2024-09-15T01:21:54.097 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T01:21:54.114 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T01:21:54.190 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043 branch: wip-bharath9-testing-2024-09-13-1444-quincy description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_latest} thrashers/force-sync-many workloads/rados_mon_workunits} email: skanta@redhat.com first_in_suite: false job_id: '7905043' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath9-testing-2024-09-13-1444-quincy ceph: conf: client: debug monc: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 paxos service trim min: 5 osd: bluestore zero block detection: true debug ms: 1 debug osd: 20 osd debug verify cached snaps: true osd debug verify missing on start: true osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: filestore osd op queue: debug_random osd op queue cut off: debug_random osd sloppy crc: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(TOO_FEW_PGS\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(AUTH_BAD_CAPS\) sha1: d75f23820806da622f561ee5ee7551015285993f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: enable_coprs: - ceph/el9 flavor: default sha1: d75f23820806da622f561ee5ee7551015285993f workunit: branch: wip-bharath9-testing-2024-09-13-1444-quincy sha1: d75f23820806da622f561ee5ee7551015285993f owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mon.d - mon.e - osd.0 - osd.1 - osd.2 - - mon.f - mon.g - mon.h - mon.i - mgr.x - osd.3 - osd.4 - osd.5 - client.0 seed: 2126 sha1: d75f23820806da622f561ee5ee7551015285993f sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath9-testing-2024-09-13-1444-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d75f23820806da622f561ee5ee7551015285993f targets: smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLIFDTnvESD8w4Wq/enD710safWO8CiXAT0Mj0IWMGijE23DYvl0/awJBcwZK7+qChwlj+rkcvjX16OFOKvG2+k= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF2VLgh93C95lAQ6Mb6iTXJLdPclMarnWJpOdg8gGGaC5ex+vieo/kHH1TXKwXJcNABCtcfsJGUbsXUMfMzqHu8= tasks: - install: null - ceph: null - mon_thrash: revive_delay: 90 store_thrash: true thrash_delay: 1 thrash_many: true - workunit: clients: client.0: - mon/pool_ops.sh - mon/crush_ops.sh - mon/osd.sh - mon/caps.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_00:54:39 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T01:21:54.190 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa; will attempt to use it 2024-09-15T01:21:54.191 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa/tasks 2024-09-15T01:21:54.191 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T01:21:54.193 INFO:teuthology.task.internal:Checking packages... 2024-09-15T01:21:54.214 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd75f23820806da622f561ee5ee7551015285993f' 2024-09-15T01:21:54.214 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T01:21:54.214 INFO:teuthology.packaging:ref: None 2024-09-15T01:21:54.214 INFO:teuthology.packaging:tag: None 2024-09-15T01:21:54.214 INFO:teuthology.packaging:branch: wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-15T01:21:54.214 INFO:teuthology.packaging:sha1: d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:21:54.214 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-15T01:21:54.370 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1643.gd75f2382 2024-09-15T01:21:54.371 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T01:21:54.378 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T01:21:54.378 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T01:21:54.401 INFO:teuthology.task.internal:Saving configuration 2024-09-15T01:21:54.417 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T01:21:54.424 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T01:21:54.451 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 01:16:35.136679', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLIFDTnvESD8w4Wq/enD710safWO8CiXAT0Mj0IWMGijE23DYvl0/awJBcwZK7+qChwlj+rkcvjX16OFOKvG2+k='} 2024-09-15T01:21:54.478 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 01:16:35.138651', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF2VLgh93C95lAQ6Mb6iTXJLdPclMarnWJpOdg8gGGaC5ex+vieo/kHH1TXKwXJcNABCtcfsJGUbsXUMfMzqHu8='} 2024-09-15T01:21:54.478 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T01:21:54.485 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'] 2024-09-15T01:21:54.485 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2024-09-15T01:21:54.485 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T01:21:54.553 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5084b5bac0>, signals=[15]) 2024-09-15T01:21:54.554 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T01:21:54.560 INFO:teuthology.task.internal:Opening connections... 2024-09-15T01:21:54.560 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-09-15T01:21:54.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:21:54.646 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-09-15T01:21:54.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:21:54.731 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T01:21:54.737 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-09-15T01:21:54.756 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-09-15T01:21:54.756 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-09-15T01:21:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-09-15T01:21:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-09-15T01:21:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-09-15T01:21:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-09-15T01:21:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-09-15T01:21:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-09-15T01:21:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T01:21:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-09-15T01:21:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-09-15T01:21:54.815 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T01:21:54.815 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-09-15T01:21:54.815 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T01:21:54.815 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T01:21:54.815 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T01:21:54.815 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-09-15T01:21:54.843 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-09-15T01:21:54.863 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-09-15T01:21:54.863 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-09-15T01:21:54.921 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-09-15T01:21:54.922 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-09-15T01:21:54.922 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-09-15T01:21:54.922 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-09-15T01:21:54.922 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-09-15T01:21:54.923 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-09-15T01:21:54.923 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T01:21:54.923 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-09-15T01:21:54.923 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-09-15T01:21:54.923 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T01:21:54.923 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-09-15T01:21:54.924 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T01:21:54.924 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T01:21:54.924 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T01:21:54.925 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-09-15T01:21:54.943 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T01:21:54.952 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T01:21:54.958 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T01:21:54.958 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-09-15T01:21:54.962 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-09-15T01:21:54.979 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T01:21:54.985 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T01:21:54.985 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-09-15T01:21:55.019 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-09-15T01:21:55.046 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T01:21:55.104 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T01:21:55.118 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T01:21:55.119 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.f': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T01:21:55.119 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T01:21:55.119 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T01:21:55.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:21:55.137 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-09-15T01:21:55.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:21:55.138 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-09-15T01:21:55.195 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-503.el9.x86_64 2024-09-15T01:21:55.195 INFO:teuthology.task.kernel:Running kernel on smithi178: 5.14.0-503.el9.x86_64 2024-09-15T01:21:55.196 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum install -y kernel 2024-09-15T01:21:55.197 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-503.el9.x86_64 2024-09-15T01:21:55.197 INFO:teuthology.task.kernel:Running kernel on smithi204: 5.14.0-503.el9.x86_64 2024-09-15T01:21:55.197 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum install -y kernel 2024-09-15T01:21:57.614 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-15T01:21:58.445 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.2 MB 00:01 2024-09-15T01:21:59.323 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-15T01:22:02.470 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 20 MB 00:02 2024-09-15T01:22:03.724 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-15T01:22:05.200 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 19 kB 00:00 2024-09-15T01:22:06.004 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-15T01:22:06.779 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-15T01:22:08.271 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 19 kB 00:00 2024-09-15T01:22:09.367 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-15T01:22:11.485 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-09-15T01:22:13.509 INFO:teuthology.orchestra.run.smithi204.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T01:22:13.509 INFO:teuthology.orchestra.run.smithi204.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T01:22:13.602 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:22:13.608 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout: Package Architecture Version Repository Size 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout:Install 4 Packages 2024-09-15T01:22:13.609 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:22:13.610 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 79 M 2024-09-15T01:22:13.610 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 126 M 2024-09-15T01:22:13.610 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:22:13.754 INFO:teuthology.orchestra.run.smithi204.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 236 kB/s | 23 kB 00:00 2024-09-15T01:22:14.856 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-15T01:22:16.370 INFO:teuthology.orchestra.run.smithi204.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.8 MB/s | 16 MB 00:02 2024-09-15T01:22:16.904 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T01:22:16.904 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T01:22:17.001 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:22:17.011 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:22:17.011 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-15T01:22:17.011 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:22:17.012 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:22:17.012 INFO:teuthology.orchestra.run.smithi178.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T01:22:17.012 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-15T01:22:17.012 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T01:22:17.012 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T01:22:17.012 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T01:22:17.012 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:22:17.012 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:22:17.012 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:22:17.012 INFO:teuthology.orchestra.run.smithi178.stdout:Install 4 Packages 2024-09-15T01:22:17.012 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:22:17.013 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 79 M 2024-09-15T01:22:17.013 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 126 M 2024-09-15T01:22:17.013 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:22:17.824 INFO:teuthology.orchestra.run.smithi178.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 43 kB/s | 23 kB 00:00 2024-09-15T01:22:18.120 INFO:teuthology.orchestra.run.smithi204.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 7.7 MB/s | 35 MB 00:04 2024-09-15T01:22:18.708 INFO:teuthology.orchestra.run.smithi178.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-15T01:22:19.499 INFO:teuthology.orchestra.run.smithi178.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-15T01:22:20.266 INFO:teuthology.orchestra.run.smithi178.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-15T01:22:20.266 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:22:20.266 INFO:teuthology.orchestra.run.smithi178.stdout:Total 24 MB/s | 79 MB 00:03 2024-09-15T01:22:20.622 INFO:teuthology.orchestra.run.smithi204.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 4.2 MB/s | 29 MB 00:06 2024-09-15T01:22:20.622 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:22:20.622 INFO:teuthology.orchestra.run.smithi204.stdout:Total 11 MB/s | 79 MB 00:07 2024-09-15T01:22:20.946 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:22:21.298 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:22:21.299 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:22:21.302 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:22:21.661 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:22:21.661 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:22:22.250 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:22:22.251 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:22:22.621 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:22:22.622 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:22:24.698 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:22:25.056 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:22:25.374 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T01:22:25.474 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:22:25.770 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T01:22:25.855 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:22:28.264 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:22:28.404 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:22:28.643 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:22:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:22:33.241 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:22:33.363 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:22:33.602 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:22:33.702 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:22:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:22:38.561 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:16.713 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:16.760 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:16.764 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:16.811 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:17.184 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:17.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T01:23:17.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:23:17.184 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:23:17.201 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:17.201 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T01:23:17.203 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T01:23:17.204 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T01:23:18.003 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:18.003 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:23:18.003 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:23:18.003 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:18.003 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T01:23:18.003 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T01:23:18.004 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T01:23:18.004 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:23:18.004 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:23:18.062 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T01:23:18.063 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:23:18.063 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:23:18.063 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:18.063 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T01:23:18.063 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T01:23:18.063 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T01:23:18.063 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:23:18.063 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:23:18.347 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T01:23:18.430 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T01:23:18.436 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:18.437 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T01:23:18.437 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T01:23:18.437 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T01:23:18.437 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T01:23:18.437 INFO:teuthology.task.kernel:Installing distro kernel on mon.f... 2024-09-15T01:23:18.437 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T01:23:18.438 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T01:23:18.496 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:18.496 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T01:23:18.496 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -qi grub2-tools 2024-09-15T01:23:18.521 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:18.522 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T01:23:18.522 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T01:23:18.522 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T01:23:18.522 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T01:23:18.522 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T01:23:18.522 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi178.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T01:23:18.522 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T01:23:18.530 INFO:teuthology.orchestra.run.smithi204.stdout:Name : grub2-tools 2024-09-15T01:23:18.530 INFO:teuthology.orchestra.run.smithi204.stdout:Epoch : 1 2024-09-15T01:23:18.530 INFO:teuthology.orchestra.run.smithi204.stdout:Version : 2.06 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:Release : 82.el9 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: x86_64 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:Group : Unspecified 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:Size : 8274670 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:License : GPLv3+ 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:Packager : builder@centos.org 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:Vendor : CentOS 2024-09-15T01:23:18.531 INFO:teuthology.orchestra.run.smithi204.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T01:23:18.532 INFO:teuthology.orchestra.run.smithi204.stdout:Summary : Support tools for GRUB. 2024-09-15T01:23:18.532 INFO:teuthology.orchestra.run.smithi204.stdout:Description : 2024-09-15T01:23:18.532 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:23:18.532 INFO:teuthology.orchestra.run.smithi204.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T01:23:18.532 INFO:teuthology.orchestra.run.smithi204.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T01:23:18.532 INFO:teuthology.orchestra.run.smithi204.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T01:23:18.532 INFO:teuthology.orchestra.run.smithi204.stdout:hardware devices. 2024-09-15T01:23:18.532 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:23:18.532 INFO:teuthology.orchestra.run.smithi204.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T01:23:18.533 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T01:23:18.533 INFO:teuthology.task.kernel:Updating grub on smithi204 to boot 5.14.0-505.el9.x86_64 2024-09-15T01:23:18.533 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T01:23:18.579 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T01:23:18.580 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T01:23:18.581 DEBUG:teuthology.orchestra.run.smithi178:> sudo rpm -qi grub2-tools 2024-09-15T01:23:18.612 INFO:teuthology.orchestra.run.smithi178.stdout:Name : grub2-tools 2024-09-15T01:23:18.612 INFO:teuthology.orchestra.run.smithi178.stdout:Epoch : 1 2024-09-15T01:23:18.612 INFO:teuthology.orchestra.run.smithi178.stdout:Version : 2.06 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:Release : 82.el9 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: x86_64 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:Group : Unspecified 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:Size : 8274670 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:License : GPLv3+ 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:Packager : builder@centos.org 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:Vendor : CentOS 2024-09-15T01:23:18.613 INFO:teuthology.orchestra.run.smithi178.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T01:23:18.614 INFO:teuthology.orchestra.run.smithi178.stdout:Summary : Support tools for GRUB. 2024-09-15T01:23:18.614 INFO:teuthology.orchestra.run.smithi178.stdout:Description : 2024-09-15T01:23:18.614 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:23:18.614 INFO:teuthology.orchestra.run.smithi178.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T01:23:18.614 INFO:teuthology.orchestra.run.smithi178.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T01:23:18.614 INFO:teuthology.orchestra.run.smithi178.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T01:23:18.614 INFO:teuthology.orchestra.run.smithi178.stdout:hardware devices. 2024-09-15T01:23:18.614 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:23:18.614 INFO:teuthology.orchestra.run.smithi178.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T01:23:18.615 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T01:23:18.615 INFO:teuthology.task.kernel:Updating grub on smithi178 to boot 5.14.0-505.el9.x86_64 2024-09-15T01:23:18.615 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T01:23:19.138 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-09-15T01:23:19.252 INFO:teuthology.orchestra.run.smithi178.stderr:Generating grub configuration file ... 2024-09-15T01:23:20.456 INFO:teuthology.orchestra.run.smithi204.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T01:23:20.487 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-09-15T01:23:20.489 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-09-15T01:23:20.505 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.tUjP799t5Z 2024-09-15T01:23:20.505 DEBUG:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.tUjP799t5Z 2024-09-15T01:23:20.566 INFO:teuthology.orchestra.run.smithi178.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T01:23:20.572 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.tUjP799t5Z 2024-09-15T01:23:20.598 INFO:teuthology.orchestra.run.smithi178.stderr:done 2024-09-15T01:23:20.600 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-09-15T01:23:20.615 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.Ie333EB3K8 2024-09-15T01:23:20.616 DEBUG:teuthology.orchestra.run.smithi178:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Ie333EB3K8 2024-09-15T01:23:20.689 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.Ie333EB3K8 2024-09-15T01:23:20.714 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.tUjP799t5Z is 6KB 2024-09-15T01:23:20.765 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.tUjP799t5Z 2024-09-15T01:23:20.780 DEBUG:teuthology.orchestra.run.smithi204:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T01:23:20.836 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.Ie333EB3K8 is 6KB 2024-09-15T01:23:20.852 INFO:teuthology.orchestra.run.smithi204.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T01:23:20.852 INFO:teuthology.orchestra.run.smithi204.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T01:23:20.852 INFO:teuthology.orchestra.run.smithi204.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T01:23:20.852 INFO:teuthology.orchestra.run.smithi204.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T01:23:20.853 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T01:23:20.885 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.Ie333EB3K8 2024-09-15T01:23:20.900 DEBUG:teuthology.orchestra.run.smithi178:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T01:23:20.962 INFO:teuthology.orchestra.run.smithi178.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T01:23:20.962 INFO:teuthology.orchestra.run.smithi178.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T01:23:20.963 INFO:teuthology.orchestra.run.smithi178.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T01:23:20.963 INFO:teuthology.orchestra.run.smithi178.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T01:23:20.964 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T01:23:21.005 DEBUG:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2024-09-15T01:23:21.107 DEBUG:teuthology.orchestra.run.smithi178:> sudo shutdown -r now 2024-09-15T01:23:51.041 INFO:teuthology.task.kernel:Checking client mon.f for new kernel version... 2024-09-15T01:23:51.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-15T01:23:51.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:23:51.110 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T01:23:51.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-15T01:23:51.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:24:25.533 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-09-15T01:24:25.789 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-09-15T01:24:34.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-15T01:24:34.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:24:34.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-15T01:24:34.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:24:37.853 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-09-15T01:24:49.865 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-15T01:24:49.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:24:52.925 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-09-15T01:24:52.929 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-09-15T01:25:04.937 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-15T01:25:04.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:25:05.263 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-09-15T01:25:05.641 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-15T01:25:05.641 INFO:teuthology.task.kernel:Checking kernel version of mon.f, want "5.14.0-505.el9.x86_64"... 2024-09-15T01:25:05.642 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-09-15T01:25:05.656 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-505.el9.x86_64 2024-09-15T01:25:05.656 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T01:25:05.657 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T01:25:05.657 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T01:25:06.657 INFO:teuthology.task.kernel:Enabling kdb on mon.f... 2024-09-15T01:25:06.657 DEBUG:teuthology.orchestra.run.smithi204:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T01:25:06.751 INFO:teuthology.orchestra.run.smithi204.stdout:ttyS1 2024-09-15T01:25:06.780 DEBUG:teuthology.parallel:result is None 2024-09-15T01:25:07.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-15T01:25:07.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:25:11.005 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-09-15T01:25:29.006 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-15T01:25:29.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:25:29.414 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-09-15T01:25:29.799 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-15T01:25:29.799 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-15T01:25:29.799 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-09-15T01:25:29.815 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-505.el9.x86_64 2024-09-15T01:25:29.815 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T01:25:29.815 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T01:25:29.815 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T01:25:30.816 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T01:25:30.817 DEBUG:teuthology.orchestra.run.smithi178:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T01:25:30.894 INFO:teuthology.orchestra.run.smithi178.stdout:ttyS1 2024-09-15T01:25:30.914 DEBUG:teuthology.parallel:result is None 2024-09-15T01:25:30.915 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T01:25:30.921 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T01:25:30.921 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T01:25:30.924 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T01:25:30.945 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T01:25:30.953 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T01:25:30.963 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T01:25:30.963 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T01:25:30.982 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T01:25:31.057 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T01:25:31.064 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T01:25:31.065 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T01:25:31.068 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T01:25:31.100 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:25:31.102 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:25:31.112 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:25:31.116 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T01:25:31.117 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T01:25:31.126 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T01:25:31.126 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T01:25:31.159 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T01:25:31.187 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T01:25:31.201 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T01:25:31.201 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T01:25:31.228 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T01:25:31.244 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:25:31.329 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:25:31.370 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:25:31.453 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:25:31.495 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:25:31.495 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T01:25:31.560 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:25:31.605 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T01:25:31.656 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:25:31.736 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T01:25:31.781 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:25:31.781 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T01:25:31.847 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-09-15T01:25:31.849 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-09-15T01:25:31.898 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T01:25:31.944 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T01:25:32.402 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T01:25:32.413 INFO:teuthology.task.internal:Starting timer... 2024-09-15T01:25:32.413 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T01:25:32.429 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T01:25:32.448 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-09-15T01:25:32.504 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-09-15T01:25:32.506 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-09-15T01:25:32.560 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-09-15T01:25:32.562 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T01:25:32.562 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-09-15T01:25:32.592 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-09-15T01:25:32.592 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-09-15T01:25:32.628 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-09-15T01:25:32.629 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi178.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-09-15T01:25:32.629 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T01:25:32.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:25:32.662 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T01:25:32.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:25:32.695 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T01:25:32.695 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-09-15T01:25:32.731 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-09-15T01:25:32.763 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T01:25:32.776 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-15T01:25:32.779 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-15T01:25:32.894 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T01:25:32.905 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T01:25:32.906 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi178.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T01:29:17.021 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi178.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-09-15T01:29:17.022 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-15T01:29:17.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:29:17.099 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-09-15T01:29:17.176 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-15T01:29:17.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-15T01:29:17.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T01:29:17.250 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-09-15T01:29:17.286 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-15T01:29:17.287 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T01:29:17.297 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T01:29:17.297 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T01:29:17.298 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:29:17.300 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T01:29:17.301 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:29:17.326 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T01:29:17.341 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T01:29:17.356 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T01:29:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-09-15T01:29:17.370 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T01:29:17.376 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-09-15T01:29:17.390 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T01:29:17.395 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-09-15T01:29:17.404 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T01:29:17.406 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-09-15T01:29:17.421 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T01:29:17.435 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T01:29:17.458 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-09-15T01:29:17.461 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T01:29:17.461 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-09-15T01:29:17.461 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:17.461 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:17.461 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:17.461 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:17.493 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-09-15T01:29:17.496 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T01:29:17.496 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-09-15T01:29:17.496 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:17.496 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:17.496 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:17.496 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:29:17.497 INFO:teuthology.run_tasks:Running task install... 2024-09-15T01:29:17.506 DEBUG:teuthology.task.install:project ceph 2024-09-15T01:29:17.506 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-15T01:29:17.506 DEBUG:teuthology.task.install:config {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'} 2024-09-15T01:29:17.506 INFO:teuthology.task.install:Using flavor: default 2024-09-15T01:29:17.511 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T01:29:17.511 INFO:teuthology.task.install:extra packages: [] 2024-09-15T01:29:17.512 DEBUG:teuthology.orchestra.run.smithi178:> sudo dnf -y install 'dnf-command(copr)' 2024-09-15T01:29:17.512 DEBUG:teuthology.orchestra.run.smithi204:> sudo dnf -y install 'dnf-command(copr)' 2024-09-15T01:29:18.068 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:45 ago on Sun 15 Sep 2024 01:28:32 AM UTC. 2024-09-15T01:29:18.095 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:00:45 ago on Sun 15 Sep 2024 01:28:32 AM UTC. 2024-09-15T01:29:18.153 INFO:teuthology.orchestra.run.smithi178.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-15T01:29:18.181 INFO:teuthology.orchestra.run.smithi204.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-15T01:29:18.195 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:29:18.196 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-15T01:29:18.196 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:29:18.223 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:29:18.224 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-09-15T01:29:18.225 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:29:18.258 DEBUG:teuthology.orchestra.run.smithi178:> sudo dnf -y copr enable ceph/el9 2024-09-15T01:29:18.284 DEBUG:teuthology.orchestra.run.smithi204:> sudo dnf -y copr enable ceph/el9 2024-09-15T01:29:18.428 INFO:teuthology.orchestra.run.smithi178.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-15T01:29:18.429 INFO:teuthology.orchestra.run.smithi178.stderr:of the main distribution, and quality may vary. 2024-09-15T01:29:18.429 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:29:18.429 INFO:teuthology.orchestra.run.smithi178.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-15T01:29:18.429 INFO:teuthology.orchestra.run.smithi178.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-15T01:29:18.429 INFO:teuthology.orchestra.run.smithi178.stderr:, 2024-09-15T01:29:18.429 INFO:teuthology.orchestra.run.smithi178.stderr:and packages are not held to any quality or security level. 2024-09-15T01:29:18.429 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:29:18.429 INFO:teuthology.orchestra.run.smithi178.stderr:Please do not file bug reports about these packages in Fedora 2024-09-15T01:29:18.429 INFO:teuthology.orchestra.run.smithi178.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-15T01:29:18.456 INFO:teuthology.orchestra.run.smithi204.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-15T01:29:18.456 INFO:teuthology.orchestra.run.smithi204.stderr:of the main distribution, and quality may vary. 2024-09-15T01:29:18.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:29:18.456 INFO:teuthology.orchestra.run.smithi204.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-15T01:29:18.456 INFO:teuthology.orchestra.run.smithi204.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-15T01:29:18.456 INFO:teuthology.orchestra.run.smithi204.stderr:, 2024-09-15T01:29:18.456 INFO:teuthology.orchestra.run.smithi204.stderr:and packages are not held to any quality or security level. 2024-09-15T01:29:18.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:29:18.457 INFO:teuthology.orchestra.run.smithi204.stderr:Please do not file bug reports about these packages in Fedora 2024-09-15T01:29:18.457 INFO:teuthology.orchestra.run.smithi204.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-15T01:29:18.494 INFO:teuthology.orchestra.run.smithi178.stdout:Repository successfully enabled. 2024-09-15T01:29:18.510 INFO:teuthology.orchestra.run.smithi204.stdout:Repository successfully enabled. 2024-09-15T01:29:18.531 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'tag': None, 'wait_for_package': False} 2024-09-15T01:29:18.531 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:29:18.546 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'tag': None, 'wait_for_package': False} 2024-09-15T01:29:18.546 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:29:18.684 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/ 2024-09-15T01:29:18.684 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1643.gd75f2382 2024-09-15T01:29:18.690 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/ 2024-09-15T01:29:18.690 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1643.gd75f2382 2024-09-15T01:29:18.794 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T01:29:18.794 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:29:18.794 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T01:29:18.801 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T01:29:18.801 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:29:18.801 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T01:29:18.820 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T01:29:18.820 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d75f23820806da622f561ee5ee7551015285993f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T01:29:18.826 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T01:29:18.826 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d75f23820806da622f561ee5ee7551015285993f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T01:29:18.885 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T01:29:18.891 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T01:29:18.960 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T01:29:18.968 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T01:29:19.025 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-09-15T01:29:19.027 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-09-15T01:29:19.032 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-09-15T01:29:19.033 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-09-15T01:29:19.260 INFO:teuthology.orchestra.run.smithi178.stdout:45 files removed 2024-09-15T01:29:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:45 files removed 2024-09-15T01:29:19.287 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw 2024-09-15T01:29:19.293 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw 2024-09-15T01:29:19.832 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 304 kB/s | 77 kB 00:00 2024-09-15T01:29:19.846 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 280 kB/s | 77 kB 00:00 2024-09-15T01:29:20.108 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-15T01:29:20.130 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-15T01:29:20.392 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 7.2 kB/s | 1.8 kB 00:00 2024-09-15T01:29:20.414 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 7.2 kB/s | 1.8 kB 00:00 2024-09-15T01:29:20.576 INFO:teuthology.orchestra.run.smithi204.stdout:Copr repo for el9 owned by ceph 51 kB/s | 7.6 kB 00:00 2024-09-15T01:29:20.581 INFO:teuthology.orchestra.run.smithi178.stdout:Copr repo for el9 owned by ceph 58 kB/s | 7.6 kB 00:00 2024-09-15T01:29:21.068 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.2 MB 00:00 2024-09-15T01:29:21.265 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-15T01:29:22.744 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-15T01:29:24.391 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-15T01:29:27.237 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.5 MB 00:00 2024-09-15T01:29:28.671 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 19 kB 00:00 2024-09-15T01:29:28.950 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.5 MB 00:00 2024-09-15T01:29:29.467 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-15T01:29:30.867 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 19 kB 00:00 2024-09-15T01:29:31.796 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-15T01:29:34.908 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-15T01:29:37.260 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-09-15T01:29:37.373 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:29:37.390 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:29:37.391 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-15T01:29:37.391 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:29:37.391 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:29:37.391 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 14 M 2024-09-15T01:29:37.391 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-09-15T01:29:37.391 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.1 M 2024-09-15T01:29:37.391 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-15T01:29:37.391 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-15T01:29:37.391 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 5.0 M 2024-09-15T01:29:37.391 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 22 M 2024-09-15T01:29:37.391 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 637 k 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 458 k 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 4.9 M 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 45 k 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 120 k 2024-09-15T01:29:37.392 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 156 k 2024-09-15T01:29:37.393 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T01:29:37.393 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 320 k 2024-09-15T01:29:37.393 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 295 k 2024-09-15T01:29:37.393 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 100 k 2024-09-15T01:29:37.393 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T01:29:37.393 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:29:37.393 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:29:37.393 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:29:37.393 INFO:teuthology.orchestra.run.smithi204.stdout:Install 22 Packages 2024-09-15T01:29:37.393 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 2 Packages 2024-09-15T01:29:37.393 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:29:37.394 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 58 M 2024-09-15T01:29:37.394 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:29:38.121 INFO:teuthology.orchestra.run.smithi204.stdout:(1/24): ceph-base-17.2.7-1643.gd75f2382.el9.x86 7.7 MB/s | 5.0 MB 00:00 2024-09-15T01:29:38.183 INFO:teuthology.orchestra.run.smithi204.stdout:(2/24): ceph-selinux-17.2.7-1643.gd75f2382.el9. 407 kB/s | 25 kB 00:00 2024-09-15T01:29:38.284 INFO:teuthology.orchestra.run.smithi204.stdout:(3/24): libcephfs2-17.2.7-1643.gd75f2382.el9.x8 6.2 MB/s | 637 kB 00:00 2024-09-15T01:29:38.484 INFO:teuthology.orchestra.run.smithi204.stdout:(4/24): ceph-radosgw-17.2.7-1643.gd75f2382.el9. 14 MB/s | 14 MB 00:01 2024-09-15T01:29:38.509 INFO:teuthology.orchestra.run.smithi204.stdout:(5/24): libradosstriper1-17.2.7-1643.gd75f2382. 2.0 MB/s | 458 kB 00:00 2024-09-15T01:29:38.734 INFO:teuthology.orchestra.run.smithi204.stdout:(6/24): ceph-common-17.2.7-1643.gd75f2382.el9.x 18 MB/s | 22 MB 00:01 2024-09-15T01:29:38.759 INFO:teuthology.orchestra.run.smithi204.stdout:(7/24): python3-ceph-argparse-17.2.7-1643.gd75f 180 kB/s | 45 kB 00:00 2024-09-15T01:29:38.784 INFO:teuthology.orchestra.run.smithi204.stdout:(8/24): python3-ceph-common-17.2.7-1643.gd75f23 2.3 MB/s | 120 kB 00:00 2024-09-15T01:29:38.810 INFO:teuthology.orchestra.run.smithi204.stdout:(9/24): python3-cephfs-17.2.7-1643.gd75f2382.el 3.0 MB/s | 156 kB 00:00 2024-09-15T01:29:38.835 INFO:teuthology.orchestra.run.smithi204.stdout:(10/24): python3-rados-17.2.7-1643.gd75f2382.el 6.2 MB/s | 320 kB 00:00 2024-09-15T01:29:38.869 INFO:teuthology.orchestra.run.smithi204.stdout:(11/24): python3-rbd-17.2.7-1643.gd75f2382.el9. 4.9 MB/s | 295 kB 00:00 2024-09-15T01:29:38.894 INFO:teuthology.orchestra.run.smithi204.stdout:(12/24): python3-rgw-17.2.7-1643.gd75f2382.el9. 1.7 MB/s | 100 kB 00:00 2024-09-15T01:29:38.970 INFO:teuthology.orchestra.run.smithi204.stdout:(13/24): librgw2-17.2.7-1643.gd75f2382.el9.x86_ 10 MB/s | 4.9 MB 00:00 2024-09-15T01:29:39.070 INFO:teuthology.orchestra.run.smithi204.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 258 kB/s | 45 kB 00:00 2024-09-15T01:29:39.095 INFO:teuthology.orchestra.run.smithi204.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 709 kB/s | 160 kB 00:00 2024-09-15T01:29:39.137 INFO:teuthology.orchestra.run.smithi204.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-15T01:29:39.187 INFO:teuthology.orchestra.run.smithi204.stdout:(17/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 6.0 MB/s | 308 kB 00:00 2024-09-15T01:29:39.212 INFO:teuthology.orchestra.run.smithi204.stdout:(18/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-15T01:29:39.238 INFO:teuthology.orchestra.run.smithi204.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 964 kB/s | 49 kB 00:00 2024-09-15T01:29:39.272 INFO:teuthology.orchestra.run.smithi204.stdout:(20/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-09-15T01:29:39.305 INFO:teuthology.orchestra.run.smithi204.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 727 kB/s | 67 kB 00:00 2024-09-15T01:29:39.356 INFO:teuthology.orchestra.run.smithi204.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-15T01:29:39.506 INFO:teuthology.orchestra.run.smithi204.stdout:(23/24): librbd1-17.2.7-1643.gd75f2382.el9.x86_ 15 MB/s | 2.9 MB 00:00 2024-09-15T01:29:39.564 INFO:teuthology.orchestra.run.smithi204.stdout:(24/24): librados2-17.2.7-1643.gd75f2382.el9.x8 11 MB/s | 3.1 MB 00:00 2024-09-15T01:29:39.568 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:29:39.568 INFO:teuthology.orchestra.run.smithi204.stdout:Total 27 MB/s | 58 MB 00:02 2024-09-15T01:29:39.630 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:29:39.661 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:29:39.661 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:29:39.727 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:29:39.738 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:29:39.738 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-15T01:29:39.738 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:29:39.738 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:29:39.738 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 14 M 2024-09-15T01:29:39.738 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-09-15T01:29:39.739 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.1 M 2024-09-15T01:29:39.739 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-15T01:29:39.739 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-15T01:29:39.739 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 5.0 M 2024-09-15T01:29:39.739 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 22 M 2024-09-15T01:29:39.739 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-15T01:29:39.739 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T01:29:39.739 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 637 k 2024-09-15T01:29:39.739 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T01:29:39.739 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T01:29:39.739 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T01:29:39.739 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 458 k 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 4.9 M 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 45 k 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 120 k 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 156 k 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 320 k 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 295 k 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 100 k 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T01:29:39.740 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:29:39.741 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:29:39.741 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:29:39.741 INFO:teuthology.orchestra.run.smithi178.stdout:Install 22 Packages 2024-09-15T01:29:39.741 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 2 Packages 2024-09-15T01:29:39.741 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:29:39.741 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 58 M 2024-09-15T01:29:39.741 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:29:40.335 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:29:40.335 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:29:41.353 INFO:teuthology.orchestra.run.smithi178.stdout:(1/24): ceph-radosgw-17.2.7-1643.gd75f2382.el9. 15 MB/s | 14 MB 00:00 2024-09-15T01:29:41.428 INFO:teuthology.orchestra.run.smithi178.stdout:(2/24): ceph-base-17.2.7-1643.gd75f2382.el9.x86 4.8 MB/s | 5.0 MB 00:01 2024-09-15T01:29:41.445 INFO:teuthology.orchestra.run.smithi178.stdout:(3/24): ceph-selinux-17.2.7-1643.gd75f2382.el9. 274 kB/s | 25 kB 00:00 2024-09-15T01:29:41.504 INFO:teuthology.orchestra.run.smithi178.stdout:(4/24): libradosstriper1-17.2.7-1643.gd75f2382. 7.7 MB/s | 458 kB 00:00 2024-09-15T01:29:41.537 INFO:teuthology.orchestra.run.smithi178.stdout:(5/24): libcephfs2-17.2.7-1643.gd75f2382.el9.x8 5.7 MB/s | 637 kB 00:00 2024-09-15T01:29:41.576 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:29:41.754 INFO:teuthology.orchestra.run.smithi178.stdout:(6/24): ceph-common-17.2.7-1643.gd75f2382.el9.x 17 MB/s | 22 MB 00:01 2024-09-15T01:29:41.779 INFO:teuthology.orchestra.run.smithi178.stdout:(7/24): python3-ceph-argparse-17.2.7-1643.gd75f 186 kB/s | 45 kB 00:00 2024-09-15T01:29:41.804 INFO:teuthology.orchestra.run.smithi178.stdout:(8/24): python3-ceph-common-17.2.7-1643.gd75f23 2.4 MB/s | 120 kB 00:00 2024-09-15T01:29:41.830 INFO:teuthology.orchestra.run.smithi178.stdout:(9/24): python3-cephfs-17.2.7-1643.gd75f2382.el 3.0 MB/s | 156 kB 00:00 2024-09-15T01:29:41.855 INFO:teuthology.orchestra.run.smithi178.stdout:(10/24): python3-rados-17.2.7-1643.gd75f2382.el 6.2 MB/s | 320 kB 00:00 2024-09-15T01:29:41.880 INFO:teuthology.orchestra.run.smithi178.stdout:(11/24): python3-rbd-17.2.7-1643.gd75f2382.el9. 5.7 MB/s | 295 kB 00:00 2024-09-15T01:29:41.897 INFO:teuthology.orchestra.run.smithi178.stdout:(12/24): python3-rgw-17.2.7-1643.gd75f2382.el9. 2.3 MB/s | 100 kB 00:00 2024-09-15T01:29:41.956 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-15T01:29:41.981 INFO:teuthology.orchestra.run.smithi178.stdout:(13/24): librgw2-17.2.7-1643.gd75f2382.el9.x86_ 10 MB/s | 4.9 MB 00:00 2024-09-15T01:29:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:42.126 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:42.212 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-15T01:29:42.252 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:42.273 INFO:teuthology.orchestra.run.smithi178.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 120 kB/s | 45 kB 00:00 2024-09-15T01:29:42.341 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:42.394 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-15T01:29:42.431 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-15T01:29:42.439 INFO:teuthology.orchestra.run.smithi178.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 287 kB/s | 160 kB 00:00 2024-09-15T01:29:42.557 INFO:teuthology.orchestra.run.smithi178.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 362 kB/s | 42 kB 00:00 2024-09-15T01:29:42.582 INFO:teuthology.orchestra.run.smithi178.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 947 kB/s | 292 kB 00:00 2024-09-15T01:29:42.640 INFO:teuthology.orchestra.run.smithi178.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 1.0 MB/s | 662 kB 00:00 2024-09-15T01:29:42.674 INFO:teuthology.orchestra.run.smithi178.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-09-15T01:29:42.708 INFO:teuthology.orchestra.run.smithi178.stdout:(20/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.0 MB/s | 308 kB 00:00 2024-09-15T01:29:42.733 INFO:teuthology.orchestra.run.smithi178.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 746 kB/s | 67 kB 00:00 2024-09-15T01:29:42.783 INFO:teuthology.orchestra.run.smithi178.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-15T01:29:42.871 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-15T01:29:42.897 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-15T01:29:42.925 INFO:teuthology.orchestra.run.smithi178.stdout:(23/24): librados2-17.2.7-1643.gd75f2382.el9.x8 14 MB/s | 3.1 MB 00:00 2024-09-15T01:29:42.959 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-15T01:29:42.988 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-15T01:29:42.992 INFO:teuthology.orchestra.run.smithi178.stdout:(24/24): librbd1-17.2.7-1643.gd75f2382.el9.x86_ 11 MB/s | 2.9 MB 00:00 2024-09-15T01:29:42.998 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:29:42.998 INFO:teuthology.orchestra.run.smithi178.stdout:Total 18 MB/s | 58 MB 00:03 2024-09-15T01:29:43.031 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 10/26 2024-09-15T01:29:43.064 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:29:43.068 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-15T01:29:43.098 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:29:43.098 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:29:43.138 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-15T01:29:43.174 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-15T01:29:43.240 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-15T01:29:43.295 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-15T01:29:43.333 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-15T01:29:43.366 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-15T01:29:43.660 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-15T01:29:43.703 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-15T01:29:43.761 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:29:43.761 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:29:43.794 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-15T01:29:43.868 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 19/26 2024-09-15T01:29:43.898 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 20/26 2024-09-15T01:29:45.015 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:29:45.397 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-15T01:29:45.436 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:45.528 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:45.606 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-15T01:29:45.629 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:45.709 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:45.754 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-15T01:29:45.783 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-15T01:29:46.223 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-15T01:29:46.265 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-15T01:29:46.311 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-15T01:29:46.327 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:46.348 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-15T01:29:46.361 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:46.391 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 10/26 2024-09-15T01:29:46.428 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-15T01:29:46.490 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-15T01:29:46.517 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-15T01:29:46.583 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-15T01:29:46.623 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-15T01:29:46.659 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-15T01:29:46.691 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-15T01:29:46.900 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:46.947 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-15T01:29:46.993 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-15T01:29:47.029 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-15T01:29:47.041 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-15T01:29:47.042 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T01:29:47.042 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T01:29:47.042 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:29:47.083 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:29:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-15T01:29:47.177 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 19/26 2024-09-15T01:29:47.207 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 20/26 2024-09-15T01:29:49.636 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:49.670 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:50.189 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-15T01:29:50.232 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-15T01:29:50.324 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-15T01:29:50.324 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T01:29:50.324 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T01:29:50.324 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:29:50.357 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:29:56.999 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:29:56.999 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-09-15T01:29:57.000 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-09-15T01:29:57.000 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-09-15T01:29:57.000 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-09-15T01:29:57.000 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-09-15T01:29:57.000 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-09-15T01:29:57.000 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-09-15T01:29:57.000 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:29:58.362 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-15T01:29:58.423 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-15T01:29:58.424 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:29:58.424 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T01:29:58.424 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:29:58.424 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:29:58.424 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:29:58.425 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T01:29:58.515 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T01:29:58.515 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:29:59.830 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:29:59.830 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 1/26 2024-09-15T01:29:59.830 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:29:59.831 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-15T01:29:59.831 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:29:59.831 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 5/26 2024-09-15T01:29:59.831 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 6/26 2024-09-15T01:29:59.831 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 7/26 2024-09-15T01:29:59.831 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 8/26 2024-09-15T01:29:59.831 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 9/26 2024-09-15T01:29:59.831 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 10/26 2024-09-15T01:29:59.831 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-15T01:29:59.831 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-15T01:29:59.831 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 13/26 2024-09-15T01:29:59.831 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-15T01:29:59.832 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-15T01:29:59.832 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-15T01:29:59.832 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-15T01:29:59.832 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-15T01:29:59.832 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-15T01:29:59.832 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-15T01:29:59.832 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-15T01:29:59.832 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-15T01:29:59.832 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:29:59.832 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-15T01:29:59.832 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 25/26 2024-09-15T01:30:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:30:00.045 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-09-15T01:30:00.045 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-09-15T01:30:00.045 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-09-15T01:30:00.045 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-09-15T01:30:00.045 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-09-15T01:30:00.048 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-09-15T01:30:00.048 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-09-15T01:30:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:00.670 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:00.670 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T01:30:00.671 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T01:30:00.672 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:00.673 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:30:01.042 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-test 2024-09-15T01:30:01.347 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-15T01:30:01.431 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-15T01:30:01.431 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:01.431 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T01:30:01.431 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:30:01.432 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T01:30:01.432 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:01.433 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T01:30:01.500 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T01:30:01.501 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:01.610 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:00:27 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:30:01.746 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:30:01.747 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:01.747 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-15T01:30:01.747 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:01.747 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:30:01.747 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 48 M 2024-09-15T01:30:01.747 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-15T01:30:01.747 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 163 k 2024-09-15T01:30:01.747 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T01:30:01.747 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T01:30:01.747 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:01.747 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:30:01.748 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:01.748 INFO:teuthology.orchestra.run.smithi204.stdout:Install 4 Packages 2024-09-15T01:30:01.748 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:01.748 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 48 M 2024-09-15T01:30:01.748 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 193 M 2024-09-15T01:30:01.748 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:30:02.179 INFO:teuthology.orchestra.run.smithi204.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.1 MB/s | 304 kB 00:00 2024-09-15T01:30:02.221 INFO:teuthology.orchestra.run.smithi204.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-15T01:30:02.271 INFO:teuthology.orchestra.run.smithi204.stdout:(3/4): libcephsqlite-17.2.7-1643.gd75f2382.el9. 691 kB/s | 163 kB 00:00 2024-09-15T01:30:02.788 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:02.788 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 1/26 2024-09-15T01:30:02.788 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-15T01:30:02.788 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-15T01:30:02.788 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-15T01:30:02.788 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 5/26 2024-09-15T01:30:02.788 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 6/26 2024-09-15T01:30:02.788 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 7/26 2024-09-15T01:30:02.788 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 8/26 2024-09-15T01:30:02.788 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 9/26 2024-09-15T01:30:02.788 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 10/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 13/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-15T01:30:02.789 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-15T01:30:02.790 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 25/26 2024-09-15T01:30:03.525 INFO:teuthology.orchestra.run.smithi204.stdout:(4/4): ceph-test-17.2.7-1643.gd75f2382.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-15T01:30:03.525 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:03.525 INFO:teuthology.orchestra.run.smithi204.stdout:Total 27 MB/s | 48 MB 00:01 2024-09-15T01:30:03.552 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:30:03.571 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:30:03.572 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:30:03.857 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T01:30:03.857 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:03.857 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-09-15T01:30:03.857 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.857 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.857 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:30:03.857 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.857 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.857 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.857 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.857 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.858 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T01:30:03.859 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.859 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.859 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:03.859 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T01:30:03.859 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:03.859 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:30:03.936 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:30:03.937 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:30:04.221 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-test 2024-09-15T01:30:04.454 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:30:04.574 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T01:30:04.623 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T01:30:04.648 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-15T01:30:04.785 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:27 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:30:04.917 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:30:04.917 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 48 M 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 163 k 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout:Install 4 Packages 2024-09-15T01:30:04.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:04.919 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 48 M 2024-09-15T01:30:04.919 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 193 M 2024-09-15T01:30:04.919 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:30:05.174 INFO:teuthology.orchestra.run.smithi178.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-15T01:30:05.216 INFO:teuthology.orchestra.run.smithi178.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-15T01:30:05.275 INFO:teuthology.orchestra.run.smithi178.stdout:(3/4): libcephsqlite-17.2.7-1643.gd75f2382.el9. 741 kB/s | 163 kB 00:00 2024-09-15T01:30:06.541 INFO:teuthology.orchestra.run.smithi178.stdout:(4/4): ceph-test-17.2.7-1643.gd75f2382.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-15T01:30:06.543 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:06.544 INFO:teuthology.orchestra.run.smithi178.stdout:Total 30 MB/s | 48 MB 00:01 2024-09-15T01:30:06.572 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:30:06.593 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:30:06.593 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:30:06.956 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:30:06.956 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:30:07.479 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:30:07.560 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T01:30:07.599 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T01:30:07.626 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-15T01:30:09.107 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-15T01:30:09.179 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-15T01:30:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-15T01:30:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 1/4 2024-09-15T01:30:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2/4 2024-09-15T01:30:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T01:30:11.685 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T01:30:11.685 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:11.685 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:30:11.685 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.685 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:11.685 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T01:30:11.685 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T01:30:11.685 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:11.685 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:30:11.976 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph 2024-09-15T01:30:12.068 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-15T01:30:12.141 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-15T01:30:12.543 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:00:38 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:30:12.681 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:30:12.684 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:12.684 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-15T01:30:12.684 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:12.684 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:30:12.684 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 6.5 k 2024-09-15T01:30:12.684 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-15T01:30:12.684 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.1 M 2024-09-15T01:30:12.684 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 1.4 M 2024-09-15T01:30:12.684 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 240 k 2024-09-15T01:30:12.684 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.9 M 2024-09-15T01:30:12.684 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 15 M 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T01:30:12.685 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T01:30:12.686 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T01:30:12.687 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T01:30:12.687 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T01:30:12.687 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T01:30:12.687 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:12.687 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:30:12.687 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:12.687 INFO:teuthology.orchestra.run.smithi204.stdout:Install 34 Packages 2024-09-15T01:30:12.687 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:12.688 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 27 M 2024-09-15T01:30:12.688 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 96 M 2024-09-15T01:30:12.688 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:30:13.056 INFO:teuthology.orchestra.run.smithi204.stdout:(1/34): ceph-17.2.7-1643.gd75f2382.el9.x86_64.r 34 kB/s | 6.5 kB 00:00 2024-09-15T01:30:13.981 INFO:teuthology.orchestra.run.smithi204.stdout:(2/34): ceph-mgr-17.2.7-1643.gd75f2382.el9.x86_ 1.3 MB/s | 1.4 MB 00:01 2024-09-15T01:30:14.048 INFO:teuthology.orchestra.run.smithi204.stdout:(3/34): ceph-mds-17.2.7-1643.gd75f2382.el9.x86_ 1.8 MB/s | 2.1 MB 00:01 2024-09-15T01:30:14.199 INFO:teuthology.orchestra.run.smithi204.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1643.gd75f 1.6 MB/s | 240 kB 00:00 2024-09-15T01:30:14.340 INFO:teuthology.orchestra.run.smithi204.stdout:(5/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-09-15T01:30:14.410 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-15T01:30:14.410 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 1/4 2024-09-15T01:30:14.410 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2/4 2024-09-15T01:30:14.410 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T01:30:14.457 INFO:teuthology.orchestra.run.smithi204.stdout:(6/34): python3-cryptography-36.0.1-4.el9.x86_6 11 MB/s | 1.2 MB 00:00 2024-09-15T01:30:14.549 INFO:teuthology.orchestra.run.smithi204.stdout:(7/34): ceph-mon-17.2.7-1643.gd75f2382.el9.x86_ 2.6 MB/s | 3.9 MB 00:01 2024-09-15T01:30:14.575 INFO:teuthology.orchestra.run.smithi204.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 910 kB/s | 106 kB 00:00 2024-09-15T01:30:14.658 INFO:teuthology.orchestra.run.smithi204.stdout:(9/34): python3-requests-2.25.1-8.el9.noarch.rp 1.5 MB/s | 125 kB 00:00 2024-09-15T01:30:14.700 INFO:teuthology.orchestra.run.smithi204.stdout:(10/34): python3-pycparser-2.20-6.el9.noarch.rp 897 kB/s | 135 kB 00:00 2024-09-15T01:30:14.725 INFO:teuthology.orchestra.run.smithi204.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-09-15T01:30:14.727 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T01:30:14.727 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:14.727 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:30:14.727 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:14.727 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:14.727 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T01:30:14.727 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T01:30:14.727 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:14.727 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:30:15.007 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph 2024-09-15T01:30:15.276 INFO:teuthology.orchestra.run.smithi204.stdout:(12/34): python3-markupsafe-1.1.1-12.el9.x86_64 63 kB/s | 35 kB 00:00 2024-09-15T01:30:15.476 INFO:teuthology.orchestra.run.smithi204.stdout:(13/34): python3-mako-1.1.4-6.el9.noarch.rpm 222 kB/s | 172 kB 00:00 2024-09-15T01:30:15.573 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:38 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:30:15.713 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:30:15.715 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:15.715 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-15T01:30:15.715 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:15.715 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:30:15.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 6.5 k 2024-09-15T01:30:15.715 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-15T01:30:15.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.1 M 2024-09-15T01:30:15.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 1.4 M 2024-09-15T01:30:15.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 240 k 2024-09-15T01:30:15.715 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.9 M 2024-09-15T01:30:15.716 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 15 M 2024-09-15T01:30:15.716 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T01:30:15.716 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T01:30:15.716 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T01:30:15.716 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T01:30:15.716 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T01:30:15.716 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T01:30:15.716 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T01:30:15.716 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections 2024-09-15T01:30:15.716 INFO:teuthology.orchestra.run.smithi178.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-15T01:30:15.716 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T01:30:15.716 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T01:30:15.717 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T01:30:15.718 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T01:30:15.718 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T01:30:15.718 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T01:30:15.718 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T01:30:15.718 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:15.718 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:30:15.718 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:15.718 INFO:teuthology.orchestra.run.smithi178.stdout:Install 34 Packages 2024-09-15T01:30:15.718 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:15.719 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 27 M 2024-09-15T01:30:15.719 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 96 M 2024-09-15T01:30:15.719 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:30:15.951 INFO:teuthology.orchestra.run.smithi204.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 114 kB/s | 77 kB 00:00 2024-09-15T01:30:16.077 INFO:teuthology.orchestra.run.smithi204.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 85 kB/s | 51 kB 00:00 2024-09-15T01:30:16.118 INFO:teuthology.orchestra.run.smithi204.stdout:(16/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2024-09-15T01:30:16.144 INFO:teuthology.orchestra.run.smithi204.stdout:(17/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 6.7 MB/s | 172 kB 00:00 2024-09-15T01:30:16.187 INFO:teuthology.orchestra.run.smithi204.stdout:(18/34): python3-cherrypy-18.6.1-2.el9.noarch.r 8.1 MB/s | 358 kB 00:00 2024-09-15T01:30:16.219 INFO:teuthology.orchestra.run.smithi204.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 434 kB/s | 11 kB 00:00 2024-09-15T01:30:16.245 INFO:teuthology.orchestra.run.smithi204.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 707 kB/s | 18 kB 00:00 2024-09-15T01:30:16.270 INFO:teuthology.orchestra.run.smithi204.stdout:(21/34): python3-jaraco-collections-3.0.0-8.el9 925 kB/s | 23 kB 00:00 2024-09-15T01:30:16.295 INFO:teuthology.orchestra.run.smithi204.stdout:(22/34): python3-jaraco-functools-3.5.0-2.el9.n 799 kB/s | 19 kB 00:00 2024-09-15T01:30:16.321 INFO:teuthology.orchestra.run.smithi204.stdout:(23/34): python3-jaraco-text-3.2.0-6.el9.noarch 775 kB/s | 20 kB 00:00 2024-09-15T01:30:16.346 INFO:teuthology.orchestra.run.smithi204.stdout:(24/34): python3-logutils-0.3.5-21.el9.noarch.r 1.8 MB/s | 46 kB 00:00 2024-09-15T01:30:16.371 INFO:teuthology.orchestra.run.smithi204.stdout:(25/34): python3-more-itertools-8.12.0-2.el9.no 3.1 MB/s | 79 kB 00:00 2024-09-15T01:30:16.397 INFO:teuthology.orchestra.run.smithi204.stdout:(26/34): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-09-15T01:30:16.422 INFO:teuthology.orchestra.run.smithi204.stdout:(27/34): python3-portend-3.1.0-2.el9.noarch.rpm 660 kB/s | 16 kB 00:00 2024-09-15T01:30:16.425 INFO:teuthology.orchestra.run.smithi178.stdout:(1/34): ceph-17.2.7-1643.gd75f2382.el9.x86_64.r 34 kB/s | 6.5 kB 00:00 2024-09-15T01:30:16.447 INFO:teuthology.orchestra.run.smithi204.stdout:(28/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.5 MB/s | 90 kB 00:00 2024-09-15T01:30:16.473 INFO:teuthology.orchestra.run.smithi204.stdout:(29/34): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-15T01:30:16.498 INFO:teuthology.orchestra.run.smithi204.stdout:(30/34): python3-webob-1.8.8-2.el9.noarch.rpm 9.1 MB/s | 230 kB 00:00 2024-09-15T01:30:16.649 INFO:teuthology.orchestra.run.smithi204.stdout:(31/34): ceph-osd-17.2.7-1643.gd75f2382.el9.x86 5.8 MB/s | 15 MB 00:02 2024-09-15T01:30:16.674 INFO:teuthology.orchestra.run.smithi204.stdout:(32/34): python3-toml-0.10.2-6.el9.noarch.rpm 58 kB/s | 42 kB 00:00 2024-09-15T01:30:16.699 INFO:teuthology.orchestra.run.smithi204.stdout:(33/34): python3-zc-lockfile-2.0-10.el9.noarch. 400 kB/s | 20 kB 00:00 2024-09-15T01:30:16.724 INFO:teuthology.orchestra.run.smithi204.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.8 MB/s | 427 kB 00:00 2024-09-15T01:30:16.728 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:16.729 INFO:teuthology.orchestra.run.smithi204.stdout:Total 6.7 MB/s | 27 MB 00:04 2024-09-15T01:30:16.856 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:30:16.858 INFO:teuthology.orchestra.run.smithi178.stdout:(2/34): ceph-mgr-17.2.7-1643.gd75f2382.el9.x86_ 2.2 MB/s | 1.4 MB 00:00 2024-09-15T01:30:16.893 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:30:16.893 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:30:17.034 INFO:teuthology.orchestra.run.smithi178.stdout:(3/34): ceph-mds-17.2.7-1643.gd75f2382.el9.x86_ 2.6 MB/s | 2.1 MB 00:00 2024-09-15T01:30:17.151 INFO:teuthology.orchestra.run.smithi178.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1643.gd75f 2.0 MB/s | 240 kB 00:00 2024-09-15T01:30:17.295 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:30:17.295 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:30:17.351 INFO:teuthology.orchestra.run.smithi178.stdout:(5/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-15T01:30:17.509 INFO:teuthology.orchestra.run.smithi178.stdout:(6/34): ceph-mon-17.2.7-1643.gd75f2382.el9.x86_ 3.6 MB/s | 3.9 MB 00:01 2024-09-15T01:30:17.551 INFO:teuthology.orchestra.run.smithi178.stdout:(7/34): python3-cryptography-36.0.1-4.el9.x86_6 6.1 MB/s | 1.2 MB 00:00 2024-09-15T01:30:17.601 INFO:teuthology.orchestra.run.smithi178.stdout:(8/34): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-09-15T01:30:17.652 INFO:teuthology.orchestra.run.smithi178.stdout:(9/34): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-15T01:30:17.694 INFO:teuthology.orchestra.run.smithi178.stdout:(10/34): python3-ply-3.11-14.el9.noarch.rpm 579 kB/s | 106 kB 00:00 2024-09-15T01:30:17.719 INFO:teuthology.orchestra.run.smithi178.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-09-15T01:30:17.819 INFO:teuthology.orchestra.run.smithi178.stdout:(12/34): python3-markupsafe-1.1.1-12.el9.x86_64 347 kB/s | 35 kB 00:00 2024-09-15T01:30:17.869 INFO:teuthology.orchestra.run.smithi178.stdout:(13/34): python3-mako-1.1.4-6.el9.noarch.rpm 982 kB/s | 172 kB 00:00 2024-09-15T01:30:17.911 INFO:teuthology.orchestra.run.smithi178.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 841 kB/s | 77 kB 00:00 2024-09-15T01:30:17.937 INFO:teuthology.orchestra.run.smithi178.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 760 kB/s | 51 kB 00:00 2024-09-15T01:30:17.970 INFO:teuthology.orchestra.run.smithi178.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 707 kB/s | 42 kB 00:00 2024-09-15T01:30:18.004 INFO:teuthology.orchestra.run.smithi178.stdout:(17/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 649 kB/s | 43 kB 00:00 2024-09-15T01:30:18.038 INFO:teuthology.orchestra.run.smithi178.stdout:(18/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.6 MB/s | 172 kB 00:00 2024-09-15T01:30:18.063 INFO:teuthology.orchestra.run.smithi178.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 428 kB/s | 11 kB 00:00 2024-09-15T01:30:18.088 INFO:teuthology.orchestra.run.smithi178.stdout:(20/34): python3-cherrypy-18.6.1-2.el9.noarch.r 4.3 MB/s | 358 kB 00:00 2024-09-15T01:30:18.113 INFO:teuthology.orchestra.run.smithi178.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 350 kB/s | 18 kB 00:00 2024-09-15T01:30:18.139 INFO:teuthology.orchestra.run.smithi178.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-09-15T01:30:18.164 INFO:teuthology.orchestra.run.smithi178.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-15T01:30:18.189 INFO:teuthology.orchestra.run.smithi178.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 389 kB/s | 20 kB 00:00 2024-09-15T01:30:18.226 INFO:teuthology.orchestra.run.smithi178.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 746 kB/s | 46 kB 00:00 2024-09-15T01:30:18.251 INFO:teuthology.orchestra.run.smithi178.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.2 MB/s | 79 kB 00:00 2024-09-15T01:30:18.277 INFO:teuthology.orchestra.run.smithi178.stdout:(27/34): python3-portend-3.1.0-2.el9.noarch.rpm 656 kB/s | 16 kB 00:00 2024-09-15T01:30:18.302 INFO:teuthology.orchestra.run.smithi178.stdout:(28/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.5 MB/s | 90 kB 00:00 2024-09-15T01:30:18.327 INFO:teuthology.orchestra.run.smithi178.stdout:(29/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-15T01:30:18.353 INFO:teuthology.orchestra.run.smithi178.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-09-15T01:30:18.386 INFO:teuthology.orchestra.run.smithi178.stdout:(31/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-09-15T01:30:18.412 INFO:teuthology.orchestra.run.smithi178.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 802 kB/s | 20 kB 00:00 2024-09-15T01:30:18.445 INFO:teuthology.orchestra.run.smithi178.stdout:(33/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-15T01:30:18.534 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:30:18.602 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-15T01:30:18.639 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-15T01:30:18.684 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-15T01:30:18.712 INFO:teuthology.orchestra.run.smithi178.stdout:(34/34): ceph-osd-17.2.7-1643.gd75f2382.el9.x86 8.3 MB/s | 15 MB 00:01 2024-09-15T01:30:18.716 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:18.716 INFO:teuthology.orchestra.run.smithi178.stdout:Total 9.1 MB/s | 27 MB 00:02 2024-09-15T01:30:18.776 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-15T01:30:18.840 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:30:18.867 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-15T01:30:18.877 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:30:18.877 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:30:18.907 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-15T01:30:18.950 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-15T01:30:18.989 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-15T01:30:19.023 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-15T01:30:19.056 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-15T01:30:19.097 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-15T01:30:19.132 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-15T01:30:19.166 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-15T01:30:19.215 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-15T01:30:19.247 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T01:30:19.288 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:30:19.288 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:30:19.321 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-15T01:30:19.466 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-15T01:30:19.547 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-15T01:30:19.600 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-15T01:30:19.656 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-15T01:30:19.713 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-15T01:30:19.768 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-15T01:30:19.963 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-15T01:30:20.010 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-15T01:30:20.088 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-15T01:30:20.177 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-15T01:30:20.217 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-15T01:30:20.335 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-15T01:30:20.483 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 29/34 2024-09-15T01:30:20.512 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:30:20.514 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-15T01:30:20.541 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-15T01:30:20.542 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:20.542 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T01:30:20.542 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T01:30:20.542 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T01:30:20.542 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:20.573 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-15T01:30:20.605 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-15T01:30:20.638 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-15T01:30:20.729 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-15T01:30:20.817 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-15T01:30:20.857 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-15T01:30:20.893 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-15T01:30:20.923 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-15T01:30:20.957 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-15T01:30:20.990 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-15T01:30:21.031 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-15T01:30:21.066 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-15T01:30:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-15T01:30:21.148 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-15T01:30:21.181 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T01:30:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-15T01:30:21.401 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-15T01:30:21.491 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-15T01:30:21.558 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-15T01:30:21.597 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-15T01:30:21.645 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-15T01:30:21.700 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-15T01:30:21.866 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-15T01:30:21.892 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-15T01:30:21.893 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:21.893 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T01:30:21.893 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T01:30:21.893 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T01:30:21.893 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:21.893 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-15T01:30:21.943 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-15T01:30:22.020 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-15T01:30:22.108 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-15T01:30:22.141 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-15T01:30:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-15T01:30:22.266 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-15T01:30:22.290 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-15T01:30:22.290 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:22.290 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T01:30:22.290 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T01:30:22.290 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T01:30:22.290 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:22.412 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 29/34 2024-09-15T01:30:22.444 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-15T01:30:22.469 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-15T01:30:22.469 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:22.470 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T01:30:22.470 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T01:30:22.470 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T01:30:22.470 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:22.491 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-15T01:30:22.515 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-15T01:30:22.515 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:22.515 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T01:30:22.515 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T01:30:22.515 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T01:30:22.515 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:22.579 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-15T01:30:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-15T01:30:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-15T01:30:23.850 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:23.850 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T01:30:23.850 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T01:30:23.850 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T01:30:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:24.237 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-15T01:30:24.262 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-15T01:30:24.262 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:24.262 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T01:30:24.262 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T01:30:24.262 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T01:30:24.262 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:24.331 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-15T01:30:24.331 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 1/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 3/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 4/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 5/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 6/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-15T01:30:24.332 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-15T01:30:24.333 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-15T01:30:24.334 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-15T01:30:24.334 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-15T01:30:24.334 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-15T01:30:24.334 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-15T01:30:24.334 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-15T01:30:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-15T01:30:24.501 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-15T01:30:24.501 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:24.501 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T01:30:24.501 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T01:30:24.501 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T01:30:24.501 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:24.567 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-15T01:30:25.387 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-15T01:30:25.387 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:25.387 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:30:25.388 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:25.388 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:25.388 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:25.388 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:25.388 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:25.388 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:25.388 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T01:30:25.388 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T01:30:25.389 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T01:30:25.389 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T01:30:25.389 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T01:30:25.389 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T01:30:25.390 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T01:30:25.390 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T01:30:25.390 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T01:30:25.390 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T01:30:25.390 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T01:30:25.390 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T01:30:25.390 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T01:30:25.391 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T01:30:25.391 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T01:30:25.391 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T01:30:25.391 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T01:30:25.391 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T01:30:25.391 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T01:30:25.391 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T01:30:25.391 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T01:30:25.392 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T01:30:25.392 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T01:30:25.392 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T01:30:25.392 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T01:30:25.392 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T01:30:25.392 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T01:30:25.392 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T01:30:25.392 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:25.393 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:30:25.732 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-base 2024-09-15T01:30:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-15T01:30:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 1/34 2024-09-15T01:30:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2/34 2024-09-15T01:30:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 3/34 2024-09-15T01:30:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 4/34 2024-09-15T01:30:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 5/34 2024-09-15T01:30:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 6/34 2024-09-15T01:30:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-15T01:30:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-15T01:30:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-15T01:30:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-15T01:30:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-15T01:30:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-15T01:30:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-15T01:30:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-15T01:30:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T01:30:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-15T01:30:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-15T01:30:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-15T01:30:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-15T01:30:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-15T01:30:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-15T01:30:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-15T01:30:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-15T01:30:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-15T01:30:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-15T01:30:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-15T01:30:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-15T01:30:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-15T01:30:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-15T01:30:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-15T01:30:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-15T01:30:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-15T01:30:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-15T01:30:26.305 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:00:52 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:30:26.395 INFO:teuthology.orchestra.run.smithi204.stdout:Package ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:30:26.437 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:30:26.437 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-09-15T01:30:26.438 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:30:26.499 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install cephadm 2024-09-15T01:30:27.067 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:00:53 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:30:27.124 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-15T01:30:27.124 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:27.124 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:30:27.125 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:27.125 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:27.125 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:27.125 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:27.125 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:27.125 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:27.125 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T01:30:27.125 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T01:30:27.125 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T01:30:27.125 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T01:30:27.125 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T01:30:27.125 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T01:30:27.126 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T01:30:27.127 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T01:30:27.127 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T01:30:27.127 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T01:30:27.127 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T01:30:27.127 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T01:30:27.127 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T01:30:27.127 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T01:30:27.127 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:27.127 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:30:27.200 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:30:27.201 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:27.201 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-15T01:30:27.201 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:27.201 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:30:27.201 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 83 k 2024-09-15T01:30:27.201 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:27.201 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:30:27.201 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:27.201 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-09-15T01:30:27.201 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:27.201 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 83 k 2024-09-15T01:30:27.201 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 375 k 2024-09-15T01:30:27.202 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:30:27.493 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-base 2024-09-15T01:30:27.635 INFO:teuthology.orchestra.run.smithi204.stdout:cephadm-17.2.7-1643.gd75f2382.el9.noarch.rpm 191 kB/s | 83 kB 00:00 2024-09-15T01:30:27.635 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:27.636 INFO:teuthology.orchestra.run.smithi204.stdout:Total 190 kB/s | 83 kB 00:00 2024-09-15T01:30:27.636 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:30:27.641 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:30:27.641 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:30:27.643 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:30:27.643 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:30:27.714 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:30:28.057 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:51 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:30:28.148 INFO:teuthology.orchestra.run.smithi178.stdout:Package ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:30:28.168 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:28.189 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:30:28.190 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-15T01:30:28.190 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:30:28.264 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install cephadm 2024-09-15T01:30:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:28.656 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:28.826 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:51 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:30:28.903 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:28.903 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:28.903 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:30:28.903 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:28.904 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:28.904 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:30:28.958 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:30:28.959 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:28.959 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-15T01:30:28.959 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:28.959 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:30:28.959 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 83 k 2024-09-15T01:30:28.959 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 83 k 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 375 k 2024-09-15T01:30:28.960 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:30:29.158 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T01:30:29.724 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:00:55 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:30:29.738 INFO:teuthology.orchestra.run.smithi178.stdout:cephadm-17.2.7-1643.gd75f2382.el9.noarch.rpm 106 kB/s | 83 kB 00:00 2024-09-15T01:30:29.738 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:29.739 INFO:teuthology.orchestra.run.smithi178.stdout:Total 106 kB/s | 83 kB 00:00 2024-09-15T01:30:29.739 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:30:29.745 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:30:29.745 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:30:29.747 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:30:29.748 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:30:29.820 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:30:29.860 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repo Size 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 137 k 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 137 k 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 379 k 2024-09-15T01:30:29.861 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:30:30.276 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:30.384 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:30.400 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-immutable-object-cache-17.2.7-1643.gd75f23 255 kB/s | 137 kB 00:00 2024-09-15T01:30:30.400 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:30.401 INFO:teuthology.orchestra.run.smithi204.stdout:Total 255 kB/s | 137 kB 00:00 2024-09-15T01:30:30.401 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:30:30.408 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:30:30.408 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:30:30.466 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:30:30.466 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:30:30.609 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:30:30.667 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:30.692 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:30.692 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:30.692 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T01:30:30.692 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:30.999 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:30:30.999 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:31.000 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:30:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:31.000 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:30:31.233 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T01:30:31.620 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:31.620 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:31.621 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:30:31.621 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:31.621 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:31.621 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:30:31.807 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:54 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:30:31.844 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-mgr 2024-09-15T01:30:31.941 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:30:31.942 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:31.942 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-09-15T01:30:31.942 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:31.942 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:30:31.942 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 137 k 2024-09-15T01:30:31.942 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:31.943 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:30:31.943 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:31.943 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-15T01:30:31.943 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:31.943 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 137 k 2024-09-15T01:30:31.943 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 379 k 2024-09-15T01:30:31.943 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:30:32.285 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-immutable-object-cache-17.2.7-1643.gd75f23 401 kB/s | 137 kB 00:00 2024-09-15T01:30:32.286 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:32.286 INFO:teuthology.orchestra.run.smithi178.stdout:Total 399 kB/s | 137 kB 00:00 2024-09-15T01:30:32.286 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:30:32.294 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:30:32.294 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:30:32.350 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:30:32.351 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:30:32.423 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:00:58 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:30:32.497 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:30:32.516 INFO:teuthology.orchestra.run.smithi204.stdout:Package ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:30:32.560 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:30:32.561 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-09-15T01:30:32.561 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:30:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:32.620 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:32.620 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:30:32.621 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T01:30:32.621 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:32.622 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T01:30:33.187 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:00:59 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:30:33.318 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:30:33.319 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:33.319 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-15T01:30:33.320 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:33.320 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:30:33.320 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 3.4 M 2024-09-15T01:30:33.320 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-15T01:30:33.320 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 24 k 2024-09-15T01:30:33.320 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 15 k 2024-09-15T01:30:33.320 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-15T01:30:33.321 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T01:30:33.321 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T01:30:33.321 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-09-15T01:30:33.321 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-15T01:30:33.321 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:33.321 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:30:33.321 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:33.321 INFO:teuthology.orchestra.run.smithi204.stdout:Install 7 Packages 2024-09-15T01:30:33.321 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:33.322 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 3.7 M 2024-09-15T01:30:33.322 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 80 M 2024-09-15T01:30:33.322 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:30:33.500 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-15T01:30:33.500 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:33.500 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:30:33.500 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:30:33.501 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:33.501 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:30:33.646 INFO:teuthology.orchestra.run.smithi204.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1643.gd75 155 kB/s | 24 kB 00:00 2024-09-15T01:30:33.671 INFO:teuthology.orchestra.run.smithi204.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1643.gd75f 80 kB/s | 15 kB 00:00 2024-09-15T01:30:33.713 INFO:teuthology.orchestra.run.smithi204.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 136 kB/s | 9.0 kB 00:00 2024-09-15T01:30:33.738 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr 2024-09-15T01:30:33.738 INFO:teuthology.orchestra.run.smithi204.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-15T01:30:33.764 INFO:teuthology.orchestra.run.smithi204.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-15T01:30:33.789 INFO:teuthology.orchestra.run.smithi204.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-15T01:30:33.964 INFO:teuthology.orchestra.run.smithi204.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1643.gd75f2382 7.1 MB/s | 3.4 MB 00:00 2024-09-15T01:30:33.967 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:33.968 INFO:teuthology.orchestra.run.smithi204.stdout:Total 5.6 MB/s | 3.7 MB 00:00 2024-09-15T01:30:33.968 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:30:33.976 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:30:33.977 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:30:34.056 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:30:34.056 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:30:34.332 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:57 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:30:34.361 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:30:34.415 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-15T01:30:34.422 INFO:teuthology.orchestra.run.smithi178.stdout:Package ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:30:34.449 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-15T01:30:34.524 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-15T01:30:34.524 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:30:34.524 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-15T01:30:34.525 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:30:34.542 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T01:30:34.545 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-15T01:30:34.574 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 5/7 2024-09-15T01:30:35.102 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:58 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:30:35.235 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:30:35.236 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-15T01:30:35.236 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:35.236 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:30:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 3.4 M 2024-09-15T01:30:35.236 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-15T01:30:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 24 k 2024-09-15T01:30:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 15 k 2024-09-15T01:30:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-15T01:30:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T01:30:35.236 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T01:30:35.237 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-09-15T01:30:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-15T01:30:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:35.237 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:30:35.237 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:30:35.237 INFO:teuthology.orchestra.run.smithi178.stdout:Install 7 Packages 2024-09-15T01:30:35.237 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:35.238 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 3.7 M 2024-09-15T01:30:35.238 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 80 M 2024-09-15T01:30:35.238 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:30:35.582 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 6/7 2024-09-15T01:30:35.619 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-15T01:30:35.756 INFO:teuthology.orchestra.run.smithi178.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1643.gd75 165 kB/s | 24 kB 00:00 2024-09-15T01:30:35.781 INFO:teuthology.orchestra.run.smithi178.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1643.gd75f 84 kB/s | 15 kB 00:00 2024-09-15T01:30:35.823 INFO:teuthology.orchestra.run.smithi178.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-15T01:30:35.848 INFO:teuthology.orchestra.run.smithi178.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-15T01:30:35.876 INFO:teuthology.orchestra.run.smithi178.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-15T01:30:35.907 INFO:teuthology.orchestra.run.smithi178.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-09-15T01:30:36.074 INFO:teuthology.orchestra.run.smithi178.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1643.gd75f2382 7.2 MB/s | 3.4 MB 00:00 2024-09-15T01:30:36.077 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:36.081 INFO:teuthology.orchestra.run.smithi178.stdout:Total 4.3 MB/s | 3.7 MB 00:00 2024-09-15T01:30:36.081 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:30:36.086 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:30:36.087 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:30:36.186 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:30:36.186 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:30:36.513 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:30:36.584 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-15T01:30:36.626 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-15T01:30:36.674 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-15T01:30:36.675 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 1/7 2024-09-15T01:30:36.675 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 2/7 2024-09-15T01:30:36.675 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 3/7 2024-09-15T01:30:36.675 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-15T01:30:36.675 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-15T01:30:36.675 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-15T01:30:36.681 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-15T01:30:36.713 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-15T01:30:36.742 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 5/7 2024-09-15T01:30:36.944 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-15T01:30:36.944 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:36.944 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:30:36.944 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:36.944 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:36.944 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:36.944 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-15T01:30:36.945 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-15T01:30:36.945 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T01:30:36.945 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T01:30:36.945 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:36.945 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:30:37.185 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T01:30:37.734 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 6/7 2024-09-15T01:30:37.752 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:03 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:30:37.770 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-15T01:30:37.887 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:30:37.888 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-09-15T01:30:37.888 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-15T01:30:37.888 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-09-15T01:30:37.888 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:30:37.888 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 7.4 M 2024-09-15T01:30:37.888 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-15T01:30:37.888 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T01:30:37.888 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T01:30:37.888 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout:Install 12 Packages 2024-09-15T01:30:37.889 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:37.890 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 38 M 2024-09-15T01:30:37.890 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 212 M 2024-09-15T01:30:37.890 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:30:38.831 INFO:teuthology.orchestra.run.smithi204.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 13 MB/s | 7.4 MB 00:00 2024-09-15T01:30:38.876 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-15T01:30:38.876 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 1/7 2024-09-15T01:30:38.876 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 2/7 2024-09-15T01:30:38.877 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 3/7 2024-09-15T01:30:38.877 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-15T01:30:38.877 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-15T01:30:38.877 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-15T01:30:39.090 INFO:teuthology.orchestra.run.smithi204.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 234 kB/s | 191 kB 00:00 2024-09-15T01:30:39.115 INFO:teuthology.orchestra.run.smithi204.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 115 kB/s | 33 kB 00:00 2024-09-15T01:30:39.157 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-15T01:30:39.157 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:39.157 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:30:39.157 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:39.157 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:39.157 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:39.157 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-15T01:30:39.157 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-15T01:30:39.157 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T01:30:39.158 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T01:30:39.158 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:39.158 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:30:39.224 INFO:teuthology.orchestra.run.smithi204.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 167 kB/s | 18 kB 00:00 2024-09-15T01:30:39.257 INFO:teuthology.orchestra.run.smithi204.stdout:(5/12): libgfortran-11.5.0-2.el9.x86_64.rpm 814 kB/s | 801 kB 00:00 2024-09-15T01:30:39.341 INFO:teuthology.orchestra.run.smithi204.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 329 kB/s | 38 kB 00:00 2024-09-15T01:30:39.399 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T01:30:39.633 INFO:teuthology.orchestra.run.smithi204.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 839 kB/s | 245 kB 00:00 2024-09-15T01:30:39.964 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:02 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:30:40.017 INFO:teuthology.orchestra.run.smithi204.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.2 MB/s | 3.0 MB 00:00 2024-09-15T01:30:40.097 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:30:40.098 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-09-15T01:30:40.098 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-15T01:30:40.098 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-09-15T01:30:40.098 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:30:40.098 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 7.4 M 2024-09-15T01:30:40.098 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-15T01:30:40.098 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T01:30:40.098 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-09-15T01:30:40.099 INFO:teuthology.orchestra.run.smithi178.stdout:Install 12 Packages 2024-09-15T01:30:40.100 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:30:40.100 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 38 M 2024-09-15T01:30:40.100 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 212 M 2024-09-15T01:30:40.100 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:30:40.183 INFO:teuthology.orchestra.run.smithi204.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.3 MB/s | 5.1 MB 00:00 2024-09-15T01:30:40.209 INFO:teuthology.orchestra.run.smithi204.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.3 MB/s | 250 kB 00:00 2024-09-15T01:30:40.276 INFO:teuthology.orchestra.run.smithi204.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.9 MB/s | 4.9 MB 00:01 2024-09-15T01:30:40.565 INFO:teuthology.orchestra.run.smithi178.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-15T01:30:40.623 INFO:teuthology.orchestra.run.smithi178.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 801 kB 00:00 2024-09-15T01:30:40.826 INFO:teuthology.orchestra.run.smithi204.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-15T01:30:40.827 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:40.827 INFO:teuthology.orchestra.run.smithi204.stdout:Total 13 MB/s | 38 MB 00:02 2024-09-15T01:30:40.857 INFO:teuthology.orchestra.run.smithi178.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 112 kB/s | 33 kB 00:00 2024-09-15T01:30:40.949 INFO:teuthology.orchestra.run.smithi178.stdout:(4/12): ceph-mgr-diskprediction-local-17.2.7-16 13 MB/s | 7.4 MB 00:00 2024-09-15T01:30:40.975 INFO:teuthology.orchestra.run.smithi178.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 155 kB/s | 18 kB 00:00 2024-09-15T01:30:41.181 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:30:41.207 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:30:41.207 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:30:41.233 INFO:teuthology.orchestra.run.smithi178.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 136 kB/s | 38 kB 00:00 2024-09-15T01:30:41.700 INFO:teuthology.orchestra.run.smithi178.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 524 kB/s | 245 kB 00:00 2024-09-15T01:30:41.733 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:30:41.733 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:30:41.758 INFO:teuthology.orchestra.run.smithi178.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-09-15T01:30:41.884 INFO:teuthology.orchestra.run.smithi178.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 250 kB 00:00 2024-09-15T01:30:42.536 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:30:42.604 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T01:30:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T01:30:42.726 INFO:teuthology.orchestra.run.smithi178.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.8 MB/s | 4.9 MB 00:01 2024-09-15T01:30:42.743 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T01:30:42.784 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T01:30:42.978 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T01:30:43.007 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T01:30:43.091 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T01:30:43.749 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T01:30:43.844 INFO:teuthology.orchestra.run.smithi178.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.4 MB/s | 5.1 MB 00:02 2024-09-15T01:30:43.872 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T01:30:45.139 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T01:30:46.195 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T01:30:46.225 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-15T01:30:48.863 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-15T01:30:48.863 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 1/12 2024-09-15T01:30:48.863 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T01:30:48.863 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T01:30:48.863 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T01:30:48.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T01:30:48.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T01:30:48.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T01:30:48.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T01:30:48.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T01:30:48.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T01:30:48.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T01:30:49.252 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T01:30:49.252 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:49.252 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:30:49.252 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:49.252 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T01:30:49.253 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T01:30:49.253 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T01:30:49.253 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T01:30:49.253 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T01:30:49.253 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T01:30:49.253 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T01:30:49.253 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T01:30:49.253 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T01:30:49.253 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T01:30:49.253 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T01:30:49.253 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:49.253 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:30:49.634 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-mgr-rook 2024-09-15T01:30:50.221 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:16 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:30:50.364 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:30:50.365 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:50.365 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-15T01:30:50.365 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:50.365 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:30:50.365 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 47 k 2024-09-15T01:30:50.365 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-15T01:30:50.365 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T01:30:50.365 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:30:50.366 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:50.367 INFO:teuthology.orchestra.run.smithi204.stdout:Install 13 Packages 2024-09-15T01:30:50.367 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:50.367 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 2.2 M 2024-09-15T01:30:50.367 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 26 M 2024-09-15T01:30:50.367 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:30:50.676 INFO:teuthology.orchestra.run.smithi204.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 190 kB/s | 19 kB 00:00 2024-09-15T01:30:50.701 INFO:teuthology.orchestra.run.smithi204.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 204 kB/s | 26 kB 00:00 2024-09-15T01:30:50.759 INFO:teuthology.orchestra.run.smithi204.stdout:(3/13): ceph-mgr-rook-17.2.7-1643.gd75f2382.el9 257 kB/s | 47 kB 00:00 2024-09-15T01:30:50.785 INFO:teuthology.orchestra.run.smithi204.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-15T01:30:50.810 INFO:teuthology.orchestra.run.smithi204.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-15T01:30:50.852 INFO:teuthology.orchestra.run.smithi204.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 771 kB/s | 32 kB 00:00 2024-09-15T01:30:50.877 INFO:teuthology.orchestra.run.smithi204.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 581 kB/s | 54 kB 00:00 2024-09-15T01:30:50.903 INFO:teuthology.orchestra.run.smithi204.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 280 kB/s | 14 kB 00:00 2024-09-15T01:30:50.936 INFO:teuthology.orchestra.run.smithi204.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-09-15T01:30:50.978 INFO:teuthology.orchestra.run.smithi204.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-15T01:30:51.003 INFO:teuthology.orchestra.run.smithi204.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 906 kB/s | 59 kB 00:00 2024-09-15T01:30:51.029 INFO:teuthology.orchestra.run.smithi204.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-15T01:30:51.062 INFO:teuthology.orchestra.run.smithi204.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.7 MB/s | 1.0 MB 00:00 2024-09-15T01:30:51.065 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:51.065 INFO:teuthology.orchestra.run.smithi204.stdout:Total 3.2 MB/s | 2.2 MB 00:00 2024-09-15T01:30:51.132 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:30:51.147 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:30:51.147 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:30:51.236 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:30:51.236 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:30:51.738 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:30:51.893 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T01:30:51.947 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T01:30:52.003 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T01:30:52.048 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T01:30:52.098 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T01:30:52.209 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T01:30:52.318 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T01:30:52.366 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T01:30:53.086 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T01:30:53.147 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T01:30:53.181 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T01:30:53.191 INFO:teuthology.orchestra.run.smithi178.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.4 MB/s | 16 MB 00:11 2024-09-15T01:30:53.192 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:53.192 INFO:teuthology.orchestra.run.smithi178.stdout:Total 2.9 MB/s | 38 MB 00:13 2024-09-15T01:30:53.227 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T01:30:53.251 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-15T01:30:53.547 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:30:53.573 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:30:53.573 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:30:54.071 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:30:54.071 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:30:54.138 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-15T01:30:54.139 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 1/13 2024-09-15T01:30:54.139 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T01:30:54.139 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T01:30:54.139 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T01:30:54.139 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T01:30:54.139 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T01:30:54.139 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T01:30:54.139 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T01:30:54.139 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T01:30:54.139 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T01:30:54.139 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T01:30:54.139 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T01:30:54.471 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T01:30:54.471 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:54.471 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:30:54.471 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:54.471 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T01:30:54.471 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T01:30:54.471 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T01:30:54.472 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T01:30:54.472 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T01:30:54.472 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T01:30:54.472 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T01:30:54.472 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T01:30:54.472 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T01:30:54.472 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T01:30:54.472 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T01:30:54.472 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T01:30:54.472 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:54.472 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:30:54.711 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T01:30:54.883 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:30:54.951 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T01:30:54.993 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T01:30:55.088 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T01:30:55.132 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T01:30:55.280 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:21 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:30:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T01:30:55.347 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T01:30:55.412 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:30:55.413 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:55.413 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-15T01:30:55.413 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:55.414 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:30:55.414 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 124 k 2024-09-15T01:30:55.414 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-15T01:30:55.414 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T01:30:55.414 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T01:30:55.414 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T01:30:55.414 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T01:30:55.414 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T01:30:55.414 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:55.414 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:30:55.414 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:55.414 INFO:teuthology.orchestra.run.smithi204.stdout:Install 6 Packages 2024-09-15T01:30:55.415 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:55.415 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 7.0 M 2024-09-15T01:30:55.415 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 33 M 2024-09-15T01:30:55.415 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:30:55.424 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T01:30:55.816 INFO:teuthology.orchestra.run.smithi204.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.8 MB/s | 248 kB 00:00 2024-09-15T01:30:55.874 INFO:teuthology.orchestra.run.smithi204.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-15T01:30:55.900 INFO:teuthology.orchestra.run.smithi204.stdout:(3/6): ceph-mgr-cephadm-17.2.7-1643.gd75f2382.e 557 kB/s | 124 kB 00:00 2024-09-15T01:30:55.925 INFO:teuthology.orchestra.run.smithi204.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-15T01:30:55.967 INFO:teuthology.orchestra.run.smithi204.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-15T01:30:56.078 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T01:30:56.202 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T01:30:56.442 INFO:teuthology.orchestra.run.smithi204.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 7.8 MB/s | 6.0 MB 00:00 2024-09-15T01:30:56.444 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:30:56.445 INFO:teuthology.orchestra.run.smithi204.stdout:Total 6.8 MB/s | 7.0 MB 00:01 2024-09-15T01:30:56.516 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:30:56.527 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:30:56.527 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:30:56.627 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:30:56.628 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:30:56.941 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:30:57.125 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T01:30:57.178 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T01:30:57.439 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T01:30:57.705 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T01:30:57.780 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T01:30:57.837 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T01:30:57.873 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-15T01:30:58.469 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T01:30:58.496 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-15T01:30:58.608 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-15T01:30:58.608 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/6 2024-09-15T01:30:58.608 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T01:30:58.608 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T01:30:58.608 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T01:30:58.608 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T01:30:58.875 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T01:30:58.876 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:58.876 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:30:58.876 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:30:58.876 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T01:30:58.876 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T01:30:58.876 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T01:30:58.876 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T01:30:58.876 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T01:30:58.876 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:58.876 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:30:59.150 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-fuse 2024-09-15T01:30:59.719 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:25 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:30:59.851 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:30:59.852 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:59.852 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-15T01:30:59.852 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:59.852 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:30:59.852 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 813 k 2024-09-15T01:30:59.852 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-15T01:30:59.853 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T01:30:59.853 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:59.853 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:30:59.853 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:30:59.853 INFO:teuthology.orchestra.run.smithi204.stdout:Install 2 Packages 2024-09-15T01:30:59.853 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:30:59.853 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 893 k 2024-09-15T01:30:59.854 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 2.6 M 2024-09-15T01:30:59.854 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:31:00.027 INFO:teuthology.orchestra.run.smithi204.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 525 kB/s | 80 kB 00:00 2024-09-15T01:31:00.236 INFO:teuthology.orchestra.run.smithi204.stdout:(2/2): ceph-fuse-17.2.7-1643.gd75f2382.el9.x86_ 2.2 MB/s | 813 kB 00:00 2024-09-15T01:31:00.236 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:00.237 INFO:teuthology.orchestra.run.smithi204.stdout:Total 2.3 MB/s | 893 kB 00:00 2024-09-15T01:31:00.248 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:31:00.269 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:31:00.270 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:31:00.348 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:31:00.348 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:31:00.533 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:31:00.661 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T01:31:00.735 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-15T01:31:01.186 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-15T01:31:01.186 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 1/12 2024-09-15T01:31:01.186 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T01:31:01.186 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T01:31:01.186 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T01:31:01.186 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T01:31:01.186 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T01:31:01.187 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T01:31:01.187 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T01:31:01.187 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T01:31:01.187 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T01:31:01.187 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T01:31:01.449 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-15T01:31:01.449 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/2 2024-09-15T01:31:01.647 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T01:31:01.647 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:01.647 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:31:01.647 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:01.647 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T01:31:01.647 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T01:31:01.647 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T01:31:01.647 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T01:31:01.647 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T01:31:01.647 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T01:31:01.647 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T01:31:01.648 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T01:31:01.648 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T01:31:01.648 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T01:31:01.648 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T01:31:01.648 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:01.648 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:01.725 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T01:31:01.725 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:01.725 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:31:01.725 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T01:31:01.725 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:01.725 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:01.993 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-volume 2024-09-15T01:31:02.041 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-rook 2024-09-15T01:31:02.581 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:28 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:02.622 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:25 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:02.718 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:02.719 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:02.719 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-15T01:31:02.719 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:02.719 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:31:02.719 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 270 k 2024-09-15T01:31:02.720 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:02.720 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:31:02.720 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:02.720 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-09-15T01:31:02.720 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:02.720 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 270 k 2024-09-15T01:31:02.720 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 1.3 M 2024-09-15T01:31:02.720 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:31:02.757 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:02.758 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:02.758 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-15T01:31:02.758 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:02.758 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:31:02.758 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 47 k 2024-09-15T01:31:02.758 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-15T01:31:02.759 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T01:31:02.759 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T01:31:02.759 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T01:31:02.759 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T01:31:02.759 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T01:31:02.759 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T01:31:02.759 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T01:31:02.760 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T01:31:02.760 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T01:31:02.760 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T01:31:02.760 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T01:31:02.760 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T01:31:02.760 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:02.761 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:31:02.761 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:02.761 INFO:teuthology.orchestra.run.smithi178.stdout:Install 13 Packages 2024-09-15T01:31:02.761 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:02.762 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 2.2 M 2024-09-15T01:31:02.762 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 26 M 2024-09-15T01:31:02.762 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:31:02.950 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-volume-17.2.7-1643.gd75f2382.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-15T01:31:02.951 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:02.951 INFO:teuthology.orchestra.run.smithi204.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-15T01:31:02.951 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:31:02.959 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:31:02.959 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:31:02.988 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:31:02.989 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:31:03.189 INFO:teuthology.orchestra.run.smithi178.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 144 kB/s | 19 kB 00:00 2024-09-15T01:31:03.203 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:31:03.214 INFO:teuthology.orchestra.run.smithi178.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 161 kB/s | 26 kB 00:00 2024-09-15T01:31:03.239 INFO:teuthology.orchestra.run.smithi178.stdout:(3/13): ceph-mgr-rook-17.2.7-1643.gd75f2382.el9 258 kB/s | 47 kB 00:00 2024-09-15T01:31:03.269 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:03.285 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:03.285 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:31:03.285 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T01:31:03.285 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:03.373 INFO:teuthology.orchestra.run.smithi178.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-09-15T01:31:03.398 INFO:teuthology.orchestra.run.smithi178.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.0 MB/s | 222 kB 00:00 2024-09-15T01:31:03.431 INFO:teuthology.orchestra.run.smithi178.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 918 kB/s | 54 kB 00:00 2024-09-15T01:31:03.465 INFO:teuthology.orchestra.run.smithi178.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 483 kB/s | 32 kB 00:00 2024-09-15T01:31:03.490 INFO:teuthology.orchestra.run.smithi178.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-09-15T01:31:03.515 INFO:teuthology.orchestra.run.smithi178.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 172 kB/s | 14 kB 00:00 2024-09-15T01:31:03.549 INFO:teuthology.orchestra.run.smithi178.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-15T01:31:03.574 INFO:teuthology.orchestra.run.smithi178.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-15T01:31:03.600 INFO:teuthology.orchestra.run.smithi178.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-15T01:31:03.642 INFO:teuthology.orchestra.run.smithi178.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-15T01:31:03.645 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:03.645 INFO:teuthology.orchestra.run.smithi178.stdout:Total 2.5 MB/s | 2.2 MB 00:00 2024-09-15T01:31:03.709 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:31:03.724 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:31:03.724 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:31:03.809 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:31:03.810 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:31:04.275 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:04.275 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:04.275 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:31:04.275 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:04.275 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:04.275 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:04.310 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:31:04.479 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T01:31:04.542 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T01:31:04.606 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T01:31:04.611 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install librados-devel 2024-09-15T01:31:04.643 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T01:31:04.684 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T01:31:04.785 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T01:31:04.888 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T01:31:04.945 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T01:31:05.192 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:31 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:05.329 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:05.330 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:05.330 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repo Size 2024-09-15T01:31:05.330 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:05.330 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:31:05.330 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 126 k 2024-09-15T01:31:05.330 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:05.330 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:31:05.330 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:05.330 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-09-15T01:31:05.330 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:05.330 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 126 k 2024-09-15T01:31:05.331 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 448 k 2024-09-15T01:31:05.331 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:31:05.515 INFO:teuthology.orchestra.run.smithi204.stdout:librados-devel-17.2.7-1643.gd75f2382.el9.x86_64 687 kB/s | 126 kB 00:00 2024-09-15T01:31:05.515 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:05.515 INFO:teuthology.orchestra.run.smithi204.stdout:Total 683 kB/s | 126 kB 00:00 2024-09-15T01:31:05.515 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:31:05.520 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:31:05.520 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:31:05.584 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:31:05.584 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:31:05.671 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T01:31:05.731 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:31:05.742 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T01:31:05.776 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T01:31:05.821 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T01:31:05.836 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:05.845 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-15T01:31:06.229 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:06.474 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:06.475 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:06.475 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:31:06.475 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:06.475 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:06.475 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:06.682 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-15T01:31:06.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 1/13 2024-09-15T01:31:06.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T01:31:06.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T01:31:06.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T01:31:06.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T01:31:06.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T01:31:06.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T01:31:06.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T01:31:06.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T01:31:06.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T01:31:06.684 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T01:31:06.684 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T01:31:06.750 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install libcephfs2 2024-09-15T01:31:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T01:31:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:07.020 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:31:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T01:31:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T01:31:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T01:31:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T01:31:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T01:31:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T01:31:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T01:31:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T01:31:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T01:31:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T01:31:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T01:31:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T01:31:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:07.021 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:07.285 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T01:31:07.320 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:33 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:07.410 INFO:teuthology.orchestra.run.smithi204.stdout:Package libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:07.455 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:07.456 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-09-15T01:31:07.456 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:07.516 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install libcephfs-devel 2024-09-15T01:31:07.843 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:30 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:07.975 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:07.976 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:07.976 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-15T01:31:07.976 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:07.976 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:31:07.976 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 124 k 2024-09-15T01:31:07.976 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-15T01:31:07.976 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T01:31:07.976 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T01:31:07.977 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T01:31:07.977 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T01:31:07.977 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T01:31:07.977 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:07.977 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:31:07.977 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:07.977 INFO:teuthology.orchestra.run.smithi178.stdout:Install 6 Packages 2024-09-15T01:31:07.977 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:07.978 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 7.0 M 2024-09-15T01:31:07.978 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 33 M 2024-09-15T01:31:07.978 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:31:08.085 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:34 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:08.218 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:08.219 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:08.219 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repo Size 2024-09-15T01:31:08.219 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:08.219 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:31:08.219 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-15T01:31:08.219 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:08.219 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:31:08.219 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:08.220 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-09-15T01:31:08.220 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:08.220 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 25 k 2024-09-15T01:31:08.220 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 110 k 2024-09-15T01:31:08.220 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:31:08.369 INFO:teuthology.orchestra.run.smithi204.stdout:libcephfs-devel-17.2.7-1643.gd75f2382.el9.x86_6 168 kB/s | 25 kB 00:00 2024-09-15T01:31:08.369 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:08.369 INFO:teuthology.orchestra.run.smithi204.stdout:Total 166 kB/s | 25 kB 00:00 2024-09-15T01:31:08.370 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:31:08.373 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:31:08.373 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:31:08.375 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:31:08.375 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:31:08.461 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:31:08.566 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:08.835 INFO:teuthology.orchestra.run.smithi178.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1643.gd75f2382.e 585 kB/s | 124 kB 00:00 2024-09-15T01:31:08.894 INFO:teuthology.orchestra.run.smithi178.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-15T01:31:08.919 INFO:teuthology.orchestra.run.smithi178.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-15T01:31:08.944 INFO:teuthology.orchestra.run.smithi178.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-15T01:31:08.954 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:09.128 INFO:teuthology.orchestra.run.smithi178.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 492 kB/s | 248 kB 00:00 2024-09-15T01:31:09.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:09.172 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:09.172 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:31:09.172 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:09.172 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:09.172 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:09.374 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install librados2 2024-09-15T01:31:09.644 INFO:teuthology.orchestra.run.smithi178.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.8 MB/s | 6.0 MB 00:01 2024-09-15T01:31:09.646 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:09.647 INFO:teuthology.orchestra.run.smithi178.stdout:Total 4.2 MB/s | 7.0 MB 00:01 2024-09-15T01:31:09.718 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:31:09.729 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:31:09.729 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:31:09.831 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:31:09.832 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:31:09.946 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:35 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:10.038 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:10.082 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:10.083 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-09-15T01:31:10.083 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:10.138 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:31:10.156 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install librbd1 2024-09-15T01:31:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T01:31:10.371 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T01:31:10.726 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:36 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:10.817 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:10.860 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:10.861 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-09-15T01:31:10.862 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:10.895 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T01:31:10.927 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install python3-rados 2024-09-15T01:31:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T01:31:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T01:31:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-15T01:31:11.498 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:37 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:11.588 INFO:teuthology.orchestra.run.smithi204.stdout:Package python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:11.631 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:11.632 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-09-15T01:31:11.632 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:11.704 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install python3-rgw 2024-09-15T01:31:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-15T01:31:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/6 2024-09-15T01:31:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T01:31:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T01:31:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T01:31:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T01:31:12.108 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T01:31:12.109 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:12.109 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:31:12.109 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:12.109 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T01:31:12.109 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T01:31:12.109 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T01:31:12.109 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T01:31:12.109 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T01:31:12.109 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:12.109 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:12.267 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:38 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:12.356 INFO:teuthology.orchestra.run.smithi204.stdout:Package python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:12.373 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-fuse 2024-09-15T01:31:12.398 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:12.399 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-09-15T01:31:12.399 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:12.473 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install python3-cephfs 2024-09-15T01:31:12.939 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:35 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:13.042 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:39 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:13.071 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:13.071 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:13.071 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-15T01:31:13.072 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:13.072 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:31:13.072 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 813 k 2024-09-15T01:31:13.072 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-15T01:31:13.072 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T01:31:13.072 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:13.072 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:31:13.072 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:13.072 INFO:teuthology.orchestra.run.smithi178.stdout:Install 2 Packages 2024-09-15T01:31:13.072 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:13.076 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 893 k 2024-09-15T01:31:13.076 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 2.6 M 2024-09-15T01:31:13.076 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:31:13.132 INFO:teuthology.orchestra.run.smithi204.stdout:Package python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:13.176 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:13.177 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-09-15T01:31:13.177 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:13.253 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install python3-rbd 2024-09-15T01:31:13.280 INFO:teuthology.orchestra.run.smithi178.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 783 kB/s | 80 kB 00:00 2024-09-15T01:31:13.463 INFO:teuthology.orchestra.run.smithi178.stdout:(2/2): ceph-fuse-17.2.7-1643.gd75f2382.el9.x86_ 2.8 MB/s | 813 kB 00:00 2024-09-15T01:31:13.464 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:13.464 INFO:teuthology.orchestra.run.smithi178.stdout:Total 2.2 MB/s | 893 kB 00:00 2024-09-15T01:31:13.475 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:31:13.497 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:31:13.497 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:31:13.575 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:31:13.575 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:31:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:31:13.820 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:39 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:13.907 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T01:31:13.909 INFO:teuthology.orchestra.run.smithi204.stdout:Package python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:13.952 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:13.953 INFO:teuthology.orchestra.run.smithi204.stdout:Nothing to do. 2024-09-15T01:31:13.953 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:13.977 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-15T01:31:14.022 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install rbd-fuse 2024-09-15T01:31:14.588 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:40 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:14.653 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-15T01:31:14.653 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/2 2024-09-15T01:31:14.723 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:14.724 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:14.724 INFO:teuthology.orchestra.run.smithi204.stdout: Package Architecture Version Repository Size 2024-09-15T01:31:14.724 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:14.724 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:31:14.724 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 82 k 2024-09-15T01:31:14.724 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:14.724 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:31:14.724 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:14.724 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-09-15T01:31:14.724 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:14.724 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 82 k 2024-09-15T01:31:14.724 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 222 k 2024-09-15T01:31:14.725 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:31:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T01:31:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:14.916 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:31:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T01:31:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:14.916 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:14.923 INFO:teuthology.orchestra.run.smithi204.stdout:rbd-fuse-17.2.7-1643.gd75f2382.el9.x86_64.rpm 413 kB/s | 82 kB 00:00 2024-09-15T01:31:14.923 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:14.924 INFO:teuthology.orchestra.run.smithi204.stdout:Total 410 kB/s | 82 kB 00:00 2024-09-15T01:31:14.924 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:31:14.930 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:31:14.930 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:31:14.993 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:31:14.994 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:31:15.133 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:31:15.208 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-volume 2024-09-15T01:31:15.236 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:15.615 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:15.779 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:38 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:15.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:15.865 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:31:15.865 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:15.865 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:15.916 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:15.917 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:15.917 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-15T01:31:15.917 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:15.917 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:31:15.917 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 270 k 2024-09-15T01:31:15.917 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:15.917 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:31:15.917 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:15.918 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-15T01:31:15.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:15.918 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 270 k 2024-09-15T01:31:15.918 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 1.3 M 2024-09-15T01:31:15.918 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:31:16.121 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install rbd-mirror 2024-09-15T01:31:16.159 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-volume-17.2.7-1643.gd75f2382.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-15T01:31:16.160 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:16.160 INFO:teuthology.orchestra.run.smithi178.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-15T01:31:16.160 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:31:16.169 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:31:16.169 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:31:16.196 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:31:16.196 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:31:16.418 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:31:16.486 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:16.502 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:16.503 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:31:16.503 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T01:31:16.503 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:16.695 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:42 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:16.829 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:16.830 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:16.830 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repo Size 2024-09-15T01:31:16.830 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:16.831 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:31:16.831 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-15T01:31:16.831 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:16.831 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:31:16.831 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:16.831 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-09-15T01:31:16.831 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:16.831 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 2.9 M 2024-09-15T01:31:16.831 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 12 M 2024-09-15T01:31:16.831 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:31:17.168 INFO:teuthology.orchestra.run.smithi204.stdout:rbd-mirror-17.2.7-1643.gd75f2382.el9.x86_64.rpm 8.7 MB/s | 2.9 MB 00:00 2024-09-15T01:31:17.169 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:17.169 INFO:teuthology.orchestra.run.smithi204.stdout:Total 8.7 MB/s | 2.9 MB 00:00 2024-09-15T01:31:17.170 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:31:17.179 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:31:17.179 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:31:17.257 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:31:17.258 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:31:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-15T01:31:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:17.550 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:31:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-15T01:31:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:17.551 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:17.668 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:31:17.719 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:17.744 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:17.744 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:31:17.744 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T01:31:17.744 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T01:31:17.744 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T01:31:17.744 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:17.869 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librados-devel 2024-09-15T01:31:18.445 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:41 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:18.581 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:18.582 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:18.582 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-09-15T01:31:18.582 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:18.582 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:31:18.582 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 126 k 2024-09-15T01:31:18.582 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:18.582 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:31:18.582 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:18.582 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-15T01:31:18.582 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:18.582 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 126 k 2024-09-15T01:31:18.583 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 448 k 2024-09-15T01:31:18.583 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:31:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:18.789 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:31:18.789 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:18.790 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:18.790 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:18.798 INFO:teuthology.orchestra.run.smithi178.stdout:librados-devel-17.2.7-1643.gd75f2382.el9.x86_64 587 kB/s | 126 kB 00:00 2024-09-15T01:31:18.798 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:18.799 INFO:teuthology.orchestra.run.smithi178.stdout:Total 583 kB/s | 126 kB 00:00 2024-09-15T01:31:18.799 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:31:18.804 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:31:18.804 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:31:18.869 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:31:18.869 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:31:19.024 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:31:19.047 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install rbd-nbd 2024-09-15T01:31:19.128 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:19.578 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:19.628 INFO:teuthology.orchestra.run.smithi204.stdout:Last metadata expiration check: 0:01:45 ago on Sun 15 Sep 2024 01:29:34 AM UTC. 2024-09-15T01:31:19.765 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-15T01:31:19.766 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:19.766 INFO:teuthology.orchestra.run.smithi204.stdout: Package Architecture Version Repository Size 2024-09-15T01:31:19.766 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:19.766 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-15T01:31:19.766 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 165 k 2024-09-15T01:31:19.767 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:19.767 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-15T01:31:19.767 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-15T01:31:19.767 INFO:teuthology.orchestra.run.smithi204.stdout:Install 1 Package 2024-09-15T01:31:19.767 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:19.767 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 165 k 2024-09-15T01:31:19.767 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 477 k 2024-09-15T01:31:19.767 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-15T01:31:19.833 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:19.833 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:19.833 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:31:19.833 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:19.833 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:19.833 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:19.998 INFO:teuthology.orchestra.run.smithi204.stdout:rbd-nbd-17.2.7-1643.gd75f2382.el9.x86_64.rpm 717 kB/s | 165 kB 00:00 2024-09-15T01:31:19.998 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:19.999 INFO:teuthology.orchestra.run.smithi204.stdout:Total 712 kB/s | 165 kB 00:00 2024-09-15T01:31:19.999 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-15T01:31:20.007 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-15T01:31:20.007 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-15T01:31:20.070 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-15T01:31:20.070 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-15T01:31:20.098 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install libcephfs2 2024-09-15T01:31:20.225 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-15T01:31:20.328 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:20.685 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:43 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:20.740 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:20.777 INFO:teuthology.orchestra.run.smithi178.stdout:Package libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:20.822 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:20.823 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-15T01:31:20.823 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:20.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install libcephfs-devel 2024-09-15T01:31:21.007 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:21.007 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:21.007 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-15T01:31:21.007 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:21.007 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:21.007 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-15T01:31:21.323 DEBUG:teuthology.parallel:result is None 2024-09-15T01:31:21.458 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:44 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:21.591 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:21.591 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:21.592 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-09-15T01:31:21.592 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:21.592 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:31:21.592 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-15T01:31:21.592 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:21.592 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:31:21.592 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:21.592 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-15T01:31:21.592 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:21.592 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 25 k 2024-09-15T01:31:21.592 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 110 k 2024-09-15T01:31:21.592 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:31:21.743 INFO:teuthology.orchestra.run.smithi178.stdout:libcephfs-devel-17.2.7-1643.gd75f2382.el9.x86_6 166 kB/s | 25 kB 00:00 2024-09-15T01:31:21.743 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:21.744 INFO:teuthology.orchestra.run.smithi178.stdout:Total 164 kB/s | 25 kB 00:00 2024-09-15T01:31:21.744 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:31:21.747 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:31:21.747 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:31:21.749 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:31:21.749 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:31:21.834 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:31:21.948 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:22.326 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:22.581 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:22.581 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:22.581 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:31:22.581 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:22.581 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:22.581 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:22.784 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librados2 2024-09-15T01:31:23.352 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:46 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:23.443 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:23.486 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:23.487 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-15T01:31:23.487 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:23.554 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librbd1 2024-09-15T01:31:24.121 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:47 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:24.210 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:24.253 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:24.254 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-15T01:31:24.254 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:24.319 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rados 2024-09-15T01:31:24.881 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:47 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:24.970 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:25.012 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:25.013 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-15T01:31:25.013 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:25.082 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rgw 2024-09-15T01:31:25.642 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:48 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:25.731 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:25.773 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:25.774 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-15T01:31:25.774 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:25.844 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-cephfs 2024-09-15T01:31:26.411 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:49 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:26.501 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:26.544 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:26.545 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-15T01:31:26.545 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:26.620 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rbd 2024-09-15T01:31:27.184 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:50 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:27.275 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-15T01:31:27.317 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:27.318 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-15T01:31:27.318 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:27.386 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-fuse 2024-09-15T01:31:27.953 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:50 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:28.087 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:28.087 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:28.087 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-15T01:31:28.087 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:28.087 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:31:28.087 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 82 k 2024-09-15T01:31:28.088 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:28.088 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:31:28.088 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:28.088 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-15T01:31:28.088 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:28.088 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 82 k 2024-09-15T01:31:28.088 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 222 k 2024-09-15T01:31:28.089 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:31:28.288 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-fuse-17.2.7-1643.gd75f2382.el9.x86_64.rpm 411 kB/s | 82 kB 00:00 2024-09-15T01:31:28.288 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:28.288 INFO:teuthology.orchestra.run.smithi178.stdout:Total 409 kB/s | 82 kB 00:00 2024-09-15T01:31:28.288 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:31:28.295 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:31:28.295 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:31:28.357 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:31:28.357 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:31:28.502 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:31:28.609 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:29.010 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:29.289 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:29.289 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:31:29.289 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:29.289 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:29.289 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:29.563 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-mirror 2024-09-15T01:31:30.163 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:53 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:30.305 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:30.305 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:30.305 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-09-15T01:31:30.306 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:30.306 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:31:30.306 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-15T01:31:30.306 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:30.306 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:31:30.306 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:30.306 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-15T01:31:30.306 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:30.307 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 2.9 M 2024-09-15T01:31:30.307 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 12 M 2024-09-15T01:31:30.307 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:31:30.633 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-mirror-17.2.7-1643.gd75f2382.el9.x86_64.rpm 9.0 MB/s | 2.9 MB 00:00 2024-09-15T01:31:30.633 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:30.634 INFO:teuthology.orchestra.run.smithi178.stdout:Total 9.0 MB/s | 2.9 MB 00:00 2024-09-15T01:31:30.634 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:31:30.643 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:31:30.643 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:31:30.723 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:31:30.723 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:31:31.143 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:31:31.292 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:31.318 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:31.318 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T01:31:31.318 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T01:31:31.319 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T01:31:31.319 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T01:31:31.319 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:32.323 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:32.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:32.323 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:31:32.323 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:32.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:32.323 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:32.594 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-nbd 2024-09-15T01:31:33.177 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:56 ago on Sun 15 Sep 2024 01:29:37 AM UTC. 2024-09-15T01:31:33.314 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-15T01:31:33.315 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:33.315 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-15T01:31:33.315 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:33.315 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-15T01:31:33.315 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 165 k 2024-09-15T01:31:33.315 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:33.316 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-15T01:31:33.316 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-15T01:31:33.316 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-15T01:31:33.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:33.316 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 165 k 2024-09-15T01:31:33.316 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 477 k 2024-09-15T01:31:33.316 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-15T01:31:33.528 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-nbd-17.2.7-1643.gd75f2382.el9.x86_64.rpm 782 kB/s | 165 kB 00:00 2024-09-15T01:31:33.528 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-15T01:31:33.529 INFO:teuthology.orchestra.run.smithi178.stdout:Total 776 kB/s | 165 kB 00:00 2024-09-15T01:31:33.529 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-15T01:31:33.536 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-15T01:31:33.536 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-15T01:31:33.600 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-15T01:31:33.600 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-15T01:31:33.757 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-15T01:31:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:34.384 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:34.681 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-15T01:31:34.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:34.681 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-15T01:31:34.681 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-15T01:31:34.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:34.681 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-15T01:31:34.923 DEBUG:teuthology.parallel:result is None 2024-09-15T01:31:34.924 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:31:35.075 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T01:31:35.096 INFO:teuthology.orchestra.run.smithi178.stdout:17.2.7-1643.gd75f2382.el9 2024-09-15T01:31:35.097 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643.gd75f2382.el9 2024-09-15T01:31:35.097 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643.gd75f2382 is installed. 2024-09-15T01:31:35.099 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:31:35.239 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T01:31:35.265 INFO:teuthology.orchestra.run.smithi204.stdout:17.2.7-1643.gd75f2382.el9 2024-09-15T01:31:35.265 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643.gd75f2382.el9 2024-09-15T01:31:35.265 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643.gd75f2382 is installed. 2024-09-15T01:31:35.267 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T01:31:35.268 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:35.268 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T01:31:35.293 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:35.293 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T01:31:35.335 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T01:31:35.335 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:35.335 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T01:31:35.359 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T01:31:35.424 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:35.424 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T01:31:35.454 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T01:31:35.523 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T01:31:35.523 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:35.523 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T01:31:35.547 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T01:31:35.625 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:35.625 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T01:31:35.653 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T01:31:35.721 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T01:31:35.721 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:35.721 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T01:31:35.745 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T01:31:35.810 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:35.810 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T01:31:35.838 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T01:31:35.906 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T01:31:35.979 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T01:31:35.979 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /var/log/ceph 2024-09-15T01:31:35.982 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /var/log/ceph 2024-09-15T01:31:36.010 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T01:31:36.010 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T01:31:36.047 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T01:31:36.078 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T01:31:36.078 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T01:31:36.112 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T01:31:36.154 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T01:31:36.154 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd sloppy crc': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(TOO_FEW_PGS\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(AUTH_BAD_CAPS\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T01:31:36.154 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043', 'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'description': 'rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_latest} thrashers/force-sync-many workloads/rados_mon_workunits}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7905043', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd sloppy crc': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(TOO_FEW_PGS\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(AUTH_BAD_CAPS\\)'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'osd objectstore': 'filestore', 'osd sloppy crc': True}}, 'filestore': True, 'fs': 'xfs'}, 'install': {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'], ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'seed': 2126, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'targets': {'smithi178.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLIFDTnvESD8w4Wq/enD710safWO8CiXAT0Mj0IWMGijE23DYvl0/awJBcwZK7+qChwlj+rkcvjX16OFOKvG2+k=', 'smithi204.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF2VLgh93C95lAQ6Mb6iTXJLdPclMarnWJpOdg8gGGaC5ex+vieo/kHH1TXKwXJcNABCtcfsJGUbsXUMfMzqHu8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'revive_delay': 90, 'store_thrash': True, 'thrash_delay': 1, 'thrash_many': True}}, {'workunit': {'clients': {'client.0': ['mon/pool_ops.sh', 'mon/crush_ops.sh', 'mon/osd.sh', 'mon/caps.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_00:54:39', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T01:31:36.154 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T01:31:36.177 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T01:31:36.201 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T01:31:36.232 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T01:31:36.267 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:36.267 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T01:31:36.310 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T01:31:36.310 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-09-15T01:31:36.365 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T01:31:36.365 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:36.365 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-15T01:31:36.365 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:36.365 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:36.365 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-15 01:31:31.602600875 +0000 2024-09-15T01:31:36.365 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-15 01:28:56.957406919 +0000 2024-09-15T01:31:36.366 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-15 01:28:56.957406919 +0000 2024-09-15T01:31:36.366 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-15 01:28:56.957406919 +0000 2024-09-15T01:31:36.366 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T01:31:36.429 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-15T01:31:36.429 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-15T01:31:36.429 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 7.7545e-05 s, 6.6 MB/s 2024-09-15T01:31:36.430 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T01:31:36.486 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-09-15T01:31:36.542 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T01:31:36.542 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:36.542 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-15T01:31:36.542 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:36.542 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:36.542 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-15 01:31:31.603600855 +0000 2024-09-15T01:31:36.542 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-15 01:28:57.421397892 +0000 2024-09-15T01:31:36.543 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-15 01:28:57.421397892 +0000 2024-09-15T01:31:36.543 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-15 01:28:57.421397892 +0000 2024-09-15T01:31:36.543 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T01:31:36.605 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-15T01:31:36.605 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-15T01:31:36.605 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000100984 s, 5.1 MB/s 2024-09-15T01:31:36.606 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T01:31:36.662 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-09-15T01:31:36.718 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T01:31:36.718 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:36.718 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-15T01:31:36.718 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:36.718 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:36.718 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-15 01:31:31.603600855 +0000 2024-09-15T01:31:36.718 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-15 01:28:57.733391822 +0000 2024-09-15T01:31:36.718 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-15 01:28:57.733391822 +0000 2024-09-15T01:31:36.718 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-15 01:28:57.733391822 +0000 2024-09-15T01:31:36.719 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T01:31:36.780 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-15T01:31:36.780 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-15T01:31:36.780 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.00010083 s, 5.1 MB/s 2024-09-15T01:31:36.781 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T01:31:36.837 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-09-15T01:31:36.893 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T01:31:36.893 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:36.893 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-15T01:31:36.893 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:36.893 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:36.893 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-15 01:31:31.604600835 +0000 2024-09-15T01:31:36.893 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-15 01:28:58.058385500 +0000 2024-09-15T01:31:36.893 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-15 01:28:58.058385500 +0000 2024-09-15T01:31:36.893 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-15 01:28:58.058385500 +0000 2024-09-15T01:31:36.894 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T01:31:36.955 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-15T01:31:36.956 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-15T01:31:36.956 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000122547 s, 4.2 MB/s 2024-09-15T01:31:36.957 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T01:31:37.013 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T01:31:37.014 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:37.014 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T01:31:37.033 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T01:31:37.034 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-09-15T01:31:37.091 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T01:31:37.092 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:37.092 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-15T01:31:37.092 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:37.092 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:37.092 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-15 01:31:17.992994512 +0000 2024-09-15T01:31:37.092 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-15 01:28:56.968760845 +0000 2024-09-15T01:31:37.092 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-15 01:28:56.968760845 +0000 2024-09-15T01:31:37.092 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-15 01:28:56.968760845 +0000 2024-09-15T01:31:37.092 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T01:31:37.160 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-15T01:31:37.160 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-15T01:31:37.160 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000220924 s, 2.3 MB/s 2024-09-15T01:31:37.161 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T01:31:37.220 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-09-15T01:31:37.277 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T01:31:37.278 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:37.278 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-15T01:31:37.278 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:37.278 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:37.278 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-15 01:31:17.992994512 +0000 2024-09-15T01:31:37.278 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-15 01:28:57.304756438 +0000 2024-09-15T01:31:37.278 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-15 01:28:57.304756438 +0000 2024-09-15T01:31:37.278 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-15 01:28:57.304756438 +0000 2024-09-15T01:31:37.278 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T01:31:37.345 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-15T01:31:37.345 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-15T01:31:37.345 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000211377 s, 2.4 MB/s 2024-09-15T01:31:37.346 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T01:31:37.406 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-09-15T01:31:37.464 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T01:31:37.464 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:37.464 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-15T01:31:37.464 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:37.464 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:37.464 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-15 01:31:17.993994499 +0000 2024-09-15T01:31:37.464 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-15 01:28:57.629752175 +0000 2024-09-15T01:31:37.464 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-15 01:28:57.629752175 +0000 2024-09-15T01:31:37.464 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-15 01:28:57.629752175 +0000 2024-09-15T01:31:37.465 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T01:31:37.530 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-15T01:31:37.530 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-15T01:31:37.530 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000197404 s, 2.6 MB/s 2024-09-15T01:31:37.531 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T01:31:37.590 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-09-15T01:31:37.647 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T01:31:37.648 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T01:31:37.648 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-15T01:31:37.648 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T01:31:37.648 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T01:31:37.648 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-15 01:31:17.993994499 +0000 2024-09-15T01:31:37.648 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-15 01:28:57.953747926 +0000 2024-09-15T01:31:37.648 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-15 01:28:57.953747926 +0000 2024-09-15T01:31:37.648 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-15 01:28:57.953747926 +0000 2024-09-15T01:31:37.648 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T01:31:37.714 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-15T01:31:37.714 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-15T01:31:37.714 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000289526 s, 1.8 MB/s 2024-09-15T01:31:37.715 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T01:31:37.772 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T01:31:37.773 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-15T01:31:37.773 INFO:tasks.ceph:Generating config... 2024-09-15T01:31:37.774 INFO:tasks.ceph:[client] debug monc = 20 2024-09-15T01:31:37.774 INFO:tasks.ceph:[client] debug ms = 1 2024-09-15T01:31:37.774 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-15T01:31:37.774 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-15T01:31:37.774 INFO:tasks.ceph:[global] ms bind msgr1 = False 2024-09-15T01:31:37.774 INFO:tasks.ceph:[global] ms bind msgr2 = True 2024-09-15T01:31:37.774 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-15T01:31:37.774 INFO:tasks.ceph:[global] ms type = async 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-15T01:31:37.775 INFO:tasks.ceph:[mon] paxos service trim min = 5 2024-09-15T01:31:37.775 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-15T01:31:37.776 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T01:31:37.776 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T01:31:37.776 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-15T01:31:37.776 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-15T01:31:37.776 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-15T01:31:37.776 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-15T01:31:37.776 INFO:tasks.ceph:[osd] osd objectstore = filestore 2024-09-15T01:31:37.776 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-15T01:31:37.776 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-15T01:31:37.776 INFO:tasks.ceph:[osd] osd sloppy crc = True 2024-09-15T01:31:37.776 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T01:31:37.777 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T01:31:37.836 INFO:teuthology.orchestra.run.smithi178.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T01:31:37.839 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T01:31:37.880 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T01:31:37.946 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.178'), ('mon.b', '[v2:172.21.15.178:3301,v1:172.21.15.178:6790]'), ('mon.c', '[v2:172.21.15.178:3302,v1:172.21.15.178:6791]'), ('mon.d', '[v2:172.21.15.178:3303,v1:172.21.15.178:6792]'), ('mon.e', '[v2:172.21.15.178:3304,v1:172.21.15.178:6793]'), ('mon.f', '172.21.15.204'), ('mon.g', '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]'), ('mon.h', '[v2:172.21.15.204:3302,v1:172.21.15.204:6791]'), ('mon.i', '[v2:172.21.15.204:3303,v1:172.21.15.204:6792]')] 2024-09-15T01:31:37.946 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790],[v2:172.21.15.178:3302,v1:172.21.15.178:6791],[v2:172.21.15.178:3303,v1:172.21.15.178:6792],[v2:172.21.15.178:3304,v1:172.21.15.178:6793],172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790],[v2:172.21.15.204:3302,v1:172.21.15.204:6791],[v2:172.21.15.204:3303,v1:172.21.15.204:6792]', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd sloppy crc': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mon.d': {}, 'mon.e': {}, 'mon.f': {}, 'mon.g': {}, 'mon.h': {}, 'mon.i': {}} 2024-09-15T01:31:37.947 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:37.947 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T01:31:38.001 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.178 --addv b '[v2:172.21.15.178:3301,v1:172.21.15.178:6790]' --addv c '[v2:172.21.15.178:3302,v1:172.21.15.178:6791]' --addv d '[v2:172.21.15.178:3303,v1:172.21.15.178:6792]' --addv e '[v2:172.21.15.178:3304,v1:172.21.15.178:6793]' --add f 172.21.15.204 --addv g '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]' --addv h '[v2:172.21.15.204:3302,v1:172.21.15.204:6791]' --addv i '[v2:172.21.15.204:3303,v1:172.21.15.204:6792]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:38.101 INFO:teuthology.orchestra.run.smithi178.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:38.101 INFO:teuthology.orchestra.run.smithi178.stdout:monmaptool: generated fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:setting min_mon_release = octopus 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:epoch 0 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:min_mon_release 15 (octopus) 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:election_strategy: 3 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:3: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:4: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:38.102 INFO:teuthology.orchestra.run.smithi178.stdout:5: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:38.103 INFO:teuthology.orchestra.run.smithi178.stdout:6: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:38.103 INFO:teuthology.orchestra.run.smithi178.stdout:7: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:38.103 INFO:teuthology.orchestra.run.smithi178.stdout:8: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:38.103 INFO:teuthology.orchestra.run.smithi178.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (9 monitors) 2024-09-15T01:31:38.104 DEBUG:teuthology.orchestra.run.smithi178:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T01:31:38.119 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6ff069e2-2827-44d3-9460-72f9fc8c094b... 2024-09-15T01:31:38.120 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T01:31:38.161 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T01:31:38.201 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-09-15T01:31:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-09-15T01:31:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T01:31:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-09-15T01:31:38.201 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-09-15T01:31:38.202 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T01:31:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T01:31:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-09-15T01:31:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-09-15T01:31:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-09-15T01:31:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-09-15T01:31:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-09-15T01:31:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-09-15T01:31:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-09-15T01:31:38.203 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-09-15T01:31:38.204 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790],[v2:172.21.15.178:3302,v1:172.21.15.178:6791],[v2:172.21.15.178:3303,v1:172.21.15.178:6792],[v2:172.21.15.178:3304,v1:172.21.15.178:6793],172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790],[v2:172.21.15.204:3302,v1:172.21.15.204:6791],[v2:172.21.15.204:3303,v1:172.21.15.204:6792]" 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: mon client directed command retry = 5 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: mon election default strategy = 3 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: ms bind msgr1 = False 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: ms bind msgr2 = True 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: ms inject socket failures = 5000 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: ms type = async 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.205 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.206 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore zero block detection = True 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-09-15T01:31:38.207 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify cached snaps = True 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify missing on start = True 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock override recovery settings = True 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = filestore 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: osd op queue = debug_random 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: osd op queue cut off = debug_random 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: osd sloppy crc = True 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-09-15T01:31:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-09-15T01:31:38.209 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-09-15T01:31:38.210 INFO:teuthology.orchestra.run.smithi178.stdout: mon min osdmap epochs = 25 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: mon osdmap full prune interval = 2 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: mon osdmap full prune min = 15 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: mon osdmap full prune txsize = 2 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: mon scrub inject crc mismatch = 0.01 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: mon scrub inject missing keys = 0.05 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: mon scrub interval = 300 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: paxos service trim min = 5 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-09-15T01:31:38.211 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T01:31:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T01:31:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: debug monc = 20 2024-09-15T01:31:38.212 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-09-15T01:31:38.212 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-09-15T01:31:38.212 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-09-15T01:31:38.212 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-09-15T01:31:38.212 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.d] 2024-09-15T01:31:38.212 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.e] 2024-09-15T01:31:38.212 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.f] 2024-09-15T01:31:38.212 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.g] 2024-09-15T01:31:38.212 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.h] 2024-09-15T01:31:38.212 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.i] 2024-09-15T01:31:38.213 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2024-09-15T01:31:38.213 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2024-09-15T01:31:38.213 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T01:31:38.213 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2024-09-15T01:31:38.213 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.213 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2024-09-15T01:31:38.213 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.213 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T01:31:38.214 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: mon_allow_pool_size_one = true 2024-09-15T01:31:38.215 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.216 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T01:31:38.216 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.216 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2024-09-15T01:31:38.216 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.216 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2024-09-15T01:31:38.216 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.216 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2024-09-15T01:31:38.216 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2024-09-15T01:31:38.216 INFO:teuthology.orchestra.run.smithi204.stdout: mon health detail to clog = false 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = "172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790],[v2:172.21.15.178:3302,v1:172.21.15.178:6791],[v2:172.21.15.178:3303,v1:172.21.15.178:6792],[v2:172.21.15.178:3304,v1:172.21.15.178:6793],172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790],[v2:172.21.15.204:3302,v1:172.21.15.204:6791],[v2:172.21.15.204:3303,v1:172.21.15.204:6792]" 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: mon client directed command retry = 5 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: mon election default strategy = 3 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: ms bind msgr1 = False 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: ms bind msgr2 = True 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: ms inject socket failures = 5000 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: ms type = async 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2024-09-15T01:31:38.217 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2024-09-15T01:31:38.218 INFO:teuthology.orchestra.run.smithi204.stdout: osd mclock profile = high_recovery_ops 2024-09-15T01:31:38.218 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.218 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2024-09-15T01:31:38.218 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2024-09-15T01:31:38.218 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.218 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2024-09-15T01:31:38.218 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2024-09-15T01:31:38.218 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2024-09-15T01:31:38.218 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.218 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2024-09-15T01:31:38.218 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2024-09-15T01:31:38.218 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.219 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2024-09-15T01:31:38.219 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore zero block detection = True 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 20 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify cached snaps = True 2024-09-15T01:31:38.220 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify missing on start = True 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout: osd mclock override recovery settings = True 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout: osd objectstore = filestore 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout: osd op queue = debug_random 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout: osd op queue cut off = debug_random 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout: osd sloppy crc = True 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:31:38.221 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:31:38.222 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2024-09-15T01:31:38.222 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.222 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2024-09-15T01:31:38.222 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-09-15T01:31:38.222 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-09-15T01:31:38.222 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2024-09-15T01:31:38.222 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-09-15T01:31:38.222 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2024-09-15T01:31:38.222 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.223 INFO:teuthology.orchestra.run.smithi204.stdout: # 1m isn't quite enough 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout: mon_down_mkfs_grace = 2m 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_filestore_osds = false 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout: mon min osdmap epochs = 25 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout: mon osdmap full prune interval = 2 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout: mon osdmap full prune min = 15 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout: mon osdmap full prune txsize = 2 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout: mon scrub inject crc mismatch = 0.01 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout: mon scrub inject missing keys = 0.05 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout: mon scrub interval = 300 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout: paxos service trim min = 5 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-15T01:31:38.224 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = true 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout: debug monc = 20 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.b] 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.c] 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.d] 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.e] 2024-09-15T01:31:38.225 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.f] 2024-09-15T01:31:38.226 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.g] 2024-09-15T01:31:38.226 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.h] 2024-09-15T01:31:38.226 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.i] 2024-09-15T01:31:38.227 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T01:31:38.227 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T01:31:38.316 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T01:31:38.316 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:38.317 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T01:31:38.331 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:38.332 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T01:31:38.387 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi178.front.sepia.ceph.com 2024-09-15T01:31:38.387 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:38.387 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T01:31:38.387 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T01:31:38.460 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:38.461 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:38.516 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi204.front.sepia.ceph.com 2024-09-15T01:31:38.516 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:38.516 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T01:31:38.516 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T01:31:38.556 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:38.556 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:38.613 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T01:31:38.614 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T01:31:38.614 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T01:31:38.712 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T01:31:38.715 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T01:31:38.715 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T01:31:38.715 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T01:31:38.753 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T01:31:38.766 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T01:31:38.766 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-15T01:31:38.766 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T01:31:38.792 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T01:31:38.792 INFO:tasks.ceph:role: osd.0 2024-09-15T01:31:38.792 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com 2024-09-15T01:31:38.792 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T01:31:38.856 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:38.856 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:38.857 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:38.857 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:38.857 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:38.888 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-09-15T01:31:38.890 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-09-15T01:31:38.890 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T01:31:38.959 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T01:31:39.029 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T01:31:39.095 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T01:31:39.095 INFO:tasks.ceph:role: osd.1 2024-09-15T01:31:39.095 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com 2024-09-15T01:31:39.095 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T01:31:39.161 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:39.161 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:39.161 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:39.161 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:39.161 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:39.162 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:39.162 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:39.193 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-09-15T01:31:39.195 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-09-15T01:31:39.195 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T01:31:39.264 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T01:31:39.333 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T01:31:39.398 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T01:31:39.398 INFO:tasks.ceph:role: osd.2 2024-09-15T01:31:39.398 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com 2024-09-15T01:31:39.398 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T01:31:39.463 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:39.463 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:39.463 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:39.463 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:39.463 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:39.463 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:39.463 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:39.463 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:39.463 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:39.463 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:39.496 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-09-15T01:31:39.498 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-09-15T01:31:39.498 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T01:31:39.570 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T01:31:39.639 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:39.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.745+0000 7f4e12239440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T01:31:39.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.745+0000 7f4e12239440 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T01:31:39.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.749+0000 7f4e12239440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:39.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.750+0000 7f4e12239440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:39.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.806+0000 7f4e12239440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:39.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.806+0000 7f4e12239440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:39.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.806+0000 7f4e12239440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:39.814 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T01:31:39.839 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:39.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.917+0000 7fbae0326440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T01:31:39.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.917+0000 7fbae0326440 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T01:31:39.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.921+0000 7fbae0326440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:39.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.921+0000 7fbae0326440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:39.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.978+0000 7fbae0326440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:39.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.978+0000 7fbae0326440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:39.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:39.978+0000 7fbae0326440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:39.987 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T01:31:40.051 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:40.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:40.130+0000 7f3c7f326440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T01:31:40.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:40.130+0000 7f3c7f326440 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T01:31:40.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:40.134+0000 7f3c7f326440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:40.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:40.134+0000 7f3c7f326440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:40.192+0000 7f3c7f326440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:40.192+0000 7f3c7f326440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:40.192+0000 7f3c7f326440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:40.200 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T01:31:40.265 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T01:31:40.295 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T01:31:40.295 INFO:tasks.ceph:role: osd.3 2024-09-15T01:31:40.296 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com 2024-09-15T01:31:40.296 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T01:31:40.374 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:40.374 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:40.374 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:40.374 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:40.374 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:40.374 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:40.374 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:40.374 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:40.375 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:40.375 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:40.881 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-09-15T01:31:40.884 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-09-15T01:31:40.884 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-15T01:31:41.029 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T01:31:41.067 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-15T01:31:41.137 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T01:31:41.137 INFO:tasks.ceph:role: osd.4 2024-09-15T01:31:41.137 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com 2024-09-15T01:31:41.137 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T01:31:41.211 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:41.211 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:41.211 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:41.211 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:41.211 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:41.211 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:41.211 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:41.211 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:41.211 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:41.211 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:41.656 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-09-15T01:31:41.658 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-09-15T01:31:41.658 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-15T01:31:41.744 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-15T01:31:41.773 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-15T01:31:41.838 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T01:31:41.838 INFO:tasks.ceph:role: osd.5 2024-09-15T01:31:41.838 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com 2024-09-15T01:31:41.839 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T01:31:41.910 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T01:31:41.910 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T01:31:41.910 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T01:31:41.911 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T01:31:41.911 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T01:31:41.911 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-09-15T01:31:41.911 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T01:31:41.911 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T01:31:41.911 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T01:31:41.911 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T01:31:42.401 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-09-15T01:31:42.403 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-09-15T01:31:42.404 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-15T01:31:42.533 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-15T01:31:42.563 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:42.667 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:42.665+0000 7f2d06c8d440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T01:31:42.672 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:42.670+0000 7f2d06c8d440 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T01:31:42.690 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:42.687+0000 7f2d06c8d440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:42.690 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:42.687+0000 7f2d06c8d440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:42.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:42.938+0000 7f2d06c8d440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:42.943 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:42.941+0000 7f2d06c8d440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:42.943 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:42.941+0000 7f2d06c8d440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:42.960 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T01:31:42.985 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:43.066 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.063+0000 7f68cab24440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-15T01:31:43.071 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.069+0000 7f68cab24440 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-15T01:31:43.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.088+0000 7f68cab24440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:43.093 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.091+0000 7f68cab24440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:43.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.323+0000 7f68cab24440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:43.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.323+0000 7f68cab24440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:43.326 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.323+0000 7f68cab24440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:43.350 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-15T01:31:43.415 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:43.496 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.494+0000 7f2ff8e8d440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-15T01:31:43.497 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.494+0000 7f2ff8e8d440 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-15T01:31:43.501 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.499+0000 7f2ff8e8d440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:43.501 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.499+0000 7f2ff8e8d440 -1 journal FileJournal::_open_file : zeroing journal 2024-09-15T01:31:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.678+0000 7f2ff8e8d440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.679+0000 7f2ff8e8d440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:43.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:43.679+0000 7f2ff8e8d440 -1 journal do_read_entry(4096): bad header magic 2024-09-15T01:31:43.688 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-15T01:31:43.753 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T01:31:43.753 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:43.753 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T01:31:43.783 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:43.783 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T01:31:43.852 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:43.852 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T01:31:43.919 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:43.919 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T01:31:43.945 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:43.945 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T01:31:44.010 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:44.010 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-15T01:31:44.074 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:44.074 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-15T01:31:44.138 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:44.138 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T01:31:44.194 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T01:31:44.194 DEBUG:teuthology.orchestra.run.smithi178:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T01:31:44.197 DEBUG:teuthology.orchestra.run.smithi204:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T01:31:44.237 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.0] 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD7OOZmOLGALBAALue+yuJLI792aioqL78Z1w== 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.1] 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD7OOZm9eHANhAAxGTIA/IaWub15FLYQHIMYA== 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.2] 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD8OOZm2GvVBxAAxmXyqxedEqUiT8nVG78iGA== 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.x] 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD6OOZmgW5yKhAAatdN5waQ+23VHw5K6LqWIQ== 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.3] 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD+OOZmx1nAJxAA2XyGck7SfuQY13IofnIJjw== 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.4] 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD/OOZmHA/mAxAAOoA3+3lVXjxViBjz7vRBnA== 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.5] 2024-09-15T01:31:44.238 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD/OOZmLRCQHRAAuKn9LE+aNosnPgnDZMXVZg== 2024-09-15T01:31:44.239 INFO:teuthology.orchestra.run.smithi178.stdout:[client.0] 2024-09-15T01:31:44.239 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD6OOZmazvcLBAA6618BRCyDxoN0w+jReLvsg== 2024-09-15T01:31:44.257 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.0] 2024-09-15T01:31:44.257 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD7OOZmOLGALBAALue+yuJLI792aioqL78Z1w== 2024-09-15T01:31:44.257 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.1] 2024-09-15T01:31:44.257 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD7OOZm9eHANhAAxGTIA/IaWub15FLYQHIMYA== 2024-09-15T01:31:44.257 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.2] 2024-09-15T01:31:44.258 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD8OOZm2GvVBxAAxmXyqxedEqUiT8nVG78iGA== 2024-09-15T01:31:44.258 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.x] 2024-09-15T01:31:44.258 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD6OOZmgW5yKhAAatdN5waQ+23VHw5K6LqWIQ== 2024-09-15T01:31:44.258 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.3] 2024-09-15T01:31:44.258 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD+OOZmx1nAJxAA2XyGck7SfuQY13IofnIJjw== 2024-09-15T01:31:44.258 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.4] 2024-09-15T01:31:44.258 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD/OOZmHA/mAxAAOoA3+3lVXjxViBjz7vRBnA== 2024-09-15T01:31:44.258 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.5] 2024-09-15T01:31:44.258 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD/OOZmLRCQHRAAuKn9LE+aNosnPgnDZMXVZg== 2024-09-15T01:31:44.258 INFO:teuthology.orchestra.run.smithi204.stdout:[client.0] 2024-09-15T01:31:44.258 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD6OOZmazvcLBAA6618BRCyDxoN0w+jReLvsg== 2024-09-15T01:31:44.259 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:44.282 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:44.346 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:44.348 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:44.395 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:44.397 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:44.443 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T01:31:44.446 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T01:31:44.492 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:44.494 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:44.540 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:44.542 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:44.588 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:44.590 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T01:31:44.636 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T01:31:44.638 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T01:31:44.687 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T01:31:44.687 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T01:31:44.711 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:31:44.802 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T01:31:44.826 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T01:31:44.891 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:31:44.981 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T01:31:45.005 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T01:31:45.070 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:31:45.161 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T01:31:45.185 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-d 2024-09-15T01:31:45.250 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i d --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:31:45.340 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-d 2024-09-15T01:31:45.365 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-e 2024-09-15T01:31:45.430 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i e --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:31:45.522 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-e 2024-09-15T01:31:45.546 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-f 2024-09-15T01:31:45.576 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i f --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:31:45.677 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-f 2024-09-15T01:31:45.704 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-g 2024-09-15T01:31:45.770 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i g --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:31:45.862 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-g 2024-09-15T01:31:45.887 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-h 2024-09-15T01:31:45.952 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i h --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:31:46.042 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-h 2024-09-15T01:31:46.066 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-i 2024-09-15T01:31:46.131 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i i --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T01:31:46.222 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-i 2024-09-15T01:31:46.246 DEBUG:teuthology.orchestra.run.smithi178:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:46.248 DEBUG:teuthology.orchestra.run.smithi204:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T01:31:46.301 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T01:31:46.302 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T01:31:46.302 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T01:31:46.306 INFO:tasks.ceph.mon.a:Started 2024-09-15T01:31:46.306 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T01:31:46.306 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T01:31:46.308 INFO:tasks.ceph.mon.b:Started 2024-09-15T01:31:46.308 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T01:31:46.308 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T01:31:46.310 INFO:tasks.ceph.mon.c:Started 2024-09-15T01:31:46.310 INFO:tasks.ceph.mon.d:Restarting daemon 2024-09-15T01:31:46.311 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-09-15T01:31:46.313 INFO:tasks.ceph.mon.d:Started 2024-09-15T01:31:46.313 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-15T01:31:46.313 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-15T01:31:46.316 INFO:tasks.ceph.mon.e:Started 2024-09-15T01:31:46.316 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-15T01:31:46.316 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-15T01:31:46.343 INFO:tasks.ceph.mon.f:Started 2024-09-15T01:31:46.343 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-15T01:31:46.343 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-15T01:31:46.347 INFO:tasks.ceph.mon.g:Started 2024-09-15T01:31:46.347 INFO:tasks.ceph.mon.h:Restarting daemon 2024-09-15T01:31:46.347 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-09-15T01:31:46.349 INFO:tasks.ceph.mon.h:Started 2024-09-15T01:31:46.349 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-15T01:31:46.349 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-15T01:31:46.351 INFO:tasks.ceph.mon.i:Started 2024-09-15T01:31:46.351 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T01:31:46.352 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T01:31:46.352 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T01:31:46.354 INFO:tasks.ceph.mgr.x:Started 2024-09-15T01:31:46.354 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T01:31:46.354 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T01:31:46.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.470+0000 7f9b4b820640 10 monclient: get_monmap_and_config 2024-09-15T01:31:46.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.470+0000 7f9b4b820640 10 monclient: build_initial_monmap 2024-09-15T01:31:46.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.470+0000 7f9b4b820640 10 monclient: monmap: 2024-09-15T01:31:46.472 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:31:46.472 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:46.472 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:46.470991+0000 2024-09-15T01:31:46.472 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:46.470991+0000 2024-09-15T01:31:46.472 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:46.472 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:46.472 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:31:46.473 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:31:46.473 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:46.473 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:46.473 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:46.473 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:46.473 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:46.473 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:46.473 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:46.473 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:31:46.473 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:31:46.473 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:31:46.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.470+0000 7f9b4b820640 1 Processor -- start 2024-09-15T01:31:46.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.470+0000 7f9b4b820640 1 -- start start 2024-09-15T01:31:46.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.470+0000 7f9b4b820640 10 monclient: init 2024-09-15T01:31:46.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b4b820640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:46.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b4b820640 10 monclient: _add_conns ranks=[6,0,1,3,5,2,4,10,8,9,7] 2024-09-15T01:31:46.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b4b820640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:46.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b4b820640 10 monclient(hunting): picked mon.noname-d con 0x7f9b4409e700 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:31:46.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b4b820640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9b4409ead0 addr v1:172.21.15.178:6789/0 2024-09-15T01:31:46.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b4b820640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9b44137db0 addr v1:172.21.15.204:6789/0 2024-09-15T01:31:46.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b4b820640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b4404bf60 con 0x7f9b4409ead0 2024-09-15T01:31:46.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b4b820640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b44069210 con 0x7f9b44137db0 2024-09-15T01:31:46.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b495bc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:46.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b48dbb640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f9b4409ead0 0x7f9b4409eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:48284/0 (socket says 172.21.15.178:48284) 2024-09-15T01:31:46.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1834149094 learned_addr learned my addr 172.21.15.178:0/1834149094 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:31:46.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b4b820640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:46.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b4b820640 1 -- 172.21.15.178:0/1834149094 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4409f5b0 con 0x7f9b4409e700 2024-09-15T01:31:46.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b4b820640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:46.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b4b820640 10 monclient(hunting): authenticate will time out at 711.221578s 2024-09-15T01:31:46.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b495bc640 10 monclient(hunting): get_auth_request con 0x7f9b4409e700 auth_method 0 2024-09-15T01:31:46.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b495bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:46.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b495bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:46.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b495bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:46.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 msgr2=0x7f9b44069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:31:46.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 msgr2=0x7f9b44069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:46.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:46.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.471+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:31:46.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.672+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:46.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.672+0000 7f9b495bc640 10 monclient(hunting): get_auth_request con 0x7f9b4409e700 auth_method 0 2024-09-15T01:31:46.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.672+0000 7f9b495bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:46.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.672+0000 7f9b495bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:46.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.672+0000 7f9b495bc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:31:46.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.672+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 msgr2=0x7f9b44069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:31:46.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.672+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 msgr2=0x7f9b44069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:46.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.672+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:46.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:46.672+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:31:47.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.073+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:47.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.073+0000 7f9b495bc640 10 monclient(hunting): get_auth_request con 0x7f9b4409e700 auth_method 0 2024-09-15T01:31:47.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.073+0000 7f9b495bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:47.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.073+0000 7f9b495bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:47.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.073+0000 7f9b495bc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:31:47.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.073+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 msgr2=0x7f9b44069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:31:47.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.073+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 msgr2=0x7f9b44069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:47.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.073+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:47.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.073+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:31:47.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.874+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:47.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.874+0000 7f9b495bc640 10 monclient(hunting): get_auth_request con 0x7f9b4409e700 auth_method 0 2024-09-15T01:31:47.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.874+0000 7f9b495bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:47.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.874+0000 7f9b495bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:47.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.874+0000 7f9b495bc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:31:47.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.874+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 msgr2=0x7f9b44069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:31:47.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.874+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 msgr2=0x7f9b44069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:47.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.874+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:47.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:47.874+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:31:49.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 10 monclient(hunting): tick 2024-09-15T01:31:49.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 1 monclient(hunting): continuing hunt 2024-09-15T01:31:49.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:31:49.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 msgr2=0x7f9b44069bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:31:49.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9b4409e700 0x7f9b44069bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:49.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 1 -- 172.21.15.178:0/1834149094 >> v1:172.21.15.204:6789/0 conn(0x7f9b44137db0 legacy=0x7f9b4413a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:49.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 1 -- 172.21.15.178:0/1834149094 >> v1:172.21.15.178:6789/0 conn(0x7f9b4409ead0 legacy=0x7f9b4409eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:49.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 10 monclient: _add_conns ranks=[9,8,4,3,2,5,7,10] 2024-09-15T01:31:49.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:49.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 10 monclient(hunting): picked mon.noname-a con 0x7f9b28003800 addr v2:172.21.15.178:3300/0 2024-09-15T01:31:49.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:49.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 10 monclient(hunting): picked mon.noname-e con 0x7f9b28006220 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:31:49.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:49.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 10 monclient(hunting): picked mon.noname-c con 0x7f9b28008df0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 1 -- 172.21.15.178:0/1834149094 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4409f5b0 con 0x7f9b28003800 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 1 -- 172.21.15.178:0/1834149094 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2800b9c0 con 0x7f9b28008df0 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 1 -- 172.21.15.178:0/1834149094 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2800bb80 con 0x7f9b28006220 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b3affd640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b495bc640 10 monclient(hunting): get_auth_request con 0x7f9b28003800 auth_method 0 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b495bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b495bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b495bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:49.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:49.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:49.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b48dbb640 10 monclient(hunting): get_auth_request con 0x7f9b28006220 auth_method 0 2024-09-15T01:31:49.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b48dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:49.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b48dbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:49.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b48dbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:49.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b49dbd640 10 monclient(hunting): get_auth_request con 0x7f9b28008df0 auth_method 0 2024-09-15T01:31:49.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b49dbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:49.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b49dbd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:49.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b49dbd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:49.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 msgr2=0x7f9b28005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:31:49.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 msgr2=0x7f9b28005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:49.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:49.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 msgr2=0x7f9b280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:31:49.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 msgr2=0x7f9b280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:49.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:31:49.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:49.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b49dbd640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 msgr2=0x7f9b2800b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:31:49.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b49dbd640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 msgr2=0x7f9b2800b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:49.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:49.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:31:49.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.471+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:31:49.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:49.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:49.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:49.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b495bc640 10 monclient(hunting): get_auth_request con 0x7f9b28003800 auth_method 0 2024-09-15T01:31:49.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b495bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:49.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b495bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:49.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b495bc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:31:49.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b48dbb640 10 monclient(hunting): get_auth_request con 0x7f9b28006220 auth_method 0 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b48dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b48dbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b48dbb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b49dbd640 10 monclient(hunting): get_auth_request con 0x7f9b28008df0 auth_method 0 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b49dbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b49dbd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b49dbd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 msgr2=0x7f9b28005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b49dbd640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 msgr2=0x7f9b2800b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b49dbd640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 msgr2=0x7f9b2800b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 msgr2=0x7f9b28005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:49.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:49.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:31:49.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 msgr2=0x7f9b280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:31:49.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 msgr2=0x7f9b280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:49.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:31:49.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:49.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:49.672+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:31:50.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:50.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:50.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b49dbd640 10 monclient(hunting): get_auth_request con 0x7f9b28008df0 auth_method 0 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b49dbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b49dbd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b49dbd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b495bc640 10 monclient(hunting): get_auth_request con 0x7f9b28003800 auth_method 0 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b495bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b495bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b495bc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b48dbb640 10 monclient(hunting): get_auth_request con 0x7f9b28006220 auth_method 0 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b48dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b48dbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b48dbb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b49dbd640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 msgr2=0x7f9b2800b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:31:50.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b49dbd640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 msgr2=0x7f9b2800b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:50.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:50.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 msgr2=0x7f9b28005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:31:50.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 msgr2=0x7f9b28005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:50.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:50.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 msgr2=0x7f9b280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:31:50.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:31:50.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 msgr2=0x7f9b280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:50.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:31:50.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:50.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.073+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:31:50.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:50.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:50.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:50.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b495bc640 10 monclient(hunting): get_auth_request con 0x7f9b28003800 auth_method 0 2024-09-15T01:31:50.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b495bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:50.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b495bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:50.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b495bc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:31:50.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b48dbb640 10 monclient(hunting): get_auth_request con 0x7f9b28006220 auth_method 0 2024-09-15T01:31:50.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b48dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:50.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b48dbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:50.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b48dbb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:31:50.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b49dbd640 10 monclient(hunting): get_auth_request con 0x7f9b28008df0 auth_method 0 2024-09-15T01:31:50.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b49dbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:50.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b49dbd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:50.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b49dbd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:31:50.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 msgr2=0x7f9b28005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:31:50.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 msgr2=0x7f9b28005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:50.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 msgr2=0x7f9b280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:31:50.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 msgr2=0x7f9b280086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:50.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:50.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:50.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b49dbd640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 msgr2=0x7f9b2800b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:31:50.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b49dbd640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 msgr2=0x7f9b2800b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:31:50.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:31:50.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:31:50.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:31:50.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:50.874+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:31:52.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 10 monclient(hunting): tick 2024-09-15T01:31:52.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 monclient(hunting): continuing hunt 2024-09-15T01:31:52.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:31:52.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 msgr2=0x7f9b280086a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:31:52.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006220 0x7f9b280086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 msgr2=0x7f9b2800b280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28008df0 0x7f9b2800b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 -- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 msgr2=0x7f9b28005c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.178:3300/0 conn(0x7f9b28003800 0x7f9b28005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b49dbd640 1 -- 172.21.15.178:0/1834149094 reap_dead start 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 10 monclient: _add_conns ranks=[2,10,7,3,5] 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b49dbd640 1 -- 172.21.15.178:0/1834149094 reap_dead start 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b28003c10 0x7f9b28006690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 10 monclient(hunting): picked mon.noname-b con 0x7f9b28003c10 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.204:3300/0 conn(0x7f9b28006bd0 0x7f9b2800b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 10 monclient(hunting): picked mon.noname-f con 0x7f9b28006bd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9b28007020 0x7f9b280074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 10 monclient(hunting): picked mon.noname-i con 0x7f9b28007020 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 -- 172.21.15.178:0/1834149094 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4409f5b0 con 0x7f9b28006bd0 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 -- 172.21.15.178:0/1834149094 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2800b9c0 con 0x7f9b28003c10 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 1 -- 172.21.15.178:0/1834149094 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2800bb80 con 0x7f9b28007020 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b3affd640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b28003c10 0x7f9b28006690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 10 monclient(hunting): get_auth_request con 0x7f9b28003c10 auth_method 0 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b49dbd640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9b28007020 0x7f9b280074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.204:3300/0 conn(0x7f9b28006bd0 0x7f9b2800b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b49dbd640 10 monclient(hunting): get_auth_request con 0x7f9b28007020 auth_method 0 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b49dbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b49dbd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b49dbd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b48dbb640 10 monclient(hunting): get_auth_request con 0x7f9b28006bd0 auth_method 0 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b48dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b48dbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b48dbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b49dbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b49dbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b49dbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b48dbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b48dbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b48dbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 10 monclient(hunting): handle_auth_done global_id 4106 payload 274 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9b28007020 msgr2=0x7f9b280074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9b28007020 0x7f9b280074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 >> v2:172.21.15.204:3300/0 conn(0x7f9b28006bd0 msgr2=0x7f9b2800b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.204:3300/0 conn(0x7f9b28006bd0 0x7f9b2800b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 10 monclient: _finish_hunting 0 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 1 monclient: found mon.noname-b 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.472+0000 7f9b495bc640 1 -- 172.21.15.178:0/1834149094 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b44068f30 con 0x7f9b28003c10 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b28003c10 0x7f9b28006690 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9b40006fd0 tx=0x7f9b40008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f70072fea1005c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b3bfff640 1 -- 172.21.15.178:0/1834149094 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9b4002cc50 con 0x7f9b28003c10 2024-09-15T01:31:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b3bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b3bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b3bfff640 10 monclient: dump: 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b3bfff640 10 monclient: _finish_auth 0 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b3bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b3bfff640 1 -- 172.21.15.178:0/1834149094 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4002ce10 con 0x7f9b28003c10 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b4b820640 5 monclient: authenticate success, global_id 4106 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b4b820640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b3bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b4b820640 10 monclient: get_monmap_and_config success 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b4b820640 4 set_mon_vals no callback set 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b3bfff640 1 -- 172.21.15.178:0/1834149094 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9b400378e0 con 0x7f9b28003c10 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b3bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b3bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.473+0000 7f9b3bfff640 10 monclient: dump: 2024-09-15T01:31:52.483 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:52.484 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.474+0000 7f9b4b820640 10 monclient: shutdown 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.474+0000 7f9b4b820640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.474+0000 7f9b4b820640 1 -- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b28003c10 msgr2=0x7f9b28006690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.474+0000 7f9b4b820640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b28003c10 0x7f9b28006690 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9b40006fd0 tx=0x7f9b40008040 comp rx=0 tx=0).stop 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.474+0000 7f9b4b820640 1 -- 172.21.15.178:0/1834149094 shutdown_connections 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.474+0000 7f9b4b820640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9b28007020 0x7f9b280074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.474+0000 7f9b4b820640 1 --2- 172.21.15.178:0/1834149094 >> v2:172.21.15.204:3300/0 conn(0x7f9b28006bd0 0x7f9b2800b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.474+0000 7f9b4b820640 1 --2- 172.21.15.178:0/1834149094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b28003c10 0x7f9b28006690 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.474+0000 7f9b4b820640 1 -- 172.21.15.178:0/1834149094 >> 172.21.15.178:0/1834149094 conn(0x7f9b440a4390 msgr2=0x7f9b44067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.474+0000 7f9b4b820640 1 -- 172.21.15.178:0/1834149094 shutdown_connections 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.474+0000 7f9b4b820640 1 -- 172.21.15.178:0/1834149094 wait complete. 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.475+0000 7f9b4b820640 10 monclient: build_initial_monmap 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.475+0000 7f9b4b820640 10 monclient: monmap: 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:31:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:31:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.475+0000 7f9b4b820640 1 Processor -- start 2024-09-15T01:31:52.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.475+0000 7f9b4b820640 1 -- start start 2024-09-15T01:31:52.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.475+0000 7f9b4b820640 10 monclient: init 2024-09-15T01:31:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 10 monclient: _add_conns ranks=[3,8,4,7,0,5,1,6,2] 2024-09-15T01:31:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b28003c10 0x7f9b441373a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 10 monclient(hunting): picked mon.noname-g con 0x7f9b28003c10 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:31:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006bd0 0x7f9b4412fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 10 monclient(hunting): picked mon.noname-e con 0x7f9b28006bd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:31:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28007020 0x7f9b44130070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 10 monclient(hunting): picked mon.noname-c con 0x7f9b28007020 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:31:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44133d00 con 0x7f9b28003c10 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44131ef0 con 0x7f9b28007020 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440687f0 con 0x7f9b28006bd0 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b4b820640 10 monclient(hunting): authenticate will time out at 717.226524s 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b49dbd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28007020 0x7f9b44130070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b48dbb640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006bd0 0x7f9b4412fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b48dbb640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006bd0 0x7f9b4412fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60434/0 (socket says 172.21.15.178:60434) 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b495bc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b28003c10 0x7f9b441373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b49dbd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28007020 0x7f9b44130070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:49414/0 (socket says 172.21.15.178:49414) 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1174939372 learned_addr learned my addr 172.21.15.178:0/1174939372 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b48dbb640 10 monclient(hunting): get_auth_request con 0x7f9b28006bd0 auth_method 0 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b48dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b48dbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b48dbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b49dbd640 10 monclient(hunting): get_auth_request con 0x7f9b28007020 auth_method 0 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b49dbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b49dbd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b49dbd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b495bc640 10 monclient(hunting): get_auth_request con 0x7f9b28003c10 auth_method 0 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b495bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b495bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b495bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b48dbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b48dbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b48dbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b49dbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b49dbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b49dbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b495bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b495bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.476+0000 7f9b495bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b48dbb640 10 monclient(hunting): handle_auth_done global_id 4130 payload 824 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1174939372 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28007020 msgr2=0x7f9b44130070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1174939372 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28007020 0x7f9b44130070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1174939372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b28003c10 msgr2=0x7f9b441373a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1174939372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b28003c10 0x7f9b441373a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b48dbb640 10 monclient: _finish_hunting 0 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b48dbb640 1 monclient: found mon.noname-e 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b48dbb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b48dbb640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b48dbb640 1 -- 172.21.15.178:0/1174939372 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4000e040 con 0x7f9b28006bd0 2024-09-15T01:31:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b495bc640 1 --2- 172.21.15.178:0/1174939372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b28003c10 0x7f9b441373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b48dbb640 1 --2- 172.21.15.178:0/1174939372 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006bd0 0x7f9b4412fb30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9b34015730 tx=0x7f9b34009040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 1 -- 172.21.15.178:0/1174939372 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9b34005c70 con 0x7f9b28006bd0 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 10 monclient: dump: 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 10 monclient: _finish_auth 0 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 1 -- 172.21.15.178:0/1174939372 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3401d030 con 0x7f9b28006bd0 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b4b820640 5 monclient: authenticate success, global_id 4130 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b4b820640 10 monclient: _renew_subs 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b4b820640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b4b820640 1 -- 172.21.15.178:0/1174939372 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b44134090 con 0x7f9b28006bd0 2024-09-15T01:31:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 1 -- 172.21.15.178:0/1174939372 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9b34005df0 con 0x7f9b28006bd0 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 10 monclient: dump: 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b4b820640 10 monclient: _renew_subs 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b3b7fe640 4 set_mon_vals no callback set 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b4b820640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b4b820640 1 -- 172.21.15.178:0/1174939372 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b441333d0 con 0x7f9b28006bd0 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 1 -- 172.21.15.178:0/1174939372 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f9b340070d0 con 0x7f9b28006bd0 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.477+0000 7f9b397fa640 1 -- 172.21.15.178:0/1174939372 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9b34001a40 con 0x7f9b28006bd0 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.478+0000 7f9b4b820640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.478+0000 7f9b4b820640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.478+0000 7f9b4b820640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:31:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.478+0000 7f9b4b820640 1 -- 172.21.15.178:0/1174939372 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4405ed70 con 0x7f9b28006bd0 2024-09-15T01:31:52.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.480+0000 7f9b397fa640 1 -- 172.21.15.178:0/1174939372 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f9b34001d10 con 0x7f9b28006bd0 2024-09-15T01:31:52.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.480+0000 7f9b397fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:52.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.480+0000 7f9b397fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:31:52.536 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:52.532+0000 7f269c3c2ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T01:31:52.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.541+0000 7f9b4b820640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-15T01:31:52.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.541+0000 7f9b4b820640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-15T01:31:52.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.541+0000 7f9b4b820640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:31:52.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.541+0000 7f9b4b820640 1 -- 172.21.15.178:0/1174939372 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f9b4404b720 con 0x7f9b28006bd0 2024-09-15T01:31:52.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.545+0000 7f9b397fa640 1 -- 172.21.15.178:0/1174939372 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9b340075b0 con 0x7f9b28006bd0 2024-09-15T01:31:52.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.545+0000 7f9b397fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-15T01:31:52.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.545+0000 7f9b397fa640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2024-09-15T01:31:52.547 INFO:teuthology.orchestra.run.smithi178.stderr:adjusted tunables profile to default 2024-09-15T01:31:52.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.546+0000 7f9b4b820640 10 monclient: shutdown 2024-09-15T01:31:52.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.546+0000 7f9b4b820640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:52.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.546+0000 7f9b4b820640 1 -- 172.21.15.178:0/1174939372 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006bd0 msgr2=0x7f9b4412fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:52.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.546+0000 7f9b4b820640 1 --2- 172.21.15.178:0/1174939372 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006bd0 0x7f9b4412fb30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9b34015730 tx=0x7f9b34009040 comp rx=0 tx=0).stop 2024-09-15T01:31:52.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.546+0000 7f9b4b820640 1 -- 172.21.15.178:0/1174939372 shutdown_connections 2024-09-15T01:31:52.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.546+0000 7f9b4b820640 1 --2- 172.21.15.178:0/1174939372 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9b28006bd0 0x7f9b4412fb30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.546+0000 7f9b4b820640 1 --2- 172.21.15.178:0/1174939372 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9b28007020 0x7f9b44130070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.546+0000 7f9b4b820640 1 --2- 172.21.15.178:0/1174939372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b28003c10 0x7f9b441373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:52.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.546+0000 7f9b4b820640 1 -- 172.21.15.178:0/1174939372 >> 172.21.15.178:0/1174939372 conn(0x7f9b440a4390 msgr2=0x7f9b440a32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:52.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.546+0000 7f9b4b820640 1 -- 172.21.15.178:0/1174939372 shutdown_connections 2024-09-15T01:31:52.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:52.546+0000 7f9b4b820640 1 -- 172.21.15.178:0/1174939372 wait complete. 2024-09-15T01:31:52.559 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T01:31:52.559 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:52.559 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T01:31:52.574 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:52.571+0000 7f269c3c2ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T01:31:52.585 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:52.585 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T01:31:52.623 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:52.621+0000 7f269c3c2ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T01:31:52.652 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-15T01:31:52.652 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T01:31:52.718 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:52.718 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T01:31:52.743 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:52.743 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-15T01:31:52.784 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:52.782+0000 7f269c3c2ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T01:31:52.809 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:31:52.809 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-15T01:31:52.874 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 690d2bc0-3a5e-4f4a-b00b-43a73741b7b1 0 2024-09-15T01:31:52.897 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:52.895+0000 7f269c3c2ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T01:31:52.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.987+0000 7fc18255d640 10 monclient: get_monmap_and_config 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.987+0000 7fc18255d640 10 monclient: build_initial_monmap 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.987+0000 7fc18255d640 10 monclient: monmap: 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 0 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:52.988763+0000 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:52.988763+0000 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:52.990 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:52.991 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:52.991 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:52.991 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:52.991 INFO:teuthology.orchestra.run.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:31:52.991 INFO:teuthology.orchestra.run.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:31:52.991 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:52.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.987+0000 7fc18255d640 1 Processor -- start 2024-09-15T01:31:52.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 1 -- start start 2024-09-15T01:31:52.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 10 monclient: init 2024-09-15T01:31:52.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:52.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 10 monclient: _add_conns ranks=[0,7,6,10,8,5,3,9,1,4,2] 2024-09-15T01:31:52.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc17c0678d0 addr v1:172.21.15.178:6789/0 2024-09-15T01:31:52.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc17c06ea30 0x7fc17c09ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:52.999 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 10 monclient(hunting): picked mon.noname-i con 0x7fc17c06ea30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc17c1359e0 0x7fc17c137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 10 monclient(hunting): picked mon.noname-d con 0x7fc17c1359e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17c04bf60 con 0x7fc17c0678d0 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c06d5b0 con 0x7fc17c1359e0 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c067ca0 con 0x7fc17c06ea30 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc18255d640 10 monclient(hunting): authenticate will time out at 728.631653s 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc17b7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc17c06ea30 0x7fc17c09ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc17b7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc17c06ea30 0x7fc17c09ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:39708/0 (socket says 172.21.15.204:39708) 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc17b7fe640 1 -- 172.21.15.204:0/3098551178 learned_addr learned my addr 172.21.15.204:0/3098551178 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:31:53.000 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc17b7fe640 10 monclient(hunting): get_auth_request con 0x7fc17c06ea30 auth_method 0 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc17b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc17b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc17b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc17b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc17b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc17b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.988+0000 7fc180afa640 1 --2- 172.21.15.204:0/3098551178 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc17c1359e0 0x7fc17c137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17b7fe640 10 monclient(hunting): handle_auth_done global_id 4129 payload 274 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17b7fe640 1 -- 172.21.15.204:0/3098551178 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc17c1359e0 msgr2=0x7fc17c137dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17b7fe640 1 --2- 172.21.15.204:0/3098551178 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc17c1359e0 0x7fc17c137dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17b7fe640 1 -- 172.21.15.204:0/3098551178 >> v1:172.21.15.178:6789/0 conn(0x7fc17c0678d0 legacy=0x7fc17c09e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17b7fe640 1 monclient: found mon.noname-i 2024-09-15T01:31:53.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17b7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17b7fe640 1 -- 172.21.15.204:0/3098551178 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c06d890 con 0x7fc17c06ea30 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc180afa640 10 monclient: get_auth_request con 0x7fc17c1359e0 auth_method 0 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc180afa640 1 --2- 172.21.15.204:0/3098551178 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc17c1359e0 0x7fc17c137dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17b7fe640 1 --2- 172.21.15.204:0/3098551178 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc17c06ea30 0x7fc17c09ee10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc170009940 tx=0x7fc1700091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=1b66d16308de8d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17affd640 1 -- 172.21.15.204:0/3098551178 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc17002ad90 con 0x7fc17c06ea30 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17affd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.989+0000 7fc17affd640 10 monclient: dump: 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.002 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc17affd640 10 monclient: _finish_auth 0 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc17affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 5 monclient: authenticate success, global_id 4129 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc17affd640 1 -- 172.21.15.204:0/3098551178 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc170031820 con 0x7fc17c06ea30 2024-09-15T01:31:53.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc17affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 10 monclient: get_monmap_and_config success 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 4 set_mon_vals no callback set 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc17affd640 1 -- 172.21.15.204:0/3098551178 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc170031e90 con 0x7fc17c06ea30 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc17affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc17affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc17affd640 10 monclient: dump: 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.004 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 10 monclient: shutdown 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 1 -- 172.21.15.204:0/3098551178 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc17c06ea30 msgr2=0x7fc17c09ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.005 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 1 --2- 172.21.15.204:0/3098551178 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc17c06ea30 0x7fc17c09ee10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc170009940 tx=0x7fc1700091d0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 1 -- 172.21.15.204:0/3098551178 shutdown_connections 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 1 --2- 172.21.15.204:0/3098551178 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc17c1359e0 0x7fc17c137dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 1 --2- 172.21.15.204:0/3098551178 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc17c06ea30 0x7fc17c09ee10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 1 -- 172.21.15.204:0/3098551178 >> 172.21.15.204:0/3098551178 conn(0x7fc17c0a4390 msgr2=0x7fc17c067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 1 -- 172.21.15.204:0/3098551178 shutdown_connections 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.990+0000 7fc18255d640 1 -- 172.21.15.204:0/3098551178 wait complete. 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.991+0000 7fc18255d640 10 monclient: build_initial_monmap 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.991+0000 7fc18255d640 10 monclient: monmap: 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 0 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 0.000000 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:created 0.000000 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 1 2024-09-15T01:31:53.006 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.991+0000 7fc18255d640 1 Processor -- start 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.991+0000 7fc18255d640 1 -- start start 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.991+0000 7fc18255d640 10 monclient: init 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 10 monclient: _add_conns ranks=[5,6,2,4,1,8,0,3,7] 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc17c0678d0 0x7fc17c11cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 10 monclient(hunting): picked mon.noname-h con 0x7fc17c0678d0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc17c06ea30 0x7fc17c11d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 10 monclient(hunting): picked mon.noname-d con 0x7fc17c06ea30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc17c1359e0 0x7fc17c11d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 10 monclient(hunting): picked mon.noname-b con 0x7fc17c1359e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c11f050 con 0x7fc17c1359e0 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c11ead0 con 0x7fc17c0678d0 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c06ccd0 con 0x7fc17c06ea30 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 10 monclient(hunting): authenticate will time out at 728.635095s 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc17c0678d0 0x7fc17c11cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.008 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc17c0678d0 0x7fc17c11cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:33812/0 (socket says 172.21.15.204:33812) 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 1 -- 172.21.15.204:0/312774939 learned_addr learned my addr 172.21.15.204:0/312774939 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 10 monclient(hunting): get_auth_request con 0x7fc17c0678d0 auth_method 0 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17b7fe640 1 --2- 172.21.15.204:0/312774939 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc17c06ea30 0x7fc17c11d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc180afa640 1 --2- 172.21.15.204:0/312774939 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc17c1359e0 0x7fc17c11d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17b7fe640 10 monclient(hunting): get_auth_request con 0x7fc17c06ea30 auth_method 0 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc180afa640 10 monclient(hunting): get_auth_request con 0x7fc17c1359e0 auth_method 0 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc180afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.009 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc180afa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc180afa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc180afa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc180afa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc180afa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 10 monclient(hunting): handle_auth_done global_id 4109 payload 824 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 1 -- 172.21.15.204:0/312774939 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc17c06ea30 msgr2=0x7fc17c11d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 1 --2- 172.21.15.204:0/312774939 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc17c06ea30 0x7fc17c11d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.010 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 1 -- 172.21.15.204:0/312774939 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc17c1359e0 msgr2=0x7fc17c11d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 1 --2- 172.21.15.204:0/312774939 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc17c1359e0 0x7fc17c11d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 1 monclient: found mon.noname-h 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 1 -- 172.21.15.204:0/312774939 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc170009450 con 0x7fc17c0678d0 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17bfff640 1 --2- 172.21.15.204:0/312774939 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc17c0678d0 0x7fc17c11cf40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00c590 tx=0x7fc16c007ef0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc163fff640 1 -- 172.21.15.204:0/312774939 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc16c012ab0 con 0x7fc17c0678d0 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc163fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc163fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc163fff640 10 monclient: dump: 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.011 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc163fff640 10 monclient: _finish_auth 0 2024-09-15T01:31:53.012 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc163fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 5 monclient: authenticate success, global_id 4109 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc163fff640 1 -- 172.21.15.204:0/312774939 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c012c90 con 0x7fc17c0678d0 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 10 monclient: _renew_subs 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc18255d640 1 -- 172.21.15.204:0/312774939 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17c06c900 con 0x7fc17c0678d0 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc163fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc163fff640 1 -- 172.21.15.204:0/312774939 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc16c012e00 con 0x7fc17c0678d0 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc163fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc163fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc163fff640 10 monclient: dump: 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.013 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.992+0000 7fc17a7fc640 4 set_mon_vals no callback set 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.993+0000 7fc18255d640 10 monclient: _renew_subs 2024-09-15T01:31:53.014 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.993+0000 7fc18255d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.015 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.993+0000 7fc18255d640 1 -- 172.21.15.204:0/312774939 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17c120590 con 0x7fc17c0678d0 2024-09-15T01:31:53.015 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.993+0000 7fc163fff640 1 -- 172.21.15.204:0/312774939 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fc16c012630 con 0x7fc17c0678d0 2024-09-15T01:31:53.015 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.993+0000 7fc163fff640 1 -- 172.21.15.204:0/312774939 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc16c01bc50 con 0x7fc17c0678d0 2024-09-15T01:31:53.015 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.993+0000 7fc18255d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.015 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.993+0000 7fc18255d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.015 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.993+0000 7fc18255d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.015 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.993+0000 7fc18255d640 1 -- 172.21.15.204:0/312774939 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17c05ed70 con 0x7fc17c0678d0 2024-09-15T01:31:53.015 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.995+0000 7fc163fff640 1 -- 172.21.15.204:0/312774939 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fc16c01be10 con 0x7fc17c0678d0 2024-09-15T01:31:53.015 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.995+0000 7fc163fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.015 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:52.995+0000 7fc163fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:31:53.063 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.061+0000 7fc18255d640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "690d2bc0-3a5e-4f4a-b00b-43a73741b7b1", "id": 0}] 2024-09-15T01:31:53.063 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.061+0000 7fc18255d640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "690d2bc0-3a5e-4f4a-b00b-43a73741b7b1", "id": 0}] 2024-09-15T01:31:53.064 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.061+0000 7fc18255d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.064 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.061+0000 7fc18255d640 1 -- 172.21.15.204:0/312774939 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd new", "uuid": "690d2bc0-3a5e-4f4a-b00b-43a73741b7b1", "id": 0} v 0) v1 -- 0x7fc17c04b720 con 0x7fc17c0678d0 2024-09-15T01:31:53.067 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.065+0000 7fc163fff640 1 -- 172.21.15.204:0/312774939 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "690d2bc0-3a5e-4f4a-b00b-43a73741b7b1", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc16c01b400 con 0x7fc17c0678d0 2024-09-15T01:31:53.067 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.065+0000 7fc163fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "690d2bc0-3a5e-4f4a-b00b-43a73741b7b1", "id": 0}] 2024-09-15T01:31:53.067 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.065+0000 7fc163fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:31:53.068 INFO:teuthology.orchestra.run.smithi204.stdout:0 2024-09-15T01:31:53.069 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.066+0000 7fc18255d640 10 monclient: shutdown 2024-09-15T01:31:53.069 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.066+0000 7fc18255d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:53.069 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.066+0000 7fc18255d640 1 -- 172.21.15.204:0/312774939 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc17c0678d0 msgr2=0x7fc17c11cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.069 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.066+0000 7fc18255d640 1 --2- 172.21.15.204:0/312774939 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc17c0678d0 0x7fc17c11cf40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00c590 tx=0x7fc16c007ef0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.069 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.066+0000 7fc18255d640 1 -- 172.21.15.204:0/312774939 shutdown_connections 2024-09-15T01:31:53.069 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.066+0000 7fc18255d640 1 --2- 172.21.15.204:0/312774939 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc17c1359e0 0x7fc17c11d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.069 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.066+0000 7fc18255d640 1 --2- 172.21.15.204:0/312774939 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc17c06ea30 0x7fc17c11d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.069 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.066+0000 7fc18255d640 1 --2- 172.21.15.204:0/312774939 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc17c0678d0 0x7fc17c11cf40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.069 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.066+0000 7fc18255d640 1 -- 172.21.15.204:0/312774939 >> 172.21.15.204:0/312774939 conn(0x7fc17c0a4390 msgr2=0x7fc17c09cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:53.069 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.066+0000 7fc18255d640 1 -- 172.21.15.204:0/312774939 shutdown_connections 2024-09-15T01:31:53.069 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.066+0000 7fc18255d640 1 -- 172.21.15.204:0/312774939 wait complete. 2024-09-15T01:31:53.079 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new bbba9997-deb4-42ca-8140-6553a9b6c45e 1 2024-09-15T01:31:53.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.169+0000 7fd89a4e8640 10 monclient: get_monmap_and_config 2024-09-15T01:31:53.171 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.169+0000 7fd89a4e8640 10 monclient: build_initial_monmap 2024-09-15T01:31:53.172 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.169+0000 7fd89a4e8640 10 monclient: monmap: 2024-09-15T01:31:53.172 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 0 2024-09-15T01:31:53.172 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.172 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:53.170530+0000 2024-09-15T01:31:53.172 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:53.170530+0000 2024-09-15T01:31:53.172 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:53.172 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.172 INFO:teuthology.orchestra.run.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:31:53.172 INFO:teuthology.orchestra.run.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:31:53.172 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:53.172 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:53.172 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:53.173 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:53.173 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:53.173 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:53.173 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:53.173 INFO:teuthology.orchestra.run.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:31:53.173 INFO:teuthology.orchestra.run.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:31:53.173 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.169+0000 7fd89a4e8640 1 Processor -- start 2024-09-15T01:31:53.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.169+0000 7fd89a4e8640 1 -- start start 2024-09-15T01:31:53.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.169+0000 7fd89a4e8640 10 monclient: init 2024-09-15T01:31:53.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:53.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 10 monclient: _add_conns ranks=[8,0,4,9,1,7,2,5,6,3,10] 2024-09-15T01:31:53.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd8940a2930 0x7fd894132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 10 monclient(hunting): picked mon.noname-e con 0x7fd8940a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:31:53.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd89409faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:31:53.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd89409fe70 0x7fd8940a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 10 monclient(hunting): picked mon.noname-c con 0x7fd89409fe70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:31:53.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89404bf60 con 0x7fd89409faa0 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd89409dd80 con 0x7fd89409fe70 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940a2d00 con 0x7fd8940a2930 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd89a4e8640 10 monclient(hunting): authenticate will time out at 728.813373s 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd89409fe70 0x7fd8940a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd8937fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd89409faa0 0x7fd8940a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.204:47192/0 (socket says 172.21.15.204:47192) 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd893fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd8940a2930 0x7fd894132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd8937fe640 1 -- 172.21.15.204:0/3805334662 learned_addr learned my addr 172.21.15.204:0/3805334662 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 10 monclient(hunting): get_auth_request con 0x7fd89409fe70 auth_method 0 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd893fff640 10 monclient(hunting): get_auth_request con 0x7fd8940a2930 auth_method 0 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd893fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd893fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd893fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd893fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd893fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd893fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 10 monclient(hunting): handle_auth_done global_id 4126 payload 274 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd892ffd640 1 -- 172.21.15.204:0/3805334662 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171133503 0 0) 0x7fd89404bf60 con 0x7fd89409faa0 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 1 -- 172.21.15.204:0/3805334662 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd8940a2930 msgr2=0x7fd894132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 1 --2- 172.21.15.204:0/3805334662 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd8940a2930 0x7fd894132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 1 -- 172.21.15.204:0/3805334662 >> v1:172.21.15.178:6789/0 conn(0x7fd89409faa0 legacy=0x7fd8940a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 10 monclient: _finish_hunting 0 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 1 monclient: found mon.noname-c 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd898a85640 1 -- 172.21.15.204:0/3805334662 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8940a0310 con 0x7fd89409fe70 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd893fff640 1 --2- 172.21.15.204:0/3805334662 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd8940a2930 0x7fd894132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.170+0000 7fd892ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd898a85640 1 --2- 172.21.15.204:0/3805334662 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd89409fe70 0x7fd8940a0fd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009e90 tx=0x7fd88c009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=daf34ad7c9a39b84 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd892ffd640 1 -- 172.21.15.204:0/3805334662 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd88c031880 con 0x7fd89409fe70 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd892ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd892ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd892ffd640 10 monclient: dump: 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.184 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.185 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd892ffd640 10 monclient: _finish_auth 0 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd892ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd892ffd640 1 -- 172.21.15.204:0/3805334662 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c02d050 con 0x7fd89409fe70 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd89a4e8640 5 monclient: authenticate success, global_id 4126 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd89a4e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd892ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd89a4e8640 10 monclient: get_monmap_and_config success 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd892ffd640 1 -- 172.21.15.204:0/3805334662 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd88c0303a0 con 0x7fd89409fe70 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd892ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd89a4e8640 4 set_mon_vals no callback set 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd892ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.171+0000 7fd892ffd640 10 monclient: dump: 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.186 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.187 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.172+0000 7fd89a4e8640 10 monclient: shutdown 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.172+0000 7fd89a4e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.172+0000 7fd89a4e8640 1 -- 172.21.15.204:0/3805334662 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd89409fe70 msgr2=0x7fd8940a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.172+0000 7fd89a4e8640 1 --2- 172.21.15.204:0/3805334662 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd89409fe70 0x7fd8940a0fd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009e90 tx=0x7fd88c009780 comp rx=0 tx=0).stop 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.172+0000 7fd89a4e8640 1 -- 172.21.15.204:0/3805334662 shutdown_connections 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.172+0000 7fd89a4e8640 1 --2- 172.21.15.204:0/3805334662 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd89409fe70 0x7fd8940a0fd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.172+0000 7fd89a4e8640 1 --2- 172.21.15.204:0/3805334662 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd8940a2930 0x7fd894132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.172+0000 7fd89a4e8640 1 -- 172.21.15.204:0/3805334662 >> 172.21.15.204:0/3805334662 conn(0x7fd8940a5ed0 msgr2=0x7fd8940a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.172+0000 7fd89a4e8640 1 -- 172.21.15.204:0/3805334662 shutdown_connections 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.172+0000 7fd89a4e8640 1 -- 172.21.15.204:0/3805334662 wait complete. 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.172+0000 7fd89a4e8640 10 monclient: build_initial_monmap 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.172+0000 7fd89a4e8640 10 monclient: monmap: 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 0 2024-09-15T01:31:53.188 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 0.000000 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:created 0.000000 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 1 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:53.189 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 1 Processor -- start 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 1 -- start start 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 10 monclient: init 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 10 monclient: _add_conns ranks=[8,3,6,2,7,4,5,1,0] 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd89409faa0 0x7fd894141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 10 monclient(hunting): picked mon.noname-e con 0x7fd89409faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89409fe70 0x7fd894131880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 10 monclient(hunting): picked mon.noname-g con 0x7fd89409fe70 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd8940a2930 0x7fd89411e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 10 monclient(hunting): picked mon.noname-d con 0x7fd8940a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd894121cb0 con 0x7fd89409fe70 2024-09-15T01:31:53.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd89411fea0 con 0x7fd8940a2930 2024-09-15T01:31:53.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941219d0 con 0x7fd89409faa0 2024-09-15T01:31:53.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:53.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd89a4e8640 10 monclient(hunting): authenticate will time out at 728.816804s 2024-09-15T01:31:53.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd8937fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89409fe70 0x7fd894131880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd8937fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89409fe70 0x7fd894131880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:38546/0 (socket says 172.21.15.204:38546) 2024-09-15T01:31:53.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd8937fe640 1 -- 172.21.15.204:0/1512277329 learned_addr learned my addr 172.21.15.204:0/1512277329 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:31:53.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd8937fe640 10 monclient(hunting): get_auth_request con 0x7fd89409fe70 auth_method 0 2024-09-15T01:31:53.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd8937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd8937fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.173+0000 7fd8937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd898a85640 1 --2- 172.21.15.204:0/1512277329 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd8940a2930 0x7fd89411e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd893fff640 1 --2- 172.21.15.204:0/1512277329 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd89409faa0 0x7fd894141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd898a85640 10 monclient(hunting): get_auth_request con 0x7fd8940a2930 auth_method 0 2024-09-15T01:31:53.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd898a85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd898a85640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd898a85640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd893fff640 10 monclient(hunting): get_auth_request con 0x7fd89409faa0 auth_method 0 2024-09-15T01:31:53.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd893fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.192 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd893fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd893fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd898a85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd898a85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd898a85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd893fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd893fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd893fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 10 monclient(hunting): handle_auth_done global_id 4134 payload 824 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 1 -- 172.21.15.204:0/1512277329 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd89409faa0 msgr2=0x7fd894141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 1 --2- 172.21.15.204:0/1512277329 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd89409faa0 0x7fd894141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.193 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 1 -- 172.21.15.204:0/1512277329 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd8940a2930 msgr2=0x7fd89411e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 1 --2- 172.21.15.204:0/1512277329 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd8940a2930 0x7fd89411e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 10 monclient: _finish_hunting 0 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 1 monclient: found mon.noname-g 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 1 -- 172.21.15.204:0/1512277329 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd884009390 con 0x7fd89409fe70 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd898a85640 1 --2- 172.21.15.204:0/1512277329 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd8940a2930 0x7fd89411e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8937fe640 1 --2- 172.21.15.204:0/1512277329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89409fe70 0x7fd894131880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00c4e0 tx=0x7fd87c007f00 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 1 -- 172.21.15.204:0/1512277329 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd87c0070d0 con 0x7fd89409fe70 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 10 monclient: dump: 2024-09-15T01:31:53.194 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.195 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 10 monclient: _finish_auth 0 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 1 -- 172.21.15.204:0/1512277329 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c007270 con 0x7fd89409fe70 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd89a4e8640 5 monclient: authenticate success, global_id 4134 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd89a4e8640 10 monclient: _renew_subs 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd89a4e8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd89a4e8640 1 -- 172.21.15.204:0/1512277329 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c009a20 con 0x7fd89409fe70 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 1 -- 172.21.15.204:0/1512277329 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd87c011360 con 0x7fd89409fe70 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd8927fc640 4 set_mon_vals no callback set 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 10 monclient: dump: 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.196 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.197 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd89a4e8640 10 monclient: _renew_subs 2024-09-15T01:31:53.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd89a4e8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:31:53.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd89a4e8640 1 -- 172.21.15.204:0/1512277329 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd894141f80 con 0x7fd89409fe70 2024-09-15T01:31:53.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 1 -- 172.21.15.204:0/1512277329 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fd87c007420 con 0x7fd89409fe70 2024-09-15T01:31:53.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.174+0000 7fd88bfff640 1 -- 172.21.15.204:0/1512277329 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fd87c011910 con 0x7fd89409fe70 2024-09-15T01:31:53.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.175+0000 7fd89a4e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.175+0000 7fd89a4e8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.175+0000 7fd89a4e8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:31:53.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.175+0000 7fd89a4e8640 1 -- 172.21.15.204:0/1512277329 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89409d6f0 con 0x7fd89409fe70 2024-09-15T01:31:53.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.177+0000 7fd88bfff640 1 -- 172.21.15.204:0/1512277329 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fd89409d6f0 con 0x7fd89409fe70 2024-09-15T01:31:53.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.177+0000 7fd88bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.177+0000 7fd88bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:31:53.199 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:53.192+0000 7f269c3c2ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T01:31:53.245 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:53.242+0000 7f269c3c2ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T01:31:53.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.243+0000 7fd89a4e8640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "bbba9997-deb4-42ca-8140-6553a9b6c45e", "id": 1}] 2024-09-15T01:31:53.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.243+0000 7fd89a4e8640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "bbba9997-deb4-42ca-8140-6553a9b6c45e", "id": 1}] 2024-09-15T01:31:53.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.243+0000 7fd89a4e8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:31:53.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.243+0000 7fd89a4e8640 1 -- 172.21.15.204:0/1512277329 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bbba9997-deb4-42ca-8140-6553a9b6c45e", "id": 1} v 0) v1 -- 0x7fd89404b720 con 0x7fd89409fe70 2024-09-15T01:31:53.249 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.247+0000 7fd88bfff640 1 -- 172.21.15.204:0/1512277329 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bbba9997-deb4-42ca-8140-6553a9b6c45e", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd87c017070 con 0x7fd89409fe70 2024-09-15T01:31:53.250 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.247+0000 7fd88bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "bbba9997-deb4-42ca-8140-6553a9b6c45e", "id": 1}] 2024-09-15T01:31:53.250 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.247+0000 7fd88bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:31:53.250 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-09-15T01:31:53.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.248+0000 7fd89a4e8640 10 monclient: shutdown 2024-09-15T01:31:53.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.248+0000 7fd89a4e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:53.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.248+0000 7fd89a4e8640 1 -- 172.21.15.204:0/1512277329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89409fe70 msgr2=0x7fd894131880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.248+0000 7fd89a4e8640 1 --2- 172.21.15.204:0/1512277329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89409fe70 0x7fd894131880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00c4e0 tx=0x7fd87c007f00 comp rx=0 tx=0).stop 2024-09-15T01:31:53.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.248+0000 7fd89a4e8640 1 -- 172.21.15.204:0/1512277329 shutdown_connections 2024-09-15T01:31:53.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.248+0000 7fd89a4e8640 1 --2- 172.21.15.204:0/1512277329 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd8940a2930 0x7fd89411e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.248+0000 7fd89a4e8640 1 --2- 172.21.15.204:0/1512277329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89409fe70 0x7fd894131880 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.248+0000 7fd89a4e8640 1 --2- 172.21.15.204:0/1512277329 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd89409faa0 0x7fd894141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.248+0000 7fd89a4e8640 1 -- 172.21.15.204:0/1512277329 >> 172.21.15.204:0/1512277329 conn(0x7fd8940a5ed0 msgr2=0x7fd89409e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:53.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.248+0000 7fd89a4e8640 1 -- 172.21.15.204:0/1512277329 shutdown_connections 2024-09-15T01:31:53.252 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.248+0000 7fd89a4e8640 1 -- 172.21.15.204:0/1512277329 wait complete. 2024-09-15T01:31:53.261 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 24b56e74-a222-40f9-a8bf-38d597553f58 2 2024-09-15T01:31:53.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.351+0000 7ff8a80c0640 10 monclient: get_monmap_and_config 2024-09-15T01:31:53.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.351+0000 7ff8a80c0640 10 monclient: build_initial_monmap 2024-09-15T01:31:53.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.351+0000 7ff8a80c0640 10 monclient: monmap: 2024-09-15T01:31:53.354 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 0 2024-09-15T01:31:53.355 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.355 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:53.353180+0000 2024-09-15T01:31:53.355 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:53.353180+0000 2024-09-15T01:31:53.355 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:53.355 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.355 INFO:teuthology.orchestra.run.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:31:53.355 INFO:teuthology.orchestra.run.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:31:53.355 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:53.355 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:53.355 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:53.355 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:53.355 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:53.356 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:53.356 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:53.356 INFO:teuthology.orchestra.run.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:31:53.356 INFO:teuthology.orchestra.run.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:31:53.356 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.356 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 1 Processor -- start 2024-09-15T01:31:53.365 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 1 -- start start 2024-09-15T01:31:53.365 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 10 monclient: init 2024-09-15T01:31:53.365 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:53.365 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 10 monclient: _add_conns ranks=[3,4,7,5,6,2,0,1,10,9,8] 2024-09-15T01:31:53.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8a0067130 0x7ff8a0067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 10 monclient(hunting): picked mon.noname-g con 0x7ff8a0067130 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:31:53.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff8a009d580 0x7ff8a0067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 10 monclient(hunting): picked mon.noname-c con 0x7ff8a009d580 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:31:53.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8a009f000 0x7ff8a012d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 10 monclient(hunting): picked mon.noname-i con 0x7ff8a009f000 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:31:53.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a009e0c0 con 0x7ff8a0067130 2024-09-15T01:31:53.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a0067f80 con 0x7ff8a009d580 2024-09-15T01:31:53.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a009dde0 con 0x7ff8a009f000 2024-09-15T01:31:53.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:53.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.352+0000 7ff8a80c0640 10 monclient(hunting): authenticate will time out at 728.995953s 2024-09-15T01:31:53.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8a009f000 0x7ff8a012d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a5e5c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8a0067130 0x7ff8a0067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a5e5c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8a0067130 0x7ff8a0067500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:38562/0 (socket says 172.21.15.204:38562) 2024-09-15T01:31:53.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8a009f000 0x7ff8a012d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:39724/0 (socket says 172.21.15.204:39724) 2024-09-15T01:31:53.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a5e5c640 1 -- 172.21.15.204:0/1715596091 learned_addr learned my addr 172.21.15.204:0/1715596091 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:31:53.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a5e5c640 10 monclient(hunting): get_auth_request con 0x7ff8a0067130 auth_method 0 2024-09-15T01:31:53.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a5e5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a5e5c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a5e5c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 10 monclient(hunting): get_auth_request con 0x7ff8a009f000 auth_method 0 2024-09-15T01:31:53.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a5e5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a5e5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a5e5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.369 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.369 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a565b640 1 --2- 172.21.15.204:0/1715596091 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff8a009d580 0x7ff8a0067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.369 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a565b640 10 monclient(hunting): get_auth_request con 0x7ff8a009d580 auth_method 0 2024-09-15T01:31:53.369 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a565b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.369 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a565b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.369 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a565b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.369 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 10 monclient(hunting): handle_auth_done global_id 4138 payload 274 2024-09-15T01:31:53.369 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 1 -- 172.21.15.204:0/1715596091 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff8a009d580 msgr2=0x7ff8a0067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:53.369 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 1 --2- 172.21.15.204:0/1715596091 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff8a009d580 0x7ff8a0067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.369 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 1 -- 172.21.15.204:0/1715596091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8a0067130 msgr2=0x7ff8a0067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:53.369 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 1 --2- 172.21.15.204:0/1715596091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8a0067130 0x7ff8a0067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 10 monclient: _finish_hunting 0 2024-09-15T01:31:53.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 1 monclient: found mon.noname-i 2024-09-15T01:31:53.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:53.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:31:53.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a665d640 1 -- 172.21.15.204:0/1715596091 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a004bf60 con 0x7ff8a009f000 2024-09-15T01:31:53.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a5e5c640 1 --2- 172.21.15.204:0/1715596091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8a0067130 0x7ff8a0067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:31:53.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.353+0000 7ff8a565b640 1 --2- 172.21.15.204:0/1715596091 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff8a009d580 0x7ff8a0067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:31:53.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a665d640 1 --2- 172.21.15.204:0/1715596091 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8a009f000 0x7ff8a012d330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff898009e90 tx=0x7ff898009780 comp rx=0 tx=0).ready entity=mon.7 client_cookie=4da207afdac69f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:53.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a4e5a640 1 -- 172.21.15.204:0/1715596091 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff8980079d0 con 0x7ff8a009f000 2024-09-15T01:31:53.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a4e5a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a4e5a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:31:53.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a4e5a640 10 monclient: dump: 2024-09-15T01:31:53.371 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.371 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.371 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.371 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.371 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.371 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.371 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.371 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.371 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.371 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.371 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.372 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.372 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.372 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.372 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.372 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a4e5a640 10 monclient: _finish_auth 0 2024-09-15T01:31:53.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a4e5a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:53.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a80c0640 5 monclient: authenticate success, global_id 4138 2024-09-15T01:31:53.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a4e5a640 1 -- 172.21.15.204:0/1715596091 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89802d050 con 0x7ff8a009f000 2024-09-15T01:31:53.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a80c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:31:53.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a4e5a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:53.373 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a80c0640 10 monclient: get_monmap_and_config success 2024-09-15T01:31:53.373 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a80c0640 4 set_mon_vals no callback set 2024-09-15T01:31:53.373 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a4e5a640 1 -- 172.21.15.204:0/1715596091 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff898007bd0 con 0x7ff8a009f000 2024-09-15T01:31:53.373 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a4e5a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.373 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a4e5a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:31:53.373 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a4e5a640 10 monclient: dump: 2024-09-15T01:31:53.373 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.373 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.373 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.373 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.373 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.373 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.374 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.374 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.374 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.374 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.374 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.374 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.374 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.374 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.374 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.374 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a80c0640 10 monclient: shutdown 2024-09-15T01:31:53.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a80c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:53.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1715596091 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8a009f000 msgr2=0x7ff8a012d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.354+0000 7ff8a80c0640 1 --2- 172.21.15.204:0/1715596091 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8a009f000 0x7ff8a012d330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff898009e90 tx=0x7ff898009780 comp rx=0 tx=0).stop 2024-09-15T01:31:53.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.355+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1715596091 shutdown_connections 2024-09-15T01:31:53.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.355+0000 7ff8a80c0640 1 --2- 172.21.15.204:0/1715596091 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8a009f000 0x7ff8a012d330 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.355+0000 7ff8a80c0640 1 --2- 172.21.15.204:0/1715596091 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff8a009d580 0x7ff8a0067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.355+0000 7ff8a80c0640 1 --2- 172.21.15.204:0/1715596091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8a0067130 0x7ff8a0067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.355+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1715596091 >> 172.21.15.204:0/1715596091 conn(0x7ff8a0068a60 msgr2=0x7ff8a00a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:53.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.355+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1715596091 shutdown_connections 2024-09-15T01:31:53.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.355+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1715596091 wait complete. 2024-09-15T01:31:53.376 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.355+0000 7ff8a80c0640 10 monclient: build_initial_monmap 2024-09-15T01:31:53.376 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.355+0000 7ff8a80c0640 10 monclient: monmap: 2024-09-15T01:31:53.376 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 0 2024-09-15T01:31:53.376 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:31:53.376 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 0.000000 2024-09-15T01:31:53.376 INFO:teuthology.orchestra.run.smithi204.stderr:created 0.000000 2024-09-15T01:31:53.376 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:53.376 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 1 2024-09-15T01:31:53.376 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:31:53.376 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:31:53.376 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:53.376 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:53.377 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:53.377 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:53.377 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:53.377 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:53.377 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:53.377 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.355+0000 7ff8a80c0640 1 Processor -- start 2024-09-15T01:31:53.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 1 -- start start 2024-09-15T01:31:53.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 10 monclient: init 2024-09-15T01:31:53.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:53.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 10 monclient: _add_conns ranks=[8,2,5,6,4,7,0,1,3] 2024-09-15T01:31:53.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8a0067130 0x7ff8a0133550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 10 monclient(hunting): picked mon.noname-e con 0x7ff8a0067130 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:31:53.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8a009d580 0x7ff8a011d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 10 monclient(hunting): picked mon.noname-b con 0x7ff8a009d580 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:31:53.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8a009f000 0x7ff8a011d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 10 monclient(hunting): picked mon.noname-h con 0x7ff8a009f000 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:31:53.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a0134cd0 con 0x7ff8a009d580 2024-09-15T01:31:53.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a009cb50 con 0x7ff8a009f000 2024-09-15T01:31:53.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a009d9e0 con 0x7ff8a0067130 2024-09-15T01:31:53.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:53.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a80c0640 10 monclient(hunting): authenticate will time out at 728.999639s 2024-09-15T01:31:53.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a665d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8a009f000 0x7ff8a011d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a665d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8a009f000 0x7ff8a011d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:33820/0 (socket says 172.21.15.204:33820) 2024-09-15T01:31:53.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a665d640 1 -- 172.21.15.204:0/1257102354 learned_addr learned my addr 172.21.15.204:0/1257102354 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:31:53.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a665d640 10 monclient(hunting): get_auth_request con 0x7ff8a009f000 auth_method 0 2024-09-15T01:31:53.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a665d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a665d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a665d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a5e5c640 1 --2- 172.21.15.204:0/1257102354 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8a0067130 0x7ff8a0133550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a665d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a665d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a665d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a5e5c640 10 monclient(hunting): get_auth_request con 0x7ff8a0067130 auth_method 0 2024-09-15T01:31:53.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a5e5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a5e5c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a5e5c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a565b640 1 --2- 172.21.15.204:0/1257102354 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8a009d580 0x7ff8a011d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a565b640 10 monclient(hunting): get_auth_request con 0x7ff8a009d580 auth_method 0 2024-09-15T01:31:53.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a565b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a565b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a565b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a5e5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a5e5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.356+0000 7ff8a5e5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a565b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a565b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a565b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a665d640 10 monclient(hunting): handle_auth_done global_id 4118 payload 824 2024-09-15T01:31:53.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a665d640 1 -- 172.21.15.204:0/1257102354 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8a0067130 msgr2=0x7ff8a0133550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a665d640 1 --2- 172.21.15.204:0/1257102354 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8a0067130 0x7ff8a0133550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a665d640 1 -- 172.21.15.204:0/1257102354 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8a009d580 msgr2=0x7ff8a011d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a665d640 1 --2- 172.21.15.204:0/1257102354 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8a009d580 0x7ff8a011d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a665d640 10 monclient: _finish_hunting 0 2024-09-15T01:31:53.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a665d640 1 monclient: found mon.noname-h 2024-09-15T01:31:53.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a665d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:53.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a665d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a665d640 1 -- 172.21.15.204:0/1257102354 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c009450 con 0x7ff8a009f000 2024-09-15T01:31:53.383 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a665d640 1 --2- 172.21.15.204:0/1257102354 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8a009f000 0x7ff8a011d5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8980099f0 tx=0x7ff898037000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:53.383 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 1 -- 172.21.15.204:0/1257102354 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff898009460 con 0x7ff8a009f000 2024-09-15T01:31:53.383 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.383 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:31:53.383 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 10 monclient: dump: 2024-09-15T01:31:53.383 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.383 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.383 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.383 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.383 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.383 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.383 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.384 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.384 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.384 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.384 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.384 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.384 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.384 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.384 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.384 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.384 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 10 monclient: _finish_auth 0 2024-09-15T01:31:53.384 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:53.384 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a80c0640 5 monclient: authenticate success, global_id 4118 2024-09-15T01:31:53.385 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 1 -- 172.21.15.204:0/1257102354 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89803c420 con 0x7ff8a009f000 2024-09-15T01:31:53.385 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:53.385 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 1 -- 172.21.15.204:0/1257102354 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff89803ca10 con 0x7ff8a009f000 2024-09-15T01:31:53.385 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.385 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:31:53.385 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff897fff640 4 set_mon_vals no callback set 2024-09-15T01:31:53.385 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 10 monclient: dump: 2024-09-15T01:31:53.385 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.385 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.385 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.385 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.386 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a80c0640 10 monclient: _renew_subs 2024-09-15T01:31:53.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a80c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1257102354 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff898009a20 con 0x7ff8a009f000 2024-09-15T01:31:53.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a80c0640 10 monclient: _renew_subs 2024-09-15T01:31:53.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a80c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1257102354 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a011f710 con 0x7ff8a009f000 2024-09-15T01:31:53.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 1 -- 172.21.15.204:0/1257102354 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ff898003040 con 0x7ff8a009f000 2024-09-15T01:31:53.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.357+0000 7ff895ffb640 1 -- 172.21.15.204:0/1257102354 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7ff89803cc00 con 0x7ff8a009f000 2024-09-15T01:31:53.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.358+0000 7ff8a80c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.358+0000 7ff8a80c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.358+0000 7ff8a80c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.358+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1257102354 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a0064650 con 0x7ff8a009f000 2024-09-15T01:31:53.387 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.360+0000 7ff895ffb640 1 -- 172.21.15.204:0/1257102354 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7ff89803c590 con 0x7ff8a009f000 2024-09-15T01:31:53.388 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.360+0000 7ff895ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.388 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.360+0000 7ff895ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:31:53.388 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:53.382+0000 7f269c3c2ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T01:31:53.430 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.428+0000 7ff8a80c0640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "24b56e74-a222-40f9-a8bf-38d597553f58", "id": 2}] 2024-09-15T01:31:53.430 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.428+0000 7ff8a80c0640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "24b56e74-a222-40f9-a8bf-38d597553f58", "id": 2}] 2024-09-15T01:31:53.431 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.428+0000 7ff8a80c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.431 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.428+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1257102354 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd new", "uuid": "24b56e74-a222-40f9-a8bf-38d597553f58", "id": 2} v 0) v1 -- 0x7ff8a004b720 con 0x7ff8a009f000 2024-09-15T01:31:53.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.433+0000 7ff895ffb640 1 -- 172.21.15.204:0/1257102354 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "24b56e74-a222-40f9-a8bf-38d597553f58", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff89803cdc0 con 0x7ff8a009f000 2024-09-15T01:31:53.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.433+0000 7ff895ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "24b56e74-a222-40f9-a8bf-38d597553f58", "id": 2}] 2024-09-15T01:31:53.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.433+0000 7ff895ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:31:53.435 INFO:teuthology.orchestra.run.smithi204.stdout:2 2024-09-15T01:31:53.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.434+0000 7ff8a80c0640 10 monclient: shutdown 2024-09-15T01:31:53.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.434+0000 7ff8a80c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:53.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.434+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1257102354 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8a009f000 msgr2=0x7ff8a011d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.434+0000 7ff8a80c0640 1 --2- 172.21.15.204:0/1257102354 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8a009f000 0x7ff8a011d5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8980099f0 tx=0x7ff898037000 comp rx=0 tx=0).stop 2024-09-15T01:31:53.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.434+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1257102354 shutdown_connections 2024-09-15T01:31:53.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.434+0000 7ff8a80c0640 1 --2- 172.21.15.204:0/1257102354 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8a009f000 0x7ff8a011d5e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.434+0000 7ff8a80c0640 1 --2- 172.21.15.204:0/1257102354 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8a009d580 0x7ff8a011d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.434+0000 7ff8a80c0640 1 --2- 172.21.15.204:0/1257102354 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8a0067130 0x7ff8a0133550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.434+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1257102354 >> 172.21.15.204:0/1257102354 conn(0x7ff8a0068a60 msgr2=0x7ff8a00a0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:53.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.434+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1257102354 shutdown_connections 2024-09-15T01:31:53.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.434+0000 7ff8a80c0640 1 -- 172.21.15.204:0/1257102354 wait complete. 2024-09-15T01:31:53.446 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new a5bf3f7d-4c59-470a-8f69-3fea018c13f1 3 2024-09-15T01:31:53.539 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.537+0000 7fa2dc020640 10 monclient: get_monmap_and_config 2024-09-15T01:31:53.539 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.537+0000 7fa2dc020640 10 monclient: build_initial_monmap 2024-09-15T01:31:53.539 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.537+0000 7fa2dc020640 10 monclient: monmap: 2024-09-15T01:31:53.540 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 0 2024-09-15T01:31:53.540 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.540 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:53.538823+0000 2024-09-15T01:31:53.540 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:53.538823+0000 2024-09-15T01:31:53.540 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:53.540 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.540 INFO:teuthology.orchestra.run.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:31:53.540 INFO:teuthology.orchestra.run.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:31:53.540 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:53.540 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:53.540 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:53.540 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:53.541 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:53.541 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:53.541 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:53.541 INFO:teuthology.orchestra.run.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:31:53.541 INFO:teuthology.orchestra.run.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:31:53.541 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.537+0000 7fa2dc020640 1 Processor -- start 2024-09-15T01:31:53.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 1 -- start start 2024-09-15T01:31:53.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 10 monclient: init 2024-09-15T01:31:53.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:53.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 10 monclient: _add_conns ranks=[4,5,6,8,3,10,2,9,7,1,0] 2024-09-15T01:31:53.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2d40678d0 0x7fa2d409e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 10 monclient(hunting): picked mon.noname-c con 0x7fa2d40678d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:31:53.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2d406ea30 0x7fa2d409ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 10 monclient(hunting): picked mon.noname-h con 0x7fa2d406ea30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2d409f180 0x7fa2d41306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 10 monclient(hunting): picked mon.noname-d con 0x7fa2d409f180 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d409f570 con 0x7fa2d40678d0 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d406d5b0 con 0x7fa2d406ea30 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d4067ca0 con 0x7fa2d409f180 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2dc020640 10 monclient(hunting): authenticate will time out at 729.181625s 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d95bb640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2d406ea30 0x7fa2d409ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d95bb640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2d406ea30 0x7fa2d409ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:33826/0 (socket says 172.21.15.204:33826) 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d95bb640 1 -- 172.21.15.204:0/428846174 learned_addr learned my addr 172.21.15.204:0/428846174 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d95bb640 10 monclient(hunting): get_auth_request con 0x7fa2d406ea30 auth_method 0 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d95bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d95bb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d95bb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d95bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d95bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d95bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d9dbc640 1 --2- 172.21.15.204:0/428846174 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2d40678d0 0x7fa2d409e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2da5bd640 1 --2- 172.21.15.204:0/428846174 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2d409f180 0x7fa2d41306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d9dbc640 10 monclient(hunting): get_auth_request con 0x7fa2d40678d0 auth_method 0 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d9dbc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d9dbc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.551 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d9dbc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2da5bd640 10 monclient(hunting): get_auth_request con 0x7fa2d409f180 auth_method 0 2024-09-15T01:31:53.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2da5bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2da5bd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2da5bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.538+0000 7fa2d95bb640 10 monclient(hunting): handle_auth_done global_id 4127 payload 274 2024-09-15T01:31:53.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d95bb640 1 -- 172.21.15.204:0/428846174 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2d409f180 msgr2=0x7fa2d41306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:53.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d95bb640 1 --2- 172.21.15.204:0/428846174 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2d409f180 0x7fa2d41306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d95bb640 1 -- 172.21.15.204:0/428846174 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2d40678d0 msgr2=0x7fa2d409e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:53.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d95bb640 1 --2- 172.21.15.204:0/428846174 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2d40678d0 0x7fa2d409e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d95bb640 10 monclient: _finish_hunting 0 2024-09-15T01:31:53.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d95bb640 1 monclient: found mon.noname-h 2024-09-15T01:31:53.552 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d95bb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:53.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d95bb640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d95bb640 1 -- 172.21.15.204:0/428846174 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d404bf60 con 0x7fa2d406ea30 2024-09-15T01:31:53.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d9dbc640 1 --2- 172.21.15.204:0/428846174 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2d40678d0 0x7fa2d409e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:31:53.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d95bb640 1 --2- 172.21.15.204:0/428846174 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2d406ea30 0x7fa2d409ec40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009940 tx=0x7fa2c40091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=9d603352b814bbf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:53.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d8dba640 1 -- 172.21.15.204:0/428846174 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2c402ad00 con 0x7fa2d406ea30 2024-09-15T01:31:53.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d8dba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d8dba640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:31:53.553 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d8dba640 10 monclient: dump: 2024-09-15T01:31:53.553 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.553 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.556 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.556 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.556 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.556 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.556 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.556 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.556 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.556 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.556 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.556 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.556 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.557 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.557 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.557 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.557 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d8dba640 10 monclient: _finish_auth 0 2024-09-15T01:31:53.557 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d8dba640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:53.557 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d8dba640 1 -- 172.21.15.204:0/428846174 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c402ae80 con 0x7fa2d406ea30 2024-09-15T01:31:53.557 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2dc020640 5 monclient: authenticate success, global_id 4127 2024-09-15T01:31:53.557 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2dc020640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:31:53.557 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.539+0000 7fa2d8dba640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:53.557 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 10 monclient: get_monmap_and_config success 2024-09-15T01:31:53.557 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2d8dba640 1 -- 172.21.15.204:0/428846174 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2c4031ca0 con 0x7fa2d406ea30 2024-09-15T01:31:53.557 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2d8dba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 4 set_mon_vals no callback set 2024-09-15T01:31:53.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2d8dba640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:31:53.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2d8dba640 10 monclient: dump: 2024-09-15T01:31:53.558 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.558 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.558 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.558 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.558 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.558 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.558 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 10 monclient: shutdown 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 1 -- 172.21.15.204:0/428846174 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2d406ea30 msgr2=0x7fa2d409ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.559 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 1 --2- 172.21.15.204:0/428846174 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2d406ea30 0x7fa2d409ec40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009940 tx=0x7fa2c40091d0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.560 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 1 -- 172.21.15.204:0/428846174 shutdown_connections 2024-09-15T01:31:53.560 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 1 --2- 172.21.15.204:0/428846174 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2d409f180 0x7fa2d41306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.560 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 1 --2- 172.21.15.204:0/428846174 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2d406ea30 0x7fa2d409ec40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.560 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 1 --2- 172.21.15.204:0/428846174 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2d40678d0 0x7fa2d409e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.560 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 1 -- 172.21.15.204:0/428846174 >> 172.21.15.204:0/428846174 conn(0x7fa2d40a4390 msgr2=0x7fa2d4067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:53.560 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 1 -- 172.21.15.204:0/428846174 shutdown_connections 2024-09-15T01:31:53.560 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.540+0000 7fa2dc020640 1 -- 172.21.15.204:0/428846174 wait complete. 2024-09-15T01:31:53.560 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.541+0000 7fa2dc020640 10 monclient: build_initial_monmap 2024-09-15T01:31:53.560 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.541+0000 7fa2dc020640 10 monclient: monmap: 2024-09-15T01:31:53.560 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 0 2024-09-15T01:31:53.560 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:31:53.560 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 0.000000 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr:created 0.000000 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 1 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:53.561 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.541+0000 7fa2dc020640 1 Processor -- start 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.541+0000 7fa2dc020640 1 -- start start 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.541+0000 7fa2dc020640 10 monclient: init 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 10 monclient: _add_conns ranks=[6,7,2,4,0,5,8,3,1] 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2d40678d0 0x7fa2d4133530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 10 monclient(hunting): picked mon.noname-d con 0x7fa2d40678d0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa2d406ea30 0x7fa2d411d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 10 monclient(hunting): picked mon.noname-i con 0x7fa2d406ea30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2d409f180 0x7fa2d411d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 10 monclient(hunting): picked mon.noname-b con 0x7fa2d409f180 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d411f0a0 con 0x7fa2d409f180 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d4140640 con 0x7fa2d40678d0 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d406ccd0 con 0x7fa2d406ea30 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2dc020640 10 monclient(hunting): authenticate will time out at 729.185145s 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa2d406ea30 0x7fa2d411d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa2d406ea30 0x7fa2d411d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:39730/0 (socket says 172.21.15.204:39730) 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 1 -- 172.21.15.204:0/2014376402 learned_addr learned my addr 172.21.15.204:0/2014376402 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 10 monclient(hunting): get_auth_request con 0x7fa2d406ea30 auth_method 0 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d9dbc640 1 --2- 172.21.15.204:0/2014376402 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2d40678d0 0x7fa2d4133530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d9dbc640 10 monclient(hunting): get_auth_request con 0x7fa2d40678d0 auth_method 0 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d9dbc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d9dbc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d9dbc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2da5bd640 1 --2- 172.21.15.204:0/2014376402 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2d409f180 0x7fa2d411d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d9dbc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d9dbc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d9dbc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2da5bd640 10 monclient(hunting): get_auth_request con 0x7fa2d409f180 auth_method 0 2024-09-15T01:31:53.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2da5bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2da5bd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2da5bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2da5bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2da5bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2da5bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 10 monclient(hunting): handle_auth_done global_id 4147 payload 824 2024-09-15T01:31:53.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 1 -- 172.21.15.204:0/2014376402 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2d40678d0 msgr2=0x7fa2d4133530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 1 --2- 172.21.15.204:0/2014376402 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2d40678d0 0x7fa2d4133530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 1 -- 172.21.15.204:0/2014376402 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2d409f180 msgr2=0x7fa2d411d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 1 --2- 172.21.15.204:0/2014376402 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2d409f180 0x7fa2d411d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.565 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 10 monclient: _finish_hunting 0 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 1 monclient: found mon.noname-i 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 1 -- 172.21.15.204:0/2014376402 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c4009450 con 0x7fa2d406ea30 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2d95bb640 1 --2- 172.21.15.204:0/2014376402 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa2d406ea30 0x7fa2d411d0d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40097e0 tx=0x7fa2c4032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2c9ffb640 1 -- 172.21.15.204:0/2014376402 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2c4006b30 con 0x7fa2d406ea30 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2c9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2c9ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.542+0000 7fa2c9ffb640 10 monclient: dump: 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.566 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2c9ffb640 10 monclient: _finish_auth 0 2024-09-15T01:31:53.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2c9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2dc020640 5 monclient: authenticate success, global_id 4147 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2c9ffb640 1 -- 172.21.15.204:0/2014376402 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c402ad00 con 0x7fa2d406ea30 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2dc020640 10 monclient: _renew_subs 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2dc020640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2dc020640 1 -- 172.21.15.204:0/2014376402 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d406c900 con 0x7fa2d406ea30 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2c9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2c9ffb640 1 -- 172.21.15.204:0/2014376402 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2c4032e20 con 0x7fa2d406ea30 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2c9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2c9ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2c9ffb640 10 monclient: dump: 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.568 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2cbfff640 4 set_mon_vals no callback set 2024-09-15T01:31:53.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2dc020640 10 monclient: _renew_subs 2024-09-15T01:31:53.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2dc020640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:31:53.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2dc020640 1 -- 172.21.15.204:0/2014376402 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d4120650 con 0x7fa2d406ea30 2024-09-15T01:31:53.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2c9ffb640 1 -- 172.21.15.204:0/2014376402 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fa2c4031cf0 con 0x7fa2d406ea30 2024-09-15T01:31:53.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.543+0000 7fa2c9ffb640 1 -- 172.21.15.204:0/2014376402 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fa2c403c400 con 0x7fa2d406ea30 2024-09-15T01:31:53.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.544+0000 7fa2dc020640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.544+0000 7fa2dc020640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.544+0000 7fa2dc020640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:31:53.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.544+0000 7fa2dc020640 1 -- 172.21.15.204:0/2014376402 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d405ed70 con 0x7fa2d406ea30 2024-09-15T01:31:53.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.546+0000 7fa2c9ffb640 1 -- 172.21.15.204:0/2014376402 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fa2c4037070 con 0x7fa2d406ea30 2024-09-15T01:31:53.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.546+0000 7fa2c9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.546+0000 7fa2c9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:31:53.615 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.613+0000 7fa2dc020640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "a5bf3f7d-4c59-470a-8f69-3fea018c13f1", "id": 3}] 2024-09-15T01:31:53.615 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.613+0000 7fa2dc020640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "a5bf3f7d-4c59-470a-8f69-3fea018c13f1", "id": 3}] 2024-09-15T01:31:53.615 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.613+0000 7fa2dc020640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:31:53.615 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.613+0000 7fa2dc020640 1 -- 172.21.15.204:0/2014376402 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd new", "uuid": "a5bf3f7d-4c59-470a-8f69-3fea018c13f1", "id": 3} v 0) v1 -- 0x7fa2d404b720 con 0x7fa2d406ea30 2024-09-15T01:31:53.619 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.617+0000 7fa2c9ffb640 1 -- 172.21.15.204:0/2014376402 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a5bf3f7d-4c59-470a-8f69-3fea018c13f1", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa2c4044450 con 0x7fa2d406ea30 2024-09-15T01:31:53.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.617+0000 7fa2c9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "a5bf3f7d-4c59-470a-8f69-3fea018c13f1", "id": 3}] 2024-09-15T01:31:53.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.617+0000 7fa2c9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:31:53.620 INFO:teuthology.orchestra.run.smithi204.stdout:3 2024-09-15T01:31:53.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.618+0000 7fa2dc020640 10 monclient: shutdown 2024-09-15T01:31:53.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.618+0000 7fa2dc020640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:53.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.618+0000 7fa2dc020640 1 -- 172.21.15.204:0/2014376402 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa2d406ea30 msgr2=0x7fa2d411d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.618+0000 7fa2dc020640 1 --2- 172.21.15.204:0/2014376402 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa2d406ea30 0x7fa2d411d0d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40097e0 tx=0x7fa2c4032000 comp rx=0 tx=0).stop 2024-09-15T01:31:53.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.618+0000 7fa2dc020640 1 -- 172.21.15.204:0/2014376402 shutdown_connections 2024-09-15T01:31:53.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.618+0000 7fa2dc020640 1 --2- 172.21.15.204:0/2014376402 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2d409f180 0x7fa2d411d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.618+0000 7fa2dc020640 1 --2- 172.21.15.204:0/2014376402 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa2d406ea30 0x7fa2d411d0d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.618+0000 7fa2dc020640 1 --2- 172.21.15.204:0/2014376402 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2d40678d0 0x7fa2d4133530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.618+0000 7fa2dc020640 1 -- 172.21.15.204:0/2014376402 >> 172.21.15.204:0/2014376402 conn(0x7fa2d40a4390 msgr2=0x7fa2d409d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:53.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.618+0000 7fa2dc020640 1 -- 172.21.15.204:0/2014376402 shutdown_connections 2024-09-15T01:31:53.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.618+0000 7fa2dc020640 1 -- 172.21.15.204:0/2014376402 wait complete. 2024-09-15T01:31:53.630 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new cdaf5723-3a5c-4238-a8e6-38f2325ccbfb 4 2024-09-15T01:31:53.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.724+0000 7f0b9999d640 10 monclient: get_monmap_and_config 2024-09-15T01:31:53.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.724+0000 7f0b9999d640 10 monclient: build_initial_monmap 2024-09-15T01:31:53.727 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.724+0000 7f0b9999d640 10 monclient: monmap: 2024-09-15T01:31:53.727 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 0 2024-09-15T01:31:53.727 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.727 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:53.725544+0000 2024-09-15T01:31:53.727 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:53.725544+0000 2024-09-15T01:31:53.727 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:53.727 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.727 INFO:teuthology.orchestra.run.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.724+0000 7f0b9999d640 1 Processor -- start 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.724+0000 7f0b9999d640 1 -- start start 2024-09-15T01:31:53.728 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.724+0000 7f0b9999d640 10 monclient: init 2024-09-15T01:31:53.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:53.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 10 monclient: _add_conns ranks=[2,9,0,10,3,5,7,6,4,1,8] 2024-09-15T01:31:53.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b94098c40 0x7f0b9409ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 10 monclient(hunting): picked mon.noname-b con 0x7f0b94098c40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:31:53.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0b9409dfd0 0x7f0b9409d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 10 monclient(hunting): picked mon.noname-a con 0x7f0b9409dfd0 addr v2:172.21.15.178:3300/0 2024-09-15T01:31:53.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0b9409d750 addr v1:172.21.15.178:6789/0 2024-09-15T01:31:53.736 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9404bf60 con 0x7f0b9409d750 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b94099e80 con 0x7f0b9409dfd0 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b94099b30 con 0x7f0b94098c40 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b9999d640 10 monclient(hunting): authenticate will time out at 729.368184s 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b92ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b94098c40 0x7f0b9409ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0b9409dfd0 0x7f0b9409d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0b9409dfd0 0x7f0b9409d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:33582/0 (socket says 172.21.15.204:33582) 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b92ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b94098c40 0x7f0b9409ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:47122/0 (socket says 172.21.15.204:47122) 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 1 -- 172.21.15.204:0/1397593952 learned_addr learned my addr 172.21.15.204:0/1397593952 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 10 monclient(hunting): get_auth_request con 0x7f0b9409dfd0 auth_method 0 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b92ffd640 10 monclient(hunting): get_auth_request con 0x7f0b94098c40 auth_method 0 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b92ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b92ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b92ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b92ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b92ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 10 monclient(hunting): handle_auth_done global_id 4122 payload 274 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 1 -- 172.21.15.204:0/1397593952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b94098c40 msgr2=0x7f0b9409ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 1 --2- 172.21.15.204:0/1397593952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b94098c40 0x7f0b9409ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 1 -- 172.21.15.204:0/1397593952 >> v1:172.21.15.178:6789/0 conn(0x7f0b9409d750 legacy=0x7f0b94130ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 10 monclient: _finish_hunting 0 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 1 monclient: found mon.noname-a 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b927fc640 1 -- 172.21.15.204:0/1397593952 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b94063620 con 0x7f0b9409dfd0 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.725+0000 7f0b92ffd640 1 --2- 172.21.15.204:0/1397593952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b94098c40 0x7f0b9409ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b927fc640 1 --2- 172.21.15.204:0/1397593952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9409dfd0 0x7f0b9409d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009940 tx=0x7f0b840091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf47e8bf96e658e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b91ffb640 1 -- 172.21.15.204:0/1397593952 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b8402ad20 con 0x7f0b9409dfd0 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b91ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b91ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b91ffb640 10 monclient: dump: 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.739 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b91ffb640 10 monclient: _finish_auth 0 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b91ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b91ffb640 1 -- 172.21.15.204:0/1397593952 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b84031820 con 0x7f0b9409dfd0 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b9999d640 5 monclient: authenticate success, global_id 4122 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b9999d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b91ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b9999d640 10 monclient: get_monmap_and_config success 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b91ffb640 1 -- 172.21.15.204:0/1397593952 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b8402aea0 con 0x7f0b9409dfd0 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b9999d640 4 set_mon_vals no callback set 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b91ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b91ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.726+0000 7f0b91ffb640 10 monclient: dump: 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.741 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.727+0000 7f0b9999d640 10 monclient: shutdown 2024-09-15T01:31:53.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.727+0000 7f0b9999d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.727+0000 7f0b9999d640 1 -- 172.21.15.204:0/1397593952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9409dfd0 msgr2=0x7f0b9409d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.727+0000 7f0b9999d640 1 --2- 172.21.15.204:0/1397593952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9409dfd0 0x7f0b9409d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009940 tx=0x7f0b840091d0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.727+0000 7f0b9999d640 1 -- 172.21.15.204:0/1397593952 shutdown_connections 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.727+0000 7f0b9999d640 1 --2- 172.21.15.204:0/1397593952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9409dfd0 0x7f0b9409d210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.727+0000 7f0b9999d640 1 --2- 172.21.15.204:0/1397593952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b94098c40 0x7f0b9409ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.727+0000 7f0b9999d640 1 -- 172.21.15.204:0/1397593952 >> 172.21.15.204:0/1397593952 conn(0x7f0b940a22e0 msgr2=0x7f0b940a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.727+0000 7f0b9999d640 1 -- 172.21.15.204:0/1397593952 shutdown_connections 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.727+0000 7f0b9999d640 1 -- 172.21.15.204:0/1397593952 wait complete. 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient: build_initial_monmap 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient: monmap: 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 0 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 0.000000 2024-09-15T01:31:53.743 INFO:teuthology.orchestra.run.smithi204.stderr:created 0.000000 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 1 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 1 Processor -- start 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 1 -- start start 2024-09-15T01:31:53.744 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient: init 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient: _add_conns ranks=[6,5,2,7,8,3,1,0,4] 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b94098c40 0x7f0b9413fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient(hunting): picked mon.noname-d con 0x7f0b94098c40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b9409d750 0x7f0b94140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient(hunting): picked mon.noname-h con 0x7f0b9409d750 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b9409dfd0 0x7f0b94140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient(hunting): picked mon.noname-b con 0x7f0b9409dfd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9409bd50 con 0x7f0b9409dfd0 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9409e660 con 0x7f0b9409d750 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0b94121c80 con 0x7f0b94098c40 2024-09-15T01:31:53.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.728+0000 7f0b9999d640 10 monclient(hunting): authenticate will time out at 729.371947s 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b9409d750 0x7f0b94140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b9409d750 0x7f0b94140300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:33834/0 (socket says 172.21.15.204:33834) 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 1 -- 172.21.15.204:0/1857912379 learned_addr learned my addr 172.21.15.204:0/1857912379 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 10 monclient(hunting): get_auth_request con 0x7f0b9409d750 auth_method 0 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b92ffd640 1 --2- 172.21.15.204:0/1857912379 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b94098c40 0x7f0b9413fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b937fe640 1 --2- 172.21.15.204:0/1857912379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b9409dfd0 0x7f0b94140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b92ffd640 10 monclient(hunting): get_auth_request con 0x7f0b94098c40 auth_method 0 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b92ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b92ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b937fe640 10 monclient(hunting): get_auth_request con 0x7f0b9409dfd0 auth_method 0 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b937fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b92ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b92ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b92ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.747 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 10 monclient(hunting): handle_auth_done global_id 4136 payload 824 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 1 -- 172.21.15.204:0/1857912379 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b94098c40 msgr2=0x7f0b9413fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 1 --2- 172.21.15.204:0/1857912379 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b94098c40 0x7f0b9413fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 1 -- 172.21.15.204:0/1857912379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b9409dfd0 msgr2=0x7f0b94140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 1 --2- 172.21.15.204:0/1857912379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b9409dfd0 0x7f0b94140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 10 monclient: _finish_hunting 0 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 1 monclient: found mon.noname-h 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 1 -- 172.21.15.204:0/1857912379 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b88009390 con 0x7f0b9409d750 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b927fc640 1 --2- 172.21.15.204:0/1857912379 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b9409d750 0x7f0b94140300 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0b840097e0 tx=0x7f0b84032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 1 -- 172.21.15.204:0/1857912379 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b84033cb0 con 0x7f0b9409d750 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:31:53.748 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 10 monclient: dump: 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.749 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 10 monclient: _finish_auth 0 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 1 -- 172.21.15.204:0/1857912379 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b84033e50 con 0x7f0b9409d750 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b9999d640 5 monclient: authenticate success, global_id 4136 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b9999d640 10 monclient: _renew_subs 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b9999d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b9999d640 1 -- 172.21.15.204:0/1857912379 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b84009450 con 0x7f0b9409d750 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 1 -- 172.21.15.204:0/1857912379 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b8403a810 con 0x7f0b9409d750 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 10 monclient: dump: 2024-09-15T01:31:53.750 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.751 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b9999d640 10 monclient: _renew_subs 2024-09-15T01:31:53.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b9999d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b9999d640 1 -- 172.21.15.204:0/1857912379 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9411f3d0 con 0x7f0b9409d750 2024-09-15T01:31:53.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b917fa640 4 set_mon_vals no callback set 2024-09-15T01:31:53.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 1 -- 172.21.15.204:0/1857912379 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f0b84039040 con 0x7f0b9409d750 2024-09-15T01:31:53.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.729+0000 7f0b7affd640 1 -- 172.21.15.204:0/1857912379 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f0b84033830 con 0x7f0b9409d750 2024-09-15T01:31:53.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.730+0000 7f0b9999d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.730+0000 7f0b9999d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.730+0000 7f0b9999d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.730+0000 7f0b9999d640 1 -- 172.21.15.204:0/1857912379 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9404b720 con 0x7f0b9409d750 2024-09-15T01:31:53.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.732+0000 7f0b7affd640 1 -- 172.21.15.204:0/1857912379 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f0b8405b7a0 con 0x7f0b9409d750 2024-09-15T01:31:53.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.732+0000 7f0b7affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.732+0000 7f0b7affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:31:53.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.799+0000 7f0b9999d640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "cdaf5723-3a5c-4238-a8e6-38f2325ccbfb", "id": 4}] 2024-09-15T01:31:53.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.799+0000 7f0b9999d640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "cdaf5723-3a5c-4238-a8e6-38f2325ccbfb", "id": 4}] 2024-09-15T01:31:53.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.799+0000 7f0b9999d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:31:53.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.799+0000 7f0b9999d640 1 -- 172.21.15.204:0/1857912379 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd new", "uuid": "cdaf5723-3a5c-4238-a8e6-38f2325ccbfb", "id": 4} v 0) v1 -- 0x7f0b94070770 con 0x7f0b9409d750 2024-09-15T01:31:53.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.810+0000 7f0b7affd640 1 -- 172.21.15.204:0/1857912379 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cdaf5723-3a5c-4238-a8e6-38f2325ccbfb", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0b8404a070 con 0x7f0b9409d750 2024-09-15T01:31:53.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.810+0000 7f0b7affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "cdaf5723-3a5c-4238-a8e6-38f2325ccbfb", "id": 4}] 2024-09-15T01:31:53.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.810+0000 7f0b7affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:31:53.813 INFO:teuthology.orchestra.run.smithi204.stdout:4 2024-09-15T01:31:53.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.811+0000 7f0b9999d640 10 monclient: shutdown 2024-09-15T01:31:53.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.811+0000 7f0b9999d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:53.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.811+0000 7f0b9999d640 1 -- 172.21.15.204:0/1857912379 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b9409d750 msgr2=0x7f0b94140300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.811+0000 7f0b9999d640 1 --2- 172.21.15.204:0/1857912379 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b9409d750 0x7f0b94140300 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0b840097e0 tx=0x7f0b84032000 comp rx=0 tx=0).stop 2024-09-15T01:31:53.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.811+0000 7f0b9999d640 1 -- 172.21.15.204:0/1857912379 shutdown_connections 2024-09-15T01:31:53.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.811+0000 7f0b9999d640 1 --2- 172.21.15.204:0/1857912379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b9409dfd0 0x7f0b94140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.811+0000 7f0b9999d640 1 --2- 172.21.15.204:0/1857912379 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b9409d750 0x7f0b94140300 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.815 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.811+0000 7f0b9999d640 1 --2- 172.21.15.204:0/1857912379 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b94098c40 0x7f0b9413fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.815 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.811+0000 7f0b9999d640 1 -- 172.21.15.204:0/1857912379 >> 172.21.15.204:0/1857912379 conn(0x7f0b940a22e0 msgr2=0x7f0b9406de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:53.815 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.811+0000 7f0b9999d640 1 -- 172.21.15.204:0/1857912379 shutdown_connections 2024-09-15T01:31:53.815 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.811+0000 7f0b9999d640 1 -- 172.21.15.204:0/1857912379 wait complete. 2024-09-15T01:31:53.824 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 6b67035f-8718-4762-98bf-0dd0aa86e6e8 5 2024-09-15T01:31:53.918 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.915+0000 7fcffef9d640 10 monclient: get_monmap_and_config 2024-09-15T01:31:53.918 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.915+0000 7fcffef9d640 10 monclient: build_initial_monmap 2024-09-15T01:31:53.918 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.915+0000 7fcffef9d640 10 monclient: monmap: 2024-09-15T01:31:53.918 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 0 2024-09-15T01:31:53.918 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.918 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:53.917242+0000 2024-09-15T01:31:53.918 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:53.917242+0000 2024-09-15T01:31:53.918 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:53.918 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.919 INFO:teuthology.orchestra.run.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:31:53.919 INFO:teuthology.orchestra.run.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:31:53.919 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:53.919 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:53.919 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:53.919 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:53.919 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:53.919 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:53.919 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:53.919 INFO:teuthology.orchestra.run.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:31:53.919 INFO:teuthology.orchestra.run.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:31:53.919 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.927 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.916+0000 7fcffef9d640 1 Processor -- start 2024-09-15T01:31:53.927 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.916+0000 7fcffef9d640 1 -- start start 2024-09-15T01:31:53.927 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.916+0000 7fcffef9d640 10 monclient: init 2024-09-15T01:31:53.927 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.916+0000 7fcffef9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:53.928 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.916+0000 7fcffef9d640 10 monclient: _add_conns ranks=[7,2,8,1,5,4,6,10,0,9,3] 2024-09-15T01:31:53.928 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.916+0000 7fcffef9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcff809c020 0x7fcff809ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.928 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.916+0000 7fcffef9d640 10 monclient(hunting): picked mon.noname-i con 0x7fcff809c020 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:31:53.928 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.916+0000 7fcffef9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff8067130 0x7fcff8067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.928 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.916+0000 7fcffef9d640 10 monclient(hunting): picked mon.noname-b con 0x7fcff8067130 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:31:53.928 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffef9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcff8067a40 0x7fcff81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.928 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffef9d640 10 monclient(hunting): picked mon.noname-e con 0x7fcff8067a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:31:53.928 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.928 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffef9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff809e220 con 0x7fcff8067130 2024-09-15T01:31:53.928 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.928 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffef9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcff809b270 con 0x7fcff809c020 2024-09-15T01:31:53.928 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffef9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcff809af90 con 0x7fcff8067a40 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffef9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffef9d640 10 monclient(hunting): authenticate will time out at 729.560032s 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcff809c020 0x7fcff809ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcff809c020 0x7fcff809ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:39746/0 (socket says 172.21.15.204:39746) 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 1 -- 172.21.15.204:0/1368751013 learned_addr learned my addr 172.21.15.204:0/1368751013 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 10 monclient(hunting): get_auth_request con 0x7fcff809c020 auth_method 0 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcff7fff640 1 --2- 172.21.15.204:0/1368751013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff8067130 0x7fcff8067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffd53a640 1 --2- 172.21.15.204:0/1368751013 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcff8067a40 0x7fcff81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 10 monclient(hunting): handle_auth_done global_id 4156 payload 274 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 1 -- 172.21.15.204:0/1368751013 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcff8067a40 msgr2=0x7fcff81393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 1 --2- 172.21.15.204:0/1368751013 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcff8067a40 0x7fcff81393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 1 -- 172.21.15.204:0/1368751013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff8067130 msgr2=0x7fcff8067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 1 --2- 172.21.15.204:0/1368751013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff8067130 0x7fcff8067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 10 monclient: _finish_hunting 0 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 1 monclient: found mon.noname-i 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcffcd39640 1 -- 172.21.15.204:0/1368751013 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff804bf60 con 0x7fcff809c020 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcff7fff640 10 monclient: get_auth_request con 0x7fcff8067130 auth_method 0 2024-09-15T01:31:53.930 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.917+0000 7fcff7fff640 1 --2- 172.21.15.204:0/1368751013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff8067130 0x7fcff8067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcffcd39640 1 --2- 172.21.15.204:0/1368751013 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcff809c020 0x7fcff809ea80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009940 tx=0x7fcfec0091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=7c8e5cf25ab8bd9b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcff77fe640 1 -- 172.21.15.204:0/1368751013 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfec02ac60 con 0x7fcff809c020 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcff77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcff77fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcff77fe640 10 monclient: dump: 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.931 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcff77fe640 10 monclient: _finish_auth 0 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcff77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcff77fe640 1 -- 172.21.15.204:0/1368751013 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec02ade0 con 0x7fcff809c020 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcffef9d640 5 monclient: authenticate success, global_id 4156 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcffef9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:31:53.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcff77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcffef9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcff77fe640 1 -- 172.21.15.204:0/1368751013 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfec031ca0 con 0x7fcff809c020 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcff77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcffef9d640 4 set_mon_vals no callback set 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcff77fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcff77fe640 10 monclient: dump: 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.933 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcffef9d640 10 monclient: shutdown 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcffef9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcffef9d640 1 -- 172.21.15.204:0/1368751013 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcff809c020 msgr2=0x7fcff809ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.918+0000 7fcffef9d640 1 --2- 172.21.15.204:0/1368751013 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcff809c020 0x7fcff809ea80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009940 tx=0x7fcfec0091d0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.919+0000 7fcffef9d640 1 -- 172.21.15.204:0/1368751013 shutdown_connections 2024-09-15T01:31:53.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.919+0000 7fcffef9d640 1 --2- 172.21.15.204:0/1368751013 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcff8067a40 0x7fcff81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.919+0000 7fcffef9d640 1 --2- 172.21.15.204:0/1368751013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcff8067130 0x7fcff8067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.919+0000 7fcffef9d640 1 --2- 172.21.15.204:0/1368751013 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcff809c020 0x7fcff809ea80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.919+0000 7fcffef9d640 1 -- 172.21.15.204:0/1368751013 >> 172.21.15.204:0/1368751013 conn(0x7fcff8068a60 msgr2=0x7fcff80a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.919+0000 7fcffef9d640 1 -- 172.21.15.204:0/1368751013 shutdown_connections 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.919+0000 7fcffef9d640 1 -- 172.21.15.204:0/1368751013 wait complete. 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.919+0000 7fcffef9d640 10 monclient: build_initial_monmap 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.919+0000 7fcffef9d640 10 monclient: monmap: 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 0 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 0.000000 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:created 0.000000 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:53.935 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 1 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.919+0000 7fcffef9d640 1 Processor -- start 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 1 -- start start 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 10 monclient: init 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:53.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 10 monclient: _add_conns ranks=[1,4,8,0,7,5,6,3,2] 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff8067130 0x7fcff8132450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 10 monclient(hunting): picked mon.noname-f con 0x7fcff8067130 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcff8067a40 0x7fcff811bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 10 monclient(hunting): picked mon.noname-c con 0x7fcff8067a40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcff809c020 0x7fcff811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 10 monclient(hunting): picked mon.noname-e con 0x7fcff809c020 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff806e750 con 0x7fcff8067130 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcff81427e0 con 0x7fcff8067a40 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcff811fb30 con 0x7fcff809c020 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:53.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffef9d640 10 monclient(hunting): authenticate will time out at 729.563507s 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff8067130 0x7fcff8132450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff8067130 0x7fcff8132450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39968/0 (socket says 172.21.15.204:39968) 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 1 -- 172.21.15.204:0/3589373910 learned_addr learned my addr 172.21.15.204:0/3589373910 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 10 monclient(hunting): get_auth_request con 0x7fcff8067130 auth_method 0 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffd53a640 1 --2- 172.21.15.204:0/3589373910 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcff809c020 0x7fcff811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffd53a640 10 monclient(hunting): get_auth_request con 0x7fcff809c020 auth_method 0 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffd53a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffd53a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffd53a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.938 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcff7fff640 1 --2- 172.21.15.204:0/3589373910 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcff8067a40 0x7fcff811bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcff7fff640 10 monclient(hunting): get_auth_request con 0x7fcff8067a40 auth_method 0 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcff7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcff7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcff7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffd53a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffd53a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffd53a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcff7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcff7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcff7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:53.939 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 10 monclient(hunting): handle_auth_done global_id 4114 payload 824 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 1 -- 172.21.15.204:0/3589373910 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcff809c020 msgr2=0x7fcff811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 1 --2- 172.21.15.204:0/3589373910 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcff809c020 0x7fcff811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 1 -- 172.21.15.204:0/3589373910 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcff8067a40 msgr2=0x7fcff811bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 1 --2- 172.21.15.204:0/3589373910 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcff8067a40 0x7fcff811bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 10 monclient: _finish_hunting 0 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 1 monclient: found mon.noname-f 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.920+0000 7fcffcd39640 1 -- 172.21.15.204:0/3589373910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec009450 con 0x7fcff8067130 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcffcd39640 1 --2- 172.21.15.204:0/3589373910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff8067130 0x7fcff8132450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcfec035040 tx=0x7fcfec032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 1 -- 172.21.15.204:0/3589373910 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfec031820 con 0x7fcff8067130 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.940 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 10 monclient: dump: 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.941 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 10 monclient: _finish_auth 0 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 1 -- 172.21.15.204:0/3589373910 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec0319e0 con 0x7fcff8067130 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcffef9d640 5 monclient: authenticate success, global_id 4114 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 1 -- 172.21.15.204:0/3589373910 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfec03a390 con 0x7fcff8067130 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcffef9d640 10 monclient: _renew_subs 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcffef9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:31:53.942 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcffef9d640 1 -- 172.21.15.204:0/3589373910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff811f850 con 0x7fcff8067130 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff6ffd640 4 set_mon_vals no callback set 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 10 monclient: dump: 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:epoch 1 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:election_strategy: 3 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:53.943 INFO:teuthology.orchestra.run.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:53.944 INFO:teuthology.orchestra.run.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:53.944 INFO:teuthology.orchestra.run.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:53.944 INFO:teuthology.orchestra.run.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:53.944 INFO:teuthology.orchestra.run.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:53.944 INFO:teuthology.orchestra.run.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:53.944 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:31:53.944 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcffef9d640 10 monclient: _renew_subs 2024-09-15T01:31:53.944 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcffef9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:31:53.944 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcffef9d640 1 -- 172.21.15.204:0/3589373910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff811e610 con 0x7fcff8067130 2024-09-15T01:31:53.944 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 1 -- 172.21.15.204:0/3589373910 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fcfec039040 con 0x7fcff8067130 2024-09-15T01:31:53.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.921+0000 7fcff4ff9640 1 -- 172.21.15.204:0/3589373910 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7fcfec031dd0 con 0x7fcff8067130 2024-09-15T01:31:53.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.922+0000 7fcffef9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.922+0000 7fcffef9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.922+0000 7fcffef9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:31:53.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.922+0000 7fcffef9d640 1 -- 172.21.15.204:0/3589373910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff804b720 con 0x7fcff8067130 2024-09-15T01:31:53.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.926+0000 7fcff4ff9640 1 -- 172.21.15.204:0/3589373910 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fcfec041070 con 0x7fcff8067130 2024-09-15T01:31:53.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.926+0000 7fcff4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:53.945 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.926+0000 7fcff4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:31:53.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.993+0000 7fcffef9d640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "6b67035f-8718-4762-98bf-0dd0aa86e6e8", "id": 5}] 2024-09-15T01:31:53.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.993+0000 7fcffef9d640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "6b67035f-8718-4762-98bf-0dd0aa86e6e8", "id": 5}] 2024-09-15T01:31:53.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.993+0000 7fcffef9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:31:53.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.993+0000 7fcffef9d640 1 -- 172.21.15.204:0/3589373910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6b67035f-8718-4762-98bf-0dd0aa86e6e8", "id": 5} v 0) v1 -- 0x7fcff8070770 con 0x7fcff8067130 2024-09-15T01:31:54.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.998+0000 7fcff4ff9640 1 -- 172.21.15.204:0/3589373910 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6b67035f-8718-4762-98bf-0dd0aa86e6e8", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcfec037070 con 0x7fcff8067130 2024-09-15T01:31:54.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.999+0000 7fcff4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "6b67035f-8718-4762-98bf-0dd0aa86e6e8", "id": 5}] 2024-09-15T01:31:54.001 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.999+0000 7fcff4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:31:54.001 INFO:teuthology.orchestra.run.smithi204.stdout:5 2024-09-15T01:31:54.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.999+0000 7fcffef9d640 10 monclient: shutdown 2024-09-15T01:31:54.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.999+0000 7fcffef9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:54.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.999+0000 7fcffef9d640 1 -- 172.21.15.204:0/3589373910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff8067130 msgr2=0x7fcff8132450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:54.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.999+0000 7fcffef9d640 1 --2- 172.21.15.204:0/3589373910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff8067130 0x7fcff8132450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcfec035040 tx=0x7fcfec032000 comp rx=0 tx=0).stop 2024-09-15T01:31:54.002 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.999+0000 7fcffef9d640 1 -- 172.21.15.204:0/3589373910 shutdown_connections 2024-09-15T01:31:54.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:53.999+0000 7fcffef9d640 1 --2- 172.21.15.204:0/3589373910 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcff809c020 0x7fcff811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:54.000+0000 7fcffef9d640 1 --2- 172.21.15.204:0/3589373910 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcff8067a40 0x7fcff811bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:54.000+0000 7fcffef9d640 1 --2- 172.21.15.204:0/3589373910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcff8067130 0x7fcff8132450 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:54.000+0000 7fcffef9d640 1 -- 172.21.15.204:0/3589373910 >> 172.21.15.204:0/3589373910 conn(0x7fcff8068a60 msgr2=0x7fcff80a1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:54.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:54.000+0000 7fcffef9d640 1 -- 172.21.15.204:0/3589373910 shutdown_connections 2024-09-15T01:31:54.003 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-15T01:31:54.000+0000 7fcffef9d640 1 -- 172.21.15.204:0/3589373910 wait complete. 2024-09-15T01:31:54.012 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T01:31:54.012 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T01:31:54.015 INFO:tasks.ceph.osd.0:Started 2024-09-15T01:31:54.015 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T01:31:54.015 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T01:31:54.017 INFO:tasks.ceph.osd.1:Started 2024-09-15T01:31:54.018 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T01:31:54.018 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T01:31:54.020 INFO:tasks.ceph.osd.2:Started 2024-09-15T01:31:54.020 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T01:31:54.020 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T01:31:54.022 INFO:tasks.ceph.osd.3:Started 2024-09-15T01:31:54.022 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T01:31:54.022 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T01:31:54.024 INFO:tasks.ceph.osd.4:Started 2024-09-15T01:31:54.024 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T01:31:54.024 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T01:31:54.026 INFO:tasks.ceph.osd.5:Started 2024-09-15T01:31:54.026 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:31:54.094 INFO:tasks.ceph.osd.3.smithi204.stderr:2024-09-15T01:31:54.091+0000 7feb19235440 -1 Falling back to public interface 2024-09-15T01:31:54.094 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-09-15T01:31:54.091+0000 7f061d609440 -1 Falling back to public interface 2024-09-15T01:31:54.094 INFO:tasks.ceph.osd.3.smithi204.stderr:2024-09-15T01:31:54.091+0000 7feb17bfd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-09-15T01:31:54.095 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-09-15T01:31:54.091+0000 7f061b5fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6802/0: (98) Address already in use 2024-09-15T01:31:54.099 INFO:tasks.ceph.osd.0.smithi178.stderr:2024-09-15T01:31:54.097+0000 7f0459186440 -1 Falling back to public interface 2024-09-15T01:31:54.100 INFO:tasks.ceph.osd.1.smithi178.stderr:2024-09-15T01:31:54.099+0000 7f6ef8924440 -1 Falling back to public interface 2024-09-15T01:31:54.102 INFO:tasks.ceph.osd.2.smithi178.stderr:2024-09-15T01:31:54.101+0000 7fb8a6b23440 -1 Falling back to public interface 2024-09-15T01:31:54.104 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-09-15T01:31:54.102+0000 7f061d609440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:54.104 INFO:tasks.ceph.osd.3.smithi204.stderr:2024-09-15T01:31:54.102+0000 7feb19235440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:54.104 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-09-15T01:31:54.102+0000 7f061d609440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:31:54.105 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-09-15T01:31:54.102+0000 7f061d609440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:31:54.105 INFO:tasks.ceph.osd.3.smithi204.stderr:2024-09-15T01:31:54.102+0000 7feb19235440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:31:54.105 INFO:tasks.ceph.osd.3.smithi204.stderr:2024-09-15T01:31:54.102+0000 7feb19235440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:31:54.107 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-09-15T01:31:54.105+0000 7f404aa12440 -1 Falling back to public interface 2024-09-15T01:31:54.109 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-09-15T01:31:54.107+0000 7f061d609440 -1 osd.4 0 log_to_monitors true 2024-09-15T01:31:54.110 INFO:tasks.ceph.osd.3.smithi204.stderr:2024-09-15T01:31:54.107+0000 7feb19235440 -1 osd.3 0 log_to_monitors true 2024-09-15T01:31:54.114 INFO:tasks.ceph.osd.2.smithi178.stderr:2024-09-15T01:31:54.112+0000 7fb8a6b23440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:54.114 INFO:tasks.ceph.osd.2.smithi178.stderr:2024-09-15T01:31:54.112+0000 7fb8a6b23440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:31:54.114 INFO:tasks.ceph.osd.2.smithi178.stderr:2024-09-15T01:31:54.112+0000 7fb8a6b23440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:31:54.116 INFO:tasks.ceph.osd.0.smithi178.stderr:2024-09-15T01:31:54.114+0000 7f0459186440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:54.116 INFO:tasks.ceph.osd.0.smithi178.stderr:2024-09-15T01:31:54.114+0000 7f0459186440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:31:54.116 INFO:tasks.ceph.osd.0.smithi178.stderr:2024-09-15T01:31:54.114+0000 7f0459186440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:31:54.117 INFO:tasks.ceph.osd.1.smithi178.stderr:2024-09-15T01:31:54.115+0000 7f6ef8924440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:54.117 INFO:tasks.ceph.osd.1.smithi178.stderr:2024-09-15T01:31:54.115+0000 7f6ef8924440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:31:54.117 INFO:tasks.ceph.osd.1.smithi178.stderr:2024-09-15T01:31:54.115+0000 7f6ef8924440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:31:54.117 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-09-15T01:31:54.115+0000 7f404aa12440 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2024-09-15T01:31:54.117 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-09-15T01:31:54.115+0000 7f404aa12440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:31:54.118 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-09-15T01:31:54.115+0000 7f404aa12440 -1 journal do_read_entry(8192): bad header magic 2024-09-15T01:31:54.119 INFO:tasks.ceph.osd.2.smithi178.stderr:2024-09-15T01:31:54.118+0000 7fb8a6b23440 -1 osd.2 0 log_to_monitors true 2024-09-15T01:31:54.122 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-09-15T01:31:54.120+0000 7f404aa12440 -1 osd.5 0 log_to_monitors true 2024-09-15T01:31:54.127 INFO:tasks.ceph.osd.0.smithi178.stderr:2024-09-15T01:31:54.126+0000 7f0459186440 -1 osd.0 0 log_to_monitors true 2024-09-15T01:31:54.128 INFO:tasks.ceph.osd.1.smithi178.stderr:2024-09-15T01:31:54.126+0000 7f6ef8924440 -1 osd.1 0 log_to_monitors true 2024-09-15T01:31:54.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.140+0000 7f79fc71d640 10 monclient: get_monmap_and_config 2024-09-15T01:31:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.140+0000 7f79fc71d640 10 monclient: build_initial_monmap 2024-09-15T01:31:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.140+0000 7f79fc71d640 10 monclient: monmap: 2024-09-15T01:31:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:31:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:54.141053+0000 2024-09-15T01:31:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:54.141053+0000 2024-09-15T01:31:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:31:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:31:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:31:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:31:54.143 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:31:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.140+0000 7f79fc71d640 1 Processor -- start 2024-09-15T01:31:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.140+0000 7f79fc71d640 1 -- start start 2024-09-15T01:31:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.140+0000 7f79fc71d640 10 monclient: init 2024-09-15T01:31:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 10 monclient: _add_conns ranks=[1,6,5,4,3,7,0,2,8,10,9] 2024-09-15T01:31:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f79f409c020 addr v1:172.21.15.204:6789/0 2024-09-15T01:31:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f41353a0 0x7f79f4137780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 10 monclient(hunting): picked mon.noname-d con 0x7f79f41353a0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:31:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79f4137cc0 0x7f79f413a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 10 monclient(hunting): picked mon.noname-h con 0x7f79f4137cc0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:31:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f404bf60 con 0x7f79f409c020 2024-09-15T01:31:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79f409e220 con 0x7f79f4137cc0 2024-09-15T01:31:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f41353a0 0x7f79f4137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f79f409b270 con 0x7f79f41353a0 2024-09-15T01:31:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79fc71d640 10 monclient(hunting): authenticate will time out at 718.891655s 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f41353a0 0x7f79f4137780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:36300/0 (socket says 172.21.15.178:36300) 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 1 -- 172.21.15.178:0/1562120350 learned_addr learned my addr 172.21.15.178:0/1562120350 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 10 monclient(hunting): get_auth_request con 0x7f79f41353a0 auth_method 0 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79facba640 1 --2- 172.21.15.178:0/1562120350 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79f4137cc0 0x7f79f413a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79facba640 10 monclient(hunting): get_auth_request con 0x7f79f4137cc0 auth_method 0 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79facba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79facba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79facba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79facba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79facba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79facba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 10 monclient(hunting): handle_auth_done global_id 4200 payload 274 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f94b7640 1 -- 172.21.15.178:0/1562120350 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891088542 0 0) 0x7f79f404bf60 con 0x7f79f409c020 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 1 -- 172.21.15.178:0/1562120350 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79f4137cc0 msgr2=0x7f79f413a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 1 --2- 172.21.15.178:0/1562120350 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79f4137cc0 0x7f79f413a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 1 -- 172.21.15.178:0/1562120350 >> v1:172.21.15.204:6789/0 conn(0x7f79f409c020 legacy=0x7f79f409ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 10 monclient: _finish_hunting 0 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 1 monclient: found mon.noname-d 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:31:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f9cb8640 1 -- 172.21.15.178:0/1562120350 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f409af90 con 0x7f79f41353a0 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.141+0000 7f79f94b7640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f9cb8640 1 --2- 172.21.15.178:0/1562120350 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f41353a0 0x7f79f4137780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009940 tx=0x7f79e80091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=1420d931a0a3731b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f94b7640 1 -- 172.21.15.178:0/1562120350 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79e802ad00 con 0x7f79f41353a0 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f94b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f94b7640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f94b7640 10 monclient: dump: 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:54.155 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f94b7640 10 monclient: _finish_auth 0 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f94b7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f94b7640 1 -- 172.21.15.178:0/1562120350 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e802ae80 con 0x7f79f41353a0 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79fc71d640 5 monclient: authenticate success, global_id 4200 2024-09-15T01:31:54.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79fc71d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f94b7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79fc71d640 10 monclient: get_monmap_and_config success 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f94b7640 1 -- 172.21.15.178:0/1562120350 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79e8031ca0 con 0x7f79f41353a0 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79fc71d640 4 set_mon_vals no callback set 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f94b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f94b7640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.142+0000 7f79f94b7640 10 monclient: dump: 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:54.157 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.143+0000 7f79fc71d640 10 monclient: shutdown 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.143+0000 7f79fc71d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.143+0000 7f79fc71d640 1 -- 172.21.15.178:0/1562120350 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f41353a0 msgr2=0x7f79f4137780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:54.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.143+0000 7f79fc71d640 1 --2- 172.21.15.178:0/1562120350 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f41353a0 0x7f79f4137780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009940 tx=0x7f79e80091d0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.143+0000 7f79fc71d640 1 -- 172.21.15.178:0/1562120350 shutdown_connections 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.143+0000 7f79fc71d640 1 --2- 172.21.15.178:0/1562120350 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79f4137cc0 0x7f79f413a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.143+0000 7f79fc71d640 1 --2- 172.21.15.178:0/1562120350 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f41353a0 0x7f79f4137780 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.143+0000 7f79fc71d640 1 -- 172.21.15.178:0/1562120350 >> 172.21.15.178:0/1562120350 conn(0x7f79f4068a60 msgr2=0x7f79f40a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.143+0000 7f79fc71d640 1 -- 172.21.15.178:0/1562120350 shutdown_connections 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.143+0000 7f79fc71d640 1 -- 172.21.15.178:0/1562120350 wait complete. 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient: build_initial_monmap 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient: monmap: 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:31:54.159 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 1 Processor -- start 2024-09-15T01:31:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 1 -- start start 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient: init 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient: _add_conns ranks=[0,7,6,2,8,4,5,1,3] 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79f409c020 0x7f79f406ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient(hunting): picked mon.noname-a con 0x7f79f409c020 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f79f41353a0 0x7f79f406cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient(hunting): picked mon.noname-i con 0x7f79f41353a0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f4137cc0 0x7f79f406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient(hunting): picked mon.noname-d con 0x7f79f4137cc0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f406e750 con 0x7f79f409c020 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f79f409f2d0 con 0x7f79f4137cc0 2024-09-15T01:31:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f79f411b100 con 0x7f79f41353a0 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.144+0000 7f79fc71d640 10 monclient(hunting): authenticate will time out at 718.894950s 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fa4b9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79f409c020 0x7f79f406ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79facba640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f4137cc0 0x7f79f406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fa4b9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79f409c020 0x7f79f406ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40540/0 (socket says 172.21.15.178:40540) 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f79f41353a0 0x7f79f406cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fa4b9640 1 -- 172.21.15.178:0/2270984423 learned_addr learned my addr 172.21.15.178:0/2270984423 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fa4b9640 10 monclient(hunting): get_auth_request con 0x7f79f409c020 auth_method 0 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fa4b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fa4b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fa4b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:54.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 10 monclient(hunting): get_auth_request con 0x7f79f41353a0 auth_method 0 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79facba640 10 monclient(hunting): get_auth_request con 0x7f79f4137cc0 auth_method 0 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79facba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79facba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79facba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fa4b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fa4b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:54.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fa4b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79facba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79facba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79facba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 10 monclient(hunting): handle_auth_done global_id 4183 payload 824 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 1 -- 172.21.15.178:0/2270984423 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f4137cc0 msgr2=0x7f79f406d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 1 --2- 172.21.15.178:0/2270984423 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f4137cc0 0x7f79f406d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 1 -- 172.21.15.178:0/2270984423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79f409c020 msgr2=0x7f79f406ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 1 --2- 172.21.15.178:0/2270984423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79f409c020 0x7f79f406ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 10 monclient: _finish_hunting 0 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 1 monclient: found mon.noname-i 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 1 -- 172.21.15.178:0/2270984423 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e8009450 con 0x7f79f41353a0 2024-09-15T01:31:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79facba640 1 --2- 172.21.15.178:0/2270984423 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f4137cc0 0x7f79f406d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f9cb8640 1 --2- 172.21.15.178:0/2270984423 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f79f41353a0 0x7f79f406cfd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f79e8031000 tx=0x7f79e8032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79e67fc640 1 -- 172.21.15.178:0/2270984423 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79e8006b30 con 0x7f79f41353a0 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79e67fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79e67fc640 10 monclient: dump: 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:54.165 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79e67fc640 10 monclient: _finish_auth 0 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fc71d640 5 monclient: authenticate success, global_id 4183 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79e67fc640 1 -- 172.21.15.178:0/2270984423 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e802ad00 con 0x7f79f41353a0 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fc71d640 10 monclient: _renew_subs 2024-09-15T01:31:54.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fc71d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79fc71d640 1 -- 172.21.15.178:0/2270984423 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f412ff00 con 0x7f79f41353a0 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79e67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79e67fc640 1 -- 172.21.15.178:0/2270984423 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79e803b820 con 0x7f79f41353a0 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79e67fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79e67fc640 10 monclient: dump: 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:54.167 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.145+0000 7f79f8cb6640 4 set_mon_vals no callback set 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.146+0000 7f79fc71d640 10 monclient: _renew_subs 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.146+0000 7f79fc71d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.146+0000 7f79fc71d640 1 -- 172.21.15.178:0/2270984423 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f4119360 con 0x7f79f41353a0 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.146+0000 7f79e67fc640 1 -- 172.21.15.178:0/2270984423 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f79e8031a40 con 0x7f79f41353a0 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.146+0000 7f79e67fc640 1 -- 172.21.15.178:0/2270984423 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f79e8031cf0 con 0x7f79f41353a0 2024-09-15T01:31:54.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.146+0000 7f79fc71d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:54.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.146+0000 7f79fc71d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:54.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.147+0000 7f79fc71d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:31:54.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.147+0000 7f79fc71d640 1 -- 172.21.15.178:0/2270984423 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f411b2b0 con 0x7f79f41353a0 2024-09-15T01:31:54.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.149+0000 7f79e67fc640 1 -- 172.21.15.178:0/2270984423 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f79f411b2b0 con 0x7f79f41353a0 2024-09-15T01:31:54.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.149+0000 7f79e67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:54.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.149+0000 7f79e67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:31:54.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.203+0000 7f79fc71d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:31:54.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.203+0000 7f79fc71d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:31:54.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.203+0000 7f79fc71d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:31:54.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.203+0000 7f79fc71d640 1 -- 172.21.15.178:0/2270984423 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79f404b720 con 0x7f79f41353a0 2024-09-15T01:31:54.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.203+0000 7f79e67fc640 1 -- 172.21.15.178:0/2270984423 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5560 (secure 0 0 0) 0x7f79e8037070 con 0x7f79f41353a0 2024-09-15T01:31:54.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.203+0000 7f79e67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:31:54.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.203+0000 7f79e67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:31:54.206 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:31:54.206 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":8,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","created":"2024-09-15T01:31:51.657329+0000","modified":"2024-09-15T01:31:53.995822+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T01:31:53.995822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"690d2bc0-3a5e-4f4a-b00b-43a73741b7b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bbba9997-deb4-42ca-8140-6553a9b6c45e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"24b56e74-a222-40f9-a8bf-38d597553f58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a5bf3f7d-4c59-470a-8f69-3fea018c13f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cdaf5723-3a5c-4238-a8e6-38f2325ccbfb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b67035f-8718-4762-98bf-0dd0aa86e6e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:31:54.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.204+0000 7f79fc71d640 10 monclient: shutdown 2024-09-15T01:31:54.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.204+0000 7f79fc71d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:54.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.204+0000 7f79fc71d640 1 -- 172.21.15.178:0/2270984423 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f79f41353a0 msgr2=0x7f79f406cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:54.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.204+0000 7f79fc71d640 1 --2- 172.21.15.178:0/2270984423 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f79f41353a0 0x7f79f406cfd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f79e8031000 tx=0x7f79e8032000 comp rx=0 tx=0).stop 2024-09-15T01:31:54.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.204+0000 7f79fc71d640 1 -- 172.21.15.178:0/2270984423 shutdown_connections 2024-09-15T01:31:54.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.204+0000 7f79fc71d640 1 --2- 172.21.15.178:0/2270984423 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79f4137cc0 0x7f79f406d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.204+0000 7f79fc71d640 1 --2- 172.21.15.178:0/2270984423 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f79f41353a0 0x7f79f406cfd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.204+0000 7f79fc71d640 1 --2- 172.21.15.178:0/2270984423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79f409c020 0x7f79f406ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.204+0000 7f79fc71d640 1 -- 172.21.15.178:0/2270984423 >> 172.21.15.178:0/2270984423 conn(0x7f79f4068a60 msgr2=0x7f79f40a0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:54.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.204+0000 7f79fc71d640 1 -- 172.21.15.178:0/2270984423 shutdown_connections 2024-09-15T01:31:54.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:31:54.204+0000 7f79fc71d640 1 -- 172.21.15.178:0/2270984423 wait complete. 2024-09-15T01:31:54.216 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T01:31:54.216 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T01:31:54.371 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:54.369+0000 7f269c3c2ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T01:31:54.420 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:54.418+0000 7f269c3c2ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T01:31:54.784 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:54.781+0000 7f269c3c2ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T01:31:54.820 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T01:31:54.865 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:54.863+0000 7f269c3c2ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T01:31:54.909 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.907+0000 7f9568f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:31:54.909 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.907+0000 7f9568f9d640 10 monclient: build_initial_monmap 2024-09-15T01:31:54.909 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.907+0000 7f9568f9d640 10 monclient: monmap: 2024-09-15T01:31:54.909 INFO:teuthology.misc.health.smithi178.stderr:epoch 0 2024-09-15T01:31:54.909 INFO:teuthology.misc.health.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:last_changed 2024-09-15T01:31:54.908248+0000 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:created 2024-09-15T01:31:54.908248+0000 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:54.910 INFO:teuthology.misc.health.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:31:54.911 INFO:teuthology.misc.health.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:31:54.911 INFO:teuthology.misc.health.smithi178.stderr: 2024-09-15T01:31:54.911 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.907+0000 7f9568f9d640 1 Processor -- start 2024-09-15T01:31:54.911 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.907+0000 7f9568f9d640 1 -- start start 2024-09-15T01:31:54.911 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.907+0000 7f9568f9d640 10 monclient: init 2024-09-15T01:31:54.919 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:54.919 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 10 monclient: _add_conns ranks=[8,9,0,3,2,7,6,1,10,4,5] 2024-09-15T01:31:54.919 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95640a2ae0 0x7f95640a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:54.919 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f95640a2ae0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:31:54.919 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f956409f8c0 0x7f9564132860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:54.919 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 10 monclient(hunting): picked mon.noname-a con 0x7f956409f8c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:31:54.919 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f956409fc90 addr v1:172.21.15.178:6789/0 2024-09-15T01:31:54.919 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f956404bf60 con 0x7f956409fc90 2024-09-15T01:31:54.919 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:54.919 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95640a1160 con 0x7f956409f8c0 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f956409e0b0 con 0x7f95640a2ae0 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9568f9d640 10 monclient(hunting): authenticate will time out at 719.658639s 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9561d9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f956409f8c0 0x7f9564132860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f956259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95640a2ae0 0x7f95640a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9561d9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f956409f8c0 0x7f9564132860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40550/0 (socket says 172.21.15.178:40550) 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f956259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95640a2ae0 0x7f95640a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60452/0 (socket says 172.21.15.178:60452) 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9561d9b640 1 -- 172.21.15.178:0/3152822765 learned_addr learned my addr 172.21.15.178:0/3152822765 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9561d9b640 10 monclient(hunting): get_auth_request con 0x7f956409f8c0 auth_method 0 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9561d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9561d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:54.920 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9561d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f956259c640 10 monclient(hunting): get_auth_request con 0x7f95640a2ae0 auth_method 0 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f956259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f956259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f956259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f956259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f956259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f956259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9561d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9561d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.908+0000 7f9561d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f9561d9b640 10 monclient(hunting): handle_auth_done global_id 4185 payload 274 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f9561d9b640 1 -- 172.21.15.178:0/3152822765 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95640a2ae0 msgr2=0x7f95640a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:31:54.921 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f9561d9b640 1 --2- 172.21.15.178:0/3152822765 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95640a2ae0 0x7f95640a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f9561d9b640 1 -- 172.21.15.178:0/3152822765 >> v1:172.21.15.178:6789/0 conn(0x7f956409fc90 legacy=0x7f9564132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f9561d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f9561d9b640 1 monclient: found mon.noname-a 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f9561d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f9561d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f9561d9b640 1 -- 172.21.15.178:0/3152822765 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956409db60 con 0x7f956409f8c0 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f956259c640 1 --2- 172.21.15.178:0/3152822765 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95640a2ae0 0x7f95640a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f9561d9b640 1 --2- 172.21.15.178:0/3152822765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f956409f8c0 0x7f9564132860 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f954c009910 tx=0x7f954c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa53978d22c3c26 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f956159a640 1 -- 172.21.15.178:0/3152822765 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f954c02ad20 con 0x7f956409f8c0 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f956159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f956159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f956159a640 10 monclient: dump: 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:epoch 1 2024-09-15T01:31:54.922 INFO:teuthology.misc.health.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:54.923 INFO:teuthology.misc.health.smithi178.stderr: 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f956159a640 10 monclient: _finish_auth 0 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f956159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f956159a640 1 -- 172.21.15.178:0/3152822765 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c031820 con 0x7f956409f8c0 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.909+0000 7f9568f9d640 5 monclient: authenticate success, global_id 4185 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f956159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f956159a640 1 -- 172.21.15.178:0/3152822765 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f954c02aea0 con 0x7f956409f8c0 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f956159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 4 set_mon_vals no callback set 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f956159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f956159a640 10 monclient: dump: 2024-09-15T01:31:54.924 INFO:teuthology.misc.health.smithi178.stderr:epoch 1 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:54.925 INFO:teuthology.misc.health.smithi178.stderr: 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 10 monclient: shutdown 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 1 -- 172.21.15.178:0/3152822765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f956409f8c0 msgr2=0x7f9564132860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 1 --2- 172.21.15.178:0/3152822765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f956409f8c0 0x7f9564132860 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f954c009910 tx=0x7f954c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 1 -- 172.21.15.178:0/3152822765 shutdown_connections 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 1 --2- 172.21.15.178:0/3152822765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f956409f8c0 0x7f9564132860 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 1 --2- 172.21.15.178:0/3152822765 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95640a2ae0 0x7f95640a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 1 -- 172.21.15.178:0/3152822765 >> 172.21.15.178:0/3152822765 conn(0x7f956406e7f0 msgr2=0x7f956406ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 1 -- 172.21.15.178:0/3152822765 shutdown_connections 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.910+0000 7f9568f9d640 1 -- 172.21.15.178:0/3152822765 wait complete. 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient: build_initial_monmap 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient: monmap: 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:epoch 0 2024-09-15T01:31:54.926 INFO:teuthology.misc.health.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:last_changed 0.000000 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:created 0.000000 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:election_strategy: 1 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:31:54.927 INFO:teuthology.misc.health.smithi178.stderr: 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 1 Processor -- start 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 1 -- start start 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient: init 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient: _add_conns ranks=[5,8,2,0,4,6,1,3,7] 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f956409f8c0 0x7f9564141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient(hunting): picked mon.noname-h con 0x7f956409f8c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f956409fc90 0x7f9564134ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f956409fc90 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95640a2ae0 0x7f956411e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f95640a2ae0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:54.928 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9564121bf0 con 0x7f95640a2ae0 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95641206b0 con 0x7f956409f8c0 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f956409ccb0 con 0x7f956409fc90 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9568f9d640 10 monclient(hunting): authenticate will time out at 719.661903s 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.911+0000 7f9561d9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f956409fc90 0x7f9564134ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f956409fc90 0x7f9564134ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60456/0 (socket says 172.21.15.178:60456) 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 1 -- 172.21.15.178:0/1798690861 learned_addr learned my addr 172.21.15.178:0/1798690861 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 10 monclient(hunting): get_auth_request con 0x7f956409fc90 auth_method 0 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9562d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95640a2ae0 0x7f956411e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:54.929 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9562d9d640 10 monclient(hunting): get_auth_request con 0x7f95640a2ae0 auth_method 0 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9562d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9562d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9562d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9562d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9562d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f956259c640 1 --2- 172.21.15.178:0/1798690861 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f956409f8c0 0x7f9564141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9562d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f956259c640 10 monclient(hunting): get_auth_request con 0x7f956409f8c0 auth_method 0 2024-09-15T01:31:54.930 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f956259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f956259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f956259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f956259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f956259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f956259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 10 monclient(hunting): handle_auth_done global_id 4211 payload 824 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 1 -- 172.21.15.178:0/1798690861 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f956409f8c0 msgr2=0x7f9564141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 1 --2- 172.21.15.178:0/1798690861 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f956409f8c0 0x7f9564141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 1 -- 172.21.15.178:0/1798690861 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95640a2ae0 msgr2=0x7f956411e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 1 --2- 172.21.15.178:0/1798690861 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95640a2ae0 0x7f956411e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 1 monclient: found mon.noname-e 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:31:54.931 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 1 -- 172.21.15.178:0/1798690861 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9554009390 con 0x7f956409fc90 2024-09-15T01:31:54.932 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9562d9d640 1 --2- 172.21.15.178:0/1798690861 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95640a2ae0 0x7f956411e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:31:54.932 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9561d9b640 1 --2- 172.21.15.178:0/1798690861 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f956409fc90 0x7f9564134ad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f954c0097e0 tx=0x7f954c032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:31:54.932 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 1 -- 172.21.15.178:0/1798690861 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f954c033d30 con 0x7f956409fc90 2024-09-15T01:31:54.932 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:54.932 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:31:54.932 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 10 monclient: dump: 2024-09-15T01:31:54.932 INFO:teuthology.misc.health.smithi178.stderr:epoch 1 2024-09-15T01:31:54.932 INFO:teuthology.misc.health.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:54.932 INFO:teuthology.misc.health.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.932 INFO:teuthology.misc.health.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.932 INFO:teuthology.misc.health.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:54.932 INFO:teuthology.misc.health.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr: 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:31:54.933 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 1 -- 172.21.15.178:0/1798690861 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c03b3f0 con 0x7f956409fc90 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9568f9d640 5 monclient: authenticate success, global_id 4211 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 1 -- 172.21.15.178:0/1798690861 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f954c03b9e0 con 0x7f956409fc90 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9568f9d640 10 monclient: _renew_subs 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9568f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9568f9d640 1 -- 172.21.15.178:0/1798690861 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c009450 con 0x7f956409fc90 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9560d99640 4 set_mon_vals no callback set 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 10 monclient: dump: 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:epoch 1 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:31:54.934 INFO:teuthology.misc.health.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr:election_strategy: 3 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr: 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9568f9d640 10 monclient: _renew_subs 2024-09-15T01:31:54.935 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9568f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:31:54.936 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f9568f9d640 1 -- 172.21.15.178:0/1798690861 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956409d330 con 0x7f956409fc90 2024-09-15T01:31:54.936 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 1 -- 172.21.15.178:0/1798690861 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f954c039040 con 0x7f956409fc90 2024-09-15T01:31:54.936 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.912+0000 7f955a7fc640 1 -- 172.21.15.178:0/1798690861 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f954c033a40 con 0x7f956409fc90 2024-09-15T01:31:54.936 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.913+0000 7f9568f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:54.936 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.913+0000 7f9568f9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:54.936 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.913+0000 7f9568f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:31:54.936 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.913+0000 7f9568f9d640 1 -- 172.21.15.178:0/1798690861 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956404b720 con 0x7f956409fc90 2024-09-15T01:31:54.936 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.915+0000 7f955a7fc640 1 -- 172.21.15.178:0/1798690861 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f954c042070 con 0x7f956409fc90 2024-09-15T01:31:54.936 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.915+0000 7f955a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:31:54.936 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.915+0000 7f955a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:31:54.956 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:54.953+0000 7f269c3c2ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T01:31:54.972 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.971+0000 7f9568f9d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:31:54.972 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.971+0000 7f9568f9d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:31:54.972 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.971+0000 7f9568f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:31:54.973 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.971+0000 7f9568f9d640 1 -- 172.21.15.178:0/1798690861 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9564070bd0 con 0x7f956409fc90 2024-09-15T01:31:54.973 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.971+0000 7f955a7fc640 1 -- 172.21.15.178:0/1798690861 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5560 (secure 0 0 0) 0x7f954c037070 con 0x7f956409fc90 2024-09-15T01:31:54.973 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.971+0000 7f955a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:31:54.973 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.971+0000 7f955a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:31:54.974 INFO:teuthology.misc.health.smithi178.stdout: 2024-09-15T01:31:54.974 INFO:teuthology.misc.health.smithi178.stdout:{"epoch":8,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","created":"2024-09-15T01:31:51.657329+0000","modified":"2024-09-15T01:31:53.995822+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T01:31:53.995822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"690d2bc0-3a5e-4f4a-b00b-43a73741b7b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bbba9997-deb4-42ca-8140-6553a9b6c45e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"24b56e74-a222-40f9-a8bf-38d597553f58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a5bf3f7d-4c59-470a-8f69-3fea018c13f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cdaf5723-3a5c-4238-a8e6-38f2325ccbfb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b67035f-8718-4762-98bf-0dd0aa86e6e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:31:54.975 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.972+0000 7f9568f9d640 10 monclient: shutdown 2024-09-15T01:31:54.975 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.972+0000 7f9568f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:31:54.975 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.972+0000 7f9568f9d640 1 -- 172.21.15.178:0/1798690861 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f956409fc90 msgr2=0x7f9564134ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:31:54.975 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.972+0000 7f9568f9d640 1 --2- 172.21.15.178:0/1798690861 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f956409fc90 0x7f9564134ad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f954c0097e0 tx=0x7f954c032000 comp rx=0 tx=0).stop 2024-09-15T01:31:54.975 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.972+0000 7f9568f9d640 1 -- 172.21.15.178:0/1798690861 shutdown_connections 2024-09-15T01:31:54.975 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.972+0000 7f9568f9d640 1 --2- 172.21.15.178:0/1798690861 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95640a2ae0 0x7f956411e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.975 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.972+0000 7f9568f9d640 1 --2- 172.21.15.178:0/1798690861 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f956409fc90 0x7f9564134ad0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.975 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.972+0000 7f9568f9d640 1 --2- 172.21.15.178:0/1798690861 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f956409f8c0 0x7f9564141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:31:54.975 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.972+0000 7f9568f9d640 1 -- 172.21.15.178:0/1798690861 >> 172.21.15.178:0/1798690861 conn(0x7f956406e7f0 msgr2=0x7f95640a46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:31:54.975 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.972+0000 7f9568f9d640 1 -- 172.21.15.178:0/1798690861 shutdown_connections 2024-09-15T01:31:54.975 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:31:54.972+0000 7f9568f9d640 1 -- 172.21.15.178:0/1798690861 wait complete. 2024-09-15T01:31:54.982 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-15T01:31:55.001 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:54.999+0000 7f269c3c2ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T01:31:55.097 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:55.094+0000 7f269c3c2ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T01:31:55.149 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:55.147+0000 7f269c3c2ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T01:31:55.192 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:55.189+0000 7f269c3c2ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T01:31:55.283 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:55.281+0000 7f269c3c2ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T01:31:55.331 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:55.328+0000 7f269c3c2ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T01:31:55.380 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:55.378+0000 7f269c3c2ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T01:31:55.447 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:55.444+0000 7f269c3c2ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T01:31:55.489 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:55.486+0000 7f269c3c2ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T01:31:55.531 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:55.528+0000 7f269c3c2ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T01:31:56.003 INFO:tasks.ceph.osd.3.smithi204.stderr:2024-09-15T01:31:56.000+0000 7feb14bf7640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T01:31:56.003 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-09-15T01:31:56.000+0000 7f06185f7640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T01:31:56.003 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-09-15T01:31:56.001+0000 7f40459f7640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T01:31:56.004 INFO:tasks.ceph.osd.1.smithi178.stderr:2024-09-15T01:31:56.001+0000 7f6ef3bf7640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T01:31:56.004 INFO:tasks.ceph.osd.0.smithi178.stderr:2024-09-15T01:31:56.001+0000 7f04543f7640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T01:31:56.004 INFO:tasks.ceph.osd.2.smithi178.stderr:2024-09-15T01:31:56.002+0000 7fb8a1df7640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T01:31:56.005 INFO:tasks.ceph.osd.3.smithi204.stderr:2024-09-15T01:31:56.002+0000 7feb1128c640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:31:56.005 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-09-15T01:31:56.002+0000 7f0614df0640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:31:56.005 INFO:tasks.ceph.osd.0.smithi178.stderr:2024-09-15T01:31:56.004+0000 7f0450bf0640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:31:56.006 INFO:tasks.ceph.osd.1.smithi178.stderr:2024-09-15T01:31:56.004+0000 7f6ef028c640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:31:56.006 INFO:tasks.ceph.osd.2.smithi178.stderr:2024-09-15T01:31:56.004+0000 7fb89e5f0640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:31:56.007 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-09-15T01:31:56.005+0000 7f40421f0640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T01:31:56.147 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:56.145+0000 7f269c3c2ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T01:31:57.166 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:57.163+0000 7f26809f6640 -1 mgr.server handle_report got status from non-daemon mon.g 2024-09-15T01:31:57.166 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:57.163+0000 7f26809f6640 -1 mgr.server handle_report got status from non-daemon mon.f 2024-09-15T01:31:57.166 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:57.163+0000 7f26809f6640 -1 mgr.server handle_report got status from non-daemon mon.i 2024-09-15T01:31:57.166 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:57.163+0000 7f26809f6640 -1 mgr.server handle_report got status from non-daemon mon.h 2024-09-15T01:31:57.166 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:57.163+0000 7f26809f6640 -1 mgr.server handle_report got status from non-daemon mon.d 2024-09-15T01:31:57.166 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:57.163+0000 7f26809f6640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T01:31:57.166 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:57.163+0000 7f26809f6640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T01:31:57.166 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-09-15T01:31:57.164+0000 7f26809f6640 -1 mgr.server handle_report got status from non-daemon mon.e 2024-09-15T01:32:01.588 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T01:32:01.675 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.673+0000 7f7757d9d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:01.675 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.673+0000 7f7757d9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:01.675 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.673+0000 7f7757d9d640 10 monclient: monmap: 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:epoch 0 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:last_changed 2024-09-15T01:32:01.674304+0000 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:created 2024-09-15T01:32:01.674304+0000 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:01.676 INFO:teuthology.misc.health.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:01.677 INFO:teuthology.misc.health.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:01.677 INFO:teuthology.misc.health.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:01.677 INFO:teuthology.misc.health.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:01.677 INFO:teuthology.misc.health.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:01.677 INFO:teuthology.misc.health.smithi178.stderr: 2024-09-15T01:32:01.677 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.673+0000 7f7757d9d640 1 Processor -- start 2024-09-15T01:32:01.677 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 1 -- start start 2024-09-15T01:32:01.677 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 10 monclient: init 2024-09-15T01:32:01.684 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:01.684 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 10 monclient: _add_conns ranks=[7,10,6,0,1,8,2,5,9,3,4] 2024-09-15T01:32:01.684 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f77500a2ae0 0x7f77500a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.684 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 10 monclient(hunting): picked mon.noname-i con 0x7f77500a2ae0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:01.685 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f775009f8c0 0x7f7750132860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.685 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 10 monclient(hunting): picked mon.noname-f con 0x7f775009f8c0 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:01.685 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f775009fc90 0x7f7750132da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.685 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 10 monclient(hunting): picked mon.noname-d con 0x7f775009fc90 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:01.685 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:01.685 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77500a1160 con 0x7f775009f8c0 2024-09-15T01:32:01.685 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:01.685 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f775009e0b0 con 0x7f775009fc90 2024-09-15T01:32:01.685 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:01.685 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f775009db60 con 0x7f77500a2ae0 2024-09-15T01:32:01.685 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:01.685 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f7757d9d640 10 monclient(hunting): authenticate will time out at 726.424687s 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f775633a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f775009fc90 0x7f7750132da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f775633a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f775009fc90 0x7f7750132da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:41862/0 (socket says 172.21.15.178:41862) 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f775633a640 1 -- 172.21.15.178:0/660035665 learned_addr learned my addr 172.21.15.178:0/660035665 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f775633a640 10 monclient(hunting): get_auth_request con 0x7f775009fc90 auth_method 0 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f775633a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f775633a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f775633a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f775633a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f775633a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.674+0000 7f775633a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f7755b39640 1 --2- 172.21.15.178:0/660035665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f77500a2ae0 0x7f77500a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f7755b39640 10 monclient(hunting): get_auth_request con 0x7f77500a2ae0 auth_method 0 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f7755b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:01.686 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f7755b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:01.687 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f7755b39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:01.687 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f7755338640 1 --2- 172.21.15.178:0/660035665 >> v2:172.21.15.204:3300/0 conn(0x7f775009f8c0 0x7f7750132860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.687 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f775633a640 10 monclient(hunting): handle_auth_done global_id 4263 payload 274 2024-09-15T01:32:01.687 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f775633a640 1 -- 172.21.15.178:0/660035665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f77500a2ae0 msgr2=0x7f77500a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:01.687 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f775633a640 1 --2- 172.21.15.178:0/660035665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f77500a2ae0 0x7f77500a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.687 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f775633a640 1 -- 172.21.15.178:0/660035665 >> v2:172.21.15.204:3300/0 conn(0x7f775009f8c0 msgr2=0x7f7750132860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:01.687 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f775633a640 1 --2- 172.21.15.178:0/660035665 >> v2:172.21.15.204:3300/0 conn(0x7f775009f8c0 0x7f7750132860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.687 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f775633a640 10 monclient: _finish_hunting 0 2024-09-15T01:32:01.687 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f775633a640 1 monclient: found mon.noname-d 2024-09-15T01:32:01.687 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f775633a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:01.687 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f775633a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:32:01.687 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f775633a640 1 -- 172.21.15.178:0/660035665 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775004bf60 con 0x7f775009fc90 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f7755338640 10 monclient: get_auth_request con 0x7f775009f8c0 auth_method 0 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f7755338640 1 --2- 172.21.15.178:0/660035665 >> v2:172.21.15.204:3300/0 conn(0x7f775009f8c0 0x7f7750132860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f775633a640 1 --2- 172.21.15.178:0/660035665 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f775009fc90 0x7f7750132da0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7748009f50 tx=0x7f7748009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=1bfdf01b8945d8d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.675+0000 7f7754b37640 1 -- 172.21.15.178:0/660035665 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7748007880 con 0x7f775009fc90 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7754b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7754b37640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7754b37640 10 monclient: dump: 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:epoch 1 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:01.688 INFO:teuthology.misc.health.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr: 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7754b37640 10 monclient: _finish_auth 0 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7754b37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 5 monclient: authenticate success, global_id 4263 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:01.689 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7754b37640 1 -- 172.21.15.178:0/660035665 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774802d050 con 0x7f775009fc90 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7754b37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 4 set_mon_vals no callback set 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7754b37640 1 -- 172.21.15.178:0/660035665 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7748030360 con 0x7f775009fc90 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7754b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7754b37640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7754b37640 10 monclient: dump: 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:epoch 1 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:01.690 INFO:teuthology.misc.health.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr: 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 10 monclient: shutdown 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 1 -- 172.21.15.178:0/660035665 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f775009fc90 msgr2=0x7f7750132da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:01.691 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 1 --2- 172.21.15.178:0/660035665 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f775009fc90 0x7f7750132da0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7748009f50 tx=0x7f7748009840 comp rx=0 tx=0).stop 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 1 -- 172.21.15.178:0/660035665 shutdown_connections 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 1 --2- 172.21.15.178:0/660035665 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f775009fc90 0x7f7750132da0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 1 --2- 172.21.15.178:0/660035665 >> v2:172.21.15.204:3300/0 conn(0x7f775009f8c0 0x7f7750132860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 1 --2- 172.21.15.178:0/660035665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f77500a2ae0 0x7f77500a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 1 -- 172.21.15.178:0/660035665 >> 172.21.15.178:0/660035665 conn(0x7f775006e7f0 msgr2=0x7f775006ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 1 -- 172.21.15.178:0/660035665 shutdown_connections 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.676+0000 7f7757d9d640 1 -- 172.21.15.178:0/660035665 wait complete. 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient: monmap: 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:epoch 0 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:01.692 INFO:teuthology.misc.health.smithi178.stderr:created 0.000000 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr: 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 1 Processor -- start 2024-09-15T01:32:01.693 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 1 -- start start 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient: init 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient: _add_conns ranks=[5,8,2,4,0,6,7,3,1] 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f775009f8c0 0x7f7750141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient(hunting): picked mon.noname-h con 0x7f775009f8c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f775009fc90 0x7f7750135b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient(hunting): picked mon.noname-e con 0x7f775009fc90 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77500a2ae0 0x7f775011f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient(hunting): picked mon.noname-b con 0x7f77500a2ae0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775009ccb0 con 0x7f77500a2ae0 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7750123200 con 0x7f775009f8c0 2024-09-15T01:32:01.694 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7750122f20 con 0x7f775009fc90 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f7757d9d640 10 monclient(hunting): authenticate will time out at 726.427881s 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.677+0000 7f775633a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77500a2ae0 0x7f775011f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f775009fc90 0x7f7750135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f775633a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77500a2ae0 0x7f775011f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:49326/0 (socket says 172.21.15.178:49326) 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f775633a640 1 -- 172.21.15.178:0/3295338152 learned_addr learned my addr 172.21.15.178:0/3295338152 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f775633a640 10 monclient(hunting): get_auth_request con 0x7f77500a2ae0 auth_method 0 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f775633a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f775633a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f775633a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 10 monclient(hunting): get_auth_request con 0x7f775009fc90 auth_method 0 2024-09-15T01:32:01.695 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755b39640 1 --2- 172.21.15.178:0/3295338152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f775009f8c0 0x7f7750141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755b39640 10 monclient(hunting): get_auth_request con 0x7f775009f8c0 auth_method 0 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755b39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f775633a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f775633a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f775633a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:01.696 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755b39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755b39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755b39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 10 monclient(hunting): handle_auth_done global_id 4238 payload 824 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 1 -- 172.21.15.178:0/3295338152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f775009f8c0 msgr2=0x7f7750141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 1 --2- 172.21.15.178:0/3295338152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f775009f8c0 0x7f7750141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 1 -- 172.21.15.178:0/3295338152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77500a2ae0 msgr2=0x7f775011f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 1 --2- 172.21.15.178:0/3295338152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77500a2ae0 0x7f775011f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 10 monclient: _finish_hunting 0 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 1 monclient: found mon.noname-e 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 1 -- 172.21.15.178:0/3295338152 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7748009ae0 con 0x7f775009fc90 2024-09-15T01:32:01.697 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f775633a640 1 --2- 172.21.15.178:0/3295338152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77500a2ae0 0x7f775011f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7755338640 1 --2- 172.21.15.178:0/3295338152 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f775009fc90 0x7f7750135b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f774000c8b0 tx=0x7f7740008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7745ffb640 1 -- 172.21.15.178:0/3295338152 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f774000ce40 con 0x7f775009fc90 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7745ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7745ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7745ffb640 10 monclient: dump: 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:epoch 1 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:01.698 INFO:teuthology.misc.health.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr: 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7745ffb640 10 monclient: _finish_auth 0 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7745ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7745ffb640 1 -- 172.21.15.178:0/3295338152 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7740007cb0 con 0x7f775009fc90 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7757d9d640 5 monclient: authenticate success, global_id 4238 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7757d9d640 10 monclient: _renew_subs 2024-09-15T01:32:01.699 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7757d9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7757d9d640 1 -- 172.21.15.178:0/3295338152 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7750121110 con 0x7f775009fc90 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7745ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7745ffb640 1 -- 172.21.15.178:0/3295338152 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7740007e20 con 0x7f775009fc90 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7745ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7745ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7745ffb640 10 monclient: dump: 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:epoch 1 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:01.700 INFO:teuthology.misc.health.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr: 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7747fff640 4 set_mon_vals no callback set 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7757d9d640 10 monclient: _renew_subs 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7757d9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.678+0000 7f7757d9d640 1 -- 172.21.15.178:0/3295338152 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7750121ce0 con 0x7f775009fc90 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.679+0000 7f7745ffb640 1 -- 172.21.15.178:0/3295338152 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7740007830 con 0x7f775009fc90 2024-09-15T01:32:01.701 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.679+0000 7f7757d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:01.702 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.679+0000 7f7757d9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:01.702 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.679+0000 7f7757d9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:01.702 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.679+0000 7f7757d9d640 1 -- 172.21.15.178:0/3295338152 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775004b720 con 0x7f775009fc90 2024-09-15T01:32:01.702 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.679+0000 7f7745ffb640 1 --2- 172.21.15.178:0/3295338152 >> v2:172.21.15.204:6812/28210 conn(0x7f77340396e0 0x7f773403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.702 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.679+0000 7f7745ffb640 1 -- 172.21.15.178:0/3295338152 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3149+0+0 (secure 0 0 0) 0x7f774004ae50 con 0x7f775009fc90 2024-09-15T01:32:01.702 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.680+0000 7f7755b39640 1 --2- 172.21.15.178:0/3295338152 >> v2:172.21.15.204:6812/28210 conn(0x7f77340396e0 0x7f773403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.702 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.680+0000 7f7755b39640 10 monclient: get_auth_request con 0x7f77340396e0 auth_method 0 2024-09-15T01:32:01.702 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.680+0000 7f7755b39640 1 --2- 172.21.15.178:0/3295338152 >> v2:172.21.15.204:6812/28210 conn(0x7f77340396e0 0x7f773403bb90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f774c00bd40 tx=0x7f774c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:01.702 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.684+0000 7f7745ffb640 1 -- 172.21.15.178:0/3295338152 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f77400113f0 con 0x7f775009fc90 2024-09-15T01:32:01.703 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.684+0000 7f7745ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:01.703 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.684+0000 7f7745ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:01.841 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.839+0000 7f7757d9d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:32:01.842 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.839+0000 7f7757d9d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:32:01.842 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.839+0000 7f7757d9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:01.842 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.839+0000 7f7757d9d640 1 -- 172.21.15.178:0/3295338152 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7750070bd0 con 0x7f775009fc90 2024-09-15T01:32:01.843 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.840+0000 7f7745ffb640 1 -- 172.21.15.178:0/3295338152 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+8481 (secure 0 0 0) 0x7f774001a450 con 0x7f775009fc90 2024-09-15T01:32:01.843 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.840+0000 7f7745ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:32:01.843 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.840+0000 7f7745ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:01.843 INFO:teuthology.misc.health.smithi178.stdout: 2024-09-15T01:32:01.843 INFO:teuthology.misc.health.smithi178.stdout:{"epoch":15,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","created":"2024-09-15T01:31:51.657329+0000","modified":"2024-09-15T01:32:01.028009+0000","last_up_change":"2024-09-15T01:31:56.999458+0000","last_in_change":"2024-09-15T01:31:53.995822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:31:58.183767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"690d2bc0-3a5e-4f4a-b00b-43a73741b7b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":29232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6801","nonce":29232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":29232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":29232}]},"public_addr":"172.21.15.178:6800/29232","cluster_addr":"172.21.15.178:6801/29232","heartbeat_back_addr":"172.21.15.178:6803/29232","heartbeat_front_addr":"172.21.15.178:6802/29232","state":["exists","up"]},{"osd":1,"uuid":"bbba9997-deb4-42ca-8140-6553a9b6c45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":29233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6805","nonce":29233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6807","nonce":29233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":29233}]},"public_addr":"172.21.15.178:6804/29233","cluster_addr":"172.21.15.178:6805/29233","heartbeat_back_addr":"172.21.15.178:6807/29233","heartbeat_front_addr":"172.21.15.178:6806/29233","state":["exists","up"]},{"osd":2,"uuid":"24b56e74-a222-40f9-a8bf-38d597553f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":29234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":29234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6811","nonce":29234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":29234}]},"public_addr":"172.21.15.178:6808/29234","cluster_addr":"172.21.15.178:6809/29234","heartbeat_back_addr":"172.21.15.178:6811/29234","heartbeat_front_addr":"172.21.15.178:6810/29234","state":["exists","up"]},{"osd":3,"uuid":"a5bf3f7d-4c59-470a-8f69-3fea018c13f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":28752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":28752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":28752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":28752}]},"public_addr":"172.21.15.204:6801/28752","cluster_addr":"172.21.15.204:6802/28752","heartbeat_back_addr":"172.21.15.204:6806/28752","heartbeat_front_addr":"172.21.15.204:6804/28752","state":["exists","up"]},{"osd":4,"uuid":"cdaf5723-3a5c-4238-a8e6-38f2325ccbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":28753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":28753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":28753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6805","nonce":28753}]},"public_addr":"172.21.15.204:6800/28753","cluster_addr":"172.21.15.204:6803/28753","heartbeat_back_addr":"172.21.15.204:6807/28753","heartbeat_front_addr":"172.21.15.204:6805/28753","state":["exists","up"]},{"osd":5,"uuid":"6b67035f-8718-4762-98bf-0dd0aa86e6e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":28754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":28754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":28754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":28754}]},"public_addr":"172.21.15.204:6808/28754","cluster_addr":"172.21.15.204:6809/28754","heartbeat_back_addr":"172.21.15.204:6811/28754","heartbeat_front_addr":"172.21.15.204:6810/28754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.093711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.117496+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.123704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.082038+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.088260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.090342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:32:01.844 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 1 -- 172.21.15.178:0/3295338152 >> v2:172.21.15.204:6812/28210 conn(0x7f77340396e0 msgr2=0x7f773403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:01.844 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 1 --2- 172.21.15.178:0/3295338152 >> v2:172.21.15.204:6812/28210 conn(0x7f77340396e0 0x7f773403bb90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f774c00bd40 tx=0x7f774c008040 comp rx=0 tx=0).stop 2024-09-15T01:32:01.844 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 10 monclient: shutdown 2024-09-15T01:32:01.845 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:01.845 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 1 -- 172.21.15.178:0/3295338152 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f775009fc90 msgr2=0x7f7750135b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:01.845 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 1 --2- 172.21.15.178:0/3295338152 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f775009fc90 0x7f7750135b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f774000c8b0 tx=0x7f7740008040 comp rx=0 tx=0).stop 2024-09-15T01:32:01.845 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 1 -- 172.21.15.178:0/3295338152 shutdown_connections 2024-09-15T01:32:01.845 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 1 --2- 172.21.15.178:0/3295338152 >> v2:172.21.15.204:6812/28210 conn(0x7f77340396e0 0x7f773403bb90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.845 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 1 --2- 172.21.15.178:0/3295338152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77500a2ae0 0x7f775011f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.845 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 1 --2- 172.21.15.178:0/3295338152 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f775009fc90 0x7f7750135b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.845 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 1 --2- 172.21.15.178:0/3295338152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f775009f8c0 0x7f7750141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.845 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 1 -- 172.21.15.178:0/3295338152 >> 172.21.15.178:0/3295338152 conn(0x7f775006e7f0 msgr2=0x7f77500a3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:01.845 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 1 -- 172.21.15.178:0/3295338152 shutdown_connections 2024-09-15T01:32:01.845 INFO:teuthology.misc.health.smithi178.stderr:2024-09-15T01:32:01.842+0000 7f7757d9d640 1 -- 172.21.15.178:0/3295338152 wait complete. 2024-09-15T01:32:01.853 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-15T01:32:01.853 INFO:tasks.ceph:Creating RBD pool 2024-09-15T01:32:01.854 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T01:32:01.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.944+0000 7f7be7b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:01.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.944+0000 7f7be7b9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:01.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.944+0000 7f7be7b9d640 10 monclient: monmap: 2024-09-15T01:32:01.946 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:01.946 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:01.946 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:01.944920+0000 2024-09-15T01:32:01.947 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:01.944920+0000 2024-09-15T01:32:01.947 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:01.947 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:01.947 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:01.947 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:01.947 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:01.947 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:01.947 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:01.947 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:01.947 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:01.947 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:01.948 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:01.948 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:01.948 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:01.948 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:01.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.944+0000 7f7be7b9d640 1 Processor -- start 2024-09-15T01:32:01.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.944+0000 7f7be7b9d640 1 -- start start 2024-09-15T01:32:01.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.944+0000 7f7be7b9d640 10 monclient: init 2024-09-15T01:32:01.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:01.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 10 monclient: _add_conns ranks=[1,4,3,9,2,8,7,0,10,6,5] 2024-09-15T01:32:01.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7be012f5d0 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:01.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7be006e510 0x7f7be0134900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 10 monclient(hunting): picked mon.noname-c con 0x7f7be006e510 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:01.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7be006e8e0 0x7f7be0136e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 10 monclient(hunting): picked mon.noname-g con 0x7f7be006e8e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:01.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be004bf60 con 0x7f7be012f5d0 2024-09-15T01:32:01.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:01.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be009f2c0 con 0x7f7be006e8e0 2024-09-15T01:32:01.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:01.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7be006b850 con 0x7f7be006e510 2024-09-15T01:32:01.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:01.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be7b9d640 10 monclient(hunting): authenticate will time out at 726.695367s 2024-09-15T01:32:01.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7be006e510 0x7f7be0134900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7be006e510 0x7f7be0134900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:40198/0 (socket says 172.21.15.178:40198) 2024-09-15T01:32:01.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 1 -- 172.21.15.178:0/115929811 learned_addr learned my addr 172.21.15.178:0/115929811 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:01.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 10 monclient(hunting): get_auth_request con 0x7f7be006e510 auth_method 0 2024-09-15T01:32:01.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:01.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:01.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:01.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:01.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:01.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:01.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be613a640 1 --2- 172.21.15.178:0/115929811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7be006e8e0 0x7f7be0136e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be613a640 10 monclient(hunting): get_auth_request con 0x7f7be006e8e0 auth_method 0 2024-09-15T01:32:01.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be613a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:01.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be613a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be613a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 10 monclient(hunting): handle_auth_done global_id 4198 payload 274 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 1 -- 172.21.15.178:0/115929811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7be006e8e0 msgr2=0x7f7be0136e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 1 --2- 172.21.15.178:0/115929811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7be006e8e0 0x7f7be0136e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 1 -- 172.21.15.178:0/115929811 >> v1:172.21.15.204:6789/0 conn(0x7f7be012f5d0 legacy=0x7f7be012f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 10 monclient: _finish_hunting 0 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 1 monclient: found mon.noname-c 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be5138640 1 -- 172.21.15.178:0/115929811 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be006c740 con 0x7f7be006e510 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.945+0000 7f7be613a640 1 --2- 172.21.15.178:0/115929811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7be006e8e0 0x7f7be0136e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be5138640 1 --2- 172.21.15.178:0/115929811 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7be006e510 0x7f7be0134900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009940 tx=0x7f7bd00091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=75251e2edb2e86fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:01.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be4937640 1 -- 172.21.15.178:0/115929811 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bd002ad00 con 0x7f7be006e510 2024-09-15T01:32:01.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be4937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:01.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be4937640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:01.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be4937640 10 monclient: dump: 2024-09-15T01:32:01.963 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:01.963 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:01.963 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.963 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.963 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:01.963 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:01.964 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:01.964 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:01.964 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:01.964 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:01.964 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:01.964 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:01.964 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:01.964 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:01.964 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be4937640 10 monclient: _finish_auth 0 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be4937640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be4937640 1 -- 172.21.15.178:0/115929811 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd002ae80 con 0x7f7be006e510 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be7b9d640 5 monclient: authenticate success, global_id 4198 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be7b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be4937640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be7b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be7b9d640 4 set_mon_vals no callback set 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be4937640 1 -- 172.21.15.178:0/115929811 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bd0031ca0 con 0x7f7be006e510 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be4937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be4937640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:01.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.946+0000 7f7be4937640 10 monclient: dump: 2024-09-15T01:32:01.966 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:01.966 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:01.966 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.966 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.966 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:01.966 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:01.966 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:01.966 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:01.966 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:01.966 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:01.967 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:01.967 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:01.967 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:01.967 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:01.967 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:01.967 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:01.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.947+0000 7f7be7b9d640 10 monclient: shutdown 2024-09-15T01:32:01.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.947+0000 7f7be7b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:01.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.947+0000 7f7be7b9d640 1 -- 172.21.15.178:0/115929811 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7be006e510 msgr2=0x7f7be0134900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:01.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.947+0000 7f7be7b9d640 1 --2- 172.21.15.178:0/115929811 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7be006e510 0x7f7be0134900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009940 tx=0x7f7bd00091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.947+0000 7f7be7b9d640 1 -- 172.21.15.178:0/115929811 shutdown_connections 2024-09-15T01:32:01.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.947+0000 7f7be7b9d640 1 --2- 172.21.15.178:0/115929811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7be006e8e0 0x7f7be0136e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.947+0000 7f7be7b9d640 1 --2- 172.21.15.178:0/115929811 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7be006e510 0x7f7be0134900 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.947+0000 7f7be7b9d640 1 -- 172.21.15.178:0/115929811 >> 172.21.15.178:0/115929811 conn(0x7f7be0068a60 msgr2=0x7f7be00a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:01.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.947+0000 7f7be7b9d640 1 -- 172.21.15.178:0/115929811 shutdown_connections 2024-09-15T01:32:01.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.947+0000 7f7be7b9d640 1 -- 172.21.15.178:0/115929811 wait complete. 2024-09-15T01:32:01.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.947+0000 7f7be7b9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:01.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.947+0000 7f7be7b9d640 10 monclient: monmap: 2024-09-15T01:32:01.968 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:01.968 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:01.969 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:01.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 1 Processor -- start 2024-09-15T01:32:01.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 1 -- start start 2024-09-15T01:32:01.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 10 monclient: init 2024-09-15T01:32:01.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:01.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 10 monclient: _add_conns ranks=[5,6,7,4,3,2,0,1,8] 2024-09-15T01:32:01.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7be006e510 0x7f7be013fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 10 monclient(hunting): picked mon.noname-h con 0x7f7be006e510 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:01.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7be006e8e0 0x7f7be012e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 10 monclient(hunting): picked mon.noname-d con 0x7f7be006e8e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:01.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7be012f5d0 0x7f7be012eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 10 monclient(hunting): picked mon.noname-i con 0x7f7be012f5d0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:01.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7be011c980 con 0x7f7be006e510 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7be006bb00 con 0x7f7be006e8e0 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7be009ccc0 con 0x7f7be012f5d0 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be7b9d640 10 monclient(hunting): authenticate will time out at 726.698563s 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7be006e8e0 0x7f7be012e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7be006e8e0 0x7f7be012e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:41876/0 (socket says 172.21.15.178:41876) 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be613a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7be012f5d0 0x7f7be012eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 1 -- 172.21.15.178:0/3514595553 learned_addr learned my addr 172.21.15.178:0/3514595553 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 10 monclient(hunting): get_auth_request con 0x7f7be006e8e0 auth_method 0 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:01.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:01.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:01.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5939640 1 --2- 172.21.15.178:0/3514595553 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7be006e510 0x7f7be013fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be613a640 10 monclient(hunting): get_auth_request con 0x7f7be012f5d0 auth_method 0 2024-09-15T01:32:01.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be613a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:01.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be613a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:01.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be613a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5939640 10 monclient(hunting): get_auth_request con 0x7f7be006e510 auth_method 0 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be613a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be613a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be613a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:01.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 10 monclient(hunting): handle_auth_done global_id 4272 payload 824 2024-09-15T01:32:01.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 1 -- 172.21.15.178:0/3514595553 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7be012f5d0 msgr2=0x7f7be012eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:01.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 1 --2- 172.21.15.178:0/3514595553 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7be012f5d0 0x7f7be012eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 1 -- 172.21.15.178:0/3514595553 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7be006e510 msgr2=0x7f7be013fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:01.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 1 --2- 172.21.15.178:0/3514595553 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7be006e510 0x7f7be013fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:01.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 10 monclient: _finish_hunting 0 2024-09-15T01:32:01.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 1 monclient: found mon.noname-d 2024-09-15T01:32:01.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:01.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:32:01.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.948+0000 7f7be5138640 1 -- 172.21.15.178:0/3514595553 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd0009450 con 0x7f7be006e8e0 2024-09-15T01:32:01.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7be5138640 1 --2- 172.21.15.178:0/3514595553 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7be006e8e0 0x7f7be012e570 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00097e0 tx=0x7f7bd0032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:01.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd5ffb640 1 -- 172.21.15.178:0/3514595553 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bd003c880 con 0x7f7be006e8e0 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd5ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd5ffb640 10 monclient: dump: 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:01.975 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd5ffb640 1 -- 172.21.15.178:0/3514595553 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd003ca00 con 0x7f7be006e8e0 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7be7b9d640 5 monclient: authenticate success, global_id 4272 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7be7b9d640 10 monclient: _renew_subs 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7be7b9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:01.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7be7b9d640 1 -- 172.21.15.178:0/3514595553 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be011c6a0 con 0x7f7be006e8e0 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd5ffb640 1 -- 172.21.15.178:0/3514595553 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bd003b3a0 con 0x7f7be006e8e0 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd5ffb640 10 monclient: dump: 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:01.977 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7bd7fff640 4 set_mon_vals no callback set 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7be7b9d640 10 monclient: _renew_subs 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7be7b9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.949+0000 7f7be7b9d640 1 -- 172.21.15.178:0/3514595553 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be0067410 con 0x7f7be006e8e0 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.950+0000 7f7bd5ffb640 1 -- 172.21.15.178:0/3514595553 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7bd003c400 con 0x7f7be006e8e0 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.950+0000 7f7be7b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:01.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.950+0000 7f7be7b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:01.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.950+0000 7f7be7b9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:01.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.950+0000 7f7bd5ffb640 1 --2- 172.21.15.178:0/3514595553 >> v2:172.21.15.204:6812/28210 conn(0x7f7bc40397b0 0x7f7bc403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:01.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.950+0000 7f7be7b9d640 1 -- 172.21.15.178:0/3514595553 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be00675a0 con 0x7f7be006e8e0 2024-09-15T01:32:01.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.950+0000 7f7bd5ffb640 1 -- 172.21.15.178:0/3514595553 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3149+0+0 (secure 0 0 0) 0x7f7bd006a950 con 0x7f7be006e8e0 2024-09-15T01:32:01.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.950+0000 7f7be5939640 1 --2- 172.21.15.178:0/3514595553 >> v2:172.21.15.204:6812/28210 conn(0x7f7bc40397b0 0x7f7bc403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:01.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.950+0000 7f7be5939640 10 monclient: get_auth_request con 0x7f7bc40397b0 auth_method 0 2024-09-15T01:32:01.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.951+0000 7f7be5939640 1 --2- 172.21.15.178:0/3514595553 >> v2:172.21.15.204:6812/28210 conn(0x7f7bc40397b0 0x7f7bc403bc60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc007fd0 tx=0x7f7bdc007e10 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:01.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.955+0000 7f7bd5ffb640 1 -- 172.21.15.178:0/3514595553 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7bd0037080 con 0x7f7be006e8e0 2024-09-15T01:32:01.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.955+0000 7f7bd5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:01.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:01.955+0000 7f7bd5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:02.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:02.112+0000 7f7be7b9d640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-15T01:32:02.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:02.112+0000 7f7be7b9d640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-15T01:32:02.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:02.112+0000 7f7be7b9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:02.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:02.112+0000 7f7be7b9d640 1 -- 172.21.15.178:0/3514595553 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f7be004b720 con 0x7f7be006e8e0 2024-09-15T01:32:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.052+0000 7f7bd5ffb640 1 -- 172.21.15.178:0/3514595553 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7f7bd0043020 con 0x7f7be006e8e0 2024-09-15T01:32:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.052+0000 7f7bd5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-15T01:32:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.052+0000 7f7bd5ffb640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2024-09-15T01:32:03.056 INFO:teuthology.orchestra.run.smithi178.stderr:pool 'rbd' created 2024-09-15T01:32:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 1 -- 172.21.15.178:0/3514595553 >> v2:172.21.15.204:6812/28210 conn(0x7f7bc40397b0 msgr2=0x7f7bc403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 1 --2- 172.21.15.178:0/3514595553 >> v2:172.21.15.204:6812/28210 conn(0x7f7bc40397b0 0x7f7bc403bc60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc007fd0 tx=0x7f7bdc007e10 comp rx=0 tx=0).stop 2024-09-15T01:32:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 10 monclient: shutdown 2024-09-15T01:32:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 1 -- 172.21.15.178:0/3514595553 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7be006e8e0 msgr2=0x7f7be012e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 1 --2- 172.21.15.178:0/3514595553 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7be006e8e0 0x7f7be012e570 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00097e0 tx=0x7f7bd0032000 comp rx=0 tx=0).stop 2024-09-15T01:32:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 1 -- 172.21.15.178:0/3514595553 shutdown_connections 2024-09-15T01:32:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 1 --2- 172.21.15.178:0/3514595553 >> v2:172.21.15.204:6812/28210 conn(0x7f7bc40397b0 0x7f7bc403bc60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 1 --2- 172.21.15.178:0/3514595553 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7be012f5d0 0x7f7be012eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 1 --2- 172.21.15.178:0/3514595553 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7be006e8e0 0x7f7be012e570 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 1 --2- 172.21.15.178:0/3514595553 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7be006e510 0x7f7be013fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 1 -- 172.21.15.178:0/3514595553 >> 172.21.15.178:0/3514595553 conn(0x7f7be0068a60 msgr2=0x7f7be0132660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 1 -- 172.21.15.178:0/3514595553 shutdown_connections 2024-09-15T01:32:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.056+0000 7f7be7b9d640 1 -- 172.21.15.178:0/3514595553 wait complete. 2024-09-15T01:32:03.070 DEBUG:teuthology.orchestra.run.smithi178:> rbd --cluster ceph pool init rbd 2024-09-15T01:32:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.128+0000 7fb3ba237b80 10 monclient: get_monmap_and_config 2024-09-15T01:32:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.128+0000 7fb3ba237b80 10 monclient: build_initial_monmap 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.128+0000 7fb3ba237b80 10 monclient: monmap: 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:03.129977+0000 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:03.129977+0000 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:03.132 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.128+0000 7fb3ba237b80 1 Processor -- start 2024-09-15T01:32:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.128+0000 7fb3ba237b80 1 -- start start 2024-09-15T01:32:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.128+0000 7fb3ba237b80 10 monclient: init 2024-09-15T01:32:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 10 monclient: _reopen_session rank -1 2024-09-15T01:32:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 10 monclient: _add_conns ranks=[6,8,2,7,4,5,10,1,0,3,9] 2024-09-15T01:32:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x557c69144090 0x557c69143850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 10 monclient(hunting): picked mon.noname-d con 0x557c69144090 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x557c69144460 0x557c69144830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 10 monclient(hunting): picked mon.noname-e con 0x557c69144460 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557c691d56c0 0x557c691d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 10 monclient(hunting): picked mon.noname-b con 0x557c691d56c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c6909e260 con 0x557c691d56c0 2024-09-15T01:32:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x557c69099f40 con 0x557c69144090 2024-09-15T01:32:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x557c6905cba0 con 0x557c69144460 2024-09-15T01:32:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 10 monclient(hunting): _renew_subs 2024-09-15T01:32:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3ba237b80 10 monclient(hunting): authenticate will time out at 727.880488s 2024-09-15T01:32:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x557c69144090 0x557c69143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3aaffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x557c69144460 0x557c69144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557c691d56c0 0x557c691d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x557c69144090 0x557c69143850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:41892/0 (socket says 172.21.15.178:41892) 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3aaffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x557c69144460 0x557c69144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:51862/0 (socket says 172.21.15.178:51862) 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 1 -- 172.21.15.178:0/1974159272 learned_addr learned my addr 172.21.15.178:0/1974159272 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 10 monclient(hunting): get_auth_request con 0x557c69144090 auth_method 0 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3aaffd640 10 monclient(hunting): get_auth_request con 0x557c69144460 auth_method 0 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b3fff640 10 monclient(hunting): get_auth_request con 0x557c691d56c0 auth_method 0 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_done global_id 4281 payload 274 2024-09-15T01:32:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 1 -- 172.21.15.178:0/1974159272 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x557c69144460 msgr2=0x557c69144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 1 --2- 172.21.15.178:0/1974159272 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x557c69144460 0x557c69144830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 1 -- 172.21.15.178:0/1974159272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557c691d56c0 msgr2=0x557c691d5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 1 --2- 172.21.15.178:0/1974159272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557c691d56c0 0x557c691d5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.129+0000 7fb3b37fe640 1 monclient: found mon.noname-d 2024-09-15T01:32:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.130+0000 7fb3b37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.130+0000 7fb3b37fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:32:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.130+0000 7fb3b37fe640 1 -- 172.21.15.178:0/1974159272 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c690650e0 con 0x557c69144090 2024-09-15T01:32:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.130+0000 7fb3b3fff640 1 --2- 172.21.15.178:0/1974159272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557c691d56c0 0x557c691d5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.130+0000 7fb3aaffd640 1 --2- 172.21.15.178:0/1974159272 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x557c69144460 0x557c69144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.131+0000 7fb3b37fe640 1 --2- 172.21.15.178:0/1974159272 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x557c69144090 0x557c69143850 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x557c691430c0 tx=0x7fb3a4045d50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=6e1347cf2f5367fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:03.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.131+0000 7fb3b2ffd640 1 -- 172.21.15.178:0/1974159272 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3a404d9e0 con 0x557c69144090 2024-09-15T01:32:03.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.131+0000 7fb3b2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:03.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.131+0000 7fb3b2ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:32:03.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.131+0000 7fb3b2ffd640 10 monclient: dump: 2024-09-15T01:32:03.145 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:03.145 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:03.145 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:03.145 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:03.145 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:03.145 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:03.145 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:03.145 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3b2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3b2ffd640 1 -- 172.21.15.178:0/1974159272 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a404db60 con 0x557c69144090 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 5 monclient: authenticate success, global_id 4281 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3b2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 10 monclient: get_monmap_and_config success 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 4 set_mon_vals no callback set 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3b2ffd640 1 -- 172.21.15.178:0/1974159272 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3a404dcd0 con 0x557c69144090 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3b2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3b2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3b2ffd640 10 monclient: dump: 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 10 monclient: shutdown 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 1 -- 172.21.15.178:0/1974159272 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x557c69144090 msgr2=0x557c69143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 1 --2- 172.21.15.178:0/1974159272 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x557c69144090 0x557c69143850 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x557c691430c0 tx=0x7fb3a4045d50 comp rx=0 tx=0).stop 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 1 -- 172.21.15.178:0/1974159272 shutdown_connections 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 1 --2- 172.21.15.178:0/1974159272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557c691d56c0 0x557c691d5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 1 --2- 172.21.15.178:0/1974159272 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x557c69144460 0x557c69144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 1 --2- 172.21.15.178:0/1974159272 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x557c69144090 0x557c69143850 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 1 -- 172.21.15.178:0/1974159272 >> 172.21.15.178:0/1974159272 conn(0x557c691459a0 msgr2=0x557c69145da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 1 -- 172.21.15.178:0/1974159272 shutdown_connections 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.132+0000 7fb3ba237b80 1 -- 172.21.15.178:0/1974159272 wait complete. 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.133+0000 7fb3ba237b80 10 monclient: build_initial_monmap 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.133+0000 7fb3ba237b80 10 monclient: monmap: 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.133+0000 7fb3ba237b80 1 Processor -- start 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.133+0000 7fb3ba237b80 1 -- start start 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.133+0000 7fb3ba237b80 10 monclient: init 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.133+0000 7fb3ba237b80 10 monclient: _reopen_session rank -1 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.133+0000 7fb3ba237b80 10 monclient: _add_conns ranks=[7,5,3,8,6,0,1,4,2] 2024-09-15T01:32:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x557c69144090 0x557c6913b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 10 monclient(hunting): picked mon.noname-i con 0x557c69144090 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x557c69144460 0x557c6913b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 10 monclient(hunting): picked mon.noname-h con 0x557c69144460 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c691d56c0 0x557c6913bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 10 monclient(hunting): picked mon.noname-g con 0x557c691d56c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c6905c840 con 0x557c691d56c0 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x557c6901f7a0 con 0x557c69144460 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x557c69141190 con 0x557c69144090 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 10 monclient(hunting): _renew_subs 2024-09-15T01:32:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3ba237b80 10 monclient(hunting): authenticate will time out at 727.885061s 2024-09-15T01:32:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x557c69144090 0x557c6913b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b3fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c691d56c0 0x557c6913bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x557c69144090 0x557c6913b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:44348/0 (socket says 172.21.15.178:44348) 2024-09-15T01:32:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 1 -- 172.21.15.178:0/769777824 learned_addr learned my addr 172.21.15.178:0/769777824 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 10 monclient(hunting): get_auth_request con 0x557c69144090 auth_method 0 2024-09-15T01:32:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3aaffd640 1 --2- 172.21.15.178:0/769777824 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x557c69144460 0x557c6913b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3aaffd640 10 monclient(hunting): get_auth_request con 0x557c69144460 auth_method 0 2024-09-15T01:32:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b3fff640 10 monclient(hunting): get_auth_request con 0x557c691d56c0 auth_method 0 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_done global_id 4237 payload 824 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 1 -- 172.21.15.178:0/769777824 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x557c69144460 msgr2=0x557c6913b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 1 --2- 172.21.15.178:0/769777824 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x557c69144460 0x557c6913b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 1 -- 172.21.15.178:0/769777824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c691d56c0 msgr2=0x557c6913bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 1 --2- 172.21.15.178:0/769777824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c691d56c0 0x557c6913bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 1 monclient: found mon.noname-i 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 1 -- 172.21.15.178:0/769777824 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c005cf0 con 0x557c69144090 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.134+0000 7fb3b37fe640 1 --2- 172.21.15.178:0/769777824 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x557c69144090 0x557c6913b1c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4000c00 tx=0x7fb3a4058000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3a404ee30 con 0x557c69144090 2024-09-15T01:32:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3abfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3abfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3abfff640 10 monclient: dump: 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3abfff640 10 monclient: _finish_auth 0 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a40558c0 con 0x557c69144090 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3ba237b80 5 monclient: authenticate success, global_id 4237 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3ba237b80 10 monclient: _renew_subs 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3ba237b80 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac00a0e0 con 0x557c69144090 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3abfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3a405f410 con 0x557c69144090 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3abfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3abfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:32:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3abfff640 10 monclient: dump: 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3b27fc640 4 set_mon_vals no callback set 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3ba237b80 10 monclient: _renew_subs 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3ba237b80 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.135+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a4005db0 con 0x557c69144090 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.136+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb3a405c070 con 0x557c69144090 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.136+0000 7fb3abfff640 1 --2- 172.21.15.178:0/769777824 >> v2:172.21.15.204:6812/28210 conn(0x7fb3940380e0 0x7fb39403a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.136+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3491+0+0 (secure 0 0 0) 0x7fb3a408d7c0 con 0x557c69144090 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.136+0000 7fb3ba237b80 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.136+0000 7fb3ba237b80 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.136+0000 7fb3ba237b80 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.136+0000 7fb3aaffd640 1 --2- 172.21.15.178:0/769777824 >> v2:172.21.15.204:6812/28210 conn(0x7fb3940380e0 0x7fb39403a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.136+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) v1 -- 0x557c6903e270 con 0x557c69144090 2024-09-15T01:32:03.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.136+0000 7fb3aaffd640 10 monclient: get_auth_request con 0x7fb3940380e0 auth_method 0 2024-09-15T01:32:03.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:03.137+0000 7fb3aaffd640 1 --2- 172.21.15.178:0/769777824 >> v2:172.21.15.204:6812/28210 conn(0x7fb3940380e0 0x7fb39403a590 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009c40 tx=0x7fb39c00c040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:04.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.054+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3a405a030 con 0x557c69144090 2024-09-15T01:32:04.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.054+0000 7fb3abfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-15T01:32:04.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.054+0000 7fb3abfff640 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2024-09-15T01:32:04.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.054+0000 7fb3ba237b80 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:04.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.054+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x557c69110470 con 0x557c69144090 2024-09-15T01:32:04.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.059+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3a4055440 con 0x557c69144090 2024-09-15T01:32:04.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.059+0000 7fb3abfff640 10 monclient: handle_get_version_reply finishing 1 version 17 2024-09-15T01:32:04.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.060+0000 7fb3b27fc640 10 monclient: _renew_subs 2024-09-15T01:32:04.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.060+0000 7fb3b27fc640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:04.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.060+0000 7fb3b27fc640 1 -- 172.21.15.178:0/769777824 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=17}) v3 -- 0x7fb398002b00 con 0x557c69144090 2024-09-15T01:32:04.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.060+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 <== mon.7 v2:172.21.15.204:3303/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 666+0+0 (secure 0 0 0) 0x7fb3a4076050 con 0x557c69144090 2024-09-15T01:32:04.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.060+0000 7fb3ba237b80 1 --2- 172.21.15.178:0/769777824 >> v2:172.21.15.178:6804/29233 conn(0x557c691eb640 0x557c691eda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:04.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.060+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 --> v2:172.21.15.178:6804/29233 -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17) v8 -- 0x557c691edf60 con 0x557c691eb640 2024-09-15T01:32:04.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.060+0000 7fb3b3fff640 1 --2- 172.21.15.178:0/769777824 >> v2:172.21.15.178:6804/29233 conn(0x557c691eb640 0x557c691eda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:04.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.060+0000 7fb3b3fff640 10 monclient: get_auth_request con 0x557c691eb640 auth_method 0 2024-09-15T01:32:04.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.061+0000 7fb3b3fff640 1 --2- 172.21.15.178:0/769777824 >> v2:172.21.15.178:6804/29233 conn(0x557c691eb640 0x557c691eda20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:04.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.062+0000 7fb3b3fff640 1 -- 172.21.15.178:0/769777824 <== osd.1 v2:172.21.15.178:6804/29233 1 ==== osd_op_reply(1 rbd_trash [create] v17'1 uv1 ondisk = 0) v8 ==== 153+0+0 (crc 0 0 0) 0x7fb3ac00aba0 con 0x557c691eb640 2024-09-15T01:32:04.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.064+0000 7fb3ba237b80 1 --2- 172.21.15.178:0/769777824 >> v2:172.21.15.178:6800/29232 conn(0x557c6923af10 0x557c6923d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:04.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.064+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 --> v2:172.21.15.178:6800/29232 -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17) v8 -- 0x557c6923d920 con 0x557c6923af10 2024-09-15T01:32:04.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.064+0000 7fb3b37fe640 1 --2- 172.21.15.178:0/769777824 >> v2:172.21.15.178:6800/29232 conn(0x557c6923af10 0x557c6923d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:04.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.064+0000 7fb3b37fe640 10 monclient: get_auth_request con 0x557c6923af10 auth_method 0 2024-09-15T01:32:04.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.064+0000 7fb3b37fe640 1 --2- 172.21.15.178:0/769777824 >> v2:172.21.15.178:6800/29232 conn(0x557c6923af10 0x557c6923d390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:04.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.065+0000 7fb3b37fe640 1 -- 172.21.15.178:0/769777824 <== osd.0 v2:172.21.15.178:6800/29232 1 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3a40544a0 con 0x557c6923af10 2024-09-15T01:32:04.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.065+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 --> v2:172.21.15.178:6800/29232 -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17) v8 -- 0x557c692399a0 con 0x557c6923af10 2024-09-15T01:32:04.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.065+0000 7fb3b37fe640 1 -- 172.21.15.178:0/769777824 <== osd.0 v2:172.21.15.178:6800/29232 2 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3a40544a0 con 0x557c6923af10 2024-09-15T01:32:04.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.065+0000 7fb3b1ffb640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:04.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:04.065+0000 7fb3b1ffb640 1 -- 172.21.15.178:0/769777824 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) v4 -- 0x7fb38c001ee0 con 0x557c69144090 2024-09-15T01:32:05.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:05.071+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 <== mon.7 v2:172.21.15.204:3303/0 9 ==== pool_op_reply(tid 4 (0) Success v18) v1 ==== 55+0+0 (secure 0 0 0) 0x7fb3a405a030 con 0x557c69144090 2024-09-15T01:32:05.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:05.071+0000 7fb3abfff640 10 monclient: _renew_subs 2024-09-15T01:32:05.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:05.071+0000 7fb3abfff640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:05.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:05.071+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=18}) v3 -- 0x7fb394002830 con 0x557c69144090 2024-09-15T01:32:05.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:05.071+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 <== mon.7 v2:172.21.15.204:3303/0 10 ==== osd_map(18..18 src has 1..18) v4 ==== 618+0+0 (secure 0 0 0) 0x7fb3a4080040 con 0x557c69144090 2024-09-15T01:32:05.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:05.071+0000 7fb3b27fc640 1 -- 172.21.15.178:0/769777824 --> v2:172.21.15.178:6800/29232 -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) v8 -- 0x7fb3980055c0 con 0x557c6923af10 2024-09-15T01:32:05.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:05.072+0000 7fb3b37fe640 1 -- 172.21.15.178:0/769777824 <== osd.0 v2:172.21.15.178:6800/29232 3 ==== osd_op_reply(5 rbd_info [create,write 0~8] v18'1 uv1 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7fb3a40544a0 con 0x557c6923af10 2024-09-15T01:32:05.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:05.072+0000 7fb3b1ffb640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:05.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:05.072+0000 7fb3b1ffb640 1 -- 172.21.15.178:0/769777824 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v18) v4 -- 0x7fb38c003360 con 0x557c69144090 2024-09-15T01:32:06.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.070+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 <== mon.7 v2:172.21.15.204:3303/0 11 ==== pool_op_reply(tid 6 (0) Success v19) v1 ==== 43+0+0 (secure 0 0 0) 0x7fb3a405a030 con 0x557c69144090 2024-09-15T01:32:06.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.070+0000 7fb3abfff640 10 monclient: _renew_subs 2024-09-15T01:32:06.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.070+0000 7fb3abfff640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:06.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.070+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=19}) v3 -- 0x7fb394002830 con 0x557c69144090 2024-09-15T01:32:06.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.071+0000 7fb3abfff640 1 -- 172.21.15.178:0/769777824 <== mon.7 v2:172.21.15.204:3303/0 12 ==== osd_map(19..19 src has 1..19) v4 ==== 646+0+0 (secure 0 0 0) 0x7fb3a408db20 con 0x557c69144090 2024-09-15T01:32:06.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.071+0000 7fb3b27fc640 1 -- 172.21.15.178:0/769777824 --> v2:172.21.15.178:6800/29232 -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19) v8 -- 0x7fb398005990 con 0x557c6923af10 2024-09-15T01:32:06.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.073+0000 7fb3b37fe640 1 -- 172.21.15.178:0/769777824 <== osd.0 v2:172.21.15.178:6800/29232 4 ==== osd_op_reply(7 rbd_info [write 0~19] v19'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3a40544a0 con 0x557c6923af10 2024-09-15T01:32:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 >> v2:172.21.15.178:6800/29232 conn(0x557c6923af10 msgr2=0x557c6923d390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 --2- 172.21.15.178:0/769777824 >> v2:172.21.15.178:6800/29232 conn(0x557c6923af10 0x557c6923d390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 >> v2:172.21.15.178:6804/29233 conn(0x557c691eb640 msgr2=0x557c691eda20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 --2- 172.21.15.178:0/769777824 >> v2:172.21.15.178:6804/29233 conn(0x557c691eb640 0x557c691eda20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 >> v2:172.21.15.204:6812/28210 conn(0x7fb3940380e0 msgr2=0x7fb39403a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 --2- 172.21.15.178:0/769777824 >> v2:172.21.15.204:6812/28210 conn(0x7fb3940380e0 0x7fb39403a590 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009c40 tx=0x7fb39c00c040 comp rx=0 tx=0).stop 2024-09-15T01:32:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3b3fff640 1 -- 172.21.15.178:0/769777824 reap_dead start 2024-09-15T01:32:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 10 monclient: shutdown 2024-09-15T01:32:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x557c69144090 msgr2=0x557c6913b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 --2- 172.21.15.178:0/769777824 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x557c69144090 0x557c6913b1c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4000c00 tx=0x7fb3a4058000 comp rx=0 tx=0).stop 2024-09-15T01:32:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 shutdown_connections 2024-09-15T01:32:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 --2- 172.21.15.178:0/769777824 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x557c69144090 0x557c6913b1c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 >> 172.21.15.178:0/769777824 conn(0x557c691459a0 msgr2=0x557c691471d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 shutdown_connections 2024-09-15T01:32:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.074+0000 7fb3ba237b80 1 -- 172.21.15.178:0/769777824 wait complete. 2024-09-15T01:32:06.081 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T01:32:06.082 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T01:32:06.082 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T01:32:06.082 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T01:32:06.083 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T01:32:06.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.178+0000 7fd6640ef640 10 monclient: get_monmap_and_config 2024-09-15T01:32:06.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.178+0000 7fd6640ef640 10 monclient: build_initial_monmap 2024-09-15T01:32:06.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.178+0000 7fd6640ef640 10 monclient: monmap: 2024-09-15T01:32:06.181 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:06.181 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.181 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:06.179947+0000 2024-09-15T01:32:06.181 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:06.179947+0000 2024-09-15T01:32:06.181 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.178+0000 7fd6640ef640 1 Processor -- start 2024-09-15T01:32:06.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.178+0000 7fd6640ef640 1 -- start start 2024-09-15T01:32:06.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.178+0000 7fd6640ef640 10 monclient: init 2024-09-15T01:32:06.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:06.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 10 monclient: _add_conns ranks=[0,5,6,4,9,10,2,1,7,8,3] 2024-09-15T01:32:06.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd65c067130 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:06.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd65c09d580 0x7fd65c135bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 10 monclient(hunting): picked mon.noname-h con 0x7fd65c09d580 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd65c09f000 0x7fd65c138120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 10 monclient(hunting): picked mon.noname-d con 0x7fd65c09f000 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c04bf60 con 0x7fd65c067130 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c09e710 con 0x7fd65c09d580 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c09e0c0 con 0x7fd65c09f000 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd6640ef640 10 monclient(hunting): authenticate will time out at 730.930342s 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd661e8b640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd65c067130 0x7fd65c067500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:59172/0 (socket says 172.21.15.178:59172) 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd661e8b640 1 -- 172.21.15.178:0/1648536412 learned_addr learned my addr 172.21.15.178:0/1648536412 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 1 --2- 172.21.15.178:0/1648536412 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd65c09f000 0x7fd65c138120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 10 monclient(hunting): get_auth_request con 0x7fd65c09f000 auth_method 0 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66168a640 1 --2- 172.21.15.178:0/1648536412 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd65c09d580 0x7fd65c135bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66168a640 10 monclient(hunting): get_auth_request con 0x7fd65c09d580 auth_method 0 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66168a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66168a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66168a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 10 monclient(hunting): handle_auth_done global_id 4290 payload 274 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd660e89640 1 -- 172.21.15.178:0/1648536412 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964751462 0 0) 0x7fd65c04bf60 con 0x7fd65c067130 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 1 -- 172.21.15.178:0/1648536412 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd65c09d580 msgr2=0x7fd65c135bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 1 --2- 172.21.15.178:0/1648536412 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd65c09d580 0x7fd65c135bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 1 -- 172.21.15.178:0/1648536412 >> v1:172.21.15.178:6789/0 conn(0x7fd65c067130 legacy=0x7fd65c067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 1 monclient: found mon.noname-d 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd66268c640 1 -- 172.21.15.178:0/1648536412 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c09dde0 con 0x7fd65c09f000 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.179+0000 7fd660e89640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd66268c640 1 --2- 172.21.15.178:0/1648536412 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd65c09f000 0x7fd65c138120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd658009e90 tx=0x7fd658009780 comp rx=0 tx=0).ready entity=mon.6 client_cookie=789a4cddd1f19620 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd660e89640 1 -- 172.21.15.178:0/1648536412 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd658031880 con 0x7fd65c09f000 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd660e89640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd660e89640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd660e89640 10 monclient: dump: 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.193 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd660e89640 10 monclient: _finish_auth 0 2024-09-15T01:32:06.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd660e89640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd6640ef640 5 monclient: authenticate success, global_id 4290 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd660e89640 1 -- 172.21.15.178:0/1648536412 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65802d050 con 0x7fd65c09f000 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd6640ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd660e89640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd6640ef640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd6640ef640 4 set_mon_vals no callback set 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd660e89640 1 -- 172.21.15.178:0/1648536412 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6580303a0 con 0x7fd65c09f000 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd660e89640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd660e89640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.180+0000 7fd660e89640 10 monclient: dump: 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.195 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.181+0000 7fd6640ef640 10 monclient: shutdown 2024-09-15T01:32:06.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.181+0000 7fd6640ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.181+0000 7fd6640ef640 1 -- 172.21.15.178:0/1648536412 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd65c09f000 msgr2=0x7fd65c138120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.181+0000 7fd6640ef640 1 --2- 172.21.15.178:0/1648536412 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd65c09f000 0x7fd65c138120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd658009e90 tx=0x7fd658009780 comp rx=0 tx=0).stop 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.181+0000 7fd6640ef640 1 -- 172.21.15.178:0/1648536412 shutdown_connections 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.181+0000 7fd6640ef640 1 --2- 172.21.15.178:0/1648536412 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd65c09f000 0x7fd65c138120 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.181+0000 7fd6640ef640 1 --2- 172.21.15.178:0/1648536412 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd65c09d580 0x7fd65c135bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.181+0000 7fd6640ef640 1 -- 172.21.15.178:0/1648536412 >> 172.21.15.178:0/1648536412 conn(0x7fd65c068a60 msgr2=0x7fd65c0a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.181+0000 7fd6640ef640 1 -- 172.21.15.178:0/1648536412 shutdown_connections 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.181+0000 7fd6640ef640 1 -- 172.21.15.178:0/1648536412 wait complete. 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.181+0000 7fd6640ef640 10 monclient: build_initial_monmap 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.181+0000 7fd6640ef640 10 monclient: monmap: 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:06.197 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 1 Processor -- start 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 1 -- start start 2024-09-15T01:32:06.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 10 monclient: init 2024-09-15T01:32:06.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:06.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 10 monclient: _add_conns ranks=[7,8,3,4,1,6,0,5,2] 2024-09-15T01:32:06.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd65c067130 0x7fd65c141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 10 monclient(hunting): picked mon.noname-i con 0x7fd65c067130 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:06.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd65c09d580 0x7fd65c1421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 10 monclient(hunting): picked mon.noname-e con 0x7fd65c09d580 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:06.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd65c09f000 0x7fd65c06b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 10 monclient(hunting): picked mon.noname-g con 0x7fd65c09f000 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c11e9d0 con 0x7fd65c09f000 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c09d9e0 con 0x7fd65c067130 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c09cb50 con 0x7fd65c09d580 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd6640ef640 10 monclient(hunting): authenticate will time out at 730.933607s 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66168a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd65c09d580 0x7fd65c1421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66168a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd65c09d580 0x7fd65c1421b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:51870/0 (socket says 172.21.15.178:51870) 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66168a640 1 -- 172.21.15.178:0/3114424211 learned_addr learned my addr 172.21.15.178:0/3114424211 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66168a640 10 monclient(hunting): get_auth_request con 0x7fd65c09d580 auth_method 0 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66168a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66168a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66168a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66268c640 1 --2- 172.21.15.178:0/3114424211 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd65c09f000 0x7fd65c06b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd661e8b640 1 --2- 172.21.15.178:0/3114424211 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd65c067130 0x7fd65c141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66168a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:06.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66168a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:06.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66168a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:06.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66268c640 10 monclient(hunting): get_auth_request con 0x7fd65c09f000 auth_method 0 2024-09-15T01:32:06.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66268c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66268c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66268c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd661e8b640 10 monclient(hunting): get_auth_request con 0x7fd65c067130 auth_method 0 2024-09-15T01:32:06.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd661e8b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd661e8b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd661e8b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66268c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66268c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd66268c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd661e8b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd661e8b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.182+0000 7fd661e8b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd66168a640 10 monclient(hunting): handle_auth_done global_id 4256 payload 824 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd66168a640 1 -- 172.21.15.178:0/3114424211 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd65c067130 msgr2=0x7fd65c141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd66168a640 1 --2- 172.21.15.178:0/3114424211 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd65c067130 0x7fd65c141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd66168a640 1 -- 172.21.15.178:0/3114424211 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd65c09f000 msgr2=0x7fd65c06b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd66168a640 1 --2- 172.21.15.178:0/3114424211 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd65c09f000 0x7fd65c06b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd66168a640 10 monclient: _finish_hunting 0 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd66168a640 1 monclient: found mon.noname-e 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd66168a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd66168a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd66168a640 1 -- 172.21.15.178:0/3114424211 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd658009a20 con 0x7fd65c09d580 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd66168a640 1 --2- 172.21.15.178:0/3114424211 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd65c09d580 0x7fd65c1421b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00d8c0 tx=0x7fd64c008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd651ffb640 1 -- 172.21.15.178:0/3114424211 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd64c00de50 con 0x7fd65c09d580 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd651ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd651ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd651ffb640 10 monclient: dump: 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.203 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd651ffb640 10 monclient: _finish_auth 0 2024-09-15T01:32:06.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd651ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd651ffb640 1 -- 172.21.15.178:0/3114424211 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c00bd50 con 0x7fd65c09d580 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd651ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd651ffb640 1 -- 172.21.15.178:0/3114424211 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd64c0148b0 con 0x7fd65c09d580 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd6640ef640 5 monclient: authenticate success, global_id 4256 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd6640ef640 10 monclient: _renew_subs 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd6640ef640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd6640ef640 1 -- 172.21.15.178:0/3114424211 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c11d800 con 0x7fd65c09d580 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd653fff640 4 set_mon_vals no callback set 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd651ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd651ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd651ffb640 10 monclient: dump: 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd6640ef640 10 monclient: _renew_subs 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd6640ef640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.183+0000 7fd6640ef640 1 -- 172.21.15.178:0/3114424211 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c06cc90 con 0x7fd65c09d580 2024-09-15T01:32:06.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.184+0000 7fd651ffb640 1 -- 172.21.15.178:0/3114424211 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd64c014430 con 0x7fd65c09d580 2024-09-15T01:32:06.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.184+0000 7fd651ffb640 1 --2- 172.21.15.178:0/3114424211 >> v2:172.21.15.204:6812/28210 conn(0x7fd6400396e0 0x7fd64003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.184+0000 7fd6640ef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:06.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.184+0000 7fd6640ef640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:06.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.184+0000 7fd6640ef640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:06.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.184+0000 7fd651ffb640 1 -- 172.21.15.178:0/3114424211 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7fd64c04b660 con 0x7fd65c09d580 2024-09-15T01:32:06.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.184+0000 7fd6640ef640 1 -- 172.21.15.178:0/3114424211 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c04b720 con 0x7fd65c09d580 2024-09-15T01:32:06.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.185+0000 7fd661e8b640 1 --2- 172.21.15.178:0/3114424211 >> v2:172.21.15.204:6812/28210 conn(0x7fd6400396e0 0x7fd64003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.185+0000 7fd661e8b640 10 monclient: get_auth_request con 0x7fd6400396e0 auth_method 0 2024-09-15T01:32:06.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.185+0000 7fd661e8b640 1 --2- 172.21.15.178:0/3114424211 >> v2:172.21.15.204:6812/28210 conn(0x7fd6400396e0 0x7fd64003bb90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd6540011e0 tx=0x7fd654007c80 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:06.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.189+0000 7fd651ffb640 1 -- 172.21.15.178:0/3114424211 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd64c014a30 con 0x7fd65c09d580 2024-09-15T01:32:06.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.189+0000 7fd651ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:06.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.189+0000 7fd651ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:06.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.381+0000 7fd6640ef640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2024-09-15T01:32:06.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.381+0000 7fd6640ef640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2024-09-15T01:32:06.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.381+0000 7fd6640ef640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:06.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.381+0000 7fd6640ef640 1 -- 172.21.15.178:0/3114424211 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd65c070770 con 0x7fd65c09d580 2024-09-15T01:32:06.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.383+0000 7fd651ffb640 1 -- 172.21.15.178:0/3114424211 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+87608 (secure 0 0 0) 0x7fd64c011340 con 0x7fd65c09d580 2024-09-15T01:32:06.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.383+0000 7fd651ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2024-09-15T01:32:06.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.383+0000 7fd651ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:06.387 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:32:06.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 1 -- 172.21.15.178:0/3114424211 >> v2:172.21.15.204:6812/28210 conn(0x7fd6400396e0 msgr2=0x7fd64003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 1 --2- 172.21.15.178:0/3114424211 >> v2:172.21.15.204:6812/28210 conn(0x7fd6400396e0 0x7fd64003bb90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd6540011e0 tx=0x7fd654007c80 comp rx=0 tx=0).stop 2024-09-15T01:32:06.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 10 monclient: shutdown 2024-09-15T01:32:06.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:06.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 1 -- 172.21.15.178:0/3114424211 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd65c09d580 msgr2=0x7fd65c1421b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 1 --2- 172.21.15.178:0/3114424211 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd65c09d580 0x7fd65c1421b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00d8c0 tx=0x7fd64c008040 comp rx=0 tx=0).stop 2024-09-15T01:32:06.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 1 -- 172.21.15.178:0/3114424211 shutdown_connections 2024-09-15T01:32:06.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 1 --2- 172.21.15.178:0/3114424211 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd65c09f000 0x7fd65c06b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 1 --2- 172.21.15.178:0/3114424211 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd65c09d580 0x7fd65c1421b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 1 --2- 172.21.15.178:0/3114424211 >> v2:172.21.15.204:6812/28210 conn(0x7fd6400396e0 0x7fd64003bb90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 1 --2- 172.21.15.178:0/3114424211 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd65c067130 0x7fd65c141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 1 -- 172.21.15.178:0/3114424211 >> 172.21.15.178:0/3114424211 conn(0x7fd65c068a60 msgr2=0x7fd65c0a2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:06.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 1 -- 172.21.15.178:0/3114424211 shutdown_connections 2024-09-15T01:32:06.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.385+0000 7fd6640ef640 1 -- 172.21.15.178:0/3114424211 wait complete. 2024-09-15T01:32:06.398 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":5,"flags":0,"active_gid":4121,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":28210}]},"active_addr":"172.21.15.204:6812/28210","active_change":"2024-09-15T01:31:56.148501+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":2185353560}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":4288662165}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":981305801}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":3299400510}]}]} 2024-09-15T01:32:06.400 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T01:32:06.400 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T01:32:06.400 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:32:06.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.490+0000 7f9a9e690640 10 monclient: get_monmap_and_config 2024-09-15T01:32:06.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.490+0000 7f9a9e690640 10 monclient: build_initial_monmap 2024-09-15T01:32:06.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.490+0000 7f9a9e690640 10 monclient: monmap: 2024-09-15T01:32:06.493 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:06.493 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.493 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:06.492033+0000 2024-09-15T01:32:06.493 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:06.492033+0000 2024-09-15T01:32:06.493 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:06.493 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.493 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:06.493 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:06.493 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:06.494 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:06.494 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:06.494 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:06.494 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:06.494 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:06.494 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:06.494 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:06.494 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:06.494 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.490+0000 7f9a9e690640 1 Processor -- start 2024-09-15T01:32:06.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.490+0000 7f9a9e690640 1 -- start start 2024-09-15T01:32:06.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.490+0000 7f9a9e690640 10 monclient: init 2024-09-15T01:32:06.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:06.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 10 monclient: _add_conns ranks=[8,3,10,4,9,6,7,1,5,2,0] 2024-09-15T01:32:06.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 0x7f9a9809e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 10 monclient(hunting): picked mon.noname-e con 0x7f9a980678d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a9806ea30 0x7f9a9809ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 10 monclient(hunting): picked mon.noname-g con 0x7f9a9806ea30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9a9809f180 0x7f9a981306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 10 monclient(hunting): picked mon.noname-f con 0x7f9a9809f180 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9809f590 con 0x7f9a9809f180 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9806d5b0 con 0x7f9a9806ea30 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9a98067ca0 con 0x7f9a980678d0 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9e690640 10 monclient(hunting): authenticate will time out at 731.242392s 2024-09-15T01:32:06.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 0x7f9a9809e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 0x7f9a9809e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:51886/0 (socket says 172.21.15.178:51886) 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 1 -- 172.21.15.178:0/1995611598 learned_addr learned my addr 172.21.15.178:0/1995611598 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 10 monclient(hunting): get_auth_request con 0x7f9a980678d0 auth_method 0 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a977fe640 1 --2- 172.21.15.178:0/1995611598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a9806ea30 0x7f9a9809ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9cc2d640 1 --2- 172.21.15.178:0/1995611598 >> v2:172.21.15.204:3300/0 conn(0x7f9a9809f180 0x7f9a981306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a977fe640 10 monclient(hunting): get_auth_request con 0x7f9a9806ea30 auth_method 0 2024-09-15T01:32:06.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a977fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9cc2d640 10 monclient(hunting): get_auth_request con 0x7f9a9809f180 auth_method 0 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9cc2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9cc2d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9cc2d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 10 monclient(hunting): handle_auth_done global_id 4265 payload 274 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 1 -- 172.21.15.178:0/1995611598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a9806ea30 msgr2=0x7f9a9809ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 1 --2- 172.21.15.178:0/1995611598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a9806ea30 0x7f9a9809ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 1 -- 172.21.15.178:0/1995611598 >> v2:172.21.15.204:3300/0 conn(0x7f9a9809f180 msgr2=0x7f9a981306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 1 --2- 172.21.15.178:0/1995611598 >> v2:172.21.15.204:3300/0 conn(0x7f9a9809f180 0x7f9a981306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 10 monclient: _finish_hunting 0 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 1 monclient: found mon.noname-e 2024-09-15T01:32:06.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a97fff640 1 -- 172.21.15.178:0/1995611598 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9804bf60 con 0x7f9a980678d0 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a977fe640 1 --2- 172.21.15.178:0/1995611598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a9806ea30 0x7f9a9809ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.491+0000 7f9a9cc2d640 1 --2- 172.21.15.178:0/1995611598 >> v2:172.21.15.204:3300/0 conn(0x7f9a9809f180 0x7f9a981306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a97fff640 1 --2- 172.21.15.178:0/1995611598 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 0x7f9a9809e700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009940 tx=0x7f9a880091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=902548c2bf04ef1f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a96ffd640 1 -- 172.21.15.178:0/1995611598 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a8802ad00 con 0x7f9a980678d0 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a96ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a96ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a96ffd640 10 monclient: dump: 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.507 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a96ffd640 10 monclient: _finish_auth 0 2024-09-15T01:32:06.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a96ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a9e690640 5 monclient: authenticate success, global_id 4265 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a96ffd640 1 -- 172.21.15.178:0/1995611598 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8802ae80 con 0x7f9a980678d0 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a9e690640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a96ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a9e690640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a9e690640 4 set_mon_vals no callback set 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a96ffd640 1 -- 172.21.15.178:0/1995611598 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a88031ca0 con 0x7f9a980678d0 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a96ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a96ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.492+0000 7f9a96ffd640 10 monclient: dump: 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.509 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.510 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:06.510 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.510 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:06.510 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:06.510 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:06.510 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:06.510 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:06.510 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:06.510 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:06.510 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:06.510 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 10 monclient: shutdown 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 1 -- 172.21.15.178:0/1995611598 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 msgr2=0x7f9a9809e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 1 --2- 172.21.15.178:0/1995611598 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 0x7f9a9809e700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009940 tx=0x7f9a880091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 1 -- 172.21.15.178:0/1995611598 shutdown_connections 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 1 --2- 172.21.15.178:0/1995611598 >> v2:172.21.15.204:3300/0 conn(0x7f9a9809f180 0x7f9a981306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 1 --2- 172.21.15.178:0/1995611598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a9806ea30 0x7f9a9809ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 1 --2- 172.21.15.178:0/1995611598 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 0x7f9a9809e700 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 1 -- 172.21.15.178:0/1995611598 >> 172.21.15.178:0/1995611598 conn(0x7f9a980a4390 msgr2=0x7f9a98067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 1 -- 172.21.15.178:0/1995611598 shutdown_connections 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 1 -- 172.21.15.178:0/1995611598 wait complete. 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 10 monclient: build_initial_monmap 2024-09-15T01:32:06.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.493+0000 7f9a9e690640 10 monclient: monmap: 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 1 Processor -- start 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 1 -- start start 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 10 monclient: init 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 10 monclient: _add_conns ranks=[8,5,3,4,6,1,7,0,2] 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 0x7f9a98141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 10 monclient(hunting): picked mon.noname-e con 0x7f9a980678d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a9806ea30 0x7f9a98133720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 10 monclient(hunting): picked mon.noname-h con 0x7f9a9806ea30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a9809f180 0x7f9a98068260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 10 monclient(hunting): picked mon.noname-g con 0x7f9a9809f180 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a98122b40 con 0x7f9a9809f180 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a981426d0 con 0x7f9a9806ea30 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9a98122860 con 0x7f9a980678d0 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9e690640 10 monclient(hunting): authenticate will time out at 731.245800s 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 0x7f9a98141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 0x7f9a98141860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:51896/0 (socket says 172.21.15.178:51896) 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a97fff640 1 -- 172.21.15.178:0/2477562176 learned_addr learned my addr 172.21.15.178:0/2477562176 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a97fff640 10 monclient(hunting): get_auth_request con 0x7f9a980678d0 auth_method 0 2024-09-15T01:32:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a97fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9cc2d640 1 --2- 172.21.15.178:0/2477562176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a9809f180 0x7f9a98068260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a977fe640 1 --2- 172.21.15.178:0/2477562176 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a9806ea30 0x7f9a98133720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9cc2d640 10 monclient(hunting): get_auth_request con 0x7f9a9809f180 auth_method 0 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9cc2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9cc2d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.494+0000 7f9a9cc2d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a977fe640 10 monclient(hunting): get_auth_request con 0x7f9a9806ea30 auth_method 0 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a977fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a9cc2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a9cc2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a9cc2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a97fff640 10 monclient(hunting): handle_auth_done global_id 4274 payload 824 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a97fff640 1 -- 172.21.15.178:0/2477562176 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a9806ea30 msgr2=0x7f9a98133720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a97fff640 1 --2- 172.21.15.178:0/2477562176 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a9806ea30 0x7f9a98133720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a97fff640 1 -- 172.21.15.178:0/2477562176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a9809f180 msgr2=0x7f9a98068260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a97fff640 1 --2- 172.21.15.178:0/2477562176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a9809f180 0x7f9a98068260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a97fff640 10 monclient: _finish_hunting 0 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a97fff640 1 monclient: found mon.noname-e 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a97fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a97fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a97fff640 1 -- 172.21.15.178:0/2477562176 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a88009450 con 0x7f9a980678d0 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a97fff640 1 --2- 172.21.15.178:0/2477562176 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 0x7f9a98141860 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9a880097e0 tx=0x7f9a88032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a7ffff640 1 -- 172.21.15.178:0/2477562176 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a88040070 con 0x7f9a980678d0 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a7ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a7ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a7ffff640 10 monclient: dump: 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.517 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a7ffff640 10 monclient: _finish_auth 0 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a7ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a9e690640 5 monclient: authenticate success, global_id 4274 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a9e690640 10 monclient: _renew_subs 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a7ffff640 1 -- 172.21.15.178:0/2477562176 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a88006b30 con 0x7f9a980678d0 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a9e690640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a9e690640 1 -- 172.21.15.178:0/2477562176 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a98122e20 con 0x7f9a980678d0 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a7ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a7ffff640 1 -- 172.21.15.178:0/2477562176 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a8802ad00 con 0x7f9a980678d0 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a7ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a7ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a967fc640 4 set_mon_vals no callback set 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a7ffff640 10 monclient: dump: 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a9e690640 10 monclient: _renew_subs 2024-09-15T01:32:06.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a9e690640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:06.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.495+0000 7f9a9e690640 1 -- 172.21.15.178:0/2477562176 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a98141da0 con 0x7f9a980678d0 2024-09-15T01:32:06.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.496+0000 7f9a7ffff640 1 -- 172.21.15.178:0/2477562176 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9a88031a60 con 0x7f9a980678d0 2024-09-15T01:32:06.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.496+0000 7f9a9e690640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:06.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.496+0000 7f9a9e690640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:06.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.496+0000 7f9a9e690640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:06.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.496+0000 7f9a7ffff640 1 --2- 172.21.15.178:0/2477562176 >> v2:172.21.15.204:6812/28210 conn(0x7f9a840397b0 0x7f9a8403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.496+0000 7f9a9e690640 1 -- 172.21.15.178:0/2477562176 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9805ed70 con 0x7f9a980678d0 2024-09-15T01:32:06.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.496+0000 7f9a7ffff640 1 -- 172.21.15.178:0/2477562176 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f9a8806aa40 con 0x7f9a980678d0 2024-09-15T01:32:06.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.496+0000 7f9a977fe640 1 --2- 172.21.15.178:0/2477562176 >> v2:172.21.15.204:6812/28210 conn(0x7f9a840397b0 0x7f9a8403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.496+0000 7f9a977fe640 10 monclient: get_auth_request con 0x7f9a840397b0 auth_method 0 2024-09-15T01:32:06.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.497+0000 7f9a977fe640 1 --2- 172.21.15.178:0/2477562176 >> v2:172.21.15.204:6812/28210 conn(0x7f9a840397b0 0x7f9a8403bc60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c006e70 tx=0x7f9a8c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:06.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.501+0000 7f9a7ffff640 1 -- 172.21.15.178:0/2477562176 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9a88037080 con 0x7f9a980678d0 2024-09-15T01:32:06.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.501+0000 7f9a7ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:06.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.501+0000 7f9a7ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:06.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.655+0000 7f9a9e690640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:32:06.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.655+0000 7f9a9e690640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:32:06.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.655+0000 7f9a9e690640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:06.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.655+0000 7f9a9e690640 1 -- 172.21.15.178:0/2477562176 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a9804b720 con 0x7f9a980678d0 2024-09-15T01:32:06.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.656+0000 7f9a7ffff640 1 -- 172.21.15.178:0/2477562176 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+10110 (secure 0 0 0) 0x7f9a88032e40 con 0x7f9a980678d0 2024-09-15T01:32:06.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.656+0000 7f9a7ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:32:06.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.656+0000 7f9a7ffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:06.659 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:32:06.659 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":19,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","created":"2024-09-15T01:31:51.657329+0000","modified":"2024-09-15T01:32:06.067435+0000","last_up_change":"2024-09-15T01:31:56.999458+0000","last_in_change":"2024-09-15T01:31:53.995822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:31:58.183767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:32:02.130544+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"690d2bc0-3a5e-4f4a-b00b-43a73741b7b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":29232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6801","nonce":29232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":29232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":29232}]},"public_addr":"172.21.15.178:6800/29232","cluster_addr":"172.21.15.178:6801/29232","heartbeat_back_addr":"172.21.15.178:6803/29232","heartbeat_front_addr":"172.21.15.178:6802/29232","state":["exists","up"]},{"osd":1,"uuid":"bbba9997-deb4-42ca-8140-6553a9b6c45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":29233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6805","nonce":29233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6807","nonce":29233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":29233}]},"public_addr":"172.21.15.178:6804/29233","cluster_addr":"172.21.15.178:6805/29233","heartbeat_back_addr":"172.21.15.178:6807/29233","heartbeat_front_addr":"172.21.15.178:6806/29233","state":["exists","up"]},{"osd":2,"uuid":"24b56e74-a222-40f9-a8bf-38d597553f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":29234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":29234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6811","nonce":29234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":29234}]},"public_addr":"172.21.15.178:6808/29234","cluster_addr":"172.21.15.178:6809/29234","heartbeat_back_addr":"172.21.15.178:6811/29234","heartbeat_front_addr":"172.21.15.178:6810/29234","state":["exists","up"]},{"osd":3,"uuid":"a5bf3f7d-4c59-470a-8f69-3fea018c13f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":28752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":28752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":28752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":28752}]},"public_addr":"172.21.15.204:6801/28752","cluster_addr":"172.21.15.204:6802/28752","heartbeat_back_addr":"172.21.15.204:6806/28752","heartbeat_front_addr":"172.21.15.204:6804/28752","state":["exists","up"]},{"osd":4,"uuid":"cdaf5723-3a5c-4238-a8e6-38f2325ccbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":28753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":28753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":28753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6805","nonce":28753}]},"public_addr":"172.21.15.204:6800/28753","cluster_addr":"172.21.15.204:6803/28753","heartbeat_back_addr":"172.21.15.204:6807/28753","heartbeat_front_addr":"172.21.15.204:6805/28753","state":["exists","up"]},{"osd":5,"uuid":"6b67035f-8718-4762-98bf-0dd0aa86e6e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":28754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":28754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":28754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":28754}]},"public_addr":"172.21.15.204:6808/28754","cluster_addr":"172.21.15.204:6809/28754","heartbeat_back_addr":"172.21.15.204:6811/28754","heartbeat_front_addr":"172.21.15.204:6810/28754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.093711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.117496+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.123704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.082038+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.088260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.090342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:32:06.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 1 -- 172.21.15.178:0/2477562176 >> v2:172.21.15.204:6812/28210 conn(0x7f9a840397b0 msgr2=0x7f9a8403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 1 --2- 172.21.15.178:0/2477562176 >> v2:172.21.15.204:6812/28210 conn(0x7f9a840397b0 0x7f9a8403bc60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c006e70 tx=0x7f9a8c008040 comp rx=0 tx=0).stop 2024-09-15T01:32:06.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 10 monclient: shutdown 2024-09-15T01:32:06.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:06.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 1 -- 172.21.15.178:0/2477562176 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 msgr2=0x7f9a98141860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 1 --2- 172.21.15.178:0/2477562176 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 0x7f9a98141860 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9a880097e0 tx=0x7f9a88032000 comp rx=0 tx=0).stop 2024-09-15T01:32:06.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 1 -- 172.21.15.178:0/2477562176 shutdown_connections 2024-09-15T01:32:06.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 1 --2- 172.21.15.178:0/2477562176 >> v2:172.21.15.204:6812/28210 conn(0x7f9a840397b0 0x7f9a8403bc60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 1 --2- 172.21.15.178:0/2477562176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a9809f180 0x7f9a98068260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 1 --2- 172.21.15.178:0/2477562176 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a9806ea30 0x7f9a98133720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 1 --2- 172.21.15.178:0/2477562176 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a980678d0 0x7f9a98141860 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 1 -- 172.21.15.178:0/2477562176 >> 172.21.15.178:0/2477562176 conn(0x7f9a980a4390 msgr2=0x7f9a9809c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:06.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 1 -- 172.21.15.178:0/2477562176 shutdown_connections 2024-09-15T01:32:06.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.658+0000 7f9a9e690640 1 -- 172.21.15.178:0/2477562176 wait complete. 2024-09-15T01:32:06.671 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T01:32:06.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:32:06.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.761+0000 7efef291d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:06.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.761+0000 7efef291d640 10 monclient: build_initial_monmap 2024-09-15T01:32:06.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.761+0000 7efef291d640 10 monclient: monmap: 2024-09-15T01:32:06.764 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:06.764 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.765 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:06.763068+0000 2024-09-15T01:32:06.765 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:06.763068+0000 2024-09-15T01:32:06.765 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:06.765 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.765 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:06.765 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:06.765 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:06.765 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:06.765 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:06.765 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:06.766 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:06.766 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:06.766 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:06.766 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:06.766 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:06.766 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.761+0000 7efef291d640 1 Processor -- start 2024-09-15T01:32:06.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.761+0000 7efef291d640 1 -- start start 2024-09-15T01:32:06.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.761+0000 7efef291d640 10 monclient: init 2024-09-15T01:32:06.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:06.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 10 monclient: _add_conns ranks=[4,9,5,1,10,6,8,2,0,7,3] 2024-09-15T01:32:06.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efeec0678d0 0x7efeec09e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 10 monclient(hunting): picked mon.noname-c con 0x7efeec0678d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:06.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efeec06ea30 0x7efeec09ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 10 monclient(hunting): picked mon.noname-a con 0x7efeec06ea30 addr v2:172.21.15.178:3300/0 2024-09-15T01:32:06.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efeec09f180 0x7efeec1306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 10 monclient(hunting): picked mon.noname-h con 0x7efeec09f180 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:06.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeec09f5c0 con 0x7efeec06ea30 2024-09-15T01:32:06.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efeec06d5b0 con 0x7efeec0678d0 2024-09-15T01:32:06.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efeec067ca0 con 0x7efeec09f180 2024-09-15T01:32:06.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:06.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef291d640 10 monclient(hunting): authenticate will time out at 731.513508s 2024-09-15T01:32:06.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeeb7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efeec06ea30 0x7efeec09ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efeec0678d0 0x7efeec09e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeeb7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efeec06ea30 0x7efeec09ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33192/0 (socket says 172.21.15.178:33192) 2024-09-15T01:32:06.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efeec0678d0 0x7efeec09e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:40206/0 (socket says 172.21.15.178:40206) 2024-09-15T01:32:06.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeeb7fe640 1 -- 172.21.15.178:0/2789638442 learned_addr learned my addr 172.21.15.178:0/2789638442 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:06.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeeb7fe640 10 monclient(hunting): get_auth_request con 0x7efeec06ea30 auth_method 0 2024-09-15T01:32:06.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeeb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeeb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeeb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 10 monclient(hunting): get_auth_request con 0x7efeec0678d0 auth_method 0 2024-09-15T01:32:06.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeeb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:06.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeeb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:06.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeeb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:06.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:06.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:06.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef0eba640 1 --2- 172.21.15.178:0/2789638442 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efeec09f180 0x7efeec1306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:06.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef0eba640 10 monclient(hunting): get_auth_request con 0x7efeec09f180 auth_method 0 2024-09-15T01:32:06.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef0eba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef0eba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef0eba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 10 monclient(hunting): handle_auth_done global_id 4207 payload 274 2024-09-15T01:32:06.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 1 -- 172.21.15.178:0/2789638442 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efeec09f180 msgr2=0x7efeec1306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:06.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 1 --2- 172.21.15.178:0/2789638442 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efeec09f180 0x7efeec1306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 1 -- 172.21.15.178:0/2789638442 >> v2:172.21.15.178:3300/0 conn(0x7efeec06ea30 msgr2=0x7efeec09ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:06.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 1 --2- 172.21.15.178:0/2789638442 >> v2:172.21.15.178:3300/0 conn(0x7efeec06ea30 0x7efeec09ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 10 monclient: _finish_hunting 0 2024-09-15T01:32:06.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 1 monclient: found mon.noname-c 2024-09-15T01:32:06.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:06.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:06.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeebfff640 1 -- 172.21.15.178:0/2789638442 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec04bf60 con 0x7efeec0678d0 2024-09-15T01:32:06.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efeeb7fe640 1 --2- 172.21.15.178:0/2789638442 >> v2:172.21.15.178:3300/0 conn(0x7efeec06ea30 0x7efeec09ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:06.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.762+0000 7efef0eba640 1 --2- 172.21.15.178:0/2789638442 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efeec09f180 0x7efeec1306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:06.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeebfff640 1 --2- 172.21.15.178:0/2789638442 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efeec0678d0 0x7efeec09e700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efedc009880 tx=0x7efedc0090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1c21cd806dee8727 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:06.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeeaffd640 1 -- 172.21.15.178:0/2789638442 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efedc02ac40 con 0x7efeec0678d0 2024-09-15T01:32:06.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeeaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:06.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeeaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:06.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeeaffd640 10 monclient: dump: 2024-09-15T01:32:06.784 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:06.784 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.784 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.784 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.785 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:06.785 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.785 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:06.785 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:06.785 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:06.785 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:06.785 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:06.785 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:06.785 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:06.785 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:06.786 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:06.786 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeeaffd640 10 monclient: _finish_auth 0 2024-09-15T01:32:06.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeeaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:06.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeeaffd640 1 -- 172.21.15.178:0/2789638442 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc02adc0 con 0x7efeec0678d0 2024-09-15T01:32:06.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efef291d640 5 monclient: authenticate success, global_id 4207 2024-09-15T01:32:06.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efef291d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:06.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeeaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:06.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeeaffd640 1 -- 172.21.15.178:0/2789638442 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efedc031ca0 con 0x7efeec0678d0 2024-09-15T01:32:06.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efef291d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:06.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efef291d640 4 set_mon_vals no callback set 2024-09-15T01:32:06.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeeaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:06.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeeaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:06.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.763+0000 7efeeaffd640 10 monclient: dump: 2024-09-15T01:32:06.787 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:06.787 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.787 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.787 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.787 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:06.787 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.788 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:06.788 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:06.788 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:06.788 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:06.788 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:06.788 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:06.788 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:06.788 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:06.788 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:06.788 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 10 monclient: shutdown 2024-09-15T01:32:06.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:06.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 1 -- 172.21.15.178:0/2789638442 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efeec0678d0 msgr2=0x7efeec09e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 1 --2- 172.21.15.178:0/2789638442 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efeec0678d0 0x7efeec09e700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efedc009880 tx=0x7efedc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 1 -- 172.21.15.178:0/2789638442 shutdown_connections 2024-09-15T01:32:06.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 1 --2- 172.21.15.178:0/2789638442 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efeec09f180 0x7efeec1306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 1 --2- 172.21.15.178:0/2789638442 >> v2:172.21.15.178:3300/0 conn(0x7efeec06ea30 0x7efeec09ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 1 --2- 172.21.15.178:0/2789638442 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efeec0678d0 0x7efeec09e700 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 1 -- 172.21.15.178:0/2789638442 >> 172.21.15.178:0/2789638442 conn(0x7efeec0a4390 msgr2=0x7efeec067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:06.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 1 -- 172.21.15.178:0/2789638442 shutdown_connections 2024-09-15T01:32:06.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 1 -- 172.21.15.178:0/2789638442 wait complete. 2024-09-15T01:32:06.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 10 monclient: build_initial_monmap 2024-09-15T01:32:06.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.764+0000 7efef291d640 10 monclient: monmap: 2024-09-15T01:32:06.790 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:06.790 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:06.790 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:06.790 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:06.790 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:06.790 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:06.791 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:06.791 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:06.791 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:06.791 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:06.791 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:06.791 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:06.791 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:06.791 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:06.791 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:06.791 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 1 Processor -- start 2024-09-15T01:32:06.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 1 -- start start 2024-09-15T01:32:06.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 10 monclient: init 2024-09-15T01:32:06.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:06.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 10 monclient: _add_conns ranks=[2,0,5,6,4,3,1,8,7] 2024-09-15T01:32:06.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efeec0678d0 0x7efeec1418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 10 monclient(hunting): picked mon.noname-b con 0x7efeec0678d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:06.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efeec06ea30 0x7efeec133730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 10 monclient(hunting): picked mon.noname-a con 0x7efeec06ea30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:06.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efeec09f180 0x7efeec068280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 10 monclient(hunting): picked mon.noname-h con 0x7efeec09f180 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:06.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec1427d0 con 0x7efeec06ea30 2024-09-15T01:32:06.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec06c900 con 0x7efeec0678d0 2024-09-15T01:32:06.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:06.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efeec122b50 con 0x7efeec09f180 2024-09-15T01:32:06.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:06.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efef291d640 10 monclient(hunting): authenticate will time out at 731.516826s 2024-09-15T01:32:06.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efeeb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efeec06ea30 0x7efeec133730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efeebfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efeec0678d0 0x7efeec1418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efeeb7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efeec06ea30 0x7efeec133730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33206/0 (socket says 172.21.15.178:33206) 2024-09-15T01:32:06.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efeebfff640 10 monclient(hunting): get_auth_request con 0x7efeec0678d0 auth_method 0 2024-09-15T01:32:06.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efeeb7fe640 1 -- 172.21.15.178:0/518463935 learned_addr learned my addr 172.21.15.178:0/518463935 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:06.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efeebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efeebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efeebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efeeb7fe640 10 monclient(hunting): get_auth_request con 0x7efeec06ea30 auth_method 0 2024-09-15T01:32:06.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efeeb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efeeb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.765+0000 7efeeb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:06.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:06.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:06.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:06.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:06.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:06.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efef0eba640 1 --2- 172.21.15.178:0/518463935 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efeec09f180 0x7efeec068280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efef0eba640 10 monclient(hunting): get_auth_request con 0x7efeec09f180 auth_method 0 2024-09-15T01:32:06.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efef0eba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:06.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efef0eba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:06.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efef0eba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:06.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 10 monclient(hunting): handle_auth_done global_id 4248 payload 824 2024-09-15T01:32:06.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 1 -- 172.21.15.178:0/518463935 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efeec09f180 msgr2=0x7efeec068280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 1 --2- 172.21.15.178:0/518463935 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efeec09f180 0x7efeec068280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 1 -- 172.21.15.178:0/518463935 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efeec0678d0 msgr2=0x7efeec1418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 1 --2- 172.21.15.178:0/518463935 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efeec0678d0 0x7efeec1418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 1 monclient: found mon.noname-a 2024-09-15T01:32:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 1 -- 172.21.15.178:0/518463935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed4009450 con 0x7efeec06ea30 2024-09-15T01:32:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeebfff640 1 --2- 172.21.15.178:0/518463935 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efeec0678d0 0x7efeec1418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeeb7fe640 1 --2- 172.21.15.178:0/518463935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efeec06ea30 0x7efeec133730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efed400ecb0 tx=0x7efed40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efee3fff640 1 -- 172.21.15.178:0/518463935 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efed4009730 con 0x7efeec06ea30 2024-09-15T01:32:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efee3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efee3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efee3fff640 10 monclient: dump: 2024-09-15T01:32:06.798 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:06.798 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.798 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.798 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.798 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:06.798 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.798 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:06.798 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:06.798 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:06.798 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:06.799 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:06.799 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:06.799 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:06.799 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:06.799 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:06.799 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efee3fff640 10 monclient: _finish_auth 0 2024-09-15T01:32:06.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efee3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:06.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efee3fff640 1 -- 172.21.15.178:0/518463935 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed4013470 con 0x7efeec06ea30 2024-09-15T01:32:06.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efef291d640 5 monclient: authenticate success, global_id 4248 2024-09-15T01:32:06.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efef291d640 10 monclient: _renew_subs 2024-09-15T01:32:06.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efef291d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:06.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efef291d640 1 -- 172.21.15.178:0/518463935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efedc009390 con 0x7efeec06ea30 2024-09-15T01:32:06.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efee3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:06.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efee3fff640 1 -- 172.21.15.178:0/518463935 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efed4013a60 con 0x7efeec06ea30 2024-09-15T01:32:06.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efee3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:06.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efee3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:06.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efee3fff640 10 monclient: dump: 2024-09-15T01:32:06.800 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:06.801 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:06.801 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.801 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:06.801 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:06.801 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:06.801 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:06.801 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:06.801 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:06.801 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:06.801 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:06.802 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:06.802 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:06.802 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:06.802 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:06.802 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:06.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efef291d640 10 monclient: _renew_subs 2024-09-15T01:32:06.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efef291d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:06.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efef291d640 1 -- 172.21.15.178:0/518463935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec141e10 con 0x7efeec06ea30 2024-09-15T01:32:06.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.766+0000 7efeea7fc640 4 set_mon_vals no callback set 2024-09-15T01:32:06.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.767+0000 7efee3fff640 1 -- 172.21.15.178:0/518463935 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7efed4013be0 con 0x7efeec06ea30 2024-09-15T01:32:06.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.767+0000 7efef291d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:06.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.767+0000 7efef291d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:06.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.767+0000 7efef291d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:06.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.767+0000 7efef291d640 1 -- 172.21.15.178:0/518463935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeec05ed70 con 0x7efeec06ea30 2024-09-15T01:32:06.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.767+0000 7efee3fff640 1 --2- 172.21.15.178:0/518463935 >> v2:172.21.15.204:6812/28210 conn(0x7efecc0397b0 0x7efecc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:06.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.767+0000 7efee3fff640 1 -- 172.21.15.178:0/518463935 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7efed404b330 con 0x7efeec06ea30 2024-09-15T01:32:06.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.767+0000 7efeebfff640 1 --2- 172.21.15.178:0/518463935 >> v2:172.21.15.204:6812/28210 conn(0x7efecc0397b0 0x7efecc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:06.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.767+0000 7efeebfff640 10 monclient: get_auth_request con 0x7efecc0397b0 auth_method 0 2024-09-15T01:32:06.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.768+0000 7efeebfff640 1 --2- 172.21.15.178:0/518463935 >> v2:172.21.15.204:6812/28210 conn(0x7efecc0397b0 0x7efecc03bc60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efedc034000 tx=0x7efedc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:06.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.772+0000 7efee3fff640 1 -- 172.21.15.178:0/518463935 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7efed40138c0 con 0x7efeec06ea30 2024-09-15T01:32:06.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.772+0000 7efee3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:06.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.772+0000 7efee3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:06.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.924+0000 7efef291d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:32:06.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.924+0000 7efef291d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:32:06.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.924+0000 7efef291d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:06.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.924+0000 7efef291d640 1 -- 172.21.15.178:0/518463935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efeec122cc0 con 0x7efeec06ea30 2024-09-15T01:32:06.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.924+0000 7efee3fff640 1 -- 172.21.15.178:0/518463935 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+10110 (secure 0 0 0) 0x7efed40192c0 con 0x7efeec06ea30 2024-09-15T01:32:06.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.924+0000 7efee3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:32:06.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.924+0000 7efee3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:06.928 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:32:06.928 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":19,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","created":"2024-09-15T01:31:51.657329+0000","modified":"2024-09-15T01:32:06.067435+0000","last_up_change":"2024-09-15T01:31:56.999458+0000","last_in_change":"2024-09-15T01:31:53.995822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:31:58.183767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:32:02.130544+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"690d2bc0-3a5e-4f4a-b00b-43a73741b7b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":29232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6801","nonce":29232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":29232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":29232}]},"public_addr":"172.21.15.178:6800/29232","cluster_addr":"172.21.15.178:6801/29232","heartbeat_back_addr":"172.21.15.178:6803/29232","heartbeat_front_addr":"172.21.15.178:6802/29232","state":["exists","up"]},{"osd":1,"uuid":"bbba9997-deb4-42ca-8140-6553a9b6c45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":29233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6805","nonce":29233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6807","nonce":29233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":29233}]},"public_addr":"172.21.15.178:6804/29233","cluster_addr":"172.21.15.178:6805/29233","heartbeat_back_addr":"172.21.15.178:6807/29233","heartbeat_front_addr":"172.21.15.178:6806/29233","state":["exists","up"]},{"osd":2,"uuid":"24b56e74-a222-40f9-a8bf-38d597553f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":29234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":29234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6811","nonce":29234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":29234}]},"public_addr":"172.21.15.178:6808/29234","cluster_addr":"172.21.15.178:6809/29234","heartbeat_back_addr":"172.21.15.178:6811/29234","heartbeat_front_addr":"172.21.15.178:6810/29234","state":["exists","up"]},{"osd":3,"uuid":"a5bf3f7d-4c59-470a-8f69-3fea018c13f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":28752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":28752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":28752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":28752}]},"public_addr":"172.21.15.204:6801/28752","cluster_addr":"172.21.15.204:6802/28752","heartbeat_back_addr":"172.21.15.204:6806/28752","heartbeat_front_addr":"172.21.15.204:6804/28752","state":["exists","up"]},{"osd":4,"uuid":"cdaf5723-3a5c-4238-a8e6-38f2325ccbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":28753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":28753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":28753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6805","nonce":28753}]},"public_addr":"172.21.15.204:6800/28753","cluster_addr":"172.21.15.204:6803/28753","heartbeat_back_addr":"172.21.15.204:6807/28753","heartbeat_front_addr":"172.21.15.204:6805/28753","state":["exists","up"]},{"osd":5,"uuid":"6b67035f-8718-4762-98bf-0dd0aa86e6e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":28754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":28754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":28754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":28754}]},"public_addr":"172.21.15.204:6808/28754","cluster_addr":"172.21.15.204:6809/28754","heartbeat_back_addr":"172.21.15.204:6811/28754","heartbeat_front_addr":"172.21.15.204:6810/28754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.093711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.117496+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.123704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.082038+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.088260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.090342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:32:06.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.926+0000 7efef291d640 1 -- 172.21.15.178:0/518463935 >> v2:172.21.15.204:6812/28210 conn(0x7efecc0397b0 msgr2=0x7efecc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.926+0000 7efef291d640 1 --2- 172.21.15.178:0/518463935 >> v2:172.21.15.204:6812/28210 conn(0x7efecc0397b0 0x7efecc03bc60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efedc034000 tx=0x7efedc032000 comp rx=0 tx=0).stop 2024-09-15T01:32:06.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.926+0000 7efef291d640 10 monclient: shutdown 2024-09-15T01:32:06.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.926+0000 7efef291d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:06.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.926+0000 7efef291d640 1 -- 172.21.15.178:0/518463935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efeec06ea30 msgr2=0x7efeec133730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:06.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.926+0000 7efef291d640 1 --2- 172.21.15.178:0/518463935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efeec06ea30 0x7efeec133730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efed400ecb0 tx=0x7efed40091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.926+0000 7efef291d640 1 -- 172.21.15.178:0/518463935 shutdown_connections 2024-09-15T01:32:06.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.926+0000 7efef291d640 1 --2- 172.21.15.178:0/518463935 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efeec09f180 0x7efeec068280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.926+0000 7efef291d640 1 --2- 172.21.15.178:0/518463935 >> v2:172.21.15.204:6812/28210 conn(0x7efecc0397b0 0x7efecc03bc60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.926+0000 7efef291d640 1 --2- 172.21.15.178:0/518463935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efeec06ea30 0x7efeec133730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.926+0000 7efef291d640 1 --2- 172.21.15.178:0/518463935 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efeec0678d0 0x7efeec1418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:06.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.926+0000 7efef291d640 1 -- 172.21.15.178:0/518463935 >> 172.21.15.178:0/518463935 conn(0x7efeec0a4390 msgr2=0x7efeec09b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:06.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.927+0000 7efef291d640 1 -- 172.21.15.178:0/518463935 shutdown_connections 2024-09-15T01:32:06.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:06.927+0000 7efef291d640 1 -- 172.21.15.178:0/518463935 wait complete. 2024-09-15T01:32:06.940 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T01:32:06.940 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T01:32:06.941 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T01:32:06.941 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T01:32:06.941 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T01:32:06.942 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T01:32:07.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.045+0000 7f1f36c20640 10 monclient: get_monmap_and_config 2024-09-15T01:32:07.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.045+0000 7f078a95d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:07.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.045+0000 7f078a95d640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.045+0000 7f078a95d640 10 monclient: monmap: 2024-09-15T01:32:07.049 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.049 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.049 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:07.047629+0000 2024-09-15T01:32:07.049 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:07.047629+0000 2024-09-15T01:32:07.049 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.049 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.049 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:07.049 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:07.050 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.050 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.050 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.050 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.050 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.050 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.050 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.050 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:07.050 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:07.050 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.045+0000 7f1f36c20640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.045+0000 7f1f36c20640 10 monclient: monmap: 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:07.047627+0000 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:07.047627+0000 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.052 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.052 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.052 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.052 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:07.052 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:07.052 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.046+0000 7f078a95d640 1 Processor -- start 2024-09-15T01:32:07.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.046+0000 7f1f36c20640 1 Processor -- start 2024-09-15T01:32:07.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.046+0000 7f1f36c20640 1 -- start start 2024-09-15T01:32:07.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.046+0000 7f1f36c20640 10 monclient: init 2024-09-15T01:32:07.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.046+0000 7f078a95d640 1 -- start start 2024-09-15T01:32:07.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.046+0000 7f078a95d640 10 monclient: init 2024-09-15T01:32:07.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 10 monclient: _add_conns ranks=[3,1,4,5,2,10,9,6,8,7,0] 2024-09-15T01:32:07.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f300a2930 0x7f1f30132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 10 monclient(hunting): picked mon.noname-g con 0x7f1f300a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:07.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1f3009faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:07.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 10 monclient: _add_conns ranks=[2,4,5,8,10,9,7,6,0,1,3] 2024-09-15T01:32:07.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07840a2930 0x7f0784132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f3009fe70 0x7f1f300a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 10 monclient(hunting): picked mon.noname-c con 0x7f1f3009fe70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f3004bf60 con 0x7f1f3009faa0 2024-09-15T01:32:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3009dd80 con 0x7f1f300a2930 2024-09-15T01:32:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f300a2d00 con 0x7f1f3009fe70 2024-09-15T01:32:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f36c20640 10 monclient(hunting): authenticate will time out at 731.798398s 2024-09-15T01:32:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 10 monclient(hunting): picked mon.noname-b con 0x7f07840a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f351bd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f3009fe70 0x7f1f300a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f2ffff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f1f3009faa0 0x7f1f300a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:44284/0 (socket says 172.21.15.178:44284) 2024-09-15T01:32:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f2ffff640 1 -- 172.21.15.178:0/3637211126 learned_addr learned my addr 172.21.15.178:0/3637211126 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f078409faa0 0x7f078409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 10 monclient(hunting): picked mon.noname-c con 0x7f078409faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07840a08c0 0x7f07840a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 10 monclient(hunting): picked mon.noname-h con 0x7f07840a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078409e0f0 con 0x7f07840a2930 2024-09-15T01:32:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f078409dd80 con 0x7f078409faa0 2024-09-15T01:32:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07840a2d00 con 0x7f07840a08c0 2024-09-15T01:32:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0783fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07840a2930 0x7f0784132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f078a95d640 10 monclient(hunting): authenticate will time out at 731.798659s 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f07837fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f078409faa0 0x7f078409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f07837fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f078409faa0 0x7f078409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:40226/0 (socket says 172.21.15.178:40226) 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0783fff640 10 monclient(hunting): get_auth_request con 0x7f07840a2930 auth_method 0 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0783fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0783fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0783fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f07837fe640 1 -- 172.21.15.178:0/2256635707 learned_addr learned my addr 172.21.15.178:0/2256635707 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f07837fe640 10 monclient(hunting): get_auth_request con 0x7f078409faa0 auth_method 0 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f07837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f07837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f07837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0788efa640 1 --2- 172.21.15.178:0/2256635707 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07840a08c0 0x7f07840a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0788efa640 10 monclient(hunting): get_auth_request con 0x7f07840a08c0 auth_method 0 2024-09-15T01:32:07.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0788efa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0788efa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0788efa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f1f349bc640 1 --2- 172.21.15.178:0/3637211126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f300a2930 0x7f1f30132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f07837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f07837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f07837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0788efa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0788efa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.047+0000 7f0788efa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 10 monclient(hunting): get_auth_request con 0x7f1f3009fe70 auth_method 0 2024-09-15T01:32:07.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0783fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0783fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f349bc640 10 monclient(hunting): get_auth_request con 0x7f1f300a2930 auth_method 0 2024-09-15T01:32:07.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f349bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f349bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f349bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0783fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0788efa640 10 monclient(hunting): handle_auth_done global_id 4325 payload 274 2024-09-15T01:32:07.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0788efa640 1 -- 172.21.15.178:0/2256635707 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f078409faa0 msgr2=0x7f078409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0788efa640 1 --2- 172.21.15.178:0/2256635707 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f078409faa0 0x7f078409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f2f7fe640 1 -- 172.21.15.178:0/3637211126 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374097717 0 0) 0x7f1f3004bf60 con 0x7f1f3009faa0 2024-09-15T01:32:07.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f349bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f349bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f349bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0788efa640 1 -- 172.21.15.178:0/2256635707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07840a2930 msgr2=0x7f0784132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f2f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f2f7fe640 10 monclient(hunting): my global_id is 4222 2024-09-15T01:32:07.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f2f7fe640 1 -- 172.21.15.178:0/3637211126 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f200034e0 con 0x7f1f3009faa0 2024-09-15T01:32:07.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0788efa640 1 --2- 172.21.15.178:0/2256635707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07840a2930 0x7f0784132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 10 monclient(hunting): handle_auth_done global_id 4225 payload 274 2024-09-15T01:32:07.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 1 -- 172.21.15.178:0/3637211126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f300a2930 msgr2=0x7f1f30132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 1 --2- 172.21.15.178:0/3637211126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f300a2930 0x7f1f30132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 1 -- 172.21.15.178:0/3637211126 >> v1:172.21.15.204:6789/0 conn(0x7f1f3009faa0 legacy=0x7f1f300a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 1 monclient: found mon.noname-c 2024-09-15T01:32:07.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:07.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f351bd640 1 -- 172.21.15.178:0/3637211126 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f300a0310 con 0x7f1f3009fe70 2024-09-15T01:32:07.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f1f349bc640 1 --2- 172.21.15.178:0/3637211126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f300a2930 0x7f1f30132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0788efa640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0788efa640 1 monclient: found mon.noname-h 2024-09-15T01:32:07.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0788efa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0788efa640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:32:07.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0788efa640 1 -- 172.21.15.178:0/2256635707 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078404bf60 con 0x7f07840a08c0 2024-09-15T01:32:07.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f07837fe640 1 --2- 172.21.15.178:0/2256635707 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f078409faa0 0x7f078409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.048+0000 7f0783fff640 1 --2- 172.21.15.178:0/2256635707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07840a2930 0x7f0784132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f351bd640 1 --2- 172.21.15.178:0/3637211126 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f3009fe70 0x7f1f300a0fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009e90 tx=0x7f1f28009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=cf1764805534338f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f2f7fe640 1 -- 172.21.15.178:0/3637211126 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f28031880 con 0x7f1f3009fe70 2024-09-15T01:32:07.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f2f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f0788efa640 1 --2- 172.21.15.178:0/2256635707 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07840a08c0 0x7f07840a0d30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f077c009e90 tx=0x7f077c009780 comp rx=0 tx=0).ready entity=mon.5 client_cookie=82eb7698d5c2a36a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f2f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:07.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f2f7fe640 10 monclient: dump: 2024-09-15T01:32:07.071 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.071 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.071 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.071 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.071 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.071 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f2f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f2f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f2f7fe640 1 -- 172.21.15.178:0/3637211126 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2802d050 con 0x7f1f3009fe70 2024-09-15T01:32:07.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f36c20640 5 monclient: authenticate success, global_id 4225 2024-09-15T01:32:07.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f36c20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:07.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f2f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f2f7fe640 1 -- 172.21.15.178:0/3637211126 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f280303a0 con 0x7f1f3009fe70 2024-09-15T01:32:07.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f36c20640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:07.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f36c20640 4 set_mon_vals no callback set 2024-09-15T01:32:07.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f2f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f2f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:07.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.049+0000 7f1f2f7fe640 10 monclient: dump: 2024-09-15T01:32:07.073 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.074 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.075 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.075 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f1f36c20640 10 monclient: shutdown 2024-09-15T01:32:07.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f1f36c20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f1f36c20640 1 -- 172.21.15.178:0/3637211126 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f3009fe70 msgr2=0x7f1f300a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f1f36c20640 1 --2- 172.21.15.178:0/3637211126 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f3009fe70 0x7f1f300a0fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009e90 tx=0x7f1f28009780 comp rx=0 tx=0).stop 2024-09-15T01:32:07.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f1f36c20640 1 -- 172.21.15.178:0/3637211126 shutdown_connections 2024-09-15T01:32:07.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f1f36c20640 1 --2- 172.21.15.178:0/3637211126 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f3009fe70 0x7f1f300a0fd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f1f36c20640 1 --2- 172.21.15.178:0/3637211126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f300a2930 0x7f1f30132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f1f36c20640 1 -- 172.21.15.178:0/3637211126 >> 172.21.15.178:0/3637211126 conn(0x7f1f300a5ed0 msgr2=0x7f1f300a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f0782ffd640 1 -- 172.21.15.178:0/2256635707 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f077c0079d0 con 0x7f07840a08c0 2024-09-15T01:32:07.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f0782ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f0782ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:32:07.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f0782ffd640 10 monclient: dump: 2024-09-15T01:32:07.076 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.076 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.076 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.076 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.076 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f0782ffd640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f0782ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f0782ffd640 1 -- 172.21.15.178:0/2256635707 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c02d050 con 0x7f07840a08c0 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f0782ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f0782ffd640 1 -- 172.21.15.178:0/2256635707 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f077c007bd0 con 0x7f07840a08c0 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f0782ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f0782ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f0782ffd640 10 monclient: dump: 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.078 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.079 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.079 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.079 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.079 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.079 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.079 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.079 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.079 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.079 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f078a95d640 5 monclient: authenticate success, global_id 4325 2024-09-15T01:32:07.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f078a95d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:07.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f078a95d640 4 set_mon_vals no callback set 2024-09-15T01:32:07.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f1f36c20640 1 -- 172.21.15.178:0/3637211126 shutdown_connections 2024-09-15T01:32:07.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f1f36c20640 1 -- 172.21.15.178:0/3637211126 wait complete. 2024-09-15T01:32:07.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f078a95d640 10 monclient: shutdown 2024-09-15T01:32:07.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f078a95d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f078a95d640 1 -- 172.21.15.178:0/2256635707 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07840a08c0 msgr2=0x7f07840a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f078a95d640 1 --2- 172.21.15.178:0/2256635707 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07840a08c0 0x7f07840a0d30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f077c009e90 tx=0x7f077c009780 comp rx=0 tx=0).stop 2024-09-15T01:32:07.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f078a95d640 1 -- 172.21.15.178:0/2256635707 shutdown_connections 2024-09-15T01:32:07.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f078a95d640 1 --2- 172.21.15.178:0/2256635707 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07840a08c0 0x7f07840a0d30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f078a95d640 1 --2- 172.21.15.178:0/2256635707 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f078409faa0 0x7f078409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f078a95d640 1 --2- 172.21.15.178:0/2256635707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07840a2930 0x7f0784132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.050+0000 7f078a95d640 1 -- 172.21.15.178:0/2256635707 >> 172.21.15.178:0/2256635707 conn(0x7f07840a5ed0 msgr2=0x7f07840a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.051+0000 7f078a95d640 1 -- 172.21.15.178:0/2256635707 shutdown_connections 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.051+0000 7f078a95d640 1 -- 172.21.15.178:0/2256635707 wait complete. 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.051+0000 7f078a95d640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.051+0000 7f078a95d640 10 monclient: monmap: 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.082 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:07.083 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:07.083 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:07.083 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.083 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.083 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.083 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.083 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.084 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.084 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.084 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.051+0000 7f1f36c20640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.051+0000 7f1f36c20640 10 monclient: monmap: 2024-09-15T01:32:07.084 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.084 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.085 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.086 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.051+0000 7f1f36c20640 1 Processor -- start 2024-09-15T01:32:07.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 1 -- start start 2024-09-15T01:32:07.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 10 monclient: init 2024-09-15T01:32:07.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f078a95d640 1 Processor -- start 2024-09-15T01:32:07.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f078a95d640 1 -- start start 2024-09-15T01:32:07.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f078a95d640 10 monclient: init 2024-09-15T01:32:07.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 10 monclient: _add_conns ranks=[0,8,6,3,7,2,5,4,1] 2024-09-15T01:32:07.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f3009faa0 0x7f1f3009af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 10 monclient(hunting): picked mon.noname-a con 0x7f1f3009faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:07.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f3009fe70 0x7f1f301317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 10 monclient(hunting): picked mon.noname-e con 0x7f1f3009fe70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:07.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f300a2930 0x7f1f30131d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 10 monclient: _add_conns ranks=[6,0,5,8,7,3,1,4,2] 2024-09-15T01:32:07.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 10 monclient(hunting): picked mon.noname-d con 0x7f1f300a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f300a2390 con 0x7f1f3009faa0 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3009d630 con 0x7f1f300a2930 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30142ab0 con 0x7f1f3009fe70 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f36c20640 10 monclient(hunting): authenticate will time out at 731.803748s 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f351bd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f300a2930 0x7f1f30131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.052+0000 7f1f351bd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f300a2930 0x7f1f30131d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:41908/0 (socket says 172.21.15.178:41908) 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 1 -- 172.21.15.178:0/2569542299 learned_addr learned my addr 172.21.15.178:0/2569542299 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 10 monclient(hunting): get_auth_request con 0x7f1f300a2930 auth_method 0 2024-09-15T01:32:07.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f078409faa0 0x7f078409af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 10 monclient(hunting): picked mon.noname-d con 0x7f078409faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:07.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07840a08c0 0x7f07841317a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f2ffff640 1 --2- 172.21.15.178:0/2569542299 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f3009fe70 0x7f1f301317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f349bc640 1 --2- 172.21.15.178:0/2569542299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f3009faa0 0x7f1f3009af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f349bc640 10 monclient(hunting): get_auth_request con 0x7f1f3009faa0 auth_method 0 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f349bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f349bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f349bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f2ffff640 10 monclient(hunting): get_auth_request con 0x7f1f3009fe70 auth_method 0 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f2ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f2ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f2ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 10 monclient(hunting): picked mon.noname-a con 0x7f07840a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:07.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07840a2930 0x7f078411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 10 monclient(hunting): picked mon.noname-h con 0x7f07840a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:07.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07840a2390 con 0x7f07840a08c0 2024-09-15T01:32:07.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f078409d630 con 0x7f07840a2930 2024-09-15T01:32:07.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f078411d9d0 con 0x7f078409faa0 2024-09-15T01:32:07.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f078a95d640 10 monclient(hunting): authenticate will time out at 731.804316s 2024-09-15T01:32:07.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f2ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f2ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f2ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f349bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f349bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f07837fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07840a08c0 0x7f07841317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0783fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f078409faa0 0x7f078409af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0783fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f078409faa0 0x7f078409af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:41910/0 (socket says 172.21.15.178:41910) 2024-09-15T01:32:07.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f07837fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07840a08c0 0x7f07841317a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33222/0 (socket says 172.21.15.178:33222) 2024-09-15T01:32:07.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0783fff640 1 -- 172.21.15.178:0/3383149563 learned_addr learned my addr 172.21.15.178:0/3383149563 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0783fff640 10 monclient(hunting): get_auth_request con 0x7f078409faa0 auth_method 0 2024-09-15T01:32:07.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0783fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0783fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0783fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f07837fe640 10 monclient(hunting): get_auth_request con 0x7f07840a08c0 auth_method 0 2024-09-15T01:32:07.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f07837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f07837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f07837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0783fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0783fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0783fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0788efa640 1 --2- 172.21.15.178:0/3383149563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07840a2930 0x7f078411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0788efa640 10 monclient(hunting): get_auth_request con 0x7f07840a2930 auth_method 0 2024-09-15T01:32:07.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0788efa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0788efa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f0788efa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f349bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 10 monclient(hunting): handle_auth_done global_id 4299 payload 824 2024-09-15T01:32:07.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 1 -- 172.21.15.178:0/2569542299 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f3009fe70 msgr2=0x7f1f301317c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 1 --2- 172.21.15.178:0/2569542299 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f3009fe70 0x7f1f301317c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 1 -- 172.21.15.178:0/2569542299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f3009faa0 msgr2=0x7f1f3009af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 1 --2- 172.21.15.178:0/2569542299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f3009faa0 0x7f1f3009af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 1 monclient: found mon.noname-d 2024-09-15T01:32:07.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f351bd640 1 -- 172.21.15.178:0/2569542299 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f24009390 con 0x7f1f300a2930 2024-09-15T01:32:07.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f1f349bc640 1 --2- 172.21.15.178:0/2569542299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f3009faa0 0x7f1f3009af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f07837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.053+0000 7f07837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f07837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f0788efa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f0788efa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f0788efa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f07837fe640 10 monclient(hunting): handle_auth_done global_id 4266 payload 824 2024-09-15T01:32:07.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f1f351bd640 1 --2- 172.21.15.178:0/2569542299 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f300a2930 0x7f1f30131d00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009ef0 tx=0x7f1f28037000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f07837fe640 1 -- 172.21.15.178:0/3383149563 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f078409faa0 msgr2=0x7f078409af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f07837fe640 1 --2- 172.21.15.178:0/3383149563 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f078409faa0 0x7f078409af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f07837fe640 1 -- 172.21.15.178:0/3383149563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07840a2930 msgr2=0x7f078411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f07837fe640 1 --2- 172.21.15.178:0/3383149563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07840a2930 0x7f078411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f07837fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f07837fe640 1 monclient: found mon.noname-a 2024-09-15T01:32:07.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f07837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f07837fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:07.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f07837fe640 1 -- 172.21.15.178:0/3383149563 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778009450 con 0x7f07840a08c0 2024-09-15T01:32:07.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f07837fe640 1 --2- 172.21.15.178:0/3383149563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07840a08c0 0x7f07841317a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f077800e950 tx=0x7f07780091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f1f2cff9640 1 -- 172.21.15.178:0/2569542299 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f2803dac0 con 0x7f1f300a2930 2024-09-15T01:32:07.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f1f2cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f1f2cff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:32:07.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f076bfff640 1 -- 172.21.15.178:0/3383149563 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f077800cdc0 con 0x7f07840a08c0 2024-09-15T01:32:07.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f076bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f076bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f076bfff640 10 monclient: dump: 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.102 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.103 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.103 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.103 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.103 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f076bfff640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f076bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f076bfff640 1 -- 172.21.15.178:0/3383149563 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0778013470 con 0x7f07840a08c0 2024-09-15T01:32:07.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f076bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f076bfff640 1 -- 172.21.15.178:0/3383149563 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0778013a60 con 0x7f07840a08c0 2024-09-15T01:32:07.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f076bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f076bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:07.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.054+0000 7f076bfff640 10 monclient: dump: 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.104 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.105 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.105 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f2cff9640 10 monclient: dump: 2024-09-15T01:32:07.105 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.105 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.105 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.105 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.105 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.105 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.105 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.105 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.106 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.106 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.106 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.106 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.106 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.106 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.106 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.106 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f07827fc640 4 set_mon_vals no callback set 2024-09-15T01:32:07.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f2cff9640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f2cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f2cff9640 1 -- 172.21.15.178:0/2569542299 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2803dc40 con 0x7f1f300a2930 2024-09-15T01:32:07.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f2cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f2effd640 4 set_mon_vals no callback set 2024-09-15T01:32:07.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f2cff9640 1 -- 172.21.15.178:0/2569542299 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f2803ddb0 con 0x7f1f300a2930 2024-09-15T01:32:07.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f2cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f2cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f2cff9640 10 monclient: dump: 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.109 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.110 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.110 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.110 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.110 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f36c20640 5 monclient: authenticate success, global_id 4299 2024-09-15T01:32:07.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f36c20640 10 monclient: _renew_subs 2024-09-15T01:32:07.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f36c20640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f36c20640 1 -- 172.21.15.178:0/2569542299 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f28009a20 con 0x7f1f300a2930 2024-09-15T01:32:07.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f36c20640 10 monclient: _renew_subs 2024-09-15T01:32:07.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f36c20640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.055+0000 7f1f36c20640 1 -- 172.21.15.178:0/2569542299 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3009b950 con 0x7f1f300a2930 2024-09-15T01:32:07.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.056+0000 7f078a95d640 5 monclient: authenticate success, global_id 4266 2024-09-15T01:32:07.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.056+0000 7f078a95d640 10 monclient: _renew_subs 2024-09-15T01:32:07.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.056+0000 7f078a95d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:07.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.056+0000 7f078a95d640 1 -- 172.21.15.178:0/3383149563 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0774009390 con 0x7f07840a08c0 2024-09-15T01:32:07.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.056+0000 7f078a95d640 10 monclient: _renew_subs 2024-09-15T01:32:07.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.056+0000 7f078a95d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:07.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.056+0000 7f078a95d640 1 -- 172.21.15.178:0/3383149563 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c009a20 con 0x7f07840a08c0 2024-09-15T01:32:07.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.057+0000 7f1f2cff9640 1 -- 172.21.15.178:0/2569542299 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1f28003040 con 0x7f1f300a2930 2024-09-15T01:32:07.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.057+0000 7f1f36c20640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.057+0000 7f1f36c20640 1 -- 172.21.15.178:0/2569542299 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f3004c4a0 con 0x7f1f300a2930 2024-09-15T01:32:07.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.057+0000 7f076bfff640 1 -- 172.21.15.178:0/3383149563 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0778013bc0 con 0x7f07840a08c0 2024-09-15T01:32:07.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.057+0000 7f078a95d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:07.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.057+0000 7f078a95d640 1 -- 172.21.15.178:0/3383149563 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f078404c4a0 con 0x7f07840a08c0 2024-09-15T01:32:07.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.057+0000 7f076bfff640 1 --2- 172.21.15.178:0/3383149563 >> v2:172.21.15.204:6812/28210 conn(0x7f07700397b0 0x7f077003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.057+0000 7f076bfff640 1 -- 172.21.15.178:0/3383149563 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f077804b260 con 0x7f07840a08c0 2024-09-15T01:32:07.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.057+0000 7f076bfff640 1 --2- 172.21.15.178:0/3383149563 >> v2:172.21.15.178:6808/29234 conn(0x7f077003f250 0x7f0770041680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.057+0000 7f076bfff640 1 -- 172.21.15.178:0/3383149563 --> v2:172.21.15.178:6808/29234 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0770041d10 con 0x7f077003f250 2024-09-15T01:32:07.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f1f2cff9640 1 --2- 172.21.15.178:0/2569542299 >> v2:172.21.15.204:6812/28210 conn(0x7f1f140397b0 0x7f1f1403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f0783fff640 1 --2- 172.21.15.178:0/3383149563 >> v2:172.21.15.204:6812/28210 conn(0x7f07700397b0 0x7f077003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f076bfff640 1 -- 172.21.15.178:0/3383149563 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07780123d0 con 0x7f07840a08c0 2024-09-15T01:32:07.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f076bfff640 10 monclient: handle_get_version_reply finishing 1 version 19 2024-09-15T01:32:07.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f0783fff640 10 monclient: get_auth_request con 0x7f07700397b0 auth_method 0 2024-09-15T01:32:07.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f1f2cff9640 1 -- 172.21.15.178:0/2569542299 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f1f2806f220 con 0x7f1f300a2930 2024-09-15T01:32:07.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f0788efa640 1 --2- 172.21.15.178:0/3383149563 >> v2:172.21.15.178:6808/29234 conn(0x7f077003f250 0x7f0770041680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f0788efa640 10 monclient: get_auth_request con 0x7f077003f250 auth_method 0 2024-09-15T01:32:07.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f1f349bc640 1 --2- 172.21.15.178:0/2569542299 >> v2:172.21.15.204:6812/28210 conn(0x7f1f140397b0 0x7f1f1403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f1f349bc640 10 monclient: get_auth_request con 0x7f1f140397b0 auth_method 0 2024-09-15T01:32:07.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f1f2cff9640 1 --2- 172.21.15.178:0/2569542299 >> v2:172.21.15.178:6804/29233 conn(0x7f1f1403f250 0x7f1f14041680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f1f2cff9640 1 -- 172.21.15.178:0/2569542299 --> v2:172.21.15.178:6804/29233 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f14041d10 con 0x7f1f1403f250 2024-09-15T01:32:07.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f0788efa640 1 --2- 172.21.15.178:0/3383149563 >> v2:172.21.15.178:6808/29234 conn(0x7f077003f250 0x7f0770041680 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f0783fff640 1 --2- 172.21.15.178:0/3383149563 >> v2:172.21.15.204:6812/28210 conn(0x7f07700397b0 0x7f077003bc60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0774009fb0 tx=0x7f07740090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f1f2ffff640 1 --2- 172.21.15.178:0/2569542299 >> v2:172.21.15.178:6804/29233 conn(0x7f1f1403f250 0x7f1f14041680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.058+0000 7f1f2ffff640 10 monclient: get_auth_request con 0x7f1f1403f250 auth_method 0 2024-09-15T01:32:07.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.059+0000 7f1f349bc640 1 --2- 172.21.15.178:0/2569542299 >> v2:172.21.15.204:6812/28210 conn(0x7f1f140397b0 0x7f1f1403bc60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f24011580 tx=0x7f1f240090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.059+0000 7f1f2ffff640 1 --2- 172.21.15.178:0/2569542299 >> v2:172.21.15.178:6804/29233 conn(0x7f1f1403f250 0x7f1f14041680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.059+0000 7f1f2cff9640 1 -- 172.21.15.178:0/2569542299 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f2806f4a0 con 0x7f1f300a2930 2024-09-15T01:32:07.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.059+0000 7f1f2cff9640 10 monclient: handle_get_version_reply finishing 1 version 19 2024-09-15T01:32:07.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.059+0000 7f076bfff640 1 -- 172.21.15.178:0/3383149563 <== osd.2 v2:172.21.15.178:6808/29234 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f0770041d10 con 0x7f077003f250 2024-09-15T01:32:07.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.060+0000 7f1f2cff9640 1 -- 172.21.15.178:0/2569542299 <== osd.1 v2:172.21.15.178:6804/29233 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f1f14041d10 con 0x7f1f1403f250 2024-09-15T01:32:07.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.081+0000 7fcad1f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:07.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.081+0000 7fcad1f9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.081+0000 7fcad1f9d640 10 monclient: monmap: 2024-09-15T01:32:07.118 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.118 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.118 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:07.083399+0000 2024-09-15T01:32:07.118 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:07.083399+0000 2024-09-15T01:32:07.118 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.118 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.118 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:07.118 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:07.119 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.119 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.119 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.119 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.119 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.119 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.119 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.119 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:07.119 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:07.119 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.084+0000 7f1f36c20640 1 -- 172.21.15.178:0/2569542299 --> v2:172.21.15.178:6804/29233 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1f3004bc80 con 0x7f1f1403f250 2024-09-15T01:32:07.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.085+0000 7fcad1f9d640 1 Processor -- start 2024-09-15T01:32:07.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.085+0000 7fcad1f9d640 1 -- start start 2024-09-15T01:32:07.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.085+0000 7fcad1f9d640 10 monclient: init 2024-09-15T01:32:07.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 10 monclient: _add_conns ranks=[10,7,1,9,5,2,4,8,3,6,0] 2024-09-15T01:32:07.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcacc0a25a0 0x7fcacc09b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 10 monclient(hunting): picked mon.noname-f con 0x7fcacc0a25a0 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:07.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcacc098900 0x7fcacc09bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 10 monclient(hunting): picked mon.noname-i con 0x7fcacc098900 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:07.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fcacc0993e0 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:07.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc04bf60 con 0x7fcacc0993e0 2024-09-15T01:32:07.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcacc09e2f0 con 0x7fcacc0a25a0 2024-09-15T01:32:07.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7f1f2cff9640 1 -- 172.21.15.178:0/2569542299 <== osd.1 v2:172.21.15.178:6804/29233 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1f3004bc80 con 0x7f1f1403f250 2024-09-15T01:32:07.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc097950 con 0x7fcacc098900 2024-09-15T01:32:07.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcad1f9d640 10 monclient(hunting): authenticate will time out at 731.837256s 2024-09-15T01:32:07.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacb7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcacc0a25a0 0x7fcacc09b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacb7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcacc0a25a0 0x7fcacc09b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:52330/0 (socket says 172.21.15.178:52330) 2024-09-15T01:32:07.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacb7fe640 1 -- 172.21.15.178:0/687989765 learned_addr learned my addr 172.21.15.178:0/687989765 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacb7fe640 10 monclient(hunting): get_auth_request con 0x7fcacc0a25a0 auth_method 0 2024-09-15T01:32:07.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacaffd640 1 --2- 172.21.15.178:0/687989765 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcacc098900 0x7fcacc09bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacaffd640 10 monclient(hunting): get_auth_request con 0x7fcacc098900 auth_method 0 2024-09-15T01:32:07.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7fe28eb9d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:07.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7fe28eb9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.086+0000 7fcacaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7fe28eb9d640 10 monclient: monmap: 2024-09-15T01:32:07.126 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.126 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.126 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:07.089167+0000 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:07.089167+0000 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:07.127 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7fcacaffd640 10 monclient(hunting): handle_auth_done global_id 4255 payload 274 2024-09-15T01:32:07.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7fcacaffd640 1 -- 172.21.15.178:0/687989765 >> v2:172.21.15.204:3300/0 conn(0x7fcacc0a25a0 msgr2=0x7fcacc09b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7fcacaffd640 1 --2- 172.21.15.178:0/687989765 >> v2:172.21.15.204:3300/0 conn(0x7fcacc0a25a0 0x7fcacc09b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7fcacaffd640 1 -- 172.21.15.178:0/687989765 >> v1:172.21.15.204:6789/0 conn(0x7fcacc0993e0 legacy=0x7fcacc09c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7fcacaffd640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7fcacaffd640 1 monclient: found mon.noname-i 2024-09-15T01:32:07.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7fcacaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7fcacaffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:32:07.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7fcacaffd640 1 -- 172.21.15.178:0/687989765 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc09c320 con 0x7fcacc098900 2024-09-15T01:32:07.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7f1f1a7fc640 1 -- 172.21.15.178:0/2569542299 >> v2:172.21.15.178:6804/29233 conn(0x7f1f1403f250 msgr2=0x7f1f14041680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7f1f1a7fc640 1 --2- 172.21.15.178:0/2569542299 >> v2:172.21.15.178:6804/29233 conn(0x7f1f1403f250 0x7f1f14041680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7f1f1a7fc640 1 -- 172.21.15.178:0/2569542299 >> v2:172.21.15.204:6812/28210 conn(0x7f1f140397b0 msgr2=0x7f1f1403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7f1f1a7fc640 1 --2- 172.21.15.178:0/2569542299 >> v2:172.21.15.204:6812/28210 conn(0x7f1f140397b0 0x7f1f1403bc60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f24011580 tx=0x7f1f240090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7f1f1a7fc640 10 monclient: shutdown 2024-09-15T01:32:07.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7f1f1a7fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7f1f1a7fc640 1 -- 172.21.15.178:0/2569542299 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f300a2930 msgr2=0x7f1f30131d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7f1f1a7fc640 1 --2- 172.21.15.178:0/2569542299 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f300a2930 0x7f1f30131d00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009ef0 tx=0x7f1f28037000 comp rx=0 tx=0).stop 2024-09-15T01:32:07.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7f1f351bd640 1 -- 172.21.15.178:0/2569542299 reap_dead start 2024-09-15T01:32:07.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7f1f1a7fc640 1 -- 172.21.15.178:0/2569542299 shutdown_connections 2024-09-15T01:32:07.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.087+0000 7f1f1a7fc640 1 -- 172.21.15.178:0/2569542299 >> 172.21.15.178:0/2569542299 conn(0x7f1f300a5ed0 msgr2=0x7f1f3009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.088+0000 7f1f1a7fc640 1 -- 172.21.15.178:0/2569542299 shutdown_connections 2024-09-15T01:32:07.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.088+0000 7f1f1a7fc640 1 -- 172.21.15.178:0/2569542299 wait complete. 2024-09-15T01:32:07.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.088+0000 7fe28eb9d640 1 Processor -- start 2024-09-15T01:32:07.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.088+0000 7fe28eb9d640 1 -- start start 2024-09-15T01:32:07.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.088+0000 7fe28eb9d640 10 monclient: init 2024-09-15T01:32:07.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.088+0000 7f078a95d640 1 -- 172.21.15.178:0/3383149563 --> v2:172.21.15.178:6808/29234 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f078404bc80 con 0x7f077003f250 2024-09-15T01:32:07.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.088+0000 7fcacaffd640 1 --2- 172.21.15.178:0/687989765 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcacc098900 0x7fcacc09bde0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009850 tx=0x7fcac00090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=2d4f847cc9c9fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.088+0000 7f076bfff640 1 -- 172.21.15.178:0/3383149563 <== osd.2 v2:172.21.15.178:6808/29234 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f078404bc80 con 0x7f077003f250 2024-09-15T01:32:07.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcaca7fc640 1 -- 172.21.15.178:0/687989765 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcac002ac70 con 0x7fcacc098900 2024-09-15T01:32:07.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcaca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcaca7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:32:07.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 10 monclient: _add_conns ranks=[1,3,6,9,7,0,2,8,4,5,10] 2024-09-15T01:32:07.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe2880a25a0 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:07.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe288098900 0x7fe28809bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcaca7fc640 10 monclient: dump: 2024-09-15T01:32:07.134 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.134 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.134 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.134 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.134 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.134 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcaca7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcaca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcaca7fc640 1 -- 172.21.15.178:0/687989765 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac002adf0 con 0x7fcacc098900 2024-09-15T01:32:07.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcaca7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 10 monclient(hunting): picked mon.noname-g con 0x7fe288098900 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:07.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2880993e0 0x7fe28813e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcaca7fc640 1 -- 172.21.15.178:0/687989765 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcac0031ca0 con 0x7fcacc098900 2024-09-15T01:32:07.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcaca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcaca7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:32:07.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcaca7fc640 10 monclient: dump: 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.137 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.138 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcad1f9d640 5 monclient: authenticate success, global_id 4255 2024-09-15T01:32:07.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcad1f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:07.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fcad1f9d640 4 set_mon_vals no callback set 2024-09-15T01:32:07.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7f078a95d640 1 -- 172.21.15.178:0/3383149563 >> v2:172.21.15.178:6808/29234 conn(0x7f077003f250 msgr2=0x7f0770041680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7f078a95d640 1 --2- 172.21.15.178:0/3383149563 >> v2:172.21.15.178:6808/29234 conn(0x7f077003f250 0x7f0770041680 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7f078a95d640 1 -- 172.21.15.178:0/3383149563 >> v2:172.21.15.204:6812/28210 conn(0x7f07700397b0 msgr2=0x7f077003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7f078a95d640 1 --2- 172.21.15.178:0/3383149563 >> v2:172.21.15.204:6812/28210 conn(0x7f07700397b0 0x7f077003bc60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0774009fb0 tx=0x7f07740090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7f078a95d640 10 monclient: shutdown 2024-09-15T01:32:07.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7f078a95d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7f078a95d640 1 -- 172.21.15.178:0/3383149563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07840a08c0 msgr2=0x7f07841317a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7f078a95d640 1 --2- 172.21.15.178:0/3383149563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07840a08c0 0x7f07841317a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f077800e950 tx=0x7f07780091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28c939640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fe2880a25a0 0x7fe28809b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:44300/0 (socket says 172.21.15.178:44300) 2024-09-15T01:32:07.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 10 monclient(hunting): picked mon.noname-d con 0x7fe2880993e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:07.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7f0788efa640 1 -- 172.21.15.178:0/3383149563 reap_dead start 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28c939640 1 -- 172.21.15.178:0/3126757520 learned_addr learned my addr 172.21.15.178:0/3126757520 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3126757520 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe28804bf60 con 0x7fe2880a25a0 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3126757520 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28809e2f0 con 0x7fe288098900 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3126757520 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe288097950 con 0x7fe2880993e0 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28eb9d640 10 monclient(hunting): authenticate will time out at 731.840451s 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe287fff640 1 --2- 172.21.15.178:0/3126757520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe288098900 0x7fe28809bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe287fff640 10 monclient(hunting): get_auth_request con 0x7fe288098900 auth_method 0 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe287fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe287fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe287fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28d13a640 1 --2- 172.21.15.178:0/3126757520 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2880993e0 0x7fe28813e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28d13a640 10 monclient(hunting): get_auth_request con 0x7fe2880993e0 auth_method 0 2024-09-15T01:32:07.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28d13a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28d13a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28d13a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe287fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe287fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe287fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28d13a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28d13a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe28d13a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe287fff640 10 monclient(hunting): handle_auth_done global_id 4269 payload 274 2024-09-15T01:32:07.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe287fff640 1 -- 172.21.15.178:0/3126757520 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2880993e0 msgr2=0x7fe28813e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.089+0000 7fe287fff640 1 --2- 172.21.15.178:0/3126757520 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2880993e0 0x7fe28813e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fe287fff640 1 -- 172.21.15.178:0/3126757520 >> v1:172.21.15.204:6789/0 conn(0x7fe2880a25a0 legacy=0x7fe28809b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fe2877fe640 1 -- 172.21.15.178:0/3126757520 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019033262 0 0) 0x7fe28804bf60 con 0x7fe2880a25a0 2024-09-15T01:32:07.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fe287fff640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fe287fff640 1 monclient: found mon.noname-g 2024-09-15T01:32:07.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fe287fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fe287fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:32:07.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fe287fff640 1 -- 172.21.15.178:0/3126757520 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880a2970 con 0x7fe288098900 2024-09-15T01:32:07.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fe2877fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:32:07.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fe28d13a640 1 --2- 172.21.15.178:0/3126757520 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2880993e0 0x7fe28813e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7f078a95d640 1 -- 172.21.15.178:0/3383149563 shutdown_connections 2024-09-15T01:32:07.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7f078a95d640 1 -- 172.21.15.178:0/3383149563 >> 172.21.15.178:0/3383149563 conn(0x7f07840a5ed0 msgr2=0x7f0784135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fcad1f9d640 10 monclient: shutdown 2024-09-15T01:32:07.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fcad1f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fcad1f9d640 1 -- 172.21.15.178:0/687989765 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcacc098900 msgr2=0x7fcacc09bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fcad1f9d640 1 --2- 172.21.15.178:0/687989765 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcacc098900 0x7fcacc09bde0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009850 tx=0x7fcac00090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7f078a95d640 1 -- 172.21.15.178:0/3383149563 shutdown_connections 2024-09-15T01:32:07.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fcad1f9d640 1 -- 172.21.15.178:0/687989765 shutdown_connections 2024-09-15T01:32:07.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fcad1f9d640 1 --2- 172.21.15.178:0/687989765 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcacc098900 0x7fcacc09bde0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fcad1f9d640 1 --2- 172.21.15.178:0/687989765 >> v2:172.21.15.204:3300/0 conn(0x7fcacc0a25a0 0x7fcacc09b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fcad1f9d640 1 -- 172.21.15.178:0/687989765 >> 172.21.15.178:0/687989765 conn(0x7fcacc067710 msgr2=0x7fcacc0a1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7f078a95d640 1 -- 172.21.15.178:0/3383149563 wait complete. 2024-09-15T01:32:07.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fcad1f9d640 1 -- 172.21.15.178:0/687989765 shutdown_connections 2024-09-15T01:32:07.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fcad1f9d640 1 -- 172.21.15.178:0/687989765 wait complete. 2024-09-15T01:32:07.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fe351cb8640 10 monclient: get_monmap_and_config 2024-09-15T01:32:07.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fe351cb8640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.090+0000 7fe351cb8640 10 monclient: monmap: 2024-09-15T01:32:07.146 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.146 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.146 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:07.092553+0000 2024-09-15T01:32:07.147 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:07.092553+0000 2024-09-15T01:32:07.147 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.147 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.147 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:07.147 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:07.147 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.147 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.147 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.147 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.147 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.147 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.148 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.148 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:07.148 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:07.148 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe287fff640 1 --2- 172.21.15.178:0/3126757520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe288098900 0x7fe28809bfb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe274009880 tx=0x7fe2740090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=465a1149ece32c27 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fcad1f9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fcad1f9d640 10 monclient: monmap: 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.149 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.150 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe2877fe640 1 -- 172.21.15.178:0/3126757520 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe27402ac50 con 0x7fe288098900 2024-09-15T01:32:07.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe2877fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe2877fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:32:07.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe2877fe640 10 monclient: dump: 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.151 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe2877fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe2877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe2877fe640 1 -- 172.21.15.178:0/3126757520 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27402add0 con 0x7fe288098900 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe2877fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe2877fe640 1 -- 172.21.15.178:0/3126757520 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe274031ca0 con 0x7fe288098900 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe2877fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe2877fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe2877fe640 10 monclient: dump: 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.152 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe28eb9d640 5 monclient: authenticate success, global_id 4269 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe28eb9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:07.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe28eb9d640 4 set_mon_vals no callback set 2024-09-15T01:32:07.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe351cb8640 1 Processor -- start 2024-09-15T01:32:07.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe351cb8640 1 -- start start 2024-09-15T01:32:07.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fe351cb8640 10 monclient: init 2024-09-15T01:32:07.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fcad1f9d640 1 Processor -- start 2024-09-15T01:32:07.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fcad1f9d640 1 -- start start 2024-09-15T01:32:07.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.091+0000 7fcad1f9d640 10 monclient: init 2024-09-15T01:32:07.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 10 monclient: _add_conns ranks=[1,4,5,2,8,7,0,3,6] 2024-09-15T01:32:07.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcacc098900 0x7fcacc1380a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 10 monclient(hunting): picked mon.noname-f con 0x7fcacc098900 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:07.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe28eb9d640 10 monclient: shutdown 2024-09-15T01:32:07.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe28eb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3126757520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe288098900 msgr2=0x7fe28809bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe28eb9d640 1 --2- 172.21.15.178:0/3126757520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe288098900 0x7fe28809bfb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe274009880 tx=0x7fe2740090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcacc0993e0 0x7fcacc123c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 10 monclient(hunting): picked mon.noname-c con 0x7fcacc0993e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:07.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcacc0a25a0 0x7fcacc124140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 10 monclient(hunting): picked mon.noname-h con 0x7fcacc0a25a0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:07.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc096aa0 con 0x7fcacc098900 2024-09-15T01:32:07.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc06ee90 con 0x7fcacc0993e0 2024-09-15T01:32:07.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc1274c0 con 0x7fcacc0a25a0 2024-09-15T01:32:07.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcad1f9d640 10 monclient(hunting): authenticate will time out at 731.843487s 2024-09-15T01:32:07.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 10 monclient: _add_conns ranks=[8,2,0,3,10,7,6,9,5,1,4] 2024-09-15T01:32:07.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7f8e02a67640 10 monclient: get_monmap_and_config 2024-09-15T01:32:07.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7f8e02a67640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7f8e02a67640 10 monclient: monmap: 2024-09-15T01:32:07.157 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:07.094405+0000 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:07.094405+0000 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.158 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.159 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.159 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:07.159 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:07.159 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe34c12f5d0 0x7fe34c12f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3126757520 shutdown_connections 2024-09-15T01:32:07.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe28eb9d640 1 --2- 172.21.15.178:0/3126757520 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2880993e0 0x7fe28813e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe28eb9d640 1 --2- 172.21.15.178:0/3126757520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe288098900 0x7fe28809bfb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3126757520 >> 172.21.15.178:0/3126757520 conn(0x7fe288067710 msgr2=0x7fe2880a1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 10 monclient(hunting): picked mon.noname-e con 0x7fe34c12f5d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:07.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e510 0x7fe34c09d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 10 monclient(hunting): picked mon.noname-b con 0x7fe34c06e510 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:07.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe34c06e8e0 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:07.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34c04bf60 con 0x7fe34c06e8e0 2024-09-15T01:32:07.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c09f2c0 con 0x7fe34c06e510 2024-09-15T01:32:07.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c06b850 con 0x7fe34c12f5d0 2024-09-15T01:32:07.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe351cb8640 10 monclient(hunting): authenticate will time out at 731.843811s 2024-09-15T01:32:07.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e510 0x7fe34c09d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e510 0x7fe34c09d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:49358/0 (socket says 172.21.15.178:49358) 2024-09-15T01:32:07.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 1 -- 172.21.15.178:0/3346844609 learned_addr learned my addr 172.21.15.178:0/3346844609 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 10 monclient(hunting): get_auth_request con 0x7fe34c06e510 auth_method 0 2024-09-15T01:32:07.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacaffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcacc0993e0 0x7fcacc123c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcacc098900 0x7fcacc1380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacaffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcacc0993e0 0x7fcacc123c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:40238/0 (socket says 172.21.15.178:40238) 2024-09-15T01:32:07.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacaffd640 1 -- 172.21.15.178:0/631585416 learned_addr learned my addr 172.21.15.178:0/631585416 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacaffd640 10 monclient(hunting): get_auth_request con 0x7fcacc0993e0 auth_method 0 2024-09-15T01:32:07.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacb7fe640 10 monclient(hunting): get_auth_request con 0x7fcacc098900 auth_method 0 2024-09-15T01:32:07.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fcacaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacbfff640 1 --2- 172.21.15.178:0/631585416 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcacc0a25a0 0x7fcacc124140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacbfff640 10 monclient(hunting): get_auth_request con 0x7fcacc0a25a0 auth_method 0 2024-09-15T01:32:07.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3126757520 shutdown_connections 2024-09-15T01:32:07.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7f8e02a67640 1 Processor -- start 2024-09-15T01:32:07.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 10 monclient(hunting): handle_auth_done global_id 4258 payload 824 2024-09-15T01:32:07.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7f8e02a67640 1 -- start start 2024-09-15T01:32:07.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7f8e02a67640 10 monclient: init 2024-09-15T01:32:07.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 1 -- 172.21.15.178:0/631585416 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcacc0a25a0 msgr2=0x7fcacc124140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 1 --2- 172.21.15.178:0/631585416 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcacc0a25a0 0x7fcacc124140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 1 -- 172.21.15.178:0/631585416 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcacc0993e0 msgr2=0x7fcacc123c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 1 --2- 172.21.15.178:0/631585416 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcacc0993e0 0x7fcacc123c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 1 monclient: found mon.noname-f 2024-09-15T01:32:07.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 1 -- 172.21.15.178:0/631585416 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac0009390 con 0x7fcacc098900 2024-09-15T01:32:07.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacaffd640 1 --2- 172.21.15.178:0/631585416 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcacc0993e0 0x7fcacc123c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcacb7fe640 1 --2- 172.21.15.178:0/631585416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcacc098900 0x7fcacc1380a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00e900 tx=0x7fcabc0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.092+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3126757520 wait complete. 2024-09-15T01:32:07.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe28eb9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe28eb9d640 10 monclient: monmap: 2024-09-15T01:32:07.167 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.167 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.168 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34b7fe640 1 --2- 172.21.15.178:0/3346844609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe34c12f5d0 0x7fe34c12f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 10 monclient(hunting): handle_auth_done global_id 4295 payload 274 2024-09-15T01:32:07.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 1 -- 172.21.15.178:0/3346844609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe34c12f5d0 msgr2=0x7fe34c12f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 1 --2- 172.21.15.178:0/3346844609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe34c12f5d0 0x7fe34c12f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 1 -- 172.21.15.178:0/3346844609 >> v1:172.21.15.178:6789/0 conn(0x7fe34c06e8e0 legacy=0x7fe34c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fcab37fe640 1 -- 172.21.15.178:0/631585416 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcabc009420 con 0x7fcacc098900 2024-09-15T01:32:07.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcab37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 1 monclient: found mon.noname-b 2024-09-15T01:32:07.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:07.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34affd640 1 -- 172.21.15.178:0/3346844609 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c06c740 con 0x7fe34c06e510 2024-09-15T01:32:07.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34b7fe640 10 monclient: get_auth_request con 0x7fe34c12f5d0 auth_method 0 2024-09-15T01:32:07.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.093+0000 7fe34b7fe640 1 --2- 172.21.15.178:0/3346844609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe34c12f5d0 0x7fe34c12f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:32:07.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcab37fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:32:07.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcab37fe640 10 monclient: dump: 2024-09-15T01:32:07.171 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.171 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.171 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.171 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.171 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.171 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.171 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.171 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcab37fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcab37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcab37fe640 1 -- 172.21.15.178:0/631585416 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcabc00ee90 con 0x7fcacc098900 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcab37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcac9ffb640 4 set_mon_vals no callback set 2024-09-15T01:32:07.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fe28eb9d640 1 Processor -- start 2024-09-15T01:32:07.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 10 monclient: _add_conns ranks=[1,3,4,7,10,8,9,0,5,6,2] 2024-09-15T01:32:07.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8df4025750 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:07.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8df40bc860 0x7f8df40c1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 10 monclient(hunting): picked mon.noname-g con 0x7f8df40bc860 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:07.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8df40bcc30 0x7f8df40c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 10 monclient(hunting): picked mon.noname-c con 0x7f8df40bcc30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:07.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df40042a0 con 0x7f8df4025750 2024-09-15T01:32:07.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4024fe0 con 0x7f8df40bc860 2024-09-15T01:32:07.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4025b20 con 0x7f8df40bcc30 2024-09-15T01:32:07.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02a67640 10 monclient(hunting): authenticate will time out at 731.845675s 2024-09-15T01:32:07.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e01a65640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f8df4025750 0x7f8df40bc150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:44308/0 (socket says 172.21.15.178:44308) 2024-09-15T01:32:07.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e01a65640 1 -- 172.21.15.178:0/2278738365 learned_addr learned my addr 172.21.15.178:0/2278738365 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:07.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7f8e02266640 1 --2- 172.21.15.178:0/2278738365 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8df40bcc30 0x7f8df40c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 10 monclient(hunting): get_auth_request con 0x7f8df40bcc30 auth_method 0 2024-09-15T01:32:07.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fe28eb9d640 1 -- start start 2024-09-15T01:32:07.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fe28eb9d640 10 monclient: init 2024-09-15T01:32:07.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 10 monclient: _add_conns ranks=[8,3,4,7,2,1,5,0,6] 2024-09-15T01:32:07.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe288098900 0x7fe288142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e00a63640 1 -- 172.21.15.178:0/2278738365 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951204590 0 0) 0x7f8df40042a0 con 0x7f8df4025750 2024-09-15T01:32:07.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e00a63640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e00a63640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e00a63640 10 monclient(hunting): my global_id is 4267 2024-09-15T01:32:07.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e00a63640 1 -- 172.21.15.178:0/2278738365 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8de80034e0 con 0x7f8df4025750 2024-09-15T01:32:07.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e01264640 1 --2- 172.21.15.178:0/2278738365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8df40bc860 0x7f8df40c1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e01264640 10 monclient(hunting): get_auth_request con 0x7f8df40bc860 auth_method 0 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e01264640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e01264640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e01264640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 10 monclient(hunting): handle_auth_done global_id 4243 payload 274 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 1 -- 172.21.15.178:0/2278738365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8df40bc860 msgr2=0x7f8df40c1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e00a63640 1 -- 172.21.15.178:0/2278738365 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1678471382 0 0) 0x7f8de80034e0 con 0x7f8df4025750 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 1 --2- 172.21.15.178:0/2278738365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8df40bc860 0x7f8df40c1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 1 -- 172.21.15.178:0/2278738365 >> v1:172.21.15.204:6789/0 conn(0x7f8df4025750 legacy=0x7f8df40bc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 1 monclient: found mon.noname-c 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:07.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e02266640 1 -- 172.21.15.178:0/2278738365 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df4024c70 con 0x7f8df40bcc30 2024-09-15T01:32:07.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7f8e00a63640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:32:07.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 10 monclient(hunting): picked mon.noname-e con 0x7fe288098900 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:07.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2880993e0 0x7fe288141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 10 monclient(hunting): picked mon.noname-g con 0x7fe2880993e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:07.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe2880a25a0 0x7fe288145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 10 monclient(hunting): picked mon.noname-c con 0x7fe2880a25a0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:07.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe288125390 con 0x7fe2880993e0 2024-09-15T01:32:07.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe288127030 con 0x7fe2880a25a0 2024-09-15T01:32:07.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe288124ba0 con 0x7fe288098900 2024-09-15T01:32:07.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28eb9d640 10 monclient(hunting): authenticate will time out at 731.846350s 2024-09-15T01:32:07.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe287fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2880993e0 0x7fe288141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcab37fe640 1 -- 172.21.15.178:0/631585416 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcabc00cdc0 con 0x7fcacc098900 2024-09-15T01:32:07.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcab37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcab37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:32:07.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe287fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2880993e0 0x7fe288141cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:55362/0 (socket says 172.21.15.178:55362) 2024-09-15T01:32:07.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe287fff640 1 -- 172.21.15.178:0/3799520982 learned_addr learned my addr 172.21.15.178:0/3799520982 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe287fff640 10 monclient(hunting): get_auth_request con 0x7fe2880993e0 auth_method 0 2024-09-15T01:32:07.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe287fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe287fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe287fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28d13a640 1 --2- 172.21.15.178:0/3799520982 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe2880a25a0 0x7fe288145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcab37fe640 10 monclient: dump: 2024-09-15T01:32:07.181 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.181 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.181 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.181 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.181 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcad1f9d640 5 monclient: authenticate success, global_id 4258 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcad1f9d640 10 monclient: _renew_subs 2024-09-15T01:32:07.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcad1f9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcad1f9d640 1 -- 172.21.15.178:0/631585416 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc1271e0 con 0x7fcacc098900 2024-09-15T01:32:07.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcad1f9d640 10 monclient: _renew_subs 2024-09-15T01:32:07.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcad1f9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fcad1f9d640 1 -- 172.21.15.178:0/631585416 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc1396d0 con 0x7fcacc098900 2024-09-15T01:32:07.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28d13a640 10 monclient(hunting): get_auth_request con 0x7fe2880a25a0 auth_method 0 2024-09-15T01:32:07.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28d13a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28d13a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe28d13a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe287fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe287fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe287fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 1 --2- 172.21.15.178:0/3799520982 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe288098900 0x7fe288142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 10 monclient(hunting): get_auth_request con 0x7fe288098900 auth_method 0 2024-09-15T01:32:07.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 10 monclient(hunting): handle_auth_done global_id 4292 payload 824 2024-09-15T01:32:07.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 1 -- 172.21.15.178:0/3799520982 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe2880a25a0 msgr2=0x7fe288145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 1 --2- 172.21.15.178:0/3799520982 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe2880a25a0 0x7fe288145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 1 -- 172.21.15.178:0/3799520982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2880993e0 msgr2=0x7fe288141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 1 --2- 172.21.15.178:0/3799520982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2880993e0 0x7fe288141cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 1 monclient: found mon.noname-e 2024-09-15T01:32:07.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:07.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28c939640 1 -- 172.21.15.178:0/3799520982 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe280009a20 con 0x7fe288098900 2024-09-15T01:32:07.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7f8e02266640 1 --2- 172.21.15.178:0/2278738365 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8df40bcc30 0x7f8df40c39d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc071a80 tx=0x7f8dfc065680 comp rx=0 tx=0).ready entity=mon.4 client_cookie=6e068c55a7b60a51 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe287fff640 1 --2- 172.21.15.178:0/3799520982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2880993e0 0x7fe288141cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.094+0000 7fe34affd640 1 --2- 172.21.15.178:0/3346844609 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e510 0x7fe34c09d580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe340009970 tx=0x7fe3400091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f9b26ee47c71794 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe28d13a640 1 --2- 172.21.15.178:0/3799520982 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe2880a25a0 0x7fe288145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:07.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fcab37fe640 1 -- 172.21.15.178:0/631585416 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fcabc00c940 con 0x7fcacc098900 2024-09-15T01:32:07.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fcad1f9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fcad1f9d640 1 -- 172.21.15.178:0/631585416 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcacc04c4a0 con 0x7fcacc098900 2024-09-15T01:32:07.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fcab37fe640 1 --2- 172.21.15.178:0/631585416 >> v2:172.21.15.204:6812/28210 conn(0x7fcab80397b0 0x7fcab803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fcab37fe640 1 -- 172.21.15.178:0/631585416 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7fcabc04b1a0 con 0x7fcacc098900 2024-09-15T01:32:07.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fcacaffd640 1 --2- 172.21.15.178:0/631585416 >> v2:172.21.15.204:6812/28210 conn(0x7fcab80397b0 0x7fcab803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fcacaffd640 10 monclient: get_auth_request con 0x7fcab80397b0 auth_method 0 2024-09-15T01:32:07.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fcab37fe640 1 --2- 172.21.15.178:0/631585416 >> v2:172.21.15.204:6800/28753 conn(0x7fcab803f250 0x7fcab8041680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fcab37fe640 1 -- 172.21.15.178:0/631585416 --> v2:172.21.15.204:6800/28753 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcab8041d10 con 0x7fcab803f250 2024-09-15T01:32:07.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fcab37fe640 1 -- 172.21.15.178:0/631585416 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcabc04d700 con 0x7fcacc098900 2024-09-15T01:32:07.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fcab37fe640 10 monclient: handle_get_version_reply finishing 1 version 19 2024-09-15T01:32:07.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7f8e00a63640 1 -- 172.21.15.178:0/2278738365 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8df80018f0 con 0x7f8df40bcc30 2024-09-15T01:32:07.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7f8e00a63640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7f8e00a63640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7f8e00a63640 10 monclient: dump: 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.190 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7f8e00a63640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7f8e00a63640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7f8e00a63640 1 -- 172.21.15.178:0/2278738365 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc0a4050 con 0x7f8df40bcc30 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7f8e02a67640 5 monclient: authenticate success, global_id 4243 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7f8e02a67640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7f8e00a63640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7f8e02a67640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7f8e00a63640 1 -- 172.21.15.178:0/2278738365 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8dfc0a8400 con 0x7f8df40bcc30 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7f8e02a67640 4 set_mon_vals no callback set 2024-09-15T01:32:07.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7f8e00a63640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7f8e00a63640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7f8e00a63640 10 monclient: dump: 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.192 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.193 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.193 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.193 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.193 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.193 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe34a7fc640 1 -- 172.21.15.178:0/3346844609 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe34002ad90 con 0x7fe34c06e510 2024-09-15T01:32:07.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe34a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe34a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:07.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe34a7fc640 10 monclient: dump: 2024-09-15T01:32:07.193 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.193 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.194 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe34a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe28c939640 1 --2- 172.21.15.178:0/3799520982 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe288098900 0x7fe288142600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00c440 tx=0x7fe27c007ea0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe34a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe351cb8640 5 monclient: authenticate success, global_id 4295 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe351cb8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe34a7fc640 1 -- 172.21.15.178:0/3346844609 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3400313a0 con 0x7fe34c06e510 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe34a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe351cb8640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.095+0000 7fe351cb8640 4 set_mon_vals no callback set 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe351cb8640 10 monclient: shutdown 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe351cb8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe351cb8640 1 -- 172.21.15.178:0/3346844609 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e510 msgr2=0x7fe34c09d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe34a7fc640 1 -- 172.21.15.178:0/3346844609 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe34002ad90 con 0x7fe34c06e510 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe351cb8640 1 --2- 172.21.15.178:0/3346844609 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e510 0x7fe34c09d580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe340009970 tx=0x7fe3400091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.096+0000 7fe34a7fc640 10 monclient: discarding stray monitor message mon_map magic: 0 v1 2024-09-15T01:32:07.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe351cb8640 1 -- 172.21.15.178:0/3346844609 shutdown_connections 2024-09-15T01:32:07.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe351cb8640 1 --2- 172.21.15.178:0/3346844609 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e510 0x7fe34c09d580 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe351cb8640 1 --2- 172.21.15.178:0/3346844609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe34c12f5d0 0x7fe34c12f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe351cb8640 1 -- 172.21.15.178:0/3346844609 >> 172.21.15.178:0/3346844609 conn(0x7fe34c068a60 msgr2=0x7fe34c0a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fcacbfff640 1 --2- 172.21.15.178:0/631585416 >> v2:172.21.15.204:6800/28753 conn(0x7fcab803f250 0x7fcab8041680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fcacbfff640 10 monclient: get_auth_request con 0x7fcab803f250 auth_method 0 2024-09-15T01:32:07.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe351cb8640 1 -- 172.21.15.178:0/3346844609 shutdown_connections 2024-09-15T01:32:07.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe284ff9640 1 -- 172.21.15.178:0/3799520982 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe27c000fe0 con 0x7fe288098900 2024-09-15T01:32:07.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe284ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe284ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:07.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe284ff9640 10 monclient: dump: 2024-09-15T01:32:07.198 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.198 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.198 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.198 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.198 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.198 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.198 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.198 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe284ff9640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe284ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe284ff9640 1 -- 172.21.15.178:0/3799520982 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c0070d0 con 0x7fe288098900 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe28eb9d640 5 monclient: authenticate success, global_id 4292 2024-09-15T01:32:07.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe28eb9d640 10 monclient: _renew_subs 2024-09-15T01:32:07.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe351cb8640 1 -- 172.21.15.178:0/3346844609 wait complete. 2024-09-15T01:32:07.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe28eb9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:07.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3799520982 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe274009390 con 0x7fe288098900 2024-09-15T01:32:07.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe284ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe284ff9640 1 -- 172.21.15.178:0/3799520982 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe27c0076c0 con 0x7fe288098900 2024-09-15T01:32:07.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe284ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe284ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe284ff9640 10 monclient: dump: 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.201 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.202 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.202 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.202 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.202 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe28eb9d640 10 monclient: _renew_subs 2024-09-15T01:32:07.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe28eb9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:07.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3799520982 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe288127310 con 0x7fe288098900 2024-09-15T01:32:07.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.097+0000 7fe286ffd640 4 set_mon_vals no callback set 2024-09-15T01:32:07.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7f8e02a67640 10 monclient: shutdown 2024-09-15T01:32:07.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7f8e02a67640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7f8e02a67640 1 -- 172.21.15.178:0/2278738365 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8df40bcc30 msgr2=0x7f8df40c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7f8e02a67640 1 --2- 172.21.15.178:0/2278738365 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8df40bcc30 0x7f8df40c39d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc071a80 tx=0x7f8dfc065680 comp rx=0 tx=0).stop 2024-09-15T01:32:07.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7fe351cb8640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7fe351cb8640 10 monclient: monmap: 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.204 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.205 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.205 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.205 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.205 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7f8e02a67640 1 -- 172.21.15.178:0/2278738365 shutdown_connections 2024-09-15T01:32:07.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7f8e02a67640 1 --2- 172.21.15.178:0/2278738365 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8df40bcc30 0x7f8df40c39d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7f8e02a67640 1 --2- 172.21.15.178:0/2278738365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8df40bc860 0x7f8df40c1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7f8e02a67640 1 -- 172.21.15.178:0/2278738365 >> 172.21.15.178:0/2278738365 conn(0x7f8df402c1f0 msgr2=0x7f8df402c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7fcacaffd640 1 --2- 172.21.15.178:0/631585416 >> v2:172.21.15.204:6812/28210 conn(0x7fcab80397b0 0x7fcab803bc60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcac002bfd0 tx=0x7fcac0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7fe28eb9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:07.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3799520982 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe28804c4a0 con 0x7fe288098900 2024-09-15T01:32:07.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7f8e02a67640 1 -- 172.21.15.178:0/2278738365 shutdown_connections 2024-09-15T01:32:07.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7fcacbfff640 1 --2- 172.21.15.178:0/631585416 >> v2:172.21.15.204:6800/28753 conn(0x7fcab803f250 0x7fcab8041680 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7f8e02a67640 1 -- 172.21.15.178:0/2278738365 wait complete. 2024-09-15T01:32:07.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.098+0000 7fe284ff9640 1 -- 172.21.15.178:0/3799520982 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe27c007240 con 0x7fe288098900 2024-09-15T01:32:07.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe351cb8640 1 Processor -- start 2024-09-15T01:32:07.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe351cb8640 1 -- start start 2024-09-15T01:32:07.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe351cb8640 10 monclient: init 2024-09-15T01:32:07.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe284ff9640 1 --2- 172.21.15.178:0/3799520982 >> v2:172.21.15.204:6812/28210 conn(0x7fe26c0397e0 0x7fe26c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe284ff9640 1 -- 172.21.15.178:0/3799520982 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7fe27c049ff0 con 0x7fe288098900 2024-09-15T01:32:07.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe284ff9640 1 --2- 172.21.15.178:0/3799520982 >> v2:172.21.15.204:6808/28754 conn(0x7fe26c03f280 0x7fe26c0416b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe287fff640 1 --2- 172.21.15.178:0/3799520982 >> v2:172.21.15.204:6812/28210 conn(0x7fe26c0397e0 0x7fe26c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe284ff9640 1 -- 172.21.15.178:0/3799520982 --> v2:172.21.15.204:6808/28754 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe26c041d40 con 0x7fe26c03f280 2024-09-15T01:32:07.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe284ff9640 1 -- 172.21.15.178:0/3799520982 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe27c04c550 con 0x7fe288098900 2024-09-15T01:32:07.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe284ff9640 10 monclient: handle_get_version_reply finishing 1 version 19 2024-09-15T01:32:07.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe287fff640 10 monclient: get_auth_request con 0x7fe26c0397e0 auth_method 0 2024-09-15T01:32:07.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fcab37fe640 1 -- 172.21.15.178:0/631585416 <== osd.4 v2:172.21.15.204:6800/28753 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fcab8041d10 con 0x7fcab803f250 2024-09-15T01:32:07.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7f8e02a67640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7f8e02a67640 10 monclient: monmap: 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.210 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe28d13a640 1 --2- 172.21.15.178:0/3799520982 >> v2:172.21.15.204:6808/28754 conn(0x7fe26c03f280 0x7fe26c0416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe28d13a640 10 monclient: get_auth_request con 0x7fe26c03f280 auth_method 0 2024-09-15T01:32:07.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe287fff640 1 --2- 172.21.15.178:0/3799520982 >> v2:172.21.15.204:6812/28210 conn(0x7fe26c0397e0 0x7fe26c03bc90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe274009130 tx=0x7fe274032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe351cb8640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe351cb8640 10 monclient: _add_conns ranks=[6,2,0,7,8,3,4,5,1] 2024-09-15T01:32:07.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe351cb8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe34c06e510 0x7fe34c141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe351cb8640 10 monclient(hunting): picked mon.noname-d con 0x7fe34c06e510 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:07.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe351cb8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e8e0 0x7fe34c12e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe351cb8640 10 monclient(hunting): picked mon.noname-b con 0x7fe34c06e8e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:07.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe351cb8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe34c12f5d0 0x7fe34c1285c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7fe28d13a640 1 --2- 172.21.15.178:0/3799520982 >> v2:172.21.15.204:6808/28754 conn(0x7fe26c03f280 0x7fe26c0416b0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe34b7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe34c06e510 0x7fe34c141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.099+0000 7f8e02a67640 1 Processor -- start 2024-09-15T01:32:07.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe34b7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe34c06e510 0x7fe34c141ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:41924/0 (socket says 172.21.15.178:41924) 2024-09-15T01:32:07.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 1 -- start start 2024-09-15T01:32:07.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 10 monclient: init 2024-09-15T01:32:07.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe34affd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e8e0 0x7fe34c12e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe34affd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e8e0 0x7fe34c12e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:49372/0 (socket says 172.21.15.178:49372) 2024-09-15T01:32:07.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 10 monclient: _add_conns ranks=[0,6,8,1,2,7,5,3,4] 2024-09-15T01:32:07.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe34bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe34c12f5d0 0x7fe34c1285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8df4025750 0x7f8df40b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe284ff9640 1 -- 172.21.15.178:0/3799520982 <== osd.5 v2:172.21.15.204:6808/28754 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fe26c041d40 con 0x7fe26c03f280 2024-09-15T01:32:07.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe34bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe34c12f5d0 0x7fe34c1285c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33232/0 (socket says 172.21.15.178:33232) 2024-09-15T01:32:07.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 10 monclient(hunting): picked mon.noname-a con 0x7f8df4025750 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:07.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe351cb8640 10 monclient(hunting): picked mon.noname-a con 0x7fe34c12f5d0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:07.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe351cb8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe34b7fe640 1 -- 172.21.15.178:0/1307698058 learned_addr learned my addr 172.21.15.178:0/1307698058 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c12bd90 con 0x7fe34c12f5d0 2024-09-15T01:32:07.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe351cb8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c06bb00 con 0x7fe34c06e8e0 2024-09-15T01:32:07.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe351cb8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c09ccc0 con 0x7fe34c06e510 2024-09-15T01:32:07.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe351cb8640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe351cb8640 10 monclient(hunting): authenticate will time out at 731.851703s 2024-09-15T01:32:07.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe34bfff640 10 monclient(hunting): get_auth_request con 0x7fe34c12f5d0 auth_method 0 2024-09-15T01:32:07.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe34bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe34bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7fe34bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8df40bc860 0x7f8df40a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 10 monclient(hunting): get_auth_request con 0x7fe34c06e510 auth_method 0 2024-09-15T01:32:07.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 10 monclient(hunting): picked mon.noname-d con 0x7f8df40bc860 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e01a65640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8df4025750 0x7f8df40b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8df40bcc30 0x7f8df40a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e01a65640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8df4025750 0x7f8df40b8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33246/0 (socket says 172.21.15.178:33246) 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 10 monclient(hunting): picked mon.noname-e con 0x7f8df40bcc30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4024410 con 0x7f8df4025750 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4024140 con 0x7f8df40bc860 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40a90f0 con 0x7f8df40bcc30 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.100+0000 7f8e02a67640 10 monclient(hunting): authenticate will time out at 731.851978s 2024-09-15T01:32:07.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01a65640 1 -- 172.21.15.178:0/2040917564 learned_addr learned my addr 172.21.15.178:0/2040917564 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01a65640 10 monclient(hunting): get_auth_request con 0x7f8df4025750 auth_method 0 2024-09-15T01:32:07.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01a65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01a65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01a65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 1 --2- 172.21.15.178:0/2040917564 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8df40bcc30 0x7f8df40a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 10 monclient(hunting): get_auth_request con 0x7f8df40bcc30 auth_method 0 2024-09-15T01:32:07.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01a65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01a65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01a65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01264640 1 --2- 172.21.15.178:0/2040917564 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8df40bc860 0x7f8df40a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 10 monclient(hunting): handle_auth_done global_id 4326 payload 824 2024-09-15T01:32:07.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 1 -- 172.21.15.178:0/1307698058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e8e0 msgr2=0x7fe34c12e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 1 --2- 172.21.15.178:0/1307698058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e8e0 0x7fe34c12e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 1 -- 172.21.15.178:0/1307698058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe34c12f5d0 msgr2=0x7fe34c1285c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34affd640 1 --2- 172.21.15.178:0/1307698058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe34c06e8e0 0x7fe34c12e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-15T01:32:07.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 1 --2- 172.21.15.178:0/1307698058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe34c12f5d0 0x7fe34c1285c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 1 monclient: found mon.noname-d 2024-09-15T01:32:07.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34b7fe640 1 -- 172.21.15.178:0/1307698058 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe340009450 con 0x7fe34c06e510 2024-09-15T01:32:07.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01264640 10 monclient(hunting): get_auth_request con 0x7f8df40bc860 auth_method 0 2024-09-15T01:32:07.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01264640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01264640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01264640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01264640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01264640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01264640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 10 monclient(hunting): handle_auth_done global_id 4301 payload 824 2024-09-15T01:32:07.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 1 -- 172.21.15.178:0/2040917564 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8df40bc860 msgr2=0x7f8df40a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7fe34bfff640 1 --2- 172.21.15.178:0/1307698058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe34c12f5d0 0x7fe34c1285c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:07.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 1 --2- 172.21.15.178:0/2040917564 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8df40bc860 0x7f8df40a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 1 -- 172.21.15.178:0/2040917564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8df4025750 msgr2=0x7f8df40b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 1 --2- 172.21.15.178:0/2040917564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8df4025750 0x7f8df40b8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 1 monclient: found mon.noname-e 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e02266640 1 -- 172.21.15.178:0/2040917564 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc04c4a0 con 0x7f8df40bcc30 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01264640 1 --2- 172.21.15.178:0/2040917564 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8df40bc860 0x7f8df40a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.101+0000 7f8e01a65640 1 --2- 172.21.15.178:0/2040917564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8df4025750 0x7f8df40b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8e02266640 1 --2- 172.21.15.178:0/2040917564 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8df40bcc30 0x7f8df40a5fe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc066fa0 tx=0x7f8dfc0ac000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df1ffb640 1 -- 172.21.15.178:0/2040917564 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8dfc0a8b90 con 0x7f8df40bcc30 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df1ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df1ffb640 10 monclient: dump: 2024-09-15T01:32:07.228 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.229 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.229 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.229 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.229 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.229 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.229 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.230 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.230 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.230 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.230 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.230 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.230 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.230 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.230 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.231 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df1ffb640 1 -- 172.21.15.178:0/2040917564 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc0a8cf0 con 0x7f8df40bcc30 2024-09-15T01:32:07.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df1ffb640 1 -- 172.21.15.178:0/2040917564 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8dfc0a8e60 con 0x7f8df40bcc30 2024-09-15T01:32:07.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df1ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:07.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df1ffb640 10 monclient: dump: 2024-09-15T01:32:07.232 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.232 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.232 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.232 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.232 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.232 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.232 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.232 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.232 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.232 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.232 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.232 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.233 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.233 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.233 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.233 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8df3fff640 4 set_mon_vals no callback set 2024-09-15T01:32:07.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8e02a67640 5 monclient: authenticate success, global_id 4301 2024-09-15T01:32:07.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8e02a67640 10 monclient: _renew_subs 2024-09-15T01:32:07.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8e02a67640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:07.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7f8e02a67640 1 -- 172.21.15.178:0/2040917564 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df40a8e70 con 0x7f8df40bcc30 2024-09-15T01:32:07.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.102+0000 7fe34b7fe640 1 --2- 172.21.15.178:0/1307698058 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe34c06e510 0x7fe34c141ff0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00c880 tx=0x7fe33c008040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7f8e02a67640 10 monclient: _renew_subs 2024-09-15T01:32:07.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7f8e02a67640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:07.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7f8e02a67640 1 -- 172.21.15.178:0/2040917564 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df40cbfa0 con 0x7f8df40bcc30 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7fe3337fe640 1 -- 172.21.15.178:0/1307698058 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe33c00ce10 con 0x7fe34c06e510 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7fe3337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7fe3337fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7fe3337fe640 10 monclient: dump: 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.235 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.236 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.236 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.236 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.236 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.236 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.236 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.236 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7fe3337fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7fe3337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7fe3337fe640 1 -- 172.21.15.178:0/1307698058 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33c007cb0 con 0x7fe34c06e510 2024-09-15T01:32:07.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7fe3337fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7f8df1ffb640 1 -- 172.21.15.178:0/2040917564 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8dfc0bc460 con 0x7f8df40bcc30 2024-09-15T01:32:07.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7f8df1ffb640 1 --2- 172.21.15.178:0/2040917564 >> v2:172.21.15.204:6812/28210 conn(0x7f8de00397b0 0x7f8de003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7f8df1ffb640 1 -- 172.21.15.178:0/2040917564 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f8dfc0e3490 con 0x7f8df40bcc30 2024-09-15T01:32:07.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.103+0000 7fe3497fa640 4 set_mon_vals no callback set 2024-09-15T01:32:07.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7fe3337fe640 1 -- 172.21.15.178:0/1307698058 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe33c007e20 con 0x7fe34c06e510 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7fe3337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7fe3337fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7fe3337fe640 10 monclient: dump: 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.238 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.239 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.239 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.239 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.239 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7fe351cb8640 5 monclient: authenticate success, global_id 4326 2024-09-15T01:32:07.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7fe351cb8640 10 monclient: _renew_subs 2024-09-15T01:32:07.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7fe351cb8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34c142ea0 con 0x7fe34c06e510 2024-09-15T01:32:07.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7f8e02a67640 1 --2- 172.21.15.178:0/2040917564 >> v2:172.21.15.204:6801/28752 conn(0x7f8df40286d0 0x7f8df40a6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.240 INFO:teuthology.orchestra.run.smithi178.stdout:47244640259 2024-09-15T01:32:07.240 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T01:32:07.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7fe351cb8640 10 monclient: _renew_subs 2024-09-15T01:32:07.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7fe351cb8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34c12ba20 con 0x7fe34c06e510 2024-09-15T01:32:07.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7f8e02a67640 1 -- 172.21.15.178:0/2040917564 --> v2:172.21.15.204:6801/28752 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8df40a8e70 con 0x7f8df40286d0 2024-09-15T01:32:07.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7f8e01a65640 1 --2- 172.21.15.178:0/2040917564 >> v2:172.21.15.204:6812/28210 conn(0x7f8de00397b0 0x7f8de003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7f8e01a65640 10 monclient: get_auth_request con 0x7f8de00397b0 auth_method 0 2024-09-15T01:32:07.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.104+0000 7f8e01264640 1 --2- 172.21.15.178:0/2040917564 >> v2:172.21.15.204:6801/28752 conn(0x7f8df40286d0 0x7f8df40a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.105+0000 7f8e01264640 10 monclient: get_auth_request con 0x7f8df40286d0 auth_method 0 2024-09-15T01:32:07.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.105+0000 7f8e01a65640 1 --2- 172.21.15.178:0/2040917564 >> v2:172.21.15.204:6812/28210 conn(0x7f8de00397b0 0x7f8de003bc60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8df8007fd0 tx=0x7f8df8000bc0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.105+0000 7f8e01264640 1 --2- 172.21.15.178:0/2040917564 >> v2:172.21.15.204:6801/28752 conn(0x7f8df40286d0 0x7f8df40a6520 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.105+0000 7fe351cb8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.105+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe34c04c4a0 con 0x7fe34c06e510 2024-09-15T01:32:07.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.105+0000 7fe3337fe640 1 -- 172.21.15.178:0/1307698058 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe33c007830 con 0x7fe34c06e510 2024-09-15T01:32:07.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.106+0000 7fe3337fe640 1 --2- 172.21.15.178:0/1307698058 >> v2:172.21.15.204:6812/28210 conn(0x7fe3380397b0 0x7fe33803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.106+0000 7fe3337fe640 1 -- 172.21.15.178:0/1307698058 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7fe33c04b3c0 con 0x7fe34c06e510 2024-09-15T01:32:07.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.106+0000 7f8df1ffb640 1 -- 172.21.15.178:0/2040917564 <== osd.3 v2:172.21.15.204:6801/28752 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f8df40a8e70 con 0x7f8df40286d0 2024-09-15T01:32:07.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.106+0000 7fe3337fe640 1 --2- 172.21.15.178:0/1307698058 >> v2:172.21.15.178:6800/29232 conn(0x7fe33803f2d0 0x7fe338041700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.106+0000 7fe34affd640 1 --2- 172.21.15.178:0/1307698058 >> v2:172.21.15.204:6812/28210 conn(0x7fe3380397b0 0x7fe33803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.106+0000 7fe3337fe640 1 -- 172.21.15.178:0/1307698058 --> v2:172.21.15.178:6800/29232 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe338041d90 con 0x7fe33803f2d0 2024-09-15T01:32:07.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.106+0000 7fe3337fe640 1 -- 172.21.15.178:0/1307698058 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe33c04d950 con 0x7fe34c06e510 2024-09-15T01:32:07.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.106+0000 7fe3337fe640 10 monclient: handle_get_version_reply finishing 1 version 19 2024-09-15T01:32:07.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.106+0000 7fe34affd640 10 monclient: get_auth_request con 0x7fe3380397b0 auth_method 0 2024-09-15T01:32:07.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.106+0000 7fe34bfff640 1 --2- 172.21.15.178:0/1307698058 >> v2:172.21.15.178:6800/29232 conn(0x7fe33803f2d0 0x7fe338041700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.106+0000 7fe34bfff640 10 monclient: get_auth_request con 0x7fe33803f2d0 auth_method 0 2024-09-15T01:32:07.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.107+0000 7fe34bfff640 1 --2- 172.21.15.178:0/1307698058 >> v2:172.21.15.178:6800/29232 conn(0x7fe33803f2d0 0x7fe338041700 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.107+0000 7fe3337fe640 1 -- 172.21.15.178:0/1307698058 <== osd.0 v2:172.21.15.178:6800/29232 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fe338041d90 con 0x7fe33803f2d0 2024-09-15T01:32:07.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.107+0000 7fe34affd640 1 --2- 172.21.15.178:0/1307698058 >> v2:172.21.15.204:6812/28210 conn(0x7fe3380397b0 0x7fe33803bc60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe340034000 tx=0x7fe340032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.246 INFO:teuthology.orchestra.run.smithi178.stdout:47244640259 2024-09-15T01:32:07.246 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T01:32:07.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.115+0000 7fcad1f9d640 1 -- 172.21.15.178:0/631585416 --> v2:172.21.15.204:6800/28753 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcacc04bc80 con 0x7fcab803f250 2024-09-15T01:32:07.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.115+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3799520982 --> v2:172.21.15.204:6808/28754 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe28804bc80 con 0x7fe26c03f280 2024-09-15T01:32:07.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcab37fe640 1 -- 172.21.15.178:0/631585416 <== osd.4 v2:172.21.15.204:6800/28753 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcacc04bc80 con 0x7fcab803f250 2024-09-15T01:32:07.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fe284ff9640 1 -- 172.21.15.178:0/3799520982 <== osd.5 v2:172.21.15.204:6808/28754 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe28804bc80 con 0x7fe26c03f280 2024-09-15T01:32:07.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcad1f9d640 1 -- 172.21.15.178:0/631585416 >> v2:172.21.15.204:6800/28753 conn(0x7fcab803f250 msgr2=0x7fcab8041680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcad1f9d640 1 --2- 172.21.15.178:0/631585416 >> v2:172.21.15.204:6800/28753 conn(0x7fcab803f250 0x7fcab8041680 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3799520982 >> v2:172.21.15.204:6808/28754 conn(0x7fe26c03f280 msgr2=0x7fe26c0416b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fe28eb9d640 1 --2- 172.21.15.178:0/3799520982 >> v2:172.21.15.204:6808/28754 conn(0x7fe26c03f280 0x7fe26c0416b0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcad1f9d640 1 -- 172.21.15.178:0/631585416 >> v2:172.21.15.204:6812/28210 conn(0x7fcab80397b0 msgr2=0x7fcab803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcad1f9d640 1 --2- 172.21.15.178:0/631585416 >> v2:172.21.15.204:6812/28210 conn(0x7fcab80397b0 0x7fcab803bc60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcac002bfd0 tx=0x7fcac0032000 comp rx=0 tx=0).stop 2024-09-15T01:32:07.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcad1f9d640 10 monclient: shutdown 2024-09-15T01:32:07.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcad1f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcad1f9d640 1 -- 172.21.15.178:0/631585416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcacc098900 msgr2=0x7fcacc1380a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcad1f9d640 1 --2- 172.21.15.178:0/631585416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcacc098900 0x7fcacc1380a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00e900 tx=0x7fcabc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcacbfff640 1 -- 172.21.15.178:0/631585416 reap_dead start 2024-09-15T01:32:07.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcad1f9d640 1 -- 172.21.15.178:0/631585416 shutdown_connections 2024-09-15T01:32:07.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcad1f9d640 1 -- 172.21.15.178:0/631585416 >> 172.21.15.178:0/631585416 conn(0x7fcacc067710 msgr2=0x7fcacc0a07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3799520982 >> v2:172.21.15.204:6812/28210 conn(0x7fe26c0397e0 msgr2=0x7fe26c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fe28eb9d640 1 --2- 172.21.15.178:0/3799520982 >> v2:172.21.15.204:6812/28210 conn(0x7fe26c0397e0 0x7fe26c03bc90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe274009130 tx=0x7fe274032000 comp rx=0 tx=0).stop 2024-09-15T01:32:07.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcad1f9d640 1 -- 172.21.15.178:0/631585416 shutdown_connections 2024-09-15T01:32:07.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fe28eb9d640 10 monclient: shutdown 2024-09-15T01:32:07.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fcad1f9d640 1 -- 172.21.15.178:0/631585416 wait complete. 2024-09-15T01:32:07.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fe28eb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3799520982 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe288098900 msgr2=0x7fe288142600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fe28eb9d640 1 --2- 172.21.15.178:0/3799520982 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe288098900 0x7fe288142600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00c440 tx=0x7fe27c007ea0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fe28d13a640 1 -- 172.21.15.178:0/3799520982 reap_dead start 2024-09-15T01:32:07.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3799520982 shutdown_connections 2024-09-15T01:32:07.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.116+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3799520982 >> 172.21.15.178:0/3799520982 conn(0x7fe288067710 msgr2=0x7fe288099de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.117+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3799520982 shutdown_connections 2024-09-15T01:32:07.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.117+0000 7fe28eb9d640 1 -- 172.21.15.178:0/3799520982 wait complete. 2024-09-15T01:32:07.252 INFO:teuthology.orchestra.run.smithi178.stdout:47244640259 2024-09-15T01:32:07.253 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T01:32:07.253 INFO:teuthology.orchestra.run.smithi178.stdout:47244640259 2024-09-15T01:32:07.254 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T01:32:07.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.131+0000 7f8e02a67640 1 -- 172.21.15.178:0/2040917564 --> v2:172.21.15.204:6801/28752 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8df4006e90 con 0x7f8df40286d0 2024-09-15T01:32:07.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.131+0000 7f8df1ffb640 1 -- 172.21.15.178:0/2040917564 <== osd.3 v2:172.21.15.204:6801/28752 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8df4006e90 con 0x7f8df40286d0 2024-09-15T01:32:07.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02a67640 1 -- 172.21.15.178:0/2040917564 >> v2:172.21.15.204:6801/28752 conn(0x7f8df40286d0 msgr2=0x7f8df40a6520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02a67640 1 --2- 172.21.15.178:0/2040917564 >> v2:172.21.15.204:6801/28752 conn(0x7f8df40286d0 0x7f8df40a6520 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02a67640 1 -- 172.21.15.178:0/2040917564 >> v2:172.21.15.204:6812/28210 conn(0x7f8de00397b0 msgr2=0x7f8de003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02a67640 1 --2- 172.21.15.178:0/2040917564 >> v2:172.21.15.204:6812/28210 conn(0x7f8de00397b0 0x7f8de003bc60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8df8007fd0 tx=0x7f8df8000bc0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02a67640 10 monclient: shutdown 2024-09-15T01:32:07.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02a67640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02a67640 1 -- 172.21.15.178:0/2040917564 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8df40bcc30 msgr2=0x7f8df40a5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02a67640 1 --2- 172.21.15.178:0/2040917564 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8df40bcc30 0x7f8df40a5fe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc066fa0 tx=0x7f8dfc0ac000 comp rx=0 tx=0).stop 2024-09-15T01:32:07.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02266640 1 -- 172.21.15.178:0/2040917564 reap_dead start 2024-09-15T01:32:07.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02a67640 1 -- 172.21.15.178:0/2040917564 shutdown_connections 2024-09-15T01:32:07.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02a67640 1 -- 172.21.15.178:0/2040917564 >> 172.21.15.178:0/2040917564 conn(0x7f8df402c1f0 msgr2=0x7f8df40ba930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02a67640 1 -- 172.21.15.178:0/2040917564 shutdown_connections 2024-09-15T01:32:07.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7f8e02a67640 1 -- 172.21.15.178:0/2040917564 wait complete. 2024-09-15T01:32:07.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 --> v2:172.21.15.178:6800/29232 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe34c04bc80 con 0x7fe33803f2d0 2024-09-15T01:32:07.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.132+0000 7fe3337fe640 1 -- 172.21.15.178:0/1307698058 <== osd.0 v2:172.21.15.178:6800/29232 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe34c04bc80 con 0x7fe33803f2d0 2024-09-15T01:32:07.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 >> v2:172.21.15.178:6800/29232 conn(0x7fe33803f2d0 msgr2=0x7fe338041700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe351cb8640 1 --2- 172.21.15.178:0/1307698058 >> v2:172.21.15.178:6800/29232 conn(0x7fe33803f2d0 0x7fe338041700 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 >> v2:172.21.15.204:6812/28210 conn(0x7fe3380397b0 msgr2=0x7fe33803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe351cb8640 1 --2- 172.21.15.178:0/1307698058 >> v2:172.21.15.204:6812/28210 conn(0x7fe3380397b0 0x7fe33803bc60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe340034000 tx=0x7fe340032000 comp rx=0 tx=0).stop 2024-09-15T01:32:07.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe351cb8640 10 monclient: shutdown 2024-09-15T01:32:07.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe351cb8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe34c06e510 msgr2=0x7fe34c141ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe351cb8640 1 --2- 172.21.15.178:0/1307698058 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe34c06e510 0x7fe34c141ff0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00c880 tx=0x7fe33c008040 comp rx=0 tx=0).stop 2024-09-15T01:32:07.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe34bfff640 1 -- 172.21.15.178:0/1307698058 reap_dead start 2024-09-15T01:32:07.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 shutdown_connections 2024-09-15T01:32:07.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 >> 172.21.15.178:0/1307698058 conn(0x7fe34c068a60 msgr2=0x7fe34c0a0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 shutdown_connections 2024-09-15T01:32:07.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.133+0000 7fe351cb8640 1 -- 172.21.15.178:0/1307698058 wait complete. 2024-09-15T01:32:07.259 INFO:teuthology.orchestra.run.smithi178.stdout:47244640259 2024-09-15T01:32:07.260 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T01:32:07.260 INFO:teuthology.orchestra.run.smithi178.stdout:47244640259 2024-09-15T01:32:07.261 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T01:32:07.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.390+0000 7f87520b8640 10 monclient: get_monmap_and_config 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.390+0000 7f87520b8640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.390+0000 7f87520b8640 10 monclient: monmap: 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:07.392829+0000 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:07.392829+0000 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.394 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.395 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.395 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.395 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.395 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.395 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:07.395 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:07.395 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.391+0000 7f87520b8640 1 Processor -- start 2024-09-15T01:32:07.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.391+0000 7f87520b8640 1 -- start start 2024-09-15T01:32:07.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.391+0000 7f87520b8640 10 monclient: init 2024-09-15T01:32:07.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 10 monclient: _add_conns ranks=[1,2,4,6,9,5,10,8,3,7,0] 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f874c09c020 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f874c1353a0 0x7f874c137780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 10 monclient(hunting): picked mon.noname-b con 0x7f874c1353a0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f874c137cc0 0x7f874c13a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 10 monclient(hunting): picked mon.noname-c con 0x7f874c137cc0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874c04bf60 con 0x7f874c09c020 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c09e220 con 0x7f874c1353a0 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f874c09b270 con 0x7f874c137cc0 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f87520b8640 10 monclient(hunting): authenticate will time out at 732.143594s 2024-09-15T01:32:07.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874bfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f874c137cc0 0x7f874c13a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874b7fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f874c09c020 0x7f874c09ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:44312/0 (socket says 172.21.15.178:44312) 2024-09-15T01:32:07.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874b7fe640 1 -- 172.21.15.178:0/2665699868 learned_addr learned my addr 172.21.15.178:0/2665699868 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:07.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874bfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f874c137cc0 0x7f874c13a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:40274/0 (socket says 172.21.15.178:40274) 2024-09-15T01:32:07.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874bfff640 10 monclient(hunting): get_auth_request con 0x7f874c137cc0 auth_method 0 2024-09-15T01:32:07.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874affd640 1 --2- 172.21.15.178:0/2665699868 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f874c1353a0 0x7f874c137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874affd640 10 monclient(hunting): get_auth_request con 0x7f874c1353a0 auth_method 0 2024-09-15T01:32:07.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.392+0000 7f874affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f874affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f874bfff640 10 monclient(hunting): handle_auth_done global_id 4261 payload 274 2024-09-15T01:32:07.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f874bfff640 1 -- 172.21.15.178:0/2665699868 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f874c1353a0 msgr2=0x7f874c137780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f874bfff640 1 --2- 172.21.15.178:0/2665699868 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f874c1353a0 0x7f874c137780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f874bfff640 1 -- 172.21.15.178:0/2665699868 >> v1:172.21.15.204:6789/0 conn(0x7f874c09c020 legacy=0x7f874c09ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f874bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f874bfff640 1 monclient: found mon.noname-c 2024-09-15T01:32:07.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f874bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f874bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:07.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f874bfff640 1 -- 172.21.15.178:0/2665699868 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c09af90 con 0x7f874c137cc0 2024-09-15T01:32:07.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f874affd640 1 --2- 172.21.15.178:0/2665699868 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f874c1353a0 0x7f874c137780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f66a595d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f66a595d640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.393+0000 7f66a595d640 10 monclient: monmap: 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:07.395764+0000 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:07.395764+0000 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.405 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.406 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.406 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.406 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.406 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.406 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:07.406 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:07.406 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874bfff640 1 --2- 172.21.15.178:0/2665699868 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f874c137cc0 0x7f874c13a0a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8744009e90 tx=0x7f8744009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=534bc030d73ffd34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874a7fc640 1 -- 172.21.15.178:0/2665699868 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8744031880 con 0x7f874c137cc0 2024-09-15T01:32:07.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:07.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f66a595d640 1 Processor -- start 2024-09-15T01:32:07.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874a7fc640 10 monclient: dump: 2024-09-15T01:32:07.407 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.408 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.409 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f66a595d640 1 -- start start 2024-09-15T01:32:07.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f66a595d640 10 monclient: init 2024-09-15T01:32:07.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874a7fc640 1 -- 172.21.15.178:0/2665699868 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874402d050 con 0x7f874c137cc0 2024-09-15T01:32:07.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874a7fc640 1 -- 172.21.15.178:0/2665699868 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87440303a0 con 0x7f874c137cc0 2024-09-15T01:32:07.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f874a7fc640 10 monclient: dump: 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.411 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.412 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.412 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f87520b8640 5 monclient: authenticate success, global_id 4261 2024-09-15T01:32:07.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f87520b8640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:07.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 10 monclient: _add_conns ranks=[9,2,6,4,7,1,8,10,0,3,5] 2024-09-15T01:32:07.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f66a00a3550 0x7f66a0137220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 10 monclient(hunting): picked mon.noname-a con 0x7f66a00a3550 addr v2:172.21.15.178:3300/0 2024-09-15T01:32:07.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66a006e3b0 0x7f66a006e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 10 monclient(hunting): picked mon.noname-b con 0x7f66a006e3b0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:07.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a0067130 0x7f66a00675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 10 monclient(hunting): picked mon.noname-d con 0x7f66a0067130 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:07.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a006bb90 con 0x7f66a00a3550 2024-09-15T01:32:07.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a00a3920 con 0x7f66a006e3b0 2024-09-15T01:32:07.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f66a009e3f0 con 0x7f66a0067130 2024-09-15T01:32:07.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f66a595d640 10 monclient(hunting): authenticate will time out at 732.146219s 2024-09-15T01:32:07.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669e7fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66a006e3b0 0x7f66a006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669effd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f66a00a3550 0x7f66a0137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669e7fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66a006e3b0 0x7f66a006e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:49388/0 (socket says 172.21.15.178:49388) 2024-09-15T01:32:07.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669e7fc640 1 -- 172.21.15.178:0/3427855023 learned_addr learned my addr 172.21.15.178:0/3427855023 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669e7fc640 10 monclient(hunting): get_auth_request con 0x7f66a006e3b0 auth_method 0 2024-09-15T01:32:07.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669effd640 10 monclient(hunting): get_auth_request con 0x7f66a00a3550 auth_method 0 2024-09-15T01:32:07.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669f7fe640 1 --2- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a0067130 0x7f66a00675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669f7fe640 10 monclient(hunting): get_auth_request con 0x7f66a0067130 auth_method 0 2024-09-15T01:32:07.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.394+0000 7f87520b8640 4 set_mon_vals no callback set 2024-09-15T01:32:07.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f87520b8640 10 monclient: shutdown 2024-09-15T01:32:07.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669effd640 10 monclient(hunting): handle_auth_done global_id 4302 payload 274 2024-09-15T01:32:07.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f669effd640 1 -- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a0067130 msgr2=0x7f66a00675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f87520b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f87520b8640 1 -- 172.21.15.178:0/2665699868 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f874c137cc0 msgr2=0x7f874c13a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f87520b8640 1 --2- 172.21.15.178:0/2665699868 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f874c137cc0 0x7f874c13a0a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8744009e90 tx=0x7f8744009780 comp rx=0 tx=0).stop 2024-09-15T01:32:07.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f87520b8640 1 -- 172.21.15.178:0/2665699868 shutdown_connections 2024-09-15T01:32:07.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f87520b8640 1 --2- 172.21.15.178:0/2665699868 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f874c137cc0 0x7f874c13a0a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f87520b8640 1 --2- 172.21.15.178:0/2665699868 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f874c1353a0 0x7f874c137780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.395+0000 7f87520b8640 1 -- 172.21.15.178:0/2665699868 >> 172.21.15.178:0/2665699868 conn(0x7f874c068a60 msgr2=0x7f874c0a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f87520b8640 1 -- 172.21.15.178:0/2665699868 shutdown_connections 2024-09-15T01:32:07.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f87520b8640 1 -- 172.21.15.178:0/2665699868 wait complete. 2024-09-15T01:32:07.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f669effd640 1 --2- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a0067130 0x7f66a00675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f669effd640 1 -- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66a006e3b0 msgr2=0x7f66a006e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f669effd640 1 --2- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66a006e3b0 0x7f66a006e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f669effd640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f669effd640 1 monclient: found mon.noname-a 2024-09-15T01:32:07.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f669effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f669effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:07.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f669effd640 1 -- 172.21.15.178:0/3427855023 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a004bf60 con 0x7f66a00a3550 2024-09-15T01:32:07.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f669e7fc640 1 --2- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66a006e3b0 0x7f66a006e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f669f7fe640 1 --2- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a0067130 0x7f66a00675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f87520b8640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.396+0000 7f87520b8640 10 monclient: monmap: 2024-09-15T01:32:07.424 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.424 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:07.424 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:07.424 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:07.424 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.424 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:07.424 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:07.424 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:07.425 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.425 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.425 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.425 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.425 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.425 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.425 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.425 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 1 Processor -- start 2024-09-15T01:32:07.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 1 -- start start 2024-09-15T01:32:07.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 10 monclient: init 2024-09-15T01:32:07.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f669effd640 1 --2- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a00a3550 0x7f66a0137220 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6690009850 tx=0x7f66900090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9074486142a93067 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 10 monclient: _add_conns ranks=[7,3,1,8,2,4,5,6,0] 2024-09-15T01:32:07.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f874c09c020 0x7f874c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 10 monclient(hunting): picked mon.noname-i con 0x7f874c09c020 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:07.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f874c1353a0 0x7f874c06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 10 monclient(hunting): picked mon.noname-g con 0x7f874c1353a0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:07.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c137cc0 0x7f874c06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 10 monclient(hunting): picked mon.noname-f con 0x7f874c137cc0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:07.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c09f2d0 con 0x7f874c137cc0 2024-09-15T01:32:07.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c11b3e0 con 0x7f874c1353a0 2024-09-15T01:32:07.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f874c11c320 con 0x7f874c09c020 2024-09-15T01:32:07.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f87520b8640 10 monclient(hunting): authenticate will time out at 732.148878s 2024-09-15T01:32:07.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f669dffb640 1 -- 172.21.15.178:0/3427855023 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f669002abc0 con 0x7f66a00a3550 2024-09-15T01:32:07.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f874bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c137cc0 0x7f874c06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f669dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f874bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c137cc0 0x7f874c06dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:52342/0 (socket says 172.21.15.178:52342) 2024-09-15T01:32:07.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 1 -- 172.21.15.178:0/1536686792 learned_addr learned my addr 172.21.15.178:0/1536686792 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 10 monclient(hunting): get_auth_request con 0x7f874c137cc0 auth_method 0 2024-09-15T01:32:07.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f669dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:07.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f669dffb640 10 monclient: dump: 2024-09-15T01:32:07.431 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.432 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.433 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.433 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874affd640 1 --2- 172.21.15.178:0/1536686792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f874c1353a0 0x7f874c06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874b7fe640 1 --2- 172.21.15.178:0/1536686792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f874c09c020 0x7f874c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f669dffb640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874affd640 10 monclient(hunting): get_auth_request con 0x7f874c1353a0 auth_method 0 2024-09-15T01:32:07.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874b7fe640 10 monclient(hunting): get_auth_request con 0x7f874c09c020 auth_method 0 2024-09-15T01:32:07.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f669dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f669dffb640 1 -- 172.21.15.178:0/3427855023 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669002ad20 con 0x7f66a00a3550 2024-09-15T01:32:07.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f66a595d640 5 monclient: authenticate success, global_id 4302 2024-09-15T01:32:07.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.397+0000 7f66a595d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:07.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f669dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f66a595d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:07.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 10 monclient(hunting): handle_auth_done global_id 4285 payload 824 2024-09-15T01:32:07.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 1 -- 172.21.15.178:0/1536686792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f874c09c020 msgr2=0x7f874c06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 1 --2- 172.21.15.178:0/1536686792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f874c09c020 0x7f874c06d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 1 -- 172.21.15.178:0/1536686792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f874c1353a0 msgr2=0x7f874c06d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 1 --2- 172.21.15.178:0/1536686792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f874c1353a0 0x7f874c06d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 1 monclient: found mon.noname-f 2024-09-15T01:32:07.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 1 -- 172.21.15.178:0/1536686792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8740009390 con 0x7f874c137cc0 2024-09-15T01:32:07.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f66a595d640 4 set_mon_vals no callback set 2024-09-15T01:32:07.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f669dffb640 1 -- 172.21.15.178:0/3427855023 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f669002abc0 con 0x7f66a00a3550 2024-09-15T01:32:07.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f669dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f669dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:07.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f669dffb640 10 monclient: dump: 2024-09-15T01:32:07.438 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.438 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.438 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.438 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.439 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.439 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.439 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.439 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.439 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.439 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.439 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.439 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.439 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.439 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.439 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.439 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f874bfff640 1 --2- 172.21.15.178:0/1536686792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c137cc0 0x7f874c06dc20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8744009d30 tx=0x7f8744037000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f87337fe640 1 -- 172.21.15.178:0/1536686792 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f874403cab0 con 0x7f874c137cc0 2024-09-15T01:32:07.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f87337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f87337fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:32:07.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f87337fe640 10 monclient: dump: 2024-09-15T01:32:07.441 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.441 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.441 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f87337fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f87337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f87520b8640 5 monclient: authenticate success, global_id 4285 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f87337fe640 1 -- 172.21.15.178:0/1536686792 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874403cc50 con 0x7f874c137cc0 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f87337fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f87337fe640 1 -- 172.21.15.178:0/1536686792 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f874403cdc0 con 0x7f874c137cc0 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.398+0000 7f87337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f87337fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f87337fe640 10 monclient: dump: 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.443 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f87520b8640 10 monclient: _renew_subs 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f87520b8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f87520b8640 1 -- 172.21.15.178:0/1536686792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8744009a20 con 0x7f874c137cc0 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f8749ffb640 4 set_mon_vals no callback set 2024-09-15T01:32:07.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f87520b8640 10 monclient: _renew_subs 2024-09-15T01:32:07.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f87520b8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f87520b8640 1 -- 172.21.15.178:0/1536686792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c11b100 con 0x7f874c137cc0 2024-09-15T01:32:07.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f66a595d640 10 monclient: shutdown 2024-09-15T01:32:07.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f66a595d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f66a595d640 1 -- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a00a3550 msgr2=0x7f66a0137220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f66a595d640 1 --2- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a00a3550 0x7f66a0137220 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6690009850 tx=0x7f66900090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f66a595d640 1 -- 172.21.15.178:0/3427855023 shutdown_connections 2024-09-15T01:32:07.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f66a595d640 1 --2- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a0067130 0x7f66a00675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f66a595d640 1 --2- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66a006e3b0 0x7f66a006e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f66a595d640 1 --2- 172.21.15.178:0/3427855023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a00a3550 0x7f66a0137220 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f66a595d640 1 -- 172.21.15.178:0/3427855023 >> 172.21.15.178:0/3427855023 conn(0x7f66a0068a60 msgr2=0x7f66a00a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f66a595d640 1 -- 172.21.15.178:0/3427855023 shutdown_connections 2024-09-15T01:32:07.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.399+0000 7f66a595d640 1 -- 172.21.15.178:0/3427855023 wait complete. 2024-09-15T01:32:07.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f87337fe640 1 -- 172.21.15.178:0/1536686792 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8744003040 con 0x7f874c137cc0 2024-09-15T01:32:07.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f87337fe640 1 --2- 172.21.15.178:0/1536686792 >> v2:172.21.15.204:6812/28210 conn(0x7f87380397b0 0x7f873803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f66a595d640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f66a595d640 10 monclient: monmap: 2024-09-15T01:32:07.447 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.447 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:07.447 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:07.447 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:07.447 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.447 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:07.447 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:07.447 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:07.448 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.448 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.448 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.448 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.448 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.448 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.448 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.448 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f87337fe640 1 -- 172.21.15.178:0/1536686792 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f874406c6f0 con 0x7f874c137cc0 2024-09-15T01:32:07.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f87520b8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f87520b8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f87520b8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f87520b8640 1 -- 172.21.15.178:0/1536686792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c04b720 con 0x7f874c137cc0 2024-09-15T01:32:07.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f66a595d640 1 Processor -- start 2024-09-15T01:32:07.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f66a595d640 1 -- start start 2024-09-15T01:32:07.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f66a595d640 10 monclient: init 2024-09-15T01:32:07.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f874b7fe640 1 --2- 172.21.15.178:0/1536686792 >> v2:172.21.15.204:6812/28210 conn(0x7f87380397b0 0x7f873803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.400+0000 7f874b7fe640 10 monclient: get_auth_request con 0x7f87380397b0 auth_method 0 2024-09-15T01:32:07.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 10 monclient: _add_conns ranks=[1,5,6,2,3,4,8,0,7] 2024-09-15T01:32:07.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0067130 0x7f66a0135a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 10 monclient(hunting): picked mon.noname-f con 0x7f66a0067130 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:07.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a006e3b0 0x7f66a0122580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f874b7fe640 1 --2- 172.21.15.178:0/1536686792 >> v2:172.21.15.204:6812/28210 conn(0x7f87380397b0 0x7f873803bc60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f873c000c00 tx=0x7f873c009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 10 monclient(hunting): picked mon.noname-h con 0x7f66a006e3b0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:07.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a00a3550 0x7f66a0122ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669e7fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a006e3b0 0x7f66a0122580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669e7fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a006e3b0 0x7f66a0122580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:59150/0 (socket says 172.21.15.178:59150) 2024-09-15T01:32:07.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669f7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a00a3550 0x7f66a0122ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669f7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a00a3550 0x7f66a0122ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:41954/0 (socket says 172.21.15.178:41954) 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0067130 0x7f66a0135a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0067130 0x7f66a0135a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:52350/0 (socket says 172.21.15.178:52350) 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669e7fc640 1 -- 172.21.15.178:0/2296426124 learned_addr learned my addr 172.21.15.178:0/2296426124 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 10 monclient(hunting): picked mon.noname-d con 0x7f66a00a3550 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a009edc0 con 0x7f66a0067130 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66a009cef0 con 0x7f66a006e3b0 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f66a0124030 con 0x7f66a00a3550 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f66a595d640 10 monclient(hunting): authenticate will time out at 732.152735s 2024-09-15T01:32:07.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669e7fc640 10 monclient(hunting): get_auth_request con 0x7f66a006e3b0 auth_method 0 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669f7fe640 10 monclient(hunting): get_auth_request con 0x7f66a00a3550 auth_method 0 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.401+0000 7f669f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669f7fe640 10 monclient(hunting): handle_auth_done global_id 4353 payload 824 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669f7fe640 1 -- 172.21.15.178:0/2296426124 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a006e3b0 msgr2=0x7f66a0122580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669f7fe640 1 --2- 172.21.15.178:0/2296426124 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a006e3b0 0x7f66a0122580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669f7fe640 1 -- 172.21.15.178:0/2296426124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0067130 msgr2=0x7f66a0135a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669f7fe640 1 --2- 172.21.15.178:0/2296426124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0067130 0x7f66a0135a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669f7fe640 1 monclient: found mon.noname-d 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669f7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669f7fe640 1 -- 172.21.15.178:0/2296426124 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6698009a20 con 0x7f66a00a3550 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669f7fe640 1 --2- 172.21.15.178:0/2296426124 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a00a3550 0x7f66a0122ac0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f669800c1f0 tx=0x7f669800b040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669effd640 10 monclient: get_auth_request con 0x7f66a0067130 auth_method 0 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f669effd640 1 --2- 172.21.15.178:0/2296426124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0067130 0x7f66a0135a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f6686ffd640 1 -- 172.21.15.178:0/2296426124 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f669800cd60 con 0x7f66a00a3550 2024-09-15T01:32:07.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f6686ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f6686ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f6686ffd640 10 monclient: dump: 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.462 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f6686ffd640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f6686ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f6686ffd640 1 -- 172.21.15.178:0/2296426124 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669800f040 con 0x7f66a00a3550 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f66a595d640 5 monclient: authenticate success, global_id 4353 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f66a595d640 10 monclient: _renew_subs 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f66a595d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.402+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6694009450 con 0x7f66a00a3550 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.403+0000 7f6686ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.403+0000 7f6686ffd640 1 -- 172.21.15.178:0/2296426124 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6698014d80 con 0x7f66a00a3550 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.403+0000 7f6686ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.403+0000 7f6686ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:07.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.403+0000 7f6686ffd640 10 monclient: dump: 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.464 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.403+0000 7f66a595d640 10 monclient: _renew_subs 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.403+0000 7f66a595d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.403+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6690009390 con 0x7f66a00a3550 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.403+0000 7f669d7fa640 4 set_mon_vals no callback set 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.403+0000 7f6686ffd640 1 -- 172.21.15.178:0/2296426124 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6698003020 con 0x7f66a00a3550 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.404+0000 7f6686ffd640 1 --2- 172.21.15.178:0/2296426124 >> v2:172.21.15.204:6812/28210 conn(0x7f668c0396e0 0x7f668c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.404+0000 7f66a595d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.404+0000 7f6686ffd640 1 -- 172.21.15.178:0/2296426124 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f669804c560 con 0x7f66a00a3550 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.404+0000 7f669effd640 1 --2- 172.21.15.178:0/2296426124 >> v2:172.21.15.204:6812/28210 conn(0x7f668c0396e0 0x7f668c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.404+0000 7f66a595d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.404+0000 7f66a595d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.404+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a004b720 con 0x7f66a00a3550 2024-09-15T01:32:07.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.404+0000 7f669effd640 10 monclient: get_auth_request con 0x7f668c0396e0 auth_method 0 2024-09-15T01:32:07.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.405+0000 7f669effd640 1 --2- 172.21.15.178:0/2296426124 >> v2:172.21.15.204:6812/28210 conn(0x7f668c0396e0 0x7f668c03bb90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6690034000 tx=0x7f6690032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.406+0000 7f87337fe640 1 -- 172.21.15.178:0/1536686792 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8744030420 con 0x7f874c137cc0 2024-09-15T01:32:07.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.406+0000 7f87337fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.406+0000 7f87337fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:07.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.410+0000 7f6686ffd640 1 -- 172.21.15.178:0/2296426124 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6698018020 con 0x7f66a00a3550 2024-09-15T01:32:07.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.410+0000 7f6686ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.410+0000 7f6686ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:07.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.428+0000 7f478d15d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:07.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.428+0000 7f478d15d640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.428+0000 7f478d15d640 10 monclient: monmap: 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:07.430291+0000 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:07.430291+0000 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.468 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.469 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.469 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:07.469 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:07.469 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.429+0000 7fa49028b640 10 monclient: get_monmap_and_config 2024-09-15T01:32:07.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.429+0000 7fa49028b640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.429+0000 7f478d15d640 1 Processor -- start 2024-09-15T01:32:07.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.429+0000 7fa49028b640 10 monclient: monmap: 2024-09-15T01:32:07.470 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.470 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.470 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:07.431014+0000 2024-09-15T01:32:07.470 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:07.431014+0000 2024-09-15T01:32:07.470 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.470 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.471 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:07.471 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:07.471 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.471 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.471 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.471 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.471 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.471 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.471 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.471 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:07.471 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:07.471 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.429+0000 7f478d15d640 1 -- start start 2024-09-15T01:32:07.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.429+0000 7f478d15d640 10 monclient: init 2024-09-15T01:32:07.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.429+0000 7f478d15d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.429+0000 7f478d15d640 10 monclient: _add_conns ranks=[0,10,6,3,9,2,7,4,1,8,5] 2024-09-15T01:32:07.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.429+0000 7f478d15d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f47880a25a0 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:07.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.429+0000 7fa49028b640 1 Processor -- start 2024-09-15T01:32:07.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478d15d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4788098900 0x7f478809bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478d15d640 10 monclient(hunting): picked mon.noname-f con 0x7f4788098900 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:07.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478d15d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f47880993e0 0x7f478813e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478d15d640 10 monclient(hunting): picked mon.noname-d con 0x7f47880993e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:07.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478d15d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478804bf60 con 0x7f47880a25a0 2024-09-15T01:32:07.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478d15d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478809e2f0 con 0x7f4788098900 2024-09-15T01:32:07.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478d15d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4788097950 con 0x7f47880993e0 2024-09-15T01:32:07.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478d15d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478d15d640 10 monclient(hunting): authenticate will time out at 732.181131s 2024-09-15T01:32:07.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478759e640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f47880993e0 0x7f478813e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478759e640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f47880993e0 0x7f478813e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:41958/0 (socket says 172.21.15.178:41958) 2024-09-15T01:32:07.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7fa49028b640 1 -- start start 2024-09-15T01:32:07.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7fa49028b640 10 monclient: init 2024-09-15T01:32:07.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478759e640 1 -- 172.21.15.178:0/2747571390 learned_addr learned my addr 172.21.15.178:0/2747571390 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478759e640 10 monclient(hunting): get_auth_request con 0x7f47880993e0 auth_method 0 2024-09-15T01:32:07.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 1 --2- 172.21.15.178:0/2747571390 >> v2:172.21.15.204:3300/0 conn(0x7f4788098900 0x7f478809bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 10 monclient(hunting): get_auth_request con 0x7f4788098900 auth_method 0 2024-09-15T01:32:07.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 10 monclient(hunting): handle_auth_done global_id 4294 payload 274 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 1 -- 172.21.15.178:0/2747571390 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f47880993e0 msgr2=0x7f478813e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 1 --2- 172.21.15.178:0/2747571390 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f47880993e0 0x7f478813e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 1 -- 172.21.15.178:0/2747571390 >> v1:172.21.15.178:6789/0 conn(0x7f47880a25a0 legacy=0x7f478809b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 1 monclient: found mon.noname-f 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.430+0000 7f478659c640 1 -- 172.21.15.178:0/2747571390 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47880a2970 con 0x7f4788098900 2024-09-15T01:32:07.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 10 monclient: _add_conns ranks=[8,0,4,7,6,10,2,5,3,1,9] 2024-09-15T01:32:07.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa48809fa10 0x7fa4880a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 10 monclient(hunting): picked mon.noname-e con 0x7fa48809fa10 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:07.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa48809d950 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa48809fde0 0x7fa48813e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 10 monclient(hunting): picked mon.noname-c con 0x7fa48809fde0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48804bf60 con 0x7fa48809d950 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa48809ec50 con 0x7fa48809fde0 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa48809f280 con 0x7fa48809fa10 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa49028b640 10 monclient(hunting): authenticate will time out at 732.182345s 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48d826640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa48809d950 0x7fa4880a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:59182/0 (socket says 172.21.15.178:59182) 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48d826640 1 -- 172.21.15.178:0/180290009 learned_addr learned my addr 172.21.15.178:0/180290009 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48e828640 1 --2- 172.21.15.178:0/180290009 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa48809fde0 0x7fa48813e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48e828640 10 monclient(hunting): get_auth_request con 0x7fa48809fde0 auth_method 0 2024-09-15T01:32:07.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48e828640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7f478659c640 1 --2- 172.21.15.178:0/2747571390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4788098900 0x7f478809bfb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4778009850 tx=0x7f47780090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1804f1f1793cde99 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48e828640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48e828640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48e828640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48e828640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48e828640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48e027640 1 --2- 172.21.15.178:0/180290009 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa48809fa10 0x7fa4880a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48d025640 1 -- 172.21.15.178:0/180290009 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017393037 0 0) 0x7fa48804bf60 con 0x7fa48809d950 2024-09-15T01:32:07.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48d025640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48d025640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48d025640 10 monclient(hunting): my global_id is 4311 2024-09-15T01:32:07.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.431+0000 7fa48d025640 1 -- 172.21.15.178:0/180290009 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4740034e0 con 0x7fa48809d950 2024-09-15T01:32:07.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e027640 10 monclient(hunting): get_auth_request con 0x7fa48809fa10 auth_method 0 2024-09-15T01:32:07.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e027640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e027640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e027640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e828640 10 monclient(hunting): handle_auth_done global_id 4270 payload 274 2024-09-15T01:32:07.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e828640 1 -- 172.21.15.178:0/180290009 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa48809fa10 msgr2=0x7fa4880a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e828640 1 --2- 172.21.15.178:0/180290009 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa48809fa10 0x7fa4880a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e828640 1 -- 172.21.15.178:0/180290009 >> v1:172.21.15.178:6789/0 conn(0x7fa48809d950 legacy=0x7fa4880a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48d025640 1 -- 172.21.15.178:0/180290009 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2398563911 0 0) 0x7fa4740034e0 con 0x7fa48809d950 2024-09-15T01:32:07.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e828640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e828640 1 monclient: found mon.noname-c 2024-09-15T01:32:07.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e828640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e828640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:07.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e828640 1 -- 172.21.15.178:0/180290009 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48806a340 con 0x7fa48809fde0 2024-09-15T01:32:07.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48e027640 1 --2- 172.21.15.178:0/180290009 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa48809fa10 0x7fa4880a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:07.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7fa48d025640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:32:07.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f4785d9b640 1 -- 172.21.15.178:0/2747571390 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f477802ac80 con 0x7f4788098900 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f4785d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f4785d9b640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f4785d9b640 10 monclient: dump: 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.486 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f4785d9b640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f4785d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f4785d9b640 1 -- 172.21.15.178:0/2747571390 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477802ae00 con 0x7f4788098900 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f4785d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f4785d9b640 1 -- 172.21.15.178:0/2747571390 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4778031ca0 con 0x7f4788098900 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f4785d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f4785d9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f4785d9b640 10 monclient: dump: 2024-09-15T01:32:07.487 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.488 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f478d15d640 5 monclient: authenticate success, global_id 4294 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f478d15d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.432+0000 7f478d15d640 4 set_mon_vals no callback set 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f478d15d640 10 monclient: shutdown 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f478d15d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f478d15d640 1 -- 172.21.15.178:0/2747571390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4788098900 msgr2=0x7f478809bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f478d15d640 1 --2- 172.21.15.178:0/2747571390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4788098900 0x7f478809bfb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4778009850 tx=0x7f47780090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f478d15d640 1 -- 172.21.15.178:0/2747571390 shutdown_connections 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f478d15d640 1 --2- 172.21.15.178:0/2747571390 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f47880993e0 0x7f478813e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f478d15d640 1 --2- 172.21.15.178:0/2747571390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4788098900 0x7f478809bfb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f478d15d640 1 -- 172.21.15.178:0/2747571390 >> 172.21.15.178:0/2747571390 conn(0x7f4788067710 msgr2=0x7f47880a1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f478d15d640 1 -- 172.21.15.178:0/2747571390 shutdown_connections 2024-09-15T01:32:07.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa48e828640 1 --2- 172.21.15.178:0/180290009 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa48809fde0 0x7fa48813e2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa480009e90 tx=0x7fa480009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=7589c47ff9620dcf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f478d15d640 1 -- 172.21.15.178:0/2747571390 wait complete. 2024-09-15T01:32:07.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa48d025640 1 -- 172.21.15.178:0/180290009 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4840018f0 con 0x7fa48809fde0 2024-09-15T01:32:07.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f04d0f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f04d0f9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7f04d0f9d640 10 monclient: monmap: 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:07.435493+0000 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:07.435493+0000 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.491 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.492 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.492 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.492 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.492 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.492 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:07.492 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:07.492 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa48d025640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa48d025640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:07.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa48d025640 10 monclient: dump: 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.493 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.494 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.494 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa48d025640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa48d025640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa49028b640 5 monclient: authenticate success, global_id 4270 2024-09-15T01:32:07.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa49028b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:07.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa48d025640 1 -- 172.21.15.178:0/180290009 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48002d050 con 0x7fa48809fde0 2024-09-15T01:32:07.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa48d025640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa49028b640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:07.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.433+0000 7fa49028b640 4 set_mon_vals no callback set 2024-09-15T01:32:07.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7f478d15d640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7f478d15d640 10 monclient: monmap: 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.495 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.496 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.496 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7f04d0f9d640 1 Processor -- start 2024-09-15T01:32:07.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7f04d0f9d640 1 -- start start 2024-09-15T01:32:07.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7f04d0f9d640 10 monclient: init 2024-09-15T01:32:07.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7f478d15d640 1 Processor -- start 2024-09-15T01:32:07.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7fa49028b640 10 monclient: shutdown 2024-09-15T01:32:07.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7fa49028b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7fa49028b640 1 -- 172.21.15.178:0/180290009 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa48809fde0 msgr2=0x7fa48813e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7fa49028b640 1 --2- 172.21.15.178:0/180290009 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa48809fde0 0x7fa48813e2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa480009e90 tx=0x7fa480009780 comp rx=0 tx=0).stop 2024-09-15T01:32:07.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7fa48d025640 1 -- 172.21.15.178:0/180290009 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4800079d0 con 0x7fa48809fde0 2024-09-15T01:32:07.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7fa48d025640 10 monclient: discarding stray monitor message mon_map magic: 0 v1 2024-09-15T01:32:07.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7f478d15d640 1 -- start start 2024-09-15T01:32:07.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7f478d15d640 10 monclient: init 2024-09-15T01:32:07.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7fa49028b640 1 -- 172.21.15.178:0/180290009 shutdown_connections 2024-09-15T01:32:07.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7fa49028b640 1 --2- 172.21.15.178:0/180290009 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa48809fde0 0x7fa48813e2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7fa49028b640 1 --2- 172.21.15.178:0/180290009 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa48809fa10 0x7fa4880a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.434+0000 7fa49028b640 1 -- 172.21.15.178:0/180290009 >> 172.21.15.178:0/180290009 conn(0x7fa4880a5ed0 msgr2=0x7fa4880a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7fa49028b640 1 -- 172.21.15.178:0/180290009 shutdown_connections 2024-09-15T01:32:07.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7fa49028b640 1 -- 172.21.15.178:0/180290009 wait complete. 2024-09-15T01:32:07.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7fd57ed54640 10 monclient: get_monmap_and_config 2024-09-15T01:32:07.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7fd57ed54640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7fd57ed54640 10 monclient: monmap: 2024-09-15T01:32:07.499 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.499 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.499 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:07.437120+0000 2024-09-15T01:32:07.499 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:07.437120+0000 2024-09-15T01:32:07.499 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.499 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.500 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:07.500 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:07.500 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.500 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.500 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.500 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.500 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.500 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.500 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.500 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:07.500 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:07.500 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f04d0f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f04d0f9d640 10 monclient: _add_conns ranks=[2,6,8,7,3,5,9,10,4,1,0] 2024-09-15T01:32:07.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f04d0f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04cc0a2630 0x7f04cc12edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f04d0f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f04cc0a2630 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:07.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f04d0f9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f04cc12f310 0x7f04cc09c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f04d0f9d640 10 monclient(hunting): picked mon.noname-d con 0x7f04cc12f310 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:07.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f04ca59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04cc0a2630 0x7f04cc12edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f04d0f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f04cc09cdc0 0x7f04cc09d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f04ca59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04cc0a2630 0x7f04cc12edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:49392/0 (socket says 172.21.15.178:49392) 2024-09-15T01:32:07.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f04c9d9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f04cc12f310 0x7f04cc09c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7fa49028b640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7fa49028b640 10 monclient: monmap: 2024-09-15T01:32:07.502 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.502 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:07.502 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:07.502 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:07.502 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.502 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:07.502 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:07.502 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:07.502 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.503 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.503 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.503 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.503 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.503 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.503 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.503 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 10 monclient: _add_conns ranks=[6,3,0,4,5,1,7,2,8] 2024-09-15T01:32:07.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4788098900 0x7f47881461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 10 monclient(hunting): picked mon.noname-d con 0x7f4788098900 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:07.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47880993e0 0x7f478813a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 10 monclient(hunting): picked mon.noname-g con 0x7f47880993e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:07.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47880a25a0 0x7f4788123c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 10 monclient(hunting): picked mon.noname-a con 0x7f47880a25a0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:07.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4788096a30 con 0x7f47880a25a0 2024-09-15T01:32:07.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478806ee90 con 0x7f47880993e0 2024-09-15T01:32:07.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4788146710 con 0x7f4788098900 2024-09-15T01:32:07.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478d15d640 10 monclient(hunting): authenticate will time out at 732.186530s 2024-09-15T01:32:07.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478759e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47880a25a0 0x7f4788123c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f04c9d9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f04cc12f310 0x7f04cc09c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:41960/0 (socket says 172.21.15.178:41960) 2024-09-15T01:32:07.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478759e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47880a25a0 0x7f4788123c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33276/0 (socket says 172.21.15.178:33276) 2024-09-15T01:32:07.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478759e640 1 -- 172.21.15.178:0/130406943 learned_addr learned my addr 172.21.15.178:0/130406943 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478759e640 10 monclient(hunting): get_auth_request con 0x7f47880a25a0 auth_method 0 2024-09-15T01:32:07.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.435+0000 7f478759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f4786d9d640 1 --2- 172.21.15.178:0/130406943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4788098900 0x7f47881461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04cad9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f04cc09cdc0 0x7f04cc09d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04cad9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f04cc09cdc0 0x7f04cc09d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:51950/0 (socket says 172.21.15.178:51950) 2024-09-15T01:32:07.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04d0f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f04cc09cdc0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:07.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04d0f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04d0f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc09d730 con 0x7f04cc0a2630 2024-09-15T01:32:07.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04d0f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04d0f9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc09e650 con 0x7f04cc12f310 2024-09-15T01:32:07.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04d0f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04d0f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc097d70 con 0x7f04cc09cdc0 2024-09-15T01:32:07.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04d0f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04d0f9d640 10 monclient(hunting): authenticate will time out at 732.187439s 2024-09-15T01:32:07.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f4786d9d640 10 monclient(hunting): get_auth_request con 0x7f4788098900 auth_method 0 2024-09-15T01:32:07.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f4786d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f4786d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f4786d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f4786d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f4786d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478659c640 1 --2- 172.21.15.178:0/130406943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47880993e0 0x7f478813a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f4786d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478659c640 10 monclient(hunting): get_auth_request con 0x7f47880993e0 auth_method 0 2024-09-15T01:32:07.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7fa49028b640 1 Processor -- start 2024-09-15T01:32:07.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7fa49028b640 1 -- start start 2024-09-15T01:32:07.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7fa49028b640 10 monclient: init 2024-09-15T01:32:07.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 10 monclient(hunting): handle_auth_done global_id 4320 payload 824 2024-09-15T01:32:07.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 1 -- 172.21.15.178:0/130406943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4788098900 msgr2=0x7f47881461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7fd57ed54640 1 Processor -- start 2024-09-15T01:32:07.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7fd57ed54640 1 -- start start 2024-09-15T01:32:07.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7fd57ed54640 10 monclient: init 2024-09-15T01:32:07.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 1 --2- 172.21.15.178:0/130406943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4788098900 0x7f47881461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 1 -- 172.21.15.178:0/130406943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47880993e0 msgr2=0x7f478813a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 1 --2- 172.21.15.178:0/130406943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47880993e0 0x7f478813a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 1 monclient: found mon.noname-a 2024-09-15T01:32:07.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:07.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04ca59c640 1 -- 172.21.15.178:0/2650279063 learned_addr learned my addr 172.21.15.178:0/2650279063 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04ca59c640 10 monclient(hunting): get_auth_request con 0x7f04cc0a2630 auth_method 0 2024-09-15T01:32:07.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04ca59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 1 -- 172.21.15.178:0/130406943 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778009390 con 0x7f47880a25a0 2024-09-15T01:32:07.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04c9d9b640 10 monclient(hunting): get_auth_request con 0x7f04cc12f310 auth_method 0 2024-09-15T01:32:07.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04c9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478659c640 1 --2- 172.21.15.178:0/130406943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47880993e0 0x7f478813a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:07.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04c9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04c9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04cad9d640 10 monclient(hunting): get_auth_request con 0x7f04cc09cdc0 auth_method 0 2024-09-15T01:32:07.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04cad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04c9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f4786d9d640 1 --2- 172.21.15.178:0/130406943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4788098900 0x7f47881461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04c9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04c9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f478759e640 1 --2- 172.21.15.178:0/130406943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47880a25a0 0x7f4788123c10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f478000c1c0 tx=0x7f478000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7f04ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f04ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f04c9d9b640 10 monclient(hunting): handle_auth_done global_id 4380 payload 274 2024-09-15T01:32:07.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f04c9d9b640 1 -- 172.21.15.178:0/2650279063 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f04cc09cdc0 msgr2=0x7f04cc09d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.436+0000 7fd57ed54640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 10 monclient: _add_conns ranks=[3,0,9,6,8,5,10,1,2,4,7] 2024-09-15T01:32:07.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd57812f5d0 0x7fd57812f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 10 monclient(hunting): picked mon.noname-g con 0x7fd57812f5d0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:07.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd57806e510 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:07.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd57806e8e0 0x7fd57809dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 10 monclient(hunting): picked mon.noname-a con 0x7fd57806e8e0 addr v2:172.21.15.178:3300/0 2024-09-15T01:32:07.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57804bf60 con 0x7fd57806e510 2024-09-15T01:32:07.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57809f2c0 con 0x7fd57806e8e0 2024-09-15T01:32:07.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57806b850 con 0x7fd57812f5d0 2024-09-15T01:32:07.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57ed54640 10 monclient(hunting): authenticate will time out at 732.188145s 2024-09-15T01:32:07.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f04c9d9b640 1 --2- 172.21.15.178:0/2650279063 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f04cc09cdc0 0x7f04cc09d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f04c9d9b640 1 -- 172.21.15.178:0/2650279063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04cc0a2630 msgr2=0x7f04cc12edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f04c9d9b640 1 --2- 172.21.15.178:0/2650279063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04cc0a2630 0x7f04cc12edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57caf0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd57812f5d0 0x7fd57812f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd57806e8e0 0x7fd57809dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57caf0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd57812f5d0 0x7fd57812f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:55400/0 (socket says 172.21.15.178:55400) 2024-09-15T01:32:07.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57caf0640 1 -- 172.21.15.178:0/2217180251 learned_addr learned my addr 172.21.15.178:0/2217180251 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57caf0640 10 monclient(hunting): get_auth_request con 0x7fd57812f5d0 auth_method 0 2024-09-15T01:32:07.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57caf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57caf0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57caf0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 10 monclient: _add_conns ranks=[5,1,7,3,6,8,4,2,0] 2024-09-15T01:32:07.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa48809d950 0x7fa488146780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 10 monclient(hunting): picked mon.noname-h con 0x7fa48809d950 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:07.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa48809fa10 0x7fa488146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 10 monclient(hunting): picked mon.noname-f con 0x7fa48809fa10 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:07.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa488126f50 0x7fa48809a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 10 monclient(hunting): picked mon.noname-i con 0x7fa488126f50 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:07.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48809bfe0 con 0x7fa48809fa10 2024-09-15T01:32:07.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa488138080 con 0x7fa48809d950 2024-09-15T01:32:07.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa488125cc0 con 0x7fa488126f50 2024-09-15T01:32:07.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa49028b640 10 monclient(hunting): authenticate will time out at 732.188118s 2024-09-15T01:32:07.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f04c9d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f04c9d9b640 1 monclient: found mon.noname-d 2024-09-15T01:32:07.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f04c9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 10 monclient(hunting): get_auth_request con 0x7fd57806e8e0 auth_method 0 2024-09-15T01:32:07.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f04c9d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f476effd640 1 -- 172.21.15.178:0/130406943 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f478000cd30 con 0x7f47880a25a0 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f476effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f476effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f476effd640 10 monclient: dump: 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.522 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.522 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.522 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.522 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.522 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f476effd640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f476effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f04c9d9b640 1 -- 172.21.15.178:0/2650279063 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc04bf60 con 0x7f04cc12f310 2024-09-15T01:32:07.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f04cad9d640 1 --2- 172.21.15.178:0/2650279063 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f04cc09cdc0 0x7f04cc09d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57caf0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57caf0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57caf0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 10 monclient(hunting): handle_auth_done global_id 4329 payload 274 2024-09-15T01:32:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 1 -- 172.21.15.178:0/2217180251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd57812f5d0 msgr2=0x7fd57812f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 1 --2- 172.21.15.178:0/2217180251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd57812f5d0 0x7fd57812f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 1 -- 172.21.15.178:0/2217180251 >> v1:172.21.15.178:6789/0 conn(0x7fd57806e510 legacy=0x7fd57809d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 1 monclient: found mon.noname-a 2024-09-15T01:32:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fd57d2f1640 1 -- 172.21.15.178:0/2217180251 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57806c740 con 0x7fd57806e8e0 2024-09-15T01:32:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa48809fa10 0x7fa488146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa48809fa10 0x7fa488146cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:52358/0 (socket says 172.21.15.178:52358) 2024-09-15T01:32:07.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 1 -- 172.21.15.178:0/10545450 learned_addr learned my addr 172.21.15.178:0/10545450 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 10 monclient(hunting): get_auth_request con 0x7fa48809fa10 auth_method 0 2024-09-15T01:32:07.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48e027640 1 --2- 172.21.15.178:0/10545450 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa48809d950 0x7fa488146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 10 monclient(hunting): handle_auth_done global_id 4303 payload 824 2024-09-15T01:32:07.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 1 -- 172.21.15.178:0/10545450 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa488126f50 msgr2=0x7fa48809a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 1 --2- 172.21.15.178:0/10545450 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa488126f50 0x7fa48809a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 1 -- 172.21.15.178:0/10545450 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa48809d950 msgr2=0x7fa488146780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 1 --2- 172.21.15.178:0/10545450 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa48809d950 0x7fa488146780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 1 monclient: found mon.noname-f 2024-09-15T01:32:07.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7fa48d826640 1 -- 172.21.15.178:0/10545450 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa480009a20 con 0x7fa48809fa10 2024-09-15T01:32:07.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f476effd640 1 -- 172.21.15.178:0/130406943 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478000f040 con 0x7f47880a25a0 2024-09-15T01:32:07.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f476effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f476effd640 1 -- 172.21.15.178:0/130406943 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f478000ce90 con 0x7f47880a25a0 2024-09-15T01:32:07.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f476effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f476effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:07.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f478559a640 4 set_mon_vals no callback set 2024-09-15T01:32:07.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.437+0000 7f476effd640 10 monclient: dump: 2024-09-15T01:32:07.527 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.527 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.527 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.527 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.527 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.528 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.528 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.528 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.528 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.528 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.528 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.528 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.528 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.528 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.528 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.528 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa48d826640 1 --2- 172.21.15.178:0/10545450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa48809fa10 0x7fa488146cc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa48400c810 tx=0x7fa484001430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa47e7fc640 1 -- 172.21.15.178:0/10545450 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4840075d0 con 0x7fa48809fa10 2024-09-15T01:32:07.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa47e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa47e7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:32:07.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa47e7fc640 10 monclient: dump: 2024-09-15T01:32:07.529 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.529 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.529 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.529 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.529 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.530 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.530 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.530 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.530 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.530 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.530 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.530 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.530 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.530 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.530 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.531 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa47e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa47e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa49028b640 5 monclient: authenticate success, global_id 4303 2024-09-15T01:32:07.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa49028b640 10 monclient: _renew_subs 2024-09-15T01:32:07.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa49028b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa49028b640 1 -- 172.21.15.178:0/10545450 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa488125fa0 con 0x7fa48809fa10 2024-09-15T01:32:07.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa47e7fc640 1 -- 172.21.15.178:0/10545450 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4840047a0 con 0x7fa48809fa10 2024-09-15T01:32:07.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa49028b640 10 monclient: _renew_subs 2024-09-15T01:32:07.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa49028b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa49028b640 1 -- 172.21.15.178:0/10545450 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa488124d80 con 0x7fa48809fa10 2024-09-15T01:32:07.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7f04c9d9b640 1 --2- 172.21.15.178:0/2650279063 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f04cc12f310 0x7f04cc09c880 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009880 tx=0x7f04c00090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=a7932fb841f9ac44 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04c959a640 1 -- 172.21.15.178:0/2650279063 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f04c002abb0 con 0x7f04cc12f310 2024-09-15T01:32:07.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04c959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04c959a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:32:07.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04c959a640 10 monclient: dump: 2024-09-15T01:32:07.532 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.532 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.532 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.532 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.532 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.532 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.533 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.533 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.533 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.533 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.533 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.533 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.533 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.533 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.533 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.533 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04c959a640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04c959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04d0f9d640 5 monclient: authenticate success, global_id 4380 2024-09-15T01:32:07.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04d0f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:07.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04c959a640 1 -- 172.21.15.178:0/2650279063 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c002ad10 con 0x7f04cc12f310 2024-09-15T01:32:07.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04c959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04d0f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:07.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04d0f9d640 4 set_mon_vals no callback set 2024-09-15T01:32:07.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fd57d2f1640 1 --2- 172.21.15.178:0/2217180251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57806e8e0 0x7fd57809dc90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd568009850 tx=0x7fd5680090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc59a1296f643499 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd5777fe640 1 -- 172.21.15.178:0/2217180251 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd56802ac70 con 0x7fd57806e8e0 2024-09-15T01:32:07.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd5777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd5777fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:07.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd5777fe640 10 monclient: dump: 2024-09-15T01:32:07.535 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.535 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.535 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.535 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.535 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.535 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.535 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.535 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.535 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.535 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd5777fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd5777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd5777fe640 1 -- 172.21.15.178:0/2217180251 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56802adf0 con 0x7fd57806e8e0 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd5777fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd5777fe640 1 -- 172.21.15.178:0/2217180251 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd568031ca0 con 0x7fd57806e8e0 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd5777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd5777fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:07.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd5777fe640 10 monclient: dump: 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.537 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.538 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.538 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.538 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa47e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7f478d15d640 5 monclient: authenticate success, global_id 4320 2024-09-15T01:32:07.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7f478d15d640 10 monclient: _renew_subs 2024-09-15T01:32:07.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7f478d15d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:07.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7f478d15d640 1 -- 172.21.15.178:0/130406943 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4788127150 con 0x7f47880a25a0 2024-09-15T01:32:07.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7f478d15d640 10 monclient: _renew_subs 2024-09-15T01:32:07.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7f478d15d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:07.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7f478d15d640 1 -- 172.21.15.178:0/130406943 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4788146fa0 con 0x7f47880a25a0 2024-09-15T01:32:07.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.438+0000 7fa48c824640 4 set_mon_vals no callback set 2024-09-15T01:32:07.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd57ed54640 5 monclient: authenticate success, global_id 4329 2024-09-15T01:32:07.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd57ed54640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:07.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fd57ed54640 4 set_mon_vals no callback set 2024-09-15T01:32:07.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fa47e7fc640 1 -- 172.21.15.178:0/10545450 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa484004d90 con 0x7fa48809fa10 2024-09-15T01:32:07.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fa47e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fa49028b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fa47e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:32:07.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04c959a640 1 -- 172.21.15.178:0/2650279063 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f04c002abb0 con 0x7f04cc12f310 2024-09-15T01:32:07.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f04c959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fa47e7fc640 10 monclient: dump: 2024-09-15T01:32:07.541 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.541 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.541 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.541 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.542 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.542 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.542 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.542 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.542 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.542 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.542 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.542 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.542 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.542 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.542 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.543 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7fa47e7fc640 1 -- 172.21.15.178:0/10545450 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa484007150 con 0x7fa48809fa10 2024-09-15T01:32:07.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f04c959a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:07.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fa49028b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fa49028b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f04c959a640 10 monclient: dump: 2024-09-15T01:32:07.544 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.544 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.544 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.544 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.544 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.544 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.544 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.544 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.544 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.544 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.544 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.545 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.545 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.545 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.545 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.545 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f476effd640 1 -- 172.21.15.178:0/130406943 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4780009110 con 0x7f47880a25a0 2024-09-15T01:32:07.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f476effd640 1 --2- 172.21.15.178:0/130406943 >> v2:172.21.15.204:6812/28210 conn(0x7f47740396e0 0x7f477403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f476effd640 1 -- 172.21.15.178:0/130406943 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f478004c290 con 0x7f47880a25a0 2024-09-15T01:32:07.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fd57ed54640 10 monclient: shutdown 2024-09-15T01:32:07.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fd57ed54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fd57ed54640 1 -- 172.21.15.178:0/2217180251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57806e8e0 msgr2=0x7fd57809dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fd57ed54640 1 --2- 172.21.15.178:0/2217180251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57806e8e0 0x7fd57809dc90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd568009850 tx=0x7fd5680090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f04d0f9d640 10 monclient: shutdown 2024-09-15T01:32:07.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.439+0000 7f478d15d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f478d15d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f04d0f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f04d0f9d640 1 -- 172.21.15.178:0/2650279063 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f04cc12f310 msgr2=0x7f04cc09c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fa49028b640 1 -- 172.21.15.178:0/10545450 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa470005010 con 0x7fa48809fa10 2024-09-15T01:32:07.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f478d15d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:07.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f04d0f9d640 1 --2- 172.21.15.178:0/2650279063 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f04cc12f310 0x7f04cc09c880 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009880 tx=0x7f04c00090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f478d15d640 1 -- 172.21.15.178:0/130406943 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478804b720 con 0x7f47880a25a0 2024-09-15T01:32:07.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f4786d9d640 1 --2- 172.21.15.178:0/130406943 >> v2:172.21.15.204:6812/28210 conn(0x7f47740396e0 0x7f477403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f4786d9d640 10 monclient: get_auth_request con 0x7f47740396e0 auth_method 0 2024-09-15T01:32:07.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f04d0f9d640 1 -- 172.21.15.178:0/2650279063 shutdown_connections 2024-09-15T01:32:07.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fa47e7fc640 1 --2- 172.21.15.178:0/10545450 >> v2:172.21.15.204:6812/28210 conn(0x7fa46c0397b0 0x7fa46c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fa47e7fc640 1 -- 172.21.15.178:0/10545450 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7fa48404a840 con 0x7fa48809fa10 2024-09-15T01:32:07.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f04d0f9d640 1 --2- 172.21.15.178:0/2650279063 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f04cc12f310 0x7f04cc09c880 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f04d0f9d640 1 --2- 172.21.15.178:0/2650279063 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f04cc09cdc0 0x7f04cc09d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f04d0f9d640 1 --2- 172.21.15.178:0/2650279063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04cc0a2630 0x7f04cc12edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7f04d0f9d640 1 -- 172.21.15.178:0/2650279063 >> 172.21.15.178:0/2650279063 conn(0x7f04cc067710 msgr2=0x7f04cc0a1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fd57ed54640 1 -- 172.21.15.178:0/2217180251 shutdown_connections 2024-09-15T01:32:07.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.441+0000 7fa48e027640 1 --2- 172.21.15.178:0/10545450 >> v2:172.21.15.204:6812/28210 conn(0x7fa46c0397b0 0x7fa46c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.441+0000 7fa48e027640 10 monclient: get_auth_request con 0x7fa46c0397b0 auth_method 0 2024-09-15T01:32:07.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.441+0000 7f04d0f9d640 1 -- 172.21.15.178:0/2650279063 shutdown_connections 2024-09-15T01:32:07.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fd57ed54640 1 --2- 172.21.15.178:0/2217180251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57806e8e0 0x7fd57809dc90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.441+0000 7f04d0f9d640 1 -- 172.21.15.178:0/2650279063 wait complete. 2024-09-15T01:32:07.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fd57ed54640 1 --2- 172.21.15.178:0/2217180251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd57812f5d0 0x7fd57812f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.440+0000 7fd57ed54640 1 -- 172.21.15.178:0/2217180251 >> 172.21.15.178:0/2217180251 conn(0x7fd578068a60 msgr2=0x7fd5780a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.441+0000 7f4786d9d640 1 --2- 172.21.15.178:0/130406943 >> v2:172.21.15.204:6812/28210 conn(0x7f47740396e0 0x7f477403bb90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f477c007fd0 tx=0x7f477c009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.441+0000 7fd57ed54640 1 -- 172.21.15.178:0/2217180251 shutdown_connections 2024-09-15T01:32:07.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.441+0000 7fd57ed54640 1 -- 172.21.15.178:0/2217180251 wait complete. 2024-09-15T01:32:07.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.441+0000 7fa48e027640 1 --2- 172.21.15.178:0/10545450 >> v2:172.21.15.204:6812/28210 conn(0x7fa46c0397b0 0x7fa46c03bc60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa478006e70 tx=0x7fa478008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.441+0000 7fd57ed54640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.441+0000 7fd57ed54640 10 monclient: monmap: 2024-09-15T01:32:07.554 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.554 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:07.554 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:07.554 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:07.554 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.554 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:07.554 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:07.555 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:07.555 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.555 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.555 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.555 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.555 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.555 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.555 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.555 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.441+0000 7f04d0f9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7f04d0f9d640 10 monclient: monmap: 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:07.556 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:07.557 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:07.557 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 1 Processor -- start 2024-09-15T01:32:07.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 1 -- start start 2024-09-15T01:32:07.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 10 monclient: init 2024-09-15T01:32:07.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7f04d0f9d640 1 Processor -- start 2024-09-15T01:32:07.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 10 monclient: _add_conns ranks=[2,0,6,3,4,5,7,8,1] 2024-09-15T01:32:07.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7f04d0f9d640 1 -- start start 2024-09-15T01:32:07.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7f04d0f9d640 10 monclient: init 2024-09-15T01:32:07.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd57806e510 0x7fd57812e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 10 monclient(hunting): picked mon.noname-b con 0x7fd57806e510 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:07.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57806e8e0 0x7fd57812eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 10 monclient(hunting): picked mon.noname-a con 0x7fd57806e8e0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:07.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd57812f5d0 0x7fd5781314e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 10 monclient(hunting): picked mon.noname-d con 0x7fd57812f5d0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:07.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57809ccc0 con 0x7fd57806e8e0 2024-09-15T01:32:07.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7f04d0f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:07.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7f04d0f9d640 10 monclient: _add_conns ranks=[3,1,7,2,0,6,8,5,4] 2024-09-15T01:32:07.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd578135110 con 0x7fd57806e510 2024-09-15T01:32:07.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7f04d0f9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc09cdc0 0x7f04cc13da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7f04d0f9d640 10 monclient(hunting): picked mon.noname-g con 0x7f04cc09cdc0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:07.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd578134e30 con 0x7fd57812f5d0 2024-09-15T01:32:07.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57ed54640 10 monclient(hunting): authenticate will time out at 732.193819s 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd577fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57806e8e0 0x7fd57812eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd57caf0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd57806e510 0x7fd57812e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd577fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57806e8e0 0x7fd57812eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33292/0 (socket says 172.21.15.178:33292) 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd577fff640 1 -- 172.21.15.178:0/2090546385 learned_addr learned my addr 172.21.15.178:0/2090546385 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd577fff640 10 monclient(hunting): get_auth_request con 0x7fd57806e8e0 auth_method 0 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd577fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd577fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.442+0000 7fd577fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 10 monclient(hunting): get_auth_request con 0x7fd57806e510 auth_method 0 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd577fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd577fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd577fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 10 monclient(hunting): handle_auth_done global_id 4331 payload 824 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 1 -- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd57812f5d0 msgr2=0x7fd5781314e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57d2f1640 1 --2- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd57812f5d0 0x7fd5781314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 1 --2- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd57812f5d0 0x7fd5781314e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 1 -- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57806e8e0 msgr2=0x7fd57812eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 1 --2- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57806e8e0 0x7fd57812eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 1 monclient: found mon.noname-b 2024-09-15T01:32:07.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:07.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 1 -- 172.21.15.178:0/2090546385 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd568009390 con 0x7fd57806e510 2024-09-15T01:32:07.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd577fff640 1 --2- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57806e8e0 0x7fd57812eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:07.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57d2f1640 10 monclient: get_auth_request con 0x7fd57812f5d0 auth_method 0 2024-09-15T01:32:07.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57d2f1640 1 --2- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd57812f5d0 0x7fd5781314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:32:07.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7fd57caf0640 1 --2- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd57806e510 0x7fd57812e570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd57000ed60 tx=0x7fd57000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7f04d0f9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0a2630 0x7f04cc12da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.443+0000 7f04d0f9d640 10 monclient(hunting): picked mon.noname-f con 0x7f04cc0a2630 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:07.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc09cdc0 0x7f04cc13da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc09cdc0 0x7f04cc13da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:55406/0 (socket says 172.21.15.178:55406) 2024-09-15T01:32:07.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 1 -- 172.21.15.178:0/606544895 learned_addr learned my addr 172.21.15.178:0/606544895 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:07.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04d0f9d640 1 --2- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f04cc12f310 0x7f04cc11a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04c9d9b640 1 --2- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0a2630 0x7f04cc12da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd574ff9640 1 -- 172.21.15.178:0/2090546385 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5700092e0 con 0x7fd57806e510 2024-09-15T01:32:07.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04d0f9d640 10 monclient(hunting): picked mon.noname-i con 0x7f04cc12f310 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:07.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd574ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04d0f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04d0f9d640 1 -- 172.21.15.178:0/606544895 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc11db60 con 0x7f04cc0a2630 2024-09-15T01:32:07.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04d0f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd574ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:07.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04d0f9d640 1 -- 172.21.15.178:0/606544895 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc06ec60 con 0x7f04cc09cdc0 2024-09-15T01:32:07.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04cad9d640 1 --2- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f04cc12f310 0x7f04cc11a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04d0f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:07.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04d0f9d640 1 -- 172.21.15.178:0/606544895 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc11c7f0 con 0x7f04cc12f310 2024-09-15T01:32:07.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04d0f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:07.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04d0f9d640 10 monclient(hunting): authenticate will time out at 732.195553s 2024-09-15T01:32:07.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 10 monclient(hunting): get_auth_request con 0x7f04cc09cdc0 auth_method 0 2024-09-15T01:32:07.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:07.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:07.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:07.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:07.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:07.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:07.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd574ff9640 10 monclient: dump: 2024-09-15T01:32:07.570 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.571 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.571 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.571 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.571 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.571 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.571 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.571 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.571 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.571 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.571 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.571 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.571 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.572 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.572 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.572 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd574ff9640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 10 monclient(hunting): handle_auth_done global_id 4323 payload 824 2024-09-15T01:32:07.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd574ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd57ed54640 5 monclient: authenticate success, global_id 4331 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd574ff9640 1 -- 172.21.15.178:0/2090546385 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57000f040 con 0x7fd57806e510 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd574ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd57ed54640 10 monclient: _renew_subs 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd574ff9640 1 -- 172.21.15.178:0/2090546385 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd570009ab0 con 0x7fd57806e510 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd57ed54640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd57ed54640 1 -- 172.21.15.178:0/2090546385 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd578133080 con 0x7fd57806e510 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd576ffd640 4 set_mon_vals no callback set 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd57ed54640 10 monclient: _renew_subs 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd57ed54640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd57ed54640 1 -- 172.21.15.178:0/2090546385 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd578134840 con 0x7fd57806e510 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd574ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd574ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7fd574ff9640 10 monclient: dump: 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.574 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.575 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.575 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.575 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.575 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 1 -- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f04cc12f310 msgr2=0x7f04cc11a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 1 --2- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f04cc12f310 0x7f04cc11a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 1 -- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0a2630 msgr2=0x7f04cc12da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 1 --2- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0a2630 0x7f04cc12da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:07.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 1 monclient: found mon.noname-g 2024-09-15T01:32:07.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:07.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:32:07.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.444+0000 7f04ca59c640 1 -- 172.21.15.178:0/606544895 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c4009a20 con 0x7f04cc09cdc0 2024-09-15T01:32:07.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7f04c9d9b640 10 monclient: get_auth_request con 0x7f04cc0a2630 auth_method 0 2024-09-15T01:32:07.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7f04cad9d640 10 monclient: get_auth_request con 0x7f04cc12f310 auth_method 0 2024-09-15T01:32:07.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7f04cad9d640 1 --2- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f04cc12f310 0x7f04cc11a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:32:07.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7f04c9d9b640 1 --2- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0a2630 0x7f04cc12da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:32:07.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7fa47e7fc640 1 -- 172.21.15.178:0/10545450 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa48400f490 con 0x7fa48809fa10 2024-09-15T01:32:07.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7fa47e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7fa47e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:07.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7fd57ed54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7fd57ed54640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7fd57ed54640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:07.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7fd57ed54640 1 -- 172.21.15.178:0/2090546385 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57804b720 con 0x7fd57806e510 2024-09-15T01:32:07.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7f04ca59c640 1 --2- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc09cdc0 0x7f04cc13da60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0109c0 tx=0x7f04bc0091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04ba7fc640 1 -- 172.21.15.178:0/606544895 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f04bc0095a0 con 0x7f04cc09cdc0 2024-09-15T01:32:07.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04ba7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:32:07.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04ba7fc640 10 monclient: dump: 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.581 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04ba7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04ba7fc640 1 -- 172.21.15.178:0/606544895 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc00e750 con 0x7f04cc09cdc0 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04ba7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04ba7fc640 1 -- 172.21.15.178:0/606544895 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f04bc00ed40 con 0x7f04cc09cdc0 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04ba7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04ba7fc640 10 monclient: dump: 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:07.582 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:07.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04c8d99640 4 set_mon_vals no callback set 2024-09-15T01:32:07.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04d0f9d640 5 monclient: authenticate success, global_id 4323 2024-09-15T01:32:07.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04d0f9d640 10 monclient: _renew_subs 2024-09-15T01:32:07.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04d0f9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:32:07.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.446+0000 7f04d0f9d640 1 -- 172.21.15.178:0/606544895 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c0009390 con 0x7f04cc09cdc0 2024-09-15T01:32:07.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.445+0000 7fd574ff9640 1 -- 172.21.15.178:0/2090546385 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd570003020 con 0x7fd57806e510 2024-09-15T01:32:07.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.447+0000 7fd574ff9640 1 --2- 172.21.15.178:0/2090546385 >> v2:172.21.15.204:6812/28210 conn(0x7fd5640397a0 0x7fd56403bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.447+0000 7fd574ff9640 1 -- 172.21.15.178:0/2090546385 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7fd57004af80 con 0x7fd57806e510 2024-09-15T01:32:07.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.447+0000 7f04d0f9d640 10 monclient: _renew_subs 2024-09-15T01:32:07.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.447+0000 7f04d0f9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:32:07.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.447+0000 7f04d0f9d640 1 -- 172.21.15.178:0/606544895 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04cc11cc00 con 0x7f04cc09cdc0 2024-09-15T01:32:07.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.447+0000 7fd577fff640 1 --2- 172.21.15.178:0/2090546385 >> v2:172.21.15.204:6812/28210 conn(0x7fd5640397a0 0x7fd56403bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.447+0000 7fd577fff640 10 monclient: get_auth_request con 0x7fd5640397a0 auth_method 0 2024-09-15T01:32:07.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.448+0000 7fd577fff640 1 --2- 172.21.15.178:0/2090546385 >> v2:172.21.15.204:6812/28210 conn(0x7fd5640397a0 0x7fd56403bc50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd56c001de0 tx=0x7fd56c009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.448+0000 7f476effd640 1 -- 172.21.15.178:0/130406943 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4780014440 con 0x7f47880a25a0 2024-09-15T01:32:07.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.448+0000 7f476effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.448+0000 7f476effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:07.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.448+0000 7f04ba7fc640 1 -- 172.21.15.178:0/606544895 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f04bc00e8c0 con 0x7f04cc09cdc0 2024-09-15T01:32:07.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.449+0000 7f04ba7fc640 1 --2- 172.21.15.178:0/606544895 >> v2:172.21.15.204:6812/28210 conn(0x7f04ac0397b0 0x7f04ac03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:07.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.449+0000 7f04ba7fc640 1 -- 172.21.15.178:0/606544895 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f04bc04f0f0 con 0x7f04cc09cdc0 2024-09-15T01:32:07.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.449+0000 7f04c9d9b640 1 --2- 172.21.15.178:0/606544895 >> v2:172.21.15.204:6812/28210 conn(0x7f04ac0397b0 0x7f04ac03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:07.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.449+0000 7f04c9d9b640 10 monclient: get_auth_request con 0x7f04ac0397b0 auth_method 0 2024-09-15T01:32:07.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.449+0000 7f04abfff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.449+0000 7f04abfff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.449+0000 7f04abfff640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:32:07.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.449+0000 7f04abfff640 1 -- 172.21.15.178:0/606544895 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a4005010 con 0x7f04cc09cdc0 2024-09-15T01:32:07.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.450+0000 7f04c9d9b640 1 --2- 172.21.15.178:0/606544895 >> v2:172.21.15.204:6812/28210 conn(0x7f04ac0397b0 0x7f04ac03bc60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009360 tx=0x7f04c0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:07.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.454+0000 7f04ba7fc640 1 -- 172.21.15.178:0/606544895 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f04bc017a90 con 0x7f04cc09cdc0 2024-09-15T01:32:07.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.454+0000 7f04ba7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.454+0000 7f04ba7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:07.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.455+0000 7fd574ff9640 1 -- 172.21.15.178:0/2090546385 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd57000cdc0 con 0x7fd57806e510 2024-09-15T01:32:07.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.455+0000 7fd574ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:07.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.455+0000 7fd574ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:07.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.581+0000 7f87520b8640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T01:32:07.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.581+0000 7f87520b8640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T01:32:07.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.581+0000 7f87520b8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.581+0000 7f87520b8640 1 -- 172.21.15.178:0/1536686792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f874c070770 con 0x7f874c137cc0 2024-09-15T01:32:07.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.582+0000 7f87337fe640 1 -- 172.21.15.178:0/1536686792 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8744049070 con 0x7f874c137cc0 2024-09-15T01:32:07.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.582+0000 7f87337fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T01:32:07.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.582+0000 7f87337fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:07.590 INFO:teuthology.orchestra.run.smithi178.stdout:47244640258 2024-09-15T01:32:07.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.584+0000 7f87520b8640 1 -- 172.21.15.178:0/1536686792 >> v2:172.21.15.204:6812/28210 conn(0x7f87380397b0 msgr2=0x7f873803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.584+0000 7f87520b8640 1 --2- 172.21.15.178:0/1536686792 >> v2:172.21.15.204:6812/28210 conn(0x7f87380397b0 0x7f873803bc60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f873c000c00 tx=0x7f873c009040 comp rx=0 tx=0).stop 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.584+0000 7f87520b8640 10 monclient: shutdown 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.584+0000 7f87520b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.584+0000 7f87520b8640 1 -- 172.21.15.178:0/1536686792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c137cc0 msgr2=0x7f874c06dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.584+0000 7f87520b8640 1 --2- 172.21.15.178:0/1536686792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c137cc0 0x7f874c06dc20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8744009d30 tx=0x7f8744037000 comp rx=0 tx=0).stop 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.584+0000 7f87520b8640 1 -- 172.21.15.178:0/1536686792 shutdown_connections 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.584+0000 7f87520b8640 1 --2- 172.21.15.178:0/1536686792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f874c137cc0 0x7f874c06dc20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.584+0000 7f87520b8640 1 --2- 172.21.15.178:0/1536686792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f874c1353a0 0x7f874c06d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.584+0000 7f87520b8640 1 --2- 172.21.15.178:0/1536686792 >> v2:172.21.15.204:6812/28210 conn(0x7f87380397b0 0x7f873803bc60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.584+0000 7f87520b8640 1 --2- 172.21.15.178:0/1536686792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f874c09c020 0x7f874c06d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.584+0000 7f87520b8640 1 -- 172.21.15.178:0/1536686792 >> 172.21.15.178:0/1536686792 conn(0x7f874c068a60 msgr2=0x7f874c09f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.585+0000 7f87520b8640 1 -- 172.21.15.178:0/1536686792 shutdown_connections 2024-09-15T01:32:07.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.585+0000 7f87520b8640 1 -- 172.21.15.178:0/1536686792 wait complete. 2024-09-15T01:32:07.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.1 2024-09-15T01:32:07.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.614+0000 7f478d15d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T01:32:07.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.614+0000 7f478d15d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T01:32:07.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.614+0000 7f478d15d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:07.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.614+0000 7f478d15d640 1 -- 172.21.15.178:0/130406943 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4788070770 con 0x7f47880a25a0 2024-09-15T01:32:07.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.614+0000 7f476effd640 1 -- 172.21.15.178:0/130406943 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f478001c360 con 0x7f47880a25a0 2024-09-15T01:32:07.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.614+0000 7f476effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T01:32:07.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.614+0000 7f476effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:07.618 INFO:teuthology.orchestra.run.smithi178.stdout:47244640258 2024-09-15T01:32:07.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.616+0000 7f478d15d640 1 -- 172.21.15.178:0/130406943 >> v2:172.21.15.204:6812/28210 conn(0x7f47740396e0 msgr2=0x7f477403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.616+0000 7f478d15d640 1 --2- 172.21.15.178:0/130406943 >> v2:172.21.15.204:6812/28210 conn(0x7f47740396e0 0x7f477403bb90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f477c007fd0 tx=0x7f477c009040 comp rx=0 tx=0).stop 2024-09-15T01:32:07.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.616+0000 7f478d15d640 10 monclient: shutdown 2024-09-15T01:32:07.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.616+0000 7f478d15d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.616+0000 7f478d15d640 1 -- 172.21.15.178:0/130406943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47880a25a0 msgr2=0x7f4788123c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.616+0000 7f478d15d640 1 --2- 172.21.15.178:0/130406943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47880a25a0 0x7f4788123c10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f478000c1c0 tx=0x7f478000b040 comp rx=0 tx=0).stop 2024-09-15T01:32:07.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.616+0000 7f478d15d640 1 -- 172.21.15.178:0/130406943 shutdown_connections 2024-09-15T01:32:07.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.616+0000 7f478d15d640 1 --2- 172.21.15.178:0/130406943 >> v2:172.21.15.204:6812/28210 conn(0x7f47740396e0 0x7f477403bb90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.616+0000 7f478d15d640 1 --2- 172.21.15.178:0/130406943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47880a25a0 0x7f4788123c10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.616+0000 7f478d15d640 1 --2- 172.21.15.178:0/130406943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47880993e0 0x7f478813a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.616+0000 7f478d15d640 1 --2- 172.21.15.178:0/130406943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4788098900 0x7f47881461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.616+0000 7f478d15d640 1 -- 172.21.15.178:0/130406943 >> 172.21.15.178:0/130406943 conn(0x7f4788067710 msgr2=0x7f47880a04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.617+0000 7f478d15d640 1 -- 172.21.15.178:0/130406943 shutdown_connections 2024-09-15T01:32:07.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.617+0000 7f478d15d640 1 -- 172.21.15.178:0/130406943 wait complete. 2024-09-15T01:32:07.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.0 2024-09-15T01:32:07.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.705+0000 7f66a595d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T01:32:07.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.705+0000 7f66a595d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T01:32:07.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.705+0000 7f66a595d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:07.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.705+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f66a0070770 con 0x7f66a00a3550 2024-09-15T01:32:07.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.706+0000 7f6686ffd640 1 -- 172.21.15.178:0/2296426124 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6698012b20 con 0x7f66a00a3550 2024-09-15T01:32:07.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.706+0000 7f6686ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T01:32:07.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.706+0000 7f6686ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:07.710 INFO:teuthology.orchestra.run.smithi178.stdout:47244640258 2024-09-15T01:32:07.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 >> v2:172.21.15.204:6812/28210 conn(0x7f668c0396e0 msgr2=0x7f668c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 1 --2- 172.21.15.178:0/2296426124 >> v2:172.21.15.204:6812/28210 conn(0x7f668c0396e0 0x7f668c03bb90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6690034000 tx=0x7f6690032000 comp rx=0 tx=0).stop 2024-09-15T01:32:07.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 10 monclient: shutdown 2024-09-15T01:32:07.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a00a3550 msgr2=0x7f66a0122ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 1 --2- 172.21.15.178:0/2296426124 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a00a3550 0x7f66a0122ac0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f669800c1f0 tx=0x7f669800b040 comp rx=0 tx=0).stop 2024-09-15T01:32:07.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 shutdown_connections 2024-09-15T01:32:07.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 1 --2- 172.21.15.178:0/2296426124 >> v2:172.21.15.204:6812/28210 conn(0x7f668c0396e0 0x7f668c03bb90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 1 --2- 172.21.15.178:0/2296426124 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a00a3550 0x7f66a0122ac0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 1 --2- 172.21.15.178:0/2296426124 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a006e3b0 0x7f66a0122580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 1 --2- 172.21.15.178:0/2296426124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a0067130 0x7f66a0135a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 >> 172.21.15.178:0/2296426124 conn(0x7f66a0068a60 msgr2=0x7f66a006c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 shutdown_connections 2024-09-15T01:32:07.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.708+0000 7f66a595d640 1 -- 172.21.15.178:0/2296426124 wait complete. 2024-09-15T01:32:07.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.5 2024-09-15T01:32:07.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.736+0000 7f04abfff640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T01:32:07.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.736+0000 7f04abfff640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T01:32:07.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.736+0000 7f04abfff640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:32:07.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.736+0000 7f04abfff640 1 -- 172.21.15.178:0/606544895 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f04a40052c0 con 0x7f04cc09cdc0 2024-09-15T01:32:07.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.736+0000 7f04ba7fc640 1 -- 172.21.15.178:0/606544895 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f04bc024030 con 0x7f04cc09cdc0 2024-09-15T01:32:07.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.736+0000 7f04ba7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T01:32:07.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.736+0000 7f04ba7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:07.740 INFO:teuthology.orchestra.run.smithi178.stdout:47244640258 2024-09-15T01:32:07.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7fa49028b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T01:32:07.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7fa49028b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T01:32:07.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7fa49028b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:07.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7fa49028b640 1 -- 172.21.15.178:0/10545450 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa4700052c0 con 0x7fa48809fa10 2024-09-15T01:32:07.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 1 -- 172.21.15.178:0/606544895 >> v2:172.21.15.204:6812/28210 conn(0x7f04ac0397b0 msgr2=0x7f04ac03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 1 --2- 172.21.15.178:0/606544895 >> v2:172.21.15.204:6812/28210 conn(0x7f04ac0397b0 0x7f04ac03bc60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009360 tx=0x7f04c0032000 comp rx=0 tx=0).stop 2024-09-15T01:32:07.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 10 monclient: shutdown 2024-09-15T01:32:07.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 1 -- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc09cdc0 msgr2=0x7f04cc13da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 1 --2- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc09cdc0 0x7f04cc13da60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0109c0 tx=0x7f04bc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 1 -- 172.21.15.178:0/606544895 shutdown_connections 2024-09-15T01:32:07.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 1 --2- 172.21.15.178:0/606544895 >> v2:172.21.15.204:6812/28210 conn(0x7f04ac0397b0 0x7f04ac03bc60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 1 --2- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f04cc12f310 0x7f04cc11a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 1 --2- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0a2630 0x7f04cc12da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 1 --2- 172.21.15.178:0/606544895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc09cdc0 0x7f04cc13da60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 1 -- 172.21.15.178:0/606544895 >> 172.21.15.178:0/606544895 conn(0x7f04cc067710 msgr2=0x7f04cc0992e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 1 -- 172.21.15.178:0/606544895 shutdown_connections 2024-09-15T01:32:07.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.738+0000 7f04abfff640 1 -- 172.21.15.178:0/606544895 wait complete. 2024-09-15T01:32:07.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.739+0000 7fa47e7fc640 1 -- 172.21.15.178:0/10545450 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa484018020 con 0x7fa48809fa10 2024-09-15T01:32:07.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.739+0000 7fa47e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T01:32:07.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.739+0000 7fa47e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:07.744 INFO:teuthology.orchestra.run.smithi178.stdout:47244640258 2024-09-15T01:32:07.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 1 -- 172.21.15.178:0/10545450 >> v2:172.21.15.204:6812/28210 conn(0x7fa46c0397b0 msgr2=0x7fa46c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 1 --2- 172.21.15.178:0/10545450 >> v2:172.21.15.204:6812/28210 conn(0x7fa46c0397b0 0x7fa46c03bc60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa478006e70 tx=0x7fa478008040 comp rx=0 tx=0).stop 2024-09-15T01:32:07.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 10 monclient: shutdown 2024-09-15T01:32:07.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 1 -- 172.21.15.178:0/10545450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa48809fa10 msgr2=0x7fa488146cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 1 --2- 172.21.15.178:0/10545450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa48809fa10 0x7fa488146cc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa48400c810 tx=0x7fa484001430 comp rx=0 tx=0).stop 2024-09-15T01:32:07.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 1 -- 172.21.15.178:0/10545450 shutdown_connections 2024-09-15T01:32:07.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 1 --2- 172.21.15.178:0/10545450 >> v2:172.21.15.204:6812/28210 conn(0x7fa46c0397b0 0x7fa46c03bc60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 1 --2- 172.21.15.178:0/10545450 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa488126f50 0x7fa48809a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 1 --2- 172.21.15.178:0/10545450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa48809fa10 0x7fa488146cc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 1 --2- 172.21.15.178:0/10545450 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa48809d950 0x7fa488146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 1 -- 172.21.15.178:0/10545450 >> 172.21.15.178:0/10545450 conn(0x7fa4880a5ed0 msgr2=0x7fa488138b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 1 -- 172.21.15.178:0/10545450 shutdown_connections 2024-09-15T01:32:07.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.741+0000 7fa49028b640 1 -- 172.21.15.178:0/10545450 wait complete. 2024-09-15T01:32:07.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.745+0000 7fd57ed54640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T01:32:07.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.745+0000 7fd57ed54640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T01:32:07.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.745+0000 7fd57ed54640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:07.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.745+0000 7fd57ed54640 1 -- 172.21.15.178:0/2090546385 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd578070770 con 0x7fd57806e510 2024-09-15T01:32:07.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.745+0000 7fd574ff9640 1 -- 172.21.15.178:0/2090546385 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd57001cba0 con 0x7fd57806e510 2024-09-15T01:32:07.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.745+0000 7fd574ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T01:32:07.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.745+0000 7fd574ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:07.749 INFO:teuthology.orchestra.run.smithi178.stdout:47244640258 2024-09-15T01:32:07.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 1 -- 172.21.15.178:0/2090546385 >> v2:172.21.15.204:6812/28210 conn(0x7fd5640397a0 msgr2=0x7fd56403bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 1 --2- 172.21.15.178:0/2090546385 >> v2:172.21.15.204:6812/28210 conn(0x7fd5640397a0 0x7fd56403bc50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd56c001de0 tx=0x7fd56c009040 comp rx=0 tx=0).stop 2024-09-15T01:32:07.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 10 monclient: shutdown 2024-09-15T01:32:07.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:07.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 1 -- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd57806e510 msgr2=0x7fd57812e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:07.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 1 --2- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd57806e510 0x7fd57812e570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd57000ed60 tx=0x7fd57000b040 comp rx=0 tx=0).stop 2024-09-15T01:32:07.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 1 -- 172.21.15.178:0/2090546385 shutdown_connections 2024-09-15T01:32:07.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 1 --2- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd57812f5d0 0x7fd5781314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 1 --2- 172.21.15.178:0/2090546385 >> v2:172.21.15.204:6812/28210 conn(0x7fd5640397a0 0x7fd56403bc50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 1 --2- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd57806e8e0 0x7fd57812eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 1 --2- 172.21.15.178:0/2090546385 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd57806e510 0x7fd57812e570 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:07.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 1 -- 172.21.15.178:0/2090546385 >> 172.21.15.178:0/2090546385 conn(0x7fd578068a60 msgr2=0x7fd5780a0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:07.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 1 -- 172.21.15.178:0/2090546385 shutdown_connections 2024-09-15T01:32:07.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:07.748+0000 7fd57ed54640 1 -- 172.21.15.178:0/2090546385 wait complete. 2024-09-15T01:32:07.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.2 2024-09-15T01:32:07.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.3 2024-09-15T01:32:07.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.4 2024-09-15T01:32:08.600 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T01:32:08.630 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T01:32:08.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.699+0000 7feac991d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:08.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.699+0000 7feac991d640 10 monclient: build_initial_monmap 2024-09-15T01:32:08.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.699+0000 7feac991d640 10 monclient: monmap: 2024-09-15T01:32:08.703 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:08.703 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.703 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:08.700950+0000 2024-09-15T01:32:08.703 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:08.700950+0000 2024-09-15T01:32:08.703 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:08.703 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.703 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:08.703 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:08.703 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:08.703 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:08.704 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:08.704 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:08.704 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:08.704 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:08.704 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:08.704 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:08.704 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:08.704 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.699+0000 7feac991d640 1 Processor -- start 2024-09-15T01:32:08.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.699+0000 7feac991d640 1 -- start start 2024-09-15T01:32:08.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.699+0000 7feac991d640 10 monclient: init 2024-09-15T01:32:08.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:08.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 10 monclient: _add_conns ranks=[9,7,0,10,6,2,8,1,5,4,3] 2024-09-15T01:32:08.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feac409e700 0x7feac4069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 10 monclient(hunting): picked mon.noname-a con 0x7feac409e700 addr v2:172.21.15.178:3300/0 2024-09-15T01:32:08.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feac409ead0 0x7feac409eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 10 monclient(hunting): picked mon.noname-i con 0x7feac409ead0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:08.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feac412f4c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:08.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac404bf60 con 0x7feac412f4c0 2024-09-15T01:32:08.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac4068340 con 0x7feac409e700 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feac4069210 con 0x7feac409ead0 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac991d640 10 monclient(hunting): authenticate will time out at 733.451307s 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feac409e700 0x7feac4069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feac409e700 0x7feac4069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:34966/0 (socket says 172.21.15.178:34966) 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac37fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7feac412f4c0 0x7feac412f920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:49232/0 (socket says 172.21.15.178:49232) 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 1 -- 172.21.15.178:0/932966090 learned_addr learned my addr 172.21.15.178:0/932966090 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 10 monclient(hunting): get_auth_request con 0x7feac409e700 auth_method 0 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac27fc640 1 --2- 172.21.15.178:0/932966090 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feac409ead0 0x7feac409eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac27fc640 10 monclient(hunting): get_auth_request con 0x7feac409ead0 auth_method 0 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 10 monclient(hunting): handle_auth_done global_id 4356 payload 274 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 1 -- 172.21.15.178:0/932966090 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feac409ead0 msgr2=0x7feac409eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 1 --2- 172.21.15.178:0/932966090 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feac409ead0 0x7feac409eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 1 -- 172.21.15.178:0/932966090 >> v1:172.21.15.178:6789/0 conn(0x7feac412f4c0 legacy=0x7feac412f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 1 monclient: found mon.noname-a 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac2ffd640 1 -- 172.21.15.178:0/932966090 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac4068f30 con 0x7feac409e700 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac1ffb640 1 -- 172.21.15.178:0/932966090 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210021337 0 0) 0x7feac404bf60 con 0x7feac412f4c0 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.700+0000 7feac1ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.701+0000 7feac2ffd640 1 --2- 172.21.15.178:0/932966090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feac409e700 0x7feac4069bf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feabc009910 tx=0x7feabc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e5cc90ec1dba37 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.701+0000 7feac1ffb640 1 -- 172.21.15.178:0/932966090 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feabc02ad20 con 0x7feac409e700 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.701+0000 7feac1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac1ffb640 10 monclient: dump: 2024-09-15T01:32:08.716 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.717 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.718 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:32:08.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:08.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 5 monclient: authenticate success, global_id 4356 2024-09-15T01:32:08.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac1ffb640 1 -- 172.21.15.178:0/932966090 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc031820 con 0x7feac409e700 2024-09-15T01:32:08.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:08.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:08.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:08.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 4 set_mon_vals no callback set 2024-09-15T01:32:08.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac1ffb640 1 -- 172.21.15.178:0/932966090 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feabc02aea0 con 0x7feac409e700 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac1ffb640 10 monclient: dump: 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.719 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.720 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.720 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.720 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.720 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.720 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.720 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 10 monclient: shutdown 2024-09-15T01:32:08.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:08.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 1 -- 172.21.15.178:0/932966090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feac409e700 msgr2=0x7feac4069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 1 --2- 172.21.15.178:0/932966090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feac409e700 0x7feac4069bf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feabc009910 tx=0x7feabc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 1 -- 172.21.15.178:0/932966090 shutdown_connections 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 1 --2- 172.21.15.178:0/932966090 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feac409ead0 0x7feac409eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 1 --2- 172.21.15.178:0/932966090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feac409e700 0x7feac4069bf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 1 -- 172.21.15.178:0/932966090 >> 172.21.15.178:0/932966090 conn(0x7feac40a4390 msgr2=0x7feac4067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 1 -- 172.21.15.178:0/932966090 shutdown_connections 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.702+0000 7feac991d640 1 -- 172.21.15.178:0/932966090 wait complete. 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.703+0000 7feac991d640 10 monclient: build_initial_monmap 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.703+0000 7feac991d640 10 monclient: monmap: 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:08.721 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.703+0000 7feac991d640 1 Processor -- start 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.703+0000 7feac991d640 1 -- start start 2024-09-15T01:32:08.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.703+0000 7feac991d640 10 monclient: init 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 10 monclient: _add_conns ranks=[7,3,4,6,5,8,1,0,2] 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feac409e700 0x7feac41419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 10 monclient(hunting): picked mon.noname-i con 0x7feac409e700 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feac409ead0 0x7feac4141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 10 monclient(hunting): picked mon.noname-g con 0x7feac409ead0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feac412f4c0 0x7feac4142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 10 monclient(hunting): picked mon.noname-c con 0x7feac412f4c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac411f6e0 con 0x7feac409ead0 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feac411fc10 con 0x7feac412f4c0 2024-09-15T01:32:08.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feac411db20 con 0x7feac409e700 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 10 monclient(hunting): authenticate will time out at 733.455138s 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feac412f4c0 0x7feac4142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feac412f4c0 0x7feac4142450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:36730/0 (socket says 172.21.15.178:36730) 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 1 -- 172.21.15.178:0/972844153 learned_addr learned my addr 172.21.15.178:0/972844153 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 10 monclient(hunting): get_auth_request con 0x7feac412f4c0 auth_method 0 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac2ffd640 1 --2- 172.21.15.178:0/972844153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feac409e700 0x7feac41419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac27fc640 1 --2- 172.21.15.178:0/972844153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feac409ead0 0x7feac4141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac2ffd640 10 monclient(hunting): get_auth_request con 0x7feac409e700 auth_method 0 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac27fc640 10 monclient(hunting): get_auth_request con 0x7feac409ead0 auth_method 0 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 10 monclient(hunting): handle_auth_done global_id 4279 payload 824 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 1 -- 172.21.15.178:0/972844153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feac409e700 msgr2=0x7feac41419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 1 --2- 172.21.15.178:0/972844153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feac409e700 0x7feac41419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 1 -- 172.21.15.178:0/972844153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feac409ead0 msgr2=0x7feac4141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 1 --2- 172.21.15.178:0/972844153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feac409ead0 0x7feac4141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 1 monclient: found mon.noname-c 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 1 -- 172.21.15.178:0/972844153 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc009450 con 0x7feac412f4c0 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac37fe640 1 --2- 172.21.15.178:0/972844153 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feac412f4c0 0x7feac4142450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feab400c520 tx=0x7feab4007eb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feaaaffd640 1 -- 172.21.15.178:0/972844153 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feab40047a0 con 0x7feac412f4c0 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feaaaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feaaaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feaaaffd640 10 monclient: dump: 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.726 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feaaaffd640 10 monclient: _finish_auth 0 2024-09-15T01:32:08.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feaaaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:08.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feaaaffd640 1 -- 172.21.15.178:0/972844153 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab4004940 con 0x7feac412f4c0 2024-09-15T01:32:08.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 5 monclient: authenticate success, global_id 4279 2024-09-15T01:32:08.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 10 monclient: _renew_subs 2024-09-15T01:32:08.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:08.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac991d640 1 -- 172.21.15.178:0/972844153 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac4068520 con 0x7feac412f4c0 2024-09-15T01:32:08.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feaaaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:08.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feaaaffd640 1 -- 172.21.15.178:0/972844153 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feab4013400 con 0x7feac412f4c0 2024-09-15T01:32:08.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feaaaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feaaaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:08.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feaaaffd640 10 monclient: dump: 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.729 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.705+0000 7feac991d640 10 monclient: _renew_subs 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.705+0000 7feac991d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.705+0000 7feac991d640 1 -- 172.21.15.178:0/972844153 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac411e470 con 0x7feac412f4c0 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.704+0000 7feac17fa640 4 set_mon_vals no callback set 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.705+0000 7feaaaffd640 1 -- 172.21.15.178:0/972844153 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7feab40072e0 con 0x7feac412f4c0 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.705+0000 7feac991d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.705+0000 7feac991d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.705+0000 7feac991d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.705+0000 7feac991d640 1 -- 172.21.15.178:0/972844153 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac40687f0 con 0x7feac412f4c0 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.706+0000 7feaaaffd640 1 --2- 172.21.15.178:0/972844153 >> v2:172.21.15.204:6812/28210 conn(0x7feab00397b0 0x7feab003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.706+0000 7feaaaffd640 1 -- 172.21.15.178:0/972844153 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7feab404a280 con 0x7feac412f4c0 2024-09-15T01:32:08.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.706+0000 7feac2ffd640 1 --2- 172.21.15.178:0/972844153 >> v2:172.21.15.204:6812/28210 conn(0x7feab00397b0 0x7feab003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.706+0000 7feac2ffd640 10 monclient: get_auth_request con 0x7feab00397b0 auth_method 0 2024-09-15T01:32:08.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.706+0000 7feac2ffd640 1 --2- 172.21.15.178:0/972844153 >> v2:172.21.15.204:6812/28210 conn(0x7feab00397b0 0x7feab003bc60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feabc009420 tx=0x7feabc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.731 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T01:32:08.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.710+0000 7feaaaffd640 1 -- 172.21.15.178:0/972844153 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7feab400ea70 con 0x7feac412f4c0 2024-09-15T01:32:08.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.710+0000 7feaaaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.710+0000 7feaaaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:08.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.740+0000 7f2ac9860640 10 monclient: get_monmap_and_config 2024-09-15T01:32:08.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.740+0000 7f2ac9860640 10 monclient: build_initial_monmap 2024-09-15T01:32:08.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.740+0000 7f2ac9860640 10 monclient: monmap: 2024-09-15T01:32:08.743 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:08.743 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.743 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:08.742327+0000 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:08.742327+0000 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:08.744 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:08.745 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:08.745 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 1 Processor -- start 2024-09-15T01:32:08.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 1 -- start start 2024-09-15T01:32:08.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 10 monclient: init 2024-09-15T01:32:08.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 10 monclient: _add_conns ranks=[10,8,7,0,3,6,5,1,2,9,4] 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2ac40a2930 0x7f2ac4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 10 monclient(hunting): picked mon.noname-f con 0x7f2ac40a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ac409faa0 0x7f2ac409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 10 monclient(hunting): picked mon.noname-e con 0x7f2ac409faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2ac40a08c0 0x7f2ac40a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 10 monclient(hunting): picked mon.noname-i con 0x7f2ac40a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac409e0f0 con 0x7f2ac40a2930 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac409dd80 con 0x7f2ac40a08c0 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac40a2d00 con 0x7f2ac409faa0 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.741+0000 7f2ac9860640 10 monclient(hunting): authenticate will time out at 733.492904s 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ac409faa0 0x7f2ac409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ac409faa0 0x7f2ac409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:57932/0 (socket says 172.21.15.178:57932) 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 1 -- 172.21.15.178:0/1712183426 learned_addr learned my addr 172.21.15.178:0/1712183426 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 10 monclient(hunting): get_auth_request con 0x7f2ac409faa0 auth_method 0 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac37fe640 1 --2- 172.21.15.178:0/1712183426 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2ac40a08c0 0x7f2ac40a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac37fe640 10 monclient(hunting): get_auth_request con 0x7f2ac40a08c0 auth_method 0 2024-09-15T01:32:08.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac2ffd640 1 --2- 172.21.15.178:0/1712183426 >> v2:172.21.15.204:3300/0 conn(0x7f2ac40a2930 0x7f2ac4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac2ffd640 10 monclient(hunting): get_auth_request con 0x7f2ac40a2930 auth_method 0 2024-09-15T01:32:08.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 10 monclient(hunting): handle_auth_done global_id 4328 payload 274 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 1 -- 172.21.15.178:0/1712183426 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2ac40a08c0 msgr2=0x7f2ac40a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 1 --2- 172.21.15.178:0/1712183426 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2ac40a08c0 0x7f2ac40a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 1 -- 172.21.15.178:0/1712183426 >> v2:172.21.15.204:3300/0 conn(0x7f2ac40a2930 msgr2=0x7f2ac4132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 1 --2- 172.21.15.178:0/1712183426 >> v2:172.21.15.204:3300/0 conn(0x7f2ac40a2930 0x7f2ac4132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 10 monclient: _finish_hunting 0 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 1 monclient: found mon.noname-e 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac27fc640 1 -- 172.21.15.178:0/1712183426 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac404bf60 con 0x7f2ac409faa0 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.742+0000 7f2ac2ffd640 1 --2- 172.21.15.178:0/1712183426 >> v2:172.21.15.204:3300/0 conn(0x7f2ac40a2930 0x7f2ac4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:08.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac27fc640 1 --2- 172.21.15.178:0/1712183426 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ac409faa0 0x7f2ac409fe70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009910 tx=0x7f2ab80091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e49d7dc22ce45cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac1ffb640 1 -- 172.21.15.178:0/1712183426 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2ab802ac80 con 0x7f2ac409faa0 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac1ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac1ffb640 10 monclient: dump: 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.758 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac1ffb640 1 -- 172.21.15.178:0/1712183426 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab802ae00 con 0x7f2ac409faa0 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac1ffb640 1 -- 172.21.15.178:0/1712183426 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2ab8031ca0 con 0x7f2ac409faa0 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac9860640 5 monclient: authenticate success, global_id 4328 2024-09-15T01:32:08.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac9860640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac9860640 4 set_mon_vals no callback set 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac1ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.743+0000 7f2ac1ffb640 10 monclient: dump: 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.760 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.744+0000 7f2ac9860640 10 monclient: shutdown 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.744+0000 7f2ac9860640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.744+0000 7f2ac9860640 1 -- 172.21.15.178:0/1712183426 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ac409faa0 msgr2=0x7f2ac409fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.744+0000 7f2ac9860640 1 --2- 172.21.15.178:0/1712183426 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ac409faa0 0x7f2ac409fe70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009910 tx=0x7f2ab80091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.744+0000 7f2ac9860640 1 -- 172.21.15.178:0/1712183426 shutdown_connections 2024-09-15T01:32:08.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.744+0000 7f2ac9860640 1 --2- 172.21.15.178:0/1712183426 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2ac40a08c0 0x7f2ac40a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.744+0000 7f2ac9860640 1 --2- 172.21.15.178:0/1712183426 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ac409faa0 0x7f2ac409fe70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.744+0000 7f2ac9860640 1 --2- 172.21.15.178:0/1712183426 >> v2:172.21.15.204:3300/0 conn(0x7f2ac40a2930 0x7f2ac4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.744+0000 7f2ac9860640 1 -- 172.21.15.178:0/1712183426 >> 172.21.15.178:0/1712183426 conn(0x7f2ac40a5ed0 msgr2=0x7f2ac40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.744+0000 7f2ac9860640 1 -- 172.21.15.178:0/1712183426 shutdown_connections 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.744+0000 7f2ac9860640 1 -- 172.21.15.178:0/1712183426 wait complete. 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient: build_initial_monmap 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient: monmap: 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:08.762 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:08.763 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:08.763 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:08.763 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:08.763 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:08.763 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:08.763 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:08.763 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:08.763 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:08.763 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 1 Processor -- start 2024-09-15T01:32:08.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 1 -- start start 2024-09-15T01:32:08.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient: init 2024-09-15T01:32:08.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:08.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient: _add_conns ranks=[6,7,1,2,3,5,8,4,0] 2024-09-15T01:32:08.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2ac409faa0 0x7f2ac409b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient(hunting): picked mon.noname-d con 0x7f2ac409faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:08.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2ac40a08c0 0x7f2ac41317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient(hunting): picked mon.noname-i con 0x7f2ac40a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:08.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac40a2930 0x7f2ac4131d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient(hunting): picked mon.noname-f con 0x7f2ac40a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:08.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac409ba90 con 0x7f2ac40a2930 2024-09-15T01:32:08.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac40a2390 con 0x7f2ac409faa0 2024-09-15T01:32:08.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac409d630 con 0x7f2ac40a08c0 2024-09-15T01:32:08.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:08.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.745+0000 7f2ac9860640 10 monclient(hunting): authenticate will time out at 733.496954s 2024-09-15T01:32:08.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac2ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2ac409faa0 0x7f2ac409b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac2ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2ac409faa0 0x7f2ac409b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:40486/0 (socket says 172.21.15.178:40486) 2024-09-15T01:32:08.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2ac40a08c0 0x7f2ac41317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac2ffd640 1 -- 172.21.15.178:0/1607424138 learned_addr learned my addr 172.21.15.178:0/1607424138 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:08.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac2ffd640 10 monclient(hunting): get_auth_request con 0x7f2ac409faa0 auth_method 0 2024-09-15T01:32:08.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 10 monclient(hunting): get_auth_request con 0x7f2ac40a08c0 auth_method 0 2024-09-15T01:32:08.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac37fe640 1 --2- 172.21.15.178:0/1607424138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac40a2930 0x7f2ac4131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac37fe640 10 monclient(hunting): get_auth_request con 0x7f2ac40a2930 auth_method 0 2024-09-15T01:32:08.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 10 monclient(hunting): handle_auth_done global_id 4300 payload 824 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 1 -- 172.21.15.178:0/1607424138 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2ac409faa0 msgr2=0x7f2ac409b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 1 --2- 172.21.15.178:0/1607424138 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2ac409faa0 0x7f2ac409b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 1 -- 172.21.15.178:0/1607424138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac40a2930 msgr2=0x7f2ac4131d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 1 --2- 172.21.15.178:0/1607424138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac40a2930 0x7f2ac4131d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 10 monclient: _finish_hunting 0 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 1 monclient: found mon.noname-i 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 1 -- 172.21.15.178:0/1607424138 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab8009450 con 0x7f2ac40a08c0 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac2ffd640 1 --2- 172.21.15.178:0/1607424138 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2ac409faa0 0x7f2ac409b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac37fe640 1 --2- 172.21.15.178:0/1607424138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac40a2930 0x7f2ac4131d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac27fc640 1 --2- 172.21.15.178:0/1607424138 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2ac40a08c0 0x7f2ac41317c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40a2e40 tx=0x7f2ab8032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2aaaffd640 1 -- 172.21.15.178:0/1607424138 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2ab8031820 con 0x7f2ac40a08c0 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2aaaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2aaaffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2aaaffd640 10 monclient: dump: 2024-09-15T01:32:08.768 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.769 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2aaaffd640 10 monclient: _finish_auth 0 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2aaaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2aaaffd640 1 -- 172.21.15.178:0/1607424138 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab80319c0 con 0x7f2ac40a08c0 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac9860640 5 monclient: authenticate success, global_id 4300 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2aaaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac9860640 10 monclient: _renew_subs 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2aaaffd640 1 -- 172.21.15.178:0/1607424138 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2ab803a390 con 0x7f2ac40a08c0 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac9860640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac9860640 1 -- 172.21.15.178:0/1607424138 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac411e8e0 con 0x7f2ac40a08c0 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2aaaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2aaaffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2ac17fa640 4 set_mon_vals no callback set 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.746+0000 7f2aaaffd640 10 monclient: dump: 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.771 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.747+0000 7f2ac9860640 10 monclient: _renew_subs 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.747+0000 7f2ac9860640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.747+0000 7f2ac9860640 1 -- 172.21.15.178:0/1607424138 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac4137080 con 0x7f2ac40a08c0 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.748+0000 7f2aaaffd640 1 -- 172.21.15.178:0/1607424138 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2ab8039040 con 0x7f2ac40a08c0 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.748+0000 7f2ac9860640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.748+0000 7f2ac9860640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.748+0000 7f2ac9860640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.748+0000 7f2ac9860640 1 -- 172.21.15.178:0/1607424138 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac4137210 con 0x7f2ac40a08c0 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.748+0000 7f2aaaffd640 1 --2- 172.21.15.178:0/1607424138 >> v2:172.21.15.204:6812/28210 conn(0x7f2ab0039710 0x7f2ab003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.748+0000 7f2aaaffd640 1 -- 172.21.15.178:0/1607424138 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f2ab806a100 con 0x7f2ac40a08c0 2024-09-15T01:32:08.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.748+0000 7f2ac2ffd640 1 --2- 172.21.15.178:0/1607424138 >> v2:172.21.15.204:6812/28210 conn(0x7f2ab0039710 0x7f2ab003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.748+0000 7f2ac2ffd640 10 monclient: get_auth_request con 0x7f2ab0039710 auth_method 0 2024-09-15T01:32:08.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.749+0000 7f2ac2ffd640 1 --2- 172.21.15.178:0/1607424138 >> v2:172.21.15.204:6812/28210 conn(0x7f2ab0039710 0x7f2ab003bbc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4010370 tx=0x7f2ab4008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.773 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T01:32:08.774 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T01:32:08.774 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T01:32:08.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.753+0000 7f2aaaffd640 1 -- 172.21.15.178:0/1607424138 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2ab80313a0 con 0x7f2ac40a08c0 2024-09-15T01:32:08.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.753+0000 7f2aaaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.753+0000 7f2aaaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:08.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.850+0000 7f711799d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:08.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.850+0000 7f711799d640 10 monclient: build_initial_monmap 2024-09-15T01:32:08.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.850+0000 7f711799d640 10 monclient: monmap: 2024-09-15T01:32:08.853 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:08.853 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.853 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:08.852174+0000 2024-09-15T01:32:08.853 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:08.852174+0000 2024-09-15T01:32:08.853 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:08.854 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.851+0000 7f711799d640 1 Processor -- start 2024-09-15T01:32:08.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.851+0000 7f711799d640 1 -- start start 2024-09-15T01:32:08.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.851+0000 7f711799d640 10 monclient: init 2024-09-15T01:32:08.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:08.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 10 monclient: _add_conns ranks=[0,3,5,8,6,4,1,9,10,2,7] 2024-09-15T01:32:08.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f71100a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:08.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7110132de0 0x7f7110137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 10 monclient(hunting): picked mon.noname-g con 0x7f7110132de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:08.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f71101380e0 0x7f711013a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 10 monclient(hunting): picked mon.noname-h con 0x7f71101380e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:08.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711004bf60 con 0x7f71100a2930 2024-09-15T01:32:08.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711009dd80 con 0x7f7110132de0 2024-09-15T01:32:08.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71100a2d00 con 0x7f71101380e0 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f711799d640 10 monclient(hunting): authenticate will time out at 733.603476s 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7110132de0 0x7f7110137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7110132de0 0x7f7110137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:37440/0 (socket says 172.21.15.178:37440) 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 1 -- 172.21.15.178:0/1611258943 learned_addr learned my addr 172.21.15.178:0/1611258943 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 10 monclient(hunting): get_auth_request con 0x7f7110132de0 auth_method 0 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7115f3a640 1 --2- 172.21.15.178:0/1611258943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f71101380e0 0x7f711013a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7115f3a640 10 monclient(hunting): get_auth_request con 0x7f71101380e0 auth_method 0 2024-09-15T01:32:08.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7115f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7115f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7115f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7115f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7115f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7115f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 10 monclient(hunting): handle_auth_done global_id 4341 payload 274 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 1 -- 172.21.15.178:0/1611258943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f71101380e0 msgr2=0x7f711013a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 1 --2- 172.21.15.178:0/1611258943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f71101380e0 0x7f711013a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.852+0000 7f7114f38640 1 -- 172.21.15.178:0/1611258943 >> v1:172.21.15.178:6789/0 conn(0x7f71100a2930 legacy=0x7f711009faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:08.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.853+0000 7f7114f38640 10 monclient: _finish_hunting 0 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.853+0000 7f7114f38640 1 monclient: found mon.noname-g 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.853+0000 7f7114f38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.853+0000 7f7114f38640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.853+0000 7f7114f38640 1 -- 172.21.15.178:0/1611258943 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71101331b0 con 0x7f7110132de0 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7114f38640 1 --2- 172.21.15.178:0/1611258943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7110132de0 0x7f7110137ba0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7100009940 tx=0x7f71000091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=6416fbe9318ebb43 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7107fff640 1 -- 172.21.15.178:0/1611258943 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f710002ad00 con 0x7f7110132de0 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7107fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7107fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7107fff640 10 monclient: dump: 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.872 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7107fff640 10 monclient: _finish_auth 0 2024-09-15T01:32:08.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7107fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7107fff640 1 -- 172.21.15.178:0/1611258943 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710002ae80 con 0x7f7110132de0 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7107fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7107fff640 1 -- 172.21.15.178:0/1611258943 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7100031ca0 con 0x7f7110132de0 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7107fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7107fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f7107fff640 10 monclient: dump: 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.874 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f711799d640 5 monclient: authenticate success, global_id 4341 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f711799d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.854+0000 7f711799d640 4 set_mon_vals no callback set 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.855+0000 7f711799d640 10 monclient: shutdown 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.855+0000 7f711799d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:08.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.855+0000 7f711799d640 1 -- 172.21.15.178:0/1611258943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7110132de0 msgr2=0x7f7110137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.855+0000 7f711799d640 1 --2- 172.21.15.178:0/1611258943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7110132de0 0x7f7110137ba0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7100009940 tx=0x7f71000091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.855+0000 7f711799d640 1 -- 172.21.15.178:0/1611258943 shutdown_connections 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.855+0000 7f711799d640 1 --2- 172.21.15.178:0/1611258943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f71101380e0 0x7f711013a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.855+0000 7f711799d640 1 --2- 172.21.15.178:0/1611258943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7110132de0 0x7f7110137ba0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.855+0000 7f711799d640 1 -- 172.21.15.178:0/1611258943 >> 172.21.15.178:0/1611258943 conn(0x7f71100a5ed0 msgr2=0x7f71100a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.855+0000 7f711799d640 1 -- 172.21.15.178:0/1611258943 shutdown_connections 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.855+0000 7f711799d640 1 -- 172.21.15.178:0/1611258943 wait complete. 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.856+0000 7f711799d640 10 monclient: build_initial_monmap 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.856+0000 7f711799d640 10 monclient: monmap: 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:08.876 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.857+0000 7f711799d640 1 Processor -- start 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.857+0000 7f711799d640 1 -- start start 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.857+0000 7f711799d640 10 monclient: init 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 10 monclient: _add_conns ranks=[4,5,8,2,3,7,1,6,0] 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f71100a2930 0x7f711009c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7115739640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f71100a2930 0x7f711009c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7115739640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f71100a2930 0x7f711009c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:36738/0 (socket says 172.21.15.178:36738) 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7115739640 1 -- 172.21.15.178:0/497167695 learned_addr learned my addr 172.21.15.178:0/497167695 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 10 monclient(hunting): picked mon.noname-c con 0x7f71100a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7110132de0 0x7f7110096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 10 monclient(hunting): picked mon.noname-h con 0x7f7110132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71101380e0 0x7f7110096f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7115f3a640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71101380e0 0x7f7110096f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 10 monclient(hunting): picked mon.noname-e con 0x7f71101380e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71100a2390 con 0x7f71100a2930 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71100984a0 con 0x7f7110132de0 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f711009d630 con 0x7f71101380e0 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f711799d640 10 monclient(hunting): authenticate will time out at 733.609832s 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7115739640 10 monclient(hunting): get_auth_request con 0x7f71100a2930 auth_method 0 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7115739640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7115739640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7115739640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7114f38640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7110132de0 0x7f7110096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7114f38640 10 monclient(hunting): get_auth_request con 0x7f7110132de0 auth_method 0 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7114f38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7114f38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.858+0000 7f7114f38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7115f3a640 10 monclient(hunting): get_auth_request con 0x7f71101380e0 auth_method 0 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7115f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7115f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7115f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7115f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7115f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7115f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 10 monclient(hunting): handle_auth_done global_id 4370 payload 824 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 1 -- 172.21.15.178:0/497167695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71101380e0 msgr2=0x7f7110096f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71101380e0 0x7f7110096f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 1 -- 172.21.15.178:0/497167695 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f71100a2930 msgr2=0x7f711009c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f71100a2930 0x7f711009c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 10 monclient: _finish_hunting 0 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 1 monclient: found mon.noname-h 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 1 -- 172.21.15.178:0/497167695 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100009450 con 0x7f7110132de0 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7115739640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f71100a2930 0x7f711009c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7115f3a640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71101380e0 0x7f7110096f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:08.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f7114f38640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7110132de0 0x7f7110096a30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7100031000 tx=0x7f7100032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f71057fa640 1 -- 172.21.15.178:0/497167695 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f710003ca90 con 0x7f7110132de0 2024-09-15T01:32:08.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f71057fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f71057fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:32:08.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f71057fa640 10 monclient: dump: 2024-09-15T01:32:08.884 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.884 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.884 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.884 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.884 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.884 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.884 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f71057fa640 10 monclient: _finish_auth 0 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f71057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.859+0000 7f71057fa640 1 -- 172.21.15.178:0/497167695 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710003cc10 con 0x7f7110132de0 2024-09-15T01:32:08.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.860+0000 7f71057fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.860+0000 7f71057fa640 1 -- 172.21.15.178:0/497167695 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f710003cd80 con 0x7f7110132de0 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.860+0000 7f71057fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.860+0000 7f71057fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.860+0000 7f71057fa640 10 monclient: dump: 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.886 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.860+0000 7f711799d640 5 monclient: authenticate success, global_id 4370 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.860+0000 7f711799d640 10 monclient: _renew_subs 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.860+0000 7f711799d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.860+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711009a010 con 0x7f7110132de0 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.860+0000 7f71077fe640 4 set_mon_vals no callback set 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.861+0000 7f711799d640 10 monclient: _renew_subs 2024-09-15T01:32:08.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.861+0000 7f711799d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:32:08.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.861+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7110097ef0 con 0x7f7110132de0 2024-09-15T01:32:08.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.861+0000 7f71057fa640 1 -- 172.21.15.178:0/497167695 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f710003c610 con 0x7f7110132de0 2024-09-15T01:32:08.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.861+0000 7f71057fa640 1 --2- 172.21.15.178:0/497167695 >> v2:172.21.15.204:6812/28210 conn(0x7f70f40397e0 0x7f70f403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.861+0000 7f71057fa640 1 -- 172.21.15.178:0/497167695 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f710006e120 con 0x7f7110132de0 2024-09-15T01:32:08.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.862+0000 7f7115739640 1 --2- 172.21.15.178:0/497167695 >> v2:172.21.15.204:6812/28210 conn(0x7f70f40397e0 0x7f70f403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.862+0000 7f7115739640 10 monclient: get_auth_request con 0x7f70f40397e0 auth_method 0 2024-09-15T01:32:08.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.862+0000 7f711799d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.862+0000 7f711799d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.862+0000 7f711799d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:32:08.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.862+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711004b720 con 0x7f7110132de0 2024-09-15T01:32:08.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.862+0000 7f7115739640 1 --2- 172.21.15.178:0/497167695 >> v2:172.21.15.204:6812/28210 conn(0x7f70f40397e0 0x7f70f403bc90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f710c001800 tx=0x7f710c009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.867+0000 7f71057fa640 1 -- 172.21.15.178:0/497167695 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7100070060 con 0x7f7110132de0 2024-09-15T01:32:08.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.867+0000 7f71057fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.867+0000 7f71057fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:08.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.883+0000 7f2012a34640 10 monclient: get_monmap_and_config 2024-09-15T01:32:08.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.883+0000 7f2012a34640 10 monclient: build_initial_monmap 2024-09-15T01:32:08.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.883+0000 7f2012a34640 10 monclient: monmap: 2024-09-15T01:32:08.897 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:08.897 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.897 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:08.885714+0000 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:08.885714+0000 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:08.898 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.884+0000 7f2012a34640 1 Processor -- start 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.884+0000 7f2012a34640 1 -- start start 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.884+0000 7f2012a34640 10 monclient: init 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 10 monclient: _add_conns ranks=[0,9,7,8,3,10,2,4,1,5,6] 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f200c0a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f200c132de0 0x7f200c137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 10 monclient(hunting): picked mon.noname-a con 0x7f200c132de0 addr v2:172.21.15.178:3300/0 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f200c1380e0 0x7f200c13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 10 monclient(hunting): picked mon.noname-i con 0x7f200c1380e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200c04bf60 con 0x7f200c0a2930 2024-09-15T01:32:08.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f200c09dd80 con 0x7f200c132de0 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0a2d00 con 0x7f200c1380e0 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2012a34640 10 monclient(hunting): authenticate will time out at 733.636539s 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f200c132de0 0x7f200c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f200c132de0 0x7f200c137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:34980/0 (socket says 172.21.15.178:34980) 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 1 -- 172.21.15.178:0/3444278994 learned_addr learned my addr 172.21.15.178:0/3444278994 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 10 monclient(hunting): get_auth_request con 0x7f200c132de0 auth_method 0 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2010fd1640 1 --2- 172.21.15.178:0/3444278994 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f200c1380e0 0x7f200c13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2010fd1640 10 monclient(hunting): get_auth_request con 0x7f200c1380e0 auth_method 0 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2010fd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2010fd1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f2010fd1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 10 monclient(hunting): handle_auth_done global_id 4383 payload 274 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 1 -- 172.21.15.178:0/3444278994 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f200c1380e0 msgr2=0x7f200c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 1 --2- 172.21.15.178:0/3444278994 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f200c1380e0 0x7f200c13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.885+0000 7f200b7fe640 1 -- 172.21.15.178:0/3444278994 >> v1:172.21.15.178:6789/0 conn(0x7f200c0a2930 legacy=0x7f200c09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200b7fe640 1 monclient: found mon.noname-a 2024-09-15T01:32:08.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200b7fe640 1 -- 172.21.15.178:0/3444278994 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200c1331b0 con 0x7f200c132de0 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200b7fe640 1 --2- 172.21.15.178:0/3444278994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f200c132de0 0x7f200c137ba0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2000009910 tx=0x7f20000091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe913ae96779c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200affd640 1 -- 172.21.15.178:0/3444278994 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f200002ad20 con 0x7f200c132de0 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200affd640 10 monclient: dump: 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.902 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200affd640 10 monclient: _finish_auth 0 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f2012a34640 5 monclient: authenticate success, global_id 4383 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200affd640 1 -- 172.21.15.178:0/3444278994 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2000031820 con 0x7f200c132de0 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f2012a34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.886+0000 7f200affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:08.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f200affd640 1 -- 172.21.15.178:0/3444278994 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f200002aea0 con 0x7f200c132de0 2024-09-15T01:32:08.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f200affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f200affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:08.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f200affd640 10 monclient: dump: 2024-09-15T01:32:08.904 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.904 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.904 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.904 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.904 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.904 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.904 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.904 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.904 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f2012a34640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f2012a34640 4 set_mon_vals no callback set 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f2012a34640 10 monclient: shutdown 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f2012a34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f2012a34640 1 -- 172.21.15.178:0/3444278994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f200c132de0 msgr2=0x7f200c137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f2012a34640 1 --2- 172.21.15.178:0/3444278994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f200c132de0 0x7f200c137ba0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2000009910 tx=0x7f20000091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f2012a34640 1 -- 172.21.15.178:0/3444278994 shutdown_connections 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f2012a34640 1 --2- 172.21.15.178:0/3444278994 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f200c1380e0 0x7f200c13a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f2012a34640 1 --2- 172.21.15.178:0/3444278994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f200c132de0 0x7f200c137ba0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f2012a34640 1 -- 172.21.15.178:0/3444278994 >> 172.21.15.178:0/3444278994 conn(0x7f200c0a5ed0 msgr2=0x7f200c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f2012a34640 1 -- 172.21.15.178:0/3444278994 shutdown_connections 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.887+0000 7f2012a34640 1 -- 172.21.15.178:0/3444278994 wait complete. 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.888+0000 7f2012a34640 10 monclient: build_initial_monmap 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.888+0000 7f2012a34640 10 monclient: monmap: 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:08.906 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.888+0000 7f2012a34640 1 Processor -- start 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.888+0000 7f2012a34640 1 -- start start 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.888+0000 7f2012a34640 10 monclient: init 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 10 monclient: _add_conns ranks=[8,0,3,6,7,2,4,1,5] 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f200c0a2930 0x7f200c096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 10 monclient(hunting): picked mon.noname-e con 0x7f200c0a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f200c132de0 0x7f200c096f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 10 monclient(hunting): picked mon.noname-a con 0x7f200c132de0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f200c1380e0 0x7f200c0974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 10 monclient(hunting): picked mon.noname-g con 0x7f200c1380e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0a2390 con 0x7f200c132de0 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200c098e80 con 0x7f200c1380e0 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f200c098900 con 0x7f200c0a2930 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2012a34640 10 monclient(hunting): authenticate will time out at 733.640515s 2024-09-15T01:32:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f200b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f200c132de0 0x7f200c096f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f200b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f200c132de0 0x7f200c096f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:34994/0 (socket says 172.21.15.178:34994) 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f200b7fe640 1 -- 172.21.15.178:0/2820809855 learned_addr learned my addr 172.21.15.178:0/2820809855 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f200b7fe640 10 monclient(hunting): get_auth_request con 0x7f200c132de0 auth_method 0 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f200b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f200b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f200b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2010fd1640 1 --2- 172.21.15.178:0/2820809855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f200c1380e0 0x7f200c0974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2010fd1640 10 monclient(hunting): get_auth_request con 0x7f200c1380e0 auth_method 0 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2010fd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2010fd1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2010fd1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2010fd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2010fd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f2010fd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f200b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f200b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.889+0000 7f200b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2010fd1640 10 monclient(hunting): handle_auth_done global_id 4350 payload 824 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2010fd1640 1 -- 172.21.15.178:0/2820809855 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f200c0a2930 msgr2=0x7f200c096a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f200bfff640 1 --2- 172.21.15.178:0/2820809855 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f200c0a2930 0x7f200c096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2010fd1640 1 --2- 172.21.15.178:0/2820809855 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f200c0a2930 0x7f200c096a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2010fd1640 1 -- 172.21.15.178:0/2820809855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f200c132de0 msgr2=0x7f200c096f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2010fd1640 1 --2- 172.21.15.178:0/2820809855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f200c132de0 0x7f200c096f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2010fd1640 10 monclient: _finish_hunting 0 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2010fd1640 1 monclient: found mon.noname-g 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2010fd1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:08.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2010fd1640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2010fd1640 1 -- 172.21.15.178:0/2820809855 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2000009450 con 0x7f200c1380e0 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f200b7fe640 1 --2- 172.21.15.178:0/2820809855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f200c132de0 0x7f200c096f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f200bfff640 10 monclient: get_auth_request con 0x7f200c0a2930 auth_method 0 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f200bfff640 1 --2- 172.21.15.178:0/2820809855 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f200c0a2930 0x7f200c096a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2010fd1640 1 --2- 172.21.15.178:0/2820809855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f200c1380e0 0x7f200c0974b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2004007f00 tx=0x7f2004009850 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f1ff3fff640 1 -- 172.21.15.178:0/2820809855 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f200400ee30 con 0x7f200c1380e0 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f1ff3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f1ff3fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f1ff3fff640 10 monclient: dump: 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.911 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f1ff3fff640 10 monclient: _finish_auth 0 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f1ff3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:08.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2012a34640 5 monclient: authenticate success, global_id 4350 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f1ff3fff640 1 -- 172.21.15.178:0/2820809855 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20040098e0 con 0x7f200c1380e0 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f1ff3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f1ff3fff640 1 -- 172.21.15.178:0/2820809855 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2004014d20 con 0x7f200c1380e0 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2012a34640 10 monclient: _renew_subs 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2012a34640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2012a34640 1 -- 172.21.15.178:0/2820809855 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200c09d5c0 con 0x7f200c1380e0 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f200a7fc640 4 set_mon_vals no callback set 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f1ff3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f1ff3fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f1ff3fff640 10 monclient: dump: 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.913 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2012a34640 10 monclient: _renew_subs 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2012a34640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:32:08.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.890+0000 7f2012a34640 1 -- 172.21.15.178:0/2820809855 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200c136e60 con 0x7f200c1380e0 2024-09-15T01:32:08.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.891+0000 7f2012a34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.891+0000 7f2012a34640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.892+0000 7f2012a34640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:32:08.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.892+0000 7f2012a34640 1 -- 172.21.15.178:0/2820809855 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200c0990c0 con 0x7f200c1380e0 2024-09-15T01:32:08.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.892+0000 7f1ff3fff640 1 -- 172.21.15.178:0/2820809855 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2004003020 con 0x7f200c1380e0 2024-09-15T01:32:08.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.892+0000 7f1ff3fff640 1 --2- 172.21.15.178:0/2820809855 >> v2:172.21.15.204:6812/28210 conn(0x7f1ff80397b0 0x7f1ff803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.892+0000 7f1ff3fff640 1 -- 172.21.15.178:0/2820809855 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f200404b3d0 con 0x7f200c1380e0 2024-09-15T01:32:08.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.892+0000 7f200bfff640 1 --2- 172.21.15.178:0/2820809855 >> v2:172.21.15.204:6812/28210 conn(0x7f1ff80397b0 0x7f1ff803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.892+0000 7f200bfff640 10 monclient: get_auth_request con 0x7f1ff80397b0 auth_method 0 2024-09-15T01:32:08.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.893+0000 7f200bfff640 1 --2- 172.21.15.178:0/2820809855 >> v2:172.21.15.204:6812/28210 conn(0x7f1ff80397b0 0x7f1ff803bc60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc001700 tx=0x7f1ffc009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.897+0000 7f1ff3fff640 1 -- 172.21.15.178:0/2820809855 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f200401b690 con 0x7f200c1380e0 2024-09-15T01:32:08.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.897+0000 7f1ff3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.897+0000 7f1ff3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.909+0000 7f7e7b26f640 10 monclient: get_monmap_and_config 2024-09-15T01:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.909+0000 7f7e7b26f640 10 monclient: build_initial_monmap 2024-09-15T01:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.909+0000 7f7e7b26f640 10 monclient: monmap: 2024-09-15T01:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:08.911353+0000 2024-09-15T01:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:08.911353+0000 2024-09-15T01:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.910+0000 7f7e7b26f640 1 Processor -- start 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.910+0000 7f7e7b26f640 1 -- start start 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.910+0000 7f7e7b26f640 10 monclient: init 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 10 monclient: _add_conns ranks=[10,8,4,5,3,1,7,2,0,9,6] 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7e7412f5d0 0x7f7e7412f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 10 monclient(hunting): picked mon.noname-f con 0x7f7e7412f5d0 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 0x7f7e7409d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 10 monclient(hunting): picked mon.noname-e con 0x7f7e7406e510 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7e7406e8e0 0x7f7e7409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 10 monclient(hunting): picked mon.noname-c con 0x7f7e7406e8e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7409f2c0 con 0x7f7e7412f5d0 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7406b850 con 0x7f7e7406e8e0 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7406c740 con 0x7f7e7406e510 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.911+0000 7f7e7b26f640 10 monclient(hunting): authenticate will time out at 733.662981s 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7880a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 0x7f7e7409d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7880a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 0x7f7e7409d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:57960/0 (socket says 172.21.15.178:57960) 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7880a640 1 -- 172.21.15.178:0/3337434611 learned_addr learned my addr 172.21.15.178:0/3337434611 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7880a640 10 monclient(hunting): get_auth_request con 0x7f7e7406e510 auth_method 0 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7880a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7880a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7880a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 1 --2- 172.21.15.178:0/3337434611 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7e7406e8e0 0x7f7e7409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 10 monclient(hunting): get_auth_request con 0x7f7e7406e8e0 auth_method 0 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7880a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7880a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7880a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 10 monclient(hunting): handle_auth_done global_id 4297 payload 274 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 1 -- 172.21.15.178:0/3337434611 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 msgr2=0x7f7e7409d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 1 --2- 172.21.15.178:0/3337434611 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 0x7f7e7409d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 1 -- 172.21.15.178:0/3337434611 >> v2:172.21.15.204:3300/0 conn(0x7f7e7412f5d0 msgr2=0x7f7e7412f9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 1 --2- 172.21.15.178:0/3337434611 >> v2:172.21.15.204:3300/0 conn(0x7f7e7412f5d0 0x7f7e7412f9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 1 monclient: found mon.noname-c 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.912+0000 7f7e7980c640 1 -- 172.21.15.178:0/3337434611 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7404bf60 con 0x7f7e7406e8e0 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.913+0000 7f7e7980c640 1 --2- 172.21.15.178:0/3337434611 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7e7406e8e0 0x7f7e7409dac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009e60 tx=0x7f7e6c009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=3bac60927541f14 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.913+0000 7f7e6bfff640 1 -- 172.21.15.178:0/3337434611 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e6c0079d0 con 0x7f7e7406e8e0 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.913+0000 7f7e6bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.913+0000 7f7e6bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.913+0000 7f7e6bfff640 10 monclient: dump: 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e6bfff640 10 monclient: _finish_auth 0 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e6bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e7b26f640 5 monclient: authenticate success, global_id 4297 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e6bfff640 1 -- 172.21.15.178:0/3337434611 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c02d050 con 0x7f7e7406e8e0 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e7b26f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e6bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e7b26f640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e7b26f640 4 set_mon_vals no callback set 2024-09-15T01:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e6bfff640 1 -- 172.21.15.178:0/3337434611 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e6c007bd0 con 0x7f7e7406e8e0 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e6bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e6bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e6bfff640 10 monclient: dump: 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e7b26f640 10 monclient: shutdown 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e7b26f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e7b26f640 1 -- 172.21.15.178:0/3337434611 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7e7406e8e0 msgr2=0x7f7e7409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.914+0000 7f7e7b26f640 1 --2- 172.21.15.178:0/3337434611 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7e7406e8e0 0x7f7e7409dac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009e60 tx=0x7f7e6c009780 comp rx=0 tx=0).stop 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.915+0000 7f7e7b26f640 1 -- 172.21.15.178:0/3337434611 shutdown_connections 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.915+0000 7f7e7b26f640 1 --2- 172.21.15.178:0/3337434611 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7e7406e8e0 0x7f7e7409dac0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.915+0000 7f7e7b26f640 1 --2- 172.21.15.178:0/3337434611 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 0x7f7e7409d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.915+0000 7f7e7b26f640 1 --2- 172.21.15.178:0/3337434611 >> v2:172.21.15.204:3300/0 conn(0x7f7e7412f5d0 0x7f7e7412f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.915+0000 7f7e7b26f640 1 -- 172.21.15.178:0/3337434611 >> 172.21.15.178:0/3337434611 conn(0x7f7e74068a60 msgr2=0x7f7e740a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.915+0000 7f7e7b26f640 1 -- 172.21.15.178:0/3337434611 shutdown_connections 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.915+0000 7f7e7b26f640 1 -- 172.21.15.178:0/3337434611 wait complete. 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.915+0000 7f7e7b26f640 10 monclient: build_initial_monmap 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.916+0000 7f7e7b26f640 10 monclient: monmap: 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.916+0000 7f7e7b26f640 1 Processor -- start 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.916+0000 7f7e7b26f640 1 -- start start 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.916+0000 7f7e7b26f640 10 monclient: init 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 10 monclient: _add_conns ranks=[8,5,3,2,6,0,4,1,7] 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 0x7f7e7409b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 10 monclient(hunting): picked mon.noname-e con 0x7f7e7406e510 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e7406e8e0 0x7f7e7412e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 10 monclient(hunting): picked mon.noname-h con 0x7f7e7406e8e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e7412f5d0 0x7f7e741191f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 10 monclient(hunting): picked mon.noname-g con 0x7f7e7412f5d0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e74132d90 con 0x7f7e7412f5d0 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7406bb00 con 0x7f7e7406e8e0 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7409ccc0 con 0x7f7e7406e510 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7b26f640 10 monclient(hunting): authenticate will time out at 733.668668s 2024-09-15T01:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7900b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 0x7f7e7409b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7900b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 0x7f7e7409b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:57970/0 (socket says 172.21.15.178:57970) 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7900b640 1 -- 172.21.15.178:0/317668489 learned_addr learned my addr 172.21.15.178:0/317668489 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7900b640 10 monclient(hunting): get_auth_request con 0x7f7e7406e510 auth_method 0 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7900b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7900b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7900b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7880a640 1 --2- 172.21.15.178:0/317668489 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e7406e8e0 0x7f7e7412e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7880a640 10 monclient(hunting): get_auth_request con 0x7f7e7406e8e0 auth_method 0 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7880a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7880a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7880a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7980c640 1 --2- 172.21.15.178:0/317668489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e7412f5d0 0x7f7e741191f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7900b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7900b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7900b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.917+0000 7f7e7980c640 10 monclient(hunting): get_auth_request con 0x7f7e7412f5d0 auth_method 0 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7980c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7980c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7980c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7980c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7980c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7980c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 10 monclient(hunting): handle_auth_done global_id 4379 payload 824 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 1 -- 172.21.15.178:0/317668489 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 msgr2=0x7f7e7409b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 1 --2- 172.21.15.178:0/317668489 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 0x7f7e7409b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 1 -- 172.21.15.178:0/317668489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e7412f5d0 msgr2=0x7f7e741191f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 1 --2- 172.21.15.178:0/317668489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e7412f5d0 0x7f7e741191f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 10 monclient: _finish_hunting 0 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 1 monclient: found mon.noname-h 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 1 -- 172.21.15.178:0/317668489 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c009a20 con 0x7f7e7406e8e0 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7900b640 1 --2- 172.21.15.178:0/317668489 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 0x7f7e7409b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7980c640 1 --2- 172.21.15.178:0/317668489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e7412f5d0 0x7f7e741191f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7880a640 1 --2- 172.21.15.178:0/317668489 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e7406e8e0 0x7f7e7412e3a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400ee10 tx=0x7f7e6400b460 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e697fa640 1 -- 172.21.15.178:0/317668489 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e6400be00 con 0x7f7e7406e8e0 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e697fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e697fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e697fa640 10 monclient: dump: 2024-09-15T01:32:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.945 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e697fa640 10 monclient: _finish_auth 0 2024-09-15T01:32:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7b26f640 5 monclient: authenticate success, global_id 4379 2024-09-15T01:32:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e697fa640 1 -- 172.21.15.178:0/317668489 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e640108c0 con 0x7f7e7406e8e0 2024-09-15T01:32:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e697fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e697fa640 1 -- 172.21.15.178:0/317668489 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e64018400 con 0x7f7e7406e8e0 2024-09-15T01:32:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7b26f640 10 monclient: _renew_subs 2024-09-15T01:32:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7b26f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:32:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7b26f640 1 -- 172.21.15.178:0/317668489 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7411a560 con 0x7f7e7406e8e0 2024-09-15T01:32:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e6b7fe640 4 set_mon_vals no callback set 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e697fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e697fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e697fa640 10 monclient: dump: 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7b26f640 10 monclient: _renew_subs 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7b26f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.918+0000 7f7e7b26f640 1 -- 172.21.15.178:0/317668489 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7411b4c0 con 0x7f7e7406e8e0 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.920+0000 7f7e5effd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.920+0000 7f7e5effd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.920+0000 7f7e5effd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.920+0000 7f7e5effd640 1 -- 172.21.15.178:0/317668489 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4c005010 con 0x7f7e7406e8e0 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.920+0000 7f7e697fa640 1 -- 172.21.15.178:0/317668489 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7e64010440 con 0x7f7e7406e8e0 2024-09-15T01:32:08.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.921+0000 7f7e697fa640 1 --2- 172.21.15.178:0/317668489 >> v2:172.21.15.204:6812/28210 conn(0x7f7e580397b0 0x7f7e5803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.921+0000 7f7e697fa640 1 -- 172.21.15.178:0/317668489 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f7e6404b710 con 0x7f7e7406e8e0 2024-09-15T01:32:08.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.921+0000 7f7e7900b640 1 --2- 172.21.15.178:0/317668489 >> v2:172.21.15.204:6812/28210 conn(0x7f7e580397b0 0x7f7e5803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.921+0000 7f7e7900b640 10 monclient: get_auth_request con 0x7f7e580397b0 auth_method 0 2024-09-15T01:32:08.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.921+0000 7f7e7900b640 1 --2- 172.21.15.178:0/317668489 >> v2:172.21.15.204:6812/28210 conn(0x7f7e580397b0 0x7f7e5803bc60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009f60 tx=0x7f7e70009060 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.923+0000 7f735b700640 10 monclient: get_monmap_and_config 2024-09-15T01:32:08.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.923+0000 7f735b700640 10 monclient: build_initial_monmap 2024-09-15T01:32:08.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.924+0000 7f735b700640 10 monclient: monmap: 2024-09-15T01:32:08.949 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:08.949 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.949 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:08.925857+0000 2024-09-15T01:32:08.949 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:08.925857+0000 2024-09-15T01:32:08.949 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:08.949 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.949 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:08.949 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:08.949 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:08.949 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:08.950 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:08.950 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:08.950 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:08.950 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:08.950 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:08.950 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:08.950 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:08.950 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.924+0000 7f735b700640 1 Processor -- start 2024-09-15T01:32:08.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.925+0000 7f735b700640 1 -- start start 2024-09-15T01:32:08.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.925+0000 7f735b700640 10 monclient: init 2024-09-15T01:32:08.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.925+0000 7f7e697fa640 1 -- 172.21.15.178:0/317668489 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7e64016530 con 0x7f7e7406e8e0 2024-09-15T01:32:08.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.925+0000 7f7e697fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.925+0000 7f7e697fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:08.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f735b700640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:08.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f735b700640 10 monclient: _add_conns ranks=[6,10,7,4,1,3,2,0,9,5,8] 2024-09-15T01:32:08.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f735b700640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735412f5d0 0x7f735412f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f735b700640 10 monclient(hunting): picked mon.noname-d con 0x7f735412f5d0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:08.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f735b700640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f735406e510 0x7f735409d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f735b700640 10 monclient(hunting): picked mon.noname-f con 0x7f735406e510 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f735b700640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f735406e8e0 0x7f735409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f735949c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735412f5d0 0x7f735412f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f735949c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735412f5d0 0x7f735412f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:40492/0 (socket says 172.21.15.178:40492) 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f735b700640 10 monclient(hunting): picked mon.noname-i con 0x7f735406e8e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f735949c640 1 -- 172.21.15.178:0/329575589 learned_addr learned my addr 172.21.15.178:0/329575589 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f7358c9b640 1 --2- 172.21.15.178:0/329575589 >> v2:172.21.15.204:3300/0 conn(0x7f735406e510 0x7f735409d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f735b700640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.926+0000 7f7359c9d640 1 --2- 172.21.15.178:0/329575589 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f735406e8e0 0x7f735409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735b700640 1 -- 172.21.15.178:0/329575589 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735409f2c0 con 0x7f735406e510 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735b700640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735b700640 1 -- 172.21.15.178:0/329575589 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f735406b850 con 0x7f735412f5d0 2024-09-15T01:32:08.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735b700640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735b700640 1 -- 172.21.15.178:0/329575589 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f735406c740 con 0x7f735406e8e0 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735b700640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735b700640 10 monclient(hunting): authenticate will time out at 733.678434s 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 10 monclient(hunting): get_auth_request con 0x7f735412f5d0 auth_method 0 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7358c9b640 10 monclient(hunting): get_auth_request con 0x7f735406e510 auth_method 0 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7358c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7358c9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7358c9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7359c9d640 10 monclient(hunting): get_auth_request con 0x7f735406e8e0 auth_method 0 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7359c9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7359c9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7359c9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7358c9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7358c9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7358c9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7359c9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7359c9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7359c9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 10 monclient(hunting): handle_auth_done global_id 4398 payload 274 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 1 -- 172.21.15.178:0/329575589 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f735406e8e0 msgr2=0x7f735409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:08.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 1 --2- 172.21.15.178:0/329575589 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f735406e8e0 0x7f735409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 1 -- 172.21.15.178:0/329575589 >> v2:172.21.15.204:3300/0 conn(0x7f735406e510 msgr2=0x7f735409d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 1 --2- 172.21.15.178:0/329575589 >> v2:172.21.15.204:3300/0 conn(0x7f735406e510 0x7f735409d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 1 monclient: found mon.noname-d 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f735949c640 1 -- 172.21.15.178:0/329575589 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735404bf60 con 0x7f735412f5d0 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.927+0000 7f7358c9b640 1 --2- 172.21.15.178:0/329575589 >> v2:172.21.15.204:3300/0 conn(0x7f735406e510 0x7f735409d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f735949c640 1 --2- 172.21.15.178:0/329575589 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735412f5d0 0x7f735412f9a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7350009910 tx=0x7f73500091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=9a5cc27b1657926f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f734bfff640 1 -- 172.21.15.178:0/329575589 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f735002ac80 con 0x7f735412f5d0 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f734bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f734bfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:32:08.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f734bfff640 10 monclient: dump: 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.968 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f734bfff640 10 monclient: _finish_auth 0 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f734bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f734bfff640 1 -- 172.21.15.178:0/329575589 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735002ae00 con 0x7f735412f5d0 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f734bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f735b700640 5 monclient: authenticate success, global_id 4398 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f735b700640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f735b700640 4 set_mon_vals no callback set 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f734bfff640 1 -- 172.21.15.178:0/329575589 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7350031820 con 0x7f735412f5d0 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f734bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f734bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.929+0000 7f734bfff640 10 monclient: dump: 2024-09-15T01:32:08.969 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.930+0000 7f735b700640 10 monclient: shutdown 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.930+0000 7f735b700640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.930+0000 7f735b700640 1 -- 172.21.15.178:0/329575589 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735412f5d0 msgr2=0x7f735412f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.930+0000 7f735b700640 1 --2- 172.21.15.178:0/329575589 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735412f5d0 0x7f735412f9a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7350009910 tx=0x7f73500091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.930+0000 7f735b700640 1 -- 172.21.15.178:0/329575589 shutdown_connections 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.930+0000 7f735b700640 1 --2- 172.21.15.178:0/329575589 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f735406e8e0 0x7f735409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.930+0000 7f735b700640 1 --2- 172.21.15.178:0/329575589 >> v2:172.21.15.204:3300/0 conn(0x7f735406e510 0x7f735409d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.930+0000 7f735b700640 1 --2- 172.21.15.178:0/329575589 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735412f5d0 0x7f735412f9a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.930+0000 7f735b700640 1 -- 172.21.15.178:0/329575589 >> 172.21.15.178:0/329575589 conn(0x7f7354068a60 msgr2=0x7f73540a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.930+0000 7f735b700640 1 -- 172.21.15.178:0/329575589 shutdown_connections 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.930+0000 7f735b700640 1 -- 172.21.15.178:0/329575589 wait complete. 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.931+0000 7f735b700640 10 monclient: build_initial_monmap 2024-09-15T01:32:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.931+0000 7f735b700640 10 monclient: monmap: 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:08.972 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.932+0000 7f735b700640 1 Processor -- start 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.932+0000 7f735b700640 1 -- start start 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.932+0000 7f735b700640 10 monclient: init 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.933+0000 7f735b700640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.933+0000 7f735b700640 10 monclient: _add_conns ranks=[2,6,4,1,0,3,8,7,5] 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.933+0000 7f735b700640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f735406e510 0x7f735413d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.933+0000 7f735b700640 10 monclient(hunting): picked mon.noname-b con 0x7f735406e510 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.933+0000 7f735b700640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735406e8e0 0x7f735412e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.933+0000 7f735949c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f735406e510 0x7f735413d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.933+0000 7f735949c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f735406e510 0x7f735413d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:33690/0 (socket says 172.21.15.178:33690) 2024-09-15T01:32:08.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.933+0000 7f7358c9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735406e8e0 0x7f735412e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.933+0000 7f7358c9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735406e8e0 0x7f735412e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:40494/0 (socket says 172.21.15.178:40494) 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.933+0000 7f735b700640 10 monclient(hunting): picked mon.noname-d con 0x7f735406e8e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.933+0000 7f735949c640 1 -- 172.21.15.178:0/1149952459 learned_addr learned my addr 172.21.15.178:0/1149952459 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735b700640 1 --2- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f735412f5d0 0x7f7354119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735b700640 10 monclient(hunting): picked mon.noname-c con 0x7f735412f5d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735b700640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735409ccc0 con 0x7f735406e510 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735b700640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f735411d9d0 con 0x7f735412f5d0 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735b700640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f735413e600 con 0x7f735406e8e0 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735b700640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735b700640 10 monclient(hunting): authenticate will time out at 733.685276s 2024-09-15T01:32:08.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 10 monclient(hunting): get_auth_request con 0x7f735406e510 auth_method 0 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f7359c9d640 1 --2- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f735412f5d0 0x7f7354119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f7358c9b640 10 monclient(hunting): get_auth_request con 0x7f735406e8e0 auth_method 0 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f7358c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f7358c9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f7358c9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f7358c9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:08.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f7358c9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f7358c9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f7359c9d640 10 monclient(hunting): get_auth_request con 0x7f735412f5d0 auth_method 0 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f7359c9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f7359c9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f7359c9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 10 monclient(hunting): handle_auth_done global_id 4340 payload 824 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 1 -- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735406e8e0 msgr2=0x7f735412e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 1 --2- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735406e8e0 0x7f735412e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 1 -- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f735412f5d0 msgr2=0x7f7354119ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 1 --2- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f735412f5d0 0x7f7354119ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 1 monclient: found mon.noname-b 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:08.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.934+0000 7f735949c640 1 -- 172.21.15.178:0/1149952459 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7344009390 con 0x7f735406e510 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f7359c9d640 1 --2- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f735412f5d0 0x7f7354119ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f7358c9b640 1 --2- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735406e8e0 0x7f735412e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f735949c640 1 --2- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f735406e510 0x7f735413d790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7350035040 tx=0x7f7350032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f73497fa640 1 -- 172.21.15.178:0/1149952459 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f735002ac80 con 0x7f735406e510 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f73497fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f73497fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f73497fa640 10 monclient: dump: 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.977 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f73497fa640 10 monclient: _finish_auth 0 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f73497fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:08.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f735b700640 5 monclient: authenticate success, global_id 4340 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f73497fa640 1 -- 172.21.15.178:0/1149952459 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735002a790 con 0x7f735406e510 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f73497fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f73497fa640 1 -- 172.21.15.178:0/1149952459 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7350031ca0 con 0x7f735406e510 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f735b700640 10 monclient: _renew_subs 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f735b700640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7350009450 con 0x7f735406e510 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f734b7fe640 4 set_mon_vals no callback set 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f73497fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f73497fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f73497fa640 10 monclient: dump: 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:08.979 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:08.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f735b700640 10 monclient: _renew_subs 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f735b700640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.935+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735411bb30 con 0x7f735406e510 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.937+0000 7f735b700640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.937+0000 7f73497fa640 1 -- 172.21.15.178:0/1149952459 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7350039040 con 0x7f735406e510 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.937+0000 7f735b700640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.937+0000 7f735b700640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.937+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735411e0b0 con 0x7f735406e510 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.937+0000 7f73497fa640 1 --2- 172.21.15.178:0/1149952459 >> v2:172.21.15.204:6812/28210 conn(0x7f73380397b0 0x7f733803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.937+0000 7f73497fa640 1 -- 172.21.15.178:0/1149952459 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f735006a680 con 0x7f735406e510 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.937+0000 7f7358c9b640 1 --2- 172.21.15.178:0/1149952459 >> v2:172.21.15.204:6812/28210 conn(0x7f73380397b0 0x7f733803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.937+0000 7f7358c9b640 10 monclient: get_auth_request con 0x7f73380397b0 auth_method 0 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.938+0000 7f7358c9b640 1 --2- 172.21.15.178:0/1149952459 >> v2:172.21.15.204:6812/28210 conn(0x7f73380397b0 0x7f733803bc60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f73540a35a0 tx=0x7f7344016040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:08.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.946+0000 7f73497fa640 1 -- 172.21.15.178:0/1149952459 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f735411e0b0 con 0x7f735406e510 2024-09-15T01:32:08.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.946+0000 7f73497fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:08.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.946+0000 7f73497fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:08.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.951+0000 7feac991d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T01:32:08.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.952+0000 7feac991d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T01:32:08.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.952+0000 7feac991d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:08.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.952+0000 7feac991d640 1 -- 172.21.15.178:0/972844153 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7feac405ed70 con 0x7feac412f4c0 2024-09-15T01:32:08.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.952+0000 7feaaaffd640 1 -- 172.21.15.178:0/972844153 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feab400e290 con 0x7feac412f4c0 2024-09-15T01:32:08.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.952+0000 7feaaaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T01:32:08.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.952+0000 7feaaaffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:08.983 INFO:teuthology.orchestra.run.smithi178.stdout:47244640258 2024-09-15T01:32:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.955+0000 7feac991d640 1 -- 172.21.15.178:0/972844153 >> v2:172.21.15.204:6812/28210 conn(0x7feab00397b0 msgr2=0x7feab003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.955+0000 7feac991d640 1 --2- 172.21.15.178:0/972844153 >> v2:172.21.15.204:6812/28210 conn(0x7feab00397b0 0x7feab003bc60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feabc009420 tx=0x7feabc032000 comp rx=0 tx=0).stop 2024-09-15T01:32:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.955+0000 7feac991d640 10 monclient: shutdown 2024-09-15T01:32:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.955+0000 7feac991d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.955+0000 7feac991d640 1 -- 172.21.15.178:0/972844153 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feac412f4c0 msgr2=0x7feac4142450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.955+0000 7feac991d640 1 --2- 172.21.15.178:0/972844153 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feac412f4c0 0x7feac4142450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feab400c520 tx=0x7feab4007eb0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.956+0000 7feac991d640 1 -- 172.21.15.178:0/972844153 shutdown_connections 2024-09-15T01:32:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.956+0000 7feac991d640 1 --2- 172.21.15.178:0/972844153 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feac412f4c0 0x7feac4142450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.956+0000 7feac991d640 1 --2- 172.21.15.178:0/972844153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feac409ead0 0x7feac4141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.956+0000 7feac991d640 1 --2- 172.21.15.178:0/972844153 >> v2:172.21.15.204:6812/28210 conn(0x7feab00397b0 0x7feab003bc60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.956+0000 7feac991d640 1 --2- 172.21.15.178:0/972844153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feac409e700 0x7feac41419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.956+0000 7feac991d640 1 -- 172.21.15.178:0/972844153 >> 172.21.15.178:0/972844153 conn(0x7feac40a4390 msgr2=0x7feac40a24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:08.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.956+0000 7feac991d640 1 -- 172.21.15.178:0/972844153 shutdown_connections 2024-09-15T01:32:08.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.956+0000 7feac991d640 1 -- 172.21.15.178:0/972844153 wait complete. 2024-09-15T01:32:08.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.959+0000 7f2ac9860640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T01:32:08.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.959+0000 7f2ac9860640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T01:32:08.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.959+0000 7f2ac9860640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:32:08.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.959+0000 7f2ac9860640 1 -- 172.21.15.178:0/1607424138 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2ac404b720 con 0x7f2ac40a08c0 2024-09-15T01:32:08.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.960+0000 7f2aaaffd640 1 -- 172.21.15.178:0/1607424138 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2ab8037090 con 0x7f2ac40a08c0 2024-09-15T01:32:08.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.960+0000 7f2aaaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T01:32:08.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.960+0000 7f2aaaffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:08.987 INFO:teuthology.orchestra.run.smithi178.stdout:47244640258 2024-09-15T01:32:08.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.963+0000 7f2ac9860640 1 -- 172.21.15.178:0/1607424138 >> v2:172.21.15.204:6812/28210 conn(0x7f2ab0039710 msgr2=0x7f2ab003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.963+0000 7f2ac9860640 1 --2- 172.21.15.178:0/1607424138 >> v2:172.21.15.204:6812/28210 conn(0x7f2ab0039710 0x7f2ab003bbc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4010370 tx=0x7f2ab4008040 comp rx=0 tx=0).stop 2024-09-15T01:32:08.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.963+0000 7f2ac9860640 10 monclient: shutdown 2024-09-15T01:32:08.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.963+0000 7f2ac9860640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:08.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.963+0000 7f2ac9860640 1 -- 172.21.15.178:0/1607424138 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2ac40a08c0 msgr2=0x7f2ac41317c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:08.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.963+0000 7f2ac9860640 1 --2- 172.21.15.178:0/1607424138 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2ac40a08c0 0x7f2ac41317c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40a2e40 tx=0x7f2ab8032000 comp rx=0 tx=0).stop 2024-09-15T01:32:08.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.964+0000 7f2ac9860640 1 -- 172.21.15.178:0/1607424138 shutdown_connections 2024-09-15T01:32:08.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.964+0000 7f2ac9860640 1 --2- 172.21.15.178:0/1607424138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ac40a2930 0x7f2ac4131d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.964+0000 7f2ac9860640 1 --2- 172.21.15.178:0/1607424138 >> v2:172.21.15.204:6812/28210 conn(0x7f2ab0039710 0x7f2ab003bbc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.964+0000 7f2ac9860640 1 --2- 172.21.15.178:0/1607424138 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2ac40a08c0 0x7f2ac41317c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.964+0000 7f2ac9860640 1 --2- 172.21.15.178:0/1607424138 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2ac409faa0 0x7f2ac409b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:08.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.964+0000 7f2ac9860640 1 -- 172.21.15.178:0/1607424138 >> 172.21.15.178:0/1607424138 conn(0x7f2ac40a5ed0 msgr2=0x7f2ac4135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:08.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.964+0000 7f2ac9860640 1 -- 172.21.15.178:0/1607424138 shutdown_connections 2024-09-15T01:32:08.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:08.964+0000 7f2ac9860640 1 -- 172.21.15.178:0/1607424138 wait complete. 2024-09-15T01:32:08.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.1 2024-09-15T01:32:08.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.0 2024-09-15T01:32:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.076+0000 7f2012a34640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T01:32:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.076+0000 7f2012a34640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T01:32:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.076+0000 7f2012a34640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:32:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.076+0000 7f2012a34640 1 -- 172.21.15.178:0/2820809855 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f200c04b720 con 0x7f200c1380e0 2024-09-15T01:32:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.076+0000 7f1ff3fff640 1 -- 172.21.15.178:0/2820809855 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f200401b690 con 0x7f200c1380e0 2024-09-15T01:32:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.076+0000 7f1ff3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T01:32:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.076+0000 7f1ff3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:09.080 INFO:teuthology.orchestra.run.smithi178.stdout:47244640258 2024-09-15T01:32:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.078+0000 7f2012a34640 1 -- 172.21.15.178:0/2820809855 >> v2:172.21.15.204:6812/28210 conn(0x7f1ff80397b0 msgr2=0x7f1ff803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.078+0000 7f2012a34640 1 --2- 172.21.15.178:0/2820809855 >> v2:172.21.15.204:6812/28210 conn(0x7f1ff80397b0 0x7f1ff803bc60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc001700 tx=0x7f1ffc009040 comp rx=0 tx=0).stop 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.078+0000 7f2012a34640 10 monclient: shutdown 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.078+0000 7f2012a34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.078+0000 7f2012a34640 1 -- 172.21.15.178:0/2820809855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f200c1380e0 msgr2=0x7f200c0974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.078+0000 7f2012a34640 1 --2- 172.21.15.178:0/2820809855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f200c1380e0 0x7f200c0974b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2004007f00 tx=0x7f2004009850 comp rx=0 tx=0).stop 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.079+0000 7f2012a34640 1 -- 172.21.15.178:0/2820809855 shutdown_connections 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.079+0000 7f2012a34640 1 --2- 172.21.15.178:0/2820809855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f200c1380e0 0x7f200c0974b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.079+0000 7f2012a34640 1 --2- 172.21.15.178:0/2820809855 >> v2:172.21.15.204:6812/28210 conn(0x7f1ff80397b0 0x7f1ff803bc60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.079+0000 7f2012a34640 1 --2- 172.21.15.178:0/2820809855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f200c132de0 0x7f200c096f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.079+0000 7f2012a34640 1 --2- 172.21.15.178:0/2820809855 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f200c0a2930 0x7f200c096a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.079+0000 7f2012a34640 1 -- 172.21.15.178:0/2820809855 >> 172.21.15.178:0/2820809855 conn(0x7f200c0a5ed0 msgr2=0x7f200c0a48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.079+0000 7f2012a34640 1 -- 172.21.15.178:0/2820809855 shutdown_connections 2024-09-15T01:32:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.079+0000 7f2012a34640 1 -- 172.21.15.178:0/2820809855 wait complete. 2024-09-15T01:32:09.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.4 2024-09-15T01:32:09.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.095+0000 7f711799d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T01:32:09.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.095+0000 7f711799d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T01:32:09.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.095+0000 7f711799d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:32:09.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.095+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f711009cef0 con 0x7f7110132de0 2024-09-15T01:32:09.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.095+0000 7f71057fa640 1 -- 172.21.15.178:0/497167695 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7100037020 con 0x7f7110132de0 2024-09-15T01:32:09.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.095+0000 7f71057fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T01:32:09.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.095+0000 7f71057fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:09.098 INFO:teuthology.orchestra.run.smithi178.stdout:47244640260 2024-09-15T01:32:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 >> v2:172.21.15.204:6812/28210 conn(0x7f70f40397e0 msgr2=0x7f70f403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 1 --2- 172.21.15.178:0/497167695 >> v2:172.21.15.204:6812/28210 conn(0x7f70f40397e0 0x7f70f403bc90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f710c001800 tx=0x7f710c009040 comp rx=0 tx=0).stop 2024-09-15T01:32:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 10 monclient: shutdown 2024-09-15T01:32:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7110132de0 msgr2=0x7f7110096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7110132de0 0x7f7110096a30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7100031000 tx=0x7f7100032000 comp rx=0 tx=0).stop 2024-09-15T01:32:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 shutdown_connections 2024-09-15T01:32:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 1 --2- 172.21.15.178:0/497167695 >> v2:172.21.15.204:6812/28210 conn(0x7f70f40397e0 0x7f70f403bc90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71101380e0 0x7f7110096f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7110132de0 0x7f7110096a30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 1 --2- 172.21.15.178:0/497167695 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f71100a2930 0x7f711009c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 >> 172.21.15.178:0/497167695 conn(0x7f71100a5ed0 msgr2=0x7f7110135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 shutdown_connections 2024-09-15T01:32:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.097+0000 7f711799d640 1 -- 172.21.15.178:0/497167695 wait complete. 2024-09-15T01:32:09.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640260 for osd.5 2024-09-15T01:32:09.110 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:09.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.125+0000 7f735b700640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T01:32:09.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.125+0000 7f735b700640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T01:32:09.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.125+0000 7f735b700640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:09.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.125+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f735411e0b0 con 0x7f735406e510 2024-09-15T01:32:09.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.125+0000 7f73497fa640 1 -- 172.21.15.178:0/1149952459 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f735411e0b0 con 0x7f735406e510 2024-09-15T01:32:09.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.125+0000 7f73497fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T01:32:09.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.125+0000 7f73497fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:09.129 INFO:teuthology.orchestra.run.smithi178.stdout:47244640260 2024-09-15T01:32:09.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.127+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 >> v2:172.21.15.204:6812/28210 conn(0x7f73380397b0 msgr2=0x7f733803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:09.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.127+0000 7f735b700640 1 --2- 172.21.15.178:0/1149952459 >> v2:172.21.15.204:6812/28210 conn(0x7f73380397b0 0x7f733803bc60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f73540a35a0 tx=0x7f7344016040 comp rx=0 tx=0).stop 2024-09-15T01:32:09.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.127+0000 7f735b700640 10 monclient: shutdown 2024-09-15T01:32:09.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.127+0000 7f735b700640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:09.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.127+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f735406e510 msgr2=0x7f735413d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:09.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.127+0000 7f735b700640 1 --2- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f735406e510 0x7f735413d790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7350035040 tx=0x7f7350032000 comp rx=0 tx=0).stop 2024-09-15T01:32:09.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.127+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 shutdown_connections 2024-09-15T01:32:09.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.127+0000 7f735b700640 1 --2- 172.21.15.178:0/1149952459 >> v2:172.21.15.204:6812/28210 conn(0x7f73380397b0 0x7f733803bc60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.127+0000 7f735b700640 1 --2- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f735412f5d0 0x7f7354119ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.127+0000 7f735b700640 1 --2- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f735406e8e0 0x7f735412e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.127+0000 7f735b700640 1 --2- 172.21.15.178:0/1149952459 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f735406e510 0x7f735413d790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.127+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 >> 172.21.15.178:0/1149952459 conn(0x7f7354068a60 msgr2=0x7f73540a0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:09.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.128+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 shutdown_connections 2024-09-15T01:32:09.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.128+0000 7f735b700640 1 -- 172.21.15.178:0/1149952459 wait complete. 2024-09-15T01:32:09.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.135+0000 7f7e5effd640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T01:32:09.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.135+0000 7f7e5effd640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T01:32:09.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.135+0000 7f7e5effd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:32:09.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.135+0000 7f7e5effd640 1 -- 172.21.15.178:0/317668489 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7e4c0052c0 con 0x7f7e7406e8e0 2024-09-15T01:32:09.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.135+0000 7f7e697fa640 1 -- 172.21.15.178:0/317668489 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7e6401d020 con 0x7f7e7406e8e0 2024-09-15T01:32:09.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.135+0000 7f7e697fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T01:32:09.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.135+0000 7f7e697fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:09.139 INFO:teuthology.orchestra.run.smithi178.stdout:47244640260 2024-09-15T01:32:09.141 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640260 for osd.2 2024-09-15T01:32:09.141 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:09.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 1 -- 172.21.15.178:0/317668489 >> v2:172.21.15.204:6812/28210 conn(0x7f7e580397b0 msgr2=0x7f7e5803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:09.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 1 --2- 172.21.15.178:0/317668489 >> v2:172.21.15.204:6812/28210 conn(0x7f7e580397b0 0x7f7e5803bc60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009f60 tx=0x7f7e70009060 comp rx=0 tx=0).stop 2024-09-15T01:32:09.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 10 monclient: shutdown 2024-09-15T01:32:09.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:09.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 1 -- 172.21.15.178:0/317668489 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e7406e8e0 msgr2=0x7f7e7412e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:09.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 1 --2- 172.21.15.178:0/317668489 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e7406e8e0 0x7f7e7412e3a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400ee10 tx=0x7f7e6400b460 comp rx=0 tx=0).stop 2024-09-15T01:32:09.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 1 -- 172.21.15.178:0/317668489 shutdown_connections 2024-09-15T01:32:09.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 1 --2- 172.21.15.178:0/317668489 >> v2:172.21.15.204:6812/28210 conn(0x7f7e580397b0 0x7f7e5803bc60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 1 --2- 172.21.15.178:0/317668489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e7412f5d0 0x7f7e741191f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 1 --2- 172.21.15.178:0/317668489 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e7406e8e0 0x7f7e7412e3a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 1 --2- 172.21.15.178:0/317668489 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7e7406e510 0x7f7e7409b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:09.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 1 -- 172.21.15.178:0/317668489 >> 172.21.15.178:0/317668489 conn(0x7f7e74068a60 msgr2=0x7f7e740a0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:09.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 1 -- 172.21.15.178:0/317668489 shutdown_connections 2024-09-15T01:32:09.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:09.138+0000 7f7e5effd640 1 -- 172.21.15.178:0/317668489 wait complete. 2024-09-15T01:32:09.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640260 for osd.3 2024-09-15T01:32:09.151 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:09.991 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T01:32:09.991 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T01:32:10.095 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T01:32:10.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.099+0000 7f28660e8640 10 monclient: get_monmap_and_config 2024-09-15T01:32:10.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.099+0000 7f28660e8640 10 monclient: build_initial_monmap 2024-09-15T01:32:10.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.099+0000 7f28660e8640 10 monclient: monmap: 2024-09-15T01:32:10.102 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:10.102 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.102 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:10.100994+0000 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:10.100994+0000 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:10.103 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:10.104 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.099+0000 7f6ba8e4f640 10 monclient: get_monmap_and_config 2024-09-15T01:32:10.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.099+0000 7f6ba8e4f640 10 monclient: build_initial_monmap 2024-09-15T01:32:10.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.099+0000 7f6ba8e4f640 10 monclient: monmap: 2024-09-15T01:32:10.104 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:10.104 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.104 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:10.101468+0000 2024-09-15T01:32:10.104 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:10.101468+0000 2024-09-15T01:32:10.104 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:10.105 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.105 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:10.105 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:10.105 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:10.105 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:10.105 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:10.105 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:10.105 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:10.105 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:10.106 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:10.106 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:10.106 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:10.106 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.099+0000 7f28660e8640 1 Processor -- start 2024-09-15T01:32:10.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 1 -- start start 2024-09-15T01:32:10.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 10 monclient: init 2024-09-15T01:32:10.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f6ba8e4f640 1 Processor -- start 2024-09-15T01:32:10.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f6ba8e4f640 1 -- start start 2024-09-15T01:32:10.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f6ba8e4f640 10 monclient: init 2024-09-15T01:32:10.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:10.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 10 monclient: _add_conns ranks=[5,4,6,0,3,8,2,1,9,7,10] 2024-09-15T01:32:10.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f286009ccd0 0x7f2860097ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 10 monclient(hunting): picked mon.noname-h con 0x7f286009ccd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:10.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f286009d0a0 0x7f286009d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 10 monclient(hunting): picked mon.noname-c con 0x7f286009d0a0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:10.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2860136ed0 0x7f28601392b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 10 monclient(hunting): picked mon.noname-d con 0x7f2860136ed0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:10.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f286009dfe0 con 0x7f286009d0a0 2024-09-15T01:32:10.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28600975a0 con 0x7f286009ccd0 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f28600972c0 con 0x7f2860136ed0 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f28660e8640 10 monclient(hunting): authenticate will time out at 734.851555s 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285ffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2860136ed0 0x7f28601392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285effd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f286009d0a0 0x7f286009d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285ffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2860136ed0 0x7f28601392b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:40500/0 (socket says 172.21.15.178:40500) 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285ffff640 1 -- 172.21.15.178:0/2859016271 learned_addr learned my addr 172.21.15.178:0/2859016271 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285ffff640 10 monclient(hunting): get_auth_request con 0x7f2860136ed0 auth_method 0 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285f7fe640 1 --2- 172.21.15.178:0/2859016271 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f286009ccd0 0x7f2860097ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285effd640 10 monclient(hunting): get_auth_request con 0x7f286009d0a0 auth_method 0 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285f7fe640 10 monclient(hunting): get_auth_request con 0x7f286009ccd0 auth_method 0 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.100+0000 7f285effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:10.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 10 monclient: _add_conns ranks=[5,4,7,1,9,6,10,0,8,3,2] 2024-09-15T01:32:10.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f285ffff640 10 monclient(hunting): handle_auth_done global_id 4416 payload 274 2024-09-15T01:32:10.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f285ffff640 1 -- 172.21.15.178:0/2859016271 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f286009ccd0 msgr2=0x7f2860097ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:10.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ba4096a30 0x7f6ba412fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 10 monclient(hunting): picked mon.noname-h con 0x7f6ba4096a30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:10.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f285ffff640 1 --2- 172.21.15.178:0/2859016271 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f286009ccd0 0x7f2860097ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f285ffff640 1 -- 172.21.15.178:0/2859016271 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f286009d0a0 msgr2=0x7f286009d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:10.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f285ffff640 1 --2- 172.21.15.178:0/2859016271 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f286009d0a0 0x7f286009d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f285ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:32:10.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f285ffff640 1 monclient: found mon.noname-d 2024-09-15T01:32:10.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6ba412fff0 0x7f6ba41303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 10 monclient(hunting): picked mon.noname-c con 0x7f6ba412fff0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:10.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ba4096e00 0x7f6ba4097270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f285ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:10.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f285ffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:32:10.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f285ffff640 1 -- 172.21.15.178:0/2859016271 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286004bf60 con 0x7f2860136ed0 2024-09-15T01:32:10.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 10 monclient(hunting): picked mon.noname-i con 0x7f6ba4096e00 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:10.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40997c0 con 0x7f6ba412fff0 2024-09-15T01:32:10.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba4098640 con 0x7f6ba4096a30 2024-09-15T01:32:10.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40982d0 con 0x7f6ba4096e00 2024-09-15T01:32:10.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:10.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba8e4f640 10 monclient(hunting): authenticate will time out at 734.852185s 2024-09-15T01:32:10.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f285effd640 1 --2- 172.21.15.178:0/2859016271 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f286009d0a0 0x7f286009d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:10.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ba4096a30 0x7f6ba412fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ba4096a30 0x7f6ba412fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:48110/0 (socket says 172.21.15.178:48110) 2024-09-15T01:32:10.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba1d9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6ba412fff0 0x7f6ba41303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 1 -- 172.21.15.178:0/3202640187 learned_addr learned my addr 172.21.15.178:0/3202640187 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:10.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 10 monclient(hunting): get_auth_request con 0x7f6ba4096a30 auth_method 0 2024-09-15T01:32:10.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba2d9d640 1 --2- 172.21.15.178:0/3202640187 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ba4096e00 0x7f6ba4097270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba2d9d640 10 monclient(hunting): get_auth_request con 0x7f6ba4096e00 auth_method 0 2024-09-15T01:32:10.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba1d9b640 10 monclient(hunting): get_auth_request con 0x7f6ba412fff0 auth_method 0 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 10 monclient(hunting): handle_auth_done global_id 4397 payload 274 2024-09-15T01:32:10.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 1 -- 172.21.15.178:0/3202640187 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ba4096e00 msgr2=0x7f6ba4097270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:10.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 1 --2- 172.21.15.178:0/3202640187 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ba4096e00 0x7f6ba4097270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 1 -- 172.21.15.178:0/3202640187 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6ba412fff0 msgr2=0x7f6ba41303c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:10.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 1 --2- 172.21.15.178:0/3202640187 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6ba412fff0 0x7f6ba41303c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:10.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 1 monclient: found mon.noname-h 2024-09-15T01:32:10.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:10.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:32:10.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba259c640 1 -- 172.21.15.178:0/3202640187 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba404bf60 con 0x7f6ba4096a30 2024-09-15T01:32:10.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.101+0000 7f6ba1d9b640 1 --2- 172.21.15.178:0/3202640187 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6ba412fff0 0x7f6ba41303c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:10.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285ffff640 1 --2- 172.21.15.178:0/2859016271 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2860136ed0 0x7f28601392b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2858009f80 tx=0x7f2858009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b7cddf987af0754f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285e7fc640 1 -- 172.21.15.178:0/2859016271 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2858007880 con 0x7f2860136ed0 2024-09-15T01:32:10.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:32:10.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285e7fc640 10 monclient: dump: 2024-09-15T01:32:10.121 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.121 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.122 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285e7fc640 1 -- 172.21.15.178:0/2859016271 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285802d050 con 0x7f2860136ed0 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285e7fc640 1 -- 172.21.15.178:0/2859016271 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2858030360 con 0x7f2860136ed0 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f285e7fc640 10 monclient: dump: 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.124 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.124 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.124 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.124 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.124 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.124 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.124 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.124 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.124 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba259c640 1 --2- 172.21.15.178:0/3202640187 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ba4096a30 0x7f6ba412fab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009970 tx=0x7f6b940091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a379ad1984c2cabf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f28660e8640 5 monclient: authenticate success, global_id 4416 2024-09-15T01:32:10.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f28660e8640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:10.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f28660e8640 4 set_mon_vals no callback set 2024-09-15T01:32:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba159a640 1 -- 172.21.15.178:0/3202640187 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6b9402ac50 con 0x7f6ba4096a30 2024-09-15T01:32:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba159a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:32:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba159a640 10 monclient: dump: 2024-09-15T01:32:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba159a640 10 monclient: _finish_auth 0 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba159a640 1 -- 172.21.15.178:0/3202640187 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9402add0 con 0x7f6ba4096a30 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba159a640 1 -- 172.21.15.178:0/3202640187 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6b94031ca0 con 0x7f6ba4096a30 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba8e4f640 5 monclient: authenticate success, global_id 4397 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba8e4f640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba8e4f640 4 set_mon_vals no callback set 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba159a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.102+0000 7f6ba159a640 10 monclient: dump: 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.128 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.129 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.129 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.129 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.129 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.129 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.129 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.129 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.129 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.129 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.129 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.129 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f28660e8640 10 monclient: shutdown 2024-09-15T01:32:10.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f28660e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:10.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f28660e8640 1 -- 172.21.15.178:0/2859016271 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2860136ed0 msgr2=0x7f28601392b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f28660e8640 1 --2- 172.21.15.178:0/2859016271 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2860136ed0 0x7f28601392b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2858009f80 tx=0x7f2858009840 comp rx=0 tx=0).stop 2024-09-15T01:32:10.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f28660e8640 1 -- 172.21.15.178:0/2859016271 shutdown_connections 2024-09-15T01:32:10.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f28660e8640 1 --2- 172.21.15.178:0/2859016271 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2860136ed0 0x7f28601392b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f28660e8640 1 --2- 172.21.15.178:0/2859016271 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f286009d0a0 0x7f286009d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f28660e8640 1 --2- 172.21.15.178:0/2859016271 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f286009ccd0 0x7f2860097ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f6ba8e4f640 10 monclient: shutdown 2024-09-15T01:32:10.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f6ba8e4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:10.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/3202640187 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ba4096a30 msgr2=0x7f6ba412fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f6ba8e4f640 1 --2- 172.21.15.178:0/3202640187 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ba4096a30 0x7f6ba412fab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009970 tx=0x7f6b940091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f28660e8640 1 -- 172.21.15.178:0/2859016271 >> 172.21.15.178:0/2859016271 conn(0x7f28600a22e0 msgr2=0x7f28600a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:10.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f28660e8640 1 -- 172.21.15.178:0/2859016271 shutdown_connections 2024-09-15T01:32:10.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f28660e8640 1 -- 172.21.15.178:0/2859016271 wait complete. 2024-09-15T01:32:10.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/3202640187 shutdown_connections 2024-09-15T01:32:10.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f6ba8e4f640 1 --2- 172.21.15.178:0/3202640187 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ba4096e00 0x7f6ba4097270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f6ba8e4f640 1 --2- 172.21.15.178:0/3202640187 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6ba412fff0 0x7f6ba41303c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f6ba8e4f640 1 --2- 172.21.15.178:0/3202640187 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ba4096a30 0x7f6ba412fab0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/3202640187 >> 172.21.15.178:0/3202640187 conn(0x7f6ba40a22e0 msgr2=0x7f6ba40a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:10.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/3202640187 shutdown_connections 2024-09-15T01:32:10.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.103+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/3202640187 wait complete. 2024-09-15T01:32:10.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.104+0000 7f6ba8e4f640 10 monclient: build_initial_monmap 2024-09-15T01:32:10.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.104+0000 7f28660e8640 10 monclient: build_initial_monmap 2024-09-15T01:32:10.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.104+0000 7f28660e8640 10 monclient: monmap: 2024-09-15T01:32:10.134 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:10.134 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:10.134 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:10.134 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:10.134 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:10.134 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:10.134 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:10.134 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:10.134 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:10.135 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:10.135 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:10.135 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:10.135 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:10.135 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:10.135 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:10.135 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.104+0000 7f6ba8e4f640 10 monclient: monmap: 2024-09-15T01:32:10.136 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:10.136 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:10.136 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:10.136 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:10.136 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:10.136 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:10.136 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:10.136 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:10.136 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:10.136 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:10.136 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:10.137 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:10.137 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:10.137 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:10.137 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:10.137 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 1 Processor -- start 2024-09-15T01:32:10.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 1 -- start start 2024-09-15T01:32:10.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 10 monclient: init 2024-09-15T01:32:10.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f28660e8640 1 Processor -- start 2024-09-15T01:32:10.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f28660e8640 1 -- start start 2024-09-15T01:32:10.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f28660e8640 10 monclient: init 2024-09-15T01:32:10.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:10.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 10 monclient: _add_conns ranks=[0,3,6,8,5,7,2,1,4] 2024-09-15T01:32:10.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4096a30 0x7f6ba41325e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 10 monclient(hunting): picked mon.noname-a con 0x7f6ba4096a30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:10.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba4096e00 0x7f6ba4132b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 10 monclient(hunting): picked mon.noname-g con 0x7f6ba4096e00 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:10.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ba412fff0 0x7f6ba4133060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 10 monclient(hunting): picked mon.noname-d con 0x7f6ba412fff0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:10.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba4134650 con 0x7f6ba4096a30 2024-09-15T01:32:10.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba409bbd0 con 0x7f6ba4096e00 2024-09-15T01:32:10.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba409e840 con 0x7f6ba412fff0 2024-09-15T01:32:10.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:10.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba8e4f640 10 monclient(hunting): authenticate will time out at 734.856866s 2024-09-15T01:32:10.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f6ba259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4096a30 0x7f6ba41325e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.105+0000 7f28660e8640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:10.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 10 monclient: _add_conns ranks=[2,3,1,4,7,0,6,5,8] 2024-09-15T01:32:10.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f286009ccd0 0x7f2860141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 10 monclient(hunting): picked mon.noname-b con 0x7f286009ccd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:10.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba2d9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ba412fff0 0x7f6ba4133060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4096a30 0x7f6ba41325e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35004/0 (socket says 172.21.15.178:35004) 2024-09-15T01:32:10.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 1 -- 172.21.15.178:0/2230841741 learned_addr learned my addr 172.21.15.178:0/2230841741 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:10.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f286009d0a0 0x7f28601325c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 10 monclient(hunting): get_auth_request con 0x7f6ba4096a30 auth_method 0 2024-09-15T01:32:10.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba2d9d640 10 monclient(hunting): get_auth_request con 0x7f6ba412fff0 auth_method 0 2024-09-15T01:32:10.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba1d9b640 1 --2- 172.21.15.178:0/2230841741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba4096e00 0x7f6ba4132b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba1d9b640 10 monclient(hunting): get_auth_request con 0x7f6ba4096e00 auth_method 0 2024-09-15T01:32:10.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 10 monclient(hunting): picked mon.noname-g con 0x7f286009d0a0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:10.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2860136ed0 0x7f286006d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 10 monclient(hunting): picked mon.noname-f con 0x7f2860136ed0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:10.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286009e840 con 0x7f2860136ed0 2024-09-15T01:32:10.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286011ec90 con 0x7f286009ccd0 2024-09-15T01:32:10.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286011d0b0 con 0x7f286009d0a0 2024-09-15T01:32:10.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:10.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f28660e8640 10 monclient(hunting): authenticate will time out at 734.857206s 2024-09-15T01:32:10.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f286009ccd0 0x7f2860141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f286009ccd0 0x7f2860141860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:33706/0 (socket says 172.21.15.178:33706) 2024-09-15T01:32:10.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 1 -- 172.21.15.178:0/67265464 learned_addr learned my addr 172.21.15.178:0/67265464 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:10.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 10 monclient(hunting): get_auth_request con 0x7f286009ccd0 auth_method 0 2024-09-15T01:32:10.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285effd640 1 --2- 172.21.15.178:0/67265464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f286009d0a0 0x7f28601325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285effd640 10 monclient(hunting): get_auth_request con 0x7f286009d0a0 auth_method 0 2024-09-15T01:32:10.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285ffff640 1 --2- 172.21.15.178:0/67265464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2860136ed0 0x7f286006d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285ffff640 10 monclient(hunting): get_auth_request con 0x7f2860136ed0 auth_method 0 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 10 monclient(hunting): handle_auth_done global_id 4349 payload 824 2024-09-15T01:32:10.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 1 -- 172.21.15.178:0/67265464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f286009d0a0 msgr2=0x7f28601325c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 1 --2- 172.21.15.178:0/67265464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f286009d0a0 0x7f28601325c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 10 monclient(hunting): handle_auth_done global_id 4401 payload 824 2024-09-15T01:32:10.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 1 -- 172.21.15.178:0/2230841741 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ba412fff0 msgr2=0x7f6ba4133060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 1 --2- 172.21.15.178:0/2230841741 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ba412fff0 0x7f6ba4133060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 1 -- 172.21.15.178:0/2230841741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba4096e00 msgr2=0x7f6ba4132b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 1 --2- 172.21.15.178:0/2230841741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba4096e00 0x7f6ba4132b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 1 -- 172.21.15.178:0/67265464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2860136ed0 msgr2=0x7f286006d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 1 --2- 172.21.15.178:0/67265464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2860136ed0 0x7f286006d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:10.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 1 monclient: found mon.noname-b 2024-09-15T01:32:10.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:10.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:10.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:10.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 1 monclient: found mon.noname-a 2024-09-15T01:32:10.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:10.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:10.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 1 -- 172.21.15.178:0/2230841741 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b98009390 con 0x7f6ba4096a30 2024-09-15T01:32:10.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba1d9b640 1 --2- 172.21.15.178:0/2230841741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba4096e00 0x7f6ba4132b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:10.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 1 -- 172.21.15.178:0/67265464 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2854009390 con 0x7f286009ccd0 2024-09-15T01:32:10.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285ffff640 1 --2- 172.21.15.178:0/67265464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2860136ed0 0x7f286006d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:10.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f6ba259c640 1 --2- 172.21.15.178:0/2230841741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4096a30 0x7f6ba41325e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6b94035040 tx=0x7f6b94032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.106+0000 7f285f7fe640 1 --2- 172.21.15.178:0/67265464 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f286009ccd0 0x7f2860141860 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f285000ea20 tx=0x7f28500090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6b927fc640 1 -- 172.21.15.178:0/2230841741 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6b94031820 con 0x7f6ba4096a30 2024-09-15T01:32:10.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6b927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6b927fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:10.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6b927fc640 10 monclient: dump: 2024-09-15T01:32:10.152 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.152 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.152 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.152 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6b927fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:10.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6b927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:10.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28477fe640 1 -- 172.21.15.178:0/67265464 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f285000ce90 con 0x7f286009ccd0 2024-09-15T01:32:10.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28477fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:10.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6b927fc640 1 -- 172.21.15.178:0/2230841741 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b940319e0 con 0x7f6ba4096a30 2024-09-15T01:32:10.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6ba8e4f640 5 monclient: authenticate success, global_id 4401 2024-09-15T01:32:10.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6ba8e4f640 10 monclient: _renew_subs 2024-09-15T01:32:10.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6ba8e4f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:10.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/2230841741 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b94009450 con 0x7f6ba4096a30 2024-09-15T01:32:10.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28477fe640 10 monclient: dump: 2024-09-15T01:32:10.155 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.155 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.155 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.155 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.155 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28477fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:10.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28660e8640 5 monclient: authenticate success, global_id 4349 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28477fe640 1 -- 172.21.15.178:0/67265464 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2850009600 con 0x7f286009ccd0 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28477fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28477fe640 1 -- 172.21.15.178:0/67265464 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28500138f0 con 0x7f286009ccd0 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28477fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28477fe640 10 monclient: dump: 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.157 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.158 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.158 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.158 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.158 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.158 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.158 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.158 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.158 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6b927fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:10.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6b927fc640 1 -- 172.21.15.178:0/2230841741 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6b9403a390 con 0x7f6ba4096a30 2024-09-15T01:32:10.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6b927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f285dffb640 4 set_mon_vals no callback set 2024-09-15T01:32:10.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28660e8640 10 monclient: _renew_subs 2024-09-15T01:32:10.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28660e8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:10.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28660e8640 1 -- 172.21.15.178:0/67265464 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2858009ae0 con 0x7f286009ccd0 2024-09-15T01:32:10.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28660e8640 10 monclient: _renew_subs 2024-09-15T01:32:10.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28660e8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:10.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f28660e8640 1 -- 172.21.15.178:0/67265464 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2860141da0 con 0x7f286009ccd0 2024-09-15T01:32:10.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6b927fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:10.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6b927fc640 10 monclient: dump: 2024-09-15T01:32:10.160 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.160 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.160 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.160 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6ba0d99640 4 set_mon_vals no callback set 2024-09-15T01:32:10.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6ba8e4f640 10 monclient: _renew_subs 2024-09-15T01:32:10.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6ba8e4f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:10.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.107+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/2230841741 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba41340d0 con 0x7f6ba4096a30 2024-09-15T01:32:10.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.108+0000 7f28477fe640 1 -- 172.21.15.178:0/67265464 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2850013470 con 0x7f286009ccd0 2024-09-15T01:32:10.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.108+0000 7f6b927fc640 1 -- 172.21.15.178:0/2230841741 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6b94039040 con 0x7f6ba4096a30 2024-09-15T01:32:10.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.108+0000 7f28477fe640 1 --2- 172.21.15.178:0/67265464 >> v2:172.21.15.204:6812/28210 conn(0x7f284c0397b0 0x7f284c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.108+0000 7f28477fe640 1 -- 172.21.15.178:0/67265464 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f285004b2b0 con 0x7f286009ccd0 2024-09-15T01:32:10.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.108+0000 7f285effd640 1 --2- 172.21.15.178:0/67265464 >> v2:172.21.15.204:6812/28210 conn(0x7f284c0397b0 0x7f284c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.108+0000 7f6b927fc640 1 --2- 172.21.15.178:0/2230841741 >> v2:172.21.15.204:6812/28210 conn(0x7f6b840397b0 0x7f6b8403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.108+0000 7f285effd640 10 monclient: get_auth_request con 0x7f284c0397b0 auth_method 0 2024-09-15T01:32:10.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.108+0000 7f6b927fc640 1 -- 172.21.15.178:0/2230841741 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f6b9406d1c0 con 0x7f6ba4096a30 2024-09-15T01:32:10.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.108+0000 7f6ba1d9b640 1 --2- 172.21.15.178:0/2230841741 >> v2:172.21.15.204:6812/28210 conn(0x7f6b840397b0 0x7f6b8403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.108+0000 7f6ba1d9b640 10 monclient: get_auth_request con 0x7f6b840397b0 auth_method 0 2024-09-15T01:32:10.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.108+0000 7f28660e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.108+0000 7f28660e8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.109+0000 7f28660e8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:10.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.109+0000 7f28660e8640 1 -- 172.21.15.178:0/67265464 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286011e870 con 0x7f286009ccd0 2024-09-15T01:32:10.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.109+0000 7f285effd640 1 --2- 172.21.15.178:0/67265464 >> v2:172.21.15.204:6812/28210 conn(0x7f284c0397b0 0x7f284c03bc60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2854009eb0 tx=0x7f28540090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.109+0000 7f6ba8e4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.109+0000 7f6ba8e4f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.109+0000 7f6ba8e4f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:10.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.109+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/2230841741 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba404b720 con 0x7f6ba4096a30 2024-09-15T01:32:10.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.109+0000 7f6ba1d9b640 1 --2- 172.21.15.178:0/2230841741 >> v2:172.21.15.204:6812/28210 conn(0x7f6b840397b0 0x7f6b8403bc60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009fd0 tx=0x7f6b980090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.113+0000 7f28477fe640 1 -- 172.21.15.178:0/67265464 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f28500123d0 con 0x7f286009ccd0 2024-09-15T01:32:10.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.113+0000 7f28477fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.113+0000 7f28477fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:10.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.113+0000 7f6b927fc640 1 -- 172.21.15.178:0/2230841741 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6b94037080 con 0x7f6ba4096a30 2024-09-15T01:32:10.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.113+0000 7f6b927fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.113+0000 7f6b927fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:10.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.192+0000 7f93ed39d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:10.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.192+0000 7f93ed39d640 10 monclient: build_initial_monmap 2024-09-15T01:32:10.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.192+0000 7f93ed39d640 10 monclient: monmap: 2024-09-15T01:32:10.196 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:10.196 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.196 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:10.194390+0000 2024-09-15T01:32:10.196 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:10.194390+0000 2024-09-15T01:32:10.196 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:10.196 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.196 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:10.196 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:10.196 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:10.197 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:10.197 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:10.197 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:10.197 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:10.197 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:10.197 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:10.197 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:10.197 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:10.197 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 1 Processor -- start 2024-09-15T01:32:10.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 1 -- start start 2024-09-15T01:32:10.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 10 monclient: init 2024-09-15T01:32:10.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:10.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 10 monclient: _add_conns ranks=[7,0,10,8,5,3,4,1,6,2,9] 2024-09-15T01:32:10.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f93e8098c40 0x7f93e809ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 10 monclient(hunting): picked mon.noname-i con 0x7f93e8098c40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:10.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f93e809dfd0 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:10.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f93e8135f10 0x7f93e81382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 10 monclient(hunting): picked mon.noname-f con 0x7f93e8135f10 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:10.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e804bf60 con 0x7f93e809dfd0 2024-09-15T01:32:10.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e8099e80 con 0x7f93e8135f10 2024-09-15T01:32:10.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f93e8099b30 con 0x7f93e8098c40 2024-09-15T01:32:10.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:10.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93ed39d640 10 monclient(hunting): authenticate will time out at 734.944737s 2024-09-15T01:32:10.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93e67fc640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f93e809dfd0 0x7f93e809d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:49276/0 (socket says 172.21.15.178:49276) 2024-09-15T01:32:10.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.193+0000 7f93e67fc640 1 -- 172.21.15.178:0/923997974 learned_addr learned my addr 172.21.15.178:0/923997974 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:10.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e6ffd640 1 --2- 172.21.15.178:0/923997974 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f93e8098c40 0x7f93e809ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 1 --2- 172.21.15.178:0/923997974 >> v2:172.21.15.204:3300/0 conn(0x7f93e8135f10 0x7f93e81382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e6ffd640 10 monclient(hunting): get_auth_request con 0x7f93e8098c40 auth_method 0 2024-09-15T01:32:10.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 10 monclient(hunting): get_auth_request con 0x7f93e8135f10 auth_method 0 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e5ffb640 1 -- 172.21.15.178:0/923997974 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534977879 0 0) 0x7f93e804bf60 con 0x7f93e809dfd0 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e5ffb640 10 monclient(hunting): my global_id is 4410 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e5ffb640 1 -- 172.21.15.178:0/923997974 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93d00034e0 con 0x7f93e809dfd0 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 10 monclient(hunting): handle_auth_done global_id 4348 payload 274 2024-09-15T01:32:10.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 1 -- 172.21.15.178:0/923997974 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f93e8098c40 msgr2=0x7f93e809ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:10.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 1 --2- 172.21.15.178:0/923997974 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f93e8098c40 0x7f93e809ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 1 -- 172.21.15.178:0/923997974 >> v1:172.21.15.178:6789/0 conn(0x7f93e809dfd0 legacy=0x7f93e809d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:10.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 1 monclient: found mon.noname-f 2024-09-15T01:32:10.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:10.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:32:10.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.194+0000 7f93e77fe640 1 -- 172.21.15.178:0/923997974 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e8063620 con 0x7f93e8135f10 2024-09-15T01:32:10.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e77fe640 1 --2- 172.21.15.178:0/923997974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93e8135f10 0x7f93e81382f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009e90 tx=0x7f93e0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad3a1736ccae99c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e5ffb640 1 -- 172.21.15.178:0/923997974 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93dc0018f0 con 0x7f93e8135f10 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e5ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e5ffb640 10 monclient: dump: 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.209 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e5ffb640 1 -- 172.21.15.178:0/923997974 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e002d050 con 0x7f93e8135f10 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93ed39d640 5 monclient: authenticate success, global_id 4348 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93ed39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93ed39d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:10.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e5ffb640 1 -- 172.21.15.178:0/923997974 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93e0007e50 con 0x7f93e8135f10 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93ed39d640 4 set_mon_vals no callback set 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93e5ffb640 10 monclient: dump: 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93ed39d640 10 monclient: shutdown 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93ed39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93ed39d640 1 -- 172.21.15.178:0/923997974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93e8135f10 msgr2=0x7f93e81382f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93ed39d640 1 --2- 172.21.15.178:0/923997974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93e8135f10 0x7f93e81382f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009e90 tx=0x7f93e0009780 comp rx=0 tx=0).stop 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93ed39d640 1 -- 172.21.15.178:0/923997974 shutdown_connections 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93ed39d640 1 --2- 172.21.15.178:0/923997974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93e8135f10 0x7f93e81382f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.195+0000 7f93ed39d640 1 --2- 172.21.15.178:0/923997974 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f93e8098c40 0x7f93e809ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.196+0000 7f93ed39d640 1 -- 172.21.15.178:0/923997974 >> 172.21.15.178:0/923997974 conn(0x7f93e80a22e0 msgr2=0x7f93e80a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:10.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.196+0000 7f93ed39d640 1 -- 172.21.15.178:0/923997974 shutdown_connections 2024-09-15T01:32:10.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.196+0000 7f93ed39d640 1 -- 172.21.15.178:0/923997974 wait complete. 2024-09-15T01:32:10.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.196+0000 7f93ed39d640 10 monclient: build_initial_monmap 2024-09-15T01:32:10.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.196+0000 7f93ed39d640 10 monclient: monmap: 2024-09-15T01:32:10.213 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:10.213 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:10.213 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:10.213 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:10.213 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:10.213 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:10.213 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:10.213 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 1 Processor -- start 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 1 -- start start 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 10 monclient: init 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 10 monclient: _add_conns ranks=[7,2,4,0,6,3,8,1,5] 2024-09-15T01:32:10.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f93e8098c40 0x7f93e813f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 10 monclient(hunting): picked mon.noname-i con 0x7f93e8098c40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:10.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93e809dfd0 0x7f93e812f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 10 monclient(hunting): picked mon.noname-b con 0x7f93e809dfd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:10.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f93e8135f10 0x7f93e806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 10 monclient(hunting): picked mon.noname-c con 0x7f93e8135f10 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:10.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e8118c10 con 0x7f93e809dfd0 2024-09-15T01:32:10.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93e809e660 con 0x7f93e8135f10 2024-09-15T01:32:10.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f93e806f1f0 con 0x7f93e8098c40 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93ed39d640 10 monclient(hunting): authenticate will time out at 734.948560s 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e77fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f93e8135f10 0x7f93e806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e67fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93e809dfd0 0x7f93e812f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e77fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f93e8135f10 0x7f93e806d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:36770/0 (socket says 172.21.15.178:36770) 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e77fe640 1 -- 172.21.15.178:0/2734505667 learned_addr learned my addr 172.21.15.178:0/2734505667 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e77fe640 10 monclient(hunting): get_auth_request con 0x7f93e8135f10 auth_method 0 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e67fc640 10 monclient(hunting): get_auth_request con 0x7f93e809dfd0 auth_method 0 2024-09-15T01:32:10.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e6ffd640 1 --2- 172.21.15.178:0/2734505667 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f93e8098c40 0x7f93e813f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e6ffd640 10 monclient(hunting): get_auth_request con 0x7f93e8098c40 auth_method 0 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.197+0000 7f93e77fe640 10 monclient(hunting): handle_auth_done global_id 4333 payload 824 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93e77fe640 1 -- 172.21.15.178:0/2734505667 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f93e8098c40 msgr2=0x7f93e813f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93e77fe640 1 --2- 172.21.15.178:0/2734505667 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f93e8098c40 0x7f93e813f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93e77fe640 1 -- 172.21.15.178:0/2734505667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93e809dfd0 msgr2=0x7f93e812f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93e77fe640 1 --2- 172.21.15.178:0/2734505667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93e809dfd0 0x7f93e812f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93e77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93e77fe640 1 monclient: found mon.noname-c 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93e77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93e77fe640 1 -- 172.21.15.178:0/2734505667 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e0009a20 con 0x7f93e8135f10 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93e77fe640 1 --2- 172.21.15.178:0/2734505667 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f93e8135f10 0x7f93e806d6f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009e90 tx=0x7f93e0036000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ceffd640 1 -- 172.21.15.178:0/2734505667 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93e0031c10 con 0x7f93e8135f10 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ceffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ceffd640 10 monclient: dump: 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ceffd640 10 monclient: _finish_auth 0 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ceffd640 1 -- 172.21.15.178:0/2734505667 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e0031db0 con 0x7f93e8135f10 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ed39d640 5 monclient: authenticate success, global_id 4333 2024-09-15T01:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ed39d640 10 monclient: _renew_subs 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ed39d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ed39d640 1 -- 172.21.15.178:0/2734505667 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e809bd50 con 0x7f93e8135f10 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ceffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ceffd640 1 -- 172.21.15.178:0/2734505667 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93e003d900 con 0x7f93e8135f10 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ceffd640 10 monclient: dump: 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.221 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ed39d640 10 monclient: _renew_subs 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ed39d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93ed39d640 1 -- 172.21.15.178:0/2734505667 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e813fd20 con 0x7f93e8135f10 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.198+0000 7f93e57fa640 4 set_mon_vals no callback set 2024-09-15T01:32:10.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.199+0000 7f93ceffd640 1 -- 172.21.15.178:0/2734505667 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f93e0039040 con 0x7f93e8135f10 2024-09-15T01:32:10.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.199+0000 7f93ed39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.199+0000 7f93ed39d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.199+0000 7f93ed39d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:10.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.199+0000 7f93ed39d640 1 -- 172.21.15.178:0/2734505667 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e804b720 con 0x7f93e8135f10 2024-09-15T01:32:10.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.199+0000 7f93ceffd640 1 --2- 172.21.15.178:0/2734505667 >> v2:172.21.15.204:6812/28210 conn(0x7f93d40397e0 0x7f93d403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.199+0000 7f93ceffd640 1 -- 172.21.15.178:0/2734505667 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f93e006c690 con 0x7f93e8135f10 2024-09-15T01:32:10.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.200+0000 7f93e6ffd640 1 --2- 172.21.15.178:0/2734505667 >> v2:172.21.15.204:6812/28210 conn(0x7f93d40397e0 0x7f93d403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.200+0000 7f93e6ffd640 10 monclient: get_auth_request con 0x7f93d40397e0 auth_method 0 2024-09-15T01:32:10.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.200+0000 7f93e6ffd640 1 --2- 172.21.15.178:0/2734505667 >> v2:172.21.15.204:6812/28210 conn(0x7f93d40397e0 0x7f93d403bc90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009c80 tx=0x7f93d80090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.204+0000 7f93ceffd640 1 -- 172.21.15.178:0/2734505667 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f93e0031790 con 0x7f93e8135f10 2024-09-15T01:32:10.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.204+0000 7f93ceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.204+0000 7f93ceffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:10.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.279+0000 7f28660e8640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T01:32:10.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.279+0000 7f28660e8640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T01:32:10.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.279+0000 7f28660e8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:10.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.279+0000 7f28660e8640 1 -- 172.21.15.178:0/67265464 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f286004b720 con 0x7f286009ccd0 2024-09-15T01:32:10.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.279+0000 7f28477fe640 1 -- 172.21.15.178:0/67265464 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f285001b3b0 con 0x7f286009ccd0 2024-09-15T01:32:10.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.279+0000 7f28477fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T01:32:10.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.279+0000 7f28477fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:10.283 INFO:teuthology.orchestra.run.smithi178.stdout:47244640260 2024-09-15T01:32:10.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.280+0000 7f6ba8e4f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T01:32:10.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.280+0000 7f6ba8e4f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T01:32:10.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.280+0000 7f6ba8e4f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:10.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.280+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/2230841741 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6ba4070770 con 0x7f6ba4096a30 2024-09-15T01:32:10.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.280+0000 7f6b927fc640 1 -- 172.21.15.178:0/2230841741 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6b94041030 con 0x7f6ba4096a30 2024-09-15T01:32:10.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.280+0000 7f6b927fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T01:32:10.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.280+0000 7f6b927fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:10.284 INFO:teuthology.orchestra.run.smithi178.stdout:47244640260 2024-09-15T01:32:10.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.281+0000 7f28660e8640 1 -- 172.21.15.178:0/67265464 >> v2:172.21.15.204:6812/28210 conn(0x7f284c0397b0 msgr2=0x7f284c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.281+0000 7f28660e8640 1 --2- 172.21.15.178:0/67265464 >> v2:172.21.15.204:6812/28210 conn(0x7f284c0397b0 0x7f284c03bc60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2854009eb0 tx=0x7f28540090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.281+0000 7f28660e8640 10 monclient: shutdown 2024-09-15T01:32:10.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.281+0000 7f28660e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:10.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.281+0000 7f28660e8640 1 -- 172.21.15.178:0/67265464 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f286009ccd0 msgr2=0x7f2860141860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.281+0000 7f28660e8640 1 --2- 172.21.15.178:0/67265464 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f286009ccd0 0x7f2860141860 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f285000ea20 tx=0x7f28500090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.281+0000 7f28660e8640 1 -- 172.21.15.178:0/67265464 shutdown_connections 2024-09-15T01:32:10.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.281+0000 7f28660e8640 1 --2- 172.21.15.178:0/67265464 >> v2:172.21.15.204:6812/28210 conn(0x7f284c0397b0 0x7f284c03bc60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.281+0000 7f28660e8640 1 --2- 172.21.15.178:0/67265464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2860136ed0 0x7f286006d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.281+0000 7f28660e8640 1 --2- 172.21.15.178:0/67265464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f286009d0a0 0x7f28601325c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.281+0000 7f28660e8640 1 --2- 172.21.15.178:0/67265464 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f286009ccd0 0x7f2860141860 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.281+0000 7f28660e8640 1 -- 172.21.15.178:0/67265464 >> 172.21.15.178:0/67265464 conn(0x7f28600a22e0 msgr2=0x7f28601364c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:10.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f28660e8640 1 -- 172.21.15.178:0/67265464 shutdown_connections 2024-09-15T01:32:10.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f28660e8640 1 -- 172.21.15.178:0/67265464 wait complete. 2024-09-15T01:32:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/2230841741 >> v2:172.21.15.204:6812/28210 conn(0x7f6b840397b0 msgr2=0x7f6b8403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f6ba8e4f640 1 --2- 172.21.15.178:0/2230841741 >> v2:172.21.15.204:6812/28210 conn(0x7f6b840397b0 0x7f6b8403bc60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009fd0 tx=0x7f6b980090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f6ba8e4f640 10 monclient: shutdown 2024-09-15T01:32:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f6ba8e4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/2230841741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4096a30 msgr2=0x7f6ba41325e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f6ba8e4f640 1 --2- 172.21.15.178:0/2230841741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4096a30 0x7f6ba41325e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6b94035040 tx=0x7f6b94032000 comp rx=0 tx=0).stop 2024-09-15T01:32:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/2230841741 shutdown_connections 2024-09-15T01:32:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f6ba8e4f640 1 --2- 172.21.15.178:0/2230841741 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ba412fff0 0x7f6ba4133060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f6ba8e4f640 1 --2- 172.21.15.178:0/2230841741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba4096e00 0x7f6ba4132b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f6ba8e4f640 1 --2- 172.21.15.178:0/2230841741 >> v2:172.21.15.204:6812/28210 conn(0x7f6b840397b0 0x7f6b8403bc60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f6ba8e4f640 1 --2- 172.21.15.178:0/2230841741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ba4096a30 0x7f6ba41325e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.282+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/2230841741 >> 172.21.15.178:0/2230841741 conn(0x7f6ba40a22e0 msgr2=0x7f6ba40a01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:10.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.283+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/2230841741 shutdown_connections 2024-09-15T01:32:10.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.283+0000 7f6ba8e4f640 1 -- 172.21.15.178:0/2230841741 wait complete. 2024-09-15T01:32:10.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640260 for osd.0 2024-09-15T01:32:10.297 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:10.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640260 for osd.1 2024-09-15T01:32:10.298 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:10.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.365+0000 7f93ed39d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T01:32:10.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.365+0000 7f93ed39d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T01:32:10.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.365+0000 7f93ed39d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:10.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.365+0000 7f93ed39d640 1 -- 172.21.15.178:0/2734505667 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f93e8070770 con 0x7f93e8135f10 2024-09-15T01:32:10.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.366+0000 7f93ceffd640 1 -- 172.21.15.178:0/2734505667 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f93e0035070 con 0x7f93e8135f10 2024-09-15T01:32:10.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.366+0000 7f93ceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T01:32:10.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.366+0000 7f93ceffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:10.370 INFO:teuthology.orchestra.run.smithi178.stdout:47244640260 2024-09-15T01:32:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 1 -- 172.21.15.178:0/2734505667 >> v2:172.21.15.204:6812/28210 conn(0x7f93d40397e0 msgr2=0x7f93d403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 1 --2- 172.21.15.178:0/2734505667 >> v2:172.21.15.204:6812/28210 conn(0x7f93d40397e0 0x7f93d403bc90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009c80 tx=0x7f93d80090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 10 monclient: shutdown 2024-09-15T01:32:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 1 -- 172.21.15.178:0/2734505667 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f93e8135f10 msgr2=0x7f93e806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 1 --2- 172.21.15.178:0/2734505667 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f93e8135f10 0x7f93e806d6f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009e90 tx=0x7f93e0036000 comp rx=0 tx=0).stop 2024-09-15T01:32:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 1 -- 172.21.15.178:0/2734505667 shutdown_connections 2024-09-15T01:32:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 1 --2- 172.21.15.178:0/2734505667 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f93e8135f10 0x7f93e806d6f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 1 --2- 172.21.15.178:0/2734505667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93e809dfd0 0x7f93e812f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 1 --2- 172.21.15.178:0/2734505667 >> v2:172.21.15.204:6812/28210 conn(0x7f93d40397e0 0x7f93d403bc90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 1 --2- 172.21.15.178:0/2734505667 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f93e8098c40 0x7f93e813f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 1 -- 172.21.15.178:0/2734505667 >> 172.21.15.178:0/2734505667 conn(0x7f93e80a22e0 msgr2=0x7f93e81330b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:10.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 1 -- 172.21.15.178:0/2734505667 shutdown_connections 2024-09-15T01:32:10.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.368+0000 7f93ed39d640 1 -- 172.21.15.178:0/2734505667 wait complete. 2024-09-15T01:32:10.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640260 for osd.4 2024-09-15T01:32:10.382 DEBUG:teuthology.parallel:result is None 2024-09-15T01:32:10.382 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T01:32:10.382 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:32:10.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.513+0000 7faf44c34640 10 monclient: get_monmap_and_config 2024-09-15T01:32:10.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.513+0000 7faf44c34640 10 monclient: build_initial_monmap 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.513+0000 7faf44c34640 10 monclient: monmap: 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:10.515607+0000 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:10.515607+0000 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:10.517 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:10.518 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:10.518 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:10.518 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:10.518 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:10.518 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:10.518 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.514+0000 7faf44c34640 1 Processor -- start 2024-09-15T01:32:10.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.514+0000 7faf44c34640 1 -- start start 2024-09-15T01:32:10.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.514+0000 7faf44c34640 10 monclient: init 2024-09-15T01:32:10.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:10.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 10 monclient: _add_conns ranks=[8,4,7,6,5,10,2,9,0,3,1] 2024-09-15T01:32:10.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 0x7faf40132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 10 monclient(hunting): picked mon.noname-e con 0x7faf400a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:10.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 0x7faf4009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 10 monclient(hunting): picked mon.noname-c con 0x7faf4009faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:10.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7faf400a08c0 0x7faf400a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 10 monclient(hunting): picked mon.noname-i con 0x7faf400a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:10.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf4009e0f0 con 0x7faf4009faa0 2024-09-15T01:32:10.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7faf4009dd80 con 0x7faf400a08c0 2024-09-15T01:32:10.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7faf400a2d00 con 0x7faf400a2930 2024-09-15T01:32:10.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf44c34640 10 monclient(hunting): authenticate will time out at 735.266126s 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 0x7faf40132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 0x7faf40132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:57980/0 (socket says 172.21.15.178:57980) 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 1 -- 172.21.15.178:0/3758284125 learned_addr learned my addr 172.21.15.178:0/3758284125 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3dd9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 0x7faf4009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 10 monclient(hunting): get_auth_request con 0x7faf400a2930 auth_method 0 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3dd9b640 10 monclient(hunting): get_auth_request con 0x7faf4009faa0 auth_method 0 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3ed9d640 1 --2- 172.21.15.178:0/3758284125 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7faf400a08c0 0x7faf400a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3ed9d640 10 monclient(hunting): get_auth_request con 0x7faf400a08c0 auth_method 0 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 10 monclient(hunting): handle_auth_done global_id 4364 payload 274 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 1 -- 172.21.15.178:0/3758284125 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7faf400a08c0 msgr2=0x7faf400a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 1 --2- 172.21.15.178:0/3758284125 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7faf400a08c0 0x7faf400a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 1 -- 172.21.15.178:0/3758284125 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 msgr2=0x7faf4009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 1 --2- 172.21.15.178:0/3758284125 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 0x7faf4009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 1 monclient: found mon.noname-e 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3e59c640 1 -- 172.21.15.178:0/3758284125 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4004bf60 con 0x7faf400a2930 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.515+0000 7faf3dd9b640 1 --2- 172.21.15.178:0/3758284125 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 0x7faf4009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3e59c640 1 --2- 172.21.15.178:0/3758284125 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 0x7faf40132de0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faf38009f50 tx=0x7faf38009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=5152dd65694049db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3d59a640 1 -- 172.21.15.178:0/3758284125 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf38007880 con 0x7faf400a2930 2024-09-15T01:32:10.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3d59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:10.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3d59a640 10 monclient: dump: 2024-09-15T01:32:10.533 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.533 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.533 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.533 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.533 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3d59a640 10 monclient: _finish_auth 0 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:10.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3d59a640 1 -- 172.21.15.178:0/3758284125 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3802d050 con 0x7faf400a2930 2024-09-15T01:32:10.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf44c34640 5 monclient: authenticate success, global_id 4364 2024-09-15T01:32:10.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf44c34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:10.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:10.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf44c34640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:10.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3d59a640 1 -- 172.21.15.178:0/3758284125 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf38030360 con 0x7faf400a2930 2024-09-15T01:32:10.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf44c34640 4 set_mon_vals no callback set 2024-09-15T01:32:10.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3d59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:10.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf3d59a640 10 monclient: dump: 2024-09-15T01:32:10.535 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.535 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.536 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf44c34640 10 monclient: shutdown 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf44c34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf44c34640 1 -- 172.21.15.178:0/3758284125 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 msgr2=0x7faf40132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.516+0000 7faf44c34640 1 --2- 172.21.15.178:0/3758284125 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 0x7faf40132de0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faf38009f50 tx=0x7faf38009840 comp rx=0 tx=0).stop 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.517+0000 7faf44c34640 1 -- 172.21.15.178:0/3758284125 shutdown_connections 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.517+0000 7faf44c34640 1 --2- 172.21.15.178:0/3758284125 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7faf400a08c0 0x7faf400a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.517+0000 7faf44c34640 1 --2- 172.21.15.178:0/3758284125 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 0x7faf4009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.517+0000 7faf44c34640 1 --2- 172.21.15.178:0/3758284125 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 0x7faf40132de0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.517+0000 7faf44c34640 1 -- 172.21.15.178:0/3758284125 >> 172.21.15.178:0/3758284125 conn(0x7faf400a5ed0 msgr2=0x7faf400a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.517+0000 7faf44c34640 1 -- 172.21.15.178:0/3758284125 shutdown_connections 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.517+0000 7faf44c34640 1 -- 172.21.15.178:0/3758284125 wait complete. 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.517+0000 7faf44c34640 10 monclient: build_initial_monmap 2024-09-15T01:32:10.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.517+0000 7faf44c34640 10 monclient: monmap: 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:10.538 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.517+0000 7faf44c34640 1 Processor -- start 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.517+0000 7faf44c34640 1 -- start start 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.517+0000 7faf44c34640 10 monclient: init 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 10 monclient: _add_conns ranks=[4,5,8,6,3,0,1,7,2] 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 0x7faf4009b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 10 monclient(hunting): picked mon.noname-c con 0x7faf4009faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf400a08c0 0x7faf401318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 10 monclient(hunting): picked mon.noname-h con 0x7faf400a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 0x7faf4011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 10 monclient(hunting): picked mon.noname-e con 0x7faf400a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf4009d630 con 0x7faf4009faa0 2024-09-15T01:32:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf4011ef90 con 0x7faf400a08c0 2024-09-15T01:32:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7faf4011ecb0 con 0x7faf400a2930 2024-09-15T01:32:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf44c34640 10 monclient(hunting): authenticate will time out at 735.269448s 2024-09-15T01:32:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 0x7faf4009b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 0x7faf4009b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:36786/0 (socket says 172.21.15.178:36786) 2024-09-15T01:32:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 1 -- 172.21.15.178:0/3993127709 learned_addr learned my addr 172.21.15.178:0/3993127709 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 10 monclient(hunting): get_auth_request con 0x7faf4009faa0 auth_method 0 2024-09-15T01:32:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3ed9d640 1 --2- 172.21.15.178:0/3993127709 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 0x7faf4011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3ed9d640 10 monclient(hunting): get_auth_request con 0x7faf400a2930 auth_method 0 2024-09-15T01:32:10.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3dd9b640 1 --2- 172.21.15.178:0/3993127709 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf400a08c0 0x7faf401318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3dd9b640 10 monclient(hunting): get_auth_request con 0x7faf400a08c0 auth_method 0 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 10 monclient(hunting): handle_auth_done global_id 4351 payload 824 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 1 -- 172.21.15.178:0/3993127709 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 msgr2=0x7faf4011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 1 --2- 172.21.15.178:0/3993127709 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 0x7faf4011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 1 -- 172.21.15.178:0/3993127709 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf400a08c0 msgr2=0x7faf401318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 1 --2- 172.21.15.178:0/3993127709 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf400a08c0 0x7faf401318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 1 monclient: found mon.noname-c 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3e59c640 1 -- 172.21.15.178:0/3993127709 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf34009390 con 0x7faf4009faa0 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3dd9b640 1 --2- 172.21.15.178:0/3993127709 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf400a08c0 0x7faf401318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.518+0000 7faf3ed9d640 1 --2- 172.21.15.178:0/3993127709 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 0x7faf4011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf3e59c640 1 --2- 172.21.15.178:0/3993127709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 0x7faf4009b010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faf38000f80 tx=0x7faf38036000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 1 -- 172.21.15.178:0/3993127709 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf3803cab0 con 0x7faf4009faa0 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 10 monclient: dump: 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.543 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 1 -- 172.21.15.178:0/3993127709 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3803cc50 con 0x7faf4009faa0 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf44c34640 5 monclient: authenticate success, global_id 4351 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf44c34640 10 monclient: _renew_subs 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf44c34640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf44c34640 1 -- 172.21.15.178:0/3993127709 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf38009ae0 con 0x7faf4009faa0 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 1 -- 172.21.15.178:0/3993127709 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf3803cdc0 con 0x7faf4009faa0 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 10 monclient: dump: 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.545 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf3cd99640 4 set_mon_vals no callback set 2024-09-15T01:32:10.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf44c34640 10 monclient: _renew_subs 2024-09-15T01:32:10.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf44c34640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:10.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf44c34640 1 -- 172.21.15.178:0/3993127709 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4011e9d0 con 0x7faf4009faa0 2024-09-15T01:32:10.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.519+0000 7faf2e7fc640 1 -- 172.21.15.178:0/3993127709 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7faf38003040 con 0x7faf4009faa0 2024-09-15T01:32:10.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.520+0000 7faf2e7fc640 1 --2- 172.21.15.178:0/3993127709 >> v2:172.21.15.204:6812/28210 conn(0x7faf200397b0 0x7faf2003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.520+0000 7faf2e7fc640 1 -- 172.21.15.178:0/3993127709 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7faf3806b970 con 0x7faf4009faa0 2024-09-15T01:32:10.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.520+0000 7faf44c34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.520+0000 7faf44c34640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.520+0000 7faf44c34640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:10.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.520+0000 7faf44c34640 1 -- 172.21.15.178:0/3993127709 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4004b720 con 0x7faf4009faa0 2024-09-15T01:32:10.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.520+0000 7faf3dd9b640 1 --2- 172.21.15.178:0/3993127709 >> v2:172.21.15.204:6812/28210 conn(0x7faf200397b0 0x7faf2003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.520+0000 7faf3dd9b640 10 monclient: get_auth_request con 0x7faf200397b0 auth_method 0 2024-09-15T01:32:10.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.520+0000 7faf3dd9b640 1 --2- 172.21.15.178:0/3993127709 >> v2:172.21.15.204:6812/28210 conn(0x7faf200397b0 0x7faf2003bc60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf34009360 tx=0x7faf34008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.525+0000 7faf2e7fc640 1 -- 172.21.15.178:0/3993127709 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7faf38040080 con 0x7faf4009faa0 2024-09-15T01:32:10.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.525+0000 7faf2e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.525+0000 7faf2e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:10.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.676+0000 7faf44c34640 1 -- 172.21.15.178:0/3993127709 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf4005ed70 con 0x7faf200397b0 2024-09-15T01:32:10.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.680+0000 7faf2e7fc640 1 -- 172.21.15.178:0/3993127709 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+39208 (secure 0 0 0) 0x7faf4005ed70 con 0x7faf200397b0 2024-09-15T01:32:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:32:10.682 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-09-15T01:32:10.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 1 -- 172.21.15.178:0/3993127709 >> v2:172.21.15.204:6812/28210 conn(0x7faf200397b0 msgr2=0x7faf2003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 1 --2- 172.21.15.178:0/3993127709 >> v2:172.21.15.204:6812/28210 conn(0x7faf200397b0 0x7faf2003bc60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf34009360 tx=0x7faf34008040 comp rx=0 tx=0).stop 2024-09-15T01:32:10.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 10 monclient: shutdown 2024-09-15T01:32:10.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:10.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 1 -- 172.21.15.178:0/3993127709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 msgr2=0x7faf4009b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 1 --2- 172.21.15.178:0/3993127709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 0x7faf4009b010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faf38000f80 tx=0x7faf38036000 comp rx=0 tx=0).stop 2024-09-15T01:32:10.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 1 -- 172.21.15.178:0/3993127709 shutdown_connections 2024-09-15T01:32:10.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 1 --2- 172.21.15.178:0/3993127709 >> v2:172.21.15.204:6812/28210 conn(0x7faf200397b0 0x7faf2003bc60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 1 --2- 172.21.15.178:0/3993127709 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf400a2930 0x7faf4011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 1 --2- 172.21.15.178:0/3993127709 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf400a08c0 0x7faf401318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 1 --2- 172.21.15.178:0/3993127709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7faf4009faa0 0x7faf4009b010 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 1 -- 172.21.15.178:0/3993127709 >> 172.21.15.178:0/3993127709 conn(0x7faf400a5ed0 msgr2=0x7faf40135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:10.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 1 -- 172.21.15.178:0/3993127709 shutdown_connections 2024-09-15T01:32:10.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.681+0000 7faf44c34640 1 -- 172.21.15.178:0/3993127709 wait complete. 2024-09-15T01:32:10.694 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-15T01:32:10.157729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562077696,"kb_used":4777156,"kb_used_data":557300696,"kb_used_omap":438,"kb_used_meta":0,"kb_avail":557300540,"statfs":{"total":575567560704,"available":570675752960,"internally_reserved":159744,"allocated":570675912704,"data_stored":570675912704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":448785,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.104548"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071141+0000","last_change":"2024-09-15T01:32:06.071392+0000","last_active":"2024-09-15T01:32:06.071141+0000","last_peered":"2024-09-15T01:32:06.071141+0000","last_clean":"2024-09-15T01:32:06.071141+0000","last_became_active":"2024-09-15T01:32:04.055827+0000","last_became_peered":"2024-09-15T01:32:04.055827+0000","last_unstale":"2024-09-15T01:32:06.071141+0000","last_undegraded":"2024-09-15T01:32:06.071141+0000","last_fullsized":"2024-09-15T01:32:06.071141+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:37:11.817932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065783200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071224+0000","last_change":"2024-09-15T01:32:06.071478+0000","last_active":"2024-09-15T01:32:06.071224+0000","last_peered":"2024-09-15T01:32:06.071224+0000","last_clean":"2024-09-15T01:32:06.071224+0000","last_became_active":"2024-09-15T01:32:04.057737+0000","last_became_peered":"2024-09-15T01:32:04.057737+0000","last_unstale":"2024-09-15T01:32:06.071224+0000","last_undegraded":"2024-09-15T01:32:06.071224+0000","last_fullsized":"2024-09-15T01:32:06.071224+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:56:30.331754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041942500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071380+0000","last_change":"2024-09-15T01:32:06.071587+0000","last_active":"2024-09-15T01:32:06.071380+0000","last_peered":"2024-09-15T01:32:06.071380+0000","last_clean":"2024-09-15T01:32:06.071380+0000","last_became_active":"2024-09-15T01:32:04.056136+0000","last_became_peered":"2024-09-15T01:32:04.056136+0000","last_unstale":"2024-09-15T01:32:06.071380+0000","last_undegraded":"2024-09-15T01:32:06.071380+0000","last_fullsized":"2024-09-15T01:32:06.071380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:54.622691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077134000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.156568+0000","last_change":"2024-09-15T01:32:06.156744+0000","last_active":"2024-09-15T01:32:06.156568+0000","last_peered":"2024-09-15T01:32:06.156568+0000","last_clean":"2024-09-15T01:32:06.156568+0000","last_became_active":"2024-09-15T01:32:04.057683+0000","last_became_peered":"2024-09-15T01:32:04.057683+0000","last_unstale":"2024-09-15T01:32:06.156568+0000","last_undegraded":"2024-09-15T01:32:06.156568+0000","last_fullsized":"2024-09-15T01:32:06.156568+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.248853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031034999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.075171+0000","last_change":"2024-09-15T01:32:06.075171+0000","last_active":"2024-09-15T01:32:06.075171+0000","last_peered":"2024-09-15T01:32:06.075171+0000","last_clean":"2024-09-15T01:32:06.075171+0000","last_became_active":"2024-09-15T01:32:04.057076+0000","last_became_peered":"2024-09-15T01:32:04.057076+0000","last_unstale":"2024-09-15T01:32:06.075171+0000","last_undegraded":"2024-09-15T01:32:06.075171+0000","last_fullsized":"2024-09-15T01:32:06.075171+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:06:21.344504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093022000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.072519+0000","last_change":"2024-09-15T01:32:06.072807+0000","last_active":"2024-09-15T01:32:06.072519+0000","last_peered":"2024-09-15T01:32:06.072519+0000","last_clean":"2024-09-15T01:32:06.072519+0000","last_became_active":"2024-09-15T01:32:04.058720+0000","last_became_peered":"2024-09-15T01:32:04.058720+0000","last_unstale":"2024-09-15T01:32:06.072519+0000","last_undegraded":"2024-09-15T01:32:06.072519+0000","last_fullsized":"2024-09-15T01:32:06.072519+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:48:11.629787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046258900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.072351+0000","last_change":"2024-09-15T01:32:06.072595+0000","last_active":"2024-09-15T01:32:06.072351+0000","last_peered":"2024-09-15T01:32:06.072351+0000","last_clean":"2024-09-15T01:32:06.072351+0000","last_became_active":"2024-09-15T01:32:04.058893+0000","last_became_peered":"2024-09-15T01:32:04.058893+0000","last_unstale":"2024-09-15T01:32:06.072351+0000","last_undegraded":"2024-09-15T01:32:06.072351+0000","last_fullsized":"2024-09-15T01:32:06.072351+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:38:08.540821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037511000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071100+0000","last_change":"2024-09-15T01:32:06.071341+0000","last_active":"2024-09-15T01:32:06.071100+0000","last_peered":"2024-09-15T01:32:06.071100+0000","last_clean":"2024-09-15T01:32:06.071100+0000","last_became_active":"2024-09-15T01:32:04.057931+0000","last_became_peered":"2024-09-15T01:32:04.057931+0000","last_unstale":"2024-09-15T01:32:06.071100+0000","last_undegraded":"2024-09-15T01:32:06.071100+0000","last_fullsized":"2024-09-15T01:32:06.071100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:10:36.309941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076976900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'72","reported_seq":103,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T01:32:05.068805+0000","last_change":"2024-09-15T01:32:00.032305+0000","last_active":"2024-09-15T01:32:05.068805+0000","last_peered":"2024-09-15T01:32:05.068805+0000","last_clean":"2024-09-15T01:32:05.068805+0000","last_became_active":"2024-09-15T01:32:00.032048+0000","last_became_peered":"2024-09-15T01:32:00.032048+0000","last_unstale":"2024-09-15T01:32:05.068805+0000","last_undegraded":"2024-09-15T01:32:05.068805+0000","last_fullsized":"2024-09-15T01:32:05.068805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_clean_scrub_stamp":"2024-09-15T01:31:59.015115+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:23:36.195853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796940,"kb_used_data":92882684,"kb_used_omap":100,"kb_used_meta":0,"kb_avail":92882676,"statfs":{"total":95927926784,"available":95111860224,"internally_reserved":8192,"allocated":95111868416,"data_stored":95111868416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":102448,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795820,"kb_used_data":92883832,"kb_used_omap":62,"kb_used_meta":0,"kb_avail":92883796,"statfs":{"total":95927926784,"available":95113007104,"internally_reserved":36864,"allocated":95113043968,"data_stored":95113043968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":64366,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795784,"kb_used_data":92883856,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92883832,"statfs":{"total":95927926784,"available":95113043968,"internally_reserved":24576,"allocated":95113068544,"data_stored":95113068544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":55119,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795784,"kb_used_data":92883856,"kb_used_omap":55,"kb_used_meta":0,"kb_avail":92883832,"statfs":{"total":95927926784,"available":95113043968,"internally_reserved":24576,"allocated":95113068544,"data_stored":95113068544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56457,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]}]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":797008,"kb_used_data":92882636,"kb_used_omap":104,"kb_used_meta":0,"kb_avail":92882608,"statfs":{"total":95927926784,"available":95111790592,"internally_reserved":28672,"allocated":95111819264,"data_stored":95111819264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":107425,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795820,"kb_used_data":92883832,"kb_used_omap":61,"kb_used_meta":0,"kb_avail":92883796,"statfs":{"total":95927926784,"available":95113007104,"internally_reserved":36864,"allocated":95113043968,"data_stored":95113043968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":62970,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.153}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.222}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[]}} 2024-09-15T01:32:10.695 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:32:10.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.786+0000 7ff0a4634640 10 monclient: get_monmap_and_config 2024-09-15T01:32:10.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.786+0000 7ff0a4634640 10 monclient: build_initial_monmap 2024-09-15T01:32:10.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.786+0000 7ff0a4634640 10 monclient: monmap: 2024-09-15T01:32:10.788 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:10.788 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.788 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:10.786951+0000 2024-09-15T01:32:10.788 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:10.786951+0000 2024-09-15T01:32:10.788 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:10.789 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.786+0000 7ff0a4634640 1 Processor -- start 2024-09-15T01:32:10.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.786+0000 7ff0a4634640 1 -- start start 2024-09-15T01:32:10.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.786+0000 7ff0a4634640 10 monclient: init 2024-09-15T01:32:10.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:10.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 10 monclient: _add_conns ranks=[3,2,1,5,7,4,0,8,6,9,10] 2024-09-15T01:32:10.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c0a2930 0x7ff09c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 10 monclient(hunting): picked mon.noname-g con 0x7ff09c0a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:10.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c09faa0 0x7ff09c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 10 monclient(hunting): picked mon.noname-b con 0x7ff09c09faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:10.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff09c0a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:10.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c04bf60 con 0x7ff09c0a08c0 2024-09-15T01:32:10.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c09e0f0 con 0x7ff09c09faa0 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c09dd80 con 0x7ff09c0a2930 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 10 monclient(hunting): authenticate will time out at 735.537390s 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c09faa0 0x7ff09c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c09faa0 0x7ff09c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:33728/0 (socket says 172.21.15.178:33728) 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 1 -- 172.21.15.178:0/3397164737 learned_addr learned my addr 172.21.15.178:0/3397164737 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 10 monclient(hunting): get_auth_request con 0x7ff09c09faa0 auth_method 0 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a23d0640 1 --2- 172.21.15.178:0/3397164737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c0a2930 0x7ff09c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 10 monclient(hunting): handle_auth_done global_id 4367 payload 274 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 1 -- 172.21.15.178:0/3397164737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c0a2930 msgr2=0x7ff09c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 1 --2- 172.21.15.178:0/3397164737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c0a2930 0x7ff09c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 1 -- 172.21.15.178:0/3397164737 >> v1:172.21.15.204:6789/0 conn(0x7ff09c0a08c0 legacy=0x7ff09c0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 10 monclient: _finish_hunting 0 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 1 monclient: found mon.noname-b 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 1 -- 172.21.15.178:0/3397164737 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0a2d00 con 0x7ff09c09faa0 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a23d0640 10 monclient: get_auth_request con 0x7ff09c0a2930 auth_method 0 2024-09-15T01:32:10.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a23d0640 1 --2- 172.21.15.178:0/3397164737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c0a2930 0x7ff09c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:32:10.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a1bcf640 1 --2- 172.21.15.178:0/3397164737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c09faa0 0x7ff09c09fe70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009940 tx=0x7ff08c0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c68079f69f8fc0db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a13ce640 1 -- 172.21.15.178:0/3397164737 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff08c02ad00 con 0x7ff09c09faa0 2024-09-15T01:32:10.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a13ce640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a13ce640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:10.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a13ce640 10 monclient: dump: 2024-09-15T01:32:10.801 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.801 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.801 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.801 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.801 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a13ce640 10 monclient: _finish_auth 0 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a13ce640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:10.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a13ce640 1 -- 172.21.15.178:0/3397164737 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c02ae80 con 0x7ff09c09faa0 2024-09-15T01:32:10.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 5 monclient: authenticate success, global_id 4367 2024-09-15T01:32:10.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:10.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a13ce640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:10.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:10.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a13ce640 1 -- 172.21.15.178:0/3397164737 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff08c031ca0 con 0x7ff09c09faa0 2024-09-15T01:32:10.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a13ce640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a4634640 4 set_mon_vals no callback set 2024-09-15T01:32:10.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a13ce640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:10.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.787+0000 7ff0a13ce640 10 monclient: dump: 2024-09-15T01:32:10.803 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.803 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.803 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.804 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.804 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.804 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.804 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.804 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.804 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.804 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.804 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.804 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.788+0000 7ff0a4634640 10 monclient: shutdown 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.788+0000 7ff0a4634640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.788+0000 7ff0a4634640 1 -- 172.21.15.178:0/3397164737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c09faa0 msgr2=0x7ff09c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.788+0000 7ff0a4634640 1 --2- 172.21.15.178:0/3397164737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c09faa0 0x7ff09c09fe70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009940 tx=0x7ff08c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.788+0000 7ff0a4634640 1 -- 172.21.15.178:0/3397164737 shutdown_connections 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.788+0000 7ff0a4634640 1 --2- 172.21.15.178:0/3397164737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c09faa0 0x7ff09c09fe70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.788+0000 7ff0a4634640 1 --2- 172.21.15.178:0/3397164737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c0a2930 0x7ff09c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.788+0000 7ff0a4634640 1 -- 172.21.15.178:0/3397164737 >> 172.21.15.178:0/3397164737 conn(0x7ff09c0a5ed0 msgr2=0x7ff09c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:10.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.788+0000 7ff0a4634640 1 -- 172.21.15.178:0/3397164737 shutdown_connections 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.788+0000 7ff0a4634640 1 -- 172.21.15.178:0/3397164737 wait complete. 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.788+0000 7ff0a4634640 10 monclient: build_initial_monmap 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.788+0000 7ff0a4634640 10 monclient: monmap: 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:10.806 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:10.807 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:10.807 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:10.807 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:10.807 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:10.807 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:10.807 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 1 Processor -- start 2024-09-15T01:32:10.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 1 -- start start 2024-09-15T01:32:10.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 10 monclient: init 2024-09-15T01:32:10.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:10.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 10 monclient: _add_conns ranks=[3,7,2,8,1,4,6,0,5] 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c09faa0 0x7ff09c09b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 10 monclient(hunting): picked mon.noname-g con 0x7ff09c09faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff09c0a08c0 0x7ff09c131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 10 monclient(hunting): picked mon.noname-i con 0x7ff09c0a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0a2930 0x7ff09c141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 10 monclient(hunting): picked mon.noname-b con 0x7ff09c0a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c11d700 con 0x7ff09c0a2930 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c09d630 con 0x7ff09c09faa0 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c11dae0 con 0x7ff09c0a08c0 2024-09-15T01:32:10.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a4634640 10 monclient(hunting): authenticate will time out at 735.540813s 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a2bd1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0a2930 0x7ff09c141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a2bd1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0a2930 0x7ff09c141d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:33732/0 (socket says 172.21.15.178:33732) 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a2bd1640 1 -- 172.21.15.178:0/2398435116 learned_addr learned my addr 172.21.15.178:0/2398435116 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a2bd1640 10 monclient(hunting): get_auth_request con 0x7ff09c0a2930 auth_method 0 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a2bd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a2bd1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a2bd1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a1bcf640 1 --2- 172.21.15.178:0/2398435116 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff09c0a08c0 0x7ff09c131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a23d0640 1 --2- 172.21.15.178:0/2398435116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c09faa0 0x7ff09c09b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a1bcf640 10 monclient(hunting): get_auth_request con 0x7ff09c0a08c0 auth_method 0 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a1bcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a1bcf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a1bcf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a2bd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.789+0000 7ff0a2bd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a2bd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a23d0640 10 monclient(hunting): get_auth_request con 0x7ff09c09faa0 auth_method 0 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a23d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a23d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a23d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a1bcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a1bcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a1bcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a23d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a23d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:10.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a23d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:10.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a2bd1640 10 monclient(hunting): handle_auth_done global_id 4376 payload 824 2024-09-15T01:32:10.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a2bd1640 1 -- 172.21.15.178:0/2398435116 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff09c0a08c0 msgr2=0x7ff09c131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a2bd1640 1 --2- 172.21.15.178:0/2398435116 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff09c0a08c0 0x7ff09c131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a2bd1640 1 -- 172.21.15.178:0/2398435116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c09faa0 msgr2=0x7ff09c09b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a2bd1640 1 --2- 172.21.15.178:0/2398435116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c09faa0 0x7ff09c09b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a2bd1640 10 monclient: _finish_hunting 0 2024-09-15T01:32:10.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a2bd1640 1 monclient: found mon.noname-b 2024-09-15T01:32:10.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a2bd1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:10.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a2bd1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:10.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a2bd1640 1 -- 172.21.15.178:0/2398435116 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c009450 con 0x7ff09c0a2930 2024-09-15T01:32:10.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a2bd1640 1 --2- 172.21.15.178:0/2398435116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0a2930 0x7ff09c141d40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff09400c860 tx=0x7ff094001690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0927fc640 1 -- 172.21.15.178:0/2398435116 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff09400ce50 con 0x7ff09c0a2930 2024-09-15T01:32:10.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0927fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:10.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0927fc640 10 monclient: dump: 2024-09-15T01:32:10.812 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.812 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.812 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.812 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.812 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.812 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.813 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.813 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.813 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.813 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.813 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.813 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.813 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.813 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.813 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.813 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0927fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:10.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:10.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a4634640 5 monclient: authenticate success, global_id 4376 2024-09-15T01:32:10.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0927fc640 1 -- 172.21.15.178:0/2398435116 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09400acd0 con 0x7ff09c0a2930 2024-09-15T01:32:10.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a4634640 10 monclient: _renew_subs 2024-09-15T01:32:10.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a4634640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:10.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a4634640 1 -- 172.21.15.178:0/2398435116 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c11ea40 con 0x7ff09c0a2930 2024-09-15T01:32:10.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0927fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:10.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0927fc640 1 -- 172.21.15.178:0/2398435116 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff09400ae40 con 0x7ff09c0a2930 2024-09-15T01:32:10.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:10.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0927fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:10.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a0bcd640 4 set_mon_vals no callback set 2024-09-15T01:32:10.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0927fc640 10 monclient: dump: 2024-09-15T01:32:10.815 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:10.815 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:10.815 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.815 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:10.815 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:10.815 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:10.815 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:10.815 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:10.815 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:10.815 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:10.815 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:10.815 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:10.816 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:10.816 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:10.816 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:10.816 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:10.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a4634640 10 monclient: _renew_subs 2024-09-15T01:32:10.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a4634640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:10.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.791+0000 7ff0a4634640 1 -- 172.21.15.178:0/2398435116 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c11f000 con 0x7ff09c0a2930 2024-09-15T01:32:10.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.792+0000 7ff0927fc640 1 -- 172.21.15.178:0/2398435116 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff094003030 con 0x7ff09c0a2930 2024-09-15T01:32:10.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.792+0000 7ff0a4634640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.792+0000 7ff0a4634640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.792+0000 7ff0a4634640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:10.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.792+0000 7ff0927fc640 1 --2- 172.21.15.178:0/2398435116 >> v2:172.21.15.204:6812/28210 conn(0x7ff0800396e0 0x7ff08003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:10.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.792+0000 7ff0a4634640 1 -- 172.21.15.178:0/2398435116 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c04b720 con 0x7ff09c0a2930 2024-09-15T01:32:10.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.792+0000 7ff0927fc640 1 -- 172.21.15.178:0/2398435116 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7ff09404aef0 con 0x7ff09c0a2930 2024-09-15T01:32:10.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.792+0000 7ff0a23d0640 1 --2- 172.21.15.178:0/2398435116 >> v2:172.21.15.204:6812/28210 conn(0x7ff0800396e0 0x7ff08003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:10.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.792+0000 7ff0a23d0640 10 monclient: get_auth_request con 0x7ff0800396e0 auth_method 0 2024-09-15T01:32:10.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.793+0000 7ff0a23d0640 1 --2- 172.21.15.178:0/2398435116 >> v2:172.21.15.204:6812/28210 conn(0x7ff0800396e0 0x7ff08003bb90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff09800acc0 tx=0x7ff098008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:10.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.795+0000 7ff0927fc640 1 -- 172.21.15.178:0/2398435116 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff094012ca0 con 0x7ff09c0a2930 2024-09-15T01:32:10.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.795+0000 7ff0927fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:10.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.795+0000 7ff0927fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:10.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.947+0000 7ff0a4634640 1 -- 172.21.15.178:0/2398435116 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff09c05ed70 con 0x7ff0800396e0 2024-09-15T01:32:10.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.949+0000 7ff0927fc640 1 -- 172.21.15.178:0/2398435116 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+39208 (secure 0 0 0) 0x7ff09c05ed70 con 0x7ff0800396e0 2024-09-15T01:32:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:32:10.952 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-09-15T01:32:10.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 1 -- 172.21.15.178:0/2398435116 >> v2:172.21.15.204:6812/28210 conn(0x7ff0800396e0 msgr2=0x7ff08003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 1 --2- 172.21.15.178:0/2398435116 >> v2:172.21.15.204:6812/28210 conn(0x7ff0800396e0 0x7ff08003bb90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff09800acc0 tx=0x7ff098008040 comp rx=0 tx=0).stop 2024-09-15T01:32:10.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 10 monclient: shutdown 2024-09-15T01:32:10.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:10.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 1 -- 172.21.15.178:0/2398435116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0a2930 msgr2=0x7ff09c141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:10.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 1 --2- 172.21.15.178:0/2398435116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0a2930 0x7ff09c141d40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff09400c860 tx=0x7ff094001690 comp rx=0 tx=0).stop 2024-09-15T01:32:10.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 1 -- 172.21.15.178:0/2398435116 shutdown_connections 2024-09-15T01:32:10.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 1 --2- 172.21.15.178:0/2398435116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff09c0a2930 0x7ff09c141d40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 1 --2- 172.21.15.178:0/2398435116 >> v2:172.21.15.204:6812/28210 conn(0x7ff0800396e0 0x7ff08003bb90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 1 --2- 172.21.15.178:0/2398435116 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff09c0a08c0 0x7ff09c131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 1 --2- 172.21.15.178:0/2398435116 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c09faa0 0x7ff09c09b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:10.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 1 -- 172.21.15.178:0/2398435116 >> 172.21.15.178:0/2398435116 conn(0x7ff09c0a5ed0 msgr2=0x7ff09c135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:10.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 1 -- 172.21.15.178:0/2398435116 shutdown_connections 2024-09-15T01:32:10.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:10.951+0000 7ff0a4634640 1 -- 172.21.15.178:0/2398435116 wait complete. 2024-09-15T01:32:10.964 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-15T01:32:10.157729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562077696,"kb_used":4777156,"kb_used_data":557300696,"kb_used_omap":438,"kb_used_meta":0,"kb_avail":557300540,"statfs":{"total":575567560704,"available":570675752960,"internally_reserved":159744,"allocated":570675912704,"data_stored":570675912704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":448785,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.104548"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071141+0000","last_change":"2024-09-15T01:32:06.071392+0000","last_active":"2024-09-15T01:32:06.071141+0000","last_peered":"2024-09-15T01:32:06.071141+0000","last_clean":"2024-09-15T01:32:06.071141+0000","last_became_active":"2024-09-15T01:32:04.055827+0000","last_became_peered":"2024-09-15T01:32:04.055827+0000","last_unstale":"2024-09-15T01:32:06.071141+0000","last_undegraded":"2024-09-15T01:32:06.071141+0000","last_fullsized":"2024-09-15T01:32:06.071141+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:37:11.817932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065783200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071224+0000","last_change":"2024-09-15T01:32:06.071478+0000","last_active":"2024-09-15T01:32:06.071224+0000","last_peered":"2024-09-15T01:32:06.071224+0000","last_clean":"2024-09-15T01:32:06.071224+0000","last_became_active":"2024-09-15T01:32:04.057737+0000","last_became_peered":"2024-09-15T01:32:04.057737+0000","last_unstale":"2024-09-15T01:32:06.071224+0000","last_undegraded":"2024-09-15T01:32:06.071224+0000","last_fullsized":"2024-09-15T01:32:06.071224+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:56:30.331754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041942500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071380+0000","last_change":"2024-09-15T01:32:06.071587+0000","last_active":"2024-09-15T01:32:06.071380+0000","last_peered":"2024-09-15T01:32:06.071380+0000","last_clean":"2024-09-15T01:32:06.071380+0000","last_became_active":"2024-09-15T01:32:04.056136+0000","last_became_peered":"2024-09-15T01:32:04.056136+0000","last_unstale":"2024-09-15T01:32:06.071380+0000","last_undegraded":"2024-09-15T01:32:06.071380+0000","last_fullsized":"2024-09-15T01:32:06.071380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:54.622691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077134000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.156568+0000","last_change":"2024-09-15T01:32:06.156744+0000","last_active":"2024-09-15T01:32:06.156568+0000","last_peered":"2024-09-15T01:32:06.156568+0000","last_clean":"2024-09-15T01:32:06.156568+0000","last_became_active":"2024-09-15T01:32:04.057683+0000","last_became_peered":"2024-09-15T01:32:04.057683+0000","last_unstale":"2024-09-15T01:32:06.156568+0000","last_undegraded":"2024-09-15T01:32:06.156568+0000","last_fullsized":"2024-09-15T01:32:06.156568+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.248853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031034999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.075171+0000","last_change":"2024-09-15T01:32:06.075171+0000","last_active":"2024-09-15T01:32:06.075171+0000","last_peered":"2024-09-15T01:32:06.075171+0000","last_clean":"2024-09-15T01:32:06.075171+0000","last_became_active":"2024-09-15T01:32:04.057076+0000","last_became_peered":"2024-09-15T01:32:04.057076+0000","last_unstale":"2024-09-15T01:32:06.075171+0000","last_undegraded":"2024-09-15T01:32:06.075171+0000","last_fullsized":"2024-09-15T01:32:06.075171+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:06:21.344504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093022000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.072519+0000","last_change":"2024-09-15T01:32:06.072807+0000","last_active":"2024-09-15T01:32:06.072519+0000","last_peered":"2024-09-15T01:32:06.072519+0000","last_clean":"2024-09-15T01:32:06.072519+0000","last_became_active":"2024-09-15T01:32:04.058720+0000","last_became_peered":"2024-09-15T01:32:04.058720+0000","last_unstale":"2024-09-15T01:32:06.072519+0000","last_undegraded":"2024-09-15T01:32:06.072519+0000","last_fullsized":"2024-09-15T01:32:06.072519+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:48:11.629787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046258900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.072351+0000","last_change":"2024-09-15T01:32:06.072595+0000","last_active":"2024-09-15T01:32:06.072351+0000","last_peered":"2024-09-15T01:32:06.072351+0000","last_clean":"2024-09-15T01:32:06.072351+0000","last_became_active":"2024-09-15T01:32:04.058893+0000","last_became_peered":"2024-09-15T01:32:04.058893+0000","last_unstale":"2024-09-15T01:32:06.072351+0000","last_undegraded":"2024-09-15T01:32:06.072351+0000","last_fullsized":"2024-09-15T01:32:06.072351+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:38:08.540821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037511000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071100+0000","last_change":"2024-09-15T01:32:06.071341+0000","last_active":"2024-09-15T01:32:06.071100+0000","last_peered":"2024-09-15T01:32:06.071100+0000","last_clean":"2024-09-15T01:32:06.071100+0000","last_became_active":"2024-09-15T01:32:04.057931+0000","last_became_peered":"2024-09-15T01:32:04.057931+0000","last_unstale":"2024-09-15T01:32:06.071100+0000","last_undegraded":"2024-09-15T01:32:06.071100+0000","last_fullsized":"2024-09-15T01:32:06.071100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:10:36.309941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076976900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'72","reported_seq":103,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T01:32:05.068805+0000","last_change":"2024-09-15T01:32:00.032305+0000","last_active":"2024-09-15T01:32:05.068805+0000","last_peered":"2024-09-15T01:32:05.068805+0000","last_clean":"2024-09-15T01:32:05.068805+0000","last_became_active":"2024-09-15T01:32:00.032048+0000","last_became_peered":"2024-09-15T01:32:00.032048+0000","last_unstale":"2024-09-15T01:32:05.068805+0000","last_undegraded":"2024-09-15T01:32:05.068805+0000","last_fullsized":"2024-09-15T01:32:05.068805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_clean_scrub_stamp":"2024-09-15T01:31:59.015115+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:23:36.195853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796940,"kb_used_data":92882684,"kb_used_omap":100,"kb_used_meta":0,"kb_avail":92882676,"statfs":{"total":95927926784,"available":95111860224,"internally_reserved":8192,"allocated":95111868416,"data_stored":95111868416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":102448,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795820,"kb_used_data":92883832,"kb_used_omap":62,"kb_used_meta":0,"kb_avail":92883796,"statfs":{"total":95927926784,"available":95113007104,"internally_reserved":36864,"allocated":95113043968,"data_stored":95113043968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":64366,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795784,"kb_used_data":92883856,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92883832,"statfs":{"total":95927926784,"available":95113043968,"internally_reserved":24576,"allocated":95113068544,"data_stored":95113068544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":55119,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795784,"kb_used_data":92883856,"kb_used_omap":55,"kb_used_meta":0,"kb_avail":92883832,"statfs":{"total":95927926784,"available":95113043968,"internally_reserved":24576,"allocated":95113068544,"data_stored":95113068544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56457,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]}]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":797008,"kb_used_data":92882636,"kb_used_omap":104,"kb_used_meta":0,"kb_avail":92882608,"statfs":{"total":95927926784,"available":95111790592,"internally_reserved":28672,"allocated":95111819264,"data_stored":95111819264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":107425,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795820,"kb_used_data":92883832,"kb_used_omap":61,"kb_used_meta":0,"kb_avail":92883796,"statfs":{"total":95927926784,"available":95113007104,"internally_reserved":36864,"allocated":95113043968,"data_stored":95113043968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":62970,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.153}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.222}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[]}} 2024-09-15T01:32:10.965 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T01:32:10.965 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T01:32:10.965 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T01:32:10.965 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T01:32:11.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.054+0000 7fc0d7289640 10 monclient: get_monmap_and_config 2024-09-15T01:32:11.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.054+0000 7fc0d7289640 10 monclient: build_initial_monmap 2024-09-15T01:32:11.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.054+0000 7fc0d7289640 10 monclient: monmap: 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:11.055717+0000 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:11.055717+0000 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:11.057 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:11.058 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:11.058 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:11.058 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:11.058 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:11.058 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.055+0000 7fc0d7289640 1 Processor -- start 2024-09-15T01:32:11.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.055+0000 7fc0d7289640 1 -- start start 2024-09-15T01:32:11.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.055+0000 7fc0d7289640 10 monclient: init 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 10 monclient: _add_conns ranks=[7,8,1,0,9,5,6,3,10,4,2] 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0d00a2930 0x7fc0d0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 10 monclient(hunting): picked mon.noname-i con 0x7fc0d00a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0d009faa0 0x7fc0d009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 10 monclient(hunting): picked mon.noname-e con 0x7fc0d009faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc0d00a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d004bf60 con 0x7fc0d00a08c0 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d009e0f0 con 0x7fc0d00a2930 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d009dd80 con 0x7fc0d009faa0 2024-09-15T01:32:11.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d7289640 10 monclient(hunting): authenticate will time out at 735.806253s 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0d009faa0 0x7fc0d009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0d009faa0 0x7fc0d009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:58006/0 (socket says 172.21.15.178:58006) 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 1 -- 172.21.15.178:0/2672629960 learned_addr learned my addr 172.21.15.178:0/2672629960 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 10 monclient(hunting): get_auth_request con 0x7fc0d009faa0 auth_method 0 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d5025640 1 --2- 172.21.15.178:0/2672629960 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0d00a2930 0x7fc0d0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d5025640 10 monclient(hunting): get_auth_request con 0x7fc0d00a2930 auth_method 0 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d5025640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d5025640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d5025640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 10 monclient(hunting): handle_auth_done global_id 4382 payload 274 2024-09-15T01:32:11.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 1 -- 172.21.15.178:0/2672629960 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0d00a2930 msgr2=0x7fc0d0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:11.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 1 --2- 172.21.15.178:0/2672629960 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0d00a2930 0x7fc0d0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 1 -- 172.21.15.178:0/2672629960 >> v1:172.21.15.204:6789/0 conn(0x7fc0d00a08c0 legacy=0x7fc0d00a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:11.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 10 monclient: _finish_hunting 0 2024-09-15T01:32:11.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 1 monclient: found mon.noname-e 2024-09-15T01:32:11.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:11.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:11.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.056+0000 7fc0d4824640 1 -- 172.21.15.178:0/2672629960 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d00a2d00 con 0x7fc0d009faa0 2024-09-15T01:32:11.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0d4824640 1 --2- 172.21.15.178:0/2672629960 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0d009faa0 0x7fc0d009fe70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009940 tx=0x7fc0c00091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=854c48c8914df7bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:11.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0c7fff640 1 -- 172.21.15.178:0/2672629960 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0c002ac70 con 0x7fc0d009faa0 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0c7fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0c7fff640 10 monclient: dump: 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.069 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0c7fff640 10 monclient: _finish_auth 0 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0d7289640 5 monclient: authenticate success, global_id 4382 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0c7fff640 1 -- 172.21.15.178:0/2672629960 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c002adf0 con 0x7fc0d009faa0 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0d7289640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0c7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0d7289640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0c7fff640 1 -- 172.21.15.178:0/2672629960 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0c0031ca0 con 0x7fc0d009faa0 2024-09-15T01:32:11.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0d7289640 4 set_mon_vals no callback set 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0c7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.057+0000 7fc0c7fff640 10 monclient: dump: 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.071 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 10 monclient: shutdown 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 1 -- 172.21.15.178:0/2672629960 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0d009faa0 msgr2=0x7fc0d009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 1 --2- 172.21.15.178:0/2672629960 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0d009faa0 0x7fc0d009fe70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009940 tx=0x7fc0c00091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 1 -- 172.21.15.178:0/2672629960 shutdown_connections 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 1 --2- 172.21.15.178:0/2672629960 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0d009faa0 0x7fc0d009fe70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 1 --2- 172.21.15.178:0/2672629960 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0d00a2930 0x7fc0d0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 1 -- 172.21.15.178:0/2672629960 >> 172.21.15.178:0/2672629960 conn(0x7fc0d00a5ed0 msgr2=0x7fc0d00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 1 -- 172.21.15.178:0/2672629960 shutdown_connections 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 1 -- 172.21.15.178:0/2672629960 wait complete. 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 10 monclient: build_initial_monmap 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 10 monclient: monmap: 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:11.073 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.058+0000 7fc0d7289640 1 Processor -- start 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 1 -- start start 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 10 monclient: init 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 10 monclient: _add_conns ranks=[2,7,4,0,5,8,3,6,1] 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0d009faa0 0x7fc0d009b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 10 monclient(hunting): picked mon.noname-b con 0x7fc0d009faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0d00a08c0 0x7fc0d0131970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 10 monclient(hunting): picked mon.noname-i con 0x7fc0d00a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc0d00a2930 0x7fc0d0131eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 10 monclient(hunting): picked mon.noname-c con 0x7fc0d00a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d009bee0 con 0x7fc0d009faa0 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d009d630 con 0x7fc0d00a2930 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d011d400 con 0x7fc0d00a08c0 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d7289640 10 monclient(hunting): authenticate will time out at 735.809597s 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5826640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc0d00a2930 0x7fc0d0131eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5025640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0d009faa0 0x7fc0d009b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5826640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc0d00a2930 0x7fc0d0131eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:36802/0 (socket says 172.21.15.178:36802) 2024-09-15T01:32:11.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5025640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0d009faa0 0x7fc0d009b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:33736/0 (socket says 172.21.15.178:33736) 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5826640 1 -- 172.21.15.178:0/185808776 learned_addr learned my addr 172.21.15.178:0/185808776 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5826640 10 monclient(hunting): get_auth_request con 0x7fc0d00a2930 auth_method 0 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5826640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5826640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5826640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5025640 10 monclient(hunting): get_auth_request con 0x7fc0d009faa0 auth_method 0 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d4824640 1 --2- 172.21.15.178:0/185808776 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0d00a08c0 0x7fc0d0131970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5025640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5025640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5025640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d4824640 10 monclient(hunting): get_auth_request con 0x7fc0d00a08c0 auth_method 0 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d4824640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d4824640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d4824640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5826640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5826640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5826640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5025640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5025640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d5025640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d4824640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d4824640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.059+0000 7fc0d4824640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d5025640 10 monclient(hunting): handle_auth_done global_id 4385 payload 824 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d5025640 1 -- 172.21.15.178:0/185808776 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0d00a08c0 msgr2=0x7fc0d0131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d5025640 1 --2- 172.21.15.178:0/185808776 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0d00a08c0 0x7fc0d0131970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d5025640 1 -- 172.21.15.178:0/185808776 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc0d00a2930 msgr2=0x7fc0d0131eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d5025640 1 --2- 172.21.15.178:0/185808776 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc0d00a2930 0x7fc0d0131eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d5025640 10 monclient: _finish_hunting 0 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d5025640 1 monclient: found mon.noname-b 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d5025640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d5025640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d5025640 1 -- 172.21.15.178:0/185808776 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0009450 con 0x7fc0d009faa0 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d5826640 1 --2- 172.21.15.178:0/185808776 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc0d00a2930 0x7fc0d0131eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d4824640 1 --2- 172.21.15.178:0/185808776 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0d00a08c0 0x7fc0d0131970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d5025640 1 --2- 172.21.15.178:0/185808776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0d009faa0 0x7fc0d009b350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00d890 tx=0x7fc0cc008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c57fa640 1 -- 172.21.15.178:0/185808776 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0cc00bb40 con 0x7fc0d009faa0 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c57fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c57fa640 10 monclient: dump: 2024-09-15T01:32:11.078 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.079 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c57fa640 10 monclient: _finish_auth 0 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d7289640 5 monclient: authenticate success, global_id 4385 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c57fa640 1 -- 172.21.15.178:0/185808776 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0cc00bce0 con 0x7fc0d009faa0 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d7289640 10 monclient: _renew_subs 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d7289640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d7289640 1 -- 172.21.15.178:0/185808776 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d011eab0 con 0x7fc0d009faa0 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c57fa640 1 -- 172.21.15.178:0/185808776 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0cc00be50 con 0x7fc0d009faa0 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c77fe640 4 set_mon_vals no callback set 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0c57fa640 10 monclient: dump: 2024-09-15T01:32:11.080 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.081 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d7289640 10 monclient: _renew_subs 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d7289640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.060+0000 7fc0d7289640 1 -- 172.21.15.178:0/185808776 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d011db50 con 0x7fc0d009faa0 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.061+0000 7fc0c57fa640 1 -- 172.21.15.178:0/185808776 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc0cc00b6c0 con 0x7fc0d009faa0 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.061+0000 7fc0c57fa640 1 --2- 172.21.15.178:0/185808776 >> v2:172.21.15.204:6812/28210 conn(0x7fc0b40396e0 0x7fc0b403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.061+0000 7fc0c57fa640 1 -- 172.21.15.178:0/185808776 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7fc0cc04b3f0 con 0x7fc0d009faa0 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.061+0000 7fc0d7289640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.061+0000 7fc0d7289640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.061+0000 7fc0d7289640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.061+0000 7fc0d7289640 1 -- 172.21.15.178:0/185808776 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d011f190 con 0x7fc0d009faa0 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.061+0000 7fc0d4824640 1 --2- 172.21.15.178:0/185808776 >> v2:172.21.15.204:6812/28210 conn(0x7fc0b40396e0 0x7fc0b403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.061+0000 7fc0d4824640 10 monclient: get_auth_request con 0x7fc0b40396e0 auth_method 0 2024-09-15T01:32:11.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.062+0000 7fc0d4824640 1 --2- 172.21.15.178:0/185808776 >> v2:172.21.15.204:6812/28210 conn(0x7fc0b40396e0 0x7fc0b403bb90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0034000 tx=0x7fc0c0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:11.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.065+0000 7fc0c57fa640 1 -- 172.21.15.178:0/185808776 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc0cc014430 con 0x7fc0d009faa0 2024-09-15T01:32:11.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.065+0000 7fc0c57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:11.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.065+0000 7fc0c57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:11.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.256+0000 7fc0d7289640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2024-09-15T01:32:11.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.256+0000 7fc0d7289640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2024-09-15T01:32:11.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.256+0000 7fc0d7289640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:11.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.256+0000 7fc0d7289640 1 -- 172.21.15.178:0/185808776 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc0d011f470 con 0x7fc0d009faa0 2024-09-15T01:32:11.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.257+0000 7fc0c57fa640 1 -- 172.21.15.178:0/185808776 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc0cc02b040 con 0x7fc0d009faa0 2024-09-15T01:32:11.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.257+0000 7fc0c57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2024-09-15T01:32:11.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.257+0000 7fc0c57fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:32:11.259 INFO:teuthology.orchestra.run.smithi178.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T01:32:11.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 1 -- 172.21.15.178:0/185808776 >> v2:172.21.15.204:6812/28210 conn(0x7fc0b40396e0 msgr2=0x7fc0b403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 1 --2- 172.21.15.178:0/185808776 >> v2:172.21.15.204:6812/28210 conn(0x7fc0b40396e0 0x7fc0b403bb90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0034000 tx=0x7fc0c0032000 comp rx=0 tx=0).stop 2024-09-15T01:32:11.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 10 monclient: shutdown 2024-09-15T01:32:11.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:11.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 1 -- 172.21.15.178:0/185808776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0d009faa0 msgr2=0x7fc0d009b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 1 --2- 172.21.15.178:0/185808776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0d009faa0 0x7fc0d009b350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00d890 tx=0x7fc0cc008040 comp rx=0 tx=0).stop 2024-09-15T01:32:11.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 1 -- 172.21.15.178:0/185808776 shutdown_connections 2024-09-15T01:32:11.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 1 --2- 172.21.15.178:0/185808776 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc0d00a2930 0x7fc0d0131eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 1 --2- 172.21.15.178:0/185808776 >> v2:172.21.15.204:6812/28210 conn(0x7fc0b40396e0 0x7fc0b403bb90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 1 --2- 172.21.15.178:0/185808776 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0d00a08c0 0x7fc0d0131970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 1 --2- 172.21.15.178:0/185808776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0d009faa0 0x7fc0d009b350 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 1 -- 172.21.15.178:0/185808776 >> 172.21.15.178:0/185808776 conn(0x7fc0d00a5ed0 msgr2=0x7fc0d00a4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:11.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 1 -- 172.21.15.178:0/185808776 shutdown_connections 2024-09-15T01:32:11.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.259+0000 7fc0d7289640 1 -- 172.21.15.178:0/185808776 wait complete. 2024-09-15T01:32:11.271 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T01:32:11.272 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-09-15T01:32:11.284 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-09-15T01:32:11.285 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:32:11.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.374+0000 7f72d691d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.374+0000 7f72d691d640 10 monclient: build_initial_monmap 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.374+0000 7f72d691d640 10 monclient: monmap: 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:11.375818+0000 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:11.375818+0000 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:11.377 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:11.378 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:11.378 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:11.378 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:11.378 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:11.378 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:11.378 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:11.378 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.375+0000 7f72d691d640 1 Processor -- start 2024-09-15T01:32:11.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.375+0000 7f72d691d640 1 -- start start 2024-09-15T01:32:11.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.375+0000 7f72d691d640 10 monclient: init 2024-09-15T01:32:11.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:11.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 10 monclient: _add_conns ranks=[3,1,0,4,9,8,5,7,10,6,2] 2024-09-15T01:32:11.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72d00a2930 0x7f72d0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 10 monclient(hunting): picked mon.noname-g con 0x7f72d00a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:11.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f72d009faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f72d009fe70 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d004bf60 con 0x7f72d009fe70 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d009dd80 con 0x7f72d009faa0 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00a2d00 con 0x7f72d00a2930 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 10 monclient(hunting): authenticate will time out at 736.126247s 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d4eba640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f72d009fe70 0x7f72d00a0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:49282/0 (socket says 172.21.15.178:49282) 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d4eba640 1 -- 172.21.15.178:0/1798860636 learned_addr learned my addr 172.21.15.178:0/1798860636 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234160929 0 0) 0x7f72d004bf60 con 0x7f72d009fe70 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72cffff640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72d00a2930 0x7f72d0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 10 monclient(hunting): my global_id is 4419 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72bc0034e0 con 0x7f72d009fe70 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72cffff640 10 monclient(hunting): get_auth_request con 0x7f72d00a2930 auth_method 0 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 72763571 0 0) 0x7f72bc0034e0 con 0x7f72d009fe70 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72d00a2930 msgr2=0x7f72d0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72d00a2930 0x7f72d0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 >> v1:172.21.15.204:6789/0 conn(0x7f72d009faa0 legacy=0x7f72d00a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 10 monclient: _finish_hunting 0 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d00a16e0 con 0x7f72d009fe70 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 10 monclient: _finish_auth 0 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 5 monclient: authenticate success, global_id 4419 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f72c8001c70 con 0x7f72d009fe70 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72d691d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 10 monclient: dump: 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.391 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 >> v1:172.21.15.178:6789/0 conn(0x7f72d009fe70 legacy=0x7f72d00a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 10 monclient: _add_conns ranks=[8,2,0,5,6,7,4,1,3] 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.376+0000 7f72ceffd640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f72bc004250 0x7f72bc004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 10 monclient(hunting): picked mon.e con 0x7f72bc004250 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72bc004cc0 0x7f72bc005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 10 monclient(hunting): picked mon.b con 0x7f72bc004cc0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc009c10 0x7f72bc00c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 10 monclient(hunting): picked mon.a con 0x7f72bc009c10 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00a2d00 con 0x7f72bc009c10 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc00c970 con 0x7f72bc004cc0 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc00cae0 con 0x7f72bc004250 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72ceffd640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d691d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc009c10 0x7f72bc00c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cf7fe640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72bc004cc0 0x7f72bc005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 10 monclient(hunting): get_auth_request con 0x7f72bc009c10 auth_method 0 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cf7fe640 10 monclient(hunting): get_auth_request con 0x7f72bc004cc0 auth_method 0 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cf7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cffff640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f72bc004250 0x7f72bc004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cffff640 10 monclient(hunting): get_auth_request con 0x7f72bc004250 auth_method 0 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 10 monclient(hunting): handle_auth_done global_id 4419 payload 293 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 1 -- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f72bc004250 msgr2=0x7f72bc004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f72bc004250 0x7f72bc004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 1 -- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72bc004cc0 msgr2=0x7f72bc005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72bc004cc0 0x7f72bc005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 10 monclient: _finish_hunting 0 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 1 monclient: found mon.a 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 1 -- 172.21.15.178:0/1798860636 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc0034e0 con 0x7f72bc009c10 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 10 monclient: _finish_auth 0 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72d4eba640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.377+0000 7f72cf7fe640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72bc004cc0 0x7f72bc005130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d4eba640 1 -- 172.21.15.178:0/1798860636 reap_dead start 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d4eba640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc009c10 0x7f72bc00c090 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f72c800c720 tx=0x7f72c80019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71779199fc13a126 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72c8007570 con 0x7f72bc009c10 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72ceffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72ceffd640 10 monclient: dump: 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c8007710 con 0x7f72bc009c10 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d691d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72ceffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72ceffd640 1 -- 172.21.15.178:0/1798860636 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72c8007880 con 0x7f72bc009c10 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d691d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:11.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d691d640 4 set_mon_vals no callback set 2024-09-15T01:32:11.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72ceffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:11.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72ceffd640 10 monclient: dump: 2024-09-15T01:32:11.399 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.399 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.399 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.399 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.399 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.399 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.399 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.399 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.399 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d691d640 10 monclient: shutdown 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d691d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d691d640 1 -- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc009c10 msgr2=0x7f72bc00c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d691d640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc009c10 0x7f72bc00c090 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f72c800c720 tx=0x7f72c80019d0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d691d640 1 -- 172.21.15.178:0/1798860636 shutdown_connections 2024-09-15T01:32:11.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d691d640 1 --2- 172.21.15.178:0/1798860636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc009c10 0x7f72bc00c090 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d691d640 1 -- 172.21.15.178:0/1798860636 >> 172.21.15.178:0/1798860636 conn(0x7f72d00a5ed0 msgr2=0x7f72d00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d691d640 1 -- 172.21.15.178:0/1798860636 shutdown_connections 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.378+0000 7f72d691d640 1 -- 172.21.15.178:0/1798860636 wait complete. 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.379+0000 7f72d691d640 10 monclient: build_initial_monmap 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.379+0000 7f72d691d640 10 monclient: monmap: 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:11.401 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.379+0000 7f72d691d640 1 Processor -- start 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.379+0000 7f72d691d640 1 -- start start 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.379+0000 7f72d691d640 10 monclient: init 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 10 monclient: _add_conns ranks=[7,0,6,5,2,3,1,8,4] 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f72bc009c10 0x7f72d01316b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 10 monclient(hunting): picked mon.noname-i con 0x7f72bc009c10 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:11.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d0131bf0 0x7f72d011e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 10 monclient(hunting): picked mon.noname-a con 0x7f72d0131bf0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f72d011e770 0x7f72d011eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 10 monclient(hunting): picked mon.noname-d con 0x7f72d011e770 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d0131fc0 con 0x7f72d0131bf0 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f72d009f870 con 0x7f72d011e770 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f72d0120490 con 0x7f72bc009c10 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d691d640 10 monclient(hunting): authenticate will time out at 736.130314s 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d0131bf0 0x7f72d011e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d4eba640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f72d011e770 0x7f72d011eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d0131bf0 0x7f72d011e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35030/0 (socket says 172.21.15.178:35030) 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 1 -- 172.21.15.178:0/2490018316 learned_addr learned my addr 172.21.15.178:0/2490018316 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 10 monclient(hunting): get_auth_request con 0x7f72d0131bf0 auth_method 0 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d4eba640 10 monclient(hunting): get_auth_request con 0x7f72d011e770 auth_method 0 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d4eba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d4eba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d4eba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cffff640 1 --2- 172.21.15.178:0/2490018316 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f72bc009c10 0x7f72d01316b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cffff640 10 monclient(hunting): get_auth_request con 0x7f72bc009c10 auth_method 0 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d4eba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d4eba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d4eba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 10 monclient(hunting): handle_auth_done global_id 4428 payload 824 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 1 -- 172.21.15.178:0/2490018316 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f72bc009c10 msgr2=0x7f72d01316b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 1 --2- 172.21.15.178:0/2490018316 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f72bc009c10 0x7f72d01316b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 1 -- 172.21.15.178:0/2490018316 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f72d011e770 msgr2=0x7f72d011eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 1 --2- 172.21.15.178:0/2490018316 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f72d011e770 0x7f72d011eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 1 monclient: found mon.noname-a 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 1 -- 172.21.15.178:0/2490018316 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d009d630 con 0x7f72d0131bf0 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72d4eba640 1 --2- 172.21.15.178:0/2490018316 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f72d011e770 0x7f72d011eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72cf7fe640 1 --2- 172.21.15.178:0/2490018316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d0131bf0 0x7f72d011e230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f72b80077f0 tx=0x7f72b8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.380+0000 7f72c7fff640 1 -- 172.21.15.178:0/2490018316 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72b800fe60 con 0x7f72d0131bf0 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72c7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72c7fff640 10 monclient: dump: 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72c7fff640 10 monclient: _finish_auth 0 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72d691d640 5 monclient: authenticate success, global_id 4428 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72c7fff640 1 -- 172.21.15.178:0/2490018316 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800dbc0 con 0x7f72d0131bf0 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72d691d640 10 monclient: _renew_subs 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72d691d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72d691d640 1 -- 172.21.15.178:0/2490018316 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d01208a0 con 0x7f72d0131bf0 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72c7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72c7fff640 1 -- 172.21.15.178:0/2490018316 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72b800dd30 con 0x7f72d0131bf0 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72c7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72ce7fc640 4 set_mon_vals no callback set 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72c7fff640 10 monclient: dump: 2024-09-15T01:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.409 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72d691d640 10 monclient: _renew_subs 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72d691d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72d691d640 1 -- 172.21.15.178:0/2490018316 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d0142780 con 0x7f72d0131bf0 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.381+0000 7f72c7fff640 1 -- 172.21.15.178:0/2490018316 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f72b801d460 con 0x7f72d0131bf0 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.382+0000 7f72d691d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.382+0000 7f72d691d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.382+0000 7f72d691d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.382+0000 7f72c7fff640 1 --2- 172.21.15.178:0/2490018316 >> v2:172.21.15.204:6812/28210 conn(0x7f72b00396e0 0x7f72b003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.382+0000 7f72d691d640 1 -- 172.21.15.178:0/2490018316 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d004b720 con 0x7f72d0131bf0 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.382+0000 7f72c7fff640 1 -- 172.21.15.178:0/2490018316 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3558+0+0 (secure 0 0 0) 0x7f72b804b360 con 0x7f72d0131bf0 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.382+0000 7f72cffff640 1 --2- 172.21.15.178:0/2490018316 >> v2:172.21.15.204:6812/28210 conn(0x7f72b00396e0 0x7f72b003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.382+0000 7f72cffff640 10 monclient: get_auth_request con 0x7f72b00396e0 auth_method 0 2024-09-15T01:32:11.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.382+0000 7f72cffff640 1 --2- 172.21.15.178:0/2490018316 >> v2:172.21.15.204:6812/28210 conn(0x7f72b00396e0 0x7f72b003bb90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f72c0006e70 tx=0x7f72c00090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:11.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.386+0000 7f72c7fff640 1 -- 172.21.15.178:0/2490018316 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f72b801d8f0 con 0x7f72d0131bf0 2024-09-15T01:32:11.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.386+0000 7f72c7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:11.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.386+0000 7f72c7fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:11.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.542+0000 7f72d691d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:32:11.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.542+0000 7f72d691d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:32:11.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.542+0000 7f72d691d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:11.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.542+0000 7f72d691d640 1 -- 172.21.15.178:0/2490018316 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72d00a19b0 con 0x7f72d0131bf0 2024-09-15T01:32:11.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.543+0000 7f72c7fff640 1 -- 172.21.15.178:0/2490018316 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+10067 (secure 0 0 0) 0x7f72b801a310 con 0x7f72d0131bf0 2024-09-15T01:32:11.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.543+0000 7f72c7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:32:11.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.543+0000 7f72c7fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:32:11.545 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":20,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","created":"2024-09-15T01:31:51.657329+0000","modified":"2024-09-15T01:32:11.420634+0000","last_up_change":"2024-09-15T01:31:56.999458+0000","last_in_change":"2024-09-15T01:31:53.995822+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:31:58.183767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:32:02.130544+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"690d2bc0-3a5e-4f4a-b00b-43a73741b7b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":29232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6801","nonce":29232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":29232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":29232}]},"public_addr":"172.21.15.178:6800/29232","cluster_addr":"172.21.15.178:6801/29232","heartbeat_back_addr":"172.21.15.178:6803/29232","heartbeat_front_addr":"172.21.15.178:6802/29232","state":["exists","up"]},{"osd":1,"uuid":"bbba9997-deb4-42ca-8140-6553a9b6c45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":29233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6805","nonce":29233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6807","nonce":29233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":29233}]},"public_addr":"172.21.15.178:6804/29233","cluster_addr":"172.21.15.178:6805/29233","heartbeat_back_addr":"172.21.15.178:6807/29233","heartbeat_front_addr":"172.21.15.178:6806/29233","state":["exists","up"]},{"osd":2,"uuid":"24b56e74-a222-40f9-a8bf-38d597553f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":29234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":29234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6811","nonce":29234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":29234}]},"public_addr":"172.21.15.178:6808/29234","cluster_addr":"172.21.15.178:6809/29234","heartbeat_back_addr":"172.21.15.178:6811/29234","heartbeat_front_addr":"172.21.15.178:6810/29234","state":["exists","up"]},{"osd":3,"uuid":"a5bf3f7d-4c59-470a-8f69-3fea018c13f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":28752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":28752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":28752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":28752}]},"public_addr":"172.21.15.204:6801/28752","cluster_addr":"172.21.15.204:6802/28752","heartbeat_back_addr":"172.21.15.204:6806/28752","heartbeat_front_addr":"172.21.15.204:6804/28752","state":["exists","up"]},{"osd":4,"uuid":"cdaf5723-3a5c-4238-a8e6-38f2325ccbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":28753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":28753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":28753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6805","nonce":28753}]},"public_addr":"172.21.15.204:6800/28753","cluster_addr":"172.21.15.204:6803/28753","heartbeat_back_addr":"172.21.15.204:6807/28753","heartbeat_front_addr":"172.21.15.204:6805/28753","state":["exists","up"]},{"osd":5,"uuid":"6b67035f-8718-4762-98bf-0dd0aa86e6e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":28754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":28754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":28754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":28754}]},"public_addr":"172.21.15.204:6808/28754","cluster_addr":"172.21.15.204:6809/28754","heartbeat_back_addr":"172.21.15.204:6811/28754","heartbeat_front_addr":"172.21.15.204:6810/28754","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.093711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.117496+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.123704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.082038+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.088260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.090342+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:32:11.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.544+0000 7f72d691d640 1 -- 172.21.15.178:0/2490018316 >> v2:172.21.15.204:6812/28210 conn(0x7f72b00396e0 msgr2=0x7f72b003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.544+0000 7f72d691d640 1 --2- 172.21.15.178:0/2490018316 >> v2:172.21.15.204:6812/28210 conn(0x7f72b00396e0 0x7f72b003bb90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f72c0006e70 tx=0x7f72c00090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.544+0000 7f72d691d640 10 monclient: shutdown 2024-09-15T01:32:11.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.544+0000 7f72d691d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:11.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.544+0000 7f72d691d640 1 -- 172.21.15.178:0/2490018316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d0131bf0 msgr2=0x7f72d011e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.544+0000 7f72d691d640 1 --2- 172.21.15.178:0/2490018316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d0131bf0 0x7f72d011e230 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f72b80077f0 tx=0x7f72b8009040 comp rx=0 tx=0).stop 2024-09-15T01:32:11.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.544+0000 7f72d691d640 1 -- 172.21.15.178:0/2490018316 shutdown_connections 2024-09-15T01:32:11.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.544+0000 7f72d691d640 1 --2- 172.21.15.178:0/2490018316 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f72d011e770 0x7f72d011eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.544+0000 7f72d691d640 1 --2- 172.21.15.178:0/2490018316 >> v2:172.21.15.204:6812/28210 conn(0x7f72b00396e0 0x7f72b003bb90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.544+0000 7f72d691d640 1 --2- 172.21.15.178:0/2490018316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72d0131bf0 0x7f72d011e230 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.544+0000 7f72d691d640 1 --2- 172.21.15.178:0/2490018316 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f72bc009c10 0x7f72d01316b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.544+0000 7f72d691d640 1 -- 172.21.15.178:0/2490018316 >> 172.21.15.178:0/2490018316 conn(0x7f72d00a5ed0 msgr2=0x7f72d009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:11.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.545+0000 7f72d691d640 1 -- 172.21.15.178:0/2490018316 shutdown_connections 2024-09-15T01:32:11.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.545+0000 7f72d691d640 1 -- 172.21.15.178:0/2490018316 wait complete. 2024-09-15T01:32:11.557 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T01:31:58.183767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T01:32:02.130544+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-09-15T01:32:11.557 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-15T01:32:11.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.652+0000 7fd44379d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:11.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.652+0000 7fd44379d640 10 monclient: build_initial_monmap 2024-09-15T01:32:11.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.652+0000 7fd44379d640 10 monclient: monmap: 2024-09-15T01:32:11.654 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:11.655 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.655 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:11.653680+0000 2024-09-15T01:32:11.655 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:11.653680+0000 2024-09-15T01:32:11.655 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:11.655 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.655 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:11.655 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:11.655 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:11.655 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:11.655 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:11.655 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:11.655 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:11.656 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:11.656 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:11.656 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:11.656 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:11.656 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.653+0000 7fd44379d640 1 Processor -- start 2024-09-15T01:32:11.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.653+0000 7fd44379d640 1 -- start start 2024-09-15T01:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.653+0000 7fd44379d640 10 monclient: init 2024-09-15T01:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 10 monclient: _add_conns ranks=[9,4,8,0,3,1,6,10,5,7,2] 2024-09-15T01:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd43c0a2930 0x7fd43c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 10 monclient(hunting): picked mon.noname-a con 0x7fd43c0a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd43c09faa0 0x7fd43c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 10 monclient(hunting): picked mon.noname-c con 0x7fd43c09faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd43c0a08c0 0x7fd43c0a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 10 monclient(hunting): picked mon.noname-e con 0x7fd43c0a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43c09e0f0 con 0x7fd43c0a2930 2024-09-15T01:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c09dd80 con 0x7fd43c09faa0 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c0a2d00 con 0x7fd43c0a08c0 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd44379d640 10 monclient(hunting): authenticate will time out at 736.404160s 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441539640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd43c0a2930 0x7fd43c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441d3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd43c0a08c0 0x7fd43c0a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd43c09faa0 0x7fd43c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441d3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd43c0a08c0 0x7fd43c0a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:58022/0 (socket says 172.21.15.178:58022) 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441539640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd43c0a2930 0x7fd43c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35046/0 (socket says 172.21.15.178:35046) 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441d3a640 1 -- 172.21.15.178:0/3389913434 learned_addr learned my addr 172.21.15.178:0/3389913434 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441d3a640 10 monclient(hunting): get_auth_request con 0x7fd43c0a08c0 auth_method 0 2024-09-15T01:32:11.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441d3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441d3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441d3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441539640 10 monclient(hunting): get_auth_request con 0x7fd43c0a2930 auth_method 0 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441539640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441539640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 10 monclient(hunting): get_auth_request con 0x7fd43c09faa0 auth_method 0 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441d3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441d3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441d3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441539640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441539640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441539640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 10 monclient(hunting): handle_auth_done global_id 4369 payload 274 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 1 -- 172.21.15.178:0/3389913434 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd43c0a08c0 msgr2=0x7fd43c0a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 1 --2- 172.21.15.178:0/3389913434 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd43c0a08c0 0x7fd43c0a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 1 -- 172.21.15.178:0/3389913434 >> v2:172.21.15.178:3300/0 conn(0x7fd43c0a2930 msgr2=0x7fd43c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 1 --2- 172.21.15.178:0/3389913434 >> v2:172.21.15.178:3300/0 conn(0x7fd43c0a2930 0x7fd43c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 10 monclient: _finish_hunting 0 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 1 monclient: found mon.noname-c 2024-09-15T01:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd440d38640 1 -- 172.21.15.178:0/3389913434 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c04bf60 con 0x7fd43c09faa0 2024-09-15T01:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441d3a640 1 --2- 172.21.15.178:0/3389913434 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd43c0a08c0 0x7fd43c0a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.654+0000 7fd441539640 1 --2- 172.21.15.178:0/3389913434 >> v2:172.21.15.178:3300/0 conn(0x7fd43c0a2930 0x7fd43c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd440d38640 1 --2- 172.21.15.178:0/3389913434 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd43c09faa0 0x7fd43c09fe70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009850 tx=0x7fd42c0090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=ab36caf087b277d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd433fff640 1 -- 172.21.15.178:0/3389913434 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd42c02abc0 con 0x7fd43c09faa0 2024-09-15T01:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd433fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd433fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd433fff640 10 monclient: dump: 2024-09-15T01:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd433fff640 10 monclient: _finish_auth 0 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd433fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd44379d640 5 monclient: authenticate success, global_id 4369 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd433fff640 1 -- 172.21.15.178:0/3389913434 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42c02ad40 con 0x7fd43c09faa0 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd44379d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd433fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd44379d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd44379d640 4 set_mon_vals no callback set 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd433fff640 1 -- 172.21.15.178:0/3389913434 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd42c031ca0 con 0x7fd43c09faa0 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd433fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd433fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.655+0000 7fd433fff640 10 monclient: dump: 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.671 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 10 monclient: shutdown 2024-09-15T01:32:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 1 -- 172.21.15.178:0/3389913434 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd43c09faa0 msgr2=0x7fd43c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 1 --2- 172.21.15.178:0/3389913434 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd43c09faa0 0x7fd43c09fe70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009850 tx=0x7fd42c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 1 -- 172.21.15.178:0/3389913434 shutdown_connections 2024-09-15T01:32:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 1 --2- 172.21.15.178:0/3389913434 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd43c0a08c0 0x7fd43c0a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 1 --2- 172.21.15.178:0/3389913434 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd43c09faa0 0x7fd43c09fe70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 1 --2- 172.21.15.178:0/3389913434 >> v2:172.21.15.178:3300/0 conn(0x7fd43c0a2930 0x7fd43c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 1 -- 172.21.15.178:0/3389913434 >> 172.21.15.178:0/3389913434 conn(0x7fd43c0a5ed0 msgr2=0x7fd43c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 1 -- 172.21.15.178:0/3389913434 shutdown_connections 2024-09-15T01:32:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 1 -- 172.21.15.178:0/3389913434 wait complete. 2024-09-15T01:32:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 10 monclient: build_initial_monmap 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.656+0000 7fd44379d640 10 monclient: monmap: 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:11.673 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 1 Processor -- start 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 1 -- start start 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 10 monclient: init 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 10 monclient: _add_conns ranks=[3,6,7,4,1,2,8,5,0] 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd43c09faa0 0x7fd43c09b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 10 monclient(hunting): picked mon.noname-g con 0x7fd43c09faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd43c0a08c0 0x7fd43c131990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 10 monclient(hunting): picked mon.noname-d con 0x7fd43c0a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:11.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd43c0a2930 0x7fd43c141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 10 monclient(hunting): picked mon.noname-i con 0x7fd43c0a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:11.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c11f070 con 0x7fd43c09faa0 2024-09-15T01:32:11.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c11ed90 con 0x7fd43c0a08c0 2024-09-15T01:32:11.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c11eab0 con 0x7fd43c0a2930 2024-09-15T01:32:11.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:11.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd44379d640 10 monclient(hunting): authenticate will time out at 736.407744s 2024-09-15T01:32:11.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd440d38640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd43c0a08c0 0x7fd43c131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd440d38640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd43c0a08c0 0x7fd43c131990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:40534/0 (socket says 172.21.15.178:40534) 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd440d38640 1 -- 172.21.15.178:0/952575728 learned_addr learned my addr 172.21.15.178:0/952575728 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd440d38640 10 monclient(hunting): get_auth_request con 0x7fd43c0a08c0 auth_method 0 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd440d38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd440d38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd440d38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd440d38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd440d38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd440d38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.657+0000 7fd441539640 1 --2- 172.21.15.178:0/952575728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd43c09faa0 0x7fd43c09b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd441d3a640 1 --2- 172.21.15.178:0/952575728 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd43c0a2930 0x7fd43c141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd441539640 10 monclient(hunting): get_auth_request con 0x7fd43c09faa0 auth_method 0 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd441539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd441539640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd441539640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd441d3a640 10 monclient(hunting): get_auth_request con 0x7fd43c0a2930 auth_method 0 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd441d3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd441d3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd441d3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd440d38640 10 monclient(hunting): handle_auth_done global_id 4443 payload 824 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd440d38640 1 -- 172.21.15.178:0/952575728 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd43c0a2930 msgr2=0x7fd43c141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd440d38640 1 --2- 172.21.15.178:0/952575728 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd43c0a2930 0x7fd43c141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd440d38640 1 -- 172.21.15.178:0/952575728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd43c09faa0 msgr2=0x7fd43c09b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd440d38640 1 --2- 172.21.15.178:0/952575728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd43c09faa0 0x7fd43c09b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd440d38640 10 monclient: _finish_hunting 0 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd440d38640 1 monclient: found mon.noname-d 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd440d38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd440d38640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:32:11.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd440d38640 1 -- 172.21.15.178:0/952575728 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd434009ae0 con 0x7fd43c0a08c0 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd440d38640 1 --2- 172.21.15.178:0/952575728 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd43c0a08c0 0x7fd43c131990 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009360 tx=0x7fd42c032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4317fa640 1 -- 172.21.15.178:0/952575728 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd42c03b870 con 0x7fd43c0a08c0 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4317fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4317fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4317fa640 10 monclient: dump: 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4317fa640 10 monclient: _finish_auth 0 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4317fa640 1 -- 172.21.15.178:0/952575728 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42c03b9f0 con 0x7fd43c0a08c0 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4317fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4317fa640 1 -- 172.21.15.178:0/952575728 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd42c0313a0 con 0x7fd43c0a08c0 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd44379d640 5 monclient: authenticate success, global_id 4443 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4317fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4317fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4337fe640 4 set_mon_vals no callback set 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd4317fa640 10 monclient: dump: 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.680 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd44379d640 10 monclient: _renew_subs 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd44379d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd44379d640 1 -- 172.21.15.178:0/952575728 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd438009390 con 0x7fd43c0a08c0 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd44379d640 10 monclient: _renew_subs 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd44379d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.658+0000 7fd44379d640 1 -- 172.21.15.178:0/952575728 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42c009390 con 0x7fd43c0a08c0 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.659+0000 7fd4317fa640 1 -- 172.21.15.178:0/952575728 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd42c039040 con 0x7fd43c0a08c0 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.659+0000 7fd44379d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:11.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.659+0000 7fd44379d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:11.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.659+0000 7fd44379d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:11.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.659+0000 7fd4317fa640 1 --2- 172.21.15.178:0/952575728 >> v2:172.21.15.204:6812/28210 conn(0x7fd4200397b0 0x7fd42003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.659+0000 7fd44379d640 1 -- 172.21.15.178:0/952575728 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43c04b720 con 0x7fd43c0a08c0 2024-09-15T01:32:11.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.659+0000 7fd4317fa640 1 -- 172.21.15.178:0/952575728 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7fd42c06a710 con 0x7fd43c0a08c0 2024-09-15T01:32:11.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.659+0000 7fd441539640 1 --2- 172.21.15.178:0/952575728 >> v2:172.21.15.204:6812/28210 conn(0x7fd4200397b0 0x7fd42003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.659+0000 7fd441539640 10 monclient: get_auth_request con 0x7fd4200397b0 auth_method 0 2024-09-15T01:32:11.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.660+0000 7fd441539640 1 --2- 172.21.15.178:0/952575728 >> v2:172.21.15.204:6812/28210 conn(0x7fd4200397b0 0x7fd42003bc60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd438009360 tx=0x7fd438008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:11.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.664+0000 7fd4317fa640 1 -- 172.21.15.178:0/952575728 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd42c042020 con 0x7fd43c0a08c0 2024-09-15T01:32:11.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.664+0000 7fd4317fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:11.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.664+0000 7fd4317fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:11.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.816+0000 7fd44379d640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-15T01:32:11.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.816+0000 7fd44379d640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-15T01:32:11.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.816+0000 7fd44379d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:32:11.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.816+0000 7fd44379d640 1 -- 172.21.15.178:0/952575728 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd43c11f1b0 con 0x7fd43c0a08c0 2024-09-15T01:32:11.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.816+0000 7fd4317fa640 1 -- 172.21.15.178:0/952575728 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd43c11f1b0 con 0x7fd43c0a08c0 2024-09-15T01:32:11.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.816+0000 7fd4317fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-15T01:32:11.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.816+0000 7fd4317fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:11.819 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2024-09-15T01:32:11.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.818+0000 7fd44379d640 1 -- 172.21.15.178:0/952575728 >> v2:172.21.15.204:6812/28210 conn(0x7fd4200397b0 msgr2=0x7fd42003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.818+0000 7fd44379d640 1 --2- 172.21.15.178:0/952575728 >> v2:172.21.15.204:6812/28210 conn(0x7fd4200397b0 0x7fd42003bc60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd438009360 tx=0x7fd438008040 comp rx=0 tx=0).stop 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.818+0000 7fd44379d640 10 monclient: shutdown 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.818+0000 7fd44379d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.818+0000 7fd44379d640 1 -- 172.21.15.178:0/952575728 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd43c0a08c0 msgr2=0x7fd43c131990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.818+0000 7fd44379d640 1 --2- 172.21.15.178:0/952575728 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd43c0a08c0 0x7fd43c131990 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009360 tx=0x7fd42c032000 comp rx=0 tx=0).stop 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.818+0000 7fd44379d640 1 -- 172.21.15.178:0/952575728 shutdown_connections 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.818+0000 7fd44379d640 1 --2- 172.21.15.178:0/952575728 >> v2:172.21.15.204:6812/28210 conn(0x7fd4200397b0 0x7fd42003bc60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.818+0000 7fd44379d640 1 --2- 172.21.15.178:0/952575728 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd43c0a2930 0x7fd43c141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.818+0000 7fd44379d640 1 --2- 172.21.15.178:0/952575728 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd43c0a08c0 0x7fd43c131990 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.818+0000 7fd44379d640 1 --2- 172.21.15.178:0/952575728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd43c09faa0 0x7fd43c09b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.818+0000 7fd44379d640 1 -- 172.21.15.178:0/952575728 >> 172.21.15.178:0/952575728 conn(0x7fd43c0a5ed0 msgr2=0x7fd43c09e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.819+0000 7fd44379d640 1 -- 172.21.15.178:0/952575728 shutdown_connections 2024-09-15T01:32:11.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.819+0000 7fd44379d640 1 -- 172.21.15.178:0/952575728 wait complete. 2024-09-15T01:32:11.831 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-15T01:32:11.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.923+0000 7fd397514640 10 monclient: get_monmap_and_config 2024-09-15T01:32:11.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.923+0000 7fd397514640 10 monclient: build_initial_monmap 2024-09-15T01:32:11.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.924+0000 7fd397514640 10 monclient: monmap: 2024-09-15T01:32:11.926 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:11.926 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.926 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:11.924889+0000 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:11.924889+0000 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:11.927 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:11.928 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:11.928 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.924+0000 7fd397514640 1 Processor -- start 2024-09-15T01:32:11.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.924+0000 7fd397514640 1 -- start start 2024-09-15T01:32:11.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.924+0000 7fd397514640 10 monclient: init 2024-09-15T01:32:11.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:11.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 10 monclient: _add_conns ranks=[4,10,2,8,3,0,9,5,1,7,6] 2024-09-15T01:32:11.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd3900a2930 0x7fd390132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 10 monclient(hunting): picked mon.noname-c con 0x7fd3900a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:11.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd39009faa0 0x7fd39009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 10 monclient(hunting): picked mon.noname-f con 0x7fd39009faa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:11.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3900a08c0 0x7fd3900a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 10 monclient(hunting): picked mon.noname-b con 0x7fd3900a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39009e0f0 con 0x7fd39009faa0 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39009dd80 con 0x7fd3900a08c0 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3900a2d00 con 0x7fd3900a2930 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd397514640 10 monclient(hunting): authenticate will time out at 736.675262s 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd3952b0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd3900a2930 0x7fd390132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3900a08c0 0x7fd3900a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd3952b0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd3900a2930 0x7fd390132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:36818/0 (socket says 172.21.15.178:36818) 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd3952b0640 1 -- 172.21.15.178:0/1561580896 learned_addr learned my addr 172.21.15.178:0/1561580896 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 10 monclient(hunting): get_auth_request con 0x7fd3900a08c0 auth_method 0 2024-09-15T01:32:11.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd3952b0640 10 monclient(hunting): get_auth_request con 0x7fd3900a2930 auth_method 0 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd3952b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd3952b0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd3952b0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd3952b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd3952b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd3952b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd394aaf640 1 --2- 172.21.15.178:0/1561580896 >> v2:172.21.15.204:3300/0 conn(0x7fd39009faa0 0x7fd39009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd394aaf640 10 monclient(hunting): get_auth_request con 0x7fd39009faa0 auth_method 0 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd394aaf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd394aaf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd394aaf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 10 monclient(hunting): handle_auth_done global_id 4394 payload 274 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 1 -- 172.21.15.178:0/1561580896 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd3900a2930 msgr2=0x7fd390132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 1 --2- 172.21.15.178:0/1561580896 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd3900a2930 0x7fd390132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 1 -- 172.21.15.178:0/1561580896 >> v2:172.21.15.204:3300/0 conn(0x7fd39009faa0 msgr2=0x7fd39009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 1 --2- 172.21.15.178:0/1561580896 >> v2:172.21.15.204:3300/0 conn(0x7fd39009faa0 0x7fd39009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 10 monclient: _finish_hunting 0 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 1 monclient: found mon.noname-b 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:11.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd395ab1640 1 -- 172.21.15.178:0/1561580896 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39004bf60 con 0x7fd3900a08c0 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.925+0000 7fd3952b0640 1 --2- 172.21.15.178:0/1561580896 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd3900a2930 0x7fd390132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd395ab1640 1 --2- 172.21.15.178:0/1561580896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3900a08c0 0x7fd3900a0d60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd380009e60 tx=0x7fd380009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87b8e9a95b075c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd38ffff640 1 -- 172.21.15.178:0/1561580896 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd3800079d0 con 0x7fd3900a08c0 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd38ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd38ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd38ffff640 10 monclient: dump: 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.941 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd38ffff640 10 monclient: _finish_auth 0 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd38ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd38ffff640 1 -- 172.21.15.178:0/1561580896 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38002d050 con 0x7fd3900a08c0 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd397514640 5 monclient: authenticate success, global_id 4394 2024-09-15T01:32:11.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd397514640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd38ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd38ffff640 1 -- 172.21.15.178:0/1561580896 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd380007bd0 con 0x7fd3900a08c0 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd397514640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd397514640 4 set_mon_vals no callback set 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd38ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd38ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.926+0000 7fd38ffff640 10 monclient: dump: 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.943 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 10 monclient: shutdown 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 1 -- 172.21.15.178:0/1561580896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3900a08c0 msgr2=0x7fd3900a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 1 --2- 172.21.15.178:0/1561580896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3900a08c0 0x7fd3900a0d60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd380009e60 tx=0x7fd380009780 comp rx=0 tx=0).stop 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 1 -- 172.21.15.178:0/1561580896 shutdown_connections 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 1 --2- 172.21.15.178:0/1561580896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3900a08c0 0x7fd3900a0d60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 1 --2- 172.21.15.178:0/1561580896 >> v2:172.21.15.204:3300/0 conn(0x7fd39009faa0 0x7fd39009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 1 --2- 172.21.15.178:0/1561580896 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd3900a2930 0x7fd390132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 1 -- 172.21.15.178:0/1561580896 >> 172.21.15.178:0/1561580896 conn(0x7fd3900a5ed0 msgr2=0x7fd3900a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 1 -- 172.21.15.178:0/1561580896 shutdown_connections 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 1 -- 172.21.15.178:0/1561580896 wait complete. 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 10 monclient: build_initial_monmap 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.927+0000 7fd397514640 10 monclient: monmap: 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 1 Processor -- start 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 1 -- start start 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 10 monclient: init 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 10 monclient: _add_conns ranks=[0,1,8,5,3,2,7,4,6] 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd39009faa0 0x7fd39009b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 10 monclient(hunting): picked mon.noname-a con 0x7fd39009faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3900a08c0 0x7fd390131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 10 monclient(hunting): picked mon.noname-f con 0x7fd3900a08c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd3900a2930 0x7fd39011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 10 monclient(hunting): picked mon.noname-e con 0x7fd3900a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39011ef20 con 0x7fd39009faa0 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39009c0b0 con 0x7fd3900a08c0 2024-09-15T01:32:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd390142120 con 0x7fd3900a2930 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd397514640 10 monclient(hunting): authenticate will time out at 736.678781s 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd3952b0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd39009faa0 0x7fd39009b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd3952b0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd39009faa0 0x7fd39009b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35056/0 (socket says 172.21.15.178:35056) 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd3952b0640 1 -- 172.21.15.178:0/2356005006 learned_addr learned my addr 172.21.15.178:0/2356005006 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd3952b0640 10 monclient(hunting): get_auth_request con 0x7fd39009faa0 auth_method 0 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd3952b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd3952b0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd3952b0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd395ab1640 1 --2- 172.21.15.178:0/2356005006 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd3900a2930 0x7fd39011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd394aaf640 1 --2- 172.21.15.178:0/2356005006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3900a08c0 0x7fd390131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd395ab1640 10 monclient(hunting): get_auth_request con 0x7fd3900a2930 auth_method 0 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd395ab1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd395ab1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd395ab1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd3952b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd3952b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd3952b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd394aaf640 10 monclient(hunting): get_auth_request con 0x7fd3900a08c0 auth_method 0 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd394aaf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd394aaf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.928+0000 7fd394aaf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd394aaf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd394aaf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd394aaf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd395ab1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:11.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd395ab1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd395ab1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd3952b0640 10 monclient(hunting): handle_auth_done global_id 4446 payload 824 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd3952b0640 1 -- 172.21.15.178:0/2356005006 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd3900a2930 msgr2=0x7fd39011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd3952b0640 1 --2- 172.21.15.178:0/2356005006 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd3900a2930 0x7fd39011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd3952b0640 1 -- 172.21.15.178:0/2356005006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3900a08c0 msgr2=0x7fd390131840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd3952b0640 1 --2- 172.21.15.178:0/2356005006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3900a08c0 0x7fd390131840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd3952b0640 10 monclient: _finish_hunting 0 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd3952b0640 1 monclient: found mon.noname-a 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd3952b0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd3952b0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd3952b0640 1 -- 172.21.15.178:0/2356005006 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd388009450 con 0x7fd39009faa0 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd3952b0640 1 --2- 172.21.15.178:0/2356005006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd39009faa0 0x7fd39009b0a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd38800ec80 tx=0x7fd3880091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38d7fa640 1 -- 172.21.15.178:0/2356005006 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd38800eec0 con 0x7fd39009faa0 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38d7fa640 10 monclient: dump: 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38d7fa640 10 monclient: _finish_auth 0 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38d7fa640 1 -- 172.21.15.178:0/2356005006 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38800cbc0 con 0x7fd39009faa0 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd397514640 5 monclient: authenticate success, global_id 4446 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd397514640 10 monclient: _renew_subs 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38d7fa640 1 -- 172.21.15.178:0/2356005006 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd388013d70 con 0x7fd39009faa0 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd397514640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd397514640 1 -- 172.21.15.178:0/2356005006 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd380009a20 con 0x7fd39009faa0 2024-09-15T01:32:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38d7fa640 10 monclient: dump: 2024-09-15T01:32:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:11.954 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:11.954 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:11.954 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:11.954 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:11.954 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:11.954 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:11.954 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:11.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd38f7fe640 4 set_mon_vals no callback set 2024-09-15T01:32:11.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd397514640 10 monclient: _renew_subs 2024-09-15T01:32:11.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd397514640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:11.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.929+0000 7fd397514640 1 -- 172.21.15.178:0/2356005006 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39011ec40 con 0x7fd39009faa0 2024-09-15T01:32:11.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.930+0000 7fd38d7fa640 1 -- 172.21.15.178:0/2356005006 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd38800cd30 con 0x7fd39009faa0 2024-09-15T01:32:11.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.930+0000 7fd397514640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:11.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.930+0000 7fd397514640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:11.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.930+0000 7fd397514640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:11.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.930+0000 7fd397514640 1 -- 172.21.15.178:0/2356005006 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39011f140 con 0x7fd39009faa0 2024-09-15T01:32:11.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.930+0000 7fd38d7fa640 1 --2- 172.21.15.178:0/2356005006 >> v2:172.21.15.204:6812/28210 conn(0x7fd37c0397b0 0x7fd37c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:11.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.930+0000 7fd38d7fa640 1 -- 172.21.15.178:0/2356005006 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7fd38804b6b0 con 0x7fd39009faa0 2024-09-15T01:32:11.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.930+0000 7fd394aaf640 1 --2- 172.21.15.178:0/2356005006 >> v2:172.21.15.204:6812/28210 conn(0x7fd37c0397b0 0x7fd37c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:11.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.930+0000 7fd394aaf640 10 monclient: get_auth_request con 0x7fd37c0397b0 auth_method 0 2024-09-15T01:32:11.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.931+0000 7fd394aaf640 1 --2- 172.21.15.178:0/2356005006 >> v2:172.21.15.204:6812/28210 conn(0x7fd37c0397b0 0x7fd37c03bc60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd384006fd0 tx=0x7fd384008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:11.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.935+0000 7fd38d7fa640 1 -- 172.21.15.178:0/2356005006 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd39011f140 con 0x7fd39009faa0 2024-09-15T01:32:11.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.935+0000 7fd38d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:11.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:11.935+0000 7fd38d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:12.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.088+0000 7fd397514640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-15T01:32:12.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.088+0000 7fd397514640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-15T01:32:12.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.088+0000 7fd397514640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:12.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.088+0000 7fd397514640 1 -- 172.21.15.178:0/2356005006 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd39011f140 con 0x7fd39009faa0 2024-09-15T01:32:12.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.089+0000 7fd38d7fa640 1 -- 172.21.15.178:0/2356005006 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd39011f140 con 0x7fd39009faa0 2024-09-15T01:32:12.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.089+0000 7fd38d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-15T01:32:12.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.089+0000 7fd38d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:12.091 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 8 2024-09-15T01:32:12.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 1 -- 172.21.15.178:0/2356005006 >> v2:172.21.15.204:6812/28210 conn(0x7fd37c0397b0 msgr2=0x7fd37c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 1 --2- 172.21.15.178:0/2356005006 >> v2:172.21.15.204:6812/28210 conn(0x7fd37c0397b0 0x7fd37c03bc60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd384006fd0 tx=0x7fd384008040 comp rx=0 tx=0).stop 2024-09-15T01:32:12.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 10 monclient: shutdown 2024-09-15T01:32:12.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:12.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 1 -- 172.21.15.178:0/2356005006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd39009faa0 msgr2=0x7fd39009b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 1 --2- 172.21.15.178:0/2356005006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd39009faa0 0x7fd39009b0a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd38800ec80 tx=0x7fd3880091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 1 -- 172.21.15.178:0/2356005006 shutdown_connections 2024-09-15T01:32:12.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 1 --2- 172.21.15.178:0/2356005006 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd3900a2930 0x7fd39011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 1 --2- 172.21.15.178:0/2356005006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3900a08c0 0x7fd390131840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 1 --2- 172.21.15.178:0/2356005006 >> v2:172.21.15.204:6812/28210 conn(0x7fd37c0397b0 0x7fd37c03bc60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 1 --2- 172.21.15.178:0/2356005006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd39009faa0 0x7fd39009b0a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 1 -- 172.21.15.178:0/2356005006 >> 172.21.15.178:0/2356005006 conn(0x7fd3900a5ed0 msgr2=0x7fd390135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:12.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 1 -- 172.21.15.178:0/2356005006 shutdown_connections 2024-09-15T01:32:12.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.091+0000 7fd397514640 1 -- 172.21.15.178:0/2356005006 wait complete. 2024-09-15T01:32:12.104 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-09-15T01:32:12.104 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:32:12.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.195+0000 7f461e79d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:12.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.195+0000 7f461e79d640 10 monclient: build_initial_monmap 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.196+0000 7f461e79d640 10 monclient: monmap: 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:12.196889+0000 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:12.196889+0000 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:12.198 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:12.199 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:12.199 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:12.199 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:12.199 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:12.199 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.196+0000 7f461e79d640 1 Processor -- start 2024-09-15T01:32:12.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.196+0000 7f461e79d640 1 -- start start 2024-09-15T01:32:12.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.196+0000 7f461e79d640 10 monclient: init 2024-09-15T01:32:12.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:12.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 10 monclient: _add_conns ranks=[5,2,8,7,1,9,4,10,6,0,3] 2024-09-15T01:32:12.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f46180a2930 0x7f4618132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 10 monclient(hunting): picked mon.noname-h con 0x7f46180a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f461809faa0 0x7f461809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 10 monclient(hunting): picked mon.noname-b con 0x7f461809faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46180a08c0 0x7f46180a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 10 monclient(hunting): picked mon.noname-e con 0x7f46180a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461809e0f0 con 0x7f461809faa0 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f461809dd80 con 0x7f46180a2930 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f46180a2d00 con 0x7f46180a08c0 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461e79d640 10 monclient(hunting): authenticate will time out at 736.947313s 2024-09-15T01:32:12.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461cd3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46180a08c0 0x7f46180a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f461809faa0 0x7f461809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461cd3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46180a08c0 0x7f46180a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:58030/0 (socket says 172.21.15.178:58030) 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 10 monclient(hunting): get_auth_request con 0x7f461809faa0 auth_method 0 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461cd3a640 1 -- 172.21.15.178:0/1831371378 learned_addr learned my addr 172.21.15.178:0/1831371378 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461cd3a640 10 monclient(hunting): get_auth_request con 0x7f46180a08c0 auth_method 0 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461cd3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461cd3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461cd3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:12.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461cd3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461cd3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461cd3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 10 monclient(hunting): handle_auth_done global_id 4403 payload 274 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 1 -- 172.21.15.178:0/1831371378 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46180a08c0 msgr2=0x7f46180a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 1 --2- 172.21.15.178:0/1831371378 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46180a08c0 0x7f46180a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 1 -- 172.21.15.178:0/1831371378 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f46180a2930 msgr2=0x7f4618132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f4617fff640 1 --2- 172.21.15.178:0/1831371378 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f46180a2930 0x7f4618132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 1 --2- 172.21.15.178:0/1831371378 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f46180a2930 0x7f4618132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 1 monclient: found mon.noname-b 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:12.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f46177fe640 1 -- 172.21.15.178:0/1831371378 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461804bf60 con 0x7f461809faa0 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.197+0000 7f461cd3a640 1 --2- 172.21.15.178:0/1831371378 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46180a08c0 0x7f46180a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f46177fe640 1 --2- 172.21.15.178:0/1831371378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f461809faa0 0x7f461809fe70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4600009880 tx=0x7f46000090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f8041149c62ac2a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f4616ffd640 1 -- 172.21.15.178:0/1831371378 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f460002abb0 con 0x7f461809faa0 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f4616ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f4616ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f4616ffd640 10 monclient: dump: 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:12.211 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f4616ffd640 10 monclient: _finish_auth 0 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f4616ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f461e79d640 5 monclient: authenticate success, global_id 4403 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f461e79d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:12.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f4616ffd640 1 -- 172.21.15.178:0/1831371378 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460002ad30 con 0x7f461809faa0 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f4616ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f461e79d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f461e79d640 4 set_mon_vals no callback set 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f4616ffd640 1 -- 172.21.15.178:0/1831371378 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4600031ca0 con 0x7f461809faa0 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f4616ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f4616ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.198+0000 7f4616ffd640 10 monclient: dump: 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.213 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 10 monclient: shutdown 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 1 -- 172.21.15.178:0/1831371378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f461809faa0 msgr2=0x7f461809fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 1 --2- 172.21.15.178:0/1831371378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f461809faa0 0x7f461809fe70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4600009880 tx=0x7f46000090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 1 -- 172.21.15.178:0/1831371378 shutdown_connections 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 1 --2- 172.21.15.178:0/1831371378 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46180a08c0 0x7f46180a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 1 --2- 172.21.15.178:0/1831371378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f461809faa0 0x7f461809fe70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 1 --2- 172.21.15.178:0/1831371378 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f46180a2930 0x7f4618132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 1 -- 172.21.15.178:0/1831371378 >> 172.21.15.178:0/1831371378 conn(0x7f46180a5ed0 msgr2=0x7f46180a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 1 -- 172.21.15.178:0/1831371378 shutdown_connections 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 1 -- 172.21.15.178:0/1831371378 wait complete. 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 10 monclient: build_initial_monmap 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.199+0000 7f461e79d640 10 monclient: monmap: 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:12.215 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 1 Processor -- start 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 1 -- start start 2024-09-15T01:32:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 10 monclient: init 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 10 monclient: _add_conns ranks=[7,0,1,5,6,2,3,4,8] 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f461809faa0 0x7f461809b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 10 monclient(hunting): picked mon.noname-i con 0x7f461809faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46180a08c0 0x7f46181316c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 10 monclient(hunting): picked mon.noname-a con 0x7f46180a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46180a2930 0x7f461811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 10 monclient(hunting): picked mon.noname-f con 0x7f46180a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461809c2c0 con 0x7f46180a08c0 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180a2390 con 0x7f46180a2930 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f461809d630 con 0x7f461809faa0 2024-09-15T01:32:12.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461e79d640 10 monclient(hunting): authenticate will time out at 736.950653s 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f46177fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46180a08c0 0x7f46181316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f46177fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46180a08c0 0x7f46181316c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35058/0 (socket says 172.21.15.178:35058) 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f46177fe640 1 -- 172.21.15.178:0/1810789847 learned_addr learned my addr 172.21.15.178:0/1810789847 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f46177fe640 10 monclient(hunting): get_auth_request con 0x7f46180a08c0 auth_method 0 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f46177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f46177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f46177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f46177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f46177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f46177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f4617fff640 1 --2- 172.21.15.178:0/1810789847 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f461809faa0 0x7f461809b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f4617fff640 10 monclient(hunting): get_auth_request con 0x7f461809faa0 auth_method 0 2024-09-15T01:32:12.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f4617fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f4617fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f4617fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461cd3a640 1 --2- 172.21.15.178:0/1810789847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46180a2930 0x7f461811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461cd3a640 10 monclient(hunting): get_auth_request con 0x7f46180a2930 auth_method 0 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461cd3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461cd3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.200+0000 7f461cd3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f46177fe640 10 monclient(hunting): handle_auth_done global_id 4455 payload 824 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f46177fe640 1 -- 172.21.15.178:0/1810789847 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f461809faa0 msgr2=0x7f461809b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f46177fe640 1 --2- 172.21.15.178:0/1810789847 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f461809faa0 0x7f461809b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f46177fe640 1 -- 172.21.15.178:0/1810789847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46180a2930 msgr2=0x7f461811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f46177fe640 1 --2- 172.21.15.178:0/1810789847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46180a2930 0x7f461811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f46177fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:12.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f46177fe640 1 monclient: found mon.noname-a 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f46177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f46177fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f46177fe640 1 -- 172.21.15.178:0/1810789847 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610009ae0 con 0x7f46180a08c0 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f4617fff640 1 --2- 172.21.15.178:0/1810789847 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f461809faa0 0x7f461809b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f46177fe640 1 --2- 172.21.15.178:0/1810789847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46180a08c0 0x7f46181316c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4600035040 tx=0x7f4600032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f460ffff640 1 -- 172.21.15.178:0/1810789847 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f460002aea0 con 0x7f46180a08c0 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f460ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f460ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f460ffff640 10 monclient: dump: 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.220 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f460ffff640 10 monclient: _finish_auth 0 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f460ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f460ffff640 1 -- 172.21.15.178:0/1810789847 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460003b870 con 0x7f46180a08c0 2024-09-15T01:32:12.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f461e79d640 5 monclient: authenticate success, global_id 4455 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f461e79d640 10 monclient: _renew_subs 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f461e79d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f461e79d640 1 -- 172.21.15.178:0/1810789847 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4600009390 con 0x7f46180a08c0 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f460ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f460ffff640 1 -- 172.21.15.178:0/1810789847 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f460003be60 con 0x7f46180a08c0 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f460ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f460ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f460ffff640 10 monclient: dump: 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.222 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f46167fc640 4 set_mon_vals no callback set 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f461e79d640 10 monclient: _renew_subs 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f461e79d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.201+0000 7f461e79d640 1 -- 172.21.15.178:0/1810789847 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461811e7e0 con 0x7f46180a08c0 2024-09-15T01:32:12.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.202+0000 7f460ffff640 1 -- 172.21.15.178:0/1810789847 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4600039040 con 0x7f46180a08c0 2024-09-15T01:32:12.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.202+0000 7f461e79d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:12.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.202+0000 7f461e79d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:12.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.202+0000 7f461e79d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:12.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.202+0000 7f460ffff640 1 --2- 172.21.15.178:0/1810789847 >> v2:172.21.15.204:6812/28210 conn(0x7f45f80396d0 0x7f45f803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.202+0000 7f461e79d640 1 -- 172.21.15.178:0/1810789847 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461804b720 con 0x7f46180a08c0 2024-09-15T01:32:12.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.202+0000 7f460ffff640 1 -- 172.21.15.178:0/1810789847 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f460006a7e0 con 0x7f46180a08c0 2024-09-15T01:32:12.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.202+0000 7f4617fff640 1 --2- 172.21.15.178:0/1810789847 >> v2:172.21.15.204:6812/28210 conn(0x7f45f80396d0 0x7f45f803bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.202+0000 7f4617fff640 10 monclient: get_auth_request con 0x7f45f80396d0 auth_method 0 2024-09-15T01:32:12.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.203+0000 7f4617fff640 1 --2- 172.21.15.178:0/1810789847 >> v2:172.21.15.204:6812/28210 conn(0x7f45f80396d0 0x7f45f803bb80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4608006fd0 tx=0x7f4608008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:12.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.206+0000 7f460ffff640 1 -- 172.21.15.178:0/1810789847 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4600030940 con 0x7f46180a08c0 2024-09-15T01:32:12.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.206+0000 7f460ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:12.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.206+0000 7f460ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:12.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.355+0000 7f461e79d640 1 -- 172.21.15.178:0/1810789847 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f461805ed70 con 0x7f45f80396d0 2024-09-15T01:32:12.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.357+0000 7f460ffff640 1 -- 172.21.15.178:0/1810789847 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+39208 (secure 0 0 0) 0x7f461805ed70 con 0x7f45f80396d0 2024-09-15T01:32:12.360 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:32:12.360 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-09-15T01:32:12.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.359+0000 7f461e79d640 1 -- 172.21.15.178:0/1810789847 >> v2:172.21.15.204:6812/28210 conn(0x7f45f80396d0 msgr2=0x7f45f803bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.359+0000 7f461e79d640 1 --2- 172.21.15.178:0/1810789847 >> v2:172.21.15.204:6812/28210 conn(0x7f45f80396d0 0x7f45f803bb80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4608006fd0 tx=0x7f4608008040 comp rx=0 tx=0).stop 2024-09-15T01:32:12.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.359+0000 7f461e79d640 10 monclient: shutdown 2024-09-15T01:32:12.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.359+0000 7f461e79d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:12.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.359+0000 7f461e79d640 1 -- 172.21.15.178:0/1810789847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46180a08c0 msgr2=0x7f46181316c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.359+0000 7f461e79d640 1 --2- 172.21.15.178:0/1810789847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46180a08c0 0x7f46181316c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4600035040 tx=0x7f4600032000 comp rx=0 tx=0).stop 2024-09-15T01:32:12.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.360+0000 7f461e79d640 1 -- 172.21.15.178:0/1810789847 shutdown_connections 2024-09-15T01:32:12.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.360+0000 7f461e79d640 1 --2- 172.21.15.178:0/1810789847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46180a2930 0x7f461811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.360+0000 7f461e79d640 1 --2- 172.21.15.178:0/1810789847 >> v2:172.21.15.204:6812/28210 conn(0x7f45f80396d0 0x7f45f803bb80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.360+0000 7f461e79d640 1 --2- 172.21.15.178:0/1810789847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46180a08c0 0x7f46181316c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.360+0000 7f461e79d640 1 --2- 172.21.15.178:0/1810789847 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f461809faa0 0x7f461809b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.360+0000 7f461e79d640 1 -- 172.21.15.178:0/1810789847 >> 172.21.15.178:0/1810789847 conn(0x7f46180a5ed0 msgr2=0x7f461809e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:12.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.360+0000 7f461e79d640 1 -- 172.21.15.178:0/1810789847 shutdown_connections 2024-09-15T01:32:12.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.360+0000 7f461e79d640 1 -- 172.21.15.178:0/1810789847 wait complete. 2024-09-15T01:32:12.372 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-15T01:32:12.158120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562077696,"kb_used":4777156,"kb_used_data":557300696,"kb_used_omap":438,"kb_used_meta":0,"kb_avail":557300540,"statfs":{"total":575567560704,"available":570675752960,"internally_reserved":159744,"allocated":570675912704,"data_stored":570675912704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":448785,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.087382"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071141+0000","last_change":"2024-09-15T01:32:06.071392+0000","last_active":"2024-09-15T01:32:06.071141+0000","last_peered":"2024-09-15T01:32:06.071141+0000","last_clean":"2024-09-15T01:32:06.071141+0000","last_became_active":"2024-09-15T01:32:04.055827+0000","last_became_peered":"2024-09-15T01:32:04.055827+0000","last_unstale":"2024-09-15T01:32:06.071141+0000","last_undegraded":"2024-09-15T01:32:06.071141+0000","last_fullsized":"2024-09-15T01:32:06.071141+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:37:11.817932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065783200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071224+0000","last_change":"2024-09-15T01:32:06.071478+0000","last_active":"2024-09-15T01:32:06.071224+0000","last_peered":"2024-09-15T01:32:06.071224+0000","last_clean":"2024-09-15T01:32:06.071224+0000","last_became_active":"2024-09-15T01:32:04.057737+0000","last_became_peered":"2024-09-15T01:32:04.057737+0000","last_unstale":"2024-09-15T01:32:06.071224+0000","last_undegraded":"2024-09-15T01:32:06.071224+0000","last_fullsized":"2024-09-15T01:32:06.071224+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:56:30.331754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041942500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071380+0000","last_change":"2024-09-15T01:32:06.071587+0000","last_active":"2024-09-15T01:32:06.071380+0000","last_peered":"2024-09-15T01:32:06.071380+0000","last_clean":"2024-09-15T01:32:06.071380+0000","last_became_active":"2024-09-15T01:32:04.056136+0000","last_became_peered":"2024-09-15T01:32:04.056136+0000","last_unstale":"2024-09-15T01:32:06.071380+0000","last_undegraded":"2024-09-15T01:32:06.071380+0000","last_fullsized":"2024-09-15T01:32:06.071380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:54.622691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077134000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.156568+0000","last_change":"2024-09-15T01:32:06.156744+0000","last_active":"2024-09-15T01:32:06.156568+0000","last_peered":"2024-09-15T01:32:06.156568+0000","last_clean":"2024-09-15T01:32:06.156568+0000","last_became_active":"2024-09-15T01:32:04.057683+0000","last_became_peered":"2024-09-15T01:32:04.057683+0000","last_unstale":"2024-09-15T01:32:06.156568+0000","last_undegraded":"2024-09-15T01:32:06.156568+0000","last_fullsized":"2024-09-15T01:32:06.156568+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.248853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031034999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.075171+0000","last_change":"2024-09-15T01:32:06.075171+0000","last_active":"2024-09-15T01:32:06.075171+0000","last_peered":"2024-09-15T01:32:06.075171+0000","last_clean":"2024-09-15T01:32:06.075171+0000","last_became_active":"2024-09-15T01:32:04.057076+0000","last_became_peered":"2024-09-15T01:32:04.057076+0000","last_unstale":"2024-09-15T01:32:06.075171+0000","last_undegraded":"2024-09-15T01:32:06.075171+0000","last_fullsized":"2024-09-15T01:32:06.075171+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:06:21.344504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093022000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.072519+0000","last_change":"2024-09-15T01:32:06.072807+0000","last_active":"2024-09-15T01:32:06.072519+0000","last_peered":"2024-09-15T01:32:06.072519+0000","last_clean":"2024-09-15T01:32:06.072519+0000","last_became_active":"2024-09-15T01:32:04.058720+0000","last_became_peered":"2024-09-15T01:32:04.058720+0000","last_unstale":"2024-09-15T01:32:06.072519+0000","last_undegraded":"2024-09-15T01:32:06.072519+0000","last_fullsized":"2024-09-15T01:32:06.072519+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:48:11.629787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046258900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.072351+0000","last_change":"2024-09-15T01:32:06.072595+0000","last_active":"2024-09-15T01:32:06.072351+0000","last_peered":"2024-09-15T01:32:06.072351+0000","last_clean":"2024-09-15T01:32:06.072351+0000","last_became_active":"2024-09-15T01:32:04.058893+0000","last_became_peered":"2024-09-15T01:32:04.058893+0000","last_unstale":"2024-09-15T01:32:06.072351+0000","last_undegraded":"2024-09-15T01:32:06.072351+0000","last_fullsized":"2024-09-15T01:32:06.072351+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:38:08.540821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037511000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071100+0000","last_change":"2024-09-15T01:32:06.071341+0000","last_active":"2024-09-15T01:32:06.071100+0000","last_peered":"2024-09-15T01:32:06.071100+0000","last_clean":"2024-09-15T01:32:06.071100+0000","last_became_active":"2024-09-15T01:32:04.057931+0000","last_became_peered":"2024-09-15T01:32:04.057931+0000","last_unstale":"2024-09-15T01:32:06.071100+0000","last_undegraded":"2024-09-15T01:32:06.071100+0000","last_fullsized":"2024-09-15T01:32:06.071100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:10:36.309941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076976900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'72","reported_seq":103,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T01:32:05.068805+0000","last_change":"2024-09-15T01:32:00.032305+0000","last_active":"2024-09-15T01:32:05.068805+0000","last_peered":"2024-09-15T01:32:05.068805+0000","last_clean":"2024-09-15T01:32:05.068805+0000","last_became_active":"2024-09-15T01:32:00.032048+0000","last_became_peered":"2024-09-15T01:32:00.032048+0000","last_unstale":"2024-09-15T01:32:05.068805+0000","last_undegraded":"2024-09-15T01:32:05.068805+0000","last_fullsized":"2024-09-15T01:32:05.068805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_clean_scrub_stamp":"2024-09-15T01:31:59.015115+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:23:36.195853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796940,"kb_used_data":92882684,"kb_used_omap":100,"kb_used_meta":0,"kb_avail":92882676,"statfs":{"total":95927926784,"available":95111860224,"internally_reserved":8192,"allocated":95111868416,"data_stored":95111868416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":102448,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795820,"kb_used_data":92883832,"kb_used_omap":62,"kb_used_meta":0,"kb_avail":92883796,"statfs":{"total":95927926784,"available":95113007104,"internally_reserved":36864,"allocated":95113043968,"data_stored":95113043968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":64366,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795784,"kb_used_data":92883856,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92883832,"statfs":{"total":95927926784,"available":95113043968,"internally_reserved":24576,"allocated":95113068544,"data_stored":95113068544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":55119,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795784,"kb_used_data":92883856,"kb_used_omap":55,"kb_used_meta":0,"kb_avail":92883832,"statfs":{"total":95927926784,"available":95113043968,"internally_reserved":24576,"allocated":95113068544,"data_stored":95113068544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56457,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]}]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":797008,"kb_used_data":92882636,"kb_used_omap":104,"kb_used_meta":0,"kb_avail":92882608,"statfs":{"total":95927926784,"available":95111790592,"internally_reserved":28672,"allocated":95111819264,"data_stored":95111819264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":107425,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795820,"kb_used_data":92883832,"kb_used_omap":61,"kb_used_meta":0,"kb_avail":92883796,"statfs":{"total":95927926784,"available":95113007104,"internally_reserved":36864,"allocated":95113043968,"data_stored":95113043968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":62970,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.153}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.222}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[]}} 2024-09-15T01:32:12.373 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:32:12.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.463+0000 7f967f39d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:12.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.463+0000 7f967f39d640 10 monclient: build_initial_monmap 2024-09-15T01:32:12.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.463+0000 7f967f39d640 10 monclient: monmap: 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:12.464690+0000 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:12.464690+0000 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:12.467 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:12.467 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:12.467 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:12.467 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:12.467 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.464+0000 7f967f39d640 1 Processor -- start 2024-09-15T01:32:12.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.464+0000 7f967f39d640 1 -- start start 2024-09-15T01:32:12.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.464+0000 7f967f39d640 10 monclient: init 2024-09-15T01:32:12.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.464+0000 7f967f39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:12.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.464+0000 7f967f39d640 10 monclient: _add_conns ranks=[5,0,10,6,9,3,7,8,4,2,1] 2024-09-15T01:32:12.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967f39d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96780a2930 0x7f9678132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967f39d640 10 monclient(hunting): picked mon.noname-h con 0x7f96780a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:12.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967f39d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f967809faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:12.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967f39d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f967809fe70 0x7f96780a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967f39d640 10 monclient(hunting): picked mon.noname-f con 0x7f967809fe70 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:12.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967f39d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967804bf60 con 0x7f967809faa0 2024-09-15T01:32:12.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967f39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967f39d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967809dd80 con 0x7f967809fe70 2024-09-15T01:32:12.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967f39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967f39d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96780a2d00 con 0x7f96780a2930 2024-09-15T01:32:12.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967f39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:12.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967f39d640 10 monclient(hunting): authenticate will time out at 737.215096s 2024-09-15T01:32:12.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967c938640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f967809faa0 0x7f96780a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:49284/0 (socket says 172.21.15.178:49284) 2024-09-15T01:32:12.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967c938640 1 -- 172.21.15.178:0/3055612438 learned_addr learned my addr 172.21.15.178:0/3055612438 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:12.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 1 --2- 172.21.15.178:0/3055612438 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96780a2930 0x7f9678132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d93a640 1 --2- 172.21.15.178:0/3055612438 >> v2:172.21.15.204:3300/0 conn(0x7f967809fe70 0x7f96780a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 10 monclient(hunting): get_auth_request con 0x7f96780a2930 auth_method 0 2024-09-15T01:32:12.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d93a640 10 monclient(hunting): get_auth_request con 0x7f967809fe70 auth_method 0 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d93a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d93a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d93a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f966ffff640 1 -- 172.21.15.178:0/3055612438 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191981922 0 0) 0x7f967804bf60 con 0x7f967809faa0 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d93a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d93a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:12.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d93a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f966ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f966ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f966ffff640 10 monclient(hunting): my global_id is 4464 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f966ffff640 1 -- 172.21.15.178:0/3055612438 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96640034e0 con 0x7f967809faa0 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 10 monclient(hunting): handle_auth_done global_id 4415 payload 274 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 1 -- 172.21.15.178:0/3055612438 >> v2:172.21.15.204:3300/0 conn(0x7f967809fe70 msgr2=0x7f96780a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 1 --2- 172.21.15.178:0/3055612438 >> v2:172.21.15.204:3300/0 conn(0x7f967809fe70 0x7f96780a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 1 -- 172.21.15.178:0/3055612438 >> v1:172.21.15.178:6789/0 conn(0x7f967809faa0 legacy=0x7f96780a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f966ffff640 1 -- 172.21.15.178:0/3055612438 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4236499582 0 0) 0x7f96640034e0 con 0x7f967809faa0 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 10 monclient: _finish_hunting 0 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 1 monclient: found mon.noname-h 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:32:12.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d139640 1 -- 172.21.15.178:0/3055612438 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96780a0310 con 0x7f96780a2930 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f966ffff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.465+0000 7f967d93a640 1 --2- 172.21.15.178:0/3055612438 >> v2:172.21.15.204:3300/0 conn(0x7f967809fe70 0x7f96780a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f967d139640 1 --2- 172.21.15.178:0/3055612438 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96780a2930 0x7f9678132de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9674009850 tx=0x7f96740090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d3aff810bcd1d18b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f966ffff640 1 -- 172.21.15.178:0/3055612438 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96680018f0 con 0x7f96780a2930 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f966ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f966ffff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f966ffff640 10 monclient: dump: 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.480 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f966ffff640 10 monclient: _finish_auth 0 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f966ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f966ffff640 1 -- 172.21.15.178:0/3055612438 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96740313a0 con 0x7f96780a2930 2024-09-15T01:32:12.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f967f39d640 5 monclient: authenticate success, global_id 4415 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f966ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f967f39d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f967f39d640 4 set_mon_vals no callback set 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f966ffff640 1 -- 172.21.15.178:0/3055612438 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9674031510 con 0x7f96780a2930 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f966ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f966ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.466+0000 7f966ffff640 10 monclient: dump: 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.482 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.467+0000 7f967f39d640 10 monclient: shutdown 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.467+0000 7f967f39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.467+0000 7f967f39d640 1 -- 172.21.15.178:0/3055612438 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96780a2930 msgr2=0x7f9678132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.467+0000 7f967f39d640 1 --2- 172.21.15.178:0/3055612438 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96780a2930 0x7f9678132de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9674009850 tx=0x7f96740090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.467+0000 7f967f39d640 1 -- 172.21.15.178:0/3055612438 shutdown_connections 2024-09-15T01:32:12.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.467+0000 7f967f39d640 1 --2- 172.21.15.178:0/3055612438 >> v2:172.21.15.204:3300/0 conn(0x7f967809fe70 0x7f96780a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.467+0000 7f967f39d640 1 --2- 172.21.15.178:0/3055612438 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96780a2930 0x7f9678132de0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.467+0000 7f967f39d640 1 -- 172.21.15.178:0/3055612438 >> 172.21.15.178:0/3055612438 conn(0x7f96780a5ed0 msgr2=0x7f96780a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.467+0000 7f967f39d640 1 -- 172.21.15.178:0/3055612438 shutdown_connections 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.467+0000 7f967f39d640 1 -- 172.21.15.178:0/3055612438 wait complete. 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.467+0000 7f967f39d640 10 monclient: build_initial_monmap 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.467+0000 7f967f39d640 10 monclient: monmap: 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:12.484 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 1 Processor -- start 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 1 -- start start 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 10 monclient: init 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:12.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 10 monclient: _add_conns ranks=[7,4,0,2,5,3,1,6,8] 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f967809faa0 0x7f967809b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 10 monclient(hunting): picked mon.noname-i con 0x7f967809faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f967809fe70 0x7f96781318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 10 monclient(hunting): picked mon.noname-c con 0x7f967809fe70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96780a2930 0x7f9678141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 10 monclient(hunting): picked mon.noname-a con 0x7f96780a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967811f370 con 0x7f96780a2930 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f967811d700 con 0x7f967809fe70 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f967811e9d0 con 0x7f967809faa0 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967f39d640 10 monclient(hunting): authenticate will time out at 737.218646s 2024-09-15T01:32:12.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d93a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96780a2930 0x7f9678141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967c938640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f967809fe70 0x7f96781318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d93a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96780a2930 0x7f9678141cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35064/0 (socket says 172.21.15.178:35064) 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d93a640 1 -- 172.21.15.178:0/1582227040 learned_addr learned my addr 172.21.15.178:0/1582227040 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967c938640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f967809fe70 0x7f96781318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:36828/0 (socket says 172.21.15.178:36828) 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d93a640 10 monclient(hunting): get_auth_request con 0x7f96780a2930 auth_method 0 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d139640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f967809faa0 0x7f967809b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d93a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d93a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d93a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967c938640 10 monclient(hunting): get_auth_request con 0x7f967809fe70 auth_method 0 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967c938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967c938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967c938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d139640 10 monclient(hunting): get_auth_request con 0x7f967809faa0 auth_method 0 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d93a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d93a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d93a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967c938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967c938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967c938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.468+0000 7f967d139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967c938640 10 monclient(hunting): handle_auth_done global_id 4387 payload 824 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967c938640 1 -- 172.21.15.178:0/1582227040 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f967809faa0 msgr2=0x7f967809b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967c938640 1 --2- 172.21.15.178:0/1582227040 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f967809faa0 0x7f967809b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967c938640 1 -- 172.21.15.178:0/1582227040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96780a2930 msgr2=0x7f9678141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967c938640 1 --2- 172.21.15.178:0/1582227040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96780a2930 0x7f9678141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967c938640 10 monclient: _finish_hunting 0 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967c938640 1 monclient: found mon.noname-c 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967c938640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967c938640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967c938640 1 -- 172.21.15.178:0/1582227040 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9670009a20 con 0x7f967809fe70 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967d93a640 1 --2- 172.21.15.178:0/1582227040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96780a2930 0x7f9678141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967c938640 1 --2- 172.21.15.178:0/1582227040 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f967809fe70 0x7f96781318b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f966800c5b0 tx=0x7f9668007940 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966d7fa640 1 -- 172.21.15.178:0/1582227040 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96680070f0 con 0x7f967809fe70 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966d7fa640 10 monclient: dump: 2024-09-15T01:32:12.489 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:12.490 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966d7fa640 10 monclient: _finish_auth 0 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966d7fa640 1 -- 172.21.15.178:0/1582227040 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9668007290 con 0x7f967809fe70 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967f39d640 5 monclient: authenticate success, global_id 4387 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967f39d640 10 monclient: _renew_subs 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966d7fa640 1 -- 172.21.15.178:0/1582227040 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9668012400 con 0x7f967809fe70 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967f39d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967f39d640 1 -- 172.21.15.178:0/1582227040 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9674009390 con 0x7f967809fe70 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966d7fa640 10 monclient: dump: 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.491 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f966f7fe640 4 set_mon_vals no callback set 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967f39d640 10 monclient: _renew_subs 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967f39d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.469+0000 7f967f39d640 1 -- 172.21.15.178:0/1582227040 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967811ef90 con 0x7f967809fe70 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.470+0000 7f966d7fa640 1 -- 172.21.15.178:0/1582227040 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9668007400 con 0x7f967809fe70 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.470+0000 7f967f39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.470+0000 7f967f39d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.470+0000 7f967f39d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.470+0000 7f966d7fa640 1 --2- 172.21.15.178:0/1582227040 >> v2:172.21.15.204:6812/28210 conn(0x7f965c0397e0 0x7f965c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.470+0000 7f967f39d640 1 -- 172.21.15.178:0/1582227040 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967804b720 con 0x7f967809fe70 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.470+0000 7f966d7fa640 1 -- 172.21.15.178:0/1582227040 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f966804a890 con 0x7f967809fe70 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.470+0000 7f967d139640 1 --2- 172.21.15.178:0/1582227040 >> v2:172.21.15.204:6812/28210 conn(0x7f965c0397e0 0x7f965c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.470+0000 7f967d139640 10 monclient: get_auth_request con 0x7f965c0397e0 auth_method 0 2024-09-15T01:32:12.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.471+0000 7f967d139640 1 --2- 172.21.15.178:0/1582227040 >> v2:172.21.15.204:6812/28210 conn(0x7f965c0397e0 0x7f965c03bc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9674034000 tx=0x7f9674032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:12.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.474+0000 7f966d7fa640 1 -- 172.21.15.178:0/1582227040 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f966801a590 con 0x7f967809fe70 2024-09-15T01:32:12.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.475+0000 7f966d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:12.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.475+0000 7f966d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:12.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.624+0000 7f967f39d640 1 -- 172.21.15.178:0/1582227040 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f967805ed70 con 0x7f965c0397e0 2024-09-15T01:32:12.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.626+0000 7f966d7fa640 1 -- 172.21.15.178:0/1582227040 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+39208 (secure 0 0 0) 0x7f967805ed70 con 0x7f965c0397e0 2024-09-15T01:32:12.629 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:32:12.629 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-09-15T01:32:12.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 1 -- 172.21.15.178:0/1582227040 >> v2:172.21.15.204:6812/28210 conn(0x7f965c0397e0 msgr2=0x7f965c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 1 --2- 172.21.15.178:0/1582227040 >> v2:172.21.15.204:6812/28210 conn(0x7f965c0397e0 0x7f965c03bc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9674034000 tx=0x7f9674032000 comp rx=0 tx=0).stop 2024-09-15T01:32:12.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 10 monclient: shutdown 2024-09-15T01:32:12.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:12.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 1 -- 172.21.15.178:0/1582227040 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f967809fe70 msgr2=0x7f96781318b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 1 --2- 172.21.15.178:0/1582227040 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f967809fe70 0x7f96781318b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f966800c5b0 tx=0x7f9668007940 comp rx=0 tx=0).stop 2024-09-15T01:32:12.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 1 -- 172.21.15.178:0/1582227040 shutdown_connections 2024-09-15T01:32:12.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 1 --2- 172.21.15.178:0/1582227040 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f967809fe70 0x7f96781318b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 1 --2- 172.21.15.178:0/1582227040 >> v2:172.21.15.204:6812/28210 conn(0x7f965c0397e0 0x7f965c03bc90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 1 --2- 172.21.15.178:0/1582227040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96780a2930 0x7f9678141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 1 --2- 172.21.15.178:0/1582227040 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f967809faa0 0x7f967809b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 1 -- 172.21.15.178:0/1582227040 >> 172.21.15.178:0/1582227040 conn(0x7f96780a5ed0 msgr2=0x7f96780a52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:12.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 1 -- 172.21.15.178:0/1582227040 shutdown_connections 2024-09-15T01:32:12.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.628+0000 7f967f39d640 1 -- 172.21.15.178:0/1582227040 wait complete. 2024-09-15T01:32:12.640 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-15T01:32:12.158120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562077696,"kb_used":4777156,"kb_used_data":557300696,"kb_used_omap":438,"kb_used_meta":0,"kb_avail":557300540,"statfs":{"total":575567560704,"available":570675752960,"internally_reserved":159744,"allocated":570675912704,"data_stored":570675912704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":448785,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.087382"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071141+0000","last_change":"2024-09-15T01:32:06.071392+0000","last_active":"2024-09-15T01:32:06.071141+0000","last_peered":"2024-09-15T01:32:06.071141+0000","last_clean":"2024-09-15T01:32:06.071141+0000","last_became_active":"2024-09-15T01:32:04.055827+0000","last_became_peered":"2024-09-15T01:32:04.055827+0000","last_unstale":"2024-09-15T01:32:06.071141+0000","last_undegraded":"2024-09-15T01:32:06.071141+0000","last_fullsized":"2024-09-15T01:32:06.071141+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:37:11.817932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065783200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071224+0000","last_change":"2024-09-15T01:32:06.071478+0000","last_active":"2024-09-15T01:32:06.071224+0000","last_peered":"2024-09-15T01:32:06.071224+0000","last_clean":"2024-09-15T01:32:06.071224+0000","last_became_active":"2024-09-15T01:32:04.057737+0000","last_became_peered":"2024-09-15T01:32:04.057737+0000","last_unstale":"2024-09-15T01:32:06.071224+0000","last_undegraded":"2024-09-15T01:32:06.071224+0000","last_fullsized":"2024-09-15T01:32:06.071224+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:56:30.331754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041942500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071380+0000","last_change":"2024-09-15T01:32:06.071587+0000","last_active":"2024-09-15T01:32:06.071380+0000","last_peered":"2024-09-15T01:32:06.071380+0000","last_clean":"2024-09-15T01:32:06.071380+0000","last_became_active":"2024-09-15T01:32:04.056136+0000","last_became_peered":"2024-09-15T01:32:04.056136+0000","last_unstale":"2024-09-15T01:32:06.071380+0000","last_undegraded":"2024-09-15T01:32:06.071380+0000","last_fullsized":"2024-09-15T01:32:06.071380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:54.622691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077134000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.156568+0000","last_change":"2024-09-15T01:32:06.156744+0000","last_active":"2024-09-15T01:32:06.156568+0000","last_peered":"2024-09-15T01:32:06.156568+0000","last_clean":"2024-09-15T01:32:06.156568+0000","last_became_active":"2024-09-15T01:32:04.057683+0000","last_became_peered":"2024-09-15T01:32:04.057683+0000","last_unstale":"2024-09-15T01:32:06.156568+0000","last_undegraded":"2024-09-15T01:32:06.156568+0000","last_fullsized":"2024-09-15T01:32:06.156568+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.248853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031034999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.075171+0000","last_change":"2024-09-15T01:32:06.075171+0000","last_active":"2024-09-15T01:32:06.075171+0000","last_peered":"2024-09-15T01:32:06.075171+0000","last_clean":"2024-09-15T01:32:06.075171+0000","last_became_active":"2024-09-15T01:32:04.057076+0000","last_became_peered":"2024-09-15T01:32:04.057076+0000","last_unstale":"2024-09-15T01:32:06.075171+0000","last_undegraded":"2024-09-15T01:32:06.075171+0000","last_fullsized":"2024-09-15T01:32:06.075171+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:06:21.344504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093022000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.072519+0000","last_change":"2024-09-15T01:32:06.072807+0000","last_active":"2024-09-15T01:32:06.072519+0000","last_peered":"2024-09-15T01:32:06.072519+0000","last_clean":"2024-09-15T01:32:06.072519+0000","last_became_active":"2024-09-15T01:32:04.058720+0000","last_became_peered":"2024-09-15T01:32:04.058720+0000","last_unstale":"2024-09-15T01:32:06.072519+0000","last_undegraded":"2024-09-15T01:32:06.072519+0000","last_fullsized":"2024-09-15T01:32:06.072519+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:48:11.629787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046258900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.072351+0000","last_change":"2024-09-15T01:32:06.072595+0000","last_active":"2024-09-15T01:32:06.072351+0000","last_peered":"2024-09-15T01:32:06.072351+0000","last_clean":"2024-09-15T01:32:06.072351+0000","last_became_active":"2024-09-15T01:32:04.058893+0000","last_became_peered":"2024-09-15T01:32:04.058893+0000","last_unstale":"2024-09-15T01:32:06.072351+0000","last_undegraded":"2024-09-15T01:32:06.072351+0000","last_fullsized":"2024-09-15T01:32:06.072351+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:38:08.540821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037511000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T01:32:06.071100+0000","last_change":"2024-09-15T01:32:06.071341+0000","last_active":"2024-09-15T01:32:06.071100+0000","last_peered":"2024-09-15T01:32:06.071100+0000","last_clean":"2024-09-15T01:32:06.071100+0000","last_became_active":"2024-09-15T01:32:04.057931+0000","last_became_peered":"2024-09-15T01:32:04.057931+0000","last_unstale":"2024-09-15T01:32:06.071100+0000","last_undegraded":"2024-09-15T01:32:06.071100+0000","last_fullsized":"2024-09-15T01:32:06.071100+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:10:36.309941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076976900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'72","reported_seq":103,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T01:32:05.068805+0000","last_change":"2024-09-15T01:32:00.032305+0000","last_active":"2024-09-15T01:32:05.068805+0000","last_peered":"2024-09-15T01:32:05.068805+0000","last_clean":"2024-09-15T01:32:05.068805+0000","last_became_active":"2024-09-15T01:32:00.032048+0000","last_became_peered":"2024-09-15T01:32:00.032048+0000","last_unstale":"2024-09-15T01:32:05.068805+0000","last_undegraded":"2024-09-15T01:32:05.068805+0000","last_fullsized":"2024-09-15T01:32:05.068805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_clean_scrub_stamp":"2024-09-15T01:31:59.015115+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:23:36.195853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":796940,"kb_used_data":92882684,"kb_used_omap":100,"kb_used_meta":0,"kb_avail":92882676,"statfs":{"total":95927926784,"available":95111860224,"internally_reserved":8192,"allocated":95111868416,"data_stored":95111868416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":102448,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795820,"kb_used_data":92883832,"kb_used_omap":62,"kb_used_meta":0,"kb_avail":92883796,"statfs":{"total":95927926784,"available":95113007104,"internally_reserved":36864,"allocated":95113043968,"data_stored":95113043968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":64366,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795784,"kb_used_data":92883856,"kb_used_omap":53,"kb_used_meta":0,"kb_avail":92883832,"statfs":{"total":95927926784,"available":95113043968,"internally_reserved":24576,"allocated":95113068544,"data_stored":95113068544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":55119,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795784,"kb_used_data":92883856,"kb_used_omap":55,"kb_used_meta":0,"kb_avail":92883832,"statfs":{"total":95927926784,"available":95113043968,"internally_reserved":24576,"allocated":95113068544,"data_stored":95113068544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56457,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]}]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":797008,"kb_used_data":92882636,"kb_used_omap":104,"kb_used_meta":0,"kb_avail":92882608,"statfs":{"total":95927926784,"available":95111790592,"internally_reserved":28672,"allocated":95111819264,"data_stored":95111819264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":107425,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":795820,"kb_used_data":92883832,"kb_used_omap":61,"kb_used_meta":0,"kb_avail":92883796,"statfs":{"total":95927926784,"available":95113007104,"internally_reserved":36864,"allocated":95113043968,"data_stored":95113043968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":62970,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.153}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.222}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[]}} 2024-09-15T01:32:12.641 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-09-15T01:32:12.642 DEBUG:tasks.mon_thrash:Yielding 2024-09-15T01:32:12.642 INFO:teuthology.run_tasks:Running task workunit... 2024-09-15T01:32:12.655 INFO:tasks.workunit:Pulling workunits from ref d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:32:12.655 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-15T01:32:12.655 DEBUG:teuthology.orchestra.run.smithi204:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-15T01:32:12.656 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-09-15T01:32:12.656 INFO:tasks.mon_thrash.mon_thrasher:seed: 1726363932, revive delay: 90.0, thrash delay: 1.0 thrash many: True, maintain quorum: True store thrash: True, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-09-15T01:32:12.656 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:32:12.656 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:32:12.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:32:12.676 INFO:teuthology.orchestra.run.smithi204.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-15T01:32:12.676 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-15T01:32:12.734 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-15T01:32:12.734 DEBUG:teuthology.orchestra.run.smithi204:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.746+0000 7f30964c9640 10 monclient: get_monmap_and_config 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.746+0000 7f30964c9640 10 monclient: build_initial_monmap 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.746+0000 7f30964c9640 10 monclient: monmap: 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:12.747080+0000 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:12.747080+0000 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:12.749 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:12.749 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:12.749 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:12.749 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:12.749 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:12.749 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:12.749 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:12.749 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.746+0000 7f30964c9640 1 Processor -- start 2024-09-15T01:32:12.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.746+0000 7f30964c9640 1 -- start start 2024-09-15T01:32:12.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.746+0000 7f30964c9640 10 monclient: init 2024-09-15T01:32:12.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 10 monclient: _add_conns ranks=[5,10,6,7,8,9,3,4,0,1,2] 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30900a2930 0x7f3090132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 10 monclient(hunting): picked mon.noname-h con 0x7f30900a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f309009faa0 0x7f309009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 10 monclient(hunting): picked mon.noname-f con 0x7f309009faa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f30900a08c0 0x7f30900a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 10 monclient(hunting): picked mon.noname-d con 0x7f30900a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309009e0f0 con 0x7f309009faa0 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f309009dd80 con 0x7f30900a2930 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f30900a2d00 con 0x7f30900a08c0 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f30964c9640 10 monclient(hunting): authenticate will time out at 737.497492s 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f30900a08c0 0x7f30900a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f30900a08c0 0x7f30900a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:40546/0 (socket says 172.21.15.178:40546) 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 1 -- 172.21.15.178:0/3921949772 learned_addr learned my addr 172.21.15.178:0/3921949772 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 10 monclient(hunting): get_auth_request con 0x7f30900a08c0 auth_method 0 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f308ffff640 1 --2- 172.21.15.178:0/3921949772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30900a2930 0x7f3090132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f308f7fe640 1 --2- 172.21.15.178:0/3921949772 >> v2:172.21.15.204:3300/0 conn(0x7f309009faa0 0x7f309009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f308ffff640 10 monclient(hunting): get_auth_request con 0x7f30900a2930 auth_method 0 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f308ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f308ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f308ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f308f7fe640 10 monclient(hunting): get_auth_request con 0x7f309009faa0 auth_method 0 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f308f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f308f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f308f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 10 monclient(hunting): handle_auth_done global_id 4452 payload 274 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 1 -- 172.21.15.178:0/3921949772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30900a2930 msgr2=0x7f3090132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 1 --2- 172.21.15.178:0/3921949772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30900a2930 0x7f3090132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 1 -- 172.21.15.178:0/3921949772 >> v2:172.21.15.204:3300/0 conn(0x7f309009faa0 msgr2=0x7f309009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 1 --2- 172.21.15.178:0/3921949772 >> v2:172.21.15.204:3300/0 conn(0x7f309009faa0 0x7f309009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 10 monclient: _finish_hunting 0 2024-09-15T01:32:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 1 monclient: found mon.noname-d 2024-09-15T01:32:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:32:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.747+0000 7f3094a66640 1 -- 172.21.15.178:0/3921949772 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309004bf60 con 0x7f30900a08c0 2024-09-15T01:32:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f3094a66640 1 --2- 172.21.15.178:0/3921949772 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f30900a08c0 0x7f30900a0d60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3088009f20 tx=0x7f3088009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=595e56bc9ba92be7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f308effd640 1 -- 172.21.15.178:0/3921949772 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3088007880 con 0x7f30900a08c0 2024-09-15T01:32:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f308effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f308effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:32:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f308effd640 10 monclient: dump: 2024-09-15T01:32:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f308effd640 10 monclient: _finish_auth 0 2024-09-15T01:32:12.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f308effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f30964c9640 5 monclient: authenticate success, global_id 4452 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f308effd640 1 -- 172.21.15.178:0/3921949772 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308802d050 con 0x7f30900a08c0 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f30964c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f308effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f30964c9640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f30964c9640 4 set_mon_vals no callback set 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f308effd640 1 -- 172.21.15.178:0/3921949772 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3088030360 con 0x7f30900a08c0 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f308effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f308effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.748+0000 7f308effd640 10 monclient: dump: 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.763 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:12.764 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 10 monclient: shutdown 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 1 -- 172.21.15.178:0/3921949772 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f30900a08c0 msgr2=0x7f30900a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 1 --2- 172.21.15.178:0/3921949772 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f30900a08c0 0x7f30900a0d60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3088009f20 tx=0x7f3088009840 comp rx=0 tx=0).stop 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 1 -- 172.21.15.178:0/3921949772 shutdown_connections 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 1 --2- 172.21.15.178:0/3921949772 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f30900a08c0 0x7f30900a0d60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 1 --2- 172.21.15.178:0/3921949772 >> v2:172.21.15.204:3300/0 conn(0x7f309009faa0 0x7f309009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 1 --2- 172.21.15.178:0/3921949772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30900a2930 0x7f3090132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 1 -- 172.21.15.178:0/3921949772 >> 172.21.15.178:0/3921949772 conn(0x7f30900a5ed0 msgr2=0x7f30900a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 1 -- 172.21.15.178:0/3921949772 shutdown_connections 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 1 -- 172.21.15.178:0/3921949772 wait complete. 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 10 monclient: build_initial_monmap 2024-09-15T01:32:12.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.749+0000 7f30964c9640 10 monclient: monmap: 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 1 Processor -- start 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 1 -- start start 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 10 monclient: init 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 10 monclient: _add_conns ranks=[3,5,1,6,4,0,2,7,8] 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f309009faa0 0x7f309009b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 10 monclient(hunting): picked mon.noname-g con 0x7f309009faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30900a08c0 0x7f30901318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 10 monclient(hunting): picked mon.noname-h con 0x7f30900a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:12.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30900a2930 0x7f3090141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 10 monclient(hunting): picked mon.noname-f con 0x7f30900a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:12.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309011e9d0 con 0x7f30900a2930 2024-09-15T01:32:12.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309011ecb0 con 0x7f309009faa0 2024-09-15T01:32:12.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:12.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f309011ef90 con 0x7f30900a08c0 2024-09-15T01:32:12.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:12.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.750+0000 7f30964c9640 10 monclient(hunting): authenticate will time out at 737.500913s 2024-09-15T01:32:12.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308f7fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30900a08c0 0x7f30901318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f309009faa0 0x7f309009b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308f7fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30900a08c0 0x7f30901318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:48152/0 (socket says 172.21.15.178:48152) 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3094a66640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30900a2930 0x7f3090141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 10 monclient(hunting): get_auth_request con 0x7f309009faa0 auth_method 0 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308f7fe640 1 -- 172.21.15.178:0/3898885098 learned_addr learned my addr 172.21.15.178:0/3898885098 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308f7fe640 10 monclient(hunting): get_auth_request con 0x7f30900a08c0 auth_method 0 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3094a66640 10 monclient(hunting): get_auth_request con 0x7f30900a2930 auth_method 0 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3094a66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3094a66640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3094a66640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3094a66640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3094a66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3094a66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 10 monclient(hunting): handle_auth_done global_id 4413 payload 824 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 1 -- 172.21.15.178:0/3898885098 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30900a08c0 msgr2=0x7f30901318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 1 --2- 172.21.15.178:0/3898885098 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30900a08c0 0x7f30901318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 1 -- 172.21.15.178:0/3898885098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30900a2930 msgr2=0x7f3090141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 1 --2- 172.21.15.178:0/3898885098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30900a2930 0x7f3090141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 1 monclient: found mon.noname-g 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 1 -- 172.21.15.178:0/3898885098 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3088009ae0 con 0x7f309009faa0 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3094a66640 1 --2- 172.21.15.178:0/3898885098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30900a2930 0x7f3090141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308ffff640 1 --2- 172.21.15.178:0/3898885098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f309009faa0 0x7f309009b0b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f308000d890 tx=0x7f3080008040 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3087fff640 1 -- 172.21.15.178:0/3898885098 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f308000bb40 con 0x7f309009faa0 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3087fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3087fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3087fff640 10 monclient: dump: 2024-09-15T01:32:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:12.773 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3087fff640 10 monclient: _finish_auth 0 2024-09-15T01:32:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3087fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f30964c9640 5 monclient: authenticate success, global_id 4413 2024-09-15T01:32:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3087fff640 1 -- 172.21.15.178:0/3898885098 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308000bce0 con 0x7f309009faa0 2024-09-15T01:32:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3087fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f30964c9640 10 monclient: _renew_subs 2024-09-15T01:32:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3087fff640 1 -- 172.21.15.178:0/3898885098 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f308000be50 con 0x7f309009faa0 2024-09-15T01:32:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f30964c9640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:32:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f30964c9640 1 -- 172.21.15.178:0/3898885098 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309011f270 con 0x7f309009faa0 2024-09-15T01:32:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3087fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f308e7fc640 4 set_mon_vals no callback set 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3087fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f3087fff640 10 monclient: dump: 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:12.774 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f30964c9640 10 monclient: _renew_subs 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f30964c9640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.751+0000 7f30964c9640 1 -- 172.21.15.178:0/3898885098 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309011dd00 con 0x7f309009faa0 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.752+0000 7f3087fff640 1 -- 172.21.15.178:0/3898885098 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f308000b6c0 con 0x7f309009faa0 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.752+0000 7f30964c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.752+0000 7f30964c9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:12.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.752+0000 7f30964c9640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:32:12.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.752+0000 7f30964c9640 1 -- 172.21.15.178:0/3898885098 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309004b720 con 0x7f309009faa0 2024-09-15T01:32:12.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.752+0000 7f3087fff640 1 --2- 172.21.15.178:0/3898885098 >> v2:172.21.15.204:6812/28210 conn(0x7f30700396e0 0x7f307003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:12.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.753+0000 7f3087fff640 1 -- 172.21.15.178:0/3898885098 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f308004b510 con 0x7f309009faa0 2024-09-15T01:32:12.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.753+0000 7f308f7fe640 1 --2- 172.21.15.178:0/3898885098 >> v2:172.21.15.204:6812/28210 conn(0x7f30700396e0 0x7f307003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:12.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.753+0000 7f308f7fe640 10 monclient: get_auth_request con 0x7f30700396e0 auth_method 0 2024-09-15T01:32:12.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.753+0000 7f308f7fe640 1 --2- 172.21.15.178:0/3898885098 >> v2:172.21.15.204:6812/28210 conn(0x7f30700396e0 0x7f307003bb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f309009c290 tx=0x7f3078008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:12.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.758+0000 7f3087fff640 1 -- 172.21.15.178:0/3898885098 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f308000b970 con 0x7f309009faa0 2024-09-15T01:32:12.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.758+0000 7f3087fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:12.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.758+0000 7f3087fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:12.793 INFO:tasks.workunit:timeout=3h 2024-09-15T01:32:12.794 INFO:tasks.workunit:cleanup=True 2024-09-15T01:32:12.794 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d75f23820806da622f561ee5ee7551015285993f 2024-09-15T01:32:12.853 INFO:tasks.workunit.client.0.smithi204.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-15T01:32:12.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.970+0000 7f30964c9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:32:12.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.970+0000 7f30964c9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:32:12.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.970+0000 7f30964c9640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:32:12.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.970+0000 7f30964c9640 1 -- 172.21.15.178:0/3898885098 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f309009cef0 con 0x7f309009faa0 2024-09-15T01:32:12.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.971+0000 7f3087fff640 1 -- 172.21.15.178:0/3898885098 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7f308001d3c0 con 0x7f309009faa0 2024-09-15T01:32:12.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.971+0000 7f3087fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:32:12.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.971+0000 7f3087fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:12.973 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":12,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:32:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 1 -- 172.21.15.178:0/3898885098 >> v2:172.21.15.204:6812/28210 conn(0x7f30700396e0 msgr2=0x7f307003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 1 --2- 172.21.15.178:0/3898885098 >> v2:172.21.15.204:6812/28210 conn(0x7f30700396e0 0x7f307003bb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f309009c290 tx=0x7f3078008040 comp rx=0 tx=0).stop 2024-09-15T01:32:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 10 monclient: shutdown 2024-09-15T01:32:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 1 -- 172.21.15.178:0/3898885098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f309009faa0 msgr2=0x7f309009b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 1 --2- 172.21.15.178:0/3898885098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f309009faa0 0x7f309009b0b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f308000d890 tx=0x7f3080008040 comp rx=0 tx=0).stop 2024-09-15T01:32:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 1 -- 172.21.15.178:0/3898885098 shutdown_connections 2024-09-15T01:32:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 1 --2- 172.21.15.178:0/3898885098 >> v2:172.21.15.204:6812/28210 conn(0x7f30700396e0 0x7f307003bb90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 1 --2- 172.21.15.178:0/3898885098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30900a2930 0x7f3090141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 1 --2- 172.21.15.178:0/3898885098 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30900a08c0 0x7f30901318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 1 --2- 172.21.15.178:0/3898885098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f309009faa0 0x7f309009b0b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 1 -- 172.21.15.178:0/3898885098 >> 172.21.15.178:0/3898885098 conn(0x7f30900a5ed0 msgr2=0x7f309009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 1 -- 172.21.15.178:0/3898885098 shutdown_connections 2024-09-15T01:32:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:12.973+0000 7f30964c9640 1 -- 172.21.15.178:0/3898885098 wait complete. 2024-09-15T01:32:12.986 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:32:12.986 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T01:32:12.986 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:32:12.986 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:32:13.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.078+0000 7fcf8ad5d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:13.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.078+0000 7fcf8ad5d640 10 monclient: build_initial_monmap 2024-09-15T01:32:13.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.078+0000 7fcf8ad5d640 10 monclient: monmap: 2024-09-15T01:32:13.080 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:13.080 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.080 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:13.079303+0000 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:13.079303+0000 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:13.081 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:13.082 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 1 Processor -- start 2024-09-15T01:32:13.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 1 -- start start 2024-09-15T01:32:13.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 10 monclient: init 2024-09-15T01:32:13.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 10 monclient: _add_conns ranks=[3,5,9,6,0,1,8,4,10,2,7] 2024-09-15T01:32:13.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf840a2930 0x7fcf84132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 10 monclient(hunting): picked mon.noname-g con 0x7fcf840a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:13.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcf8409faa0 0x7fcf8409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 10 monclient(hunting): picked mon.noname-h con 0x7fcf8409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:13.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcf840a08c0 0x7fcf840a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 10 monclient(hunting): picked mon.noname-a con 0x7fcf840a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:32:13.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf8409e0f0 con 0x7fcf840a08c0 2024-09-15T01:32:13.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8409dd80 con 0x7fcf840a2930 2024-09-15T01:32:13.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840a2d00 con 0x7fcf8409faa0 2024-09-15T01:32:13.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf8ad5d640 10 monclient(hunting): authenticate will time out at 737.829699s 2024-09-15T01:32:13.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf892fa640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcf840a08c0 0x7fcf840a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf892fa640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcf840a08c0 0x7fcf840a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35070/0 (socket says 172.21.15.178:35070) 2024-09-15T01:32:13.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf892fa640 1 -- 172.21.15.178:0/456182694 learned_addr learned my addr 172.21.15.178:0/456182694 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:13.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf892fa640 10 monclient(hunting): get_auth_request con 0x7fcf840a08c0 auth_method 0 2024-09-15T01:32:13.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf892fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf892fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf892fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf892fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf892fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf892fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf83fff640 1 --2- 172.21.15.178:0/456182694 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcf8409faa0 0x7fcf8409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.079+0000 7fcf88af9640 1 --2- 172.21.15.178:0/456182694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf840a2930 0x7fcf84132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf83fff640 10 monclient(hunting): get_auth_request con 0x7fcf8409faa0 auth_method 0 2024-09-15T01:32:13.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf83fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf88af9640 10 monclient(hunting): get_auth_request con 0x7fcf840a2930 auth_method 0 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf88af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf88af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf88af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf892fa640 10 monclient(hunting): handle_auth_done global_id 4482 payload 274 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf892fa640 1 -- 172.21.15.178:0/456182694 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcf8409faa0 msgr2=0x7fcf8409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf892fa640 1 --2- 172.21.15.178:0/456182694 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcf8409faa0 0x7fcf8409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf892fa640 1 -- 172.21.15.178:0/456182694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf840a2930 msgr2=0x7fcf84132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf892fa640 1 --2- 172.21.15.178:0/456182694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf840a2930 0x7fcf84132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf892fa640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf892fa640 1 monclient: found mon.noname-a 2024-09-15T01:32:13.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf892fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf892fa640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf892fa640 1 -- 172.21.15.178:0/456182694 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8404bf60 con 0x7fcf840a08c0 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf83fff640 1 --2- 172.21.15.178:0/456182694 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcf8409faa0 0x7fcf8409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf88af9640 1 --2- 172.21.15.178:0/456182694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf840a2930 0x7fcf84132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf892fa640 1 --2- 172.21.15.178:0/456182694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf840a08c0 0x7fcf840a0d30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009f20 tx=0x7fcf7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af174d958997dd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf837fe640 1 -- 172.21.15.178:0/456182694 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcf7c007880 con 0x7fcf840a08c0 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf837fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf837fe640 10 monclient: dump: 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.095 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf837fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf837fe640 1 -- 172.21.15.178:0/456182694 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7c02d050 con 0x7fcf840a08c0 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf8ad5d640 5 monclient: authenticate success, global_id 4482 2024-09-15T01:32:13.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf8ad5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf837fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf8ad5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf837fe640 1 -- 172.21.15.178:0/456182694 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcf7c030360 con 0x7fcf840a08c0 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.080+0000 7fcf8ad5d640 4 set_mon_vals no callback set 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf837fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf837fe640 10 monclient: dump: 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.097 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf8ad5d640 10 monclient: shutdown 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf8ad5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/456182694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf840a08c0 msgr2=0x7fcf840a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf8ad5d640 1 --2- 172.21.15.178:0/456182694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf840a08c0 0x7fcf840a0d30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009f20 tx=0x7fcf7c009840 comp rx=0 tx=0).stop 2024-09-15T01:32:13.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/456182694 shutdown_connections 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf8ad5d640 1 --2- 172.21.15.178:0/456182694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf840a08c0 0x7fcf840a0d30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf8ad5d640 1 --2- 172.21.15.178:0/456182694 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcf8409faa0 0x7fcf8409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf8ad5d640 1 --2- 172.21.15.178:0/456182694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf840a2930 0x7fcf84132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/456182694 >> 172.21.15.178:0/456182694 conn(0x7fcf840a5ed0 msgr2=0x7fcf840a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/456182694 shutdown_connections 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.081+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/456182694 wait complete. 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 10 monclient: build_initial_monmap 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 10 monclient: monmap: 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:13.099 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 1 Processor -- start 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 1 -- start start 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 10 monclient: init 2024-09-15T01:32:13.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 10 monclient: _add_conns ranks=[0,3,1,5,8,7,6,2,4] 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf8409faa0 0x7fcf8409b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 10 monclient(hunting): picked mon.noname-a con 0x7fcf8409faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf840a08c0 0x7fcf8411ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 10 monclient(hunting): picked mon.noname-g con 0x7fcf840a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf840a2930 0x7fcf84131a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.082+0000 7fcf8ad5d640 10 monclient(hunting): picked mon.noname-f con 0x7fcf840a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8409d630 con 0x7fcf8409faa0 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8411f190 con 0x7fcf840a2930 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8411d7e0 con 0x7fcf840a08c0 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 10 monclient(hunting): authenticate will time out at 737.833026s 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf8409faa0 0x7fcf8409b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf8409faa0 0x7fcf8409b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35078/0 (socket says 172.21.15.178:35078) 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 1 -- 172.21.15.178:0/1458614602 learned_addr learned my addr 172.21.15.178:0/1458614602 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 10 monclient(hunting): get_auth_request con 0x7fcf8409faa0 auth_method 0 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf83fff640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf840a08c0 0x7fcf8411ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf83fff640 10 monclient(hunting): get_auth_request con 0x7fcf840a08c0 auth_method 0 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf83fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf892fa640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf840a2930 0x7fcf84131a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf892fa640 10 monclient(hunting): get_auth_request con 0x7fcf840a2930 auth_method 0 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf892fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf892fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf892fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 10 monclient(hunting): handle_auth_done global_id 4491 payload 824 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 1 -- 172.21.15.178:0/1458614602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf840a08c0 msgr2=0x7fcf8411ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf840a08c0 0x7fcf8411ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 1 -- 172.21.15.178:0/1458614602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf840a2930 msgr2=0x7fcf84131a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf840a2930 0x7fcf84131a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 1 monclient: found mon.noname-a 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 1 -- 172.21.15.178:0/1458614602 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c009ae0 con 0x7fcf8409faa0 2024-09-15T01:32:13.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf83fff640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcf840a08c0 0x7fcf8411ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf88af9640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf8409faa0 0x7fcf8409b260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800d890 tx=0x7fcf78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcf7800de80 con 0x7fcf8409faa0 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf80ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf80ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf80ff9640 10 monclient: dump: 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.104 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf80ff9640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf80ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf78009d50 con 0x7fcf8409faa0 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 5 monclient: authenticate success, global_id 4491 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf80ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:13.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcf78014890 con 0x7fcf8409faa0 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 10 monclient: _renew_subs 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/1458614602 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8411dbe0 con 0x7fcf8409faa0 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf80ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf80ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf82ffd640 4 set_mon_vals no callback set 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf80ff9640 10 monclient: dump: 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.106 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 10 monclient: _renew_subs 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.083+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/1458614602 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8411ee20 con 0x7fcf8409faa0 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf8ad5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf8ad5d640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84131f60 0x7fcf84141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf8ad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/1458614602 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84132330 con 0x7fcf84131f60 2024-09-15T01:32:13.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fcf78014410 con 0x7fcf8409faa0 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf83fff640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84131f60 0x7fcf84141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf83fff640 10 monclient: get_auth_request con 0x7fcf84131f60 auth_method 0 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf83fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf80ff9640 1 --2- 172.21.15.178:0/1458614602 >> v2:172.21.15.204:6812/28210 conn(0x7fcf680397b0 0x7fcf6803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf83fff640 10 monclient(hunting): handle_auth_done global_id 4500 payload 824 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf83fff640 1 -- 172.21.15.178:0/1458614602 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf8404c4a0 con 0x7fcf84131f60 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7fcf7804b760 con 0x7fcf8409faa0 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf892fa640 1 --2- 172.21.15.178:0/1458614602 >> v2:172.21.15.204:6812/28210 conn(0x7fcf680397b0 0x7fcf6803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf83fff640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84131f60 0x7fcf84141dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000f560 tx=0x7fcf70008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.084+0000 7fcf892fa640 10 monclient: get_auth_request con 0x7fcf680397b0 auth_method 0 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.085+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcf70014af0 con 0x7fcf84131f60 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.085+0000 7fcf80ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.085+0000 7fcf80ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.085+0000 7fcf80ff9640 10 monclient: dump: 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.109 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.085+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fcf70014c50 con 0x7fcf84131f60 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.085+0000 7fcf80ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.085+0000 7fcf80ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.085+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84131f60 msgr2=0x7fcf84141dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.085+0000 7fcf80ff9640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84131f60 0x7fcf84141dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000f560 tx=0x7fcf70008040 comp rx=0 tx=0).stop 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.085+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84131f60 msgr2=0x7fcf84141dc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:13.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.085+0000 7fcf80ff9640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84131f60 0x7fcf84141dc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.085+0000 7fcf892fa640 1 --2- 172.21.15.178:0/1458614602 >> v2:172.21.15.204:6812/28210 conn(0x7fcf680397b0 0x7fcf6803bc60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c033000 tx=0x7fcf7c036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.090+0000 7fcf8ad5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:13.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.090+0000 7fcf8ad5d640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84142300 0x7fcf84142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.090+0000 7fcf8ad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.090+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/1458614602 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840a0180 con 0x7fcf84142300 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf83fff640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84142300 0x7fcf84142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf83fff640 10 monclient: get_auth_request con 0x7fcf84142300 auth_method 0 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf83fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf83fff640 10 monclient(hunting): handle_auth_done global_id 4509 payload 824 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf83fff640 1 -- 172.21.15.178:0/1458614602 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcf8404bc80 con 0x7fcf84142300 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf83fff640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84142300 0x7fcf84142700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000b5f0 tx=0x7fcf7001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcf70015b80 con 0x7fcf84142300 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf80ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf80ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf80ff9640 10 monclient: dump: 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.115 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.115 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9144 (secure 0 0 0) 0x7fcf70015ce0 con 0x7fcf84142300 2024-09-15T01:32:13.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf80ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:13.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf80ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:13.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84142300 msgr2=0x7fcf84142700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf80ff9640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84142300 0x7fcf84142700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000b5f0 tx=0x7fcf7001b000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf80ff9640 1 -- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84142300 msgr2=0x7fcf84142700 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:13.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.091+0000 7fcf80ff9640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84142300 0x7fcf84142700 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000b5f0 tx=0x7fcf7001b000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.115 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:13.115 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 12, 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:13.116 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 21, 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.117 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:13.118 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:13.119 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.120 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:13.121 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:13.121 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.121 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:13.121 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.121 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:13.122 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:13.123 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.123 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:13.123 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.123 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:13.123 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.123 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.124 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.125 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:13.126 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.127 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.128 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.129 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:13.130 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.132 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.133 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.135 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.137 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:13.137 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:13.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.092+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/1458614602 >> v2:172.21.15.204:6812/28210 conn(0x7fcf680397b0 msgr2=0x7fcf6803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.092+0000 7fcf8ad5d640 1 --2- 172.21.15.178:0/1458614602 >> v2:172.21.15.204:6812/28210 conn(0x7fcf680397b0 0x7fcf6803bc60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c033000 tx=0x7fcf7c036000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.092+0000 7fcf8ad5d640 10 monclient: shutdown 2024-09-15T01:32:13.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.092+0000 7fcf8ad5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:13.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.092+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf8409faa0 msgr2=0x7fcf8409b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.092+0000 7fcf8ad5d640 1 --2- 172.21.15.178:0/1458614602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf8409faa0 0x7fcf8409b260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800d890 tx=0x7fcf78008040 comp rx=0 tx=0).stop 2024-09-15T01:32:13.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.092+0000 7fcf892fa640 1 -- 172.21.15.178:0/1458614602 reap_dead start 2024-09-15T01:32:13.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.092+0000 7fcf892fa640 1 -- 172.21.15.178:0/1458614602 reap_dead start 2024-09-15T01:32:13.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.092+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/1458614602 shutdown_connections 2024-09-15T01:32:13.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.092+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/1458614602 >> 172.21.15.178:0/1458614602 conn(0x7fcf840a5ed0 msgr2=0x7fcf8409e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:13.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.092+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/1458614602 shutdown_connections 2024-09-15T01:32:13.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.092+0000 7fcf8ad5d640 1 -- 172.21.15.178:0/1458614602 wait complete. 2024-09-15T01:32:13.139 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:32:13.139 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:32:13.139 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:32:13.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.231+0000 7f46e9460640 10 monclient: get_monmap_and_config 2024-09-15T01:32:13.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.231+0000 7f46e9460640 10 monclient: build_initial_monmap 2024-09-15T01:32:13.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.231+0000 7f46e9460640 10 monclient: monmap: 2024-09-15T01:32:13.233 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:13.233 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.234 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:13.232223+0000 2024-09-15T01:32:13.234 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:13.232223+0000 2024-09-15T01:32:13.234 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:13.234 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.234 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:13.234 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:13.234 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:13.234 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:13.234 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:13.234 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:13.234 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:13.234 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:13.235 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:13.235 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:13.235 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:13.235 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 1 Processor -- start 2024-09-15T01:32:13.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 1 -- start start 2024-09-15T01:32:13.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 10 monclient: init 2024-09-15T01:32:13.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 10 monclient: _add_conns ranks=[3,4,9,0,10,7,1,6,2,5,8] 2024-09-15T01:32:13.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46e40a2930 0x7f46e4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 10 monclient(hunting): picked mon.noname-g con 0x7f46e40a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:13.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46e409faa0 0x7f46e409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 10 monclient(hunting): picked mon.noname-c con 0x7f46e409faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:13.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f46e40a08c0 0x7f46e40a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 10 monclient(hunting): picked mon.noname-a con 0x7f46e40a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:32:13.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e409e0f0 con 0x7f46e40a08c0 2024-09-15T01:32:13.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e409dd80 con 0x7f46e40a2930 2024-09-15T01:32:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40a2d00 con 0x7f46e409faa0 2024-09-15T01:32:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e9460640 10 monclient(hunting): authenticate will time out at 737.982633s 2024-09-15T01:32:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e37fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f46e40a08c0 0x7f46e40a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46dbfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46e409faa0 0x7f46e409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e37fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f46e40a08c0 0x7f46e40a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35104/0 (socket says 172.21.15.178:35104) 2024-09-15T01:32:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e37fe640 1 -- 172.21.15.178:0/796749258 learned_addr learned my addr 172.21.15.178:0/796749258 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e37fe640 10 monclient(hunting): get_auth_request con 0x7f46e40a08c0 auth_method 0 2024-09-15T01:32:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46dbfff640 10 monclient(hunting): get_auth_request con 0x7f46e409faa0 auth_method 0 2024-09-15T01:32:13.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e2ffd640 1 --2- 172.21.15.178:0/796749258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46e40a2930 0x7f46e4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e2ffd640 10 monclient(hunting): get_auth_request con 0x7f46e40a2930 auth_method 0 2024-09-15T01:32:13.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.232+0000 7f46dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e37fe640 10 monclient(hunting): handle_auth_done global_id 4518 payload 274 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e37fe640 1 -- 172.21.15.178:0/796749258 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46e409faa0 msgr2=0x7f46e409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e37fe640 1 --2- 172.21.15.178:0/796749258 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46e409faa0 0x7f46e409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e37fe640 1 -- 172.21.15.178:0/796749258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46e40a2930 msgr2=0x7f46e4132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e37fe640 1 --2- 172.21.15.178:0/796749258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46e40a2930 0x7f46e4132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e37fe640 1 monclient: found mon.noname-a 2024-09-15T01:32:13.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e37fe640 1 -- 172.21.15.178:0/796749258 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e404bf60 con 0x7f46e40a08c0 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46dbfff640 1 --2- 172.21.15.178:0/796749258 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46e409faa0 0x7f46e409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e37fe640 1 --2- 172.21.15.178:0/796749258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e40a08c0 0x7f46e40a0d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009f20 tx=0x7f46dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dab16217fdce11 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e27fc640 1 -- 172.21.15.178:0/796749258 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46dc007880 con 0x7f46e40a08c0 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e27fc640 10 monclient: dump: 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.249 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e27fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e27fc640 1 -- 172.21.15.178:0/796749258 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc02d050 con 0x7f46e40a08c0 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e9460640 5 monclient: authenticate success, global_id 4518 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e9460640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:13.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e27fc640 1 -- 172.21.15.178:0/796749258 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46dc030360 con 0x7f46e40a08c0 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.233+0000 7f46e9460640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e9460640 4 set_mon_vals no callback set 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e27fc640 10 monclient: dump: 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e9460640 10 monclient: shutdown 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e9460640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e9460640 1 -- 172.21.15.178:0/796749258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e40a08c0 msgr2=0x7f46e40a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e9460640 1 --2- 172.21.15.178:0/796749258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e40a08c0 0x7f46e40a0d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009f20 tx=0x7f46dc009840 comp rx=0 tx=0).stop 2024-09-15T01:32:13.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e9460640 1 -- 172.21.15.178:0/796749258 shutdown_connections 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e9460640 1 --2- 172.21.15.178:0/796749258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e40a08c0 0x7f46e40a0d30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e9460640 1 --2- 172.21.15.178:0/796749258 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46e409faa0 0x7f46e409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e9460640 1 --2- 172.21.15.178:0/796749258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46e40a2930 0x7f46e4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e9460640 1 -- 172.21.15.178:0/796749258 >> 172.21.15.178:0/796749258 conn(0x7f46e40a5ed0 msgr2=0x7f46e40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e9460640 1 -- 172.21.15.178:0/796749258 shutdown_connections 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.234+0000 7f46e9460640 1 -- 172.21.15.178:0/796749258 wait complete. 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient: build_initial_monmap 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient: monmap: 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 1 Processor -- start 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 1 -- start start 2024-09-15T01:32:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient: init 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient: _add_conns ranks=[6,0,2,4,1,5,3,7,8] 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f46e409faa0 0x7f46e409b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient(hunting): picked mon.noname-d con 0x7f46e409faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e40a08c0 0x7f46e4131800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient(hunting): picked mon.noname-a con 0x7f46e40a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e40a2930 0x7f46e411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient(hunting): picked mon.noname-b con 0x7f46e40a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40a2390 con 0x7f46e40a08c0 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e411ec00 con 0x7f46e40a2930 2024-09-15T01:32:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f46e411eee0 con 0x7f46e409faa0 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.235+0000 7f46e9460640 10 monclient(hunting): authenticate will time out at 737.985929s 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e2ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f46e409faa0 0x7f46e409b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e40a08c0 0x7f46e4131800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e2ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f46e409faa0 0x7f46e409b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:40556/0 (socket says 172.21.15.178:40556) 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e2ffd640 1 -- 172.21.15.178:0/619619923 learned_addr learned my addr 172.21.15.178:0/619619923 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e2ffd640 10 monclient(hunting): get_auth_request con 0x7f46e409faa0 auth_method 0 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e40a08c0 0x7f46e4131800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35118/0 (socket says 172.21.15.178:35118) 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 10 monclient(hunting): get_auth_request con 0x7f46e40a08c0 auth_method 0 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e37fe640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e40a2930 0x7f46e411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e37fe640 10 monclient(hunting): get_auth_request con 0x7f46e40a2930 auth_method 0 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 10 monclient(hunting): handle_auth_done global_id 4527 payload 824 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 1 -- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f46e409faa0 msgr2=0x7f46e409b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f46e409faa0 0x7f46e409b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 1 -- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e40a2930 msgr2=0x7f46e411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e40a2930 0x7f46e411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 1 monclient: found mon.noname-a 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 1 -- 172.21.15.178:0/619619923 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc009390 con 0x7f46e40a08c0 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e2ffd640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f46e409faa0 0x7f46e409b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46dbfff640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e40a08c0 0x7f46e4131800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46e40a6ec0 tx=0x7f46cc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46cc0095d0 con 0x7f46e40a08c0 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46daffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46daffd640 10 monclient: dump: 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46daffd640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc00cdc0 con 0x7f46e40a08c0 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e9460640 5 monclient: authenticate success, global_id 4527 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e9460640 10 monclient: _renew_subs 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e9460640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e9460640 1 -- 172.21.15.178:0/619619923 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46dc009ae0 con 0x7f46e40a08c0 2024-09-15T01:32:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46daffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46cc0138f0 con 0x7f46e40a08c0 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46daffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46daffd640 10 monclient: dump: 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e1ffb640 4 set_mon_vals no callback set 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e9460640 10 monclient: _renew_subs 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e9460640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.236+0000 7f46e9460640 1 -- 172.21.15.178:0/619619923 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e411e920 con 0x7f46e40a08c0 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f46cc013470 con 0x7f46e40a08c0 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e9460640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46daffd640 1 --2- 172.21.15.178:0/619619923 >> v2:172.21.15.204:6812/28210 conn(0x7f46c40397e0 0x7f46c403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e9460640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e4131d40 0x7f46e411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e9460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e9460640 1 -- 172.21.15.178:0/619619923 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e4132110 con 0x7f46e4131d40 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f46cc04b130 con 0x7f46e40a08c0 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e37fe640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e4131d40 0x7f46e411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e37fe640 10 monclient: get_auth_request con 0x7f46e4131d40 auth_method 0 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.237+0000 7f46e2ffd640 1 --2- 172.21.15.178:0/619619923 >> v2:172.21.15.204:6812/28210 conn(0x7f46c40397e0 0x7f46c403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46e2ffd640 10 monclient: get_auth_request con 0x7f46c40397e0 auth_method 0 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46e37fe640 10 monclient(hunting): handle_auth_done global_id 4421 payload 824 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46e37fe640 1 -- 172.21.15.178:0/619619923 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46e404c4a0 con 0x7f46e4131d40 2024-09-15T01:32:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46e37fe640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e4131d40 0x7f46e411ca70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f46e40a3290 tx=0x7f46dc036000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46dc03ca80 con 0x7f46e4131d40 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46daffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46daffd640 10 monclient: dump: 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46e2ffd640 1 --2- 172.21.15.178:0/619619923 >> v2:172.21.15.204:6812/28210 conn(0x7f46c40397e0 0x7f46c403bc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46d40114a0 tx=0x7f46d40090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f46dc03cbe0 con 0x7f46e4131d40 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46daffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46daffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e4131d40 msgr2=0x7f46e411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46daffd640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e4131d40 0x7f46e411ca70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f46e40a3290 tx=0x7f46dc036000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e4131d40 msgr2=0x7f46e411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.238+0000 7f46daffd640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e4131d40 0x7f46e411ca70 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f46e40a3290 tx=0x7f46dc036000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e9460640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e9460640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e411cfb0 0x7f46e4096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e9460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e9460640 1 -- 172.21.15.178:0/619619923 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e411ed70 con 0x7f46e411cfb0 2024-09-15T01:32:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e37fe640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e411cfb0 0x7f46e4096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e37fe640 10 monclient: get_auth_request con 0x7f46e411cfb0 auth_method 0 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e37fe640 10 monclient(hunting): handle_auth_done global_id 4430 payload 824 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e37fe640 1 -- 172.21.15.178:0/619619923 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f46e404bc80 con 0x7f46e411cfb0 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46e37fe640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e411cfb0 0x7f46e4096a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f46e409b8d0 tx=0x7f46dc03e070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46dc049070 con 0x7f46e411cfb0 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46daffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.244+0000 7f46daffd640 10 monclient: dump: 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.270 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f46dc03b3f0 con 0x7f46e411cfb0 2024-09-15T01:32:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46daffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46daffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e411cfb0 msgr2=0x7f46e4096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46daffd640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e411cfb0 0x7f46e4096a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f46e409b8d0 tx=0x7f46dc03e070 comp rx=0 tx=0).stop 2024-09-15T01:32:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46daffd640 1 -- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e411cfb0 msgr2=0x7f46e4096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46daffd640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46e411cfb0 0x7f46e4096a30 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f46e409b8d0 tx=0x7f46dc03e070 comp rx=0 tx=0).stop 2024-09-15T01:32:13.270 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:13.270 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 12, 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:13.271 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 21, 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.272 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.273 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:13.274 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:13.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.276 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:13.277 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.278 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.279 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:13.280 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.280 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.280 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.280 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.280 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:13.280 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.280 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.280 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.280 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.280 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:13.280 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:13.280 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.281 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:13.282 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.283 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.284 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.285 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.286 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:13.287 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.288 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.289 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.290 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:13.291 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.292 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.292 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:13.292 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:13.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46e9460640 1 -- 172.21.15.178:0/619619923 >> v2:172.21.15.204:6812/28210 conn(0x7f46c40397e0 msgr2=0x7f46c403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46e9460640 1 --2- 172.21.15.178:0/619619923 >> v2:172.21.15.204:6812/28210 conn(0x7f46c40397e0 0x7f46c403bc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46d40114a0 tx=0x7f46d40090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46e9460640 10 monclient: shutdown 2024-09-15T01:32:13.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46e9460640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:13.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46e9460640 1 -- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e40a08c0 msgr2=0x7f46e4131800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46e37fe640 1 -- 172.21.15.178:0/619619923 reap_dead start 2024-09-15T01:32:13.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46e9460640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e40a08c0 0x7f46e4131800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46e40a6ec0 tx=0x7f46cc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46e9460640 1 -- 172.21.15.178:0/619619923 shutdown_connections 2024-09-15T01:32:13.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46e9460640 1 --2- 172.21.15.178:0/619619923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e40a08c0 0x7f46e4131800 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46e9460640 1 -- 172.21.15.178:0/619619923 >> 172.21.15.178:0/619619923 conn(0x7f46e40a5ed0 msgr2=0x7f46e409e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:13.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46e9460640 1 -- 172.21.15.178:0/619619923 shutdown_connections 2024-09-15T01:32:13.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.245+0000 7f46e9460640 1 -- 172.21.15.178:0/619619923 wait complete. 2024-09-15T01:32:13.294 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:32:13.294 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:32:13.294 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:32:13.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.386+0000 7f33f0ea8640 10 monclient: get_monmap_and_config 2024-09-15T01:32:13.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.386+0000 7f33f0ea8640 10 monclient: build_initial_monmap 2024-09-15T01:32:13.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.386+0000 7f33f0ea8640 10 monclient: monmap: 2024-09-15T01:32:13.388 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:13.388 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.388 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:13.387284+0000 2024-09-15T01:32:13.388 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:13.387284+0000 2024-09-15T01:32:13.388 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:13.388 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.389 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:13.389 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:13.389 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:13.389 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:13.389 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:13.389 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:13.389 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:13.389 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:13.392 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:13.392 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:13.392 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:13.392 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.386+0000 7f33f0ea8640 1 Processor -- start 2024-09-15T01:32:13.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 1 -- start start 2024-09-15T01:32:13.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 10 monclient: init 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 10 monclient: _add_conns ranks=[3,7,8,5,2,4,6,10,1,9,0] 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ec0a2930 0x7f33ec132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 10 monclient(hunting): picked mon.noname-g con 0x7f33ec0a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f33ec09faa0 0x7f33ec09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 10 monclient(hunting): picked mon.noname-i con 0x7f33ec09faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 0x7f33ec0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 10 monclient(hunting): picked mon.noname-e con 0x7f33ec0a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec09e0f0 con 0x7f33ec0a2930 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec09dd80 con 0x7f33ec09faa0 2024-09-15T01:32:13.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0a2d00 con 0x7f33ec0a08c0 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33f0ea8640 10 monclient(hunting): authenticate will time out at 738.137667s 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ead9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 0x7f33ec0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ead9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 0x7f33ec0a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:58040/0 (socket says 172.21.15.178:58040) 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ead9d640 1 -- 172.21.15.178:0/1218667307 learned_addr learned my addr 172.21.15.178:0/1218667307 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ead9d640 10 monclient(hunting): get_auth_request con 0x7f33ec0a08c0 auth_method 0 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ead9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33e9d9b640 1 --2- 172.21.15.178:0/1218667307 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f33ec09faa0 0x7f33ec09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ea59c640 1 --2- 172.21.15.178:0/1218667307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ec0a2930 0x7f33ec132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33e9d9b640 10 monclient(hunting): get_auth_request con 0x7f33ec09faa0 auth_method 0 2024-09-15T01:32:13.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33e9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33e9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33e9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ea59c640 10 monclient(hunting): get_auth_request con 0x7f33ec0a2930 auth_method 0 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ead9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ead9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.387+0000 7f33ead9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33e9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33e9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33e9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ead9d640 10 monclient(hunting): handle_auth_done global_id 4418 payload 274 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ead9d640 1 -- 172.21.15.178:0/1218667307 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f33ec09faa0 msgr2=0x7f33ec09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ead9d640 1 --2- 172.21.15.178:0/1218667307 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f33ec09faa0 0x7f33ec09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ead9d640 1 -- 172.21.15.178:0/1218667307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ec0a2930 msgr2=0x7f33ec132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ead9d640 1 --2- 172.21.15.178:0/1218667307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ec0a2930 0x7f33ec132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ead9d640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ead9d640 1 monclient: found mon.noname-e 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ead9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ead9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ead9d640 1 -- 172.21.15.178:0/1218667307 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec04bf60 con 0x7f33ec0a08c0 2024-09-15T01:32:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ea59c640 1 --2- 172.21.15.178:0/1218667307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ec0a2930 0x7f33ec132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.388+0000 7f33ead9d640 1 --2- 172.21.15.178:0/1218667307 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 0x7f33ec0a0d30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009f50 tx=0x7f33e4009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=65924096b3555f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33e959a640 1 -- 172.21.15.178:0/1218667307 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33e4007880 con 0x7f33ec0a08c0 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33e959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33e959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33e959a640 10 monclient: dump: 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.409 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.409 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.409 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.409 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.409 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.409 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.409 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.409 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33e959a640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33e959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33e959a640 1 -- 172.21.15.178:0/1218667307 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e402d050 con 0x7f33ec0a08c0 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 5 monclient: authenticate success, global_id 4418 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33e959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33e959a640 1 -- 172.21.15.178:0/1218667307 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33e4030360 con 0x7f33ec0a08c0 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 4 set_mon_vals no callback set 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33e959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33e959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33e959a640 10 monclient: dump: 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.410 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 10 monclient: shutdown 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 1 -- 172.21.15.178:0/1218667307 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 msgr2=0x7f33ec0a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 1 --2- 172.21.15.178:0/1218667307 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 0x7f33ec0a0d30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009f50 tx=0x7f33e4009840 comp rx=0 tx=0).stop 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 1 -- 172.21.15.178:0/1218667307 shutdown_connections 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 1 --2- 172.21.15.178:0/1218667307 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 0x7f33ec0a0d30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 1 --2- 172.21.15.178:0/1218667307 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f33ec09faa0 0x7f33ec09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 1 --2- 172.21.15.178:0/1218667307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ec0a2930 0x7f33ec132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 1 -- 172.21.15.178:0/1218667307 >> 172.21.15.178:0/1218667307 conn(0x7f33ec0a5ed0 msgr2=0x7f33ec0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 1 -- 172.21.15.178:0/1218667307 shutdown_connections 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.389+0000 7f33f0ea8640 1 -- 172.21.15.178:0/1218667307 wait complete. 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient: build_initial_monmap 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient: monmap: 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:13.412 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:13.413 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 1 Processor -- start 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 1 -- start start 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient: init 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient: _add_conns ranks=[3,8,1,6,2,7,4,0,5] 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ec09faa0 0x7f33ec09b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient(hunting): picked mon.noname-g con 0x7f33ec09faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 0x7f33ec131b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient(hunting): picked mon.noname-e con 0x7f33ec0a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33ec0a2930 0x7f33ec1320a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient(hunting): picked mon.noname-f con 0x7f33ec0a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec09c100 con 0x7f33ec0a2930 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0a2390 con 0x7f33ec09faa0 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec09d630 con 0x7f33ec0a08c0 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.390+0000 7f33f0ea8640 10 monclient(hunting): authenticate will time out at 738.140968s 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 0x7f33ec131b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 0x7f33ec131b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:58052/0 (socket says 172.21.15.178:58052) 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 1 -- 172.21.15.178:0/856165557 learned_addr learned my addr 172.21.15.178:0/856165557 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 10 monclient(hunting): get_auth_request con 0x7f33ec0a08c0 auth_method 0 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33ead9d640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33ec0a2930 0x7f33ec1320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33ea59c640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ec09faa0 0x7f33ec09b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33ead9d640 10 monclient(hunting): get_auth_request con 0x7f33ec0a2930 auth_method 0 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33ead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33ead9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33ead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33ea59c640 10 monclient(hunting): get_auth_request con 0x7f33ec09faa0 auth_method 0 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33ea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33ea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33ea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 10 monclient(hunting): handle_auth_done global_id 4427 payload 824 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 1 -- 172.21.15.178:0/856165557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ec09faa0 msgr2=0x7f33ec09b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ec09faa0 0x7f33ec09b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 1 -- 172.21.15.178:0/856165557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33ec0a2930 msgr2=0x7f33ec1320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33ec0a2930 0x7f33ec1320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 1 monclient: found mon.noname-e 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 1 -- 172.21.15.178:0/856165557 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc009390 con 0x7f33ec0a08c0 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e9d9b640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 0x7f33ec131b60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f33e000e900 tx=0x7f33e00090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33e0009480 con 0x7f33ec0a08c0 2024-09-15T01:32:13.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33da7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33da7fc640 10 monclient: dump: 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33da7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e000cd60 con 0x7f33ec0a08c0 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33f0ea8640 5 monclient: authenticate success, global_id 4427 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33f0ea8640 10 monclient: _renew_subs 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33f0ea8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33f0ea8640 1 -- 172.21.15.178:0/856165557 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e4009ae0 con 0x7f33ec0a08c0 2024-09-15T01:32:13.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33da7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33e000ced0 con 0x7f33ec0a08c0 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33da7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33da7fc640 10 monclient: dump: 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.420 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33f0ea8640 10 monclient: _renew_subs 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33f0ea8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33f0ea8640 1 -- 172.21.15.178:0/856165557 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ec11ec80 con 0x7f33ec0a08c0 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.391+0000 7f33e8d99640 4 set_mon_vals no callback set 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f33e000c8e0 con 0x7f33ec0a08c0 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33f0ea8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33f0ea8640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec142520 0x7f33ec11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33f0ea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33f0ea8640 1 -- 172.21.15.178:0/856165557 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec04c4a0 con 0x7f33ec142520 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33da7fc640 1 --2- 172.21.15.178:0/856165557 >> v2:172.21.15.204:6812/28210 conn(0x7f33cc0397e0 0x7f33cc03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33ea59c640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec142520 0x7f33ec11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f33e004afd0 con 0x7f33ec0a08c0 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33ea59c640 10 monclient: get_auth_request con 0x7f33ec142520 auth_method 0 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33ea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33ea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33ea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33ea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33ea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33ea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33ead9d640 1 --2- 172.21.15.178:0/856165557 >> v2:172.21.15.204:6812/28210 conn(0x7f33cc0397e0 0x7f33cc03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33ead9d640 10 monclient: get_auth_request con 0x7f33cc0397e0 auth_method 0 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33ea59c640 10 monclient(hunting): handle_auth_done global_id 4405 payload 824 2024-09-15T01:32:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.392+0000 7f33ea59c640 1 -- 172.21.15.178:0/856165557 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33ec05ed70 con 0x7f33ec142520 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33ea59c640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec142520 0x7f33ec11c530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33dc011840 tx=0x7f33dc008040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33dc00fdc0 con 0x7f33ec142520 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33da7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33da7fc640 10 monclient: dump: 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33ead9d640 1 --2- 172.21.15.178:0/856165557 >> v2:172.21.15.204:6812/28210 conn(0x7f33cc0397e0 0x7f33cc03bc90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f33e4033000 tx=0x7f33e4036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f33dc00f940 con 0x7f33ec142520 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33da7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33da7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec142520 msgr2=0x7f33ec11c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33da7fc640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec142520 0x7f33ec11c530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33dc011840 tx=0x7f33dc008040 comp rx=0 tx=0).stop 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec142520 msgr2=0x7f33ec11c530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:13.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.393+0000 7f33da7fc640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec142520 0x7f33ec11c530 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33dc011840 tx=0x7f33dc008040 comp rx=0 tx=0).stop 2024-09-15T01:32:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33f0ea8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33f0ea8640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec11ca70 0x7f33ec11ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33f0ea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33f0ea8640 1 -- 172.21.15.178:0/856165557 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec04bc80 con 0x7f33ec11ca70 2024-09-15T01:32:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33ea59c640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec11ca70 0x7f33ec11ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33ea59c640 10 monclient: get_auth_request con 0x7f33ec11ca70 auth_method 0 2024-09-15T01:32:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33ea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33ea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33ea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33ea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33ea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33ea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33ea59c640 10 monclient(hunting): handle_auth_done global_id 4414 payload 824 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33ea59c640 1 -- 172.21.15.178:0/856165557 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f33ec09fe70 con 0x7f33ec11ca70 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33ea59c640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec11ca70 0x7f33ec11ce70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f33dc011840 tx=0x7f33dc01c000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33dc022070 con 0x7f33ec11ca70 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33da7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33da7fc640 10 monclient: dump: 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f33dc0143e0 con 0x7f33ec11ca70 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33da7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33da7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec11ca70 msgr2=0x7f33ec11ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.399+0000 7f33da7fc640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec11ca70 0x7f33ec11ce70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f33dc011840 tx=0x7f33dc01c000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33da7fc640 1 -- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec11ca70 msgr2=0x7f33ec11ce70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33da7fc640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ec11ca70 0x7f33ec11ce70 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f33dc011840 tx=0x7f33dc01c000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.428 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:13.428 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:13.428 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:13.428 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:13.428 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 12, 2024-09-15T01:32:13.428 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:13.428 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:13.428 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:13.428 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:13.428 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:32:13.428 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 21, 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.429 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.430 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:13.431 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.432 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:13.434 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.437 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.438 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:13.439 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.441 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.442 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.443 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.444 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.445 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.446 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.448 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:13.449 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:13.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33f0ea8640 1 -- 172.21.15.178:0/856165557 >> v2:172.21.15.204:6812/28210 conn(0x7f33cc0397e0 msgr2=0x7f33cc03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33f0ea8640 1 --2- 172.21.15.178:0/856165557 >> v2:172.21.15.204:6812/28210 conn(0x7f33cc0397e0 0x7f33cc03bc90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f33e4033000 tx=0x7f33e4036000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33f0ea8640 10 monclient: shutdown 2024-09-15T01:32:13.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33f0ea8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:13.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33f0ea8640 1 -- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 msgr2=0x7f33ec131b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33f0ea8640 1 --2- 172.21.15.178:0/856165557 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ec0a08c0 0x7f33ec131b60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f33e000e900 tx=0x7f33e00090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33ead9d640 1 -- 172.21.15.178:0/856165557 reap_dead start 2024-09-15T01:32:13.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33ead9d640 1 -- 172.21.15.178:0/856165557 reap_dead start 2024-09-15T01:32:13.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33f0ea8640 1 -- 172.21.15.178:0/856165557 shutdown_connections 2024-09-15T01:32:13.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33f0ea8640 1 -- 172.21.15.178:0/856165557 >> 172.21.15.178:0/856165557 conn(0x7f33ec0a5ed0 msgr2=0x7f33ec0a53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:13.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33f0ea8640 1 -- 172.21.15.178:0/856165557 shutdown_connections 2024-09-15T01:32:13.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.400+0000 7f33f0ea8640 1 -- 172.21.15.178:0/856165557 wait complete. 2024-09-15T01:32:13.451 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:32:13.451 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:32:13.451 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:32:13.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.543+0000 7fccbef5d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:13.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.543+0000 7fccbef5d640 10 monclient: build_initial_monmap 2024-09-15T01:32:13.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.543+0000 7fccbef5d640 10 monclient: monmap: 2024-09-15T01:32:13.545 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:13.545 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.545 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:13.543976+0000 2024-09-15T01:32:13.545 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:13.543976+0000 2024-09-15T01:32:13.545 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:13.547 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.543+0000 7fccbef5d640 1 Processor -- start 2024-09-15T01:32:13.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.543+0000 7fccbef5d640 1 -- start start 2024-09-15T01:32:13.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.543+0000 7fccbef5d640 10 monclient: init 2024-09-15T01:32:13.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 10 monclient: _add_conns ranks=[2,4,9,1,3,0,8,7,6,5,10] 2024-09-15T01:32:13.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb80a2930 0x7fccb8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 10 monclient(hunting): picked mon.noname-b con 0x7fccb80a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:13.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccb809faa0 0x7fccb809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 10 monclient(hunting): picked mon.noname-c con 0x7fccb809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:13.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fccb80a08c0 0x7fccb80a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 10 monclient(hunting): picked mon.noname-a con 0x7fccb80a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:32:13.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb809e0f0 con 0x7fccb80a08c0 2024-09-15T01:32:13.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb809dd80 con 0x7fccb80a2930 2024-09-15T01:32:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccb80a2d00 con 0x7fccb809faa0 2024-09-15T01:32:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbef5d640 10 monclient(hunting): authenticate will time out at 738.294434s 2024-09-15T01:32:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbccf9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb80a2930 0x7fccb8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccaffff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccb809faa0 0x7fccb809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fccb80a08c0 0x7fccb80a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbccf9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb80a2930 0x7fccb8132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:33804/0 (socket says 172.21.15.178:33804) 2024-09-15T01:32:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fccb80a08c0 0x7fccb80a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35122/0 (socket says 172.21.15.178:35122) 2024-09-15T01:32:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 1 -- 172.21.15.178:0/326337927 learned_addr learned my addr 172.21.15.178:0/326337927 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 10 monclient(hunting): get_auth_request con 0x7fccb80a08c0 auth_method 0 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccaffff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccb809faa0 0x7fccb809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:36854/0 (socket says 172.21.15.178:36854) 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccaffff640 10 monclient(hunting): get_auth_request con 0x7fccb809faa0 auth_method 0 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbccf9640 10 monclient(hunting): get_auth_request con 0x7fccb80a2930 auth_method 0 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbccf9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbccf9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbccf9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccaffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccaffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccaffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbccf9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbccf9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbccf9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 10 monclient(hunting): handle_auth_done global_id 4536 payload 274 2024-09-15T01:32:13.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 1 -- 172.21.15.178:0/326337927 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccb809faa0 msgr2=0x7fccb809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 1 --2- 172.21.15.178:0/326337927 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccb809faa0 0x7fccb809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 1 -- 172.21.15.178:0/326337927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb80a2930 msgr2=0x7fccb8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 1 --2- 172.21.15.178:0/326337927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb80a2930 0x7fccb8132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 1 monclient: found mon.noname-a 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbd4fa640 1 -- 172.21.15.178:0/326337927 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb804bf60 con 0x7fccb80a08c0 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccaffff640 1 --2- 172.21.15.178:0/326337927 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccb809faa0 0x7fccb809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.544+0000 7fccbccf9640 1 --2- 172.21.15.178:0/326337927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb80a2930 0x7fccb8132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccbd4fa640 1 --2- 172.21.15.178:0/326337927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb80a08c0 0x7fccb80a0d30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009f20 tx=0x7fccb0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec01b159164192f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccaf7fe640 1 -- 172.21.15.178:0/326337927 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccb0007880 con 0x7fccb80a08c0 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccaf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccaf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccaf7fe640 10 monclient: dump: 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.562 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccaf7fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccaf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccbef5d640 5 monclient: authenticate success, global_id 4536 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccbef5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccaf7fe640 1 -- 172.21.15.178:0/326337927 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb002d050 con 0x7fccb80a08c0 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccaf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccbef5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccbef5d640 4 set_mon_vals no callback set 2024-09-15T01:32:13.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccaf7fe640 1 -- 172.21.15.178:0/326337927 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccb0030360 con 0x7fccb80a08c0 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccaf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccaf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.545+0000 7fccaf7fe640 10 monclient: dump: 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.564 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.565 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.565 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.565 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.565 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.565 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.565 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 10 monclient: shutdown 2024-09-15T01:32:13.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:13.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 1 -- 172.21.15.178:0/326337927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb80a08c0 msgr2=0x7fccb80a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 1 --2- 172.21.15.178:0/326337927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb80a08c0 0x7fccb80a0d30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009f20 tx=0x7fccb0009840 comp rx=0 tx=0).stop 2024-09-15T01:32:13.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 1 -- 172.21.15.178:0/326337927 shutdown_connections 2024-09-15T01:32:13.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 1 --2- 172.21.15.178:0/326337927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb80a08c0 0x7fccb80a0d30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 1 --2- 172.21.15.178:0/326337927 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccb809faa0 0x7fccb809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 1 --2- 172.21.15.178:0/326337927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb80a2930 0x7fccb8132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 1 -- 172.21.15.178:0/326337927 >> 172.21.15.178:0/326337927 conn(0x7fccb80a5ed0 msgr2=0x7fccb80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:13.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 1 -- 172.21.15.178:0/326337927 shutdown_connections 2024-09-15T01:32:13.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 1 -- 172.21.15.178:0/326337927 wait complete. 2024-09-15T01:32:13.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 10 monclient: build_initial_monmap 2024-09-15T01:32:13.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.546+0000 7fccbef5d640 10 monclient: monmap: 2024-09-15T01:32:13.566 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:13.566 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:13.567 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:13.568 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 1 Processor -- start 2024-09-15T01:32:13.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 1 -- start start 2024-09-15T01:32:13.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 10 monclient: init 2024-09-15T01:32:13.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 10 monclient: _add_conns ranks=[0,3,4,1,6,5,7,2,8] 2024-09-15T01:32:13.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb809faa0 0x7fccb809af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 10 monclient(hunting): picked mon.noname-a con 0x7fccb809faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:13.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccb80a08c0 0x7fccb81317a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 10 monclient(hunting): picked mon.noname-g con 0x7fccb80a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:13.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccb80a2930 0x7fccb8131ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 10 monclient(hunting): picked mon.noname-c con 0x7fccb80a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:13.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb811ebc0 con 0x7fccb809faa0 2024-09-15T01:32:13.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb8142a20 con 0x7fccb80a08c0 2024-09-15T01:32:13.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccb809b940 con 0x7fccb80a2930 2024-09-15T01:32:13.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbef5d640 10 monclient(hunting): authenticate will time out at 738.297719s 2024-09-15T01:32:13.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbccf9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb809faa0 0x7fccb809af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbccf9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb809faa0 0x7fccb809af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35130/0 (socket says 172.21.15.178:35130) 2024-09-15T01:32:13.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbccf9640 1 -- 172.21.15.178:0/1080037762 learned_addr learned my addr 172.21.15.178:0/1080037762 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbccf9640 10 monclient(hunting): get_auth_request con 0x7fccb809faa0 auth_method 0 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbd4fa640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccb80a2930 0x7fccb8131ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbccf9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbccf9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbccf9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbd4fa640 10 monclient(hunting): get_auth_request con 0x7fccb80a2930 auth_method 0 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbd4fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbd4fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbd4fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbccf9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbccf9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbccf9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbd4fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbd4fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccbd4fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.547+0000 7fccaffff640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccb80a08c0 0x7fccb81317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccaffff640 10 monclient(hunting): get_auth_request con 0x7fccb80a08c0 auth_method 0 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbccf9640 10 monclient(hunting): handle_auth_done global_id 4545 payload 824 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbccf9640 1 -- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccb80a2930 msgr2=0x7fccb8131ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbccf9640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccb80a2930 0x7fccb8131ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbccf9640 1 -- 172.21.15.178:0/1080037762 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccb80a08c0 msgr2=0x7fccb81317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbccf9640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fccb80a08c0 0x7fccb81317a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbccf9640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbccf9640 1 monclient: found mon.noname-a 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbccf9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbccf9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbccf9640 1 -- 172.21.15.178:0/1080037762 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8009390 con 0x7fccb809faa0 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbd4fa640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccb80a2930 0x7fccb8131ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbccf9640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb809faa0 0x7fccb809af80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fccb400ec80 tx=0x7fccb40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccb400eec0 con 0x7fccb809faa0 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccacff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccacff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccacff9640 10 monclient: dump: 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.572 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.573 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccacff9640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccacff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbef5d640 5 monclient: authenticate success, global_id 4545 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb4013470 con 0x7fccb809faa0 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccacff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccb4013a60 con 0x7fccb809faa0 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbef5d640 10 monclient: _renew_subs 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbef5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbef5d640 1 -- 172.21.15.178:0/1080037762 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb4009390 con 0x7fccb809faa0 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccacff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccacff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccaeffd640 4 set_mon_vals no callback set 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccacff9640 10 monclient: dump: 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.574 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.575 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbef5d640 10 monclient: _renew_subs 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbef5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.548+0000 7fccbef5d640 1 -- 172.21.15.178:0/1080037762 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb0009ae0 con 0x7fccb809faa0 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fccb4013bc0 con 0x7fccb809faa0 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccbef5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccbef5d640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb8137000 0x7fccb811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccbef5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccbef5d640 1 -- 172.21.15.178:0/1080037762 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fccb804c4a0 con 0x7fccb8137000 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccaffff640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb8137000 0x7fccb811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccacff9640 1 --2- 172.21.15.178:0/1080037762 >> v2:172.21.15.204:6812/28210 conn(0x7fcca40397b0 0x7fcca403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccaffff640 10 monclient: get_auth_request con 0x7fccb8137000 auth_method 0 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7fccb404b6b0 con 0x7fccb809faa0 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccaffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccaffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccaffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccaffff640 10 monclient(hunting): handle_auth_done global_id 4470 payload 824 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccbd4fa640 1 --2- 172.21.15.178:0/1080037762 >> v2:172.21.15.204:6812/28210 conn(0x7fcca40397b0 0x7fcca403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccaffff640 1 -- 172.21.15.178:0/1080037762 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccb805ed70 con 0x7fccb8137000 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccbd4fa640 10 monclient: get_auth_request con 0x7fcca40397b0 auth_method 0 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccaffff640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb8137000 0x7fccb811c530 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcca8011760 tx=0x7fcca8008040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.549+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcca800fd30 con 0x7fccb8137000 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.550+0000 7fccacff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.550+0000 7fccacff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.550+0000 7fccacff9640 10 monclient: dump: 2024-09-15T01:32:13.577 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.578 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.579 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.550+0000 7fccbd4fa640 1 --2- 172.21.15.178:0/1080037762 >> v2:172.21.15.204:6812/28210 conn(0x7fcca40397b0 0x7fcca403bc60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fccb0033000 tx=0x7fccb0036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.550+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fcca800fe90 con 0x7fccb8137000 2024-09-15T01:32:13.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.550+0000 7fccacff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:13.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.550+0000 7fccacff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:13.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.550+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb8137000 msgr2=0x7fccb811c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.550+0000 7fccacff9640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb8137000 0x7fccb811c530 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcca8011760 tx=0x7fcca8008040 comp rx=0 tx=0).stop 2024-09-15T01:32:13.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.550+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb8137000 msgr2=0x7fccb811c530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:13.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.550+0000 7fccacff9640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb8137000 0x7fccb811c530 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcca8011760 tx=0x7fcca8008040 comp rx=0 tx=0).stop 2024-09-15T01:32:13.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.555+0000 7fccbef5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:13.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.555+0000 7fccbef5d640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb811ed30 0x7fccb811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.555+0000 7fccbef5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.555+0000 7fccbef5d640 1 -- 172.21.15.178:0/1080037762 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fccb804bc80 con 0x7fccb811ed30 2024-09-15T01:32:13.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccaffff640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb811ed30 0x7fccb811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccaffff640 10 monclient: get_auth_request con 0x7fccb811ed30 auth_method 0 2024-09-15T01:32:13.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccaffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccaffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccaffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccaffff640 10 monclient(hunting): handle_auth_done global_id 4479 payload 824 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccaffff640 1 -- 172.21.15.178:0/1080037762 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fccb804b720 con 0x7fccb811ed30 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccaffff640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb811ed30 0x7fccb811ca70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcca8011760 tx=0x7fcca801c000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcca8022070 con 0x7fccb811ed30 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccacff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccacff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccacff9640 10 monclient: dump: 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.582 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7fcca800f300 con 0x7fccb811ed30 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccacff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccacff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:13.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb811ed30 msgr2=0x7fccb811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccacff9640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb811ed30 0x7fccb811ca70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcca8011760 tx=0x7fcca801c000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccacff9640 1 -- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb811ed30 msgr2=0x7fccb811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:13.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.556+0000 7fccacff9640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccb811ed30 0x7fccb811ca70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.584 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 12, 2024-09-15T01:32:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:13.584 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 21, 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:13.585 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.586 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:13.587 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.588 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:13.589 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:13.593 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.596 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.596 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.596 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:13.596 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:13.596 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.596 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.596 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.597 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:13.597 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:13.597 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.597 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:13.597 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.597 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:13.598 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.600 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:13.601 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:13.605 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.607 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.608 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:13.609 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.609 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.609 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:13.609 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:13.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.557+0000 7fccbef5d640 1 -- 172.21.15.178:0/1080037762 >> v2:172.21.15.204:6812/28210 conn(0x7fcca40397b0 msgr2=0x7fcca403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.557+0000 7fccbef5d640 1 --2- 172.21.15.178:0/1080037762 >> v2:172.21.15.204:6812/28210 conn(0x7fcca40397b0 0x7fcca403bc60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fccb0033000 tx=0x7fccb0036000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.557+0000 7fccbef5d640 10 monclient: shutdown 2024-09-15T01:32:13.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.557+0000 7fccbef5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:13.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.557+0000 7fccbef5d640 1 -- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb809faa0 msgr2=0x7fccb809af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.557+0000 7fccbef5d640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb809faa0 0x7fccb809af80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fccb400ec80 tx=0x7fccb40090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.557+0000 7fccbd4fa640 1 -- 172.21.15.178:0/1080037762 reap_dead start 2024-09-15T01:32:13.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.557+0000 7fccbef5d640 1 -- 172.21.15.178:0/1080037762 shutdown_connections 2024-09-15T01:32:13.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.557+0000 7fccbef5d640 1 --2- 172.21.15.178:0/1080037762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb809faa0 0x7fccb809af80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.557+0000 7fccbef5d640 1 -- 172.21.15.178:0/1080037762 >> 172.21.15.178:0/1080037762 conn(0x7fccb80a5ed0 msgr2=0x7fccb809e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:13.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.557+0000 7fccbef5d640 1 -- 172.21.15.178:0/1080037762 shutdown_connections 2024-09-15T01:32:13.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.557+0000 7fccbef5d640 1 -- 172.21.15.178:0/1080037762 wait complete. 2024-09-15T01:32:13.611 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:32:13.612 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:32:13.612 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:32:13.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.706+0000 7f9e43b1d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:13.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.706+0000 7f9e43b1d640 10 monclient: build_initial_monmap 2024-09-15T01:32:13.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.706+0000 7f9e43b1d640 10 monclient: monmap: 2024-09-15T01:32:13.708 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:13.708 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.708 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:13.707040+0000 2024-09-15T01:32:13.708 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:13.707040+0000 2024-09-15T01:32:13.708 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:13.708 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.708 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:13.708 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:13.708 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:13.708 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:13.709 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:13.709 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:13.709 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:13.709 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:13.709 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:13.709 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:13.709 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:13.709 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.706+0000 7f9e43b1d640 1 Processor -- start 2024-09-15T01:32:13.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.706+0000 7f9e43b1d640 1 -- start start 2024-09-15T01:32:13.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.706+0000 7f9e43b1d640 10 monclient: init 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 10 monclient: _add_conns ranks=[5,3,1,0,6,9,2,10,8,4,7] 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9e3c12f5d0 0x7f9e3c12f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 10 monclient(hunting): picked mon.noname-h con 0x7f9e3c12f5d0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e3c06e510 0x7f9e3c09d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 10 monclient(hunting): picked mon.noname-g con 0x7f9e3c06e510 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9e3c06e8e0 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c04bf60 con 0x7f9e3c06e8e0 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c09f2c0 con 0x7f9e3c06e510 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c06b850 con 0x7f9e3c12f5d0 2024-09-15T01:32:13.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e43b1d640 10 monclient(hunting): authenticate will time out at 738.457437s 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e3c06e510 0x7f9e3c09d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e418b9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9e3c12f5d0 0x7f9e3c12f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e420ba640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f9e3c06e8e0 0x7f9e3c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:36658/0 (socket says 172.21.15.178:36658) 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e420ba640 1 -- 172.21.15.178:0/795699951 learned_addr learned my addr 172.21.15.178:0/795699951 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 10 monclient(hunting): get_auth_request con 0x7f9e3c06e510 auth_method 0 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e418b9640 10 monclient(hunting): get_auth_request con 0x7f9e3c12f5d0 auth_method 0 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e418b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e418b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e418b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e418b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e418b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e418b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 10 monclient(hunting): handle_auth_done global_id 4476 payload 274 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e408b7640 1 -- 172.21.15.178:0/795699951 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165441886 0 0) 0x7f9e3c04bf60 con 0x7f9e3c06e8e0 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 1 -- 172.21.15.178:0/795699951 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9e3c12f5d0 msgr2=0x7f9e3c12f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 1 --2- 172.21.15.178:0/795699951 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9e3c12f5d0 0x7f9e3c12f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 1 -- 172.21.15.178:0/795699951 >> v1:172.21.15.204:6789/0 conn(0x7f9e3c06e8e0 legacy=0x7f9e3c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 1 monclient: found mon.noname-g 2024-09-15T01:32:13.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.707+0000 7f9e410b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e410b8640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:32:13.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e410b8640 1 -- 172.21.15.178:0/795699951 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c06c740 con 0x7f9e3c06e510 2024-09-15T01:32:13.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e418b9640 1 --2- 172.21.15.178:0/795699951 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9e3c12f5d0 0x7f9e3c12f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:13.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:32:13.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e410b8640 1 --2- 172.21.15.178:0/795699951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e3c06e510 0x7f9e3c09d580 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009880 tx=0x7f9e2c0090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=1b7a9ee5b0c1fe53 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 1 -- 172.21.15.178:0/795699951 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e2c02acd0 con 0x7f9e3c06e510 2024-09-15T01:32:13.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:32:13.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 10 monclient: dump: 2024-09-15T01:32:13.722 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.722 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.723 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.724 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 1 -- 172.21.15.178:0/795699951 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c02ae50 con 0x7f9e3c06e510 2024-09-15T01:32:13.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e43b1d640 5 monclient: authenticate success, global_id 4476 2024-09-15T01:32:13.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e43b1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:13.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:13.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e43b1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:13.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 1 -- 172.21.15.178:0/795699951 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e2c031ca0 con 0x7f9e3c06e510 2024-09-15T01:32:13.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e43b1d640 4 set_mon_vals no callback set 2024-09-15T01:32:13.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:32:13.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.708+0000 7f9e408b7640 10 monclient: dump: 2024-09-15T01:32:13.725 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.725 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.725 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.725 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.725 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.725 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.725 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.725 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.725 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.725 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.709+0000 7f9e43b1d640 10 monclient: shutdown 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.709+0000 7f9e43b1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.709+0000 7f9e43b1d640 1 -- 172.21.15.178:0/795699951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e3c06e510 msgr2=0x7f9e3c09d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.709+0000 7f9e43b1d640 1 --2- 172.21.15.178:0/795699951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e3c06e510 0x7f9e3c09d580 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009880 tx=0x7f9e2c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.709+0000 7f9e43b1d640 1 -- 172.21.15.178:0/795699951 shutdown_connections 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.709+0000 7f9e43b1d640 1 --2- 172.21.15.178:0/795699951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e3c06e510 0x7f9e3c09d580 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.709+0000 7f9e43b1d640 1 --2- 172.21.15.178:0/795699951 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9e3c12f5d0 0x7f9e3c12f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.709+0000 7f9e43b1d640 1 -- 172.21.15.178:0/795699951 >> 172.21.15.178:0/795699951 conn(0x7f9e3c068a60 msgr2=0x7f9e3c0a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.709+0000 7f9e43b1d640 1 -- 172.21.15.178:0/795699951 shutdown_connections 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.709+0000 7f9e43b1d640 1 -- 172.21.15.178:0/795699951 wait complete. 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.709+0000 7f9e43b1d640 10 monclient: build_initial_monmap 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 10 monclient: monmap: 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:13.727 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 1 Processor -- start 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 1 -- start start 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 10 monclient: init 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 10 monclient: _add_conns ranks=[2,3,8,4,1,7,6,5,0] 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e3c06e510 0x7f9e3c067130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 10 monclient(hunting): picked mon.noname-b con 0x7f9e3c06e510 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:13.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e3c06e8e0 0x7f9e3c12e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 10 monclient(hunting): picked mon.noname-g con 0x7f9e3c06e8e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c12f5d0 0x7f9e3c12ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 10 monclient(hunting): picked mon.noname-e con 0x7f9e3c12f5d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c067db0 con 0x7f9e3c06e510 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c06bb30 con 0x7f9e3c06e8e0 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c09ccf0 con 0x7f9e3c12f5d0 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e43b1d640 10 monclient(hunting): authenticate will time out at 738.460829s 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e418b9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e3c06e510 0x7f9e3c067130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e420ba640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c12f5d0 0x7f9e3c12ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e418b9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e3c06e510 0x7f9e3c067130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:33818/0 (socket says 172.21.15.178:33818) 2024-09-15T01:32:13.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e420ba640 10 monclient(hunting): get_auth_request con 0x7f9e3c12f5d0 auth_method 0 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e418b9640 1 -- 172.21.15.178:0/2769983177 learned_addr learned my addr 172.21.15.178:0/2769983177 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e420ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e420ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e420ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e418b9640 10 monclient(hunting): get_auth_request con 0x7f9e3c06e510 auth_method 0 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e418b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e418b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.710+0000 7f9e418b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e420ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e420ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e420ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e410b8640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e3c06e8e0 0x7f9e3c12e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e410b8640 10 monclient(hunting): get_auth_request con 0x7f9e3c06e8e0 auth_method 0 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e410b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e410b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e410b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 10 monclient(hunting): handle_auth_done global_id 4448 payload 824 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 1 -- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c12f5d0 msgr2=0x7f9e3c12ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c12f5d0 0x7f9e3c12ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 1 -- 172.21.15.178:0/2769983177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e3c06e8e0 msgr2=0x7f9e3c12e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e3c06e8e0 0x7f9e3c12e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 1 monclient: found mon.noname-b 2024-09-15T01:32:13.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 1 -- 172.21.15.178:0/2769983177 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e34009390 con 0x7f9e3c06e510 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e410b8640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e3c06e8e0 0x7f9e3c12e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e420ba640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c12f5d0 0x7f9e3c12ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e418b9640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e3c06e510 0x7f9e3c067130 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400ea70 tx=0x7f9e340090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e3400ce90 con 0x7f9e3c06e510 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e31ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e31ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e31ffb640 10 monclient: dump: 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.732 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e31ffb640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e31ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e34013470 con 0x7f9e3c06e510 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e43b1d640 5 monclient: authenticate success, global_id 4448 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e43b1d640 10 monclient: _renew_subs 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e43b1d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e43b1d640 1 -- 172.21.15.178:0/2769983177 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2c009390 con 0x7f9e3c06e510 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e31ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e34013a60 con 0x7f9e3c06e510 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e31ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e31ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e31ffb640 10 monclient: dump: 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.734 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e33fff640 4 set_mon_vals no callback set 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e43b1d640 10 monclient: _renew_subs 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e43b1d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.711+0000 7f9e43b1d640 1 -- 172.21.15.178:0/2769983177 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c11e910 con 0x7f9e3c06e510 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9e34013bc0 con 0x7f9e3c06e510 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e43b1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e43b1d640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c11d730 0x7f9e3c13f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e43b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e43b1d640 1 -- 172.21.15.178:0/2769983177 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c11f9a0 con 0x7f9e3c11d730 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e31ffb640 1 --2- 172.21.15.178:0/2769983177 >> v2:172.21.15.204:6812/28210 conn(0x7f9e200397b0 0x7f9e2003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e410b8640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c11d730 0x7f9e3c13f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e410b8640 10 monclient: get_auth_request con 0x7f9e3c11d730 auth_method 0 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f9e3404b250 con 0x7f9e3c06e510 2024-09-15T01:32:13.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e410b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e410b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e410b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e410b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e410b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e410b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e420ba640 1 --2- 172.21.15.178:0/2769983177 >> v2:172.21.15.204:6812/28210 conn(0x7f9e200397b0 0x7f9e2003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e410b8640 10 monclient(hunting): handle_auth_done global_id 4445 payload 824 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e410b8640 1 -- 172.21.15.178:0/2769983177 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e3c04c4a0 con 0x7f9e3c11d730 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.712+0000 7f9e420ba640 10 monclient: get_auth_request con 0x7f9e200397b0 auth_method 0 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e410b8640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c11d730 0x7f9e3c13f920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0a3a90 tx=0x7f9e2c032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e2c02acd0 con 0x7f9e3c11d730 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e31ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e31ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:13.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e31ffb640 10 monclient: dump: 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.738 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.739 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e420ba640 1 --2- 172.21.15.178:0/2769983177 >> v2:172.21.15.204:6812/28210 conn(0x7f9e200397b0 0x7f9e2003bc60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e38001760 tx=0x7f9e38009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f9e2c031a40 con 0x7f9e3c11d730 2024-09-15T01:32:13.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e31ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:13.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e31ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:13.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c11d730 msgr2=0x7f9e3c13f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e31ffb640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c11d730 0x7f9e3c13f920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0a3a90 tx=0x7f9e2c032000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c11d730 msgr2=0x7f9e3c13f920 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:13.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.713+0000 7f9e31ffb640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c11d730 0x7f9e3c13f920 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e43b1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:13.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e43b1d640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c13fe60 0x7f9e3c140260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e43b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e43b1d640 1 -- 172.21.15.178:0/2769983177 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0641f0 con 0x7f9e3c13fe60 2024-09-15T01:32:13.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e410b8640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c13fe60 0x7f9e3c140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e410b8640 10 monclient: get_auth_request con 0x7f9e3c13fe60 auth_method 0 2024-09-15T01:32:13.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e410b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e410b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e410b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e410b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e410b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e410b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e410b8640 10 monclient(hunting): handle_auth_done global_id 4454 payload 824 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e410b8640 1 -- 172.21.15.178:0/2769983177 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9e3c04bc80 con 0x7f9e3c13fe60 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e410b8640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c13fe60 0x7f9e3c140260 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009720 tx=0x7f9e2c03c000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e2c049bc0 con 0x7f9e3c13fe60 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e31ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e31ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.719+0000 7f9e31ffb640 10 monclient: dump: 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.742 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f9e2c049d20 con 0x7f9e3c13fe60 2024-09-15T01:32:13.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e31ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:13.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e31ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:13.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c13fe60 msgr2=0x7f9e3c140260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e31ffb640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c13fe60 0x7f9e3c140260 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009720 tx=0x7f9e2c03c000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e31ffb640 1 -- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c13fe60 msgr2=0x7f9e3c140260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:13.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e31ffb640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e3c13fe60 0x7f9e3c140260 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.744 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:13.744 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:13.744 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:13.744 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:13.744 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 12, 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 22, 2024-09-15T01:32:13.745 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:13.746 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:13.747 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:13.748 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.749 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.750 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:13.751 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.751 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.751 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.751 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.751 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:13.751 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:13.751 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.751 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.751 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.751 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.751 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.751 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:13.752 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:13.752 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.752 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.752 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.752 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.752 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:13.752 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.752 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.752 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.752 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:13.752 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:13.753 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.754 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.754 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.754 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.754 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:13.754 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.754 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.754 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.754 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:13.754 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.754 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.754 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.755 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.755 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:13.755 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:13.755 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.755 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.755 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.755 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.755 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.755 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:13.755 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:13.755 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.755 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.756 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:13.757 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:13.757 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.757 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.757 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.757 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.757 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.757 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:13.757 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:13.757 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.757 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.757 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:13.758 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:13.759 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.759 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.759 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.759 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.759 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:13.759 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:13.759 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.759 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.759 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.759 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.759 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.760 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.761 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.761 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:13.761 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:13.761 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.761 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:13.761 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.762 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.765 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:13.766 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.767 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.767 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:13.767 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:13.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e43b1d640 1 -- 172.21.15.178:0/2769983177 >> v2:172.21.15.204:6812/28210 conn(0x7f9e200397b0 msgr2=0x7f9e2003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e43b1d640 1 --2- 172.21.15.178:0/2769983177 >> v2:172.21.15.204:6812/28210 conn(0x7f9e200397b0 0x7f9e2003bc60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e38001760 tx=0x7f9e38009040 comp rx=0 tx=0).stop 2024-09-15T01:32:13.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e43b1d640 10 monclient: shutdown 2024-09-15T01:32:13.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e43b1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:13.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e43b1d640 1 -- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e3c06e510 msgr2=0x7f9e3c067130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e43b1d640 1 --2- 172.21.15.178:0/2769983177 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e3c06e510 0x7f9e3c067130 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400ea70 tx=0x7f9e340090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e420ba640 1 -- 172.21.15.178:0/2769983177 reap_dead start 2024-09-15T01:32:13.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e420ba640 1 -- 172.21.15.178:0/2769983177 reap_dead start 2024-09-15T01:32:13.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e43b1d640 1 -- 172.21.15.178:0/2769983177 shutdown_connections 2024-09-15T01:32:13.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e43b1d640 1 -- 172.21.15.178:0/2769983177 >> 172.21.15.178:0/2769983177 conn(0x7f9e3c068a60 msgr2=0x7f9e3c09f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:13.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e43b1d640 1 -- 172.21.15.178:0/2769983177 shutdown_connections 2024-09-15T01:32:13.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.720+0000 7f9e43b1d640 1 -- 172.21.15.178:0/2769983177 wait complete. 2024-09-15T01:32:13.769 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:32:13.770 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:32:13.770 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:32:13.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.862+0000 7fb6af4c0640 10 monclient: get_monmap_and_config 2024-09-15T01:32:13.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.862+0000 7fb6af4c0640 10 monclient: build_initial_monmap 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.862+0000 7fb6af4c0640 10 monclient: monmap: 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:13.863782+0000 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:13.863782+0000 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:13.865 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:13.866 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:13.866 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:13.866 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:13.866 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:13.866 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:13.866 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.863+0000 7fb6af4c0640 1 Processor -- start 2024-09-15T01:32:13.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.863+0000 7fb6af4c0640 1 -- start start 2024-09-15T01:32:13.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.863+0000 7fb6af4c0640 10 monclient: init 2024-09-15T01:32:13.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 10 monclient: _add_conns ranks=[5,10,0,1,6,8,2,3,4,7,9] 2024-09-15T01:32:13.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb6a80a2930 0x7fb6a8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 10 monclient(hunting): picked mon.noname-h con 0x7fb6a80a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb6a809faa0 0x7fb6a809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 10 monclient(hunting): picked mon.noname-f con 0x7fb6a809faa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb6a80a08c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a804bf60 con 0x7fb6a80a08c0 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a809e0f0 con 0x7fb6a809faa0 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a809dd80 con 0x7fb6a80a2930 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 10 monclient(hunting): authenticate will time out at 738.614305s 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6ada5d640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fb6a80a08c0 0x7fb6a80a0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:49290/0 (socket says 172.21.15.178:49290) 2024-09-15T01:32:13.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6ada5d640 1 -- 172.21.15.178:0/571893228 learned_addr learned my addr 172.21.15.178:0/571893228 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6ad25c640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb6a80a2930 0x7fb6a8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6aca5b640 1 --2- 172.21.15.178:0/571893228 >> v2:172.21.15.204:3300/0 conn(0x7fb6a809faa0 0x7fb6a809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412476193 0 0) 0x7fb6a804bf60 con 0x7fb6a80a08c0 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 10 monclient(hunting): my global_id is 4554 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6940034e0 con 0x7fb6a80a08c0 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6ad25c640 10 monclient(hunting): get_auth_request con 0x7fb6a80a2930 auth_method 0 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6ad25c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6ad25c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6ad25c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6aca5b640 10 monclient(hunting): get_auth_request con 0x7fb6a809faa0 auth_method 0 2024-09-15T01:32:13.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6aca5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6aca5b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6aca5b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6aca5b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6aca5b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6aca5b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1099245375 0 0) 0x7fb6940034e0 con 0x7fb6a80a08c0 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb6a80a2930 msgr2=0x7fb6a8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb6a80a2930 0x7fb6a8132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 >> v2:172.21.15.204:3300/0 conn(0x7fb6a809faa0 msgr2=0x7fb6a809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 1 --2- 172.21.15.178:0/571893228 >> v2:172.21.15.204:3300/0 conn(0x7fb6a809faa0 0x7fb6a809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a80a2d00 con 0x7fb6a80a08c0 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 5 monclient: authenticate success, global_id 4554 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6af4c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fb6a0001c70 con 0x7fb6a80a08c0 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb6ad25c640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb6a80a2930 0x7fb6a8132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:32:13.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.864+0000 7fb69ffff640 10 monclient: dump: 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.882 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 >> v1:172.21.15.178:6789/0 conn(0x7fb6a80a08c0 legacy=0x7fb6a80a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 10 monclient: _add_conns ranks=[3,8,4,7,5,1,2,0,6] 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb694004220 0x7fb694004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 10 monclient(hunting): picked mon.g con 0x7fb694004220 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb694004d10 0x7fb694005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 10 monclient(hunting): picked mon.e con 0x7fb694004d10 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb694009ab0 0x7fb69400bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 10 monclient(hunting): picked mon.c con 0x7fb694009ab0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a809e0f0 con 0x7fb694004220 2024-09-15T01:32:13.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a809dd80 con 0x7fb694009ab0 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb69400c820 con 0x7fb694004d10 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb69ffff640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6af4c0640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb694009ab0 0x7fb69400bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6aca5b640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb694004d10 0x7fb694005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ad25c640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb694004220 0x7fb694004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6aca5b640 10 monclient(hunting): get_auth_request con 0x7fb694004d10 auth_method 0 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6aca5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6aca5b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6aca5b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 10 monclient(hunting): get_auth_request con 0x7fb694009ab0 auth_method 0 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ad25c640 10 monclient(hunting): get_auth_request con 0x7fb694004220 auth_method 0 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ad25c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ad25c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ad25c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6aca5b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6aca5b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6aca5b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 10 monclient(hunting): handle_auth_done global_id 4554 payload 293 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 1 -- 172.21.15.178:0/571893228 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb694004d10 msgr2=0x7fb694005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb694004d10 0x7fb694005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 1 -- 172.21.15.178:0/571893228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb694004220 msgr2=0x7fb694004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb694004220 0x7fb694004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 1 monclient: found mon.c 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 1 -- 172.21.15.178:0/571893228 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6940034e0 con 0x7fb694009ab0 2024-09-15T01:32:13.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6ada5d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.865+0000 7fb6aca5b640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb694004d10 0x7fb694005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb6ada5d640 1 -- 172.21.15.178:0/571893228 reap_dead start 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb6ada5d640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb694009ab0 0x7fb69400bf10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000a740 tx=0x7fb6a00019f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d52ebae3092799c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6a0033070 con 0x7fb694009ab0 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb69ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb69ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb69ffff640 10 monclient: dump: 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.887 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0007550 con 0x7fb694009ab0 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb6af4c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb69ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:13.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb69ffff640 1 -- 172.21.15.178:0/571893228 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6a002e380 con 0x7fb694009ab0 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb6af4c0640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb6af4c0640 4 set_mon_vals no callback set 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb69ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb69ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb69ffff640 10 monclient: dump: 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.889 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb6af4c0640 10 monclient: shutdown 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb6af4c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb6af4c0640 1 -- 172.21.15.178:0/571893228 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb694009ab0 msgr2=0x7fb69400bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb6af4c0640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb694009ab0 0x7fb69400bf10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000a740 tx=0x7fb6a00019f0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb6af4c0640 1 -- 172.21.15.178:0/571893228 shutdown_connections 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb6af4c0640 1 --2- 172.21.15.178:0/571893228 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb694009ab0 0x7fb69400bf10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.866+0000 7fb6af4c0640 1 -- 172.21.15.178:0/571893228 >> 172.21.15.178:0/571893228 conn(0x7fb6a80a5ed0 msgr2=0x7fb6a80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.867+0000 7fb6af4c0640 1 -- 172.21.15.178:0/571893228 shutdown_connections 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.867+0000 7fb6af4c0640 1 -- 172.21.15.178:0/571893228 wait complete. 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.867+0000 7fb6af4c0640 10 monclient: build_initial_monmap 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.867+0000 7fb6af4c0640 10 monclient: monmap: 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:13.891 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.867+0000 7fb6af4c0640 1 Processor -- start 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.867+0000 7fb6af4c0640 1 -- start start 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.867+0000 7fb6af4c0640 10 monclient: init 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 10 monclient: _add_conns ranks=[2,8,6,5,4,3,1,0,7] 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb694009ab0 0x7fb6a81316b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 10 monclient(hunting): picked mon.noname-b con 0x7fb694009ab0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb6a8131bf0 0x7fb6a811e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 10 monclient(hunting): picked mon.noname-e con 0x7fb6a8131bf0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6a811e6f0 0x7fb6a811eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 10 monclient(hunting): picked mon.noname-d con 0x7fb6a811e6f0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8121800 con 0x7fb694009ab0 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8131fc0 con 0x7fb6a811e6f0 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8121e50 con 0x7fb6a8131bf0 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6af4c0640 10 monclient(hunting): authenticate will time out at 738.618420s 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ad25c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb694009ab0 0x7fb6a81316b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb6a8131bf0 0x7fb6a811e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ad25c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb694009ab0 0x7fb6a81316b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:33834/0 (socket says 172.21.15.178:33834) 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb6a8131bf0 0x7fb6a811e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:58114/0 (socket says 172.21.15.178:58114) 2024-09-15T01:32:13.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ad25c640 1 -- 172.21.15.178:0/2825762517 learned_addr learned my addr 172.21.15.178:0/2825762517 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ad25c640 10 monclient(hunting): get_auth_request con 0x7fb694009ab0 auth_method 0 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ad25c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ad25c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ad25c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 10 monclient(hunting): get_auth_request con 0x7fb6a8131bf0 auth_method 0 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ada5d640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6a811e6f0 0x7fb6a811eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ada5d640 10 monclient(hunting): get_auth_request con 0x7fb6a811e6f0 auth_method 0 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ada5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ada5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ada5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ada5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ada5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ada5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 10 monclient(hunting): handle_auth_done global_id 4463 payload 824 2024-09-15T01:32:13.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 1 -- 172.21.15.178:0/2825762517 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6a811e6f0 msgr2=0x7fb6a811eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6a811e6f0 0x7fb6a811eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 1 -- 172.21.15.178:0/2825762517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb694009ab0 msgr2=0x7fb6a81316b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb694009ab0 0x7fb6a81316b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 10 monclient: _finish_hunting 0 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 1 monclient: found mon.noname-e 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6aca5b640 1 -- 172.21.15.178:0/2825762517 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a809d630 con 0x7fb6a8131bf0 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.868+0000 7fb6ad25c640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb694009ab0 0x7fb6a81316b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb6aca5b640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb6a8131bf0 0x7fb6a811e1b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb698006e70 tx=0x7fb6980090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb69800fc80 con 0x7fb6a8131bf0 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69d7fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69d7fa640 10 monclient: dump: 2024-09-15T01:32:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.897 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69d7fa640 10 monclient: _finish_auth 0 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb6af4c0640 5 monclient: authenticate success, global_id 4463 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69800fde0 con 0x7fb6a8131bf0 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb698014d40 con 0x7fb6a8131bf0 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb6af4c0640 10 monclient: _renew_subs 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb6af4c0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb6af4c0640 1 -- 172.21.15.178:0/2825762517 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a80a2390 con 0x7fb6a8131bf0 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69f7fe640 4 set_mon_vals no callback set 2024-09-15T01:32:13.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb69d7fa640 10 monclient: dump: 2024-09-15T01:32:13.899 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.899 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.899 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.899 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.899 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.899 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.899 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.899 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.899 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.899 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.899 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.900 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.900 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.900 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.900 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.900 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb6af4c0640 10 monclient: _renew_subs 2024-09-15T01:32:13.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb6af4c0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:13.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb6af4c0640 1 -- 172.21.15.178:0/2825762517 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a811fd40 con 0x7fb6a8131bf0 2024-09-15T01:32:13.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.869+0000 7fb6af4c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:13.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6af4c0640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a811f020 0x7fb6a8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6af4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6af4c0640 1 -- 172.21.15.178:0/2825762517 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a81428a0 con 0x7fb6a811f020 2024-09-15T01:32:13.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb698014440 con 0x7fb6a8131bf0 2024-09-15T01:32:13.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ad25c640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a811f020 0x7fb6a8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb69d7fa640 1 --2- 172.21.15.178:0/2825762517 >> v2:172.21.15.204:6812/28210 conn(0x7fb68c0397b0 0x7fb68c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ad25c640 10 monclient: get_auth_request con 0x7fb6a811f020 auth_method 0 2024-09-15T01:32:13.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ad25c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ad25c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ad25c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7fb698018030 con 0x7fb6a8131bf0 2024-09-15T01:32:13.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ada5d640 1 --2- 172.21.15.178:0/2825762517 >> v2:172.21.15.204:6812/28210 conn(0x7fb68c0397b0 0x7fb68c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ada5d640 10 monclient: get_auth_request con 0x7fb68c0397b0 auth_method 0 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_done global_id 4465 payload 824 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ad25c640 1 -- 172.21.15.178:0/2825762517 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6a804c4a0 con 0x7fb6a811f020 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ad25c640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a811f020 0x7fb6a8099a50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4012590 tx=0x7fb6a40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb6ada5d640 1 --2- 172.21.15.178:0/2825762517 >> v2:172.21.15.204:6812/28210 conn(0x7fb68c0397b0 0x7fb68c03bc60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c7f0 tx=0x7fb6a0033000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6a4011c70 con 0x7fb6a811f020 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb69d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb69d7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.870+0000 7fb69d7fa640 10 monclient: dump: 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.871+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fb6a4011dd0 con 0x7fb6a811f020 2024-09-15T01:32:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.871+0000 7fb69d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.871+0000 7fb69d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.871+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a811f020 msgr2=0x7fb6a8099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.871+0000 7fb69d7fa640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a811f020 0x7fb6a8099a50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4012590 tx=0x7fb6a40090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.871+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a811f020 msgr2=0x7fb6a8099a50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.871+0000 7fb69d7fa640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a811f020 0x7fb6a8099a50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.876+0000 7fb6af4c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.876+0000 7fb6af4c0640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a8099f90 0x7fb6a809a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.876+0000 7fb6af4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.876+0000 7fb6af4c0640 1 -- 172.21.15.178:0/2825762517 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a804bc80 con 0x7fb6a8099f90 2024-09-15T01:32:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb6ad25c640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a8099f90 0x7fb6a809a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb6ad25c640 10 monclient: get_auth_request con 0x7fb6a8099f90 auth_method 0 2024-09-15T01:32:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb6ad25c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb6ad25c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb6ad25c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb6ad25c640 10 monclient(hunting): handle_auth_done global_id 4474 payload 824 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb6ad25c640 1 -- 172.21.15.178:0/2825762517 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb6a805ed70 con 0x7fb6a8099f90 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb6ad25c640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a8099f90 0x7fb6a809a390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb6a401f040 tx=0x7fb6a401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6a4006810 con 0x7fb6a8099f90 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb69d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb69d7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.877+0000 7fb69d7fa640 10 monclient: dump: 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:13.908 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7fb6a401d440 con 0x7fb6a8099f90 2024-09-15T01:32:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb69d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb69d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a8099f90 msgr2=0x7fb6a809a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb69d7fa640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a8099f90 0x7fb6a809a390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb6a401f040 tx=0x7fb6a401d000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb69d7fa640 1 -- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a8099f90 msgr2=0x7fb6a809a390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb69d7fa640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6a8099f90 0x7fb6a809a390 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb6a401f040 tx=0x7fb6a401d000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.909 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:13.909 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 12, 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:13.910 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 22, 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.911 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.912 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:13.913 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:13.915 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:13.915 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:13.915 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.915 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:13.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.915 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:13.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.915 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:13.915 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:13.915 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.915 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:13.916 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:13.917 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.918 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.919 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.920 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:13.921 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:13.921 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.921 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.921 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.921 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:13.921 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.921 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.921 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:13.921 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:13.923 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:13.924 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.925 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.926 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:13.927 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.929 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:13.930 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.931 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:13.932 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:13.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:13.933 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:13.933 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:13.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb6af4c0640 1 -- 172.21.15.178:0/2825762517 >> v2:172.21.15.204:6812/28210 conn(0x7fb68c0397b0 msgr2=0x7fb68c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb6af4c0640 1 --2- 172.21.15.178:0/2825762517 >> v2:172.21.15.204:6812/28210 conn(0x7fb68c0397b0 0x7fb68c03bc60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c7f0 tx=0x7fb6a0033000 comp rx=0 tx=0).stop 2024-09-15T01:32:13.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb6af4c0640 10 monclient: shutdown 2024-09-15T01:32:13.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb6af4c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:13.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb6af4c0640 1 -- 172.21.15.178:0/2825762517 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb6a8131bf0 msgr2=0x7fb6a811e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:13.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb6af4c0640 1 --2- 172.21.15.178:0/2825762517 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb6a8131bf0 0x7fb6a811e1b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb698006e70 tx=0x7fb6980090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:13.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb6ada5d640 1 -- 172.21.15.178:0/2825762517 reap_dead start 2024-09-15T01:32:13.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb6ada5d640 1 -- 172.21.15.178:0/2825762517 reap_dead start 2024-09-15T01:32:13.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb6af4c0640 1 -- 172.21.15.178:0/2825762517 shutdown_connections 2024-09-15T01:32:13.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb6af4c0640 1 -- 172.21.15.178:0/2825762517 >> 172.21.15.178:0/2825762517 conn(0x7fb6a80a5ed0 msgr2=0x7fb6a80a47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:13.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb6af4c0640 1 -- 172.21.15.178:0/2825762517 shutdown_connections 2024-09-15T01:32:13.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:13.878+0000 7fb6af4c0640 1 -- 172.21.15.178:0/2825762517 wait complete. 2024-09-15T01:32:13.935 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:32:13.935 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:32:13.935 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:32:14.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.027+0000 7f0ecff9d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:14.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.027+0000 7f0ecff9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:14.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.027+0000 7f0ecff9d640 10 monclient: monmap: 2024-09-15T01:32:14.029 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:14.029 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.029 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:14.028336+0000 2024-09-15T01:32:14.029 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:14.028336+0000 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:14.030 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:14.031 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 1 Processor -- start 2024-09-15T01:32:14.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 1 -- start start 2024-09-15T01:32:14.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 10 monclient: init 2024-09-15T01:32:14.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:14.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 10 monclient: _add_conns ranks=[5,8,1,7,6,10,3,2,4,9,0] 2024-09-15T01:32:14.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0ec80a2930 0x7f0ec8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 10 monclient(hunting): picked mon.noname-h con 0x7f0ec80a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:14.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ec809faa0 0x7f0ec809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 10 monclient(hunting): picked mon.noname-e con 0x7f0ec809faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:14.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0ec80a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:14.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec804bf60 con 0x7f0ec80a08c0 2024-09-15T01:32:14.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec809e0f0 con 0x7f0ec80a2930 2024-09-15T01:32:14.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec809dd80 con 0x7f0ec809faa0 2024-09-15T01:32:14.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:14.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecff9d640 10 monclient(hunting): authenticate will time out at 738.778752s 2024-09-15T01:32:14.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecd538640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ec809faa0 0x7f0ec809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecd538640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ec809faa0 0x7f0ec809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:58120/0 (socket says 172.21.15.178:58120) 2024-09-15T01:32:14.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecd538640 1 -- 172.21.15.178:0/3965000807 learned_addr learned my addr 172.21.15.178:0/3965000807 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:14.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecd538640 10 monclient(hunting): get_auth_request con 0x7f0ec809faa0 auth_method 0 2024-09-15T01:32:14.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecd538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecd538640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.028+0000 7f0ecd538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecdd39640 1 --2- 172.21.15.178:0/3965000807 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0ec80a2930 0x7f0ec8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecdd39640 10 monclient(hunting): get_auth_request con 0x7f0ec80a2930 auth_method 0 2024-09-15T01:32:14.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecdd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecdd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecdd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecdd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecdd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecdd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 10 monclient(hunting): handle_auth_done global_id 4472 payload 274 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 1 -- 172.21.15.178:0/3965000807 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0ec80a2930 msgr2=0x7f0ec8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 1 --2- 172.21.15.178:0/3965000807 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0ec80a2930 0x7f0ec8132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 1 -- 172.21.15.178:0/3965000807 >> v1:172.21.15.204:6789/0 conn(0x7f0ec80a08c0 legacy=0x7f0ec80a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 10 monclient: _finish_hunting 0 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 1 monclient: found mon.noname-e 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 1 -- 172.21.15.178:0/3965000807 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec80a2d00 con 0x7f0ec809faa0 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.029+0000 7f0ecd538640 1 --2- 172.21.15.178:0/3965000807 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ec809faa0 0x7f0ec809fe70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009940 tx=0x7f0ebc0091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=40b4ec8b4958ca26 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0eccd37640 1 -- 172.21.15.178:0/3965000807 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ebc02ad00 con 0x7f0ec809faa0 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0eccd37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0eccd37640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:14.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0eccd37640 10 monclient: dump: 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.044 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.045 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.045 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.045 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0eccd37640 10 monclient: _finish_auth 0 2024-09-15T01:32:14.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0eccd37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:14.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0eccd37640 1 -- 172.21.15.178:0/3965000807 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc02ae80 con 0x7f0ec809faa0 2024-09-15T01:32:14.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 5 monclient: authenticate success, global_id 4472 2024-09-15T01:32:14.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:14.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0eccd37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:14.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0eccd37640 1 -- 172.21.15.178:0/3965000807 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ebc031ca0 con 0x7f0ec809faa0 2024-09-15T01:32:14.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:14.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 4 set_mon_vals no callback set 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0eccd37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0eccd37640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0eccd37640 10 monclient: dump: 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.046 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 10 monclient: shutdown 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 1 -- 172.21.15.178:0/3965000807 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ec809faa0 msgr2=0x7f0ec809fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 1 --2- 172.21.15.178:0/3965000807 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ec809faa0 0x7f0ec809fe70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009940 tx=0x7f0ebc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 1 -- 172.21.15.178:0/3965000807 shutdown_connections 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 1 --2- 172.21.15.178:0/3965000807 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ec809faa0 0x7f0ec809fe70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 1 --2- 172.21.15.178:0/3965000807 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0ec80a2930 0x7f0ec8132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 1 -- 172.21.15.178:0/3965000807 >> 172.21.15.178:0/3965000807 conn(0x7f0ec80a5ed0 msgr2=0x7f0ec80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 1 -- 172.21.15.178:0/3965000807 shutdown_connections 2024-09-15T01:32:14.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.030+0000 7f0ecff9d640 1 -- 172.21.15.178:0/3965000807 wait complete. 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.031+0000 7f0ecff9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.031+0000 7f0ecff9d640 10 monclient: monmap: 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:14.048 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.031+0000 7f0ecff9d640 1 Processor -- start 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.031+0000 7f0ecff9d640 1 -- start start 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.031+0000 7f0ecff9d640 10 monclient: init 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.031+0000 7f0ecff9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.031+0000 7f0ecff9d640 10 monclient: _add_conns ranks=[4,5,1,8,6,0,2,3,7] 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0ec809faa0 0x7f0ec809b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 10 monclient(hunting): picked mon.noname-c con 0x7f0ec809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0ec80a08c0 0x7f0ec8131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 10 monclient(hunting): picked mon.noname-h con 0x7f0ec80a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ec80a2930 0x7f0ec811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 10 monclient(hunting): picked mon.noname-f con 0x7f0ec80a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec809d630 con 0x7f0ec80a2930 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec811ec40 con 0x7f0ec809faa0 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec811e960 con 0x7f0ec80a08c0 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 10 monclient(hunting): authenticate will time out at 738.782081s 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0ec809faa0 0x7f0ec809b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0ec809faa0 0x7f0ec809b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:36888/0 (socket says 172.21.15.178:36888) 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 1 -- 172.21.15.178:0/4171963574 learned_addr learned my addr 172.21.15.178:0/4171963574 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:14.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 10 monclient(hunting): get_auth_request con 0x7f0ec809faa0 auth_method 0 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecd538640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0ec80a08c0 0x7f0ec8131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecd538640 10 monclient(hunting): get_auth_request con 0x7f0ec80a08c0 auth_method 0 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecd538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecd538640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecd538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ece53a640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ec80a2930 0x7f0ec811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ece53a640 10 monclient(hunting): get_auth_request con 0x7f0ec80a2930 auth_method 0 2024-09-15T01:32:14.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ece53a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ece53a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ece53a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecd538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecd538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecd538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ece53a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ece53a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ece53a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 10 monclient(hunting): handle_auth_done global_id 4441 payload 824 2024-09-15T01:32:14.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 1 -- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0ec80a08c0 msgr2=0x7f0ec8131840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0ec80a08c0 0x7f0ec8131840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 1 -- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ec80a2930 msgr2=0x7f0ec811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ec80a2930 0x7f0ec811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 10 monclient: _finish_hunting 0 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 1 monclient: found mon.noname-c 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 1 -- 172.21.15.178:0/4171963574 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc009450 con 0x7f0ec809faa0 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecdd39640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0ec809faa0 0x7f0ec809b030 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000ea50 tx=0x7f0ec00090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ec000ce90 con 0x7f0ec809faa0 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec5ffb640 10 monclient: dump: 2024-09-15T01:32:14.053 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.054 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec0009600 con 0x7f0ec809faa0 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 5 monclient: authenticate success, global_id 4441 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 10 monclient: _renew_subs 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ec00138f0 con 0x7f0ec809faa0 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ecff9d640 1 -- 172.21.15.178:0/4171963574 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec809bbb0 con 0x7f0ec809faa0 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec7fff640 4 set_mon_vals no callback set 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.032+0000 7f0ec5ffb640 10 monclient: dump: 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.056 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ecff9d640 10 monclient: _renew_subs 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ecff9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ecff9d640 1 -- 172.21.15.178:0/4171963574 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec81421e0 con 0x7f0ec809faa0 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0ec0013470 con 0x7f0ec809faa0 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ecff9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ec5ffb640 1 --2- 172.21.15.178:0/4171963574 >> v2:172.21.15.204:6812/28210 conn(0x7f0eb40396e0 0x7f0eb403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ecff9d640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec8131d80 0x7f0ec811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ecff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ecff9d640 1 -- 172.21.15.178:0/4171963574 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec8132150 con 0x7f0ec8131d80 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f0ec004b270 con 0x7f0ec809faa0 2024-09-15T01:32:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ecd538640 1 --2- 172.21.15.178:0/4171963574 >> v2:172.21.15.204:6812/28210 conn(0x7f0eb40396e0 0x7f0eb403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ece53a640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec8131d80 0x7f0ec811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ecd538640 10 monclient: get_auth_request con 0x7f0eb40396e0 auth_method 0 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ece53a640 10 monclient: get_auth_request con 0x7f0ec8131d80 auth_method 0 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ece53a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ece53a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.033+0000 7f0ece53a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ece53a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ece53a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ece53a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ece53a640 10 monclient(hunting): handle_auth_done global_id 4494 payload 824 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ece53a640 1 -- 172.21.15.178:0/4171963574 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ec805ed70 con 0x7f0ec8131d80 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ecd538640 1 --2- 172.21.15.178:0/4171963574 >> v2:172.21.15.204:6812/28210 conn(0x7f0eb40396e0 0x7f0eb403bb90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc034000 tx=0x7f0ebc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ece53a640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec8131d80 0x7f0ec811ca70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ec809b9e0 tx=0x7f0eb8001610 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0eb800c7c0 con 0x7f0ec8131d80 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ec5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ec5ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ec5ffb640 10 monclient: dump: 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.059 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f0eb8001ac0 con 0x7f0ec8131d80 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ec5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ec5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec8131d80 msgr2=0x7f0ec811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ec5ffb640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec8131d80 0x7f0ec811ca70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ec809b9e0 tx=0x7f0eb8001610 comp rx=0 tx=0).stop 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec8131d80 msgr2=0x7f0ec811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:14.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.034+0000 7f0ec5ffb640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec8131d80 0x7f0ec811ca70 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ec809b9e0 tx=0x7f0eb8001610 comp rx=0 tx=0).stop 2024-09-15T01:32:14.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.040+0000 7f0ecff9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:14.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.040+0000 7f0ecff9d640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec811cfb0 0x7f0ec8096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.040+0000 7f0ecff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.040+0000 7f0ecff9d640 1 -- 172.21.15.178:0/4171963574 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec804bc80 con 0x7f0ec811cfb0 2024-09-15T01:32:14.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.040+0000 7f0ece53a640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec811cfb0 0x7f0ec8096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.040+0000 7f0ece53a640 10 monclient: get_auth_request con 0x7f0ec811cfb0 auth_method 0 2024-09-15T01:32:14.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.040+0000 7f0ece53a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.040+0000 7f0ece53a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.040+0000 7f0ece53a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.041+0000 7f0ece53a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.041+0000 7f0ece53a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.041+0000 7f0ece53a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.041+0000 7f0ece53a640 10 monclient(hunting): handle_auth_done global_id 4503 payload 824 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.041+0000 7f0ece53a640 1 -- 172.21.15.178:0/4171963574 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0ec811d3b0 con 0x7f0ec811cfb0 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.041+0000 7f0ece53a640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec811cfb0 0x7f0ec8096a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8018000 tx=0x7f0eb8019000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.041+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0eb8013420 con 0x7f0ec811cfb0 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.041+0000 7f0ec5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.041+0000 7f0ec5ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.041+0000 7f0ec5ffb640 10 monclient: dump: 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.063 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.064 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.065 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.041+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f0eb800c550 con 0x7f0ec811cfb0 2024-09-15T01:32:14.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ec5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:14.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ec5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:14.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec811cfb0 msgr2=0x7f0ec8096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ec5ffb640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec811cfb0 0x7f0ec8096a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8018000 tx=0x7f0eb8019000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ec5ffb640 1 -- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec811cfb0 msgr2=0x7f0ec8096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:14.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ec5ffb640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ec811cfb0 0x7f0ec8096a30 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8018000 tx=0x7f0eb8019000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.065 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:14.065 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 12, 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 22, 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.074 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.075 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.076 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.077 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:14.078 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.081 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.082 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:14.083 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:32:14.086 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.087 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.087 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:14.087 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ecff9d640 1 -- 172.21.15.178:0/4171963574 >> v2:172.21.15.204:6812/28210 conn(0x7f0eb40396e0 msgr2=0x7f0eb403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ecff9d640 1 --2- 172.21.15.178:0/4171963574 >> v2:172.21.15.204:6812/28210 conn(0x7f0eb40396e0 0x7f0eb403bb90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc034000 tx=0x7f0ebc032000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ecff9d640 10 monclient: shutdown 2024-09-15T01:32:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ecff9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ecff9d640 1 -- 172.21.15.178:0/4171963574 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0ec809faa0 msgr2=0x7f0ec809b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ecff9d640 1 --2- 172.21.15.178:0/4171963574 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0ec809faa0 0x7f0ec809b030 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000ea50 tx=0x7f0ec00090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ece53a640 1 -- 172.21.15.178:0/4171963574 reap_dead start 2024-09-15T01:32:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ece53a640 1 -- 172.21.15.178:0/4171963574 reap_dead start 2024-09-15T01:32:14.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ecff9d640 1 -- 172.21.15.178:0/4171963574 shutdown_connections 2024-09-15T01:32:14.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ecff9d640 1 -- 172.21.15.178:0/4171963574 >> 172.21.15.178:0/4171963574 conn(0x7f0ec80a5ed0 msgr2=0x7f0ec809e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:14.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ecff9d640 1 -- 172.21.15.178:0/4171963574 shutdown_connections 2024-09-15T01:32:14.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.042+0000 7f0ecff9d640 1 -- 172.21.15.178:0/4171963574 wait complete. 2024-09-15T01:32:14.089 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:32:14.089 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:32:14.089 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:32:14.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.183+0000 7f40b5aaf640 10 monclient: get_monmap_and_config 2024-09-15T01:32:14.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.183+0000 7f40b5aaf640 10 monclient: build_initial_monmap 2024-09-15T01:32:14.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.183+0000 7f40b5aaf640 10 monclient: monmap: 2024-09-15T01:32:14.185 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:14.185 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.185 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:14.184487+0000 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:14.184487+0000 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:14.187 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 1 Processor -- start 2024-09-15T01:32:14.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 1 -- start start 2024-09-15T01:32:14.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 10 monclient: init 2024-09-15T01:32:14.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:14.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 10 monclient: _add_conns ranks=[10,3,1,8,9,6,2,4,7,5,0] 2024-09-15T01:32:14.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f40b00a2930 0x7f40b0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 10 monclient(hunting): picked mon.noname-f con 0x7f40b00a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40b009faa0 0x7f40b009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 10 monclient(hunting): picked mon.noname-g con 0x7f40b009faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f40b00a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b004bf60 con 0x7f40b00a08c0 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b009e0f0 con 0x7f40b00a2930 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b009dd80 con 0x7f40b009faa0 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.184+0000 7f40b5aaf640 10 monclient(hunting): authenticate will time out at 738.934954s 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f40b00a2930 0x7f40b0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40b009faa0 0x7f40b009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f40b00a2930 0x7f40b0132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:47088/0 (socket says 172.21.15.178:47088) 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 1 -- 172.21.15.178:0/1340461213 learned_addr learned my addr 172.21.15.178:0/1340461213 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 10 monclient(hunting): get_auth_request con 0x7f40b00a2930 auth_method 0 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40aeffd640 10 monclient(hunting): get_auth_request con 0x7f40b009faa0 auth_method 0 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40affff640 1 --1- 172.21.15.178:0/1340461213 >> v1:172.21.15.204:6789/0 conn(0x7f40b00a08c0 0x7f40b00a0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:36682/0 (socket says 172.21.15.178:36682) 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 10 monclient(hunting): handle_auth_done global_id 4501 payload 274 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 1 -- 172.21.15.178:0/1340461213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40b009faa0 msgr2=0x7f40b009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 1 --2- 172.21.15.178:0/1340461213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40b009faa0 0x7f40b009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 1 -- 172.21.15.178:0/1340461213 >> v1:172.21.15.204:6789/0 conn(0x7f40b00a08c0 legacy=0x7f40b00a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 1 monclient: found mon.noname-f 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.185+0000 7f40af7fe640 1 -- 172.21.15.178:0/1340461213 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00a2d00 con 0x7f40b00a2930 2024-09-15T01:32:14.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40af7fe640 1 --2- 172.21.15.178:0/1340461213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40b00a2930 0x7f40b0132de0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009910 tx=0x7f40a40091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb0a6f74680fa239 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40ae7fc640 1 -- 172.21.15.178:0/1340461213 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40a402ad20 con 0x7f40b00a2930 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40ae7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40ae7fc640 10 monclient: dump: 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.200 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40ae7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40ae7fc640 1 -- 172.21.15.178:0/1340461213 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a4031820 con 0x7f40b00a2930 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40ae7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40ae7fc640 1 -- 172.21.15.178:0/1340461213 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40a402aea0 con 0x7f40b00a2930 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40b5aaf640 5 monclient: authenticate success, global_id 4501 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40b5aaf640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:14.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40b5aaf640 4 set_mon_vals no callback set 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40ae7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40ae7fc640 10 monclient: dump: 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.202 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40b5aaf640 10 monclient: shutdown 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40b5aaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40b5aaf640 1 -- 172.21.15.178:0/1340461213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40b00a2930 msgr2=0x7f40b0132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40b5aaf640 1 --2- 172.21.15.178:0/1340461213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40b00a2930 0x7f40b0132de0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009910 tx=0x7f40a40091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40b5aaf640 1 -- 172.21.15.178:0/1340461213 shutdown_connections 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40b5aaf640 1 --2- 172.21.15.178:0/1340461213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40b009faa0 0x7f40b009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40b5aaf640 1 --2- 172.21.15.178:0/1340461213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40b00a2930 0x7f40b0132de0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.186+0000 7f40b5aaf640 1 -- 172.21.15.178:0/1340461213 >> 172.21.15.178:0/1340461213 conn(0x7f40b00a5ed0 msgr2=0x7f40b00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.187+0000 7f40b5aaf640 1 -- 172.21.15.178:0/1340461213 shutdown_connections 2024-09-15T01:32:14.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.187+0000 7f40b5aaf640 1 -- 172.21.15.178:0/1340461213 wait complete. 2024-09-15T01:32:14.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.187+0000 7f40b5aaf640 10 monclient: build_initial_monmap 2024-09-15T01:32:14.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.187+0000 7f40b5aaf640 10 monclient: monmap: 2024-09-15T01:32:14.204 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:14.204 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:14.204 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:14.204 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:14.204 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:14.204 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:14.204 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:14.204 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:14.204 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:14.204 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:14.205 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:14.205 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:14.205 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:14.205 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:14.205 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:14.205 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.187+0000 7f40b5aaf640 1 Processor -- start 2024-09-15T01:32:14.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 1 -- start start 2024-09-15T01:32:14.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 10 monclient: init 2024-09-15T01:32:14.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:14.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 10 monclient: _add_conns ranks=[2,1,3,5,6,4,0,7,8] 2024-09-15T01:32:14.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b009faa0 0x7f40b009b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 10 monclient(hunting): picked mon.noname-b con 0x7f40b009faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40b00a08c0 0x7f40b0131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 10 monclient(hunting): picked mon.noname-f con 0x7f40b00a08c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40b00a2930 0x7f40b0141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 10 monclient(hunting): picked mon.noname-g con 0x7f40b00a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b011f000 con 0x7f40b00a08c0 2024-09-15T01:32:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b011ed20 con 0x7f40b009faa0 2024-09-15T01:32:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b009d610 con 0x7f40b00a2930 2024-09-15T01:32:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40b5aaf640 10 monclient(hunting): authenticate will time out at 738.938486s 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b009faa0 0x7f40b009b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b009faa0 0x7f40b009b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:33850/0 (socket says 172.21.15.178:33850) 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 1 -- 172.21.15.178:0/869151563 learned_addr learned my addr 172.21.15.178:0/869151563 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 10 monclient(hunting): get_auth_request con 0x7f40b009faa0 auth_method 0 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40aeffd640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40b00a08c0 0x7f40b0131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40affff640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40b00a2930 0x7f40b0141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40aeffd640 10 monclient(hunting): get_auth_request con 0x7f40b00a08c0 auth_method 0 2024-09-15T01:32:14.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40affff640 10 monclient(hunting): get_auth_request con 0x7f40b00a2930 auth_method 0 2024-09-15T01:32:14.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 10 monclient(hunting): handle_auth_done global_id 4466 payload 824 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 1 -- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40b00a2930 msgr2=0x7f40b0141e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40b00a2930 0x7f40b0141e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 1 -- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40b00a08c0 msgr2=0x7f40b0131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40b00a08c0 0x7f40b0131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 1 monclient: found mon.noname-b 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.188+0000 7f40af7fe640 1 -- 172.21.15.178:0/869151563 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4009450 con 0x7f40b009faa0 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40af7fe640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b009faa0 0x7f40b009b350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40b009d7e0 tx=0x7f40a4032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40a4033d30 con 0x7f40b009faa0 2024-09-15T01:32:14.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40977fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40977fe640 10 monclient: dump: 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40977fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a403a390 con 0x7f40b009faa0 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40b5aaf640 5 monclient: authenticate success, global_id 4466 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40b5aaf640 10 monclient: _renew_subs 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40b5aaf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40b5aaf640 1 -- 172.21.15.178:0/869151563 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b011ea40 con 0x7f40b009faa0 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40977fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40a403a500 con 0x7f40b009faa0 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40977fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40977fe640 10 monclient: dump: 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40adffb640 4 set_mon_vals no callback set 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40b5aaf640 10 monclient: _renew_subs 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40b5aaf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.189+0000 7f40b5aaf640 1 -- 172.21.15.178:0/869151563 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b011ca00 con 0x7f40b009faa0 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40b5aaf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40b5aaf640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b011dae0 0x7f40b0142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40b5aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40b5aaf640 1 -- 172.21.15.178:0/869151563 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40b0132670 con 0x7f40b011dae0 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f40a4039040 con 0x7f40b009faa0 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40977fe640 1 --2- 172.21.15.178:0/869151563 >> v2:172.21.15.204:6812/28210 conn(0x7f409c0396e0 0x7f409c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40aeffd640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b011dae0 0x7f40b0142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40aeffd640 10 monclient: get_auth_request con 0x7f40b011dae0 auth_method 0 2024-09-15T01:32:14.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f40a406a710 con 0x7f40b009faa0 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40affff640 1 --2- 172.21.15.178:0/869151563 >> v2:172.21.15.204:6812/28210 conn(0x7f409c0396e0 0x7f409c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40affff640 10 monclient: get_auth_request con 0x7f409c0396e0 auth_method 0 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40aeffd640 10 monclient(hunting): handle_auth_done global_id 4487 payload 824 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40aeffd640 1 -- 172.21.15.178:0/869151563 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40b004c4a0 con 0x7f40b011dae0 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40aeffd640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b011dae0 0x7f40b0142360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f40a0011bf0 tx=0x7f40a00090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40affff640 1 --2- 172.21.15.178:0/869151563 >> v2:172.21.15.204:6812/28210 conn(0x7f409c0396e0 0x7f409c03bb90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40a8004fd0 tx=0x7f40a80015e0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.190+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40a000fc80 con 0x7f40b011dae0 2024-09-15T01:32:14.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.191+0000 7f40977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.191+0000 7f40977fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.191+0000 7f40977fe640 10 monclient: dump: 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.218 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.219 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.219 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.219 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.219 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.219 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.191+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f40a000fde0 con 0x7f40b011dae0 2024-09-15T01:32:14.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.191+0000 7f40977fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:14.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.191+0000 7f40977fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:14.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.191+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b011dae0 msgr2=0x7f40b0142360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.191+0000 7f40977fe640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b011dae0 0x7f40b0142360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f40a0011bf0 tx=0x7f40a00090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.191+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b011dae0 msgr2=0x7f40b0142360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:14.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.191+0000 7f40977fe640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b011dae0 0x7f40b0142360 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f40a0011bf0 tx=0x7f40a00090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.196+0000 7f40b5aaf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:14.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.196+0000 7f40b5aaf640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b0131e60 0x7f40b01428a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.196+0000 7f40b5aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.196+0000 7f40b5aaf640 1 -- 172.21.15.178:0/869151563 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40b004bc80 con 0x7f40b0131e60 2024-09-15T01:32:14.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40aeffd640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b0131e60 0x7f40b01428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40aeffd640 10 monclient: get_auth_request con 0x7f40b0131e60 auth_method 0 2024-09-15T01:32:14.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40aeffd640 10 monclient(hunting): handle_auth_done global_id 4496 payload 824 2024-09-15T01:32:14.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40aeffd640 1 -- 172.21.15.178:0/869151563 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f40b005ed70 con 0x7f40b0131e60 2024-09-15T01:32:14.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40aeffd640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b0131e60 0x7f40b01428a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40a0011bf0 tx=0x7f40a001c000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40a0024040 con 0x7f40b0131e60 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.197+0000 7f40977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40977fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40977fe640 10 monclient: dump: 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.223 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.224 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.224 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.224 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.224 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f40a00133a0 con 0x7f40b0131e60 2024-09-15T01:32:14.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40977fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:14.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40977fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:14.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b0131e60 msgr2=0x7f40b01428a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40977fe640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b0131e60 0x7f40b01428a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40a0011bf0 tx=0x7f40a001c000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40977fe640 1 -- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b0131e60 msgr2=0x7f40b01428a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:14.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40977fe640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f40b0131e60 0x7f40b01428a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.224 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 12, 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:32:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 22, 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.232 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.233 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.233 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:14.233 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.233 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.233 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.233 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.233 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:14.233 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:14.233 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.233 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.233 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:14.234 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.236 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.237 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.237 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.237 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.237 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:14.237 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:14.237 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.237 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.237 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.237 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.237 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:14.237 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.237 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.238 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.242 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:14.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.243 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.243 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:14.243 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:14.243 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.243 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.243 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.243 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:14.245 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.247 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:14.247 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40b5aaf640 1 -- 172.21.15.178:0/869151563 >> v2:172.21.15.204:6812/28210 conn(0x7f409c0396e0 msgr2=0x7f409c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40b5aaf640 1 --2- 172.21.15.178:0/869151563 >> v2:172.21.15.204:6812/28210 conn(0x7f409c0396e0 0x7f409c03bb90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40a8004fd0 tx=0x7f40a80015e0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40b5aaf640 10 monclient: shutdown 2024-09-15T01:32:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40b5aaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40b5aaf640 1 -- 172.21.15.178:0/869151563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b009faa0 msgr2=0x7f40b009b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40b5aaf640 1 --2- 172.21.15.178:0/869151563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b009faa0 0x7f40b009b350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40b009d7e0 tx=0x7f40a4032000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40affff640 1 -- 172.21.15.178:0/869151563 reap_dead start 2024-09-15T01:32:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40affff640 1 -- 172.21.15.178:0/869151563 reap_dead start 2024-09-15T01:32:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40b5aaf640 1 -- 172.21.15.178:0/869151563 shutdown_connections 2024-09-15T01:32:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40b5aaf640 1 -- 172.21.15.178:0/869151563 >> 172.21.15.178:0/869151563 conn(0x7f40b00a5ed0 msgr2=0x7f40b009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40b5aaf640 1 -- 172.21.15.178:0/869151563 shutdown_connections 2024-09-15T01:32:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.198+0000 7f40b5aaf640 1 -- 172.21.15.178:0/869151563 wait complete. 2024-09-15T01:32:14.249 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:32:14.249 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:32:14.249 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:32:14.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.341+0000 7fb2c859d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:14.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.341+0000 7fb2c859d640 10 monclient: build_initial_monmap 2024-09-15T01:32:14.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.341+0000 7fb2c859d640 10 monclient: monmap: 2024-09-15T01:32:14.343 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:14.342126+0000 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:14.342126+0000 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:14.344 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:14.345 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:14.345 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:14.345 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.341+0000 7fb2c859d640 1 Processor -- start 2024-09-15T01:32:14.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.341+0000 7fb2c859d640 1 -- start start 2024-09-15T01:32:14.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.341+0000 7fb2c859d640 10 monclient: init 2024-09-15T01:32:14.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:14.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 10 monclient: _add_conns ranks=[8,10,5,7,2,3,4,9,1,0,6] 2024-09-15T01:32:14.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb2c00678d0 0x7fb2c009e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 10 monclient(hunting): picked mon.noname-e con 0x7fb2c00678d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:14.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb2c006ea30 0x7fb2c009ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 10 monclient(hunting): picked mon.noname-f con 0x7fb2c006ea30 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:14.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb2c009f180 0x7fb2c01306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 10 monclient(hunting): picked mon.noname-h con 0x7fb2c009f180 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:14.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c009f5c0 con 0x7fb2c006ea30 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c006d5b0 con 0x7fb2c009f180 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c0067ca0 con 0x7fb2c00678d0 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c859d640 10 monclient(hunting): authenticate will time out at 739.092493s 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb2c00678d0 0x7fb2c009e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb2c00678d0 0x7fb2c009e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:58122/0 (socket says 172.21.15.178:58122) 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 1 -- 172.21.15.178:0/4237212785 learned_addr learned my addr 172.21.15.178:0/4237212785 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 10 monclient(hunting): get_auth_request con 0x7fb2c00678d0 auth_method 0 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6b3a640 1 --2- 172.21.15.178:0/4237212785 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb2c009f180 0x7fb2c01306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c5b38640 1 --2- 172.21.15.178:0/4237212785 >> v2:172.21.15.204:3300/0 conn(0x7fb2c006ea30 0x7fb2c009ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c5b38640 10 monclient(hunting): get_auth_request con 0x7fb2c006ea30 auth_method 0 2024-09-15T01:32:14.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c5b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c5b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c5b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6b3a640 10 monclient(hunting): get_auth_request con 0x7fb2c009f180 auth_method 0 2024-09-15T01:32:14.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 10 monclient(hunting): handle_auth_done global_id 4481 payload 274 2024-09-15T01:32:14.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 1 -- 172.21.15.178:0/4237212785 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb2c009f180 msgr2=0x7fb2c01306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:14.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 1 --2- 172.21.15.178:0/4237212785 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb2c009f180 0x7fb2c01306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 1 -- 172.21.15.178:0/4237212785 >> v2:172.21.15.204:3300/0 conn(0x7fb2c006ea30 msgr2=0x7fb2c009ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:14.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 1 --2- 172.21.15.178:0/4237212785 >> v2:172.21.15.204:3300/0 conn(0x7fb2c006ea30 0x7fb2c009ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 10 monclient: _finish_hunting 0 2024-09-15T01:32:14.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 1 monclient: found mon.noname-e 2024-09-15T01:32:14.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:14.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:14.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c6339640 1 -- 172.21.15.178:0/4237212785 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c004bf60 con 0x7fb2c00678d0 2024-09-15T01:32:14.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.342+0000 7fb2c5b38640 1 --2- 172.21.15.178:0/4237212785 >> v2:172.21.15.204:3300/0 conn(0x7fb2c006ea30 0x7fb2c009ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:14.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c6339640 1 --2- 172.21.15.178:0/4237212785 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb2c00678d0 0x7fb2c009e700 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009940 tx=0x7fb2bc0091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=40dc81f0163ca881 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c5337640 1 -- 172.21.15.178:0/4237212785 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb2bc02ad00 con 0x7fb2c00678d0 2024-09-15T01:32:14.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c5337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c5337640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:14.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c5337640 10 monclient: dump: 2024-09-15T01:32:14.359 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.359 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.359 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.359 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.359 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.359 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c5337640 10 monclient: _finish_auth 0 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c5337640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:14.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c5337640 1 -- 172.21.15.178:0/4237212785 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc02ae80 con 0x7fb2c00678d0 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c859d640 5 monclient: authenticate success, global_id 4481 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c859d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c5337640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c859d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c5337640 1 -- 172.21.15.178:0/4237212785 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb2bc031ca0 con 0x7fb2c00678d0 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c859d640 4 set_mon_vals no callback set 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c5337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c5337640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.343+0000 7fb2c5337640 10 monclient: dump: 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.361 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 10 monclient: shutdown 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 1 -- 172.21.15.178:0/4237212785 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb2c00678d0 msgr2=0x7fb2c009e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 1 --2- 172.21.15.178:0/4237212785 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb2c00678d0 0x7fb2c009e700 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009940 tx=0x7fb2bc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 1 -- 172.21.15.178:0/4237212785 shutdown_connections 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 1 --2- 172.21.15.178:0/4237212785 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb2c009f180 0x7fb2c01306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 1 --2- 172.21.15.178:0/4237212785 >> v2:172.21.15.204:3300/0 conn(0x7fb2c006ea30 0x7fb2c009ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 1 --2- 172.21.15.178:0/4237212785 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb2c00678d0 0x7fb2c009e700 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 1 -- 172.21.15.178:0/4237212785 >> 172.21.15.178:0/4237212785 conn(0x7fb2c00a4390 msgr2=0x7fb2c0067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 1 -- 172.21.15.178:0/4237212785 shutdown_connections 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 1 -- 172.21.15.178:0/4237212785 wait complete. 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 10 monclient: build_initial_monmap 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.344+0000 7fb2c859d640 10 monclient: monmap: 2024-09-15T01:32:14.363 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:14.364 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:14.364 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:14.364 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:14.364 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:14.364 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:14.364 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:14.364 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:14.364 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:14.364 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:14.364 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:14.364 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:14.364 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:14.365 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:14.365 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:14.365 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 1 Processor -- start 2024-09-15T01:32:14.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 1 -- start start 2024-09-15T01:32:14.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 10 monclient: init 2024-09-15T01:32:14.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:14.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 10 monclient: _add_conns ranks=[1,5,4,7,2,8,6,3,0] 2024-09-15T01:32:14.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2c00678d0 0x7fb2c0141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 10 monclient(hunting): picked mon.noname-f con 0x7fb2c00678d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:14.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb2c006ea30 0x7fb2c01337d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 10 monclient(hunting): picked mon.noname-h con 0x7fb2c006ea30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb2c009f180 0x7fb2c0133d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 10 monclient(hunting): picked mon.noname-c con 0x7fb2c009f180 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c01217e0 con 0x7fb2c00678d0 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c006ccd0 con 0x7fb2c009f180 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c006c900 con 0x7fb2c006ea30 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c859d640 10 monclient(hunting): authenticate will time out at 739.095691s 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6b3a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb2c009f180 0x7fb2c0133d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6b3a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb2c009f180 0x7fb2c0133d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:36894/0 (socket says 172.21.15.178:36894) 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6b3a640 1 -- 172.21.15.178:0/1184498709 learned_addr learned my addr 172.21.15.178:0/1184498709 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6b3a640 10 monclient(hunting): get_auth_request con 0x7fb2c009f180 auth_method 0 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6339640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2c00678d0 0x7fb2c0141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6339640 10 monclient(hunting): get_auth_request con 0x7fb2c00678d0 auth_method 0 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c5b38640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb2c006ea30 0x7fb2c01337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.345+0000 7fb2c6339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c5b38640 10 monclient(hunting): get_auth_request con 0x7fb2c006ea30 auth_method 0 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c5b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c5b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c5b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c6b3a640 10 monclient(hunting): handle_auth_done global_id 4450 payload 824 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c6b3a640 1 -- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb2c006ea30 msgr2=0x7fb2c01337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c6b3a640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb2c006ea30 0x7fb2c01337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c6b3a640 1 -- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2c00678d0 msgr2=0x7fb2c0141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c6b3a640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2c00678d0 0x7fb2c0141960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c6b3a640 10 monclient: _finish_hunting 0 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c6b3a640 1 monclient: found mon.noname-c 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c6b3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:14.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c6b3a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:14.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c6b3a640 1 -- 172.21.15.178:0/1184498709 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc009450 con 0x7fb2c009f180 2024-09-15T01:32:14.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c6339640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2c00678d0 0x7fb2c0141960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:14.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c6b3a640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb2c009f180 0x7fb2c0133d10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8007f50 tx=0x7fb2b8009860 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb2b800ee10 con 0x7fb2c009f180 2024-09-15T01:32:14.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2b67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:14.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2b67fc640 10 monclient: dump: 2024-09-15T01:32:14.369 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.369 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.369 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.369 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.370 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.370 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.370 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.370 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.370 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.370 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.370 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.370 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.370 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.370 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.370 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2b67fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b80098f0 con 0x7fb2c009f180 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c859d640 5 monclient: authenticate success, global_id 4450 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2b67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c859d640 10 monclient: _renew_subs 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb2b8014d20 con 0x7fb2c009f180 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c859d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c859d640 1 -- 172.21.15.178:0/1184498709 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c01427b0 con 0x7fb2c009f180 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2b67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2b67fc640 10 monclient: dump: 2024-09-15T01:32:14.371 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.372 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.372 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.372 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.372 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.372 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.372 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.372 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.372 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.372 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.372 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.372 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.372 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.373 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.373 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.373 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c4b36640 4 set_mon_vals no callback set 2024-09-15T01:32:14.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c859d640 10 monclient: _renew_subs 2024-09-15T01:32:14.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c859d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:14.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.346+0000 7fb2c859d640 1 -- 172.21.15.178:0/1184498709 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c0069d90 con 0x7fb2c009f180 2024-09-15T01:32:14.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb2b8014420 con 0x7fb2c009f180 2024-09-15T01:32:14.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c859d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:14.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c859d640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c00682c0 0x7fb2c0068690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c859d640 1 -- 172.21.15.178:0/1184498709 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c0122cb0 con 0x7fb2c00682c0 2024-09-15T01:32:14.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2b67fc640 1 --2- 172.21.15.178:0/1184498709 >> v2:172.21.15.204:6812/28210 conn(0x7fb2a40397b0 0x7fb2a403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7fb2b804b650 con 0x7fb2c009f180 2024-09-15T01:32:14.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c6339640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c00682c0 0x7fb2c0068690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c5b38640 1 --2- 172.21.15.178:0/1184498709 >> v2:172.21.15.204:6812/28210 conn(0x7fb2a40397b0 0x7fb2a403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c5b38640 10 monclient: get_auth_request con 0x7fb2a40397b0 auth_method 0 2024-09-15T01:32:14.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c6339640 10 monclient: get_auth_request con 0x7fb2c00682c0 auth_method 0 2024-09-15T01:32:14.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c6339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c6339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c6339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c6339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c6339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c6339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c6339640 10 monclient(hunting): handle_auth_done global_id 4435 payload 824 2024-09-15T01:32:14.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c5b38640 1 --2- 172.21.15.178:0/1184498709 >> v2:172.21.15.204:6812/28210 conn(0x7fb2a40397b0 0x7fb2a403bc60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0006e70 tx=0x7fb2b0008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.347+0000 7fb2c6339640 1 -- 172.21.15.178:0/1184498709 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2c004bc80 con 0x7fb2c00682c0 2024-09-15T01:32:14.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.348+0000 7fb2c6339640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c00682c0 0x7fb2c0068690 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb2c006e7d0 tx=0x7fb2bc032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.348+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb2bc03caa0 con 0x7fb2c00682c0 2024-09-15T01:32:14.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.348+0000 7fb2b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.348+0000 7fb2b67fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:32:14.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.348+0000 7fb2b67fc640 10 monclient: dump: 2024-09-15T01:32:14.375 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.375 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.376 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.377 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.348+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fb2bc03cc00 con 0x7fb2c00682c0 2024-09-15T01:32:14.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.348+0000 7fb2b67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:14.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.348+0000 7fb2b67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:14.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.348+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c00682c0 msgr2=0x7fb2c0068690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.348+0000 7fb2b67fc640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c00682c0 0x7fb2c0068690 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb2c006e7d0 tx=0x7fb2bc032000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.348+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c00682c0 msgr2=0x7fb2c0068690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:14.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.348+0000 7fb2b67fc640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c00682c0 0x7fb2c0068690 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb2c006e7d0 tx=0x7fb2bc032000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c859d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:14.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c859d640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c0068bd0 0x7fb2c0098a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c859d640 1 -- 172.21.15.178:0/1184498709 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c005ed70 con 0x7fb2c0068bd0 2024-09-15T01:32:14.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c6339640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c0068bd0 0x7fb2c0098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c6339640 10 monclient: get_auth_request con 0x7fb2c0068bd0 auth_method 0 2024-09-15T01:32:14.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c6339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c6339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c6339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c6339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c6339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c6339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c6339640 10 monclient(hunting): handle_auth_done global_id 4444 payload 824 2024-09-15T01:32:14.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.354+0000 7fb2c6339640 1 -- 172.21.15.178:0/1184498709 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb2c004b720 con 0x7fb2c0068bd0 2024-09-15T01:32:14.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2c6339640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c0068bd0 0x7fb2c0098a40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc040000 tx=0x7fb2bc03c000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb2bc050460 con 0x7fb2c0068bd0 2024-09-15T01:32:14.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2b67fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:32:14.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2b67fc640 10 monclient: dump: 2024-09-15T01:32:14.381 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.381 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.381 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.381 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.381 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.381 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.381 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.381 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.381 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.381 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.381 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.382 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.382 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.382 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.382 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.382 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9482 (secure 0 0 0) 0x7fb2bc045a80 con 0x7fb2c0068bd0 2024-09-15T01:32:14.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2b67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:14.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2b67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:14.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c0068bd0 msgr2=0x7fb2c0098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2b67fc640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c0068bd0 0x7fb2c0098a40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc040000 tx=0x7fb2bc03c000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2b67fc640 1 -- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c0068bd0 msgr2=0x7fb2c0098a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:14.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2b67fc640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb2c0068bd0 0x7fb2c0098a40 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc040000 tx=0x7fb2bc03c000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.383 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:14.383 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:14.383 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:14.383 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:14.383 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 12, 2024-09-15T01:32:14.383 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:14.383 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:14.383 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:14.383 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:14.383 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 22, 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:14.384 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:14.385 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:14.386 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:14.387 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:14.388 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.389 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:14.390 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:14.390 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.390 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.390 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.390 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.390 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.390 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:14.390 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:14.390 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.390 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.390 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.391 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.391 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:14.391 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.391 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.391 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:14.391 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.391 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.391 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.391 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.393 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:14.393 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.393 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.393 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:14.393 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.393 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.393 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.393 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:14.393 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.397 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.397 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.397 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:14.397 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:14.397 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.397 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.397 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.397 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.397 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.397 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:14.397 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:14.397 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.398 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:14.399 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.401 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:14.401 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:14.401 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.401 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.401 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.401 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:14.401 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:14.403 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.404 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.405 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:14.406 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2c859d640 1 -- 172.21.15.178:0/1184498709 >> v2:172.21.15.204:6812/28210 conn(0x7fb2a40397b0 msgr2=0x7fb2a403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2c859d640 1 --2- 172.21.15.178:0/1184498709 >> v2:172.21.15.204:6812/28210 conn(0x7fb2a40397b0 0x7fb2a403bc60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0006e70 tx=0x7fb2b0008040 comp rx=0 tx=0).stop 2024-09-15T01:32:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2c859d640 10 monclient: shutdown 2024-09-15T01:32:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2c859d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2c859d640 1 -- 172.21.15.178:0/1184498709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb2c009f180 msgr2=0x7fb2c0133d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2c859d640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb2c009f180 0x7fb2c0133d10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8007f50 tx=0x7fb2b8009860 comp rx=0 tx=0).stop 2024-09-15T01:32:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2c6b3a640 1 -- 172.21.15.178:0/1184498709 reap_dead start 2024-09-15T01:32:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2c859d640 1 -- 172.21.15.178:0/1184498709 shutdown_connections 2024-09-15T01:32:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2c859d640 1 --2- 172.21.15.178:0/1184498709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb2c009f180 0x7fb2c0133d10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.355+0000 7fb2c859d640 1 -- 172.21.15.178:0/1184498709 >> 172.21.15.178:0/1184498709 conn(0x7fb2c00a4390 msgr2=0x7fb2c009e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.356+0000 7fb2c859d640 1 -- 172.21.15.178:0/1184498709 shutdown_connections 2024-09-15T01:32:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.356+0000 7fb2c859d640 1 -- 172.21.15.178:0/1184498709 wait complete. 2024-09-15T01:32:14.413 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:32:14.413 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['d', 'g'] 2024-09-15T01:32:14.413 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-09-15T01:32:14.413 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d 2024-09-15T01:32:14.413 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d store 2024-09-15T01:32:14.413 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d sync_force --yes-i-really-mean-it 2024-09-15T01:32:14.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.505+0000 7f01b319d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:14.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.505+0000 7f01b319d640 10 monclient: build_initial_monmap 2024-09-15T01:32:14.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.505+0000 7f01b319d640 10 monclient: monmap: 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:14.506556+0000 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:14.506556+0000 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:14.508 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:14.509 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 1 Processor -- start 2024-09-15T01:32:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 1 -- start start 2024-09-15T01:32:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 10 monclient: init 2024-09-15T01:32:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 10 monclient: _add_conns ranks=[9,2,8,0,7,6,3,1,5,10,4] 2024-09-15T01:32:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f01ac0a2930 0x7f01ac132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 10 monclient(hunting): picked mon.noname-a con 0x7f01ac0a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:32:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01ac09faa0 0x7f01ac09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 10 monclient(hunting): picked mon.noname-b con 0x7f01ac09faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f01ac0a08c0 0x7f01ac0a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 10 monclient(hunting): picked mon.noname-e con 0x7f01ac0a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac09e0f0 con 0x7f01ac0a2930 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac09dd80 con 0x7f01ac09faa0 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0a2d00 con 0x7f01ac0a08c0 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.506+0000 7f01b319d640 10 monclient(hunting): authenticate will time out at 739.256915s 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b0f39640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f01ac0a2930 0x7f01ac132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f01ac0a08c0 0x7f01ac0a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01abfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01ac09faa0 0x7f01ac09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01abfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01ac09faa0 0x7f01ac09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:33866/0 (socket says 172.21.15.178:33866) 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f01ac0a08c0 0x7f01ac0a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:58126/0 (socket says 172.21.15.178:58126) 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b0f39640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f01ac0a2930 0x7f01ac132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35140/0 (socket says 172.21.15.178:35140) 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01abfff640 1 -- 172.21.15.178:0/215591415 learned_addr learned my addr 172.21.15.178:0/215591415 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01abfff640 10 monclient(hunting): get_auth_request con 0x7f01ac09faa0 auth_method 0 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 10 monclient(hunting): get_auth_request con 0x7f01ac0a08c0 auth_method 0 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b0f39640 10 monclient(hunting): get_auth_request con 0x7f01ac0a2930 auth_method 0 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b0f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b0f39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b0f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b0f39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b0f39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b0f39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 10 monclient(hunting): handle_auth_done global_id 4490 payload 274 2024-09-15T01:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 1 -- 172.21.15.178:0/215591415 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01ac09faa0 msgr2=0x7f01ac09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 1 --2- 172.21.15.178:0/215591415 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01ac09faa0 0x7f01ac09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 1 -- 172.21.15.178:0/215591415 >> v2:172.21.15.178:3300/0 conn(0x7f01ac0a2930 msgr2=0x7f01ac132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 1 --2- 172.21.15.178:0/215591415 >> v2:172.21.15.178:3300/0 conn(0x7f01ac0a2930 0x7f01ac132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 10 monclient: _finish_hunting 0 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 1 monclient: found mon.noname-e 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b173a640 1 -- 172.21.15.178:0/215591415 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac04bf60 con 0x7f01ac0a08c0 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01b0f39640 1 --2- 172.21.15.178:0/215591415 >> v2:172.21.15.178:3300/0 conn(0x7f01ac0a2930 0x7f01ac132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.507+0000 7f01abfff640 1 --2- 172.21.15.178:0/215591415 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01ac09faa0 0x7f01ac09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b173a640 1 --2- 172.21.15.178:0/215591415 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f01ac0a08c0 0x7f01ac0a0d60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009e60 tx=0x7f01a4009780 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9aad5150e6b85e9e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01ab7fe640 1 -- 172.21.15.178:0/215591415 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01a40079d0 con 0x7f01ac0a08c0 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01ab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01ab7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01ab7fe640 10 monclient: dump: 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01ab7fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01ab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01ab7fe640 1 -- 172.21.15.178:0/215591415 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a402d050 con 0x7f01ac0a08c0 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01ab7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01ab7fe640 1 -- 172.21.15.178:0/215591415 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01a4007bd0 con 0x7f01ac0a08c0 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01ab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01ab7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01ab7fe640 10 monclient: dump: 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 5 monclient: authenticate success, global_id 4490 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 4 set_mon_vals no callback set 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 10 monclient: shutdown 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 1 -- 172.21.15.178:0/215591415 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f01ac0a08c0 msgr2=0x7f01ac0a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 1 --2- 172.21.15.178:0/215591415 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f01ac0a08c0 0x7f01ac0a0d60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009e60 tx=0x7f01a4009780 comp rx=0 tx=0).stop 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 1 -- 172.21.15.178:0/215591415 shutdown_connections 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 1 --2- 172.21.15.178:0/215591415 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f01ac0a08c0 0x7f01ac0a0d60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 1 --2- 172.21.15.178:0/215591415 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01ac09faa0 0x7f01ac09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 1 --2- 172.21.15.178:0/215591415 >> v2:172.21.15.178:3300/0 conn(0x7f01ac0a2930 0x7f01ac132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 1 -- 172.21.15.178:0/215591415 >> 172.21.15.178:0/215591415 conn(0x7f01ac0a5ed0 msgr2=0x7f01ac0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 1 -- 172.21.15.178:0/215591415 shutdown_connections 2024-09-15T01:32:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.508+0000 7f01b319d640 1 -- 172.21.15.178:0/215591415 wait complete. 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.509+0000 7f01b319d640 10 monclient: build_initial_monmap 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.509+0000 7f01b319d640 10 monclient: monmap: 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:14.527 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.509+0000 7f01b319d640 1 Processor -- start 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.509+0000 7f01b319d640 1 -- start start 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.509+0000 7f01b319d640 10 monclient: init 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 10 monclient: _add_conns ranks=[4,3,5,7,6,0,8,2,1] 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac09faa0 0x7f01ac09b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 10 monclient(hunting): picked mon.noname-c con 0x7f01ac09faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01ac0a08c0 0x7f01ac1317e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 10 monclient(hunting): picked mon.noname-g con 0x7f01ac0a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:14.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f01ac0a2930 0x7f01ac131d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 10 monclient(hunting): picked mon.noname-h con 0x7f01ac0a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac11ebe0 con 0x7f01ac0a08c0 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac142b30 con 0x7f01ac09faa0 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac09c020 con 0x7f01ac0a2930 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b319d640 10 monclient(hunting): authenticate will time out at 739.260312s 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac09faa0 0x7f01ac09b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac09faa0 0x7f01ac09b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:36904/0 (socket says 172.21.15.178:36904) 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 1 -- 172.21.15.178:0/2228024804 learned_addr learned my addr 172.21.15.178:0/2228024804 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:14.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 10 monclient(hunting): get_auth_request con 0x7f01ac09faa0 auth_method 0 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b173a640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f01ac0a2930 0x7f01ac131d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01abfff640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01ac0a08c0 0x7f01ac1317e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 10 monclient(hunting): handle_auth_done global_id 4459 payload 824 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 1 -- 172.21.15.178:0/2228024804 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f01ac0a2930 msgr2=0x7f01ac131d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f01ac0a2930 0x7f01ac131d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 1 -- 172.21.15.178:0/2228024804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01ac0a08c0 msgr2=0x7f01ac1317e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01ac0a08c0 0x7f01ac1317e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 10 monclient: _finish_hunting 0 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 1 monclient: found mon.noname-c 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 1 -- 172.21.15.178:0/2228024804 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0198009450 con 0x7f01ac09faa0 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01abfff640 10 monclient: get_auth_request con 0x7f01ac0a08c0 auth_method 0 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01abfff640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01ac0a08c0 0x7f01ac1317e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b173a640 10 monclient: get_auth_request con 0x7f01ac0a2930 auth_method 0 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b173a640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f01ac0a2930 0x7f01ac131d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01b0f39640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac09faa0 0x7f01ac09b580 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01a000ec80 tx=0x7f01a00090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01a000eec0 con 0x7f01ac09faa0 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01a8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01a8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.510+0000 7f01a8ff9640 10 monclient: dump: 2024-09-15T01:32:14.531 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01a8ff9640 10 monclient: _finish_auth 0 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01a8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a0013470 con 0x7f01ac09faa0 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01b319d640 5 monclient: authenticate success, global_id 4459 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01a8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01a0013a60 con 0x7f01ac09faa0 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01a8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01a8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01a8ff9640 10 monclient: dump: 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01aaffd640 4 set_mon_vals no callback set 2024-09-15T01:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01b319d640 10 monclient: _renew_subs 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01b319d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01b319d640 1 -- 172.21.15.178:0/2228024804 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a0009390 con 0x7f01ac09faa0 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01b319d640 10 monclient: _renew_subs 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01b319d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01b319d640 1 -- 172.21.15.178:0/2228024804 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a4009a20 con 0x7f01ac09faa0 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f01a0013bc0 con 0x7f01ac09faa0 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01b319d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01b319d640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11c530 0x7f01ac11c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01b319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.511+0000 7f01b319d640 1 -- 172.21.15.178:0/2228024804 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac09f940 con 0x7f01ac11c530 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 1 --2- 172.21.15.178:0/2228024804 >> v2:172.21.15.204:6812/28210 conn(0x7f01900396e0 0x7f019003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01abfff640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11c530 0x7f01ac11c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01abfff640 10 monclient: get_auth_request con 0x7f01ac11c530 auth_method 0 2024-09-15T01:32:14.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f01a004b6e0 con 0x7f01ac09faa0 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01b173a640 1 --2- 172.21.15.178:0/2228024804 >> v2:172.21.15.204:6812/28210 conn(0x7f01900396e0 0x7f019003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01b173a640 10 monclient: get_auth_request con 0x7f01900396e0 auth_method 0 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01abfff640 10 monclient(hunting): handle_auth_done global_id 4497 payload 824 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01abfff640 1 -- 172.21.15.178:0/2228024804 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f01ac04c4a0 con 0x7f01ac11c530 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01abfff640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11c530 0x7f01ac11c900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0a6ec0 tx=0x7f0198008040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f019801c400 con 0x7f01ac11c530 2024-09-15T01:32:14.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 10 monclient: dump: 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.537 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.538 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.538 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.538 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.538 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01b173a640 1 --2- 172.21.15.178:0/2228024804 >> v2:172.21.15.204:6812/28210 conn(0x7f01900396e0 0x7f019003bb90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f01a4034000 tx=0x7f01a4037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f019800ea90 con 0x7f01ac11c530 2024-09-15T01:32:14.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:14.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:14.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11c530 msgr2=0x7f01ac11c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11c530 0x7f01ac11c900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0a6ec0 tx=0x7f0198008040 comp rx=0 tx=0).stop 2024-09-15T01:32:14.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11c530 msgr2=0x7f01ac11c900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:14.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.512+0000 7f01a8ff9640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11c530 0x7f01ac11c900 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0a6ec0 tx=0x7f0198008040 comp rx=0 tx=0).stop 2024-09-15T01:32:14.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01b319d640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:32:14.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01b319d640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11ce40 0x7f01ac096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01b319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01b319d640 1 -- 172.21.15.178:0/2228024804 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac04bc80 con 0x7f01ac11ce40 2024-09-15T01:32:14.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01abfff640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11ce40 0x7f01ac096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01abfff640 10 monclient: get_auth_request con 0x7f01ac11ce40 auth_method 0 2024-09-15T01:32:14.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01abfff640 10 monclient(hunting): handle_auth_done global_id 4506 payload 824 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01abfff640 1 -- 172.21.15.178:0/2228024804 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7f01ac05ed70 con 0x7f01ac11ce40 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01abfff640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11ce40 0x7f01ac096a30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f01980098c0 tx=0x7f019801c000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0198006680 con 0x7f01ac11ce40 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.519+0000 7f01a8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01a8ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01a8ff9640 10 monclient: dump: 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.541 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.542 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.543 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7f01980067e0 con 0x7f01ac11ce40 2024-09-15T01:32:14.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01a8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:32:14.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01a8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:14.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11ce40 msgr2=0x7f01ac096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01a8ff9640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11ce40 0x7f01ac096a30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f01980098c0 tx=0x7f019801c000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01a8ff9640 1 -- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11ce40 msgr2=0x7f01ac096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:14.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01a8ff9640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01ac11ce40 0x7f01ac096a30 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f01980098c0 tx=0x7f019801c000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.543 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: "ret": 0, 2024-09-15T01:32:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-15T01:32:14.543 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01b319d640 1 -- 172.21.15.178:0/2228024804 >> v2:172.21.15.204:6812/28210 conn(0x7f01900396e0 msgr2=0x7f019003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01b319d640 1 --2- 172.21.15.178:0/2228024804 >> v2:172.21.15.204:6812/28210 conn(0x7f01900396e0 0x7f019003bb90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f01a4034000 tx=0x7f01a4037000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01b319d640 10 monclient: shutdown 2024-09-15T01:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01b319d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01b319d640 1 -- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac09faa0 msgr2=0x7f01ac09b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01b319d640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac09faa0 0x7f01ac09b580 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01a000ec80 tx=0x7f01a00090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01b173a640 1 -- 172.21.15.178:0/2228024804 reap_dead start 2024-09-15T01:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01b319d640 1 -- 172.21.15.178:0/2228024804 shutdown_connections 2024-09-15T01:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01b319d640 1 --2- 172.21.15.178:0/2228024804 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac09faa0 0x7f01ac09b580 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01b319d640 1 -- 172.21.15.178:0/2228024804 >> 172.21.15.178:0/2228024804 conn(0x7f01ac0a5ed0 msgr2=0x7f01ac09e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01b319d640 1 -- 172.21.15.178:0/2228024804 shutdown_connections 2024-09-15T01:32:14.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.520+0000 7f01b319d640 1 -- 172.21.15.178:0/2228024804 wait complete. 2024-09-15T01:32:14.546 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-15T01:32:14.546 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-09-15T01:32:14.546 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:32:14.556 INFO:tasks.ceph.mon.d:Stopped 2024-09-15T01:32:14.556 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-15T01:32:14.556 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g store 2024-09-15T01:32:14.556 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g sync_force --yes-i-really-mean-it 2024-09-15T01:32:14.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.689+0000 7fdb54c57640 10 monclient: get_monmap_and_config 2024-09-15T01:32:14.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.689+0000 7fdb54c57640 10 monclient: build_initial_monmap 2024-09-15T01:32:14.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.689+0000 7fdb54c57640 10 monclient: monmap: 2024-09-15T01:32:14.691 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:14.691 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:14.690427+0000 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:14.690427+0000 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:14.692 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:14.693 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:14.693 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:14.693 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 1 Processor -- start 2024-09-15T01:32:14.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 1 -- start start 2024-09-15T01:32:14.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 10 monclient: init 2024-09-15T01:32:14.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:14.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 10 monclient: _add_conns ranks=[1,10,3,6,4,0,8,2,9,7,5] 2024-09-15T01:32:14.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fdb500a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:14.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdb50132de0 0x7fdb50137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 10 monclient(hunting): picked mon.noname-f con 0x7fdb50132de0 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb501380e0 0x7fdb5013a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 10 monclient(hunting): picked mon.noname-g con 0x7fdb501380e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5004bf60 con 0x7fdb500a2930 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb5009dd80 con 0x7fdb50132de0 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb500a2d00 con 0x7fdb501380e0 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.690+0000 7fdb54c57640 10 monclient(hunting): authenticate will time out at 739.440910s 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdb50132de0 0x7fdb50137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdb50132de0 0x7fdb50137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:47120/0 (socket says 172.21.15.178:47120) 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 1 -- 172.21.15.178:0/3106636821 learned_addr learned my addr 172.21.15.178:0/3106636821 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:14.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 10 monclient(hunting): get_auth_request con 0x7fdb50132de0 auth_method 0 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4ed9d640 1 --2- 172.21.15.178:0/3106636821 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb501380e0 0x7fdb5013a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4ed9d640 10 monclient(hunting): get_auth_request con 0x7fdb501380e0 auth_method 0 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_done global_id 4546 payload 274 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 1 -- 172.21.15.178:0/3106636821 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb501380e0 msgr2=0x7fdb5013a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 1 --2- 172.21.15.178:0/3106636821 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb501380e0 0x7fdb5013a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 1 -- 172.21.15.178:0/3106636821 >> v1:172.21.15.204:6789/0 conn(0x7fdb500a2930 legacy=0x7fdb5009faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 10 monclient: _finish_hunting 0 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 1 monclient: found mon.noname-f 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.691+0000 7fdb4dd9b640 1 -- 172.21.15.178:0/3106636821 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb501331b0 con 0x7fdb50132de0 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4dd9b640 1 --2- 172.21.15.178:0/3106636821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb50132de0 0x7fdb50137ba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009910 tx=0x7fdb440091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87aa187b41b45b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4d59a640 1 -- 172.21.15.178:0/3106636821 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb4402ad20 con 0x7fdb50132de0 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4d59a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4d59a640 10 monclient: dump: 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.706 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4d59a640 10 monclient: _finish_auth 0 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4d59a640 1 -- 172.21.15.178:0/3106636821 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb44031820 con 0x7fdb50132de0 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb54c57640 5 monclient: authenticate success, global_id 4546 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb54c57640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb54c57640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb54c57640 4 set_mon_vals no callback set 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4d59a640 1 -- 172.21.15.178:0/3106636821 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb4402aea0 con 0x7fdb50132de0 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4d59a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb4d59a640 10 monclient: dump: 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb54c57640 10 monclient: shutdown 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb54c57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb54c57640 1 -- 172.21.15.178:0/3106636821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb50132de0 msgr2=0x7fdb50137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb54c57640 1 --2- 172.21.15.178:0/3106636821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb50132de0 0x7fdb50137ba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009910 tx=0x7fdb440091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb54c57640 1 -- 172.21.15.178:0/3106636821 shutdown_connections 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb54c57640 1 --2- 172.21.15.178:0/3106636821 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb501380e0 0x7fdb5013a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb54c57640 1 --2- 172.21.15.178:0/3106636821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb50132de0 0x7fdb50137ba0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.692+0000 7fdb54c57640 1 -- 172.21.15.178:0/3106636821 >> 172.21.15.178:0/3106636821 conn(0x7fdb500a5ed0 msgr2=0x7fdb500a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.693+0000 7fdb54c57640 1 -- 172.21.15.178:0/3106636821 shutdown_connections 2024-09-15T01:32:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.693+0000 7fdb54c57640 1 -- 172.21.15.178:0/3106636821 wait complete. 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.693+0000 7fdb54c57640 10 monclient: build_initial_monmap 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.693+0000 7fdb54c57640 10 monclient: monmap: 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.693+0000 7fdb54c57640 1 Processor -- start 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.693+0000 7fdb54c57640 1 -- start start 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.693+0000 7fdb54c57640 10 monclient: init 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 10 monclient: _add_conns ranks=[2,5,6,0,1,3,8,7,4] 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb500a2930 0x7fdb5009ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 10 monclient(hunting): picked mon.noname-b con 0x7fdb500a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdb50132de0 0x7fdb5009b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 10 monclient(hunting): picked mon.noname-h con 0x7fdb50132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:14.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb501380e0 0x7fdb5009b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 10 monclient(hunting): picked mon.noname-d con 0x7fdb501380e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5009c4a0 con 0x7fdb500a2930 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb500a2390 con 0x7fdb50132de0 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5009d630 con 0x7fdb501380e0 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb54c57640 10 monclient(hunting): authenticate will time out at 739.444224s 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4ed9d640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb501380e0 msgr2=0x7fdb5009b5e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4ed9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb501380e0 0x7fdb5009b5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb500a2930 0x7fdb5009ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb500a2930 0x7fdb5009ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:33874/0 (socket says 172.21.15.178:33874) 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 1 -- 172.21.15.178:0/2798410869 learned_addr learned my addr 172.21.15.178:0/2798410869 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 10 monclient(hunting): get_auth_request con 0x7fdb500a2930 auth_method 0 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4dd9b640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdb50132de0 0x7fdb5009b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4dd9b640 10 monclient(hunting): get_auth_request con 0x7fdb50132de0 auth_method 0 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 10 monclient(hunting): handle_auth_done global_id 4484 payload 824 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 1 -- 172.21.15.178:0/2798410869 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb501380e0 msgr2=0x7fdb5009b5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb501380e0 0x7fdb5009b5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 1 -- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdb50132de0 msgr2=0x7fdb5009b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdb50132de0 0x7fdb5009b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 1 monclient: found mon.noname-b 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 1 -- 172.21.15.178:0/2798410869 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb44009450 con 0x7fdb500a2930 2024-09-15T01:32:14.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb4e59c640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb500a2930 0x7fdb5009ab60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000c4f0 tx=0x7fdb40007ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb40001080 con 0x7fdb500a2930 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb3e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb3e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb3e7fc640 10 monclient: dump: 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.715 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb3e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.694+0000 7fdb3e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb54c57640 5 monclient: authenticate success, global_id 4484 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb54c57640 10 monclient: _renew_subs 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb40011a10 con 0x7fdb500a2930 2024-09-15T01:32:14.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb54c57640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:14.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb54c57640 1 -- 172.21.15.178:0/2798410869 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5011d7d0 con 0x7fdb500a2930 2024-09-15T01:32:14.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb3e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:14.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb400102e0 con 0x7fdb500a2930 2024-09-15T01:32:14.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb3e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb3e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:14.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb3e7fc640 10 monclient: dump: 2024-09-15T01:32:14.720 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.720 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.720 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.720 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.720 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.720 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.720 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.720 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb4cd99640 4 set_mon_vals no callback set 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb54c57640 10 monclient: _renew_subs 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb54c57640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb54c57640 1 -- 172.21.15.178:0/2798410869 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5011c870 con 0x7fdb500a2930 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb54c57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:14.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdb40011b80 con 0x7fdb500a2930 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb54c57640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb5009bb20 0x7fdb50096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb54c57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.695+0000 7fdb54c57640 1 -- 172.21.15.178:0/2798410869 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb501329c0 con 0x7fdb5009bb20 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb3e7fc640 1 --2- 172.21.15.178:0/2798410869 >> v2:172.21.15.204:6812/28210 conn(0x7fdb300397b0 0x7fdb3003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4dd9b640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb5009bb20 0x7fdb50096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4dd9b640 10 monclient: get_auth_request con 0x7fdb5009bb20 auth_method 0 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7fdb40049f90 con 0x7fdb500a2930 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4ed9d640 1 --2- 172.21.15.178:0/2798410869 >> v2:172.21.15.204:6812/28210 conn(0x7fdb300397b0 0x7fdb3003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4ed9d640 10 monclient: get_auth_request con 0x7fdb300397b0 auth_method 0 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_done global_id 4539 payload 824 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4dd9b640 1 -- 172.21.15.178:0/2798410869 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb5004c4a0 con 0x7fdb5009bb20 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4dd9b640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb5009bb20 0x7fdb50096a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdb500a2f70 tx=0x7fdb44032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb4ed9d640 1 --2- 172.21.15.178:0/2798410869 >> v2:172.21.15.204:6812/28210 conn(0x7fdb300397b0 0x7fdb3003bc60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009fd0 tx=0x7fdb4800b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb44032d30 con 0x7fdb5009bb20 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb3e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb3e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.696+0000 7fdb3e7fc640 10 monclient: dump: 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.723 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.697+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fdb44032e90 con 0x7fdb5009bb20 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.697+0000 7fdb3e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:14.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.697+0000 7fdb3e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:14.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.697+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb5009bb20 msgr2=0x7fdb50096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.697+0000 7fdb3e7fc640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb5009bb20 0x7fdb50096a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdb500a2f70 tx=0x7fdb44032000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.697+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb5009bb20 msgr2=0x7fdb50096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:14.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.697+0000 7fdb3e7fc640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb5009bb20 0x7fdb50096a30 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdb500a2f70 tx=0x7fdb44032000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.703+0000 7fdb54c57640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:32:14.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.703+0000 7fdb54c57640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb50096f70 0x7fdb50097340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.703+0000 7fdb54c57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.703+0000 7fdb54c57640 1 -- 172.21.15.178:0/2798410869 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5004bc80 con 0x7fdb50096f70 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.703+0000 7fdb4dd9b640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb50096f70 0x7fdb50097340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.703+0000 7fdb4dd9b640 10 monclient: get_auth_request con 0x7fdb50096f70 auth_method 0 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.703+0000 7fdb4dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.703+0000 7fdb4dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.703+0000 7fdb4dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.703+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.703+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.703+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.704+0000 7fdb4dd9b640 10 monclient(hunting): handle_auth_done global_id 4548 payload 824 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.704+0000 7fdb4dd9b640 1 -- 172.21.15.178:0/2798410869 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7fdb5005ed70 con 0x7fdb50096f70 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.704+0000 7fdb4dd9b640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb50096f70 0x7fdb50097340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009910 tx=0x7fdb4403d000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.704+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb4403c860 con 0x7fdb50096f70 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.704+0000 7fdb3e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.704+0000 7fdb3e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.704+0000 7fdb3e7fc640 10 monclient: dump: 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.704+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7fdb4403c3e0 con 0x7fdb50096f70 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.704+0000 7fdb3e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb3e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb50096f70 msgr2=0x7fdb50097340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb3e7fc640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb50096f70 0x7fdb50097340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009910 tx=0x7fdb4403d000 comp rx=0 tx=0).stop 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb3e7fc640 1 -- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb50096f70 msgr2=0x7fdb50097340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb3e7fc640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb50096f70 0x7fdb50097340 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.730 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:14.730 INFO:teuthology.orchestra.run.smithi178.stdout: "ret": 0, 2024-09-15T01:32:14.730 INFO:teuthology.orchestra.run.smithi178.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-15T01:32:14.730 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:14.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb54c57640 1 -- 172.21.15.178:0/2798410869 >> v2:172.21.15.204:6812/28210 conn(0x7fdb300397b0 msgr2=0x7fdb3003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb54c57640 1 --2- 172.21.15.178:0/2798410869 >> v2:172.21.15.204:6812/28210 conn(0x7fdb300397b0 0x7fdb3003bc60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009fd0 tx=0x7fdb4800b040 comp rx=0 tx=0).stop 2024-09-15T01:32:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb54c57640 10 monclient: shutdown 2024-09-15T01:32:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb54c57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb54c57640 1 -- 172.21.15.178:0/2798410869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb500a2930 msgr2=0x7fdb5009ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb54c57640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb500a2930 0x7fdb5009ab60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000c4f0 tx=0x7fdb40007ef0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb4ed9d640 1 -- 172.21.15.178:0/2798410869 reap_dead start 2024-09-15T01:32:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb54c57640 1 -- 172.21.15.178:0/2798410869 shutdown_connections 2024-09-15T01:32:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb54c57640 1 --2- 172.21.15.178:0/2798410869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb500a2930 0x7fdb5009ab60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb54c57640 1 -- 172.21.15.178:0/2798410869 >> 172.21.15.178:0/2798410869 conn(0x7fdb500a5ed0 msgr2=0x7fdb500a47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb54c57640 1 -- 172.21.15.178:0/2798410869 shutdown_connections 2024-09-15T01:32:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:14.705+0000 7fdb54c57640 1 -- 172.21.15.178:0/2798410869 wait complete. 2024-09-15T01:32:14.733 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T01:32:14.733 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-15T01:32:14.733 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:32:14.745 INFO:tasks.ceph.mon.g:Stopped 2024-09-15T01:32:14.745 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-15T01:32:14.745 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-15T01:32:14.746 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T01:32:29.746 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-15T01:32:29.747 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-15T01:32:29.747 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-15T01:32:29.747 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:32:29.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.849+0000 7f8ddc889640 10 monclient: get_monmap_and_config 2024-09-15T01:32:29.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.849+0000 7f8ddc889640 10 monclient: build_initial_monmap 2024-09-15T01:32:29.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.849+0000 7f8ddc889640 10 monclient: monmap: 2024-09-15T01:32:29.852 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:29.852 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:29.852 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:29.850565+0000 2024-09-15T01:32:29.852 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:29.850565+0000 2024-09-15T01:32:29.852 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:29.852 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:29.852 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.850+0000 7f8ddc889640 1 Processor -- start 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.850+0000 7f8ddc889640 1 -- start start 2024-09-15T01:32:29.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.850+0000 7f8ddc889640 10 monclient: init 2024-09-15T01:32:29.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.850+0000 7f8ddc889640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:29.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.850+0000 7f8ddc889640 10 monclient: _add_conns ranks=[5,10,3,6,2,0,1,7,4,8,9] 2024-09-15T01:32:29.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.850+0000 7f8ddc889640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:29.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.850+0000 7f8ddc889640 10 monclient(hunting): picked mon.noname-h con 0x7f8dd80a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:29.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.850+0000 7f8ddc889640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:29.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.850+0000 7f8ddc889640 10 monclient(hunting): picked mon.noname-f con 0x7f8dd809faa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:29.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.850+0000 7f8ddc889640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd80a08c0 0x7f8dd80a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:29.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8ddc889640 10 monclient(hunting): picked mon.noname-g con 0x7f8dd80a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:29.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8ddc889640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:29.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8ddc889640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd809e0f0 con 0x7f8dd809faa0 2024-09-15T01:32:29.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8ddc889640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:29.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8ddc889640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd809dd80 con 0x7f8dd80a08c0 2024-09-15T01:32:29.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8ddc889640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:29.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8ddc889640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd80a2d00 con 0x7f8dd80a2930 2024-09-15T01:32:29.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8ddc889640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:29.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8ddc889640 10 monclient(hunting): authenticate will time out at 754.601028s 2024-09-15T01:32:29.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd6d9d640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd80a08c0 msgr2=0x7f8dd80a0d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:32:29.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd6d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd80a08c0 0x7f8dd80a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:29.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd5d9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:29.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd659c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:29.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd659c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:58446/0 (socket says 172.21.15.178:58446) 2024-09-15T01:32:29.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd5d9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:51120/0 (socket says 172.21.15.178:51120) 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 learned_addr learned my addr 172.21.15.178:0/1256812943 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd659c640 10 monclient(hunting): get_auth_request con 0x7f8dd80a2930 auth_method 0 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request con 0x7f8dd809faa0 auth_method 0 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 msgr2=0x7f8dd8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 msgr2=0x7f8dd8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:29.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 msgr2=0x7f8dd809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:32:29.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:29.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 msgr2=0x7f8dd809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:29.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:29.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:29.851+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd6d9d640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd80a08c0 msgr2=0x7f8dd80a0d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd6d9d640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd80a08c0 0x7f8dd80a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd659c640 10 monclient(hunting): get_auth_request con 0x7f8dd80a2930 auth_method 0 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request con 0x7f8dd809faa0 auth_method 0 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 msgr2=0x7f8dd8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 msgr2=0x7f8dd8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 msgr2=0x7f8dd809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:32:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 msgr2=0x7f8dd809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:30.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.052+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:32:30.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd6d9d640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd80a08c0 msgr2=0x7f8dd80a0d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:32:30.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd6d9d640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd80a08c0 0x7f8dd80a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:32:30.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd659c640 10 monclient(hunting): get_auth_request con 0x7f8dd80a2930 auth_method 0 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request con 0x7f8dd809faa0 auth_method 0 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 msgr2=0x7f8dd8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 msgr2=0x7f8dd8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:30.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 msgr2=0x7f8dd809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:32:30.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 msgr2=0x7f8dd809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:30.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:30.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:32:30.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:30.453+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:32:31.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd6d9d640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd80a08c0 msgr2=0x7f8dd80a0d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:32:31.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd6d9d640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd80a08c0 0x7f8dd80a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:32:31.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:31.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:31.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd659c640 10 monclient(hunting): get_auth_request con 0x7f8dd80a2930 auth_method 0 2024-09-15T01:32:31.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:31.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:31.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:31.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request con 0x7f8dd809faa0 auth_method 0 2024-09-15T01:32:31.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:31.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:31.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:31.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 msgr2=0x7f8dd8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:32:31.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 msgr2=0x7f8dd8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:31.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:31.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 msgr2=0x7f8dd809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:32:31.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:32:31.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 msgr2=0x7f8dd809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:31.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:31.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:31.254+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:32:32.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 10 monclient(hunting): tick 2024-09-15T01:32:32.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 monclient(hunting): continuing hunt 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 msgr2=0x7f8dd8132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8dd80a2930 0x7f8dd8132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd80a08c0 msgr2=0x7f8dd80a0d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8dd80a08c0 0x7f8dd80a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 msgr2=0x7f8dd809fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.204:3300/0 conn(0x7f8dd809faa0 0x7f8dd809fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 10 monclient: _add_conns ranks=[4,9,6,0,1,7,2,8] 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 10 monclient(hunting): picked mon.noname-c con 0x7f8db80037d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 10 monclient(hunting): picked mon.noname-a con 0x7f8db80061f0 addr v2:172.21.15.178:3300/0 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8db8008cb0 0x7f8db800b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 10 monclient(hunting): picked mon.noname-d con 0x7f8db8008cb0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 -- 172.21.15.178:0/1256812943 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd809e0f0 con 0x7f8db80061f0 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 -- 172.21.15.178:0/1256812943 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd809dd80 con 0x7f8db80037d0 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 1 -- 172.21.15.178:0/1256812943 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd80a2d00 con 0x7f8db8008cb0 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dc7fff640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd6d9d640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8db8008cb0 msgr2=0x7f8db800b160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd6d9d640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8db8008cb0 0x7f8db800b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request con 0x7f8db80061f0 auth_method 0 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd659c640 10 monclient(hunting): get_auth_request con 0x7f8db80037d0 auth_method 0 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 msgr2=0x7f8db8008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 msgr2=0x7f8db8008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 msgr2=0x7f8db8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 msgr2=0x7f8db8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:32.851+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:33.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd6d9d640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8db8008cb0 msgr2=0x7f8db800b160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:32:33.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd6d9d640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8db8008cb0 0x7f8db800b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:32:33.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:33.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:33.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request con 0x7f8db80061f0 auth_method 0 2024-09-15T01:32:33.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:33.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:33.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:33.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd659c640 10 monclient(hunting): get_auth_request con 0x7f8db80037d0 auth_method 0 2024-09-15T01:32:33.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:33.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:33.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:33.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 msgr2=0x7f8db8008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:32:33.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 msgr2=0x7f8db8008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:33.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:33.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 msgr2=0x7f8db8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:32:33.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 msgr2=0x7f8db8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:33.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:33.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:32:33.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.052+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:32:33.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd6d9d640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8db8008cb0 msgr2=0x7f8db800b160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:32:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd6d9d640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8db8008cb0 0x7f8db800b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:32:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request con 0x7f8db80061f0 auth_method 0 2024-09-15T01:32:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd659c640 10 monclient(hunting): get_auth_request con 0x7f8db80037d0 auth_method 0 2024-09-15T01:32:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 msgr2=0x7f8db8008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:32:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 msgr2=0x7f8db8008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 msgr2=0x7f8db8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:32:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 msgr2=0x7f8db8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:32:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:33.453+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:32:34.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd6d9d640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8db8008cb0 msgr2=0x7f8db800b160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:32:34.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd6d9d640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8db8008cb0 0x7f8db800b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:32:34.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:34.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd659c640 10 monclient(hunting): get_auth_request con 0x7f8db80037d0 auth_method 0 2024-09-15T01:32:34.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:34.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:34.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:34.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:34.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request con 0x7f8db80061f0 auth_method 0 2024-09-15T01:32:34.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:34.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:34.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:34.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 msgr2=0x7f8db8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:32:34.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 msgr2=0x7f8db8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:34.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:34.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:32:34.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 msgr2=0x7f8db8008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:32:34.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 msgr2=0x7f8db8008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:34.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:34.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:34.254+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:32:35.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 10 monclient(hunting): tick 2024-09-15T01:32:35.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 1 monclient(hunting): continuing hunt 2024-09-15T01:32:35.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8db8008cb0 msgr2=0x7f8db800b160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8db8008cb0 0x7f8db800b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 msgr2=0x7f8db8005bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8db80037d0 0x7f8db8005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 1 -- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 msgr2=0x7f8db8008650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 1 --2- 172.21.15.178:0/1256812943 >> v2:172.21.15.178:3300/0 conn(0x7f8db80061f0 0x7f8db8008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 10 monclient: _add_conns ranks=[8,1,2,0,7] 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd6d9d640 1 -- 172.21.15.178:0/1256812943 reap_dead start 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8db800bdb0 0x7f8db800e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 10 monclient(hunting): picked mon.noname-e con 0x7f8db800bdb0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8db8006640 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:35.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db8005300 0x7f8db800b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 10 monclient(hunting): picked mon.noname-b con 0x7f8db8005300 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 1 -- 172.21.15.178:0/1256812943 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd804bf60 con 0x7f8db8006640 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 1 -- 172.21.15.178:0/1256812943 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd809e0f0 con 0x7f8db8005300 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 1 -- 172.21.15.178:0/1256812943 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd809dd80 con 0x7f8db800bdb0 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dc7fff640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8db800bdb0 0x7f8db800e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 10 monclient(hunting): get_auth_request con 0x7f8db800bdb0 auth_method 0 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:35.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd6d9d640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db8005300 0x7f8db800b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:35.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd6d9d640 10 monclient(hunting): get_auth_request con 0x7f8db8005300 auth_method 0 2024-09-15T01:32:35.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:35.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:35.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:35.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:35.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:35.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:35.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:35.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:35.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:35.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 10 monclient(hunting): handle_auth_done global_id 4499 payload 274 2024-09-15T01:32:35.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db8005300 msgr2=0x7f8db800b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db8005300 0x7f8db800b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 >> v1:172.21.15.204:6789/0 conn(0x7f8db8006640 legacy=0x7f8db8003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 1 monclient: found mon.noname-e 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.852+0000 7f8dd659c640 1 -- 172.21.15.178:0/1256812943 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db80058a0 con 0x7f8db800bdb0 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd659c640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8db800bdb0 0x7f8db800e190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8006f60 tx=0x7f8dc8008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=921328250716eb13 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd559a640 1 -- 172.21.15.178:0/1256812943 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8dc802cbe0 con 0x7f8db800bdb0 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd559a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd559a640 10 monclient: dump: 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:35.862 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd559a640 10 monclient: _finish_auth 0 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd559a640 1 -- 172.21.15.178:0/1256812943 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc800c040 con 0x7f8db800bdb0 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8ddc889640 5 monclient: authenticate success, global_id 4499 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8ddc889640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd559a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8ddc889640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd559a640 1 -- 172.21.15.178:0/1256812943 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8dc802cdc0 con 0x7f8db800bdb0 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8ddc889640 4 set_mon_vals no callback set 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd559a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.853+0000 7f8dd559a640 10 monclient: dump: 2024-09-15T01:32:35.863 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:35.864 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:35.865 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:35.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.854+0000 7f8ddc889640 10 monclient: shutdown 2024-09-15T01:32:35.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.854+0000 7f8ddc889640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:35.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.854+0000 7f8ddc889640 1 -- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8db800bdb0 msgr2=0x7f8db800e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:35.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.854+0000 7f8ddc889640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8db800bdb0 0x7f8db800e190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8006f60 tx=0x7f8dc8008040 comp rx=0 tx=0).stop 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.854+0000 7f8ddc889640 1 -- 172.21.15.178:0/1256812943 shutdown_connections 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.854+0000 7f8ddc889640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db8005300 0x7f8db800b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.854+0000 7f8ddc889640 1 --2- 172.21.15.178:0/1256812943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8db800bdb0 0x7f8db800e190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.854+0000 7f8ddc889640 1 -- 172.21.15.178:0/1256812943 >> 172.21.15.178:0/1256812943 conn(0x7f8dd80a5ed0 msgr2=0x7f8dd80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.854+0000 7f8ddc889640 1 -- 172.21.15.178:0/1256812943 shutdown_connections 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.854+0000 7f8ddc889640 1 -- 172.21.15.178:0/1256812943 wait complete. 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.855+0000 7f8ddc889640 10 monclient: build_initial_monmap 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.855+0000 7f8ddc889640 10 monclient: monmap: 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:35.866 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.855+0000 7f8ddc889640 1 Processor -- start 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.855+0000 7f8ddc889640 1 -- start start 2024-09-15T01:32:35.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.855+0000 7f8ddc889640 10 monclient: init 2024-09-15T01:32:35.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:35.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 10 monclient: _add_conns ranks=[2,8,7,6,4,0,3,1,5] 2024-09-15T01:32:35.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db8005300 0x7f8dd809b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 10 monclient(hunting): picked mon.noname-b con 0x7f8db8005300 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8db8006640 0x7f8dd8131800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 10 monclient(hunting): picked mon.noname-e con 0x7f8db8006640 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8db800bdb0 0x7f8dd811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 10 monclient(hunting): picked mon.noname-i con 0x7f8db800bdb0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd81429b0 con 0x7f8db8005300 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd80a2390 con 0x7f8db800bdb0 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd809d630 con 0x7f8db8006640 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8ddc889640 10 monclient(hunting): authenticate will time out at 760.606445s 2024-09-15T01:32:35.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd659c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db8005300 0x7f8dd809b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8db8006640 0x7f8dd8131800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd659c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db8005300 0x7f8dd809b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:37582/0 (socket says 172.21.15.178:37582) 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd659c640 1 -- 172.21.15.178:0/3424075793 learned_addr learned my addr 172.21.15.178:0/3424075793 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd659c640 10 monclient(hunting): get_auth_request con 0x7f8db8005300 auth_method 0 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request con 0x7f8db8006640 auth_method 0 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd6d9d640 1 --2- 172.21.15.178:0/3424075793 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8db800bdb0 0x7f8dd811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd6d9d640 10 monclient(hunting): get_auth_request con 0x7f8db800bdb0 auth_method 0 2024-09-15T01:32:35.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 10 monclient(hunting): handle_auth_done global_id 4508 payload 824 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/3424075793 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8db800bdb0 msgr2=0x7f8dd811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:35.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/3424075793 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8db800bdb0 0x7f8dd811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/3424075793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db8005300 msgr2=0x7f8dd809b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/3424075793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db8005300 0x7f8dd809b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 1 monclient: found mon.noname-e 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.856+0000 7f8dd5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dd5d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dd5d9b640 1 -- 172.21.15.178:0/3424075793 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc800e040 con 0x7f8db8006640 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dd659c640 1 --2- 172.21.15.178:0/3424075793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db8005300 0x7f8dd809b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dd5d9b640 1 --2- 172.21.15.178:0/3424075793 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8db8006640 0x7f8dd8131800 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc006fd0 tx=0x7f8dcc008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dc67fc640 1 -- 172.21.15.178:0/3424075793 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8dcc01e070 con 0x7f8db8006640 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dc67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dc67fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:35.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dc67fc640 10 monclient: dump: 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:35.877 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dc67fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dc67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8ddc889640 5 monclient: authenticate success, global_id 4508 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dc67fc640 1 -- 172.21.15.178:0/3424075793 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc017ea0 con 0x7f8db8006640 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8ddc889640 10 monclient: _renew_subs 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8ddc889640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8ddc889640 1 -- 172.21.15.178:0/3424075793 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd811ec00 con 0x7f8db8006640 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dc67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dc67fc640 1 -- 172.21.15.178:0/3424075793 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8dcc0195c0 con 0x7f8db8006640 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dc67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:35.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dc67fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dc67fc640 10 monclient: dump: 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:35.879 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8dd4d99640 4 set_mon_vals no callback set 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8ddc889640 10 monclient: _renew_subs 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8ddc889640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.857+0000 7f8ddc889640 1 -- 172.21.15.178:0/3424075793 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd809bbe0 con 0x7f8db8006640 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.858+0000 7f8dc67fc640 1 -- 172.21.15.178:0/3424075793 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8dcc016b90 con 0x7f8db8006640 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.858+0000 7f8ddc889640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.858+0000 7f8ddc889640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.858+0000 7f8ddc889640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.858+0000 7f8dc67fc640 1 --2- 172.21.15.178:0/3424075793 >> v2:172.21.15.204:6812/28210 conn(0x7f8db803b8e0 0x7f8db803dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.858+0000 7f8ddc889640 1 -- 172.21.15.178:0/3424075793 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd811ed90 con 0x7f8db8006640 2024-09-15T01:32:35.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.858+0000 7f8dc67fc640 1 -- 172.21.15.178:0/3424075793 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f8dcc04bf20 con 0x7f8db8006640 2024-09-15T01:32:35.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.858+0000 7f8dd659c640 1 --2- 172.21.15.178:0/3424075793 >> v2:172.21.15.204:6812/28210 conn(0x7f8db803b8e0 0x7f8db803dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:35.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.858+0000 7f8dd659c640 10 monclient: get_auth_request con 0x7f8db803b8e0 auth_method 0 2024-09-15T01:32:35.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.859+0000 7f8dd659c640 1 --2- 172.21.15.178:0/3424075793 >> v2:172.21.15.204:6812/28210 conn(0x7f8db803b8e0 0x7f8db803dd90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8008080 tx=0x7f8dc8036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:35.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.863+0000 7f8dc67fc640 1 -- 172.21.15.178:0/3424075793 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8dcc03b070 con 0x7f8db8006640 2024-09-15T01:32:35.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.863+0000 7f8dc67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:35.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:35.863+0000 7f8dc67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:36.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.085+0000 7f8ddc889640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:32:36.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.085+0000 7f8ddc889640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:32:36.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.085+0000 7f8ddc889640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:36.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.085+0000 7f8ddc889640 1 -- 172.21.15.178:0/3424075793 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8dd804b720 con 0x7f8db8006640 2024-09-15T01:32:36.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.086+0000 7f8dc67fc640 1 -- 172.21.15.178:0/3424075793 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3171 (secure 0 0 0) 0x7f8dcc029540 con 0x7f8db8006640 2024-09-15T01:32:36.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.086+0000 7f8dc67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:32:36.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.086+0000 7f8dc67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:36.088 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":20,"quorum":[0,1,2,4,5,7,8],"quorum_names":["a","f","b","c","h","i","e"],"quorum_leader_name":"h","quorum_age":1,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:32:36.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.087+0000 7f8ddc889640 1 -- 172.21.15.178:0/3424075793 >> v2:172.21.15.204:6812/28210 conn(0x7f8db803b8e0 msgr2=0x7f8db803dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.087+0000 7f8ddc889640 1 --2- 172.21.15.178:0/3424075793 >> v2:172.21.15.204:6812/28210 conn(0x7f8db803b8e0 0x7f8db803dd90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8008080 tx=0x7f8dc8036000 comp rx=0 tx=0).stop 2024-09-15T01:32:36.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.087+0000 7f8ddc889640 10 monclient: shutdown 2024-09-15T01:32:36.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.087+0000 7f8ddc889640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:36.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.087+0000 7f8ddc889640 1 -- 172.21.15.178:0/3424075793 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8db8006640 msgr2=0x7f8dd8131800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.087+0000 7f8ddc889640 1 --2- 172.21.15.178:0/3424075793 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8db8006640 0x7f8dd8131800 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc006fd0 tx=0x7f8dcc008040 comp rx=0 tx=0).stop 2024-09-15T01:32:36.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.088+0000 7f8ddc889640 1 -- 172.21.15.178:0/3424075793 shutdown_connections 2024-09-15T01:32:36.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.088+0000 7f8ddc889640 1 --2- 172.21.15.178:0/3424075793 >> v2:172.21.15.204:6812/28210 conn(0x7f8db803b8e0 0x7f8db803dd90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.088+0000 7f8ddc889640 1 --2- 172.21.15.178:0/3424075793 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8db800bdb0 0x7f8dd811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.088+0000 7f8ddc889640 1 --2- 172.21.15.178:0/3424075793 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8db8006640 0x7f8dd8131800 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.088+0000 7f8ddc889640 1 --2- 172.21.15.178:0/3424075793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db8005300 0x7f8dd809b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.088+0000 7f8ddc889640 1 -- 172.21.15.178:0/3424075793 >> 172.21.15.178:0/3424075793 conn(0x7f8dd80a5ed0 msgr2=0x7f8dd80a5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:36.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.088+0000 7f8ddc889640 1 -- 172.21.15.178:0/3424075793 shutdown_connections 2024-09-15T01:32:36.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.088+0000 7f8ddc889640 1 -- 172.21.15.178:0/3424075793 wait complete. 2024-09-15T01:32:36.100 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-15T01:32:36.100 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-09-15T01:32:36.101 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:32:36.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.193+0000 7efc66eaf640 10 monclient: get_monmap_and_config 2024-09-15T01:32:36.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.193+0000 7efc66eaf640 10 monclient: build_initial_monmap 2024-09-15T01:32:36.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.193+0000 7efc66eaf640 10 monclient: monmap: 2024-09-15T01:32:36.195 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:36.195 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.195 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:36.194357+0000 2024-09-15T01:32:36.195 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:36.194357+0000 2024-09-15T01:32:36.195 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:36.196 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 1 Processor -- start 2024-09-15T01:32:36.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 1 -- start start 2024-09-15T01:32:36.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 10 monclient: init 2024-09-15T01:32:36.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:36.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 10 monclient: _add_conns ranks=[1,5,4,3,9,10,6,0,2,7,8] 2024-09-15T01:32:36.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7efc600a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:36.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efc60132de0 0x7efc60137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 10 monclient(hunting): picked mon.noname-h con 0x7efc60132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc601380e0 0x7efc6013a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 10 monclient(hunting): picked mon.noname-c con 0x7efc601380e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc6004bf60 con 0x7efc600a2930 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc6009dd80 con 0x7efc601380e0 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc600a2d00 con 0x7efc60132de0 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc66eaf640 10 monclient(hunting): authenticate will time out at 760.944877s 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.194+0000 7efc6544c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc601380e0 0x7efc6013a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc601380e0 0x7efc6013a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:55884/0 (socket says 172.21.15.178:55884) 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 1 -- 172.21.15.178:0/2110658004 learned_addr learned my addr 172.21.15.178:0/2110658004 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc5ffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efc60132de0 0x7efc60137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 10 monclient(hunting): get_auth_request con 0x7efc601380e0 auth_method 0 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc64c4b640 1 --1- 172.21.15.178:0/2110658004 >> v1:172.21.15.204:6789/0 conn(0x7efc600a2930 0x7efc6009faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:58422/0 (socket says 172.21.15.178:58422) 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc5ffff640 10 monclient(hunting): get_auth_request con 0x7efc60132de0 auth_method 0 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 10 monclient(hunting): handle_auth_done global_id 4468 payload 274 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 1 -- 172.21.15.178:0/2110658004 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efc60132de0 msgr2=0x7efc60137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 1 --2- 172.21.15.178:0/2110658004 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efc60132de0 0x7efc60137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 1 -- 172.21.15.178:0/2110658004 >> v1:172.21.15.204:6789/0 conn(0x7efc600a2930 legacy=0x7efc6009faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 1 monclient: found mon.noname-c 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 1 -- 172.21.15.178:0/2110658004 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc601331b0 con 0x7efc601380e0 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.195+0000 7efc6544c640 1 --2- 172.21.15.178:0/2110658004 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc601380e0 0x7efc6013a4c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efc50009f50 tx=0x7efc50009840 comp rx=0 tx=0).ready entity=mon.4 client_cookie=238f36c0906afceb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc5f7fe640 1 -- 172.21.15.178:0/2110658004 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc50007880 con 0x7efc601380e0 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc5f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc5f7fe640 10 monclient: dump: 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc5f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc5f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc5f7fe640 1 -- 172.21.15.178:0/2110658004 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5002d050 con 0x7efc601380e0 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 5 monclient: authenticate success, global_id 4468 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc5f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc5f7fe640 1 -- 172.21.15.178:0/2110658004 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc50030360 con 0x7efc601380e0 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 4 set_mon_vals no callback set 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc5f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc5f7fe640 10 monclient: dump: 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 10 monclient: shutdown 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 1 -- 172.21.15.178:0/2110658004 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc601380e0 msgr2=0x7efc6013a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 1 --2- 172.21.15.178:0/2110658004 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc601380e0 0x7efc6013a4c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efc50009f50 tx=0x7efc50009840 comp rx=0 tx=0).stop 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 1 -- 172.21.15.178:0/2110658004 shutdown_connections 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 1 --2- 172.21.15.178:0/2110658004 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc601380e0 0x7efc6013a4c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 1 --2- 172.21.15.178:0/2110658004 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efc60132de0 0x7efc60137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 1 -- 172.21.15.178:0/2110658004 >> 172.21.15.178:0/2110658004 conn(0x7efc600a5ed0 msgr2=0x7efc600a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 1 -- 172.21.15.178:0/2110658004 shutdown_connections 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.196+0000 7efc66eaf640 1 -- 172.21.15.178:0/2110658004 wait complete. 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.197+0000 7efc66eaf640 10 monclient: build_initial_monmap 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.197+0000 7efc66eaf640 10 monclient: monmap: 2024-09-15T01:32:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:36.214 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.197+0000 7efc66eaf640 1 Processor -- start 2024-09-15T01:32:36.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.197+0000 7efc66eaf640 1 -- start start 2024-09-15T01:32:36.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.197+0000 7efc66eaf640 10 monclient: init 2024-09-15T01:32:36.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:36.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 10 monclient: _add_conns ranks=[0,4,8,7,3,6,2,5,1] 2024-09-15T01:32:36.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc600a2930 0x7efc60132520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 10 monclient(hunting): picked mon.noname-a con 0x7efc600a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:36.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc60132de0 0x7efc6009a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 10 monclient(hunting): picked mon.noname-c con 0x7efc60132de0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:36.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc601380e0 0x7efc6009ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 10 monclient(hunting): picked mon.noname-e con 0x7efc601380e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6011db20 con 0x7efc600a2930 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc600a2390 con 0x7efc60132de0 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7efc6009d630 con 0x7efc601380e0 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc66eaf640 10 monclient(hunting): authenticate will time out at 760.948269s 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5ffff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc60132de0 0x7efc6009a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc600a2930 0x7efc60132520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc6544c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc601380e0 0x7efc6009ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5ffff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc60132de0 0x7efc6009a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:55890/0 (socket says 172.21.15.178:55890) 2024-09-15T01:32:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5ffff640 1 -- 172.21.15.178:0/1240110010 learned_addr learned my addr 172.21.15.178:0/1240110010 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5ffff640 10 monclient(hunting): get_auth_request con 0x7efc60132de0 auth_method 0 2024-09-15T01:32:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 10 monclient(hunting): get_auth_request con 0x7efc600a2930 auth_method 0 2024-09-15T01:32:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc6544c640 10 monclient(hunting): get_auth_request con 0x7efc601380e0 auth_method 0 2024-09-15T01:32:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc6544c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc6544c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc6544c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc6544c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc6544c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc6544c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 10 monclient(hunting): handle_auth_done global_id 4572 payload 824 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 1 -- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc601380e0 msgr2=0x7efc6009ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc601380e0 0x7efc6009ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 1 -- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc60132de0 msgr2=0x7efc6009a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc60132de0 0x7efc6009a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 10 monclient: _finish_hunting 0 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 1 monclient: found mon.noname-a 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 1 -- 172.21.15.178:0/1240110010 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50009ae0 con 0x7efc600a2930 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5ffff640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc60132de0 0x7efc6009a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc64c4b640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc600a2930 0x7efc60132520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efc5800c470 tx=0x7efc58007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc580047a0 con 0x7efc600a2930 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.198+0000 7efc5cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc5cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc5cff9640 10 monclient: dump: 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.220 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.220 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.220 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.220 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.220 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.220 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.220 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.220 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.220 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.220 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.220 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.221 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.221 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.221 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc5cff9640 10 monclient: _finish_auth 0 2024-09-15T01:32:36.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc5cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:36.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc58004940 con 0x7efc600a2930 2024-09-15T01:32:36.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc66eaf640 5 monclient: authenticate success, global_id 4572 2024-09-15T01:32:36.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc66eaf640 10 monclient: _renew_subs 2024-09-15T01:32:36.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc66eaf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:36.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc66eaf640 1 -- 172.21.15.178:0/1240110010 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6009ca30 con 0x7efc600a2930 2024-09-15T01:32:36.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc5cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc58013400 con 0x7efc600a2930 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc5cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc5cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc5cff9640 10 monclient: dump: 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.222 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc5effd640 4 set_mon_vals no callback set 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc66eaf640 10 monclient: _renew_subs 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc66eaf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc66eaf640 1 -- 172.21.15.178:0/1240110010 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc601319d0 con 0x7efc600a2930 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.199+0000 7efc66eaf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc66eaf640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6011c570 0x7efc6009b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc66eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc66eaf640 1 -- 172.21.15.178:0/1240110010 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc60131bc0 con 0x7efc6011c570 2024-09-15T01:32:36.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7efc58004ab0 con 0x7efc600a2930 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5ffff640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6011c570 0x7efc6009b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5ffff640 10 monclient: get_auth_request con 0x7efc6011c570 auth_method 0 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 1 --2- 172.21.15.178:0/1240110010 >> v2:172.21.15.204:6812/28210 conn(0x7efc400396e0 0x7efc4003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5ffff640 10 monclient(hunting): handle_auth_done global_id 4581 payload 824 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7efc5804a160 con 0x7efc600a2930 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5ffff640 1 -- 172.21.15.178:0/1240110010 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc6004c4a0 con 0x7efc6011c570 2024-09-15T01:32:36.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5ffff640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6011c570 0x7efc6009b1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc48009f30 tx=0x7efc480090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc6544c640 1 --2- 172.21.15.178:0/1240110010 >> v2:172.21.15.204:6812/28210 conn(0x7efc400396e0 0x7efc4003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc6544c640 10 monclient: get_auth_request con 0x7efc400396e0 auth_method 0 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc48014ad0 con 0x7efc6011c570 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 10 monclient: dump: 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.225 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7efc48014c30 con 0x7efc6011c570 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6011c570 msgr2=0x7efc6009b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6011c570 0x7efc6009b1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc48009f30 tx=0x7efc480090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6011c570 msgr2=0x7efc6009b1a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:36.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc5cff9640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc6011c570 0x7efc6009b1a0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc48009f30 tx=0x7efc480090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.200+0000 7efc6544c640 1 --2- 172.21.15.178:0/1240110010 >> v2:172.21.15.204:6812/28210 conn(0x7efc400396e0 0x7efc4003bb90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efc50033000 tx=0x7efc50036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc66eaf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:36.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc66eaf640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60096a30 0x7efc60096e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc66eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc66eaf640 1 -- 172.21.15.178:0/1240110010 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6004bc80 con 0x7efc60096a30 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc5ffff640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60096a30 0x7efc60096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc5ffff640 10 monclient: get_auth_request con 0x7efc60096a30 auth_method 0 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc5ffff640 10 monclient(hunting): handle_auth_done global_id 4590 payload 824 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc5ffff640 1 -- 172.21.15.178:0/1240110010 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efc6005ed70 con 0x7efc60096a30 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.206+0000 7efc5ffff640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60096a30 0x7efc60096e30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efc4801b000 tx=0x7efc4801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc48024040 con 0x7efc60096a30 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc5cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc5cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc5cff9640 10 monclient: dump: 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.230 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.231 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.231 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.231 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.231 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9119 (secure 0 0 0) 0x7efc480133a0 con 0x7efc60096a30 2024-09-15T01:32:36.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc5cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:36.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc5cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:36.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60096a30 msgr2=0x7efc60096e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc5cff9640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60096a30 0x7efc60096e30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efc4801b000 tx=0x7efc4801c000 comp rx=0 tx=0).stop 2024-09-15T01:32:36.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc5cff9640 1 -- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60096a30 msgr2=0x7efc60096e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:36.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc5cff9640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc60096a30 0x7efc60096e30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.232 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:36.232 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:36.232 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:36.232 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:36.232 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 20, 2024-09-15T01:32:36.232 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:36.232 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:36.232 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:36.232 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:36.232 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:36.232 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:36.233 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:36.234 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:36.234 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:36.234 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:36.234 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.234 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:36.234 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:36.234 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:36.234 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:36.234 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:36.234 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:36.234 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:36.234 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:36.235 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:36.236 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:36.236 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:36.236 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:36.236 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:36.236 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:36.236 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:36.236 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:36.236 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:36.236 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:36.237 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:36.237 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:36.237 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:36.237 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:36.237 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:36.237 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:36.237 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:36.237 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:36.237 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.237 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:36.237 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.237 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:36.238 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:36.240 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:36.241 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.243 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.244 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:36.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.245 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:36.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.245 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.245 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.247 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:36.249 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.249 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.249 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:36.249 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:36.249 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.249 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.249 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.249 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.250 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:36.252 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:36.253 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.253 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.253 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.253 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.253 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.253 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:36.253 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:36.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.253 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:36.253 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:36.254 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:36.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc66eaf640 1 -- 172.21.15.178:0/1240110010 >> v2:172.21.15.204:6812/28210 conn(0x7efc400396e0 msgr2=0x7efc4003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc66eaf640 1 --2- 172.21.15.178:0/1240110010 >> v2:172.21.15.204:6812/28210 conn(0x7efc400396e0 0x7efc4003bb90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efc50033000 tx=0x7efc50036000 comp rx=0 tx=0).stop 2024-09-15T01:32:36.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc66eaf640 10 monclient: shutdown 2024-09-15T01:32:36.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc66eaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:36.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc66eaf640 1 -- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc600a2930 msgr2=0x7efc60132520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc66eaf640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc600a2930 0x7efc60132520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efc5800c470 tx=0x7efc58007e20 comp rx=0 tx=0).stop 2024-09-15T01:32:36.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc6544c640 1 -- 172.21.15.178:0/1240110010 reap_dead start 2024-09-15T01:32:36.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc66eaf640 1 -- 172.21.15.178:0/1240110010 shutdown_connections 2024-09-15T01:32:36.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc66eaf640 1 --2- 172.21.15.178:0/1240110010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc600a2930 0x7efc60132520 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc66eaf640 1 -- 172.21.15.178:0/1240110010 >> 172.21.15.178:0/1240110010 conn(0x7efc600a5ed0 msgr2=0x7efc60135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:36.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc66eaf640 1 -- 172.21.15.178:0/1240110010 shutdown_connections 2024-09-15T01:32:36.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.207+0000 7efc66eaf640 1 -- 172.21.15.178:0/1240110010 wait complete. 2024-09-15T01:32:36.257 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:32:36.257 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-09-15T01:32:36.257 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:32:36.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.350+0000 7f9993f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:36.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.350+0000 7f9993f9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:36.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.350+0000 7f9993f9d640 10 monclient: monmap: 2024-09-15T01:32:36.353 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:36.353 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.353 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:36.351533+0000 2024-09-15T01:32:36.353 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:36.351533+0000 2024-09-15T01:32:36.353 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:36.353 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.353 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:36.353 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:36.354 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:36.354 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:36.354 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:36.354 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:36.354 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:36.354 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:36.354 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:36.354 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:36.354 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:36.354 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 1 Processor -- start 2024-09-15T01:32:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 1 -- start start 2024-09-15T01:32:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 10 monclient: init 2024-09-15T01:32:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 10 monclient: _add_conns ranks=[2,7,5,4,3,10,6,8,9,1,0] 2024-09-15T01:32:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c0a2930 0x7f998c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f998c0a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f998c09faa0 0x7f998c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 10 monclient(hunting): picked mon.noname-i con 0x7f998c09faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f998c0a08c0 0x7f998c0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 10 monclient(hunting): picked mon.noname-h con 0x7f998c0a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c09e0f0 con 0x7f998c0a2930 2024-09-15T01:32:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f998c09dd80 con 0x7f998c0a08c0 2024-09-15T01:32:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f998c0a2d00 con 0x7f998c09faa0 2024-09-15T01:32:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.351+0000 7f9993f9d640 10 monclient(hunting): authenticate will time out at 761.101940s 2024-09-15T01:32:36.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c0a2930 0x7f998c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c0a2930 0x7f998c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:37584/0 (socket says 172.21.15.178:37584) 2024-09-15T01:32:36.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 1 -- 172.21.15.178:0/2653277010 learned_addr learned my addr 172.21.15.178:0/2653277010 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:36.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 10 monclient(hunting): get_auth_request con 0x7f998c0a2930 auth_method 0 2024-09-15T01:32:36.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991538640 1 --2- 172.21.15.178:0/2653277010 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f998c09faa0 0x7f998c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f999253a640 1 --2- 172.21.15.178:0/2653277010 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f998c0a08c0 0x7f998c0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991538640 10 monclient(hunting): get_auth_request con 0x7f998c09faa0 auth_method 0 2024-09-15T01:32:36.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991538640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f999253a640 10 monclient(hunting): get_auth_request con 0x7f998c0a08c0 auth_method 0 2024-09-15T01:32:36.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f999253a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f999253a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f999253a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 10 monclient(hunting): handle_auth_done global_id 4511 payload 274 2024-09-15T01:32:36.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 1 -- 172.21.15.178:0/2653277010 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f998c09faa0 msgr2=0x7f998c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:36.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 1 --2- 172.21.15.178:0/2653277010 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f998c09faa0 0x7f998c09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 1 -- 172.21.15.178:0/2653277010 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f998c0a08c0 msgr2=0x7f998c0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 1 --2- 172.21.15.178:0/2653277010 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f998c0a08c0 0x7f998c0a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 10 monclient: _finish_hunting 0 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 1 monclient: found mon.noname-b 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f9991d39640 1 -- 172.21.15.178:0/2653277010 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c04bf60 con 0x7f998c0a2930 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.352+0000 7f999253a640 1 --2- 172.21.15.178:0/2653277010 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f998c0a08c0 0x7f998c0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9991d39640 1 --2- 172.21.15.178:0/2653277010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c0a2930 0x7f998c132de0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9988009940 tx=0x7f99880091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a562303c8d139dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9990d37640 1 -- 172.21.15.178:0/2653277010 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f998802ac60 con 0x7f998c0a2930 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9990d37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9990d37640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9990d37640 10 monclient: dump: 2024-09-15T01:32:36.374 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9990d37640 10 monclient: _finish_auth 0 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9990d37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9990d37640 1 -- 172.21.15.178:0/2653277010 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998802ade0 con 0x7f998c0a2930 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 5 monclient: authenticate success, global_id 4511 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9990d37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9990d37640 1 -- 172.21.15.178:0/2653277010 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9988031ca0 con 0x7f998c0a2930 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9990d37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 4 set_mon_vals no callback set 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9990d37640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9990d37640 10 monclient: dump: 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 10 monclient: shutdown 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 1 -- 172.21.15.178:0/2653277010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c0a2930 msgr2=0x7f998c132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 1 --2- 172.21.15.178:0/2653277010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c0a2930 0x7f998c132de0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9988009940 tx=0x7f99880091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 1 -- 172.21.15.178:0/2653277010 shutdown_connections 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 1 --2- 172.21.15.178:0/2653277010 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f998c0a08c0 0x7f998c0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 1 --2- 172.21.15.178:0/2653277010 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f998c09faa0 0x7f998c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 1 --2- 172.21.15.178:0/2653277010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c0a2930 0x7f998c132de0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 1 -- 172.21.15.178:0/2653277010 >> 172.21.15.178:0/2653277010 conn(0x7f998c0a5ed0 msgr2=0x7f998c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 1 -- 172.21.15.178:0/2653277010 shutdown_connections 2024-09-15T01:32:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.353+0000 7f9993f9d640 1 -- 172.21.15.178:0/2653277010 wait complete. 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.354+0000 7f9993f9d640 10 monclient: build_initial_monmap 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.354+0000 7f9993f9d640 10 monclient: monmap: 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.354+0000 7f9993f9d640 1 Processor -- start 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.354+0000 7f9993f9d640 1 -- start start 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.354+0000 7f9993f9d640 10 monclient: init 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 10 monclient: _add_conns ranks=[2,8,1,0,5,7,6,4,3] 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c09faa0 0x7f998c09b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f998c09faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f998c0a08c0 0x7f998c131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f998c0a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f998c0a2930 0x7f998c141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 10 monclient(hunting): picked mon.noname-f con 0x7f998c0a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c09d630 con 0x7f998c0a2930 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c11f090 con 0x7f998c09faa0 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f998c11ed20 con 0x7f998c0a08c0 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 10 monclient(hunting): authenticate will time out at 761.105139s 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c09faa0 0x7f998c09b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c09faa0 0x7f998c09b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:37590/0 (socket says 172.21.15.178:37590) 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 1 -- 172.21.15.178:0/3178474375 learned_addr learned my addr 172.21.15.178:0/3178474375 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 10 monclient(hunting): get_auth_request con 0x7f998c09faa0 auth_method 0 2024-09-15T01:32:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991538640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f998c0a08c0 0x7f998c131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991538640 10 monclient(hunting): get_auth_request con 0x7f998c0a08c0 auth_method 0 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991538640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f999253a640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f998c0a2930 0x7f998c141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f999253a640 10 monclient(hunting): get_auth_request con 0x7f998c0a2930 auth_method 0 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f999253a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f999253a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f999253a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f999253a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f999253a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f999253a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 10 monclient(hunting): handle_auth_done global_id 4520 payload 824 2024-09-15T01:32:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 1 -- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f998c0a08c0 msgr2=0x7f998c131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f998c0a08c0 0x7f998c131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 1 -- 172.21.15.178:0/3178474375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f998c0a2930 msgr2=0x7f998c141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f998c0a2930 0x7f998c141d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 10 monclient: _finish_hunting 0 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 1 monclient: found mon.noname-b 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 1 -- 172.21.15.178:0/3178474375 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9988009450 con 0x7f998c09faa0 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9991d39640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c09faa0 0x7f998c09b160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f99880097e0 tx=0x7f9988032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f998803bb50 con 0x7f998c09faa0 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9981ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9981ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9981ffb640 10 monclient: dump: 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9981ffb640 10 monclient: _finish_auth 0 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9981ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 5 monclient: authenticate success, global_id 4520 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998803bd10 con 0x7f998c09faa0 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9981ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f998803be80 con 0x7f998c09faa0 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9981ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9981ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9981ffb640 10 monclient: dump: 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.386 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.387 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.387 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.387 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.387 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.387 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.387 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.387 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.387 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.387 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.387 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9983fff640 4 set_mon_vals no callback set 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 10 monclient: _renew_subs 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.355+0000 7f9993f9d640 1 -- 172.21.15.178:0/3178474375 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998c11eaa0 con 0x7f998c09faa0 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.356+0000 7f9993f9d640 10 monclient: _renew_subs 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.356+0000 7f9993f9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.356+0000 7f9993f9d640 1 -- 172.21.15.178:0/3178474375 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c11db50 con 0x7f998c09faa0 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.356+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9988039040 con 0x7f998c09faa0 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.356+0000 7f9993f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:36.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.356+0000 7f9993f9d640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c11ca10 0x7f998c142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.356+0000 7f9993f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.356+0000 7f9993f9d640 1 -- 172.21.15.178:0/3178474375 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c11f090 con 0x7f998c11ca10 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 1 --2- 172.21.15.178:0/3178474375 >> v2:172.21.15.204:6812/28210 conn(0x7f9970039710 0x7f997003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9991538640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c11ca10 0x7f998c142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9991538640 10 monclient: get_auth_request con 0x7f998c11ca10 auth_method 0 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9991538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9991538640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9991538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f998806aaf0 con 0x7f998c09faa0 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9991538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9991538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9991538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f999253a640 1 --2- 172.21.15.178:0/3178474375 >> v2:172.21.15.204:6812/28210 conn(0x7f9970039710 0x7f997003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f999253a640 10 monclient: get_auth_request con 0x7f9970039710 auth_method 0 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9991538640 10 monclient(hunting): handle_auth_done global_id 4529 payload 824 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9991538640 1 -- 172.21.15.178:0/3178474375 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f998c04c4a0 con 0x7f998c11ca10 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9991538640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c11ca10 0x7f998c142290 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f997c011bf0 tx=0x7f997c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f997c00fc80 con 0x7f998c11ca10 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 10 monclient: dump: 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.390 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f999253a640 1 --2- 172.21.15.178:0/3178474375 >> v2:172.21.15.204:6812/28210 conn(0x7f9970039710 0x7f997003bbc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f998c09b0b0 tx=0x7f99840099b0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f997c00fde0 con 0x7f998c11ca10 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:36.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c11ca10 msgr2=0x7f998c142290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c11ca10 0x7f998c142290 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f997c011bf0 tx=0x7f997c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c11ca10 msgr2=0x7f998c142290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:36.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.357+0000 7f9981ffb640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c11ca10 0x7f998c142290 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f997c011bf0 tx=0x7f997c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9993f9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:36.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9993f9d640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c131e60 0x7f998c1427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9993f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9993f9d640 1 -- 172.21.15.178:0/3178474375 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c09f940 con 0x7f998c131e60 2024-09-15T01:32:36.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9991538640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c131e60 0x7f998c1427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9991538640 10 monclient: get_auth_request con 0x7f998c131e60 auth_method 0 2024-09-15T01:32:36.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9991538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9991538640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9991538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9991538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9991538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9991538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9991538640 10 monclient(hunting): handle_auth_done global_id 4538 payload 824 2024-09-15T01:32:36.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.363+0000 7f9991538640 1 -- 172.21.15.178:0/3178474375 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f998c11ee90 con 0x7f998c131e60 2024-09-15T01:32:36.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9991538640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c131e60 0x7f998c1427d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f997c011bf0 tx=0x7f997c01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f997c024040 con 0x7f998c131e60 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9981ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9981ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9981ffb640 10 monclient: dump: 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.394 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9119 (secure 0 0 0) 0x7f997c0133a0 con 0x7f998c131e60 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9981ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9981ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c131e60 msgr2=0x7f998c1427d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9981ffb640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c131e60 0x7f998c1427d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f997c011bf0 tx=0x7f997c01c000 comp rx=0 tx=0).stop 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9981ffb640 1 -- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c131e60 msgr2=0x7f998c1427d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:36.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9981ffb640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c131e60 0x7f998c1427d0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f997c011bf0 tx=0x7f997c01c000 comp rx=0 tx=0).stop 2024-09-15T01:32:36.396 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:36.396 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:36.396 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:36.396 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:36.396 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 20, 2024-09-15T01:32:36.396 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:36.396 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:36.396 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:36.396 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:36.396 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:36.396 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:36.397 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:36.398 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:36.399 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:36.400 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.401 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.402 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.403 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.404 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.405 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.406 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:36.407 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.407 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:36.407 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.407 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.407 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:36.407 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:36.408 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:36.410 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.415 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.416 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:36.416 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:36.416 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.416 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.416 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.416 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.416 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:36.417 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:32:36.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.418 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.418 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:36.418 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:36.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9993f9d640 1 -- 172.21.15.178:0/3178474375 >> v2:172.21.15.204:6812/28210 conn(0x7f9970039710 msgr2=0x7f997003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9993f9d640 1 --2- 172.21.15.178:0/3178474375 >> v2:172.21.15.204:6812/28210 conn(0x7f9970039710 0x7f997003bbc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f998c09b0b0 tx=0x7f99840099b0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9993f9d640 10 monclient: shutdown 2024-09-15T01:32:36.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9993f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:36.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9993f9d640 1 -- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c09faa0 msgr2=0x7f998c09b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9993f9d640 1 --2- 172.21.15.178:0/3178474375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f998c09faa0 0x7f998c09b160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f99880097e0 tx=0x7f9988032000 comp rx=0 tx=0).stop 2024-09-15T01:32:36.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f999253a640 1 -- 172.21.15.178:0/3178474375 reap_dead start 2024-09-15T01:32:36.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f999253a640 1 -- 172.21.15.178:0/3178474375 reap_dead start 2024-09-15T01:32:36.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9993f9d640 1 -- 172.21.15.178:0/3178474375 shutdown_connections 2024-09-15T01:32:36.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.364+0000 7f9993f9d640 1 -- 172.21.15.178:0/3178474375 >> 172.21.15.178:0/3178474375 conn(0x7f998c0a5ed0 msgr2=0x7f998c135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:36.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.365+0000 7f9993f9d640 1 -- 172.21.15.178:0/3178474375 shutdown_connections 2024-09-15T01:32:36.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.365+0000 7f9993f9d640 1 -- 172.21.15.178:0/3178474375 wait complete. 2024-09-15T01:32:36.420 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:32:36.420 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-09-15T01:32:36.421 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:32:36.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.517+0000 7f913c64f640 10 monclient: get_monmap_and_config 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.517+0000 7f913c64f640 10 monclient: build_initial_monmap 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.517+0000 7f913c64f640 10 monclient: monmap: 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:36.518237+0000 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:36.518237+0000 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.517+0000 7f913c64f640 1 Processor -- start 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.517+0000 7f913c64f640 1 -- start start 2024-09-15T01:32:36.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.517+0000 7f913c64f640 10 monclient: init 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 10 monclient: _add_conns ranks=[0,2,4,10,6,7,9,5,8,1,3] 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f91340a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9134132de0 0x7f9134137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 10 monclient(hunting): picked mon.noname-b con 0x7f9134132de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f91341380e0 0x7f913413a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 10 monclient(hunting): picked mon.noname-c con 0x7f91341380e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913404bf60 con 0x7f91340a2930 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913409dd80 con 0x7f9134132de0 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91340a2d00 con 0x7f91341380e0 2024-09-15T01:32:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913c64f640 10 monclient(hunting): authenticate will time out at 761.268683s 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913abec640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f91341380e0 0x7f913413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f9139bea640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9134132de0 0x7f9134137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913a3eb640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f91340a2930 0x7f913409faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:53344/0 (socket says 172.21.15.178:53344) 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913a3eb640 1 -- 172.21.15.178:0/1443628146 learned_addr learned my addr 172.21.15.178:0/1443628146 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f9139bea640 10 monclient(hunting): get_auth_request con 0x7f9134132de0 auth_method 0 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f9139bea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f9139bea640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f9139bea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913abec640 10 monclient(hunting): get_auth_request con 0x7f91341380e0 auth_method 0 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913abec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913abec640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913abec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f9139bea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f9139bea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f9139bea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913abec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913abec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.518+0000 7f913abec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f913abec640 10 monclient(hunting): handle_auth_done global_id 4486 payload 274 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f913abec640 1 -- 172.21.15.178:0/1443628146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9134132de0 msgr2=0x7f9134137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f913abec640 1 --2- 172.21.15.178:0/1443628146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9134132de0 0x7f9134137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f91393e9640 1 -- 172.21.15.178:0/1443628146 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941873791 0 0) 0x7f913404bf60 con 0x7f91340a2930 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f913abec640 1 -- 172.21.15.178:0/1443628146 >> v1:172.21.15.178:6789/0 conn(0x7f91340a2930 legacy=0x7f913409faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f913abec640 10 monclient: _finish_hunting 0 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f913abec640 1 monclient: found mon.noname-c 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f913abec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f913abec640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f913abec640 1 -- 172.21.15.178:0/1443628146 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91341331b0 con 0x7f91341380e0 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f9139bea640 1 --2- 172.21.15.178:0/1443628146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9134132de0 0x7f9134137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f91393e9640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:32:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f913abec640 1 --2- 172.21.15.178:0/1443628146 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f91341380e0 0x7f913413a4c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f912c009e90 tx=0x7f912c009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=fefe55e95b7f29b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f91393e9640 1 -- 172.21.15.178:0/1443628146 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f912c031880 con 0x7f91341380e0 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f91393e9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f91393e9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f91393e9640 10 monclient: dump: 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f91393e9640 10 monclient: _finish_auth 0 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.519+0000 7f91393e9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f91393e9640 1 -- 172.21.15.178:0/1443628146 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c02d050 con 0x7f91341380e0 2024-09-15T01:32:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 5 monclient: authenticate success, global_id 4486 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f91393e9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 4 set_mon_vals no callback set 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f91393e9640 1 -- 172.21.15.178:0/1443628146 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f912c0303a0 con 0x7f91341380e0 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f91393e9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f91393e9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f91393e9640 10 monclient: dump: 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 10 monclient: shutdown 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 1 -- 172.21.15.178:0/1443628146 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f91341380e0 msgr2=0x7f913413a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 1 --2- 172.21.15.178:0/1443628146 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f91341380e0 0x7f913413a4c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f912c009e90 tx=0x7f912c009780 comp rx=0 tx=0).stop 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 1 -- 172.21.15.178:0/1443628146 shutdown_connections 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 1 --2- 172.21.15.178:0/1443628146 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f91341380e0 0x7f913413a4c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 1 --2- 172.21.15.178:0/1443628146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9134132de0 0x7f9134137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 1 -- 172.21.15.178:0/1443628146 >> 172.21.15.178:0/1443628146 conn(0x7f91340a5ed0 msgr2=0x7f91340a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 1 -- 172.21.15.178:0/1443628146 shutdown_connections 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.520+0000 7f913c64f640 1 -- 172.21.15.178:0/1443628146 wait complete. 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient: build_initial_monmap 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient: monmap: 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 1 Processor -- start 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 1 -- start start 2024-09-15T01:32:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient: init 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient: _add_conns ranks=[6,7,4,5,0,1,2,8,3] 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f91340a2930 0x7f9134141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient(hunting): picked mon.noname-d con 0x7f91340a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9134132de0 0x7f913409c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient(hunting): picked mon.noname-i con 0x7f9134132de0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f91341380e0 0x7f913409c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient(hunting): picked mon.noname-c con 0x7f91341380e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91340a2390 con 0x7f91341380e0 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f913409d630 con 0x7f91340a2930 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f91340971b0 con 0x7f9134132de0 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913a3eb640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f91340a2930 msgr2=0x7f9134141930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913c64f640 10 monclient(hunting): authenticate will time out at 761.271919s 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913a3eb640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f91340a2930 0x7f9134141930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.521+0000 7f913abec640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f91341380e0 0x7f913409c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f91341380e0 0x7f913409c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:55912/0 (socket says 172.21.15.178:55912) 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 1 -- 172.21.15.178:0/4030693293 learned_addr learned my addr 172.21.15.178:0/4030693293 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 10 monclient(hunting): get_auth_request con 0x7f91341380e0 auth_method 0 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f9139bea640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9134132de0 0x7f913409c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f9139bea640 10 monclient(hunting): get_auth_request con 0x7f9134132de0 auth_method 0 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f9139bea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f9139bea640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f9139bea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f9139bea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f9139bea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f9139bea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 10 monclient(hunting): handle_auth_done global_id 4495 payload 824 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 1 -- 172.21.15.178:0/4030693293 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9134132de0 msgr2=0x7f913409c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9134132de0 0x7f913409c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 1 -- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f91340a2930 msgr2=0x7f9134141930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f91340a2930 0x7f9134141930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 10 monclient: _finish_hunting 0 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 1 monclient: found mon.noname-c 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 1 -- 172.21.15.178:0/4030693293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9124009390 con 0x7f91341380e0 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913abec640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f91341380e0 0x7f913409c750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f912c0099f0 tx=0x7f912c037000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f912c030c00 con 0x7f91341380e0 2024-09-15T01:32:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f912a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f912a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f912a7fc640 10 monclient: dump: 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f912a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f912a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c030d60 con 0x7f91341380e0 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913c64f640 5 monclient: authenticate success, global_id 4495 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913c64f640 10 monclient: _renew_subs 2024-09-15T01:32:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913c64f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913c64f640 1 -- 172.21.15.178:0/4030693293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c009a20 con 0x7f91341380e0 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f912a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f912c03d8b0 con 0x7f91341380e0 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f912a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f912a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f912a7fc640 10 monclient: dump: 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f9138be8640 4 set_mon_vals no callback set 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913c64f640 10 monclient: _renew_subs 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913c64f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:32:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.522+0000 7f913c64f640 1 -- 172.21.15.178:0/4030693293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9134098420 con 0x7f91341380e0 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913c64f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f912c003040 con 0x7f91341380e0 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913c64f640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134141e70 0x7f9134135c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913c64f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913c64f640 1 -- 172.21.15.178:0/4030693293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9134098e60 con 0x7f9134141e70 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913a3eb640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134141e70 0x7f9134135c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913a3eb640 10 monclient: get_auth_request con 0x7f9134141e70 auth_method 0 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913a3eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913a3eb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913a3eb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913a3eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913a3eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f912a7fc640 1 --2- 172.21.15.178:0/4030693293 >> v2:172.21.15.204:6812/28210 conn(0x7f91180397b0 0x7f911803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913a3eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f912c06c280 con 0x7f91341380e0 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913a3eb640 10 monclient(hunting): handle_auth_done global_id 4504 payload 824 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913a3eb640 1 -- 172.21.15.178:0/4030693293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9134142860 con 0x7f9134141e70 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f9139bea640 1 --2- 172.21.15.178:0/4030693293 >> v2:172.21.15.204:6812/28210 conn(0x7f91180397b0 0x7f911803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f9139bea640 10 monclient: get_auth_request con 0x7f91180397b0 auth_method 0 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.523+0000 7f913a3eb640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134141e70 0x7f9134135c00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9130004830 tx=0x7f9130007d90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.524+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91300070d0 con 0x7f9134141e70 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.524+0000 7f912a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.524+0000 7f912a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.524+0000 7f912a7fc640 10 monclient: dump: 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.524+0000 7f9139bea640 1 --2- 172.21.15.178:0/4030693293 >> v2:172.21.15.204:6812/28210 conn(0x7f91180397b0 0x7f911803bc60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91340a6ec0 tx=0x7f91240090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.524+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f9130005b70 con 0x7f9134141e70 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.524+0000 7f912a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.524+0000 7f912a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.524+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134141e70 msgr2=0x7f9134135c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.524+0000 7f912a7fc640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134141e70 0x7f9134135c00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9130004830 tx=0x7f9130007d90 comp rx=0 tx=0).stop 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.524+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134141e70 msgr2=0x7f9134135c00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.524+0000 7f912a7fc640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134141e70 0x7f9134135c00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.529+0000 7f913c64f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.529+0000 7f913c64f640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134136140 0x7f9134136540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.529+0000 7f913c64f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.529+0000 7f913c64f640 1 -- 172.21.15.178:0/4030693293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9134070c80 con 0x7f9134136140 2024-09-15T01:32:36.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f913a3eb640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134136140 0x7f9134136540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f913a3eb640 10 monclient: get_auth_request con 0x7f9134136140 auth_method 0 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f913a3eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f913a3eb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f913a3eb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f913a3eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f913a3eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f913a3eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f913a3eb640 10 monclient(hunting): handle_auth_done global_id 4513 payload 824 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f913a3eb640 1 -- 172.21.15.178:0/4030693293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f913404bc80 con 0x7f9134136140 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f913a3eb640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134136140 0x7f9134136540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9130014000 tx=0x7f9130010000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9130012df0 con 0x7f9134136140 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f912a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f912a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f912a7fc640 10 monclient: dump: 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.557 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9119 (secure 0 0 0) 0x7f913001e360 con 0x7f9134136140 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f912a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f912a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134136140 msgr2=0x7f9134136540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f912a7fc640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134136140 0x7f9134136540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9130014000 tx=0x7f9130010000 comp rx=0 tx=0).stop 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f912a7fc640 1 -- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134136140 msgr2=0x7f9134136540 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:36.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.530+0000 7f912a7fc640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9134136140 0x7f9134136540 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9130014000 tx=0x7f9130010000 comp rx=0 tx=0).stop 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 20, 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:36.559 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:36.560 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:36.561 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:36.562 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:36.563 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:36.564 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:36.565 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.567 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.568 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.569 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:36.570 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:36.571 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.575 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:36.577 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.579 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:36.580 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:36.581 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.531+0000 7f913c64f640 1 -- 172.21.15.178:0/4030693293 >> v2:172.21.15.204:6812/28210 conn(0x7f91180397b0 msgr2=0x7f911803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.531+0000 7f913c64f640 1 --2- 172.21.15.178:0/4030693293 >> v2:172.21.15.204:6812/28210 conn(0x7f91180397b0 0x7f911803bc60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91340a6ec0 tx=0x7f91240090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.531+0000 7f913c64f640 10 monclient: shutdown 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.531+0000 7f913c64f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.531+0000 7f913c64f640 1 -- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f91341380e0 msgr2=0x7f913409c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.531+0000 7f913c64f640 1 --2- 172.21.15.178:0/4030693293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f91341380e0 0x7f913409c750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f912c0099f0 tx=0x7f912c037000 comp rx=0 tx=0).stop 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.531+0000 7f913abec640 1 -- 172.21.15.178:0/4030693293 reap_dead start 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.531+0000 7f913abec640 1 -- 172.21.15.178:0/4030693293 reap_dead start 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.531+0000 7f913c64f640 1 -- 172.21.15.178:0/4030693293 shutdown_connections 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.531+0000 7f913c64f640 1 -- 172.21.15.178:0/4030693293 >> 172.21.15.178:0/4030693293 conn(0x7f91340a5ed0 msgr2=0x7f91340a4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.531+0000 7f913c64f640 1 -- 172.21.15.178:0/4030693293 shutdown_connections 2024-09-15T01:32:36.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.531+0000 7f913c64f640 1 -- 172.21.15.178:0/4030693293 wait complete. 2024-09-15T01:32:36.584 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:32:36.584 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-09-15T01:32:36.584 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:32:36.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.676+0000 7f8b24494640 10 monclient: get_monmap_and_config 2024-09-15T01:32:36.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.676+0000 7f8b24494640 10 monclient: build_initial_monmap 2024-09-15T01:32:36.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.676+0000 7f8b24494640 10 monclient: monmap: 2024-09-15T01:32:36.678 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:36.678 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.678 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:36.677436+0000 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:36.677436+0000 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:36.679 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 1 Processor -- start 2024-09-15T01:32:36.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 1 -- start start 2024-09-15T01:32:36.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 10 monclient: init 2024-09-15T01:32:36.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:36.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 10 monclient: _add_conns ranks=[1,7,0,9,2,4,8,6,10,3,5] 2024-09-15T01:32:36.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8b1c0a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:36.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b1c132de0 0x7f8b1c137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 10 monclient(hunting): picked mon.noname-i con 0x7f8b1c132de0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8b1c1380e0 addr v1:172.21.15.178:6789/0 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1c04bf60 con 0x7f8b1c1380e0 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1c09dd80 con 0x7f8b1c0a2930 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c0a2d00 con 0x7f8b1c132de0 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b24494640 10 monclient(hunting): authenticate will time out at 761.427817s 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b22a31640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f8b1c1380e0 0x7f8b1c13a4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:53346/0 (socket says 172.21.15.178:53346) 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.677+0000 7f8b22a31640 1 -- 172.21.15.178:0/1608601692 learned_addr learned my addr 172.21.15.178:0/1608601692 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 1 --2- 172.21.15.178:0/1608601692 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b1c132de0 0x7f8b1c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 10 monclient(hunting): get_auth_request con 0x7f8b1c132de0 auth_method 0 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b2122e640 1 -- 172.21.15.178:0/1608601692 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922847239 0 0) 0x7f8b1c04bf60 con 0x7f8b1c1380e0 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b2122e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b2122e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b2122e640 10 monclient(hunting): my global_id is 4608 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b2122e640 1 -- 172.21.15.178:0/1608601692 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b080034e0 con 0x7f8b1c1380e0 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b2122e640 1 -- 172.21.15.178:0/1608601692 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181886626 0 0) 0x7f8b1c09dd80 con 0x7f8b1c0a2930 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b2122e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b2122e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b2122e640 10 monclient(hunting): my global_id is 4591 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b2122e640 1 -- 172.21.15.178:0/1608601692 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b1c04bf60 con 0x7f8b1c0a2930 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 10 monclient(hunting): handle_auth_done global_id 4480 payload 274 2024-09-15T01:32:36.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 1 -- 172.21.15.178:0/1608601692 >> v1:172.21.15.204:6789/0 conn(0x7f8b1c0a2930 legacy=0x7f8b1c09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 1 -- 172.21.15.178:0/1608601692 >> v1:172.21.15.178:6789/0 conn(0x7f8b1c1380e0 legacy=0x7f8b1c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 10 monclient: _finish_hunting 0 2024-09-15T01:32:36.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 1 monclient: found mon.noname-i 2024-09-15T01:32:36.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:36.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:32:36.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.678+0000 7f8b21a2f640 1 -- 172.21.15.178:0/1608601692 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c1331b0 con 0x7f8b1c132de0 2024-09-15T01:32:36.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b21a2f640 1 --2- 172.21.15.178:0/1608601692 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b1c132de0 0x7f8b1c137ba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c09df10 tx=0x7f8b0c0090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=2e1c74d0e4016a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b2122e640 1 -- 172.21.15.178:0/1608601692 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b0c02acd0 con 0x7f8b1c132de0 2024-09-15T01:32:36.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b2122e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b2122e640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:32:36.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b2122e640 10 monclient: dump: 2024-09-15T01:32:36.694 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.694 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.694 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.694 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.694 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.694 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.694 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.694 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.694 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.694 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.694 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.695 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.695 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.695 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.695 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.695 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b2122e640 10 monclient: _finish_auth 0 2024-09-15T01:32:36.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b2122e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:36.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 5 monclient: authenticate success, global_id 4480 2024-09-15T01:32:36.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b2122e640 1 -- 172.21.15.178:0/1608601692 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c02ae50 con 0x7f8b1c132de0 2024-09-15T01:32:36.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:36.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b2122e640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b2122e640 1 -- 172.21.15.178:0/1608601692 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b0c031ca0 con 0x7f8b1c132de0 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b2122e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 4 set_mon_vals no callback set 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b2122e640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b2122e640 10 monclient: dump: 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 10 monclient: shutdown 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 1 -- 172.21.15.178:0/1608601692 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b1c132de0 msgr2=0x7f8b1c137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 1 --2- 172.21.15.178:0/1608601692 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b1c132de0 0x7f8b1c137ba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c09df10 tx=0x7f8b0c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 1 -- 172.21.15.178:0/1608601692 shutdown_connections 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 1 --2- 172.21.15.178:0/1608601692 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b1c132de0 0x7f8b1c137ba0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 1 -- 172.21.15.178:0/1608601692 >> 172.21.15.178:0/1608601692 conn(0x7f8b1c0a5ed0 msgr2=0x7f8b1c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 1 -- 172.21.15.178:0/1608601692 shutdown_connections 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.679+0000 7f8b24494640 1 -- 172.21.15.178:0/1608601692 wait complete. 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.680+0000 7f8b24494640 10 monclient: build_initial_monmap 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.680+0000 7f8b24494640 10 monclient: monmap: 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.680+0000 7f8b24494640 1 Processor -- start 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.680+0000 7f8b24494640 1 -- start start 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.680+0000 7f8b24494640 10 monclient: init 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:36.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 10 monclient: _add_conns ranks=[4,8,6,7,2,1,3,5,0] 2024-09-15T01:32:36.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8b1c0a2930 0x7f8b1c09aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 10 monclient(hunting): picked mon.noname-c con 0x7f8b1c0a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:36.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c132de0 0x7f8b1c09afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 10 monclient(hunting): picked mon.noname-e con 0x7f8b1c132de0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:36.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b1c1380e0 0x7f8b1c09b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 10 monclient(hunting): picked mon.noname-d con 0x7f8b1c1380e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c0a2390 con 0x7f8b1c0a2930 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c09d630 con 0x7f8b1c1380e0 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c11db40 con 0x7f8b1c132de0 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 10 monclient(hunting): authenticate will time out at 761.431175s 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22a31640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b1c1380e0 msgr2=0x7f8b1c09b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22a31640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b1c1380e0 0x7f8b1c09b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22230640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8b1c0a2930 0x7f8b1c09aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c132de0 0x7f8b1c09afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22230640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8b1c0a2930 0x7f8b1c09aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:55946/0 (socket says 172.21.15.178:55946) 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c132de0 0x7f8b1c09afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42502/0 (socket says 172.21.15.178:42502) 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22230640 1 -- 172.21.15.178:0/3895557775 learned_addr learned my addr 172.21.15.178:0/3895557775 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22230640 10 monclient(hunting): get_auth_request con 0x7f8b1c0a2930 auth_method 0 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22230640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22230640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22230640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 10 monclient(hunting): get_auth_request con 0x7f8b1c132de0 auth_method 0 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22230640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22230640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22230640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 10 monclient(hunting): handle_auth_done global_id 4535 payload 824 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 1 -- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b1c1380e0 msgr2=0x7f8b1c09b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b1c1380e0 0x7f8b1c09b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 1 -- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8b1c0a2930 msgr2=0x7f8b1c09aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8b1c0a2930 0x7f8b1c09aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 10 monclient: _finish_hunting 0 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 1 monclient: found mon.noname-e 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 1 -- 172.21.15.178:0/3895557775 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c009390 con 0x7f8b1c132de0 2024-09-15T01:32:36.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b22230640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8b1c0a2930 0x7f8b1c09aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:32:36.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b21a2f640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c132de0 0x7f8b1c09afd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0a3d10 tx=0x7f8b0c032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b0c031820 con 0x7f8b1c132de0 2024-09-15T01:32:36.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b127fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:36.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b127fc640 10 monclient: dump: 2024-09-15T01:32:36.705 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.705 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.705 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.705 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.705 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.705 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.706 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.706 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.706 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.706 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.706 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.706 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.707 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b127fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 5 monclient: authenticate success, global_id 4535 2024-09-15T01:32:36.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c031980 con 0x7f8b1c132de0 2024-09-15T01:32:36.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b127fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:36.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 10 monclient: _renew_subs 2024-09-15T01:32:36.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b0c03a390 con 0x7f8b1c132de0 2024-09-15T01:32:36.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b24494640 1 -- 172.21.15.178:0/3895557775 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1c132930 con 0x7f8b1c132de0 2024-09-15T01:32:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b127fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b20a2d640 4 set_mon_vals no callback set 2024-09-15T01:32:36.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.681+0000 7f8b127fc640 10 monclient: dump: 2024-09-15T01:32:36.710 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.710 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.710 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.710 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.710 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.711 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.711 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.711 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.711 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.711 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.711 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.712 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.712 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.712 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.712 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.712 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.682+0000 7f8b24494640 10 monclient: _renew_subs 2024-09-15T01:32:36.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.682+0000 7f8b24494640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:36.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.682+0000 7f8b24494640 1 -- 172.21.15.178:0/3895557775 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1c09c4c0 con 0x7f8b1c132de0 2024-09-15T01:32:36.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.682+0000 7f8b24494640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:36.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.682+0000 7f8b24494640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c09ba50 0x7f8b1c096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.682+0000 7f8b24494640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.682+0000 7f8b24494640 1 -- 172.21.15.178:0/3895557775 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c04c4a0 con 0x7f8b1c09ba50 2024-09-15T01:32:36.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.682+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8b0c039040 con 0x7f8b1c132de0 2024-09-15T01:32:36.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.682+0000 7f8b22230640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c09ba50 0x7f8b1c096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22230640 10 monclient: get_auth_request con 0x7f8b1c09ba50 auth_method 0 2024-09-15T01:32:36.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22230640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22230640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22230640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22230640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22230640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22230640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 1 --2- 172.21.15.178:0/3895557775 >> v2:172.21.15.204:6812/28210 conn(0x7f8b000397b0 0x7f8b0003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f8b0c06a2a0 con 0x7f8b1c132de0 2024-09-15T01:32:36.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22230640 10 monclient(hunting): handle_auth_done global_id 4544 payload 824 2024-09-15T01:32:36.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22230640 1 -- 172.21.15.178:0/3895557775 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b1c05ed70 con 0x7f8b1c09ba50 2024-09-15T01:32:36.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22a31640 1 --2- 172.21.15.178:0/3895557775 >> v2:172.21.15.204:6812/28210 conn(0x7f8b000397b0 0x7f8b0003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22a31640 10 monclient: get_auth_request con 0x7f8b000397b0 auth_method 0 2024-09-15T01:32:36.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22230640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c09ba50 0x7f8b1c096a30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800fb00 tx=0x7f8b18007d30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b18014470 con 0x7f8b1c09ba50 2024-09-15T01:32:36.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:36.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 10 monclient: dump: 2024-09-15T01:32:36.718 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.718 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.718 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.718 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.718 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.718 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.719 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.719 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.719 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.719 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.719 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.720 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.720 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.720 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.720 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.720 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f8b1800db10 con 0x7f8b1c09ba50 2024-09-15T01:32:36.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:36.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:36.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c09ba50 msgr2=0x7f8b1c096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c09ba50 0x7f8b1c096a30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800fb00 tx=0x7f8b18007d30 comp rx=0 tx=0).stop 2024-09-15T01:32:36.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c09ba50 msgr2=0x7f8b1c096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:36.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b22a31640 1 --2- 172.21.15.178:0/3895557775 >> v2:172.21.15.204:6812/28210 conn(0x7f8b000397b0 0x7f8b0003bc60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b14001460 tx=0x7f8b140016e0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.683+0000 7f8b127fc640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c09ba50 0x7f8b1c096a30 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800fb00 tx=0x7f8b18007d30 comp rx=0 tx=0).stop 2024-09-15T01:32:36.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b24494640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:36.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b24494640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c096f70 0x7f8b1c097370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b24494640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b24494640 1 -- 172.21.15.178:0/3895557775 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c04bc80 con 0x7f8b1c096f70 2024-09-15T01:32:36.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b22230640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c096f70 0x7f8b1c097370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b22230640 10 monclient: get_auth_request con 0x7f8b1c096f70 auth_method 0 2024-09-15T01:32:36.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b22230640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b22230640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b22230640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b22230640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b22230640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b22230640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b22230640 10 monclient(hunting): handle_auth_done global_id 4553 payload 824 2024-09-15T01:32:36.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b22230640 1 -- 172.21.15.178:0/3895557775 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8b1c04b720 con 0x7f8b1c096f70 2024-09-15T01:32:36.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.689+0000 7f8b22230640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c096f70 0x7f8b1c097370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800fb00 tx=0x7f8b1801a000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:36.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b1801ec10 con 0x7f8b1c096f70 2024-09-15T01:32:36.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:36.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b127fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:36.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b127fc640 10 monclient: dump: 2024-09-15T01:32:36.729 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:36.729 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.729 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.729 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:36.730 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:36.730 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.730 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:36.730 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:36.730 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:36.731 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:36.731 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:36.731 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:36.731 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:36.731 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:36.731 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:36.732 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9119 (secure 0 0 0) 0x7f8b1801ed70 con 0x7f8b1c096f70 2024-09-15T01:32:36.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b127fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:36.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b127fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:36.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c096f70 msgr2=0x7f8b1c097370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b127fc640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c096f70 0x7f8b1c097370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800fb00 tx=0x7f8b1801a000 comp rx=0 tx=0).stop 2024-09-15T01:32:36.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b127fc640 1 -- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c096f70 msgr2=0x7f8b1c097370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:36.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b127fc640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c096f70 0x7f8b1c097370 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800fb00 tx=0x7f8b1801a000 comp rx=0 tx=0).stop 2024-09-15T01:32:36.733 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:36.734 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:36.734 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:36.734 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:36.734 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 20, 2024-09-15T01:32:36.734 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:36.734 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:36.735 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:36.735 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:36.735 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:36.735 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:36.735 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:36.736 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:36.736 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.736 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:32:36.736 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:36.736 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:36.736 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:36.737 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:36.737 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:36.737 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:36.737 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:36.738 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:36.738 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:36.738 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:36.738 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:36.738 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:36.739 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.739 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:36.739 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:36.739 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:36.739 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:36.739 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:36.740 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:36.740 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:36.740 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:36.740 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:36.740 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:36.741 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:36.741 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.741 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.741 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:36.741 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:36.741 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:36.742 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:36.742 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:36.742 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:36.742 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:36.742 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:36.742 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:36.743 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:36.743 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:36.743 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:36.743 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:36.743 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:36.744 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:36.744 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:36.744 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:36.744 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:36.744 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:36.745 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:36.745 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:36.745 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:36.745 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:36.745 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:36.745 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:36.746 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:36.746 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.746 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:36.746 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.746 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:36.746 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.747 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:36.747 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:36.747 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.747 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.747 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.748 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.748 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:36.748 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.748 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.748 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.749 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.749 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:36.749 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.749 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.749 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.750 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:36.750 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:36.750 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.750 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.750 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.750 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.751 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.751 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:36.751 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:36.751 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.751 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.751 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.752 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.752 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:36.752 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.752 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.752 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.753 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:36.753 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.753 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.753 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.753 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.754 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.755 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.756 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.757 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.758 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.759 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.760 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.761 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:36.762 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.764 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:36.766 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:36.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:36.767 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:36.767 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:36.767 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:32:36.767 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:36.767 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:36.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:36.767 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:36.767 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:36.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b24494640 1 -- 172.21.15.178:0/3895557775 >> v2:172.21.15.204:6812/28210 conn(0x7f8b000397b0 msgr2=0x7f8b0003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b24494640 1 --2- 172.21.15.178:0/3895557775 >> v2:172.21.15.204:6812/28210 conn(0x7f8b000397b0 0x7f8b0003bc60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b14001460 tx=0x7f8b140016e0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b24494640 10 monclient: shutdown 2024-09-15T01:32:36.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b24494640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:36.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b24494640 1 -- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c132de0 msgr2=0x7f8b1c09afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:36.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b24494640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c132de0 0x7f8b1c09afd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0a3d10 tx=0x7f8b0c032000 comp rx=0 tx=0).stop 2024-09-15T01:32:36.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b22a31640 1 -- 172.21.15.178:0/3895557775 reap_dead start 2024-09-15T01:32:36.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b24494640 1 -- 172.21.15.178:0/3895557775 shutdown_connections 2024-09-15T01:32:36.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b24494640 1 --2- 172.21.15.178:0/3895557775 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b1c132de0 0x7f8b1c09afd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b24494640 1 -- 172.21.15.178:0/3895557775 >> 172.21.15.178:0/3895557775 conn(0x7f8b1c0a5ed0 msgr2=0x7f8b1c135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:36.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b24494640 1 -- 172.21.15.178:0/3895557775 shutdown_connections 2024-09-15T01:32:36.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.690+0000 7f8b24494640 1 -- 172.21.15.178:0/3895557775 wait complete. 2024-09-15T01:32:36.770 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:32:36.770 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-09-15T01:32:36.770 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:32:36.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.862+0000 7fd606100640 10 monclient: get_monmap_and_config 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.862+0000 7fd606100640 10 monclient: build_initial_monmap 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.862+0000 7fd606100640 10 monclient: monmap: 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:36.863757+0000 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:36.863757+0000 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:36.865 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:36.866 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:36.866 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:36.866 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:36.866 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:36.866 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:36.866 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:36.866 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:36.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.863+0000 7fd606100640 1 Processor -- start 2024-09-15T01:32:36.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.863+0000 7fd606100640 1 -- start start 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.863+0000 7fd606100640 10 monclient: init 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 10 monclient: _add_conns ranks=[5,7,9,10,3,0,6,8,4,1,2] 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6000a2930 0x7fd600132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 10 monclient(hunting): picked mon.noname-h con 0x7fd6000a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd60009faa0 0x7fd60009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 10 monclient(hunting): picked mon.noname-i con 0x7fd60009faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd6000a08c0 0x7fd6000a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 10 monclient(hunting): picked mon.noname-a con 0x7fd6000a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd60009e0f0 con 0x7fd6000a08c0 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd60009dd80 con 0x7fd6000a2930 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000a2d00 con 0x7fd60009faa0 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd606100640 10 monclient(hunting): authenticate will time out at 761.614188s 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd6000a08c0 0x7fd6000a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd6000a08c0 0x7fd6000a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44456/0 (socket says 172.21.15.178:44456) 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 1 -- 172.21.15.178:0/2262655063 learned_addr learned my addr 172.21.15.178:0/2262655063 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 10 monclient(hunting): get_auth_request con 0x7fd6000a08c0 auth_method 0 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5ff7fe640 1 --2- 172.21.15.178:0/2262655063 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6000a2930 0x7fd600132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5feffd640 1 --2- 172.21.15.178:0/2262655063 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd60009faa0 0x7fd60009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:36.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5ff7fe640 10 monclient(hunting): get_auth_request con 0x7fd6000a2930 auth_method 0 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5feffd640 10 monclient(hunting): get_auth_request con 0x7fd60009faa0 auth_method 0 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5feffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 10 monclient(hunting): handle_auth_done global_id 4617 payload 274 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 1 -- 172.21.15.178:0/2262655063 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd60009faa0 msgr2=0x7fd60009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 1 --2- 172.21.15.178:0/2262655063 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd60009faa0 0x7fd60009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 1 -- 172.21.15.178:0/2262655063 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6000a2930 msgr2=0x7fd600132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 1 --2- 172.21.15.178:0/2262655063 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6000a2930 0x7fd600132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 10 monclient: _finish_hunting 0 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 1 monclient: found mon.noname-a 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.864+0000 7fd5fffff640 1 -- 172.21.15.178:0/2262655063 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60004bf60 con 0x7fd6000a08c0 2024-09-15T01:32:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.865+0000 7fd5fffff640 1 -- 172.21.15.178:0/2262655063 >> v2:172.21.15.178:3300/0 conn(0x7fd6000a08c0 msgr2=0x7fd6000a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:32:36.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.865+0000 7fd5fffff640 1 -- 172.21.15.178:0/2262655063 >> v2:172.21.15.178:3300/0 conn(0x7fd6000a08c0 msgr2=0x7fd6000a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:32:36.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.865+0000 7fd5fffff640 1 --2- 172.21.15.178:0/2262655063 >> v2:172.21.15.178:3300/0 conn(0x7fd6000a08c0 0x7fd6000a0d30 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7fd5f8009910 tx=0x7fd5f80091d0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:32:36.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:36.865+0000 7fd5fffff640 1 --2- 172.21.15.178:0/2262655063 >> v2:172.21.15.178:3300/0 conn(0x7fd6000a08c0 0x7fd6000a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:37.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fffff640 1 --2- 172.21.15.178:0/2262655063 >> v2:172.21.15.178:3300/0 conn(0x7fd6000a08c0 0x7fd6000a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fffff640 10 monclient: get_auth_request con 0x7fd6000a08c0 auth_method 0 2024-09-15T01:32:37.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fffff640 0 --2- 172.21.15.178:0/2262655063 >> v2:172.21.15.178:3300/0 conn(0x7fd6000a08c0 0x7fd6000a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T01:32:37.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fffff640 1 --2- 172.21.15.178:0/2262655063 >> v2:172.21.15.178:3300/0 conn(0x7fd6000a08c0 0x7fd6000a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 10 monclient: ms_handle_reset current mon v2:172.21.15.178:3300/0 2024-09-15T01:32:37.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 1 -- 172.21.15.178:0/2262655063 >> v2:172.21.15.178:3300/0 conn(0x7fd6000a08c0 msgr2=0x7fd6000a0d30 unknown :-1 s=STATE_CLOSED l=0).mark_down 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 1 --2- 172.21.15.178:0/2262655063 >> v2:172.21.15.178:3300/0 conn(0x7fd6000a08c0 0x7fd6000a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 10 monclient: _add_conns ranks=[2,3,10,8,4,6,0,1] 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 1 --2- 172.21.15.178:0/2262655063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5e80035f0 0x7fd5e8005a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 10 monclient(hunting): picked mon.noname-b con 0x7fd5e80035f0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 1 --2- 172.21.15.178:0/2262655063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5e8006340 0x7fd5e80087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 10 monclient(hunting): picked mon.noname-g con 0x7fd5e8006340 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 1 --2- 172.21.15.178:0/2262655063 >> v2:172.21.15.204:3300/0 conn(0x7fd5e8009110 0x7fd5e800b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 10 monclient(hunting): picked mon.noname-f con 0x7fd5e8009110 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 1 -- 172.21.15.178:0/2262655063 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e800bed0 con 0x7fd5e8009110 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 1 -- 172.21.15.178:0/2262655063 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e800c010 con 0x7fd5e80035f0 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 1 -- 172.21.15.178:0/2262655063 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e800c1d0 con 0x7fd5e8006340 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 10 monclient(hunting): ms_handle_reset stray mon v2:172.21.15.178:3300/0 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5feffd640 1 -- 172.21.15.178:0/2262655063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5e8006340 msgr2=0x7fd5e80087c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5feffd640 1 --2- 172.21.15.178:0/2262655063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5e8006340 0x7fd5e80087c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 1 --2- 172.21.15.178:0/2262655063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5e80035f0 0x7fd5e8005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 10 monclient(hunting): get_auth_request con 0x7fd5e80035f0 auth_method 0 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fffff640 1 --2- 172.21.15.178:0/2262655063 >> v2:172.21.15.204:3300/0 conn(0x7fd5e8009110 0x7fd5e800b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fffff640 10 monclient(hunting): get_auth_request con 0x7fd5e8009110 auth_method 0 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_done global_id 4617 payload 293 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 1 -- 172.21.15.178:0/2262655063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5e8006340 msgr2=0x7fd5e80087c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 1 --2- 172.21.15.178:0/2262655063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5e8006340 0x7fd5e80087c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 1 -- 172.21.15.178:0/2262655063 >> v2:172.21.15.204:3300/0 conn(0x7fd5e8009110 msgr2=0x7fd5e800b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 1 --2- 172.21.15.178:0/2262655063 >> v2:172.21.15.204:3300/0 conn(0x7fd5e8009110 0x7fd5e800b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 1 monclient: found mon.noname-b 2024-09-15T01:32:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fffff640 1 -- 172.21.15.178:0/2262655063 reap_dead start 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 1 -- 172.21.15.178:0/2262655063 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e800c450 con 0x7fd5e80035f0 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5ff7fe640 1 --2- 172.21.15.178:0/2262655063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5e80035f0 0x7fd5e8005a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00094d0 tx=0x7fd5f000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da7746822f9299ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 1 -- 172.21.15.178:0/2262655063 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5f000ed30 con 0x7fd5e80035f0 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.066+0000 7fd5fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd5fe7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd5fe7fc640 10 monclient: dump: 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd5fe7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd5fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd5fe7fc640 1 -- 172.21.15.178:0/2262655063 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f000f040 con 0x7fd5e80035f0 2024-09-15T01:32:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 5 monclient: authenticate success, global_id 4617 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd5fe7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 4 set_mon_vals no callback set 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd5fe7fc640 1 -- 172.21.15.178:0/2262655063 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5f0014d80 con 0x7fd5e80035f0 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd5fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd5fe7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd5fe7fc640 10 monclient: dump: 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 10 monclient: shutdown 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 1 -- 172.21.15.178:0/2262655063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5e80035f0 msgr2=0x7fd5e8005a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 1 --2- 172.21.15.178:0/2262655063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5e80035f0 0x7fd5e8005a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00094d0 tx=0x7fd5f000b040 comp rx=0 tx=0).stop 2024-09-15T01:32:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 1 -- 172.21.15.178:0/2262655063 shutdown_connections 2024-09-15T01:32:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 1 --2- 172.21.15.178:0/2262655063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5e80035f0 0x7fd5e8005a30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 1 -- 172.21.15.178:0/2262655063 >> 172.21.15.178:0/2262655063 conn(0x7fd6000a5ed0 msgr2=0x7fd6000a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 1 -- 172.21.15.178:0/2262655063 shutdown_connections 2024-09-15T01:32:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.067+0000 7fd606100640 1 -- 172.21.15.178:0/2262655063 wait complete. 2024-09-15T01:32:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.068+0000 7fd606100640 10 monclient: build_initial_monmap 2024-09-15T01:32:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.068+0000 7fd606100640 10 monclient: monmap: 2024-09-15T01:32:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:37.086 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.068+0000 7fd606100640 1 Processor -- start 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.068+0000 7fd606100640 1 -- start start 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.068+0000 7fd606100640 10 monclient: init 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 10 monclient: _add_conns ranks=[5,8,7,1,6,0,4,2,3] 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5e80035f0 0x7fd60009b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 10 monclient(hunting): picked mon.noname-h con 0x7fd5e80035f0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd60011ca40 0x7fd600131970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 10 monclient(hunting): picked mon.noname-e con 0x7fd60011ca40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd600131eb0 0x7fd600141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 10 monclient(hunting): picked mon.noname-i con 0x7fd600131eb0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd60009be10 con 0x7fd5e80035f0 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000a2390 con 0x7fd600131eb0 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd60009d630 con 0x7fd60011ca40 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd606100640 10 monclient(hunting): authenticate will time out at 761.819280s 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd60011ca40 0x7fd600131970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5ff7fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5e80035f0 0x7fd60009b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd60011ca40 0x7fd600131970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42534/0 (socket says 172.21.15.178:42534) 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5ff7fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5e80035f0 0x7fd60009b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:58524/0 (socket says 172.21.15.178:58524) 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 1 -- 172.21.15.178:0/2825795771 learned_addr learned my addr 172.21.15.178:0/2825795771 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 10 monclient(hunting): get_auth_request con 0x7fd60011ca40 auth_method 0 2024-09-15T01:32:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5fffff640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd600131eb0 0x7fd600141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5ff7fe640 10 monclient(hunting): get_auth_request con 0x7fd5e80035f0 auth_method 0 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5fffff640 10 monclient(hunting): get_auth_request con 0x7fd600131eb0 auth_method 0 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5fffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 10 monclient(hunting): handle_auth_done global_id 4562 payload 824 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 1 -- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd600131eb0 msgr2=0x7fd600141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd600131eb0 0x7fd600141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 1 -- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5e80035f0 msgr2=0x7fd60009b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5e80035f0 0x7fd60009b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 10 monclient: _finish_hunting 0 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 1 monclient: found mon.noname-e 2024-09-15T01:32:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 1 -- 172.21.15.178:0/2825795771 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60011ea90 con 0x7fd60011ca40 2024-09-15T01:32:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5feffd640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd60011ca40 0x7fd600131970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400ea80 tx=0x7fd5f40090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.069+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5f400ce10 con 0x7fd60011ca40 2024-09-15T01:32:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5e77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5e77fe640 10 monclient: dump: 2024-09-15T01:32:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5e77fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f4013470 con 0x7fd60011ca40 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5e77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5f4013a60 con 0x7fd60011ca40 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd606100640 5 monclient: authenticate success, global_id 4562 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd606100640 10 monclient: _renew_subs 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd606100640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd606100640 1 -- 172.21.15.178:0/2825795771 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60011ed70 con 0x7fd60011ca40 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5e77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5e77fe640 10 monclient: dump: 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5fdffb640 4 set_mon_vals no callback set 2024-09-15T01:32:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd606100640 10 monclient: _renew_subs 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd606100640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd606100640 1 -- 172.21.15.178:0/2825795771 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60011d680 con 0x7fd60011ca40 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.070+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd5f4013bc0 con 0x7fd60011ca40 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd606100640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd606100640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600142310 0x7fd6001426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd606100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd606100640 1 -- 172.21.15.178:0/2825795771 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60011ef00 con 0x7fd600142310 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5e77fe640 1 --2- 172.21.15.178:0/2825795771 >> v2:172.21.15.204:6812/28210 conn(0x7fd5ec0396e0 0x7fd5ec03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7fd5f404aec0 con 0x7fd60011ca40 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5ff7fe640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600142310 0x7fd6001426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5ff7fe640 10 monclient: get_auth_request con 0x7fd600142310 auth_method 0 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5fffff640 1 --2- 172.21.15.178:0/2825795771 >> v2:172.21.15.204:6812/28210 conn(0x7fd5ec0396e0 0x7fd5ec03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5fffff640 10 monclient: get_auth_request con 0x7fd5ec0396e0 auth_method 0 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_done global_id 4600 payload 824 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5ff7fe640 1 -- 172.21.15.178:0/2825795771 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd60004c4a0 con 0x7fd600142310 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5ff7fe640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600142310 0x7fd6001426e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8003400 tx=0x7fd5f001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5fffff640 1 --2- 172.21.15.178:0/2825795771 >> v2:172.21.15.204:6812/28210 conn(0x7fd5ec0396e0 0x7fd5ec03bb90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd600133890 tx=0x7fd6000a3cd0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5f000e050 con 0x7fd600142310 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5e77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:32:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.071+0000 7fd5e77fe640 10 monclient: dump: 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.098 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.072+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fd5f0029460 con 0x7fd600142310 2024-09-15T01:32:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.072+0000 7fd5e77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.072+0000 7fd5e77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.072+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600142310 msgr2=0x7fd6001426e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.072+0000 7fd5e77fe640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600142310 0x7fd6001426e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8003400 tx=0x7fd5f001a000 comp rx=0 tx=0).stop 2024-09-15T01:32:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.072+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600142310 msgr2=0x7fd6001426e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.072+0000 7fd5e77fe640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600142310 0x7fd6001426e0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8003400 tx=0x7fd5f001a000 comp rx=0 tx=0).stop 2024-09-15T01:32:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd606100640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd606100640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600096a30 0x7fd600096e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd606100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd606100640 1 -- 172.21.15.178:0/2825795771 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd600132280 con 0x7fd600096a30 2024-09-15T01:32:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5ff7fe640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600096a30 0x7fd600096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5ff7fe640 10 monclient: get_auth_request con 0x7fd600096a30 auth_method 0 2024-09-15T01:32:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5ff7fe640 10 monclient(hunting): handle_auth_done global_id 4609 payload 824 2024-09-15T01:32:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5ff7fe640 1 -- 172.21.15.178:0/2825795771 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd60004bc80 con 0x7fd600096a30 2024-09-15T01:32:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5ff7fe640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600096a30 0x7fd600096e30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd5f001f000 tx=0x7fd5f0024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5f0024070 con 0x7fd600096a30 2024-09-15T01:32:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5e77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:32:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.078+0000 7fd5e77fe640 10 monclient: dump: 2024-09-15T01:32:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9289 (secure 0 0 0) 0x7fd5f0009ac0 con 0x7fd600096a30 2024-09-15T01:32:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd5e77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd5e77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600096a30 msgr2=0x7fd600096e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd5e77fe640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600096a30 0x7fd600096e30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd5f001f000 tx=0x7fd5f0024000 comp rx=0 tx=0).stop 2024-09-15T01:32:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd5e77fe640 1 -- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600096a30 msgr2=0x7fd600096e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd5e77fe640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd600096a30 0x7fd600096e30 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd5f001f000 tx=0x7fd5f0024000 comp rx=0 tx=0).stop 2024-09-15T01:32:37.103 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:37.103 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:37.103 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:37.103 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 20, 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:37.104 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:37.105 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:37.106 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:37.107 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:37.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.109 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.110 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.111 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.113 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.114 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.116 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.117 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:37.118 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:37.120 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.122 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:37.123 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:37.124 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:37.125 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:37.125 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.125 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.125 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:37.125 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:37.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd606100640 1 -- 172.21.15.178:0/2825795771 >> v2:172.21.15.204:6812/28210 conn(0x7fd5ec0396e0 msgr2=0x7fd5ec03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd606100640 1 --2- 172.21.15.178:0/2825795771 >> v2:172.21.15.204:6812/28210 conn(0x7fd5ec0396e0 0x7fd5ec03bb90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd600133890 tx=0x7fd6000a3cd0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd606100640 10 monclient: shutdown 2024-09-15T01:32:37.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd606100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:37.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd606100640 1 -- 172.21.15.178:0/2825795771 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd60011ca40 msgr2=0x7fd600131970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd606100640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd60011ca40 0x7fd600131970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400ea80 tx=0x7fd5f40090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd5fffff640 1 -- 172.21.15.178:0/2825795771 reap_dead start 2024-09-15T01:32:37.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd606100640 1 -- 172.21.15.178:0/2825795771 shutdown_connections 2024-09-15T01:32:37.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd606100640 1 --2- 172.21.15.178:0/2825795771 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd60011ca40 0x7fd600131970 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd606100640 1 -- 172.21.15.178:0/2825795771 >> 172.21.15.178:0/2825795771 conn(0x7fd6000a5ed0 msgr2=0x7fd60009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:37.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd606100640 1 -- 172.21.15.178:0/2825795771 shutdown_connections 2024-09-15T01:32:37.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.079+0000 7fd606100640 1 -- 172.21.15.178:0/2825795771 wait complete. 2024-09-15T01:32:37.127 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:32:37.127 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-15T01:32:37.127 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:32:37.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.220+0000 7ffa31ed4640 10 monclient: get_monmap_and_config 2024-09-15T01:32:37.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.220+0000 7ffa31ed4640 10 monclient: build_initial_monmap 2024-09-15T01:32:37.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.220+0000 7ffa31ed4640 10 monclient: monmap: 2024-09-15T01:32:37.222 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:37.222 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.222 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:37.221350+0000 2024-09-15T01:32:37.222 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:37.221350+0000 2024-09-15T01:32:37.222 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:37.222 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.223 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:37.223 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:37.223 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:37.223 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:37.223 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:37.223 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:37.223 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:37.223 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:37.223 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:37.223 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:37.223 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:37.223 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 1 Processor -- start 2024-09-15T01:32:37.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 1 -- start start 2024-09-15T01:32:37.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 10 monclient: init 2024-09-15T01:32:37.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:37.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 10 monclient: _add_conns ranks=[8,1,10,3,6,2,0,7,5,4,9] 2024-09-15T01:32:37.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 0x7ffa2c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 10 monclient(hunting): picked mon.noname-e con 0x7ffa2c0a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:37.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ffa2c09faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:32:37.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffa2c09fe70 0x7ffa2c0a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 10 monclient(hunting): picked mon.noname-f con 0x7ffa2c09fe70 addr v2:172.21.15.204:3300/0 2024-09-15T01:32:37.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa2c04bf60 con 0x7ffa2c09faa0 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa2c09dd80 con 0x7ffa2c09fe70 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c0a2d00 con 0x7ffa2c0a2930 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa31ed4640 10 monclient(hunting): authenticate will time out at 761.971858s 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 0x7ffa2c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 0x7ffa2c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42548/0 (socket says 172.21.15.178:42548) 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa2b7fe640 1 -- 172.21.15.178:0/105934813 learned_addr learned my addr 172.21.15.178:0/105934813 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa2b7fe640 10 monclient(hunting): get_auth_request con 0x7ffa2c0a2930 auth_method 0 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa2b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.221+0000 7ffa2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2bfff640 1 --2- 172.21.15.178:0/105934813 >> v2:172.21.15.204:3300/0 conn(0x7ffa2c09fe70 0x7ffa2c0a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2bfff640 10 monclient(hunting): get_auth_request con 0x7ffa2c09fe70 auth_method 0 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_done global_id 4571 payload 274 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 1 -- 172.21.15.178:0/105934813 >> v2:172.21.15.204:3300/0 conn(0x7ffa2c09fe70 msgr2=0x7ffa2c0a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 1 --2- 172.21.15.178:0/105934813 >> v2:172.21.15.204:3300/0 conn(0x7ffa2c09fe70 0x7ffa2c0a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 1 -- 172.21.15.178:0/105934813 >> v1:172.21.15.204:6789/0 conn(0x7ffa2c09faa0 legacy=0x7ffa2c0a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 1 monclient: found mon.noname-e 2024-09-15T01:32:37.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 1 -- 172.21.15.178:0/105934813 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c0a0310 con 0x7ffa2c0a2930 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2b7fe640 1 --2- 172.21.15.178:0/105934813 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 0x7ffa2c132de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009910 tx=0x7ffa1c0091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e8ba6aa32a77fcf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2a7fc640 1 -- 172.21.15.178:0/105934813 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa1c02ac90 con 0x7ffa2c0a2930 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.222+0000 7ffa2a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa2a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa2a7fc640 10 monclient: dump: 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.236 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa2a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa2a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa2a7fc640 1 -- 172.21.15.178:0/105934813 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1c02ae10 con 0x7ffa2c0a2930 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 5 monclient: authenticate success, global_id 4571 2024-09-15T01:32:37.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa2a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa2a7fc640 1 -- 172.21.15.178:0/105934813 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa1c031ca0 con 0x7ffa2c0a2930 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 4 set_mon_vals no callback set 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa2a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa2a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa2a7fc640 10 monclient: dump: 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.238 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 10 monclient: shutdown 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 1 -- 172.21.15.178:0/105934813 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 msgr2=0x7ffa2c132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 1 --2- 172.21.15.178:0/105934813 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 0x7ffa2c132de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009910 tx=0x7ffa1c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 1 -- 172.21.15.178:0/105934813 shutdown_connections 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 1 --2- 172.21.15.178:0/105934813 >> v2:172.21.15.204:3300/0 conn(0x7ffa2c09fe70 0x7ffa2c0a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 1 --2- 172.21.15.178:0/105934813 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 0x7ffa2c132de0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 1 -- 172.21.15.178:0/105934813 >> 172.21.15.178:0/105934813 conn(0x7ffa2c0a5ed0 msgr2=0x7ffa2c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 1 -- 172.21.15.178:0/105934813 shutdown_connections 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 1 -- 172.21.15.178:0/105934813 wait complete. 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.223+0000 7ffa31ed4640 10 monclient: build_initial_monmap 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 10 monclient: monmap: 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:37.240 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 1 Processor -- start 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 1 -- start start 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 10 monclient: init 2024-09-15T01:32:37.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 10 monclient: _add_conns ranks=[0,7,8,2,3,6,4,1,5] 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa2c09faa0 0x7ffa2c09b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 10 monclient(hunting): picked mon.noname-a con 0x7ffa2c09faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffa2c09fe70 0x7ffa2c131b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 10 monclient(hunting): picked mon.noname-i con 0x7ffa2c09fe70 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 0x7ffa2c1320a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 10 monclient(hunting): picked mon.noname-e con 0x7ffa2c0a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c09f870 con 0x7ffa2c09faa0 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c11ec80 con 0x7ffa2c09fe70 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c11ef60 con 0x7ffa2c0a2930 2024-09-15T01:32:37.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.224+0000 7ffa31ed4640 10 monclient(hunting): authenticate will time out at 761.974968s 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 0x7ffa2c1320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa2c09faa0 0x7ffa2c09b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 0x7ffa2c1320a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42552/0 (socket says 172.21.15.178:42552) 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2b7fe640 10 monclient(hunting): get_auth_request con 0x7ffa2c09faa0 auth_method 0 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 1 -- 172.21.15.178:0/3159337846 learned_addr learned my addr 172.21.15.178:0/3159337846 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 10 monclient(hunting): get_auth_request con 0x7ffa2c0a2930 auth_method 0 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2affd640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffa2c09fe70 0x7ffa2c131b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2affd640 10 monclient(hunting): get_auth_request con 0x7ffa2c09fe70 auth_method 0 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 10 monclient(hunting): handle_auth_done global_id 4580 payload 824 2024-09-15T01:32:37.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 1 -- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffa2c09fe70 msgr2=0x7ffa2c131b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffa2c09fe70 0x7ffa2c131b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 1 -- 172.21.15.178:0/3159337846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa2c09faa0 msgr2=0x7ffa2c09b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa2c09faa0 0x7ffa2c09b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 1 monclient: found mon.noname-e 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 1 -- 172.21.15.178:0/3159337846 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c009450 con 0x7ffa2c0a2930 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa2bfff640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 0x7ffa2c1320a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009fa0 tx=0x7ffa24009900 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa2400ee60 con 0x7ffa2c0a2930 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa137fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa137fe640 10 monclient: dump: 2024-09-15T01:32:37.245 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.246 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa137fe640 10 monclient: _finish_auth 0 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa31ed4640 5 monclient: authenticate success, global_id 4580 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa24009a40 con 0x7ffa2c0a2930 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa31ed4640 10 monclient: _renew_subs 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa31ed4640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa31ed4640 1 -- 172.21.15.178:0/3159337846 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c11dd20 con 0x7ffa2c0a2930 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa137fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa24014d20 con 0x7ffa2c0a2930 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa137fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa137fe640 10 monclient: dump: 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.247 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.248 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa29ffb640 4 set_mon_vals no callback set 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa31ed4640 10 monclient: _renew_subs 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa31ed4640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.225+0000 7ffa31ed4640 1 -- 172.21.15.178:0/3159337846 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2c1427e0 con 0x7ffa2c0a2930 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ffa24014420 con 0x7ffa2c0a2930 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa31ed4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa31ed4640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11c530 0x7ffa2c11c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa31ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa31ed4640 1 -- 172.21.15.178:0/3159337846 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c11edf0 con 0x7ffa2c11c530 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa137fe640 1 --2- 172.21.15.178:0/3159337846 >> v2:172.21.15.204:6812/28210 conn(0x7ffa180397b0 0x7ffa1803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7ffa2404b6e0 con 0x7ffa2c0a2930 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa2b7fe640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11c530 0x7ffa2c11c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa2b7fe640 10 monclient: get_auth_request con 0x7ffa2c11c530 auth_method 0 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa2b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa2affd640 1 --2- 172.21.15.178:0/3159337846 >> v2:172.21.15.204:6812/28210 conn(0x7ffa180397b0 0x7ffa1803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.226+0000 7ffa2affd640 10 monclient: get_auth_request con 0x7ffa180397b0 auth_method 0 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_done global_id 4568 payload 824 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa2b7fe640 1 -- 172.21.15.178:0/3159337846 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa2c04c4a0 con 0x7ffa2c11c530 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa2b7fe640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11c530 0x7ffa2c11c900 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c067590 tx=0x7ffa1c032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa2affd640 1 --2- 172.21.15.178:0/3159337846 >> v2:172.21.15.204:6812/28210 conn(0x7ffa180397b0 0x7ffa1803bc60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0a6ec0 tx=0x7ffa20006f50 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa1c03c860 con 0x7ffa2c11c530 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa137fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:32:37.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa137fe640 10 monclient: dump: 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.251 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.252 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7ffa1c03c3e0 con 0x7ffa2c11c530 2024-09-15T01:32:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa137fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa137fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11c530 msgr2=0x7ffa2c11c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa137fe640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11c530 0x7ffa2c11c900 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c067590 tx=0x7ffa1c032000 comp rx=0 tx=0).stop 2024-09-15T01:32:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11c530 msgr2=0x7ffa2c11c900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.227+0000 7ffa137fe640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11c530 0x7ffa2c11c900 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c067590 tx=0x7ffa1c032000 comp rx=0 tx=0).stop 2024-09-15T01:32:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.233+0000 7ffa31ed4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.233+0000 7ffa31ed4640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11ce40 0x7ffa2c096de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.233+0000 7ffa31ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.233+0000 7ffa31ed4640 1 -- 172.21.15.178:0/3159337846 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c04bc80 con 0x7ffa2c11ce40 2024-09-15T01:32:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.233+0000 7ffa2b7fe640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11ce40 0x7ffa2c096de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.233+0000 7ffa2b7fe640 10 monclient: get_auth_request con 0x7ffa2c11ce40 auth_method 0 2024-09-15T01:32:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.233+0000 7ffa2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.233+0000 7ffa2b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.233+0000 7ffa2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.233+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.233+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.233+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.234+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_done global_id 4577 payload 824 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.234+0000 7ffa2b7fe640 1 -- 172.21.15.178:0/3159337846 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffa2c05ed70 con 0x7ffa2c11ce40 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.234+0000 7ffa2b7fe640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11ce40 0x7ffa2c096de0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0097e0 tx=0x7ffa1c03d000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.234+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa1c04e070 con 0x7ffa2c11ce40 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.234+0000 7ffa137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.234+0000 7ffa137fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.234+0000 7ffa137fe640 10 monclient: dump: 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9291 (secure 0 0 0) 0x7ffa1c04a470 con 0x7ffa2c11ce40 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa137fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa137fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:37.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11ce40 msgr2=0x7ffa2c096de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa137fe640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11ce40 0x7ffa2c096de0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0097e0 tx=0x7ffa1c03d000 comp rx=0 tx=0).stop 2024-09-15T01:32:37.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa137fe640 1 -- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11ce40 msgr2=0x7ffa2c096de0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:37.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa137fe640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa2c11ce40 0x7ffa2c096de0 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0097e0 tx=0x7ffa1c03d000 comp rx=0 tx=0).stop 2024-09-15T01:32:37.257 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:37.257 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:37.257 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:37.257 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:32:37.257 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 20, 2024-09-15T01:32:37.257 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:37.257 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:37.258 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:37.259 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:37.260 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:37.260 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:37.260 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:37.260 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:37.260 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.260 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:37.260 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:37.260 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:37.260 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:37.260 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:37.260 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:37.261 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:37.262 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:37.263 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.266 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:37.268 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:37.269 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.269 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.269 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.269 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.269 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:37.269 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:37.269 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.269 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.269 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.269 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.269 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.270 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.271 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.272 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.273 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.274 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.275 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.276 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:37.277 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:37.278 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.279 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:37.279 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:37.279 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:32:37.279 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.279 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.279 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.279 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:37.279 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:37.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa31ed4640 1 -- 172.21.15.178:0/3159337846 >> v2:172.21.15.204:6812/28210 conn(0x7ffa180397b0 msgr2=0x7ffa1803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa31ed4640 1 --2- 172.21.15.178:0/3159337846 >> v2:172.21.15.204:6812/28210 conn(0x7ffa180397b0 0x7ffa1803bc60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0a6ec0 tx=0x7ffa20006f50 comp rx=0 tx=0).stop 2024-09-15T01:32:37.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa31ed4640 10 monclient: shutdown 2024-09-15T01:32:37.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa31ed4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:37.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa31ed4640 1 -- 172.21.15.178:0/3159337846 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 msgr2=0x7ffa2c1320a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa31ed4640 1 --2- 172.21.15.178:0/3159337846 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa2c0a2930 0x7ffa2c1320a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009fa0 tx=0x7ffa24009900 comp rx=0 tx=0).stop 2024-09-15T01:32:37.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa2bfff640 1 -- 172.21.15.178:0/3159337846 reap_dead start 2024-09-15T01:32:37.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa2bfff640 1 -- 172.21.15.178:0/3159337846 reap_dead start 2024-09-15T01:32:37.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa31ed4640 1 -- 172.21.15.178:0/3159337846 shutdown_connections 2024-09-15T01:32:37.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa31ed4640 1 -- 172.21.15.178:0/3159337846 >> 172.21.15.178:0/3159337846 conn(0x7ffa2c0a5ed0 msgr2=0x7ffa2c135a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:37.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa31ed4640 1 -- 172.21.15.178:0/3159337846 shutdown_connections 2024-09-15T01:32:37.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.235+0000 7ffa31ed4640 1 -- 172.21.15.178:0/3159337846 wait complete. 2024-09-15T01:32:37.281 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:32:37.281 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-09-15T01:32:37.282 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:32:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.373+0000 7f87d8f5d640 10 monclient: get_monmap_and_config 2024-09-15T01:32:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.373+0000 7f87d8f5d640 10 monclient: build_initial_monmap 2024-09-15T01:32:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.373+0000 7f87d8f5d640 10 monclient: monmap: 2024-09-15T01:32:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:32:37.374090+0000 2024-09-15T01:32:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:32:37.374090+0000 2024-09-15T01:32:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:32:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:32:37.376 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:37.376 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:37.376 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:37.376 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:37.376 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:37.376 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:37.376 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:37.376 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:32:37.376 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:32:37.376 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.373+0000 7f87d8f5d640 1 Processor -- start 2024-09-15T01:32:37.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.373+0000 7f87d8f5d640 1 -- start start 2024-09-15T01:32:37.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.373+0000 7f87d8f5d640 10 monclient: init 2024-09-15T01:32:37.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:37.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 10 monclient: _add_conns ranks=[3,6,4,2,7,8,10,0,9,5,1] 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d40a2930 0x7f87d4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 10 monclient(hunting): picked mon.noname-g con 0x7f87d40a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87d409faa0 0x7f87d409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 10 monclient(hunting): picked mon.noname-d con 0x7f87d409faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87d40a08c0 0x7f87d40a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 10 monclient(hunting): picked mon.noname-c con 0x7f87d40a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d409e0f0 con 0x7f87d40a2930 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87d409dd80 con 0x7f87d40a08c0 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f87d40a2d00 con 0x7f87d409faa0 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:37.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d8f5d640 10 monclient(hunting): authenticate will time out at 762.124461s 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d1d9b640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87d409faa0 msgr2=0x7f87d409fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d1d9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87d409faa0 0x7f87d409fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d259c640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d40a2930 msgr2=0x7f87d4132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d259c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d40a2930 0x7f87d4132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87d40a08c0 0x7f87d40a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87d40a08c0 0x7f87d40a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:55950/0 (socket says 172.21.15.178:55950) 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 1 -- 172.21.15.178:0/404701982 learned_addr learned my addr 172.21.15.178:0/404701982 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 10 monclient(hunting): get_auth_request con 0x7f87d40a08c0 auth_method 0 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 10 monclient(hunting): handle_auth_done global_id 4531 payload 274 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 1 -- 172.21.15.178:0/404701982 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87d409faa0 msgr2=0x7f87d409fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 1 --2- 172.21.15.178:0/404701982 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87d409faa0 0x7f87d409fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 1 -- 172.21.15.178:0/404701982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d40a2930 msgr2=0x7f87d4132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 1 --2- 172.21.15.178:0/404701982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d40a2930 0x7f87d4132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 1 monclient: found mon.noname-c 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.374+0000 7f87d2d9d640 1 -- 172.21.15.178:0/404701982 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d404bf60 con 0x7f87d40a08c0 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d2d9d640 1 --2- 172.21.15.178:0/404701982 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87d40a08c0 0x7f87d40a0d30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009f50 tx=0x7f87c4009840 comp rx=0 tx=0).ready entity=mon.4 client_cookie=e466e0f2aadcda1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d159a640 1 -- 172.21.15.178:0/404701982 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87c4007880 con 0x7f87d40a08c0 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d159a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d159a640 10 monclient: dump: 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.389 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d159a640 10 monclient: _finish_auth 0 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d8f5d640 5 monclient: authenticate success, global_id 4531 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d8f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d159a640 1 -- 172.21.15.178:0/404701982 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c402d050 con 0x7f87d40a08c0 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d8f5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d8f5d640 4 set_mon_vals no callback set 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d159a640 1 -- 172.21.15.178:0/404701982 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87c4030360 con 0x7f87d40a08c0 2024-09-15T01:32:37.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.375+0000 7f87d159a640 10 monclient: dump: 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.391 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 10 monclient: shutdown 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 1 -- 172.21.15.178:0/404701982 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87d40a08c0 msgr2=0x7f87d40a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 1 --2- 172.21.15.178:0/404701982 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87d40a08c0 0x7f87d40a0d30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009f50 tx=0x7f87c4009840 comp rx=0 tx=0).stop 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 1 -- 172.21.15.178:0/404701982 shutdown_connections 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 1 --2- 172.21.15.178:0/404701982 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87d409faa0 0x7f87d409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 1 --2- 172.21.15.178:0/404701982 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87d40a08c0 0x7f87d40a0d30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 1 --2- 172.21.15.178:0/404701982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d40a2930 0x7f87d4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 1 -- 172.21.15.178:0/404701982 >> 172.21.15.178:0/404701982 conn(0x7f87d40a5ed0 msgr2=0x7f87d40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 1 -- 172.21.15.178:0/404701982 shutdown_connections 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 1 -- 172.21.15.178:0/404701982 wait complete. 2024-09-15T01:32:37.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 10 monclient: build_initial_monmap 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.376+0000 7f87d8f5d640 10 monclient: monmap: 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:32:37.393 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 1 Processor -- start 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 1 -- start start 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 10 monclient: init 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 10 monclient: _add_conns ranks=[1,3,6,2,5,0,8,7,4] 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d409faa0 0x7f87d409af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 10 monclient(hunting): picked mon.noname-f con 0x7f87d409faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d40a08c0 0x7f87d41318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 10 monclient(hunting): picked mon.noname-g con 0x7f87d40a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87d40a2930 0x7f87d411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 10 monclient(hunting): picked mon.noname-d con 0x7f87d40a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d411ecb0 con 0x7f87d409faa0 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d409bb00 con 0x7f87d40a08c0 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f87d409d630 con 0x7f87d40a2930 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d8f5d640 10 monclient(hunting): authenticate will time out at 762.127575s 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d2d9d640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87d40a2930 msgr2=0x7f87d411c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d1d9b640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d40a08c0 msgr2=0x7f87d41318b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d2d9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87d40a2930 0x7f87d411c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d1d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d40a08c0 0x7f87d41318b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d409faa0 0x7f87d409af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d409faa0 0x7f87d409af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:51198/0 (socket says 172.21.15.178:51198) 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d259c640 1 -- 172.21.15.178:0/2556407483 learned_addr learned my addr 172.21.15.178:0/2556407483 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d259c640 10 monclient(hunting): get_auth_request con 0x7f87d409faa0 auth_method 0 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.377+0000 7f87d259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d259c640 10 monclient(hunting): handle_auth_done global_id 4627 payload 824 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d259c640 1 -- 172.21.15.178:0/2556407483 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87d40a2930 msgr2=0x7f87d411c530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d259c640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87d40a2930 0x7f87d411c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d259c640 1 -- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d40a08c0 msgr2=0x7f87d41318b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d259c640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87d40a08c0 0x7f87d41318b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d259c640 10 monclient: _finish_hunting 0 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d259c640 1 monclient: found mon.noname-f 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d259c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d259c640 1 -- 172.21.15.178:0/2556407483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c4009ae0 con 0x7f87d409faa0 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d259c640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d409faa0 0x7f87d409af60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00b600 tx=0x7f87cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87cc015070 con 0x7f87d409faa0 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87ca7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87ca7fc640 10 monclient: dump: 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.397 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87ca7fc640 10 monclient: _finish_auth 0 2024-09-15T01:32:37.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc006150 con 0x7f87d409faa0 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d8f5d640 5 monclient: authenticate success, global_id 4627 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d8f5d640 10 monclient: _renew_subs 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d8f5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d8f5d640 1 -- 172.21.15.178:0/2556407483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d41421d0 con 0x7f87d409faa0 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87ca7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87cc0062c0 con 0x7f87d409faa0 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87ca7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87ca7fc640 10 monclient: dump: 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.399 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d0d99640 4 set_mon_vals no callback set 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d8f5d640 10 monclient: _renew_subs 2024-09-15T01:32:37.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d8f5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.378+0000 7f87d8f5d640 1 -- 172.21.15.178:0/2556407483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d4142ac0 con 0x7f87d409faa0 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d8f5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d8f5d640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d411da70 0x7f87d411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d8f5d640 1 -- 172.21.15.178:0/2556407483 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f87d411eb70 con 0x7f87d411da70 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d1d9b640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d411da70 0x7f87d411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f87cc003b80 con 0x7f87d409faa0 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d1d9b640 10 monclient: get_auth_request con 0x7f87d411da70 auth_method 0 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87ca7fc640 1 --2- 172.21.15.178:0/2556407483 >> v2:172.21.15.204:6812/28210 conn(0x7f87b40397b0 0x7f87b403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f87cc04a9f0 con 0x7f87d409faa0 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d1d9b640 10 monclient(hunting): handle_auth_done global_id 4516 payload 824 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d1d9b640 1 -- 172.21.15.178:0/2556407483 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87d404c4a0 con 0x7f87d411da70 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d2d9d640 1 --2- 172.21.15.178:0/2556407483 >> v2:172.21.15.204:6812/28210 conn(0x7f87b40397b0 0x7f87b403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.379+0000 7f87d2d9d640 10 monclient: get_auth_request con 0x7f87b40397b0 auth_method 0 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87d1d9b640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d411da70 0x7f87d411ca70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00b460 tx=0x7f87bc0090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87bc0050f0 con 0x7f87d411da70 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87ca7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87ca7fc640 10 monclient: dump: 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.402 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.403 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87d2d9d640 1 --2- 172.21.15.178:0/2556407483 >> v2:172.21.15.204:6812/28210 conn(0x7f87b40397b0 0x7f87b403bc60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f87c4033000 tx=0x7f87c4036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f87bc003b30 con 0x7f87d411da70 2024-09-15T01:32:37.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87ca7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:32:37.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87ca7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:32:37.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d411da70 msgr2=0x7f87d411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87ca7fc640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d411da70 0x7f87d411ca70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00b460 tx=0x7f87bc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d411da70 msgr2=0x7f87d411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:37.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.380+0000 7f87ca7fc640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d411da70 0x7f87d411ca70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d8f5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:32:37.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d8f5d640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d4131df0 0x7f87d411cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:32:37.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:32:37.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d8f5d640 1 -- 172.21.15.178:0/2556407483 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f87d411ee20 con 0x7f87d4131df0 2024-09-15T01:32:37.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d1d9b640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d4131df0 0x7f87d411cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:32:37.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d1d9b640 10 monclient: get_auth_request con 0x7f87d4131df0 auth_method 0 2024-09-15T01:32:37.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:32:37.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:32:37.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:32:37.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:32:37.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:32:37.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.386+0000 7f87d1d9b640 10 monclient(hunting): handle_auth_done global_id 4525 payload 824 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.387+0000 7f87d1d9b640 1 -- 172.21.15.178:0/2556407483 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f87d404bc80 con 0x7f87d4131df0 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.387+0000 7f87d1d9b640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d4131df0 0x7f87d411cfb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0093a0 tx=0x7f87bc015000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.387+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87bc010de0 con 0x7f87d4131df0 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.387+0000 7f87ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.387+0000 7f87ca7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.387+0000 7f87ca7fc640 10 monclient: dump: 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:32:37.407 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9459 (secure 0 0 0) 0x7f87bc028460 con 0x7f87d4131df0 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87ca7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87ca7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d4131df0 msgr2=0x7f87d411cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87ca7fc640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d4131df0 0x7f87d411cfb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0093a0 tx=0x7f87bc015000 comp rx=0 tx=0).stop 2024-09-15T01:32:37.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87ca7fc640 1 -- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d4131df0 msgr2=0x7f87d411cfb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:32:37.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87ca7fc640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87d4131df0 0x7f87d411cfb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.409 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:32:37.409 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:37.409 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:37.409 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:32:37.409 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 20, 2024-09-15T01:32:37.409 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:32:37.409 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:32:37.409 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:32:37.409 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:32:37.409 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:37.410 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:37.411 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:32:37.412 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:32:37.413 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.415 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:32:37.417 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:32:37.419 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.420 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:32:37.422 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:32:37.423 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.425 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.426 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:32:37.427 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.428 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:37.429 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:37.430 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:32:37.431 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:32:37.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87d8f5d640 1 -- 172.21.15.178:0/2556407483 >> v2:172.21.15.204:6812/28210 conn(0x7f87b40397b0 msgr2=0x7f87b403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87d8f5d640 1 --2- 172.21.15.178:0/2556407483 >> v2:172.21.15.204:6812/28210 conn(0x7f87b40397b0 0x7f87b403bc60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f87c4033000 tx=0x7f87c4036000 comp rx=0 tx=0).stop 2024-09-15T01:32:37.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87d8f5d640 10 monclient: shutdown 2024-09-15T01:32:37.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87d8f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:32:37.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87d8f5d640 1 -- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d409faa0 msgr2=0x7f87d409af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:32:37.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87d8f5d640 1 --2- 172.21.15.178:0/2556407483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d409faa0 0x7f87d409af60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00b600 tx=0x7f87cc0091a0 comp rx=0 tx=0).stop 2024-09-15T01:32:37.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87d2d9d640 1 -- 172.21.15.178:0/2556407483 reap_dead start 2024-09-15T01:32:37.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87d2d9d640 1 -- 172.21.15.178:0/2556407483 reap_dead start 2024-09-15T01:32:37.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87d8f5d640 1 -- 172.21.15.178:0/2556407483 shutdown_connections 2024-09-15T01:32:37.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87d8f5d640 1 -- 172.21.15.178:0/2556407483 >> 172.21.15.178:0/2556407483 conn(0x7f87d40a5ed0 msgr2=0x7f87d409e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:32:37.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87d8f5d640 1 -- 172.21.15.178:0/2556407483 shutdown_connections 2024-09-15T01:32:37.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:32:37.388+0000 7f87d8f5d640 1 -- 172.21.15.178:0/2556407483 wait complete. 2024-09-15T01:32:37.434 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:32:37.434 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T01:33:03.146 INFO:tasks.workunit.client.0.smithi204.stderr:Note: switching to 'd75f23820806da622f561ee5ee7551015285993f'. 2024-09-15T01:33:03.146 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:03.146 INFO:tasks.workunit.client.0.smithi204.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-15T01:33:03.146 INFO:tasks.workunit.client.0.smithi204.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-15T01:33:03.146 INFO:tasks.workunit.client.0.smithi204.stderr:state without impacting any branches by switching back to a branch. 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr: git switch -c 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr:Or undo this operation with: 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr: git switch - 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:03.147 INFO:tasks.workunit.client.0.smithi204.stderr:HEAD is now at d75f2382080 Merge branch 'wip-67762-quincy' of https://github.com/nbalacha/ceph into wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-15T01:33:03.151 DEBUG:teuthology.orchestra.run.smithi204:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-15T01:33:03.201 INFO:tasks.workunit.client.0.smithi204.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-15T01:33:03.203 INFO:tasks.workunit.client.0.smithi204.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T01:33:03.203 INFO:tasks.workunit.client.0.smithi204.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-15T01:33:03.800 INFO:tasks.workunit.client.0.smithi204.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-15T01:33:03.883 INFO:tasks.workunit.client.0.smithi204.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-15T01:33:03.925 INFO:tasks.workunit.client.0.smithi204.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T01:33:03.927 INFO:tasks.workunit.client.0.smithi204.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T01:33:03.927 INFO:tasks.workunit.client.0.smithi204.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-15T01:33:03.967 INFO:tasks.workunit.client.0.smithi204.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T01:33:03.970 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-15T01:33:03.970 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-15T01:33:03.987 INFO:tasks.workunit:Running workunits matching mon/pool_ops.sh on client.0... 2024-09-15T01:33:03.988 INFO:tasks.workunit:Running workunit mon/pool_ops.sh... 2024-09-15T01:33:03.988 DEBUG:teuthology.orchestra.run.smithi204:workunit test mon/pool_ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d75f23820806da622f561ee5ee7551015285993f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/pool_ops.sh 2024-09-15T01:33:04.047 INFO:tasks.workunit.client.0.smithi204.stderr:+ TEST_POOL=testpool1234 2024-09-15T01:33:04.047 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool create testpool1234 8 --autoscale-mode off 2024-09-15T01:33:04.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.120+0000 7f614ff9d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:04.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.120+0000 7f614ff9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:04.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.120+0000 7f614ff9d640 10 monclient: monmap: 2024-09-15T01:33:04.123 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:04.123 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:04.121802+0000 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:04.121802+0000 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:04.124 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:04.125 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:04.125 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:04.125 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:04.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.120+0000 7f614ff9d640 1 Processor -- start 2024-09-15T01:33:04.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 1 -- start start 2024-09-15T01:33:04.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 10 monclient: init 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 10 monclient: _add_conns ranks=[5,9,10,0,2,8,6,4,3,7,1] 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6148097c60 0x7f614812f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 10 monclient(hunting): picked mon.noname-h con 0x7f6148097c60 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f614809b820 0x7f614812b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 10 monclient(hunting): picked mon.noname-a con 0x7f614809b820 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f614809bc50 0x7f614812bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 10 monclient(hunting): picked mon.noname-f con 0x7f614809bc50 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6148097640 con 0x7f614809b820 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6148099320 con 0x7f614809bc50 2024-09-15T01:33:04.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6148097390 con 0x7f6148097c60 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614ff9d640 10 monclient(hunting): authenticate will time out at 799.764588s 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614e53a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f614809bc50 0x7f614812bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614dd39640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6148097c60 0x7f614812f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614e53a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f614809bc50 0x7f614812bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:60698/0 (socket says 172.21.15.204:60698) 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614dd39640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6148097c60 0x7f614812f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:42510/0 (socket says 172.21.15.204:42510) 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614e53a640 1 -- 172.21.15.204:0/3117621364 learned_addr learned my addr 172.21.15.204:0/3117621364 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614e53a640 10 monclient(hunting): get_auth_request con 0x7f614809bc50 auth_method 0 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614e53a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614e53a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614e53a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:04.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614dd39640 10 monclient(hunting): get_auth_request con 0x7f6148097c60 auth_method 0 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614dd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614dd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614dd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614e53a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614e53a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614e53a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614dd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614dd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614dd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614d538640 1 --2- 172.21.15.204:0/3117621364 >> v2:172.21.15.178:3300/0 conn(0x7f614809b820 0x7f614812b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614d538640 10 monclient(hunting): get_auth_request con 0x7f614809b820 auth_method 0 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614d538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:04.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614d538640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614d538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614e53a640 10 monclient(hunting): handle_auth_done global_id 4636 payload 274 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.121+0000 7f614e53a640 1 -- 172.21.15.204:0/3117621364 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6148097c60 msgr2=0x7f614812f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614e53a640 1 --2- 172.21.15.204:0/3117621364 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6148097c60 0x7f614812f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614e53a640 1 -- 172.21.15.204:0/3117621364 >> v2:172.21.15.178:3300/0 conn(0x7f614809b820 msgr2=0x7f614812b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614e53a640 1 --2- 172.21.15.204:0/3117621364 >> v2:172.21.15.178:3300/0 conn(0x7f614809b820 0x7f614812b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614e53a640 10 monclient: _finish_hunting 0 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614e53a640 1 monclient: found mon.noname-f 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614e53a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614e53a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614e53a640 1 -- 172.21.15.204:0/3117621364 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61480648a0 con 0x7f614809bc50 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614dd39640 1 --2- 172.21.15.204:0/3117621364 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6148097c60 0x7f614812f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:04.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614e53a640 1 --2- 172.21.15.204:0/3117621364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f614809bc50 0x7f614812bcf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6148099020 tx=0x7f6140009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5278281d53440c38 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614cd37640 1 -- 172.21.15.204:0/3117621364 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6140007880 con 0x7f614809bc50 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614cd37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614cd37640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614cd37640 10 monclient: dump: 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:04.137 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614cd37640 10 monclient: _finish_auth 0 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614cd37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614cd37640 1 -- 172.21.15.204:0/3117621364 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614002d050 con 0x7f614809bc50 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614ff9d640 5 monclient: authenticate success, global_id 4636 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614ff9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:04.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614cd37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614ff9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614cd37640 1 -- 172.21.15.204:0/3117621364 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6140030360 con 0x7f614809bc50 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614cd37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614ff9d640 4 set_mon_vals no callback set 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614cd37640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.122+0000 7f614cd37640 10 monclient: dump: 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:04.139 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:04.140 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:04.140 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:04.140 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:04.140 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:04.140 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:04.140 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:04.140 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:04.140 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:04.140 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 10 monclient: shutdown 2024-09-15T01:33:04.140 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:04.140 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 1 -- 172.21.15.204:0/3117621364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f614809bc50 msgr2=0x7f614812bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:04.140 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 1 --2- 172.21.15.204:0/3117621364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f614809bc50 0x7f614812bcf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6148099020 tx=0x7f6140009840 comp rx=0 tx=0).stop 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 1 -- 172.21.15.204:0/3117621364 shutdown_connections 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 1 --2- 172.21.15.204:0/3117621364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f614809bc50 0x7f614812bcf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 1 --2- 172.21.15.204:0/3117621364 >> v2:172.21.15.178:3300/0 conn(0x7f614809b820 0x7f614812b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 1 --2- 172.21.15.204:0/3117621364 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6148097c60 0x7f614812f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 1 -- 172.21.15.204:0/3117621364 >> 172.21.15.204:0/3117621364 conn(0x7f61480a1e40 msgr2=0x7f61480a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 1 -- 172.21.15.204:0/3117621364 shutdown_connections 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 1 -- 172.21.15.204:0/3117621364 wait complete. 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.123+0000 7f614ff9d640 10 monclient: monmap: 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:04.141 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 1 Processor -- start 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 1 -- start start 2024-09-15T01:33:04.142 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 10 monclient: init 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 10 monclient: _add_conns ranks=[2,0,8,5,7,4,3,6,1] 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6148097c60 0x7f6148141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 10 monclient(hunting): picked mon.noname-b con 0x7f6148097c60 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614809b820 0x7f6148142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 10 monclient(hunting): picked mon.noname-a con 0x7f614809b820 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f614809bc50 0x7f614806e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 10 monclient(hunting): picked mon.noname-e con 0x7f614809bc50 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6148098370 con 0x7f614809b820 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614812b3e0 con 0x7f6148097c60 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f614811c7b0 con 0x7f614809bc50 2024-09-15T01:33:04.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.124+0000 7f614ff9d640 10 monclient(hunting): authenticate will time out at 799.767768s 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614e53a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f614809bc50 0x7f614806e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6148097c60 0x7f6148141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614d538640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614809b820 0x7f6148142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614e53a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f614809bc50 0x7f614806e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:43536/0 (socket says 172.21.15.204:43536) 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614e53a640 1 -- 172.21.15.204:0/3685357513 learned_addr learned my addr 172.21.15.204:0/3685357513 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614e53a640 10 monclient(hunting): get_auth_request con 0x7f614809bc50 auth_method 0 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614e53a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614e53a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614e53a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 10 monclient(hunting): get_auth_request con 0x7f6148097c60 auth_method 0 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:04.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614d538640 10 monclient(hunting): get_auth_request con 0x7f614809b820 auth_method 0 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614d538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614d538640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614d538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614e53a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614e53a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614e53a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614d538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614d538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:04.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614d538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 10 monclient(hunting): handle_auth_done global_id 4556 payload 824 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 1 -- 172.21.15.204:0/3685357513 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f614809bc50 msgr2=0x7f614806e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 1 --2- 172.21.15.204:0/3685357513 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f614809bc50 0x7f614806e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 1 -- 172.21.15.204:0/3685357513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614809b820 msgr2=0x7f6148142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 1 --2- 172.21.15.204:0/3685357513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614809b820 0x7f6148142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 10 monclient: _finish_hunting 0 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 1 monclient: found mon.noname-b 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 1 -- 172.21.15.204:0/3685357513 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614400a530 con 0x7f6148097c60 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614dd39640 1 --2- 172.21.15.204:0/3685357513 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6148097c60 0x7f6148141e30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f614400e9a0 tx=0x7f614400a2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f6139ffb640 1 -- 172.21.15.204:0/3685357513 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61440113f0 con 0x7f6148097c60 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f6139ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:04.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f6139ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f6139ffb640 10 monclient: dump: 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:04.147 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f6139ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f6139ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614ff9d640 5 monclient: authenticate success, global_id 4556 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f6139ffb640 1 -- 172.21.15.204:0/3685357513 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6144011570 con 0x7f6148097c60 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f6139ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f6139ffb640 1 -- 172.21.15.204:0/3685357513 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6144011b60 con 0x7f6148097c60 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f6139ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f6139ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f6139ffb640 10 monclient: dump: 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:04.148 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:04.149 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f613bfff640 4 set_mon_vals no callback set 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614ff9d640 10 monclient: _renew_subs 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614ff9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.125+0000 7f614ff9d640 1 -- 172.21.15.204:0/3685357513 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6140009ae0 con 0x7f6148097c60 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.126+0000 7f614ff9d640 10 monclient: _renew_subs 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.126+0000 7f614ff9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.126+0000 7f614ff9d640 1 -- 172.21.15.204:0/3685357513 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614811ca70 con 0x7f6148097c60 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.126+0000 7f6139ffb640 1 -- 172.21.15.204:0/3685357513 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6144017030 con 0x7f6148097c60 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.126+0000 7f6139ffb640 1 --2- 172.21.15.204:0/3685357513 >> v2:172.21.15.204:6812/28210 conn(0x7f612c0397b0 0x7f612c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.127+0000 7f614ff9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.127+0000 7f614ff9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.127+0000 7f614ff9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.127+0000 7f6139ffb640 1 -- 172.21.15.204:0/3685357513 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3530+0+0 (secure 0 0 0) 0x7f6144015070 con 0x7f6148097c60 2024-09-15T01:33:04.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.127+0000 7f614ff9d640 1 -- 172.21.15.204:0/3685357513 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6148070fc0 con 0x7f6148097c60 2024-09-15T01:33:04.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.127+0000 7f614d538640 1 --2- 172.21.15.204:0/3685357513 >> v2:172.21.15.204:6812/28210 conn(0x7f612c0397b0 0x7f612c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:04.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.127+0000 7f614d538640 10 monclient: get_auth_request con 0x7f612c0397b0 auth_method 0 2024-09-15T01:33:04.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.127+0000 7f614d538640 1 --2- 172.21.15.204:0/3685357513 >> v2:172.21.15.204:6812/28210 conn(0x7f612c0397b0 0x7f612c03bc60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f613c007ba0 tx=0x7f613c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:04.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.132+0000 7f6139ffb640 1 -- 172.21.15.204:0/3685357513 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f614401f070 con 0x7f6148097c60 2024-09-15T01:33:04.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.132+0000 7f6139ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:04.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.132+0000 7f6139ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:04.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.288+0000 7f614ff9d640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-09-15T01:33:04.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.288+0000 7f614ff9d640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-09-15T01:33:04.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.288+0000 7f614ff9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:04.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:04.288+0000 7f614ff9d640 1 -- 172.21.15.204:0/3685357513 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"} v 0) v1 -- 0x7f61480008d0 con 0x7f6148097c60 2024-09-15T01:33:05.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.253+0000 7f6139ffb640 1 -- 172.21.15.204:0/3685357513 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}]=0 pool 'testpool1234' created v21) v1 ==== 152+0+0 (secure 0 0 0) 0x7f6144021d30 con 0x7f6148097c60 2024-09-15T01:33:05.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.253+0000 7f6139ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-09-15T01:33:05.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.253+0000 7f6139ffb640 10 monclient: _finish_command 2 = system:0 pool 'testpool1234' created 2024-09-15T01:33:05.355 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.353+0000 7f614ff9d640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-09-15T01:33:05.356 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.353+0000 7f614ff9d640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-09-15T01:33:05.356 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.353+0000 7f614ff9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:05.356 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.353+0000 7f614ff9d640 1 -- 172.21.15.204:0/3685357513 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"} v 0) v1 -- 0x7f6148129b60 con 0x7f6148097c60 2024-09-15T01:33:05.356 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.354+0000 7f6139ffb640 1 -- 172.21.15.204:0/3685357513 <== mon.2 v2:172.21.15.178:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}]=0 pool 'testpool1234' already exists v21) v1 ==== 159+0+0 (secure 0 0 0) 0x7f6144019eb0 con 0x7f6148097c60 2024-09-15T01:33:05.356 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.354+0000 7f6139ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-09-15T01:33:05.357 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.354+0000 7f6139ffb640 10 monclient: _finish_command 3 = system:0 pool 'testpool1234' already exists 2024-09-15T01:33:05.357 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'testpool1234' already exists 2024-09-15T01:33:05.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 1 -- 172.21.15.204:0/3685357513 >> v2:172.21.15.204:6812/28210 conn(0x7f612c0397b0 msgr2=0x7f612c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:05.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 1 --2- 172.21.15.204:0/3685357513 >> v2:172.21.15.204:6812/28210 conn(0x7f612c0397b0 0x7f612c03bc60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f613c007ba0 tx=0x7f613c008040 comp rx=0 tx=0).stop 2024-09-15T01:33:05.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 10 monclient: shutdown 2024-09-15T01:33:05.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:05.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 1 -- 172.21.15.204:0/3685357513 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6148097c60 msgr2=0x7f6148141e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:05.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 1 --2- 172.21.15.204:0/3685357513 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6148097c60 0x7f6148141e30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f614400e9a0 tx=0x7f614400a2b0 comp rx=0 tx=0).stop 2024-09-15T01:33:05.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 1 -- 172.21.15.204:0/3685357513 shutdown_connections 2024-09-15T01:33:05.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 1 --2- 172.21.15.204:0/3685357513 >> v2:172.21.15.204:6812/28210 conn(0x7f612c0397b0 0x7f612c03bc60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:05.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 1 --2- 172.21.15.204:0/3685357513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614809b820 0x7f6148142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:05.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 1 --2- 172.21.15.204:0/3685357513 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f614809bc50 0x7f614806e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:05.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 1 --2- 172.21.15.204:0/3685357513 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6148097c60 0x7f6148141e30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:05.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 1 -- 172.21.15.204:0/3685357513 >> 172.21.15.204:0/3685357513 conn(0x7f61480a1e40 msgr2=0x7f614812b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:05.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 1 -- 172.21.15.204:0/3685357513 shutdown_connections 2024-09-15T01:33:05.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.356+0000 7f614ff9d640 1 -- 172.21.15.204:0/3685357513 wait complete. 2024-09-15T01:33:05.368 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set testpool1234 pg_num_min 2 2024-09-15T01:33:05.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.440+0000 7ff237657640 10 monclient: get_monmap_and_config 2024-09-15T01:33:05.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.440+0000 7ff237657640 10 monclient: build_initial_monmap 2024-09-15T01:33:05.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.440+0000 7ff237657640 10 monclient: monmap: 2024-09-15T01:33:05.443 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:05.443 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:05.443 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:05.442402+0000 2024-09-15T01:33:05.443 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:05.442402+0000 2024-09-15T01:33:05.444 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:05.444 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:05.444 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:05.444 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:05.444 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:05.444 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:05.444 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:05.444 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:05.444 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:05.444 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:05.444 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:05.444 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:05.445 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:05.445 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:05.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.441+0000 7ff237657640 1 Processor -- start 2024-09-15T01:33:05.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.441+0000 7ff237657640 1 -- start start 2024-09-15T01:33:05.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.441+0000 7ff237657640 10 monclient: init 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 10 monclient: _add_conns ranks=[3,4,5,8,9,6,0,10,1,2,7] 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2300a2cd0 0x7ff230132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 10 monclient(hunting): picked mon.noname-g con 0x7ff2300a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff23009fa20 0x7ff23009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 10 monclient(hunting): picked mon.noname-c con 0x7ff23009fa20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff2300a08e0 0x7ff2300a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 10 monclient(hunting): picked mon.noname-h con 0x7ff2300a08e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300a30a0 con 0x7ff2300a2cd0 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300a0330 con 0x7ff23009fa20 2024-09-15T01:33:05.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:05.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300a1db0 con 0x7ff2300a08e0 2024-09-15T01:33:05.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:05.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff237657640 10 monclient(hunting): authenticate will time out at 801.085179s 2024-09-15T01:33:05.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff2353f3640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2300a2cd0 msgr2=0x7ff230132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:05.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff2353f3640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2300a2cd0 0x7ff230132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:05.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff2300a08e0 0x7ff2300a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:05.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff2300a08e0 0x7ff2300a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:42524/0 (socket says 172.21.15.204:42524) 2024-09-15T01:33:05.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 1 -- 172.21.15.204:0/601719638 learned_addr learned my addr 172.21.15.204:0/601719638 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:05.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 10 monclient(hunting): get_auth_request con 0x7ff2300a08e0 auth_method 0 2024-09-15T01:33:05.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:05.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:05.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff234bf2640 1 --2- 172.21.15.204:0/601719638 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff23009fa20 0x7ff23009fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff234bf2640 10 monclient(hunting): get_auth_request con 0x7ff23009fa20 auth_method 0 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff234bf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff234bf2640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff234bf2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 10 monclient(hunting): handle_auth_done global_id 4595 payload 274 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 1 -- 172.21.15.204:0/601719638 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff23009fa20 msgr2=0x7ff23009fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 1 --2- 172.21.15.204:0/601719638 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff23009fa20 0x7ff23009fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 1 -- 172.21.15.204:0/601719638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2300a2cd0 msgr2=0x7ff230132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 1 --2- 172.21.15.204:0/601719638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2300a2cd0 0x7ff230132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:05.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 10 monclient: _finish_hunting 0 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 1 monclient: found mon.noname-h 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.442+0000 7ff235bf4640 1 -- 172.21.15.204:0/601719638 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2300648a0 con 0x7ff2300a08e0 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff235bf4640 1 --2- 172.21.15.204:0/601719638 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff2300a08e0 0x7ff2300a0d50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff228009f50 tx=0x7ff228009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=31ee0cd83f633699 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff227fff640 1 -- 172.21.15.204:0/601719638 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff228007880 con 0x7ff2300a08e0 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff227fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff227fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff227fff640 10 monclient: dump: 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:05.457 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff227fff640 10 monclient: _finish_auth 0 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff227fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:05.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff237657640 5 monclient: authenticate success, global_id 4595 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff237657640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff227fff640 1 -- 172.21.15.204:0/601719638 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22802d050 con 0x7ff2300a08e0 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff227fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff237657640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff237657640 4 set_mon_vals no callback set 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff227fff640 1 -- 172.21.15.204:0/601719638 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff228030360 con 0x7ff2300a08e0 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff227fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff227fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.443+0000 7ff227fff640 10 monclient: dump: 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:05.459 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 10 monclient: shutdown 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:05.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 1 -- 172.21.15.204:0/601719638 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff2300a08e0 msgr2=0x7ff2300a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 1 --2- 172.21.15.204:0/601719638 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff2300a08e0 0x7ff2300a0d50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff228009f50 tx=0x7ff228009840 comp rx=0 tx=0).stop 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 1 -- 172.21.15.204:0/601719638 shutdown_connections 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 1 --2- 172.21.15.204:0/601719638 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff2300a08e0 0x7ff2300a0d50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 1 --2- 172.21.15.204:0/601719638 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff23009fa20 0x7ff23009fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 1 --2- 172.21.15.204:0/601719638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2300a2cd0 0x7ff230132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 1 -- 172.21.15.204:0/601719638 >> 172.21.15.204:0/601719638 conn(0x7ff23006e860 msgr2=0x7ff23006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 1 -- 172.21.15.204:0/601719638 shutdown_connections 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 1 -- 172.21.15.204:0/601719638 wait complete. 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 10 monclient: build_initial_monmap 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 10 monclient: monmap: 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:05.461 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 1 Processor -- start 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.444+0000 7ff237657640 1 -- start start 2024-09-15T01:33:05.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 10 monclient: init 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 10 monclient: _add_conns ranks=[1,0,7,4,8,2,3,5,6] 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff23009fa20 0x7ff230142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 10 monclient(hunting): picked mon.noname-f con 0x7ff23009fa20 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2300a08e0 0x7ff23009aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 10 monclient(hunting): picked mon.noname-a con 0x7ff2300a08e0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff2300a2cd0 0x7ff23009b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 10 monclient(hunting): picked mon.noname-i con 0x7ff2300a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23009d3d0 con 0x7ff2300a08e0 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23011dc70 con 0x7ff23009fa20 2024-09-15T01:33:05.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff23011c920 con 0x7ff2300a2cd0 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff237657640 10 monclient(hunting): authenticate will time out at 801.088585s 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff2353f3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff23009fa20 0x7ff230142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff235bf4640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff2300a2cd0 0x7ff23009b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff2353f3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff23009fa20 0x7ff230142290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:60714/0 (socket says 172.21.15.204:60714) 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff2353f3640 1 -- 172.21.15.204:0/3453129929 learned_addr learned my addr 172.21.15.204:0/3453129929 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff2353f3640 10 monclient(hunting): get_auth_request con 0x7ff23009fa20 auth_method 0 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff2353f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff2353f3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff2353f3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff235bf4640 10 monclient(hunting): get_auth_request con 0x7ff2300a2cd0 auth_method 0 2024-09-15T01:33:05.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff235bf4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:05.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff235bf4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:05.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff235bf4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:05.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff234bf2640 1 --2- 172.21.15.204:0/3453129929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2300a08e0 0x7ff23009aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:05.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff2353f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:05.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff2353f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:05.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff2353f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:05.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff234bf2640 10 monclient(hunting): get_auth_request con 0x7ff2300a08e0 auth_method 0 2024-09-15T01:33:05.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff234bf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:05.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff234bf2640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:05.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff234bf2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:05.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff235bf4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:05.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff235bf4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff235bf4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff234bf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff234bf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff234bf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.445+0000 7ff2353f3640 10 monclient(hunting): handle_auth_done global_id 4645 payload 824 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2353f3640 1 -- 172.21.15.204:0/3453129929 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff2300a2cd0 msgr2=0x7ff23009b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2353f3640 1 --2- 172.21.15.204:0/3453129929 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff2300a2cd0 0x7ff23009b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2353f3640 1 -- 172.21.15.204:0/3453129929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2300a08e0 msgr2=0x7ff23009aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2353f3640 1 --2- 172.21.15.204:0/3453129929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2300a08e0 0x7ff23009aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2353f3640 10 monclient: _finish_hunting 0 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2353f3640 1 monclient: found mon.noname-f 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2353f3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:05.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2353f3640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2353f3640 1 -- 172.21.15.204:0/3453129929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff228009ae0 con 0x7ff23009fa20 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff235bf4640 1 --2- 172.21.15.204:0/3453129929 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff2300a2cd0 0x7ff23009b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2353f3640 1 --2- 172.21.15.204:0/3453129929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff23009fa20 0x7ff230142290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00b5f0 tx=0x7ff22c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2257fa640 1 -- 172.21.15.204:0/3453129929 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff22c015070 con 0x7ff23009fa20 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2257fa640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2257fa640 10 monclient: dump: 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:05.467 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2257fa640 10 monclient: _finish_auth 0 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2257fa640 1 -- 172.21.15.204:0/3453129929 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c006150 con 0x7ff23009fa20 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff237657640 5 monclient: authenticate success, global_id 4645 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2257fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:05.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2257fa640 1 -- 172.21.15.204:0/3453129929 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff22c006780 con 0x7ff23009fa20 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff237657640 10 monclient: _renew_subs 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff237657640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff237657640 1 -- 172.21.15.204:0/3453129929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23011ded0 con 0x7ff23009fa20 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2277fe640 4 set_mon_vals no callback set 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2257fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff2257fa640 10 monclient: dump: 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:05.469 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:05.470 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:05.470 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:05.470 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:05.470 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:05.470 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:05.470 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:05.470 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:05.470 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:05.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff237657640 10 monclient: _renew_subs 2024-09-15T01:33:05.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff237657640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:05.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.446+0000 7ff237657640 1 -- 172.21.15.204:0/3453129929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23009c9d0 con 0x7ff23009fa20 2024-09-15T01:33:05.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.447+0000 7ff2257fa640 1 -- 172.21.15.204:0/3453129929 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff22c006300 con 0x7ff23009fa20 2024-09-15T01:33:05.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.447+0000 7ff237657640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:05.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.447+0000 7ff237657640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:05.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.447+0000 7ff237657640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:05.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.447+0000 7ff237657640 1 -- 172.21.15.204:0/3453129929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23011e2d0 con 0x7ff23009fa20 2024-09-15T01:33:05.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.447+0000 7ff2257fa640 1 --2- 172.21.15.204:0/3453129929 >> v2:172.21.15.204:6812/28210 conn(0x7ff214039710 0x7ff21403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:05.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.447+0000 7ff2257fa640 1 -- 172.21.15.204:0/3453129929 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3853+0+0 (secure 0 0 0) 0x7ff22c04aa20 con 0x7ff23009fa20 2024-09-15T01:33:05.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.447+0000 7ff234bf2640 1 --2- 172.21.15.204:0/3453129929 >> v2:172.21.15.204:6812/28210 conn(0x7ff214039710 0x7ff21403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:05.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.447+0000 7ff234bf2640 10 monclient: get_auth_request con 0x7ff214039710 auth_method 0 2024-09-15T01:33:05.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.448+0000 7ff234bf2640 1 --2- 172.21.15.204:0/3453129929 >> v2:172.21.15.204:6812/28210 conn(0x7ff214039710 0x7ff21403bbc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff23009ae20 tx=0x7ff220008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:05.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.452+0000 7ff2257fa640 1 -- 172.21.15.204:0/3453129929 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff22c010440 con 0x7ff23009fa20 2024-09-15T01:33:05.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.452+0000 7ff2257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:05.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.452+0000 7ff2257fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:05.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.605+0000 7ff237657640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-09-15T01:33:05.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.605+0000 7ff237657640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-09-15T01:33:05.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.605+0000 7ff237657640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:05.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:05.605+0000 7ff237657640 1 -- 172.21.15.204:0/3453129929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"} v 0) v1 -- 0x7ff230070b80 con 0x7ff23009fa20 2024-09-15T01:33:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:06.256+0000 7ff2257fa640 1 -- 172.21.15.204:0/3453129929 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}]=0 set pool 3 pg_num_min to 2 v22) v1 ==== 143+0+0 (secure 0 0 0) 0x7ff22c00d370 con 0x7ff23009fa20 2024-09-15T01:33:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:06.256+0000 7ff2257fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-09-15T01:33:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:06.256+0000 7ff2257fa640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_min to 2 2024-09-15T01:33:06.357 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:06.354+0000 7ff237657640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-09-15T01:33:06.357 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:06.354+0000 7ff237657640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-09-15T01:33:06.357 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:06.354+0000 7ff237657640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:06.357 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:06.354+0000 7ff237657640 1 -- 172.21.15.204:0/3453129929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"} v 0) v1 -- 0x7ff23011e060 con 0x7ff23009fa20 2024-09-15T01:33:07.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.253+0000 7ff2257fa640 1 -- 172.21.15.204:0/3453129929 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}]=0 set pool 3 pg_num_min to 2 v23) v1 ==== 143+0+0 (secure 0 0 0) 0x7ff22c00eb30 con 0x7ff23009fa20 2024-09-15T01:33:07.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.253+0000 7ff2257fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-09-15T01:33:07.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.253+0000 7ff2257fa640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_min to 2 2024-09-15T01:33:07.256 INFO:tasks.workunit.client.0.smithi204.stderr:set pool 3 pg_num_min to 2 2024-09-15T01:33:07.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 1 -- 172.21.15.204:0/3453129929 >> v2:172.21.15.204:6812/28210 conn(0x7ff214039710 msgr2=0x7ff21403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:07.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 1 --2- 172.21.15.204:0/3453129929 >> v2:172.21.15.204:6812/28210 conn(0x7ff214039710 0x7ff21403bbc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff23009ae20 tx=0x7ff220008040 comp rx=0 tx=0).stop 2024-09-15T01:33:07.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 10 monclient: shutdown 2024-09-15T01:33:07.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:07.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 1 -- 172.21.15.204:0/3453129929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff23009fa20 msgr2=0x7ff230142290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:07.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 1 --2- 172.21.15.204:0/3453129929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff23009fa20 0x7ff230142290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00b5f0 tx=0x7ff22c009120 comp rx=0 tx=0).stop 2024-09-15T01:33:07.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 1 -- 172.21.15.204:0/3453129929 shutdown_connections 2024-09-15T01:33:07.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 1 --2- 172.21.15.204:0/3453129929 >> v2:172.21.15.204:6812/28210 conn(0x7ff214039710 0x7ff21403bbc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 1 --2- 172.21.15.204:0/3453129929 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff2300a2cd0 0x7ff23009b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 1 --2- 172.21.15.204:0/3453129929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2300a08e0 0x7ff23009aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 1 --2- 172.21.15.204:0/3453129929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff23009fa20 0x7ff230142290 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 1 -- 172.21.15.204:0/3453129929 >> 172.21.15.204:0/3453129929 conn(0x7ff23006e860 msgr2=0x7ff2300a5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:07.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 1 -- 172.21.15.204:0/3453129929 shutdown_connections 2024-09-15T01:33:07.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.256+0000 7ff237657640 1 -- 172.21.15.204:0/3453129929 wait complete. 2024-09-15T01:33:07.269 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool get testpool1234 pg_num_min 2024-09-15T01:33:07.269 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 2 2024-09-15T01:33:07.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.346+0000 7fc8ea95d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:07.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.346+0000 7fc8ea95d640 10 monclient: build_initial_monmap 2024-09-15T01:33:07.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.346+0000 7fc8ea95d640 10 monclient: monmap: 2024-09-15T01:33:07.349 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:07.349 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:07.349 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:07.347955+0000 2024-09-15T01:33:07.349 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:07.347955+0000 2024-09-15T01:33:07.349 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:07.349 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:07.349 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:07.350 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:07.350 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:07.350 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:07.350 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:07.350 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:07.350 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:07.350 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:07.350 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:07.350 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:07.350 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:07.350 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:07.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 1 Processor -- start 2024-09-15T01:33:07.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 1 -- start start 2024-09-15T01:33:07.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 10 monclient: init 2024-09-15T01:33:07.359 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:07.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 10 monclient: _add_conns ranks=[0,10,3,8,7,5,6,1,2,9,4] 2024-09-15T01:33:07.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc8e40a2e30 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:07.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc8e409fb80 0x7fc8e409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 10 monclient(hunting): picked mon.noname-f con 0x7fc8e409fb80 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:07.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8e4137d20 0x7fc8e413a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 10 monclient(hunting): picked mon.noname-g con 0x7fc8e4137d20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:07.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e404b720 con 0x7fc8e40a2e30 2024-09-15T01:33:07.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:07.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e409e6c0 con 0x7fc8e409fb80 2024-09-15T01:33:07.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:07.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40a04b0 con 0x7fc8e4137d20 2024-09-15T01:33:07.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:07.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8ea95d640 10 monclient(hunting): authenticate will time out at 802.990779s 2024-09-15T01:33:07.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8e8efa640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8e4137d20 msgr2=0x7fc8e413a100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:07.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8e8efa640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8e4137d20 0x7fc8e413a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:07.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8e37fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc8e409fb80 0x7fc8e409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:07.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8e37fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc8e409fb80 0x7fc8e409ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:34414/0 (socket says 172.21.15.204:34414) 2024-09-15T01:33:07.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8e37fe640 1 -- 172.21.15.204:0/2912478946 learned_addr learned my addr 172.21.15.204:0/2912478946 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:07.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8e37fe640 10 monclient(hunting): get_auth_request con 0x7fc8e409fb80 auth_method 0 2024-09-15T01:33:07.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:07.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:07.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.347+0000 7fc8e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:07.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 10 monclient(hunting): handle_auth_done global_id 4654 payload 274 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 1 -- 172.21.15.204:0/2912478946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8e4137d20 msgr2=0x7fc8e413a100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 1 --2- 172.21.15.204:0/2912478946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8e4137d20 0x7fc8e413a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 1 -- 172.21.15.204:0/2912478946 >> v1:172.21.15.178:6789/0 conn(0x7fc8e40a2e30 legacy=0x7fc8e4132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 1 monclient: found mon.noname-f 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 1 -- 172.21.15.204:0/2912478946 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e4133170 con 0x7fc8e409fb80 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.348+0000 7fc8e37fe640 1 --2- 172.21.15.204:0/2912478946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e409fb80 0x7fc8e409ff70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009910 tx=0x7fc8d80091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d875158c7c07bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:07.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8e2ffd640 1 -- 172.21.15.204:0/2912478946 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8d802ad40 con 0x7fc8e409fb80 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8e2ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8e2ffd640 10 monclient: dump: 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:07.363 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8e2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8e2ffd640 1 -- 172.21.15.204:0/2912478946 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d8031820 con 0x7fc8e409fb80 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 5 monclient: authenticate success, global_id 4654 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8e2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:07.364 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8e2ffd640 1 -- 172.21.15.204:0/2912478946 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8d8031e90 con 0x7fc8e409fb80 2024-09-15T01:33:07.365 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 4 set_mon_vals no callback set 2024-09-15T01:33:07.365 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:07.365 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8e2ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:07.365 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8e2ffd640 10 monclient: dump: 2024-09-15T01:33:07.365 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:07.365 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:07.365 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.365 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.365 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:07.365 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:07.365 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:07.365 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 10 monclient: shutdown 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 1 -- 172.21.15.204:0/2912478946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e409fb80 msgr2=0x7fc8e409ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 1 --2- 172.21.15.204:0/2912478946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e409fb80 0x7fc8e409ff70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009910 tx=0x7fc8d80091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 1 -- 172.21.15.204:0/2912478946 shutdown_connections 2024-09-15T01:33:07.366 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 1 --2- 172.21.15.204:0/2912478946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8e4137d20 0x7fc8e413a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 1 --2- 172.21.15.204:0/2912478946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e409fb80 0x7fc8e409ff70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 1 -- 172.21.15.204:0/2912478946 >> 172.21.15.204:0/2912478946 conn(0x7fc8e406e860 msgr2=0x7fc8e406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 1 -- 172.21.15.204:0/2912478946 shutdown_connections 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.349+0000 7fc8ea95d640 1 -- 172.21.15.204:0/2912478946 wait complete. 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.350+0000 7fc8ea95d640 10 monclient: build_initial_monmap 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.350+0000 7fc8ea95d640 10 monclient: monmap: 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:07.367 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.350+0000 7fc8ea95d640 1 Processor -- start 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.350+0000 7fc8ea95d640 1 -- start start 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.350+0000 7fc8ea95d640 10 monclient: init 2024-09-15T01:33:07.368 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:07.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient: _add_conns ranks=[4,5,7,3,2,6,0,1,8] 2024-09-15T01:33:07.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc8e409fb80 0x7fc8e4099160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient(hunting): picked mon.noname-c con 0x7fc8e409fb80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:07.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc8e40a2e30 0x7fc8e40996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient(hunting): picked mon.noname-h con 0x7fc8e40a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:07.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8e4137d20 0x7fc8e4099be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient(hunting): picked mon.noname-i con 0x7fc8e4137d20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:07.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:07.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e409c7d0 con 0x7fc8e409fb80 2024-09-15T01:33:07.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:07.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e409ca40 con 0x7fc8e40a2e30 2024-09-15T01:33:07.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e409b220 con 0x7fc8e4137d20 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient(hunting): authenticate will time out at 802.994114s 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e37fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc8e40a2e30 0x7fc8e40996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8e4137d20 0x7fc8e4099be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e37fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc8e40a2e30 0x7fc8e40996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:50638/0 (socket says 172.21.15.204:50638) 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e37fe640 1 -- 172.21.15.204:0/3865698787 learned_addr learned my addr 172.21.15.204:0/3865698787 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 10 monclient(hunting): get_auth_request con 0x7fc8e4137d20 auth_method 0 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e37fe640 10 monclient(hunting): get_auth_request con 0x7fc8e40a2e30 auth_method 0 2024-09-15T01:33:07.370 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e3fff640 1 --2- 172.21.15.204:0/3865698787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc8e409fb80 0x7fc8e4099160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e3fff640 10 monclient(hunting): get_auth_request con 0x7fc8e409fb80 auth_method 0 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:07.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 10 monclient(hunting): handle_auth_done global_id 4543 payload 824 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 1 -- 172.21.15.204:0/3865698787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc8e40a2e30 msgr2=0x7fc8e40996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 1 --2- 172.21.15.204:0/3865698787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc8e40a2e30 0x7fc8e40996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 1 -- 172.21.15.204:0/3865698787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc8e409fb80 msgr2=0x7fc8e4099160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 1 --2- 172.21.15.204:0/3865698787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc8e409fb80 0x7fc8e4099160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 10 monclient: _finish_hunting 0 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 1 monclient: found mon.noname-i 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 1 -- 172.21.15.204:0/3865698787 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d8009450 con 0x7fc8e4137d20 2024-09-15T01:33:07.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e37fe640 1 --2- 172.21.15.204:0/3865698787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc8e40a2e30 0x7fc8e40996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e8efa640 1 --2- 172.21.15.204:0/3865698787 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8e4137d20 0x7fc8e4099be0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00bdf0 tx=0x7fc8dc009650 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8cbfff640 1 -- 172.21.15.204:0/3865698787 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8dc0067e0 con 0x7fc8e4137d20 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8cbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8cbfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8cbfff640 10 monclient: dump: 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:07.373 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8cbfff640 10 monclient: _finish_auth 0 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8cbfff640 1 -- 172.21.15.204:0/3865698787 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc00f070 con 0x7fc8e4137d20 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 5 monclient: authenticate success, global_id 4543 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient: _renew_subs 2024-09-15T01:33:07.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:07.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 1 -- 172.21.15.204:0/3865698787 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e409d650 con 0x7fc8e4137d20 2024-09-15T01:33:07.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8cbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:07.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8cbfff640 1 -- 172.21.15.204:0/3865698787 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8dc006a00 con 0x7fc8e4137d20 2024-09-15T01:33:07.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8cbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:07.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8cbfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:07.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8cbfff640 10 monclient: dump: 2024-09-15T01:33:07.375 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:07.375 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:07.375 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.375 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.375 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:07.375 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient: _renew_subs 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8e27fc640 4 set_mon_vals no callback set 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:07.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.351+0000 7fc8ea95d640 1 -- 172.21.15.204:0/3865698787 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e409ad10 con 0x7fc8e4137d20 2024-09-15T01:33:07.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.352+0000 7fc8cbfff640 1 -- 172.21.15.204:0/3865698787 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc8dc006360 con 0x7fc8e4137d20 2024-09-15T01:33:07.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.352+0000 7fc8ea95d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:07.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.352+0000 7fc8ea95d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:07.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.352+0000 7fc8ea95d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:07.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.352+0000 7fc8ea95d640 1 -- 172.21.15.204:0/3865698787 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e4070cc0 con 0x7fc8e4137d20 2024-09-15T01:33:07.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.352+0000 7fc8cbfff640 1 --2- 172.21.15.204:0/3865698787 >> v2:172.21.15.204:6812/28210 conn(0x7fc8d00397b0 0x7fc8d003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.353+0000 7fc8cbfff640 1 -- 172.21.15.204:0/3865698787 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3869+0+0 (secure 0 0 0) 0x7fc8dc04bc60 con 0x7fc8e4137d20 2024-09-15T01:33:07.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.353+0000 7fc8e3fff640 1 --2- 172.21.15.204:0/3865698787 >> v2:172.21.15.204:6812/28210 conn(0x7fc8d00397b0 0x7fc8d003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:07.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.353+0000 7fc8e3fff640 10 monclient: get_auth_request con 0x7fc8d00397b0 auth_method 0 2024-09-15T01:33:07.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.353+0000 7fc8e3fff640 1 --2- 172.21.15.204:0/3865698787 >> v2:172.21.15.204:6812/28210 conn(0x7fc8d00397b0 0x7fc8d003bc60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4007fd0 tx=0x7fc8d4000bc0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:07.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.357+0000 7fc8cbfff640 1 -- 172.21.15.204:0/3865698787 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc8dc011510 con 0x7fc8e4137d20 2024-09-15T01:33:07.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.357+0000 7fc8cbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:07.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.357+0000 7fc8cbfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:07.513 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.510+0000 7fc8ea95d640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T01:33:07.513 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.510+0000 7fc8ea95d640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T01:33:07.513 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.510+0000 7fc8ea95d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:07.513 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.510+0000 7fc8ea95d640 1 -- 172.21.15.204:0/3865698787 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"} v 0) v1 -- 0x7fc8e4067590 con 0x7fc8e4137d20 2024-09-15T01:33:07.513 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.510+0000 7fc8cbfff640 1 -- 172.21.15.204:0/3865698787 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}]=0 v23) v1 ==== 105+0+14 (secure 0 0 0) 0x7fc8dc01a020 con 0x7fc8e4137d20 2024-09-15T01:33:07.513 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.510+0000 7fc8cbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T01:33:07.514 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.510+0000 7fc8cbfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:07.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.601+0000 7fc8ea95d640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T01:33:07.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.601+0000 7fc8ea95d640 10 monclient: _send_command 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T01:33:07.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.601+0000 7fc8ea95d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:07.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.601+0000 7fc8ea95d640 1 -- 172.21.15.204:0/3865698787 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"} v 0) v1 -- 0x7fc8e409b8e0 con 0x7fc8e4137d20 2024-09-15T01:33:07.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.601+0000 7fc8cbfff640 1 -- 172.21.15.204:0/3865698787 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}]=0 v23) v1 ==== 105+0+14 (secure 0 0 0) 0x7fc8dc013560 con 0x7fc8e4137d20 2024-09-15T01:33:07.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.601+0000 7fc8cbfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T01:33:07.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.601+0000 7fc8cbfff640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:07.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 1 -- 172.21.15.204:0/3865698787 >> v2:172.21.15.204:6812/28210 conn(0x7fc8d00397b0 msgr2=0x7fc8d003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:07.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 1 --2- 172.21.15.204:0/3865698787 >> v2:172.21.15.204:6812/28210 conn(0x7fc8d00397b0 0x7fc8d003bc60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4007fd0 tx=0x7fc8d4000bc0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 10 monclient: shutdown 2024-09-15T01:33:07.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:07.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 1 -- 172.21.15.204:0/3865698787 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8e4137d20 msgr2=0x7fc8e4099be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:07.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 1 --2- 172.21.15.204:0/3865698787 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8e4137d20 0x7fc8e4099be0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00bdf0 tx=0x7fc8dc009650 comp rx=0 tx=0).stop 2024-09-15T01:33:07.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 1 -- 172.21.15.204:0/3865698787 shutdown_connections 2024-09-15T01:33:07.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 1 --2- 172.21.15.204:0/3865698787 >> v2:172.21.15.204:6812/28210 conn(0x7fc8d00397b0 0x7fc8d003bc60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 1 --2- 172.21.15.204:0/3865698787 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8e4137d20 0x7fc8e4099be0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 1 --2- 172.21.15.204:0/3865698787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc8e40a2e30 0x7fc8e40996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 1 --2- 172.21.15.204:0/3865698787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc8e409fb80 0x7fc8e4099160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 1 -- 172.21.15.204:0/3865698787 >> 172.21.15.204:0/3865698787 conn(0x7fc8e406e860 msgr2=0x7fc8e40a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:07.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 1 -- 172.21.15.204:0/3865698787 shutdown_connections 2024-09-15T01:33:07.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.603+0000 7fc8ea95d640 1 -- 172.21.15.204:0/3865698787 wait complete. 2024-09-15T01:33:07.614 INFO:tasks.workunit.client.0.smithi204.stdout:pg_num_min: 2 2024-09-15T01:33:07.614 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set testpool1234 pg_num_max 33 2024-09-15T01:33:07.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.688+0000 7fad5d114640 10 monclient: get_monmap_and_config 2024-09-15T01:33:07.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.688+0000 7fad5d114640 10 monclient: build_initial_monmap 2024-09-15T01:33:07.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.688+0000 7fad5d114640 10 monclient: monmap: 2024-09-15T01:33:07.691 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:07.691 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:07.691 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:07.690013+0000 2024-09-15T01:33:07.691 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:07.690013+0000 2024-09-15T01:33:07.691 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:07.691 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:07.691 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:07.691 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:07.692 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:07.692 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:07.692 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:07.692 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:07.692 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:07.692 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:07.692 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:07.692 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:07.692 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:07.692 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:07.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 1 Processor -- start 2024-09-15T01:33:07.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 1 -- start start 2024-09-15T01:33:07.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 10 monclient: init 2024-09-15T01:33:07.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:07.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 10 monclient: _add_conns ranks=[3,10,9,7,1,0,8,6,5,2,4] 2024-09-15T01:33:07.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad580a2cd0 0x7fad58132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 10 monclient(hunting): picked mon.noname-g con 0x7fad580a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:07.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fad5809fa20 0x7fad5809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 10 monclient(hunting): picked mon.noname-f con 0x7fad5809fa20 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:07.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fad580a08e0 0x7fad580a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 10 monclient(hunting): picked mon.noname-a con 0x7fad580a08e0 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:07.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad580a30a0 con 0x7fad580a08e0 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad580a0330 con 0x7fad5809fa20 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad580a1db0 con 0x7fad580a2cd0 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad5d114640 10 monclient(hunting): authenticate will time out at 803.332801s 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad56d9d640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad580a2cd0 msgr2=0x7fad58132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad56d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad580a2cd0 0x7fad58132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad4ffff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fad5809fa20 0x7fad5809fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad4ffff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fad5809fa20 0x7fad5809fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:34420/0 (socket says 172.21.15.204:34420) 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad4ffff640 1 -- 172.21.15.204:0/3055481148 learned_addr learned my addr 172.21.15.204:0/3055481148 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad4ffff640 10 monclient(hunting): get_auth_request con 0x7fad5809fa20 auth_method 0 2024-09-15T01:33:07.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.689+0000 7fad4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 10 monclient(hunting): handle_auth_done global_id 4663 payload 274 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad5759e640 1 --2- 172.21.15.204:0/3055481148 >> v2:172.21.15.178:3300/0 conn(0x7fad580a08e0 0x7fad580a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 1 -- 172.21.15.204:0/3055481148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad580a2cd0 msgr2=0x7fad58132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 1 --2- 172.21.15.204:0/3055481148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad580a2cd0 0x7fad58132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 1 -- 172.21.15.204:0/3055481148 >> v2:172.21.15.178:3300/0 conn(0x7fad580a08e0 msgr2=0x7fad580a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 1 --2- 172.21.15.204:0/3055481148 >> v2:172.21.15.178:3300/0 conn(0x7fad580a08e0 0x7fad580a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:07.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 1 monclient: found mon.noname-f 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 1 -- 172.21.15.204:0/3055481148 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad580648a0 con 0x7fad5809fa20 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad4ffff640 1 --2- 172.21.15.204:0/3055481148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad5809fa20 0x7fad5809fdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad580a2940 tx=0x7fad400091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac51ca7baa447825 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad5659c640 1 -- 172.21.15.204:0/3055481148 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad4002acc0 con 0x7fad5809fa20 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad5659c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad5659c640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.690+0000 7fad5659c640 10 monclient: dump: 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:07.703 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5659c640 10 monclient: _finish_auth 0 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5659c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5659c640 1 -- 172.21.15.204:0/3055481148 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4002ae40 con 0x7fad5809fa20 2024-09-15T01:33:07.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 5 monclient: authenticate success, global_id 4663 2024-09-15T01:33:07.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:07.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5659c640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:07.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:07.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5659c640 1 -- 172.21.15.204:0/3055481148 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad40031ca0 con 0x7fad5809fa20 2024-09-15T01:33:07.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 4 set_mon_vals no callback set 2024-09-15T01:33:07.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5659c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:07.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5659c640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:07.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5659c640 10 monclient: dump: 2024-09-15T01:33:07.705 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:07.705 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:07.705 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.705 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.706 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:07.706 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:07.706 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:07.706 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:07.706 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:07.706 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:07.706 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:07.706 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:07.706 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:07.706 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:07.706 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:07.706 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 10 monclient: shutdown 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 1 -- 172.21.15.204:0/3055481148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad5809fa20 msgr2=0x7fad5809fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 1 --2- 172.21.15.204:0/3055481148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad5809fa20 0x7fad5809fdf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad580a2940 tx=0x7fad400091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 1 -- 172.21.15.204:0/3055481148 shutdown_connections 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 1 --2- 172.21.15.204:0/3055481148 >> v2:172.21.15.178:3300/0 conn(0x7fad580a08e0 0x7fad580a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 1 --2- 172.21.15.204:0/3055481148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad5809fa20 0x7fad5809fdf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 1 --2- 172.21.15.204:0/3055481148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad580a2cd0 0x7fad58132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 1 -- 172.21.15.204:0/3055481148 >> 172.21.15.204:0/3055481148 conn(0x7fad5806e860 msgr2=0x7fad5806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 1 -- 172.21.15.204:0/3055481148 shutdown_connections 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 1 -- 172.21.15.204:0/3055481148 wait complete. 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.691+0000 7fad5d114640 10 monclient: build_initial_monmap 2024-09-15T01:33:07.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 10 monclient: monmap: 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:07.708 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:07.709 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:07.709 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:07.709 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:07.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 1 Processor -- start 2024-09-15T01:33:07.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 1 -- start start 2024-09-15T01:33:07.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 10 monclient: init 2024-09-15T01:33:07.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:07.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 10 monclient: _add_conns ranks=[0,2,7,4,3,1,5,8,6] 2024-09-15T01:33:07.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad5809fa20 0x7fad58130fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 10 monclient(hunting): picked mon.noname-a con 0x7fad5809fa20 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:07.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad580a08e0 0x7fad5811db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 10 monclient(hunting): picked mon.noname-b con 0x7fad580a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fad580a2cd0 0x7fad5811e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 10 monclient(hunting): picked mon.noname-i con 0x7fad580a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5809cb60 con 0x7fad5809fa20 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5811fe30 con 0x7fad580a08e0 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fad5809d3d0 con 0x7fad580a2cd0 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5d114640 10 monclient(hunting): authenticate will time out at 803.335840s 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5759e640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fad580a2cd0 0x7fad5811e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:07.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5759e640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fad580a2cd0 0x7fad5811e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49504/0 (socket says 172.21.15.204:49504) 2024-09-15T01:33:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5759e640 1 -- 172.21.15.204:0/2208845942 learned_addr learned my addr 172.21.15.204:0/2208845942 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5759e640 10 monclient(hunting): get_auth_request con 0x7fad580a2cd0 auth_method 0 2024-09-15T01:33:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.692+0000 7fad5759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4ffff640 1 --2- 172.21.15.204:0/2208845942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad580a08e0 0x7fad5811db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad56d9d640 1 --2- 172.21.15.204:0/2208845942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad5809fa20 0x7fad58130fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 10 monclient(hunting): handle_auth_done global_id 4552 payload 824 2024-09-15T01:33:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 1 -- 172.21.15.204:0/2208845942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad580a08e0 msgr2=0x7fad5811db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 1 --2- 172.21.15.204:0/2208845942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad580a08e0 0x7fad5811db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 1 -- 172.21.15.204:0/2208845942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad5809fa20 msgr2=0x7fad58130fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 1 --2- 172.21.15.204:0/2208845942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad5809fa20 0x7fad58130fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 10 monclient: _finish_hunting 0 2024-09-15T01:33:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 1 monclient: found mon.noname-i 2024-09-15T01:33:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 1 -- 172.21.15.204:0/2208845942 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad40009450 con 0x7fad580a2cd0 2024-09-15T01:33:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad56d9d640 10 monclient: get_auth_request con 0x7fad5809fa20 auth_method 0 2024-09-15T01:33:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad56d9d640 1 --2- 172.21.15.204:0/2208845942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad5809fa20 0x7fad58130fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:33:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5759e640 1 --2- 172.21.15.204:0/2208845942 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fad580a2cd0 0x7fad5811e050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad5000ceb0 tx=0x7fad50007870 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4effd640 1 -- 172.21.15.204:0/2208845942 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad500079b0 con 0x7fad580a2cd0 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4effd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4effd640 10 monclient: dump: 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4effd640 10 monclient: _finish_auth 0 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4effd640 1 -- 172.21.15.204:0/2208845942 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad500168a0 con 0x7fad580a2cd0 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5d114640 5 monclient: authenticate success, global_id 4552 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4effd640 1 -- 172.21.15.204:0/2208845942 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad5000e400 con 0x7fad580a2cd0 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad4effd640 10 monclient: dump: 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:07.715 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:07.716 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad55d9b640 4 set_mon_vals no callback set 2024-09-15T01:33:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5d114640 10 monclient: _renew_subs 2024-09-15T01:33:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5d114640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5d114640 1 -- 172.21.15.204:0/2208845942 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad58120240 con 0x7fad580a2cd0 2024-09-15T01:33:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5d114640 10 monclient: _renew_subs 2024-09-15T01:33:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5d114640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.693+0000 7fad5d114640 1 -- 172.21.15.204:0/2208845942 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad58142810 con 0x7fad580a2cd0 2024-09-15T01:33:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.694+0000 7fad5d114640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.694+0000 7fad4effd640 1 -- 172.21.15.204:0/2208845942 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fad50003020 con 0x7fad580a2cd0 2024-09-15T01:33:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.694+0000 7fad5d114640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:07.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.694+0000 7fad5d114640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:07.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.694+0000 7fad5d114640 1 -- 172.21.15.204:0/2208845942 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad58070b80 con 0x7fad580a2cd0 2024-09-15T01:33:07.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.694+0000 7fad4effd640 1 --2- 172.21.15.204:0/2208845942 >> v2:172.21.15.204:6812/28210 conn(0x7fad380397b0 0x7fad3803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:07.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.694+0000 7fad4effd640 1 -- 172.21.15.204:0/2208845942 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3869+0+0 (secure 0 0 0) 0x7fad50012070 con 0x7fad580a2cd0 2024-09-15T01:33:07.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.694+0000 7fad56d9d640 1 --2- 172.21.15.204:0/2208845942 >> v2:172.21.15.204:6812/28210 conn(0x7fad380397b0 0x7fad3803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:07.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.695+0000 7fad56d9d640 10 monclient: get_auth_request con 0x7fad380397b0 auth_method 0 2024-09-15T01:33:07.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.695+0000 7fad56d9d640 1 --2- 172.21.15.204:0/2208845942 >> v2:172.21.15.204:6812/28210 conn(0x7fad380397b0 0x7fad3803bc60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad48003850 tx=0x7fad48008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:07.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.699+0000 7fad4effd640 1 -- 172.21.15.204:0/2208845942 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fad5001d540 con 0x7fad580a2cd0 2024-09-15T01:33:07.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.699+0000 7fad4effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:07.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.699+0000 7fad4effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:07.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.855+0000 7fad5d114640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-09-15T01:33:07.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.855+0000 7fad5d114640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-09-15T01:33:07.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.855+0000 7fad5d114640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:07.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:07.855+0000 7fad5d114640 1 -- 172.21.15.204:0/2208845942 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"} v 0) v1 -- 0x7fad581212d0 con 0x7fad580a2cd0 2024-09-15T01:33:08.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:08.290+0000 7fad4effd640 1 -- 172.21.15.204:0/2208845942 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}]=0 set pool 3 pg_num_max to 33 v24) v1 ==== 145+0+0 (secure 0 0 0) 0x7fad50014b20 con 0x7fad580a2cd0 2024-09-15T01:33:08.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:08.290+0000 7fad4effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-09-15T01:33:08.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:08.290+0000 7fad4effd640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_max to 33 2024-09-15T01:33:08.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:08.384+0000 7fad5d114640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-09-15T01:33:08.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:08.384+0000 7fad5d114640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-09-15T01:33:08.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:08.384+0000 7fad5d114640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:08.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:08.384+0000 7fad5d114640 1 -- 172.21.15.204:0/2208845942 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"} v 0) v1 -- 0x7fad58067450 con 0x7fad580a2cd0 2024-09-15T01:33:09.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.298+0000 7fad4effd640 1 -- 172.21.15.204:0/2208845942 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}]=0 set pool 3 pg_num_max to 33 v25) v1 ==== 145+0+0 (secure 0 0 0) 0x7fad5001d310 con 0x7fad580a2cd0 2024-09-15T01:33:09.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.298+0000 7fad4effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-09-15T01:33:09.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.298+0000 7fad4effd640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_max to 33 2024-09-15T01:33:09.301 INFO:tasks.workunit.client.0.smithi204.stderr:set pool 3 pg_num_max to 33 2024-09-15T01:33:09.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.300+0000 7fad5d114640 1 -- 172.21.15.204:0/2208845942 >> v2:172.21.15.204:6812/28210 conn(0x7fad380397b0 msgr2=0x7fad3803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:09.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.300+0000 7fad5d114640 1 --2- 172.21.15.204:0/2208845942 >> v2:172.21.15.204:6812/28210 conn(0x7fad380397b0 0x7fad3803bc60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad48003850 tx=0x7fad48008040 comp rx=0 tx=0).stop 2024-09-15T01:33:09.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.300+0000 7fad5d114640 10 monclient: shutdown 2024-09-15T01:33:09.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.300+0000 7fad5d114640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:09.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.300+0000 7fad5d114640 1 -- 172.21.15.204:0/2208845942 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fad580a2cd0 msgr2=0x7fad5811e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:09.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.300+0000 7fad5d114640 1 --2- 172.21.15.204:0/2208845942 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fad580a2cd0 0x7fad5811e050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad5000ceb0 tx=0x7fad50007870 comp rx=0 tx=0).stop 2024-09-15T01:33:09.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.300+0000 7fad5d114640 1 -- 172.21.15.204:0/2208845942 shutdown_connections 2024-09-15T01:33:09.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.300+0000 7fad5d114640 1 --2- 172.21.15.204:0/2208845942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad580a08e0 0x7fad5811db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.301+0000 7fad5d114640 1 --2- 172.21.15.204:0/2208845942 >> v2:172.21.15.204:6812/28210 conn(0x7fad380397b0 0x7fad3803bc60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.301+0000 7fad5d114640 1 --2- 172.21.15.204:0/2208845942 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fad580a2cd0 0x7fad5811e050 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.301+0000 7fad5d114640 1 --2- 172.21.15.204:0/2208845942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad5809fa20 0x7fad58130fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.301+0000 7fad5d114640 1 -- 172.21.15.204:0/2208845942 >> 172.21.15.204:0/2208845942 conn(0x7fad5806e860 msgr2=0x7fad580a5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:09.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.301+0000 7fad5d114640 1 -- 172.21.15.204:0/2208845942 shutdown_connections 2024-09-15T01:33:09.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.301+0000 7fad5d114640 1 -- 172.21.15.204:0/2208845942 wait complete. 2024-09-15T01:33:09.313 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool get testpool1234 pg_num_max 2024-09-15T01:33:09.313 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 33 2024-09-15T01:33:09.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.389+0000 7f08f8f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:09.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.389+0000 7f08f8f9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:09.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.389+0000 7f08f8f9d640 10 monclient: monmap: 2024-09-15T01:33:09.392 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:09.392 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:09.392 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:09.390617+0000 2024-09-15T01:33:09.392 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:09.390617+0000 2024-09-15T01:33:09.392 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:09.392 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:09.392 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:09.392 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:09.392 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:09.393 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:09.393 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:09.393 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:09.393 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:09.393 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:09.393 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:09.393 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:09.393 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:09.393 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.389+0000 7f08f8f9d640 1 Processor -- start 2024-09-15T01:33:09.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.389+0000 7f08f8f9d640 1 -- start start 2024-09-15T01:33:09.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.389+0000 7f08f8f9d640 10 monclient: init 2024-09-15T01:33:09.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:09.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 10 monclient: _add_conns ranks=[8,2,9,7,1,4,0,3,5,6,10] 2024-09-15T01:33:09.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f40a4680 0x7f08f409e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f08f40a4680 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:09.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f406eea0 0x7f08f409ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f08f406eea0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:09.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f08f409f2d0 0x7f08f41308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 10 monclient(hunting): picked mon.noname-a con 0x7f08f409f2d0 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:09.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f409f6e0 con 0x7f08f409f2d0 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40a4a50 con 0x7f08f406eea0 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f08f406dc70 con 0x7f08f40a4680 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f8f9d640 10 monclient(hunting): authenticate will time out at 805.033398s 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f40a4680 0x7f08f409e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f2d9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f08f409f2d0 0x7f08f41308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f2d9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f08f409f2d0 0x7f08f41308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:40800/0 (socket says 172.21.15.204:40800) 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f40a4680 0x7f08f409e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:43806/0 (socket says 172.21.15.204:43806) 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f2d9d640 1 -- 172.21.15.204:0/965794808 learned_addr learned my addr 172.21.15.204:0/965794808 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:09.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f2d9d640 10 monclient(hunting): get_auth_request con 0x7f08f409f2d0 auth_method 0 2024-09-15T01:33:09.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:09.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:09.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:09.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f1d9b640 1 --2- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f406eea0 0x7f08f409ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f259c640 10 monclient(hunting): get_auth_request con 0x7f08f40a4680 auth_method 0 2024-09-15T01:33:09.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:09.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:09.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:09.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f1d9b640 10 monclient(hunting): get_auth_request con 0x7f08f406eea0 auth_method 0 2024-09-15T01:33:09.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:09.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:09.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:09.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:09.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:09.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:09.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:09.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:09.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:09.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:09.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:09.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.390+0000 7f08f1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:09.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f2d9d640 10 monclient(hunting): handle_auth_done global_id 4671 payload 274 2024-09-15T01:33:09.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f2d9d640 1 -- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f40a4680 msgr2=0x7f08f409e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:09.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f2d9d640 1 --2- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f40a4680 0x7f08f409e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f2d9d640 1 -- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f406eea0 msgr2=0x7f08f409ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:09.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f2d9d640 1 --2- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f406eea0 0x7f08f409ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f2d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:33:09.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f2d9d640 1 monclient: found mon.noname-a 2024-09-15T01:33:09.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:09.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f2d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:33:09.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f2d9d640 1 -- 172.21.15.204:0/965794808 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f404b720 con 0x7f08f409f2d0 2024-09-15T01:33:09.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f259c640 1 --2- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f40a4680 0x7f08f409e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:09.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f1d9b640 1 --2- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f406eea0 0x7f08f409ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:09.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.391+0000 7f08f2d9d640 1 --2- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f409f2d0 0x7f08f41308a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009940 tx=0x7f08e40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72dcd888b6a670ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:09.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f159a640 1 -- 172.21.15.204:0/965794808 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08e402ac70 con 0x7f08f409f2d0 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f159a640 10 monclient: dump: 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:09.407 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:09.408 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:09.408 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:09.408 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:09.408 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:09.408 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f159a640 10 monclient: _finish_auth 0 2024-09-15T01:33:09.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:09.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f159a640 1 -- 172.21.15.204:0/965794808 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e402adf0 con 0x7f08f409f2d0 2024-09-15T01:33:09.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 5 monclient: authenticate success, global_id 4671 2024-09-15T01:33:09.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:09.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:09.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f159a640 1 -- 172.21.15.204:0/965794808 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08e4031ca0 con 0x7f08f409f2d0 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 4 set_mon_vals no callback set 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f159a640 10 monclient: dump: 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:09.409 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 10 monclient: shutdown 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 1 -- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f409f2d0 msgr2=0x7f08f41308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 1 --2- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f409f2d0 0x7f08f41308a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009940 tx=0x7f08e40091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 1 -- 172.21.15.204:0/965794808 shutdown_connections 2024-09-15T01:33:09.410 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 1 --2- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f409f2d0 0x7f08f41308a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.411 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 1 --2- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f406eea0 0x7f08f409ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.411 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 1 --2- 172.21.15.204:0/965794808 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f40a4680 0x7f08f409e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.411 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 1 -- 172.21.15.204:0/965794808 >> 172.21.15.204:0/965794808 conn(0x7f08f4067740 msgr2=0x7f08f40a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:09.411 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 1 -- 172.21.15.204:0/965794808 shutdown_connections 2024-09-15T01:33:09.411 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.392+0000 7f08f8f9d640 1 -- 172.21.15.204:0/965794808 wait complete. 2024-09-15T01:33:09.411 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.393+0000 7f08f8f9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:09.411 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.393+0000 7f08f8f9d640 10 monclient: monmap: 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:09.412 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.393+0000 7f08f8f9d640 1 Processor -- start 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.393+0000 7f08f8f9d640 1 -- start start 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.393+0000 7f08f8f9d640 10 monclient: init 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 10 monclient: _add_conns ranks=[8,6,2,5,7,1,4,3,0] 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f406eea0 0x7f08f413c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f08f406eea0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f08f409f2d0 0x7f08f413b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 10 monclient(hunting): picked mon.noname-d con 0x7f08f409f2d0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:09.413 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f40a4680 0x7f08f413f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f08f40a4680 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f411ef60 con 0x7f08f40a4680 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f08f413ce40 con 0x7f08f409f2d0 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4120a50 con 0x7f08f406eea0 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f8f9d640 10 monclient(hunting): authenticate will time out at 805.037132s 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f1d9b640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f08f409f2d0 msgr2=0x7f08f413b9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f1d9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f08f409f2d0 0x7f08f413b9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f406eea0 0x7f08f413c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f406eea0 0x7f08f413c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:43814/0 (socket says 172.21.15.204:43814) 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f40a4680 0x7f08f413f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f259c640 1 -- 172.21.15.204:0/2505844298 learned_addr learned my addr 172.21.15.204:0/2505844298 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f259c640 10 monclient(hunting): get_auth_request con 0x7f08f406eea0 auth_method 0 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 10 monclient(hunting): get_auth_request con 0x7f08f40a4680 auth_method 0 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:09.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 10 monclient(hunting): handle_auth_done global_id 4574 payload 824 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 1 -- 172.21.15.204:0/2505844298 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f406eea0 msgr2=0x7f08f413c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 1 --2- 172.21.15.204:0/2505844298 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f406eea0 0x7f08f413c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 1 -- 172.21.15.204:0/2505844298 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f08f409f2d0 msgr2=0x7f08f413b9b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 1 --2- 172.21.15.204:0/2505844298 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f08f409f2d0 0x7f08f413b9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 1 monclient: found mon.noname-b 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 1 -- 172.21.15.204:0/2505844298 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc009390 con 0x7f08f40a4680 2024-09-15T01:33:09.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.394+0000 7f08f2d9d640 1 --2- 172.21.15.204:0/2505844298 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f40a4680 0x7f08f413f850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08e4031000 tx=0x7f08e4032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 1 -- 172.21.15.204:0/2505844298 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08e403b870 con 0x7f08f40a4680 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 10 monclient: dump: 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:09.417 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:09.418 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:09.418 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:09.418 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:09.418 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:09.418 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:09.418 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:09.418 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:09.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:09.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08f8f9d640 5 monclient: authenticate success, global_id 4574 2024-09-15T01:33:09.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08f8f9d640 10 monclient: _renew_subs 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 1 -- 172.21.15.204:0/2505844298 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e403b9f0 con 0x7f08f40a4680 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08f8f9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08f8f9d640 1 -- 172.21.15.204:0/2505844298 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec009a20 con 0x7f08f40a4680 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 1 -- 172.21.15.204:0/2505844298 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08e40313a0 con 0x7f08f40a4680 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 10 monclient: dump: 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:09.419 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:09.420 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:09.420 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:09.420 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:09.420 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:09.420 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:09.420 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:09.420 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:09.420 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:09.420 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:09.420 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08f0d99640 4 set_mon_vals no callback set 2024-09-15T01:33:09.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08f8f9d640 10 monclient: _renew_subs 2024-09-15T01:33:09.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08f8f9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:09.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08f8f9d640 1 -- 172.21.15.204:0/2505844298 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e4009450 con 0x7f08f40a4680 2024-09-15T01:33:09.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08f8f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:09.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08f8f9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:09.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08f8f9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:09.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08f8f9d640 1 -- 172.21.15.204:0/2505844298 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f406ccd0 con 0x7f08f40a4680 2024-09-15T01:33:09.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.395+0000 7f08ea7fc640 1 -- 172.21.15.204:0/2505844298 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f08e4039040 con 0x7f08f40a4680 2024-09-15T01:33:09.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.396+0000 7f08ea7fc640 1 --2- 172.21.15.204:0/2505844298 >> v2:172.21.15.204:6812/28210 conn(0x7f08d40396e0 0x7f08d403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.396+0000 7f08ea7fc640 1 -- 172.21.15.204:0/2505844298 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3885+0+0 (secure 0 0 0) 0x7f08e406aa60 con 0x7f08f40a4680 2024-09-15T01:33:09.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.396+0000 7f08f259c640 1 --2- 172.21.15.204:0/2505844298 >> v2:172.21.15.204:6812/28210 conn(0x7f08d40396e0 0x7f08d403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.396+0000 7f08f259c640 10 monclient: get_auth_request con 0x7f08d40396e0 auth_method 0 2024-09-15T01:33:09.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.396+0000 7f08f259c640 1 --2- 172.21.15.204:0/2505844298 >> v2:172.21.15.204:6812/28210 conn(0x7f08d40396e0 0x7f08d403bb90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00f800 tx=0x7f08ec00b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:09.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.400+0000 7f08ea7fc640 1 -- 172.21.15.204:0/2505844298 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f08e403d650 con 0x7f08f40a4680 2024-09-15T01:33:09.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.400+0000 7f08ea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:09.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.400+0000 7f08ea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:09.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.553+0000 7f08f8f9d640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T01:33:09.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.553+0000 7f08f8f9d640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T01:33:09.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.553+0000 7f08f8f9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:09.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.553+0000 7f08f8f9d640 1 -- 172.21.15.204:0/2505844298 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"} v 0) v1 -- 0x7f08f4070cc0 con 0x7f08f40a4680 2024-09-15T01:33:09.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.554+0000 7f08ea7fc640 1 -- 172.21.15.204:0/2505844298 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}]=0 v25) v1 ==== 105+0+15 (secure 0 0 0) 0x7f08e40420b0 con 0x7f08f40a4680 2024-09-15T01:33:09.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.554+0000 7f08ea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T01:33:09.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.554+0000 7f08ea7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:09.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.649+0000 7f08f8f9d640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T01:33:09.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.649+0000 7f08f8f9d640 10 monclient: _send_command 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T01:33:09.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.649+0000 7f08f8f9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:09.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.649+0000 7f08f8f9d640 1 -- 172.21.15.204:0/2505844298 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"} v 0) v1 -- 0x7f08f409b940 con 0x7f08f40a4680 2024-09-15T01:33:09.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.649+0000 7f08ea7fc640 1 -- 172.21.15.204:0/2505844298 <== mon.2 v2:172.21.15.178:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}]=0 v25) v1 ==== 105+0+15 (secure 0 0 0) 0x7f08e4037070 con 0x7f08f40a4680 2024-09-15T01:33:09.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.649+0000 7f08ea7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T01:33:09.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.649+0000 7f08ea7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:09.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.651+0000 7f08f8f9d640 1 -- 172.21.15.204:0/2505844298 >> v2:172.21.15.204:6812/28210 conn(0x7f08d40396e0 msgr2=0x7f08d403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:09.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 1 --2- 172.21.15.204:0/2505844298 >> v2:172.21.15.204:6812/28210 conn(0x7f08d40396e0 0x7f08d403bb90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00f800 tx=0x7f08ec00b040 comp rx=0 tx=0).stop 2024-09-15T01:33:09.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 10 monclient: shutdown 2024-09-15T01:33:09.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:09.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 1 -- 172.21.15.204:0/2505844298 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f40a4680 msgr2=0x7f08f413f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:09.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 1 --2- 172.21.15.204:0/2505844298 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f40a4680 0x7f08f413f850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08e4031000 tx=0x7f08e4032000 comp rx=0 tx=0).stop 2024-09-15T01:33:09.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 1 -- 172.21.15.204:0/2505844298 shutdown_connections 2024-09-15T01:33:09.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 1 --2- 172.21.15.204:0/2505844298 >> v2:172.21.15.204:6812/28210 conn(0x7f08d40396e0 0x7f08d403bb90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 1 --2- 172.21.15.204:0/2505844298 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f08f409f2d0 0x7f08f413b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 1 --2- 172.21.15.204:0/2505844298 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f40a4680 0x7f08f413f850 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 1 --2- 172.21.15.204:0/2505844298 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f08f406eea0 0x7f08f413c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 1 -- 172.21.15.204:0/2505844298 >> 172.21.15.204:0/2505844298 conn(0x7f08f4067740 msgr2=0x7f08f409d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:09.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 1 -- 172.21.15.204:0/2505844298 shutdown_connections 2024-09-15T01:33:09.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.652+0000 7f08f8f9d640 1 -- 172.21.15.204:0/2505844298 wait complete. 2024-09-15T01:33:09.663 INFO:tasks.workunit.client.0.smithi204.stdout:pg_num_max: 33 2024-09-15T01:33:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool set testpool1234 pg_num_min 9 2024-09-15T01:33:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set testpool1234 pg_num_min 9 2024-09-15T01:33:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.737+0000 7f707422b640 10 monclient: get_monmap_and_config 2024-09-15T01:33:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.737+0000 7f707422b640 10 monclient: build_initial_monmap 2024-09-15T01:33:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.737+0000 7f707422b640 10 monclient: monmap: 2024-09-15T01:33:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:09.738643+0000 2024-09-15T01:33:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:09.738643+0000 2024-09-15T01:33:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.737+0000 7f707422b640 1 Processor -- start 2024-09-15T01:33:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.737+0000 7f707422b640 1 -- start start 2024-09-15T01:33:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.737+0000 7f707422b640 10 monclient: init 2024-09-15T01:33:09.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:09.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 10 monclient: _add_conns ranks=[8,3,7,9,0,5,2,10,4,6,1] 2024-09-15T01:33:09.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f706c0a4520 0x7f706c130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 10 monclient(hunting): picked mon.noname-e con 0x7f706c0a4520 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:09.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f706c06ecf0 0x7f706c130f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 10 monclient(hunting): picked mon.noname-g con 0x7f706c06ecf0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:09.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f706c09e750 0x7f706c09eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 10 monclient(hunting): picked mon.noname-i con 0x7f706c09e750 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706c06de30 con 0x7f706c06ecf0 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f706c06db50 con 0x7f706c09e750 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f706c0a48f0 con 0x7f706c0a4520 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f707422b640 10 monclient(hunting): authenticate will time out at 805.381457s 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70717c6640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f706c06ecf0 msgr2=0x7f706c130f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70717c6640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f706c06ecf0 0x7f706c130f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f706c09e750 0x7f706c09eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f706c09e750 0x7f706c09eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49520/0 (socket says 172.21.15.204:49520) 2024-09-15T01:33:09.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 1 -- 172.21.15.204:0/3672044547 learned_addr learned my addr 172.21.15.204:0/3672044547 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:09.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 10 monclient(hunting): get_auth_request con 0x7f706c09e750 auth_method 0 2024-09-15T01:33:09.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:09.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:09.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:09.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:09.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:09.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f7071fc7640 1 --2- 172.21.15.204:0/3672044547 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f706c0a4520 0x7f706c130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:09.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f7071fc7640 10 monclient(hunting): get_auth_request con 0x7f706c0a4520 auth_method 0 2024-09-15T01:33:09.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f7071fc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:09.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f7071fc7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:09.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f7071fc7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f7071fc7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f7071fc7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f7071fc7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 10 monclient(hunting): handle_auth_done global_id 4561 payload 274 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 1 -- 172.21.15.204:0/3672044547 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f706c0a4520 msgr2=0x7f706c130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 1 --2- 172.21.15.204:0/3672044547 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f706c0a4520 0x7f706c130a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 1 -- 172.21.15.204:0/3672044547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f706c06ecf0 msgr2=0x7f706c130f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 1 --2- 172.21.15.204:0/3672044547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f706c06ecf0 0x7f706c130f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 10 monclient: _finish_hunting 0 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 1 monclient: found mon.noname-i 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.738+0000 7f70727c8640 1 -- 172.21.15.204:0/3672044547 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c0648a0 con 0x7f706c09e750 2024-09-15T01:33:09.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f70727c8640 1 --2- 172.21.15.204:0/3672044547 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f706c09e750 0x7f706c09eba0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7064009f50 tx=0x7f7064009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=a2308c2139c01818 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f7070fc5640 1 -- 172.21.15.204:0/3672044547 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7064007880 con 0x7f706c09e750 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f7070fc5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f7070fc5640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f7070fc5640 10 monclient: dump: 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:09.754 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f7070fc5640 10 monclient: _finish_auth 0 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f7070fc5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f707422b640 5 monclient: authenticate success, global_id 4561 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f707422b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f7070fc5640 1 -- 172.21.15.204:0/3672044547 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706402d050 con 0x7f706c09e750 2024-09-15T01:33:09.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f7070fc5640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f707422b640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f707422b640 4 set_mon_vals no callback set 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f7070fc5640 1 -- 172.21.15.204:0/3672044547 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7064030360 con 0x7f706c09e750 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f7070fc5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f7070fc5640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.739+0000 7f7070fc5640 10 monclient: dump: 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:09.756 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:09.757 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:09.757 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:09.757 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:09.757 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:09.757 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:09.757 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:09.757 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:09.757 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:09.757 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 10 monclient: shutdown 2024-09-15T01:33:09.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:09.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 1 -- 172.21.15.204:0/3672044547 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f706c09e750 msgr2=0x7f706c09eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 1 --2- 172.21.15.204:0/3672044547 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f706c09e750 0x7f706c09eba0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7064009f50 tx=0x7f7064009840 comp rx=0 tx=0).stop 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 1 -- 172.21.15.204:0/3672044547 shutdown_connections 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 1 --2- 172.21.15.204:0/3672044547 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f706c09e750 0x7f706c09eba0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 1 --2- 172.21.15.204:0/3672044547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f706c06ecf0 0x7f706c130f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 1 --2- 172.21.15.204:0/3672044547 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f706c0a4520 0x7f706c130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 1 -- 172.21.15.204:0/3672044547 >> 172.21.15.204:0/3672044547 conn(0x7f706c0676a0 msgr2=0x7f706c0a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 1 -- 172.21.15.204:0/3672044547 shutdown_connections 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 1 -- 172.21.15.204:0/3672044547 wait complete. 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 10 monclient: build_initial_monmap 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.740+0000 7f707422b640 10 monclient: monmap: 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:09.758 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 1 Processor -- start 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 1 -- start start 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 10 monclient: init 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 10 monclient: _add_conns ranks=[2,7,5,3,1,0,8,4,6] 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f706c06ecf0 0x7f706c11c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 10 monclient(hunting): picked mon.noname-b con 0x7f706c06ecf0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f706c09e750 0x7f706c11c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 10 monclient(hunting): picked mon.noname-i con 0x7f706c09e750 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f706c0a4520 0x7f706c11ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 10 monclient(hunting): picked mon.noname-h con 0x7f706c0a4520 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706c11e4a0 con 0x7f706c06ecf0 2024-09-15T01:33:09.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f706c06cc30 con 0x7f706c0a4520 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f706c12f5f0 con 0x7f706c09e750 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f707422b640 10 monclient(hunting): authenticate will time out at 805.384638s 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70717c6640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f706c09e750 0x7f706c11c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70717c6640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f706c09e750 0x7f706c11c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49524/0 (socket says 172.21.15.204:49524) 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70717c6640 1 -- 172.21.15.204:0/3830324562 learned_addr learned my addr 172.21.15.204:0/3830324562 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70717c6640 10 monclient(hunting): get_auth_request con 0x7f706c09e750 auth_method 0 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70717c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70717c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70717c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:09.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70727c8640 1 --2- 172.21.15.204:0/3830324562 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f706c0a4520 0x7f706c11ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70727c8640 10 monclient(hunting): get_auth_request con 0x7f706c0a4520 auth_method 0 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70727c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70727c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70727c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70717c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70717c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70717c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70727c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70727c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f70727c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f7071fc7640 1 --2- 172.21.15.204:0/3830324562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f706c06ecf0 0x7f706c11c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f7071fc7640 10 monclient(hunting): get_auth_request con 0x7f706c06ecf0 auth_method 0 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f7071fc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:09.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f7071fc7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f7071fc7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f7071fc7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.741+0000 7f7071fc7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7071fc7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f70717c6640 10 monclient(hunting): handle_auth_done global_id 4570 payload 824 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f70717c6640 1 -- 172.21.15.204:0/3830324562 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f706c0a4520 msgr2=0x7f706c11ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f70717c6640 1 --2- 172.21.15.204:0/3830324562 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f706c0a4520 0x7f706c11ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f70717c6640 1 -- 172.21.15.204:0/3830324562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f706c06ecf0 msgr2=0x7f706c11c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f70717c6640 1 --2- 172.21.15.204:0/3830324562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f706c06ecf0 0x7f706c11c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f70717c6640 10 monclient: _finish_hunting 0 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f70717c6640 1 monclient: found mon.noname-i 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f70717c6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f70717c6640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f70717c6640 1 -- 172.21.15.204:0/3830324562 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7064009ae0 con 0x7f706c09e750 2024-09-15T01:33:09.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f70727c8640 1 --2- 172.21.15.204:0/3830324562 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f706c0a4520 0x7f706c11ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f70717c6640 1 --2- 172.21.15.204:0/3830324562 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f706c09e750 0x7f706c11c770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f705c00b5c0 tx=0x7f705c009120 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7061ffb640 1 -- 172.21.15.204:0/3830324562 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f705c006150 con 0x7f706c09e750 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7061ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7061ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7061ffb640 10 monclient: dump: 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:09.764 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7061ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7061ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7061ffb640 1 -- 172.21.15.204:0/3830324562 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c0062d0 con 0x7f706c09e750 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f707422b640 5 monclient: authenticate success, global_id 4570 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f707422b640 10 monclient: _renew_subs 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f707422b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:09.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f707422b640 1 -- 172.21.15.204:0/3830324562 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706c06d010 con 0x7f706c09e750 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7061ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7061ffb640 1 -- 172.21.15.204:0/3830324562 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f705c0103c0 con 0x7f706c09e750 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7061ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7061ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7061ffb640 10 monclient: dump: 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:09.766 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f7063fff640 4 set_mon_vals no callback set 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f707422b640 10 monclient: _renew_subs 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f707422b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.742+0000 7f707422b640 1 -- 172.21.15.204:0/3830324562 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706c11daa0 con 0x7f706c09e750 2024-09-15T01:33:09.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.743+0000 7f7061ffb640 1 -- 172.21.15.204:0/3830324562 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f705c0064c0 con 0x7f706c09e750 2024-09-15T01:33:09.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.743+0000 7f7061ffb640 1 --2- 172.21.15.204:0/3830324562 >> v2:172.21.15.204:6812/28210 conn(0x7f70500397b0 0x7f705003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.743+0000 7f7061ffb640 1 -- 172.21.15.204:0/3830324562 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3885+0+0 (secure 0 0 0) 0x7f705c04a410 con 0x7f706c09e750 2024-09-15T01:33:09.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.743+0000 7f7071fc7640 1 --2- 172.21.15.204:0/3830324562 >> v2:172.21.15.204:6812/28210 conn(0x7f70500397b0 0x7f705003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.743+0000 7f707422b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:09.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.743+0000 7f707422b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:09.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.743+0000 7f707422b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:09.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.743+0000 7f707422b640 1 -- 172.21.15.204:0/3830324562 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706c11f560 con 0x7f706c09e750 2024-09-15T01:33:09.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.743+0000 7f7071fc7640 10 monclient: get_auth_request con 0x7f70500397b0 auth_method 0 2024-09-15T01:33:09.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.743+0000 7f7071fc7640 1 --2- 172.21.15.204:0/3830324562 >> v2:172.21.15.204:6812/28210 conn(0x7f70500397b0 0x7f705003bc60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7068009d20 tx=0x7f70680090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:09.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.747+0000 7f7061ffb640 1 -- 172.21.15.204:0/3830324562 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f705c017070 con 0x7f706c09e750 2024-09-15T01:33:09.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.748+0000 7f7061ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:09.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.748+0000 7f7061ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:09.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.902+0000 7f707422b640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}] 2024-09-15T01:33:09.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.902+0000 7f707422b640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}] 2024-09-15T01:33:09.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.902+0000 7f707422b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:09.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.902+0000 7f707422b640 1 -- 172.21.15.204:0/3830324562 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"} v 0) v1 -- 0x7f706c070b80 con 0x7f706c09e750 2024-09-15T01:33:09.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.903+0000 7f7061ffb640 1 -- 172.21.15.204:0/3830324562 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}]=-22 specified pg_num_min 9 > pg_num 8 v25) v1 ==== 150+0+0 (secure 0 0 0) 0x7f705c006770 con 0x7f706c09e750 2024-09-15T01:33:09.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.903+0000 7f7061ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}] 2024-09-15T01:33:09.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.903+0000 7f7061ffb640 10 monclient: _finish_command 2 = mon:22 specified pg_num_min 9 > pg_num 8 2024-09-15T01:33:09.906 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: specified pg_num_min 9 > pg_num 8 2024-09-15T01:33:09.909 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:09.909 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool set testpool1234 pg_num_max 7 2024-09-15T01:33:09.909 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:09.909 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set testpool1234 pg_num_max 7 2024-09-15T01:33:09.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.981+0000 7f0fa2d5d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:09.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.981+0000 7f0fa2d5d640 10 monclient: build_initial_monmap 2024-09-15T01:33:09.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.981+0000 7f0fa2d5d640 10 monclient: monmap: 2024-09-15T01:33:09.984 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:09.984 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:09.984 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:09.983326+0000 2024-09-15T01:33:09.984 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:09.983326+0000 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:09.985 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:09.986 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:09.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.982+0000 7f0fa2d5d640 1 Processor -- start 2024-09-15T01:33:09.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.982+0000 7f0fa2d5d640 1 -- start start 2024-09-15T01:33:09.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.982+0000 7f0fa2d5d640 10 monclient: init 2024-09-15T01:33:09.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:09.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 10 monclient: _add_conns ranks=[2,7,8,6,5,9,3,10,4,0,1] 2024-09-15T01:33:09.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0a2cd0 0x7f0f9c132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 10 monclient(hunting): picked mon.noname-b con 0x7f0f9c0a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:09.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0f9c09fa20 0x7f0f9c09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 10 monclient(hunting): picked mon.noname-i con 0x7f0f9c09fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:09.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0f9c0a08e0 0x7f0f9c0a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:09.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 10 monclient(hunting): picked mon.noname-e con 0x7f0f9c0a08e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:09.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c0a30a0 con 0x7f0f9c0a2cd0 2024-09-15T01:33:09.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c0a0330 con 0x7f0f9c09fa20 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c0a1db0 con 0x7f0f9c0a08e0 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa2d5d640 10 monclient(hunting): authenticate will time out at 805.626114s 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0f9c09fa20 0x7f0f9c09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0f9c09fa20 0x7f0f9c09fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49530/0 (socket says 172.21.15.204:49530) 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 1 -- 172.21.15.204:0/3114034607 learned_addr learned my addr 172.21.15.204:0/3114034607 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 10 monclient(hunting): get_auth_request con 0x7f0f9c09fa20 auth_method 0 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:09.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:09.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:09.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:09.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa12fa640 1 --2- 172.21.15.204:0/3114034607 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0f9c0a08e0 0x7f0f9c0a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa0af9640 1 --2- 172.21.15.204:0/3114034607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0a2cd0 0x7f0f9c132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:09.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 10 monclient(hunting): handle_auth_done global_id 4579 payload 274 2024-09-15T01:33:09.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 1 -- 172.21.15.204:0/3114034607 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0f9c0a08e0 msgr2=0x7f0f9c0a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:09.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 1 --2- 172.21.15.204:0/3114034607 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0f9c0a08e0 0x7f0f9c0a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 1 -- 172.21.15.204:0/3114034607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0a2cd0 msgr2=0x7f0f9c132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:09.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 1 --2- 172.21.15.204:0/3114034607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0a2cd0 0x7f0f9c132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:09.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:09.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 1 monclient: found mon.noname-i 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0f9bfff640 1 -- 172.21.15.204:0/3114034607 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c0648a0 con 0x7f0f9c09fa20 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa12fa640 10 monclient: get_auth_request con 0x7f0f9c0a08e0 auth_method 0 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa12fa640 1 --2- 172.21.15.204:0/3114034607 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0f9c0a08e0 0x7f0f9c0a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa0af9640 10 monclient: get_auth_request con 0x7f0f9c0a2cd0 auth_method 0 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.983+0000 7f0fa0af9640 1 --2- 172.21.15.204:0/3114034607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0a2cd0 0x7f0f9c132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9bfff640 1 --2- 172.21.15.204:0/3114034607 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0f9c09fa20 0x7f0f9c09fdf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0f90009940 tx=0x7f0f900091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=b8b5c44de8017334 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9b7fe640 1 -- 172.21.15.204:0/3114034607 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f9002ac60 con 0x7f0f9c09fa20 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9b7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9b7fe640 10 monclient: dump: 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:09.998 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:09.999 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.999 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:09.999 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:09.999 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:09.999 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:09.999 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:09.999 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:09.999 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:09.999 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:09.999 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:09.999 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:09.999 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9b7fe640 1 -- 172.21.15.204:0/3114034607 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9002ade0 con 0x7f0f9c09fa20 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0fa2d5d640 5 monclient: authenticate success, global_id 4579 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0fa2d5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9b7fe640 1 -- 172.21.15.204:0/3114034607 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f90031ca0 con 0x7f0f9c09fa20 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0f9b7fe640 10 monclient: dump: 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0fa2d5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0fa2d5d640 4 set_mon_vals no callback set 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0fa2d5d640 10 monclient: shutdown 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0fa2d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0fa2d5d640 1 -- 172.21.15.204:0/3114034607 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0f9c09fa20 msgr2=0x7f0f9c09fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.984+0000 7f0fa2d5d640 1 --2- 172.21.15.204:0/3114034607 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0f9c09fa20 0x7f0f9c09fdf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0f90009940 tx=0x7f0f900091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.985+0000 7f0fa2d5d640 1 -- 172.21.15.204:0/3114034607 shutdown_connections 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.985+0000 7f0fa2d5d640 1 --2- 172.21.15.204:0/3114034607 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0f9c09fa20 0x7f0f9c09fdf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.985+0000 7f0fa2d5d640 1 --2- 172.21.15.204:0/3114034607 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0f9c0a08e0 0x7f0f9c0a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.985+0000 7f0fa2d5d640 1 --2- 172.21.15.204:0/3114034607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f9c0a2cd0 0x7f0f9c132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.985+0000 7f0fa2d5d640 1 -- 172.21.15.204:0/3114034607 >> 172.21.15.204:0/3114034607 conn(0x7f0f9c06e860 msgr2=0x7f0f9c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.985+0000 7f0fa2d5d640 1 -- 172.21.15.204:0/3114034607 shutdown_connections 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.985+0000 7f0fa2d5d640 1 -- 172.21.15.204:0/3114034607 wait complete. 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.985+0000 7f0fa2d5d640 10 monclient: build_initial_monmap 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.985+0000 7f0fa2d5d640 10 monclient: monmap: 2024-09-15T01:33:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:10.003 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.985+0000 7f0fa2d5d640 1 Processor -- start 2024-09-15T01:33:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 1 -- start start 2024-09-15T01:33:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 10 monclient: init 2024-09-15T01:33:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 10 monclient: _add_conns ranks=[1,8,4,7,3,2,6,0,5] 2024-09-15T01:33:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f9c09fa20 0x7f0f9c145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 10 monclient(hunting): picked mon.noname-f con 0x7f0f9c09fa20 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0f9c0a08e0 0x7f0f9c131140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 10 monclient(hunting): picked mon.noname-e con 0x7f0f9c0a08e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0f9c0a2cd0 0x7f0f9c06a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 10 monclient(hunting): picked mon.noname-c con 0x7f0f9c0a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c09d3d0 con 0x7f0f9c09fa20 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c09cb60 con 0x7f0f9c0a2cd0 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c146ad0 con 0x7f0f9c0a08e0 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa2d5d640 10 monclient(hunting): authenticate will time out at 805.629484s 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f9c09fa20 0x7f0f9c145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f9c09fa20 0x7f0f9c145c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:34430/0 (socket says 172.21.15.204:34430) 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 1 -- 172.21.15.204:0/32398532 learned_addr learned my addr 172.21.15.204:0/32398532 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 10 monclient(hunting): get_auth_request con 0x7f0f9c09fa20 auth_method 0 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0f9bfff640 1 --2- 172.21.15.204:0/32398532 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0f9c0a08e0 0x7f0f9c131140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0f9bfff640 10 monclient(hunting): get_auth_request con 0x7f0f9c0a08e0 auth_method 0 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0f9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0f9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0f9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa12fa640 1 --2- 172.21.15.204:0/32398532 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0f9c0a2cd0 0x7f0f9c06a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa12fa640 10 monclient(hunting): get_auth_request con 0x7f0f9c0a2cd0 auth_method 0 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa12fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa12fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa12fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0f9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0f9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0f9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa12fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa12fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa12fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 10 monclient(hunting): handle_auth_done global_id 4672 payload 824 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 1 -- 172.21.15.204:0/32398532 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0f9c0a08e0 msgr2=0x7f0f9c131140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 1 --2- 172.21.15.204:0/32398532 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0f9c0a08e0 0x7f0f9c131140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 1 -- 172.21.15.204:0/32398532 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0f9c0a2cd0 msgr2=0x7f0f9c06a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 1 --2- 172.21.15.204:0/32398532 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0f9c0a2cd0 0x7f0f9c06a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 10 monclient: _finish_hunting 0 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 1 monclient: found mon.noname-f 2024-09-15T01:33:10.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:10.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:10.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.986+0000 7f0fa0af9640 1 -- 172.21.15.204:0/32398532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f90009450 con 0x7f0f9c09fa20 2024-09-15T01:33:10.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0fa0af9640 1 --2- 172.21.15.204:0/32398532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f9c09fa20 0x7f0f9c145c40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00c880 tx=0x7f0f8c008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:10.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f98ff9640 1 -- 172.21.15.204:0/32398532 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f8c00ce10 con 0x7f0f9c09fa20 2024-09-15T01:33:10.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f98ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f98ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:10.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f98ff9640 10 monclient: dump: 2024-09-15T01:33:10.008 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.008 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.008 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.008 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f98ff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f98ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:10.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f98ff9640 1 -- 172.21.15.204:0/32398532 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c007cb0 con 0x7f0f9c09fa20 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0fa2d5d640 5 monclient: authenticate success, global_id 4672 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0fa2d5d640 10 monclient: _renew_subs 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0fa2d5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0fa2d5d640 1 -- 172.21.15.204:0/32398532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9c06bf40 con 0x7f0f9c09fa20 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f98ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f98ff9640 1 -- 172.21.15.204:0/32398532 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f8c007e20 con 0x7f0f9c09fa20 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f98ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f98ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f98ff9640 10 monclient: dump: 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.010 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0fa2d5d640 10 monclient: _renew_subs 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0fa2d5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:10.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0fa2d5d640 1 -- 172.21.15.204:0/32398532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9c06c400 con 0x7f0f9c09fa20 2024-09-15T01:33:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.987+0000 7f0f9affd640 4 set_mon_vals no callback set 2024-09-15T01:33:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.988+0000 7f0f98ff9640 1 -- 172.21.15.204:0/32398532 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0f8c007830 con 0x7f0f9c09fa20 2024-09-15T01:33:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.988+0000 7f0fa2d5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.988+0000 7f0fa2d5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.988+0000 7f0fa2d5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.988+0000 7f0fa2d5d640 1 -- 172.21.15.204:0/32398532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9c070b80 con 0x7f0f9c09fa20 2024-09-15T01:33:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.988+0000 7f0f98ff9640 1 --2- 172.21.15.204:0/32398532 >> v2:172.21.15.204:6812/28210 conn(0x7f0f880397b0 0x7f0f8803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.988+0000 7f0f98ff9640 1 -- 172.21.15.204:0/32398532 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3885+0+0 (secure 0 0 0) 0x7f0f8c04b140 con 0x7f0f9c09fa20 2024-09-15T01:33:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.988+0000 7f0f9bfff640 1 --2- 172.21.15.204:0/32398532 >> v2:172.21.15.204:6812/28210 conn(0x7f0f880397b0 0x7f0f8803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.988+0000 7f0f9bfff640 10 monclient: get_auth_request con 0x7f0f880397b0 auth_method 0 2024-09-15T01:33:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.988+0000 7f0f9bfff640 1 --2- 172.21.15.204:0/32398532 >> v2:172.21.15.204:6812/28210 conn(0x7f0f880397b0 0x7f0f8803bc60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f90034000 tx=0x7f0f90032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:10.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.992+0000 7f0f98ff9640 1 -- 172.21.15.204:0/32398532 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0f8c0113f0 con 0x7f0f9c09fa20 2024-09-15T01:33:10.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.992+0000 7f0f98ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:10.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:09.992+0000 7f0f98ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:10.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.145+0000 7f0fa2d5d640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}] 2024-09-15T01:33:10.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.145+0000 7f0fa2d5d640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}] 2024-09-15T01:33:10.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.145+0000 7f0fa2d5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:10.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.145+0000 7f0fa2d5d640 1 -- 172.21.15.204:0/32398532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"} v 0) v1 -- 0x7f0f9c067450 con 0x7f0f9c09fa20 2024-09-15T01:33:10.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.146+0000 7f0f98ff9640 1 -- 172.21.15.204:0/32398532 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}]=-22 specified pg_num_max 7 < pg_num 8 v25) v1 ==== 150+0+0 (secure 0 0 0) 0x7f0f8c01a450 con 0x7f0f9c09fa20 2024-09-15T01:33:10.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.146+0000 7f0f98ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}] 2024-09-15T01:33:10.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.146+0000 7f0f98ff9640 10 monclient: _finish_command 2 = mon:22 specified pg_num_max 7 < pg_num 8 2024-09-15T01:33:10.150 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: specified pg_num_max 7 < pg_num 8 2024-09-15T01:33:10.153 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:10.153 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool set testpool1234 pg_num 1 2024-09-15T01:33:10.153 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:10.153 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set testpool1234 pg_num 1 2024-09-15T01:33:10.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.226+0000 7f156dd9d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:10.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.226+0000 7f156dd9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:10.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.226+0000 7f156dd9d640 10 monclient: monmap: 2024-09-15T01:33:10.228 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:10.227676+0000 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:10.227676+0000 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:10.229 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:10.230 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:10.230 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:10.230 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.226+0000 7f156dd9d640 1 Processor -- start 2024-09-15T01:33:10.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 1 -- start start 2024-09-15T01:33:10.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 10 monclient: init 2024-09-15T01:33:10.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:10.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 10 monclient: _add_conns ranks=[1,7,2,3,9,5,0,10,6,8,4] 2024-09-15T01:33:10.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f15680a2cd0 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:10.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f156809fa20 0x7f156809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 10 monclient(hunting): picked mon.noname-i con 0x7f156809fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:10.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1568137c80 0x7f156813a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 10 monclient(hunting): picked mon.noname-b con 0x7f1568137c80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:10.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15680648a0 con 0x7f15680a2cd0 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1568133310 con 0x7f1568137c80 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f15680a30a0 con 0x7f156809fa20 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f156dd9d640 10 monclient(hunting): authenticate will time out at 805.870611s 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f156809fa20 0x7f156809fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f15677fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f15680a2cd0 0x7f1568132c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:58098/0 (socket says 172.21.15.204:58098) 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f15677fe640 1 -- 172.21.15.204:0/2003779477 learned_addr learned my addr 172.21.15.204:0/2003779477 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 10 monclient(hunting): get_auth_request con 0x7f156809fa20 auth_method 0 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f15667fc640 1 -- 172.21.15.204:0/2003779477 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442917381 0 0) 0x7f15680648a0 con 0x7f15680a2cd0 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f15667fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f15667fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f15667fc640 10 monclient(hunting): my global_id is 4681 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1567fff640 1 --2- 172.21.15.204:0/2003779477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1568137c80 0x7f156813a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f15667fc640 1 -- 172.21.15.204:0/2003779477 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15500034e0 con 0x7f15680a2cd0 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 10 monclient(hunting): handle_auth_done global_id 4588 payload 274 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 1 -- 172.21.15.204:0/2003779477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1568137c80 msgr2=0x7f156813a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 1 --2- 172.21.15.204:0/2003779477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1568137c80 0x7f156813a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 1 -- 172.21.15.204:0/2003779477 >> v1:172.21.15.204:6789/0 conn(0x7f15680a2cd0 legacy=0x7f1568132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:10.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 1 monclient: found mon.noname-i 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1566ffd640 1 -- 172.21.15.204:0/2003779477 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15680a0330 con 0x7f156809fa20 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1567fff640 10 monclient: get_auth_request con 0x7f1568137c80 auth_method 0 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.227+0000 7f1567fff640 1 --2- 172.21.15.204:0/2003779477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1568137c80 0x7f156813a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f1566ffd640 1 --2- 172.21.15.204:0/2003779477 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f156809fa20 0x7f156809fdf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f155c009880 tx=0x7f155c0090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=943ebd917526a42a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f15667fc640 1 -- 172.21.15.204:0/2003779477 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f155c02ac50 con 0x7f156809fa20 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f15667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f15667fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f15667fc640 10 monclient: dump: 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.241 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f15667fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f15667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f15667fc640 1 -- 172.21.15.204:0/2003779477 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c02add0 con 0x7f156809fa20 2024-09-15T01:33:10.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f156dd9d640 5 monclient: authenticate success, global_id 4588 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f156dd9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f15667fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f156dd9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f15667fc640 1 -- 172.21.15.204:0/2003779477 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f155c031ca0 con 0x7f156809fa20 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f15667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f156dd9d640 4 set_mon_vals no callback set 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f15667fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.228+0000 7f15667fc640 10 monclient: dump: 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.243 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.229+0000 7f156dd9d640 10 monclient: shutdown 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.229+0000 7f156dd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.229+0000 7f156dd9d640 1 -- 172.21.15.204:0/2003779477 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f156809fa20 msgr2=0x7f156809fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:10.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.229+0000 7f156dd9d640 1 --2- 172.21.15.204:0/2003779477 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f156809fa20 0x7f156809fdf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f155c009880 tx=0x7f155c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.229+0000 7f156dd9d640 1 -- 172.21.15.204:0/2003779477 shutdown_connections 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.229+0000 7f156dd9d640 1 --2- 172.21.15.204:0/2003779477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1568137c80 0x7f156813a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.229+0000 7f156dd9d640 1 --2- 172.21.15.204:0/2003779477 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f156809fa20 0x7f156809fdf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.229+0000 7f156dd9d640 1 -- 172.21.15.204:0/2003779477 >> 172.21.15.204:0/2003779477 conn(0x7f156806e860 msgr2=0x7f156806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.229+0000 7f156dd9d640 1 -- 172.21.15.204:0/2003779477 shutdown_connections 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.229+0000 7f156dd9d640 1 -- 172.21.15.204:0/2003779477 wait complete. 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.229+0000 7f156dd9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.229+0000 7f156dd9d640 10 monclient: monmap: 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:10.245 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 1 Processor -- start 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 1 -- start start 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 10 monclient: init 2024-09-15T01:33:10.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 10 monclient: _add_conns ranks=[6,5,4,2,8,7,3,0,1] 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f156809fa20 0x7f156809c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 10 monclient(hunting): picked mon.noname-d con 0x7f156809fa20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f15680a2cd0 0x7f1568133f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 10 monclient(hunting): picked mon.noname-h con 0x7f15680a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1568137c80 0x7f1568134450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 10 monclient(hunting): picked mon.noname-c con 0x7f1568137c80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1568096bf0 con 0x7f1568137c80 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f15681364a0 con 0x7f15680a2cd0 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f156809cb60 con 0x7f156809fa20 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f156dd9d640 10 monclient(hunting): authenticate will time out at 805.873898s 2024-09-15T01:33:10.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f15677fe640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f156809fa20 msgr2=0x7f156809c540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.230+0000 7f15677fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f156809fa20 0x7f156809c540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f15680a2cd0 0x7f1568133f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f15680a2cd0 0x7f1568133f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:50650/0 (socket says 172.21.15.204:50650) 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 1 -- 172.21.15.204:0/4278432402 learned_addr learned my addr 172.21.15.204:0/4278432402 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 10 monclient(hunting): get_auth_request con 0x7f15680a2cd0 auth_method 0 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1567fff640 1 --2- 172.21.15.204:0/4278432402 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1568137c80 0x7f1568134450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1567fff640 10 monclient(hunting): get_auth_request con 0x7f1568137c80 auth_method 0 2024-09-15T01:33:10.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1567fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1567fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1567fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1567fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1567fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1567fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 10 monclient(hunting): handle_auth_done global_id 4622 payload 824 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 1 -- 172.21.15.204:0/4278432402 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f156809fa20 msgr2=0x7f156809c540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 1 --2- 172.21.15.204:0/4278432402 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f156809fa20 0x7f156809c540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 1 -- 172.21.15.204:0/4278432402 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1568137c80 msgr2=0x7f1568134450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 1 --2- 172.21.15.204:0/4278432402 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1568137c80 0x7f1568134450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 1 monclient: found mon.noname-h 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:10.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 1 -- 172.21.15.204:0/4278432402 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c009390 con 0x7f15680a2cd0 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1566ffd640 1 --2- 172.21.15.204:0/4278432402 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f15680a2cd0 0x7f1568133f10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f155c009720 tx=0x7f155c032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f154f7fe640 1 -- 172.21.15.204:0/4278432402 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f155c03b870 con 0x7f15680a2cd0 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f154f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f154f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f154f7fe640 10 monclient: dump: 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.250 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f154f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f154f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f154f7fe640 1 -- 172.21.15.204:0/4278432402 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c03b9f0 con 0x7f15680a2cd0 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f156dd9d640 5 monclient: authenticate success, global_id 4622 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f154f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f156dd9d640 10 monclient: _renew_subs 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f154f7fe640 1 -- 172.21.15.204:0/4278432402 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f155c0313a0 con 0x7f15680a2cd0 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f156dd9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f156dd9d640 1 -- 172.21.15.204:0/4278432402 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156809ba30 con 0x7f15680a2cd0 2024-09-15T01:33:10.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f154f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f154f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f1565ffb640 4 set_mon_vals no callback set 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f154f7fe640 10 monclient: dump: 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.252 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f156dd9d640 10 monclient: _renew_subs 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f156dd9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.231+0000 7f156dd9d640 1 -- 172.21.15.204:0/4278432402 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1568135aa0 con 0x7f15680a2cd0 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.232+0000 7f154f7fe640 1 -- 172.21.15.204:0/4278432402 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f155c039040 con 0x7f15680a2cd0 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.232+0000 7f156dd9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.232+0000 7f156dd9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.232+0000 7f156dd9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.232+0000 7f156dd9d640 1 -- 172.21.15.204:0/4278432402 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1568070b80 con 0x7f15680a2cd0 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.232+0000 7f154f7fe640 1 --2- 172.21.15.204:0/4278432402 >> v2:172.21.15.204:6812/28210 conn(0x7f15540397b0 0x7f155403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.232+0000 7f154f7fe640 1 -- 172.21.15.204:0/4278432402 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3885+0+0 (secure 0 0 0) 0x7f155c06aaf0 con 0x7f15680a2cd0 2024-09-15T01:33:10.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.232+0000 7f15677fe640 1 --2- 172.21.15.204:0/4278432402 >> v2:172.21.15.204:6812/28210 conn(0x7f15540397b0 0x7f155403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.232+0000 7f15677fe640 10 monclient: get_auth_request con 0x7f15540397b0 auth_method 0 2024-09-15T01:33:10.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.233+0000 7f15677fe640 1 --2- 172.21.15.204:0/4278432402 >> v2:172.21.15.204:6812/28210 conn(0x7f15540397b0 0x7f155403bc60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1558001460 tx=0x7f1558001790 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:10.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.237+0000 7f154f7fe640 1 -- 172.21.15.204:0/4278432402 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f155c030570 con 0x7f15680a2cd0 2024-09-15T01:33:10.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.237+0000 7f154f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:10.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.237+0000 7f154f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:10.397 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.395+0000 7f156dd9d640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}] 2024-09-15T01:33:10.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.395+0000 7f156dd9d640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}] 2024-09-15T01:33:10.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.395+0000 7f156dd9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:10.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.395+0000 7f156dd9d640 1 -- 172.21.15.204:0/4278432402 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"} v 0) v1 -- 0x7f15680008d0 con 0x7f15680a2cd0 2024-09-15T01:33:10.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.395+0000 7f154f7fe640 1 -- 172.21.15.204:0/4278432402 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}]=-22 specified pg_num 1 < pg_num_min 2 v25) v1 ==== 146+0+0 (secure 0 0 0) 0x7f155c045b80 con 0x7f15680a2cd0 2024-09-15T01:33:10.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.395+0000 7f154f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}] 2024-09-15T01:33:10.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.395+0000 7f154f7fe640 10 monclient: _finish_command 2 = mon:22 specified pg_num 1 < pg_num_min 2 2024-09-15T01:33:10.398 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: specified pg_num 1 < pg_num_min 2 2024-09-15T01:33:10.401 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:10.401 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool set testpool1234 pg_num 44 2024-09-15T01:33:10.401 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:10.401 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set testpool1234 pg_num 44 2024-09-15T01:33:10.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.474+0000 7fd15a19d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:10.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.474+0000 7fd15a19d640 10 monclient: build_initial_monmap 2024-09-15T01:33:10.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.474+0000 7fd15a19d640 10 monclient: monmap: 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:10.475839+0000 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:10.475839+0000 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:10.477 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:10.478 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:10.478 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:10.478 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:10.478 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:10.478 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 1 Processor -- start 2024-09-15T01:33:10.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 1 -- start start 2024-09-15T01:33:10.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 10 monclient: init 2024-09-15T01:33:10.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:10.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 10 monclient: _add_conns ranks=[2,9,6,5,3,10,7,1,4,0,8] 2024-09-15T01:33:10.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1540a2cd0 0x7fd154132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 10 monclient(hunting): picked mon.noname-b con 0x7fd1540a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:10.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd15409fa20 0x7fd15409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 10 monclient(hunting): picked mon.noname-a con 0x7fd15409fa20 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd1540a08e0 0x7fd1540a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 10 monclient(hunting): picked mon.noname-d con 0x7fd1540a08e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1540a30a0 con 0x7fd15409fa20 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540a0330 con 0x7fd1540a2cd0 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540a1db0 con 0x7fd1540a08e0 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd15a19d640 10 monclient(hunting): authenticate will time out at 806.118605s 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd153fff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd1540a08e0 msgr2=0x7fd1540a0d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.475+0000 7fd153fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd1540a08e0 0x7fd1540a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1540a2cd0 0x7fd154132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd152ffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd15409fa20 0x7fd15409fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1540a2cd0 0x7fd154132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:35248/0 (socket says 172.21.15.204:35248) 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 1 -- 172.21.15.204:0/2334848879 learned_addr learned my addr 172.21.15.204:0/2334848879 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 10 monclient(hunting): get_auth_request con 0x7fd1540a2cd0 auth_method 0 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd152ffd640 10 monclient(hunting): get_auth_request con 0x7fd15409fa20 auth_method 0 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd152ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd152ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd152ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd152ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd152ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd152ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 10 monclient(hunting): handle_auth_done global_id 4592 payload 274 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 1 -- 172.21.15.204:0/2334848879 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd1540a08e0 msgr2=0x7fd1540a0d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 1 --2- 172.21.15.204:0/2334848879 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd1540a08e0 0x7fd1540a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 1 -- 172.21.15.204:0/2334848879 >> v2:172.21.15.178:3300/0 conn(0x7fd15409fa20 msgr2=0x7fd15409fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 1 --2- 172.21.15.204:0/2334848879 >> v2:172.21.15.178:3300/0 conn(0x7fd15409fa20 0x7fd15409fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 1 monclient: found mon.noname-b 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd1537fe640 1 -- 172.21.15.204:0/2334848879 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1540648a0 con 0x7fd1540a2cd0 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.476+0000 7fd152ffd640 1 --2- 172.21.15.204:0/2334848879 >> v2:172.21.15.178:3300/0 conn(0x7fd15409fa20 0x7fd15409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:10.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1537fe640 1 --2- 172.21.15.204:0/2334848879 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1540a2cd0 0x7fd154132c00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd144009910 tx=0x7fd1440091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e28654b83839cf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1527fc640 1 -- 172.21.15.204:0/2334848879 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd14402acb0 con 0x7fd1540a2cd0 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1527fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1527fc640 10 monclient: dump: 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.491 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1527fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1527fc640 1 -- 172.21.15.204:0/2334848879 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14402ae30 con 0x7fd1540a2cd0 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 5 monclient: authenticate success, global_id 4592 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:10.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1527fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:10.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:10.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1527fc640 1 -- 172.21.15.204:0/2334848879 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd144031ca0 con 0x7fd1540a2cd0 2024-09-15T01:33:10.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 4 set_mon_vals no callback set 2024-09-15T01:33:10.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1527fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:33:10.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd1527fc640 10 monclient: dump: 2024-09-15T01:33:10.493 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.493 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.493 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.493 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.493 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.493 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 10 monclient: shutdown 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 1 -- 172.21.15.204:0/2334848879 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1540a2cd0 msgr2=0x7fd154132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:10.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 1 --2- 172.21.15.204:0/2334848879 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1540a2cd0 0x7fd154132c00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd144009910 tx=0x7fd1440091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 1 -- 172.21.15.204:0/2334848879 shutdown_connections 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 1 --2- 172.21.15.204:0/2334848879 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd1540a08e0 0x7fd1540a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 1 --2- 172.21.15.204:0/2334848879 >> v2:172.21.15.178:3300/0 conn(0x7fd15409fa20 0x7fd15409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 1 --2- 172.21.15.204:0/2334848879 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1540a2cd0 0x7fd154132c00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 1 -- 172.21.15.204:0/2334848879 >> 172.21.15.204:0/2334848879 conn(0x7fd15406e860 msgr2=0x7fd15406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 1 -- 172.21.15.204:0/2334848879 shutdown_connections 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.477+0000 7fd15a19d640 1 -- 172.21.15.204:0/2334848879 wait complete. 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.478+0000 7fd15a19d640 10 monclient: build_initial_monmap 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.478+0000 7fd15a19d640 10 monclient: monmap: 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:10.495 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.478+0000 7fd15a19d640 1 Processor -- start 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.478+0000 7fd15a19d640 1 -- start start 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.478+0000 7fd15a19d640 10 monclient: init 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 10 monclient: _add_conns ranks=[2,7,5,8,1,4,0,3,6] 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15409fa20 0x7fd154131000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 10 monclient(hunting): picked mon.noname-b con 0x7fd15409fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1540a08e0 0x7fd15411db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 10 monclient(hunting): picked mon.noname-i con 0x7fd1540a08e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd1540a2cd0 0x7fd15411e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 10 monclient(hunting): picked mon.noname-h con 0x7fd1540a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15409d440 con 0x7fd15409fa20 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd15409cb60 con 0x7fd1540a2cd0 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd154131da0 con 0x7fd1540a08e0 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd15a19d640 10 monclient(hunting): authenticate will time out at 806.122403s 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1540a08e0 0x7fd15411db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1540a08e0 0x7fd15411db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49540/0 (socket says 172.21.15.204:49540) 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 1 -- 172.21.15.204:0/666491973 learned_addr learned my addr 172.21.15.204:0/666491973 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 10 monclient(hunting): get_auth_request con 0x7fd1540a08e0 auth_method 0 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd153fff640 1 --2- 172.21.15.204:0/666491973 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd1540a2cd0 0x7fd15411e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd153fff640 10 monclient(hunting): get_auth_request con 0x7fd1540a2cd0 auth_method 0 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd153fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd153fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd153fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd1537fe640 1 --2- 172.21.15.204:0/666491973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15409fa20 0x7fd154131000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd1537fe640 10 monclient(hunting): get_auth_request con 0x7fd15409fa20 auth_method 0 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd1537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd1537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd1537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd153fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd153fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd153fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd1537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd1537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd1537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 10 monclient(hunting): handle_auth_done global_id 4597 payload 824 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 1 -- 172.21.15.204:0/666491973 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd1540a2cd0 msgr2=0x7fd15411e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 1 --2- 172.21.15.204:0/666491973 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd1540a2cd0 0x7fd15411e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 1 -- 172.21.15.204:0/666491973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15409fa20 msgr2=0x7fd154131000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 1 --2- 172.21.15.204:0/666491973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15409fa20 0x7fd154131000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 1 monclient: found mon.noname-i 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 1 -- 172.21.15.204:0/666491973 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd148009390 con 0x7fd1540a08e0 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.479+0000 7fd152ffd640 1 --2- 172.21.15.204:0/666491973 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1540a08e0 0x7fd15411db90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd14800ecb0 tx=0x7fd1480090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 1 -- 172.21.15.204:0/666491973 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd148013470 con 0x7fd1540a08e0 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 10 monclient: dump: 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.501 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd15a19d640 5 monclient: authenticate success, global_id 4597 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 1 -- 172.21.15.204:0/666491973 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1480135f0 con 0x7fd1540a08e0 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd15a19d640 10 monclient: _renew_subs 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd15a19d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd15a19d640 1 -- 172.21.15.204:0/666491973 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd144009450 con 0x7fd1540a08e0 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 1 -- 172.21.15.204:0/666491973 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd148013be0 con 0x7fd1540a08e0 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 10 monclient: dump: 2024-09-15T01:33:10.502 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.503 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd15a19d640 10 monclient: _renew_subs 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd15a19d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd151ffb640 4 set_mon_vals no callback set 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd15a19d640 1 -- 172.21.15.204:0/666491973 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd154120260 con 0x7fd1540a08e0 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.480+0000 7fd13b7fe640 1 -- 172.21.15.204:0/666491973 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd148013d40 con 0x7fd1540a08e0 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.481+0000 7fd13b7fe640 1 --2- 172.21.15.204:0/666491973 >> v2:172.21.15.204:6812/28210 conn(0x7fd1400397b0 0x7fd14003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.481+0000 7fd15a19d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.481+0000 7fd15a19d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.481+0000 7fd15a19d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.481+0000 7fd13b7fe640 1 -- 172.21.15.204:0/666491973 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3885+0+0 (secure 0 0 0) 0x7fd14804b8e0 con 0x7fd1540a08e0 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.481+0000 7fd15a19d640 1 -- 172.21.15.204:0/666491973 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd154070b80 con 0x7fd1540a08e0 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.481+0000 7fd1537fe640 1 --2- 172.21.15.204:0/666491973 >> v2:172.21.15.204:6812/28210 conn(0x7fd1400397b0 0x7fd14003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.481+0000 7fd1537fe640 10 monclient: get_auth_request con 0x7fd1400397b0 auth_method 0 2024-09-15T01:33:10.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.481+0000 7fd1537fe640 1 --2- 172.21.15.204:0/666491973 >> v2:172.21.15.204:6812/28210 conn(0x7fd1400397b0 0x7fd14003bc60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd144034000 tx=0x7fd144032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:10.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.485+0000 7fd13b7fe640 1 -- 172.21.15.204:0/666491973 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd148013a40 con 0x7fd1540a08e0 2024-09-15T01:33:10.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.485+0000 7fd13b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:10.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.485+0000 7fd13b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:10.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.639+0000 7fd15a19d640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}] 2024-09-15T01:33:10.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.639+0000 7fd15a19d640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}] 2024-09-15T01:33:10.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.639+0000 7fd15a19d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:10.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.639+0000 7fd15a19d640 1 -- 172.21.15.204:0/666491973 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"} v 0) v1 -- 0x7fd1540008d0 con 0x7fd1540a08e0 2024-09-15T01:33:10.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.640+0000 7fd13b7fe640 1 -- 172.21.15.204:0/666491973 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}]=-22 specified pg_num 44 < pg_num_max 33 v25) v1 ==== 149+0+0 (secure 0 0 0) 0x7fd14801b3b0 con 0x7fd1540a08e0 2024-09-15T01:33:10.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.640+0000 7fd13b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}] 2024-09-15T01:33:10.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.640+0000 7fd13b7fe640 10 monclient: _finish_command 2 = mon:22 specified pg_num 44 < pg_num_max 33 2024-09-15T01:33:10.643 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: specified pg_num 44 < pg_num_max 33 2024-09-15T01:33:10.646 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:10.646 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set testpool1234 pg_num_min 0 2024-09-15T01:33:10.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.720+0000 7f543419d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:10.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.720+0000 7f543419d640 10 monclient: build_initial_monmap 2024-09-15T01:33:10.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.720+0000 7f543419d640 10 monclient: monmap: 2024-09-15T01:33:10.722 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:10.722 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.723 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:10.721685+0000 2024-09-15T01:33:10.723 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:10.721685+0000 2024-09-15T01:33:10.723 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:10.723 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.723 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:10.723 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:10.723 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:10.723 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:10.723 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:10.723 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:10.723 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:10.723 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:10.724 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:10.724 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:10.724 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:10.724 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.720+0000 7f543419d640 1 Processor -- start 2024-09-15T01:33:10.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.720+0000 7f543419d640 1 -- start start 2024-09-15T01:33:10.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 10 monclient: init 2024-09-15T01:33:10.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:10.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 10 monclient: _add_conns ranks=[7,10,5,4,1,8,3,9,0,2,6] 2024-09-15T01:33:10.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c067130 0x7f542c09d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 10 monclient(hunting): picked mon.noname-i con 0x7f542c067130 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:10.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f542c09c650 0x7f542c09ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 10 monclient(hunting): picked mon.noname-f con 0x7f542c09c650 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f542c132970 0x7f542c132e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 10 monclient(hunting): picked mon.noname-h con 0x7f542c132970 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c09abd0 con 0x7f542c09c650 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f542c067500 con 0x7f542c132970 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f542c09ae40 con 0x7f542c067130 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543419d640 10 monclient(hunting): authenticate will time out at 806.364471s 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c067130 0x7f542c09d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431738640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f542c09c650 0x7f542c09ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543273a640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f542c132970 0x7f542c132e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c067130 0x7f542c09d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49552/0 (socket says 172.21.15.204:49552) 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431738640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f542c09c650 0x7f542c09ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:34444/0 (socket says 172.21.15.204:34444) 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 1 -- 172.21.15.204:0/3684992388 learned_addr learned my addr 172.21.15.204:0/3684992388 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 10 monclient(hunting): get_auth_request con 0x7f542c067130 auth_method 0 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543273a640 10 monclient(hunting): get_auth_request con 0x7f542c132970 auth_method 0 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543273a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543273a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543273a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431738640 10 monclient(hunting): get_auth_request con 0x7f542c09c650 auth_method 0 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431738640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431738640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431738640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543273a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543273a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543273a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431738640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431738640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431738640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 10 monclient(hunting): handle_auth_done global_id 4606 payload 274 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 1 -- 172.21.15.204:0/3684992388 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f542c132970 msgr2=0x7f542c132e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 1 --2- 172.21.15.204:0/3684992388 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f542c132970 0x7f542c132e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 1 -- 172.21.15.204:0/3684992388 >> v2:172.21.15.204:3300/0 conn(0x7f542c09c650 msgr2=0x7f542c09ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 1 --2- 172.21.15.204:0/3684992388 >> v2:172.21.15.204:3300/0 conn(0x7f542c09c650 0x7f542c09ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 10 monclient: _finish_hunting 0 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 1 monclient: found mon.noname-i 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431f39640 1 -- 172.21.15.204:0/3684992388 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c0648a0 con 0x7f542c067130 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f543273a640 1 --2- 172.21.15.204:0/3684992388 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f542c132970 0x7f542c132e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.721+0000 7f5431738640 1 --2- 172.21.15.204:0/3684992388 >> v2:172.21.15.204:3300/0 conn(0x7f542c09c650 0x7f542c09ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5431f39640 1 --2- 172.21.15.204:0/3684992388 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c067130 0x7f542c09d900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5428009910 tx=0x7f54280091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=32e78280aeb43228 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5430f37640 1 -- 172.21.15.204:0/3684992388 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f542802ac80 con 0x7f542c067130 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5430f37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5430f37640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:10.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5430f37640 10 monclient: dump: 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.737 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5430f37640 10 monclient: _finish_auth 0 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5430f37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f543419d640 5 monclient: authenticate success, global_id 4606 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f543419d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5430f37640 1 -- 172.21.15.204:0/3684992388 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542802ae00 con 0x7f542c067130 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5430f37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f543419d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f543419d640 4 set_mon_vals no callback set 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5430f37640 1 -- 172.21.15.204:0/3684992388 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5428031ca0 con 0x7f542c067130 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5430f37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5430f37640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.722+0000 7f5430f37640 10 monclient: dump: 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.739 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 10 monclient: shutdown 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 1 -- 172.21.15.204:0/3684992388 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c067130 msgr2=0x7f542c09d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 1 --2- 172.21.15.204:0/3684992388 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c067130 0x7f542c09d900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5428009910 tx=0x7f54280091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 1 -- 172.21.15.204:0/3684992388 shutdown_connections 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 1 --2- 172.21.15.204:0/3684992388 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f542c132970 0x7f542c132e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 1 --2- 172.21.15.204:0/3684992388 >> v2:172.21.15.204:3300/0 conn(0x7f542c09c650 0x7f542c09ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 1 --2- 172.21.15.204:0/3684992388 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c067130 0x7f542c09d900 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 1 -- 172.21.15.204:0/3684992388 >> 172.21.15.204:0/3684992388 conn(0x7f542c0a2c40 msgr2=0x7f542c0a3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 1 -- 172.21.15.204:0/3684992388 shutdown_connections 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 1 -- 172.21.15.204:0/3684992388 wait complete. 2024-09-15T01:33:10.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 10 monclient: build_initial_monmap 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.723+0000 7f543419d640 10 monclient: monmap: 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:10.741 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 1 Processor -- start 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 1 -- start start 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 10 monclient: init 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 10 monclient: _add_conns ranks=[4,7,8,0,5,3,2,6,1] 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f542c067130 0x7f542c1317b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 10 monclient(hunting): picked mon.noname-c con 0x7f542c067130 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c09c650 0x7f542c11e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 10 monclient(hunting): picked mon.noname-i con 0x7f542c09c650 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f542c132970 0x7f542c11e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 10 monclient(hunting): picked mon.noname-e con 0x7f542c132970 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f542c099a10 con 0x7f542c067130 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f542c09a190 con 0x7f542c09c650 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f542c142900 con 0x7f542c132970 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f543419d640 10 monclient(hunting): authenticate will time out at 806.367796s 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f5431738640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c09c650 0x7f542c11e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f5431738640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c09c650 0x7f542c11e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49560/0 (socket says 172.21.15.204:49560) 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f5431738640 1 -- 172.21.15.204:0/2811756927 learned_addr learned my addr 172.21.15.204:0/2811756927 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f5431738640 10 monclient(hunting): get_auth_request con 0x7f542c09c650 auth_method 0 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f5431738640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f5431738640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f5431738640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f5431738640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:10.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f5431738640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:10.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.724+0000 7f5431738640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:10.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431f39640 1 --2- 172.21.15.204:0/2811756927 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f542c067130 0x7f542c1317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f543273a640 1 --2- 172.21.15.204:0/2811756927 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f542c132970 0x7f542c11e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f543273a640 10 monclient(hunting): get_auth_request con 0x7f542c132970 auth_method 0 2024-09-15T01:33:10.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f543273a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f543273a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f543273a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431f39640 10 monclient(hunting): get_auth_request con 0x7f542c067130 auth_method 0 2024-09-15T01:33:10.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:10.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431f39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:10.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:10.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431738640 10 monclient(hunting): handle_auth_done global_id 4615 payload 824 2024-09-15T01:33:10.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431738640 1 -- 172.21.15.204:0/2811756927 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f542c132970 msgr2=0x7f542c11e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:10.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431738640 1 --2- 172.21.15.204:0/2811756927 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f542c132970 0x7f542c11e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431738640 1 -- 172.21.15.204:0/2811756927 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f542c067130 msgr2=0x7f542c1317b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:10.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431738640 1 --2- 172.21.15.204:0/2811756927 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f542c067130 0x7f542c1317b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:10.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431738640 10 monclient: _finish_hunting 0 2024-09-15T01:33:10.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431738640 1 monclient: found mon.noname-i 2024-09-15T01:33:10.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431738640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:10.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431738640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:10.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431738640 1 -- 172.21.15.204:0/2811756927 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c009390 con 0x7f542c09c650 2024-09-15T01:33:10.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5431738640 1 --2- 172.21.15.204:0/2811756927 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c09c650 0x7f542c11e3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f541c00ec80 tx=0x7f541c0090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:10.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5421ffb640 1 -- 172.21.15.204:0/2811756927 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f541c00eec0 con 0x7f542c09c650 2024-09-15T01:33:10.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5421ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5421ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5421ffb640 10 monclient: dump: 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.746 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5421ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5421ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f543419d640 5 monclient: authenticate success, global_id 4615 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5421ffb640 1 -- 172.21.15.204:0/2811756927 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c013470 con 0x7f542c09c650 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5421ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5421ffb640 1 -- 172.21.15.204:0/2811756927 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f541c013a60 con 0x7f542c09c650 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f543419d640 10 monclient: _renew_subs 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f543419d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:10.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f543419d640 1 -- 172.21.15.204:0/2811756927 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5424009a20 con 0x7f542c09c650 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5421ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5423fff640 4 set_mon_vals no callback set 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5421ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f5421ffb640 10 monclient: dump: 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:10.748 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f543419d640 10 monclient: _renew_subs 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f543419d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.725+0000 7f543419d640 1 -- 172.21.15.204:0/2811756927 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5428009450 con 0x7f542c09c650 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.726+0000 7f5421ffb640 1 -- 172.21.15.204:0/2811756927 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f541c013bc0 con 0x7f542c09c650 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.726+0000 7f543419d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:10.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.726+0000 7f543419d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:10.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.726+0000 7f543419d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:10.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.726+0000 7f5421ffb640 1 --2- 172.21.15.204:0/2811756927 >> v2:172.21.15.204:6812/28210 conn(0x7f54100397b0 0x7f541003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:10.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.726+0000 7f543419d640 1 -- 172.21.15.204:0/2811756927 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c069250 con 0x7f542c09c650 2024-09-15T01:33:10.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.726+0000 7f5421ffb640 1 -- 172.21.15.204:0/2811756927 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3885+0+0 (secure 0 0 0) 0x7f541c04b3d0 con 0x7f542c09c650 2024-09-15T01:33:10.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.726+0000 7f5431f39640 1 --2- 172.21.15.204:0/2811756927 >> v2:172.21.15.204:6812/28210 conn(0x7f54100397b0 0x7f541003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:10.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.726+0000 7f5431f39640 10 monclient: get_auth_request con 0x7f54100397b0 auth_method 0 2024-09-15T01:33:10.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.727+0000 7f5431f39640 1 --2- 172.21.15.204:0/2811756927 >> v2:172.21.15.204:6812/28210 conn(0x7f54100397b0 0x7f541003bc60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5428034000 tx=0x7f5428032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:10.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.731+0000 7f5421ffb640 1 -- 172.21.15.204:0/2811756927 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f541c0123d0 con 0x7f542c09c650 2024-09-15T01:33:10.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.731+0000 7f5421ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:10.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.731+0000 7f5421ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:10.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.884+0000 7f543419d640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-09-15T01:33:10.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.884+0000 7f543419d640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-09-15T01:33:10.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.884+0000 7f543419d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:10.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:10.884+0000 7f543419d640 1 -- 172.21.15.204:0/2811756927 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"} v 0) v1 -- 0x7f542c070b80 con 0x7f542c09c650 2024-09-15T01:33:11.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:11.309+0000 7f5421ffb640 1 -- 172.21.15.204:0/2811756927 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}]=0 set pool 3 pg_num_min to 0 v26) v1 ==== 143+0+0 (secure 0 0 0) 0x7f541c01b3b0 con 0x7f542c09c650 2024-09-15T01:33:11.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:11.309+0000 7f5421ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-09-15T01:33:11.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:11.309+0000 7f5421ffb640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_min to 0 2024-09-15T01:33:11.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:11.400+0000 7f543419d640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-09-15T01:33:11.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:11.400+0000 7f543419d640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-09-15T01:33:11.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:11.400+0000 7f543419d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:11.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:11.400+0000 7f543419d640 1 -- 172.21.15.204:0/2811756927 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"} v 0) v1 -- 0x7f542c09b120 con 0x7f542c09c650 2024-09-15T01:33:12.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.314+0000 7f5421ffb640 1 -- 172.21.15.204:0/2811756927 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}]=0 set pool 3 pg_num_min to 0 v27) v1 ==== 143+0+0 (secure 0 0 0) 0x7f541c0232f0 con 0x7f542c09c650 2024-09-15T01:33:12.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.314+0000 7f5421ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-09-15T01:33:12.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.314+0000 7f5421ffb640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_min to 0 2024-09-15T01:33:12.317 INFO:tasks.workunit.client.0.smithi204.stderr:set pool 3 pg_num_min to 0 2024-09-15T01:33:12.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 1 -- 172.21.15.204:0/2811756927 >> v2:172.21.15.204:6812/28210 conn(0x7f54100397b0 msgr2=0x7f541003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:12.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 1 --2- 172.21.15.204:0/2811756927 >> v2:172.21.15.204:6812/28210 conn(0x7f54100397b0 0x7f541003bc60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5428034000 tx=0x7f5428032000 comp rx=0 tx=0).stop 2024-09-15T01:33:12.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 10 monclient: shutdown 2024-09-15T01:33:12.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:12.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 1 -- 172.21.15.204:0/2811756927 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c09c650 msgr2=0x7f542c11e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:12.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 1 --2- 172.21.15.204:0/2811756927 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c09c650 0x7f542c11e3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f541c00ec80 tx=0x7f541c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 1 -- 172.21.15.204:0/2811756927 shutdown_connections 2024-09-15T01:33:12.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 1 --2- 172.21.15.204:0/2811756927 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f542c132970 0x7f542c11e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 1 --2- 172.21.15.204:0/2811756927 >> v2:172.21.15.204:6812/28210 conn(0x7f54100397b0 0x7f541003bc60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 1 --2- 172.21.15.204:0/2811756927 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f542c09c650 0x7f542c11e3c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 1 --2- 172.21.15.204:0/2811756927 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f542c067130 0x7f542c1317b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 1 -- 172.21.15.204:0/2811756927 >> 172.21.15.204:0/2811756927 conn(0x7f542c0a2c40 msgr2=0x7f542c0989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:12.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 1 -- 172.21.15.204:0/2811756927 shutdown_connections 2024-09-15T01:33:12.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.318+0000 7f543419d640 1 -- 172.21.15.204:0/2811756927 wait complete. 2024-09-15T01:33:12.330 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool get testpool1234 pg_num_min 2024-09-15T01:33:12.330 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:12.330 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool get testpool1234 pg_num_min 2024-09-15T01:33:12.409 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.407+0000 7f1c42b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:12.409 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.407+0000 7f1c42b9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:12.409 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.407+0000 7f1c42b9d640 10 monclient: monmap: 2024-09-15T01:33:12.409 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:12.409 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:12.408613+0000 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:12.408613+0000 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:12.410 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:12.411 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:12.411 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:12.411 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:12.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.407+0000 7f1c42b9d640 1 Processor -- start 2024-09-15T01:33:12.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.407+0000 7f1c42b9d640 1 -- start start 2024-09-15T01:33:12.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.407+0000 7f1c42b9d640 10 monclient: init 2024-09-15T01:33:12.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:12.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 10 monclient: _add_conns ranks=[7,3,10,6,2,4,0,1,9,5,8] 2024-09-15T01:33:12.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c3c0a2e30 0x7f1c3c132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 10 monclient(hunting): picked mon.noname-i con 0x7f1c3c0a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c09fb80 0x7f1c3c09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 10 monclient(hunting): picked mon.noname-g con 0x7f1c3c09fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1c3c0a0a40 0x7f1c3c0a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 10 monclient(hunting): picked mon.noname-f con 0x7f1c3c0a0a40 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3c09e6c0 con 0x7f1c3c0a0a40 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c0a04b0 con 0x7f1c3c09fb80 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c0a3200 con 0x7f1c3c0a2e30 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c42b9d640 10 monclient(hunting): authenticate will time out at 808.051441s 2024-09-15T01:33:12.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c3bfff640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c09fb80 msgr2=0x7f1c3c09ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c3bfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c09fb80 0x7f1c3c09ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c40939640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c3c0a2e30 0x7f1c3c132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1c3c0a0a40 0x7f1c3c0a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1c3c0a0a40 0x7f1c3c0a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:34456/0 (socket says 172.21.15.204:34456) 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c40939640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c3c0a2e30 0x7f1c3c132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49574/0 (socket says 172.21.15.204:49574) 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 1 -- 172.21.15.204:0/3869723088 learned_addr learned my addr 172.21.15.204:0/3869723088 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 10 monclient(hunting): get_auth_request con 0x7f1c3c0a0a40 auth_method 0 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c40939640 10 monclient(hunting): get_auth_request con 0x7f1c3c0a2e30 auth_method 0 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c40939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:12.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c40939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c40939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c40939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c40939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c40939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 10 monclient(hunting): handle_auth_done global_id 4699 payload 274 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 1 -- 172.21.15.204:0/3869723088 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c3c0a2e30 msgr2=0x7f1c3c132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 1 --2- 172.21.15.204:0/3869723088 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c3c0a2e30 0x7f1c3c132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 1 -- 172.21.15.204:0/3869723088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c09fb80 msgr2=0x7f1c3c09ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 1 --2- 172.21.15.204:0/3869723088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c09fb80 0x7f1c3c09ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 10 monclient: _finish_hunting 0 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 1 monclient: found mon.noname-f 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c4113a640 1 -- 172.21.15.204:0/3869723088 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c04b720 con 0x7f1c3c0a0a40 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.408+0000 7f1c40939640 1 --2- 172.21.15.204:0/3869723088 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c3c0a2e30 0x7f1c3c132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c4113a640 1 --2- 172.21.15.204:0/3869723088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3c0a0a40 0x7f1c3c0a0ee0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009f20 tx=0x7f1c34009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1b5edde775367e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c3b7fe640 1 -- 172.21.15.204:0/3869723088 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c34007880 con 0x7f1c3c0a0a40 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c3b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c3b7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c3b7fe640 10 monclient: dump: 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.423 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.424 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:12.424 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:12.424 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:12.424 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:12.424 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:12.424 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:12.424 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:12.424 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:12.424 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:12.424 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:12.424 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:12.424 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c3b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c3b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c42b9d640 5 monclient: authenticate success, global_id 4699 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c3b7fe640 1 -- 172.21.15.204:0/3869723088 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3402d050 con 0x7f1c3c0a0a40 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c42b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c3b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c42b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c42b9d640 4 set_mon_vals no callback set 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c3b7fe640 1 -- 172.21.15.204:0/3869723088 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c34030360 con 0x7f1c3c0a0a40 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c3b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c3b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.409+0000 7f1c3b7fe640 10 monclient: dump: 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:12.425 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:12.426 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 10 monclient: shutdown 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 1 -- 172.21.15.204:0/3869723088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3c0a0a40 msgr2=0x7f1c3c0a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 1 --2- 172.21.15.204:0/3869723088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3c0a0a40 0x7f1c3c0a0ee0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009f20 tx=0x7f1c34009840 comp rx=0 tx=0).stop 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 1 -- 172.21.15.204:0/3869723088 shutdown_connections 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 1 --2- 172.21.15.204:0/3869723088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3c0a0a40 0x7f1c3c0a0ee0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 1 --2- 172.21.15.204:0/3869723088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c09fb80 0x7f1c3c09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 1 --2- 172.21.15.204:0/3869723088 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c3c0a2e30 0x7f1c3c132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 1 -- 172.21.15.204:0/3869723088 >> 172.21.15.204:0/3869723088 conn(0x7f1c3c06e860 msgr2=0x7f1c3c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 1 -- 172.21.15.204:0/3869723088 shutdown_connections 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 1 -- 172.21.15.204:0/3869723088 wait complete. 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.410+0000 7f1c42b9d640 10 monclient: monmap: 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:12.427 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:12.428 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.411+0000 7f1c42b9d640 1 Processor -- start 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.411+0000 7f1c42b9d640 1 -- start start 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.411+0000 7f1c42b9d640 10 monclient: init 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.411+0000 7f1c42b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient: _add_conns ranks=[7,2,3,0,6,1,8,5,4] 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c3c09fb80 0x7f1c3c130690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient(hunting): picked mon.noname-i con 0x7f1c3c09fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c3c0a0a40 0x7f1c3c11d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient(hunting): picked mon.noname-b con 0x7f1c3c0a0a40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c0a2e30 0x7f1c3c11d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient(hunting): picked mon.noname-g con 0x7f1c3c0a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c09d650 con 0x7f1c3c0a0a40 2024-09-15T01:33:12.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c120da0 con 0x7f1c3c0a2e30 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c120ac0 con 0x7f1c3c09fb80 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient(hunting): authenticate will time out at 808.055074s 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c4113a640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c0a2e30 msgr2=0x7f1c3c11d710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c4113a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c0a2e30 0x7f1c3c11d710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c3c09fb80 0x7f1c3c130690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c3c09fb80 0x7f1c3c130690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49586/0 (socket says 172.21.15.204:49586) 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 1 -- 172.21.15.204:0/1388423460 learned_addr learned my addr 172.21.15.204:0/1388423460 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 10 monclient(hunting): get_auth_request con 0x7f1c3c09fb80 auth_method 0 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:12.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c3bfff640 1 --2- 172.21.15.204:0/1388423460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c3c0a0a40 0x7f1c3c11d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c3bfff640 10 monclient(hunting): get_auth_request con 0x7f1c3c0a0a40 auth_method 0 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 10 monclient(hunting): handle_auth_done global_id 4633 payload 824 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 1 -- 172.21.15.204:0/1388423460 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c0a2e30 msgr2=0x7f1c3c11d710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 1 --2- 172.21.15.204:0/1388423460 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c0a2e30 0x7f1c3c11d710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 1 -- 172.21.15.204:0/1388423460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c3c0a0a40 msgr2=0x7f1c3c11d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 1 --2- 172.21.15.204:0/1388423460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c3c0a0a40 0x7f1c3c11d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 10 monclient: _finish_hunting 0 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 1 monclient: found mon.noname-i 2024-09-15T01:33:12.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 1 -- 172.21.15.204:0/1388423460 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c30009390 con 0x7f1c3c09fb80 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c40939640 1 --2- 172.21.15.204:0/1388423460 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c3c09fb80 0x7f1c3c130690 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000ea70 tx=0x7f1c300090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c38ff9640 1 -- 172.21.15.204:0/1388423460 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c3000ce90 con 0x7f1c3c09fb80 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c38ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c38ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c38ff9640 10 monclient: dump: 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:12.432 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c38ff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c38ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c38ff9640 1 -- 172.21.15.204:0/1388423460 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c30013470 con 0x7f1c3c09fb80 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 5 monclient: authenticate success, global_id 4633 2024-09-15T01:33:12.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient: _renew_subs 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 1 -- 172.21.15.204:0/1388423460 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c34009ae0 con 0x7f1c3c09fb80 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c38ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c38ff9640 1 -- 172.21.15.204:0/1388423460 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c30013a60 con 0x7f1c3c09fb80 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c38ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c38ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c38ff9640 10 monclient: dump: 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:12.434 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c3affd640 4 set_mon_vals no callback set 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient: _renew_subs 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.412+0000 7f1c42b9d640 1 -- 172.21.15.204:0/1388423460 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3c1207e0 con 0x7f1c3c09fb80 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.413+0000 7f1c38ff9640 1 -- 172.21.15.204:0/1388423460 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1c30013be0 con 0x7f1c3c09fb80 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.413+0000 7f1c42b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:12.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.413+0000 7f1c42b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:12.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.413+0000 7f1c42b9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:12.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.413+0000 7f1c38ff9640 1 --2- 172.21.15.204:0/1388423460 >> v2:172.21.15.204:6812/28210 conn(0x7f1c200397b0 0x7f1c2003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.413+0000 7f1c42b9d640 1 -- 172.21.15.204:0/1388423460 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3c070cc0 con 0x7f1c3c09fb80 2024-09-15T01:33:12.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.413+0000 7f1c38ff9640 1 -- 172.21.15.204:0/1388423460 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3869+0+0 (secure 0 0 0) 0x7f1c3004b4d0 con 0x7f1c3c09fb80 2024-09-15T01:33:12.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.414+0000 7f1c3bfff640 1 --2- 172.21.15.204:0/1388423460 >> v2:172.21.15.204:6812/28210 conn(0x7f1c200397b0 0x7f1c2003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:12.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.414+0000 7f1c3bfff640 10 monclient: get_auth_request con 0x7f1c200397b0 auth_method 0 2024-09-15T01:33:12.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.414+0000 7f1c3bfff640 1 --2- 172.21.15.204:0/1388423460 >> v2:172.21.15.204:6812/28210 conn(0x7f1c200397b0 0x7f1c2003bc60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1c28003850 tx=0x7f1c28006ec0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:12.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.419+0000 7f1c38ff9640 1 -- 172.21.15.204:0/1388423460 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1c3000ca10 con 0x7f1c3c09fb80 2024-09-15T01:33:12.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.419+0000 7f1c38ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:12.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.419+0000 7f1c38ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:12.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.570+0000 7f1c42b9d640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T01:33:12.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.570+0000 7f1c42b9d640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T01:33:12.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.570+0000 7f1c42b9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:12.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.570+0000 7f1c42b9d640 1 -- 172.21.15.204:0/1388423460 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"} v 0) v1 -- 0x7f1c3c133ed0 con 0x7f1c3c09fb80 2024-09-15T01:33:12.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.571+0000 7f1c38ff9640 1 -- 172.21.15.204:0/1388423460 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}]=-2 option 'pg_num_min' is not set on pool 'testpool1234' v27) v1 ==== 158+0+0 (secure 0 0 0) 0x7f1c300192c0 con 0x7f1c3c09fb80 2024-09-15T01:33:12.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.571+0000 7f1c38ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-09-15T01:33:12.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.571+0000 7f1c38ff9640 10 monclient: _finish_command 2 = mon:2 option 'pg_num_min' is not set on pool 'testpool1234' 2024-09-15T01:33:12.574 INFO:tasks.workunit.client.0.smithi204.stderr:Error ENOENT: option 'pg_num_min' is not set on pool 'testpool1234' 2024-09-15T01:33:12.577 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:12.577 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set testpool1234 pg_num_max 0 2024-09-15T01:33:12.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.650+0000 7f5840f1d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.650+0000 7f5840f1d640 10 monclient: build_initial_monmap 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.650+0000 7f5840f1d640 10 monclient: monmap: 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:12.651903+0000 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:12.651903+0000 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:12.653 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:12.654 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:12.654 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:12.654 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:12.654 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:12.654 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:12.654 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:12.654 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:12.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.650+0000 7f5840f1d640 1 Processor -- start 2024-09-15T01:33:12.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 1 -- start start 2024-09-15T01:33:12.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 10 monclient: init 2024-09-15T01:33:12.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:12.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 10 monclient: _add_conns ranks=[10,0,9,1,6,7,4,5,3,2,8] 2024-09-15T01:33:12.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f583c0a4520 0x7f583c130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 10 monclient(hunting): picked mon.noname-f con 0x7f583c0a4520 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:12.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f583c06ecf0 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:12.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f583c135ad0 0x7f583c137eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 10 monclient(hunting): picked mon.noname-a con 0x7f583c135ad0 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583c0648a0 con 0x7f583c06ecf0 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583c131690 con 0x7f583c135ad0 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583c06de30 con 0x7f583c0a4520 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f5840f1d640 10 monclient(hunting): authenticate will time out at 808.294646s 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583a59c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f583c0a4520 0x7f583c130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583a59c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f583c0a4520 0x7f583c130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:34458/0 (socket says 172.21.15.204:34458) 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583a59c640 1 -- 172.21.15.204:0/4198717823 learned_addr learned my addr 172.21.15.204:0/4198717823 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583a59c640 10 monclient(hunting): get_auth_request con 0x7f583c0a4520 auth_method 0 2024-09-15T01:33:12.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583ad9d640 1 --2- 172.21.15.204:0/4198717823 >> v2:172.21.15.178:3300/0 conn(0x7f583c135ad0 0x7f583c137eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583ad9d640 10 monclient(hunting): get_auth_request con 0x7f583c135ad0 auth_method 0 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:12.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.651+0000 7f583ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:12.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583a59c640 10 monclient(hunting): handle_auth_done global_id 4708 payload 274 2024-09-15T01:33:12.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583a59c640 1 -- 172.21.15.204:0/4198717823 >> v2:172.21.15.178:3300/0 conn(0x7f583c135ad0 msgr2=0x7f583c137eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:12.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583a59c640 1 --2- 172.21.15.204:0/4198717823 >> v2:172.21.15.178:3300/0 conn(0x7f583c135ad0 0x7f583c137eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583a59c640 1 -- 172.21.15.204:0/4198717823 >> v1:172.21.15.178:6789/0 conn(0x7f583c06ecf0 legacy=0x7f583c130f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:12.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583a59c640 10 monclient: _finish_hunting 0 2024-09-15T01:33:12.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583a59c640 1 monclient: found mon.noname-f 2024-09-15T01:33:12.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:12.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583a59c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:12.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583a59c640 1 -- 172.21.15.204:0/4198717823 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c06db50 con 0x7f583c0a4520 2024-09-15T01:33:12.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583a59c640 1 --2- 172.21.15.204:0/4198717823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c0a4520 0x7f583c130a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f582c009910 tx=0x7f582c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab68ea043c3d68c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:12.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583959a640 1 -- 172.21.15.204:0/4198717823 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f582c02ad40 con 0x7f583c0a4520 2024-09-15T01:33:12.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:12.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583959a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:12.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583959a640 10 monclient: dump: 2024-09-15T01:33:12.668 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:12.668 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:12.668 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.668 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:12.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583959a640 10 monclient: _finish_auth 0 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583959a640 1 -- 172.21.15.204:0/4198717823 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c031820 con 0x7f583c0a4520 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f5840f1d640 5 monclient: authenticate success, global_id 4708 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f5840f1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f5840f1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583959a640 1 -- 172.21.15.204:0/4198717823 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f582c031e90 con 0x7f583c0a4520 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f5840f1d640 4 set_mon_vals no callback set 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583959a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.652+0000 7f583959a640 10 monclient: dump: 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:12.670 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:12.671 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.671 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.671 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:12.671 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:12.671 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:12.671 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:12.671 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:12.671 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:12.671 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:12.671 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:12.671 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:12.671 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.653+0000 7f5840f1d640 10 monclient: shutdown 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.653+0000 7f5840f1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.653+0000 7f5840f1d640 1 -- 172.21.15.204:0/4198717823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c0a4520 msgr2=0x7f583c130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.653+0000 7f5840f1d640 1 --2- 172.21.15.204:0/4198717823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c0a4520 0x7f583c130a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f582c009910 tx=0x7f582c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.653+0000 7f5840f1d640 1 -- 172.21.15.204:0/4198717823 shutdown_connections 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.653+0000 7f5840f1d640 1 --2- 172.21.15.204:0/4198717823 >> v2:172.21.15.178:3300/0 conn(0x7f583c135ad0 0x7f583c137eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.653+0000 7f5840f1d640 1 --2- 172.21.15.204:0/4198717823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c0a4520 0x7f583c130a40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.653+0000 7f5840f1d640 1 -- 172.21.15.204:0/4198717823 >> 172.21.15.204:0/4198717823 conn(0x7f583c0676a0 msgr2=0x7f583c0a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.653+0000 7f5840f1d640 1 -- 172.21.15.204:0/4198717823 shutdown_connections 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.653+0000 7f5840f1d640 1 -- 172.21.15.204:0/4198717823 wait complete. 2024-09-15T01:33:12.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.653+0000 7f5840f1d640 10 monclient: build_initial_monmap 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.653+0000 7f5840f1d640 10 monclient: monmap: 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:12.673 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 1 Processor -- start 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 1 -- start start 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 10 monclient: init 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 10 monclient: _add_conns ranks=[5,4,1,6,7,2,0,3,8] 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f583c06ecf0 0x7f583c135150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 10 monclient(hunting): picked mon.noname-h con 0x7f583c06ecf0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f583c0a4520 0x7f583c0969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 10 monclient(hunting): picked mon.noname-c con 0x7f583c0a4520 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c135ad0 0x7f583c096f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 10 monclient(hunting): picked mon.noname-f con 0x7f583c135ad0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c06d010 con 0x7f583c135ad0 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f583c06cc30 con 0x7f583c0a4520 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f583c0998b0 con 0x7f583c06ecf0 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5840f1d640 10 monclient(hunting): authenticate will time out at 808.297763s 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583a59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f583c06ecf0 0x7f583c135150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c135ad0 0x7f583c096f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583a59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f583c06ecf0 0x7f583c135150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:50692/0 (socket says 172.21.15.204:50692) 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583a59c640 1 -- 172.21.15.204:0/782238826 learned_addr learned my addr 172.21.15.204:0/782238826 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:12.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583a59c640 10 monclient(hunting): get_auth_request con 0x7f583c06ecf0 auth_method 0 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c135ad0 0x7f583c096f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:34468/0 (socket says 172.21.15.204:34468) 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583ad9d640 10 monclient(hunting): get_auth_request con 0x7f583c135ad0 auth_method 0 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:12.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5839d9b640 1 --2- 172.21.15.204:0/782238826 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f583c0a4520 0x7f583c0969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f583ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5839d9b640 10 monclient(hunting): get_auth_request con 0x7f583c0a4520 auth_method 0 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5839d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5839d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.654+0000 7f5839d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f5839d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f5839d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f5839d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f583ad9d640 10 monclient(hunting): handle_auth_done global_id 4717 payload 824 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f583ad9d640 1 -- 172.21.15.204:0/782238826 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f583c06ecf0 msgr2=0x7f583c135150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f583ad9d640 1 --2- 172.21.15.204:0/782238826 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f583c06ecf0 0x7f583c135150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f583ad9d640 1 -- 172.21.15.204:0/782238826 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f583c0a4520 msgr2=0x7f583c0969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f583ad9d640 1 --2- 172.21.15.204:0/782238826 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f583c0a4520 0x7f583c0969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f583ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f583ad9d640 1 monclient: found mon.noname-f 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f583ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f583ad9d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f583ad9d640 1 -- 172.21.15.204:0/782238826 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c009450 con 0x7f583c135ad0 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f583a59c640 1 --2- 172.21.15.204:0/782238826 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f583c06ecf0 0x7f583c135150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f583ad9d640 1 --2- 172.21.15.204:0/782238826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c135ad0 0x7f583c096f20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5834009e50 tx=0x7f583400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f582a7fc640 1 -- 172.21.15.204:0/782238826 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f583400cd40 con 0x7f583c135ad0 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f582a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f582a7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f582a7fc640 10 monclient: dump: 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:12.678 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:12.679 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f582a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f582a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f582a7fc640 1 -- 172.21.15.204:0/782238826 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583400f040 con 0x7f583c135ad0 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f5840f1d640 5 monclient: authenticate success, global_id 4717 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f5840f1d640 10 monclient: _renew_subs 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f5840f1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f5840f1d640 1 -- 172.21.15.204:0/782238826 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583c099b90 con 0x7f583c135ad0 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f582a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f582a7fc640 1 -- 172.21.15.204:0/782238826 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5834014d80 con 0x7f583c135ad0 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f582a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f582a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:12.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f582a7fc640 10 monclient: dump: 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:12.681 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:12.682 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:12.682 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:12.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f5838d99640 4 set_mon_vals no callback set 2024-09-15T01:33:12.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f5840f1d640 10 monclient: _renew_subs 2024-09-15T01:33:12.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f5840f1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:12.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.655+0000 7f5840f1d640 1 -- 172.21.15.204:0/782238826 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583c09a570 con 0x7f583c135ad0 2024-09-15T01:33:12.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.656+0000 7f582a7fc640 1 -- 172.21.15.204:0/782238826 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5834014480 con 0x7f583c135ad0 2024-09-15T01:33:12.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.656+0000 7f5840f1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:12.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.656+0000 7f5840f1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:12.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.656+0000 7f5840f1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:12.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.656+0000 7f582a7fc640 1 --2- 172.21.15.204:0/782238826 >> v2:172.21.15.204:6812/28210 conn(0x7f581c0397b0 0x7f581c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:12.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.656+0000 7f5840f1d640 1 -- 172.21.15.204:0/782238826 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583c070b80 con 0x7f583c135ad0 2024-09-15T01:33:12.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.656+0000 7f582a7fc640 1 -- 172.21.15.204:0/782238826 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3869+0+0 (secure 0 0 0) 0x7f583404bae0 con 0x7f583c135ad0 2024-09-15T01:33:12.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.656+0000 7f583a59c640 1 --2- 172.21.15.204:0/782238826 >> v2:172.21.15.204:6812/28210 conn(0x7f581c0397b0 0x7f581c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:12.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.656+0000 7f583a59c640 10 monclient: get_auth_request con 0x7f581c0397b0 auth_method 0 2024-09-15T01:33:12.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.657+0000 7f583a59c640 1 --2- 172.21.15.204:0/782238826 >> v2:172.21.15.204:6812/28210 conn(0x7f581c0397b0 0x7f581c03bc60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f582c009420 tx=0x7f582c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:12.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.661+0000 7f582a7fc640 1 -- 172.21.15.204:0/782238826 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5834018080 con 0x7f583c135ad0 2024-09-15T01:33:12.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.661+0000 7f582a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:12.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.661+0000 7f582a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:12.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.814+0000 7f5840f1d640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-09-15T01:33:12.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.814+0000 7f5840f1d640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-09-15T01:33:12.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.814+0000 7f5840f1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:12.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:12.814+0000 7f5840f1d640 1 -- 172.21.15.204:0/782238826 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"} v 0) v1 -- 0x7f583c06a790 con 0x7f583c135ad0 2024-09-15T01:33:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:13.320+0000 7f582a7fc640 1 -- 172.21.15.204:0/782238826 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}]=0 set pool 3 pg_num_max to 0 v28) v1 ==== 143+0+0 (secure 0 0 0) 0x7f5834014730 con 0x7f583c135ad0 2024-09-15T01:33:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:13.321+0000 7f582a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-09-15T01:33:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:13.321+0000 7f582a7fc640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_max to 0 2024-09-15T01:33:13.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:13.413+0000 7f5840f1d640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-09-15T01:33:13.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:13.413+0000 7f5840f1d640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-09-15T01:33:13.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:13.413+0000 7f5840f1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:13.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:13.413+0000 7f5840f1d640 1 -- 172.21.15.204:0/782238826 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"} v 0) v1 -- 0x7f583c099d20 con 0x7f583c135ad0 2024-09-15T01:33:14.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.322+0000 7f582a7fc640 1 -- 172.21.15.204:0/782238826 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}]=0 set pool 3 pg_num_max to 0 v29) v1 ==== 143+0+0 (secure 0 0 0) 0x7f583401d400 con 0x7f583c135ad0 2024-09-15T01:33:14.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.322+0000 7f582a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-09-15T01:33:14.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.322+0000 7f582a7fc640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_max to 0 2024-09-15T01:33:14.324 INFO:tasks.workunit.client.0.smithi204.stderr:set pool 3 pg_num_max to 0 2024-09-15T01:33:14.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 1 -- 172.21.15.204:0/782238826 >> v2:172.21.15.204:6812/28210 conn(0x7f581c0397b0 msgr2=0x7f581c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:14.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 1 --2- 172.21.15.204:0/782238826 >> v2:172.21.15.204:6812/28210 conn(0x7f581c0397b0 0x7f581c03bc60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f582c009420 tx=0x7f582c032000 comp rx=0 tx=0).stop 2024-09-15T01:33:14.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 10 monclient: shutdown 2024-09-15T01:33:14.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:14.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 1 -- 172.21.15.204:0/782238826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c135ad0 msgr2=0x7f583c096f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:14.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 1 --2- 172.21.15.204:0/782238826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c135ad0 0x7f583c096f20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5834009e50 tx=0x7f583400b040 comp rx=0 tx=0).stop 2024-09-15T01:33:14.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 1 -- 172.21.15.204:0/782238826 shutdown_connections 2024-09-15T01:33:14.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 1 --2- 172.21.15.204:0/782238826 >> v2:172.21.15.204:6812/28210 conn(0x7f581c0397b0 0x7f581c03bc60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 1 --2- 172.21.15.204:0/782238826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c135ad0 0x7f583c096f20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 1 --2- 172.21.15.204:0/782238826 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f583c0a4520 0x7f583c0969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 1 --2- 172.21.15.204:0/782238826 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f583c06ecf0 0x7f583c135150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 1 -- 172.21.15.204:0/782238826 >> 172.21.15.204:0/782238826 conn(0x7f583c0676a0 msgr2=0x7f583c0a3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:14.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 1 -- 172.21.15.204:0/782238826 shutdown_connections 2024-09-15T01:33:14.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.325+0000 7f5840f1d640 1 -- 172.21.15.204:0/782238826 wait complete. 2024-09-15T01:33:14.338 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool get testpool1234 pg_num_max 2024-09-15T01:33:14.338 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:14.338 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool get testpool1234 pg_num_max 2024-09-15T01:33:14.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.416+0000 7ff81d55d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:14.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.416+0000 7ff81d55d640 10 monclient: build_initial_monmap 2024-09-15T01:33:14.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.416+0000 7ff81d55d640 10 monclient: monmap: 2024-09-15T01:33:14.418 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:14.418 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:14.418 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:14.416924+0000 2024-09-15T01:33:14.418 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:14.416924+0000 2024-09-15T01:33:14.418 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:14.418 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:14.418 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:14.419 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:14.419 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:14.419 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:14.419 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:14.419 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:14.419 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:14.419 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:14.419 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:14.419 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:14.419 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:14.419 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:14.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 1 Processor -- start 2024-09-15T01:33:14.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 1 -- start start 2024-09-15T01:33:14.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 10 monclient: init 2024-09-15T01:33:14.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:14.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 10 monclient: _add_conns ranks=[0,10,5,6,3,9,4,2,7,1,8] 2024-09-15T01:33:14.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff8180a2e30 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:14.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff81809fb80 0x7ff81809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:14.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 10 monclient(hunting): picked mon.noname-f con 0x7ff81809fb80 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:14.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff818137d20 0x7ff81813a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:14.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 10 monclient(hunting): picked mon.noname-h con 0x7ff818137d20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:14.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81804b720 con 0x7ff8180a2e30 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81809e6c0 con 0x7ff81809fb80 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8180a04b0 con 0x7ff818137d20 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff81d55d640 10 monclient(hunting): authenticate will time out at 810.059849s 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff8177fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff818137d20 0x7ff81813a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.417+0000 7ff80ffff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff81809fb80 0x7ff81809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff818137d20 0x7ff81813a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:50706/0 (socket says 172.21.15.204:50706) 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 1 -- 172.21.15.204:0/1949734532 learned_addr learned my addr 172.21.15.204:0/1949734532 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff80ffff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff81809fb80 0x7ff81809ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:34472/0 (socket says 172.21.15.204:34472) 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 10 monclient(hunting): get_auth_request con 0x7ff818137d20 auth_method 0 2024-09-15T01:33:14.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff80ffff640 10 monclient(hunting): get_auth_request con 0x7ff81809fb80 auth_method 0 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff80ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff80ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff80ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff80ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff80ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff80ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:14.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 10 monclient(hunting): handle_auth_done global_id 4658 payload 274 2024-09-15T01:33:14.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 1 -- 172.21.15.204:0/1949734532 >> v2:172.21.15.204:3300/0 conn(0x7ff81809fb80 msgr2=0x7ff81809ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:14.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 1 --2- 172.21.15.204:0/1949734532 >> v2:172.21.15.204:3300/0 conn(0x7ff81809fb80 0x7ff81809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 1 -- 172.21.15.204:0/1949734532 >> v1:172.21.15.178:6789/0 conn(0x7ff8180a2e30 legacy=0x7ff818132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:14.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:14.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 1 monclient: found mon.noname-h 2024-09-15T01:33:14.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:14.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:14.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff8177fe640 1 -- 172.21.15.204:0/1949734532 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff818133170 con 0x7ff818137d20 2024-09-15T01:33:14.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.418+0000 7ff80ffff640 1 --2- 172.21.15.204:0/1949734532 >> v2:172.21.15.204:3300/0 conn(0x7ff81809fb80 0x7ff81809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:14.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8177fe640 1 --2- 172.21.15.204:0/1949734532 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff818137d20 0x7ff81813a100 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff810009f20 tx=0x7ff810009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a011037879b41a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:14.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8167fc640 1 -- 172.21.15.204:0/1949734532 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff810007880 con 0x7ff818137d20 2024-09-15T01:33:14.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8167fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8167fc640 10 monclient: dump: 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:14.432 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8167fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8167fc640 1 -- 172.21.15.204:0/1949734532 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81002d050 con 0x7ff818137d20 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 5 monclient: authenticate success, global_id 4658 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8167fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:14.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8167fc640 1 -- 172.21.15.204:0/1949734532 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff810030360 con 0x7ff818137d20 2024-09-15T01:33:14.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 4 set_mon_vals no callback set 2024-09-15T01:33:14.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:14.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8167fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:14.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff8167fc640 10 monclient: dump: 2024-09-15T01:33:14.434 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:14.434 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:14.434 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.434 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.434 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:14.434 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:14.434 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:14.435 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:14.435 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:14.435 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:14.435 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:14.435 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:14.435 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:14.435 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:14.435 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:14.435 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:14.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 10 monclient: shutdown 2024-09-15T01:33:14.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 1 -- 172.21.15.204:0/1949734532 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff818137d20 msgr2=0x7ff81813a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 1 --2- 172.21.15.204:0/1949734532 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff818137d20 0x7ff81813a100 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff810009f20 tx=0x7ff810009840 comp rx=0 tx=0).stop 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 1 -- 172.21.15.204:0/1949734532 shutdown_connections 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 1 --2- 172.21.15.204:0/1949734532 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff818137d20 0x7ff81813a100 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 1 --2- 172.21.15.204:0/1949734532 >> v2:172.21.15.204:3300/0 conn(0x7ff81809fb80 0x7ff81809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 1 -- 172.21.15.204:0/1949734532 >> 172.21.15.204:0/1949734532 conn(0x7ff81806e860 msgr2=0x7ff81806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 1 -- 172.21.15.204:0/1949734532 shutdown_connections 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.419+0000 7ff81d55d640 1 -- 172.21.15.204:0/1949734532 wait complete. 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.420+0000 7ff81d55d640 10 monclient: build_initial_monmap 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.420+0000 7ff81d55d640 10 monclient: monmap: 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:14.436 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:14.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.420+0000 7ff81d55d640 1 Processor -- start 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.420+0000 7ff81d55d640 1 -- start start 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.420+0000 7ff81d55d640 10 monclient: init 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient: _add_conns ranks=[7,1,0,2,5,3,4,6,8] 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff81809fb80 0x7ff818098fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient(hunting): picked mon.noname-i con 0x7ff81809fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8180a2e30 0x7ff8180994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient(hunting): picked mon.noname-f con 0x7ff8180a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff818137d20 0x7ff818099a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient(hunting): picked mon.noname-a con 0x7ff818137d20 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81811c7e0 con 0x7ff818137d20 2024-09-15T01:33:14.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81809afb0 con 0x7ff8180a2e30 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff81809c560 con 0x7ff81809fb80 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient(hunting): authenticate will time out at 810.063118s 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff816ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff81809fb80 0x7ff818098fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8180a2e30 0x7ff8180994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff816ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff81809fb80 0x7ff818098fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49596/0 (socket says 172.21.15.204:49596) 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff816ffd640 1 -- 172.21.15.204:0/3630512457 learned_addr learned my addr 172.21.15.204:0/3630512457 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8180a2e30 0x7ff8180994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:34476/0 (socket says 172.21.15.204:34476) 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff816ffd640 10 monclient(hunting): get_auth_request con 0x7ff81809fb80 auth_method 0 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff816ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff816ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff816ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:14.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 10 monclient(hunting): get_auth_request con 0x7ff8180a2e30 auth_method 0 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff8177fe640 1 --2- 172.21.15.204:0/3630512457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff818137d20 0x7ff818099a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff8177fe640 10 monclient(hunting): get_auth_request con 0x7ff818137d20 auth_method 0 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff8177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff8177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff8177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff816ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff816ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff816ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:14.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff8177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff8177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff8177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 10 monclient(hunting): handle_auth_done global_id 4735 payload 824 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 1 -- 172.21.15.204:0/3630512457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff81809fb80 msgr2=0x7ff818098fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 1 --2- 172.21.15.204:0/3630512457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff81809fb80 0x7ff818098fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 1 -- 172.21.15.204:0/3630512457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff818137d20 msgr2=0x7ff818099a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 1 --2- 172.21.15.204:0/3630512457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff818137d20 0x7ff818099a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 1 monclient: found mon.noname-f 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 1 -- 172.21.15.204:0/3630512457 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff800009390 con 0x7ff8180a2e30 2024-09-15T01:33:14.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff816ffd640 1 --2- 172.21.15.204:0/3630512457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff81809fb80 0x7ff818098fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80ffff640 1 --2- 172.21.15.204:0/3630512457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8180a2e30 0x7ff8180994e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff80000ea50 tx=0x7ff8000090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80effd640 1 -- 172.21.15.204:0/3630512457 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff800009670 con 0x7ff8180a2e30 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80effd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80effd640 10 monclient: dump: 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:14.442 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80effd640 10 monclient: _finish_auth 0 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80effd640 1 -- 172.21.15.204:0/3630512457 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80000ce90 con 0x7ff8180a2e30 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 5 monclient: authenticate success, global_id 4735 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient: _renew_subs 2024-09-15T01:33:14.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 1 -- 172.21.15.204:0/3630512457 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff810009ae0 con 0x7ff8180a2e30 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80effd640 1 -- 172.21.15.204:0/3630512457 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff8000138f0 con 0x7ff8180a2e30 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff80effd640 10 monclient: dump: 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:14.444 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff815ffb640 4 set_mon_vals no callback set 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient: _renew_subs 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.421+0000 7ff81d55d640 1 -- 172.21.15.204:0/3630512457 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81809b670 con 0x7ff8180a2e30 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.422+0000 7ff80effd640 1 -- 172.21.15.204:0/3630512457 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff800013470 con 0x7ff8180a2e30 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.422+0000 7ff80effd640 1 --2- 172.21.15.204:0/3630512457 >> v2:172.21.15.204:6812/28210 conn(0x7ff7f80397b0 0x7ff7f803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.422+0000 7ff81d55d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:14.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.422+0000 7ff81d55d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:14.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.422+0000 7ff81d55d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:14.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.422+0000 7ff80effd640 1 -- 172.21.15.204:0/3630512457 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3853+0+0 (secure 0 0 0) 0x7ff80004b6b0 con 0x7ff8180a2e30 2024-09-15T01:33:14.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.422+0000 7ff81d55d640 1 -- 172.21.15.204:0/3630512457 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81809b8c0 con 0x7ff8180a2e30 2024-09-15T01:33:14.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.422+0000 7ff816ffd640 1 --2- 172.21.15.204:0/3630512457 >> v2:172.21.15.204:6812/28210 conn(0x7ff7f80397b0 0x7ff7f803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:14.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.423+0000 7ff816ffd640 10 monclient: get_auth_request con 0x7ff7f80397b0 auth_method 0 2024-09-15T01:33:14.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.423+0000 7ff816ffd640 1 --2- 172.21.15.204:0/3630512457 >> v2:172.21.15.204:6812/28210 conn(0x7ff7f80397b0 0x7ff7f803bc60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff808007fd0 tx=0x7ff808009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:14.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.427+0000 7ff80effd640 1 -- 172.21.15.204:0/3630512457 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff81809b8c0 con 0x7ff8180a2e30 2024-09-15T01:33:14.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.427+0000 7ff80effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:14.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.427+0000 7ff80effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:14.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.579+0000 7ff81d55d640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T01:33:14.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.579+0000 7ff81d55d640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T01:33:14.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.579+0000 7ff81d55d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.579+0000 7ff81d55d640 1 -- 172.21.15.204:0/3630512457 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"} v 0) v1 -- 0x7ff81809b8c0 con 0x7ff8180a2e30 2024-09-15T01:33:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.580+0000 7ff80effd640 1 -- 172.21.15.204:0/3630512457 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}]=-2 option 'pg_num_max' is not set on pool 'testpool1234' v29) v1 ==== 158+0+0 (secure 0 0 0) 0x7ff81809b8c0 con 0x7ff8180a2e30 2024-09-15T01:33:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.580+0000 7ff80effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-09-15T01:33:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.580+0000 7ff80effd640 10 monclient: _finish_command 2 = mon:2 option 'pg_num_max' is not set on pool 'testpool1234' 2024-09-15T01:33:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:Error ENOENT: option 'pg_num_max' is not set on pool 'testpool1234' 2024-09-15T01:33:14.585 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:14.585 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete testpool1234 testpool1234 --yes-i-really-really-mean-it 2024-09-15T01:33:14.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.659+0000 7fbbb199d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.659+0000 7fbbb199d640 10 monclient: build_initial_monmap 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.659+0000 7fbbb199d640 10 monclient: monmap: 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:14.659843+0000 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:14.659843+0000 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:14.661 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:14.662 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:14.662 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:14.662 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:14.662 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:14.662 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:14.662 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:14.662 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:14.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.659+0000 7fbbb199d640 1 Processor -- start 2024-09-15T01:33:14.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.659+0000 7fbbb199d640 1 -- start start 2024-09-15T01:33:14.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.659+0000 7fbbb199d640 10 monclient: init 2024-09-15T01:33:14.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:14.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 10 monclient: _add_conns ranks=[7,4,0,3,8,6,1,5,9,2,10] 2024-09-15T01:33:14.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbbac0a2cd0 0x7fbbac132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:14.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 10 monclient(hunting): picked mon.noname-i con 0x7fbbac0a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:14.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbac09fa20 0x7fbbac09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:14.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 10 monclient(hunting): picked mon.noname-c con 0x7fbbac09fa20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:14.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbbac0a08e0 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:14.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbac0648a0 con 0x7fbbac0a08e0 2024-09-15T01:33:14.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:14.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac0a30a0 con 0x7fbbac09fa20 2024-09-15T01:33:14.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:14.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac0a0330 con 0x7fbbac0a2cd0 2024-09-15T01:33:14.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbb199d640 10 monclient(hunting): authenticate will time out at 810.302555s 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbbac0a2cd0 0x7fbbac132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbbac0a2cd0 0x7fbbac132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49602/0 (socket says 172.21.15.204:49602) 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 1 -- 172.21.15.204:0/3366533834 learned_addr learned my addr 172.21.15.204:0/3366533834 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 10 monclient(hunting): get_auth_request con 0x7fbbac0a2cd0 auth_method 0 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaa7fc640 1 --2- 172.21.15.204:0/3366533834 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbac09fa20 0x7fbbac09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:14.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaa7fc640 10 monclient(hunting): get_auth_request con 0x7fbbac09fa20 auth_method 0 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 10 monclient(hunting): handle_auth_done global_id 4651 payload 274 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 1 -- 172.21.15.204:0/3366533834 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbac09fa20 msgr2=0x7fbbac09fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 1 --2- 172.21.15.204:0/3366533834 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbac09fa20 0x7fbbac09fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 1 -- 172.21.15.204:0/3366533834 >> v1:172.21.15.178:6789/0 conn(0x7fbbac0a08e0 legacy=0x7fbbac0a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 1 monclient: found mon.noname-i 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.660+0000 7fbbaaffd640 1 -- 172.21.15.204:0/3366533834 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac0a1db0 con 0x7fbbac0a2cd0 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbbaaffd640 1 --2- 172.21.15.204:0/3366533834 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbbac0a2cd0 0x7fbbac132c00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009f50 tx=0x7fbba4009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=f4a55979d4991859 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:14.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbba9ffb640 1 -- 172.21.15.204:0/3366533834 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbba4007880 con 0x7fbbac0a2cd0 2024-09-15T01:33:14.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbba9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:14.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbba9ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:14.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbba9ffb640 10 monclient: dump: 2024-09-15T01:33:14.674 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:14.674 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:14.674 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.674 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.674 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:14.674 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:14.674 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:14.674 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:14.674 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbba9ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbba9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbba9ffb640 1 -- 172.21.15.204:0/3366533834 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba402d050 con 0x7fbbac0a2cd0 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbbb199d640 5 monclient: authenticate success, global_id 4651 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbbb199d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbba9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:14.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbbb199d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:14.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbba9ffb640 1 -- 172.21.15.204:0/3366533834 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbba4030360 con 0x7fbbac0a2cd0 2024-09-15T01:33:14.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbbb199d640 4 set_mon_vals no callback set 2024-09-15T01:33:14.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbba9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:14.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbba9ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:14.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.661+0000 7fbba9ffb640 10 monclient: dump: 2024-09-15T01:33:14.676 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:14.676 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:14.676 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.676 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.676 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.662+0000 7fbbb199d640 10 monclient: shutdown 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.662+0000 7fbbb199d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:14.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.662+0000 7fbbb199d640 1 -- 172.21.15.204:0/3366533834 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbbac0a2cd0 msgr2=0x7fbbac132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:14.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.662+0000 7fbbb199d640 1 --2- 172.21.15.204:0/3366533834 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbbac0a2cd0 0x7fbbac132c00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009f50 tx=0x7fbba4009840 comp rx=0 tx=0).stop 2024-09-15T01:33:14.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.662+0000 7fbbb199d640 1 -- 172.21.15.204:0/3366533834 shutdown_connections 2024-09-15T01:33:14.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.662+0000 7fbbb199d640 1 --2- 172.21.15.204:0/3366533834 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbac09fa20 0x7fbbac09fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.662+0000 7fbbb199d640 1 --2- 172.21.15.204:0/3366533834 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbbac0a2cd0 0x7fbbac132c00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.662+0000 7fbbb199d640 1 -- 172.21.15.204:0/3366533834 >> 172.21.15.204:0/3366533834 conn(0x7fbbac06e860 msgr2=0x7fbbac06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:14.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.662+0000 7fbbb199d640 1 -- 172.21.15.204:0/3366533834 shutdown_connections 2024-09-15T01:33:14.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.662+0000 7fbbb199d640 1 -- 172.21.15.204:0/3366533834 wait complete. 2024-09-15T01:33:14.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient: build_initial_monmap 2024-09-15T01:33:14.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient: monmap: 2024-09-15T01:33:14.678 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:14.678 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:14.678 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 1 Processor -- start 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 1 -- start start 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient: init 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient: _add_conns ranks=[2,5,3,4,6,7,0,8,1] 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbac09fa20 0x7fbbac131950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient(hunting): picked mon.noname-b con 0x7fbbac09fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbbac0a08e0 0x7fbbac11d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient(hunting): picked mon.noname-h con 0x7fbbac0a08e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbac0a2cd0 0x7fbbac11da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient(hunting): picked mon.noname-g con 0x7fbbac0a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac09d3d0 con 0x7fbbac09fa20 2024-09-15T01:33:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac09cb60 con 0x7fbbac0a2cd0 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac142b10 con 0x7fbbac0a08e0 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbb199d640 10 monclient(hunting): authenticate will time out at 810.305893s 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbab7fe640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbac0a2cd0 msgr2=0x7fbbac11da10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbab7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbac0a2cd0 0x7fbbac11da10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.663+0000 7fbbaa7fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbbac0a08e0 0x7fbbac11d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbbac0a08e0 0x7fbbac11d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:50722/0 (socket says 172.21.15.204:50722) 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 1 -- 172.21.15.204:0/3788155325 learned_addr learned my addr 172.21.15.204:0/3788155325 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 10 monclient(hunting): get_auth_request con 0x7fbbac0a08e0 auth_method 0 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaaffd640 1 --2- 172.21.15.204:0/3788155325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbac09fa20 0x7fbbac131950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaaffd640 10 monclient(hunting): get_auth_request con 0x7fbbac09fa20 auth_method 0 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 10 monclient(hunting): handle_auth_done global_id 4667 payload 824 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 1 -- 172.21.15.204:0/3788155325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbac0a2cd0 msgr2=0x7fbbac11da10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 1 --2- 172.21.15.204:0/3788155325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbac0a2cd0 0x7fbbac11da10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 1 -- 172.21.15.204:0/3788155325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbac09fa20 msgr2=0x7fbbac131950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 1 --2- 172.21.15.204:0/3788155325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbac09fa20 0x7fbbac131950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 1 monclient: found mon.noname-h 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 1 -- 172.21.15.204:0/3788155325 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba4009ae0 con 0x7fbbac0a08e0 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaaffd640 1 --2- 172.21.15.204:0/3788155325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbac09fa20 0x7fbbac131950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbaa7fc640 1 --2- 172.21.15.204:0/3788155325 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbbac0a08e0 0x7fbbac11d4d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400d8f0 tx=0x7fbb94008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba2ffd640 1 -- 172.21.15.204:0/3788155325 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb9400de80 con 0x7fbbac0a08e0 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba2ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba2ffd640 10 monclient: dump: 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:14.684 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba2ffd640 1 -- 172.21.15.204:0/3788155325 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9400bd50 con 0x7fbbac0a08e0 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbb199d640 5 monclient: authenticate success, global_id 4667 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba2ffd640 1 -- 172.21.15.204:0/3788155325 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb9400bec0 con 0x7fbbac0a08e0 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbb199d640 10 monclient: _renew_subs 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbb199d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbb199d640 1 -- 172.21.15.204:0/3788155325 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac120e00 con 0x7fbbac0a08e0 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba97fa640 4 set_mon_vals no callback set 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbba2ffd640 10 monclient: dump: 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbb199d640 10 monclient: _renew_subs 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbb199d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.664+0000 7fbbb199d640 1 -- 172.21.15.204:0/3788155325 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbac11efd0 con 0x7fbbac0a08e0 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.665+0000 7fbba2ffd640 1 -- 172.21.15.204:0/3788155325 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fbb9400b8d0 con 0x7fbbac0a08e0 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.665+0000 7fbbb199d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.665+0000 7fbbb199d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.665+0000 7fbbb199d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.665+0000 7fbba2ffd640 1 --2- 172.21.15.204:0/3788155325 >> v2:172.21.15.204:6812/28210 conn(0x7fbb8c0397b0 0x7fbb8c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.665+0000 7fbbb199d640 1 -- 172.21.15.204:0/3788155325 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbac0a0120 con 0x7fbbac0a08e0 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.665+0000 7fbba2ffd640 1 -- 172.21.15.204:0/3788155325 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3853+0+0 (secure 0 0 0) 0x7fbb9404b6d0 con 0x7fbbac0a08e0 2024-09-15T01:33:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.665+0000 7fbbaaffd640 1 --2- 172.21.15.204:0/3788155325 >> v2:172.21.15.204:6812/28210 conn(0x7fbb8c0397b0 0x7fbb8c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.665+0000 7fbbaaffd640 10 monclient: get_auth_request con 0x7fbb8c0397b0 auth_method 0 2024-09-15T01:33:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.666+0000 7fbbaaffd640 1 --2- 172.21.15.204:0/3788155325 >> v2:172.21.15.204:6812/28210 conn(0x7fbb8c0397b0 0x7fbb8c03bc60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbba4033000 tx=0x7fbba4036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.670+0000 7fbba2ffd640 1 -- 172.21.15.204:0/3788155325 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fbb9400bb80 con 0x7fbbac0a08e0 2024-09-15T01:33:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.670+0000 7fbba2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.670+0000 7fbba2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:14.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.828+0000 7fbbb199d640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:14.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.828+0000 7fbbb199d640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:14.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.828+0000 7fbbb199d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:14.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:14.828+0000 7fbbb199d640 1 -- 172.21.15.204:0/3788155325 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbbac070b80 con 0x7fbbac0a08e0 2024-09-15T01:33:15.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.327+0000 7fbba2ffd640 1 -- 172.21.15.204:0/3788155325 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}]=0 pool 'testpool1234' removed v30) v1 ==== 176+0+0 (secure 0 0 0) 0x7fbb9401d3c0 con 0x7fbbac0a08e0 2024-09-15T01:33:15.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.327+0000 7fbba2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:15.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.327+0000 7fbba2ffd640 10 monclient: _finish_command 2 = system:0 pool 'testpool1234' removed 2024-09-15T01:33:15.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.424+0000 7fbbb199d640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:15.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.425+0000 7fbbb199d640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:15.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.425+0000 7fbbb199d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:15.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.425+0000 7fbbb199d640 1 -- 172.21.15.204:0/3788155325 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbbac120f90 con 0x7fbbac0a08e0 2024-09-15T01:33:15.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.425+0000 7fbba2ffd640 1 -- 172.21.15.204:0/3788155325 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}]=0 pool 'testpool1234' does not exist v30) v1 ==== 183+0+0 (secure 0 0 0) 0x7fbb94011340 con 0x7fbbac0a08e0 2024-09-15T01:33:15.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.425+0000 7fbba2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:15.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.425+0000 7fbba2ffd640 10 monclient: _finish_command 3 = system:0 pool 'testpool1234' does not exist 2024-09-15T01:33:15.427 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'testpool1234' does not exist 2024-09-15T01:33:15.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 1 -- 172.21.15.204:0/3788155325 >> v2:172.21.15.204:6812/28210 conn(0x7fbb8c0397b0 msgr2=0x7fbb8c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:15.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 1 --2- 172.21.15.204:0/3788155325 >> v2:172.21.15.204:6812/28210 conn(0x7fbb8c0397b0 0x7fbb8c03bc60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbba4033000 tx=0x7fbba4036000 comp rx=0 tx=0).stop 2024-09-15T01:33:15.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 10 monclient: shutdown 2024-09-15T01:33:15.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:15.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 1 -- 172.21.15.204:0/3788155325 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbbac0a08e0 msgr2=0x7fbbac11d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:15.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 1 --2- 172.21.15.204:0/3788155325 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbbac0a08e0 0x7fbbac11d4d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400d8f0 tx=0x7fbb94008040 comp rx=0 tx=0).stop 2024-09-15T01:33:15.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 1 -- 172.21.15.204:0/3788155325 shutdown_connections 2024-09-15T01:33:15.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 1 --2- 172.21.15.204:0/3788155325 >> v2:172.21.15.204:6812/28210 conn(0x7fbb8c0397b0 0x7fbb8c03bc60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 1 --2- 172.21.15.204:0/3788155325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbac0a2cd0 0x7fbbac11da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.429 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 1 --2- 172.21.15.204:0/3788155325 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbbac0a08e0 0x7fbbac11d4d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 1 --2- 172.21.15.204:0/3788155325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbac09fa20 0x7fbbac131950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 1 -- 172.21.15.204:0/3788155325 >> 172.21.15.204:0/3788155325 conn(0x7fbbac06e860 msgr2=0x7fbbac0a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:15.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 1 -- 172.21.15.204:0/3788155325 shutdown_connections 2024-09-15T01:33:15.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.427+0000 7fbbb199d640 1 -- 172.21.15.204:0/3788155325 wait complete. 2024-09-15T01:33:15.437 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool create foo 123 123 invalid foo-profile foo-rule 2024-09-15T01:33:15.437 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:15.437 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool create foo 123 123 invalid foo-profile foo-rule 2024-09-15T01:33:15.515 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.514+0000 7fe9e159d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:15.515 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.514+0000 7fe9e159d640 10 monclient: build_initial_monmap 2024-09-15T01:33:15.515 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.514+0000 7fe9e159d640 10 monclient: monmap: 2024-09-15T01:33:15.515 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:15.515 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:15.514622+0000 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:15.514622+0000 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:15.516 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:15.517 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:15.517 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:15.517 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:15.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.514+0000 7fe9e159d640 1 Processor -- start 2024-09-15T01:33:15.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.514+0000 7fe9e159d640 1 -- start start 2024-09-15T01:33:15.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.514+0000 7fe9e159d640 10 monclient: init 2024-09-15T01:33:15.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:15.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 10 monclient: _add_conns ranks=[1,8,7,2,3,4,9,6,10,0,5] 2024-09-15T01:33:15.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe9dc0a3560 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:15.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9dc09b3f0 0x7fe9dc137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 10 monclient(hunting): picked mon.noname-e con 0x7fe9dc09b3f0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc09b7c0 0x7fe9dc139f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 10 monclient(hunting): picked mon.noname-i con 0x7fe9dc09b7c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9dc04b720 con 0x7fe9dc0a3560 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc09a440 con 0x7fe9dc09b7c0 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0a3930 con 0x7fe9dc09b3f0 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9e159d640 10 monclient(hunting): authenticate will time out at 811.157440s 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc09b7c0 0x7fe9dc139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9daffd640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fe9dc0a3560 0x7fe9dc09cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:58102/0 (socket says 172.21.15.204:58102) 2024-09-15T01:33:15.526 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9daffd640 1 -- 172.21.15.204:0/3204689594 learned_addr learned my addr 172.21.15.204:0/3204689594 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 10 monclient(hunting): get_auth_request con 0x7fe9dc09b7c0 auth_method 0 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 10 monclient(hunting): handle_auth_done global_id 4660 payload 274 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 1 -- 172.21.15.204:0/3204689594 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9dc09b3f0 msgr2=0x7fe9dc137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9da7fc640 1 -- 172.21.15.204:0/3204689594 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228109658 0 0) 0x7fe9dc04b720 con 0x7fe9dc0a3560 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9d3fff640 1 --2- 172.21.15.204:0/3204689594 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9dc09b3f0 0x7fe9dc137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 1 --2- 172.21.15.204:0/3204689594 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9dc09b3f0 0x7fe9dc137a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 1 -- 172.21.15.204:0/3204689594 >> v1:172.21.15.204:6789/0 conn(0x7fe9dc0a3560 legacy=0x7fe9dc09cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:15.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 1 monclient: found mon.noname-i 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9db7fe640 1 -- 172.21.15.204:0/3204689594 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc09e0d0 con 0x7fe9dc09b7c0 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.515+0000 7fe9da7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9db7fe640 1 --2- 172.21.15.204:0/3204689594 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc09b7c0 0x7fe9dc139f90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009e90 tx=0x7fe9d4009780 comp rx=0 tx=0).ready entity=mon.7 client_cookie=e803b25b755a2113 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9da7fc640 1 -- 172.21.15.204:0/3204689594 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9d40079d0 con 0x7fe9dc09b7c0 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9da7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9da7fc640 10 monclient: dump: 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:15.528 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:15.529 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9da7fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9da7fc640 1 -- 172.21.15.204:0/3204689594 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d402d050 con 0x7fe9dc09b7c0 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9e159d640 5 monclient: authenticate success, global_id 4660 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9e159d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9da7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9e159d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9e159d640 4 set_mon_vals no callback set 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9da7fc640 1 -- 172.21.15.204:0/3204689594 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9d4007b90 con 0x7fe9dc09b7c0 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9da7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.516+0000 7fe9da7fc640 10 monclient: dump: 2024-09-15T01:33:15.530 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:15.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.517+0000 7fe9e159d640 10 monclient: shutdown 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.517+0000 7fe9e159d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.517+0000 7fe9e159d640 1 -- 172.21.15.204:0/3204689594 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc09b7c0 msgr2=0x7fe9dc139f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.517+0000 7fe9e159d640 1 --2- 172.21.15.204:0/3204689594 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc09b7c0 0x7fe9dc139f90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009e90 tx=0x7fe9d4009780 comp rx=0 tx=0).stop 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.517+0000 7fe9e159d640 1 -- 172.21.15.204:0/3204689594 shutdown_connections 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.517+0000 7fe9e159d640 1 --2- 172.21.15.204:0/3204689594 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc09b7c0 0x7fe9dc139f90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.517+0000 7fe9e159d640 1 --2- 172.21.15.204:0/3204689594 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9dc09b3f0 0x7fe9dc137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.517+0000 7fe9e159d640 1 -- 172.21.15.204:0/3204689594 >> 172.21.15.204:0/3204689594 conn(0x7fe9dc0687b0 msgr2=0x7fe9dc0a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.517+0000 7fe9e159d640 1 -- 172.21.15.204:0/3204689594 shutdown_connections 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.517+0000 7fe9e159d640 1 -- 172.21.15.204:0/3204689594 wait complete. 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.517+0000 7fe9e159d640 10 monclient: build_initial_monmap 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.517+0000 7fe9e159d640 10 monclient: monmap: 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:15.532 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:15.533 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 1 Processor -- start 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 1 -- start start 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 10 monclient: init 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 10 monclient: _add_conns ranks=[2,3,8,1,0,5,4,6,7] 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9dc09b3f0 0x7fe9dc13e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 10 monclient(hunting): picked mon.noname-b con 0x7fe9dc09b3f0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9dc09b7c0 0x7fe9dc13d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 10 monclient(hunting): picked mon.noname-g con 0x7fe9dc09b7c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9dc0a3560 0x7fe9dc141690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 10 monclient(hunting): picked mon.noname-e con 0x7fe9dc0a3560 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:15.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc06f000 con 0x7fe9dc09b3f0 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc06ea80 con 0x7fe9dc09b7c0 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc13ed10 con 0x7fe9dc0a3560 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9e159d640 10 monclient(hunting): authenticate will time out at 811.160664s 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9d3fff640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9dc09b7c0 msgr2=0x7fe9dc13d7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9d3fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9dc09b7c0 0x7fe9dc13d7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9daffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9dc09b3f0 0x7fe9dc13e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9db7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9dc0a3560 0x7fe9dc141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9daffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9dc09b3f0 0x7fe9dc13e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:35268/0 (socket says 172.21.15.204:35268) 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9daffd640 1 -- 172.21.15.204:0/1927971486 learned_addr learned my addr 172.21.15.204:0/1927971486 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:15.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9daffd640 10 monclient(hunting): get_auth_request con 0x7fe9dc09b3f0 auth_method 0 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9daffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9daffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9db7fe640 10 monclient(hunting): get_auth_request con 0x7fe9dc0a3560 auth_method 0 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.518+0000 7fe9db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:15.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9daffd640 10 monclient(hunting): handle_auth_done global_id 4628 payload 824 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9daffd640 1 -- 172.21.15.204:0/1927971486 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9dc0a3560 msgr2=0x7fe9dc141690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9daffd640 1 --2- 172.21.15.204:0/1927971486 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9dc0a3560 0x7fe9dc141690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9daffd640 1 -- 172.21.15.204:0/1927971486 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9dc09b7c0 msgr2=0x7fe9dc13d7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9daffd640 1 --2- 172.21.15.204:0/1927971486 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9dc09b7c0 0x7fe9dc13d7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9daffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9daffd640 1 monclient: found mon.noname-b 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9daffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9daffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9daffd640 1 -- 172.21.15.204:0/1927971486 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d4009a20 con 0x7fe9dc09b3f0 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9daffd640 1 --2- 172.21.15.204:0/1927971486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9dc09b3f0 0x7fe9dc13e140 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00c510 tx=0x7fe9cc007f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d2ffd640 1 -- 172.21.15.204:0/1927971486 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9cc0070d0 con 0x7fe9dc09b3f0 2024-09-15T01:33:15.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d2ffd640 10 monclient: dump: 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:15.538 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9e159d640 5 monclient: authenticate success, global_id 4628 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d2ffd640 1 -- 172.21.15.204:0/1927971486 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc007250 con 0x7fe9dc09b3f0 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9e159d640 10 monclient: _renew_subs 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d2ffd640 1 -- 172.21.15.204:0/1927971486 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9cc011360 con 0x7fe9dc09b3f0 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9e159d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:15.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9e159d640 1 -- 172.21.15.204:0/1927971486 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc122890 con 0x7fe9dc09b3f0 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d9ffb640 4 set_mon_vals no callback set 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9d2ffd640 10 monclient: dump: 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:15.540 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:15.541 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:15.541 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:15.541 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:15.541 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:15.541 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:15.541 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:15.541 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:15.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9e159d640 10 monclient: _renew_subs 2024-09-15T01:33:15.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9e159d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:15.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.519+0000 7fe9e159d640 1 -- 172.21.15.204:0/1927971486 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc13dd30 con 0x7fe9dc09b3f0 2024-09-15T01:33:15.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.520+0000 7fe9d2ffd640 1 -- 172.21.15.204:0/1927971486 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe9cc007400 con 0x7fe9dc09b3f0 2024-09-15T01:33:15.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.520+0000 7fe9e159d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:15.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.520+0000 7fe9e159d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:15.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.520+0000 7fe9e159d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:15.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.520+0000 7fe9e159d640 1 -- 172.21.15.204:0/1927971486 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc070cc0 con 0x7fe9dc09b3f0 2024-09-15T01:33:15.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.520+0000 7fe9d2ffd640 1 --2- 172.21.15.204:0/1927971486 >> v2:172.21.15.204:6812/28210 conn(0x7fe9bc0397b0 0x7fe9bc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:15.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.520+0000 7fe9d2ffd640 1 -- 172.21.15.204:0/1927971486 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3502+0+0 (secure 0 0 0) 0x7fe9cc04a090 con 0x7fe9dc09b3f0 2024-09-15T01:33:15.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.520+0000 7fe9d3fff640 1 --2- 172.21.15.204:0/1927971486 >> v2:172.21.15.204:6812/28210 conn(0x7fe9bc0397b0 0x7fe9bc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:15.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.520+0000 7fe9d3fff640 10 monclient: get_auth_request con 0x7fe9bc0397b0 auth_method 0 2024-09-15T01:33:15.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.521+0000 7fe9d3fff640 1 --2- 172.21.15.204:0/1927971486 >> v2:172.21.15.204:6812/28210 conn(0x7fe9bc0397b0 0x7fe9bc03bc60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006fd0 tx=0x7fe9c4008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:15.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.525+0000 7fe9d2ffd640 1 -- 172.21.15.204:0/1927971486 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe9cc00e890 con 0x7fe9dc09b3f0 2024-09-15T01:33:15.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.525+0000 7fe9d2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:15.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.525+0000 7fe9d2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:15.684 INFO:tasks.workunit.client.0.smithi204.stderr:foo-rule not valid: foo-rule doesn't represent a float 2024-09-15T01:33:15.684 INFO:tasks.workunit.client.0.smithi204.stderr:Invalid command: unused arguments: ['foo-rule'] 2024-09-15T01:33:15.684 INFO:tasks.workunit.client.0.smithi204.stderr:osd pool create [] [] [] [] [] [] [] [] [] [] [--bulk] [] [] : create pool 2024-09-15T01:33:15.684 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: invalid command 2024-09-15T01:33:15.685 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:15.685 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool create foo 123 123 replicated 2024-09-15T01:33:15.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.760+0000 7f7988868640 10 monclient: get_monmap_and_config 2024-09-15T01:33:15.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.760+0000 7f7988868640 10 monclient: build_initial_monmap 2024-09-15T01:33:15.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.760+0000 7f7988868640 10 monclient: monmap: 2024-09-15T01:33:15.761 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:15.762 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:15.762 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:15.760650+0000 2024-09-15T01:33:15.762 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:15.760650+0000 2024-09-15T01:33:15.762 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:15.762 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:15.762 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:15.762 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:15.762 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:15.762 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:15.762 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:15.762 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:15.762 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:15.763 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:15.763 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:15.763 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:15.763 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:15.763 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:15.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.760+0000 7f7988868640 1 Processor -- start 2024-09-15T01:33:15.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.760+0000 7f7988868640 1 -- start start 2024-09-15T01:33:15.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.760+0000 7f7988868640 10 monclient: init 2024-09-15T01:33:15.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:15.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 10 monclient: _add_conns ranks=[6,7,0,9,2,8,5,3,10,1,4] 2024-09-15T01:33:15.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7984067130 0x7f7984130d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:15.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 10 monclient(hunting): picked mon.noname-d con 0x7f7984067130 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:15.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7984067500 0x7f79840678d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:15.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 10 monclient(hunting): picked mon.noname-i con 0x7f7984067500 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:15.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f798412d450 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:15.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79840648a0 con 0x7f798412d450 2024-09-15T01:33:15.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:15.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f798409ddd0 con 0x7f7984067130 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f798409ef00 con 0x7f7984067500 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7988868640 10 monclient(hunting): authenticate will time out at 811.403473s 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7984067500 0x7f79840678d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7984067500 0x7f79840678d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49606/0 (socket says 172.21.15.204:49606) 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 1 -- 172.21.15.204:0/3689343965 learned_addr learned my addr 172.21.15.204:0/3689343965 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f798259c640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7984067130 msgr2=0x7f7984130d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 10 monclient(hunting): get_auth_request con 0x7f7984067500 auth_method 0 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:15.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f798259c640 1 --2- 172.21.15.204:0/3689343965 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7984067130 0x7f7984130d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 10 monclient(hunting): handle_auth_done global_id 4669 payload 274 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 1 -- 172.21.15.204:0/3689343965 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7984067130 msgr2=0x7f7984130d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 1 --2- 172.21.15.204:0/3689343965 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7984067130 0x7f7984130d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 1 -- 172.21.15.204:0/3689343965 >> v1:172.21.15.178:6789/0 conn(0x7f798412d450 legacy=0x7f798412d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 1 monclient: found mon.noname-i 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.761+0000 7f7981d9b640 1 -- 172.21.15.204:0/3689343965 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798409d8e0 con 0x7f7984067500 2024-09-15T01:33:15.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f7981d9b640 1 --2- 172.21.15.204:0/3689343965 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7984067500 0x7f79840678d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7978009970 tx=0x7f79780091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=5c0b0dc8c3413c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f798159a640 1 -- 172.21.15.204:0/3689343965 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f797802acc0 con 0x7f7984067500 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f798159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f798159a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f798159a640 10 monclient: dump: 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:15.775 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:15.776 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:15.776 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:15.776 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:15.776 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:15.776 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:15.776 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:15.776 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:15.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f798159a640 10 monclient: _finish_auth 0 2024-09-15T01:33:15.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f798159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:15.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f798159a640 1 -- 172.21.15.204:0/3689343965 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797802ae40 con 0x7f7984067500 2024-09-15T01:33:15.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f7988868640 5 monclient: authenticate success, global_id 4669 2024-09-15T01:33:15.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f7988868640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f798159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f7988868640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f798159a640 1 -- 172.21.15.204:0/3689343965 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7978031ca0 con 0x7f7984067500 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f798159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f7988868640 4 set_mon_vals no callback set 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f798159a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.762+0000 7f798159a640 10 monclient: dump: 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:15.777 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:15.778 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:15.778 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:15.778 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:15.778 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:15.778 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:15.778 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:15.778 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:15.778 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:15.778 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:15.778 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:15.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.763+0000 7f7988868640 10 monclient: shutdown 2024-09-15T01:33:15.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.763+0000 7f7988868640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:15.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.763+0000 7f7988868640 1 -- 172.21.15.204:0/3689343965 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7984067500 msgr2=0x7f79840678d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:15.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.763+0000 7f7988868640 1 --2- 172.21.15.204:0/3689343965 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7984067500 0x7f79840678d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7978009970 tx=0x7f79780091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.763+0000 7f7988868640 1 -- 172.21.15.204:0/3689343965 shutdown_connections 2024-09-15T01:33:15.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.763+0000 7f7988868640 1 --2- 172.21.15.204:0/3689343965 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7984067500 0x7f79840678d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.763+0000 7f7988868640 1 --2- 172.21.15.204:0/3689343965 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7984067130 0x7f7984130d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.763+0000 7f7988868640 1 -- 172.21.15.204:0/3689343965 >> 172.21.15.204:0/3689343965 conn(0x7f79840a2cb0 msgr2=0x7f79840a30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:15.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.763+0000 7f7988868640 1 -- 172.21.15.204:0/3689343965 shutdown_connections 2024-09-15T01:33:15.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.763+0000 7f7988868640 1 -- 172.21.15.204:0/3689343965 wait complete. 2024-09-15T01:33:15.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.763+0000 7f7988868640 10 monclient: build_initial_monmap 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.763+0000 7f7988868640 10 monclient: monmap: 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:15.780 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 1 Processor -- start 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 1 -- start start 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 10 monclient: init 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 10 monclient: _add_conns ranks=[4,8,7,6,1,5,2,3,0] 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7984067130 0x7f798412c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 10 monclient(hunting): picked mon.noname-c con 0x7f7984067130 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7984067500 0x7f798412cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:15.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 10 monclient(hunting): picked mon.noname-e con 0x7f7984067500 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798412d450 0x7f7984119310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 10 monclient(hunting): picked mon.noname-i con 0x7f798412d450 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f798409e250 con 0x7f7984067130 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f798411cca0 con 0x7f798412d450 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f798411c9c0 con 0x7f7984067500 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7988868640 10 monclient(hunting): authenticate will time out at 811.406614s 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798412d450 0x7f7984119310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798412d450 0x7f7984119310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49620/0 (socket says 172.21.15.204:49620) 2024-09-15T01:33:15.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 1 -- 172.21.15.204:0/108378880 learned_addr learned my addr 172.21.15.204:0/108378880 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 10 monclient(hunting): get_auth_request con 0x7f798412d450 auth_method 0 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f798259c640 1 --2- 172.21.15.204:0/108378880 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7984067130 0x7f798412c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7981d9b640 1 --2- 172.21.15.204:0/108378880 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7984067500 0x7f798412cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7981d9b640 10 monclient(hunting): get_auth_request con 0x7f7984067500 auth_method 0 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7981d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7981d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:15.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7981d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:15.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f798259c640 10 monclient(hunting): get_auth_request con 0x7f7984067130 auth_method 0 2024-09-15T01:33:15.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f798259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:15.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f798259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:15.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f798259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:15.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7981d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:15.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7981d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:15.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7981d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:15.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 10 monclient(hunting): handle_auth_done global_id 4678 payload 824 2024-09-15T01:33:15.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 1 -- 172.21.15.204:0/108378880 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7984067500 msgr2=0x7f798412cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:15.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 1 --2- 172.21.15.204:0/108378880 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7984067500 0x7f798412cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 1 -- 172.21.15.204:0/108378880 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7984067130 msgr2=0x7f798412c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:15.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 1 --2- 172.21.15.204:0/108378880 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7984067130 0x7f798412c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:15.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:33:15.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 1 monclient: found mon.noname-i 2024-09-15T01:33:15.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:15.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:15.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.764+0000 7f7982d9d640 1 -- 172.21.15.204:0/108378880 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7978009450 con 0x7f798412d450 2024-09-15T01:33:15.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f798259c640 1 --2- 172.21.15.204:0/108378880 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7984067130 0x7f798412c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:15.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f7982d9d640 1 --2- 172.21.15.204:0/108378880 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798412d450 0x7f7984119310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f797c00b8e0 tx=0x7f797c009400 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:15.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f79727fc640 1 -- 172.21.15.204:0/108378880 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f797c00cd40 con 0x7f798412d450 2024-09-15T01:33:15.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f79727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:15.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f79727fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:15.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f79727fc640 10 monclient: dump: 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:15.786 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f79727fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f79727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f79727fc640 1 -- 172.21.15.204:0/108378880 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c00c040 con 0x7f798412d450 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f7988868640 5 monclient: authenticate success, global_id 4678 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f7988868640 10 monclient: _renew_subs 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f7988868640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f7988868640 1 -- 172.21.15.204:0/108378880 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7984131f10 con 0x7f798412d450 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f79727fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f79727fc640 1 -- 172.21.15.204:0/108378880 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f797c014890 con 0x7f798412d450 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f79727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:15.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f79727fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f79727fc640 10 monclient: dump: 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:15.788 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f7980d99640 4 set_mon_vals no callback set 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f7988868640 10 monclient: _renew_subs 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f7988868640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.765+0000 7f7988868640 1 -- 172.21.15.204:0/108378880 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7984131c40 con 0x7f798412d450 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.766+0000 7f79727fc640 1 -- 172.21.15.204:0/108378880 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f797c014410 con 0x7f798412d450 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.766+0000 7f79727fc640 1 --2- 172.21.15.204:0/108378880 >> v2:172.21.15.204:6812/28210 conn(0x7f79640397b0 0x7f796403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.766+0000 7f79727fc640 1 -- 172.21.15.204:0/108378880 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3502+0+0 (secure 0 0 0) 0x7f797c04a8e0 con 0x7f798412d450 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.766+0000 7f7988868640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.766+0000 7f7988868640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.766+0000 7f7988868640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:15.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.766+0000 7f798259c640 1 --2- 172.21.15.204:0/108378880 >> v2:172.21.15.204:6812/28210 conn(0x7f79640397b0 0x7f796403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:15.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.766+0000 7f7988868640 1 -- 172.21.15.204:0/108378880 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f798411ab30 con 0x7f798412d450 2024-09-15T01:33:15.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.766+0000 7f798259c640 10 monclient: get_auth_request con 0x7f79640397b0 auth_method 0 2024-09-15T01:33:15.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.766+0000 7f798259c640 1 --2- 172.21.15.204:0/108378880 >> v2:172.21.15.204:6812/28210 conn(0x7f79640397b0 0x7f796403bc60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7974004fd0 tx=0x7f7974004980 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:15.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.770+0000 7f79727fc640 1 -- 172.21.15.204:0/108378880 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f798411ab30 con 0x7f798412d450 2024-09-15T01:33:15.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.770+0000 7f79727fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:15.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.770+0000 7f79727fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:15.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.925+0000 7f7988868640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-09-15T01:33:15.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.925+0000 7f7988868640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-09-15T01:33:15.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.925+0000 7f7988868640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:15.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:15.925+0000 7f7988868640 1 -- 172.21.15.204:0/108378880 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"} v 0) v1 -- 0x7f798411ab30 con 0x7f798412d450 2024-09-15T01:33:16.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.342+0000 7f79727fc640 1 -- 172.21.15.204:0/108378880 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}]=0 pool 'foo' created v31) v1 ==== 154+0+0 (secure 0 0 0) 0x7f798411ab30 con 0x7f798412d450 2024-09-15T01:33:16.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.342+0000 7f79727fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-09-15T01:33:16.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.342+0000 7f79727fc640 10 monclient: _finish_command 2 = system:0 pool 'foo' created 2024-09-15T01:33:16.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.449+0000 7f7988868640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-09-15T01:33:16.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.449+0000 7f7988868640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-09-15T01:33:16.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.449+0000 7f7988868640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:16.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.449+0000 7f7988868640 1 -- 172.21.15.204:0/108378880 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"} v 0) v1 -- 0x7f798411ab30 con 0x7f798412d450 2024-09-15T01:33:16.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.450+0000 7f79727fc640 1 -- 172.21.15.204:0/108378880 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}]=0 pool 'foo' already exists v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7f798411ab30 con 0x7f798412d450 2024-09-15T01:33:16.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.450+0000 7f79727fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-09-15T01:33:16.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.450+0000 7f79727fc640 10 monclient: _finish_command 3 = system:0 pool 'foo' already exists 2024-09-15T01:33:16.452 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'foo' already exists 2024-09-15T01:33:16.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 1 -- 172.21.15.204:0/108378880 >> v2:172.21.15.204:6812/28210 conn(0x7f79640397b0 msgr2=0x7f796403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:16.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 1 --2- 172.21.15.204:0/108378880 >> v2:172.21.15.204:6812/28210 conn(0x7f79640397b0 0x7f796403bc60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7974004fd0 tx=0x7f7974004980 comp rx=0 tx=0).stop 2024-09-15T01:33:16.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 10 monclient: shutdown 2024-09-15T01:33:16.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:16.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 1 -- 172.21.15.204:0/108378880 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798412d450 msgr2=0x7f7984119310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:16.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 1 --2- 172.21.15.204:0/108378880 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798412d450 0x7f7984119310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f797c00b8e0 tx=0x7f797c009400 comp rx=0 tx=0).stop 2024-09-15T01:33:16.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 1 -- 172.21.15.204:0/108378880 shutdown_connections 2024-09-15T01:33:16.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 1 --2- 172.21.15.204:0/108378880 >> v2:172.21.15.204:6812/28210 conn(0x7f79640397b0 0x7f796403bc60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:16.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 1 --2- 172.21.15.204:0/108378880 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798412d450 0x7f7984119310 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:16.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 1 --2- 172.21.15.204:0/108378880 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7984067500 0x7f798412cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:16.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 1 --2- 172.21.15.204:0/108378880 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7984067130 0x7f798412c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:16.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 1 -- 172.21.15.204:0/108378880 >> 172.21.15.204:0/108378880 conn(0x7f79840a2cb0 msgr2=0x7f79840a1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:16.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 1 -- 172.21.15.204:0/108378880 shutdown_connections 2024-09-15T01:33:16.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.452+0000 7f7988868640 1 -- 172.21.15.204:0/108378880 wait complete. 2024-09-15T01:33:16.464 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool create fooo 123 123 erasure default 2024-09-15T01:33:16.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.540+0000 7ff02008b640 10 monclient: get_monmap_and_config 2024-09-15T01:33:16.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.540+0000 7ff02008b640 10 monclient: build_initial_monmap 2024-09-15T01:33:16.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.540+0000 7ff02008b640 10 monclient: monmap: 2024-09-15T01:33:16.542 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:16.542 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:16.542 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:16.541080+0000 2024-09-15T01:33:16.542 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:16.541080+0000 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:16.543 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:16.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 1 Processor -- start 2024-09-15T01:33:16.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 1 -- start start 2024-09-15T01:33:16.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 10 monclient: init 2024-09-15T01:33:16.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:16.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 10 monclient: _add_conns ranks=[9,2,4,8,7,10,6,0,3,1,5] 2024-09-15T01:33:16.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff0180a2e30 0x7ff018132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:16.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 10 monclient(hunting): picked mon.noname-a con 0x7ff0180a2e30 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:16.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff01809fb80 0x7ff01809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:16.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 10 monclient(hunting): picked mon.noname-b con 0x7ff01809fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:16.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff0180a0a40 0x7ff0180a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:16.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 10 monclient(hunting): picked mon.noname-c con 0x7ff0180a0a40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:16.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:16.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01809e6c0 con 0x7ff0180a2e30 2024-09-15T01:33:16.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180a04b0 con 0x7ff01809fb80 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180a3200 con 0x7ff0180a0a40 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.541+0000 7ff02008b640 10 monclient(hunting): authenticate will time out at 812.183828s 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01e628640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff0180a0a40 0x7ff0180a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01de27640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff0180a2e30 0x7ff018132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff01809fb80 0x7ff01809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01de27640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff0180a2e30 0x7ff018132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:40826/0 (socket says 172.21.15.204:40826) 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01e628640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff0180a0a40 0x7ff0180a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:42346/0 (socket says 172.21.15.204:42346) 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01de27640 1 -- 172.21.15.204:0/3269257416 learned_addr learned my addr 172.21.15.204:0/3269257416 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 10 monclient(hunting): get_auth_request con 0x7ff01809fb80 auth_method 0 2024-09-15T01:33:16.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01de27640 10 monclient(hunting): get_auth_request con 0x7ff0180a2e30 auth_method 0 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01de27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01de27640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01de27640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01e628640 10 monclient(hunting): get_auth_request con 0x7ff0180a0a40 auth_method 0 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01e628640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01e628640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01e628640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:16.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01e628640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01e628640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01e628640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01de27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01de27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01de27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 10 monclient(hunting): handle_auth_done global_id 4637 payload 274 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 1 -- 172.21.15.204:0/3269257416 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff0180a0a40 msgr2=0x7ff0180a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 1 --2- 172.21.15.204:0/3269257416 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff0180a0a40 0x7ff0180a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 1 -- 172.21.15.204:0/3269257416 >> v2:172.21.15.178:3300/0 conn(0x7ff0180a2e30 msgr2=0x7ff018132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 1 --2- 172.21.15.204:0/3269257416 >> v2:172.21.15.178:3300/0 conn(0x7ff0180a2e30 0x7ff018132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 10 monclient: _finish_hunting 0 2024-09-15T01:33:16.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 1 monclient: found mon.noname-b 2024-09-15T01:33:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:33:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01d626640 1 -- 172.21.15.204:0/3269257416 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01804b720 con 0x7ff01809fb80 2024-09-15T01:33:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01de27640 1 --2- 172.21.15.204:0/3269257416 >> v2:172.21.15.178:3300/0 conn(0x7ff0180a2e30 0x7ff018132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.542+0000 7ff01e628640 1 --2- 172.21.15.204:0/3269257416 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff0180a0a40 0x7ff0180a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01d626640 1 --2- 172.21.15.204:0/3269257416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff01809fb80 0x7ff01809ff70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff008009850 tx=0x7ff0080090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2003512af3b2655 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01ce25640 1 -- 172.21.15.204:0/3269257416 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff00802abc0 con 0x7ff01809fb80 2024-09-15T01:33:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01ce25640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01ce25640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:33:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01ce25640 10 monclient: dump: 2024-09-15T01:33:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:16.557 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01ce25640 10 monclient: _finish_auth 0 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01ce25640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01ce25640 1 -- 172.21.15.204:0/3269257416 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00802ad40 con 0x7ff01809fb80 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01ce25640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01ce25640 1 -- 172.21.15.204:0/3269257416 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff008031ca0 con 0x7ff01809fb80 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01ce25640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01ce25640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff01ce25640 10 monclient: dump: 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:16.558 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff02008b640 5 monclient: authenticate success, global_id 4637 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff02008b640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:16.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff02008b640 4 set_mon_vals no callback set 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff02008b640 10 monclient: shutdown 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff02008b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff02008b640 1 -- 172.21.15.204:0/3269257416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff01809fb80 msgr2=0x7ff01809ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff02008b640 1 --2- 172.21.15.204:0/3269257416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff01809fb80 0x7ff01809ff70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff008009850 tx=0x7ff0080090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff02008b640 1 -- 172.21.15.204:0/3269257416 shutdown_connections 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff02008b640 1 --2- 172.21.15.204:0/3269257416 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff0180a0a40 0x7ff0180a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff02008b640 1 --2- 172.21.15.204:0/3269257416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff01809fb80 0x7ff01809ff70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff02008b640 1 --2- 172.21.15.204:0/3269257416 >> v2:172.21.15.178:3300/0 conn(0x7ff0180a2e30 0x7ff018132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.543+0000 7ff02008b640 1 -- 172.21.15.204:0/3269257416 >> 172.21.15.204:0/3269257416 conn(0x7ff01806e860 msgr2=0x7ff01806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.544+0000 7ff02008b640 1 -- 172.21.15.204:0/3269257416 shutdown_connections 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.544+0000 7ff02008b640 1 -- 172.21.15.204:0/3269257416 wait complete. 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.544+0000 7ff02008b640 10 monclient: build_initial_monmap 2024-09-15T01:33:16.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.544+0000 7ff02008b640 10 monclient: monmap: 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:16.561 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.544+0000 7ff02008b640 1 Processor -- start 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.544+0000 7ff02008b640 1 -- start start 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.544+0000 7ff02008b640 10 monclient: init 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 10 monclient: _add_conns ranks=[5,2,1,3,7,0,4,6,8] 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff01809fb80 0x7ff01809b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 10 monclient(hunting): picked mon.noname-h con 0x7ff01809fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0180a0a40 0x7ff01812e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 10 monclient(hunting): picked mon.noname-b con 0x7ff0180a0a40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0180a2e30 0x7ff01811c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:16.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 10 monclient(hunting): picked mon.noname-f con 0x7ff0180a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff018142a00 con 0x7ff0180a2e30 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01809d650 con 0x7ff0180a0a40 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff01811efb0 con 0x7ff01809fb80 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff02008b640 10 monclient(hunting): authenticate will time out at 812.187280s 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff01809fb80 0x7ff01809b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff01809fb80 0x7ff01809b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:50736/0 (socket says 172.21.15.204:50736) 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01e628640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0180a2e30 0x7ff01811c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 1 -- 172.21.15.204:0/3964324395 learned_addr learned my addr 172.21.15.204:0/3964324395 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 10 monclient(hunting): get_auth_request con 0x7ff01809fb80 auth_method 0 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:16.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01e628640 10 monclient(hunting): get_auth_request con 0x7ff0180a2e30 auth_method 0 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01e628640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01e628640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01e628640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01d626640 1 --2- 172.21.15.204:0/3964324395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0180a0a40 0x7ff01812e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01d626640 10 monclient(hunting): get_auth_request con 0x7ff0180a0a40 auth_method 0 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01d626640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01d626640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01d626640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:16.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01e628640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01e628640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01e628640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01d626640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01d626640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01d626640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 10 monclient(hunting): handle_auth_done global_id 4676 payload 824 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 1 -- 172.21.15.204:0/3964324395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0180a0a40 msgr2=0x7ff01812e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 1 --2- 172.21.15.204:0/3964324395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0180a0a40 0x7ff01812e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 1 -- 172.21.15.204:0/3964324395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0180a2e30 msgr2=0x7ff01811c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 1 --2- 172.21.15.204:0/3964324395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0180a2e30 0x7ff01811c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 10 monclient: _finish_hunting 0 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 1 monclient: found mon.noname-h 2024-09-15T01:33:16.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 1 -- 172.21.15.204:0/3964324395 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff010009a20 con 0x7ff01809fb80 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01e628640 1 --2- 172.21.15.204:0/3964324395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0180a2e30 0x7ff01811c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff01de27640 1 --2- 172.21.15.204:0/3964324395 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff01809fb80 0x7ff01809b7b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff01400ea60 tx=0x7ff0140091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff00dffb640 1 -- 172.21.15.204:0/3964324395 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff014009690 con 0x7ff01809fb80 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.545+0000 7ff00dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff00dffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff00dffb640 10 monclient: dump: 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:16.566 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff00dffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff00dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff02008b640 5 monclient: authenticate success, global_id 4676 2024-09-15T01:33:16.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff00dffb640 1 -- 172.21.15.204:0/3964324395 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01400cdc0 con 0x7ff01809fb80 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff00dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff00dffb640 1 -- 172.21.15.204:0/3964324395 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff0140138f0 con 0x7ff01809fb80 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff02008b640 10 monclient: _renew_subs 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff02008b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff02008b640 1 -- 172.21.15.204:0/3964324395 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff014009450 con 0x7ff01809fb80 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff00dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff00dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff00ffff640 4 set_mon_vals no callback set 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff00dffb640 10 monclient: dump: 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:16.568 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:16.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff02008b640 10 monclient: _renew_subs 2024-09-15T01:33:16.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff02008b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:16.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff02008b640 1 -- 172.21.15.204:0/3964324395 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff008009390 con 0x7ff01809fb80 2024-09-15T01:33:16.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.546+0000 7ff00dffb640 1 -- 172.21.15.204:0/3964324395 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff014013470 con 0x7ff01809fb80 2024-09-15T01:33:16.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.547+0000 7ff00dffb640 1 --2- 172.21.15.204:0/3964324395 >> v2:172.21.15.204:6812/28210 conn(0x7feffc0397b0 0x7feffc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:16.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.547+0000 7ff02008b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:16.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.547+0000 7ff02008b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:16.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.547+0000 7ff02008b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:16.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.547+0000 7ff00dffb640 1 -- 172.21.15.204:0/3964324395 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3844+0+0 (secure 0 0 0) 0x7ff01404b370 con 0x7ff01809fb80 2024-09-15T01:33:16.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.547+0000 7ff02008b640 1 -- 172.21.15.204:0/3964324395 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff018070cc0 con 0x7ff01809fb80 2024-09-15T01:33:16.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.547+0000 7ff01d626640 1 --2- 172.21.15.204:0/3964324395 >> v2:172.21.15.204:6812/28210 conn(0x7feffc0397b0 0x7feffc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:16.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.547+0000 7ff01d626640 10 monclient: get_auth_request con 0x7feffc0397b0 auth_method 0 2024-09-15T01:33:16.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.547+0000 7ff01d626640 1 --2- 172.21.15.204:0/3964324395 >> v2:172.21.15.204:6812/28210 conn(0x7feffc0397b0 0x7feffc03bc60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff008034000 tx=0x7ff008032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:16.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.551+0000 7ff00dffb640 1 -- 172.21.15.204:0/3964324395 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff01401bbb0 con 0x7ff01809fb80 2024-09-15T01:33:16.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.551+0000 7ff00dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:16.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.551+0000 7ff00dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:16.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.708+0000 7ff02008b640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-09-15T01:33:16.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.708+0000 7ff02008b640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-09-15T01:33:16.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.708+0000 7ff02008b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:16.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:16.708+0000 7ff02008b640 1 -- 172.21.15.204:0/3964324395 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"} v 0) v1 -- 0x7ff0180a1f10 con 0x7ff01809fb80 2024-09-15T01:33:18.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.348+0000 7ff00dffb640 1 -- 172.21.15.204:0/3964324395 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}]=0 pool 'fooo' created v33) v1 ==== 188+0+0 (secure 0 0 0) 0x7ff01401bbb0 con 0x7ff01809fb80 2024-09-15T01:33:18.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.348+0000 7ff00dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-09-15T01:33:18.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.348+0000 7ff00dffb640 10 monclient: _finish_command 2 = system:0 pool 'fooo' created 2024-09-15T01:33:18.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.464+0000 7ff02008b640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-09-15T01:33:18.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.464+0000 7ff02008b640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-09-15T01:33:18.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.464+0000 7ff02008b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:18.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.464+0000 7ff02008b640 1 -- 172.21.15.204:0/3964324395 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"} v 0) v1 -- 0x7ff0180a41f0 con 0x7ff01809fb80 2024-09-15T01:33:18.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.464+0000 7ff00dffb640 1 -- 172.21.15.204:0/3964324395 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}]=0 pool 'fooo' already exists v33) v1 ==== 195+0+0 (secure 0 0 0) 0x7ff01401bbb0 con 0x7ff01809fb80 2024-09-15T01:33:18.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.464+0000 7ff00dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-09-15T01:33:18.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.464+0000 7ff00dffb640 10 monclient: _finish_command 3 = system:0 pool 'fooo' already exists 2024-09-15T01:33:18.466 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'fooo' already exists 2024-09-15T01:33:18.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 1 -- 172.21.15.204:0/3964324395 >> v2:172.21.15.204:6812/28210 conn(0x7feffc0397b0 msgr2=0x7feffc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:18.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 1 --2- 172.21.15.204:0/3964324395 >> v2:172.21.15.204:6812/28210 conn(0x7feffc0397b0 0x7feffc03bc60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff008034000 tx=0x7ff008032000 comp rx=0 tx=0).stop 2024-09-15T01:33:18.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 10 monclient: shutdown 2024-09-15T01:33:18.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:18.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 1 -- 172.21.15.204:0/3964324395 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff01809fb80 msgr2=0x7ff01809b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:18.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 1 --2- 172.21.15.204:0/3964324395 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff01809fb80 0x7ff01809b7b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff01400ea60 tx=0x7ff0140091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:18.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 1 -- 172.21.15.204:0/3964324395 shutdown_connections 2024-09-15T01:33:18.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 1 --2- 172.21.15.204:0/3964324395 >> v2:172.21.15.204:6812/28210 conn(0x7feffc0397b0 0x7feffc03bc60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:18.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 1 --2- 172.21.15.204:0/3964324395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0180a2e30 0x7ff01811c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:18.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 1 --2- 172.21.15.204:0/3964324395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0180a0a40 0x7ff01812e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:18.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 1 --2- 172.21.15.204:0/3964324395 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff01809fb80 0x7ff01809b7b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:18.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 1 -- 172.21.15.204:0/3964324395 >> 172.21.15.204:0/3964324395 conn(0x7ff01806e860 msgr2=0x7ff01809ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:18.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 1 -- 172.21.15.204:0/3964324395 shutdown_connections 2024-09-15T01:33:18.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.466+0000 7ff02008b640 1 -- 172.21.15.204:0/3964324395 wait complete. 2024-09-15T01:33:18.477 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool create foooo 123 2024-09-15T01:33:18.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.550+0000 7fe874899640 10 monclient: get_monmap_and_config 2024-09-15T01:33:18.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.550+0000 7fe874899640 10 monclient: build_initial_monmap 2024-09-15T01:33:18.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.550+0000 7fe874899640 10 monclient: monmap: 2024-09-15T01:33:18.552 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:18.552 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:18.552 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:18.551020+0000 2024-09-15T01:33:18.552 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:18.551020+0000 2024-09-15T01:33:18.552 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:18.552 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:18.552 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:18.553 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:18.553 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:18.553 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:18.553 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:18.553 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:18.553 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:18.553 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:18.553 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:18.553 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:18.553 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:18.553 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:18.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 1 Processor -- start 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 1 -- start start 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 10 monclient: init 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 10 monclient: _add_conns ranks=[10,9,0,6,2,7,4,5,1,3,8] 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe8700a2e30 0x7fe870132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 10 monclient(hunting): picked mon.noname-f con 0x7fe8700a2e30 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe87009fb80 0x7fe87009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 10 monclient(hunting): picked mon.noname-a con 0x7fe87009fb80 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe8700a0a40 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87004b720 con 0x7fe8700a0a40 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87009e6c0 con 0x7fe87009fb80 2024-09-15T01:33:18.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8700a04b0 con 0x7fe8700a2e30 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe874899640 10 monclient(hunting): authenticate will time out at 814.193815s 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe86e59c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe8700a2e30 0x7fe870132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe86e59c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe8700a2e30 0x7fe870132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41580/0 (socket says 172.21.15.204:41580) 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe86e59c640 1 -- 172.21.15.204:0/3934749927 learned_addr learned my addr 172.21.15.204:0/3934749927 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe86e59c640 10 monclient(hunting): get_auth_request con 0x7fe8700a2e30 auth_method 0 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe86e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe86e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.551+0000 7fe86e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:18.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe867fff640 1 --2- 172.21.15.204:0/3934749927 >> v2:172.21.15.178:3300/0 conn(0x7fe87009fb80 0x7fe87009ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe867fff640 10 monclient(hunting): get_auth_request con 0x7fe87009fb80 auth_method 0 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe867fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe867fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe867fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 10 monclient(hunting): handle_auth_done global_id 4762 payload 274 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 1 -- 172.21.15.204:0/3934749927 >> v2:172.21.15.178:3300/0 conn(0x7fe87009fb80 msgr2=0x7fe87009ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 1 --2- 172.21.15.204:0/3934749927 >> v2:172.21.15.178:3300/0 conn(0x7fe87009fb80 0x7fe87009ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 1 -- 172.21.15.204:0/3934749927 >> v1:172.21.15.178:6789/0 conn(0x7fe8700a0a40 legacy=0x7fe8700a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 1 monclient: found mon.noname-f 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:18.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 1 -- 172.21.15.204:0/3934749927 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8700a3200 con 0x7fe8700a2e30 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe867fff640 1 --2- 172.21.15.204:0/3934749927 >> v2:172.21.15.178:3300/0 conn(0x7fe87009fb80 0x7fe87009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.552+0000 7fe86e59c640 1 --2- 172.21.15.204:0/3934749927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8700a2e30 0x7fe870132a60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe87006f710 tx=0x7fe8600091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=755ee789fdec4c87 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe86dd9b640 1 -- 172.21.15.204:0/3934749927 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe86002ad40 con 0x7fe8700a2e30 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe86dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe86dd9b640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe86dd9b640 10 monclient: dump: 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:18.565 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe86dd9b640 10 monclient: _finish_auth 0 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe86dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe86dd9b640 1 -- 172.21.15.204:0/3934749927 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe860031820 con 0x7fe8700a2e30 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 5 monclient: authenticate success, global_id 4762 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:18.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe86dd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe86dd9b640 1 -- 172.21.15.204:0/3934749927 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe860031e90 con 0x7fe8700a2e30 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe86dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 4 set_mon_vals no callback set 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe86dd9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe86dd9b640 10 monclient: dump: 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:18.567 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 10 monclient: shutdown 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 1 -- 172.21.15.204:0/3934749927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8700a2e30 msgr2=0x7fe870132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 1 --2- 172.21.15.204:0/3934749927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8700a2e30 0x7fe870132a60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe87006f710 tx=0x7fe8600091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:18.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 1 -- 172.21.15.204:0/3934749927 shutdown_connections 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 1 --2- 172.21.15.204:0/3934749927 >> v2:172.21.15.178:3300/0 conn(0x7fe87009fb80 0x7fe87009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 1 --2- 172.21.15.204:0/3934749927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8700a2e30 0x7fe870132a60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 1 -- 172.21.15.204:0/3934749927 >> 172.21.15.204:0/3934749927 conn(0x7fe87006e860 msgr2=0x7fe87006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 1 -- 172.21.15.204:0/3934749927 shutdown_connections 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.553+0000 7fe874899640 1 -- 172.21.15.204:0/3934749927 wait complete. 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient: build_initial_monmap 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient: monmap: 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:18.569 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 1 Processor -- start 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 1 -- start start 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient: init 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient: _add_conns ranks=[4,6,0,8,2,5,7,3,1] 2024-09-15T01:33:18.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe87009fb80 0x7fe870131830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient(hunting): picked mon.noname-c con 0x7fe87009fb80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe8700a0a40 0x7fe87011e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient(hunting): picked mon.noname-d con 0x7fe8700a0a40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8700a2e30 0x7fe87011e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient(hunting): picked mon.noname-a con 0x7fe8700a2e30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87009ca90 con 0x7fe8700a2e30 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe87009d650 con 0x7fe87009fb80 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe870121c60 con 0x7fe8700a0a40 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe874899640 10 monclient(hunting): authenticate will time out at 814.196893s 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe867fff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe8700a0a40 msgr2=0x7fe87011e450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:18.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.554+0000 7fe867fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe8700a0a40 0x7fe87011e450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe87009fb80 0x7fe870131830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe87009fb80 0x7fe870131830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:37582/0 (socket says 172.21.15.204:37582) 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 1 -- 172.21.15.204:0/77438107 learned_addr learned my addr 172.21.15.204:0/77438107 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 10 monclient(hunting): get_auth_request con 0x7fe87009fb80 auth_method 0 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86ed9d640 1 --2- 172.21.15.204:0/77438107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8700a2e30 0x7fe87011e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86ed9d640 10 monclient(hunting): get_auth_request con 0x7fe8700a2e30 auth_method 0 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:18.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 10 monclient(hunting): handle_auth_done global_id 4621 payload 824 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 1 -- 172.21.15.204:0/77438107 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe8700a0a40 msgr2=0x7fe87011e450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 1 --2- 172.21.15.204:0/77438107 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe8700a0a40 0x7fe87011e450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 1 -- 172.21.15.204:0/77438107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8700a2e30 msgr2=0x7fe87011e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 1 --2- 172.21.15.204:0/77438107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8700a2e30 0x7fe87011e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 1 monclient: found mon.noname-c 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 1 -- 172.21.15.204:0/77438107 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe860009450 con 0x7fe87009fb80 2024-09-15T01:33:18.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86ed9d640 1 --2- 172.21.15.204:0/77438107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8700a2e30 0x7fe87011e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86e59c640 1 --2- 172.21.15.204:0/77438107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe87009fb80 0x7fe870131830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe860000c00 tx=0x7fe860032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe8667fc640 1 -- 172.21.15.204:0/77438107 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe860033c60 con 0x7fe87009fb80 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe8667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe8667fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe8667fc640 10 monclient: dump: 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:18.574 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe8667fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe8667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe874899640 5 monclient: authenticate success, global_id 4621 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe8667fc640 1 -- 172.21.15.204:0/77438107 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe860033de0 con 0x7fe87009fb80 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe874899640 10 monclient: _renew_subs 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe874899640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe874899640 1 -- 172.21.15.204:0/77438107 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe870121a00 con 0x7fe87009fb80 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe8667fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:18.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe8667fc640 1 -- 172.21.15.204:0/77438107 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe86003b3f0 con 0x7fe87009fb80 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe8667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe8667fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe86d59a640 4 set_mon_vals no callback set 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe8667fc640 10 monclient: dump: 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:18.576 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe874899640 10 monclient: _renew_subs 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe874899640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.555+0000 7fe874899640 1 -- 172.21.15.204:0/77438107 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe870142910 con 0x7fe87009fb80 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.556+0000 7fe8667fc640 1 -- 172.21.15.204:0/77438107 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe860039040 con 0x7fe87009fb80 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.556+0000 7fe8667fc640 1 --2- 172.21.15.204:0/77438107 >> v2:172.21.15.204:6812/28210 conn(0x7fe8500397b0 0x7fe85003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.556+0000 7fe8667fc640 1 -- 172.21.15.204:0/77438107 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4286+0+0 (secure 0 0 0) 0x7fe86006ad20 con 0x7fe87009fb80 2024-09-15T01:33:18.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.556+0000 7fe874899640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:18.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.556+0000 7fe874899640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:18.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.556+0000 7fe874899640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:18.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.556+0000 7fe874899640 1 -- 172.21.15.204:0/77438107 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8701219e0 con 0x7fe87009fb80 2024-09-15T01:33:18.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.556+0000 7fe867fff640 1 --2- 172.21.15.204:0/77438107 >> v2:172.21.15.204:6812/28210 conn(0x7fe8500397b0 0x7fe85003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:18.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.556+0000 7fe867fff640 10 monclient: get_auth_request con 0x7fe8500397b0 auth_method 0 2024-09-15T01:33:18.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.557+0000 7fe867fff640 1 --2- 172.21.15.204:0/77438107 >> v2:172.21.15.204:6812/28210 conn(0x7fe8500397b0 0x7fe85003bc60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe858006e70 tx=0x7fe8580090c0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:18.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.561+0000 7fe8667fc640 1 -- 172.21.15.204:0/77438107 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe860042080 con 0x7fe87009fb80 2024-09-15T01:33:18.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.561+0000 7fe8667fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:18.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.561+0000 7fe8667fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.717+0000 7fe874899640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-09-15T01:33:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.717+0000 7fe874899640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-09-15T01:33:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.717+0000 7fe874899640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:18.717+0000 7fe874899640 1 -- 172.21.15.204:0/77438107 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "foooo", "pg_num": 123} v 0) v1 -- 0x7fe870070cc0 con 0x7fe87009fb80 2024-09-15T01:33:19.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.350+0000 7fe8667fc640 1 -- 172.21.15.204:0/77438107 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}]=0 pool 'foooo' created v34) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe860037020 con 0x7fe87009fb80 2024-09-15T01:33:19.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.350+0000 7fe8667fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-09-15T01:33:19.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.350+0000 7fe8667fc640 10 monclient: _finish_command 2 = system:0 pool 'foooo' created 2024-09-15T01:33:19.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.475+0000 7fe874899640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-09-15T01:33:19.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.475+0000 7fe874899640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-09-15T01:33:19.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.475+0000 7fe874899640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:19.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.475+0000 7fe874899640 1 -- 172.21.15.204:0/77438107 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "foooo", "pg_num": 123} v 0) v1 -- 0x7fe8700a1f10 con 0x7fe87009fb80 2024-09-15T01:33:19.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.476+0000 7fe8667fc640 1 -- 172.21.15.204:0/77438107 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}]=0 pool 'foooo' already exists v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe8600337e0 con 0x7fe87009fb80 2024-09-15T01:33:19.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.476+0000 7fe8667fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-09-15T01:33:19.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.476+0000 7fe8667fc640 10 monclient: _finish_command 3 = system:0 pool 'foooo' already exists 2024-09-15T01:33:19.477 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'foooo' already exists 2024-09-15T01:33:19.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 1 -- 172.21.15.204:0/77438107 >> v2:172.21.15.204:6812/28210 conn(0x7fe8500397b0 msgr2=0x7fe85003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:19.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 1 --2- 172.21.15.204:0/77438107 >> v2:172.21.15.204:6812/28210 conn(0x7fe8500397b0 0x7fe85003bc60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe858006e70 tx=0x7fe8580090c0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 10 monclient: shutdown 2024-09-15T01:33:19.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:19.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 1 -- 172.21.15.204:0/77438107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe87009fb80 msgr2=0x7fe870131830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:19.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 1 --2- 172.21.15.204:0/77438107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe87009fb80 0x7fe870131830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe860000c00 tx=0x7fe860032000 comp rx=0 tx=0).stop 2024-09-15T01:33:19.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 1 -- 172.21.15.204:0/77438107 shutdown_connections 2024-09-15T01:33:19.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 1 --2- 172.21.15.204:0/77438107 >> v2:172.21.15.204:6812/28210 conn(0x7fe8500397b0 0x7fe85003bc60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 1 --2- 172.21.15.204:0/77438107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8700a2e30 0x7fe87011e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 1 --2- 172.21.15.204:0/77438107 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe8700a0a40 0x7fe87011e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 1 --2- 172.21.15.204:0/77438107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe87009fb80 0x7fe870131830 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 1 -- 172.21.15.204:0/77438107 >> 172.21.15.204:0/77438107 conn(0x7fe87006e860 msgr2=0x7fe87009bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:19.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 1 -- 172.21.15.204:0/77438107 shutdown_connections 2024-09-15T01:33:19.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.478+0000 7fe874899640 1 -- 172.21.15.204:0/77438107 wait complete. 2024-09-15T01:33:19.489 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool create foo 123 2024-09-15T01:33:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.571+0000 7f57bb6a8640 10 monclient: get_monmap_and_config 2024-09-15T01:33:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.571+0000 7f57bb6a8640 10 monclient: build_initial_monmap 2024-09-15T01:33:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.571+0000 7f57bb6a8640 10 monclient: monmap: 2024-09-15T01:33:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:19.571742+0000 2024-09-15T01:33:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:19.571742+0000 2024-09-15T01:33:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.571+0000 7f57bb6a8640 1 Processor -- start 2024-09-15T01:33:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 1 -- start start 2024-09-15T01:33:19.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 10 monclient: init 2024-09-15T01:33:19.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:19.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 10 monclient: _add_conns ranks=[1,6,5,9,8,3,2,0,4,7,10] 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f57b40a35e0 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57b4099a10 0x7f57b4134640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 10 monclient(hunting): picked mon.noname-d con 0x7f57b4099a10 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f57b4134b80 0x7f57b4136f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 10 monclient(hunting): picked mon.noname-h con 0x7f57b4134b80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b404b720 con 0x7f57b40a35e0 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57b409d340 con 0x7f57b4134b80 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f57b409cc70 con 0x7f57b4099a10 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57bb6a8640 10 monclient(hunting): authenticate will time out at 815.214561s 2024-09-15T01:33:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9444640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f57b40a35e0 0x7f57b409c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:33822/0 (socket says 172.21.15.204:33822) 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9444640 1 -- 172.21.15.204:0/1466884049 learned_addr learned my addr 172.21.15.204:0/1466884049 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9c45640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f57b4134b80 0x7f57b4136f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b8c43640 1 -- 172.21.15.204:0/1466884049 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57b4099a10 msgr2=0x7f57b4134640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b8c43640 1 --2- 172.21.15.204:0/1466884049 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57b4099a10 0x7f57b4134640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9c45640 10 monclient(hunting): get_auth_request con 0x7f57b4134b80 auth_method 0 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9c45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9c45640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9c45640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9c45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9c45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9c45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9c45640 10 monclient(hunting): handle_auth_done global_id 4685 payload 274 2024-09-15T01:33:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57abfff640 1 -- 172.21.15.204:0/1466884049 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492181304 0 0) 0x7f57b404b720 con 0x7f57b40a35e0 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9c45640 1 -- 172.21.15.204:0/1466884049 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57b4099a10 msgr2=0x7f57b4134640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9c45640 1 --2- 172.21.15.204:0/1466884049 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57b4099a10 0x7f57b4134640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.572+0000 7f57b9c45640 1 -- 172.21.15.204:0/1466884049 >> v1:172.21.15.204:6789/0 conn(0x7f57b40a35e0 legacy=0x7f57b409c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.573+0000 7f57b9c45640 10 monclient: _finish_hunting 0 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.573+0000 7f57b9c45640 1 monclient: found mon.noname-h 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.573+0000 7f57b9c45640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.573+0000 7f57b9c45640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.573+0000 7f57b9c45640 1 -- 172.21.15.204:0/1466884049 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b40a39b0 con 0x7f57b4134b80 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.573+0000 7f57abfff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.573+0000 7f57b9c45640 1 --2- 172.21.15.204:0/1466884049 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f57b4134b80 0x7f57b4136f60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009e90 tx=0x7f57ac009780 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d99bbcf518b96d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.573+0000 7f57abfff640 1 -- 172.21.15.204:0/1466884049 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57ac0079d0 con 0x7f57b4134b80 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.573+0000 7f57abfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57abfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57abfff640 10 monclient: dump: 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57abfff640 10 monclient: _finish_auth 0 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 5 monclient: authenticate success, global_id 4685 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57abfff640 1 -- 172.21.15.204:0/1466884049 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac02d050 con 0x7f57b4134b80 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57abfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57abfff640 1 -- 172.21.15.204:0/1466884049 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57ac031400 con 0x7f57b4134b80 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57abfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 4 set_mon_vals no callback set 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57abfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57abfff640 10 monclient: dump: 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 10 monclient: shutdown 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 1 -- 172.21.15.204:0/1466884049 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f57b4134b80 msgr2=0x7f57b4136f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 1 --2- 172.21.15.204:0/1466884049 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f57b4134b80 0x7f57b4136f60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009e90 tx=0x7f57ac009780 comp rx=0 tx=0).stop 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 1 -- 172.21.15.204:0/1466884049 shutdown_connections 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 1 --2- 172.21.15.204:0/1466884049 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f57b4134b80 0x7f57b4136f60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 1 --2- 172.21.15.204:0/1466884049 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57b4099a10 0x7f57b4134640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 1 -- 172.21.15.204:0/1466884049 >> 172.21.15.204:0/1466884049 conn(0x7f57b40687e0 msgr2=0x7f57b40a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 1 -- 172.21.15.204:0/1466884049 shutdown_connections 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.574+0000 7f57bb6a8640 1 -- 172.21.15.204:0/1466884049 wait complete. 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.575+0000 7f57bb6a8640 10 monclient: build_initial_monmap 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.575+0000 7f57bb6a8640 10 monclient: monmap: 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.575+0000 7f57bb6a8640 1 Processor -- start 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.575+0000 7f57bb6a8640 1 -- start start 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.575+0000 7f57bb6a8640 10 monclient: init 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 10 monclient: _add_conns ranks=[2,3,1,4,8,5,7,6,0] 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b4099a10 0x7f57b40a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 10 monclient(hunting): picked mon.noname-b con 0x7f57b4099a10 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57b40a35e0 0x7f57b40a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 10 monclient(hunting): picked mon.noname-g con 0x7f57b40a35e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4134b80 0x7f57b40a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 10 monclient(hunting): picked mon.noname-f con 0x7f57b4134b80 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b41236b0 con 0x7f57b4134b80 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b4122420 con 0x7f57b4099a10 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b406ea80 con 0x7f57b40a35e0 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57bb6a8640 10 monclient(hunting): authenticate will time out at 815.218518s 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b8c43640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57b40a35e0 msgr2=0x7f57b40a15d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b8c43640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57b40a35e0 0x7f57b40a15d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9c45640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4134b80 0x7f57b40a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9c45640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4134b80 0x7f57b40a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41596/0 (socket says 172.21.15.204:41596) 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9c45640 1 -- 172.21.15.204:0/3627901533 learned_addr learned my addr 172.21.15.204:0/3627901533 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9c45640 10 monclient(hunting): get_auth_request con 0x7f57b4134b80 auth_method 0 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9c45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9c45640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9c45640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9c45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9c45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9444640 1 --2- 172.21.15.204:0/3627901533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b4099a10 0x7f57b40a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9c45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9444640 10 monclient(hunting): get_auth_request con 0x7f57b4099a10 auth_method 0 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9444640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9444640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.576+0000 7f57b9444640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 10 monclient(hunting): handle_auth_done global_id 4655 payload 824 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 1 -- 172.21.15.204:0/3627901533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57b40a35e0 msgr2=0x7f57b40a15d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 1 --2- 172.21.15.204:0/3627901533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57b40a35e0 0x7f57b40a15d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 1 -- 172.21.15.204:0/3627901533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4134b80 msgr2=0x7f57b40a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 1 --2- 172.21.15.204:0/3627901533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4134b80 0x7f57b40a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 10 monclient: _finish_hunting 0 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 1 monclient: found mon.noname-b 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 1 -- 172.21.15.204:0/3627901533 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac009a20 con 0x7f57b4099a10 2024-09-15T01:33:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9c45640 1 --2- 172.21.15.204:0/3627901533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4134b80 0x7f57b40a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57b9444640 1 --2- 172.21.15.204:0/3627901533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b4099a10 0x7f57b40a1090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f57b0000c00 tx=0x7f57b0007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57a97fa640 1 -- 172.21.15.204:0/3627901533 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57b00070d0 con 0x7f57b4099a10 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57a97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57a97fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57a97fa640 10 monclient: dump: 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57a97fa640 10 monclient: _finish_auth 0 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57a97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57a97fa640 1 -- 172.21.15.204:0/3627901533 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b0007250 con 0x7f57b4099a10 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57bb6a8640 5 monclient: authenticate success, global_id 4655 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57bb6a8640 10 monclient: _renew_subs 2024-09-15T01:33:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57bb6a8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57bb6a8640 1 -- 172.21.15.204:0/3627901533 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b4122700 con 0x7f57b4099a10 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57a97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57a97fa640 1 -- 172.21.15.204:0/3627901533 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57b0007440 con 0x7f57b4099a10 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57a97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57a97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57a97fa640 10 monclient: dump: 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57ab7fe640 4 set_mon_vals no callback set 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57bb6a8640 10 monclient: _renew_subs 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57bb6a8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.577+0000 7f57bb6a8640 1 -- 172.21.15.204:0/3627901533 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b4120fd0 con 0x7f57b4099a10 2024-09-15T01:33:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.578+0000 7f57a97fa640 1 -- 172.21.15.204:0/3627901533 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f57b00075a0 con 0x7f57b4099a10 2024-09-15T01:33:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.578+0000 7f57a97fa640 1 --2- 172.21.15.204:0/3627901533 >> v2:172.21.15.204:6812/28210 conn(0x7f57980397e0 0x7f579803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.578+0000 7f57a97fa640 1 -- 172.21.15.204:0/3627901533 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4630+0+0 (secure 0 0 0) 0x7f57b004a410 con 0x7f57b4099a10 2024-09-15T01:33:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.578+0000 7f57b8c43640 1 --2- 172.21.15.204:0/3627901533 >> v2:172.21.15.204:6812/28210 conn(0x7f57980397e0 0x7f579803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.579+0000 7f57b8c43640 10 monclient: get_auth_request con 0x7f57980397e0 auth_method 0 2024-09-15T01:33:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.578+0000 7f57bb6a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.579+0000 7f57bb6a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.579+0000 7f57bb6a8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.579+0000 7f57bb6a8640 1 -- 172.21.15.204:0/3627901533 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b406f070 con 0x7f57b4099a10 2024-09-15T01:33:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.579+0000 7f57b8c43640 1 --2- 172.21.15.204:0/3627901533 >> v2:172.21.15.204:6812/28210 conn(0x7f57980397e0 0x7f579803bc90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f57b40a3c00 tx=0x7f57a4006d00 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:19.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.583+0000 7f57a97fa640 1 -- 172.21.15.204:0/3627901533 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f57b406f070 con 0x7f57b4099a10 2024-09-15T01:33:19.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.583+0000 7f57a97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:19.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.583+0000 7f57a97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:19.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.767+0000 7f57bb6a8640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-09-15T01:33:19.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.767+0000 7f57bb6a8640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-09-15T01:33:19.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.767+0000 7f57bb6a8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:19.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.767+0000 7f57bb6a8640 1 -- 172.21.15.204:0/3627901533 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123} v 0) v1 -- 0x7f57b406f070 con 0x7f57b4099a10 2024-09-15T01:33:19.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.768+0000 7f57a97fa640 1 -- 172.21.15.204:0/3627901533 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}]=0 pool 'foo' already exists v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f57b406f070 con 0x7f57b4099a10 2024-09-15T01:33:19.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.768+0000 7f57a97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-09-15T01:33:19.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.768+0000 7f57a97fa640 10 monclient: _finish_command 2 = system:0 pool 'foo' already exists 2024-09-15T01:33:19.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.858+0000 7f57bb6a8640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-09-15T01:33:19.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.858+0000 7f57bb6a8640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-09-15T01:33:19.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.858+0000 7f57bb6a8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:19.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.858+0000 7f57bb6a8640 1 -- 172.21.15.204:0/3627901533 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123} v 0) v1 -- 0x7f57b406f070 con 0x7f57b4099a10 2024-09-15T01:33:19.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.859+0000 7f57a97fa640 1 -- 172.21.15.204:0/3627901533 <== mon.2 v2:172.21.15.178:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}]=0 pool 'foo' already exists v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f57b406f070 con 0x7f57b4099a10 2024-09-15T01:33:19.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.859+0000 7f57a97fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-09-15T01:33:19.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.859+0000 7f57a97fa640 10 monclient: _finish_command 3 = system:0 pool 'foo' already exists 2024-09-15T01:33:19.861 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'foo' already exists 2024-09-15T01:33:19.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.861+0000 7f57bb6a8640 1 -- 172.21.15.204:0/3627901533 >> v2:172.21.15.204:6812/28210 conn(0x7f57980397e0 msgr2=0x7f579803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:19.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.861+0000 7f57bb6a8640 1 --2- 172.21.15.204:0/3627901533 >> v2:172.21.15.204:6812/28210 conn(0x7f57980397e0 0x7f579803bc90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f57b40a3c00 tx=0x7f57a4006d00 comp rx=0 tx=0).stop 2024-09-15T01:33:19.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.861+0000 7f57bb6a8640 10 monclient: shutdown 2024-09-15T01:33:19.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.861+0000 7f57bb6a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:19.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.861+0000 7f57bb6a8640 1 -- 172.21.15.204:0/3627901533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b4099a10 msgr2=0x7f57b40a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:19.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.861+0000 7f57bb6a8640 1 --2- 172.21.15.204:0/3627901533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b4099a10 0x7f57b40a1090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f57b0000c00 tx=0x7f57b0007c80 comp rx=0 tx=0).stop 2024-09-15T01:33:19.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.862+0000 7f57bb6a8640 1 -- 172.21.15.204:0/3627901533 shutdown_connections 2024-09-15T01:33:19.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.862+0000 7f57bb6a8640 1 --2- 172.21.15.204:0/3627901533 >> v2:172.21.15.204:6812/28210 conn(0x7f57980397e0 0x7f579803bc90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.862+0000 7f57bb6a8640 1 --2- 172.21.15.204:0/3627901533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57b4134b80 0x7f57b40a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.862+0000 7f57bb6a8640 1 --2- 172.21.15.204:0/3627901533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57b40a35e0 0x7f57b40a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.862+0000 7f57bb6a8640 1 --2- 172.21.15.204:0/3627901533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b4099a10 0x7f57b40a1090 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.862+0000 7f57bb6a8640 1 -- 172.21.15.204:0/3627901533 >> 172.21.15.204:0/3627901533 conn(0x7f57b40687e0 msgr2=0x7f57b406ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:19.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.862+0000 7f57bb6a8640 1 -- 172.21.15.204:0/3627901533 shutdown_connections 2024-09-15T01:33:19.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.862+0000 7f57bb6a8640 1 -- 172.21.15.204:0/3627901533 wait complete. 2024-09-15T01:33:19.872 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set foo size 1 --yes-i-really-mean-it 2024-09-15T01:33:19.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.946+0000 7f4c0d709640 10 monclient: get_monmap_and_config 2024-09-15T01:33:19.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.946+0000 7f4c0d709640 10 monclient: build_initial_monmap 2024-09-15T01:33:19.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.946+0000 7f4c0d709640 10 monclient: monmap: 2024-09-15T01:33:19.948 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:19.948 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:19.948 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:19.946992+0000 2024-09-15T01:33:19.948 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:19.946992+0000 2024-09-15T01:33:19.948 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:19.948 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:19.948 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:19.948 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:19.948 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:19.949 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:19.949 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:19.949 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:19.949 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:19.949 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:19.949 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:19.949 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:19.949 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:19.949 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:19.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 1 Processor -- start 2024-09-15T01:33:19.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 1 -- start start 2024-09-15T01:33:19.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 10 monclient: init 2024-09-15T01:33:19.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:19.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 10 monclient: _add_conns ranks=[1,5,6,7,4,2,3,0,10,8,9] 2024-09-15T01:33:19.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4c080a24a0 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:19.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4c080a00c0 0x7f4c08132b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:19.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 10 monclient(hunting): picked mon.noname-h con 0x7f4c080a00c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:19.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c08137c60 0x7f4c0813a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:19.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 10 monclient(hunting): picked mon.noname-d con 0x7f4c08137c60 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080648a0 con 0x7f4c080a24a0 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080a0490 con 0x7f4c080a00c0 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080a2870 con 0x7f4c08137c60 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c0d709640 10 monclient(hunting): authenticate will time out at 815.589799s 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4bfffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4c080a00c0 0x7f4c08132b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c06ffd640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f4c080a24a0 0x7f4c0809f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:33828/0 (socket says 172.21.15.204:33828) 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c06ffd640 1 -- 172.21.15.204:0/905363220 learned_addr learned my addr 172.21.15.204:0/905363220 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4bfffff640 10 monclient(hunting): get_auth_request con 0x7f4c080a00c0 auth_method 0 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4bfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:19.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4bfffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:19.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4bfffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:19.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c077fe640 1 -- 172.21.15.204:0/905363220 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c08137c60 msgr2=0x7f4c0813a040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:19.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4c077fe640 1 --2- 172.21.15.204:0/905363220 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c08137c60 0x7f4c0813a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:19.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.947+0000 7f4bfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:19.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4bfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:19.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4bfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:19.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4bfffff640 10 monclient(hunting): handle_auth_done global_id 4694 payload 274 2024-09-15T01:33:19.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4bfffff640 1 -- 172.21.15.204:0/905363220 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c08137c60 msgr2=0x7f4c0813a040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:19.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4bfffff640 1 --2- 172.21.15.204:0/905363220 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c08137c60 0x7f4c0813a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4bfffff640 1 -- 172.21.15.204:0/905363220 >> v1:172.21.15.204:6789/0 conn(0x7f4c080a24a0 legacy=0x7f4c0809f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:19.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4bfffff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:19.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4bfffff640 1 monclient: found mon.noname-h 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4c067fc640 1 -- 172.21.15.204:0/905363220 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768334697 0 0) 0x7f4c080648a0 con 0x7f4c080a24a0 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4bfffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4bfffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4bfffff640 1 -- 172.21.15.204:0/905363220 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080a1bd0 con 0x7f4c080a00c0 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4c067fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.948+0000 7f4bfffff640 1 --2- 172.21.15.204:0/905363220 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4c080a00c0 0x7f4c08132b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009880 tx=0x7f4bf00090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=f3332d647d095826 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c067fc640 1 -- 172.21.15.204:0/905363220 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4bf002ac80 con 0x7f4c080a00c0 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c067fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c067fc640 10 monclient: dump: 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.962 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c067fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:19.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c067fc640 1 -- 172.21.15.204:0/905363220 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf002ae00 con 0x7f4c080a00c0 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 5 monclient: authenticate success, global_id 4694 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c067fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c067fc640 1 -- 172.21.15.204:0/905363220 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4bf0031ca0 con 0x7f4c080a00c0 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 4 set_mon_vals no callback set 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c067fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c067fc640 10 monclient: dump: 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:19.964 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:19.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 10 monclient: shutdown 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 1 -- 172.21.15.204:0/905363220 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4c080a00c0 msgr2=0x7f4c08132b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 1 --2- 172.21.15.204:0/905363220 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4c080a00c0 0x7f4c08132b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009880 tx=0x7f4bf00090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 1 -- 172.21.15.204:0/905363220 shutdown_connections 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 1 --2- 172.21.15.204:0/905363220 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c08137c60 0x7f4c0813a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 1 --2- 172.21.15.204:0/905363220 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4c080a00c0 0x7f4c08132b10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 1 -- 172.21.15.204:0/905363220 >> 172.21.15.204:0/905363220 conn(0x7f4c080a5f70 msgr2=0x7f4c080a6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 1 -- 172.21.15.204:0/905363220 shutdown_connections 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.949+0000 7f4c0d709640 1 -- 172.21.15.204:0/905363220 wait complete. 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient: build_initial_monmap 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient: monmap: 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:19.966 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:19.967 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 1 Processor -- start 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 1 -- start start 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient: init 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient: _add_conns ranks=[0,8,2,5,3,6,1,4,7] 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c080a00c0 0x7f4c08133a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient(hunting): picked mon.noname-a con 0x7f4c080a00c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4c080a24a0 0x7f4c08133f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient(hunting): picked mon.noname-e con 0x7f4c080a24a0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c08137c60 0x7f4c08134480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient(hunting): picked mon.noname-b con 0x7f4c08137c60 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:19.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0809d470 con 0x7f4c080a00c0 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c081367d0 con 0x7f4c08137c60 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4c08136ad0 con 0x7f4c080a24a0 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.950+0000 7f4c0d709640 10 monclient(hunting): authenticate will time out at 815.592927s 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c06ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c080a00c0 0x7f4c08133a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4bfffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4c080a24a0 0x7f4c08133f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c08137c60 0x7f4c08134480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c06ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c080a00c0 0x7f4c08133a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:44016/0 (socket says 172.21.15.204:44016) 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c06ffd640 1 -- 172.21.15.204:0/1103682317 learned_addr learned my addr 172.21.15.204:0/1103682317 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4bfffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4c080a24a0 0x7f4c08133f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:49038/0 (socket says 172.21.15.204:49038) 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c06ffd640 10 monclient(hunting): get_auth_request con 0x7f4c080a00c0 auth_method 0 2024-09-15T01:33:19.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c06ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c06ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c06ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4bfffff640 10 monclient(hunting): get_auth_request con 0x7f4c080a24a0 auth_method 0 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4bfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4bfffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4bfffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 10 monclient(hunting): get_auth_request con 0x7f4c08137c60 auth_method 0 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c06ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c06ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:19.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c06ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4bfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4bfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4bfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 10 monclient(hunting): handle_auth_done global_id 4664 payload 824 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 1 -- 172.21.15.204:0/1103682317 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4c080a24a0 msgr2=0x7f4c08133f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 1 --2- 172.21.15.204:0/1103682317 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4c080a24a0 0x7f4c08133f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 1 -- 172.21.15.204:0/1103682317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c080a00c0 msgr2=0x7f4c08133a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 1 --2- 172.21.15.204:0/1103682317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c080a00c0 0x7f4c08133a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 1 monclient: found mon.noname-b 2024-09-15T01:33:19.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 1 -- 172.21.15.204:0/1103682317 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf0009390 con 0x7f4c08137c60 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4bfffff640 1 --2- 172.21.15.204:0/1103682317 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4c080a24a0 0x7f4c08133f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c06ffd640 1 --2- 172.21.15.204:0/1103682317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c080a00c0 0x7f4c08133a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4c077fe640 1 --2- 172.21.15.204:0/1103682317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c08137c60 0x7f4c08134480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000be20 tx=0x7f4c000096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.951+0000 7f4bfeffd640 1 -- 172.21.15.204:0/1103682317 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c000067e0 con 0x7f4c08137c60 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4bfeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4bfeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4bfeffd640 10 monclient: dump: 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.972 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4bfeffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4bfeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4c0d709640 5 monclient: authenticate success, global_id 4664 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4bfeffd640 1 -- 172.21.15.204:0/1103682317 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0000f070 con 0x7f4c08137c60 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4bfeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4bfeffd640 1 -- 172.21.15.204:0/1103682317 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c000069c0 con 0x7f4c08137c60 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4c0d709640 10 monclient: _renew_subs 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4c0d709640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4c0d709640 1 -- 172.21.15.204:0/1103682317 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c08136db0 con 0x7f4c08137c60 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4c05ffb640 4 set_mon_vals no callback set 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4bfeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4bfeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4bfeffd640 10 monclient: dump: 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:19.975 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4c0d709640 10 monclient: _renew_subs 2024-09-15T01:33:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4c0d709640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4c0d709640 1 -- 172.21.15.204:0/1103682317 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0809c990 con 0x7f4c08137c60 2024-09-15T01:33:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4bfeffd640 1 -- 172.21.15.204:0/1103682317 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4c00006360 con 0x7f4c08137c60 2024-09-15T01:33:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4c0d709640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4c0d709640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4c0d709640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.952+0000 7f4c0d709640 1 -- 172.21.15.204:0/1103682317 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c080298c0 con 0x7f4c08137c60 2024-09-15T01:33:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.953+0000 7f4bfeffd640 1 --2- 172.21.15.204:0/1103682317 >> v2:172.21.15.204:6812/28210 conn(0x7f4be80397b0 0x7f4be803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.953+0000 7f4bfeffd640 1 -- 172.21.15.204:0/1103682317 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4630+0+0 (secure 0 0 0) 0x7f4c0004c180 con 0x7f4c08137c60 2024-09-15T01:33:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.953+0000 7f4c06ffd640 1 --2- 172.21.15.204:0/1103682317 >> v2:172.21.15.204:6812/28210 conn(0x7f4be80397b0 0x7f4be803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.953+0000 7f4c06ffd640 10 monclient: get_auth_request con 0x7f4be80397b0 auth_method 0 2024-09-15T01:33:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.953+0000 7f4c06ffd640 1 --2- 172.21.15.204:0/1103682317 >> v2:172.21.15.204:6812/28210 conn(0x7f4be80397b0 0x7f4be803bc60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8001e20 tx=0x7f4bf8009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.957+0000 7f4bfeffd640 1 -- 172.21.15.204:0/1103682317 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4c00006610 con 0x7f4c08137c60 2024-09-15T01:33:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.957+0000 7f4bfeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:19.957+0000 7f4bfeffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:20.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:20.111+0000 7f4c0d709640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-09-15T01:33:20.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:20.111+0000 7f4c0d709640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-09-15T01:33:20.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:20.111+0000 7f4c0d709640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:20.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:20.111+0000 7f4c0d709640 1 -- 172.21.15.204:0/1103682317 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4c081331d0 con 0x7f4c08137c60 2024-09-15T01:33:20.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:20.350+0000 7f4bfeffd640 1 -- 172.21.15.204:0/1103682317 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}]=0 set pool 4 size to 1 v35) v1 ==== 152+0+0 (secure 0 0 0) 0x7f4c00012330 con 0x7f4c08137c60 2024-09-15T01:33:20.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:20.350+0000 7f4bfeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-09-15T01:33:20.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:20.350+0000 7f4bfeffd640 10 monclient: _finish_command 2 = system:0 set pool 4 size to 1 2024-09-15T01:33:20.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:20.493+0000 7f4c0d709640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-09-15T01:33:20.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:20.493+0000 7f4c0d709640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-09-15T01:33:20.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:20.493+0000 7f4c0d709640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:20.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:20.493+0000 7f4c0d709640 1 -- 172.21.15.204:0/1103682317 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4c080a6880 con 0x7f4c08137c60 2024-09-15T01:33:21.353 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.351+0000 7f4bfeffd640 1 -- 172.21.15.204:0/1103682317 <== mon.2 v2:172.21.15.178:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}]=0 set pool 4 size to 1 v36) v1 ==== 152+0+0 (secure 0 0 0) 0x7f4c000350c0 con 0x7f4c08137c60 2024-09-15T01:33:21.353 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.351+0000 7f4bfeffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-09-15T01:33:21.353 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.351+0000 7f4bfeffd640 10 monclient: _finish_command 3 = system:0 set pool 4 size to 1 2024-09-15T01:33:21.353 INFO:tasks.workunit.client.0.smithi204.stderr:set pool 4 size to 1 2024-09-15T01:33:21.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.358+0000 7f4c0d709640 1 -- 172.21.15.204:0/1103682317 >> v2:172.21.15.204:6812/28210 conn(0x7f4be80397b0 msgr2=0x7f4be803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:21.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.358+0000 7f4c0d709640 1 --2- 172.21.15.204:0/1103682317 >> v2:172.21.15.204:6812/28210 conn(0x7f4be80397b0 0x7f4be803bc60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8001e20 tx=0x7f4bf8009040 comp rx=0 tx=0).stop 2024-09-15T01:33:21.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.358+0000 7f4c0d709640 10 monclient: shutdown 2024-09-15T01:33:21.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.358+0000 7f4c0d709640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:21.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.358+0000 7f4c0d709640 1 -- 172.21.15.204:0/1103682317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c08137c60 msgr2=0x7f4c08134480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:21.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.358+0000 7f4c0d709640 1 --2- 172.21.15.204:0/1103682317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c08137c60 0x7f4c08134480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000be20 tx=0x7f4c000096a0 comp rx=0 tx=0).stop 2024-09-15T01:33:21.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.358+0000 7f4c0d709640 1 -- 172.21.15.204:0/1103682317 shutdown_connections 2024-09-15T01:33:21.360 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.359+0000 7f4c0d709640 1 --2- 172.21.15.204:0/1103682317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c08137c60 0x7f4c08134480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:21.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.359+0000 7f4c0d709640 1 --2- 172.21.15.204:0/1103682317 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4c080a24a0 0x7f4c08133f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:21.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.359+0000 7f4c0d709640 1 --2- 172.21.15.204:0/1103682317 >> v2:172.21.15.204:6812/28210 conn(0x7f4be80397b0 0x7f4be803bc60 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8001e20 tx=0x7f4bf8009040 comp rx=0 tx=0).stop 2024-09-15T01:33:21.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.359+0000 7f4c0d709640 1 --2- 172.21.15.204:0/1103682317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c080a00c0 0x7f4c08133a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:21.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.359+0000 7f4c0d709640 1 -- 172.21.15.204:0/1103682317 >> 172.21.15.204:0/1103682317 conn(0x7f4c080a5f70 msgr2=0x7f4c080a4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:21.361 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.360+0000 7f4c0d709640 1 -- 172.21.15.204:0/1103682317 shutdown_connections 2024-09-15T01:33:21.362 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.360+0000 7f4c0d709640 1 -- 172.21.15.204:0/1103682317 wait complete. 2024-09-15T01:33:21.381 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_config_value foo min_size 1 2024-09-15T01:33:21.381 INFO:tasks.workunit.client.0.smithi204.stderr:+ local pool_name config_opt expected_val val 2024-09-15T01:33:21.381 INFO:tasks.workunit.client.0.smithi204.stderr:+ pool_name=foo 2024-09-15T01:33:21.381 INFO:tasks.workunit.client.0.smithi204.stderr:+ config_opt=min_size 2024-09-15T01:33:21.382 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_val=1 2024-09-15T01:33:21.382 INFO:tasks.workunit.client.0.smithi204.stderr:++ get_config_value_or_die foo min_size 2024-09-15T01:33:21.382 INFO:tasks.workunit.client.0.smithi204.stderr:++ local pool_name config_opt raw val 2024-09-15T01:33:21.382 INFO:tasks.workunit.client.0.smithi204.stderr:++ pool_name=foo 2024-09-15T01:33:21.382 INFO:tasks.workunit.client.0.smithi204.stderr:++ config_opt=min_size 2024-09-15T01:33:21.382 INFO:tasks.workunit.client.0.smithi204.stderr:+++ ceph osd pool get foo min_size 2024-09-15T01:33:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:++ raw='min_size: 1' 2024-09-15T01:33:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:++ [[ 0 -ne 0 ]] 2024-09-15T01:33:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:+++ echo min_size: 1 2024-09-15T01:33:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:+++ sed -e 's/[{} "]//g' 2024-09-15T01:33:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:++ raw=min_size:1 2024-09-15T01:33:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:+++ echo min_size:1 2024-09-15T01:33:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:+++ cut -f2 -d: 2024-09-15T01:33:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:++ val=1 2024-09-15T01:33:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:++ echo 1 2024-09-15T01:33:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:++ return 0 2024-09-15T01:33:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:+ val=1 2024-09-15T01:33:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 1 != \1 ]] 2024-09-15T01:33:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set foo size 4 2024-09-15T01:33:21.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.804+0000 7f723fa57640 10 monclient: get_monmap_and_config 2024-09-15T01:33:21.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.804+0000 7f723fa57640 10 monclient: build_initial_monmap 2024-09-15T01:33:21.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.804+0000 7f723fa57640 10 monclient: monmap: 2024-09-15T01:33:21.806 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:21.806 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:21.806 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:21.804531+0000 2024-09-15T01:33:21.806 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:21.804531+0000 2024-09-15T01:33:21.806 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:21.806 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.804+0000 7f723fa57640 1 Processor -- start 2024-09-15T01:33:21.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.804+0000 7f723fa57640 1 -- start start 2024-09-15T01:33:21.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.804+0000 7f723fa57640 10 monclient: init 2024-09-15T01:33:21.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:21.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 10 monclient: _add_conns ranks=[10,6,0,9,5,1,4,2,7,8,3] 2024-09-15T01:33:21.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f72380a2cd0 0x7f7238132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:21.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 10 monclient(hunting): picked mon.noname-f con 0x7f72380a2cd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:21.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f723809fa20 0x7f723809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 10 monclient(hunting): picked mon.noname-d con 0x7f723809fa20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f72380a08e0 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72380648a0 con 0x7f72380a08e0 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72380a30a0 con 0x7f72380a2cd0 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f72380a0330 con 0x7f723809fa20 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723fa57640 10 monclient(hunting): authenticate will time out at 817.447327s 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f72380a2cd0 0x7f7238132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f72380a2cd0 0x7f7238132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41614/0 (socket says 172.21.15.204:41614) 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 1 -- 172.21.15.204:0/3096226323 learned_addr learned my addr 172.21.15.204:0/3096226323 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:21.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 10 monclient(hunting): get_auth_request con 0x7f72380a2cd0 auth_method 0 2024-09-15T01:33:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723cff2640 1 -- 172.21.15.204:0/3096226323 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f723809fa20 msgr2=0x7f723809fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723cff2640 1 --2- 172.21.15.204:0/3096226323 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f723809fa20 0x7f723809fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 10 monclient(hunting): handle_auth_done global_id 4807 payload 274 2024-09-15T01:33:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 1 -- 172.21.15.204:0/3096226323 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f723809fa20 msgr2=0x7f723809fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 1 --2- 172.21.15.204:0/3096226323 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f723809fa20 0x7f723809fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 1 -- 172.21.15.204:0/3096226323 >> v1:172.21.15.178:6789/0 conn(0x7f72380a08e0 legacy=0x7f72380a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 10 monclient: _finish_hunting 0 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 1 monclient: found mon.noname-f 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.805+0000 7f723d7f3640 1 -- 172.21.15.204:0/3096226323 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72380a1db0 con 0x7f72380a2cd0 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f723d7f3640 1 --2- 172.21.15.204:0/3096226323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72380a2cd0 0x7f7238132c00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7230009f20 tx=0x7f7230009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29179493f1d95c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f722ffff640 1 -- 172.21.15.204:0/3096226323 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7230007880 con 0x7f72380a2cd0 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f722ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f722ffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f722ffff640 10 monclient: dump: 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:21.820 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:21.820 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:21.820 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:21.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f722ffff640 10 monclient: _finish_auth 0 2024-09-15T01:33:21.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f722ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:21.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f722ffff640 1 -- 172.21.15.204:0/3096226323 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723002d050 con 0x7f72380a2cd0 2024-09-15T01:33:21.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f723fa57640 5 monclient: authenticate success, global_id 4807 2024-09-15T01:33:21.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f722ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:21.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f723fa57640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:21.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f722ffff640 1 -- 172.21.15.204:0/3096226323 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7230030360 con 0x7f72380a2cd0 2024-09-15T01:33:21.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f723fa57640 4 set_mon_vals no callback set 2024-09-15T01:33:21.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f722ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:21.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f722ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:21.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.806+0000 7f722ffff640 10 monclient: dump: 2024-09-15T01:33:21.821 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:21.821 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:21.821 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:21.821 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:21.821 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:21.821 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:21.822 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:21.822 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:21.822 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:21.822 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:21.822 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:21.822 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:21.822 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:21.822 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:21.822 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:21.822 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:21.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.807+0000 7f723fa57640 10 monclient: shutdown 2024-09-15T01:33:21.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.807+0000 7f723fa57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.807+0000 7f723fa57640 1 -- 172.21.15.204:0/3096226323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72380a2cd0 msgr2=0x7f7238132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.807+0000 7f723fa57640 1 --2- 172.21.15.204:0/3096226323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72380a2cd0 0x7f7238132c00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7230009f20 tx=0x7f7230009840 comp rx=0 tx=0).stop 2024-09-15T01:33:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.807+0000 7f723fa57640 1 -- 172.21.15.204:0/3096226323 shutdown_connections 2024-09-15T01:33:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.807+0000 7f723fa57640 1 --2- 172.21.15.204:0/3096226323 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f723809fa20 0x7f723809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.807+0000 7f723fa57640 1 --2- 172.21.15.204:0/3096226323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72380a2cd0 0x7f7238132c00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.807+0000 7f723fa57640 1 -- 172.21.15.204:0/3096226323 >> 172.21.15.204:0/3096226323 conn(0x7f723806e860 msgr2=0x7f723806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.807+0000 7f723fa57640 1 -- 172.21.15.204:0/3096226323 shutdown_connections 2024-09-15T01:33:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.807+0000 7f723fa57640 1 -- 172.21.15.204:0/3096226323 wait complete. 2024-09-15T01:33:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.807+0000 7f723fa57640 10 monclient: build_initial_monmap 2024-09-15T01:33:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.807+0000 7f723fa57640 10 monclient: monmap: 2024-09-15T01:33:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:21.824 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:21.824 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:21.824 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:21.824 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:21.824 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:21.824 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:21.824 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:21.824 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:21.824 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:21.824 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:21.824 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:21.824 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:21.825 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:21.825 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:21.825 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:21.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 1 Processor -- start 2024-09-15T01:33:21.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 1 -- start start 2024-09-15T01:33:21.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 10 monclient: init 2024-09-15T01:33:21.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:21.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 10 monclient: _add_conns ranks=[6,4,8,1,3,7,2,5,0] 2024-09-15T01:33:21.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f723809fa20 0x7f7238141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:21.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 10 monclient(hunting): picked mon.noname-d con 0x7f723809fa20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:21.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f72380a08e0 0x7f7238131310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 10 monclient(hunting): picked mon.noname-c con 0x7f72380a08e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f72380a2cd0 0x7f723811dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 10 monclient(hunting): picked mon.noname-e con 0x7f72380a2cd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7238121a20 con 0x7f72380a08e0 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f723809d3d0 con 0x7f723809fa20 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f723811f9a0 con 0x7f72380a2cd0 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723fa57640 10 monclient(hunting): authenticate will time out at 817.450669s 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723d7f3640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f723809fa20 msgr2=0x7f7238141c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723d7f3640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f723809fa20 0x7f7238141c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:21.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723cff2640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f72380a08e0 0x7f7238131310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723cff2640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f72380a08e0 0x7f7238131310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:37590/0 (socket says 172.21.15.204:37590) 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723cff2640 1 -- 172.21.15.204:0/3756834192 learned_addr learned my addr 172.21.15.204:0/3756834192 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723cff2640 10 monclient(hunting): get_auth_request con 0x7f72380a08e0 auth_method 0 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723dff4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f72380a2cd0 0x7f723811dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723cff2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723cff2640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723cff2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723dff4640 10 monclient(hunting): get_auth_request con 0x7f72380a2cd0 auth_method 0 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723dff4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723dff4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723dff4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723cff2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:21.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.808+0000 7f723cff2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:21.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723cff2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:21.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:21.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:21.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:21.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 10 monclient(hunting): handle_auth_done global_id 4679 payload 824 2024-09-15T01:33:21.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 1 -- 172.21.15.204:0/3756834192 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f723809fa20 msgr2=0x7f7238141c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:21.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 1 --2- 172.21.15.204:0/3756834192 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f723809fa20 0x7f7238141c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:21.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 1 -- 172.21.15.204:0/3756834192 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f72380a08e0 msgr2=0x7f7238131310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:21.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 1 --2- 172.21.15.204:0/3756834192 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f72380a08e0 0x7f7238131310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:21.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 10 monclient: _finish_hunting 0 2024-09-15T01:33:21.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 1 monclient: found mon.noname-e 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 1 -- 172.21.15.204:0/3756834192 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7230009ae0 con 0x7f72380a2cd0 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723cff2640 1 --2- 172.21.15.204:0/3756834192 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f72380a08e0 0x7f7238131310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723dff4640 1 --2- 172.21.15.204:0/3756834192 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f72380a2cd0 0x7f723811dff0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f723400b4c0 tx=0x7f7234006f50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722d7fa640 1 -- 172.21.15.204:0/3756834192 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f723400f990 con 0x7f72380a2cd0 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722d7fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722d7fa640 10 monclient: dump: 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:21.829 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722d7fa640 10 monclient: _finish_auth 0 2024-09-15T01:33:21.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722d7fa640 1 -- 172.21.15.204:0/3756834192 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723400fb50 con 0x7f72380a2cd0 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723fa57640 5 monclient: authenticate success, global_id 4679 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723fa57640 10 monclient: _renew_subs 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723fa57640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723fa57640 1 -- 172.21.15.204:0/3756834192 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f723809cb40 con 0x7f72380a2cd0 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722d7fa640 1 -- 172.21.15.204:0/3756834192 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f723400fcc0 con 0x7f72380a2cd0 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722d7fa640 10 monclient: dump: 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:21.831 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:21.832 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:21.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f722f7fe640 4 set_mon_vals no callback set 2024-09-15T01:33:21.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723fa57640 10 monclient: _renew_subs 2024-09-15T01:33:21.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723fa57640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:33:21.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.809+0000 7f723fa57640 1 -- 172.21.15.204:0/3756834192 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7238120030 con 0x7f72380a2cd0 2024-09-15T01:33:21.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.810+0000 7f722d7fa640 1 -- 172.21.15.204:0/3756834192 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f723400f510 con 0x7f72380a2cd0 2024-09-15T01:33:21.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.810+0000 7f723fa57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:21.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.810+0000 7f723fa57640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:21.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.810+0000 7f723fa57640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:33:21.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.810+0000 7f723fa57640 1 -- 172.21.15.204:0/3756834192 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7238070b80 con 0x7f72380a2cd0 2024-09-15T01:33:21.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.810+0000 7f722d7fa640 1 --2- 172.21.15.204:0/3756834192 >> v2:172.21.15.204:6812/28210 conn(0x7f721c0397b0 0x7f721c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:21.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.810+0000 7f722d7fa640 1 -- 172.21.15.204:0/3756834192 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4630+0+0 (secure 0 0 0) 0x7f723404a690 con 0x7f72380a2cd0 2024-09-15T01:33:21.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.810+0000 7f723d7f3640 1 --2- 172.21.15.204:0/3756834192 >> v2:172.21.15.204:6812/28210 conn(0x7f721c0397b0 0x7f721c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:21.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.810+0000 7f723d7f3640 10 monclient: get_auth_request con 0x7f721c0397b0 auth_method 0 2024-09-15T01:33:21.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.811+0000 7f723d7f3640 1 --2- 172.21.15.204:0/3756834192 >> v2:172.21.15.204:6812/28210 conn(0x7f721c0397b0 0x7f721c03bc60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7230033000 tx=0x7f7230036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:21.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.815+0000 7f722d7fa640 1 -- 172.21.15.204:0/3756834192 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f723400f7c0 con 0x7f72380a2cd0 2024-09-15T01:33:21.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.816+0000 7f722d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:21.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.816+0000 7f722d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:21.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.968+0000 7f723fa57640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-09-15T01:33:21.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.968+0000 7f723fa57640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-09-15T01:33:21.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.968+0000 7f723fa57640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:33:21.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:21.968+0000 7f723fa57640 1 -- 172.21.15.204:0/3756834192 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"} v 0) v1 -- 0x7f7238132710 con 0x7f72380a2cd0 2024-09-15T01:33:22.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:22.436+0000 7f722d7fa640 1 -- 172.21.15.204:0/3756834192 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}]=0 set pool 4 size to 4 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7234018b00 con 0x7f72380a2cd0 2024-09-15T01:33:22.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:22.436+0000 7f722d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-09-15T01:33:22.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:22.436+0000 7f722d7fa640 10 monclient: _finish_command 2 = system:0 set pool 4 size to 4 2024-09-15T01:33:22.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:22.539+0000 7f723fa57640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-09-15T01:33:22.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:22.539+0000 7f723fa57640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-09-15T01:33:22.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:22.539+0000 7f723fa57640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:33:22.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:22.539+0000 7f723fa57640 1 -- 172.21.15.204:0/3756834192 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"} v 0) v1 -- 0x7f72380008d0 con 0x7f72380a2cd0 2024-09-15T01:33:23.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.433+0000 7f722d7fa640 1 -- 172.21.15.204:0/3756834192 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}]=0 set pool 4 size to 4 v38) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7234018ce0 con 0x7f72380a2cd0 2024-09-15T01:33:23.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.433+0000 7f722d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-09-15T01:33:23.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.433+0000 7f722d7fa640 10 monclient: _finish_command 3 = system:0 set pool 4 size to 4 2024-09-15T01:33:23.438 INFO:tasks.workunit.client.0.smithi204.stderr:set pool 4 size to 4 2024-09-15T01:33:23.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.438+0000 7f723fa57640 1 -- 172.21.15.204:0/3756834192 >> v2:172.21.15.204:6812/28210 conn(0x7f721c0397b0 msgr2=0x7f721c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:23.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.438+0000 7f723fa57640 1 --2- 172.21.15.204:0/3756834192 >> v2:172.21.15.204:6812/28210 conn(0x7f721c0397b0 0x7f721c03bc60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7230033000 tx=0x7f7230036000 comp rx=0 tx=0).stop 2024-09-15T01:33:23.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.438+0000 7f723fa57640 10 monclient: shutdown 2024-09-15T01:33:23.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.438+0000 7f723fa57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:23.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.438+0000 7f723fa57640 1 -- 172.21.15.204:0/3756834192 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f72380a2cd0 msgr2=0x7f723811dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:23.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.438+0000 7f723fa57640 1 --2- 172.21.15.204:0/3756834192 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f72380a2cd0 0x7f723811dff0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f723400b4c0 tx=0x7f7234006f50 comp rx=0 tx=0).stop 2024-09-15T01:33:23.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.438+0000 7f723fa57640 1 -- 172.21.15.204:0/3756834192 shutdown_connections 2024-09-15T01:33:23.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.438+0000 7f723fa57640 1 --2- 172.21.15.204:0/3756834192 >> v2:172.21.15.204:6812/28210 conn(0x7f721c0397b0 0x7f721c03bc60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:23.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.438+0000 7f723fa57640 1 --2- 172.21.15.204:0/3756834192 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f72380a2cd0 0x7f723811dff0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:23.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.438+0000 7f723fa57640 1 --2- 172.21.15.204:0/3756834192 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f72380a08e0 0x7f7238131310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:23.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.438+0000 7f723fa57640 1 --2- 172.21.15.204:0/3756834192 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f723809fa20 0x7f7238141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:23.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.438+0000 7f723fa57640 1 -- 172.21.15.204:0/3756834192 >> 172.21.15.204:0/3756834192 conn(0x7f723806e860 msgr2=0x7f72380a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:23.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.442+0000 7f723fa57640 1 -- 172.21.15.204:0/3756834192 shutdown_connections 2024-09-15T01:33:23.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.442+0000 7f723fa57640 1 -- 172.21.15.204:0/3756834192 wait complete. 2024-09-15T01:33:23.461 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_config_value foo min_size 2 2024-09-15T01:33:23.462 INFO:tasks.workunit.client.0.smithi204.stderr:+ local pool_name config_opt expected_val val 2024-09-15T01:33:23.462 INFO:tasks.workunit.client.0.smithi204.stderr:+ pool_name=foo 2024-09-15T01:33:23.462 INFO:tasks.workunit.client.0.smithi204.stderr:+ config_opt=min_size 2024-09-15T01:33:23.462 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_val=2 2024-09-15T01:33:23.462 INFO:tasks.workunit.client.0.smithi204.stderr:++ get_config_value_or_die foo min_size 2024-09-15T01:33:23.462 INFO:tasks.workunit.client.0.smithi204.stderr:++ local pool_name config_opt raw val 2024-09-15T01:33:23.463 INFO:tasks.workunit.client.0.smithi204.stderr:++ pool_name=foo 2024-09-15T01:33:23.463 INFO:tasks.workunit.client.0.smithi204.stderr:++ config_opt=min_size 2024-09-15T01:33:23.463 INFO:tasks.workunit.client.0.smithi204.stderr:+++ ceph osd pool get foo min_size 2024-09-15T01:33:23.867 INFO:tasks.workunit.client.0.smithi204.stderr:++ raw='min_size: 2' 2024-09-15T01:33:23.867 INFO:tasks.workunit.client.0.smithi204.stderr:++ [[ 0 -ne 0 ]] 2024-09-15T01:33:23.867 INFO:tasks.workunit.client.0.smithi204.stderr:+++ echo min_size: 2 2024-09-15T01:33:23.867 INFO:tasks.workunit.client.0.smithi204.stderr:+++ sed -e 's/[{} "]//g' 2024-09-15T01:33:23.868 INFO:tasks.workunit.client.0.smithi204.stderr:++ raw=min_size:2 2024-09-15T01:33:23.869 INFO:tasks.workunit.client.0.smithi204.stderr:+++ echo min_size:2 2024-09-15T01:33:23.869 INFO:tasks.workunit.client.0.smithi204.stderr:+++ cut -f2 -d: 2024-09-15T01:33:23.870 INFO:tasks.workunit.client.0.smithi204.stderr:++ val=2 2024-09-15T01:33:23.870 INFO:tasks.workunit.client.0.smithi204.stderr:++ echo 2 2024-09-15T01:33:23.870 INFO:tasks.workunit.client.0.smithi204.stderr:++ return 0 2024-09-15T01:33:23.870 INFO:tasks.workunit.client.0.smithi204.stderr:+ val=2 2024-09-15T01:33:23.870 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 2 != \2 ]] 2024-09-15T01:33:23.870 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set foo size 10 2024-09-15T01:33:23.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.944+0000 7fa87dcef640 10 monclient: get_monmap_and_config 2024-09-15T01:33:23.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.944+0000 7fa87dcef640 10 monclient: build_initial_monmap 2024-09-15T01:33:23.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.944+0000 7fa87dcef640 10 monclient: monmap: 2024-09-15T01:33:23.945 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:23.944563+0000 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:23.944563+0000 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:23.946 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:23.947 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:23.947 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:23.947 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:23.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.944+0000 7fa87dcef640 1 Processor -- start 2024-09-15T01:33:23.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.944+0000 7fa87dcef640 1 -- start start 2024-09-15T01:33:23.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.944+0000 7fa87dcef640 10 monclient: init 2024-09-15T01:33:23.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:23.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 10 monclient: _add_conns ranks=[7,10,9,3,6,4,1,0,5,8,2] 2024-09-15T01:33:23.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa878067130 0x7fa878099a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:23.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 10 monclient(hunting): picked mon.noname-i con 0x7fa878067130 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:23.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa87809b030 0x7fa87809d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:23.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 10 monclient(hunting): picked mon.noname-f con 0x7fa87809b030 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:23.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa87809dbe0 0x7fa87809e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:23.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 10 monclient(hunting): picked mon.noname-a con 0x7fa87809dbe0 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:23.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:23.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa878067b00 con 0x7fa87809dbe0 2024-09-15T01:33:23.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:23.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa878099f50 con 0x7fa87809b030 2024-09-15T01:33:23.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:23.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa87809cde0 con 0x7fa878067130 2024-09-15T01:33:23.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:23.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa87dcef640 10 monclient(hunting): authenticate will time out at 819.587499s 2024-09-15T01:33:23.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa87809b030 0x7fa87809d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:23.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa8777fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa878067130 0x7fa878099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:23.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa87809b030 0x7fa87809d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41616/0 (socket says 172.21.15.204:41616) 2024-09-15T01:33:23.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 1 -- 172.21.15.204:0/3896162174 learned_addr learned my addr 172.21.15.204:0/3896162174 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:23.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa8777fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa878067130 0x7fa878099a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:51626/0 (socket says 172.21.15.204:51626) 2024-09-15T01:33:23.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 10 monclient(hunting): get_auth_request con 0x7fa87809b030 auth_method 0 2024-09-15T01:33:23.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:23.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:23.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:23.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa8777fe640 10 monclient(hunting): get_auth_request con 0x7fa878067130 auth_method 0 2024-09-15T01:33:23.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa8777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:23.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa8777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:23.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa8777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:23.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:23.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:23.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:23.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa8777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:23.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa8777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:23.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa8777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:23.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 10 monclient(hunting): handle_auth_done global_id 4816 payload 274 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 1 -- 172.21.15.204:0/3896162174 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa878067130 msgr2=0x7fa878099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 1 --2- 172.21.15.204:0/3896162174 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa878067130 0x7fa878099a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 1 -- 172.21.15.204:0/3896162174 >> v2:172.21.15.178:3300/0 conn(0x7fa87809dbe0 msgr2=0x7fa87809e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa877fff640 1 --2- 172.21.15.204:0/3896162174 >> v2:172.21.15.178:3300/0 conn(0x7fa87809dbe0 0x7fa87809e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 1 --2- 172.21.15.204:0/3896162174 >> v2:172.21.15.178:3300/0 conn(0x7fa87809dbe0 0x7fa87809e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 1 monclient: found mon.noname-f 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa86ffff640 1 -- 172.21.15.204:0/3896162174 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8780648a0 con 0x7fa87809b030 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.945+0000 7fa8777fe640 1 --2- 172.21.15.204:0/3896162174 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa878067130 0x7fa878099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa86ffff640 1 --2- 172.21.15.204:0/3896162174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87809b030 0x7fa87809d6a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa87809e980 tx=0x7fa8600091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6844258002e50a32 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:23.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa876ffd640 1 -- 172.21.15.204:0/3896162174 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa86002acb0 con 0x7fa87809b030 2024-09-15T01:33:23.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa876ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:23.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa876ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:23.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa876ffd640 10 monclient: dump: 2024-09-15T01:33:23.960 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:23.960 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:23.960 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:23.960 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:23.960 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:23.960 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:23.960 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:23.960 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:23.960 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa876ffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa876ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa876ffd640 1 -- 172.21.15.204:0/3896162174 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86002ae30 con 0x7fa87809b030 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa87dcef640 5 monclient: authenticate success, global_id 4816 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa87dcef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:23.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa876ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:23.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa87dcef640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:23.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa876ffd640 1 -- 172.21.15.204:0/3896162174 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa860031ca0 con 0x7fa87809b030 2024-09-15T01:33:23.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa876ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:23.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa87dcef640 4 set_mon_vals no callback set 2024-09-15T01:33:23.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa876ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:23.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.946+0000 7fa876ffd640 10 monclient: dump: 2024-09-15T01:33:23.962 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:23.962 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:23.962 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:23.962 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:23.962 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:23.962 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:23.963 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:23.963 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:23.963 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:23.963 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:23.963 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:23.963 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:23.963 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:23.963 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:23.963 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:23.963 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:23.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 10 monclient: shutdown 2024-09-15T01:33:23.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:23.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 1 -- 172.21.15.204:0/3896162174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87809b030 msgr2=0x7fa87809d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:23.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 1 --2- 172.21.15.204:0/3896162174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87809b030 0x7fa87809d6a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa87809e980 tx=0x7fa8600091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:23.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 1 -- 172.21.15.204:0/3896162174 shutdown_connections 2024-09-15T01:33:23.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 1 --2- 172.21.15.204:0/3896162174 >> v2:172.21.15.178:3300/0 conn(0x7fa87809dbe0 0x7fa87809e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:23.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 1 --2- 172.21.15.204:0/3896162174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87809b030 0x7fa87809d6a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:23.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 1 --2- 172.21.15.204:0/3896162174 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa878067130 0x7fa878099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:23.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 1 -- 172.21.15.204:0/3896162174 >> 172.21.15.204:0/3896162174 conn(0x7fa878068af0 msgr2=0x7fa8780a2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:23.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 1 -- 172.21.15.204:0/3896162174 shutdown_connections 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 1 -- 172.21.15.204:0/3896162174 wait complete. 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 10 monclient: build_initial_monmap 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.947+0000 7fa87dcef640 10 monclient: monmap: 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:23.965 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 1 Processor -- start 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 1 -- start start 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 10 monclient: init 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 10 monclient: _add_conns ranks=[8,4,1,6,5,3,2,0,7] 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa878067130 0x7fa878141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:23.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 10 monclient(hunting): picked mon.noname-e con 0x7fa878067130 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa87809b030 0x7fa878135b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 10 monclient(hunting): picked mon.noname-c con 0x7fa87809b030 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87809dbe0 0x7fa87811f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 10 monclient(hunting): picked mon.noname-f con 0x7fa87809dbe0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878122f70 con 0x7fa87809dbe0 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa878142bb0 con 0x7fa87809b030 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa87809cab0 con 0x7fa878067130 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa87dcef640 10 monclient(hunting): authenticate will time out at 819.590862s 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa877fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87809dbe0 0x7fa87811f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:23.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa877fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87809dbe0 0x7fa87811f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41624/0 (socket says 172.21.15.204:41624) 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa877fff640 1 -- 172.21.15.204:0/2787778383 learned_addr learned my addr 172.21.15.204:0/2787778383 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa877fff640 10 monclient(hunting): get_auth_request con 0x7fa87809dbe0 auth_method 0 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa877fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa877fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.948+0000 7fa877fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86ffff640 1 --2- 172.21.15.204:0/2787778383 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa87809b030 0x7fa878135b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86ffff640 10 monclient(hunting): get_auth_request con 0x7fa87809b030 auth_method 0 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:23.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 10 monclient(hunting): handle_auth_done global_id 4825 payload 824 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa8777fe640 1 --2- 172.21.15.204:0/2787778383 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa878067130 0x7fa878141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 1 -- 172.21.15.204:0/2787778383 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa878067130 msgr2=0x7fa878141cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 1 --2- 172.21.15.204:0/2787778383 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa878067130 0x7fa878141cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 1 -- 172.21.15.204:0/2787778383 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa87809b030 msgr2=0x7fa878135b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 1 --2- 172.21.15.204:0/2787778383 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa87809b030 0x7fa878135b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 1 monclient: found mon.noname-f 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 1 -- 172.21.15.204:0/2787778383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa868009390 con 0x7fa87809dbe0 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa877fff640 1 --2- 172.21.15.204:0/2787778383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87809dbe0 0x7fa87811f8a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa87000cdb0 tx=0x7fa8700077d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:23.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86f7fe640 1 -- 172.21.15.204:0/2787778383 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa87000ac10 con 0x7fa87809dbe0 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86f7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86f7fe640 10 monclient: dump: 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:23.970 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa87dcef640 5 monclient: authenticate success, global_id 4825 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86f7fe640 1 -- 172.21.15.204:0/2787778383 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87000adb0 con 0x7fa87809dbe0 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa87dcef640 10 monclient: _renew_subs 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa87dcef640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:23.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa87dcef640 1 -- 172.21.15.204:0/2787778383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa860009450 con 0x7fa87809dbe0 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86f7fe640 1 -- 172.21.15.204:0/2787778383 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa870018d80 con 0x7fa87809dbe0 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa86f7fe640 10 monclient: dump: 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:23.972 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa8767fc640 4 set_mon_vals no callback set 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa87dcef640 10 monclient: _renew_subs 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa87dcef640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.949+0000 7fa87dcef640 1 -- 172.21.15.204:0/2787778383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa878142210 con 0x7fa87809dbe0 2024-09-15T01:33:23.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.950+0000 7fa86f7fe640 1 -- 172.21.15.204:0/2787778383 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa870003020 con 0x7fa87809dbe0 2024-09-15T01:33:23.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.950+0000 7fa86f7fe640 1 --2- 172.21.15.204:0/2787778383 >> v2:172.21.15.204:6812/28210 conn(0x7fa8580397b0 0x7fa85803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:23.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.950+0000 7fa86f7fe640 1 -- 172.21.15.204:0/2787778383 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4630+0+0 (secure 0 0 0) 0x7fa870013070 con 0x7fa87809dbe0 2024-09-15T01:33:23.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.950+0000 7fa87dcef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:23.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.950+0000 7fa8777fe640 1 --2- 172.21.15.204:0/2787778383 >> v2:172.21.15.204:6812/28210 conn(0x7fa8580397b0 0x7fa85803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:23.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.950+0000 7fa87dcef640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:23.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.950+0000 7fa87dcef640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:23.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.950+0000 7fa87dcef640 1 -- 172.21.15.204:0/2787778383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87809d080 con 0x7fa87809dbe0 2024-09-15T01:33:23.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.950+0000 7fa8777fe640 10 monclient: get_auth_request con 0x7fa8580397b0 auth_method 0 2024-09-15T01:33:23.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.951+0000 7fa8777fe640 1 --2- 172.21.15.204:0/2787778383 >> v2:172.21.15.204:6812/28210 conn(0x7fa8580397b0 0x7fa85803bc60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa868006e70 tx=0x7fa868008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:23.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.955+0000 7fa86f7fe640 1 -- 172.21.15.204:0/2787778383 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa87809d080 con 0x7fa87809dbe0 2024-09-15T01:33:23.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.955+0000 7fa86f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:23.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:23.955+0000 7fa86f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:24.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:24.109+0000 7fa87dcef640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-09-15T01:33:24.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:24.109+0000 7fa87dcef640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-09-15T01:33:24.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:24.109+0000 7fa87dcef640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:24.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:24.109+0000 7fa87dcef640 1 -- 172.21.15.204:0/2787778383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"} v 0) v1 -- 0x7fa87809d080 con 0x7fa87809dbe0 2024-09-15T01:33:24.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:24.448+0000 7fa86f7fe640 1 -- 172.21.15.204:0/2787778383 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}]=0 set pool 4 size to 10 v39) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa87809d080 con 0x7fa87809dbe0 2024-09-15T01:33:24.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:24.448+0000 7fa86f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-09-15T01:33:24.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:24.448+0000 7fa86f7fe640 10 monclient: _finish_command 2 = system:0 set pool 4 size to 10 2024-09-15T01:33:24.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:24.574+0000 7fa87dcef640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-09-15T01:33:24.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:24.575+0000 7fa87dcef640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-09-15T01:33:24.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:24.575+0000 7fa87dcef640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:24.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:24.575+0000 7fa87dcef640 1 -- 172.21.15.204:0/2787778383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"} v 0) v1 -- 0x7fa87809d080 con 0x7fa87809dbe0 2024-09-15T01:33:25.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.458+0000 7fa86f7fe640 1 -- 172.21.15.204:0/2787778383 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}]=0 set pool 4 size to 10 v40) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa87809d080 con 0x7fa87809dbe0 2024-09-15T01:33:25.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.458+0000 7fa86f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-09-15T01:33:25.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.458+0000 7fa86f7fe640 10 monclient: _finish_command 3 = system:0 set pool 4 size to 10 2024-09-15T01:33:25.463 INFO:tasks.workunit.client.0.smithi204.stderr:set pool 4 size to 10 2024-09-15T01:33:25.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.463+0000 7fa87dcef640 1 -- 172.21.15.204:0/2787778383 >> v2:172.21.15.204:6812/28210 conn(0x7fa8580397b0 msgr2=0x7fa85803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:25.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.463+0000 7fa87dcef640 1 --2- 172.21.15.204:0/2787778383 >> v2:172.21.15.204:6812/28210 conn(0x7fa8580397b0 0x7fa85803bc60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa868006e70 tx=0x7fa868008040 comp rx=0 tx=0).stop 2024-09-15T01:33:25.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.463+0000 7fa87dcef640 10 monclient: shutdown 2024-09-15T01:33:25.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.463+0000 7fa87dcef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:25.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.463+0000 7fa87dcef640 1 -- 172.21.15.204:0/2787778383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87809dbe0 msgr2=0x7fa87811f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:25.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.463+0000 7fa87dcef640 1 --2- 172.21.15.204:0/2787778383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87809dbe0 0x7fa87811f8a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa87000cdb0 tx=0x7fa8700077d0 comp rx=0 tx=0).stop 2024-09-15T01:33:25.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.464+0000 7fa87dcef640 1 -- 172.21.15.204:0/2787778383 shutdown_connections 2024-09-15T01:33:25.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.464+0000 7fa87dcef640 1 --2- 172.21.15.204:0/2787778383 >> v2:172.21.15.204:6812/28210 conn(0x7fa8580397b0 0x7fa85803bc60 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa868006e70 tx=0x7fa868008040 comp rx=0 tx=0).stop 2024-09-15T01:33:25.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.464+0000 7fa87dcef640 1 --2- 172.21.15.204:0/2787778383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87809dbe0 0x7fa87811f8a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:25.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.464+0000 7fa87dcef640 1 --2- 172.21.15.204:0/2787778383 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa87809b030 0x7fa878135b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:25.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.464+0000 7fa87dcef640 1 --2- 172.21.15.204:0/2787778383 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa878067130 0x7fa878141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:25.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.464+0000 7fa87dcef640 1 -- 172.21.15.204:0/2787778383 >> 172.21.15.204:0/2787778383 conn(0x7fa878068af0 msgr2=0x7fa87806d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:25.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.468+0000 7fa87dcef640 1 -- 172.21.15.204:0/2787778383 shutdown_connections 2024-09-15T01:33:25.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.469+0000 7fa87dcef640 1 -- 172.21.15.204:0/2787778383 wait complete. 2024-09-15T01:33:25.489 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_config_value foo min_size 5 2024-09-15T01:33:25.489 INFO:tasks.workunit.client.0.smithi204.stderr:+ local pool_name config_opt expected_val val 2024-09-15T01:33:25.489 INFO:tasks.workunit.client.0.smithi204.stderr:+ pool_name=foo 2024-09-15T01:33:25.489 INFO:tasks.workunit.client.0.smithi204.stderr:+ config_opt=min_size 2024-09-15T01:33:25.490 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_val=5 2024-09-15T01:33:25.490 INFO:tasks.workunit.client.0.smithi204.stderr:++ get_config_value_or_die foo min_size 2024-09-15T01:33:25.490 INFO:tasks.workunit.client.0.smithi204.stderr:++ local pool_name config_opt raw val 2024-09-15T01:33:25.490 INFO:tasks.workunit.client.0.smithi204.stderr:++ pool_name=foo 2024-09-15T01:33:25.490 INFO:tasks.workunit.client.0.smithi204.stderr:++ config_opt=min_size 2024-09-15T01:33:25.490 INFO:tasks.workunit.client.0.smithi204.stderr:+++ ceph osd pool get foo min_size 2024-09-15T01:33:25.890 INFO:tasks.workunit.client.0.smithi204.stderr:++ raw='min_size: 5' 2024-09-15T01:33:25.890 INFO:tasks.workunit.client.0.smithi204.stderr:++ [[ 0 -ne 0 ]] 2024-09-15T01:33:25.891 INFO:tasks.workunit.client.0.smithi204.stderr:+++ echo min_size: 5 2024-09-15T01:33:25.891 INFO:tasks.workunit.client.0.smithi204.stderr:+++ sed -e 's/[{} "]//g' 2024-09-15T01:33:25.891 INFO:tasks.workunit.client.0.smithi204.stderr:++ raw=min_size:5 2024-09-15T01:33:25.892 INFO:tasks.workunit.client.0.smithi204.stderr:+++ echo min_size:5 2024-09-15T01:33:25.892 INFO:tasks.workunit.client.0.smithi204.stderr:+++ cut -f2 -d: 2024-09-15T01:33:25.893 INFO:tasks.workunit.client.0.smithi204.stderr:++ val=5 2024-09-15T01:33:25.893 INFO:tasks.workunit.client.0.smithi204.stderr:++ echo 5 2024-09-15T01:33:25.893 INFO:tasks.workunit.client.0.smithi204.stderr:++ return 0 2024-09-15T01:33:25.893 INFO:tasks.workunit.client.0.smithi204.stderr:+ val=5 2024-09-15T01:33:25.893 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 5 != \5 ]] 2024-09-15T01:33:25.894 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool set foo size 0 2024-09-15T01:33:25.894 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:25.894 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set foo size 0 2024-09-15T01:33:25.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.967+0000 7f01d459d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:25.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.967+0000 7f01d459d640 10 monclient: build_initial_monmap 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.967+0000 7f01d459d640 10 monclient: monmap: 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:25.967815+0000 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:25.967815+0000 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:25.969 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:25.970 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:25.970 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:25.970 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:25.970 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:25.970 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:25.970 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:25.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.967+0000 7f01d459d640 1 Processor -- start 2024-09-15T01:33:25.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 1 -- start start 2024-09-15T01:33:25.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 10 monclient: init 2024-09-15T01:33:25.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:25.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 10 monclient: _add_conns ranks=[3,0,6,2,7,8,4,1,9,10,5] 2024-09-15T01:33:25.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc0a4590 0x7f01cc09e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:25.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 10 monclient(hunting): picked mon.noname-g con 0x7f01cc0a4590 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:25.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f01cc06edd0 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:25.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01cc135ad0 0x7f01cc137eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 10 monclient(hunting): picked mon.noname-d con 0x7f01cc135ad0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01cc0648a0 con 0x7f01cc06edd0 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc0a4960 con 0x7f01cc0a4590 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc06dba0 con 0x7f01cc135ad0 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d459d640 10 monclient(hunting): authenticate will time out at 821.610757s 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d2339640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc0a4590 msgr2=0x7f01cc09e750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d2339640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc0a4590 0x7f01cc09e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d2b3a640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01cc135ad0 msgr2=0x7f01cc137eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.968+0000 7f01d2b3a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01cc135ad0 0x7f01cc137eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1b38640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f01cc06edd0 0x7f01cc09ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.204:42068/0 (socket says 172.21.15.204:42068) 2024-09-15T01:33:25.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1b38640 1 -- 172.21.15.204:0/944846876 learned_addr learned my addr 172.21.15.204:0/944846876 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230376026 0 0) 0x7f01cc0648a0 con 0x7f01cc06edd0 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient(hunting): my global_id is 4779 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01b80034e0 con 0x7f01cc06edd0 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3998912965 0 0) 0x7f01b80034e0 con 0x7f01cc06edd0 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01cc135ad0 msgr2=0x7f01cc137eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f01cc135ad0 0x7f01cc137eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc0a4590 msgr2=0x7f01cc09e750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc0a4590 0x7f01cc09e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient: _finish_hunting 0 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:25.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc0a04c0 con 0x7f01cc06edd0 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient: _finish_auth 0 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d459d640 5 monclient: authenticate success, global_id 4779 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d459d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f01bc0018f0 con 0x7f01cc06edd0 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient: dump: 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:25.984 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:33:25.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 >> v1:172.21.15.178:6789/0 conn(0x7f01cc06edd0 legacy=0x7f01cc09ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient: _add_conns ranks=[7,4,8,0,5,1,3,2,6] 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01b8004250 0x7f01b8004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient(hunting): picked mon.i con 0x7f01b8004250 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01b8004ec0 0x7f01b80099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient(hunting): picked mon.c con 0x7f01b8004ec0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f01b8009f00 0x7f01b800c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient(hunting): picked mon.e con 0x7f01b8009f00 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc0a4960 con 0x7f01b8004ec0 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc06dba0 con 0x7f01b8004250 2024-09-15T01:33:25.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f01b800cb80 con 0x7f01b8009f00 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1337640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d459d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2339640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01b8004250 0x7f01b8004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1b38640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01b8004ec0 0x7f01b80099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2339640 10 monclient(hunting): get_auth_request con 0x7f01b8004250 auth_method 0 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2339640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1b38640 10 monclient(hunting): get_auth_request con 0x7f01b8004ec0 auth_method 0 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:25.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1b38640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2b3a640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f01b8009f00 0x7f01b800c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2b3a640 10 monclient(hunting): get_auth_request con 0x7f01b8009f00 auth_method 0 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2b3a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2339640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1b38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1b38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d1b38640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:25.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.969+0000 7f01d2b3a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 10 monclient(hunting): handle_auth_done global_id 4779 payload 293 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 1 -- 172.21.15.204:0/944846876 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f01b8009f00 msgr2=0x7f01b800c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f01b8009f00 0x7f01b800c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 1 -- 172.21.15.204:0/944846876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01b8004250 msgr2=0x7f01b8004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01b8004250 0x7f01b8004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 10 monclient: _finish_hunting 0 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 1 monclient: found mon.c 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 1 -- 172.21.15.204:0/944846876 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b80034e0 con 0x7f01b8004ec0 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 10 monclient: _finish_auth 0 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:25.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d2b3a640 1 -- 172.21.15.204:0/944846876 reap_dead start 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d2339640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01b8004250 0x7f01b8004780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1b38640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01b8004ec0 0x7f01b80099c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0018c0 tx=0x7f01bc001630 comp rx=0 tx=0).ready entity=mon.4 client_cookie=be6e572802817a16 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01bc005d10 con 0x7f01b8004ec0 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1337640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1337640 10 monclient: dump: 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:25.990 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d459d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc02d070 con 0x7f01b8004ec0 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1337640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d459d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:25.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d459d640 4 set_mon_vals no callback set 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1337640 1 -- 172.21.15.204:0/944846876 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01bc007570 con 0x7f01b8004ec0 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.970+0000 7f01d1337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.971+0000 7f01d1337640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.971+0000 7f01d1337640 10 monclient: dump: 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:25.992 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.971+0000 7f01d459d640 10 monclient: shutdown 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.971+0000 7f01d459d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.971+0000 7f01d459d640 1 -- 172.21.15.204:0/944846876 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01b8004ec0 msgr2=0x7f01b80099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.971+0000 7f01d459d640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01b8004ec0 0x7f01b80099c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0018c0 tx=0x7f01bc001630 comp rx=0 tx=0).stop 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.971+0000 7f01d459d640 1 -- 172.21.15.204:0/944846876 shutdown_connections 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.971+0000 7f01d459d640 1 --2- 172.21.15.204:0/944846876 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01b8004ec0 0x7f01b80099c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.971+0000 7f01d459d640 1 -- 172.21.15.204:0/944846876 >> 172.21.15.204:0/944846876 conn(0x7f01cc067700 msgr2=0x7f01cc0a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:25.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.971+0000 7f01d459d640 1 -- 172.21.15.204:0/944846876 shutdown_connections 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.971+0000 7f01d459d640 1 -- 172.21.15.204:0/944846876 wait complete. 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.971+0000 7f01d459d640 10 monclient: build_initial_monmap 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 10 monclient: monmap: 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:25.994 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 1 Processor -- start 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 1 -- start start 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 10 monclient: init 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 10 monclient: _add_conns ranks=[4,3,7,2,6,8,1,0,5] 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01b8004ec0 0x7f01cc135320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 10 monclient(hunting): picked mon.noname-c con 0x7f01b8004ec0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:25.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc12cb50 0x7f01cc12cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 10 monclient(hunting): picked mon.noname-g con 0x7f01cc12cb50 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01cc12d460 0x7f01cc145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 10 monclient(hunting): picked mon.noname-i con 0x7f01cc12d460 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc06d070 con 0x7f01cc12cb50 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc06cc80 con 0x7f01b8004ec0 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc12f310 con 0x7f01cc12d460 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d1b38640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc12cb50 msgr2=0x7f01cc12cf20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d1b38640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc12cb50 0x7f01cc12cf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:25.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d459d640 10 monclient(hunting): authenticate will time out at 821.614831s 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d2339640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01b8004ec0 0x7f01cc135320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d2b3a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01cc12d460 0x7f01cc145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d2339640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01b8004ec0 0x7f01cc135320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:37620/0 (socket says 172.21.15.204:37620) 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d2339640 1 -- 172.21.15.204:0/830912970 learned_addr learned my addr 172.21.15.204:0/830912970 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d2339640 10 monclient(hunting): get_auth_request con 0x7f01b8004ec0 auth_method 0 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d2339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d2339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d2339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d2b3a640 10 monclient(hunting): get_auth_request con 0x7f01cc12d460 auth_method 0 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d2b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d2b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.972+0000 7f01d2b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:25.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 10 monclient(hunting): handle_auth_done global_id 4648 payload 824 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 1 -- 172.21.15.204:0/830912970 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01cc12d460 msgr2=0x7f01cc145f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 1 --2- 172.21.15.204:0/830912970 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01cc12d460 0x7f01cc145f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 1 -- 172.21.15.204:0/830912970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc12cb50 msgr2=0x7f01cc12cf20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 1 --2- 172.21.15.204:0/830912970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01cc12cb50 0x7f01cc12cf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 10 monclient: _finish_hunting 0 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 1 monclient: found mon.noname-c 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:25.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 1 -- 172.21.15.204:0/830912970 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc12f8d0 con 0x7f01b8004ec0 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d2339640 1 --2- 172.21.15.204:0/830912970 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01b8004ec0 0x7f01cc135320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009770 tx=0x7f01c8009060 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01c27fc640 1 -- 172.21.15.204:0/830912970 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01c8012a80 con 0x7f01b8004ec0 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01c27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01c27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01c27fc640 10 monclient: dump: 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:25.999 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01c27fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d459d640 5 monclient: authenticate success, global_id 4648 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01c27fc640 1 -- 172.21.15.204:0/830912970 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c8012be0 con 0x7f01b8004ec0 2024-09-15T01:33:26.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d459d640 10 monclient: _renew_subs 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d459d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d459d640 1 -- 172.21.15.204:0/830912970 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01cc09a720 con 0x7f01b8004ec0 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01c27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01c27fc640 1 -- 172.21.15.204:0/830912970 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01c8012d50 con 0x7f01b8004ec0 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01c27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01c27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d0b36640 4 set_mon_vals no callback set 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01c27fc640 10 monclient: dump: 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.001 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d459d640 10 monclient: _renew_subs 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d459d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:26.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.973+0000 7f01d459d640 1 -- 172.21.15.204:0/830912970 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01cc12e150 con 0x7f01b8004ec0 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.974+0000 7f01c27fc640 1 -- 172.21.15.204:0/830912970 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f01c8012600 con 0x7f01b8004ec0 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.974+0000 7f01d459d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.974+0000 7f01d459d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.974+0000 7f01d459d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.974+0000 7f01d459d640 1 -- 172.21.15.204:0/830912970 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01cc070b80 con 0x7f01b8004ec0 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.974+0000 7f01c27fc640 1 --2- 172.21.15.204:0/830912970 >> v2:172.21.15.204:6812/28210 conn(0x7f01b00397b0 0x7f01b003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.974+0000 7f01c27fc640 1 -- 172.21.15.204:0/830912970 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4630+0+0 (secure 0 0 0) 0x7f01c804ae90 con 0x7f01b8004ec0 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.974+0000 7f01d1b38640 1 --2- 172.21.15.204:0/830912970 >> v2:172.21.15.204:6812/28210 conn(0x7f01b00397b0 0x7f01b003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.974+0000 7f01d1b38640 10 monclient: get_auth_request con 0x7f01b00397b0 auth_method 0 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.975+0000 7f01d1b38640 1 --2- 172.21.15.204:0/830912970 >> v2:172.21.15.204:6812/28210 conn(0x7f01b00397b0 0x7f01b003bc60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f01bc032000 tx=0x7f01bc033000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.979+0000 7f01c27fc640 1 -- 172.21.15.204:0/830912970 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f01c80128b0 con 0x7f01b8004ec0 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.979+0000 7f01c27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:25.979+0000 7f01c27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:26.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.134+0000 7f01d459d640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}] 2024-09-15T01:33:26.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.134+0000 7f01d459d640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}] 2024-09-15T01:33:26.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.134+0000 7f01d459d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:26.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.134+0000 7f01d459d640 1 -- 172.21.15.204:0/830912970 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"} v 0) v1 -- 0x7f01cc06d8c0 con 0x7f01b8004ec0 2024-09-15T01:33:26.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.135+0000 7f01c27fc640 1 -- 172.21.15.204:0/830912970 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}]=-22 pool size must be between 1 and 10 v40) v1 ==== 136+0+0 (secure 0 0 0) 0x7f01c804c020 con 0x7f01b8004ec0 2024-09-15T01:33:26.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.135+0000 7f01c27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}] 2024-09-15T01:33:26.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.135+0000 7f01c27fc640 10 monclient: _finish_command 2 = mon:22 pool size must be between 1 and 10 2024-09-15T01:33:26.136 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: pool size must be between 1 and 10 2024-09-15T01:33:26.140 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:26.140 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool set foo size 20 2024-09-15T01:33:26.140 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:26.140 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set foo size 20 2024-09-15T01:33:26.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.215+0000 7fe0b8878640 10 monclient: get_monmap_and_config 2024-09-15T01:33:26.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.215+0000 7fe0b8878640 10 monclient: build_initial_monmap 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.215+0000 7fe0b8878640 10 monclient: monmap: 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:26.215618+0000 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:26.215618+0000 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:26.217 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:26.218 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:26.218 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:26.218 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:26.218 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:26.218 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.215+0000 7fe0b8878640 1 Processor -- start 2024-09-15T01:33:26.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.215+0000 7fe0b8878640 1 -- start start 2024-09-15T01:33:26.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.215+0000 7fe0b8878640 10 monclient: init 2024-09-15T01:33:26.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:26.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 10 monclient: _add_conns ranks=[1,3,5,4,2,9,6,0,8,10,7] 2024-09-15T01:33:26.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe0b4067130 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:26.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0b409f380 0x7fe0b4099a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 10 monclient(hunting): picked mon.noname-g con 0x7fe0b409f380 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:26.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe0b409ddb0 0x7fe0b413a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 10 monclient(hunting): picked mon.noname-h con 0x7fe0b409ddb0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b40648a0 con 0x7fe0b4067130 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b409a930 con 0x7fe0b409f380 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b409e790 con 0x7fe0b409ddb0 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b8878640 10 monclient(hunting): authenticate will time out at 821.858420s 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0a9d9b640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0b409f380 msgr2=0x7fe0b4099a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0a9d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0b409f380 0x7fe0b4099a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b259c640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fe0b4067130 0x7fe0b409d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:33844/0 (socket says 172.21.15.204:33844) 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b259c640 1 -- 172.21.15.204:0/3955879248 learned_addr learned my addr 172.21.15.204:0/3955879248 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 1 --2- 172.21.15.204:0/3955879248 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe0b409ddb0 0x7fe0b413a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 10 monclient(hunting): get_auth_request con 0x7fe0b409ddb0 auth_method 0 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b1d9b640 1 -- 172.21.15.204:0/3955879248 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225450972 0 0) 0x7fe0b40648a0 con 0x7fe0b4067130 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b1d9b640 10 monclient(hunting): my global_id is 4834 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b1d9b640 1 -- 172.21.15.204:0/3955879248 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a00034e0 con 0x7fe0b4067130 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 10 monclient(hunting): handle_auth_done global_id 4721 payload 274 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 1 -- 172.21.15.204:0/3955879248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0b409f380 msgr2=0x7fe0b4099a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:26.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 1 --2- 172.21.15.204:0/3955879248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0b409f380 0x7fe0b4099a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 1 -- 172.21.15.204:0/3955879248 >> v1:172.21.15.204:6789/0 conn(0x7fe0b4067130 legacy=0x7fe0b409d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 1 monclient: found mon.noname-h 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.216+0000 7fe0b2d9d640 1 -- 172.21.15.204:0/3955879248 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b4067ec0 con 0x7fe0b409ddb0 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b2d9d640 1 --2- 172.21.15.204:0/3955879248 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe0b409ddb0 0x7fe0b413a8c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009e90 tx=0x7fe0ac009780 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a8c24b13349726c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b1d9b640 1 -- 172.21.15.204:0/3955879248 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0ac0079d0 con 0x7fe0b409ddb0 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b1d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b1d9b640 10 monclient: dump: 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.229 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.230 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b1d9b640 10 monclient: _finish_auth 0 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b1d9b640 1 -- 172.21.15.204:0/3955879248 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac02d050 con 0x7fe0b409ddb0 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b1d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b1d9b640 1 -- 172.21.15.204:0/3955879248 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0ac031400 con 0x7fe0b409ddb0 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b8878640 5 monclient: authenticate success, global_id 4721 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b8878640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b8878640 4 set_mon_vals no callback set 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b1d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.217+0000 7fe0b1d9b640 10 monclient: dump: 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.231 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.232 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.218+0000 7fe0b8878640 10 monclient: shutdown 2024-09-15T01:33:26.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.218+0000 7fe0b8878640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:26.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.218+0000 7fe0b8878640 1 -- 172.21.15.204:0/3955879248 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe0b409ddb0 msgr2=0x7fe0b413a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:26.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.218+0000 7fe0b8878640 1 --2- 172.21.15.204:0/3955879248 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe0b409ddb0 0x7fe0b413a8c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009e90 tx=0x7fe0ac009780 comp rx=0 tx=0).stop 2024-09-15T01:33:26.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.218+0000 7fe0b8878640 1 -- 172.21.15.204:0/3955879248 shutdown_connections 2024-09-15T01:33:26.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.218+0000 7fe0b8878640 1 --2- 172.21.15.204:0/3955879248 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe0b409ddb0 0x7fe0b413a8c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.218+0000 7fe0b8878640 1 --2- 172.21.15.204:0/3955879248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0b409f380 0x7fe0b4099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.218+0000 7fe0b8878640 1 -- 172.21.15.204:0/3955879248 >> 172.21.15.204:0/3955879248 conn(0x7fe0b4068af0 msgr2=0x7fe0b40a2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:26.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.218+0000 7fe0b8878640 1 -- 172.21.15.204:0/3955879248 shutdown_connections 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.218+0000 7fe0b8878640 1 -- 172.21.15.204:0/3955879248 wait complete. 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.218+0000 7fe0b8878640 10 monclient: build_initial_monmap 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.218+0000 7fe0b8878640 10 monclient: monmap: 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:26.234 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 1 Processor -- start 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 1 -- start start 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 10 monclient: init 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 10 monclient: _add_conns ranks=[1,3,2,7,8,4,6,5,0] 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0b4067130 0x7fe0b4135990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 10 monclient(hunting): picked mon.noname-f con 0x7fe0b4067130 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0b409ddb0 0x7fe0b411f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 10 monclient(hunting): picked mon.noname-g con 0x7fe0b409ddb0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0b409f380 0x7fe0b411f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 10 monclient(hunting): picked mon.noname-b con 0x7fe0b409f380 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b409cab0 con 0x7fe0b4067130 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b4142a10 con 0x7fe0b409f380 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b4122d90 con 0x7fe0b409ddb0 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0a9d9b640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0b409ddb0 msgr2=0x7fe0b411f460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:26.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b8878640 10 monclient(hunting): authenticate will time out at 821.861835s 2024-09-15T01:33:26.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0b4067130 0x7fe0b4135990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0a9d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0b409ddb0 0x7fe0b411f460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:26.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0b4067130 0x7fe0b4135990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41628/0 (socket says 172.21.15.204:41628) 2024-09-15T01:33:26.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b259c640 1 -- 172.21.15.204:0/3039990568 learned_addr learned my addr 172.21.15.204:0/3039990568 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:26.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b259c640 10 monclient(hunting): get_auth_request con 0x7fe0b4067130 auth_method 0 2024-09-15T01:33:26.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.219+0000 7fe0b259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b259c640 10 monclient(hunting): handle_auth_done global_id 4843 payload 824 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b259c640 1 -- 172.21.15.204:0/3039990568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0b409ddb0 msgr2=0x7fe0b411f460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b259c640 1 --2- 172.21.15.204:0/3039990568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0b409ddb0 0x7fe0b411f460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b2d9d640 1 --2- 172.21.15.204:0/3039990568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0b409f380 0x7fe0b411f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b259c640 1 -- 172.21.15.204:0/3039990568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0b409f380 msgr2=0x7fe0b411f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b259c640 1 --2- 172.21.15.204:0/3039990568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0b409f380 0x7fe0b411f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b259c640 10 monclient: _finish_hunting 0 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b259c640 1 monclient: found mon.noname-f 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b259c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b259c640 1 -- 172.21.15.204:0/3039990568 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac009a20 con 0x7fe0b4067130 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b2d9d640 10 monclient: get_auth_request con 0x7fe0b409f380 auth_method 0 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b2d9d640 1 --2- 172.21.15.204:0/3039990568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0b409f380 0x7fe0b411f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b259c640 1 --2- 172.21.15.204:0/3039990568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0b4067130 0x7fe0b4135990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400c4b0 tx=0x7fe0a4007d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0aaffd640 1 -- 172.21.15.204:0/3039990568 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0a40070d0 con 0x7fe0b4067130 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0aaffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0aaffd640 10 monclient: dump: 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.239 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0aaffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b8878640 5 monclient: authenticate success, global_id 4843 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0aaffd640 1 -- 172.21.15.204:0/3039990568 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a4007290 con 0x7fe0b4067130 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b8878640 10 monclient: _renew_subs 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b8878640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b8878640 1 -- 172.21.15.204:0/3039990568 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b4120ff0 con 0x7fe0b4067130 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0aaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:26.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0aaffd640 1 -- 172.21.15.204:0/3039990568 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0a4011360 con 0x7fe0b4067130 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0aaffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b159a640 4 set_mon_vals no callback set 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0aaffd640 10 monclient: dump: 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.241 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b8878640 10 monclient: _renew_subs 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b8878640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.220+0000 7fe0b8878640 1 -- 172.21.15.204:0/3039990568 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b4121bc0 con 0x7fe0b4067130 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.221+0000 7fe0aaffd640 1 -- 172.21.15.204:0/3039990568 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe0a4007400 con 0x7fe0b4067130 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.221+0000 7fe0b8878640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.221+0000 7fe0b8878640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.221+0000 7fe0b8878640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:26.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.221+0000 7fe0b8878640 1 -- 172.21.15.204:0/3039990568 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b4070b80 con 0x7fe0b4067130 2024-09-15T01:33:26.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.221+0000 7fe0aaffd640 1 --2- 172.21.15.204:0/3039990568 >> v2:172.21.15.204:6812/28210 conn(0x7fe0940397b0 0x7fe09403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.221+0000 7fe0aaffd640 1 -- 172.21.15.204:0/3039990568 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4630+0+0 (secure 0 0 0) 0x7fe0a404a350 con 0x7fe0b4067130 2024-09-15T01:33:26.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.221+0000 7fe0a9d9b640 1 --2- 172.21.15.204:0/3039990568 >> v2:172.21.15.204:6812/28210 conn(0x7fe0940397b0 0x7fe09403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.221+0000 7fe0a9d9b640 10 monclient: get_auth_request con 0x7fe0940397b0 auth_method 0 2024-09-15T01:33:26.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.221+0000 7fe0a9d9b640 1 --2- 172.21.15.204:0/3039990568 >> v2:172.21.15.204:6812/28210 conn(0x7fe0940397b0 0x7fe09403bc60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe09c006fd0 tx=0x7fe09c006cf0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.225+0000 7fe0aaffd640 1 -- 172.21.15.204:0/3039990568 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe0a400ea70 con 0x7fe0b4067130 2024-09-15T01:33:26.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.225+0000 7fe0aaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.225+0000 7fe0aaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:26.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.378+0000 7fe0b8878640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}] 2024-09-15T01:33:26.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.378+0000 7fe0b8878640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}] 2024-09-15T01:33:26.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.378+0000 7fe0b8878640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:26.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.378+0000 7fe0b8878640 1 -- 172.21.15.204:0/3039990568 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"} v 0) v1 -- 0x7fe0b406f750 con 0x7fe0b4067130 2024-09-15T01:33:26.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.379+0000 7fe0aaffd640 1 -- 172.21.15.204:0/3039990568 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}]=-22 pool size must be between 1 and 10 v40) v1 ==== 137+0+0 (secure 0 0 0) 0x7fe0a40170a0 con 0x7fe0b4067130 2024-09-15T01:33:26.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.379+0000 7fe0aaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}] 2024-09-15T01:33:26.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.379+0000 7fe0aaffd640 10 monclient: _finish_command 2 = mon:22 pool size must be between 1 and 10 2024-09-15T01:33:26.381 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: pool size must be between 1 and 10 2024-09-15T01:33:26.384 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:26.384 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool delete foo 2024-09-15T01:33:26.384 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:26.384 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete foo 2024-09-15T01:33:26.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.457+0000 7fa436e34640 10 monclient: get_monmap_and_config 2024-09-15T01:33:26.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.457+0000 7fa436e34640 10 monclient: build_initial_monmap 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.457+0000 7fa436e34640 10 monclient: monmap: 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:26.457522+0000 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:26.457522+0000 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:26.460 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.457+0000 7fa436e34640 1 Processor -- start 2024-09-15T01:33:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.457+0000 7fa436e34640 1 -- start start 2024-09-15T01:33:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.457+0000 7fa436e34640 10 monclient: init 2024-09-15T01:33:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 10 monclient: _add_conns ranks=[1,4,9,8,3,6,5,0,7,10,2] 2024-09-15T01:33:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa4300a2e30 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa43009fb80 0x7fa43009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 10 monclient(hunting): picked mon.noname-c con 0x7fa43009fb80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa430137d20 0x7fa43013a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 10 monclient(hunting): picked mon.noname-a con 0x7fa430137d20 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa43004b720 con 0x7fa4300a2e30 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43009e6c0 con 0x7fa430137d20 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4300a04b0 con 0x7fa43009fb80 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 10 monclient(hunting): authenticate will time out at 822.100322s 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa434bd0640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa4300a2e30 0x7fa430132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:33860/0 (socket says 172.21.15.204:33860) 2024-09-15T01:33:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa434bd0640 1 -- 172.21.15.204:0/938026331 learned_addr learned my addr 172.21.15.204:0/938026331 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42ffff640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa43009fb80 0x7fa43009ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42ffff640 10 monclient(hunting): get_auth_request con 0x7fa43009fb80 auth_method 0 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023742120 0 0) 0x7fa43004b720 con 0x7fa4300a2e30 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa4353d1640 1 --2- 172.21.15.204:0/938026331 >> v2:172.21.15.178:3300/0 conn(0x7fa430137d20 0x7fa43013a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 10 monclient(hunting): my global_id is 4852 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa41c0034e0 con 0x7fa4300a2e30 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa4353d1640 10 monclient(hunting): get_auth_request con 0x7fa430137d20 auth_method 0 2024-09-15T01:33:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa4353d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa4353d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa4353d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa4353d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa4353d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa4353d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4018312253 0 0) 0x7fa41c0034e0 con 0x7fa4300a2e30 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa43009fb80 msgr2=0x7fa43009ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa43009fb80 0x7fa43009ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 >> v2:172.21.15.178:3300/0 conn(0x7fa430137d20 msgr2=0x7fa43013a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 1 --2- 172.21.15.204:0/938026331 >> v2:172.21.15.178:3300/0 conn(0x7fa430137d20 0x7fa43013a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa430133170 con 0x7fa4300a2e30 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fa4280018f0 con 0x7fa4300a2e30 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 5 monclient: authenticate success, global_id 4852 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa436e34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42ffff640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa43009fb80 0x7fa43009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 10 monclient: dump: 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 >> v1:172.21.15.204:6789/0 conn(0x7fa4300a2e30 legacy=0x7fa430132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.458+0000 7fa42f7fe640 10 monclient: _add_conns ranks=[8,1,0,5,6,3,7,2,4] 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa41c004220 0x7fa41c004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 10 monclient(hunting): picked mon.e con 0x7fa41c004220 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa41c004d10 0x7fa41c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 10 monclient(hunting): picked mon.f con 0x7fa41c004d10 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa41c009ab0 0x7fa41c00bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 10 monclient(hunting): picked mon.a con 0x7fa41c009ab0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43009e6c0 con 0x7fa41c009ab0 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4300a04b0 con 0x7fa41c004d10 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c00c820 con 0x7fa41c004220 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42f7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa436e34640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa41c004d10 0x7fa41c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 10 monclient(hunting): get_auth_request con 0x7fa41c004d10 auth_method 0 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa434bd0640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa41c004220 0x7fa41c004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa4353d1640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa41c009ab0 0x7fa41c00bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa434bd0640 10 monclient(hunting): get_auth_request con 0x7fa41c004220 auth_method 0 2024-09-15T01:33:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa434bd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa434bd0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa434bd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa4353d1640 10 monclient(hunting): get_auth_request con 0x7fa41c009ab0 auth_method 0 2024-09-15T01:33:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa4353d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa4353d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa4353d1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa434bd0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa434bd0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa434bd0640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa4353d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa4353d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa4353d1640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 10 monclient(hunting): handle_auth_done global_id 4852 payload 293 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 1 -- 172.21.15.204:0/938026331 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa41c004220 msgr2=0x7fa41c004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa41c004220 0x7fa41c004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 1 -- 172.21.15.204:0/938026331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa41c009ab0 msgr2=0x7fa41c00bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa41c009ab0 0x7fa41c00bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 1 monclient: found mon.f 2024-09-15T01:33:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa4353d1640 1 -- 172.21.15.204:0/938026331 reap_dead start 2024-09-15T01:33:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 1 -- 172.21.15.204:0/938026331 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c0034e0 con 0x7fa41c004d10 2024-09-15T01:33:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 10 monclient: _finish_auth 0 2024-09-15T01:33:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.459+0000 7fa42ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa42ffff640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa41c004d10 0x7fa41c005180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa4300a2e00 tx=0x7fa4180090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f56e268a4f93d16c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa418031880 con 0x7fa41c004d10 2024-09-15T01:33:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa42f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa42f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa42f7fe640 10 monclient: dump: 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.481 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41802d050 con 0x7fa41c004d10 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa436e34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa42f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa436e34640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa436e34640 4 set_mon_vals no callback set 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa42f7fe640 1 -- 172.21.15.204:0/938026331 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa418031e60 con 0x7fa41c004d10 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa42f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa42f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa42f7fe640 10 monclient: dump: 2024-09-15T01:33:26.482 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.483 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa436e34640 10 monclient: shutdown 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa436e34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa436e34640 1 -- 172.21.15.204:0/938026331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa41c004d10 msgr2=0x7fa41c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa436e34640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa41c004d10 0x7fa41c005180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa4300a2e00 tx=0x7fa4180090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa436e34640 1 -- 172.21.15.204:0/938026331 shutdown_connections 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa436e34640 1 --2- 172.21.15.204:0/938026331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa41c004d10 0x7fa41c005180 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa436e34640 1 -- 172.21.15.204:0/938026331 >> 172.21.15.204:0/938026331 conn(0x7fa43006e860 msgr2=0x7fa43006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa436e34640 1 -- 172.21.15.204:0/938026331 shutdown_connections 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.460+0000 7fa436e34640 1 -- 172.21.15.204:0/938026331 wait complete. 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.461+0000 7fa436e34640 10 monclient: build_initial_monmap 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.461+0000 7fa436e34640 10 monclient: monmap: 2024-09-15T01:33:26.484 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:26.485 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:26.485 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:26.485 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:26.485 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:26.485 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:26.485 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:26.485 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:26.485 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:26.485 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:26.485 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:26.485 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:26.485 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.461+0000 7fa436e34640 1 Processor -- start 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.461+0000 7fa436e34640 1 -- start start 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.461+0000 7fa436e34640 10 monclient: init 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 10 monclient: _add_conns ranks=[8,2,4,7,0,1,5,6,3] 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa41c004d10 0x7fa430132b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 10 monclient(hunting): picked mon.noname-e con 0x7fa41c004d10 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa43011cbe0 0x7fa43009a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 10 monclient(hunting): picked mon.noname-b con 0x7fa43011cbe0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa43009ac70 0x7fa43009b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 10 monclient(hunting): picked mon.noname-c con 0x7fa43009ac70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43011dad0 con 0x7fa43011cbe0 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa43011e090 con 0x7fa43009ac70 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa43011e370 con 0x7fa41c004d10 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa436e34640 10 monclient(hunting): authenticate will time out at 822.104298s 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa434bd0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa41c004d10 0x7fa430132b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa43011cbe0 0x7fa43009a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa4353d1640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa43009ac70 0x7fa43009b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa43011cbe0 0x7fa43009a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:54256/0 (socket says 172.21.15.204:54256) 2024-09-15T01:33:26.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa434bd0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa41c004d10 0x7fa430132b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:49114/0 (socket says 172.21.15.204:49114) 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 1 -- 172.21.15.204:0/3331754828 learned_addr learned my addr 172.21.15.204:0/3331754828 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 10 monclient(hunting): get_auth_request con 0x7fa43011cbe0 auth_method 0 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa434bd0640 10 monclient(hunting): get_auth_request con 0x7fa41c004d10 auth_method 0 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa434bd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa434bd0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa434bd0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa4353d1640 10 monclient(hunting): get_auth_request con 0x7fa43009ac70 auth_method 0 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa4353d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa4353d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa4353d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa4353d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa4353d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa4353d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa434bd0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa434bd0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa434bd0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 10 monclient(hunting): handle_auth_done global_id 4673 payload 824 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 1 -- 172.21.15.204:0/3331754828 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa41c004d10 msgr2=0x7fa430132b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 1 --2- 172.21.15.204:0/3331754828 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa41c004d10 0x7fa430132b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 1 -- 172.21.15.204:0/3331754828 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa43009ac70 msgr2=0x7fa43009b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 1 --2- 172.21.15.204:0/3331754828 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa43009ac70 0x7fa43009b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 1 monclient: found mon.noname-b 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 1 -- 172.21.15.204:0/3331754828 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4301320b0 con 0x7fa43011cbe0 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa4353d1640 1 --2- 172.21.15.204:0/3331754828 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa43009ac70 0x7fa43009b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.462+0000 7fa42ffff640 1 --2- 172.21.15.204:0/3331754828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa43011cbe0 0x7fa43009a730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa418009840 tx=0x7fa418035000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 1 -- 172.21.15.204:0/3331754828 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa418030b30 con 0x7fa43011cbe0 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 10 monclient: dump: 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.490 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.491 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 1 -- 172.21.15.204:0/3331754828 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa418030c90 con 0x7fa43011cbe0 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa436e34640 5 monclient: authenticate success, global_id 4673 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa436e34640 10 monclient: _renew_subs 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa436e34640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa436e34640 1 -- 172.21.15.204:0/3331754828 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43011e790 con 0x7fa43011cbe0 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 1 -- 172.21.15.204:0/3331754828 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa418030e00 con 0x7fa43011cbe0 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 10 monclient: dump: 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.492 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.493 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42effd640 4 set_mon_vals no callback set 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa436e34640 10 monclient: _renew_subs 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa436e34640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa436e34640 1 -- 172.21.15.204:0/3331754828 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43009ca20 con 0x7fa43011cbe0 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.463+0000 7fa42cff9640 1 -- 172.21.15.204:0/3331754828 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa4180306b0 con 0x7fa43011cbe0 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.464+0000 7fa436e34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.464+0000 7fa436e34640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.464+0000 7fa436e34640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.464+0000 7fa436e34640 1 -- 172.21.15.204:0/3331754828 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43009c370 con 0x7fa43011cbe0 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.464+0000 7fa42cff9640 1 --2- 172.21.15.204:0/3331754828 >> v2:172.21.15.204:6812/28210 conn(0x7fa4100397b0 0x7fa41003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.464+0000 7fa42cff9640 1 -- 172.21.15.204:0/3331754828 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4630+0+0 (secure 0 0 0) 0x7fa41806c280 con 0x7fa43011cbe0 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.464+0000 7fa434bd0640 1 --2- 172.21.15.204:0/3331754828 >> v2:172.21.15.204:6812/28210 conn(0x7fa4100397b0 0x7fa41003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.464+0000 7fa434bd0640 10 monclient: get_auth_request con 0x7fa4100397b0 auth_method 0 2024-09-15T01:33:26.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.464+0000 7fa434bd0640 1 --2- 172.21.15.204:0/3331754828 >> v2:172.21.15.204:6812/28210 conn(0x7fa4100397b0 0x7fa41003bc60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa428000c00 tx=0x7fa4300a1df0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.468+0000 7fa42cff9640 1 -- 172.21.15.204:0/3331754828 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa418030960 con 0x7fa43011cbe0 2024-09-15T01:33:26.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.468+0000 7fa42cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.468+0000 7fa42cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:26.626 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.624+0000 7fa436e34640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo"}] 2024-09-15T01:33:26.626 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.624+0000 7fa436e34640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo"}] 2024-09-15T01:33:26.626 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.624+0000 7fa436e34640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:26.626 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.624+0000 7fa436e34640 1 -- 172.21.15.204:0/3331754828 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo"} v 0) v1 -- 0x7fa430070cc0 con 0x7fa43011cbe0 2024-09-15T01:33:26.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.625+0000 7fa42cff9640 1 -- 172.21.15.204:0/3331754828 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v40) v1 ==== 274+0+0 (secure 0 0 0) 0x7fa418039b50 con 0x7fa43011cbe0 2024-09-15T01:33:26.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.625+0000 7fa42cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo"}] 2024-09-15T01:33:26.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.625+0000 7fa42cff9640 10 monclient: _finish_command 2 = mon:1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-09-15T01:33:26.627 INFO:tasks.workunit.client.0.smithi204.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-09-15T01:33:26.630 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:26.630 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool delete foo foo 2024-09-15T01:33:26.630 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:26.631 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete foo foo 2024-09-15T01:33:26.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.703+0000 7eff7af1d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:26.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.703+0000 7eff7af1d640 10 monclient: build_initial_monmap 2024-09-15T01:33:26.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.703+0000 7eff7af1d640 10 monclient: monmap: 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:26.704106+0000 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:26.704106+0000 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:26.706 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:26.707 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:26.707 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:26.707 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:26.707 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:26.707 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 1 Processor -- start 2024-09-15T01:33:26.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 1 -- start start 2024-09-15T01:33:26.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 10 monclient: init 2024-09-15T01:33:26.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:26.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 10 monclient: _add_conns ranks=[5,7,2,4,3,0,1,10,9,6,8] 2024-09-15T01:33:26.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7eff740a2e30 0x7eff74132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 10 monclient(hunting): picked mon.noname-h con 0x7eff740a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7eff7409fb80 0x7eff7409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 10 monclient(hunting): picked mon.noname-i con 0x7eff7409fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff740a0a40 0x7eff740a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 10 monclient(hunting): picked mon.noname-b con 0x7eff740a0a40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7409e6c0 con 0x7eff740a0a40 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff740a04b0 con 0x7eff740a2e30 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7eff740a3200 con 0x7eff7409fb80 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff7af1d640 10 monclient(hunting): authenticate will time out at 822.346863s 2024-09-15T01:33:26.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff6bfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7eff7409fb80 0x7eff7409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.704+0000 7eff78cb9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7eff740a2e30 0x7eff74132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7eff7409fb80 0x7eff7409ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:51676/0 (socket says 172.21.15.204:51676) 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 1 -- 172.21.15.204:0/1044459529 learned_addr learned my addr 172.21.15.204:0/1044459529 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 10 monclient(hunting): get_auth_request con 0x7eff7409fb80 auth_method 0 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff78cb9640 10 monclient(hunting): get_auth_request con 0x7eff740a2e30 auth_method 0 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff78cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff78cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff78cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff78cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff78cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff78cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff794ba640 1 --2- 172.21.15.204:0/1044459529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff740a0a40 0x7eff740a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff794ba640 10 monclient(hunting): get_auth_request con 0x7eff740a0a40 auth_method 0 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff794ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff794ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff794ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 10 monclient(hunting): handle_auth_done global_id 4723 payload 274 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 1 -- 172.21.15.204:0/1044459529 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7eff740a2e30 msgr2=0x7eff74132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 1 --2- 172.21.15.204:0/1044459529 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7eff740a2e30 0x7eff74132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 1 -- 172.21.15.204:0/1044459529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff740a0a40 msgr2=0x7eff740a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:26.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 1 --2- 172.21.15.204:0/1044459529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff740a0a40 0x7eff740a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 1 monclient: found mon.noname-i 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 1 -- 172.21.15.204:0/1044459529 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7404b720 con 0x7eff7409fb80 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff78cb9640 1 --2- 172.21.15.204:0/1044459529 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7eff740a2e30 0x7eff74132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff794ba640 1 --2- 172.21.15.204:0/1044459529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff740a0a40 0x7eff740a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.705+0000 7eff6bfff640 1 --2- 172.21.15.204:0/1044459529 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7eff7409fb80 0x7eff7409ff70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff64009940 tx=0x7eff640091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=2b91531e758b524e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff6b7fe640 1 -- 172.21.15.204:0/1044459529 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff6402ac60 con 0x7eff7409fb80 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff6b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff6b7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff6b7fe640 10 monclient: dump: 2024-09-15T01:33:26.718 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.719 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff6b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff6b7fe640 1 -- 172.21.15.204:0/1044459529 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6402ade0 con 0x7eff7409fb80 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff6b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff6b7fe640 1 -- 172.21.15.204:0/1044459529 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff64031ca0 con 0x7eff7409fb80 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff6b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff6b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff6b7fe640 10 monclient: dump: 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.720 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 5 monclient: authenticate success, global_id 4723 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 4 set_mon_vals no callback set 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 10 monclient: shutdown 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 1 -- 172.21.15.204:0/1044459529 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7eff7409fb80 msgr2=0x7eff7409ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 1 --2- 172.21.15.204:0/1044459529 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7eff7409fb80 0x7eff7409ff70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff64009940 tx=0x7eff640091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 1 -- 172.21.15.204:0/1044459529 shutdown_connections 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 1 --2- 172.21.15.204:0/1044459529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff740a0a40 0x7eff740a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 1 --2- 172.21.15.204:0/1044459529 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7eff7409fb80 0x7eff7409ff70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 1 --2- 172.21.15.204:0/1044459529 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7eff740a2e30 0x7eff74132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 1 -- 172.21.15.204:0/1044459529 >> 172.21.15.204:0/1044459529 conn(0x7eff7406e860 msgr2=0x7eff7406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 1 -- 172.21.15.204:0/1044459529 shutdown_connections 2024-09-15T01:33:26.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.706+0000 7eff7af1d640 1 -- 172.21.15.204:0/1044459529 wait complete. 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.707+0000 7eff7af1d640 10 monclient: build_initial_monmap 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.707+0000 7eff7af1d640 10 monclient: monmap: 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:26.723 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.707+0000 7eff7af1d640 1 Processor -- start 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.707+0000 7eff7af1d640 1 -- start start 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.707+0000 7eff7af1d640 10 monclient: init 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 10 monclient: _add_conns ranks=[5,6,8,3,1,2,4,7,0] 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7eff7409fb80 0x7eff741306f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 10 monclient(hunting): picked mon.noname-h con 0x7eff7409fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7eff740a0a40 0x7eff7411d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 10 monclient(hunting): picked mon.noname-d con 0x7eff740a0a40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:26.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7eff740a2e30 0x7eff7411d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 10 monclient(hunting): picked mon.noname-e con 0x7eff740a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff741210e0 con 0x7eff7409fb80 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7eff74120b20 con 0x7eff740a0a40 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7eff74120e00 con 0x7eff740a2e30 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 10 monclient(hunting): authenticate will time out at 822.350330s 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7eff7409fb80 0x7eff741306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7eff7409fb80 0x7eff741306f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:38202/0 (socket says 172.21.15.204:38202) 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 1 -- 172.21.15.204:0/1327271835 learned_addr learned my addr 172.21.15.204:0/1327271835 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:26.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 10 monclient(hunting): get_auth_request con 0x7eff7409fb80 auth_method 0 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff6bfff640 1 -- 172.21.15.204:0/1327271835 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7eff740a0a40 msgr2=0x7eff7411d250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff6bfff640 1 --2- 172.21.15.204:0/1327271835 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7eff740a0a40 0x7eff7411d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 10 monclient(hunting): handle_auth_done global_id 4739 payload 824 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 1 -- 172.21.15.204:0/1327271835 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7eff740a2e30 msgr2=0x7eff7411d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff794ba640 1 --2- 172.21.15.204:0/1327271835 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7eff740a2e30 0x7eff7411d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 1 --2- 172.21.15.204:0/1327271835 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7eff740a2e30 0x7eff7411d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 1 -- 172.21.15.204:0/1327271835 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7eff740a0a40 msgr2=0x7eff7411d250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:26.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 1 --2- 172.21.15.204:0/1327271835 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7eff740a0a40 0x7eff7411d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 10 monclient: _finish_hunting 0 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 1 monclient: found mon.noname-h 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 1 -- 172.21.15.204:0/1327271835 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff70009390 con 0x7eff7409fb80 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff78cb9640 1 --2- 172.21.15.204:0/1327271835 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7eff7409fb80 0x7eff741306f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7eff7000ea70 tx=0x7eff700090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff68ff9640 1 -- 172.21.15.204:0/1327271835 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff7000ce90 con 0x7eff7409fb80 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff68ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff68ff9640 10 monclient: dump: 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.727 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff68ff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff68ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff68ff9640 1 -- 172.21.15.204:0/1327271835 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff70013470 con 0x7eff7409fb80 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 5 monclient: authenticate success, global_id 4739 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 10 monclient: _renew_subs 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.708+0000 7eff7af1d640 1 -- 172.21.15.204:0/1327271835 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff64009450 con 0x7eff7409fb80 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.709+0000 7eff68ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.709+0000 7eff68ff9640 1 -- 172.21.15.204:0/1327271835 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff70013a60 con 0x7eff7409fb80 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.709+0000 7eff68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.709+0000 7eff68ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.709+0000 7eff68ff9640 10 monclient: dump: 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.729 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.709+0000 7eff6affd640 4 set_mon_vals no callback set 2024-09-15T01:33:26.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.709+0000 7eff7af1d640 10 monclient: _renew_subs 2024-09-15T01:33:26.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.709+0000 7eff7af1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:26.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.709+0000 7eff7af1d640 1 -- 172.21.15.204:0/1327271835 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff74120840 con 0x7eff7409fb80 2024-09-15T01:33:26.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.709+0000 7eff68ff9640 1 -- 172.21.15.204:0/1327271835 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7eff70013bc0 con 0x7eff7409fb80 2024-09-15T01:33:26.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.710+0000 7eff68ff9640 1 --2- 172.21.15.204:0/1327271835 >> v2:172.21.15.204:6812/28210 conn(0x7eff600397e0 0x7eff6003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.710+0000 7eff7af1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.710+0000 7eff68ff9640 1 -- 172.21.15.204:0/1327271835 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4630+0+0 (secure 0 0 0) 0x7eff7004ba30 con 0x7eff7409fb80 2024-09-15T01:33:26.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.710+0000 7eff7af1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.710+0000 7eff7af1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:26.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.710+0000 7eff7af1d640 1 -- 172.21.15.204:0/1327271835 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff74070cc0 con 0x7eff7409fb80 2024-09-15T01:33:26.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.710+0000 7eff6bfff640 1 --2- 172.21.15.204:0/1327271835 >> v2:172.21.15.204:6812/28210 conn(0x7eff600397e0 0x7eff6003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.710+0000 7eff6bfff640 10 monclient: get_auth_request con 0x7eff600397e0 auth_method 0 2024-09-15T01:33:26.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.710+0000 7eff6bfff640 1 --2- 172.21.15.204:0/1327271835 >> v2:172.21.15.204:6812/28210 conn(0x7eff600397e0 0x7eff6003bc90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff64034000 tx=0x7eff64032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.714+0000 7eff68ff9640 1 -- 172.21.15.204:0/1327271835 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7eff7000ca10 con 0x7eff7409fb80 2024-09-15T01:33:26.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.714+0000 7eff68ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.714+0000 7eff68ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:26.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.870+0000 7eff7af1d640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}] 2024-09-15T01:33:26.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.870+0000 7eff7af1d640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}] 2024-09-15T01:33:26.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.870+0000 7eff7af1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:26.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.870+0000 7eff7af1d640 1 -- 172.21.15.204:0/1327271835 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"} v 0) v1 -- 0x7eff740a1f10 con 0x7eff7409fb80 2024-09-15T01:33:26.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.870+0000 7eff68ff9640 1 -- 172.21.15.204:0/1327271835 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v40) v1 ==== 290+0+0 (secure 0 0 0) 0x7eff7001b3b0 con 0x7eff7409fb80 2024-09-15T01:33:26.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.870+0000 7eff68ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}] 2024-09-15T01:33:26.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.870+0000 7eff68ff9640 10 monclient: _finish_command 2 = mon:1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-09-15T01:33:26.872 INFO:tasks.workunit.client.0.smithi204.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-09-15T01:33:26.875 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:26.875 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool delete foo foo --force 2024-09-15T01:33:26.875 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:26.875 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete foo foo --force 2024-09-15T01:33:26.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.949+0000 7f03d6cdd640 10 monclient: get_monmap_and_config 2024-09-15T01:33:26.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.949+0000 7f03d6cdd640 10 monclient: build_initial_monmap 2024-09-15T01:33:26.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.949+0000 7f03d6cdd640 10 monclient: monmap: 2024-09-15T01:33:26.951 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:26.951 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.951 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:26.950376+0000 2024-09-15T01:33:26.951 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:26.950376+0000 2024-09-15T01:33:26.951 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:26.951 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.952 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:26.952 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:26.952 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:26.952 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:26.952 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:26.952 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:26.952 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:26.952 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:26.952 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:26.952 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:26.952 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:26.952 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.950+0000 7f03d6cdd640 1 Processor -- start 2024-09-15T01:33:26.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.950+0000 7f03d6cdd640 1 -- start start 2024-09-15T01:33:26.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.950+0000 7f03d6cdd640 10 monclient: init 2024-09-15T01:33:26.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:26.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 10 monclient: _add_conns ranks=[7,8,9,4,5,6,2,0,1,10,3] 2024-09-15T01:33:26.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f03d00a2cd0 0x7f03d0132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 10 monclient(hunting): picked mon.noname-i con 0x7f03d00a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:26.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f03d009fa20 0x7f03d009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 10 monclient(hunting): picked mon.noname-e con 0x7f03d009fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:26.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f03d00a08e0 0x7f03d00a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 10 monclient(hunting): picked mon.noname-a con 0x7f03d00a08e0 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:26.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d00a30a0 con 0x7f03d00a08e0 2024-09-15T01:33:26.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f03d00a0330 con 0x7f03d00a2cd0 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f03d00a1db0 con 0x7f03d009fa20 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d6cdd640 10 monclient(hunting): authenticate will time out at 822.593177s 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f03d00a2cd0 0x7f03d0132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f03d00a2cd0 0x7f03d0132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:51690/0 (socket says 172.21.15.204:51690) 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 1 -- 172.21.15.204:0/1919660616 learned_addr learned my addr 172.21.15.204:0/1919660616 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 10 monclient(hunting): get_auth_request con 0x7f03d00a2cd0 auth_method 0 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03cffff640 1 --2- 172.21.15.204:0/1919660616 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f03d009fa20 0x7f03d009fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d527a640 1 --2- 172.21.15.204:0/1919660616 >> v2:172.21.15.178:3300/0 conn(0x7f03d00a08e0 0x7f03d00a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 10 monclient(hunting): handle_auth_done global_id 4732 payload 274 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 1 -- 172.21.15.204:0/1919660616 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f03d009fa20 msgr2=0x7f03d009fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 1 --2- 172.21.15.204:0/1919660616 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f03d009fa20 0x7f03d009fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 1 -- 172.21.15.204:0/1919660616 >> v2:172.21.15.178:3300/0 conn(0x7f03d00a08e0 msgr2=0x7f03d00a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 1 --2- 172.21.15.204:0/1919660616 >> v2:172.21.15.178:3300/0 conn(0x7f03d00a08e0 0x7f03d00a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 10 monclient: _finish_hunting 0 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 1 monclient: found mon.noname-i 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d4a79640 1 -- 172.21.15.204:0/1919660616 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d00648a0 con 0x7f03d00a2cd0 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03cffff640 10 monclient: get_auth_request con 0x7f03d009fa20 auth_method 0 2024-09-15T01:33:26.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03cffff640 1 --2- 172.21.15.204:0/1919660616 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f03d009fa20 0x7f03d009fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d527a640 10 monclient: get_auth_request con 0x7f03d00a08e0 auth_method 0 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.951+0000 7f03d527a640 1 --2- 172.21.15.204:0/1919660616 >> v2:172.21.15.178:3300/0 conn(0x7f03d00a08e0 0x7f03d00a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03d4a79640 1 --2- 172.21.15.204:0/1919660616 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f03d00a2cd0 0x7f03d0132c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009910 tx=0x7f03c00091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=ab2207ac61c63a18 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03cf7fe640 1 -- 172.21.15.204:0/1919660616 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03c002ac80 con 0x7f03d00a2cd0 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03cf7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03cf7fe640 10 monclient: dump: 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.964 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03cf7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03cf7fe640 1 -- 172.21.15.204:0/1919660616 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c002ae00 con 0x7f03d00a2cd0 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03d6cdd640 5 monclient: authenticate success, global_id 4732 2024-09-15T01:33:26.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03d6cdd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03cf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03d6cdd640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03cf7fe640 1 -- 172.21.15.204:0/1919660616 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03c0031ca0 con 0x7f03d00a2cd0 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03d6cdd640 4 set_mon_vals no callback set 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03cf7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03cf7fe640 10 monclient: dump: 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.966 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03d6cdd640 10 monclient: shutdown 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03d6cdd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03d6cdd640 1 -- 172.21.15.204:0/1919660616 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f03d00a2cd0 msgr2=0x7f03d0132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:26.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.952+0000 7f03d6cdd640 1 --2- 172.21.15.204:0/1919660616 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f03d00a2cd0 0x7f03d0132c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009910 tx=0x7f03c00091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.953+0000 7f03d6cdd640 1 -- 172.21.15.204:0/1919660616 shutdown_connections 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.953+0000 7f03d6cdd640 1 --2- 172.21.15.204:0/1919660616 >> v2:172.21.15.178:3300/0 conn(0x7f03d00a08e0 0x7f03d00a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.953+0000 7f03d6cdd640 1 --2- 172.21.15.204:0/1919660616 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f03d009fa20 0x7f03d009fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.953+0000 7f03d6cdd640 1 --2- 172.21.15.204:0/1919660616 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f03d00a2cd0 0x7f03d0132c00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.953+0000 7f03d6cdd640 1 -- 172.21.15.204:0/1919660616 >> 172.21.15.204:0/1919660616 conn(0x7f03d006e860 msgr2=0x7f03d006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.953+0000 7f03d6cdd640 1 -- 172.21.15.204:0/1919660616 shutdown_connections 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.953+0000 7f03d6cdd640 1 -- 172.21.15.204:0/1919660616 wait complete. 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.953+0000 7f03d6cdd640 10 monclient: build_initial_monmap 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.953+0000 7f03d6cdd640 10 monclient: monmap: 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:26.968 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.953+0000 7f03d6cdd640 1 Processor -- start 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 1 -- start start 2024-09-15T01:33:26.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 10 monclient: init 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 10 monclient: _add_conns ranks=[7,2,3,6,8,5,4,1,0] 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f03d009fa20 0x7f03d0130e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 10 monclient(hunting): picked mon.noname-i con 0x7f03d009fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03d00a08e0 0x7f03d011d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 10 monclient(hunting): picked mon.noname-b con 0x7f03d00a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d00a2cd0 0x7f03d011dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 10 monclient(hunting): picked mon.noname-g con 0x7f03d00a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d009d3d0 con 0x7f03d00a08e0 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d009cb60 con 0x7f03d00a2cd0 2024-09-15T01:33:26.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:26.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f03d011f450 con 0x7f03d009fa20 2024-09-15T01:33:26.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:26.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d6cdd640 10 monclient(hunting): authenticate will time out at 822.596616s 2024-09-15T01:33:26.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d527a640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d00a2cd0 msgr2=0x7f03d011dea0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:26.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d527a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d00a2cd0 0x7f03d011dea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:26.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f03d009fa20 0x7f03d0130e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f03d009fa20 0x7f03d0130e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:51692/0 (socket says 172.21.15.204:51692) 2024-09-15T01:33:26.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 1 -- 172.21.15.204:0/3702614053 learned_addr learned my addr 172.21.15.204:0/3702614053 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:26.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 10 monclient(hunting): get_auth_request con 0x7f03d009fa20 auth_method 0 2024-09-15T01:33:26.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03cffff640 1 --2- 172.21.15.204:0/3702614053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03d00a08e0 0x7f03d011d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03cffff640 10 monclient(hunting): get_auth_request con 0x7f03d00a08e0 auth_method 0 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 10 monclient(hunting): handle_auth_done global_id 4741 payload 824 2024-09-15T01:33:26.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 1 -- 172.21.15.204:0/3702614053 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d00a2cd0 msgr2=0x7f03d011dea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 1 --2- 172.21.15.204:0/3702614053 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d00a2cd0 0x7f03d011dea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 1 -- 172.21.15.204:0/3702614053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03d00a08e0 msgr2=0x7f03d011d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 1 --2- 172.21.15.204:0/3702614053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03d00a08e0 0x7f03d011d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 10 monclient: _finish_hunting 0 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 1 monclient: found mon.noname-i 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.954+0000 7f03d4a79640 1 -- 172.21.15.204:0/3702614053 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c0009450 con 0x7f03d009fa20 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03d4a79640 1 --2- 172.21.15.204:0/3702614053 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f03d009fa20 0x7f03d0130e30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f03c0000c00 tx=0x7f03c0032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ccff9640 1 -- 172.21.15.204:0/3702614053 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03c002ac80 con 0x7f03d009fa20 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ccff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ccff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ccff9640 10 monclient: dump: 2024-09-15T01:33:26.973 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.974 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ccff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ccff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ccff9640 1 -- 172.21.15.204:0/3702614053 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c0033e50 con 0x7f03d009fa20 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03d6cdd640 5 monclient: authenticate success, global_id 4741 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03d6cdd640 10 monclient: _renew_subs 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03d6cdd640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03d6cdd640 1 -- 172.21.15.204:0/3702614053 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d0120f80 con 0x7f03d009fa20 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ccff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ccff9640 1 -- 172.21.15.204:0/3702614053 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03c003a810 con 0x7f03d009fa20 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ccff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ccff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ccff9640 10 monclient: dump: 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:26.975 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:26.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03ceffd640 4 set_mon_vals no callback set 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03d6cdd640 10 monclient: _renew_subs 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03d6cdd640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.955+0000 7f03d6cdd640 1 -- 172.21.15.204:0/3702614053 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d011fb50 con 0x7f03d009fa20 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.956+0000 7f03ccff9640 1 -- 172.21.15.204:0/3702614053 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f03c0039040 con 0x7f03d009fa20 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.956+0000 7f03d6cdd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.956+0000 7f03d6cdd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.956+0000 7f03d6cdd640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.956+0000 7f03d6cdd640 1 -- 172.21.15.204:0/3702614053 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d0070b80 con 0x7f03d009fa20 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.956+0000 7f03ccff9640 1 --2- 172.21.15.204:0/3702614053 >> v2:172.21.15.204:6812/28210 conn(0x7f03bc0397b0 0x7f03bc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.956+0000 7f03ccff9640 1 -- 172.21.15.204:0/3702614053 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4630+0+0 (secure 0 0 0) 0x7f03c006aa60 con 0x7f03d009fa20 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.956+0000 7f03cffff640 1 --2- 172.21.15.204:0/3702614053 >> v2:172.21.15.204:6812/28210 conn(0x7f03bc0397b0 0x7f03bc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.956+0000 7f03cffff640 10 monclient: get_auth_request con 0x7f03bc0397b0 auth_method 0 2024-09-15T01:33:26.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.956+0000 7f03cffff640 1 --2- 172.21.15.204:0/3702614053 >> v2:172.21.15.204:6812/28210 conn(0x7f03bc0397b0 0x7f03bc03bc60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f03c400ab90 tx=0x7f03c4008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:26.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.960+0000 7f03ccff9640 1 -- 172.21.15.204:0/3702614053 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f03c0039860 con 0x7f03d009fa20 2024-09-15T01:33:26.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.960+0000 7f03ccff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:26.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:26.960+0000 7f03ccff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:27.116 INFO:tasks.workunit.client.0.smithi204.stderr:--force not valid: --force not one of 'true', 'false' 2024-09-15T01:33:27.116 INFO:tasks.workunit.client.0.smithi204.stderr:Invalid command: unused arguments: ['--force'] 2024-09-15T01:33:27.116 INFO:tasks.workunit.client.0.smithi204.stderr:osd pool delete [] [--yes-i-really-really-mean-it] [--yes-i-really-really-mean-it-not-faking] : delete pool 2024-09-15T01:33:27.116 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: invalid command 2024-09-15T01:33:27.118 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:27.118 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool delete foo fooo --yes-i-really-mean-it 2024-09-15T01:33:27.118 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:27.118 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete foo fooo --yes-i-really-mean-it 2024-09-15T01:33:27.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.191+0000 7f517f19d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:27.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.191+0000 7f517f19d640 10 monclient: build_initial_monmap 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.191+0000 7f517f19d640 10 monclient: monmap: 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:27.192196+0000 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:27.192196+0000 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:27.194 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:27.195 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:27.195 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:27.195 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:27.195 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:27.195 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:27.195 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.203 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 1 Processor -- start 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 1 -- start start 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 10 monclient: init 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 10 monclient: _add_conns ranks=[10,1,0,4,7,5,2,8,3,6,9] 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f51780a2cd0 0x7f5178132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 10 monclient(hunting): picked mon.noname-f con 0x7f51780a2cd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f517809fa20 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5178137c80 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51780648a0 con 0x7f5178137c80 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51780a30a0 con 0x7f517809fa20 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51780a1db0 con 0x7f51780a2cd0 2024-09-15T01:33:27.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:27.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.192+0000 7f517f19d640 10 monclient(hunting): authenticate will time out at 822.834983s 2024-09-15T01:33:27.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f51780a2cd0 0x7f5178132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f51780a2cd0 0x7f5178132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33180/0 (socket says 172.21.15.204:33180) 2024-09-15T01:33:27.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f516ffff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f517809fa20 0x7f517809fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:58924/0 (socket says 172.21.15.204:58924) 2024-09-15T01:33:27.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 1 -- 172.21.15.204:0/3324433841 learned_addr learned my addr 172.21.15.204:0/3324433841 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:27.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 10 monclient(hunting): get_auth_request con 0x7f51780a2cd0 auth_method 0 2024-09-15T01:33:27.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:27.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:27.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:27.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:27.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:27.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:27.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 10 monclient(hunting): handle_auth_done global_id 4861 payload 274 2024-09-15T01:33:27.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 1 -- 172.21.15.204:0/3324433841 >> v1:172.21.15.204:6789/0 conn(0x7f517809fa20 legacy=0x7f517809fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:27.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 1 -- 172.21.15.204:0/3324433841 >> v1:172.21.15.178:6789/0 conn(0x7f5178137c80 legacy=0x7f517813a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:27.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 10 monclient: _finish_hunting 0 2024-09-15T01:33:27.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 1 monclient: found mon.noname-f 2024-09-15T01:33:27.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:27.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:27.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.193+0000 7f517cf39640 1 -- 172.21.15.204:0/3324433841 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517809dc50 con 0x7f51780a2cd0 2024-09-15T01:33:27.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517cf39640 1 --2- 172.21.15.204:0/3324433841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51780a2cd0 0x7f5178132c00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5174009910 tx=0x7f51740091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5c05e2339cacc82 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:27.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f516f7fe640 1 -- 172.21.15.204:0/3324433841 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f517402add0 con 0x7f51780a2cd0 2024-09-15T01:33:27.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f516f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:27.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f516f7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:27.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f516f7fe640 10 monclient: dump: 2024-09-15T01:33:27.208 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:27.208 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.208 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.208 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.209 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:27.209 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.209 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:27.209 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:27.209 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:27.209 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:27.209 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:27.210 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:27.210 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:27.210 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:27.210 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:27.210 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.210 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f516f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:27.210 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f516f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:27.210 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f516f7fe640 1 -- 172.21.15.204:0/3324433841 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5174031820 con 0x7f51780a2cd0 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 5 monclient: authenticate success, global_id 4861 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f516f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f516f7fe640 1 -- 172.21.15.204:0/3324433841 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5174031e90 con 0x7f51780a2cd0 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f516f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 4 set_mon_vals no callback set 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f516f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f516f7fe640 10 monclient: dump: 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.211 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 10 monclient: shutdown 2024-09-15T01:33:27.212 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:27.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 1 -- 172.21.15.204:0/3324433841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51780a2cd0 msgr2=0x7f5178132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:27.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 1 --2- 172.21.15.204:0/3324433841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51780a2cd0 0x7f5178132c00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5174009910 tx=0x7f51740091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 1 -- 172.21.15.204:0/3324433841 shutdown_connections 2024-09-15T01:33:27.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 1 --2- 172.21.15.204:0/3324433841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51780a2cd0 0x7f5178132c00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 1 -- 172.21.15.204:0/3324433841 >> 172.21.15.204:0/3324433841 conn(0x7f517806e860 msgr2=0x7f517806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:27.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 1 -- 172.21.15.204:0/3324433841 shutdown_connections 2024-09-15T01:33:27.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.194+0000 7f517f19d640 1 -- 172.21.15.204:0/3324433841 wait complete. 2024-09-15T01:33:27.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.195+0000 7f517f19d640 10 monclient: build_initial_monmap 2024-09-15T01:33:27.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.195+0000 7f517f19d640 10 monclient: monmap: 2024-09-15T01:33:27.213 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:27.213 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:27.213 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.214 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.195+0000 7f517f19d640 1 Processor -- start 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.195+0000 7f517f19d640 1 -- start start 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.195+0000 7f517f19d640 10 monclient: init 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 10 monclient: _add_conns ranks=[7,5,8,1,0,4,6,3,2] 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f517809fa20 0x7f51780997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 10 monclient(hunting): picked mon.noname-i con 0x7f517809fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f51780a2cd0 0x7f5178099ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 10 monclient(hunting): picked mon.noname-h con 0x7f51780a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5178137c80 0x7f517809a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 10 monclient(hunting): picked mon.noname-e con 0x7f5178137c80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5178130d30 con 0x7f51780a2cd0 2024-09-15T01:33:27.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f517809cb60 con 0x7f517809fa20 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f517811c6c0 con 0x7f5178137c80 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517f19d640 10 monclient(hunting): authenticate will time out at 822.838378s 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517cf39640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f517809fa20 0x7f51780997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f51780a2cd0 0x7f5178099ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517cf39640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f517809fa20 0x7f51780997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:45640/0 (socket says 172.21.15.204:45640) 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517cf39640 1 -- 172.21.15.204:0/2744614886 learned_addr learned my addr 172.21.15.204:0/2744614886 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517cf39640 10 monclient(hunting): get_auth_request con 0x7f517809fa20 auth_method 0 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517cf39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517cf39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517cf39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:27.216 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 10 monclient(hunting): get_auth_request con 0x7f51780a2cd0 auth_method 0 2024-09-15T01:33:27.217 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:27.217 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:27.217 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:27.217 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517cf39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:27.217 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517cf39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:27.217 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517cf39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:27.217 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:27.217 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:27.217 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:27.217 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517d73a640 1 --2- 172.21.15.204:0/2744614886 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5178137c80 0x7f517809a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517d73a640 10 monclient(hunting): get_auth_request con 0x7f5178137c80 auth_method 0 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517d73a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517d73a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517d73a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 10 monclient(hunting): handle_auth_done global_id 4748 payload 824 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 1 -- 172.21.15.204:0/2744614886 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5178137c80 msgr2=0x7f517809a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 1 --2- 172.21.15.204:0/2744614886 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5178137c80 0x7f517809a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 1 -- 172.21.15.204:0/2744614886 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f517809fa20 msgr2=0x7f51780997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 1 --2- 172.21.15.204:0/2744614886 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f517809fa20 0x7f51780997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 1 monclient: found mon.noname-h 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:27.218 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 1 -- 172.21.15.204:0/2744614886 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5174009450 con 0x7f51780a2cd0 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517d73a640 1 --2- 172.21.15.204:0/2744614886 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5178137c80 0x7f517809a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f517cf39640 1 --2- 172.21.15.204:0/2744614886 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f517809fa20 0x7f51780997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.196+0000 7f516ffff640 1 --2- 172.21.15.204:0/2744614886 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f51780a2cd0 0x7f5178099ce0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f516800c470 tx=0x7f5168007e40 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 1 -- 172.21.15.204:0/2744614886 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5168000fe0 con 0x7f51780a2cd0 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 10 monclient: dump: 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:27.219 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f517f19d640 5 monclient: authenticate success, global_id 4748 2024-09-15T01:33:27.220 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 1 -- 172.21.15.204:0/2744614886 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51680047a0 con 0x7f51780a2cd0 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 1 -- 172.21.15.204:0/2744614886 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5168004d90 con 0x7f51780a2cd0 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f517f19d640 10 monclient: _renew_subs 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f517f19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f517f19d640 1 -- 172.21.15.204:0/2744614886 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517809d3d0 con 0x7f51780a2cd0 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516effd640 4 set_mon_vals no callback set 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 10 monclient: dump: 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.221 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f517f19d640 10 monclient: _renew_subs 2024-09-15T01:33:27.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f517f19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f517f19d640 1 -- 172.21.15.204:0/2744614886 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517811cc80 con 0x7f51780a2cd0 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.197+0000 7f516cff9640 1 -- 172.21.15.204:0/2744614886 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5168004910 con 0x7f51780a2cd0 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.198+0000 7f517f19d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.198+0000 7f517f19d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.198+0000 7f517f19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.198+0000 7f517f19d640 1 -- 172.21.15.204:0/2744614886 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517811c8e0 con 0x7f51780a2cd0 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.198+0000 7f516cff9640 1 --2- 172.21.15.204:0/2744614886 >> v2:172.21.15.204:6812/28210 conn(0x7f51640397b0 0x7f516403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.198+0000 7f516cff9640 1 -- 172.21.15.204:0/2744614886 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4630+0+0 (secure 0 0 0) 0x7f516804a9a0 con 0x7f51780a2cd0 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.198+0000 7f517cf39640 1 --2- 172.21.15.204:0/2744614886 >> v2:172.21.15.204:6812/28210 conn(0x7f51640397b0 0x7f516403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.198+0000 7f517cf39640 10 monclient: get_auth_request con 0x7f51640397b0 auth_method 0 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.198+0000 7f517cf39640 1 --2- 172.21.15.204:0/2744614886 >> v2:172.21.15.204:6812/28210 conn(0x7f51640397b0 0x7f516403bc60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5174009910 tx=0x7f5174032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.202+0000 7f516cff9640 1 -- 172.21.15.204:0/2744614886 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f517811c8e0 con 0x7f51780a2cd0 2024-09-15T01:33:27.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.202+0000 7f516cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:27.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.202+0000 7f516cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:27.356 INFO:tasks.workunit.client.0.smithi204.stderr:--yes-i-really-mean-it not valid: --yes-i-really-mean-it not one of 'true', 'false' 2024-09-15T01:33:27.357 INFO:tasks.workunit.client.0.smithi204.stderr:Invalid command: unused arguments: ['--yes-i-really-mean-it'] 2024-09-15T01:33:27.357 INFO:tasks.workunit.client.0.smithi204.stderr:osd pool delete [] [--yes-i-really-really-mean-it] [--yes-i-really-really-mean-it-not-faking] : delete pool 2024-09-15T01:33:27.357 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: invalid command 2024-09-15T01:33:27.358 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:27.359 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd pool delete foo --yes-i-really-mean-it foo 2024-09-15T01:33:27.359 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:27.359 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete foo --yes-i-really-mean-it foo 2024-09-15T01:33:27.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.433+0000 7f663739d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:27.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.433+0000 7f663739d640 10 monclient: build_initial_monmap 2024-09-15T01:33:27.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.433+0000 7f663739d640 10 monclient: monmap: 2024-09-15T01:33:27.435 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:27.435 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.435 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:27.434010+0000 2024-09-15T01:33:27.435 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:27.434010+0000 2024-09-15T01:33:27.435 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:27.436 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 1 Processor -- start 2024-09-15T01:33:27.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 1 -- start start 2024-09-15T01:33:27.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 10 monclient: init 2024-09-15T01:33:27.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:27.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 10 monclient: _add_conns ranks=[3,7,8,9,6,0,2,1,10,4,5] 2024-09-15T01:33:27.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66300a2cd0 0x7f6630132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 10 monclient(hunting): picked mon.noname-g con 0x7f66300a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f663009fa20 0x7f663009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 10 monclient(hunting): picked mon.noname-i con 0x7f663009fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66300a08e0 0x7f66300a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 10 monclient(hunting): picked mon.noname-e con 0x7f66300a08e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66300a30a0 con 0x7f66300a2cd0 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f66300a0330 con 0x7f663009fa20 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f6635139640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66300a2cd0 msgr2=0x7f6630132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f66300a1db0 con 0x7f66300a08e0 2024-09-15T01:33:27.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:27.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f663739d640 10 monclient(hunting): authenticate will time out at 823.076889s 2024-09-15T01:33:27.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f6635139640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66300a2cd0 0x7f6630132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:27.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.434+0000 7f6634938640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f663009fa20 0x7f663009fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f663009fa20 0x7f663009fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:45644/0 (socket says 172.21.15.204:45644) 2024-09-15T01:33:27.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 1 -- 172.21.15.204:0/3457686047 learned_addr learned my addr 172.21.15.204:0/3457686047 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:27.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 10 monclient(hunting): get_auth_request con 0x7f663009fa20 auth_method 0 2024-09-15T01:33:27.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:27.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:27.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:27.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:27.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:27.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 10 monclient(hunting): handle_auth_done global_id 4759 payload 274 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f663593a640 1 --2- 172.21.15.204:0/3457686047 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66300a08e0 0x7f66300a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 1 -- 172.21.15.204:0/3457686047 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66300a08e0 msgr2=0x7f66300a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 1 --2- 172.21.15.204:0/3457686047 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66300a08e0 0x7f66300a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 1 -- 172.21.15.204:0/3457686047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66300a2cd0 msgr2=0x7f6630132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 1 --2- 172.21.15.204:0/3457686047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66300a2cd0 0x7f6630132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 10 monclient: _finish_hunting 0 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 1 monclient: found mon.noname-i 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 1 -- 172.21.15.204:0/3457686047 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66300648a0 con 0x7f663009fa20 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.435+0000 7f6634938640 1 --2- 172.21.15.204:0/3457686047 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f663009fa20 0x7f663009fdf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6620009940 tx=0x7f66200091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=220830d5075bc2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:27.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f6627fff640 1 -- 172.21.15.204:0/3457686047 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f662002ac80 con 0x7f663009fa20 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f6627fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f6627fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f6627fff640 10 monclient: dump: 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:27.448 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f6627fff640 10 monclient: _finish_auth 0 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f6627fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f6627fff640 1 -- 172.21.15.204:0/3457686047 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662002ae00 con 0x7f663009fa20 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 5 monclient: authenticate success, global_id 4759 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f6627fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:27.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f6627fff640 1 -- 172.21.15.204:0/3457686047 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6620031ca0 con 0x7f663009fa20 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 4 set_mon_vals no callback set 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f6627fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f6627fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f6627fff640 10 monclient: dump: 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:27.450 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 10 monclient: shutdown 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 1 -- 172.21.15.204:0/3457686047 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f663009fa20 msgr2=0x7f663009fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 1 --2- 172.21.15.204:0/3457686047 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f663009fa20 0x7f663009fdf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6620009940 tx=0x7f66200091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 1 -- 172.21.15.204:0/3457686047 shutdown_connections 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 1 --2- 172.21.15.204:0/3457686047 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66300a08e0 0x7f66300a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 1 --2- 172.21.15.204:0/3457686047 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f663009fa20 0x7f663009fdf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 1 --2- 172.21.15.204:0/3457686047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66300a2cd0 0x7f6630132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 1 -- 172.21.15.204:0/3457686047 >> 172.21.15.204:0/3457686047 conn(0x7f663006e860 msgr2=0x7f663006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 1 -- 172.21.15.204:0/3457686047 shutdown_connections 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.436+0000 7f663739d640 1 -- 172.21.15.204:0/3457686047 wait complete. 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.437+0000 7f663739d640 10 monclient: build_initial_monmap 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.437+0000 7f663739d640 10 monclient: monmap: 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:27.452 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.437+0000 7f663739d640 1 Processor -- start 2024-09-15T01:33:27.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.437+0000 7f663739d640 1 -- start start 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.437+0000 7f663739d640 10 monclient: init 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 10 monclient: _add_conns ranks=[2,8,6,7,4,0,3,5,1] 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f663009fa20 0x7f6630131030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 10 monclient(hunting): picked mon.noname-b con 0x7f663009fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66300a08e0 0x7f663011dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 10 monclient(hunting): picked mon.noname-e con 0x7f66300a08e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66300a2cd0 0x7f663011e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 10 monclient(hunting): picked mon.noname-d con 0x7f66300a2cd0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663009d3d0 con 0x7f663009fa20 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f663011fe80 con 0x7f66300a2cd0 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f663009cb60 con 0x7f66300a08e0 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663739d640 10 monclient(hunting): authenticate will time out at 823.080260s 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663593a640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66300a2cd0 msgr2=0x7f663011e100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f663593a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66300a2cd0 0x7f663011e100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66300a08e0 0x7f663011dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66300a08e0 0x7f663011dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:47972/0 (socket says 172.21.15.204:47972) 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6635139640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f663009fa20 0x7f6630131030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 1 -- 172.21.15.204:0/3275861627 learned_addr learned my addr 172.21.15.204:0/3275861627 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 10 monclient(hunting): get_auth_request con 0x7f66300a08e0 auth_method 0 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:27.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6635139640 10 monclient(hunting): get_auth_request con 0x7f663009fa20 auth_method 0 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6635139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6635139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6635139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6635139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6635139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6635139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 10 monclient(hunting): handle_auth_done global_id 4733 payload 824 2024-09-15T01:33:27.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 1 -- 172.21.15.204:0/3275861627 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66300a2cd0 msgr2=0x7f663011e100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 1 --2- 172.21.15.204:0/3275861627 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66300a2cd0 0x7f663011e100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 1 -- 172.21.15.204:0/3275861627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f663009fa20 msgr2=0x7f6630131030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 1 --2- 172.21.15.204:0/3275861627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f663009fa20 0x7f6630131030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 10 monclient: _finish_hunting 0 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 1 monclient: found mon.noname-e 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 1 -- 172.21.15.204:0/3275861627 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620009450 con 0x7f66300a08e0 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6635139640 1 --2- 172.21.15.204:0/3275861627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f663009fa20 0x7f6630131030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.438+0000 7f6634938640 1 --2- 172.21.15.204:0/3275861627 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66300a08e0 0x7f663011dbc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6620000c00 tx=0x7f6620032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66257fa640 1 -- 172.21.15.204:0/3275861627 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f662002ac80 con 0x7f66300a08e0 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:27.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66257fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:33:27.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66257fa640 10 monclient: dump: 2024-09-15T01:33:27.458 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:27.458 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.458 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.458 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.458 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:27.458 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.458 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:27.458 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:27.458 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:27.458 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:27.458 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66257fa640 10 monclient: _finish_auth 0 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66257fa640 1 -- 172.21.15.204:0/3275861627 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6620033e30 con 0x7f66300a08e0 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66257fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66257fa640 1 -- 172.21.15.204:0/3275861627 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f662003a810 con 0x7f66300a08e0 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f663739d640 5 monclient: authenticate success, global_id 4733 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:27.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66257fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66277fe640 4 set_mon_vals no callback set 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f66257fa640 10 monclient: dump: 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:27.460 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f663739d640 10 monclient: _renew_subs 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f663739d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f663739d640 1 -- 172.21.15.204:0/3275861627 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6630120290 con 0x7f66300a08e0 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f663739d640 10 monclient: _renew_subs 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f663739d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.439+0000 7f663739d640 1 -- 172.21.15.204:0/3275861627 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66301427c0 con 0x7f66300a08e0 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.440+0000 7f66257fa640 1 -- 172.21.15.204:0/3275861627 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6620039040 con 0x7f66300a08e0 2024-09-15T01:33:27.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.440+0000 7f663739d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:27.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.440+0000 7f663739d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:27.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.440+0000 7f663739d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:33:27.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.440+0000 7f663739d640 1 -- 172.21.15.204:0/3275861627 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6630070b80 con 0x7f66300a08e0 2024-09-15T01:33:27.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.440+0000 7f66257fa640 1 --2- 172.21.15.204:0/3275861627 >> v2:172.21.15.204:6812/28210 conn(0x7f66140396e0 0x7f661403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.440+0000 7f66257fa640 1 -- 172.21.15.204:0/3275861627 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4630+0+0 (secure 0 0 0) 0x7f662006aa60 con 0x7f66300a08e0 2024-09-15T01:33:27.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.440+0000 7f6635139640 1 --2- 172.21.15.204:0/3275861627 >> v2:172.21.15.204:6812/28210 conn(0x7f66140396e0 0x7f661403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.440+0000 7f6635139640 10 monclient: get_auth_request con 0x7f66140396e0 auth_method 0 2024-09-15T01:33:27.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.440+0000 7f6635139640 1 --2- 172.21.15.204:0/3275861627 >> v2:172.21.15.204:6812/28210 conn(0x7f66140396e0 0x7f661403bb90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f662c003850 tx=0x7f662c0090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:27.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.445+0000 7f66257fa640 1 -- 172.21.15.204:0/3275861627 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f66200438f0 con 0x7f66300a08e0 2024-09-15T01:33:27.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.445+0000 7f66257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:27.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.445+0000 7f66257fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:27.604 INFO:tasks.workunit.client.0.smithi204.stderr:foo not valid: foo not one of 'true', 'false' 2024-09-15T01:33:27.604 INFO:tasks.workunit.client.0.smithi204.stderr:Invalid command: unused arguments: ['foo'] 2024-09-15T01:33:27.604 INFO:tasks.workunit.client.0.smithi204.stderr:osd pool delete [] [--yes-i-really-really-mean-it] [--yes-i-really-really-mean-it-not-faking] : delete pool 2024-09-15T01:33:27.604 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: invalid command 2024-09-15T01:33:27.606 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:27.606 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete foooo foooo --yes-i-really-really-mean-it 2024-09-15T01:33:27.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.683+0000 7ffbda500640 10 monclient: get_monmap_and_config 2024-09-15T01:33:27.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.683+0000 7ffbda500640 10 monclient: build_initial_monmap 2024-09-15T01:33:27.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.683+0000 7ffbda500640 10 monclient: monmap: 2024-09-15T01:33:27.685 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:27.685 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.685 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:27.683822+0000 2024-09-15T01:33:27.685 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:27.683822+0000 2024-09-15T01:33:27.685 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:27.685 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.685 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:27.685 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:27.685 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:27.685 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:27.686 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:27.686 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:27.686 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:27.686 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:27.686 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:27.686 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:27.686 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:27.686 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 1 Processor -- start 2024-09-15T01:33:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 1 -- start start 2024-09-15T01:33:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 10 monclient: init 2024-09-15T01:33:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 10 monclient: _add_conns ranks=[7,3,10,1,2,8,6,9,0,5,4] 2024-09-15T01:33:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffbd406e790 0x7ffbd4130810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 10 monclient(hunting): picked mon.noname-i con 0x7ffbd406e790 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbd4130d50 0x7ffbd4131120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 10 monclient(hunting): picked mon.noname-g con 0x7ffbd4130d50 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffbd409ec20 0x7ffbd409f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 10 monclient(hunting): picked mon.noname-f con 0x7ffbd409ec20 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd406d9d0 con 0x7ffbd409ec20 2024-09-15T01:33:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4131660 con 0x7ffbd4130d50 2024-09-15T01:33:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd406eb60 con 0x7ffbd406e790 2024-09-15T01:33:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbda500640 10 monclient(hunting): authenticate will time out at 823.326597s 2024-09-15T01:33:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd37fe640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbd4130d50 msgr2=0x7ffbd4131120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd37fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbd4130d50 0x7ffbd4131120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffbd406e790 0x7ffbd4130810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffbd406e790 0x7ffbd4130810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:45660/0 (socket says 172.21.15.204:45660) 2024-09-15T01:33:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd3fff640 1 -- 172.21.15.204:0/2735759719 learned_addr learned my addr 172.21.15.204:0/2735759719 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd3fff640 10 monclient(hunting): get_auth_request con 0x7ffbd406e790 auth_method 0 2024-09-15T01:33:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd8a9d640 1 --2- 172.21.15.204:0/2735759719 >> v2:172.21.15.204:3300/0 conn(0x7ffbd409ec20 0x7ffbd409f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd8a9d640 10 monclient(hunting): get_auth_request con 0x7ffbd409ec20 auth_method 0 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd8a9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd8a9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd8a9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd8a9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd8a9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd8a9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd8a9d640 10 monclient(hunting): handle_auth_done global_id 4879 payload 274 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd8a9d640 1 -- 172.21.15.204:0/2735759719 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffbd406e790 msgr2=0x7ffbd4130810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd8a9d640 1 --2- 172.21.15.204:0/2735759719 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffbd406e790 0x7ffbd4130810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.684+0000 7ffbd8a9d640 1 -- 172.21.15.204:0/2735759719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbd4130d50 msgr2=0x7ffbd4131120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd8a9d640 1 --2- 172.21.15.204:0/2735759719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbd4130d50 0x7ffbd4131120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd8a9d640 10 monclient: _finish_hunting 0 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd8a9d640 1 monclient: found mon.noname-f 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd8a9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd8a9d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd8a9d640 1 -- 172.21.15.204:0/2735759719 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd40648a0 con 0x7ffbd409ec20 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd3fff640 1 --2- 172.21.15.204:0/2735759719 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffbd406e790 0x7ffbd4130810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd8a9d640 1 --2- 172.21.15.204:0/2735759719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd409ec20 0x7ffbd409f090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009e60 tx=0x7ffbcc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4be88ae6704d6e97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd2ffd640 1 -- 172.21.15.204:0/2735759719 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffbcc0079d0 con 0x7ffbd409ec20 2024-09-15T01:33:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd2ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd2ffd640 10 monclient: dump: 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd2ffd640 1 -- 172.21.15.204:0/2735759719 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc02d050 con 0x7ffbd409ec20 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbda500640 5 monclient: authenticate success, global_id 4879 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbda500640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbda500640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd2ffd640 1 -- 172.21.15.204:0/2735759719 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffbcc007bd0 con 0x7ffbd409ec20 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbda500640 4 set_mon_vals no callback set 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd2ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.685+0000 7ffbd2ffd640 10 monclient: dump: 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 10 monclient: shutdown 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 1 -- 172.21.15.204:0/2735759719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd409ec20 msgr2=0x7ffbd409f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 1 --2- 172.21.15.204:0/2735759719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd409ec20 0x7ffbd409f090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009e60 tx=0x7ffbcc009780 comp rx=0 tx=0).stop 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 1 -- 172.21.15.204:0/2735759719 shutdown_connections 2024-09-15T01:33:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 1 --2- 172.21.15.204:0/2735759719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd409ec20 0x7ffbd409f090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 1 --2- 172.21.15.204:0/2735759719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbd4130d50 0x7ffbd4131120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 1 --2- 172.21.15.204:0/2735759719 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffbd406e790 0x7ffbd4130810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 1 -- 172.21.15.204:0/2735759719 >> 172.21.15.204:0/2735759719 conn(0x7ffbd4067840 msgr2=0x7ffbd40a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 1 -- 172.21.15.204:0/2735759719 shutdown_connections 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 1 -- 172.21.15.204:0/2735759719 wait complete. 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 10 monclient: build_initial_monmap 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.686+0000 7ffbda500640 10 monclient: monmap: 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:27.704 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 1 Processor -- start 2024-09-15T01:33:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 1 -- start start 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 10 monclient: init 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 10 monclient: _add_conns ranks=[8,3,1,7,5,4,2,6,0] 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffbd406e790 0x7ffbd412e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 10 monclient(hunting): picked mon.noname-e con 0x7ffbd406e790 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbd409ec20 0x7ffbd411b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 10 monclient(hunting): picked mon.noname-g con 0x7ffbd409ec20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd4130d50 0x7ffbd411b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 10 monclient(hunting): picked mon.noname-f con 0x7ffbd4130d50 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd406d1c0 con 0x7ffbd4130d50 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd406cd90 con 0x7ffbd409ec20 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4131ce0 con 0x7ffbd406e790 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbda500640 10 monclient(hunting): authenticate will time out at 823.329848s 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbd37fe640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbd409ec20 msgr2=0x7ffbd411b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbd37fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbd409ec20 0x7ffbd411b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbd8a9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd4130d50 0x7ffbd411b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbd8a9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd4130d50 0x7ffbd411b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33198/0 (socket says 172.21.15.204:33198) 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbd8a9d640 1 -- 172.21.15.204:0/3982900863 learned_addr learned my addr 172.21.15.204:0/3982900863 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbd8a9d640 10 monclient(hunting): get_auth_request con 0x7ffbd4130d50 auth_method 0 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbd8a9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbd8a9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.687+0000 7ffbd8a9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd3fff640 1 --2- 172.21.15.204:0/3982900863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffbd406e790 0x7ffbd412e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd3fff640 10 monclient(hunting): get_auth_request con 0x7ffbd406e790 auth_method 0 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 10 monclient(hunting): handle_auth_done global_id 4888 payload 824 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 1 -- 172.21.15.204:0/3982900863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffbd406e790 msgr2=0x7ffbd412e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 1 --2- 172.21.15.204:0/3982900863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffbd406e790 0x7ffbd412e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 1 -- 172.21.15.204:0/3982900863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbd409ec20 msgr2=0x7ffbd411b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 1 --2- 172.21.15.204:0/3982900863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbd409ec20 0x7ffbd411b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 10 monclient: _finish_hunting 0 2024-09-15T01:33:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 1 monclient: found mon.noname-f 2024-09-15T01:33:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 1 -- 172.21.15.204:0/3982900863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc4009450 con 0x7ffbd4130d50 2024-09-15T01:33:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd8a9d640 1 --2- 172.21.15.204:0/3982900863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd4130d50 0x7ffbd411b9d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0099f0 tx=0x7ffbcc037000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbbbfff640 1 -- 172.21.15.204:0/3982900863 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffbcc007eb0 con 0x7ffbd4130d50 2024-09-15T01:33:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbbbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbbbfff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbbbfff640 10 monclient: dump: 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:27.709 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbbbfff640 10 monclient: _finish_auth 0 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbbbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbbbfff640 1 -- 172.21.15.204:0/3982900863 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc03d430 con 0x7ffbd4130d50 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbda500640 5 monclient: authenticate success, global_id 4888 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbda500640 10 monclient: _renew_subs 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbda500640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbda500640 1 -- 172.21.15.204:0/3982900863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbcc009a20 con 0x7ffbd4130d50 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbbbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbbbfff640 1 -- 172.21.15.204:0/3982900863 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffbcc03da20 con 0x7ffbd4130d50 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbbbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:27.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbbbfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbbbfff640 10 monclient: dump: 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:27.711 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:27.712 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:27.712 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:27.712 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:27.712 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:27.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbda500640 10 monclient: _renew_subs 2024-09-15T01:33:27.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbda500640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:27.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbda500640 1 -- 172.21.15.204:0/3982900863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd41406c0 con 0x7ffbd4130d50 2024-09-15T01:33:27.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.688+0000 7ffbd27fc640 4 set_mon_vals no callback set 2024-09-15T01:33:27.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.689+0000 7ffbbbfff640 1 -- 172.21.15.204:0/3982900863 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ffbcc003040 con 0x7ffbd4130d50 2024-09-15T01:33:27.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.689+0000 7ffbda500640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:27.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.689+0000 7ffbda500640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:27.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.689+0000 7ffbda500640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:27.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.689+0000 7ffbbbfff640 1 --2- 172.21.15.204:0/3982900863 >> v2:172.21.15.204:6812/28210 conn(0x7ffbc00397e0 0x7ffbc003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:27.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.689+0000 7ffbda500640 1 -- 172.21.15.204:0/3982900863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd411ee70 con 0x7ffbd4130d50 2024-09-15T01:33:27.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.689+0000 7ffbd3fff640 1 --2- 172.21.15.204:0/3982900863 >> v2:172.21.15.204:6812/28210 conn(0x7ffbc00397e0 0x7ffbc003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:27.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.689+0000 7ffbbbfff640 1 -- 172.21.15.204:0/3982900863 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4630+0+0 (secure 0 0 0) 0x7ffbcc06c8c0 con 0x7ffbd4130d50 2024-09-15T01:33:27.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.689+0000 7ffbd3fff640 10 monclient: get_auth_request con 0x7ffbc00397e0 auth_method 0 2024-09-15T01:33:27.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.690+0000 7ffbd3fff640 1 --2- 172.21.15.204:0/3982900863 >> v2:172.21.15.204:6812/28210 conn(0x7ffbc00397e0 0x7ffbc003bc90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4006c90 tx=0x7ffbc40091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:27.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.694+0000 7ffbbbfff640 1 -- 172.21.15.204:0/3982900863 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ffbcc0445e0 con 0x7ffbd4130d50 2024-09-15T01:33:27.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.694+0000 7ffbbbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:27.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.694+0000 7ffbbbfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:27.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.849+0000 7ffbda500640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:27.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.850+0000 7ffbda500640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:27.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.850+0000 7ffbda500640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:27.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:27.850+0000 7ffbda500640 1 -- 172.21.15.204:0/3982900863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffbd4070b80 con 0x7ffbd4130d50 2024-09-15T01:33:28.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.488+0000 7ffbbbfff640 1 -- 172.21.15.204:0/3982900863 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}]=0 pool 'foooo' removed v41) v1 ==== 155+0+0 (secure 0 0 0) 0x7ffbcc030b50 con 0x7ffbd4130d50 2024-09-15T01:33:28.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.488+0000 7ffbbbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:28.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.488+0000 7ffbbbfff640 10 monclient: _finish_command 2 = system:0 pool 'foooo' removed 2024-09-15T01:33:28.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.593+0000 7ffbda500640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:28.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.593+0000 7ffbda500640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:28.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.593+0000 7ffbda500640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:28.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.593+0000 7ffbda500640 1 -- 172.21.15.204:0/3982900863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffbd411f020 con 0x7ffbd4130d50 2024-09-15T01:33:28.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.594+0000 7ffbbbfff640 1 -- 172.21.15.204:0/3982900863 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}]=0 pool 'foooo' does not exist v41) v1 ==== 162+0+0 (secure 0 0 0) 0x7ffbcc0443b0 con 0x7ffbd4130d50 2024-09-15T01:33:28.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.594+0000 7ffbbbfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:28.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.594+0000 7ffbbbfff640 10 monclient: _finish_command 3 = system:0 pool 'foooo' does not exist 2024-09-15T01:33:28.596 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'foooo' does not exist 2024-09-15T01:33:28.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 1 -- 172.21.15.204:0/3982900863 >> v2:172.21.15.204:6812/28210 conn(0x7ffbc00397e0 msgr2=0x7ffbc003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:28.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 1 --2- 172.21.15.204:0/3982900863 >> v2:172.21.15.204:6812/28210 conn(0x7ffbc00397e0 0x7ffbc003bc90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4006c90 tx=0x7ffbc40091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 10 monclient: shutdown 2024-09-15T01:33:28.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:28.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 1 -- 172.21.15.204:0/3982900863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd4130d50 msgr2=0x7ffbd411b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:28.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 1 --2- 172.21.15.204:0/3982900863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd4130d50 0x7ffbd411b9d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0099f0 tx=0x7ffbcc037000 comp rx=0 tx=0).stop 2024-09-15T01:33:28.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 1 -- 172.21.15.204:0/3982900863 shutdown_connections 2024-09-15T01:33:28.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 1 --2- 172.21.15.204:0/3982900863 >> v2:172.21.15.204:6812/28210 conn(0x7ffbc00397e0 0x7ffbc003bc90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 1 --2- 172.21.15.204:0/3982900863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd4130d50 0x7ffbd411b9d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 1 --2- 172.21.15.204:0/3982900863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbd409ec20 0x7ffbd411b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 1 --2- 172.21.15.204:0/3982900863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffbd406e790 0x7ffbd412e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 1 -- 172.21.15.204:0/3982900863 >> 172.21.15.204:0/3982900863 conn(0x7ffbd4067840 msgr2=0x7ffbd40a1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:28.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 1 -- 172.21.15.204:0/3982900863 shutdown_connections 2024-09-15T01:33:28.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.596+0000 7ffbda500640 1 -- 172.21.15.204:0/3982900863 wait complete. 2024-09-15T01:33:28.606 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete fooo fooo --yes-i-really-really-mean-it 2024-09-15T01:33:28.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.681+0000 7f63a479d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:28.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.681+0000 7f63a479d640 10 monclient: build_initial_monmap 2024-09-15T01:33:28.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.681+0000 7f63a479d640 10 monclient: monmap: 2024-09-15T01:33:28.683 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:28.683 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:28.683 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:28.681837+0000 2024-09-15T01:33:28.683 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:28.681837+0000 2024-09-15T01:33:28.683 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:28.683 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:28.683 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:28.683 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:28.683 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:28.683 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:28.683 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:28.684 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:28.684 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:28.684 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:28.684 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:28.684 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:28.684 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:28.684 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.681+0000 7f63a479d640 1 Processor -- start 2024-09-15T01:33:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.681+0000 7f63a479d640 1 -- start start 2024-09-15T01:33:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.681+0000 7f63a479d640 10 monclient: init 2024-09-15T01:33:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 10 monclient: _add_conns ranks=[3,5,9,8,10,2,4,6,1,7,0] 2024-09-15T01:33:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c0a2cd0 0x7f639c132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 10 monclient(hunting): picked mon.noname-g con 0x7f639c0a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f639c09fa20 0x7f639c09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 10 monclient(hunting): picked mon.noname-h con 0x7f639c09fa20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f639c0a08e0 0x7f639c0a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 10 monclient(hunting): picked mon.noname-a con 0x7f639c0a08e0 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639c0a30a0 con 0x7f639c0a08e0 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639c0a0330 con 0x7f639c0a2cd0 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f639c0a1db0 con 0x7f639c09fa20 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:28.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a479d640 10 monclient(hunting): authenticate will time out at 824.324620s 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a2539640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c0a2cd0 msgr2=0x7f639c132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a2539640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c0a2cd0 0x7f639c132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f639c09fa20 0x7f639c09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f639c09fa20 0x7f639c09fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:52882/0 (socket says 172.21.15.204:52882) 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 1 -- 172.21.15.204:0/2753164497 learned_addr learned my addr 172.21.15.204:0/2753164497 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 10 monclient(hunting): get_auth_request con 0x7f639c09fa20 auth_method 0 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:28.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a2d3a640 1 --2- 172.21.15.204:0/2753164497 >> v2:172.21.15.178:3300/0 conn(0x7f639c0a08e0 0x7f639c0a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 10 monclient(hunting): handle_auth_done global_id 4757 payload 274 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 1 -- 172.21.15.204:0/2753164497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c0a2cd0 msgr2=0x7f639c132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 1 --2- 172.21.15.204:0/2753164497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c0a2cd0 0x7f639c132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 1 -- 172.21.15.204:0/2753164497 >> v2:172.21.15.178:3300/0 conn(0x7f639c0a08e0 msgr2=0x7f639c0a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 1 --2- 172.21.15.204:0/2753164497 >> v2:172.21.15.178:3300/0 conn(0x7f639c0a08e0 0x7f639c0a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 10 monclient: _finish_hunting 0 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 1 monclient: found mon.noname-h 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a1d38640 1 -- 172.21.15.204:0/2753164497 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c0648a0 con 0x7f639c09fa20 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a2d3a640 10 monclient: get_auth_request con 0x7f639c0a08e0 auth_method 0 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.682+0000 7f63a2d3a640 1 --2- 172.21.15.204:0/2753164497 >> v2:172.21.15.178:3300/0 conn(0x7f639c0a08e0 0x7f639c0a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1d38640 1 --2- 172.21.15.204:0/2753164497 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f639c09fa20 0x7f639c09fdf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f638c009910 tx=0x7f638c0091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=6be37ab1178bc113 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:28.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1537640 1 -- 172.21.15.204:0/2753164497 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f638c02acb0 con 0x7f639c09fa20 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1537640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1537640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1537640 10 monclient: dump: 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:28.695 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1537640 10 monclient: _finish_auth 0 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1537640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1537640 1 -- 172.21.15.204:0/2753164497 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c02ae30 con 0x7f639c09fa20 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a479d640 5 monclient: authenticate success, global_id 4757 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a479d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1537640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1537640 1 -- 172.21.15.204:0/2753164497 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f638c031ca0 con 0x7f639c09fa20 2024-09-15T01:33:28.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a479d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a479d640 4 set_mon_vals no callback set 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1537640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1537640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.683+0000 7f63a1537640 10 monclient: dump: 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:28.697 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 10 monclient: shutdown 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 1 -- 172.21.15.204:0/2753164497 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f639c09fa20 msgr2=0x7f639c09fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 1 --2- 172.21.15.204:0/2753164497 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f639c09fa20 0x7f639c09fdf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f638c009910 tx=0x7f638c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 1 -- 172.21.15.204:0/2753164497 shutdown_connections 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 1 --2- 172.21.15.204:0/2753164497 >> v2:172.21.15.178:3300/0 conn(0x7f639c0a08e0 0x7f639c0a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 1 --2- 172.21.15.204:0/2753164497 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f639c09fa20 0x7f639c09fdf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 1 --2- 172.21.15.204:0/2753164497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c0a2cd0 0x7f639c132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 1 -- 172.21.15.204:0/2753164497 >> 172.21.15.204:0/2753164497 conn(0x7f639c06e860 msgr2=0x7f639c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 1 -- 172.21.15.204:0/2753164497 shutdown_connections 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 1 -- 172.21.15.204:0/2753164497 wait complete. 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 10 monclient: build_initial_monmap 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.684+0000 7f63a479d640 10 monclient: monmap: 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 1 Processor -- start 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 1 -- start start 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 10 monclient: init 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 10 monclient: _add_conns ranks=[2,1,3,4,8,5,7,6,0] 2024-09-15T01:33:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f639c09fa20 0x7f639c130f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 10 monclient(hunting): picked mon.noname-b con 0x7f639c09fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c0a08e0 0x7f639c11da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 10 monclient(hunting): picked mon.noname-f con 0x7f639c0a08e0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c0a2cd0 0x7f639c11dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 10 monclient(hunting): picked mon.noname-g con 0x7f639c0a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c142910 con 0x7f639c0a08e0 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639c09d3d0 con 0x7f639c09fa20 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639c09cb60 con 0x7f639c0a2cd0 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a479d640 10 monclient(hunting): authenticate will time out at 824.327839s 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a2d3a640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c0a2cd0 msgr2=0x7f639c11dfd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a2d3a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c0a2cd0 0x7f639c11dfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a1d38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c0a08e0 0x7f639c11da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a1d38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c0a08e0 0x7f639c11da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33212/0 (socket says 172.21.15.204:33212) 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a1d38640 1 -- 172.21.15.204:0/1598474572 learned_addr learned my addr 172.21.15.204:0/1598474572 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a1d38640 10 monclient(hunting): get_auth_request con 0x7f639c0a08e0 auth_method 0 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a1d38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a1d38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a1d38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a1d38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.685+0000 7f63a1d38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a1d38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a1d38640 10 monclient(hunting): handle_auth_done global_id 4897 payload 824 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a1d38640 1 -- 172.21.15.204:0/1598474572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c0a2cd0 msgr2=0x7f639c11dfd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a1d38640 1 --2- 172.21.15.204:0/1598474572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c0a2cd0 0x7f639c11dfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a1d38640 1 -- 172.21.15.204:0/1598474572 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f639c09fa20 msgr2=0x7f639c130f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a1d38640 1 --2- 172.21.15.204:0/1598474572 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f639c09fa20 0x7f639c130f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a1d38640 10 monclient: _finish_hunting 0 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a1d38640 1 monclient: found mon.noname-f 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a1d38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a1d38640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a1d38640 1 -- 172.21.15.204:0/1598474572 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c009450 con 0x7f639c0a08e0 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a1d38640 1 --2- 172.21.15.204:0/1598474572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c0a08e0 0x7f639c11da90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f638c0097e0 tx=0x7f638c032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63927fc640 1 -- 172.21.15.204:0/1598474572 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f638c031820 con 0x7f639c0a08e0 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63927fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63927fc640 10 monclient: dump: 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63927fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63927fc640 1 -- 172.21.15.204:0/1598474572 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c031980 con 0x7f639c0a08e0 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a479d640 5 monclient: authenticate success, global_id 4897 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63927fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a479d640 10 monclient: _renew_subs 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63927fc640 1 -- 172.21.15.204:0/1598474572 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f638c030340 con 0x7f639c0a08e0 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a479d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a479d640 1 -- 172.21.15.204:0/1598474572 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c11fd50 con 0x7f639c0a08e0 2024-09-15T01:33:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a0d36640 4 set_mon_vals no callback set 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63927fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63927fc640 10 monclient: dump: 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a479d640 10 monclient: _renew_subs 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a479d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.686+0000 7f63a479d640 1 -- 172.21.15.204:0/1598474572 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c121050 con 0x7f639c0a08e0 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.687+0000 7f63927fc640 1 -- 172.21.15.204:0/1598474572 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f638c039040 con 0x7f639c0a08e0 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.687+0000 7f63927fc640 1 --2- 172.21.15.204:0/1598474572 >> v2:172.21.15.204:6812/28210 conn(0x7f63800397b0 0x7f638003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.687+0000 7f63927fc640 1 -- 172.21.15.204:0/1598474572 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4286+0+0 (secure 0 0 0) 0x7f638c06abe0 con 0x7f639c0a08e0 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.687+0000 7f63a479d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.687+0000 7f63a479d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.687+0000 7f63a479d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.687+0000 7f63a479d640 1 -- 172.21.15.204:0/1598474572 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639c070b80 con 0x7f639c0a08e0 2024-09-15T01:33:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.687+0000 7f63a2539640 1 --2- 172.21.15.204:0/1598474572 >> v2:172.21.15.204:6812/28210 conn(0x7f63800397b0 0x7f638003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.687+0000 7f63a2539640 10 monclient: get_auth_request con 0x7f63800397b0 auth_method 0 2024-09-15T01:33:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.687+0000 7f63a2539640 1 --2- 172.21.15.204:0/1598474572 >> v2:172.21.15.204:6812/28210 conn(0x7f63800397b0 0x7f638003bc60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6398003940 tx=0x7f6398006cb0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.691+0000 7f63927fc640 1 -- 172.21.15.204:0/1598474572 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f638c041080 con 0x7f639c0a08e0 2024-09-15T01:33:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.692+0000 7f63927fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.692+0000 7f63927fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:28.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.846+0000 7f63a479d640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:28.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.846+0000 7f63a479d640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:28.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.846+0000 7f63a479d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:28.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:28.846+0000 7f63a479d640 1 -- 172.21.15.204:0/1598474572 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f639c0008d0 con 0x7f639c0a08e0 2024-09-15T01:33:29.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.485+0000 7f63927fc640 1 -- 172.21.15.204:0/1598474572 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' removed v42) v1 ==== 152+0+0 (secure 0 0 0) 0x7f638c037020 con 0x7f639c0a08e0 2024-09-15T01:33:29.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.485+0000 7f63927fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:29.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.485+0000 7f63927fc640 10 monclient: _finish_command 2 = system:0 pool 'fooo' removed 2024-09-15T01:33:29.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.611+0000 7f63a479d640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:29.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.611+0000 7f63a479d640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:29.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.611+0000 7f63a479d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:29.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.611+0000 7f63a479d640 1 -- 172.21.15.204:0/1598474572 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f639c133e80 con 0x7f639c0a08e0 2024-09-15T01:33:29.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.612+0000 7f63927fc640 1 -- 172.21.15.204:0/1598474572 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v42) v1 ==== 159+0+0 (secure 0 0 0) 0x7f638c02f8a0 con 0x7f639c0a08e0 2024-09-15T01:33:29.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.612+0000 7f63927fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:29.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.612+0000 7f63927fc640 10 monclient: _finish_command 3 = system:0 pool 'fooo' does not exist 2024-09-15T01:33:29.614 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'fooo' does not exist 2024-09-15T01:33:29.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 1 -- 172.21.15.204:0/1598474572 >> v2:172.21.15.204:6812/28210 conn(0x7f63800397b0 msgr2=0x7f638003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:29.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 1 --2- 172.21.15.204:0/1598474572 >> v2:172.21.15.204:6812/28210 conn(0x7f63800397b0 0x7f638003bc60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6398003940 tx=0x7f6398006cb0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 10 monclient: shutdown 2024-09-15T01:33:29.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:29.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 1 -- 172.21.15.204:0/1598474572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c0a08e0 msgr2=0x7f639c11da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:29.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 1 --2- 172.21.15.204:0/1598474572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c0a08e0 0x7f639c11da90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f638c0097e0 tx=0x7f638c032000 comp rx=0 tx=0).stop 2024-09-15T01:33:29.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 1 -- 172.21.15.204:0/1598474572 shutdown_connections 2024-09-15T01:33:29.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 1 --2- 172.21.15.204:0/1598474572 >> v2:172.21.15.204:6812/28210 conn(0x7f63800397b0 0x7f638003bc60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 1 --2- 172.21.15.204:0/1598474572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f639c0a2cd0 0x7f639c11dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 1 --2- 172.21.15.204:0/1598474572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f639c0a08e0 0x7f639c11da90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 1 --2- 172.21.15.204:0/1598474572 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f639c09fa20 0x7f639c130f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 1 -- 172.21.15.204:0/1598474572 >> 172.21.15.204:0/1598474572 conn(0x7f639c06e860 msgr2=0x7f639c0a5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:29.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 1 -- 172.21.15.204:0/1598474572 shutdown_connections 2024-09-15T01:33:29.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.614+0000 7f63a479d640 1 -- 172.21.15.204:0/1598474572 wait complete. 2024-09-15T01:33:29.625 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete foo foo --yes-i-really-really-mean-it 2024-09-15T01:33:29.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.699+0000 7f5add06f640 10 monclient: get_monmap_and_config 2024-09-15T01:33:29.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.699+0000 7f5add06f640 10 monclient: build_initial_monmap 2024-09-15T01:33:29.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.699+0000 7f5add06f640 10 monclient: monmap: 2024-09-15T01:33:29.701 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:29.701 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:29.701 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:29.700090+0000 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:29.700090+0000 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:29.702 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:29.703 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:29.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 1 Processor -- start 2024-09-15T01:33:29.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 1 -- start start 2024-09-15T01:33:29.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 10 monclient: init 2024-09-15T01:33:29.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:29.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 10 monclient: _add_conns ranks=[3,8,6,1,7,9,2,4,5,0,10] 2024-09-15T01:33:29.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ad80a2cd0 0x7f5ad8132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 10 monclient(hunting): picked mon.noname-g con 0x7f5ad80a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ad809fa20 0x7f5ad809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 10 monclient(hunting): picked mon.noname-e con 0x7f5ad809fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ad80a08e0 0x7f5ad80a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 10 monclient(hunting): picked mon.noname-d con 0x7f5ad80a08e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad80a30a0 con 0x7f5ad80a2cd0 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad80a0330 con 0x7f5ad80a08e0 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad80a1db0 con 0x7f5ad809fa20 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5add06f640 10 monclient(hunting): authenticate will time out at 825.342707s 2024-09-15T01:33:29.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5ad6d9d640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ad80a2cd0 msgr2=0x7f5ad8132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5ad6d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ad80a2cd0 0x7f5ad8132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5ad759e640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ad80a08e0 msgr2=0x7f5ad80a0d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5ad759e640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ad80a08e0 0x7f5ad80a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5ad659c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ad809fa20 0x7f5ad809fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.700+0000 7f5ad659c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ad809fa20 0x7f5ad809fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:47990/0 (socket says 172.21.15.204:47990) 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 1 -- 172.21.15.204:0/3766558795 learned_addr learned my addr 172.21.15.204:0/3766558795 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 10 monclient(hunting): get_auth_request con 0x7f5ad809fa20 auth_method 0 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:29.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 10 monclient(hunting): handle_auth_done global_id 4751 payload 274 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 1 -- 172.21.15.204:0/3766558795 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ad80a08e0 msgr2=0x7f5ad80a0d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 1 --2- 172.21.15.204:0/3766558795 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ad80a08e0 0x7f5ad80a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 1 -- 172.21.15.204:0/3766558795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ad80a2cd0 msgr2=0x7f5ad8132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 1 --2- 172.21.15.204:0/3766558795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ad80a2cd0 0x7f5ad8132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 10 monclient: _finish_hunting 0 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 1 monclient: found mon.noname-e 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.701+0000 7f5ad659c640 1 -- 172.21.15.204:0/3766558795 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad80648a0 con 0x7f5ad809fa20 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad659c640 1 --2- 172.21.15.204:0/3766558795 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ad809fa20 0x7f5ad809fdf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009940 tx=0x7f5acc0091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=aa5a376d68771f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad5d9b640 1 -- 172.21.15.204:0/3766558795 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5acc02acb0 con 0x7f5ad809fa20 2024-09-15T01:33:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad5d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad5d9b640 10 monclient: dump: 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad5d9b640 10 monclient: _finish_auth 0 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad5d9b640 1 -- 172.21.15.204:0/3766558795 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5acc02ae30 con 0x7f5ad809fa20 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad5d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad5d9b640 1 -- 172.21.15.204:0/3766558795 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5acc031ca0 con 0x7f5ad809fa20 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad5d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:33:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5ad5d9b640 10 monclient: dump: 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 5 monclient: authenticate success, global_id 4751 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 4 set_mon_vals no callback set 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 10 monclient: shutdown 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 1 -- 172.21.15.204:0/3766558795 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ad809fa20 msgr2=0x7f5ad809fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 1 --2- 172.21.15.204:0/3766558795 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ad809fa20 0x7f5ad809fdf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009940 tx=0x7f5acc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 1 -- 172.21.15.204:0/3766558795 shutdown_connections 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 1 --2- 172.21.15.204:0/3766558795 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ad80a08e0 0x7f5ad80a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 1 --2- 172.21.15.204:0/3766558795 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ad809fa20 0x7f5ad809fdf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 1 --2- 172.21.15.204:0/3766558795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ad80a2cd0 0x7f5ad8132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 1 -- 172.21.15.204:0/3766558795 >> 172.21.15.204:0/3766558795 conn(0x7f5ad806e860 msgr2=0x7f5ad806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 1 -- 172.21.15.204:0/3766558795 shutdown_connections 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.702+0000 7f5add06f640 1 -- 172.21.15.204:0/3766558795 wait complete. 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.703+0000 7f5add06f640 10 monclient: build_initial_monmap 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.703+0000 7f5add06f640 10 monclient: monmap: 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.703+0000 7f5add06f640 1 Processor -- start 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.703+0000 7f5add06f640 1 -- start start 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.703+0000 7f5add06f640 10 monclient: init 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 10 monclient: _add_conns ranks=[7,8,6,5,3,1,0,2,4] 2024-09-15T01:33:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ad809fa20 0x7f5ad8141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 10 monclient(hunting): picked mon.noname-i con 0x7f5ad809fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ad80a08e0 0x7f5ad8131130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 10 monclient(hunting): picked mon.noname-e con 0x7f5ad80a08e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ad80a2cd0 0x7f5ad811dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 10 monclient(hunting): picked mon.noname-d con 0x7f5ad80a2cd0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad809cb60 con 0x7f5ad80a2cd0 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad8121560 con 0x7f5ad809fa20 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad8142990 con 0x7f5ad80a08e0 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5add06f640 10 monclient(hunting): authenticate will time out at 825.346260s 2024-09-15T01:33:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ad809fa20 0x7f5ad8141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ad809fa20 0x7f5ad8141b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:45668/0 (socket says 172.21.15.204:45668) 2024-09-15T01:33:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 -- 172.21.15.204:0/706262649 learned_addr learned my addr 172.21.15.204:0/706262649 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 10 monclient(hunting): get_auth_request con 0x7f5ad809fa20 auth_method 0 2024-09-15T01:33:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad759e640 1 -- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ad80a2cd0 msgr2=0x7f5ad811dc20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad759e640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ad80a2cd0 0x7f5ad811dc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad659c640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ad80a08e0 0x7f5ad8131130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad659c640 10 monclient(hunting): get_auth_request con 0x7f5ad80a08e0 auth_method 0 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 10 monclient(hunting): handle_auth_done global_id 4777 payload 824 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 -- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ad80a08e0 msgr2=0x7f5ad8131130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ad80a08e0 0x7f5ad8131130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 -- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ad80a2cd0 msgr2=0x7f5ad811dc20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ad80a2cd0 0x7f5ad811dc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:33:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 monclient: found mon.noname-i 2024-09-15T01:33:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 -- 172.21.15.204:0/706262649 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc009450 con 0x7f5ad809fa20 2024-09-15T01:33:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad659c640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ad80a08e0 0x7f5ad8131130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 -- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ad809fa20 msgr2=0x7f5ad8141b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:33:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 -- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ad809fa20 msgr2=0x7f5ad8141b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:33:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ad809fa20 0x7f5ad8141b00 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800b5c0 tx=0x7f5ac8009120 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:33:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.704+0000 7f5ad6d9d640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ad809fa20 0x7f5ad8141b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:29.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad6d9d640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ad809fa20 0x7f5ad8141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:29.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad6d9d640 10 monclient: get_auth_request con 0x7f5ad809fa20 auth_method 0 2024-09-15T01:33:29.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad6d9d640 0 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ad809fa20 0x7f5ad8141b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T01:33:29.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad6d9d640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ad809fa20 0x7f5ad8141b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient: ms_handle_reset current mon [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:29.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:29.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 1 -- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ad809fa20 msgr2=0x7f5ad8141b00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ad809fa20 0x7f5ad8141b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient: _add_conns ranks=[2,5,0,3,1,4] 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac4003590 0x7f5ac40059d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient(hunting): picked mon.noname-b con 0x7f5ac4003590 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5ac4006520 0x7f5ac40089a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient(hunting): picked mon.noname-h con 0x7f5ac4006520 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ac4009540 0x7f5ac400b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient(hunting): picked mon.noname-a con 0x7f5ac4009540 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 1 -- 172.21.15.204:0/706262649 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac400c5e0 con 0x7f5ac4009540 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 1 -- 172.21.15.204:0/706262649 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac400c720 con 0x7f5ac4003590 2024-09-15T01:33:29.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:29.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 1 -- 172.21.15.204:0/706262649 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac400c8e0 con 0x7f5ac4006520 2024-09-15T01:33:29.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:29.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5ac4006520 0x7f5ac40089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:29.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 10 monclient(hunting): get_auth_request con 0x7f5ac4006520 auth_method 0 2024-09-15T01:33:29.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:29.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:29.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:29.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:29.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:29.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:29.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 10 monclient(hunting): handle_auth_done global_id 4777 payload 293 2024-09-15T01:33:29.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad759e640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ac4009540 0x7f5ac400b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 1 -- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac4003590 msgr2=0x7f5ac40059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ac4003590 0x7f5ac40059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 1 -- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ac4009540 msgr2=0x7f5ac400b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ac4009540 0x7f5ac400b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 10 monclient: _finish_hunting 0 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 1 monclient: found mon.noname-h 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 1 -- 172.21.15.204:0/706262649 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac400cb60 con 0x7f5ac4006520 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad759e640 10 monclient: get_auth_request con 0x7f5ac4009540 auth_method 0 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad759e640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ac4009540 0x7f5ac400b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad759e640 1 -- 172.21.15.204:0/706262649 reap_dead start 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5ad659c640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5ac4006520 0x7f5ac40089a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5acc031000 tx=0x7f5acc032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:29.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 1 -- 172.21.15.204:0/706262649 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5acc032d60 con 0x7f5ac4006520 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.905+0000 7f5abeffd640 10 monclient: dump: 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:29.914 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5abeffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5abeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5add06f640 5 monclient: authenticate success, global_id 4777 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5abeffd640 1 -- 172.21.15.204:0/706262649 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5acc03c470 con 0x7f5ac4006520 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5abeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5add06f640 10 monclient: _renew_subs 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5abeffd640 1 -- 172.21.15.204:0/706262649 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5acc03ca60 con 0x7f5ac4006520 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5add06f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:29.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5add06f640 1 -- 172.21.15.204:0/706262649 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad8121300 con 0x7f5ac4006520 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5abeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5abeffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5abeffd640 10 monclient: dump: 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:29.916 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5ad559a640 4 set_mon_vals no callback set 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5add06f640 10 monclient: _renew_subs 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5add06f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.906+0000 7f5add06f640 1 -- 172.21.15.204:0/706262649 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad8120390 con 0x7f5ac4006520 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.907+0000 7f5abeffd640 1 -- 172.21.15.204:0/706262649 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5acc039040 con 0x7f5ac4006520 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.907+0000 7f5abeffd640 1 --2- 172.21.15.204:0/706262649 >> v2:172.21.15.204:6812/28210 conn(0x7f5ac403e060 0x7f5ac4040510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.907+0000 7f5abeffd640 1 -- 172.21.15.204:0/706262649 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 3936+0+0 (secure 0 0 0) 0x7f5acc040070 con 0x7f5ac4006520 2024-09-15T01:33:29.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.907+0000 7f5add06f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:29.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.907+0000 7f5add06f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:29.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.907+0000 7f5add06f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:29.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.907+0000 7f5ad6d9d640 1 --2- 172.21.15.204:0/706262649 >> v2:172.21.15.204:6812/28210 conn(0x7f5ac403e060 0x7f5ac4040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:29.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.907+0000 7f5add06f640 1 -- 172.21.15.204:0/706262649 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad8070b80 con 0x7f5ac4006520 2024-09-15T01:33:29.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.907+0000 7f5ad6d9d640 10 monclient: get_auth_request con 0x7f5ac403e060 auth_method 0 2024-09-15T01:33:29.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.907+0000 7f5ad6d9d640 1 --2- 172.21.15.204:0/706262649 >> v2:172.21.15.204:6812/28210 conn(0x7f5ac403e060 0x7f5ac4040510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8006fd0 tx=0x7f5ac8009120 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:29.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.911+0000 7f5abeffd640 1 -- 172.21.15.204:0/706262649 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5acc037080 con 0x7f5ac4006520 2024-09-15T01:33:29.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.911+0000 7f5abeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:29.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:29.911+0000 7f5abeffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:30.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.065+0000 7f5add06f640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:30.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.065+0000 7f5add06f640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:30.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.065+0000 7f5add06f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:30.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.065+0000 7f5add06f640 1 -- 172.21.15.204:0/706262649 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5ad8120520 con 0x7f5ac4006520 2024-09-15T01:33:30.515 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.513+0000 7f5abeffd640 1 -- 172.21.15.204:0/706262649 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' removed v43) v1 ==== 149+0+0 (secure 0 0 0) 0x7f5acc0433d0 con 0x7f5ac4006520 2024-09-15T01:33:30.515 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.513+0000 7f5abeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:30.515 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.513+0000 7f5abeffd640 10 monclient: _finish_command 2 = system:0 pool 'foo' removed 2024-09-15T01:33:30.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.637+0000 7f5add06f640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:30.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.637+0000 7f5add06f640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:30.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.637+0000 7f5add06f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:30.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.637+0000 7f5add06f640 1 -- 172.21.15.204:0/706262649 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5ad80008d0 con 0x7f5ac4006520 2024-09-15T01:33:30.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.637+0000 7f5abeffd640 1 -- 172.21.15.204:0/706262649 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' does not exist v43) v1 ==== 156+0+0 (secure 0 0 0) 0x7f5acc030340 con 0x7f5ac4006520 2024-09-15T01:33:30.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.637+0000 7f5abeffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:30.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.637+0000 7f5abeffd640 10 monclient: _finish_command 3 = system:0 pool 'foo' does not exist 2024-09-15T01:33:30.639 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'foo' does not exist 2024-09-15T01:33:30.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.639+0000 7f5add06f640 1 -- 172.21.15.204:0/706262649 >> v2:172.21.15.204:6812/28210 conn(0x7f5ac403e060 msgr2=0x7f5ac4040510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:30.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.639+0000 7f5add06f640 1 --2- 172.21.15.204:0/706262649 >> v2:172.21.15.204:6812/28210 conn(0x7f5ac403e060 0x7f5ac4040510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8006fd0 tx=0x7f5ac8009120 comp rx=0 tx=0).stop 2024-09-15T01:33:30.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.639+0000 7f5add06f640 10 monclient: shutdown 2024-09-15T01:33:30.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.639+0000 7f5add06f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:30.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.639+0000 7f5add06f640 1 -- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5ac4006520 msgr2=0x7f5ac40089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:30.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.639+0000 7f5add06f640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5ac4006520 0x7f5ac40089a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5acc031000 tx=0x7f5acc032000 comp rx=0 tx=0).stop 2024-09-15T01:33:30.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.640+0000 7f5add06f640 1 -- 172.21.15.204:0/706262649 shutdown_connections 2024-09-15T01:33:30.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.640+0000 7f5add06f640 1 --2- 172.21.15.204:0/706262649 >> v2:172.21.15.204:6812/28210 conn(0x7f5ac403e060 0x7f5ac4040510 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:30.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.640+0000 7f5add06f640 1 --2- 172.21.15.204:0/706262649 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5ac4006520 0x7f5ac40089a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:30.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.640+0000 7f5add06f640 1 -- 172.21.15.204:0/706262649 >> 172.21.15.204:0/706262649 conn(0x7f5ad806e860 msgr2=0x7f5ad80a5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:30.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.640+0000 7f5add06f640 1 -- 172.21.15.204:0/706262649 shutdown_connections 2024-09-15T01:33:30.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.640+0000 7f5add06f640 1 -- 172.21.15.204:0/706262649 wait complete. 2024-09-15T01:33:30.650 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete foo foo --yes-i-really-really-mean-it 2024-09-15T01:33:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.724+0000 7fa4992b8640 10 monclient: get_monmap_and_config 2024-09-15T01:33:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.724+0000 7fa4992b8640 10 monclient: build_initial_monmap 2024-09-15T01:33:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.724+0000 7fa4992b8640 10 monclient: monmap: 2024-09-15T01:33:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:30.725298+0000 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:30.725298+0000 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:30.728 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:30.728 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:30.728 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:30.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 1 Processor -- start 2024-09-15T01:33:30.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 1 -- start start 2024-09-15T01:33:30.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 10 monclient: init 2024-09-15T01:33:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 10 monclient: _add_conns ranks=[2,1,4,3,9,6,7,0,5,10,8] 2024-09-15T01:33:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4940a2cd0 0x7fa494132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 10 monclient(hunting): picked mon.noname-b con 0x7fa4940a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa49409fa20 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa494137c80 0x7fa49413a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 10 monclient(hunting): picked mon.noname-c con 0x7fa494137c80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4940648a0 con 0x7fa49409fa20 2024-09-15T01:33:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4940a30a0 con 0x7fa4940a2cd0 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4940a1db0 con 0x7fa494137c80 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.725+0000 7fa4992b8640 10 monclient(hunting): authenticate will time out at 826.367922s 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4927fc640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa49409fa20 0x7fa49409fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:58940/0 (socket says 172.21.15.204:58940) 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4927fc640 1 -- 172.21.15.204:0/2556710707 learned_addr learned my addr 172.21.15.204:0/2556710707 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa494137c80 0x7fa49413a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4940a2cd0 0x7fa494132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133456282 0 0) 0x7fa4940648a0 con 0x7fa49409fa20 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient(hunting): my global_id is 4906 2024-09-15T01:33:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4800034e0 con 0x7fa49409fa20 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): get_auth_request con 0x7fa4940a2cd0 auth_method 0 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 10 monclient(hunting): get_auth_request con 0x7fa494137c80 auth_method 0 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 18780399 0 0) 0x7fa4800034e0 con 0x7fa49409fa20 2024-09-15T01:33:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa494137c80 msgr2=0x7fa49413a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa494137c80 0x7fa49413a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4940a2cd0 msgr2=0x7fa494132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4940a2cd0 0x7fa494132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient: _finish_hunting 0 2024-09-15T01:33:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:33:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:33:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49409dc50 con 0x7fa49409fa20 2024-09-15T01:33:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4992b8640 5 monclient: authenticate success, global_id 4906 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fa47c0018f0 con 0x7fa49409fa20 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient: dump: 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 >> v1:172.21.15.204:6789/0 conn(0x7fa49409fa20 legacy=0x7fa49409fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient: _add_conns ranks=[4,6,8,0,7,3,5,1,2] 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa494137c80 0x7fa480004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient(hunting): picked mon.c con 0x7fa494137c80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa480004330 0x7fa480004d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient(hunting): picked mon.d con 0x7fa480004330 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa4800053b0 0x7fa48000ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient(hunting): picked mon.e con 0x7fa4800053b0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4940a30a0 con 0x7fa494137c80 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa4940a1db0 con 0x7fa480004330 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa48000c380 con 0x7fa4800053b0 2024-09-15T01:33:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa491ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4992b8640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4927fc640 1 -- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa480004330 msgr2=0x7fa480004d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4927fc640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa480004330 0x7fa480004d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa494137c80 0x7fa480004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): get_auth_request con 0x7fa494137c80 auth_method 0 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa4800053b0 0x7fa48000ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa492ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 10 monclient(hunting): get_auth_request con 0x7fa4800053b0 auth_method 0 2024-09-15T01:33:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.726+0000 7fa4937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa4937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa4937fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 10 monclient(hunting): handle_auth_done global_id 4906 payload 293 2024-09-15T01:33:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 1 -- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa4800053b0 msgr2=0x7fa48000ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa4800053b0 0x7fa48000ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 1 -- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa480004330 msgr2=0x7fa480004d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa480004330 0x7fa480004d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 1 monclient: found mon.c 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 1 -- 172.21.15.204:0/2556710707 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4800034e0 con 0x7fa494137c80 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa492ffd640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa494137c80 0x7fa480004780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00e210 tx=0x7fa48c00b040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=4afb808bdf7c1ffb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa48c02c040 con 0x7fa494137c80 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa491ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa491ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa491ffb640 10 monclient: dump: 2024-09-15T01:33:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:30.751 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:30.751 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:30.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa4992b8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:30.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c00f040 con 0x7fa494137c80 2024-09-15T01:33:30.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa491ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:30.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa4992b8640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:30.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa4992b8640 4 set_mon_vals no callback set 2024-09-15T01:33:30.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa491ffb640 1 -- 172.21.15.204:0/2556710707 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa48c030d00 con 0x7fa494137c80 2024-09-15T01:33:30.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa491ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:30.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa491ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:30.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.727+0000 7fa491ffb640 10 monclient: dump: 2024-09-15T01:33:30.751 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:30.752 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:30.753 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.728+0000 7fa4992b8640 10 monclient: shutdown 2024-09-15T01:33:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.728+0000 7fa4992b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.728+0000 7fa4992b8640 1 -- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa494137c80 msgr2=0x7fa480004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.728+0000 7fa4992b8640 1 --2- 172.21.15.204:0/2556710707 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa494137c80 0x7fa480004780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00e210 tx=0x7fa48c00b040 comp rx=0 tx=0).stop 2024-09-15T01:33:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.728+0000 7fa4937fe640 1 -- 172.21.15.204:0/2556710707 reap_dead start 2024-09-15T01:33:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.728+0000 7fa4992b8640 1 -- 172.21.15.204:0/2556710707 shutdown_connections 2024-09-15T01:33:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.728+0000 7fa4992b8640 1 -- 172.21.15.204:0/2556710707 >> 172.21.15.204:0/2556710707 conn(0x7fa49406e860 msgr2=0x7fa49406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.728+0000 7fa4992b8640 1 -- 172.21.15.204:0/2556710707 shutdown_connections 2024-09-15T01:33:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.728+0000 7fa4992b8640 1 -- 172.21.15.204:0/2556710707 wait complete. 2024-09-15T01:33:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient: build_initial_monmap 2024-09-15T01:33:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient: monmap: 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 1 Processor -- start 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 1 -- start start 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient: init 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient: _add_conns ranks=[2,0,7,4,5,3,1,6,8] 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa494137c80 0x7fa494134270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient(hunting): picked mon.noname-b con 0x7fa494137c80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4941347b0 0x7fa494134b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient(hunting): picked mon.noname-a con 0x7fa4941347b0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa4941350c0 0x7fa494148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient(hunting): picked mon.noname-i con 0x7fa4941350c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49409bb30 con 0x7fa4941347b0 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941368d0 con 0x7fa494137c80 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941354f0 con 0x7fa4941350c0 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.729+0000 7fa4992b8640 10 monclient(hunting): authenticate will time out at 826.371923s 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa4941350c0 0x7fa494148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4927fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4941347b0 0x7fa494134b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa4941350c0 0x7fa494148120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:45688/0 (socket says 172.21.15.204:45688) 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 1 -- 172.21.15.204:0/2551524891 learned_addr learned my addr 172.21.15.204:0/2551524891 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 10 monclient(hunting): get_auth_request con 0x7fa4941350c0 auth_method 0 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4927fc640 10 monclient(hunting): get_auth_request con 0x7fa4941347b0 auth_method 0 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4927fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4927fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4927fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa492ffd640 1 --2- 172.21.15.204:0/2551524891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa494137c80 0x7fa494134270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa492ffd640 10 monclient(hunting): get_auth_request con 0x7fa494137c80 auth_method 0 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa492ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa492ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa492ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4927fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4927fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4927fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa492ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa492ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa492ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 10 monclient(hunting): handle_auth_done global_id 4786 payload 824 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 1 -- 172.21.15.204:0/2551524891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa494137c80 msgr2=0x7fa494134270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 1 --2- 172.21.15.204:0/2551524891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa494137c80 0x7fa494134270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 1 -- 172.21.15.204:0/2551524891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4941347b0 msgr2=0x7fa494134b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 1 --2- 172.21.15.204:0/2551524891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4941347b0 0x7fa494134b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 1 monclient: found mon.noname-i 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 1 -- 172.21.15.204:0/2551524891 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa494136280 con 0x7fa4941350c0 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4927fc640 1 --2- 172.21.15.204:0/2551524891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4941347b0 0x7fa494134b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4937fe640 1 --2- 172.21.15.204:0/2551524891 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa4941350c0 0x7fa494148120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa484006e70 tx=0x7fa4840090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa48affd640 1 -- 172.21.15.204:0/2551524891 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4840148c0 con 0x7fa4941350c0 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa48affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa48affd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa48affd640 10 monclient: dump: 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:30.760 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa48affd640 10 monclient: _finish_auth 0 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa48affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa48affd640 1 -- 172.21.15.204:0/2551524891 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa484014a60 con 0x7fa4941350c0 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4992b8640 5 monclient: authenticate success, global_id 4786 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa48affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa48affd640 1 -- 172.21.15.204:0/2551524891 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa484014bd0 con 0x7fa4941350c0 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa48affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa48affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa48affd640 10 monclient: dump: 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:30.761 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4917fa640 4 set_mon_vals no callback set 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4992b8640 10 monclient: _renew_subs 2024-09-15T01:33:30.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4992b8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4992b8640 1 -- 172.21.15.204:0/2551524891 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49409c590 con 0x7fa4941350c0 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4992b8640 10 monclient: _renew_subs 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4992b8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.730+0000 7fa4992b8640 1 -- 172.21.15.204:0/2551524891 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49409d450 con 0x7fa4941350c0 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.731+0000 7fa48affd640 1 -- 172.21.15.204:0/2551524891 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa484014d30 con 0x7fa4941350c0 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.731+0000 7fa4992b8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.731+0000 7fa4992b8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.731+0000 7fa4992b8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.731+0000 7fa4992b8640 1 -- 172.21.15.204:0/2551524891 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa494137650 con 0x7fa4941350c0 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.731+0000 7fa48affd640 1 --2- 172.21.15.204:0/2551524891 >> v2:172.21.15.204:6812/28210 conn(0x7fa4740397b0 0x7fa47403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.731+0000 7fa48affd640 1 -- 172.21.15.204:0/2551524891 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 3594+0+0 (secure 0 0 0) 0x7fa48404afb0 con 0x7fa4941350c0 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.731+0000 7fa492ffd640 1 --2- 172.21.15.204:0/2551524891 >> v2:172.21.15.204:6812/28210 conn(0x7fa4740397b0 0x7fa47403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.731+0000 7fa492ffd640 10 monclient: get_auth_request con 0x7fa4740397b0 auth_method 0 2024-09-15T01:33:30.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.732+0000 7fa492ffd640 1 --2- 172.21.15.204:0/2551524891 >> v2:172.21.15.204:6812/28210 conn(0x7fa4740397b0 0x7fa47403bc60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa48c035000 tx=0x7fa48c036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:30.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.735+0000 7fa48affd640 1 -- 172.21.15.204:0/2551524891 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa48407ce40 con 0x7fa4941350c0 2024-09-15T01:33:30.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.736+0000 7fa48affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:30.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.736+0000 7fa48affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:30.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.908+0000 7fa4992b8640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:30.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.908+0000 7fa4992b8640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:30.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.908+0000 7fa4992b8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:30.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.908+0000 7fa4992b8640 1 -- 172.21.15.204:0/2551524891 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa494096f60 con 0x7fa4941350c0 2024-09-15T01:33:30.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.908+0000 7fa48affd640 1 -- 172.21.15.204:0/2551524891 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' does not exist v43) v1 ==== 156+0+0 (secure 0 0 0) 0x7fa494137650 con 0x7fa4941350c0 2024-09-15T01:33:30.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.908+0000 7fa48affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:30.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:30.908+0000 7fa48affd640 10 monclient: _finish_command 2 = system:0 pool 'foo' does not exist 2024-09-15T01:33:31.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.000+0000 7fa4992b8640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.000+0000 7fa4992b8640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.000+0000 7fa4992b8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:31.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.000+0000 7fa4992b8640 1 -- 172.21.15.204:0/2551524891 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa494137650 con 0x7fa4941350c0 2024-09-15T01:33:31.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.001+0000 7fa48affd640 1 -- 172.21.15.204:0/2551524891 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' does not exist v43) v1 ==== 156+0+0 (secure 0 0 0) 0x7fa494137650 con 0x7fa4941350c0 2024-09-15T01:33:31.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.001+0000 7fa48affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.001+0000 7fa48affd640 10 monclient: _finish_command 3 = system:0 pool 'foo' does not exist 2024-09-15T01:33:31.003 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'foo' does not exist 2024-09-15T01:33:31.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 1 -- 172.21.15.204:0/2551524891 >> v2:172.21.15.204:6812/28210 conn(0x7fa4740397b0 msgr2=0x7fa47403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 1 --2- 172.21.15.204:0/2551524891 >> v2:172.21.15.204:6812/28210 conn(0x7fa4740397b0 0x7fa47403bc60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa48c035000 tx=0x7fa48c036000 comp rx=0 tx=0).stop 2024-09-15T01:33:31.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 10 monclient: shutdown 2024-09-15T01:33:31.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:31.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 1 -- 172.21.15.204:0/2551524891 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa4941350c0 msgr2=0x7fa494148120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 1 --2- 172.21.15.204:0/2551524891 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa4941350c0 0x7fa494148120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa484006e70 tx=0x7fa4840090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 1 -- 172.21.15.204:0/2551524891 shutdown_connections 2024-09-15T01:33:31.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 1 --2- 172.21.15.204:0/2551524891 >> v2:172.21.15.204:6812/28210 conn(0x7fa4740397b0 0x7fa47403bc60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 1 --2- 172.21.15.204:0/2551524891 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa4941350c0 0x7fa494148120 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 1 --2- 172.21.15.204:0/2551524891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4941347b0 0x7fa494134b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 1 --2- 172.21.15.204:0/2551524891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa494137c80 0x7fa494134270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 1 -- 172.21.15.204:0/2551524891 >> 172.21.15.204:0/2551524891 conn(0x7fa49406e860 msgr2=0x7fa49412da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:31.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 1 -- 172.21.15.204:0/2551524891 shutdown_connections 2024-09-15T01:33:31.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.003+0000 7fa4992b8640 1 -- 172.21.15.204:0/2551524891 wait complete. 2024-09-15T01:33:31.013 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete fooo fooo --yes-i-really-really-mean-it 2024-09-15T01:33:31.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.087+0000 7f997af9d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:31.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.087+0000 7f997af9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:31.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.087+0000 7f997af9d640 10 monclient: monmap: 2024-09-15T01:33:31.089 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:31.089 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.089 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:31.087649+0000 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:31.087649+0000 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:31.090 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:31.091 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.087+0000 7f997af9d640 1 Processor -- start 2024-09-15T01:33:31.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.087+0000 7f997af9d640 1 -- start start 2024-09-15T01:33:31.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.087+0000 7f997af9d640 10 monclient: init 2024-09-15T01:33:31.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:31.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 10 monclient: _add_conns ranks=[10,7,2,1,3,4,8,5,9,6,0] 2024-09-15T01:33:31.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f99740a4590 0x7f997409e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 10 monclient(hunting): picked mon.noname-f con 0x7f99740a4590 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:31.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f997406edd0 0x7f997409ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 10 monclient(hunting): picked mon.noname-i con 0x7f997406edd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:31.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997409f1d0 0x7f99741307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 10 monclient(hunting): picked mon.noname-b con 0x7f997409f1d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:31.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997409f600 con 0x7f99740a4590 2024-09-15T01:33:31.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99740a4960 con 0x7f997409f1d0 2024-09-15T01:33:31.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f997406dba0 con 0x7f997406edd0 2024-09-15T01:33:31.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f997af9d640 10 monclient(hunting): authenticate will time out at 826.730452s 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f997406edd0 0x7f997409ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9978d39640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f99740a4590 0x7f997409e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f997406edd0 0x7f997409ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:45694/0 (socket says 172.21.15.204:45694) 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 1 -- 172.21.15.204:0/687683535 learned_addr learned my addr 172.21.15.204:0/687683535 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 10 monclient(hunting): get_auth_request con 0x7f997406edd0 auth_method 0 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9978d39640 10 monclient(hunting): get_auth_request con 0x7f99740a4590 auth_method 0 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9978d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9978d39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9978d39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9978d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9978d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9978d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 10 monclient(hunting): handle_auth_done global_id 4795 payload 274 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 1 -- 172.21.15.204:0/687683535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997409f1d0 msgr2=0x7f99741307a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 1 --2- 172.21.15.204:0/687683535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997409f1d0 0x7f99741307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 1 -- 172.21.15.204:0/687683535 >> v2:172.21.15.204:3300/0 conn(0x7f99740a4590 msgr2=0x7f997409e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 1 --2- 172.21.15.204:0/687683535 >> v2:172.21.15.204:3300/0 conn(0x7f99740a4590 0x7f997409e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 1 monclient: found mon.noname-i 2024-09-15T01:33:31.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9973fff640 1 -- 172.21.15.204:0/687683535 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740648a0 con 0x7f997406edd0 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.088+0000 7f9978d39640 1 --2- 172.21.15.204:0/687683535 >> v2:172.21.15.204:3300/0 conn(0x7f99740a4590 0x7f997409e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f9973fff640 1 --2- 172.21.15.204:0/687683535 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f997406edd0 0x7f997409ec90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f995c009910 tx=0x7f995c0091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=2a246fcad3bd9e8f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f99737fe640 1 -- 172.21.15.204:0/687683535 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f995c02ac90 con 0x7f997406edd0 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f99737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f99737fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f99737fe640 10 monclient: dump: 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.103 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.104 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.104 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.104 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.104 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.104 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.104 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.104 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.104 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.104 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.104 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f99737fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:31.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f99737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:31.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f99737fe640 1 -- 172.21.15.204:0/687683535 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c02ae10 con 0x7f997406edd0 2024-09-15T01:33:31.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f997af9d640 5 monclient: authenticate success, global_id 4795 2024-09-15T01:33:31.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f997af9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:31.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f99737fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:31.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f997af9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:31.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f99737fe640 1 -- 172.21.15.204:0/687683535 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f995c031ca0 con 0x7f997406edd0 2024-09-15T01:33:31.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f99737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f997af9d640 4 set_mon_vals no callback set 2024-09-15T01:33:31.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f99737fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:31.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.089+0000 7f99737fe640 10 monclient: dump: 2024-09-15T01:33:31.105 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.105 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.106 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.106 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.106 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.106 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.106 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.106 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.106 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.106 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.106 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.106 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.106 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 10 monclient: shutdown 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 1 -- 172.21.15.204:0/687683535 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f997406edd0 msgr2=0x7f997409ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 1 --2- 172.21.15.204:0/687683535 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f997406edd0 0x7f997409ec90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f995c009910 tx=0x7f995c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 1 -- 172.21.15.204:0/687683535 shutdown_connections 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 1 --2- 172.21.15.204:0/687683535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f997409f1d0 0x7f99741307a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 1 --2- 172.21.15.204:0/687683535 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f997406edd0 0x7f997409ec90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 1 --2- 172.21.15.204:0/687683535 >> v2:172.21.15.204:3300/0 conn(0x7f99740a4590 0x7f997409e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 1 -- 172.21.15.204:0/687683535 >> 172.21.15.204:0/687683535 conn(0x7f9974067700 msgr2=0x7f99740a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 1 -- 172.21.15.204:0/687683535 shutdown_connections 2024-09-15T01:33:31.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 1 -- 172.21.15.204:0/687683535 wait complete. 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.090+0000 7f997af9d640 10 monclient: monmap: 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:31.108 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 1 Processor -- start 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 1 -- start start 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 10 monclient: init 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 10 monclient: _add_conns ranks=[1,3,4,7,2,6,8,0,5] 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997406edd0 0x7f9974130750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 10 monclient(hunting): picked mon.noname-f con 0x7f997406edd0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997409f1d0 0x7f997411d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 10 monclient(hunting): picked mon.noname-g con 0x7f997409f1d0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99740a4590 0x7f997411d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 10 monclient(hunting): picked mon.noname-c con 0x7f99740a4590 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997406d070 con 0x7f997406edd0 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997406cc80 con 0x7f997409f1d0 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f997411ed70 con 0x7f99740a4590 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997af9d640 10 monclient(hunting): authenticate will time out at 826.733799s 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f9973fff640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997409f1d0 msgr2=0x7f997411d400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f9973fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997409f1d0 0x7f997411d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f9978d39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997406edd0 0x7f9974130750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f9978d39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997406edd0 0x7f9974130750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33236/0 (socket says 172.21.15.204:33236) 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f9978d39640 1 -- 172.21.15.204:0/2740809950 learned_addr learned my addr 172.21.15.204:0/2740809950 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f9978d39640 10 monclient(hunting): get_auth_request con 0x7f997406edd0 auth_method 0 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f9978d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f9978d39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f9978d39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f9978d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f9978d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f9978d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.091+0000 7f997953a640 1 --2- 172.21.15.204:0/2740809950 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99740a4590 0x7f997411d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997953a640 10 monclient(hunting): get_auth_request con 0x7f99740a4590 auth_method 0 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997953a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997953a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997953a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997953a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997953a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997953a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9978d39640 10 monclient(hunting): handle_auth_done global_id 4924 payload 824 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9978d39640 1 -- 172.21.15.204:0/2740809950 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99740a4590 msgr2=0x7f997411d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9978d39640 1 --2- 172.21.15.204:0/2740809950 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99740a4590 0x7f997411d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9978d39640 1 -- 172.21.15.204:0/2740809950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997409f1d0 msgr2=0x7f997411d400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9978d39640 1 --2- 172.21.15.204:0/2740809950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997409f1d0 0x7f997411d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9978d39640 10 monclient: _finish_hunting 0 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9978d39640 1 monclient: found mon.noname-f 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9978d39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9978d39640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9978d39640 1 -- 172.21.15.204:0/2740809950 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9964009a20 con 0x7f997406edd0 2024-09-15T01:33:31.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9978d39640 1 --2- 172.21.15.204:0/2740809950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997406edd0 0x7f9974130750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f996400c0c0 tx=0x7f996400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:31.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9970ff9640 1 -- 172.21.15.204:0/2740809950 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f996400cea0 con 0x7f997406edd0 2024-09-15T01:33:31.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9970ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9970ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:31.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9970ff9640 10 monclient: dump: 2024-09-15T01:33:31.113 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.114 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9970ff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9970ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9970ff9640 1 -- 172.21.15.204:0/2740809950 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996400f040 con 0x7f997406edd0 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997af9d640 5 monclient: authenticate success, global_id 4924 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9970ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9970ff9640 1 -- 172.21.15.204:0/2740809950 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9964014d80 con 0x7f997406edd0 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997af9d640 10 monclient: _renew_subs 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997af9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997af9d640 1 -- 172.21.15.204:0/2740809950 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995c009450 con 0x7f997406edd0 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9972ffd640 4 set_mon_vals no callback set 2024-09-15T01:33:31.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9970ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9970ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:31.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f9970ff9640 10 monclient: dump: 2024-09-15T01:33:31.117 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.117 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.117 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.117 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.117 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.117 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.117 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.117 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.117 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.117 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.117 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997af9d640 10 monclient: _renew_subs 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997af9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.092+0000 7f997af9d640 1 -- 172.21.15.204:0/2740809950 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99741208a0 con 0x7f997406edd0 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.093+0000 7f9970ff9640 1 -- 172.21.15.204:0/2740809950 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9964003020 con 0x7f997406edd0 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.093+0000 7f997af9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.093+0000 7f997af9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.093+0000 7f997af9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.093+0000 7f997af9d640 1 -- 172.21.15.204:0/2740809950 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9974070b80 con 0x7f997406edd0 2024-09-15T01:33:31.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.093+0000 7f9970ff9640 1 --2- 172.21.15.204:0/2740809950 >> v2:172.21.15.204:6812/28210 conn(0x7f99540397b0 0x7f995403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.093+0000 7f9970ff9640 1 -- 172.21.15.204:0/2740809950 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 3594+0+0 (secure 0 0 0) 0x7f996404b9a0 con 0x7f997406edd0 2024-09-15T01:33:31.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.093+0000 7f9973fff640 1 --2- 172.21.15.204:0/2740809950 >> v2:172.21.15.204:6812/28210 conn(0x7f99540397b0 0x7f995403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.093+0000 7f9973fff640 10 monclient: get_auth_request con 0x7f99540397b0 auth_method 0 2024-09-15T01:33:31.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.094+0000 7f9973fff640 1 --2- 172.21.15.204:0/2740809950 >> v2:172.21.15.204:6812/28210 conn(0x7f99540397b0 0x7f995403bc60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f995c034000 tx=0x7f995c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:31.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.098+0000 7f9970ff9640 1 -- 172.21.15.204:0/2740809950 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f996401aa80 con 0x7f997406edd0 2024-09-15T01:33:31.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.098+0000 7f9970ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:31.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.098+0000 7f9970ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:31.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.253+0000 7f997af9d640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.253+0000 7f997af9d640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.253+0000 7f997af9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.253+0000 7f997af9d640 1 -- 172.21.15.204:0/2740809950 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99740a04c0 con 0x7f997406edd0 2024-09-15T01:33:31.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.254+0000 7f9970ff9640 1 -- 172.21.15.204:0/2740809950 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v43) v1 ==== 159+0+0 (secure 0 0 0) 0x7f9964018030 con 0x7f997406edd0 2024-09-15T01:33:31.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.254+0000 7f9970ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.254+0000 7f9970ff9640 10 monclient: _finish_command 2 = system:0 pool 'fooo' does not exist 2024-09-15T01:33:31.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.347+0000 7f997af9d640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.347+0000 7f997af9d640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.347+0000 7f997af9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.347+0000 7f997af9d640 1 -- 172.21.15.204:0/2740809950 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99740008d0 con 0x7f997406edd0 2024-09-15T01:33:31.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.348+0000 7f9970ff9640 1 -- 172.21.15.204:0/2740809950 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v43) v1 ==== 159+0+0 (secure 0 0 0) 0x7f9964018020 con 0x7f997406edd0 2024-09-15T01:33:31.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.348+0000 7f9970ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.348+0000 7f9970ff9640 10 monclient: _finish_command 3 = system:0 pool 'fooo' does not exist 2024-09-15T01:33:31.350 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'fooo' does not exist 2024-09-15T01:33:31.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 1 -- 172.21.15.204:0/2740809950 >> v2:172.21.15.204:6812/28210 conn(0x7f99540397b0 msgr2=0x7f995403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 1 --2- 172.21.15.204:0/2740809950 >> v2:172.21.15.204:6812/28210 conn(0x7f99540397b0 0x7f995403bc60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f995c034000 tx=0x7f995c032000 comp rx=0 tx=0).stop 2024-09-15T01:33:31.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 10 monclient: shutdown 2024-09-15T01:33:31.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:31.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 1 -- 172.21.15.204:0/2740809950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997406edd0 msgr2=0x7f9974130750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 1 --2- 172.21.15.204:0/2740809950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997406edd0 0x7f9974130750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f996400c0c0 tx=0x7f996400b040 comp rx=0 tx=0).stop 2024-09-15T01:33:31.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 1 -- 172.21.15.204:0/2740809950 shutdown_connections 2024-09-15T01:33:31.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 1 --2- 172.21.15.204:0/2740809950 >> v2:172.21.15.204:6812/28210 conn(0x7f99540397b0 0x7f995403bc60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 1 --2- 172.21.15.204:0/2740809950 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99740a4590 0x7f997411d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 1 --2- 172.21.15.204:0/2740809950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997409f1d0 0x7f997411d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.353 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 1 --2- 172.21.15.204:0/2740809950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997406edd0 0x7f9974130750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.353 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 1 -- 172.21.15.204:0/2740809950 >> 172.21.15.204:0/2740809950 conn(0x7f9974067700 msgr2=0x7f99740a24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:31.353 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 1 -- 172.21.15.204:0/2740809950 shutdown_connections 2024-09-15T01:33:31.353 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.350+0000 7f997af9d640 1 -- 172.21.15.204:0/2740809950 wait complete. 2024-09-15T01:33:31.360 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete fooo fooo --yes-i-really-really-mean-it 2024-09-15T01:33:31.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.433+0000 7f1eb00af640 10 monclient: get_monmap_and_config 2024-09-15T01:33:31.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.433+0000 7f1eb00af640 10 monclient: build_initial_monmap 2024-09-15T01:33:31.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.434+0000 7f1eb00af640 10 monclient: monmap: 2024-09-15T01:33:31.436 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:31.436 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.436 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:31.434508+0000 2024-09-15T01:33:31.436 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:31.434508+0000 2024-09-15T01:33:31.436 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.434+0000 7f1eb00af640 1 Processor -- start 2024-09-15T01:33:31.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.434+0000 7f1eb00af640 1 -- start start 2024-09-15T01:33:31.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.434+0000 7f1eb00af640 10 monclient: init 2024-09-15T01:33:31.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:31.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 10 monclient: _add_conns ranks=[1,9,6,10,4,8,2,5,7,3,0] 2024-09-15T01:33:31.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1ea8067130 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:31.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1ea8067500 0x7f1ea80678d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 10 monclient(hunting): picked mon.noname-a con 0x7f1ea8067500 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:31.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1ea8135cb0 0x7f1ea8138090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 10 monclient(hunting): picked mon.noname-d con 0x7f1ea8135cb0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:31.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea80648a0 con 0x7f1ea8067130 2024-09-15T01:33:31.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea8099ab0 con 0x7f1ea8067500 2024-09-15T01:33:31.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8099e00 con 0x7f1ea8135cb0 2024-09-15T01:33:31.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:31.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eb00af640 10 monclient(hunting): authenticate will time out at 827.077246s 2024-09-15T01:33:31.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eade4b640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f1ea8067130 0x7f1ea809a1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:58946/0 (socket says 172.21.15.204:58946) 2024-09-15T01:33:31.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eade4b640 1 -- 172.21.15.204:0/1647974786 learned_addr learned my addr 172.21.15.204:0/1647974786 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:31.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eae64c640 1 -- 172.21.15.204:0/1647974786 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1ea8135cb0 msgr2=0x7f1ea8138090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:31.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eae64c640 1 --2- 172.21.15.204:0/1647974786 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1ea8135cb0 0x7f1ea8138090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:31.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237985718 0 0) 0x7f1ea80648a0 con 0x7f1ea8067130 2024-09-15T01:33:31.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:31.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient(hunting): my global_id is 4933 2024-09-15T01:33:31.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e940034e0 con 0x7f1ea8067130 2024-09-15T01:33:31.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1ead64a640 1 --2- 172.21.15.204:0/1647974786 >> v2:172.21.15.178:3300/0 conn(0x7f1ea8067500 0x7f1ea80678d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1ead64a640 10 monclient(hunting): get_auth_request con 0x7f1ea8067500 auth_method 0 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1ead64a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1ead64a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1ead64a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1ead64a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1ead64a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1ead64a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2412678348 0 0) 0x7f1e940034e0 con 0x7f1ea8067130 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1ea8135cb0 msgr2=0x7f1ea8138090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 --2- 172.21.15.204:0/1647974786 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1ea8135cb0 0x7f1ea8138090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 >> v2:172.21.15.178:3300/0 conn(0x7f1ea8067500 msgr2=0x7f1ea80678d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 --2- 172.21.15.204:0/1647974786 >> v2:172.21.15.178:3300/0 conn(0x7f1ea8067500 0x7f1ea80678d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient: _finish_hunting 0 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:33:31.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea809c710 con 0x7f1ea8067130 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient: _finish_auth 0 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f1ea40018f0 con 0x7f1ea8067130 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient: dump: 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.450 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 >> v1:172.21.15.204:6789/0 conn(0x7f1ea8067130 legacy=0x7f1ea809a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient: _add_conns ranks=[3,1,6,0,5,7,8,2,4] 2024-09-15T01:33:31.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 --2- 172.21.15.204:0/1647974786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e94004250 0x7f1e94004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient(hunting): picked mon.g con 0x7f1e94004250 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:31.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 --2- 172.21.15.204:0/1647974786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e94004d10 0x7f1e94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient(hunting): picked mon.f con 0x7f1e94004d10 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:31.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 --2- 172.21.15.204:0/1647974786 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1ea8135cb0 0x7f1e9400bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient(hunting): picked mon.d con 0x7f1ea8135cb0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:31.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8099e00 con 0x7f1e94004d10 2024-09-15T01:33:31.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eade4b640 1 -- 172.21.15.204:0/1647974786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e94004250 msgr2=0x7f1e94004780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:31.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9400c400 con 0x7f1e94004250 2024-09-15T01:33:31.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eade4b640 1 --2- 172.21.15.204:0/1647974786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e94004250 0x7f1e94004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:31.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9400c570 con 0x7f1ea8135cb0 2024-09-15T01:33:31.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:31.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eace49640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:33:31.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.435+0000 7f1eae64c640 1 -- 172.21.15.204:0/1647974786 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1ea8135cb0 msgr2=0x7f1e9400bcf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:31.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 1 --2- 172.21.15.204:0/1647974786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e94004d10 0x7f1e94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1eae64c640 1 --2- 172.21.15.204:0/1647974786 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1ea8135cb0 0x7f1e9400bcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:31.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 10 monclient(hunting): get_auth_request con 0x7f1e94004d10 auth_method 0 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 10 monclient(hunting): handle_auth_done global_id 4933 payload 293 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 1 -- 172.21.15.204:0/1647974786 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1ea8135cb0 msgr2=0x7f1e9400bcf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 1 --2- 172.21.15.204:0/1647974786 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1ea8135cb0 0x7f1e9400bcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 1 -- 172.21.15.204:0/1647974786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e94004250 msgr2=0x7f1e94004780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 1 --2- 172.21.15.204:0/1647974786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e94004250 0x7f1e94004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 10 monclient: _finish_hunting 0 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 1 monclient: found mon.f 2024-09-15T01:33:31.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 1 -- 172.21.15.204:0/1647974786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e940034e0 con 0x7f1e94004d10 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 10 monclient: _finish_auth 0 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1eb00af640 5 monclient: authenticate success, global_id 4933 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1eb00af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.436+0000 7f1ead64a640 1 --2- 172.21.15.204:0/1647974786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e94004d10 0x7f1e94005180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e98006e70 tx=0x7f1e980090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17263ffcc3cd4f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1e98031880 con 0x7f1e94004d10 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eace49640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eace49640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eace49640 10 monclient: dump: 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.455 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.456 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eb00af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9802d050 con 0x7f1e94004d10 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eace49640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eb00af640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eace49640 1 -- 172.21.15.204:0/1647974786 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1e98031e80 con 0x7f1e94004d10 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eb00af640 4 set_mon_vals no callback set 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eace49640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eace49640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eace49640 10 monclient: dump: 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.457 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eb00af640 10 monclient: shutdown 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eb00af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:31.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eb00af640 1 -- 172.21.15.204:0/1647974786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e94004d10 msgr2=0x7f1e94005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eb00af640 1 --2- 172.21.15.204:0/1647974786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e94004d10 0x7f1e94005180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e98006e70 tx=0x7f1e980090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eae64c640 1 -- 172.21.15.204:0/1647974786 reap_dead start 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eb00af640 1 -- 172.21.15.204:0/1647974786 shutdown_connections 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eb00af640 1 -- 172.21.15.204:0/1647974786 >> 172.21.15.204:0/1647974786 conn(0x7f1ea80a2ce0 msgr2=0x7f1ea80a0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eb00af640 1 -- 172.21.15.204:0/1647974786 shutdown_connections 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.437+0000 7f1eb00af640 1 -- 172.21.15.204:0/1647974786 wait complete. 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient: build_initial_monmap 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient: monmap: 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:31.459 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 1 Processor -- start 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 1 -- start start 2024-09-15T01:33:31.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient: init 2024-09-15T01:33:31.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:31.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient: _add_conns ranks=[6,1,8,0,5,7,2,3,4] 2024-09-15T01:33:31.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1e94004d10 0x7f1ea806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient(hunting): picked mon.noname-d con 0x7f1e94004d10 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:31.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea8119fb0 0x7f1ea81188f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient(hunting): picked mon.noname-f con 0x7f1ea8119fb0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:31.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1ea8118e30 0x7f1ea8119200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient(hunting): picked mon.noname-e con 0x7f1ea8118e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:31.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea809bc30 con 0x7f1ea8119fb0 2024-09-15T01:33:31.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8119740 con 0x7f1e94004d10 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea811b870 con 0x7f1ea8118e30 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eb00af640 10 monclient(hunting): authenticate will time out at 827.080869s 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1ead64a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea8119fb0 0x7f1ea81188f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eade4b640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1e94004d10 msgr2=0x7f1ea806ebc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1ead64a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea8119fb0 0x7f1ea81188f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33250/0 (socket says 172.21.15.204:33250) 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1eade4b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1e94004d10 0x7f1ea806ebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1ead64a640 1 -- 172.21.15.204:0/1789202608 learned_addr learned my addr 172.21.15.204:0/1789202608 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1ead64a640 10 monclient(hunting): get_auth_request con 0x7f1ea8119fb0 auth_method 0 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1ead64a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1ead64a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.438+0000 7f1ead64a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:31.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1eae64c640 1 --2- 172.21.15.204:0/1789202608 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1ea8118e30 0x7f1ea8119200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1eae64c640 10 monclient(hunting): get_auth_request con 0x7f1ea8118e30 auth_method 0 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1eae64c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1eae64c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1eae64c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 10 monclient(hunting): handle_auth_done global_id 4942 payload 824 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 1 -- 172.21.15.204:0/1789202608 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1ea8118e30 msgr2=0x7f1ea8119200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 1 --2- 172.21.15.204:0/1789202608 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1ea8118e30 0x7f1ea8119200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 1 -- 172.21.15.204:0/1789202608 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1e94004d10 msgr2=0x7f1ea806ebc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 1 --2- 172.21.15.204:0/1789202608 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1e94004d10 0x7f1ea806ebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 10 monclient: _finish_hunting 0 2024-09-15T01:33:31.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 1 monclient: found mon.noname-f 2024-09-15T01:33:31.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:31.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 1 -- 172.21.15.204:0/1789202608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea8119d00 con 0x7f1ea8119fb0 2024-09-15T01:33:31.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1ead64a640 1 --2- 172.21.15.204:0/1789202608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea8119fb0 0x7f1ea81188f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009ba0 tx=0x7f1e98035000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:31.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9dffb640 1 -- 172.21.15.204:0/1789202608 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1e9803cc50 con 0x7f1ea8119fb0 2024-09-15T01:33:31.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9dffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:31.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9dffb640 10 monclient: dump: 2024-09-15T01:33:31.464 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.464 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.464 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.465 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9dffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9dffb640 1 -- 172.21.15.204:0/1789202608 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9803cdd0 con 0x7f1ea8119fb0 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1eb00af640 5 monclient: authenticate success, global_id 4942 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1eb00af640 10 monclient: _renew_subs 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1eb00af640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1eb00af640 1 -- 172.21.15.204:0/1789202608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea811b590 con 0x7f1ea8119fb0 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9dffb640 1 -- 172.21.15.204:0/1789202608 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1e98044880 con 0x7f1ea8119fb0 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9dffb640 10 monclient: dump: 2024-09-15T01:33:31.466 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.467 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1eb00af640 10 monclient: _renew_subs 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1e9ffff640 4 set_mon_vals no callback set 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1eb00af640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.439+0000 7f1eb00af640 1 -- 172.21.15.204:0/1789202608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea811a3c0 con 0x7f1ea8119fb0 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.440+0000 7f1e9dffb640 1 -- 172.21.15.204:0/1789202608 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1e98044400 con 0x7f1ea8119fb0 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.440+0000 7f1e9dffb640 1 --2- 172.21.15.204:0/1789202608 >> v2:172.21.15.204:6812/28210 conn(0x7f1e8c0397b0 0x7f1e8c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.440+0000 7f1eb00af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.440+0000 7f1eb00af640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.440+0000 7f1eb00af640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.440+0000 7f1eb00af640 1 -- 172.21.15.204:0/1789202608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea8070b80 con 0x7f1ea8119fb0 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.440+0000 7f1e9dffb640 1 -- 172.21.15.204:0/1789202608 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 3594+0+0 (secure 0 0 0) 0x7f1e9806c030 con 0x7f1ea8119fb0 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.440+0000 7f1eade4b640 1 --2- 172.21.15.204:0/1789202608 >> v2:172.21.15.204:6812/28210 conn(0x7f1e8c0397b0 0x7f1e8c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.440+0000 7f1eade4b640 10 monclient: get_auth_request con 0x7f1e8c0397b0 auth_method 0 2024-09-15T01:33:31.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.441+0000 7f1eade4b640 1 --2- 172.21.15.204:0/1789202608 >> v2:172.21.15.204:6812/28210 conn(0x7f1e8c0397b0 0x7f1e8c03bc60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4001a20 tx=0x7f1ea4001e30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:31.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.445+0000 7f1e9dffb640 1 -- 172.21.15.204:0/1789202608 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1e98040030 con 0x7f1ea8119fb0 2024-09-15T01:33:31.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.445+0000 7f1e9dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:31.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.445+0000 7f1e9dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:31.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.607+0000 7f1eb00af640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.607+0000 7f1eb00af640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.607+0000 7f1eb00af640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.607+0000 7f1eb00af640 1 -- 172.21.15.204:0/1789202608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1ea80a0c50 con 0x7f1ea8119fb0 2024-09-15T01:33:31.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.608+0000 7f1e9dffb640 1 -- 172.21.15.204:0/1789202608 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v44) v1 ==== 159+0+0 (secure 0 0 0) 0x7f1e98040030 con 0x7f1ea8119fb0 2024-09-15T01:33:31.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.608+0000 7f1e9dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.608+0000 7f1e9dffb640 10 monclient: _finish_command 2 = system:0 pool 'fooo' does not exist 2024-09-15T01:33:31.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.704+0000 7f1eb00af640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.704+0000 7f1eb00af640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.704+0000 7f1eb00af640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.704+0000 7f1eb00af640 1 -- 172.21.15.204:0/1789202608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1ea80008d0 con 0x7f1ea8119fb0 2024-09-15T01:33:31.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.705+0000 7f1e9dffb640 1 -- 172.21.15.204:0/1789202608 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v44) v1 ==== 159+0+0 (secure 0 0 0) 0x7f1e98040020 con 0x7f1ea8119fb0 2024-09-15T01:33:31.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.705+0000 7f1e9dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.705+0000 7f1e9dffb640 10 monclient: _finish_command 3 = system:0 pool 'fooo' does not exist 2024-09-15T01:33:31.707 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'fooo' does not exist 2024-09-15T01:33:31.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.707+0000 7f1eb00af640 1 -- 172.21.15.204:0/1789202608 >> v2:172.21.15.204:6812/28210 conn(0x7f1e8c0397b0 msgr2=0x7f1e8c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.707+0000 7f1eb00af640 1 --2- 172.21.15.204:0/1789202608 >> v2:172.21.15.204:6812/28210 conn(0x7f1e8c0397b0 0x7f1e8c03bc60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4001a20 tx=0x7f1ea4001e30 comp rx=0 tx=0).stop 2024-09-15T01:33:31.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.707+0000 7f1eb00af640 10 monclient: shutdown 2024-09-15T01:33:31.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.707+0000 7f1eb00af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:31.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.707+0000 7f1eb00af640 1 -- 172.21.15.204:0/1789202608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea8119fb0 msgr2=0x7f1ea81188f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.707+0000 7f1eb00af640 1 --2- 172.21.15.204:0/1789202608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea8119fb0 0x7f1ea81188f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009ba0 tx=0x7f1e98035000 comp rx=0 tx=0).stop 2024-09-15T01:33:31.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.707+0000 7f1eb00af640 1 -- 172.21.15.204:0/1789202608 shutdown_connections 2024-09-15T01:33:31.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.707+0000 7f1eb00af640 1 --2- 172.21.15.204:0/1789202608 >> v2:172.21.15.204:6812/28210 conn(0x7f1e8c0397b0 0x7f1e8c03bc60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.707+0000 7f1eb00af640 1 --2- 172.21.15.204:0/1789202608 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1ea8118e30 0x7f1ea8119200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.707+0000 7f1eb00af640 1 --2- 172.21.15.204:0/1789202608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ea8119fb0 0x7f1ea81188f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.707+0000 7f1eb00af640 1 --2- 172.21.15.204:0/1789202608 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1e94004d10 0x7f1ea806ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.707+0000 7f1eb00af640 1 -- 172.21.15.204:0/1789202608 >> 172.21.15.204:0/1789202608 conn(0x7f1ea80a2ce0 msgr2=0x7f1ea809a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:31.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.708+0000 7f1eb00af640 1 -- 172.21.15.204:0/1789202608 shutdown_connections 2024-09-15T01:33:31.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.708+0000 7f1eb00af640 1 -- 172.21.15.204:0/1789202608 wait complete. 2024-09-15T01:33:31.717 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool delete fuggg fuggg --yes-i-really-really-mean-it 2024-09-15T01:33:31.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.792+0000 7fedbb434640 10 monclient: get_monmap_and_config 2024-09-15T01:33:31.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.792+0000 7fedbb434640 10 monclient: build_initial_monmap 2024-09-15T01:33:31.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.792+0000 7fedbb434640 10 monclient: monmap: 2024-09-15T01:33:31.794 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:31.794 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.794 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:31.792762+0000 2024-09-15T01:33:31.794 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:31.792762+0000 2024-09-15T01:33:31.794 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:31.794 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.794 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:31.795 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:31.795 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:31.795 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:31.795 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:31.795 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:31.795 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:31.795 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:31.795 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:31.795 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:31.795 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:31.795 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.792+0000 7fedbb434640 1 Processor -- start 2024-09-15T01:33:31.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 1 -- start start 2024-09-15T01:33:31.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 10 monclient: init 2024-09-15T01:33:31.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:31.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 10 monclient: _add_conns ranks=[0,3,10,6,9,4,2,1,8,7,5] 2024-09-15T01:33:31.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fedb40a4590 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:31.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedb406edd0 0x7fedb409ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 10 monclient(hunting): picked mon.noname-g con 0x7fedb406edd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:31.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fedb4135ad0 0x7fedb4137eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 10 monclient(hunting): picked mon.noname-f con 0x7fedb4135ad0 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:31.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb40648a0 con 0x7fedb40a4590 2024-09-15T01:33:31.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb40a4960 con 0x7fedb4135ad0 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb406dba0 con 0x7fedb406edd0 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedbb434640 10 monclient(hunting): authenticate will time out at 827.435570s 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb89cf640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedb406edd0 msgr2=0x7fedb409ee60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb89cf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedb406edd0 0x7fedb409ee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fedb4135ad0 0x7fedb4137eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fedb4135ad0 0x7fedb4137eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33280/0 (socket says 172.21.15.204:33280) 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 1 -- 172.21.15.204:0/841250395 learned_addr learned my addr 172.21.15.204:0/841250395 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 10 monclient(hunting): get_auth_request con 0x7fedb4135ad0 auth_method 0 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:31.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 10 monclient(hunting): handle_auth_done global_id 4951 payload 274 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 1 -- 172.21.15.204:0/841250395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedb406edd0 msgr2=0x7fedb409ee60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 1 --2- 172.21.15.204:0/841250395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedb406edd0 0x7fedb409ee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 1 -- 172.21.15.204:0/841250395 >> v1:172.21.15.178:6789/0 conn(0x7fedb40a4590 legacy=0x7fedb409e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 10 monclient: _finish_hunting 0 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 1 monclient: found mon.noname-f 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.793+0000 7fedb99d1640 1 -- 172.21.15.204:0/841250395 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb40a04c0 con 0x7fedb4135ad0 2024-09-15T01:33:31.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedb99d1640 1 --2- 172.21.15.204:0/841250395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb4135ad0 0x7fedb4137eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fedac009f20 tx=0x7fedac009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb8bfa54f7933cd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedabfff640 1 -- 172.21.15.204:0/841250395 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedac007880 con 0x7fedb4135ad0 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedabfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedabfff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedabfff640 10 monclient: dump: 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.806 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedabfff640 10 monclient: _finish_auth 0 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedabfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedabfff640 1 -- 172.21.15.204:0/841250395 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac02d050 con 0x7fedb4135ad0 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedbb434640 5 monclient: authenticate success, global_id 4951 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedbb434640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedabfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedbb434640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedabfff640 1 -- 172.21.15.204:0/841250395 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedac030360 con 0x7fedb4135ad0 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedabfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedbb434640 4 set_mon_vals no callback set 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedabfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.794+0000 7fedabfff640 10 monclient: dump: 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.808 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.795+0000 7fedbb434640 10 monclient: shutdown 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.795+0000 7fedbb434640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.795+0000 7fedbb434640 1 -- 172.21.15.204:0/841250395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb4135ad0 msgr2=0x7fedb4137eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.795+0000 7fedbb434640 1 --2- 172.21.15.204:0/841250395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb4135ad0 0x7fedb4137eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fedac009f20 tx=0x7fedac009840 comp rx=0 tx=0).stop 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.795+0000 7fedbb434640 1 -- 172.21.15.204:0/841250395 shutdown_connections 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.795+0000 7fedbb434640 1 --2- 172.21.15.204:0/841250395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb4135ad0 0x7fedb4137eb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.795+0000 7fedbb434640 1 --2- 172.21.15.204:0/841250395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedb406edd0 0x7fedb409ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.795+0000 7fedbb434640 1 -- 172.21.15.204:0/841250395 >> 172.21.15.204:0/841250395 conn(0x7fedb4067700 msgr2=0x7fedb40a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.795+0000 7fedbb434640 1 -- 172.21.15.204:0/841250395 shutdown_connections 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.795+0000 7fedbb434640 1 -- 172.21.15.204:0/841250395 wait complete. 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.795+0000 7fedbb434640 10 monclient: build_initial_monmap 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.795+0000 7fedbb434640 10 monclient: monmap: 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 1 Processor -- start 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 1 -- start start 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 10 monclient: init 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 10 monclient: _add_conns ranks=[8,2,0,6,7,3,5,1,4] 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb406edd0 0x7fedb4141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 10 monclient(hunting): picked mon.noname-e con 0x7fedb406edd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb40a4590 0x7fedb41424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 10 monclient(hunting): picked mon.noname-b con 0x7fedb40a4590 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedb4135ad0 0x7fedb41429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 10 monclient(hunting): picked mon.noname-a con 0x7fedb4135ad0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb406cc80 con 0x7fedb4135ad0 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb406d070 con 0x7fedb40a4590 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fedb411c9b0 con 0x7fedb406edd0 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.796+0000 7fedbb434640 10 monclient(hunting): authenticate will time out at 827.438865s 2024-09-15T01:33:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedb4135ad0 0x7fedb41429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb89cf640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb40a4590 0x7fedb41424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedb4135ad0 0x7fedb41429e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:45556/0 (socket says 172.21.15.204:45556) 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 1 -- 172.21.15.204:0/1060551991 learned_addr learned my addr 172.21.15.204:0/1060551991 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 10 monclient(hunting): get_auth_request con 0x7fedb4135ad0 auth_method 0 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb91d0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb406edd0 0x7fedb4141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb89cf640 10 monclient(hunting): get_auth_request con 0x7fedb40a4590 auth_method 0 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb89cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb89cf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb89cf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb91d0640 10 monclient(hunting): get_auth_request con 0x7fedb406edd0 auth_method 0 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb91d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb91d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb91d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb89cf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb89cf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb89cf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb91d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb91d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb91d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 10 monclient(hunting): handle_auth_done global_id 4815 payload 824 2024-09-15T01:33:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 1 -- 172.21.15.204:0/1060551991 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb406edd0 msgr2=0x7fedb4141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 1 --2- 172.21.15.204:0/1060551991 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb406edd0 0x7fedb4141f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 1 -- 172.21.15.204:0/1060551991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb40a4590 msgr2=0x7fedb41424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 1 --2- 172.21.15.204:0/1060551991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb40a4590 0x7fedb41424a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 10 monclient: _finish_hunting 0 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 1 monclient: found mon.noname-a 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 1 -- 172.21.15.204:0/1060551991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac009ae0 con 0x7fedb4135ad0 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedb99d1640 1 --2- 172.21.15.204:0/1060551991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedb4135ad0 0x7fedb41429e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fedac009880 tx=0x7fedac036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7feda97fa640 1 -- 172.21.15.204:0/1060551991 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedac03baa0 con 0x7fedb4135ad0 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7feda97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7feda97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:33:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7feda97fa640 10 monclient: dump: 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7feda97fa640 10 monclient: _finish_auth 0 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7feda97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7feda97fa640 1 -- 172.21.15.204:0/1060551991 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac03bc40 con 0x7fedb4135ad0 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedbb434640 5 monclient: authenticate success, global_id 4815 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedbb434640 10 monclient: _renew_subs 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedbb434640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedbb434640 1 -- 172.21.15.204:0/1060551991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb411b640 con 0x7fedb4135ad0 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7feda97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7feda97fa640 1 -- 172.21.15.204:0/1060551991 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedac03bdb0 con 0x7fedb4135ad0 2024-09-15T01:33:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7feda97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7feda97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7feda97fa640 10 monclient: dump: 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedab7fe640 4 set_mon_vals no callback set 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedbb434640 10 monclient: _renew_subs 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedbb434640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.797+0000 7fedbb434640 1 -- 172.21.15.204:0/1060551991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb409a170 con 0x7fedb4135ad0 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.798+0000 7feda97fa640 1 -- 172.21.15.204:0/1060551991 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fedac038070 con 0x7fedb4135ad0 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.798+0000 7fedbb434640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.798+0000 7fedbb434640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.798+0000 7fedbb434640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.798+0000 7fedbb434640 1 -- 172.21.15.204:0/1060551991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb4070b80 con 0x7fedb4135ad0 2024-09-15T01:33:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.798+0000 7feda97fa640 1 --2- 172.21.15.204:0/1060551991 >> v2:172.21.15.204:6812/28210 conn(0x7fed980397e0 0x7fed9803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.798+0000 7feda97fa640 1 -- 172.21.15.204:0/1060551991 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 3594+0+0 (secure 0 0 0) 0x7fedac06bb40 con 0x7fedb4135ad0 2024-09-15T01:33:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.799+0000 7fedb91d0640 1 --2- 172.21.15.204:0/1060551991 >> v2:172.21.15.204:6812/28210 conn(0x7fed980397e0 0x7fed9803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.799+0000 7fedb91d0640 10 monclient: get_auth_request con 0x7fed980397e0 auth_method 0 2024-09-15T01:33:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.799+0000 7fedb91d0640 1 --2- 172.21.15.204:0/1060551991 >> v2:172.21.15.204:6812/28210 conn(0x7fed980397e0 0x7fed9803bc90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fedb0007fd0 tx=0x7fedb0000bc0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.803+0000 7feda97fa640 1 -- 172.21.15.204:0/1060551991 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fedac03b620 con 0x7fedb4135ad0 2024-09-15T01:33:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.803+0000 7feda97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.803+0000 7feda97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:31.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.958+0000 7fedbb434640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.958+0000 7fedbb434640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.958+0000 7fedbb434640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:31.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.958+0000 7fedbb434640 1 -- 172.21.15.204:0/1060551991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fedb406d8c0 con 0x7fedb4135ad0 2024-09-15T01:33:31.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.959+0000 7feda97fa640 1 -- 172.21.15.204:0/1060551991 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}]=0 pool 'fuggg' does not exist v44) v1 ==== 162+0+0 (secure 0 0 0) 0x7fedac03f070 con 0x7fedb4135ad0 2024-09-15T01:33:31.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.959+0000 7feda97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:31.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:31.959+0000 7feda97fa640 10 monclient: _finish_command 2 = system:0 pool 'fuggg' does not exist 2024-09-15T01:33:32.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.051+0000 7fedbb434640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:32.053 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.051+0000 7fedbb434640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:32.053 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.051+0000 7fedbb434640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:32.053 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.051+0000 7fedbb434640 1 -- 172.21.15.204:0/1060551991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fedb402ae50 con 0x7fedb4135ad0 2024-09-15T01:33:32.054 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.052+0000 7feda97fa640 1 -- 172.21.15.204:0/1060551991 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}]=0 pool 'fuggg' does not exist v44) v1 ==== 162+0+0 (secure 0 0 0) 0x7fedb402ae50 con 0x7fedb4135ad0 2024-09-15T01:33:32.054 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.052+0000 7feda97fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:32.054 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.052+0000 7feda97fa640 10 monclient: _finish_command 3 = system:0 pool 'fuggg' does not exist 2024-09-15T01:33:32.054 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'fuggg' does not exist 2024-09-15T01:33:32.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.054+0000 7fedbb434640 1 -- 172.21.15.204:0/1060551991 >> v2:172.21.15.204:6812/28210 conn(0x7fed980397e0 msgr2=0x7fed9803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.054+0000 7fedbb434640 1 --2- 172.21.15.204:0/1060551991 >> v2:172.21.15.204:6812/28210 conn(0x7fed980397e0 0x7fed9803bc90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fedb0007fd0 tx=0x7fedb0000bc0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.054+0000 7fedbb434640 10 monclient: shutdown 2024-09-15T01:33:32.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.054+0000 7fedbb434640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:32.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.054+0000 7fedbb434640 1 -- 172.21.15.204:0/1060551991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedb4135ad0 msgr2=0x7fedb41429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.054+0000 7fedbb434640 1 --2- 172.21.15.204:0/1060551991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedb4135ad0 0x7fedb41429e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fedac009880 tx=0x7fedac036000 comp rx=0 tx=0).stop 2024-09-15T01:33:32.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.054+0000 7fedbb434640 1 -- 172.21.15.204:0/1060551991 shutdown_connections 2024-09-15T01:33:32.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.054+0000 7fedbb434640 1 --2- 172.21.15.204:0/1060551991 >> v2:172.21.15.204:6812/28210 conn(0x7fed980397e0 0x7fed9803bc90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.054+0000 7fedbb434640 1 --2- 172.21.15.204:0/1060551991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedb4135ad0 0x7fedb41429e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.054+0000 7fedbb434640 1 --2- 172.21.15.204:0/1060551991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb40a4590 0x7fedb41424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.054+0000 7fedbb434640 1 --2- 172.21.15.204:0/1060551991 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb406edd0 0x7fedb4141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.054+0000 7fedbb434640 1 -- 172.21.15.204:0/1060551991 >> 172.21.15.204:0/1060551991 conn(0x7fedb4067700 msgr2=0x7fedb40688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:32.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.055+0000 7fedbb434640 1 -- 172.21.15.204:0/1060551991 shutdown_connections 2024-09-15T01:33:32.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.055+0000 7fedbb434640 1 -- 172.21.15.204:0/1060551991 wait complete. 2024-09-15T01:33:32.065 INFO:tasks.workunit.client.0.smithi204.stdout:OK 2024-09-15T01:33:32.065 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo OK 2024-09-15T01:33:32.066 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T01:33:32.066 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T01:33:32.091 INFO:tasks.workunit:Running workunits matching mon/crush_ops.sh on client.0... 2024-09-15T01:33:32.092 INFO:tasks.workunit:Running workunit mon/crush_ops.sh... 2024-09-15T01:33:32.093 DEBUG:teuthology.orchestra.run.smithi204:workunit test mon/crush_ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d75f23820806da622f561ee5ee7551015285993f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/crush_ops.sh 2024-09-15T01:33:32.151 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush dump 2024-09-15T01:33:32.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.224+0000 7fd49022b640 10 monclient: get_monmap_and_config 2024-09-15T01:33:32.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.224+0000 7fd49022b640 10 monclient: build_initial_monmap 2024-09-15T01:33:32.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.224+0000 7fd49022b640 10 monclient: monmap: 2024-09-15T01:33:32.226 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:32.226 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.226 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:32.224670+0000 2024-09-15T01:33:32.226 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:32.224670+0000 2024-09-15T01:33:32.226 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:32.226 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.226 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.224+0000 7fd49022b640 1 Processor -- start 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.224+0000 7fd49022b640 1 -- start start 2024-09-15T01:33:32.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.224+0000 7fd49022b640 10 monclient: init 2024-09-15T01:33:32.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:32.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 10 monclient: _add_conns ranks=[1,4,2,7,6,9,10,3,5,8,0] 2024-09-15T01:33:32.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd48806e670 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:32.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd48809e750 0x7fd488130fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 10 monclient(hunting): picked mon.noname-c con 0x7fd48809e750 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:32.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd488135bd0 0x7fd488137fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 10 monclient(hunting): picked mon.noname-b con 0x7fd488135bd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:32.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4880648a0 con 0x7fd48806e670 2024-09-15T01:33:32.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48809eb20 con 0x7fd488135bd0 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd48806d630 con 0x7fd48809e750 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd49022b640 10 monclient(hunting): authenticate will time out at 827.867443s 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48dfc7640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fd48806e670 0x7fd4881308a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:58962/0 (socket says 172.21.15.204:58962) 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48dfc7640 1 -- 172.21.15.204:0/3033045328 learned_addr learned my addr 172.21.15.204:0/3033045328 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48d7c6640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd48809e750 0x7fd488130fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd488135bd0 0x7fd488137fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48d7c6640 10 monclient(hunting): get_auth_request con 0x7fd48809e750 auth_method 0 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48d7c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184582484 0 0) 0x7fd4880648a0 con 0x7fd48806e670 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48d7c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48d7c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48cfc5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48cfc5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48cfc5640 10 monclient(hunting): my global_id is 4960 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4740034e0 con 0x7fd48806e670 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48e7c8640 10 monclient(hunting): get_auth_request con 0x7fd488135bd0 auth_method 0 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48e7c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48e7c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48e7c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48d7c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48d7c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48d7c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:32.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.225+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:32.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 294963763 0 0) 0x7fd4740034e0 con 0x7fd48806e670 2024-09-15T01:33:32.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd48809e750 msgr2=0x7fd488130fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:32.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd48809e750 0x7fd488130fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd488135bd0 msgr2=0x7fd488137fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:32.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd488135bd0 0x7fd488137fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient: _finish_hunting 0 2024-09-15T01:33:32.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:33:32.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:32.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:33:32.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48806ea40 con 0x7fd48806e670 2024-09-15T01:33:32.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient: _finish_auth 0 2024-09-15T01:33:32.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fd4840018f0 con 0x7fd48806e670 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd49022b640 5 monclient: authenticate success, global_id 4960 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd49022b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient: dump: 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.245 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 >> v1:172.21.15.204:6789/0 conn(0x7fd48806e670 legacy=0x7fd4881308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient: _add_conns ranks=[3,0,5,6,2,1,4,7,8] 2024-09-15T01:33:32.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd474004220 0x7fd474004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient(hunting): picked mon.g con 0x7fd474004220 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd474004d10 0x7fd474005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient(hunting): picked mon.a con 0x7fd474004d10 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd474009ab0 0x7fd47400bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient(hunting): picked mon.h con 0x7fd474009ab0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48809eb20 con 0x7fd474004d10 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48806d630 con 0x7fd474004220 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48dfc7640 1 -- 172.21.15.204:0/3033045328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd474004220 msgr2=0x7fd474004780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd47400c820 con 0x7fd474009ab0 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:32.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48cfc5640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:33:32.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48dfc7640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd474004220 0x7fd474004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:32.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd49022b640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:32.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd474009ab0 0x7fd47400bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 10 monclient(hunting): get_auth_request con 0x7fd474009ab0 auth_method 0 2024-09-15T01:33:32.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:32.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48d7c6640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd474004d10 0x7fd474005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48d7c6640 10 monclient(hunting): get_auth_request con 0x7fd474004d10 auth_method 0 2024-09-15T01:33:32.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48d7c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48d7c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48d7c6640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48d7c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48d7c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48d7c6640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_done global_id 4960 payload 293 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3033045328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd474004220 msgr2=0x7fd474004780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd474004220 0x7fd474004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3033045328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd474004d10 msgr2=0x7fd474005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd474004d10 0x7fd474005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 10 monclient: _finish_hunting 0 2024-09-15T01:33:32.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 1 monclient: found mon.h 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3033045328 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4740034e0 con 0x7fd474009ab0 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 10 monclient: _finish_auth 0 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.226+0000 7fd48e7c8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3033045328 reap_dead start 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd474009ab0 0x7fd47400bf10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd488068110 tx=0x7fd48000b040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=3d983ae03422a8e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd48002c040 con 0x7fd474009ab0 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd48cfc5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd48cfc5640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd48cfc5640 10 monclient: dump: 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.250 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd49022b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48000f040 con 0x7fd474009ab0 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd48cfc5640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd49022b640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd49022b640 4 set_mon_vals no callback set 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd48cfc5640 1 -- 172.21.15.204:0/3033045328 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd480030d00 con 0x7fd474009ab0 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd48cfc5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd48cfc5640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd48cfc5640 10 monclient: dump: 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.252 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.253 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.253 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.253 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.253 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.253 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.253 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.253 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.253 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.253 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.253 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd49022b640 10 monclient: shutdown 2024-09-15T01:33:32.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd49022b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd49022b640 1 -- 172.21.15.204:0/3033045328 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd474009ab0 msgr2=0x7fd47400bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.227+0000 7fd49022b640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd474009ab0 0x7fd47400bf10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd488068110 tx=0x7fd48000b040 comp rx=0 tx=0).stop 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.228+0000 7fd49022b640 1 -- 172.21.15.204:0/3033045328 shutdown_connections 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.228+0000 7fd49022b640 1 --2- 172.21.15.204:0/3033045328 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd474009ab0 0x7fd47400bf10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.228+0000 7fd49022b640 1 -- 172.21.15.204:0/3033045328 >> 172.21.15.204:0/3033045328 conn(0x7fd488067c00 msgr2=0x7fd4880a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.228+0000 7fd49022b640 1 -- 172.21.15.204:0/3033045328 shutdown_connections 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.228+0000 7fd49022b640 1 -- 172.21.15.204:0/3033045328 wait complete. 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.228+0000 7fd49022b640 10 monclient: build_initial_monmap 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.228+0000 7fd49022b640 10 monclient: monmap: 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:32.254 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 1 Processor -- start 2024-09-15T01:33:32.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 1 -- start start 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 10 monclient: init 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 10 monclient: _add_conns ranks=[6,3,1,2,7,8,0,5,4] 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd474009ab0 0x7fd488130e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 10 monclient(hunting): picked mon.noname-d con 0x7fd474009ab0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4880690c0 0x7fd488134920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 10 monclient(hunting): picked mon.noname-g con 0x7fd4880690c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd488134e60 0x7fd488135250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 10 monclient(hunting): picked mon.noname-f con 0x7fd488134e60 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48806d0f0 con 0x7fd488134e60 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48806cc70 con 0x7fd4880690c0 2024-09-15T01:33:32.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd488135790 con 0x7fd474009ab0 2024-09-15T01:33:32.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:32.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd49022b640 10 monclient(hunting): authenticate will time out at 827.871532s 2024-09-15T01:33:32.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48d7c6640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4880690c0 msgr2=0x7fd488134920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:32.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48d7c6640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4880690c0 0x7fd488134920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:32.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48dfc7640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd474009ab0 msgr2=0x7fd488130e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:32.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48dfc7640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd474009ab0 0x7fd488130e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:32.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd488134e60 0x7fd488135250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd488134e60 0x7fd488135250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33290/0 (socket says 172.21.15.204:33290) 2024-09-15T01:33:32.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3186880890 learned_addr learned my addr 172.21.15.204:0/3186880890 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:32.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 10 monclient(hunting): get_auth_request con 0x7fd488134e60 auth_method 0 2024-09-15T01:33:32.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:32.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:32.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:32.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_done global_id 4969 payload 824 2024-09-15T01:33:32.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3186880890 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd474009ab0 msgr2=0x7fd488130e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:32.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd474009ab0 0x7fd488130e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4880690c0 msgr2=0x7fd488134920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4880690c0 0x7fd488134920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 10 monclient: _finish_hunting 0 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 1 monclient: found mon.noname-f 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.229+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3186880890 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4880684f0 con 0x7fd488134e60 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd488134e60 0x7fd488135250 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd4800094d0 tx=0x7fd480036000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd480030b90 con 0x7fd488134e60 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47dffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47dffb640 10 monclient: dump: 2024-09-15T01:33:32.259 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.260 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.261 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.261 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47dffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:32.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:32.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd480030d30 con 0x7fd488134e60 2024-09-15T01:33:32.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd49022b640 5 monclient: authenticate success, global_id 4969 2024-09-15T01:33:32.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd49022b640 10 monclient: _renew_subs 2024-09-15T01:33:32.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd49022b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:32.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd49022b640 1 -- 172.21.15.204:0/3186880890 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd488069fb0 con 0x7fd488134e60 2024-09-15T01:33:32.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:32.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd480030ea0 con 0x7fd488134e60 2024-09-15T01:33:32.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47dffb640 10 monclient: dump: 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.262 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd47ffff640 4 set_mon_vals no callback set 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd49022b640 10 monclient: _renew_subs 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd49022b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd49022b640 1 -- 172.21.15.204:0/3186880890 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd488097210 con 0x7fd488134e60 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd488134e60 msgr2=0x7fd488135250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd488134e60 msgr2=0x7fd488135250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd488134e60 0x7fd488135250 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd4800094d0 tx=0x7fd480036000 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:33:32.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.230+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd488134e60 0x7fd488135250 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd4800094d0 tx=0x7fd480036000 comp rx=0 tx=0).stop 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 10 monclient: ms_handle_reset current mon [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd488134e60 msgr2=0x7fd488135250 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd488134e60 0x7fd488135250 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 10 monclient: _add_conns ranks=[6,3,2,5,7,4,8,1,0] 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd474009ab0 0x7fd46c006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 10 monclient(hunting): picked mon.d con 0x7fd474009ab0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4880690c0 0x7fd46c009420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 10 monclient(hunting): picked mon.g con 0x7fd4880690c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd46c009d00 0x7fd46c00c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 10 monclient(hunting): picked mon.b con 0x7fd46c009d00 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c00cc80 con 0x7fd46c009d00 2024-09-15T01:33:32.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c00cdc0 con 0x7fd4880690c0 2024-09-15T01:33:32.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c00cf30 con 0x7fd474009ab0 2024-09-15T01:33:32.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd47dffb640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:32.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48d7c6640 1 -- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4880690c0 msgr2=0x7fd46c009420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:32.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48d7c6640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4880690c0 0x7fd46c009420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:32.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48dfc7640 1 -- 172.21.15.204:0/3186880890 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd474009ab0 msgr2=0x7fd46c006cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:32.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48dfc7640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd474009ab0 0x7fd46c006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:32.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd46c009d00 0x7fd46c00c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient(hunting): get_auth_request con 0x7fd46c009d00 auth_method 0 2024-09-15T01:33:32.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd49022b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd49022b640 10 monclient(hunting): _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient(hunting): handle_auth_done global_id 4969 payload 293 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3186880890 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd474009ab0 msgr2=0x7fd46c006cc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd474009ab0 0x7fd46c006cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4880690c0 msgr2=0x7fd46c009420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4880690c0 0x7fd46c009420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient: _finish_hunting 0 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 1 monclient: found mon.b 2024-09-15T01:33:32.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3186880890 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) v3 -- 0x7fd46c003f00 con 0x7fd46c009d00 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3186880890 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd488070b80 con 0x7fd46c009d00 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3186880890 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd488097210 con 0x7fd46c009d00 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 10 monclient: _finish_auth 0 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.231+0000 7fd48e7c8640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd46c009d00 0x7fd46c00c180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd480030ab0 tx=0x7fd48003c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.232+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd480044870 con 0x7fd46c009d00 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.232+0000 7fd47dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.232+0000 7fd47dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:33:32.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.232+0000 7fd47dffb640 10 monclient: dump: 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.268 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.232+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4800449d0 con 0x7fd46c009d00 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.232+0000 7fd47dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.232+0000 7fd47f7fe640 4 set_mon_vals no callback set 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.232+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd4800443f0 con 0x7fd46c009d00 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.233+0000 7fd47dffb640 1 --2- 172.21.15.204:0/3186880890 >> v2:172.21.15.204:6812/28210 conn(0x7fd46c042290 0x7fd46c044740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.233+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 <== mon.2 v2:172.21.15.178:3301/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 3594+0+0 (secure 0 0 0) 0x7fd480053020 con 0x7fd46c009d00 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.233+0000 7fd48dfc7640 1 --2- 172.21.15.204:0/3186880890 >> v2:172.21.15.204:6812/28210 conn(0x7fd46c042290 0x7fd46c044740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.233+0000 7fd48dfc7640 10 monclient: get_auth_request con 0x7fd46c042290 auth_method 0 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.233+0000 7fd48dfc7640 1 --2- 172.21.15.204:0/3186880890 >> v2:172.21.15.204:6812/28210 conn(0x7fd46c042290 0x7fd46c044740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd4840047d0 tx=0x7fd48806dc20 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.237+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 <== mon.2 v2:172.21.15.178:3301/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd48004b980 con 0x7fd46c009d00 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.237+0000 7fd47dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:32.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.237+0000 7fd47dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:32.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.240+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd48002f360 con 0x7fd46c009d00 2024-09-15T01:33:32.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.240+0000 7fd47dffb640 10 monclient: handle_mon_command_ack 1 not found 2024-09-15T01:33:32.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.393+0000 7fd49022b640 10 start_mon_command cmd=[{"prefix": "osd crush dump"}] 2024-09-15T01:33:32.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.393+0000 7fd49022b640 10 monclient: _send_command 2 [{"prefix": "osd crush dump"}] 2024-09-15T01:33:32.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.393+0000 7fd49022b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:32.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.393+0000 7fd49022b640 1 -- 172.21.15.204:0/3186880890 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd crush dump"} v 0) v1 -- 0x7fd488131410 con 0x7fd46c009d00 2024-09-15T01:33:32.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.394+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush dump"}]=0 v44) v1 ==== 62+0+7238 (secure 0 0 0) 0x7fd48004b350 con 0x7fd46c009d00 2024-09-15T01:33:32.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.394+0000 7fd47dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush dump"}] 2024-09-15T01:33:32.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.394+0000 7fd47dffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:32.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.488+0000 7fd49022b640 10 start_mon_command cmd=[{"prefix": "osd crush dump"}] 2024-09-15T01:33:32.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.488+0000 7fd49022b640 10 monclient: _send_command 3 [{"prefix": "osd crush dump"}] 2024-09-15T01:33:32.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.488+0000 7fd49022b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:32.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.488+0000 7fd49022b640 1 -- 172.21.15.204:0/3186880890 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd crush dump"} v 0) v1 -- 0x7fd488131410 con 0x7fd46c009d00 2024-09-15T01:33:32.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.489+0000 7fd47dffb640 1 -- 172.21.15.204:0/3186880890 <== mon.2 v2:172.21.15.178:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush dump"}]=0 v44) v1 ==== 62+0+7238 (secure 0 0 0) 0x7fd48004b980 con 0x7fd46c009d00 2024-09-15T01:33:32.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.489+0000 7fd47dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush dump"}] 2024-09-15T01:33:32.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.489+0000 7fd47dffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:32.491 INFO:tasks.workunit.client.0.smithi204.stdout:{ 2024-09-15T01:33:32.491 INFO:tasks.workunit.client.0.smithi204.stdout: "devices": [ 2024-09-15T01:33:32.491 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.491 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 0, 2024-09-15T01:33:32.491 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "osd.0", 2024-09-15T01:33:32.492 INFO:tasks.workunit.client.0.smithi204.stdout: "class": "ssd" 2024-09-15T01:33:32.492 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.492 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.492 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 1, 2024-09-15T01:33:32.492 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "osd.1", 2024-09-15T01:33:32.492 INFO:tasks.workunit.client.0.smithi204.stdout: "class": "ssd" 2024-09-15T01:33:32.492 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.492 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.492 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 2, 2024-09-15T01:33:32.492 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "osd.2", 2024-09-15T01:33:32.492 INFO:tasks.workunit.client.0.smithi204.stdout: "class": "ssd" 2024-09-15T01:33:32.493 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.493 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.493 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 3, 2024-09-15T01:33:32.493 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "osd.3", 2024-09-15T01:33:32.493 INFO:tasks.workunit.client.0.smithi204.stdout: "class": "ssd" 2024-09-15T01:33:32.493 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.493 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.493 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 4, 2024-09-15T01:33:32.493 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "osd.4", 2024-09-15T01:33:32.493 INFO:tasks.workunit.client.0.smithi204.stdout: "class": "ssd" 2024-09-15T01:33:32.493 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.493 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 5, 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "osd.5", 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: "class": "ssd" 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: "types": [ 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 0, 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "osd" 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 1, 2024-09-15T01:33:32.494 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "host" 2024-09-15T01:33:32.495 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.495 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.495 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 2, 2024-09-15T01:33:32.495 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "chassis" 2024-09-15T01:33:32.495 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.495 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.495 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 3, 2024-09-15T01:33:32.495 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "rack" 2024-09-15T01:33:32.495 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.495 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.495 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 4, 2024-09-15T01:33:32.495 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "row" 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 5, 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "pdu" 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 6, 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "pod" 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 7, 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "room" 2024-09-15T01:33:32.496 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 8, 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "datacenter" 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 9, 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "zone" 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 10, 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "region" 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.497 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.498 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 11, 2024-09-15T01:33:32.498 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "root" 2024-09-15T01:33:32.498 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.498 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:33:32.498 INFO:tasks.workunit.client.0.smithi204.stdout: "buckets": [ 2024-09-15T01:33:32.498 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.498 INFO:tasks.workunit.client.0.smithi204.stdout: "id": -1, 2024-09-15T01:33:32.498 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "default", 2024-09-15T01:33:32.498 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 11, 2024-09-15T01:33:32.498 INFO:tasks.workunit.client.0.smithi204.stdout: "type_name": "root", 2024-09-15T01:33:32.498 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 34284, 2024-09-15T01:33:32.498 INFO:tasks.workunit.client.0.smithi204.stdout: "alg": "straw2", 2024-09-15T01:33:32.499 INFO:tasks.workunit.client.0.smithi204.stdout: "hash": "rjenkins1", 2024-09-15T01:33:32.499 INFO:tasks.workunit.client.0.smithi204.stdout: "items": [ 2024-09-15T01:33:32.499 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.499 INFO:tasks.workunit.client.0.smithi204.stdout: "id": -3, 2024-09-15T01:33:32.499 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 17142, 2024-09-15T01:33:32.499 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 0 2024-09-15T01:33:32.499 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.499 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.499 INFO:tasks.workunit.client.0.smithi204.stdout: "id": -5, 2024-09-15T01:33:32.499 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 17142, 2024-09-15T01:33:32.499 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 1 2024-09-15T01:33:32.500 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.500 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:33:32.500 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.500 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.500 INFO:tasks.workunit.client.0.smithi204.stdout: "id": -2, 2024-09-15T01:33:32.500 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "default~ssd", 2024-09-15T01:33:32.500 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 11, 2024-09-15T01:33:32.500 INFO:tasks.workunit.client.0.smithi204.stdout: "type_name": "root", 2024-09-15T01:33:32.500 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 34284, 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: "alg": "straw2", 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: "hash": "rjenkins1", 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: "items": [ 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: "id": -4, 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 17142, 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 0 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: "id": -6, 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 17142, 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 1 2024-09-15T01:33:32.501 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.502 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:33:32.502 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.502 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.502 INFO:tasks.workunit.client.0.smithi204.stdout: "id": -3, 2024-09-15T01:33:32.502 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "smithi204", 2024-09-15T01:33:32.502 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 1, 2024-09-15T01:33:32.502 INFO:tasks.workunit.client.0.smithi204.stdout: "type_name": "host", 2024-09-15T01:33:32.502 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 17142, 2024-09-15T01:33:32.502 INFO:tasks.workunit.client.0.smithi204.stdout: "alg": "straw2", 2024-09-15T01:33:32.502 INFO:tasks.workunit.client.0.smithi204.stdout: "hash": "rjenkins1", 2024-09-15T01:33:32.503 INFO:tasks.workunit.client.0.smithi204.stdout: "items": [ 2024-09-15T01:33:32.503 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.503 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 4, 2024-09-15T01:33:32.503 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 5714, 2024-09-15T01:33:32.503 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 0 2024-09-15T01:33:32.503 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.503 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.503 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 3, 2024-09-15T01:33:32.503 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 5714, 2024-09-15T01:33:32.503 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 1 2024-09-15T01:33:32.503 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.503 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.504 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 5, 2024-09-15T01:33:32.504 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 5714, 2024-09-15T01:33:32.504 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 2 2024-09-15T01:33:32.504 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.504 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:33:32.504 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.504 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.504 INFO:tasks.workunit.client.0.smithi204.stdout: "id": -4, 2024-09-15T01:33:32.504 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "smithi204~ssd", 2024-09-15T01:33:32.504 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 1, 2024-09-15T01:33:32.504 INFO:tasks.workunit.client.0.smithi204.stdout: "type_name": "host", 2024-09-15T01:33:32.504 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 17142, 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: "alg": "straw2", 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: "hash": "rjenkins1", 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: "items": [ 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 4, 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 5714, 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 0 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 3, 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 5714, 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 1 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.505 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.506 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 5, 2024-09-15T01:33:32.506 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 5714, 2024-09-15T01:33:32.506 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 2 2024-09-15T01:33:32.506 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.506 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:33:32.506 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.506 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.506 INFO:tasks.workunit.client.0.smithi204.stdout: "id": -5, 2024-09-15T01:33:32.506 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "smithi178", 2024-09-15T01:33:32.506 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 1, 2024-09-15T01:33:32.506 INFO:tasks.workunit.client.0.smithi204.stdout: "type_name": "host", 2024-09-15T01:33:32.506 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 17142, 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: "alg": "straw2", 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: "hash": "rjenkins1", 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: "items": [ 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 1, 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 5714, 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 0 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 0, 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 5714, 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 1 2024-09-15T01:33:32.507 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 2, 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 5714, 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 2 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: "id": -6, 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: "name": "smithi178~ssd", 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: "type_id": 1, 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: "type_name": "host", 2024-09-15T01:33:32.508 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 17142, 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: "alg": "straw2", 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: "hash": "rjenkins1", 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: "items": [ 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 1, 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 5714, 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 0 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 0, 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 5714, 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 1 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.509 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.510 INFO:tasks.workunit.client.0.smithi204.stdout: "id": 2, 2024-09-15T01:33:32.510 INFO:tasks.workunit.client.0.smithi204.stdout: "weight": 5714, 2024-09-15T01:33:32.510 INFO:tasks.workunit.client.0.smithi204.stdout: "pos": 2 2024-09-15T01:33:32.510 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.510 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:33:32.510 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.510 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:33:32.510 INFO:tasks.workunit.client.0.smithi204.stdout: "rules": [ 2024-09-15T01:33:32.510 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.510 INFO:tasks.workunit.client.0.smithi204.stdout: "rule_id": 0, 2024-09-15T01:33:32.511 INFO:tasks.workunit.client.0.smithi204.stdout: "rule_name": "replicated_rule", 2024-09-15T01:33:32.511 INFO:tasks.workunit.client.0.smithi204.stdout: "type": 1, 2024-09-15T01:33:32.511 INFO:tasks.workunit.client.0.smithi204.stdout: "steps": [ 2024-09-15T01:33:32.511 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.511 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "take", 2024-09-15T01:33:32.511 INFO:tasks.workunit.client.0.smithi204.stdout: "item": -1, 2024-09-15T01:33:32.511 INFO:tasks.workunit.client.0.smithi204.stdout: "item_name": "default" 2024-09-15T01:33:32.511 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.511 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.511 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "choose_firstn", 2024-09-15T01:33:32.511 INFO:tasks.workunit.client.0.smithi204.stdout: "num": 0, 2024-09-15T01:33:32.511 INFO:tasks.workunit.client.0.smithi204.stdout: "type": "osd" 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "emit" 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: "rule_id": 1, 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: "rule_name": "erasure-code", 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: "type": 3, 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: "steps": [ 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.512 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "set_chooseleaf_tries", 2024-09-15T01:33:32.513 INFO:tasks.workunit.client.0.smithi204.stdout: "num": 5 2024-09-15T01:33:32.513 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.513 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.513 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "set_choose_tries", 2024-09-15T01:33:32.513 INFO:tasks.workunit.client.0.smithi204.stdout: "num": 100 2024-09-15T01:33:32.513 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.513 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.513 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "take", 2024-09-15T01:33:32.513 INFO:tasks.workunit.client.0.smithi204.stdout: "item": -1, 2024-09-15T01:33:32.513 INFO:tasks.workunit.client.0.smithi204.stdout: "item_name": "default" 2024-09-15T01:33:32.513 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.513 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.514 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "choose_indep", 2024-09-15T01:33:32.514 INFO:tasks.workunit.client.0.smithi204.stdout: "num": 0, 2024-09-15T01:33:32.514 INFO:tasks.workunit.client.0.smithi204.stdout: "type": "osd" 2024-09-15T01:33:32.514 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.514 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "emit" 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: "tunables": { 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: "choose_local_tries": 0, 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: "choose_local_fallback_tries": 0, 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: "choose_total_tries": 50, 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: "chooseleaf_descend_once": 1, 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: "chooseleaf_vary_r": 1, 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: "chooseleaf_stable": 1, 2024-09-15T01:33:32.515 INFO:tasks.workunit.client.0.smithi204.stdout: "straw_calc_version": 1, 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "allowed_bucket_algs": 54, 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "profile": "jewel", 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "optimal_tunables": 1, 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "legacy_tunables": 0, 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "minimum_required_version": "jewel", 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "require_feature_tunables": 1, 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "require_feature_tunables2": 1, 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "has_v2_rules": 1, 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "require_feature_tunables3": 1, 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "has_v3_rules": 0, 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "has_v4_buckets": 1, 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "require_feature_tunables5": 1, 2024-09-15T01:33:32.516 INFO:tasks.workunit.client.0.smithi204.stdout: "has_v5_rules": 0 2024-09-15T01:33:32.517 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.517 INFO:tasks.workunit.client.0.smithi204.stdout: "choose_args": {} 2024-09-15T01:33:32.517 INFO:tasks.workunit.client.0.smithi204.stdout:} 2024-09-15T01:33:32.517 INFO:tasks.workunit.client.0.smithi204.stdout: 2024-09-15T01:33:32.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.491+0000 7fd49022b640 1 -- 172.21.15.204:0/3186880890 >> v2:172.21.15.204:6812/28210 conn(0x7fd46c042290 msgr2=0x7fd46c044740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.491+0000 7fd49022b640 1 --2- 172.21.15.204:0/3186880890 >> v2:172.21.15.204:6812/28210 conn(0x7fd46c042290 0x7fd46c044740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd4840047d0 tx=0x7fd48806dc20 comp rx=0 tx=0).stop 2024-09-15T01:33:32.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.491+0000 7fd49022b640 10 monclient: shutdown 2024-09-15T01:33:32.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.491+0000 7fd49022b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:32.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.491+0000 7fd49022b640 1 -- 172.21.15.204:0/3186880890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd46c009d00 msgr2=0x7fd46c00c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.491+0000 7fd49022b640 1 --2- 172.21.15.204:0/3186880890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd46c009d00 0x7fd46c00c180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd480030ab0 tx=0x7fd48003c040 comp rx=0 tx=0).stop 2024-09-15T01:33:32.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.491+0000 7fd48e7c8640 1 -- 172.21.15.204:0/3186880890 reap_dead start 2024-09-15T01:33:32.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.491+0000 7fd49022b640 1 -- 172.21.15.204:0/3186880890 shutdown_connections 2024-09-15T01:33:32.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.491+0000 7fd49022b640 1 -- 172.21.15.204:0/3186880890 >> 172.21.15.204:0/3186880890 conn(0x7fd488067c00 msgr2=0x7fd4880a1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:32.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.491+0000 7fd49022b640 1 -- 172.21.15.204:0/3186880890 shutdown_connections 2024-09-15T01:33:32.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.492+0000 7fd49022b640 1 -- 172.21.15.204:0/3186880890 wait complete. 2024-09-15T01:33:32.519 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule dump 2024-09-15T01:33:32.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.574+0000 7fd226b09640 10 monclient: get_monmap_and_config 2024-09-15T01:33:32.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.574+0000 7fd226b09640 10 monclient: build_initial_monmap 2024-09-15T01:33:32.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.574+0000 7fd226b09640 10 monclient: monmap: 2024-09-15T01:33:32.576 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:32.576 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.576 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:32.575361+0000 2024-09-15T01:33:32.576 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:32.575361+0000 2024-09-15T01:33:32.577 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:32.577 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.577 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:32.577 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:32.577 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:32.577 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:32.577 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:32.577 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:32.577 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:32.577 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:32.577 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:32.577 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:32.578 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:32.578 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.575+0000 7fd226b09640 1 Processor -- start 2024-09-15T01:33:32.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.575+0000 7fd226b09640 1 -- start start 2024-09-15T01:33:32.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.575+0000 7fd226b09640 10 monclient: init 2024-09-15T01:33:32.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:32.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 10 monclient: _add_conns ranks=[8,0,1,4,3,6,5,7,10,9,2] 2024-09-15T01:33:32.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2200a2e30 0x7fd220132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 10 monclient(hunting): picked mon.noname-e con 0x7fd2200a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:32.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd22009fb80 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:32.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd22009ff70 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:32.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22004b720 con 0x7fd22009fb80 2024-09-15T01:33:32.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2200a3200 con 0x7fd22009ff70 2024-09-15T01:33:32.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd22009e6c0 con 0x7fd2200a2e30 2024-09-15T01:33:32.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:32.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 10 monclient(hunting): authenticate will time out at 828.218157s 2024-09-15T01:33:32.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd2250a6640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fd22009ff70 0x7fd2200a1150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:58966/0 (socket says 172.21.15.204:58966) 2024-09-15T01:33:32.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd2250a6640 1 -- 172.21.15.204:0/3001675112 learned_addr learned my addr 172.21.15.204:0/3001675112 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:32.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074259859 0 0) 0x7fd2200a3200 con 0x7fd22009ff70 2024-09-15T01:33:32.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient(hunting): my global_id is 4978 2024-09-15T01:33:32.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2080034e0 con 0x7fd22009ff70 2024-09-15T01:33:32.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd2248a5640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2200a2e30 0x7fd220132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd2248a5640 10 monclient(hunting): get_auth_request con 0x7fd2200a2e30 auth_method 0 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd2248a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd2248a5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd2248a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2587438338 0 0) 0x7fd2080034e0 con 0x7fd22009ff70 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2200a2e30 msgr2=0x7fd220132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2200a2e30 0x7fd220132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 >> v1:172.21.15.178:6789/0 conn(0x7fd22009fb80 legacy=0x7fd2200a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2200a1860 con 0x7fd22009ff70 2024-09-15T01:33:32.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fd2180018f0 con 0x7fd22009ff70 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 5 monclient: authenticate success, global_id 4978 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd226b09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd2248a5640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2200a2e30 0x7fd220132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient: dump: 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.590 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:32.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 >> v1:172.21.15.204:6789/0 conn(0x7fd22009ff70 legacy=0x7fd2200a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient: _add_conns ranks=[7,5,6,2,0,8,4,3,1] 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd208004220 0x7fd208004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient(hunting): picked mon.i con 0x7fd208004220 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 0x7fd208005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient(hunting): picked mon.h con 0x7fd208004de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd208009ce0 0x7fd20800c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient(hunting): picked mon.d con 0x7fd208009ce0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd22009e6c0 con 0x7fd208004de0 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd20800ca20 con 0x7fd208009ce0 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd20800cb90 con 0x7fd208004220 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.576+0000 7fd21f7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd226b09640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd2250a6640 1 -- 172.21.15.204:0/3001675112 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd208009ce0 msgr2=0x7fd20800c140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd2248a5640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd208004220 0x7fd208004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd2250a6640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd208009ce0 0x7fd20800c140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 0x7fd208005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd2248a5640 10 monclient(hunting): get_auth_request con 0x7fd208004220 auth_method 0 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd2248a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd2248a5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd2248a5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 10 monclient(hunting): get_auth_request con 0x7fd208004de0 auth_method 0 2024-09-15T01:33:32.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:32.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd2248a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:32.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd2248a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:32.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd2248a5640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:32.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:32.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:32.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:32.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 10 monclient(hunting): handle_auth_done global_id 4978 payload 293 2024-09-15T01:33:32.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 1 -- 172.21.15.204:0/3001675112 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd208004220 msgr2=0x7fd208004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:32.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd208004220 0x7fd208004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 1 -- 172.21.15.204:0/3001675112 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd208009ce0 msgr2=0x7fd20800c140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd208009ce0 0x7fd20800c140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 1 monclient: found mon.h 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 1 -- 172.21.15.204:0/3001675112 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2080034e0 con 0x7fd208004de0 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 10 monclient: _finish_auth 0 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd2250a6640 1 -- 172.21.15.204:0/3001675112 reap_dead start 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd2248a5640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd208004220 0x7fd208004780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.577+0000 7fd21ffff640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 0x7fd208005250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd214001ef0 tx=0x7fd214007f90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=2e70732d77724696 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd21402ac60 con 0x7fd208004de0 2024-09-15T01:33:32.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd21f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd21f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd21f7fe640 10 monclient: dump: 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.596 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21402ade0 con 0x7fd208004de0 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd226b09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd21f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd226b09640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd21f7fe640 1 -- 172.21.15.204:0/3001675112 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd214032d30 con 0x7fd208004de0 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd226b09640 4 set_mon_vals no callback set 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd21f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd21f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:32.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd21f7fe640 10 monclient: dump: 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.598 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd226b09640 10 monclient: shutdown 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd226b09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd226b09640 1 -- 172.21.15.204:0/3001675112 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 msgr2=0x7fd208005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd226b09640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 0x7fd208005250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd214001ef0 tx=0x7fd214007f90 comp rx=0 tx=0).stop 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd226b09640 1 -- 172.21.15.204:0/3001675112 shutdown_connections 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd226b09640 1 --2- 172.21.15.204:0/3001675112 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 0x7fd208005250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd226b09640 1 -- 172.21.15.204:0/3001675112 >> 172.21.15.204:0/3001675112 conn(0x7fd22006e860 msgr2=0x7fd22006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd226b09640 1 -- 172.21.15.204:0/3001675112 shutdown_connections 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.578+0000 7fd226b09640 1 -- 172.21.15.204:0/3001675112 wait complete. 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.579+0000 7fd226b09640 10 monclient: build_initial_monmap 2024-09-15T01:33:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.579+0000 7fd226b09640 10 monclient: monmap: 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:32.600 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.579+0000 7fd226b09640 1 Processor -- start 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.579+0000 7fd226b09640 1 -- start start 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.579+0000 7fd226b09640 10 monclient: init 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 10 monclient: _add_conns ranks=[5,0,2,1,7,6,4,3,8] 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 0x7fd220130770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 10 monclient(hunting): picked mon.noname-h con 0x7fd208004de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd220130cb0 0x7fd22011d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 10 monclient(hunting): picked mon.noname-a con 0x7fd220130cb0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd22011d820 0x7fd22011dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 10 monclient(hunting): picked mon.noname-b con 0x7fd22011d820 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd220131080 con 0x7fd220130cb0 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22009ca40 con 0x7fd22011d820 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd22009d650 con 0x7fd208004de0 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 10 monclient(hunting): authenticate will time out at 828.222312s 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 0x7fd220130770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 0x7fd220130770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:52924/0 (socket says 172.21.15.204:52924) 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 1 -- 172.21.15.204:0/1704012417 learned_addr learned my addr 172.21.15.204:0/1704012417 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 10 monclient(hunting): get_auth_request con 0x7fd208004de0 auth_method 0 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21ffff640 1 --2- 172.21.15.204:0/1704012417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd220130cb0 0x7fd22011d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21ffff640 10 monclient(hunting): get_auth_request con 0x7fd220130cb0 auth_method 0 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2250a6640 1 --2- 172.21.15.204:0/1704012417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd22011d820 0x7fd22011dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2250a6640 10 monclient(hunting): get_auth_request con 0x7fd22011d820 auth_method 0 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2250a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2250a6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2250a6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 10 monclient(hunting): handle_auth_done global_id 4766 payload 824 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 1 -- 172.21.15.204:0/1704012417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd22011d820 msgr2=0x7fd22011dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 1 --2- 172.21.15.204:0/1704012417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd22011d820 0x7fd22011dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 1 -- 172.21.15.204:0/1704012417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd220130cb0 msgr2=0x7fd22011d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 1 --2- 172.21.15.204:0/1704012417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd220130cb0 0x7fd22011d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 10 monclient: _finish_hunting 0 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 1 monclient: found mon.noname-h 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 1 -- 172.21.15.204:0/1704012417 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2201208c0 con 0x7fd208004de0 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21ffff640 1 --2- 172.21.15.204:0/1704012417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd220130cb0 0x7fd22011d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:32.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd2248a5640 1 --2- 172.21.15.204:0/1704012417 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 0x7fd220130770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd210010ce0 tx=0x7fd210008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21cff9640 1 -- 172.21.15.204:0/1704012417 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd210010ea0 con 0x7fd208004de0 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21cff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21cff9640 10 monclient: dump: 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.605 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21cff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21cff9640 1 -- 172.21.15.204:0/1704012417 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21000ee40 con 0x7fd208004de0 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21cff9640 1 -- 172.21.15.204:0/1704012417 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd210016900 con 0x7fd208004de0 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 5 monclient: authenticate success, global_id 4766 2024-09-15T01:33:32.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 10 monclient: _renew_subs 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd226b09640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd226b09640 1 -- 172.21.15.204:0/1704012417 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22011ed90 con 0x7fd208004de0 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.580+0000 7fd21effd640 4 set_mon_vals no callback set 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd21cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd21cff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd21cff9640 10 monclient: dump: 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.607 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd226b09640 10 monclient: _renew_subs 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd226b09640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd226b09640 1 -- 172.21.15.204:0/1704012417 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22011f960 con 0x7fd208004de0 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd21cff9640 1 -- 172.21.15.204:0/1704012417 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd210016480 con 0x7fd208004de0 2024-09-15T01:33:32.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd226b09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:32.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd226b09640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:32.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd226b09640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:32.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd21cff9640 1 --2- 172.21.15.204:0/1704012417 >> v2:172.21.15.204:6812/28210 conn(0x7fd20c0397b0 0x7fd20c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.581+0000 7fd226b09640 1 -- 172.21.15.204:0/1704012417 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd220120d60 con 0x7fd208004de0 2024-09-15T01:33:32.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.582+0000 7fd21cff9640 1 -- 172.21.15.204:0/1704012417 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 3594+0+0 (secure 0 0 0) 0x7fd21004b590 con 0x7fd208004de0 2024-09-15T01:33:32.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.582+0000 7fd21ffff640 1 --2- 172.21.15.204:0/1704012417 >> v2:172.21.15.204:6812/28210 conn(0x7fd20c0397b0 0x7fd20c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.582+0000 7fd21ffff640 10 monclient: get_auth_request con 0x7fd20c0397b0 auth_method 0 2024-09-15T01:33:32.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.582+0000 7fd21ffff640 1 --2- 172.21.15.204:0/1704012417 >> v2:172.21.15.204:6812/28210 conn(0x7fd20c0397b0 0x7fd20c03bc60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd214033000 tx=0x7fd214031000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:32.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.586+0000 7fd21cff9640 1 -- 172.21.15.204:0/1704012417 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd21001a020 con 0x7fd208004de0 2024-09-15T01:33:32.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.586+0000 7fd21cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:32.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.586+0000 7fd21cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:32.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.744+0000 7fd226b09640 10 start_mon_command cmd=[{"prefix": "osd crush rule dump"}] 2024-09-15T01:33:32.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.744+0000 7fd226b09640 10 monclient: _send_command 2 [{"prefix": "osd crush rule dump"}] 2024-09-15T01:33:32.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.744+0000 7fd226b09640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:32.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.744+0000 7fd226b09640 1 -- 172.21.15.204:0/1704012417 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule dump"} v 0) v1 -- 0x7fd220070cc0 con 0x7fd208004de0 2024-09-15T01:33:32.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.744+0000 7fd21cff9640 1 -- 172.21.15.204:0/1704012417 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule dump"}]=0 v44) v1 ==== 67+0+1058 (secure 0 0 0) 0x7fd21001d540 con 0x7fd208004de0 2024-09-15T01:33:32.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.744+0000 7fd21cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule dump"}] 2024-09-15T01:33:32.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.744+0000 7fd21cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:32.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.840+0000 7fd226b09640 10 start_mon_command cmd=[{"prefix": "osd crush rule dump"}] 2024-09-15T01:33:32.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.840+0000 7fd226b09640 10 monclient: _send_command 3 [{"prefix": "osd crush rule dump"}] 2024-09-15T01:33:32.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.840+0000 7fd226b09640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:32.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.840+0000 7fd226b09640 1 -- 172.21.15.204:0/1704012417 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule dump"} v 0) v1 -- 0x7fd2200a1f10 con 0x7fd208004de0 2024-09-15T01:33:32.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.841+0000 7fd21cff9640 1 -- 172.21.15.204:0/1704012417 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule dump"}]=0 v44) v1 ==== 67+0+1058 (secure 0 0 0) 0x7fd21002f440 con 0x7fd208004de0 2024-09-15T01:33:32.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.841+0000 7fd21cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule dump"}] 2024-09-15T01:33:32.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.841+0000 7fd21cff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:32.843 INFO:tasks.workunit.client.0.smithi204.stdout:[ 2024-09-15T01:33:32.843 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.843 INFO:tasks.workunit.client.0.smithi204.stdout: "rule_id": 0, 2024-09-15T01:33:32.843 INFO:tasks.workunit.client.0.smithi204.stdout: "rule_name": "replicated_rule", 2024-09-15T01:33:32.843 INFO:tasks.workunit.client.0.smithi204.stdout: "type": 1, 2024-09-15T01:33:32.843 INFO:tasks.workunit.client.0.smithi204.stdout: "steps": [ 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "take", 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: "item": -1, 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: "item_name": "default" 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "choose_firstn", 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: "num": 0, 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: "type": "osd" 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "emit" 2024-09-15T01:33:32.844 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.845 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:33:32.845 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.845 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.845 INFO:tasks.workunit.client.0.smithi204.stdout: "rule_id": 1, 2024-09-15T01:33:32.845 INFO:tasks.workunit.client.0.smithi204.stdout: "rule_name": "erasure-code", 2024-09-15T01:33:32.845 INFO:tasks.workunit.client.0.smithi204.stdout: "type": 3, 2024-09-15T01:33:32.845 INFO:tasks.workunit.client.0.smithi204.stdout: "steps": [ 2024-09-15T01:33:32.845 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.845 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "set_chooseleaf_tries", 2024-09-15T01:33:32.845 INFO:tasks.workunit.client.0.smithi204.stdout: "num": 5 2024-09-15T01:33:32.845 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.845 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "set_choose_tries", 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: "num": 100 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "take", 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: "item": -1, 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: "item_name": "default" 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "choose_indep", 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: "num": 0, 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: "type": "osd" 2024-09-15T01:33:32.846 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:33:32.847 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:33:32.847 INFO:tasks.workunit.client.0.smithi204.stdout: "op": "emit" 2024-09-15T01:33:32.847 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.847 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:33:32.847 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:33:32.847 INFO:tasks.workunit.client.0.smithi204.stdout:] 2024-09-15T01:33:32.847 INFO:tasks.workunit.client.0.smithi204.stdout: 2024-09-15T01:33:32.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 1 -- 172.21.15.204:0/1704012417 >> v2:172.21.15.204:6812/28210 conn(0x7fd20c0397b0 msgr2=0x7fd20c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 1 --2- 172.21.15.204:0/1704012417 >> v2:172.21.15.204:6812/28210 conn(0x7fd20c0397b0 0x7fd20c03bc60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd214033000 tx=0x7fd214031000 comp rx=0 tx=0).stop 2024-09-15T01:33:32.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 10 monclient: shutdown 2024-09-15T01:33:32.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:32.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 1 -- 172.21.15.204:0/1704012417 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 msgr2=0x7fd220130770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 1 --2- 172.21.15.204:0/1704012417 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 0x7fd220130770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd210010ce0 tx=0x7fd210008040 comp rx=0 tx=0).stop 2024-09-15T01:33:32.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 1 -- 172.21.15.204:0/1704012417 shutdown_connections 2024-09-15T01:33:32.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 1 --2- 172.21.15.204:0/1704012417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd22011d820 0x7fd22011dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 1 --2- 172.21.15.204:0/1704012417 >> v2:172.21.15.204:6812/28210 conn(0x7fd20c0397b0 0x7fd20c03bc60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 1 --2- 172.21.15.204:0/1704012417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd220130cb0 0x7fd22011d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 1 --2- 172.21.15.204:0/1704012417 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd208004de0 0x7fd220130770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 1 -- 172.21.15.204:0/1704012417 >> 172.21.15.204:0/1704012417 conn(0x7fd22006e860 msgr2=0x7fd2200a3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:32.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 1 -- 172.21.15.204:0/1704012417 shutdown_connections 2024-09-15T01:33:32.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.843+0000 7fd226b09640 1 -- 172.21.15.204:0/1704012417 wait complete. 2024-09-15T01:33:32.853 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule ls 2024-09-15T01:33:32.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.927+0000 7fcbfe31d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:32.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.927+0000 7fcbfe31d640 10 monclient: build_initial_monmap 2024-09-15T01:33:32.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.927+0000 7fcbfe31d640 10 monclient: monmap: 2024-09-15T01:33:32.929 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:32.929 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.929 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:32.927828+0000 2024-09-15T01:33:32.929 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:32.927828+0000 2024-09-15T01:33:32.929 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 1 Processor -- start 2024-09-15T01:33:32.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 1 -- start start 2024-09-15T01:33:32.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 10 monclient: init 2024-09-15T01:33:32.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:32.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 10 monclient: _add_conns ranks=[3,4,1,8,10,7,0,2,6,9,5] 2024-09-15T01:33:32.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbf80a4720 0x7fcbf81308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 10 monclient(hunting): picked mon.noname-g con 0x7fcbf80a4720 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:32.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbf8130de0 0x7fcbf81311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 10 monclient(hunting): picked mon.noname-c con 0x7fcbf8130de0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fcbf809e850 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf804b720 con 0x7fcbf809e850 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf8131730 con 0x7fcbf80a4720 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf80a4af0 con 0x7fcbf8130de0 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfe31d640 10 monclient(hunting): authenticate will time out at 828.570649s 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbf7fff640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbf80a4720 msgr2=0x7fcbf81308a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbf7fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbf80a4720 0x7fcbf81308a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfc8ba640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fcbf809e850 0x7fcbf809ecf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:58972/0 (socket says 172.21.15.204:58972) 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbfc8ba640 1 -- 172.21.15.204:0/665800366 learned_addr learned my addr 172.21.15.204:0/665800366 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:32.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbf77fe640 1 --2- 172.21.15.204:0/665800366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbf8130de0 0x7fcbf81311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbf77fe640 10 monclient(hunting): get_auth_request con 0x7fcbf8130de0 auth_method 0 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbf77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbf77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.928+0000 7fcbf77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf6ffd640 1 -- 172.21.15.204:0/665800366 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359615438 0 0) 0x7fcbf804b720 con 0x7fcbf809e850 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf6ffd640 10 monclient(hunting): my global_id is 4987 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf6ffd640 1 -- 172.21.15.204:0/665800366 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe00034e0 con 0x7fcbf809e850 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:32.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf77fe640 10 monclient(hunting): handle_auth_done global_id 4702 payload 274 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf6ffd640 1 -- 172.21.15.204:0/665800366 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3318870166 0 0) 0x7fcbe00034e0 con 0x7fcbf809e850 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf77fe640 1 -- 172.21.15.204:0/665800366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbf80a4720 msgr2=0x7fcbf81308a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf77fe640 1 --2- 172.21.15.204:0/665800366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbf80a4720 0x7fcbf81308a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf77fe640 1 -- 172.21.15.204:0/665800366 >> v1:172.21.15.204:6789/0 conn(0x7fcbf809e850 legacy=0x7fcbf809ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf77fe640 1 monclient: found mon.noname-c 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf77fe640 1 -- 172.21.15.204:0/665800366 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf80a0630 con 0x7fcbf8130de0 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.929+0000 7fcbf6ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf77fe640 1 --2- 172.21.15.204:0/665800366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbf8130de0 0x7fcbf81311f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009880 tx=0x7fcbec0090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8c8e6b1501f90eca server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf6ffd640 1 -- 172.21.15.204:0/665800366 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbf0001c70 con 0x7fcbf8130de0 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf6ffd640 10 monclient: dump: 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.942 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf6ffd640 1 -- 172.21.15.204:0/665800366 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec0313a0 con 0x7fcbf8130de0 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 5 monclient: authenticate success, global_id 4702 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 4 set_mon_vals no callback set 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf6ffd640 1 -- 172.21.15.204:0/665800366 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbec02ab40 con 0x7fcbf8130de0 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbf6ffd640 10 monclient: dump: 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.944 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 10 monclient: shutdown 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 1 -- 172.21.15.204:0/665800366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbf8130de0 msgr2=0x7fcbf81311f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 1 --2- 172.21.15.204:0/665800366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbf8130de0 0x7fcbf81311f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009880 tx=0x7fcbec0090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 1 -- 172.21.15.204:0/665800366 shutdown_connections 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 1 --2- 172.21.15.204:0/665800366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbf8130de0 0x7fcbf81311f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 1 --2- 172.21.15.204:0/665800366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbf80a4720 0x7fcbf81308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 1 -- 172.21.15.204:0/665800366 >> 172.21.15.204:0/665800366 conn(0x7fcbf80677e0 msgr2=0x7fcbf80a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.930+0000 7fcbfe31d640 1 -- 172.21.15.204:0/665800366 shutdown_connections 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.931+0000 7fcbfe31d640 1 -- 172.21.15.204:0/665800366 wait complete. 2024-09-15T01:33:32.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.931+0000 7fcbfe31d640 10 monclient: build_initial_monmap 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.931+0000 7fcbfe31d640 10 monclient: monmap: 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:32.946 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:32.947 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:32.947 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.931+0000 7fcbfe31d640 1 Processor -- start 2024-09-15T01:33:32.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.931+0000 7fcbfe31d640 1 -- start start 2024-09-15T01:33:32.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 10 monclient: init 2024-09-15T01:33:32.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:32.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 10 monclient: _add_conns ranks=[8,1,2,0,7,4,5,3,6] 2024-09-15T01:33:32.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcbf809e850 0x7fcbf811a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 10 monclient(hunting): picked mon.noname-e con 0x7fcbf809e850 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbf80a4720 0x7fcbf811ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 10 monclient(hunting): picked mon.noname-f con 0x7fcbf80a4720 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf8130de0 0x7fcbf811b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 10 monclient(hunting): picked mon.noname-b con 0x7fcbf8130de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf811c960 con 0x7fcbf80a4720 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf811dfe0 con 0x7fcbf8130de0 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf811c470 con 0x7fcbf809e850 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfe31d640 10 monclient(hunting): authenticate will time out at 828.574492s 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbf80a4720 0x7fcbf811ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbf80a4720 0x7fcbf811ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33306/0 (socket says 172.21.15.204:33306) 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 1 -- 172.21.15.204:0/673012194 learned_addr learned my addr 172.21.15.204:0/673012194 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 10 monclient(hunting): get_auth_request con 0x7fcbf80a4720 auth_method 0 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf7fff640 1 --2- 172.21.15.204:0/673012194 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcbf809e850 0x7fcbf811a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf7fff640 10 monclient(hunting): get_auth_request con 0x7fcbf809e850 auth_method 0 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfc8ba640 1 --2- 172.21.15.204:0/673012194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf8130de0 0x7fcbf811b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfc8ba640 10 monclient(hunting): get_auth_request con 0x7fcbf8130de0 auth_method 0 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfc8ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfc8ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfc8ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 10 monclient(hunting): handle_auth_done global_id 4996 payload 824 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 1 -- 172.21.15.204:0/673012194 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcbf809e850 msgr2=0x7fcbf811a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 1 --2- 172.21.15.204:0/673012194 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcbf809e850 0x7fcbf811a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 1 -- 172.21.15.204:0/673012194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf8130de0 msgr2=0x7fcbf811b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 1 --2- 172.21.15.204:0/673012194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf8130de0 0x7fcbf811b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 1 monclient: found mon.noname-f 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:32.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbf77fe640 1 -- 172.21.15.204:0/673012194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec009390 con 0x7fcbf80a4720 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.932+0000 7fcbfc8ba640 1 --2- 172.21.15.204:0/673012194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf8130de0 0x7fcbf811b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbf77fe640 1 --2- 172.21.15.204:0/673012194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbf80a4720 0x7fcbf811ac90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcbec035040 tx=0x7fcbec032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbdffff640 1 -- 172.21.15.204:0/673012194 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbec031e90 con 0x7fcbf80a4720 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbdffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbdffff640 10 monclient: dump: 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.951 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbdffff640 10 monclient: _finish_auth 0 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbdffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbdffff640 1 -- 172.21.15.204:0/673012194 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec032ad0 con 0x7fcbf80a4720 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbfe31d640 5 monclient: authenticate success, global_id 4996 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbfe31d640 10 monclient: _renew_subs 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbfe31d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbfe31d640 1 -- 172.21.15.204:0/673012194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf806d220 con 0x7fcbf80a4720 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbdffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbdffff640 1 -- 172.21.15.204:0/673012194 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbec032c40 con 0x7fcbf80a4720 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbdffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbdffff640 10 monclient: dump: 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbfe31d640 10 monclient: _renew_subs 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbfe31d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbf67fc640 4 set_mon_vals no callback set 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.933+0000 7fcbfe31d640 1 -- 172.21.15.204:0/673012194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf811bf60 con 0x7fcbf80a4720 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.934+0000 7fcbdffff640 1 -- 172.21.15.204:0/673012194 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fcbec039040 con 0x7fcbf80a4720 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.934+0000 7fcbfe31d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.934+0000 7fcbfe31d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.934+0000 7fcbfe31d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:32.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.934+0000 7fcbfe31d640 1 -- 172.21.15.204:0/673012194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf8070cc0 con 0x7fcbf80a4720 2024-09-15T01:33:32.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.934+0000 7fcbdffff640 1 --2- 172.21.15.204:0/673012194 >> v2:172.21.15.204:6812/28210 conn(0x7fcbe40397b0 0x7fcbe403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:32.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.934+0000 7fcbdffff640 1 -- 172.21.15.204:0/673012194 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 3594+0+0 (secure 0 0 0) 0x7fcbec06a2f0 con 0x7fcbf80a4720 2024-09-15T01:33:32.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.934+0000 7fcbf7fff640 1 --2- 172.21.15.204:0/673012194 >> v2:172.21.15.204:6812/28210 conn(0x7fcbe40397b0 0x7fcbe403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:32.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.934+0000 7fcbf7fff640 10 monclient: get_auth_request con 0x7fcbe40397b0 auth_method 0 2024-09-15T01:33:32.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.934+0000 7fcbf7fff640 1 --2- 172.21.15.204:0/673012194 >> v2:172.21.15.204:6812/28210 conn(0x7fcbe40397b0 0x7fcbe403bc60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8003850 tx=0x7fcbe80090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:32.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.938+0000 7fcbdffff640 1 -- 172.21.15.204:0/673012194 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcbec032650 con 0x7fcbf80a4720 2024-09-15T01:33:32.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.938+0000 7fcbdffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:32.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:32.938+0000 7fcbdffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:33.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.100+0000 7fcbfe31d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:33.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.100+0000 7fcbfe31d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:33.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.100+0000 7fcbfe31d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:33.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.100+0000 7fcbfe31d640 1 -- 172.21.15.204:0/673012194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7fcbf806a8f0 con 0x7fcbf80a4720 2024-09-15T01:33:33.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.100+0000 7fcbdffff640 1 -- 172.21.15.204:0/673012194 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v44) v1 ==== 65+0+29 (secure 0 0 0) 0x7fcbec040080 con 0x7fcbf80a4720 2024-09-15T01:33:33.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.100+0000 7fcbdffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:33.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.100+0000 7fcbdffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.195+0000 7fcbfe31d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.195+0000 7fcbfe31d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.195+0000 7fcbfe31d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.195+0000 7fcbfe31d640 1 -- 172.21.15.204:0/673012194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7fcbf811cb30 con 0x7fcbf80a4720 2024-09-15T01:33:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.195+0000 7fcbdffff640 1 -- 172.21.15.204:0/673012194 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v44) v1 ==== 65+0+29 (secure 0 0 0) 0x7fcbec037020 con 0x7fcbf80a4720 2024-09-15T01:33:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.195+0000 7fcbdffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.195+0000 7fcbdffff640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:33.197 INFO:tasks.workunit.client.0.smithi204.stdout:replicated_rule 2024-09-15T01:33:33.197 INFO:tasks.workunit.client.0.smithi204.stdout:erasure-code 2024-09-15T01:33:33.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 1 -- 172.21.15.204:0/673012194 >> v2:172.21.15.204:6812/28210 conn(0x7fcbe40397b0 msgr2=0x7fcbe403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:33.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 1 --2- 172.21.15.204:0/673012194 >> v2:172.21.15.204:6812/28210 conn(0x7fcbe40397b0 0x7fcbe403bc60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8003850 tx=0x7fcbe80090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 10 monclient: shutdown 2024-09-15T01:33:33.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:33.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 1 -- 172.21.15.204:0/673012194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbf80a4720 msgr2=0x7fcbf811ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:33.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 1 --2- 172.21.15.204:0/673012194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbf80a4720 0x7fcbf811ac90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcbec035040 tx=0x7fcbec032000 comp rx=0 tx=0).stop 2024-09-15T01:33:33.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 1 -- 172.21.15.204:0/673012194 shutdown_connections 2024-09-15T01:33:33.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 1 --2- 172.21.15.204:0/673012194 >> v2:172.21.15.204:6812/28210 conn(0x7fcbe40397b0 0x7fcbe403bc60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 1 --2- 172.21.15.204:0/673012194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbf80a4720 0x7fcbf811ac90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 1 --2- 172.21.15.204:0/673012194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf8130de0 0x7fcbf811b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 1 --2- 172.21.15.204:0/673012194 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcbf809e850 0x7fcbf811a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 1 -- 172.21.15.204:0/673012194 >> 172.21.15.204:0/673012194 conn(0x7fcbf80677e0 msgr2=0x7fcbf809d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:33.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 1 -- 172.21.15.204:0/673012194 shutdown_connections 2024-09-15T01:33:33.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.197+0000 7fcbfe31d640 1 -- 172.21.15.204:0/673012194 wait complete. 2024-09-15T01:33:33.208 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule list 2024-09-15T01:33:33.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.281+0000 7f0902b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:33.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.281+0000 7f0902b9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:33.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.281+0000 7f0902b9d640 10 monclient: monmap: 2024-09-15T01:33:33.282 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:33.283 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:33.283 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:33.281579+0000 2024-09-15T01:33:33.283 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:33.281579+0000 2024-09-15T01:33:33.283 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:33.283 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:33.284 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:33.284 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:33.284 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:33.284 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:33.284 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:33.284 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:33.284 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:33.284 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:33.284 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:33.284 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:33.284 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:33.285 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:33.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.281+0000 7f0902b9d640 1 Processor -- start 2024-09-15T01:33:33.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.281+0000 7f0902b9d640 1 -- start start 2024-09-15T01:33:33.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.281+0000 7f0902b9d640 10 monclient: init 2024-09-15T01:33:33.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:33.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 10 monclient: _add_conns ranks=[10,9,3,2,8,1,6,4,5,0,7] 2024-09-15T01:33:33.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f08fc09f380 0x7f08fc09e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 10 monclient(hunting): picked mon.noname-f con 0x7f08fc09f380 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:33.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f08fc06cc50 0x7f08fc09f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 10 monclient(hunting): picked mon.noname-a con 0x7f08fc06cc50 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:33.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08fc130790 0x7f08fc130bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 10 monclient(hunting): picked mon.noname-g con 0x7f08fc130790 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:33.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:33.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc06dfc0 con 0x7f08fc06cc50 2024-09-15T01:33:33.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:33.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc09ed90 con 0x7f08fc09f380 2024-09-15T01:33:33.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:33.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc09fc90 con 0x7f08fc130790 2024-09-15T01:33:33.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:33.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0902b9d640 10 monclient(hunting): authenticate will time out at 828.924461s 2024-09-15T01:33:33.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f090113a640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08fc130790 msgr2=0x7f08fc130bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:33.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f090113a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08fc130790 0x7f08fc130bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f08fc09f380 0x7f08fc09e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f08fc09f380 0x7f08fc09e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33314/0 (socket says 172.21.15.204:33314) 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 1 -- 172.21.15.204:0/2268040189 learned_addr learned my addr 172.21.15.204:0/2268040189 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 10 monclient(hunting): get_auth_request con 0x7f08fc09f380 auth_method 0 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f08fbfff640 1 --2- 172.21.15.204:0/2268040189 >> v2:172.21.15.178:3300/0 conn(0x7f08fc06cc50 0x7f08fc09f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f08fbfff640 10 monclient(hunting): get_auth_request con 0x7f08fc06cc50 auth_method 0 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f08fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:33.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f08fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f08fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f08fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f08fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f08fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 10 monclient(hunting): handle_auth_done global_id 5005 payload 274 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 1 -- 172.21.15.204:0/2268040189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08fc130790 msgr2=0x7f08fc130bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 1 --2- 172.21.15.204:0/2268040189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08fc130790 0x7f08fc130bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 1 -- 172.21.15.204:0/2268040189 >> v2:172.21.15.178:3300/0 conn(0x7f08fc06cc50 msgr2=0x7f08fc09f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 1 --2- 172.21.15.204:0/2268040189 >> v2:172.21.15.178:3300/0 conn(0x7f08fc06cc50 0x7f08fc09f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 10 monclient: _finish_hunting 0 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 1 monclient: found mon.noname-f 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:33.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.282+0000 7f0900939640 1 -- 172.21.15.204:0/2268040189 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc04b720 con 0x7f08fc09f380 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f0900939640 1 --2- 172.21.15.204:0/2268040189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08fc09f380 0x7f08fc09e850 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f08fc02adb0 tx=0x7f08ec009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb6cbb17512de8bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f08fb7fe640 1 -- 172.21.15.204:0/2268040189 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08ec007880 con 0x7f08fc09f380 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f08fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f08fb7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f08fb7fe640 10 monclient: dump: 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:33.298 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:33.299 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:33.299 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:33.299 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:33.299 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:33.299 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:33.299 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:33.299 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:33.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f08fb7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:33.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f08fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:33.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f08fb7fe640 1 -- 172.21.15.204:0/2268040189 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec02d050 con 0x7f08fc09f380 2024-09-15T01:33:33.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f0902b9d640 5 monclient: authenticate success, global_id 5005 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f08fb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f0902b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f08fb7fe640 1 -- 172.21.15.204:0/2268040189 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08ec030360 con 0x7f08fc09f380 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f08fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f0902b9d640 4 set_mon_vals no callback set 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f08fb7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.283+0000 7f08fb7fe640 10 monclient: dump: 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:33.300 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 10 monclient: shutdown 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 1 -- 172.21.15.204:0/2268040189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08fc09f380 msgr2=0x7f08fc09e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 1 --2- 172.21.15.204:0/2268040189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08fc09f380 0x7f08fc09e850 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f08fc02adb0 tx=0x7f08ec009840 comp rx=0 tx=0).stop 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 1 -- 172.21.15.204:0/2268040189 shutdown_connections 2024-09-15T01:33:33.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 1 --2- 172.21.15.204:0/2268040189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08fc130790 0x7f08fc130bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 1 --2- 172.21.15.204:0/2268040189 >> v2:172.21.15.178:3300/0 conn(0x7f08fc06cc50 0x7f08fc09f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 1 --2- 172.21.15.204:0/2268040189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08fc09f380 0x7f08fc09e850 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 1 -- 172.21.15.204:0/2268040189 >> 172.21.15.204:0/2268040189 conn(0x7f08fc0a3e60 msgr2=0x7f08fc0a4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 1 -- 172.21.15.204:0/2268040189 shutdown_connections 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 1 -- 172.21.15.204:0/2268040189 wait complete. 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.284+0000 7f0902b9d640 10 monclient: monmap: 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:33.302 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 1 Processor -- start 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 1 -- start start 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 10 monclient: init 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 10 monclient: _add_conns ranks=[3,1,0,5,7,6,2,8,4] 2024-09-15T01:33:33.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08fc06cc50 0x7f08fc12e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 10 monclient(hunting): picked mon.noname-g con 0x7f08fc06cc50 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:33.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08fc09f380 0x7f08fc12ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 10 monclient(hunting): picked mon.noname-f con 0x7f08fc09f380 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:33.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc130790 0x7f08fc12f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 10 monclient(hunting): picked mon.noname-a con 0x7f08fc130790 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:33.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:33.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc134190 con 0x7f08fc130790 2024-09-15T01:33:33.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:33.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc06d630 con 0x7f08fc09f380 2024-09-15T01:33:33.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:33.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc09ac50 con 0x7f08fc06cc50 2024-09-15T01:33:33.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:33.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0902b9d640 10 monclient(hunting): authenticate will time out at 828.927784s 2024-09-15T01:33:33.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0900939640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08fc06cc50 msgr2=0x7f08fc12e9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:33.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f0900939640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08fc06cc50 0x7f08fc12e9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:33.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f08fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08fc09f380 0x7f08fc12ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:33.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f08fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08fc09f380 0x7f08fc12ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33322/0 (socket says 172.21.15.204:33322) 2024-09-15T01:33:33.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f08fbfff640 1 -- 172.21.15.204:0/3111640489 learned_addr learned my addr 172.21.15.204:0/3111640489 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:33.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f08fbfff640 10 monclient(hunting): get_auth_request con 0x7f08fc09f380 auth_method 0 2024-09-15T01:33:33.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f08fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:33.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f08fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:33.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f08fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:33.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f08fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:33.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f08fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:33.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f08fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:33.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.285+0000 7f090113a640 1 --2- 172.21.15.204:0/3111640489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc130790 0x7f08fc12f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:33.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f090113a640 10 monclient(hunting): get_auth_request con 0x7f08fc130790 auth_method 0 2024-09-15T01:33:33.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f090113a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:33.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f090113a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:33.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f090113a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:33.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f090113a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:33.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f090113a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:33.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f090113a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:33.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08fbfff640 10 monclient(hunting): handle_auth_done global_id 5014 payload 824 2024-09-15T01:33:33.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08fbfff640 1 -- 172.21.15.204:0/3111640489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08fc06cc50 msgr2=0x7f08fc12e9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:33.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08fbfff640 1 --2- 172.21.15.204:0/3111640489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08fc06cc50 0x7f08fc12e9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08fbfff640 1 -- 172.21.15.204:0/3111640489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc130790 msgr2=0x7f08fc12f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:33.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08fbfff640 1 --2- 172.21.15.204:0/3111640489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc130790 0x7f08fc12f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08fbfff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:33.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08fbfff640 1 monclient: found mon.noname-f 2024-09-15T01:33:33.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:33.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08fbfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:33.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08fbfff640 1 -- 172.21.15.204:0/3111640489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec009ae0 con 0x7f08fc09f380 2024-09-15T01:33:33.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08fbfff640 1 --2- 172.21.15.204:0/3111640489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08fc09f380 0x7f08fc12ef10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f08f000ea50 tx=0x7f08f00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:33.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08f8ff9640 1 -- 172.21.15.204:0/3111640489 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08f000ce90 con 0x7f08fc09f380 2024-09-15T01:33:33.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08f8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:33.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08f8ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:33.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08f8ff9640 10 monclient: dump: 2024-09-15T01:33:33.308 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:33.309 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:33.309 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.309 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.309 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:33.309 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:33.309 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:33.309 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:33.309 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:33.310 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:33.310 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:33.310 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:33.310 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:33.310 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:33.310 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:33.310 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:33.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08f8ff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:33.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08f8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:33.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08f8ff9640 1 -- 172.21.15.204:0/3111640489 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f0009600 con 0x7f08fc09f380 2024-09-15T01:33:33.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f0902b9d640 5 monclient: authenticate success, global_id 5014 2024-09-15T01:33:33.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f0902b9d640 10 monclient: _renew_subs 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f0902b9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f0902b9d640 1 -- 172.21.15.204:0/3111640489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc135190 con 0x7f08fc09f380 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08f8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08f8ff9640 1 -- 172.21.15.204:0/3111640489 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08f00138f0 con 0x7f08fc09f380 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08f8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08f8ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08f8ff9640 10 monclient: dump: 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:33.311 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f0902b9d640 10 monclient: _renew_subs 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f0902b9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:33.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f0902b9d640 1 -- 172.21.15.204:0/3111640489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08fc132360 con 0x7f08fc09f380 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.286+0000 7f08faffd640 4 set_mon_vals no callback set 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.287+0000 7f08f8ff9640 1 -- 172.21.15.204:0/3111640489 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f08f0013470 con 0x7f08fc09f380 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.287+0000 7f0902b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.287+0000 7f08f8ff9640 1 --2- 172.21.15.204:0/3111640489 >> v2:172.21.15.204:6812/28210 conn(0x7f08e80397b0 0x7f08e803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.287+0000 7f0902b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.287+0000 7f0902b9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.287+0000 7f0902b9d640 1 -- 172.21.15.204:0/3111640489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08fc070cc0 con 0x7f08fc09f380 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.287+0000 7f08f8ff9640 1 -- 172.21.15.204:0/3111640489 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 3594+0+0 (secure 0 0 0) 0x7f08f004b5b0 con 0x7f08fc09f380 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.287+0000 7f0900939640 1 --2- 172.21.15.204:0/3111640489 >> v2:172.21.15.204:6812/28210 conn(0x7f08e80397b0 0x7f08e803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.287+0000 7f0900939640 10 monclient: get_auth_request con 0x7f08e80397b0 auth_method 0 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.287+0000 7f0900939640 1 --2- 172.21.15.204:0/3111640489 >> v2:172.21.15.204:6812/28210 conn(0x7f08e80397b0 0x7f08e803bc60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f08ec033000 tx=0x7f08ec036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.291+0000 7f08f8ff9640 1 -- 172.21.15.204:0/3111640489 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f08f001c450 con 0x7f08fc09f380 2024-09-15T01:33:33.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.291+0000 7f08f8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:33.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.291+0000 7f08f8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:33.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.456+0000 7f0902b9d640 10 start_mon_command cmd=[{"prefix": "osd crush rule list"}] 2024-09-15T01:33:33.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.456+0000 7f0902b9d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule list"}] 2024-09-15T01:33:33.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.456+0000 7f0902b9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:33.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.456+0000 7f0902b9d640 1 -- 172.21.15.204:0/3111640489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule list"} v 0) v1 -- 0x7f08fc09ae80 con 0x7f08fc09f380 2024-09-15T01:33:33.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.456+0000 7f08f8ff9640 1 -- 172.21.15.204:0/3111640489 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule list"}]=0 v44) v1 ==== 67+0+29 (secure 0 0 0) 0x7f08f001b3b0 con 0x7f08fc09f380 2024-09-15T01:33:33.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.457+0000 7f08f8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule list"}] 2024-09-15T01:33:33.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.457+0000 7f08f8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:33.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.561+0000 7f0902b9d640 10 start_mon_command cmd=[{"prefix": "osd crush rule list"}] 2024-09-15T01:33:33.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.561+0000 7f0902b9d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule list"}] 2024-09-15T01:33:33.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.561+0000 7f0902b9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:33.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.561+0000 7f0902b9d640 1 -- 172.21.15.204:0/3111640489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule list"} v 0) v1 -- 0x7f08fc0008d0 con 0x7f08fc09f380 2024-09-15T01:33:33.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.561+0000 7f08f8ff9640 1 -- 172.21.15.204:0/3111640489 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule list"}]=0 v44) v1 ==== 67+0+29 (secure 0 0 0) 0x7f08f001b590 con 0x7f08fc09f380 2024-09-15T01:33:33.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.561+0000 7f08f8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule list"}] 2024-09-15T01:33:33.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.561+0000 7f08f8ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:33.563 INFO:tasks.workunit.client.0.smithi204.stdout:replicated_rule 2024-09-15T01:33:33.564 INFO:tasks.workunit.client.0.smithi204.stdout:erasure-code 2024-09-15T01:33:33.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.563+0000 7f0902b9d640 1 -- 172.21.15.204:0/3111640489 >> v2:172.21.15.204:6812/28210 conn(0x7f08e80397b0 msgr2=0x7f08e803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:33.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.563+0000 7f0902b9d640 1 --2- 172.21.15.204:0/3111640489 >> v2:172.21.15.204:6812/28210 conn(0x7f08e80397b0 0x7f08e803bc60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f08ec033000 tx=0x7f08ec036000 comp rx=0 tx=0).stop 2024-09-15T01:33:33.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.563+0000 7f0902b9d640 10 monclient: shutdown 2024-09-15T01:33:33.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.563+0000 7f0902b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:33.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.563+0000 7f0902b9d640 1 -- 172.21.15.204:0/3111640489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08fc09f380 msgr2=0x7f08fc12ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:33.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.563+0000 7f0902b9d640 1 --2- 172.21.15.204:0/3111640489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08fc09f380 0x7f08fc12ef10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f08f000ea50 tx=0x7f08f00090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.564+0000 7f0902b9d640 1 -- 172.21.15.204:0/3111640489 shutdown_connections 2024-09-15T01:33:33.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.564+0000 7f0902b9d640 1 --2- 172.21.15.204:0/3111640489 >> v2:172.21.15.204:6812/28210 conn(0x7f08e80397b0 0x7f08e803bc60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.564+0000 7f0902b9d640 1 --2- 172.21.15.204:0/3111640489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc130790 0x7f08fc12f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.564+0000 7f0902b9d640 1 --2- 172.21.15.204:0/3111640489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08fc09f380 0x7f08fc12ef10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.564+0000 7f0902b9d640 1 --2- 172.21.15.204:0/3111640489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08fc06cc50 0x7f08fc12e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.564+0000 7f0902b9d640 1 -- 172.21.15.204:0/3111640489 >> 172.21.15.204:0/3111640489 conn(0x7f08fc0a3e60 msgr2=0x7f08fc069340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:33.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.564+0000 7f0902b9d640 1 -- 172.21.15.204:0/3111640489 shutdown_connections 2024-09-15T01:33:33.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.564+0000 7f0902b9d640 1 -- 172.21.15.204:0/3111640489 wait complete. 2024-09-15T01:33:33.574 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule create-simple foo default host 2024-09-15T01:33:33.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.651+0000 7fe83159d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:33.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.651+0000 7fe83159d640 10 monclient: build_initial_monmap 2024-09-15T01:33:33.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.651+0000 7fe83159d640 10 monclient: monmap: 2024-09-15T01:33:33.653 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:33.653 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:33.653 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:33.652057+0000 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:33.652057+0000 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:33.654 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:33.655 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:33.655 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:33.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 1 Processor -- start 2024-09-15T01:33:33.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 1 -- start start 2024-09-15T01:33:33.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 10 monclient: init 2024-09-15T01:33:33.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:33.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 10 monclient: _add_conns ranks=[2,5,1,6,10,3,9,7,0,8,4] 2024-09-15T01:33:33.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82c0969e0 0x7fe82c068240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 10 monclient(hunting): picked mon.noname-b con 0x7fe82c0969e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:33.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe82c096db0 0x7fe82c097180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 10 monclient(hunting): picked mon.noname-h con 0x7fe82c096db0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:33.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe82c12fa60 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:33.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82c0648a0 con 0x7fe82c12fa60 2024-09-15T01:33:33.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c098310 con 0x7fe82c0969e0 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c098940 con 0x7fe82c096db0 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe83159d640 10 monclient(hunting): authenticate will time out at 829.294834s 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe82a7fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe82c096db0 0x7fe82c097180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe82b7fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fe82c12fa60 0x7fe82c12ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:58974/0 (socket says 172.21.15.204:58974) 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.652+0000 7fe82b7fe640 1 -- 172.21.15.204:0/2403127432 learned_addr learned my addr 172.21.15.204:0/2403127432 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 10 monclient(hunting): get_auth_request con 0x7fe82c096db0 auth_method 0 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82affd640 1 --2- 172.21.15.204:0/2403127432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82c0969e0 0x7fe82c068240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:33.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82affd640 10 monclient(hunting): get_auth_request con 0x7fe82c0969e0 auth_method 0 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe829ffb640 1 -- 172.21.15.204:0/2403127432 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692120862 0 0) 0x7fe82c0648a0 con 0x7fe82c12fa60 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe829ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe829ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe829ffb640 10 monclient(hunting): my global_id is 5023 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe829ffb640 1 -- 172.21.15.204:0/2403127432 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8140034e0 con 0x7fe82c12fa60 2024-09-15T01:33:33.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 10 monclient(hunting): handle_auth_done global_id 4775 payload 274 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 1 -- 172.21.15.204:0/2403127432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82c0969e0 msgr2=0x7fe82c068240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 1 --2- 172.21.15.204:0/2403127432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82c0969e0 0x7fe82c068240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 1 -- 172.21.15.204:0/2403127432 >> v1:172.21.15.204:6789/0 conn(0x7fe82c12fa60 legacy=0x7fe82c12ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 1 monclient: found mon.noname-h 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.653+0000 7fe82a7fc640 1 -- 172.21.15.204:0/2403127432 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c099770 con 0x7fe82c096db0 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe82a7fc640 1 --2- 172.21.15.204:0/2403127432 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe82c096db0 0x7fe82c097180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe820009880 tx=0x7fe8200090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=c2c0730bc6837c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe829ffb640 1 -- 172.21.15.204:0/2403127432 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe82002ac50 con 0x7fe82c096db0 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe829ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe829ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:33.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe829ffb640 10 monclient: dump: 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:33.666 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe829ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe829ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe829ffb640 1 -- 172.21.15.204:0/2403127432 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82002add0 con 0x7fe82c096db0 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe83159d640 5 monclient: authenticate success, global_id 4775 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe83159d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe829ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe83159d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe829ffb640 1 -- 172.21.15.204:0/2403127432 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe820031ca0 con 0x7fe82c096db0 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe83159d640 4 set_mon_vals no callback set 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe829ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:33.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe829ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe829ffb640 10 monclient: dump: 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:33.668 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe83159d640 10 monclient: shutdown 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe83159d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe83159d640 1 -- 172.21.15.204:0/2403127432 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe82c096db0 msgr2=0x7fe82c097180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe83159d640 1 --2- 172.21.15.204:0/2403127432 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe82c096db0 0x7fe82c097180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe820009880 tx=0x7fe8200090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe83159d640 1 -- 172.21.15.204:0/2403127432 shutdown_connections 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe83159d640 1 --2- 172.21.15.204:0/2403127432 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe82c096db0 0x7fe82c097180 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe83159d640 1 --2- 172.21.15.204:0/2403127432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe82c0969e0 0x7fe82c068240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.654+0000 7fe83159d640 1 -- 172.21.15.204:0/2403127432 >> 172.21.15.204:0/2403127432 conn(0x7fe82c0a2ce0 msgr2=0x7fe82c0a0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:33.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.655+0000 7fe83159d640 1 -- 172.21.15.204:0/2403127432 shutdown_connections 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.655+0000 7fe83159d640 1 -- 172.21.15.204:0/2403127432 wait complete. 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.655+0000 7fe83159d640 10 monclient: build_initial_monmap 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.655+0000 7fe83159d640 10 monclient: monmap: 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:33.670 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.655+0000 7fe83159d640 1 Processor -- start 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 1 -- start start 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 10 monclient: init 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 10 monclient: _add_conns ranks=[4,5,1,8,7,2,6,3,0] 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe82c0969e0 0x7fe82c118dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 10 monclient(hunting): picked mon.noname-c con 0x7fe82c0969e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe82c096db0 0x7fe82c067130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 10 monclient(hunting): picked mon.noname-h con 0x7fe82c096db0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe82c12fa60 0x7fe82c12eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 10 monclient(hunting): picked mon.noname-f con 0x7fe82c12fa60 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c11f040 con 0x7fe82c12fa60 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c067ae0 con 0x7fe82c0969e0 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c09bc40 con 0x7fe82c096db0 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe83159d640 10 monclient(hunting): authenticate will time out at 829.298485s 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe82c12fa60 0x7fe82c12eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe82c096db0 0x7fe82c067130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:33.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe82c12fa60 0x7fe82c12eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33330/0 (socket says 172.21.15.204:33330) 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82b7fe640 1 -- 172.21.15.204:0/2047858219 learned_addr learned my addr 172.21.15.204:0/2047858219 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 10 monclient(hunting): get_auth_request con 0x7fe82c096db0 auth_method 0 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82affd640 1 --2- 172.21.15.204:0/2047858219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe82c0969e0 0x7fe82c118dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82b7fe640 10 monclient(hunting): get_auth_request con 0x7fe82c12fa60 auth_method 0 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82affd640 10 monclient(hunting): get_auth_request con 0x7fe82c0969e0 auth_method 0 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:33.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 10 monclient(hunting): handle_auth_done global_id 4784 payload 824 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 1 -- 172.21.15.204:0/2047858219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe82c0969e0 msgr2=0x7fe82c118dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 1 --2- 172.21.15.204:0/2047858219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe82c0969e0 0x7fe82c118dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 1 -- 172.21.15.204:0/2047858219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe82c12fa60 msgr2=0x7fe82c12eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:33.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 1 --2- 172.21.15.204:0/2047858219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe82c12fa60 0x7fe82c12eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 1 monclient: found mon.noname-h 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82a7fc640 1 -- 172.21.15.204:0/2047858219 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820009390 con 0x7fe82c096db0 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.656+0000 7fe82b7fe640 1 --2- 172.21.15.204:0/2047858219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe82c12fa60 0x7fe82c12eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe82a7fc640 1 --2- 172.21.15.204:0/2047858219 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe82c096db0 0x7fe82c067130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe820031000 tx=0x7fe820032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe812ffd640 1 -- 172.21.15.204:0/2047858219 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe82003b870 con 0x7fe82c096db0 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe812ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe812ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe812ffd640 10 monclient: dump: 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:33.675 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:33.676 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe812ffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe812ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe812ffd640 1 -- 172.21.15.204:0/2047858219 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82003b9d0 con 0x7fe82c096db0 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe83159d640 5 monclient: authenticate success, global_id 4784 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe812ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe812ffd640 1 -- 172.21.15.204:0/2047858219 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe8200313a0 con 0x7fe82c096db0 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe83159d640 10 monclient: _renew_subs 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe83159d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe83159d640 1 -- 172.21.15.204:0/2047858219 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82c0a0d80 con 0x7fe82c096db0 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe812ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe812ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe812ffd640 10 monclient: dump: 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:33.677 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:33.678 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe8297fa640 4 set_mon_vals no callback set 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe83159d640 10 monclient: _renew_subs 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe83159d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.657+0000 7fe83159d640 1 -- 172.21.15.204:0/2047858219 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82c12f430 con 0x7fe82c096db0 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.658+0000 7fe812ffd640 1 -- 172.21.15.204:0/2047858219 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe820039040 con 0x7fe82c096db0 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.658+0000 7fe83159d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.658+0000 7fe83159d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.658+0000 7fe83159d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.658+0000 7fe83159d640 1 -- 172.21.15.204:0/2047858219 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82c070b80 con 0x7fe82c096db0 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.658+0000 7fe812ffd640 1 --2- 172.21.15.204:0/2047858219 >> v2:172.21.15.204:6812/28210 conn(0x7fe8180397b0 0x7fe81803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.658+0000 7fe812ffd640 1 -- 172.21.15.204:0/2047858219 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 3594+0+0 (secure 0 0 0) 0x7fe82006aa00 con 0x7fe82c096db0 2024-09-15T01:33:33.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.658+0000 7fe82affd640 1 --2- 172.21.15.204:0/2047858219 >> v2:172.21.15.204:6812/28210 conn(0x7fe8180397b0 0x7fe81803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:33.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.658+0000 7fe82affd640 10 monclient: get_auth_request con 0x7fe8180397b0 auth_method 0 2024-09-15T01:33:33.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.658+0000 7fe82affd640 1 --2- 172.21.15.204:0/2047858219 >> v2:172.21.15.204:6812/28210 conn(0x7fe8180397b0 0x7fe81803bc60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009d20 tx=0x7fe81c0090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:33.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.662+0000 7fe812ffd640 1 -- 172.21.15.204:0/2047858219 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe820030b20 con 0x7fe82c096db0 2024-09-15T01:33:33.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.662+0000 7fe812ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:33.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.662+0000 7fe812ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:33.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.825+0000 7fe83159d640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T01:33:33.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.825+0000 7fe83159d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T01:33:33.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.825+0000 7fe83159d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:33.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:33.825+0000 7fe83159d640 1 -- 172.21.15.204:0/2047858219 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) v1 -- 0x7fe82c11f3e0 con 0x7fe82c096db0 2024-09-15T01:33:34.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.519+0000 7fe812ffd640 1 -- 172.21.15.204:0/2047858219 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 v45) v1 ==== 126+0+0 (secure 0 0 0) 0x7fe820037020 con 0x7fe82c096db0 2024-09-15T01:33:34.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.519+0000 7fe812ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T01:33:34.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.519+0000 7fe812ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:34.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.614+0000 7fe83159d640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T01:33:34.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.614+0000 7fe83159d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T01:33:34.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.614+0000 7fe83159d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:34.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.614+0000 7fe83159d640 1 -- 172.21.15.204:0/2047858219 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) v1 -- 0x7fe82c0008d0 con 0x7fe82c096db0 2024-09-15T01:33:34.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.615+0000 7fe812ffd640 1 -- 172.21.15.204:0/2047858219 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 rule foo already exists v45) v1 ==== 149+0+0 (secure 0 0 0) 0x7fe820037070 con 0x7fe82c096db0 2024-09-15T01:33:34.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.615+0000 7fe812ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T01:33:34.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.615+0000 7fe812ffd640 10 monclient: _finish_command 3 = system:0 rule foo already exists 2024-09-15T01:33:34.617 INFO:tasks.workunit.client.0.smithi204.stderr:rule foo already exists 2024-09-15T01:33:34.618 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 1 -- 172.21.15.204:0/2047858219 >> v2:172.21.15.204:6812/28210 conn(0x7fe8180397b0 msgr2=0x7fe81803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:34.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 1 --2- 172.21.15.204:0/2047858219 >> v2:172.21.15.204:6812/28210 conn(0x7fe8180397b0 0x7fe81803bc60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009d20 tx=0x7fe81c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 10 monclient: shutdown 2024-09-15T01:33:34.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:34.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 1 -- 172.21.15.204:0/2047858219 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe82c096db0 msgr2=0x7fe82c067130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:34.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 1 --2- 172.21.15.204:0/2047858219 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe82c096db0 0x7fe82c067130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe820031000 tx=0x7fe820032000 comp rx=0 tx=0).stop 2024-09-15T01:33:34.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 1 -- 172.21.15.204:0/2047858219 shutdown_connections 2024-09-15T01:33:34.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 1 --2- 172.21.15.204:0/2047858219 >> v2:172.21.15.204:6812/28210 conn(0x7fe8180397b0 0x7fe81803bc60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 1 --2- 172.21.15.204:0/2047858219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe82c12fa60 0x7fe82c12eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 1 --2- 172.21.15.204:0/2047858219 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe82c096db0 0x7fe82c067130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 1 --2- 172.21.15.204:0/2047858219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe82c0969e0 0x7fe82c118dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.620 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 1 -- 172.21.15.204:0/2047858219 >> 172.21.15.204:0/2047858219 conn(0x7fe82c0a2ce0 msgr2=0x7fe82c09a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:34.620 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 1 -- 172.21.15.204:0/2047858219 shutdown_connections 2024-09-15T01:33:34.620 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.617+0000 7fe83159d640 1 -- 172.21.15.204:0/2047858219 wait complete. 2024-09-15T01:33:34.627 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule create-simple foo default host 2024-09-15T01:33:34.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.702+0000 7fb8dd289640 10 monclient: get_monmap_and_config 2024-09-15T01:33:34.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.702+0000 7fb8dd289640 10 monclient: build_initial_monmap 2024-09-15T01:33:34.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.702+0000 7fb8dd289640 10 monclient: monmap: 2024-09-15T01:33:34.704 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:34.704 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:34.704 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:34.703284+0000 2024-09-15T01:33:34.704 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:34.703284+0000 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:34.705 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:34.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 1 Processor -- start 2024-09-15T01:33:34.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 1 -- start start 2024-09-15T01:33:34.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 10 monclient: init 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 10 monclient: _add_conns ranks=[10,0,3,1,6,7,4,2,5,9,8] 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb8d80a2cd0 0x7fb8d8132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 10 monclient(hunting): picked mon.noname-f con 0x7fb8d80a2cd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb8d809fa20 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8d8137c80 0x7fb8d813a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 10 monclient(hunting): picked mon.noname-g con 0x7fb8d8137c80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d80648a0 con 0x7fb8d809fa20 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d80a30a0 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d80a1db0 con 0x7fb8d8137c80 2024-09-15T01:33:34.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.703+0000 7fb8dd289640 10 monclient(hunting): authenticate will time out at 830.345992s 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d77fe640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8d8137c80 msgr2=0x7fb8d813a060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d77fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8d8137c80 0x7fb8d813a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb8d80a2cd0 0x7fb8d8132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb8d80a2cd0 0x7fb8d8132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33342/0 (socket says 172.21.15.204:33342) 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 1 -- 172.21.15.204:0/1092879188 learned_addr learned my addr 172.21.15.204:0/1092879188 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 10 monclient(hunting): get_auth_request con 0x7fb8d80a2cd0 auth_method 0 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:34.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 10 monclient(hunting): handle_auth_done global_id 5041 payload 274 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 1 -- 172.21.15.204:0/1092879188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8d8137c80 msgr2=0x7fb8d813a060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 1 --2- 172.21.15.204:0/1092879188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8d8137c80 0x7fb8d813a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 1 -- 172.21.15.204:0/1092879188 >> v1:172.21.15.178:6789/0 conn(0x7fb8d809fa20 legacy=0x7fb8d809fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 1 monclient: found mon.noname-f 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.704+0000 7fb8d6ffd640 1 -- 172.21.15.204:0/1092879188 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d809dc50 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d6ffd640 1 --2- 172.21.15.204:0/1092879188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d80a2cd0 0x7fb8d8132c00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009f20 tx=0x7fb8d0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f1108efb280cd7d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d5ffb640 1 -- 172.21.15.204:0/1092879188 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb8d0007880 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d5ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:34.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d5ffb640 10 monclient: dump: 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:34.718 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d5ffb640 1 -- 172.21.15.204:0/1092879188 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d002d050 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 5 monclient: authenticate success, global_id 5041 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d5ffb640 1 -- 172.21.15.204:0/1092879188 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb8d0030360 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 4 set_mon_vals no callback set 2024-09-15T01:33:34.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:34.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8d5ffb640 10 monclient: dump: 2024-09-15T01:33:34.720 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:34.720 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:34.720 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:34.720 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:34.720 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:34.720 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:34.720 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:34.720 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:34.720 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 10 monclient: shutdown 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 1 -- 172.21.15.204:0/1092879188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d80a2cd0 msgr2=0x7fb8d8132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 1 --2- 172.21.15.204:0/1092879188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d80a2cd0 0x7fb8d8132c00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009f20 tx=0x7fb8d0009840 comp rx=0 tx=0).stop 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 1 -- 172.21.15.204:0/1092879188 shutdown_connections 2024-09-15T01:33:34.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 1 --2- 172.21.15.204:0/1092879188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8d8137c80 0x7fb8d813a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 1 --2- 172.21.15.204:0/1092879188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d80a2cd0 0x7fb8d8132c00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 1 -- 172.21.15.204:0/1092879188 >> 172.21.15.204:0/1092879188 conn(0x7fb8d806e860 msgr2=0x7fb8d806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 1 -- 172.21.15.204:0/1092879188 shutdown_connections 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.705+0000 7fb8dd289640 1 -- 172.21.15.204:0/1092879188 wait complete. 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.706+0000 7fb8dd289640 10 monclient: build_initial_monmap 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.706+0000 7fb8dd289640 10 monclient: monmap: 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:34.723 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:34.723 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:34.723 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:34.723 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:34.723 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:34.723 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:34.723 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:34.723 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:34.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.706+0000 7fb8dd289640 1 Processor -- start 2024-09-15T01:33:34.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.706+0000 7fb8dd289640 1 -- start start 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.706+0000 7fb8dd289640 10 monclient: init 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 10 monclient: _add_conns ranks=[1,5,3,4,0,7,8,6,2] 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d809fa20 0x7fb8d80994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 10 monclient(hunting): picked mon.noname-f con 0x7fb8d809fa20 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb8d80a2cd0 0x7fb8d80999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 10 monclient(hunting): picked mon.noname-h con 0x7fb8d80a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8d8137c80 0x7fb8d8099f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 10 monclient(hunting): picked mon.noname-g con 0x7fb8d8137c80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d809d3d0 con 0x7fb8d809fa20 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d809cb60 con 0x7fb8d8137c80 2024-09-15T01:33:34.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d809bdf0 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8dd289640 10 monclient(hunting): authenticate will time out at 830.349345s 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d77fe640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8d8137c80 msgr2=0x7fb8d8099f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d77fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8d8137c80 0x7fb8d8099f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d809fa20 0x7fb8d80994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb8d80a2cd0 0x7fb8d80999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d809fa20 0x7fb8d80994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33346/0 (socket says 172.21.15.204:33346) 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d6ffd640 1 -- 172.21.15.204:0/3655382952 learned_addr learned my addr 172.21.15.204:0/3655382952 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d6ffd640 10 monclient(hunting): get_auth_request con 0x7fb8d809fa20 auth_method 0 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:34.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 10 monclient(hunting): get_auth_request con 0x7fb8d80a2cd0 auth_method 0 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 10 monclient(hunting): handle_auth_done global_id 4793 payload 824 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 1 -- 172.21.15.204:0/3655382952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8d8137c80 msgr2=0x7fb8d8099f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:34.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 1 --2- 172.21.15.204:0/3655382952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8d8137c80 0x7fb8d8099f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 1 -- 172.21.15.204:0/3655382952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d809fa20 msgr2=0x7fb8d80994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 1 --2- 172.21.15.204:0/3655382952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d809fa20 0x7fb8d80994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 10 monclient: _finish_hunting 0 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 1 monclient: found mon.noname-h 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 1 -- 172.21.15.204:0/3655382952 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0009ae0 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d6ffd640 1 --2- 172.21.15.204:0/3655382952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d809fa20 0x7fb8d80994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.707+0000 7fb8d67fc640 1 --2- 172.21.15.204:0/3655382952 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb8d80a2cd0 0x7fb8d80999e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0001eb0 tx=0x7fb8c0007e10 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 1 -- 172.21.15.204:0/3655382952 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb8c0011a10 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 10 monclient: dump: 2024-09-15T01:33:34.727 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:34.728 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:34.728 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:34.728 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:34.728 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:34.728 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:34.728 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:34.728 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:34.728 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:34.728 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:34.728 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:34.728 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:34.728 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 1 -- 172.21.15.204:0/3655382952 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c0011bd0 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8dd289640 5 monclient: authenticate success, global_id 4793 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8dd289640 10 monclient: _renew_subs 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8dd289640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8dd289640 1 -- 172.21.15.204:0/3655382952 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d809b0a0 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 1 -- 172.21.15.204:0/3655382952 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb8c0011d40 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 10 monclient: dump: 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:34.730 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8d57fa640 4 set_mon_vals no callback set 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8dd289640 10 monclient: _renew_subs 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8dd289640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8dd289640 1 -- 172.21.15.204:0/3655382952 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d811c5f0 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.708+0000 7fb8ceffd640 1 -- 172.21.15.204:0/3655382952 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb8c0011590 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.709+0000 7fb8dd289640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.709+0000 7fb8dd289640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.709+0000 7fb8dd289640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:34.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.709+0000 7fb8dd289640 1 -- 172.21.15.204:0/3655382952 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d811ca70 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.709+0000 7fb8ceffd640 1 --2- 172.21.15.204:0/3655382952 >> v2:172.21.15.204:6812/28210 conn(0x7fb8b80397e0 0x7fb8b803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:34.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.709+0000 7fb8ceffd640 1 -- 172.21.15.204:0/3655382952 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 3653+0+0 (secure 0 0 0) 0x7fb8c0049f00 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.709+0000 7fb8d6ffd640 1 --2- 172.21.15.204:0/3655382952 >> v2:172.21.15.204:6812/28210 conn(0x7fb8b80397e0 0x7fb8b803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:34.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.709+0000 7fb8d6ffd640 10 monclient: get_auth_request con 0x7fb8b80397e0 auth_method 0 2024-09-15T01:33:34.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.709+0000 7fb8d6ffd640 1 --2- 172.21.15.204:0/3655382952 >> v2:172.21.15.204:6812/28210 conn(0x7fb8b80397e0 0x7fb8b803bc90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009ab0 tx=0x7fb8d0036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:34.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.713+0000 7fb8ceffd640 1 -- 172.21.15.204:0/3655382952 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb8c0011840 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.713+0000 7fb8ceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:34.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.713+0000 7fb8ceffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:34.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.871+0000 7fb8dd289640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T01:33:34.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.871+0000 7fb8dd289640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T01:33:34.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.871+0000 7fb8dd289640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:34.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.871+0000 7fb8dd289640 1 -- 172.21.15.204:0/3655382952 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) v1 -- 0x7fb8d8070b80 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.872+0000 7fb8ceffd640 1 -- 172.21.15.204:0/3655382952 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 rule foo already exists v45) v1 ==== 149+0+0 (secure 0 0 0) 0x7fb8c0017070 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.872+0000 7fb8ceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T01:33:34.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.872+0000 7fb8ceffd640 10 monclient: _finish_command 2 = system:0 rule foo already exists 2024-09-15T01:33:34.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.966+0000 7fb8dd289640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T01:33:34.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.966+0000 7fb8dd289640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T01:33:34.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.966+0000 7fb8dd289640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:34.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.966+0000 7fb8dd289640 1 -- 172.21.15.204:0/3655382952 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) v1 -- 0x7fb8d80008d0 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.966+0000 7fb8ceffd640 1 -- 172.21.15.204:0/3655382952 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 rule foo already exists v45) v1 ==== 149+0+0 (secure 0 0 0) 0x7fb8c0010e00 con 0x7fb8d80a2cd0 2024-09-15T01:33:34.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.966+0000 7fb8ceffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-09-15T01:33:34.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.966+0000 7fb8ceffd640 10 monclient: _finish_command 3 = system:0 rule foo already exists 2024-09-15T01:33:34.968 INFO:tasks.workunit.client.0.smithi204.stderr:rule foo already exists 2024-09-15T01:33:34.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 1 -- 172.21.15.204:0/3655382952 >> v2:172.21.15.204:6812/28210 conn(0x7fb8b80397e0 msgr2=0x7fb8b803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:34.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 1 --2- 172.21.15.204:0/3655382952 >> v2:172.21.15.204:6812/28210 conn(0x7fb8b80397e0 0x7fb8b803bc90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009ab0 tx=0x7fb8d0036000 comp rx=0 tx=0).stop 2024-09-15T01:33:34.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 10 monclient: shutdown 2024-09-15T01:33:34.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:34.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 1 -- 172.21.15.204:0/3655382952 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb8d80a2cd0 msgr2=0x7fb8d80999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:34.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 1 --2- 172.21.15.204:0/3655382952 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb8d80a2cd0 0x7fb8d80999e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0001eb0 tx=0x7fb8c0007e10 comp rx=0 tx=0).stop 2024-09-15T01:33:34.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 1 -- 172.21.15.204:0/3655382952 shutdown_connections 2024-09-15T01:33:34.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 1 --2- 172.21.15.204:0/3655382952 >> v2:172.21.15.204:6812/28210 conn(0x7fb8b80397e0 0x7fb8b803bc90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 1 --2- 172.21.15.204:0/3655382952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8d8137c80 0x7fb8d8099f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 1 --2- 172.21.15.204:0/3655382952 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb8d80a2cd0 0x7fb8d80999e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 1 --2- 172.21.15.204:0/3655382952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d809fa20 0x7fb8d80994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:34.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 1 -- 172.21.15.204:0/3655382952 >> 172.21.15.204:0/3655382952 conn(0x7fb8d806e860 msgr2=0x7fb8d80a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:34.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 1 -- 172.21.15.204:0/3655382952 shutdown_connections 2024-09-15T01:33:34.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:34.968+0000 7fb8dd289640 1 -- 172.21.15.204:0/3655382952 wait complete. 2024-09-15T01:33:34.978 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule create-simple bar default host 2024-09-15T01:33:35.054 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.052+0000 7f7ea2aaf640 10 monclient: get_monmap_and_config 2024-09-15T01:33:35.054 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.052+0000 7f7ea2aaf640 10 monclient: build_initial_monmap 2024-09-15T01:33:35.054 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.052+0000 7f7ea2aaf640 10 monclient: monmap: 2024-09-15T01:33:35.054 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:35.054 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:35.054 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:35.053313+0000 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:35.053313+0000 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:35.055 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:35.056 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:35.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.053+0000 7f7ea2aaf640 1 Processor -- start 2024-09-15T01:33:35.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.053+0000 7f7ea2aaf640 1 -- start start 2024-09-15T01:33:35.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.053+0000 7f7ea2aaf640 10 monclient: init 2024-09-15T01:33:35.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:35.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 10 monclient: _add_conns ranks=[4,5,6,8,10,2,9,7,0,1,3] 2024-09-15T01:33:35.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7e9c0a2cd0 0x7f7e9c132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 10 monclient(hunting): picked mon.noname-c con 0x7f7e9c0a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:35.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e9c09fa20 0x7f7e9c09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 10 monclient(hunting): picked mon.noname-h con 0x7f7e9c09fa20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:35.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a08e0 0x7f7e9c0a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.064 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 10 monclient(hunting): picked mon.noname-d con 0x7f7e9c0a08e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:35.064 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:35.064 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0a30a0 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.064 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:35.064 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0a0330 con 0x7f7e9c09fa20 2024-09-15T01:33:35.064 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:35.064 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0a1db0 con 0x7f7e9c0a08e0 2024-09-15T01:33:35.064 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:35.064 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea2aaf640 10 monclient(hunting): authenticate will time out at 830.696098s 2024-09-15T01:33:35.064 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e9c09fa20 0x7f7e9c09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:35.064 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea104c640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a08e0 msgr2=0x7f7e9c0a0d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e9c09fa20 0x7f7e9c09fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:52964/0 (socket says 172.21.15.204:52964) 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 1 -- 172.21.15.204:0/2385382025 learned_addr learned my addr 172.21.15.204:0/2385382025 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 10 monclient(hunting): get_auth_request con 0x7f7e9c09fa20 auth_method 0 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7ea104c640 1 --2- 172.21.15.204:0/2385382025 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a08e0 0x7f7e9c0a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 10 monclient(hunting): handle_auth_done global_id 4802 payload 274 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 1 -- 172.21.15.204:0/2385382025 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a08e0 msgr2=0x7f7e9c0a0d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:35.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 1 --2- 172.21.15.204:0/2385382025 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a08e0 0x7f7e9c0a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 1 -- 172.21.15.204:0/2385382025 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7e9c0a2cd0 msgr2=0x7f7e9c132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 1 --2- 172.21.15.204:0/2385382025 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7e9c0a2cd0 0x7f7e9c132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 1 monclient: found mon.noname-h 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.054+0000 7f7e93fff640 1 -- 172.21.15.204:0/2385382025 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c0648a0 con 0x7f7e9c09fa20 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e93fff640 1 --2- 172.21.15.204:0/2385382025 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e9c09fa20 0x7f7e9c09fdf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009940 tx=0x7f7e8c0091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=57572e055bc17e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e937fe640 1 -- 172.21.15.204:0/2385382025 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e8c02ac80 con 0x7f7e9c09fa20 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e937fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e937fe640 10 monclient: dump: 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:35.066 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:35.067 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:35.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e937fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:35.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:35.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e937fe640 1 -- 172.21.15.204:0/2385382025 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c02ae00 con 0x7f7e9c09fa20 2024-09-15T01:33:35.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 5 monclient: authenticate success, global_id 4802 2024-09-15T01:33:35.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e937fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:35.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:35.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e937fe640 1 -- 172.21.15.204:0/2385382025 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e8c031ca0 con 0x7f7e9c09fa20 2024-09-15T01:33:35.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:35.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 4 set_mon_vals no callback set 2024-09-15T01:33:35.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e937fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:35.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7e937fe640 10 monclient: dump: 2024-09-15T01:33:35.068 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:35.069 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 10 monclient: shutdown 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/2385382025 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e9c09fa20 msgr2=0x7f7e9c09fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 1 --2- 172.21.15.204:0/2385382025 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e9c09fa20 0x7f7e9c09fdf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009940 tx=0x7f7e8c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/2385382025 shutdown_connections 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 1 --2- 172.21.15.204:0/2385382025 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a08e0 0x7f7e9c0a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 1 --2- 172.21.15.204:0/2385382025 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7e9c09fa20 0x7f7e9c09fdf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 1 --2- 172.21.15.204:0/2385382025 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7e9c0a2cd0 0x7f7e9c132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/2385382025 >> 172.21.15.204:0/2385382025 conn(0x7f7e9c06e860 msgr2=0x7f7e9c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/2385382025 shutdown_connections 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.055+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/2385382025 wait complete. 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.056+0000 7f7ea2aaf640 10 monclient: build_initial_monmap 2024-09-15T01:33:35.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.056+0000 7f7ea2aaf640 10 monclient: monmap: 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:35.071 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.056+0000 7f7ea2aaf640 1 Processor -- start 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.056+0000 7f7ea2aaf640 1 -- start start 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.056+0000 7f7ea2aaf640 10 monclient: init 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 10 monclient: _add_conns ranks=[6,3,0,8,7,4,5,1,2] 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c09fa20 0x7f7e9c141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 10 monclient(hunting): picked mon.noname-d con 0x7f7e9c09fa20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a08e0 0x7f7e9c12fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 10 monclient(hunting): picked mon.noname-g con 0x7f7e9c0a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c0a2cd0 0x7f7e9c099d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 10 monclient(hunting): picked mon.noname-a con 0x7f7e9c0a2cd0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c11d3e0 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c09d3d0 con 0x7f7e9c0a08e0 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c09b8e0 con 0x7f7e9c09fa20 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea2aaf640 10 monclient(hunting): authenticate will time out at 830.699279s 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7e93fff640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a08e0 msgr2=0x7f7e9c12fc70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a08e0 0x7f7e9c12fc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea084b640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c09fa20 msgr2=0x7f7e9c141f90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea084b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c09fa20 0x7f7e9c141f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea104c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c0a2cd0 0x7f7e9c099d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:35.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea104c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c0a2cd0 0x7f7e9c099d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:45620/0 (socket says 172.21.15.204:45620) 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea104c640 1 -- 172.21.15.204:0/101264941 learned_addr learned my addr 172.21.15.204:0/101264941 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea104c640 10 monclient(hunting): get_auth_request con 0x7f7e9c0a2cd0 auth_method 0 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea104c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea104c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea104c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea104c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea104c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.057+0000 7f7ea104c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea104c640 10 monclient(hunting): handle_auth_done global_id 4860 payload 824 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea104c640 1 -- 172.21.15.204:0/101264941 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c09fa20 msgr2=0x7f7e9c141f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea104c640 1 --2- 172.21.15.204:0/101264941 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c09fa20 0x7f7e9c141f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea104c640 1 -- 172.21.15.204:0/101264941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a08e0 msgr2=0x7f7e9c12fc70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea104c640 1 --2- 172.21.15.204:0/101264941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a08e0 0x7f7e9c12fc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea104c640 10 monclient: _finish_hunting 0 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea104c640 1 monclient: found mon.noname-a 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea104c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea104c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea104c640 1 -- 172.21.15.204:0/101264941 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c009450 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea104c640 1 --2- 172.21.15.204:0/101264941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c0a2cd0 0x7f7e9c099d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400bf80 tx=0x7f7e94009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e90ff9640 1 -- 172.21.15.204:0/101264941 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e9400cdf0 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e90ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e90ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e90ff9640 10 monclient: dump: 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.075 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e90ff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:35.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e90ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea2aaf640 5 monclient: authenticate success, global_id 4860 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea2aaf640 10 monclient: _renew_subs 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e90ff9640 1 -- 172.21.15.204:0/101264941 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e940065d0 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea2aaf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/101264941 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c11cd90 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e90ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e90ff9640 1 -- 172.21.15.204:0/101264941 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e940153e0 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e90ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e90ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e90ff9640 10 monclient: dump: 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.077 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea2aaf640 10 monclient: _renew_subs 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea2aaf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:35.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/101264941 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c09c4b0 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.058+0000 7f7e92ffd640 4 set_mon_vals no callback set 2024-09-15T01:33:35.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.059+0000 7f7ea2aaf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:35.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.059+0000 7f7ea2aaf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:35.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.059+0000 7f7ea2aaf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:35.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.059+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/101264941 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c070b80 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.059+0000 7f7e90ff9640 1 -- 172.21.15.204:0/101264941 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7e94012020 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.059+0000 7f7e90ff9640 1 --2- 172.21.15.204:0/101264941 >> v2:172.21.15.204:6812/28210 conn(0x7f7e880397b0 0x7f7e8803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.059+0000 7f7e90ff9640 1 -- 172.21.15.204:0/101264941 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 3653+0+0 (secure 0 0 0) 0x7f7e9404b390 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.059+0000 7f7ea084b640 1 --2- 172.21.15.204:0/101264941 >> v2:172.21.15.204:6812/28210 conn(0x7f7e880397b0 0x7f7e8803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:35.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.059+0000 7f7ea084b640 10 monclient: get_auth_request con 0x7f7e880397b0 auth_method 0 2024-09-15T01:33:35.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.060+0000 7f7ea084b640 1 --2- 172.21.15.204:0/101264941 >> v2:172.21.15.204:6812/28210 conn(0x7f7e880397b0 0x7f7e8803bc60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e980025d0 tx=0x7f7e98002a60 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:35.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.064+0000 7f7e90ff9640 1 -- 172.21.15.204:0/101264941 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7e94006780 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.064+0000 7f7e90ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:35.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.064+0000 7f7e90ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:35.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.221+0000 7f7ea2aaf640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-09-15T01:33:35.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.222+0000 7f7ea2aaf640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-09-15T01:33:35.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.222+0000 7f7ea2aaf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:35.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.222+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/101264941 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"} v 0) v1 -- 0x7f7e9c0a3ec0 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.523 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.521+0000 7f7e90ff9640 1 -- 172.21.15.204:0/101264941 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}]=0 v46) v1 ==== 126+0+0 (secure 0 0 0) 0x7f7e9401b030 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.523 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.521+0000 7f7e90ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-09-15T01:33:35.523 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.521+0000 7f7e90ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:35.621 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.620+0000 7f7ea2aaf640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-09-15T01:33:35.621 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.620+0000 7f7ea2aaf640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-09-15T01:33:35.622 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.620+0000 7f7ea2aaf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:35.622 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.620+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/101264941 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"} v 0) v1 -- 0x7f7e9c0008d0 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.622 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.621+0000 7f7e90ff9640 1 -- 172.21.15.204:0/101264941 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}]=0 rule bar already exists v46) v1 ==== 149+0+0 (secure 0 0 0) 0x7f7e9401b020 con 0x7f7e9c0a2cd0 2024-09-15T01:33:35.622 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.621+0000 7f7e90ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-09-15T01:33:35.622 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.621+0000 7f7e90ff9640 10 monclient: _finish_command 3 = system:0 rule bar already exists 2024-09-15T01:33:35.623 INFO:tasks.workunit.client.0.smithi204.stderr:rule bar already exists 2024-09-15T01:33:35.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/101264941 >> v2:172.21.15.204:6812/28210 conn(0x7f7e880397b0 msgr2=0x7f7e8803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:35.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 1 --2- 172.21.15.204:0/101264941 >> v2:172.21.15.204:6812/28210 conn(0x7f7e880397b0 0x7f7e8803bc60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e980025d0 tx=0x7f7e98002a60 comp rx=0 tx=0).stop 2024-09-15T01:33:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 10 monclient: shutdown 2024-09-15T01:33:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/101264941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c0a2cd0 msgr2=0x7f7e9c099d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 1 --2- 172.21.15.204:0/101264941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c0a2cd0 0x7f7e9c099d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400bf80 tx=0x7f7e94009730 comp rx=0 tx=0).stop 2024-09-15T01:33:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/101264941 shutdown_connections 2024-09-15T01:33:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 1 --2- 172.21.15.204:0/101264941 >> v2:172.21.15.204:6812/28210 conn(0x7f7e880397b0 0x7f7e8803bc60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 1 --2- 172.21.15.204:0/101264941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e9c0a2cd0 0x7f7e9c099d90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 1 --2- 172.21.15.204:0/101264941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a08e0 0x7f7e9c12fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 1 --2- 172.21.15.204:0/101264941 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c09fa20 0x7f7e9c141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/101264941 >> 172.21.15.204:0/101264941 conn(0x7f7e9c06e860 msgr2=0x7f7e9c0a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/101264941 shutdown_connections 2024-09-15T01:33:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.623+0000 7f7ea2aaf640 1 -- 172.21.15.204:0/101264941 wait complete. 2024-09-15T01:33:35.633 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm-device-class all 2024-09-15T01:33:35.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.707+0000 7fadf0f5d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:35.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.707+0000 7fadf0f5d640 10 monclient: build_initial_monmap 2024-09-15T01:33:35.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.707+0000 7fadf0f5d640 10 monclient: monmap: 2024-09-15T01:33:35.709 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:35.709 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:35.709 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:35.707702+0000 2024-09-15T01:33:35.709 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:35.707702+0000 2024-09-15T01:33:35.709 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:35.709 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:35.709 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:35.709 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:35.710 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:35.710 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:35.710 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:35.710 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:35.710 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:35.710 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:35.710 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:35.710 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:35.710 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:35.710 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:35.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 1 Processor -- start 2024-09-15T01:33:35.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 1 -- start start 2024-09-15T01:33:35.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 10 monclient: init 2024-09-15T01:33:35.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:35.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 10 monclient: _add_conns ranks=[8,2,3,6,5,10,1,9,7,4,0] 2024-09-15T01:33:35.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fadec09ac00 0x7fadec09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 10 monclient(hunting): picked mon.noname-e con 0x7fadec09ac00 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:35.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadec09afd0 0x7fadec099a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 10 monclient(hunting): picked mon.noname-b con 0x7fadec09afd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:35.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadec09d7b0 0x7fadec09dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 10 monclient(hunting): picked mon.noname-g con 0x7fadec09d7b0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec099f50 con 0x7fadec09afd0 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec09b770 con 0x7fadec09d7b0 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fadec09b490 con 0x7fadec09ac00 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadf0f5d640 10 monclient(hunting): authenticate will time out at 831.350540s 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadead9d640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadec09d7b0 msgr2=0x7fadec09dba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadead9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadec09d7b0 0x7fadec09dba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadea59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fadec09ac00 0x7fadec09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:35.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fade3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadec09afd0 0x7fadec099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadea59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fadec09ac00 0x7fadec09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:48044/0 (socket says 172.21.15.204:48044) 2024-09-15T01:33:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadea59c640 1 -- 172.21.15.204:0/3522339065 learned_addr learned my addr 172.21.15.204:0/3522339065 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fade3fff640 10 monclient(hunting): get_auth_request con 0x7fadec09afd0 auth_method 0 2024-09-15T01:33:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fade3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fade3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fade3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadea59c640 10 monclient(hunting): get_auth_request con 0x7fadec09ac00 auth_method 0 2024-09-15T01:33:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.708+0000 7fadea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fade3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fade3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fade3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 10 monclient(hunting): handle_auth_done global_id 4805 payload 274 2024-09-15T01:33:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 1 -- 172.21.15.204:0/3522339065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadec09d7b0 msgr2=0x7fadec09dba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 1 --2- 172.21.15.204:0/3522339065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadec09d7b0 0x7fadec09dba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 1 -- 172.21.15.204:0/3522339065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadec09afd0 msgr2=0x7fadec099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 1 --2- 172.21.15.204:0/3522339065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadec09afd0 0x7fadec099a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 10 monclient: _finish_hunting 0 2024-09-15T01:33:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 1 monclient: found mon.noname-e 2024-09-15T01:33:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:33:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fadea59c640 1 -- 172.21.15.204:0/3522339065 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec04b720 con 0x7fadec09ac00 2024-09-15T01:33:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.709+0000 7fade3fff640 1 --2- 172.21.15.204:0/3522339065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadec09afd0 0x7fadec099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadea59c640 1 --2- 172.21.15.204:0/3522339065 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fadec09ac00 0x7fadec09c150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faddc009970 tx=0x7faddc0091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1e20f5e4935ba3c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fade9d9b640 1 -- 172.21.15.204:0/3522339065 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faddc02ac70 con 0x7fadec09ac00 2024-09-15T01:33:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fade9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fade9d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:33:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fade9d9b640 10 monclient: dump: 2024-09-15T01:33:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:35.725 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:35.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fade9d9b640 10 monclient: _finish_auth 0 2024-09-15T01:33:35.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fade9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:35.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fade9d9b640 1 -- 172.21.15.204:0/3522339065 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc02adf0 con 0x7fadec09ac00 2024-09-15T01:33:35.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 5 monclient: authenticate success, global_id 4805 2024-09-15T01:33:35.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:35.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fade9d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:35.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:35.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fade9d9b640 1 -- 172.21.15.204:0/3522339065 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faddc031ca0 con 0x7fadec09ac00 2024-09-15T01:33:35.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fade9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:35.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 4 set_mon_vals no callback set 2024-09-15T01:33:35.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fade9d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:33:35.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fade9d9b640 10 monclient: dump: 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:35.727 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 10 monclient: shutdown 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 1 -- 172.21.15.204:0/3522339065 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fadec09ac00 msgr2=0x7fadec09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 1 --2- 172.21.15.204:0/3522339065 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fadec09ac00 0x7fadec09c150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faddc009970 tx=0x7faddc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 1 -- 172.21.15.204:0/3522339065 shutdown_connections 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 1 --2- 172.21.15.204:0/3522339065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadec09d7b0 0x7fadec09dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 1 --2- 172.21.15.204:0/3522339065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadec09afd0 0x7fadec099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 1 --2- 172.21.15.204:0/3522339065 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fadec09ac00 0x7fadec09c150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 1 -- 172.21.15.204:0/3522339065 >> 172.21.15.204:0/3522339065 conn(0x7fadec068ac0 msgr2=0x7fadec0a2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:35.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 1 -- 172.21.15.204:0/3522339065 shutdown_connections 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.710+0000 7fadf0f5d640 1 -- 172.21.15.204:0/3522339065 wait complete. 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.711+0000 7fadf0f5d640 10 monclient: build_initial_monmap 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.711+0000 7fadf0f5d640 10 monclient: monmap: 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:35.729 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.711+0000 7fadf0f5d640 1 Processor -- start 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.711+0000 7fadf0f5d640 1 -- start start 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.711+0000 7fadf0f5d640 10 monclient: init 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 10 monclient: _add_conns ranks=[4,8,6,3,5,2,1,0,7] 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fadec09ac00 0x7fadec067130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 10 monclient(hunting): picked mon.noname-c con 0x7fadec09ac00 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:35.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fadec09afd0 0x7fadec132920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 10 monclient(hunting): picked mon.noname-e con 0x7fadec09afd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fadec09d7b0 0x7fadec141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 10 monclient(hunting): picked mon.noname-d con 0x7fadec09d7b0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fadec123030 con 0x7fadec09ac00 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fadec122d50 con 0x7fadec09d7b0 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fadec11d230 con 0x7fadec09afd0 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadf0f5d640 10 monclient(hunting): authenticate will time out at 831.354187s 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadead9d640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fadec09d7b0 msgr2=0x7fadec141a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadead9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fadec09d7b0 0x7fadec141a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:35.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fade3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fadec09afd0 0x7fadec132920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fadec09ac00 0x7fadec067130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fadec09ac00 0x7fadec067130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:50292/0 (socket says 172.21.15.204:50292) 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 1 -- 172.21.15.204:0/624298070 learned_addr learned my addr 172.21.15.204:0/624298070 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 10 monclient(hunting): get_auth_request con 0x7fadec09ac00 auth_method 0 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fade3fff640 10 monclient(hunting): get_auth_request con 0x7fadec09afd0 auth_method 0 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fade3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fade3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fade3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:35.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fade3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fade3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fade3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 10 monclient(hunting): handle_auth_done global_id 4720 payload 824 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 1 -- 172.21.15.204:0/624298070 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fadec09afd0 msgr2=0x7fadec132920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 1 --2- 172.21.15.204:0/624298070 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fadec09afd0 0x7fadec132920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 1 -- 172.21.15.204:0/624298070 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fadec09d7b0 msgr2=0x7fadec141a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 1 --2- 172.21.15.204:0/624298070 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fadec09d7b0 0x7fadec141a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 10 monclient: _finish_hunting 0 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 1 monclient: found mon.noname-c 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 1 -- 172.21.15.204:0/624298070 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd4009390 con 0x7fadec09ac00 2024-09-15T01:33:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.712+0000 7fadea59c640 1 --2- 172.21.15.204:0/624298070 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fadec09ac00 0x7fadec067130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faddc009420 tx=0x7faddc032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 1 -- 172.21.15.204:0/624298070 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faddc03b870 con 0x7fadec09ac00 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 10 monclient: dump: 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:35.735 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 1 -- 172.21.15.204:0/624298070 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc03b9d0 con 0x7fadec09ac00 2024-09-15T01:33:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fadf0f5d640 5 monclient: authenticate success, global_id 4720 2024-09-15T01:33:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fadf0f5d640 10 monclient: _renew_subs 2024-09-15T01:33:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fadf0f5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fadf0f5d640 1 -- 172.21.15.204:0/624298070 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faddc009450 con 0x7fadec09ac00 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 1 -- 172.21.15.204:0/624298070 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faddc0313a0 con 0x7fadec09ac00 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 10 monclient: dump: 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade959a640 4 set_mon_vals no callback set 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fadf0f5d640 10 monclient: _renew_subs 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fadf0f5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fadf0f5d640 1 -- 172.21.15.204:0/624298070 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec122a70 con 0x7fadec09ac00 2024-09-15T01:33:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fadf0f5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fadf0f5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fadf0f5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fade27fc640 1 -- 172.21.15.204:0/624298070 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7faddc039040 con 0x7fadec09ac00 2024-09-15T01:33:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.713+0000 7fadf0f5d640 1 -- 172.21.15.204:0/624298070 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec070cc0 con 0x7fadec09ac00 2024-09-15T01:33:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.714+0000 7fade27fc640 1 --2- 172.21.15.204:0/624298070 >> v2:172.21.15.204:6812/28210 conn(0x7fadcc0397e0 0x7fadcc03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.714+0000 7fade27fc640 1 -- 172.21.15.204:0/624298070 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 3712+0+0 (secure 0 0 0) 0x7faddc06a980 con 0x7fadec09ac00 2024-09-15T01:33:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.714+0000 7fade3fff640 1 --2- 172.21.15.204:0/624298070 >> v2:172.21.15.204:6812/28210 conn(0x7fadcc0397e0 0x7fadcc03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.714+0000 7fade3fff640 10 monclient: get_auth_request con 0x7fadcc0397e0 auth_method 0 2024-09-15T01:33:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.714+0000 7fade3fff640 1 --2- 172.21.15.204:0/624298070 >> v2:172.21.15.204:6812/28210 conn(0x7fadcc0397e0 0x7fadcc03bc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009eb0 tx=0x7fadd40090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.718+0000 7fade27fc640 1 -- 172.21.15.204:0/624298070 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7faddc031520 con 0x7fadec09ac00 2024-09-15T01:33:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.718+0000 7fade27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.718+0000 7fade27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:35.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.876+0000 7fadf0f5d640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-09-15T01:33:35.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.876+0000 7fadf0f5d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-09-15T01:33:35.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.876+0000 7fadf0f5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:35.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:35.876+0000 7fadf0f5d640 1 -- 172.21.15.204:0/624298070 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["all"]} v 0) v1 -- 0x7fadec069880 con 0x7fadec09ac00 2024-09-15T01:33:36.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:36.461+0000 7fade27fc640 1 -- 172.21.15.204:0/624298070 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["all"]}]=0 done removing class of osd(s): 0,1,2,3,4,5 v47) v1 ==== 131+0+0 (secure 0 0 0) 0x7faddc0420a0 con 0x7fadec09ac00 2024-09-15T01:33:36.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:36.461+0000 7fade27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-09-15T01:33:36.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:36.461+0000 7fade27fc640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0,1,2,3,4,5 2024-09-15T01:33:36.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:36.564+0000 7fadf0f5d640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-09-15T01:33:36.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:36.564+0000 7fadf0f5d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-09-15T01:33:36.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:36.564+0000 7fadf0f5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:36.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:36.564+0000 7fadf0f5d640 1 -- 172.21.15.204:0/624298070 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["all"]} v 0) v1 -- 0x7fadec132e60 con 0x7fadec09ac00 2024-09-15T01:33:37.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.533+0000 7fade27fc640 1 -- 172.21.15.204:0/624298070 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["all"]}]=0 osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, osd.3 belongs to no class, osd.4 belongs to no class, osd.5 belongs to no class, done removing class of osd(s): v48) v1 ==== 282+0+0 (secure 0 0 0) 0x7faddc037030 con 0x7fadec09ac00 2024-09-15T01:33:37.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.533+0000 7fade27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-09-15T01:33:37.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.533+0000 7fade27fc640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, osd.3 belongs to no class, osd.4 belongs to no class, osd.5 belongs to no class, done removing class of osd(s): 2024-09-15T01:33:37.535 INFO:tasks.workunit.client.0.smithi204.stderr:osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, osd.3 belongs to no class, osd.4 belongs to no class, osd.5 belongs to no class, done removing class of osd(s): 2024-09-15T01:33:37.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 1 -- 172.21.15.204:0/624298070 >> v2:172.21.15.204:6812/28210 conn(0x7fadcc0397e0 msgr2=0x7fadcc03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:37.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 1 --2- 172.21.15.204:0/624298070 >> v2:172.21.15.204:6812/28210 conn(0x7fadcc0397e0 0x7fadcc03bc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009eb0 tx=0x7fadd40090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 10 monclient: shutdown 2024-09-15T01:33:37.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:37.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 1 -- 172.21.15.204:0/624298070 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fadec09ac00 msgr2=0x7fadec067130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:37.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 1 --2- 172.21.15.204:0/624298070 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fadec09ac00 0x7fadec067130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faddc009420 tx=0x7faddc032000 comp rx=0 tx=0).stop 2024-09-15T01:33:37.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 1 -- 172.21.15.204:0/624298070 shutdown_connections 2024-09-15T01:33:37.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 1 --2- 172.21.15.204:0/624298070 >> v2:172.21.15.204:6812/28210 conn(0x7fadcc0397e0 0x7fadcc03bc90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 1 --2- 172.21.15.204:0/624298070 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fadec09d7b0 0x7fadec141a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 1 --2- 172.21.15.204:0/624298070 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fadec09afd0 0x7fadec132920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 1 --2- 172.21.15.204:0/624298070 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fadec09ac00 0x7fadec067130 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 1 -- 172.21.15.204:0/624298070 >> 172.21.15.204:0/624298070 conn(0x7fadec068ac0 msgr2=0x7fadec09e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:37.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 1 -- 172.21.15.204:0/624298070 shutdown_connections 2024-09-15T01:33:37.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.537+0000 7fadf0f5d640 1 -- 172.21.15.204:0/624298070 wait complete. 2024-09-15T01:33:37.549 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush set-device-class ssd osd.0 2024-09-15T01:33:37.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.628+0000 7f53bda34640 10 monclient: get_monmap_and_config 2024-09-15T01:33:37.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.628+0000 7f53bda34640 10 monclient: build_initial_monmap 2024-09-15T01:33:37.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.628+0000 7f53bda34640 10 monclient: monmap: 2024-09-15T01:33:37.630 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:37.630 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:37.630 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:37.629149+0000 2024-09-15T01:33:37.630 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:37.629149+0000 2024-09-15T01:33:37.630 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:37.630 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:37.630 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:37.630 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:37.631 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:37.631 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:37.631 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:37.631 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:37.631 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:37.631 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:37.631 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:37.631 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:37.631 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:37.631 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:37.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 1 Processor -- start 2024-09-15T01:33:37.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 1 -- start start 2024-09-15T01:33:37.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 10 monclient: init 2024-09-15T01:33:37.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:37.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 10 monclient: _add_conns ranks=[7,6,10,8,4,3,5,0,9,1,2] 2024-09-15T01:33:37.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53b80a2e30 0x7f53b8132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:37.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 10 monclient(hunting): picked mon.noname-i con 0x7f53b80a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:37.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f53b809fb80 0x7f53b809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:37.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 10 monclient(hunting): picked mon.noname-d con 0x7f53b809fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:37.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f53b80a0a40 0x7f53b80a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:37.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 10 monclient(hunting): picked mon.noname-f con 0x7f53b80a0a40 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:37.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:37.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.629+0000 7f53bda34640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b809e6c0 con 0x7f53b80a0a40 2024-09-15T01:33:37.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53bda34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:37.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53bda34640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80a04b0 con 0x7f53b809fb80 2024-09-15T01:33:37.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53bda34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:37.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53bda34640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80a3200 con 0x7f53b80a2e30 2024-09-15T01:33:37.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53bda34640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:37.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53bda34640 10 monclient(hunting): authenticate will time out at 833.272020s 2024-09-15T01:33:37.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53b80a2e30 0x7f53b8132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:37.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53affff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f53b809fb80 msgr2=0x7f53b809ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:37.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53affff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f53b809fb80 0x7f53b809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:37.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53b80a2e30 0x7f53b8132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:58798/0 (socket says 172.21.15.204:58798) 2024-09-15T01:33:37.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 1 -- 172.21.15.204:0/3549851451 learned_addr learned my addr 172.21.15.204:0/3549851451 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:37.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 10 monclient(hunting): get_auth_request con 0x7f53b80a2e30 auth_method 0 2024-09-15T01:33:37.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:37.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:37.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:37.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b77fe640 1 --2- 172.21.15.204:0/3549851451 >> v2:172.21.15.204:3300/0 conn(0x7f53b80a0a40 0x7f53b80a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b77fe640 10 monclient(hunting): get_auth_request con 0x7f53b80a0a40 auth_method 0 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 10 monclient(hunting): handle_auth_done global_id 4804 payload 274 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 1 -- 172.21.15.204:0/3549851451 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f53b809fb80 msgr2=0x7f53b809ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:37.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 1 --2- 172.21.15.204:0/3549851451 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f53b809fb80 0x7f53b809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 1 -- 172.21.15.204:0/3549851451 >> v2:172.21.15.204:3300/0 conn(0x7f53b80a0a40 msgr2=0x7f53b80a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 1 --2- 172.21.15.204:0/3549851451 >> v2:172.21.15.204:3300/0 conn(0x7f53b80a0a40 0x7f53b80a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 1 monclient: found mon.noname-i 2024-09-15T01:33:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.630+0000 7f53b6ffd640 1 -- 172.21.15.204:0/3549851451 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b804b720 con 0x7f53b80a2e30 2024-09-15T01:33:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b6ffd640 1 --2- 172.21.15.204:0/3549851451 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53b80a2e30 0x7f53b8132a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009910 tx=0x7f53a80091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=4adbb578916071b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b67fc640 1 -- 172.21.15.204:0/3549851451 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53a802acb0 con 0x7f53b80a2e30 2024-09-15T01:33:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b67fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b67fc640 10 monclient: dump: 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b67fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b67fc640 1 -- 172.21.15.204:0/3549851451 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a802ae30 con 0x7f53b80a2e30 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 5 monclient: authenticate success, global_id 4804 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b67fc640 1 -- 172.21.15.204:0/3549851451 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53a8031ca0 con 0x7f53b80a2e30 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 4 set_mon_vals no callback set 2024-09-15T01:33:37.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b67fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53b67fc640 10 monclient: dump: 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:37.646 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 10 monclient: shutdown 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 1 -- 172.21.15.204:0/3549851451 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53b80a2e30 msgr2=0x7f53b8132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 1 --2- 172.21.15.204:0/3549851451 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53b80a2e30 0x7f53b8132a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009910 tx=0x7f53a80091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 1 -- 172.21.15.204:0/3549851451 shutdown_connections 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 1 --2- 172.21.15.204:0/3549851451 >> v2:172.21.15.204:3300/0 conn(0x7f53b80a0a40 0x7f53b80a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 1 --2- 172.21.15.204:0/3549851451 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f53b809fb80 0x7f53b809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 1 --2- 172.21.15.204:0/3549851451 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53b80a2e30 0x7f53b8132a60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 1 -- 172.21.15.204:0/3549851451 >> 172.21.15.204:0/3549851451 conn(0x7f53b806e860 msgr2=0x7f53b806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:37.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.631+0000 7f53bda34640 1 -- 172.21.15.204:0/3549851451 shutdown_connections 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.632+0000 7f53bda34640 1 -- 172.21.15.204:0/3549851451 wait complete. 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.632+0000 7f53bda34640 10 monclient: build_initial_monmap 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.632+0000 7f53bda34640 10 monclient: monmap: 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:37.648 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.632+0000 7f53bda34640 1 Processor -- start 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.632+0000 7f53bda34640 1 -- start start 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.632+0000 7f53bda34640 10 monclient: init 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 10 monclient: _add_conns ranks=[1,3,6,5,0,2,4,8,7] 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b809fb80 0x7f53b81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:37.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 10 monclient(hunting): picked mon.noname-f con 0x7f53b809fb80 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53b80a0a40 0x7f53b811e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 10 monclient(hunting): picked mon.noname-g con 0x7f53b80a0a40 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f53b80a2e30 0x7f53b811e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 10 monclient(hunting): picked mon.noname-d con 0x7f53b80a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b81429c0 con 0x7f53b809fb80 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b809ca40 con 0x7f53b80a0a40 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f53b809d650 con 0x7f53b80a2e30 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53bda34640 10 monclient(hunting): authenticate will time out at 833.275376s 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53affff640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53b80a0a40 msgr2=0x7f53b811e380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:37.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53affff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53b80a0a40 0x7f53b811e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:37.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b809fb80 0x7f53b81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:37.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b809fb80 0x7f53b81317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55270/0 (socket says 172.21.15.204:55270) 2024-09-15T01:33:37.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 1 -- 172.21.15.204:0/237394105 learned_addr learned my addr 172.21.15.204:0/237394105 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:37.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 10 monclient(hunting): get_auth_request con 0x7f53b809fb80 auth_method 0 2024-09-15T01:33:37.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:37.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:37.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:37.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b77fe640 1 -- 172.21.15.204:0/237394105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f53b80a2e30 msgr2=0x7f53b811e8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:37.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:37.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:37.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b77fe640 1 --2- 172.21.15.204:0/237394105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f53b80a2e30 0x7f53b811e8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:37.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:37.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 10 monclient(hunting): handle_auth_done global_id 5068 payload 824 2024-09-15T01:33:37.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 1 -- 172.21.15.204:0/237394105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f53b80a2e30 msgr2=0x7f53b811e8c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:37.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 1 --2- 172.21.15.204:0/237394105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f53b80a2e30 0x7f53b811e8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 1 -- 172.21.15.204:0/237394105 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53b80a0a40 msgr2=0x7f53b811e380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:37.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 1 --2- 172.21.15.204:0/237394105 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53b80a0a40 0x7f53b811e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:37.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:37.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 1 monclient: found mon.noname-f 2024-09-15T01:33:37.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:37.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:37.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 1 -- 172.21.15.204:0/237394105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a8009450 con 0x7f53b809fb80 2024-09-15T01:33:37.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53b6ffd640 1 --2- 172.21.15.204:0/237394105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b809fb80 0x7f53b81317f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f53a8035040 tx=0x7f53a8032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:37.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53aeffd640 1 -- 172.21.15.204:0/237394105 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53a8031820 con 0x7f53b809fb80 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.633+0000 7f53aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53aeffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53aeffd640 10 monclient: dump: 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53aeffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53aeffd640 1 -- 172.21.15.204:0/237394105 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a80319a0 con 0x7f53b809fb80 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53bda34640 5 monclient: authenticate success, global_id 5068 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53bda34640 10 monclient: _renew_subs 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53bda34640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53bda34640 1 -- 172.21.15.204:0/237394105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b8120640 con 0x7f53b809fb80 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53aeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:37.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53aeffd640 1 -- 172.21.15.204:0/237394105 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53a803a390 con 0x7f53b809fb80 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53aeffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53aeffd640 10 monclient: dump: 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:37.656 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53b5ffb640 4 set_mon_vals no callback set 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53bda34640 10 monclient: _renew_subs 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53bda34640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53bda34640 1 -- 172.21.15.204:0/237394105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b8121940 con 0x7f53b809fb80 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.634+0000 7f53aeffd640 1 -- 172.21.15.204:0/237394105 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f53a8039040 con 0x7f53b809fb80 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.635+0000 7f53bda34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:37.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.635+0000 7f53bda34640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:37.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.635+0000 7f53bda34640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:37.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.635+0000 7f53bda34640 1 -- 172.21.15.204:0/237394105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b80668b0 con 0x7f53b809fb80 2024-09-15T01:33:37.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.635+0000 7f53aeffd640 1 --2- 172.21.15.204:0/237394105 >> v2:172.21.15.204:6812/28210 conn(0x7f53980397b0 0x7f539803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:37.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.635+0000 7f53aeffd640 1 -- 172.21.15.204:0/237394105 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 3408+0+0 (secure 0 0 0) 0x7f53a806a880 con 0x7f53b809fb80 2024-09-15T01:33:37.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.635+0000 7f53affff640 1 --2- 172.21.15.204:0/237394105 >> v2:172.21.15.204:6812/28210 conn(0x7f53980397b0 0x7f539803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:37.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.635+0000 7f53affff640 10 monclient: get_auth_request con 0x7f53980397b0 auth_method 0 2024-09-15T01:33:37.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.635+0000 7f53affff640 1 --2- 172.21.15.204:0/237394105 >> v2:172.21.15.204:6812/28210 conn(0x7f53980397b0 0x7f539803bc60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f53b80a2e00 tx=0x7f53a0006d00 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:37.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.639+0000 7f53aeffd640 1 -- 172.21.15.204:0/237394105 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f53a8031e40 con 0x7f53b809fb80 2024-09-15T01:33:37.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.639+0000 7f53aeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:37.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.639+0000 7f53aeffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:37.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.801+0000 7f53bda34640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-09-15T01:33:37.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.802+0000 7f53bda34640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-09-15T01:33:37.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.802+0000 7f53bda34640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:37.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:37.802+0000 7f53bda34640 1 -- 172.21.15.204:0/237394105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]} v 0) v1 -- 0x7f53b8070cc0 con 0x7f53b809fb80 2024-09-15T01:33:38.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:38.542+0000 7f53aeffd640 1 -- 172.21.15.204:0/237394105 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}]=0 set osd(s) 0 to class 'ssd' v49) v1 ==== 135+0+0 (secure 0 0 0) 0x7f53a8037070 con 0x7f53b809fb80 2024-09-15T01:33:38.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:38.542+0000 7f53aeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-09-15T01:33:38.544 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:38.542+0000 7f53aeffd640 10 monclient: _finish_command 2 = system:0 set osd(s) 0 to class 'ssd' 2024-09-15T01:33:38.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:38.641+0000 7f53bda34640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-09-15T01:33:38.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:38.641+0000 7f53bda34640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-09-15T01:33:38.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:38.641+0000 7f53bda34640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:38.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:38.641+0000 7f53bda34640 1 -- 172.21.15.204:0/237394105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]} v 0) v1 -- 0x7f53b8070cc0 con 0x7f53b809fb80 2024-09-15T01:33:39.544 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.542+0000 7f53aeffd640 1 -- 172.21.15.204:0/237394105 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}]=0 osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' v50) v1 ==== 237+0+0 (secure 0 0 0) 0x7f53a802f710 con 0x7f53b809fb80 2024-09-15T01:33:39.544 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.543+0000 7f53aeffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-09-15T01:33:39.545 INFO:tasks.workunit.client.0.smithi204.stderr:osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-09-15T01:33:39.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.543+0000 7f53aeffd640 10 monclient: _finish_command 3 = system:0 osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-09-15T01:33:39.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 1 -- 172.21.15.204:0/237394105 >> v2:172.21.15.204:6812/28210 conn(0x7f53980397b0 msgr2=0x7f539803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:39.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 1 --2- 172.21.15.204:0/237394105 >> v2:172.21.15.204:6812/28210 conn(0x7f53980397b0 0x7f539803bc60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f53b80a2e00 tx=0x7f53a0006d00 comp rx=0 tx=0).stop 2024-09-15T01:33:39.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 10 monclient: shutdown 2024-09-15T01:33:39.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:39.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 1 -- 172.21.15.204:0/237394105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b809fb80 msgr2=0x7f53b81317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:39.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 1 --2- 172.21.15.204:0/237394105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b809fb80 0x7f53b81317f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f53a8035040 tx=0x7f53a8032000 comp rx=0 tx=0).stop 2024-09-15T01:33:39.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 1 -- 172.21.15.204:0/237394105 shutdown_connections 2024-09-15T01:33:39.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 1 --2- 172.21.15.204:0/237394105 >> v2:172.21.15.204:6812/28210 conn(0x7f53980397b0 0x7f539803bc60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 1 --2- 172.21.15.204:0/237394105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f53b80a2e30 0x7f53b811e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 1 --2- 172.21.15.204:0/237394105 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53b80a0a40 0x7f53b811e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 1 --2- 172.21.15.204:0/237394105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b809fb80 0x7f53b81317f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 1 -- 172.21.15.204:0/237394105 >> 172.21.15.204:0/237394105 conn(0x7f53b806e860 msgr2=0x7f53b809bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:39.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 1 -- 172.21.15.204:0/237394105 shutdown_connections 2024-09-15T01:33:39.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.545+0000 7f53bda34640 1 -- 172.21.15.204:0/237394105 wait complete. 2024-09-15T01:33:39.559 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush set-device-class hdd osd.1 2024-09-15T01:33:39.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.636+0000 7f4a2cb5d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:39.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.636+0000 7f4a2cb5d640 10 monclient: build_initial_monmap 2024-09-15T01:33:39.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.636+0000 7f4a2cb5d640 10 monclient: monmap: 2024-09-15T01:33:39.638 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:39.638 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:39.638 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:39.636950+0000 2024-09-15T01:33:39.638 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:39.636950+0000 2024-09-15T01:33:39.638 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:39.639 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:39.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 1 Processor -- start 2024-09-15T01:33:39.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 1 -- start start 2024-09-15T01:33:39.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 10 monclient: init 2024-09-15T01:33:39.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:39.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 10 monclient: _add_conns ranks=[4,5,3,9,8,10,2,6,0,7,1] 2024-09-15T01:33:39.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a280a2e30 0x7f4a28132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:39.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 10 monclient(hunting): picked mon.noname-c con 0x7f4a280a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:39.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a2809fb80 0x7f4a2809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:39.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 10 monclient(hunting): picked mon.noname-h con 0x7f4a2809fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:39.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a280a0a40 0x7f4a280a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:39.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 10 monclient(hunting): picked mon.noname-g con 0x7f4a280a0a40 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:39.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2809e6c0 con 0x7f4a280a0a40 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280a04b0 con 0x7f4a280a2e30 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280a3200 con 0x7f4a2809fb80 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a2cb5d640 10 monclient(hunting): authenticate will time out at 835.279807s 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a26d9d640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a280a0a40 msgr2=0x7f4a280a0e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a26d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a280a0a40 0x7f4a280a0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a25d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a2809fb80 0x7f4a2809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a25d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a2809fb80 0x7f4a2809ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:34778/0 (socket says 172.21.15.204:34778) 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a25d9b640 1 -- 172.21.15.204:0/1428234335 learned_addr learned my addr 172.21.15.204:0/1428234335 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a25d9b640 10 monclient(hunting): get_auth_request con 0x7f4a2809fb80 auth_method 0 2024-09-15T01:33:39.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a25d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a25d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.637+0000 7f4a25d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a2659c640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a280a2e30 0x7f4a28132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a2659c640 10 monclient(hunting): get_auth_request con 0x7f4a280a2e30 auth_method 0 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a2659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a2659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a2659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 10 monclient(hunting): handle_auth_done global_id 4811 payload 274 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 1 -- 172.21.15.204:0/1428234335 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a280a2e30 msgr2=0x7f4a28132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a280a2e30 0x7f4a28132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 1 -- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a280a0a40 msgr2=0x7f4a280a0e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:39.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a280a0a40 0x7f4a280a0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:33:39.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 1 monclient: found mon.noname-h 2024-09-15T01:33:39.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:39.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:39.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 1 -- 172.21.15.204:0/1428234335 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2804b720 con 0x7f4a2809fb80 2024-09-15T01:33:39.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 1 -- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a2809fb80 msgr2=0x7f4a2809ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:33:39.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 1 -- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a2809fb80 msgr2=0x7f4a2809ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:33:39.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a2809fb80 0x7f4a2809ff70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7f4a1c009940 tx=0x7f4a1c0091d0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:33:39.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.638+0000 7f4a25d9b640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a2809fb80 0x7f4a2809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:39.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a2809fb80 0x7f4a2809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:39.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 10 monclient: get_auth_request con 0x7f4a2809fb80 auth_method 0 2024-09-15T01:33:39.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 0 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a2809fb80 0x7f4a2809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T01:33:39.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a2809fb80 0x7f4a2809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 10 monclient: ms_handle_reset current mon [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 1 -- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a2809fb80 msgr2=0x7f4a2809ff70 unknown :-1 s=STATE_CLOSED l=0).mark_down 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a2809fb80 0x7f4a2809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 10 monclient: _add_conns ranks=[2,10,9,7,1,6,8,0] 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a100035f0 0x7f4a10005a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 10 monclient(hunting): picked mon.noname-b con 0x7f4a100035f0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 1 --2- 172.21.15.204:0/1428234335 >> v2:172.21.15.204:3300/0 conn(0x7f4a100062a0 0x7f4a10008720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 10 monclient(hunting): picked mon.noname-f con 0x7f4a100062a0 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 1 --2- 172.21.15.204:0/1428234335 >> v2:172.21.15.178:3300/0 conn(0x7f4a10009020 0x7f4a1000b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 10 monclient(hunting): picked mon.noname-a con 0x7f4a10009020 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 1 -- 172.21.15.204:0/1428234335 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1000bde0 con 0x7f4a10009020 2024-09-15T01:33:39.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 1 -- 172.21.15.204:0/1428234335 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1000bf20 con 0x7f4a100062a0 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 1 -- 172.21.15.204:0/1428234335 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1000c0e0 con 0x7f4a100035f0 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2559a640 10 monclient(hunting): ms_handle_reset stray mon [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 1 --2- 172.21.15.204:0/1428234335 >> v2:172.21.15.204:3300/0 conn(0x7f4a100062a0 0x7f4a10008720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 10 monclient(hunting): get_auth_request con 0x7f4a100062a0 auth_method 0 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:39.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 10 monclient(hunting): handle_auth_done global_id 4811 payload 293 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 1 -- 172.21.15.204:0/1428234335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a100035f0 msgr2=0x7f4a10005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a2659c640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a100035f0 0x7f4a10005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a100035f0 0x7f4a10005a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a26d9d640 1 --2- 172.21.15.204:0/1428234335 >> v2:172.21.15.178:3300/0 conn(0x7f4a10009020 0x7f4a1000b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 1 -- 172.21.15.204:0/1428234335 >> v2:172.21.15.178:3300/0 conn(0x7f4a10009020 msgr2=0x7f4a1000b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 1 --2- 172.21.15.204:0/1428234335 >> v2:172.21.15.178:3300/0 conn(0x7f4a10009020 0x7f4a1000b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 1 monclient: found mon.noname-f 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a25d9b640 1 -- 172.21.15.204:0/1428234335 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1000c360 con 0x7f4a100062a0 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a26d9d640 10 monclient: get_auth_request con 0x7f4a10009020 auth_method 0 2024-09-15T01:33:39.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a26d9d640 1 --2- 172.21.15.204:0/1428234335 >> v2:172.21.15.178:3300/0 conn(0x7f4a10009020 0x7f4a1000b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:33:39.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.839+0000 7f4a26d9d640 1 -- 172.21.15.204:0/1428234335 reap_dead start 2024-09-15T01:33:39.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a25d9b640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a100062a0 0x7f4a10008720 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009940 tx=0x7f4a1c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=337e41cb7e2adf71 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:39.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2559a640 1 -- 172.21.15.204:0/1428234335 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a1c038070 con 0x7f4a100062a0 2024-09-15T01:33:39.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:39.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2559a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:39.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2559a640 10 monclient: dump: 2024-09-15T01:33:39.852 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:39.852 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:39.852 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:39.852 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:39.852 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:39.852 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:39.853 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:39.853 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:39.853 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:39.853 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:39.853 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:39.856 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:39.856 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:39.856 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:39.856 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:39.856 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2559a640 10 monclient: _finish_auth 0 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2559a640 1 -- 172.21.15.204:0/1428234335 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c033c50 con 0x7f4a100062a0 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2cb5d640 5 monclient: authenticate success, global_id 4811 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2cb5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2559a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2559a640 1 -- 172.21.15.204:0/1428234335 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a1c033dc0 con 0x7f4a100062a0 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2cb5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2cb5d640 4 set_mon_vals no callback set 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2559a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2559a640 10 monclient: dump: 2024-09-15T01:33:39.857 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:39.858 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2cb5d640 10 monclient: shutdown 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2cb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a100062a0 msgr2=0x7f4a10008720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.840+0000 7f4a2cb5d640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a100062a0 0x7f4a10008720 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009940 tx=0x7f4a1c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.841+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1428234335 shutdown_connections 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.841+0000 7f4a2cb5d640 1 --2- 172.21.15.204:0/1428234335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a100062a0 0x7f4a10008720 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.841+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1428234335 >> 172.21.15.204:0/1428234335 conn(0x7f4a2806e860 msgr2=0x7f4a2806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.841+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1428234335 shutdown_connections 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.841+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1428234335 wait complete. 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.841+0000 7f4a2cb5d640 10 monclient: build_initial_monmap 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.841+0000 7f4a2cb5d640 10 monclient: monmap: 2024-09-15T01:33:39.859 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:39.860 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.841+0000 7f4a2cb5d640 1 Processor -- start 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 1 -- start start 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 10 monclient: init 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 10 monclient: _add_conns ranks=[2,6,4,5,1,3,8,0,7] 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a100062a0 0x7f4a28141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 10 monclient(hunting): picked mon.noname-b con 0x7f4a100062a0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a281327e0 0x7f4a28132bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 10 monclient(hunting): picked mon.noname-d con 0x7f4a281327e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a2811f380 0x7f4a2811f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 10 monclient(hunting): picked mon.noname-c con 0x7f4a2811f380 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:39.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a28133110 con 0x7f4a100062a0 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a28122ef0 con 0x7f4a2811f380 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4a28122c10 con 0x7f4a281327e0 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2cb5d640 10 monclient(hunting): authenticate will time out at 835.484548s 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a25d9b640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a281327e0 msgr2=0x7f4a28132bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a25d9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a281327e0 0x7f4a28132bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2659c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a100062a0 0x7f4a28141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a26d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a2811f380 0x7f4a2811f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2659c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a100062a0 0x7f4a28141da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:48304/0 (socket says 172.21.15.204:48304) 2024-09-15T01:33:39.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2659c640 1 -- 172.21.15.204:0/1519310871 learned_addr learned my addr 172.21.15.204:0/1519310871 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2659c640 10 monclient(hunting): get_auth_request con 0x7f4a100062a0 auth_method 0 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a26d9d640 10 monclient(hunting): get_auth_request con 0x7f4a2811f380 auth_method 0 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a26d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a26d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a26d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a2659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a26d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a26d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:39.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.842+0000 7f4a26d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:39.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a26d9d640 10 monclient(hunting): handle_auth_done global_id 4738 payload 824 2024-09-15T01:33:39.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a26d9d640 1 -- 172.21.15.204:0/1519310871 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a281327e0 msgr2=0x7f4a28132bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:39.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a26d9d640 1 --2- 172.21.15.204:0/1519310871 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a281327e0 0x7f4a28132bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a26d9d640 1 -- 172.21.15.204:0/1519310871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a100062a0 msgr2=0x7f4a28141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:39.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a26d9d640 1 --2- 172.21.15.204:0/1519310871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a100062a0 0x7f4a28141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:39.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a26d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:33:39.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a26d9d640 1 monclient: found mon.noname-c 2024-09-15T01:33:39.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a26d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:39.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a26d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:33:39.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a26d9d640 1 -- 172.21.15.204:0/1519310871 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a281219d0 con 0x7f4a2811f380 2024-09-15T01:33:39.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a2659c640 1 --2- 172.21.15.204:0/1519310871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a100062a0 0x7f4a28141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:39.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a26d9d640 1 --2- 172.21.15.204:0/1519310871 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a2811f380 0x7f4a2811f7d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4a2806f710 tx=0x7f4a20007410 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a167fc640 1 -- 172.21.15.204:0/1519310871 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a2000cd50 con 0x7f4a2811f380 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a167fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a167fc640 10 monclient: dump: 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:39.865 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a167fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a167fc640 1 -- 172.21.15.204:0/1519310871 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2000f8f0 con 0x7f4a2811f380 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a2cb5d640 5 monclient: authenticate success, global_id 4738 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a2cb5d640 10 monclient: _renew_subs 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a2cb5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1519310871 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a28122930 con 0x7f4a2811f380 2024-09-15T01:33:39.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a167fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a167fc640 1 -- 172.21.15.204:0/1519310871 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a200050d0 con 0x7f4a2811f380 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a167fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a167fc640 10 monclient: dump: 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:39.867 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a24d99640 4 set_mon_vals no callback set 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a2cb5d640 10 monclient: _renew_subs 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a2cb5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.843+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1519310871 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a28142b50 con 0x7f4a2811f380 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.844+0000 7f4a167fc640 1 -- 172.21.15.204:0/1519310871 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4a2000f470 con 0x7f4a2811f380 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.844+0000 7f4a2cb5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:39.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.844+0000 7f4a2cb5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:39.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.844+0000 7f4a2cb5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:39.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.844+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1519310871 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a28122d50 con 0x7f4a2811f380 2024-09-15T01:33:39.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.844+0000 7f4a167fc640 1 --2- 172.21.15.204:0/1519310871 >> v2:172.21.15.204:6812/28210 conn(0x7f4a080397b0 0x7f4a0803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:39.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.844+0000 7f4a167fc640 1 -- 172.21.15.204:0/1519310871 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 3632+0+0 (secure 0 0 0) 0x7f4a2004ab80 con 0x7f4a2811f380 2024-09-15T01:33:39.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.844+0000 7f4a2659c640 1 --2- 172.21.15.204:0/1519310871 >> v2:172.21.15.204:6812/28210 conn(0x7f4a080397b0 0x7f4a0803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:39.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.844+0000 7f4a2659c640 10 monclient: get_auth_request con 0x7f4a080397b0 auth_method 0 2024-09-15T01:33:39.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.845+0000 7f4a2659c640 1 --2- 172.21.15.204:0/1519310871 >> v2:172.21.15.204:6812/28210 conn(0x7f4a080397b0 0x7f4a0803bc60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4a18006a70 tx=0x7f4a18008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:39.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.849+0000 7f4a167fc640 1 -- 172.21.15.204:0/1519310871 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4a28122d50 con 0x7f4a2811f380 2024-09-15T01:33:39.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.849+0000 7f4a167fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:39.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:39.849+0000 7f4a167fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:40.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:40.005+0000 7f4a2cb5d640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-09-15T01:33:40.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:40.005+0000 7f4a2cb5d640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-09-15T01:33:40.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:40.005+0000 7f4a2cb5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:40.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:40.005+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1519310871 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]} v 0) v1 -- 0x7f4a28122d50 con 0x7f4a2811f380 2024-09-15T01:33:40.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:40.553+0000 7f4a167fc640 1 -- 172.21.15.204:0/1519310871 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}]=0 set osd(s) 1 to class 'hdd' v51) v1 ==== 135+0+0 (secure 0 0 0) 0x7f4a28122d50 con 0x7f4a2811f380 2024-09-15T01:33:40.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:40.553+0000 7f4a167fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-09-15T01:33:40.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:40.553+0000 7f4a167fc640 10 monclient: _finish_command 2 = system:0 set osd(s) 1 to class 'hdd' 2024-09-15T01:33:40.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:40.650+0000 7f4a2cb5d640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-09-15T01:33:40.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:40.650+0000 7f4a2cb5d640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-09-15T01:33:40.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:40.650+0000 7f4a2cb5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:40.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:40.650+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1519310871 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]} v 0) v1 -- 0x7f4a28122d50 con 0x7f4a2811f380 2024-09-15T01:33:41.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.559+0000 7f4a167fc640 1 -- 172.21.15.204:0/1519310871 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}]=0 osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. set osd(s) to class 'hdd' v52) v1 ==== 237+0+0 (secure 0 0 0) 0x7f4a28122d50 con 0x7f4a2811f380 2024-09-15T01:33:41.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.559+0000 7f4a167fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-09-15T01:33:41.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.559+0000 7f4a167fc640 10 monclient: _finish_command 3 = system:0 osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. set osd(s) to class 'hdd' 2024-09-15T01:33:41.561 INFO:tasks.workunit.client.0.smithi204.stderr:osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. set osd(s) to class 'hdd' 2024-09-15T01:33:41.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.561+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1519310871 >> v2:172.21.15.204:6812/28210 conn(0x7f4a080397b0 msgr2=0x7f4a0803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:41.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.561+0000 7f4a2cb5d640 1 --2- 172.21.15.204:0/1519310871 >> v2:172.21.15.204:6812/28210 conn(0x7f4a080397b0 0x7f4a0803bc60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4a18006a70 tx=0x7f4a18008040 comp rx=0 tx=0).stop 2024-09-15T01:33:41.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.561+0000 7f4a2cb5d640 10 monclient: shutdown 2024-09-15T01:33:41.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.561+0000 7f4a2cb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:41.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.561+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1519310871 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a2811f380 msgr2=0x7f4a2811f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:41.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.561+0000 7f4a2cb5d640 1 --2- 172.21.15.204:0/1519310871 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a2811f380 0x7f4a2811f7d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4a2806f710 tx=0x7f4a20007410 comp rx=0 tx=0).stop 2024-09-15T01:33:41.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.561+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1519310871 shutdown_connections 2024-09-15T01:33:41.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.561+0000 7f4a2cb5d640 1 --2- 172.21.15.204:0/1519310871 >> v2:172.21.15.204:6812/28210 conn(0x7f4a080397b0 0x7f4a0803bc60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:41.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.561+0000 7f4a2cb5d640 1 --2- 172.21.15.204:0/1519310871 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a2811f380 0x7f4a2811f7d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:41.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.561+0000 7f4a2cb5d640 1 --2- 172.21.15.204:0/1519310871 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a281327e0 0x7f4a28132bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:41.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.561+0000 7f4a2cb5d640 1 --2- 172.21.15.204:0/1519310871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a100062a0 0x7f4a28141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:41.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.561+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1519310871 >> 172.21.15.204:0/1519310871 conn(0x7f4a2806e860 msgr2=0x7f4a280969e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:41.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.562+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1519310871 shutdown_connections 2024-09-15T01:33:41.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.562+0000 7f4a2cb5d640 1 -- 172.21.15.204:0/1519310871 wait complete. 2024-09-15T01:33:41.575 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule create-replicated foo-ssd default host ssd 2024-09-15T01:33:41.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.657+0000 7f90ea99d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:41.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.657+0000 7f90ea99d640 10 monclient: build_initial_monmap 2024-09-15T01:33:41.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.657+0000 7f90ea99d640 10 monclient: monmap: 2024-09-15T01:33:41.659 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:41.659 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:41.659 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:41.657647+0000 2024-09-15T01:33:41.659 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:41.657647+0000 2024-09-15T01:33:41.659 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:41.659 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:41.659 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:41.659 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:41.660 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:41.660 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:41.660 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:41.660 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:41.660 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:41.660 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:41.660 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:41.660 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:41.660 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:41.660 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:41.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.657+0000 7f90ea99d640 1 Processor -- start 2024-09-15T01:33:41.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 1 -- start start 2024-09-15T01:33:41.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 10 monclient: init 2024-09-15T01:33:41.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:41.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 10 monclient: _add_conns ranks=[7,6,4,9,8,0,1,5,2,3,10] 2024-09-15T01:33:41.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4068410 0x7f90e412f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:41.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 10 monclient(hunting): picked mon.noname-i con 0x7f90e4068410 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:41.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f90e409c190 0x7f90e412fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:41.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 10 monclient(hunting): picked mon.noname-d con 0x7f90e409c190 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:41.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f90e4099870 0x7f90e4130350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:41.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 10 monclient(hunting): picked mon.noname-c con 0x7f90e4099870 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:41.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:41.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90e409b910 con 0x7f90e4099870 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f90e4068ca0 con 0x7f90e409c190 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f90e409b5c0 con 0x7f90e4068410 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90ea99d640 10 monclient(hunting): authenticate will time out at 837.300538s 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e37fe640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f90e409c190 msgr2=0x7f90e412fe10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4068410 0x7f90e412f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e37fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f90e409c190 0x7f90e412fe10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4068410 0x7f90e412f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:58808/0 (socket says 172.21.15.204:58808) 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 1 -- 172.21.15.204:0/3611026286 learned_addr learned my addr 172.21.15.204:0/3611026286 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 10 monclient(hunting): get_auth_request con 0x7f90e4068410 auth_method 0 2024-09-15T01:33:41.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:41.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:41.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:41.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:41.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:41.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:41.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e8f3a640 1 --2- 172.21.15.204:0/3611026286 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f90e4099870 0x7f90e4130350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:41.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e8f3a640 10 monclient(hunting): get_auth_request con 0x7f90e4099870 auth_method 0 2024-09-15T01:33:41.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e8f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:41.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e8f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:41.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e8f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:41.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 10 monclient(hunting): handle_auth_done global_id 4813 payload 274 2024-09-15T01:33:41.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 1 -- 172.21.15.204:0/3611026286 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f90e409c190 msgr2=0x7f90e412fe10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:41.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 1 --2- 172.21.15.204:0/3611026286 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f90e409c190 0x7f90e412fe10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:41.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 1 -- 172.21.15.204:0/3611026286 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f90e4099870 msgr2=0x7f90e4130350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:41.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 1 --2- 172.21.15.204:0/3611026286 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f90e4099870 0x7f90e4130350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:41.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:41.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 1 monclient: found mon.noname-i 2024-09-15T01:33:41.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:41.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:41.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.658+0000 7f90e3fff640 1 -- 172.21.15.204:0/3611026286 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e404b720 con 0x7f90e4068410 2024-09-15T01:33:41.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e3fff640 1 --2- 172.21.15.204:0/3611026286 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4068410 0x7f90e412f8d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009940 tx=0x7f90d40091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=f085a05b67b2f5cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:41.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e2ffd640 1 -- 172.21.15.204:0/3611026286 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90d402ac80 con 0x7f90e4068410 2024-09-15T01:33:41.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:41.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e2ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e2ffd640 10 monclient: dump: 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:41.674 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90ea99d640 5 monclient: authenticate success, global_id 4813 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90ea99d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e2ffd640 1 -- 172.21.15.204:0/3611026286 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d402ae00 con 0x7f90e4068410 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90ea99d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90ea99d640 4 set_mon_vals no callback set 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e2ffd640 1 -- 172.21.15.204:0/3611026286 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90d4031ca0 con 0x7f90e4068410 2024-09-15T01:33:41.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.659+0000 7f90e2ffd640 10 monclient: dump: 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:41.676 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 10 monclient: shutdown 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 1 -- 172.21.15.204:0/3611026286 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4068410 msgr2=0x7f90e412f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 1 --2- 172.21.15.204:0/3611026286 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4068410 0x7f90e412f8d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009940 tx=0x7f90d40091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 1 -- 172.21.15.204:0/3611026286 shutdown_connections 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 1 --2- 172.21.15.204:0/3611026286 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f90e4099870 0x7f90e4130350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 1 --2- 172.21.15.204:0/3611026286 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f90e409c190 0x7f90e412fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:41.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 1 --2- 172.21.15.204:0/3611026286 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4068410 0x7f90e412f8d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 1 -- 172.21.15.204:0/3611026286 >> 172.21.15.204:0/3611026286 conn(0x7f90e40a2e30 msgr2=0x7f90e40a0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 1 -- 172.21.15.204:0/3611026286 shutdown_connections 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 1 -- 172.21.15.204:0/3611026286 wait complete. 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 10 monclient: build_initial_monmap 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.660+0000 7f90ea99d640 10 monclient: monmap: 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:41.678 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 1 Processor -- start 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 1 -- start start 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 10 monclient: init 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:41.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 10 monclient: _add_conns ranks=[7,1,0,5,6,3,4,8,2] 2024-09-15T01:33:41.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4099870 0x7f90e41299f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:41.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 10 monclient(hunting): picked mon.noname-i con 0x7f90e4099870 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:41.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e409c190 0x7f90e412c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:41.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 10 monclient(hunting): picked mon.noname-f con 0x7f90e409c190 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:41.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90e412cac0 0x7f90e4095410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:41.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 10 monclient(hunting): picked mon.noname-a con 0x7f90e412cac0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:41.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:41.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e4067b60 con 0x7f90e412cac0 2024-09-15T01:33:41.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:41.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e402a800 con 0x7f90e409c190 2024-09-15T01:33:41.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:41.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f90e4128d30 con 0x7f90e4099870 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90ea99d640 10 monclient(hunting): authenticate will time out at 837.303773s 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4099870 0x7f90e41299f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4099870 0x7f90e41299f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:58820/0 (socket says 172.21.15.204:58820) 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e409c190 0x7f90e412c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e3fff640 1 -- 172.21.15.204:0/2144320303 learned_addr learned my addr 172.21.15.204:0/2144320303 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e3fff640 10 monclient(hunting): get_auth_request con 0x7f90e4099870 auth_method 0 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e37fe640 10 monclient(hunting): get_auth_request con 0x7f90e409c190 auth_method 0 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:41.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e8f3a640 1 --2- 172.21.15.204:0/2144320303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90e412cac0 0x7f90e4095410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.661+0000 7f90e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e8f3a640 10 monclient(hunting): get_auth_request con 0x7f90e412cac0 auth_method 0 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e8f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e8f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e8f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:41.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e8f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e8f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e8f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e37fe640 10 monclient(hunting): handle_auth_done global_id 5077 payload 824 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e37fe640 1 -- 172.21.15.204:0/2144320303 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4099870 msgr2=0x7f90e41299f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e37fe640 1 --2- 172.21.15.204:0/2144320303 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4099870 0x7f90e41299f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e37fe640 1 -- 172.21.15.204:0/2144320303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90e412cac0 msgr2=0x7f90e4095410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e37fe640 1 --2- 172.21.15.204:0/2144320303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90e412cac0 0x7f90e4095410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e37fe640 1 monclient: found mon.noname-f 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e37fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e37fe640 1 -- 172.21.15.204:0/2144320303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d4009450 con 0x7f90e409c190 2024-09-15T01:33:41.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e3fff640 1 --2- 172.21.15.204:0/2144320303 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4099870 0x7f90e41299f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e37fe640 1 --2- 172.21.15.204:0/2144320303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e409c190 0x7f90e412c580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90e40687f0 tx=0x7f90d80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90cbfff640 1 -- 172.21.15.204:0/2144320303 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90d8006150 con 0x7f90e409c190 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90cbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90cbfff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90cbfff640 10 monclient: dump: 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:41.684 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90cbfff640 10 monclient: _finish_auth 0 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90cbfff640 1 -- 172.21.15.204:0/2144320303 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d80062b0 con 0x7f90e409c190 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90ea99d640 5 monclient: authenticate success, global_id 5077 2024-09-15T01:33:41.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90ea99d640 10 monclient: _renew_subs 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90ea99d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90ea99d640 1 -- 172.21.15.204:0/2144320303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e4128770 con 0x7f90e409c190 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90cbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90cbfff640 1 -- 172.21.15.204:0/2144320303 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90d8010360 con 0x7f90e409c190 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90cbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90cbfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90cbfff640 10 monclient: dump: 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:41.686 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90e27fc640 4 set_mon_vals no callback set 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90ea99d640 10 monclient: _renew_subs 2024-09-15T01:33:41.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90ea99d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.662+0000 7f90ea99d640 1 -- 172.21.15.204:0/2144320303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e4129f30 con 0x7f90e409c190 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.663+0000 7f90cbfff640 1 -- 172.21.15.204:0/2144320303 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f90d8003b70 con 0x7f90e409c190 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.663+0000 7f90ea99d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.663+0000 7f90ea99d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.663+0000 7f90ea99d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.663+0000 7f90ea99d640 1 -- 172.21.15.204:0/2144320303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e4070cc0 con 0x7f90e409c190 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.663+0000 7f90cbfff640 1 --2- 172.21.15.204:0/2144320303 >> v2:172.21.15.204:6812/28210 conn(0x7f90d00397b0 0x7f90d003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.663+0000 7f90cbfff640 1 -- 172.21.15.204:0/2144320303 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 3864+0+0 (secure 0 0 0) 0x7f90d804a1c0 con 0x7f90e409c190 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.663+0000 7f90e3fff640 1 --2- 172.21.15.204:0/2144320303 >> v2:172.21.15.204:6812/28210 conn(0x7f90d00397b0 0x7f90d003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.663+0000 7f90e3fff640 10 monclient: get_auth_request con 0x7f90d00397b0 auth_method 0 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.664+0000 7f90e3fff640 1 --2- 172.21.15.204:0/2144320303 >> v2:172.21.15.204:6812/28210 conn(0x7f90d00397b0 0x7f90d003bc60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009940 tx=0x7f90d4032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.668+0000 7f90cbfff640 1 -- 172.21.15.204:0/2144320303 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f90d800da70 con 0x7f90e409c190 2024-09-15T01:33:41.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.668+0000 7f90cbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:41.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.668+0000 7f90cbfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:41.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.827+0000 7f90ea99d640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-09-15T01:33:41.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.827+0000 7f90ea99d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-09-15T01:33:41.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.827+0000 7f90ea99d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:41.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:41.827+0000 7f90ea99d640 1 -- 172.21.15.204:0/2144320303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"} v 0) v1 -- 0x7f90e4129070 con 0x7f90e409c190 2024-09-15T01:33:42.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.571+0000 7f90cbfff640 1 -- 172.21.15.204:0/2144320303 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}]=0 v53) v1 ==== 150+0+0 (secure 0 0 0) 0x7f90e4129070 con 0x7f90e409c190 2024-09-15T01:33:42.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.571+0000 7f90cbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-09-15T01:33:42.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.571+0000 7f90cbfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:42.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.670+0000 7f90ea99d640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-09-15T01:33:42.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.670+0000 7f90ea99d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-09-15T01:33:42.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.670+0000 7f90ea99d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:42.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.670+0000 7f90ea99d640 1 -- 172.21.15.204:0/2144320303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"} v 0) v1 -- 0x7f90e4129070 con 0x7f90e409c190 2024-09-15T01:33:42.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.671+0000 7f90cbfff640 1 -- 172.21.15.204:0/2144320303 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}]=0 rule foo-ssd already exists v53) v1 ==== 177+0+0 (secure 0 0 0) 0x7f90e4129070 con 0x7f90e409c190 2024-09-15T01:33:42.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.671+0000 7f90cbfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-09-15T01:33:42.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.671+0000 7f90cbfff640 10 monclient: _finish_command 3 = system:0 rule foo-ssd already exists 2024-09-15T01:33:42.673 INFO:tasks.workunit.client.0.smithi204.stderr:rule foo-ssd already exists 2024-09-15T01:33:42.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 1 -- 172.21.15.204:0/2144320303 >> v2:172.21.15.204:6812/28210 conn(0x7f90d00397b0 msgr2=0x7f90d003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:42.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 1 --2- 172.21.15.204:0/2144320303 >> v2:172.21.15.204:6812/28210 conn(0x7f90d00397b0 0x7f90d003bc60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009940 tx=0x7f90d4032000 comp rx=0 tx=0).stop 2024-09-15T01:33:42.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 10 monclient: shutdown 2024-09-15T01:33:42.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:42.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 1 -- 172.21.15.204:0/2144320303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e409c190 msgr2=0x7f90e412c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:42.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 1 --2- 172.21.15.204:0/2144320303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e409c190 0x7f90e412c580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90e40687f0 tx=0x7f90d80090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 1 -- 172.21.15.204:0/2144320303 shutdown_connections 2024-09-15T01:33:42.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 1 --2- 172.21.15.204:0/2144320303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e409c190 0x7f90e412c580 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 1 --2- 172.21.15.204:0/2144320303 >> v2:172.21.15.204:6812/28210 conn(0x7f90d00397b0 0x7f90d003bc60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 1 --2- 172.21.15.204:0/2144320303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90e412cac0 0x7f90e4095410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 1 --2- 172.21.15.204:0/2144320303 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f90e4099870 0x7f90e41299f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 1 -- 172.21.15.204:0/2144320303 >> 172.21.15.204:0/2144320303 conn(0x7f90e40a2e30 msgr2=0x7f90e4098270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:42.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 1 -- 172.21.15.204:0/2144320303 shutdown_connections 2024-09-15T01:33:42.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.673+0000 7f90ea99d640 1 -- 172.21.15.204:0/2144320303 wait complete. 2024-09-15T01:33:42.684 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule create-replicated foo-hdd default host hdd 2024-09-15T01:33:42.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.758+0000 7f12a939d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:42.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.758+0000 7f12a939d640 10 monclient: build_initial_monmap 2024-09-15T01:33:42.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.758+0000 7f12a939d640 10 monclient: monmap: 2024-09-15T01:33:42.760 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:42.760 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:42.759352+0000 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:42.759352+0000 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:42.761 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:42.762 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:42.762 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:42.762 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:42.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 1 Processor -- start 2024-09-15T01:33:42.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 1 -- start start 2024-09-15T01:33:42.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 10 monclient: init 2024-09-15T01:33:42.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:42.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 10 monclient: _add_conns ranks=[7,8,5,3,0,4,2,9,1,10,6] 2024-09-15T01:33:42.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12a40a0b40 0x7f12a409acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:42.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 10 monclient(hunting): picked mon.noname-i con 0x7f12a40a0b40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:42.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f12a4067130 0x7f12a4067520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:42.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 10 monclient(hunting): picked mon.noname-e con 0x7f12a4067130 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:42.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067a60 0x7f12a4139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:42.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 10 monclient(hunting): picked mon.noname-h con 0x7f12a4067a60 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:42.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:42.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12a4099d50 con 0x7f12a4067a60 2024-09-15T01:33:42.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:42.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f12a409a8c0 con 0x7f12a40a0b40 2024-09-15T01:33:42.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:42.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40a0f10 con 0x7f12a4067130 2024-09-15T01:33:42.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:42.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.759+0000 7f12a939d640 10 monclient(hunting): authenticate will time out at 838.401944s 2024-09-15T01:33:42.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12a40a0b40 0x7f12a409acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:42.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a37fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067a60 0x7f12a4139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:42.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12a40a0b40 0x7f12a409acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:58824/0 (socket says 172.21.15.204:58824) 2024-09-15T01:33:42.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 1 -- 172.21.15.204:0/1729146936 learned_addr learned my addr 172.21.15.204:0/1729146936 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:42.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a37fe640 10 monclient(hunting): get_auth_request con 0x7f12a4067a60 auth_method 0 2024-09-15T01:33:42.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:42.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:42.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:42.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 10 monclient(hunting): get_auth_request con 0x7f12a40a0b40 auth_method 0 2024-09-15T01:33:42.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:42.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:42.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:42.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a27fc640 1 --2- 172.21.15.204:0/1729146936 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f12a4067130 0x7f12a4067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:42.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:42.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:42.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:42.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:42.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:42.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a27fc640 10 monclient(hunting): get_auth_request con 0x7f12a4067130 auth_method 0 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 10 monclient(hunting): handle_auth_done global_id 4831 payload 274 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 1 -- 172.21.15.204:0/1729146936 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f12a4067130 msgr2=0x7f12a4067520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 1 --2- 172.21.15.204:0/1729146936 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f12a4067130 0x7f12a4067520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 1 -- 172.21.15.204:0/1729146936 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067a60 msgr2=0x7f12a4139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 1 --2- 172.21.15.204:0/1729146936 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067a60 0x7f12a4139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 1 monclient: found mon.noname-i 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:42.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a2ffd640 1 -- 172.21.15.204:0/1729146936 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a404b720 con 0x7f12a40a0b40 2024-09-15T01:33:42.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a27fc640 1 --2- 172.21.15.204:0/1729146936 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f12a4067130 0x7f12a4067520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:42.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.760+0000 7f12a37fe640 1 --2- 172.21.15.204:0/1729146936 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067a60 0x7f12a4139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:42.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a2ffd640 1 --2- 172.21.15.204:0/1729146936 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12a40a0b40 0x7f12a409acf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f129c009970 tx=0x7f129c0091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=2df103609c660dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:42.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a1ffb640 1 -- 172.21.15.204:0/1729146936 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f129c02ac50 con 0x7f12a40a0b40 2024-09-15T01:33:42.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a1ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a1ffb640 10 monclient: dump: 2024-09-15T01:33:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:42.779 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:42.779 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:42.779 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:42.779 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:42.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:42.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:42.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a1ffb640 1 -- 172.21.15.204:0/1729146936 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c02add0 con 0x7f12a40a0b40 2024-09-15T01:33:42.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 5 monclient: authenticate success, global_id 4831 2024-09-15T01:33:42.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:42.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:42.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:42.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a1ffb640 1 -- 172.21.15.204:0/1729146936 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f129c031ca0 con 0x7f12a40a0b40 2024-09-15T01:33:42.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 4 set_mon_vals no callback set 2024-09-15T01:33:42.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:42.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a1ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:42.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a1ffb640 10 monclient: dump: 2024-09-15T01:33:42.780 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:42.780 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:42.780 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:42.781 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:42.781 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:42.781 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:42.781 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:42.781 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:42.781 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:42.781 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:42.781 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:42.781 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:42.781 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 10 monclient: shutdown 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 1 -- 172.21.15.204:0/1729146936 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12a40a0b40 msgr2=0x7f12a409acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 1 --2- 172.21.15.204:0/1729146936 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12a40a0b40 0x7f12a409acf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f129c009970 tx=0x7f129c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 1 -- 172.21.15.204:0/1729146936 shutdown_connections 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 1 --2- 172.21.15.204:0/1729146936 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067a60 0x7f12a4139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 1 --2- 172.21.15.204:0/1729146936 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f12a4067130 0x7f12a4067520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 1 --2- 172.21.15.204:0/1729146936 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12a40a0b40 0x7f12a409acf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 1 -- 172.21.15.204:0/1729146936 >> 172.21.15.204:0/1729146936 conn(0x7f12a40a2e30 msgr2=0x7f12a40a0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 1 -- 172.21.15.204:0/1729146936 shutdown_connections 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.761+0000 7f12a939d640 1 -- 172.21.15.204:0/1729146936 wait complete. 2024-09-15T01:33:42.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.762+0000 7f12a939d640 10 monclient: build_initial_monmap 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.762+0000 7f12a939d640 10 monclient: monmap: 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:42.783 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:42.784 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:42.784 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:42.784 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:42.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.762+0000 7f12a939d640 1 Processor -- start 2024-09-15T01:33:42.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.762+0000 7f12a939d640 1 -- start start 2024-09-15T01:33:42.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.762+0000 7f12a939d640 10 monclient: init 2024-09-15T01:33:42.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:42.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 10 monclient: _add_conns ranks=[5,0,8,4,1,6,3,2,7] 2024-09-15T01:33:42.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067130 0x7f12a4141c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:42.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 10 monclient(hunting): picked mon.noname-h con 0x7f12a4067130 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:42.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12a4067a60 0x7f12a4094630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:42.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 10 monclient(hunting): picked mon.noname-a con 0x7f12a4067a60 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:42.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f12a40a0b40 0x7f12a406d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:42.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 10 monclient(hunting): picked mon.noname-e con 0x7f12a40a0b40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:42.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:42.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a408e140 con 0x7f12a4067a60 2024-09-15T01:33:42.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:42.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12a408d810 con 0x7f12a4067130 2024-09-15T01:33:42.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:42.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f12a408daf0 con 0x7f12a40a0b40 2024-09-15T01:33:42.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:42.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 10 monclient(hunting): authenticate will time out at 838.405266s 2024-09-15T01:33:42.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067130 0x7f12a4141c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067130 0x7f12a4141c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:34808/0 (socket says 172.21.15.204:34808) 2024-09-15T01:33:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 1 -- 172.21.15.204:0/2454104107 learned_addr learned my addr 172.21.15.204:0/2454104107 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 10 monclient(hunting): get_auth_request con 0x7f12a4067130 auth_method 0 2024-09-15T01:33:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a27fc640 1 --2- 172.21.15.204:0/2454104107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12a4067a60 0x7f12a4094630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a37fe640 1 --2- 172.21.15.204:0/2454104107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f12a40a0b40 0x7f12a406d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a27fc640 10 monclient(hunting): get_auth_request con 0x7f12a4067a60 auth_method 0 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a37fe640 10 monclient(hunting): get_auth_request con 0x7f12a40a0b40 auth_method 0 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 10 monclient(hunting): handle_auth_done global_id 4829 payload 824 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 1 -- 172.21.15.204:0/2454104107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f12a40a0b40 msgr2=0x7f12a406d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 1 --2- 172.21.15.204:0/2454104107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f12a40a0b40 0x7f12a406d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 1 -- 172.21.15.204:0/2454104107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12a4067a60 msgr2=0x7f12a4094630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 1 --2- 172.21.15.204:0/2454104107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12a4067a60 0x7f12a4094630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:42.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 1 monclient: found mon.noname-h 2024-09-15T01:33:42.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:42.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:42.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 1 -- 172.21.15.204:0/2454104107 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1294009a20 con 0x7f12a4067130 2024-09-15T01:33:42.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a27fc640 1 --2- 172.21.15.204:0/2454104107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12a4067a60 0x7f12a4094630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:42.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a37fe640 1 --2- 172.21.15.204:0/2454104107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f12a40a0b40 0x7f12a406d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:42.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a2ffd640 1 --2- 172.21.15.204:0/2454104107 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067130 0x7f12a4141c60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f129c035040 tx=0x7f129c032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:42.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f128affd640 1 -- 172.21.15.204:0/2454104107 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f129c031820 con 0x7f12a4067130 2024-09-15T01:33:42.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f128affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:42.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f128affd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:42.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f128affd640 10 monclient: dump: 2024-09-15T01:33:42.788 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:42.788 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:42.789 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:42.789 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:42.789 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:42.789 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:42.789 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:42.789 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:42.789 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:42.789 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:42.789 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:42.789 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:42.789 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:42.790 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:42.790 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:42.790 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:42.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f128affd640 10 monclient: _finish_auth 0 2024-09-15T01:33:42.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f128affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:42.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 5 monclient: authenticate success, global_id 4829 2024-09-15T01:33:42.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f128affd640 1 -- 172.21.15.204:0/2454104107 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c0319c0 con 0x7f12a4067130 2024-09-15T01:33:42.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 10 monclient: _renew_subs 2024-09-15T01:33:42.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:42.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f12a939d640 1 -- 172.21.15.204:0/2454104107 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129c009450 con 0x7f12a4067130 2024-09-15T01:33:42.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.763+0000 7f128affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:42.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.764+0000 7f128affd640 1 -- 172.21.15.204:0/2454104107 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f129c03a390 con 0x7f12a4067130 2024-09-15T01:33:42.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.764+0000 7f128affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:42.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.764+0000 7f128affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:42.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.764+0000 7f12a17fa640 4 set_mon_vals no callback set 2024-09-15T01:33:42.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.764+0000 7f128affd640 10 monclient: dump: 2024-09-15T01:33:42.792 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:42.792 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:42.792 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:42.792 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:42.792 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:42.792 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:42.792 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:42.792 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:42.792 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:42.792 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:42.792 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:42.792 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.764+0000 7f12a939d640 10 monclient: _renew_subs 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.764+0000 7f12a939d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.764+0000 7f12a939d640 1 -- 172.21.15.204:0/2454104107 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a408c920 con 0x7f12a4067130 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.764+0000 7f128affd640 1 -- 172.21.15.204:0/2454104107 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f129c039040 con 0x7f12a4067130 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.765+0000 7f128affd640 1 --2- 172.21.15.204:0/2454104107 >> v2:172.21.15.204:6812/28210 conn(0x7f12900397b0 0x7f129003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.765+0000 7f12a939d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.765+0000 7f12a939d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.765+0000 7f12a939d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.765+0000 7f12a939d640 1 -- 172.21.15.204:0/2454104107 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a4070cc0 con 0x7f12a4067130 2024-09-15T01:33:42.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.765+0000 7f128affd640 1 -- 172.21.15.204:0/2454104107 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 3927+0+0 (secure 0 0 0) 0x7f129c06a890 con 0x7f12a4067130 2024-09-15T01:33:42.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.765+0000 7f12a27fc640 1 --2- 172.21.15.204:0/2454104107 >> v2:172.21.15.204:6812/28210 conn(0x7f12900397b0 0x7f129003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:42.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.765+0000 7f12a27fc640 10 monclient: get_auth_request con 0x7f12900397b0 auth_method 0 2024-09-15T01:33:42.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.765+0000 7f12a27fc640 1 --2- 172.21.15.204:0/2454104107 >> v2:172.21.15.204:6812/28210 conn(0x7f12900397b0 0x7f129003bc60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f12a406ee20 tx=0x7f1298008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:42.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.769+0000 7f128affd640 1 -- 172.21.15.204:0/2454104107 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f129c037070 con 0x7f12a4067130 2024-09-15T01:33:42.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.769+0000 7f128affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:42.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.769+0000 7f128affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:42.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.926+0000 7f12a939d640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-09-15T01:33:42.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.926+0000 7f12a939d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-09-15T01:33:42.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.926+0000 7f12a939d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:42.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:42.926+0000 7f12a939d640 1 -- 172.21.15.204:0/2454104107 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"} v 0) v1 -- 0x7f12a40a10f0 con 0x7f12a4067130 2024-09-15T01:33:43.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.572+0000 7f128affd640 1 -- 172.21.15.204:0/2454104107 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}]=0 v54) v1 ==== 150+0+0 (secure 0 0 0) 0x7f129c0313a0 con 0x7f12a4067130 2024-09-15T01:33:43.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.572+0000 7f128affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-09-15T01:33:43.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.572+0000 7f128affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:43.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.674+0000 7f12a939d640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-09-15T01:33:43.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.674+0000 7f12a939d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-09-15T01:33:43.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.674+0000 7f12a939d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:43.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.674+0000 7f12a939d640 1 -- 172.21.15.204:0/2454104107 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"} v 0) v1 -- 0x7f12a408cab0 con 0x7f12a4067130 2024-09-15T01:33:43.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.674+0000 7f128affd640 1 -- 172.21.15.204:0/2454104107 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}]=0 rule foo-hdd already exists v54) v1 ==== 177+0+0 (secure 0 0 0) 0x7f129c041350 con 0x7f12a4067130 2024-09-15T01:33:43.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.674+0000 7f128affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-09-15T01:33:43.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.674+0000 7f128affd640 10 monclient: _finish_command 3 = system:0 rule foo-hdd already exists 2024-09-15T01:33:43.676 INFO:tasks.workunit.client.0.smithi204.stderr:rule foo-hdd already exists 2024-09-15T01:33:43.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 1 -- 172.21.15.204:0/2454104107 >> v2:172.21.15.204:6812/28210 conn(0x7f12900397b0 msgr2=0x7f129003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:43.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 1 --2- 172.21.15.204:0/2454104107 >> v2:172.21.15.204:6812/28210 conn(0x7f12900397b0 0x7f129003bc60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f12a406ee20 tx=0x7f1298008040 comp rx=0 tx=0).stop 2024-09-15T01:33:43.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 10 monclient: shutdown 2024-09-15T01:33:43.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:43.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 1 -- 172.21.15.204:0/2454104107 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067130 msgr2=0x7f12a4141c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:43.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 1 --2- 172.21.15.204:0/2454104107 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067130 0x7f12a4141c60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f129c035040 tx=0x7f129c032000 comp rx=0 tx=0).stop 2024-09-15T01:33:43.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 1 -- 172.21.15.204:0/2454104107 shutdown_connections 2024-09-15T01:33:43.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 1 --2- 172.21.15.204:0/2454104107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f12a40a0b40 0x7f12a406d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:43.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 1 --2- 172.21.15.204:0/2454104107 >> v2:172.21.15.204:6812/28210 conn(0x7f12900397b0 0x7f129003bc60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:43.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 1 --2- 172.21.15.204:0/2454104107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12a4067a60 0x7f12a4094630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:43.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 1 --2- 172.21.15.204:0/2454104107 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f12a4067130 0x7f12a4141c60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:43.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 1 -- 172.21.15.204:0/2454104107 >> 172.21.15.204:0/2454104107 conn(0x7f12a40a2e30 msgr2=0x7f12a40298c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:43.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 1 -- 172.21.15.204:0/2454104107 shutdown_connections 2024-09-15T01:33:43.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.676+0000 7f12a939d640 1 -- 172.21.15.204:0/2454104107 wait complete. 2024-09-15T01:33:43.687 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule ls-by-class ssd 2024-09-15T01:33:43.687 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep foo-ssd 2024-09-15T01:33:43.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.759+0000 7ffb47b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:43.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.759+0000 7ffb47b9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:43.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.759+0000 7ffb47b9d640 10 monclient: monmap: 2024-09-15T01:33:43.761 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:43.761 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:43.761 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:43.760430+0000 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:43.760430+0000 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:43.762 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:43.763 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:43.763 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:43.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.760+0000 7ffb47b9d640 1 Processor -- start 2024-09-15T01:33:43.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.760+0000 7ffb47b9d640 1 -- start start 2024-09-15T01:33:43.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.760+0000 7ffb47b9d640 10 monclient: init 2024-09-15T01:33:43.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:43.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 10 monclient: _add_conns ranks=[0,3,10,8,5,2,1,7,6,9,4] 2024-09-15T01:33:43.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffb40067130 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:43.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb4009e940 0x7ffb4009c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:43.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 10 monclient(hunting): picked mon.noname-g con 0x7ffb4009e940 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:43.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffb4009cd20 0x7ffb40139d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:43.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 10 monclient(hunting): picked mon.noname-f con 0x7ffb4009cd20 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:43.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4004b720 con 0x7ffb40067130 2024-09-15T01:33:43.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:43.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4009d190 con 0x7ffb4009cd20 2024-09-15T01:33:43.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:43.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40067500 con 0x7ffb4009e940 2024-09-15T01:33:43.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:43.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb47b9d640 10 monclient(hunting): authenticate will time out at 839.403338s 2024-09-15T01:33:43.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb45138640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb4009e940 msgr2=0x7ffb4009c7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:43.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb45138640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb4009e940 0x7ffb4009c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:43.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffb4009cd20 0x7ffb40139d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:43.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffb4009cd20 0x7ffb40139d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55298/0 (socket says 172.21.15.204:55298) 2024-09-15T01:33:43.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 1 -- 172.21.15.204:0/355198977 learned_addr learned my addr 172.21.15.204:0/355198977 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:43.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 10 monclient(hunting): get_auth_request con 0x7ffb4009cd20 auth_method 0 2024-09-15T01:33:43.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:43.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:43.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:43.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:43.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:43.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 10 monclient(hunting): handle_auth_done global_id 5086 payload 274 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 1 -- 172.21.15.204:0/355198977 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb4009e940 msgr2=0x7ffb4009c7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 1 --2- 172.21.15.204:0/355198977 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb4009e940 0x7ffb4009c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 1 -- 172.21.15.204:0/355198977 >> v1:172.21.15.178:6789/0 conn(0x7ffb40067130 legacy=0x7ffb4009db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 10 monclient: _finish_hunting 0 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 1 monclient: found mon.noname-f 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.761+0000 7ffb4613a640 1 -- 172.21.15.204:0/355198977 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4009e220 con 0x7ffb4009cd20 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb4613a640 1 --2- 172.21.15.204:0/355198977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb4009cd20 0x7ffb40139d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009f20 tx=0x7ffb38009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18d8c09f9939a7be server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb44937640 1 -- 172.21.15.204:0/355198977 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb38007880 con 0x7ffb4009cd20 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb44937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:43.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb44937640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb44937640 10 monclient: dump: 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:43.776 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb44937640 10 monclient: _finish_auth 0 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb44937640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb44937640 1 -- 172.21.15.204:0/355198977 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3802d050 con 0x7ffb4009cd20 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb47b9d640 5 monclient: authenticate success, global_id 5086 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb47b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb44937640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb47b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb44937640 1 -- 172.21.15.204:0/355198977 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb38030360 con 0x7ffb4009cd20 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb47b9d640 4 set_mon_vals no callback set 2024-09-15T01:33:43.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb44937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb44937640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.762+0000 7ffb44937640 10 monclient: dump: 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:43.778 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:43.779 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:43.779 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:43.779 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:43.779 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:43.779 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:43.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 10 monclient: shutdown 2024-09-15T01:33:43.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:43.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 1 -- 172.21.15.204:0/355198977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb4009cd20 msgr2=0x7ffb40139d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:43.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 1 --2- 172.21.15.204:0/355198977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb4009cd20 0x7ffb40139d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009f20 tx=0x7ffb38009840 comp rx=0 tx=0).stop 2024-09-15T01:33:43.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 1 -- 172.21.15.204:0/355198977 shutdown_connections 2024-09-15T01:33:43.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 1 --2- 172.21.15.204:0/355198977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb4009cd20 0x7ffb40139d90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:43.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 1 --2- 172.21.15.204:0/355198977 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb4009e940 0x7ffb4009c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 1 -- 172.21.15.204:0/355198977 >> 172.21.15.204:0/355198977 conn(0x7ffb400a2e50 msgr2=0x7ffb400a3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 1 -- 172.21.15.204:0/355198977 shutdown_connections 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 1 -- 172.21.15.204:0/355198977 wait complete. 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 10 monclient: monmap: 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:43.780 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.763+0000 7ffb47b9d640 1 Processor -- start 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 1 -- start start 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 10 monclient: init 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:43.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 10 monclient: _add_conns ranks=[6,2,4,1,8,0,3,7,5] 2024-09-15T01:33:43.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb40067130 0x7ffb4013e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:43.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 10 monclient(hunting): picked mon.noname-d con 0x7ffb40067130 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:43.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb4009cd20 0x7ffb4013daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:43.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 10 monclient(hunting): picked mon.noname-b con 0x7ffb4009cd20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:43.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4009e940 0x7ffb401419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:43.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 10 monclient(hunting): picked mon.noname-c con 0x7ffb4009e940 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:43.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:43.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40123430 con 0x7ffb4009cd20 2024-09-15T01:33:43.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:43.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40122e70 con 0x7ffb4009e940 2024-09-15T01:33:43.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:43.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40120990 con 0x7ffb40067130 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb47b9d640 10 monclient(hunting): authenticate will time out at 839.406474s 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb45939640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb40067130 msgr2=0x7ffb4013e440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb45939640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb40067130 0x7ffb4013e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb45138640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb4009cd20 0x7ffb4013daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4009e940 0x7ffb401419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4009e940 0x7ffb401419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:38822/0 (socket says 172.21.15.204:38822) 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb45138640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb4009cd20 0x7ffb4013daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:48320/0 (socket says 172.21.15.204:48320) 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 1 -- 172.21.15.204:0/2964056280 learned_addr learned my addr 172.21.15.204:0/2964056280 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 10 monclient(hunting): get_auth_request con 0x7ffb4009e940 auth_method 0 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:43.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:43.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb45138640 10 monclient(hunting): get_auth_request con 0x7ffb4009cd20 auth_method 0 2024-09-15T01:33:43.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb45138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:43.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb45138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:43.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb45138640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:43.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:43.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:43.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:43.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb45138640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:43.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb45138640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:43.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb45138640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:43.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 10 monclient(hunting): handle_auth_done global_id 4756 payload 824 2024-09-15T01:33:43.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 1 -- 172.21.15.204:0/2964056280 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb40067130 msgr2=0x7ffb4013e440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 1 --2- 172.21.15.204:0/2964056280 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb40067130 0x7ffb4013e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 1 -- 172.21.15.204:0/2964056280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb4009cd20 msgr2=0x7ffb4013daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 1 --2- 172.21.15.204:0/2964056280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb4009cd20 0x7ffb4013daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 10 monclient: _finish_hunting 0 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 1 monclient: found mon.noname-c 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.764+0000 7ffb4613a640 1 -- 172.21.15.204:0/2964056280 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb38009ae0 con 0x7ffb4009e940 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb45138640 1 --2- 172.21.15.204:0/2964056280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb4009cd20 0x7ffb4013daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb4613a640 1 --2- 172.21.15.204:0/2964056280 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4009e940 0x7ffb401419b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009880 tx=0x7ffb38036000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb35ffb640 1 -- 172.21.15.204:0/2964056280 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb38008c20 con 0x7ffb4009e940 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb35ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:43.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb35ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb35ffb640 10 monclient: dump: 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:43.786 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:43.787 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:43.787 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:43.787 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:43.787 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:43.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb35ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:43.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb35ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:43.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb47b9d640 5 monclient: authenticate success, global_id 4756 2024-09-15T01:33:43.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb35ffb640 1 -- 172.21.15.204:0/2964056280 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb38008da0 con 0x7ffb4009e940 2024-09-15T01:33:43.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb35ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:43.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb35ffb640 1 -- 172.21.15.204:0/2964056280 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb3803b890 con 0x7ffb4009e940 2024-09-15T01:33:43.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb35ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:43.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb35ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:43.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb37fff640 4 set_mon_vals no callback set 2024-09-15T01:33:43.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb35ffb640 10 monclient: dump: 2024-09-15T01:33:43.788 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:43.788 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:43.788 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:43.788 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:43.788 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:43.788 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:43.788 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:43.788 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:43.788 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:43.788 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb47b9d640 10 monclient: _renew_subs 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb47b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb47b9d640 1 -- 172.21.15.204:0/2964056280 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb40122b90 con 0x7ffb4009e940 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb47b9d640 10 monclient: _renew_subs 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb47b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.765+0000 7ffb47b9d640 1 -- 172.21.15.204:0/2964056280 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4013e030 con 0x7ffb4009e940 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.766+0000 7ffb47b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:43.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.766+0000 7ffb47b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:43.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.766+0000 7ffb47b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:43.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.766+0000 7ffb47b9d640 1 -- 172.21.15.204:0/2964056280 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb40070cc0 con 0x7ffb4009e940 2024-09-15T01:33:43.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.766+0000 7ffb35ffb640 1 -- 172.21.15.204:0/2964056280 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ffb38038050 con 0x7ffb4009e940 2024-09-15T01:33:43.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.766+0000 7ffb35ffb640 1 --2- 172.21.15.204:0/2964056280 >> v2:172.21.15.204:6812/28210 conn(0x7ffb240397b0 0x7ffb2403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:43.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.767+0000 7ffb35ffb640 1 -- 172.21.15.204:0/2964056280 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 3990+0+0 (secure 0 0 0) 0x7ffb3806bcc0 con 0x7ffb4009e940 2024-09-15T01:33:43.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.767+0000 7ffb45939640 1 --2- 172.21.15.204:0/2964056280 >> v2:172.21.15.204:6812/28210 conn(0x7ffb240397b0 0x7ffb2403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:43.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.767+0000 7ffb45939640 10 monclient: get_auth_request con 0x7ffb240397b0 auth_method 0 2024-09-15T01:33:43.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.767+0000 7ffb45939640 1 --2- 172.21.15.204:0/2964056280 >> v2:172.21.15.204:6812/28210 conn(0x7ffb240397b0 0x7ffb2403bc60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c007fd0 tx=0x7ffb3c007a70 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:43.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.771+0000 7ffb35ffb640 1 -- 172.21.15.204:0/2964056280 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ffb3803b410 con 0x7ffb4009e940 2024-09-15T01:33:43.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.771+0000 7ffb35ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:43.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.771+0000 7ffb35ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:43.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.926+0000 7ffb47b9d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T01:33:43.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.926+0000 7ffb47b9d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T01:33:43.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.926+0000 7ffb47b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:43.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.926+0000 7ffb47b9d640 1 -- 172.21.15.204:0/2964056280 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) v1 -- 0x7ffb4009aa10 con 0x7ffb4009e940 2024-09-15T01:33:43.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.927+0000 7ffb35ffb640 1 -- 172.21.15.204:0/2964056280 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v54) v1 ==== 90+0+8 (secure 0 0 0) 0x7ffb3803f070 con 0x7ffb4009e940 2024-09-15T01:33:43.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.927+0000 7ffb35ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T01:33:43.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:43.927+0000 7ffb35ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:44.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.020+0000 7ffb47b9d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T01:33:44.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.020+0000 7ffb47b9d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T01:33:44.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.020+0000 7ffb47b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:44.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.020+0000 7ffb47b9d640 1 -- 172.21.15.204:0/2964056280 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) v1 -- 0x7ffb400008d0 con 0x7ffb4009e940 2024-09-15T01:33:44.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.021+0000 7ffb35ffb640 1 -- 172.21.15.204:0/2964056280 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v54) v1 ==== 90+0+8 (secure 0 0 0) 0x7ffb3804cb40 con 0x7ffb4009e940 2024-09-15T01:33:44.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.021+0000 7ffb35ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T01:33:44.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.021+0000 7ffb35ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:44.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.022+0000 7ffb47b9d640 1 -- 172.21.15.204:0/2964056280 >> v2:172.21.15.204:6812/28210 conn(0x7ffb240397b0 msgr2=0x7ffb2403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.022+0000 7ffb47b9d640 1 --2- 172.21.15.204:0/2964056280 >> v2:172.21.15.204:6812/28210 conn(0x7ffb240397b0 0x7ffb2403bc60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c007fd0 tx=0x7ffb3c007a70 comp rx=0 tx=0).stop 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.022+0000 7ffb47b9d640 10 monclient: shutdown 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.022+0000 7ffb47b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.022+0000 7ffb47b9d640 1 -- 172.21.15.204:0/2964056280 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4009e940 msgr2=0x7ffb401419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.022+0000 7ffb47b9d640 1 --2- 172.21.15.204:0/2964056280 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4009e940 0x7ffb401419b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009880 tx=0x7ffb38036000 comp rx=0 tx=0).stop 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.023+0000 7ffb47b9d640 1 -- 172.21.15.204:0/2964056280 shutdown_connections 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.023+0000 7ffb47b9d640 1 --2- 172.21.15.204:0/2964056280 >> v2:172.21.15.204:6812/28210 conn(0x7ffb240397b0 0x7ffb2403bc60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.023+0000 7ffb47b9d640 1 --2- 172.21.15.204:0/2964056280 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4009e940 0x7ffb401419b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.023+0000 7ffb47b9d640 1 --2- 172.21.15.204:0/2964056280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb4009cd20 0x7ffb4013daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.023+0000 7ffb47b9d640 1 --2- 172.21.15.204:0/2964056280 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb40067130 0x7ffb4013e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.023+0000 7ffb47b9d640 1 -- 172.21.15.204:0/2964056280 >> 172.21.15.204:0/2964056280 conn(0x7ffb400a2e50 msgr2=0x7ffb4009b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.023+0000 7ffb47b9d640 1 -- 172.21.15.204:0/2964056280 shutdown_connections 2024-09-15T01:33:44.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.023+0000 7ffb47b9d640 1 -- 172.21.15.204:0/2964056280 wait complete. 2024-09-15T01:33:44.033 INFO:tasks.workunit.client.0.smithi204.stdout:foo-ssd 2024-09-15T01:33:44.034 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule ls-by-class ssd 2024-09-15T01:33:44.034 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false grep foo-hdd 2024-09-15T01:33:44.034 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:44.034 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep foo-hdd 2024-09-15T01:33:44.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.107+0000 7f3f6a114640 10 monclient: get_monmap_and_config 2024-09-15T01:33:44.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.107+0000 7f3f6a114640 10 monclient: build_initial_monmap 2024-09-15T01:33:44.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.107+0000 7f3f6a114640 10 monclient: monmap: 2024-09-15T01:33:44.109 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:44.109 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.109 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:44.108161+0000 2024-09-15T01:33:44.109 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:44.108161+0000 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:44.110 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.108+0000 7f3f6a114640 1 Processor -- start 2024-09-15T01:33:44.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.108+0000 7f3f6a114640 1 -- start start 2024-09-15T01:33:44.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.108+0000 7f3f6a114640 10 monclient: init 2024-09-15T01:33:44.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.108+0000 7f3f6a114640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.108+0000 7f3f6a114640 10 monclient: _add_conns ranks=[4,7,3,1,2,9,10,6,5,8,0] 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.108+0000 7f3f6a114640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3f640a2e30 0x7f3f64132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.108+0000 7f3f6a114640 10 monclient(hunting): picked mon.noname-c con 0x7f3f640a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.108+0000 7f3f6a114640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3f6409fb80 0x7f3f6409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f6a114640 10 monclient(hunting): picked mon.noname-i con 0x7f3f6409fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f6a114640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f640a0a40 0x7f3f640a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f6a114640 10 monclient(hunting): picked mon.noname-g con 0x7f3f640a0a40 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f6a114640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f6a114640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6409e6c0 con 0x7f3f640a0a40 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f6a114640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f6a114640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640a04b0 con 0x7f3f640a2e30 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f6a114640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f6a114640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640a3200 con 0x7f3f6409fb80 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f6a114640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f6a114640 10 monclient(hunting): authenticate will time out at 839.751056s 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f63fff640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f640a0a40 msgr2=0x7f3f640a0e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f63fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f640a0a40 0x7f3f640a0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3f6409fb80 0x7f3f6409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3f6409fb80 0x7f3f6409ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:58832/0 (socket says 172.21.15.204:58832) 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 1 -- 172.21.15.204:0/3353329774 learned_addr learned my addr 172.21.15.204:0/3353329774 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 10 monclient(hunting): get_auth_request con 0x7f3f6409fb80 auth_method 0 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:44.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f637fe640 1 --2- 172.21.15.204:0/3353329774 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3f640a2e30 0x7f3f64132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f637fe640 10 monclient(hunting): get_auth_request con 0x7f3f640a2e30 auth_method 0 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 10 monclient(hunting): handle_auth_done global_id 4840 payload 274 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 1 -- 172.21.15.204:0/3353329774 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3f640a2e30 msgr2=0x7f3f64132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 1 --2- 172.21.15.204:0/3353329774 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3f640a2e30 0x7f3f64132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 1 -- 172.21.15.204:0/3353329774 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f640a0a40 msgr2=0x7f3f640a0e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 1 --2- 172.21.15.204:0/3353329774 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f640a0a40 0x7f3f640a0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 1 monclient: found mon.noname-i 2024-09-15T01:33:44.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.109+0000 7f3f62ffd640 1 -- 172.21.15.204:0/3353329774 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6404b720 con 0x7f3f6409fb80 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f62ffd640 1 --2- 172.21.15.204:0/3353329774 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3f6409fb80 0x7f3f6409ff70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009940 tx=0x7f3f580091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=a3b4e175e1ab0193 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f627fc640 1 -- 172.21.15.204:0/3353329774 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f5802ac80 con 0x7f3f6409fb80 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f627fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f627fc640 10 monclient: dump: 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.123 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f627fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f627fc640 1 -- 172.21.15.204:0/3353329774 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5802ae00 con 0x7f3f6409fb80 2024-09-15T01:33:44.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 5 monclient: authenticate success, global_id 4840 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f627fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f627fc640 1 -- 172.21.15.204:0/3353329774 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f58031ca0 con 0x7f3f6409fb80 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 4 set_mon_vals no callback set 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f627fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f627fc640 10 monclient: dump: 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:44.125 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 10 monclient: shutdown 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 1 -- 172.21.15.204:0/3353329774 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3f6409fb80 msgr2=0x7f3f6409ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 1 --2- 172.21.15.204:0/3353329774 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3f6409fb80 0x7f3f6409ff70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009940 tx=0x7f3f580091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 1 -- 172.21.15.204:0/3353329774 shutdown_connections 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 1 --2- 172.21.15.204:0/3353329774 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3f6409fb80 0x7f3f6409ff70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 1 --2- 172.21.15.204:0/3353329774 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f640a0a40 0x7f3f640a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 1 --2- 172.21.15.204:0/3353329774 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3f640a2e30 0x7f3f64132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 1 -- 172.21.15.204:0/3353329774 >> 172.21.15.204:0/3353329774 conn(0x7f3f6406e860 msgr2=0x7f3f6406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 1 -- 172.21.15.204:0/3353329774 shutdown_connections 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.110+0000 7f3f6a114640 1 -- 172.21.15.204:0/3353329774 wait complete. 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.111+0000 7f3f6a114640 10 monclient: build_initial_monmap 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.111+0000 7f3f6a114640 10 monclient: monmap: 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:44.127 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.111+0000 7f3f6a114640 1 Processor -- start 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.111+0000 7f3f6a114640 1 -- start start 2024-09-15T01:33:44.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.111+0000 7f3f6a114640 10 monclient: init 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 10 monclient: _add_conns ranks=[4,1,0,7,8,5,6,3,2] 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3f6409fb80 0x7f3f64141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 10 monclient(hunting): picked mon.noname-c con 0x7f3f6409fb80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f640a0a40 0x7f3f64132700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 10 monclient(hunting): picked mon.noname-f con 0x7f3f640a0a40 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f640a2e30 0x7f3f6411f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 10 monclient(hunting): picked mon.noname-a con 0x7f3f640a2e30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f64142bc0 con 0x7f3f640a2e30 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6409ca40 con 0x7f3f640a0a40 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f64121500 con 0x7f3f6409fb80 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f6a114640 10 monclient(hunting): authenticate will time out at 839.754336s 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f640a0a40 0x7f3f64132700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f640a0a40 0x7f3f64132700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55302/0 (socket says 172.21.15.204:55302) 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 1 -- 172.21.15.204:0/4041741537 learned_addr learned my addr 172.21.15.204:0/4041741537 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 10 monclient(hunting): get_auth_request con 0x7f3f640a0a40 auth_method 0 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f637fe640 1 --2- 172.21.15.204:0/4041741537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3f6409fb80 0x7f3f64141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f637fe640 10 monclient(hunting): get_auth_request con 0x7f3f6409fb80 auth_method 0 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:44.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f63fff640 1 --2- 172.21.15.204:0/4041741537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f640a2e30 0x7f3f6411f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f63fff640 10 monclient(hunting): get_auth_request con 0x7f3f640a2e30 auth_method 0 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f63fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 10 monclient(hunting): handle_auth_done global_id 5095 payload 824 2024-09-15T01:33:44.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 1 -- 172.21.15.204:0/4041741537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3f6409fb80 msgr2=0x7f3f64141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 1 --2- 172.21.15.204:0/4041741537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3f6409fb80 0x7f3f64141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 1 -- 172.21.15.204:0/4041741537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f640a2e30 msgr2=0x7f3f6411f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 1 --2- 172.21.15.204:0/4041741537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f640a2e30 0x7f3f6411f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:44.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 1 monclient: found mon.noname-f 2024-09-15T01:33:44.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:44.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:44.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 1 -- 172.21.15.204:0/4041741537 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f58009450 con 0x7f3f640a0a40 2024-09-15T01:33:44.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f63fff640 1 --2- 172.21.15.204:0/4041741537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f640a2e30 0x7f3f6411f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:44.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.112+0000 7f3f62ffd640 1 --2- 172.21.15.204:0/4041741537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f640a0a40 0x7f3f64132700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f580097e0 tx=0x7f3f58032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:44.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f4b7fe640 1 -- 172.21.15.204:0/4041741537 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f58031820 con 0x7f3f640a0a40 2024-09-15T01:33:44.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f4b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f4b7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f4b7fe640 10 monclient: dump: 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:44.133 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f4b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f4b7fe640 1 -- 172.21.15.204:0/4041741537 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f580319a0 con 0x7f3f640a0a40 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f6a114640 5 monclient: authenticate success, global_id 5095 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f6a114640 10 monclient: _renew_subs 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f6a114640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f6a114640 1 -- 172.21.15.204:0/4041741537 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6409d650 con 0x7f3f640a0a40 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f4b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:44.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f4b7fe640 1 -- 172.21.15.204:0/4041741537 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f5803a390 con 0x7f3f640a0a40 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f4b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f4b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f4b7fe640 10 monclient: dump: 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f6a114640 10 monclient: _renew_subs 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f6a114640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f6a114640 1 -- 172.21.15.204:0/4041741537 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f64122850 con 0x7f3f640a0a40 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.113+0000 7f3f61ffb640 4 set_mon_vals no callback set 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.114+0000 7f3f4b7fe640 1 -- 172.21.15.204:0/4041741537 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3f58039040 con 0x7f3f640a0a40 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.114+0000 7f3f6a114640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.114+0000 7f3f6a114640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.114+0000 7f3f6a114640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:44.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.114+0000 7f3f6a114640 1 -- 172.21.15.204:0/4041741537 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f64070cc0 con 0x7f3f640a0a40 2024-09-15T01:33:44.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.114+0000 7f3f4b7fe640 1 --2- 172.21.15.204:0/4041741537 >> v2:172.21.15.204:6812/28210 conn(0x7f3f500397e0 0x7f3f5003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.114+0000 7f3f4b7fe640 1 -- 172.21.15.204:0/4041741537 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 3990+0+0 (secure 0 0 0) 0x7f3f5806a7e0 con 0x7f3f640a0a40 2024-09-15T01:33:44.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.114+0000 7f3f637fe640 1 --2- 172.21.15.204:0/4041741537 >> v2:172.21.15.204:6812/28210 conn(0x7f3f500397e0 0x7f3f5003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.114+0000 7f3f637fe640 10 monclient: get_auth_request con 0x7f3f500397e0 auth_method 0 2024-09-15T01:33:44.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.114+0000 7f3f637fe640 1 --2- 172.21.15.204:0/4041741537 >> v2:172.21.15.204:6812/28210 conn(0x7f3f500397e0 0x7f3f5003bc90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400bd40 tx=0x7f3f54008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:44.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.118+0000 7f3f4b7fe640 1 -- 172.21.15.204:0/4041741537 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f3f5802f8a0 con 0x7f3f640a0a40 2024-09-15T01:33:44.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.118+0000 7f3f4b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:44.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.118+0000 7f3f4b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:44.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.278+0000 7f3f6a114640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T01:33:44.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.278+0000 7f3f6a114640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T01:33:44.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.278+0000 7f3f6a114640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:44.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.278+0000 7f3f6a114640 1 -- 172.21.15.204:0/4041741537 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) v1 -- 0x7f3f640a1f10 con 0x7f3f640a0a40 2024-09-15T01:33:44.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.279+0000 7f3f4b7fe640 1 -- 172.21.15.204:0/4041741537 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v54) v1 ==== 90+0+8 (secure 0 0 0) 0x7f3f580370a0 con 0x7f3f640a0a40 2024-09-15T01:33:44.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.279+0000 7f3f4b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T01:33:44.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.279+0000 7f3f4b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:44.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.374+0000 7f3f6a114640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T01:33:44.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.374+0000 7f3f6a114640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T01:33:44.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.374+0000 7f3f6a114640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:44.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.374+0000 7f3f6a114640 1 -- 172.21.15.204:0/4041741537 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) v1 -- 0x7f3f641218f0 con 0x7f3f640a0a40 2024-09-15T01:33:44.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.374+0000 7f3f4b7fe640 1 -- 172.21.15.204:0/4041741537 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v54) v1 ==== 90+0+8 (secure 0 0 0) 0x7f3f58041070 con 0x7f3f640a0a40 2024-09-15T01:33:44.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.374+0000 7f3f4b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-09-15T01:33:44.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.374+0000 7f3f4b7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:44.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 1 -- 172.21.15.204:0/4041741537 >> v2:172.21.15.204:6812/28210 conn(0x7f3f500397e0 msgr2=0x7f3f5003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 1 --2- 172.21.15.204:0/4041741537 >> v2:172.21.15.204:6812/28210 conn(0x7f3f500397e0 0x7f3f5003bc90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400bd40 tx=0x7f3f54008040 comp rx=0 tx=0).stop 2024-09-15T01:33:44.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 10 monclient: shutdown 2024-09-15T01:33:44.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:44.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 1 -- 172.21.15.204:0/4041741537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f640a0a40 msgr2=0x7f3f64132700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 1 --2- 172.21.15.204:0/4041741537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f640a0a40 0x7f3f64132700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f580097e0 tx=0x7f3f58032000 comp rx=0 tx=0).stop 2024-09-15T01:33:44.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 1 -- 172.21.15.204:0/4041741537 shutdown_connections 2024-09-15T01:33:44.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 1 --2- 172.21.15.204:0/4041741537 >> v2:172.21.15.204:6812/28210 conn(0x7f3f500397e0 0x7f3f5003bc90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 1 --2- 172.21.15.204:0/4041741537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f640a2e30 0x7f3f6411f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 1 --2- 172.21.15.204:0/4041741537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f640a0a40 0x7f3f64132700 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 1 --2- 172.21.15.204:0/4041741537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3f6409fb80 0x7f3f64141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 1 -- 172.21.15.204:0/4041741537 >> 172.21.15.204:0/4041741537 conn(0x7f3f6406e860 msgr2=0x7f3f6409ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:44.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 1 -- 172.21.15.204:0/4041741537 shutdown_connections 2024-09-15T01:33:44.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.376+0000 7f3f6a114640 1 -- 172.21.15.204:0/4041741537 wait complete. 2024-09-15T01:33:44.387 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:44.387 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule ls-by-class hdd 2024-09-15T01:33:44.387 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep foo-hdd 2024-09-15T01:33:44.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.460+0000 7fc0cb39d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:44.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.460+0000 7fc0cb39d640 10 monclient: build_initial_monmap 2024-09-15T01:33:44.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.460+0000 7fc0cb39d640 10 monclient: monmap: 2024-09-15T01:33:44.461 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:44.461 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.462 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:44.460529+0000 2024-09-15T01:33:44.462 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:44.460529+0000 2024-09-15T01:33:44.462 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:44.462 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.462 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:44.462 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:44.462 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:44.462 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:44.462 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:44.462 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:44.462 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:44.462 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:44.463 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:44.463 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:44.463 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:44.463 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.460+0000 7fc0cb39d640 1 Processor -- start 2024-09-15T01:33:44.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.460+0000 7fc0cb39d640 1 -- start start 2024-09-15T01:33:44.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.460+0000 7fc0cb39d640 10 monclient: init 2024-09-15T01:33:44.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:44.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 10 monclient: _add_conns ranks=[7,5,6,1,4,2,10,3,0,8,9] 2024-09-15T01:33:44.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 0x7fc0c4132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 10 monclient(hunting): picked mon.noname-i con 0x7fc0c40a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:44.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc0c409fb80 0x7fc0c409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 10 monclient(hunting): picked mon.noname-h con 0x7fc0c409fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:44.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc0c40a0a40 0x7fc0c40a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 10 monclient(hunting): picked mon.noname-d con 0x7fc0c40a0a40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:44.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c409e6c0 con 0x7fc0c409fb80 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40a04b0 con 0x7fc0c40a0a40 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40a3200 con 0x7fc0c40a2e30 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0cb39d640 10 monclient(hunting): authenticate will time out at 840.103261s 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c8938640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc0c409fb80 0x7fc0c409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 0x7fc0c4132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 0x7fc0c4132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:58834/0 (socket says 172.21.15.204:58834) 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c8938640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc0c409fb80 0x7fc0c409ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:34816/0 (socket says 172.21.15.204:34816) 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 1 -- 172.21.15.204:0/328242477 learned_addr learned my addr 172.21.15.204:0/328242477 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 10 monclient(hunting): get_auth_request con 0x7fc0c40a2e30 auth_method 0 2024-09-15T01:33:44.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c8938640 10 monclient(hunting): get_auth_request con 0x7fc0c409fb80 auth_method 0 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c8938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c8938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c8938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c993a640 1 -- 172.21.15.204:0/328242477 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc0c40a0a40 msgr2=0x7fc0c40a0e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c993a640 1 --2- 172.21.15.204:0/328242477 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc0c40a0a40 0x7fc0c40a0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c8938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c8938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:44.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c8938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 10 monclient(hunting): handle_auth_done global_id 4849 payload 274 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 1 -- 172.21.15.204:0/328242477 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc0c40a0a40 msgr2=0x7fc0c40a0e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 1 --2- 172.21.15.204:0/328242477 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc0c40a0a40 0x7fc0c40a0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 1 -- 172.21.15.204:0/328242477 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc0c409fb80 msgr2=0x7fc0c409ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 1 --2- 172.21.15.204:0/328242477 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc0c409fb80 0x7fc0c409ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 10 monclient: _finish_hunting 0 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 1 monclient: found mon.noname-i 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c9139640 1 -- 172.21.15.204:0/328242477 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c404b720 con 0x7fc0c40a2e30 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.461+0000 7fc0c8938640 1 --2- 172.21.15.204:0/328242477 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc0c409fb80 0x7fc0c409ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0c9139640 1 --2- 172.21.15.204:0/328242477 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 0x7fc0c4132a60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009940 tx=0x7fc0c00091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=24dfa8d81c0dff8f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:44.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0bbfff640 1 -- 172.21.15.204:0/328242477 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0c002ac80 con 0x7fc0c40a2e30 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0bbfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0bbfff640 10 monclient: dump: 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:44.475 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0bbfff640 10 monclient: _finish_auth 0 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0bbfff640 1 -- 172.21.15.204:0/328242477 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c002ae00 con 0x7fc0c40a2e30 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0cb39d640 5 monclient: authenticate success, global_id 4849 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0cb39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0bbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0cb39d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:44.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0bbfff640 1 -- 172.21.15.204:0/328242477 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0c0031ca0 con 0x7fc0c40a2e30 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0cb39d640 4 set_mon_vals no callback set 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0bbfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0bbfff640 10 monclient: dump: 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:44.477 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0cb39d640 10 monclient: shutdown 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0cb39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.462+0000 7fc0cb39d640 1 -- 172.21.15.204:0/328242477 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 msgr2=0x7fc0c4132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 1 --2- 172.21.15.204:0/328242477 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 0x7fc0c4132a60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009940 tx=0x7fc0c00091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 1 -- 172.21.15.204:0/328242477 shutdown_connections 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 1 --2- 172.21.15.204:0/328242477 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc0c40a0a40 0x7fc0c40a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 1 --2- 172.21.15.204:0/328242477 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc0c409fb80 0x7fc0c409ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 1 --2- 172.21.15.204:0/328242477 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 0x7fc0c4132a60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 1 -- 172.21.15.204:0/328242477 >> 172.21.15.204:0/328242477 conn(0x7fc0c406e860 msgr2=0x7fc0c406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 1 -- 172.21.15.204:0/328242477 shutdown_connections 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 1 -- 172.21.15.204:0/328242477 wait complete. 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 10 monclient: build_initial_monmap 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 10 monclient: monmap: 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:44.479 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 1 Processor -- start 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 1 -- start start 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.463+0000 7fc0cb39d640 10 monclient: init 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:44.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 10 monclient: _add_conns ranks=[0,2,7,5,4,1,6,3,8] 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c409fb80 0x7fc0c4131930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 10 monclient(hunting): picked mon.noname-a con 0x7fc0c409fb80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c40a0a40 0x7fc0c411e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 10 monclient(hunting): picked mon.noname-b con 0x7fc0c40a0a40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 0x7fc0c411e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 10 monclient(hunting): picked mon.noname-i con 0x7fc0c40a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c4142a20 con 0x7fc0c409fb80 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c4121a80 con 0x7fc0c40a0a40 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c409ca40 con 0x7fc0c40a2e30 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:44.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0cb39d640 10 monclient(hunting): authenticate will time out at 840.106569s 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 0x7fc0c411e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 0x7fc0c411e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:58844/0 (socket says 172.21.15.204:58844) 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 1 -- 172.21.15.204:0/76509620 learned_addr learned my addr 172.21.15.204:0/76509620 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 10 monclient(hunting): get_auth_request con 0x7fc0c40a2e30 auth_method 0 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 10 monclient(hunting): handle_auth_done global_id 4858 payload 824 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 1 -- 172.21.15.204:0/76509620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c40a0a40 msgr2=0x7fc0c411e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 1 --2- 172.21.15.204:0/76509620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c40a0a40 0x7fc0c411e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 1 -- 172.21.15.204:0/76509620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c409fb80 msgr2=0x7fc0c4131930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 1 --2- 172.21.15.204:0/76509620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c409fb80 0x7fc0c4131930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 10 monclient: _finish_hunting 0 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 1 monclient: found mon.noname-i 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.464+0000 7fc0c993a640 1 -- 172.21.15.204:0/76509620 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b4009390 con 0x7fc0c40a2e30 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0c993a640 1 --2- 172.21.15.204:0/76509620 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 0x7fc0c411e9b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00bf80 tx=0x7fc0bc009730 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0b97fa640 1 -- 172.21.15.204:0/76509620 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0bc00cdf0 con 0x7fc0c40a2e30 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0b97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0b97fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0b97fa640 10 monclient: dump: 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:44.483 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:44.484 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0b97fa640 10 monclient: _finish_auth 0 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0b97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0b97fa640 1 -- 172.21.15.204:0/76509620 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc0065d0 con 0x7fc0c40a2e30 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0cb39d640 5 monclient: authenticate success, global_id 4858 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0b97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0b97fa640 1 -- 172.21.15.204:0/76509620 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0bc0153e0 con 0x7fc0c40a2e30 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0cb39d640 10 monclient: _renew_subs 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0cb39d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0cb39d640 1 -- 172.21.15.204:0/76509620 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c0009450 con 0x7fc0c40a2e30 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0b97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0bb7fe640 4 set_mon_vals no callback set 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0b97fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:44.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0b97fa640 10 monclient: dump: 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:44.486 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0cb39d640 10 monclient: _renew_subs 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0cb39d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.465+0000 7fc0cb39d640 1 -- 172.21.15.204:0/76509620 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c4120730 con 0x7fc0c40a2e30 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.466+0000 7fc0b97fa640 1 -- 172.21.15.204:0/76509620 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc0bc012020 con 0x7fc0c40a2e30 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.466+0000 7fc0cb39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.466+0000 7fc0cb39d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.466+0000 7fc0cb39d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.466+0000 7fc0cb39d640 1 -- 172.21.15.204:0/76509620 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c4070cc0 con 0x7fc0c40a2e30 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.466+0000 7fc0b97fa640 1 --2- 172.21.15.204:0/76509620 >> v2:172.21.15.204:6812/28210 conn(0x7fc0a80397b0 0x7fc0a803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.466+0000 7fc0b97fa640 1 -- 172.21.15.204:0/76509620 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 3990+0+0 (secure 0 0 0) 0x7fc0bc04b570 con 0x7fc0c40a2e30 2024-09-15T01:33:44.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.466+0000 7fc0c9139640 1 --2- 172.21.15.204:0/76509620 >> v2:172.21.15.204:6812/28210 conn(0x7fc0a80397b0 0x7fc0a803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.466+0000 7fc0c9139640 10 monclient: get_auth_request con 0x7fc0a80397b0 auth_method 0 2024-09-15T01:33:44.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.466+0000 7fc0c9139640 1 --2- 172.21.15.204:0/76509620 >> v2:172.21.15.204:6812/28210 conn(0x7fc0a80397b0 0x7fc0a803bc60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0034000 tx=0x7fc0c0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:44.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.470+0000 7fc0b97fa640 1 -- 172.21.15.204:0/76509620 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc0bc006150 con 0x7fc0c40a2e30 2024-09-15T01:33:44.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.470+0000 7fc0b97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:44.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.470+0000 7fc0b97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:44.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.631+0000 7fc0cb39d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T01:33:44.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.631+0000 7fc0cb39d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T01:33:44.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.631+0000 7fc0cb39d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:44.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.631+0000 7fc0cb39d640 1 -- 172.21.15.204:0/76509620 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) v1 -- 0x7fc0c40a1f10 con 0x7fc0c40a2e30 2024-09-15T01:33:44.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.631+0000 7fc0b97fa640 1 -- 172.21.15.204:0/76509620 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v54) v1 ==== 90+0+8 (secure 0 0 0) 0x7fc0bc0072f0 con 0x7fc0c40a2e30 2024-09-15T01:33:44.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.631+0000 7fc0b97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T01:33:44.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.631+0000 7fc0b97fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:44.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.728+0000 7fc0cb39d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T01:33:44.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.728+0000 7fc0cb39d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T01:33:44.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.728+0000 7fc0cb39d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:44.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.728+0000 7fc0cb39d640 1 -- 172.21.15.204:0/76509620 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) v1 -- 0x7fc0c4066880 con 0x7fc0c40a2e30 2024-09-15T01:33:44.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.729+0000 7fc0b97fa640 1 -- 172.21.15.204:0/76509620 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v54) v1 ==== 90+0+8 (secure 0 0 0) 0x7fc0bc0078f0 con 0x7fc0c40a2e30 2024-09-15T01:33:44.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.729+0000 7fc0b97fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T01:33:44.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.729+0000 7fc0b97fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:44.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 1 -- 172.21.15.204:0/76509620 >> v2:172.21.15.204:6812/28210 conn(0x7fc0a80397b0 msgr2=0x7fc0a803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 1 --2- 172.21.15.204:0/76509620 >> v2:172.21.15.204:6812/28210 conn(0x7fc0a80397b0 0x7fc0a803bc60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0034000 tx=0x7fc0c0032000 comp rx=0 tx=0).stop 2024-09-15T01:33:44.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 10 monclient: shutdown 2024-09-15T01:33:44.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:44.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 1 -- 172.21.15.204:0/76509620 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 msgr2=0x7fc0c411e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 1 --2- 172.21.15.204:0/76509620 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 0x7fc0c411e9b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00bf80 tx=0x7fc0bc009730 comp rx=0 tx=0).stop 2024-09-15T01:33:44.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 1 -- 172.21.15.204:0/76509620 shutdown_connections 2024-09-15T01:33:44.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 1 --2- 172.21.15.204:0/76509620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc0c40a0a40 0x7fc0c411e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 1 --2- 172.21.15.204:0/76509620 >> v2:172.21.15.204:6812/28210 conn(0x7fc0a80397b0 0x7fc0a803bc60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 1 --2- 172.21.15.204:0/76509620 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0c40a2e30 0x7fc0c411e9b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 1 --2- 172.21.15.204:0/76509620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0c409fb80 0x7fc0c4131930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 1 -- 172.21.15.204:0/76509620 >> 172.21.15.204:0/76509620 conn(0x7fc0c406e860 msgr2=0x7fc0c409bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:44.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 1 -- 172.21.15.204:0/76509620 shutdown_connections 2024-09-15T01:33:44.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.731+0000 7fc0cb39d640 1 -- 172.21.15.204:0/76509620 wait complete. 2024-09-15T01:33:44.741 INFO:tasks.workunit.client.0.smithi204.stdout:foo-hdd 2024-09-15T01:33:44.742 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule ls-by-class hdd 2024-09-15T01:33:44.742 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false grep foo-ssd 2024-09-15T01:33:44.742 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:44.742 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep foo-ssd 2024-09-15T01:33:44.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.815+0000 7f8f4eb1d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:44.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.815+0000 7f8f4eb1d640 10 monclient: build_initial_monmap 2024-09-15T01:33:44.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.815+0000 7f8f4eb1d640 10 monclient: monmap: 2024-09-15T01:33:44.817 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:44.817 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.817 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:44.815649+0000 2024-09-15T01:33:44.817 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:44.815649+0000 2024-09-15T01:33:44.817 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:44.817 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.817 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:44.817 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:44.817 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:44.817 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:44.817 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:44.818 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:44.818 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:44.818 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:44.818 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:44.818 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:44.818 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:44.818 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.815+0000 7f8f4eb1d640 1 Processor -- start 2024-09-15T01:33:44.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.815+0000 7f8f4eb1d640 1 -- start start 2024-09-15T01:33:44.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.815+0000 7f8f4eb1d640 10 monclient: init 2024-09-15T01:33:44.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:44.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 10 monclient: _add_conns ranks=[4,3,8,10,7,0,9,5,2,1,6] 2024-09-15T01:33:44.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8f48132d60 0x7f8f48133130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 10 monclient(hunting): picked mon.noname-c con 0x7f8f48132d60 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:44.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f4806ee90 0x7f8f4809e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 10 monclient(hunting): picked mon.noname-g con 0x7f8f4806ee90 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:44.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f48133670 0x7f8f4809ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 10 monclient(hunting): picked mon.noname-e con 0x7f8f48133670 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:44.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48068480 con 0x7f8f4806ee90 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480a00e0 con 0x7f8f48132d60 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48068ff0 con 0x7f8f48133670 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4eb1d640 10 monclient(hunting): authenticate will time out at 840.458463s 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f47fff640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f4806ee90 msgr2=0x7f8f4809e850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f47fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f4806ee90 0x7f8f4809e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4c8b9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8f48132d60 0x7f8f48133130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4d0ba640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f48133670 0x7f8f4809ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4c8b9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8f48132d60 0x7f8f48133130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:38844/0 (socket says 172.21.15.204:38844) 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4c8b9640 1 -- 172.21.15.204:0/768721892 learned_addr learned my addr 172.21.15.204:0/768721892 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:44.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4c8b9640 10 monclient(hunting): get_auth_request con 0x7f8f48132d60 auth_method 0 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4c8b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4c8b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4c8b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4d0ba640 10 monclient(hunting): get_auth_request con 0x7f8f48133670 auth_method 0 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4d0ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4d0ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4d0ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4c8b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4c8b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4c8b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4d0ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4d0ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4d0ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:44.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.816+0000 7f8f4c8b9640 10 monclient(hunting): handle_auth_done global_id 4783 payload 274 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4c8b9640 1 -- 172.21.15.204:0/768721892 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f48133670 msgr2=0x7f8f4809ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4c8b9640 1 --2- 172.21.15.204:0/768721892 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f48133670 0x7f8f4809ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4c8b9640 1 -- 172.21.15.204:0/768721892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f4806ee90 msgr2=0x7f8f4809e850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4c8b9640 1 --2- 172.21.15.204:0/768721892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f4806ee90 0x7f8f4809e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4c8b9640 10 monclient: _finish_hunting 0 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4c8b9640 1 monclient: found mon.noname-c 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4c8b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4c8b9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4c8b9640 1 -- 172.21.15.204:0/768721892 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4804b720 con 0x7f8f48132d60 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4d0ba640 1 --2- 172.21.15.204:0/768721892 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f48133670 0x7f8f4809ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4c8b9640 1 --2- 172.21.15.204:0/768721892 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8f48132d60 0x7f8f48133130 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009970 tx=0x7f8f3c0091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=208b08598d76db0d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f477fe640 1 -- 172.21.15.204:0/768721892 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8f3c02ac70 con 0x7f8f48132d60 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:44.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f477fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f477fe640 10 monclient: dump: 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:44.830 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f477fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f477fe640 1 -- 172.21.15.204:0/768721892 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c02adf0 con 0x7f8f48132d60 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4eb1d640 5 monclient: authenticate success, global_id 4783 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4eb1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f477fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4eb1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f4eb1d640 4 set_mon_vals no callback set 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f477fe640 1 -- 172.21.15.204:0/768721892 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8f3c031ca0 con 0x7f8f48132d60 2024-09-15T01:33:44.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.817+0000 7f8f477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f477fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f477fe640 10 monclient: dump: 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:44.832 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:44.833 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:44.833 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:44.833 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:44.833 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:44.833 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f4eb1d640 10 monclient: shutdown 2024-09-15T01:33:44.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f4eb1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:44.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/768721892 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8f48132d60 msgr2=0x7f8f48133130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f4eb1d640 1 --2- 172.21.15.204:0/768721892 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8f48132d60 0x7f8f48133130 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009970 tx=0x7f8f3c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/768721892 shutdown_connections 2024-09-15T01:33:44.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f4eb1d640 1 --2- 172.21.15.204:0/768721892 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f48133670 0x7f8f4809ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f4eb1d640 1 --2- 172.21.15.204:0/768721892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f4806ee90 0x7f8f4809e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f4eb1d640 1 --2- 172.21.15.204:0/768721892 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8f48132d60 0x7f8f48133130 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/768721892 >> 172.21.15.204:0/768721892 conn(0x7f8f48067770 msgr2=0x7f8f480a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:44.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/768721892 shutdown_connections 2024-09-15T01:33:44.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/768721892 wait complete. 2024-09-15T01:33:44.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.818+0000 7f8f4eb1d640 10 monclient: build_initial_monmap 2024-09-15T01:33:44.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 10 monclient: monmap: 2024-09-15T01:33:44.834 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:44.834 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:44.834 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:44.834 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:44.834 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:44.834 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:44.835 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:44.835 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:44.835 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:44.835 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:44.835 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:44.835 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:44.835 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:44.835 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:44.835 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:44.835 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 1 Processor -- start 2024-09-15T01:33:44.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 1 -- start start 2024-09-15T01:33:44.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 10 monclient: init 2024-09-15T01:33:44.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:44.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 10 monclient: _add_conns ranks=[7,5,3,2,0,8,1,6,4] 2024-09-15T01:33:44.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8f4806ee90 0x7f8f4811e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 10 monclient(hunting): picked mon.noname-i con 0x7f8f4806ee90 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:44.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8f48132d60 0x7f8f4811cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 10 monclient(hunting): picked mon.noname-h con 0x7f8f48132d60 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:44.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f48133670 0x7f8f4812fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 10 monclient(hunting): picked mon.noname-g con 0x7f8f48133670 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:44.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4811dc90 con 0x7f8f48133670 2024-09-15T01:33:44.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f481204d0 con 0x7f8f48132d60 2024-09-15T01:33:44.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:44.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8f481201f0 con 0x7f8f4806ee90 2024-09-15T01:33:44.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:44.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.819+0000 7f8f4eb1d640 10 monclient(hunting): authenticate will time out at 840.461982s 2024-09-15T01:33:44.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4d0ba640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f48133670 msgr2=0x7f8f4812fae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:44.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4d0ba640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f48133670 0x7f8f4812fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:44.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4c8b9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8f4806ee90 0x7f8f4811e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8f48132d60 0x7f8f4811cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4c8b9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8f4806ee90 0x7f8f4811e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:58854/0 (socket says 172.21.15.204:58854) 2024-09-15T01:33:44.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4c8b9640 1 -- 172.21.15.204:0/3205478155 learned_addr learned my addr 172.21.15.204:0/3205478155 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4c8b9640 10 monclient(hunting): get_auth_request con 0x7f8f4806ee90 auth_method 0 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4c8b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4c8b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4c8b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 10 monclient(hunting): get_auth_request con 0x7f8f48132d60 auth_method 0 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4c8b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4c8b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4c8b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:44.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 10 monclient(hunting): handle_auth_done global_id 4847 payload 824 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 1 -- 172.21.15.204:0/3205478155 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8f4806ee90 msgr2=0x7f8f4811e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 1 --2- 172.21.15.204:0/3205478155 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8f4806ee90 0x7f8f4811e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 1 -- 172.21.15.204:0/3205478155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f48133670 msgr2=0x7f8f4812fae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 1 --2- 172.21.15.204:0/3205478155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f48133670 0x7f8f4812fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 1 monclient: found mon.noname-h 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 1 -- 172.21.15.204:0/3205478155 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f40009a20 con 0x7f8f48132d60 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4c8b9640 1 --2- 172.21.15.204:0/3205478155 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8f4806ee90 0x7f8f4811e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f47fff640 1 --2- 172.21.15.204:0/3205478155 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8f48132d60 0x7f8f4811cec0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f4809ae50 tx=0x7f8f34009060 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f44ff9640 1 -- 172.21.15.204:0/3205478155 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8f34006150 con 0x7f8f48132d60 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f44ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:44.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f44ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f44ff9640 10 monclient: dump: 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:44.840 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:44.841 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:44.841 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:44.841 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f44ff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:44.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f44ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:44.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f44ff9640 1 -- 172.21.15.204:0/3205478155 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f340062f0 con 0x7f8f48132d60 2024-09-15T01:33:44.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4eb1d640 5 monclient: authenticate success, global_id 4847 2024-09-15T01:33:44.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4eb1d640 10 monclient: _renew_subs 2024-09-15T01:33:44.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4eb1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:44.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/3205478155 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c009450 con 0x7f8f48132d60 2024-09-15T01:33:44.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f44ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:44.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f44ff9640 1 -- 172.21.15.204:0/3205478155 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8f34010360 con 0x7f8f48132d60 2024-09-15T01:33:44.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f44ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:44.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f44ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:44.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f44ff9640 10 monclient: dump: 2024-09-15T01:33:44.842 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:44.842 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:44.842 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.843 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:44.843 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:44.843 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:44.843 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:44.843 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:44.843 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:44.843 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:44.844 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:44.844 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:44.844 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:44.844 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:44.844 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:44.844 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:44.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4eb1d640 10 monclient: _renew_subs 2024-09-15T01:33:44.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f46ffd640 4 set_mon_vals no callback set 2024-09-15T01:33:44.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4eb1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:44.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.820+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/3205478155 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4811d400 con 0x7f8f48132d60 2024-09-15T01:33:44.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.821+0000 7f8f44ff9640 1 -- 172.21.15.204:0/3205478155 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8f34003b50 con 0x7f8f48132d60 2024-09-15T01:33:44.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.821+0000 7f8f4eb1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:44.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.821+0000 7f8f4eb1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:44.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.821+0000 7f8f4eb1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:44.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.821+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/3205478155 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4811e3a0 con 0x7f8f48132d60 2024-09-15T01:33:44.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.821+0000 7f8f44ff9640 1 --2- 172.21.15.204:0/3205478155 >> v2:172.21.15.204:6812/28210 conn(0x7f8f2c0397b0 0x7f8f2c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:44.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.821+0000 7f8f44ff9640 1 -- 172.21.15.204:0/3205478155 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 3990+0+0 (secure 0 0 0) 0x7f8f3404a190 con 0x7f8f48132d60 2024-09-15T01:33:44.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.821+0000 7f8f4c8b9640 1 --2- 172.21.15.204:0/3205478155 >> v2:172.21.15.204:6812/28210 conn(0x7f8f2c0397b0 0x7f8f2c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:44.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.821+0000 7f8f4c8b9640 10 monclient: get_auth_request con 0x7f8f2c0397b0 auth_method 0 2024-09-15T01:33:44.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.822+0000 7f8f4c8b9640 1 --2- 172.21.15.204:0/3205478155 >> v2:172.21.15.204:6812/28210 conn(0x7f8f2c0397b0 0x7f8f2c03bc60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009210 tx=0x7f8f3c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:44.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.826+0000 7f8f44ff9640 1 -- 172.21.15.204:0/3205478155 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8f4811e3a0 con 0x7f8f48132d60 2024-09-15T01:33:44.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.826+0000 7f8f44ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:44.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.826+0000 7f8f44ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:44.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.982+0000 7f8f4eb1d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T01:33:44.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.982+0000 7f8f4eb1d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T01:33:44.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.982+0000 7f8f4eb1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:44.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.982+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/3205478155 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) v1 -- 0x7f8f4811e3a0 con 0x7f8f48132d60 2024-09-15T01:33:44.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.982+0000 7f8f44ff9640 1 -- 172.21.15.204:0/3205478155 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v54) v1 ==== 90+0+8 (secure 0 0 0) 0x7f8f4811e3a0 con 0x7f8f48132d60 2024-09-15T01:33:44.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.982+0000 7f8f44ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T01:33:44.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:44.982+0000 7f8f44ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:45.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.077+0000 7f8f4eb1d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T01:33:45.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.077+0000 7f8f4eb1d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T01:33:45.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.077+0000 7f8f4eb1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:45.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.077+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/3205478155 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) v1 -- 0x7f8f48120610 con 0x7f8f48132d60 2024-09-15T01:33:45.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.077+0000 7f8f44ff9640 1 -- 172.21.15.204:0/3205478155 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v54) v1 ==== 90+0+8 (secure 0 0 0) 0x7f8f48120610 con 0x7f8f48132d60 2024-09-15T01:33:45.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.077+0000 7f8f44ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-09-15T01:33:45.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.077+0000 7f8f44ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:45.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/3205478155 >> v2:172.21.15.204:6812/28210 conn(0x7f8f2c0397b0 msgr2=0x7f8f2c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:45.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 1 --2- 172.21.15.204:0/3205478155 >> v2:172.21.15.204:6812/28210 conn(0x7f8f2c0397b0 0x7f8f2c03bc60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009210 tx=0x7f8f3c032000 comp rx=0 tx=0).stop 2024-09-15T01:33:45.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 10 monclient: shutdown 2024-09-15T01:33:45.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:45.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/3205478155 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8f48132d60 msgr2=0x7f8f4811cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:45.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 1 --2- 172.21.15.204:0/3205478155 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8f48132d60 0x7f8f4811cec0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f4809ae50 tx=0x7f8f34009060 comp rx=0 tx=0).stop 2024-09-15T01:33:45.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/3205478155 shutdown_connections 2024-09-15T01:33:45.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 1 --2- 172.21.15.204:0/3205478155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f48133670 0x7f8f4812fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 1 --2- 172.21.15.204:0/3205478155 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8f48132d60 0x7f8f4811cec0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 1 --2- 172.21.15.204:0/3205478155 >> v2:172.21.15.204:6812/28210 conn(0x7f8f2c0397b0 0x7f8f2c03bc60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 1 --2- 172.21.15.204:0/3205478155 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8f4806ee90 0x7f8f4811e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/3205478155 >> 172.21.15.204:0/3205478155 conn(0x7f8f48067770 msgr2=0x7f8f480a2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:45.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/3205478155 shutdown_connections 2024-09-15T01:33:45.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.079+0000 7f8f4eb1d640 1 -- 172.21.15.204:0/3205478155 wait complete. 2024-09-15T01:33:45.089 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:45.089 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd erasure-code-profile set ec-foo-ssd crush-device-class=ssd m=2 k=2 2024-09-15T01:33:45.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.164+0000 7fde16ac9640 10 monclient: get_monmap_and_config 2024-09-15T01:33:45.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.164+0000 7fde16ac9640 10 monclient: build_initial_monmap 2024-09-15T01:33:45.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.164+0000 7fde16ac9640 10 monclient: monmap: 2024-09-15T01:33:45.166 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:45.166 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:45.166 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:45.165421+0000 2024-09-15T01:33:45.166 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:45.165421+0000 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:45.167 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:45.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.165+0000 7fde16ac9640 1 Processor -- start 2024-09-15T01:33:45.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.165+0000 7fde16ac9640 1 -- start start 2024-09-15T01:33:45.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.165+0000 7fde16ac9640 10 monclient: init 2024-09-15T01:33:45.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:45.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 10 monclient: _add_conns ranks=[10,3,2,4,9,7,8,5,0,1,6] 2024-09-15T01:33:45.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fde100a2cd0 0x7fde10132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 10 monclient(hunting): picked mon.noname-f con 0x7fde100a2cd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:45.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde1009fa20 0x7fde1009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 10 monclient(hunting): picked mon.noname-g con 0x7fde1009fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:45.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde100a08e0 0x7fde100a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 10 monclient(hunting): picked mon.noname-b con 0x7fde100a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde100a30a0 con 0x7fde100a2cd0 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde100a0330 con 0x7fde100a08e0 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde100a1db0 con 0x7fde1009fa20 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde16ac9640 10 monclient(hunting): authenticate will time out at 840.808353s 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde0ffff640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde1009fa20 msgr2=0x7fde1009fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde0ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde1009fa20 0x7fde1009fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fde100a2cd0 0x7fde10132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:45.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fde100a2cd0 0x7fde10132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55314/0 (socket says 172.21.15.204:55314) 2024-09-15T01:33:45.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 1 -- 172.21.15.204:0/230069585 learned_addr learned my addr 172.21.15.204:0/230069585 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:45.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 10 monclient(hunting): get_auth_request con 0x7fde100a2cd0 auth_method 0 2024-09-15T01:33:45.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:45.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:45.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:45.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:45.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:45.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:45.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 10 monclient(hunting): handle_auth_done global_id 5104 payload 274 2024-09-15T01:33:45.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 1 -- 172.21.15.204:0/230069585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde1009fa20 msgr2=0x7fde1009fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:45.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 1 --2- 172.21.15.204:0/230069585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde1009fa20 0x7fde1009fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 1 -- 172.21.15.204:0/230069585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde100a08e0 msgr2=0x7fde100a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 1 --2- 172.21.15.204:0/230069585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde100a08e0 0x7fde100a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 10 monclient: _finish_hunting 0 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 1 monclient: found mon.noname-f 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.166+0000 7fde14865640 1 -- 172.21.15.204:0/230069585 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde100648a0 con 0x7fde100a2cd0 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde14865640 1 --2- 172.21.15.204:0/230069585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde100a2cd0 0x7fde10132c00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009910 tx=0x7fddf80091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40bcd7bb58fde00 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde0f7fe640 1 -- 172.21.15.204:0/230069585 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fddf802acb0 con 0x7fde100a2cd0 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde0f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde0f7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde0f7fe640 10 monclient: dump: 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:45.179 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde0f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:45.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde0f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde16ac9640 5 monclient: authenticate success, global_id 5104 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde16ac9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde0f7fe640 1 -- 172.21.15.204:0/230069585 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf802ae30 con 0x7fde100a2cd0 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde0f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde16ac9640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde16ac9640 4 set_mon_vals no callback set 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde0f7fe640 1 -- 172.21.15.204:0/230069585 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fddf8031ca0 con 0x7fde100a2cd0 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde0f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde0f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.167+0000 7fde0f7fe640 10 monclient: dump: 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:45.181 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:45.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 10 monclient: shutdown 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 1 -- 172.21.15.204:0/230069585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde100a2cd0 msgr2=0x7fde10132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 1 --2- 172.21.15.204:0/230069585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde100a2cd0 0x7fde10132c00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009910 tx=0x7fddf80091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 1 -- 172.21.15.204:0/230069585 shutdown_connections 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 1 --2- 172.21.15.204:0/230069585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde100a08e0 0x7fde100a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 1 --2- 172.21.15.204:0/230069585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde1009fa20 0x7fde1009fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 1 --2- 172.21.15.204:0/230069585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde100a2cd0 0x7fde10132c00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 1 -- 172.21.15.204:0/230069585 >> 172.21.15.204:0/230069585 conn(0x7fde1006e860 msgr2=0x7fde1006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 1 -- 172.21.15.204:0/230069585 shutdown_connections 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 1 -- 172.21.15.204:0/230069585 wait complete. 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 10 monclient: build_initial_monmap 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 10 monclient: monmap: 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:45.183 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:45.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.168+0000 7fde16ac9640 1 Processor -- start 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 1 -- start start 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 10 monclient: init 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 10 monclient: _add_conns ranks=[3,6,0,5,4,8,2,7,1] 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde1009fa20 0x7fde101317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 10 monclient(hunting): picked mon.noname-g con 0x7fde1009fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fde100a08e0 0x7fde1011d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 10 monclient(hunting): picked mon.noname-d con 0x7fde100a08e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde100a2cd0 0x7fde1011d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 10 monclient(hunting): picked mon.noname-a con 0x7fde100a2cd0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1009cb60 con 0x7fde100a2cd0 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:45.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde10120c70 con 0x7fde1009fa20 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fde101350c0 con 0x7fde100a08e0 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde16ac9640 10 monclient(hunting): authenticate will time out at 840.811626s 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde14865640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde1009fa20 msgr2=0x7fde101317c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde14865640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde1009fa20 0x7fde101317c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde0ffff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fde100a08e0 msgr2=0x7fde1011d260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde0ffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fde100a08e0 0x7fde1011d260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde15066640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde100a2cd0 0x7fde1011d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde15066640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde100a2cd0 0x7fde1011d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:37410/0 (socket says 172.21.15.204:37410) 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde15066640 1 -- 172.21.15.204:0/1217886848 learned_addr learned my addr 172.21.15.204:0/1217886848 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde15066640 10 monclient(hunting): get_auth_request con 0x7fde100a2cd0 auth_method 0 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde15066640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:45.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde15066640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde15066640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde15066640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde15066640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.169+0000 7fde15066640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde15066640 10 monclient(hunting): handle_auth_done global_id 4905 payload 824 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde15066640 1 -- 172.21.15.204:0/1217886848 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fde100a08e0 msgr2=0x7fde1011d260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde15066640 1 --2- 172.21.15.204:0/1217886848 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fde100a08e0 0x7fde1011d260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde15066640 1 -- 172.21.15.204:0/1217886848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde1009fa20 msgr2=0x7fde101317c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde15066640 1 --2- 172.21.15.204:0/1217886848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde1009fa20 0x7fde101317c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde15066640 10 monclient: _finish_hunting 0 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde15066640 1 monclient: found mon.noname-a 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde15066640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde15066640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde15066640 1 -- 172.21.15.204:0/1217886848 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf8009450 con 0x7fde100a2cd0 2024-09-15T01:33:45.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde15066640 1 --2- 172.21.15.204:0/1217886848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde100a2cd0 0x7fde1011d7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde0800bc30 tx=0x7fde08009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0cff9640 1 -- 172.21.15.204:0/1217886848 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde080092e0 con 0x7fde100a2cd0 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0cff9640 10 monclient: dump: 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:45.188 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0cff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0cff9640 1 -- 172.21.15.204:0/1217886848 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0800cce0 con 0x7fde100a2cd0 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde16ac9640 5 monclient: authenticate success, global_id 4905 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde16ac9640 10 monclient: _renew_subs 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde16ac9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde16ac9640 1 -- 172.21.15.204:0/1217886848 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde10120610 con 0x7fde100a2cd0 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:45.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0cff9640 1 -- 172.21.15.204:0/1217886848 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde0800ce90 con 0x7fde100a2cd0 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0cff9640 10 monclient: dump: 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:45.190 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde0effd640 4 set_mon_vals no callback set 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde16ac9640 10 monclient: _renew_subs 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde16ac9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.170+0000 7fde16ac9640 1 -- 172.21.15.204:0/1217886848 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde10120900 con 0x7fde100a2cd0 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.171+0000 7fde0cff9640 1 -- 172.21.15.204:0/1217886848 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fde08012070 con 0x7fde100a2cd0 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.171+0000 7fde16ac9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.171+0000 7fde16ac9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.171+0000 7fde16ac9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.171+0000 7fde16ac9640 1 -- 172.21.15.204:0/1217886848 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde10070b80 con 0x7fde100a2cd0 2024-09-15T01:33:45.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.171+0000 7fde0cff9640 1 --2- 172.21.15.204:0/1217886848 >> v2:172.21.15.204:6812/28210 conn(0x7fddf00397b0 0x7fddf003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.171+0000 7fde0cff9640 1 -- 172.21.15.204:0/1217886848 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 3990+0+0 (secure 0 0 0) 0x7fde0804aca0 con 0x7fde100a2cd0 2024-09-15T01:33:45.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.172+0000 7fde14865640 1 --2- 172.21.15.204:0/1217886848 >> v2:172.21.15.204:6812/28210 conn(0x7fddf00397b0 0x7fddf003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:45.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.172+0000 7fde14865640 10 monclient: get_auth_request con 0x7fddf00397b0 auth_method 0 2024-09-15T01:33:45.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.172+0000 7fde14865640 1 --2- 172.21.15.204:0/1217886848 >> v2:172.21.15.204:6812/28210 conn(0x7fddf00397b0 0x7fddf003bc60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fddf8034000 tx=0x7fddf8032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:45.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.176+0000 7fde0cff9640 1 -- 172.21.15.204:0/1217886848 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fde0804f070 con 0x7fde100a2cd0 2024-09-15T01:33:45.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.176+0000 7fde0cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:45.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.176+0000 7fde0cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:45.341 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.339+0000 7fde16ac9640 10 start_mon_command cmd=[{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-09-15T01:33:45.341 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.339+0000 7fde16ac9640 10 monclient: _send_command 2 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-09-15T01:33:45.341 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.339+0000 7fde16ac9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:45.341 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.339+0000 7fde16ac9640 1 -- 172.21.15.204:0/1217886848 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]} v 0) v1 -- 0x7fde1011ee40 con 0x7fde100a2cd0 2024-09-15T01:33:45.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.590+0000 7fde0cff9640 1 -- 172.21.15.204:0/1217886848 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}]=0 v55) v1 ==== 151+0+0 (secure 0 0 0) 0x7fde0801ee60 con 0x7fde100a2cd0 2024-09-15T01:33:45.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.590+0000 7fde0cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-09-15T01:33:45.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.590+0000 7fde0cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.693+0000 7fde16ac9640 10 start_mon_command cmd=[{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-09-15T01:33:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.693+0000 7fde16ac9640 10 monclient: _send_command 3 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-09-15T01:33:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.693+0000 7fde16ac9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.693+0000 7fde16ac9640 1 -- 172.21.15.204:0/1217886848 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]} v 0) v1 -- 0x7fde100008d0 con 0x7fde100a2cd0 2024-09-15T01:33:45.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.694+0000 7fde0cff9640 1 -- 172.21.15.204:0/1217886848 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}]=0 v55) v1 ==== 151+0+0 (secure 0 0 0) 0x7fde0800c860 con 0x7fde100a2cd0 2024-09-15T01:33:45.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.694+0000 7fde0cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-09-15T01:33:45.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.694+0000 7fde0cff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:45.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 1 -- 172.21.15.204:0/1217886848 >> v2:172.21.15.204:6812/28210 conn(0x7fddf00397b0 msgr2=0x7fddf003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:45.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 1 --2- 172.21.15.204:0/1217886848 >> v2:172.21.15.204:6812/28210 conn(0x7fddf00397b0 0x7fddf003bc60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fddf8034000 tx=0x7fddf8032000 comp rx=0 tx=0).stop 2024-09-15T01:33:45.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 10 monclient: shutdown 2024-09-15T01:33:45.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:45.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 1 -- 172.21.15.204:0/1217886848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde100a2cd0 msgr2=0x7fde1011d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:45.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 1 --2- 172.21.15.204:0/1217886848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde100a2cd0 0x7fde1011d7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde0800bc30 tx=0x7fde08009840 comp rx=0 tx=0).stop 2024-09-15T01:33:45.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 1 -- 172.21.15.204:0/1217886848 shutdown_connections 2024-09-15T01:33:45.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 1 --2- 172.21.15.204:0/1217886848 >> v2:172.21.15.204:6812/28210 conn(0x7fddf00397b0 0x7fddf003bc60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 1 --2- 172.21.15.204:0/1217886848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde100a2cd0 0x7fde1011d7a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 1 --2- 172.21.15.204:0/1217886848 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fde100a08e0 0x7fde1011d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 1 --2- 172.21.15.204:0/1217886848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fde1009fa20 0x7fde101317c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 1 -- 172.21.15.204:0/1217886848 >> 172.21.15.204:0/1217886848 conn(0x7fde1006e860 msgr2=0x7fde100a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:45.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 1 -- 172.21.15.204:0/1217886848 shutdown_connections 2024-09-15T01:33:45.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.697+0000 7fde16ac9640 1 -- 172.21.15.204:0/1217886848 wait complete. 2024-09-15T01:33:45.707 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool create ec-foo 2 erasure ec-foo-ssd 2024-09-15T01:33:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.780+0000 7f17d1d54640 10 monclient: get_monmap_and_config 2024-09-15T01:33:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.780+0000 7f17d1d54640 10 monclient: build_initial_monmap 2024-09-15T01:33:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.780+0000 7f17d1d54640 10 monclient: monmap: 2024-09-15T01:33:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:45.782332+0000 2024-09-15T01:33:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:45.782332+0000 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:45.784 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:45.785 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:45.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.781+0000 7f17d1d54640 1 Processor -- start 2024-09-15T01:33:45.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.781+0000 7f17d1d54640 1 -- start start 2024-09-15T01:33:45.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.781+0000 7f17d1d54640 10 monclient: init 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 10 monclient: _add_conns ranks=[2,10,7,5,1,6,9,4,0,3,8] 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17cc0a2cd0 0x7f17cc132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 10 monclient(hunting): picked mon.noname-b con 0x7f17cc0a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f17cc09fa20 0x7f17cc09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 10 monclient(hunting): picked mon.noname-f con 0x7f17cc09fa20 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f17cc0a08e0 0x7f17cc0a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 10 monclient(hunting): picked mon.noname-i con 0x7f17cc0a08e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17cc0a30a0 con 0x7f17cc09fa20 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0a0330 con 0x7f17cc0a2cd0 2024-09-15T01:33:45.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0a1db0 con 0x7f17cc0a08e0 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17d1d54640 10 monclient(hunting): authenticate will time out at 841.425132s 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17caffd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f17cc09fa20 0x7f17cc09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f17cc0a08e0 0x7f17cc0a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17caffd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f17cc09fa20 0x7f17cc09fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55330/0 (socket says 172.21.15.204:55330) 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f17cc0a08e0 0x7f17cc0a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:58862/0 (socket says 172.21.15.204:58862) 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17caffd640 1 -- 172.21.15.204:0/2602808685 learned_addr learned my addr 172.21.15.204:0/2602808685 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17caffd640 10 monclient(hunting): get_auth_request con 0x7f17cc09fa20 auth_method 0 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17caffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17caffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:45.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 10 monclient(hunting): get_auth_request con 0x7f17cc0a08e0 auth_method 0 2024-09-15T01:33:45.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:45.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:45.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:45.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17caffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:45.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17caffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:45.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17caffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:45.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:45.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:45.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:45.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cb7fe640 1 --2- 172.21.15.204:0/2602808685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17cc0a2cd0 0x7f17cc132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:45.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cb7fe640 10 monclient(hunting): get_auth_request con 0x7f17cc0a2cd0 auth_method 0 2024-09-15T01:33:45.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:45.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:45.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:45.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 10 monclient(hunting): handle_auth_done global_id 4876 payload 274 2024-09-15T01:33:45.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 1 -- 172.21.15.204:0/2602808685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17cc0a2cd0 msgr2=0x7f17cc132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:45.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 1 --2- 172.21.15.204:0/2602808685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17cc0a2cd0 0x7f17cc132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 1 -- 172.21.15.204:0/2602808685 >> v2:172.21.15.204:3300/0 conn(0x7f17cc09fa20 msgr2=0x7f17cc09fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:45.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 1 --2- 172.21.15.204:0/2602808685 >> v2:172.21.15.204:3300/0 conn(0x7f17cc09fa20 0x7f17cc09fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:45.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 1 monclient: found mon.noname-i 2024-09-15T01:33:45.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:45.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:45.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17cbfff640 1 -- 172.21.15.204:0/2602808685 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc0648a0 con 0x7f17cc0a08e0 2024-09-15T01:33:45.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.782+0000 7f17caffd640 1 --2- 172.21.15.204:0/2602808685 >> v2:172.21.15.204:3300/0 conn(0x7f17cc09fa20 0x7f17cc09fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:45.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17cbfff640 1 --2- 172.21.15.204:0/2602808685 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f17cc0a08e0 0x7f17cc0a0d50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009e60 tx=0x7f17c4009780 comp rx=0 tx=0).ready entity=mon.7 client_cookie=791fde2351e42540 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:45.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17ca7fc640 1 -- 172.21.15.204:0/2602808685 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17c40079d0 con 0x7f17cc0a08e0 2024-09-15T01:33:45.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:45.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17ca7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:45.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17ca7fc640 10 monclient: dump: 2024-09-15T01:33:45.797 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:45.797 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:45.797 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.797 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.797 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:45.797 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17ca7fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:45.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17d1d54640 5 monclient: authenticate success, global_id 4876 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17ca7fc640 1 -- 172.21.15.204:0/2602808685 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c402d050 con 0x7f17cc0a08e0 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17d1d54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17ca7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17d1d54640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17d1d54640 4 set_mon_vals no callback set 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17ca7fc640 1 -- 172.21.15.204:0/2602808685 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17c4007bd0 con 0x7f17cc0a08e0 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17ca7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17ca7fc640 10 monclient: dump: 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.799 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:45.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17d1d54640 10 monclient: shutdown 2024-09-15T01:33:45.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17d1d54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:45.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17d1d54640 1 -- 172.21.15.204:0/2602808685 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f17cc0a08e0 msgr2=0x7f17cc0a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:45.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.783+0000 7f17d1d54640 1 --2- 172.21.15.204:0/2602808685 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f17cc0a08e0 0x7f17cc0a0d50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009e60 tx=0x7f17c4009780 comp rx=0 tx=0).stop 2024-09-15T01:33:45.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.784+0000 7f17d1d54640 1 -- 172.21.15.204:0/2602808685 shutdown_connections 2024-09-15T01:33:45.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.784+0000 7f17d1d54640 1 --2- 172.21.15.204:0/2602808685 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f17cc0a08e0 0x7f17cc0a0d50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.784+0000 7f17d1d54640 1 --2- 172.21.15.204:0/2602808685 >> v2:172.21.15.204:3300/0 conn(0x7f17cc09fa20 0x7f17cc09fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.784+0000 7f17d1d54640 1 --2- 172.21.15.204:0/2602808685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17cc0a2cd0 0x7f17cc132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.784+0000 7f17d1d54640 1 -- 172.21.15.204:0/2602808685 >> 172.21.15.204:0/2602808685 conn(0x7f17cc06e860 msgr2=0x7f17cc06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:45.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.784+0000 7f17d1d54640 1 -- 172.21.15.204:0/2602808685 shutdown_connections 2024-09-15T01:33:45.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.784+0000 7f17d1d54640 1 -- 172.21.15.204:0/2602808685 wait complete. 2024-09-15T01:33:45.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.784+0000 7f17d1d54640 10 monclient: build_initial_monmap 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.784+0000 7f17d1d54640 10 monclient: monmap: 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:45.802 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.784+0000 7f17d1d54640 1 Processor -- start 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 1 -- start start 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 10 monclient: init 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 10 monclient: _add_conns ranks=[1,5,0,7,3,8,2,6,4] 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17cc09fa20 0x7f17cc11d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 10 monclient(hunting): picked mon.noname-f con 0x7f17cc09fa20 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f17cc0a08e0 0x7f17cc11d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 10 monclient(hunting): picked mon.noname-h con 0x7f17cc0a08e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:45.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0a2cd0 0x7f17cc11dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 10 monclient(hunting): picked mon.noname-a con 0x7f17cc0a2cd0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:45.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:45.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc11f100 con 0x7f17cc0a2cd0 2024-09-15T01:33:45.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:45.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc09d3d0 con 0x7f17cc09fa20 2024-09-15T01:33:45.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:45.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc09cb60 con 0x7f17cc0a08e0 2024-09-15T01:33:45.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:45.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17d1d54640 10 monclient(hunting): authenticate will time out at 841.428480s 2024-09-15T01:33:45.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17cc09fa20 0x7f17cc11d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:45.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17caffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f17cc0a08e0 0x7f17cc11d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:45.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17cc09fa20 0x7f17cc11d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55336/0 (socket says 172.21.15.204:55336) 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 1 -- 172.21.15.204:0/1519078223 learned_addr learned my addr 172.21.15.204:0/1519078223 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 10 monclient(hunting): get_auth_request con 0x7f17cc09fa20 auth_method 0 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17caffd640 10 monclient(hunting): get_auth_request con 0x7f17cc0a08e0 auth_method 0 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17caffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17caffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17caffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cbfff640 1 --2- 172.21.15.204:0/1519078223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0a2cd0 0x7f17cc11dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:45.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17caffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17caffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17caffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cbfff640 10 monclient(hunting): get_auth_request con 0x7f17cc0a2cd0 auth_method 0 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 10 monclient(hunting): handle_auth_done global_id 5122 payload 824 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 1 -- 172.21.15.204:0/1519078223 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f17cc0a08e0 msgr2=0x7f17cc11d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 1 --2- 172.21.15.204:0/1519078223 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f17cc0a08e0 0x7f17cc11d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 1 -- 172.21.15.204:0/1519078223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0a2cd0 msgr2=0x7f17cc11dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 1 --2- 172.21.15.204:0/1519078223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0a2cd0 0x7f17cc11dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 1 monclient: found mon.noname-f 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17cb7fe640 1 -- 172.21.15.204:0/1519078223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c0009450 con 0x7f17cc09fa20 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.785+0000 7f17caffd640 1 --2- 172.21.15.204:0/1519078223 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f17cc0a08e0 0x7f17cc11d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17cb7fe640 1 --2- 172.21.15.204:0/1519078223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17cc09fa20 0x7f17cc11d200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00d890 tx=0x7f17bc008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17b37fe640 1 -- 172.21.15.204:0/1519078223 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17bc00de20 con 0x7f17cc09fa20 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17b37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17b37fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17b37fe640 10 monclient: dump: 2024-09-15T01:33:45.807 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:45.808 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:45.809 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:45.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17b37fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:45.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17b37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:45.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17b37fe640 1 -- 172.21.15.204:0/1519078223 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc009d30 con 0x7f17cc09fa20 2024-09-15T01:33:45.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17d1d54640 5 monclient: authenticate success, global_id 5122 2024-09-15T01:33:45.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17b37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:45.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17b37fe640 1 -- 172.21.15.204:0/1519078223 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17bc009ea0 con 0x7f17cc09fa20 2024-09-15T01:33:45.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17d1d54640 10 monclient: _renew_subs 2024-09-15T01:33:45.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17d1d54640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:45.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17d1d54640 1 -- 172.21.15.204:0/1519078223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c4009a20 con 0x7f17cc09fa20 2024-09-15T01:33:45.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17c9ffb640 4 set_mon_vals no callback set 2024-09-15T01:33:45.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17b37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17b37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17b37fe640 10 monclient: dump: 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:45.810 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17d1d54640 10 monclient: _renew_subs 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17d1d54640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.786+0000 7f17d1d54640 1 -- 172.21.15.204:0/1519078223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc1326f0 con 0x7f17cc09fa20 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.787+0000 7f17b37fe640 1 -- 172.21.15.204:0/1519078223 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f17bc0098b0 con 0x7f17cc09fa20 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.787+0000 7f17d1d54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.787+0000 7f17d1d54640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.787+0000 7f17d1d54640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.787+0000 7f17d1d54640 1 -- 172.21.15.204:0/1519078223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17cc070b80 con 0x7f17cc09fa20 2024-09-15T01:33:45.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.787+0000 7f17b37fe640 1 --2- 172.21.15.204:0/1519078223 >> v2:172.21.15.204:6812/28210 conn(0x7f17b80397e0 0x7f17b803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:45.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.787+0000 7f17b37fe640 1 -- 172.21.15.204:0/1519078223 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4215+0+0 (secure 0 0 0) 0x7f17bc04b8b0 con 0x7f17cc09fa20 2024-09-15T01:33:45.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.787+0000 7f17caffd640 1 --2- 172.21.15.204:0/1519078223 >> v2:172.21.15.204:6812/28210 conn(0x7f17b80397e0 0x7f17b803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:45.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.787+0000 7f17caffd640 10 monclient: get_auth_request con 0x7f17b80397e0 auth_method 0 2024-09-15T01:33:45.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.787+0000 7f17caffd640 1 --2- 172.21.15.204:0/1519078223 >> v2:172.21.15.204:6812/28210 conn(0x7f17b80397e0 0x7f17b803bc90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f17c0006c90 tx=0x7f17c00091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:45.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.791+0000 7f17b37fe640 1 -- 172.21.15.204:0/1519078223 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f17bc025580 con 0x7f17cc09fa20 2024-09-15T01:33:45.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.791+0000 7f17b37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:45.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.791+0000 7f17b37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:45.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.944+0000 7f17d1d54640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-09-15T01:33:45.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.945+0000 7f17d1d54640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-09-15T01:33:45.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.945+0000 7f17d1d54640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:45.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:45.945+0000 7f17d1d54640 1 -- 172.21.15.204:0/1519078223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"} v 0) v1 -- 0x7f17cc11eb80 con 0x7f17cc09fa20 2024-09-15T01:33:47.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.605+0000 7f17b37fe640 1 -- 172.21.15.204:0/1519078223 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}]=0 pool 'ec-foo' created v57) v1 ==== 177+0+0 (secure 0 0 0) 0x7f17bc0180e0 con 0x7f17cc09fa20 2024-09-15T01:33:47.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.606+0000 7f17b37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-09-15T01:33:47.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.606+0000 7f17b37fe640 10 monclient: _finish_command 2 = system:0 pool 'ec-foo' created 2024-09-15T01:33:47.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.702+0000 7f17d1d54640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-09-15T01:33:47.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.702+0000 7f17d1d54640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-09-15T01:33:47.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.702+0000 7f17d1d54640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:47.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.702+0000 7f17d1d54640 1 -- 172.21.15.204:0/1519078223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"} v 0) v1 -- 0x7f17cc09a0c0 con 0x7f17cc09fa20 2024-09-15T01:33:47.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.703+0000 7f17b37fe640 1 -- 172.21.15.204:0/1519078223 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}]=0 pool 'ec-foo' already exists v57) v1 ==== 184+0+0 (secure 0 0 0) 0x7f17bc013b60 con 0x7f17cc09fa20 2024-09-15T01:33:47.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.703+0000 7f17b37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-09-15T01:33:47.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.703+0000 7f17b37fe640 10 monclient: _finish_command 3 = system:0 pool 'ec-foo' already exists 2024-09-15T01:33:47.706 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'ec-foo' already exists 2024-09-15T01:33:47.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 1 -- 172.21.15.204:0/1519078223 >> v2:172.21.15.204:6812/28210 conn(0x7f17b80397e0 msgr2=0x7f17b803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:47.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 1 --2- 172.21.15.204:0/1519078223 >> v2:172.21.15.204:6812/28210 conn(0x7f17b80397e0 0x7f17b803bc90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f17c0006c90 tx=0x7f17c00091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 10 monclient: shutdown 2024-09-15T01:33:47.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:47.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 1 -- 172.21.15.204:0/1519078223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17cc09fa20 msgr2=0x7f17cc11d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:47.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 1 --2- 172.21.15.204:0/1519078223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17cc09fa20 0x7f17cc11d200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00d890 tx=0x7f17bc008040 comp rx=0 tx=0).stop 2024-09-15T01:33:47.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 1 -- 172.21.15.204:0/1519078223 shutdown_connections 2024-09-15T01:33:47.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 1 --2- 172.21.15.204:0/1519078223 >> v2:172.21.15.204:6812/28210 conn(0x7f17b80397e0 0x7f17b803bc90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 1 --2- 172.21.15.204:0/1519078223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17cc0a2cd0 0x7f17cc11dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 1 --2- 172.21.15.204:0/1519078223 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f17cc0a08e0 0x7f17cc11d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 1 --2- 172.21.15.204:0/1519078223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17cc09fa20 0x7f17cc11d200 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 1 -- 172.21.15.204:0/1519078223 >> 172.21.15.204:0/1519078223 conn(0x7f17cc06e860 msgr2=0x7f17cc0a4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:47.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 1 -- 172.21.15.204:0/1519078223 shutdown_connections 2024-09-15T01:33:47.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.705+0000 7f17d1d54640 1 -- 172.21.15.204:0/1519078223 wait complete. 2024-09-15T01:33:47.717 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool rm ec-foo ec-foo --yes-i-really-really-mean-it 2024-09-15T01:33:47.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.791+0000 7f317bd1d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:47.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.791+0000 7f317bd1d640 10 monclient: build_initial_monmap 2024-09-15T01:33:47.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.792+0000 7f317bd1d640 10 monclient: monmap: 2024-09-15T01:33:47.794 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:47.794 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:47.794 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:47.793485+0000 2024-09-15T01:33:47.795 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:47.793485+0000 2024-09-15T01:33:47.795 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:47.795 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:47.795 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:47.795 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:47.795 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:47.795 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:47.795 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:47.795 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:47.795 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:47.795 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:47.795 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:47.796 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:47.796 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:47.796 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:47.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.792+0000 7f317bd1d640 1 Processor -- start 2024-09-15T01:33:47.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.792+0000 7f317bd1d640 1 -- start start 2024-09-15T01:33:47.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.792+0000 7f317bd1d640 10 monclient: init 2024-09-15T01:33:47.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:47.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 10 monclient: _add_conns ranks=[5,10,3,6,8,9,7,2,4,0,1] 2024-09-15T01:33:47.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a2cd0 0x7f3174132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:47.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 10 monclient(hunting): picked mon.noname-h con 0x7f31740a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:47.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f317409fa20 0x7f317409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:47.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 10 monclient(hunting): picked mon.noname-f con 0x7f317409fa20 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:47.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31740a08e0 0x7f31740a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:47.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 10 monclient(hunting): picked mon.noname-g con 0x7f31740a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:47.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:47.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31740a30a0 con 0x7f317409fa20 2024-09-15T01:33:47.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:47.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31740a0330 con 0x7f31740a08e0 2024-09-15T01:33:47.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31740a1db0 con 0x7f31740a2cd0 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317bd1d640 10 monclient(hunting): authenticate will time out at 843.436292s 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317a2ba640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31740a08e0 msgr2=0x7f31740a0d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f317a2ba640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31740a08e0 0x7f31740a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a2cd0 0x7f3174132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f31792b8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f317409fa20 0x7f317409fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a2cd0 0x7f3174132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49514/0 (socket says 172.21.15.204:49514) 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 1 -- 172.21.15.204:0/2448364919 learned_addr learned my addr 172.21.15.204:0/2448364919 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 10 monclient(hunting): get_auth_request con 0x7f31740a2cd0 auth_method 0 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:47.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f31792b8640 10 monclient(hunting): get_auth_request con 0x7f317409fa20 auth_method 0 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f31792b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f31792b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f31792b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f31792b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f31792b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f31792b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 10 monclient(hunting): handle_auth_done global_id 4865 payload 274 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 1 -- 172.21.15.204:0/2448364919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31740a08e0 msgr2=0x7f31740a0d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:47.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 1 --2- 172.21.15.204:0/2448364919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31740a08e0 0x7f31740a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 1 -- 172.21.15.204:0/2448364919 >> v2:172.21.15.204:3300/0 conn(0x7f317409fa20 msgr2=0x7f317409fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 1 --2- 172.21.15.204:0/2448364919 >> v2:172.21.15.204:3300/0 conn(0x7f317409fa20 0x7f317409fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 10 monclient: _finish_hunting 0 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 1 monclient: found mon.noname-h 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f3179ab9640 1 -- 172.21.15.204:0/2448364919 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31740648a0 con 0x7f31740a2cd0 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.793+0000 7f31792b8640 1 --2- 172.21.15.204:0/2448364919 >> v2:172.21.15.204:3300/0 conn(0x7f317409fa20 0x7f317409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3179ab9640 1 --2- 172.21.15.204:0/2448364919 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a2cd0 0x7f3174132c00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3170009910 tx=0x7f31700091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=6a828ea118bb5309 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3178ab7640 1 -- 172.21.15.204:0/2448364919 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f317002acb0 con 0x7f31740a2cd0 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3178ab7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3178ab7640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:47.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3178ab7640 10 monclient: dump: 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:47.809 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3178ab7640 10 monclient: _finish_auth 0 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3178ab7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f317bd1d640 5 monclient: authenticate success, global_id 4865 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3178ab7640 1 -- 172.21.15.204:0/2448364919 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317002ae30 con 0x7f31740a2cd0 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f317bd1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3178ab7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f317bd1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f317bd1d640 4 set_mon_vals no callback set 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3178ab7640 1 -- 172.21.15.204:0/2448364919 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3170031ca0 con 0x7f31740a2cd0 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3178ab7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:47.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3178ab7640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.794+0000 7f3178ab7640 10 monclient: dump: 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:47.811 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 10 monclient: shutdown 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 1 -- 172.21.15.204:0/2448364919 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a2cd0 msgr2=0x7f3174132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 1 --2- 172.21.15.204:0/2448364919 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a2cd0 0x7f3174132c00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3170009910 tx=0x7f31700091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 1 -- 172.21.15.204:0/2448364919 shutdown_connections 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 1 --2- 172.21.15.204:0/2448364919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31740a08e0 0x7f31740a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 1 --2- 172.21.15.204:0/2448364919 >> v2:172.21.15.204:3300/0 conn(0x7f317409fa20 0x7f317409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 1 --2- 172.21.15.204:0/2448364919 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a2cd0 0x7f3174132c00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 1 -- 172.21.15.204:0/2448364919 >> 172.21.15.204:0/2448364919 conn(0x7f317406e860 msgr2=0x7f317406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:47.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 1 -- 172.21.15.204:0/2448364919 shutdown_connections 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 1 -- 172.21.15.204:0/2448364919 wait complete. 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 10 monclient: build_initial_monmap 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.795+0000 7f317bd1d640 10 monclient: monmap: 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:47.813 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 1 Processor -- start 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 1 -- start start 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 10 monclient: init 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 10 monclient: _add_conns ranks=[3,5,6,8,1,4,2,7,0] 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f317409fa20 0x7f3174131000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 10 monclient(hunting): picked mon.noname-g con 0x7f317409fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:47.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a08e0 0x7f317411db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 10 monclient(hunting): picked mon.noname-h con 0x7f31740a08e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f31740a2cd0 0x7f317411e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 10 monclient(hunting): picked mon.noname-d con 0x7f31740a2cd0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f3179ab9640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f317409fa20 msgr2=0x7f3174131000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317409d440 con 0x7f317409fa20 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f317409cb60 con 0x7f31740a08e0 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3174131da0 con 0x7f31740a2cd0 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317bd1d640 10 monclient(hunting): authenticate will time out at 843.439566s 2024-09-15T01:33:47.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f3179ab9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f317409fa20 0x7f3174131000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a08e0 0x7f317411db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a08e0 0x7f317411db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49526/0 (socket says 172.21.15.204:49526) 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 1 -- 172.21.15.204:0/2588137554 learned_addr learned my addr 172.21.15.204:0/2588137554 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 10 monclient(hunting): get_auth_request con 0x7f31740a08e0 auth_method 0 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317a2ba640 1 -- 172.21.15.204:0/2588137554 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f31740a2cd0 msgr2=0x7f317411e0d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f317a2ba640 1 --2- 172.21.15.204:0/2588137554 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f31740a2cd0 0x7f317411e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:47.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 10 monclient(hunting): handle_auth_done global_id 4874 payload 824 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 1 -- 172.21.15.204:0/2588137554 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f31740a2cd0 msgr2=0x7f317411e0d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 1 --2- 172.21.15.204:0/2588137554 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f31740a2cd0 0x7f317411e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 1 -- 172.21.15.204:0/2588137554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f317409fa20 msgr2=0x7f3174131000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 1 --2- 172.21.15.204:0/2588137554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f317409fa20 0x7f3174131000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 10 monclient: _finish_hunting 0 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 1 monclient: found mon.noname-h 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.796+0000 7f31792b8640 1 -- 172.21.15.204:0/2588137554 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3164009390 con 0x7f31740a08e0 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f31792b8640 1 --2- 172.21.15.204:0/2588137554 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a08e0 0x7f317411db90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f316400ecb0 tx=0x7f31640090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f3169ffb640 1 -- 172.21.15.204:0/2588137554 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3164013470 con 0x7f31740a08e0 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f3169ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f3169ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:47.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f3169ffb640 10 monclient: dump: 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:47.818 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f3169ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f3169ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f3169ffb640 1 -- 172.21.15.204:0/2588137554 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31640135f0 con 0x7f31740a08e0 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f317bd1d640 5 monclient: authenticate success, global_id 4874 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f3169ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f3169ffb640 1 -- 172.21.15.204:0/2588137554 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3164013be0 con 0x7f31740a08e0 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f3169ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f3169ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f3169ffb640 10 monclient: dump: 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:47.819 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:47.820 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f316bfff640 4 set_mon_vals no callback set 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f317bd1d640 10 monclient: _renew_subs 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f317bd1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f317bd1d640 1 -- 172.21.15.204:0/2588137554 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3170009450 con 0x7f31740a08e0 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f317bd1d640 10 monclient: _renew_subs 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f317bd1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.797+0000 7f317bd1d640 1 -- 172.21.15.204:0/2588137554 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3174120260 con 0x7f31740a08e0 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.798+0000 7f3169ffb640 1 -- 172.21.15.204:0/2588137554 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3164013d40 con 0x7f31740a08e0 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.798+0000 7f3169ffb640 1 --2- 172.21.15.204:0/2588137554 >> v2:172.21.15.204:6812/28210 conn(0x7f31580397b0 0x7f315803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.798+0000 7f3169ffb640 1 -- 172.21.15.204:0/2588137554 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4656+0+0 (secure 0 0 0) 0x7f316404baf0 con 0x7f31740a08e0 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.798+0000 7f317bd1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.798+0000 7f317bd1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:47.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.798+0000 7f317bd1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:47.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.798+0000 7f3179ab9640 1 --2- 172.21.15.204:0/2588137554 >> v2:172.21.15.204:6812/28210 conn(0x7f31580397b0 0x7f315803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:47.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.798+0000 7f317bd1d640 1 -- 172.21.15.204:0/2588137554 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3174070b80 con 0x7f31740a08e0 2024-09-15T01:33:47.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.798+0000 7f3179ab9640 10 monclient: get_auth_request con 0x7f31580397b0 auth_method 0 2024-09-15T01:33:47.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.798+0000 7f3179ab9640 1 --2- 172.21.15.204:0/2588137554 >> v2:172.21.15.204:6812/28210 conn(0x7f31580397b0 0x7f315803bc60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3170034000 tx=0x7f3170032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:47.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.803+0000 7f3169ffb640 1 -- 172.21.15.204:0/2588137554 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f3164013a40 con 0x7f31740a08e0 2024-09-15T01:33:47.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.803+0000 7f3169ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:47.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.803+0000 7f3169ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:47.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.956+0000 7f317bd1d640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:47.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.956+0000 7f317bd1d640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:47.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.956+0000 7f317bd1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:47.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:47.956+0000 7f317bd1d640 1 -- 172.21.15.204:0/2588137554 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f31740008d0 con 0x7f31740a08e0 2024-09-15T01:33:48.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.611+0000 7f3169ffb640 1 -- 172.21.15.204:0/2588137554 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}]=0 pool 'ec-foo' removed v58) v1 ==== 154+0+0 (secure 0 0 0) 0x7f316401b3b0 con 0x7f31740a08e0 2024-09-15T01:33:48.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.611+0000 7f3169ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:48.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.611+0000 7f3169ffb640 10 monclient: _finish_command 2 = system:0 pool 'ec-foo' removed 2024-09-15T01:33:48.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.708+0000 7f317bd1d640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:48.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.708+0000 7f317bd1d640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:48.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.708+0000 7f317bd1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:48.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.708+0000 7f317bd1d640 1 -- 172.21.15.204:0/2588137554 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3174133e20 con 0x7f31740a08e0 2024-09-15T01:33:48.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.708+0000 7f3169ffb640 1 -- 172.21.15.204:0/2588137554 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}]=0 pool 'ec-foo' does not exist v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7f316401b590 con 0x7f31740a08e0 2024-09-15T01:33:48.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.708+0000 7f3169ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:48.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.708+0000 7f3169ffb640 10 monclient: _finish_command 3 = system:0 pool 'ec-foo' does not exist 2024-09-15T01:33:48.711 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'ec-foo' does not exist 2024-09-15T01:33:48.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 1 -- 172.21.15.204:0/2588137554 >> v2:172.21.15.204:6812/28210 conn(0x7f31580397b0 msgr2=0x7f315803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:48.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 1 --2- 172.21.15.204:0/2588137554 >> v2:172.21.15.204:6812/28210 conn(0x7f31580397b0 0x7f315803bc60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3170034000 tx=0x7f3170032000 comp rx=0 tx=0).stop 2024-09-15T01:33:48.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 10 monclient: shutdown 2024-09-15T01:33:48.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:48.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 1 -- 172.21.15.204:0/2588137554 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a08e0 msgr2=0x7f317411db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:48.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 1 --2- 172.21.15.204:0/2588137554 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a08e0 0x7f317411db90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f316400ecb0 tx=0x7f31640090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 1 -- 172.21.15.204:0/2588137554 shutdown_connections 2024-09-15T01:33:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 1 --2- 172.21.15.204:0/2588137554 >> v2:172.21.15.204:6812/28210 conn(0x7f31580397b0 0x7f315803bc60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 1 --2- 172.21.15.204:0/2588137554 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f31740a2cd0 0x7f317411e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 1 --2- 172.21.15.204:0/2588137554 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31740a08e0 0x7f317411db90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 1 --2- 172.21.15.204:0/2588137554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f317409fa20 0x7f3174131000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 1 -- 172.21.15.204:0/2588137554 >> 172.21.15.204:0/2588137554 conn(0x7f317406e860 msgr2=0x7f31740a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 1 -- 172.21.15.204:0/2588137554 shutdown_connections 2024-09-15T01:33:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.710+0000 7f317bd1d640 1 -- 172.21.15.204:0/2588137554 wait complete. 2024-09-15T01:33:48.722 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule ls 2024-09-15T01:33:48.722 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep foo 2024-09-15T01:33:48.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.794+0000 7fe754b5d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:48.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.794+0000 7fe754b5d640 10 monclient: build_initial_monmap 2024-09-15T01:33:48.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.794+0000 7fe754b5d640 10 monclient: monmap: 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:48.795657+0000 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:48.795657+0000 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:48.797 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:48.798 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:48.798 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:48.798 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:48.798 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:48.798 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:48.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.794+0000 7fe754b5d640 1 Processor -- start 2024-09-15T01:33:48.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.794+0000 7fe754b5d640 1 -- start start 2024-09-15T01:33:48.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.794+0000 7fe754b5d640 10 monclient: init 2024-09-15T01:33:48.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:48.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 10 monclient: _add_conns ranks=[5,8,2,1,9,0,7,6,4,10,3] 2024-09-15T01:33:48.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe75006e980 0x7fe750130c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:48.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 10 monclient(hunting): picked mon.noname-h con 0x7fe75006e980 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:48.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe750099a10 0x7fe750067130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:48.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 10 monclient(hunting): picked mon.noname-e con 0x7fe750099a10 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:48.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe750067670 0x7fe750067ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:48.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 10 monclient(hunting): picked mon.noname-b con 0x7fe750067670 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:48.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:48.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7501311a0 con 0x7fe750067670 2024-09-15T01:33:48.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:48.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe750068020 con 0x7fe75006e980 2024-09-15T01:33:48.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe7501316d0 con 0x7fe750099a10 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe754b5d640 10 monclient(hunting): authenticate will time out at 844.438354s 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe75006e980 0x7fe750130c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe75006e980 0x7fe750130c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49540/0 (socket says 172.21.15.204:49540) 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 1 -- 172.21.15.204:0/1372197676 learned_addr learned my addr 172.21.15.204:0/1372197676 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 10 monclient(hunting): get_auth_request con 0x7fe75006e980 auth_method 0 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:48.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:48.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe745d9b640 1 --2- 172.21.15.204:0/1372197676 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe750099a10 0x7fe750067130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:48.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74ed9d640 1 --2- 172.21.15.204:0/1372197676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe750067670 0x7fe750067ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:48.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe745d9b640 10 monclient(hunting): get_auth_request con 0x7fe750099a10 auth_method 0 2024-09-15T01:33:48.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe745d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:48.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe745d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:48.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe745d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:48.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74ed9d640 10 monclient(hunting): get_auth_request con 0x7fe750067670 auth_method 0 2024-09-15T01:33:48.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:48.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:48.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:48.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 10 monclient(hunting): handle_auth_done global_id 4883 payload 274 2024-09-15T01:33:48.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 1 -- 172.21.15.204:0/1372197676 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe750099a10 msgr2=0x7fe750067130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 1 --2- 172.21.15.204:0/1372197676 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe750099a10 0x7fe750067130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 1 -- 172.21.15.204:0/1372197676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe750067670 msgr2=0x7fe750067ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 1 --2- 172.21.15.204:0/1372197676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe750067670 0x7fe750067ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:33:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 1 monclient: found mon.noname-h 2024-09-15T01:33:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74e59c640 1 -- 172.21.15.204:0/1372197676 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75004b720 con 0x7fe75006e980 2024-09-15T01:33:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe745d9b640 1 --2- 172.21.15.204:0/1372197676 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe750099a10 0x7fe750067130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.795+0000 7fe74ed9d640 1 --2- 172.21.15.204:0/1372197676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe750067670 0x7fe750067ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74e59c640 1 --2- 172.21.15.204:0/1372197676 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe75006e980 0x7fe750130c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe740009970 tx=0x7fe7400091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=cf28827f1c801ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74dd9b640 1 -- 172.21.15.204:0/1372197676 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe74002ac50 con 0x7fe75006e980 2024-09-15T01:33:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74dd9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74dd9b640 10 monclient: dump: 2024-09-15T01:33:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:48.814 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74dd9b640 10 monclient: _finish_auth 0 2024-09-15T01:33:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74dd9b640 1 -- 172.21.15.204:0/1372197676 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74002add0 con 0x7fe75006e980 2024-09-15T01:33:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74dd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74dd9b640 1 -- 172.21.15.204:0/1372197676 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe740031ca0 con 0x7fe75006e980 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe754b5d640 5 monclient: authenticate success, global_id 4883 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe754b5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe754b5d640 4 set_mon_vals no callback set 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74dd9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.796+0000 7fe74dd9b640 10 monclient: dump: 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:48.816 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 10 monclient: shutdown 2024-09-15T01:33:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 1 -- 172.21.15.204:0/1372197676 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe75006e980 msgr2=0x7fe750130c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 1 --2- 172.21.15.204:0/1372197676 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe75006e980 0x7fe750130c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe740009970 tx=0x7fe7400091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 1 -- 172.21.15.204:0/1372197676 shutdown_connections 2024-09-15T01:33:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 1 --2- 172.21.15.204:0/1372197676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe750067670 0x7fe750067ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 1 --2- 172.21.15.204:0/1372197676 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe750099a10 0x7fe750067130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 1 --2- 172.21.15.204:0/1372197676 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe75006e980 0x7fe750130c60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 1 -- 172.21.15.204:0/1372197676 >> 172.21.15.204:0/1372197676 conn(0x7fe7500687e0 msgr2=0x7fe7500a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 1 -- 172.21.15.204:0/1372197676 shutdown_connections 2024-09-15T01:33:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 1 -- 172.21.15.204:0/1372197676 wait complete. 2024-09-15T01:33:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 10 monclient: build_initial_monmap 2024-09-15T01:33:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.797+0000 7fe754b5d640 10 monclient: monmap: 2024-09-15T01:33:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:48.819 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 1 Processor -- start 2024-09-15T01:33:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 1 -- start start 2024-09-15T01:33:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 10 monclient: init 2024-09-15T01:33:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 10 monclient: _add_conns ranks=[0,4,2,1,3,8,6,5,7] 2024-09-15T01:33:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe750067670 0x7fe75012f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 10 monclient(hunting): picked mon.noname-a con 0x7fe750067670 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe75006e980 0x7fe75011c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 10 monclient(hunting): picked mon.noname-c con 0x7fe75006e980 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe750099a10 0x7fe75011c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 10 monclient(hunting): picked mon.noname-b con 0x7fe750099a10 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75009ded0 con 0x7fe750067670 2024-09-15T01:33:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75011fba0 con 0x7fe750099a10 2024-09-15T01:33:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe75006f030 con 0x7fe75006e980 2024-09-15T01:33:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe754b5d640 10 monclient(hunting): authenticate will time out at 844.441681s 2024-09-15T01:33:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe745d9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe75006e980 0x7fe75011c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe74ed9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe750099a10 0x7fe75011c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe74e59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe750067670 0x7fe75012f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe745d9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe75006e980 0x7fe75011c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:38320/0 (socket says 172.21.15.204:38320) 2024-09-15T01:33:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe745d9b640 1 -- 172.21.15.204:0/918300842 learned_addr learned my addr 172.21.15.204:0/918300842 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe74ed9d640 10 monclient(hunting): get_auth_request con 0x7fe750099a10 auth_method 0 2024-09-15T01:33:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe74ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe74ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe74ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe745d9b640 10 monclient(hunting): get_auth_request con 0x7fe75006e980 auth_method 0 2024-09-15T01:33:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe745d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe745d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe745d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe74e59c640 10 monclient(hunting): get_auth_request con 0x7fe750067670 auth_method 0 2024-09-15T01:33:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe74e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe74e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe74e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.798+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe745d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe745d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe745d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 10 monclient(hunting): handle_auth_done global_id 4923 payload 824 2024-09-15T01:33:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 1 -- 172.21.15.204:0/918300842 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe75006e980 msgr2=0x7fe75011c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 1 --2- 172.21.15.204:0/918300842 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe75006e980 0x7fe75011c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 1 -- 172.21.15.204:0/918300842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe750099a10 msgr2=0x7fe75011c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 1 --2- 172.21.15.204:0/918300842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe750099a10 0x7fe75011c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:33:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 1 monclient: found mon.noname-a 2024-09-15T01:33:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:33:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 1 -- 172.21.15.204:0/918300842 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe740009450 con 0x7fe750067670 2024-09-15T01:33:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe745d9b640 1 --2- 172.21.15.204:0/918300842 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe75006e980 0x7fe75011c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74ed9d640 1 --2- 172.21.15.204:0/918300842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe750099a10 0x7fe75011c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74e59c640 1 --2- 172.21.15.204:0/918300842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe750067670 0x7fe75012f770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe740009940 tx=0x7fe740032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe746ffd640 1 -- 172.21.15.204:0/918300842 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe740006b30 con 0x7fe750067670 2024-09-15T01:33:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe746ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe746ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:33:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe746ffd640 10 monclient: dump: 2024-09-15T01:33:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:48.826 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe746ffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe746ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe746ffd640 1 -- 172.21.15.204:0/918300842 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74002ac50 con 0x7fe750067670 2024-09-15T01:33:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe754b5d640 5 monclient: authenticate success, global_id 4923 2024-09-15T01:33:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe746ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe754b5d640 10 monclient: _renew_subs 2024-09-15T01:33:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe746ffd640 1 -- 172.21.15.204:0/918300842 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe740031a60 con 0x7fe750067670 2024-09-15T01:33:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe754b5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe754b5d640 1 -- 172.21.15.204:0/918300842 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75011f8c0 con 0x7fe750067670 2024-09-15T01:33:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe746ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe746ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:33:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe746ffd640 10 monclient: dump: 2024-09-15T01:33:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:48.829 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe74d59a640 4 set_mon_vals no callback set 2024-09-15T01:33:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe754b5d640 10 monclient: _renew_subs 2024-09-15T01:33:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe754b5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.799+0000 7fe754b5d640 1 -- 172.21.15.204:0/918300842 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75011eb00 con 0x7fe750067670 2024-09-15T01:33:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.800+0000 7fe746ffd640 1 -- 172.21.15.204:0/918300842 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe740039040 con 0x7fe750067670 2024-09-15T01:33:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.800+0000 7fe754b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.800+0000 7fe754b5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.800+0000 7fe754b5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.800+0000 7fe746ffd640 1 --2- 172.21.15.204:0/918300842 >> v2:172.21.15.204:6812/28210 conn(0x7fe7300397b0 0x7fe73003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.800+0000 7fe754b5d640 1 -- 172.21.15.204:0/918300842 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe750070cc0 con 0x7fe750067670 2024-09-15T01:33:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.800+0000 7fe746ffd640 1 -- 172.21.15.204:0/918300842 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4301+0+0 (secure 0 0 0) 0x7fe74006a9d0 con 0x7fe750067670 2024-09-15T01:33:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.800+0000 7fe745d9b640 1 --2- 172.21.15.204:0/918300842 >> v2:172.21.15.204:6812/28210 conn(0x7fe7300397b0 0x7fe73003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.800+0000 7fe745d9b640 10 monclient: get_auth_request con 0x7fe7300397b0 auth_method 0 2024-09-15T01:33:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.801+0000 7fe745d9b640 1 --2- 172.21.15.204:0/918300842 >> v2:172.21.15.204:6812/28210 conn(0x7fe7300397b0 0x7fe73003bc60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe7380102a0 tx=0x7fe738008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.805+0000 7fe746ffd640 1 -- 172.21.15.204:0/918300842 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe740037030 con 0x7fe750067670 2024-09-15T01:33:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.805+0000 7fe746ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.805+0000 7fe746ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:48.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.959+0000 7fe754b5d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:48.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.959+0000 7fe754b5d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:48.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.959+0000 7fe754b5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:48.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.959+0000 7fe754b5d640 1 -- 172.21.15.204:0/918300842 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7fe75011fde0 con 0x7fe750067670 2024-09-15T01:33:48.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.960+0000 7fe746ffd640 1 -- 172.21.15.204:0/918300842 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v58) v1 ==== 65+0+60 (secure 0 0 0) 0x7fe740049080 con 0x7fe750067670 2024-09-15T01:33:48.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.960+0000 7fe746ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:48.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:48.960+0000 7fe746ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:49.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.052+0000 7fe754b5d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:49.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.052+0000 7fe754b5d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:49.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.052+0000 7fe754b5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:49.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.052+0000 7fe754b5d640 1 -- 172.21.15.204:0/918300842 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7fe750066830 con 0x7fe750067670 2024-09-15T01:33:49.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.053+0000 7fe746ffd640 1 -- 172.21.15.204:0/918300842 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v58) v1 ==== 65+0+60 (secure 0 0 0) 0x7fe74004c080 con 0x7fe750067670 2024-09-15T01:33:49.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.053+0000 7fe746ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:49.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.053+0000 7fe746ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:49.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 1 -- 172.21.15.204:0/918300842 >> v2:172.21.15.204:6812/28210 conn(0x7fe7300397b0 msgr2=0x7fe73003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:49.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 1 --2- 172.21.15.204:0/918300842 >> v2:172.21.15.204:6812/28210 conn(0x7fe7300397b0 0x7fe73003bc60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe7380102a0 tx=0x7fe738008040 comp rx=0 tx=0).stop 2024-09-15T01:33:49.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 10 monclient: shutdown 2024-09-15T01:33:49.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:49.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 1 -- 172.21.15.204:0/918300842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe750067670 msgr2=0x7fe75012f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:49.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 1 --2- 172.21.15.204:0/918300842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe750067670 0x7fe75012f770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe740009940 tx=0x7fe740032000 comp rx=0 tx=0).stop 2024-09-15T01:33:49.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 1 -- 172.21.15.204:0/918300842 shutdown_connections 2024-09-15T01:33:49.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 1 --2- 172.21.15.204:0/918300842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe750099a10 0x7fe75011c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 1 --2- 172.21.15.204:0/918300842 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe75006e980 0x7fe75011c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 1 --2- 172.21.15.204:0/918300842 >> v2:172.21.15.204:6812/28210 conn(0x7fe7300397b0 0x7fe73003bc60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 1 --2- 172.21.15.204:0/918300842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe750067670 0x7fe75012f770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 1 -- 172.21.15.204:0/918300842 >> 172.21.15.204:0/918300842 conn(0x7fe7500687e0 msgr2=0x7fe7500989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:49.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 1 -- 172.21.15.204:0/918300842 shutdown_connections 2024-09-15T01:33:49.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.055+0000 7fe754b5d640 1 -- 172.21.15.204:0/918300842 wait complete. 2024-09-15T01:33:49.066 INFO:tasks.workunit.client.0.smithi204.stdout:foo 2024-09-15T01:33:49.066 INFO:tasks.workunit.client.0.smithi204.stdout:foo-ssd 2024-09-15T01:33:49.067 INFO:tasks.workunit.client.0.smithi204.stdout:foo-hdd 2024-09-15T01:33:49.067 INFO:tasks.workunit.client.0.smithi204.stdout:ec-foo 2024-09-15T01:33:49.067 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rename foo foo-asdf 2024-09-15T01:33:49.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.140+0000 7f960955d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:49.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.140+0000 7f960955d640 10 monclient: build_initial_monmap 2024-09-15T01:33:49.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.141+0000 7f960955d640 10 monclient: monmap: 2024-09-15T01:33:49.144 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:49.144 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:49.144 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:49.142504+0000 2024-09-15T01:33:49.144 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:49.142504+0000 2024-09-15T01:33:49.144 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:49.144 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:49.144 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:49.144 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:49.144 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:49.144 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:49.145 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:49.145 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:49.145 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:49.145 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:49.145 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:49.145 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:49.145 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:49.145 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:49.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.141+0000 7f960955d640 1 Processor -- start 2024-09-15T01:33:49.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.141+0000 7f960955d640 1 -- start start 2024-09-15T01:33:49.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.141+0000 7f960955d640 10 monclient: init 2024-09-15T01:33:49.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:49.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 10 monclient: _add_conns ranks=[5,8,10,0,7,2,3,9,4,1,6] 2024-09-15T01:33:49.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96040a2cd0 0x7f9604132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 10 monclient(hunting): picked mon.noname-h con 0x7f96040a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:49.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960409fa20 0x7f960409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 10 monclient(hunting): picked mon.noname-e con 0x7f960409fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f96040a08e0 0x7f96040a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 10 monclient(hunting): picked mon.noname-f con 0x7f96040a08e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96040a30a0 con 0x7f96040a08e0 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96040a0330 con 0x7f96040a2cd0 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f96040a1db0 con 0x7f960409fa20 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f960955d640 10 monclient(hunting): authenticate will time out at 844.785192s 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f9602ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96040a2cd0 0x7f9604132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f96040a08e0 0x7f96040a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f9602ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96040a2cd0 0x7f9604132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49556/0 (socket says 172.21.15.204:49556) 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f9602ffd640 1 -- 172.21.15.204:0/3394493628 learned_addr learned my addr 172.21.15.204:0/3394493628 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f9602ffd640 10 monclient(hunting): get_auth_request con 0x7f96040a2cd0 auth_method 0 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f9602ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f9602ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f9602ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 10 monclient(hunting): get_auth_request con 0x7f96040a08e0 auth_method 0 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:49.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f9602ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:49.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f9602ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:49.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f9602ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:49.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96027fc640 1 --2- 172.21.15.204:0/3394493628 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960409fa20 0x7f960409fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96027fc640 10 monclient(hunting): get_auth_request con 0x7f960409fa20 auth_method 0 2024-09-15T01:33:49.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:49.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96027fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:49.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96027fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:49.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 10 monclient(hunting): handle_auth_done global_id 5140 payload 274 2024-09-15T01:33:49.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 1 -- 172.21.15.204:0/3394493628 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960409fa20 msgr2=0x7f960409fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:49.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 1 --2- 172.21.15.204:0/3394493628 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960409fa20 0x7f960409fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 1 -- 172.21.15.204:0/3394493628 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96040a2cd0 msgr2=0x7f9604132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:49.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 1 --2- 172.21.15.204:0/3394493628 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96040a2cd0 0x7f9604132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 1 monclient: found mon.noname-f 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f96037fe640 1 -- 172.21.15.204:0/3394493628 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96040648a0 con 0x7f96040a08e0 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.142+0000 7f9602ffd640 1 --2- 172.21.15.204:0/3394493628 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96040a2cd0 0x7f9604132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f96037fe640 1 --2- 172.21.15.204:0/3394493628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96040a08e0 0x7f96040a0d50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009e60 tx=0x7f95fc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=121dcf407dac3e61 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f9601ffb640 1 -- 172.21.15.204:0/3394493628 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95fc0079d0 con 0x7f96040a08e0 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f9601ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f9601ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f9601ffb640 10 monclient: dump: 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:49.158 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:49.159 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f9601ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f9601ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f9601ffb640 1 -- 172.21.15.204:0/3394493628 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc02d050 con 0x7f96040a08e0 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f960955d640 5 monclient: authenticate success, global_id 5140 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f960955d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f9601ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f960955d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f9601ffb640 1 -- 172.21.15.204:0/3394493628 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95fc007bd0 con 0x7f96040a08e0 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f960955d640 4 set_mon_vals no callback set 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f9601ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f9601ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:49.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f9601ffb640 10 monclient: dump: 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:49.161 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f960955d640 10 monclient: shutdown 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f960955d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f960955d640 1 -- 172.21.15.204:0/3394493628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96040a08e0 msgr2=0x7f96040a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.143+0000 7f960955d640 1 --2- 172.21.15.204:0/3394493628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96040a08e0 0x7f96040a0d50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009e60 tx=0x7f95fc009780 comp rx=0 tx=0).stop 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.144+0000 7f960955d640 1 -- 172.21.15.204:0/3394493628 shutdown_connections 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.144+0000 7f960955d640 1 --2- 172.21.15.204:0/3394493628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96040a08e0 0x7f96040a0d50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.144+0000 7f960955d640 1 --2- 172.21.15.204:0/3394493628 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960409fa20 0x7f960409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.144+0000 7f960955d640 1 --2- 172.21.15.204:0/3394493628 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f96040a2cd0 0x7f9604132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.144+0000 7f960955d640 1 -- 172.21.15.204:0/3394493628 >> 172.21.15.204:0/3394493628 conn(0x7f960406e860 msgr2=0x7f960406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:49.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.144+0000 7f960955d640 1 -- 172.21.15.204:0/3394493628 shutdown_connections 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.144+0000 7f960955d640 1 -- 172.21.15.204:0/3394493628 wait complete. 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.144+0000 7f960955d640 10 monclient: build_initial_monmap 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.144+0000 7f960955d640 10 monclient: monmap: 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:49.163 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 1 Processor -- start 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 1 -- start start 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 10 monclient: init 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 10 monclient: _add_conns ranks=[4,2,8,0,6,1,7,5,3] 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960409fa20 0x7f960411d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 10 monclient(hunting): picked mon.noname-c con 0x7f960409fa20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:49.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96040a08e0 0x7f960411d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 10 monclient(hunting): picked mon.noname-b con 0x7f96040a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96040a2cd0 0x7f960411dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 10 monclient(hunting): picked mon.noname-e con 0x7f96040a2cd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960411eb70 con 0x7f96040a08e0 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f960409d3d0 con 0x7f960409fa20 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f960409cb60 con 0x7f96040a2cd0 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f960955d640 10 monclient(hunting): authenticate will time out at 844.788510s 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960409fa20 0x7f960411d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96027fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96040a08e0 0x7f960411d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960409fa20 0x7f960411d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:38322/0 (socket says 172.21.15.204:38322) 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 1 -- 172.21.15.204:0/3020786656 learned_addr learned my addr 172.21.15.204:0/3020786656 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 10 monclient(hunting): get_auth_request con 0x7f960409fa20 auth_method 0 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96037fe640 1 --2- 172.21.15.204:0/3020786656 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96040a2cd0 0x7f960411dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96027fc640 10 monclient(hunting): get_auth_request con 0x7f96040a08e0 auth_method 0 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96027fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96027fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96037fe640 10 monclient(hunting): get_auth_request con 0x7f96040a2cd0 auth_method 0 2024-09-15T01:33:49.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96037fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96027fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96027fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96027fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f96037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 10 monclient(hunting): handle_auth_done global_id 4801 payload 824 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 1 -- 172.21.15.204:0/3020786656 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96040a2cd0 msgr2=0x7f960411dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:49.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 1 --2- 172.21.15.204:0/3020786656 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96040a2cd0 0x7f960411dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.145+0000 7f9602ffd640 1 -- 172.21.15.204:0/3020786656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96040a08e0 msgr2=0x7f960411d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f9602ffd640 1 --2- 172.21.15.204:0/3020786656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96040a08e0 0x7f960411d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f9602ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f9602ffd640 1 monclient: found mon.noname-c 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f9602ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f9602ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f9602ffd640 1 -- 172.21.15.204:0/3020786656 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4009450 con 0x7f960409fa20 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f9602ffd640 1 --2- 172.21.15.204:0/3020786656 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960409fa20 0x7f960411d270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f95f400ecb0 tx=0x7f95f40091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f95eaffd640 1 -- 172.21.15.204:0/3020786656 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95f4009730 con 0x7f960409fa20 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f95eaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f95eaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f95eaffd640 10 monclient: dump: 2024-09-15T01:33:49.168 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:49.169 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f95eaffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f95eaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f95eaffd640 1 -- 172.21.15.204:0/3020786656 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f4013470 con 0x7f960409fa20 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f960955d640 5 monclient: authenticate success, global_id 4801 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f95eaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f960955d640 10 monclient: _renew_subs 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f95eaffd640 1 -- 172.21.15.204:0/3020786656 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95f4013a60 con 0x7f960409fa20 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f960955d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f960955d640 1 -- 172.21.15.204:0/3020786656 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc009a20 con 0x7f960409fa20 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f95eaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:49.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f95eaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:49.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f95eaffd640 10 monclient: dump: 2024-09-15T01:33:49.171 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:49.171 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:49.171 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.171 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.171 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:49.171 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:49.171 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:49.171 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:49.171 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:49.171 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:49.172 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:49.172 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:49.172 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:49.172 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:49.172 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:49.172 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:49.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f96017fa640 4 set_mon_vals no callback set 2024-09-15T01:33:49.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f960955d640 10 monclient: _renew_subs 2024-09-15T01:33:49.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f960955d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:49.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.146+0000 7f960955d640 1 -- 172.21.15.204:0/3020786656 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9604132650 con 0x7f960409fa20 2024-09-15T01:33:49.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.147+0000 7f95eaffd640 1 -- 172.21.15.204:0/3020786656 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f95f4013be0 con 0x7f960409fa20 2024-09-15T01:33:49.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.147+0000 7f95eaffd640 1 --2- 172.21.15.204:0/3020786656 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00397e0 0x7f95f003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.147+0000 7f960955d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:49.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.147+0000 7f95eaffd640 1 -- 172.21.15.204:0/3020786656 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4301+0+0 (secure 0 0 0) 0x7f95f404b6b0 con 0x7f960409fa20 2024-09-15T01:33:49.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.147+0000 7f960955d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:49.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.147+0000 7f960955d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:49.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.147+0000 7f960955d640 1 -- 172.21.15.204:0/3020786656 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9604070b80 con 0x7f960409fa20 2024-09-15T01:33:49.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.147+0000 7f96027fc640 1 --2- 172.21.15.204:0/3020786656 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00397e0 0x7f95f003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.147+0000 7f96027fc640 10 monclient: get_auth_request con 0x7f95f00397e0 auth_method 0 2024-09-15T01:33:49.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.148+0000 7f96027fc640 1 --2- 172.21.15.204:0/3020786656 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00397e0 0x7f95f003bc90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f95f800bd40 tx=0x7f95f8008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:49.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.152+0000 7f95eaffd640 1 -- 172.21.15.204:0/3020786656 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f95f400cb60 con 0x7f960409fa20 2024-09-15T01:33:49.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.152+0000 7f95eaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:49.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.152+0000 7f95eaffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:49.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.310+0000 7f960955d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T01:33:49.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.310+0000 7f960955d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T01:33:49.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.310+0000 7f960955d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:49.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.310+0000 7f960955d640 1 -- 172.21.15.204:0/3020786656 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) v1 -- 0x7f9604120da0 con 0x7f960409fa20 2024-09-15T01:33:49.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.614+0000 7f95eaffd640 1 -- 172.21.15.204:0/3020786656 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 v59) v1 ==== 110+0+0 (secure 0 0 0) 0x7f95f40192c0 con 0x7f960409fa20 2024-09-15T01:33:49.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.614+0000 7f95eaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T01:33:49.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.614+0000 7f95eaffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:49.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.713+0000 7f960955d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T01:33:49.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.713+0000 7f960955d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T01:33:49.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.713+0000 7f960955d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:49.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.713+0000 7f960955d640 1 -- 172.21.15.204:0/3020786656 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) v1 -- 0x7f96041207b0 con 0x7f960409fa20 2024-09-15T01:33:49.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.714+0000 7f95eaffd640 1 -- 172.21.15.204:0/3020786656 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 already renamed to 'foo-asdf' v59) v1 ==== 139+0+0 (secure 0 0 0) 0x7f95f40194a0 con 0x7f960409fa20 2024-09-15T01:33:49.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.714+0000 7f95eaffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T01:33:49.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.714+0000 7f95eaffd640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo-asdf' 2024-09-15T01:33:49.717 INFO:tasks.workunit.client.0.smithi204.stderr:already renamed to 'foo-asdf' 2024-09-15T01:33:49.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 1 -- 172.21.15.204:0/3020786656 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00397e0 msgr2=0x7f95f003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:49.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 1 --2- 172.21.15.204:0/3020786656 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00397e0 0x7f95f003bc90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f95f800bd40 tx=0x7f95f8008040 comp rx=0 tx=0).stop 2024-09-15T01:33:49.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 10 monclient: shutdown 2024-09-15T01:33:49.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:49.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 1 -- 172.21.15.204:0/3020786656 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960409fa20 msgr2=0x7f960411d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:49.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 1 --2- 172.21.15.204:0/3020786656 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960409fa20 0x7f960411d270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f95f400ecb0 tx=0x7f95f40091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 1 -- 172.21.15.204:0/3020786656 shutdown_connections 2024-09-15T01:33:49.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 1 --2- 172.21.15.204:0/3020786656 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00397e0 0x7f95f003bc90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 1 --2- 172.21.15.204:0/3020786656 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96040a2cd0 0x7f960411dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 1 --2- 172.21.15.204:0/3020786656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96040a08e0 0x7f960411d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 1 --2- 172.21.15.204:0/3020786656 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960409fa20 0x7f960411d270 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 1 -- 172.21.15.204:0/3020786656 >> 172.21.15.204:0/3020786656 conn(0x7f960406e860 msgr2=0x7f96040a4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:49.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 1 -- 172.21.15.204:0/3020786656 shutdown_connections 2024-09-15T01:33:49.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.716+0000 7f960955d640 1 -- 172.21.15.204:0/3020786656 wait complete. 2024-09-15T01:33:49.728 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rename foo foo-asdf 2024-09-15T01:33:49.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.801+0000 7fc6d0e94640 10 monclient: get_monmap_and_config 2024-09-15T01:33:49.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.801+0000 7fc6d0e94640 10 monclient: build_initial_monmap 2024-09-15T01:33:49.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.801+0000 7fc6d0e94640 10 monclient: monmap: 2024-09-15T01:33:49.804 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:49.804 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:49.804 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:49.802784+0000 2024-09-15T01:33:49.804 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:49.802784+0000 2024-09-15T01:33:49.805 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:49.805 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:49.806 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:49.806 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:49.806 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:49.806 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:49.806 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:49.806 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:49.806 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:49.806 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:49.806 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:49.806 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:49.806 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:49.806 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:49.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.801+0000 7fc6d0e94640 1 Processor -- start 2024-09-15T01:33:49.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.801+0000 7fc6d0e94640 1 -- start start 2024-09-15T01:33:49.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.801+0000 7fc6d0e94640 10 monclient: init 2024-09-15T01:33:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 10 monclient: _add_conns ranks=[8,10,7,5,0,9,1,3,2,4,6] 2024-09-15T01:33:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc6cc0a2cd0 0x7fc6cc132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 10 monclient(hunting): picked mon.noname-e con 0x7fc6cc0a2cd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc6cc09fa20 0x7fc6cc09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 10 monclient(hunting): picked mon.noname-f con 0x7fc6cc09fa20 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a08e0 0x7fc6cc0a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 10 monclient(hunting): picked mon.noname-i con 0x7fc6cc0a08e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6cc0a30a0 con 0x7fc6cc09fa20 2024-09-15T01:33:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0a0330 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0a1db0 con 0x7fc6cc0a2cd0 2024-09-15T01:33:49.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:49.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6d0e94640 10 monclient(hunting): authenticate will time out at 845.445538s 2024-09-15T01:33:49.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6cad9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a08e0 0x7fc6cc0a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc6cc09fa20 0x7fc6cc09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6cad9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a08e0 0x7fc6cc0a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:56750/0 (socket says 172.21.15.204:56750) 2024-09-15T01:33:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc6cc09fa20 0x7fc6cc09fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43954/0 (socket says 172.21.15.204:43954) 2024-09-15T01:33:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6cad9d640 1 -- 172.21.15.204:0/227344083 learned_addr learned my addr 172.21.15.204:0/227344083 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6cad9d640 10 monclient(hunting): get_auth_request con 0x7fc6cc0a08e0 auth_method 0 2024-09-15T01:33:49.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:49.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6cad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:49.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:49.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 10 monclient(hunting): get_auth_request con 0x7fc6cc09fa20 auth_method 0 2024-09-15T01:33:49.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:49.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:49.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:49.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:49.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:49.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:49.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:49.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:49.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:49.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6ca59c640 1 --2- 172.21.15.204:0/227344083 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc6cc0a2cd0 0x7fc6cc132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6ca59c640 10 monclient(hunting): get_auth_request con 0x7fc6cc0a2cd0 auth_method 0 2024-09-15T01:33:49.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:49.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6ca59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:49.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:49.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 10 monclient(hunting): handle_auth_done global_id 5149 payload 274 2024-09-15T01:33:49.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 1 -- 172.21.15.204:0/227344083 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc6cc0a2cd0 msgr2=0x7fc6cc132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:49.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 1 --2- 172.21.15.204:0/227344083 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc6cc0a2cd0 0x7fc6cc132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 1 -- 172.21.15.204:0/227344083 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a08e0 msgr2=0x7fc6cc0a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:49.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 1 --2- 172.21.15.204:0/227344083 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a08e0 0x7fc6cc0a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 1 monclient: found mon.noname-f 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.802+0000 7fc6c9d9b640 1 -- 172.21.15.204:0/227344083 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc0648a0 con 0x7fc6cc09fa20 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6cad9d640 1 --2- 172.21.15.204:0/227344083 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a08e0 0x7fc6cc0a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6ca59c640 1 --2- 172.21.15.204:0/227344083 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc6cc0a2cd0 0x7fc6cc132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c9d9b640 1 --2- 172.21.15.204:0/227344083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6cc09fa20 0x7fc6cc09fdf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009850 tx=0x7fc6c00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81728dfc89c790d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c959a640 1 -- 172.21.15.204:0/227344083 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6c002abc0 con 0x7fc6cc09fa20 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c959a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c959a640 10 monclient: dump: 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:49.823 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:49.824 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.824 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.824 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:49.824 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:49.824 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:49.824 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:49.824 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:49.824 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:49.824 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:49.824 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c959a640 10 monclient: _finish_auth 0 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6d0e94640 5 monclient: authenticate success, global_id 5149 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c959a640 1 -- 172.21.15.204:0/227344083 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c002ad40 con 0x7fc6cc09fa20 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6d0e94640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6d0e94640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c959a640 1 -- 172.21.15.204:0/227344083 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6c0031ca0 con 0x7fc6cc09fa20 2024-09-15T01:33:49.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6d0e94640 4 set_mon_vals no callback set 2024-09-15T01:33:49.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:49.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c959a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:49.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.803+0000 7fc6c959a640 10 monclient: dump: 2024-09-15T01:33:49.826 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:49.826 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:49.826 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.826 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.826 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:49.826 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:49.826 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:49.826 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:49.826 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 10 monclient: shutdown 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 1 -- 172.21.15.204:0/227344083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6cc09fa20 msgr2=0x7fc6cc09fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 1 --2- 172.21.15.204:0/227344083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6cc09fa20 0x7fc6cc09fdf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009850 tx=0x7fc6c00090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 1 -- 172.21.15.204:0/227344083 shutdown_connections 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 1 --2- 172.21.15.204:0/227344083 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a08e0 0x7fc6cc0a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 1 --2- 172.21.15.204:0/227344083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6cc09fa20 0x7fc6cc09fdf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 1 --2- 172.21.15.204:0/227344083 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc6cc0a2cd0 0x7fc6cc132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 1 -- 172.21.15.204:0/227344083 >> 172.21.15.204:0/227344083 conn(0x7fc6cc06e860 msgr2=0x7fc6cc06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 1 -- 172.21.15.204:0/227344083 shutdown_connections 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 1 -- 172.21.15.204:0/227344083 wait complete. 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 10 monclient: build_initial_monmap 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.804+0000 7fc6d0e94640 10 monclient: monmap: 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:49.830 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 1 Processor -- start 2024-09-15T01:33:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 1 -- start start 2024-09-15T01:33:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 10 monclient: init 2024-09-15T01:33:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 10 monclient: _add_conns ranks=[3,5,7,8,0,4,1,2,6] 2024-09-15T01:33:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6cc09fa20 0x7fc6cc130dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 10 monclient(hunting): picked mon.noname-g con 0x7fc6cc09fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc6cc0a08e0 0x7fc6cc11d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 10 monclient(hunting): picked mon.noname-h con 0x7fc6cc0a08e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a2cd0 0x7fc6cc11dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 10 monclient(hunting): picked mon.noname-i con 0x7fc6cc0a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc133e40 con 0x7fc6cc09fa20 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc09d3d0 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc09cb60 con 0x7fc6cc0a2cd0 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6d0e94640 10 monclient(hunting): authenticate will time out at 845.448710s 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6ca59c640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6cc09fa20 msgr2=0x7fc6cc130dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6ca59c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6cc09fa20 0x7fc6cc130dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6cad9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a2cd0 0x7fc6cc11dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6c9d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc6cc0a08e0 0x7fc6cc11d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6cad9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a2cd0 0x7fc6cc11dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:56762/0 (socket says 172.21.15.204:56762) 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6c9d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc6cc0a08e0 0x7fc6cc11d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49572/0 (socket says 172.21.15.204:49572) 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6c9d9b640 1 -- 172.21.15.204:0/3330140365 learned_addr learned my addr 172.21.15.204:0/3330140365 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6c9d9b640 10 monclient(hunting): get_auth_request con 0x7fc6cc0a08e0 auth_method 0 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6c9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6c9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6c9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6cad9d640 10 monclient(hunting): get_auth_request con 0x7fc6cc0a2cd0 auth_method 0 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6cad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6c9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6c9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6c9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.805+0000 7fc6cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6c9d9b640 10 monclient(hunting): handle_auth_done global_id 4901 payload 824 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6c9d9b640 1 -- 172.21.15.204:0/3330140365 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a2cd0 msgr2=0x7fc6cc11dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6c9d9b640 1 --2- 172.21.15.204:0/3330140365 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a2cd0 0x7fc6cc11dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6c9d9b640 1 -- 172.21.15.204:0/3330140365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6cc09fa20 msgr2=0x7fc6cc130dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6c9d9b640 1 --2- 172.21.15.204:0/3330140365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6cc09fa20 0x7fc6cc130dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6c9d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:33:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6c9d9b640 1 monclient: found mon.noname-h 2024-09-15T01:33:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6c9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6c9d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6c9d9b640 1 -- 172.21.15.204:0/3330140365 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c4009ae0 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6c9d9b640 1 --2- 172.21.15.204:0/3330140365 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc6cc0a08e0 0x7fc6cc11d9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009360 tx=0x7fc6c0032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6ba7fc640 1 -- 172.21.15.204:0/3330140365 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6c003b870 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6ba7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6ba7fc640 10 monclient: dump: 2024-09-15T01:33:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:49.836 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6ba7fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6ba7fc640 1 -- 172.21.15.204:0/3330140365 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c003ba10 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6d0e94640 5 monclient: authenticate success, global_id 4901 2024-09-15T01:33:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6d0e94640 10 monclient: _renew_subs 2024-09-15T01:33:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6d0e94640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6d0e94640 1 -- 172.21.15.204:0/3330140365 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c0009390 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6ba7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6ba7fc640 1 -- 172.21.15.204:0/3330140365 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6c00313a0 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6ba7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6ba7fc640 10 monclient: dump: 2024-09-15T01:33:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6d0e94640 10 monclient: _renew_subs 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6d0e94640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6c8d99640 4 set_mon_vals no callback set 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.806+0000 7fc6d0e94640 1 -- 172.21.15.204:0/3330140365 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc11ffc0 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.807+0000 7fc6ba7fc640 1 -- 172.21.15.204:0/3330140365 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc6c0039040 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.807+0000 7fc6d0e94640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.807+0000 7fc6ba7fc640 1 --2- 172.21.15.204:0/3330140365 >> v2:172.21.15.204:6812/28210 conn(0x7fc6ac0397b0 0x7fc6ac03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.807+0000 7fc6d0e94640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.807+0000 7fc6d0e94640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.807+0000 7fc6d0e94640 1 -- 172.21.15.204:0/3330140365 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc070b80 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.807+0000 7fc6ba7fc640 1 -- 172.21.15.204:0/3330140365 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4306+0+0 (secure 0 0 0) 0x7fc6c006ab50 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.807+0000 7fc6ca59c640 1 --2- 172.21.15.204:0/3330140365 >> v2:172.21.15.204:6812/28210 conn(0x7fc6ac0397b0 0x7fc6ac03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.807+0000 7fc6ca59c640 10 monclient: get_auth_request con 0x7fc6ac0397b0 auth_method 0 2024-09-15T01:33:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.807+0000 7fc6ca59c640 1 --2- 172.21.15.204:0/3330140365 >> v2:172.21.15.204:6812/28210 conn(0x7fc6ac0397b0 0x7fc6ac03bc60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc006f40 tx=0x7fc6bc008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.811+0000 7fc6ba7fc640 1 -- 172.21.15.204:0/3330140365 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc6c0042020 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.811+0000 7fc6ba7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.811+0000 7fc6ba7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:49.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.971+0000 7fc6d0e94640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T01:33:49.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.971+0000 7fc6d0e94640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T01:33:49.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.971+0000 7fc6d0e94640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:49.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.971+0000 7fc6d0e94640 1 -- 172.21.15.204:0/3330140365 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) v1 -- 0x7fc6cc11f3f0 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.971+0000 7fc6ba7fc640 1 -- 172.21.15.204:0/3330140365 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 already renamed to 'foo-asdf' v59) v1 ==== 139+0+0 (secure 0 0 0) 0x7fc6c0037070 con 0x7fc6cc0a08e0 2024-09-15T01:33:49.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.971+0000 7fc6ba7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T01:33:49.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:49.971+0000 7fc6ba7fc640 10 monclient: _finish_command 2 = system:0 already renamed to 'foo-asdf' 2024-09-15T01:33:50.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.066+0000 7fc6d0e94640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T01:33:50.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.066+0000 7fc6d0e94640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T01:33:50.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.066+0000 7fc6d0e94640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:50.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.066+0000 7fc6d0e94640 1 -- 172.21.15.204:0/3330140365 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) v1 -- 0x7fc6cc1427b0 con 0x7fc6cc0a08e0 2024-09-15T01:33:50.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.066+0000 7fc6ba7fc640 1 -- 172.21.15.204:0/3330140365 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 already renamed to 'foo-asdf' v59) v1 ==== 139+0+0 (secure 0 0 0) 0x7fc6c0031520 con 0x7fc6cc0a08e0 2024-09-15T01:33:50.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.066+0000 7fc6ba7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-09-15T01:33:50.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.066+0000 7fc6ba7fc640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo-asdf' 2024-09-15T01:33:50.069 INFO:tasks.workunit.client.0.smithi204.stderr:already renamed to 'foo-asdf' 2024-09-15T01:33:50.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 1 -- 172.21.15.204:0/3330140365 >> v2:172.21.15.204:6812/28210 conn(0x7fc6ac0397b0 msgr2=0x7fc6ac03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:50.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 1 --2- 172.21.15.204:0/3330140365 >> v2:172.21.15.204:6812/28210 conn(0x7fc6ac0397b0 0x7fc6ac03bc60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc006f40 tx=0x7fc6bc008040 comp rx=0 tx=0).stop 2024-09-15T01:33:50.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 10 monclient: shutdown 2024-09-15T01:33:50.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:50.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 1 -- 172.21.15.204:0/3330140365 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc6cc0a08e0 msgr2=0x7fc6cc11d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:50.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 1 --2- 172.21.15.204:0/3330140365 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc6cc0a08e0 0x7fc6cc11d9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009360 tx=0x7fc6c0032000 comp rx=0 tx=0).stop 2024-09-15T01:33:50.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 1 -- 172.21.15.204:0/3330140365 shutdown_connections 2024-09-15T01:33:50.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 1 --2- 172.21.15.204:0/3330140365 >> v2:172.21.15.204:6812/28210 conn(0x7fc6ac0397b0 0x7fc6ac03bc60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 1 --2- 172.21.15.204:0/3330140365 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc6cc0a2cd0 0x7fc6cc11dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 1 --2- 172.21.15.204:0/3330140365 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc6cc0a08e0 0x7fc6cc11d9a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 1 --2- 172.21.15.204:0/3330140365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6cc09fa20 0x7fc6cc130dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 1 -- 172.21.15.204:0/3330140365 >> 172.21.15.204:0/3330140365 conn(0x7fc6cc06e860 msgr2=0x7fc6cc0a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:50.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 1 -- 172.21.15.204:0/3330140365 shutdown_connections 2024-09-15T01:33:50.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.068+0000 7fc6d0e94640 1 -- 172.21.15.204:0/3330140365 wait complete. 2024-09-15T01:33:50.080 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rename bar bar-asdf 2024-09-15T01:33:50.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.153+0000 7f961331d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:50.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.153+0000 7f961331d640 10 monclient: build_initial_monmap 2024-09-15T01:33:50.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.153+0000 7f961331d640 10 monclient: monmap: 2024-09-15T01:33:50.156 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:50.156 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:50.156 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:50.154558+0000 2024-09-15T01:33:50.156 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:50.154558+0000 2024-09-15T01:33:50.156 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:50.156 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:50.156 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:50.156 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:50.156 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:50.157 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:50.157 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:50.157 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:50.157 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:50.157 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:50.157 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:50.157 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:50.157 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:50.158 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:50.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.153+0000 7f961331d640 1 Processor -- start 2024-09-15T01:33:50.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.153+0000 7f961331d640 1 -- start start 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.153+0000 7f961331d640 10 monclient: init 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 10 monclient: _add_conns ranks=[9,3,7,6,0,5,8,2,4,1,10] 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f960c0a4590 0x7f960c09e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 10 monclient(hunting): picked mon.noname-a con 0x7f960c0a4590 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f960c06edd0 0x7f960c09ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 10 monclient(hunting): picked mon.noname-g con 0x7f960c06edd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c09f1d0 0x7f960c1307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 10 monclient(hunting): picked mon.noname-i con 0x7f960c09f1d0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960c09f600 con 0x7f960c0a4590 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:50.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c0a4960 con 0x7f960c06edd0 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f960c06dba0 con 0x7f960c09f1d0 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f961331d640 10 monclient(hunting): authenticate will time out at 845.797420s 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96108b8640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f960c06edd0 msgr2=0x7f960c09ec90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96108b8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f960c06edd0 0x7f960c09ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c09f1d0 0x7f960c1307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c09f1d0 0x7f960c1307a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:56766/0 (socket says 172.21.15.204:56766) 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 1 -- 172.21.15.204:0/2881720284 learned_addr learned my addr 172.21.15.204:0/2881720284 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 10 monclient(hunting): get_auth_request con 0x7f960c09f1d0 auth_method 0 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:50.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 10 monclient(hunting): handle_auth_done global_id 4903 payload 274 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 1 -- 172.21.15.204:0/2881720284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f960c06edd0 msgr2=0x7f960c09ec90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 1 --2- 172.21.15.204:0/2881720284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f960c06edd0 0x7f960c09ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 1 -- 172.21.15.204:0/2881720284 >> v2:172.21.15.178:3300/0 conn(0x7f960c0a4590 msgr2=0x7f960c09e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96110b9640 1 --2- 172.21.15.204:0/2881720284 >> v2:172.21.15.178:3300/0 conn(0x7f960c0a4590 0x7f960c09e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 1 --2- 172.21.15.204:0/2881720284 >> v2:172.21.15.178:3300/0 conn(0x7f960c0a4590 0x7f960c09e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 10 monclient: _finish_hunting 0 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 1 monclient: found mon.noname-i 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:50.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.154+0000 7f96118ba640 1 -- 172.21.15.204:0/2881720284 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c0648a0 con 0x7f960c09f1d0 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f96118ba640 1 --2- 172.21.15.204:0/2881720284 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c09f1d0 0x7f960c1307a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9604009f20 tx=0x7f9604009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=dfe7b762cbd621c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f9603fff640 1 -- 172.21.15.204:0/2881720284 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9604007880 con 0x7f960c09f1d0 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f9603fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f9603fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f9603fff640 10 monclient: dump: 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:50.170 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f9603fff640 10 monclient: _finish_auth 0 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f9603fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f9603fff640 1 -- 172.21.15.204:0/2881720284 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960402d050 con 0x7f960c09f1d0 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f961331d640 5 monclient: authenticate success, global_id 4903 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f961331d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:50.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f9603fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f961331d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f961331d640 4 set_mon_vals no callback set 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f9603fff640 1 -- 172.21.15.204:0/2881720284 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9604030360 con 0x7f960c09f1d0 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f9603fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f9603fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.155+0000 7f9603fff640 10 monclient: dump: 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:50.172 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 10 monclient: shutdown 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 1 -- 172.21.15.204:0/2881720284 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c09f1d0 msgr2=0x7f960c1307a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 1 --2- 172.21.15.204:0/2881720284 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c09f1d0 0x7f960c1307a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9604009f20 tx=0x7f9604009840 comp rx=0 tx=0).stop 2024-09-15T01:33:50.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 1 -- 172.21.15.204:0/2881720284 shutdown_connections 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 1 --2- 172.21.15.204:0/2881720284 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c09f1d0 0x7f960c1307a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 1 --2- 172.21.15.204:0/2881720284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f960c06edd0 0x7f960c09ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 1 --2- 172.21.15.204:0/2881720284 >> v2:172.21.15.178:3300/0 conn(0x7f960c0a4590 0x7f960c09e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 1 -- 172.21.15.204:0/2881720284 >> 172.21.15.204:0/2881720284 conn(0x7f960c067700 msgr2=0x7f960c0a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 1 -- 172.21.15.204:0/2881720284 shutdown_connections 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 1 -- 172.21.15.204:0/2881720284 wait complete. 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 10 monclient: build_initial_monmap 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.156+0000 7f961331d640 10 monclient: monmap: 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:50.174 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 1 Processor -- start 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 1 -- start start 2024-09-15T01:33:50.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 10 monclient: init 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 10 monclient: _add_conns ranks=[2,1,6,4,0,8,3,5,7] 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f960c06edd0 0x7f960c13e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 10 monclient(hunting): picked mon.noname-b con 0x7f960c06edd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f960c09f1d0 0x7f960c13d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 10 monclient(hunting): picked mon.noname-f con 0x7f960c09f1d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c0a4590 0x7f960c141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 10 monclient(hunting): picked mon.noname-d con 0x7f960c0a4590 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c122d30 con 0x7f960c09f1d0 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c06d070 con 0x7f960c06edd0 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:50.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f960c06cc80 con 0x7f960c0a4590 2024-09-15T01:33:50.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:50.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f961331d640 10 monclient(hunting): authenticate will time out at 845.800846s 2024-09-15T01:33:50.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f96108b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f960c09f1d0 0x7f960c13d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:50.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f96118ba640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c0a4590 msgr2=0x7f960c141850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:50.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f96118ba640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c0a4590 0x7f960c141850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:50.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f96108b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f960c09f1d0 0x7f960c13d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43962/0 (socket says 172.21.15.204:43962) 2024-09-15T01:33:50.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f96108b8640 1 -- 172.21.15.204:0/191787788 learned_addr learned my addr 172.21.15.204:0/191787788 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:50.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f96108b8640 10 monclient(hunting): get_auth_request con 0x7f960c09f1d0 auth_method 0 2024-09-15T01:33:50.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f96108b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:50.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f96108b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:50.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.157+0000 7f96108b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:50.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96110b9640 1 --2- 172.21.15.204:0/191787788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f960c06edd0 0x7f960c13e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96110b9640 10 monclient(hunting): get_auth_request con 0x7f960c06edd0 auth_method 0 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96110b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96110b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96110b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96110b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96110b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96110b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 10 monclient(hunting): handle_auth_done global_id 5158 payload 824 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 1 -- 172.21.15.204:0/191787788 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c0a4590 msgr2=0x7f960c141850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 1 --2- 172.21.15.204:0/191787788 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c0a4590 0x7f960c141850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 1 -- 172.21.15.204:0/191787788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f960c06edd0 msgr2=0x7f960c13e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 1 --2- 172.21.15.204:0/191787788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f960c06edd0 0x7f960c13e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 10 monclient: _finish_hunting 0 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 1 monclient: found mon.noname-f 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 1 -- 172.21.15.204:0/191787788 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9604009ae0 con 0x7f960c09f1d0 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96108b8640 1 --2- 172.21.15.204:0/191787788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f960c09f1d0 0x7f960c13d9b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00b5c0 tx=0x7f95fc009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96017fa640 1 -- 172.21.15.204:0/191787788 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95fc006150 con 0x7f960c09f1d0 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96017fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96017fa640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96017fa640 10 monclient: dump: 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:50.179 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:50.180 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96017fa640 10 monclient: _finish_auth 0 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96017fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96017fa640 1 -- 172.21.15.204:0/191787788 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc0062d0 con 0x7f960c09f1d0 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f961331d640 5 monclient: authenticate success, global_id 5158 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96017fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96017fa640 1 -- 172.21.15.204:0/191787788 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95fc0103c0 con 0x7f960c09f1d0 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f961331d640 10 monclient: _renew_subs 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f961331d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f961331d640 1 -- 172.21.15.204:0/191787788 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c13eed0 con 0x7f960c09f1d0 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96017fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96037fe640 4 set_mon_vals no callback set 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96017fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f96017fa640 10 monclient: dump: 2024-09-15T01:33:50.181 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:50.182 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f961331d640 10 monclient: _renew_subs 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f961331d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.158+0000 7f961331d640 1 -- 172.21.15.204:0/191787788 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c13def0 con 0x7f960c09f1d0 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.159+0000 7f961331d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.159+0000 7f961331d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.159+0000 7f961331d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.159+0000 7f96017fa640 1 -- 172.21.15.204:0/191787788 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f95fc003b80 con 0x7f960c09f1d0 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.159+0000 7f961331d640 1 -- 172.21.15.204:0/191787788 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960c070b80 con 0x7f960c09f1d0 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.159+0000 7f96017fa640 1 --2- 172.21.15.204:0/191787788 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00397b0 0x7f95f003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.159+0000 7f96017fa640 1 -- 172.21.15.204:0/191787788 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4306+0+0 (secure 0 0 0) 0x7f95fc04a530 con 0x7f960c09f1d0 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.159+0000 7f96110b9640 1 --2- 172.21.15.204:0/191787788 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00397b0 0x7f95f003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.159+0000 7f96110b9640 10 monclient: get_auth_request con 0x7f95f00397b0 auth_method 0 2024-09-15T01:33:50.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.160+0000 7f96110b9640 1 --2- 172.21.15.204:0/191787788 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00397b0 0x7f95f003bc60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9608006fd0 tx=0x7f9608008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:50.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.164+0000 7f96017fa640 1 -- 172.21.15.204:0/191787788 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f95fc00dad0 con 0x7f960c09f1d0 2024-09-15T01:33:50.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.164+0000 7f96017fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:50.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.164+0000 7f96017fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:50.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.321+0000 7f961331d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-09-15T01:33:50.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.321+0000 7f961331d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-09-15T01:33:50.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.321+0000 7f961331d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:50.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.321+0000 7f961331d640 1 -- 172.21.15.204:0/191787788 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"} v 0) v1 -- 0x7f960c0a04c0 con 0x7f960c09f1d0 2024-09-15T01:33:50.635 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.633+0000 7f96017fa640 1 -- 172.21.15.204:0/191787788 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}]=0 v60) v1 ==== 110+0+0 (secure 0 0 0) 0x7f95fc017070 con 0x7f960c09f1d0 2024-09-15T01:33:50.635 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.633+0000 7f96017fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-09-15T01:33:50.635 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.633+0000 7f96017fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:50.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.730+0000 7f961331d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-09-15T01:33:50.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.730+0000 7f961331d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-09-15T01:33:50.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.730+0000 7f961331d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:50.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.730+0000 7f961331d640 1 -- 172.21.15.204:0/191787788 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"} v 0) v1 -- 0x7f960c0008d0 con 0x7f960c09f1d0 2024-09-15T01:33:50.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.731+0000 7f96017fa640 1 -- 172.21.15.204:0/191787788 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}]=0 already renamed to 'bar-asdf' v60) v1 ==== 139+0+0 (secure 0 0 0) 0x7f95fc00d2f0 con 0x7f960c09f1d0 2024-09-15T01:33:50.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.731+0000 7f96017fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-09-15T01:33:50.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.731+0000 7f96017fa640 10 monclient: _finish_command 3 = system:0 already renamed to 'bar-asdf' 2024-09-15T01:33:50.734 INFO:tasks.workunit.client.0.smithi204.stderr:already renamed to 'bar-asdf' 2024-09-15T01:33:50.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 1 -- 172.21.15.204:0/191787788 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00397b0 msgr2=0x7f95f003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:50.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 1 --2- 172.21.15.204:0/191787788 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00397b0 0x7f95f003bc60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9608006fd0 tx=0x7f9608008040 comp rx=0 tx=0).stop 2024-09-15T01:33:50.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 10 monclient: shutdown 2024-09-15T01:33:50.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:50.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 1 -- 172.21.15.204:0/191787788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f960c09f1d0 msgr2=0x7f960c13d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:50.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 1 --2- 172.21.15.204:0/191787788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f960c09f1d0 0x7f960c13d9b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00b5c0 tx=0x7f95fc009120 comp rx=0 tx=0).stop 2024-09-15T01:33:50.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 1 -- 172.21.15.204:0/191787788 shutdown_connections 2024-09-15T01:33:50.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 1 --2- 172.21.15.204:0/191787788 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00397b0 0x7f95f003bc60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 1 --2- 172.21.15.204:0/191787788 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c0a4590 0x7f960c141850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 1 --2- 172.21.15.204:0/191787788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f960c09f1d0 0x7f960c13d9b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 1 --2- 172.21.15.204:0/191787788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f960c06edd0 0x7f960c13e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 1 -- 172.21.15.204:0/191787788 >> 172.21.15.204:0/191787788 conn(0x7f960c067700 msgr2=0x7f960c09d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:50.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 1 -- 172.21.15.204:0/191787788 shutdown_connections 2024-09-15T01:33:50.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.733+0000 7f961331d640 1 -- 172.21.15.204:0/191787788 wait complete. 2024-09-15T01:33:50.744 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule ls 2024-09-15T01:33:50.745 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep foo-asdf 2024-09-15T01:33:50.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.816+0000 7f07f071d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:50.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.816+0000 7f07f071d640 10 monclient: build_initial_monmap 2024-09-15T01:33:50.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.817+0000 7f07f071d640 10 monclient: monmap: 2024-09-15T01:33:50.820 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:50.820 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:50.820 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:50.818476+0000 2024-09-15T01:33:50.821 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:50.818476+0000 2024-09-15T01:33:50.821 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:50.821 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:50.821 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:50.821 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:50.821 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:50.821 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:50.822 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:50.822 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:50.822 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:50.822 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:50.822 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:50.822 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:50.822 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:50.822 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:50.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.817+0000 7f07f071d640 1 Processor -- start 2024-09-15T01:33:50.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.817+0000 7f07f071d640 1 -- start start 2024-09-15T01:33:50.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.817+0000 7f07f071d640 10 monclient: init 2024-09-15T01:33:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:50.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 10 monclient: _add_conns ranks=[8,10,5,7,9,3,1,2,4,0,6] 2024-09-15T01:33:50.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f07e80a2e30 0x7f07e8132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 10 monclient(hunting): picked mon.noname-e con 0x7f07e80a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:50.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f07e809fb80 0x7f07e809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 10 monclient(hunting): picked mon.noname-f con 0x7f07e809fb80 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:50.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07e80a0a40 0x7f07e80a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 10 monclient(hunting): picked mon.noname-h con 0x7f07e80a0a40 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:50.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:50.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e809e6c0 con 0x7f07e809fb80 2024-09-15T01:33:50.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:50.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80a04b0 con 0x7f07e80a0a40 2024-09-15T01:33:50.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:50.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80a3200 con 0x7f07e80a2e30 2024-09-15T01:33:50.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:50.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07f071d640 10 monclient(hunting): authenticate will time out at 846.461298s 2024-09-15T01:33:50.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07e80a0a40 0x7f07e80a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:50.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07edcb8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f07e809fb80 0x7f07e809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:50.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07e80a0a40 0x7f07e80a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49576/0 (socket says 172.21.15.204:49576) 2024-09-15T01:33:50.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 1 -- 172.21.15.204:0/2798263859 learned_addr learned my addr 172.21.15.204:0/2798263859 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:50.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 10 monclient(hunting): get_auth_request con 0x7f07e80a0a40 auth_method 0 2024-09-15T01:33:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07edcb8640 10 monclient(hunting): get_auth_request con 0x7f07e809fb80 auth_method 0 2024-09-15T01:33:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07edcb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07edcb8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07edcb8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07edcb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07edcb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07edcb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07ee4b9640 1 --2- 172.21.15.204:0/2798263859 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f07e80a2e30 0x7f07e8132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07ee4b9640 10 monclient(hunting): get_auth_request con 0x7f07e80a2e30 auth_method 0 2024-09-15T01:33:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07ee4b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07ee4b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07ee4b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 10 monclient(hunting): handle_auth_done global_id 4910 payload 274 2024-09-15T01:33:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 1 -- 172.21.15.204:0/2798263859 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f07e80a2e30 msgr2=0x7f07e8132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 1 --2- 172.21.15.204:0/2798263859 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f07e80a2e30 0x7f07e8132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 1 -- 172.21.15.204:0/2798263859 >> v2:172.21.15.204:3300/0 conn(0x7f07e809fb80 msgr2=0x7f07e809ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 1 --2- 172.21.15.204:0/2798263859 >> v2:172.21.15.204:3300/0 conn(0x7f07e809fb80 0x7f07e809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 10 monclient: _finish_hunting 0 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 1 monclient: found mon.noname-h 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07eecba640 1 -- 172.21.15.204:0/2798263859 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e804b720 con 0x7f07e80a0a40 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07edcb8640 1 --2- 172.21.15.204:0/2798263859 >> v2:172.21.15.204:3300/0 conn(0x7f07e809fb80 0x7f07e809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.818+0000 7f07ee4b9640 1 --2- 172.21.15.204:0/2798263859 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f07e80a2e30 0x7f07e8132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07eecba640 1 --2- 172.21.15.204:0/2798263859 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07e80a0a40 0x7f07e80a0ee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009f20 tx=0x7f07d8009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=19b2077187459751 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07ed4b7640 1 -- 172.21.15.204:0/2798263859 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f07d8007880 con 0x7f07e80a0a40 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07ed4b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07ed4b7640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07ed4b7640 10 monclient: dump: 2024-09-15T01:33:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:50.840 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07ed4b7640 10 monclient: _finish_auth 0 2024-09-15T01:33:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07ed4b7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07f071d640 5 monclient: authenticate success, global_id 4910 2024-09-15T01:33:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07f071d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07ed4b7640 1 -- 172.21.15.204:0/2798263859 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d802d050 con 0x7f07e80a0a40 2024-09-15T01:33:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07ed4b7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07f071d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07f071d640 4 set_mon_vals no callback set 2024-09-15T01:33:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07ed4b7640 1 -- 172.21.15.204:0/2798263859 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f07d8030360 con 0x7f07e80a0a40 2024-09-15T01:33:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07ed4b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07ed4b7640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.819+0000 7f07ed4b7640 10 monclient: dump: 2024-09-15T01:33:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:50.842 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 10 monclient: shutdown 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 1 -- 172.21.15.204:0/2798263859 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07e80a0a40 msgr2=0x7f07e80a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 1 --2- 172.21.15.204:0/2798263859 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07e80a0a40 0x7f07e80a0ee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009f20 tx=0x7f07d8009840 comp rx=0 tx=0).stop 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 1 -- 172.21.15.204:0/2798263859 shutdown_connections 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 1 --2- 172.21.15.204:0/2798263859 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f07e80a0a40 0x7f07e80a0ee0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 1 --2- 172.21.15.204:0/2798263859 >> v2:172.21.15.204:3300/0 conn(0x7f07e809fb80 0x7f07e809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 1 --2- 172.21.15.204:0/2798263859 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f07e80a2e30 0x7f07e8132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 1 -- 172.21.15.204:0/2798263859 >> 172.21.15.204:0/2798263859 conn(0x7f07e806e860 msgr2=0x7f07e806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 1 -- 172.21.15.204:0/2798263859 shutdown_connections 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 1 -- 172.21.15.204:0/2798263859 wait complete. 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 10 monclient: build_initial_monmap 2024-09-15T01:33:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.820+0000 7f07f071d640 10 monclient: monmap: 2024-09-15T01:33:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:50.846 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 1 Processor -- start 2024-09-15T01:33:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 1 -- start start 2024-09-15T01:33:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 10 monclient: init 2024-09-15T01:33:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 10 monclient: _add_conns ranks=[2,1,7,8,3,6,5,0,4] 2024-09-15T01:33:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e809fb80 0x7f07e8141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 10 monclient(hunting): picked mon.noname-b con 0x7f07e809fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07e80a0a40 0x7f07e8142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 10 monclient(hunting): picked mon.noname-f con 0x7f07e80a0a40 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07e80a2e30 0x7f07e811e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 10 monclient(hunting): picked mon.noname-i con 0x7f07e80a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e81203e0 con 0x7f07e80a0a40 2024-09-15T01:33:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e81216e0 con 0x7f07e809fb80 2024-09-15T01:33:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f07e809ca40 con 0x7f07e80a2e30 2024-09-15T01:33:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07f071d640 10 monclient(hunting): authenticate will time out at 846.464518s 2024-09-15T01:33:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07e80a0a40 0x7f07e8142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07eecba640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07e80a2e30 0x7f07e811e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07e80a0a40 0x7f07e8142320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43978/0 (socket says 172.21.15.204:43978) 2024-09-15T01:33:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 1 -- 172.21.15.204:0/3075909979 learned_addr learned my addr 172.21.15.204:0/3075909979 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07eecba640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07e80a2e30 0x7f07e811e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:56780/0 (socket says 172.21.15.204:56780) 2024-09-15T01:33:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 10 monclient(hunting): get_auth_request con 0x7f07e80a0a40 auth_method 0 2024-09-15T01:33:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07eecba640 10 monclient(hunting): get_auth_request con 0x7f07e80a2e30 auth_method 0 2024-09-15T01:33:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07eecba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07eecba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07eecba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07ee4b9640 1 --2- 172.21.15.204:0/3075909979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e809fb80 0x7f07e8141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07ee4b9640 10 monclient(hunting): get_auth_request con 0x7f07e809fb80 auth_method 0 2024-09-15T01:33:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07ee4b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07ee4b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07ee4b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07eecba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07eecba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07eecba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07ee4b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07ee4b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07ee4b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 10 monclient(hunting): handle_auth_done global_id 5176 payload 824 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 1 -- 172.21.15.204:0/3075909979 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07e80a2e30 msgr2=0x7f07e811e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 1 --2- 172.21.15.204:0/3075909979 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07e80a2e30 0x7f07e811e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 1 -- 172.21.15.204:0/3075909979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e809fb80 msgr2=0x7f07e8141de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 1 --2- 172.21.15.204:0/3075909979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e809fb80 0x7f07e8141de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 10 monclient: _finish_hunting 0 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 1 monclient: found mon.noname-f 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.821+0000 7f07edcb8640 1 -- 172.21.15.204:0/3075909979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc009390 con 0x7f07e80a0a40 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07edcb8640 1 --2- 172.21.15.204:0/3075909979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07e80a0a40 0x7f07e8142320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00ec50 tx=0x7f07dc0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07e67fc640 1 -- 172.21.15.204:0/3075909979 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f07dc00ee90 con 0x7f07e80a0a40 2024-09-15T01:33:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07e67fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07e67fc640 10 monclient: dump: 2024-09-15T01:33:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07e67fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07f071d640 5 monclient: authenticate success, global_id 5176 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07e67fc640 1 -- 172.21.15.204:0/3075909979 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc00cbc0 con 0x7f07e80a0a40 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07f071d640 10 monclient: _renew_subs 2024-09-15T01:33:50.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07f071d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07f071d640 1 -- 172.21.15.204:0/3075909979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d8009ae0 con 0x7f07e80a0a40 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07e67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07e67fc640 1 -- 172.21.15.204:0/3075909979 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f07dc013d70 con 0x7f07e80a0a40 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07e67fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07e67fc640 10 monclient: dump: 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:50.854 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07f071d640 10 monclient: _renew_subs 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07f071d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07eccb6640 4 set_mon_vals no callback set 2024-09-15T01:33:50.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.822+0000 7f07f071d640 1 -- 172.21.15.204:0/3075909979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e81207f0 con 0x7f07e80a0a40 2024-09-15T01:33:50.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.823+0000 7f07e67fc640 1 -- 172.21.15.204:0/3075909979 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f07dc00cd30 con 0x7f07e80a0a40 2024-09-15T01:33:50.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.823+0000 7f07f071d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:50.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.823+0000 7f07f071d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:50.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.823+0000 7f07f071d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:50.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.823+0000 7f07f071d640 1 -- 172.21.15.204:0/3075909979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e8070cc0 con 0x7f07e80a0a40 2024-09-15T01:33:50.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.823+0000 7f07e67fc640 1 --2- 172.21.15.204:0/3075909979 >> v2:172.21.15.204:6812/28210 conn(0x7f07d40397b0 0x7f07d403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:50.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.823+0000 7f07e67fc640 1 -- 172.21.15.204:0/3075909979 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4311+0+0 (secure 0 0 0) 0x7f07dc04b5d0 con 0x7f07e80a0a40 2024-09-15T01:33:50.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.823+0000 7f07ee4b9640 1 --2- 172.21.15.204:0/3075909979 >> v2:172.21.15.204:6812/28210 conn(0x7f07d40397b0 0x7f07d403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:50.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.823+0000 7f07ee4b9640 10 monclient: get_auth_request con 0x7f07d40397b0 auth_method 0 2024-09-15T01:33:50.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.823+0000 7f07ee4b9640 1 --2- 172.21.15.204:0/3075909979 >> v2:172.21.15.204:6812/28210 conn(0x7f07d40397b0 0x7f07d403bc60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f07e000bc80 tx=0x7f07e0008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:50.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.827+0000 7f07e67fc640 1 -- 172.21.15.204:0/3075909979 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f07dc0238f0 con 0x7f07e80a0a40 2024-09-15T01:33:50.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.827+0000 7f07e67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:50.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.827+0000 7f07e67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:50.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.981+0000 7f07f071d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:50.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.981+0000 7f07f071d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:50.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.981+0000 7f07f071d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:50.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.981+0000 7f07f071d640 1 -- 172.21.15.204:0/3075909979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f07e80a1f10 con 0x7f07e80a0a40 2024-09-15T01:33:50.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.981+0000 7f07e67fc640 1 -- 172.21.15.204:0/3075909979 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v60) v1 ==== 65+0+70 (secure 0 0 0) 0x7f07dc0232f0 con 0x7f07e80a0a40 2024-09-15T01:33:50.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.981+0000 7f07e67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:50.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:50.981+0000 7f07e67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:51.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.074+0000 7f07f071d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:51.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.074+0000 7f07f071d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:51.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.074+0000 7f07f071d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:51.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.074+0000 7f07f071d640 1 -- 172.21.15.204:0/3075909979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f07e811fc20 con 0x7f07e80a0a40 2024-09-15T01:33:51.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.074+0000 7f07e67fc640 1 -- 172.21.15.204:0/3075909979 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v60) v1 ==== 65+0+70 (secure 0 0 0) 0x7f07dc0234d0 con 0x7f07e80a0a40 2024-09-15T01:33:51.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.074+0000 7f07e67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:51.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.074+0000 7f07e67fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:51.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 1 -- 172.21.15.204:0/3075909979 >> v2:172.21.15.204:6812/28210 conn(0x7f07d40397b0 msgr2=0x7f07d403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:51.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 1 --2- 172.21.15.204:0/3075909979 >> v2:172.21.15.204:6812/28210 conn(0x7f07d40397b0 0x7f07d403bc60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f07e000bc80 tx=0x7f07e0008040 comp rx=0 tx=0).stop 2024-09-15T01:33:51.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 10 monclient: shutdown 2024-09-15T01:33:51.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:51.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 1 -- 172.21.15.204:0/3075909979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07e80a0a40 msgr2=0x7f07e8142320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:51.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 1 --2- 172.21.15.204:0/3075909979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07e80a0a40 0x7f07e8142320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00ec50 tx=0x7f07dc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 1 -- 172.21.15.204:0/3075909979 shutdown_connections 2024-09-15T01:33:51.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 1 --2- 172.21.15.204:0/3075909979 >> v2:172.21.15.204:6812/28210 conn(0x7f07d40397b0 0x7f07d403bc60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 1 --2- 172.21.15.204:0/3075909979 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07e80a2e30 0x7f07e811e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 1 --2- 172.21.15.204:0/3075909979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07e80a0a40 0x7f07e8142320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 1 --2- 172.21.15.204:0/3075909979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e809fb80 0x7f07e8141de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 1 -- 172.21.15.204:0/3075909979 >> 172.21.15.204:0/3075909979 conn(0x7f07e806e860 msgr2=0x7f07e809ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:51.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 1 -- 172.21.15.204:0/3075909979 shutdown_connections 2024-09-15T01:33:51.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.076+0000 7f07f071d640 1 -- 172.21.15.204:0/3075909979 wait complete. 2024-09-15T01:33:51.088 INFO:tasks.workunit.client.0.smithi204.stdout:foo-asdf 2024-09-15T01:33:51.088 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule ls 2024-09-15T01:33:51.088 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep bar-asdf 2024-09-15T01:33:51.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.159+0000 7f7c8644f640 10 monclient: get_monmap_and_config 2024-09-15T01:33:51.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.159+0000 7f7c8644f640 10 monclient: build_initial_monmap 2024-09-15T01:33:51.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.159+0000 7f7c8644f640 10 monclient: monmap: 2024-09-15T01:33:51.162 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:51.162 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:51.162 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:51.161014+0000 2024-09-15T01:33:51.162 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:51.161014+0000 2024-09-15T01:33:51.162 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:51.162 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:51.162 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:51.162 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:51.163 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:51.163 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:51.163 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:51.163 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:51.163 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:51.163 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:51.163 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:51.163 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:51.163 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:51.163 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:51.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 1 Processor -- start 2024-09-15T01:33:51.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 1 -- start start 2024-09-15T01:33:51.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 10 monclient: init 2024-09-15T01:33:51.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:51.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 10 monclient: _add_conns ranks=[8,10,9,4,6,5,2,3,7,1,0] 2024-09-15T01:33:51.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a2e30 0x7f7c80132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:51.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 10 monclient(hunting): picked mon.noname-e con 0x7f7c800a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:51.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7c8009fb80 0x7f7c8009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:51.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 10 monclient(hunting): picked mon.noname-f con 0x7f7c8009fb80 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:51.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7c800a0a40 0x7f7c800a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:51.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 10 monclient(hunting): picked mon.noname-a con 0x7f7c800a0a40 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:51.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:51.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8009e6c0 con 0x7f7c800a0a40 2024-09-15T01:33:51.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:51.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c800a04b0 con 0x7f7c8009fb80 2024-09-15T01:33:51.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:51.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800a3200 con 0x7f7c800a2e30 2024-09-15T01:33:51.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:51.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.160+0000 7f7c8644f640 10 monclient(hunting): authenticate will time out at 846.803878s 2024-09-15T01:33:51.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7c8009fb80 0x7f7c8009ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:51.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7c8009fb80 0x7f7c8009ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43986/0 (socket says 172.21.15.204:43986) 2024-09-15T01:33:51.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 1 -- 172.21.15.204:0/1690104698 learned_addr learned my addr 172.21.15.204:0/1690104698 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:51.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 10 monclient(hunting): get_auth_request con 0x7f7c8009fb80 auth_method 0 2024-09-15T01:33:51.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:51.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:51.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7ffff640 1 --2- 172.21.15.204:0/1690104698 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a2e30 0x7f7c80132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 10 monclient(hunting): handle_auth_done global_id 5185 payload 274 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 1 -- 172.21.15.204:0/1690104698 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a2e30 msgr2=0x7f7c80132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 1 --2- 172.21.15.204:0/1690104698 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a2e30 0x7f7c80132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c849ec640 1 --2- 172.21.15.204:0/1690104698 >> v2:172.21.15.178:3300/0 conn(0x7f7c800a0a40 0x7f7c800a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 1 -- 172.21.15.204:0/1690104698 >> v2:172.21.15.178:3300/0 conn(0x7f7c800a0a40 msgr2=0x7f7c800a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 1 --2- 172.21.15.204:0/1690104698 >> v2:172.21.15.178:3300/0 conn(0x7f7c800a0a40 0x7f7c800a0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 1 monclient: found mon.noname-f 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:51.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 1 -- 172.21.15.204:0/1690104698 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8004b720 con 0x7f7c8009fb80 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7ffff640 10 monclient: get_auth_request con 0x7f7c800a2e30 auth_method 0 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7ffff640 1 --2- 172.21.15.204:0/1690104698 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a2e30 0x7f7c80132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.161+0000 7f7c7f7fe640 1 --2- 172.21.15.204:0/1690104698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c8009fb80 0x7f7c8009ff70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7c80133920 tx=0x7f7c740091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e743f695a0d78712 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c7effd640 1 -- 172.21.15.204:0/1690104698 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c7402acb0 con 0x7f7c8009fb80 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c7effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c7effd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c7effd640 10 monclient: dump: 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:51.175 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c7effd640 10 monclient: _finish_auth 0 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c7effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 5 monclient: authenticate success, global_id 5185 2024-09-15T01:33:51.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c7effd640 1 -- 172.21.15.204:0/1690104698 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7402ae30 con 0x7f7c8009fb80 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c7effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 4 set_mon_vals no callback set 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c7effd640 1 -- 172.21.15.204:0/1690104698 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c74031ca0 con 0x7f7c8009fb80 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c7effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c7effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c7effd640 10 monclient: dump: 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:51.177 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 10 monclient: shutdown 2024-09-15T01:33:51.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 1 -- 172.21.15.204:0/1690104698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c8009fb80 msgr2=0x7f7c8009ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 1 --2- 172.21.15.204:0/1690104698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c8009fb80 0x7f7c8009ff70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7c80133920 tx=0x7f7c740091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 1 -- 172.21.15.204:0/1690104698 shutdown_connections 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 1 --2- 172.21.15.204:0/1690104698 >> v2:172.21.15.178:3300/0 conn(0x7f7c800a0a40 0x7f7c800a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 1 --2- 172.21.15.204:0/1690104698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c8009fb80 0x7f7c8009ff70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 1 --2- 172.21.15.204:0/1690104698 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a2e30 0x7f7c80132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 1 -- 172.21.15.204:0/1690104698 >> 172.21.15.204:0/1690104698 conn(0x7f7c8006e860 msgr2=0x7f7c8006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 1 -- 172.21.15.204:0/1690104698 shutdown_connections 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.162+0000 7f7c8644f640 1 -- 172.21.15.204:0/1690104698 wait complete. 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.163+0000 7f7c8644f640 10 monclient: build_initial_monmap 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.163+0000 7f7c8644f640 10 monclient: monmap: 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:51.179 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:51.180 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.163+0000 7f7c8644f640 1 Processor -- start 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.163+0000 7f7c8644f640 1 -- start start 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.163+0000 7f7c8644f640 10 monclient: init 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 10 monclient: _add_conns ranks=[3,8,0,1,6,2,4,5,7] 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c8009fb80 0x7f7c801316c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 10 monclient(hunting): picked mon.noname-g con 0x7f7c8009fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a0a40 0x7f7c8011e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 10 monclient(hunting): picked mon.noname-e con 0x7f7c800a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c800a2e30 0x7f7c8011e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 10 monclient(hunting): picked mon.noname-a con 0x7f7c800a2e30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:51.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8009d650 con 0x7f7c800a2e30 2024-09-15T01:33:51.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:51.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c801204a0 con 0x7f7c8009fb80 2024-09-15T01:33:51.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:51.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8009ca40 con 0x7f7c800a0a40 2024-09-15T01:33:51.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:51.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c7ffff640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c8009fb80 msgr2=0x7f7c801316c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:51.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c8644f640 10 monclient(hunting): authenticate will time out at 846.807315s 2024-09-15T01:33:51.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c7ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c8009fb80 0x7f7c801316c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:51.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c800a2e30 0x7f7c8011e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:51.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c7f7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a0a40 0x7f7c8011e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:51.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c800a2e30 0x7f7c8011e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:53004/0 (socket says 172.21.15.204:53004) 2024-09-15T01:33:51.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c7f7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a0a40 0x7f7c8011e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:34016/0 (socket says 172.21.15.204:34016) 2024-09-15T01:33:51.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 1 -- 172.21.15.204:0/3358601696 learned_addr learned my addr 172.21.15.204:0/3358601696 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:51.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 10 monclient(hunting): get_auth_request con 0x7f7c800a2e30 auth_method 0 2024-09-15T01:33:51.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:51.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:51.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:51.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c7f7fe640 10 monclient(hunting): get_auth_request con 0x7f7c800a0a40 auth_method 0 2024-09-15T01:33:51.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:51.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c7f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:51.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c7f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:51.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:51.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:51.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 10 monclient(hunting): handle_auth_done global_id 4932 payload 824 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 1 -- 172.21.15.204:0/3358601696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a0a40 msgr2=0x7f7c8011e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 1 --2- 172.21.15.204:0/3358601696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a0a40 0x7f7c8011e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 1 -- 172.21.15.204:0/3358601696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c8009fb80 msgr2=0x7f7c801316c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 1 --2- 172.21.15.204:0/3358601696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c8009fb80 0x7f7c801316c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 10 monclient: _finish_hunting 0 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 1 monclient: found mon.noname-a 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:33:51.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c849ec640 1 -- 172.21.15.204:0/3358601696 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c74009450 con 0x7f7c800a2e30 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.164+0000 7f7c7f7fe640 1 --2- 172.21.15.204:0/3358601696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a0a40 0x7f7c8011e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c849ec640 1 --2- 172.21.15.204:0/3358601696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c800a2e30 0x7f7c8011e7d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800ceb0 tx=0x7f7c78007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c67fff640 1 -- 172.21.15.204:0/3358601696 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c780079b0 con 0x7f7c800a2e30 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c67fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c67fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c67fff640 10 monclient: dump: 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:51.185 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c67fff640 10 monclient: _finish_auth 0 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c67fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c67fff640 1 -- 172.21.15.204:0/3358601696 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c780168a0 con 0x7f7c800a2e30 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c8644f640 5 monclient: authenticate success, global_id 4932 2024-09-15T01:33:51.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c67fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c67fff640 1 -- 172.21.15.204:0/3358601696 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c7800e400 con 0x7f7c800a2e30 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c8644f640 10 monclient: _renew_subs 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c8644f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c8644f640 1 -- 172.21.15.204:0/3358601696 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8011fce0 con 0x7f7c800a2e30 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c67fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c67fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c67fff640 10 monclient: dump: 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:51.187 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c7e7fc640 4 set_mon_vals no callback set 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c8644f640 10 monclient: _renew_subs 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c8644f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.165+0000 7f7c8644f640 1 -- 172.21.15.204:0/3358601696 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c80121520 con 0x7f7c800a2e30 2024-09-15T01:33:51.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.166+0000 7f7c67fff640 1 -- 172.21.15.204:0/3358601696 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7c78003020 con 0x7f7c800a2e30 2024-09-15T01:33:51.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.166+0000 7f7c8644f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:51.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.166+0000 7f7c8644f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:51.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.166+0000 7f7c8644f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:51.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.166+0000 7f7c8644f640 1 -- 172.21.15.204:0/3358601696 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c80066b80 con 0x7f7c800a2e30 2024-09-15T01:33:51.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.166+0000 7f7c67fff640 1 --2- 172.21.15.204:0/3358601696 >> v2:172.21.15.204:6812/28210 conn(0x7f7c6c0397b0 0x7f7c6c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:51.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.166+0000 7f7c67fff640 1 -- 172.21.15.204:0/3358601696 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4311+0+0 (secure 0 0 0) 0x7f7c78012070 con 0x7f7c800a2e30 2024-09-15T01:33:51.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.166+0000 7f7c7ffff640 1 --2- 172.21.15.204:0/3358601696 >> v2:172.21.15.204:6812/28210 conn(0x7f7c6c0397b0 0x7f7c6c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:51.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.166+0000 7f7c7ffff640 10 monclient: get_auth_request con 0x7f7c6c0397b0 auth_method 0 2024-09-15T01:33:51.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.166+0000 7f7c7ffff640 1 --2- 172.21.15.204:0/3358601696 >> v2:172.21.15.204:6812/28210 conn(0x7f7c6c0397b0 0x7f7c6c03bc60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c70006c00 tx=0x7f7c70008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:51.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.171+0000 7f7c67fff640 1 -- 172.21.15.204:0/3358601696 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7c78016630 con 0x7f7c800a2e30 2024-09-15T01:33:51.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.171+0000 7f7c67fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:51.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.171+0000 7f7c67fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:51.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.326+0000 7f7c8644f640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:51.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.326+0000 7f7c8644f640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:51.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.326+0000 7f7c8644f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:51.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.326+0000 7f7c8644f640 1 -- 172.21.15.204:0/3358601696 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f7c80070cc0 con 0x7f7c800a2e30 2024-09-15T01:33:51.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.326+0000 7f7c67fff640 1 -- 172.21.15.204:0/3358601696 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v60) v1 ==== 65+0+70 (secure 0 0 0) 0x7f7c78014c30 con 0x7f7c800a2e30 2024-09-15T01:33:51.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.326+0000 7f7c67fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:51.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.326+0000 7f7c67fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:51.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.419+0000 7f7c8644f640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:51.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.419+0000 7f7c8644f640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:51.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.419+0000 7f7c8644f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:33:51.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.419+0000 7f7c8644f640 1 -- 172.21.15.204:0/3358601696 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f7c800a1f10 con 0x7f7c800a2e30 2024-09-15T01:33:51.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.420+0000 7f7c67fff640 1 -- 172.21.15.204:0/3358601696 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v60) v1 ==== 65+0+70 (secure 0 0 0) 0x7f7c78014df0 con 0x7f7c800a2e30 2024-09-15T01:33:51.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.420+0000 7f7c67fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:51.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.420+0000 7f7c67fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:51.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 1 -- 172.21.15.204:0/3358601696 >> v2:172.21.15.204:6812/28210 conn(0x7f7c6c0397b0 msgr2=0x7f7c6c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:51.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 1 --2- 172.21.15.204:0/3358601696 >> v2:172.21.15.204:6812/28210 conn(0x7f7c6c0397b0 0x7f7c6c03bc60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c70006c00 tx=0x7f7c70008040 comp rx=0 tx=0).stop 2024-09-15T01:33:51.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 10 monclient: shutdown 2024-09-15T01:33:51.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:51.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 1 -- 172.21.15.204:0/3358601696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c800a2e30 msgr2=0x7f7c8011e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:51.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 1 --2- 172.21.15.204:0/3358601696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c800a2e30 0x7f7c8011e7d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800ceb0 tx=0x7f7c78007870 comp rx=0 tx=0).stop 2024-09-15T01:33:51.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 1 -- 172.21.15.204:0/3358601696 shutdown_connections 2024-09-15T01:33:51.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 1 --2- 172.21.15.204:0/3358601696 >> v2:172.21.15.204:6812/28210 conn(0x7f7c6c0397b0 0x7f7c6c03bc60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 1 --2- 172.21.15.204:0/3358601696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c800a2e30 0x7f7c8011e7d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 1 --2- 172.21.15.204:0/3358601696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c800a0a40 0x7f7c8011e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 1 --2- 172.21.15.204:0/3358601696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c8009fb80 0x7f7c801316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:51.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 1 -- 172.21.15.204:0/3358601696 >> 172.21.15.204:0/3358601696 conn(0x7f7c8006e860 msgr2=0x7f7c8009ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:51.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 1 -- 172.21.15.204:0/3358601696 shutdown_connections 2024-09-15T01:33:51.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:51.422+0000 7f7c8644f640 1 -- 172.21.15.204:0/3358601696 wait complete. 2024-09-15T01:33:51.434 INFO:tasks.workunit.client.0.smithi204.stdout:bar-asdf 2024-09-15T01:33:51.434 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rm foo 2024-09-15T01:33:51.434 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 'does not exist' 2024-09-15T01:33:51.791 INFO:tasks.workunit.client.0.smithi204.stdout:2024-09-15T01:33:51.681+0000 7f0630ff9640 1 -- 172.21.15.204:0/674498915 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v61) v1 ==== 103+0+0 (secure 0 0 0) 0x7f0624010300 con 0x7f063406e840 2024-09-15T01:33:51.792 INFO:tasks.workunit.client.0.smithi204.stdout:2024-09-15T01:33:51.681+0000 7f0630ff9640 10 monclient: _finish_command 2 = system:0 rule foo does not exist 2024-09-15T01:33:51.792 INFO:tasks.workunit.client.0.smithi204.stdout:2024-09-15T01:33:51.778+0000 7f0630ff9640 1 -- 172.21.15.204:0/674498915 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v61) v1 ==== 103+0+0 (secure 0 0 0) 0x7f06240104e0 con 0x7f063406e840 2024-09-15T01:33:51.792 INFO:tasks.workunit.client.0.smithi204.stdout:2024-09-15T01:33:51.778+0000 7f0630ff9640 10 monclient: _finish_command 3 = system:0 rule foo does not exist 2024-09-15T01:33:51.792 INFO:tasks.workunit.client.0.smithi204.stdout:rule foo does not exist 2024-09-15T01:33:51.793 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rm bar 2024-09-15T01:33:51.793 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 'does not exist' 2024-09-15T01:33:52.140 INFO:tasks.workunit.client.0.smithi204.stdout:2024-09-15T01:33:52.031+0000 7fe9a8ff9640 1 -- 172.21.15.204:0/826092639 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 rule bar does not exist v61) v1 ==== 103+0+0 (secure 0 0 0) 0x7fe9a0044da0 con 0x7fe9ac09fb80 2024-09-15T01:33:52.140 INFO:tasks.workunit.client.0.smithi204.stdout:2024-09-15T01:33:52.031+0000 7fe9a8ff9640 10 monclient: _finish_command 2 = system:0 rule bar does not exist 2024-09-15T01:33:52.140 INFO:tasks.workunit.client.0.smithi204.stdout:2024-09-15T01:33:52.126+0000 7fe9a8ff9640 1 -- 172.21.15.204:0/826092639 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 rule bar does not exist v61) v1 ==== 103+0+0 (secure 0 0 0) 0x7fe9a00447c0 con 0x7fe9ac09fb80 2024-09-15T01:33:52.140 INFO:tasks.workunit.client.0.smithi204.stdout:2024-09-15T01:33:52.126+0000 7fe9a8ff9640 10 monclient: _finish_command 3 = system:0 rule bar does not exist 2024-09-15T01:33:52.140 INFO:tasks.workunit.client.0.smithi204.stdout:rule bar does not exist 2024-09-15T01:33:52.140 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rename foo-asdf foo 2024-09-15T01:33:52.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.213+0000 7fda3379d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:52.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.213+0000 7fda3379d640 10 monclient: build_initial_monmap 2024-09-15T01:33:52.215 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.213+0000 7fda3379d640 10 monclient: monmap: 2024-09-15T01:33:52.216 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:52.216 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:52.216 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:52.214714+0000 2024-09-15T01:33:52.216 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:52.214714+0000 2024-09-15T01:33:52.216 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:52.216 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:52.216 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:52.216 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:52.216 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:52.216 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:52.217 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:52.217 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:52.217 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:52.217 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:52.217 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:52.217 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:52.218 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:52.218 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:52.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.213+0000 7fda3379d640 1 Processor -- start 2024-09-15T01:33:52.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 1 -- start start 2024-09-15T01:33:52.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 10 monclient: init 2024-09-15T01:33:52.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:52.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 10 monclient: _add_conns ranks=[0,9,10,4,7,3,1,8,6,5,2] 2024-09-15T01:33:52.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fda2c0a2cd0 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:52.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fda2c09fa20 0x7fda2c09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 10 monclient(hunting): picked mon.noname-a con 0x7fda2c09fa20 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:52.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fda2c137c80 0x7fda2c13a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 10 monclient(hunting): picked mon.noname-f con 0x7fda2c137c80 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c0648a0 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2c133310 con 0x7fda2c09fa20 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2c0a30a0 con 0x7fda2c137c80 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda3379d640 10 monclient(hunting): authenticate will time out at 847.857649s 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda31d3a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fda2c137c80 0x7fda2c13a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda31d3a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fda2c137c80 0x7fda2c13a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44006/0 (socket says 172.21.15.204:44006) 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda31d3a640 1 -- 172.21.15.204:0/1387947500 learned_addr learned my addr 172.21.15.204:0/1387947500 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda31d3a640 10 monclient(hunting): get_auth_request con 0x7fda2c137c80 auth_method 0 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda31d3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:52.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda31d3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:52.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda31d3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:52.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda31d3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:52.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda31d3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:52.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda31d3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:52.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda30d38640 1 --2- 172.21.15.204:0/1387947500 >> v2:172.21.15.178:3300/0 conn(0x7fda2c09fa20 0x7fda2c09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:52.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda30d38640 10 monclient(hunting): get_auth_request con 0x7fda2c09fa20 auth_method 0 2024-09-15T01:33:52.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda30d38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:52.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda30d38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:52.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.214+0000 7fda30d38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:52.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda31d3a640 10 monclient(hunting): handle_auth_done global_id 5212 payload 274 2024-09-15T01:33:52.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda31d3a640 1 -- 172.21.15.204:0/1387947500 >> v2:172.21.15.178:3300/0 conn(0x7fda2c09fa20 msgr2=0x7fda2c09fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:52.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda31d3a640 1 --2- 172.21.15.204:0/1387947500 >> v2:172.21.15.178:3300/0 conn(0x7fda2c09fa20 0x7fda2c09fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda31d3a640 1 -- 172.21.15.204:0/1387947500 >> v1:172.21.15.178:6789/0 conn(0x7fda2c0a2cd0 legacy=0x7fda2c132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda31d3a640 10 monclient: _finish_hunting 0 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda31d3a640 1 monclient: found mon.noname-f 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda31d3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda31d3a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda31d3a640 1 -- 172.21.15.204:0/1387947500 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c0a0330 con 0x7fda2c137c80 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda30d38640 1 --2- 172.21.15.204:0/1387947500 >> v2:172.21.15.178:3300/0 conn(0x7fda2c09fa20 0x7fda2c09fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda31d3a640 1 --2- 172.21.15.204:0/1387947500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda2c137c80 0x7fda2c13a060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0a1980 tx=0x7fda24009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6f07255ec05d7b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda23fff640 1 -- 172.21.15.204:0/1387947500 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fda24007880 con 0x7fda2c137c80 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda23fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda23fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda23fff640 10 monclient: dump: 2024-09-15T01:33:52.233 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:52.234 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:52.235 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:52.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda23fff640 10 monclient: _finish_auth 0 2024-09-15T01:33:52.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda23fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:52.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda23fff640 1 -- 172.21.15.204:0/1387947500 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2402d050 con 0x7fda2c137c80 2024-09-15T01:33:52.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda3379d640 5 monclient: authenticate success, global_id 5212 2024-09-15T01:33:52.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda3379d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:52.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda23fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:52.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda3379d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:52.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda23fff640 1 -- 172.21.15.204:0/1387947500 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fda24030360 con 0x7fda2c137c80 2024-09-15T01:33:52.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda3379d640 4 set_mon_vals no callback set 2024-09-15T01:33:52.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda23fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda23fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.215+0000 7fda23fff640 10 monclient: dump: 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:52.236 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.216+0000 7fda3379d640 10 monclient: shutdown 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.216+0000 7fda3379d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.216+0000 7fda3379d640 1 -- 172.21.15.204:0/1387947500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda2c137c80 msgr2=0x7fda2c13a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.216+0000 7fda3379d640 1 --2- 172.21.15.204:0/1387947500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda2c137c80 0x7fda2c13a060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0a1980 tx=0x7fda24009840 comp rx=0 tx=0).stop 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.216+0000 7fda3379d640 1 -- 172.21.15.204:0/1387947500 shutdown_connections 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.216+0000 7fda3379d640 1 --2- 172.21.15.204:0/1387947500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda2c137c80 0x7fda2c13a060 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.216+0000 7fda3379d640 1 --2- 172.21.15.204:0/1387947500 >> v2:172.21.15.178:3300/0 conn(0x7fda2c09fa20 0x7fda2c09fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.216+0000 7fda3379d640 1 -- 172.21.15.204:0/1387947500 >> 172.21.15.204:0/1387947500 conn(0x7fda2c06e860 msgr2=0x7fda2c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.216+0000 7fda3379d640 1 -- 172.21.15.204:0/1387947500 shutdown_connections 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.216+0000 7fda3379d640 1 -- 172.21.15.204:0/1387947500 wait complete. 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.216+0000 7fda3379d640 10 monclient: build_initial_monmap 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.216+0000 7fda3379d640 10 monclient: monmap: 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:52.238 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 1 Processor -- start 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 1 -- start start 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 10 monclient: init 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 10 monclient: _add_conns ranks=[4,1,7,2,3,5,0,8,6] 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fda2c09fa20 0x7fda2c1341d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 10 monclient(hunting): picked mon.noname-c con 0x7fda2c09fa20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda2c0a2cd0 0x7fda2c134710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 10 monclient(hunting): picked mon.noname-f con 0x7fda2c0a2cd0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fda2c137c80 0x7fda2c134c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 10 monclient(hunting): picked mon.noname-i con 0x7fda2c137c80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c096bf0 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c09d3d0 con 0x7fda2c09fa20 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c09cb60 con 0x7fda2c137c80 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.217+0000 7fda3379d640 10 monclient(hunting): authenticate will time out at 847.860951s 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda2c0a2cd0 0x7fda2c134710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31d3a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fda2c137c80 0x7fda2c134c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:52.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda2c0a2cd0 0x7fda2c134710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44022/0 (socket says 172.21.15.204:44022) 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 1 -- 172.21.15.204:0/1370209727 learned_addr learned my addr 172.21.15.204:0/1370209727 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 10 monclient(hunting): get_auth_request con 0x7fda2c0a2cd0 auth_method 0 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31d3a640 10 monclient(hunting): get_auth_request con 0x7fda2c137c80 auth_method 0 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31d3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31d3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31d3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31539640 1 --2- 172.21.15.204:0/1370209727 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fda2c09fa20 0x7fda2c1341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:52.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31539640 10 monclient(hunting): get_auth_request con 0x7fda2c09fa20 auth_method 0 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31539640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31539640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31d3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31d3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31d3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31539640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31539640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31539640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 10 monclient(hunting): handle_auth_done global_id 5221 payload 824 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 1 -- 172.21.15.204:0/1370209727 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fda2c137c80 msgr2=0x7fda2c134c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 1 --2- 172.21.15.204:0/1370209727 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fda2c137c80 0x7fda2c134c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 1 -- 172.21.15.204:0/1370209727 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fda2c09fa20 msgr2=0x7fda2c1341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:52.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 1 --2- 172.21.15.204:0/1370209727 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fda2c09fa20 0x7fda2c1341d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 10 monclient: _finish_hunting 0 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 1 monclient: found mon.noname-f 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 1 -- 172.21.15.204:0/1370209727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda24009ae0 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda31d3a640 1 --2- 172.21.15.204:0/1370209727 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fda2c137c80 0x7fda2c134c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda30d38640 1 --2- 172.21.15.204:0/1370209727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda2c0a2cd0 0x7fda2c134710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00d890 tx=0x7fda1c008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda217fa640 1 -- 172.21.15.204:0/1370209727 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fda1c007490 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda217fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda217fa640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda217fa640 10 monclient: dump: 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:52.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda217fa640 10 monclient: _finish_auth 0 2024-09-15T01:33:52.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda217fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:52.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda3379d640 5 monclient: authenticate success, global_id 5221 2024-09-15T01:33:52.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda217fa640 1 -- 172.21.15.204:0/1370209727 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1c00de80 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda217fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:52.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda3379d640 10 monclient: _renew_subs 2024-09-15T01:33:52.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda217fa640 1 -- 172.21.15.204:0/1370209727 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fda1c00bd50 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda3379d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:52.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda3379d640 1 -- 172.21.15.204:0/1370209727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c135b40 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda237fe640 4 set_mon_vals no callback set 2024-09-15T01:33:52.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda217fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:52.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda217fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:52.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda217fa640 10 monclient: dump: 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:52.246 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda3379d640 10 monclient: _renew_subs 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda3379d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.218+0000 7fda3379d640 1 -- 172.21.15.204:0/1370209727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c136870 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.219+0000 7fda217fa640 1 -- 172.21.15.204:0/1370209727 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fda1c00b8d0 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.219+0000 7fda217fa640 1 --2- 172.21.15.204:0/1370209727 >> v2:172.21.15.204:6812/28210 conn(0x7fda100397e0 0x7fda1003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.219+0000 7fda217fa640 1 -- 172.21.15.204:0/1370209727 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4311+0+0 (secure 0 0 0) 0x7fda1c04b7a0 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.219+0000 7fda31539640 1 --2- 172.21.15.204:0/1370209727 >> v2:172.21.15.204:6812/28210 conn(0x7fda100397e0 0x7fda1003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.219+0000 7fda3379d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.219+0000 7fda3379d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.219+0000 7fda3379d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.219+0000 7fda3379d640 1 -- 172.21.15.204:0/1370209727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c070b80 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.219+0000 7fda31539640 10 monclient: get_auth_request con 0x7fda100397e0 auth_method 0 2024-09-15T01:33:52.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.220+0000 7fda31539640 1 --2- 172.21.15.204:0/1370209727 >> v2:172.21.15.204:6812/28210 conn(0x7fda100397e0 0x7fda1003bc90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fda28007fd0 tx=0x7fda28007e10 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:52.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.224+0000 7fda217fa640 1 -- 172.21.15.204:0/1370209727 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fda1c014430 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.224+0000 7fda217fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:52.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.224+0000 7fda217fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:52.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.380+0000 7fda3379d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T01:33:52.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.380+0000 7fda3379d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T01:33:52.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.380+0000 7fda3379d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:52.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.380+0000 7fda3379d640 1 -- 172.21.15.204:0/1370209727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) v1 -- 0x7fda2c067450 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.637+0000 7fda217fa640 1 -- 172.21.15.204:0/1370209727 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 v62) v1 ==== 110+0+0 (secure 0 0 0) 0x7fda1c01c360 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.637+0000 7fda217fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T01:33:52.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.637+0000 7fda217fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:52.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.734+0000 7fda3379d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T01:33:52.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.734+0000 7fda3379d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T01:33:52.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.734+0000 7fda3379d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:52.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.734+0000 7fda3379d640 1 -- 172.21.15.204:0/1370209727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) v1 -- 0x7fda2c136200 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.735+0000 7fda217fa640 1 -- 172.21.15.204:0/1370209727 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 already renamed to 'foo' v62) v1 ==== 134+0+0 (secure 0 0 0) 0x7fda1c01d3c0 con 0x7fda2c0a2cd0 2024-09-15T01:33:52.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.735+0000 7fda217fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T01:33:52.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.735+0000 7fda217fa640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo' 2024-09-15T01:33:52.738 INFO:tasks.workunit.client.0.smithi204.stderr:already renamed to 'foo' 2024-09-15T01:33:52.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.736+0000 7fda3379d640 1 -- 172.21.15.204:0/1370209727 >> v2:172.21.15.204:6812/28210 conn(0x7fda100397e0 msgr2=0x7fda1003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:52.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.736+0000 7fda3379d640 1 --2- 172.21.15.204:0/1370209727 >> v2:172.21.15.204:6812/28210 conn(0x7fda100397e0 0x7fda1003bc90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fda28007fd0 tx=0x7fda28007e10 comp rx=0 tx=0).stop 2024-09-15T01:33:52.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.736+0000 7fda3379d640 10 monclient: shutdown 2024-09-15T01:33:52.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.736+0000 7fda3379d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:52.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.736+0000 7fda3379d640 1 -- 172.21.15.204:0/1370209727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda2c0a2cd0 msgr2=0x7fda2c134710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:52.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.736+0000 7fda3379d640 1 --2- 172.21.15.204:0/1370209727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda2c0a2cd0 0x7fda2c134710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00d890 tx=0x7fda1c008040 comp rx=0 tx=0).stop 2024-09-15T01:33:52.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.737+0000 7fda3379d640 1 -- 172.21.15.204:0/1370209727 shutdown_connections 2024-09-15T01:33:52.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.737+0000 7fda3379d640 1 --2- 172.21.15.204:0/1370209727 >> v2:172.21.15.204:6812/28210 conn(0x7fda100397e0 0x7fda1003bc90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.737+0000 7fda3379d640 1 --2- 172.21.15.204:0/1370209727 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fda2c137c80 0x7fda2c134c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.737+0000 7fda3379d640 1 --2- 172.21.15.204:0/1370209727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda2c0a2cd0 0x7fda2c134710 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.737+0000 7fda3379d640 1 --2- 172.21.15.204:0/1370209727 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fda2c09fa20 0x7fda2c1341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.737+0000 7fda3379d640 1 -- 172.21.15.204:0/1370209727 >> 172.21.15.204:0/1370209727 conn(0x7fda2c06e860 msgr2=0x7fda2c0a3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:52.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.737+0000 7fda3379d640 1 -- 172.21.15.204:0/1370209727 shutdown_connections 2024-09-15T01:33:52.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.737+0000 7fda3379d640 1 -- 172.21.15.204:0/1370209727 wait complete. 2024-09-15T01:33:52.748 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rename foo-asdf foo 2024-09-15T01:33:52.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.820+0000 7f07fcc57640 10 monclient: get_monmap_and_config 2024-09-15T01:33:52.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.820+0000 7f07fcc57640 10 monclient: build_initial_monmap 2024-09-15T01:33:52.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.820+0000 7f07fcc57640 10 monclient: monmap: 2024-09-15T01:33:52.823 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:52.823 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:52.823 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:52.821812+0000 2024-09-15T01:33:52.823 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:52.821812+0000 2024-09-15T01:33:52.823 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:52.823 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:52.823 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.820+0000 7f07fcc57640 1 Processor -- start 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 1 -- start start 2024-09-15T01:33:52.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 10 monclient: init 2024-09-15T01:33:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 10 monclient: _add_conns ranks=[4,7,10,2,6,1,3,9,0,8,5] 2024-09-15T01:33:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 0x7f07f8132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 10 monclient(hunting): picked mon.noname-c con 0x7f07f80a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07f809fa20 0x7f07f809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 10 monclient(hunting): picked mon.noname-i con 0x7f07f809fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f07f80a08e0 0x7f07f80a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 10 monclient(hunting): picked mon.noname-f con 0x7f07f80a08e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f80a30a0 con 0x7f07f80a08e0 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07f80a0330 con 0x7f07f80a2cd0 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f07f80a1db0 con 0x7f07f809fa20 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07fcc57640 10 monclient(hunting): authenticate will time out at 848.464646s 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f6d9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f07f80a08e0 0x7f07f80a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f5d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07f809fa20 0x7f07f809fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f5d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07f809fa20 0x7f07f809fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:56854/0 (socket says 172.21.15.204:56854) 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f6d9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f07f80a08e0 0x7f07f80a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44028/0 (socket says 172.21.15.204:44028) 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f5d9b640 1 -- 172.21.15.204:0/3365649502 learned_addr learned my addr 172.21.15.204:0/3365649502 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f5d9b640 10 monclient(hunting): get_auth_request con 0x7f07f809fa20 auth_method 0 2024-09-15T01:33:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f6d9d640 10 monclient(hunting): get_auth_request con 0x7f07f80a08e0 auth_method 0 2024-09-15T01:33:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.821+0000 7f07f6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f659c640 1 --2- 172.21.15.204:0/3365649502 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 0x7f07f8132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f5d9b640 10 monclient(hunting): handle_auth_done global_id 4966 payload 274 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f5d9b640 1 -- 172.21.15.204:0/3365649502 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 msgr2=0x7f07f8132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f5d9b640 1 --2- 172.21.15.204:0/3365649502 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 0x7f07f8132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f5d9b640 1 -- 172.21.15.204:0/3365649502 >> v2:172.21.15.204:3300/0 conn(0x7f07f80a08e0 msgr2=0x7f07f80a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f5d9b640 1 --2- 172.21.15.204:0/3365649502 >> v2:172.21.15.204:3300/0 conn(0x7f07f80a08e0 0x7f07f80a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f5d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f5d9b640 1 monclient: found mon.noname-i 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f5d9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:52.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f5d9b640 1 -- 172.21.15.204:0/3365649502 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f80648a0 con 0x7f07f809fa20 2024-09-15T01:33:52.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f659c640 10 monclient: get_auth_request con 0x7f07f80a2cd0 auth_method 0 2024-09-15T01:33:52.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f6d9d640 1 --2- 172.21.15.204:0/3365649502 >> v2:172.21.15.204:3300/0 conn(0x7f07f80a08e0 0x7f07f80a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:52.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f659c640 1 --2- 172.21.15.204:0/3365649502 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 0x7f07f8132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:33:52.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f5d9b640 1 --2- 172.21.15.204:0/3365649502 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07f809fa20 0x7f07f809fdf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009910 tx=0x7f07ec0091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=49b0c9f1c472467e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:52.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f559a640 1 -- 172.21.15.204:0/3365649502 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f07ec02ac80 con 0x7f07f809fa20 2024-09-15T01:33:52.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.822+0000 7f07f559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:52.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07f559a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:52.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07f559a640 10 monclient: dump: 2024-09-15T01:33:52.840 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:52.840 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:52.840 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.840 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07f559a640 10 monclient: _finish_auth 0 2024-09-15T01:33:52.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07f559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07f559a640 1 -- 172.21.15.204:0/3365649502 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ec02ae00 con 0x7f07f809fa20 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 5 monclient: authenticate success, global_id 4966 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07f559a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07f559a640 1 -- 172.21.15.204:0/3365649502 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f07ec031ca0 con 0x7f07f809fa20 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 4 set_mon_vals no callback set 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07f559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07f559a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07f559a640 10 monclient: dump: 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:52.842 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.843 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.843 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:52.843 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:52.843 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:52.843 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:52.843 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:52.843 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:52.843 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:52.843 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:52.843 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:52.843 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 10 monclient: shutdown 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 1 -- 172.21.15.204:0/3365649502 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07f809fa20 msgr2=0x7f07f809fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 1 --2- 172.21.15.204:0/3365649502 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07f809fa20 0x7f07f809fdf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009910 tx=0x7f07ec0091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 1 -- 172.21.15.204:0/3365649502 shutdown_connections 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 1 --2- 172.21.15.204:0/3365649502 >> v2:172.21.15.204:3300/0 conn(0x7f07f80a08e0 0x7f07f80a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 1 --2- 172.21.15.204:0/3365649502 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f07f809fa20 0x7f07f809fdf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 1 --2- 172.21.15.204:0/3365649502 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 0x7f07f8132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 1 -- 172.21.15.204:0/3365649502 >> 172.21.15.204:0/3365649502 conn(0x7f07f806e860 msgr2=0x7f07f806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 1 -- 172.21.15.204:0/3365649502 shutdown_connections 2024-09-15T01:33:52.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.823+0000 7f07fcc57640 1 -- 172.21.15.204:0/3365649502 wait complete. 2024-09-15T01:33:52.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient: build_initial_monmap 2024-09-15T01:33:52.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient: monmap: 2024-09-15T01:33:52.845 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:52.845 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:52.845 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:52.848 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 1 Processor -- start 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 1 -- start start 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient: init 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient: _add_conns ranks=[3,2,4,7,8,1,5,6,0] 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07f809fa20 0x7f07f8130c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient(hunting): picked mon.noname-g con 0x7f07f809fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07f80a08e0 0x7f07f811d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient(hunting): picked mon.noname-b con 0x7f07f80a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 0x7f07f811dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient(hunting): picked mon.noname-c con 0x7f07f80a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f811fad0 con 0x7f07f80a08e0 2024-09-15T01:33:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f809d3d0 con 0x7f07f809fa20 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07f809cb60 con 0x7f07f80a2cd0 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07fcc57640 10 monclient(hunting): authenticate will time out at 848.467956s 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07f659c640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07f809fa20 msgr2=0x7f07f8130c10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.824+0000 7f07f659c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07f809fa20 0x7f07f8130c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 0x7f07f811dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f5d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07f80a08e0 0x7f07f811d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 0x7f07f811dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:38372/0 (socket says 172.21.15.204:38372) 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f5d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07f80a08e0 0x7f07f811d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:49306/0 (socket says 172.21.15.204:49306) 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 1 -- 172.21.15.204:0/315925251 learned_addr learned my addr 172.21.15.204:0/315925251 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 10 monclient(hunting): get_auth_request con 0x7f07f80a2cd0 auth_method 0 2024-09-15T01:33:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f5d9b640 10 monclient(hunting): get_auth_request con 0x7f07f80a08e0 auth_method 0 2024-09-15T01:33:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 10 monclient(hunting): handle_auth_done global_id 4819 payload 824 2024-09-15T01:33:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 1 -- 172.21.15.204:0/315925251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07f809fa20 msgr2=0x7f07f8130c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 1 --2- 172.21.15.204:0/315925251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07f809fa20 0x7f07f8130c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 1 -- 172.21.15.204:0/315925251 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07f80a08e0 msgr2=0x7f07f811d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 1 --2- 172.21.15.204:0/315925251 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07f80a08e0 0x7f07f811d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:33:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 1 monclient: found mon.noname-c 2024-09-15T01:33:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:33:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 1 -- 172.21.15.204:0/315925251 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f0009a20 con 0x7f07f80a2cd0 2024-09-15T01:33:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f5d9b640 1 --2- 172.21.15.204:0/315925251 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07f80a08e0 0x7f07f811d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07f6d9d640 1 --2- 172.21.15.204:0/315925251 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 0x7f07f811dc00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07f000c0f0 tx=0x7f07f000b040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.825+0000 7f07e67fc640 1 -- 172.21.15.204:0/315925251 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f07f000ced0 con 0x7f07f80a2cd0 2024-09-15T01:33:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07e67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:33:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07e67fc640 10 monclient: dump: 2024-09-15T01:33:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:52.856 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07e67fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07e67fc640 1 -- 172.21.15.204:0/315925251 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f000f040 con 0x7f07f80a2cd0 2024-09-15T01:33:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07fcc57640 5 monclient: authenticate success, global_id 4819 2024-09-15T01:33:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07fcc57640 10 monclient: _renew_subs 2024-09-15T01:33:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07fcc57640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07fcc57640 1 -- 172.21.15.204:0/315925251 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ec009450 con 0x7f07f80a2cd0 2024-09-15T01:33:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07e67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07e67fc640 1 -- 172.21.15.204:0/315925251 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f07f0009d60 con 0x7f07f80a2cd0 2024-09-15T01:33:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07e67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07e67fc640 10 monclient: dump: 2024-09-15T01:33:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:52.859 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07f4d99640 4 set_mon_vals no callback set 2024-09-15T01:33:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07fcc57640 10 monclient: _renew_subs 2024-09-15T01:33:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07fcc57640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07fcc57640 1 -- 172.21.15.204:0/315925251 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f8120d60 con 0x7f07f80a2cd0 2024-09-15T01:33:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07fcc57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07fcc57640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07fcc57640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07fcc57640 1 -- 172.21.15.204:0/315925251 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f8121190 con 0x7f07f80a2cd0 2024-09-15T01:33:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.826+0000 7f07e67fc640 1 -- 172.21.15.204:0/315925251 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f07f0003020 con 0x7f07f80a2cd0 2024-09-15T01:33:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.827+0000 7f07e67fc640 1 --2- 172.21.15.204:0/315925251 >> v2:172.21.15.204:6812/28210 conn(0x7f07d80397e0 0x7f07d803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.827+0000 7f07e67fc640 1 -- 172.21.15.204:0/315925251 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4306+0+0 (secure 0 0 0) 0x7f07f004be00 con 0x7f07f80a2cd0 2024-09-15T01:33:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.827+0000 7f07f659c640 1 --2- 172.21.15.204:0/315925251 >> v2:172.21.15.204:6812/28210 conn(0x7f07d80397e0 0x7f07d803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.827+0000 7f07f659c640 10 monclient: get_auth_request con 0x7f07d80397e0 auth_method 0 2024-09-15T01:33:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.827+0000 7f07f659c640 1 --2- 172.21.15.204:0/315925251 >> v2:172.21.15.204:6812/28210 conn(0x7f07d80397e0 0x7f07d803bc90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f07e8006fd0 tx=0x7f07e8008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.832+0000 7f07e67fc640 1 -- 172.21.15.204:0/315925251 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f07f0012940 con 0x7f07f80a2cd0 2024-09-15T01:33:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.832+0000 7f07e67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.832+0000 7f07e67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:52.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.988+0000 7f07fcc57640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T01:33:52.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.988+0000 7f07fcc57640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T01:33:52.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.988+0000 7f07fcc57640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:52.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.988+0000 7f07fcc57640 1 -- 172.21.15.204:0/315925251 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) v1 -- 0x7f07f8070b80 con 0x7f07f80a2cd0 2024-09-15T01:33:52.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.990+0000 7f07e67fc640 1 -- 172.21.15.204:0/315925251 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 already renamed to 'foo' v62) v1 ==== 134+0+0 (secure 0 0 0) 0x7f07f0018070 con 0x7f07f80a2cd0 2024-09-15T01:33:52.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.990+0000 7f07e67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T01:33:52.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:52.990+0000 7f07e67fc640 10 monclient: _finish_command 2 = system:0 already renamed to 'foo' 2024-09-15T01:33:53.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.086+0000 7f07fcc57640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T01:33:53.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.086+0000 7f07fcc57640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T01:33:53.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.086+0000 7f07fcc57640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:53.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.086+0000 7f07fcc57640 1 -- 172.21.15.204:0/315925251 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) v1 -- 0x7f07f811f2a0 con 0x7f07f80a2cd0 2024-09-15T01:33:53.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.087+0000 7f07e67fc640 1 -- 172.21.15.204:0/315925251 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 already renamed to 'foo' v62) v1 ==== 134+0+0 (secure 0 0 0) 0x7f07f001b360 con 0x7f07f80a2cd0 2024-09-15T01:33:53.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.087+0000 7f07e67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-09-15T01:33:53.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.087+0000 7f07e67fc640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo' 2024-09-15T01:33:53.090 INFO:tasks.workunit.client.0.smithi204.stderr:already renamed to 'foo' 2024-09-15T01:33:53.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 1 -- 172.21.15.204:0/315925251 >> v2:172.21.15.204:6812/28210 conn(0x7f07d80397e0 msgr2=0x7f07d803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:53.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 1 --2- 172.21.15.204:0/315925251 >> v2:172.21.15.204:6812/28210 conn(0x7f07d80397e0 0x7f07d803bc90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f07e8006fd0 tx=0x7f07e8008040 comp rx=0 tx=0).stop 2024-09-15T01:33:53.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 10 monclient: shutdown 2024-09-15T01:33:53.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:53.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 1 -- 172.21.15.204:0/315925251 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 msgr2=0x7f07f811dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:53.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 1 --2- 172.21.15.204:0/315925251 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 0x7f07f811dc00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07f000c0f0 tx=0x7f07f000b040 comp rx=0 tx=0).stop 2024-09-15T01:33:53.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 1 -- 172.21.15.204:0/315925251 shutdown_connections 2024-09-15T01:33:53.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 1 --2- 172.21.15.204:0/315925251 >> v2:172.21.15.204:6812/28210 conn(0x7f07d80397e0 0x7f07d803bc90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 1 --2- 172.21.15.204:0/315925251 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07f80a08e0 0x7f07f811d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 1 --2- 172.21.15.204:0/315925251 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f07f80a2cd0 0x7f07f811dc00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 1 --2- 172.21.15.204:0/315925251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07f809fa20 0x7f07f8130c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 1 -- 172.21.15.204:0/315925251 >> 172.21.15.204:0/315925251 conn(0x7f07f806e860 msgr2=0x7f07f80a5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:53.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 1 -- 172.21.15.204:0/315925251 shutdown_connections 2024-09-15T01:33:53.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.089+0000 7f07fcc57640 1 -- 172.21.15.204:0/315925251 wait complete. 2024-09-15T01:33:53.101 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rename bar-asdf bar 2024-09-15T01:33:53.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.173+0000 7ff8c784f640 10 monclient: get_monmap_and_config 2024-09-15T01:33:53.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.173+0000 7ff8c784f640 10 monclient: build_initial_monmap 2024-09-15T01:33:53.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.173+0000 7ff8c784f640 10 monclient: monmap: 2024-09-15T01:33:53.175 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:53.174571+0000 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:53.174571+0000 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:53.176 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:53.177 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:53.177 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:53.177 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:53.177 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:53.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.173+0000 7ff8c784f640 1 Processor -- start 2024-09-15T01:33:53.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.173+0000 7ff8c784f640 1 -- start start 2024-09-15T01:33:53.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.173+0000 7ff8c784f640 10 monclient: init 2024-09-15T01:33:53.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:53.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 10 monclient: _add_conns ranks=[8,10,5,3,6,4,7,1,0,9,2] 2024-09-15T01:33:53.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8c0067130 0x7ff8c0132ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 10 monclient(hunting): picked mon.noname-e con 0x7ff8c0067130 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:53.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff8c009c880 0x7ff8c009cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 10 monclient(hunting): picked mon.noname-f con 0x7ff8c009c880 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8c009d6a0 0x7ff8c009db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 10 monclient(hunting): picked mon.noname-h con 0x7ff8c009d6a0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c009d1b0 con 0x7ff8c009c880 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c009a8f0 con 0x7ff8c009d6a0 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0067500 con 0x7ff8c0067130 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c784f640 10 monclient(hunting): authenticate will time out at 848.817575s 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c4dea640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff8c009c880 0x7ff8c009cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:53.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c5dec640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8c009d6a0 0x7ff8c009db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c4dea640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff8c009c880 0x7ff8c009cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44042/0 (socket says 172.21.15.204:44042) 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c5dec640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8c009d6a0 0x7ff8c009db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49582/0 (socket says 172.21.15.204:49582) 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c4dea640 1 -- 172.21.15.204:0/1930914979 learned_addr learned my addr 172.21.15.204:0/1930914979 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c4dea640 10 monclient(hunting): get_auth_request con 0x7ff8c009c880 auth_method 0 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c4dea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c4dea640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c4dea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c5dec640 10 monclient(hunting): get_auth_request con 0x7ff8c009d6a0 auth_method 0 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c5dec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c5dec640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c5dec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c5dec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:53.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c5dec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c5dec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c4dea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c4dea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c4dea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c55eb640 1 --2- 172.21.15.204:0/1930914979 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8c0067130 0x7ff8c0132ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c55eb640 10 monclient(hunting): get_auth_request con 0x7ff8c0067130 auth_method 0 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c55eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c55eb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c55eb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c5dec640 10 monclient(hunting): handle_auth_done global_id 4928 payload 274 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.174+0000 7ff8c5dec640 1 -- 172.21.15.204:0/1930914979 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8c0067130 msgr2=0x7ff8c0132ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c5dec640 1 --2- 172.21.15.204:0/1930914979 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8c0067130 0x7ff8c0132ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c5dec640 1 -- 172.21.15.204:0/1930914979 >> v2:172.21.15.204:3300/0 conn(0x7ff8c009c880 msgr2=0x7ff8c009cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c5dec640 1 --2- 172.21.15.204:0/1930914979 >> v2:172.21.15.204:3300/0 conn(0x7ff8c009c880 0x7ff8c009cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c5dec640 10 monclient: _finish_hunting 0 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c5dec640 1 monclient: found mon.noname-h 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c5dec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c5dec640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c5dec640 1 -- 172.21.15.204:0/1930914979 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c00648a0 con 0x7ff8c009d6a0 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c4dea640 1 --2- 172.21.15.204:0/1930914979 >> v2:172.21.15.204:3300/0 conn(0x7ff8c009c880 0x7ff8c009cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c5dec640 1 --2- 172.21.15.204:0/1930914979 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8c009d6a0 0x7ff8c009db10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80098b0 tx=0x7ff8b80090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=da77a409df89f979 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8bffff640 1 -- 172.21.15.204:0/1930914979 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b802abe0 con 0x7ff8c009d6a0 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8bffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8bffff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8bffff640 10 monclient: dump: 2024-09-15T01:33:53.188 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:53.189 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8bffff640 10 monclient: _finish_auth 0 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8bffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8bffff640 1 -- 172.21.15.204:0/1930914979 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b802ad60 con 0x7ff8c009d6a0 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c784f640 5 monclient: authenticate success, global_id 4928 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c784f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8bffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c784f640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8bffff640 1 -- 172.21.15.204:0/1930914979 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b8031ca0 con 0x7ff8c009d6a0 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8c784f640 4 set_mon_vals no callback set 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8bffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8bffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:53.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.175+0000 7ff8bffff640 10 monclient: dump: 2024-09-15T01:33:53.191 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:53.191 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:53.191 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.191 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.191 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:53.191 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:53.191 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:53.191 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:53.191 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:53.191 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:53.191 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:53.191 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.176+0000 7ff8c784f640 10 monclient: shutdown 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.176+0000 7ff8c784f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.176+0000 7ff8c784f640 1 -- 172.21.15.204:0/1930914979 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8c009d6a0 msgr2=0x7ff8c009db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.176+0000 7ff8c784f640 1 --2- 172.21.15.204:0/1930914979 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8c009d6a0 0x7ff8c009db10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80098b0 tx=0x7ff8b80090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.176+0000 7ff8c784f640 1 -- 172.21.15.204:0/1930914979 shutdown_connections 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.176+0000 7ff8c784f640 1 --2- 172.21.15.204:0/1930914979 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8c009d6a0 0x7ff8c009db10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.176+0000 7ff8c784f640 1 --2- 172.21.15.204:0/1930914979 >> v2:172.21.15.204:3300/0 conn(0x7ff8c009c880 0x7ff8c009cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.176+0000 7ff8c784f640 1 --2- 172.21.15.204:0/1930914979 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8c0067130 0x7ff8c0132ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.176+0000 7ff8c784f640 1 -- 172.21.15.204:0/1930914979 >> 172.21.15.204:0/1930914979 conn(0x7ff8c00a2cb0 msgr2=0x7ff8c00a30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.176+0000 7ff8c784f640 1 -- 172.21.15.204:0/1930914979 shutdown_connections 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.176+0000 7ff8c784f640 1 -- 172.21.15.204:0/1930914979 wait complete. 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.177+0000 7ff8c784f640 10 monclient: build_initial_monmap 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.177+0000 7ff8c784f640 10 monclient: monmap: 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:53.193 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.177+0000 7ff8c784f640 1 Processor -- start 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.177+0000 7ff8c784f640 1 -- start start 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.177+0000 7ff8c784f640 10 monclient: init 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.177+0000 7ff8c784f640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.177+0000 7ff8c784f640 10 monclient: _add_conns ranks=[8,2,1,6,3,7,5,0,4] 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8c0067130 0x7ff8c01318c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 10 monclient(hunting): picked mon.noname-e con 0x7ff8c0067130 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c009c880 0x7ff8c006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 10 monclient(hunting): picked mon.noname-b con 0x7ff8c009c880 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c009d6a0 0x7ff8c006db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 10 monclient(hunting): picked mon.noname-f con 0x7ff8c009d6a0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c006f660 con 0x7ff8c009d6a0 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c009f3e0 con 0x7ff8c009c880 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c009a190 con 0x7ff8c0067130 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 10 monclient(hunting): authenticate will time out at 848.821049s 2024-09-15T01:33:53.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c009d6a0 0x7ff8c006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c009d6a0 0x7ff8c006db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44044/0 (socket says 172.21.15.204:44044) 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 1 -- 172.21.15.204:0/3230740941 learned_addr learned my addr 172.21.15.204:0/3230740941 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 10 monclient(hunting): get_auth_request con 0x7ff8c009d6a0 auth_method 0 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c4dea640 1 --2- 172.21.15.204:0/3230740941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c009c880 0x7ff8c006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 10 monclient(hunting): handle_auth_done global_id 5248 payload 824 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 1 -- 172.21.15.204:0/3230740941 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8c0067130 msgr2=0x7ff8c01318c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c55eb640 1 --2- 172.21.15.204:0/3230740941 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8c0067130 0x7ff8c01318c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:53.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 1 --2- 172.21.15.204:0/3230740941 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8c0067130 0x7ff8c01318c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 1 -- 172.21.15.204:0/3230740941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c009c880 msgr2=0x7ff8c006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 1 --2- 172.21.15.204:0/3230740941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c009c880 0x7ff8c006d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 10 monclient: _finish_hunting 0 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 1 monclient: found mon.noname-f 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 1 -- 172.21.15.204:0/3230740941 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b4009450 con 0x7ff8c009d6a0 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c5dec640 1 --2- 172.21.15.204:0/3230740941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c009d6a0 0x7ff8c006db20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009360 tx=0x7ff8b8032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bd7fa640 1 -- 172.21.15.204:0/3230740941 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b803ca90 con 0x7ff8c009d6a0 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bd7fa640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bd7fa640 10 monclient: dump: 2024-09-15T01:33:53.197 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:53.198 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bd7fa640 10 monclient: _finish_auth 0 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 5 monclient: authenticate success, global_id 5248 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bd7fa640 1 -- 172.21.15.204:0/3230740941 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b803cc30 con 0x7ff8c009d6a0 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 10 monclient: _renew_subs 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 1 -- 172.21.15.204:0/3230740941 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b8009390 con 0x7ff8c009d6a0 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bd7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bd7fa640 1 -- 172.21.15.204:0/3230740941 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b803cda0 con 0x7ff8c009d6a0 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bd7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:53.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bd7fa640 10 monclient: dump: 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:53.200 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 10 monclient: _renew_subs 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8bf7fe640 4 set_mon_vals no callback set 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.178+0000 7ff8c784f640 1 -- 172.21.15.204:0/3230740941 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c011e9e0 con 0x7ff8c009d6a0 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.179+0000 7ff8bd7fa640 1 -- 172.21.15.204:0/3230740941 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff8b803c610 con 0x7ff8c009d6a0 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.179+0000 7ff8c784f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.179+0000 7ff8c784f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.179+0000 7ff8c784f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.179+0000 7ff8c784f640 1 -- 172.21.15.204:0/3230740941 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c0070b80 con 0x7ff8c009d6a0 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.179+0000 7ff8bd7fa640 1 --2- 172.21.15.204:0/3230740941 >> v2:172.21.15.204:6812/28210 conn(0x7ff8ac0397b0 0x7ff8ac03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.179+0000 7ff8bd7fa640 1 -- 172.21.15.204:0/3230740941 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4306+0+0 (secure 0 0 0) 0x7ff8b806b780 con 0x7ff8c009d6a0 2024-09-15T01:33:53.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.179+0000 7ff8c55eb640 1 --2- 172.21.15.204:0/3230740941 >> v2:172.21.15.204:6812/28210 conn(0x7ff8ac0397b0 0x7ff8ac03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:53.202 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.179+0000 7ff8c55eb640 10 monclient: get_auth_request con 0x7ff8ac0397b0 auth_method 0 2024-09-15T01:33:53.202 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.180+0000 7ff8c55eb640 1 --2- 172.21.15.204:0/3230740941 >> v2:172.21.15.204:6812/28210 conn(0x7ff8ac0397b0 0x7ff8ac03bc60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009fd0 tx=0x7ff8b000b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:53.202 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.184+0000 7ff8bd7fa640 1 -- 172.21.15.204:0/3230740941 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff8b803c8c0 con 0x7ff8c009d6a0 2024-09-15T01:33:53.202 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.184+0000 7ff8bd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:53.202 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.184+0000 7ff8bd7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:53.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.341+0000 7ff8c784f640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-09-15T01:33:53.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.341+0000 7ff8c784f640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-09-15T01:33:53.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.341+0000 7ff8c784f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:53.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.341+0000 7ff8c784f640 1 -- 172.21.15.204:0/3230740941 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"} v 0) v1 -- 0x7ff8c009a4f0 con 0x7ff8c009d6a0 2024-09-15T01:33:53.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.638+0000 7ff8bd7fa640 1 -- 172.21.15.204:0/3230740941 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}]=0 v63) v1 ==== 110+0+0 (secure 0 0 0) 0x7ff8b8037030 con 0x7ff8c009d6a0 2024-09-15T01:33:53.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.638+0000 7ff8bd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-09-15T01:33:53.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.638+0000 7ff8bd7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:53.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.734+0000 7ff8c784f640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-09-15T01:33:53.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.734+0000 7ff8c784f640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-09-15T01:33:53.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.734+0000 7ff8c784f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:53.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.734+0000 7ff8c784f640 1 -- 172.21.15.204:0/3230740941 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"} v 0) v1 -- 0x7ff8c0142a40 con 0x7ff8c009d6a0 2024-09-15T01:33:53.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.735+0000 7ff8bd7fa640 1 -- 172.21.15.204:0/3230740941 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}]=0 already renamed to 'bar' v63) v1 ==== 134+0+0 (secure 0 0 0) 0x7ff8b8037020 con 0x7ff8c009d6a0 2024-09-15T01:33:53.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.735+0000 7ff8bd7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-09-15T01:33:53.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.735+0000 7ff8bd7fa640 10 monclient: _finish_command 3 = system:0 already renamed to 'bar' 2024-09-15T01:33:53.738 INFO:tasks.workunit.client.0.smithi204.stderr:already renamed to 'bar' 2024-09-15T01:33:53.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 1 -- 172.21.15.204:0/3230740941 >> v2:172.21.15.204:6812/28210 conn(0x7ff8ac0397b0 msgr2=0x7ff8ac03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:53.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 1 --2- 172.21.15.204:0/3230740941 >> v2:172.21.15.204:6812/28210 conn(0x7ff8ac0397b0 0x7ff8ac03bc60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009fd0 tx=0x7ff8b000b040 comp rx=0 tx=0).stop 2024-09-15T01:33:53.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 10 monclient: shutdown 2024-09-15T01:33:53.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:53.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 1 -- 172.21.15.204:0/3230740941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c009d6a0 msgr2=0x7ff8c006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:53.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 1 --2- 172.21.15.204:0/3230740941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c009d6a0 0x7ff8c006db20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009360 tx=0x7ff8b8032000 comp rx=0 tx=0).stop 2024-09-15T01:33:53.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 1 -- 172.21.15.204:0/3230740941 shutdown_connections 2024-09-15T01:33:53.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 1 --2- 172.21.15.204:0/3230740941 >> v2:172.21.15.204:6812/28210 conn(0x7ff8ac0397b0 0x7ff8ac03bc60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 1 --2- 172.21.15.204:0/3230740941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c009d6a0 0x7ff8c006db20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 1 --2- 172.21.15.204:0/3230740941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c009c880 0x7ff8c006d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 1 --2- 172.21.15.204:0/3230740941 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8c0067130 0x7ff8c01318c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 1 -- 172.21.15.204:0/3230740941 >> 172.21.15.204:0/3230740941 conn(0x7ff8c00a2cb0 msgr2=0x7ff8c00989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:53.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 1 -- 172.21.15.204:0/3230740941 shutdown_connections 2024-09-15T01:33:53.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.737+0000 7ff8c784f640 1 -- 172.21.15.204:0/3230740941 wait complete. 2024-09-15T01:33:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule ls 2024-09-15T01:33:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false grep foo-asdf 2024-09-15T01:33:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep foo-asdf 2024-09-15T01:33:53.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.820+0000 7f60ff39d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:53.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.820+0000 7f60ff39d640 10 monclient: build_initial_monmap 2024-09-15T01:33:53.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.820+0000 7f60ff39d640 10 monclient: monmap: 2024-09-15T01:33:53.823 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:53.823 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:53.823 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:53.822046+0000 2024-09-15T01:33:53.823 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:53.822046+0000 2024-09-15T01:33:53.823 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:53.823 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:53.823 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:53.823 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:53.823 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:53.824 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:53.824 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:53.824 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:53.824 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:53.824 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:53.824 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:53.824 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:53.824 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:53.824 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:53.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 1 Processor -- start 2024-09-15T01:33:53.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 1 -- start start 2024-09-15T01:33:53.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 10 monclient: init 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 10 monclient: _add_conns ranks=[9,3,8,5,7,1,0,4,2,10,6] 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f60f80a2e30 0x7f60f8132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 10 monclient(hunting): picked mon.noname-a con 0x7f60f80a2e30 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60f809fb80 0x7f60f809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 10 monclient(hunting): picked mon.noname-g con 0x7f60f809fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 0x7f60f80a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 10 monclient(hunting): picked mon.noname-e con 0x7f60f80a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f809e6c0 con 0x7f60f80a2e30 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f80a04b0 con 0x7f60f809fb80 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:53.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f60f80a3200 con 0x7f60f80a0a40 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60ff39d640 10 monclient(hunting): authenticate will time out at 849.464830s 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60fc938640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60f809fb80 msgr2=0x7f60f809ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.821+0000 7f60fc938640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60f809fb80 0x7f60f809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd93a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 0x7f60f80a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f60f80a2e30 0x7f60f8132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd93a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 0x7f60f80a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:34046/0 (socket says 172.21.15.204:34046) 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd93a640 1 -- 172.21.15.204:0/1188797422 learned_addr learned my addr 172.21.15.204:0/1188797422 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd93a640 10 monclient(hunting): get_auth_request con 0x7f60f80a0a40 auth_method 0 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd93a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd93a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd93a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:53.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 10 monclient(hunting): get_auth_request con 0x7f60f80a2e30 auth_method 0 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd93a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd93a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd93a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 10 monclient(hunting): handle_auth_done global_id 4959 payload 274 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 1 -- 172.21.15.204:0/1188797422 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 msgr2=0x7f60f80a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 1 --2- 172.21.15.204:0/1188797422 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 0x7f60f80a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 1 -- 172.21.15.204:0/1188797422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60f809fb80 msgr2=0x7f60f809ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 1 --2- 172.21.15.204:0/1188797422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60f809fb80 0x7f60f809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 10 monclient: _finish_hunting 0 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 1 monclient: found mon.noname-a 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd139640 1 -- 172.21.15.204:0/1188797422 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f804b720 con 0x7f60f80a2e30 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.822+0000 7f60fd93a640 1 --2- 172.21.15.204:0/1188797422 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 0x7f60f80a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60fd139640 1 --2- 172.21.15.204:0/1188797422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60f80a2e30 0x7f60f8132a60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009850 tx=0x7f60f40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802603237e9a201b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60effff640 1 -- 172.21.15.204:0/1188797422 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60f402abe0 con 0x7f60f80a2e30 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60effff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60effff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60effff640 10 monclient: dump: 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:53.837 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60effff640 10 monclient: _finish_auth 0 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60effff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60ff39d640 5 monclient: authenticate success, global_id 4959 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60effff640 1 -- 172.21.15.204:0/1188797422 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f402ad60 con 0x7f60f80a2e30 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60ff39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60effff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60ff39d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60ff39d640 4 set_mon_vals no callback set 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60effff640 1 -- 172.21.15.204:0/1188797422 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60f4031ca0 con 0x7f60f80a2e30 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60effff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60effff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.823+0000 7f60effff640 10 monclient: dump: 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:53.838 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:53.839 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 10 monclient: shutdown 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 1 -- 172.21.15.204:0/1188797422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60f80a2e30 msgr2=0x7f60f8132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 1 --2- 172.21.15.204:0/1188797422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60f80a2e30 0x7f60f8132a60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009850 tx=0x7f60f40090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 1 -- 172.21.15.204:0/1188797422 shutdown_connections 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 1 --2- 172.21.15.204:0/1188797422 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 0x7f60f80a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 1 --2- 172.21.15.204:0/1188797422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60f809fb80 0x7f60f809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 1 --2- 172.21.15.204:0/1188797422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60f80a2e30 0x7f60f8132a60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 1 -- 172.21.15.204:0/1188797422 >> 172.21.15.204:0/1188797422 conn(0x7f60f806e860 msgr2=0x7f60f806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 1 -- 172.21.15.204:0/1188797422 shutdown_connections 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 1 -- 172.21.15.204:0/1188797422 wait complete. 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 10 monclient: build_initial_monmap 2024-09-15T01:33:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 10 monclient: monmap: 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.824+0000 7f60ff39d640 1 Processor -- start 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 1 -- start start 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 10 monclient: init 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 10 monclient: _add_conns ranks=[6,8,2,4,7,0,3,1,5] 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f60f809fb80 0x7f60f8131850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 10 monclient(hunting): picked mon.noname-d con 0x7f60f809fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 0x7f60f811e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 10 monclient(hunting): picked mon.noname-e con 0x7f60f80a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60f80a2e30 0x7f60f811e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 10 monclient(hunting): picked mon.noname-b con 0x7f60f80a2e30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:53.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f809cab0 con 0x7f60f80a2e30 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f60f809d650 con 0x7f60f809fb80 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f60f8142970 con 0x7f60f80a0a40 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60ff39d640 10 monclient(hunting): authenticate will time out at 849.468576s 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fd139640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f60f809fb80 msgr2=0x7f60f8131850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fd139640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f60f809fb80 0x7f60f8131850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fc938640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 0x7f60f811e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fd93a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60f80a2e30 0x7f60f811e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fc938640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 0x7f60f811e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:34062/0 (socket says 172.21.15.204:34062) 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fd93a640 10 monclient(hunting): get_auth_request con 0x7f60f80a2e30 auth_method 0 2024-09-15T01:33:53.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fc938640 1 -- 172.21.15.204:0/2833238662 learned_addr learned my addr 172.21.15.204:0/2833238662 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fd93a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fd93a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fd93a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fc938640 10 monclient(hunting): get_auth_request con 0x7f60f80a0a40 auth_method 0 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fc938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fc938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fc938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fd93a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fd93a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fd93a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fc938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fc938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.825+0000 7f60fc938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:53.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60fd93a640 10 monclient(hunting): handle_auth_done global_id 4871 payload 824 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60fd93a640 1 -- 172.21.15.204:0/2833238662 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 msgr2=0x7f60f811e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60fd93a640 1 --2- 172.21.15.204:0/2833238662 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 0x7f60f811e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60fd93a640 1 -- 172.21.15.204:0/2833238662 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f60f809fb80 msgr2=0x7f60f8131850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60fd93a640 1 --2- 172.21.15.204:0/2833238662 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f60f809fb80 0x7f60f8131850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60fd93a640 10 monclient: _finish_hunting 0 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60fd93a640 1 monclient: found mon.noname-b 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60fd93a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60fd93a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60fd93a640 1 -- 172.21.15.204:0/2833238662 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f0009ae0 con 0x7f60f80a2e30 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60fc938640 1 --2- 172.21.15.204:0/2833238662 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 0x7f60f811e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60fd93a640 1 --2- 172.21.15.204:0/2833238662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60f80a2e30 0x7f60f811e8d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009f10 tx=0x7f60f000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ed7fa640 1 -- 172.21.15.204:0/2833238662 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60f0018070 con 0x7f60f80a2e30 2024-09-15T01:33:53.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ed7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ed7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ed7fa640 10 monclient: dump: 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:53.846 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ed7fa640 10 monclient: _finish_auth 0 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ed7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ed7fa640 1 -- 172.21.15.204:0/2833238662 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f000f040 con 0x7f60f80a2e30 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ff39d640 5 monclient: authenticate success, global_id 4871 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ff39d640 10 monclient: _renew_subs 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ff39d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ff39d640 1 -- 172.21.15.204:0/2833238662 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f4009390 con 0x7f60f80a2e30 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ed7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ed7fa640 1 -- 172.21.15.204:0/2833238662 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60f00092e0 con 0x7f60f80a2e30 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ed7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:53.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ed7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ed7fa640 10 monclient: dump: 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:53.848 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ff39d640 10 monclient: _renew_subs 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ef7fe640 4 set_mon_vals no callback set 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ff39d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.826+0000 7f60ff39d640 1 -- 172.21.15.204:0/2833238662 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f81219a0 con 0x7f60f80a2e30 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.827+0000 7f60ff39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.827+0000 7f60ff39d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.827+0000 7f60ff39d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.827+0000 7f60ff39d640 1 -- 172.21.15.204:0/2833238662 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f8070cc0 con 0x7f60f80a2e30 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.827+0000 7f60ed7fa640 1 -- 172.21.15.204:0/2833238662 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f60f0003020 con 0x7f60f80a2e30 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.827+0000 7f60ed7fa640 1 --2- 172.21.15.204:0/2833238662 >> v2:172.21.15.204:6812/28210 conn(0x7f60dc0397b0 0x7f60dc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:53.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.827+0000 7f60ed7fa640 1 -- 172.21.15.204:0/2833238662 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4301+0+0 (secure 0 0 0) 0x7f60f004b360 con 0x7f60f80a2e30 2024-09-15T01:33:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.827+0000 7f60fd139640 1 --2- 172.21.15.204:0/2833238662 >> v2:172.21.15.204:6812/28210 conn(0x7f60dc0397b0 0x7f60dc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.827+0000 7f60fd139640 10 monclient: get_auth_request con 0x7f60dc0397b0 auth_method 0 2024-09-15T01:33:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.828+0000 7f60fd139640 1 --2- 172.21.15.204:0/2833238662 >> v2:172.21.15.204:6812/28210 conn(0x7f60dc0397b0 0x7f60dc03bc60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60f4034000 tx=0x7f60f4032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.832+0000 7f60ed7fa640 1 -- 172.21.15.204:0/2833238662 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f60f000ce30 con 0x7f60f80a2e30 2024-09-15T01:33:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.832+0000 7f60ed7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.832+0000 7f60ed7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:53.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.988+0000 7f60ff39d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:53.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.988+0000 7f60ff39d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:53.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.988+0000 7f60ff39d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:53.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.988+0000 7f60ff39d640 1 -- 172.21.15.204:0/2833238662 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f60f80a1f10 con 0x7f60f80a2e30 2024-09-15T01:33:53.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.988+0000 7f60ed7fa640 1 -- 172.21.15.204:0/2833238662 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v63) v1 ==== 65+0+60 (secure 0 0 0) 0x7f60f001de60 con 0x7f60f80a2e30 2024-09-15T01:33:53.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.988+0000 7f60ed7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:53.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:53.988+0000 7f60ed7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:54.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.082+0000 7f60ff39d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:54.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.083+0000 7f60ff39d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:54.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.083+0000 7f60ff39d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:33:54.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.083+0000 7f60ff39d640 1 -- 172.21.15.204:0/2833238662 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f60f81325f0 con 0x7f60f80a2e30 2024-09-15T01:33:54.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.083+0000 7f60ed7fa640 1 -- 172.21.15.204:0/2833238662 <== mon.2 v2:172.21.15.178:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v63) v1 ==== 65+0+60 (secure 0 0 0) 0x7f60f003f050 con 0x7f60f80a2e30 2024-09-15T01:33:54.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.083+0000 7f60ed7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:54.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.083+0000 7f60ed7fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:54.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 1 -- 172.21.15.204:0/2833238662 >> v2:172.21.15.204:6812/28210 conn(0x7f60dc0397b0 msgr2=0x7f60dc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:54.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 1 --2- 172.21.15.204:0/2833238662 >> v2:172.21.15.204:6812/28210 conn(0x7f60dc0397b0 0x7f60dc03bc60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60f4034000 tx=0x7f60f4032000 comp rx=0 tx=0).stop 2024-09-15T01:33:54.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 10 monclient: shutdown 2024-09-15T01:33:54.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:54.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 1 -- 172.21.15.204:0/2833238662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60f80a2e30 msgr2=0x7f60f811e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:54.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 1 --2- 172.21.15.204:0/2833238662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60f80a2e30 0x7f60f811e8d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009f10 tx=0x7f60f000b040 comp rx=0 tx=0).stop 2024-09-15T01:33:54.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 1 -- 172.21.15.204:0/2833238662 shutdown_connections 2024-09-15T01:33:54.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 1 --2- 172.21.15.204:0/2833238662 >> v2:172.21.15.204:6812/28210 conn(0x7f60dc0397b0 0x7f60dc03bc60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 1 --2- 172.21.15.204:0/2833238662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60f80a2e30 0x7f60f811e8d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 1 --2- 172.21.15.204:0/2833238662 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f60f80a0a40 0x7f60f811e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 1 --2- 172.21.15.204:0/2833238662 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f60f809fb80 0x7f60f8131850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 1 -- 172.21.15.204:0/2833238662 >> 172.21.15.204:0/2833238662 conn(0x7f60f806e860 msgr2=0x7f60f809bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:54.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 1 -- 172.21.15.204:0/2833238662 shutdown_connections 2024-09-15T01:33:54.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.085+0000 7f60ff39d640 1 -- 172.21.15.204:0/2833238662 wait complete. 2024-09-15T01:33:54.097 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:54.097 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule ls 2024-09-15T01:33:54.097 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false grep bar-asdf 2024-09-15T01:33:54.097 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:54.097 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep bar-asdf 2024-09-15T01:33:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.168+0000 7fe04495d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.168+0000 7fe04495d640 10 monclient: build_initial_monmap 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.168+0000 7fe04495d640 10 monclient: monmap: 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:54.170007+0000 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:54.170007+0000 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:54.172 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 1 Processor -- start 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 1 -- start start 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 10 monclient: init 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 10 monclient: _add_conns ranks=[2,6,7,8,1,0,9,4,5,10,3] 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0400a2e30 0x7fe040132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 10 monclient(hunting): picked mon.noname-b con 0x7fe0400a2e30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe04009fb80 0x7fe04009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 10 monclient(hunting): picked mon.noname-d con 0x7fe04009fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0400a0a40 0x7fe0400a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 10 monclient(hunting): picked mon.noname-i con 0x7fe0400a0a40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:54.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04009e6c0 con 0x7fe0400a2e30 2024-09-15T01:33:54.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:54.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe0400a04b0 con 0x7fe04009fb80 2024-09-15T01:33:54.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:54.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe0400a3200 con 0x7fe0400a0a40 2024-09-15T01:33:54.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:54.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe04495d640 10 monclient(hunting): authenticate will time out at 849.812834s 2024-09-15T01:33:54.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe03dd9b640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe04009fb80 msgr2=0x7fe04009ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:54.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe03ed9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0400a0a40 0x7fe0400a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:54.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.169+0000 7fe03dd9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe04009fb80 0x7fe04009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:54.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0400a0a40 0x7fe0400a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:56858/0 (socket says 172.21.15.204:56858) 2024-09-15T01:33:54.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 1 -- 172.21.15.204:0/2813531921 learned_addr learned my addr 172.21.15.204:0/2813531921 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:54.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 10 monclient(hunting): get_auth_request con 0x7fe0400a0a40 auth_method 0 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03e59c640 1 --2- 172.21.15.204:0/2813531921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0400a2e30 0x7fe040132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03e59c640 10 monclient(hunting): get_auth_request con 0x7fe0400a2e30 auth_method 0 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:54.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 10 monclient(hunting): handle_auth_done global_id 4975 payload 274 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 1 -- 172.21.15.204:0/2813531921 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe04009fb80 msgr2=0x7fe04009ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 1 --2- 172.21.15.204:0/2813531921 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe04009fb80 0x7fe04009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 1 -- 172.21.15.204:0/2813531921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0400a2e30 msgr2=0x7fe040132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 1 --2- 172.21.15.204:0/2813531921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0400a2e30 0x7fe040132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 1 monclient: found mon.noname-i 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 1 -- 172.21.15.204:0/2813531921 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04004b720 con 0x7fe0400a0a40 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.170+0000 7fe03ed9d640 1 --2- 172.21.15.204:0/2813531921 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0400a0a40 0x7fe0400a0e90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe038009f50 tx=0x7fe038009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=2c669eb358e06b3b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe03d59a640 1 -- 172.21.15.204:0/2813531921 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe038007880 con 0x7fe0400a0a40 2024-09-15T01:33:54.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe03d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe03d59a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe03d59a640 10 monclient: dump: 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:54.185 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe03d59a640 10 monclient: _finish_auth 0 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe03d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe03d59a640 1 -- 172.21.15.204:0/2813531921 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03802d050 con 0x7fe0400a0a40 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 5 monclient: authenticate success, global_id 4975 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe03d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe03d59a640 1 -- 172.21.15.204:0/2813531921 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe038030360 con 0x7fe0400a0a40 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe03d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:54.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 4 set_mon_vals no callback set 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe03d59a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe03d59a640 10 monclient: dump: 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:54.187 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 10 monclient: shutdown 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 1 -- 172.21.15.204:0/2813531921 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0400a0a40 msgr2=0x7fe0400a0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 1 --2- 172.21.15.204:0/2813531921 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0400a0a40 0x7fe0400a0e90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe038009f50 tx=0x7fe038009840 comp rx=0 tx=0).stop 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 1 -- 172.21.15.204:0/2813531921 shutdown_connections 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 1 --2- 172.21.15.204:0/2813531921 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0400a0a40 0x7fe0400a0e90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 1 --2- 172.21.15.204:0/2813531921 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe04009fb80 0x7fe04009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 1 --2- 172.21.15.204:0/2813531921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0400a2e30 0x7fe040132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 1 -- 172.21.15.204:0/2813531921 >> 172.21.15.204:0/2813531921 conn(0x7fe04006e860 msgr2=0x7fe04006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 1 -- 172.21.15.204:0/2813531921 shutdown_connections 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.171+0000 7fe04495d640 1 -- 172.21.15.204:0/2813531921 wait complete. 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.172+0000 7fe04495d640 10 monclient: build_initial_monmap 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.172+0000 7fe04495d640 10 monclient: monmap: 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:54.189 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.172+0000 7fe04495d640 1 Processor -- start 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.172+0000 7fe04495d640 1 -- start start 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.172+0000 7fe04495d640 10 monclient: init 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.172+0000 7fe04495d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.172+0000 7fe04495d640 10 monclient: _add_conns ranks=[4,8,6,3,7,2,5,1,0] 2024-09-15T01:33:54.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe04009fb80 0x7fe040131890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 10 monclient(hunting): picked mon.noname-c con 0x7fe04009fb80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe0400a0a40 0x7fe04011e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 10 monclient(hunting): picked mon.noname-e con 0x7fe0400a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe0400a2e30 0x7fe04011ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 10 monclient(hunting): picked mon.noname-d con 0x7fe0400a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe04009d650 con 0x7fe04009fb80 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe040121cc0 con 0x7fe0400a2e30 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe0401219e0 con 0x7fe0400a0a40 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe04495d640 10 monclient(hunting): authenticate will time out at 849.816049s 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03ed9d640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe0400a2e30 msgr2=0x7fe04011ea80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:54.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03ed9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe0400a2e30 0x7fe04011ea80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:54.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe04009fb80 0x7fe040131890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:54.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03dd9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe0400a0a40 0x7fe04011e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:54.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe04009fb80 0x7fe040131890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:38374/0 (socket says 172.21.15.204:38374) 2024-09-15T01:33:54.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03dd9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe0400a0a40 0x7fe04011e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:34064/0 (socket says 172.21.15.204:34064) 2024-09-15T01:33:54.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 1 -- 172.21.15.204:0/3020220590 learned_addr learned my addr 172.21.15.204:0/3020220590 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:54.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 10 monclient(hunting): get_auth_request con 0x7fe04009fb80 auth_method 0 2024-09-15T01:33:54.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:54.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:54.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:54.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03dd9b640 10 monclient(hunting): get_auth_request con 0x7fe0400a0a40 auth_method 0 2024-09-15T01:33:54.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:54.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:54.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:54.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:54.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:54.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:54.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:54.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:54.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:54.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 10 monclient(hunting): handle_auth_done global_id 4828 payload 824 2024-09-15T01:33:54.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 1 -- 172.21.15.204:0/3020220590 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe0400a0a40 msgr2=0x7fe04011e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:54.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 1 --2- 172.21.15.204:0/3020220590 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe0400a0a40 0x7fe04011e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 1 -- 172.21.15.204:0/3020220590 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe0400a2e30 msgr2=0x7fe04011ea80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:54.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 1 --2- 172.21.15.204:0/3020220590 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe0400a2e30 0x7fe04011ea80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 1 monclient: found mon.noname-c 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 1 -- 172.21.15.204:0/3020220590 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe038009ae0 con 0x7fe04009fb80 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03dd9b640 1 --2- 172.21.15.204:0/3020220590 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe0400a0a40 0x7fe04011e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe03e59c640 1 --2- 172.21.15.204:0/3020220590 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe04009fb80 0x7fe040131890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe03000e900 tx=0x7fe0300090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.173+0000 7fe02e7fc640 1 -- 172.21.15.204:0/3020220590 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe030009480 con 0x7fe04009fb80 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe02e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe02e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe02e7fc640 10 monclient: dump: 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:54.194 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:54.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe02e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe02e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe02e7fc640 1 -- 172.21.15.204:0/3020220590 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03000cd60 con 0x7fe04009fb80 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe02e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe02e7fc640 1 -- 172.21.15.204:0/3020220590 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0300138f0 con 0x7fe04009fb80 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe02e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe02e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe02e7fc640 10 monclient: dump: 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:54.196 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe03cd99640 4 set_mon_vals no callback set 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe04495d640 5 monclient: authenticate success, global_id 4828 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe04495d640 10 monclient: _renew_subs 2024-09-15T01:33:54.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe04495d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe04495d640 1 -- 172.21.15.204:0/3020220590 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04011feb0 con 0x7fe04009fb80 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe04495d640 10 monclient: _renew_subs 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe04495d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe04495d640 1 -- 172.21.15.204:0/3020220590 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe040133f50 con 0x7fe04009fb80 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.174+0000 7fe02e7fc640 1 -- 172.21.15.204:0/3020220590 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe030013470 con 0x7fe04009fb80 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.175+0000 7fe04495d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.175+0000 7fe04495d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.175+0000 7fe04495d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.175+0000 7fe04495d640 1 -- 172.21.15.204:0/3020220590 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe040070cc0 con 0x7fe04009fb80 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.175+0000 7fe02e7fc640 1 --2- 172.21.15.204:0/3020220590 >> v2:172.21.15.204:6812/28210 conn(0x7fe0200397b0 0x7fe02003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.175+0000 7fe02e7fc640 1 -- 172.21.15.204:0/3020220590 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4301+0+0 (secure 0 0 0) 0x7fe03004b7c0 con 0x7fe04009fb80 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.175+0000 7fe03dd9b640 1 --2- 172.21.15.204:0/3020220590 >> v2:172.21.15.204:6812/28210 conn(0x7fe0200397b0 0x7fe02003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.175+0000 7fe03dd9b640 10 monclient: get_auth_request con 0x7fe0200397b0 auth_method 0 2024-09-15T01:33:54.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.175+0000 7fe03dd9b640 1 --2- 172.21.15.204:0/3020220590 >> v2:172.21.15.204:6812/28210 conn(0x7fe0200397b0 0x7fe02003bc60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe034006fd0 tx=0x7fe0340090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:54.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.180+0000 7fe02e7fc640 1 -- 172.21.15.204:0/3020220590 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe03001b990 con 0x7fe04009fb80 2024-09-15T01:33:54.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.180+0000 7fe02e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:54.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.180+0000 7fe02e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:54.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.336+0000 7fe04495d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:54.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.336+0000 7fe04495d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:54.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.336+0000 7fe04495d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:54.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.336+0000 7fe04495d640 1 -- 172.21.15.204:0/3020220590 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7fe0400a1f10 con 0x7fe04009fb80 2024-09-15T01:33:54.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.337+0000 7fe02e7fc640 1 -- 172.21.15.204:0/3020220590 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v63) v1 ==== 65+0+60 (secure 0 0 0) 0x7fe03001b3b0 con 0x7fe04009fb80 2024-09-15T01:33:54.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.337+0000 7fe02e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:54.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.337+0000 7fe02e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:54.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.431+0000 7fe04495d640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:54.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.431+0000 7fe04495d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:54.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.431+0000 7fe04495d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:54.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.431+0000 7fe04495d640 1 -- 172.21.15.204:0/3020220590 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7fe0400008d0 con 0x7fe04009fb80 2024-09-15T01:33:54.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.431+0000 7fe02e7fc640 1 -- 172.21.15.204:0/3020220590 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v63) v1 ==== 65+0+60 (secure 0 0 0) 0x7fe03001b990 con 0x7fe04009fb80 2024-09-15T01:33:54.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.431+0000 7fe02e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-09-15T01:33:54.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.431+0000 7fe02e7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:33:54.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.433+0000 7fe04495d640 1 -- 172.21.15.204:0/3020220590 >> v2:172.21.15.204:6812/28210 conn(0x7fe0200397b0 msgr2=0x7fe02003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:54.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.433+0000 7fe04495d640 1 --2- 172.21.15.204:0/3020220590 >> v2:172.21.15.204:6812/28210 conn(0x7fe0200397b0 0x7fe02003bc60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe034006fd0 tx=0x7fe0340090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.433+0000 7fe04495d640 10 monclient: shutdown 2024-09-15T01:33:54.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.433+0000 7fe04495d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:54.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.433+0000 7fe04495d640 1 -- 172.21.15.204:0/3020220590 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe04009fb80 msgr2=0x7fe040131890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:54.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.433+0000 7fe04495d640 1 --2- 172.21.15.204:0/3020220590 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe04009fb80 0x7fe040131890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe03000e900 tx=0x7fe0300090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.433+0000 7fe04495d640 1 -- 172.21.15.204:0/3020220590 shutdown_connections 2024-09-15T01:33:54.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.433+0000 7fe04495d640 1 --2- 172.21.15.204:0/3020220590 >> v2:172.21.15.204:6812/28210 conn(0x7fe0200397b0 0x7fe02003bc60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.433+0000 7fe04495d640 1 --2- 172.21.15.204:0/3020220590 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe0400a2e30 0x7fe04011ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.433+0000 7fe04495d640 1 --2- 172.21.15.204:0/3020220590 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe0400a0a40 0x7fe04011e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.433+0000 7fe04495d640 1 --2- 172.21.15.204:0/3020220590 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe04009fb80 0x7fe040131890 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.433+0000 7fe04495d640 1 -- 172.21.15.204:0/3020220590 >> 172.21.15.204:0/3020220590 conn(0x7fe04006e860 msgr2=0x7fe04009bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:54.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.434+0000 7fe04495d640 1 -- 172.21.15.204:0/3020220590 shutdown_connections 2024-09-15T01:33:54.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.434+0000 7fe04495d640 1 -- 172.21.15.204:0/3020220590 wait complete. 2024-09-15T01:33:54.445 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:54.445 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rm foo 2024-09-15T01:33:54.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.517+0000 7f0e9c19d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:54.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.517+0000 7f0e9c19d640 10 monclient: build_initial_monmap 2024-09-15T01:33:54.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.517+0000 7f0e9c19d640 10 monclient: monmap: 2024-09-15T01:33:54.519 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:54.520 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:54.520 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:54.518662+0000 2024-09-15T01:33:54.520 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:54.518662+0000 2024-09-15T01:33:54.520 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:54.520 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:54.520 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:54.520 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:54.520 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:54.520 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:54.520 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:54.520 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:54.520 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:54.521 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:54.521 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:54.521 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:54.521 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:54.521 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:54.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.517+0000 7f0e9c19d640 1 Processor -- start 2024-09-15T01:33:54.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.517+0000 7f0e9c19d640 1 -- start start 2024-09-15T01:33:54.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.517+0000 7f0e9c19d640 10 monclient: init 2024-09-15T01:33:54.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:54.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 10 monclient: _add_conns ranks=[8,6,10,9,5,7,3,2,4,0,1] 2024-09-15T01:33:54.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0e940a35e0 0x7f0e94099a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 10 monclient(hunting): picked mon.noname-e con 0x7f0e940a35e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:54.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0e9409c560 0x7f0e9409d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 10 monclient(hunting): picked mon.noname-d con 0x7f0e9409c560 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:54.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0e9409c950 0x7f0e9409dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 10 monclient(hunting): picked mon.noname-f con 0x7f0e9409c950 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e94099f50 con 0x7f0e9409c950 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9409d340 con 0x7f0e9409c560 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0e940a39b0 con 0x7f0e940a35e0 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9c19d640 10 monclient(hunting): authenticate will time out at 850.161495s 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e99738640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0e9409c560 msgr2=0x7f0e9409d840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e99738640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0e9409c560 0x7f0e9409d840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0e9409c950 0x7f0e9409dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0e9409c950 0x7f0e9409dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44054/0 (socket says 172.21.15.204:44054) 2024-09-15T01:33:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 1 -- 172.21.15.204:0/1352012220 learned_addr learned my addr 172.21.15.204:0/1352012220 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 10 monclient(hunting): get_auth_request con 0x7f0e9409c950 auth_method 0 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e99f39640 1 --2- 172.21.15.204:0/1352012220 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0e940a35e0 0x7f0e94099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e99f39640 10 monclient(hunting): get_auth_request con 0x7f0e940a35e0 auth_method 0 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e99f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e99f39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e99f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e99f39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e99f39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e99f39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 10 monclient(hunting): handle_auth_done global_id 5257 payload 274 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 1 -- 172.21.15.204:0/1352012220 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0e940a35e0 msgr2=0x7f0e94099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 1 --2- 172.21.15.204:0/1352012220 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0e940a35e0 0x7f0e94099a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 1 -- 172.21.15.204:0/1352012220 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0e9409c560 msgr2=0x7f0e9409d840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 1 --2- 172.21.15.204:0/1352012220 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0e9409c560 0x7f0e9409d840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 10 monclient: _finish_hunting 0 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 1 monclient: found mon.noname-f 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.518+0000 7f0e9a73a640 1 -- 172.21.15.204:0/1352012220 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9404b720 con 0x7f0e9409c950 2024-09-15T01:33:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e9a73a640 1 --2- 172.21.15.204:0/1352012220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e9409c950 0x7f0e9409dd80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00d1a0 tx=0x7f0e8c00ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5760a5e836d74eef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e98f37640 1 -- 172.21.15.204:0/1352012220 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0e8c00b2e0 con 0x7f0e9409c950 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e98f37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e98f37640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e98f37640 10 monclient: dump: 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:54.534 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:54.535 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:54.535 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:54.535 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:54.535 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:54.535 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:54.535 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:54.535 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:54.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e98f37640 10 monclient: _finish_auth 0 2024-09-15T01:33:54.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e98f37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:54.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e98f37640 1 -- 172.21.15.204:0/1352012220 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c00b460 con 0x7f0e9409c950 2024-09-15T01:33:54.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e9c19d640 5 monclient: authenticate success, global_id 5257 2024-09-15T01:33:54.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e9c19d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e98f37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e9c19d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e98f37640 1 -- 172.21.15.204:0/1352012220 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0e8c0333a0 con 0x7f0e9409c950 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e9c19d640 4 set_mon_vals no callback set 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e98f37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e98f37640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.519+0000 7f0e98f37640 10 monclient: dump: 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:54.536 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 10 monclient: shutdown 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1352012220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e9409c950 msgr2=0x7f0e9409dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:54.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 1 --2- 172.21.15.204:0/1352012220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e9409c950 0x7f0e9409dd80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00d1a0 tx=0x7f0e8c00ba50 comp rx=0 tx=0).stop 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1352012220 shutdown_connections 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 1 --2- 172.21.15.204:0/1352012220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e9409c950 0x7f0e9409dd80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 1 --2- 172.21.15.204:0/1352012220 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0e9409c560 0x7f0e9409d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 1 --2- 172.21.15.204:0/1352012220 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0e940a35e0 0x7f0e94099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1352012220 >> 172.21.15.204:0/1352012220 conn(0x7f0e940687e0 msgr2=0x7f0e940a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1352012220 shutdown_connections 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1352012220 wait complete. 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 10 monclient: build_initial_monmap 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.520+0000 7f0e9c19d640 10 monclient: monmap: 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:54.538 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 1 Processor -- start 2024-09-15T01:33:54.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 1 -- start start 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 10 monclient: init 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 10 monclient: _add_conns ranks=[4,1,8,5,2,0,6,7,3] 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0e9409c560 0x7f0e9413e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 10 monclient(hunting): picked mon.noname-c con 0x7f0e9409c560 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e9409c950 0x7f0e9413da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 10 monclient(hunting): picked mon.noname-f con 0x7f0e9409c950 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0e940a35e0 0x7f0e94141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 10 monclient(hunting): picked mon.noname-e con 0x7f0e940a35e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e941233c0 con 0x7f0e9409c950 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94122e00 con 0x7f0e9409c560 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94122b20 con 0x7f0e940a35e0 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9c19d640 10 monclient(hunting): authenticate will time out at 850.164709s 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e99738640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e9409c950 0x7f0e9413da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e99738640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e9409c950 0x7f0e9413da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44066/0 (socket says 172.21.15.204:44066) 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e99738640 1 -- 172.21.15.204:0/1799389825 learned_addr learned my addr 172.21.15.204:0/1799389825 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e99738640 10 monclient(hunting): get_auth_request con 0x7f0e9409c950 auth_method 0 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e99738640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e99738640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e99738640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e99738640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e99738640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e99738640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9a73a640 1 --2- 172.21.15.204:0/1799389825 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0e940a35e0 0x7f0e94141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9a73a640 10 monclient(hunting): get_auth_request con 0x7f0e940a35e0 auth_method 0 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9a73a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e99f39640 1 --2- 172.21.15.204:0/1799389825 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0e9409c560 0x7f0e9413e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9a73a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.521+0000 7f0e9a73a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99f39640 10 monclient(hunting): get_auth_request con 0x7f0e9409c560 auth_method 0 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99f39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99738640 10 monclient(hunting): handle_auth_done global_id 5266 payload 824 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99738640 1 -- 172.21.15.204:0/1799389825 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0e940a35e0 msgr2=0x7f0e94141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:54.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99738640 1 --2- 172.21.15.204:0/1799389825 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0e940a35e0 0x7f0e94141940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99738640 1 -- 172.21.15.204:0/1799389825 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0e9409c560 msgr2=0x7f0e9413e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99738640 1 --2- 172.21.15.204:0/1799389825 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0e9409c560 0x7f0e9413e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99738640 10 monclient: _finish_hunting 0 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99738640 1 monclient: found mon.noname-f 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99738640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99738640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99738640 1 -- 172.21.15.204:0/1799389825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c00bcf0 con 0x7f0e9409c950 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e99738640 1 --2- 172.21.15.204:0/1799389825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e9409c950 0x7f0e9413da80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400b5e0 tx=0x7f0e84006f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e89ffb640 1 -- 172.21.15.204:0/1799389825 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0e84006150 con 0x7f0e9409c950 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e89ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e89ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e89ffb640 10 monclient: dump: 2024-09-15T01:33:54.543 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:54.544 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e89ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e89ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e89ffb640 1 -- 172.21.15.204:0/1799389825 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e840062f0 con 0x7f0e9409c950 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e89ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e89ffb640 1 -- 172.21.15.204:0/1799389825 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0e840103e0 con 0x7f0e9409c950 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e9c19d640 5 monclient: authenticate success, global_id 5266 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e9c19d640 10 monclient: _renew_subs 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e9c19d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1799389825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e941230e0 con 0x7f0e9409c950 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e8bfff640 4 set_mon_vals no callback set 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e89ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e89ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:54.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e89ffb640 10 monclient: dump: 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:54.546 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e9c19d640 10 monclient: _renew_subs 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e9c19d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.522+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1799389825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9413dfc0 con 0x7f0e9409c950 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.523+0000 7f0e89ffb640 1 -- 172.21.15.204:0/1799389825 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0e840064a0 con 0x7f0e9409c950 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.523+0000 7f0e89ffb640 1 --2- 172.21.15.204:0/1799389825 >> v2:172.21.15.204:6812/28210 conn(0x7f0e780397b0 0x7f0e7803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.523+0000 7f0e89ffb640 1 -- 172.21.15.204:0/1799389825 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4301+0+0 (secure 0 0 0) 0x7f0e8404a610 con 0x7f0e9409c950 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.523+0000 7f0e99f39640 1 --2- 172.21.15.204:0/1799389825 >> v2:172.21.15.204:6812/28210 conn(0x7f0e780397b0 0x7f0e7803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.523+0000 7f0e99f39640 10 monclient: get_auth_request con 0x7f0e780397b0 auth_method 0 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.523+0000 7f0e9c19d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.523+0000 7f0e9c19d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:54.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.523+0000 7f0e9c19d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:54.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.523+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1799389825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e94070cc0 con 0x7f0e9409c950 2024-09-15T01:33:54.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.523+0000 7f0e99f39640 1 --2- 172.21.15.204:0/1799389825 >> v2:172.21.15.204:6812/28210 conn(0x7f0e780397b0 0x7f0e7803bc60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009c10 tx=0x7f0e90008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:54.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.528+0000 7f0e89ffb640 1 -- 172.21.15.204:0/1799389825 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0e84017070 con 0x7f0e9409c950 2024-09-15T01:33:54.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.528+0000 7f0e89ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:54.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.528+0000 7f0e89ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.688+0000 7f0e9c19d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T01:33:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.689+0000 7f0e9c19d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T01:33:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.689+0000 7f0e9c19d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:54.689+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1799389825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) v1 -- 0x7f0e94069880 con 0x7f0e9409c950 2024-09-15T01:33:55.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.648+0000 7f0e89ffb640 1 -- 172.21.15.204:0/1799389825 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 v64) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0e840108a0 con 0x7f0e9409c950 2024-09-15T01:33:55.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.648+0000 7f0e89ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T01:33:55.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.648+0000 7f0e89ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:55.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.748+0000 7f0e9c19d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T01:33:55.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.748+0000 7f0e9c19d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T01:33:55.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.748+0000 7f0e9c19d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:55.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.748+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1799389825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) v1 -- 0x7f0e94121c50 con 0x7f0e9409c950 2024-09-15T01:33:55.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.749+0000 7f0e89ffb640 1 -- 172.21.15.204:0/1799389825 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v64) v1 ==== 103+0+0 (secure 0 0 0) 0x7f0e84023030 con 0x7f0e9409c950 2024-09-15T01:33:55.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.749+0000 7f0e89ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T01:33:55.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.749+0000 7f0e89ffb640 10 monclient: _finish_command 3 = system:0 rule foo does not exist 2024-09-15T01:33:55.752 INFO:tasks.workunit.client.0.smithi204.stderr:rule foo does not exist 2024-09-15T01:33:55.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1799389825 >> v2:172.21.15.204:6812/28210 conn(0x7f0e780397b0 msgr2=0x7f0e7803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:55.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 1 --2- 172.21.15.204:0/1799389825 >> v2:172.21.15.204:6812/28210 conn(0x7f0e780397b0 0x7f0e7803bc60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009c10 tx=0x7f0e90008040 comp rx=0 tx=0).stop 2024-09-15T01:33:55.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 10 monclient: shutdown 2024-09-15T01:33:55.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:55.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1799389825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e9409c950 msgr2=0x7f0e9413da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:55.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 1 --2- 172.21.15.204:0/1799389825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e9409c950 0x7f0e9413da80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400b5e0 tx=0x7f0e84006f90 comp rx=0 tx=0).stop 2024-09-15T01:33:55.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1799389825 shutdown_connections 2024-09-15T01:33:55.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 1 --2- 172.21.15.204:0/1799389825 >> v2:172.21.15.204:6812/28210 conn(0x7f0e780397b0 0x7f0e7803bc60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:55.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 1 --2- 172.21.15.204:0/1799389825 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0e940a35e0 0x7f0e94141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:55.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 1 --2- 172.21.15.204:0/1799389825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e9409c950 0x7f0e9413da80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:55.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 1 --2- 172.21.15.204:0/1799389825 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0e9409c560 0x7f0e9413e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:55.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1799389825 >> 172.21.15.204:0/1799389825 conn(0x7f0e940687e0 msgr2=0x7f0e940a1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:55.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1799389825 shutdown_connections 2024-09-15T01:33:55.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.751+0000 7f0e9c19d640 1 -- 172.21.15.204:0/1799389825 wait complete. 2024-09-15T01:33:55.762 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rm foo 2024-09-15T01:33:55.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.833+0000 7f588bb1d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:55.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.833+0000 7f588bb1d640 10 monclient: build_initial_monmap 2024-09-15T01:33:55.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.833+0000 7f588bb1d640 10 monclient: monmap: 2024-09-15T01:33:55.836 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:55.836 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:55.837 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:55.835300+0000 2024-09-15T01:33:55.837 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:55.835300+0000 2024-09-15T01:33:55.837 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:55.837 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:55.837 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:55.837 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:55.837 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:55.837 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:55.837 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:55.837 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:55.837 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:55.837 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:55.838 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:55.838 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:55.838 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:55.838 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:55.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.834+0000 7f588bb1d640 1 Processor -- start 2024-09-15T01:33:55.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.834+0000 7f588bb1d640 1 -- start start 2024-09-15T01:33:55.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.834+0000 7f588bb1d640 10 monclient: init 2024-09-15T01:33:55.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:55.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 10 monclient: _add_conns ranks=[3,9,1,7,5,8,6,0,4,10,2] 2024-09-15T01:33:55.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58840a2e30 0x7f5884132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:55.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 10 monclient(hunting): picked mon.noname-g con 0x7f58840a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:55.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f588409fb80 0x7f588409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:55.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 10 monclient(hunting): picked mon.noname-a con 0x7f588409fb80 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:55.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f58840a0a40 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588404b720 con 0x7f58840a0a40 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588409e6c0 con 0x7f588409fb80 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58840a04b0 con 0x7f58840a2e30 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 10 monclient(hunting): authenticate will time out at 851.478184s 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58898b9640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58840a2e30 msgr2=0x7f5884132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58898b9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58840a2e30 0x7f5884132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588a0ba640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f58840a0a40 0x7f58840a0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:34466/0 (socket says 172.21.15.204:34466) 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588a0ba640 1 -- 172.21.15.204:0/2512638549 learned_addr learned my addr 172.21.15.204:0/2512638549 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831850233 0 0) 0x7f588404b720 con 0x7f58840a0a40 2024-09-15T01:33:55.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient(hunting): my global_id is 5275 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58700034e0 con 0x7f58840a0a40 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58890b8640 1 --2- 172.21.15.204:0/2512638549 >> v2:172.21.15.178:3300/0 conn(0x7f588409fb80 0x7f588409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58890b8640 10 monclient(hunting): get_auth_request con 0x7f588409fb80 auth_method 0 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58890b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58890b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58890b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58890b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58890b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58890b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3899642459 0 0) 0x7f58700034e0 con 0x7f58840a0a40 2024-09-15T01:33:55.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58840a2e30 msgr2=0x7f5884132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 --2- 172.21.15.204:0/2512638549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58840a2e30 0x7f5884132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 >> v2:172.21.15.178:3300/0 conn(0x7f588409fb80 msgr2=0x7f588409ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 --2- 172.21.15.204:0/2512638549 >> v2:172.21.15.178:3300/0 conn(0x7f588409fb80 0x7f588409ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient: _finish_hunting 0 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840a3200 con 0x7f58840a0a40 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient: _finish_auth 0 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f587c0018f0 con 0x7f58840a0a40 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 5 monclient: authenticate success, global_id 5275 2024-09-15T01:33:55.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient: dump: 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:55.853 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:55.854 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:55.854 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:55.854 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:55.854 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:55.854 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:55.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:33:55.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:55.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 >> v1:172.21.15.204:6789/0 conn(0x7f58840a0a40 legacy=0x7f58840a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:55.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient: _add_conns ranks=[6,2,7,5,8,1,0,3,4] 2024-09-15T01:33:55.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 --2- 172.21.15.204:0/2512638549 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5870004220 0x7f5870004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:55.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient(hunting): picked mon.d con 0x7f5870004220 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:55.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 --2- 172.21.15.204:0/2512638549 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5870004d10 0x7f5870005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient(hunting): picked mon.b con 0x7f5870004d10 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 --2- 172.21.15.204:0/2512638549 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 0x7f587000c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient(hunting): picked mon.i con 0x7f5870009c40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588409e6c0 con 0x7f5870004d10 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f58840a04b0 con 0x7f5870004220 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f587000c9b0 con 0x7f5870009c40 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58888b7640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f588bb1d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.835+0000 7f58898b9640 1 -- 172.21.15.204:0/2512638549 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5870004220 msgr2=0x7f5870004780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f58898b9640 1 --2- 172.21.15.204:0/2512638549 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5870004220 0x7f5870004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:55.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 1 --2- 172.21.15.204:0/2512638549 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 0x7f587000c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f58890b8640 1 --2- 172.21.15.204:0/2512638549 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5870004d10 0x7f5870005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 10 monclient(hunting): get_auth_request con 0x7f5870009c40 auth_method 0 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f58890b8640 10 monclient(hunting): get_auth_request con 0x7f5870004d10 auth_method 0 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f58890b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f58890b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f58890b8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:55.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f58890b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f58890b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f58890b8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 10 monclient(hunting): handle_auth_done global_id 5275 payload 293 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 1 -- 172.21.15.204:0/2512638549 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5870004220 msgr2=0x7f5870004780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 1 --2- 172.21.15.204:0/2512638549 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5870004220 0x7f5870004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 1 -- 172.21.15.204:0/2512638549 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5870004d10 msgr2=0x7f5870005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 1 --2- 172.21.15.204:0/2512638549 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5870004d10 0x7f5870005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 10 monclient: _finish_hunting 0 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 1 monclient: found mon.i 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 1 -- 172.21.15.204:0/2512638549 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58700034e0 con 0x7f5870009c40 2024-09-15T01:33:55.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 10 monclient: _finish_auth 0 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.836+0000 7f588a0ba640 1 -- 172.21.15.204:0/2512638549 reap_dead start 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588a0ba640 1 --2- 172.21.15.204:0/2512638549 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 0x7f587000c0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f587c007f70 tx=0x7f587c001650 comp rx=0 tx=0).ready entity=mon.7 client_cookie=6a5abcde2ce1e782 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f587c005d10 con 0x7f5870009c40 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f58888b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f58888b7640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f58888b7640 10 monclient: dump: 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:55.858 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c02d070 con 0x7f5870009c40 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588bb1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f58888b7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f58888b7640 1 -- 172.21.15.204:0/2512638549 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f587c007570 con 0x7f5870009c40 2024-09-15T01:33:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588bb1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588bb1d640 4 set_mon_vals no callback set 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f58888b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f58888b7640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f58888b7640 10 monclient: dump: 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588bb1d640 10 monclient: shutdown 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588bb1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588bb1d640 1 -- 172.21.15.204:0/2512638549 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 msgr2=0x7f587000c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588bb1d640 1 --2- 172.21.15.204:0/2512638549 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 0x7f587000c0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f587c007f70 tx=0x7f587c001650 comp rx=0 tx=0).stop 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588bb1d640 1 -- 172.21.15.204:0/2512638549 shutdown_connections 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588bb1d640 1 --2- 172.21.15.204:0/2512638549 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 0x7f587000c0a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588bb1d640 1 -- 172.21.15.204:0/2512638549 >> 172.21.15.204:0/2512638549 conn(0x7f588406e860 msgr2=0x7f588406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588bb1d640 1 -- 172.21.15.204:0/2512638549 shutdown_connections 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.837+0000 7f588bb1d640 1 -- 172.21.15.204:0/2512638549 wait complete. 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.838+0000 7f588bb1d640 10 monclient: build_initial_monmap 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.838+0000 7f588bb1d640 10 monclient: monmap: 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:55.863 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.838+0000 7f588bb1d640 1 Processor -- start 2024-09-15T01:33:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.838+0000 7f588bb1d640 1 -- start start 2024-09-15T01:33:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.838+0000 7f588bb1d640 10 monclient: init 2024-09-15T01:33:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 10 monclient: _add_conns ranks=[7,3,4,5,2,1,0,8,6] 2024-09-15T01:33:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 0x7f5884130830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 10 monclient(hunting): picked mon.noname-i con 0x7f5870009c40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5884130d70 0x7f588411d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 10 monclient(hunting): picked mon.noname-g con 0x7f5884130d70 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f588411d870 0x7f588411dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 10 monclient(hunting): picked mon.noname-c con 0x7f588411d870 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5884120a80 con 0x7f5884130d70 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5884131140 con 0x7f588411d870 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5884120fe0 con 0x7f5870009c40 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58890b8640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5884130d70 msgr2=0x7f588411d330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 10 monclient(hunting): authenticate will time out at 851.482220s 2024-09-15T01:33:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58890b8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5884130d70 0x7f588411d330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 0x7f5884130830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 0x7f5884130830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:56872/0 (socket says 172.21.15.204:56872) 2024-09-15T01:33:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 1 -- 172.21.15.204:0/4095917183 learned_addr learned my addr 172.21.15.204:0/4095917183 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 10 monclient(hunting): get_auth_request con 0x7f5870009c40 auth_method 0 2024-09-15T01:33:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588a0ba640 1 --2- 172.21.15.204:0/4095917183 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f588411d870 0x7f588411dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588a0ba640 10 monclient(hunting): get_auth_request con 0x7f588411d870 auth_method 0 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588a0ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588a0ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588a0ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 10 monclient(hunting): handle_auth_done global_id 4984 payload 824 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 1 -- 172.21.15.204:0/4095917183 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f588411d870 msgr2=0x7f588411dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 1 --2- 172.21.15.204:0/4095917183 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f588411d870 0x7f588411dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 1 -- 172.21.15.204:0/4095917183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5884130d70 msgr2=0x7f588411d330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 1 --2- 172.21.15.204:0/4095917183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5884130d70 0x7f588411d330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 10 monclient: _finish_hunting 0 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 1 monclient: found mon.noname-i 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 1 -- 172.21.15.204:0/4095917183 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588409d650 con 0x7f5870009c40 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588a0ba640 1 --2- 172.21.15.204:0/4095917183 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f588411d870 0x7f588411dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f58898b9640 1 --2- 172.21.15.204:0/4095917183 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 0x7f5884130830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f588000bbb0 tx=0x7f58800097f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f5879ffb640 1 -- 172.21.15.204:0/4095917183 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5880006150 con 0x7f5870009c40 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f5879ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f5879ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f5879ffb640 10 monclient: dump: 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:55.868 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f5879ffb640 10 monclient: _finish_auth 0 2024-09-15T01:33:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f5879ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f5879ffb640 1 -- 172.21.15.204:0/4095917183 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58800062f0 con 0x7f5870009c40 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 5 monclient: authenticate success, global_id 4984 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 10 monclient: _renew_subs 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f588bb1d640 1 -- 172.21.15.204:0/4095917183 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5884142970 con 0x7f5870009c40 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f5879ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f5879ffb640 1 -- 172.21.15.204:0/4095917183 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5880006920 con 0x7f5870009c40 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f5879ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f5879ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f5879ffb640 10 monclient: dump: 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.839+0000 7f587bfff640 4 set_mon_vals no callback set 2024-09-15T01:33:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.840+0000 7f588bb1d640 10 monclient: _renew_subs 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.840+0000 7f588bb1d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.840+0000 7f588bb1d640 1 -- 172.21.15.204:0/4095917183 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588411f5c0 con 0x7f5870009c40 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.840+0000 7f5879ffb640 1 -- 172.21.15.204:0/4095917183 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5880012020 con 0x7f5870009c40 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.841+0000 7f5879ffb640 1 --2- 172.21.15.204:0/4095917183 >> v2:172.21.15.204:6812/28210 conn(0x7f5868039710 0x7f586803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.841+0000 7f5879ffb640 1 -- 172.21.15.204:0/4095917183 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4246+0+0 (secure 0 0 0) 0x7f588004b280 con 0x7f5870009c40 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.841+0000 7f588bb1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.841+0000 7f588bb1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.841+0000 7f588bb1d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.841+0000 7f588bb1d640 1 -- 172.21.15.204:0/4095917183 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5884070cc0 con 0x7f5870009c40 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.841+0000 7f58890b8640 1 --2- 172.21.15.204:0/4095917183 >> v2:172.21.15.204:6812/28210 conn(0x7f5868039710 0x7f586803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.841+0000 7f58890b8640 10 monclient: get_auth_request con 0x7f5868039710 auth_method 0 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.841+0000 7f58890b8640 1 --2- 172.21.15.204:0/4095917183 >> v2:172.21.15.204:6812/28210 conn(0x7f5868039710 0x7f586803bbc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5870005760 tx=0x7f58740090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.845+0000 7f5879ffb640 1 -- 172.21.15.204:0/4095917183 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f588001b080 con 0x7f5870009c40 2024-09-15T01:33:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.845+0000 7f5879ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:55.845+0000 7f5879ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:56.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.002+0000 7f588bb1d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T01:33:56.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.002+0000 7f588bb1d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T01:33:56.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.002+0000 7f588bb1d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:56.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.002+0000 7f588bb1d640 1 -- 172.21.15.204:0/4095917183 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) v1 -- 0x7f58840a1f10 con 0x7f5870009c40 2024-09-15T01:33:56.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.003+0000 7f5879ffb640 1 -- 172.21.15.204:0/4095917183 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v64) v1 ==== 103+0+0 (secure 0 0 0) 0x7f588001d530 con 0x7f5870009c40 2024-09-15T01:33:56.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.003+0000 7f5879ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T01:33:56.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.003+0000 7f5879ffb640 10 monclient: _finish_command 2 = system:0 rule foo does not exist 2024-09-15T01:33:56.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.097+0000 7f588bb1d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T01:33:56.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.097+0000 7f588bb1d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T01:33:56.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.097+0000 7f588bb1d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:33:56.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.097+0000 7f588bb1d640 1 -- 172.21.15.204:0/4095917183 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) v1 -- 0x7f58840a5040 con 0x7f5870009c40 2024-09-15T01:33:56.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.098+0000 7f5879ffb640 1 -- 172.21.15.204:0/4095917183 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v64) v1 ==== 103+0+0 (secure 0 0 0) 0x7f588001e350 con 0x7f5870009c40 2024-09-15T01:33:56.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.098+0000 7f5879ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-09-15T01:33:56.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.098+0000 7f5879ffb640 10 monclient: _finish_command 3 = system:0 rule foo does not exist 2024-09-15T01:33:56.101 INFO:tasks.workunit.client.0.smithi204.stderr:rule foo does not exist 2024-09-15T01:33:56.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 1 -- 172.21.15.204:0/4095917183 >> v2:172.21.15.204:6812/28210 conn(0x7f5868039710 msgr2=0x7f586803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:56.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 1 --2- 172.21.15.204:0/4095917183 >> v2:172.21.15.204:6812/28210 conn(0x7f5868039710 0x7f586803bbc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5870005760 tx=0x7f58740090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 10 monclient: shutdown 2024-09-15T01:33:56.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:56.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 1 -- 172.21.15.204:0/4095917183 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 msgr2=0x7f5884130830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:56.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 1 --2- 172.21.15.204:0/4095917183 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 0x7f5884130830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f588000bbb0 tx=0x7f58800097f0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 1 -- 172.21.15.204:0/4095917183 shutdown_connections 2024-09-15T01:33:56.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 1 --2- 172.21.15.204:0/4095917183 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f588411d870 0x7f588411dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 1 --2- 172.21.15.204:0/4095917183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5884130d70 0x7f588411d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 1 --2- 172.21.15.204:0/4095917183 >> v2:172.21.15.204:6812/28210 conn(0x7f5868039710 0x7f586803bbc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 1 --2- 172.21.15.204:0/4095917183 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5870009c40 0x7f5884130830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 1 -- 172.21.15.204:0/4095917183 >> 172.21.15.204:0/4095917183 conn(0x7f588406e860 msgr2=0x7f58840a3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:56.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 1 -- 172.21.15.204:0/4095917183 shutdown_connections 2024-09-15T01:33:56.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.100+0000 7f588bb1d640 1 -- 172.21.15.204:0/4095917183 wait complete. 2024-09-15T01:33:56.111 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rm bar 2024-09-15T01:33:56.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.184+0000 7f4ea199d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:56.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.184+0000 7f4ea199d640 10 monclient: build_initial_monmap 2024-09-15T01:33:56.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.184+0000 7f4ea199d640 10 monclient: monmap: 2024-09-15T01:33:56.186 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:56.186 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:56.186 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:56.185557+0000 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:56.185557+0000 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:56.187 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:56.188 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:56.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.184+0000 7f4ea199d640 1 Processor -- start 2024-09-15T01:33:56.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.184+0000 7f4ea199d640 1 -- start start 2024-09-15T01:33:56.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.184+0000 7f4ea199d640 10 monclient: init 2024-09-15T01:33:56.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:56.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 10 monclient: _add_conns ranks=[4,7,3,5,2,8,0,10,9,1,6] 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4e9c0a0b40 0x7f4e9c130bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 10 monclient(hunting): picked mon.noname-c con 0x7f4e9c0a0b40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4e9c098940 0x7f4e9c067130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 10 monclient(hunting): picked mon.noname-i con 0x7f4e9c098940 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c098d50 0x7f4e9c067670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 10 monclient(hunting): picked mon.noname-g con 0x7f4e9c098d50 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0a0f10 con 0x7f4e9c098d50 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c02a7c0 con 0x7f4e9c0a0b40 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c09c1c0 con 0x7f4e9c098940 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:56.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4ea199d640 10 monclient(hunting): authenticate will time out at 851.828203s 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e9b7fe640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c098d50 msgr2=0x7f4e9c067670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e9b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c098d50 0x7f4e9c067670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4e9c098940 0x7f4e9c067130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4e9c098940 0x7f4e9c067130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:56888/0 (socket says 172.21.15.204:56888) 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 1 -- 172.21.15.204:0/4189314387 learned_addr learned my addr 172.21.15.204:0/4189314387 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 10 monclient(hunting): get_auth_request con 0x7f4e9c098940 auth_method 0 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:56.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e9affd640 1 --2- 172.21.15.204:0/4189314387 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4e9c0a0b40 0x7f4e9c130bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e9affd640 10 monclient(hunting): get_auth_request con 0x7f4e9c0a0b40 auth_method 0 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e9affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e9affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e9affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 10 monclient(hunting): handle_auth_done global_id 4993 payload 274 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 1 -- 172.21.15.204:0/4189314387 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4e9c0a0b40 msgr2=0x7f4e9c130bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 1 --2- 172.21.15.204:0/4189314387 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4e9c0a0b40 0x7f4e9c130bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 1 -- 172.21.15.204:0/4189314387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c098d50 msgr2=0x7f4e9c067670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 1 --2- 172.21.15.204:0/4189314387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c098d50 0x7f4e9c067670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 10 monclient: _finish_hunting 0 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 1 monclient: found mon.noname-i 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:33:56.199 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.185+0000 7f4e927fc640 1 -- 172.21.15.204:0/4189314387 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c04b720 con 0x7f4e9c098940 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e927fc640 1 --2- 172.21.15.204:0/4189314387 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4e9c098940 0x7f4e9c067130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009940 tx=0x7f4e840091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=cb69a5ad5c550dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e9a7fc640 1 -- 172.21.15.204:0/4189314387 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e8402ac80 con 0x7f4e9c098940 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e9a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e9a7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e9a7fc640 10 monclient: dump: 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:56.200 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e9a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e9a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e9a7fc640 1 -- 172.21.15.204:0/4189314387 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8402ae00 con 0x7f4e9c098940 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e9a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:56.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e9a7fc640 1 -- 172.21.15.204:0/4189314387 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e84031ca0 con 0x7f4e9c098940 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4ea199d640 5 monclient: authenticate success, global_id 4993 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4ea199d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4ea199d640 4 set_mon_vals no callback set 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e9a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e9a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4e9a7fc640 10 monclient: dump: 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:56.202 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4ea199d640 10 monclient: shutdown 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4ea199d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4ea199d640 1 -- 172.21.15.204:0/4189314387 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4e9c098940 msgr2=0x7f4e9c067130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.186+0000 7f4ea199d640 1 --2- 172.21.15.204:0/4189314387 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4e9c098940 0x7f4e9c067130 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009940 tx=0x7f4e840091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.203 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.187+0000 7f4ea199d640 1 -- 172.21.15.204:0/4189314387 shutdown_connections 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.187+0000 7f4ea199d640 1 --2- 172.21.15.204:0/4189314387 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4e9c098940 0x7f4e9c067130 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.187+0000 7f4ea199d640 1 --2- 172.21.15.204:0/4189314387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c098d50 0x7f4e9c067670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.187+0000 7f4ea199d640 1 --2- 172.21.15.204:0/4189314387 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4e9c0a0b40 0x7f4e9c130bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.187+0000 7f4ea199d640 1 -- 172.21.15.204:0/4189314387 >> 172.21.15.204:0/4189314387 conn(0x7f4e9c0a2e30 msgr2=0x7f4e9c0a0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.187+0000 7f4ea199d640 1 -- 172.21.15.204:0/4189314387 shutdown_connections 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.187+0000 7f4ea199d640 1 -- 172.21.15.204:0/4189314387 wait complete. 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.187+0000 7f4ea199d640 10 monclient: build_initial_monmap 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.187+0000 7f4ea199d640 10 monclient: monmap: 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:56.204 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.187+0000 7f4ea199d640 1 Processor -- start 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 1 -- start start 2024-09-15T01:33:56.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 10 monclient: init 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 10 monclient: _add_conns ranks=[5,3,8,7,6,0,2,4,1] 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4e9c098940 0x7f4e9c095330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 10 monclient(hunting): picked mon.noname-h con 0x7f4e9c098940 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c098d50 0x7f4e9c095870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 10 monclient(hunting): picked mon.noname-g con 0x7f4e9c098d50 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4e9c0a0b40 0x7f4e9c06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 10 monclient(hunting): picked mon.noname-e con 0x7f4e9c0a0b40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c06d4f0 con 0x7f4e9c098d50 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c09bdd0 con 0x7f4e9c098940 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:56.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c12b480 con 0x7f4e9c0a0b40 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4ea199d640 10 monclient(hunting): authenticate will time out at 851.831496s 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e927fc640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c098d50 msgr2=0x7f4e9c095870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e927fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c098d50 0x7f4e9c095870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4e9c098940 0x7f4e9c095330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4e9c098940 0x7f4e9c095330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49596/0 (socket says 172.21.15.204:49596) 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 1 -- 172.21.15.204:0/307094096 learned_addr learned my addr 172.21.15.204:0/307094096 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 10 monclient(hunting): get_auth_request con 0x7f4e9c098940 auth_method 0 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9b7fe640 1 --2- 172.21.15.204:0/307094096 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4e9c0a0b40 0x7f4e9c06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:56.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9b7fe640 10 monclient(hunting): get_auth_request con 0x7f4e9c0a0b40 auth_method 0 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 10 monclient(hunting): handle_auth_done global_id 4937 payload 824 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 1 -- 172.21.15.204:0/307094096 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4e9c0a0b40 msgr2=0x7f4e9c06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 1 --2- 172.21.15.204:0/307094096 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4e9c0a0b40 0x7f4e9c06b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 1 -- 172.21.15.204:0/307094096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c098d50 msgr2=0x7f4e9c095870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:56.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 1 --2- 172.21.15.204:0/307094096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c098d50 0x7f4e9c095870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 10 monclient: _finish_hunting 0 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 1 monclient: found mon.noname-h 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.188+0000 7f4e9affd640 1 -- 172.21.15.204:0/307094096 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e84009450 con 0x7f4e9c098940 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e9affd640 1 --2- 172.21.15.204:0/307094096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4e9c098940 0x7f4e9c095330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00d8c0 tx=0x7f4e8c008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e937fe640 1 -- 172.21.15.204:0/307094096 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e8c00de50 con 0x7f4e9c098940 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e937fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e937fe640 10 monclient: dump: 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.209 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e937fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:56.210 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4ea199d640 5 monclient: authenticate success, global_id 4937 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e937fe640 1 -- 172.21.15.204:0/307094096 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c00bd50 con 0x7f4e9c098940 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4ea199d640 10 monclient: _renew_subs 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4ea199d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4ea199d640 1 -- 172.21.15.204:0/307094096 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c06eea0 con 0x7f4e9c098940 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e937fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e937fe640 1 -- 172.21.15.204:0/307094096 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e8c0148b0 con 0x7f4e9c098940 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e937fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e99ffb640 4 set_mon_vals no callback set 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4e937fe640 10 monclient: dump: 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:56.211 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:56.212 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4ea199d640 10 monclient: _renew_subs 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4ea199d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.189+0000 7f4ea199d640 1 -- 172.21.15.204:0/307094096 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c095db0 con 0x7f4e9c098940 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.190+0000 7f4e937fe640 1 -- 172.21.15.204:0/307094096 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4e8c014430 con 0x7f4e9c098940 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.190+0000 7f4ea199d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.190+0000 7f4ea199d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.190+0000 7f4ea199d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.190+0000 7f4ea199d640 1 -- 172.21.15.204:0/307094096 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c070cc0 con 0x7f4e9c098940 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.190+0000 7f4e937fe640 1 --2- 172.21.15.204:0/307094096 >> v2:172.21.15.204:6812/28210 conn(0x7f4e7c0397b0 0x7f4e7c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.190+0000 7f4e937fe640 1 -- 172.21.15.204:0/307094096 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4246+0+0 (secure 0 0 0) 0x7f4e8c04b8d0 con 0x7f4e9c098940 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.190+0000 7f4e927fc640 1 --2- 172.21.15.204:0/307094096 >> v2:172.21.15.204:6812/28210 conn(0x7f4e7c0397b0 0x7f4e7c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.190+0000 7f4e927fc640 10 monclient: get_auth_request con 0x7f4e7c0397b0 auth_method 0 2024-09-15T01:33:56.213 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.190+0000 7f4e927fc640 1 --2- 172.21.15.204:0/307094096 >> v2:172.21.15.204:6812/28210 conn(0x7f4e7c0397b0 0x7f4e7c03bc60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e84034000 tx=0x7f4e84032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:56.214 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.194+0000 7f4e937fe640 1 -- 172.21.15.204:0/307094096 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4e8c014a30 con 0x7f4e9c098940 2024-09-15T01:33:56.214 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.194+0000 7f4e937fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:56.214 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.194+0000 7f4e937fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:56.356 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.353+0000 7f4ea199d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "bar"}] 2024-09-15T01:33:56.356 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.353+0000 7f4ea199d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-09-15T01:33:56.356 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.353+0000 7f4ea199d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:56.356 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.353+0000 7f4ea199d640 1 -- 172.21.15.204:0/307094096 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule rm", "name": "bar"} v 0) v1 -- 0x7f4e9c0a10b0 con 0x7f4e9c098940 2024-09-15T01:33:56.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.464+0000 7f4e937fe640 1 -- 172.21.15.204:0/307094096 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4e8c011340 con 0x7f4e9c098940 2024-09-15T01:33:56.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.464+0000 7f4e937fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-09-15T01:33:56.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.464+0000 7f4e937fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:33:56.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.559+0000 7f4ea199d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "bar"}] 2024-09-15T01:33:56.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.559+0000 7f4ea199d640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-09-15T01:33:56.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.559+0000 7f4ea199d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:56.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.559+0000 7f4ea199d640 1 -- 172.21.15.204:0/307094096 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rule rm", "name": "bar"} v 0) v1 -- 0x7f4e9c068f00 con 0x7f4e9c098940 2024-09-15T01:33:56.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.559+0000 7f4e937fe640 1 -- 172.21.15.204:0/307094096 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 rule bar does not exist v65) v1 ==== 103+0+0 (secure 0 0 0) 0x7f4e8c011520 con 0x7f4e9c098940 2024-09-15T01:33:56.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.559+0000 7f4e937fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-09-15T01:33:56.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.559+0000 7f4e937fe640 10 monclient: _finish_command 3 = system:0 rule bar does not exist 2024-09-15T01:33:56.562 INFO:tasks.workunit.client.0.smithi204.stderr:rule bar does not exist 2024-09-15T01:33:56.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 1 -- 172.21.15.204:0/307094096 >> v2:172.21.15.204:6812/28210 conn(0x7f4e7c0397b0 msgr2=0x7f4e7c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:56.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 1 --2- 172.21.15.204:0/307094096 >> v2:172.21.15.204:6812/28210 conn(0x7f4e7c0397b0 0x7f4e7c03bc60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e84034000 tx=0x7f4e84032000 comp rx=0 tx=0).stop 2024-09-15T01:33:56.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 10 monclient: shutdown 2024-09-15T01:33:56.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:56.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 1 -- 172.21.15.204:0/307094096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4e9c098940 msgr2=0x7f4e9c095330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:56.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 1 --2- 172.21.15.204:0/307094096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4e9c098940 0x7f4e9c095330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00d8c0 tx=0x7f4e8c008040 comp rx=0 tx=0).stop 2024-09-15T01:33:56.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 1 -- 172.21.15.204:0/307094096 shutdown_connections 2024-09-15T01:33:56.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 1 --2- 172.21.15.204:0/307094096 >> v2:172.21.15.204:6812/28210 conn(0x7f4e7c0397b0 0x7f4e7c03bc60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 1 --2- 172.21.15.204:0/307094096 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4e9c0a0b40 0x7f4e9c06b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 1 --2- 172.21.15.204:0/307094096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c098d50 0x7f4e9c095870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 1 --2- 172.21.15.204:0/307094096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4e9c098940 0x7f4e9c095330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 1 -- 172.21.15.204:0/307094096 >> 172.21.15.204:0/307094096 conn(0x7f4e9c0a2e30 msgr2=0x7f4e9c133940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:56.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 1 -- 172.21.15.204:0/307094096 shutdown_connections 2024-09-15T01:33:56.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.561+0000 7f4ea199d640 1 -- 172.21.15.204:0/307094096 wait complete. 2024-09-15T01:33:56.572 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool create pinning_pool 1 2024-09-15T01:33:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.644+0000 7f1d3a75d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.644+0000 7f1d3a75d640 10 monclient: build_initial_monmap 2024-09-15T01:33:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.644+0000 7f1d3a75d640 10 monclient: monmap: 2024-09-15T01:33:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:56.646306+0000 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:56.646306+0000 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:56.649 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.645+0000 7f1d3a75d640 1 Processor -- start 2024-09-15T01:33:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.645+0000 7f1d3a75d640 1 -- start start 2024-09-15T01:33:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.645+0000 7f1d3a75d640 10 monclient: init 2024-09-15T01:33:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 10 monclient: _add_conns ranks=[10,6,2,4,0,5,3,1,8,9,7] 2024-09-15T01:33:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1d3406e0f0 0x7f1d3406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 10 monclient(hunting): picked mon.noname-f con 0x7f1d3406e0f0 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1d3406e4c0 0x7f1d3406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 10 monclient(hunting): picked mon.noname-d con 0x7f1d3406e4c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d3409e850 0x7f1d3409ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 10 monclient(hunting): picked mon.noname-b con 0x7f1d3409e850 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d34069840 con 0x7f1d3406e0f0 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d34069560 con 0x7f1d3409e850 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3406edd0 con 0x7f1d3406e4c0 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d3a75d640 10 monclient(hunting): authenticate will time out at 852.289339s 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d337fe640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1d3406e4c0 msgr2=0x7f1d3406e890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d337fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1d3406e4c0 0x7f1d3406e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1d3406e0f0 0x7f1d3406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1d3406e0f0 0x7f1d3406c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44080/0 (socket says 172.21.15.204:44080) 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 1 -- 172.21.15.204:0/2624360534 learned_addr learned my addr 172.21.15.204:0/2624360534 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 10 monclient(hunting): get_auth_request con 0x7f1d3406e0f0 auth_method 0 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d38cfa640 1 --2- 172.21.15.204:0/2624360534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d3409e850 0x7f1d3409ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d38cfa640 10 monclient(hunting): get_auth_request con 0x7f1d3409e850 auth_method 0 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d38cfa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d38cfa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d38cfa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d38cfa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d38cfa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d38cfa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 10 monclient(hunting): handle_auth_done global_id 5284 payload 274 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 1 -- 172.21.15.204:0/2624360534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1d3406e4c0 msgr2=0x7f1d3406e890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 1 --2- 172.21.15.204:0/2624360534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1d3406e4c0 0x7f1d3406e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 1 -- 172.21.15.204:0/2624360534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d3409e850 msgr2=0x7f1d3409ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 1 --2- 172.21.15.204:0/2624360534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d3409e850 0x7f1d3409ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 1 monclient: found mon.noname-f 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.646+0000 7f1d33fff640 1 -- 172.21.15.204:0/2624360534 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3404b720 con 0x7f1d3406e0f0 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.647+0000 7f1d33fff640 1 --2- 172.21.15.204:0/2624360534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3406e0f0 0x7f1d3406c980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009910 tx=0x7f1d240091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bed2af5cf8a5959 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d32ffd640 1 -- 172.21.15.204:0/2624360534 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1d2402acb0 con 0x7f1d3406e0f0 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d32ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d32ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d32ffd640 10 monclient: dump: 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d32ffd640 10 monclient: _finish_auth 0 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d32ffd640 1 -- 172.21.15.204:0/2624360534 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2402ae30 con 0x7f1d3406e0f0 2024-09-15T01:33:56.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d32ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d32ffd640 1 -- 172.21.15.204:0/2624360534 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1d24031ca0 con 0x7f1d3406e0f0 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d32ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d32ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d32ffd640 10 monclient: dump: 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:56.658 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d3a75d640 5 monclient: authenticate success, global_id 5284 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d3a75d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d3a75d640 4 set_mon_vals no callback set 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d3a75d640 10 monclient: shutdown 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d3a75d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d3a75d640 1 -- 172.21.15.204:0/2624360534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3406e0f0 msgr2=0x7f1d3406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:56.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.649+0000 7f1d3a75d640 1 --2- 172.21.15.204:0/2624360534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3406e0f0 0x7f1d3406c980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009910 tx=0x7f1d240091d0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.650+0000 7f1d3a75d640 1 -- 172.21.15.204:0/2624360534 shutdown_connections 2024-09-15T01:33:56.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.650+0000 7f1d3a75d640 1 --2- 172.21.15.204:0/2624360534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d3409e850 0x7f1d3409ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.650+0000 7f1d3a75d640 1 --2- 172.21.15.204:0/2624360534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1d3406e4c0 0x7f1d3406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.650+0000 7f1d3a75d640 1 --2- 172.21.15.204:0/2624360534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3406e0f0 0x7f1d3406c980 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.650+0000 7f1d3a75d640 1 -- 172.21.15.204:0/2624360534 >> 172.21.15.204:0/2624360534 conn(0x7f1d34067740 msgr2=0x7f1d340a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:56.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.650+0000 7f1d3a75d640 1 -- 172.21.15.204:0/2624360534 shutdown_connections 2024-09-15T01:33:56.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.650+0000 7f1d3a75d640 1 -- 172.21.15.204:0/2624360534 wait complete. 2024-09-15T01:33:56.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.650+0000 7f1d3a75d640 10 monclient: build_initial_monmap 2024-09-15T01:33:56.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.650+0000 7f1d3a75d640 10 monclient: monmap: 2024-09-15T01:33:56.664 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:56.665 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:56.665 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:56.665 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:56.665 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:56.665 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:56.665 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:56.665 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:56.665 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:56.665 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:56.665 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:56.665 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.651+0000 7f1d3a75d640 1 Processor -- start 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.651+0000 7f1d3a75d640 1 -- start start 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.651+0000 7f1d3a75d640 10 monclient: init 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 10 monclient: _add_conns ranks=[6,4,5,7,0,2,3,1,8] 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1d3406e0f0 0x7f1d34142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 10 monclient(hunting): picked mon.noname-d con 0x7f1d3406e0f0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d3406e4c0 0x7f1d34142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 10 monclient(hunting): picked mon.noname-c con 0x7f1d3406e4c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:56.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d3409e850 0x7f1d34145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 10 monclient(hunting): picked mon.noname-h con 0x7f1d3409e850 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3406f000 con 0x7f1d3406e4c0 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d34068480 con 0x7f1d3409e850 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1d341270e0 con 0x7f1d3406e0f0 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d3a75d640 10 monclient(hunting): authenticate will time out at 852.295266s 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d33fff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1d3406e0f0 msgr2=0x7f1d34142990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d33fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1d3406e0f0 0x7f1d34142990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d3409e850 0x7f1d34145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d3409e850 0x7f1d34145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49608/0 (socket says 172.21.15.204:49608) 2024-09-15T01:33:56.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 1 -- 172.21.15.204:0/3874397395 learned_addr learned my addr 172.21.15.204:0/3874397395 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 10 monclient(hunting): get_auth_request con 0x7f1d3409e850 auth_method 0 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d337fe640 1 --2- 172.21.15.204:0/3874397395 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d3406e4c0 0x7f1d34142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d337fe640 10 monclient(hunting): get_auth_request con 0x7f1d3406e4c0 auth_method 0 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:56.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 10 monclient(hunting): handle_auth_done global_id 4946 payload 824 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 1 -- 172.21.15.204:0/3874397395 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1d3406e0f0 msgr2=0x7f1d34142990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 1 --2- 172.21.15.204:0/3874397395 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1d3406e0f0 0x7f1d34142990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 1 -- 172.21.15.204:0/3874397395 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d3406e4c0 msgr2=0x7f1d34142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 1 --2- 172.21.15.204:0/3874397395 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d3406e4c0 0x7f1d34142040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 10 monclient: _finish_hunting 0 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 1 monclient: found mon.noname-h 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 1 -- 172.21.15.204:0/3874397395 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d24009450 con 0x7f1d3409e850 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.652+0000 7f1d38cfa640 1 --2- 172.21.15.204:0/3874397395 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d3409e850 0x7f1d34145ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009df0 tx=0x7f1d2c00b040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d1bfff640 1 -- 172.21.15.204:0/3874397395 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1d2c00cce0 con 0x7f1d3409e850 2024-09-15T01:33:56.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d1bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d1bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d1bfff640 10 monclient: dump: 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:56.670 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d1bfff640 10 monclient: _finish_auth 0 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d1bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d3a75d640 5 monclient: authenticate success, global_id 4946 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d3a75d640 10 monclient: _renew_subs 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d3a75d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d3a75d640 1 -- 172.21.15.204:0/3874397395 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d341273c0 con 0x7f1d3409e850 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d3a75d640 10 monclient: _renew_subs 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d3a75d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:56.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d3a75d640 1 -- 172.21.15.204:0/3874397395 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d34143470 con 0x7f1d3409e850 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d1bfff640 1 -- 172.21.15.204:0/3874397395 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c00f040 con 0x7f1d3409e850 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d1bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d1bfff640 1 -- 172.21.15.204:0/3874397395 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1d2c00ce40 con 0x7f1d3409e850 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d1bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d1bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d1bfff640 10 monclient: dump: 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:56.672 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.653+0000 7f1d327fc640 4 set_mon_vals no callback set 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.654+0000 7f1d1bfff640 1 -- 172.21.15.204:0/3874397395 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1d2c014480 con 0x7f1d3409e850 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.654+0000 7f1d1bfff640 1 --2- 172.21.15.204:0/3874397395 >> v2:172.21.15.204:6812/28210 conn(0x7f1d200397b0 0x7f1d2003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.654+0000 7f1d1bfff640 1 -- 172.21.15.204:0/3874397395 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4191+0+0 (secure 0 0 0) 0x7f1d2c04bd00 con 0x7f1d3409e850 2024-09-15T01:33:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.654+0000 7f1d3a75d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.654+0000 7f1d3a75d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.654+0000 7f1d3a75d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.654+0000 7f1d33fff640 1 --2- 172.21.15.204:0/3874397395 >> v2:172.21.15.204:6812/28210 conn(0x7f1d200397b0 0x7f1d2003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.654+0000 7f1d3a75d640 1 -- 172.21.15.204:0/3874397395 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d34125650 con 0x7f1d3409e850 2024-09-15T01:33:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.654+0000 7f1d33fff640 10 monclient: get_auth_request con 0x7f1d200397b0 auth_method 0 2024-09-15T01:33:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.655+0000 7f1d33fff640 1 --2- 172.21.15.204:0/3874397395 >> v2:172.21.15.204:6812/28210 conn(0x7f1d200397b0 0x7f1d2003bc60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1d24034000 tx=0x7f1d24032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.659+0000 7f1d1bfff640 1 -- 172.21.15.204:0/3874397395 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1d34125650 con 0x7f1d3409e850 2024-09-15T01:33:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.659+0000 7f1d1bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.659+0000 7f1d1bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:56.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.815+0000 7f1d3a75d640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-09-15T01:33:56.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.815+0000 7f1d3a75d640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-09-15T01:33:56.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.815+0000 7f1d3a75d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:56.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:56.815+0000 7f1d3a75d640 1 -- 172.21.15.204:0/3874397395 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1} v 0) v1 -- 0x7f1d34125650 con 0x7f1d3409e850 2024-09-15T01:33:57.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.658+0000 7f1d1bfff640 1 -- 172.21.15.204:0/3874397395 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}]=0 pool 'pinning_pool' created v66) v1 ==== 127+0+0 (secure 0 0 0) 0x7f1d34125650 con 0x7f1d3409e850 2024-09-15T01:33:57.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.658+0000 7f1d1bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-09-15T01:33:57.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.658+0000 7f1d1bfff640 10 monclient: _finish_command 2 = system:0 pool 'pinning_pool' created 2024-09-15T01:33:57.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.752+0000 7f1d3a75d640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-09-15T01:33:57.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.752+0000 7f1d3a75d640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-09-15T01:33:57.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.752+0000 7f1d3a75d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:33:57.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.752+0000 7f1d3a75d640 1 -- 172.21.15.204:0/3874397395 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1} v 0) v1 -- 0x7f1d34125650 con 0x7f1d3409e850 2024-09-15T01:33:57.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.753+0000 7f1d1bfff640 1 -- 172.21.15.204:0/3874397395 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}]=0 pool 'pinning_pool' already exists v66) v1 ==== 134+0+0 (secure 0 0 0) 0x7f1d34125650 con 0x7f1d3409e850 2024-09-15T01:33:57.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.753+0000 7f1d1bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-09-15T01:33:57.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.753+0000 7f1d1bfff640 10 monclient: _finish_command 3 = system:0 pool 'pinning_pool' already exists 2024-09-15T01:33:57.756 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'pinning_pool' already exists 2024-09-15T01:33:57.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 1 -- 172.21.15.204:0/3874397395 >> v2:172.21.15.204:6812/28210 conn(0x7f1d200397b0 msgr2=0x7f1d2003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:57.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 1 --2- 172.21.15.204:0/3874397395 >> v2:172.21.15.204:6812/28210 conn(0x7f1d200397b0 0x7f1d2003bc60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1d24034000 tx=0x7f1d24032000 comp rx=0 tx=0).stop 2024-09-15T01:33:57.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 10 monclient: shutdown 2024-09-15T01:33:57.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:57.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 1 -- 172.21.15.204:0/3874397395 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d3409e850 msgr2=0x7f1d34145ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:57.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 1 --2- 172.21.15.204:0/3874397395 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d3409e850 0x7f1d34145ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009df0 tx=0x7f1d2c00b040 comp rx=0 tx=0).stop 2024-09-15T01:33:57.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 1 -- 172.21.15.204:0/3874397395 shutdown_connections 2024-09-15T01:33:57.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 1 --2- 172.21.15.204:0/3874397395 >> v2:172.21.15.204:6812/28210 conn(0x7f1d200397b0 0x7f1d2003bc60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:57.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 1 --2- 172.21.15.204:0/3874397395 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d3409e850 0x7f1d34145ee0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:57.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 1 --2- 172.21.15.204:0/3874397395 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d3406e4c0 0x7f1d34142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:57.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 1 --2- 172.21.15.204:0/3874397395 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1d3406e0f0 0x7f1d34142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:57.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 1 -- 172.21.15.204:0/3874397395 >> 172.21.15.204:0/3874397395 conn(0x7f1d34067740 msgr2=0x7f1d3409d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:57.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 1 -- 172.21.15.204:0/3874397395 shutdown_connections 2024-09-15T01:33:57.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.755+0000 7f1d3a75d640 1 -- 172.21.15.204:0/3874397395 wait complete. 2024-09-15T01:33:57.767 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush rule rm replicated_rule 2024-09-15T01:33:57.767 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:57.767 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rm replicated_rule 2024-09-15T01:33:57.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.837+0000 7f453831d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:57.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.837+0000 7f453831d640 10 monclient: build_initial_monmap 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.837+0000 7f453831d640 10 monclient: monmap: 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:57.839150+0000 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:57.839150+0000 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:57.841 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:57.842 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:57.842 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:57.842 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:57.842 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:57.842 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:57.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 1 Processor -- start 2024-09-15T01:33:57.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 1 -- start start 2024-09-15T01:33:57.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 10 monclient: init 2024-09-15T01:33:57.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:57.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 10 monclient: _add_conns ranks=[4,9,8,6,5,3,0,7,1,10,2] 2024-09-15T01:33:57.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f45300a2e30 0x7f4530132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:57.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 10 monclient(hunting): picked mon.noname-c con 0x7f45300a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:57.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f453009fb80 0x7f453009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:57.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 10 monclient(hunting): picked mon.noname-a con 0x7f453009fb80 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:57.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45300a0a40 0x7f45300a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:57.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 10 monclient(hunting): picked mon.noname-e con 0x7f45300a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:57.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:57.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453009e6c0 con 0x7f453009fb80 2024-09-15T01:33:57.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:57.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45300a04b0 con 0x7f45300a2e30 2024-09-15T01:33:57.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:57.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f45300a3200 con 0x7f45300a0a40 2024-09-15T01:33:57.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:57.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.838+0000 7f453831d640 10 monclient(hunting): authenticate will time out at 853.481896s 2024-09-15T01:33:57.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45368ba640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45300a0a40 0x7f45300a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:57.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45358b8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f453009fb80 0x7f453009ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:57.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f45300a2e30 0x7f4530132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:57.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45368ba640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45300a0a40 0x7f45300a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:46130/0 (socket says 172.21.15.204:46130) 2024-09-15T01:33:57.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45358b8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f453009fb80 0x7f453009ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:34244/0 (socket says 172.21.15.204:34244) 2024-09-15T01:33:57.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45368ba640 1 -- 172.21.15.204:0/2062039842 learned_addr learned my addr 172.21.15.204:0/2062039842 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:57.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 10 monclient(hunting): get_auth_request con 0x7f45300a2e30 auth_method 0 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45368ba640 10 monclient(hunting): get_auth_request con 0x7f45300a0a40 auth_method 0 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45368ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45368ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45368ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45358b8640 10 monclient(hunting): get_auth_request con 0x7f453009fb80 auth_method 0 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45358b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45358b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45358b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:57.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45368ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45368ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45368ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45358b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45358b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45358b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 10 monclient(hunting): handle_auth_done global_id 4873 payload 274 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 1 -- 172.21.15.204:0/2062039842 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45300a0a40 msgr2=0x7f45300a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 1 --2- 172.21.15.204:0/2062039842 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45300a0a40 0x7f45300a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 1 -- 172.21.15.204:0/2062039842 >> v2:172.21.15.178:3300/0 conn(0x7f453009fb80 msgr2=0x7f453009ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 1 --2- 172.21.15.204:0/2062039842 >> v2:172.21.15.178:3300/0 conn(0x7f453009fb80 0x7f453009ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 10 monclient: _finish_hunting 0 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 1 monclient: found mon.noname-c 2024-09-15T01:33:57.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45360b9640 1 -- 172.21.15.204:0/2062039842 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453004b720 con 0x7f45300a2e30 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.839+0000 7f45368ba640 1 --2- 172.21.15.204:0/2062039842 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45300a0a40 0x7f45300a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45360b9640 1 --2- 172.21.15.204:0/2062039842 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f45300a2e30 0x7f4530132a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f452c009850 tx=0x7f452c0090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=76102244e234f84b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45350b7640 1 -- 172.21.15.204:0/2062039842 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f452c02abc0 con 0x7f45300a2e30 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45350b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45350b7640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45350b7640 10 monclient: dump: 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:57.855 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45350b7640 10 monclient: _finish_auth 0 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45350b7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45350b7640 1 -- 172.21.15.204:0/2062039842 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c02ad40 con 0x7f45300a2e30 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45350b7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:57.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45350b7640 1 -- 172.21.15.204:0/2062039842 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f452c031ca0 con 0x7f45300a2e30 2024-09-15T01:33:57.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45350b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:57.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45350b7640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:57.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f45350b7640 10 monclient: dump: 2024-09-15T01:33:57.857 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:57.857 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:57.857 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:57.857 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:57.857 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:57.857 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:57.857 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:57.857 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:57.857 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f453831d640 5 monclient: authenticate success, global_id 4873 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f453831d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f453831d640 4 set_mon_vals no callback set 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f453831d640 10 monclient: shutdown 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f453831d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f453831d640 1 -- 172.21.15.204:0/2062039842 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f45300a2e30 msgr2=0x7f4530132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.840+0000 7f453831d640 1 --2- 172.21.15.204:0/2062039842 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f45300a2e30 0x7f4530132a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f452c009850 tx=0x7f452c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.841+0000 7f453831d640 1 -- 172.21.15.204:0/2062039842 shutdown_connections 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.841+0000 7f453831d640 1 --2- 172.21.15.204:0/2062039842 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45300a0a40 0x7f45300a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.841+0000 7f453831d640 1 --2- 172.21.15.204:0/2062039842 >> v2:172.21.15.178:3300/0 conn(0x7f453009fb80 0x7f453009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.841+0000 7f453831d640 1 --2- 172.21.15.204:0/2062039842 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f45300a2e30 0x7f4530132a60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.841+0000 7f453831d640 1 -- 172.21.15.204:0/2062039842 >> 172.21.15.204:0/2062039842 conn(0x7f453006e860 msgr2=0x7f453006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.841+0000 7f453831d640 1 -- 172.21.15.204:0/2062039842 shutdown_connections 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.841+0000 7f453831d640 1 -- 172.21.15.204:0/2062039842 wait complete. 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.841+0000 7f453831d640 10 monclient: build_initial_monmap 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.841+0000 7f453831d640 10 monclient: monmap: 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:57.860 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.841+0000 7f453831d640 1 Processor -- start 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.841+0000 7f453831d640 1 -- start start 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.841+0000 7f453831d640 10 monclient: init 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 10 monclient: _add_conns ranks=[8,2,0,4,6,1,3,5,7] 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f453009fb80 0x7f453009b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 10 monclient(hunting): picked mon.noname-e con 0x7f453009fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45300a0a40 0x7f453012e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 10 monclient(hunting): picked mon.noname-b con 0x7f45300a0a40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45300a2e30 0x7f453011c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 10 monclient(hunting): picked mon.noname-a con 0x7f45300a2e30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453009d650 con 0x7f45300a2e30 2024-09-15T01:33:57.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453011ef60 con 0x7f45300a0a40 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f453011ebf0 con 0x7f453009fb80 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f453831d640 10 monclient(hunting): authenticate will time out at 853.485365s 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45368ba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45300a2e30 0x7f453011c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45360b9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f453009fb80 0x7f453009b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45358b8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45300a0a40 0x7f453012e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45368ba640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45300a2e30 0x7f453011c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:34256/0 (socket says 172.21.15.204:34256) 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45360b9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f453009fb80 0x7f453009b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:46132/0 (socket says 172.21.15.204:46132) 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45368ba640 1 -- 172.21.15.204:0/4137319329 learned_addr learned my addr 172.21.15.204:0/4137319329 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45368ba640 10 monclient(hunting): get_auth_request con 0x7f45300a2e30 auth_method 0 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45368ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45368ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:57.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45368ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45360b9640 10 monclient(hunting): get_auth_request con 0x7f453009fb80 auth_method 0 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45360b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45360b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45360b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45358b8640 10 monclient(hunting): get_auth_request con 0x7f45300a0a40 auth_method 0 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45358b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45358b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45358b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45360b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45360b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45360b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45368ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45368ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:57.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45368ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45358b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45358b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.842+0000 7f45358b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45360b9640 10 monclient(hunting): handle_auth_done global_id 4976 payload 824 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45360b9640 1 -- 172.21.15.204:0/4137319329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45300a0a40 msgr2=0x7f453012e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45360b9640 1 --2- 172.21.15.204:0/4137319329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45300a0a40 0x7f453012e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45360b9640 1 -- 172.21.15.204:0/4137319329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45300a2e30 msgr2=0x7f453011c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45360b9640 1 --2- 172.21.15.204:0/4137319329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45300a2e30 0x7f453011c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45360b9640 10 monclient: _finish_hunting 0 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45360b9640 1 monclient: found mon.noname-e 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45360b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45360b9640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:33:57.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45360b9640 1 -- 172.21.15.204:0/4137319329 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528009ae0 con 0x7f453009fb80 2024-09-15T01:33:57.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45368ba640 1 --2- 172.21.15.204:0/4137319329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45300a2e30 0x7f453011c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:57.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45358b8640 1 --2- 172.21.15.204:0/4137319329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45300a0a40 0x7f453012e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:33:57.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45360b9640 1 --2- 172.21.15.204:0/4137319329 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f453009fb80 0x7f453009b600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f452c031000 tx=0x7f452c032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:57.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45267fc640 1 -- 172.21.15.204:0/4137319329 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f452c03cad0 con 0x7f453009fb80 2024-09-15T01:33:57.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:57.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45267fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:33:57.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45267fc640 10 monclient: dump: 2024-09-15T01:33:57.865 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:57.865 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:57.865 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:57.865 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:57.865 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45267fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45267fc640 1 -- 172.21.15.204:0/4137319329 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c03cc30 con 0x7f453009fb80 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f453831d640 5 monclient: authenticate success, global_id 4976 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45267fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45267fc640 1 -- 172.21.15.204:0/4137319329 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f452c03cda0 con 0x7f453009fb80 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f453831d640 10 monclient: _renew_subs 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f453831d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f453831d640 1 -- 172.21.15.204:0/4137319329 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c009390 con 0x7f453009fb80 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45348b6640 4 set_mon_vals no callback set 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45267fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f45267fc640 10 monclient: dump: 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f453831d640 10 monclient: _renew_subs 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f453831d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.843+0000 7f453831d640 1 -- 172.21.15.204:0/4137319329 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453011e910 con 0x7f453009fb80 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.844+0000 7f453831d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.844+0000 7f453831d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.844+0000 7f453831d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.844+0000 7f453831d640 1 -- 172.21.15.204:0/4137319329 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4530070cc0 con 0x7f453009fb80 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.844+0000 7f45267fc640 1 -- 172.21.15.204:0/4137319329 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f452c03c650 con 0x7f453009fb80 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.844+0000 7f45267fc640 1 --2- 172.21.15.204:0/4137319329 >> v2:172.21.15.204:6812/28210 conn(0x7f45140397e0 0x7f451403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.844+0000 7f45267fc640 1 -- 172.21.15.204:0/4137319329 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4542+0+0 (secure 0 0 0) 0x7f452c06b780 con 0x7f453009fb80 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.844+0000 7f45358b8640 1 --2- 172.21.15.204:0/4137319329 >> v2:172.21.15.204:6812/28210 conn(0x7f45140397e0 0x7f451403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.844+0000 7f45358b8640 10 monclient: get_auth_request con 0x7f45140397e0 auth_method 0 2024-09-15T01:33:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.845+0000 7f45358b8640 1 --2- 172.21.15.204:0/4137319329 >> v2:172.21.15.204:6812/28210 conn(0x7f45140397e0 0x7f451403bc90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f453009b550 tx=0x7f45200090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.849+0000 7f45267fc640 1 -- 172.21.15.204:0/4137319329 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f452c03b3a0 con 0x7f453009fb80 2024-09-15T01:33:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.849+0000 7f45267fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:57.849+0000 7f45267fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:58.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.005+0000 7f453831d640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "replicated_rule"}] 2024-09-15T01:33:58.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.005+0000 7f453831d640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "replicated_rule"}] 2024-09-15T01:33:58.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.005+0000 7f453831d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:33:58.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.005+0000 7f453831d640 1 -- 172.21.15.204:0/4137319329 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush rule rm", "name": "replicated_rule"} v 0) v1 -- 0x7f45300a1f10 con 0x7f453009fb80 2024-09-15T01:33:58.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.007+0000 7f45267fc640 1 -- 172.21.15.204:0/4137319329 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "replicated_rule"}]=-16 crush rule replicated_rule (0) is in use v66) v1 ==== 132+0+0 (secure 0 0 0) 0x7f452c037070 con 0x7f453009fb80 2024-09-15T01:33:58.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.007+0000 7f45267fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "replicated_rule"}] 2024-09-15T01:33:58.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.007+0000 7f45267fc640 10 monclient: _finish_command 2 = mon:16 crush rule replicated_rule (0) is in use 2024-09-15T01:33:58.009 INFO:tasks.workunit.client.0.smithi204.stderr:Error EBUSY: crush rule replicated_rule (0) is in use 2024-09-15T01:33:58.013 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:58.013 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool rm pinning_pool pinning_pool --yes-i-really-really-mean-it 2024-09-15T01:33:58.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.086+0000 7f523ab9d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:58.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.086+0000 7f523ab9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:58.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.086+0000 7f523ab9d640 10 monclient: monmap: 2024-09-15T01:33:58.089 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:58.089 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:58.089 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:58.087804+0000 2024-09-15T01:33:58.089 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:58.087804+0000 2024-09-15T01:33:58.089 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:58.089 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:58.089 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:58.089 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:58.089 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:58.090 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:58.090 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:58.090 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:58.090 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:58.090 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:58.090 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:58.090 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:58.090 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:58.090 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 1 Processor -- start 2024-09-15T01:33:58.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 1 -- start start 2024-09-15T01:33:58.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 10 monclient: init 2024-09-15T01:33:58.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:58.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 10 monclient: _add_conns ranks=[2,6,1,3,10,4,9,7,0,5,8] 2024-09-15T01:33:58.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52340a2cd0 0x7f5234132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 10 monclient(hunting): picked mon.noname-b con 0x7f52340a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:58.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f523409fa20 0x7f523409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 10 monclient(hunting): picked mon.noname-d con 0x7f523409fa20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f52340a08e0 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52340648a0 con 0x7f52340a08e0 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52340a30a0 con 0x7f52340a2cd0 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f52340a0330 con 0x7f523409fa20 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523ab9d640 10 monclient(hunting): authenticate will time out at 853.730640s 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f5233fff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f523409fa20 msgr2=0x7f523409fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f5233fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f523409fa20 0x7f523409fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523913a640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f52340a08e0 0x7f52340a0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:48770/0 (socket says 172.21.15.204:48770) 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f523913a640 1 -- 172.21.15.204:0/3659137685 learned_addr learned my addr 172.21.15.204:0/3659137685 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:58.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.087+0000 7f5238939640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52340a2cd0 0x7f5234132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): get_auth_request con 0x7f52340a2cd0 auth_method 0 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926947200 0 0) 0x7f52340648a0 con 0x7f52340a08e0 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient(hunting): my global_id is 5293 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52240034e0 con 0x7f52340a08e0 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2660664215 0 0) 0x7f52240034e0 con 0x7f52340a08e0 2024-09-15T01:33:58.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f523409fa20 msgr2=0x7f523409fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f523409fa20 0x7f523409fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52340a2cd0 msgr2=0x7f5234132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52340a2cd0 0x7f5234132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52340a1db0 con 0x7f52340a08e0 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f522c001c70 con 0x7f52340a08e0 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523ab9d640 5 monclient: authenticate success, global_id 5293 2024-09-15T01:33:58.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523ab9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient: dump: 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:58.105 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 >> v1:172.21.15.204:6789/0 conn(0x7f52340a08e0 legacy=0x7f52340a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient: _add_conns ranks=[4,2,1,7,8,6,5,3,0] 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5224004220 0x7f5224004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient(hunting): picked mon.c con 0x7f5224004220 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52340a2cd0 0x7f5224004d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient(hunting): picked mon.b con 0x7f52340a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:58.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240099c0 0x7f522400bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient(hunting): picked mon.f con 0x7f52240099c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52340a30a0 con 0x7f52240099c0 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52340a0330 con 0x7f52340a2cd0 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f522400c3c0 con 0x7f5224004220 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f52337fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523ab9d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240099c0 0x7f522400bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 10 monclient(hunting): get_auth_request con 0x7f52240099c0 auth_method 0 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:58.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5233fff640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52340a2cd0 0x7f5224004d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5224004220 0x7f5224004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5233fff640 10 monclient(hunting): get_auth_request con 0x7f52340a2cd0 auth_method 0 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5233fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5233fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5233fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): get_auth_request con 0x7f5224004220 auth_method 0 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:58.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5233fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5233fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5233fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f5238939640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 10 monclient(hunting): handle_auth_done global_id 5293 payload 293 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 1 -- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5224004220 msgr2=0x7f5224004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5224004220 0x7f5224004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 1 -- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52340a2cd0 msgr2=0x7f5224004d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52340a2cd0 0x7f5224004d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 10 monclient: _finish_hunting 0 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 1 monclient: found mon.f 2024-09-15T01:33:58.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 1 -- 172.21.15.204:0/3659137685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240034e0 con 0x7f52240099c0 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 10 monclient: _finish_auth 0 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.088+0000 7f523913a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f523913a640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240099c0 0x7f522400bda0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c750 tx=0x7f523409db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0cdca0089ddfbe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f522c007570 con 0x7f52240099c0 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f52337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f52337fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f52337fe640 10 monclient: dump: 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.110 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c0076f0 con 0x7f52240099c0 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f523ab9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f52337fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f523ab9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f52337fe640 1 -- 172.21.15.204:0/3659137685 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f522c007860 con 0x7f52240099c0 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f52337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f523ab9d640 4 set_mon_vals no callback set 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f52337fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.089+0000 7f52337fe640 10 monclient: dump: 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:58.112 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.090+0000 7f523ab9d640 10 monclient: shutdown 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.090+0000 7f523ab9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.090+0000 7f523ab9d640 1 -- 172.21.15.204:0/3659137685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240099c0 msgr2=0x7f522400bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:58.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.090+0000 7f523ab9d640 1 --2- 172.21.15.204:0/3659137685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240099c0 0x7f522400bda0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c750 tx=0x7f523409db10 comp rx=0 tx=0).stop 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.090+0000 7f523913a640 1 -- 172.21.15.204:0/3659137685 reap_dead start 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.090+0000 7f523ab9d640 1 -- 172.21.15.204:0/3659137685 shutdown_connections 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.090+0000 7f523ab9d640 1 -- 172.21.15.204:0/3659137685 >> 172.21.15.204:0/3659137685 conn(0x7f523406e860 msgr2=0x7f523406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.090+0000 7f523ab9d640 1 -- 172.21.15.204:0/3659137685 shutdown_connections 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.090+0000 7f523ab9d640 1 -- 172.21.15.204:0/3659137685 wait complete. 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.090+0000 7f523ab9d640 10 monclient: build_initial_monmap 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.090+0000 7f523ab9d640 10 monclient: monmap: 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:58.114 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.090+0000 7f523ab9d640 1 Processor -- start 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 1 -- start start 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 10 monclient: init 2024-09-15T01:33:58.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 10 monclient: _add_conns ranks=[1,8,6,7,2,3,0,5,4] 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240099c0 0x7f523411c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 10 monclient(hunting): picked mon.noname-f con 0x7f52240099c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f523411ccf0 0x7f523411d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 10 monclient(hunting): picked mon.noname-e con 0x7f523411ccf0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f523411d600 0x7f5234147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 10 monclient(hunting): picked mon.noname-d con 0x7f523411d600 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523411e3e0 con 0x7f52240099c0 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f523411fd30 con 0x7f523411d600 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f523409d3d0 con 0x7f523411ccf0 2024-09-15T01:33:58.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523ab9d640 10 monclient(hunting): authenticate will time out at 853.734582s 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523913a640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f523411d600 msgr2=0x7f5234147ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240099c0 0x7f523411c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f523913a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f523411d600 0x7f5234147ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240099c0 0x7f523411c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38090/0 (socket says 172.21.15.204:38090) 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 1 -- 172.21.15.204:0/731707480 learned_addr learned my addr 172.21.15.204:0/731707480 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 10 monclient(hunting): get_auth_request con 0x7f52240099c0 auth_method 0 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:58.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5233fff640 1 --2- 172.21.15.204:0/731707480 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f523411ccf0 0x7f523411d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5233fff640 10 monclient(hunting): get_auth_request con 0x7f523411ccf0 auth_method 0 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5233fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5233fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5233fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 10 monclient(hunting): handle_auth_done global_id 5302 payload 824 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 1 -- 172.21.15.204:0/731707480 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f523411ccf0 msgr2=0x7f523411d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 1 --2- 172.21.15.204:0/731707480 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f523411ccf0 0x7f523411d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 1 -- 172.21.15.204:0/731707480 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f523411d600 msgr2=0x7f5234147ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 1 --2- 172.21.15.204:0/731707480 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f523411d600 0x7f5234147ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 10 monclient: _finish_hunting 0 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 1 monclient: found mon.noname-f 2024-09-15T01:33:58.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5238939640 1 -- 172.21.15.204:0/731707480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523411f440 con 0x7f52240099c0 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.091+0000 7f5233fff640 1 --2- 172.21.15.204:0/731707480 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f523411ccf0 0x7f523411d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5238939640 1 --2- 172.21.15.204:0/731707480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240099c0 0x7f523411c7b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5228011b70 tx=0x7f52280090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5230ff9640 1 -- 172.21.15.204:0/731707480 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5228014440 con 0x7f52240099c0 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5230ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5230ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5230ff9640 10 monclient: dump: 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:58.119 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5230ff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5230ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f523ab9d640 5 monclient: authenticate success, global_id 5302 2024-09-15T01:33:58.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5230ff9640 1 -- 172.21.15.204:0/731707480 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52280145a0 con 0x7f52240099c0 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f523ab9d640 10 monclient: _renew_subs 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f523ab9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f523ab9d640 1 -- 172.21.15.204:0/731707480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523411dbb0 con 0x7f52240099c0 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5230ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5230ff9640 1 -- 172.21.15.204:0/731707480 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5228014b90 con 0x7f52240099c0 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5230ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5230ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5232ffd640 4 set_mon_vals no callback set 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f5230ff9640 10 monclient: dump: 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.121 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f523ab9d640 10 monclient: _renew_subs 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f523ab9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:58.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.092+0000 7f523ab9d640 1 -- 172.21.15.204:0/731707480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523411e780 con 0x7f52240099c0 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.093+0000 7f5230ff9640 1 -- 172.21.15.204:0/731707480 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5228014cf0 con 0x7f52240099c0 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.093+0000 7f5230ff9640 1 --2- 172.21.15.204:0/731707480 >> v2:172.21.15.204:6812/28210 conn(0x7f52180397e0 0x7f521803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.093+0000 7f5230ff9640 1 -- 172.21.15.204:0/731707480 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4542+0+0 (secure 0 0 0) 0x7f5228018030 con 0x7f52240099c0 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.093+0000 7f523ab9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.093+0000 7f523ab9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.093+0000 7f523ab9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.093+0000 7f5233fff640 1 --2- 172.21.15.204:0/731707480 >> v2:172.21.15.204:6812/28210 conn(0x7f52180397e0 0x7f521803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.093+0000 7f523ab9d640 1 -- 172.21.15.204:0/731707480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5234070b80 con 0x7f52240099c0 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.093+0000 7f5233fff640 10 monclient: get_auth_request con 0x7f52180397e0 auth_method 0 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.093+0000 7f5233fff640 1 --2- 172.21.15.204:0/731707480 >> v2:172.21.15.204:6812/28210 conn(0x7f52180397e0 0x7f521803bc90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f523409c6b0 tx=0x7f5220008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.097+0000 7f5230ff9640 1 -- 172.21.15.204:0/731707480 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f522800fb60 con 0x7f52240099c0 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.097+0000 7f5230ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:58.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.097+0000 7f5230ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:58.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.252+0000 7f523ab9d640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:58.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.252+0000 7f523ab9d640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:58.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.252+0000 7f523ab9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:58.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.252+0000 7f523ab9d640 1 -- 172.21.15.204:0/731707480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5234067450 con 0x7f52240099c0 2024-09-15T01:33:58.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.663+0000 7f5230ff9640 1 -- 172.21.15.204:0/731707480 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}]=0 pool 'pinning_pool' removed v67) v1 ==== 172+0+0 (secure 0 0 0) 0x7f522802b030 con 0x7f52240099c0 2024-09-15T01:33:58.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.663+0000 7f5230ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:58.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.663+0000 7f5230ff9640 10 monclient: _finish_command 2 = system:0 pool 'pinning_pool' removed 2024-09-15T01:33:58.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.757+0000 7f523ab9d640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:58.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.757+0000 7f523ab9d640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:58.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.757+0000 7f523ab9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:58.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.757+0000 7f523ab9d640 1 -- 172.21.15.204:0/731707480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5234120970 con 0x7f52240099c0 2024-09-15T01:33:58.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.758+0000 7f5230ff9640 1 -- 172.21.15.204:0/731707480 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}]=0 pool 'pinning_pool' does not exist v67) v1 ==== 179+0+0 (secure 0 0 0) 0x7f522802b020 con 0x7f52240099c0 2024-09-15T01:33:58.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.758+0000 7f5230ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-09-15T01:33:58.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.758+0000 7f5230ff9640 10 monclient: _finish_command 3 = system:0 pool 'pinning_pool' does not exist 2024-09-15T01:33:58.761 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'pinning_pool' does not exist 2024-09-15T01:33:58.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 1 -- 172.21.15.204:0/731707480 >> v2:172.21.15.204:6812/28210 conn(0x7f52180397e0 msgr2=0x7f521803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:58.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 1 --2- 172.21.15.204:0/731707480 >> v2:172.21.15.204:6812/28210 conn(0x7f52180397e0 0x7f521803bc90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f523409c6b0 tx=0x7f5220008040 comp rx=0 tx=0).stop 2024-09-15T01:33:58.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 10 monclient: shutdown 2024-09-15T01:33:58.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:58.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 1 -- 172.21.15.204:0/731707480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240099c0 msgr2=0x7f523411c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:58.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 1 --2- 172.21.15.204:0/731707480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240099c0 0x7f523411c7b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5228011b70 tx=0x7f52280090f0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 1 -- 172.21.15.204:0/731707480 shutdown_connections 2024-09-15T01:33:58.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 1 --2- 172.21.15.204:0/731707480 >> v2:172.21.15.204:6812/28210 conn(0x7f52180397e0 0x7f521803bc90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 1 --2- 172.21.15.204:0/731707480 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f523411d600 0x7f5234147ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 1 --2- 172.21.15.204:0/731707480 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f523411ccf0 0x7f523411d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 1 --2- 172.21.15.204:0/731707480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240099c0 0x7f523411c7b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 1 -- 172.21.15.204:0/731707480 >> 172.21.15.204:0/731707480 conn(0x7f523406e860 msgr2=0x7f52340a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:58.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 1 -- 172.21.15.204:0/731707480 shutdown_connections 2024-09-15T01:33:58.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.760+0000 7f523ab9d640 1 -- 172.21.15.204:0/731707480 wait complete. 2024-09-15T01:33:58.772 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush add-bucket foo osd 2024-09-15T01:33:58.772 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:33:58.772 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket foo osd 2024-09-15T01:33:58.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.844+0000 7f33af15d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:58.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.844+0000 7f33af15d640 10 monclient: build_initial_monmap 2024-09-15T01:33:58.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.844+0000 7f33af15d640 10 monclient: monmap: 2024-09-15T01:33:58.847 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:58.847 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:58.847 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:58.845553+0000 2024-09-15T01:33:58.847 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:58.845553+0000 2024-09-15T01:33:58.847 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:58.847 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:58.847 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:58.847 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:58.847 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:58.848 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:58.848 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:58.848 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:58.848 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:58.848 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:58.848 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:58.848 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:58.848 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:58.848 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.844+0000 7f33af15d640 1 Processor -- start 2024-09-15T01:33:58.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.844+0000 7f33af15d640 1 -- start start 2024-09-15T01:33:58.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.844+0000 7f33af15d640 10 monclient: init 2024-09-15T01:33:58.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:58.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 10 monclient: _add_conns ranks=[0,8,10,6,2,1,3,9,7,5,4] 2024-09-15T01:33:58.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f33a80a4680 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:58.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33a806eea0 0x7f33a809ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 10 monclient(hunting): picked mon.noname-e con 0x7f33a806eea0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:58.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f33a8135bd0 0x7f33a8137fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 10 monclient(hunting): picked mon.noname-f con 0x7f33a8135bd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a804b720 con 0x7f33a80a4680 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a80a4a50 con 0x7f33a8135bd0 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f33a806dc70 con 0x7f33a806eea0 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33af15d640 10 monclient(hunting): authenticate will time out at 854.488281s 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f33a8135bd0 0x7f33a8137fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f33a8135bd0 0x7f33a8137fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38094/0 (socket says 172.21.15.204:38094) 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 1 -- 172.21.15.204:0/2098490010 learned_addr learned my addr 172.21.15.204:0/2098490010 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 10 monclient(hunting): get_auth_request con 0x7f33a8135bd0 auth_method 0 2024-09-15T01:33:58.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33a7fff640 1 --2- 172.21.15.204:0/2098490010 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33a806eea0 0x7f33a809ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 10 monclient(hunting): handle_auth_done global_id 5311 payload 274 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 1 -- 172.21.15.204:0/2098490010 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33a806eea0 msgr2=0x7f33a809ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 1 --2- 172.21.15.204:0/2098490010 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33a806eea0 0x7f33a809ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 1 -- 172.21.15.204:0/2098490010 >> v1:172.21.15.178:6789/0 conn(0x7f33a80a4680 legacy=0x7f33a809e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 10 monclient: _finish_hunting 0 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 1 monclient: found mon.noname-f 2024-09-15T01:33:58.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.845+0000 7f33ad6fa640 1 -- 172.21.15.204:0/2098490010 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a80a0710 con 0x7f33a8135bd0 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33ad6fa640 1 --2- 172.21.15.204:0/2098490010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a8135bd0 0x7f33a8137fb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009f20 tx=0x7f33a0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=825aa2448f3e468e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33a77fe640 1 -- 172.21.15.204:0/2098490010 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33a0007880 con 0x7f33a8135bd0 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33a77fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33a77fe640 10 monclient: dump: 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:58.861 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33a77fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:58.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33a77fe640 1 -- 172.21.15.204:0/2098490010 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a002d050 con 0x7f33a8135bd0 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33a77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33a77fe640 1 -- 172.21.15.204:0/2098490010 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33a0030360 con 0x7f33a8135bd0 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33af15d640 5 monclient: authenticate success, global_id 5311 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33af15d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33af15d640 4 set_mon_vals no callback set 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33a77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.846+0000 7f33a77fe640 10 monclient: dump: 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:58.863 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:58.864 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:58.864 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:58.864 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:58.864 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:58.864 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:58.864 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:58.864 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:58.864 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:58.864 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:58.864 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.847+0000 7f33af15d640 10 monclient: shutdown 2024-09-15T01:33:58.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.847+0000 7f33af15d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.847+0000 7f33af15d640 1 -- 172.21.15.204:0/2098490010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a8135bd0 msgr2=0x7f33a8137fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.847+0000 7f33af15d640 1 --2- 172.21.15.204:0/2098490010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a8135bd0 0x7f33a8137fb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009f20 tx=0x7f33a0009840 comp rx=0 tx=0).stop 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.847+0000 7f33af15d640 1 -- 172.21.15.204:0/2098490010 shutdown_connections 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.847+0000 7f33af15d640 1 --2- 172.21.15.204:0/2098490010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a8135bd0 0x7f33a8137fb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.847+0000 7f33af15d640 1 --2- 172.21.15.204:0/2098490010 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33a806eea0 0x7f33a809ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.847+0000 7f33af15d640 1 -- 172.21.15.204:0/2098490010 >> 172.21.15.204:0/2098490010 conn(0x7f33a8067740 msgr2=0x7f33a80a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.847+0000 7f33af15d640 1 -- 172.21.15.204:0/2098490010 shutdown_connections 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.847+0000 7f33af15d640 1 -- 172.21.15.204:0/2098490010 wait complete. 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.847+0000 7f33af15d640 10 monclient: build_initial_monmap 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.847+0000 7f33af15d640 10 monclient: monmap: 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:58.865 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 1 Processor -- start 2024-09-15T01:33:58.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 1 -- start start 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 10 monclient: init 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 10 monclient: _add_conns ranks=[3,4,6,5,0,8,7,1,2] 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a806eea0 0x7f33a8135390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 10 monclient(hunting): picked mon.noname-g con 0x7f33a806eea0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33a80a4680 0x7f33a809cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 10 monclient(hunting): picked mon.noname-c con 0x7f33a80a4680 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f33a8135bd0 0x7f33a809d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 10 monclient(hunting): picked mon.noname-d con 0x7f33a8135bd0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a8132150 con 0x7f33a806eea0 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33a806d0d0 con 0x7f33a80a4680 2024-09-15T01:33:58.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f33a806cce0 con 0x7f33a8135bd0 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33acef9640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a806eea0 msgr2=0x7f33a8135390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33af15d640 10 monclient(hunting): authenticate will time out at 854.491733s 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33acef9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a806eea0 0x7f33a8135390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33ad6fa640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f33a8135bd0 msgr2=0x7f33a809d080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33ad6fa640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f33a8135bd0 0x7f33a809d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33a7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33a80a4680 0x7f33a809cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33a7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33a80a4680 0x7f33a809cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:60332/0 (socket says 172.21.15.204:60332) 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33a7fff640 1 -- 172.21.15.204:0/642529023 learned_addr learned my addr 172.21.15.204:0/642529023 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33a7fff640 10 monclient(hunting): get_auth_request con 0x7f33a80a4680 auth_method 0 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:58.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.848+0000 7f33a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 10 monclient(hunting): handle_auth_done global_id 4882 payload 824 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 1 -- 172.21.15.204:0/642529023 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f33a8135bd0 msgr2=0x7f33a809d080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 1 --2- 172.21.15.204:0/642529023 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f33a8135bd0 0x7f33a809d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 1 -- 172.21.15.204:0/642529023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a806eea0 msgr2=0x7f33a8135390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 1 --2- 172.21.15.204:0/642529023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a806eea0 0x7f33a8135390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 1 monclient: found mon.noname-c 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:33:58.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 1 -- 172.21.15.204:0/642529023 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a0009ae0 con 0x7f33a80a4680 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a7fff640 1 --2- 172.21.15.204:0/642529023 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33a80a4680 0x7f33a809cb40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f339400c8b0 tx=0x7f3394008040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a4ff9640 1 -- 172.21.15.204:0/642529023 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f339400ce40 con 0x7f33a80a4680 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a4ff9640 10 monclient: dump: 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a4ff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a4ff9640 1 -- 172.21.15.204:0/642529023 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3394007cb0 con 0x7f33a80a4680 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33af15d640 5 monclient: authenticate success, global_id 4882 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33af15d640 10 monclient: _renew_subs 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33af15d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33af15d640 1 -- 172.21.15.204:0/642529023 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a8133450 con 0x7f33a80a4680 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a4ff9640 1 -- 172.21.15.204:0/642529023 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3394007e20 con 0x7f33a80a4680 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a4ff9640 10 monclient: dump: 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33a6ffd640 4 set_mon_vals no callback set 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33af15d640 10 monclient: _renew_subs 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33af15d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.849+0000 7f33af15d640 1 -- 172.21.15.204:0/642529023 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a8131920 con 0x7f33a80a4680 2024-09-15T01:33:58.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.850+0000 7f33a4ff9640 1 -- 172.21.15.204:0/642529023 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3394007830 con 0x7f33a80a4680 2024-09-15T01:33:58.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.850+0000 7f33af15d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:58.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.850+0000 7f33af15d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:58.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.850+0000 7f33af15d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:58.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.850+0000 7f33af15d640 1 -- 172.21.15.204:0/642529023 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a8070cc0 con 0x7f33a80a4680 2024-09-15T01:33:58.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.850+0000 7f33a4ff9640 1 --2- 172.21.15.204:0/642529023 >> v2:172.21.15.204:6812/28210 conn(0x7f338c0397b0 0x7f338c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:58.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.850+0000 7f33a4ff9640 1 -- 172.21.15.204:0/642529023 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4191+0+0 (secure 0 0 0) 0x7f339404b060 con 0x7f33a80a4680 2024-09-15T01:33:58.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.850+0000 7f33acef9640 1 --2- 172.21.15.204:0/642529023 >> v2:172.21.15.204:6812/28210 conn(0x7f338c0397b0 0x7f338c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:58.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.850+0000 7f33acef9640 10 monclient: get_auth_request con 0x7f338c0397b0 auth_method 0 2024-09-15T01:33:58.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.851+0000 7f33acef9640 1 --2- 172.21.15.204:0/642529023 >> v2:172.21.15.204:6812/28210 conn(0x7f338c0397b0 0x7f338c03bc60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f339c002060 tx=0x7f339c007ab0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:58.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.855+0000 7f33a4ff9640 1 -- 172.21.15.204:0/642529023 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f339407ce40 con 0x7f33a80a4680 2024-09-15T01:33:58.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.855+0000 7f33a4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:58.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:58.855+0000 7f33a4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:59.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.013+0000 7f33af15d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}] 2024-09-15T01:33:59.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.013+0000 7f33af15d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}] 2024-09-15T01:33:59.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.013+0000 7f33af15d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:33:59.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.013+0000 7f33af15d640 1 -- 172.21.15.204:0/642529023 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"} v 0) v1 -- 0x7f33a806a8b0 con 0x7f33a80a4680 2024-09-15T01:33:59.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.014+0000 7f33a4ff9640 1 -- 172.21.15.204:0/642529023 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}]=-22 type 'osd' is for devices, not buckets v67) v1 ==== 136+0+0 (secure 0 0 0) 0x7f3394007ae0 con 0x7f33a80a4680 2024-09-15T01:33:59.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.014+0000 7f33a4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}] 2024-09-15T01:33:59.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.014+0000 7f33a4ff9640 10 monclient: _finish_command 2 = mon:22 type 'osd' is for devices, not buckets 2024-09-15T01:33:59.017 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: type 'osd' is for devices, not buckets 2024-09-15T01:33:59.020 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:33:59.020 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket foo root 2024-09-15T01:33:59.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.092+0000 7f93a311d640 10 monclient: get_monmap_and_config 2024-09-15T01:33:59.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.092+0000 7f93a311d640 10 monclient: build_initial_monmap 2024-09-15T01:33:59.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.092+0000 7f93a311d640 10 monclient: monmap: 2024-09-15T01:33:59.095 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:59.096 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:59.096 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:59.094436+0000 2024-09-15T01:33:59.096 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:59.094436+0000 2024-09-15T01:33:59.096 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:59.096 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:59.096 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:59.096 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:59.096 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:59.096 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:59.096 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:59.096 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:59.096 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:59.097 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:59.097 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:59.097 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:59.097 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:59.097 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.093+0000 7f93a311d640 1 Processor -- start 2024-09-15T01:33:59.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.093+0000 7f93a311d640 1 -- start start 2024-09-15T01:33:59.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.093+0000 7f93a311d640 10 monclient: init 2024-09-15T01:33:59.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:59.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 10 monclient: _add_conns ranks=[3,1,6,9,7,0,5,2,10,8,4] 2024-09-15T01:33:59.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c0a2e30 0x7f939c132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 10 monclient(hunting): picked mon.noname-g con 0x7f939c0a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:59.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f939c09fb80 addr v1:172.21.15.204:6789/0 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f939c09ff70 0x7f939c0a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 10 monclient(hunting): picked mon.noname-d con 0x7f939c09ff70 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939c04b720 con 0x7f939c09fb80 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c09e6c0 con 0x7f939c0a2e30 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f939c0a3200 con 0x7f939c09ff70 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 10 monclient(hunting): authenticate will time out at 854.737220s 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a0eb9640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c0a2e30 msgr2=0x7f939c132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a0eb9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c0a2e30 0x7f939c132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939bfff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f939c09fb80 0x7f939c0a0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:48786/0 (socket says 172.21.15.204:48786) 2024-09-15T01:33:59.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a16ba640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f939c09ff70 msgr2=0x7f939c0a1150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:59.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939bfff640 1 -- 172.21.15.204:0/3537858753 learned_addr learned my addr 172.21.15.204:0/3537858753 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:33:59.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a16ba640 1 --2- 172.21.15.204:0/3537858753 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f939c09ff70 0x7f939c0a1150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:59.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604157452 0 0) 0x7f939c04b720 con 0x7f939c09fb80 2024-09-15T01:33:59.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:59.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:59.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient(hunting): my global_id is 5320 2024-09-15T01:33:59.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93840034e0 con 0x7f939c09fb80 2024-09-15T01:33:59.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 470810528 0 0) 0x7f93840034e0 con 0x7f939c09fb80 2024-09-15T01:33:59.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f939c09ff70 msgr2=0x7f939c0a1150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:59.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 --2- 172.21.15.204:0/3537858753 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f939c09ff70 0x7f939c0a1150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c0a2e30 msgr2=0x7f939c132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 --2- 172.21.15.204:0/3537858753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c0a2e30 0x7f939c132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c0a1f10 con 0x7f939c09fb80 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 5 monclient: authenticate success, global_id 5320 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f93a311d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f9390001c70 con 0x7f939c09fb80 2024-09-15T01:33:59.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient: dump: 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:59.111 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:59.112 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:59.112 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:59.112 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:59.112 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:59.112 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:33:59.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:59.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 >> v1:172.21.15.204:6789/0 conn(0x7f939c09fb80 legacy=0x7f939c0a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:59.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient: _add_conns ranks=[1,2,3,8,0,7,6,4,5] 2024-09-15T01:33:59.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 --2- 172.21.15.204:0/3537858753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9384004220 0x7f9384004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient(hunting): picked mon.f con 0x7f9384004220 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 --2- 172.21.15.204:0/3537858753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9384004ec0 0x7f93840099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient(hunting): picked mon.b con 0x7f9384004ec0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 --2- 172.21.15.204:0/3537858753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c0a2e30 0x7f938400bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient(hunting): picked mon.g con 0x7f939c0a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c09e6c0 con 0x7f9384004220 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c0a3200 con 0x7f9384004ec0 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.094+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938400c5e0 con 0x7f939c0a2e30 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f939b7fe640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f939b7fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a311d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:59.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 1 --2- 172.21.15.204:0/3537858753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9384004220 0x7f9384004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:59.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a16ba640 1 -- 172.21.15.204:0/3537858753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c0a2e30 msgr2=0x7f938400bf10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:59.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a16ba640 1 --2- 172.21.15.204:0/3537858753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c0a2e30 0x7f938400bf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:59.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 10 monclient(hunting): get_auth_request con 0x7f9384004220 auth_method 0 2024-09-15T01:33:59.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:59.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:59.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:33:59.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:59.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:59.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:33:59.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 10 monclient(hunting): handle_auth_done global_id 5320 payload 293 2024-09-15T01:33:59.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 1 -- 172.21.15.204:0/3537858753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c0a2e30 msgr2=0x7f938400bf10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:59.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 1 --2- 172.21.15.204:0/3537858753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c0a2e30 0x7f938400bf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 1 -- 172.21.15.204:0/3537858753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9384004ec0 msgr2=0x7f93840099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 1 --2- 172.21.15.204:0/3537858753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9384004ec0 0x7f93840099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 10 monclient: _finish_hunting 0 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 1 monclient: found mon.f 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 1 -- 172.21.15.204:0/3537858753 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93840034e0 con 0x7f9384004220 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 10 monclient: _finish_auth 0 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.095+0000 7f93a0eb9640 1 --2- 172.21.15.204:0/3537858753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9384004220 0x7f9384004780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f938c0099c0 tx=0x7f938c009060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22869032d6b02ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f938c003df0 con 0x7f9384004220 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f939b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f939b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:59.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f939b7fe640 10 monclient: dump: 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:59.116 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:59.117 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:59.117 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c02a930 con 0x7f9384004220 2024-09-15T01:33:59.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f93a311d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:59.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f939b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:59.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f939b7fe640 1 -- 172.21.15.204:0/3537858753 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f938c0050f0 con 0x7f9384004220 2024-09-15T01:33:59.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f93a311d640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:59.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f939b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f93a311d640 4 set_mon_vals no callback set 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f939b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f939b7fe640 10 monclient: dump: 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:59.118 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:59.119 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:59.119 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:59.119 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:59.119 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:59.119 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:59.119 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f93a311d640 10 monclient: shutdown 2024-09-15T01:33:59.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f93a311d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:59.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f93a311d640 1 -- 172.21.15.204:0/3537858753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9384004220 msgr2=0x7f9384004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:59.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f93a311d640 1 --2- 172.21.15.204:0/3537858753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9384004220 0x7f9384004780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f938c0099c0 tx=0x7f938c009060 comp rx=0 tx=0).stop 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f93a16ba640 1 -- 172.21.15.204:0/3537858753 reap_dead start 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f93a311d640 1 -- 172.21.15.204:0/3537858753 shutdown_connections 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f93a311d640 1 -- 172.21.15.204:0/3537858753 >> 172.21.15.204:0/3537858753 conn(0x7f939c06e860 msgr2=0x7f939c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f93a311d640 1 -- 172.21.15.204:0/3537858753 shutdown_connections 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.096+0000 7f93a311d640 1 -- 172.21.15.204:0/3537858753 wait complete. 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.097+0000 7f93a311d640 10 monclient: build_initial_monmap 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.097+0000 7f93a311d640 10 monclient: monmap: 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:59.120 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:59.121 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:59.121 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:59.121 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:59.121 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:59.121 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:59.121 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:59.121 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:59.121 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:59.121 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:59.122 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.097+0000 7f93a311d640 1 Processor -- start 2024-09-15T01:33:59.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.097+0000 7f93a311d640 1 -- start start 2024-09-15T01:33:59.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.097+0000 7f93a311d640 10 monclient: init 2024-09-15T01:33:59.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:59.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient: _add_conns ranks=[5,3,1,7,6,2,4,8,0] 2024-09-15T01:33:59.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9384004220 0x7f939c131870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient(hunting): picked mon.noname-h con 0x7f9384004220 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:33:59.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c131db0 0x7f939c11e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient(hunting): picked mon.noname-g con 0x7f939c131db0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:33:59.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f939c11e9c0 0x7f939c11edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient(hunting): picked mon.noname-f con 0x7f939c11e9c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c132180 con 0x7f939c11e9c0 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c1326a0 con 0x7f939c131db0 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f939c121ca0 con 0x7f9384004220 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient(hunting): authenticate will time out at 854.741110s 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f939bfff640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c131db0 msgr2=0x7f939c11e480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f939bfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c131db0 0x7f939c11e480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f939c11e9c0 0x7f939c11edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a0eb9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9384004220 0x7f939c131870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:59.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f939c11e9c0 0x7f939c11edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38108/0 (socket says 172.21.15.204:38108) 2024-09-15T01:33:59.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 1 -- 172.21.15.204:0/3595462104 learned_addr learned my addr 172.21.15.204:0/3595462104 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:59.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 10 monclient(hunting): get_auth_request con 0x7f939c11e9c0 auth_method 0 2024-09-15T01:33:59.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:59.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:59.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:59.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a0eb9640 10 monclient(hunting): get_auth_request con 0x7f9384004220 auth_method 0 2024-09-15T01:33:59.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a0eb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:59.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a0eb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:59.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a0eb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:59.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:59.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:59.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a0eb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a0eb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a0eb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 10 monclient(hunting): handle_auth_done global_id 5329 payload 824 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 1 -- 172.21.15.204:0/3595462104 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9384004220 msgr2=0x7f939c131870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 1 --2- 172.21.15.204:0/3595462104 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9384004220 0x7f939c131870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 1 -- 172.21.15.204:0/3595462104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c131db0 msgr2=0x7f939c11e480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 1 --2- 172.21.15.204:0/3595462104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c131db0 0x7f939c11e480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 10 monclient: _finish_hunting 0 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 1 monclient: found mon.noname-f 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 1 -- 172.21.15.204:0/3595462104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c11fe90 con 0x7f939c11e9c0 2024-09-15T01:33:59.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a16ba640 1 --2- 172.21.15.204:0/3595462104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f939c11e9c0 0x7f939c11edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f939c0a3610 tx=0x7f9394006eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f9398ff9640 1 -- 172.21.15.204:0/3595462104 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9394003c50 con 0x7f939c11e9c0 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f9398ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f9398ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f9398ff9640 10 monclient: dump: 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:59.126 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f9398ff9640 10 monclient: _finish_auth 0 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f9398ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 5 monclient: authenticate success, global_id 5329 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f9398ff9640 1 -- 172.21.15.204:0/3595462104 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9394003df0 con 0x7f939c11e9c0 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient: _renew_subs 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 1 -- 172.21.15.204:0/3595462104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c1219c0 con 0x7f939c11e9c0 2024-09-15T01:33:59.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f9398ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f9398ff9640 1 -- 172.21.15.204:0/3595462104 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93940065f0 con 0x7f939c11e9c0 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f9398ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f9398ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f9398ff9640 10 monclient: dump: 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:59.128 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f939affd640 4 set_mon_vals no callback set 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient: _renew_subs 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.098+0000 7f93a311d640 1 -- 172.21.15.204:0/3595462104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c133f50 con 0x7f939c11e9c0 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.099+0000 7f9398ff9640 1 -- 172.21.15.204:0/3595462104 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9394006170 con 0x7f939c11e9c0 2024-09-15T01:33:59.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.099+0000 7f9398ff9640 1 --2- 172.21.15.204:0/3595462104 >> v2:172.21.15.204:6812/28210 conn(0x7f93880397b0 0x7f938803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.099+0000 7f93a311d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:59.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.099+0000 7f93a311d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:59.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.099+0000 7f93a311d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:59.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.099+0000 7f9398ff9640 1 -- 172.21.15.204:0/3595462104 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4191+0+0 (secure 0 0 0) 0x7f9394049880 con 0x7f939c11e9c0 2024-09-15T01:33:59.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.099+0000 7f93a311d640 1 -- 172.21.15.204:0/3595462104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939c070cc0 con 0x7f939c11e9c0 2024-09-15T01:33:59.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.099+0000 7f93a0eb9640 1 --2- 172.21.15.204:0/3595462104 >> v2:172.21.15.204:6812/28210 conn(0x7f93880397b0 0x7f938803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:59.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.100+0000 7f93a0eb9640 10 monclient: get_auth_request con 0x7f93880397b0 auth_method 0 2024-09-15T01:33:59.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.100+0000 7f93a0eb9640 1 --2- 172.21.15.204:0/3595462104 >> v2:172.21.15.204:6812/28210 conn(0x7f93880397b0 0x7f938803bc60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f938c032000 tx=0x7f938c030000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:59.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.104+0000 7f9398ff9640 1 -- 172.21.15.204:0/3595462104 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9394016070 con 0x7f939c11e9c0 2024-09-15T01:33:59.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.104+0000 7f9398ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:59.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.104+0000 7f9398ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:33:59.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.260+0000 7f93a311d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-09-15T01:33:59.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.260+0000 7f93a311d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-09-15T01:33:59.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.260+0000 7f93a311d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:59.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.260+0000 7f93a311d640 1 -- 172.21.15.204:0/3595462104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "root"} v 0) v1 -- 0x7f939c09ddb0 con 0x7f939c11e9c0 2024-09-15T01:33:59.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.675+0000 7f9398ff9640 1 -- 172.21.15.204:0/3595462104 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}]=0 added bucket foo type root to crush map v68) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9394006750 con 0x7f939c11e9c0 2024-09-15T01:33:59.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.675+0000 7f9398ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-09-15T01:33:59.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.675+0000 7f9398ff9640 10 monclient: _finish_command 2 = system:0 added bucket foo type root to crush map 2024-09-15T01:33:59.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.772+0000 7f93a311d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-09-15T01:33:59.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.772+0000 7f93a311d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-09-15T01:33:59.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.772+0000 7f93a311d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:59.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.772+0000 7f93a311d640 1 -- 172.21.15.204:0/3595462104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "root"} v 0) v1 -- 0x7f939c120ac0 con 0x7f939c11e9c0 2024-09-15T01:33:59.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.773+0000 7f9398ff9640 1 -- 172.21.15.204:0/3595462104 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}]=0 bucket 'foo' already exists v68) v1 ==== 126+0+0 (secure 0 0 0) 0x7f9394010e60 con 0x7f939c11e9c0 2024-09-15T01:33:59.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.773+0000 7f9398ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-09-15T01:33:59.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.773+0000 7f9398ff9640 10 monclient: _finish_command 3 = system:0 bucket 'foo' already exists 2024-09-15T01:33:59.776 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'foo' already exists 2024-09-15T01:33:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 1 -- 172.21.15.204:0/3595462104 >> v2:172.21.15.204:6812/28210 conn(0x7f93880397b0 msgr2=0x7f938803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 1 --2- 172.21.15.204:0/3595462104 >> v2:172.21.15.204:6812/28210 conn(0x7f93880397b0 0x7f938803bc60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f938c032000 tx=0x7f938c030000 comp rx=0 tx=0).stop 2024-09-15T01:33:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 10 monclient: shutdown 2024-09-15T01:33:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 1 -- 172.21.15.204:0/3595462104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f939c11e9c0 msgr2=0x7f939c11edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 1 --2- 172.21.15.204:0/3595462104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f939c11e9c0 0x7f939c11edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f939c0a3610 tx=0x7f9394006eb0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 1 -- 172.21.15.204:0/3595462104 shutdown_connections 2024-09-15T01:33:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 1 --2- 172.21.15.204:0/3595462104 >> v2:172.21.15.204:6812/28210 conn(0x7f93880397b0 0x7f938803bc60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 1 --2- 172.21.15.204:0/3595462104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f939c11e9c0 0x7f939c11edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 1 --2- 172.21.15.204:0/3595462104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f939c131db0 0x7f939c11e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 1 --2- 172.21.15.204:0/3595462104 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9384004220 0x7f939c131870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 1 -- 172.21.15.204:0/3595462104 >> 172.21.15.204:0/3595462104 conn(0x7f939c06e860 msgr2=0x7f939c09bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:59.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 1 -- 172.21.15.204:0/3595462104 shutdown_connections 2024-09-15T01:33:59.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.775+0000 7f93a311d640 1 -- 172.21.15.204:0/3595462104 wait complete. 2024-09-15T01:33:59.787 INFO:tasks.workunit.client.0.smithi204.stderr:++ ceph osd create 2024-09-15T01:33:59.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.856+0000 7f8b1464f640 10 monclient: get_monmap_and_config 2024-09-15T01:33:59.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.856+0000 7f8b1464f640 10 monclient: build_initial_monmap 2024-09-15T01:33:59.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.856+0000 7f8b1464f640 10 monclient: monmap: 2024-09-15T01:33:59.859 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:59.859 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:59.859 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:33:59.857941+0000 2024-09-15T01:33:59.859 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:33:59.857941+0000 2024-09-15T01:33:59.859 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:59.859 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:59.859 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:33:59.859 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:33:59.859 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:59.859 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:59.860 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:59.860 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:59.860 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:59.860 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:59.860 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:59.860 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:33:59.860 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:33:59.860 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 1 Processor -- start 2024-09-15T01:33:59.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 1 -- start start 2024-09-15T01:33:59.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 10 monclient: init 2024-09-15T01:33:59.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:59.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 10 monclient: _add_conns ranks=[6,0,9,10,1,7,8,3,5,2,4] 2024-09-15T01:33:59.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b0c0a2400 0x7f8b0c0a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 10 monclient(hunting): picked mon.noname-d con 0x7f8b0c0a2400 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:33:59.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8b0c09f990 addr v1:172.21.15.178:6789/0 2024-09-15T01:33:59.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b0c137c60 0x7f8b0c13a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 10 monclient(hunting): picked mon.noname-a con 0x7f8b0c137c60 addr v2:172.21.15.178:3300/0 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0c0648a0 con 0x7f8b0c09f990 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0c09dc50 con 0x7f8b0c137c60 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c09e1a0 con 0x7f8b0c0a2400 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b1464f640 10 monclient(hunting): authenticate will time out at 855.500814s 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b123eb640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b0c0a2400 msgr2=0x7f8b0c0a0ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.857+0000 7f8b123eb640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b0c0a2400 0x7f8b0c0a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b0c137c60 0x7f8b0c13a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b0c137c60 0x7f8b0c13a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:34262/0 (socket says 172.21.15.204:34262) 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 1 -- 172.21.15.204:0/1000649741 learned_addr learned my addr 172.21.15.204:0/1000649741 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 10 monclient(hunting): get_auth_request con 0x7f8b0c137c60 auth_method 0 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 10 monclient(hunting): handle_auth_done global_id 4995 payload 274 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 1 -- 172.21.15.204:0/1000649741 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b0c0a2400 msgr2=0x7f8b0c0a0ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 1 --2- 172.21.15.204:0/1000649741 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b0c0a2400 0x7f8b0c0a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 1 -- 172.21.15.204:0/1000649741 >> v1:172.21.15.178:6789/0 conn(0x7f8b0c09f990 legacy=0x7f8b0c09fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 10 monclient: _finish_hunting 0 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 1 monclient: found mon.noname-a 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.858+0000 7f8b12bec640 1 -- 172.21.15.204:0/1000649741 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c0a27d0 con 0x7f8b0c137c60 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b12bec640 1 --2- 172.21.15.204:0/1000649741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c137c60 0x7f8b0c13a040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009f20 tx=0x7f8b04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e16e021e74f96f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b113e9640 1 -- 172.21.15.204:0/1000649741 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b04007880 con 0x7f8b0c137c60 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b113e9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b113e9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b113e9640 10 monclient: dump: 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:59.873 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b113e9640 10 monclient: _finish_auth 0 2024-09-15T01:33:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b113e9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b113e9640 1 -- 172.21.15.204:0/1000649741 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0402d050 con 0x7f8b0c137c60 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b1464f640 5 monclient: authenticate success, global_id 4995 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b1464f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b113e9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b1464f640 10 monclient: get_monmap_and_config success 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b113e9640 1 -- 172.21.15.204:0/1000649741 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b04030360 con 0x7f8b0c137c60 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b1464f640 4 set_mon_vals no callback set 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b113e9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b113e9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.859+0000 7f8b113e9640 10 monclient: dump: 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.874 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:59.875 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:59.875 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:59.875 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:59.875 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:59.875 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:59.875 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:59.875 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:59.875 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:59.875 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:59.876 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:59.876 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 10 monclient: shutdown 2024-09-15T01:33:59.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:33:59.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 1 -- 172.21.15.204:0/1000649741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c137c60 msgr2=0x7f8b0c13a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:59.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 1 --2- 172.21.15.204:0/1000649741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c137c60 0x7f8b0c13a040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009f20 tx=0x7f8b04009840 comp rx=0 tx=0).stop 2024-09-15T01:33:59.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 1 -- 172.21.15.204:0/1000649741 shutdown_connections 2024-09-15T01:33:59.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 1 --2- 172.21.15.204:0/1000649741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c137c60 0x7f8b0c13a040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 1 --2- 172.21.15.204:0/1000649741 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b0c0a2400 0x7f8b0c0a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 1 -- 172.21.15.204:0/1000649741 >> 172.21.15.204:0/1000649741 conn(0x7f8b0c06e860 msgr2=0x7f8b0c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:33:59.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 1 -- 172.21.15.204:0/1000649741 shutdown_connections 2024-09-15T01:33:59.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 1 -- 172.21.15.204:0/1000649741 wait complete. 2024-09-15T01:33:59.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 10 monclient: build_initial_monmap 2024-09-15T01:33:59.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 10 monclient: monmap: 2024-09-15T01:33:59.877 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:33:59.878 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:33:59.878 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:33:59.878 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:33:59.878 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:33:59.878 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:33:59.878 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:33:59.878 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:33:59.878 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:33:59.878 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:33:59.878 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:33:59.878 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:33:59.878 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:33:59.879 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:33:59.879 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:33:59.879 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.860+0000 7f8b1464f640 1 Processor -- start 2024-09-15T01:33:59.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 1 -- start start 2024-09-15T01:33:59.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 10 monclient: init 2024-09-15T01:33:59.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 10 monclient: _reopen_session rank -1 2024-09-15T01:33:59.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 10 monclient: _add_conns ranks=[0,1,8,2,5,7,4,6,3] 2024-09-15T01:33:59.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c09f990 0x7f8b0c09baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 10 monclient(hunting): picked mon.noname-a con 0x7f8b0c09f990 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:33:59.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b0c0a2400 0x7f8b0c12eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 10 monclient(hunting): picked mon.noname-f con 0x7f8b0c0a2400 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b0c137c60 0x7f8b0c12f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 10 monclient(hunting): picked mon.noname-e con 0x7f8b0c137c60 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c09d3d0 con 0x7f8b0c09f990 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c09cad0 con 0x7f8b0c0a2400 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c131930 con 0x7f8b0c137c60 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 10 monclient(hunting): _renew_subs 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b1464f640 10 monclient(hunting): authenticate will time out at 855.504471s 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b0c0a2400 0x7f8b0c12eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b0c0a2400 0x7f8b0c12eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38116/0 (socket says 172.21.15.204:38116) 2024-09-15T01:33:59.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 1 -- 172.21.15.204:0/2999236506 learned_addr learned my addr 172.21.15.204:0/2999236506 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:33:59.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 10 monclient(hunting): get_auth_request con 0x7f8b0c0a2400 auth_method 0 2024-09-15T01:33:59.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:59.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:59.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:59.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b123eb640 1 --2- 172.21.15.204:0/2999236506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c09f990 0x7f8b0c09baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:59.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b12bec640 1 --2- 172.21.15.204:0/2999236506 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b0c137c60 0x7f8b0c12f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:59.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b123eb640 10 monclient(hunting): get_auth_request con 0x7f8b0c09f990 auth_method 0 2024-09-15T01:33:59.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b123eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:59.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b123eb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:59.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b123eb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:59.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b12bec640 10 monclient(hunting): get_auth_request con 0x7f8b0c137c60 auth_method 0 2024-09-15T01:33:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b12bec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:33:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b12bec640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:33:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b12bec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:33:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b123eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b123eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b123eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b12bec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:33:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b12bec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:33:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b12bec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 10 monclient(hunting): handle_auth_done global_id 5338 payload 824 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 1 -- 172.21.15.204:0/2999236506 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b0c137c60 msgr2=0x7f8b0c12f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 1 --2- 172.21.15.204:0/2999236506 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b0c137c60 0x7f8b0c12f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 1 -- 172.21.15.204:0/2999236506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c09f990 msgr2=0x7f8b0c09baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 1 --2- 172.21.15.204:0/2999236506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c09f990 0x7f8b0c09baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 10 monclient: _finish_hunting 0 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 1 monclient: found mon.noname-f 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.861+0000 7f8b11bea640 1 -- 172.21.15.204:0/2999236506 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b04009ae0 con 0x7f8b0c0a2400 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b11bea640 1 --2- 172.21.15.204:0/2999236506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b0c0a2400 0x7f8b0c12eed0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00c590 tx=0x7f8afc007eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b027fc640 1 -- 172.21.15.204:0/2999236506 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8afc012ab0 con 0x7f8b0c0a2400 2024-09-15T01:33:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b027fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:33:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b027fc640 10 monclient: dump: 2024-09-15T01:33:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:59.885 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b027fc640 10 monclient: _finish_auth 0 2024-09-15T01:33:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:33:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b1464f640 5 monclient: authenticate success, global_id 5338 2024-09-15T01:33:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b1464f640 10 monclient: _renew_subs 2024-09-15T01:33:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b1464f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b1464f640 1 -- 172.21.15.204:0/2999236506 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c131650 con 0x7f8b0c0a2400 2024-09-15T01:33:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b027fc640 1 -- 172.21.15.204:0/2999236506 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc012c10 con 0x7f8b0c0a2400 2024-09-15T01:33:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b027fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:33:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b1464f640 10 monclient: _renew_subs 2024-09-15T01:33:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b027fc640 1 -- 172.21.15.204:0/2999236506 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8afc012d80 con 0x7f8b0c0a2400 2024-09-15T01:33:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b1464f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b1464f640 1 -- 172.21.15.204:0/2999236506 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c1306f0 con 0x7f8b0c0a2400 2024-09-15T01:33:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:33:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b027fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:33:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b027fc640 10 monclient: dump: 2024-09-15T01:33:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:33:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:33:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:33:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:33:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:33:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:33:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:33:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:33:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:33:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:33:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:33:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:33:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:33:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:33:59.888 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:33:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.862+0000 7f8b10be8640 4 set_mon_vals no callback set 2024-09-15T01:33:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.863+0000 7f8b027fc640 1 -- 172.21.15.204:0/2999236506 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8afc012630 con 0x7f8b0c0a2400 2024-09-15T01:33:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.863+0000 7f8b027fc640 1 --2- 172.21.15.204:0/2999236506 >> v2:172.21.15.204:6812/28210 conn(0x7f8af00397b0 0x7f8af003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:33:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.863+0000 7f8b027fc640 1 -- 172.21.15.204:0/2999236506 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4218+0+0 (secure 0 0 0) 0x7f8afc04ac90 con 0x7f8b0c0a2400 2024-09-15T01:33:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.863+0000 7f8b1464f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.863+0000 7f8b1464f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.863+0000 7f8b1464f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:33:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.863+0000 7f8b1464f640 1 -- 172.21.15.204:0/2999236506 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af4005010 con 0x7f8b0c0a2400 2024-09-15T01:33:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.863+0000 7f8b123eb640 1 --2- 172.21.15.204:0/2999236506 >> v2:172.21.15.204:6812/28210 conn(0x7f8af00397b0 0x7f8af003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:33:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.863+0000 7f8b123eb640 10 monclient: get_auth_request con 0x7f8af00397b0 auth_method 0 2024-09-15T01:33:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.863+0000 7f8b123eb640 1 --2- 172.21.15.204:0/2999236506 >> v2:172.21.15.204:6812/28210 conn(0x7f8af00397b0 0x7f8af003bc60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8b080038f0 tx=0x7f8b080090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:33:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.867+0000 7f8b027fc640 1 -- 172.21.15.204:0/2999236506 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8afc011380 con 0x7f8b0c0a2400 2024-09-15T01:33:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.867+0000 7f8b027fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:33:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:33:59.867+0000 7f8b027fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:00.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:00.033+0000 7f8b1464f640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T01:34:00.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:00.033+0000 7f8b1464f640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-09-15T01:34:00.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:00.033+0000 7f8b1464f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:00.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:00.033+0000 7f8b1464f640 1 -- 172.21.15.204:0/2999236506 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7f8af4005590 con 0x7f8b0c0a2400 2024-09-15T01:34:00.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:00.686+0000 7f8b027fc640 1 -- 172.21.15.204:0/2999236506 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v69) v1 ==== 58+0+2 (secure 0 0 0) 0x7f8afc0182b0 con 0x7f8b0c0a2400 2024-09-15T01:34:00.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:00.686+0000 7f8b027fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-09-15T01:34:00.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:00.686+0000 7f8b027fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:00.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:00.792+0000 7f8b1464f640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T01:34:00.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:00.792+0000 7f8b1464f640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-09-15T01:34:00.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:00.792+0000 7f8b1464f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:00.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:00.792+0000 7f8b1464f640 1 -- 172.21.15.204:0/2999236506 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7f8af40023c0 con 0x7f8b0c0a2400 2024-09-15T01:34:01.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.688+0000 7f8b027fc640 1 -- 172.21.15.204:0/2999236506 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v70) v1 ==== 58+0+2 (secure 0 0 0) 0x7f8afc018490 con 0x7f8b0c0a2400 2024-09-15T01:34:01.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.688+0000 7f8b027fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-09-15T01:34:01.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.688+0000 7f8b027fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:01.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 1 -- 172.21.15.204:0/2999236506 >> v2:172.21.15.204:6812/28210 conn(0x7f8af00397b0 msgr2=0x7f8af003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:01.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 1 --2- 172.21.15.204:0/2999236506 >> v2:172.21.15.204:6812/28210 conn(0x7f8af00397b0 0x7f8af003bc60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8b080038f0 tx=0x7f8b080090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 10 monclient: shutdown 2024-09-15T01:34:01.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:01.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 1 -- 172.21.15.204:0/2999236506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b0c0a2400 msgr2=0x7f8b0c12eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:01.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 1 --2- 172.21.15.204:0/2999236506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b0c0a2400 0x7f8b0c12eed0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00c590 tx=0x7f8afc007eb0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 1 -- 172.21.15.204:0/2999236506 shutdown_connections 2024-09-15T01:34:01.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 1 --2- 172.21.15.204:0/2999236506 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b0c137c60 0x7f8b0c12f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 1 --2- 172.21.15.204:0/2999236506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b0c0a2400 0x7f8b0c12eed0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 1 --2- 172.21.15.204:0/2999236506 >> v2:172.21.15.204:6812/28210 conn(0x7f8af00397b0 0x7f8af003bc60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 1 --2- 172.21.15.204:0/2999236506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c09f990 0x7f8b0c09baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 1 -- 172.21.15.204:0/2999236506 >> 172.21.15.204:0/2999236506 conn(0x7f8b0c06e860 msgr2=0x7f8b0c0a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:01.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 1 -- 172.21.15.204:0/2999236506 shutdown_connections 2024-09-15T01:34:01.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.691+0000 7f8b1464f640 1 -- 172.21.15.204:0/2999236506 wait complete. 2024-09-15T01:34:01.704 INFO:tasks.workunit.client.0.smithi204.stderr:+ o1=7 2024-09-15T01:34:01.705 INFO:tasks.workunit.client.0.smithi204.stderr:++ ceph osd create 2024-09-15T01:34:01.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.778+0000 7f32b8e0f640 10 monclient: get_monmap_and_config 2024-09-15T01:34:01.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.778+0000 7f32b8e0f640 10 monclient: build_initial_monmap 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.778+0000 7f32b8e0f640 10 monclient: monmap: 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:01.779721+0000 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:01.779721+0000 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:01.781 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:01.782 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:01.782 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:01.782 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:01.782 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:01.782 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:01.782 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:01.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.778+0000 7f32b8e0f640 1 Processor -- start 2024-09-15T01:34:01.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.778+0000 7f32b8e0f640 1 -- start start 2024-09-15T01:34:01.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.778+0000 7f32b8e0f640 10 monclient: init 2024-09-15T01:34:01.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:01.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 10 monclient: _add_conns ranks=[9,10,4,6,1,0,5,7,2,3,8] 2024-09-15T01:34:01.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f32b40a2400 0x7f32b40a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:01.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 10 monclient(hunting): picked mon.noname-a con 0x7f32b40a2400 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:01.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f32b409f990 0x7f32b409fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:01.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 10 monclient(hunting): picked mon.noname-f con 0x7f32b409f990 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:01.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32b4132930 0x7f32b4132db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:01.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 10 monclient(hunting): picked mon.noname-c con 0x7f32b4132930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:01.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:01.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b409e1a0 con 0x7f32b40a2400 2024-09-15T01:34:01.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:01.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b40a27d0 con 0x7f32b409f990 2024-09-15T01:34:01.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:01.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32b409dec0 con 0x7f32b4132930 2024-09-15T01:34:01.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:01.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b8e0f640 10 monclient(hunting): authenticate will time out at 857.422614s 2024-09-15T01:34:01.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b1d9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f32b409f990 0x7f32b409fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:01.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b1d9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f32b409f990 0x7f32b409fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38122/0 (socket says 172.21.15.204:38122) 2024-09-15T01:34:01.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b1d9b640 1 -- 172.21.15.204:0/1055853415 learned_addr learned my addr 172.21.15.204:0/1055853415 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:01.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b1d9b640 10 monclient(hunting): get_auth_request con 0x7f32b409f990 auth_method 0 2024-09-15T01:34:01.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:01.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:01.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:01.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b259c640 1 --2- 172.21.15.204:0/1055853415 >> v2:172.21.15.178:3300/0 conn(0x7f32b40a2400 0x7f32b40a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:01.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:01.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:01.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:01.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b259c640 10 monclient(hunting): get_auth_request con 0x7f32b40a2400 auth_method 0 2024-09-15T01:34:01.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b2d9d640 1 --2- 172.21.15.204:0/1055853415 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32b4132930 0x7f32b4132db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b2d9d640 10 monclient(hunting): get_auth_request con 0x7f32b4132930 auth_method 0 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b2d9d640 1 -- 172.21.15.204:0/1055853415 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32b4132930 msgr2=0x7f32b4132db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b2d9d640 1 -- 172.21.15.204:0/1055853415 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32b4132930 msgr2=0x7f32b4132db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b2d9d640 1 --2- 172.21.15.204:0/1055853415 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32b4132930 0x7f32b4132db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:01.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:01.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b2d9d640 1 --2- 172.21.15.204:0/1055853415 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32b4132930 0x7f32b4132db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:01.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.779+0000 7f32b1d9b640 10 monclient(hunting): handle_auth_done global_id 5347 payload 274 2024-09-15T01:34:01.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b1d9b640 1 -- 172.21.15.204:0/1055853415 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32b4132930 msgr2=0x7f32b4132db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:01.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b1d9b640 1 --2- 172.21.15.204:0/1055853415 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32b4132930 0x7f32b4132db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b1d9b640 1 -- 172.21.15.204:0/1055853415 >> v2:172.21.15.178:3300/0 conn(0x7f32b40a2400 msgr2=0x7f32b40a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:01.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b1d9b640 1 --2- 172.21.15.204:0/1055853415 >> v2:172.21.15.178:3300/0 conn(0x7f32b40a2400 0x7f32b40a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b1d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:34:01.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b1d9b640 1 monclient: found mon.noname-f 2024-09-15T01:34:01.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:01.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b1d9b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:01.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b1d9b640 1 -- 172.21.15.204:0/1055853415 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b40648a0 con 0x7f32b409f990 2024-09-15T01:34:01.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b1d9b640 1 --2- 172.21.15.204:0/1055853415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b409f990 0x7f32b409fd60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f32b40a1a50 tx=0x7f32a40091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e618ddc23cae1e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:01.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b159a640 1 -- 172.21.15.204:0/1055853415 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32a402acb0 con 0x7f32b409f990 2024-09-15T01:34:01.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:01.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b159a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:01.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b159a640 10 monclient: dump: 2024-09-15T01:34:01.797 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:01.797 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:01.797 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:01.797 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:01.797 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:01.797 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:01.797 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:01.797 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b159a640 10 monclient: _finish_auth 0 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b159a640 1 -- 172.21.15.204:0/1055853415 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a402ae30 con 0x7f32b409f990 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b8e0f640 5 monclient: authenticate success, global_id 5347 2024-09-15T01:34:01.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b8e0f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b8e0f640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b159a640 1 -- 172.21.15.204:0/1055853415 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32a4031ca0 con 0x7f32b409f990 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b8e0f640 4 set_mon_vals no callback set 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b159a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.780+0000 7f32b159a640 10 monclient: dump: 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:01.799 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:01.800 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:01.800 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:01.800 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:01.800 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:01.800 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:01.800 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:01.800 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:01.800 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:01.800 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:01.800 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:01.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.781+0000 7f32b8e0f640 10 monclient: shutdown 2024-09-15T01:34:01.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.781+0000 7f32b8e0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:01.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.781+0000 7f32b8e0f640 1 -- 172.21.15.204:0/1055853415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b409f990 msgr2=0x7f32b409fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:01.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.781+0000 7f32b8e0f640 1 --2- 172.21.15.204:0/1055853415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b409f990 0x7f32b409fd60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f32b40a1a50 tx=0x7f32a40091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.781+0000 7f32b8e0f640 1 -- 172.21.15.204:0/1055853415 shutdown_connections 2024-09-15T01:34:01.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.781+0000 7f32b8e0f640 1 --2- 172.21.15.204:0/1055853415 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32b4132930 0x7f32b4132db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.781+0000 7f32b8e0f640 1 --2- 172.21.15.204:0/1055853415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b409f990 0x7f32b409fd60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.781+0000 7f32b8e0f640 1 --2- 172.21.15.204:0/1055853415 >> v2:172.21.15.178:3300/0 conn(0x7f32b40a2400 0x7f32b40a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.781+0000 7f32b8e0f640 1 -- 172.21.15.204:0/1055853415 >> 172.21.15.204:0/1055853415 conn(0x7f32b406e860 msgr2=0x7f32b406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:01.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.781+0000 7f32b8e0f640 1 -- 172.21.15.204:0/1055853415 shutdown_connections 2024-09-15T01:34:01.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.781+0000 7f32b8e0f640 1 -- 172.21.15.204:0/1055853415 wait complete. 2024-09-15T01:34:01.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.781+0000 7f32b8e0f640 10 monclient: build_initial_monmap 2024-09-15T01:34:01.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 10 monclient: monmap: 2024-09-15T01:34:01.802 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:01.802 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:01.802 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:01.802 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:01.802 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:01.802 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:01.802 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:01.802 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:01.802 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 1 Processor -- start 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 1 -- start start 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 10 monclient: init 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 10 monclient: _add_conns ranks=[1,5,8,7,2,0,4,3,6] 2024-09-15T01:34:01.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b409f990 0x7f32b4141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:01.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 10 monclient(hunting): picked mon.noname-f con 0x7f32b409f990 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:01.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32b40a2400 0x7f32b41423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:01.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 10 monclient(hunting): picked mon.noname-h con 0x7f32b40a2400 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:01.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f32b4132930 0x7f32b4142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:01.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 10 monclient(hunting): picked mon.noname-e con 0x7f32b4132930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:01.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:01.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b409cb20 con 0x7f32b409f990 2024-09-15T01:34:01.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:01.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4120110 con 0x7f32b40a2400 2024-09-15T01:34:01.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:01.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4121350 con 0x7f32b4132930 2024-09-15T01:34:01.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b8e0f640 10 monclient(hunting): authenticate will time out at 857.425869s 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b409f990 0x7f32b4141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b1d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32b40a2400 0x7f32b41423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b409f990 0x7f32b4141ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38126/0 (socket says 172.21.15.204:38126) 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b259c640 1 -- 172.21.15.204:0/3312630695 learned_addr learned my addr 172.21.15.204:0/3312630695 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b259c640 10 monclient(hunting): get_auth_request con 0x7f32b409f990 auth_method 0 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.782+0000 7f32b259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b1d9b640 10 monclient(hunting): get_auth_request con 0x7f32b40a2400 auth_method 0 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:01.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:01.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b2d9d640 1 --2- 172.21.15.204:0/3312630695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f32b4132930 0x7f32b4142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:01.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:01.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:01.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:01.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:01.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:01.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:01.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b2d9d640 10 monclient(hunting): get_auth_request con 0x7f32b4132930 auth_method 0 2024-09-15T01:34:01.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:01.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:01.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:01.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:01.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:01.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:01.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 10 monclient(hunting): handle_auth_done global_id 5356 payload 824 2024-09-15T01:34:01.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 1 -- 172.21.15.204:0/3312630695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f32b4132930 msgr2=0x7f32b4142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:01.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 1 --2- 172.21.15.204:0/3312630695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f32b4132930 0x7f32b4142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 1 -- 172.21.15.204:0/3312630695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32b40a2400 msgr2=0x7f32b41423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:01.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 1 --2- 172.21.15.204:0/3312630695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32b40a2400 0x7f32b41423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:01.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:01.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 1 monclient: found mon.noname-f 2024-09-15T01:34:01.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:01.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:01.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 1 -- 172.21.15.204:0/3312630695 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4009450 con 0x7f32b409f990 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b1d9b640 1 --2- 172.21.15.204:0/3312630695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32b40a2400 0x7f32b41423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b259c640 1 --2- 172.21.15.204:0/3312630695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b409f990 0x7f32b4141ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f329c00ea20 tx=0x7f329c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32aa7fc640 1 -- 172.21.15.204:0/3312630695 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f329c00ce90 con 0x7f32b409f990 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32aa7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32aa7fc640 10 monclient: dump: 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:01.808 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:01.809 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32aa7fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32aa7fc640 1 -- 172.21.15.204:0/3312630695 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c009600 con 0x7f32b409f990 2024-09-15T01:34:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b8e0f640 5 monclient: authenticate success, global_id 5356 2024-09-15T01:34:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32aa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32aa7fc640 1 -- 172.21.15.204:0/3312630695 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f329c0138f0 con 0x7f32b409f990 2024-09-15T01:34:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b8e0f640 10 monclient: _renew_subs 2024-09-15T01:34:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b8e0f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b8e0f640 1 -- 172.21.15.204:0/3312630695 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b409d3d0 con 0x7f32b409f990 2024-09-15T01:34:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b0d99640 4 set_mon_vals no callback set 2024-09-15T01:34:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32aa7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32aa7fc640 10 monclient: dump: 2024-09-15T01:34:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:01.811 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:01.811 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:01.811 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:01.811 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:01.811 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:01.811 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:01.811 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:01.811 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:01.811 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:01.811 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:01.811 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b8e0f640 10 monclient: _renew_subs 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b8e0f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.783+0000 7f32b8e0f640 1 -- 172.21.15.204:0/3312630695 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b4121070 con 0x7f32b409f990 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.784+0000 7f32aa7fc640 1 -- 172.21.15.204:0/3312630695 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f329c013470 con 0x7f32b409f990 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.784+0000 7f32b8e0f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.784+0000 7f32b8e0f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.784+0000 7f32b8e0f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.784+0000 7f32b8e0f640 1 -- 172.21.15.204:0/3312630695 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b4070b80 con 0x7f32b409f990 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.784+0000 7f32aa7fc640 1 --2- 172.21.15.204:0/3312630695 >> v2:172.21.15.204:6812/28210 conn(0x7f32940396e0 0x7f329403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:01.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.784+0000 7f32aa7fc640 1 -- 172.21.15.204:0/3312630695 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4448+0+0 (secure 0 0 0) 0x7f329c04b600 con 0x7f32b409f990 2024-09-15T01:34:01.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.784+0000 7f32b1d9b640 1 --2- 172.21.15.204:0/3312630695 >> v2:172.21.15.204:6812/28210 conn(0x7f32940396e0 0x7f329403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:01.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.785+0000 7f32b1d9b640 10 monclient: get_auth_request con 0x7f32940396e0 auth_method 0 2024-09-15T01:34:01.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.785+0000 7f32b1d9b640 1 --2- 172.21.15.204:0/3312630695 >> v2:172.21.15.204:6812/28210 conn(0x7f32940396e0 0x7f329403bb90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f32a4034000 tx=0x7f32a4032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:01.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.789+0000 7f32aa7fc640 1 -- 172.21.15.204:0/3312630695 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f329c013720 con 0x7f32b409f990 2024-09-15T01:34:01.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.789+0000 7f32aa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:01.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.789+0000 7f32aa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:01.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.957+0000 7f32b8e0f640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T01:34:01.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.957+0000 7f32b8e0f640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-09-15T01:34:01.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.957+0000 7f32b8e0f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:01.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:01.957+0000 7f32b8e0f640 1 -- 172.21.15.204:0/3312630695 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7f32b4121590 con 0x7f32b409f990 2024-09-15T01:34:02.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:02.710+0000 7f32aa7fc640 1 -- 172.21.15.204:0/3312630695 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v71) v1 ==== 58+0+2 (secure 0 0 0) 0x7f329c01b3b0 con 0x7f32b409f990 2024-09-15T01:34:02.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:02.710+0000 7f32aa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-09-15T01:34:02.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:02.710+0000 7f32aa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:02.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:02.818+0000 7f32b8e0f640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T01:34:02.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:02.818+0000 7f32b8e0f640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-09-15T01:34:02.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:02.818+0000 7f32b8e0f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:02.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:02.818+0000 7f32b8e0f640 1 -- 172.21.15.204:0/3312630695 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7f32b409e480 con 0x7f32b409f990 2024-09-15T01:34:03.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.713+0000 7f32aa7fc640 1 -- 172.21.15.204:0/3312630695 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v72) v1 ==== 58+0+2 (secure 0 0 0) 0x7f329c0192c0 con 0x7f32b409f990 2024-09-15T01:34:03.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.713+0000 7f32aa7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-09-15T01:34:03.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.713+0000 7f32aa7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:03.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.716+0000 7f32b8e0f640 1 -- 172.21.15.204:0/3312630695 >> v2:172.21.15.204:6812/28210 conn(0x7f32940396e0 msgr2=0x7f329403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:03.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.716+0000 7f32b8e0f640 1 --2- 172.21.15.204:0/3312630695 >> v2:172.21.15.204:6812/28210 conn(0x7f32940396e0 0x7f329403bb90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f32a4034000 tx=0x7f32a4032000 comp rx=0 tx=0).stop 2024-09-15T01:34:03.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.716+0000 7f32b8e0f640 10 monclient: shutdown 2024-09-15T01:34:03.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.716+0000 7f32b8e0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:03.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.716+0000 7f32b8e0f640 1 -- 172.21.15.204:0/3312630695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b409f990 msgr2=0x7f32b4141ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:03.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.716+0000 7f32b8e0f640 1 --2- 172.21.15.204:0/3312630695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b409f990 0x7f32b4141ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f329c00ea20 tx=0x7f329c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:03.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.716+0000 7f32b8e0f640 1 -- 172.21.15.204:0/3312630695 shutdown_connections 2024-09-15T01:34:03.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.716+0000 7f32b8e0f640 1 --2- 172.21.15.204:0/3312630695 >> v2:172.21.15.204:6812/28210 conn(0x7f32940396e0 0x7f329403bb90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:03.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.716+0000 7f32b8e0f640 1 --2- 172.21.15.204:0/3312630695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f32b4132930 0x7f32b4142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:03.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.716+0000 7f32b8e0f640 1 --2- 172.21.15.204:0/3312630695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32b40a2400 0x7f32b41423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:03.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.716+0000 7f32b8e0f640 1 --2- 172.21.15.204:0/3312630695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b409f990 0x7f32b4141ea0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:03.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.717+0000 7f32b8e0f640 1 -- 172.21.15.204:0/3312630695 >> 172.21.15.204:0/3312630695 conn(0x7f32b406e860 msgr2=0x7f32b4131310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:03.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.717+0000 7f32b8e0f640 1 -- 172.21.15.204:0/3312630695 shutdown_connections 2024-09-15T01:34:03.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.717+0000 7f32b8e0f640 1 -- 172.21.15.204:0/3312630695 wait complete. 2024-09-15T01:34:03.730 INFO:tasks.workunit.client.0.smithi204.stderr:+ o2=9 2024-09-15T01:34:03.730 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add 7 1 host=host1 root=foo 2024-09-15T01:34:03.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.807+0000 7fc8d00af640 10 monclient: get_monmap_and_config 2024-09-15T01:34:03.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.807+0000 7fc8d00af640 10 monclient: build_initial_monmap 2024-09-15T01:34:03.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.807+0000 7fc8d00af640 10 monclient: monmap: 2024-09-15T01:34:03.810 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:03.810 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:03.810 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:03.808734+0000 2024-09-15T01:34:03.810 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:03.808734+0000 2024-09-15T01:34:03.810 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:03.810 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:03.810 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:03.810 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:03.810 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:03.811 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:03.811 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:03.811 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:03.811 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:03.811 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:03.811 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:03.811 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:03.811 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:03.811 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:03.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.807+0000 7fc8d00af640 1 Processor -- start 2024-09-15T01:34:03.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 1 -- start start 2024-09-15T01:34:03.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 10 monclient: init 2024-09-15T01:34:03.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:03.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 10 monclient: _add_conns ranks=[7,3,1,8,2,6,4,9,0,10,5] 2024-09-15T01:34:03.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8c80a2cd0 0x7fc8c8132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:03.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 10 monclient(hunting): picked mon.noname-i con 0x7fc8c80a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:03.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c809fa20 0x7fc8c809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:03.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 10 monclient(hunting): picked mon.noname-g con 0x7fc8c809fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:03.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc8c80a08e0 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:03.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c80648a0 con 0x7fc8c80a08e0 2024-09-15T01:34:03.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:03.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c80a30a0 con 0x7fc8c809fa20 2024-09-15T01:34:03.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:03.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c80a0330 con 0x7fc8c80a2cd0 2024-09-15T01:34:03.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:03.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8d00af640 10 monclient(hunting): authenticate will time out at 859.451516s 2024-09-15T01:34:03.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cd64a640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c809fa20 msgr2=0x7fc8c809fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:34:03.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cd64a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c809fa20 0x7fc8c809fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:03.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8c80a2cd0 0x7fc8c8132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:03.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8ce64c640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc8c80a08e0 0x7fc8c80a0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:48796/0 (socket says 172.21.15.204:48796) 2024-09-15T01:34:03.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 10 monclient(hunting): get_auth_request con 0x7fc8c80a2cd0 auth_method 0 2024-09-15T01:34:03.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8ce64c640 1 -- 172.21.15.204:0/44366630 learned_addr learned my addr 172.21.15.204:0/44366630 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:34:03.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:03.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:03.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:03.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:03.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:03.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:03.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 10 monclient(hunting): handle_auth_done global_id 5002 payload 274 2024-09-15T01:34:03.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 1 -- 172.21.15.204:0/44366630 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c809fa20 msgr2=0x7fc8c809fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:03.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 1 --2- 172.21.15.204:0/44366630 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c809fa20 0x7fc8c809fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:03.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 1 -- 172.21.15.204:0/44366630 >> v1:172.21.15.204:6789/0 conn(0x7fc8c80a08e0 legacy=0x7fc8c80a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:03.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 10 monclient: _finish_hunting 0 2024-09-15T01:34:03.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 1 monclient: found mon.noname-i 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cde4b640 1 -- 172.21.15.204:0/44366630 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c80a1db0 con 0x7fc8c80a2cd0 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cce49640 1 -- 172.21.15.204:0/44366630 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48074827 0 0) 0x7fc8c80648a0 con 0x7fc8c80a08e0 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.808+0000 7fc8cce49640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cde4b640 1 --2- 172.21.15.204:0/44366630 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8c80a2cd0 0x7fc8c8132c00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009880 tx=0x7fc8c40090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=159b6f57c8e88f41 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cce49640 1 -- 172.21.15.204:0/44366630 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8c402ac80 con 0x7fc8c80a2cd0 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cce49640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cce49640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cce49640 10 monclient: dump: 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:03.823 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:03.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cce49640 10 monclient: _finish_auth 0 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cce49640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cce49640 1 -- 172.21.15.204:0/44366630 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c402ae00 con 0x7fc8c80a2cd0 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8d00af640 5 monclient: authenticate success, global_id 5002 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8d00af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cce49640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8d00af640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8d00af640 4 set_mon_vals no callback set 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cce49640 1 -- 172.21.15.204:0/44366630 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8c4031ca0 con 0x7fc8c80a2cd0 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cce49640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cce49640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.809+0000 7fc8cce49640 10 monclient: dump: 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:03.825 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:03.826 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.810+0000 7fc8d00af640 10 monclient: shutdown 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.810+0000 7fc8d00af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.810+0000 7fc8d00af640 1 -- 172.21.15.204:0/44366630 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8c80a2cd0 msgr2=0x7fc8c8132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.810+0000 7fc8d00af640 1 --2- 172.21.15.204:0/44366630 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8c80a2cd0 0x7fc8c8132c00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009880 tx=0x7fc8c40090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.810+0000 7fc8d00af640 1 -- 172.21.15.204:0/44366630 shutdown_connections 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.810+0000 7fc8d00af640 1 --2- 172.21.15.204:0/44366630 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8c809fa20 0x7fc8c809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.810+0000 7fc8d00af640 1 --2- 172.21.15.204:0/44366630 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8c80a2cd0 0x7fc8c8132c00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.810+0000 7fc8d00af640 1 -- 172.21.15.204:0/44366630 >> 172.21.15.204:0/44366630 conn(0x7fc8c806e860 msgr2=0x7fc8c806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.810+0000 7fc8d00af640 1 -- 172.21.15.204:0/44366630 shutdown_connections 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.810+0000 7fc8d00af640 1 -- 172.21.15.204:0/44366630 wait complete. 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.810+0000 7fc8d00af640 10 monclient: build_initial_monmap 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.810+0000 7fc8d00af640 10 monclient: monmap: 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:03.827 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:03.828 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:03.828 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:03.828 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:03.828 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:03.828 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:03.828 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:03.828 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:03.828 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:03.828 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:03.828 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:03.828 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:03.828 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 1 Processor -- start 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 1 -- start start 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 10 monclient: init 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 10 monclient: _add_conns ranks=[1,0,7,2,6,4,5,3,8] 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8c809fa20 0x7fc8c8131960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 10 monclient(hunting): picked mon.noname-f con 0x7fc8c809fa20 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8c80a08e0 0x7fc8c811d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 10 monclient(hunting): picked mon.noname-a con 0x7fc8c80a08e0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8c80a2cd0 0x7fc8c811d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 10 monclient(hunting): picked mon.noname-i con 0x7fc8c80a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:03.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c8120aa0 con 0x7fc8c80a08e0 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c81210d0 con 0x7fc8c809fa20 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c811f560 con 0x7fc8c80a2cd0 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8d00af640 10 monclient(hunting): authenticate will time out at 859.454696s 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8cde4b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8c809fa20 0x7fc8c8131960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8cde4b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8c809fa20 0x7fc8c8131960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38140/0 (socket says 172.21.15.204:38140) 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8cde4b640 1 -- 172.21.15.204:0/477226483 learned_addr learned my addr 172.21.15.204:0/477226483 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8cde4b640 10 monclient(hunting): get_auth_request con 0x7fc8c809fa20 auth_method 0 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8ce64c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8c80a2cd0 0x7fc8c811d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8cde4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:03.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8cde4b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8cde4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8ce64c640 10 monclient(hunting): get_auth_request con 0x7fc8c80a2cd0 auth_method 0 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8ce64c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8ce64c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8ce64c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8cde4b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8cde4b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8cde4b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8ce64c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8ce64c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8ce64c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.811+0000 7fc8cd64a640 1 --2- 172.21.15.204:0/477226483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8c80a08e0 0x7fc8c811d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:03.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cd64a640 10 monclient(hunting): get_auth_request con 0x7fc8c80a08e0 auth_method 0 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cd64a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cd64a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cd64a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cde4b640 10 monclient(hunting): handle_auth_done global_id 5374 payload 824 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cde4b640 1 -- 172.21.15.204:0/477226483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8c80a2cd0 msgr2=0x7fc8c811d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cde4b640 1 --2- 172.21.15.204:0/477226483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8c80a2cd0 0x7fc8c811d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cde4b640 1 -- 172.21.15.204:0/477226483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8c80a08e0 msgr2=0x7fc8c811d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cde4b640 1 --2- 172.21.15.204:0/477226483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8c80a08e0 0x7fc8c811d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cde4b640 10 monclient: _finish_hunting 0 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cde4b640 1 monclient: found mon.noname-f 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cde4b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cde4b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:03.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cde4b640 1 -- 172.21.15.204:0/477226483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c4009390 con 0x7fc8c809fa20 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8ce64c640 1 --2- 172.21.15.204:0/477226483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8c80a2cd0 0x7fc8c811d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8cde4b640 1 --2- 172.21.15.204:0/477226483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8c809fa20 0x7fc8c8131960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009720 tx=0x7fc8c4032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bdffb640 1 -- 172.21.15.204:0/477226483 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8c4032e20 con 0x7fc8c809fa20 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bdffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bdffb640 10 monclient: dump: 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:03.833 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:03.834 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:03.834 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:03.834 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:03.834 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:03.834 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:03.834 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:03.834 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:03.834 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:03.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bdffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:03.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:03.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bdffb640 1 -- 172.21.15.204:0/477226483 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c403c440 con 0x7fc8c809fa20 2024-09-15T01:34:03.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8d00af640 5 monclient: authenticate success, global_id 5374 2024-09-15T01:34:03.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:03.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bdffb640 1 -- 172.21.15.204:0/477226483 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8c403ca30 con 0x7fc8c809fa20 2024-09-15T01:34:03.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8d00af640 10 monclient: _renew_subs 2024-09-15T01:34:03.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8d00af640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:03.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8d00af640 1 -- 172.21.15.204:0/477226483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c809d430 con 0x7fc8c809fa20 2024-09-15T01:34:03.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:03.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bdffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:03.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bdffb640 10 monclient: dump: 2024-09-15T01:34:03.835 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:03.835 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:03.835 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:03.835 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:03.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8bffff640 4 set_mon_vals no callback set 2024-09-15T01:34:03.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8d00af640 10 monclient: _renew_subs 2024-09-15T01:34:03.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8d00af640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:03.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.812+0000 7fc8d00af640 1 -- 172.21.15.204:0/477226483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c811efe0 con 0x7fc8c809fa20 2024-09-15T01:34:03.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.813+0000 7fc8bdffb640 1 -- 172.21.15.204:0/477226483 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc8c403cb90 con 0x7fc8c809fa20 2024-09-15T01:34:03.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.813+0000 7fc8bdffb640 1 --2- 172.21.15.204:0/477226483 >> v2:172.21.15.204:6812/28210 conn(0x7fc8ac0397b0 0x7fc8ac03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:03.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.813+0000 7fc8d00af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:03.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.813+0000 7fc8d00af640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:03.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.813+0000 7fc8d00af640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:03.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.813+0000 7fc8d00af640 1 -- 172.21.15.204:0/477226483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c8070b80 con 0x7fc8c809fa20 2024-09-15T01:34:03.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.813+0000 7fc8bdffb640 1 -- 172.21.15.204:0/477226483 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4678+0+0 (secure 0 0 0) 0x7fc8c406bb10 con 0x7fc8c809fa20 2024-09-15T01:34:03.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.813+0000 7fc8cd64a640 1 --2- 172.21.15.204:0/477226483 >> v2:172.21.15.204:6812/28210 conn(0x7fc8ac0397b0 0x7fc8ac03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:03.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.813+0000 7fc8cd64a640 10 monclient: get_auth_request con 0x7fc8ac0397b0 auth_method 0 2024-09-15T01:34:03.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.814+0000 7fc8cd64a640 1 --2- 172.21.15.204:0/477226483 >> v2:172.21.15.204:6812/28210 conn(0x7fc8ac0397b0 0x7fc8ac03bc60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8003850 tx=0x7fc8b8006ed0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:03.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.818+0000 7fc8bdffb640 1 -- 172.21.15.204:0/477226483 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc8c4037030 con 0x7fc8c809fa20 2024-09-15T01:34:03.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.818+0000 7fc8bdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:03.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.818+0000 7fc8bdffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:03.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.975+0000 7fc8d00af640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T01:34:03.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.975+0000 7fc8d00af640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T01:34:03.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.975+0000 7fc8d00af640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:03.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:03.975+0000 7fc8d00af640 1 -- 172.21.15.204:0/477226483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) v1 -- 0x7fc8c8133570 con 0x7fc8c809fa20 2024-09-15T01:34:04.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.731+0000 7fc8bdffb640 1 -- 172.21.15.204:0/477226483 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 add item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo} to crush map v73) v1 ==== 203+0+0 (secure 0 0 0) 0x7fc8c403ce40 con 0x7fc8c809fa20 2024-09-15T01:34:04.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.731+0000 7fc8bdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T01:34:04.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.731+0000 7fc8bdffb640 10 monclient: _finish_command 2 = system:0 add item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo} to crush map 2024-09-15T01:34:04.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.829+0000 7fc8d00af640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T01:34:04.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.829+0000 7fc8d00af640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T01:34:04.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.829+0000 7fc8d00af640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:04.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.829+0000 7fc8d00af640 1 -- 172.21.15.204:0/477226483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) v1 -- 0x7fc8c8133570 con 0x7fc8c809fa20 2024-09-15T01:34:04.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.830+0000 7fc8bdffb640 1 -- 172.21.15.204:0/477226483 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change v73) v1 ==== 201+0+0 (secure 0 0 0) 0x7fc8c403b3a0 con 0x7fc8c809fa20 2024-09-15T01:34:04.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.830+0000 7fc8bdffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T01:34:04.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.830+0000 7fc8bdffb640 10 monclient: _finish_command 3 = system:0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-09-15T01:34:04.832 INFO:tasks.workunit.client.0.smithi204.stderr:set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-09-15T01:34:04.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 1 -- 172.21.15.204:0/477226483 >> v2:172.21.15.204:6812/28210 conn(0x7fc8ac0397b0 msgr2=0x7fc8ac03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:04.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 1 --2- 172.21.15.204:0/477226483 >> v2:172.21.15.204:6812/28210 conn(0x7fc8ac0397b0 0x7fc8ac03bc60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8003850 tx=0x7fc8b8006ed0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 10 monclient: shutdown 2024-09-15T01:34:04.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:04.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 1 -- 172.21.15.204:0/477226483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8c809fa20 msgr2=0x7fc8c8131960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:04.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 1 --2- 172.21.15.204:0/477226483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8c809fa20 0x7fc8c8131960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009720 tx=0x7fc8c4032000 comp rx=0 tx=0).stop 2024-09-15T01:34:04.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 1 -- 172.21.15.204:0/477226483 shutdown_connections 2024-09-15T01:34:04.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 1 --2- 172.21.15.204:0/477226483 >> v2:172.21.15.204:6812/28210 conn(0x7fc8ac0397b0 0x7fc8ac03bc60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 1 --2- 172.21.15.204:0/477226483 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc8c80a2cd0 0x7fc8c811d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 1 --2- 172.21.15.204:0/477226483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8c80a08e0 0x7fc8c811d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 1 --2- 172.21.15.204:0/477226483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8c809fa20 0x7fc8c8131960 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 1 -- 172.21.15.204:0/477226483 >> 172.21.15.204:0/477226483 conn(0x7fc8c806e860 msgr2=0x7fc8c80a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:04.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 1 -- 172.21.15.204:0/477226483 shutdown_connections 2024-09-15T01:34:04.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.832+0000 7fc8d00af640 1 -- 172.21.15.204:0/477226483 wait complete. 2024-09-15T01:34:04.843 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add 7 1 host=host1 root=foo 2024-09-15T01:34:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.917+0000 7faea0d00640 10 monclient: get_monmap_and_config 2024-09-15T01:34:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.917+0000 7faea0d00640 10 monclient: build_initial_monmap 2024-09-15T01:34:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.917+0000 7faea0d00640 10 monclient: monmap: 2024-09-15T01:34:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:04.919037+0000 2024-09-15T01:34:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:04.919037+0000 2024-09-15T01:34:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:04.921 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 1 Processor -- start 2024-09-15T01:34:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 1 -- start start 2024-09-15T01:34:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 10 monclient: init 2024-09-15T01:34:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 10 monclient: _add_conns ranks=[10,2,3,9,6,7,4,0,1,8,5] 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fae9c0a2cd0 0x7fae9c132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 10 monclient(hunting): picked mon.noname-f con 0x7fae9c0a2cd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9c09fa20 0x7fae9c09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 10 monclient(hunting): picked mon.noname-b con 0x7fae9c09fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae9c0a08e0 0x7fae9c0a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 10 monclient(hunting): picked mon.noname-g con 0x7fae9c0a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c0a30a0 con 0x7fae9c0a2cd0 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0a0330 con 0x7fae9c09fa20 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0a1db0 con 0x7fae9c0a08e0 2024-09-15T01:34:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7faea0d00640 10 monclient(hunting): authenticate will time out at 860.561892s 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7fae9ad9d640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae9c0a08e0 msgr2=0x7fae9c0a0d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.918+0000 7fae9ad9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae9c0a08e0 0x7fae9c0a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fae9c0a2cd0 0x7fae9c132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fae9c0a2cd0 0x7fae9c132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38150/0 (socket says 172.21.15.204:38150) 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 1 -- 172.21.15.204:0/3481356051 learned_addr learned my addr 172.21.15.204:0/3481356051 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 10 monclient(hunting): get_auth_request con 0x7fae9c0a2cd0 auth_method 0 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae93fff640 1 --2- 172.21.15.204:0/3481356051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9c09fa20 0x7fae9c09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae93fff640 10 monclient(hunting): get_auth_request con 0x7fae9c09fa20 auth_method 0 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae93fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae93fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 10 monclient(hunting): handle_auth_done global_id 5383 payload 274 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 1 -- 172.21.15.204:0/3481356051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae9c0a08e0 msgr2=0x7fae9c0a0d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 1 --2- 172.21.15.204:0/3481356051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae9c0a08e0 0x7fae9c0a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 1 -- 172.21.15.204:0/3481356051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9c09fa20 msgr2=0x7fae9c09fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 1 --2- 172.21.15.204:0/3481356051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9c09fa20 0x7fae9c09fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 1 monclient: found mon.noname-f 2024-09-15T01:34:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 1 -- 172.21.15.204:0/3481356051 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c0648a0 con 0x7fae9c0a2cd0 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.919+0000 7fae9a59c640 1 --2- 172.21.15.204:0/3481356051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9c0a2cd0 0x7fae9c132c00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009910 tx=0x7fae8c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8298c7786e3e6230 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7fae99d9b640 1 -- 172.21.15.204:0/3481356051 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fae8c02acb0 con 0x7fae9c0a2cd0 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7fae99d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7fae99d9b640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7fae99d9b640 10 monclient: dump: 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7fae99d9b640 10 monclient: _finish_auth 0 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7fae99d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 5 monclient: authenticate success, global_id 5383 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7fae99d9b640 1 -- 172.21.15.204:0/3481356051 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c02ae30 con 0x7fae9c0a2cd0 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7fae99d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 4 set_mon_vals no callback set 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7fae99d9b640 1 -- 172.21.15.204:0/3481356051 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fae8c031ca0 con 0x7fae9c0a2cd0 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7fae99d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7fae99d9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7fae99d9b640 10 monclient: dump: 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 10 monclient: shutdown 2024-09-15T01:34:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 1 -- 172.21.15.204:0/3481356051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9c0a2cd0 msgr2=0x7fae9c132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 1 --2- 172.21.15.204:0/3481356051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9c0a2cd0 0x7fae9c132c00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009910 tx=0x7fae8c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 1 -- 172.21.15.204:0/3481356051 shutdown_connections 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 1 --2- 172.21.15.204:0/3481356051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae9c0a08e0 0x7fae9c0a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 1 --2- 172.21.15.204:0/3481356051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9c09fa20 0x7fae9c09fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 1 --2- 172.21.15.204:0/3481356051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9c0a2cd0 0x7fae9c132c00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 1 -- 172.21.15.204:0/3481356051 >> 172.21.15.204:0/3481356051 conn(0x7fae9c06e860 msgr2=0x7fae9c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 1 -- 172.21.15.204:0/3481356051 shutdown_connections 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.920+0000 7faea0d00640 1 -- 172.21.15.204:0/3481356051 wait complete. 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient: build_initial_monmap 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient: monmap: 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:04.939 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:04.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 1 Processor -- start 2024-09-15T01:34:04.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 1 -- start start 2024-09-15T01:34:04.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient: init 2024-09-15T01:34:04.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:04.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient: _add_conns ranks=[7,1,5,2,0,8,6,4,3] 2024-09-15T01:34:04.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fae9c09fa20 0x7fae9c131050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:04.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient(hunting): picked mon.noname-i con 0x7fae9c09fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:04.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9c0a08e0 0x7fae9c09b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:04.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient(hunting): picked mon.noname-f con 0x7fae9c0a08e0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:04.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fae9c0a2cd0 0x7fae9c09b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:04.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient(hunting): picked mon.noname-h con 0x7fae9c0a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:04.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:04.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c11e270 con 0x7fae9c0a08e0 2024-09-15T01:34:04.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:04.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c11e7a0 con 0x7fae9c0a2cd0 2024-09-15T01:34:04.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:04.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c09d3d0 con 0x7fae9c09fa20 2024-09-15T01:34:04.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:04.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.921+0000 7faea0d00640 10 monclient(hunting): authenticate will time out at 860.564992s 2024-09-15T01:34:04.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9a59c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fae9c09fa20 0x7fae9c131050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:04.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9c0a08e0 0x7fae9c09b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:04.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9c0a08e0 0x7fae9c09b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38152/0 (socket says 172.21.15.204:38152) 2024-09-15T01:34:04.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9a59c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fae9c09fa20 0x7fae9c131050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:35102/0 (socket says 172.21.15.204:35102) 2024-09-15T01:34:04.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 1 -- 172.21.15.204:0/2223569664 learned_addr learned my addr 172.21.15.204:0/2223569664 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 10 monclient(hunting): get_auth_request con 0x7fae9c0a08e0 auth_method 0 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9a59c640 10 monclient(hunting): get_auth_request con 0x7fae9c09fa20 auth_method 0 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9ad9d640 1 --2- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fae9c0a2cd0 0x7fae9c09b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9ad9d640 10 monclient(hunting): get_auth_request con 0x7fae9c0a2cd0 auth_method 0 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:04.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 10 monclient(hunting): handle_auth_done global_id 5392 payload 824 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 1 -- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fae9c09fa20 msgr2=0x7fae9c131050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 1 --2- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fae9c09fa20 0x7fae9c131050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 1 -- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fae9c0a2cd0 msgr2=0x7fae9c09b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:04.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 1 --2- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fae9c0a2cd0 0x7fae9c09b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:04.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:04.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 1 monclient: found mon.noname-f 2024-09-15T01:34:04.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:04.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:04.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 1 -- 172.21.15.204:0/2223569664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c009450 con 0x7fae9c0a08e0 2024-09-15T01:34:04.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9a59c640 1 --2- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fae9c09fa20 0x7fae9c131050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:04.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9ad9d640 1 --2- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fae9c0a2cd0 0x7fae9c09b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:04.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae93fff640 1 --2- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9c0a08e0 0x7fae9c09b180 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fae8400d8c0 tx=0x7fae84008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:04.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae927fc640 1 -- 172.21.15.204:0/2223569664 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fae8400de50 con 0x7fae9c0a08e0 2024-09-15T01:34:04.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:04.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae927fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:04.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae927fc640 10 monclient: dump: 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:04.945 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae927fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae927fc640 1 -- 172.21.15.204:0/2223569664 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8400bd50 con 0x7fae9c0a08e0 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7faea0d00640 5 monclient: authenticate success, global_id 5392 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7faea0d00640 10 monclient: _renew_subs 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7faea0d00640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7faea0d00640 1 -- 172.21.15.204:0/2223569664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9c09cb60 con 0x7fae9c0a08e0 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae927fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae927fc640 1 -- 172.21.15.204:0/2223569664 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fae840148b0 con 0x7fae9c0a08e0 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:04.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae927fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae927fc640 10 monclient: dump: 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:04.947 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:04.948 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:04.948 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:04.948 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:04.948 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:04.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7faea0d00640 10 monclient: _renew_subs 2024-09-15T01:34:04.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7faea0d00640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:04.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7fae9959a640 4 set_mon_vals no callback set 2024-09-15T01:34:04.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.922+0000 7faea0d00640 1 -- 172.21.15.204:0/2223569664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c11c6b0 con 0x7fae9c0a08e0 2024-09-15T01:34:04.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.923+0000 7fae927fc640 1 -- 172.21.15.204:0/2223569664 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fae84014430 con 0x7fae9c0a08e0 2024-09-15T01:34:04.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.923+0000 7faea0d00640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:04.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.923+0000 7faea0d00640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:04.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.923+0000 7faea0d00640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:04.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.923+0000 7faea0d00640 1 -- 172.21.15.204:0/2223569664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c070b80 con 0x7fae9c0a08e0 2024-09-15T01:34:04.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.923+0000 7fae927fc640 1 --2- 172.21.15.204:0/2223569664 >> v2:172.21.15.204:6812/28210 conn(0x7fae7c0397e0 0x7fae7c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:04.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.923+0000 7fae927fc640 1 -- 172.21.15.204:0/2223569664 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4960+0+0 (secure 0 0 0) 0x7fae8404bb60 con 0x7fae9c0a08e0 2024-09-15T01:34:04.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.923+0000 7fae9a59c640 1 --2- 172.21.15.204:0/2223569664 >> v2:172.21.15.204:6812/28210 conn(0x7fae7c0397e0 0x7fae7c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:04.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.924+0000 7fae9a59c640 10 monclient: get_auth_request con 0x7fae7c0397e0 auth_method 0 2024-09-15T01:34:04.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.924+0000 7fae9a59c640 1 --2- 172.21.15.204:0/2223569664 >> v2:172.21.15.204:6812/28210 conn(0x7fae7c0397e0 0x7fae7c03bc90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fae8c034000 tx=0x7fae8c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:04.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.928+0000 7fae927fc640 1 -- 172.21.15.204:0/2223569664 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fae84014a30 con 0x7fae9c0a08e0 2024-09-15T01:34:04.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.928+0000 7fae927fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:04.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:04.928+0000 7fae927fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:05.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.086+0000 7faea0d00640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T01:34:05.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.086+0000 7faea0d00640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T01:34:05.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.086+0000 7faea0d00640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:05.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.086+0000 7faea0d00640 1 -- 172.21.15.204:0/2223569664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) v1 -- 0x7fae9c142a30 con 0x7fae9c0a08e0 2024-09-15T01:34:05.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.087+0000 7fae927fc640 1 -- 172.21.15.204:0/2223569664 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change v73) v1 ==== 201+0+0 (secure 0 0 0) 0x7fae84011340 con 0x7fae9c0a08e0 2024-09-15T01:34:05.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.087+0000 7fae927fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T01:34:05.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.087+0000 7fae927fc640 10 monclient: _finish_command 2 = system:0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-09-15T01:34:05.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.183+0000 7faea0d00640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T01:34:05.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.183+0000 7faea0d00640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T01:34:05.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.183+0000 7faea0d00640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:05.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.183+0000 7faea0d00640 1 -- 172.21.15.204:0/2223569664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) v1 -- 0x7fae9c0008d0 con 0x7fae9c0a08e0 2024-09-15T01:34:05.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.184+0000 7fae927fc640 1 -- 172.21.15.204:0/2223569664 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change v73) v1 ==== 201+0+0 (secure 0 0 0) 0x7fae84011520 con 0x7fae9c0a08e0 2024-09-15T01:34:05.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.184+0000 7fae927fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-09-15T01:34:05.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.184+0000 7fae927fc640 10 monclient: _finish_command 3 = system:0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-09-15T01:34:05.186 INFO:tasks.workunit.client.0.smithi204.stderr:set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-09-15T01:34:05.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.185+0000 7faea0d00640 1 -- 172.21.15.204:0/2223569664 >> v2:172.21.15.204:6812/28210 conn(0x7fae7c0397e0 msgr2=0x7fae7c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:05.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.185+0000 7faea0d00640 1 --2- 172.21.15.204:0/2223569664 >> v2:172.21.15.204:6812/28210 conn(0x7fae7c0397e0 0x7fae7c03bc90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fae8c034000 tx=0x7fae8c032000 comp rx=0 tx=0).stop 2024-09-15T01:34:05.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.185+0000 7faea0d00640 10 monclient: shutdown 2024-09-15T01:34:05.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.185+0000 7faea0d00640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:05.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.185+0000 7faea0d00640 1 -- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9c0a08e0 msgr2=0x7fae9c09b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:05.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.185+0000 7faea0d00640 1 --2- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9c0a08e0 0x7fae9c09b180 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fae8400d8c0 tx=0x7fae84008040 comp rx=0 tx=0).stop 2024-09-15T01:34:05.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.185+0000 7faea0d00640 1 -- 172.21.15.204:0/2223569664 shutdown_connections 2024-09-15T01:34:05.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.185+0000 7faea0d00640 1 --2- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fae9c0a2cd0 0x7fae9c09b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.185+0000 7faea0d00640 1 --2- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fae9c0a08e0 0x7fae9c09b180 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.185+0000 7faea0d00640 1 --2- 172.21.15.204:0/2223569664 >> v2:172.21.15.204:6812/28210 conn(0x7fae7c0397e0 0x7fae7c03bc90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.185+0000 7faea0d00640 1 --2- 172.21.15.204:0/2223569664 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fae9c09fa20 0x7fae9c131050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.185+0000 7faea0d00640 1 -- 172.21.15.204:0/2223569664 >> 172.21.15.204:0/2223569664 conn(0x7fae9c06e860 msgr2=0x7fae9c0a5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:05.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.186+0000 7faea0d00640 1 -- 172.21.15.204:0/2223569664 shutdown_connections 2024-09-15T01:34:05.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.186+0000 7faea0d00640 1 -- 172.21.15.204:0/2223569664 wait complete. 2024-09-15T01:34:05.197 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add 9 1 host=host2 root=foo 2024-09-15T01:34:05.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.270+0000 7f862f99d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.270+0000 7f862f99d640 10 monclient: build_initial_monmap 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.270+0000 7f862f99d640 10 monclient: monmap: 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:05.271568+0000 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:05.271568+0000 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:05.273 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:05.274 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:05.274 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:05.274 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:05.274 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:05.274 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:05.274 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:05.274 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:05.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.270+0000 7f862f99d640 1 Processor -- start 2024-09-15T01:34:05.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.270+0000 7f862f99d640 1 -- start start 2024-09-15T01:34:05.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.270+0000 7f862f99d640 10 monclient: init 2024-09-15T01:34:05.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:05.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 10 monclient: _add_conns ranks=[6,8,4,1,9,7,0,5,3,2,10] 2024-09-15T01:34:05.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f862809f1c0 0x7f862809f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 10 monclient(hunting): picked mon.noname-d con 0x7f862809f1c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:05.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f862809e750 0x7f8628130690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 10 monclient(hunting): picked mon.noname-e con 0x7f862809e750 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:05.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f862809eb20 0x7f8628130bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 10 monclient(hunting): picked mon.noname-c con 0x7f862809eb20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f862806cd10 con 0x7f862809eb20 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f862806de80 con 0x7f862809f1c0 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f862806dba0 con 0x7f862809e750 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862f99d640 10 monclient(hunting): authenticate will time out at 860.914433s 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862d739640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f862809f1c0 msgr2=0x7f862809f590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862d739640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f862809f1c0 0x7f862809f590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862cf38640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f862809e750 0x7f8628130690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862df3a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f862809eb20 0x7f8628130bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:05.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862cf38640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f862809e750 0x7f8628130690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:46182/0 (socket says 172.21.15.204:46182) 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862cf38640 1 -- 172.21.15.204:0/170118041 learned_addr learned my addr 172.21.15.204:0/170118041 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862cf38640 10 monclient(hunting): get_auth_request con 0x7f862809e750 auth_method 0 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862cf38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862cf38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862cf38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862df3a640 10 monclient(hunting): get_auth_request con 0x7f862809eb20 auth_method 0 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862df3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862df3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862df3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862cf38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862cf38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862cf38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:05.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862df3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862df3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.271+0000 7f862df3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f862cf38640 10 monclient(hunting): handle_auth_done global_id 5012 payload 274 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f862cf38640 1 -- 172.21.15.204:0/170118041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f862809f1c0 msgr2=0x7f862809f590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f862cf38640 1 --2- 172.21.15.204:0/170118041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f862809f1c0 0x7f862809f590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f862cf38640 1 -- 172.21.15.204:0/170118041 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f862809eb20 msgr2=0x7f8628130bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f862cf38640 1 --2- 172.21.15.204:0/170118041 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f862809eb20 0x7f8628130bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f862cf38640 10 monclient: _finish_hunting 0 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f862cf38640 1 monclient: found mon.noname-e 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f862cf38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f862cf38640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f862cf38640 1 -- 172.21.15.204:0/170118041 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86280648a0 con 0x7f862809e750 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f862df3a640 1 --2- 172.21.15.204:0/170118041 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f862809eb20 0x7f8628130bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:05.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f862cf38640 1 --2- 172.21.15.204:0/170118041 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f862809e750 0x7f8628130690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f861c009970 tx=0x7f861c0091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1261aaa6ca8f1dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.272+0000 7f8627fff640 1 -- 172.21.15.204:0/170118041 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f861c02ad00 con 0x7f862809e750 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f8627fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f8627fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f8627fff640 10 monclient: dump: 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:05.286 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f8627fff640 10 monclient: _finish_auth 0 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f8627fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f8627fff640 1 -- 172.21.15.204:0/170118041 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c02ae80 con 0x7f862809e750 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 5 monclient: authenticate success, global_id 5012 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:05.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f8627fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f8627fff640 1 -- 172.21.15.204:0/170118041 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f861c031ca0 con 0x7f862809e750 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f8627fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 4 set_mon_vals no callback set 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f8627fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f8627fff640 10 monclient: dump: 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:05.288 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 10 monclient: shutdown 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 1 -- 172.21.15.204:0/170118041 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f862809e750 msgr2=0x7f8628130690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 1 --2- 172.21.15.204:0/170118041 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f862809e750 0x7f8628130690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f861c009970 tx=0x7f861c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 1 -- 172.21.15.204:0/170118041 shutdown_connections 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 1 --2- 172.21.15.204:0/170118041 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f862809eb20 0x7f8628130bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 1 --2- 172.21.15.204:0/170118041 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f862809e750 0x7f8628130690 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 1 --2- 172.21.15.204:0/170118041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f862809f1c0 0x7f862809f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 1 -- 172.21.15.204:0/170118041 >> 172.21.15.204:0/170118041 conn(0x7f8628067c00 msgr2=0x7f86280a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 1 -- 172.21.15.204:0/170118041 shutdown_connections 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.273+0000 7f862f99d640 1 -- 172.21.15.204:0/170118041 wait complete. 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient: build_initial_monmap 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient: monmap: 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:05.290 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 1 Processor -- start 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 1 -- start start 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient: init 2024-09-15T01:34:05.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient: _add_conns ranks=[1,7,3,8,6,0,2,4,5] 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f862809e750 0x7f8628141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient(hunting): picked mon.noname-f con 0x7f862809e750 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f862809eb20 0x7f8628130830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient(hunting): picked mon.noname-i con 0x7f862809eb20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f862809f1c0 0x7f8628068240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient(hunting): picked mon.noname-g con 0x7f862809f1c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862811f970 con 0x7f862809e750 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862809ac50 con 0x7f862809f1c0 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:05.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8628069fa0 con 0x7f862809eb20 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.274+0000 7f862f99d640 10 monclient(hunting): authenticate will time out at 860.917991s 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862df3a640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f862809f1c0 msgr2=0x7f8628068240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862df3a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f862809f1c0 0x7f8628068240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862cf38640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f862809eb20 0x7f8628130830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862cf38640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f862809eb20 0x7f8628130830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:35110/0 (socket says 172.21.15.204:35110) 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f862809e750 0x7f8628141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862cf38640 1 -- 172.21.15.204:0/2916548412 learned_addr learned my addr 172.21.15.204:0/2916548412 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862cf38640 10 monclient(hunting): get_auth_request con 0x7f862809eb20 auth_method 0 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862cf38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862cf38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862cf38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:05.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 10 monclient(hunting): get_auth_request con 0x7f862809e750 auth_method 0 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862cf38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862cf38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862cf38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 10 monclient(hunting): handle_auth_done global_id 5401 payload 824 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 1 -- 172.21.15.204:0/2916548412 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f862809eb20 msgr2=0x7f8628130830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 1 --2- 172.21.15.204:0/2916548412 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f862809eb20 0x7f8628130830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 1 -- 172.21.15.204:0/2916548412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f862809f1c0 msgr2=0x7f8628068240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 1 --2- 172.21.15.204:0/2916548412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f862809f1c0 0x7f8628068240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 10 monclient: _finish_hunting 0 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 1 monclient: found mon.noname-f 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 1 -- 172.21.15.204:0/2916548412 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8618009a20 con 0x7f862809e750 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862cf38640 1 --2- 172.21.15.204:0/2916548412 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f862809eb20 0x7f8628130830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862d739640 1 --2- 172.21.15.204:0/2916548412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f862809e750 0x7f8628141ce0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f862000b5e0 tx=0x7f86200090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86257fa640 1 -- 172.21.15.204:0/2916548412 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8620006150 con 0x7f862809e750 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86257fa640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86257fa640 10 monclient: dump: 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:05.295 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:05.296 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86257fa640 10 monclient: _finish_auth 0 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86257fa640 1 -- 172.21.15.204:0/2916548412 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86200062f0 con 0x7f862809e750 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862f99d640 5 monclient: authenticate success, global_id 5401 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862f99d640 10 monclient: _renew_subs 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862f99d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862f99d640 1 -- 172.21.15.204:0/2916548412 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861c009450 con 0x7f862809e750 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86257fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86257fa640 1 -- 172.21.15.204:0/2916548412 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f86200103e0 con 0x7f862809e750 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86257fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86257fa640 10 monclient: dump: 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:05.297 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:05.298 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f86277fe640 4 set_mon_vals no callback set 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862f99d640 10 monclient: _renew_subs 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862f99d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.275+0000 7f862f99d640 1 -- 172.21.15.204:0/2916548412 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862811ea10 con 0x7f862809e750 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.276+0000 7f86257fa640 1 -- 172.21.15.204:0/2916548412 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8620003b80 con 0x7f862809e750 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.276+0000 7f86257fa640 1 --2- 172.21.15.204:0/2916548412 >> v2:172.21.15.204:6812/28210 conn(0x7f86140396e0 0x7f861403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.276+0000 7f86257fa640 1 -- 172.21.15.204:0/2916548412 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4960+0+0 (secure 0 0 0) 0x7f862004a6e0 con 0x7f862809e750 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.276+0000 7f862cf38640 1 --2- 172.21.15.204:0/2916548412 >> v2:172.21.15.204:6812/28210 conn(0x7f86140396e0 0x7f861403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.276+0000 7f862cf38640 10 monclient: get_auth_request con 0x7f86140396e0 auth_method 0 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.276+0000 7f862f99d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.276+0000 7f862f99d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.276+0000 7f862f99d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.276+0000 7f862f99d640 1 -- 172.21.15.204:0/2916548412 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8628070b80 con 0x7f862809e750 2024-09-15T01:34:05.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.277+0000 7f862cf38640 1 --2- 172.21.15.204:0/2916548412 >> v2:172.21.15.204:6812/28210 conn(0x7f86140396e0 0x7f861403bb90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8628064e60 tx=0x7f861c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:05.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.281+0000 7f86257fa640 1 -- 172.21.15.204:0/2916548412 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8620017080 con 0x7f862809e750 2024-09-15T01:34:05.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.281+0000 7f86257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:05.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.281+0000 7f86257fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:05.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.438+0000 7f862f99d640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T01:34:05.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.438+0000 7f862f99d640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T01:34:05.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.438+0000 7f862f99d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:05.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.438+0000 7f862f99d640 1 -- 172.21.15.204:0/2916548412 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) v1 -- 0x7f862806d8c0 con 0x7f862809e750 2024-09-15T01:34:05.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.729+0000 7f86257fa640 1 -- 172.21.15.204:0/2916548412 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 add item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo} to crush map v74) v1 ==== 203+0+0 (secure 0 0 0) 0x7f862000daf0 con 0x7f862809e750 2024-09-15T01:34:05.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.729+0000 7f86257fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T01:34:05.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.729+0000 7f86257fa640 10 monclient: _finish_command 2 = system:0 add item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo} to crush map 2024-09-15T01:34:05.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.827+0000 7f862f99d640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T01:34:05.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.827+0000 7f862f99d640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T01:34:05.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.827+0000 7f862f99d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:05.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.827+0000 7f862f99d640 1 -- 172.21.15.204:0/2916548412 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) v1 -- 0x7f862809ae80 con 0x7f862809e750 2024-09-15T01:34:05.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.827+0000 7f86257fa640 1 -- 172.21.15.204:0/2916548412 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change v74) v1 ==== 201+0+0 (secure 0 0 0) 0x7f862000d310 con 0x7f862809e750 2024-09-15T01:34:05.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.827+0000 7f86257fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T01:34:05.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.827+0000 7f86257fa640 10 monclient: _finish_command 3 = system:0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-09-15T01:34:05.830 INFO:tasks.workunit.client.0.smithi204.stderr:set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-09-15T01:34:05.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.829+0000 7f862f99d640 1 -- 172.21.15.204:0/2916548412 >> v2:172.21.15.204:6812/28210 conn(0x7f86140396e0 msgr2=0x7f861403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:05.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.829+0000 7f862f99d640 1 --2- 172.21.15.204:0/2916548412 >> v2:172.21.15.204:6812/28210 conn(0x7f86140396e0 0x7f861403bb90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8628064e60 tx=0x7f861c032000 comp rx=0 tx=0).stop 2024-09-15T01:34:05.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.829+0000 7f862f99d640 10 monclient: shutdown 2024-09-15T01:34:05.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.829+0000 7f862f99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:05.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.829+0000 7f862f99d640 1 -- 172.21.15.204:0/2916548412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f862809e750 msgr2=0x7f8628141ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:05.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.829+0000 7f862f99d640 1 --2- 172.21.15.204:0/2916548412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f862809e750 0x7f8628141ce0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f862000b5e0 tx=0x7f86200090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.830+0000 7f862f99d640 1 -- 172.21.15.204:0/2916548412 shutdown_connections 2024-09-15T01:34:05.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.830+0000 7f862f99d640 1 --2- 172.21.15.204:0/2916548412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f862809f1c0 0x7f8628068240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.830+0000 7f862f99d640 1 --2- 172.21.15.204:0/2916548412 >> v2:172.21.15.204:6812/28210 conn(0x7f86140396e0 0x7f861403bb90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.830+0000 7f862f99d640 1 --2- 172.21.15.204:0/2916548412 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f862809eb20 0x7f8628130830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.830+0000 7f862f99d640 1 --2- 172.21.15.204:0/2916548412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f862809e750 0x7f8628141ce0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.830+0000 7f862f99d640 1 -- 172.21.15.204:0/2916548412 >> 172.21.15.204:0/2916548412 conn(0x7f8628067c00 msgr2=0x7f862809c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:05.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.830+0000 7f862f99d640 1 -- 172.21.15.204:0/2916548412 shutdown_connections 2024-09-15T01:34:05.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.830+0000 7f862f99d640 1 -- 172.21.15.204:0/2916548412 wait complete. 2024-09-15T01:34:05.841 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add 9 1 host=host2 root=foo 2024-09-15T01:34:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.914+0000 7f09d4f09640 10 monclient: get_monmap_and_config 2024-09-15T01:34:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.914+0000 7f09d4f09640 10 monclient: build_initial_monmap 2024-09-15T01:34:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.915+0000 7f09d4f09640 10 monclient: monmap: 2024-09-15T01:34:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:05.916477+0000 2024-09-15T01:34:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:05.916477+0000 2024-09-15T01:34:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:05.919 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:05.919 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:05.919 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:05.919 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:05.919 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:05.919 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:05.919 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:05.919 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:05.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.915+0000 7f09d4f09640 1 Processor -- start 2024-09-15T01:34:05.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.915+0000 7f09d4f09640 1 -- start start 2024-09-15T01:34:05.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.915+0000 7f09d4f09640 10 monclient: init 2024-09-15T01:34:05.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:05.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 10 monclient: _add_conns ranks=[5,10,3,4,7,1,8,0,2,9,6] 2024-09-15T01:34:05.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09d00a2cd0 0x7f09d0132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 10 monclient(hunting): picked mon.noname-h con 0x7f09d00a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:05.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f09d009fa20 0x7f09d009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 10 monclient(hunting): picked mon.noname-f con 0x7f09d009fa20 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:05.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09d00a08e0 0x7f09d00a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 10 monclient(hunting): picked mon.noname-g con 0x7f09d00a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:05.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:05.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d00a30a0 con 0x7f09d009fa20 2024-09-15T01:34:05.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:05.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00a0330 con 0x7f09d00a08e0 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00a1db0 con 0x7f09d00a2cd0 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09d4f09640 10 monclient(hunting): authenticate will time out at 861.559375s 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ced9d640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09d00a08e0 msgr2=0x7f09d00a0d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ced9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09d00a08e0 0x7f09d00a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ce59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09d00a2cd0 0x7f09d0132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f09d009fa20 0x7f09d009fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ce59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09d00a2cd0 0x7f09d0132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:51698/0 (socket says 172.21.15.204:51698) 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ce59c640 1 -- 172.21.15.204:0/3824256313 learned_addr learned my addr 172.21.15.204:0/3824256313 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 10 monclient(hunting): get_auth_request con 0x7f09d009fa20 auth_method 0 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:05.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:05.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:05.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ce59c640 10 monclient(hunting): get_auth_request con 0x7f09d00a2cd0 auth_method 0 2024-09-15T01:34:05.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:05.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:05.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:05.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:05.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:05.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:05.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ce59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:05.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ce59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:05.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ce59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:05.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 10 monclient(hunting): handle_auth_done global_id 5410 payload 274 2024-09-15T01:34:05.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 1 -- 172.21.15.204:0/3824256313 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09d00a2cd0 msgr2=0x7f09d0132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:05.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 1 --2- 172.21.15.204:0/3824256313 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09d00a2cd0 0x7f09d0132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 1 -- 172.21.15.204:0/3824256313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09d00a08e0 msgr2=0x7f09d00a0d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:05.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 1 --2- 172.21.15.204:0/3824256313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09d00a08e0 0x7f09d00a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 10 monclient: _finish_hunting 0 2024-09-15T01:34:05.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 1 monclient: found mon.noname-f 2024-09-15T01:34:05.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:05.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09cdd9b640 1 -- 172.21.15.204:0/3824256313 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d00648a0 con 0x7f09d009fa20 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.916+0000 7f09ce59c640 1 --2- 172.21.15.204:0/3824256313 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09d00a2cd0 0x7f09d0132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cdd9b640 1 --2- 172.21.15.204:0/3824256313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d009fa20 0x7f09d009fdf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009850 tx=0x7f09c40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f27de63086c439 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cd59a640 1 -- 172.21.15.204:0/3824256313 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f09c402abe0 con 0x7f09d009fa20 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cd59a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cd59a640 10 monclient: dump: 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:05.934 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cd59a640 10 monclient: _finish_auth 0 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:05.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09d4f09640 5 monclient: authenticate success, global_id 5410 2024-09-15T01:34:05.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cd59a640 1 -- 172.21.15.204:0/3824256313 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c402ad60 con 0x7f09d009fa20 2024-09-15T01:34:05.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09d4f09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:05.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cd59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:05.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09d4f09640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:05.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09d4f09640 4 set_mon_vals no callback set 2024-09-15T01:34:05.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cd59a640 1 -- 172.21.15.204:0/3824256313 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f09c4031ca0 con 0x7f09d009fa20 2024-09-15T01:34:05.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:05.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cd59a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:05.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.917+0000 7f09cd59a640 10 monclient: dump: 2024-09-15T01:34:05.936 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:05.937 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 10 monclient: shutdown 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 1 -- 172.21.15.204:0/3824256313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d009fa20 msgr2=0x7f09d009fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 1 --2- 172.21.15.204:0/3824256313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d009fa20 0x7f09d009fdf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009850 tx=0x7f09c40090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 1 -- 172.21.15.204:0/3824256313 shutdown_connections 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 1 --2- 172.21.15.204:0/3824256313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09d00a08e0 0x7f09d00a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 1 --2- 172.21.15.204:0/3824256313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09d009fa20 0x7f09d009fdf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 1 --2- 172.21.15.204:0/3824256313 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09d00a2cd0 0x7f09d0132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 1 -- 172.21.15.204:0/3824256313 >> 172.21.15.204:0/3824256313 conn(0x7f09d006e860 msgr2=0x7f09d006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 1 -- 172.21.15.204:0/3824256313 shutdown_connections 2024-09-15T01:34:05.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 1 -- 172.21.15.204:0/3824256313 wait complete. 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 10 monclient: build_initial_monmap 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.918+0000 7f09d4f09640 10 monclient: monmap: 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:05.939 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:05.940 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:05.940 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:05.940 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:05.940 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:05.940 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:05.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 1 Processor -- start 2024-09-15T01:34:05.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 1 -- start start 2024-09-15T01:34:05.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 10 monclient: init 2024-09-15T01:34:05.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:05.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 10 monclient: _add_conns ranks=[2,6,3,0,7,8,4,1,5] 2024-09-15T01:34:05.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d009fa20 0x7f09d0130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 10 monclient(hunting): picked mon.noname-b con 0x7f09d009fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f09d00a08e0 0x7f09d011db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 10 monclient(hunting): picked mon.noname-d con 0x7f09d00a08e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09d00a2cd0 0x7f09d011e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 10 monclient(hunting): picked mon.noname-g con 0x7f09d00a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d009d3d0 con 0x7f09d009fa20 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d009cb60 con 0x7f09d00a2cd0 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f09d0142870 con 0x7f09d00a08e0 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09d4f09640 10 monclient(hunting): authenticate will time out at 861.562618s 2024-09-15T01:34:05.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09ced9d640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09d00a2cd0 msgr2=0x7f09d011e080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:34:05.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09ced9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09d00a2cd0 0x7f09d011e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:05.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09cdd9b640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f09d00a08e0 msgr2=0x7f09d011db40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:05.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09cdd9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f09d00a08e0 0x7f09d011db40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:05.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09ce59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d009fa20 0x7f09d0130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:05.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.919+0000 7f09ce59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d009fa20 0x7f09d0130f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:33398/0 (socket says 172.21.15.204:33398) 2024-09-15T01:34:05.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 1 -- 172.21.15.204:0/443831195 learned_addr learned my addr 172.21.15.204:0/443831195 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:05.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 10 monclient(hunting): get_auth_request con 0x7f09d009fa20 auth_method 0 2024-09-15T01:34:05.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:05.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:05.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 10 monclient(hunting): handle_auth_done global_id 4925 payload 824 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 1 -- 172.21.15.204:0/443831195 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f09d00a08e0 msgr2=0x7f09d011db40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 1 --2- 172.21.15.204:0/443831195 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f09d00a08e0 0x7f09d011db40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 1 -- 172.21.15.204:0/443831195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09d00a2cd0 msgr2=0x7f09d011e080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 1 --2- 172.21.15.204:0/443831195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09d00a2cd0 0x7f09d011e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 1 monclient: found mon.noname-b 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:34:05.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 1 -- 172.21.15.204:0/443831195 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0009450 con 0x7f09d009fa20 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ce59c640 1 --2- 172.21.15.204:0/443831195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d009fa20 0x7f09d0130f20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09c000ea60 tx=0x7f09c00091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09be7fc640 1 -- 172.21.15.204:0/443831195 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f09c0009690 con 0x7f09d009fa20 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09be7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09be7fc640 10 monclient: dump: 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:05.944 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:05.945 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:05.945 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:05.945 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:05.945 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:05.945 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:05.945 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:05.945 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:05.945 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:05.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09be7fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:05.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:05.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09d4f09640 5 monclient: authenticate success, global_id 4925 2024-09-15T01:34:05.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09be7fc640 1 -- 172.21.15.204:0/443831195 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c000ce90 con 0x7f09d009fa20 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09be7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09be7fc640 1 -- 172.21.15.204:0/443831195 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f09c00138f0 con 0x7f09d009fa20 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09d4f09640 10 monclient: _renew_subs 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09d4f09640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09d4f09640 1 -- 172.21.15.204:0/443831195 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c4009390 con 0x7f09d009fa20 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09ccd99640 4 set_mon_vals no callback set 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09be7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09be7fc640 10 monclient: dump: 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.946 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:05.947 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:05.947 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:05.947 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:05.947 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:05.947 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:05.947 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:05.947 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:05.947 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:05.947 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:05.947 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:05.947 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:05.948 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:05.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09d4f09640 10 monclient: _renew_subs 2024-09-15T01:34:05.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09d4f09640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:05.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.920+0000 7f09d4f09640 1 -- 172.21.15.204:0/443831195 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d011f540 con 0x7f09d009fa20 2024-09-15T01:34:05.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.921+0000 7f09be7fc640 1 -- 172.21.15.204:0/443831195 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f09c0013470 con 0x7f09d009fa20 2024-09-15T01:34:05.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.921+0000 7f09d4f09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:05.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.921+0000 7f09d4f09640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:05.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.921+0000 7f09d4f09640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:05.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.921+0000 7f09d4f09640 1 -- 172.21.15.204:0/443831195 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d01214f0 con 0x7f09d009fa20 2024-09-15T01:34:05.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.921+0000 7f09be7fc640 1 --2- 172.21.15.204:0/443831195 >> v2:172.21.15.204:6812/28210 conn(0x7f09b00397b0 0x7f09b003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:05.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.921+0000 7f09be7fc640 1 -- 172.21.15.204:0/443831195 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5204+0+0 (secure 0 0 0) 0x7f09c004bc00 con 0x7f09d009fa20 2024-09-15T01:34:05.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.921+0000 7f09cdd9b640 1 --2- 172.21.15.204:0/443831195 >> v2:172.21.15.204:6812/28210 conn(0x7f09b00397b0 0x7f09b003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:05.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.921+0000 7f09cdd9b640 10 monclient: get_auth_request con 0x7f09b00397b0 auth_method 0 2024-09-15T01:34:05.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.922+0000 7f09cdd9b640 1 --2- 172.21.15.204:0/443831195 >> v2:172.21.15.204:6812/28210 conn(0x7f09b00397b0 0x7f09b003bc60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f09c4034000 tx=0x7f09c4032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:05.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.926+0000 7f09be7fc640 1 -- 172.21.15.204:0/443831195 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f09c0051020 con 0x7f09d009fa20 2024-09-15T01:34:05.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.926+0000 7f09be7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:05.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:05.926+0000 7f09be7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:06.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.081+0000 7f09d4f09640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T01:34:06.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.081+0000 7f09d4f09640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T01:34:06.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.081+0000 7f09d4f09640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:06.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.081+0000 7f09d4f09640 1 -- 172.21.15.204:0/443831195 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) v1 -- 0x7f09d0070b80 con 0x7f09d009fa20 2024-09-15T01:34:06.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.082+0000 7f09be7fc640 1 -- 172.21.15.204:0/443831195 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change v74) v1 ==== 201+0+0 (secure 0 0 0) 0x7f09c001b990 con 0x7f09d009fa20 2024-09-15T01:34:06.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.082+0000 7f09be7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T01:34:06.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.082+0000 7f09be7fc640 10 monclient: _finish_command 2 = system:0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-09-15T01:34:06.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.178+0000 7f09d4f09640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T01:34:06.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.178+0000 7f09d4f09640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T01:34:06.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.178+0000 7f09d4f09640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:06.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.178+0000 7f09d4f09640 1 -- 172.21.15.204:0/443831195 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) v1 -- 0x7f09d01216c0 con 0x7f09d009fa20 2024-09-15T01:34:06.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.179+0000 7f09be7fc640 1 -- 172.21.15.204:0/443831195 <== mon.2 v2:172.21.15.178:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change v74) v1 ==== 201+0+0 (secure 0 0 0) 0x7f09c001b3b0 con 0x7f09d009fa20 2024-09-15T01:34:06.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.180+0000 7f09be7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-09-15T01:34:06.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.180+0000 7f09be7fc640 10 monclient: _finish_command 3 = system:0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-09-15T01:34:06.182 INFO:tasks.workunit.client.0.smithi204.stderr:set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 1 -- 172.21.15.204:0/443831195 >> v2:172.21.15.204:6812/28210 conn(0x7f09b00397b0 msgr2=0x7f09b003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 1 --2- 172.21.15.204:0/443831195 >> v2:172.21.15.204:6812/28210 conn(0x7f09b00397b0 0x7f09b003bc60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f09c4034000 tx=0x7f09c4032000 comp rx=0 tx=0).stop 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 10 monclient: shutdown 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 1 -- 172.21.15.204:0/443831195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d009fa20 msgr2=0x7f09d0130f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 1 --2- 172.21.15.204:0/443831195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d009fa20 0x7f09d0130f20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09c000ea60 tx=0x7f09c00091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 1 -- 172.21.15.204:0/443831195 shutdown_connections 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 1 --2- 172.21.15.204:0/443831195 >> v2:172.21.15.204:6812/28210 conn(0x7f09b00397b0 0x7f09b003bc60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 1 --2- 172.21.15.204:0/443831195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09d00a2cd0 0x7f09d011e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 1 --2- 172.21.15.204:0/443831195 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f09d00a08e0 0x7f09d011db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 1 --2- 172.21.15.204:0/443831195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09d009fa20 0x7f09d0130f20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 1 -- 172.21.15.204:0/443831195 >> 172.21.15.204:0/443831195 conn(0x7f09d006e860 msgr2=0x7f09d00a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:06.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 1 -- 172.21.15.204:0/443831195 shutdown_connections 2024-09-15T01:34:06.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.182+0000 7f09d4f09640 1 -- 172.21.15.204:0/443831195 wait complete. 2024-09-15T01:34:06.193 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket bar root 2024-09-15T01:34:06.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.265+0000 7fbdb10c9640 10 monclient: get_monmap_and_config 2024-09-15T01:34:06.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.265+0000 7fbdb10c9640 10 monclient: build_initial_monmap 2024-09-15T01:34:06.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.265+0000 7fbdb10c9640 10 monclient: monmap: 2024-09-15T01:34:06.268 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:06.268 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:06.269 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:06.267010+0000 2024-09-15T01:34:06.269 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:06.267010+0000 2024-09-15T01:34:06.269 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:06.269 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:06.269 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:06.269 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:06.269 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:06.269 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:06.269 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:06.269 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:06.269 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:06.269 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:06.270 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:06.270 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:06.270 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:06.270 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:06.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 1 Processor -- start 2024-09-15T01:34:06.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 1 -- start start 2024-09-15T01:34:06.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 10 monclient: init 2024-09-15T01:34:06.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:06.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 10 monclient: _add_conns ranks=[1,4,7,5,10,2,8,9,6,3,0] 2024-09-15T01:34:06.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbdac0a2e30 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:06.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdac09fb80 0x7fbdac09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:06.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 10 monclient(hunting): picked mon.noname-c con 0x7fbdac09fb80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:06.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdac137d20 0x7fbdac13a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:06.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 10 monclient(hunting): picked mon.noname-i con 0x7fbdac137d20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:06.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac04b720 con 0x7fbdac0a2e30 2024-09-15T01:34:06.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:06.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac09e6c0 con 0x7fbdac09fb80 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac0a04b0 con 0x7fbdac137d20 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdb10c9640 10 monclient(hunting): authenticate will time out at 861.909655s 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdab59e640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdac137d20 0x7fbdac13a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdaad9d640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fbdac0a2e30 0x7fbdac132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:48798/0 (socket says 172.21.15.204:48798) 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdaad9d640 1 -- 172.21.15.204:0/1034854559 learned_addr learned my addr 172.21.15.204:0/1034854559 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdab59e640 10 monclient(hunting): get_auth_request con 0x7fbdac137d20 auth_method 0 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdab59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdab59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdab59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdab59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdab59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:06.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.266+0000 7fbdab59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:06.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdab59e640 10 monclient(hunting): handle_auth_done global_id 5038 payload 274 2024-09-15T01:34:06.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 1 -- 172.21.15.204:0/1034854559 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508655647 0 0) 0x7fbdac04b720 con 0x7fbdac0a2e30 2024-09-15T01:34:06.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdab59e640 1 -- 172.21.15.204:0/1034854559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdac09fb80 msgr2=0x7fbdac09ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:06.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdaa59c640 1 --2- 172.21.15.204:0/1034854559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdac09fb80 0x7fbdac09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:06.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdab59e640 1 --2- 172.21.15.204:0/1034854559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdac09fb80 0x7fbdac09ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdab59e640 1 -- 172.21.15.204:0/1034854559 >> v1:172.21.15.204:6789/0 conn(0x7fbdac0a2e30 legacy=0x7fbdac132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:06.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdab59e640 10 monclient: _finish_hunting 0 2024-09-15T01:34:06.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdab59e640 1 monclient: found mon.noname-i 2024-09-15T01:34:06.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdab59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:06.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdab59e640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:34:06.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdab59e640 1 -- 172.21.15.204:0/1034854559 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdac133170 con 0x7fbdac137d20 2024-09-15T01:34:06.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdab59e640 1 --2- 172.21.15.204:0/1034854559 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdac137d20 0x7fbdac13a100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009e90 tx=0x7fbda4009780 comp rx=0 tx=0).ready entity=mon.7 client_cookie=d3bb8f30ae809f31 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 1 -- 172.21.15.204:0/1034854559 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbda4031880 con 0x7fbdac137d20 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 10 monclient: dump: 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:06.281 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 10 monclient: _finish_auth 0 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 1 -- 172.21.15.204:0/1034854559 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda402d050 con 0x7fbdac137d20 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdb10c9640 5 monclient: authenticate success, global_id 5038 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdb10c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:06.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 1 -- 172.21.15.204:0/1034854559 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbda40303a0 con 0x7fbdac137d20 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdb10c9640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbdb10c9640 4 set_mon_vals no callback set 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.267+0000 7fbda9d9b640 10 monclient: dump: 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:06.283 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:06.284 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:06.284 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:06.284 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:06.284 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:06.284 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:06.284 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:06.284 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:06.284 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:06.284 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:06.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.268+0000 7fbdb10c9640 10 monclient: shutdown 2024-09-15T01:34:06.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.268+0000 7fbdb10c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:06.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.268+0000 7fbdb10c9640 1 -- 172.21.15.204:0/1034854559 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdac137d20 msgr2=0x7fbdac13a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.268+0000 7fbdb10c9640 1 --2- 172.21.15.204:0/1034854559 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdac137d20 0x7fbdac13a100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009e90 tx=0x7fbda4009780 comp rx=0 tx=0).stop 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.268+0000 7fbdb10c9640 1 -- 172.21.15.204:0/1034854559 shutdown_connections 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.268+0000 7fbdb10c9640 1 --2- 172.21.15.204:0/1034854559 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdac137d20 0x7fbdac13a100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.268+0000 7fbdb10c9640 1 --2- 172.21.15.204:0/1034854559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdac09fb80 0x7fbdac09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.268+0000 7fbdb10c9640 1 -- 172.21.15.204:0/1034854559 >> 172.21.15.204:0/1034854559 conn(0x7fbdac06e860 msgr2=0x7fbdac06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.268+0000 7fbdb10c9640 1 -- 172.21.15.204:0/1034854559 shutdown_connections 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.268+0000 7fbdb10c9640 1 -- 172.21.15.204:0/1034854559 wait complete. 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient: build_initial_monmap 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient: monmap: 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:06.285 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 1 Processor -- start 2024-09-15T01:34:06.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 1 -- start start 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient: init 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient: _add_conns ranks=[6,4,2,3,8,7,0,1,5] 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbdac09fb80 0x7fbdac099210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient(hunting): picked mon.noname-d con 0x7fbdac09fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdac0a2e30 0x7fbdac099750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient(hunting): picked mon.noname-c con 0x7fbdac0a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdac137d20 0x7fbdac099c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient(hunting): picked mon.noname-b con 0x7fbdac137d20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac130ac0 con 0x7fbdac137d20 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:06.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac131450 con 0x7fbdac0a2e30 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac09d650 con 0x7fbdac09fb80 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.269+0000 7fbdb10c9640 10 monclient(hunting): authenticate will time out at 861.912917s 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaad9d640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbdac09fb80 msgr2=0x7fbdac099210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaad9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbdac09fb80 0x7fbdac099210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdac0a2e30 0x7fbdac099750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdab59e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdac137d20 0x7fbdac099c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdac0a2e30 0x7fbdac099750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:60388/0 (socket says 172.21.15.204:60388) 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdab59e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdac137d20 0x7fbdac099c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:33404/0 (socket says 172.21.15.204:33404) 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 1 -- 172.21.15.204:0/4224064642 learned_addr learned my addr 172.21.15.204:0/4224064642 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 10 monclient(hunting): get_auth_request con 0x7fbdac0a2e30 auth_method 0 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:06.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdab59e640 10 monclient(hunting): get_auth_request con 0x7fbdac137d20 auth_method 0 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdab59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdab59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdab59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdab59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdab59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdab59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 10 monclient(hunting): handle_auth_done global_id 4900 payload 824 2024-09-15T01:34:06.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 1 -- 172.21.15.204:0/4224064642 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbdac09fb80 msgr2=0x7fbdac099210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 1 --2- 172.21.15.204:0/4224064642 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbdac09fb80 0x7fbdac099210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 1 -- 172.21.15.204:0/4224064642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdac137d20 msgr2=0x7fbdac099c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 1 --2- 172.21.15.204:0/4224064642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdac137d20 0x7fbdac099c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 1 monclient: found mon.noname-c 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 1 -- 172.21.15.204:0/4224064642 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4009a20 con 0x7fbdac0a2e30 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdab59e640 1 --2- 172.21.15.204:0/4224064642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdac137d20 0x7fbdac099c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdaa59c640 1 --2- 172.21.15.204:0/4224064642 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdac0a2e30 0x7fbdac099750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbda000c8b0 tx=0x7fbda0008040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbd92ffd640 1 -- 172.21.15.204:0/4224064642 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbda000ce40 con 0x7fbdac0a2e30 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbd92ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbd92ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:06.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbd92ffd640 10 monclient: dump: 2024-09-15T01:34:06.291 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:06.291 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:06.291 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.291 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.291 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:06.291 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:06.291 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:06.291 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:06.291 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:06.291 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:06.291 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:06.291 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbd92ffd640 10 monclient: _finish_auth 0 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbd92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbd92ffd640 1 -- 172.21.15.204:0/4224064642 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda0007cb0 con 0x7fbdac0a2e30 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbdb10c9640 5 monclient: authenticate success, global_id 4900 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbd92ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbd92ffd640 1 -- 172.21.15.204:0/4224064642 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbda0007e20 con 0x7fbdac0a2e30 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbd92ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbd92ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:06.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbd92ffd640 10 monclient: dump: 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:06.293 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.270+0000 7fbda959a640 4 set_mon_vals no callback set 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.271+0000 7fbdb10c9640 10 monclient: _renew_subs 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.271+0000 7fbdb10c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.271+0000 7fbdb10c9640 1 -- 172.21.15.204:0/4224064642 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdac11ca30 con 0x7fbdac0a2e30 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.271+0000 7fbdb10c9640 10 monclient: _renew_subs 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.271+0000 7fbdb10c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.271+0000 7fbdb10c9640 1 -- 172.21.15.204:0/4224064642 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdac09b920 con 0x7fbdac0a2e30 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.271+0000 7fbdb10c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.271+0000 7fbdb10c9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.271+0000 7fbdb10c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:06.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.271+0000 7fbdb10c9640 1 -- 172.21.15.204:0/4224064642 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdac0668b0 con 0x7fbdac0a2e30 2024-09-15T01:34:06.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.272+0000 7fbd92ffd640 1 -- 172.21.15.204:0/4224064642 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fbda0007830 con 0x7fbdac0a2e30 2024-09-15T01:34:06.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.272+0000 7fbd92ffd640 1 --2- 172.21.15.204:0/4224064642 >> v2:172.21.15.204:6812/28210 conn(0x7fbd980397b0 0x7fbd9803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:06.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.272+0000 7fbd92ffd640 1 -- 172.21.15.204:0/4224064642 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5204+0+0 (secure 0 0 0) 0x7fbda004b620 con 0x7fbdac0a2e30 2024-09-15T01:34:06.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.272+0000 7fbdaad9d640 1 --2- 172.21.15.204:0/4224064642 >> v2:172.21.15.204:6812/28210 conn(0x7fbd980397b0 0x7fbd9803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:06.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.272+0000 7fbdaad9d640 10 monclient: get_auth_request con 0x7fbd980397b0 auth_method 0 2024-09-15T01:34:06.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.272+0000 7fbdaad9d640 1 --2- 172.21.15.204:0/4224064642 >> v2:172.21.15.204:6812/28210 conn(0x7fbd980397b0 0x7fbd9803bc60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0010e0 tx=0x7fbd9c007ae0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:06.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.276+0000 7fbd92ffd640 1 -- 172.21.15.204:0/4224064642 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fbdac0668b0 con 0x7fbdac0a2e30 2024-09-15T01:34:06.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.276+0000 7fbd92ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:06.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.276+0000 7fbd92ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:06.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.434+0000 7fbdb10c9640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T01:34:06.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.434+0000 7fbdb10c9640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T01:34:06.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.434+0000 7fbdb10c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:06.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.434+0000 7fbdb10c9640 1 -- 172.21.15.204:0/4224064642 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) v1 -- 0x7fbdac09ad30 con 0x7fbdac0a2e30 2024-09-15T01:34:06.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.465+0000 7fbd92ffd640 1 -- 172.21.15.204:0/4224064642 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 added bucket bar type root to crush map v75) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbda001e020 con 0x7fbdac0a2e30 2024-09-15T01:34:06.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.465+0000 7fbd92ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T01:34:06.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.465+0000 7fbd92ffd640 10 monclient: _finish_command 2 = system:0 added bucket bar type root to crush map 2024-09-15T01:34:06.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.560+0000 7fbdb10c9640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T01:34:06.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.560+0000 7fbdb10c9640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T01:34:06.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.560+0000 7fbdb10c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:06.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.560+0000 7fbdb10c9640 1 -- 172.21.15.204:0/4224064642 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) v1 -- 0x7fbdac11cbc0 con 0x7fbdac0a2e30 2024-09-15T01:34:06.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.561+0000 7fbd92ffd640 1 -- 172.21.15.204:0/4224064642 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 bucket 'bar' already exists v75) v1 ==== 126+0+0 (secure 0 0 0) 0x7fbda0017300 con 0x7fbdac0a2e30 2024-09-15T01:34:06.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.561+0000 7fbd92ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T01:34:06.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.561+0000 7fbd92ffd640 10 monclient: _finish_command 3 = system:0 bucket 'bar' already exists 2024-09-15T01:34:06.564 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'bar' already exists 2024-09-15T01:34:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 1 -- 172.21.15.204:0/4224064642 >> v2:172.21.15.204:6812/28210 conn(0x7fbd980397b0 msgr2=0x7fbd9803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 1 --2- 172.21.15.204:0/4224064642 >> v2:172.21.15.204:6812/28210 conn(0x7fbd980397b0 0x7fbd9803bc60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0010e0 tx=0x7fbd9c007ae0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 10 monclient: shutdown 2024-09-15T01:34:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 1 -- 172.21.15.204:0/4224064642 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdac0a2e30 msgr2=0x7fbdac099750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 1 --2- 172.21.15.204:0/4224064642 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdac0a2e30 0x7fbdac099750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbda000c8b0 tx=0x7fbda0008040 comp rx=0 tx=0).stop 2024-09-15T01:34:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 1 -- 172.21.15.204:0/4224064642 shutdown_connections 2024-09-15T01:34:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 1 --2- 172.21.15.204:0/4224064642 >> v2:172.21.15.204:6812/28210 conn(0x7fbd980397b0 0x7fbd9803bc60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 1 --2- 172.21.15.204:0/4224064642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdac137d20 0x7fbdac099c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 1 --2- 172.21.15.204:0/4224064642 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdac0a2e30 0x7fbdac099750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 1 --2- 172.21.15.204:0/4224064642 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbdac09fb80 0x7fbdac099210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 1 -- 172.21.15.204:0/4224064642 >> 172.21.15.204:0/4224064642 conn(0x7fbdac06e860 msgr2=0x7fbdac0a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 1 -- 172.21.15.204:0/4224064642 shutdown_connections 2024-09-15T01:34:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.563+0000 7fbdb10c9640 1 -- 172.21.15.204:0/4224064642 wait complete. 2024-09-15T01:34:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket bar root 2024-09-15T01:34:06.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.646+0000 7fb208f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:06.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.646+0000 7fb208f9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:06.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.646+0000 7fb208f9d640 10 monclient: monmap: 2024-09-15T01:34:06.650 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:06.650 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:06.650 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:06.648061+0000 2024-09-15T01:34:06.650 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:06.648061+0000 2024-09-15T01:34:06.650 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:06.650 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:06.650 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:06.650 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:06.650 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:06.651 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:06.651 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:06.651 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:06.651 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:06.651 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:06.651 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:06.651 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:06.651 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:06.651 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:06.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 1 Processor -- start 2024-09-15T01:34:06.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 1 -- start start 2024-09-15T01:34:06.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 10 monclient: init 2024-09-15T01:34:06.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:06.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 10 monclient: _add_conns ranks=[0,4,7,8,3,2,5,9,1,10,6] 2024-09-15T01:34:06.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb2040a2e30 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:06.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb20409fb80 0x7fb20409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:06.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 10 monclient(hunting): picked mon.noname-c con 0x7fb20409fb80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:06.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb204137d20 0x7fb20413a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:06.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 10 monclient(hunting): picked mon.noname-i con 0x7fb204137d20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:06.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20404b720 con 0x7fb2040a2e30 2024-09-15T01:34:06.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:06.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb20409e6c0 con 0x7fb20409fb80 2024-09-15T01:34:06.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:06.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040a04b0 con 0x7fb204137d20 2024-09-15T01:34:06.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:06.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.647+0000 7fb208f9d640 10 monclient(hunting): authenticate will time out at 862.290921s 2024-09-15T01:34:06.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb204137d20 0x7fb20413a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb204137d20 0x7fb20413a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:35128/0 (socket says 172.21.15.204:35128) 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 1 -- 172.21.15.204:0/1317401974 learned_addr learned my addr 172.21.15.204:0/1317401974 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 10 monclient(hunting): get_auth_request con 0x7fb204137d20 auth_method 0 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb201d9b640 1 --2- 172.21.15.204:0/1317401974 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb20409fb80 0x7fb20409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 10 monclient(hunting): handle_auth_done global_id 5047 payload 274 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 1 -- 172.21.15.204:0/1317401974 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb20409fb80 msgr2=0x7fb20409ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:06.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 1 --2- 172.21.15.204:0/1317401974 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb20409fb80 0x7fb20409ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 1 -- 172.21.15.204:0/1317401974 >> v1:172.21.15.178:6789/0 conn(0x7fb2040a2e30 legacy=0x7fb204132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 1 monclient: found mon.noname-i 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 1 -- 172.21.15.204:0/1317401974 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb204133170 con 0x7fb204137d20 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb201d9b640 10 monclient: get_auth_request con 0x7fb20409fb80 auth_method 0 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb201d9b640 1 --2- 172.21.15.204:0/1317401974 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb20409fb80 0x7fb20409ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.648+0000 7fb202d9d640 1 --2- 172.21.15.204:0/1317401974 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb204137d20 0x7fb20413a100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009f50 tx=0x7fb1fc009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=87fd034a47d4286c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb20159a640 1 -- 172.21.15.204:0/1317401974 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1fc007880 con 0x7fb204137d20 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb20159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb20159a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:34:06.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb20159a640 10 monclient: dump: 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:06.664 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb20159a640 10 monclient: _finish_auth 0 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb20159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb20159a640 1 -- 172.21.15.204:0/1317401974 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1fc02d050 con 0x7fb204137d20 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 5 monclient: authenticate success, global_id 5047 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb20159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb20159a640 1 -- 172.21.15.204:0/1317401974 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1fc030360 con 0x7fb204137d20 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb20159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb20159a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb20159a640 10 monclient: dump: 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:06.665 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:06.666 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 4 set_mon_vals no callback set 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 10 monclient: shutdown 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 1 -- 172.21.15.204:0/1317401974 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb204137d20 msgr2=0x7fb20413a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 1 --2- 172.21.15.204:0/1317401974 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb204137d20 0x7fb20413a100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009f50 tx=0x7fb1fc009840 comp rx=0 tx=0).stop 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 1 -- 172.21.15.204:0/1317401974 shutdown_connections 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 1 --2- 172.21.15.204:0/1317401974 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb204137d20 0x7fb20413a100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 1 --2- 172.21.15.204:0/1317401974 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb20409fb80 0x7fb20409ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 1 -- 172.21.15.204:0/1317401974 >> 172.21.15.204:0/1317401974 conn(0x7fb20406e860 msgr2=0x7fb20406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 1 -- 172.21.15.204:0/1317401974 shutdown_connections 2024-09-15T01:34:06.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.649+0000 7fb208f9d640 1 -- 172.21.15.204:0/1317401974 wait complete. 2024-09-15T01:34:06.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.650+0000 7fb208f9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:06.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.650+0000 7fb208f9d640 10 monclient: monmap: 2024-09-15T01:34:06.668 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:06.668 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:06.668 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:06.668 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:06.668 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:06.668 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:06.668 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:06.668 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:06.668 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.650+0000 7fb208f9d640 1 Processor -- start 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.650+0000 7fb208f9d640 1 -- start start 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.650+0000 7fb208f9d640 10 monclient: init 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 10 monclient: _add_conns ranks=[0,1,6,8,5,2,7,3,4] 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb20409fb80 0x7fb204098f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 10 monclient(hunting): picked mon.noname-a con 0x7fb20409fb80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2040a2e30 0x7fb2040994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 10 monclient(hunting): picked mon.noname-f con 0x7fb2040a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb204137d20 0x7fb2040999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 10 monclient(hunting): picked mon.noname-d con 0x7fb204137d20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20409b070 con 0x7fb20409fb80 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20409d650 con 0x7fb2040a2e30 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb20411c7e0 con 0x7fb204137d20 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 10 monclient(hunting): authenticate will time out at 862.294194s 2024-09-15T01:34:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb202d9d640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb204137d20 msgr2=0x7fb2040999e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb202d9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb204137d20 0x7fb2040999e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2040a2e30 0x7fb2040994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2040a2e30 0x7fb2040994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38184/0 (socket says 172.21.15.204:38184) 2024-09-15T01:34:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 1 -- 172.21.15.204:0/3250070792 learned_addr learned my addr 172.21.15.204:0/3250070792 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 10 monclient(hunting): get_auth_request con 0x7fb2040a2e30 auth_method 0 2024-09-15T01:34:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb20259c640 1 --2- 172.21.15.204:0/3250070792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb20409fb80 0x7fb204098f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:06.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:06.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:06.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:06.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb20259c640 10 monclient(hunting): get_auth_request con 0x7fb20409fb80 auth_method 0 2024-09-15T01:34:06.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb20259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb20259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb20259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 10 monclient(hunting): handle_auth_done global_id 5428 payload 824 2024-09-15T01:34:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 1 -- 172.21.15.204:0/3250070792 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb204137d20 msgr2=0x7fb2040999e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 1 --2- 172.21.15.204:0/3250070792 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb204137d20 0x7fb2040999e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 1 -- 172.21.15.204:0/3250070792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb20409fb80 msgr2=0x7fb204098f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 1 --2- 172.21.15.204:0/3250070792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb20409fb80 0x7fb204098f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 1 monclient: found mon.noname-f 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 1 -- 172.21.15.204:0/3250070792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc009ae0 con 0x7fb2040a2e30 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb20259c640 1 --2- 172.21.15.204:0/3250070792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb20409fb80 0x7fb204098f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb201d9b640 1 --2- 172.21.15.204:0/3250070792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2040a2e30 0x7fb2040994a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800c850 tx=0x7fb1f8008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb1f27fc640 1 -- 172.21.15.204:0/3250070792 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1f8007b00 con 0x7fb2040a2e30 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb1f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb1f27fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb1f27fc640 10 monclient: dump: 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:06.675 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:06.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb1f27fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:06.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb1f27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:06.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb1f27fc640 1 -- 172.21.15.204:0/3250070792 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f8007cc0 con 0x7fb2040a2e30 2024-09-15T01:34:06.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 5 monclient: authenticate success, global_id 5428 2024-09-15T01:34:06.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 10 monclient: _renew_subs 2024-09-15T01:34:06.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:06.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb208f9d640 1 -- 172.21.15.204:0/3250070792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb204131300 con 0x7fb2040a2e30 2024-09-15T01:34:06.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb1f27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:06.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb1f27fc640 1 -- 172.21.15.204:0/3250070792 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1f8007e30 con 0x7fb2040a2e30 2024-09-15T01:34:06.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb1f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:06.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb1f27fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:06.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb1f27fc640 10 monclient: dump: 2024-09-15T01:34:06.677 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:06.677 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:06.677 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.677 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:06.677 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:06.677 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:06.677 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:06.677 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:06.677 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:06.677 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:06.677 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:06.678 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:06.678 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:06.678 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:06.678 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:06.678 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:06.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.652+0000 7fb208f9d640 10 monclient: _renew_subs 2024-09-15T01:34:06.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.651+0000 7fb200d99640 4 set_mon_vals no callback set 2024-09-15T01:34:06.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.652+0000 7fb208f9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:06.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.652+0000 7fb208f9d640 1 -- 172.21.15.204:0/3250070792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20409c990 con 0x7fb2040a2e30 2024-09-15T01:34:06.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.652+0000 7fb1f27fc640 1 -- 172.21.15.204:0/3250070792 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb1f80113f0 con 0x7fb2040a2e30 2024-09-15T01:34:06.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.652+0000 7fb1f27fc640 1 --2- 172.21.15.204:0/3250070792 >> v2:172.21.15.204:6812/28210 conn(0x7fb1e40396e0 0x7fb1e403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:06.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.652+0000 7fb1f27fc640 1 -- 172.21.15.204:0/3250070792 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5231+0+0 (secure 0 0 0) 0x7fb1f804b7d0 con 0x7fb2040a2e30 2024-09-15T01:34:06.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.653+0000 7fb208f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:06.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.653+0000 7fb208f9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:06.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.653+0000 7fb208f9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:06.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.653+0000 7fb20259c640 1 --2- 172.21.15.204:0/3250070792 >> v2:172.21.15.204:6812/28210 conn(0x7fb1e40396e0 0x7fb1e403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:06.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.653+0000 7fb208f9d640 1 -- 172.21.15.204:0/3250070792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb204070cc0 con 0x7fb2040a2e30 2024-09-15T01:34:06.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.653+0000 7fb20259c640 10 monclient: get_auth_request con 0x7fb1e40396e0 auth_method 0 2024-09-15T01:34:06.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.653+0000 7fb20259c640 1 --2- 172.21.15.204:0/3250070792 >> v2:172.21.15.204:6812/28210 conn(0x7fb1e40396e0 0x7fb1e403bb90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4002860 tx=0x7fb1f4009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:06.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.657+0000 7fb1f27fc640 1 -- 172.21.15.204:0/3250070792 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb1f8018350 con 0x7fb2040a2e30 2024-09-15T01:34:06.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.657+0000 7fb1f27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:06.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.657+0000 7fb1f27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:06.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.821+0000 7fb208f9d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T01:34:06.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.821+0000 7fb208f9d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T01:34:06.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.821+0000 7fb208f9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:06.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.821+0000 7fb208f9d640 1 -- 172.21.15.204:0/3250070792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) v1 -- 0x7fb20409aaf0 con 0x7fb2040a2e30 2024-09-15T01:34:06.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.822+0000 7fb1f27fc640 1 -- 172.21.15.204:0/3250070792 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 bucket 'bar' already exists v75) v1 ==== 126+0+0 (secure 0 0 0) 0x7fb1f8020300 con 0x7fb2040a2e30 2024-09-15T01:34:06.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.822+0000 7fb1f27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T01:34:06.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.822+0000 7fb1f27fc640 10 monclient: _finish_command 2 = system:0 bucket 'bar' already exists 2024-09-15T01:34:06.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.919+0000 7fb208f9d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T01:34:06.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.919+0000 7fb208f9d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T01:34:06.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.919+0000 7fb208f9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:06.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.919+0000 7fb208f9d640 1 -- 172.21.15.204:0/3250070792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) v1 -- 0x7fb2040008d0 con 0x7fb2040a2e30 2024-09-15T01:34:06.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.920+0000 7fb1f27fc640 1 -- 172.21.15.204:0/3250070792 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 bucket 'bar' already exists v75) v1 ==== 126+0+0 (secure 0 0 0) 0x7fb1f8017300 con 0x7fb2040a2e30 2024-09-15T01:34:06.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.920+0000 7fb1f27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-09-15T01:34:06.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.920+0000 7fb1f27fc640 10 monclient: _finish_command 3 = system:0 bucket 'bar' already exists 2024-09-15T01:34:06.923 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'bar' already exists 2024-09-15T01:34:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 1 -- 172.21.15.204:0/3250070792 >> v2:172.21.15.204:6812/28210 conn(0x7fb1e40396e0 msgr2=0x7fb1e403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 1 --2- 172.21.15.204:0/3250070792 >> v2:172.21.15.204:6812/28210 conn(0x7fb1e40396e0 0x7fb1e403bb90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4002860 tx=0x7fb1f4009040 comp rx=0 tx=0).stop 2024-09-15T01:34:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 10 monclient: shutdown 2024-09-15T01:34:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 1 -- 172.21.15.204:0/3250070792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2040a2e30 msgr2=0x7fb2040994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 1 --2- 172.21.15.204:0/3250070792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2040a2e30 0x7fb2040994a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800c850 tx=0x7fb1f8008040 comp rx=0 tx=0).stop 2024-09-15T01:34:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 1 -- 172.21.15.204:0/3250070792 shutdown_connections 2024-09-15T01:34:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 1 --2- 172.21.15.204:0/3250070792 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb204137d20 0x7fb2040999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 1 --2- 172.21.15.204:0/3250070792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2040a2e30 0x7fb2040994a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 1 --2- 172.21.15.204:0/3250070792 >> v2:172.21.15.204:6812/28210 conn(0x7fb1e40396e0 0x7fb1e403bb90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 1 --2- 172.21.15.204:0/3250070792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb20409fb80 0x7fb204098f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 1 -- 172.21.15.204:0/3250070792 >> 172.21.15.204:0/3250070792 conn(0x7fb20406e860 msgr2=0x7fb2040a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 1 -- 172.21.15.204:0/3250070792 shutdown_connections 2024-09-15T01:34:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:06.922+0000 7fb208f9d640 1 -- 172.21.15.204:0/3250070792 wait complete. 2024-09-15T01:34:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush link host1 root=bar 2024-09-15T01:34:07.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.005+0000 7f27108c0640 10 monclient: get_monmap_and_config 2024-09-15T01:34:07.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.005+0000 7f27108c0640 10 monclient: build_initial_monmap 2024-09-15T01:34:07.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.006+0000 7f27108c0640 10 monclient: monmap: 2024-09-15T01:34:07.009 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:07.009 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:07.009 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:07.007478+0000 2024-09-15T01:34:07.009 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:07.007478+0000 2024-09-15T01:34:07.009 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:07.009 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:07.009 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:07.009 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:07.009 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:07.010 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:07.010 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:07.010 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:07.010 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:07.010 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:07.010 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:07.010 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:07.010 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:07.010 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:07.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.006+0000 7f27108c0640 1 Processor -- start 2024-09-15T01:34:07.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.006+0000 7f27108c0640 1 -- start start 2024-09-15T01:34:07.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.006+0000 7f27108c0640 10 monclient: init 2024-09-15T01:34:07.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 10 monclient: _add_conns ranks=[5,7,10,0,2,9,4,6,3,8,1] 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f270c0a2e30 0x7f270c132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 10 monclient(hunting): picked mon.noname-h con 0x7f270c0a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f270c09fb80 0x7f270c09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 10 monclient(hunting): picked mon.noname-i con 0x7f270c09fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f270c0a0a40 0x7f270c0a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 10 monclient(hunting): picked mon.noname-f con 0x7f270c0a0a40 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270c09e6c0 con 0x7f270c0a0a40 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f270c0a04b0 con 0x7f270c0a2e30 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:07.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f270c0a3200 con 0x7f270c09fb80 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f27108c0640 10 monclient(hunting): authenticate will time out at 862.650321s 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270ad9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f270c0a0a40 0x7f270c0a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f270c0a2e30 0x7f270c132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270ad9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f270c0a0a40 0x7f270c0a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38194/0 (socket says 172.21.15.204:38194) 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270ad9d640 1 -- 172.21.15.204:0/838511988 learned_addr learned my addr 172.21.15.204:0/838511988 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270ad9d640 10 monclient(hunting): get_auth_request con 0x7f270c0a0a40 auth_method 0 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f2709d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f270c09fb80 0x7f270c09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 10 monclient(hunting): get_auth_request con 0x7f270c0a2e30 auth_method 0 2024-09-15T01:34:07.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:07.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:07.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:07.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f2709d9b640 10 monclient(hunting): get_auth_request con 0x7f270c09fb80 auth_method 0 2024-09-15T01:34:07.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f2709d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:07.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f2709d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:07.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f2709d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:07.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:07.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:07.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:07.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:07.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:07.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:07.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f2709d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:07.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f2709d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:07.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f2709d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:07.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 10 monclient(hunting): handle_auth_done global_id 4991 payload 274 2024-09-15T01:34:07.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 1 -- 172.21.15.204:0/838511988 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f270c09fb80 msgr2=0x7f270c09ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:07.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 1 --2- 172.21.15.204:0/838511988 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f270c09fb80 0x7f270c09ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:07.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 1 -- 172.21.15.204:0/838511988 >> v2:172.21.15.204:3300/0 conn(0x7f270c0a0a40 msgr2=0x7f270c0a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:07.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 1 --2- 172.21.15.204:0/838511988 >> v2:172.21.15.204:3300/0 conn(0x7f270c0a0a40 0x7f270c0a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:07.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:07.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 1 monclient: found mon.noname-h 2024-09-15T01:34:07.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:07.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:07.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270a59c640 1 -- 172.21.15.204:0/838511988 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c04b720 con 0x7f270c0a2e30 2024-09-15T01:34:07.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f270ad9d640 1 --2- 172.21.15.204:0/838511988 >> v2:172.21.15.204:3300/0 conn(0x7f270c0a0a40 0x7f270c0a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:07.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.007+0000 7f2709d9b640 1 --2- 172.21.15.204:0/838511988 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f270c09fb80 0x7f270c09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:07.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270a59c640 1 --2- 172.21.15.204:0/838511988 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f270c0a2e30 0x7f270c132a60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009850 tx=0x7f26fc0090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=2c1b79544f07bed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:07.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270959a640 1 -- 172.21.15.204:0/838511988 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f26fc02abc0 con 0x7f270c0a2e30 2024-09-15T01:34:07.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:07.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270959a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:07.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270959a640 10 monclient: dump: 2024-09-15T01:34:07.030 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:07.030 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:07.030 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:07.030 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:07.031 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:07.031 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:07.031 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:07.031 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:07.031 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:07.031 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:07.031 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:07.032 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:07.032 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:07.032 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:07.032 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:07.032 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:07.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270959a640 10 monclient: _finish_auth 0 2024-09-15T01:34:07.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:07.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270959a640 1 -- 172.21.15.204:0/838511988 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26fc02ad40 con 0x7f270c0a2e30 2024-09-15T01:34:07.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f27108c0640 5 monclient: authenticate success, global_id 4991 2024-09-15T01:34:07.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f27108c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:07.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:07.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f27108c0640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:07.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270959a640 1 -- 172.21.15.204:0/838511988 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f26fc031ca0 con 0x7f270c0a2e30 2024-09-15T01:34:07.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f27108c0640 4 set_mon_vals no callback set 2024-09-15T01:34:07.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:07.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270959a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:07.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.008+0000 7f270959a640 10 monclient: dump: 2024-09-15T01:34:07.034 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:07.035 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:07.035 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:07.035 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:07.035 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:07.035 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:07.035 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:07.035 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:07.036 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:07.036 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:07.036 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:07.036 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:07.036 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:07.036 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:07.036 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:07.037 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:07.037 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 10 monclient: shutdown 2024-09-15T01:34:07.037 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:07.037 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 1 -- 172.21.15.204:0/838511988 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f270c0a2e30 msgr2=0x7f270c132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:07.037 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 1 --2- 172.21.15.204:0/838511988 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f270c0a2e30 0x7f270c132a60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009850 tx=0x7f26fc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:07.037 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 1 -- 172.21.15.204:0/838511988 shutdown_connections 2024-09-15T01:34:07.037 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 1 --2- 172.21.15.204:0/838511988 >> v2:172.21.15.204:3300/0 conn(0x7f270c0a0a40 0x7f270c0a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:07.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 1 --2- 172.21.15.204:0/838511988 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f270c09fb80 0x7f270c09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:07.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 1 --2- 172.21.15.204:0/838511988 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f270c0a2e30 0x7f270c132a60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:07.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 1 -- 172.21.15.204:0/838511988 >> 172.21.15.204:0/838511988 conn(0x7f270c06e860 msgr2=0x7f270c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:07.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 1 -- 172.21.15.204:0/838511988 shutdown_connections 2024-09-15T01:34:07.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 1 -- 172.21.15.204:0/838511988 wait complete. 2024-09-15T01:34:07.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 10 monclient: build_initial_monmap 2024-09-15T01:34:07.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.009+0000 7f27108c0640 10 monclient: monmap: 2024-09-15T01:34:07.039 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:07.039 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:07.039 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:07.039 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:07.039 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:07.039 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:07.039 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:07.039 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:07.040 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:07.040 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:07.040 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:07.040 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:07.040 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:07.040 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:07.040 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:07.040 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:07.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 1 Processor -- start 2024-09-15T01:34:07.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 1 -- start start 2024-09-15T01:34:07.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 10 monclient: init 2024-09-15T01:34:07.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:07.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 10 monclient: _add_conns ranks=[0,1,3,8,7,5,6,4,2] 2024-09-15T01:34:07.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f270c09fb80 0x7f270c130670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:07.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 10 monclient(hunting): picked mon.noname-a con 0x7f270c09fb80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:07.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f270c0a0a40 0x7f270c11d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:07.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 10 monclient(hunting): picked mon.noname-f con 0x7f270c0a0a40 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:07.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f270c0a2e30 0x7f270c11d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:07.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 10 monclient(hunting): picked mon.noname-g con 0x7f270c0a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:07.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:07.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c09ca40 con 0x7f270c09fb80 2024-09-15T01:34:07.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:07.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c09d650 con 0x7f270c0a0a40 2024-09-15T01:34:07.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:07.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270c1207c0 con 0x7f270c0a2e30 2024-09-15T01:34:07.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:07.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f27108c0640 10 monclient(hunting): authenticate will time out at 862.653592s 2024-09-15T01:34:07.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f270ad9d640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f270c0a2e30 msgr2=0x7f270c11d6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:34:07.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f270ad9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f270c0a2e30 0x7f270c11d6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:07.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f270c0a0a40 0x7f270c11d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:07.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f270c0a0a40 0x7f270c11d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38208/0 (socket says 172.21.15.204:38208) 2024-09-15T01:34:07.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 1 -- 172.21.15.204:0/2365523065 learned_addr learned my addr 172.21.15.204:0/2365523065 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:07.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 10 monclient(hunting): get_auth_request con 0x7f270c0a0a40 auth_method 0 2024-09-15T01:34:07.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:07.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:07.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:07.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:07.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:07.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:07.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 10 monclient(hunting): handle_auth_done global_id 5446 payload 824 2024-09-15T01:34:07.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 1 -- 172.21.15.204:0/2365523065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f270c0a2e30 msgr2=0x7f270c11d6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:07.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 1 --2- 172.21.15.204:0/2365523065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f270c0a2e30 0x7f270c11d6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:07.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 1 -- 172.21.15.204:0/2365523065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f270c09fb80 msgr2=0x7f270c130670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:07.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 1 --2- 172.21.15.204:0/2365523065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f270c09fb80 0x7f270c130670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:07.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:34:07.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 1 monclient: found mon.noname-f 2024-09-15T01:34:07.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:07.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:07.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.010+0000 7f2709d9b640 1 -- 172.21.15.204:0/2365523065 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2700009390 con 0x7f270c0a0a40 2024-09-15T01:34:07.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f2709d9b640 1 --2- 172.21.15.204:0/2365523065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f270c0a0a40 0x7f270c11d1a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f270000ec50 tx=0x7f27000090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:07.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f26fa7fc640 1 -- 172.21.15.204:0/2365523065 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f270000ee90 con 0x7f270c0a0a40 2024-09-15T01:34:07.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f26fa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:07.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f26fa7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:07.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f26fa7fc640 10 monclient: dump: 2024-09-15T01:34:07.048 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:07.048 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:07.049 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:07.049 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:07.049 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:07.049 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:07.049 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:07.049 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:07.050 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:07.050 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:07.050 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:07.050 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:07.050 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:07.050 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:07.050 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:07.051 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:07.051 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f26fa7fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:07.051 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f26fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:07.051 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f27108c0640 5 monclient: authenticate success, global_id 5446 2024-09-15T01:34:07.051 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f26fa7fc640 1 -- 172.21.15.204:0/2365523065 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270000cbc0 con 0x7f270c0a0a40 2024-09-15T01:34:07.051 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f27108c0640 10 monclient: _renew_subs 2024-09-15T01:34:07.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f27108c0640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:07.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f27108c0640 1 -- 172.21.15.204:0/2365523065 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2704009ae0 con 0x7f270c0a0a40 2024-09-15T01:34:07.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f26fa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:07.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f26fa7fc640 1 -- 172.21.15.204:0/2365523065 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2700013d70 con 0x7f270c0a0a40 2024-09-15T01:34:07.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f26fa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:07.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f26fa7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:07.053 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f2708d99640 4 set_mon_vals no callback set 2024-09-15T01:34:07.053 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f26fa7fc640 10 monclient: dump: 2024-09-15T01:34:07.053 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:07.053 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:07.053 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:07.053 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:07.054 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:07.054 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:07.054 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:07.054 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:07.054 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:07.055 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:07.055 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:07.055 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:07.055 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:07.055 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:07.055 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:07.056 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:07.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f27108c0640 10 monclient: _renew_subs 2024-09-15T01:34:07.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f27108c0640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:07.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.011+0000 7f27108c0640 1 -- 172.21.15.204:0/2365523065 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc009390 con 0x7f270c0a0a40 2024-09-15T01:34:07.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.012+0000 7f27108c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:07.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.012+0000 7f27108c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:07.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.012+0000 7f27108c0640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:07.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.012+0000 7f26fa7fc640 1 -- 172.21.15.204:0/2365523065 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f270000cd30 con 0x7f270c0a0a40 2024-09-15T01:34:07.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.012+0000 7f27108c0640 1 -- 172.21.15.204:0/2365523065 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270c070cc0 con 0x7f270c0a0a40 2024-09-15T01:34:07.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.012+0000 7f26fa7fc640 1 --2- 172.21.15.204:0/2365523065 >> v2:172.21.15.204:6812/28210 conn(0x7f26ec0397b0 0x7f26ec03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:07.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.012+0000 7f26fa7fc640 1 -- 172.21.15.204:0/2365523065 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5231+0+0 (secure 0 0 0) 0x7f270004b9c0 con 0x7f270c0a0a40 2024-09-15T01:34:07.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.012+0000 7f270a59c640 1 --2- 172.21.15.204:0/2365523065 >> v2:172.21.15.204:6812/28210 conn(0x7f26ec0397b0 0x7f26ec03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:07.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.012+0000 7f270a59c640 10 monclient: get_auth_request con 0x7f26ec0397b0 auth_method 0 2024-09-15T01:34:07.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.013+0000 7f270a59c640 1 --2- 172.21.15.204:0/2365523065 >> v2:172.21.15.204:6812/28210 conn(0x7f26ec0397b0 0x7f26ec03bc60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f26fc034000 tx=0x7f26fc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:07.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.016+0000 7f26fa7fc640 1 -- 172.21.15.204:0/2365523065 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f27000238f0 con 0x7f270c0a0a40 2024-09-15T01:34:07.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.016+0000 7f26fa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:07.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.016+0000 7f26fa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:07.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.172+0000 7f27108c0640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T01:34:07.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.172+0000 7f27108c0640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T01:34:07.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.172+0000 7f27108c0640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:07.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:07.172+0000 7f27108c0640 1 -- 172.21.15.204:0/2365523065 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) v1 -- 0x7f270c11ec90 con 0x7f270c0a0a40 2024-09-15T01:34:07.434 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-15T01:34:07.434 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.d 2024-09-15T01:34:07.435 INFO:tasks.ceph.mon.d:Restarting daemon 2024-09-15T01:34:07.435 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-09-15T01:34:07.438 INFO:tasks.ceph.mon.d:Started 2024-09-15T01:34:07.438 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T01:34:07.439 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-15T01:34:07.439 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-15T01:34:07.439 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-15T01:34:07.441 INFO:tasks.ceph.mon.g:Started 2024-09-15T01:34:07.441 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-15T01:34:07.442 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-15T01:34:07.442 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T01:34:07.520 INFO:tasks.ceph.mon.g.smithi204.stderr:2024-09-15T01:34:07.517+0000 7f3c04129900 -1 mon.g@-1(???).log v0 log_external_backlog rewinding external_log_to from 24 -> 0 (sync_force? mon rebuild?) 2024-09-15T01:34:07.527 INFO:tasks.ceph.mon.d.smithi178.stderr:2024-09-15T01:34:07.526+0000 7fb276a49900 -1 mon.d@-1(???).log v0 log_external_backlog rewinding external_log_to from 24 -> 0 (sync_force? mon rebuild?) 2024-09-15T01:34:10.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:10.011+0000 7f26f97fa640 10 monclient: tick 2024-09-15T01:34:10.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:10.011+0000 7f26f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:10.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:10.011+0000 7f26f97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:11.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:11.010+0000 7f26f97fa640 10 monclient: tick 2024-09-15T01:34:11.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:11.010+0000 7f26f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:12.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:12.011+0000 7f26f97fa640 10 monclient: tick 2024-09-15T01:34:12.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:12.011+0000 7f26f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:13.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:13.011+0000 7f26f97fa640 10 monclient: tick 2024-09-15T01:34:13.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:13.011+0000 7f26f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:14.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:14.012+0000 7f26f97fa640 10 monclient: tick 2024-09-15T01:34:14.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:14.012+0000 7f26f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:15.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:15.012+0000 7f26f97fa640 10 monclient: tick 2024-09-15T01:34:15.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:15.012+0000 7f26f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:16.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:16.012+0000 7f26f97fa640 10 monclient: tick 2024-09-15T01:34:16.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:16.012+0000 7f26f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:17.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.012+0000 7f26f97fa640 10 monclient: tick 2024-09-15T01:34:17.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.012+0000 7f26f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:17.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.821+0000 7f26fa7fc640 1 -- 172.21.15.204:0/2365523065 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 linked item id -11 name 'host1' to location {root=bar} in crush map v76) v1 ==== 168+0+0 (secure 0 0 0) 0x7f2700021030 con 0x7f270c0a0a40 2024-09-15T01:34:17.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.822+0000 7f26fa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T01:34:17.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.822+0000 7f26fa7fc640 10 monclient: _finish_command 2 = system:0 linked item id -11 name 'host1' to location {root=bar} in crush map 2024-09-15T01:34:17.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.919+0000 7f27108c0640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T01:34:17.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.919+0000 7f27108c0640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T01:34:17.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.919+0000 7f27108c0640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:17.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.919+0000 7f27108c0640 1 -- 172.21.15.204:0/2365523065 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) v1 -- 0x7f270c11f860 con 0x7f270c0a0a40 2024-09-15T01:34:17.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.920+0000 7f26fa7fc640 1 -- 172.21.15.204:0/2365523065 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 no need to move item id -11 name 'host1' to location {root=bar} in crush map v76) v1 ==== 177+0+0 (secure 0 0 0) 0x7f2700021020 con 0x7f270c0a0a40 2024-09-15T01:34:17.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.920+0000 7f26fa7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T01:34:17.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.920+0000 7f26fa7fc640 10 monclient: _finish_command 3 = system:0 no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-09-15T01:34:17.923 INFO:tasks.workunit.client.0.smithi204.stderr:no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-09-15T01:34:17.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 1 -- 172.21.15.204:0/2365523065 >> v2:172.21.15.204:6812/28210 conn(0x7f26ec0397b0 msgr2=0x7f26ec03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:17.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 1 --2- 172.21.15.204:0/2365523065 >> v2:172.21.15.204:6812/28210 conn(0x7f26ec0397b0 0x7f26ec03bc60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f26fc034000 tx=0x7f26fc032000 comp rx=0 tx=0).stop 2024-09-15T01:34:17.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 10 monclient: shutdown 2024-09-15T01:34:17.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:17.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 1 -- 172.21.15.204:0/2365523065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f270c0a0a40 msgr2=0x7f270c11d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:17.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 1 --2- 172.21.15.204:0/2365523065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f270c0a0a40 0x7f270c11d1a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f270000ec50 tx=0x7f27000090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:17.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 1 -- 172.21.15.204:0/2365523065 shutdown_connections 2024-09-15T01:34:17.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 1 --2- 172.21.15.204:0/2365523065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f270c0a2e30 0x7f270c11d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:17.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 1 --2- 172.21.15.204:0/2365523065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f270c0a0a40 0x7f270c11d1a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:17.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 1 --2- 172.21.15.204:0/2365523065 >> v2:172.21.15.204:6812/28210 conn(0x7f26ec0397b0 0x7f26ec03bc60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:17.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 1 --2- 172.21.15.204:0/2365523065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f270c09fb80 0x7f270c130670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:17.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 1 -- 172.21.15.204:0/2365523065 >> 172.21.15.204:0/2365523065 conn(0x7f270c06e860 msgr2=0x7f270c09ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:17.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 1 -- 172.21.15.204:0/2365523065 shutdown_connections 2024-09-15T01:34:17.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:17.922+0000 7f27108c0640 1 -- 172.21.15.204:0/2365523065 wait complete. 2024-09-15T01:34:17.933 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush link host1 root=bar 2024-09-15T01:34:18.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.005+0000 7f338859d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:18.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.005+0000 7f338859d640 10 monclient: build_initial_monmap 2024-09-15T01:34:18.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.005+0000 7f338859d640 10 monclient: monmap: 2024-09-15T01:34:18.008 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:18.007437+0000 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:18.007437+0000 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:18.009 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:18.010 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:18.010 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:18.010 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:18.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.006+0000 7f338859d640 1 Processor -- start 2024-09-15T01:34:18.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.006+0000 7f338859d640 1 -- start start 2024-09-15T01:34:18.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.006+0000 7f338859d640 10 monclient: init 2024-09-15T01:34:18.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:18.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 10 monclient: _add_conns ranks=[4,6,7,1,3,5,2,9,10,8,0] 2024-09-15T01:34:18.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33800a2e30 0x7f3380132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 10 monclient(hunting): picked mon.noname-c con 0x7f33800a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:18.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f338009fb80 0x7f338009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 10 monclient(hunting): picked mon.noname-d con 0x7f338009fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:18.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f33800a0a40 0x7f33800a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 10 monclient(hunting): picked mon.noname-i con 0x7f33800a0a40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:18.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:18.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f338009e6c0 con 0x7f33800a2e30 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f33800a04b0 con 0x7f338009fb80 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f33800a3200 con 0x7f33800a0a40 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f338859d640 10 monclient(hunting): authenticate will time out at 873.650381s 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f33800a0a40 0x7f33800a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f33800a0a40 0x7f33800a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:34050/0 (socket says 172.21.15.204:34050) 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 1 -- 172.21.15.204:0/2064654449 learned_addr learned my addr 172.21.15.204:0/2064654449 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 10 monclient(hunting): get_auth_request con 0x7f33800a0a40 auth_method 0 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:18.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3385b38640 1 --2- 172.21.15.204:0/2064654449 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f338009fb80 0x7f338009ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386339640 1 --2- 172.21.15.204:0/2064654449 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33800a2e30 0x7f3380132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386339640 10 monclient(hunting): get_auth_request con 0x7f33800a2e30 auth_method 0 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3385b38640 10 monclient(hunting): get_auth_request con 0x7f338009fb80 auth_method 0 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3385b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3385b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:18.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3385b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 10 monclient(hunting): handle_auth_done global_id 5065 payload 274 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 1 -- 172.21.15.204:0/2064654449 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f338009fb80 msgr2=0x7f338009ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 1 --2- 172.21.15.204:0/2064654449 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f338009fb80 0x7f338009ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 1 -- 172.21.15.204:0/2064654449 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33800a2e30 msgr2=0x7f3380132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 1 --2- 172.21.15.204:0/2064654449 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33800a2e30 0x7f3380132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 10 monclient: _finish_hunting 0 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 1 monclient: found mon.noname-i 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.007+0000 7f3386b3a640 1 -- 172.21.15.204:0/2064654449 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338004b720 con 0x7f33800a0a40 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3386b3a640 1 --2- 172.21.15.204:0/2064654449 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f33800a0a40 0x7f33800a0e90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3378009f50 tx=0x7f3378009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=4d94f658271f939d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3385337640 1 -- 172.21.15.204:0/2064654449 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3378007880 con 0x7f33800a0a40 2024-09-15T01:34:18.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3385337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3385337640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3385337640 10 monclient: dump: 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:18.022 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3385337640 10 monclient: _finish_auth 0 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3385337640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3385337640 1 -- 172.21.15.204:0/2064654449 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337802d050 con 0x7f33800a0a40 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f338859d640 5 monclient: authenticate success, global_id 5065 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f338859d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3385337640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f338859d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3385337640 1 -- 172.21.15.204:0/2064654449 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3378030360 con 0x7f33800a0a40 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3385337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:18.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f338859d640 4 set_mon_vals no callback set 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3385337640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.008+0000 7f3385337640 10 monclient: dump: 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:18.024 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 10 monclient: shutdown 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 1 -- 172.21.15.204:0/2064654449 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f33800a0a40 msgr2=0x7f33800a0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 1 --2- 172.21.15.204:0/2064654449 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f33800a0a40 0x7f33800a0e90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3378009f50 tx=0x7f3378009840 comp rx=0 tx=0).stop 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 1 -- 172.21.15.204:0/2064654449 shutdown_connections 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 1 --2- 172.21.15.204:0/2064654449 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f33800a0a40 0x7f33800a0e90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 1 --2- 172.21.15.204:0/2064654449 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f338009fb80 0x7f338009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 1 --2- 172.21.15.204:0/2064654449 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33800a2e30 0x7f3380132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 1 -- 172.21.15.204:0/2064654449 >> 172.21.15.204:0/2064654449 conn(0x7f338006e860 msgr2=0x7f338006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:18.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 1 -- 172.21.15.204:0/2064654449 shutdown_connections 2024-09-15T01:34:18.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 1 -- 172.21.15.204:0/2064654449 wait complete. 2024-09-15T01:34:18.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 10 monclient: build_initial_monmap 2024-09-15T01:34:18.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.009+0000 7f338859d640 10 monclient: monmap: 2024-09-15T01:34:18.026 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:18.026 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:18.026 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:18.026 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:18.026 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:18.026 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:18.026 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:18.026 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 1 Processor -- start 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 1 -- start start 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 10 monclient: init 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 10 monclient: _add_conns ranks=[4,5,1,0,2,3,7,8,6] 2024-09-15T01:34:18.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f338009fb80 0x7f3380131850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 10 monclient(hunting): picked mon.noname-c con 0x7f338009fb80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:18.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33800a0a40 0x7f338011e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 10 monclient(hunting): picked mon.noname-h con 0x7f33800a0a40 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:18.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33800a2e30 0x7f338011e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 10 monclient(hunting): picked mon.noname-f con 0x7f33800a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:18.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:18.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33801429f0 con 0x7f33800a2e30 2024-09-15T01:34:18.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:18.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f338009cab0 con 0x7f338009fb80 2024-09-15T01:34:18.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:18.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33801219a0 con 0x7f33800a0a40 2024-09-15T01:34:18.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f338859d640 10 monclient(hunting): authenticate will time out at 873.653509s 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3385b38640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33800a0a40 0x7f338011e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33800a2e30 0x7f338011e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33800a2e30 0x7f338011e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48870/0 (socket says 172.21.15.204:48870) 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 1 -- 172.21.15.204:0/1340935757 learned_addr learned my addr 172.21.15.204:0/1340935757 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3385b38640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33800a0a40 0x7f338011e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:55686/0 (socket says 172.21.15.204:55686) 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 10 monclient(hunting): get_auth_request con 0x7f33800a2e30 auth_method 0 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3385b38640 10 monclient(hunting): get_auth_request con 0x7f33800a0a40 auth_method 0 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386339640 1 --2- 172.21.15.204:0/1340935757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f338009fb80 0x7f3380131850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:18.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3385b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3385b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3385b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386339640 10 monclient(hunting): get_auth_request con 0x7f338009fb80 auth_method 0 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3385b38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3385b38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3385b38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:18.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 10 monclient(hunting): handle_auth_done global_id 5455 payload 824 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 1 -- 172.21.15.204:0/1340935757 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33800a0a40 msgr2=0x7f338011e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 1 --2- 172.21.15.204:0/1340935757 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33800a0a40 0x7f338011e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 1 -- 172.21.15.204:0/1340935757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f338009fb80 msgr2=0x7f3380131850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 1 --2- 172.21.15.204:0/1340935757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f338009fb80 0x7f3380131850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 10 monclient: _finish_hunting 0 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 1 monclient: found mon.noname-f 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.010+0000 7f3386b3a640 1 -- 172.21.15.204:0/1340935757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3378009ae0 con 0x7f33800a2e30 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f3386b3a640 1 --2- 172.21.15.204:0/1340935757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33800a2e30 0x7f338011e840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3378009df0 tx=0x7f3378036000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:18.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f33767fc640 1 -- 172.21.15.204:0/1340935757 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3378007170 con 0x7f33800a2e30 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f33767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f33767fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f33767fc640 10 monclient: dump: 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:18.032 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f33767fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f33767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f338859d640 5 monclient: authenticate success, global_id 5455 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f33767fc640 1 -- 172.21.15.204:0/1340935757 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337803baa0 con 0x7f33800a2e30 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f338859d640 10 monclient: _renew_subs 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f338859d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f338859d640 1 -- 172.21.15.204:0/1340935757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33801325f0 con 0x7f33800a2e30 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f33767fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:18.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f33767fc640 1 -- 172.21.15.204:0/1340935757 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f337803bc10 con 0x7f33800a2e30 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f33767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f33767fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f3384b36640 4 set_mon_vals no callback set 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f33767fc640 10 monclient: dump: 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:18.034 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f338859d640 10 monclient: _renew_subs 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f338859d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.011+0000 7f338859d640 1 -- 172.21.15.204:0/1340935757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3380120ab0 con 0x7f33800a2e30 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.012+0000 7f33767fc640 1 -- 172.21.15.204:0/1340935757 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3378038070 con 0x7f33800a2e30 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.012+0000 7f338859d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.012+0000 7f338859d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:18.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.012+0000 7f338859d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:18.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.012+0000 7f338859d640 1 -- 172.21.15.204:0/1340935757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3380070cc0 con 0x7f33800a2e30 2024-09-15T01:34:18.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.012+0000 7f33767fc640 1 --2- 172.21.15.204:0/1340935757 >> v2:172.21.15.204:6812/28210 conn(0x7f33640397b0 0x7f336403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.012+0000 7f33767fc640 1 -- 172.21.15.204:0/1340935757 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f337806b9c0 con 0x7f33800a2e30 2024-09-15T01:34:18.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.012+0000 7f3386339640 1 --2- 172.21.15.204:0/1340935757 >> v2:172.21.15.204:6812/28210 conn(0x7f33640397b0 0x7f336403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:18.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.012+0000 7f3386339640 10 monclient: get_auth_request con 0x7f33640397b0 auth_method 0 2024-09-15T01:34:18.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.012+0000 7f3386339640 1 --2- 172.21.15.204:0/1340935757 >> v2:172.21.15.204:6812/28210 conn(0x7f33640397b0 0x7f336403bc60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f337c00bd40 tx=0x7f337c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:18.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.016+0000 7f33767fc640 1 -- 172.21.15.204:0/1340935757 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f3378039ae0 con 0x7f33800a2e30 2024-09-15T01:34:18.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.016+0000 7f33767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:18.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.016+0000 7f33767fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:18.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.175+0000 7f338859d640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T01:34:18.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.176+0000 7f338859d640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T01:34:18.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.176+0000 7f338859d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:18.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.176+0000 7f338859d640 1 -- 172.21.15.204:0/1340935757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) v1 -- 0x7f33800a1f10 con 0x7f33800a2e30 2024-09-15T01:34:18.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.176+0000 7f33767fc640 1 -- 172.21.15.204:0/1340935757 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 no need to move item id -11 name 'host1' to location {root=bar} in crush map v76) v1 ==== 177+0+0 (secure 0 0 0) 0x7f337803f020 con 0x7f33800a2e30 2024-09-15T01:34:18.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.176+0000 7f33767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T01:34:18.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.176+0000 7f33767fc640 10 monclient: _finish_command 2 = system:0 no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-09-15T01:34:18.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.274+0000 7f338859d640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T01:34:18.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.274+0000 7f338859d640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T01:34:18.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.274+0000 7f338859d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:18.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.274+0000 7f338859d640 1 -- 172.21.15.204:0/1340935757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) v1 -- 0x7f3380066a70 con 0x7f33800a2e30 2024-09-15T01:34:18.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.275+0000 7f33767fc640 1 -- 172.21.15.204:0/1340935757 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 no need to move item id -11 name 'host1' to location {root=bar} in crush map v76) v1 ==== 177+0+0 (secure 0 0 0) 0x7f3378039300 con 0x7f33800a2e30 2024-09-15T01:34:18.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.275+0000 7f33767fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-09-15T01:34:18.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.275+0000 7f33767fc640 10 monclient: _finish_command 3 = system:0 no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-09-15T01:34:18.278 INFO:tasks.workunit.client.0.smithi204.stderr:no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-09-15T01:34:18.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 1 -- 172.21.15.204:0/1340935757 >> v2:172.21.15.204:6812/28210 conn(0x7f33640397b0 msgr2=0x7f336403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:18.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 1 --2- 172.21.15.204:0/1340935757 >> v2:172.21.15.204:6812/28210 conn(0x7f33640397b0 0x7f336403bc60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f337c00bd40 tx=0x7f337c008040 comp rx=0 tx=0).stop 2024-09-15T01:34:18.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 10 monclient: shutdown 2024-09-15T01:34:18.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:18.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 1 -- 172.21.15.204:0/1340935757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33800a2e30 msgr2=0x7f338011e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:18.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 1 --2- 172.21.15.204:0/1340935757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33800a2e30 0x7f338011e840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3378009df0 tx=0x7f3378036000 comp rx=0 tx=0).stop 2024-09-15T01:34:18.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 1 -- 172.21.15.204:0/1340935757 shutdown_connections 2024-09-15T01:34:18.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 1 --2- 172.21.15.204:0/1340935757 >> v2:172.21.15.204:6812/28210 conn(0x7f33640397b0 0x7f336403bc60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 1 --2- 172.21.15.204:0/1340935757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33800a2e30 0x7f338011e840 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 1 --2- 172.21.15.204:0/1340935757 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33800a0a40 0x7f338011e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 1 --2- 172.21.15.204:0/1340935757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f338009fb80 0x7f3380131850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 1 -- 172.21.15.204:0/1340935757 >> 172.21.15.204:0/1340935757 conn(0x7f338006e860 msgr2=0x7f33800a5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:18.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 1 -- 172.21.15.204:0/1340935757 shutdown_connections 2024-09-15T01:34:18.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.277+0000 7f338859d640 1 -- 172.21.15.204:0/1340935757 wait complete. 2024-09-15T01:34:18.288 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush link host2 root=bar 2024-09-15T01:34:18.363 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.361+0000 7f4682eef640 10 monclient: get_monmap_and_config 2024-09-15T01:34:18.363 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.361+0000 7f4682eef640 10 monclient: build_initial_monmap 2024-09-15T01:34:18.363 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.361+0000 7f4682eef640 10 monclient: monmap: 2024-09-15T01:34:18.363 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:18.362534+0000 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:18.362534+0000 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:18.364 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:18.365 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:18.365 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:18.365 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:18.365 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:18.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.361+0000 7f4682eef640 1 Processor -- start 2024-09-15T01:34:18.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.361+0000 7f4682eef640 1 -- start start 2024-09-15T01:34:18.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.361+0000 7f4682eef640 10 monclient: init 2024-09-15T01:34:18.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:18.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 10 monclient: _add_conns ranks=[6,5,8,10,3,0,7,9,2,1,4] 2024-09-15T01:34:18.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f467c0a4680 0x7f467c09e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 10 monclient(hunting): picked mon.noname-d con 0x7f467c0a4680 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f467c06eea0 0x7f467c09ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 10 monclient(hunting): picked mon.noname-h con 0x7f467c06eea0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f467c09f2d0 0x7f467c1308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 10 monclient(hunting): picked mon.noname-e con 0x7f467c09f2d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f467c09f6e0 con 0x7f467c06eea0 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0a4a50 con 0x7f467c0a4680 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f467c06dc70 con 0x7f467c09f2d0 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4682eef640 10 monclient(hunting): authenticate will time out at 874.005365s 2024-09-15T01:34:18.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f467c06eea0 0x7f467c09ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:18.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f467c06eea0 0x7f467c09ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:55692/0 (socket says 172.21.15.204:55692) 2024-09-15T01:34:18.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 1 -- 172.21.15.204:0/2495859200 learned_addr learned my addr 172.21.15.204:0/2495859200 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:18.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 10 monclient(hunting): get_auth_request con 0x7f467c06eea0 auth_method 0 2024-09-15T01:34:18.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:18.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:18.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:18.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:18.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:18.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:18.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 10 monclient(hunting): handle_auth_done global_id 5009 payload 274 2024-09-15T01:34:18.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 1 -- 172.21.15.204:0/2495859200 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f467c09f2d0 msgr2=0x7f467c1308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:18.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 1 --2- 172.21.15.204:0/2495859200 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f467c09f2d0 0x7f467c1308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 1 -- 172.21.15.204:0/2495859200 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f467c0a4680 msgr2=0x7f467c09e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f4680c8b640 1 --2- 172.21.15.204:0/2495859200 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f467c0a4680 0x7f467c09e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 1 --2- 172.21.15.204:0/2495859200 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f467c0a4680 0x7f467c09e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 1 monclient: found mon.noname-h 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.362+0000 7f467bfff640 1 -- 172.21.15.204:0/2495859200 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c04b720 con 0x7f467c06eea0 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467bfff640 1 --2- 172.21.15.204:0/2495859200 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f467c06eea0 0x7f467c09ed90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4670009970 tx=0x7f46700091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d7be7368521f437f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467b7fe640 1 -- 172.21.15.204:0/2495859200 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f467002ac50 con 0x7f467c06eea0 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467b7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:18.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467b7fe640 10 monclient: dump: 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:18.376 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467b7fe640 1 -- 172.21.15.204:0/2495859200 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467002add0 con 0x7f467c06eea0 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f4682eef640 5 monclient: authenticate success, global_id 5009 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f4682eef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467b7fe640 1 -- 172.21.15.204:0/2495859200 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4670031ca0 con 0x7f467c06eea0 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f4682eef640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f4682eef640 4 set_mon_vals no callback set 2024-09-15T01:34:18.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.363+0000 7f467b7fe640 10 monclient: dump: 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:18.378 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 10 monclient: shutdown 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 1 -- 172.21.15.204:0/2495859200 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f467c06eea0 msgr2=0x7f467c09ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 1 --2- 172.21.15.204:0/2495859200 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f467c06eea0 0x7f467c09ed90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4670009970 tx=0x7f46700091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 1 -- 172.21.15.204:0/2495859200 shutdown_connections 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 1 --2- 172.21.15.204:0/2495859200 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f467c09f2d0 0x7f467c1308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 1 --2- 172.21.15.204:0/2495859200 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f467c06eea0 0x7f467c09ed90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 1 --2- 172.21.15.204:0/2495859200 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f467c0a4680 0x7f467c09e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 1 -- 172.21.15.204:0/2495859200 >> 172.21.15.204:0/2495859200 conn(0x7f467c067740 msgr2=0x7f467c0a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 1 -- 172.21.15.204:0/2495859200 shutdown_connections 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 1 -- 172.21.15.204:0/2495859200 wait complete. 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 10 monclient: build_initial_monmap 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.364+0000 7f4682eef640 10 monclient: monmap: 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:18.380 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 1 Processor -- start 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 1 -- start start 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 10 monclient: init 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 10 monclient: _add_conns ranks=[0,2,1,7,4,3,5,8,6] 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c06eea0 0x7f467c13c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 10 monclient(hunting): picked mon.noname-a con 0x7f467c06eea0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f467c09f2d0 0x7f467c13bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 10 monclient(hunting): picked mon.noname-b con 0x7f467c09f2d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f467c0a4680 0x7f467c13fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 10 monclient(hunting): picked mon.noname-f con 0x7f467c0a4680 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c11f220 con 0x7f467c06eea0 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c120ec0 con 0x7f467c0a4680 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c120be0 con 0x7f467c09f2d0 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4682eef640 10 monclient(hunting): authenticate will time out at 874.008796s 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f468148c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f467c0a4680 0x7f467c13fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:18.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4680c8b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c06eea0 0x7f467c13c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f468148c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f467c0a4680 0x7f467c13fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48886/0 (socket says 172.21.15.204:48886) 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f468148c640 1 -- 172.21.15.204:0/3131519141 learned_addr learned my addr 172.21.15.204:0/3131519141 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f468148c640 10 monclient(hunting): get_auth_request con 0x7f467c0a4680 auth_method 0 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f468148c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f468148c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f468148c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4680c8b640 10 monclient(hunting): get_auth_request con 0x7f467c06eea0 auth_method 0 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4680c8b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4680c8b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.365+0000 7f4680c8b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:18.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4680c8b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4680c8b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4680c8b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 10 monclient(hunting): handle_auth_done global_id 5464 payload 824 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 1 -- 172.21.15.204:0/3131519141 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f467c09f2d0 msgr2=0x7f467c13bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 1 --2- 172.21.15.204:0/3131519141 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f467c09f2d0 0x7f467c13bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 1 -- 172.21.15.204:0/3131519141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c06eea0 msgr2=0x7f467c13c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 1 --2- 172.21.15.204:0/3131519141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c06eea0 0x7f467c13c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 1 monclient: found mon.noname-f 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 1 -- 172.21.15.204:0/3131519141 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4670009450 con 0x7f467c0a4680 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f468148c640 1 --2- 172.21.15.204:0/3131519141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f467c0a4680 0x7f467c13fa00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f467400bb90 tx=0x7f4674009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:18.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4678ff9640 1 -- 172.21.15.204:0/3131519141 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4674014410 con 0x7f467c0a4680 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4678ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4678ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4678ff9640 10 monclient: dump: 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:18.385 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4678ff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4678ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4682eef640 5 monclient: authenticate success, global_id 5464 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4678ff9640 1 -- 172.21.15.204:0/3131519141 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46740145b0 con 0x7f467c0a4680 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4682eef640 10 monclient: _renew_subs 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4682eef640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4682eef640 1 -- 172.21.15.204:0/3131519141 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c1211a0 con 0x7f467c0a4680 2024-09-15T01:34:18.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4678ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4678ff9640 1 -- 172.21.15.204:0/3131519141 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4674014ba0 con 0x7f467c0a4680 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4678ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4678ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4678ff9640 10 monclient: dump: 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:18.387 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4682eef640 10 monclient: _renew_subs 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4682eef640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f4682eef640 1 -- 172.21.15.204:0/3131519141 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c13c080 con 0x7f467c0a4680 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.366+0000 7f467affd640 4 set_mon_vals no callback set 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.367+0000 7f4678ff9640 1 -- 172.21.15.204:0/3131519141 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4674014d20 con 0x7f467c0a4680 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.367+0000 7f4682eef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:18.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.367+0000 7f4682eef640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:18.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.367+0000 7f4682eef640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:18.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.367+0000 7f4682eef640 1 -- 172.21.15.204:0/3131519141 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c11eaf0 con 0x7f467c0a4680 2024-09-15T01:34:18.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.367+0000 7f4678ff9640 1 --2- 172.21.15.204:0/3131519141 >> v2:172.21.15.204:6812/28210 conn(0x7f46680397b0 0x7f466803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:18.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.367+0000 7f4678ff9640 1 -- 172.21.15.204:0/3131519141 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f467404b770 con 0x7f467c0a4680 2024-09-15T01:34:18.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.367+0000 7f4680c8b640 1 --2- 172.21.15.204:0/3131519141 >> v2:172.21.15.204:6812/28210 conn(0x7f46680397b0 0x7f466803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:18.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.367+0000 7f4680c8b640 10 monclient: get_auth_request con 0x7f46680397b0 auth_method 0 2024-09-15T01:34:18.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.368+0000 7f4680c8b640 1 --2- 172.21.15.204:0/3131519141 >> v2:172.21.15.204:6812/28210 conn(0x7f46680397b0 0x7f466803bc60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f466c006fd0 tx=0x7f466c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:18.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.372+0000 7f4678ff9640 1 -- 172.21.15.204:0/3131519141 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f46740073d0 con 0x7f467c0a4680 2024-09-15T01:34:18.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.372+0000 7f4678ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:18.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.372+0000 7f4678ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:18.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.531+0000 7f4682eef640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T01:34:18.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.531+0000 7f4682eef640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T01:34:18.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.531+0000 7f4682eef640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:18.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.531+0000 7f4682eef640 1 -- 172.21.15.204:0/3131519141 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) v1 -- 0x7f467c070cc0 con 0x7f467c0a4680 2024-09-15T01:34:18.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.830+0000 7f4678ff9640 1 -- 172.21.15.204:0/3131519141 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 linked item id -16 name 'host2' to location {root=bar} in crush map v77) v1 ==== 168+0+0 (secure 0 0 0) 0x7f467401b310 con 0x7f467c0a4680 2024-09-15T01:34:18.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.830+0000 7f4678ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T01:34:18.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.830+0000 7f4678ff9640 10 monclient: _finish_command 2 = system:0 linked item id -16 name 'host2' to location {root=bar} in crush map 2024-09-15T01:34:18.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.926+0000 7f4682eef640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T01:34:18.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.926+0000 7f4682eef640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T01:34:18.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.926+0000 7f4682eef640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:18.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.926+0000 7f4682eef640 1 -- 172.21.15.204:0/3131519141 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) v1 -- 0x7f467c06caa0 con 0x7f467c0a4680 2024-09-15T01:34:18.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.927+0000 7f4678ff9640 1 -- 172.21.15.204:0/3131519141 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 no need to move item id -16 name 'host2' to location {root=bar} in crush map v77) v1 ==== 177+0+0 (secure 0 0 0) 0x7f467401b4f0 con 0x7f467c0a4680 2024-09-15T01:34:18.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.927+0000 7f4678ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T01:34:18.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.927+0000 7f4678ff9640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-09-15T01:34:18.930 INFO:tasks.workunit.client.0.smithi204.stderr:no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-09-15T01:34:18.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 1 -- 172.21.15.204:0/3131519141 >> v2:172.21.15.204:6812/28210 conn(0x7f46680397b0 msgr2=0x7f466803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:18.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 1 --2- 172.21.15.204:0/3131519141 >> v2:172.21.15.204:6812/28210 conn(0x7f46680397b0 0x7f466803bc60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f466c006fd0 tx=0x7f466c008040 comp rx=0 tx=0).stop 2024-09-15T01:34:18.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 10 monclient: shutdown 2024-09-15T01:34:18.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:18.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 1 -- 172.21.15.204:0/3131519141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f467c0a4680 msgr2=0x7f467c13fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:18.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 1 --2- 172.21.15.204:0/3131519141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f467c0a4680 0x7f467c13fa00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f467400bb90 tx=0x7f4674009810 comp rx=0 tx=0).stop 2024-09-15T01:34:18.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 1 -- 172.21.15.204:0/3131519141 shutdown_connections 2024-09-15T01:34:18.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 1 --2- 172.21.15.204:0/3131519141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f467c0a4680 0x7f467c13fa00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 1 --2- 172.21.15.204:0/3131519141 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f467c09f2d0 0x7f467c13bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 1 --2- 172.21.15.204:0/3131519141 >> v2:172.21.15.204:6812/28210 conn(0x7f46680397b0 0x7f466803bc60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 1 --2- 172.21.15.204:0/3131519141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c06eea0 0x7f467c13c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:18.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 1 -- 172.21.15.204:0/3131519141 >> 172.21.15.204:0/3131519141 conn(0x7f467c067740 msgr2=0x7f467c0a25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:18.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 1 -- 172.21.15.204:0/3131519141 shutdown_connections 2024-09-15T01:34:18.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:18.929+0000 7f4682eef640 1 -- 172.21.15.204:0/3131519141 wait complete. 2024-09-15T01:34:18.940 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush link host2 root=bar 2024-09-15T01:34:19.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.011+0000 7fce9919d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:19.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.011+0000 7fce9919d640 10 monclient: build_initial_monmap 2024-09-15T01:34:19.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.011+0000 7fce9919d640 10 monclient: monmap: 2024-09-15T01:34:19.014 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:19.014 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.014 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:19.013383+0000 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:19.013383+0000 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:19.015 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:19.016 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:19.016 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.012+0000 7fce9919d640 1 Processor -- start 2024-09-15T01:34:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.012+0000 7fce9919d640 1 -- start start 2024-09-15T01:34:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.012+0000 7fce9919d640 10 monclient: init 2024-09-15T01:34:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.012+0000 7fce9919d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.012+0000 7fce9919d640 10 monclient: _add_conns ranks=[0,8,6,4,2,1,9,3,10,7,5] 2024-09-15T01:34:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.012+0000 7fce9919d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fce940a2e30 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.012+0000 7fce9919d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fce9409fb80 0x7fce9409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.012+0000 7fce9919d640 10 monclient(hunting): picked mon.noname-e con 0x7fce9409fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.012+0000 7fce9919d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fce94137d20 0x7fce9413a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.012+0000 7fce9919d640 10 monclient(hunting): picked mon.noname-d con 0x7fce94137d20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9919d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce9404b720 con 0x7fce940a2e30 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9919d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9919d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fce9409e6c0 con 0x7fce94137d20 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9919d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9919d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fce940a04b0 con 0x7fce9409fb80 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9919d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9919d640 10 monclient(hunting): authenticate will time out at 874.656023s 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9359e640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fce94137d20 0x7fce9413a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fce9409fb80 0x7fce9409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9359e640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fce94137d20 0x7fce9413a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:53960/0 (socket says 172.21.15.204:53960) 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9359e640 1 -- 172.21.15.204:0/4009720296 learned_addr learned my addr 172.21.15.204:0/4009720296 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9359e640 10 monclient(hunting): get_auth_request con 0x7fce94137d20 auth_method 0 2024-09-15T01:34:19.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce92d9d640 1 --1- 172.21.15.204:0/4009720296 >> v1:172.21.15.178:6789/0 conn(0x7fce940a2e30 0x7fce94132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.204:59144/0 (socket says 172.21.15.204:59144) 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 10 monclient(hunting): get_auth_request con 0x7fce9409fb80 auth_method 0 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9359e640 1 -- 172.21.15.204:0/4009720296 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fce94137d20 msgr2=0x7fce9413a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9359e640 1 -- 172.21.15.204:0/4009720296 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fce94137d20 msgr2=0x7fce9413a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9359e640 1 --2- 172.21.15.204:0/4009720296 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fce94137d20 0x7fce9413a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:19.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9359e640 1 --2- 172.21.15.204:0/4009720296 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fce94137d20 0x7fce9413a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 10 monclient(hunting): handle_auth_done global_id 5021 payload 274 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 1 -- 172.21.15.204:0/4009720296 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fce94137d20 msgr2=0x7fce9413a100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 1 --2- 172.21.15.204:0/4009720296 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fce94137d20 0x7fce9413a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 1 -- 172.21.15.204:0/4009720296 >> v1:172.21.15.178:6789/0 conn(0x7fce940a2e30 legacy=0x7fce94132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce91d9b640 1 -- 172.21.15.204:0/4009720296 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020507117 0 0) 0x7fce9404b720 con 0x7fce940a2e30 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 1 monclient: found mon.noname-e 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce9259c640 1 -- 172.21.15.204:0/4009720296 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce94133170 con 0x7fce9409fb80 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.013+0000 7fce91d9b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:19.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce9259c640 1 --2- 172.21.15.204:0/4009720296 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fce9409fb80 0x7fce9409ff70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce88009880 tx=0x7fce880090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e37b2ca9c9a2ed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce91d9b640 1 -- 172.21.15.204:0/4009720296 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce8802ac50 con 0x7fce9409fb80 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce91d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce91d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce91d9b640 10 monclient: dump: 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:19.027 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce91d9b640 10 monclient: _finish_auth 0 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce91d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce91d9b640 1 -- 172.21.15.204:0/4009720296 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8802add0 con 0x7fce9409fb80 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce9919d640 5 monclient: authenticate success, global_id 5021 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce9919d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce91d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:19.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce9919d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce9919d640 4 set_mon_vals no callback set 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce91d9b640 1 -- 172.21.15.204:0/4009720296 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce88031ca0 con 0x7fce9409fb80 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce91d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce91d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.014+0000 7fce91d9b640 10 monclient: dump: 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:19.029 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.015+0000 7fce9919d640 10 monclient: shutdown 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.015+0000 7fce9919d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.015+0000 7fce9919d640 1 -- 172.21.15.204:0/4009720296 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fce9409fb80 msgr2=0x7fce9409ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.015+0000 7fce9919d640 1 --2- 172.21.15.204:0/4009720296 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fce9409fb80 0x7fce9409ff70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce88009880 tx=0x7fce880090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.015+0000 7fce9919d640 1 -- 172.21.15.204:0/4009720296 shutdown_connections 2024-09-15T01:34:19.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.015+0000 7fce9919d640 1 --2- 172.21.15.204:0/4009720296 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fce94137d20 0x7fce9413a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.015+0000 7fce9919d640 1 --2- 172.21.15.204:0/4009720296 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fce9409fb80 0x7fce9409ff70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.015+0000 7fce9919d640 1 -- 172.21.15.204:0/4009720296 >> 172.21.15.204:0/4009720296 conn(0x7fce9406e860 msgr2=0x7fce9406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.015+0000 7fce9919d640 1 -- 172.21.15.204:0/4009720296 shutdown_connections 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.015+0000 7fce9919d640 1 -- 172.21.15.204:0/4009720296 wait complete. 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.015+0000 7fce9919d640 10 monclient: build_initial_monmap 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.015+0000 7fce9919d640 10 monclient: monmap: 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:19.031 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 1 Processor -- start 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 1 -- start start 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 10 monclient: init 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:19.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 10 monclient: _add_conns ranks=[1,8,0,5,3,7,4,6,2] 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9409fb80 0x7fce94098f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 10 monclient(hunting): picked mon.noname-f con 0x7fce9409fb80 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fce940a2e30 0x7fce940994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 10 monclient(hunting): picked mon.noname-e con 0x7fce940a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce94137d20 0x7fce940999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 10 monclient(hunting): picked mon.noname-a con 0x7fce94137d20 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9411c970 con 0x7fce94137d20 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce94130920 con 0x7fce9409fb80 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fce9409ca40 con 0x7fce940a2e30 2024-09-15T01:34:19.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9919d640 10 monclient(hunting): authenticate will time out at 874.659679s 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce92d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9409fb80 0x7fce94098f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce92d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9409fb80 0x7fce94098f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48888/0 (socket says 172.21.15.204:48888) 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce92d9d640 1 -- 172.21.15.204:0/271015507 learned_addr learned my addr 172.21.15.204:0/271015507 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce92d9d640 10 monclient(hunting): get_auth_request con 0x7fce9409fb80 auth_method 0 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce92d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce92d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce92d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9259c640 1 --2- 172.21.15.204:0/271015507 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fce940a2e30 0x7fce940994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9359e640 1 --2- 172.21.15.204:0/271015507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce94137d20 0x7fce940999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce92d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce92d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce92d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:19.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9259c640 10 monclient(hunting): get_auth_request con 0x7fce940a2e30 auth_method 0 2024-09-15T01:34:19.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9359e640 10 monclient(hunting): get_auth_request con 0x7fce94137d20 auth_method 0 2024-09-15T01:34:19.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:19.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.016+0000 7fce9259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:19.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce9259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:19.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce9359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:19.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce9359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce9359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce92d9d640 10 monclient(hunting): handle_auth_done global_id 5473 payload 824 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce92d9d640 1 -- 172.21.15.204:0/271015507 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fce940a2e30 msgr2=0x7fce940994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce92d9d640 1 --2- 172.21.15.204:0/271015507 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fce940a2e30 0x7fce940994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce92d9d640 1 -- 172.21.15.204:0/271015507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce94137d20 msgr2=0x7fce940999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce92d9d640 1 --2- 172.21.15.204:0/271015507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce94137d20 0x7fce940999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce92d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce92d9d640 1 monclient: found mon.noname-f 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce92d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce92d9d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce92d9d640 1 -- 172.21.15.204:0/271015507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce88009390 con 0x7fce9409fb80 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce92d9d640 1 --2- 172.21.15.204:0/271015507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9409fb80 0x7fce94098f70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fce8400c610 tx=0x7fce84007d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce7affd640 1 -- 172.21.15.204:0/271015507 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce840047a0 con 0x7fce9409fb80 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:19.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce7affd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce7affd640 10 monclient: dump: 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:19.037 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce7affd640 10 monclient: _finish_auth 0 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce7affd640 1 -- 172.21.15.204:0/271015507 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce84004940 con 0x7fce9409fb80 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce9919d640 5 monclient: authenticate success, global_id 5473 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce9919d640 10 monclient: _renew_subs 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce9919d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce9919d640 1 -- 172.21.15.204:0/271015507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9409b080 con 0x7fce9409fb80 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce7affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:19.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce7affd640 1 -- 172.21.15.204:0/271015507 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce840113a0 con 0x7fce9409fb80 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce7affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce7affd640 10 monclient: dump: 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:19.039 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce9919d640 10 monclient: _renew_subs 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce9919d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce9919d640 1 -- 172.21.15.204:0/271015507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9409b470 con 0x7fce9409fb80 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.017+0000 7fce9159a640 4 set_mon_vals no callback set 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.018+0000 7fce7affd640 1 -- 172.21.15.204:0/271015507 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fce84007300 con 0x7fce9409fb80 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.018+0000 7fce9919d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.018+0000 7fce9919d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.018+0000 7fce9919d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.018+0000 7fce9919d640 1 -- 172.21.15.204:0/271015507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce940a22c0 con 0x7fce9409fb80 2024-09-15T01:34:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.018+0000 7fce7affd640 1 --2- 172.21.15.204:0/271015507 >> v2:172.21.15.204:6812/28210 conn(0x7fce800397b0 0x7fce8003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.018+0000 7fce7affd640 1 -- 172.21.15.204:0/271015507 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7fce8404a630 con 0x7fce9409fb80 2024-09-15T01:34:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.018+0000 7fce9259c640 1 --2- 172.21.15.204:0/271015507 >> v2:172.21.15.204:6812/28210 conn(0x7fce800397b0 0x7fce8003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.018+0000 7fce9259c640 10 monclient: get_auth_request con 0x7fce800397b0 auth_method 0 2024-09-15T01:34:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.019+0000 7fce9259c640 1 --2- 172.21.15.204:0/271015507 >> v2:172.21.15.204:6812/28210 conn(0x7fce800397b0 0x7fce8003bc60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fce88034000 tx=0x7fce88032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.022+0000 7fce7affd640 1 -- 172.21.15.204:0/271015507 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fce840075b0 con 0x7fce9409fb80 2024-09-15T01:34:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.022+0000 7fce7affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.022+0000 7fce7affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:19.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.180+0000 7fce9919d640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T01:34:19.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.180+0000 7fce9919d640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T01:34:19.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.180+0000 7fce9919d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:19.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.180+0000 7fce9919d640 1 -- 172.21.15.204:0/271015507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) v1 -- 0x7fce94070cc0 con 0x7fce9409fb80 2024-09-15T01:34:19.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.181+0000 7fce7affd640 1 -- 172.21.15.204:0/271015507 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 no need to move item id -16 name 'host2' to location {root=bar} in crush map v77) v1 ==== 177+0+0 (secure 0 0 0) 0x7fce840180a0 con 0x7fce9409fb80 2024-09-15T01:34:19.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.181+0000 7fce7affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T01:34:19.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.181+0000 7fce7affd640 10 monclient: _finish_command 2 = system:0 no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-09-15T01:34:19.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.277+0000 7fce9919d640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T01:34:19.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.277+0000 7fce9919d640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T01:34:19.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.277+0000 7fce9919d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:19.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.277+0000 7fce9919d640 1 -- 172.21.15.204:0/271015507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) v1 -- 0x7fce940008d0 con 0x7fce9409fb80 2024-09-15T01:34:19.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.278+0000 7fce7affd640 1 -- 172.21.15.204:0/271015507 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 no need to move item id -16 name 'host2' to location {root=bar} in crush map v77) v1 ==== 177+0+0 (secure 0 0 0) 0x7fce8400e290 con 0x7fce9409fb80 2024-09-15T01:34:19.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.278+0000 7fce7affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-09-15T01:34:19.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.278+0000 7fce7affd640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-09-15T01:34:19.282 INFO:tasks.workunit.client.0.smithi204.stderr:no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-09-15T01:34:19.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 1 -- 172.21.15.204:0/271015507 >> v2:172.21.15.204:6812/28210 conn(0x7fce800397b0 msgr2=0x7fce8003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 1 --2- 172.21.15.204:0/271015507 >> v2:172.21.15.204:6812/28210 conn(0x7fce800397b0 0x7fce8003bc60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fce88034000 tx=0x7fce88032000 comp rx=0 tx=0).stop 2024-09-15T01:34:19.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 10 monclient: shutdown 2024-09-15T01:34:19.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:19.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 1 -- 172.21.15.204:0/271015507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9409fb80 msgr2=0x7fce94098f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 1 --2- 172.21.15.204:0/271015507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9409fb80 0x7fce94098f70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fce8400c610 tx=0x7fce84007d00 comp rx=0 tx=0).stop 2024-09-15T01:34:19.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 1 -- 172.21.15.204:0/271015507 shutdown_connections 2024-09-15T01:34:19.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 1 --2- 172.21.15.204:0/271015507 >> v2:172.21.15.204:6812/28210 conn(0x7fce800397b0 0x7fce8003bc60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 1 --2- 172.21.15.204:0/271015507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce94137d20 0x7fce940999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 1 --2- 172.21.15.204:0/271015507 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fce940a2e30 0x7fce940994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 1 --2- 172.21.15.204:0/271015507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9409fb80 0x7fce94098f70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 1 -- 172.21.15.204:0/271015507 >> 172.21.15.204:0/271015507 conn(0x7fce9406e860 msgr2=0x7fce940a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:19.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 1 -- 172.21.15.204:0/271015507 shutdown_connections 2024-09-15T01:34:19.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.280+0000 7fce9919d640 1 -- 172.21.15.204:0/271015507 wait complete. 2024-09-15T01:34:19.293 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:34:19.293 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -c osd.7 2024-09-15T01:34:19.293 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -q 2 2024-09-15T01:34:19.363 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.361+0000 7f5ae0a34640 10 monclient: get_monmap_and_config 2024-09-15T01:34:19.363 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.361+0000 7f5ae0a34640 10 monclient: build_initial_monmap 2024-09-15T01:34:19.363 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.361+0000 7f5ae0a34640 10 monclient: monmap: 2024-09-15T01:34:19.364 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:19.364 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.364 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:19.362683+0000 2024-09-15T01:34:19.364 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:19.362683+0000 2024-09-15T01:34:19.364 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:19.364 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.364 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:19.364 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:19.364 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:19.364 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:19.364 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:19.364 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:19.365 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:19.365 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:19.365 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:19.365 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:19.365 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:19.365 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.361+0000 7f5ae0a34640 1 Processor -- start 2024-09-15T01:34:19.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 1 -- start start 2024-09-15T01:34:19.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 10 monclient: init 2024-09-15T01:34:19.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:19.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 10 monclient: _add_conns ranks=[9,7,4,1,3,6,2,8,10,0,5] 2024-09-15T01:34:19.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5adc0a2400 0x7f5adc0a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 10 monclient(hunting): picked mon.noname-a con 0x7f5adc0a2400 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:19.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5adc09f990 0x7f5adc09fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 10 monclient(hunting): picked mon.noname-i con 0x7f5adc09f990 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:19.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5adc132930 0x7f5adc132d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 10 monclient(hunting): picked mon.noname-c con 0x7f5adc132930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:19.374 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5adc09e1a0 con 0x7f5adc0a2400 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc0a27d0 con 0x7f5adc132930 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc09dec0 con 0x7f5adc09f990 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ae0a34640 10 monclient(hunting): authenticate will time out at 875.005599s 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5adc09f990 0x7f5adc09fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5adc09f990 0x7f5adc09fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:34064/0 (socket says 172.21.15.204:34064) 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ad3fff640 1 -- 172.21.15.204:0/2164313051 learned_addr learned my addr 172.21.15.204:0/2164313051 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ad3fff640 10 monclient(hunting): get_auth_request con 0x7f5adc09f990 auth_method 0 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ad3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ad3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.375 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ad3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ad3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ad3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ad3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ada59c640 1 --2- 172.21.15.204:0/2164313051 >> v2:172.21.15.178:3300/0 conn(0x7f5adc0a2400 0x7f5adc0a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ada59c640 10 monclient(hunting): get_auth_request con 0x7f5adc0a2400 auth_method 0 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ada59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ada59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ada59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5adad9d640 1 --2- 172.21.15.204:0/2164313051 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5adc132930 0x7f5adc132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5adad9d640 10 monclient(hunting): get_auth_request con 0x7f5adc132930 auth_method 0 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5adad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5adad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5adad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ad3fff640 10 monclient(hunting): handle_auth_done global_id 5074 payload 274 2024-09-15T01:34:19.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.362+0000 7f5ad3fff640 1 -- 172.21.15.204:0/2164313051 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5adc132930 msgr2=0x7f5adc132d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad3fff640 1 --2- 172.21.15.204:0/2164313051 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5adc132930 0x7f5adc132d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad3fff640 1 -- 172.21.15.204:0/2164313051 >> v2:172.21.15.178:3300/0 conn(0x7f5adc0a2400 msgr2=0x7f5adc0a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad3fff640 1 --2- 172.21.15.204:0/2164313051 >> v2:172.21.15.178:3300/0 conn(0x7f5adc0a2400 0x7f5adc0a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad3fff640 1 monclient: found mon.noname-i 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad3fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad3fff640 1 -- 172.21.15.204:0/2164313051 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc0648a0 con 0x7f5adc09f990 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ada59c640 1 --2- 172.21.15.204:0/2164313051 >> v2:172.21.15.178:3300/0 conn(0x7f5adc0a2400 0x7f5adc0a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad3fff640 1 --2- 172.21.15.204:0/2164313051 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5adc09f990 0x7f5adc09fd60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009910 tx=0x7f5ac40091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=ca1acb4121c46396 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad9d9b640 1 -- 172.21.15.204:0/2164313051 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ac402ac80 con 0x7f5adc09f990 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad9d9b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:34:19.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad9d9b640 10 monclient: dump: 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:19.378 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad9d9b640 10 monclient: _finish_auth 0 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ae0a34640 5 monclient: authenticate success, global_id 5074 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad9d9b640 1 -- 172.21.15.204:0/2164313051 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac402ae00 con 0x7f5adc09f990 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ae0a34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad9d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ae0a34640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ae0a34640 4 set_mon_vals no callback set 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad9d9b640 1 -- 172.21.15.204:0/2164313051 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ac4031ca0 con 0x7f5adc09f990 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad9d9b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:19.379 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.363+0000 7f5ad9d9b640 10 monclient: dump: 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:19.380 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.364+0000 7f5ae0a34640 10 monclient: shutdown 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.364+0000 7f5ae0a34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.364+0000 7f5ae0a34640 1 -- 172.21.15.204:0/2164313051 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5adc09f990 msgr2=0x7f5adc09fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.364+0000 7f5ae0a34640 1 --2- 172.21.15.204:0/2164313051 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5adc09f990 0x7f5adc09fd60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009910 tx=0x7f5ac40091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.364+0000 7f5ae0a34640 1 -- 172.21.15.204:0/2164313051 shutdown_connections 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.364+0000 7f5ae0a34640 1 --2- 172.21.15.204:0/2164313051 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5adc132930 0x7f5adc132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.364+0000 7f5ae0a34640 1 --2- 172.21.15.204:0/2164313051 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5adc09f990 0x7f5adc09fd60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.364+0000 7f5ae0a34640 1 --2- 172.21.15.204:0/2164313051 >> v2:172.21.15.178:3300/0 conn(0x7f5adc0a2400 0x7f5adc0a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.364+0000 7f5ae0a34640 1 -- 172.21.15.204:0/2164313051 >> 172.21.15.204:0/2164313051 conn(0x7f5adc06e860 msgr2=0x7f5adc06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.364+0000 7f5ae0a34640 1 -- 172.21.15.204:0/2164313051 shutdown_connections 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.364+0000 7f5ae0a34640 1 -- 172.21.15.204:0/2164313051 wait complete. 2024-09-15T01:34:19.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient: build_initial_monmap 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient: monmap: 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:19.382 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:19.383 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:19.383 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:19.383 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 1 Processor -- start 2024-09-15T01:34:19.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 1 -- start start 2024-09-15T01:34:19.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient: init 2024-09-15T01:34:19.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:19.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient: _add_conns ranks=[6,3,8,0,5,1,4,7,2] 2024-09-15T01:34:19.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5adc09f990 0x7f5adc09a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient(hunting): picked mon.noname-d con 0x7f5adc09f990 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:19.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5adc0a2400 0x7f5adc09a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient(hunting): picked mon.noname-g con 0x7f5adc0a2400 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5adc132930 0x7f5adc09acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient(hunting): picked mon.noname-e con 0x7f5adc132930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc11d950 con 0x7f5adc0a2400 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc09d3d0 con 0x7f5adc09f990 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc09cad0 con 0x7f5adc132930 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ae0a34640 10 monclient(hunting): authenticate will time out at 875.008868s 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5adc0a2400 0x7f5adc09a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ad3fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5adc0a2400 0x7f5adc09a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54462/0 (socket says 172.21.15.204:54462) 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ad3fff640 1 -- 172.21.15.204:0/1451091556 learned_addr learned my addr 172.21.15.204:0/1451091556 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:19.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ad3fff640 10 monclient(hunting): get_auth_request con 0x7f5adc0a2400 auth_method 0 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ad3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ad3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.365+0000 7f5ad3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ada59c640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5adc09f990 0x7f5adc09a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5adc132930 0x7f5adc09acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ada59c640 10 monclient(hunting): get_auth_request con 0x7f5adc09f990 auth_method 0 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ada59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ada59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ada59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad3fff640 1 -- 172.21.15.204:0/1451091556 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5adc0a2400 msgr2=0x7f5adc09a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad3fff640 1 -- 172.21.15.204:0/1451091556 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5adc0a2400 msgr2=0x7f5adc09a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad3fff640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5adc0a2400 0x7f5adc09a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:19.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 10 monclient(hunting): get_auth_request con 0x7f5adc132930 auth_method 0 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad3fff640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5adc0a2400 0x7f5adc09a780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ada59c640 1 -- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5adc09f990 msgr2=0x7f5adc09a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ada59c640 1 -- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5adc09f990 msgr2=0x7f5adc09a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ada59c640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5adc09f990 0x7f5adc09a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ada59c640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5adc09f990 0x7f5adc09a240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 10 monclient(hunting): handle_auth_done global_id 5039 payload 824 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 1 -- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5adc09f990 msgr2=0x7f5adc09a240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:19.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5adc09f990 0x7f5adc09a240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 1 -- 172.21.15.204:0/1451091556 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5adc0a2400 msgr2=0x7f5adc09a780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5adc0a2400 0x7f5adc09a780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 1 monclient: found mon.noname-e 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 1 -- 172.21.15.204:0/1451091556 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac4009450 con 0x7f5adc132930 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5adad9d640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5adc132930 0x7f5adc09acc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4007ea0 tx=0x7f5ad40097b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad27fc640 1 -- 172.21.15.204:0/1451091556 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ad4009840 con 0x7f5adc132930 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad27fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad27fc640 10 monclient: dump: 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:19.387 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:19.388 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad27fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad27fc640 1 -- 172.21.15.204:0/1451091556 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad400ed60 con 0x7f5adc132930 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ae0a34640 5 monclient: authenticate success, global_id 5039 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ae0a34640 10 monclient: _renew_subs 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ae0a34640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ae0a34640 1 -- 172.21.15.204:0/1451091556 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5adc12d440 con 0x7f5adc132930 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad27fc640 1 -- 172.21.15.204:0/1451091556 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ad400eed0 con 0x7f5adc132930 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad27fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad27fc640 10 monclient: dump: 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.389 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ae0a34640 10 monclient: _renew_subs 2024-09-15T01:34:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ad959a640 4 set_mon_vals no callback set 2024-09-15T01:34:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ae0a34640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.366+0000 7f5ae0a34640 1 -- 172.21.15.204:0/1451091556 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc09be40 con 0x7f5adc132930 2024-09-15T01:34:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.367+0000 7f5ae0a34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.367+0000 7f5ae0a34640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.367+0000 7f5ae0a34640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.367+0000 7f5ae0a34640 1 -- 172.21.15.204:0/1451091556 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5adc070b80 con 0x7f5adc132930 2024-09-15T01:34:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.367+0000 7f5ad27fc640 1 -- 172.21.15.204:0/1451091556 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5ad4003020 con 0x7f5adc132930 2024-09-15T01:34:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.367+0000 7f5ad27fc640 1 --2- 172.21.15.204:0/1451091556 >> v2:172.21.15.204:6812/28210 conn(0x7f5abc0397b0 0x7f5abc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.368+0000 7f5ad27fc640 1 -- 172.21.15.204:0/1451091556 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7f5ad404bac0 con 0x7f5adc132930 2024-09-15T01:34:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.368+0000 7f5ada59c640 1 --2- 172.21.15.204:0/1451091556 >> v2:172.21.15.204:6812/28210 conn(0x7f5abc0397b0 0x7f5abc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.368+0000 7f5ada59c640 10 monclient: get_auth_request con 0x7f5abc0397b0 auth_method 0 2024-09-15T01:34:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.368+0000 7f5ada59c640 1 --2- 172.21.15.204:0/1451091556 >> v2:172.21.15.204:6812/28210 conn(0x7f5abc0397b0 0x7f5abc03bc60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5acc006e70 tx=0x7f5acc008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:19.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.372+0000 7f5ad27fc640 1 -- 172.21.15.204:0/1451091556 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5ad401baf0 con 0x7f5adc132930 2024-09-15T01:34:19.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.372+0000 7f5ad27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:19.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.372+0000 7f5ad27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:19.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.527+0000 7f5ae0a34640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:19.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.527+0000 7f5ae0a34640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:19.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.527+0000 7f5ae0a34640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:19.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.527+0000 7f5ae0a34640 1 -- 172.21.15.204:0/1451091556 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f5adc09e480 con 0x7f5adc132930 2024-09-15T01:34:19.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.528+0000 7f5ad27fc640 1 -- 172.21.15.204:0/1451091556 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v77) v1 ==== 56+0+1474 (secure 0 0 0) 0x7f5ad401b310 con 0x7f5adc132930 2024-09-15T01:34:19.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.528+0000 7f5ad27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:19.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.528+0000 7f5ad27fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:19.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.621+0000 7f5ae0a34640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:19.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.621+0000 7f5ae0a34640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:19.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.621+0000 7f5ae0a34640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:19.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.621+0000 7f5ae0a34640 1 -- 172.21.15.204:0/1451091556 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f5adc0008d0 con 0x7f5adc132930 2024-09-15T01:34:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.621+0000 7f5ad27fc640 1 -- 172.21.15.204:0/1451091556 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v77) v1 ==== 56+0+1474 (secure 0 0 0) 0x7f5ad401b4f0 con 0x7f5adc132930 2024-09-15T01:34:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.621+0000 7f5ad27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.621+0000 7f5ad27fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:19.626 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 1 -- 172.21.15.204:0/1451091556 >> v2:172.21.15.204:6812/28210 conn(0x7f5abc0397b0 msgr2=0x7f5abc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 1 --2- 172.21.15.204:0/1451091556 >> v2:172.21.15.204:6812/28210 conn(0x7f5abc0397b0 0x7f5abc03bc60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5acc006e70 tx=0x7f5acc008040 comp rx=0 tx=0).stop 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 10 monclient: shutdown 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 1 -- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5adc132930 msgr2=0x7f5adc09acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5adc132930 0x7f5adc09acc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4007ea0 tx=0x7f5ad40097b0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 1 -- 172.21.15.204:0/1451091556 shutdown_connections 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 1 --2- 172.21.15.204:0/1451091556 >> v2:172.21.15.204:6812/28210 conn(0x7f5abc0397b0 0x7f5abc03bc60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5adc132930 0x7f5adc09acc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5adc0a2400 0x7f5adc09a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 1 --2- 172.21.15.204:0/1451091556 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5adc09f990 0x7f5adc09a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 1 -- 172.21.15.204:0/1451091556 >> 172.21.15.204:0/1451091556 conn(0x7f5adc06e860 msgr2=0x7f5adc130150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 1 -- 172.21.15.204:0/1451091556 shutdown_connections 2024-09-15T01:34:19.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.624+0000 7f5ae0a34640 1 -- 172.21.15.204:0/1451091556 wait complete. 2024-09-15T01:34:19.636 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:34:19.636 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -c host1 2024-09-15T01:34:19.636 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -q 2 2024-09-15T01:34:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.705+0000 7ff238b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.705+0000 7ff238b9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.706+0000 7ff238b9d640 10 monclient: monmap: 2024-09-15T01:34:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:19.707464+0000 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:19.707464+0000 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:19.710 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.706+0000 7ff238b9d640 1 Processor -- start 2024-09-15T01:34:19.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.706+0000 7ff238b9d640 1 -- start start 2024-09-15T01:34:19.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.706+0000 7ff238b9d640 10 monclient: init 2024-09-15T01:34:19.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 10 monclient: _add_conns ranks=[2,9,8,4,7,1,3,0,6,5,10] 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2340a2400 0x7ff2340a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 10 monclient(hunting): picked mon.noname-b con 0x7ff2340a2400 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff23409f990 0x7ff23409fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 10 monclient(hunting): picked mon.noname-a con 0x7ff23409f990 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 0x7ff234132d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 10 monclient(hunting): picked mon.noname-e con 0x7ff234132930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23409e1a0 con 0x7ff23409f990 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2340a27d0 con 0x7ff2340a2400 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff23409dec0 con 0x7ff234132930 2024-09-15T01:34:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff238b9d640 10 monclient(hunting): authenticate will time out at 875.350149s 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 0x7ff234132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff229d9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff23409f990 0x7ff23409fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff229d9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff23409f990 0x7ff23409fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:54154/0 (socket says 172.21.15.204:54154) 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff229d9b640 1 -- 172.21.15.204:0/2591332168 learned_addr learned my addr 172.21.15.204:0/2591332168 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff229d9b640 10 monclient(hunting): get_auth_request con 0x7ff23409f990 auth_method 0 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 0x7ff234132d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:56332/0 (socket says 172.21.15.204:56332) 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff229d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff229d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff229d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 10 monclient(hunting): get_auth_request con 0x7ff234132930 auth_method 0 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff229d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff229d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff229d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 10 monclient(hunting): handle_auth_done global_id 5048 payload 274 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 1 -- 172.21.15.204:0/2591332168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2340a2400 msgr2=0x7ff2340a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 1 --2- 172.21.15.204:0/2591332168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2340a2400 0x7ff2340a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 1 -- 172.21.15.204:0/2591332168 >> v2:172.21.15.178:3300/0 conn(0x7ff23409f990 msgr2=0x7ff23409fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 1 --2- 172.21.15.204:0/2591332168 >> v2:172.21.15.178:3300/0 conn(0x7ff23409f990 0x7ff23409fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 1 monclient: found mon.noname-e 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff232d9d640 1 -- 172.21.15.204:0/2591332168 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2340648a0 con 0x7ff234132930 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.707+0000 7ff229d9b640 1 --2- 172.21.15.204:0/2591332168 >> v2:172.21.15.178:3300/0 conn(0x7ff23409f990 0x7ff23409fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff232d9d640 1 --2- 172.21.15.204:0/2591332168 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 0x7ff234132d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009e60 tx=0x7ff22c009780 comp rx=0 tx=0).ready entity=mon.8 client_cookie=28f9136a4cef87f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff231d9b640 1 -- 172.21.15.204:0/2591332168 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff22c0079d0 con 0x7ff234132930 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff231d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff231d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff231d9b640 10 monclient: dump: 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff231d9b640 10 monclient: _finish_auth 0 2024-09-15T01:34:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff231d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff231d9b640 1 -- 172.21.15.204:0/2591332168 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c02d050 con 0x7ff234132930 2024-09-15T01:34:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff231d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff231d9b640 1 -- 172.21.15.204:0/2591332168 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff22c007bd0 con 0x7ff234132930 2024-09-15T01:34:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff231d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff231d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff231d9b640 10 monclient: dump: 2024-09-15T01:34:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:19.725 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff238b9d640 5 monclient: authenticate success, global_id 5048 2024-09-15T01:34:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff238b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.708+0000 7ff238b9d640 4 set_mon_vals no callback set 2024-09-15T01:34:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 10 monclient: shutdown 2024-09-15T01:34:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 1 -- 172.21.15.204:0/2591332168 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 msgr2=0x7ff234132d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 1 --2- 172.21.15.204:0/2591332168 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 0x7ff234132d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009e60 tx=0x7ff22c009780 comp rx=0 tx=0).stop 2024-09-15T01:34:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 1 -- 172.21.15.204:0/2591332168 shutdown_connections 2024-09-15T01:34:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 1 --2- 172.21.15.204:0/2591332168 >> v2:172.21.15.178:3300/0 conn(0x7ff23409f990 0x7ff23409fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 1 --2- 172.21.15.204:0/2591332168 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 0x7ff234132d80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 1 --2- 172.21.15.204:0/2591332168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2340a2400 0x7ff2340a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 1 -- 172.21.15.204:0/2591332168 >> 172.21.15.204:0/2591332168 conn(0x7ff23406e860 msgr2=0x7ff23406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 1 -- 172.21.15.204:0/2591332168 shutdown_connections 2024-09-15T01:34:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 1 -- 172.21.15.204:0/2591332168 wait complete. 2024-09-15T01:34:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.709+0000 7ff238b9d640 10 monclient: monmap: 2024-09-15T01:34:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:19.728 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 1 Processor -- start 2024-09-15T01:34:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 1 -- start start 2024-09-15T01:34:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 10 monclient: init 2024-09-15T01:34:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 10 monclient: _add_conns ranks=[4,6,8,0,2,3,1,7,5] 2024-09-15T01:34:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff23409f990 0x7ff23411d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 10 monclient(hunting): picked mon.noname-c con 0x7ff23409f990 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff2340a2400 0x7ff23411d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 10 monclient(hunting): picked mon.noname-d con 0x7ff2340a2400 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 0x7ff23411dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 10 monclient(hunting): picked mon.noname-e con 0x7ff234132930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff23411f4f0 con 0x7ff23409f990 2024-09-15T01:34:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff23409d3d0 con 0x7ff2340a2400 2024-09-15T01:34:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff23409cad0 con 0x7ff234132930 2024-09-15T01:34:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff238b9d640 10 monclient(hunting): authenticate will time out at 875.353686s 2024-09-15T01:34:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.710+0000 7ff229d9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff2340a2400 0x7ff23411d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff23409f990 0x7ff23411d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff229d9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff2340a2400 0x7ff23411d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:53982/0 (socket says 172.21.15.204:53982) 2024-09-15T01:34:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff229d9b640 1 -- 172.21.15.204:0/2644794105 learned_addr learned my addr 172.21.15.204:0/2644794105 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff229d9b640 10 monclient(hunting): get_auth_request con 0x7ff2340a2400 auth_method 0 2024-09-15T01:34:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff229d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff229d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff229d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 10 monclient(hunting): get_auth_request con 0x7ff23409f990 auth_method 0 2024-09-15T01:34:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff232d9d640 1 --2- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 0x7ff23411dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff232d9d640 10 monclient(hunting): get_auth_request con 0x7ff234132930 auth_method 0 2024-09-15T01:34:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff232d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff232d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff232d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff229d9b640 1 -- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff2340a2400 msgr2=0x7ff23411d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:34:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff229d9b640 1 -- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff2340a2400 msgr2=0x7ff23411d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:34:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff229d9b640 1 --2- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff2340a2400 0x7ff23411d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff229d9b640 1 --2- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff2340a2400 0x7ff23411d730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff232d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff232d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff232d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 10 monclient(hunting): handle_auth_done global_id 4936 payload 824 2024-09-15T01:34:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 1 -- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 msgr2=0x7ff23411dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 1 --2- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 0x7ff23411dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 1 -- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff2340a2400 msgr2=0x7ff23411d730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 1 --2- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff2340a2400 0x7ff23411d730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 1 monclient: found mon.noname-c 2024-09-15T01:34:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 1 -- 172.21.15.204:0/2644794105 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c009450 con 0x7ff23409f990 2024-09-15T01:34:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff232d9d640 1 --2- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 0x7ff23411dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23259c640 1 --2- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff23409f990 0x7ff23411d1f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff22400b560 tx=0x7ff224009120 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff22affd640 1 -- 172.21.15.204:0/2644794105 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff22400f990 con 0x7ff23409f990 2024-09-15T01:34:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff22affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff22affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff22affd640 10 monclient: dump: 2024-09-15T01:34:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:19.735 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff22affd640 10 monclient: _finish_auth 0 2024-09-15T01:34:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff22affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff22affd640 1 -- 172.21.15.204:0/2644794105 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22400fb10 con 0x7ff23409f990 2024-09-15T01:34:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff238b9d640 5 monclient: authenticate success, global_id 4936 2024-09-15T01:34:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff238b9d640 10 monclient: _renew_subs 2024-09-15T01:34:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff238b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff238b9d640 1 -- 172.21.15.204:0/2644794105 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22c009a20 con 0x7ff23409f990 2024-09-15T01:34:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff22affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff22affd640 1 -- 172.21.15.204:0/2644794105 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff22400fc80 con 0x7ff23409f990 2024-09-15T01:34:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff22affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff22affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff22affd640 10 monclient: dump: 2024-09-15T01:34:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:19.737 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:19.737 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:19.737 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:19.737 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:19.737 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:19.737 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:19.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff238b9d640 10 monclient: _renew_subs 2024-09-15T01:34:19.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff238b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:19.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff23159a640 4 set_mon_vals no callback set 2024-09-15T01:34:19.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.711+0000 7ff238b9d640 1 -- 172.21.15.204:0/2644794105 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23411eaf0 con 0x7ff23409f990 2024-09-15T01:34:19.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.712+0000 7ff238b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:19.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.712+0000 7ff238b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:19.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.712+0000 7ff238b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:19.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.712+0000 7ff238b9d640 1 -- 172.21.15.204:0/2644794105 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff234120a20 con 0x7ff23409f990 2024-09-15T01:34:19.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.712+0000 7ff22affd640 1 -- 172.21.15.204:0/2644794105 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff22400f510 con 0x7ff23409f990 2024-09-15T01:34:19.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.713+0000 7ff22affd640 1 --2- 172.21.15.204:0/2644794105 >> v2:172.21.15.204:6812/28210 conn(0x7ff2140397b0 0x7ff21403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:19.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.713+0000 7ff22affd640 1 -- 172.21.15.204:0/2644794105 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7ff22404b150 con 0x7ff23409f990 2024-09-15T01:34:19.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.713+0000 7ff229d9b640 1 --2- 172.21.15.204:0/2644794105 >> v2:172.21.15.204:6812/28210 conn(0x7ff2140397b0 0x7ff21403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:19.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.713+0000 7ff229d9b640 10 monclient: get_auth_request con 0x7ff2140397b0 auth_method 0 2024-09-15T01:34:19.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.713+0000 7ff229d9b640 1 --2- 172.21.15.204:0/2644794105 >> v2:172.21.15.204:6812/28210 conn(0x7ff2140397b0 0x7ff21403bc60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff21c006cb0 tx=0x7ff21c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:19.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.717+0000 7ff22affd640 1 -- 172.21.15.204:0/2644794105 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff2240172b0 con 0x7ff23409f990 2024-09-15T01:34:19.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.717+0000 7ff22affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:19.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.717+0000 7ff22affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:19.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.877+0000 7ff238b9d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:19.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.877+0000 7ff238b9d640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:19.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.877+0000 7ff238b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:19.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.877+0000 7ff238b9d640 1 -- 172.21.15.204:0/2644794105 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7ff234070b80 con 0x7ff23409f990 2024-09-15T01:34:19.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.878+0000 7ff22affd640 1 -- 172.21.15.204:0/2644794105 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v77) v1 ==== 56+0+1474 (secure 0 0 0) 0x7ff224018af0 con 0x7ff23409f990 2024-09-15T01:34:19.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.878+0000 7ff22affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:19.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.878+0000 7ff22affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.973+0000 7ff238b9d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.973+0000 7ff238b9d640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.973+0000 7ff238b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.973+0000 7ff238b9d640 1 -- 172.21.15.204:0/2644794105 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7ff23409e480 con 0x7ff23409f990 2024-09-15T01:34:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.974+0000 7ff22affd640 1 -- 172.21.15.204:0/2644794105 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v77) v1 ==== 56+0+1474 (secure 0 0 0) 0x7ff224018cd0 con 0x7ff23409f990 2024-09-15T01:34:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.974+0000 7ff22affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.974+0000 7ff22affd640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:19.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.975+0000 7ff238b9d640 1 -- 172.21.15.204:0/2644794105 >> v2:172.21.15.204:6812/28210 conn(0x7ff2140397b0 msgr2=0x7ff21403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 1 --2- 172.21.15.204:0/2644794105 >> v2:172.21.15.204:6812/28210 conn(0x7ff2140397b0 0x7ff21403bc60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff21c006cb0 tx=0x7ff21c008040 comp rx=0 tx=0).stop 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 10 monclient: shutdown 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 1 -- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff23409f990 msgr2=0x7ff23411d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 1 --2- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff23409f990 0x7ff23411d1f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff22400b560 tx=0x7ff224009120 comp rx=0 tx=0).stop 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 1 -- 172.21.15.204:0/2644794105 shutdown_connections 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 1 --2- 172.21.15.204:0/2644794105 >> v2:172.21.15.204:6812/28210 conn(0x7ff2140397b0 0x7ff21403bc60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 1 --2- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff234132930 0x7ff23411dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 1 --2- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff2340a2400 0x7ff23411d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 1 --2- 172.21.15.204:0/2644794105 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff23409f990 0x7ff23411d1f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 1 -- 172.21.15.204:0/2644794105 >> 172.21.15.204:0/2644794105 conn(0x7ff23406e860 msgr2=0x7ff2340a45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 1 -- 172.21.15.204:0/2644794105 shutdown_connections 2024-09-15T01:34:19.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:19.976+0000 7ff238b9d640 1 -- 172.21.15.204:0/2644794105 wait complete. 2024-09-15T01:34:19.987 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:34:19.988 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -c osd.9 2024-09-15T01:34:19.988 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -q 2 2024-09-15T01:34:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.056+0000 7efea7f09640 10 monclient: get_monmap_and_config 2024-09-15T01:34:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.056+0000 7efea7f09640 10 monclient: build_initial_monmap 2024-09-15T01:34:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.056+0000 7efea7f09640 10 monclient: monmap: 2024-09-15T01:34:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:20.058282+0000 2024-09-15T01:34:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:20.058282+0000 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:20.060 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:20.061 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.057+0000 7efea7f09640 1 Processor -- start 2024-09-15T01:34:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.057+0000 7efea7f09640 1 -- start start 2024-09-15T01:34:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.057+0000 7efea7f09640 10 monclient: init 2024-09-15T01:34:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 10 monclient: _add_conns ranks=[6,3,0,7,9,8,2,10,4,1,5] 2024-09-15T01:34:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efea00a2400 0x7efea00a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 10 monclient(hunting): picked mon.noname-d con 0x7efea00a2400 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 0x7efea009fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 10 monclient(hunting): picked mon.noname-g con 0x7efea009f990 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efea0132930 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea00648a0 con 0x7efea0132930 2024-09-15T01:34:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea009e1a0 con 0x7efea009f990 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efea00a27d0 con 0x7efea00a2400 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea7f09640 10 monclient(hunting): authenticate will time out at 875.701243s 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea54a4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 0x7efea009fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea54a4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 0x7efea009fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54468/0 (socket says 172.21.15.204:54468) 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea54a4640 1 -- 172.21.15.204:0/566166054 learned_addr learned my addr 172.21.15.204:0/566166054 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea54a4640 10 monclient(hunting): get_auth_request con 0x7efea009f990 auth_method 0 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea54a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea54a4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea54a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea54a4640 1 -- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 msgr2=0x7efea009fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:34:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea54a4640 1 -- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 msgr2=0x7efea009fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea54a4640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 0x7efea009fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea54a4640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 0x7efea009fd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea5ca5640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efea00a2400 0x7efea00a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea5ca5640 10 monclient(hunting): get_auth_request con 0x7efea00a2400 auth_method 0 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea5ca5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea5ca5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea5ca5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea5ca5640 1 -- 172.21.15.204:0/566166054 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efea00a2400 msgr2=0x7efea00a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea5ca5640 1 -- 172.21.15.204:0/566166054 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efea00a2400 msgr2=0x7efea00a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea5ca5640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efea00a2400 0x7efea00a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea5ca5640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efea00a2400 0x7efea00a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885374280 0 0) 0x7efea00648a0 con 0x7efea0132930 2024-09-15T01:34:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea4ca3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea4ca3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea4ca3640 10 monclient(hunting): my global_id is 5094 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.058+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe8c0034e0 con 0x7efea0132930 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2463119498 0 0) 0x7efe8c0034e0 con 0x7efea0132930 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efea00a2400 msgr2=0x7efea00a0ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efea00a2400 0x7efea00a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 msgr2=0x7efea009fd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 0x7efea009fd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient: _finish_hunting 0 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:34:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea009dec0 con 0x7efea0132930 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient: _finish_auth 0 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7efe98001c70 con 0x7efea0132930 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea7f09640 5 monclient: authenticate success, global_id 5094 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea7f09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient: dump: 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:34:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 >> v1:172.21.15.178:6789/0 conn(0x7efea0132930 legacy=0x7efea0132d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient: _add_conns ranks=[1,3,5,2,6,0,4,8,7] 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe8c004220 0x7efe8c004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient(hunting): picked mon.f con 0x7efe8c004220 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 0x7efe8c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient(hunting): picked mon.g con 0x7efea009f990 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe8c004220 0x7efe8c004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efe8c0099c0 0x7efe8c00bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient(hunting): picked mon.h con 0x7efe8c0099c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea009e1a0 con 0x7efe8c004220 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea00a27d0 con 0x7efea009f990 2024-09-15T01:34:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c00c3e0 con 0x7efe8c0099c0 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea4ca3640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 10 monclient(hunting): get_auth_request con 0x7efe8c004220 auth_method 0 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea7f09640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea54a4640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 0x7efe8c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea64a6640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efe8c0099c0 0x7efe8c00bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea54a4640 10 monclient(hunting): get_auth_request con 0x7efea009f990 auth_method 0 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea54a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea54a4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea54a4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea64a6640 10 monclient(hunting): get_auth_request con 0x7efe8c0099c0 auth_method 0 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea64a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea64a6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea64a6640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea64a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea64a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea64a6640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 10 monclient(hunting): handle_auth_done global_id 5094 payload 293 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 1 -- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efe8c0099c0 msgr2=0x7efe8c00bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efe8c0099c0 0x7efe8c00bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 1 -- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 msgr2=0x7efe8c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 0x7efe8c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 10 monclient: _finish_hunting 0 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 1 monclient: found mon.f 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 1 -- 172.21.15.204:0/566166054 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c0034e0 con 0x7efe8c004220 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 10 monclient: _finish_auth 0 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea5ca5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.059+0000 7efea54a4640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efea009f990 0x7efe8c004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea5ca5640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe8c004220 0x7efe8c004780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe9c006fd0 tx=0x7efe9c008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40bd453b9124beb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efe9c02abd0 con 0x7efe8c004220 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea4ca3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea4ca3640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea4ca3640 10 monclient: dump: 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c02e070 con 0x7efe8c004220 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea7f09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea4ca3640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea7f09640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea4ca3640 1 -- 172.21.15.204:0/566166054 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efe9c02ad50 con 0x7efe8c004220 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea4ca3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea7f09640 4 set_mon_vals no callback set 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea4ca3640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.060+0000 7efea4ca3640 10 monclient: dump: 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.061+0000 7efea7f09640 10 monclient: shutdown 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.061+0000 7efea7f09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.061+0000 7efea7f09640 1 -- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe8c004220 msgr2=0x7efe8c004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.061+0000 7efea7f09640 1 --2- 172.21.15.204:0/566166054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe8c004220 0x7efe8c004780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe9c006fd0 tx=0x7efe9c008040 comp rx=0 tx=0).stop 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.061+0000 7efea64a6640 1 -- 172.21.15.204:0/566166054 reap_dead start 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.061+0000 7efea7f09640 1 -- 172.21.15.204:0/566166054 shutdown_connections 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.061+0000 7efea7f09640 1 -- 172.21.15.204:0/566166054 >> 172.21.15.204:0/566166054 conn(0x7efea006e860 msgr2=0x7efea006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.061+0000 7efea7f09640 1 -- 172.21.15.204:0/566166054 shutdown_connections 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.061+0000 7efea7f09640 1 -- 172.21.15.204:0/566166054 wait complete. 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.061+0000 7efea7f09640 10 monclient: build_initial_monmap 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.061+0000 7efea7f09640 10 monclient: monmap: 2024-09-15T01:34:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:20.085 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:20.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 1 Processor -- start 2024-09-15T01:34:20.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 1 -- start start 2024-09-15T01:34:20.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 10 monclient: init 2024-09-15T01:34:20.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:20.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 10 monclient: _add_conns ranks=[0,8,2,1,4,6,5,7,3] 2024-09-15T01:34:20.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe8c004220 0x7efea009c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 10 monclient(hunting): picked mon.noname-a con 0x7efe8c004220 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:20.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efea011e300 0x7efea011c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 10 monclient(hunting): picked mon.noname-e con 0x7efea011e300 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:20.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efea011ca90 0x7efea011cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 10 monclient(hunting): picked mon.noname-b con 0x7efea011ca90 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:20.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea011d420 con 0x7efe8c004220 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea011d7c0 con 0x7efea011ca90 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7efea009d3d0 con 0x7efea011e300 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea7f09640 10 monclient(hunting): authenticate will time out at 875.705707s 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea5ca5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe8c004220 0x7efea009c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea54a4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efea011e300 0x7efea011c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea54a4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efea011e300 0x7efea011c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:56360/0 (socket says 172.21.15.204:56360) 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea5ca5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe8c004220 0x7efea009c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:54156/0 (socket says 172.21.15.204:54156) 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea54a4640 1 -- 172.21.15.204:0/2442440404 learned_addr learned my addr 172.21.15.204:0/2442440404 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:20.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea54a4640 10 monclient(hunting): get_auth_request con 0x7efea011e300 auth_method 0 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.062+0000 7efea54a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea5ca5640 10 monclient(hunting): get_auth_request con 0x7efe8c004220 auth_method 0 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea5ca5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea5ca5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea5ca5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea5ca5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea5ca5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea5ca5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:20.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 10 monclient(hunting): handle_auth_done global_id 5066 payload 824 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 1 -- 172.21.15.204:0/2442440404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efea011ca90 msgr2=0x7efea011cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 1 --2- 172.21.15.204:0/2442440404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efea011ca90 0x7efea011cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 1 -- 172.21.15.204:0/2442440404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe8c004220 msgr2=0x7efea009c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 1 --2- 172.21.15.204:0/2442440404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe8c004220 0x7efea009c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 10 monclient: _finish_hunting 0 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 1 monclient: found mon.noname-e 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 1 -- 172.21.15.204:0/2442440404 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea009cad0 con 0x7efea011e300 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea54a4640 1 --2- 172.21.15.204:0/2442440404 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efea011e300 0x7efea011c550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe90010900 tx=0x7efe90008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe95ffb640 1 -- 172.21.15.204:0/2442440404 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efe9000ede0 con 0x7efea011e300 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe95ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe95ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:20.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe95ffb640 10 monclient: dump: 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:20.090 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe95ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe95ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe95ffb640 1 -- 172.21.15.204:0/2442440404 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9000c050 con 0x7efea011e300 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea7f09640 5 monclient: authenticate success, global_id 5066 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea7f09640 10 monclient: _renew_subs 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea7f09640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea7f09640 1 -- 172.21.15.204:0/2442440404 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea011f260 con 0x7efea011e300 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe95ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe95ffb640 1 -- 172.21.15.204:0/2442440404 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efe900158f0 con 0x7efea011e300 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe95ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:20.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe95ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe95ffb640 10 monclient: dump: 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:20.092 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea7f09640 10 monclient: _renew_subs 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efe97fff640 4 set_mon_vals no callback set 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea7f09640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.063+0000 7efea7f09640 1 -- 172.21.15.204:0/2442440404 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea0147e80 con 0x7efea011e300 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.064+0000 7efe95ffb640 1 -- 172.21.15.204:0/2442440404 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7efe90015470 con 0x7efea011e300 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.064+0000 7efea7f09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.064+0000 7efea7f09640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.064+0000 7efea7f09640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.064+0000 7efea7f09640 1 -- 172.21.15.204:0/2442440404 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea011f660 con 0x7efea011e300 2024-09-15T01:34:20.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.064+0000 7efe95ffb640 1 --2- 172.21.15.204:0/2442440404 >> v2:172.21.15.204:6812/28210 conn(0x7efe840397b0 0x7efe8403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.064+0000 7efe95ffb640 1 -- 172.21.15.204:0/2442440404 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7efe9004baf0 con 0x7efea011e300 2024-09-15T01:34:20.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.065+0000 7efea5ca5640 1 --2- 172.21.15.204:0/2442440404 >> v2:172.21.15.204:6812/28210 conn(0x7efe840397b0 0x7efe8403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.065+0000 7efea5ca5640 10 monclient: get_auth_request con 0x7efe840397b0 auth_method 0 2024-09-15T01:34:20.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.065+0000 7efea5ca5640 1 --2- 172.21.15.204:0/2442440404 >> v2:172.21.15.204:6812/28210 conn(0x7efe840397b0 0x7efe8403bc60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efe9c034000 tx=0x7efe9c035000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:20.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.069+0000 7efe95ffb640 1 -- 172.21.15.204:0/2442440404 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7efe90015720 con 0x7efea011e300 2024-09-15T01:34:20.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.069+0000 7efe95ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:20.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.069+0000 7efe95ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:20.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.226+0000 7efea7f09640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:20.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.226+0000 7efea7f09640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:20.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.226+0000 7efea7f09640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:20.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.226+0000 7efea7f09640 1 -- 172.21.15.204:0/2442440404 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7efea0070b80 con 0x7efea011e300 2024-09-15T01:34:20.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.226+0000 7efe95ffb640 1 -- 172.21.15.204:0/2442440404 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v77) v1 ==== 56+0+1474 (secure 0 0 0) 0x7efe9001a070 con 0x7efea011e300 2024-09-15T01:34:20.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.226+0000 7efe95ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:20.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.226+0000 7efe95ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:20.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.322+0000 7efea7f09640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:20.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.322+0000 7efea7f09640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:20.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.322+0000 7efea7f09640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:20.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.322+0000 7efea7f09640 1 -- 172.21.15.204:0/2442440404 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7efea009e480 con 0x7efea011e300 2024-09-15T01:34:20.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.323+0000 7efe95ffb640 1 -- 172.21.15.204:0/2442440404 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v77) v1 ==== 56+0+1474 (secure 0 0 0) 0x7efe90013cc0 con 0x7efea011e300 2024-09-15T01:34:20.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.323+0000 7efe95ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:20.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.323+0000 7efe95ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:20.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 1 -- 172.21.15.204:0/2442440404 >> v2:172.21.15.204:6812/28210 conn(0x7efe840397b0 msgr2=0x7efe8403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:20.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 1 --2- 172.21.15.204:0/2442440404 >> v2:172.21.15.204:6812/28210 conn(0x7efe840397b0 0x7efe8403bc60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efe9c034000 tx=0x7efe9c035000 comp rx=0 tx=0).stop 2024-09-15T01:34:20.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 10 monclient: shutdown 2024-09-15T01:34:20.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:20.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 1 -- 172.21.15.204:0/2442440404 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efea011e300 msgr2=0x7efea011c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:20.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 1 --2- 172.21.15.204:0/2442440404 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efea011e300 0x7efea011c550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe90010900 tx=0x7efe90008040 comp rx=0 tx=0).stop 2024-09-15T01:34:20.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 1 -- 172.21.15.204:0/2442440404 shutdown_connections 2024-09-15T01:34:20.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 1 --2- 172.21.15.204:0/2442440404 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efea011ca90 0x7efea011cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 1 --2- 172.21.15.204:0/2442440404 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efea011e300 0x7efea011c550 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 1 --2- 172.21.15.204:0/2442440404 >> v2:172.21.15.204:6812/28210 conn(0x7efe840397b0 0x7efe8403bc60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 1 --2- 172.21.15.204:0/2442440404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe8c004220 0x7efea009c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 1 -- 172.21.15.204:0/2442440404 >> 172.21.15.204:0/2442440404 conn(0x7efea006e860 msgr2=0x7efea012ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:20.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 1 -- 172.21.15.204:0/2442440404 shutdown_connections 2024-09-15T01:34:20.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.325+0000 7efea7f09640 1 -- 172.21.15.204:0/2442440404 wait complete. 2024-09-15T01:34:20.337 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:34:20.337 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -c host2 2024-09-15T01:34:20.337 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -q 2 2024-09-15T01:34:20.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.406+0000 7f784299d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:20.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.406+0000 7f784299d640 10 monclient: build_initial_monmap 2024-09-15T01:34:20.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.406+0000 7f784299d640 10 monclient: monmap: 2024-09-15T01:34:20.409 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:20.409 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:20.409 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:20.407795+0000 2024-09-15T01:34:20.409 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:20.407795+0000 2024-09-15T01:34:20.409 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:20.409 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:20.409 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:20.409 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:20.409 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:20.409 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:20.409 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:20.409 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:20.410 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:20.410 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:20.410 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:20.410 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:20.410 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:20.410 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:20.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 1 Processor -- start 2024-09-15T01:34:20.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 1 -- start start 2024-09-15T01:34:20.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 10 monclient: init 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 10 monclient: _add_conns ranks=[10,0,6,5,8,4,3,1,2,9,7] 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f783c09f0e0 0x7f783c09e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 10 monclient(hunting): picked mon.noname-f con 0x7f783c09f0e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f783c09f4b0 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c133040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 10 monclient(hunting): picked mon.noname-d con 0x7f783c09f880 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c0648a0 con 0x7f783c09f4b0 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783c0a0270 con 0x7f783c09f0e0 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0683f0 con 0x7f783c09f880 2024-09-15T01:34:20.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f784299d640 10 monclient(hunting): authenticate will time out at 876.050748s 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f783bfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f783c09f0e0 0x7f783c09e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f783bfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f783c09f0e0 0x7f783c09e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48896/0 (socket says 172.21.15.204:48896) 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f783bfff640 1 -- 172.21.15.204:0/2409703947 learned_addr learned my addr 172.21.15.204:0/2409703947 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f783bfff640 10 monclient(hunting): get_auth_request con 0x7f783c09f0e0 auth_method 0 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f783bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f783bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f783bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f783bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f783bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.407+0000 7f783bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f7840f3a640 1 --2- 172.21.15.204:0/2409703947 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c133040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f7840f3a640 10 monclient(hunting): get_auth_request con 0x7f783c09f880 auth_method 0 2024-09-15T01:34:20.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f7840f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f7840f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f7840f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783bfff640 10 monclient(hunting): handle_auth_done global_id 5482 payload 274 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783bfff640 1 -- 172.21.15.204:0/2409703947 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 msgr2=0x7f783c133040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783bfff640 1 --2- 172.21.15.204:0/2409703947 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c133040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783bfff640 1 -- 172.21.15.204:0/2409703947 >> v1:172.21.15.178:6789/0 conn(0x7f783c09f4b0 legacy=0x7f783c132930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783bfff640 1 monclient: found mon.noname-f 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783bfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783bfff640 1 -- 172.21.15.204:0/2409703947 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c069590 con 0x7f783c09f0e0 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783bfff640 1 --2- 172.21.15.204:0/2409703947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f783c09f0e0 0x7f783c09e750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f782c009940 tx=0x7f782c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2f011791259c34e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783affd640 1 -- 172.21.15.204:0/2409703947 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f782c02ad50 con 0x7f783c09f0e0 2024-09-15T01:34:20.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783affd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783affd640 10 monclient: dump: 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:20.420 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:20.421 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:20.421 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:20.421 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:20.421 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:20.421 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:20.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783affd640 10 monclient: _finish_auth 0 2024-09-15T01:34:20.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:20.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783affd640 1 -- 172.21.15.204:0/2409703947 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c031820 con 0x7f783c09f0e0 2024-09-15T01:34:20.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f784299d640 5 monclient: authenticate success, global_id 5482 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f784299d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.408+0000 7f783affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f783affd640 1 -- 172.21.15.204:0/2409703947 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f782c031e50 con 0x7f783c09f0e0 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 4 set_mon_vals no callback set 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f783affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f783affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f783affd640 10 monclient: dump: 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:20.422 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 10 monclient: shutdown 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 1 -- 172.21.15.204:0/2409703947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f783c09f0e0 msgr2=0x7f783c09e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 1 --2- 172.21.15.204:0/2409703947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f783c09f0e0 0x7f783c09e750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f782c009940 tx=0x7f782c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.423 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 1 -- 172.21.15.204:0/2409703947 shutdown_connections 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 1 --2- 172.21.15.204:0/2409703947 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c133040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 1 --2- 172.21.15.204:0/2409703947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f783c09f0e0 0x7f783c09e750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 1 -- 172.21.15.204:0/2409703947 >> 172.21.15.204:0/2409703947 conn(0x7f783c067c00 msgr2=0x7f783c0a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 1 -- 172.21.15.204:0/2409703947 shutdown_connections 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 1 -- 172.21.15.204:0/2409703947 wait complete. 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.409+0000 7f784299d640 10 monclient: build_initial_monmap 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 10 monclient: monmap: 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:20.424 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 1 Processor -- start 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 1 -- start start 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 10 monclient: init 2024-09-15T01:34:20.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:20.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 10 monclient: _add_conns ranks=[3,2,6,7,8,0,5,1,4] 2024-09-15T01:34:20.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 10 monclient(hunting): picked mon.noname-g con 0x7f783c09f0e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:20.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f783c09f4b0 0x7f783c11c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 10 monclient(hunting): picked mon.noname-b con 0x7f783c09f4b0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:20.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:20.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 10 monclient(hunting): picked mon.noname-d con 0x7f783c09f880 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:20.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783c11f9e0 con 0x7f783c09f4b0 2024-09-15T01:34:20.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783c11ff10 con 0x7f783c09f0e0 2024-09-15T01:34:20.426 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f783c068920 con 0x7f783c09f880 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.410+0000 7f784299d640 10 monclient(hunting): authenticate will time out at 876.053950s 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f783bfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f783bfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54492/0 (socket says 172.21.15.204:54492) 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f783bfff640 1 -- 172.21.15.204:0/1479839062 learned_addr learned my addr 172.21.15.204:0/1479839062 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f783bfff640 10 monclient(hunting): get_auth_request con 0x7f783c09f0e0 auth_method 0 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f783bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f783bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f783bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f783bfff640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 msgr2=0x7f783c1327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f783bfff640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 msgr2=0x7f783c1327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:20.427 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:20.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f7840f3a640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f7840f3a640 10 monclient(hunting): get_auth_request con 0x7f783c09f880 auth_method 0 2024-09-15T01:34:20.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f7840f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f7840f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f7840f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:20.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f7840f3a640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 msgr2=0x7f783c11c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:34:20.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f7840f3a640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 msgr2=0x7f783c11c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:34:20.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f7840f3a640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:20.428 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.411+0000 7f7840f3a640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:20.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f783bfff640 10 monclient(hunting): get_auth_request con 0x7f783c09f0e0 auth_method 0 2024-09-15T01:34:20.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f783bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f783bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f783bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:20.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f783bfff640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 msgr2=0x7f783c1327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:34:20.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f783bfff640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 msgr2=0x7f783c1327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:34:20.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:20.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:34:20.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f7840f3a640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:20.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f7840f3a640 10 monclient(hunting): get_auth_request con 0x7f783c09f880 auth_method 0 2024-09-15T01:34:20.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f7840f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:20.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f7840f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f7840f3a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f7840f3a640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 msgr2=0x7f783c11c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:34:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f7840f3a640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 msgr2=0x7f783c11c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:34:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f7840f3a640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:20.612+0000 7f7840f3a640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:34:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f783bfff640 10 monclient(hunting): get_auth_request con 0x7f783c09f0e0 auth_method 0 2024-09-15T01:34:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f783bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f783bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f783bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f783bfff640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 msgr2=0x7f783c1327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:34:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f783bfff640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 msgr2=0x7f783c1327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:34:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:34:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f7840f3a640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f7840f3a640 10 monclient(hunting): get_auth_request con 0x7f783c09f880 auth_method 0 2024-09-15T01:34:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f7840f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f7840f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f7840f3a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f7840f3a640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 msgr2=0x7f783c11c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:34:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f7840f3a640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 msgr2=0x7f783c11c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:34:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f7840f3a640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.013+0000 7f7840f3a640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:34:21.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f783bfff640 10 monclient(hunting): get_auth_request con 0x7f783c09f0e0 auth_method 0 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f783bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f783bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f783bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f783bfff640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 msgr2=0x7f783c1327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f783bfff640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 msgr2=0x7f783c1327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f7840f3a640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f7840f3a640 10 monclient(hunting): get_auth_request con 0x7f783c09f880 auth_method 0 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f7840f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f7840f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f7840f3a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:21.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f7840f3a640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 msgr2=0x7f783c11c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:34:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f7840f3a640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 msgr2=0x7f783c11c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:34:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f7840f3a640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:21.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:21.814+0000 7f7840f3a640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:34:22.443 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-15T01:34:22.443 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-15T01:34:22.443 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:34:22.444 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:34:22.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.443+0000 7f783b7fe640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f783c09f4b0 0x7f783c11c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.443+0000 7f783b7fe640 10 monclient(hunting): get_auth_request con 0x7f783c09f4b0 auth_method 0 2024-09-15T01:34:22.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.443+0000 7f783b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.443+0000 7f783b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.443+0000 7f783b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.443+0000 7f783b7fe640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f783c09f4b0 msgr2=0x7f783c11c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:34:22.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.443+0000 7f783b7fe640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f783c09f4b0 msgr2=0x7f783c11c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:34:22.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.443+0000 7f783b7fe640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f783c09f4b0 0x7f783c11c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:22.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.443+0000 7f783b7fe640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f783c09f4b0 0x7f783c11c440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:22.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.555+0000 7f2adab9d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:22.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.555+0000 7f2adab9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:22.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.555+0000 7f2adab9d640 10 monclient: monmap: 2024-09-15T01:34:22.558 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:22.556811+0000 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:22.556811+0000 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:22.559 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:22.560 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:22.560 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:22.560 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.556+0000 7f2adab9d640 1 Processor -- start 2024-09-15T01:34:22.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.556+0000 7f2adab9d640 1 -- start start 2024-09-15T01:34:22.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.556+0000 7f2adab9d640 10 monclient: init 2024-09-15T01:34:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 10 monclient: _add_conns ranks=[1,10,8,5,3,0,6,7,4,2,9] 2024-09-15T01:34:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2ad40a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2ad4132de0 0x7f2ad4137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 10 monclient(hunting): picked mon.noname-f con 0x7f2ad4132de0 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ad41380e0 0x7f2ad413a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 10 monclient(hunting): picked mon.noname-e con 0x7f2ad41380e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad404bf60 con 0x7f2ad40a2930 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad409dd80 con 0x7f2ad4132de0 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad40a2d00 con 0x7f2ad41380e0 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2adab9d640 10 monclient(hunting): authenticate will time out at 867.307327s 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ad41380e0 0x7f2ad413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ad41380e0 0x7f2ad413a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54144/0 (socket says 172.21.15.178:54144) 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 1 -- 172.21.15.178:0/3421197819 learned_addr learned my addr 172.21.15.178:0/3421197819 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 10 monclient(hunting): get_auth_request con 0x7f2ad41380e0 auth_method 0 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad3fff640 1 --2- 172.21.15.178:0/3421197819 >> v2:172.21.15.204:3300/0 conn(0x7f2ad4132de0 0x7f2ad4137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 10 monclient(hunting): handle_auth_done global_id 5075 payload 274 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 1 -- 172.21.15.178:0/3421197819 >> v2:172.21.15.204:3300/0 conn(0x7f2ad4132de0 msgr2=0x7f2ad4137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 1 --2- 172.21.15.178:0/3421197819 >> v2:172.21.15.204:3300/0 conn(0x7f2ad4132de0 0x7f2ad4137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 1 -- 172.21.15.178:0/3421197819 >> v1:172.21.15.204:6789/0 conn(0x7f2ad40a2930 legacy=0x7f2ad409faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 10 monclient: _finish_hunting 0 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 1 monclient: found mon.noname-e 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad913a640 1 -- 172.21.15.178:0/3421197819 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad41331b0 con 0x7f2ad41380e0 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad3fff640 10 monclient: get_auth_request con 0x7f2ad4132de0 auth_method 0 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.557+0000 7f2ad3fff640 1 --2- 172.21.15.178:0/3421197819 >> v2:172.21.15.204:3300/0 conn(0x7f2ad4132de0 0x7f2ad4137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad913a640 1 --2- 172.21.15.178:0/3421197819 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ad41380e0 0x7f2ad413a4c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009f20 tx=0x7f2acc009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6d77fcf78b237e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad37fe640 1 -- 172.21.15.178:0/3421197819 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2acc007880 con 0x7f2ad41380e0 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad37fe640 10 monclient: dump: 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad37fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad37fe640 1 -- 172.21.15.178:0/3421197819 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2acc02d050 con 0x7f2ad41380e0 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2adab9d640 5 monclient: authenticate success, global_id 5075 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2adab9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad37fe640 1 -- 172.21.15.178:0/3421197819 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2acc030360 con 0x7f2ad41380e0 2024-09-15T01:34:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2adab9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2adab9d640 4 set_mon_vals no callback set 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.558+0000 7f2ad37fe640 10 monclient: dump: 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.559+0000 7f2adab9d640 10 monclient: shutdown 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.559+0000 7f2adab9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.559+0000 7f2adab9d640 1 -- 172.21.15.178:0/3421197819 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ad41380e0 msgr2=0x7f2ad413a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.559+0000 7f2adab9d640 1 --2- 172.21.15.178:0/3421197819 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ad41380e0 0x7f2ad413a4c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009f20 tx=0x7f2acc009840 comp rx=0 tx=0).stop 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.559+0000 7f2adab9d640 1 -- 172.21.15.178:0/3421197819 shutdown_connections 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.559+0000 7f2adab9d640 1 --2- 172.21.15.178:0/3421197819 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2ad41380e0 0x7f2ad413a4c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.559+0000 7f2adab9d640 1 --2- 172.21.15.178:0/3421197819 >> v2:172.21.15.204:3300/0 conn(0x7f2ad4132de0 0x7f2ad4137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.559+0000 7f2adab9d640 1 -- 172.21.15.178:0/3421197819 >> 172.21.15.178:0/3421197819 conn(0x7f2ad40a5ed0 msgr2=0x7f2ad40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.559+0000 7f2adab9d640 1 -- 172.21.15.178:0/3421197819 shutdown_connections 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.559+0000 7f2adab9d640 1 -- 172.21.15.178:0/3421197819 wait complete. 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.559+0000 7f2adab9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.559+0000 7f2adab9d640 10 monclient: monmap: 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 1 Processor -- start 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 1 -- start start 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 10 monclient: init 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 10 monclient: _add_conns ranks=[5,2,1,4,8,0,7,6,3] 2024-09-15T01:34:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2ad40a2930 0x7f2ad4132500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 10 monclient(hunting): picked mon.noname-h con 0x7f2ad40a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ad4132de0 0x7f2ad409a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 10 monclient(hunting): picked mon.noname-b con 0x7f2ad4132de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ad41380e0 0x7f2ad409ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 10 monclient(hunting): picked mon.noname-f con 0x7f2ad41380e0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad40a2390 con 0x7f2ad41380e0 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad409d630 con 0x7f2ad4132de0 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad411d770 con 0x7f2ad40a2930 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2adab9d640 10 monclient(hunting): authenticate will time out at 867.310778s 2024-09-15T01:34:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ad4132de0 0x7f2ad409a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad913a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ad41380e0 0x7f2ad409ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ad4132de0 0x7f2ad409a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42894/0 (socket says 172.21.15.178:42894) 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad8939640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2ad40a2930 0x7f2ad4132500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad3fff640 1 -- 172.21.15.178:0/3880995429 learned_addr learned my addr 172.21.15.178:0/3880995429 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad3fff640 10 monclient(hunting): get_auth_request con 0x7f2ad4132de0 auth_method 0 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad913a640 10 monclient(hunting): get_auth_request con 0x7f2ad41380e0 auth_method 0 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad913a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad913a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad913a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad8939640 10 monclient(hunting): get_auth_request con 0x7f2ad40a2930 auth_method 0 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad8939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad8939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.560+0000 7f2ad8939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad913a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad913a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad913a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad8939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad8939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad8939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 10 monclient(hunting): handle_auth_done global_id 4943 payload 824 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 1 -- 172.21.15.178:0/3880995429 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2ad40a2930 msgr2=0x7f2ad4132500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 1 --2- 172.21.15.178:0/3880995429 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2ad40a2930 0x7f2ad4132500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 1 -- 172.21.15.178:0/3880995429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ad41380e0 msgr2=0x7f2ad409ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 1 --2- 172.21.15.178:0/3880995429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ad41380e0 0x7f2ad409ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 1 monclient: found mon.noname-b 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 1 -- 172.21.15.178:0/3880995429 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2acc009ae0 con 0x7f2ad4132de0 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad3fff640 1 --2- 172.21.15.178:0/3880995429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ad4132de0 0x7f2ad409a6c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00c880 tx=0x7f2abc008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad0ff9640 1 -- 172.21.15.178:0/3880995429 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2abc00ce10 con 0x7f2ad4132de0 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad0ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad0ff9640 10 monclient: dump: 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:22.586 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad0ff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2adab9d640 5 monclient: authenticate success, global_id 4943 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad0ff9640 1 -- 172.21.15.178:0/3880995429 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2abc007d10 con 0x7f2ad4132de0 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2adab9d640 10 monclient: _renew_subs 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2adab9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2adab9d640 1 -- 172.21.15.178:0/3880995429 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad411e230 con 0x7f2ad4132de0 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad0ff9640 1 -- 172.21.15.178:0/3880995429 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2abc007e80 con 0x7f2ad4132de0 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad2ffd640 4 set_mon_vals no callback set 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2ad0ff9640 10 monclient: dump: 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:22.588 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2adab9d640 10 monclient: _renew_subs 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2adab9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.561+0000 7f2adab9d640 1 -- 172.21.15.178:0/3880995429 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad411d490 con 0x7f2ad4132de0 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.562+0000 7f2ad0ff9640 1 -- 172.21.15.178:0/3880995429 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2abc007890 con 0x7f2ad4132de0 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.562+0000 7f2adab9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.562+0000 7f2adab9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.562+0000 7f2adab9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.562+0000 7f2adab9d640 1 -- 172.21.15.178:0/3880995429 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad411d920 con 0x7f2ad4132de0 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.562+0000 7f2ad0ff9640 1 --2- 172.21.15.178:0/3880995429 >> v2:172.21.15.204:6812/28210 conn(0x7f2ab40397b0 0x7f2ab403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.562+0000 7f2ad0ff9640 1 -- 172.21.15.178:0/3880995429 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7f2abc04b6e0 con 0x7f2ad4132de0 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.562+0000 7f2ad8939640 1 --2- 172.21.15.178:0/3880995429 >> v2:172.21.15.204:6812/28210 conn(0x7f2ab40397b0 0x7f2ab403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.562+0000 7f2ad8939640 10 monclient: get_auth_request con 0x7f2ab40397b0 auth_method 0 2024-09-15T01:34:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.563+0000 7f2ad8939640 1 --2- 172.21.15.178:0/3880995429 >> v2:172.21.15.204:6812/28210 conn(0x7f2ab40397b0 0x7f2ab403bc60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009a60 tx=0x7f2ac4006f50 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:22.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.566+0000 7f2ad0ff9640 1 -- 172.21.15.178:0/3880995429 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2ad411d920 con 0x7f2ad4132de0 2024-09-15T01:34:22.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.566+0000 7f2ad0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:22.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.566+0000 7f2ad0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:22.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f783c09f4b0 0x7f783c11c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 10 monclient(hunting): get_auth_request con 0x7f783c09f4b0 auth_method 0 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 10 monclient(hunting): handle_auth_done global_id 4952 payload 824 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 msgr2=0x7f783c11c980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 msgr2=0x7f783c1327c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 1 monclient: found mon.noname-b 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 1 -- 172.21.15.204:0/1479839062 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c009450 con 0x7f783c09f4b0 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783b7fe640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f783c09f4b0 0x7f783c11c440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f783000e590 tx=0x7f7830009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f7823fff640 1 -- 172.21.15.204:0/1479839062 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7830017070 con 0x7f783c09f4b0 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f7823fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f7823fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f7823fff640 10 monclient: dump: 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f7823fff640 10 monclient: _finish_auth 0 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f7823fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f7823fff640 1 -- 172.21.15.204:0/1479839062 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7830012ae0 con 0x7f783c09f4b0 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f784299d640 5 monclient: authenticate success, global_id 4952 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f784299d640 10 monclient: _renew_subs 2024-09-15T01:34:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f784299d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f784299d640 1 -- 172.21.15.204:0/1479839062 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783c0685c0 con 0x7f783c09f4b0 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f7823fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f7823fff640 1 -- 172.21.15.204:0/1479839062 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7830012c50 con 0x7f783c09f4b0 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f7823fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f7823fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f7823fff640 10 monclient: dump: 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:22.652 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f783a7fc640 4 set_mon_vals no callback set 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f784299d640 10 monclient: _renew_subs 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f784299d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.644+0000 7f784299d640 1 -- 172.21.15.204:0/1479839062 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783c11e2c0 con 0x7f783c09f4b0 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.645+0000 7f7823fff640 1 -- 172.21.15.204:0/1479839062 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7830012660 con 0x7f783c09f4b0 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.645+0000 7f7823fff640 1 --2- 172.21.15.204:0/1479839062 >> v2:172.21.15.204:6812/28210 conn(0x7f78280397b0 0x7f782803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.645+0000 7f784299d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.645+0000 7f7823fff640 1 -- 172.21.15.204:0/1479839062 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7f783004ac70 con 0x7f783c09f4b0 2024-09-15T01:34:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.645+0000 7f784299d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.645+0000 7f784299d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.645+0000 7f784299d640 1 -- 172.21.15.204:0/1479839062 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783c11de10 con 0x7f783c09f4b0 2024-09-15T01:34:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.645+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> v2:172.21.15.204:6812/28210 conn(0x7f78280397b0 0x7f782803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.645+0000 7f783bfff640 10 monclient: get_auth_request con 0x7f78280397b0 auth_method 0 2024-09-15T01:34:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.646+0000 7f783bfff640 1 --2- 172.21.15.204:0/1479839062 >> v2:172.21.15.204:6812/28210 conn(0x7f78280397b0 0x7f782803bc60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f782c009940 tx=0x7f782c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.650+0000 7f7823fff640 1 -- 172.21.15.204:0/1479839062 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7830012910 con 0x7f783c09f4b0 2024-09-15T01:34:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.650+0000 7f7823fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.650+0000 7f7823fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:22.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.782+0000 7f2adab9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:34:22.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.782+0000 7f2adab9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:34:22.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.782+0000 7f2adab9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:22.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.782+0000 7f2adab9d640 1 -- 172.21.15.178:0/3880995429 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2ad404b720 con 0x7f2ad4132de0 2024-09-15T01:34:22.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.783+0000 7f2ad0ff9640 1 -- 172.21.15.178:0/3880995429 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7f2abc01ec80 con 0x7f2ad4132de0 2024-09-15T01:34:22.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.783+0000 7f2ad0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:34:22.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.783+0000 7f2ad0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:22.785 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":38,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:34:22.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 1 -- 172.21.15.178:0/3880995429 >> v2:172.21.15.204:6812/28210 conn(0x7f2ab40397b0 msgr2=0x7f2ab403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 1 --2- 172.21.15.178:0/3880995429 >> v2:172.21.15.204:6812/28210 conn(0x7f2ab40397b0 0x7f2ab403bc60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009a60 tx=0x7f2ac4006f50 comp rx=0 tx=0).stop 2024-09-15T01:34:22.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 10 monclient: shutdown 2024-09-15T01:34:22.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:22.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 1 -- 172.21.15.178:0/3880995429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ad4132de0 msgr2=0x7f2ad409a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 1 --2- 172.21.15.178:0/3880995429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ad4132de0 0x7f2ad409a6c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00c880 tx=0x7f2abc008040 comp rx=0 tx=0).stop 2024-09-15T01:34:22.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 1 -- 172.21.15.178:0/3880995429 shutdown_connections 2024-09-15T01:34:22.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 1 --2- 172.21.15.178:0/3880995429 >> v2:172.21.15.204:6812/28210 conn(0x7f2ab40397b0 0x7f2ab403bc60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 1 --2- 172.21.15.178:0/3880995429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ad41380e0 0x7f2ad409ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 1 --2- 172.21.15.178:0/3880995429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ad4132de0 0x7f2ad409a6c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 1 --2- 172.21.15.178:0/3880995429 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2ad40a2930 0x7f2ad4132500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 1 -- 172.21.15.178:0/3880995429 >> 172.21.15.178:0/3880995429 conn(0x7f2ad40a5ed0 msgr2=0x7f2ad41358c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:22.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 1 -- 172.21.15.178:0/3880995429 shutdown_connections 2024-09-15T01:34:22.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.785+0000 7f2adab9d640 1 -- 172.21.15.178:0/3880995429 wait complete. 2024-09-15T01:34:22.797 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:34:22.797 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:34:22.797 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:34:22.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.812+0000 7f784299d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:22.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.813+0000 7f784299d640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:22.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.813+0000 7f784299d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:22.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.813+0000 7f784299d640 1 -- 172.21.15.204:0/1479839062 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f783c070b80 con 0x7f783c09f4b0 2024-09-15T01:34:22.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.813+0000 7f7823fff640 1 -- 172.21.15.204:0/1479839062 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v77) v1 ==== 56+0+1474 (secure 0 0 0) 0x7f7830011ad0 con 0x7f783c09f4b0 2024-09-15T01:34:22.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.813+0000 7f7823fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:22.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.813+0000 7f7823fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:22.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.888+0000 7f478f6c9640 10 monclient: get_monmap_and_config 2024-09-15T01:34:22.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.888+0000 7f478f6c9640 10 monclient: build_initial_monmap 2024-09-15T01:34:22.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.888+0000 7f478f6c9640 10 monclient: monmap: 2024-09-15T01:34:22.890 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:22.889582+0000 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:22.889582+0000 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:22.891 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:22.892 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:22.892 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:22.892 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 1 Processor -- start 2024-09-15T01:34:22.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 1 -- start start 2024-09-15T01:34:22.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 10 monclient: init 2024-09-15T01:34:22.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:22.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 10 monclient: _add_conns ranks=[8,10,3,0,9,4,2,7,6,1,5] 2024-09-15T01:34:22.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f47880a2930 0x7f4788132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 10 monclient(hunting): picked mon.noname-e con 0x7f47880a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:22.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f478809faa0 0x7f478809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 10 monclient(hunting): picked mon.noname-f con 0x7f478809faa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47880a08c0 0x7f47880a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 10 monclient(hunting): picked mon.noname-g con 0x7f47880a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478809e0f0 con 0x7f478809faa0 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478809dd80 con 0x7f47880a08c0 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f47880a2d00 con 0x7f47880a2930 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.889+0000 7f478f6c9640 10 monclient(hunting): authenticate will time out at 867.639946s 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f47880a2930 0x7f4788132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f47880a2930 0x7f4788132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54150/0 (socket says 172.21.15.178:54150) 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 1 -- 172.21.15.178:0/2899886312 learned_addr learned my addr 172.21.15.178:0/2899886312 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:22.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 10 monclient(hunting): get_auth_request con 0x7f47880a2930 auth_method 0 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478dc66640 1 --2- 172.21.15.178:0/2899886312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47880a08c0 0x7f47880a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478cc64640 1 --2- 172.21.15.178:0/2899886312 >> v2:172.21.15.204:3300/0 conn(0x7f478809faa0 0x7f478809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478dc66640 10 monclient(hunting): get_auth_request con 0x7f47880a08c0 auth_method 0 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478dc66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478dc66640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478dc66640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478cc64640 10 monclient(hunting): get_auth_request con 0x7f478809faa0 auth_method 0 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478cc64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478cc64640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478cc64640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478cc64640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478cc64640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478cc64640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478dc66640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478dc66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478dc66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 10 monclient(hunting): handle_auth_done global_id 5084 payload 274 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 1 -- 172.21.15.178:0/2899886312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47880a08c0 msgr2=0x7f47880a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 1 --2- 172.21.15.178:0/2899886312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47880a08c0 0x7f47880a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 1 -- 172.21.15.178:0/2899886312 >> v2:172.21.15.204:3300/0 conn(0x7f478809faa0 msgr2=0x7f478809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:22.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 1 --2- 172.21.15.178:0/2899886312 >> v2:172.21.15.204:3300/0 conn(0x7f478809faa0 0x7f478809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 10 monclient: _finish_hunting 0 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 1 monclient: found mon.noname-e 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.890+0000 7f478d465640 1 -- 172.21.15.178:0/2899886312 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478804bf60 con 0x7f47880a2930 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478d465640 1 --2- 172.21.15.178:0/2899886312 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f47880a2930 0x7f4788132de0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4784009910 tx=0x7f47840091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=435d81b2d26720cb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f477ffff640 1 -- 172.21.15.178:0/2899886312 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f478402ac80 con 0x7f47880a2930 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f477ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f477ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f477ffff640 10 monclient: dump: 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.905 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f477ffff640 10 monclient: _finish_auth 0 2024-09-15T01:34:22.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f477ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f477ffff640 1 -- 172.21.15.178:0/2899886312 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478402ae00 con 0x7f47880a2930 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 5 monclient: authenticate success, global_id 5084 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f477ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f477ffff640 1 -- 172.21.15.178:0/2899886312 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4784031ca0 con 0x7f47880a2930 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 4 set_mon_vals no callback set 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f477ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f477ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f477ffff640 10 monclient: dump: 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.907 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 10 monclient: shutdown 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 1 -- 172.21.15.178:0/2899886312 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f47880a2930 msgr2=0x7f4788132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 1 --2- 172.21.15.178:0/2899886312 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f47880a2930 0x7f4788132de0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4784009910 tx=0x7f47840091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 1 -- 172.21.15.178:0/2899886312 shutdown_connections 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 1 --2- 172.21.15.178:0/2899886312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47880a08c0 0x7f47880a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 1 --2- 172.21.15.178:0/2899886312 >> v2:172.21.15.204:3300/0 conn(0x7f478809faa0 0x7f478809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 1 --2- 172.21.15.178:0/2899886312 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f47880a2930 0x7f4788132de0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 1 -- 172.21.15.178:0/2899886312 >> 172.21.15.178:0/2899886312 conn(0x7f47880a5ed0 msgr2=0x7f47880a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 1 -- 172.21.15.178:0/2899886312 shutdown_connections 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.891+0000 7f478f6c9640 1 -- 172.21.15.178:0/2899886312 wait complete. 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.892+0000 7f478f6c9640 10 monclient: build_initial_monmap 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.892+0000 7f478f6c9640 10 monclient: monmap: 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:22.909 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.892+0000 7f478f6c9640 1 Processor -- start 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.892+0000 7f478f6c9640 1 -- start start 2024-09-15T01:34:22.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.892+0000 7f478f6c9640 10 monclient: init 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 10 monclient: _add_conns ranks=[3,5,1,0,7,6,4,8,2] 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f478809faa0 0x7f478809b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 10 monclient(hunting): picked mon.noname-g con 0x7f478809faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f47880a08c0 0x7f4788131730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 10 monclient(hunting): picked mon.noname-h con 0x7f47880a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47880a2930 0x7f478811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 10 monclient(hunting): picked mon.noname-f con 0x7f47880a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4788142c30 con 0x7f47880a2930 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47880a2390 con 0x7f478809faa0 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f478809d630 con 0x7f47880a08c0 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478f6c9640 10 monclient(hunting): authenticate will time out at 867.643345s 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478d465640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f478809faa0 0x7f478809b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478dc66640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47880a2930 0x7f478811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478d465640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f478809faa0 0x7f478809b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:39160/0 (socket says 172.21.15.178:39160) 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478dc66640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47880a2930 0x7f478811c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:39504/0 (socket says 172.21.15.178:39504) 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f47880a08c0 0x7f4788131730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478d465640 1 -- 172.21.15.178:0/2234376816 learned_addr learned my addr 172.21.15.178:0/2234376816 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478d465640 10 monclient(hunting): get_auth_request con 0x7f478809faa0 auth_method 0 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478d465640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478d465640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478d465640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478dc66640 10 monclient(hunting): get_auth_request con 0x7f47880a2930 auth_method 0 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478dc66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478dc66640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478dc66640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 10 monclient(hunting): get_auth_request con 0x7f47880a08c0 auth_method 0 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478d465640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478d465640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478d465640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478dc66640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478dc66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478dc66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 10 monclient(hunting): handle_auth_done global_id 5027 payload 824 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 1 -- 172.21.15.178:0/2234376816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f478809faa0 msgr2=0x7f478809b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f478809faa0 0x7f478809b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 1 -- 172.21.15.178:0/2234376816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47880a2930 msgr2=0x7f478811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47880a2930 0x7f478811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 10 monclient: _finish_hunting 0 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 1 monclient: found mon.noname-h 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 1 -- 172.21.15.178:0/2234376816 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4784009450 con 0x7f47880a08c0 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478d465640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f478809faa0 0x7f478809b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478dc66640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47880a2930 0x7f478811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.893+0000 7f478cc64640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f47880a08c0 0x7f4788131730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f477800e8d0 tx=0x7f47780090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f477800ee60 con 0x7f47880a08c0 2024-09-15T01:34:22.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477d7fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477d7fa640 10 monclient: dump: 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:22.915 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477d7fa640 10 monclient: _finish_auth 0 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f478f6c9640 5 monclient: authenticate success, global_id 5027 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477800cdc0 con 0x7f47880a08c0 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f478f6c9640 10 monclient: _renew_subs 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f478f6c9640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f478f6c9640 1 -- 172.21.15.178:0/2234376816 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478811e850 con 0x7f47880a08c0 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47780138f0 con 0x7f47880a08c0 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:22.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477d7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477d7fa640 10 monclient: dump: 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:22.917 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f478f6c9640 10 monclient: _renew_subs 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f478f6c9640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f477f7fe640 4 set_mon_vals no callback set 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f478f6c9640 1 -- 172.21.15.178:0/2234376816 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478809bc60 con 0x7f47880a08c0 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.894+0000 7f478f6c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478f6c9640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47881371d0 0x7f478811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478f6c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478f6c9640 1 -- 172.21.15.178:0/2234376816 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478804c4a0 con 0x7f47881371d0 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4778013470 con 0x7f47880a08c0 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478d465640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47881371d0 0x7f478811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478d465640 10 monclient: get_auth_request con 0x7f47881371d0 auth_method 0 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478d465640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478d465640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478d465640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478d465640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478d465640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478d465640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f477d7fa640 1 --2- 172.21.15.178:0/2234376816 >> v2:172.21.15.204:6812/28210 conn(0x7f476c0397e0 0x7f476c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7f477804b6b0 con 0x7f47880a08c0 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478d465640 10 monclient(hunting): handle_auth_done global_id 5112 payload 824 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478d465640 1 -- 172.21.15.178:0/2234376816 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f478805ed70 con 0x7f47881371d0 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478dc66640 1 --2- 172.21.15.178:0/2234376816 >> v2:172.21.15.204:6812/28210 conn(0x7f476c0397e0 0x7f476c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478dc66640 10 monclient: get_auth_request con 0x7f476c0397e0 auth_method 0 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478d465640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47881371d0 0x7f478811ca70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f47840097e0 tx=0x7f4784032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:22.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f478402ac80 con 0x7f47881371d0 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f477d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f477d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f477d7fa640 10 monclient: dump: 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:22.920 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:22.921 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:22.921 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:22.921 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:22.921 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.895+0000 7f478dc66640 1 --2- 172.21.15.178:0/2234376816 >> v2:172.21.15.204:6812/28210 conn(0x7f476c0397e0 0x7f476c03bc90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f478809b8c0 tx=0x7f478000b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:22.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.896+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f4784031930 con 0x7f47881371d0 2024-09-15T01:34:22.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.896+0000 7f477d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:22.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.896+0000 7f477d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:22.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.896+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47881371d0 msgr2=0x7f478811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.896+0000 7f477d7fa640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47881371d0 0x7f478811ca70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f47840097e0 tx=0x7f4784032000 comp rx=0 tx=0).stop 2024-09-15T01:34:22.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.896+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47881371d0 msgr2=0x7f478811ca70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:22.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.896+0000 7f477d7fa640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47881371d0 0x7f478811ca70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.907+0000 7f784299d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:22.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.907+0000 7f784299d640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:22.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.907+0000 7f784299d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:22.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.907+0000 7f784299d640 1 -- 172.21.15.204:0/1479839062 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f783c06e850 con 0x7f783c09f4b0 2024-09-15T01:34:22.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.907+0000 7f7823fff640 1 -- 172.21.15.204:0/1479839062 <== mon.2 v2:172.21.15.178:3301/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v77) v1 ==== 56+0+1474 (secure 0 0 0) 0x7f7830011cb0 con 0x7f783c09f4b0 2024-09-15T01:34:22.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.907+0000 7f7823fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:22.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.907+0000 7f7823fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:22.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 1 -- 172.21.15.204:0/1479839062 >> v2:172.21.15.204:6812/28210 conn(0x7f78280397b0 msgr2=0x7f782803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 1 --2- 172.21.15.204:0/1479839062 >> v2:172.21.15.204:6812/28210 conn(0x7f78280397b0 0x7f782803bc60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f782c009940 tx=0x7f782c032000 comp rx=0 tx=0).stop 2024-09-15T01:34:22.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 10 monclient: shutdown 2024-09-15T01:34:22.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:22.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 1 -- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f783c09f4b0 msgr2=0x7f783c11c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f783c09f4b0 0x7f783c11c440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f783000e590 tx=0x7f7830009040 comp rx=0 tx=0).stop 2024-09-15T01:34:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 1 -- 172.21.15.204:0/1479839062 shutdown_connections 2024-09-15T01:34:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 1 --2- 172.21.15.204:0/1479839062 >> v2:172.21.15.204:6812/28210 conn(0x7f78280397b0 0x7f782803bc60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f783c09f880 0x7f783c11c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f783c09f4b0 0x7f783c11c440 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 1 --2- 172.21.15.204:0/1479839062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f783c09f0e0 0x7f783c1327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 1 -- 172.21.15.204:0/1479839062 >> 172.21.15.204:0/1479839062 conn(0x7f783c067c00 msgr2=0x7f783c06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 1 -- 172.21.15.204:0/1479839062 shutdown_connections 2024-09-15T01:34:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.909+0000 7f784299d640 1 -- 172.21.15.204:0/1479839062 wait complete. 2024-09-15T01:34:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush rm host1 foo 2024-09-15T01:34:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:34:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm host1 foo 2024-09-15T01:34:22.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478f6c9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:22.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478f6c9640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4788131c70 0x7f478811cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:22.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478f6c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:22.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478f6c9640 1 -- 172.21.15.178:0/2234376816 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478804bc80 con 0x7f4788131c70 2024-09-15T01:34:22.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478d465640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4788131c70 0x7f478811cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:22.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478d465640 10 monclient: get_auth_request con 0x7f4788131c70 auth_method 0 2024-09-15T01:34:22.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478d465640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:22.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478d465640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478d465640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478d465640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478d465640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478d465640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478d465640 10 monclient(hunting): handle_auth_done global_id 5121 payload 824 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.901+0000 7f478d465640 1 -- 172.21.15.178:0/2234376816 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f478804b720 con 0x7f4788131c70 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f478d465640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4788131c70 0x7f478811cfb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f478804c5e0 tx=0x7f478403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f478403ce20 con 0x7f4788131c70 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f477d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f477d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f477d7fa640 10 monclient: dump: 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.927 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:22.928 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:22.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f478403b3d0 con 0x7f4788131c70 2024-09-15T01:34:22.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f477d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:22.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f477d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:22.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4788131c70 msgr2=0x7f478811cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f477d7fa640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4788131c70 0x7f478811cfb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f478804c5e0 tx=0x7f478403d000 comp rx=0 tx=0).stop 2024-09-15T01:34:22.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f477d7fa640 1 -- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4788131c70 msgr2=0x7f478811cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:22.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f477d7fa640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4788131c70 0x7f478811cfb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.929 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:22.930 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:22.930 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:22.930 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:22.930 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:22.930 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:22.930 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:22.930 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:22.930 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:22.930 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:22.930 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:22.930 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:22.930 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:22.931 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:22.932 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:22.933 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:22.934 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.935 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:22.936 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:22.937 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:22.937 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:22.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.937 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:22.937 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:22.937 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:22.937 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:22.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:22.938 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.939 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.941 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:22.942 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:22.943 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:22.945 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:22.946 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:22.947 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.948 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:22.949 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:22.949 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:22.949 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:22.949 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:22.949 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.949 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:22.949 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:22.949 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:22.949 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:22.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.952 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:22.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:22.952 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.952 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:22.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:22.952 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:22.952 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:22.952 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:22.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:22.953 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:22.953 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:22.953 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:22.953 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:22.953 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:22.953 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:22.953 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.953 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:22.953 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:22.953 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.953 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:22.954 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:22.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f478f6c9640 1 -- 172.21.15.178:0/2234376816 >> v2:172.21.15.204:6812/28210 conn(0x7f476c0397e0 msgr2=0x7f476c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f478f6c9640 1 --2- 172.21.15.178:0/2234376816 >> v2:172.21.15.204:6812/28210 conn(0x7f476c0397e0 0x7f476c03bc90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f478809b8c0 tx=0x7f478000b040 comp rx=0 tx=0).stop 2024-09-15T01:34:22.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f478f6c9640 10 monclient: shutdown 2024-09-15T01:34:22.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f478f6c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:22.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f478f6c9640 1 -- 172.21.15.178:0/2234376816 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f47880a08c0 msgr2=0x7f4788131730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:22.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f478f6c9640 1 --2- 172.21.15.178:0/2234376816 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f47880a08c0 0x7f4788131730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f477800e8d0 tx=0x7f47780090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:22.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f478dc66640 1 -- 172.21.15.178:0/2234376816 reap_dead start 2024-09-15T01:34:22.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f478dc66640 1 -- 172.21.15.178:0/2234376816 reap_dead start 2024-09-15T01:34:22.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f478f6c9640 1 -- 172.21.15.178:0/2234376816 shutdown_connections 2024-09-15T01:34:22.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.902+0000 7f478f6c9640 1 -- 172.21.15.178:0/2234376816 >> 172.21.15.178:0/2234376816 conn(0x7f47880a5ed0 msgr2=0x7f478809e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:22.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.903+0000 7f478f6c9640 1 -- 172.21.15.178:0/2234376816 shutdown_connections 2024-09-15T01:34:22.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:22.903+0000 7f478f6c9640 1 -- 172.21.15.178:0/2234376816 wait complete. 2024-09-15T01:34:22.957 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:22.957 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:34:22.957 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.993+0000 7f6b96b5d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.993+0000 7f6b96b5d640 10 monclient: build_initial_monmap 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.993+0000 7f6b96b5d640 10 monclient: monmap: 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:22.995429+0000 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:22.995429+0000 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:22.997 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:22.998 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:22.998 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:22.998 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:22.998 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:22.998 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:22.998 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:22.998 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:22.998 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.994+0000 7f6b96b5d640 1 Processor -- start 2024-09-15T01:34:23.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.994+0000 7f6b96b5d640 1 -- start start 2024-09-15T01:34:23.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.994+0000 7f6b96b5d640 10 monclient: init 2024-09-15T01:34:23.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 10 monclient: _add_conns ranks=[5,2,0,10,6,9,8,4,3,7,1] 2024-09-15T01:34:23.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6b900a2e30 0x7f6b90132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 10 monclient(hunting): picked mon.noname-h con 0x7f6b900a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:23.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9009fb80 0x7f6b9009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 10 monclient(hunting): picked mon.noname-b con 0x7f6b9009fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:23.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6b900a0a40 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:23.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b9004b720 con 0x7f6b900a0a40 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9009e6c0 con 0x7f6b9009fb80 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900a04b0 con 0x7f6b900a2e30 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b96b5d640 10 monclient(hunting): authenticate will time out at 878.638282s 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6b900a2e30 0x7f6b90132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6b900a2e30 0x7f6b90132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:55714/0 (socket says 172.21.15.204:55714) 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 1 -- 172.21.15.204:0/2469016053 learned_addr learned my addr 172.21.15.204:0/2469016053 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 10 monclient(hunting): get_auth_request con 0x7f6b900a2e30 auth_method 0 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b8ffff640 1 --2- 172.21.15.204:0/2469016053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9009fb80 0x7f6b9009ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b8ffff640 10 monclient(hunting): get_auth_request con 0x7f6b9009fb80 auth_method 0 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 10 monclient(hunting): handle_auth_done global_id 5036 payload 274 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 1 -- 172.21.15.204:0/2469016053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9009fb80 msgr2=0x7f6b9009ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 1 --2- 172.21.15.204:0/2469016053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9009fb80 0x7f6b9009ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 1 -- 172.21.15.204:0/2469016053 >> v1:172.21.15.178:6789/0 conn(0x7f6b900a0a40 legacy=0x7f6b900a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 1 monclient: found mon.noname-h 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b948f9640 1 -- 172.21.15.204:0/2469016053 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b900a3200 con 0x7f6b900a2e30 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.995+0000 7f6b8ffff640 1 --2- 172.21.15.204:0/2469016053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9009fb80 0x7f6b9009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b948f9640 1 --2- 172.21.15.204:0/2469016053 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6b900a2e30 0x7f6b90132a60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009940 tx=0x7f6b800091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=2bc5cdd55f82706a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b8f7fe640 1 -- 172.21.15.204:0/2469016053 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6b8002ad00 con 0x7f6b900a2e30 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b8f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b8f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b8f7fe640 10 monclient: dump: 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.008 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b8f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b8f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b8f7fe640 1 -- 172.21.15.204:0/2469016053 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8002ae80 con 0x7f6b900a2e30 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b96b5d640 5 monclient: authenticate success, global_id 5036 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b96b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b8f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b96b5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b8f7fe640 1 -- 172.21.15.204:0/2469016053 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6b80031ca0 con 0x7f6b900a2e30 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b8f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b96b5d640 4 set_mon_vals no callback set 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b8f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.996+0000 7f6b8f7fe640 10 monclient: dump: 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.010 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 10 monclient: shutdown 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 1 -- 172.21.15.204:0/2469016053 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6b900a2e30 msgr2=0x7f6b90132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 1 --2- 172.21.15.204:0/2469016053 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6b900a2e30 0x7f6b90132a60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009940 tx=0x7f6b800091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 1 -- 172.21.15.204:0/2469016053 shutdown_connections 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 1 --2- 172.21.15.204:0/2469016053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9009fb80 0x7f6b9009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 1 --2- 172.21.15.204:0/2469016053 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6b900a2e30 0x7f6b90132a60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 1 -- 172.21.15.204:0/2469016053 >> 172.21.15.204:0/2469016053 conn(0x7f6b9006e860 msgr2=0x7f6b9006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 1 -- 172.21.15.204:0/2469016053 shutdown_connections 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 1 -- 172.21.15.204:0/2469016053 wait complete. 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 10 monclient: monmap: 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:23.012 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.997+0000 7f6b96b5d640 1 Processor -- start 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 1 -- start start 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 10 monclient: init 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 10 monclient: _add_conns ranks=[2,4,3,1,0,6,8,5,7] 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9009fb80 0x7f6b90141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 10 monclient(hunting): picked mon.noname-b con 0x7f6b9009fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6b900a0a40 0x7f6b90131980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 10 monclient(hunting): picked mon.noname-c con 0x7f6b900a0a40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b900a2e30 0x7f6b9011e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 10 monclient(hunting): picked mon.noname-g con 0x7f6b900a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90120840 con 0x7f6b9009fb80 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9009ca40 con 0x7f6b900a2e30 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9009d650 con 0x7f6b900a0a40 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b96b5d640 10 monclient(hunting): authenticate will time out at 878.641592s 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b950fa640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b900a2e30 0x7f6b9011e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b948f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9009fb80 0x7f6b90141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b950fa640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b900a2e30 0x7f6b9011e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54536/0 (socket says 172.21.15.204:54536) 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b8ffff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6b900a0a40 0x7f6b90131980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b950fa640 1 -- 172.21.15.204:0/3550434123 learned_addr learned my addr 172.21.15.204:0/3550434123 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b950fa640 10 monclient(hunting): get_auth_request con 0x7f6b900a2e30 auth_method 0 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b950fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b950fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b950fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b948f9640 1 --2- 172.21.15.204:0/3550434123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9009fb80 0x7f6b90141bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:58892/0 (socket says 172.21.15.204:58892) 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b948f9640 10 monclient(hunting): get_auth_request con 0x7f6b9009fb80 auth_method 0 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b948f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b948f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b948f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b8ffff640 10 monclient(hunting): get_auth_request con 0x7f6b900a0a40 auth_method 0 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b950fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b950fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b950fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b948f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b948f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b948f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.998+0000 7f6b8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b950fa640 10 monclient(hunting): handle_auth_done global_id 14124 payload 824 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b950fa640 1 -- 172.21.15.204:0/3550434123 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6b900a0a40 msgr2=0x7f6b90131980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b950fa640 1 --2- 172.21.15.204:0/3550434123 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6b900a0a40 0x7f6b90131980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b950fa640 1 -- 172.21.15.204:0/3550434123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9009fb80 msgr2=0x7f6b90141bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b950fa640 1 --2- 172.21.15.204:0/3550434123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b9009fb80 0x7f6b90141bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b950fa640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b950fa640 1 monclient: found mon.noname-g 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b950fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b950fa640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b950fa640 1 -- 172.21.15.204:0/3550434123 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b80009450 con 0x7f6b900a2e30 2024-09-15T01:34:23.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b950fa640 1 --2- 172.21.15.204:0/3550434123 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b900a2e30 0x7f6b9011e4c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800c8c0 tx=0x7f6b88001690 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8cff9640 1 -- 172.21.15.204:0/3550434123 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6b8800acd0 con 0x7f6b900a2e30 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8cff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8cff9640 10 monclient: dump: 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.018 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8cff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8cff9640 1 -- 172.21.15.204:0/3550434123 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8800ae70 con 0x7f6b900a2e30 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b96b5d640 5 monclient: authenticate success, global_id 14124 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b96b5d640 10 monclient: _renew_subs 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b96b5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b96b5d640 1 -- 172.21.15.204:0/3550434123 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b90142100 con 0x7f6b900a2e30 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8cff9640 1 -- 172.21.15.204:0/3550434123 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6b880128a0 con 0x7f6b900a2e30 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8cff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8cff9640 10 monclient: dump: 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b96b5d640 10 monclient: _renew_subs 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b96b5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b96b5d640 1 -- 172.21.15.204:0/3550434123 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b90120010 con 0x7f6b900a2e30 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:22.999+0000 7f6b8effd640 4 set_mon_vals no callback set 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.000+0000 7f6b8cff9640 1 -- 172.21.15.204:0/3550434123 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6b88003030 con 0x7f6b900a2e30 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.000+0000 7f6b8cff9640 1 --2- 172.21.15.204:0/3550434123 >> v2:172.21.15.204:6812/28210 conn(0x7f6b7c0397b0 0x7f6b7c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.000+0000 7f6b96b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.000+0000 7f6b96b5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.000+0000 7f6b96b5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.000+0000 7f6b8cff9640 1 -- 172.21.15.204:0/3550434123 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7f6b8804b520 con 0x7f6b900a2e30 2024-09-15T01:34:23.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.000+0000 7f6b96b5d640 1 -- 172.21.15.204:0/3550434123 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b90070cc0 con 0x7f6b900a2e30 2024-09-15T01:34:23.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.000+0000 7f6b948f9640 1 --2- 172.21.15.204:0/3550434123 >> v2:172.21.15.204:6812/28210 conn(0x7f6b7c0397b0 0x7f6b7c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.000+0000 7f6b948f9640 10 monclient: get_auth_request con 0x7f6b7c0397b0 auth_method 0 2024-09-15T01:34:23.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.000+0000 7f6b948f9640 1 --2- 172.21.15.204:0/3550434123 >> v2:172.21.15.204:6812/28210 conn(0x7f6b7c0397b0 0x7f6b7c03bc60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6b80034000 tx=0x7f6b80032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.005+0000 7f6b8cff9640 1 -- 172.21.15.204:0/3550434123 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6b880214d0 con 0x7f6b900a2e30 2024-09-15T01:34:23.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.005+0000 7f6b8cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.005+0000 7f6b8cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.050+0000 7f1eb6f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.050+0000 7f1eb6f9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.050+0000 7f1eb6f9d640 10 monclient: monmap: 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:23.050987+0000 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:23.050987+0000 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.052 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.053 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.053 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.053 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.053 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.053 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.053 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:23.053 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:23.053 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.050+0000 7f1eb6f9d640 1 Processor -- start 2024-09-15T01:34:23.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.050+0000 7f1eb6f9d640 1 -- start start 2024-09-15T01:34:23.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.050+0000 7f1eb6f9d640 10 monclient: init 2024-09-15T01:34:23.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 10 monclient: _add_conns ranks=[8,6,10,3,7,9,4,5,1,0,2] 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1eb00678d0 0x7f1eb009e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f1eb00678d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1eb006ea30 0x7f1eb009ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 10 monclient(hunting): picked mon.noname-d con 0x7f1eb006ea30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1eb009f180 0x7f1eb01306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 10 monclient(hunting): picked mon.noname-f con 0x7f1eb009f180 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb009f590 con 0x7f1eb009f180 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb006d5b0 con 0x7f1eb006ea30 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb0067ca0 con 0x7f1eb00678d0 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb6f9d640 10 monclient(hunting): authenticate will time out at 867.801430s 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb4d39640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1eb00678d0 0x7f1eb009e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1eb006ea30 0x7f1eb009ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb4d39640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1eb00678d0 0x7f1eb009e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54166/0 (socket says 172.21.15.178:54166) 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb4d39640 1 -- 172.21.15.178:0/324191921 learned_addr learned my addr 172.21.15.178:0/324191921 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb4d39640 10 monclient(hunting): get_auth_request con 0x7f1eb00678d0 auth_method 0 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb4d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb4d39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb4d39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 10 monclient(hunting): get_auth_request con 0x7f1eb006ea30 auth_method 0 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb4d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb4d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb4d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb553a640 1 --2- 172.21.15.178:0/324191921 >> v2:172.21.15.204:3300/0 conn(0x7f1eb009f180 0x7f1eb01306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb553a640 10 monclient(hunting): get_auth_request con 0x7f1eb009f180 auth_method 0 2024-09-15T01:34:23.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb553a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb553a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb553a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 10 monclient(hunting): handle_auth_done global_id 14109 payload 274 2024-09-15T01:34:23.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 1 -- 172.21.15.178:0/324191921 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1eb00678d0 msgr2=0x7f1eb009e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 1 --2- 172.21.15.178:0/324191921 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1eb00678d0 0x7f1eb009e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 1 -- 172.21.15.178:0/324191921 >> v2:172.21.15.204:3300/0 conn(0x7f1eb009f180 msgr2=0x7f1eb01306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 1 --2- 172.21.15.178:0/324191921 >> v2:172.21.15.204:3300/0 conn(0x7f1eb009f180 0x7f1eb01306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 1 monclient: found mon.noname-d 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eaffff640 1 -- 172.21.15.178:0/324191921 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb004bf60 con 0x7f1eb006ea30 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.051+0000 7f1eb4d39640 1 --2- 172.21.15.178:0/324191921 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1eb00678d0 0x7f1eb009e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaffff640 1 --2- 172.21.15.178:0/324191921 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1eb006ea30 0x7f1eb009ec40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400f330 tx=0x7f1ea400db90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=6bf6cf548171e2ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaf7fe640 1 -- 172.21.15.178:0/324191921 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1ea400b750 con 0x7f1eb006ea30 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaf7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:34:23.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaf7fe640 10 monclient: dump: 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.066 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaf7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaf7fe640 1 -- 172.21.15.178:0/324191921 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea400b8d0 con 0x7f1eb006ea30 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eb6f9d640 5 monclient: authenticate success, global_id 14109 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eb6f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eb6f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaf7fe640 1 -- 172.21.15.178:0/324191921 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1ea4033430 con 0x7f1eb006ea30 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eb6f9d640 4 set_mon_vals no callback set 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaf7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.052+0000 7f1eaf7fe640 10 monclient: dump: 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.067 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 10 monclient: shutdown 2024-09-15T01:34:23.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/324191921 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1eb006ea30 msgr2=0x7f1eb009ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 1 --2- 172.21.15.178:0/324191921 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1eb006ea30 0x7f1eb009ec40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400f330 tx=0x7f1ea400db90 comp rx=0 tx=0).stop 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/324191921 shutdown_connections 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 1 --2- 172.21.15.178:0/324191921 >> v2:172.21.15.204:3300/0 conn(0x7f1eb009f180 0x7f1eb01306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 1 --2- 172.21.15.178:0/324191921 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1eb006ea30 0x7f1eb009ec40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 1 --2- 172.21.15.178:0/324191921 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1eb00678d0 0x7f1eb009e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/324191921 >> 172.21.15.178:0/324191921 conn(0x7f1eb00a4390 msgr2=0x7f1eb0067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/324191921 shutdown_connections 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/324191921 wait complete. 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.053+0000 7f1eb6f9d640 10 monclient: monmap: 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:23.069 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 1 Processor -- start 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 1 -- start start 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 10 monclient: init 2024-09-15T01:34:23.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 10 monclient: _add_conns ranks=[0,2,8,3,1,7,6,4,5] 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eb00678d0 0x7f1eb011c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 10 monclient(hunting): picked mon.noname-a con 0x7f1eb00678d0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb006ea30 0x7f1eb0139630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f1eb006ea30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1eb009f180 0x7f1eb013d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f1eb009f180 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb006ccd0 con 0x7f1eb00678d0 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb006c900 con 0x7f1eb006ea30 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb011e6d0 con 0x7f1eb009f180 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb6f9d640 10 monclient(hunting): authenticate will time out at 867.804787s 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb4d39640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eb00678d0 0x7f1eb011c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb4d39640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eb00678d0 0x7f1eb011c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:58508/0 (socket says 172.21.15.178:58508) 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb4d39640 1 -- 172.21.15.178:0/746484811 learned_addr learned my addr 172.21.15.178:0/746484811 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb553a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1eb009f180 0x7f1eb013d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb4d39640 10 monclient(hunting): get_auth_request con 0x7f1eb00678d0 auth_method 0 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eaffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb006ea30 0x7f1eb0139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb4d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb4d39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb4d39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb553a640 10 monclient(hunting): get_auth_request con 0x7f1eb009f180 auth_method 0 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb553a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb553a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb553a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eaffff640 10 monclient(hunting): get_auth_request con 0x7f1eb006ea30 auth_method 0 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb4d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb4d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb4d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb553a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.054+0000 7f1eb553a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb553a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eaffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eaffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eaffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb4d39640 10 monclient(hunting): handle_auth_done global_id 5139 payload 824 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb4d39640 1 -- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1eb009f180 msgr2=0x7f1eb013d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb4d39640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1eb009f180 0x7f1eb013d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb4d39640 1 -- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb006ea30 msgr2=0x7f1eb0139630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb4d39640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb006ea30 0x7f1eb0139630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb4d39640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb4d39640 1 monclient: found mon.noname-a 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb4d39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb4d39640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb4d39640 1 -- 172.21.15.178:0/746484811 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea0009450 con 0x7f1eb00678d0 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb553a640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1eb009f180 0x7f1eb013d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb4d39640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eb00678d0 0x7f1eb011c0b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000ec80 tx=0x7f1ea00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1ea0009730 con 0x7f1eb00678d0 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eacff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eacff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eacff9640 10 monclient: dump: 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.074 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eacff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eacff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb6f9d640 5 monclient: authenticate success, global_id 5139 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea000eec0 con 0x7f1eb00678d0 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb6f9d640 10 monclient: _renew_subs 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb6f9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/746484811 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea400de30 con 0x7f1eb00678d0 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eacff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1ea0013d70 con 0x7f1eb00678d0 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eacff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eacff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eaeffd640 4 set_mon_vals no callback set 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eacff9640 10 monclient: dump: 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.076 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.077 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb6f9d640 10 monclient: _renew_subs 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb6f9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.055+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/746484811 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb013a240 con 0x7f1eb00678d0 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eb6f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eb6f9d640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb011d7b0 0x7f1eb00298c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eb6f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/746484811 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb011eb50 con 0x7f1eb011d7b0 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1ea000cb60 con 0x7f1eb00678d0 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eaffff640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb011d7b0 0x7f1eb00298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eaffff640 10 monclient: get_auth_request con 0x7f1eb011d7b0 auth_method 0 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eaffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eaffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eaffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eacff9640 1 --2- 172.21.15.178:0/746484811 >> v2:172.21.15.204:6812/28210 conn(0x7f1e9c0397b0 0x7f1e9c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eaffff640 10 monclient(hunting): handle_auth_done global_id 4988 payload 824 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eaffff640 1 -- 172.21.15.178:0/746484811 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1eb004c4a0 con 0x7f1eb011d7b0 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7f1ea004bd50 con 0x7f1eb00678d0 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eb553a640 1 --2- 172.21.15.178:0/746484811 >> v2:172.21.15.204:6812/28210 conn(0x7f1e9c0397b0 0x7f1e9c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eaffff640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb011d7b0 0x7f1eb00298c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00678a0 tx=0x7f1ea403b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.056+0000 7f1eb553a640 10 monclient: get_auth_request con 0x7f1e9c0397b0 auth_method 0 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.057+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1ea4044070 con 0x7f1eb011d7b0 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.057+0000 7f1eacff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.057+0000 7f1eacff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.057+0000 7f1eacff9640 10 monclient: dump: 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.079 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.057+0000 7f1eb553a640 1 --2- 172.21.15.178:0/746484811 >> v2:172.21.15.204:6812/28210 conn(0x7f1e9c0397b0 0x7f1e9c03bc60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1eb011f460 tx=0x7f1ea8008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.057+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f1ea400b740 con 0x7f1eb011d7b0 2024-09-15T01:34:23.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.057+0000 7f1eacff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.057+0000 7f1eacff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:23.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.057+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb011d7b0 msgr2=0x7f1eb00298c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.057+0000 7f1eacff9640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb011d7b0 0x7f1eb00298c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00678a0 tx=0x7f1ea403b000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.057+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb011d7b0 msgr2=0x7f1eb00298c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:23.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.057+0000 7f1eacff9640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb011d7b0 0x7f1eb00298c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.062+0000 7f1eb6f9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:23.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eb6f9d640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb0029e00 0x7f1eb002a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eb6f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/746484811 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb011e860 con 0x7f1eb0029e00 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eaffff640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb0029e00 0x7f1eb002a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eaffff640 10 monclient: get_auth_request con 0x7f1eb0029e00 auth_method 0 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eaffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eaffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eaffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eaffff640 10 monclient(hunting): handle_auth_done global_id 4997 payload 824 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eaffff640 1 -- 172.21.15.178:0/746484811 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1eb004bc80 con 0x7f1eb0029e00 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eaffff640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb0029e00 0x7f1eb002a1d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4000f80 tx=0x7f1ea4044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1ea403de10 con 0x7f1eb0029e00 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eacff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eacff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eacff9640 10 monclient: dump: 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.084 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.085 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.085 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.085 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.085 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.085 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9311 (secure 0 0 0) 0x7f1ea4033430 con 0x7f1eb0029e00 2024-09-15T01:34:23.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eacff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:23.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eacff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:23.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb0029e00 msgr2=0x7f1eb002a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eacff9640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb0029e00 0x7f1eb002a1d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4000f80 tx=0x7f1ea4044000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eacff9640 1 -- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb0029e00 msgr2=0x7f1eb002a1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:23.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.063+0000 7f1eacff9640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1eb0029e00 0x7f1eb002a1d0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4000f80 tx=0x7f1ea4044000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:23.086 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.087 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.088 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:23.089 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.090 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.091 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.092 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:23.093 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.094 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:23.096 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:23.098 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:23.100 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.102 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.103 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.104 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:23.105 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.106 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.106 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:23.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.106 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.106 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.106 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:23.106 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.106 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.106 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:23.106 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:23.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.064+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/746484811 >> v2:172.21.15.204:6812/28210 conn(0x7f1e9c0397b0 msgr2=0x7f1e9c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.064+0000 7f1eb6f9d640 1 --2- 172.21.15.178:0/746484811 >> v2:172.21.15.204:6812/28210 conn(0x7f1e9c0397b0 0x7f1e9c03bc60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1eb011f460 tx=0x7f1ea8008040 comp rx=0 tx=0).stop 2024-09-15T01:34:23.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.064+0000 7f1eb6f9d640 10 monclient: shutdown 2024-09-15T01:34:23.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.064+0000 7f1eb6f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.064+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eb00678d0 msgr2=0x7f1eb011c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.064+0000 7f1eb6f9d640 1 --2- 172.21.15.178:0/746484811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eb00678d0 0x7f1eb011c0b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000ec80 tx=0x7f1ea00091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.064+0000 7f1eb553a640 1 -- 172.21.15.178:0/746484811 reap_dead start 2024-09-15T01:34:23.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.064+0000 7f1eb553a640 1 -- 172.21.15.178:0/746484811 reap_dead start 2024-09-15T01:34:23.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.064+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/746484811 shutdown_connections 2024-09-15T01:34:23.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.064+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/746484811 >> 172.21.15.178:0/746484811 conn(0x7f1eb00a4390 msgr2=0x7f1eb009e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.064+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/746484811 shutdown_connections 2024-09-15T01:34:23.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.064+0000 7f1eb6f9d640 1 -- 172.21.15.178:0/746484811 wait complete. 2024-09-15T01:34:23.108 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:23.109 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:34:23.109 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:34:23.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.169+0000 7f6b96b5d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.169+0000 7f6b96b5d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.169+0000 7f6b96b5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:23.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.169+0000 7f6b96b5d640 1 -- 172.21.15.204:0/3550434123 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"} v 0) v1 -- 0x7f6b90121f00 con 0x7f6b900a2e30 2024-09-15T01:34:23.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.169+0000 7f6b8cff9640 1 -- 172.21.15.204:0/3550434123 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}]=-39 (39) Directory not empty v77) v1 ==== 120+0+0 (secure 0 0 0) 0x7f6b88010ad0 con 0x7f6b900a2e30 2024-09-15T01:34:23.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.169+0000 7f6b8cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.169+0000 7f6b8cff9640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-09-15T01:34:23.172 INFO:tasks.workunit.client.0.smithi204.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-09-15T01:34:23.175 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:34:23.175 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush unlink host1 foo 2024-09-15T01:34:23.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.200+0000 7fe15dd1d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:23.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.200+0000 7fe15dd1d640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.200+0000 7fe15dd1d640 10 monclient: monmap: 2024-09-15T01:34:23.202 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.202 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.202 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:23.201455+0000 2024-09-15T01:34:23.202 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:23.201455+0000 2024-09-15T01:34:23.202 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:23.203 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 1 Processor -- start 2024-09-15T01:34:23.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 1 -- start start 2024-09-15T01:34:23.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 10 monclient: init 2024-09-15T01:34:23.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 10 monclient: _add_conns ranks=[9,7,4,5,3,0,8,2,6,1,10] 2024-09-15T01:34:23.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe1580a2930 0x7fe158132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 10 monclient(hunting): picked mon.noname-a con 0x7fe1580a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe15809faa0 0x7fe15809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 10 monclient(hunting): picked mon.noname-i con 0x7fe15809faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1580a08c0 0x7fe1580a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 10 monclient(hunting): picked mon.noname-c con 0x7fe1580a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15809e0f0 con 0x7fe1580a2930 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe15809dd80 con 0x7fe1580a08c0 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580a2d00 con 0x7fe15809faa0 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.201+0000 7fe15dd1d640 10 monclient(hunting): authenticate will time out at 867.951998s 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe1580a2930 0x7fe158132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe157fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1580a08c0 0x7fe1580a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe1580a2930 0x7fe158132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:58524/0 (socket says 172.21.15.178:58524) 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 1 -- 172.21.15.178:0/4031488246 learned_addr learned my addr 172.21.15.178:0/4031488246 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 10 monclient(hunting): get_auth_request con 0x7fe1580a2930 auth_method 0 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe157fff640 10 monclient(hunting): get_auth_request con 0x7fe1580a08c0 auth_method 0 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe157fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe157fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe157fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe156ffd640 1 --2- 172.21.15.178:0/4031488246 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe15809faa0 0x7fe15809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe157fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe157fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe157fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe156ffd640 10 monclient(hunting): get_auth_request con 0x7fe15809faa0 auth_method 0 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe156ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe156ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe156ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe156ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe156ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe156ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 10 monclient(hunting): handle_auth_done global_id 5148 payload 274 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 1 -- 172.21.15.178:0/4031488246 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe15809faa0 msgr2=0x7fe15809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 1 --2- 172.21.15.178:0/4031488246 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe15809faa0 0x7fe15809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 1 -- 172.21.15.178:0/4031488246 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1580a08c0 msgr2=0x7fe1580a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 1 --2- 172.21.15.178:0/4031488246 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1580a08c0 0x7fe1580a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 1 monclient: found mon.noname-a 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe1577fe640 1 -- 172.21.15.178:0/4031488246 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15804bf60 con 0x7fe1580a2930 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.202+0000 7fe157fff640 1 --2- 172.21.15.178:0/4031488246 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1580a08c0 0x7fe1580a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1577fe640 1 --2- 172.21.15.178:0/4031488246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1580a2930 0x7fe158132de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe148009910 tx=0x7fe1480091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e403a0ebc468c298 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1567fc640 1 -- 172.21.15.178:0/4031488246 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe14802ac90 con 0x7fe1580a2930 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1567fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1567fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1567fc640 10 monclient: dump: 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.217 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1567fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1567fc640 1 -- 172.21.15.178:0/4031488246 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14802ae10 con 0x7fe1580a2930 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 5 monclient: authenticate success, global_id 5148 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1567fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1567fc640 1 -- 172.21.15.178:0/4031488246 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe148031ca0 con 0x7fe1580a2930 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1567fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 4 set_mon_vals no callback set 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1567fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe1567fc640 10 monclient: dump: 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.219 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 10 monclient: shutdown 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 1 -- 172.21.15.178:0/4031488246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1580a2930 msgr2=0x7fe158132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 1 --2- 172.21.15.178:0/4031488246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1580a2930 0x7fe158132de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe148009910 tx=0x7fe1480091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 1 -- 172.21.15.178:0/4031488246 shutdown_connections 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 1 --2- 172.21.15.178:0/4031488246 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1580a08c0 0x7fe1580a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 1 --2- 172.21.15.178:0/4031488246 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe15809faa0 0x7fe15809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 1 --2- 172.21.15.178:0/4031488246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1580a2930 0x7fe158132de0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 1 -- 172.21.15.178:0/4031488246 >> 172.21.15.178:0/4031488246 conn(0x7fe1580a5ed0 msgr2=0x7fe1580a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 1 -- 172.21.15.178:0/4031488246 shutdown_connections 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.203+0000 7fe15dd1d640 1 -- 172.21.15.178:0/4031488246 wait complete. 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.204+0000 7fe15dd1d640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.204+0000 7fe15dd1d640 10 monclient: monmap: 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.221 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.204+0000 7fe15dd1d640 1 Processor -- start 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.204+0000 7fe15dd1d640 1 -- start start 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.204+0000 7fe15dd1d640 10 monclient: init 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 10 monclient: _add_conns ranks=[8,5,4,3,0,7,1,2,6] 2024-09-15T01:34:23.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe15809faa0 0x7fe15809b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 10 monclient(hunting): picked mon.noname-e con 0x7fe15809faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe1580a08c0 0x7fe1581317d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 10 monclient(hunting): picked mon.noname-h con 0x7fe1580a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1580a2930 0x7fe15811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 10 monclient(hunting): picked mon.noname-c con 0x7fe1580a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe15811ebd0 con 0x7fe1580a2930 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580a2400 con 0x7fe1580a08c0 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe15809d630 con 0x7fe15809faa0 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe15dd1d640 10 monclient(hunting): authenticate will time out at 867.955357s 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe15809faa0 0x7fe15809b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe15809faa0 0x7fe15809b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54182/0 (socket says 172.21.15.178:54182) 2024-09-15T01:34:23.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe157fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1580a2930 0x7fe15811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 1 -- 172.21.15.178:0/3672711461 learned_addr learned my addr 172.21.15.178:0/3672711461 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 10 monclient(hunting): get_auth_request con 0x7fe15809faa0 auth_method 0 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe157fff640 10 monclient(hunting): get_auth_request con 0x7fe1580a2930 auth_method 0 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe157fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe157fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe157fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe157fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe157fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe157fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe156ffd640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe1580a08c0 0x7fe1581317d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe156ffd640 10 monclient(hunting): get_auth_request con 0x7fe1580a08c0 auth_method 0 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe156ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe156ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe156ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe156ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe156ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe156ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 10 monclient(hunting): handle_auth_done global_id 5111 payload 824 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 1 -- 172.21.15.178:0/3672711461 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe1580a08c0 msgr2=0x7fe1581317d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe1580a08c0 0x7fe1581317d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 1 -- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1580a2930 msgr2=0x7fe15811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1580a2930 0x7fe15811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 1 monclient: found mon.noname-e 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 1 -- 172.21.15.178:0/3672711461 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe150009a20 con 0x7fe15809faa0 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe157fff640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1580a2930 0x7fe15811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.205+0000 7fe1577fe640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe15809faa0 0x7fe15809b000 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe148000c00 tx=0x7fe148032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe148031820 con 0x7fe15809faa0 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 10 monclient: dump: 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.226 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe15dd1d640 5 monclient: authenticate success, global_id 5111 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1480319c0 con 0x7fe15809faa0 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe15dd1d640 10 monclient: _renew_subs 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe15dd1d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe15dd1d640 1 -- 172.21.15.178:0/3672711461 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe148009450 con 0x7fe15809faa0 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe14803a390 con 0x7fe15809faa0 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe155ffb640 4 set_mon_vals no callback set 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 10 monclient: dump: 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.228 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe15dd1d640 10 monclient: _renew_subs 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe15dd1d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe15dd1d640 1 -- 172.21.15.178:0/3672711461 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15809bbb0 con 0x7fe15809faa0 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe15dd1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe15dd1d640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe158131d10 0x7fe15811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe15dd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe15dd1d640 1 -- 172.21.15.178:0/3672711461 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1581320e0 con 0x7fe158131d10 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.206+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe148039040 con 0x7fe15809faa0 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe156ffd640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe158131d10 0x7fe15811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe156ffd640 10 monclient: get_auth_request con 0x7fe158131d10 auth_method 0 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe156ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe156ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe156ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe156ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe156ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe156ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 1 --2- 172.21.15.178:0/3672711461 >> v2:172.21.15.204:6812/28210 conn(0x7fe1440397b0 0x7fe14403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7fe14806ae50 con 0x7fe15809faa0 2024-09-15T01:34:23.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe156ffd640 10 monclient(hunting): handle_auth_done global_id 4972 payload 824 2024-09-15T01:34:23.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe156ffd640 1 -- 172.21.15.178:0/3672711461 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe15804c4a0 con 0x7fe158131d10 2024-09-15T01:34:23.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe157fff640 1 --2- 172.21.15.178:0/3672711461 >> v2:172.21.15.204:6812/28210 conn(0x7fe1440397b0 0x7fe14403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe157fff640 10 monclient: get_auth_request con 0x7fe1440397b0 auth_method 0 2024-09-15T01:34:23.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe156ffd640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe158131d10 0x7fe15811ca70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe14c011bf0 tx=0x7fe14c0090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe14c00fc80 con 0x7fe158131d10 2024-09-15T01:34:23.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 10 monclient: dump: 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.232 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.233 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.233 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.233 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.233 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe157fff640 1 --2- 172.21.15.178:0/3672711461 >> v2:172.21.15.204:6812/28210 conn(0x7fe1440397b0 0x7fe14403bc60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe15809b720 tx=0x7fe15000b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fe14c00fde0 con 0x7fe158131d10 2024-09-15T01:34:23.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:23.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe158131d10 msgr2=0x7fe15811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe158131d10 0x7fe15811ca70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe14c011bf0 tx=0x7fe14c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe158131d10 msgr2=0x7fe15811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:23.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.207+0000 7fe13f7fe640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe158131d10 0x7fe15811ca70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe15dd1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:23.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe15dd1d640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe15811cfb0 0x7fe158096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe15dd1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe15dd1d640 1 -- 172.21.15.178:0/3672711461 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe15804bc80 con 0x7fe15811cfb0 2024-09-15T01:34:23.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe156ffd640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe15811cfb0 0x7fe158096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe156ffd640 10 monclient: get_auth_request con 0x7fe15811cfb0 auth_method 0 2024-09-15T01:34:23.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe156ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe156ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe156ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe156ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe156ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe156ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe156ffd640 10 monclient(hunting): handle_auth_done global_id 4981 payload 824 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.213+0000 7fe156ffd640 1 -- 172.21.15.178:0/3672711461 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe15805ed70 con 0x7fe15811cfb0 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe156ffd640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe15811cfb0 0x7fe158096a30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe14c011bf0 tx=0x7fe14c01c000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe14c024040 con 0x7fe15811cfb0 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe13f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe13f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe13f7fe640 10 monclient: dump: 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.236 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7fe14c0133a0 con 0x7fe15811cfb0 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe13f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe13f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:23.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe15811cfb0 msgr2=0x7fe158096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe13f7fe640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe15811cfb0 0x7fe158096a30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe14c011bf0 tx=0x7fe14c01c000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe13f7fe640 1 -- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe15811cfb0 msgr2=0x7fe158096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:23.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe13f7fe640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe15811cfb0 0x7fe158096a30 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe14c011bf0 tx=0x7fe14c01c000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.238 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:23.238 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:23.238 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:23.238 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:23.238 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:23.238 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:23.238 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:23.238 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.239 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.240 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:23.241 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.242 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:23.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:23.250 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.251 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:23.252 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:23.253 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:23.255 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:23.257 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.258 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.258 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:23.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.258 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.258 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.258 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:23.258 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.258 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.258 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:23.258 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:23.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe15dd1d640 1 -- 172.21.15.178:0/3672711461 >> v2:172.21.15.204:6812/28210 conn(0x7fe1440397b0 msgr2=0x7fe14403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe15dd1d640 1 --2- 172.21.15.178:0/3672711461 >> v2:172.21.15.204:6812/28210 conn(0x7fe1440397b0 0x7fe14403bc60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe15809b720 tx=0x7fe15000b040 comp rx=0 tx=0).stop 2024-09-15T01:34:23.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe15dd1d640 10 monclient: shutdown 2024-09-15T01:34:23.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe15dd1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe15dd1d640 1 -- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe15809faa0 msgr2=0x7fe15809b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe15dd1d640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe15809faa0 0x7fe15809b000 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe148000c00 tx=0x7fe148032000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe157fff640 1 -- 172.21.15.178:0/3672711461 reap_dead start 2024-09-15T01:34:23.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe15dd1d640 1 -- 172.21.15.178:0/3672711461 shutdown_connections 2024-09-15T01:34:23.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe15dd1d640 1 --2- 172.21.15.178:0/3672711461 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe15809faa0 0x7fe15809b000 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe15dd1d640 1 -- 172.21.15.178:0/3672711461 >> 172.21.15.178:0/3672711461 conn(0x7fe1580a5ed0 msgr2=0x7fe15809e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe15dd1d640 1 -- 172.21.15.178:0/3672711461 shutdown_connections 2024-09-15T01:34:23.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.214+0000 7fe15dd1d640 1 -- 172.21.15.178:0/3672711461 wait complete. 2024-09-15T01:34:23.261 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:23.261 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:34:23.261 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:34:23.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.247+0000 7fa48722b640 10 monclient: get_monmap_and_config 2024-09-15T01:34:23.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.247+0000 7fa48722b640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.247+0000 7fa48722b640 10 monclient: monmap: 2024-09-15T01:34:23.269 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:23.269 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.269 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:23.249096+0000 2024-09-15T01:34:23.269 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:23.249096+0000 2024-09-15T01:34:23.269 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.269 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.269 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:23.269 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 1 Processor -- start 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 1 -- start start 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 10 monclient: init 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 10 monclient: _add_conns ranks=[10,3,6,8,0,9,7,2,4,5,1] 2024-09-15T01:34:23.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa4800a2e30 0x7fa480132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 10 monclient(hunting): picked mon.noname-f con 0x7fa4800a2e30 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa48009fb80 0x7fa48009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 10 monclient(hunting): picked mon.noname-g con 0x7fa48009fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa4800a0a40 0x7fa4800a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 10 monclient(hunting): picked mon.noname-d con 0x7fa4800a0a40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48009e6c0 con 0x7fa4800a2e30 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4800a04b0 con 0x7fa48009fb80 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa4800a3200 con 0x7fa4800a0a40 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa48722b640 10 monclient(hunting): authenticate will time out at 878.891863s 2024-09-15T01:34:23.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa47ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa48009fb80 0x7fa48009ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.248+0000 7fa484fc7640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa4800a2e30 0x7fa480132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa4800a2e30 0x7fa480132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48902/0 (socket says 172.21.15.204:48902) 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa47ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa48009fb80 0x7fa48009ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54550/0 (socket says 172.21.15.204:54550) 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 1 -- 172.21.15.204:0/2555149473 learned_addr learned my addr 172.21.15.204:0/2555149473 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 10 monclient(hunting): get_auth_request con 0x7fa4800a2e30 auth_method 0 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa47ffff640 10 monclient(hunting): get_auth_request con 0x7fa48009fb80 auth_method 0 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa47ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa47ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa47ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa47ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa47ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa47ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa4857c8640 1 --2- 172.21.15.204:0/2555149473 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa4800a0a40 0x7fa4800a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa4857c8640 10 monclient(hunting): get_auth_request con 0x7fa4800a0a40 auth_method 0 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa4857c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa4857c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa4857c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 10 monclient(hunting): handle_auth_done global_id 5527 payload 274 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 1 -- 172.21.15.204:0/2555149473 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa4800a0a40 msgr2=0x7fa4800a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 1 --2- 172.21.15.204:0/2555149473 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa4800a0a40 0x7fa4800a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 1 -- 172.21.15.204:0/2555149473 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa48009fb80 msgr2=0x7fa48009ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 1 --2- 172.21.15.204:0/2555149473 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa48009fb80 0x7fa48009ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 1 monclient: found mon.noname-f 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 1 -- 172.21.15.204:0/2555149473 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48004b720 con 0x7fa4800a2e30 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.249+0000 7fa484fc7640 1 --2- 172.21.15.204:0/2555149473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4800a2e30 0x7fa480132a60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa474009910 tx=0x7fa4740091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95b01787e0f3ca66 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa47f7fe640 1 -- 172.21.15.204:0/2555149473 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa47402ac80 con 0x7fa4800a2e30 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa47f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa47f7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa47f7fe640 10 monclient: dump: 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.274 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa47f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa47f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa47f7fe640 1 -- 172.21.15.204:0/2555149473 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47402ae00 con 0x7fa4800a2e30 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 5 monclient: authenticate success, global_id 5527 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa47f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa47f7fe640 1 -- 172.21.15.204:0/2555149473 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa474031ca0 con 0x7fa4800a2e30 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 4 set_mon_vals no callback set 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa47f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa47f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa47f7fe640 10 monclient: dump: 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.276 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 10 monclient: shutdown 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 1 -- 172.21.15.204:0/2555149473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4800a2e30 msgr2=0x7fa480132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 1 --2- 172.21.15.204:0/2555149473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4800a2e30 0x7fa480132a60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa474009910 tx=0x7fa4740091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 1 -- 172.21.15.204:0/2555149473 shutdown_connections 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 1 --2- 172.21.15.204:0/2555149473 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa48009fb80 0x7fa48009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 1 --2- 172.21.15.204:0/2555149473 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa4800a0a40 0x7fa4800a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 1 --2- 172.21.15.204:0/2555149473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4800a2e30 0x7fa480132a60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 1 -- 172.21.15.204:0/2555149473 >> 172.21.15.204:0/2555149473 conn(0x7fa48006e860 msgr2=0x7fa48006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 1 -- 172.21.15.204:0/2555149473 shutdown_connections 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.250+0000 7fa48722b640 1 -- 172.21.15.204:0/2555149473 wait complete. 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.251+0000 7fa48722b640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.251+0000 7fa48722b640 10 monclient: monmap: 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:23.278 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.251+0000 7fa48722b640 1 Processor -- start 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.251+0000 7fa48722b640 1 -- start start 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.251+0000 7fa48722b640 10 monclient: init 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 10 monclient: _add_conns ranks=[7,0,3,5,1,2,8,4,6] 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa48009fb80 0x7fa4801316c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 10 monclient(hunting): picked mon.noname-i con 0x7fa48009fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4800a0a40 0x7fa48011e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 10 monclient(hunting): picked mon.noname-a con 0x7fa4800a0a40 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4800a2e30 0x7fa48011e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 10 monclient(hunting): picked mon.noname-g con 0x7fa4800a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa480120510 con 0x7fa4800a0a40 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48009ca40 con 0x7fa4800a2e30 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa48009d650 con 0x7fa48009fb80 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa48722b640 10 monclient(hunting): authenticate will time out at 878.895481s 2024-09-15T01:34:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4800a2e30 0x7fa48011e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa484fc7640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa48009fb80 0x7fa4801316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4800a2e30 0x7fa48011e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54564/0 (socket says 172.21.15.204:54564) 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 1 -- 172.21.15.204:0/2628863251 learned_addr learned my addr 172.21.15.204:0/2628863251 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa484fc7640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa48009fb80 0x7fa4801316c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:34078/0 (socket says 172.21.15.204:34078) 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 10 monclient(hunting): get_auth_request con 0x7fa4800a2e30 auth_method 0 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa484fc7640 10 monclient(hunting): get_auth_request con 0x7fa48009fb80 auth_method 0 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa484fc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa484fc7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa484fc7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa47ffff640 1 --2- 172.21.15.204:0/2628863251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4800a0a40 0x7fa48011e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa484fc7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa484fc7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa484fc7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa47ffff640 10 monclient(hunting): get_auth_request con 0x7fa4800a0a40 auth_method 0 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa47ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa47ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa47ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa47ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa47ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa47ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 10 monclient(hunting): handle_auth_done global_id 14142 payload 824 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 1 -- 172.21.15.204:0/2628863251 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa48009fb80 msgr2=0x7fa4801316c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 1 --2- 172.21.15.204:0/2628863251 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa48009fb80 0x7fa4801316c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 1 -- 172.21.15.204:0/2628863251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4800a0a40 msgr2=0x7fa48011e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 1 --2- 172.21.15.204:0/2628863251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4800a0a40 0x7fa48011e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 1 monclient: found mon.noname-g 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa4857c8640 1 -- 172.21.15.204:0/2628863251 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa474009450 con 0x7fa4800a2e30 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.252+0000 7fa484fc7640 1 --2- 172.21.15.204:0/2628863251 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa48009fb80 0x7fa4801316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa4857c8640 1 --2- 172.21.15.204:0/2628863251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4800a2e30 0x7fa48011e730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa478009fd0 tx=0x7fa478009900 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47cff9640 1 -- 172.21.15.204:0/2628863251 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa47800ee90 con 0x7fa4800a2e30 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47cff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47cff9640 10 monclient: dump: 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:23.283 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47cff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47cff9640 1 -- 172.21.15.204:0/2628863251 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa478009990 con 0x7fa4800a2e30 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa48722b640 5 monclient: authenticate success, global_id 14142 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa48722b640 10 monclient: _renew_subs 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa48722b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa48722b640 1 -- 172.21.15.204:0/2628863251 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa480121520 con 0x7fa4800a2e30 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47cff9640 1 -- 172.21.15.204:0/2628863251 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa478014d20 con 0x7fa4800a2e30 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47cff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47cff9640 10 monclient: dump: 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.285 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa48722b640 10 monclient: _renew_subs 2024-09-15T01:34:23.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa47effd640 4 set_mon_vals no callback set 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa48722b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.253+0000 7fa48722b640 1 -- 172.21.15.204:0/2628863251 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48011fce0 con 0x7fa4800a2e30 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.254+0000 7fa47cff9640 1 -- 172.21.15.204:0/2628863251 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa478014420 con 0x7fa4800a2e30 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.254+0000 7fa48722b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.254+0000 7fa48722b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.254+0000 7fa48722b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.254+0000 7fa48722b640 1 -- 172.21.15.204:0/2628863251 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa480070cc0 con 0x7fa4800a2e30 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.254+0000 7fa47cff9640 1 --2- 172.21.15.204:0/2628863251 >> v2:172.21.15.204:6812/28210 conn(0x7fa4640397e0 0x7fa46403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.254+0000 7fa47cff9640 1 -- 172.21.15.204:0/2628863251 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7fa47804be50 con 0x7fa4800a2e30 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.254+0000 7fa484fc7640 1 --2- 172.21.15.204:0/2628863251 >> v2:172.21.15.204:6812/28210 conn(0x7fa4640397e0 0x7fa46403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.254+0000 7fa484fc7640 10 monclient: get_auth_request con 0x7fa4640397e0 auth_method 0 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.254+0000 7fa484fc7640 1 --2- 172.21.15.204:0/2628863251 >> v2:172.21.15.204:6812/28210 conn(0x7fa4640397e0 0x7fa46403bc90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa474034000 tx=0x7fa474032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.258+0000 7fa47cff9640 1 -- 172.21.15.204:0/2628863251 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa4780146d0 con 0x7fa4800a2e30 2024-09-15T01:34:23.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.258+0000 7fa47cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.258+0000 7fa47cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:23.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.356+0000 7f9707eaf640 10 monclient: get_monmap_and_config 2024-09-15T01:34:23.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.356+0000 7f9707eaf640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.356+0000 7f9707eaf640 10 monclient: monmap: 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:23.357496+0000 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:23.357496+0000 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.370 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 1 Processor -- start 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 1 -- start start 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 10 monclient: init 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 10 monclient: _add_conns ranks=[10,8,5,4,6,2,7,1,0,3,9] 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f97000a2930 0x7f9700132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 10 monclient(hunting): picked mon.noname-f con 0x7f97000a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f970009faa0 0x7f970009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 10 monclient(hunting): picked mon.noname-e con 0x7f970009faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97000a08c0 0x7f97000a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 10 monclient(hunting): picked mon.noname-h con 0x7f97000a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970009e0f0 con 0x7f97000a2930 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f970009dd80 con 0x7f97000a08c0 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f97000a2d00 con 0x7f970009faa0 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f9707eaf640 10 monclient(hunting): authenticate will time out at 868.107850s 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.357+0000 7f970544a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f970009faa0 0x7f970009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f970009faa0 0x7f970009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54196/0 (socket says 172.21.15.178:54196) 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 1 -- 172.21.15.178:0/3932723558 learned_addr learned my addr 172.21.15.178:0/3932723558 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 10 monclient(hunting): get_auth_request con 0x7f970009faa0 auth_method 0 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970644c640 1 --2- 172.21.15.178:0/3932723558 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97000a08c0 0x7f97000a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f9705c4b640 1 --2- 172.21.15.178:0/3932723558 >> v2:172.21.15.204:3300/0 conn(0x7f97000a2930 0x7f9700132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970644c640 10 monclient(hunting): get_auth_request con 0x7f97000a08c0 auth_method 0 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970644c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970644c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970644c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f9705c4b640 10 monclient(hunting): get_auth_request con 0x7f97000a2930 auth_method 0 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f9705c4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f9705c4b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f9705c4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970644c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970644c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970644c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f9705c4b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f9705c4b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f9705c4b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 10 monclient(hunting): handle_auth_done global_id 5120 payload 274 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 1 -- 172.21.15.178:0/3932723558 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97000a08c0 msgr2=0x7f97000a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 1 --2- 172.21.15.178:0/3932723558 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97000a08c0 0x7f97000a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 1 -- 172.21.15.178:0/3932723558 >> v2:172.21.15.204:3300/0 conn(0x7f97000a2930 msgr2=0x7f9700132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 1 --2- 172.21.15.178:0/3932723558 >> v2:172.21.15.204:3300/0 conn(0x7f97000a2930 0x7f9700132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 1 monclient: found mon.noname-e 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 1 -- 172.21.15.178:0/3932723558 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970004bf60 con 0x7f970009faa0 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.358+0000 7f970544a640 1 --2- 172.21.15.178:0/3932723558 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f970009faa0 0x7f970009fe70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009910 tx=0x7f96f00091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=337e651210422552 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9704c49640 1 -- 172.21.15.178:0/3932723558 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96f002ac80 con 0x7f970009faa0 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9704c49640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9704c49640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9704c49640 10 monclient: dump: 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.375 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9704c49640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9704c49640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 5 monclient: authenticate success, global_id 5120 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9704c49640 1 -- 172.21.15.178:0/3932723558 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f002ae00 con 0x7f970009faa0 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9704c49640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9704c49640 1 -- 172.21.15.178:0/3932723558 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96f0031ca0 con 0x7f970009faa0 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9704c49640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 4 set_mon_vals no callback set 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9704c49640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9704c49640 10 monclient: dump: 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.377 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 10 monclient: shutdown 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 1 -- 172.21.15.178:0/3932723558 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f970009faa0 msgr2=0x7f970009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 1 --2- 172.21.15.178:0/3932723558 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f970009faa0 0x7f970009fe70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009910 tx=0x7f96f00091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 1 -- 172.21.15.178:0/3932723558 shutdown_connections 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 1 --2- 172.21.15.178:0/3932723558 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97000a08c0 0x7f97000a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 1 --2- 172.21.15.178:0/3932723558 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f970009faa0 0x7f970009fe70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 1 --2- 172.21.15.178:0/3932723558 >> v2:172.21.15.204:3300/0 conn(0x7f97000a2930 0x7f9700132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 1 -- 172.21.15.178:0/3932723558 >> 172.21.15.178:0/3932723558 conn(0x7f97000a5ed0 msgr2=0x7f97000a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 1 -- 172.21.15.178:0/3932723558 shutdown_connections 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.359+0000 7f9707eaf640 1 -- 172.21.15.178:0/3932723558 wait complete. 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient: monmap: 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:23.379 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 1 Processor -- start 2024-09-15T01:34:23.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 1 -- start start 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient: init 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient: _add_conns ranks=[1,6,8,3,4,2,0,7,5] 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f970009faa0 0x7f970009b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient(hunting): picked mon.noname-f con 0x7f970009faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97000a08c0 0x7f9700131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient(hunting): picked mon.noname-d con 0x7f97000a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97000a2930 0x7f970011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient(hunting): picked mon.noname-e con 0x7f97000a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970009d630 con 0x7f970009faa0 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f970011ec40 con 0x7f97000a08c0 2024-09-15T01:34:23.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f97001429e0 con 0x7f97000a2930 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.360+0000 7f9707eaf640 10 monclient(hunting): authenticate will time out at 868.110982s 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970644c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97000a2930 0x7f970011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97000a08c0 0x7f9700131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970644c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97000a2930 0x7f970011c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54208/0 (socket says 172.21.15.178:54208) 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970644c640 1 -- 172.21.15.178:0/3940479806 learned_addr learned my addr 172.21.15.178:0/3940479806 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 10 monclient(hunting): get_auth_request con 0x7f97000a08c0 auth_method 0 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9705c4b640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f970009faa0 0x7f970009b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970644c640 10 monclient(hunting): get_auth_request con 0x7f97000a2930 auth_method 0 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970644c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970644c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970644c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9705c4b640 10 monclient(hunting): get_auth_request con 0x7f970009faa0 auth_method 0 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9705c4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9705c4b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9705c4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970644c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970644c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970644c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9705c4b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9705c4b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9705c4b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 10 monclient(hunting): handle_auth_done global_id 14127 payload 824 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 1 -- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97000a2930 msgr2=0x7f970011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97000a2930 0x7f970011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 1 -- 172.21.15.178:0/3940479806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f970009faa0 msgr2=0x7f970009b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f970009faa0 0x7f970009b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 1 monclient: found mon.noname-d 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 1 -- 172.21.15.178:0/3940479806 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0009450 con 0x7f97000a08c0 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970644c640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97000a2930 0x7f970011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9705c4b640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f970009faa0 0x7f970009b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f970544a640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97000a08c0 0x7f9700131840 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f97000a2e40 tx=0x7f96f0032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96f002ac80 con 0x7f97000a08c0 2024-09-15T01:34:23.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f5ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f5ffb640 10 monclient: dump: 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.385 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0031820 con 0x7f97000a08c0 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9707eaf640 5 monclient: authenticate success, global_id 14127 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9707eaf640 10 monclient: _renew_subs 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9707eaf640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f9707eaf640 1 -- 172.21.15.178:0/3940479806 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970011e960 con 0x7f97000a08c0 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96f0031e10 con 0x7f97000a08c0 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f5ffb640 10 monclient: dump: 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.387 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.361+0000 7f96f7fff640 4 set_mon_vals no callback set 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9707eaf640 10 monclient: _renew_subs 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9707eaf640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9707eaf640 1 -- 172.21.15.178:0/3940479806 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9700137000 con 0x7f97000a08c0 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9707eaf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9707eaf640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9700131d80 0x7f970011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9707eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9707eaf640 1 -- 172.21.15.178:0/3940479806 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f970011eaf0 con 0x7f9700131d80 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f96f0039040 con 0x7f97000a08c0 2024-09-15T01:34:23.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9705c4b640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9700131d80 0x7f970011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9705c4b640 10 monclient: get_auth_request con 0x7f9700131d80 auth_method 0 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9705c4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9705c4b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9705c4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9705c4b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9705c4b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f9705c4b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.362+0000 7f96f5ffb640 1 --2- 172.21.15.178:0/3940479806 >> v2:172.21.15.204:6812/28210 conn(0x7f96e40397b0 0x7f96e403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7f96f006aa40 con 0x7f97000a08c0 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f9705c4b640 10 monclient(hunting): handle_auth_done global_id 14136 payload 824 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f9705c4b640 1 -- 172.21.15.178:0/3940479806 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9700137190 con 0x7f9700131d80 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f970644c640 1 --2- 172.21.15.178:0/3940479806 >> v2:172.21.15.204:6812/28210 conn(0x7f96e40397b0 0x7f96e403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f970644c640 10 monclient: get_auth_request con 0x7f96e40397b0 auth_method 0 2024-09-15T01:34:23.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f9705c4b640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9700131d80 0x7f970011ca70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009f30 tx=0x7f96fc0090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96fc014ad0 con 0x7f9700131d80 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f96f5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f96f5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f96f5ffb640 10 monclient: dump: 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.390 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f96fc014c30 con 0x7f9700131d80 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f96f5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f96f5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9700131d80 msgr2=0x7f970011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f96f5ffb640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9700131d80 0x7f970011ca70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009f30 tx=0x7f96fc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9700131d80 msgr2=0x7f970011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f96f5ffb640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9700131d80 0x7f970011ca70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.363+0000 7f970644c640 1 --2- 172.21.15.178:0/3940479806 >> v2:172.21.15.204:6812/28210 conn(0x7f96e40397b0 0x7f96e403bc60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f970009ba70 tx=0x7f96f800b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9707eaf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:23.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9707eaf640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f970011cfb0 0x7f9700096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9707eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9707eaf640 1 -- 172.21.15.178:0/3940479806 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f970004bc80 con 0x7f970011cfb0 2024-09-15T01:34:23.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9705c4b640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f970011cfb0 0x7f9700096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9705c4b640 10 monclient: get_auth_request con 0x7f970011cfb0 auth_method 0 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9705c4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9705c4b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9705c4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9705c4b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9705c4b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9705c4b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9705c4b640 10 monclient(hunting): handle_auth_done global_id 14145 payload 824 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9705c4b640 1 -- 172.21.15.178:0/3940479806 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f970005ed70 con 0x7f970011cfb0 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f9705c4b640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f970011cfb0 0x7f9700096a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009f30 tx=0x7f96fc01c000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96fc024040 con 0x7f970011cfb0 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f96f5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f96f5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:23.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.369+0000 7f96f5ffb640 10 monclient: dump: 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.395 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.396 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.396 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f96fc0133a0 con 0x7f970011cfb0 2024-09-15T01:34:23.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f96f5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:23.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f96f5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:23.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f970011cfb0 msgr2=0x7f9700096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f96f5ffb640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f970011cfb0 0x7f9700096a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009f30 tx=0x7f96fc01c000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f96f5ffb640 1 -- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f970011cfb0 msgr2=0x7f9700096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:23.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f96f5ffb640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f970011cfb0 0x7f9700096a30 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009f30 tx=0x7f96fc01c000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.396 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:23.397 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.398 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.399 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:23.400 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.401 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.404 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:23.406 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.409 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.410 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.411 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:23.412 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:23.414 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.416 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.417 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.418 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:23.418 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:23.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f9707eaf640 1 -- 172.21.15.178:0/3940479806 >> v2:172.21.15.204:6812/28210 conn(0x7f96e40397b0 msgr2=0x7f96e403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f9707eaf640 1 --2- 172.21.15.178:0/3940479806 >> v2:172.21.15.204:6812/28210 conn(0x7f96e40397b0 0x7f96e403bc60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f970009ba70 tx=0x7f96f800b040 comp rx=0 tx=0).stop 2024-09-15T01:34:23.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f9707eaf640 10 monclient: shutdown 2024-09-15T01:34:23.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f9707eaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f9707eaf640 1 -- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97000a08c0 msgr2=0x7f9700131840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f9707eaf640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97000a08c0 0x7f9700131840 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f97000a2e40 tx=0x7f96f0032000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f970644c640 1 -- 172.21.15.178:0/3940479806 reap_dead start 2024-09-15T01:34:23.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f9707eaf640 1 -- 172.21.15.178:0/3940479806 shutdown_connections 2024-09-15T01:34:23.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f9707eaf640 1 --2- 172.21.15.178:0/3940479806 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97000a08c0 0x7f9700131840 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f9707eaf640 1 -- 172.21.15.178:0/3940479806 >> 172.21.15.178:0/3940479806 conn(0x7f97000a5ed0 msgr2=0x7f9700135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f9707eaf640 1 -- 172.21.15.178:0/3940479806 shutdown_connections 2024-09-15T01:34:23.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.370+0000 7f9707eaf640 1 -- 172.21.15.178:0/3940479806 wait complete. 2024-09-15T01:34:23.420 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:23.420 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:34:23.420 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:34:23.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.418+0000 7fa48722b640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.418+0000 7fa48722b640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.418+0000 7fa48722b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:23.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.418+0000 7fa48722b640 1 -- 172.21.15.204:0/2628863251 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) v1 -- 0x7fa4800a1f10 con 0x7fa4800a2e30 2024-09-15T01:34:23.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.513+0000 7f78b223f640 10 monclient: get_monmap_and_config 2024-09-15T01:34:23.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.513+0000 7f78b223f640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.513+0000 7f78b223f640 10 monclient: monmap: 2024-09-15T01:34:23.515 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:23.514605+0000 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:23.514605+0000 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.516 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.517 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.517 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:23.517 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:23.517 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.514+0000 7f78b223f640 1 Processor -- start 2024-09-15T01:34:23.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.514+0000 7f78b223f640 1 -- start start 2024-09-15T01:34:23.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.514+0000 7f78b223f640 10 monclient: init 2024-09-15T01:34:23.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.514+0000 7f78b223f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.514+0000 7f78b223f640 10 monclient: _add_conns ranks=[1,0,4,7,5,10,8,9,3,6,2] 2024-09-15T01:34:23.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.514+0000 7f78b223f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f78ac0a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:23.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.514+0000 7f78b223f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f78ac132de0 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:23.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.514+0000 7f78b223f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78ac1392c0 0x7f78ac13b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.514+0000 7f78b223f640 10 monclient(hunting): picked mon.noname-c con 0x7f78ac1392c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:23.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.514+0000 7f78b223f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac04bf60 con 0x7f78ac132de0 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.514+0000 7f78b223f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac09dd80 con 0x7f78ac0a2930 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.514+0000 7f78b223f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78b223f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0a2d00 con 0x7f78ac1392c0 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78b223f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78b223f640 10 monclient(hunting): authenticate will time out at 868.265019s 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78ac1392c0 0x7f78ac13b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78aaffd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f78ac132de0 0x7f78ac137ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:34370/0 (socket says 172.21.15.178:34370) 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78aaffd640 1 -- 172.21.15.178:0/1206338063 learned_addr learned my addr 172.21.15.178:0/1206338063 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 10 monclient(hunting): get_auth_request con 0x7f78ac1392c0 auth_method 0 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 10 monclient(hunting): handle_auth_done global_id 4990 payload 274 2024-09-15T01:34:23.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78aa7fc640 1 -- 172.21.15.178:0/1206338063 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 862938648 0 0) 0x7f78ac04bf60 con 0x7f78ac132de0 2024-09-15T01:34:23.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 1 -- 172.21.15.178:0/1206338063 >> v1:172.21.15.204:6789/0 conn(0x7f78ac0a2930 legacy=0x7f78ac09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 1 -- 172.21.15.178:0/1206338063 >> v1:172.21.15.178:6789/0 conn(0x7f78ac132de0 legacy=0x7f78ac137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 1 monclient: found mon.noname-c 2024-09-15T01:34:23.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:23.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78abfff640 1 -- 172.21.15.178:0/1206338063 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac1331b0 con 0x7f78ac1392c0 2024-09-15T01:34:23.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.515+0000 7f78aa7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:23.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78abfff640 1 --2- 172.21.15.178:0/1206338063 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78ac1392c0 0x7f78ac13b6a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78ac09df10 tx=0x7f78a40090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f69e3392be22f099 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78aa7fc640 1 -- 172.21.15.178:0/1206338063 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78a402ac50 con 0x7f78ac1392c0 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78aa7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78aa7fc640 10 monclient: dump: 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.528 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78aa7fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 5 monclient: authenticate success, global_id 4990 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78aa7fc640 1 -- 172.21.15.178:0/1206338063 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a402add0 con 0x7f78ac1392c0 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78aa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 4 set_mon_vals no callback set 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78aa7fc640 1 -- 172.21.15.178:0/1206338063 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78a4031820 con 0x7f78ac1392c0 2024-09-15T01:34:23.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78aa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78aa7fc640 10 monclient: dump: 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.530 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 10 monclient: shutdown 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 1 -- 172.21.15.178:0/1206338063 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78ac1392c0 msgr2=0x7f78ac13b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 1 --2- 172.21.15.178:0/1206338063 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78ac1392c0 0x7f78ac13b6a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78ac09df10 tx=0x7f78a40090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 1 -- 172.21.15.178:0/1206338063 shutdown_connections 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 1 --2- 172.21.15.178:0/1206338063 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78ac1392c0 0x7f78ac13b6a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 1 -- 172.21.15.178:0/1206338063 >> 172.21.15.178:0/1206338063 conn(0x7f78ac0a5ed0 msgr2=0x7f78ac0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 1 -- 172.21.15.178:0/1206338063 shutdown_connections 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.516+0000 7f78b223f640 1 -- 172.21.15.178:0/1206338063 wait complete. 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.517+0000 7f78b223f640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.517+0000 7f78b223f640 10 monclient: monmap: 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.532 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.517+0000 7f78b223f640 1 Processor -- start 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.517+0000 7f78b223f640 1 -- start start 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.517+0000 7f78b223f640 10 monclient: init 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 10 monclient: _add_conns ranks=[7,2,0,5,3,6,8,1,4] 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f78ac0a2930 0x7f78ac141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 10 monclient(hunting): picked mon.noname-i con 0x7f78ac0a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f78ac132de0 0x7f78ac09c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 10 monclient(hunting): picked mon.noname-b con 0x7f78ac132de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78ac1392c0 0x7f78ac09c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 10 monclient(hunting): picked mon.noname-a con 0x7f78ac1392c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:23.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac1367f0 con 0x7f78ac1392c0 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac09d630 con 0x7f78ac132de0 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0a2390 con 0x7f78ac0a2930 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78b223f640 10 monclient(hunting): authenticate will time out at 868.268312s 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f78ac132de0 0x7f78ac09c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78abfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78ac1392c0 0x7f78ac09c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f78ac132de0 0x7f78ac09c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42922/0 (socket says 172.21.15.178:42922) 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 1 -- 172.21.15.178:0/309501519 learned_addr learned my addr 172.21.15.178:0/309501519 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 10 monclient(hunting): get_auth_request con 0x7f78ac132de0 auth_method 0 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78abfff640 10 monclient(hunting): get_auth_request con 0x7f78ac1392c0 auth_method 0 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78ab7fe640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f78ac0a2930 0x7f78ac141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 10 monclient(hunting): handle_auth_done global_id 5006 payload 824 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 1 -- 172.21.15.178:0/309501519 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f78ac0a2930 msgr2=0x7f78ac141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f78ac0a2930 0x7f78ac141c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 1 -- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78ac1392c0 msgr2=0x7f78ac09c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78ac1392c0 0x7f78ac09c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 1 monclient: found mon.noname-b 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 1 -- 172.21.15.178:0/309501519 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a4009390 con 0x7f78ac132de0 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78ab7fe640 10 monclient: get_auth_request con 0x7f78ac0a2930 auth_method 0 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78ab7fe640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f78ac0a2930 0x7f78ac141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78aaffd640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f78ac132de0 0x7f78ac09c100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78a000c610 tx=0x7f78a0007d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78a00070f0 con 0x7f78ac132de0 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.518+0000 7f78937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78937fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78937fe640 10 monclient: dump: 2024-09-15T01:34:23.536 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78937fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78b223f640 5 monclient: authenticate success, global_id 5006 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a0007250 con 0x7f78ac132de0 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78b223f640 10 monclient: _renew_subs 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78b223f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78b223f640 1 -- 172.21.15.178:0/309501519 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ac138290 con 0x7f78ac132de0 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78937fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78a0012400 con 0x7f78ac132de0 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78937fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78937fe640 10 monclient: dump: 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.538 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78b223f640 10 monclient: _renew_subs 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78a9ffb640 4 set_mon_vals no callback set 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78b223f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:23.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78b223f640 1 -- 172.21.15.178:0/309501519 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac142310 con 0x7f78ac132de0 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.519+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f78a0007400 con 0x7f78ac132de0 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78937fe640 1 --2- 172.21.15.178:0/309501519 >> v2:172.21.15.204:6812/28210 conn(0x7f78980397b0 0x7f789803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78b223f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78b223f640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac134b70 0x7f78ac134f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78b223f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78b223f640 1 -- 172.21.15.178:0/309501519 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac138420 con 0x7f78ac134b70 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5381+0+0 (secure 0 0 0) 0x7f78a004b100 con 0x7f78ac132de0 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78abfff640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac134b70 0x7f78ac134f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78abfff640 10 monclient: get_auth_request con 0x7f78ac134b70 auth_method 0 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78ab7fe640 1 --2- 172.21.15.178:0/309501519 >> v2:172.21.15.204:6812/28210 conn(0x7f78980397b0 0x7f789803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78ab7fe640 10 monclient: get_auth_request con 0x7f78980397b0 auth_method 0 2024-09-15T01:34:23.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78abfff640 10 monclient(hunting): handle_auth_done global_id 5138 payload 824 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78abfff640 1 -- 172.21.15.178:0/309501519 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78ac04c4a0 con 0x7f78ac134b70 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78abfff640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac134b70 0x7f78ac134f40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0a6ec0 tx=0x7f78a4032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78ab7fe640 1 --2- 172.21.15.178:0/309501519 >> v2:172.21.15.204:6812/28210 conn(0x7f78980397b0 0x7f789803bc60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f789c00c330 tx=0x7f789c0074d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78a403d3f0 con 0x7f78ac134b70 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78937fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78937fe640 10 monclient: dump: 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.541 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f78a4031a90 con 0x7f78ac134b70 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78937fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78937fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac134b70 msgr2=0x7f78ac134f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.520+0000 7f78937fe640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac134b70 0x7f78ac134f40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0a6ec0 tx=0x7f78a4032000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.521+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac134b70 msgr2=0x7f78ac134f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:23.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.521+0000 7f78937fe640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac134b70 0x7f78ac134f40 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0a6ec0 tx=0x7f78a4032000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.529+0000 7fa47cff9640 1 -- 172.21.15.204:0/2628863251 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 removed item id -11 name 'host1' from crush map v78) v1 ==== 147+0+0 (secure 0 0 0) 0x7fa478001250 con 0x7fa4800a2e30 2024-09-15T01:34:23.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.529+0000 7fa47cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.529+0000 7fa47cff9640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host1' from crush map 2024-09-15T01:34:23.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78b223f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:23.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78b223f640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac135480 0x7f78ac12fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78b223f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78b223f640 1 -- 172.21.15.178:0/309501519 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac136140 con 0x7f78ac135480 2024-09-15T01:34:23.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78abfff640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac135480 0x7f78ac12fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78abfff640 10 monclient: get_auth_request con 0x7f78ac135480 auth_method 0 2024-09-15T01:34:23.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78abfff640 10 monclient(hunting): handle_auth_done global_id 5147 payload 824 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.527+0000 7f78abfff640 1 -- 172.21.15.178:0/309501519 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f78ac04bc80 con 0x7f78ac135480 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.528+0000 7f78abfff640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac135480 0x7f78ac12fd40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009360 tx=0x7f78a403e000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.530+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78a404a070 con 0x7f78ac135480 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.530+0000 7f78937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.530+0000 7f78937fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.530+0000 7f78937fe640 10 monclient: dump: 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.546 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.530+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f78a40315d0 con 0x7f78ac135480 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.530+0000 7f78937fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.530+0000 7f78937fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.530+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac135480 msgr2=0x7f78ac12fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.530+0000 7f78937fe640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac135480 0x7f78ac12fd40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009360 tx=0x7f78a403e000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.530+0000 7f78937fe640 1 -- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac135480 msgr2=0x7f78ac12fd40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:23.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.530+0000 7f78937fe640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78ac135480 0x7f78ac12fd40 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009360 tx=0x7f78a403e000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.548 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:23.548 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:23.548 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:23.548 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:23.548 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:23.548 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:23.548 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:23.548 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.550 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:23.551 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.552 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.553 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.554 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.555 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.556 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:23.558 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.560 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.564 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.564 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:23.564 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.564 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.564 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.564 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:23.565 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.566 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.567 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.568 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.569 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.570 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:34:23.571 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.571 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.571 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:23.571 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:23.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.530+0000 7f78b223f640 1 -- 172.21.15.178:0/309501519 >> v2:172.21.15.204:6812/28210 conn(0x7f78980397b0 msgr2=0x7f789803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.531+0000 7f78b223f640 1 --2- 172.21.15.178:0/309501519 >> v2:172.21.15.204:6812/28210 conn(0x7f78980397b0 0x7f789803bc60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f789c00c330 tx=0x7f789c0074d0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.531+0000 7f78abfff640 1 -- 172.21.15.178:0/309501519 reap_dead start 2024-09-15T01:34:23.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.531+0000 7f78b223f640 10 monclient: shutdown 2024-09-15T01:34:23.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.531+0000 7f78b223f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.531+0000 7f78b223f640 1 -- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f78ac132de0 msgr2=0x7f78ac09c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.531+0000 7f78b223f640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f78ac132de0 0x7f78ac09c100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78a000c610 tx=0x7f78a0007d00 comp rx=0 tx=0).stop 2024-09-15T01:34:23.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.531+0000 7f78b223f640 1 -- 172.21.15.178:0/309501519 shutdown_connections 2024-09-15T01:34:23.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.531+0000 7f78b223f640 1 --2- 172.21.15.178:0/309501519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f78ac132de0 0x7f78ac09c100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.531+0000 7f78b223f640 1 -- 172.21.15.178:0/309501519 >> 172.21.15.178:0/309501519 conn(0x7f78ac0a5ed0 msgr2=0x7f78ac12e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.531+0000 7f78b223f640 1 -- 172.21.15.178:0/309501519 shutdown_connections 2024-09-15T01:34:23.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.531+0000 7f78b223f640 1 -- 172.21.15.178:0/309501519 wait complete. 2024-09-15T01:34:23.573 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:23.573 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:34:23.574 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:34:23.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.627+0000 7fa48722b640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.627+0000 7fa48722b640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.627+0000 7fa48722b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:23.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.627+0000 7fa48722b640 1 -- 172.21.15.204:0/2628863251 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) v1 -- 0x7fa4800008d0 con 0x7fa4800a2e30 2024-09-15T01:34:23.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.628+0000 7fa47cff9640 1 -- 172.21.15.204:0/2628863251 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 item -11 does not appear in that position v78) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa47801b730 con 0x7fa4800a2e30 2024-09-15T01:34:23.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.628+0000 7fa47cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.628+0000 7fa47cff9640 10 monclient: _finish_command 3 = system:0 item -11 does not appear in that position 2024-09-15T01:34:23.630 INFO:tasks.workunit.client.0.smithi204.stderr:item -11 does not appear in that position 2024-09-15T01:34:23.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 1 -- 172.21.15.204:0/2628863251 >> v2:172.21.15.204:6812/28210 conn(0x7fa4640397e0 msgr2=0x7fa46403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 1 --2- 172.21.15.204:0/2628863251 >> v2:172.21.15.204:6812/28210 conn(0x7fa4640397e0 0x7fa46403bc90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa474034000 tx=0x7fa474032000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 10 monclient: shutdown 2024-09-15T01:34:23.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 1 -- 172.21.15.204:0/2628863251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4800a2e30 msgr2=0x7fa48011e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 1 --2- 172.21.15.204:0/2628863251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4800a2e30 0x7fa48011e730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa478009fd0 tx=0x7fa478009900 comp rx=0 tx=0).stop 2024-09-15T01:34:23.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 1 -- 172.21.15.204:0/2628863251 shutdown_connections 2024-09-15T01:34:23.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 1 --2- 172.21.15.204:0/2628863251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4800a2e30 0x7fa48011e730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 1 --2- 172.21.15.204:0/2628863251 >> v2:172.21.15.204:6812/28210 conn(0x7fa4640397e0 0x7fa46403bc90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 1 --2- 172.21.15.204:0/2628863251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4800a0a40 0x7fa48011e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 1 --2- 172.21.15.204:0/2628863251 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa48009fb80 0x7fa4801316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 1 -- 172.21.15.204:0/2628863251 >> 172.21.15.204:0/2628863251 conn(0x7fa48006e860 msgr2=0x7fa48009ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 1 -- 172.21.15.204:0/2628863251 shutdown_connections 2024-09-15T01:34:23.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.630+0000 7fa48722b640 1 -- 172.21.15.204:0/2628863251 wait complete. 2024-09-15T01:34:23.641 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush unlink host1 foo 2024-09-15T01:34:23.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.665+0000 7fab90c99640 10 monclient: get_monmap_and_config 2024-09-15T01:34:23.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.665+0000 7fab90c99640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.665+0000 7fab90c99640 10 monclient: monmap: 2024-09-15T01:34:23.667 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.667 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:23.666229+0000 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:23.666229+0000 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.668 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.669 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:23.669 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:23.669 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 1 Processor -- start 2024-09-15T01:34:23.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 1 -- start start 2024-09-15T01:34:23.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 10 monclient: init 2024-09-15T01:34:23.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 10 monclient: _add_conns ranks=[2,4,9,5,6,10,7,3,8,0,1] 2024-09-15T01:34:23.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab8c0a2930 0x7fab8c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 10 monclient(hunting): picked mon.noname-b con 0x7fab8c0a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:23.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 0x7fab8c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 10 monclient(hunting): picked mon.noname-c con 0x7fab8c09faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:23.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fab8c0a08c0 0x7fab8c0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 10 monclient(hunting): picked mon.noname-a con 0x7fab8c0a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:23.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab8c09e0f0 con 0x7fab8c0a08c0 2024-09-15T01:34:23.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c09dd80 con 0x7fab8c0a2930 2024-09-15T01:34:23.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c0a2d00 con 0x7fab8c09faa0 2024-09-15T01:34:23.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab90c99640 10 monclient(hunting): authenticate will time out at 868.416704s 2024-09-15T01:34:23.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab83fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 0x7fab8c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8ad9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fab8c0a08c0 0x7fab8c0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8a59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab8c0a2930 0x7fab8c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab83fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 0x7fab8c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:33152/0 (socket says 172.21.15.178:33152) 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab83fff640 1 -- 172.21.15.178:0/4233418805 learned_addr learned my addr 172.21.15.178:0/4233418805 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab83fff640 10 monclient(hunting): get_auth_request con 0x7fab8c09faa0 auth_method 0 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab83fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8a59c640 10 monclient(hunting): get_auth_request con 0x7fab8c0a2930 auth_method 0 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8ad9d640 10 monclient(hunting): get_auth_request con 0x7fab8c0a08c0 auth_method 0 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.666+0000 7fab8ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab83fff640 10 monclient(hunting): handle_auth_done global_id 4999 payload 274 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab83fff640 1 -- 172.21.15.178:0/4233418805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab8c0a2930 msgr2=0x7fab8c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab83fff640 1 --2- 172.21.15.178:0/4233418805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab8c0a2930 0x7fab8c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab83fff640 1 -- 172.21.15.178:0/4233418805 >> v2:172.21.15.178:3300/0 conn(0x7fab8c0a08c0 msgr2=0x7fab8c0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab83fff640 1 --2- 172.21.15.178:0/4233418805 >> v2:172.21.15.178:3300/0 conn(0x7fab8c0a08c0 0x7fab8c0a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab83fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab83fff640 1 monclient: found mon.noname-c 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab83fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab83fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab83fff640 1 -- 172.21.15.178:0/4233418805 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c04bf60 con 0x7fab8c09faa0 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab8ad9d640 1 --2- 172.21.15.178:0/4233418805 >> v2:172.21.15.178:3300/0 conn(0x7fab8c0a08c0 0x7fab8c0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab8a59c640 1 --2- 172.21.15.178:0/4233418805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab8c0a2930 0x7fab8c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab83fff640 1 --2- 172.21.15.178:0/4233418805 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 0x7fab8c09fe70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab74009910 tx=0x7fab740091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=53e9935747ebba77 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab89d9b640 1 -- 172.21.15.178:0/4233418805 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fab7402ac90 con 0x7fab8c09faa0 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.667+0000 7fab89d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab89d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:23.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab89d9b640 10 monclient: dump: 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.684 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab89d9b640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab89d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab89d9b640 1 -- 172.21.15.178:0/4233418805 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7402ae10 con 0x7fab8c09faa0 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 5 monclient: authenticate success, global_id 4999 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab89d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab89d9b640 1 -- 172.21.15.178:0/4233418805 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fab74031ca0 con 0x7fab8c09faa0 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 4 set_mon_vals no callback set 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab89d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab89d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab89d9b640 10 monclient: dump: 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 10 monclient: shutdown 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 1 -- 172.21.15.178:0/4233418805 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 msgr2=0x7fab8c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 1 --2- 172.21.15.178:0/4233418805 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 0x7fab8c09fe70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab74009910 tx=0x7fab740091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 1 -- 172.21.15.178:0/4233418805 shutdown_connections 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 1 --2- 172.21.15.178:0/4233418805 >> v2:172.21.15.178:3300/0 conn(0x7fab8c0a08c0 0x7fab8c0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 1 --2- 172.21.15.178:0/4233418805 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 0x7fab8c09fe70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 1 --2- 172.21.15.178:0/4233418805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab8c0a2930 0x7fab8c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 1 -- 172.21.15.178:0/4233418805 >> 172.21.15.178:0/4233418805 conn(0x7fab8c0a5ed0 msgr2=0x7fab8c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 1 -- 172.21.15.178:0/4233418805 shutdown_connections 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.668+0000 7fab90c99640 1 -- 172.21.15.178:0/4233418805 wait complete. 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient: monmap: 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.688 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 1 Processor -- start 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 1 -- start start 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient: init 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient: _add_conns ranks=[4,7,3,2,6,0,5,8,1] 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 0x7fab8c11ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient(hunting): picked mon.noname-c con 0x7fab8c09faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:23.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fab8c0a08c0 0x7fab8c131990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient(hunting): picked mon.noname-i con 0x7fab8c0a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab8c0a2930 0x7fab8c131ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient(hunting): picked mon.noname-g con 0x7fab8c0a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c11d830 con 0x7fab8c0a2930 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c0a23c0 con 0x7fab8c09faa0 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c09d660 con 0x7fab8c0a08c0 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab90c99640 10 monclient(hunting): authenticate will time out at 868.419877s 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab8a59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 0x7fab8c11ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab8a59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 0x7fab8c11ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:33160/0 (socket says 172.21.15.178:33160) 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab8a59c640 1 -- 172.21.15.178:0/3923192783 learned_addr learned my addr 172.21.15.178:0/3923192783 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab8a59c640 10 monclient(hunting): get_auth_request con 0x7fab8c09faa0 auth_method 0 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab8a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab8a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.669+0000 7fab8a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab83fff640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fab8c0a08c0 0x7fab8c131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8ad9d640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab8c0a2930 0x7fab8c131ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab83fff640 10 monclient(hunting): get_auth_request con 0x7fab8c0a08c0 auth_method 0 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab83fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8ad9d640 10 monclient(hunting): get_auth_request con 0x7fab8c0a2930 auth_method 0 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 10 monclient(hunting): handle_auth_done global_id 5008 payload 824 2024-09-15T01:34:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 1 -- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fab8c0a08c0 msgr2=0x7fab8c131990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fab8c0a08c0 0x7fab8c131990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 1 -- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab8c0a2930 msgr2=0x7fab8c131ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab8c0a2930 0x7fab8c131ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 1 monclient: found mon.noname-c 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 1 -- 172.21.15.178:0/3923192783 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab84009a20 con 0x7fab8c09faa0 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8a59c640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 0x7fab8c11ca00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00ec80 tx=0x7fab7c0090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fab7c00eec0 con 0x7fab8c09faa0 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab827fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab827fc640 10 monclient: dump: 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.694 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab827fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c013470 con 0x7fab8c09faa0 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab90c99640 5 monclient: authenticate success, global_id 5008 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab90c99640 10 monclient: _renew_subs 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab90c99640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab90c99640 1 -- 172.21.15.178:0/3923192783 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7c009390 con 0x7fab8c09faa0 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab827fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fab7c013a60 con 0x7fab8c09faa0 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab827fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab827fc640 10 monclient: dump: 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.696 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab8959a640 4 set_mon_vals no callback set 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab90c99640 10 monclient: _renew_subs 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab90c99640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.670+0000 7fab90c99640 1 -- 172.21.15.178:0/3923192783 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab74009450 con 0x7fab8c09faa0 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fab7c013bc0 con 0x7fab8c09faa0 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab90c99640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab90c99640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c11eab0 0x7fab8c141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab90c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab90c99640 1 -- 172.21.15.178:0/3923192783 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c04c4a0 con 0x7fab8c11eab0 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab827fc640 1 --2- 172.21.15.178:0/3923192783 >> v2:172.21.15.204:6812/28210 conn(0x7fab6c039710 0x7fab6c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5373+0+0 (secure 0 0 0) 0x7fab7c04bd10 con 0x7fab8c09faa0 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab83fff640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c11eab0 0x7fab8c141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab83fff640 10 monclient: get_auth_request con 0x7fab8c11eab0 auth_method 0 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab83fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab8ad9d640 1 --2- 172.21.15.178:0/3923192783 >> v2:172.21.15.204:6812/28210 conn(0x7fab6c039710 0x7fab6c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab8ad9d640 10 monclient: get_auth_request con 0x7fab6c039710 auth_method 0 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.671+0000 7fab83fff640 10 monclient(hunting): handle_auth_done global_id 5554 payload 824 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab83fff640 1 -- 172.21.15.178:0/3923192783 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab8c05ed70 con 0x7fab8c11eab0 2024-09-15T01:34:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab83fff640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c11eab0 0x7fab8c141d90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0a3290 tx=0x7fab74032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab8ad9d640 1 --2- 172.21.15.178:0/3923192783 >> v2:172.21.15.204:6812/28210 conn(0x7fab6c039710 0x7fab6c03bbc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fab8c09b800 tx=0x7fab8400b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fab7402ac90 con 0x7fab8c11eab0 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab827fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab827fc640 10 monclient: dump: 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.699 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fab7403c3e0 con 0x7fab8c11eab0 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab827fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab827fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c11eab0 msgr2=0x7fab8c141d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab827fc640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c11eab0 0x7fab8c141d90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0a3290 tx=0x7fab74032000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c11eab0 msgr2=0x7fab8c141d90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:23.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.672+0000 7fab827fc640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c11eab0 0x7fab8c141d90 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0a3290 tx=0x7fab74032000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab90c99640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:23.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab90c99640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c1422d0 0x7fab8c1426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab90c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab90c99640 1 -- 172.21.15.178:0/3923192783 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c04bc80 con 0x7fab8c1422d0 2024-09-15T01:34:23.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab83fff640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c1422d0 0x7fab8c1426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab83fff640 10 monclient: get_auth_request con 0x7fab8c1422d0 auth_method 0 2024-09-15T01:34:23.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab83fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab83fff640 10 monclient(hunting): handle_auth_done global_id 5563 payload 824 2024-09-15T01:34:23.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab83fff640 1 -- 172.21.15.178:0/3923192783 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fab8c142c10 con 0x7fab8c1422d0 2024-09-15T01:34:23.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.678+0000 7fab83fff640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c1422d0 0x7fab8c1426d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fab74009910 tx=0x7fab7403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fab7403cd00 con 0x7fab8c1422d0 2024-09-15T01:34:23.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab827fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab827fc640 10 monclient: dump: 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.704 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.705 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.705 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.705 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9311 (secure 0 0 0) 0x7fab7403ce60 con 0x7fab8c1422d0 2024-09-15T01:34:23.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab827fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:23.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab827fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:23.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c1422d0 msgr2=0x7fab8c1426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab827fc640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c1422d0 0x7fab8c1426d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fab74009910 tx=0x7fab7403d000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab827fc640 1 -- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c1422d0 msgr2=0x7fab8c1426d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:23.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab827fc640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab8c1422d0 0x7fab8c1426d0 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fab74009910 tx=0x7fab7403d000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.705 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:23.706 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:23.708 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:23.709 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.710 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.711 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.712 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:23.712 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:23.712 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.712 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.712 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.712 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.712 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.712 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:23.712 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:23.712 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.712 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.712 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:23.713 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.714 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.715 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.716 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.716 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:23.716 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:23.716 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.716 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.716 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.716 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.716 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:23.716 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.716 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.716 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.716 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.717 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.718 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.719 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:23.720 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:23.721 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.722 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.723 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:23.724 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.725 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.726 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.727 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.728 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:23.728 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.728 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.728 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.728 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:23.728 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:23.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab90c99640 1 -- 172.21.15.178:0/3923192783 >> v2:172.21.15.204:6812/28210 conn(0x7fab6c039710 msgr2=0x7fab6c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab90c99640 1 --2- 172.21.15.178:0/3923192783 >> v2:172.21.15.204:6812/28210 conn(0x7fab6c039710 0x7fab6c03bbc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fab8c09b800 tx=0x7fab8400b040 comp rx=0 tx=0).stop 2024-09-15T01:34:23.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab90c99640 10 monclient: shutdown 2024-09-15T01:34:23.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab90c99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab90c99640 1 -- 172.21.15.178:0/3923192783 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 msgr2=0x7fab8c11ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab90c99640 1 --2- 172.21.15.178:0/3923192783 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fab8c09faa0 0x7fab8c11ca00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00ec80 tx=0x7fab7c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab8ad9d640 1 -- 172.21.15.178:0/3923192783 reap_dead start 2024-09-15T01:34:23.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab8ad9d640 1 -- 172.21.15.178:0/3923192783 reap_dead start 2024-09-15T01:34:23.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab90c99640 1 -- 172.21.15.178:0/3923192783 shutdown_connections 2024-09-15T01:34:23.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab90c99640 1 -- 172.21.15.178:0/3923192783 >> 172.21.15.178:0/3923192783 conn(0x7fab8c0a5ed0 msgr2=0x7fab8c0a5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab90c99640 1 -- 172.21.15.178:0/3923192783 shutdown_connections 2024-09-15T01:34:23.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.679+0000 7fab90c99640 1 -- 172.21.15.178:0/3923192783 wait complete. 2024-09-15T01:34:23.731 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:23.731 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:34:23.731 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:34:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.715+0000 7f37d46a8640 10 monclient: get_monmap_and_config 2024-09-15T01:34:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.715+0000 7f37d46a8640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.716+0000 7f37d46a8640 10 monclient: monmap: 2024-09-15T01:34:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:23.717473+0000 2024-09-15T01:34:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:23.717473+0000 2024-09-15T01:34:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.716+0000 7f37d46a8640 1 Processor -- start 2024-09-15T01:34:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.716+0000 7f37d46a8640 1 -- start start 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.716+0000 7f37d46a8640 10 monclient: init 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 10 monclient: _add_conns ranks=[9,5,0,1,6,8,4,10,7,2,3] 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f37cc0a2e30 0x7f37cc132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 10 monclient(hunting): picked mon.noname-a con 0x7f37cc0a2e30 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f37cc09fb80 0x7f37cc09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 10 monclient(hunting): picked mon.noname-h con 0x7f37cc09fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f37cc0a0a40 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37cc04b720 con 0x7f37cc0a0a40 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37cc09e6c0 con 0x7f37cc0a2e30 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0a04b0 con 0x7f37cc09fb80 2024-09-15T01:34:23.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d46a8640 10 monclient(hunting): authenticate will time out at 879.360470s 2024-09-15T01:34:23.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f37cc09fb80 0x7f37cc09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f37cc09fb80 0x7f37cc09ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:55724/0 (socket says 172.21.15.204:55724) 2024-09-15T01:34:23.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 1 -- 172.21.15.204:0/3847426918 learned_addr learned my addr 172.21.15.204:0/3847426918 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:23.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 10 monclient(hunting): get_auth_request con 0x7f37cc09fb80 auth_method 0 2024-09-15T01:34:23.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d2444640 1 --2- 172.21.15.204:0/3847426918 >> v2:172.21.15.178:3300/0 conn(0x7f37cc0a2e30 0x7f37cc132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d2444640 10 monclient(hunting): get_auth_request con 0x7f37cc0a2e30 auth_method 0 2024-09-15T01:34:23.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d2444640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d2444640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d2444640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d2444640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d2444640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d2444640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 10 monclient(hunting): handle_auth_done global_id 5063 payload 274 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 1 -- 172.21.15.204:0/3847426918 >> v2:172.21.15.178:3300/0 conn(0x7f37cc0a2e30 msgr2=0x7f37cc132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 1 --2- 172.21.15.204:0/3847426918 >> v2:172.21.15.178:3300/0 conn(0x7f37cc0a2e30 0x7f37cc132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 1 -- 172.21.15.204:0/3847426918 >> v1:172.21.15.178:6789/0 conn(0x7f37cc0a0a40 legacy=0x7f37cc0a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 1 monclient: found mon.noname-h 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.717+0000 7f37d1c43640 1 -- 172.21.15.204:0/3847426918 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc0a3200 con 0x7f37cc09fb80 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1c43640 1 --2- 172.21.15.204:0/3847426918 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f37cc09fb80 0x7f37cc09ff70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009910 tx=0x7f37c80091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=3b963f399bc9b530 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1442640 1 -- 172.21.15.204:0/3847426918 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f37c802ad20 con 0x7f37cc09fb80 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1442640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1442640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1442640 10 monclient: dump: 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.746 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1442640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1442640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d46a8640 5 monclient: authenticate success, global_id 5063 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1442640 1 -- 172.21.15.204:0/3847426918 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c8031820 con 0x7f37cc09fb80 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1442640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d46a8640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1442640 1 -- 172.21.15.204:0/3847426918 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f37c802aea0 con 0x7f37cc09fb80 2024-09-15T01:34:23.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d46a8640 4 set_mon_vals no callback set 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1442640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1442640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.718+0000 7f37d1442640 10 monclient: dump: 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.748 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.719+0000 7f37d46a8640 10 monclient: shutdown 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.719+0000 7f37d46a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.719+0000 7f37d46a8640 1 -- 172.21.15.204:0/3847426918 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f37cc09fb80 msgr2=0x7f37cc09ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.719+0000 7f37d46a8640 1 --2- 172.21.15.204:0/3847426918 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f37cc09fb80 0x7f37cc09ff70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009910 tx=0x7f37c80091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.719+0000 7f37d46a8640 1 -- 172.21.15.204:0/3847426918 shutdown_connections 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.719+0000 7f37d46a8640 1 --2- 172.21.15.204:0/3847426918 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f37cc09fb80 0x7f37cc09ff70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.719+0000 7f37d46a8640 1 --2- 172.21.15.204:0/3847426918 >> v2:172.21.15.178:3300/0 conn(0x7f37cc0a2e30 0x7f37cc132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.719+0000 7f37d46a8640 1 -- 172.21.15.204:0/3847426918 >> 172.21.15.204:0/3847426918 conn(0x7f37cc06e860 msgr2=0x7f37cc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.719+0000 7f37d46a8640 1 -- 172.21.15.204:0/3847426918 shutdown_connections 2024-09-15T01:34:23.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.719+0000 7f37d46a8640 1 -- 172.21.15.204:0/3847426918 wait complete. 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.719+0000 7f37d46a8640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.719+0000 7f37d46a8640 10 monclient: monmap: 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.750 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 1 Processor -- start 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 1 -- start start 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 10 monclient: init 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 10 monclient: _add_conns ranks=[0,2,1,8,4,3,6,5,7] 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37cc09fb80 0x7f37cc1317b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 10 monclient(hunting): picked mon.noname-a con 0x7f37cc09fb80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37cc0a0a40 0x7f37cc11e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 10 monclient(hunting): picked mon.noname-b con 0x7f37cc0a0a40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37cc0a2e30 0x7f37cc11e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 10 monclient(hunting): picked mon.noname-f con 0x7f37cc0a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc09ca40 con 0x7f37cc09fb80 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc09d650 con 0x7f37cc0a2e30 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc11fe40 con 0x7f37cc0a0a40 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d46a8640 10 monclient(hunting): authenticate will time out at 879.363634s 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2c45640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37cc0a2e30 0x7f37cc11e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2c45640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37cc0a2e30 0x7f37cc11e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48918/0 (socket says 172.21.15.204:48918) 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2c45640 1 -- 172.21.15.204:0/1568560114 learned_addr learned my addr 172.21.15.204:0/1568560114 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2c45640 10 monclient(hunting): get_auth_request con 0x7f37cc0a2e30 auth_method 0 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2c45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2c45640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2c45640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2444640 1 --2- 172.21.15.204:0/1568560114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37cc09fb80 0x7f37cc1317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2444640 10 monclient(hunting): get_auth_request con 0x7f37cc09fb80 auth_method 0 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2444640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2444640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2444640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d1c43640 1 --2- 172.21.15.204:0/1568560114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37cc0a0a40 0x7f37cc11e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2c45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2c45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2c45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d1c43640 10 monclient(hunting): get_auth_request con 0x7f37cc0a0a40 auth_method 0 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d1c43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d1c43640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d1c43640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2444640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2444640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2444640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d1c43640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d1c43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d1c43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2c45640 10 monclient(hunting): handle_auth_done global_id 5572 payload 824 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.720+0000 7f37d2c45640 1 -- 172.21.15.204:0/1568560114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37cc0a0a40 msgr2=0x7f37cc11e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d2c45640 1 --2- 172.21.15.204:0/1568560114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37cc0a0a40 0x7f37cc11e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d2c45640 1 -- 172.21.15.204:0/1568560114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37cc09fb80 msgr2=0x7f37cc1317b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d2c45640 1 --2- 172.21.15.204:0/1568560114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37cc09fb80 0x7f37cc1317b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d2c45640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d2c45640 1 monclient: found mon.noname-f 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d2c45640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d2c45640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d2c45640 1 -- 172.21.15.204:0/1568560114 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c8009450 con 0x7f37cc0a2e30 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d2c45640 1 --2- 172.21.15.204:0/1568560114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37cc0a2e30 0x7f37cc11e880 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f37c400c890 tx=0x7f37c4001690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37be7fc640 1 -- 172.21.15.204:0/1568560114 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f37c400ce80 con 0x7f37cc0a2e30 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37be7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37be7fc640 10 monclient: dump: 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37be7fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37be7fc640 1 -- 172.21.15.204:0/1568560114 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c400acd0 con 0x7f37cc0a2e30 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d46a8640 5 monclient: authenticate success, global_id 5572 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37be7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d46a8640 10 monclient: _renew_subs 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37be7fc640 1 -- 172.21.15.204:0/1568560114 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f37c400ae40 con 0x7f37cc0a2e30 2024-09-15T01:34:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d46a8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d46a8640 1 -- 172.21.15.204:0/1568560114 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37cc120590 con 0x7f37cc0a2e30 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37be7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d0c41640 4 set_mon_vals no callback set 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37be7fc640 10 monclient: dump: 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d46a8640 10 monclient: _renew_subs 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d46a8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.721+0000 7f37d46a8640 1 -- 172.21.15.204:0/1568560114 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc133f80 con 0x7f37cc0a2e30 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.722+0000 7f37be7fc640 1 -- 172.21.15.204:0/1568560114 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f37c4003030 con 0x7f37cc0a2e30 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.722+0000 7f37d46a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.722+0000 7f37d46a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.722+0000 7f37d46a8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.722+0000 7f37d46a8640 1 -- 172.21.15.204:0/1568560114 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37cc070cc0 con 0x7f37cc0a2e30 2024-09-15T01:34:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.722+0000 7f37be7fc640 1 --2- 172.21.15.204:0/1568560114 >> v2:172.21.15.204:6812/28210 conn(0x7f37b00397b0 0x7f37b003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.722+0000 7f37be7fc640 1 -- 172.21.15.204:0/1568560114 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5373+0+0 (secure 0 0 0) 0x7f37c404b520 con 0x7f37cc0a2e30 2024-09-15T01:34:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.722+0000 7f37d2444640 1 --2- 172.21.15.204:0/1568560114 >> v2:172.21.15.204:6812/28210 conn(0x7f37b00397b0 0x7f37b003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.722+0000 7f37d2444640 10 monclient: get_auth_request con 0x7f37b00397b0 auth_method 0 2024-09-15T01:34:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.722+0000 7f37d2444640 1 --2- 172.21.15.204:0/1568560114 >> v2:172.21.15.204:6812/28210 conn(0x7f37b00397b0 0x7f37b003bc60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009c80 tx=0x7f37c00090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.726+0000 7f37be7fc640 1 -- 172.21.15.204:0/1568560114 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f37c4010c20 con 0x7f37cc0a2e30 2024-09-15T01:34:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.726+0000 7f37be7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.726+0000 7f37be7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:23.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.826+0000 7fa13be6f640 10 monclient: get_monmap_and_config 2024-09-15T01:34:23.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.826+0000 7fa13be6f640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.826+0000 7fa13be6f640 10 monclient: monmap: 2024-09-15T01:34:23.828 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.828 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.828 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:23.827238+0000 2024-09-15T01:34:23.828 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:23.827238+0000 2024-09-15T01:34:23.828 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.828 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.829 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:23.829 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:23.829 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.829 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.829 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.829 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.829 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.829 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.829 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.829 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:23.829 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:23.829 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.826+0000 7fa13be6f640 1 Processor -- start 2024-09-15T01:34:23.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 1 -- start start 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 10 monclient: init 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 10 monclient: _add_conns ranks=[9,0,6,7,10,5,4,8,1,2,3] 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa1340a2930 0x7fa134132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 10 monclient(hunting): picked mon.noname-a con 0x7fa1340a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa13409faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa13409fe70 0x7fa1340a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 10 monclient(hunting): picked mon.noname-d con 0x7fa13409fe70 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13404bf60 con 0x7fa13409faa0 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13409dd80 con 0x7fa1340a2930 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340a2d00 con 0x7fa13409fe70 2024-09-15T01:34:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13be6f640 10 monclient(hunting): authenticate will time out at 868.577631s 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa139c0b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa1340a2930 0x7fa134132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13a40c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa13409fe70 0x7fa1340a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa139c0b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa1340a2930 0x7fa134132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:58554/0 (socket says 172.21.15.178:58554) 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13a40c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa13409fe70 0x7fa1340a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42668/0 (socket says 172.21.15.178:42668) 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa139c0b640 1 -- 172.21.15.178:0/2761728159 learned_addr learned my addr 172.21.15.178:0/2761728159 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13940a640 1 --1- 172.21.15.178:0/2761728159 >> v1:172.21.15.178:6789/0 conn(0x7fa13409faa0 0x7fa1340a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:34378/0 (socket says 172.21.15.178:34378) 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa139c0b640 10 monclient(hunting): get_auth_request con 0x7fa1340a2930 auth_method 0 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa139c0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa139c0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa139c0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13a40c640 10 monclient(hunting): get_auth_request con 0x7fa13409fe70 auth_method 0 2024-09-15T01:34:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13a40c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13a40c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13a40c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13a40c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13a40c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa13a40c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa139c0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa139c0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.827+0000 7fa139c0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13a40c640 10 monclient(hunting): handle_auth_done global_id 14154 payload 274 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13a40c640 1 -- 172.21.15.178:0/2761728159 >> v2:172.21.15.178:3300/0 conn(0x7fa1340a2930 msgr2=0x7fa134132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13a40c640 1 --2- 172.21.15.178:0/2761728159 >> v2:172.21.15.178:3300/0 conn(0x7fa1340a2930 0x7fa134132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13a40c640 1 -- 172.21.15.178:0/2761728159 >> v1:172.21.15.178:6789/0 conn(0x7fa13409faa0 legacy=0x7fa1340a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13a40c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13a40c640 1 monclient: found mon.noname-d 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13a40c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13a40c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 1 -- 172.21.15.178:0/2761728159 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936976188 0 0) 0x7fa13404bf60 con 0x7fa13409faa0 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13a40c640 1 -- 172.21.15.178:0/2761728159 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa134133440 con 0x7fa13409fe70 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa139c0b640 1 --2- 172.21.15.178:0/2761728159 >> v2:172.21.15.178:3300/0 conn(0x7fa1340a2930 0x7fa134132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13a40c640 1 --2- 172.21.15.178:0/2761728159 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa13409fe70 0x7fa1340a0fd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009e60 tx=0x7fa12c009780 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f9ef904f561c37fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 1 -- 172.21.15.178:0/2761728159 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa12c0079d0 con 0x7fa13409fe70 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 10 monclient: dump: 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13be6f640 5 monclient: authenticate success, global_id 14154 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13be6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 1 -- 172.21.15.178:0/2761728159 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c02d050 con 0x7fa13409fe70 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13be6f640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa13be6f640 4 set_mon_vals no callback set 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 1 -- 172.21.15.178:0/2761728159 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa12c007b50 con 0x7fa13409fe70 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.828+0000 7fa138c09640 10 monclient: dump: 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.829+0000 7fa13be6f640 10 monclient: shutdown 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.829+0000 7fa13be6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.829+0000 7fa13be6f640 1 -- 172.21.15.178:0/2761728159 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa13409fe70 msgr2=0x7fa1340a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.829+0000 7fa13be6f640 1 --2- 172.21.15.178:0/2761728159 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa13409fe70 0x7fa1340a0fd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009e60 tx=0x7fa12c009780 comp rx=0 tx=0).stop 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.829+0000 7fa13be6f640 1 -- 172.21.15.178:0/2761728159 shutdown_connections 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.829+0000 7fa13be6f640 1 --2- 172.21.15.178:0/2761728159 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa13409fe70 0x7fa1340a0fd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.829+0000 7fa13be6f640 1 --2- 172.21.15.178:0/2761728159 >> v2:172.21.15.178:3300/0 conn(0x7fa1340a2930 0x7fa134132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.829+0000 7fa13be6f640 1 -- 172.21.15.178:0/2761728159 >> 172.21.15.178:0/2761728159 conn(0x7fa1340a5ed0 msgr2=0x7fa1340a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.829+0000 7fa13be6f640 1 -- 172.21.15.178:0/2761728159 shutdown_connections 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.829+0000 7fa13be6f640 1 -- 172.21.15.178:0/2761728159 wait complete. 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient: monmap: 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 1 Processor -- start 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 1 -- start start 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient: init 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient: _add_conns ranks=[2,8,5,3,6,7,1,0,4] 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa13409faa0 0x7fa134141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient(hunting): picked mon.noname-b con 0x7fa13409faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa13409fe70 0x7fa134131a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient(hunting): picked mon.noname-e con 0x7fa13409fe70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa1340a2930 0x7fa13411e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient(hunting): picked mon.noname-h con 0x7fa1340a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa134121b80 con 0x7fa13409faa0 2024-09-15T01:34:23.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340a2390 con 0x7fa1340a2930 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa13409d630 con 0x7fa13409fe70 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13be6f640 10 monclient(hunting): authenticate will time out at 868.580809s 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa139c0b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa13409faa0 0x7fa134141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13940a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa13409fe70 0x7fa134131a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa139c0b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa13409faa0 0x7fa134141c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42928/0 (socket says 172.21.15.178:42928) 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa139c0b640 1 -- 172.21.15.178:0/52517495 learned_addr learned my addr 172.21.15.178:0/52517495 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13940a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa13409fe70 0x7fa134131a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54244/0 (socket says 172.21.15.178:54244) 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa139c0b640 10 monclient(hunting): get_auth_request con 0x7fa13409faa0 auth_method 0 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa139c0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa139c0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa139c0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13940a640 10 monclient(hunting): get_auth_request con 0x7fa13409fe70 auth_method 0 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13940a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13940a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.830+0000 7fa13940a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13940a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13940a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13940a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13a40c640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa1340a2930 0x7fa13411e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13a40c640 10 monclient(hunting): get_auth_request con 0x7fa1340a2930 auth_method 0 2024-09-15T01:34:23.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13a40c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13a40c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13a40c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 10 monclient(hunting): handle_auth_done global_id 5033 payload 824 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 1 -- 172.21.15.178:0/52517495 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa13409fe70 msgr2=0x7fa134131a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa13409fe70 0x7fa134131a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 1 -- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa1340a2930 msgr2=0x7fa13411e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa1340a2930 0x7fa13411e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 10 monclient: _finish_hunting 0 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 1 monclient: found mon.noname-b 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 1 -- 172.21.15.178:0/52517495 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa128009450 con 0x7fa13409faa0 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13a40c640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa1340a2930 0x7fa13411e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:23.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa139c0b640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa13409faa0 0x7fa134141c50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa12800ec80 tx=0x7fa1280091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa128009730 con 0x7fa13409faa0 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa125ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa125ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa125ffb640 10 monclient: dump: 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa125ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa125ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13be6f640 5 monclient: authenticate success, global_id 5033 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12800eec0 con 0x7fa13409faa0 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13be6f640 10 monclient: _renew_subs 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13be6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:23.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13be6f640 1 -- 172.21.15.178:0/52517495 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12c009a20 con 0x7fa13409faa0 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa125ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa128013d70 con 0x7fa13409faa0 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa125ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa125ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa125ffb640 10 monclient: dump: 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.854 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13be6f640 10 monclient: _renew_subs 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13be6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa127fff640 4 set_mon_vals no callback set 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.831+0000 7fa13be6f640 1 -- 172.21.15.178:0/52517495 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa134120880 con 0x7fa13409faa0 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa12800cb60 con 0x7fa13409faa0 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa13be6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa13be6f640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa134131f70 0x7fa13411eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa13be6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa13be6f640 1 -- 172.21.15.178:0/52517495 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa134132340 con 0x7fa134131f70 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa125ffb640 1 --2- 172.21.15.178:0/52517495 >> v2:172.21.15.204:6812/28210 conn(0x7fa1180396e0 0x7fa11803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5373+0+0 (secure 0 0 0) 0x7fa12804baa0 con 0x7fa13409faa0 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa13940a640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa134131f70 0x7fa13411eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa13940a640 10 monclient: get_auth_request con 0x7fa134131f70 auth_method 0 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa13940a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa13940a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa13940a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.832+0000 7fa13a40c640 1 --2- 172.21.15.178:0/52517495 >> v2:172.21.15.204:6812/28210 conn(0x7fa1180396e0 0x7fa11803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa13a40c640 10 monclient: get_auth_request con 0x7fa1180396e0 auth_method 0 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa13940a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa13940a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa13940a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa13940a640 10 monclient(hunting): handle_auth_done global_id 14160 payload 824 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa13940a640 1 -- 172.21.15.178:0/52517495 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa13404c4a0 con 0x7fa134131f70 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa13a40c640 1 --2- 172.21.15.178:0/52517495 >> v2:172.21.15.204:6812/28210 conn(0x7fa1180396e0 0x7fa11803bb90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa12c034000 tx=0x7fa12c037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa13940a640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa134131f70 0x7fa13411eab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa13411ff80 tx=0x7fa130007e40 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa130011a10 con 0x7fa134131f70 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa125ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa125ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa125ffb640 10 monclient: dump: 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.857 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fa130011590 con 0x7fa134131f70 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa125ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:23.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa125ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:23.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa134131f70 msgr2=0x7fa13411eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa125ffb640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa134131f70 0x7fa13411eab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa13411ff80 tx=0x7fa130007e40 comp rx=0 tx=0).stop 2024-09-15T01:34:23.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa134131f70 msgr2=0x7fa13411eab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:23.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.833+0000 7fa125ffb640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa134131f70 0x7fa13411eab0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13be6f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:23.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13be6f640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa13411eff0 0x7fa134099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13be6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13be6f640 1 -- 172.21.15.178:0/52517495 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13404bc80 con 0x7fa13411eff0 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13940a640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa13411eff0 0x7fa134099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13940a640 10 monclient: get_auth_request con 0x7fa13411eff0 auth_method 0 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13940a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13940a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13940a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13940a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13940a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13940a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13940a640 10 monclient(hunting): handle_auth_done global_id 14169 payload 824 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.839+0000 7fa13940a640 1 -- 172.21.15.178:0/52517495 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa13405ed70 con 0x7fa13411eff0 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa13940a640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa13411eff0 0x7fa134099a50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa130007800 tx=0x7fa130015000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:23.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1300208f0 con 0x7fa13411eff0 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa125ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa125ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa125ffb640 10 monclient: dump: 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:23.862 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:23.863 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:23.863 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:23.863 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:23.863 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7fa130020470 con 0x7fa13411eff0 2024-09-15T01:34:23.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa125ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:23.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa125ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:23.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa13411eff0 msgr2=0x7fa134099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa125ffb640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa13411eff0 0x7fa134099a50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa130007800 tx=0x7fa130015000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa125ffb640 1 -- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa13411eff0 msgr2=0x7fa134099a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:23.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa125ffb640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa13411eff0 0x7fa134099a50 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa130007800 tx=0x7fa130015000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.864 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:34:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:23.886 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:23.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa13be6f640 1 -- 172.21.15.178:0/52517495 >> v2:172.21.15.204:6812/28210 conn(0x7fa1180396e0 msgr2=0x7fa11803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa13be6f640 1 --2- 172.21.15.178:0/52517495 >> v2:172.21.15.204:6812/28210 conn(0x7fa1180396e0 0x7fa11803bb90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa12c034000 tx=0x7fa12c037000 comp rx=0 tx=0).stop 2024-09-15T01:34:23.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa13be6f640 10 monclient: shutdown 2024-09-15T01:34:23.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa13be6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa13be6f640 1 -- 172.21.15.178:0/52517495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa13409faa0 msgr2=0x7fa134141c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa13be6f640 1 --2- 172.21.15.178:0/52517495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa13409faa0 0x7fa134141c50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa12800ec80 tx=0x7fa1280091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa13a40c640 1 -- 172.21.15.178:0/52517495 reap_dead start 2024-09-15T01:34:23.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa13a40c640 1 -- 172.21.15.178:0/52517495 reap_dead start 2024-09-15T01:34:23.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa13be6f640 1 -- 172.21.15.178:0/52517495 shutdown_connections 2024-09-15T01:34:23.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.840+0000 7fa13be6f640 1 -- 172.21.15.178:0/52517495 >> 172.21.15.178:0/52517495 conn(0x7fa1340a5ed0 msgr2=0x7fa13409e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.841+0000 7fa13be6f640 1 -- 172.21.15.178:0/52517495 shutdown_connections 2024-09-15T01:34:23.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.841+0000 7fa13be6f640 1 -- 172.21.15.178:0/52517495 wait complete. 2024-09-15T01:34:23.888 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:23.888 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:34:23.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:34:23.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.887+0000 7f37d46a8640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.887+0000 7f37d46a8640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.887+0000 7f37d46a8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:23.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.887+0000 7f37d46a8640 1 -- 172.21.15.204:0/1568560114 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) v1 -- 0x7f37cc0a1f10 con 0x7f37cc0a2e30 2024-09-15T01:34:23.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.888+0000 7f37be7fc640 1 -- 172.21.15.204:0/1568560114 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 item -11 does not appear in that position v78) v1 ==== 141+0+0 (secure 0 0 0) 0x7f37c4019570 con 0x7f37cc0a2e30 2024-09-15T01:34:23.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.888+0000 7f37be7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.888+0000 7f37be7fc640 10 monclient: _finish_command 2 = system:0 item -11 does not appear in that position 2024-09-15T01:34:23.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.980+0000 7f341bb00640 10 monclient: get_monmap_and_config 2024-09-15T01:34:23.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.980+0000 7f341bb00640 10 monclient: build_initial_monmap 2024-09-15T01:34:23.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.980+0000 7f341bb00640 10 monclient: monmap: 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:23.981300+0000 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:23.981300+0000 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:23.983 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:23.984 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:23.984 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:23.984 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:23.984 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:23.984 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:23.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.980+0000 7f341bb00640 1 Processor -- start 2024-09-15T01:34:23.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 1 -- start start 2024-09-15T01:34:23.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 10 monclient: init 2024-09-15T01:34:23.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.985+0000 7f37d46a8640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.985+0000 7f37d46a8640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.985+0000 7f37d46a8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:23.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.985+0000 7f37d46a8640 1 -- 172.21.15.204:0/1568560114 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) v1 -- 0x7f37cc0008d0 con 0x7f37cc0a2e30 2024-09-15T01:34:23.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.986+0000 7f37be7fc640 1 -- 172.21.15.204:0/1568560114 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 item -11 does not appear in that position v78) v1 ==== 141+0+0 (secure 0 0 0) 0x7f37c4019760 con 0x7f37cc0a2e30 2024-09-15T01:34:23.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.986+0000 7f37be7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-09-15T01:34:23.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.986+0000 7f37be7fc640 10 monclient: _finish_command 3 = system:0 item -11 does not appear in that position 2024-09-15T01:34:23.992 INFO:tasks.workunit.client.0.smithi204.stderr:item -11 does not appear in that position 2024-09-15T01:34:23.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 1 -- 172.21.15.204:0/1568560114 >> v2:172.21.15.204:6812/28210 conn(0x7f37b00397b0 msgr2=0x7f37b003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 1 --2- 172.21.15.204:0/1568560114 >> v2:172.21.15.204:6812/28210 conn(0x7f37b00397b0 0x7f37b003bc60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009c80 tx=0x7f37c00090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 10 monclient: shutdown 2024-09-15T01:34:23.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:23.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 1 -- 172.21.15.204:0/1568560114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37cc0a2e30 msgr2=0x7f37cc11e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:23.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 1 --2- 172.21.15.204:0/1568560114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37cc0a2e30 0x7f37cc11e880 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f37c400c890 tx=0x7f37c4001690 comp rx=0 tx=0).stop 2024-09-15T01:34:23.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 1 -- 172.21.15.204:0/1568560114 shutdown_connections 2024-09-15T01:34:23.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 1 --2- 172.21.15.204:0/1568560114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37cc0a2e30 0x7f37cc11e880 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 1 --2- 172.21.15.204:0/1568560114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37cc0a0a40 0x7f37cc11e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 1 --2- 172.21.15.204:0/1568560114 >> v2:172.21.15.204:6812/28210 conn(0x7f37b00397b0 0x7f37b003bc60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 1 --2- 172.21.15.204:0/1568560114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37cc09fb80 0x7f37cc1317b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:23.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 1 -- 172.21.15.204:0/1568560114 >> 172.21.15.204:0/1568560114 conn(0x7f37cc06e860 msgr2=0x7f37cc09ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:23.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 1 -- 172.21.15.204:0/1568560114 shutdown_connections 2024-09-15T01:34:23.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:23.988+0000 7f37d46a8640 1 -- 172.21.15.204:0/1568560114 wait complete. 2024-09-15T01:34:23.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:23.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 10 monclient: _add_conns ranks=[8,10,6,5,1,3,0,2,9,7,4] 2024-09-15T01:34:23.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f34140a2930 0x7f3414132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 10 monclient(hunting): picked mon.noname-e con 0x7f34140a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f341409faa0 0x7f341409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 10 monclient(hunting): picked mon.noname-f con 0x7f341409faa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34140a08c0 0x7f34140a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 10 monclient(hunting): picked mon.noname-d con 0x7f34140a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341409e0f0 con 0x7f341409faa0 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f341409dd80 con 0x7f34140a08c0 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f34140a2d00 con 0x7f34140a2930 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:23.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341bb00640 10 monclient(hunting): authenticate will time out at 868.731665s 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341a09d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34140a08c0 0x7f34140a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341989c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f34140a2930 0x7f3414132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341a09d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34140a08c0 0x7f34140a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42676/0 (socket says 172.21.15.178:42676) 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341989c640 10 monclient(hunting): get_auth_request con 0x7f34140a2930 auth_method 0 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341a09d640 1 -- 172.21.15.178:0/767410086 learned_addr learned my addr 172.21.15.178:0/767410086 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341989c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341989c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341989c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341a09d640 10 monclient(hunting): get_auth_request con 0x7f34140a08c0 auth_method 0 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341a09d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341a09d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341a09d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341909b640 1 --2- 172.21.15.178:0/767410086 >> v2:172.21.15.204:3300/0 conn(0x7f341409faa0 0x7f341409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341909b640 10 monclient(hunting): get_auth_request con 0x7f341409faa0 auth_method 0 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341909b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341909b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341909b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341a09d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341a09d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341a09d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341989c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341989c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341989c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.981+0000 7f341909b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341909b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341909b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341a09d640 10 monclient(hunting): handle_auth_done global_id 14163 payload 274 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341a09d640 1 -- 172.21.15.178:0/767410086 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f34140a2930 msgr2=0x7f3414132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341a09d640 1 --2- 172.21.15.178:0/767410086 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f34140a2930 0x7f3414132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341a09d640 1 -- 172.21.15.178:0/767410086 >> v2:172.21.15.204:3300/0 conn(0x7f341409faa0 msgr2=0x7f341409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341a09d640 1 --2- 172.21.15.178:0/767410086 >> v2:172.21.15.204:3300/0 conn(0x7f341409faa0 0x7f341409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341a09d640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341a09d640 1 monclient: found mon.noname-d 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341a09d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341a09d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341a09d640 1 -- 172.21.15.178:0/767410086 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341404bf60 con 0x7f34140a08c0 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341989c640 1 --2- 172.21.15.178:0/767410086 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f34140a2930 0x7f3414132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341a09d640 1 --2- 172.21.15.178:0/767410086 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34140a08c0 0x7f34140a0d60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f340c009f20 tx=0x7f340c009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7f48664ad747bd7c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341889a640 1 -- 172.21.15.178:0/767410086 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f340c007880 con 0x7f34140a08c0 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341889a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341889a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.982+0000 7f341889a640 10 monclient: dump: 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.003 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.004 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.004 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.004 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.004 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341889a640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341889a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 5 monclient: authenticate success, global_id 14163 2024-09-15T01:34:24.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341889a640 1 -- 172.21.15.178:0/767410086 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c02d050 con 0x7f34140a08c0 2024-09-15T01:34:24.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341889a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 4 set_mon_vals no callback set 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341889a640 1 -- 172.21.15.178:0/767410086 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f340c030360 con 0x7f34140a08c0 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341889a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341889a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341889a640 10 monclient: dump: 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.005 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 10 monclient: shutdown 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 1 -- 172.21.15.178:0/767410086 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34140a08c0 msgr2=0x7f34140a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 1 --2- 172.21.15.178:0/767410086 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34140a08c0 0x7f34140a0d60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f340c009f20 tx=0x7f340c009840 comp rx=0 tx=0).stop 2024-09-15T01:34:24.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 1 -- 172.21.15.178:0/767410086 shutdown_connections 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 1 --2- 172.21.15.178:0/767410086 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34140a08c0 0x7f34140a0d60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 1 --2- 172.21.15.178:0/767410086 >> v2:172.21.15.204:3300/0 conn(0x7f341409faa0 0x7f341409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 1 --2- 172.21.15.178:0/767410086 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f34140a2930 0x7f3414132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 1 -- 172.21.15.178:0/767410086 >> 172.21.15.178:0/767410086 conn(0x7f34140a5ed0 msgr2=0x7f34140a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 1 -- 172.21.15.178:0/767410086 shutdown_connections 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.983+0000 7f341bb00640 1 -- 172.21.15.178:0/767410086 wait complete. 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.984+0000 7f341bb00640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.984+0000 7f341bb00640 10 monclient: monmap: 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:24.007 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.984+0000 7f341bb00640 1 Processor -- start 2024-09-15T01:34:24.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.984+0000 7f341bb00640 1 -- start start 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.984+0000 7f341bb00640 10 monclient: init 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.984+0000 7f341bb00640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.984+0000 7f341bb00640 10 monclient: _add_conns ranks=[0,1,3,4,7,2,5,6,8] 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.984+0000 7f341bb00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341409faa0 0x7f341409aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.984+0000 7f341bb00640 10 monclient(hunting): picked mon.noname-a con 0x7f341409faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.984+0000 7f341bb00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34140a08c0 0x7f3414131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 10 monclient(hunting): picked mon.noname-f con 0x7f34140a08c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34140a2930 0x7f341411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 10 monclient(hunting): picked mon.noname-g con 0x7f34140a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341409d630 con 0x7f341409faa0 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341411ec40 con 0x7f34140a08c0 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3414142b10 con 0x7f34140a2930 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 10 monclient(hunting): authenticate will time out at 868.735066s 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341409faa0 0x7f341409aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341409faa0 0x7f341409aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:58562/0 (socket says 172.21.15.178:58562) 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 1 -- 172.21.15.178:0/3338222772 learned_addr learned my addr 172.21.15.178:0/3338222772 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 10 monclient(hunting): get_auth_request con 0x7f341409faa0 auth_method 0 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341a09d640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34140a2930 0x7f341411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341909b640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34140a08c0 0x7f3414131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341a09d640 10 monclient(hunting): get_auth_request con 0x7f34140a2930 auth_method 0 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341a09d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341a09d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341a09d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341909b640 10 monclient(hunting): get_auth_request con 0x7f34140a08c0 auth_method 0 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341909b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341909b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341909b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341a09d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341a09d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341a09d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341909b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341909b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341909b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 10 monclient(hunting): handle_auth_done global_id 5238 payload 824 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 1 -- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34140a2930 msgr2=0x7f341411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34140a2930 0x7f341411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 1 -- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34140a08c0 msgr2=0x7f3414131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34140a08c0 0x7f3414131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 1 monclient: found mon.noname-a 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:24.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 1 -- 172.21.15.178:0/3338222772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3410009390 con 0x7f341409faa0 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341989c640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341409faa0 0x7f341409aeb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f341000ec80 tx=0x7f34100090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f341000eec0 con 0x7f341409faa0 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3405ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3405ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3405ffb640 10 monclient: dump: 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.013 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3405ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3405ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 5 monclient: authenticate success, global_id 5238 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3410013470 con 0x7f341409faa0 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 10 monclient: _renew_subs 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f341bb00640 1 -- 172.21.15.178:0/3338222772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c009ae0 con 0x7f341409faa0 2024-09-15T01:34:24.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3405ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3410013a60 con 0x7f341409faa0 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3405ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3405ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3405ffb640 10 monclient: dump: 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.015 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.985+0000 7f3407fff640 4 set_mon_vals no callback set 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f341bb00640 10 monclient: _renew_subs 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f341bb00640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f341bb00640 1 -- 172.21.15.178:0/3338222772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341411da70 con 0x7f341409faa0 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3410013bc0 con 0x7f341409faa0 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f341bb00640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f341bb00640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414136ff0 0x7f341411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f341bb00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f341bb00640 1 -- 172.21.15.178:0/3338222772 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f341411eb00 con 0x7f3414136ff0 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f3405ffb640 1 --2- 172.21.15.178:0/3338222772 >> v2:172.21.15.204:6812/28210 conn(0x7f33f80396e0 0x7f33f803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5373+0+0 (secure 0 0 0) 0x7f341004bcf0 con 0x7f341409faa0 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f341909b640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414136ff0 0x7f341411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f341909b640 10 monclient: get_auth_request con 0x7f3414136ff0 auth_method 0 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f341909b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f341909b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.986+0000 7f341909b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f341a09d640 1 --2- 172.21.15.178:0/3338222772 >> v2:172.21.15.204:6812/28210 conn(0x7f33f80396e0 0x7f33f803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f341a09d640 10 monclient: get_auth_request con 0x7f33f80396e0 auth_method 0 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f341909b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f341909b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f341909b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f341909b640 10 monclient(hunting): handle_auth_done global_id 5081 payload 824 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f341909b640 1 -- 172.21.15.178:0/3338222772 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f341404c4a0 con 0x7f3414136ff0 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f341a09d640 1 --2- 172.21.15.178:0/3338222772 >> v2:172.21.15.204:6812/28210 conn(0x7f33f80396e0 0x7f33f803bb90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f340c033000 tx=0x7f340c036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f341909b640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414136ff0 0x7f341411ca70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3408011bf0 tx=0x7f34080090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f340800fc80 con 0x7f3414136ff0 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f3405ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f3405ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f3405ffb640 10 monclient: dump: 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.018 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f340800fde0 con 0x7f3414136ff0 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f3405ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f3405ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414136ff0 msgr2=0x7f341411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f3405ffb640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414136ff0 0x7f341411ca70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3408011bf0 tx=0x7f34080090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414136ff0 msgr2=0x7f341411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.987+0000 7f3405ffb640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414136ff0 0x7f341411ca70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341bb00640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341bb00640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414131d60 0x7f341411cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341bb00640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341bb00640 1 -- 172.21.15.178:0/3338222772 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f341404bc80 con 0x7f3414131d60 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341909b640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414131d60 0x7f341411cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341909b640 10 monclient: get_auth_request con 0x7f3414131d60 auth_method 0 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341909b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341909b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341909b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341909b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341909b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341909b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341909b640 10 monclient(hunting): handle_auth_done global_id 5090 payload 824 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.993+0000 7f341909b640 1 -- 172.21.15.178:0/3338222772 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f341405ed70 con 0x7f3414131d60 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.994+0000 7f341909b640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414131d60 0x7f341411cfb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3408011bf0 tx=0x7f340801c000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.994+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3408024040 con 0x7f3414131d60 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.994+0000 7f3405ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.994+0000 7f3405ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.994+0000 7f3405ffb640 10 monclient: dump: 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.021 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.994+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9313 (secure 0 0 0) 0x7f34080133a0 con 0x7f3414131d60 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.994+0000 7f3405ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.994+0000 7f3405ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.994+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414131d60 msgr2=0x7f341411cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.994+0000 7f3405ffb640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414131d60 0x7f341411cfb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3408011bf0 tx=0x7f340801c000 comp rx=0 tx=0).stop 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.994+0000 7f3405ffb640 1 -- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414131d60 msgr2=0x7f341411cfb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:24.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.994+0000 7f3405ffb640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3414131d60 0x7f341411cfb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.023 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:24.023 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:24.023 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:24.023 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:34:24.023 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:24.023 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:24.023 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:24.023 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:24.023 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:24.023 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:24.023 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:24.024 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:24.025 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:24.026 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:24.027 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.028 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.029 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.030 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.031 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.032 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.033 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:24.034 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.037 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.038 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.039 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.040 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.041 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:24.042 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:24.043 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.044 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:24.045 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:24.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.995+0000 7f341bb00640 1 -- 172.21.15.178:0/3338222772 >> v2:172.21.15.204:6812/28210 conn(0x7f33f80396e0 msgr2=0x7f33f803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.995+0000 7f341bb00640 1 --2- 172.21.15.178:0/3338222772 >> v2:172.21.15.204:6812/28210 conn(0x7f33f80396e0 0x7f33f803bb90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f340c033000 tx=0x7f340c036000 comp rx=0 tx=0).stop 2024-09-15T01:34:24.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.995+0000 7f341bb00640 10 monclient: shutdown 2024-09-15T01:34:24.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.995+0000 7f341bb00640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:24.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.995+0000 7f341bb00640 1 -- 172.21.15.178:0/3338222772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341409faa0 msgr2=0x7f341409aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.995+0000 7f341bb00640 1 --2- 172.21.15.178:0/3338222772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f341409faa0 0x7f341409aeb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f341000ec80 tx=0x7f34100090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.995+0000 7f341a09d640 1 -- 172.21.15.178:0/3338222772 reap_dead start 2024-09-15T01:34:24.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.995+0000 7f341a09d640 1 -- 172.21.15.178:0/3338222772 reap_dead start 2024-09-15T01:34:24.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.995+0000 7f341bb00640 1 -- 172.21.15.178:0/3338222772 shutdown_connections 2024-09-15T01:34:24.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.995+0000 7f341bb00640 1 -- 172.21.15.178:0/3338222772 >> 172.21.15.178:0/3338222772 conn(0x7f34140a5ed0 msgr2=0x7f3414135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:24.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.995+0000 7f341bb00640 1 -- 172.21.15.178:0/3338222772 shutdown_connections 2024-09-15T01:34:24.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:23.995+0000 7f341bb00640 1 -- 172.21.15.178:0/3338222772 wait complete. 2024-09-15T01:34:24.047 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:24.047 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:34:24.047 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:34:24.048 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:34:24.048 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -c host1 2024-09-15T01:34:24.048 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -q 1 2024-09-15T01:34:24.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.068+0000 7f8d0439d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:24.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.068+0000 7f8d0439d640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.069+0000 7f8d0439d640 10 monclient: monmap: 2024-09-15T01:34:24.071 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:24.071 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.071 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:24.070457+0000 2024-09-15T01:34:24.071 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:24.070457+0000 2024-09-15T01:34:24.071 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:24.072 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.069+0000 7f8d0439d640 1 Processor -- start 2024-09-15T01:34:24.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.069+0000 7f8d0439d640 1 -- start start 2024-09-15T01:34:24.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.069+0000 7f8d0439d640 10 monclient: init 2024-09-15T01:34:24.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 10 monclient: _add_conns ranks=[5,8,2,1,7,10,0,4,9,3,6] 2024-09-15T01:34:24.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cfc0a2400 0x7f8cfc0a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 10 monclient(hunting): picked mon.noname-h con 0x7f8cfc0a2400 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cfc09f990 0x7f8cfc09fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 10 monclient(hunting): picked mon.noname-e con 0x7f8cfc09f990 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc132930 0x7f8cfc132d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 10 monclient(hunting): picked mon.noname-b con 0x7f8cfc132930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc09e1a0 con 0x7f8cfc132930 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0a27d0 con 0x7f8cfc0a2400 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc09dec0 con 0x7f8cfc09f990 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0439d640 10 monclient(hunting): authenticate will time out at 879.713369s 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cfc0a2400 0x7f8cfc0a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cfc0a2400 0x7f8cfc0a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:55736/0 (socket says 172.21.15.204:55736) 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 1 -- 172.21.15.204:0/2577288772 learned_addr learned my addr 172.21.15.204:0/2577288772 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 10 monclient(hunting): get_auth_request con 0x7f8cfc0a2400 auth_method 0 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d01938640 1 --2- 172.21.15.204:0/2577288772 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cfc09f990 0x7f8cfc09fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0293a640 1 --2- 172.21.15.204:0/2577288772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc132930 0x7f8cfc132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d01938640 10 monclient(hunting): get_auth_request con 0x7f8cfc09f990 auth_method 0 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d01938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d01938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d01938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0293a640 10 monclient(hunting): get_auth_request con 0x7f8cfc132930 auth_method 0 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0293a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0293a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d0293a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 10 monclient(hunting): handle_auth_done global_id 5099 payload 274 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 1 -- 172.21.15.204:0/2577288772 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cfc09f990 msgr2=0x7f8cfc09fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 1 --2- 172.21.15.204:0/2577288772 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cfc09f990 0x7f8cfc09fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 1 -- 172.21.15.204:0/2577288772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc132930 msgr2=0x7f8cfc132d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 1 --2- 172.21.15.204:0/2577288772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc132930 0x7f8cfc132d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 1 monclient: found mon.noname-h 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.070+0000 7f8d02139640 1 -- 172.21.15.204:0/2577288772 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc0648a0 con 0x7f8cfc0a2400 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d02139640 1 --2- 172.21.15.204:0/2577288772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cfc0a2400 0x7f8cfc0a0ae0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009940 tx=0x7f8cf80091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=ea7283705e6052a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d01137640 1 -- 172.21.15.204:0/2577288772 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8cf802ac60 con 0x7f8cfc0a2400 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d01137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d01137640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d01137640 10 monclient: dump: 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.085 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d01137640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d01137640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d0439d640 5 monclient: authenticate success, global_id 5099 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d0439d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:24.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d01137640 1 -- 172.21.15.204:0/2577288772 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf802ade0 con 0x7f8cfc0a2400 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d01137640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d0439d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d01137640 1 -- 172.21.15.204:0/2577288772 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8cf8031ca0 con 0x7f8cfc0a2400 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d0439d640 4 set_mon_vals no callback set 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d01137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d01137640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.071+0000 7f8d01137640 10 monclient: dump: 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.087 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 10 monclient: shutdown 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 1 -- 172.21.15.204:0/2577288772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cfc0a2400 msgr2=0x7f8cfc0a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 1 --2- 172.21.15.204:0/2577288772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cfc0a2400 0x7f8cfc0a0ae0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009940 tx=0x7f8cf80091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 1 -- 172.21.15.204:0/2577288772 shutdown_connections 2024-09-15T01:34:24.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 1 --2- 172.21.15.204:0/2577288772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc132930 0x7f8cfc132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 1 --2- 172.21.15.204:0/2577288772 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cfc09f990 0x7f8cfc09fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 1 --2- 172.21.15.204:0/2577288772 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cfc0a2400 0x7f8cfc0a0ae0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 1 -- 172.21.15.204:0/2577288772 >> 172.21.15.204:0/2577288772 conn(0x7f8cfc06e860 msgr2=0x7f8cfc06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 1 -- 172.21.15.204:0/2577288772 shutdown_connections 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 1 -- 172.21.15.204:0/2577288772 wait complete. 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.072+0000 7f8d0439d640 10 monclient: monmap: 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:24.089 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 1 Processor -- start 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 1 -- start start 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 10 monclient: init 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 10 monclient: _add_conns ranks=[2,4,1,7,0,5,6,3,8] 2024-09-15T01:34:24.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc09f990 0x7f8cfc141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 10 monclient(hunting): picked mon.noname-b con 0x7f8cfc09f990 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8cfc0a2400 0x7f8cfc142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 10 monclient(hunting): picked mon.noname-c con 0x7f8cfc0a2400 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cfc132930 0x7f8cfc1428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 10 monclient(hunting): picked mon.noname-f con 0x7f8cfc132930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc09d3d0 con 0x7f8cfc132930 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc09cad0 con 0x7f8cfc09f990 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc121e20 con 0x7f8cfc0a2400 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0439d640 10 monclient(hunting): authenticate will time out at 879.716631s 2024-09-15T01:34:24.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0293a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cfc132930 0x7f8cfc1428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d02139640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc09f990 0x7f8cfc141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d02139640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc09f990 0x7f8cfc141e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:58920/0 (socket says 172.21.15.204:58920) 2024-09-15T01:34:24.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d02139640 1 -- 172.21.15.204:0/1433368618 learned_addr learned my addr 172.21.15.204:0/1433368618 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:24.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0293a640 10 monclient(hunting): get_auth_request con 0x7f8cfc132930 auth_method 0 2024-09-15T01:34:24.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0293a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0293a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0293a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d01938640 1 --2- 172.21.15.204:0/1433368618 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8cfc0a2400 0x7f8cfc142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d02139640 10 monclient(hunting): get_auth_request con 0x7f8cfc09f990 auth_method 0 2024-09-15T01:34:24.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d02139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d02139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d02139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d01938640 10 monclient(hunting): get_auth_request con 0x7f8cfc0a2400 auth_method 0 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d01938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d01938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d01938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0293a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0293a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d0293a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d02139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d02139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d02139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d01938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d01938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.073+0000 7f8d01938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0293a640 10 monclient(hunting): handle_auth_done global_id 5599 payload 824 2024-09-15T01:34:24.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0293a640 1 -- 172.21.15.204:0/1433368618 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8cfc0a2400 msgr2=0x7f8cfc142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0293a640 1 --2- 172.21.15.204:0/1433368618 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8cfc0a2400 0x7f8cfc142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0293a640 1 -- 172.21.15.204:0/1433368618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc09f990 msgr2=0x7f8cfc141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0293a640 1 --2- 172.21.15.204:0/1433368618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc09f990 0x7f8cfc141e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0293a640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0293a640 1 monclient: found mon.noname-f 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0293a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0293a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0293a640 1 -- 172.21.15.204:0/1433368618 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf8009450 con 0x7f8cfc132930 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d02139640 1 --2- 172.21.15.204:0/1433368618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc09f990 0x7f8cfc141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0293a640 1 --2- 172.21.15.204:0/1433368618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cfc132930 0x7f8cfc1428b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4007e40 tx=0x7f8cf40097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8cf27fc640 1 -- 172.21.15.204:0/1433368618 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8cf4009840 con 0x7f8cfc132930 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8cf27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8cf27fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8cf27fc640 10 monclient: dump: 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.095 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8cf27fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8cf27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8cf27fc640 1 -- 172.21.15.204:0/1433368618 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf400ed60 con 0x7f8cfc132930 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0439d640 5 monclient: authenticate success, global_id 5599 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8cf27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0439d640 10 monclient: _renew_subs 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8cf27fc640 1 -- 172.21.15.204:0/1433368618 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8cf4014d20 con 0x7f8cfc132930 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0439d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0439d640 1 -- 172.21.15.204:0/1433368618 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc11f090 con 0x7f8cfc132930 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8cf27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d00936640 4 set_mon_vals no callback set 2024-09-15T01:34:24.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8cf27fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8cf27fc640 10 monclient: dump: 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.097 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0439d640 10 monclient: _renew_subs 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0439d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.074+0000 7f8d0439d640 1 -- 172.21.15.204:0/1433368618 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc120b50 con 0x7f8cfc132930 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.075+0000 7f8cf27fc640 1 -- 172.21.15.204:0/1433368618 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8cf4003020 con 0x7f8cfc132930 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.075+0000 7f8d0439d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.075+0000 7f8d0439d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.075+0000 7f8d0439d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.075+0000 7f8d0439d640 1 -- 172.21.15.204:0/1433368618 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc070b80 con 0x7f8cfc132930 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.075+0000 7f8cf27fc640 1 --2- 172.21.15.204:0/1433368618 >> v2:172.21.15.204:6812/28210 conn(0x7f8ce00397b0 0x7f8ce003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.075+0000 7f8cf27fc640 1 -- 172.21.15.204:0/1433368618 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5373+0+0 (secure 0 0 0) 0x7f8cf404ba10 con 0x7f8cfc132930 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.075+0000 7f8d02139640 1 --2- 172.21.15.204:0/1433368618 >> v2:172.21.15.204:6812/28210 conn(0x7f8ce00397b0 0x7f8ce003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.075+0000 7f8d02139640 10 monclient: get_auth_request con 0x7f8ce00397b0 auth_method 0 2024-09-15T01:34:24.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.076+0000 7f8d02139640 1 --2- 172.21.15.204:0/1433368618 >> v2:172.21.15.204:6812/28210 conn(0x7f8ce00397b0 0x7f8ce003bc60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009940 tx=0x7f8cf8032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.080+0000 7f8cf27fc640 1 -- 172.21.15.204:0/1433368618 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8cf401b960 con 0x7f8cfc132930 2024-09-15T01:34:24.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.080+0000 7f8cf27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.080+0000 7f8cf27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:24.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.139+0000 7f96144d4640 10 monclient: get_monmap_and_config 2024-09-15T01:34:24.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.139+0000 7f96144d4640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.139+0000 7f96144d4640 10 monclient: monmap: 2024-09-15T01:34:24.141 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:24.141 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.141 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:24.140467+0000 2024-09-15T01:34:24.141 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:24.140467+0000 2024-09-15T01:34:24.141 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:24.142 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 1 Processor -- start 2024-09-15T01:34:24.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 1 -- start start 2024-09-15T01:34:24.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 10 monclient: init 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 10 monclient: _add_conns ranks=[4,6,8,10,5,1,2,0,9,3,7] 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960c0a2930 0x7f960c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 10 monclient(hunting): picked mon.noname-c con 0x7f960c0a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c09faa0 0x7f960c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 10 monclient(hunting): picked mon.noname-d con 0x7f960c09faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960c0a08c0 0x7f960c0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 10 monclient(hunting): picked mon.noname-e con 0x7f960c0a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f960c09e0f0 con 0x7f960c0a2930 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f960c09dd80 con 0x7f960c09faa0 2024-09-15T01:34:24.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f960c0a2d00 con 0x7f960c0a08c0 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.140+0000 7f96144d4640 10 monclient(hunting): authenticate will time out at 868.890934s 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c09faa0 0x7f960c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c09faa0 0x7f960c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42690/0 (socket says 172.21.15.178:42690) 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 1 -- 172.21.15.178:0/1874222871 learned_addr learned my addr 172.21.15.178:0/1874222871 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 10 monclient(hunting): get_auth_request con 0x7f960c09faa0 auth_method 0 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612270640 1 --2- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960c0a2930 0x7f960c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612a71640 1 --2- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960c0a08c0 0x7f960c0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612270640 10 monclient(hunting): get_auth_request con 0x7f960c0a2930 auth_method 0 2024-09-15T01:34:24.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612270640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612270640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612270640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612a71640 10 monclient(hunting): get_auth_request con 0x7f960c0a08c0 auth_method 0 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612a71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612a71640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612a71640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612270640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612270640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612270640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612a71640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612a71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612a71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 10 monclient(hunting): handle_auth_done global_id 14172 payload 274 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 1 -- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960c0a08c0 msgr2=0x7f960c0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 1 --2- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960c0a08c0 0x7f960c0a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 1 -- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960c0a2930 msgr2=0x7f960c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 1 --2- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960c0a2930 0x7f960c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 1 monclient: found mon.noname-d 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9611a6f640 1 -- 172.21.15.178:0/1874222871 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c04bf60 con 0x7f960c09faa0 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.141+0000 7f9612270640 1 --2- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960c0a2930 0x7f960c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f9611a6f640 1 --2- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c09faa0 0x7f960c09fe70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009940 tx=0x7f95fc0091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=40e0250773be922a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f961126e640 1 -- 172.21.15.178:0/1874222871 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95fc02ac60 con 0x7f960c09faa0 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f961126e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f961126e640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f961126e640 10 monclient: dump: 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.156 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f961126e640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f961126e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f961126e640 1 -- 172.21.15.178:0/1874222871 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc02ade0 con 0x7f960c09faa0 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 5 monclient: authenticate success, global_id 14172 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f961126e640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f961126e640 1 -- 172.21.15.178:0/1874222871 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95fc031ca0 con 0x7f960c09faa0 2024-09-15T01:34:24.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 4 set_mon_vals no callback set 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f961126e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f961126e640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f961126e640 10 monclient: dump: 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 10 monclient: shutdown 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 1 -- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c09faa0 msgr2=0x7f960c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 1 --2- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c09faa0 0x7f960c09fe70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009940 tx=0x7f95fc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 1 -- 172.21.15.178:0/1874222871 shutdown_connections 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 1 --2- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960c0a08c0 0x7f960c0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 1 --2- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f960c09faa0 0x7f960c09fe70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 1 --2- 172.21.15.178:0/1874222871 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f960c0a2930 0x7f960c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 1 -- 172.21.15.178:0/1874222871 >> 172.21.15.178:0/1874222871 conn(0x7f960c0a5ed0 msgr2=0x7f960c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 1 -- 172.21.15.178:0/1874222871 shutdown_connections 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.142+0000 7f96144d4640 1 -- 172.21.15.178:0/1874222871 wait complete. 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.143+0000 7f96144d4640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.143+0000 7f96144d4640 10 monclient: monmap: 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.160 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.143+0000 7f96144d4640 1 Processor -- start 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.143+0000 7f96144d4640 1 -- start start 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.143+0000 7f96144d4640 10 monclient: init 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient: _add_conns ranks=[5,8,7,1,6,2,4,0,3] 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f960c09faa0 0x7f960c11c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient(hunting): picked mon.noname-h con 0x7f960c09faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960c0a08c0 0x7f960c09b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient(hunting): picked mon.noname-e con 0x7f960c0a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c0a2930 0x7f960c131950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient(hunting): picked mon.noname-i con 0x7f960c0a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f960c11f030 con 0x7f960c09faa0 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f960c09d630 con 0x7f960c0a2930 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f960c11db80 con 0x7f960c0a08c0 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient(hunting): authenticate will time out at 868.894113s 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960c0a08c0 0x7f960c09b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960c0a08c0 0x7f960c09b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54270/0 (socket says 172.21.15.178:54270) 2024-09-15T01:34:24.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 1 -- 172.21.15.178:0/1565044612 learned_addr learned my addr 172.21.15.178:0/1565044612 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 10 monclient(hunting): get_auth_request con 0x7f960c0a08c0 auth_method 0 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9612a71640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c0a2930 0x7f960c131950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9612270640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f960c09faa0 0x7f960c11c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9612a71640 10 monclient(hunting): get_auth_request con 0x7f960c0a2930 auth_method 0 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9612a71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9612a71640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9612a71640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9612270640 10 monclient(hunting): get_auth_request con 0x7f960c09faa0 auth_method 0 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9612270640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9612270640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9612270640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 10 monclient(hunting): handle_auth_done global_id 5192 payload 824 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 1 -- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c0a2930 msgr2=0x7f960c131950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c0a2930 0x7f960c131950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 1 -- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f960c09faa0 msgr2=0x7f960c11c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f960c09faa0 0x7f960c11c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 1 monclient: found mon.noname-e 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:24.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 1 -- 172.21.15.178:0/1565044612 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9608009390 con 0x7f960c0a08c0 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9612a71640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c0a2930 0x7f960c131950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9612270640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f960c09faa0 0x7f960c11c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9611a6f640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960c0a08c0 0x7f960c09b350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95fc02ac60 con 0x7f960c0a08c0 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96027fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96027fc640 10 monclient: dump: 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.165 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96027fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc031820 con 0x7f960c0a08c0 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 5 monclient: authenticate success, global_id 5192 2024-09-15T01:34:24.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96027fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95fc031e50 con 0x7f960c0a08c0 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient: _renew_subs 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 1 -- 172.21.15.178:0/1565044612 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc009450 con 0x7f960c0a08c0 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f9610a6d640 4 set_mon_vals no callback set 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96027fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96027fc640 10 monclient: dump: 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.167 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient: _renew_subs 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:24.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.144+0000 7f96144d4640 1 -- 172.21.15.178:0/1565044612 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c11ed50 con 0x7f960c0a08c0 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f96144d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f96144d4640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c131e90 0x7f960c142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f96144d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f96144d4640 1 -- 172.21.15.178:0/1565044612 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f960c04c4a0 con 0x7f960c131e90 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f95fc039040 con 0x7f960c0a08c0 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f9612270640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c131e90 0x7f960c142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f9612270640 10 monclient: get_auth_request con 0x7f960c131e90 auth_method 0 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f9612270640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f9612270640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f9612270640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f9612270640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f9612270640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.145+0000 7f9612270640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 1 --2- 172.21.15.178:0/1565044612 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00396e0 0x7f95f003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5373+0+0 (secure 0 0 0) 0x7f95fc06a9e0 con 0x7f960c0a08c0 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f9612270640 10 monclient(hunting): handle_auth_done global_id 5119 payload 824 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f9612a71640 1 --2- 172.21.15.178:0/1565044612 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00396e0 0x7f95f003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f9612270640 1 -- 172.21.15.178:0/1565044612 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f960c05ed70 con 0x7f960c131e90 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f9612a71640 10 monclient: get_auth_request con 0x7f95f00396e0 auth_method 0 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f9612270640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c131e90 0x7f960c142090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9608011990 tx=0x7f9608008040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9608015460 con 0x7f960c131e90 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 10 monclient: dump: 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.170 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.171 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f9612a71640 1 --2- 172.21.15.178:0/1565044612 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00396e0 0x7f95f003bb90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f960c0a3290 tx=0x7f960400b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f960800fb50 con 0x7f960c131e90 2024-09-15T01:34:24.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:24.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c131e90 msgr2=0x7f960c142090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c131e90 0x7f960c142090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9608011990 tx=0x7f9608008040 comp rx=0 tx=0).stop 2024-09-15T01:34:24.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c131e90 msgr2=0x7f960c142090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:24.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.146+0000 7f96027fc640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c131e90 0x7f960c142090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f96144d4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f96144d4640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c1425d0 0x7f960c1429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f96144d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f96144d4640 1 -- 172.21.15.178:0/1565044612 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f960c11f1a0 con 0x7f960c1425d0 2024-09-15T01:34:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f9612270640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c1425d0 0x7f960c1429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f9612270640 10 monclient: get_auth_request con 0x7f960c1425d0 auth_method 0 2024-09-15T01:34:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f9612270640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f9612270640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f9612270640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f9612270640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f9612270640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f9612270640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f9612270640 10 monclient(hunting): handle_auth_done global_id 5128 payload 824 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.152+0000 7f9612270640 1 -- 172.21.15.178:0/1565044612 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f960c04bc80 con 0x7f960c1425d0 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f9612270640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c1425d0 0x7f960c1429d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9608011990 tx=0x7f960801d000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9608015a40 con 0x7f960c1425d0 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96027fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96027fc640 10 monclient: dump: 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9481 (secure 0 0 0) 0x7f9608015ba0 con 0x7f960c1425d0 2024-09-15T01:34:24.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96027fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:24.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96027fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:24.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c1425d0 msgr2=0x7f960c1429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96027fc640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c1425d0 0x7f960c1429d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9608011990 tx=0x7f960801d000 comp rx=0 tx=0).stop 2024-09-15T01:34:24.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96027fc640 1 -- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c1425d0 msgr2=0x7f960c1429d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:24.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96027fc640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f960c1425d0 0x7f960c1429d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.177 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:24.177 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:24.177 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:24.177 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:24.177 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:24.178 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:24.179 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:24.180 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:24.181 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.182 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:24.183 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.184 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.185 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:24.186 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.187 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.188 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.189 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.191 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.192 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.193 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:24.194 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:24.196 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:24.197 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:24.198 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:24.199 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:24.200 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:24.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:24.200 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:24.200 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:24.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96144d4640 1 -- 172.21.15.178:0/1565044612 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00396e0 msgr2=0x7f95f003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96144d4640 1 --2- 172.21.15.178:0/1565044612 >> v2:172.21.15.204:6812/28210 conn(0x7f95f00396e0 0x7f95f003bb90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f960c0a3290 tx=0x7f960400b040 comp rx=0 tx=0).stop 2024-09-15T01:34:24.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96144d4640 10 monclient: shutdown 2024-09-15T01:34:24.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96144d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:24.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96144d4640 1 -- 172.21.15.178:0/1565044612 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960c0a08c0 msgr2=0x7f960c09b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96144d4640 1 --2- 172.21.15.178:0/1565044612 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f960c0a08c0 0x7f960c09b350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc032000 comp rx=0 tx=0).stop 2024-09-15T01:34:24.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f9612a71640 1 -- 172.21.15.178:0/1565044612 reap_dead start 2024-09-15T01:34:24.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f9612a71640 1 -- 172.21.15.178:0/1565044612 reap_dead start 2024-09-15T01:34:24.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96144d4640 1 -- 172.21.15.178:0/1565044612 shutdown_connections 2024-09-15T01:34:24.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96144d4640 1 -- 172.21.15.178:0/1565044612 >> 172.21.15.178:0/1565044612 conn(0x7f960c0a5ed0 msgr2=0x7f960c09e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:24.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.153+0000 7f96144d4640 1 -- 172.21.15.178:0/1565044612 shutdown_connections 2024-09-15T01:34:24.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.154+0000 7f96144d4640 1 -- 172.21.15.178:0/1565044612 wait complete. 2024-09-15T01:34:24.202 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:24.202 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T01:34:24.203 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T01:34:24.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.239+0000 7f8d0439d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:24.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.239+0000 7f8d0439d640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:24.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.239+0000 7f8d0439d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:24.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.239+0000 7f8d0439d640 1 -- 172.21.15.204:0/1433368618 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f8cfc09e480 con 0x7f8cfc132930 2024-09-15T01:34:24.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.240+0000 7f8cf27fc640 1 -- 172.21.15.204:0/1433368618 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v78) v1 ==== 56+0+1340 (secure 0 0 0) 0x7f8cf4005cf0 con 0x7f8cfc132930 2024-09-15T01:34:24.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.240+0000 7f8cf27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:24.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.240+0000 7f8cf27fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:24.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.293+0000 7ff13ae6f640 10 monclient: get_monmap_and_config 2024-09-15T01:34:24.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.293+0000 7ff13ae6f640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.293+0000 7ff13ae6f640 10 monclient: monmap: 2024-09-15T01:34:24.295 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:24.295 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.295 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:24.294485+0000 2024-09-15T01:34:24.295 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:24.294485+0000 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:24.296 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:24.297 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 1 Processor -- start 2024-09-15T01:34:24.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 1 -- start start 2024-09-15T01:34:24.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 10 monclient: init 2024-09-15T01:34:24.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 10 monclient: _add_conns ranks=[8,2,0,7,5,9,1,3,10,4,6] 2024-09-15T01:34:24.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1340a2930 0x7ff134132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 10 monclient(hunting): picked mon.noname-e con 0x7ff1340a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:24.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff13409faa0 0x7ff13409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 10 monclient(hunting): picked mon.noname-b con 0x7ff13409faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:24.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff1340a08c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:24.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13404bf60 con 0x7ff1340a08c0 2024-09-15T01:34:24.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13409e0f0 con 0x7ff13409faa0 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff13409dd80 con 0x7ff1340a2930 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.294+0000 7ff13ae6f640 10 monclient(hunting): authenticate will time out at 869.044950s 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff13409faa0 0x7ff13409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1340a2930 0x7ff134132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff13409faa0 0x7ff13409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42942/0 (socket says 172.21.15.178:42942) 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff13940c640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff1340a08c0 0x7ff1340a0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:34392/0 (socket says 172.21.15.178:34392) 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1340a2930 0x7ff134132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54272/0 (socket says 172.21.15.178:54272) 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12bfff640 1 -- 172.21.15.178:0/3996108182 learned_addr learned my addr 172.21.15.178:0/3996108182 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12bfff640 10 monclient(hunting): get_auth_request con 0x7ff13409faa0 auth_method 0 2024-09-15T01:34:24.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 10 monclient(hunting): get_auth_request con 0x7ff1340a2930 auth_method 0 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 10 monclient(hunting): handle_auth_done global_id 5201 payload 274 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 1 -- 172.21.15.178:0/3996108182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff13409faa0 msgr2=0x7ff13409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:24.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 1 --2- 172.21.15.178:0/3996108182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff13409faa0 0x7ff13409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 1 -- 172.21.15.178:0/3996108182 >> v1:172.21.15.178:6789/0 conn(0x7ff1340a08c0 legacy=0x7ff1340a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12b7fe640 1 -- 172.21.15.178:0/3996108182 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787932882 0 0) 0x7ff13404bf60 con 0x7ff1340a08c0 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 1 monclient: found mon.noname-e 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff138c0b640 1 -- 172.21.15.178:0/3996108182 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1340a2d00 con 0x7ff1340a2930 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12bfff640 1 --2- 172.21.15.178:0/3996108182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff13409faa0 0x7ff13409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.295+0000 7ff12b7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff138c0b640 1 --2- 172.21.15.178:0/3996108182 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1340a2930 0x7ff134132de0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff130009880 tx=0x7ff1300090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=67d4a32bd2f81fd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff12b7fe640 1 -- 172.21.15.178:0/3996108182 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff13002ac50 con 0x7ff1340a2930 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff12b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff12b7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff12b7fe640 10 monclient: dump: 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff12b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff12b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff12b7fe640 1 -- 172.21.15.178:0/3996108182 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13002add0 con 0x7ff1340a2930 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 5 monclient: authenticate success, global_id 5201 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff12b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff12b7fe640 1 -- 172.21.15.178:0/3996108182 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff130031ca0 con 0x7ff1340a2930 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 4 set_mon_vals no callback set 2024-09-15T01:34:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff12b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff12b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff12b7fe640 10 monclient: dump: 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.311 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 10 monclient: shutdown 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3996108182 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1340a2930 msgr2=0x7ff134132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 1 --2- 172.21.15.178:0/3996108182 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1340a2930 0x7ff134132de0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff130009880 tx=0x7ff1300090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3996108182 shutdown_connections 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 1 --2- 172.21.15.178:0/3996108182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff13409faa0 0x7ff13409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 1 --2- 172.21.15.178:0/3996108182 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1340a2930 0x7ff134132de0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3996108182 >> 172.21.15.178:0/3996108182 conn(0x7ff1340a5ed0 msgr2=0x7ff1340a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:24.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3996108182 shutdown_connections 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.296+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3996108182 wait complete. 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.297+0000 7ff13ae6f640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.297+0000 7ff13ae6f640 10 monclient: monmap: 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.313 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.297+0000 7ff13ae6f640 1 Processor -- start 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.297+0000 7ff13ae6f640 1 -- start start 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.297+0000 7ff13ae6f640 10 monclient: init 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 10 monclient: _add_conns ranks=[8,2,7,6,3,4,5,1,0] 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff13409faa0 0x7ff13409ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 10 monclient(hunting): picked mon.noname-e con 0x7ff13409faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:24.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1340a08c0 0x7ff1341316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 10 monclient(hunting): picked mon.noname-b con 0x7ff1340a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff1340a2930 0x7ff13411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 10 monclient(hunting): picked mon.noname-i con 0x7ff1340a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340a2390 con 0x7ff1340a08c0 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff13409d630 con 0x7ff1340a2930 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff13409bdc0 con 0x7ff13409faa0 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13ae6f640 10 monclient(hunting): authenticate will time out at 869.048377s 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff138c0b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff13409faa0 0x7ff13409ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1340a08c0 0x7ff1341316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff138c0b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff13409faa0 0x7ff13409ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54274/0 (socket says 172.21.15.178:54274) 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1340a08c0 0x7ff1341316e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42954/0 (socket says 172.21.15.178:42954) 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff138c0b640 1 -- 172.21.15.178:0/3854594145 learned_addr learned my addr 172.21.15.178:0/3854594145 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff138c0b640 10 monclient(hunting): get_auth_request con 0x7ff13409faa0 auth_method 0 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff138c0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff138c0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff138c0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 10 monclient(hunting): get_auth_request con 0x7ff1340a08c0 auth_method 0 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff138c0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff138c0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff138c0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13940c640 1 --2- 172.21.15.178:0/3854594145 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff1340a2930 0x7ff13411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13940c640 10 monclient(hunting): get_auth_request con 0x7ff1340a2930 auth_method 0 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13940c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13940c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13940c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13940c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13940c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff13940c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 10 monclient(hunting): handle_auth_done global_id 5069 payload 824 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 1 -- 172.21.15.178:0/3854594145 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff13409faa0 msgr2=0x7ff13409ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 1 --2- 172.21.15.178:0/3854594145 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff13409faa0 0x7ff13409ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 1 -- 172.21.15.178:0/3854594145 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff1340a2930 msgr2=0x7ff13411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 1 --2- 172.21.15.178:0/3854594145 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff1340a2930 0x7ff13411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 1 monclient: found mon.noname-b 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 1 -- 172.21.15.178:0/3854594145 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff124009450 con 0x7ff1340a08c0 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff138c0b640 1 --2- 172.21.15.178:0/3854594145 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff13409faa0 0x7ff13409ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.298+0000 7ff12bfff640 1 --2- 172.21.15.178:0/3854594145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1340a08c0 0x7ff1341316e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff12400e950 tx=0x7ff1240091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 1 -- 172.21.15.178:0/3854594145 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1240095e0 con 0x7ff1340a08c0 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 10 monclient: dump: 2024-09-15T01:34:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 1 -- 172.21.15.178:0/3854594145 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12400cdc0 con 0x7ff1340a08c0 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 1 -- 172.21.15.178:0/3854594145 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1240138f0 con 0x7ff1340a08c0 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff13ae6f640 5 monclient: authenticate success, global_id 5069 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff13ae6f640 10 monclient: _renew_subs 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff13ae6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3854594145 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff130009390 con 0x7ff1340a08c0 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:34:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff12affd640 4 set_mon_vals no callback set 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 10 monclient: dump: 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff13ae6f640 10 monclient: _renew_subs 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff13ae6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3854594145 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13411e800 con 0x7ff1340a08c0 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.299+0000 7ff128ff9640 1 -- 172.21.15.178:0/3854594145 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff124013470 con 0x7ff1340a08c0 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.300+0000 7ff13ae6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.300+0000 7ff13ae6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.300+0000 7ff13ae6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.300+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3854594145 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13404b720 con 0x7ff1340a08c0 2024-09-15T01:34:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.300+0000 7ff128ff9640 1 --2- 172.21.15.178:0/3854594145 >> v2:172.21.15.204:6812/28210 conn(0x7ff1200397b0 0x7ff12003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.300+0000 7ff128ff9640 1 -- 172.21.15.178:0/3854594145 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5373+0+0 (secure 0 0 0) 0x7ff12404b860 con 0x7ff1340a08c0 2024-09-15T01:34:24.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.300+0000 7ff138c0b640 1 --2- 172.21.15.178:0/3854594145 >> v2:172.21.15.204:6812/28210 conn(0x7ff1200397b0 0x7ff12003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.300+0000 7ff138c0b640 10 monclient: get_auth_request con 0x7ff1200397b0 auth_method 0 2024-09-15T01:34:24.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.300+0000 7ff138c0b640 1 --2- 172.21.15.178:0/3854594145 >> v2:172.21.15.204:6812/28210 conn(0x7ff1200397b0 0x7ff12003bc60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff130000c00 tx=0x7ff130032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.304+0000 7ff128ff9640 1 -- 172.21.15.178:0/3854594145 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff124013720 con 0x7ff1340a08c0 2024-09-15T01:34:24.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.304+0000 7ff128ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.304+0000 7ff128ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:24.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.334+0000 7f8d0439d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:24.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.334+0000 7f8d0439d640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:24.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.334+0000 7f8d0439d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:24.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.334+0000 7f8d0439d640 1 -- 172.21.15.204:0/1433368618 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f8cfc11fc60 con 0x7f8cfc132930 2024-09-15T01:34:24.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.334+0000 7f8cf27fc640 1 -- 172.21.15.204:0/1433368618 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v78) v1 ==== 56+0+1340 (secure 0 0 0) 0x7f8cf401b780 con 0x7f8cfc132930 2024-09-15T01:34:24.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.334+0000 7f8cf27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:24.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.334+0000 7f8cf27fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:24.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 1 -- 172.21.15.204:0/1433368618 >> v2:172.21.15.204:6812/28210 conn(0x7f8ce00397b0 msgr2=0x7f8ce003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 1 --2- 172.21.15.204:0/1433368618 >> v2:172.21.15.204:6812/28210 conn(0x7f8ce00397b0 0x7f8ce003bc60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009940 tx=0x7f8cf8032000 comp rx=0 tx=0).stop 2024-09-15T01:34:24.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 10 monclient: shutdown 2024-09-15T01:34:24.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:24.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 1 -- 172.21.15.204:0/1433368618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cfc132930 msgr2=0x7f8cfc1428b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 1 --2- 172.21.15.204:0/1433368618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cfc132930 0x7f8cfc1428b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4007e40 tx=0x7f8cf40097b0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 1 -- 172.21.15.204:0/1433368618 shutdown_connections 2024-09-15T01:34:24.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 1 --2- 172.21.15.204:0/1433368618 >> v2:172.21.15.204:6812/28210 conn(0x7f8ce00397b0 0x7f8ce003bc60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 1 --2- 172.21.15.204:0/1433368618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cfc132930 0x7f8cfc1428b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 1 --2- 172.21.15.204:0/1433368618 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8cfc0a2400 0x7f8cfc142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 1 --2- 172.21.15.204:0/1433368618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cfc09f990 0x7f8cfc141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 1 -- 172.21.15.204:0/1433368618 >> 172.21.15.204:0/1433368618 conn(0x7f8cfc06e860 msgr2=0x7f8cfc09fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:24.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 1 -- 172.21.15.204:0/1433368618 shutdown_connections 2024-09-15T01:34:24.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.336+0000 7f8d0439d640 1 -- 172.21.15.204:0/1433368618 wait complete. 2024-09-15T01:34:24.348 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush rm foo 2024-09-15T01:34:24.348 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:34:24.348 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm foo 2024-09-15T01:34:24.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.418+0000 7fa969cb8640 10 monclient: get_monmap_and_config 2024-09-15T01:34:24.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.418+0000 7fa969cb8640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.421 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.418+0000 7fa969cb8640 10 monclient: monmap: 2024-09-15T01:34:24.421 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:24.421 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:24.420340+0000 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:24.420340+0000 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.422 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:24.423 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:24.423 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.419+0000 7fa969cb8640 1 Processor -- start 2024-09-15T01:34:24.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.419+0000 7fa969cb8640 1 -- start start 2024-09-15T01:34:24.430 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.419+0000 7fa969cb8640 10 monclient: init 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 10 monclient: _add_conns ranks=[4,7,1,8,9,3,5,6,2,10,0] 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa96409acd0 0x7fa96409b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 10 monclient(hunting): picked mon.noname-c con 0x7fa96409acd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa96409b9f0 0x7fa9640969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 10 monclient(hunting): picked mon.noname-i con 0x7fa96409b9f0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa964096f20 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96404b720 con 0x7fa964096f20 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa96409cee0 con 0x7fa96409acd0 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa9640994c0 con 0x7fa96409b9f0 2024-09-15T01:34:24.431 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa969cb8640 10 monclient(hunting): authenticate will time out at 880.063238s 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa96409b9f0 0x7fa9640969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa963fff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa964096f20 0x7fa9641366c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:60702/0 (socket says 172.21.15.204:60702) 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa963fff640 1 -- 172.21.15.204:0/2042177566 learned_addr learned my addr 172.21.15.204:0/2042177566 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 10 monclient(hunting): get_auth_request con 0x7fa96409b9f0 auth_method 0 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9637fe640 1 --2- 172.21.15.204:0/2042177566 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa96409acd0 0x7fa96409b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9637fe640 10 monclient(hunting): get_auth_request con 0x7fa96409acd0 auth_method 0 2024-09-15T01:34:24.432 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9627fc640 1 -- 172.21.15.204:0/2042177566 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324435201 0 0) 0x7fa96404b720 con 0x7fa964096f20 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9627fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9627fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9627fc640 10 monclient(hunting): my global_id is 5608 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa9627fc640 1 -- 172.21.15.204:0/2042177566 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9500034e0 con 0x7fa964096f20 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 10 monclient(hunting): handle_auth_done global_id 5146 payload 274 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 1 -- 172.21.15.204:0/2042177566 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa96409acd0 msgr2=0x7fa96409b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:24.433 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 1 --2- 172.21.15.204:0/2042177566 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa96409acd0 0x7fa96409b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 1 -- 172.21.15.204:0/2042177566 >> v1:172.21.15.204:6789/0 conn(0x7fa964096f20 legacy=0x7fa9641366c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 1 monclient: found mon.noname-i 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.420+0000 7fa962ffd640 1 -- 172.21.15.204:0/2042177566 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96409bdc0 con 0x7fa96409b9f0 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa962ffd640 1 --2- 172.21.15.204:0/2042177566 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa96409b9f0 0x7fa9640969e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0098b0 tx=0x7fa94c0090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=f117d22661703c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa9627fc640 1 -- 172.21.15.204:0/2042177566 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa94c02ac40 con 0x7fa96409b9f0 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa9627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa9627fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa9627fc640 10 monclient: dump: 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.434 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.435 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.435 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.435 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.435 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.435 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.435 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.435 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.435 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa9627fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa9627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa969cb8640 5 monclient: authenticate success, global_id 5146 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa969cb8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa9627fc640 1 -- 172.21.15.204:0/2042177566 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c02adc0 con 0x7fa96409b9f0 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa9627fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa969cb8640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:24.436 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa969cb8640 4 set_mon_vals no callback set 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa9627fc640 1 -- 172.21.15.204:0/2042177566 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa94c031ca0 con 0x7fa96409b9f0 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa9627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa9627fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.421+0000 7fa9627fc640 10 monclient: dump: 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.437 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.422+0000 7fa969cb8640 10 monclient: shutdown 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.422+0000 7fa969cb8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.422+0000 7fa969cb8640 1 -- 172.21.15.204:0/2042177566 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa96409b9f0 msgr2=0x7fa9640969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.422+0000 7fa969cb8640 1 --2- 172.21.15.204:0/2042177566 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa96409b9f0 0x7fa9640969e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0098b0 tx=0x7fa94c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.422+0000 7fa969cb8640 1 -- 172.21.15.204:0/2042177566 shutdown_connections 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.422+0000 7fa969cb8640 1 --2- 172.21.15.204:0/2042177566 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa96409b9f0 0x7fa9640969e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.422+0000 7fa969cb8640 1 --2- 172.21.15.204:0/2042177566 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa96409acd0 0x7fa96409b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.438 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.422+0000 7fa969cb8640 1 -- 172.21.15.204:0/2042177566 >> 172.21.15.204:0/2042177566 conn(0x7fa9640a2a80 msgr2=0x7fa96409e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.422+0000 7fa969cb8640 1 -- 172.21.15.204:0/2042177566 shutdown_connections 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.422+0000 7fa969cb8640 1 -- 172.21.15.204:0/2042177566 wait complete. 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.422+0000 7fa969cb8640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.422+0000 7fa969cb8640 10 monclient: monmap: 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.439 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 1 Processor -- start 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 1 -- start start 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 10 monclient: init 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 10 monclient: _add_conns ranks=[7,5,4,0,2,8,1,6,3] 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa964096f20 0x7fa9641308d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 10 monclient(hunting): picked mon.noname-i con 0x7fa964096f20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa96409acd0 0x7fa96411c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 10 monclient(hunting): picked mon.noname-h con 0x7fa96409acd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa96409b9f0 0x7fa96411cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 10 monclient(hunting): picked mon.noname-c con 0x7fa96409b9f0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa96406ec40 con 0x7fa96409b9f0 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa964098760 con 0x7fa96409acd0 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa96411df50 con 0x7fa964096f20 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa969cb8640 10 monclient(hunting): authenticate will time out at 880.066718s 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa9637fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa964096f20 0x7fa9641308d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa9637fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa964096f20 0x7fa9641308d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:34104/0 (socket says 172.21.15.204:34104) 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa9637fe640 1 -- 172.21.15.204:0/3035591009 learned_addr learned my addr 172.21.15.204:0/3035591009 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:24.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa9637fe640 10 monclient(hunting): get_auth_request con 0x7fa964096f20 auth_method 0 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa9637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa9637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa9637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa962ffd640 1 --2- 172.21.15.204:0/3035591009 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa96409acd0 0x7fa96411c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa962ffd640 10 monclient(hunting): get_auth_request con 0x7fa96409acd0 auth_method 0 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa962ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa962ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa962ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa9637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa9637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa9637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa962ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa962ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa963fff640 1 --2- 172.21.15.204:0/3035591009 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa96409b9f0 0x7fa96411cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa962ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa963fff640 10 monclient(hunting): get_auth_request con 0x7fa96409b9f0 auth_method 0 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa963fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa963fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.423+0000 7fa963fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa9637fe640 10 monclient(hunting): handle_auth_done global_id 5155 payload 824 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa9637fe640 1 -- 172.21.15.204:0/3035591009 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa96409acd0 msgr2=0x7fa96411c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa9637fe640 1 --2- 172.21.15.204:0/3035591009 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa96409acd0 0x7fa96411c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa9637fe640 1 -- 172.21.15.204:0/3035591009 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa96409b9f0 msgr2=0x7fa96411cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa9637fe640 1 --2- 172.21.15.204:0/3035591009 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa96409b9f0 0x7fa96411cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa9637fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa9637fe640 1 monclient: found mon.noname-i 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa9637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa9637fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa9637fe640 1 -- 172.21.15.204:0/3035591009 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c009390 con 0x7fa964096f20 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa963fff640 1 --2- 172.21.15.204:0/3035591009 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa96409b9f0 0x7fa96411cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa9637fe640 1 --2- 172.21.15.204:0/3035591009 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa964096f20 0x7fa9641308d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa95400ea80 tx=0x7fa9540090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa95b7fe640 1 -- 172.21.15.204:0/3035591009 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa95400ce90 con 0x7fa964096f20 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa95b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa95b7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa95b7fe640 10 monclient: dump: 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.444 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa95b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa95b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa969cb8640 5 monclient: authenticate success, global_id 5155 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa95b7fe640 1 -- 172.21.15.204:0/3035591009 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa954013470 con 0x7fa964096f20 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa969cb8640 10 monclient: _renew_subs 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa969cb8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa969cb8640 1 -- 172.21.15.204:0/3035591009 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96411fa10 con 0x7fa964096f20 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa95b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa95b7fe640 1 -- 172.21.15.204:0/3035591009 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa954013a60 con 0x7fa964096f20 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa95b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa95b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa95b7fe640 10 monclient: dump: 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.446 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa961ffb640 4 set_mon_vals no callback set 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa969cb8640 10 monclient: _renew_subs 2024-09-15T01:34:24.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa969cb8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:34:24.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.424+0000 7fa969cb8640 1 -- 172.21.15.204:0/3035591009 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa964131e40 con 0x7fa964096f20 2024-09-15T01:34:24.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.425+0000 7fa95b7fe640 1 -- 172.21.15.204:0/3035591009 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa954013bc0 con 0x7fa964096f20 2024-09-15T01:34:24.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.425+0000 7fa95b7fe640 1 --2- 172.21.15.204:0/3035591009 >> v2:172.21.15.204:6812/28210 conn(0x7fa9440397e0 0x7fa94403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.425+0000 7fa95b7fe640 1 -- 172.21.15.204:0/3035591009 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5373+0+0 (secure 0 0 0) 0x7fa95404b970 con 0x7fa964096f20 2024-09-15T01:34:24.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.425+0000 7fa969cb8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.425+0000 7fa962ffd640 1 --2- 172.21.15.204:0/3035591009 >> v2:172.21.15.204:6812/28210 conn(0x7fa9440397e0 0x7fa94403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.425+0000 7fa969cb8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.425+0000 7fa969cb8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:34:24.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.425+0000 7fa969cb8640 1 -- 172.21.15.204:0/3035591009 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa964071100 con 0x7fa964096f20 2024-09-15T01:34:24.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.425+0000 7fa962ffd640 10 monclient: get_auth_request con 0x7fa9440397e0 auth_method 0 2024-09-15T01:34:24.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.425+0000 7fa962ffd640 1 --2- 172.21.15.204:0/3035591009 >> v2:172.21.15.204:6812/28210 conn(0x7fa9440397e0 0x7fa94403bc90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa94c034000 tx=0x7fa94c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.430+0000 7fa95b7fe640 1 -- 172.21.15.204:0/3035591009 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa95400ca10 con 0x7fa964096f20 2024-09-15T01:34:24.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.430+0000 7fa95b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.449 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.430+0000 7fa95b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:24.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.523+0000 7ff13ae6f640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T01:34:24.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.523+0000 7ff13ae6f640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T01:34:24.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.523+0000 7ff13ae6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:24.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.523+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3854594145 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7ff13409cef0 con 0x7ff1340a08c0 2024-09-15T01:34:24.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.525+0000 7ff128ff9640 1 -- 172.21.15.178:0/3854594145 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7ff12401bba0 con 0x7ff1340a08c0 2024-09-15T01:34:24.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.525+0000 7ff128ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T01:34:24.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.525+0000 7ff128ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:24.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3854594145 >> v2:172.21.15.204:6812/28210 conn(0x7ff1200397b0 msgr2=0x7ff12003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 1 --2- 172.21.15.178:0/3854594145 >> v2:172.21.15.204:6812/28210 conn(0x7ff1200397b0 0x7ff12003bc60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff130000c00 tx=0x7ff130032000 comp rx=0 tx=0).stop 2024-09-15T01:34:24.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 10 monclient: shutdown 2024-09-15T01:34:24.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:24.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3854594145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1340a08c0 msgr2=0x7ff1341316e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 1 --2- 172.21.15.178:0/3854594145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1340a08c0 0x7ff1341316e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff12400e950 tx=0x7ff1240091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3854594145 shutdown_connections 2024-09-15T01:34:24.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 1 --2- 172.21.15.178:0/3854594145 >> v2:172.21.15.204:6812/28210 conn(0x7ff1200397b0 0x7ff12003bc60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 1 --2- 172.21.15.178:0/3854594145 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff1340a2930 0x7ff13411c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 1 --2- 172.21.15.178:0/3854594145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1340a08c0 0x7ff1341316e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 1 --2- 172.21.15.178:0/3854594145 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff13409faa0 0x7ff13409ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3854594145 >> 172.21.15.178:0/3854594145 conn(0x7ff1340a5ed0 msgr2=0x7ff1341358d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:24.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.527+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3854594145 shutdown_connections 2024-09-15T01:34:24.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:24.528+0000 7ff13ae6f640 1 -- 172.21.15.178:0/3854594145 wait complete. 2024-09-15T01:34:24.539 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.539 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=8,config=2,health=10,kv=7,logm=73} crc {auth=2304070878,config=3140514009,health=1306142553,kv=2253581948,logm=3777356027}) 2024-09-15T01:34:24.539 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=7,config=2,health=12,kv=7,logm=72} crc {auth=2620470011,config=3140514009,health=2157093694,kv=706962213,logm=3743215377}) 2024-09-15T01:34:24.539 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.539 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=8,config=2,health=10,kv=7,logm=73} crc {auth=2304070878,config=3140514009,health=1306142553,kv=2253581948,logm=3777356027}) 2024-09-15T01:34:24.539 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=8,config=2,health=12,kv=8,logm=70} crc {auth=2304070878,config=3140514009,health=2157093694,kv=706962213,logm=508332891}) 2024-09-15T01:34:24.539 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.539 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=8,config=2,health=10,kv=7,logm=73} crc {auth=2304070878,config=3140514009,health=1306142553,kv=2253581948,logm=3777356027}) 2024-09-15T01:34:24.539 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=8,config=2,health=10,kv=7,logm=73} crc {auth=2304070878,config=3140514009,health=922668212,kv=2951374574,logm=2735743339}) 2024-09-15T01:34:24.540 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.540 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=8,config=2,health=10,kv=7,logm=73} crc {auth=2304070878,config=3140514009,health=1306142553,kv=2253581948,logm=3777356027}) 2024-09-15T01:34:24.540 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=8,config=2,health=12,kv=8,logm=70} crc {auth=2304070878,config=3140514009,health=2157093694,kv=706962213,logm=979161560}) 2024-09-15T01:34:24.540 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.540 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=8,config=2,health=10,kv=7,logm=73} crc {auth=2304070878,config=3140514009,health=1306142553,kv=2253581948,logm=3777356027}) 2024-09-15T01:34:24.540 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=6,config=2,health=10,kv=8,logm=74} crc {auth=1079484665,config=3140514009,health=1218526840,kv=706962213,logm=2968451468}) 2024-09-15T01:34:24.540 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.540 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=8,config=2,health=10,kv=7,logm=73} crc {auth=2304070878,config=3140514009,health=1306142553,kv=2253581948,logm=3777356027}) 2024-09-15T01:34:24.540 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=8,config=2,health=12,kv=8,logm=70} crc {auth=2304070878,config=3140514009,health=2157093694,kv=706962213,logm=3585348074}) 2024-09-15T01:34:24.540 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.540 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=8,config=2,health=10,kv=7,logm=73} crc {auth=2304070878,config=3140514009,health=1306142553,kv=2253581948,logm=3777356027}) 2024-09-15T01:34:24.540 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=8,config=2,health=12,kv=8,logm=70} crc {auth=2304070878,config=3140514009,health=2157093694,kv=706962213,logm=3511298609}) 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=8,config=2,health=10,kv=7,logm=73} crc {auth=2304070878,config=3140514009,health=1306142553,kv=2253581948,logm=3777356027}) 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=7,config=2,health=11,kv=7,logm=73} crc {auth=2022202023,config=3140514009,health=1886805112,kv=963045094,logm=429460544}) 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3399736125}) 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2301981449}) 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3399736125}) 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3529360753}) 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3399736125}) 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=728300158}) 2024-09-15T01:34:24.541 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3399736125}) 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=701590877}) 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3399736125}) 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1049349333}) 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3399736125}) 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=434725263}) 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3399736125}) 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3939889275}) 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3399736125}) 2024-09-15T01:34:24.542 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1458739088}) 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2907810030}) 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3467545053}) 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2907810030}) 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=59733187}) 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2907810030}) 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3084092306}) 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2907810030}) 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3297287362}) 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.543 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2907810030}) 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2260761547}) 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2907810030}) 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3247826427}) 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2907810030}) 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2712532020}) 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2907810030}) 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1748863379}) 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2315564299}) 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1568952141}) 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.544 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2315564299}) 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1242819746}) 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2315564299}) 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1876377018}) 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2315564299}) 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2135214739}) 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2315564299}) 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1785606594}) 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2315564299}) 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3784240527}) 2024-09-15T01:34:24.545 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2315564299}) 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=927604950}) 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2315564299}) 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2537677776}) 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=568127612}) 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2875094474}) 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=568127612}) 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4261431893}) 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=568127612}) 2024-09-15T01:34:24.546 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3505765631}) 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=568127612}) 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3776243315}) 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=568127612}) 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=280579174}) 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=568127612}) 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1299156327}) 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=568127612}) 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=630692230}) 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.547 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=568127612}) 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1855834912}) 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2812929236}) 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1164193160}) 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2812929236}) 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3096574413}) 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2812929236}) 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2783751451}) 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2812929236}) 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3182114362}) 2024-09-15T01:34:24.548 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.549 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2812929236}) 2024-09-15T01:34:24.549 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2175285087}) 2024-09-15T01:34:24.549 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.549 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2812929236}) 2024-09-15T01:34:24.549 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3620787676}) 2024-09-15T01:34:24.549 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.549 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2812929236}) 2024-09-15T01:34:24.549 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2930975834}) 2024-09-15T01:34:24.549 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.549 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2812929236}) 2024-09-15T01:34:24.549 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3023981681}) 2024-09-15T01:34:24.551 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T01:34:24.552 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.552 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2607607534}) 2024-09-15T01:34:24.552 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1542272111}) 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2607607534}) 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1257333375}) 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2607607534}) 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3218227188}) 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2607607534}) 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2333330302}) 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2607607534}) 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4131537780}) 2024-09-15T01:34:24.553 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2607607534}) 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3860399817}) 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2607607534}) 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2244273140}) 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2607607534}) 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1932277907}) 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=85133037}) 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3053503926}) 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=85133037}) 2024-09-15T01:34:24.554 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=442708720}) 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=85133037}) 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1937817915}) 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=85133037}) 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4001151392}) 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=85133037}) 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3146019976}) 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=85133037}) 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3742268594}) 2024-09-15T01:34:24.555 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.556 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=85133037}) 2024-09-15T01:34:24.556 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3435831876}) 2024-09-15T01:34:24.556 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.556 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=85133037}) 2024-09-15T01:34:24.556 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=676699313}) 2024-09-15T01:34:24.561 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.561 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3886471479}) 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2775886060}) 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3886471479}) 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1581142804}) 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3886471479}) 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2796414657}) 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3886471479}) 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2133330596}) 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3886471479}) 2024-09-15T01:34:24.562 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1353867492}) 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3886471479}) 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2720108538}) 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3886471479}) 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1984381765}) 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3886471479}) 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2663726581}) 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=93,mdsmap=3,mgr=4} crc {logm=2660325052,mdsmap=1531151339,mgr=3154979937}) 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=91,mdsmap=3,mgr=6} crc {logm=25624377,mdsmap=1531151339,mgr=2788037318}) 2024-09-15T01:34:24.563 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=93,mdsmap=3,mgr=4} crc {logm=2660325052,mdsmap=1531151339,mgr=3154979937}) 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=93,mdsmap=3,mgr=4} crc {logm=2220883175,mdsmap=1531151339,mgr=3154979937}) 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=93,mdsmap=3,mgr=4} crc {logm=2660325052,mdsmap=1531151339,mgr=3154979937}) 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=87,mdsmap=3,mgr=7,mgr_metadata=1,mgrstat=2} crc {logm=3880384963,mdsmap=1531151339,mgr=4058211233,mgr_metadata=3150520333,mgrstat=583307694}) 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=93,mdsmap=3,mgr=4} crc {logm=2660325052,mdsmap=1531151339,mgr=3154979937}) 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=93,mdsmap=3,mgr=4} crc {logm=3775573286,mdsmap=1531151339,mgr=2509566662}) 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=93,mdsmap=3,mgr=4} crc {logm=2660325052,mdsmap=1531151339,mgr=3154979937}) 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=95,mdsmap=3,mgr=2} crc {logm=3234305830,mdsmap=1531151339,mgr=194516567}) 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=93,mdsmap=3,mgr=4} crc {logm=2660325052,mdsmap=1531151339,mgr=3154979937}) 2024-09-15T01:34:24.564 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=88,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1} crc {logm=2539576840,mdsmap=1531151339,mgr=3532922113,mgr_command_descs=3545737492,mgr_metadata=3150520333}) 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=93,mdsmap=3,mgr=4} crc {logm=2660325052,mdsmap=1531151339,mgr=3154979937}) 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=87,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1} crc {logm=1729208622,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333}) 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=93,mdsmap=3,mgr=4} crc {logm=2660325052,mdsmap=1531151339,mgr=3154979937}) 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=90,mdsmap=3,mgr=7} crc {logm=1861754380,mdsmap=1531151339,mgr=4058211233}) 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgr=3,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=5,osd_snap=2,osdmap=73} crc {mgr=2941987429,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1295865042,mon_config_key=1111662797,monmap=1281023599,osd_metadata=2302127439,osd_snap=3178633393,osdmap=2094596141}) 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgr=4,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=4,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=72} crc {mgr=622933224,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1836398471,mon_config_key=3476930649,monmap=2850442478,osd_metadata=4061885386,osd_pg_creating=107862661,osd_snap=3178633393,osdmap=3406886403}) 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgr=3,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=5,osd_snap=2,osdmap=73} crc {mgr=2941987429,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1295865042,mon_config_key=1111662797,monmap=1281023599,osd_metadata=2302127439,osd_snap=3178633393,osdmap=2094596141}) 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgr=4,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=69} crc {mgr=4041079024,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=101570119,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=107862661,osd_snap=3178633393,osdmap=4231789056}) 2024-09-15T01:34:24.565 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgr=3,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=5,osd_snap=2,osdmap=73} crc {mgr=2941987429,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1295865042,mon_config_key=1111662797,monmap=1281023599,osd_metadata=2302127439,osd_snap=3178633393,osdmap=2094596141}) 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr=4,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=70} crc {mgr=4041079024,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=4211228831,mon_config_key=1111662797,monmap=2850442478,osd_metadata=528112302,osd_pg_creating=107862661,osd_snap=3823785925,osdmap=3245395470}) 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgr=3,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=5,osd_snap=2,osdmap=73} crc {mgr=2941987429,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1295865042,mon_config_key=1111662797,monmap=1281023599,osd_metadata=2302127439,osd_snap=3178633393,osdmap=2094596141}) 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mgr=4,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=70} crc {mgr=4041079024,mgr_metadata=3150520333,mgrstat=4211228831,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=107862661,osd_snap=956710772,osdmap=2940174325}) 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgr=3,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=5,osd_snap=2,osdmap=73} crc {mgr=2941987429,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1295865042,mon_config_key=1111662797,monmap=1281023599,osd_metadata=2302127439,osd_snap=3178633393,osdmap=2094596141}) 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mgr=4,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=72} crc {mgr=4041079024,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2586851907,mon_config_key=1111662797,monmap=2850442478,osd_metadata=4279226934,osd_pg_creating=107862661,osd_snap=3823785925,osdmap=1206021387}) 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgr=3,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=5,osd_snap=2,osdmap=73} crc {mgr=2941987429,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1295865042,mon_config_key=1111662797,monmap=1281023599,osd_metadata=2302127439,osd_snap=3178633393,osdmap=2094596141}) 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mgr=3,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=70} crc {mgr=2941987429,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=4211228831,mon_config_key=1111662797,monmap=1281023599,osd_metadata=1261905835,osd_pg_creating=107862661,osd_snap=3178633393,osdmap=3188852560}) 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.566 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgr=3,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=5,osd_snap=2,osdmap=73} crc {mgr=2941987429,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1295865042,mon_config_key=1111662797,monmap=1281023599,osd_metadata=2302127439,osd_snap=3178633393,osdmap=2094596141}) 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mgr=4,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=70} crc {mgr=4041079024,mgr_metadata=3150520333,mgrstat=4211228831,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=107862661,osd_snap=956710772,osdmap=968619418}) 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgr=3,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=5,osd_snap=2,osdmap=73} crc {mgr=2941987429,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1295865042,mon_config_key=1111662797,monmap=1281023599,osd_metadata=2302127439,osd_snap=3178633393,osdmap=2094596141}) 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr=4,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=70} crc {mgr=4041079024,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=4211228831,mon_config_key=1111662797,monmap=1281023599,osd_metadata=4061885386,osd_pg_creating=107862661,osd_snap=3178633393,osdmap=2687902406}) 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=57} crc {osdmap=2190549397}) 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=57} crc {osdmap=2759464712}) 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=57} crc {osdmap=2190549397}) 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=59} crc {osdmap=3236535080}) 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=57} crc {osdmap=2190549397}) 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=57} crc {osdmap=3183878258}) 2024-09-15T01:34:24.567 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.568 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=57} crc {osdmap=2190549397}) 2024-09-15T01:34:24.568 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=54} crc {osdmap=742515694}) 2024-09-15T01:34:24.568 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.568 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=57} crc {osdmap=2190549397}) 2024-09-15T01:34:24.568 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=55} crc {osdmap=824779999}) 2024-09-15T01:34:24.568 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.568 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=57} crc {osdmap=2190549397}) 2024-09-15T01:34:24.568 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=57} crc {osdmap=3099668024}) 2024-09-15T01:34:24.568 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.568 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=57} crc {osdmap=2190549397}) 2024-09-15T01:34:24.568 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=58} crc {osdmap=3374380610}) 2024-09-15T01:34:24.568 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:34:24.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=57} crc {osdmap=2190549397}) 2024-09-15T01:34:24.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:34:24.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=57} crc {osdmap=359384561}) 2024-09-15T01:34:24.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.601+0000 7fa969cb8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:34:24.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.601+0000 7fa969cb8640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:34:24.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.601+0000 7fa969cb8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:34:24.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.601+0000 7fa969cb8640 1 -- 172.21.15.204:0/3035591009 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) v1 -- 0x7fa964068870 con 0x7fa964096f20 2024-09-15T01:34:24.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.602+0000 7fa95b7fe640 1 -- 172.21.15.204:0/3035591009 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=-39 (39) Directory not empty v78) v1 ==== 99+0+0 (secure 0 0 0) 0x7fa9540192c0 con 0x7fa964096f20 2024-09-15T01:34:24.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.602+0000 7fa95b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:34:24.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.602+0000 7fa95b7fe640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-09-15T01:34:24.605 INFO:tasks.workunit.client.0.smithi204.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-09-15T01:34:24.609 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:34:24.609 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush rm bar 2024-09-15T01:34:24.609 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:34:24.609 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm bar 2024-09-15T01:34:24.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.680+0000 7fbdd3c20640 10 monclient: get_monmap_and_config 2024-09-15T01:34:24.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.680+0000 7fbdd3c20640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.680+0000 7fbdd3c20640 10 monclient: monmap: 2024-09-15T01:34:24.683 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:24.683 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.683 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:24.682433+0000 2024-09-15T01:34:24.683 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:24.682433+0000 2024-09-15T01:34:24.683 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:24.684 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.681+0000 7fbdd3c20640 1 Processor -- start 2024-09-15T01:34:24.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.681+0000 7fbdd3c20640 1 -- start start 2024-09-15T01:34:24.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.681+0000 7fbdd3c20640 10 monclient: init 2024-09-15T01:34:24.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 10 monclient: _add_conns ranks=[5,7,8,6,3,9,2,0,10,1,4] 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbdcc0a2e30 0x7fbdcc132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 10 monclient(hunting): picked mon.noname-h con 0x7fbdcc0a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdcc09fb80 0x7fbdcc09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 10 monclient(hunting): picked mon.noname-i con 0x7fbdcc09fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdcc0a0a40 0x7fbdcc0a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 10 monclient(hunting): picked mon.noname-e con 0x7fbdcc0a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc09e6c0 con 0x7fbdcc0a2e30 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0a04b0 con 0x7fbdcc09fb80 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0a3200 con 0x7fbdcc0a0a40 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd3c20640 10 monclient(hunting): authenticate will time out at 880.325207s 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd19bc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbdcc0a2e30 0x7fbdcc132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdcc09fb80 0x7fbdcc09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdcc09fb80 0x7fbdcc09ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:34108/0 (socket says 172.21.15.204:34108) 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd19bc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbdcc0a2e30 0x7fbdcc132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:55766/0 (socket says 172.21.15.204:55766) 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 1 -- 172.21.15.204:0/909394501 learned_addr learned my addr 172.21.15.204:0/909394501 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 10 monclient(hunting): get_auth_request con 0x7fbdcc09fb80 auth_method 0 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd19bc640 10 monclient(hunting): get_auth_request con 0x7fbdcc0a2e30 auth_method 0 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd19bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd19bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd19bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd19bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd19bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd19bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd21bd640 1 --2- 172.21.15.204:0/909394501 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdcc0a0a40 0x7fbdcc0a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd21bd640 10 monclient(hunting): get_auth_request con 0x7fbdcc0a0a40 auth_method 0 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd21bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd21bd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd21bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 10 monclient(hunting): handle_auth_done global_id 5164 payload 274 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 1 -- 172.21.15.204:0/909394501 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdcc0a0a40 msgr2=0x7fbdcc0a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:24.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 1 --2- 172.21.15.204:0/909394501 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdcc0a0a40 0x7fbdcc0a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 1 -- 172.21.15.204:0/909394501 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbdcc0a2e30 msgr2=0x7fbdcc132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 1 --2- 172.21.15.204:0/909394501 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbdcc0a2e30 0x7fbdcc132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 1 monclient: found mon.noname-i 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.682+0000 7fbdd11bb640 1 -- 172.21.15.204:0/909394501 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc04b720 con 0x7fbdcc09fb80 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd11bb640 1 --2- 172.21.15.204:0/909394501 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdcc09fb80 0x7fbdcc09ff70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009940 tx=0x7fbdbc0091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=f0592d481bf6b77c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd09ba640 1 -- 172.21.15.204:0/909394501 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbdbc02ac60 con 0x7fbdcc09fb80 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd09ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd09ba640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd09ba640 10 monclient: dump: 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.696 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.697 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd09ba640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd09ba640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd09ba640 1 -- 172.21.15.204:0/909394501 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc02ade0 con 0x7fbdcc09fb80 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 5 monclient: authenticate success, global_id 5164 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd09ba640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 4 set_mon_vals no callback set 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd09ba640 1 -- 172.21.15.204:0/909394501 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbdbc031ca0 con 0x7fbdcc09fb80 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd09ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd09ba640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:24.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd09ba640 10 monclient: dump: 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.699 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 10 monclient: shutdown 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 1 -- 172.21.15.204:0/909394501 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdcc09fb80 msgr2=0x7fbdcc09ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 1 --2- 172.21.15.204:0/909394501 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdcc09fb80 0x7fbdcc09ff70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009940 tx=0x7fbdbc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 1 -- 172.21.15.204:0/909394501 shutdown_connections 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 1 --2- 172.21.15.204:0/909394501 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdcc0a0a40 0x7fbdcc0a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 1 --2- 172.21.15.204:0/909394501 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbdcc09fb80 0x7fbdcc09ff70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 1 --2- 172.21.15.204:0/909394501 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbdcc0a2e30 0x7fbdcc132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.683+0000 7fbdd3c20640 1 -- 172.21.15.204:0/909394501 >> 172.21.15.204:0/909394501 conn(0x7fbdcc06e860 msgr2=0x7fbdcc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.684+0000 7fbdd3c20640 1 -- 172.21.15.204:0/909394501 shutdown_connections 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.684+0000 7fbdd3c20640 1 -- 172.21.15.204:0/909394501 wait complete. 2024-09-15T01:34:24.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.684+0000 7fbdd3c20640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.684+0000 7fbdd3c20640 10 monclient: monmap: 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.701 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.684+0000 7fbdd3c20640 1 Processor -- start 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.684+0000 7fbdd3c20640 1 -- start start 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.684+0000 7fbdd3c20640 10 monclient: init 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 10 monclient: _add_conns ranks=[0,2,8,4,7,6,1,3,5] 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdcc09fb80 0x7fbdcc1307f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 10 monclient(hunting): picked mon.noname-a con 0x7fbdcc09fb80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdcc0a0a40 0x7fbdcc11d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 10 monclient(hunting): picked mon.noname-b con 0x7fbdcc0a0a40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:24.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdcc0a2e30 0x7fbdcc11d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 10 monclient(hunting): picked mon.noname-e con 0x7fbdcc0a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc09ca40 con 0x7fbdcc09fb80 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc09d650 con 0x7fbdcc0a0a40 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc11f660 con 0x7fbdcc0a2e30 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd3c20640 10 monclient(hunting): authenticate will time out at 880.328283s 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdcc09fb80 0x7fbdcc1307f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd11bb640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdcc0a0a40 0x7fbdcc11d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdcc09fb80 0x7fbdcc1307f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:54190/0 (socket says 172.21.15.204:54190) 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 1 -- 172.21.15.204:0/1036850042 learned_addr learned my addr 172.21.15.204:0/1036850042 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:24.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 10 monclient(hunting): get_auth_request con 0x7fbdcc09fb80 auth_method 0 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd21bd640 1 --2- 172.21.15.204:0/1036850042 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdcc0a2e30 0x7fbdcc11d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd11bb640 10 monclient(hunting): get_auth_request con 0x7fbdcc0a0a40 auth_method 0 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd11bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd11bb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd11bb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd21bd640 10 monclient(hunting): get_auth_request con 0x7fbdcc0a2e30 auth_method 0 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd21bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd21bd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd21bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd11bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd11bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd11bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd21bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd21bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd21bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 10 monclient(hunting): handle_auth_done global_id 5256 payload 824 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 1 -- 172.21.15.204:0/1036850042 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdcc0a2e30 msgr2=0x7fbdcc11d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 1 --2- 172.21.15.204:0/1036850042 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdcc0a2e30 0x7fbdcc11d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 1 -- 172.21.15.204:0/1036850042 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdcc0a0a40 msgr2=0x7fbdcc11d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 1 --2- 172.21.15.204:0/1036850042 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdcc0a0a40 0x7fbdcc11d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 1 monclient: found mon.noname-a 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 1 -- 172.21.15.204:0/1036850042 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc009450 con 0x7fbdcc09fb80 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.685+0000 7fbdd19bc640 1 --2- 172.21.15.204:0/1036850042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdcc09fb80 0x7fbdcc1307f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800e900 tx=0x7fbdc80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc1ffb640 1 -- 172.21.15.204:0/1036850042 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbdc8009480 con 0x7fbdcc09fb80 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc1ffb640 10 monclient: dump: 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.706 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc1ffb640 1 -- 172.21.15.204:0/1036850042 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc800cd60 con 0x7fbdcc09fb80 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdd3c20640 5 monclient: authenticate success, global_id 5256 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdd3c20640 10 monclient: _renew_subs 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdd3c20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdd3c20640 1 -- 172.21.15.204:0/1036850042 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdcc120c20 con 0x7fbdcc09fb80 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc1ffb640 1 -- 172.21.15.204:0/1036850042 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbdc80138f0 con 0x7fbdcc09fb80 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc1ffb640 10 monclient: dump: 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.708 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdd3c20640 10 monclient: _renew_subs 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdc3fff640 4 set_mon_vals no callback set 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdd3c20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.686+0000 7fbdd3c20640 1 -- 172.21.15.204:0/1036850042 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdcc11ee80 con 0x7fbdcc09fb80 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.687+0000 7fbdc1ffb640 1 -- 172.21.15.204:0/1036850042 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fbdc8013470 con 0x7fbdcc09fb80 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.687+0000 7fbdc1ffb640 1 --2- 172.21.15.204:0/1036850042 >> v2:172.21.15.204:6812/28210 conn(0x7fbdb00396e0 0x7fbdb003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.687+0000 7fbdd3c20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.687+0000 7fbdd3c20640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.687+0000 7fbdd3c20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.687+0000 7fbdc1ffb640 1 -- 172.21.15.204:0/1036850042 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5373+0+0 (secure 0 0 0) 0x7fbdc804bad0 con 0x7fbdcc09fb80 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.687+0000 7fbdd3c20640 1 -- 172.21.15.204:0/1036850042 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdcc070cc0 con 0x7fbdcc09fb80 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.687+0000 7fbdd11bb640 1 --2- 172.21.15.204:0/1036850042 >> v2:172.21.15.204:6812/28210 conn(0x7fbdb00396e0 0x7fbdb003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.687+0000 7fbdd11bb640 10 monclient: get_auth_request con 0x7fbdb00396e0 auth_method 0 2024-09-15T01:34:24.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.687+0000 7fbdd11bb640 1 --2- 172.21.15.204:0/1036850042 >> v2:172.21.15.204:6812/28210 conn(0x7fbdb00396e0 0x7fbdb003bb90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc034000 tx=0x7fbdbc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.692+0000 7fbdc1ffb640 1 -- 172.21.15.204:0/1036850042 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fbdc8013720 con 0x7fbdcc09fb80 2024-09-15T01:34:24.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.692+0000 7fbdc1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.692+0000 7fbdc1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:24.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.852+0000 7fbdd3c20640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T01:34:24.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.852+0000 7fbdd3c20640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T01:34:24.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.852+0000 7fbdd3c20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:24.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.852+0000 7fbdd3c20640 1 -- 172.21.15.204:0/1036850042 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) v1 -- 0x7fbdcc0a1f10 con 0x7fbdcc09fb80 2024-09-15T01:34:24.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.854+0000 7fbdc1ffb640 1 -- 172.21.15.204:0/1036850042 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=-39 (39) Directory not empty v78) v1 ==== 99+0+0 (secure 0 0 0) 0x7fbdc802a090 con 0x7fbdcc09fb80 2024-09-15T01:34:24.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.854+0000 7fbdc1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T01:34:24.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.854+0000 7fbdc1ffb640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-09-15T01:34:24.856 INFO:tasks.workunit.client.0.smithi204.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-09-15T01:34:24.860 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:34:24.860 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush unlink host1 bar 2024-09-15T01:34:24.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.931+0000 7f2747f1d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:24.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.931+0000 7f2747f1d640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.931+0000 7f2747f1d640 10 monclient: monmap: 2024-09-15T01:34:24.934 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:24.934 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.934 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:24.933420+0000 2024-09-15T01:34:24.935 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:24.933420+0000 2024-09-15T01:34:24.935 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.935 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.935 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:24.935 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:24.935 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.935 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.935 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.935 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.935 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.935 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.935 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.936 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:24.936 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:24.936 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.932+0000 7f2747f1d640 1 Processor -- start 2024-09-15T01:34:24.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.932+0000 7f2747f1d640 1 -- start start 2024-09-15T01:34:24.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.932+0000 7f2747f1d640 10 monclient: init 2024-09-15T01:34:24.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 10 monclient: _add_conns ranks=[10,5,2,8,9,4,1,3,7,0,6] 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f27400a2c60 0x7f2740132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 10 monclient(hunting): picked mon.noname-f con 0x7f27400a2c60 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f274009fc60 0x7f27400a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 10 monclient(hunting): picked mon.noname-h con 0x7f274009fc60 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27400a0ab0 0x7f27400a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 10 monclient(hunting): picked mon.noname-b con 0x7f27400a0ab0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27400a0590 con 0x7f27400a2c60 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27400a3030 con 0x7f27400a0ab0 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f274009d5a0 con 0x7f274009fc60 2024-09-15T01:34:24.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2747f1d640 10 monclient(hunting): authenticate will time out at 880.576313s 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f27400a2c60 0x7f2740132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f27400a2c60 0x7f2740132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48948/0 (socket says 172.21.15.204:48948) 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 1 -- 172.21.15.204:0/3520204719 learned_addr learned my addr 172.21.15.204:0/3520204719 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 10 monclient(hunting): get_auth_request con 0x7f27400a2c60 auth_method 0 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f27454b8640 1 --2- 172.21.15.204:0/3520204719 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f274009fc60 0x7f27400a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f27454b8640 10 monclient(hunting): get_auth_request con 0x7f274009fc60 auth_method 0 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f27454b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f27454b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f27454b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f27454b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f27454b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f27454b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 10 monclient(hunting): handle_auth_done global_id 5617 payload 274 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f27464ba640 1 --2- 172.21.15.204:0/3520204719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27400a0ab0 0x7f27400a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 1 -- 172.21.15.204:0/3520204719 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f274009fc60 msgr2=0x7f27400a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 1 --2- 172.21.15.204:0/3520204719 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f274009fc60 0x7f27400a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 1 -- 172.21.15.204:0/3520204719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27400a0ab0 msgr2=0x7f27400a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 1 --2- 172.21.15.204:0/3520204719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27400a0ab0 0x7f27400a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 1 monclient: found mon.noname-f 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f2745cb9640 1 -- 172.21.15.204:0/3520204719 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274004b720 con 0x7f27400a2c60 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f27464ba640 10 monclient: get_auth_request con 0x7f27400a0ab0 auth_method 0 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.933+0000 7f27464ba640 1 --2- 172.21.15.204:0/3520204719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27400a0ab0 0x7f27400a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2745cb9640 1 --2- 172.21.15.204:0/3520204719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27400a2c60 0x7f2740132a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f273c009910 tx=0x7f273c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5f435da9029287d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2744cb7640 1 -- 172.21.15.204:0/3520204719 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f273c02ac80 con 0x7f27400a2c60 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2744cb7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2744cb7640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2744cb7640 10 monclient: dump: 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.948 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.949 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2744cb7640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2744cb7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2744cb7640 1 -- 172.21.15.204:0/3520204719 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273c02ae00 con 0x7f27400a2c60 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2747f1d640 5 monclient: authenticate success, global_id 5617 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2747f1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2744cb7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2747f1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2744cb7640 1 -- 172.21.15.204:0/3520204719 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f273c031ca0 con 0x7f27400a2c60 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2744cb7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2747f1d640 4 set_mon_vals no callback set 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2744cb7640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.934+0000 7f2744cb7640 10 monclient: dump: 2024-09-15T01:34:24.950 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.951 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.952 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.952 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 10 monclient: shutdown 2024-09-15T01:34:24.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:24.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 1 -- 172.21.15.204:0/3520204719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27400a2c60 msgr2=0x7f2740132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 1 --2- 172.21.15.204:0/3520204719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27400a2c60 0x7f2740132a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f273c009910 tx=0x7f273c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 1 -- 172.21.15.204:0/3520204719 shutdown_connections 2024-09-15T01:34:24.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 1 --2- 172.21.15.204:0/3520204719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27400a0ab0 0x7f27400a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 1 --2- 172.21.15.204:0/3520204719 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f274009fc60 0x7f27400a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 1 --2- 172.21.15.204:0/3520204719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27400a2c60 0x7f2740132a60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 1 -- 172.21.15.204:0/3520204719 >> 172.21.15.204:0/3520204719 conn(0x7f27400a60c0 msgr2=0x7f27400a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:24.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 1 -- 172.21.15.204:0/3520204719 shutdown_connections 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 1 -- 172.21.15.204:0/3520204719 wait complete. 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 10 monclient: build_initial_monmap 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.935+0000 7f2747f1d640 10 monclient: monmap: 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:24.953 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 1 Processor -- start 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 1 -- start start 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 10 monclient: init 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 10 monclient: _add_conns ranks=[4,7,1,2,5,8,3,0,6] 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f274009fc60 0x7f2740131a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 10 monclient(hunting): picked mon.noname-c con 0x7f274009fc60 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:24.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f27400a0ab0 0x7f274011e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 10 monclient(hunting): picked mon.noname-i con 0x7f27400a0ab0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27400a2c60 0x7f274011eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 10 monclient(hunting): picked mon.noname-f con 0x7f27400a2c60 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274009d7d0 con 0x7f27400a2c60 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f274009ca40 con 0x7f274009fc60 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2740121e60 con 0x7f27400a0ab0 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2747f1d640 10 monclient(hunting): authenticate will time out at 880.579656s 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27454b8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f27400a0ab0 0x7f274011e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27464ba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27400a2c60 0x7f274011eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27454b8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f27400a0ab0 0x7f274011e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:34110/0 (socket says 172.21.15.204:34110) 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2745cb9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f274009fc60 0x7f2740131a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27454b8640 1 -- 172.21.15.204:0/3742281555 learned_addr learned my addr 172.21.15.204:0/3742281555 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27454b8640 10 monclient(hunting): get_auth_request con 0x7f27400a0ab0 auth_method 0 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27454b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27454b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27454b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27464ba640 10 monclient(hunting): get_auth_request con 0x7f27400a2c60 auth_method 0 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27464ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27464ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27464ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2745cb9640 10 monclient(hunting): get_auth_request con 0x7f274009fc60 auth_method 0 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2745cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:24.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2745cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2745cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27464ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27464ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27464ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2745cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2745cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f2745cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27454b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27454b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27454b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.936+0000 7f27464ba640 10 monclient(hunting): handle_auth_done global_id 5626 payload 824 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f27464ba640 1 -- 172.21.15.204:0/3742281555 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f27400a0ab0 msgr2=0x7f274011e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f27464ba640 1 --2- 172.21.15.204:0/3742281555 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f27400a0ab0 0x7f274011e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f27464ba640 1 -- 172.21.15.204:0/3742281555 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f274009fc60 msgr2=0x7f2740131a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f27464ba640 1 --2- 172.21.15.204:0/3742281555 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f274009fc60 0x7f2740131a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f27464ba640 10 monclient: _finish_hunting 0 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f27464ba640 1 monclient: found mon.noname-f 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f27464ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f27464ba640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f27464ba640 1 -- 172.21.15.204:0/3742281555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c009450 con 0x7f27400a2c60 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f27464ba640 1 --2- 172.21.15.204:0/3742281555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27400a2c60 0x7f274011eaa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f273800cf30 tx=0x7f2738007840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2735ffb640 1 -- 172.21.15.204:0/3742281555 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f273800ae90 con 0x7f27400a2c60 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2735ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2735ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2735ffb640 10 monclient: dump: 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.958 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.959 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2735ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2735ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2735ffb640 1 -- 172.21.15.204:0/3742281555 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273800dad0 con 0x7f27400a2c60 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2747f1d640 5 monclient: authenticate success, global_id 5626 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2747f1d640 10 monclient: _renew_subs 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2747f1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2747f1d640 1 -- 172.21.15.204:0/3742281555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2740120050 con 0x7f27400a2c60 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2735ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2735ffb640 1 -- 172.21.15.204:0/3742281555 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f273800dc80 con 0x7f27400a2c60 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2735ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2735ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:24.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2735ffb640 10 monclient: dump: 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:24.961 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:24.962 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:24.962 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:24.962 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:24.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2737fff640 4 set_mon_vals no callback set 2024-09-15T01:34:24.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2747f1d640 10 monclient: _renew_subs 2024-09-15T01:34:24.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2747f1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:24.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.937+0000 7f2747f1d640 1 -- 172.21.15.204:0/3742281555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2740120c20 con 0x7f27400a2c60 2024-09-15T01:34:24.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.938+0000 7f2735ffb640 1 -- 172.21.15.204:0/3742281555 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f273800d650 con 0x7f27400a2c60 2024-09-15T01:34:24.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.938+0000 7f2747f1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.938+0000 7f2747f1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.938+0000 7f2747f1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:24.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.938+0000 7f2747f1d640 1 -- 172.21.15.204:0/3742281555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2740122360 con 0x7f27400a2c60 2024-09-15T01:34:24.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.938+0000 7f2735ffb640 1 --2- 172.21.15.204:0/3742281555 >> v2:172.21.15.204:6812/28210 conn(0x7f27240397b0 0x7f272403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:24.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.938+0000 7f2735ffb640 1 -- 172.21.15.204:0/3742281555 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5373+0+0 (secure 0 0 0) 0x7f273804bc10 con 0x7f27400a2c60 2024-09-15T01:34:24.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.938+0000 7f2745cb9640 1 --2- 172.21.15.204:0/3742281555 >> v2:172.21.15.204:6812/28210 conn(0x7f27240397b0 0x7f272403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:24.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.938+0000 7f2745cb9640 10 monclient: get_auth_request con 0x7f27240397b0 auth_method 0 2024-09-15T01:34:24.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.939+0000 7f2745cb9640 1 --2- 172.21.15.204:0/3742281555 >> v2:172.21.15.204:6812/28210 conn(0x7f27240397b0 0x7f272403bc60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f273c009420 tx=0x7f273c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:24.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.943+0000 7f2735ffb640 1 -- 172.21.15.204:0/3742281555 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f27380143a0 con 0x7f27400a2c60 2024-09-15T01:34:24.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.943+0000 7f2735ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:24.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:24.943+0000 7f2735ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:25.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.102+0000 7f2747f1d640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-09-15T01:34:25.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.102+0000 7f2747f1d640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-09-15T01:34:25.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.102+0000 7f2747f1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:25.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.102+0000 7f2747f1d640 1 -- 172.21.15.204:0/3742281555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"} v 0) v1 -- 0x7f2740071100 con 0x7f27400a2c60 2024-09-15T01:34:25.551 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:34:25.552 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:34:25.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.563+0000 7f2735ffb640 1 -- 172.21.15.204:0/3742281555 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}]=0 removed item id -11 name 'host1' from crush map v79) v1 ==== 147+0+0 (secure 0 0 0) 0x7f273801a030 con 0x7f27400a2c60 2024-09-15T01:34:25.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.563+0000 7f2735ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-09-15T01:34:25.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.563+0000 7f2735ffb640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host1' from crush map 2024-09-15T01:34:25.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.652+0000 7f261611d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:25.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.652+0000 7f261611d640 10 monclient: build_initial_monmap 2024-09-15T01:34:25.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.652+0000 7f261611d640 10 monclient: monmap: 2024-09-15T01:34:25.654 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:25.654 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:25.653491+0000 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:25.653491+0000 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:25.655 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:25.656 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:25.656 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:25.656 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:25.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 1 Processor -- start 2024-09-15T01:34:25.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 1 -- start start 2024-09-15T01:34:25.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 10 monclient: init 2024-09-15T01:34:25.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:25.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 10 monclient: _add_conns ranks=[3,4,7,10,8,5,6,1,0,2,9] 2024-09-15T01:34:25.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26100a2930 0x7f2610132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 10 monclient(hunting): picked mon.noname-g con 0x7f26100a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:25.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 0x7f261009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 10 monclient(hunting): picked mon.noname-c con 0x7f261009faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f26100a08c0 0x7f26100a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 10 monclient(hunting): picked mon.noname-i con 0x7f26100a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261009e0f0 con 0x7f26100a2930 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f261009dd80 con 0x7f261009faa0 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.653+0000 7f261611d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f26100a2d00 con 0x7f26100a08c0 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f261611d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f261611d640 10 monclient(hunting): authenticate will time out at 870.404017s 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 0x7f261009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 0x7f261009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:33172/0 (socket says 172.21.15.178:33172) 2024-09-15T01:34:25.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 1 -- 172.21.15.178:0/3647936020 learned_addr learned my addr 172.21.15.178:0/3647936020 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:25.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 10 monclient(hunting): get_auth_request con 0x7f261009faa0 auth_method 0 2024-09-15T01:34:25.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:25.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:25.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:25.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260ffff640 1 --2- 172.21.15.178:0/3647936020 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f26100a08c0 0x7f26100a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260f7fe640 1 --2- 172.21.15.178:0/3647936020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26100a2930 0x7f2610132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260ffff640 10 monclient(hunting): get_auth_request con 0x7f26100a08c0 auth_method 0 2024-09-15T01:34:25.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:25.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:25.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:25.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260f7fe640 10 monclient(hunting): get_auth_request con 0x7f26100a2930 auth_method 0 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 10 monclient(hunting): handle_auth_done global_id 5062 payload 274 2024-09-15T01:34:25.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 1 -- 172.21.15.178:0/3647936020 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f26100a08c0 msgr2=0x7f26100a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 1 --2- 172.21.15.178:0/3647936020 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f26100a08c0 0x7f26100a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 1 -- 172.21.15.178:0/3647936020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26100a2930 msgr2=0x7f2610132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 1 --2- 172.21.15.178:0/3647936020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26100a2930 0x7f2610132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 1 monclient: found mon.noname-c 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260effd640 1 -- 172.21.15.178:0/3647936020 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261004bf60 con 0x7f261009faa0 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.654+0000 7f260ffff640 1 --2- 172.21.15.178:0/3647936020 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f26100a08c0 0x7f26100a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260effd640 1 --2- 172.21.15.178:0/3647936020 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 0x7f261009fe70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2604009940 tx=0x7f26040091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=59e8e58c2acb67f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260e7fc640 1 -- 172.21.15.178:0/3647936020 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f260402ac60 con 0x7f261009faa0 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:25.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260e7fc640 10 monclient: dump: 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:25.669 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260e7fc640 1 -- 172.21.15.178:0/3647936020 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260402ade0 con 0x7f261009faa0 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260e7fc640 1 -- 172.21.15.178:0/3647936020 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2604031ca0 con 0x7f261009faa0 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f261611d640 5 monclient: authenticate success, global_id 5062 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f261611d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f261611d640 4 set_mon_vals no callback set 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:25.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f260e7fc640 10 monclient: dump: 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:25.671 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f261611d640 10 monclient: shutdown 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f261611d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f261611d640 1 -- 172.21.15.178:0/3647936020 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 msgr2=0x7f261009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f261611d640 1 --2- 172.21.15.178:0/3647936020 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 0x7f261009fe70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2604009940 tx=0x7f26040091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f261611d640 1 -- 172.21.15.178:0/3647936020 shutdown_connections 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f261611d640 1 --2- 172.21.15.178:0/3647936020 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f26100a08c0 0x7f26100a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f261611d640 1 --2- 172.21.15.178:0/3647936020 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 0x7f261009fe70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f261611d640 1 --2- 172.21.15.178:0/3647936020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26100a2930 0x7f2610132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.655+0000 7f261611d640 1 -- 172.21.15.178:0/3647936020 >> 172.21.15.178:0/3647936020 conn(0x7f26100a5ed0 msgr2=0x7f26100a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:25.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.656+0000 7f261611d640 1 -- 172.21.15.178:0/3647936020 shutdown_connections 2024-09-15T01:34:25.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.656+0000 7f261611d640 1 -- 172.21.15.178:0/3647936020 wait complete. 2024-09-15T01:34:25.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.656+0000 7f261611d640 10 monclient: build_initial_monmap 2024-09-15T01:34:25.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.656+0000 7f261611d640 10 monclient: monmap: 2024-09-15T01:34:25.673 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:25.673 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:25.673 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:25.673 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:25.673 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:25.673 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:25.673 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:25.673 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:25.673 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.656+0000 7f261611d640 1 Processor -- start 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 1 -- start start 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 10 monclient: init 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 10 monclient: _add_conns ranks=[4,3,7,0,2,5,8,1,6] 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 0x7f261009b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 10 monclient(hunting): picked mon.noname-c con 0x7f261009faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26100a08c0 0x7f2610131870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 10 monclient(hunting): picked mon.noname-g con 0x7f26100a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f26100a2930 0x7f261011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 10 monclient(hunting): picked mon.noname-i con 0x7f26100a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261009be30 con 0x7f26100a08c0 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2610142ca0 con 0x7f261009faa0 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f261011ec70 con 0x7f26100a2930 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f261611d640 10 monclient(hunting): authenticate will time out at 870.407472s 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 0x7f261009b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 0x7f261009b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:33188/0 (socket says 172.21.15.178:33188) 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 1 -- 172.21.15.178:0/3638104876 learned_addr learned my addr 172.21.15.178:0/3638104876 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 10 monclient(hunting): get_auth_request con 0x7f261009faa0 auth_method 0 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260effd640 1 --2- 172.21.15.178:0/3638104876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26100a08c0 0x7f2610131870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260effd640 10 monclient(hunting): get_auth_request con 0x7f26100a08c0 auth_method 0 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260ffff640 1 --2- 172.21.15.178:0/3638104876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f26100a2930 0x7f261011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260ffff640 10 monclient(hunting): get_auth_request con 0x7f26100a2930 auth_method 0 2024-09-15T01:34:25.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:25.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:25.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:25.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:25.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:25.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:25.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:25.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:25.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:25.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:25.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:25.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:25.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 10 monclient(hunting): handle_auth_done global_id 5071 payload 824 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 1 -- 172.21.15.178:0/3638104876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f26100a2930 msgr2=0x7f261011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 1 --2- 172.21.15.178:0/3638104876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f26100a2930 0x7f261011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 1 -- 172.21.15.178:0/3638104876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26100a08c0 msgr2=0x7f2610131870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 1 --2- 172.21.15.178:0/3638104876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26100a08c0 0x7f2610131870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 1 monclient: found mon.noname-c 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.657+0000 7f260f7fe640 1 -- 172.21.15.178:0/3638104876 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2608009390 con 0x7f261009faa0 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f260f7fe640 1 --2- 172.21.15.178:0/3638104876 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 0x7f261009b300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2600009f20 tx=0x7f260000b040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f25f77fe640 1 -- 172.21.15.178:0/3638104876 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f260000ce10 con 0x7f261009faa0 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f25f77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f25f77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:25.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f25f77fe640 10 monclient: dump: 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:25.679 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:25.680 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:25.680 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:25.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f25f77fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:25.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f25f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:25.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f25f77fe640 1 -- 172.21.15.178:0/3638104876 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260000f040 con 0x7f261009faa0 2024-09-15T01:34:25.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f261611d640 5 monclient: authenticate success, global_id 5071 2024-09-15T01:34:25.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f25f77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:25.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f25f77fe640 1 -- 172.21.15.178:0/3638104876 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2600014d80 con 0x7f261009faa0 2024-09-15T01:34:25.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f261611d640 10 monclient: _renew_subs 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f261611d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f261611d640 1 -- 172.21.15.178:0/3638104876 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2604009450 con 0x7f261009faa0 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f25f77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f25f77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f260dffb640 4 set_mon_vals no callback set 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f25f77fe640 10 monclient: dump: 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:25.681 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f261611d640 10 monclient: _renew_subs 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f261611d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.658+0000 7f261611d640 1 -- 172.21.15.178:0/3638104876 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2610137230 con 0x7f261009faa0 2024-09-15T01:34:25.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.659+0000 7f25f77fe640 1 -- 172.21.15.178:0/3638104876 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2600003020 con 0x7f261009faa0 2024-09-15T01:34:25.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.659+0000 7f25f77fe640 1 --2- 172.21.15.178:0/3638104876 >> v2:172.21.15.204:6812/28210 conn(0x7f25fc0397b0 0x7f25fc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.659+0000 7f25f77fe640 1 -- 172.21.15.178:0/3638104876 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5333+0+0 (secure 0 0 0) 0x7f260004bf30 con 0x7f261009faa0 2024-09-15T01:34:25.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.659+0000 7f261611d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:25.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.659+0000 7f261611d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:25.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.659+0000 7f261611d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:25.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.659+0000 7f260effd640 1 --2- 172.21.15.178:0/3638104876 >> v2:172.21.15.204:6812/28210 conn(0x7f25fc0397b0 0x7f25fc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.659+0000 7f261611d640 1 -- 172.21.15.178:0/3638104876 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261009d6a0 con 0x7f261009faa0 2024-09-15T01:34:25.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.659+0000 7f260effd640 10 monclient: get_auth_request con 0x7f25fc0397b0 auth_method 0 2024-09-15T01:34:25.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.659+0000 7f260effd640 1 --2- 172.21.15.178:0/3638104876 >> v2:172.21.15.204:6812/28210 conn(0x7f25fc0397b0 0x7f25fc03bc60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2604034000 tx=0x7f2604032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:25.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.664+0000 7f25f77fe640 1 -- 172.21.15.178:0/3638104876 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f261009d6a0 con 0x7f261009faa0 2024-09-15T01:34:25.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.664+0000 7f25f77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:25.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.664+0000 7f25f77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:25.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.664+0000 7f2747f1d640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-09-15T01:34:25.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.664+0000 7f2747f1d640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-09-15T01:34:25.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.664+0000 7f2747f1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.664+0000 7f2747f1d640 1 -- 172.21.15.204:0/3742281555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"} v 0) v1 -- 0x7f274009c820 con 0x7f27400a2c60 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.664+0000 7f2735ffb640 1 -- 172.21.15.204:0/3742281555 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}]=0 item -11 does not appear in that position v79) v1 ==== 141+0+0 (secure 0 0 0) 0x7f273801a020 con 0x7f27400a2c60 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.664+0000 7f2735ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.664+0000 7f2735ffb640 10 monclient: _finish_command 3 = system:0 item -11 does not appear in that position 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:item -11 does not appear in that position 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.666+0000 7f2747f1d640 1 -- 172.21.15.204:0/3742281555 >> v2:172.21.15.204:6812/28210 conn(0x7f27240397b0 msgr2=0x7f272403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.666+0000 7f2747f1d640 1 --2- 172.21.15.204:0/3742281555 >> v2:172.21.15.204:6812/28210 conn(0x7f27240397b0 0x7f272403bc60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f273c009420 tx=0x7f273c032000 comp rx=0 tx=0).stop 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.666+0000 7f2747f1d640 10 monclient: shutdown 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.666+0000 7f2747f1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.666+0000 7f2747f1d640 1 -- 172.21.15.204:0/3742281555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27400a2c60 msgr2=0x7f274011eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.666+0000 7f2747f1d640 1 --2- 172.21.15.204:0/3742281555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27400a2c60 0x7f274011eaa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f273800cf30 tx=0x7f2738007840 comp rx=0 tx=0).stop 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.666+0000 7f2747f1d640 1 -- 172.21.15.204:0/3742281555 shutdown_connections 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.666+0000 7f2747f1d640 1 --2- 172.21.15.204:0/3742281555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27400a2c60 0x7f274011eaa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.666+0000 7f2747f1d640 1 --2- 172.21.15.204:0/3742281555 >> v2:172.21.15.204:6812/28210 conn(0x7f27240397b0 0x7f272403bc60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.666+0000 7f2747f1d640 1 --2- 172.21.15.204:0/3742281555 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f27400a0ab0 0x7f274011e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.666+0000 7f2747f1d640 1 --2- 172.21.15.204:0/3742281555 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f274009fc60 0x7f2740131a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.666+0000 7f2747f1d640 1 -- 172.21.15.204:0/3742281555 >> 172.21.15.204:0/3742281555 conn(0x7f27400a60c0 msgr2=0x7f27400a42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.667+0000 7f2747f1d640 1 -- 172.21.15.204:0/3742281555 shutdown_connections 2024-09-15T01:34:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.667+0000 7f2747f1d640 1 -- 172.21.15.204:0/3742281555 wait complete. 2024-09-15T01:34:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:34:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -c host1 2024-09-15T01:34:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -q 1 2024-09-15T01:34:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient: get_monmap_and_config 2024-09-15T01:34:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient: build_initial_monmap 2024-09-15T01:34:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient: monmap: 2024-09-15T01:34:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:25.749222+0000 2024-09-15T01:34:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:25.749222+0000 2024-09-15T01:34:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:25.751 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 1 Processor -- start 2024-09-15T01:34:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 1 -- start start 2024-09-15T01:34:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient: init 2024-09-15T01:34:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient: _add_conns ranks=[7,3,4,2,0,6,1,10,5,9,8] 2024-09-15T01:34:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 0x7fea840a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient(hunting): picked mon.noname-i con 0x7fea840a2400 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea8409f990 0x7fea8409fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient(hunting): picked mon.noname-g con 0x7fea8409f990 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea84132930 0x7fea84132d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient(hunting): picked mon.noname-c con 0x7fea84132930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8409e1a0 con 0x7fea8409f990 2024-09-15T01:34:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea840a27d0 con 0x7fea84132930 2024-09-15T01:34:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea8409dec0 con 0x7fea840a2400 2024-09-15T01:34:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.748+0000 7fea88e99640 10 monclient(hunting): authenticate will time out at 881.391961s 2024-09-15T01:34:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea8409f990 0x7fea8409fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea8259c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 0x7fea840a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea8409f990 0x7fea8409fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54580/0 (socket says 172.21.15.204:54580) 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 1 -- 172.21.15.204:0/3022520356 learned_addr learned my addr 172.21.15.204:0/3022520356 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 10 monclient(hunting): get_auth_request con 0x7fea8409f990 auth_method 0 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea8259c640 10 monclient(hunting): get_auth_request con 0x7fea840a2400 auth_method 0 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea8259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea8259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea8259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea82d9d640 1 --2- 172.21.15.204:0/3022520356 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea84132930 0x7fea84132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea82d9d640 10 monclient(hunting): get_auth_request con 0x7fea84132930 auth_method 0 2024-09-15T01:34:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea82d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea82d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea82d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea8259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea8259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea8259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea82d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea82d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea82d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 10 monclient(hunting): handle_auth_done global_id 14205 payload 274 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 1 -- 172.21.15.204:0/3022520356 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 msgr2=0x7fea840a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 1 --2- 172.21.15.204:0/3022520356 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 0x7fea840a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 1 -- 172.21.15.204:0/3022520356 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea84132930 msgr2=0x7fea84132d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:25.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 1 --2- 172.21.15.204:0/3022520356 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea84132930 0x7fea84132d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:25.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 1 monclient: found mon.noname-g 2024-09-15T01:34:25.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:25.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:25.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea7bfff640 1 -- 172.21.15.204:0/3022520356 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea840648a0 con 0x7fea8409f990 2024-09-15T01:34:25.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea82d9d640 1 --2- 172.21.15.204:0/3022520356 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea84132930 0x7fea84132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:25.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.749+0000 7fea8259c640 1 --2- 172.21.15.204:0/3022520356 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 0x7fea840a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:25.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea7bfff640 1 --2- 172.21.15.204:0/3022520356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea8409f990 0x7fea8409fd60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009940 tx=0x7fea6c0091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=dbf2c2174d819365 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:25.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea81d9b640 1 -- 172.21.15.204:0/3022520356 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea6c02ac60 con 0x7fea8409f990 2024-09-15T01:34:25.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea81d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea81d9b640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea81d9b640 10 monclient: dump: 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:25.769 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea81d9b640 10 monclient: _finish_auth 0 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea81d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea81d9b640 1 -- 172.21.15.204:0/3022520356 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c02ade0 con 0x7fea8409f990 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 5 monclient: authenticate success, global_id 14205 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea81d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea81d9b640 1 -- 172.21.15.204:0/3022520356 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea6c031ca0 con 0x7fea8409f990 2024-09-15T01:34:25.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 4 set_mon_vals no callback set 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea81d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea81d9b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea81d9b640 10 monclient: dump: 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:25.771 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 10 monclient: shutdown 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 1 -- 172.21.15.204:0/3022520356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea8409f990 msgr2=0x7fea8409fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 1 --2- 172.21.15.204:0/3022520356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea8409f990 0x7fea8409fd60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009940 tx=0x7fea6c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 1 -- 172.21.15.204:0/3022520356 shutdown_connections 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 1 --2- 172.21.15.204:0/3022520356 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea84132930 0x7fea84132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 1 --2- 172.21.15.204:0/3022520356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea8409f990 0x7fea8409fd60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 1 --2- 172.21.15.204:0/3022520356 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 0x7fea840a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.750+0000 7fea88e99640 1 -- 172.21.15.204:0/3022520356 >> 172.21.15.204:0/3022520356 conn(0x7fea8406e860 msgr2=0x7fea8406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.751+0000 7fea88e99640 1 -- 172.21.15.204:0/3022520356 shutdown_connections 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.751+0000 7fea88e99640 1 -- 172.21.15.204:0/3022520356 wait complete. 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.751+0000 7fea88e99640 10 monclient: build_initial_monmap 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.751+0000 7fea88e99640 10 monclient: monmap: 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:25.773 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.751+0000 7fea88e99640 1 Processor -- start 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.751+0000 7fea88e99640 1 -- start start 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.751+0000 7fea88e99640 10 monclient: init 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:25.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 10 monclient: _add_conns ranks=[6,7,4,0,5,1,3,8,2] 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea8409f990 0x7fea84141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 10 monclient(hunting): picked mon.noname-d con 0x7fea8409f990 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 0x7fea84142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 10 monclient(hunting): picked mon.noname-i con 0x7fea840a2400 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea84132930 0x7fea841428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 10 monclient(hunting): picked mon.noname-c con 0x7fea84132930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea84120780 con 0x7fea84132930 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea8411f8c0 con 0x7fea8409f990 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea8409d3d0 con 0x7fea840a2400 2024-09-15T01:34:25.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea88e99640 10 monclient(hunting): authenticate will time out at 881.395328s 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 0x7fea84142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 0x7fea84142370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:34138/0 (socket says 172.21.15.204:34138) 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 1 -- 172.21.15.204:0/2463570212 learned_addr learned my addr 172.21.15.204:0/2463570212 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 10 monclient(hunting): get_auth_request con 0x7fea840a2400 auth_method 0 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea8259c640 1 --2- 172.21.15.204:0/2463570212 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea8409f990 0x7fea84141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea82d9d640 1 --2- 172.21.15.204:0/2463570212 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea84132930 0x7fea841428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea8259c640 10 monclient(hunting): get_auth_request con 0x7fea8409f990 auth_method 0 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea8259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea8259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea8259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea82d9d640 10 monclient(hunting): get_auth_request con 0x7fea84132930 auth_method 0 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea82d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea82d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea82d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea8259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea8259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea8259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea82d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea82d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:25.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea82d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 10 monclient(hunting): handle_auth_done global_id 5209 payload 824 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 1 -- 172.21.15.204:0/2463570212 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea8409f990 msgr2=0x7fea84141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 1 --2- 172.21.15.204:0/2463570212 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea8409f990 0x7fea84141e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 1 -- 172.21.15.204:0/2463570212 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea84132930 msgr2=0x7fea841428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 1 --2- 172.21.15.204:0/2463570212 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea84132930 0x7fea841428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 1 monclient: found mon.noname-i 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 1 -- 172.21.15.204:0/2463570212 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea74009390 con 0x7fea840a2400 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.752+0000 7fea7bfff640 1 --2- 172.21.15.204:0/2463570212 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 0x7fea84142370 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fea6c000c00 tx=0x7fea6c032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 1 -- 172.21.15.204:0/2463570212 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea6c031820 con 0x7fea840a2400 2024-09-15T01:34:25.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 10 monclient: dump: 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:25.779 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 1 -- 172.21.15.204:0/2463570212 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c0319e0 con 0x7fea840a2400 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea88e99640 5 monclient: authenticate success, global_id 5209 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea88e99640 10 monclient: _renew_subs 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea88e99640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea88e99640 1 -- 172.21.15.204:0/2463570212 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7c009a20 con 0x7fea840a2400 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:25.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 1 -- 172.21.15.204:0/2463570212 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea6c03a390 con 0x7fea840a2400 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 10 monclient: dump: 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:25.781 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea8159a640 4 set_mon_vals no callback set 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea88e99640 10 monclient: _renew_subs 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea88e99640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea88e99640 1 -- 172.21.15.204:0/2463570212 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6c009450 con 0x7fea840a2400 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.753+0000 7fea7a7fc640 1 -- 172.21.15.204:0/2463570212 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fea6c039040 con 0x7fea840a2400 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.754+0000 7fea88e99640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:25.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.754+0000 7fea88e99640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:25.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.754+0000 7fea88e99640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:34:25.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.754+0000 7fea88e99640 1 -- 172.21.15.204:0/2463570212 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea84070b80 con 0x7fea840a2400 2024-09-15T01:34:25.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.754+0000 7fea7a7fc640 1 --2- 172.21.15.204:0/2463570212 >> v2:172.21.15.204:6812/28210 conn(0x7fea640397e0 0x7fea6403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:25.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.754+0000 7fea7a7fc640 1 -- 172.21.15.204:0/2463570212 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5333+0+0 (secure 0 0 0) 0x7fea6c06ab20 con 0x7fea840a2400 2024-09-15T01:34:25.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.754+0000 7fea8259c640 1 --2- 172.21.15.204:0/2463570212 >> v2:172.21.15.204:6812/28210 conn(0x7fea640397e0 0x7fea6403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:25.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.754+0000 7fea8259c640 10 monclient: get_auth_request con 0x7fea640397e0 auth_method 0 2024-09-15T01:34:25.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.754+0000 7fea8259c640 1 --2- 172.21.15.204:0/2463570212 >> v2:172.21.15.204:6812/28210 conn(0x7fea640397e0 0x7fea6403bc90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fea74009eb0 tx=0x7fea740090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:25.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.758+0000 7fea7a7fc640 1 -- 172.21.15.204:0/2463570212 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fea6c03a510 con 0x7fea840a2400 2024-09-15T01:34:25.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.758+0000 7fea7a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:25.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.758+0000 7fea7a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:25.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.881+0000 7f261611d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:34:25.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.881+0000 7f261611d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:34:25.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.881+0000 7f261611d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:25.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.881+0000 7f261611d640 1 -- 172.21.15.178:0/3638104876 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f261011f070 con 0x7f261009faa0 2024-09-15T01:34:25.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.882+0000 7f25f77fe640 1 -- 172.21.15.178:0/3638104876 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7f2600041080 con 0x7f261009faa0 2024-09-15T01:34:25.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.882+0000 7f25f77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:34:25.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.882+0000 7f25f77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:25.884 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":38,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:34:25.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 1 -- 172.21.15.178:0/3638104876 >> v2:172.21.15.204:6812/28210 conn(0x7f25fc0397b0 msgr2=0x7f25fc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:25.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 1 --2- 172.21.15.178:0/3638104876 >> v2:172.21.15.204:6812/28210 conn(0x7f25fc0397b0 0x7f25fc03bc60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2604034000 tx=0x7f2604032000 comp rx=0 tx=0).stop 2024-09-15T01:34:25.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 10 monclient: shutdown 2024-09-15T01:34:25.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:25.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 1 -- 172.21.15.178:0/3638104876 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 msgr2=0x7f261009b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:25.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 1 --2- 172.21.15.178:0/3638104876 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 0x7f261009b300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2600009f20 tx=0x7f260000b040 comp rx=0 tx=0).stop 2024-09-15T01:34:25.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 1 -- 172.21.15.178:0/3638104876 shutdown_connections 2024-09-15T01:34:25.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 1 --2- 172.21.15.178:0/3638104876 >> v2:172.21.15.204:6812/28210 conn(0x7f25fc0397b0 0x7f25fc03bc60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 1 --2- 172.21.15.178:0/3638104876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f26100a2930 0x7f261011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 1 --2- 172.21.15.178:0/3638104876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26100a08c0 0x7f2610131870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 1 --2- 172.21.15.178:0/3638104876 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f261009faa0 0x7f261009b300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:25.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 1 -- 172.21.15.178:0/3638104876 >> 172.21.15.178:0/3638104876 conn(0x7f26100a5ed0 msgr2=0x7f261009e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:25.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 1 -- 172.21.15.178:0/3638104876 shutdown_connections 2024-09-15T01:34:25.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.884+0000 7f261611d640 1 -- 172.21.15.178:0/3638104876 wait complete. 2024-09-15T01:34:25.896 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:34:25.896 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T01:34:25.896 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:34:25.896 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:34:25.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.915+0000 7fea88e99640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:25.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.915+0000 7fea88e99640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:25.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.915+0000 7fea88e99640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:34:25.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.915+0000 7fea88e99640 1 -- 172.21.15.204:0/2463570212 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7fea840667d0 con 0x7fea840a2400 2024-09-15T01:34:25.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.915+0000 7fea7a7fc640 1 -- 172.21.15.204:0/2463570212 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v79) v1 ==== 56+0+1340 (secure 0 0 0) 0x7fea6c041070 con 0x7fea840a2400 2024-09-15T01:34:25.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.915+0000 7fea7a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:25.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:25.915+0000 7fea7a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:25.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.987+0000 7f515871d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:25.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.987+0000 7f515871d640 10 monclient: build_initial_monmap 2024-09-15T01:34:25.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.987+0000 7f515871d640 10 monclient: monmap: 2024-09-15T01:34:25.990 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:25.990 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:25.990 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:25.988676+0000 2024-09-15T01:34:25.990 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:25.988676+0000 2024-09-15T01:34:25.990 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:25.990 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.988+0000 7f515871d640 1 Processor -- start 2024-09-15T01:34:25.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.988+0000 7f515871d640 1 -- start start 2024-09-15T01:34:25.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.988+0000 7f515871d640 10 monclient: init 2024-09-15T01:34:26.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 10 monclient: _add_conns ranks=[5,0,6,9,4,8,2,3,1,10,7] 2024-09-15T01:34:26.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f51500a2930 0x7f5150132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 10 monclient(hunting): picked mon.noname-h con 0x7f51500a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:26.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f515009faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:26.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009fe70 0x7f51500a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 10 monclient(hunting): picked mon.noname-d con 0x7f515009fe70 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:26.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515004bf60 con 0x7f515009faa0 2024-09-15T01:34:26.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f515009dd80 con 0x7f51500a2930 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f51500a2d00 con 0x7f515009fe70 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f515871d640 10 monclient(hunting): authenticate will time out at 870.739087s 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009fe70 0x7f51500a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5155cb8640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f515009faa0 0x7f51500a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:34406/0 (socket says 172.21.15.178:34406) 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5155cb8640 1 -- 172.21.15.178:0/1738259333 learned_addr learned my addr 172.21.15.178:0/1738259333 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009fe70 0x7f51500a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42704/0 (socket says 172.21.15.178:42704) 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 10 monclient(hunting): get_auth_request con 0x7f515009fe70 auth_method 0 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f51564b9640 1 --2- 172.21.15.178:0/1738259333 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f51500a2930 0x7f5150132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f51564b9640 10 monclient(hunting): get_auth_request con 0x7f51500a2930 auth_method 0 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f51564b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f51564b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f51564b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f51564b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f51564b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f51564b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 10 monclient(hunting): handle_auth_done global_id 14190 payload 274 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 1 -- 172.21.15.178:0/1738259333 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f51500a2930 msgr2=0x7f5150132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f51554b7640 1 -- 172.21.15.178:0/1738259333 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416945219 0 0) 0x7f515004bf60 con 0x7f515009faa0 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 1 --2- 172.21.15.178:0/1738259333 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f51500a2930 0x7f5150132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 1 -- 172.21.15.178:0/1738259333 >> v1:172.21.15.178:6789/0 conn(0x7f515009faa0 legacy=0x7f51500a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 1 monclient: found mon.noname-d 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f5156cba640 1 -- 172.21.15.178:0/1738259333 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51500a0310 con 0x7f515009fe70 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.989+0000 7f51554b7640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f5156cba640 1 --2- 172.21.15.178:0/1738259333 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009fe70 0x7f51500a0fd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5148009e90 tx=0x7f5148009780 comp rx=0 tx=0).ready entity=mon.6 client_cookie=33ea70a3f96933f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f51554b7640 1 -- 172.21.15.178:0/1738259333 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5148031880 con 0x7f515009fe70 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f51554b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f51554b7640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:34:26.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f51554b7640 10 monclient: dump: 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.004 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f51554b7640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f51554b7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f51554b7640 1 -- 172.21.15.178:0/1738259333 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514802d050 con 0x7f515009fe70 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f515871d640 5 monclient: authenticate success, global_id 14190 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f515871d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f51554b7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f515871d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f515871d640 4 set_mon_vals no callback set 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f51554b7640 1 -- 172.21.15.178:0/1738259333 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5148031400 con 0x7f515009fe70 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f51554b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f51554b7640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:26.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f51554b7640 10 monclient: dump: 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.006 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f515871d640 10 monclient: shutdown 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f515871d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f515871d640 1 -- 172.21.15.178:0/1738259333 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009fe70 msgr2=0x7f51500a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f515871d640 1 --2- 172.21.15.178:0/1738259333 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009fe70 0x7f51500a0fd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5148009e90 tx=0x7f5148009780 comp rx=0 tx=0).stop 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f515871d640 1 -- 172.21.15.178:0/1738259333 shutdown_connections 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f515871d640 1 --2- 172.21.15.178:0/1738259333 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009fe70 0x7f51500a0fd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f515871d640 1 --2- 172.21.15.178:0/1738259333 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f51500a2930 0x7f5150132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.990+0000 7f515871d640 1 -- 172.21.15.178:0/1738259333 >> 172.21.15.178:0/1738259333 conn(0x7f51500a5ed0 msgr2=0x7f51500a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.991+0000 7f515871d640 1 -- 172.21.15.178:0/1738259333 shutdown_connections 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.991+0000 7f515871d640 1 -- 172.21.15.178:0/1738259333 wait complete. 2024-09-15T01:34:26.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.991+0000 7f515871d640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.991+0000 7f515871d640 10 monclient: monmap: 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.008 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.991+0000 7f515871d640 1 Processor -- start 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.991+0000 7f515871d640 1 -- start start 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.991+0000 7f515871d640 10 monclient: init 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient: _add_conns ranks=[6,5,3,2,4,8,7,0,1] 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009faa0 0x7f515009af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient(hunting): picked mon.noname-d con 0x7f515009faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f515009fe70 0x7f5150131730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient(hunting): picked mon.noname-h con 0x7f515009fe70 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:26.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51500a2930 0x7f515011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient(hunting): picked mon.noname-g con 0x7f51500a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51500a2390 con 0x7f51500a2930 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f515009d630 con 0x7f515009fe70 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5150142a10 con 0x7f515009faa0 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient(hunting): authenticate will time out at 870.742252s 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009faa0 0x7f515009af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009faa0 0x7f515009af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42720/0 (socket says 172.21.15.178:42720) 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 1 -- 172.21.15.178:0/3847247113 learned_addr learned my addr 172.21.15.178:0/3847247113 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:26.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 10 monclient(hunting): get_auth_request con 0x7f515009faa0 auth_method 0 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f5155cb8640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f515009fe70 0x7f5150131730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f5156cba640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51500a2930 0x7f515011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f5156cba640 10 monclient(hunting): get_auth_request con 0x7f51500a2930 auth_method 0 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f5156cba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f5156cba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f5156cba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f5155cb8640 10 monclient(hunting): get_auth_request con 0x7f515009fe70 auth_method 0 2024-09-15T01:34:26.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f5155cb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f5155cb8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f5155cb8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 10 monclient(hunting): handle_auth_done global_id 14199 payload 824 2024-09-15T01:34:26.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 1 -- 172.21.15.178:0/3847247113 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f515009fe70 msgr2=0x7f5150131730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f515009fe70 0x7f5150131730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 1 -- 172.21.15.178:0/3847247113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51500a2930 msgr2=0x7f515011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51500a2930 0x7f515011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 1 monclient: found mon.noname-d 2024-09-15T01:34:26.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 1 -- 172.21.15.178:0/3847247113 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5148009a20 con 0x7f515009faa0 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f5156cba640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51500a2930 0x7f515011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51564b9640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009faa0 0x7f515009af50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f514c00ea50 tx=0x7f514c0090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f514c00ce90 con 0x7f515009faa0 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51467fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51467fc640 10 monclient: dump: 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.013 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51467fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 5 monclient: authenticate success, global_id 14199 2024-09-15T01:34:26.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c013470 con 0x7f515009faa0 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51467fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f514c013a60 con 0x7f515009faa0 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient: _renew_subs 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 1 -- 172.21.15.178:0/3847247113 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515011eb30 con 0x7f515009faa0 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f5154cb6640 4 set_mon_vals no callback set 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51467fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f51467fc640 10 monclient: dump: 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.015 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient: _renew_subs 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:34:26.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.992+0000 7f515871d640 1 -- 172.21.15.178:0/3847247113 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515011e850 con 0x7f515009faa0 2024-09-15T01:34:26.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f515871d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f515871d640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150136fd0 0x7f515011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f515871d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f515871d640 1 -- 172.21.15.178:0/3847247113 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515004c4a0 con 0x7f5150136fd0 2024-09-15T01:34:26.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f5155cb8640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150136fd0 0x7f515011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f514c013bc0 con 0x7f515009faa0 2024-09-15T01:34:26.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f5155cb8640 10 monclient: get_auth_request con 0x7f5150136fd0 auth_method 0 2024-09-15T01:34:26.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f5155cb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f5155cb8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f5155cb8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f5155cb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f5155cb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.993+0000 7f5155cb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 1 --2- 172.21.15.178:0/3847247113 >> v2:172.21.15.204:6812/28210 conn(0x7f51340397e0 0x7f513403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f5155cb8640 10 monclient(hunting): handle_auth_done global_id 5274 payload 824 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5333+0+0 (secure 0 0 0) 0x7f514c04bab0 con 0x7f515009faa0 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f5155cb8640 1 -- 172.21.15.178:0/3847247113 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f515005ed70 con 0x7f5150136fd0 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f5155cb8640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150136fd0 0x7f515011ca70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f51500a6ec0 tx=0x7f5140009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f5156cba640 1 --2- 172.21.15.178:0/3847247113 >> v2:172.21.15.204:6812/28210 conn(0x7f51340397e0 0x7f513403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f5156cba640 10 monclient: get_auth_request con 0x7f51340397e0 auth_method 0 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5140005810 con 0x7f5150136fd0 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 10 monclient: dump: 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.018 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.019 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f51400072e0 con 0x7f5150136fd0 2024-09-15T01:34:26.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:26.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150136fd0 msgr2=0x7f515011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150136fd0 0x7f515011ca70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f51500a6ec0 tx=0x7f5140009040 comp rx=0 tx=0).stop 2024-09-15T01:34:26.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150136fd0 msgr2=0x7f515011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f51467fc640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150136fd0 0x7f515011ca70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:25.994+0000 7f5156cba640 1 --2- 172.21.15.178:0/3847247113 >> v2:172.21.15.204:6812/28210 conn(0x7f51340397e0 0x7f513403bc90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5148034000 tx=0x7f5148037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.008+0000 7fea88e99640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:26.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.008+0000 7fea88e99640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:26.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.008+0000 7fea88e99640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:34:26.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.008+0000 7fea88e99640 1 -- 172.21.15.204:0/2463570212 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7fea84121ad0 con 0x7fea840a2400 2024-09-15T01:34:26.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.009+0000 7fea7a7fc640 1 -- 172.21.15.204:0/2463570212 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v79) v1 ==== 56+0+1340 (secure 0 0 0) 0x7fea6c037020 con 0x7fea840a2400 2024-09-15T01:34:26.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.009+0000 7fea7a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:26.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.009+0000 7fea7a7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 1 -- 172.21.15.204:0/2463570212 >> v2:172.21.15.204:6812/28210 conn(0x7fea640397e0 msgr2=0x7fea6403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 1 --2- 172.21.15.204:0/2463570212 >> v2:172.21.15.204:6812/28210 conn(0x7fea640397e0 0x7fea6403bc90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fea74009eb0 tx=0x7fea740090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 10 monclient: shutdown 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 1 -- 172.21.15.204:0/2463570212 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 msgr2=0x7fea84142370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 1 --2- 172.21.15.204:0/2463570212 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 0x7fea84142370 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fea6c000c00 tx=0x7fea6c032000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 1 -- 172.21.15.204:0/2463570212 shutdown_connections 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 1 --2- 172.21.15.204:0/2463570212 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea84132930 0x7fea841428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 1 --2- 172.21.15.204:0/2463570212 >> v2:172.21.15.204:6812/28210 conn(0x7fea640397e0 0x7fea6403bc90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 1 --2- 172.21.15.204:0/2463570212 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea840a2400 0x7fea84142370 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 1 --2- 172.21.15.204:0/2463570212 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea8409f990 0x7fea84141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 1 -- 172.21.15.204:0/2463570212 >> 172.21.15.204:0/2463570212 conn(0x7fea8406e860 msgr2=0x7fea8409fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 1 -- 172.21.15.204:0/2463570212 shutdown_connections 2024-09-15T01:34:26.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.011+0000 7fea88e99640 1 -- 172.21.15.204:0/2463570212 wait complete. 2024-09-15T01:34:26.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f515871d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:26.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f515871d640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150131c70 0x7f515011cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f515871d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f515871d640 1 -- 172.21.15.178:0/3847247113 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515004bc80 con 0x7f5150131c70 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f5155cb8640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150131c70 0x7f515011cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f5155cb8640 10 monclient: get_auth_request con 0x7f5150131c70 auth_method 0 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f5155cb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f5155cb8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f5155cb8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f5155cb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f5155cb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f5155cb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f5155cb8640 10 monclient(hunting): handle_auth_done global_id 5283 payload 824 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f5155cb8640 1 -- 172.21.15.178:0/3847247113 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f515004b720 con 0x7f5150131c70 2024-09-15T01:34:26.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f5155cb8640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150131c70 0x7f515011cfb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5140017000 tx=0x7f514001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5140028880 con 0x7f5150131c70 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f51467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f51467fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.000+0000 7f51467fc640 10 monclient: dump: 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.026 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f5140028400 con 0x7f5150131c70 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f51467fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f51467fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150131c70 msgr2=0x7f515011cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f51467fc640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150131c70 0x7f515011cfb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5140017000 tx=0x7f514001a000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f51467fc640 1 -- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150131c70 msgr2=0x7f515011cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f51467fc640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150131c70 0x7f515011cfb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.028 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:26.028 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:26.028 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:26.028 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:26.028 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:26.028 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:26.028 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:26.028 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:26.028 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.029 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.030 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:26.031 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.032 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.033 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.034 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.036 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.039 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:26.040 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.040 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.040 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.040 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.040 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.040 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.040 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.040 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.041 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.042 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.043 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.044 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.045 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.046 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.047 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.048 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:26.049 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.050 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.050 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:26.050 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.050 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.050 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.050 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:26.050 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.050 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.050 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:26.050 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:26.051 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm osd.7 host1 2024-09-15T01:34:26.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f515871d640 1 -- 172.21.15.178:0/3847247113 >> v2:172.21.15.204:6812/28210 conn(0x7f51340397e0 msgr2=0x7f513403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f515871d640 1 --2- 172.21.15.178:0/3847247113 >> v2:172.21.15.204:6812/28210 conn(0x7f51340397e0 0x7f513403bc90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5148034000 tx=0x7f5148037000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f515871d640 10 monclient: shutdown 2024-09-15T01:34:26.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f515871d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f515871d640 1 -- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009faa0 msgr2=0x7f515009af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f515871d640 1 --2- 172.21.15.178:0/3847247113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f515009faa0 0x7f515009af50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f514c00ea50 tx=0x7f514c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f5156cba640 1 -- 172.21.15.178:0/3847247113 reap_dead start 2024-09-15T01:34:26.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f5156cba640 1 -- 172.21.15.178:0/3847247113 reap_dead start 2024-09-15T01:34:26.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f515871d640 1 -- 172.21.15.178:0/3847247113 shutdown_connections 2024-09-15T01:34:26.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f515871d640 1 -- 172.21.15.178:0/3847247113 >> 172.21.15.178:0/3847247113 conn(0x7f51500a5ed0 msgr2=0x7f515009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f515871d640 1 -- 172.21.15.178:0/3847247113 shutdown_connections 2024-09-15T01:34:26.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.001+0000 7f515871d640 1 -- 172.21.15.178:0/3847247113 wait complete. 2024-09-15T01:34:26.053 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:26.053 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:34:26.053 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:34:26.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.095+0000 7fc6f0aa8640 10 monclient: get_monmap_and_config 2024-09-15T01:34:26.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.095+0000 7fc6f0aa8640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.095+0000 7fc6f0aa8640 10 monclient: monmap: 2024-09-15T01:34:26.097 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:26.097 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.097 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:26.096313+0000 2024-09-15T01:34:26.097 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:26.096313+0000 2024-09-15T01:34:26.097 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:26.098 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:26.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.095+0000 7fc6f0aa8640 1 Processor -- start 2024-09-15T01:34:26.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.095+0000 7fc6f0aa8640 1 -- start start 2024-09-15T01:34:26.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.095+0000 7fc6f0aa8640 10 monclient: init 2024-09-15T01:34:26.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.095+0000 7fc6f0aa8640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 10 monclient: _add_conns ranks=[4,1,10,2,6,7,5,0,8,3,9] 2024-09-15T01:34:26.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc6ec0a2e30 0x7fc6ec132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 10 monclient(hunting): picked mon.noname-c con 0x7fc6ec0a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:26.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc6ec09fb80 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc6ec09ff70 0x7fc6ec0a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 10 monclient(hunting): picked mon.noname-f con 0x7fc6ec09ff70 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ec04b720 con 0x7fc6ec09fb80 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ec09e6c0 con 0x7fc6ec09ff70 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec0a3200 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6f0aa8640 10 monclient(hunting): authenticate will time out at 881.739090s 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc6ec09ff70 0x7fc6ec0a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6e9d9b640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc6ec09fb80 0x7fc6ec0a0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:60708/0 (socket says 172.21.15.204:60708) 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6e9d9b640 1 -- 172.21.15.204:0/3419248799 learned_addr learned my addr 172.21.15.204:0/3419248799 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:34:26.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 10 monclient(hunting): get_auth_request con 0x7fc6ec09ff70 auth_method 0 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ea59c640 1 --2- 172.21.15.204:0/3419248799 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc6ec0a2e30 0x7fc6ec132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ea59c640 10 monclient(hunting): get_auth_request con 0x7fc6ec0a2e30 auth_method 0 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 10 monclient(hunting): handle_auth_done global_id 5635 payload 274 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 1 -- 172.21.15.204:0/3419248799 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc6ec0a2e30 msgr2=0x7fc6ec132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 1 --2- 172.21.15.204:0/3419248799 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc6ec0a2e30 0x7fc6ec132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 1 -- 172.21.15.204:0/3419248799 >> v1:172.21.15.204:6789/0 conn(0x7fc6ec09fb80 legacy=0x7fc6ec0a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 1 monclient: found mon.noname-f 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6e959a640 1 -- 172.21.15.204:0/3419248799 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187880360 0 0) 0x7fc6ec04b720 con 0x7fc6ec09fb80 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6ead9d640 1 -- 172.21.15.204:0/3419248799 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec0a1f10 con 0x7fc6ec09ff70 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.096+0000 7fc6e959a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:26.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6ead9d640 1 --2- 172.21.15.204:0/3419248799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec09ff70 0x7fc6ec0a1150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009e60 tx=0x7fc6e4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdeb768963bb703f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6e959a640 1 -- 172.21.15.204:0/3419248799 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6e40079d0 con 0x7fc6ec09ff70 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6e959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6e959a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6e959a640 10 monclient: dump: 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.110 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6e959a640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6e959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6e959a640 1 -- 172.21.15.204:0/3419248799 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e402d050 con 0x7fc6ec09ff70 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6f0aa8640 5 monclient: authenticate success, global_id 5635 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6f0aa8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6e959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6f0aa8640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:26.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6e959a640 1 -- 172.21.15.204:0/3419248799 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6e4007b90 con 0x7fc6ec09ff70 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6f0aa8640 4 set_mon_vals no callback set 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6e959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6e959a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6e959a640 10 monclient: dump: 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.112 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6f0aa8640 10 monclient: shutdown 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6f0aa8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/3419248799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec09ff70 msgr2=0x7fc6ec0a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.097+0000 7fc6f0aa8640 1 --2- 172.21.15.204:0/3419248799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec09ff70 0x7fc6ec0a1150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009e60 tx=0x7fc6e4009780 comp rx=0 tx=0).stop 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.098+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/3419248799 shutdown_connections 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.098+0000 7fc6f0aa8640 1 --2- 172.21.15.204:0/3419248799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec09ff70 0x7fc6ec0a1150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.098+0000 7fc6f0aa8640 1 --2- 172.21.15.204:0/3419248799 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc6ec0a2e30 0x7fc6ec132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.098+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/3419248799 >> 172.21.15.204:0/3419248799 conn(0x7fc6ec06e860 msgr2=0x7fc6ec06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.098+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/3419248799 shutdown_connections 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.098+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/3419248799 wait complete. 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.098+0000 7fc6f0aa8640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.098+0000 7fc6f0aa8640 10 monclient: monmap: 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.114 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.098+0000 7fc6f0aa8640 1 Processor -- start 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.098+0000 7fc6f0aa8640 1 -- start start 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.098+0000 7fc6f0aa8640 10 monclient: init 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 10 monclient: _add_conns ranks=[6,4,1,7,8,0,5,2,3] 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc6ec09fb80 0x7fc6ec130760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 10 monclient(hunting): picked mon.noname-d con 0x7fc6ec09fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc6ec09ff70 0x7fc6ec11d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 10 monclient(hunting): picked mon.noname-c con 0x7fc6ec09ff70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec0a2e30 0x7fc6ec11d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 10 monclient(hunting): picked mon.noname-f con 0x7fc6ec0a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec1209b0 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec120ee0 con 0x7fc6ec09ff70 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec09ca40 con 0x7fc6ec09fb80 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6f0aa8640 10 monclient(hunting): authenticate will time out at 881.742302s 2024-09-15T01:34:26.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec0a2e30 0x7fc6ec11d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec0a2e30 0x7fc6ec11d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48974/0 (socket says 172.21.15.204:48974) 2024-09-15T01:34:26.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 1 -- 172.21.15.204:0/2981851550 learned_addr learned my addr 172.21.15.204:0/2981851550 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:26.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 10 monclient(hunting): get_auth_request con 0x7fc6ec0a2e30 auth_method 0 2024-09-15T01:34:26.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ea59c640 1 --2- 172.21.15.204:0/2981851550 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc6ec09fb80 0x7fc6ec130760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6e9d9b640 1 --2- 172.21.15.204:0/2981851550 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc6ec09ff70 0x7fc6ec11d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ea59c640 10 monclient(hunting): get_auth_request con 0x7fc6ec09fb80 auth_method 0 2024-09-15T01:34:26.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6e9d9b640 10 monclient(hunting): get_auth_request con 0x7fc6ec09ff70 auth_method 0 2024-09-15T01:34:26.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6e9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6e9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6e9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6e9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6e9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6e9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 10 monclient(hunting): handle_auth_done global_id 5653 payload 824 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 1 -- 172.21.15.204:0/2981851550 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc6ec09fb80 msgr2=0x7fc6ec130760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 1 --2- 172.21.15.204:0/2981851550 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc6ec09fb80 0x7fc6ec130760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 1 -- 172.21.15.204:0/2981851550 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc6ec09ff70 msgr2=0x7fc6ec11d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 1 --2- 172.21.15.204:0/2981851550 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc6ec09ff70 0x7fc6ec11d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 1 monclient: found mon.noname-f 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 1 -- 172.21.15.204:0/2981851550 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e4009a20 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.099+0000 7fc6ead9d640 1 --2- 172.21.15.204:0/2981851550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec0a2e30 0x7fc6ec11d830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009e60 tx=0x7fc6e4037000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 1 -- 172.21.15.204:0/2981851550 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6e4007eb0 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:26.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 10 monclient: dump: 2024-09-15T01:34:26.120 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:26.120 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.120 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.120 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.120 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.120 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:26.120 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.120 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.120 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 1 -- 172.21.15.204:0/2981851550 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e403d430 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 1 -- 172.21.15.204:0/2981851550 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6e403da20 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6f0aa8640 5 monclient: authenticate success, global_id 5653 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6e8d99640 4 set_mon_vals no callback set 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 10 monclient: dump: 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.122 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6f0aa8640 10 monclient: _renew_subs 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6f0aa8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/2981851550 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ec11edf0 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6f0aa8640 10 monclient: _renew_subs 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6f0aa8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/2981851550 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ec142900 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.100+0000 7fc6da7fc640 1 -- 172.21.15.204:0/2981851550 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc6e4003040 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.101+0000 7fc6f0aa8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.101+0000 7fc6f0aa8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.101+0000 7fc6f0aa8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:26.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.101+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/2981851550 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ec070cc0 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.101+0000 7fc6da7fc640 1 --2- 172.21.15.204:0/2981851550 >> v2:172.21.15.204:6812/28210 conn(0x7fc6cc0397b0 0x7fc6cc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.101+0000 7fc6da7fc640 1 -- 172.21.15.204:0/2981851550 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5333+0+0 (secure 0 0 0) 0x7fc6e406cc50 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.101+0000 7fc6ea59c640 1 --2- 172.21.15.204:0/2981851550 >> v2:172.21.15.204:6812/28210 conn(0x7fc6cc0397b0 0x7fc6cc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.101+0000 7fc6ea59c640 10 monclient: get_auth_request con 0x7fc6cc0397b0 auth_method 0 2024-09-15T01:34:26.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.101+0000 7fc6ea59c640 1 --2- 172.21.15.204:0/2981851550 >> v2:172.21.15.204:6812/28210 conn(0x7fc6cc0397b0 0x7fc6cc03bc60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009c80 tx=0x7fc6dc0090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.105+0000 7fc6da7fc640 1 -- 172.21.15.204:0/2981851550 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc6e403db80 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.105+0000 7fc6da7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.105+0000 7fc6da7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:26.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.146+0000 7f32d5100640 10 monclient: get_monmap_and_config 2024-09-15T01:34:26.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.146+0000 7f32d5100640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.146+0000 7f32d5100640 10 monclient: monmap: 2024-09-15T01:34:26.148 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.148 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.148 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:26.146934+0000 2024-09-15T01:34:26.148 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:26.146934+0000 2024-09-15T01:34:26.148 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.148 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.148 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:26.148 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:26.148 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.148 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.149 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.149 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.149 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.149 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.149 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.149 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:26.149 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:26.149 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.146+0000 7f32d5100640 1 Processor -- start 2024-09-15T01:34:26.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.146+0000 7f32d5100640 1 -- start start 2024-09-15T01:34:26.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.146+0000 7f32d5100640 10 monclient: init 2024-09-15T01:34:26.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 10 monclient: _add_conns ranks=[3,6,8,2,9,10,7,1,5,4,0] 2024-09-15T01:34:26.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32d00987b0 0x7f32d009ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 10 monclient(hunting): picked mon.noname-g con 0x7f32d00987b0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:26.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f32d012ef10 0x7f32d012f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 10 monclient(hunting): picked mon.noname-d con 0x7f32d012ef10 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:26.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f32d012f880 0x7f32d012ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 10 monclient(hunting): picked mon.noname-e con 0x7f32d012f880 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0096e00 con 0x7f32d00987b0 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0097d50 con 0x7f32d012ef10 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00990b0 con 0x7f32d012f880 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32d5100640 10 monclient(hunting): authenticate will time out at 870.897270s 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32c659c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f32d012ef10 0x7f32d012f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f32d012f880 0x7f32d012ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32c659c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f32d012ef10 0x7f32d012f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42728/0 (socket says 172.21.15.178:42728) 2024-09-15T01:34:26.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f32d012f880 0x7f32d012ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54276/0 (socket says 172.21.15.178:54276) 2024-09-15T01:34:26.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32c659c640 1 -- 172.21.15.178:0/2650042752 learned_addr learned my addr 172.21.15.178:0/2650042752 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:26.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32c659c640 10 monclient(hunting): get_auth_request con 0x7f32d012ef10 auth_method 0 2024-09-15T01:34:26.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32c659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32c659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32c659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 10 monclient(hunting): get_auth_request con 0x7f32d012f880 auth_method 0 2024-09-15T01:34:26.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32ced9d640 1 --2- 172.21.15.178:0/2650042752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32d00987b0 0x7f32d009ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32ced9d640 10 monclient(hunting): get_auth_request con 0x7f32d00987b0 auth_method 0 2024-09-15T01:34:26.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32ced9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32c659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32c659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32c659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32ced9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32ced9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32ced9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 10 monclient(hunting): handle_auth_done global_id 5237 payload 274 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 1 -- 172.21.15.178:0/2650042752 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f32d012ef10 msgr2=0x7f32d012f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 1 --2- 172.21.15.178:0/2650042752 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f32d012ef10 0x7f32d012f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 1 -- 172.21.15.178:0/2650042752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32d00987b0 msgr2=0x7f32d009ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 1 --2- 172.21.15.178:0/2650042752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32d00987b0 0x7f32d009ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 1 monclient: found mon.noname-e 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32cf59e640 1 -- 172.21.15.178:0/2650042752 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d004bf60 con 0x7f32d012f880 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.147+0000 7f32c659c640 1 --2- 172.21.15.178:0/2650042752 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f32d012ef10 0x7f32d012f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32cf59e640 1 --2- 172.21.15.178:0/2650042752 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f32d012f880 0x7f32d012ba70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009ec0 tx=0x7f32c8009780 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c78599498a1248e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32ce59c640 1 -- 172.21.15.178:0/2650042752 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32c80079d0 con 0x7f32d012f880 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32ce59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32ce59c640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32ce59c640 10 monclient: dump: 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.164 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32ce59c640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32ce59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32ce59c640 1 -- 172.21.15.178:0/2650042752 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c802d050 con 0x7f32d012f880 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32d5100640 5 monclient: authenticate success, global_id 5237 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32d5100640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32ce59c640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32d5100640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:26.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32ce59c640 1 -- 172.21.15.178:0/2650042752 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32c8007b90 con 0x7f32d012f880 2024-09-15T01:34:26.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32ce59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32d5100640 4 set_mon_vals no callback set 2024-09-15T01:34:26.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32ce59c640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:26.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.148+0000 7f32ce59c640 10 monclient: dump: 2024-09-15T01:34:26.166 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.166 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.166 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.166 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.166 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.166 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.166 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.166 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 10 monclient: shutdown 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 1 -- 172.21.15.178:0/2650042752 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f32d012f880 msgr2=0x7f32d012ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 1 --2- 172.21.15.178:0/2650042752 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f32d012f880 0x7f32d012ba70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009ec0 tx=0x7f32c8009780 comp rx=0 tx=0).stop 2024-09-15T01:34:26.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 1 -- 172.21.15.178:0/2650042752 shutdown_connections 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 1 --2- 172.21.15.178:0/2650042752 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f32d012f880 0x7f32d012ba70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 1 --2- 172.21.15.178:0/2650042752 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f32d012ef10 0x7f32d012f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 1 --2- 172.21.15.178:0/2650042752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32d00987b0 0x7f32d009ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 1 -- 172.21.15.178:0/2650042752 >> 172.21.15.178:0/2650042752 conn(0x7f32d00a22e0 msgr2=0x7f32d00a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 1 -- 172.21.15.178:0/2650042752 shutdown_connections 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 1 -- 172.21.15.178:0/2650042752 wait complete. 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.149+0000 7f32d5100640 10 monclient: monmap: 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:26.168 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 1 Processor -- start 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 1 -- start start 2024-09-15T01:34:26.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 10 monclient: init 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 10 monclient: _add_conns ranks=[4,3,0,1,6,5,7,8,2] 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32d00987b0 0x7f32d006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 10 monclient(hunting): picked mon.noname-c con 0x7f32d00987b0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32d012ef10 0x7f32d012b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 10 monclient(hunting): picked mon.noname-g con 0x7f32d012ef10 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d012f880 0x7f32d012b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 10 monclient(hunting): picked mon.noname-a con 0x7f32d012f880 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d01305b0 con 0x7f32d012f880 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0097170 con 0x7f32d012ef10 2024-09-15T01:34:26.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0096f40 con 0x7f32d00987b0 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32d5100640 10 monclient(hunting): authenticate will time out at 870.900572s 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32d00987b0 0x7f32d006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32cf59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d012f880 0x7f32d012b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32d00987b0 0x7f32d006e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:33198/0 (socket says 172.21.15.178:33198) 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 1 -- 172.21.15.178:0/930185226 learned_addr learned my addr 172.21.15.178:0/930185226 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32cf59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d012f880 0x7f32d012b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:58596/0 (socket says 172.21.15.178:58596) 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 10 monclient(hunting): get_auth_request con 0x7f32d00987b0 auth_method 0 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32cf59e640 10 monclient(hunting): get_auth_request con 0x7f32d012f880 auth_method 0 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32cf59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32cf59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32cf59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32c659c640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32d012ef10 0x7f32d012b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32c659c640 10 monclient(hunting): get_auth_request con 0x7f32d012ef10 auth_method 0 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32c659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32c659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32c659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32cf59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32cf59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32cf59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32c659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32c659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32c659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 10 monclient(hunting): handle_auth_done global_id 5116 payload 824 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 1 -- 172.21.15.178:0/930185226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32d012ef10 msgr2=0x7f32d012b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32d012ef10 0x7f32d012b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.150+0000 7f32ced9d640 1 -- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d012f880 msgr2=0x7f32d012b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32ced9d640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d012f880 0x7f32d012b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32ced9d640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32ced9d640 1 monclient: found mon.noname-c 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32ced9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32ced9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:26.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32ced9d640 1 -- 172.21.15.178:0/930185226 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b8009450 con 0x7f32d00987b0 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32cf59e640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d012f880 0x7f32d012b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32ced9d640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32d00987b0 0x7f32d006e410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f32c000ea50 tx=0x7f32c00090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32c000ce90 con 0x7f32d00987b0 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32c77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32c77fe640 10 monclient: dump: 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.174 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.175 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.175 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.175 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.175 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.175 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.175 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.175 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.175 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32c77fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0009600 con 0x7f32d00987b0 2024-09-15T01:34:26.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32d5100640 5 monclient: authenticate success, global_id 5116 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32d5100640 10 monclient: _renew_subs 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32d5100640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32d5100640 1 -- 172.21.15.178:0/930185226 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c8009a20 con 0x7f32d00987b0 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32c77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32c00138f0 con 0x7f32d00987b0 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32c77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32c77fe640 10 monclient: dump: 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.176 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32cdd9b640 4 set_mon_vals no callback set 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32d5100640 10 monclient: _renew_subs 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32d5100640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:26.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.151+0000 7f32d5100640 1 -- 172.21.15.178:0/930185226 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d0135c30 con 0x7f32d00987b0 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32d5100640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32d5100640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d0130db0 0x7f32d013f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32d5100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32d5100640 1 -- 172.21.15.178:0/930185226 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0068530 con 0x7f32d0130db0 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f32c0013470 con 0x7f32d00987b0 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c659c640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d0130db0 0x7f32d013f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c659c640 10 monclient: get_auth_request con 0x7f32d0130db0 auth_method 0 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c77fe640 1 --2- 172.21.15.178:0/930185226 >> v2:172.21.15.204:6812/28210 conn(0x7f32b00397b0 0x7f32b003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5333+0+0 (secure 0 0 0) 0x7f32c004bc60 con 0x7f32d00987b0 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c659c640 10 monclient(hunting): handle_auth_done global_id 5087 payload 824 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c659c640 1 -- 172.21.15.178:0/930185226 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32d004c4a0 con 0x7f32d0130db0 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32cf59e640 1 --2- 172.21.15.178:0/930185226 >> v2:172.21.15.204:6812/28210 conn(0x7f32b00397b0 0x7f32b003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c659c640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d0130db0 0x7f32d013f770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32d009c910 tx=0x7f32b80091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32cf59e640 10 monclient: get_auth_request con 0x7f32b00397b0 auth_method 0 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32b800fdb0 con 0x7f32d0130db0 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.152+0000 7f32c77fe640 10 monclient: dump: 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.179 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.180 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.153+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f32b801d480 con 0x7f32d0130db0 2024-09-15T01:34:26.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.153+0000 7f32c77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.153+0000 7f32c77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:26.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.153+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d0130db0 msgr2=0x7f32d013f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.153+0000 7f32c77fe640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d0130db0 0x7f32d013f770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32d009c910 tx=0x7f32b80091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.153+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d0130db0 msgr2=0x7f32d013f770 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.153+0000 7f32c77fe640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d0130db0 0x7f32d013f770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.153+0000 7f32cf59e640 1 --2- 172.21.15.178:0/930185226 >> v2:172.21.15.204:6812/28210 conn(0x7f32b00397b0 0x7f32b003bc60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f32c80099f0 tx=0x7f32c8037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.158+0000 7f32d5100640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:26.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.158+0000 7f32d5100640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d012bdb0 0x7f32d013fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.158+0000 7f32d5100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.158+0000 7f32d5100640 1 -- 172.21.15.178:0/930185226 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d004bc80 con 0x7f32d012bdb0 2024-09-15T01:34:26.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.158+0000 7f32c659c640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d012bdb0 0x7f32d013fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.158+0000 7f32c659c640 10 monclient: get_auth_request con 0x7f32d012bdb0 auth_method 0 2024-09-15T01:34:26.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.158+0000 7f32c659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.158+0000 7f32c659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.158+0000 7f32c659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.158+0000 7f32c659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.158+0000 7f32c659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.158+0000 7f32c659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c659c640 10 monclient(hunting): handle_auth_done global_id 5096 payload 824 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c659c640 1 -- 172.21.15.178:0/930185226 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f32d005ed70 con 0x7f32d012bdb0 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c659c640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d012bdb0 0x7f32d013fcb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f32d009c910 tx=0x7f32b801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32b8025040 con 0x7f32d012bdb0 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c77fe640 10 monclient: dump: 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.184 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9311 (secure 0 0 0) 0x7f32b801c3e0 con 0x7f32d012bdb0 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:26.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d012bdb0 msgr2=0x7f32d013fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c77fe640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d012bdb0 0x7f32d013fcb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f32d009c910 tx=0x7f32b801d000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c77fe640 1 -- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d012bdb0 msgr2=0x7f32d013fcb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.159+0000 7f32c77fe640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32d012bdb0 0x7f32d013fcb0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f32d009c910 tx=0x7f32b801d000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.186 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:26.186 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:26.186 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:26.186 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:26.186 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:26.186 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:26.186 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:26.187 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.188 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:26.189 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.190 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:26.191 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.192 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:26.193 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:26.195 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:26.196 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.198 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:26.200 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.201 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.202 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.203 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.204 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.205 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:26.206 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:26.207 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.208 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.208 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.208 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:26.208 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:26.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.160+0000 7f32d5100640 1 -- 172.21.15.178:0/930185226 >> v2:172.21.15.204:6812/28210 conn(0x7f32b00397b0 msgr2=0x7f32b003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.160+0000 7f32d5100640 1 --2- 172.21.15.178:0/930185226 >> v2:172.21.15.204:6812/28210 conn(0x7f32b00397b0 0x7f32b003bc60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f32c80099f0 tx=0x7f32c8037000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.160+0000 7f32d5100640 10 monclient: shutdown 2024-09-15T01:34:26.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.160+0000 7f32d5100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.160+0000 7f32d5100640 1 -- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32d00987b0 msgr2=0x7f32d006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.160+0000 7f32d5100640 1 --2- 172.21.15.178:0/930185226 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f32d00987b0 0x7f32d006e410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f32c000ea50 tx=0x7f32c00090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.160+0000 7f32cf59e640 1 -- 172.21.15.178:0/930185226 reap_dead start 2024-09-15T01:34:26.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.160+0000 7f32cf59e640 1 -- 172.21.15.178:0/930185226 reap_dead start 2024-09-15T01:34:26.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.160+0000 7f32d5100640 1 -- 172.21.15.178:0/930185226 shutdown_connections 2024-09-15T01:34:26.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.160+0000 7f32d5100640 1 -- 172.21.15.178:0/930185226 >> 172.21.15.178:0/930185226 conn(0x7f32d00a22e0 msgr2=0x7f32d009d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.160+0000 7f32d5100640 1 -- 172.21.15.178:0/930185226 shutdown_connections 2024-09-15T01:34:26.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.160+0000 7f32d5100640 1 -- 172.21.15.178:0/930185226 wait complete. 2024-09-15T01:34:26.210 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:26.210 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:34:26.211 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:34:26.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.268+0000 7fc6f0aa8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-09-15T01:34:26.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.268+0000 7fc6f0aa8640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-09-15T01:34:26.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.268+0000 7fc6f0aa8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:26.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.268+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/2981851550 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"} v 0) v1 -- 0x7fc6ec09ddb0 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.302+0000 7f7f9119d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:26.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.302+0000 7f7f9119d640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.302+0000 7f7f9119d640 10 monclient: monmap: 2024-09-15T01:34:26.304 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.304 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.304 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:26.303472+0000 2024-09-15T01:34:26.304 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:26.303472+0000 2024-09-15T01:34:26.304 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:26.305 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 1 Processor -- start 2024-09-15T01:34:26.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 1 -- start start 2024-09-15T01:34:26.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 10 monclient: init 2024-09-15T01:34:26.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 10 monclient: _add_conns ranks=[0,7,4,8,3,1,10,2,6,5,9] 2024-09-15T01:34:26.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7f8c0a2390 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:26.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7f8c09f8a0 0x7f7f8c09fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 10 monclient(hunting): picked mon.noname-i con 0x7f7f8c09f8a0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c137bb0 0x7f7f8c139f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 10 monclient(hunting): picked mon.noname-c con 0x7f7f8c137bb0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c04bf60 con 0x7f7f8c0a2390 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c09d700 con 0x7f7f8c137bb0 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c09ddb0 con 0x7f7f8c09f8a0 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f9119d640 10 monclient(hunting): authenticate will time out at 871.053824s 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f8ad9d640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f7f8c0a2390 0x7f7f8c132a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:34418/0 (socket says 172.21.15.178:34418) 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f8ad9d640 1 -- 172.21.15.178:0/4136531593 learned_addr learned my addr 172.21.15.178:0/4136531593 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:34:26.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f8b59e640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c137bb0 0x7f7f8c139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.303+0000 7f7f8b59e640 10 monclient(hunting): get_auth_request con 0x7f7f8c137bb0 auth_method 0 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f83fff640 1 --2- 172.21.15.178:0/4136531593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7f8c09f8a0 0x7f7f8c09fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f83fff640 10 monclient(hunting): get_auth_request con 0x7f7f8c09f8a0 auth_method 0 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f83fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 10 monclient(hunting): handle_auth_done global_id 5125 payload 274 2024-09-15T01:34:26.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8a59c640 1 -- 172.21.15.178:0/4136531593 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798349499 0 0) 0x7f7f8c04bf60 con 0x7f7f8c0a2390 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 1 -- 172.21.15.178:0/4136531593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7f8c09f8a0 msgr2=0x7f7f8c09fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 1 --2- 172.21.15.178:0/4136531593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7f8c09f8a0 0x7f7f8c09fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 1 -- 172.21.15.178:0/4136531593 >> v1:172.21.15.178:6789/0 conn(0x7f7f8c0a2390 legacy=0x7f7f8c132a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 1 monclient: found mon.noname-c 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 1 -- 172.21.15.178:0/4136531593 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c0a2760 con 0x7f7f8c137bb0 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8a59c640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f83fff640 1 --2- 172.21.15.178:0/4136531593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7f8c09f8a0 0x7f7f8c09fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.304+0000 7f7f8b59e640 1 --2- 172.21.15.178:0/4136531593 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c137bb0 0x7f7f8c139f90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009e90 tx=0x7f7f84009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=fe1db85e29f37d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f8a59c640 1 -- 172.21.15.178:0/4136531593 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7f84031880 con 0x7f7f8c137bb0 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f8a59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f8a59c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f8a59c640 10 monclient: dump: 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.319 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f8a59c640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f8a59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f8a59c640 1 -- 172.21.15.178:0/4136531593 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8402d050 con 0x7f7f8c137bb0 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 5 monclient: authenticate success, global_id 5125 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f8a59c640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f8a59c640 1 -- 172.21.15.178:0/4136531593 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7f840303a0 con 0x7f7f8c137bb0 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 4 set_mon_vals no callback set 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f8a59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f8a59c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:26.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f8a59c640 10 monclient: dump: 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.321 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 10 monclient: shutdown 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 1 -- 172.21.15.178:0/4136531593 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c137bb0 msgr2=0x7f7f8c139f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 1 --2- 172.21.15.178:0/4136531593 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c137bb0 0x7f7f8c139f90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009e90 tx=0x7f7f84009780 comp rx=0 tx=0).stop 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 1 -- 172.21.15.178:0/4136531593 shutdown_connections 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 1 --2- 172.21.15.178:0/4136531593 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c137bb0 0x7f7f8c139f90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 1 --2- 172.21.15.178:0/4136531593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7f8c09f8a0 0x7f7f8c09fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 1 -- 172.21.15.178:0/4136531593 >> 172.21.15.178:0/4136531593 conn(0x7f7f8c0631b0 msgr2=0x7f7f8c0635b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 1 -- 172.21.15.178:0/4136531593 shutdown_connections 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.305+0000 7f7f9119d640 1 -- 172.21.15.178:0/4136531593 wait complete. 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient: monmap: 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.323 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 1 Processor -- start 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 1 -- start start 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient: init 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient: _add_conns ranks=[3,0,6,4,1,7,5,2,8] 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f8c09f8a0 0x7f7f8c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient(hunting): picked mon.noname-g con 0x7f7f8c09f8a0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f8c0a2390 0x7f7f8c1348d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient(hunting): picked mon.noname-a con 0x7f7f8c0a2390 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7f8c137bb0 0x7f7f8c134e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient(hunting): picked mon.noname-d con 0x7f7f8c137bb0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c09d3d0 con 0x7f7f8c0a2390 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c09bac0 con 0x7f7f8c09f8a0 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c097260 con 0x7f7f8c137bb0 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.306+0000 7f7f9119d640 10 monclient(hunting): authenticate will time out at 871.056925s 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f8c0a2390 0x7f7f8c1348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f8c0a2390 0x7f7f8c1348d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:58604/0 (socket says 172.21.15.178:58604) 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8b59e640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7f8c137bb0 0x7f7f8c134e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 1 -- 172.21.15.178:0/2025285751 learned_addr learned my addr 172.21.15.178:0/2025285751 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 10 monclient(hunting): get_auth_request con 0x7f7f8c0a2390 auth_method 0 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8b59e640 10 monclient(hunting): get_auth_request con 0x7f7f8c137bb0 auth_method 0 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8ad9d640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f8c09f8a0 0x7f7f8c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8ad9d640 10 monclient(hunting): get_auth_request con 0x7f7f8c09f8a0 auth_method 0 2024-09-15T01:34:26.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f8ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 10 monclient(hunting): handle_auth_done global_id 5310 payload 824 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 1 -- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7f8c137bb0 msgr2=0x7f7f8c134e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7f8c137bb0 0x7f7f8c134e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 1 -- 172.21.15.178:0/2025285751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f8c09f8a0 msgr2=0x7f7f8c0a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f8c09f8a0 0x7f7f8c0a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 1 monclient: found mon.noname-a 2024-09-15T01:34:26.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 1 -- 172.21.15.178:0/2025285751 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f84009a20 con 0x7f7f8c0a2390 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f83fff640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f8c0a2390 0x7f7f8c1348d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400d890 tx=0x7f7f74008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7f7400de80 con 0x7f7f8c0a2390 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f82ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f82ffd640 10 monclient: dump: 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.328 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f82ffd640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f82ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7400bd50 con 0x7f7f8c0a2390 2024-09-15T01:34:26.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f9119d640 5 monclient: authenticate success, global_id 5310 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f9119d640 10 monclient: _renew_subs 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f9119d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f9119d640 1 -- 172.21.15.178:0/2025285751 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c137740 con 0x7f7f8c0a2390 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f82ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7f740148b0 con 0x7f7f8c0a2390 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f82ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f82ffd640 10 monclient: dump: 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f89d9b640 4 set_mon_vals no callback set 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f9119d640 10 monclient: _renew_subs 2024-09-15T01:34:26.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f9119d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.307+0000 7f7f9119d640 1 -- 172.21.15.178:0/2025285751 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c1360a0 con 0x7f7f8c0a2390 2024-09-15T01:34:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f9119d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f9119d640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c135350 0x7f7f8c12ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f9119d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f9119d640 1 -- 172.21.15.178:0/2025285751 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c06f430 con 0x7f7f8c135350 2024-09-15T01:34:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7f74014430 con 0x7f7f8c0a2390 2024-09-15T01:34:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f8ad9d640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c135350 0x7f7f8c12ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f8ad9d640 10 monclient: get_auth_request con 0x7f7f8c135350 auth_method 0 2024-09-15T01:34:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f8ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f8ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f8ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f8ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f8ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f8ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f82ffd640 1 --2- 172.21.15.178:0/2025285751 >> v2:172.21.15.204:6812/28210 conn(0x7f7f6c0397e0 0x7f7f6c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5333+0+0 (secure 0 0 0) 0x7f7f7404bc10 con 0x7f7f8c0a2390 2024-09-15T01:34:26.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f8ad9d640 10 monclient(hunting): handle_auth_done global_id 5134 payload 824 2024-09-15T01:34:26.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.308+0000 7f7f8ad9d640 1 -- 172.21.15.178:0/2025285751 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f8c135a40 con 0x7f7f8c135350 2024-09-15T01:34:26.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f8ad9d640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c135350 0x7f7f8c12ebf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c000c00 tx=0x7f7f7c007c80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f8b59e640 1 --2- 172.21.15.178:0/2025285751 >> v2:172.21.15.204:6812/28210 conn(0x7f7f6c0397e0 0x7f7f6c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f8b59e640 10 monclient: get_auth_request con 0x7f7f6c0397e0 auth_method 0 2024-09-15T01:34:26.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7f7c013a80 con 0x7f7f8c135350 2024-09-15T01:34:26.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f82ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f82ffd640 10 monclient: dump: 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.334 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.335 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.335 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.335 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f7f7c013be0 con 0x7f7f8c135350 2024-09-15T01:34:26.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f82ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f82ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:26.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c135350 msgr2=0x7f7f8c12ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f82ffd640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c135350 0x7f7f8c12ebf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c000c00 tx=0x7f7f7c007c80 comp rx=0 tx=0).stop 2024-09-15T01:34:26.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c135350 msgr2=0x7f7f8c12ebf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f82ffd640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c135350 0x7f7f8c12ebf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.309+0000 7f7f8b59e640 1 --2- 172.21.15.178:0/2025285751 >> v2:172.21.15.204:6812/28210 conn(0x7f7f6c0397e0 0x7f7f6c03bc90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7f84034000 tx=0x7f7f84037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f9119d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:26.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f9119d640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c12f130 0x7f7f8c12f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f9119d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f9119d640 1 -- 172.21.15.178:0/2025285751 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c04bc80 con 0x7f7f8c12f130 2024-09-15T01:34:26.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f8ad9d640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c12f130 0x7f7f8c12f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f8ad9d640 10 monclient: get_auth_request con 0x7f7f8c12f130 auth_method 0 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f8ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f8ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f8ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f8ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f8ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f8ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f8ad9d640 10 monclient(hunting): handle_auth_done global_id 5143 payload 824 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f8ad9d640 1 -- 172.21.15.178:0/2025285751 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7f8c05ed70 con 0x7f7f8c12f130 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f8ad9d640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c12f130 0x7f7f8c12f530 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c000fb0 tx=0x7f7f7c019000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7f7c0047a0 con 0x7f7f8c12f130 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f82ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f82ffd640 10 monclient: dump: 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.339 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.340 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.340 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.340 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.340 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f7f7c0062e0 con 0x7f7f8c12f130 2024-09-15T01:34:26.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f82ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:26.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f82ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:26.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c12f130 msgr2=0x7f7f8c12f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f82ffd640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c12f130 0x7f7f8c12f530 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c000fb0 tx=0x7f7f7c019000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f82ffd640 1 -- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c12f130 msgr2=0x7f7f8c12f530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.315+0000 7f7f82ffd640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7f8c12f130 0x7f7f8c12f530 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.341 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:26.341 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:26.341 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:26.341 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:26.341 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:26.341 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:26.341 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:26.341 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:26.341 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:26.341 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:26.341 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:26.341 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.342 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.343 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.344 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:26.346 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.347 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.348 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.349 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:26.351 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.352 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.353 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.354 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.355 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.356 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:26.357 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.358 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:26.359 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.360 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.361 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.362 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.362 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:26.362 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:26.362 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.362 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.362 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.362 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:26.362 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.362 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.362 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:26.362 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.362 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.363 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.363 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:26.363 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.363 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.363 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.363 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:26.363 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:26.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.316+0000 7f7f9119d640 1 -- 172.21.15.178:0/2025285751 >> v2:172.21.15.204:6812/28210 conn(0x7f7f6c0397e0 msgr2=0x7f7f6c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.316+0000 7f7f9119d640 1 --2- 172.21.15.178:0/2025285751 >> v2:172.21.15.204:6812/28210 conn(0x7f7f6c0397e0 0x7f7f6c03bc90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7f84034000 tx=0x7f7f84037000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.316+0000 7f7f9119d640 10 monclient: shutdown 2024-09-15T01:34:26.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.316+0000 7f7f9119d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.316+0000 7f7f9119d640 1 -- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f8c0a2390 msgr2=0x7f7f8c1348d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.316+0000 7f7f9119d640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f8c0a2390 0x7f7f8c1348d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400d890 tx=0x7f7f74008040 comp rx=0 tx=0).stop 2024-09-15T01:34:26.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.316+0000 7f7f8b59e640 1 -- 172.21.15.178:0/2025285751 reap_dead start 2024-09-15T01:34:26.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.316+0000 7f7f9119d640 1 -- 172.21.15.178:0/2025285751 shutdown_connections 2024-09-15T01:34:26.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.316+0000 7f7f9119d640 1 --2- 172.21.15.178:0/2025285751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f8c0a2390 0x7f7f8c1348d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.316+0000 7f7f9119d640 1 -- 172.21.15.178:0/2025285751 >> 172.21.15.178:0/2025285751 conn(0x7f7f8c0631b0 msgr2=0x7f7f8c09e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.316+0000 7f7f9119d640 1 -- 172.21.15.178:0/2025285751 shutdown_connections 2024-09-15T01:34:26.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.316+0000 7f7f9119d640 1 -- 172.21.15.178:0/2025285751 wait complete. 2024-09-15T01:34:26.365 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:26.365 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:34:26.366 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:34:26.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.457+0000 7f1dfbe60640 10 monclient: get_monmap_and_config 2024-09-15T01:34:26.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.457+0000 7f1dfbe60640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.457+0000 7f1dfbe60640 10 monclient: monmap: 2024-09-15T01:34:26.460 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.460 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.460 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:26.458689+0000 2024-09-15T01:34:26.460 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:26.458689+0000 2024-09-15T01:34:26.460 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.458+0000 7f1dfbe60640 1 Processor -- start 2024-09-15T01:34:26.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.458+0000 7f1dfbe60640 1 -- start start 2024-09-15T01:34:26.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.458+0000 7f1dfbe60640 10 monclient: init 2024-09-15T01:34:26.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 10 monclient: _add_conns ranks=[4,5,2,1,6,10,3,8,9,0,7] 2024-09-15T01:34:26.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1df40a2930 0x7f1df4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 10 monclient(hunting): picked mon.noname-c con 0x7f1df40a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:26.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1df409faa0 0x7f1df409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 10 monclient(hunting): picked mon.noname-h con 0x7f1df409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:26.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1df40a08c0 0x7f1df40a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 10 monclient(hunting): picked mon.noname-b con 0x7f1df40a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:26.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df409e0f0 con 0x7f1df40a08c0 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1df409dd80 con 0x7f1df40a2930 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1df40a2d00 con 0x7f1df409faa0 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfbe60640 10 monclient(hunting): authenticate will time out at 871.209159s 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1df40a2930 0x7f1df4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfa3fd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1df40a08c0 0x7f1df40a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1df40a2930 0x7f1df4132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:33230/0 (socket says 172.21.15.178:33230) 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfa3fd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1df40a08c0 0x7f1df40a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42986/0 (socket says 172.21.15.178:42986) 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 1 -- 172.21.15.178:0/52090930 learned_addr learned my addr 172.21.15.178:0/52090930 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 10 monclient(hunting): get_auth_request con 0x7f1df40a2930 auth_method 0 2024-09-15T01:34:26.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfa3fd640 10 monclient(hunting): get_auth_request con 0x7f1df40a08c0 auth_method 0 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfa3fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfa3fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfa3fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df93fb640 1 --2- 172.21.15.178:0/52090930 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1df409faa0 0x7f1df409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfa3fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfa3fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfa3fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df93fb640 10 monclient(hunting): get_auth_request con 0x7f1df409faa0 auth_method 0 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df93fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df93fb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df93fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df93fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df93fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df93fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 10 monclient(hunting): handle_auth_done global_id 5152 payload 274 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 1 -- 172.21.15.178:0/52090930 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1df409faa0 msgr2=0x7f1df409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 1 --2- 172.21.15.178:0/52090930 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1df409faa0 0x7f1df409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 1 -- 172.21.15.178:0/52090930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1df40a08c0 msgr2=0x7f1df40a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 1 --2- 172.21.15.178:0/52090930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1df40a08c0 0x7f1df40a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 1 monclient: found mon.noname-c 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1df9bfc640 1 -- 172.21.15.178:0/52090930 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df404bf60 con 0x7f1df40a2930 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.459+0000 7f1dfa3fd640 1 --2- 172.21.15.178:0/52090930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1df40a08c0 0x7f1df40a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df9bfc640 1 --2- 172.21.15.178:0/52090930 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1df40a2930 0x7f1df4132de0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009940 tx=0x7f1df00091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1bf63ffe17ee1967 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df8bfa640 1 -- 172.21.15.178:0/52090930 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1df002ac60 con 0x7f1df40a2930 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df8bfa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df8bfa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df8bfa640 10 monclient: dump: 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.475 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.476 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df8bfa640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df8bfa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1dfbe60640 5 monclient: authenticate success, global_id 5152 2024-09-15T01:34:26.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df8bfa640 1 -- 172.21.15.178:0/52090930 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df002ade0 con 0x7f1df40a2930 2024-09-15T01:34:26.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df8bfa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1dfbe60640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:26.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df8bfa640 1 -- 172.21.15.178:0/52090930 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1df0031ca0 con 0x7f1df40a2930 2024-09-15T01:34:26.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1dfbe60640 4 set_mon_vals no callback set 2024-09-15T01:34:26.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df8bfa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df8bfa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:26.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1df8bfa640 10 monclient: dump: 2024-09-15T01:34:26.477 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.478 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1dfbe60640 10 monclient: shutdown 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1dfbe60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1dfbe60640 1 -- 172.21.15.178:0/52090930 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1df40a2930 msgr2=0x7f1df4132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.460+0000 7f1dfbe60640 1 --2- 172.21.15.178:0/52090930 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1df40a2930 0x7f1df4132de0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009940 tx=0x7f1df00091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.461+0000 7f1dfbe60640 1 -- 172.21.15.178:0/52090930 shutdown_connections 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.461+0000 7f1dfbe60640 1 --2- 172.21.15.178:0/52090930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1df40a08c0 0x7f1df40a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.461+0000 7f1dfbe60640 1 --2- 172.21.15.178:0/52090930 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1df409faa0 0x7f1df409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.461+0000 7f1dfbe60640 1 --2- 172.21.15.178:0/52090930 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1df40a2930 0x7f1df4132de0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.461+0000 7f1dfbe60640 1 -- 172.21.15.178:0/52090930 >> 172.21.15.178:0/52090930 conn(0x7f1df40a5ed0 msgr2=0x7f1df40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.461+0000 7f1dfbe60640 1 -- 172.21.15.178:0/52090930 shutdown_connections 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.461+0000 7f1dfbe60640 1 -- 172.21.15.178:0/52090930 wait complete. 2024-09-15T01:34:26.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.461+0000 7f1dfbe60640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.461+0000 7f1dfbe60640 10 monclient: monmap: 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.480 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.461+0000 7f1dfbe60640 1 Processor -- start 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.461+0000 7f1dfbe60640 1 -- start start 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.461+0000 7f1dfbe60640 10 monclient: init 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 10 monclient: _add_conns ranks=[1,3,0,4,7,6,2,5,8] 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df409faa0 0x7f1df409b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 10 monclient(hunting): picked mon.noname-f con 0x7f1df409faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1df40a08c0 0x7f1df4131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 10 monclient(hunting): picked mon.noname-g con 0x7f1df40a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:26.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1df40a2930 0x7f1df411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 10 monclient(hunting): picked mon.noname-a con 0x7f1df40a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:26.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df409d630 con 0x7f1df40a2930 2024-09-15T01:34:26.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df409bcc0 con 0x7f1df409faa0 2024-09-15T01:34:26.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df4142ba0 con 0x7f1df40a08c0 2024-09-15T01:34:26.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfbe60640 10 monclient(hunting): authenticate will time out at 871.212402s 2024-09-15T01:34:26.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1df40a2930 0x7f1df411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1df40a2930 0x7f1df411c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:58606/0 (socket says 172.21.15.178:58606) 2024-09-15T01:34:26.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 1 -- 172.21.15.178:0/2460245364 learned_addr learned my addr 172.21.15.178:0/2460245364 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 10 monclient(hunting): get_auth_request con 0x7f1df40a2930 auth_method 0 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1df93fb640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1df40a08c0 0x7f1df4131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1df9bfc640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df409faa0 0x7f1df409b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1df93fb640 10 monclient(hunting): get_auth_request con 0x7f1df40a08c0 auth_method 0 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1df93fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1df93fb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1df93fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 10 monclient(hunting): handle_auth_done global_id 5319 payload 824 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 1 -- 172.21.15.178:0/2460245364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1df40a08c0 msgr2=0x7f1df4131840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1df40a08c0 0x7f1df4131840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 1 -- 172.21.15.178:0/2460245364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df409faa0 msgr2=0x7f1df409b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df409faa0 0x7f1df409b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 1 monclient: found mon.noname-a 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 1 -- 172.21.15.178:0/2460245364 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec009a20 con 0x7f1df40a2930 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1df9bfc640 10 monclient: get_auth_request con 0x7f1df409faa0 auth_method 0 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1df9bfc640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df409faa0 0x7f1df409b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:34:26.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.462+0000 7f1dfa3fd640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1df40a2930 0x7f1df411c530 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009f90 tx=0x7f1dec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1dec00ce70 con 0x7f1df40a2930 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1de9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1de9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1de9ffb640 10 monclient: dump: 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.485 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1de9ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1de9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1dfbe60640 5 monclient: authenticate success, global_id 5319 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec00f040 con 0x7f1df40a2930 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1de9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1dfbe60640 10 monclient: _renew_subs 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1dec014d80 con 0x7f1df40a2930 2024-09-15T01:34:26.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1dfbe60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1dfbe60640 1 -- 172.21.15.178:0/2460245364 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df0009450 con 0x7f1df40a2930 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1de9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1debfff640 4 set_mon_vals no callback set 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1de9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1de9ffb640 10 monclient: dump: 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1dfbe60640 10 monclient: _renew_subs 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1dfbe60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.463+0000 7f1dfbe60640 1 -- 172.21.15.178:0/2460245364 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df411ec40 con 0x7f1df40a2930 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1dfbe60640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1dfbe60640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df4131d80 0x7f1df411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1dfbe60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1dfbe60640 1 -- 172.21.15.178:0/2460245364 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1df4132150 con 0x7f1df4131d80 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1dec014480 con 0x7f1df40a2930 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df9bfc640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df4131d80 0x7f1df411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df9bfc640 10 monclient: get_auth_request con 0x7f1df4131d80 auth_method 0 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df9bfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df9bfc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df9bfc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df9bfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df9bfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df9bfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1de9ffb640 1 --2- 172.21.15.178:0/2460245364 >> v2:172.21.15.204:6812/28210 conn(0x7f1dd80396e0 0x7f1dd803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df9bfc640 10 monclient(hunting): handle_auth_done global_id 14235 payload 824 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df9bfc640 1 -- 172.21.15.178:0/2460245364 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1df404c4a0 con 0x7f1df4131d80 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5333+0+0 (secure 0 0 0) 0x7f1dec04c630 con 0x7f1df40a2930 2024-09-15T01:34:26.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df9bfc640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df4131d80 0x7f1df411ca70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009910 tx=0x7f1df0032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df93fb640 1 --2- 172.21.15.178:0/2460245364 >> v2:172.21.15.204:6812/28210 conn(0x7f1dd80396e0 0x7f1dd803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1df93fb640 10 monclient: get_auth_request con 0x7f1dd80396e0 auth_method 0 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1df002ac60 con 0x7f1df4131d80 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1de9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1de9ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.464+0000 7f1de9ffb640 10 monclient: dump: 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.490 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.465+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f1df003c400 con 0x7f1df4131d80 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.465+0000 7f1de9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.465+0000 7f1de9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.465+0000 7f1df93fb640 1 --2- 172.21.15.178:0/2460245364 >> v2:172.21.15.204:6812/28210 conn(0x7f1dd80396e0 0x7f1dd803bb90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1df40a6ec0 tx=0x7f1de4008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.465+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df4131d80 msgr2=0x7f1df411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.465+0000 7f1de9ffb640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df4131d80 0x7f1df411ca70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009910 tx=0x7f1df0032000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.465+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df4131d80 msgr2=0x7f1df411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.465+0000 7f1de9ffb640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df4131d80 0x7f1df411ca70 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009910 tx=0x7f1df0032000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.470+0000 7f1dfbe60640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:26.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.470+0000 7f1dfbe60640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df411cfb0 0x7f1df4096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.470+0000 7f1dfbe60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.470+0000 7f1dfbe60640 1 -- 172.21.15.178:0/2460245364 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1df404bc80 con 0x7f1df411cfb0 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.470+0000 7f1df9bfc640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df411cfb0 0x7f1df4096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.470+0000 7f1df9bfc640 10 monclient: get_auth_request con 0x7f1df411cfb0 auth_method 0 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.470+0000 7f1df9bfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.470+0000 7f1df9bfc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.470+0000 7f1df9bfc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.470+0000 7f1df9bfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.470+0000 7f1df9bfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.470+0000 7f1df9bfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1df9bfc640 10 monclient(hunting): handle_auth_done global_id 14244 payload 824 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1df9bfc640 1 -- 172.21.15.178:0/2460245364 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1df405ed70 con 0x7f1df411cfb0 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1df9bfc640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df411cfb0 0x7f1df4096a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1df0000c00 tx=0x7f1df003b000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1df004a870 con 0x7f1df411cfb0 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1de9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1de9ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1de9ffb640 10 monclient: dump: 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.495 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.496 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.496 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.496 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f1df004a3f0 con 0x7f1df411cfb0 2024-09-15T01:34:26.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1de9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:26.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1de9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:26.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df411cfb0 msgr2=0x7f1df4096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1de9ffb640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df411cfb0 0x7f1df4096a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1df0000c00 tx=0x7f1df003b000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1de9ffb640 1 -- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df411cfb0 msgr2=0x7f1df4096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1de9ffb640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1df411cfb0 0x7f1df4096a30 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1df0000c00 tx=0x7f1df003b000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.496 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:26.497 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.498 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.499 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:26.500 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.501 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.502 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.503 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.504 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.505 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.506 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.508 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.509 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.510 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.511 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:26.512 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.513 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.514 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.515 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.516 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.517 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:26.518 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:26.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1dfbe60640 1 -- 172.21.15.178:0/2460245364 >> v2:172.21.15.204:6812/28210 conn(0x7f1dd80396e0 msgr2=0x7f1dd803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1dfbe60640 1 --2- 172.21.15.178:0/2460245364 >> v2:172.21.15.204:6812/28210 conn(0x7f1dd80396e0 0x7f1dd803bb90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1df40a6ec0 tx=0x7f1de4008040 comp rx=0 tx=0).stop 2024-09-15T01:34:26.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1dfbe60640 10 monclient: shutdown 2024-09-15T01:34:26.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1dfbe60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1dfbe60640 1 -- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1df40a2930 msgr2=0x7f1df411c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1dfbe60640 1 --2- 172.21.15.178:0/2460245364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1df40a2930 0x7f1df411c530 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009f90 tx=0x7f1dec00b040 comp rx=0 tx=0).stop 2024-09-15T01:34:26.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1dfa3fd640 1 -- 172.21.15.178:0/2460245364 reap_dead start 2024-09-15T01:34:26.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1dfa3fd640 1 -- 172.21.15.178:0/2460245364 reap_dead start 2024-09-15T01:34:26.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1dfbe60640 1 -- 172.21.15.178:0/2460245364 shutdown_connections 2024-09-15T01:34:26.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.471+0000 7f1dfbe60640 1 -- 172.21.15.178:0/2460245364 >> 172.21.15.178:0/2460245364 conn(0x7f1df40a5ed0 msgr2=0x7f1df40a5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.472+0000 7f1dfbe60640 1 -- 172.21.15.178:0/2460245364 shutdown_connections 2024-09-15T01:34:26.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.472+0000 7f1dfbe60640 1 -- 172.21.15.178:0/2460245364 wait complete. 2024-09-15T01:34:26.521 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:26.521 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:34:26.521 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:34:26.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.569+0000 7fc6da7fc640 1 -- 172.21.15.204:0/2981851550 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}]=0 removed item id 7 name 'osd.7' from crush map v80) v1 ==== 143+0+0 (secure 0 0 0) 0x7fc6e4039ba0 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.569+0000 7fc6da7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-09-15T01:34:26.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.569+0000 7fc6da7fc640 10 monclient: _finish_command 2 = system:0 removed item id 7 name 'osd.7' from crush map 2024-09-15T01:34:26.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.617+0000 7fc009d1d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:26.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.617+0000 7fc009d1d640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.617+0000 7fc009d1d640 10 monclient: monmap: 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:26.617968+0000 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:26.617968+0000 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.620 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.620 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.620 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.620 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:26.620 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:26.620 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.617+0000 7fc009d1d640 1 Processor -- start 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.617+0000 7fc009d1d640 1 -- start start 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.617+0000 7fc009d1d640 10 monclient: init 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 10 monclient: _add_conns ranks=[4,3,7,5,10,6,0,2,9,1,8] 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc00409c020 0x7fc00409ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 10 monclient(hunting): picked mon.noname-c con 0x7fc00409c020 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004067130 0x7fc004067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 10 monclient(hunting): picked mon.noname-g con 0x7fc004067130 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc004067a40 0x7fc0041393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 10 monclient(hunting): picked mon.noname-i con 0x7fc004067a40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00409e220 con 0x7fc004067130 2024-09-15T01:34:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc00409b270 con 0x7fc00409c020 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc00409af90 con 0x7fc004067a40 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc009d1d640 10 monclient(hunting): authenticate will time out at 871.368440s 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc00409c020 0x7fc00409ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc00409c020 0x7fc00409ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:33234/0 (socket says 172.21.15.178:33234) 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 1 -- 172.21.15.178:0/372950454 learned_addr learned my addr 172.21.15.178:0/372950454 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 10 monclient(hunting): get_auth_request con 0x7fc00409c020 auth_method 0 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc002ffd640 1 --2- 172.21.15.178:0/372950454 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004067130 0x7fc004067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc003fff640 1 --2- 172.21.15.178:0/372950454 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc004067a40 0x7fc0041393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc003fff640 10 monclient(hunting): get_auth_request con 0x7fc004067a40 auth_method 0 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc003fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc003fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc003fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc002ffd640 10 monclient(hunting): get_auth_request con 0x7fc004067130 auth_method 0 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc002ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc002ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc002ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc003fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc003fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc003fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 10 monclient(hunting): handle_auth_done global_id 5161 payload 274 2024-09-15T01:34:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 1 -- 172.21.15.178:0/372950454 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc004067a40 msgr2=0x7fc0041393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 1 --2- 172.21.15.178:0/372950454 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc004067a40 0x7fc0041393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 1 -- 172.21.15.178:0/372950454 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004067130 msgr2=0x7fc004067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 1 --2- 172.21.15.178:0/372950454 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004067130 0x7fc004067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 1 monclient: found mon.noname-c 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.618+0000 7fc0037fe640 1 -- 172.21.15.178:0/372950454 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00404bf60 con 0x7fc00409c020 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0037fe640 1 --2- 172.21.15.178:0/372950454 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc00409c020 0x7fc00409ea80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009f50 tx=0x7fbff4009840 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c664a5c859f09fcd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0027fc640 1 -- 172.21.15.178:0/372950454 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff4007880 con 0x7fc00409c020 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0027fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0027fc640 10 monclient: dump: 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.638 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0027fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0027fc640 1 -- 172.21.15.178:0/372950454 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff402d050 con 0x7fc00409c020 2024-09-15T01:34:26.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc009d1d640 5 monclient: authenticate success, global_id 5161 2024-09-15T01:34:26.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc009d1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:26.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0027fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc009d1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:26.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0027fc640 1 -- 172.21.15.178:0/372950454 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff4030360 con 0x7fc00409c020 2024-09-15T01:34:26.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc009d1d640 4 set_mon_vals no callback set 2024-09-15T01:34:26.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0027fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:26.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.619+0000 7fc0027fc640 10 monclient: dump: 2024-09-15T01:34:26.640 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.640 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.640 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.641 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 10 monclient: shutdown 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 1 -- 172.21.15.178:0/372950454 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc00409c020 msgr2=0x7fc00409ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 1 --2- 172.21.15.178:0/372950454 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc00409c020 0x7fc00409ea80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009f50 tx=0x7fbff4009840 comp rx=0 tx=0).stop 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 1 -- 172.21.15.178:0/372950454 shutdown_connections 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 1 --2- 172.21.15.178:0/372950454 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc004067a40 0x7fc0041393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 1 --2- 172.21.15.178:0/372950454 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004067130 0x7fc004067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 1 --2- 172.21.15.178:0/372950454 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc00409c020 0x7fc00409ea80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 1 -- 172.21.15.178:0/372950454 >> 172.21.15.178:0/372950454 conn(0x7fc004068a60 msgr2=0x7fc0040a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 1 -- 172.21.15.178:0/372950454 shutdown_connections 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 1 -- 172.21.15.178:0/372950454 wait complete. 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.620+0000 7fc009d1d640 10 monclient: monmap: 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.643 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 1 Processor -- start 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 1 -- start start 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 10 monclient: init 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 10 monclient: _add_conns ranks=[7,8,0,1,2,4,3,6,5] 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc004067130 0x7fc0041418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 10 monclient(hunting): picked mon.noname-i con 0x7fc004067130 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004067a40 0x7fc004134630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 10 monclient(hunting): picked mon.noname-e con 0x7fc004067a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00409c020 0x7fc00411e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 10 monclient(hunting): picked mon.noname-a con 0x7fc00409c020 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:26.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc004142790 con 0x7fc00409c020 2024-09-15T01:34:26.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc004121a30 con 0x7fc004067130 2024-09-15T01:34:26.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc004121750 con 0x7fc004067a40 2024-09-15T01:34:26.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc009d1d640 10 monclient(hunting): authenticate will time out at 871.371734s 2024-09-15T01:34:26.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc002ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004067a40 0x7fc004134630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc0037fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc004067130 0x7fc0041418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc003fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00409c020 0x7fc00411e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc002ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004067a40 0x7fc004134630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54286/0 (socket says 172.21.15.178:54286) 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc002ffd640 1 -- 172.21.15.178:0/3231967619 learned_addr learned my addr 172.21.15.178:0/3231967619 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc002ffd640 10 monclient(hunting): get_auth_request con 0x7fc004067a40 auth_method 0 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc002ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc002ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc002ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc0037fe640 10 monclient(hunting): get_auth_request con 0x7fc004067130 auth_method 0 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc0037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc0037fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc0037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc003fff640 10 monclient(hunting): get_auth_request con 0x7fc00409c020 auth_method 0 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc003fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc003fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc003fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc002ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc003fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc003fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc003fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.621+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc002ffd640 10 monclient(hunting): handle_auth_done global_id 5246 payload 824 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc002ffd640 1 -- 172.21.15.178:0/3231967619 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc004067130 msgr2=0x7fc0041418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc002ffd640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc004067130 0x7fc0041418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc002ffd640 1 -- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00409c020 msgr2=0x7fc00411e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc002ffd640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00409c020 0x7fc00411e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc002ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc002ffd640 1 monclient: found mon.noname-e 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc002ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc002ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc002ffd640 1 -- 172.21.15.178:0/3231967619 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff4009ae0 con 0x7fc004067a40 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc002ffd640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004067a40 0x7fc004134630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbff800e900 tx=0x7fbff80090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff8009480 con 0x7fc004067a40 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fbfeb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fbfeb7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fbfeb7fe640 10 monclient: dump: 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.648 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.649 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fbfeb7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fbfeb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff800cd60 con 0x7fc004067a40 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc009d1d640 5 monclient: authenticate success, global_id 5246 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc009d1d640 10 monclient: _renew_subs 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc009d1d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc009d1d640 1 -- 172.21.15.178:0/3231967619 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc004121d10 con 0x7fc004067a40 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fbfeb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff80138f0 con 0x7fc004067a40 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fbfeb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fbfeb7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc001ffb640 4 set_mon_vals no callback set 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fbfeb7fe640 10 monclient: dump: 2024-09-15T01:34:26.650 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.651 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.652 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc009d1d640 10 monclient: _renew_subs 2024-09-15T01:34:26.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc009d1d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:26.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.622+0000 7fc009d1d640 1 -- 172.21.15.178:0/3231967619 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc004141e00 con 0x7fc004067a40 2024-09-15T01:34:26.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc009d1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc009d1d640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0041207f0 0x7fc0040298c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc009d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc009d1d640 1 -- 172.21.15.178:0/3231967619 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040641f0 con 0x7fc0041207f0 2024-09-15T01:34:26.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fbff8013470 con 0x7fc004067a40 2024-09-15T01:34:26.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc0037fe640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0041207f0 0x7fc0040298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc0037fe640 10 monclient: get_auth_request con 0x7fc0041207f0 auth_method 0 2024-09-15T01:34:26.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc0037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc0037fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc0037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fbfeb7fe640 1 --2- 172.21.15.178:0/3231967619 >> v2:172.21.15.204:6812/28210 conn(0x7fbff00397b0 0x7fbff003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5312+0+0 (secure 0 0 0) 0x7fbff804ba90 con 0x7fc004067a40 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc0037fe640 10 monclient(hunting): handle_auth_done global_id 5255 payload 824 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc0037fe640 1 -- 172.21.15.178:0/3231967619 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc00404c4a0 con 0x7fc0041207f0 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc003fff640 1 --2- 172.21.15.178:0/3231967619 >> v2:172.21.15.204:6812/28210 conn(0x7fbff00397b0 0x7fbff003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc003fff640 10 monclient: get_auth_request con 0x7fbff00397b0 auth_method 0 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.623+0000 7fc0037fe640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0041207f0 0x7fc0040298c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009f20 tx=0x7fbff4036000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.624+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff403d880 con 0x7fc0041207f0 2024-09-15T01:34:26.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.624+0000 7fbfeb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.624+0000 7fbfeb7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.624+0000 7fbfeb7fe640 10 monclient: dump: 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.654 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.624+0000 7fc003fff640 1 --2- 172.21.15.178:0/3231967619 >> v2:172.21.15.204:6812/28210 conn(0x7fbff00397b0 0x7fbff003bc60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0040680f0 tx=0x7fbffc0090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.624+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fbff403d400 con 0x7fc0041207f0 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.624+0000 7fbfeb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.624+0000 7fbfeb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.624+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0041207f0 msgr2=0x7fc0040298c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.624+0000 7fbfeb7fe640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0041207f0 0x7fc0040298c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009f20 tx=0x7fbff4036000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.624+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0041207f0 msgr2=0x7fc0040298c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.624+0000 7fbfeb7fe640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc0041207f0 0x7fc0040298c0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009f20 tx=0x7fbff4036000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.629+0000 7fc009d1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.629+0000 7fc009d1d640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004134b70 0x7fc004029e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.629+0000 7fc009d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.629+0000 7fc009d1d640 1 -- 172.21.15.178:0/3231967619 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc00409b5f0 con 0x7fc004134b70 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fc0037fe640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004134b70 0x7fc004029e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fc0037fe640 10 monclient: get_auth_request con 0x7fc004134b70 auth_method 0 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fc0037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fc0037fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fc0037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fc0037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fc0037fe640 10 monclient(hunting): handle_auth_done global_id 5264 payload 824 2024-09-15T01:34:26.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fc0037fe640 1 -- 172.21.15.178:0/3231967619 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc004121ba0 con 0x7fc004134b70 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fc0037fe640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004134b70 0x7fc004029e00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009880 tx=0x7fbff403f070 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff4053070 con 0x7fc004134b70 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fbfeb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fbfeb7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fbfeb7fe640 10 monclient: dump: 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.657 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7fbff404b040 con 0x7fc004134b70 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fbfeb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fbfeb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004134b70 msgr2=0x7fc004029e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fbfeb7fe640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004134b70 0x7fc004029e00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009880 tx=0x7fbff403f070 comp rx=0 tx=0).stop 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fbfeb7fe640 1 -- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004134b70 msgr2=0x7fc004029e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.630+0000 7fbfeb7fe640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004134b70 0x7fc004029e00 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009880 tx=0x7fbff403f070 comp rx=0 tx=0).stop 2024-09-15T01:34:26.659 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:26.659 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:26.659 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:26.659 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:26.659 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:26.659 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:26.659 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:26.659 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:26.659 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.660 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.661 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:26.662 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:26.663 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.663 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.663 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:26.663 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:26.663 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:26.663 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:26.663 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:26.663 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:26.663 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:26.663 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.665 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:26.666 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.667 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.668 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.669 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.672 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.672 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:26.672 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:26.672 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.672 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.672 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.672 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.672 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:26.672 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.672 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.672 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.672 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.673 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.674 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.676 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:26.677 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.679 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.679 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:26.679 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:26.679 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.679 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.679 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.679 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.679 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:26.679 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.679 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.679 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.679 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.680 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.681 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.682 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.682 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:26.682 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:26.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.631+0000 7fc009d1d640 1 -- 172.21.15.178:0/3231967619 >> v2:172.21.15.204:6812/28210 conn(0x7fbff00397b0 msgr2=0x7fbff003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.631+0000 7fc009d1d640 1 --2- 172.21.15.178:0/3231967619 >> v2:172.21.15.204:6812/28210 conn(0x7fbff00397b0 0x7fbff003bc60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0040680f0 tx=0x7fbffc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.631+0000 7fc009d1d640 10 monclient: shutdown 2024-09-15T01:34:26.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.631+0000 7fc009d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.631+0000 7fc009d1d640 1 -- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004067a40 msgr2=0x7fc004134630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.631+0000 7fc009d1d640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004067a40 0x7fc004134630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbff800e900 tx=0x7fbff80090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.631+0000 7fc003fff640 1 -- 172.21.15.178:0/3231967619 reap_dead start 2024-09-15T01:34:26.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.631+0000 7fc009d1d640 1 -- 172.21.15.178:0/3231967619 shutdown_connections 2024-09-15T01:34:26.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.631+0000 7fc009d1d640 1 --2- 172.21.15.178:0/3231967619 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc004067a40 0x7fc004134630 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.631+0000 7fc009d1d640 1 -- 172.21.15.178:0/3231967619 >> 172.21.15.178:0/3231967619 conn(0x7fc004068a60 msgr2=0x7fc0040a19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.631+0000 7fc009d1d640 1 -- 172.21.15.178:0/3231967619 shutdown_connections 2024-09-15T01:34:26.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.631+0000 7fc009d1d640 1 -- 172.21.15.178:0/3231967619 wait complete. 2024-09-15T01:34:26.685 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:26.685 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:34:26.685 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:34:26.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.670+0000 7fc6f0aa8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-09-15T01:34:26.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.670+0000 7fc6f0aa8640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-09-15T01:34:26.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.670+0000 7fc6f0aa8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:26.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.670+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/2981851550 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"} v 0) v1 -- 0x7fc6ec09c820 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.670+0000 7fc6da7fc640 1 -- 172.21.15.204:0/2981851550 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}]=0 device 'osd.7' does not appear in the crush map v80) v1 ==== 145+0+0 (secure 0 0 0) 0x7fc6e4039d80 con 0x7fc6ec0a2e30 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.670+0000 7fc6da7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.670+0000 7fc6da7fc640 10 monclient: _finish_command 3 = system:0 device 'osd.7' does not appear in the crush map 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:device 'osd.7' does not appear in the crush map 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/2981851550 >> v2:172.21.15.204:6812/28210 conn(0x7fc6cc0397b0 msgr2=0x7fc6cc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 1 --2- 172.21.15.204:0/2981851550 >> v2:172.21.15.204:6812/28210 conn(0x7fc6cc0397b0 0x7fc6cc03bc60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009c80 tx=0x7fc6dc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 10 monclient: shutdown 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/2981851550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec0a2e30 msgr2=0x7fc6ec11d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 1 --2- 172.21.15.204:0/2981851550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec0a2e30 0x7fc6ec11d830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009e60 tx=0x7fc6e4037000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/2981851550 shutdown_connections 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 1 --2- 172.21.15.204:0/2981851550 >> v2:172.21.15.204:6812/28210 conn(0x7fc6cc0397b0 0x7fc6cc03bc60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 1 --2- 172.21.15.204:0/2981851550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec0a2e30 0x7fc6ec11d830 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 1 --2- 172.21.15.204:0/2981851550 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc6ec09ff70 0x7fc6ec11d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 1 --2- 172.21.15.204:0/2981851550 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc6ec09fb80 0x7fc6ec130760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/2981851550 >> 172.21.15.204:0/2981851550 conn(0x7fc6ec06e860 msgr2=0x7fc6ec131870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/2981851550 shutdown_connections 2024-09-15T01:34:26.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.673+0000 7fc6f0aa8640 1 -- 172.21.15.204:0/2981851550 wait complete. 2024-09-15T01:34:26.688 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm host1 2024-09-15T01:34:26.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient: monmap: 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:26.758094+0000 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:26.758094+0000 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:26.760 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 1 Processor -- start 2024-09-15T01:34:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 1 -- start start 2024-09-15T01:34:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient: init 2024-09-15T01:34:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient: _add_conns ranks=[0,10,3,1,5,8,9,4,7,6,2] 2024-09-15T01:34:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffa5c0a2e30 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffa5c09fb80 0x7ffa5c09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient(hunting): picked mon.noname-f con 0x7ffa5c09fb80 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 0x7ffa5c13a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient(hunting): picked mon.noname-g con 0x7ffa5c137d20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5c04b720 con 0x7ffa5c0a2e30 2024-09-15T01:34:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5c09e6c0 con 0x7ffa5c09fb80 2024-09-15T01:34:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c0a04b0 con 0x7ffa5c137d20 2024-09-15T01:34:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa6295d640 10 monclient(hunting): authenticate will time out at 882.400878s 2024-09-15T01:34:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa5b7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffa5c09fb80 0x7ffa5c09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa60efa640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 0x7ffa5c13a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 0x7ffa5c13a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54584/0 (socket says 172.21.15.204:54584) 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.757+0000 7ffa5b7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffa5c09fb80 0x7ffa5c09ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48988/0 (socket says 172.21.15.204:48988) 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 1 -- 172.21.15.204:0/739636507 learned_addr learned my addr 172.21.15.204:0/739636507 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 10 monclient(hunting): get_auth_request con 0x7ffa5c137d20 auth_method 0 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa5b7fe640 10 monclient(hunting): get_auth_request con 0x7ffa5c09fb80 auth_method 0 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa5b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa5b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa5b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa5b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa5b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa5b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 10 monclient(hunting): handle_auth_done global_id 14268 payload 274 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 1 -- 172.21.15.204:0/739636507 >> v2:172.21.15.204:3300/0 conn(0x7ffa5c09fb80 msgr2=0x7ffa5c09ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 1 --2- 172.21.15.204:0/739636507 >> v2:172.21.15.204:3300/0 conn(0x7ffa5c09fb80 0x7ffa5c09ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 1 -- 172.21.15.204:0/739636507 >> v1:172.21.15.178:6789/0 conn(0x7ffa5c0a2e30 legacy=0x7ffa5c132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 1 monclient: found mon.noname-g 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 1 -- 172.21.15.204:0/739636507 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5c133170 con 0x7ffa5c137d20 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa5b7fe640 1 --2- 172.21.15.204:0/739636507 >> v2:172.21.15.204:3300/0 conn(0x7ffa5c09fb80 0x7ffa5c09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.758+0000 7ffa60efa640 1 --2- 172.21.15.204:0/739636507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 0x7ffa5c13a100 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009f20 tx=0x7ffa54009840 comp rx=0 tx=0).ready entity=mon.3 client_cookie=3fdfa80369a12054 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa5affd640 1 -- 172.21.15.204:0/739636507 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa54007880 con 0x7ffa5c137d20 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa5affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa5affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa5affd640 10 monclient: dump: 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa5affd640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa5affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa5affd640 1 -- 172.21.15.204:0/739636507 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5402d050 con 0x7ffa5c137d20 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 5 monclient: authenticate success, global_id 14268 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa5affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa5affd640 1 -- 172.21.15.204:0/739636507 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa54030360 con 0x7ffa5c137d20 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 4 set_mon_vals no callback set 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa5affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa5affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa5affd640 10 monclient: dump: 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 10 monclient: shutdown 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 1 -- 172.21.15.204:0/739636507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 msgr2=0x7ffa5c13a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 1 --2- 172.21.15.204:0/739636507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 0x7ffa5c13a100 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009f20 tx=0x7ffa54009840 comp rx=0 tx=0).stop 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 1 -- 172.21.15.204:0/739636507 shutdown_connections 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 1 --2- 172.21.15.204:0/739636507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 0x7ffa5c13a100 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 1 --2- 172.21.15.204:0/739636507 >> v2:172.21.15.204:3300/0 conn(0x7ffa5c09fb80 0x7ffa5c09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 1 -- 172.21.15.204:0/739636507 >> 172.21.15.204:0/739636507 conn(0x7ffa5c06e860 msgr2=0x7ffa5c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 1 -- 172.21.15.204:0/739636507 shutdown_connections 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.759+0000 7ffa6295d640 1 -- 172.21.15.204:0/739636507 wait complete. 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.760+0000 7ffa6295d640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.760+0000 7ffa6295d640 10 monclient: monmap: 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.760+0000 7ffa6295d640 1 Processor -- start 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.760+0000 7ffa6295d640 1 -- start start 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.760+0000 7ffa6295d640 10 monclient: init 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient: _add_conns ranks=[6,0,3,5,7,2,4,1,8] 2024-09-15T01:34:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa5c09fb80 0x7ffa5c0992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient(hunting): picked mon.noname-d con 0x7ffa5c09fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa5c0a2e30 0x7ffa5c0997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient(hunting): picked mon.noname-a con 0x7ffa5c0a2e30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 0x7ffa5c099d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient(hunting): picked mon.noname-g con 0x7ffa5c137d20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c09ca40 con 0x7ffa5c0a2e30 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c09d650 con 0x7ffa5c137d20 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c11cb30 con 0x7ffa5c09fb80 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient(hunting): authenticate will time out at 882.404102s 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 0x7ffa5c099d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 0x7ffa5c099d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54588/0 (socket says 172.21.15.204:54588) 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 1 -- 172.21.15.204:0/1289498309 learned_addr learned my addr 172.21.15.204:0/1289498309 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 10 monclient(hunting): get_auth_request con 0x7ffa5c137d20 auth_method 0 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5bfff640 1 --2- 172.21.15.204:0/1289498309 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa5c09fb80 0x7ffa5c0992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5b7fe640 1 --2- 172.21.15.204:0/1289498309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa5c0a2e30 0x7ffa5c0997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5bfff640 10 monclient(hunting): get_auth_request con 0x7ffa5c09fb80 auth_method 0 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5b7fe640 10 monclient(hunting): get_auth_request con 0x7ffa5c0a2e30 auth_method 0 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 10 monclient(hunting): handle_auth_done global_id 14277 payload 824 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 1 -- 172.21.15.204:0/1289498309 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa5c09fb80 msgr2=0x7ffa5c0992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 1 --2- 172.21.15.204:0/1289498309 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa5c09fb80 0x7ffa5c0992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 1 -- 172.21.15.204:0/1289498309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa5c0a2e30 msgr2=0x7ffa5c0997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 1 --2- 172.21.15.204:0/1289498309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa5c0a2e30 0x7ffa5c0997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 1 monclient: found mon.noname-g 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 1 -- 172.21.15.204:0/1289498309 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa50009390 con 0x7ffa5c137d20 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa60efa640 1 --2- 172.21.15.204:0/1289498309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 0x7ffa5c099d20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009ab0 tx=0x7ffa54036000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa43fff640 1 -- 172.21.15.204:0/1289498309 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa5403cab0 con 0x7ffa5c137d20 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa43fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa43fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa43fff640 10 monclient: dump: 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.781 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:26.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa43fff640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa43fff640 1 -- 172.21.15.204:0/1289498309 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5403cc30 con 0x7ffa5c137d20 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 5 monclient: authenticate success, global_id 14277 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient: _renew_subs 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 1 -- 172.21.15.204:0/1289498309 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa54009ae0 con 0x7ffa5c137d20 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa43fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa43fff640 1 -- 172.21.15.204:0/1289498309 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa5403cda0 con 0x7ffa5c137d20 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa43fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa43fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa43fff640 10 monclient: dump: 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:26.783 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.784 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient: _renew_subs 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa6295d640 1 -- 172.21.15.204:0/1289498309 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c09b030 con 0x7ffa5c137d20 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.761+0000 7ffa5a7fc640 4 set_mon_vals no callback set 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.762+0000 7ffa43fff640 1 -- 172.21.15.204:0/1289498309 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ffa54003040 con 0x7ffa5c137d20 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.762+0000 7ffa6295d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.762+0000 7ffa6295d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.762+0000 7ffa6295d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.762+0000 7ffa6295d640 1 -- 172.21.15.204:0/1289498309 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5c070cc0 con 0x7ffa5c137d20 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.762+0000 7ffa43fff640 1 --2- 172.21.15.204:0/1289498309 >> v2:172.21.15.204:6812/28210 conn(0x7ffa480397b0 0x7ffa4803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.763+0000 7ffa43fff640 1 -- 172.21.15.204:0/1289498309 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5312+0+0 (secure 0 0 0) 0x7ffa5406bbc0 con 0x7ffa5c137d20 2024-09-15T01:34:26.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.763+0000 7ffa5bfff640 1 --2- 172.21.15.204:0/1289498309 >> v2:172.21.15.204:6812/28210 conn(0x7ffa480397b0 0x7ffa4803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.763+0000 7ffa5bfff640 10 monclient: get_auth_request con 0x7ffa480397b0 auth_method 0 2024-09-15T01:34:26.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.763+0000 7ffa5bfff640 1 --2- 172.21.15.204:0/1289498309 >> v2:172.21.15.204:6812/28210 conn(0x7ffa480397b0 0x7ffa4803bc60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c007fd0 tx=0x7ffa4c007e30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.767+0000 7ffa43fff640 1 -- 172.21.15.204:0/1289498309 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ffa5403a8f0 con 0x7ffa5c137d20 2024-09-15T01:34:26.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.767+0000 7ffa43fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.767+0000 7ffa43fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:26.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.780+0000 7f756acc9640 10 monclient: get_monmap_and_config 2024-09-15T01:34:26.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.780+0000 7f756acc9640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.780+0000 7f756acc9640 10 monclient: monmap: 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:26.780940+0000 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:26.780940+0000 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.798 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.780+0000 7f756acc9640 1 Processor -- start 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.780+0000 7f756acc9640 1 -- start start 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.780+0000 7f756acc9640 10 monclient: init 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 10 monclient: _add_conns ranks=[4,3,8,1,5,2,6,10,9,7,0] 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7564096a30 0x7f756412fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 10 monclient(hunting): picked mon.noname-c con 0x7f7564096a30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f756412fff0 0x7f75641303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 10 monclient(hunting): picked mon.noname-g con 0x7f756412fff0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:26.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7564096e00 0x7f7564097270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 10 monclient(hunting): picked mon.noname-e con 0x7f7564096e00 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75640997c0 con 0x7f756412fff0 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7564098640 con 0x7f7564096a30 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f75640982d0 con 0x7f7564096e00 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f756acc9640 10 monclient(hunting): authenticate will time out at 871.531273s 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7569266640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7564096e00 0x7f7564097270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7564096a30 0x7f756412fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7569266640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7564096e00 0x7f7564097270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54332/0 (socket says 172.21.15.178:54332) 2024-09-15T01:34:26.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7564096a30 0x7f756412fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:33246/0 (socket says 172.21.15.178:33246) 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7569266640 1 -- 172.21.15.178:0/182086306 learned_addr learned my addr 172.21.15.178:0/182086306 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 10 monclient(hunting): get_auth_request con 0x7f7564096a30 auth_method 0 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7569266640 10 monclient(hunting): get_auth_request con 0x7f7564096e00 auth_method 0 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7569266640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7569266640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7569266640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7563fff640 1 --2- 172.21.15.178:0/182086306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f756412fff0 0x7f75641303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7563fff640 10 monclient(hunting): get_auth_request con 0x7f756412fff0 auth_method 0 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7563fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7563fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7563fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7569266640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7569266640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7569266640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7563fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7563fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7563fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 10 monclient(hunting): handle_auth_done global_id 5170 payload 274 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 1 -- 172.21.15.178:0/182086306 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7564096e00 msgr2=0x7f7564097270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 1 --2- 172.21.15.178:0/182086306 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7564096e00 0x7f7564097270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 1 -- 172.21.15.178:0/182086306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f756412fff0 msgr2=0x7f75641303c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 1 --2- 172.21.15.178:0/182086306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f756412fff0 0x7f75641303c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 1 monclient: found mon.noname-c 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.781+0000 7f7568a65640 1 -- 172.21.15.178:0/182086306 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756404bf60 con 0x7f7564096a30 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f7568a65640 1 --2- 172.21.15.178:0/182086306 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7564096a30 0x7f756412fab0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f75580098b0 tx=0x7f75580090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f9b82098e7eb8983 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f75637fe640 1 -- 172.21.15.178:0/182086306 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f755802ab90 con 0x7f7564096a30 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f75637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f75637fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f75637fe640 10 monclient: dump: 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.803 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f75637fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f75637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f75637fe640 1 -- 172.21.15.178:0/182086306 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755802ad10 con 0x7f7564096a30 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f756acc9640 5 monclient: authenticate success, global_id 5170 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f756acc9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f75637fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f75637fe640 1 -- 172.21.15.178:0/182086306 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7558031ca0 con 0x7f7564096a30 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f756acc9640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f756acc9640 4 set_mon_vals no callback set 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f75637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f75637fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.782+0000 7f75637fe640 10 monclient: dump: 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.805 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.806 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 10 monclient: shutdown 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 1 -- 172.21.15.178:0/182086306 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7564096a30 msgr2=0x7f756412fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 1 --2- 172.21.15.178:0/182086306 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7564096a30 0x7f756412fab0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f75580098b0 tx=0x7f75580090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 1 -- 172.21.15.178:0/182086306 shutdown_connections 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 1 --2- 172.21.15.178:0/182086306 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7564096e00 0x7f7564097270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 1 --2- 172.21.15.178:0/182086306 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f756412fff0 0x7f75641303c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 1 --2- 172.21.15.178:0/182086306 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7564096a30 0x7f756412fab0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 1 -- 172.21.15.178:0/182086306 >> 172.21.15.178:0/182086306 conn(0x7f75640a22e0 msgr2=0x7f75640a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 1 -- 172.21.15.178:0/182086306 shutdown_connections 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 1 -- 172.21.15.178:0/182086306 wait complete. 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 10 monclient: monmap: 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.808 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.783+0000 7f756acc9640 1 Processor -- start 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 1 -- start start 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 10 monclient: init 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 10 monclient: _add_conns ranks=[7,6,4,3,2,1,5,0,8] 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7564096a30 0x7f7564141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 10 monclient(hunting): picked mon.noname-i con 0x7f7564096a30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7564096e00 0x7f7564141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 10 monclient(hunting): picked mon.noname-d con 0x7f7564096e00 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f756412fff0 0x7f7564142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 10 monclient(hunting): picked mon.noname-c con 0x7f756412fff0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f756411d740 con 0x7f756412fff0 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f756411ef60 con 0x7f7564096e00 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f756409bbd0 con 0x7f7564096a30 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f756acc9640 10 monclient(hunting): authenticate will time out at 871.534623s 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7569266640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f756412fff0 0x7f7564142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7563fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7564096e00 0x7f7564141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7569266640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f756412fff0 0x7f7564142520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:33252/0 (socket says 172.21.15.178:33252) 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7563fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7564096e00 0x7f7564141fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42762/0 (socket says 172.21.15.178:42762) 2024-09-15T01:34:26.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7569266640 1 -- 172.21.15.178:0/2339821362 learned_addr learned my addr 172.21.15.178:0/2339821362 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7568a65640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7564096a30 0x7f7564141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7569266640 10 monclient(hunting): get_auth_request con 0x7f756412fff0 auth_method 0 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7569266640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7569266640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7569266640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7563fff640 10 monclient(hunting): get_auth_request con 0x7f7564096e00 auth_method 0 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7563fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7563fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7563fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7568a65640 10 monclient(hunting): get_auth_request con 0x7f7564096a30 auth_method 0 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7568a65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7568a65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7568a65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7563fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7563fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7563fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7569266640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7569266640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7569266640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7568a65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7568a65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.784+0000 7f7568a65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7563fff640 10 monclient(hunting): handle_auth_done global_id 14262 payload 824 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7563fff640 1 -- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7564096a30 msgr2=0x7f7564141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7563fff640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7564096a30 0x7f7564141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7563fff640 1 -- 172.21.15.178:0/2339821362 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f756412fff0 msgr2=0x7f7564142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7563fff640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f756412fff0 0x7f7564142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7563fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7563fff640 1 monclient: found mon.noname-d 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7563fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7563fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7563fff640 1 -- 172.21.15.178:0/2339821362 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7558009390 con 0x7f7564096e00 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7569266640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f756412fff0 0x7f7564142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7568a65640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7564096a30 0x7f7564141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7563fff640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7564096e00 0x7f7564141fe0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f755000e930 tx=0x7f75500090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75500093c0 con 0x7f7564096e00 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7560ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7560ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7560ff9640 10 monclient: dump: 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.813 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7560ff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7560ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755000cdc0 con 0x7f7564096e00 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f756acc9640 5 monclient: authenticate success, global_id 14262 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f756acc9640 10 monclient: _renew_subs 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f756acc9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f756acc9640 1 -- 172.21.15.178:0/2339821362 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756411de60 con 0x7f7564096e00 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7560ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75500138f0 con 0x7f7564096e00 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7560ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7560ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7560ff9640 10 monclient: dump: 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.815 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f7562ffd640 4 set_mon_vals no callback set 2024-09-15T01:34:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f756acc9640 10 monclient: _renew_subs 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f756acc9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.785+0000 7f756acc9640 1 -- 172.21.15.178:0/2339821362 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756411d1c0 con 0x7f7564096e00 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7550013470 con 0x7f7564096e00 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f756acc9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f756acc9640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406b3d0 0x7f756406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f756acc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f756acc9640 1 -- 172.21.15.178:0/2339821362 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7564068530 con 0x7f756406b3d0 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7560ff9640 1 --2- 172.21.15.178:0/2339821362 >> v2:172.21.15.204:6812/28210 conn(0x7f75480397b0 0x7f754803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7568a65640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406b3d0 0x7f756406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5312+0+0 (secure 0 0 0) 0x7f755004b970 con 0x7f7564096e00 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7568a65640 10 monclient: get_auth_request con 0x7f756406b3d0 auth_method 0 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7568a65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7568a65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7568a65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7568a65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7568a65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7568a65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7569266640 1 --2- 172.21.15.178:0/2339821362 >> v2:172.21.15.204:6812/28210 conn(0x7f75480397b0 0x7f754803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7569266640 10 monclient: get_auth_request con 0x7f75480397b0 auth_method 0 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7568a65640 10 monclient(hunting): handle_auth_done global_id 5671 payload 824 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7568a65640 1 -- 172.21.15.178:0/2339821362 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f756404c4a0 con 0x7f756406b3d0 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7568a65640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406b3d0 0x7f756406b7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7558009880 tx=0x7f7558032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.786+0000 7f7569266640 1 --2- 172.21.15.178:0/2339821362 >> v2:172.21.15.204:6812/28210 conn(0x7f75480397b0 0x7f754803bc60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f755c00bfd0 tx=0x7f755c00d040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.787+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f755802ab90 con 0x7f756406b3d0 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.787+0000 7f7560ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.787+0000 7f7560ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:26.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.787+0000 7f7560ff9640 10 monclient: dump: 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.787+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f7558031970 con 0x7f756406b3d0 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.787+0000 7f7560ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.787+0000 7f7560ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.787+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406b3d0 msgr2=0x7f756406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.787+0000 7f7560ff9640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406b3d0 0x7f756406b7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7558009880 tx=0x7f7558032000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.787+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406b3d0 msgr2=0x7f756406b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.787+0000 7f7560ff9640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406b3d0 0x7f756406b7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.792+0000 7f756acc9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.792+0000 7f756acc9640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406bce0 0x7f756406dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.792+0000 7f756acc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.792+0000 7f756acc9640 1 -- 172.21.15.178:0/2339821362 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756404bc80 con 0x7f756406bce0 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7568a65640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406bce0 0x7f756406dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7568a65640 10 monclient: get_auth_request con 0x7f756406bce0 auth_method 0 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7568a65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7568a65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7568a65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7568a65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7568a65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7568a65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7568a65640 10 monclient(hunting): handle_auth_done global_id 5680 payload 824 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7568a65640 1 -- 172.21.15.178:0/2339821362 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f756405ed70 con 0x7f756406bce0 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7568a65640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406bce0 0x7f756406dd70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f75580098b0 tx=0x7f755803d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f755804c070 con 0x7f756406bce0 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7560ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7560ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7560ff9640 10 monclient: dump: 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.822 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.823 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.823 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.823 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.823 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.823 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9311 (secure 0 0 0) 0x7f755803c510 con 0x7f756406bce0 2024-09-15T01:34:26.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7560ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:26.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7560ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:26.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406bce0 msgr2=0x7f756406dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7560ff9640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406bce0 0x7f756406dd70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f75580098b0 tx=0x7f755803d000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7560ff9640 1 -- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406bce0 msgr2=0x7f756406dd70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.793+0000 7f7560ff9640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f756406bce0 0x7f756406dd70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.824 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:26.824 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:26.824 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:26.824 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:26.824 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:26.824 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:26.824 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:26.824 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:26.824 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:26.824 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:26.824 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.825 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.826 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:26.827 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.828 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:26.829 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.830 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.831 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.832 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.833 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.834 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.835 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:26.838 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.839 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.840 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:26.843 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:26.844 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.845 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.845 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.845 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:26.845 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.845 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.845 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:34:26.845 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.845 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.845 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.845 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:34:26.845 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.845 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.846 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:26.846 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.846 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:26.846 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:26.846 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:26.846 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.846 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.846 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:26.846 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:26.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.794+0000 7f756acc9640 1 -- 172.21.15.178:0/2339821362 >> v2:172.21.15.204:6812/28210 conn(0x7f75480397b0 msgr2=0x7f754803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.794+0000 7f756acc9640 1 --2- 172.21.15.178:0/2339821362 >> v2:172.21.15.204:6812/28210 conn(0x7f75480397b0 0x7f754803bc60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f755c00bfd0 tx=0x7f755c00d040 comp rx=0 tx=0).stop 2024-09-15T01:34:26.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.794+0000 7f756acc9640 10 monclient: shutdown 2024-09-15T01:34:26.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.794+0000 7f756acc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.794+0000 7f756acc9640 1 -- 172.21.15.178:0/2339821362 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7564096e00 msgr2=0x7f7564141fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.794+0000 7f756acc9640 1 --2- 172.21.15.178:0/2339821362 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7564096e00 0x7f7564141fe0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f755000e930 tx=0x7f75500090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.794+0000 7f7569266640 1 -- 172.21.15.178:0/2339821362 reap_dead start 2024-09-15T01:34:26.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.794+0000 7f7569266640 1 -- 172.21.15.178:0/2339821362 reap_dead start 2024-09-15T01:34:26.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.794+0000 7f756acc9640 1 -- 172.21.15.178:0/2339821362 shutdown_connections 2024-09-15T01:34:26.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.794+0000 7f756acc9640 1 -- 172.21.15.178:0/2339821362 >> 172.21.15.178:0/2339821362 conn(0x7f75640a22e0 msgr2=0x7f756409d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.794+0000 7f756acc9640 1 -- 172.21.15.178:0/2339821362 shutdown_connections 2024-09-15T01:34:26.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.794+0000 7f756acc9640 1 -- 172.21.15.178:0/2339821362 wait complete. 2024-09-15T01:34:26.848 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:26.849 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:34:26.849 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:34:26.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.927+0000 7ffa6295d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host1"}] 2024-09-15T01:34:26.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.927+0000 7ffa6295d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host1"}] 2024-09-15T01:34:26.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.927+0000 7ffa6295d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:26.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:26.927+0000 7ffa6295d640 1 -- 172.21.15.204:0/1289498309 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "host1"} v 0) v1 -- 0x7ffa5c067590 con 0x7ffa5c137d20 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.941+0000 7f71f1909640 10 monclient: get_monmap_and_config 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.941+0000 7f71f1909640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.941+0000 7f71f1909640 10 monclient: monmap: 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:26.942185+0000 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:26.942185+0000 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.943 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.944 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.944 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.944 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.944 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.944 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.944 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:26.944 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:26.944 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.941+0000 7f71f1909640 1 Processor -- start 2024-09-15T01:34:26.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.941+0000 7f71f1909640 1 -- start start 2024-09-15T01:34:26.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.941+0000 7f71f1909640 10 monclient: init 2024-09-15T01:34:26.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 10 monclient: _add_conns ranks=[0,8,10,1,3,9,6,2,4,5,7] 2024-09-15T01:34:26.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f71ec09c020 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:26.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71ec1353a0 0x7f71ec137780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 10 monclient(hunting): picked mon.noname-e con 0x7f71ec1353a0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:26.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f71ec137cc0 0x7f71ec13a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 10 monclient(hunting): picked mon.noname-f con 0x7f71ec137cc0 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:26.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ec04bf60 con 0x7f71ec09c020 2024-09-15T01:34:26.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ec09e220 con 0x7f71ec137cc0 2024-09-15T01:34:26.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec09b270 con 0x7f71ec1353a0 2024-09-15T01:34:26.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71f1909640 10 monclient(hunting): authenticate will time out at 871.692510s 2024-09-15T01:34:26.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71eaffd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f71ec09c020 0x7f71ec09ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:34424/0 (socket says 172.21.15.178:34424) 2024-09-15T01:34:26.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71ec1353a0 0x7f71ec137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71eaffd640 1 -- 172.21.15.178:0/1814971503 learned_addr learned my addr 172.21.15.178:0/1814971503 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:34:26.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 10 monclient(hunting): get_auth_request con 0x7f71ec1353a0 auth_method 0 2024-09-15T01:34:26.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71eb7fe640 1 --2- 172.21.15.178:0/1814971503 >> v2:172.21.15.204:3300/0 conn(0x7f71ec137cc0 0x7f71ec13a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71eb7fe640 10 monclient(hunting): get_auth_request con 0x7f71ec137cc0 auth_method 0 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 10 monclient(hunting): handle_auth_done global_id 5282 payload 274 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71e9ffb640 1 -- 172.21.15.178:0/1814971503 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260130059 0 0) 0x7f71ec04bf60 con 0x7f71ec09c020 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 1 -- 172.21.15.178:0/1814971503 >> v2:172.21.15.204:3300/0 conn(0x7f71ec137cc0 msgr2=0x7f71ec13a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 1 --2- 172.21.15.178:0/1814971503 >> v2:172.21.15.204:3300/0 conn(0x7f71ec137cc0 0x7f71ec13a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 1 -- 172.21.15.178:0/1814971503 >> v1:172.21.15.178:6789/0 conn(0x7f71ec09c020 legacy=0x7f71ec09ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 1 monclient: found mon.noname-e 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71ea7fc640 1 -- 172.21.15.178:0/1814971503 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ec09af90 con 0x7f71ec1353a0 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.942+0000 7f71e9ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71ea7fc640 1 --2- 172.21.15.178:0/1814971503 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71ec1353a0 0x7f71ec137780 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009850 tx=0x7f71e00090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=de884d72930bb0ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71e9ffb640 1 -- 172.21.15.178:0/1814971503 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71e002ac80 con 0x7f71ec1353a0 2024-09-15T01:34:26.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71e9ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71e9ffb640 10 monclient: dump: 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71e9ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71e9ffb640 1 -- 172.21.15.178:0/1814971503 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e002ae00 con 0x7f71ec1353a0 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71f1909640 5 monclient: authenticate success, global_id 5282 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71f1909640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71e9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71f1909640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71e9ffb640 1 -- 172.21.15.178:0/1814971503 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71e0031ca0 con 0x7f71ec1353a0 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71f1909640 4 set_mon_vals no callback set 2024-09-15T01:34:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71e9ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.943+0000 7f71e9ffb640 10 monclient: dump: 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.944+0000 7f71f1909640 10 monclient: shutdown 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.944+0000 7f71f1909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.944+0000 7f71f1909640 1 -- 172.21.15.178:0/1814971503 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71ec1353a0 msgr2=0x7f71ec137780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.944+0000 7f71f1909640 1 --2- 172.21.15.178:0/1814971503 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71ec1353a0 0x7f71ec137780 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009850 tx=0x7f71e00090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.944+0000 7f71f1909640 1 -- 172.21.15.178:0/1814971503 shutdown_connections 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.944+0000 7f71f1909640 1 --2- 172.21.15.178:0/1814971503 >> v2:172.21.15.204:3300/0 conn(0x7f71ec137cc0 0x7f71ec13a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.944+0000 7f71f1909640 1 --2- 172.21.15.178:0/1814971503 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71ec1353a0 0x7f71ec137780 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.944+0000 7f71f1909640 1 -- 172.21.15.178:0/1814971503 >> 172.21.15.178:0/1814971503 conn(0x7f71ec068a60 msgr2=0x7f71ec0a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.944+0000 7f71f1909640 1 -- 172.21.15.178:0/1814971503 shutdown_connections 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.944+0000 7f71f1909640 1 -- 172.21.15.178:0/1814971503 wait complete. 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient: build_initial_monmap 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient: monmap: 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 1 Processor -- start 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 1 -- start start 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient: init 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient: _add_conns ranks=[7,2,3,5,4,8,0,1,6] 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71ec09c020 0x7f71ec06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient(hunting): picked mon.noname-i con 0x7f71ec09c020 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71ec1353a0 0x7f71ec06cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient(hunting): picked mon.noname-b con 0x7f71ec1353a0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec137cc0 0x7f71ec06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient(hunting): picked mon.noname-g con 0x7f71ec137cc0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec06e750 con 0x7f71ec1353a0 2024-09-15T01:34:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec09f2d0 con 0x7f71ec137cc0 2024-09-15T01:34:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec11b3e0 con 0x7f71ec09c020 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.945+0000 7f71f1909640 10 monclient(hunting): authenticate will time out at 871.695997s 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71ec1353a0 0x7f71ec06cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71ec1353a0 0x7f71ec06cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:43000/0 (socket says 172.21.15.178:43000) 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 1 -- 172.21.15.178:0/3811102709 learned_addr learned my addr 172.21.15.178:0/3811102709 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 10 monclient(hunting): get_auth_request con 0x7f71ec1353a0 auth_method 0 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eb7fe640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec137cc0 0x7f71ec06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eaffd640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71ec09c020 0x7f71ec06ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eaffd640 10 monclient(hunting): get_auth_request con 0x7f71ec09c020 auth_method 0 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eb7fe640 10 monclient(hunting): get_auth_request con 0x7f71ec137cc0 auth_method 0 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 10 monclient(hunting): handle_auth_done global_id 5114 payload 824 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 1 -- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71ec09c020 msgr2=0x7f71ec06ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71ec09c020 0x7f71ec06ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 1 -- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec137cc0 msgr2=0x7f71ec06d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec137cc0 0x7f71ec06d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 1 monclient: found mon.noname-b 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 1 -- 172.21.15.178:0/3811102709 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e0009390 con 0x7f71ec1353a0 2024-09-15T01:34:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71ea7fc640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71ec1353a0 0x7f71ec06cfa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f71e0035040 tx=0x7f71e0032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71e003b870 con 0x7f71ec1353a0 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71d2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71d2ffd640 10 monclient: dump: 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71d2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71f1909640 5 monclient: authenticate success, global_id 5114 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e003b9d0 con 0x7f71ec1353a0 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71f1909640 10 monclient: _renew_subs 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71f1909640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71f1909640 1 -- 172.21.15.178:0/3811102709 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ec119870 con 0x7f71ec1353a0 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71d2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71e00313a0 con 0x7f71ec1353a0 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71d2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71d2ffd640 10 monclient: dump: 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71f1909640 10 monclient: _renew_subs 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71e97fa640 4 set_mon_vals no callback set 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71f1909640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.946+0000 7f71f1909640 1 -- 172.21.15.178:0/3811102709 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec119360 con 0x7f71ec1353a0 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71f1909640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71f1909640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec119f30 0x7f71ec06da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71f1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71f1909640 1 -- 172.21.15.178:0/3811102709 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec0641f0 con 0x7f71ec119f30 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f71e0039040 con 0x7f71ec1353a0 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71eaffd640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec119f30 0x7f71ec06da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71eaffd640 10 monclient: get_auth_request con 0x7f71ec119f30 auth_method 0 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71eaffd640 10 monclient(hunting): handle_auth_done global_id 14304 payload 824 2024-09-15T01:34:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71d2ffd640 1 --2- 172.21.15.178:0/3811102709 >> v2:172.21.15.204:6812/28210 conn(0x7f71d80396e0 0x7f71d803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.947+0000 7f71eaffd640 1 -- 172.21.15.178:0/3811102709 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71ec04c4a0 con 0x7f71ec119f30 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5312+0+0 (secure 0 0 0) 0x7f71e006b050 con 0x7f71ec1353a0 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71eb7fe640 1 --2- 172.21.15.178:0/3811102709 >> v2:172.21.15.204:6812/28210 conn(0x7f71d80396e0 0x7f71d803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71eb7fe640 10 monclient: get_auth_request con 0x7f71d80396e0 auth_method 0 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71eaffd640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec119f30 0x7f71ec06da20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f71dc000f80 tx=0x7f71dc001a00 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71dc0133f0 con 0x7f71ec119f30 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71d2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71d2ffd640 10 monclient: dump: 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71eb7fe640 1 --2- 172.21.15.178:0/3811102709 >> v2:172.21.15.204:6812/28210 conn(0x7f71d80396e0 0x7f71d803bb90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71ec06e1d0 tx=0x7f71e40090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f71dc00fae0 con 0x7f71ec119f30 2024-09-15T01:34:26.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71d2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:26.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71d2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:26.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec119f30 msgr2=0x7f71ec06da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71d2ffd640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec119f30 0x7f71ec06da20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f71dc000f80 tx=0x7f71dc001a00 comp rx=0 tx=0).stop 2024-09-15T01:34:26.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec119f30 msgr2=0x7f71ec06da20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.948+0000 7f71d2ffd640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec119f30 0x7f71ec06da20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71f1909640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:26.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71f1909640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec135ab0 0x7f71ec135eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:26.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71f1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:26.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71f1909640 1 -- 172.21.15.178:0/3811102709 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec070b90 con 0x7f71ec135ab0 2024-09-15T01:34:26.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71eaffd640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec135ab0 0x7f71ec135eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:26.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71eaffd640 10 monclient: get_auth_request con 0x7f71ec135ab0 auth_method 0 2024-09-15T01:34:26.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71eaffd640 10 monclient(hunting): handle_auth_done global_id 14313 payload 824 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71eaffd640 1 -- 172.21.15.178:0/3811102709 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f71ec04bc80 con 0x7f71ec135ab0 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71eaffd640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec135ab0 0x7f71ec135eb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f71dc001a40 tx=0x7f71dc019000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71dc004e00 con 0x7f71ec135ab0 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71d2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.954+0000 7f71d2ffd640 10 monclient: dump: 2024-09-15T01:34:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:26.980 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f71dc00f6c0 con 0x7f71ec135ab0 2024-09-15T01:34:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71d2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71d2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec135ab0 msgr2=0x7f71ec135eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71d2ffd640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec135ab0 0x7f71ec135eb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f71dc001a40 tx=0x7f71dc019000 comp rx=0 tx=0).stop 2024-09-15T01:34:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71d2ffd640 1 -- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec135ab0 msgr2=0x7f71ec135eb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71d2ffd640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71ec135ab0 0x7f71ec135eb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:26.980 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:26.981 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:26.981 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:26.981 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:26.981 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:26.981 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:26.981 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:26.981 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:26.981 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:26.981 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:26.981 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:26.981 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:26.981 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.982 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.983 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.984 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.984 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.984 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.984 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:26.984 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:26.984 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:26.984 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:26.984 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:26.984 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:26.984 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.984 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:26.985 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:26.986 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.988 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:26.989 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.990 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.991 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.992 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.993 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.994 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:26.995 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.997 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:26.998 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:26.999 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:27.000 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:27.000 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.000 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.000 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.000 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.000 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:27.000 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:27.000 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.000 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.000 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.000 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:27.002 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:27.003 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.003 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:27.003 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:27.003 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:34:27.003 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.003 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.003 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:27.003 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:27.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71f1909640 1 -- 172.21.15.178:0/3811102709 >> v2:172.21.15.204:6812/28210 conn(0x7f71d80396e0 msgr2=0x7f71d803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71f1909640 1 --2- 172.21.15.178:0/3811102709 >> v2:172.21.15.204:6812/28210 conn(0x7f71d80396e0 0x7f71d803bb90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71ec06e1d0 tx=0x7f71e40090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71f1909640 10 monclient: shutdown 2024-09-15T01:34:27.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71f1909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:27.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71f1909640 1 -- 172.21.15.178:0/3811102709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71ec1353a0 msgr2=0x7f71ec06cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71f1909640 1 --2- 172.21.15.178:0/3811102709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71ec1353a0 0x7f71ec06cfa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f71e0035040 tx=0x7f71e0032000 comp rx=0 tx=0).stop 2024-09-15T01:34:27.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71eb7fe640 1 -- 172.21.15.178:0/3811102709 reap_dead start 2024-09-15T01:34:27.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71eb7fe640 1 -- 172.21.15.178:0/3811102709 reap_dead start 2024-09-15T01:34:27.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71f1909640 1 -- 172.21.15.178:0/3811102709 shutdown_connections 2024-09-15T01:34:27.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71f1909640 1 -- 172.21.15.178:0/3811102709 >> 172.21.15.178:0/3811102709 conn(0x7f71ec068a60 msgr2=0x7f71ec0a1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:27.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71f1909640 1 -- 172.21.15.178:0/3811102709 shutdown_connections 2024-09-15T01:34:27.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:26.955+0000 7f71f1909640 1 -- 172.21.15.178:0/3811102709 wait complete. 2024-09-15T01:34:27.005 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:27.006 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:34:27.006 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:34:27.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.097+0000 7f5faa39d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:27.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.097+0000 7f5faa39d640 10 monclient: build_initial_monmap 2024-09-15T01:34:27.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.097+0000 7f5faa39d640 10 monclient: monmap: 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:27.098584+0000 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:27.098584+0000 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:27.100 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:27.101 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:27.101 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:27.101 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:27.101 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:27.101 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.098+0000 7f5faa39d640 1 Processor -- start 2024-09-15T01:34:27.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.098+0000 7f5faa39d640 1 -- start start 2024-09-15T01:34:27.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.098+0000 7f5faa39d640 10 monclient: init 2024-09-15T01:34:27.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:27.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 10 monclient: _add_conns ranks=[2,6,0,5,4,9,3,8,1,7,10] 2024-09-15T01:34:27.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa40a2930 0x7f5fa4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 10 monclient(hunting): picked mon.noname-b con 0x7f5fa40a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:27.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5fa409faa0 0x7f5fa409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 10 monclient(hunting): picked mon.noname-d con 0x7f5fa409faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:27.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5fa40a08c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:27.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa404bf60 con 0x7f5fa40a08c0 2024-09-15T01:34:27.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa409e0f0 con 0x7f5fa40a2930 2024-09-15T01:34:27.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa409dd80 con 0x7f5fa409faa0 2024-09-15T01:34:27.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:27.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5faa39d640 10 monclient(hunting): authenticate will time out at 871.849134s 2024-09-15T01:34:27.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa40a2930 0x7f5fa4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa893a640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f5fa40a08c0 0x7f5fa40a0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:34432/0 (socket says 172.21.15.178:34432) 2024-09-15T01:34:27.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa40a2930 0x7f5fa4132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:43006/0 (socket says 172.21.15.178:43006) 2024-09-15T01:34:27.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5fa409faa0 0x7f5fa409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa893a640 1 -- 172.21.15.178:0/186004828 learned_addr learned my addr 172.21.15.178:0/186004828 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:34:27.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa3fff640 10 monclient(hunting): get_auth_request con 0x7f5fa40a2930 auth_method 0 2024-09-15T01:34:27.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 10 monclient(hunting): get_auth_request con 0x7f5fa409faa0 auth_method 0 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_done global_id 14271 payload 274 2024-09-15T01:34:27.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa37fe640 1 -- 172.21.15.178:0/186004828 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479588219 0 0) 0x7f5fa404bf60 con 0x7f5fa40a08c0 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 1 -- 172.21.15.178:0/186004828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa40a2930 msgr2=0x7f5fa4132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 1 --2- 172.21.15.178:0/186004828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa40a2930 0x7f5fa4132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 1 -- 172.21.15.178:0/186004828 >> v1:172.21.15.178:6789/0 conn(0x7f5fa40a08c0 legacy=0x7f5fa40a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 1 monclient: found mon.noname-d 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5f9bfff640 1 -- 172.21.15.178:0/186004828 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa40a2d00 con 0x7f5fa409faa0 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa3fff640 1 --2- 172.21.15.178:0/186004828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa40a2930 0x7f5fa4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.099+0000 7f5fa37fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5f9bfff640 1 --2- 172.21.15.178:0/186004828 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5fa409faa0 0x7f5fa409fe70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009880 tx=0x7f5f8c0090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=28fab870a4db8f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5fa37fe640 1 -- 172.21.15.178:0/186004828 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5f8c02ac50 con 0x7f5fa409faa0 2024-09-15T01:34:27.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5fa37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5fa37fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5fa37fe640 10 monclient: dump: 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.114 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5fa37fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5fa37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5faa39d640 5 monclient: authenticate success, global_id 14271 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5faa39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5fa37fe640 1 -- 172.21.15.178:0/186004828 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c02add0 con 0x7f5fa409faa0 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5fa37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5faa39d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5faa39d640 4 set_mon_vals no callback set 2024-09-15T01:34:27.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5fa37fe640 1 -- 172.21.15.178:0/186004828 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5f8c031ca0 con 0x7f5fa409faa0 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5fa37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5fa37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.100+0000 7f5fa37fe640 10 monclient: dump: 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.116 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.117 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.117 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.117 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.117 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.117 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 10 monclient: shutdown 2024-09-15T01:34:27.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:27.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 1 -- 172.21.15.178:0/186004828 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5fa409faa0 msgr2=0x7f5fa409fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 1 --2- 172.21.15.178:0/186004828 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5fa409faa0 0x7f5fa409fe70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009880 tx=0x7f5f8c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 1 -- 172.21.15.178:0/186004828 shutdown_connections 2024-09-15T01:34:27.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 1 --2- 172.21.15.178:0/186004828 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5fa409faa0 0x7f5fa409fe70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 1 --2- 172.21.15.178:0/186004828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa40a2930 0x7f5fa4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 1 -- 172.21.15.178:0/186004828 >> 172.21.15.178:0/186004828 conn(0x7f5fa40a5ed0 msgr2=0x7f5fa40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 1 -- 172.21.15.178:0/186004828 shutdown_connections 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 1 -- 172.21.15.178:0/186004828 wait complete. 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 10 monclient: build_initial_monmap 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 10 monclient: monmap: 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:27.118 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 1 Processor -- start 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 1 -- start start 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.101+0000 7f5faa39d640 10 monclient: init 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 10 monclient: _add_conns ranks=[2,7,4,1,0,6,3,5,8] 2024-09-15T01:34:27.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa409faa0 0x7f5fa409b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 10 monclient(hunting): picked mon.noname-b con 0x7f5fa409faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5fa40a08c0 0x7f5fa4131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 10 monclient(hunting): picked mon.noname-i con 0x7f5fa40a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5fa40a2930 0x7f5fa4141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 10 monclient(hunting): picked mon.noname-c con 0x7f5fa40a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa411ed20 con 0x7f5fa409faa0 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa411ea40 con 0x7f5fa40a2930 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa411f000 con 0x7f5fa40a08c0 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:27.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5faa39d640 10 monclient(hunting): authenticate will time out at 871.852386s 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa409faa0 0x7f5fa409b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa893a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5fa40a2930 0x7f5fa4141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa409faa0 0x7f5fa409b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:43010/0 (socket says 172.21.15.178:43010) 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 1 -- 172.21.15.178:0/2556200980 learned_addr learned my addr 172.21.15.178:0/2556200980 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 10 monclient(hunting): get_auth_request con 0x7f5fa409faa0 auth_method 0 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa893a640 10 monclient(hunting): get_auth_request con 0x7f5fa40a2930 auth_method 0 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa893a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa893a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa893a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa893a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa893a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa893a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5f9bfff640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5fa40a08c0 0x7f5fa4131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 10 monclient(hunting): handle_auth_done global_id 5132 payload 824 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 1 -- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5fa40a08c0 msgr2=0x7f5fa4131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5fa40a08c0 0x7f5fa4131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 1 -- 172.21.15.178:0/2556200980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5fa40a2930 msgr2=0x7f5fa4141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5fa40a2930 0x7f5fa4141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:27.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 1 monclient: found mon.noname-b 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 1 -- 172.21.15.178:0/2556200980 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f94009390 con 0x7f5fa409faa0 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5f9bfff640 10 monclient: get_auth_request con 0x7f5fa40a08c0 auth_method 0 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa893a640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5fa40a2930 0x7f5fa4141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5f9bfff640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5fa40a08c0 0x7f5fa4131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.102+0000 7f5fa3fff640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa409faa0 0x7f5fa409b0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400ec80 tx=0x7f5f940090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5f9400eec0 con 0x7f5fa409faa0 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 10 monclient: dump: 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.123 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.124 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f94013470 con 0x7f5fa409faa0 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5faa39d640 5 monclient: authenticate success, global_id 5132 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5faa39d640 10 monclient: _renew_subs 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5f94013a60 con 0x7f5fa409faa0 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5faa39d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5faa39d640 1 -- 172.21.15.178:0/2556200980 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c009390 con 0x7f5fa409faa0 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 10 monclient: dump: 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.125 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.126 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa2ffd640 4 set_mon_vals no callback set 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5faa39d640 10 monclient: _renew_subs 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5faa39d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5faa39d640 1 -- 172.21.15.178:0/2556200980 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa411dae0 con 0x7f5fa409faa0 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5f94013bc0 con 0x7f5fa409faa0 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5faa39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5faa39d640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa411ca00 0x7f5fa4142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5faa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.103+0000 7f5faa39d640 1 -- 172.21.15.178:0/2556200980 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa404c4a0 con 0x7f5fa411ca00 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5fa0ff9640 1 --2- 172.21.15.178:0/2556200980 >> v2:172.21.15.204:6812/28210 conn(0x7f5f840396e0 0x7f5f8403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5312+0+0 (secure 0 0 0) 0x7f5f9404be00 con 0x7f5fa409faa0 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5f9bfff640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa411ca00 0x7f5fa4142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5fa893a640 1 --2- 172.21.15.178:0/2556200980 >> v2:172.21.15.204:6812/28210 conn(0x7f5f840396e0 0x7f5f8403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5f9bfff640 10 monclient: get_auth_request con 0x7f5fa411ca00 auth_method 0 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5f9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5f9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5f9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5fa893a640 10 monclient: get_auth_request con 0x7f5f840396e0 auth_method 0 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_done global_id 5171 payload 824 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5f9bfff640 1 -- 172.21.15.178:0/2556200980 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5fa405ed70 con 0x7f5fa411ca00 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5fa893a640 1 --2- 172.21.15.178:0/2556200980 >> v2:172.21.15.204:6812/28210 conn(0x7f5f840396e0 0x7f5f8403bb90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c001760 tx=0x7f5f9c009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.104+0000 7f5f9bfff640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa411ca00 0x7f5fa4142200 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40a3290 tx=0x7f5f8c032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.105+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5f8c031820 con 0x7f5fa411ca00 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.105+0000 7f5fa0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.105+0000 7f5fa0ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.105+0000 7f5fa0ff9640 10 monclient: dump: 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.129 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.130 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.130 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.130 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.130 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.130 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.105+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f5f8c0313a0 con 0x7f5fa411ca00 2024-09-15T01:34:27.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.105+0000 7f5fa0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:27.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.105+0000 7f5fa0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:27.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.105+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa411ca00 msgr2=0x7f5fa4142200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.105+0000 7f5fa0ff9640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa411ca00 0x7f5fa4142200 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40a3290 tx=0x7f5f8c032000 comp rx=0 tx=0).stop 2024-09-15T01:34:27.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.105+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa411ca00 msgr2=0x7f5fa4142200 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:27.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.105+0000 7f5fa0ff9640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa411ca00 0x7f5fa4142200 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40a3290 tx=0x7f5f8c032000 comp rx=0 tx=0).stop 2024-09-15T01:34:27.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.110+0000 7f5faa39d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:27.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.110+0000 7f5faa39d640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa4131e60 0x7f5fa4142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.110+0000 7f5faa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.110+0000 7f5faa39d640 1 -- 172.21.15.178:0/2556200980 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa411ebb0 con 0x7f5fa4131e60 2024-09-15T01:34:27.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5f9bfff640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa4131e60 0x7f5fa4142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5f9bfff640 10 monclient: get_auth_request con 0x7f5fa4131e60 auth_method 0 2024-09-15T01:34:27.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5f9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5f9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5f9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_done global_id 5180 payload 824 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5f9bfff640 1 -- 172.21.15.178:0/2556200980 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5fa404bc80 con 0x7f5fa4131e60 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5f9bfff640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa4131e60 0x7f5fa4142740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009720 tx=0x7f5f8c03b000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5f8c03c880 con 0x7f5fa4131e60 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5fa0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5fa0ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.111+0000 7f5fa0ff9640 10 monclient: dump: 2024-09-15T01:34:27.133 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.134 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.135 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9313 (secure 0 0 0) 0x7f5f8c03c400 con 0x7f5fa4131e60 2024-09-15T01:34:27.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5fa0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:27.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5fa0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:27.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa4131e60 msgr2=0x7f5fa4142740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5fa0ff9640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa4131e60 0x7f5fa4142740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009720 tx=0x7f5f8c03b000 comp rx=0 tx=0).stop 2024-09-15T01:34:27.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5fa0ff9640 1 -- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa4131e60 msgr2=0x7f5fa4142740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:27.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5fa0ff9640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5fa4131e60 0x7f5fa4142740 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009720 tx=0x7f5f8c03b000 comp rx=0 tx=0).stop 2024-09-15T01:34:27.135 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:27.135 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:27.136 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:27.137 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.138 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:27.139 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:27.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.141 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:27.142 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.145 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.146 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:27.147 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.148 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:27.150 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.151 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.152 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.153 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.154 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:27.155 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:27.156 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.158 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:27.158 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5faa39d640 1 -- 172.21.15.178:0/2556200980 >> v2:172.21.15.204:6812/28210 conn(0x7f5f840396e0 msgr2=0x7f5f8403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5faa39d640 1 --2- 172.21.15.178:0/2556200980 >> v2:172.21.15.204:6812/28210 conn(0x7f5f840396e0 0x7f5f8403bb90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c001760 tx=0x7f5f9c009040 comp rx=0 tx=0).stop 2024-09-15T01:34:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5faa39d640 10 monclient: shutdown 2024-09-15T01:34:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5faa39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5faa39d640 1 -- 172.21.15.178:0/2556200980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa409faa0 msgr2=0x7f5fa409b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5faa39d640 1 --2- 172.21.15.178:0/2556200980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa409faa0 0x7f5fa409b0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400ec80 tx=0x7f5f940090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5fa893a640 1 -- 172.21.15.178:0/2556200980 reap_dead start 2024-09-15T01:34:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5fa893a640 1 -- 172.21.15.178:0/2556200980 reap_dead start 2024-09-15T01:34:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5faa39d640 1 -- 172.21.15.178:0/2556200980 shutdown_connections 2024-09-15T01:34:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5faa39d640 1 -- 172.21.15.178:0/2556200980 >> 172.21.15.178:0/2556200980 conn(0x7f5fa40a5ed0 msgr2=0x7f5fa4135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:27.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5faa39d640 1 -- 172.21.15.178:0/2556200980 shutdown_connections 2024-09-15T01:34:27.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.112+0000 7f5faa39d640 1 -- 172.21.15.178:0/2556200980 wait complete. 2024-09-15T01:34:27.160 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:27.160 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:34:27.160 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:34:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.252+0000 7f032c09f640 10 monclient: get_monmap_and_config 2024-09-15T01:34:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.252+0000 7f032c09f640 10 monclient: build_initial_monmap 2024-09-15T01:34:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.252+0000 7f032c09f640 10 monclient: monmap: 2024-09-15T01:34:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:27.253176+0000 2024-09-15T01:34:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:27.253176+0000 2024-09-15T01:34:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:27.255 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.252+0000 7f032c09f640 1 Processor -- start 2024-09-15T01:34:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 1 -- start start 2024-09-15T01:34:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 10 monclient: init 2024-09-15T01:34:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 10 monclient: _add_conns ranks=[3,7,6,1,0,9,5,4,10,8,2] 2024-09-15T01:34:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03240a2930 0x7f0324132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 10 monclient(hunting): picked mon.noname-g con 0x7f03240a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032409faa0 0x7f032409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 10 monclient(hunting): picked mon.noname-i con 0x7f032409faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a08c0 0x7f03240a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 10 monclient(hunting): picked mon.noname-d con 0x7f03240a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032409e0f0 con 0x7f03240a2930 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f032409dd80 con 0x7f03240a08c0 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f03240a2d00 con 0x7f032409faa0 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032c09f640 10 monclient(hunting): authenticate will time out at 872.003545s 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a08c0 0x7f03240a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a08c0 0x7f03240a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42786/0 (socket says 172.21.15.178:42786) 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 1 -- 172.21.15.178:0/2472544058 learned_addr learned my addr 172.21.15.178:0/2472544058 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 10 monclient(hunting): get_auth_request con 0x7f03240a08c0 auth_method 0 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 10 monclient(hunting): handle_auth_done global_id 14280 payload 274 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032963a640 1 --2- 172.21.15.178:0/2472544058 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032409faa0 0x7f032409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f0329e3b640 1 --2- 172.21.15.178:0/2472544058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03240a2930 0x7f0324132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 1 -- 172.21.15.178:0/2472544058 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032409faa0 msgr2=0x7f032409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 1 --2- 172.21.15.178:0/2472544058 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032409faa0 0x7f032409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 1 -- 172.21.15.178:0/2472544058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03240a2930 msgr2=0x7f0324132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 1 --2- 172.21.15.178:0/2472544058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03240a2930 0x7f0324132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 1 monclient: found mon.noname-d 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032a63c640 1 -- 172.21.15.178:0/2472544058 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032404bf60 con 0x7f03240a08c0 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032963a640 10 monclient: get_auth_request con 0x7f032409faa0 auth_method 0 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f032963a640 1 --2- 172.21.15.178:0/2472544058 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032409faa0 0x7f032409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f0329e3b640 10 monclient: get_auth_request con 0x7f03240a2930 auth_method 0 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.253+0000 7f0329e3b640 1 --2- 172.21.15.178:0/2472544058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03240a2930 0x7f0324132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f032a63c640 1 --2- 172.21.15.178:0/2472544058 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a08c0 0x7f03240a0d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f031c009f50 tx=0x7f031c009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c14f0be353f1ef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f0328e39640 1 -- 172.21.15.178:0/2472544058 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f031c007880 con 0x7f03240a08c0 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f0328e39640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f0328e39640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f0328e39640 10 monclient: dump: 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.268 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f0328e39640 10 monclient: _finish_auth 0 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f0328e39640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f0328e39640 1 -- 172.21.15.178:0/2472544058 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c02d050 con 0x7f03240a08c0 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f032c09f640 5 monclient: authenticate success, global_id 14280 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f032c09f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f0328e39640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f0328e39640 1 -- 172.21.15.178:0/2472544058 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f031c030360 con 0x7f03240a08c0 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f032c09f640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f032c09f640 4 set_mon_vals no callback set 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f0328e39640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f0328e39640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.254+0000 7f0328e39640 10 monclient: dump: 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.269 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.270 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 10 monclient: shutdown 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 1 -- 172.21.15.178:0/2472544058 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a08c0 msgr2=0x7f03240a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 1 --2- 172.21.15.178:0/2472544058 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a08c0 0x7f03240a0d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f031c009f50 tx=0x7f031c009840 comp rx=0 tx=0).stop 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 1 -- 172.21.15.178:0/2472544058 shutdown_connections 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 1 --2- 172.21.15.178:0/2472544058 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a08c0 0x7f03240a0d30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 1 --2- 172.21.15.178:0/2472544058 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032409faa0 0x7f032409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 1 --2- 172.21.15.178:0/2472544058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03240a2930 0x7f0324132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 1 -- 172.21.15.178:0/2472544058 >> 172.21.15.178:0/2472544058 conn(0x7f03240a5ed0 msgr2=0x7f03240a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 1 -- 172.21.15.178:0/2472544058 shutdown_connections 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 1 -- 172.21.15.178:0/2472544058 wait complete. 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 10 monclient: build_initial_monmap 2024-09-15T01:34:27.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.255+0000 7f032c09f640 10 monclient: monmap: 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 1 Processor -- start 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 1 -- start start 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 10 monclient: init 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 10 monclient: _add_conns ranks=[0,4,6,1,8,7,3,2,5] 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f032409faa0 0x7f0324131800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 10 monclient(hunting): picked mon.noname-a con 0x7f032409faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f03240a08c0 0x7f032411e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 10 monclient(hunting): picked mon.noname-c con 0x7f03240a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a2930 0x7f032411e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 10 monclient(hunting): picked mon.noname-d con 0x7f03240a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032411feb0 con 0x7f032409faa0 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03240a2390 con 0x7f03240a08c0 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f032409d630 con 0x7f03240a2930 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032c09f640 10 monclient(hunting): authenticate will time out at 872.006690s 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032963a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f03240a08c0 0x7f032411e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032a63c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a2930 0x7f032411e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f0329e3b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f032409faa0 0x7f0324131800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032a63c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a2930 0x7f032411e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42796/0 (socket says 172.21.15.178:42796) 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f0329e3b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f032409faa0 0x7f0324131800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:58610/0 (socket says 172.21.15.178:58610) 2024-09-15T01:34:27.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032963a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f03240a08c0 0x7f032411e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:33270/0 (socket says 172.21.15.178:33270) 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032a63c640 1 -- 172.21.15.178:0/3936041022 learned_addr learned my addr 172.21.15.178:0/3936041022 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032a63c640 10 monclient(hunting): get_auth_request con 0x7f03240a2930 auth_method 0 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032a63c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032a63c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032a63c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f0329e3b640 10 monclient(hunting): get_auth_request con 0x7f032409faa0 auth_method 0 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f0329e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f0329e3b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f0329e3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032963a640 10 monclient(hunting): get_auth_request con 0x7f03240a08c0 auth_method 0 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032963a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032963a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032963a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032a63c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032a63c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032a63c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f0329e3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f0329e3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f0329e3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032963a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032963a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.256+0000 7f032963a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032a63c640 10 monclient(hunting): handle_auth_done global_id 14289 payload 824 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032a63c640 1 -- 172.21.15.178:0/3936041022 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f03240a08c0 msgr2=0x7f032411e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032a63c640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f03240a08c0 0x7f032411e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032a63c640 1 -- 172.21.15.178:0/3936041022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f032409faa0 msgr2=0x7f0324131800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032a63c640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f032409faa0 0x7f0324131800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032a63c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:27.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032a63c640 1 monclient: found mon.noname-d 2024-09-15T01:34:27.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032a63c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:27.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032a63c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:34:27.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032a63c640 1 -- 172.21.15.178:0/3936041022 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c009ae0 con 0x7f03240a2930 2024-09-15T01:34:27.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0329e3b640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f032409faa0 0x7f0324131800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:27.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032963a640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f03240a08c0 0x7f032411e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:27.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032a63c640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a2930 0x7f032411e870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f031c009fb0 tx=0x7f031c036000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f031c008b20 con 0x7f03240a2930 2024-09-15T01:34:27.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0315ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0315ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:34:27.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0315ffb640 10 monclient: dump: 2024-09-15T01:34:27.277 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.278 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0315ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0315ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c008ca0 con 0x7f03240a2930 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032c09f640 5 monclient: authenticate success, global_id 14289 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032c09f640 10 monclient: _renew_subs 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032c09f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032c09f640 1 -- 172.21.15.178:0/3936041022 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0324121c30 con 0x7f03240a2930 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0315ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f031c008e10 con 0x7f03240a2930 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0315ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0315ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:34:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0315ffb640 10 monclient: dump: 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.280 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032c09f640 10 monclient: _renew_subs 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032c09f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f032c09f640 1 -- 172.21.15.178:0/3936041022 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0324133e00 con 0x7f03240a2930 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.257+0000 7f0317fff640 4 set_mon_vals no callback set 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f032c09f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f031c038050 con 0x7f03240a2930 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f032c09f640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0324131d40 0x7f032411edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f032c09f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f032c09f640 1 -- 172.21.15.178:0/3936041022 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0324121af0 con 0x7f0324131d40 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f0315ffb640 1 --2- 172.21.15.178:0/3936041022 >> v2:172.21.15.204:6812/28210 conn(0x7f03080397b0 0x7f030803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5312+0+0 (secure 0 0 0) 0x7f031c06ba40 con 0x7f03240a2930 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f0329e3b640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0324131d40 0x7f032411edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f0329e3b640 10 monclient: get_auth_request con 0x7f0324131d40 auth_method 0 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f0329e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f0329e3b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f0329e3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f0329e3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f0329e3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f0329e3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f032963a640 1 --2- 172.21.15.178:0/3936041022 >> v2:172.21.15.204:6812/28210 conn(0x7f03080397b0 0x7f030803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.258+0000 7f032963a640 10 monclient: get_auth_request con 0x7f03080397b0 auth_method 0 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0329e3b640 10 monclient(hunting): handle_auth_done global_id 5263 payload 824 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0329e3b640 1 -- 172.21.15.178:0/3936041022 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f032404c4a0 con 0x7f0324131d40 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0329e3b640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0324131d40 0x7f032411edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f031800aeb0 tx=0x7f0318008040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f032963a640 1 --2- 172.21.15.178:0/3936041022 >> v2:172.21.15.204:6812/28210 conn(0x7f03080397b0 0x7f030803bc60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f03240a6ec0 tx=0x7f0320008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03180138c0 con 0x7f0324131d40 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0315ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0315ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0315ffb640 10 monclient: dump: 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.284 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f031800c040 con 0x7f0324131d40 2024-09-15T01:34:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0315ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0315ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0324131d40 msgr2=0x7f032411edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0315ffb640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0324131d40 0x7f032411edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f031800aeb0 tx=0x7f0318008040 comp rx=0 tx=0).stop 2024-09-15T01:34:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0324131d40 msgr2=0x7f032411edb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.259+0000 7f0315ffb640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0324131d40 0x7f032411edb0 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f031800aeb0 tx=0x7f0318008040 comp rx=0 tx=0).stop 2024-09-15T01:34:27.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.265+0000 7f032c09f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:27.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.265+0000 7f032c09f640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032411f2f0 0x7f0324099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.265+0000 7f032c09f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.265+0000 7f032c09f640 1 -- 172.21.15.178:0/3936041022 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0324132110 con 0x7f032411f2f0 2024-09-15T01:34:27.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.265+0000 7f0329e3b640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032411f2f0 0x7f0324099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.265+0000 7f0329e3b640 10 monclient: get_auth_request con 0x7f032411f2f0 auth_method 0 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.265+0000 7f0329e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.265+0000 7f0329e3b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.265+0000 7f0329e3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.265+0000 7f0329e3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.265+0000 7f0329e3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.265+0000 7f0329e3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0329e3b640 10 monclient(hunting): handle_auth_done global_id 5272 payload 824 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0329e3b640 1 -- 172.21.15.178:0/3936041022 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f032404bc80 con 0x7f032411f2f0 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0329e3b640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032411f2f0 0x7f0324099a50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0318000c00 tx=0x7f031801d000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03180263d0 con 0x7f032411f2f0 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0315ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0315ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:34:27.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0315ffb640 10 monclient: dump: 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.288 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.289 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.289 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.289 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9481 (secure 0 0 0) 0x7f031800daa0 con 0x7f032411f2f0 2024-09-15T01:34:27.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0315ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:27.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0315ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:27.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032411f2f0 msgr2=0x7f0324099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0315ffb640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032411f2f0 0x7f0324099a50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0318000c00 tx=0x7f031801d000 comp rx=0 tx=0).stop 2024-09-15T01:34:27.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.266+0000 7f0315ffb640 1 -- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032411f2f0 msgr2=0x7f0324099a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:27.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f0315ffb640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f032411f2f0 0x7f0324099a50 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0318000c00 tx=0x7f031801d000 comp rx=0 tx=0).stop 2024-09-15T01:34:27.289 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 38, 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:34:27.290 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:27.291 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:27.292 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:27.293 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:27.294 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:27.295 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.296 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.298 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:27.302 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.303 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:27.304 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.306 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.307 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:27.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:27.309 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:34:27.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:27.311 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:34:27.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:27.312 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:27.312 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:27.312 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:27.312 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:27.312 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:27.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:27.312 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:27.312 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:27.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f032c09f640 1 -- 172.21.15.178:0/3936041022 >> v2:172.21.15.204:6812/28210 conn(0x7f03080397b0 msgr2=0x7f030803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f032c09f640 1 --2- 172.21.15.178:0/3936041022 >> v2:172.21.15.204:6812/28210 conn(0x7f03080397b0 0x7f030803bc60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f03240a6ec0 tx=0x7f0320008040 comp rx=0 tx=0).stop 2024-09-15T01:34:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f032c09f640 10 monclient: shutdown 2024-09-15T01:34:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f032c09f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f032c09f640 1 -- 172.21.15.178:0/3936041022 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a2930 msgr2=0x7f032411e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f032c09f640 1 --2- 172.21.15.178:0/3936041022 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03240a2930 0x7f032411e870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f031c009fb0 tx=0x7f031c036000 comp rx=0 tx=0).stop 2024-09-15T01:34:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f032a63c640 1 -- 172.21.15.178:0/3936041022 reap_dead start 2024-09-15T01:34:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f032a63c640 1 -- 172.21.15.178:0/3936041022 reap_dead start 2024-09-15T01:34:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f032c09f640 1 -- 172.21.15.178:0/3936041022 shutdown_connections 2024-09-15T01:34:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f032c09f640 1 -- 172.21.15.178:0/3936041022 >> 172.21.15.178:0/3936041022 conn(0x7f03240a5ed0 msgr2=0x7f032409e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f032c09f640 1 -- 172.21.15.178:0/3936041022 shutdown_connections 2024-09-15T01:34:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.267+0000 7f032c09f640 1 -- 172.21.15.178:0/3936041022 wait complete. 2024-09-15T01:34:27.315 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:34:27.315 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['i', 'b', 'd'] 2024-09-15T01:34:27.315 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T01:34:27.315 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-15T01:34:27.315 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-15T01:34:27.315 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-15T01:34:27.315 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:27.327 INFO:tasks.ceph.mon.i:Stopped 2024-09-15T01:34:27.327 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T01:34:27.327 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T01:34:27.327 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T01:34:27.327 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:27.338 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T01:34:27.338 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d 2024-09-15T01:34:27.338 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-15T01:34:27.338 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-09-15T01:34:27.338 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:34:27.388 INFO:tasks.ceph.mon.d:Stopped 2024-09-15T01:34:27.389 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-09-15T01:34:27.389 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:34:27.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.518+0000 7f5926d54640 10 monclient: get_monmap_and_config 2024-09-15T01:34:27.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.518+0000 7f5926d54640 10 monclient: build_initial_monmap 2024-09-15T01:34:27.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.518+0000 7f5926d54640 10 monclient: monmap: 2024-09-15T01:34:27.520 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:27.520 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:27.519586+0000 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:27.519586+0000 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:27.521 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:27.522 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:27.522 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 1 Processor -- start 2024-09-15T01:34:27.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 1 -- start start 2024-09-15T01:34:27.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 10 monclient: init 2024-09-15T01:34:27.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:27.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 10 monclient: _add_conns ranks=[7,9,6,2,1,0,10,8,3,4,5] 2024-09-15T01:34:27.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f59200a2930 0x7f5920132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 10 monclient(hunting): picked mon.noname-i con 0x7f59200a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:27.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f592009faa0 0x7f592009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 10 monclient(hunting): picked mon.noname-a con 0x7f592009faa0 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:27.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f59200a08c0 0x7f59200a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 10 monclient(hunting): picked mon.noname-d con 0x7f59200a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592009e0f0 con 0x7f592009faa0 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f592009dd80 con 0x7f59200a08c0 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f59200a2d00 con 0x7f59200a2930 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.519+0000 7f5926d54640 10 monclient(hunting): authenticate will time out at 872.269989s 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f59252f1640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f59200a08c0 msgr2=0x7f59200a0d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f5924af0640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f59200a2930 msgr2=0x7f5920132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f59252f1640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f59200a08c0 0x7f59200a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:27.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f5924af0640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f59200a2930 0x7f5920132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f592009faa0 0x7f592009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f592009faa0 0x7f592009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:58624/0 (socket says 172.21.15.178:58624) 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 1 -- 172.21.15.178:0/3709557391 learned_addr learned my addr 172.21.15.178:0/3709557391 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 10 monclient(hunting): get_auth_request con 0x7f592009faa0 auth_method 0 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 10 monclient(hunting): handle_auth_done global_id 5382 payload 274 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 1 -- 172.21.15.178:0/3709557391 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f59200a2930 msgr2=0x7f5920132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 1 --2- 172.21.15.178:0/3709557391 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f59200a2930 0x7f5920132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 1 -- 172.21.15.178:0/3709557391 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f59200a08c0 msgr2=0x7f59200a0d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 1 --2- 172.21.15.178:0/3709557391 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f59200a08c0 0x7f59200a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 1 monclient: found mon.noname-a 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.520+0000 7f591ffff640 1 -- 172.21.15.178:0/3709557391 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592004bf60 con 0x7f592009faa0 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591ffff640 1 --2- 172.21.15.178:0/3709557391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f592009faa0 0x7f592009fe70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f590c009910 tx=0x7f590c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f5eab89a32ee18 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591f7fe640 1 -- 172.21.15.178:0/3709557391 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f590c02acd0 con 0x7f592009faa0 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591f7fe640 10 monclient: dump: 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.533 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:27.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:27.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591f7fe640 1 -- 172.21.15.178:0/3709557391 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c02ae50 con 0x7f592009faa0 2024-09-15T01:34:27.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 5 monclient: authenticate success, global_id 5382 2024-09-15T01:34:27.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:27.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:27.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:27.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591f7fe640 1 -- 172.21.15.178:0/3709557391 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f590c031ca0 con 0x7f592009faa0 2024-09-15T01:34:27.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 4 set_mon_vals no callback set 2024-09-15T01:34:27.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:27.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f591f7fe640 10 monclient: dump: 2024-09-15T01:34:27.535 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 10 monclient: shutdown 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 1 -- 172.21.15.178:0/3709557391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f592009faa0 msgr2=0x7f592009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 1 --2- 172.21.15.178:0/3709557391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f592009faa0 0x7f592009fe70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f590c009910 tx=0x7f590c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 1 -- 172.21.15.178:0/3709557391 shutdown_connections 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 1 --2- 172.21.15.178:0/3709557391 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f59200a08c0 0x7f59200a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 1 --2- 172.21.15.178:0/3709557391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f592009faa0 0x7f592009fe70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 1 --2- 172.21.15.178:0/3709557391 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f59200a2930 0x7f5920132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 1 -- 172.21.15.178:0/3709557391 >> 172.21.15.178:0/3709557391 conn(0x7f59200a5ed0 msgr2=0x7f59200a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 1 -- 172.21.15.178:0/3709557391 shutdown_connections 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.521+0000 7f5926d54640 1 -- 172.21.15.178:0/3709557391 wait complete. 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.522+0000 7f5926d54640 10 monclient: build_initial_monmap 2024-09-15T01:34:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.522+0000 7f5926d54640 10 monclient: monmap: 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.522+0000 7f5926d54640 1 Processor -- start 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.522+0000 7f5926d54640 1 -- start start 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.522+0000 7f5926d54640 10 monclient: init 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 10 monclient: _add_conns ranks=[5,7,8,4,1,6,3,2,0] 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f592009faa0 0x7f592009b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 10 monclient(hunting): picked mon.noname-h con 0x7f592009faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f59200a08c0 0x7f5920131770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 10 monclient(hunting): picked mon.noname-i con 0x7f59200a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f59200a2930 0x7f592011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 10 monclient(hunting): picked mon.noname-e con 0x7f59200a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5920142df0 con 0x7f592009faa0 2024-09-15T01:34:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f59200a2390 con 0x7f59200a08c0 2024-09-15T01:34:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f592009d630 con 0x7f59200a2930 2024-09-15T01:34:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5926d54640 10 monclient(hunting): authenticate will time out at 872.273300s 2024-09-15T01:34:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f59200a2930 0x7f592011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f591ffff640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f59200a08c0 msgr2=0x7f5920131770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f591ffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f59200a08c0 0x7f5920131770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f59200a2930 0x7f592011c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:54350/0 (socket says 172.21.15.178:54350) 2024-09-15T01:34:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 1 -- 172.21.15.178:0/2777645598 learned_addr learned my addr 172.21.15.178:0/2777645598 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 10 monclient(hunting): get_auth_request con 0x7f59200a2930 auth_method 0 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5924af0640 1 --2- 172.21.15.178:0/2777645598 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f592009faa0 0x7f592009b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5924af0640 10 monclient(hunting): get_auth_request con 0x7f592009faa0 auth_method 0 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5924af0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5924af0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5924af0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5924af0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5924af0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f5924af0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 10 monclient(hunting): handle_auth_done global_id 5291 payload 824 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 1 -- 172.21.15.178:0/2777645598 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f59200a08c0 msgr2=0x7f5920131770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 1 --2- 172.21.15.178:0/2777645598 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f59200a08c0 0x7f5920131770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 1 -- 172.21.15.178:0/2777645598 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f592009faa0 msgr2=0x7f592009b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 1 --2- 172.21.15.178:0/2777645598 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f592009faa0 0x7f592009b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 10 monclient: _finish_hunting 0 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 1 monclient: found mon.noname-e 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 1 -- 172.21.15.178:0/2777645598 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c009450 con 0x7f59200a2930 2024-09-15T01:34:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.523+0000 7f59252f1640 1 --2- 172.21.15.178:0/2777645598 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f59200a2930 0x7f592011c530 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f591800bee0 tx=0x7f59180096e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 1 -- 172.21.15.178:0/2777645598 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f591800cd00 con 0x7f59200a2930 2024-09-15T01:34:27.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:27.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 10 monclient: dump: 2024-09-15T01:34:27.543 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.543 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.543 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.543 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.543 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.543 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.543 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.543 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 1 -- 172.21.15.178:0/2777645598 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591800ce80 con 0x7f59200a2930 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f5926d54640 5 monclient: authenticate success, global_id 5291 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f5926d54640 10 monclient: _renew_subs 2024-09-15T01:34:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f5926d54640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f5926d54640 1 -- 172.21.15.178:0/2777645598 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592009bdf0 con 0x7f59200a2930 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 1 -- 172.21.15.178:0/2777645598 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5918006a50 con 0x7f59200a2930 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591effd640 4 set_mon_vals no callback set 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 10 monclient: dump: 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f5926d54640 10 monclient: _renew_subs 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f5926d54640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f5926d54640 1 -- 172.21.15.178:0/2777645598 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592011d9a0 con 0x7f59200a2930 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.524+0000 7f591cff9640 1 -- 172.21.15.178:0/2777645598 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5918012020 con 0x7f59200a2930 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.525+0000 7f5926d54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.525+0000 7f5926d54640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.525+0000 7f5926d54640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.525+0000 7f5926d54640 1 -- 172.21.15.178:0/2777645598 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592011ed90 con 0x7f59200a2930 2024-09-15T01:34:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.525+0000 7f591cff9640 1 --2- 172.21.15.178:0/2777645598 >> v2:172.21.15.204:6812/28210 conn(0x7f59040397b0 0x7f590403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.525+0000 7f591cff9640 1 -- 172.21.15.178:0/2777645598 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5312+0+0 (secure 0 0 0) 0x7f591804b920 con 0x7f59200a2930 2024-09-15T01:34:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.525+0000 7f5924af0640 1 --2- 172.21.15.178:0/2777645598 >> v2:172.21.15.204:6812/28210 conn(0x7f59040397b0 0x7f590403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.525+0000 7f5924af0640 10 monclient: get_auth_request con 0x7f59040397b0 auth_method 0 2024-09-15T01:34:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.525+0000 7f5924af0640 1 --2- 172.21.15.178:0/2777645598 >> v2:172.21.15.204:6812/28210 conn(0x7f59040397b0 0x7f590403bc60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f59140038f0 tx=0x7f59140090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.529+0000 7f591cff9640 1 -- 172.21.15.178:0/2777645598 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f59180138c0 con 0x7f59200a2930 2024-09-15T01:34:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.529+0000 7f591cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.529+0000 7f591cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:27.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.747+0000 7f5926d54640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:34:27.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.747+0000 7f5926d54640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:34:27.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.747+0000 7f5926d54640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:27.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.747+0000 7f5926d54640 1 -- 172.21.15.178:0/2777645598 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f592004b720 con 0x7f59200a2930 2024-09-15T01:34:27.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.747+0000 7f591cff9640 1 -- 172.21.15.178:0/2777645598 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7f59180132e0 con 0x7f59200a2930 2024-09-15T01:34:27.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.747+0000 7f591cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:34:27.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.747+0000 7f591cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:27.749 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":38,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":5,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:34:27.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 1 -- 172.21.15.178:0/2777645598 >> v2:172.21.15.204:6812/28210 conn(0x7f59040397b0 msgr2=0x7f590403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 1 --2- 172.21.15.178:0/2777645598 >> v2:172.21.15.204:6812/28210 conn(0x7f59040397b0 0x7f590403bc60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f59140038f0 tx=0x7f59140090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 10 monclient: shutdown 2024-09-15T01:34:27.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:27.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 1 -- 172.21.15.178:0/2777645598 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f59200a2930 msgr2=0x7f592011c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:27.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 1 --2- 172.21.15.178:0/2777645598 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f59200a2930 0x7f592011c530 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f591800bee0 tx=0x7f59180096e0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 1 -- 172.21.15.178:0/2777645598 shutdown_connections 2024-09-15T01:34:27.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 1 --2- 172.21.15.178:0/2777645598 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f59200a2930 0x7f592011c530 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 1 --2- 172.21.15.178:0/2777645598 >> v2:172.21.15.204:6812/28210 conn(0x7f59040397b0 0x7f590403bc60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 1 --2- 172.21.15.178:0/2777645598 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f59200a08c0 0x7f5920131770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 1 --2- 172.21.15.178:0/2777645598 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f592009faa0 0x7f592009b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:27.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 1 -- 172.21.15.178:0/2777645598 >> 172.21.15.178:0/2777645598 conn(0x7f59200a5ed0 msgr2=0x7f592009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:27.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 1 -- 172.21.15.178:0/2777645598 shutdown_connections 2024-09-15T01:34:27.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:27.749+0000 7f5926d54640 1 -- 172.21.15.178:0/2777645598 wait complete. 2024-09-15T01:34:29.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:29.762+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:29.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:29.762+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:29.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:29.762+0000 7ffa42ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:30.762 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:34:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:30.762+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:30.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:30.762+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:30.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.865+0000 7f7c8935d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:30.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.865+0000 7f7c8935d640 10 monclient: build_initial_monmap 2024-09-15T01:34:30.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.865+0000 7f7c8935d640 10 monclient: monmap: 2024-09-15T01:34:30.867 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:30.867 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:30.867 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:30.866329+0000 2024-09-15T01:34:30.867 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:30.866329+0000 2024-09-15T01:34:30.867 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:30.867 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:30.867 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:30.868 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:30.868 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:30.868 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:30.868 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:30.868 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:30.868 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:30.868 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:30.868 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:30.868 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:30.868 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:30.868 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:30.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 1 Processor -- start 2024-09-15T01:34:30.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 1 -- start start 2024-09-15T01:34:30.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 10 monclient: init 2024-09-15T01:34:30.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:30.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 10 monclient: _add_conns ranks=[4,6,1,0,3,9,7,8,2,5,10] 2024-09-15T01:34:30.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7c840a2930 0x7f7c84132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:30.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 10 monclient(hunting): picked mon.noname-c con 0x7f7c840a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:30.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c8409faa0 0x7f7c8409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 10 monclient(hunting): picked mon.noname-d con 0x7f7c8409faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7c840a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8404bf60 con 0x7f7c840a08c0 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8409e0f0 con 0x7f7c840a2930 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8409dd80 con 0x7f7c8409faa0 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c8935d640 10 monclient(hunting): authenticate will time out at 875.616932s 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c7bfff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c8409faa0 msgr2=0x7f7c8409fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.866+0000 7f7c7bfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c8409faa0 0x7f7c8409fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7c840a2930 0x7f7c84132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:30.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7c840a2930 0x7f7c84132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44336/0 (socket says 172.21.15.178:44336) 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 1 -- 172.21.15.178:0/219359859 learned_addr learned my addr 172.21.15.178:0/219359859 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 10 monclient(hunting): get_auth_request con 0x7f7c840a2930 auth_method 0 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 10 monclient(hunting): handle_auth_done global_id 5206 payload 274 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 1 -- 172.21.15.178:0/219359859 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c8409faa0 msgr2=0x7f7c8409fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 1 --2- 172.21.15.178:0/219359859 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c8409faa0 0x7f7c8409fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 1 -- 172.21.15.178:0/219359859 >> v1:172.21.15.204:6789/0 conn(0x7f7c840a08c0 legacy=0x7f7c840a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:30.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 1 monclient: found mon.noname-c 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.867+0000 7f7c82ffd640 1 -- 172.21.15.178:0/219359859 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c840a2d00 con 0x7f7c840a2930 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c82ffd640 1 --2- 172.21.15.178:0/219359859 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7c840a2930 0x7f7c84132de0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009940 tx=0x7f7c740091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c9099a5886c545c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c827fc640 1 -- 172.21.15.178:0/219359859 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c7402ac90 con 0x7f7c840a2930 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c827fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c827fc640 10 monclient: dump: 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:30.880 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c827fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:30.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c827fc640 1 -- 172.21.15.178:0/219359859 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7402ae10 con 0x7f7c840a2930 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 5 monclient: authenticate success, global_id 5206 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c827fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c827fc640 1 -- 172.21.15.178:0/219359859 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c74031ca0 con 0x7f7c840a2930 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 4 set_mon_vals no callback set 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c827fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c827fc640 10 monclient: dump: 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:30.882 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:30.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 10 monclient: shutdown 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 1 -- 172.21.15.178:0/219359859 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7c840a2930 msgr2=0x7f7c84132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 1 --2- 172.21.15.178:0/219359859 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7c840a2930 0x7f7c84132de0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009940 tx=0x7f7c740091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 1 -- 172.21.15.178:0/219359859 shutdown_connections 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 1 --2- 172.21.15.178:0/219359859 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c8409faa0 0x7f7c8409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 1 --2- 172.21.15.178:0/219359859 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7c840a2930 0x7f7c84132de0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 1 -- 172.21.15.178:0/219359859 >> 172.21.15.178:0/219359859 conn(0x7f7c840a5ed0 msgr2=0x7f7c840a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 1 -- 172.21.15.178:0/219359859 shutdown_connections 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.868+0000 7f7c8935d640 1 -- 172.21.15.178:0/219359859 wait complete. 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.869+0000 7f7c8935d640 10 monclient: build_initial_monmap 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.869+0000 7f7c8935d640 10 monclient: monmap: 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:30.884 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:30.885 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:30.885 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:30.885 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:30.885 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:30.885 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:30.885 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:30.885 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:30.885 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:30.885 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:30.885 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:30.885 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:30.885 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.869+0000 7f7c8935d640 1 Processor -- start 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.869+0000 7f7c8935d640 1 -- start start 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.869+0000 7f7c8935d640 10 monclient: init 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 10 monclient: _add_conns ranks=[3,5,0,1,2,7,6,4,8] 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c8409faa0 0x7f7c8409b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 10 monclient(hunting): picked mon.noname-g con 0x7f7c8409faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7c840a08c0 0x7f7c841318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 10 monclient(hunting): picked mon.noname-h con 0x7f7c840a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c840a2930 0x7f7c8411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 10 monclient(hunting): picked mon.noname-a con 0x7f7c840a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:30.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8411ef90 con 0x7f7c840a2930 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8409d600 con 0x7f7c8409faa0 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8411ecb0 con 0x7f7c840a08c0 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c8935d640 10 monclient(hunting): authenticate will time out at 875.620361s 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c840a2930 0x7f7c8411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c840a2930 0x7f7c8411c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:54740/0 (socket says 172.21.15.178:54740) 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 1 -- 172.21.15.178:0/2535634320 learned_addr learned my addr 172.21.15.178:0/2535634320 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 10 monclient(hunting): get_auth_request con 0x7f7c840a2930 auth_method 0 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:30.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c82ffd640 1 --2- 172.21.15.178:0/2535634320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c8409faa0 0x7f7c8409b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c7bfff640 1 --2- 172.21.15.178:0/2535634320 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7c840a08c0 0x7f7c841318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c82ffd640 10 monclient(hunting): get_auth_request con 0x7f7c8409faa0 auth_method 0 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c82ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c82ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c82ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c7bfff640 10 monclient(hunting): get_auth_request con 0x7f7c840a08c0 auth_method 0 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:30.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 10 monclient(hunting): handle_auth_done global_id 5391 payload 824 2024-09-15T01:34:30.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 1 -- 172.21.15.178:0/2535634320 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7c840a08c0 msgr2=0x7f7c841318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:30.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 1 --2- 172.21.15.178:0/2535634320 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7c840a08c0 0x7f7c841318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:30.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 1 -- 172.21.15.178:0/2535634320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c8409faa0 msgr2=0x7f7c8409b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:30.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 1 --2- 172.21.15.178:0/2535634320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c8409faa0 0x7f7c8409b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:30.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:30.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 1 monclient: found mon.noname-a 2024-09-15T01:34:30.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:30.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:30.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 1 -- 172.21.15.178:0/2535634320 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c74009450 con 0x7f7c840a2930 2024-09-15T01:34:30.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c82ffd640 1 --2- 172.21.15.178:0/2535634320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c8409faa0 0x7f7c8409b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:30.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.870+0000 7f7c837fe640 1 --2- 172.21.15.178:0/2535634320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c840a2930 0x7f7c8411c530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00c890 tx=0x7f7c7c002640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:30.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c7affd640 1 -- 172.21.15.178:0/2535634320 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c7c00ce80 con 0x7f7c840a2930 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c7affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c7affd640 10 monclient: dump: 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c7affd640 10 monclient: _finish_auth 0 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c7affd640 1 -- 172.21.15.178:0/2535634320 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7c00ac70 con 0x7f7c840a2930 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c8935d640 5 monclient: authenticate success, global_id 5391 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c7affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c7affd640 1 -- 172.21.15.178:0/2535634320 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c7c00ae60 con 0x7f7c840a2930 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c8935d640 10 monclient: _renew_subs 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c8935d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:30.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c8935d640 1 -- 172.21.15.178:0/2535634320 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8411e9d0 con 0x7f7c840a2930 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c7affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c81ffb640 4 set_mon_vals no callback set 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c7affd640 10 monclient: dump: 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:30.892 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c8935d640 10 monclient: _renew_subs 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c8935d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.871+0000 7f7c8935d640 1 -- 172.21.15.178:0/2535634320 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c84142250 con 0x7f7c840a2930 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.872+0000 7f7c7affd640 1 -- 172.21.15.178:0/2535634320 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7c7c00a7f0 con 0x7f7c840a2930 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.872+0000 7f7c8935d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.872+0000 7f7c8935d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.872+0000 7f7c8935d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.872+0000 7f7c8935d640 1 -- 172.21.15.178:0/2535634320 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8404b720 con 0x7f7c840a2930 2024-09-15T01:34:30.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.872+0000 7f7c7affd640 1 --2- 172.21.15.178:0/2535634320 >> v2:172.21.15.204:6812/28210 conn(0x7f7c640397b0 0x7f7c6403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:30.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.872+0000 7f7c7affd640 1 -- 172.21.15.178:0/2535634320 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5312+0+0 (secure 0 0 0) 0x7f7c7c04b0b0 con 0x7f7c840a2930 2024-09-15T01:34:30.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.872+0000 7f7c82ffd640 1 --2- 172.21.15.178:0/2535634320 >> v2:172.21.15.204:6812/28210 conn(0x7f7c640397b0 0x7f7c6403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:30.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.872+0000 7f7c82ffd640 10 monclient: get_auth_request con 0x7f7c640397b0 auth_method 0 2024-09-15T01:34:30.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.873+0000 7f7c82ffd640 1 --2- 172.21.15.178:0/2535634320 >> v2:172.21.15.204:6812/28210 conn(0x7f7c640397b0 0x7f7c6403bc60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7c74034000 tx=0x7f7c74032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:30.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.876+0000 7f7c7affd640 1 -- 172.21.15.178:0/2535634320 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7c7c018c50 con 0x7f7c840a2930 2024-09-15T01:34:30.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.876+0000 7f7c7affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:30.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:30.876+0000 7f7c7affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:31.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.088+0000 7f7c8935d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:34:31.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.088+0000 7f7c8935d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:34:31.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.088+0000 7f7c8935d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:31.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.088+0000 7f7c8935d640 1 -- 172.21.15.178:0/2535634320 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7c8409cef0 con 0x7f7c840a2930 2024-09-15T01:34:31.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.089+0000 7f7c7affd640 1 -- 172.21.15.178:0/2535634320 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7f7c7c016c80 con 0x7f7c840a2930 2024-09-15T01:34:31.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.089+0000 7f7c7affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:34:31.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.089+0000 7f7c7affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:31.091 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":38,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":8,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:34:31.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 1 -- 172.21.15.178:0/2535634320 >> v2:172.21.15.204:6812/28210 conn(0x7f7c640397b0 msgr2=0x7f7c6403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:31.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 1 --2- 172.21.15.178:0/2535634320 >> v2:172.21.15.204:6812/28210 conn(0x7f7c640397b0 0x7f7c6403bc60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7c74034000 tx=0x7f7c74032000 comp rx=0 tx=0).stop 2024-09-15T01:34:31.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 10 monclient: shutdown 2024-09-15T01:34:31.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:31.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 1 -- 172.21.15.178:0/2535634320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c840a2930 msgr2=0x7f7c8411c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:31.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 1 --2- 172.21.15.178:0/2535634320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c840a2930 0x7f7c8411c530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00c890 tx=0x7f7c7c002640 comp rx=0 tx=0).stop 2024-09-15T01:34:31.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 1 -- 172.21.15.178:0/2535634320 shutdown_connections 2024-09-15T01:34:31.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 1 --2- 172.21.15.178:0/2535634320 >> v2:172.21.15.204:6812/28210 conn(0x7f7c640397b0 0x7f7c6403bc60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:31.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 1 --2- 172.21.15.178:0/2535634320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c840a2930 0x7f7c8411c530 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:31.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 1 --2- 172.21.15.178:0/2535634320 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7c840a08c0 0x7f7c841318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:31.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 1 --2- 172.21.15.178:0/2535634320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c8409faa0 0x7f7c8409b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:31.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 1 -- 172.21.15.178:0/2535634320 >> 172.21.15.178:0/2535634320 conn(0x7f7c840a5ed0 msgr2=0x7f7c8409e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:31.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 1 -- 172.21.15.178:0/2535634320 shutdown_connections 2024-09-15T01:34:31.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:31.091+0000 7f7c8935d640 1 -- 172.21.15.178:0/2535634320 wait complete. 2024-09-15T01:34:31.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:31.762+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:31.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:31.762+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:32.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:32.762+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:32.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:32.762+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:33.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:33.762+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:33.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:33.762+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:34.104 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:34:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.206+0000 7f093cf1d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.206+0000 7f093cf1d640 10 monclient: build_initial_monmap 2024-09-15T01:34:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.206+0000 7f093cf1d640 10 monclient: monmap: 2024-09-15T01:34:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:34.207216+0000 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:34.207216+0000 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:34.210 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:34.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.206+0000 7f093cf1d640 1 Processor -- start 2024-09-15T01:34:34.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 1 -- start start 2024-09-15T01:34:34.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 10 monclient: init 2024-09-15T01:34:34.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:34.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 10 monclient: _add_conns ranks=[7,4,9,0,3,2,5,10,1,8,6] 2024-09-15T01:34:34.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f09380a2930 0x7f0938132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:34.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 10 monclient(hunting): picked mon.noname-i con 0x7f09380a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:34.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093809faa0 0x7f093809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:34.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 10 monclient(hunting): picked mon.noname-c con 0x7f093809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:34.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f09380a08c0 0x7f09380a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 10 monclient(hunting): picked mon.noname-a con 0x7f09380a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093809e0f0 con 0x7f09380a08c0 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f093809dd80 con 0x7f093809faa0 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f09380a2d00 con 0x7f09380a2930 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093cf1d640 10 monclient(hunting): authenticate will time out at 878.957719s 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093659c640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f09380a2930 msgr2=0x7f0938132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f0935d9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093809faa0 0x7f093809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f093659c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f09380a2930 0x7f0938132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f0936d9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f09380a08c0 0x7f09380a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f0935d9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093809faa0 0x7f093809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44338/0 (socket says 172.21.15.178:44338) 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f0935d9b640 1 -- 172.21.15.178:0/1415222895 learned_addr learned my addr 172.21.15.178:0/1415222895 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f0935d9b640 10 monclient(hunting): get_auth_request con 0x7f093809faa0 auth_method 0 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f0935d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f0935d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f0935d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f0936d9d640 10 monclient(hunting): get_auth_request con 0x7f09380a08c0 auth_method 0 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f0936d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f0936d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.207+0000 7f0936d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0936d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0936d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0936d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 10 monclient(hunting): handle_auth_done global_id 5215 payload 274 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 1 -- 172.21.15.178:0/1415222895 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f09380a2930 msgr2=0x7f0938132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 1 --2- 172.21.15.178:0/1415222895 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f09380a2930 0x7f0938132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 1 -- 172.21.15.178:0/1415222895 >> v2:172.21.15.178:3300/0 conn(0x7f09380a08c0 msgr2=0x7f09380a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 1 --2- 172.21.15.178:0/1415222895 >> v2:172.21.15.178:3300/0 conn(0x7f09380a08c0 0x7f09380a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 1 monclient: found mon.noname-c 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 1 -- 172.21.15.178:0/1415222895 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093804bf60 con 0x7f093809faa0 2024-09-15T01:34:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0936d9d640 1 --2- 172.21.15.178:0/1415222895 >> v2:172.21.15.178:3300/0 conn(0x7f09380a08c0 0x7f09380a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.208+0000 7f0935d9b640 1 --2- 172.21.15.178:0/1415222895 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093809faa0 0x7f093809fe70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0928009910 tx=0x7f09280091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2abca00d606ddb8a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093559a640 1 -- 172.21.15.178:0/1415222895 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f092802acb0 con 0x7f093809faa0 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093559a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093559a640 10 monclient: dump: 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093559a640 10 monclient: _finish_auth 0 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093559a640 1 -- 172.21.15.178:0/1415222895 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092802ae30 con 0x7f093809faa0 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 5 monclient: authenticate success, global_id 5215 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093559a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 4 set_mon_vals no callback set 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093559a640 1 -- 172.21.15.178:0/1415222895 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0928031ca0 con 0x7f093809faa0 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093559a640 10 monclient: dump: 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:34.224 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 10 monclient: shutdown 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 1 -- 172.21.15.178:0/1415222895 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093809faa0 msgr2=0x7f093809fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 1 --2- 172.21.15.178:0/1415222895 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093809faa0 0x7f093809fe70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0928009910 tx=0x7f09280091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 1 -- 172.21.15.178:0/1415222895 shutdown_connections 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 1 --2- 172.21.15.178:0/1415222895 >> v2:172.21.15.178:3300/0 conn(0x7f09380a08c0 0x7f09380a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 1 --2- 172.21.15.178:0/1415222895 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093809faa0 0x7f093809fe70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 1 --2- 172.21.15.178:0/1415222895 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f09380a2930 0x7f0938132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 1 -- 172.21.15.178:0/1415222895 >> 172.21.15.178:0/1415222895 conn(0x7f09380a5ed0 msgr2=0x7f09380a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 1 -- 172.21.15.178:0/1415222895 shutdown_connections 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.209+0000 7f093cf1d640 1 -- 172.21.15.178:0/1415222895 wait complete. 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.210+0000 7f093cf1d640 10 monclient: build_initial_monmap 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.210+0000 7f093cf1d640 10 monclient: monmap: 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.210+0000 7f093cf1d640 1 Processor -- start 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.210+0000 7f093cf1d640 1 -- start start 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.210+0000 7f093cf1d640 10 monclient: init 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 10 monclient: _add_conns ranks=[0,2,7,6,8,5,4,3,1] 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f093809faa0 0x7f093809b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 10 monclient(hunting): picked mon.noname-a con 0x7f093809faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09380a08c0 0x7f0938131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 10 monclient(hunting): picked mon.noname-b con 0x7f09380a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f09380a2930 0x7f093811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 10 monclient(hunting): picked mon.noname-i con 0x7f09380a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093809d630 con 0x7f093809faa0 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093811ec40 con 0x7f09380a08c0 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0938142a30 con 0x7f09380a2930 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f0935d9b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09380a08c0 msgr2=0x7f0938131840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093cf1d640 10 monclient(hunting): authenticate will time out at 878.961329s 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f0935d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09380a08c0 0x7f0938131840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f093809faa0 0x7f093809b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f093809faa0 0x7f093809b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:54750/0 (socket says 172.21.15.178:54750) 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 1 -- 172.21.15.178:0/1570655572 learned_addr learned my addr 172.21.15.178:0/1570655572 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 10 monclient(hunting): get_auth_request con 0x7f093809faa0 auth_method 0 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f0936d9d640 1 -- 172.21.15.178:0/1570655572 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f09380a2930 msgr2=0x7f093811c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f0936d9d640 1 --2- 172.21.15.178:0/1570655572 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f09380a2930 0x7f093811c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 10 monclient(hunting): handle_auth_done global_id 5409 payload 824 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 1 -- 172.21.15.178:0/1570655572 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f09380a2930 msgr2=0x7f093811c530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 1 --2- 172.21.15.178:0/1570655572 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f09380a2930 0x7f093811c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 1 -- 172.21.15.178:0/1570655572 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09380a08c0 msgr2=0x7f0938131840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 1 --2- 172.21.15.178:0/1570655572 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09380a08c0 0x7f0938131840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 1 monclient: found mon.noname-a 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 1 -- 172.21.15.178:0/1570655572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0930009a20 con 0x7f093809faa0 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f093659c640 1 --2- 172.21.15.178:0/1570655572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f093809faa0 0x7f093809b010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0938142c00 tx=0x7f092c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f09267fc640 1 -- 172.21.15.178:0/1570655572 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f092c006150 con 0x7f093809faa0 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f09267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f09267fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f09267fc640 10 monclient: dump: 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.211+0000 7f09267fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f09267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f093cf1d640 5 monclient: authenticate success, global_id 5409 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f09267fc640 1 -- 172.21.15.178:0/1570655572 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c0062f0 con 0x7f093809faa0 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f093cf1d640 10 monclient: _renew_subs 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f093cf1d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f093cf1d640 1 -- 172.21.15.178:0/1570655572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0928009450 con 0x7f093809faa0 2024-09-15T01:34:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f09267fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f09267fc640 1 -- 172.21.15.178:0/1570655572 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f092c0103e0 con 0x7f093809faa0 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f09267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f09267fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f0934d99640 4 set_mon_vals no callback set 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f09267fc640 10 monclient: dump: 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f093cf1d640 10 monclient: _renew_subs 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f093cf1d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f093cf1d640 1 -- 172.21.15.178:0/1570655572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093809bc00 con 0x7f093809faa0 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.212+0000 7f09267fc640 1 -- 172.21.15.178:0/1570655572 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f092c003b80 con 0x7f093809faa0 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.213+0000 7f09267fc640 1 --2- 172.21.15.178:0/1570655572 >> v2:172.21.15.204:6812/28210 conn(0x7f09180396e0 0x7f091803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.213+0000 7f093cf1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:34.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.213+0000 7f093cf1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:34.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.213+0000 7f093cf1d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:34.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.213+0000 7f09267fc640 1 -- 172.21.15.178:0/1570655572 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5312+0+0 (secure 0 0 0) 0x7f092c04a900 con 0x7f093809faa0 2024-09-15T01:34:34.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.213+0000 7f093cf1d640 1 -- 172.21.15.178:0/1570655572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093804b720 con 0x7f093809faa0 2024-09-15T01:34:34.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.213+0000 7f0935d9b640 1 --2- 172.21.15.178:0/1570655572 >> v2:172.21.15.204:6812/28210 conn(0x7f09180396e0 0x7f091803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:34.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.213+0000 7f0935d9b640 10 monclient: get_auth_request con 0x7f09180396e0 auth_method 0 2024-09-15T01:34:34.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.213+0000 7f0935d9b640 1 --2- 172.21.15.178:0/1570655572 >> v2:172.21.15.204:6812/28210 conn(0x7f09180396e0 0x7f091803bb90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0928034000 tx=0x7f0928032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:34.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.217+0000 7f09267fc640 1 -- 172.21.15.178:0/1570655572 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f092c017020 con 0x7f093809faa0 2024-09-15T01:34:34.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.217+0000 7f09267fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:34.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.217+0000 7f09267fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:34.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.426+0000 7f093cf1d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:34:34.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.426+0000 7f093cf1d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:34:34.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.426+0000 7f093cf1d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:34.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.426+0000 7f093cf1d640 1 -- 172.21.15.178:0/1570655572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f093809cef0 con 0x7f093809faa0 2024-09-15T01:34:34.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.427+0000 7f09267fc640 1 -- 172.21.15.178:0/1570655572 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7f092c00d6d0 con 0x7f093809faa0 2024-09-15T01:34:34.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.427+0000 7f09267fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:34:34.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.427+0000 7f09267fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:34.430 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":38,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":11,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:34:34.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 1 -- 172.21.15.178:0/1570655572 >> v2:172.21.15.204:6812/28210 conn(0x7f09180396e0 msgr2=0x7f091803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:34.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 1 --2- 172.21.15.178:0/1570655572 >> v2:172.21.15.204:6812/28210 conn(0x7f09180396e0 0x7f091803bb90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0928034000 tx=0x7f0928032000 comp rx=0 tx=0).stop 2024-09-15T01:34:34.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 10 monclient: shutdown 2024-09-15T01:34:34.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:34.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 1 -- 172.21.15.178:0/1570655572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f093809faa0 msgr2=0x7f093809b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:34.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 1 --2- 172.21.15.178:0/1570655572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f093809faa0 0x7f093809b010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0938142c00 tx=0x7f092c0091a0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 1 -- 172.21.15.178:0/1570655572 shutdown_connections 2024-09-15T01:34:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 1 --2- 172.21.15.178:0/1570655572 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09380a08c0 0x7f0938131840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 1 --2- 172.21.15.178:0/1570655572 >> v2:172.21.15.204:6812/28210 conn(0x7f09180396e0 0x7f091803bb90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 1 --2- 172.21.15.178:0/1570655572 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f09380a2930 0x7f093811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 1 --2- 172.21.15.178:0/1570655572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f093809faa0 0x7f093809b010 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 1 -- 172.21.15.178:0/1570655572 >> 172.21.15.178:0/1570655572 conn(0x7f09380a5ed0 msgr2=0x7f093809e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 1 -- 172.21.15.178:0/1570655572 shutdown_connections 2024-09-15T01:34:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:34.429+0000 7f093cf1d640 1 -- 172.21.15.178:0/1570655572 wait complete. 2024-09-15T01:34:34.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:34.762+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:34.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:34.762+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:35.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:35.763+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:35.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:35.763+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:36.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:36.763+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:36.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:36.763+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:37.442 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:34:37.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.543+0000 7f1f23b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:37.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.543+0000 7f1f23b9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:37.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.543+0000 7f1f23b9d640 10 monclient: monmap: 2024-09-15T01:34:37.546 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:37.546 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:37.545454+0000 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:37.545454+0000 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:37.547 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:37.548 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:37.548 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:37.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 1 Processor -- start 2024-09-15T01:34:37.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 1 -- start start 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 10 monclient: init 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 10 monclient: _add_conns ranks=[6,9,1,2,8,5,4,3,0,7,10] 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f1c0a2930 0x7f1f1c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 10 monclient(hunting): picked mon.noname-d con 0x7f1f1c0a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 10 monclient(hunting): picked mon.noname-a con 0x7f1f1c09faa0 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1f1c0a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f1c04bf60 con 0x7f1f1c0a08c0 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1c09e0f0 con 0x7f1f1c09faa0 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c09dd80 con 0x7f1f1c0a2930 2024-09-15T01:34:37.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f23b9d640 10 monclient(hunting): authenticate will time out at 882.295894s 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f21939640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f1c0a2930 msgr2=0x7f1f1c132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.544+0000 7f1f21939640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f1c0a2930 0x7f1f1c132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.545+0000 7f1f21138640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.545+0000 7f1f21138640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:54764/0 (socket says 172.21.15.178:54764) 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.545+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 learned_addr learned my addr 172.21.15.178:0/828014537 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.545+0000 7f1f21138640 10 monclient(hunting): get_auth_request con 0x7f1f1c09faa0 auth_method 0 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.545+0000 7f1f21138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.545+0000 7f1f21138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.545+0000 7f1f21138640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.545+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 msgr2=0x7f1f1c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.545+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 msgr2=0x7f1f1c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.545+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:37.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.545+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:37.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.746+0000 7f1f21939640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f1c0a2930 msgr2=0x7f1f1c132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:37.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.746+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f1c0a2930 0x7f1f1c132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:34:37.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.746+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:37.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.746+0000 7f1f21138640 10 monclient(hunting): get_auth_request con 0x7f1f1c09faa0 auth_method 0 2024-09-15T01:34:37.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.746+0000 7f1f21138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:37.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.746+0000 7f1f21138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:37.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.746+0000 7f1f21138640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:37.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.746+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 msgr2=0x7f1f1c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:34:37.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.746+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 msgr2=0x7f1f1c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:37.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.746+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:37.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:37.746+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:34:37.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:37.763+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:37.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:37.763+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:38.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.147+0000 7f1f21939640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f1c0a2930 msgr2=0x7f1f1c132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:38.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.147+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f1c0a2930 0x7f1f1c132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:34:38.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.147+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:38.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.147+0000 7f1f21138640 10 monclient(hunting): get_auth_request con 0x7f1f1c09faa0 auth_method 0 2024-09-15T01:34:38.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.147+0000 7f1f21138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:38.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.147+0000 7f1f21138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:38.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.147+0000 7f1f21138640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:38.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.147+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 msgr2=0x7f1f1c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:34:38.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.147+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 msgr2=0x7f1f1c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:38.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.147+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:38.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.147+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:34:38.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:38.763+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:38.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:38.763+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:38.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.948+0000 7f1f21939640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f1c0a2930 msgr2=0x7f1f1c132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:38.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.948+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f1c0a2930 0x7f1f1c132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:34:38.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.948+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:38.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.948+0000 7f1f21138640 10 monclient(hunting): get_auth_request con 0x7f1f1c09faa0 auth_method 0 2024-09-15T01:34:38.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.948+0000 7f1f21138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:38.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.948+0000 7f1f21138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:38.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.948+0000 7f1f21138640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:38.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.948+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 msgr2=0x7f1f1c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:34:38.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.948+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 msgr2=0x7f1f1c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:38.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.948+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:38.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:38.948+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:34:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:39.763+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:39.763+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:39.763+0000 7ffa42ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 10 monclient(hunting): tick 2024-09-15T01:34:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 1 monclient(hunting): continuing hunt 2024-09-15T01:34:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:34:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f1c0a2930 msgr2=0x7f1f1c132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f1c0a2930 0x7f1f1c132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 1 -- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 msgr2=0x7f1f1c09fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 1 --2- 172.21.15.178:0/828014537 >> v2:172.21.15.178:3300/0 conn(0x7f1f1c09faa0 0x7f1f1c09fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 1 -- 172.21.15.178:0/828014537 >> v1:172.21.15.204:6789/0 conn(0x7f1f1c0a08c0 legacy=0x7f1f1c0a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 10 monclient: _add_conns ranks=[8,4,7,3,10,5,2,0] 2024-09-15T01:34:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 10 monclient(hunting): picked mon.noname-e con 0x7f1f000037d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 10 monclient(hunting): picked mon.noname-c con 0x7f1f000062b0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1f00008e20 0x7f1f0000b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 10 monclient(hunting): picked mon.noname-i con 0x7f1f00008e20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 1 -- 172.21.15.178:0/828014537 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c09e0f0 con 0x7f1f000062b0 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 1 -- 172.21.15.178:0/828014537 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c09dd80 con 0x7f1f00008e20 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 1 -- 172.21.15.178:0/828014537 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0000ba40 con 0x7f1f000037d0 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f137fe640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f2213a640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1f00008e20 msgr2=0x7f1f0000b280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f2213a640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1f00008e20 0x7f1f0000b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21138640 10 monclient(hunting): get_auth_request con 0x7f1f000062b0 auth_method 0 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21138640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21939640 10 monclient(hunting): get_auth_request con 0x7f1f000037d0 auth_method 0 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 msgr2=0x7f1f00008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 msgr2=0x7f1f00008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21939640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 msgr2=0x7f1f00005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:34:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21939640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 msgr2=0x7f1f00005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.545+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:40.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f2213a640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1f00008e20 msgr2=0x7f1f0000b280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:40.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f2213a640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1f00008e20 0x7f1f0000b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:34:40.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:40.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21138640 10 monclient(hunting): get_auth_request con 0x7f1f000062b0 auth_method 0 2024-09-15T01:34:40.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:40.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:40.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21138640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:40.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:40.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21939640 10 monclient(hunting): get_auth_request con 0x7f1f000037d0 auth_method 0 2024-09-15T01:34:40.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:40.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:40.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21939640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:40.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 msgr2=0x7f1f00008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:34:40.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 msgr2=0x7f1f00008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:40.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:40.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21939640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 msgr2=0x7f1f00005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:34:40.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21939640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 msgr2=0x7f1f00005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:40.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:40.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:34:40.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:40.746+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:34:40.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:40.763+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:40.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:40.763+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:41.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f2213a640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1f00008e20 msgr2=0x7f1f0000b280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:41.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f2213a640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1f00008e20 0x7f1f0000b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:34:41.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:41.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:41.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21138640 10 monclient(hunting): get_auth_request con 0x7f1f000062b0 auth_method 0 2024-09-15T01:34:41.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:41.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:41.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21138640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:41.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21939640 10 monclient(hunting): get_auth_request con 0x7f1f000037d0 auth_method 0 2024-09-15T01:34:41.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:41.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:41.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21939640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:41.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 msgr2=0x7f1f00008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:34:41.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21138640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 msgr2=0x7f1f00008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:41.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:41.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21939640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 msgr2=0x7f1f00005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:34:41.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21939640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 msgr2=0x7f1f00005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:34:41.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:34:41.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:34:41.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.147+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:34:41.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:41.763+0000 7ffa42ffd640 10 monclient: tick 2024-09-15T01:34:41.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:41.763+0000 7ffa42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f2213a640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1f00008e20 msgr2=0x7f1f0000b280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21138640 10 monclient(hunting): get_auth_request con 0x7f1f000062b0 auth_method 0 2024-09-15T01:34:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21138640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f2213a640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1f00008e20 0x7f1f0000b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:34:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 10 monclient(hunting): get_auth_request con 0x7f1f000037d0 auth_method 0 2024-09-15T01:34:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21138640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21138640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21138640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 10 monclient(hunting): handle_auth_done global_id 5300 payload 274 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1f00008e20 msgr2=0x7f1f0000b280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1f00008e20 0x7f1f0000b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 msgr2=0x7f1f00008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 10 monclient: _finish_hunting 0 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 1 monclient: found mon.noname-e 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f2213a640 1 -- 172.21.15.178:0/828014537 reap_dead start 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21939640 1 -- 172.21.15.178:0/828014537 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c0a2d00 con 0x7f1f000037d0 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.948+0000 7f1f21138640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f000062b0 0x7f1f00008730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f21939640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009e70 tx=0x7f1f1800b040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=4af060deab0eedb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f20937640 1 -- 172.21.15.178:0/828014537 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f18033430 con 0x7f1f000037d0 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f20937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f20937640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f20937640 10 monclient: dump: 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f20937640 10 monclient: _finish_auth 0 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f20937640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f20937640 1 -- 172.21.15.178:0/828014537 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f180335b0 con 0x7f1f000037d0 2024-09-15T01:34:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f23b9d640 5 monclient: authenticate success, global_id 5300 2024-09-15T01:34:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f20937640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f23b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f23b9d640 4 set_mon_vals no callback set 2024-09-15T01:34:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f20937640 1 -- 172.21.15.178:0/828014537 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f18033ba0 con 0x7f1f000037d0 2024-09-15T01:34:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f20937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f20937640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.949+0000 7f1f20937640 10 monclient: dump: 2024-09-15T01:34:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:41.960 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.950+0000 7f1f23b9d640 10 monclient: shutdown 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.950+0000 7f1f23b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.950+0000 7f1f23b9d640 1 -- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 msgr2=0x7f1f00005c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.950+0000 7f1f23b9d640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009e70 tx=0x7f1f1800b040 comp rx=0 tx=0).stop 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.950+0000 7f1f23b9d640 1 -- 172.21.15.178:0/828014537 shutdown_connections 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.950+0000 7f1f23b9d640 1 --2- 172.21.15.178:0/828014537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1f000037d0 0x7f1f00005c10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.950+0000 7f1f23b9d640 1 -- 172.21.15.178:0/828014537 >> 172.21.15.178:0/828014537 conn(0x7f1f1c0a5ed0 msgr2=0x7f1f1c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.950+0000 7f1f23b9d640 1 -- 172.21.15.178:0/828014537 shutdown_connections 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.950+0000 7f1f23b9d640 1 -- 172.21.15.178:0/828014537 wait complete. 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.951+0000 7f1f23b9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.951+0000 7f1f23b9d640 10 monclient: monmap: 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:41.962 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.951+0000 7f1f23b9d640 1 Processor -- start 2024-09-15T01:34:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.951+0000 7f1f23b9d640 1 -- start start 2024-09-15T01:34:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.951+0000 7f1f23b9d640 10 monclient: init 2024-09-15T01:34:41.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:41.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 10 monclient: _add_conns ranks=[3,0,1,7,2,8,6,4,5] 2024-09-15T01:34:41.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f000037d0 0x7f1f1c09b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:41.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 10 monclient(hunting): picked mon.noname-g con 0x7f1f000037d0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f1c11ca00 0x7f1f1c131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 10 monclient(hunting): picked mon.noname-a con 0x7f1f1c11ca00 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c131e60 0x7f1f1c141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 10 monclient(hunting): picked mon.noname-f con 0x7f1f1c131e60 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c09be20 con 0x7f1f1c11ca00 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c09d630 con 0x7f1f1c131e60 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c11ea40 con 0x7f1f000037d0 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f23b9d640 10 monclient(hunting): authenticate will time out at 886.703484s 2024-09-15T01:34:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21138640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f1c11ca00 0x7f1f1c131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:41.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21138640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f1c11ca00 0x7f1f1c131920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44730/0 (socket says 172.21.15.178:44730) 2024-09-15T01:34:41.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21138640 1 -- 172.21.15.178:0/1075557786 learned_addr learned my addr 172.21.15.178:0/1075557786 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:41.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21138640 10 monclient(hunting): get_auth_request con 0x7f1f1c11ca00 auth_method 0 2024-09-15T01:34:41.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:41.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:41.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21138640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:41.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21138640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:41.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21138640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:41.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21138640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:41.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21939640 1 --2- 172.21.15.178:0/1075557786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f000037d0 0x7f1f1c09b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:41.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f2213a640 1 --2- 172.21.15.178:0/1075557786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c131e60 0x7f1f1c141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:41.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21939640 10 monclient(hunting): get_auth_request con 0x7f1f000037d0 auth_method 0 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f2213a640 10 monclient(hunting): get_auth_request con 0x7f1f1c131e60 auth_method 0 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f2213a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f2213a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f2213a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f21939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f2213a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.952+0000 7f1f2213a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:41.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f2213a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:41.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f21138640 10 monclient(hunting): handle_auth_done global_id 5418 payload 824 2024-09-15T01:34:41.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f21138640 1 -- 172.21.15.178:0/1075557786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f000037d0 msgr2=0x7f1f1c09b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:41.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f21138640 1 --2- 172.21.15.178:0/1075557786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f000037d0 0x7f1f1c09b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:41.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f21138640 1 -- 172.21.15.178:0/1075557786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c131e60 msgr2=0x7f1f1c141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:41.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f21138640 1 --2- 172.21.15.178:0/1075557786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c131e60 0x7f1f1c141c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:41.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f21138640 10 monclient: _finish_hunting 0 2024-09-15T01:34:41.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f21138640 1 monclient: found mon.noname-a 2024-09-15T01:34:41.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f21138640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:41.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f21138640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:41.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f21138640 1 -- 172.21.15.178:0/1075557786 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f18009a50 con 0x7f1f1c11ca00 2024-09-15T01:34:41.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f21138640 1 --2- 172.21.15.178:0/1075557786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f1c11ca00 0x7f1f1c131920 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c016a60 tx=0x7f1f0c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:41.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f11ffb640 1 -- 172.21.15.178:0/1075557786 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f0c01d050 con 0x7f1f1c11ca00 2024-09-15T01:34:41.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f11ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:41.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f11ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:41.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f11ffb640 10 monclient: dump: 2024-09-15T01:34:41.973 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:41.973 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:41.973 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:41.973 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:41.973 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:41.973 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:41.973 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:41.973 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:41.973 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f11ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f11ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f11ffb640 1 -- 172.21.15.178:0/1075557786 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c01e040 con 0x7f1f1c11ca00 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f23b9d640 5 monclient: authenticate success, global_id 5418 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f11ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:41.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f23b9d640 10 monclient: _renew_subs 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f11ffb640 1 -- 172.21.15.178:0/1075557786 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f0c006c70 con 0x7f1f1c11ca00 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f23b9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f23b9d640 1 -- 172.21.15.178:0/1075557786 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c11ed20 con 0x7f1f1c11ca00 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f11ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f11ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f11ffb640 10 monclient: dump: 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:41.975 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:41.976 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:41.976 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:41.976 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:41.976 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:41.976 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:41.976 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:41.976 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:41.976 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:41.976 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:41.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f13fff640 4 set_mon_vals no callback set 2024-09-15T01:34:41.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f23b9d640 10 monclient: _renew_subs 2024-09-15T01:34:41.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f23b9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:41.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.953+0000 7f1f23b9d640 1 -- 172.21.15.178:0/1075557786 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c11dae0 con 0x7f1f1c11ca00 2024-09-15T01:34:41.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.954+0000 7f1f11ffb640 1 -- 172.21.15.178:0/1075557786 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1f0c010800 con 0x7f1f1c11ca00 2024-09-15T01:34:41.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.955+0000 7f1f23b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:41.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.955+0000 7f1f23b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:41.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.955+0000 7f1f23b9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:41.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.955+0000 7f1f23b9d640 1 -- 172.21.15.178:0/1075557786 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1c04b720 con 0x7f1f1c11ca00 2024-09-15T01:34:41.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.955+0000 7f1f11ffb640 1 --2- 172.21.15.178:0/1075557786 >> v2:172.21.15.204:6812/28210 conn(0x7f1f0003abe0 0x7f1f0003d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:41.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.955+0000 7f1f11ffb640 1 -- 172.21.15.178:0/1075557786 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5312+0+0 (secure 0 0 0) 0x7f1f0c04c9d0 con 0x7f1f1c11ca00 2024-09-15T01:34:41.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.955+0000 7f1f21939640 1 --2- 172.21.15.178:0/1075557786 >> v2:172.21.15.204:6812/28210 conn(0x7f1f0003abe0 0x7f1f0003d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:41.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.955+0000 7f1f21939640 10 monclient: get_auth_request con 0x7f1f0003abe0 auth_method 0 2024-09-15T01:34:41.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.955+0000 7f1f21939640 1 --2- 172.21.15.178:0/1075557786 >> v2:172.21.15.204:6812/28210 conn(0x7f1f0003abe0 0x7f1f0003d090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f18034000 tx=0x7f1f18035000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:41.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.960+0000 7f1f11ffb640 1 -- 172.21.15.178:0/1075557786 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1f0c04ee60 con 0x7f1f1c11ca00 2024-09-15T01:34:41.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.960+0000 7f1f11ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:41.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:41.960+0000 7f1f11ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:42.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.175+0000 7f1f23b9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:34:42.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.175+0000 7f1f23b9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:34:42.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.175+0000 7f1f23b9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:42.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.175+0000 7f1f23b9d640 1 -- 172.21.15.178:0/1075557786 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1f1c09cef0 con 0x7f1f1c11ca00 2024-09-15T01:34:42.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.175+0000 7f1f11ffb640 1 -- 172.21.15.178:0/1075557786 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3165 (secure 0 0 0) 0x7f1f0c010cf0 con 0x7f1f1c11ca00 2024-09-15T01:34:42.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.175+0000 7f1f11ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:34:42.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.175+0000 7f1f11ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:42.179 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":44,"quorum":[0,1,3,4,5,8],"quorum_names":["a","f","g","c","h","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:34:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 1 -- 172.21.15.178:0/1075557786 >> v2:172.21.15.204:6812/28210 conn(0x7f1f0003abe0 msgr2=0x7f1f0003d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 1 --2- 172.21.15.178:0/1075557786 >> v2:172.21.15.204:6812/28210 conn(0x7f1f0003abe0 0x7f1f0003d090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f18034000 tx=0x7f1f18035000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 10 monclient: shutdown 2024-09-15T01:34:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 1 -- 172.21.15.178:0/1075557786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f1c11ca00 msgr2=0x7f1f1c131920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 1 --2- 172.21.15.178:0/1075557786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f1c11ca00 0x7f1f1c131920 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c016a60 tx=0x7f1f0c008040 comp rx=0 tx=0).stop 2024-09-15T01:34:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 1 -- 172.21.15.178:0/1075557786 shutdown_connections 2024-09-15T01:34:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 1 --2- 172.21.15.178:0/1075557786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c131e60 0x7f1f1c141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 1 --2- 172.21.15.178:0/1075557786 >> v2:172.21.15.204:6812/28210 conn(0x7f1f0003abe0 0x7f1f0003d090 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 1 --2- 172.21.15.178:0/1075557786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f1c11ca00 0x7f1f1c131920 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 1 --2- 172.21.15.178:0/1075557786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f000037d0 0x7f1f1c09b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 1 -- 172.21.15.178:0/1075557786 >> 172.21.15.178:0/1075557786 conn(0x7f1f1c0a5ed0 msgr2=0x7f1f1c09e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 1 -- 172.21.15.178:0/1075557786 shutdown_connections 2024-09-15T01:34:42.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.177+0000 7f1f23b9d640 1 -- 172.21.15.178:0/1075557786 wait complete. 2024-09-15T01:34:42.191 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-09-15T01:34:42.191 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon a 2024-09-15T01:34:42.191 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:34:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.282+0000 7f365815d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.282+0000 7f365815d640 10 monclient: build_initial_monmap 2024-09-15T01:34:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.282+0000 7f365815d640 10 monclient: monmap: 2024-09-15T01:34:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:42.283961+0000 2024-09-15T01:34:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:42.283961+0000 2024-09-15T01:34:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:42.286 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.282+0000 7f365815d640 1 Processor -- start 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.282+0000 7f365815d640 1 -- start start 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.282+0000 7f365815d640 10 monclient: init 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 10 monclient: _add_conns ranks=[3,1,9,5,8,0,10,6,2,7,4] 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3650067130 0x7f3650067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 10 monclient(hunting): picked mon.noname-g con 0x7f3650067130 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f365009d580 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f365009f000 0x7f3650067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 10 monclient(hunting): picked mon.noname-a con 0x7f365009f000 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365004bf60 con 0x7f365009d580 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365009e0c0 con 0x7f365009f000 2024-09-15T01:34:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3650067f80 con 0x7f3650067130 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f365815d640 10 monclient(hunting): authenticate will time out at 887.034518s 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f365009f000 0x7f3650067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f365009f000 0x7f3650067a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44740/0 (socket says 172.21.15.178:44740) 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 1 -- 172.21.15.178:0/1059184332 learned_addr learned my addr 172.21.15.178:0/1059184332 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 10 monclient(hunting): get_auth_request con 0x7f365009f000 auth_method 0 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f3655ef9640 1 --2- 172.21.15.178:0/1059184332 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3650067130 0x7f3650067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f3655ef9640 10 monclient(hunting): get_auth_request con 0x7f3650067130 auth_method 0 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f3655ef9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f3655ef9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f3655ef9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f3655ef9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f3655ef9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f3655ef9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 10 monclient(hunting): handle_auth_done global_id 5427 payload 274 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 1 -- 172.21.15.178:0/1059184332 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3650067130 msgr2=0x7f3650067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 1 --2- 172.21.15.178:0/1059184332 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3650067130 0x7f3650067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 1 -- 172.21.15.178:0/1059184332 >> v1:172.21.15.204:6789/0 conn(0x7f365009d580 legacy=0x7f365012d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 10 monclient: _finish_hunting 0 2024-09-15T01:34:42.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 1 monclient: found mon.noname-a 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.283+0000 7f36566fa640 1 -- 172.21.15.178:0/1059184332 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365009dde0 con 0x7f365009f000 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f36566fa640 1 --2- 172.21.15.178:0/1059184332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365009f000 0x7f3650067a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3648009f20 tx=0x7f3648009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa837b8db247aa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f3654ef7640 1 -- 172.21.15.178:0/1059184332 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3648007880 con 0x7f365009f000 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f3654ef7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f3654ef7640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f3654ef7640 10 monclient: dump: 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.299 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f3654ef7640 10 monclient: _finish_auth 0 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f3654ef7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:42.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f365815d640 5 monclient: authenticate success, global_id 5427 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f365815d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f3654ef7640 1 -- 172.21.15.178:0/1059184332 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364802d050 con 0x7f365009f000 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f3654ef7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f365815d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f365815d640 4 set_mon_vals no callback set 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f3654ef7640 1 -- 172.21.15.178:0/1059184332 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3648030360 con 0x7f365009f000 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f3654ef7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f3654ef7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.284+0000 7f3654ef7640 10 monclient: dump: 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.301 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.285+0000 7f365815d640 10 monclient: shutdown 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.285+0000 7f365815d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.285+0000 7f365815d640 1 -- 172.21.15.178:0/1059184332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365009f000 msgr2=0x7f3650067a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.285+0000 7f365815d640 1 --2- 172.21.15.178:0/1059184332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365009f000 0x7f3650067a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3648009f20 tx=0x7f3648009840 comp rx=0 tx=0).stop 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.285+0000 7f365815d640 1 -- 172.21.15.178:0/1059184332 shutdown_connections 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.285+0000 7f365815d640 1 --2- 172.21.15.178:0/1059184332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365009f000 0x7f3650067a40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.285+0000 7f365815d640 1 --2- 172.21.15.178:0/1059184332 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3650067130 0x7f3650067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.285+0000 7f365815d640 1 -- 172.21.15.178:0/1059184332 >> 172.21.15.178:0/1059184332 conn(0x7f3650068a60 msgr2=0x7f36500a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.285+0000 7f365815d640 1 -- 172.21.15.178:0/1059184332 shutdown_connections 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.285+0000 7f365815d640 1 -- 172.21.15.178:0/1059184332 wait complete. 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.285+0000 7f365815d640 10 monclient: build_initial_monmap 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.285+0000 7f365815d640 10 monclient: monmap: 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:42.303 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:42.304 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:42.304 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:42.304 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:42.304 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:42.304 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:42.304 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:42.304 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:42.304 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:42.304 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:42.304 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:42.304 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 1 Processor -- start 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 1 -- start start 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 10 monclient: init 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 10 monclient: _add_conns ranks=[7,4,0,3,1,5,6,8,2] 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3650067130 0x7f365013fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 10 monclient(hunting): picked mon.noname-i con 0x7f3650067130 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f365009d580 0x7f3650133ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 10 monclient(hunting): picked mon.noname-c con 0x7f365009d580 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365009f000 0x7f3650134000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 10 monclient(hunting): picked mon.noname-a con 0x7f365009f000 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365009cb50 con 0x7f365009f000 2024-09-15T01:34:42.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f365009d9e0 con 0x7f365009d580 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f365011f120 con 0x7f3650067130 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f365815d640 10 monclient(hunting): authenticate will time out at 887.037660s 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f3655ef9640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3650067130 msgr2=0x7f365013fce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f3655ef9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3650067130 0x7f365013fce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36566fa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365009f000 0x7f3650134000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36556f8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f365009d580 0x7f3650133ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36566fa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365009f000 0x7f3650134000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44754/0 (socket says 172.21.15.178:44754) 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36566fa640 1 -- 172.21.15.178:0/2385234166 learned_addr learned my addr 172.21.15.178:0/2385234166 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36566fa640 10 monclient(hunting): get_auth_request con 0x7f365009f000 auth_method 0 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36566fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36566fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36566fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36556f8640 10 monclient(hunting): get_auth_request con 0x7f365009d580 auth_method 0 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36556f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36556f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36556f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36566fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36566fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36566fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36556f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36556f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.286+0000 7f36556f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f36556f8640 10 monclient(hunting): handle_auth_done global_id 5233 payload 824 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f36556f8640 1 -- 172.21.15.178:0/2385234166 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3650067130 msgr2=0x7f365013fce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:42.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f36556f8640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3650067130 0x7f365013fce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f36556f8640 1 -- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365009f000 msgr2=0x7f3650134000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f36556f8640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365009f000 0x7f3650134000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f36556f8640 10 monclient: _finish_hunting 0 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f36556f8640 1 monclient: found mon.noname-c 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f36556f8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f36556f8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f36556f8640 1 -- 172.21.15.178:0/2385234166 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3648009ae0 con 0x7f365009d580 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f36566fa640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365009f000 0x7f3650134000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f36556f8640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f365009d580 0x7f3650133ac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f364000c4f0 tx=0x7f3640007df0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3640011a10 con 0x7f365009d580 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3645ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3645ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:42.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3645ffb640 10 monclient: dump: 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.309 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3645ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3645ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f365815d640 5 monclient: authenticate success, global_id 5233 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640011bb0 con 0x7f365009d580 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3645ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f365815d640 10 monclient: _renew_subs 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3640011d20 con 0x7f365009d580 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f365815d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f365815d640 1 -- 172.21.15.178:0/2385234166 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3650120be0 con 0x7f365009d580 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3645ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3647fff640 4 set_mon_vals no callback set 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3645ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:42.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f3645ffb640 10 monclient: dump: 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.311 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f365815d640 10 monclient: _renew_subs 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f365815d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.287+0000 7f365815d640 1 -- 172.21.15.178:0/2385234166 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365011fcf0 con 0x7f365009d580 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f365815d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3640011590 con 0x7f365009d580 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f365815d640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3650140220 0x7f365011d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f365815d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f365815d640 1 -- 172.21.15.178:0/2385234166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365011f6f0 con 0x7f3650140220 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3655ef9640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3650140220 0x7f365011d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3655ef9640 10 monclient: get_auth_request con 0x7f3650140220 auth_method 0 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3655ef9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3655ef9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3655ef9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3655ef9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3655ef9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3655ef9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3645ffb640 1 --2- 172.21.15.178:0/2385234166 >> v2:172.21.15.204:6812/28210 conn(0x7f36340397b0 0x7f363403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5312+0+0 (secure 0 0 0) 0x7f364004a9f0 con 0x7f365009d580 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3655ef9640 10 monclient(hunting): handle_auth_done global_id 5445 payload 824 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.288+0000 7f3655ef9640 1 -- 172.21.15.178:0/2385234166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f365004c4a0 con 0x7f3650140220 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f36566fa640 1 --2- 172.21.15.178:0/2385234166 >> v2:172.21.15.204:6812/28210 conn(0x7f36340397b0 0x7f363403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f36566fa640 10 monclient: get_auth_request con 0x7f36340397b0 auth_method 0 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f3655ef9640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3650140220 0x7f365011d620 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f364c00e8d0 tx=0x7f364c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f364c013470 con 0x7f3650140220 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f3645ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f3645ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:42.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f3645ffb640 10 monclient: dump: 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.314 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.315 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.315 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f364c00ca30 con 0x7f3650140220 2024-09-15T01:34:42.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f3645ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f3645ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:42.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3650140220 msgr2=0x7f365011d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f3645ffb640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3650140220 0x7f365011d620 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f364c00e8d0 tx=0x7f364c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3650140220 msgr2=0x7f365011d620 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:42.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f3645ffb640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3650140220 0x7f365011d620 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f364c00e8d0 tx=0x7f364c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.289+0000 7f36566fa640 1 --2- 172.21.15.178:0/2385234166 >> v2:172.21.15.204:6812/28210 conn(0x7f36340397b0 0x7f363403bc60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3648009f20 tx=0x7f3648036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f365815d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:42.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f365815d640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365011db60 0x7f365011df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f365815d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f365815d640 1 -- 172.21.15.178:0/2385234166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365009ec40 con 0x7f365011db60 2024-09-15T01:34:42.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3655ef9640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365011db60 0x7f365011df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3655ef9640 10 monclient: get_auth_request con 0x7f365011db60 auth_method 0 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3655ef9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3655ef9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3655ef9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3655ef9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3655ef9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3655ef9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3655ef9640 10 monclient(hunting): handle_auth_done global_id 5454 payload 824 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3655ef9640 1 -- 172.21.15.178:0/2385234166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f365004bc80 con 0x7f365011db60 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3655ef9640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365011db60 0x7f365011df60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f364c00e8d0 tx=0x7f364c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f364c005c50 con 0x7f365011db60 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3645ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3645ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3645ffb640 10 monclient: dump: 2024-09-15T01:34:42.318 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.319 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9108 (secure 0 0 0) 0x7f364c00c640 con 0x7f365011db60 2024-09-15T01:34:42.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3645ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:42.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3645ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:42.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365011db60 msgr2=0x7f365011df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.295+0000 7f3645ffb640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365011db60 0x7f365011df60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f364c00e8d0 tx=0x7f364c019000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f3645ffb640 1 -- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365011db60 msgr2=0x7f365011df60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:42.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f3645ffb640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f365011db60 0x7f365011df60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.320 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:42.320 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:42.320 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:42.320 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 44, 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:42.321 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.322 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:42.323 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.324 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:42.325 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:42.326 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.327 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.328 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.329 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.330 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:42.331 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.332 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:42.333 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.334 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.335 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:42.336 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.337 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.338 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.339 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:42.340 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.341 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.341 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:42.341 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.341 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:42.341 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:42.341 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:42.341 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.341 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.341 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.341 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:42.341 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:42.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f365815d640 1 -- 172.21.15.178:0/2385234166 >> v2:172.21.15.204:6812/28210 conn(0x7f36340397b0 msgr2=0x7f363403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f365815d640 1 --2- 172.21.15.178:0/2385234166 >> v2:172.21.15.204:6812/28210 conn(0x7f36340397b0 0x7f363403bc60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3648009f20 tx=0x7f3648036000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f365815d640 10 monclient: shutdown 2024-09-15T01:34:42.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f365815d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f365815d640 1 -- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f365009d580 msgr2=0x7f3650133ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f365815d640 1 --2- 172.21.15.178:0/2385234166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f365009d580 0x7f3650133ac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f364000c4f0 tx=0x7f3640007df0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f36566fa640 1 -- 172.21.15.178:0/2385234166 reap_dead start 2024-09-15T01:34:42.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f36566fa640 1 -- 172.21.15.178:0/2385234166 reap_dead start 2024-09-15T01:34:42.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f365815d640 1 -- 172.21.15.178:0/2385234166 shutdown_connections 2024-09-15T01:34:42.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f365815d640 1 -- 172.21.15.178:0/2385234166 >> 172.21.15.178:0/2385234166 conn(0x7f3650068a60 msgr2=0x7f36500a09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f365815d640 1 -- 172.21.15.178:0/2385234166 shutdown_connections 2024-09-15T01:34:42.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.296+0000 7f365815d640 1 -- 172.21.15.178:0/2385234166 wait complete. 2024-09-15T01:34:42.343 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T01:34:42.343 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon c 2024-09-15T01:34:42.344 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:34:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.433+0000 7f7601f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.433+0000 7f7601f9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.433+0000 7f7601f9d640 10 monclient: monmap: 2024-09-15T01:34:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:42.435584+0000 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:42.435584+0000 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:42.438 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 1 Processor -- start 2024-09-15T01:34:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 1 -- start start 2024-09-15T01:34:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 10 monclient: init 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 10 monclient: _add_conns ranks=[9,1,6,5,3,8,7,10,2,0,4] 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f75fc0a2930 0x7f75fc132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 10 monclient(hunting): picked mon.noname-a con 0x7f75fc0a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f75fc09faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f75fc09fe70 0x7f75fc0a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 10 monclient(hunting): picked mon.noname-d con 0x7f75fc09fe70 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75fc04bf60 con 0x7f75fc09faa0 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75fc09dd80 con 0x7f75fc0a2930 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.434+0000 7f7601f9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0a2d00 con 0x7f75fc09fe70 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f7601f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f7601f9d640 10 monclient(hunting): authenticate will time out at 887.186011s 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fbfff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f75fc09fe70 msgr2=0x7f75fc0a0fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fbfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f75fc09fe70 0x7f75fc0a0fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f75fc0a2930 0x7f75fc132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f75fc0a2930 0x7f75fc132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44786/0 (socket says 172.21.15.178:44786) 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 1 -- 172.21.15.178:0/4105950175 learned_addr learned my addr 172.21.15.178:0/4105950175 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 10 monclient(hunting): get_auth_request con 0x7f75fc0a2930 auth_method 0 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 10 monclient(hunting): handle_auth_done global_id 5463 payload 274 2024-09-15T01:34:42.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 1 -- 172.21.15.178:0/4105950175 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f75fc09fe70 msgr2=0x7f75fc0a0fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:42.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 1 --2- 172.21.15.178:0/4105950175 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f75fc09fe70 0x7f75fc0a0fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 1 -- 172.21.15.178:0/4105950175 >> v1:172.21.15.204:6789/0 conn(0x7f75fc09faa0 legacy=0x7f75fc0a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:42.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 1 monclient: found mon.noname-a 2024-09-15T01:34:42.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:42.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:42.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.435+0000 7f75fb7fe640 1 -- 172.21.15.178:0/4105950175 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc133440 con 0x7f75fc0a2930 2024-09-15T01:34:42.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fb7fe640 1 --2- 172.21.15.178:0/4105950175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75fc0a2930 0x7f75fc132de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009910 tx=0x7f75ec0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aeabf6147cb487b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fa7fc640 1 -- 172.21.15.178:0/4105950175 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75ec02ad40 con 0x7f75fc0a2930 2024-09-15T01:34:42.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fa7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fa7fc640 10 monclient: dump: 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.451 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fa7fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fa7fc640 1 -- 172.21.15.178:0/4105950175 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec031820 con 0x7f75fc0a2930 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 5 monclient: authenticate success, global_id 5463 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fa7fc640 1 -- 172.21.15.178:0/4105950175 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75ec031e90 con 0x7f75fc0a2930 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 4 set_mon_vals no callback set 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f75fa7fc640 10 monclient: dump: 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 10 monclient: shutdown 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 1 -- 172.21.15.178:0/4105950175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75fc0a2930 msgr2=0x7f75fc132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 1 --2- 172.21.15.178:0/4105950175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75fc0a2930 0x7f75fc132de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009910 tx=0x7f75ec0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 1 -- 172.21.15.178:0/4105950175 shutdown_connections 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 1 --2- 172.21.15.178:0/4105950175 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f75fc09fe70 0x7f75fc0a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 1 --2- 172.21.15.178:0/4105950175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75fc0a2930 0x7f75fc132de0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 1 -- 172.21.15.178:0/4105950175 >> 172.21.15.178:0/4105950175 conn(0x7f75fc0a5ed0 msgr2=0x7f75fc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 1 -- 172.21.15.178:0/4105950175 shutdown_connections 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.436+0000 7f7601f9d640 1 -- 172.21.15.178:0/4105950175 wait complete. 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.437+0000 7f7601f9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.437+0000 7f7601f9d640 10 monclient: monmap: 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:42.454 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:42.455 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.437+0000 7f7601f9d640 1 Processor -- start 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.437+0000 7f7601f9d640 1 -- start start 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.437+0000 7f7601f9d640 10 monclient: init 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 10 monclient: _add_conns ranks=[8,3,2,0,6,7,5,4,1] 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f75fc09faa0 0x7f75fc09b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f75fc09faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75fc09fe70 0x7f75fc131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 10 monclient(hunting): picked mon.noname-g con 0x7f75fc09fe70 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75fc0a2930 0x7f75fc141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f75fc0a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc11ed20 con 0x7f75fc0a2930 2024-09-15T01:34:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc09d610 con 0x7f75fc09fe70 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc11ea40 con 0x7f75fc09faa0 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f7601f9d640 10 monclient(hunting): authenticate will time out at 887.189458s 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fbfff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75fc0a2930 msgr2=0x7f75fc141e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fbfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75fc0a2930 0x7f75fc141e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f75fc09faa0 0x7f75fc09b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f75fc09faa0 0x7f75fc09b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:36834/0 (socket says 172.21.15.178:36834) 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 1 -- 172.21.15.178:0/1193343380 learned_addr learned my addr 172.21.15.178:0/1193343380 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 10 monclient(hunting): get_auth_request con 0x7f75fc09faa0 auth_method 0 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75faffd640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75fc09fe70 0x7f75fc131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75faffd640 10 monclient(hunting): get_auth_request con 0x7f75fc09fe70 auth_method 0 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75faffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75faffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 10 monclient(hunting): handle_auth_done global_id 5309 payload 824 2024-09-15T01:34:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 1 -- 172.21.15.178:0/1193343380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75fc09fe70 msgr2=0x7f75fc131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75fc09fe70 0x7f75fc131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 1 -- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75fc0a2930 msgr2=0x7f75fc141e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75fc0a2930 0x7f75fc141e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 1 monclient: found mon.noname-e 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 1 -- 172.21.15.178:0/1193343380 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec009450 con 0x7f75fc09faa0 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.438+0000 7f75fb7fe640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f75fc09faa0 0x7f75fc09b150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f75ec000c00 tx=0x7f75ec032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75ec03a390 con 0x7f75fc09faa0 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 10 monclient: dump: 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.460 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec03a510 con 0x7f75fc09faa0 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f7601f9d640 5 monclient: authenticate success, global_id 5309 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f7601f9d640 10 monclient: _renew_subs 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f7601f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f7601f9d640 1 -- 172.21.15.178:0/1193343380 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75fc11d630 con 0x7f75fc09faa0 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75ec03ab40 con 0x7f75fc09faa0 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:42.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 10 monclient: dump: 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.462 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75f9ffb640 4 set_mon_vals no callback set 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f7601f9d640 10 monclient: _renew_subs 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f7601f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f7601f9d640 1 -- 172.21.15.178:0/1193343380 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75fc11cba0 con 0x7f75fc09faa0 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f7601f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f7601f9d640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc11dae0 0x7f75fc142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f7601f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f7601f9d640 1 -- 172.21.15.178:0/1193343380 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc11f190 con 0x7f75fc11dae0 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.439+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f75ec039040 con 0x7f75fc09faa0 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75faffd640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc11dae0 0x7f75fc142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75faffd640 10 monclient: get_auth_request con 0x7f75fc11dae0 auth_method 0 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75faffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75faffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 1 --2- 172.21.15.178:0/1193343380 >> v2:172.21.15.204:6812/28210 conn(0x7f75e80396e0 0x7f75e803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5312+0+0 (secure 0 0 0) 0x7f75ec06b2c0 con 0x7f75fc09faa0 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75faffd640 10 monclient(hunting): handle_auth_done global_id 5242 payload 824 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75faffd640 1 -- 172.21.15.178:0/1193343380 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75fc04c4a0 con 0x7f75fc11dae0 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75fbfff640 1 --2- 172.21.15.178:0/1193343380 >> v2:172.21.15.204:6812/28210 conn(0x7f75e80396e0 0x7f75e803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75fbfff640 10 monclient: get_auth_request con 0x7f75e80396e0 auth_method 0 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75faffd640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc11dae0 0x7f75fc142360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f75f0000c00 tx=0x7f75f0007e10 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75f0002b20 con 0x7f75fc11dae0 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 10 monclient: dump: 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.465 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.466 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.466 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.466 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75fbfff640 1 --2- 172.21.15.178:0/1193343380 >> v2:172.21.15.204:6812/28210 conn(0x7f75e80396e0 0x7f75e803bb90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f75f4003b90 tx=0x7f75f4009290 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f75f0002c80 con 0x7f75fc11dae0 2024-09-15T01:34:42.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:42.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc11dae0 msgr2=0x7f75fc142360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc11dae0 0x7f75fc142360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f75f0000c00 tx=0x7f75f0007e10 comp rx=0 tx=0).stop 2024-09-15T01:34:42.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc11dae0 msgr2=0x7f75fc142360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:42.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.440+0000 7f75e37fe640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc11dae0 0x7f75fc142360 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f75f0000c00 tx=0x7f75f0007e10 comp rx=0 tx=0).stop 2024-09-15T01:34:42.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f7601f9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:42.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f7601f9d640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc131e60 0x7f75fc1428a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f7601f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f7601f9d640 1 -- 172.21.15.178:0/1193343380 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc09f800 con 0x7f75fc131e60 2024-09-15T01:34:42.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f75faffd640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc131e60 0x7f75fc1428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f75faffd640 10 monclient: get_auth_request con 0x7f75fc131e60 auth_method 0 2024-09-15T01:34:42.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f75faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f75faffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f75faffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f75faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f75faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f75faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f75faffd640 10 monclient(hunting): handle_auth_done global_id 5251 payload 824 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.446+0000 7f75faffd640 1 -- 172.21.15.178:0/1193343380 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f75fc11ee90 con 0x7f75fc131e60 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75faffd640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc131e60 0x7f75fc1428a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f75f0000c00 tx=0x7f75f0018000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75f00074b0 con 0x7f75fc131e60 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75e37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75e37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75e37fe640 10 monclient: dump: 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.469 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9108 (secure 0 0 0) 0x7f75f0002030 con 0x7f75fc131e60 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75e37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:42.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75e37fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:42.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc131e60 msgr2=0x7f75fc1428a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75e37fe640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc131e60 0x7f75fc1428a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f75f0000c00 tx=0x7f75f0018000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75e37fe640 1 -- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc131e60 msgr2=0x7f75fc1428a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:42.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75e37fe640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75fc131e60 0x7f75fc1428a0 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f75f0000c00 tx=0x7f75f0018000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.471 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 44, 2024-09-15T01:34:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.484 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.485 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.486 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.487 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.488 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.489 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.490 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:42.491 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:42.492 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:42.492 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.492 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.492 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.492 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:42.492 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:42.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f7601f9d640 1 -- 172.21.15.178:0/1193343380 >> v2:172.21.15.204:6812/28210 conn(0x7f75e80396e0 msgr2=0x7f75e803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f7601f9d640 1 --2- 172.21.15.178:0/1193343380 >> v2:172.21.15.204:6812/28210 conn(0x7f75e80396e0 0x7f75e803bb90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f75f4003b90 tx=0x7f75f4009290 comp rx=0 tx=0).stop 2024-09-15T01:34:42.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f7601f9d640 10 monclient: shutdown 2024-09-15T01:34:42.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f7601f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f7601f9d640 1 -- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f75fc09faa0 msgr2=0x7f75fc09b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f7601f9d640 1 --2- 172.21.15.178:0/1193343380 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f75fc09faa0 0x7f75fc09b150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f75ec000c00 tx=0x7f75ec032000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75fbfff640 1 -- 172.21.15.178:0/1193343380 reap_dead start 2024-09-15T01:34:42.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f75fbfff640 1 -- 172.21.15.178:0/1193343380 reap_dead start 2024-09-15T01:34:42.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f7601f9d640 1 -- 172.21.15.178:0/1193343380 shutdown_connections 2024-09-15T01:34:42.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f7601f9d640 1 -- 172.21.15.178:0/1193343380 >> 172.21.15.178:0/1193343380 conn(0x7f75fc0a5ed0 msgr2=0x7f75fc09e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f7601f9d640 1 -- 172.21.15.178:0/1193343380 shutdown_connections 2024-09-15T01:34:42.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.447+0000 7f7601f9d640 1 -- 172.21.15.178:0/1193343380 wait complete. 2024-09-15T01:34:42.494 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T01:34:42.494 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-09-15T01:34:42.494 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:34:42.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.586+0000 7fa3ed8c9640 10 monclient: get_monmap_and_config 2024-09-15T01:34:42.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.586+0000 7fa3ed8c9640 10 monclient: build_initial_monmap 2024-09-15T01:34:42.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.586+0000 7fa3ed8c9640 10 monclient: monmap: 2024-09-15T01:34:42.589 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:42.589 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.589 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:42.588408+0000 2024-09-15T01:34:42.589 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:42.588408+0000 2024-09-15T01:34:42.589 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:42.589 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.590 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:42.590 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:42.590 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:42.590 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:42.590 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:42.590 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:42.590 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:42.590 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:42.590 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:42.590 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:42.590 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:42.590 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 1 Processor -- start 2024-09-15T01:34:42.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 1 -- start start 2024-09-15T01:34:42.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 10 monclient: init 2024-09-15T01:34:42.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:42.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 10 monclient: _add_conns ranks=[4,1,10,7,2,0,8,5,6,9,3] 2024-09-15T01:34:42.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e80a2930 0x7fa3e8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 10 monclient(hunting): picked mon.noname-c con 0x7fa3e80a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:42.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa3e809faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:42.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa3e809fe70 0x7fa3e80a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 10 monclient(hunting): picked mon.noname-f con 0x7fa3e809fe70 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:42.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e804bf60 con 0x7fa3e809faa0 2024-09-15T01:34:42.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e809dd80 con 0x7fa3e809fe70 2024-09-15T01:34:42.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80a2d00 con 0x7fa3e80a2930 2024-09-15T01:34:42.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:42.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3ed8c9640 10 monclient(hunting): authenticate will time out at 887.338654s 2024-09-15T01:34:42.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e80a2930 0x7fa3e8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e80a2930 0x7fa3e8132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44870/0 (socket says 172.21.15.178:44870) 2024-09-15T01:34:42.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 1 -- 172.21.15.178:0/339687536 learned_addr learned my addr 172.21.15.178:0/339687536 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:42.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 10 monclient(hunting): get_auth_request con 0x7fa3e80a2930 auth_method 0 2024-09-15T01:34:42.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 10 monclient(hunting): handle_auth_done global_id 5260 payload 274 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 1 -- 172.21.15.178:0/339687536 >> v2:172.21.15.204:3300/0 conn(0x7fa3e809fe70 msgr2=0x7fa3e80a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 1 --2- 172.21.15.178:0/339687536 >> v2:172.21.15.204:3300/0 conn(0x7fa3e809fe70 0x7fa3e80a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 1 -- 172.21.15.178:0/339687536 >> v1:172.21.15.204:6789/0 conn(0x7fa3e809faa0 legacy=0x7fa3e80a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 1 monclient: found mon.noname-c 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:42.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:42.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.587+0000 7fa3e6ffd640 1 -- 172.21.15.178:0/339687536 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e80a0310 con 0x7fa3e80a2930 2024-09-15T01:34:42.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.589+0000 7fa3e6ffd640 1 --2- 172.21.15.178:0/339687536 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e80a2930 0x7fa3e8132de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009910 tx=0x7fa3d80091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=18b8193f5a2325dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3e5ffb640 1 -- 172.21.15.178:0/339687536 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa3d802ac90 con 0x7fa3e80a2930 2024-09-15T01:34:42.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3e5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:42.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3e5ffb640 10 monclient: dump: 2024-09-15T01:34:42.597 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.597 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.597 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.597 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.597 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.597 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3e5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:42.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3e5ffb640 1 -- 172.21.15.178:0/339687536 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d802ae10 con 0x7fa3e80a2930 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3ed8c9640 5 monclient: authenticate success, global_id 5260 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3ed8c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3e5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3e5ffb640 1 -- 172.21.15.178:0/339687536 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa3d8031ca0 con 0x7fa3e80a2930 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3e5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3e5ffb640 10 monclient: dump: 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.599 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.600 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.600 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.600 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.600 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.600 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.600 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.600 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.600 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.600 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.600 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3ed8c9640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:42.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3ed8c9640 4 set_mon_vals no callback set 2024-09-15T01:34:42.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.596+0000 7ffa43fff640 1 -- 172.21.15.204:0/1289498309 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host1"}]=0 removed item id -11 name 'host1' from crush map v81) v1 ==== 124+0+0 (secure 0 0 0) 0x7ffa54040020 con 0x7ffa5c137d20 2024-09-15T01:34:42.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.596+0000 7ffa43fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host1"}] 2024-09-15T01:34:42.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.596+0000 7ffa43fff640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host1' from crush map 2024-09-15T01:34:42.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3ed8c9640 10 monclient: shutdown 2024-09-15T01:34:42.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3ed8c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/339687536 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e80a2930 msgr2=0x7fa3e8132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3ed8c9640 1 --2- 172.21.15.178:0/339687536 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e80a2930 0x7fa3e8132de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009910 tx=0x7fa3d80091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/339687536 shutdown_connections 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3ed8c9640 1 --2- 172.21.15.178:0/339687536 >> v2:172.21.15.204:3300/0 conn(0x7fa3e809fe70 0x7fa3e80a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3ed8c9640 1 --2- 172.21.15.178:0/339687536 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e80a2930 0x7fa3e8132de0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.591+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/339687536 >> 172.21.15.178:0/339687536 conn(0x7fa3e80a5ed0 msgr2=0x7fa3e80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.592+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/339687536 shutdown_connections 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.592+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/339687536 wait complete. 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.592+0000 7fa3ed8c9640 10 monclient: build_initial_monmap 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.592+0000 7fa3ed8c9640 10 monclient: monmap: 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.592+0000 7fa3ed8c9640 1 Processor -- start 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 1 -- start start 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 10 monclient: init 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 10 monclient: _add_conns ranks=[6,4,2,8,7,5,3,1,0] 2024-09-15T01:34:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa3e809faa0 0x7fa3e809b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 10 monclient(hunting): picked mon.noname-d con 0x7fa3e809faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e809fe70 0x7fa3e811ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 10 monclient(hunting): picked mon.noname-c con 0x7fa3e809fe70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3e80a2930 0x7fa3e8131950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 10 monclient(hunting): picked mon.noname-b con 0x7fa3e80a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e811d590 con 0x7fa3e80a2930 2024-09-15T01:34:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e809f870 con 0x7fa3e809fe70 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e811db80 con 0x7fa3e809faa0 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e6ffd640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa3e809faa0 msgr2=0x7fa3e809b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3ed8c9640 10 monclient(hunting): authenticate will time out at 887.344509s 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e6ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa3e809faa0 0x7fa3e809b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e77fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3e80a2930 msgr2=0x7fa3e8131950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e77fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3e80a2930 0x7fa3e8131950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e809fe70 0x7fa3e811ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e809fe70 0x7fa3e811ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44880/0 (socket says 172.21.15.178:44880) 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 1 -- 172.21.15.178:0/1087576863 learned_addr learned my addr 172.21.15.178:0/1087576863 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 10 monclient(hunting): get_auth_request con 0x7fa3e809fe70 auth_method 0 2024-09-15T01:34:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 10 monclient(hunting): handle_auth_done global_id 5269 payload 824 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 1 -- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa3e809faa0 msgr2=0x7fa3e809b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa3e809faa0 0x7fa3e809b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 1 -- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3e80a2930 msgr2=0x7fa3e8131950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3e80a2930 0x7fa3e8131950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 10 monclient: _finish_hunting 0 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 1 monclient: found mon.noname-c 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.593+0000 7fa3e67fc640 1 -- 172.21.15.178:0/1087576863 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d8009450 con 0x7fa3e809fe70 2024-09-15T01:34:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3e67fc640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e809fe70 0x7fa3e811ca00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00b4c0 tx=0x7fa3dc009060 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa3dc00f990 con 0x7fa3e809fe70 2024-09-15T01:34:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ceffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ceffd640 10 monclient: dump: 2024-09-15T01:34:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.616 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.616 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.616 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.616 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.616 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.616 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.616 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.616 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.616 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.616 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ceffd640 10 monclient: _finish_auth 0 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc00fb10 con 0x7fa3e809fe70 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ed8c9640 5 monclient: authenticate success, global_id 5269 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ed8c9640 10 monclient: _renew_subs 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ed8c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/1087576863 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e811ea70 con 0x7fa3e809fe70 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ceffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa3dc00fd00 con 0x7fa3e809fe70 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ceffd640 10 monclient: dump: 2024-09-15T01:34:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.619 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3e57fa640 4 set_mon_vals no callback set 2024-09-15T01:34:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ed8c9640 10 monclient: _renew_subs 2024-09-15T01:34:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ed8c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.594+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/1087576863 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e811f030 con 0x7fa3e809fe70 2024-09-15T01:34:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3ed8c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3ed8c9640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e8131e90 0x7fa3e8141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3ed8c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/1087576863 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e804c4a0 con 0x7fa3e8131e90 2024-09-15T01:34:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa3dc00f510 con 0x7fa3e809fe70 2024-09-15T01:34:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e6ffd640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e8131e90 0x7fa3e8141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e6ffd640 10 monclient: get_auth_request con 0x7fa3e8131e90 auth_method 0 2024-09-15T01:34:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3ceffd640 1 --2- 172.21.15.178:0/1087576863 >> v2:172.21.15.204:6812/28210 conn(0x7fa3d40397b0 0x7fa3d403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa3dc04b050 con 0x7fa3e809fe70 2024-09-15T01:34:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e6ffd640 10 monclient(hunting): handle_auth_done global_id 5318 payload 824 2024-09-15T01:34:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e6ffd640 1 -- 172.21.15.178:0/1087576863 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3e805ed70 con 0x7fa3e8131e90 2024-09-15T01:34:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e77fe640 1 --2- 172.21.15.178:0/1087576863 >> v2:172.21.15.204:6812/28210 conn(0x7fa3d40397b0 0x7fa3d403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e77fe640 10 monclient: get_auth_request con 0x7fa3d40397b0 auth_method 0 2024-09-15T01:34:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3e6ffd640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e8131e90 0x7fa3e8141f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8067590 tx=0x7fa3d8032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa3d802ac90 con 0x7fa3e8131e90 2024-09-15T01:34:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3ceffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.595+0000 7fa3ceffd640 10 monclient: dump: 2024-09-15T01:34:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.596+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fa3d8031e40 con 0x7fa3e8131e90 2024-09-15T01:34:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.596+0000 7fa3ceffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.596+0000 7fa3ceffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.596+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e8131e90 msgr2=0x7fa3e8141f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.596+0000 7fa3ceffd640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e8131e90 0x7fa3e8141f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8067590 tx=0x7fa3d8032000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.596+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e8131e90 msgr2=0x7fa3e8141f90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.596+0000 7fa3ceffd640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e8131e90 0x7fa3e8141f90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.596+0000 7fa3e77fe640 1 --2- 172.21.15.178:0/1087576863 >> v2:172.21.15.204:6812/28210 conn(0x7fa3d40397b0 0x7fa3d403bc60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00032a0 tx=0x7fa3e0002d60 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.601+0000 7fa3ed8c9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.601+0000 7fa3ed8c9640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e81424d0 0x7fa3e81428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.601+0000 7fa3ed8c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.601+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/1087576863 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e811ec00 con 0x7fa3e81424d0 2024-09-15T01:34:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.601+0000 7fa3e6ffd640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e81424d0 0x7fa3e81428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.601+0000 7fa3e6ffd640 10 monclient: get_auth_request con 0x7fa3e81424d0 auth_method 0 2024-09-15T01:34:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.601+0000 7fa3e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.601+0000 7fa3e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.601+0000 7fa3e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.601+0000 7fa3e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.601+0000 7fa3e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.601+0000 7fa3e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3e6ffd640 10 monclient(hunting): handle_auth_done global_id 5327 payload 824 2024-09-15T01:34:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3e6ffd640 1 -- 172.21.15.178:0/1087576863 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa3e809c110 con 0x7fa3e81424d0 2024-09-15T01:34:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3e6ffd640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e81424d0 0x7fa3e81428d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009910 tx=0x7fa3d803c000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa3d804d070 con 0x7fa3e81424d0 2024-09-15T01:34:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3ceffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3ceffd640 10 monclient: dump: 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.626 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9108 (secure 0 0 0) 0x7fa3d8049680 con 0x7fa3e81424d0 2024-09-15T01:34:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3ceffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3ceffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e81424d0 msgr2=0x7fa3e81428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3ceffd640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e81424d0 0x7fa3e81428d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009910 tx=0x7fa3d803c000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3ceffd640 1 -- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e81424d0 msgr2=0x7fa3e81428d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.602+0000 7fa3ceffd640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3e81424d0 0x7fa3e81428d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 44, 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.628 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.629 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:42.630 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:42.631 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.632 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:42.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.632 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:42.632 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:42.632 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.632 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.632 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.632 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:42.632 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.632 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.633 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.633 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:42.633 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.633 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.633 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.633 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:42.633 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:42.633 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.633 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.633 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.634 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.634 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.634 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:42.634 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:42.634 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.634 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.634 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.634 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.634 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:42.634 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.634 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.634 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:42.637 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.637 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.637 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.637 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:42.637 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:42.637 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.637 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.637 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.637 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:42.638 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:42.638 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.638 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.638 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.638 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.638 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:42.638 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.638 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.638 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.638 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:42.638 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:42.639 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.641 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:42.641 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:42.641 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.641 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.641 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.641 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.641 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.641 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:42.641 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:42.641 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.641 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.641 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:42.642 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:42.649 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:42.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.603+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/1087576863 >> v2:172.21.15.204:6812/28210 conn(0x7fa3d40397b0 msgr2=0x7fa3d403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.603+0000 7fa3ed8c9640 1 --2- 172.21.15.178:0/1087576863 >> v2:172.21.15.204:6812/28210 conn(0x7fa3d40397b0 0x7fa3d403bc60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00032a0 tx=0x7fa3e0002d60 comp rx=0 tx=0).stop 2024-09-15T01:34:42.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.603+0000 7fa3ed8c9640 10 monclient: shutdown 2024-09-15T01:34:42.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.603+0000 7fa3ed8c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.603+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e809fe70 msgr2=0x7fa3e811ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.603+0000 7fa3ed8c9640 1 --2- 172.21.15.178:0/1087576863 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3e809fe70 0x7fa3e811ca00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00b4c0 tx=0x7fa3dc009060 comp rx=0 tx=0).stop 2024-09-15T01:34:42.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.603+0000 7fa3e77fe640 1 -- 172.21.15.178:0/1087576863 reap_dead start 2024-09-15T01:34:42.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.603+0000 7fa3e77fe640 1 -- 172.21.15.178:0/1087576863 reap_dead start 2024-09-15T01:34:42.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.603+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/1087576863 shutdown_connections 2024-09-15T01:34:42.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.603+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/1087576863 >> 172.21.15.178:0/1087576863 conn(0x7fa3e80a5ed0 msgr2=0x7fa3e80a57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.603+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/1087576863 shutdown_connections 2024-09-15T01:34:42.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.603+0000 7fa3ed8c9640 1 -- 172.21.15.178:0/1087576863 wait complete. 2024-09-15T01:34:42.652 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T01:34:42.652 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon f 2024-09-15T01:34:42.652 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:34:42.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.697+0000 7ffa6295d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host1"}] 2024-09-15T01:34:42.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.697+0000 7ffa6295d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "host1"}] 2024-09-15T01:34:42.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.697+0000 7ffa6295d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:42.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.697+0000 7ffa6295d640 1 -- 172.21.15.204:0/1289498309 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "host1"} v 0) v1 -- 0x7ffa5c131370 con 0x7ffa5c137d20 2024-09-15T01:34:42.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.698+0000 7ffa43fff640 1 -- 172.21.15.204:0/1289498309 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host1"}]=0 device 'host1' does not appear in the crush map v81) v1 ==== 124+0+0 (secure 0 0 0) 0x7ffa5403a310 con 0x7ffa5c137d20 2024-09-15T01:34:42.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.698+0000 7ffa43fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "host1"}] 2024-09-15T01:34:42.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.698+0000 7ffa43fff640 10 monclient: _finish_command 3 = system:0 device 'host1' does not appear in the crush map 2024-09-15T01:34:42.699 INFO:tasks.workunit.client.0.smithi204.stderr:device 'host1' does not appear in the crush map 2024-09-15T01:34:42.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.699+0000 7ffa6295d640 1 -- 172.21.15.204:0/1289498309 >> v2:172.21.15.204:6812/28210 conn(0x7ffa480397b0 msgr2=0x7ffa4803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.699+0000 7ffa6295d640 1 --2- 172.21.15.204:0/1289498309 >> v2:172.21.15.204:6812/28210 conn(0x7ffa480397b0 0x7ffa4803bc60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c007fd0 tx=0x7ffa4c007e30 comp rx=0 tx=0).stop 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.699+0000 7ffa6295d640 10 monclient: shutdown 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.699+0000 7ffa6295d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.699+0000 7ffa6295d640 1 -- 172.21.15.204:0/1289498309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 msgr2=0x7ffa5c099d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.699+0000 7ffa6295d640 1 --2- 172.21.15.204:0/1289498309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 0x7ffa5c099d20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009ab0 tx=0x7ffa54036000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.700+0000 7ffa6295d640 1 -- 172.21.15.204:0/1289498309 shutdown_connections 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.700+0000 7ffa6295d640 1 --2- 172.21.15.204:0/1289498309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffa5c137d20 0x7ffa5c099d20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.700+0000 7ffa6295d640 1 --2- 172.21.15.204:0/1289498309 >> v2:172.21.15.204:6812/28210 conn(0x7ffa480397b0 0x7ffa4803bc60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.700+0000 7ffa6295d640 1 --2- 172.21.15.204:0/1289498309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa5c0a2e30 0x7ffa5c0997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.700+0000 7ffa6295d640 1 --2- 172.21.15.204:0/1289498309 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa5c09fb80 0x7ffa5c0992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.700+0000 7ffa6295d640 1 -- 172.21.15.204:0/1289498309 >> 172.21.15.204:0/1289498309 conn(0x7ffa5c06e860 msgr2=0x7ffa5c0a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.700+0000 7ffa6295d640 1 -- 172.21.15.204:0/1289498309 shutdown_connections 2024-09-15T01:34:42.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.700+0000 7ffa6295d640 1 -- 172.21.15.204:0/1289498309 wait complete. 2024-09-15T01:34:42.710 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:34:42.710 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -c host1 2024-09-15T01:34:42.711 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -q 0 2024-09-15T01:34:42.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.743+0000 7fe38d48b640 10 monclient: get_monmap_and_config 2024-09-15T01:34:42.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.743+0000 7fe38d48b640 10 monclient: build_initial_monmap 2024-09-15T01:34:42.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.743+0000 7fe38d48b640 10 monclient: monmap: 2024-09-15T01:34:42.746 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:42.746 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:42.745534+0000 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:42.745534+0000 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:42.747 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:42.748 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:42.748 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 1 Processor -- start 2024-09-15T01:34:42.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 1 -- start start 2024-09-15T01:34:42.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 10 monclient: init 2024-09-15T01:34:42.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:42.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 10 monclient: _add_conns ranks=[3,5,0,2,6,4,7,10,9,8,1] 2024-09-15T01:34:42.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3880678d0 0x7fe38809e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 10 monclient(hunting): picked mon.noname-g con 0x7fe3880678d0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:42.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe38806ea30 0x7fe38809ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 10 monclient(hunting): picked mon.noname-h con 0x7fe38806ea30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:42.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe38809f180 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:42.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38804bf60 con 0x7fe38809f180 2024-09-15T01:34:42.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38809f590 con 0x7fe3880678d0 2024-09-15T01:34:42.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe38806d5b0 con 0x7fe38806ea30 2024-09-15T01:34:42.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe38d48b640 10 monclient(hunting): authenticate will time out at 887.495771s 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe3877fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe38809f180 0x7fe3881306b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:51358/0 (socket says 172.21.15.178:51358) 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe3877fe640 1 -- 172.21.15.178:0/3939172867 learned_addr learned my addr 172.21.15.178:0/3939172867 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe37e7fc640 1 --2- 172.21.15.178:0/3939172867 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe38806ea30 0x7fe38809ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe386ffd640 1 --2- 172.21.15.178:0/3939172867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3880678d0 0x7fe38809e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe386ffd640 10 monclient(hunting): get_auth_request con 0x7fe3880678d0 auth_method 0 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe386ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe386ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.744+0000 7fe386ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe37e7fc640 10 monclient(hunting): get_auth_request con 0x7fe38806ea30 auth_method 0 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe37e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe37e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe37e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe386ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe386ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe386ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe37e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe37e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe37e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe3867fc640 1 -- 172.21.15.178:0/3939172867 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020336942 0 0) 0x7fe38804bf60 con 0x7fe38809f180 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe3867fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe3867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe3867fc640 10 monclient(hunting): my global_id is 5472 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe3867fc640 1 -- 172.21.15.178:0/3939172867 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3740034e0 con 0x7fe38809f180 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe386ffd640 10 monclient(hunting): handle_auth_done global_id 14358 payload 274 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe386ffd640 1 -- 172.21.15.178:0/3939172867 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe38806ea30 msgr2=0x7fe38809ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:42.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe386ffd640 1 --2- 172.21.15.178:0/3939172867 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe38806ea30 0x7fe38809ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe386ffd640 1 -- 172.21.15.178:0/3939172867 >> v1:172.21.15.178:6789/0 conn(0x7fe38809f180 legacy=0x7fe3881306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe386ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe386ffd640 1 monclient: found mon.noname-g 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe386ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe386ffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe386ffd640 1 -- 172.21.15.178:0/3939172867 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe388067ca0 con 0x7fe3880678d0 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.745+0000 7fe37e7fc640 1 --2- 172.21.15.178:0/3939172867 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe38806ea30 0x7fe38809ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe386ffd640 1 --2- 172.21.15.178:0/3939172867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3880678d0 0x7fe38809e700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe378009880 tx=0x7fe3780090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=fb91642b1e4d0b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe3867fc640 1 -- 172.21.15.178:0/3939172867 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe37802acd0 con 0x7fe3880678d0 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe3867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe3867fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe3867fc640 10 monclient: dump: 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.762 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe3867fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe3867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe3867fc640 1 -- 172.21.15.178:0/3939172867 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37802ae50 con 0x7fe3880678d0 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 5 monclient: authenticate success, global_id 14358 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe3867fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe3867fc640 1 -- 172.21.15.178:0/3939172867 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe378031ca0 con 0x7fe3880678d0 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe3867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 4 set_mon_vals no callback set 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe3867fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe3867fc640 10 monclient: dump: 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.764 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.765 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 10 monclient: shutdown 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 1 -- 172.21.15.178:0/3939172867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3880678d0 msgr2=0x7fe38809e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 1 --2- 172.21.15.178:0/3939172867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3880678d0 0x7fe38809e700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe378009880 tx=0x7fe3780090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 1 -- 172.21.15.178:0/3939172867 shutdown_connections 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 1 --2- 172.21.15.178:0/3939172867 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe38806ea30 0x7fe38809ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 1 --2- 172.21.15.178:0/3939172867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3880678d0 0x7fe38809e700 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 1 -- 172.21.15.178:0/3939172867 >> 172.21.15.178:0/3939172867 conn(0x7fe3880a4390 msgr2=0x7fe388067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 1 -- 172.21.15.178:0/3939172867 shutdown_connections 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.746+0000 7fe38d48b640 1 -- 172.21.15.178:0/3939172867 wait complete. 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.747+0000 7fe38d48b640 10 monclient: build_initial_monmap 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.747+0000 7fe38d48b640 10 monclient: monmap: 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:42.767 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.747+0000 7fe38d48b640 1 Processor -- start 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.747+0000 7fe38d48b640 1 -- start start 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.747+0000 7fe38d48b640 10 monclient: init 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 10 monclient: _add_conns ranks=[2,7,5,8,1,0,3,6,4] 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3880678d0 0x7fe388141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 10 monclient(hunting): picked mon.noname-b con 0x7fe3880678d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe38806ea30 0x7fe388133750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 10 monclient(hunting): picked mon.noname-i con 0x7fe38806ea30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe38809f180 0x7fe388068240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 10 monclient(hunting): picked mon.noname-h con 0x7fe38809f180 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38806c900 con 0x7fe3880678d0 2024-09-15T01:34:42.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe388122b70 con 0x7fe38809f180 2024-09-15T01:34:42.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe388069d10 con 0x7fe38806ea30 2024-09-15T01:34:42.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:42.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe386ffd640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3880678d0 msgr2=0x7fe388141860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:42.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe38d48b640 10 monclient(hunting): authenticate will time out at 887.499372s 2024-09-15T01:34:42.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe386ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3880678d0 0x7fe388141860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:42.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe37e7fc640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe38806ea30 msgr2=0x7fe388133750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:42.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe37e7fc640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe38806ea30 0x7fe388133750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:42.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe38809f180 0x7fe388068240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe38809f180 0x7fe388068240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:42140/0 (socket says 172.21.15.178:42140) 2024-09-15T01:34:42.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 1 -- 172.21.15.178:0/85489668 learned_addr learned my addr 172.21.15.178:0/85489668 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 10 monclient(hunting): get_auth_request con 0x7fe38809f180 auth_method 0 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 10 monclient(hunting): handle_auth_done global_id 5216 payload 824 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 1 -- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe38806ea30 msgr2=0x7fe388133750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe38806ea30 0x7fe388133750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 1 -- 172.21.15.178:0/85489668 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3880678d0 msgr2=0x7fe388141860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3880678d0 0x7fe388141860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:42.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 1 monclient: found mon.noname-h 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.748+0000 7fe3877fe640 1 -- 172.21.15.178:0/85489668 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe370009390 con 0x7fe38809f180 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe3877fe640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe38809f180 0x7fe388068240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe380001460 tx=0x7fe380001a40 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe38000ce30 con 0x7fe38809f180 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe37f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe37f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe37f7fe640 10 monclient: dump: 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.771 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe37f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe37f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38000acd0 con 0x7fe38809f180 2024-09-15T01:34:42.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe38d48b640 5 monclient: authenticate success, global_id 5216 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe38d48b640 10 monclient: _renew_subs 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe38d48b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe38d48b640 1 -- 172.21.15.178:0/85489668 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe378009390 con 0x7fe38809f180 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe37f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe38000a850 con 0x7fe38809f180 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe37f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe37f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe37f7fe640 10 monclient: dump: 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.773 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe38d48b640 10 monclient: _renew_subs 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe385ffb640 4 set_mon_vals no callback set 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe38d48b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:42.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe38d48b640 1 -- 172.21.15.178:0/85489668 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe388142660 con 0x7fe38809f180 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe38d48b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe38d48b640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388121930 0x7fe388068780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe38d48b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.749+0000 7fe38d48b640 1 -- 172.21.15.178:0/85489668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe388122a30 con 0x7fe388121930 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe38000a9d0 con 0x7fe38809f180 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe386ffd640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388121930 0x7fe388068780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe386ffd640 10 monclient: get_auth_request con 0x7fe388121930 auth_method 0 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe386ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe386ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe386ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe386ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe386ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe386ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe37f7fe640 1 --2- 172.21.15.178:0/85489668 >> v2:172.21.15.204:6812/28210 conn(0x7fe3680397b0 0x7fe36803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe38004b770 con 0x7fe38809f180 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe386ffd640 10 monclient(hunting): handle_auth_done global_id 5725 payload 824 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe386ffd640 1 -- 172.21.15.178:0/85489668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe38804c4a0 con 0x7fe388121930 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe37e7fc640 1 --2- 172.21.15.178:0/85489668 >> v2:172.21.15.204:6812/28210 conn(0x7fe3680397b0 0x7fe36803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe37e7fc640 10 monclient: get_auth_request con 0x7fe3680397b0 auth_method 0 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe386ffd640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388121930 0x7fe388068780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe378034000 tx=0x7fe378032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe37802acd0 con 0x7fe388121930 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe37f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe37f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe37f7fe640 10 monclient: dump: 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.776 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.777 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.750+0000 7fe37e7fc640 1 --2- 172.21.15.178:0/85489668 >> v2:172.21.15.204:6812/28210 conn(0x7fe3680397b0 0x7fe36803bc60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe3700095c0 tx=0x7fe3700090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.751+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fe378031e80 con 0x7fe388121930 2024-09-15T01:34:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.751+0000 7fe37f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.751+0000 7fe37f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.751+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388121930 msgr2=0x7fe388068780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.751+0000 7fe37f7fe640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388121930 0x7fe388068780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe378034000 tx=0x7fe378032000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.751+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388121930 msgr2=0x7fe388068780 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.751+0000 7fe37f7fe640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388121930 0x7fe388068780 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe378034000 tx=0x7fe378032000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.756+0000 7fe38d48b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.756+0000 7fe38d48b640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388133c90 0x7fe388068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.756+0000 7fe38d48b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.756+0000 7fe38d48b640 1 -- 172.21.15.178:0/85489668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe388070e70 con 0x7fe388133c90 2024-09-15T01:34:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.756+0000 7fe386ffd640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388133c90 0x7fe388068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.756+0000 7fe386ffd640 10 monclient: get_auth_request con 0x7fe388133c90 auth_method 0 2024-09-15T01:34:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.756+0000 7fe386ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.756+0000 7fe386ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.756+0000 7fe386ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe386ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe386ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe386ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe386ffd640 10 monclient(hunting): handle_auth_done global_id 5734 payload 824 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe386ffd640 1 -- 172.21.15.178:0/85489668 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe38804bc80 con 0x7fe388133c90 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe386ffd640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388133c90 0x7fe388068cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe378009130 tx=0x7fe37803c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe378031c80 con 0x7fe388133c90 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe37f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe37f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe37f7fe640 10 monclient: dump: 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9450 (secure 0 0 0) 0x7fe378045bc0 con 0x7fe388133c90 2024-09-15T01:34:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe37f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe37f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388133c90 msgr2=0x7fe388068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe37f7fe640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388133c90 0x7fe388068cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe378009130 tx=0x7fe37803c000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe37f7fe640 1 -- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388133c90 msgr2=0x7fe388068cc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.757+0000 7fe37f7fe640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe388133c90 0x7fe388068cc0 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe378009130 tx=0x7fe37803c000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.783 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:42.783 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:42.783 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:42.783 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 44, 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:42.784 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:42.785 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:42.786 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:42.787 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:42.788 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.789 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.790 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:42.791 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.793 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.794 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.795 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:42.797 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.798 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:42.800 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:42.806 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:42.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.758+0000 7fe38d48b640 1 -- 172.21.15.178:0/85489668 >> v2:172.21.15.204:6812/28210 conn(0x7fe3680397b0 msgr2=0x7fe36803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.758+0000 7fe38d48b640 1 --2- 172.21.15.178:0/85489668 >> v2:172.21.15.204:6812/28210 conn(0x7fe3680397b0 0x7fe36803bc60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe3700095c0 tx=0x7fe3700090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.758+0000 7fe38d48b640 10 monclient: shutdown 2024-09-15T01:34:42.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.758+0000 7fe38d48b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.758+0000 7fe38d48b640 1 -- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe38809f180 msgr2=0x7fe388068240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.758+0000 7fe38d48b640 1 --2- 172.21.15.178:0/85489668 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe38809f180 0x7fe388068240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe380001460 tx=0x7fe380001a40 comp rx=0 tx=0).stop 2024-09-15T01:34:42.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.758+0000 7fe3877fe640 1 -- 172.21.15.178:0/85489668 reap_dead start 2024-09-15T01:34:42.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.758+0000 7fe3877fe640 1 -- 172.21.15.178:0/85489668 reap_dead start 2024-09-15T01:34:42.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.758+0000 7fe38d48b640 1 -- 172.21.15.178:0/85489668 shutdown_connections 2024-09-15T01:34:42.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.758+0000 7fe38d48b640 1 -- 172.21.15.178:0/85489668 >> 172.21.15.178:0/85489668 conn(0x7fe3880a4390 msgr2=0x7fe3880a23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.758+0000 7fe38d48b640 1 -- 172.21.15.178:0/85489668 shutdown_connections 2024-09-15T01:34:42.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.758+0000 7fe38d48b640 1 -- 172.21.15.178:0/85489668 wait complete. 2024-09-15T01:34:42.809 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T01:34:42.809 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon g 2024-09-15T01:34:42.810 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:34:42.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.783+0000 7fca6c260640 10 monclient: get_monmap_and_config 2024-09-15T01:34:42.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.783+0000 7fca6c260640 10 monclient: build_initial_monmap 2024-09-15T01:34:42.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.783+0000 7fca6c260640 10 monclient: monmap: 2024-09-15T01:34:42.817 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:42.817 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:42.783984+0000 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:42.783984+0000 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:42.818 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 1 Processor -- start 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 1 -- start start 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 10 monclient: init 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 10 monclient: _add_conns ranks=[1,3,7,5,2,8,4,10,9,0,6] 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fca640a2400 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca6409f990 0x7fca6409fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 10 monclient(hunting): picked mon.noname-g con 0x7fca6409f990 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fca64137c60 0x7fca6413a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 10 monclient(hunting): picked mon.noname-i con 0x7fca64137c60 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca640648a0 con 0x7fca640a2400 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6409e1a0 con 0x7fca6409f990 2024-09-15T01:34:42.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fca640a11f0 con 0x7fca64137c60 2024-09-15T01:34:42.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:42.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6c260640 10 monclient(hunting): authenticate will time out at 898.426928s 2024-09-15T01:34:42.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.784+0000 7fca6a7fd640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fca64137c60 msgr2=0x7fca6413a040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:42.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca6a7fd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fca64137c60 0x7fca6413a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:42.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca6409f990 0x7fca6409fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca69ffc640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fca640a2400 0x7fca640a0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:50678/0 (socket says 172.21.15.204:50678) 2024-09-15T01:34:42.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca69ffc640 1 -- 172.21.15.204:0/1623381033 learned_addr learned my addr 172.21.15.204:0/1623381033 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:34:42.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 10 monclient(hunting): get_auth_request con 0x7fca6409f990 auth_method 0 2024-09-15T01:34:42.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca68ffa640 1 -- 172.21.15.204:0/1623381033 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273714995 0 0) 0x7fca640648a0 con 0x7fca640a2400 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 10 monclient(hunting): handle_auth_done global_id 14367 payload 274 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 1 -- 172.21.15.204:0/1623381033 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fca64137c60 msgr2=0x7fca6413a040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 1 --2- 172.21.15.204:0/1623381033 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fca64137c60 0x7fca6413a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 1 -- 172.21.15.204:0/1623381033 >> v1:172.21.15.204:6789/0 conn(0x7fca640a2400 legacy=0x7fca640a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 10 monclient: _finish_hunting 0 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 1 monclient: found mon.noname-g 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:42.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 1 -- 172.21.15.204:0/1623381033 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca640a27d0 con 0x7fca6409f990 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca68ffa640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.785+0000 7fca697fb640 1 --2- 172.21.15.204:0/1623381033 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca6409f990 0x7fca6409fd60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fca54009880 tx=0x7fca540090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=8d7e7d28a0a0512 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca68ffa640 1 -- 172.21.15.204:0/1623381033 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca5402ac80 con 0x7fca6409f990 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca68ffa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca68ffa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca68ffa640 10 monclient: dump: 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.822 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca68ffa640 10 monclient: _finish_auth 0 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca68ffa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 5 monclient: authenticate success, global_id 14367 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca68ffa640 1 -- 172.21.15.204:0/1623381033 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5402ae00 con 0x7fca6409f990 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca68ffa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:42.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 4 set_mon_vals no callback set 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca68ffa640 1 -- 172.21.15.204:0/1623381033 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca54031ca0 con 0x7fca6409f990 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca68ffa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca68ffa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca68ffa640 10 monclient: dump: 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.824 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 10 monclient: shutdown 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 1 -- 172.21.15.204:0/1623381033 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca6409f990 msgr2=0x7fca6409fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 1 --2- 172.21.15.204:0/1623381033 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca6409f990 0x7fca6409fd60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fca54009880 tx=0x7fca540090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 1 -- 172.21.15.204:0/1623381033 shutdown_connections 2024-09-15T01:34:42.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 1 --2- 172.21.15.204:0/1623381033 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fca64137c60 0x7fca6413a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 1 --2- 172.21.15.204:0/1623381033 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca6409f990 0x7fca6409fd60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 1 -- 172.21.15.204:0/1623381033 >> 172.21.15.204:0/1623381033 conn(0x7fca6406e860 msgr2=0x7fca6406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 1 -- 172.21.15.204:0/1623381033 shutdown_connections 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.786+0000 7fca6c260640 1 -- 172.21.15.204:0/1623381033 wait complete. 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.787+0000 7fca6c260640 10 monclient: build_initial_monmap 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.787+0000 7fca6c260640 10 monclient: monmap: 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:42.826 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.787+0000 7fca6c260640 1 Processor -- start 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.787+0000 7fca6c260640 1 -- start start 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.787+0000 7fca6c260640 10 monclient: init 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 10 monclient: _add_conns ranks=[5,8,2,1,4,3,7,6,0] 2024-09-15T01:34:42.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fca6409f990 0x7fca6411f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 10 monclient(hunting): picked mon.noname-h con 0x7fca6409f990 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fca640a2400 0x7fca6411f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 10 monclient(hunting): picked mon.noname-e con 0x7fca640a2400 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca64137c60 0x7fca6411fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 10 monclient(hunting): picked mon.noname-b con 0x7fca64137c60 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca64122ab0 con 0x7fca64137c60 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca64121870 con 0x7fca6409f990 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fca6409d3d0 con 0x7fca640a2400 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 10 monclient(hunting): authenticate will time out at 898.430192s 2024-09-15T01:34:42.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fca6409f990 0x7fca6411f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fca6409f990 0x7fca6411f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:37676/0 (socket says 172.21.15.204:37676) 2024-09-15T01:34:42.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 1 -- 172.21.15.204:0/2568462634 learned_addr learned my addr 172.21.15.204:0/2568462634 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:42.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 10 monclient(hunting): get_auth_request con 0x7fca6409f990 auth_method 0 2024-09-15T01:34:42.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6a7fd640 1 -- 172.21.15.204:0/2568462634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca64137c60 msgr2=0x7fca6411fc70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:42.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6a7fd640 1 --2- 172.21.15.204:0/2568462634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca64137c60 0x7fca6411fc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:42.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca697fb640 1 --2- 172.21.15.204:0/2568462634 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fca640a2400 0x7fca6411f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 10 monclient(hunting): handle_auth_done global_id 5225 payload 824 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 1 -- 172.21.15.204:0/2568462634 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fca640a2400 msgr2=0x7fca6411f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 1 --2- 172.21.15.204:0/2568462634 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fca640a2400 0x7fca6411f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 1 -- 172.21.15.204:0/2568462634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca64137c60 msgr2=0x7fca6411fc70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 1 --2- 172.21.15.204:0/2568462634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca64137c60 0x7fca6411fc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 10 monclient: _finish_hunting 0 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 1 monclient: found mon.noname-h 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 1 -- 172.21.15.204:0/2568462634 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca54009390 con 0x7fca6409f990 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca697fb640 10 monclient: get_auth_request con 0x7fca640a2400 auth_method 0 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca697fb640 1 --2- 172.21.15.204:0/2568462634 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fca640a2400 0x7fca6411f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca69ffc640 1 --2- 172.21.15.204:0/2568462634 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fca6409f990 0x7fca6411f1f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca6000c610 tx=0x7fca60007d00 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca59ffb640 1 -- 172.21.15.204:0/2568462634 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca600070f0 con 0x7fca6409f990 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca59ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca59ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca59ffb640 10 monclient: dump: 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.831 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca59ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca59ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 5 monclient: authenticate success, global_id 5225 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca59ffb640 1 -- 172.21.15.204:0/2568462634 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca60007290 con 0x7fca6409f990 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 10 monclient: _renew_subs 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca6c260640 1 -- 172.21.15.204:0/2568462634 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6409cad0 con 0x7fca6409f990 2024-09-15T01:34:42.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca59ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca59ffb640 1 -- 172.21.15.204:0/2568462634 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca60012400 con 0x7fca6409f990 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca59ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca59ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca59ffb640 10 monclient: dump: 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.833 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.788+0000 7fca5bfff640 4 set_mon_vals no callback set 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.789+0000 7fca6c260640 10 monclient: _renew_subs 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.789+0000 7fca6c260640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.789+0000 7fca6c260640 1 -- 172.21.15.204:0/2568462634 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca64120ca0 con 0x7fca6409f990 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.789+0000 7fca59ffb640 1 -- 172.21.15.204:0/2568462634 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fca60007400 con 0x7fca6409f990 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.789+0000 7fca59ffb640 1 --2- 172.21.15.204:0/2568462634 >> v2:172.21.15.204:6812/28210 conn(0x7fca480397e0 0x7fca4803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.789+0000 7fca59ffb640 1 -- 172.21.15.204:0/2568462634 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fca6004af90 con 0x7fca6409f990 2024-09-15T01:34:42.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.789+0000 7fca6c260640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.789+0000 7fca6c260640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.789+0000 7fca6c260640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:42.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.789+0000 7fca697fb640 1 --2- 172.21.15.204:0/2568462634 >> v2:172.21.15.204:6812/28210 conn(0x7fca480397e0 0x7fca4803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.789+0000 7fca6c260640 1 -- 172.21.15.204:0/2568462634 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca64122cb0 con 0x7fca6409f990 2024-09-15T01:34:42.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.789+0000 7fca697fb640 10 monclient: get_auth_request con 0x7fca480397e0 auth_method 0 2024-09-15T01:34:42.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.790+0000 7fca697fb640 1 --2- 172.21.15.204:0/2568462634 >> v2:172.21.15.204:6812/28210 conn(0x7fca480397e0 0x7fca4803bc90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fca54034000 tx=0x7fca54032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.794+0000 7fca59ffb640 1 -- 172.21.15.204:0/2568462634 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fca60012580 con 0x7fca6409f990 2024-09-15T01:34:42.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.794+0000 7fca59ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.794+0000 7fca59ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.903+0000 7fd4cf2dd640 10 monclient: get_monmap_and_config 2024-09-15T01:34:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.903+0000 7fd4cf2dd640 10 monclient: build_initial_monmap 2024-09-15T01:34:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.903+0000 7fd4cf2dd640 10 monclient: monmap: 2024-09-15T01:34:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:42.905018+0000 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:42.905018+0000 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:42.908 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.903+0000 7fd4cf2dd640 1 Processor -- start 2024-09-15T01:34:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.903+0000 7fd4cf2dd640 1 -- start start 2024-09-15T01:34:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.903+0000 7fd4cf2dd640 10 monclient: init 2024-09-15T01:34:42.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:42.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 10 monclient: _add_conns ranks=[1,9,4,8,10,7,3,5,0,2,6] 2024-09-15T01:34:42.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd4c80678d0 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:42.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd4c806ea30 0x7fd4c809ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 10 monclient(hunting): picked mon.noname-a con 0x7fd4c806ea30 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:42.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd4c81359e0 0x7fd4c8137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 10 monclient(hunting): picked mon.noname-c con 0x7fd4c81359e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:42.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c804bf60 con 0x7fd4c80678d0 2024-09-15T01:34:42.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c806d5b0 con 0x7fd4c806ea30 2024-09-15T01:34:42.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c8067ca0 con 0x7fd4c81359e0 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cf2dd640 10 monclient(hunting): authenticate will time out at 887.655402s 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd4c806ea30 0x7fd4c809ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cd87a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd4c81359e0 0x7fd4c8137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd4c806ea30 0x7fd4c809ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44794/0 (socket says 172.21.15.178:44794) 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cd87a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd4c81359e0 0x7fd4c8137dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44886/0 (socket says 172.21.15.178:44886) 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 1 -- 172.21.15.178:0/4075213219 learned_addr learned my addr 172.21.15.178:0/4075213219 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 10 monclient(hunting): get_auth_request con 0x7fd4c806ea30 auth_method 0 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cd87a640 10 monclient(hunting): get_auth_request con 0x7fd4c81359e0 auth_method 0 2024-09-15T01:34:42.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cd87a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cd87a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cd87a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cd87a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cd87a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cd87a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 10 monclient(hunting): handle_auth_done global_id 5481 payload 274 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 1 -- 172.21.15.178:0/4075213219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd4c81359e0 msgr2=0x7fd4c8137dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 1 --2- 172.21.15.178:0/4075213219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd4c81359e0 0x7fd4c8137dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 1 -- 172.21.15.178:0/4075213219 >> v1:172.21.15.204:6789/0 conn(0x7fd4c80678d0 legacy=0x7fd4c809e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 10 monclient: _finish_hunting 0 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 1 monclient: found mon.noname-a 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cc878640 1 -- 172.21.15.178:0/4075213219 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c806d890 con 0x7fd4c806ea30 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.904+0000 7fd4cd87a640 1 --2- 172.21.15.178:0/4075213219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd4c81359e0 0x7fd4c8137dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4cc878640 1 --2- 172.21.15.178:0/4075213219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4c806ea30 0x7fd4c809ee10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009910 tx=0x7fd4bc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691d84a47d6a391e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4bbfff640 1 -- 172.21.15.178:0/4075213219 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd4bc02adb0 con 0x7fd4c806ea30 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4bbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4bbfff640 10 monclient: dump: 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.920 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4bbfff640 10 monclient: _finish_auth 0 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4bbfff640 1 -- 172.21.15.178:0/4075213219 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc031820 con 0x7fd4c806ea30 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4cf2dd640 5 monclient: authenticate success, global_id 5481 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4cf2dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4bbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4cf2dd640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4bbfff640 1 -- 172.21.15.178:0/4075213219 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd4bc031e90 con 0x7fd4c806ea30 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4cf2dd640 4 set_mon_vals no callback set 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4bbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.905+0000 7fd4bbfff640 10 monclient: dump: 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.923 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.906+0000 7fd4cf2dd640 10 monclient: shutdown 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.906+0000 7fd4cf2dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.906+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/4075213219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4c806ea30 msgr2=0x7fd4c809ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.906+0000 7fd4cf2dd640 1 --2- 172.21.15.178:0/4075213219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4c806ea30 0x7fd4c809ee10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009910 tx=0x7fd4bc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.906+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/4075213219 shutdown_connections 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.906+0000 7fd4cf2dd640 1 --2- 172.21.15.178:0/4075213219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd4c81359e0 0x7fd4c8137dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.906+0000 7fd4cf2dd640 1 --2- 172.21.15.178:0/4075213219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4c806ea30 0x7fd4c809ee10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.906+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/4075213219 >> 172.21.15.178:0/4075213219 conn(0x7fd4c80a4390 msgr2=0x7fd4c8067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.906+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/4075213219 shutdown_connections 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.906+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/4075213219 wait complete. 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.906+0000 7fd4cf2dd640 10 monclient: build_initial_monmap 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.906+0000 7fd4cf2dd640 10 monclient: monmap: 2024-09-15T01:34:42.924 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:42.925 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 1 Processor -- start 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 1 -- start start 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 10 monclient: init 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 10 monclient: _add_conns ranks=[7,1,2,4,0,6,3,5,8] 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd4c80678d0 0x7fd4c811bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 10 monclient(hunting): picked mon.noname-i con 0x7fd4c80678d0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4c806ea30 0x7fd4c811c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 10 monclient(hunting): picked mon.noname-f con 0x7fd4c806ea30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4c81359e0 0x7fd4c811c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 10 monclient(hunting): picked mon.noname-b con 0x7fd4c81359e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c8133860 con 0x7fd4c806ea30 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c806ccd0 con 0x7fd4c81359e0 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c806c900 con 0x7fd4c80678d0 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cf2dd640 10 monclient(hunting): authenticate will time out at 887.658870s 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cd87a640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4c81359e0 msgr2=0x7fd4c811c870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cd87a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4c81359e0 0x7fd4c811c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.907+0000 7fd4cd079640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd4c80678d0 msgr2=0x7fd4c811bdf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cd079640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd4c80678d0 0x7fd4c811bdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4c806ea30 0x7fd4c811c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4c806ea30 0x7fd4c811c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:47346/0 (socket says 172.21.15.178:47346) 2024-09-15T01:34:42.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 1 -- 172.21.15.178:0/95122711 learned_addr learned my addr 172.21.15.178:0/95122711 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:42.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 10 monclient(hunting): get_auth_request con 0x7fd4c806ea30 auth_method 0 2024-09-15T01:34:42.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 10 monclient(hunting): handle_auth_done global_id 5761 payload 824 2024-09-15T01:34:42.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 1 -- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd4c80678d0 msgr2=0x7fd4c811bdf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:42.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd4c80678d0 0x7fd4c811bdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 1 -- 172.21.15.178:0/95122711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4c81359e0 msgr2=0x7fd4c811c870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4c81359e0 0x7fd4c811c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 10 monclient: _finish_hunting 0 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 1 monclient: found mon.noname-f 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 1 -- 172.21.15.178:0/95122711 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c4009390 con 0x7fd4c806ea30 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cc878640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4c806ea30 0x7fd4c811c330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097e0 tx=0x7fd4bc032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd4bc033d30 con 0x7fd4c806ea30 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4b97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4b97fa640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4b97fa640 10 monclient: dump: 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.929 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4b97fa640 10 monclient: _finish_auth 0 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4b97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cf2dd640 5 monclient: authenticate success, global_id 5761 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc03a390 con 0x7fd4c806ea30 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cf2dd640 10 monclient: _renew_subs 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cf2dd640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.908+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/95122711 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4bc009450 con 0x7fd4c806ea30 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4b97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd4bc03a980 con 0x7fd4c806ea30 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4b97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4b97fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4bb7fe640 4 set_mon_vals no callback set 2024-09-15T01:34:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4b97fa640 10 monclient: dump: 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4cf2dd640 10 monclient: _renew_subs 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4cf2dd640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/95122711 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c81342c0 con 0x7fd4c806ea30 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd4bc039040 con 0x7fd4c806ea30 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4cf2dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4cf2dd640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c811cdb0 0x7fd4c8098a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4cf2dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.909+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/95122711 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c804c4a0 con 0x7fd4c811cdb0 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4b97fa640 1 --2- 172.21.15.178:0/95122711 >> v2:172.21.15.204:6812/28210 conn(0x7fd4ac0397b0 0x7fd4ac03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd4bc06b100 con 0x7fd4c806ea30 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd079640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c811cdb0 0x7fd4c8098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd079640 10 monclient: get_auth_request con 0x7fd4c811cdb0 auth_method 0 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd079640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd079640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd079640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd079640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd079640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd079640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd87a640 1 --2- 172.21.15.178:0/95122711 >> v2:172.21.15.204:6812/28210 conn(0x7fd4ac0397b0 0x7fd4ac03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd87a640 10 monclient: get_auth_request con 0x7fd4ac0397b0 auth_method 0 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd079640 10 monclient(hunting): handle_auth_done global_id 14376 payload 824 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd079640 1 -- 172.21.15.178:0/95122711 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4c805ed70 con 0x7fd4c811cdb0 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd079640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c811cdb0 0x7fd4c8098a40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00013d0 tx=0x7fd4c0001670 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4cd87a640 1 --2- 172.21.15.178:0/95122711 >> v2:172.21.15.204:6812/28210 conn(0x7fd4ac0397b0 0x7fd4ac03bc60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd4c811d2e0 tx=0x7fd4c40090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd4c0005d10 con 0x7fd4c811cdb0 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4b97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4b97fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.910+0000 7fd4b97fa640 10 monclient: dump: 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.936 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.911+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fd4c0005e70 con 0x7fd4c811cdb0 2024-09-15T01:34:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.911+0000 7fd4b97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.911+0000 7fd4b97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.911+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c811cdb0 msgr2=0x7fd4c8098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.911+0000 7fd4b97fa640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c811cdb0 0x7fd4c8098a40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00013d0 tx=0x7fd4c0001670 comp rx=0 tx=0).stop 2024-09-15T01:34:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.911+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c811cdb0 msgr2=0x7fd4c8098a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.911+0000 7fd4b97fa640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c811cdb0 0x7fd4c8098a40 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00013d0 tx=0x7fd4c0001670 comp rx=0 tx=0).stop 2024-09-15T01:34:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.916+0000 7fd4cf2dd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.916+0000 7fd4cf2dd640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c8098f80 0x7fd4c8099380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.916+0000 7fd4cf2dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.916+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/95122711 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c804bc80 con 0x7fd4c8098f80 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.916+0000 7fd4cd079640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c8098f80 0x7fd4c8099380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.916+0000 7fd4cd079640 10 monclient: get_auth_request con 0x7fd4c8098f80 auth_method 0 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.916+0000 7fd4cd079640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.916+0000 7fd4cd079640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.916+0000 7fd4cd079640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4cd079640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4cd079640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4cd079640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4cd079640 10 monclient(hunting): handle_auth_done global_id 14385 payload 824 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4cd079640 1 -- 172.21.15.178:0/95122711 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd4c811f670 con 0x7fd4c8098f80 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4cd079640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c8098f80 0x7fd4c8099380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00013d0 tx=0x7fd4c0015000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd4c000ac60 con 0x7fd4c8098f80 2024-09-15T01:34:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4b97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4b97fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4b97fa640 10 monclient: dump: 2024-09-15T01:34:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9108 (secure 0 0 0) 0x7fd4c000adc0 con 0x7fd4c8098f80 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4b97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4b97fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c8098f80 msgr2=0x7fd4c8099380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4b97fa640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c8098f80 0x7fd4c8099380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00013d0 tx=0x7fd4c0015000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4b97fa640 1 -- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c8098f80 msgr2=0x7fd4c8099380 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.917+0000 7fd4b97fa640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4c8098f80 0x7fd4c8099380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.942 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:42.942 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:42.942 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:42.942 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:42.942 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 44, 2024-09-15T01:34:42.942 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:42.942 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:42.942 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:42.942 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:42.942 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.943 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.944 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.944 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.944 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:42.944 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:42.944 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.944 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.944 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.944 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.944 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.944 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.944 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.944 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:42.945 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:42.946 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.948 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:42.949 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.950 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.951 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.952 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:42.953 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.954 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.955 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.956 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:42.963 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:42.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.953+0000 7fca6c260640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:42.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.953+0000 7fca6c260640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:42.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.953+0000 7fca6c260640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:42.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.953+0000 7fca6c260640 1 -- 172.21.15.204:0/2568462634 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7fca64070b80 con 0x7fca6409f990 2024-09-15T01:34:42.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.953+0000 7fca59ffb640 1 -- 172.21.15.204:0/2568462634 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v81) v1 ==== 56+0+1273 (secure 0 0 0) 0x7fca60029510 con 0x7fca6409f990 2024-09-15T01:34:42.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.953+0000 7fca59ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:42.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:42.953+0000 7fca59ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:42.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.918+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/95122711 >> v2:172.21.15.204:6812/28210 conn(0x7fd4ac0397b0 msgr2=0x7fd4ac03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.918+0000 7fd4cf2dd640 1 --2- 172.21.15.178:0/95122711 >> v2:172.21.15.204:6812/28210 conn(0x7fd4ac0397b0 0x7fd4ac03bc60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd4c811d2e0 tx=0x7fd4c40090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:42.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.918+0000 7fd4cf2dd640 10 monclient: shutdown 2024-09-15T01:34:42.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.918+0000 7fd4cf2dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:42.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.918+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4c806ea30 msgr2=0x7fd4c811c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:42.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.918+0000 7fd4cf2dd640 1 --2- 172.21.15.178:0/95122711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4c806ea30 0x7fd4c811c330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097e0 tx=0x7fd4bc032000 comp rx=0 tx=0).stop 2024-09-15T01:34:42.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.918+0000 7fd4cd87a640 1 -- 172.21.15.178:0/95122711 reap_dead start 2024-09-15T01:34:42.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.918+0000 7fd4cd87a640 1 -- 172.21.15.178:0/95122711 reap_dead start 2024-09-15T01:34:42.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.918+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/95122711 shutdown_connections 2024-09-15T01:34:42.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.918+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/95122711 >> 172.21.15.178:0/95122711 conn(0x7fd4c80a4390 msgr2=0x7fd4c809d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:42.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.918+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/95122711 shutdown_connections 2024-09-15T01:34:42.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:42.918+0000 7fd4cf2dd640 1 -- 172.21.15.178:0/95122711 wait complete. 2024-09-15T01:34:42.967 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T01:34:42.967 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-09-15T01:34:42.967 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:34:43.049 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.047+0000 7fca6c260640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:43.049 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.047+0000 7fca6c260640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:43.049 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.047+0000 7fca6c260640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:43.049 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.047+0000 7fca6c260640 1 -- 172.21.15.204:0/2568462634 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7fca6409dec0 con 0x7fca6409f990 2024-09-15T01:34:43.049 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.047+0000 7fca59ffb640 1 -- 172.21.15.204:0/2568462634 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v81) v1 ==== 56+0+1273 (secure 0 0 0) 0x7fca60029030 con 0x7fca6409f990 2024-09-15T01:34:43.049 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.047+0000 7fca59ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:43.049 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.047+0000 7fca59ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:43.051 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 1 -- 172.21.15.204:0/2568462634 >> v2:172.21.15.204:6812/28210 conn(0x7fca480397e0 msgr2=0x7fca4803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.051 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 1 --2- 172.21.15.204:0/2568462634 >> v2:172.21.15.204:6812/28210 conn(0x7fca480397e0 0x7fca4803bc90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fca54034000 tx=0x7fca54032000 comp rx=0 tx=0).stop 2024-09-15T01:34:43.051 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 10 monclient: shutdown 2024-09-15T01:34:43.051 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:43.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 1 -- 172.21.15.204:0/2568462634 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fca6409f990 msgr2=0x7fca6411f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 1 --2- 172.21.15.204:0/2568462634 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fca6409f990 0x7fca6411f1f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca6000c610 tx=0x7fca60007d00 comp rx=0 tx=0).stop 2024-09-15T01:34:43.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 1 -- 172.21.15.204:0/2568462634 shutdown_connections 2024-09-15T01:34:43.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 1 --2- 172.21.15.204:0/2568462634 >> v2:172.21.15.204:6812/28210 conn(0x7fca480397e0 0x7fca4803bc90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 1 --2- 172.21.15.204:0/2568462634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca64137c60 0x7fca6411fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 1 --2- 172.21.15.204:0/2568462634 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fca640a2400 0x7fca6411f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 1 --2- 172.21.15.204:0/2568462634 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fca6409f990 0x7fca6411f1f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 1 -- 172.21.15.204:0/2568462634 >> 172.21.15.204:0/2568462634 conn(0x7fca6406e860 msgr2=0x7fca6409c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:43.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.049+0000 7fca6c260640 1 -- 172.21.15.204:0/2568462634 shutdown_connections 2024-09-15T01:34:43.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.050+0000 7fca6c260640 1 -- 172.21.15.204:0/2568462634 wait complete. 2024-09-15T01:34:43.060 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd tree-from host1 2024-09-15T01:34:43.060 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:34:43.060 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree-from host1 2024-09-15T01:34:43.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.057+0000 7f28bf06f640 10 monclient: get_monmap_and_config 2024-09-15T01:34:43.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.057+0000 7f28bf06f640 10 monclient: build_initial_monmap 2024-09-15T01:34:43.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.057+0000 7f28bf06f640 10 monclient: monmap: 2024-09-15T01:34:43.071 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:43.071 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.071 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:34:43.059317+0000 2024-09-15T01:34:43.071 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:34:43.059317+0000 2024-09-15T01:34:43.071 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:43.071 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:43.071 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:43.071 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 1 Processor -- start 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 1 -- start start 2024-09-15T01:34:43.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 10 monclient: init 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 10 monclient: _add_conns ranks=[8,0,3,2,10,6,5,7,1,4,9] 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 0x7f28b8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 10 monclient(hunting): picked mon.noname-e con 0x7f28b80a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f28b809faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28b809fe70 0x7f28b80a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 10 monclient(hunting): picked mon.noname-g con 0x7f28b809fe70 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b804bf60 con 0x7f28b809faa0 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b809dd80 con 0x7f28b809fe70 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f28b80a2d00 con 0x7f28b80a2930 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:43.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bf06f640 10 monclient(hunting): authenticate will time out at 887.809733s 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bce0b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 0x7f28b8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28b7fff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f28b809faa0 0x7f28b80a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:51364/0 (socket says 172.21.15.178:51364) 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28b7fff640 1 -- 172.21.15.178:0/572165584 learned_addr learned my addr 172.21.15.178:0/572165584 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bce0b640 10 monclient(hunting): get_auth_request con 0x7f28b80a2930 auth_method 0 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bce0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bce0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bce0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bd60c640 1 --2- 172.21.15.178:0/572165584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28b809fe70 0x7f28b80a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bd60c640 10 monclient(hunting): get_auth_request con 0x7f28b809fe70 auth_method 0 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bd60c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bd60c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bd60c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bce0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.058+0000 7f28bce0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bce0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bd60c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bd60c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bd60c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28b77fe640 1 -- 172.21.15.178:0/572165584 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388751207 0 0) 0x7f28b804bf60 con 0x7f28b809faa0 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28b77fe640 10 monclient(hunting): my global_id is 5490 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28b77fe640 1 -- 172.21.15.178:0/572165584 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28a80034e0 con 0x7f28b809faa0 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bce0b640 10 monclient(hunting): handle_auth_done global_id 5336 payload 274 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bce0b640 1 -- 172.21.15.178:0/572165584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28b809fe70 msgr2=0x7f28b80a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bce0b640 1 --2- 172.21.15.178:0/572165584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28b809fe70 0x7f28b80a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bce0b640 1 -- 172.21.15.178:0/572165584 >> v1:172.21.15.178:6789/0 conn(0x7f28b809faa0 legacy=0x7f28b80a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bce0b640 10 monclient: _finish_hunting 0 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bce0b640 1 monclient: found mon.noname-e 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bce0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bce0b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bce0b640 1 -- 172.21.15.178:0/572165584 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b80a0310 con 0x7f28b80a2930 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28bce0b640 1 --2- 172.21.15.178:0/572165584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 0x7f28b8132de0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009880 tx=0x7f28ac0090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6162dcd3bd95bbc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.059+0000 7f28b77fe640 1 -- 172.21.15.178:0/572165584 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28ac02ac50 con 0x7f28b80a2930 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28b77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28b77fe640 10 monclient: dump: 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.076 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.077 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28b77fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28b77fe640 1 -- 172.21.15.178:0/572165584 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac02add0 con 0x7f28b80a2930 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 5 monclient: authenticate success, global_id 5336 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28b77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28b77fe640 1 -- 172.21.15.178:0/572165584 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28ac031ca0 con 0x7f28b80a2930 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 4 set_mon_vals no callback set 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28b77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28b77fe640 10 monclient: dump: 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.080 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 10 monclient: shutdown 2024-09-15T01:34:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 1 -- 172.21.15.178:0/572165584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 msgr2=0x7f28b8132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 1 --2- 172.21.15.178:0/572165584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 0x7f28b8132de0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009880 tx=0x7f28ac0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 1 -- 172.21.15.178:0/572165584 shutdown_connections 2024-09-15T01:34:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 1 --2- 172.21.15.178:0/572165584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28b809fe70 0x7f28b80a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 1 --2- 172.21.15.178:0/572165584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 0x7f28b8132de0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 1 -- 172.21.15.178:0/572165584 >> 172.21.15.178:0/572165584 conn(0x7f28b80a5ed0 msgr2=0x7f28b80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 1 -- 172.21.15.178:0/572165584 shutdown_connections 2024-09-15T01:34:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.060+0000 7f28bf06f640 1 -- 172.21.15.178:0/572165584 wait complete. 2024-09-15T01:34:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.061+0000 7f28bf06f640 10 monclient: build_initial_monmap 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.061+0000 7f28bf06f640 10 monclient: monmap: 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:43.081 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.061+0000 7f28bf06f640 1 Processor -- start 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.061+0000 7f28bf06f640 1 -- start start 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.061+0000 7f28bf06f640 10 monclient: init 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient: _add_conns ranks=[5,7,8,4,3,6,2,1,0] 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b809faa0 0x7f28b809af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient(hunting): picked mon.noname-h con 0x7f28b809faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f28b809fe70 0x7f28b81316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient(hunting): picked mon.noname-i con 0x7f28b809fe70 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:43.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 0x7f28b811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient(hunting): picked mon.noname-e con 0x7f28b80a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28b811ee50 con 0x7f28b809faa0 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f28b80a2390 con 0x7f28b809fe70 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f28b809d630 con 0x7f28b80a2930 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient(hunting): authenticate will time out at 887.813153s 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b7fff640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f28b809fe70 msgr2=0x7f28b81316e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b7fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f28b809fe70 0x7f28b81316e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 0x7f28b811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 0x7f28b811c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:36878/0 (socket says 172.21.15.178:36878) 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 1 -- 172.21.15.178:0/3584585588 learned_addr learned my addr 172.21.15.178:0/3584585588 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 10 monclient(hunting): get_auth_request con 0x7f28b80a2930 auth_method 0 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bce0b640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b809faa0 0x7f28b809af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bce0b640 10 monclient(hunting): get_auth_request con 0x7f28b809faa0 auth_method 0 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bce0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bce0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bce0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bce0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bce0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bce0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 10 monclient(hunting): handle_auth_done global_id 5345 payload 824 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 1 -- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f28b809fe70 msgr2=0x7f28b81316e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f28b809fe70 0x7f28b81316e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 1 -- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b809faa0 msgr2=0x7f28b809af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b809faa0 0x7f28b809af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 1 monclient: found mon.noname-e 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 1 -- 172.21.15.178:0/3584585588 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac009390 con 0x7f28b80a2930 2024-09-15T01:34:43.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bd60c640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 0x7f28b811c530 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009ef0 tx=0x7f28b000b040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28b000ce10 con 0x7f28b80a2930 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b4ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b4ff9640 10 monclient: dump: 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.086 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b4ff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b000f040 con 0x7f28b80a2930 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 5 monclient: authenticate success, global_id 5345 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28b0014d80 con 0x7f28b80a2930 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient: _renew_subs 2024-09-15T01:34:43.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 1 -- 172.21.15.178:0/3584585588 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b809bee0 con 0x7f28b80a2930 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b6ffd640 4 set_mon_vals no callback set 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28b4ff9640 10 monclient: dump: 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.088 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient: _renew_subs 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.062+0000 7f28bf06f640 1 -- 172.21.15.178:0/3584585588 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b8142b60 con 0x7f28b80a2930 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.063+0000 7f28bf06f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.063+0000 7f28bf06f640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b8131c20 0x7f28b811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.063+0000 7f28bf06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.063+0000 7f28bf06f640 1 -- 172.21.15.178:0/3584585588 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28b804c4a0 con 0x7f28b8131c20 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.063+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f28b0014480 con 0x7f28b80a2930 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.063+0000 7f28bce0b640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b8131c20 0x7f28b811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.063+0000 7f28bce0b640 10 monclient: get_auth_request con 0x7f28b8131c20 auth_method 0 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.063+0000 7f28bce0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.063+0000 7f28bce0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.063+0000 7f28bce0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.063+0000 7f28bce0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.063+0000 7f28bce0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28bce0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 1 --2- 172.21.15.178:0/3584585588 >> v2:172.21.15.204:6812/28210 conn(0x7f289c0397b0 0x7f289c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f28b004c380 con 0x7f28b80a2930 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28bce0b640 10 monclient(hunting): handle_auth_done global_id 5243 payload 824 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28bce0b640 1 -- 172.21.15.178:0/3584585588 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28b805ed70 con 0x7f28b8131c20 2024-09-15T01:34:43.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b7fff640 1 --2- 172.21.15.178:0/3584585588 >> v2:172.21.15.204:6812/28210 conn(0x7f289c0397b0 0x7f289c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b7fff640 10 monclient: get_auth_request con 0x7f289c0397b0 auth_method 0 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28bce0b640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b8131c20 0x7f28b811ca70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f28b8067590 tx=0x7f28ac032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28ac03da80 con 0x7f28b8131c20 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 10 monclient: dump: 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.091 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b7fff640 1 --2- 172.21.15.178:0/3584585588 >> v2:172.21.15.204:6812/28210 conn(0x7f289c0397b0 0x7f289c03bc60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f28a4000fb0 tx=0x7f28a4007940 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f28ac03dbe0 con 0x7f28b8131c20 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b8131c20 msgr2=0x7f28b811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b8131c20 0x7f28b811ca70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f28b8067590 tx=0x7f28ac032000 comp rx=0 tx=0).stop 2024-09-15T01:34:43.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b8131c20 msgr2=0x7f28b811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:43.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.064+0000 7f28b4ff9640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b8131c20 0x7f28b811ca70 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f28b8067590 tx=0x7f28ac032000 comp rx=0 tx=0).stop 2024-09-15T01:34:43.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bf06f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:34:43.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bf06f640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b811cfb0 0x7f28b8096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bf06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bf06f640 1 -- 172.21.15.178:0/3584585588 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28b804bc80 con 0x7f28b811cfb0 2024-09-15T01:34:43.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bce0b640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b811cfb0 0x7f28b8096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bce0b640 10 monclient: get_auth_request con 0x7f28b811cfb0 auth_method 0 2024-09-15T01:34:43.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bce0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bce0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bce0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bce0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bce0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bce0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bce0b640 10 monclient(hunting): handle_auth_done global_id 5252 payload 824 2024-09-15T01:34:43.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bce0b640 1 -- 172.21.15.178:0/3584585588 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f28b811d3b0 con 0x7f28b811cfb0 2024-09-15T01:34:43.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.070+0000 7f28bce0b640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b811cfb0 0x7f28b8096a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009130 tx=0x7f28ac03e000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28ac04b070 con 0x7f28b811cfb0 2024-09-15T01:34:43.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28b4ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:43.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28b4ff9640 10 monclient: dump: 2024-09-15T01:34:43.096 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:34:43.096 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.096 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.096 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.096 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.096 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9278 (secure 0 0 0) 0x7f28ac04e3b0 con 0x7f28b811cfb0 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28b4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28b4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:43.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b811cfb0 msgr2=0x7f28b8096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28b4ff9640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b811cfb0 0x7f28b8096a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009130 tx=0x7f28ac03e000 comp rx=0 tx=0).stop 2024-09-15T01:34:43.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28b4ff9640 1 -- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b811cfb0 msgr2=0x7f28b8096a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:34:43.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28b4ff9640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f28b811cfb0 0x7f28b8096a30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.098 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:34:43.098 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:43.098 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:43.098 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:34:43.098 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 44, 2024-09-15T01:34:43.098 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:34:43.098 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:34:43.099 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:34:43.099 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:34:43.099 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:34:43.099 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:34:43.099 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:34:43.099 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:43.099 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:34:43.099 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:43.099 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:34:43.099 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:34:43.099 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:43.099 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:43.100 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:43.101 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:43.101 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:43.101 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:43.101 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:43.101 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:43.101 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:43.101 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:43.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.101 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:34:43.101 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:34:43.101 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:34:43.101 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:34:43.102 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:34:43.103 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:34:43.103 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:34:43.103 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:34:43.103 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:34:43.103 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:34:43.103 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:34:43.103 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:34:43.103 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:34:43.103 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:34:43.103 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:43.103 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:34:43.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.104 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.105 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:43.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.107 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:34:43.107 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:34:43.107 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:43.107 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:43.107 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:43.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.107 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:34:43.107 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:34:43.107 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:43.108 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:43.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.108 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:43.108 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:34:43.108 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.108 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:43.108 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:34:43.108 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.108 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.108 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.109 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:34:43.110 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.113 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.114 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:43.115 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:43.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:34:43.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:34:43.118 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:43.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 4 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:34:43.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:34:43.121 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:34:43.121 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:34:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28bf06f640 1 -- 172.21.15.178:0/3584585588 >> v2:172.21.15.204:6812/28210 conn(0x7f289c0397b0 msgr2=0x7f289c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28bf06f640 1 --2- 172.21.15.178:0/3584585588 >> v2:172.21.15.204:6812/28210 conn(0x7f289c0397b0 0x7f289c03bc60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f28a4000fb0 tx=0x7f28a4007940 comp rx=0 tx=0).stop 2024-09-15T01:34:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28bf06f640 10 monclient: shutdown 2024-09-15T01:34:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28bf06f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28bf06f640 1 -- 172.21.15.178:0/3584585588 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 msgr2=0x7f28b811c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28bf06f640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 0x7f28b811c530 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009ef0 tx=0x7f28b000b040 comp rx=0 tx=0).stop 2024-09-15T01:34:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.071+0000 7f28bd60c640 1 -- 172.21.15.178:0/3584585588 reap_dead start 2024-09-15T01:34:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.072+0000 7f28bf06f640 1 -- 172.21.15.178:0/3584585588 shutdown_connections 2024-09-15T01:34:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.072+0000 7f28bf06f640 1 --2- 172.21.15.178:0/3584585588 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f28b80a2930 0x7f28b811c530 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.072+0000 7f28bf06f640 1 -- 172.21.15.178:0/3584585588 >> 172.21.15.178:0/3584585588 conn(0x7f28b80a5ed0 msgr2=0x7f28b81359f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.072+0000 7f28bf06f640 1 -- 172.21.15.178:0/3584585588 shutdown_connections 2024-09-15T01:34:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:34:43.072+0000 7f28bf06f640 1 -- 172.21.15.178:0/3584585588 wait complete. 2024-09-15T01:34:43.123 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T01:34:43.123 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T01:34:43.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.133+0000 7f8efcf1d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:43.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.133+0000 7f8efcf1d640 10 monclient: build_initial_monmap 2024-09-15T01:34:43.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.133+0000 7f8efcf1d640 10 monclient: monmap: 2024-09-15T01:34:43.134 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:43.134 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.134 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:43.133689+0000 2024-09-15T01:34:43.134 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:43.133689+0000 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:43.135 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:43.136 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.133+0000 7f8efcf1d640 1 Processor -- start 2024-09-15T01:34:43.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.133+0000 7f8efcf1d640 1 -- start start 2024-09-15T01:34:43.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.133+0000 7f8efcf1d640 10 monclient: init 2024-09-15T01:34:43.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:43.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 10 monclient: _add_conns ranks=[6,5,2,7,9,1,10,4,3,0,8] 2024-09-15T01:34:43.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8ef80a4520 0x7f8ef8130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 10 monclient(hunting): picked mon.noname-d con 0x7f8ef80a4520 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8ef806ecf0 0x7f8ef8130f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 10 monclient(hunting): picked mon.noname-h con 0x7f8ef806ecf0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef809e750 0x7f8ef809eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 10 monclient(hunting): picked mon.noname-b con 0x7f8ef809e750 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef806de30 con 0x7f8ef809e750 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef806db50 con 0x7f8ef806ecf0 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80a48f0 con 0x7f8ef80a4520 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:43.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8efcf1d640 10 monclient(hunting): authenticate will time out at 898.776449s 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8ef659c640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8ef80a4520 msgr2=0x7f8ef8130a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8ef659c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8ef80a4520 0x7f8ef8130a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8ef6d9d640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef809e750 msgr2=0x7f8ef809eba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8ef806ecf0 0x7f8ef8130f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8ef6d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef809e750 0x7f8ef809eba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8ef806ecf0 0x7f8ef8130f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:37682/0 (socket says 172.21.15.204:37682) 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 1 -- 172.21.15.204:0/3841504731 learned_addr learned my addr 172.21.15.204:0/3841504731 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 10 monclient(hunting): get_auth_request con 0x7f8ef806ecf0 auth_method 0 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 10 monclient(hunting): handle_auth_done global_id 5261 payload 274 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 1 -- 172.21.15.204:0/3841504731 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8ef80a4520 msgr2=0x7f8ef8130a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 1 --2- 172.21.15.204:0/3841504731 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8ef80a4520 0x7f8ef8130a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 1 -- 172.21.15.204:0/3841504731 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef809e750 msgr2=0x7f8ef809eba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 1 --2- 172.21.15.204:0/3841504731 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef809e750 0x7f8ef809eba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 1 monclient: found mon.noname-h 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.134+0000 7f8eeffff640 1 -- 172.21.15.204:0/3841504731 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80648a0 con 0x7f8ef806ecf0 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8eeffff640 1 --2- 172.21.15.204:0/3841504731 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8ef806ecf0 0x7f8ef8130f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009940 tx=0x7f8ee00091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=68428c8d312a7af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8ef5d9b640 1 -- 172.21.15.204:0/3841504731 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ee002acb0 con 0x7f8ef806ecf0 2024-09-15T01:34:43.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8ef5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8ef5d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8ef5d9b640 10 monclient: dump: 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.148 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8ef5d9b640 10 monclient: _finish_auth 0 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8ef5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8efcf1d640 5 monclient: authenticate success, global_id 5261 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8efcf1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8ef5d9b640 1 -- 172.21.15.204:0/3841504731 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee002ae30 con 0x7f8ef806ecf0 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8ef5d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8efcf1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8efcf1d640 4 set_mon_vals no callback set 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8ef5d9b640 1 -- 172.21.15.204:0/3841504731 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ee0031ca0 con 0x7f8ef806ecf0 2024-09-15T01:34:43.149 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8ef5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8ef5d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.135+0000 7f8ef5d9b640 10 monclient: dump: 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.150 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 10 monclient: shutdown 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 1 -- 172.21.15.204:0/3841504731 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8ef806ecf0 msgr2=0x7f8ef8130f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 1 --2- 172.21.15.204:0/3841504731 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8ef806ecf0 0x7f8ef8130f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009940 tx=0x7f8ee00091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 1 -- 172.21.15.204:0/3841504731 shutdown_connections 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 1 --2- 172.21.15.204:0/3841504731 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ef809e750 0x7f8ef809eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 1 --2- 172.21.15.204:0/3841504731 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8ef806ecf0 0x7f8ef8130f80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 1 --2- 172.21.15.204:0/3841504731 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8ef80a4520 0x7f8ef8130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 1 -- 172.21.15.204:0/3841504731 >> 172.21.15.204:0/3841504731 conn(0x7f8ef80676d0 msgr2=0x7f8ef80a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:43.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 1 -- 172.21.15.204:0/3841504731 shutdown_connections 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 1 -- 172.21.15.204:0/3841504731 wait complete. 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 10 monclient: build_initial_monmap 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.136+0000 7f8efcf1d640 10 monclient: monmap: 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:43.152 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 1 Processor -- start 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 1 -- start start 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 10 monclient: init 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 10 monclient: _add_conns ranks=[5,0,3,6,8,7,2,1,4] 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8ef806ecf0 0x7f8ef80999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 10 monclient(hunting): picked mon.noname-h con 0x7f8ef806ecf0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:43.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef809e750 0x7f8ef812d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 10 monclient(hunting): picked mon.noname-a con 0x7f8ef809e750 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ef80a4520 0x7f8ef811a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 10 monclient(hunting): picked mon.noname-g con 0x7f8ef80a4520 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef806cc30 con 0x7f8ef809e750 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef8140b70 con 0x7f8ef80a4520 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef811c8d0 con 0x7f8ef806ecf0 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8efcf1d640 10 monclient(hunting): authenticate will time out at 898.779535s 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8ef806ecf0 0x7f8ef80999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef6d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ef80a4520 0x7f8ef811a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8ef806ecf0 0x7f8ef80999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:37690/0 (socket says 172.21.15.204:37690) 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 1 -- 172.21.15.204:0/4157865218 learned_addr learned my addr 172.21.15.204:0/4157865218 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 10 monclient(hunting): get_auth_request con 0x7f8ef806ecf0 auth_method 0 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef6d9d640 10 monclient(hunting): get_auth_request con 0x7f8ef80a4520 auth_method 0 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8eeffff640 1 --2- 172.21.15.204:0/4157865218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef809e750 0x7f8ef812d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 10 monclient(hunting): handle_auth_done global_id 5270 payload 824 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 1 -- 172.21.15.204:0/4157865218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ef80a4520 msgr2=0x7f8ef811a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 1 --2- 172.21.15.204:0/4157865218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ef80a4520 0x7f8ef811a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 1 -- 172.21.15.204:0/4157865218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef809e750 msgr2=0x7f8ef812d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 1 --2- 172.21.15.204:0/4157865218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef809e750 0x7f8ef812d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 1 monclient: found mon.noname-h 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:43.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8ef659c640 1 -- 172.21.15.204:0/4157865218 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee0009450 con 0x7f8ef806ecf0 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8eeffff640 10 monclient: get_auth_request con 0x7f8ef809e750 auth_method 0 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.137+0000 7f8eeffff640 1 --2- 172.21.15.204:0/4157865218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ef809e750 0x7f8ef812d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8ef659c640 1 --2- 172.21.15.204:0/4157865218 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8ef806ecf0 0x7f8ef80999e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800b600 tx=0x7f8ee80091a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8eee7fc640 1 -- 172.21.15.204:0/4157865218 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ee8015070 con 0x7f8ef806ecf0 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8eee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8eee7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8eee7fc640 10 monclient: dump: 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.157 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8eee7fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8eee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8eee7fc640 1 -- 172.21.15.204:0/4157865218 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee8006150 con 0x7f8ef806ecf0 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8efcf1d640 5 monclient: authenticate success, global_id 5270 2024-09-15T01:34:43.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8efcf1d640 10 monclient: _renew_subs 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8efcf1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8efcf1d640 1 -- 172.21.15.204:0/4157865218 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef811b590 con 0x7f8ef806ecf0 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8eee7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8eee7fc640 1 -- 172.21.15.204:0/4157865218 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ee8006740 con 0x7f8ef806ecf0 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8eee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8eee7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8eee7fc640 10 monclient: dump: 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.159 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8efcf1d640 10 monclient: _renew_subs 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8ef559a640 4 set_mon_vals no callback set 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8efcf1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.138+0000 7f8efcf1d640 1 -- 172.21.15.204:0/4157865218 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef811b9e0 con 0x7f8ef806ecf0 2024-09-15T01:34:43.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.139+0000 7f8eee7fc640 1 -- 172.21.15.204:0/4157865218 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8ee80062c0 con 0x7f8ef806ecf0 2024-09-15T01:34:43.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.139+0000 7f8efcf1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.139+0000 7f8efcf1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.139+0000 7f8efcf1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:43.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.139+0000 7f8efcf1d640 1 -- 172.21.15.204:0/4157865218 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef8070b80 con 0x7f8ef806ecf0 2024-09-15T01:34:43.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.139+0000 7f8eee7fc640 1 --2- 172.21.15.204:0/4157865218 >> v2:172.21.15.204:6812/28210 conn(0x7f8ed80397b0 0x7f8ed803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.139+0000 7f8eee7fc640 1 -- 172.21.15.204:0/4157865218 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8ee804afe0 con 0x7f8ef806ecf0 2024-09-15T01:34:43.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.139+0000 7f8eeffff640 1 --2- 172.21.15.204:0/4157865218 >> v2:172.21.15.204:6812/28210 conn(0x7f8ed80397b0 0x7f8ed803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.139+0000 7f8eeffff640 10 monclient: get_auth_request con 0x7f8ed80397b0 auth_method 0 2024-09-15T01:34:43.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.139+0000 7f8eeffff640 1 --2- 172.21.15.204:0/4157865218 >> v2:172.21.15.204:6812/28210 conn(0x7f8ed80397b0 0x7f8ed803bc60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0034000 tx=0x7f8ee0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.143+0000 7f8eee7fc640 1 -- 172.21.15.204:0/4157865218 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8ee807ce40 con 0x7f8ef806ecf0 2024-09-15T01:34:43.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.143+0000 7f8eee7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.143+0000 7f8eee7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:43.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.298+0000 7f8efcf1d640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "host1"}] 2024-09-15T01:34:43.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.298+0000 7f8efcf1d640 10 monclient: _send_command 2 [{"prefix": "osd tree-from", "bucket": "host1"}] 2024-09-15T01:34:43.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.298+0000 7f8efcf1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:43.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.298+0000 7f8efcf1d640 1 -- 172.21.15.204:0/4157865218 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd tree-from", "bucket": "host1"} v 0) v1 -- 0x7f8ef8071140 con 0x7f8ef806ecf0 2024-09-15T01:34:43.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.298+0000 7f8eee7fc640 1 -- 172.21.15.204:0/4157865218 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "host1"}]=-2 bucket 'host1' does not exist v81) v1 ==== 109+0+0 (secure 0 0 0) 0x7f8ee8003e10 con 0x7f8ef806ecf0 2024-09-15T01:34:43.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.298+0000 7f8eee7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree-from", "bucket": "host1"}] 2024-09-15T01:34:43.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.298+0000 7f8eee7fc640 10 monclient: _finish_command 2 = mon:2 bucket 'host1' does not exist 2024-09-15T01:34:43.300 INFO:tasks.workunit.client.0.smithi204.stderr:Error ENOENT: bucket 'host1' does not exist 2024-09-15T01:34:43.303 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:34:43.303 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree-from host2 2024-09-15T01:34:43.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.374+0000 7f2fc1b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:43.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.374+0000 7f2fc1b9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:43.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.374+0000 7f2fc1b9d640 10 monclient: monmap: 2024-09-15T01:34:43.376 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:43.376 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.376 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:43.375110+0000 2024-09-15T01:34:43.376 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:43.375110+0000 2024-09-15T01:34:43.376 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:43.376 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.376 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:43.377 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:43.377 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:43.377 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:43.377 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:43.377 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:43.377 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:43.377 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:43.377 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:43.377 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:43.377 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:43.377 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.375+0000 7f2fc1b9d640 1 Processor -- start 2024-09-15T01:34:43.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.375+0000 7f2fc1b9d640 1 -- start start 2024-09-15T01:34:43.386 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.375+0000 7f2fc1b9d640 10 monclient: init 2024-09-15T01:34:43.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.375+0000 7f2fc1b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:43.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.375+0000 7f2fc1b9d640 10 monclient: _add_conns ranks=[3,5,10,1,4,9,6,2,0,7,8] 2024-09-15T01:34:43.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.375+0000 7f2fc1b9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fbc0a2400 0x7f2fbc0a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.375+0000 7f2fc1b9d640 10 monclient(hunting): picked mon.noname-g con 0x7f2fbc0a2400 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:43.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.375+0000 7f2fc1b9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2fbc09f990 0x7f2fbc09fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fc1b9d640 10 monclient(hunting): picked mon.noname-h con 0x7f2fbc09f990 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:43.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fc1b9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2fbc132930 0x7f2fbc132d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fc1b9d640 10 monclient(hunting): picked mon.noname-f con 0x7f2fbc132930 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:43.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fc1b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fc1b9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc09e1a0 con 0x7f2fbc132930 2024-09-15T01:34:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fc1b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fc1b9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc0a27d0 con 0x7f2fbc0a2400 2024-09-15T01:34:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fc1b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fc1b9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc09dec0 con 0x7f2fbc09f990 2024-09-15T01:34:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fc1b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fc1b9d640 10 monclient(hunting): authenticate will time out at 899.018057s 2024-09-15T01:34:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbb7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fbc0a2400 0x7f2fbc0a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbaffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2fbc09f990 0x7f2fbc09fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2fbc132930 0x7f2fbc132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbb7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fbc0a2400 0x7f2fbc0a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48622/0 (socket says 172.21.15.204:48622) 2024-09-15T01:34:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbaffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2fbc09f990 0x7f2fbc09fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:37704/0 (socket says 172.21.15.204:37704) 2024-09-15T01:34:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbb7fe640 1 -- 172.21.15.204:0/2983063399 learned_addr learned my addr 172.21.15.204:0/2983063399 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:43.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbb7fe640 10 monclient(hunting): get_auth_request con 0x7f2fbc0a2400 auth_method 0 2024-09-15T01:34:43.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbaffd640 10 monclient(hunting): get_auth_request con 0x7f2fbc09f990 auth_method 0 2024-09-15T01:34:43.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 10 monclient(hunting): get_auth_request con 0x7f2fbc132930 auth_method 0 2024-09-15T01:34:43.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 10 monclient(hunting): handle_auth_done global_id 5770 payload 274 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 1 -- 172.21.15.204:0/2983063399 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2fbc09f990 msgr2=0x7f2fbc09fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 1 --2- 172.21.15.204:0/2983063399 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2fbc09f990 0x7f2fbc09fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 1 -- 172.21.15.204:0/2983063399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fbc0a2400 msgr2=0x7f2fbc0a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:43.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 1 --2- 172.21.15.204:0/2983063399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fbc0a2400 0x7f2fbc0a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:43.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 1 monclient: found mon.noname-f 2024-09-15T01:34:43.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:43.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:43.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbbfff640 1 -- 172.21.15.204:0/2983063399 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc0648a0 con 0x7f2fbc132930 2024-09-15T01:34:43.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbb7fe640 1 --2- 172.21.15.204:0/2983063399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fbc0a2400 0x7f2fbc0a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:43.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.376+0000 7f2fbaffd640 1 --2- 172.21.15.204:0/2983063399 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2fbc09f990 0x7f2fbc09fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:43.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fbbfff640 1 --2- 172.21.15.204:0/2983063399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fbc132930 0x7f2fbc132d60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009850 tx=0x7f2fb40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0dabc898dbf505 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fba7fc640 1 -- 172.21.15.204:0/2983063399 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fb402abc0 con 0x7f2fbc132930 2024-09-15T01:34:43.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fba7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:43.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fba7fc640 10 monclient: dump: 2024-09-15T01:34:43.392 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.392 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.392 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.392 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.392 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.392 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.392 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.392 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.392 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.392 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.393 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.393 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.393 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.393 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.393 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.393 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fba7fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:43.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:43.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fba7fc640 1 -- 172.21.15.204:0/2983063399 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb402ad40 con 0x7f2fbc132930 2024-09-15T01:34:43.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 5 monclient: authenticate success, global_id 5770 2024-09-15T01:34:43.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:43.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fba7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:43.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fba7fc640 1 -- 172.21.15.204:0/2983063399 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fb4031ca0 con 0x7f2fbc132930 2024-09-15T01:34:43.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:43.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 4 set_mon_vals no callback set 2024-09-15T01:34:43.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fba7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:43.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fba7fc640 10 monclient: dump: 2024-09-15T01:34:43.394 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.394 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.394 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.394 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.394 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.394 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 10 monclient: shutdown 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:43.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/2983063399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fbc132930 msgr2=0x7f2fbc132d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 1 --2- 172.21.15.204:0/2983063399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fbc132930 0x7f2fbc132d60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009850 tx=0x7f2fb40090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/2983063399 shutdown_connections 2024-09-15T01:34:43.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 1 --2- 172.21.15.204:0/2983063399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fbc132930 0x7f2fbc132d60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 1 --2- 172.21.15.204:0/2983063399 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2fbc09f990 0x7f2fbc09fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 1 --2- 172.21.15.204:0/2983063399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fbc0a2400 0x7f2fbc0a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/2983063399 >> 172.21.15.204:0/2983063399 conn(0x7f2fbc06e860 msgr2=0x7f2fbc06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:43.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.377+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/2983063399 shutdown_connections 2024-09-15T01:34:43.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.378+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/2983063399 wait complete. 2024-09-15T01:34:43.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.378+0000 7f2fc1b9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:43.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.378+0000 7f2fc1b9d640 10 monclient: monmap: 2024-09-15T01:34:43.396 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:43.397 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.378+0000 7f2fc1b9d640 1 Processor -- start 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.378+0000 7f2fc1b9d640 1 -- start start 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.378+0000 7f2fc1b9d640 10 monclient: init 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 10 monclient: _add_conns ranks=[2,6,0,3,5,8,4,1,7] 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fbc09f990 0x7f2fbc09c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 10 monclient(hunting): picked mon.noname-b con 0x7f2fbc09f990 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2fbc0a2400 0x7f2fbc11c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 10 monclient(hunting): picked mon.noname-d con 0x7f2fbc0a2400 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fbc132930 0x7f2fbc11ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 10 monclient(hunting): picked mon.noname-a con 0x7f2fbc132930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc09d3d0 con 0x7f2fbc132930 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc11f910 con 0x7f2fbc09f990 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc11e500 con 0x7f2fbc0a2400 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fc1b9d640 10 monclient(hunting): authenticate will time out at 899.021417s 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbb7fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fbc09f990 msgr2=0x7f2fbc09c230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbaffd640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2fbc0a2400 msgr2=0x7f2fbc11c550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbb7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fbc09f990 0x7f2fbc09c230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbaffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2fbc0a2400 0x7f2fbc11c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:43.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fbc132930 0x7f2fbc11ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fbc132930 0x7f2fbc11ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:54648/0 (socket says 172.21.15.204:54648) 2024-09-15T01:34:43.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 1 -- 172.21.15.204:0/3141317603 learned_addr learned my addr 172.21.15.204:0/3141317603 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:43.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 10 monclient(hunting): get_auth_request con 0x7f2fbc132930 auth_method 0 2024-09-15T01:34:43.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 10 monclient(hunting): handle_auth_done global_id 5499 payload 824 2024-09-15T01:34:43.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 1 -- 172.21.15.204:0/3141317603 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2fbc0a2400 msgr2=0x7f2fbc11c550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:43.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 1 --2- 172.21.15.204:0/3141317603 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2fbc0a2400 0x7f2fbc11c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 1 -- 172.21.15.204:0/3141317603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fbc09f990 msgr2=0x7f2fbc09c230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 1 --2- 172.21.15.204:0/3141317603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fbc09f990 0x7f2fbc09c230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 1 monclient: found mon.noname-a 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.379+0000 7f2fbbfff640 1 -- 172.21.15.204:0/3141317603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb0009390 con 0x7f2fbc132930 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fbbfff640 1 --2- 172.21.15.204:0/3141317603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fbc132930 0x7f2fbc11ca90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc11e6d0 tx=0x7f2fb4032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fa37fe640 1 -- 172.21.15.204:0/3141317603 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fb403ca90 con 0x7f2fbc132930 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fa37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fa37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fa37fe640 10 monclient: dump: 2024-09-15T01:34:43.401 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.402 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.403 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.403 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fa37fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:43.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fa37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:43.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fa37fe640 1 -- 172.21.15.204:0/3141317603 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb403cc30 con 0x7f2fbc132930 2024-09-15T01:34:43.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fc1b9d640 5 monclient: authenticate success, global_id 5499 2024-09-15T01:34:43.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fc1b9d640 10 monclient: _renew_subs 2024-09-15T01:34:43.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fc1b9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:43.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/3141317603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac009ae0 con 0x7f2fbc132930 2024-09-15T01:34:43.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fa37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:43.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fa37fe640 1 -- 172.21.15.204:0/3141317603 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fb403cda0 con 0x7f2fbc132930 2024-09-15T01:34:43.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fa37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fa37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:43.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fa37fe640 10 monclient: dump: 2024-09-15T01:34:43.404 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.404 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.404 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.404 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.404 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.404 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.404 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.404 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.404 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.405 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.405 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.405 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.405 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.405 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.405 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.405 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fc1b9d640 10 monclient: _renew_subs 2024-09-15T01:34:43.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fb9ffb640 4 set_mon_vals no callback set 2024-09-15T01:34:43.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fc1b9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:43.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.380+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/3141317603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb4009390 con 0x7f2fbc132930 2024-09-15T01:34:43.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.381+0000 7f2fa37fe640 1 -- 172.21.15.204:0/3141317603 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2fb403c610 con 0x7f2fbc132930 2024-09-15T01:34:43.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.381+0000 7f2fc1b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.381+0000 7f2fc1b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.381+0000 7f2fc1b9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:43.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.381+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/3141317603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc070b80 con 0x7f2fbc132930 2024-09-15T01:34:43.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.381+0000 7f2fa37fe640 1 --2- 172.21.15.204:0/3141317603 >> v2:172.21.15.204:6812/28210 conn(0x7f2fa80397e0 0x7f2fa803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.381+0000 7f2fa37fe640 1 -- 172.21.15.204:0/3141317603 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2fb406baa0 con 0x7f2fbc132930 2024-09-15T01:34:43.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.381+0000 7f2fbb7fe640 1 --2- 172.21.15.204:0/3141317603 >> v2:172.21.15.204:6812/28210 conn(0x7f2fa80397e0 0x7f2fa803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.381+0000 7f2fbb7fe640 10 monclient: get_auth_request con 0x7f2fa80397e0 auth_method 0 2024-09-15T01:34:43.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.381+0000 7f2fbb7fe640 1 --2- 172.21.15.204:0/3141317603 >> v2:172.21.15.204:6812/28210 conn(0x7f2fa80397e0 0x7f2fa803bc90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2fac007250 tx=0x7f2fac00b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.407 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.385+0000 7f2fa37fe640 1 -- 172.21.15.204:0/3141317603 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2fb403c8c0 con 0x7f2fbc132930 2024-09-15T01:34:43.407 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.385+0000 7f2fa37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.407 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.385+0000 7f2fa37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:43.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.540+0000 7f2fc1b9d640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "host2"}] 2024-09-15T01:34:43.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.540+0000 7f2fc1b9d640 10 monclient: _send_command 2 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-09-15T01:34:43.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.540+0000 7f2fc1b9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:43.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.540+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/3141317603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd tree-from", "bucket": "host2"} v 0) v1 -- 0x7f2fbc09e480 con 0x7f2fbc132930 2024-09-15T01:34:43.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.541+0000 7f2fa37fe640 1 -- 172.21.15.204:0/3141317603 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "host2"}]=0 v81) v1 ==== 80+0+177 (secure 0 0 0) 0x7f2fb4037070 con 0x7f2fbc132930 2024-09-15T01:34:43.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.541+0000 7f2fa37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-09-15T01:34:43.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.541+0000 7f2fa37fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:43.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.636+0000 7f2fc1b9d640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "host2"}] 2024-09-15T01:34:43.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.636+0000 7f2fc1b9d640 10 monclient: _send_command 3 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-09-15T01:34:43.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.636+0000 7f2fc1b9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:43.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.636+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/3141317603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd tree-from", "bucket": "host2"} v 0) v1 -- 0x7f2fbc0a5b10 con 0x7f2fbc132930 2024-09-15T01:34:43.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.636+0000 7f2fa37fe640 1 -- 172.21.15.204:0/3141317603 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "host2"}]=0 v81) v1 ==== 80+0+177 (secure 0 0 0) 0x7f2fb404b910 con 0x7f2fbc132930 2024-09-15T01:34:43.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.636+0000 7f2fa37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-09-15T01:34:43.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.636+0000 7f2fa37fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:43.638 INFO:tasks.workunit.client.0.smithi204.stdout:ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF 2024-09-15T01:34:43.638 INFO:tasks.workunit.client.0.smithi204.stdout:-16 1.00000 host host2 2024-09-15T01:34:43.639 INFO:tasks.workunit.client.0.smithi204.stdout: 9 1.00000 osd.9 down 1.00000 1.00000 2024-09-15T01:34:43.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/3141317603 >> v2:172.21.15.204:6812/28210 conn(0x7f2fa80397e0 msgr2=0x7f2fa803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 1 --2- 172.21.15.204:0/3141317603 >> v2:172.21.15.204:6812/28210 conn(0x7f2fa80397e0 0x7f2fa803bc90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2fac007250 tx=0x7f2fac00b040 comp rx=0 tx=0).stop 2024-09-15T01:34:43.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 10 monclient: shutdown 2024-09-15T01:34:43.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:43.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/3141317603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fbc132930 msgr2=0x7f2fbc11ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 1 --2- 172.21.15.204:0/3141317603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fbc132930 0x7f2fbc11ca90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc11e6d0 tx=0x7f2fb4032000 comp rx=0 tx=0).stop 2024-09-15T01:34:43.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/3141317603 shutdown_connections 2024-09-15T01:34:43.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 1 --2- 172.21.15.204:0/3141317603 >> v2:172.21.15.204:6812/28210 conn(0x7f2fa80397e0 0x7f2fa803bc90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 1 --2- 172.21.15.204:0/3141317603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fbc132930 0x7f2fbc11ca90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 1 --2- 172.21.15.204:0/3141317603 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2fbc0a2400 0x7f2fbc11c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 1 --2- 172.21.15.204:0/3141317603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fbc09f990 0x7f2fbc09c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/3141317603 >> 172.21.15.204:0/3141317603 conn(0x7f2fbc06e860 msgr2=0x7f2fbc130020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:43.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/3141317603 shutdown_connections 2024-09-15T01:34:43.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.638+0000 7f2fc1b9d640 1 -- 172.21.15.204:0/3141317603 wait complete. 2024-09-15T01:34:43.649 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd tree-from osd.9 2024-09-15T01:34:43.649 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:34:43.649 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree-from osd.9 2024-09-15T01:34:43.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.720+0000 7ff277d9d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:43.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.720+0000 7ff277d9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:43.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.720+0000 7ff277d9d640 10 monclient: monmap: 2024-09-15T01:34:43.722 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:43.722 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.722 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:43.720834+0000 2024-09-15T01:34:43.722 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:43.720834+0000 2024-09-15T01:34:43.722 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:43.722 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.722 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:43.722 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:43.722 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:43.722 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:43.722 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:43.722 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:43.723 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:43.723 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:43.723 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:43.723 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:43.723 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:43.723 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.720+0000 7ff277d9d640 1 Processor -- start 2024-09-15T01:34:43.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 1 -- start start 2024-09-15T01:34:43.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 10 monclient: init 2024-09-15T01:34:43.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:43.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 10 monclient: _add_conns ranks=[0,4,7,10,9,1,3,2,8,5,6] 2024-09-15T01:34:43.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff2700969e0 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:43.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff270068240 0x7ff270134ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 10 monclient(hunting): picked mon.noname-c con 0x7ff270068240 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:43.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff2700974c0 0x7ff2701371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 10 monclient(hunting): picked mon.noname-i con 0x7ff2700974c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:43.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2700648a0 con 0x7ff2700969e0 2024-09-15T01:34:43.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff27009c740 con 0x7ff270068240 2024-09-15T01:34:43.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff270098010 con 0x7ff2700974c0 2024-09-15T01:34:43.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:43.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff277d9d640 10 monclient(hunting): authenticate will time out at 899.363711s 2024-09-15T01:34:43.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff27633a640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff2700974c0 msgr2=0x7ff2701371f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:43.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff27633a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff2700974c0 0x7ff2701371f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:43.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff275338640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff270068240 0x7ff270134ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff275b39640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff2700969e0 0x7ff270096db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.204:44494/0 (socket says 172.21.15.204:44494) 2024-09-15T01:34:43.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff275b39640 1 -- 172.21.15.204:0/3691974376 learned_addr learned my addr 172.21.15.204:0/3691974376 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:34:43.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff275338640 10 monclient(hunting): get_auth_request con 0x7ff270068240 auth_method 0 2024-09-15T01:34:43.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff275338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff275338640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.721+0000 7ff275338640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 10 monclient(hunting): handle_auth_done global_id 5287 payload 274 2024-09-15T01:34:43.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 1 -- 172.21.15.204:0/3691974376 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff2700974c0 msgr2=0x7ff2701371f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:43.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 1 --2- 172.21.15.204:0/3691974376 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff2700974c0 0x7ff2701371f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 1 -- 172.21.15.204:0/3691974376 >> v1:172.21.15.178:6789/0 conn(0x7ff2700969e0 legacy=0x7ff270096db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff274b37640 1 -- 172.21.15.204:0/3691974376 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513365079 0 0) 0x7ff2700648a0 con 0x7ff2700969e0 2024-09-15T01:34:43.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 10 monclient: _finish_hunting 0 2024-09-15T01:34:43.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 1 monclient: found mon.noname-c 2024-09-15T01:34:43.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:43.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:43.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 1 -- 172.21.15.204:0/3691974376 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff270098890 con 0x7ff270068240 2024-09-15T01:34:43.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff274b37640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:43.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.722+0000 7ff275338640 1 --2- 172.21.15.204:0/3691974376 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff270068240 0x7ff270134ca0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff260009880 tx=0x7ff2600090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=eadbe7ad6b2283b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff274b37640 1 -- 172.21.15.204:0/3691974376 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff26002ac80 con 0x7ff270068240 2024-09-15T01:34:43.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff274b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff274b37640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:43.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff274b37640 10 monclient: dump: 2024-09-15T01:34:43.736 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.736 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.736 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.736 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.736 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.736 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.736 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.736 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.737 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.737 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.737 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.737 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.737 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.737 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.737 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.737 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff274b37640 10 monclient: _finish_auth 0 2024-09-15T01:34:43.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff274b37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:43.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff274b37640 1 -- 172.21.15.204:0/3691974376 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26002ae00 con 0x7ff270068240 2024-09-15T01:34:43.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 5 monclient: authenticate success, global_id 5287 2024-09-15T01:34:43.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:43.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff274b37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:43.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:43.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff274b37640 1 -- 172.21.15.204:0/3691974376 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff260031ca0 con 0x7ff270068240 2024-09-15T01:34:43.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff274b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 4 set_mon_vals no callback set 2024-09-15T01:34:43.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff274b37640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:43.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff274b37640 10 monclient: dump: 2024-09-15T01:34:43.738 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.738 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.738 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.738 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.739 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.739 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.739 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.739 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.739 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.739 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.739 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.739 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.739 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.739 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.739 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.739 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 10 monclient: shutdown 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 1 -- 172.21.15.204:0/3691974376 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff270068240 msgr2=0x7ff270134ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 1 --2- 172.21.15.204:0/3691974376 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff270068240 0x7ff270134ca0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff260009880 tx=0x7ff2600090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 1 -- 172.21.15.204:0/3691974376 shutdown_connections 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 1 --2- 172.21.15.204:0/3691974376 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff2700974c0 0x7ff2701371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 1 --2- 172.21.15.204:0/3691974376 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff270068240 0x7ff270134ca0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 1 -- 172.21.15.204:0/3691974376 >> 172.21.15.204:0/3691974376 conn(0x7ff2700a2ce0 msgr2=0x7ff2700a0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 1 -- 172.21.15.204:0/3691974376 shutdown_connections 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.723+0000 7ff277d9d640 1 -- 172.21.15.204:0/3691974376 wait complete. 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 10 monclient: build_initial_monmap 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 10 monclient: monmap: 2024-09-15T01:34:43.740 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:43.741 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:43.741 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:43.741 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:43.741 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:43.741 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:43.741 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:43.741 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:43.741 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:43.741 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:43.741 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:43.741 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:43.741 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:43.742 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:43.742 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:43.742 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 1 Processor -- start 2024-09-15T01:34:43.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 1 -- start start 2024-09-15T01:34:43.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 10 monclient: init 2024-09-15T01:34:43.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:43.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 10 monclient: _add_conns ranks=[7,2,0,6,5,8,4,1,3] 2024-09-15T01:34:43.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff270068240 0x7ff270067130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 10 monclient(hunting): picked mon.noname-i con 0x7ff270068240 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:43.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2700969e0 0x7ff27013de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 10 monclient(hunting): picked mon.noname-b con 0x7ff2700969e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:43.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2700974c0 0x7ff27013e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 10 monclient(hunting): picked mon.noname-a con 0x7ff2700974c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:43.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.724+0000 7ff277d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff277d9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2701234d0 con 0x7ff2700974c0 2024-09-15T01:34:43.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff277d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff277d9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff270067e00 con 0x7ff2700969e0 2024-09-15T01:34:43.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff277d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff277d9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff27009ba50 con 0x7ff270068240 2024-09-15T01:34:43.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff277d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:43.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff277d9d640 10 monclient(hunting): authenticate will time out at 899.367016s 2024-09-15T01:34:43.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff275b39640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff270068240 msgr2=0x7ff270067130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:43.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff275b39640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff270068240 0x7ff270067130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:43.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff275338640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2700969e0 msgr2=0x7ff27013de70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:43.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff275338640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2700969e0 0x7ff27013de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:43.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2700974c0 0x7ff27013e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2700974c0 0x7ff27013e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:54654/0 (socket says 172.21.15.204:54654) 2024-09-15T01:34:43.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 1 -- 172.21.15.204:0/3487619736 learned_addr learned my addr 172.21.15.204:0/3487619736 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:43.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 10 monclient(hunting): get_auth_request con 0x7ff2700974c0 auth_method 0 2024-09-15T01:34:43.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 10 monclient(hunting): handle_auth_done global_id 5517 payload 824 2024-09-15T01:34:43.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 1 -- 172.21.15.204:0/3487619736 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff270068240 msgr2=0x7ff270067130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:43.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 1 --2- 172.21.15.204:0/3487619736 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff270068240 0x7ff270067130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 1 -- 172.21.15.204:0/3487619736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2700969e0 msgr2=0x7ff27013de70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:43.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 1 --2- 172.21.15.204:0/3487619736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2700969e0 0x7ff27013de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 10 monclient: _finish_hunting 0 2024-09-15T01:34:43.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 1 monclient: found mon.noname-a 2024-09-15T01:34:43.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:43.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:43.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 1 -- 172.21.15.204:0/3487619736 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff260009390 con 0x7ff2700974c0 2024-09-15T01:34:43.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff27633a640 1 --2- 172.21.15.204:0/3487619736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2700974c0 0x7ff27013e7c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff26800bee0 tx=0x7ff2680096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff265ffb640 1 -- 172.21.15.204:0/3487619736 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff26800cd00 con 0x7ff2700974c0 2024-09-15T01:34:43.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff265ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff265ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:43.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff265ffb640 10 monclient: dump: 2024-09-15T01:34:43.746 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.746 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.746 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.746 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.746 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.746 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff265ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff265ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:43.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff265ffb640 1 -- 172.21.15.204:0/3487619736 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26800ce80 con 0x7ff2700974c0 2024-09-15T01:34:43.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff277d9d640 5 monclient: authenticate success, global_id 5517 2024-09-15T01:34:43.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff277d9d640 10 monclient: _renew_subs 2024-09-15T01:34:43.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff277d9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:43.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff277d9d640 1 -- 172.21.15.204:0/3487619736 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff270122f30 con 0x7ff2700974c0 2024-09-15T01:34:43.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff265ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:43.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff265ffb640 1 -- 172.21.15.204:0/3487619736 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff268006a50 con 0x7ff2700974c0 2024-09-15T01:34:43.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff265ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff265ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:43.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff265ffb640 10 monclient: dump: 2024-09-15T01:34:43.748 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.748 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.748 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.749 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.725+0000 7ff267fff640 4 set_mon_vals no callback set 2024-09-15T01:34:43.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.726+0000 7ff277d9d640 10 monclient: _renew_subs 2024-09-15T01:34:43.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.726+0000 7ff277d9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:43.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.726+0000 7ff277d9d640 1 -- 172.21.15.204:0/3487619736 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27011dc30 con 0x7ff2700974c0 2024-09-15T01:34:43.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.726+0000 7ff265ffb640 1 -- 172.21.15.204:0/3487619736 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff268012020 con 0x7ff2700974c0 2024-09-15T01:34:43.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.727+0000 7ff277d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.727+0000 7ff277d9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.727+0000 7ff277d9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:43.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.727+0000 7ff277d9d640 1 -- 172.21.15.204:0/3487619736 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff270070b80 con 0x7ff2700974c0 2024-09-15T01:34:43.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.727+0000 7ff265ffb640 1 --2- 172.21.15.204:0/3487619736 >> v2:172.21.15.204:6812/28210 conn(0x7ff2540397e0 0x7ff25403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.727+0000 7ff265ffb640 1 -- 172.21.15.204:0/3487619736 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff26804b870 con 0x7ff2700974c0 2024-09-15T01:34:43.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.727+0000 7ff275b39640 1 --2- 172.21.15.204:0/3487619736 >> v2:172.21.15.204:6812/28210 conn(0x7ff2540397e0 0x7ff25403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.727+0000 7ff275b39640 10 monclient: get_auth_request con 0x7ff2540397e0 auth_method 0 2024-09-15T01:34:43.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.727+0000 7ff275b39640 1 --2- 172.21.15.204:0/3487619736 >> v2:172.21.15.204:6812/28210 conn(0x7ff2540397e0 0x7ff25403bc90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff26c007fd0 tx=0x7ff26c007a90 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.731+0000 7ff265ffb640 1 -- 172.21.15.204:0/3487619736 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff268013ac0 con 0x7ff2700974c0 2024-09-15T01:34:43.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.731+0000 7ff265ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.731+0000 7ff265ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:43.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.885+0000 7ff277d9d640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "osd.9"}] 2024-09-15T01:34:43.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.885+0000 7ff277d9d640 10 monclient: _send_command 2 [{"prefix": "osd tree-from", "bucket": "osd.9"}] 2024-09-15T01:34:43.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.885+0000 7ff277d9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:34:43.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.885+0000 7ff277d9d640 1 -- 172.21.15.204:0/3487619736 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd tree-from", "bucket": "osd.9"} v 0) v1 -- 0x7ff270068e10 con 0x7ff2700974c0 2024-09-15T01:34:43.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.886+0000 7ff265ffb640 1 -- 172.21.15.204:0/3487619736 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "osd.9"}]=-22 "osd.9" is not a bucket v81) v1 ==== 103+0+0 (secure 0 0 0) 0x7ff2680132e0 con 0x7ff2700974c0 2024-09-15T01:34:43.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.886+0000 7ff265ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree-from", "bucket": "osd.9"}] 2024-09-15T01:34:43.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.886+0000 7ff265ffb640 10 monclient: _finish_command 2 = mon:22 "osd.9" is not a bucket 2024-09-15T01:34:43.888 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: "osd.9" is not a bucket 2024-09-15T01:34:43.891 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:34:43.891 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush rm bar 2024-09-15T01:34:43.891 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:34:43.892 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm bar 2024-09-15T01:34:43.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.963+0000 7fe60db1d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:43.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.963+0000 7fe60db1d640 10 monclient: build_initial_monmap 2024-09-15T01:34:43.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.963+0000 7fe60db1d640 10 monclient: monmap: 2024-09-15T01:34:43.965 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:43.965 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.965 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:43.963973+0000 2024-09-15T01:34:43.965 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:43.963973+0000 2024-09-15T01:34:43.965 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:43.965 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.966 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:43.966 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:43.966 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:43.966 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:43.966 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:43.966 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:43.966 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:43.966 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:43.966 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:43.966 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:43.967 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:43.967 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 1 Processor -- start 2024-09-15T01:34:43.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 1 -- start start 2024-09-15T01:34:43.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 10 monclient: init 2024-09-15T01:34:43.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:43.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 10 monclient: _add_conns ranks=[7,6,9,1,10,8,3,5,2,0,4] 2024-09-15T01:34:43.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6080a2e30 0x7fe608132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 10 monclient(hunting): picked mon.noname-i con 0x7fe6080a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:43.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe60809fb80 0x7fe60809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 10 monclient(hunting): picked mon.noname-d con 0x7fe60809fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:43.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe6080a0a40 0x7fe6080a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 10 monclient(hunting): picked mon.noname-a con 0x7fe6080a0a40 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:43.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60809e6c0 con 0x7fe6080a0a40 2024-09-15T01:34:43.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080a04b0 con 0x7fe60809fb80 2024-09-15T01:34:43.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080a3200 con 0x7fe6080a2e30 2024-09-15T01:34:43.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:43.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe60db1d640 10 monclient(hunting): authenticate will time out at 899.606749s 2024-09-15T01:34:43.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe6077fe640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6080a2e30 msgr2=0x7fe608132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:43.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe6077fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6080a2e30 0x7fe608132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:43.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe606ffd640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe60809fb80 msgr2=0x7fe60809ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:43.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.964+0000 7fe606ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe60809fb80 0x7fe60809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:43.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe6080a0a40 0x7fe6080a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe6080a0a40 0x7fe6080a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:54670/0 (socket says 172.21.15.204:54670) 2024-09-15T01:34:43.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 1 -- 172.21.15.204:0/3080359234 learned_addr learned my addr 172.21.15.204:0/3080359234 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:43.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 10 monclient(hunting): get_auth_request con 0x7fe6080a0a40 auth_method 0 2024-09-15T01:34:43.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 10 monclient(hunting): handle_auth_done global_id 5526 payload 274 2024-09-15T01:34:43.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 1 -- 172.21.15.204:0/3080359234 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6080a2e30 msgr2=0x7fe608132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:43.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 1 --2- 172.21.15.204:0/3080359234 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6080a2e30 0x7fe608132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 1 -- 172.21.15.204:0/3080359234 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe60809fb80 msgr2=0x7fe60809ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:43.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 1 --2- 172.21.15.204:0/3080359234 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe60809fb80 0x7fe60809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:43.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 1 monclient: found mon.noname-a 2024-09-15T01:34:43.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:43.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:43.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.965+0000 7fe607fff640 1 -- 172.21.15.204:0/3080359234 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60804b720 con 0x7fe6080a0a40 2024-09-15T01:34:43.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe607fff640 1 --2- 172.21.15.204:0/3080359234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6080a0a40 0x7fe6080a0ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe600009f20 tx=0x7fe600009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aef95e522f78b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe6067fc640 1 -- 172.21.15.204:0/3080359234 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe600007880 con 0x7fe6080a0a40 2024-09-15T01:34:43.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe6067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe6067fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:43.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe6067fc640 10 monclient: dump: 2024-09-15T01:34:43.980 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.980 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.980 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.980 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.980 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.980 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.981 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.981 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.981 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.981 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.981 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.981 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.981 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.981 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.981 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.981 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe6067fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:43.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe6067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:43.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe6067fc640 1 -- 172.21.15.204:0/3080359234 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60002d050 con 0x7fe6080a0a40 2024-09-15T01:34:43.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe6067fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:43.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe6067fc640 1 -- 172.21.15.204:0/3080359234 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe600030360 con 0x7fe6080a0a40 2024-09-15T01:34:43.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe6067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe6067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:43.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe6067fc640 10 monclient: dump: 2024-09-15T01:34:43.982 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.982 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.982 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.982 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.982 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.983 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.983 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.983 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.983 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.983 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.983 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.983 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.983 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.983 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.984 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.984 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 5 monclient: authenticate success, global_id 5526 2024-09-15T01:34:43.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:43.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 4 set_mon_vals no callback set 2024-09-15T01:34:43.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 10 monclient: shutdown 2024-09-15T01:34:43.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:43.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 1 -- 172.21.15.204:0/3080359234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6080a0a40 msgr2=0x7fe6080a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 1 --2- 172.21.15.204:0/3080359234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6080a0a40 0x7fe6080a0ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe600009f20 tx=0x7fe600009840 comp rx=0 tx=0).stop 2024-09-15T01:34:43.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 1 -- 172.21.15.204:0/3080359234 shutdown_connections 2024-09-15T01:34:43.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 1 --2- 172.21.15.204:0/3080359234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6080a0a40 0x7fe6080a0ee0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 1 --2- 172.21.15.204:0/3080359234 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe60809fb80 0x7fe60809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 1 --2- 172.21.15.204:0/3080359234 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6080a2e30 0x7fe608132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 1 -- 172.21.15.204:0/3080359234 >> 172.21.15.204:0/3080359234 conn(0x7fe60806e860 msgr2=0x7fe60806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:43.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 1 -- 172.21.15.204:0/3080359234 shutdown_connections 2024-09-15T01:34:43.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.966+0000 7fe60db1d640 1 -- 172.21.15.204:0/3080359234 wait complete. 2024-09-15T01:34:43.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.967+0000 7fe60db1d640 10 monclient: build_initial_monmap 2024-09-15T01:34:43.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.967+0000 7fe60db1d640 10 monclient: monmap: 2024-09-15T01:34:43.985 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:43.985 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:43.985 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:43.986 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:43.986 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:43.986 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:43.986 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:43.986 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:43.986 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:43.986 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:43.986 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:43.986 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:43.986 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:43.986 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:43.986 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:43.987 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.967+0000 7fe60db1d640 1 Processor -- start 2024-09-15T01:34:43.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.967+0000 7fe60db1d640 1 -- start start 2024-09-15T01:34:43.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.967+0000 7fe60db1d640 10 monclient: init 2024-09-15T01:34:43.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:43.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 10 monclient: _add_conns ranks=[5,3,1,8,6,0,7,2,4] 2024-09-15T01:34:43.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe60809fb80 0x7fe6081317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 10 monclient(hunting): picked mon.noname-h con 0x7fe60809fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:43.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6080a0a40 0x7fe60811e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 10 monclient(hunting): picked mon.noname-g con 0x7fe6080a0a40 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:43.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6080a2e30 0x7fe60811e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 10 monclient(hunting): picked mon.noname-f con 0x7fe6080a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:43.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe608121f60 con 0x7fe6080a2e30 2024-09-15T01:34:43.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60809ca40 con 0x7fe6080a0a40 2024-09-15T01:34:43.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:43.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe60809d650 con 0x7fe60809fb80 2024-09-15T01:34:43.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:43.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe60db1d640 10 monclient(hunting): authenticate will time out at 899.610381s 2024-09-15T01:34:43.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe6077fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe60809fb80 0x7fe6081317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe606ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6080a0a40 0x7fe60811e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6080a2e30 0x7fe60811e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe606ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6080a0a40 0x7fe60811e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48628/0 (socket says 172.21.15.204:48628) 2024-09-15T01:34:43.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe606ffd640 1 -- 172.21.15.204:0/1019638372 learned_addr learned my addr 172.21.15.204:0/1019638372 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:43.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe606ffd640 10 monclient(hunting): get_auth_request con 0x7fe6080a0a40 auth_method 0 2024-09-15T01:34:43.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe606ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe606ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe606ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe6077fe640 10 monclient(hunting): get_auth_request con 0x7fe60809fb80 auth_method 0 2024-09-15T01:34:43.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe6077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe6077fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe6077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 10 monclient(hunting): get_auth_request con 0x7fe6080a2e30 auth_method 0 2024-09-15T01:34:43.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:43.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:43.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:43.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe606ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe606ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe606ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe6077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:43.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe6077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:43.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe6077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:43.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 10 monclient(hunting): handle_auth_done global_id 5779 payload 824 2024-09-15T01:34:43.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 1 -- 172.21.15.204:0/1019638372 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe60809fb80 msgr2=0x7fe6081317c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 1 --2- 172.21.15.204:0/1019638372 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe60809fb80 0x7fe6081317c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 1 -- 172.21.15.204:0/1019638372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6080a0a40 msgr2=0x7fe60811e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:43.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 1 --2- 172.21.15.204:0/1019638372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6080a0a40 0x7fe60811e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:43.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:43.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 1 monclient: found mon.noname-f 2024-09-15T01:34:43.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:43.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:43.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe607fff640 1 -- 172.21.15.204:0/1019638372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe600009ae0 con 0x7fe6080a2e30 2024-09-15T01:34:43.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe606ffd640 1 --2- 172.21.15.204:0/1019638372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6080a0a40 0x7fe60811e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:43.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.968+0000 7fe6077fe640 1 --2- 172.21.15.204:0/1019638372 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe60809fb80 0x7fe6081317c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:43.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe607fff640 1 --2- 172.21.15.204:0/1019638372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6080a2e30 0x7fe60811e840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe60809e800 tx=0x7fe600036000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe5ef7fe640 1 -- 172.21.15.204:0/1019638372 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe600008b20 con 0x7fe6080a2e30 2024-09-15T01:34:43.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe5ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe5ef7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:43.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe5ef7fe640 10 monclient: dump: 2024-09-15T01:34:43.992 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.992 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.992 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.992 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.992 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe5ef7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:43.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe5ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:43.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe60db1d640 5 monclient: authenticate success, global_id 5779 2024-09-15T01:34:43.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe5ef7fe640 1 -- 172.21.15.204:0/1019638372 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe600008cc0 con 0x7fe6080a2e30 2024-09-15T01:34:43.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe60db1d640 10 monclient: _renew_subs 2024-09-15T01:34:43.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe60db1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:43.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe60db1d640 1 -- 172.21.15.204:0/1019638372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe608142970 con 0x7fe6080a2e30 2024-09-15T01:34:43.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe5ef7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:43.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe5ef7fe640 1 -- 172.21.15.204:0/1019638372 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe600008e30 con 0x7fe6080a2e30 2024-09-15T01:34:43.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe5ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:43.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe5ef7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:43.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe5ef7fe640 10 monclient: dump: 2024-09-15T01:34:43.994 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:43.995 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:43.995 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.995 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:43.995 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:43.995 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:43.995 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:43.995 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:43.995 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:43.995 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:43.995 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:43.995 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:43.995 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:43.996 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:43.996 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:43.996 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:43.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe60db1d640 10 monclient: _renew_subs 2024-09-15T01:34:43.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe60db1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:43.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe605ffb640 4 set_mon_vals no callback set 2024-09-15T01:34:43.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.969+0000 7fe60db1d640 1 -- 172.21.15.204:0/1019638372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe608120550 con 0x7fe6080a2e30 2024-09-15T01:34:43.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.970+0000 7fe5ef7fe640 1 -- 172.21.15.204:0/1019638372 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe600038050 con 0x7fe6080a2e30 2024-09-15T01:34:43.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.970+0000 7fe5ef7fe640 1 --2- 172.21.15.204:0/1019638372 >> v2:172.21.15.204:6812/28210 conn(0x7fe5f40397b0 0x7fe5f403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:43.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.970+0000 7fe60db1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.970+0000 7fe60db1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.970+0000 7fe60db1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:43.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.970+0000 7fe5ef7fe640 1 -- 172.21.15.204:0/1019638372 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe60006b890 con 0x7fe6080a2e30 2024-09-15T01:34:43.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.970+0000 7fe60db1d640 1 -- 172.21.15.204:0/1019638372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe608121a30 con 0x7fe6080a2e30 2024-09-15T01:34:43.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.970+0000 7fe6077fe640 1 --2- 172.21.15.204:0/1019638372 >> v2:172.21.15.204:6812/28210 conn(0x7fe5f40397b0 0x7fe5f403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:43.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.970+0000 7fe6077fe640 10 monclient: get_auth_request con 0x7fe5f40397b0 auth_method 0 2024-09-15T01:34:43.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.970+0000 7fe6077fe640 1 --2- 172.21.15.204:0/1019638372 >> v2:172.21.15.204:6812/28210 conn(0x7fe5f40397b0 0x7fe5f403bc60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8003850 tx=0x7fe5f80090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:43.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.975+0000 7fe5ef7fe640 1 -- 172.21.15.204:0/1019638372 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe60003f080 con 0x7fe6080a2e30 2024-09-15T01:34:43.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.975+0000 7fe5ef7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:43.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:43.975+0000 7fe5ef7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.133+0000 7fe60db1d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T01:34:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.133+0000 7fe60db1d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T01:34:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.133+0000 7fe60db1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.133+0000 7fe60db1d640 1 -- 172.21.15.204:0/1019638372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) v1 -- 0x7fe6080668b0 con 0x7fe6080a2e30 2024-09-15T01:34:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.133+0000 7fe5ef7fe640 1 -- 172.21.15.204:0/1019638372 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=-39 (39) Directory not empty v81) v1 ==== 99+0+0 (secure 0 0 0) 0x7fe600039660 con 0x7fe6080a2e30 2024-09-15T01:34:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.133+0000 7fe5ef7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T01:34:44.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.133+0000 7fe5ef7fe640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-09-15T01:34:44.136 INFO:tasks.workunit.client.0.smithi204.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-09-15T01:34:44.138 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:34:44.138 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush unlink host2 2024-09-15T01:34:44.212 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.211+0000 7fdf24a99640 10 monclient: get_monmap_and_config 2024-09-15T01:34:44.212 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.211+0000 7fdf24a99640 10 monclient: build_initial_monmap 2024-09-15T01:34:44.212 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.211+0000 7fdf24a99640 10 monclient: monmap: 2024-09-15T01:34:44.212 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:44.213 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:44.213 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:44.211616+0000 2024-09-15T01:34:44.213 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:44.211616+0000 2024-09-15T01:34:44.213 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:44.213 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:44.213 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:44.213 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:44.213 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:44.213 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:44.213 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:44.214 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:44.214 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:44.214 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:44.214 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:44.214 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:44.214 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:44.214 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:44.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.211+0000 7fdf24a99640 1 Processor -- start 2024-09-15T01:34:44.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.211+0000 7fdf24a99640 1 -- start start 2024-09-15T01:34:44.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.211+0000 7fdf24a99640 10 monclient: init 2024-09-15T01:34:44.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf24a99640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:44.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf24a99640 10 monclient: _add_conns ranks=[5,0,1,8,4,7,6,10,2,3,9] 2024-09-15T01:34:44.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf24a99640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdf20067130 0x7fdf2009d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:44.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf24a99640 10 monclient(hunting): picked mon.noname-h con 0x7fdf20067130 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:44.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf24a99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdf2009c6c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf24a99640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fdf20137c80 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf24a99640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2004b720 con 0x7fdf2009c6c0 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf24a99640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2009abd0 con 0x7fdf20137c80 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf24a99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf24a99640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2009a9d0 con 0x7fdf20067130 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf24a99640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf24a99640 10 monclient(hunting): authenticate will time out at 899.854438s 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1ed9d640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fdf20137c80 0x7fdf2013a060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:50688/0 (socket says 172.21.15.204:50688) 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdf20067130 0x7fdf2009d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1ed9d640 1 -- 172.21.15.204:0/2768453652 learned_addr learned my addr 172.21.15.204:0/2768453652 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 10 monclient(hunting): get_auth_request con 0x7fdf20067130 auth_method 0 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:44.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:44.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:44.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:44.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:44.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:44.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1d59a640 1 -- 172.21.15.204:0/2768453652 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826590576 0 0) 0x7fdf2009abd0 con 0x7fdf20137c80 2024-09-15T01:34:44.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 10 monclient(hunting): handle_auth_done global_id 5297 payload 274 2024-09-15T01:34:44.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 1 -- 172.21.15.204:0/2768453652 >> v1:172.21.15.204:6789/0 conn(0x7fdf20137c80 legacy=0x7fdf2013a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:44.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 1 -- 172.21.15.204:0/2768453652 >> v1:172.21.15.178:6789/0 conn(0x7fdf2009c6c0 legacy=0x7fdf2009ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:44.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:44.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 1 monclient: found mon.noname-h 2024-09-15T01:34:44.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:44.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1e59c640 1 -- 172.21.15.204:0/2768453652 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2009b190 con 0x7fdf20067130 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.212+0000 7fdf1d59a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1e59c640 1 --2- 172.21.15.204:0/2768453652 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdf20067130 0x7fdf2009d930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009e90 tx=0x7fdf10009780 comp rx=0 tx=0).ready entity=mon.5 client_cookie=ff6dcb5084cb108f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1d59a640 1 -- 172.21.15.204:0/2768453652 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf1002aea0 con 0x7fdf20067130 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1d59a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1d59a640 10 monclient: dump: 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:44.225 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1d59a640 10 monclient: _finish_auth 0 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf24a99640 5 monclient: authenticate success, global_id 5297 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf24a99640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:44.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1d59a640 1 -- 172.21.15.204:0/2768453652 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1002d050 con 0x7fdf20067130 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf24a99640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1d59a640 1 -- 172.21.15.204:0/2768453652 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf100079f0 con 0x7fdf20067130 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf24a99640 4 set_mon_vals no callback set 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1d59a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.213+0000 7fdf1d59a640 10 monclient: dump: 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:44.227 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.214+0000 7fdf24a99640 10 monclient: shutdown 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.214+0000 7fdf24a99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.214+0000 7fdf24a99640 1 -- 172.21.15.204:0/2768453652 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdf20067130 msgr2=0x7fdf2009d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.214+0000 7fdf24a99640 1 --2- 172.21.15.204:0/2768453652 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdf20067130 0x7fdf2009d930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009e90 tx=0x7fdf10009780 comp rx=0 tx=0).stop 2024-09-15T01:34:44.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.214+0000 7fdf24a99640 1 -- 172.21.15.204:0/2768453652 shutdown_connections 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.214+0000 7fdf24a99640 1 --2- 172.21.15.204:0/2768453652 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdf20067130 0x7fdf2009d930 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.214+0000 7fdf24a99640 1 -- 172.21.15.204:0/2768453652 >> 172.21.15.204:0/2768453652 conn(0x7fdf200a2cb0 msgr2=0x7fdf200a30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.214+0000 7fdf24a99640 1 -- 172.21.15.204:0/2768453652 shutdown_connections 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.214+0000 7fdf24a99640 1 -- 172.21.15.204:0/2768453652 wait complete. 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.214+0000 7fdf24a99640 10 monclient: build_initial_monmap 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.214+0000 7fdf24a99640 10 monclient: monmap: 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:44.229 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 1 Processor -- start 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 1 -- start start 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 10 monclient: init 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:44.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 10 monclient: _add_conns ranks=[7,6,3,2,8,0,1,4,5] 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdf20067130 0x7fdf200991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 10 monclient(hunting): picked mon.noname-i con 0x7fdf20067130 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdf2009c6c0 0x7fdf2006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 10 monclient(hunting): picked mon.noname-d con 0x7fdf2009c6c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf20137c80 0x7fdf2006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 10 monclient(hunting): picked mon.noname-g con 0x7fdf20137c80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2006d780 con 0x7fdf20137c80 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2009a530 con 0x7fdf2009c6c0 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2009a190 con 0x7fdf20067130 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:44.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf24a99640 10 monclient(hunting): authenticate will time out at 899.857604s 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1e59c640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdf20067130 msgr2=0x7fdf200991f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1e59c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdf20067130 0x7fdf200991f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1dd9b640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdf2009c6c0 msgr2=0x7fdf2006b880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1dd9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdf2009c6c0 0x7fdf2006b880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf20137c80 0x7fdf2006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf20137c80 0x7fdf2006bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48644/0 (socket says 172.21.15.204:48644) 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 1 -- 172.21.15.204:0/1458259737 learned_addr learned my addr 172.21.15.204:0/1458259737 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 10 monclient(hunting): get_auth_request con 0x7fdf20137c80 auth_method 0 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:44.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:44.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 10 monclient(hunting): handle_auth_done global_id 14430 payload 824 2024-09-15T01:34:44.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 1 -- 172.21.15.204:0/1458259737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdf20067130 msgr2=0x7fdf200991f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:44.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 1 --2- 172.21.15.204:0/1458259737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdf20067130 0x7fdf200991f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:44.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 1 -- 172.21.15.204:0/1458259737 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdf2009c6c0 msgr2=0x7fdf2006b880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:44.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 1 --2- 172.21.15.204:0/1458259737 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdf2009c6c0 0x7fdf2006b880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:44.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:34:44.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.215+0000 7fdf1ed9d640 1 monclient: found mon.noname-g 2024-09-15T01:34:44.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf1ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:44.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf1ed9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:44.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf1ed9d640 1 -- 172.21.15.204:0/1458259737 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf10009a20 con 0x7fdf20137c80 2024-09-15T01:34:44.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf1ed9d640 1 --2- 172.21.15.204:0/1458259737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf20137c80 0x7fdf2006bdc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdf2006d290 tx=0x7fdf18007ec0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:44.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf167fc640 1 -- 172.21.15.204:0/1458259737 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf180070d0 con 0x7fdf20137c80 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf167fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf167fc640 10 monclient: dump: 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:44.234 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf167fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf167fc640 1 -- 172.21.15.204:0/1458259737 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf18007250 con 0x7fdf20137c80 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf24a99640 5 monclient: authenticate success, global_id 14430 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf167fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf167fc640 1 -- 172.21.15.204:0/1458259737 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf18011360 con 0x7fdf20137c80 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf24a99640 10 monclient: _renew_subs 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf24a99640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:44.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf24a99640 1 -- 172.21.15.204:0/1458259737 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2006e950 con 0x7fdf20137c80 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf167fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf167fc640 10 monclient: dump: 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:44.236 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:44.237 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:44.237 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:44.237 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:44.237 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:44.237 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:44.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf1cd99640 4 set_mon_vals no callback set 2024-09-15T01:34:44.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf24a99640 10 monclient: _renew_subs 2024-09-15T01:34:44.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf24a99640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:44.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.216+0000 7fdf24a99640 1 -- 172.21.15.204:0/1458259737 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2006cbd0 con 0x7fdf20137c80 2024-09-15T01:34:44.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.217+0000 7fdf167fc640 1 -- 172.21.15.204:0/1458259737 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdf180073c0 con 0x7fdf20137c80 2024-09-15T01:34:44.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.217+0000 7fdf24a99640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:44.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.217+0000 7fdf24a99640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:44.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.217+0000 7fdf24a99640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:44.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.217+0000 7fdf24a99640 1 -- 172.21.15.204:0/1458259737 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2006eb90 con 0x7fdf20137c80 2024-09-15T01:34:44.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.217+0000 7fdf167fc640 1 --2- 172.21.15.204:0/1458259737 >> v2:172.21.15.204:6812/28210 conn(0x7fdf000397b0 0x7fdf0003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:44.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.217+0000 7fdf167fc640 1 -- 172.21.15.204:0/1458259737 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fdf1804a590 con 0x7fdf20137c80 2024-09-15T01:34:44.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.217+0000 7fdf1e59c640 1 --2- 172.21.15.204:0/1458259737 >> v2:172.21.15.204:6812/28210 conn(0x7fdf000397b0 0x7fdf0003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:44.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.217+0000 7fdf1e59c640 10 monclient: get_auth_request con 0x7fdf000397b0 auth_method 0 2024-09-15T01:34:44.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.218+0000 7fdf1e59c640 1 --2- 172.21.15.204:0/1458259737 >> v2:172.21.15.204:6812/28210 conn(0x7fdf000397b0 0x7fdf0003bc60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdf10034000 tx=0x7fdf10037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:44.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.222+0000 7fdf167fc640 1 -- 172.21.15.204:0/1458259737 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fdf1800e850 con 0x7fdf20137c80 2024-09-15T01:34:44.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.222+0000 7fdf167fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:44.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.222+0000 7fdf167fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:44.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.380+0000 7fdf24a99640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host2"}] 2024-09-15T01:34:44.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.380+0000 7fdf24a99640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-09-15T01:34:44.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.380+0000 7fdf24a99640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:44.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.380+0000 7fdf24a99640 1 -- 172.21.15.204:0/1458259737 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush unlink", "name": "host2"} v 0) v1 -- 0x7fdf20070b80 con 0x7fdf20137c80 2024-09-15T01:34:44.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.604+0000 7fdf167fc640 1 -- 172.21.15.204:0/1458259737 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host2"}]=0 removed item id -16 name 'host2' from crush map v82) v1 ==== 128+0+0 (secure 0 0 0) 0x7fdf18017090 con 0x7fdf20137c80 2024-09-15T01:34:44.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.604+0000 7fdf167fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-09-15T01:34:44.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.604+0000 7fdf167fc640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'host2' from crush map 2024-09-15T01:34:44.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.702+0000 7fdf24a99640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host2"}] 2024-09-15T01:34:44.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.702+0000 7fdf24a99640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-09-15T01:34:44.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.702+0000 7fdf24a99640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:44.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:44.702+0000 7fdf24a99640 1 -- 172.21.15.204:0/1458259737 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush unlink", "name": "host2"} v 0) v1 -- 0x7fdf200a2a70 con 0x7fdf20137c80 2024-09-15T01:34:45.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.614+0000 7fdf167fc640 1 -- 172.21.15.204:0/1458259737 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host2"}]=0 removed item id -16 name 'host2' from crush map v83) v1 ==== 128+0+0 (secure 0 0 0) 0x7fdf1800e290 con 0x7fdf20137c80 2024-09-15T01:34:45.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.614+0000 7fdf167fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-09-15T01:34:45.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.614+0000 7fdf167fc640 10 monclient: _finish_command 3 = system:0 removed item id -16 name 'host2' from crush map 2024-09-15T01:34:45.616 INFO:tasks.workunit.client.0.smithi204.stderr:removed item id -16 name 'host2' from crush map 2024-09-15T01:34:45.618 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 1 -- 172.21.15.204:0/1458259737 >> v2:172.21.15.204:6812/28210 conn(0x7fdf000397b0 msgr2=0x7fdf0003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:45.618 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 1 --2- 172.21.15.204:0/1458259737 >> v2:172.21.15.204:6812/28210 conn(0x7fdf000397b0 0x7fdf0003bc60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdf10034000 tx=0x7fdf10037000 comp rx=0 tx=0).stop 2024-09-15T01:34:45.618 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 10 monclient: shutdown 2024-09-15T01:34:45.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:45.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 1 -- 172.21.15.204:0/1458259737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf20137c80 msgr2=0x7fdf2006bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:45.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 1 --2- 172.21.15.204:0/1458259737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf20137c80 0x7fdf2006bdc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdf2006d290 tx=0x7fdf18007ec0 comp rx=0 tx=0).stop 2024-09-15T01:34:45.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 1 -- 172.21.15.204:0/1458259737 shutdown_connections 2024-09-15T01:34:45.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 1 --2- 172.21.15.204:0/1458259737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf20137c80 0x7fdf2006bdc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:45.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 1 --2- 172.21.15.204:0/1458259737 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdf2009c6c0 0x7fdf2006b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:45.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 1 --2- 172.21.15.204:0/1458259737 >> v2:172.21.15.204:6812/28210 conn(0x7fdf000397b0 0x7fdf0003bc60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:45.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 1 --2- 172.21.15.204:0/1458259737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdf20067130 0x7fdf200991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:45.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 1 -- 172.21.15.204:0/1458259737 >> 172.21.15.204:0/1458259737 conn(0x7fdf200a2cb0 msgr2=0x7fdf200a3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:45.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 1 -- 172.21.15.204:0/1458259737 shutdown_connections 2024-09-15T01:34:45.620 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.616+0000 7fdf24a99640 1 -- 172.21.15.204:0/1458259737 wait complete. 2024-09-15T01:34:45.629 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket host-for-test host root=root-for-test rack=rack-for-test 2024-09-15T01:34:45.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.709+0000 7f53fd35d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:45.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.709+0000 7f53fd35d640 10 monclient: build_initial_monmap 2024-09-15T01:34:45.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.709+0000 7f53fd35d640 10 monclient: monmap: 2024-09-15T01:34:45.711 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:45.711 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:45.711 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:45.710211+0000 2024-09-15T01:34:45.711 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:45.710211+0000 2024-09-15T01:34:45.712 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:45.712 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:45.712 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:45.712 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:45.712 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:45.712 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:45.712 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:45.712 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:45.712 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:45.712 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:45.712 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:45.712 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:45.713 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:45.713 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:45.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 1 Processor -- start 2024-09-15T01:34:45.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 1 -- start start 2024-09-15T01:34:45.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 10 monclient: init 2024-09-15T01:34:45.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:45.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 10 monclient: _add_conns ranks=[5,3,10,4,7,8,1,9,0,6,2] 2024-09-15T01:34:45.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f53f80a2cd0 0x7f53f8132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:45.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 10 monclient(hunting): picked mon.noname-h con 0x7f53f80a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:45.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f809fa20 0x7f53f809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:45.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 10 monclient(hunting): picked mon.noname-g con 0x7f53f809fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:45.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f53f80a08e0 0x7f53f80a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:45.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 10 monclient(hunting): picked mon.noname-f con 0x7f53f80a08e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:45.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:45.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f80a30a0 con 0x7f53f80a08e0 2024-09-15T01:34:45.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:45.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80a0330 con 0x7f53f809fa20 2024-09-15T01:34:45.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:45.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53f80a1db0 con 0x7f53f80a2cd0 2024-09-15T01:34:45.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:45.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.710+0000 7f53fd35d640 10 monclient(hunting): authenticate will time out at 901.352886s 2024-09-15T01:34:45.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53ee7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f809fa20 0x7f53f809fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:45.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f53f80a08e0 0x7f53f80a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:45.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f6ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f53f80a2cd0 0x7f53f8132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:45.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f53f80a08e0 0x7f53f80a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:35512/0 (socket says 172.21.15.204:35512) 2024-09-15T01:34:45.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f6ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f53f80a2cd0 0x7f53f8132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:37734/0 (socket says 172.21.15.204:37734) 2024-09-15T01:34:45.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 1 -- 172.21.15.204:0/1128706337 learned_addr learned my addr 172.21.15.204:0/1128706337 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:45.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 10 monclient(hunting): get_auth_request con 0x7f53f80a08e0 auth_method 0 2024-09-15T01:34:45.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:45.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:45.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:45.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f6ffd640 10 monclient(hunting): get_auth_request con 0x7f53f80a2cd0 auth_method 0 2024-09-15T01:34:45.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:45.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:45.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:45.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53ee7fc640 10 monclient(hunting): get_auth_request con 0x7f53f809fa20 auth_method 0 2024-09-15T01:34:45.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:45.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:45.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:45.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:45.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:45.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:45.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53ee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:45.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53ee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:45.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53ee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:45.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:45.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:45.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:45.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 10 monclient(hunting): handle_auth_done global_id 5797 payload 274 2024-09-15T01:34:45.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 1 -- 172.21.15.204:0/1128706337 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f53f80a2cd0 msgr2=0x7f53f8132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:45.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 1 --2- 172.21.15.204:0/1128706337 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f53f80a2cd0 0x7f53f8132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:45.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 1 -- 172.21.15.204:0/1128706337 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f809fa20 msgr2=0x7f53f809fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:45.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 1 --2- 172.21.15.204:0/1128706337 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f809fa20 0x7f53f809fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:45.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:45.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 1 monclient: found mon.noname-f 2024-09-15T01:34:45.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:45.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:45.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53f77fe640 1 -- 172.21.15.204:0/1128706337 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f80648a0 con 0x7f53f80a08e0 2024-09-15T01:34:45.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.711+0000 7f53ee7fc640 1 --2- 172.21.15.204:0/1128706337 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f809fa20 0x7f53f809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:45.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f77fe640 1 --2- 172.21.15.204:0/1128706337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f80a08e0 0x7f53f80a0d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009f20 tx=0x7f53f0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=974b5188c6811356 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:45.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f67fc640 1 -- 172.21.15.204:0/1128706337 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53f0007880 con 0x7f53f80a08e0 2024-09-15T01:34:45.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:45.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f67fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:45.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f67fc640 10 monclient: dump: 2024-09-15T01:34:45.727 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:45.727 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:45.727 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:45.727 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:45.728 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:45.728 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:45.728 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:45.728 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:45.728 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:45.728 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:45.728 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:45.728 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:45.728 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:45.728 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:45.728 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:45.728 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:45.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f67fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:45.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:45.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 5 monclient: authenticate success, global_id 5797 2024-09-15T01:34:45.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:45.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f67fc640 1 -- 172.21.15.204:0/1128706337 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f002d050 con 0x7f53f80a08e0 2024-09-15T01:34:45.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:45.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:45.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 4 set_mon_vals no callback set 2024-09-15T01:34:45.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f67fc640 1 -- 172.21.15.204:0/1128706337 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53f0030360 con 0x7f53f80a08e0 2024-09-15T01:34:45.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:45.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f67fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:45.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53f67fc640 10 monclient: dump: 2024-09-15T01:34:45.730 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:45.730 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:45.730 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:45.730 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:45.730 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:45.730 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:45.730 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:45.730 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:45.730 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:45.730 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:45.730 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:45.731 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:45.731 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:45.731 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:45.731 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:45.731 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:45.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 10 monclient: shutdown 2024-09-15T01:34:45.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:45.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 1 -- 172.21.15.204:0/1128706337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f80a08e0 msgr2=0x7f53f80a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:45.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 1 --2- 172.21.15.204:0/1128706337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f80a08e0 0x7f53f80a0d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009f20 tx=0x7f53f0009840 comp rx=0 tx=0).stop 2024-09-15T01:34:45.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 1 -- 172.21.15.204:0/1128706337 shutdown_connections 2024-09-15T01:34:45.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 1 --2- 172.21.15.204:0/1128706337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f80a08e0 0x7f53f80a0d50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:45.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 1 --2- 172.21.15.204:0/1128706337 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f809fa20 0x7f53f809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:45.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 1 --2- 172.21.15.204:0/1128706337 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f53f80a2cd0 0x7f53f8132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:45.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 1 -- 172.21.15.204:0/1128706337 >> 172.21.15.204:0/1128706337 conn(0x7f53f806e860 msgr2=0x7f53f806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:45.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 1 -- 172.21.15.204:0/1128706337 shutdown_connections 2024-09-15T01:34:45.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.712+0000 7f53fd35d640 1 -- 172.21.15.204:0/1128706337 wait complete. 2024-09-15T01:34:45.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.713+0000 7f53fd35d640 10 monclient: build_initial_monmap 2024-09-15T01:34:45.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.713+0000 7f53fd35d640 10 monclient: monmap: 2024-09-15T01:34:45.732 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:45.732 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:45.732 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:45.732 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:45.732 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:45.732 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:45.733 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:45.733 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:45.733 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:45.733 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:45.733 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:45.733 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:45.733 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:45.733 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:45.733 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:45.733 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.713+0000 7f53fd35d640 1 Processor -- start 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.713+0000 7f53fd35d640 1 -- start start 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.713+0000 7f53fd35d640 10 monclient: init 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 10 monclient: _add_conns ranks=[8,7,4,2,3,6,0,5,1] 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f53f809fa20 0x7f53f8130ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 10 monclient(hunting): picked mon.noname-e con 0x7f53f809fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53f80a08e0 0x7f53f811d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 10 monclient(hunting): picked mon.noname-i con 0x7f53f80a08e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f53f80a2cd0 0x7f53f811daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 10 monclient(hunting): picked mon.noname-c con 0x7f53f80a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:45.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53f8120c00 con 0x7f53f80a2cd0 2024-09-15T01:34:45.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:45.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f53f809d3d0 con 0x7f53f80a08e0 2024-09-15T01:34:45.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:45.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f53f809cb60 con 0x7f53f809fa20 2024-09-15T01:34:45.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:45.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53fd35d640 10 monclient(hunting): authenticate will time out at 901.356468s 2024-09-15T01:34:45.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53ee7fc640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53f80a08e0 msgr2=0x7f53f811d5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:45.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53ee7fc640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53f80a08e0 0x7f53f811d5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:45.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f6ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f53f809fa20 0x7f53f8130ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:45.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f77fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f53f80a2cd0 0x7f53f811daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:45.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f6ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f53f809fa20 0x7f53f8130ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:45696/0 (socket says 172.21.15.204:45696) 2024-09-15T01:34:45.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f77fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f53f80a2cd0 0x7f53f811daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:46280/0 (socket says 172.21.15.204:46280) 2024-09-15T01:34:45.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f6ffd640 1 -- 172.21.15.204:0/3981405149 learned_addr learned my addr 172.21.15.204:0/3981405149 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:45.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f6ffd640 10 monclient(hunting): get_auth_request con 0x7f53f809fa20 auth_method 0 2024-09-15T01:34:45.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:45.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:45.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:45.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f77fe640 10 monclient(hunting): get_auth_request con 0x7f53f80a2cd0 auth_method 0 2024-09-15T01:34:45.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:45.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:45.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:45.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:45.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:45.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:45.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:45.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:45.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.714+0000 7f53f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:45.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f6ffd640 10 monclient(hunting): handle_auth_done global_id 5354 payload 824 2024-09-15T01:34:45.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f6ffd640 1 -- 172.21.15.204:0/3981405149 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53f80a08e0 msgr2=0x7f53f811d5b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:45.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f6ffd640 1 --2- 172.21.15.204:0/3981405149 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53f80a08e0 0x7f53f811d5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:45.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f6ffd640 1 -- 172.21.15.204:0/3981405149 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f53f80a2cd0 msgr2=0x7f53f811daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:45.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f6ffd640 1 --2- 172.21.15.204:0/3981405149 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f53f80a2cd0 0x7f53f811daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:45.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:45.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f6ffd640 1 monclient: found mon.noname-e 2024-09-15T01:34:45.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:45.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f6ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:45.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f6ffd640 1 -- 172.21.15.204:0/3981405149 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e0009390 con 0x7f53f809fa20 2024-09-15T01:34:45.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f77fe640 1 --2- 172.21.15.204:0/3981405149 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f53f80a2cd0 0x7f53f811daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:45.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f6ffd640 1 --2- 172.21.15.204:0/3981405149 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f53f809fa20 0x7f53f8130ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f53e800ea50 tx=0x7f53e80090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:45.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53ef7fe640 1 -- 172.21.15.204:0/3981405149 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53e800ce90 con 0x7f53f809fa20 2024-09-15T01:34:45.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:45.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53ef7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:45.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53ef7fe640 10 monclient: dump: 2024-09-15T01:34:45.738 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:45.739 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:45.739 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:45.739 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:45.739 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:45.739 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:45.739 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:45.739 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:45.739 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:45.739 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:45.739 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:45.739 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:45.739 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:45.740 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:45.740 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:45.740 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:45.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53ef7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:45.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:45.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53ef7fe640 1 -- 172.21.15.204:0/3981405149 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e8009600 con 0x7f53f809fa20 2024-09-15T01:34:45.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53fd35d640 5 monclient: authenticate success, global_id 5354 2024-09-15T01:34:45.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53ef7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:45.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53ef7fe640 1 -- 172.21.15.204:0/3981405149 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53e80138f0 con 0x7f53f809fa20 2024-09-15T01:34:45.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53fd35d640 10 monclient: _renew_subs 2024-09-15T01:34:45.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53fd35d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:45.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53fd35d640 1 -- 172.21.15.204:0/3981405149 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f0009ae0 con 0x7f53f809fa20 2024-09-15T01:34:45.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:45.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53ef7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:45.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53f5ffb640 4 set_mon_vals no callback set 2024-09-15T01:34:45.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53ef7fe640 10 monclient: dump: 2024-09-15T01:34:45.741 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:45.741 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:45.741 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:45.741 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:45.741 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:45.741 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:45.742 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:45.742 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:45.742 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:45.742 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:45.742 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:45.742 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:45.742 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:45.742 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:45.742 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:45.742 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:45.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53fd35d640 10 monclient: _renew_subs 2024-09-15T01:34:45.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53fd35d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:45.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.715+0000 7f53fd35d640 1 -- 172.21.15.204:0/3981405149 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f811f890 con 0x7f53f809fa20 2024-09-15T01:34:45.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.716+0000 7f53ef7fe640 1 -- 172.21.15.204:0/3981405149 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f53e8013470 con 0x7f53f809fa20 2024-09-15T01:34:45.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.716+0000 7f53fd35d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:45.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.716+0000 7f53fd35d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:45.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.716+0000 7f53fd35d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:45.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.716+0000 7f53fd35d640 1 -- 172.21.15.204:0/3981405149 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f8070b80 con 0x7f53f809fa20 2024-09-15T01:34:45.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.716+0000 7f53ef7fe640 1 --2- 172.21.15.204:0/3981405149 >> v2:172.21.15.204:6812/28210 conn(0x7f53d80397b0 0x7f53d803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:45.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.716+0000 7f53ef7fe640 1 -- 172.21.15.204:0/3981405149 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5129+0+0 (secure 0 0 0) 0x7f53e804b860 con 0x7f53f809fa20 2024-09-15T01:34:45.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.716+0000 7f53ee7fc640 1 --2- 172.21.15.204:0/3981405149 >> v2:172.21.15.204:6812/28210 conn(0x7f53d80397b0 0x7f53d803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:45.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.716+0000 7f53ee7fc640 10 monclient: get_auth_request con 0x7f53d80397b0 auth_method 0 2024-09-15T01:34:45.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.717+0000 7f53ee7fc640 1 --2- 172.21.15.204:0/3981405149 >> v2:172.21.15.204:6812/28210 conn(0x7f53d80397b0 0x7f53d803bc60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009540 tx=0x7f53e00090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:45.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.721+0000 7f53ef7fe640 1 -- 172.21.15.204:0/3981405149 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f53e80194f0 con 0x7f53f809fa20 2024-09-15T01:34:45.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.721+0000 7f53ef7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:45.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.721+0000 7f53ef7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.880+0000 7f53fd35d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-09-15T01:34:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.880+0000 7f53fd35d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-09-15T01:34:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.880+0000 7f53fd35d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:45.880+0000 7f53fd35d640 1 -- 172.21.15.204:0/3981405149 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]} v 0) v1 -- 0x7f53f80008d0 con 0x7f53f809fa20 2024-09-15T01:34:46.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.626+0000 7f53ef7fe640 1 -- 172.21.15.204:0/3981405149 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}]=0 added bucket host-for-test type host to location {rack=rack-for-test,root=root-for-test} v84) v1 ==== 251+0+0 (secure 0 0 0) 0x7f53e801bbf0 con 0x7f53f809fa20 2024-09-15T01:34:46.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.626+0000 7f53ef7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-09-15T01:34:46.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.626+0000 7f53ef7fe640 10 monclient: _finish_command 2 = system:0 added bucket host-for-test type host to location {rack=rack-for-test,root=root-for-test} 2024-09-15T01:34:46.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.723+0000 7f53fd35d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-09-15T01:34:46.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.723+0000 7f53fd35d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-09-15T01:34:46.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.723+0000 7f53fd35d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:46.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.723+0000 7f53fd35d640 1 -- 172.21.15.204:0/3981405149 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]} v 0) v1 -- 0x7f53f8067450 con 0x7f53f809fa20 2024-09-15T01:34:46.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.725+0000 7f53ef7fe640 1 -- 172.21.15.204:0/3981405149 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}]=0 bucket 'host-for-test' already exists v84) v1 ==== 200+0+0 (secure 0 0 0) 0x7f53e801bdd0 con 0x7f53f809fa20 2024-09-15T01:34:46.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.725+0000 7f53ef7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-09-15T01:34:46.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.725+0000 7f53ef7fe640 10 monclient: _finish_command 3 = system:0 bucket 'host-for-test' already exists 2024-09-15T01:34:46.726 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'host-for-test' already exists 2024-09-15T01:34:46.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.726+0000 7f53fd35d640 1 -- 172.21.15.204:0/3981405149 >> v2:172.21.15.204:6812/28210 conn(0x7f53d80397b0 msgr2=0x7f53d803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.726+0000 7f53fd35d640 1 --2- 172.21.15.204:0/3981405149 >> v2:172.21.15.204:6812/28210 conn(0x7f53d80397b0 0x7f53d803bc60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009540 tx=0x7f53e00090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.726+0000 7f53fd35d640 10 monclient: shutdown 2024-09-15T01:34:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.726+0000 7f53fd35d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.726+0000 7f53fd35d640 1 -- 172.21.15.204:0/3981405149 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f53f809fa20 msgr2=0x7f53f8130ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.726+0000 7f53fd35d640 1 --2- 172.21.15.204:0/3981405149 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f53f809fa20 0x7f53f8130ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f53e800ea50 tx=0x7f53e80090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.727+0000 7f53fd35d640 1 -- 172.21.15.204:0/3981405149 shutdown_connections 2024-09-15T01:34:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.727+0000 7f53fd35d640 1 --2- 172.21.15.204:0/3981405149 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f53f80a2cd0 0x7f53f811daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.727+0000 7f53fd35d640 1 --2- 172.21.15.204:0/3981405149 >> v2:172.21.15.204:6812/28210 conn(0x7f53d80397b0 0x7f53d803bc60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.727+0000 7f53fd35d640 1 --2- 172.21.15.204:0/3981405149 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f53f80a08e0 0x7f53f811d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.727+0000 7f53fd35d640 1 --2- 172.21.15.204:0/3981405149 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f53f809fa20 0x7f53f8130ab0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.727+0000 7f53fd35d640 1 -- 172.21.15.204:0/3981405149 >> 172.21.15.204:0/3981405149 conn(0x7f53f806e860 msgr2=0x7f53f80a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.727+0000 7f53fd35d640 1 -- 172.21.15.204:0/3981405149 shutdown_connections 2024-09-15T01:34:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.727+0000 7f53fd35d640 1 -- 172.21.15.204:0/3981405149 wait complete. 2024-09-15T01:34:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:34:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep host-for-test 2024-09-15T01:34:46.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.808+0000 7f6c5d95d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:46.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.808+0000 7f6c5d95d640 10 monclient: build_initial_monmap 2024-09-15T01:34:46.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.808+0000 7f6c5d95d640 10 monclient: monmap: 2024-09-15T01:34:46.809 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:46.809 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:46.809 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:46.808527+0000 2024-09-15T01:34:46.809 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:46.808527+0000 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:46.810 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:46.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.808+0000 7f6c5d95d640 1 Processor -- start 2024-09-15T01:34:46.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.808+0000 7f6c5d95d640 1 -- start start 2024-09-15T01:34:46.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.808+0000 7f6c5d95d640 10 monclient: init 2024-09-15T01:34:46.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:46.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 10 monclient: _add_conns ranks=[8,9,4,5,0,3,6,7,1,10,2] 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6c580a2400 0x7f6c580a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 10 monclient(hunting): picked mon.noname-e con 0x7f6c580a2400 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6c5809f990 0x7f6c5809fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 10 monclient(hunting): picked mon.noname-a con 0x7f6c5809f990 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c58132930 0x7f6c58132d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 10 monclient(hunting): picked mon.noname-c con 0x7f6c58132930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5809e1a0 con 0x7f6c5809f990 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580a27d0 con 0x7f6c58132930 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5809dec0 con 0x7f6c580a2400 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:46.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c5d95d640 10 monclient(hunting): authenticate will time out at 902.451155s 2024-09-15T01:34:46.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c567fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6c5809f990 0x7f6c5809fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:46.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6c580a2400 0x7f6c580a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:46.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c567fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6c5809f990 0x7f6c5809fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:54686/0 (socket says 172.21.15.204:54686) 2024-09-15T01:34:46.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6c580a2400 0x7f6c580a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:45700/0 (socket says 172.21.15.204:45700) 2024-09-15T01:34:46.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c567fc640 1 -- 172.21.15.204:0/1364701028 learned_addr learned my addr 172.21.15.204:0/1364701028 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:46.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c58132930 0x7f6c58132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:46.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c567fc640 10 monclient(hunting): get_auth_request con 0x7f6c5809f990 auth_method 0 2024-09-15T01:34:46.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:46.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c567fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:46.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c567fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:46.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 10 monclient(hunting): get_auth_request con 0x7f6c580a2400 auth_method 0 2024-09-15T01:34:46.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c577fe640 10 monclient(hunting): get_auth_request con 0x7f6c58132930 auth_method 0 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c577fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c567fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c567fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c567fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:46.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 10 monclient(hunting): handle_auth_done global_id 5363 payload 274 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 1 -- 172.21.15.204:0/1364701028 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c58132930 msgr2=0x7f6c58132d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 1 --2- 172.21.15.204:0/1364701028 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c58132930 0x7f6c58132d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 1 -- 172.21.15.204:0/1364701028 >> v2:172.21.15.178:3300/0 conn(0x7f6c5809f990 msgr2=0x7f6c5809fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 1 --2- 172.21.15.204:0/1364701028 >> v2:172.21.15.178:3300/0 conn(0x7f6c5809f990 0x7f6c5809fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 1 monclient: found mon.noname-e 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c56ffd640 1 -- 172.21.15.204:0/1364701028 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580648a0 con 0x7f6c580a2400 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c567fc640 1 --2- 172.21.15.204:0/1364701028 >> v2:172.21.15.178:3300/0 conn(0x7f6c5809f990 0x7f6c5809fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.809+0000 7f6c577fe640 1 --2- 172.21.15.204:0/1364701028 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c58132930 0x7f6c58132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:46.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c56ffd640 1 --2- 172.21.15.204:0/1364701028 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6c580a2400 0x7f6c580a0ae0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009850 tx=0x7f6c480090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9389dbcdb15614c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c55ffb640 1 -- 172.21.15.204:0/1364701028 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c4802abc0 con 0x7f6c580a2400 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c55ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c55ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c55ffb640 10 monclient: dump: 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:46.824 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c55ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c55ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c55ffb640 1 -- 172.21.15.204:0/1364701028 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4802ad40 con 0x7f6c580a2400 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c5d95d640 5 monclient: authenticate success, global_id 5363 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c5d95d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c55ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:46.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c5d95d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c5d95d640 4 set_mon_vals no callback set 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c55ffb640 1 -- 172.21.15.204:0/1364701028 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c48031ca0 con 0x7f6c580a2400 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c55ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c55ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.810+0000 7f6c55ffb640 10 monclient: dump: 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:46.826 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 10 monclient: shutdown 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 1 -- 172.21.15.204:0/1364701028 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6c580a2400 msgr2=0x7f6c580a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 1 --2- 172.21.15.204:0/1364701028 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6c580a2400 0x7f6c580a0ae0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009850 tx=0x7f6c480090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 1 -- 172.21.15.204:0/1364701028 shutdown_connections 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 1 --2- 172.21.15.204:0/1364701028 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c58132930 0x7f6c58132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 1 --2- 172.21.15.204:0/1364701028 >> v2:172.21.15.178:3300/0 conn(0x7f6c5809f990 0x7f6c5809fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 1 --2- 172.21.15.204:0/1364701028 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6c580a2400 0x7f6c580a0ae0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 1 -- 172.21.15.204:0/1364701028 >> 172.21.15.204:0/1364701028 conn(0x7f6c5806e860 msgr2=0x7f6c5806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 1 -- 172.21.15.204:0/1364701028 shutdown_connections 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 1 -- 172.21.15.204:0/1364701028 wait complete. 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 10 monclient: build_initial_monmap 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.811+0000 7f6c5d95d640 10 monclient: monmap: 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:46.828 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 1 Processor -- start 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 1 -- start start 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 10 monclient: init 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 10 monclient: _add_conns ranks=[4,6,0,8,1,2,3,7,5] 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c5809f990 0x7f6c58141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 10 monclient(hunting): picked mon.noname-c con 0x7f6c5809f990 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6c580a2400 0x7f6c58142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:46.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 10 monclient(hunting): picked mon.noname-d con 0x7f6c580a2400 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c58132930 0x7f6c581428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 10 monclient(hunting): picked mon.noname-a con 0x7f6c58132930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c581210f0 con 0x7f6c58132930 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5809cad0 con 0x7f6c5809f990 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6c58120e10 con 0x7f6c580a2400 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c5d95d640 10 monclient(hunting): authenticate will time out at 902.454745s 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c567fc640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6c580a2400 msgr2=0x7f6c58142390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c567fc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6c580a2400 0x7f6c58142390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c56ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c5809f990 0x7f6c58141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c56ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c5809f990 0x7f6c58141e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:46300/0 (socket says 172.21.15.204:46300) 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c56ffd640 1 -- 172.21.15.204:0/544247576 learned_addr learned my addr 172.21.15.204:0/544247576 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c56ffd640 10 monclient(hunting): get_auth_request con 0x7f6c5809f990 auth_method 0 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c56ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c56ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c577fe640 1 --2- 172.21.15.204:0/544247576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c58132930 0x7f6c581428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c577fe640 10 monclient(hunting): get_auth_request con 0x7f6c58132930 auth_method 0 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c577fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.812+0000 7f6c577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:46.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 10 monclient(hunting): handle_auth_done global_id 5314 payload 824 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 1 -- 172.21.15.204:0/544247576 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6c580a2400 msgr2=0x7f6c58142390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 1 --2- 172.21.15.204:0/544247576 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6c580a2400 0x7f6c58142390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 1 -- 172.21.15.204:0/544247576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c58132930 msgr2=0x7f6c581428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 1 --2- 172.21.15.204:0/544247576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c58132930 0x7f6c581428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 1 monclient: found mon.noname-c 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 1 -- 172.21.15.204:0/544247576 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c009450 con 0x7f6c5809f990 2024-09-15T01:34:46.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c56ffd640 1 --2- 172.21.15.204:0/544247576 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c5809f990 0x7f6c58141e50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6c48035040 tx=0x7f6c48032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c3effd640 1 -- 172.21.15.204:0/544247576 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c4803b870 con 0x7f6c5809f990 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c3effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c3effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c3effd640 10 monclient: dump: 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:46.833 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c3effd640 10 monclient: _finish_auth 0 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c3effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c3effd640 1 -- 172.21.15.204:0/544247576 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4803ba10 con 0x7f6c5809f990 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c3effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c3effd640 1 -- 172.21.15.204:0/544247576 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c480313a0 con 0x7f6c5809f990 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c5d95d640 5 monclient: authenticate success, global_id 5314 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c5d95d640 10 monclient: _renew_subs 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c5d95d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:46.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c5d95d640 1 -- 172.21.15.204:0/544247576 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c50009a20 con 0x7f6c5809f990 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c3effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c557fa640 4 set_mon_vals no callback set 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c3effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c3effd640 10 monclient: dump: 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:46.835 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c5d95d640 10 monclient: _renew_subs 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c5d95d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.813+0000 7f6c5d95d640 1 -- 172.21.15.204:0/544247576 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c48009390 con 0x7f6c5809f990 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.814+0000 7f6c3effd640 1 -- 172.21.15.204:0/544247576 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6c48039040 con 0x7f6c5809f990 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.814+0000 7f6c5d95d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.814+0000 7f6c5d95d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:46.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.814+0000 7f6c5d95d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:46.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.814+0000 7f6c5d95d640 1 -- 172.21.15.204:0/544247576 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c58070b80 con 0x7f6c5809f990 2024-09-15T01:34:46.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.814+0000 7f6c3effd640 1 --2- 172.21.15.204:0/544247576 >> v2:172.21.15.204:6812/28210 conn(0x7f6c440397e0 0x7f6c4403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:46.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.814+0000 7f6c3effd640 1 -- 172.21.15.204:0/544247576 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5654+0+0 (secure 0 0 0) 0x7f6c4806af50 con 0x7f6c5809f990 2024-09-15T01:34:46.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.814+0000 7f6c567fc640 1 --2- 172.21.15.204:0/544247576 >> v2:172.21.15.204:6812/28210 conn(0x7f6c440397e0 0x7f6c4403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:46.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.814+0000 7f6c567fc640 10 monclient: get_auth_request con 0x7f6c440397e0 auth_method 0 2024-09-15T01:34:46.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.815+0000 7f6c567fc640 1 --2- 172.21.15.204:0/544247576 >> v2:172.21.15.204:6812/28210 conn(0x7f6c440397e0 0x7f6c4403bc90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c006c10 tx=0x7f6c4c0091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:46.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.819+0000 7f6c3effd640 1 -- 172.21.15.204:0/544247576 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6c48044ae0 con 0x7f6c5809f990 2024-09-15T01:34:46.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.819+0000 7f6c3effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:46.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.819+0000 7f6c3effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:46.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.975+0000 7f6c5d95d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:46.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.975+0000 7f6c5d95d640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:46.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.975+0000 7f6c5d95d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:46.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.975+0000 7f6c5d95d640 1 -- 172.21.15.204:0/544247576 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f6c5809e480 con 0x7f6c5809f990 2024-09-15T01:34:46.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.976+0000 7f6c3effd640 1 -- 172.21.15.204:0/544247576 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v84) v1 ==== 56+0+1500 (secure 0 0 0) 0x7f6c48042020 con 0x7f6c5809f990 2024-09-15T01:34:46.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.976+0000 7f6c3effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:46.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:46.976+0000 7f6c3effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:47.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.068+0000 7f6c5d95d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:47.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.068+0000 7f6c5d95d640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:47.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.068+0000 7f6c5d95d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.068+0000 7f6c5d95d640 1 -- 172.21.15.204:0/544247576 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f6c5811fc70 con 0x7f6c5809f990 2024-09-15T01:34:47.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.069+0000 7f6c3effd640 1 -- 172.21.15.204:0/544247576 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v84) v1 ==== 56+0+1500 (secure 0 0 0) 0x7f6c48044300 con 0x7f6c5809f990 2024-09-15T01:34:47.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.069+0000 7f6c3effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:47.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.069+0000 7f6c3effd640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:47.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 1 -- 172.21.15.204:0/544247576 >> v2:172.21.15.204:6812/28210 conn(0x7f6c440397e0 msgr2=0x7f6c4403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 1 --2- 172.21.15.204:0/544247576 >> v2:172.21.15.204:6812/28210 conn(0x7f6c440397e0 0x7f6c4403bc90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c006c10 tx=0x7f6c4c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 10 monclient: shutdown 2024-09-15T01:34:47.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:47.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 1 -- 172.21.15.204:0/544247576 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c5809f990 msgr2=0x7f6c58141e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 1 --2- 172.21.15.204:0/544247576 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c5809f990 0x7f6c58141e50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6c48035040 tx=0x7f6c48032000 comp rx=0 tx=0).stop 2024-09-15T01:34:47.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 1 -- 172.21.15.204:0/544247576 shutdown_connections 2024-09-15T01:34:47.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 1 --2- 172.21.15.204:0/544247576 >> v2:172.21.15.204:6812/28210 conn(0x7f6c440397e0 0x7f6c4403bc90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 1 --2- 172.21.15.204:0/544247576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c58132930 0x7f6c581428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 1 --2- 172.21.15.204:0/544247576 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6c580a2400 0x7f6c58142390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 1 --2- 172.21.15.204:0/544247576 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6c5809f990 0x7f6c58141e50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 1 -- 172.21.15.204:0/544247576 >> 172.21.15.204:0/544247576 conn(0x7f6c5806e860 msgr2=0x7f6c58131110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:47.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 1 -- 172.21.15.204:0/544247576 shutdown_connections 2024-09-15T01:34:47.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.071+0000 7f6c5d95d640 1 -- 172.21.15.204:0/544247576 wait complete. 2024-09-15T01:34:47.081 INFO:tasks.workunit.client.0.smithi204.stdout:-11 0 host host-for-test 2024-09-15T01:34:47.081 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:34:47.082 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep rack-for-test 2024-09-15T01:34:47.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.151+0000 7ff75cedd640 10 monclient: get_monmap_and_config 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.151+0000 7ff75cedd640 10 monclient: build_initial_monmap 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.152+0000 7ff75cedd640 10 monclient: monmap: 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:47.152455+0000 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:47.152455+0000 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:47.154 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:47.155 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:47.155 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:47.155 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:47.155 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:47.155 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:47.155 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:47.155 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.152+0000 7ff75cedd640 1 Processor -- start 2024-09-15T01:34:47.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.152+0000 7ff75cedd640 1 -- start start 2024-09-15T01:34:47.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.152+0000 7ff75cedd640 10 monclient: init 2024-09-15T01:34:47.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:47.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 10 monclient: _add_conns ranks=[5,2,4,6,7,10,8,0,3,9,1] 2024-09-15T01:34:47.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff7580a2400 0x7ff7580a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 10 monclient(hunting): picked mon.noname-h con 0x7ff7580a2400 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:47.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75809f990 0x7ff75809fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 10 monclient(hunting): picked mon.noname-b con 0x7ff75809f990 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:47.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff758132930 0x7ff758132d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 10 monclient(hunting): picked mon.noname-c con 0x7ff758132930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:47.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75809e1a0 con 0x7ff75809f990 2024-09-15T01:34:47.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580a27d0 con 0x7ff758132930 2024-09-15T01:34:47.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff75809dec0 con 0x7ff7580a2400 2024-09-15T01:34:47.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:47.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75cedd640 10 monclient(hunting): authenticate will time out at 902.795311s 2024-09-15T01:34:47.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff755d9b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75809f990 msgr2=0x7ff75809fd60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:47.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff7580a2400 0x7ff7580a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff755d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75809f990 0x7ff75809fd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:47.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff7580a2400 0x7ff7580a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:53374/0 (socket says 172.21.15.204:53374) 2024-09-15T01:34:47.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 1 -- 172.21.15.204:0/2570227724 learned_addr learned my addr 172.21.15.204:0/2570227724 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:47.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 10 monclient(hunting): get_auth_request con 0x7ff7580a2400 auth_method 0 2024-09-15T01:34:47.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:47.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:47.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff756d9d640 1 --2- 172.21.15.204:0/2570227724 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff758132930 0x7ff758132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 10 monclient(hunting): handle_auth_done global_id 5315 payload 274 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 1 -- 172.21.15.204:0/2570227724 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff758132930 msgr2=0x7ff758132d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 1 --2- 172.21.15.204:0/2570227724 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff758132930 0x7ff758132d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 1 -- 172.21.15.204:0/2570227724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75809f990 msgr2=0x7ff75809fd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 1 --2- 172.21.15.204:0/2570227724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75809f990 0x7ff75809fd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 1 monclient: found mon.noname-h 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:47.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff75659c640 1 -- 172.21.15.204:0/2570227724 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7580648a0 con 0x7ff7580a2400 2024-09-15T01:34:47.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff756d9d640 10 monclient: get_auth_request con 0x7ff758132930 auth_method 0 2024-09-15T01:34:47.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.153+0000 7ff756d9d640 1 --2- 172.21.15.204:0/2570227724 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff758132930 0x7ff758132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:34:47.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75659c640 1 --2- 172.21.15.204:0/2570227724 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff7580a2400 0x7ff7580a0ae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff748009940 tx=0x7ff7480091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d919b2ebe7ffeb66 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:47.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75559a640 1 -- 172.21.15.204:0/2570227724 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff74802ac80 con 0x7ff7580a2400 2024-09-15T01:34:47.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:47.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75559a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:47.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75559a640 10 monclient: dump: 2024-09-15T01:34:47.167 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:47.167 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.167 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.167 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.167 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75559a640 10 monclient: _finish_auth 0 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:47.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75cedd640 5 monclient: authenticate success, global_id 5315 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75559a640 1 -- 172.21.15.204:0/2570227724 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74802ae00 con 0x7ff7580a2400 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75cedd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75559a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75cedd640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75cedd640 4 set_mon_vals no callback set 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75559a640 1 -- 172.21.15.204:0/2570227724 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff748031ca0 con 0x7ff7580a2400 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75559a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.154+0000 7ff75559a640 10 monclient: dump: 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.169 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 10 monclient: shutdown 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:47.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 1 -- 172.21.15.204:0/2570227724 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff7580a2400 msgr2=0x7ff7580a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 1 --2- 172.21.15.204:0/2570227724 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff7580a2400 0x7ff7580a0ae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff748009940 tx=0x7ff7480091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 1 -- 172.21.15.204:0/2570227724 shutdown_connections 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 1 --2- 172.21.15.204:0/2570227724 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff758132930 0x7ff758132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 1 --2- 172.21.15.204:0/2570227724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff75809f990 0x7ff75809fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 1 --2- 172.21.15.204:0/2570227724 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff7580a2400 0x7ff7580a0ae0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 1 -- 172.21.15.204:0/2570227724 >> 172.21.15.204:0/2570227724 conn(0x7ff75806e860 msgr2=0x7ff75806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 1 -- 172.21.15.204:0/2570227724 shutdown_connections 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 1 -- 172.21.15.204:0/2570227724 wait complete. 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 10 monclient: build_initial_monmap 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.155+0000 7ff75cedd640 10 monclient: monmap: 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:47.171 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 1 Processor -- start 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 1 -- start start 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 10 monclient: init 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 10 monclient: _add_conns ranks=[4,8,6,3,7,0,5,2,1] 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff75809f990 0x7ff758121a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 10 monclient(hunting): picked mon.noname-c con 0x7ff75809f990 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff7580a2400 0x7ff75812f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 10 monclient(hunting): picked mon.noname-e con 0x7ff7580a2400 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff758132930 0x7ff75811d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 10 monclient(hunting): picked mon.noname-d con 0x7ff758132930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff75809d3d0 con 0x7ff75809f990 2024-09-15T01:34:47.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff75809cad0 con 0x7ff758132930 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff75811ecc0 con 0x7ff7580a2400 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75cedd640 10 monclient(hunting): authenticate will time out at 902.798513s 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff756d9d640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff758132930 msgr2=0x7ff75811d180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff756d9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff758132930 0x7ff75811d180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff75809f990 0x7ff758121a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff75809f990 0x7ff758121a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:36962/0 (socket says 172.21.15.204:36962) 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75659c640 1 -- 172.21.15.204:0/1995339640 learned_addr learned my addr 172.21.15.204:0/1995339640 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75659c640 10 monclient(hunting): get_auth_request con 0x7ff75809f990 auth_method 0 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:47.174 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff755d9b640 1 --2- 172.21.15.204:0/1995339640 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff7580a2400 0x7ff75812f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff755d9b640 10 monclient(hunting): get_auth_request con 0x7ff7580a2400 auth_method 0 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff755d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff755d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff755d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.156+0000 7ff75659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff755d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff755d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff755d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75659c640 10 monclient(hunting): handle_auth_done global_id 5323 payload 824 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75659c640 1 -- 172.21.15.204:0/1995339640 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff7580a2400 msgr2=0x7ff75812f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75659c640 1 --2- 172.21.15.204:0/1995339640 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff7580a2400 0x7ff75812f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75659c640 1 -- 172.21.15.204:0/1995339640 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff758132930 msgr2=0x7ff75811d180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75659c640 1 --2- 172.21.15.204:0/1995339640 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff758132930 0x7ff75811d180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75659c640 10 monclient: _finish_hunting 0 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75659c640 1 monclient: found mon.noname-c 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75659c640 1 -- 172.21.15.204:0/1995339640 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff748009450 con 0x7ff75809f990 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75659c640 1 --2- 172.21.15.204:0/1995339640 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff75809f990 0x7ff758121a70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff75811ee90 tx=0x7ff748032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff7467fc640 1 -- 172.21.15.204:0/1995339640 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff74802ac80 con 0x7ff75809f990 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff7467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff7467fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff7467fc640 10 monclient: dump: 2024-09-15T01:34:47.176 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:47.177 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff7467fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff7467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75cedd640 5 monclient: authenticate success, global_id 5323 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff7467fc640 1 -- 172.21.15.204:0/1995339640 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff748033e30 con 0x7ff75809f990 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75cedd640 10 monclient: _renew_subs 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75cedd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75cedd640 1 -- 172.21.15.204:0/1995339640 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75811f610 con 0x7ff75809f990 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff7467fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff7467fc640 1 -- 172.21.15.204:0/1995339640 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff748031a60 con 0x7ff75809f990 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff7467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:47.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff7467fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff7467fc640 10 monclient: dump: 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:47.179 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff754d99640 4 set_mon_vals no callback set 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75cedd640 10 monclient: _renew_subs 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75cedd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.157+0000 7ff75cedd640 1 -- 172.21.15.204:0/1995339640 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75812fbd0 con 0x7ff75809f990 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.158+0000 7ff7467fc640 1 -- 172.21.15.204:0/1995339640 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff748039040 con 0x7ff75809f990 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.158+0000 7ff7467fc640 1 --2- 172.21.15.204:0/1995339640 >> v2:172.21.15.204:6812/28210 conn(0x7ff7380397b0 0x7ff73803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.158+0000 7ff75cedd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.158+0000 7ff75cedd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:47.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.158+0000 7ff7467fc640 1 -- 172.21.15.204:0/1995339640 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5654+0+0 (secure 0 0 0) 0x7ff74806b150 con 0x7ff75809f990 2024-09-15T01:34:47.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.158+0000 7ff75cedd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.158+0000 7ff75cedd640 1 -- 172.21.15.204:0/1995339640 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7580667d0 con 0x7ff75809f990 2024-09-15T01:34:47.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.158+0000 7ff755d9b640 1 --2- 172.21.15.204:0/1995339640 >> v2:172.21.15.204:6812/28210 conn(0x7ff7380397b0 0x7ff73803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.158+0000 7ff755d9b640 10 monclient: get_auth_request con 0x7ff7380397b0 auth_method 0 2024-09-15T01:34:47.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.159+0000 7ff755d9b640 1 --2- 172.21.15.204:0/1995339640 >> v2:172.21.15.204:6812/28210 conn(0x7ff7380397b0 0x7ff73803bc60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0038f0 tx=0x7ff74c0090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:47.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.163+0000 7ff7467fc640 1 -- 172.21.15.204:0/1995339640 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff748037080 con 0x7ff75809f990 2024-09-15T01:34:47.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.163+0000 7ff7467fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:47.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.163+0000 7ff7467fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:47.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.317+0000 7ff75cedd640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:47.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.318+0000 7ff75cedd640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:47.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.318+0000 7ff75cedd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.318+0000 7ff75cedd640 1 -- 172.21.15.204:0/1995339640 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7ff758070b80 con 0x7ff75809f990 2024-09-15T01:34:47.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.318+0000 7ff7467fc640 1 -- 172.21.15.204:0/1995339640 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v84) v1 ==== 56+0+1500 (secure 0 0 0) 0x7ff748042070 con 0x7ff75809f990 2024-09-15T01:34:47.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.318+0000 7ff7467fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:47.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.318+0000 7ff7467fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:47.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.412+0000 7ff75cedd640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:47.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.413+0000 7ff75cedd640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:47.414 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.413+0000 7ff75cedd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.413+0000 7ff75cedd640 1 -- 172.21.15.204:0/1995339640 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7ff75809e480 con 0x7ff75809f990 2024-09-15T01:34:47.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.413+0000 7ff7467fc640 1 -- 172.21.15.204:0/1995339640 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v84) v1 ==== 56+0+1500 (secure 0 0 0) 0x7ff7480446c0 con 0x7ff75809f990 2024-09-15T01:34:47.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.413+0000 7ff7467fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:47.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.413+0000 7ff7467fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:47.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.415+0000 7ff75cedd640 1 -- 172.21.15.204:0/1995339640 >> v2:172.21.15.204:6812/28210 conn(0x7ff7380397b0 msgr2=0x7ff73803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.415+0000 7ff75cedd640 1 --2- 172.21.15.204:0/1995339640 >> v2:172.21.15.204:6812/28210 conn(0x7ff7380397b0 0x7ff73803bc60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0038f0 tx=0x7ff74c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.415+0000 7ff75cedd640 10 monclient: shutdown 2024-09-15T01:34:47.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.415+0000 7ff75cedd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:47.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.415+0000 7ff75cedd640 1 -- 172.21.15.204:0/1995339640 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff75809f990 msgr2=0x7ff758121a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.415+0000 7ff75cedd640 1 --2- 172.21.15.204:0/1995339640 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff75809f990 0x7ff758121a70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff75811ee90 tx=0x7ff748032000 comp rx=0 tx=0).stop 2024-09-15T01:34:47.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.416+0000 7ff75cedd640 1 -- 172.21.15.204:0/1995339640 shutdown_connections 2024-09-15T01:34:47.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.416+0000 7ff75cedd640 1 --2- 172.21.15.204:0/1995339640 >> v2:172.21.15.204:6812/28210 conn(0x7ff7380397b0 0x7ff73803bc60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.416+0000 7ff75cedd640 1 --2- 172.21.15.204:0/1995339640 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff758132930 0x7ff75811d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.416+0000 7ff75cedd640 1 --2- 172.21.15.204:0/1995339640 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff7580a2400 0x7ff75812f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.416+0000 7ff75cedd640 1 --2- 172.21.15.204:0/1995339640 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff75809f990 0x7ff758121a70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.416+0000 7ff75cedd640 1 -- 172.21.15.204:0/1995339640 >> 172.21.15.204:0/1995339640 conn(0x7ff75806e860 msgr2=0x7ff7580a4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:47.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.416+0000 7ff75cedd640 1 -- 172.21.15.204:0/1995339640 shutdown_connections 2024-09-15T01:34:47.419 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.416+0000 7ff75cedd640 1 -- 172.21.15.204:0/1995339640 wait complete. 2024-09-15T01:34:47.426 INFO:tasks.workunit.client.0.smithi204.stdout:-12 0 rack rack-for-test 2024-09-15T01:34:47.426 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:34:47.426 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep root-for-test 2024-09-15T01:34:47.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.496+0000 7f60b1017640 10 monclient: get_monmap_and_config 2024-09-15T01:34:47.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.496+0000 7f60b1017640 10 monclient: build_initial_monmap 2024-09-15T01:34:47.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.496+0000 7f60b1017640 10 monclient: monmap: 2024-09-15T01:34:47.498 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:47.499 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.499 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:47.497161+0000 2024-09-15T01:34:47.499 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:47.497161+0000 2024-09-15T01:34:47.499 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:47.499 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.499 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:47.499 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:47.499 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:47.499 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:47.499 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:47.500 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:47.500 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:47.500 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:47.500 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:47.500 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:47.500 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:47.500 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 1 Processor -- start 2024-09-15T01:34:47.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 1 -- start start 2024-09-15T01:34:47.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 10 monclient: init 2024-09-15T01:34:47.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:47.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 10 monclient: _add_conns ranks=[7,5,10,8,3,9,6,1,4,2,0] 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f60ac0a24c0 0x7f60ac1329b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 10 monclient(hunting): picked mon.noname-i con 0x7f60ac0a24c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f60ac09f9d0 0x7f60ac09fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 10 monclient(hunting): picked mon.noname-h con 0x7f60ac09f9d0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f60ac0a0960 0x7f60ac0a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 10 monclient(hunting): picked mon.noname-f con 0x7f60ac0a0960 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ac09dc00 con 0x7f60ac0a0960 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0a2890 con 0x7f60ac09f9d0 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60b1017640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac09d810 con 0x7f60ac0a24c0 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60b1017640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:47.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60b1017640 10 monclient(hunting): authenticate will time out at 903.140014s 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.497+0000 7f60aad9d640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f60ac0a24c0 msgr2=0x7f60ac1329b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60aad9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f60ac0a24c0 0x7f60ac1329b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f60ac09f9d0 0x7f60ac09fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f60ac09f9d0 0x7f60ac09fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:53382/0 (socket says 172.21.15.204:53382) 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60ab59e640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f60ac0a0960 0x7f60ac0a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 1 -- 172.21.15.204:0/1885394831 learned_addr learned my addr 172.21.15.204:0/1885394831 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 10 monclient(hunting): get_auth_request con 0x7f60ac09f9d0 auth_method 0 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60ab59e640 10 monclient(hunting): get_auth_request con 0x7f60ac0a0960 auth_method 0 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60ab59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:47.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60ab59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60ab59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60ab59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60ab59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60ab59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 10 monclient(hunting): handle_auth_done global_id 5324 payload 274 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 1 -- 172.21.15.204:0/1885394831 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f60ac0a24c0 msgr2=0x7f60ac1329b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 1 --2- 172.21.15.204:0/1885394831 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f60ac0a24c0 0x7f60ac1329b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 1 -- 172.21.15.204:0/1885394831 >> v2:172.21.15.204:3300/0 conn(0x7f60ac0a0960 msgr2=0x7f60ac0a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 1 --2- 172.21.15.204:0/1885394831 >> v2:172.21.15.204:3300/0 conn(0x7f60ac0a0960 0x7f60ac0a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:47.512 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 1 monclient: found mon.noname-h 2024-09-15T01:34:47.512 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:47.512 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:47.512 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60a3fff640 1 -- 172.21.15.204:0/1885394831 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac0648a0 con 0x7f60ac09f9d0 2024-09-15T01:34:47.512 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.498+0000 7f60ab59e640 1 --2- 172.21.15.204:0/1885394831 >> v2:172.21.15.204:3300/0 conn(0x7f60ac0a0960 0x7f60ac0a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:47.512 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60a3fff640 1 --2- 172.21.15.204:0/1885394831 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f60ac09f9d0 0x7f60ac09fda0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6094009910 tx=0x7f60940091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=ef20ef55fa01f848 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:47.512 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60aa59c640 1 -- 172.21.15.204:0/1885394831 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f609402ad40 con 0x7f60ac09f9d0 2024-09-15T01:34:47.512 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60aa59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:47.512 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60aa59c640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:47.512 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60aa59c640 10 monclient: dump: 2024-09-15T01:34:47.512 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:47.513 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.513 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.513 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.513 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:47.513 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.513 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:47.513 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:47.513 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:47.513 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:47.513 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:47.513 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:47.513 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60aa59c640 10 monclient: _finish_auth 0 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60aa59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 5 monclient: authenticate success, global_id 5324 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60aa59c640 1 -- 172.21.15.204:0/1885394831 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6094031820 con 0x7f60ac09f9d0 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60aa59c640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 4 set_mon_vals no callback set 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60aa59c640 1 -- 172.21.15.204:0/1885394831 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6094031e90 con 0x7f60ac09f9d0 2024-09-15T01:34:47.514 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60aa59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60aa59c640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60aa59c640 10 monclient: dump: 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:47.515 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:47.516 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:47.516 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:47.516 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:47.516 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.516 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 10 monclient: shutdown 2024-09-15T01:34:47.516 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:47.516 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 1 -- 172.21.15.204:0/1885394831 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f60ac09f9d0 msgr2=0x7f60ac09fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.516 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 1 --2- 172.21.15.204:0/1885394831 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f60ac09f9d0 0x7f60ac09fda0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6094009910 tx=0x7f60940091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.516 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 1 -- 172.21.15.204:0/1885394831 shutdown_connections 2024-09-15T01:34:47.516 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 1 --2- 172.21.15.204:0/1885394831 >> v2:172.21.15.204:3300/0 conn(0x7f60ac0a0960 0x7f60ac0a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.516 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 1 --2- 172.21.15.204:0/1885394831 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f60ac09f9d0 0x7f60ac09fda0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.516 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 1 --2- 172.21.15.204:0/1885394831 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f60ac0a24c0 0x7f60ac1329b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.517 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 1 -- 172.21.15.204:0/1885394831 >> 172.21.15.204:0/1885394831 conn(0x7f60ac0a5f70 msgr2=0x7f60ac0a6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:47.517 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 1 -- 172.21.15.204:0/1885394831 shutdown_connections 2024-09-15T01:34:47.517 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.499+0000 7f60b1017640 1 -- 172.21.15.204:0/1885394831 wait complete. 2024-09-15T01:34:47.517 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.500+0000 7f60b1017640 10 monclient: build_initial_monmap 2024-09-15T01:34:47.517 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.500+0000 7f60b1017640 10 monclient: monmap: 2024-09-15T01:34:47.517 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:47.517 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:47.517 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:47.517 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:47.517 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:47.517 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:47.517 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.500+0000 7f60b1017640 1 Processor -- start 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.500+0000 7f60b1017640 1 -- start start 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.500+0000 7f60b1017640 10 monclient: init 2024-09-15T01:34:47.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 10 monclient: _add_conns ranks=[6,4,0,2,8,5,7,1,3] 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f60ac09f9d0 0x7f60ac141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 10 monclient(hunting): picked mon.noname-d con 0x7f60ac09f9d0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f60ac0a0960 0x7f60ac1327c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 10 monclient(hunting): picked mon.noname-c con 0x7f60ac0a0960 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ac0a24c0 0x7f60ac11f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 10 monclient(hunting): picked mon.noname-a con 0x7f60ac0a24c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac121680 con 0x7f60ac0a24c0 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac09d4e0 con 0x7f60ac0a0960 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.519 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac133510 con 0x7f60ac09f9d0 2024-09-15T01:34:47.520 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:47.520 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60b1017640 10 monclient(hunting): authenticate will time out at 903.143159s 2024-09-15T01:34:47.520 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60aad9d640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f60ac09f9d0 msgr2=0x7f60ac141cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:47.520 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60aad9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f60ac09f9d0 0x7f60ac141cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:47.520 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f60ac0a0960 0x7f60ac1327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.520 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60ab59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ac0a24c0 0x7f60ac11f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.520 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f60ac0a0960 0x7f60ac1327c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:36974/0 (socket says 172.21.15.204:36974) 2024-09-15T01:34:47.520 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 1 -- 172.21.15.204:0/1465902528 learned_addr learned my addr 172.21.15.204:0/1465902528 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 10 monclient(hunting): get_auth_request con 0x7f60ac0a0960 auth_method 0 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60ab59e640 1 --2- 172.21.15.204:0/1465902528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ac0a24c0 0x7f60ac11f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:56624/0 (socket says 172.21.15.204:56624) 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60ab59e640 10 monclient(hunting): get_auth_request con 0x7f60ac0a24c0 auth_method 0 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60ab59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60ab59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60ab59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:47.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60ab59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:47.522 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60ab59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:47.522 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60ab59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:47.522 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 10 monclient(hunting): handle_auth_done global_id 5332 payload 824 2024-09-15T01:34:47.522 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 1 -- 172.21.15.204:0/1465902528 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f60ac09f9d0 msgr2=0x7f60ac141cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:47.522 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 1 --2- 172.21.15.204:0/1465902528 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f60ac09f9d0 0x7f60ac141cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.522 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 1 -- 172.21.15.204:0/1465902528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ac0a24c0 msgr2=0x7f60ac11f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.522 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 1 --2- 172.21.15.204:0/1465902528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ac0a24c0 0x7f60ac11f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.522 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:47.522 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 1 monclient: found mon.noname-c 2024-09-15T01:34:47.522 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:47.522 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.523 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.501+0000 7f60a3fff640 1 -- 172.21.15.204:0/1465902528 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a4009a20 con 0x7f60ac0a0960 2024-09-15T01:34:47.523 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a3fff640 1 --2- 172.21.15.204:0/1465902528 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f60ac0a0960 0x7f60ac1327c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f60940097e0 tx=0x7f6094032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:47.523 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a2ffd640 1 -- 172.21.15.204:0/1465902528 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6094033d30 con 0x7f60ac0a0960 2024-09-15T01:34:47.523 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:47.523 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:47.523 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a2ffd640 10 monclient: dump: 2024-09-15T01:34:47.523 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:47.523 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.523 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.523 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.524 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a2ffd640 1 -- 172.21.15.204:0/1465902528 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609403b3f0 con 0x7f60ac0a0960 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60b1017640 5 monclient: authenticate success, global_id 5332 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60b1017640 10 monclient: _renew_subs 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a2ffd640 1 -- 172.21.15.204:0/1465902528 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f609403b9e0 con 0x7f60ac0a0960 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60b1017640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60b1017640 1 -- 172.21.15.204:0/1465902528 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6094009450 con 0x7f60ac0a0960 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a9d9b640 4 set_mon_vals no callback set 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60a2ffd640 10 monclient: dump: 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:47.525 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:47.526 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60b1017640 10 monclient: _renew_subs 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60b1017640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.502+0000 7f60b1017640 1 -- 172.21.15.204:0/1465902528 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac120e50 con 0x7f60ac0a0960 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.503+0000 7f60b1017640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.503+0000 7f60b1017640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.503+0000 7f60b1017640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.503+0000 7f60b1017640 1 -- 172.21.15.204:0/1465902528 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ac122c50 con 0x7f60ac0a0960 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.503+0000 7f60a2ffd640 1 -- 172.21.15.204:0/1465902528 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6094039040 con 0x7f60ac0a0960 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.503+0000 7f60a2ffd640 1 --2- 172.21.15.204:0/1465902528 >> v2:172.21.15.204:6812/28210 conn(0x7f608c0397b0 0x7f608c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.503+0000 7f60a2ffd640 1 -- 172.21.15.204:0/1465902528 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5654+0+0 (secure 0 0 0) 0x7f609406b1c0 con 0x7f60ac0a0960 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.503+0000 7f60aad9d640 1 --2- 172.21.15.204:0/1465902528 >> v2:172.21.15.204:6812/28210 conn(0x7f608c0397b0 0x7f608c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.527 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.503+0000 7f60aad9d640 10 monclient: get_auth_request con 0x7f608c0397b0 auth_method 0 2024-09-15T01:34:47.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.503+0000 7f60aad9d640 1 --2- 172.21.15.204:0/1465902528 >> v2:172.21.15.204:6812/28210 conn(0x7f608c0397b0 0x7f608c03bc60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f609c006fd0 tx=0x7f609c006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:47.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.507+0000 7f60a2ffd640 1 -- 172.21.15.204:0/1465902528 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f60940432b0 con 0x7f60ac0a0960 2024-09-15T01:34:47.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.507+0000 7f60a2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:47.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.507+0000 7f60a2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.668+0000 7f60b1017640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.668+0000 7f60b1017640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.668+0000 7f60b1017640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.668+0000 7f60b1017640 1 -- 172.21.15.204:0/1465902528 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f60ac09dee0 con 0x7f60ac0a0960 2024-09-15T01:34:47.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.669+0000 7f60a2ffd640 1 -- 172.21.15.204:0/1465902528 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v84) v1 ==== 56+0+1500 (secure 0 0 0) 0x7f609403d440 con 0x7f60ac0a0960 2024-09-15T01:34:47.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.669+0000 7f60a2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:34:47.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.669+0000 7f60a2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:47.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.763+0000 7f60b1017640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:34:47.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.763+0000 7f60b1017640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:47.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.763+0000 7f60b1017640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:47.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.763+0000 7f60b1017640 1 -- 172.21.15.204:0/1465902528 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f60ac068f90 con 0x7f60ac0a0960 2024-09-15T01:34:47.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.763+0000 7f60a2ffd640 1 -- 172.21.15.204:0/1465902528 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v84) v1 ==== 56+0+1500 (secure 0 0 0) 0x7f6094042020 con 0x7f60ac0a0960 2024-09-15T01:34:47.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.763+0000 7f60a2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:34:47.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.763+0000 7f60a2ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:34:47.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 1 -- 172.21.15.204:0/1465902528 >> v2:172.21.15.204:6812/28210 conn(0x7f608c0397b0 msgr2=0x7f608c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 1 --2- 172.21.15.204:0/1465902528 >> v2:172.21.15.204:6812/28210 conn(0x7f608c0397b0 0x7f608c03bc60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f609c006fd0 tx=0x7f609c006d30 comp rx=0 tx=0).stop 2024-09-15T01:34:47.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 10 monclient: shutdown 2024-09-15T01:34:47.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:47.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 1 -- 172.21.15.204:0/1465902528 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f60ac0a0960 msgr2=0x7f60ac1327c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 1 --2- 172.21.15.204:0/1465902528 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f60ac0a0960 0x7f60ac1327c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f60940097e0 tx=0x7f6094032000 comp rx=0 tx=0).stop 2024-09-15T01:34:47.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 1 -- 172.21.15.204:0/1465902528 shutdown_connections 2024-09-15T01:34:47.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 1 --2- 172.21.15.204:0/1465902528 >> v2:172.21.15.204:6812/28210 conn(0x7f608c0397b0 0x7f608c03bc60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 1 --2- 172.21.15.204:0/1465902528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ac0a24c0 0x7f60ac11f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 1 --2- 172.21.15.204:0/1465902528 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f60ac0a0960 0x7f60ac1327c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 1 --2- 172.21.15.204:0/1465902528 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f60ac09f9d0 0x7f60ac141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 1 -- 172.21.15.204:0/1465902528 >> 172.21.15.204:0/1465902528 conn(0x7f60ac0a5f70 msgr2=0x7f60ac0969e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:47.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 1 -- 172.21.15.204:0/1465902528 shutdown_connections 2024-09-15T01:34:47.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.766+0000 7f60b1017640 1 -- 172.21.15.204:0/1465902528 wait complete. 2024-09-15T01:34:47.776 INFO:tasks.workunit.client.0.smithi204.stdout:-14 0 root root-for-test 2024-09-15T01:34:47.776 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm host-for-test 2024-09-15T01:34:47.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.848+0000 7f954e11d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:47.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.848+0000 7f954e11d640 10 monclient: build_initial_monmap 2024-09-15T01:34:47.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.849+0000 7f954e11d640 10 monclient: monmap: 2024-09-15T01:34:47.850 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:47.850 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.850 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:47.849460+0000 2024-09-15T01:34:47.850 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:47.849460+0000 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:47.851 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:47.852 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.849+0000 7f954e11d640 1 Processor -- start 2024-09-15T01:34:47.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.849+0000 7f954e11d640 1 -- start start 2024-09-15T01:34:47.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.849+0000 7f954e11d640 10 monclient: init 2024-09-15T01:34:47.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:47.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 10 monclient: _add_conns ranks=[9,2,6,0,8,3,4,10,5,7,1] 2024-09-15T01:34:47.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f95480a2e30 0x7f9548132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 10 monclient(hunting): picked mon.noname-a con 0x7f95480a2e30 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:47.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f954809fb80 0x7f954809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 10 monclient(hunting): picked mon.noname-b con 0x7f954809fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:47.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95480a0a40 0x7f95480a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 10 monclient(hunting): picked mon.noname-d con 0x7f95480a0a40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:47.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954809e6c0 con 0x7f95480a2e30 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95480a04b0 con 0x7f954809fb80 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f95480a3200 con 0x7f95480a0a40 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f954e11d640 10 monclient(hunting): authenticate will time out at 903.492254s 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f9547fff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95480a0a40 msgr2=0x7f95480a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f953effd640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f954809fb80 msgr2=0x7f954809ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f9547fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95480a0a40 0x7f95480a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f953effd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f954809fb80 0x7f954809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f95477fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f95480a2e30 0x7f9548132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f95477fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f95480a2e30 0x7f9548132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:56626/0 (socket says 172.21.15.204:56626) 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f95477fe640 1 -- 172.21.15.204:0/71224243 learned_addr learned my addr 172.21.15.204:0/71224243 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f95477fe640 10 monclient(hunting): get_auth_request con 0x7f95480a2e30 auth_method 0 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f95477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f95477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f95477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f95477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f95477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.850+0000 7f95477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.851+0000 7f95477fe640 10 monclient(hunting): handle_auth_done global_id 5571 payload 274 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.851+0000 7f95477fe640 1 -- 172.21.15.204:0/71224243 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95480a0a40 msgr2=0x7f95480a0ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.851+0000 7f95477fe640 1 --2- 172.21.15.204:0/71224243 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95480a0a40 0x7f95480a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.851+0000 7f95477fe640 1 -- 172.21.15.204:0/71224243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f954809fb80 msgr2=0x7f954809ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.851+0000 7f95477fe640 1 --2- 172.21.15.204:0/71224243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f954809fb80 0x7f954809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.851+0000 7f95477fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.851+0000 7f95477fe640 1 monclient: found mon.noname-a 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.851+0000 7f95477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.851+0000 7f95477fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.851+0000 7f95477fe640 1 -- 172.21.15.204:0/71224243 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954804b720 con 0x7f95480a2e30 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.851+0000 7f95477fe640 1 --2- 172.21.15.204:0/71224243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95480a2e30 0x7f9548132a60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9538009910 tx=0x7f95380091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893b38b5f925deeb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f9546ffd640 1 -- 172.21.15.204:0/71224243 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f953802acd0 con 0x7f95480a2e30 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f9546ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f9546ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f9546ffd640 10 monclient: dump: 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.863 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.864 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:47.864 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.864 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:47.864 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:47.864 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:47.864 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:47.864 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:47.864 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:47.864 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:47.864 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:47.864 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:47.865 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f9546ffd640 10 monclient: _finish_auth 0 2024-09-15T01:34:47.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f9546ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:47.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f9546ffd640 1 -- 172.21.15.204:0/71224243 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953802ae50 con 0x7f95480a2e30 2024-09-15T01:34:47.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 5 monclient: authenticate success, global_id 5571 2024-09-15T01:34:47.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:47.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f9546ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:47.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f9546ffd640 1 -- 172.21.15.204:0/71224243 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9538031ca0 con 0x7f95480a2e30 2024-09-15T01:34:47.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:47.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 4 set_mon_vals no callback set 2024-09-15T01:34:47.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f9546ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:47.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f9546ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:47.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f9546ffd640 10 monclient: dump: 2024-09-15T01:34:47.866 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:47.866 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.866 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.866 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.866 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:47.866 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.866 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:47.866 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:47.866 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:47.866 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:47.866 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:47.867 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:47.867 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:47.867 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:47.867 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:47.867 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 10 monclient: shutdown 2024-09-15T01:34:47.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:47.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 1 -- 172.21.15.204:0/71224243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95480a2e30 msgr2=0x7f9548132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 1 --2- 172.21.15.204:0/71224243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95480a2e30 0x7f9548132a60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9538009910 tx=0x7f95380091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 1 -- 172.21.15.204:0/71224243 shutdown_connections 2024-09-15T01:34:47.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 1 --2- 172.21.15.204:0/71224243 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95480a0a40 0x7f95480a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 1 --2- 172.21.15.204:0/71224243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f954809fb80 0x7f954809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 1 --2- 172.21.15.204:0/71224243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95480a2e30 0x7f9548132a60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 1 -- 172.21.15.204:0/71224243 >> 172.21.15.204:0/71224243 conn(0x7f954806e860 msgr2=0x7f954806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 1 -- 172.21.15.204:0/71224243 shutdown_connections 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.852+0000 7f954e11d640 1 -- 172.21.15.204:0/71224243 wait complete. 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 10 monclient: build_initial_monmap 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 10 monclient: monmap: 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:47.868 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:47.869 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:47.869 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:47.869 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:47.869 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:47.869 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:47.869 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:47.869 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:47.869 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:47.869 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 1 Processor -- start 2024-09-15T01:34:47.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 1 -- start start 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 10 monclient: init 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 10 monclient: _add_conns ranks=[1,8,3,0,4,2,5,6,7] 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954809fb80 0x7f9548131950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 10 monclient(hunting): picked mon.noname-f con 0x7f954809fb80 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95480a0a40 0x7f954811e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 10 monclient(hunting): picked mon.noname-e con 0x7f95480a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95480a2e30 0x7f954811e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.853+0000 7f954e11d640 10 monclient(hunting): picked mon.noname-g con 0x7f95480a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954809d680 con 0x7f954809fb80 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9548142a50 con 0x7f95480a2e30 2024-09-15T01:34:47.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:47.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f954809ca40 con 0x7f95480a0a40 2024-09-15T01:34:47.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:47.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 10 monclient(hunting): authenticate will time out at 903.496022s 2024-09-15T01:34:47.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954809fb80 0x7f9548131950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954809fb80 0x7f9548131950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:46752/0 (socket says 172.21.15.204:46752) 2024-09-15T01:34:47.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 1 -- 172.21.15.204:0/3321041400 learned_addr learned my addr 172.21.15.204:0/3321041400 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:47.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 10 monclient(hunting): get_auth_request con 0x7f954809fb80 auth_method 0 2024-09-15T01:34:47.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:47.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:47.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:47.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f9547fff640 1 --2- 172.21.15.204:0/3321041400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95480a2e30 0x7f954811e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f9547fff640 10 monclient(hunting): get_auth_request con 0x7f95480a2e30 auth_method 0 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f9547fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f9547fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f9547fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f9547fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f9547fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f9547fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953effd640 1 --2- 172.21.15.204:0/3321041400 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95480a0a40 0x7f954811e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953effd640 10 monclient(hunting): get_auth_request con 0x7f95480a0a40 auth_method 0 2024-09-15T01:34:47.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 10 monclient(hunting): handle_auth_done global_id 5815 payload 824 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 1 -- 172.21.15.204:0/3321041400 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95480a0a40 msgr2=0x7f954811e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 1 --2- 172.21.15.204:0/3321041400 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95480a0a40 0x7f954811e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 1 -- 172.21.15.204:0/3321041400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95480a2e30 msgr2=0x7f954811e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 1 --2- 172.21.15.204:0/3321041400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95480a2e30 0x7f954811e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 1 monclient: found mon.noname-f 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:47.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 1 -- 172.21.15.204:0/3321041400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9538009450 con 0x7f954809fb80 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f9547fff640 1 --2- 172.21.15.204:0/3321041400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95480a2e30 0x7f954811e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95477fe640 1 --2- 172.21.15.204:0/3321041400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954809fb80 0x7f9548131950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95380097e0 tx=0x7f9538032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953ffff640 1 -- 172.21.15.204:0/3321041400 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9538031820 con 0x7f954809fb80 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953ffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953ffff640 10 monclient: dump: 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:47.874 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953ffff640 10 monclient: _finish_auth 0 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:47.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953ffff640 1 -- 172.21.15.204:0/3321041400 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95380319c0 con 0x7f954809fb80 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 5 monclient: authenticate success, global_id 5815 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 10 monclient: _renew_subs 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 1 -- 172.21.15.204:0/3321041400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954811ffe0 con 0x7f954809fb80 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953ffff640 1 -- 172.21.15.204:0/3321041400 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f953803a390 con 0x7f954809fb80 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f953ffff640 10 monclient: dump: 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:47.876 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 10 monclient: _renew_subs 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f95467fc640 4 set_mon_vals no callback set 2024-09-15T01:34:47.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:47.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.854+0000 7f954e11d640 1 -- 172.21.15.204:0/3321041400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9548120bb0 con 0x7f954809fb80 2024-09-15T01:34:47.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.855+0000 7f953ffff640 1 -- 172.21.15.204:0/3321041400 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9538039040 con 0x7f954809fb80 2024-09-15T01:34:47.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.855+0000 7f953ffff640 1 --2- 172.21.15.204:0/3321041400 >> v2:172.21.15.204:6812/28210 conn(0x7f95280397b0 0x7f952803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:47.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.855+0000 7f953ffff640 1 -- 172.21.15.204:0/3321041400 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5654+0+0 (secure 0 0 0) 0x7f953806b430 con 0x7f954809fb80 2024-09-15T01:34:47.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.855+0000 7f953effd640 1 --2- 172.21.15.204:0/3321041400 >> v2:172.21.15.204:6812/28210 conn(0x7f95280397b0 0x7f952803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:47.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.855+0000 7f953effd640 10 monclient: get_auth_request con 0x7f95280397b0 auth_method 0 2024-09-15T01:34:47.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.855+0000 7f954e11d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:47.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.855+0000 7f954e11d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:47.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.855+0000 7f954e11d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:47.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.855+0000 7f954e11d640 1 -- 172.21.15.204:0/3321041400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9548070cc0 con 0x7f954809fb80 2024-09-15T01:34:47.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.856+0000 7f953effd640 1 --2- 172.21.15.204:0/3321041400 >> v2:172.21.15.204:6812/28210 conn(0x7f95280397b0 0x7f952803bc60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9530006fd0 tx=0x7f95300090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:47.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.860+0000 7f953ffff640 1 -- 172.21.15.204:0/3321041400 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f95380313a0 con 0x7f954809fb80 2024-09-15T01:34:47.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.860+0000 7f953ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:47.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:47.860+0000 7f953ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:48.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.020+0000 7f954e11d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-09-15T01:34:48.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.020+0000 7f954e11d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-09-15T01:34:48.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.020+0000 7f954e11d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:48.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.020+0000 7f954e11d640 1 -- 172.21.15.204:0/3321041400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host-for-test"} v 0) v1 -- 0x7f95480a1f10 con 0x7f954809fb80 2024-09-15T01:34:48.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.626+0000 7f953ffff640 1 -- 172.21.15.204:0/3321041400 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host-for-test"}]=0 removed item id -11 name 'host-for-test' from crush map v85) v1 ==== 140+0+0 (secure 0 0 0) 0x7f95380392f0 con 0x7f954809fb80 2024-09-15T01:34:48.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.626+0000 7f953ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-09-15T01:34:48.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.626+0000 7f953ffff640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host-for-test' from crush map 2024-09-15T01:34:48.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.726+0000 7f954e11d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-09-15T01:34:48.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.726+0000 7f954e11d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-09-15T01:34:48.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.726+0000 7f954e11d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:48.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.726+0000 7f954e11d640 1 -- 172.21.15.204:0/3321041400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host-for-test"} v 0) v1 -- 0x7f95480008d0 con 0x7f954809fb80 2024-09-15T01:34:48.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.726+0000 7f953ffff640 1 -- 172.21.15.204:0/3321041400 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host-for-test"}]=0 device 'host-for-test' does not appear in the crush map v85) v1 ==== 140+0+0 (secure 0 0 0) 0x7f9538039940 con 0x7f954809fb80 2024-09-15T01:34:48.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.726+0000 7f953ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-09-15T01:34:48.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.726+0000 7f953ffff640 10 monclient: _finish_command 3 = system:0 device 'host-for-test' does not appear in the crush map 2024-09-15T01:34:48.729 INFO:tasks.workunit.client.0.smithi204.stderr:device 'host-for-test' does not appear in the crush map 2024-09-15T01:34:48.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 1 -- 172.21.15.204:0/3321041400 >> v2:172.21.15.204:6812/28210 conn(0x7f95280397b0 msgr2=0x7f952803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:48.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 1 --2- 172.21.15.204:0/3321041400 >> v2:172.21.15.204:6812/28210 conn(0x7f95280397b0 0x7f952803bc60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9530006fd0 tx=0x7f95300090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:48.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 10 monclient: shutdown 2024-09-15T01:34:48.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:48.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 1 -- 172.21.15.204:0/3321041400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954809fb80 msgr2=0x7f9548131950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:48.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 1 --2- 172.21.15.204:0/3321041400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954809fb80 0x7f9548131950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95380097e0 tx=0x7f9538032000 comp rx=0 tx=0).stop 2024-09-15T01:34:48.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 1 -- 172.21.15.204:0/3321041400 shutdown_connections 2024-09-15T01:34:48.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 1 --2- 172.21.15.204:0/3321041400 >> v2:172.21.15.204:6812/28210 conn(0x7f95280397b0 0x7f952803bc60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:48.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 1 --2- 172.21.15.204:0/3321041400 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95480a2e30 0x7f954811e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:48.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 1 --2- 172.21.15.204:0/3321041400 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95480a0a40 0x7f954811e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:48.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 1 --2- 172.21.15.204:0/3321041400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954809fb80 0x7f9548131950 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:48.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 1 -- 172.21.15.204:0/3321041400 >> 172.21.15.204:0/3321041400 conn(0x7f954806e860 msgr2=0x7f954809bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:48.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 1 -- 172.21.15.204:0/3321041400 shutdown_connections 2024-09-15T01:34:48.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.728+0000 7f954e11d640 1 -- 172.21.15.204:0/3321041400 wait complete. 2024-09-15T01:34:48.738 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm rack-for-test 2024-09-15T01:34:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.810+0000 7fc15bc2b640 10 monclient: get_monmap_and_config 2024-09-15T01:34:48.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.810+0000 7fc15bc2b640 10 monclient: build_initial_monmap 2024-09-15T01:34:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.810+0000 7fc15bc2b640 10 monclient: monmap: 2024-09-15T01:34:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:48.811237+0000 2024-09-15T01:34:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:48.811237+0000 2024-09-15T01:34:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:48.814 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 1 Processor -- start 2024-09-15T01:34:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 1 -- start start 2024-09-15T01:34:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 10 monclient: init 2024-09-15T01:34:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 10 monclient: _add_conns ranks=[0,5,10,9,6,4,3,8,7,1,2] 2024-09-15T01:34:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc1540a2e30 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc15409fb80 0x7fc15409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 10 monclient(hunting): picked mon.noname-h con 0x7fc15409fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc154137d20 0x7fc15413a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 10 monclient(hunting): picked mon.noname-f con 0x7fc154137d20 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15404b720 con 0x7fc1540a2e30 2024-09-15T01:34:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15409e6c0 con 0x7fc154137d20 2024-09-15T01:34:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1540a04b0 con 0x7fc15409fb80 2024-09-15T01:34:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.811+0000 7fc15bc2b640 10 monclient(hunting): authenticate will time out at 904.453970s 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc15409fb80 0x7fc15409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc15a1c8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc154137d20 0x7fc15413a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc15409fb80 0x7fc15409ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:53384/0 (socket says 172.21.15.204:53384) 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc15a1c8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc154137d20 0x7fc15413a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:46764/0 (socket says 172.21.15.204:46764) 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 1 -- 172.21.15.204:0/4281563057 learned_addr learned my addr 172.21.15.204:0/4281563057 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 10 monclient(hunting): get_auth_request con 0x7fc15409fb80 auth_method 0 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc15a1c8640 10 monclient(hunting): get_auth_request con 0x7fc154137d20 auth_method 0 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc15a1c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc15a1c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc15a1c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc15a1c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc15a1c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc15a1c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 10 monclient(hunting): handle_auth_done global_id 5333 payload 274 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 1 -- 172.21.15.204:0/4281563057 >> v2:172.21.15.204:3300/0 conn(0x7fc154137d20 msgr2=0x7fc15413a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 1 --2- 172.21.15.204:0/4281563057 >> v2:172.21.15.204:3300/0 conn(0x7fc154137d20 0x7fc15413a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 1 -- 172.21.15.204:0/4281563057 >> v1:172.21.15.178:6789/0 conn(0x7fc1540a2e30 legacy=0x7fc154132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 10 monclient: _finish_hunting 0 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 1 monclient: found mon.noname-h 2024-09-15T01:34:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc1591c6640 1 -- 172.21.15.204:0/4281563057 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc154133170 con 0x7fc15409fb80 2024-09-15T01:34:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.812+0000 7fc15a1c8640 1 --2- 172.21.15.204:0/4281563057 >> v2:172.21.15.204:3300/0 conn(0x7fc154137d20 0x7fc15413a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1591c6640 1 --2- 172.21.15.204:0/4281563057 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc15409fb80 0x7fc15409ff70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc144009910 tx=0x7fc1440091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=976e7cee4fe08f51 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1589c5640 1 -- 172.21.15.204:0/4281563057 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc14402ac90 con 0x7fc15409fb80 2024-09-15T01:34:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1589c5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1589c5640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1589c5640 10 monclient: dump: 2024-09-15T01:34:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1589c5640 10 monclient: _finish_auth 0 2024-09-15T01:34:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1589c5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1589c5640 1 -- 172.21.15.204:0/4281563057 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14402ae10 con 0x7fc15409fb80 2024-09-15T01:34:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 5 monclient: authenticate success, global_id 5333 2024-09-15T01:34:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1589c5640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1589c5640 1 -- 172.21.15.204:0/4281563057 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc144031ca0 con 0x7fc15409fb80 2024-09-15T01:34:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1589c5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 4 set_mon_vals no callback set 2024-09-15T01:34:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1589c5640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc1589c5640 10 monclient: dump: 2024-09-15T01:34:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 10 monclient: shutdown 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 1 -- 172.21.15.204:0/4281563057 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc15409fb80 msgr2=0x7fc15409ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 1 --2- 172.21.15.204:0/4281563057 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc15409fb80 0x7fc15409ff70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc144009910 tx=0x7fc1440091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 1 -- 172.21.15.204:0/4281563057 shutdown_connections 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 1 --2- 172.21.15.204:0/4281563057 >> v2:172.21.15.204:3300/0 conn(0x7fc154137d20 0x7fc15413a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 1 --2- 172.21.15.204:0/4281563057 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc15409fb80 0x7fc15409ff70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 1 -- 172.21.15.204:0/4281563057 >> 172.21.15.204:0/4281563057 conn(0x7fc15406e860 msgr2=0x7fc15406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 1 -- 172.21.15.204:0/4281563057 shutdown_connections 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.813+0000 7fc15bc2b640 1 -- 172.21.15.204:0/4281563057 wait complete. 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.814+0000 7fc15bc2b640 10 monclient: build_initial_monmap 2024-09-15T01:34:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.814+0000 7fc15bc2b640 10 monclient: monmap: 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.814+0000 7fc15bc2b640 1 Processor -- start 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.814+0000 7fc15bc2b640 1 -- start start 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.814+0000 7fc15bc2b640 10 monclient: init 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 10 monclient: _add_conns ranks=[0,4,7,2,8,1,6,5,3] 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc15409fb80 0x7fc1540990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 10 monclient(hunting): picked mon.noname-a con 0x7fc15409fb80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc1540a2e30 0x7fc154099620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 10 monclient(hunting): picked mon.noname-c con 0x7fc1540a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc154137d20 0x7fc154099b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 10 monclient(hunting): picked mon.noname-i con 0x7fc154137d20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15411c900 con 0x7fc15409fb80 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1541313b0 con 0x7fc1540a2e30 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc15409b0a0 con 0x7fc154137d20 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15bc2b640 10 monclient(hunting): authenticate will time out at 904.457262s 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15a1c8640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc154137d20 msgr2=0x7fc154099b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc15a1c8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc154137d20 0x7fc154099b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1599c7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc15409fb80 0x7fc1540990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc1540a2e30 0x7fc154099620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1599c7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc15409fb80 0x7fc1540990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:56628/0 (socket says 172.21.15.204:56628) 2024-09-15T01:34:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1599c7640 1 -- 172.21.15.204:0/2785707652 learned_addr learned my addr 172.21.15.204:0/2785707652 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1599c7640 10 monclient(hunting): get_auth_request con 0x7fc15409fb80 auth_method 0 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1599c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1599c7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1599c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 10 monclient(hunting): get_auth_request con 0x7fc1540a2e30 auth_method 0 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1599c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1599c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1599c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:48.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 10 monclient(hunting): handle_auth_done global_id 5341 payload 824 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 1 -- 172.21.15.204:0/2785707652 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc154137d20 msgr2=0x7fc154099b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 1 --2- 172.21.15.204:0/2785707652 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc154137d20 0x7fc154099b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 1 -- 172.21.15.204:0/2785707652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc15409fb80 msgr2=0x7fc1540990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 1 --2- 172.21.15.204:0/2785707652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc15409fb80 0x7fc1540990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 10 monclient: _finish_hunting 0 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 1 monclient: found mon.noname-c 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1591c6640 1 -- 172.21.15.204:0/2785707652 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c009a20 con 0x7fc1540a2e30 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.815+0000 7fc1599c7640 1 --2- 172.21.15.204:0/2785707652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc15409fb80 0x7fc1540990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc1591c6640 1 --2- 172.21.15.204:0/2785707652 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc1540a2e30 0x7fc154099620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc144031000 tx=0x7fc144032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:48.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc149ffb640 1 -- 172.21.15.204:0/2785707652 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc144031820 con 0x7fc1540a2e30 2024-09-15T01:34:48.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc149ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:48.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc149ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:34:48.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc149ffb640 10 monclient: dump: 2024-09-15T01:34:48.836 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:48.836 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:48.836 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:48.836 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:48.836 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:48.836 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:48.836 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:48.836 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:48.837 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:48.837 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:48.837 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:48.837 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:48.837 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:48.837 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:48.837 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:48.837 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:48.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc149ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:48.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc149ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:48.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc149ffb640 1 -- 172.21.15.204:0/2785707652 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc144031980 con 0x7fc1540a2e30 2024-09-15T01:34:48.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc15bc2b640 5 monclient: authenticate success, global_id 5341 2024-09-15T01:34:48.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc15bc2b640 10 monclient: _renew_subs 2024-09-15T01:34:48.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc15bc2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:48.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc15bc2b640 1 -- 172.21.15.204:0/2785707652 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc144009450 con 0x7fc1540a2e30 2024-09-15T01:34:48.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc149ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:48.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc149ffb640 1 -- 172.21.15.204:0/2785707652 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc144031af0 con 0x7fc1540a2e30 2024-09-15T01:34:48.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc149ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:48.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc149ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:34:48.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc149ffb640 10 monclient: dump: 2024-09-15T01:34:48.838 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:48.839 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:48.839 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:48.839 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:48.839 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:48.839 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:48.839 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:48.839 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:48.839 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:48.839 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:48.839 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:48.839 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:48.840 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:48.840 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:48.840 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:48.840 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:48.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc15bc2b640 10 monclient: _renew_subs 2024-09-15T01:34:48.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc15bc2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:48.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc14bfff640 4 set_mon_vals no callback set 2024-09-15T01:34:48.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.816+0000 7fc15bc2b640 1 -- 172.21.15.204:0/2785707652 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15409c6c0 con 0x7fc1540a2e30 2024-09-15T01:34:48.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.817+0000 7fc149ffb640 1 -- 172.21.15.204:0/2785707652 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc144039040 con 0x7fc1540a2e30 2024-09-15T01:34:48.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.817+0000 7fc149ffb640 1 --2- 172.21.15.204:0/2785707652 >> v2:172.21.15.204:6812/28210 conn(0x7fc1380397b0 0x7fc13803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:48.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.817+0000 7fc15bc2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:48.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.817+0000 7fc15bc2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:48.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.817+0000 7fc15bc2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:48.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.817+0000 7fc149ffb640 1 -- 172.21.15.204:0/2785707652 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5471+0+0 (secure 0 0 0) 0x7fc14406ac20 con 0x7fc1540a2e30 2024-09-15T01:34:48.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.817+0000 7fc15bc2b640 1 -- 172.21.15.204:0/2785707652 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15409ab50 con 0x7fc1540a2e30 2024-09-15T01:34:48.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.817+0000 7fc1599c7640 1 --2- 172.21.15.204:0/2785707652 >> v2:172.21.15.204:6812/28210 conn(0x7fc1380397b0 0x7fc13803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:48.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.817+0000 7fc1599c7640 10 monclient: get_auth_request con 0x7fc1380397b0 auth_method 0 2024-09-15T01:34:48.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.817+0000 7fc1599c7640 1 --2- 172.21.15.204:0/2785707652 >> v2:172.21.15.204:6812/28210 conn(0x7fc1380397b0 0x7fc13803bc60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc150002ed0 tx=0x7fc150007ed0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:48.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.822+0000 7fc149ffb640 1 -- 172.21.15.204:0/2785707652 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc144037020 con 0x7fc1540a2e30 2024-09-15T01:34:48.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.822+0000 7fc149ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:48.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.822+0000 7fc149ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:48.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.982+0000 7fc15bc2b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-09-15T01:34:48.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.982+0000 7fc15bc2b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-09-15T01:34:48.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.982+0000 7fc15bc2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:48.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:48.982+0000 7fc15bc2b640 1 -- 172.21.15.204:0/2785707652 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "rack-for-test"} v 0) v1 -- 0x7fc15409ab50 con 0x7fc1540a2e30 2024-09-15T01:34:49.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.628+0000 7fc149ffb640 1 -- 172.21.15.204:0/2785707652 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "rack-for-test"}]=0 removed item id -12 name 'rack-for-test' from crush map v86) v1 ==== 140+0+0 (secure 0 0 0) 0x7fc14402f880 con 0x7fc1540a2e30 2024-09-15T01:34:49.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.628+0000 7fc149ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-09-15T01:34:49.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.628+0000 7fc149ffb640 10 monclient: _finish_command 2 = system:0 removed item id -12 name 'rack-for-test' from crush map 2024-09-15T01:34:49.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.728+0000 7fc15bc2b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-09-15T01:34:49.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.728+0000 7fc15bc2b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-09-15T01:34:49.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.728+0000 7fc15bc2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:34:49.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.728+0000 7fc15bc2b640 1 -- 172.21.15.204:0/2785707652 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "rack-for-test"} v 0) v1 -- 0x7fc15409ab50 con 0x7fc1540a2e30 2024-09-15T01:34:49.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.729+0000 7fc149ffb640 1 -- 172.21.15.204:0/2785707652 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "rack-for-test"}]=0 device 'rack-for-test' does not appear in the crush map v86) v1 ==== 140+0+0 (secure 0 0 0) 0x7fc144034030 con 0x7fc1540a2e30 2024-09-15T01:34:49.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.729+0000 7fc149ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-09-15T01:34:49.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.729+0000 7fc149ffb640 10 monclient: _finish_command 3 = system:0 device 'rack-for-test' does not appear in the crush map 2024-09-15T01:34:49.731 INFO:tasks.workunit.client.0.smithi204.stderr:device 'rack-for-test' does not appear in the crush map 2024-09-15T01:34:49.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 1 -- 172.21.15.204:0/2785707652 >> v2:172.21.15.204:6812/28210 conn(0x7fc1380397b0 msgr2=0x7fc13803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:49.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 1 --2- 172.21.15.204:0/2785707652 >> v2:172.21.15.204:6812/28210 conn(0x7fc1380397b0 0x7fc13803bc60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc150002ed0 tx=0x7fc150007ed0 comp rx=0 tx=0).stop 2024-09-15T01:34:49.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 10 monclient: shutdown 2024-09-15T01:34:49.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:49.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 1 -- 172.21.15.204:0/2785707652 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc1540a2e30 msgr2=0x7fc154099620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:49.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 1 --2- 172.21.15.204:0/2785707652 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc1540a2e30 0x7fc154099620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc144031000 tx=0x7fc144032000 comp rx=0 tx=0).stop 2024-09-15T01:34:49.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 1 -- 172.21.15.204:0/2785707652 shutdown_connections 2024-09-15T01:34:49.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 1 --2- 172.21.15.204:0/2785707652 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc1540a2e30 0x7fc154099620 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:49.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 1 --2- 172.21.15.204:0/2785707652 >> v2:172.21.15.204:6812/28210 conn(0x7fc1380397b0 0x7fc13803bc60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:49.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 1 --2- 172.21.15.204:0/2785707652 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc154137d20 0x7fc154099b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:49.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 1 --2- 172.21.15.204:0/2785707652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc15409fb80 0x7fc1540990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:49.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 1 -- 172.21.15.204:0/2785707652 >> 172.21.15.204:0/2785707652 conn(0x7fc15406e860 msgr2=0x7fc1540a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:49.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.731+0000 7fc15bc2b640 1 -- 172.21.15.204:0/2785707652 shutdown_connections 2024-09-15T01:34:49.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.732+0000 7fc15bc2b640 1 -- 172.21.15.204:0/2785707652 wait complete. 2024-09-15T01:34:49.742 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm root-for-test 2024-09-15T01:34:49.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.813+0000 7fddbb22b640 10 monclient: get_monmap_and_config 2024-09-15T01:34:49.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.813+0000 7fddbb22b640 10 monclient: build_initial_monmap 2024-09-15T01:34:49.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.813+0000 7fddbb22b640 10 monclient: monmap: 2024-09-15T01:34:49.815 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:49.815 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:49.815 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:49.814030+0000 2024-09-15T01:34:49.815 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:49.814030+0000 2024-09-15T01:34:49.815 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:49.815 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:49.816 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 1 Processor -- start 2024-09-15T01:34:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 1 -- start start 2024-09-15T01:34:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 10 monclient: init 2024-09-15T01:34:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 10 monclient: _add_conns ranks=[1,8,0,3,2,10,5,6,4,9,7] 2024-09-15T01:34:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fddb40a2e30 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fddb409fb80 0x7fddb409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 10 monclient(hunting): picked mon.noname-e con 0x7fddb409fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fddb4137d20 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb404b720 con 0x7fddb4137d20 2024-09-15T01:34:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb40a04b0 con 0x7fddb40a2e30 2024-09-15T01:34:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fddb409e6c0 con 0x7fddb409fb80 2024-09-15T01:34:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddbb22b640 10 monclient(hunting): authenticate will time out at 905.456785s 2024-09-15T01:34:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddb8fc7640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fddb40a2e30 0x7fddb4132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:37376/0 (socket says 172.21.15.204:37376) 2024-09-15T01:34:49.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.814+0000 7fddb8fc7640 1 -- 172.21.15.204:0/754711609 learned_addr learned my addr 172.21.15.204:0/754711609 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523264471 0 0) 0x7fddb40a04b0 con 0x7fddb40a2e30 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient(hunting): my global_id is 5833 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdda40034e0 con 0x7fddb40a2e30 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fddb409fb80 0x7fddb409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 10 monclient(hunting): get_auth_request con 0x7fddb409fb80 auth_method 0 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 687203387 0 0) 0x7fdda40034e0 con 0x7fddb40a2e30 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fddb409fb80 msgr2=0x7fddb409ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:49.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fddb409fb80 0x7fddb409ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 >> v1:172.21.15.178:6789/0 conn(0x7fddb4137d20 legacy=0x7fddb413a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb4133170 con 0x7fddb40a2e30 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddbb22b640 5 monclient: authenticate success, global_id 5833 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fdda80018f0 con 0x7fddb40a2e30 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:34:49.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient: dump: 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:49.831 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 >> v1:172.21.15.204:6789/0 conn(0x7fddb40a2e30 legacy=0x7fddb4132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient: _add_conns ranks=[6,8,4,3,1,5,7,2,0] 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdda4004220 0x7fdda4004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient(hunting): picked mon.d con 0x7fdda4004220 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdda4004de0 0x7fdda4005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient(hunting): picked mon.e con 0x7fdda4004de0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdda4009c90 0x7fdda400c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient(hunting): picked mon.c con 0x7fdda4009c90 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:49.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddb409e6c0 con 0x7fdda4009c90 2024-09-15T01:34:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdda400c9c0 con 0x7fdda4004220 2024-09-15T01:34:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdda400cb30 con 0x7fdda4004de0 2024-09-15T01:34:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb37fe640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:34:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddbb22b640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fddb409fb80 0x7fddb409ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb8fc7640 1 -- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdda4004220 msgr2=0x7fdda4004780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb8fc7640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdda4004220 0x7fdda4004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:49.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdda4004de0 0x7fdda4005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb97c8640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdda4009c90 0x7fdda400c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 10 monclient(hunting): get_auth_request con 0x7fdda4004de0 auth_method 0 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb97c8640 10 monclient(hunting): get_auth_request con 0x7fdda4009c90 auth_method 0 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb97c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb97c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb97c8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:34:49.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb97c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb97c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb97c8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 10 monclient(hunting): handle_auth_done global_id 5833 payload 293 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 1 -- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdda4004220 msgr2=0x7fdda4004780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.815+0000 7fddb3fff640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdda4004220 0x7fdda4004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb3fff640 1 -- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdda4009c90 msgr2=0x7fdda400c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb3fff640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdda4009c90 0x7fdda400c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb3fff640 1 monclient: found mon.e 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb3fff640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb3fff640 1 -- 172.21.15.204:0/754711609 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda40034e0 con 0x7fdda4004de0 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb3fff640 10 monclient: _finish_auth 0 2024-09-15T01:34:49.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb97c8640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdda4009c90 0x7fdda400c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb3fff640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdda4004de0 0x7fdda4005250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fddb4133980 tx=0x7fdda0008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=44c51fe7c04ffc8e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdda002ab30 con 0x7fdda4004de0 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb37fe640 10 monclient: dump: 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:49.836 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda002e070 con 0x7fdda4004de0 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddbb22b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddbb22b640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:49.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb37fe640 1 -- 172.21.15.204:0/754711609 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdda002acf0 con 0x7fdda4004de0 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddbb22b640 4 set_mon_vals no callback set 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.816+0000 7fddb37fe640 10 monclient: dump: 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:49.838 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.817+0000 7fddbb22b640 10 monclient: shutdown 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.817+0000 7fddbb22b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.817+0000 7fddbb22b640 1 -- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdda4004de0 msgr2=0x7fdda4005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.817+0000 7fddbb22b640 1 --2- 172.21.15.204:0/754711609 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdda4004de0 0x7fdda4005250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fddb4133980 tx=0x7fdda0008040 comp rx=0 tx=0).stop 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.817+0000 7fddb97c8640 1 -- 172.21.15.204:0/754711609 reap_dead start 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.817+0000 7fddbb22b640 1 -- 172.21.15.204:0/754711609 shutdown_connections 2024-09-15T01:34:49.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.817+0000 7fddbb22b640 1 -- 172.21.15.204:0/754711609 >> 172.21.15.204:0/754711609 conn(0x7fddb406e860 msgr2=0x7fddb406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.817+0000 7fddbb22b640 1 -- 172.21.15.204:0/754711609 shutdown_connections 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.817+0000 7fddbb22b640 1 -- 172.21.15.204:0/754711609 wait complete. 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.818+0000 7fddbb22b640 10 monclient: build_initial_monmap 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.818+0000 7fddbb22b640 10 monclient: monmap: 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:49.840 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.818+0000 7fddbb22b640 1 Processor -- start 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.818+0000 7fddbb22b640 1 -- start start 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.818+0000 7fddbb22b640 10 monclient: init 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.818+0000 7fddbb22b640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.818+0000 7fddbb22b640 10 monclient: _add_conns ranks=[0,5,7,1,2,4,8,3,6] 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.818+0000 7fddbb22b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdda4004de0 0x7fddb4132ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.818+0000 7fddbb22b640 10 monclient(hunting): picked mon.noname-a con 0x7fdda4004de0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddb409a730 0x7fddb409ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:49.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 10 monclient(hunting): picked mon.noname-h con 0x7fddb409a730 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fddb409b080 0x7fddb41481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 10 monclient(hunting): picked mon.noname-i con 0x7fddb409b080 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb411e390 con 0x7fdda4004de0 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddb409d650 con 0x7fddb409a730 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fddb409ca40 con 0x7fddb409b080 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 10 monclient(hunting): authenticate will time out at 905.461053s 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb97c8640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fddb409b080 msgr2=0x7fddb41481c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb97c8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fddb409b080 0x7fddb41481c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddb409a730 0x7fddb409ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddb409a730 0x7fddb409ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:53388/0 (socket says 172.21.15.204:53388) 2024-09-15T01:34:49.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 1 -- 172.21.15.204:0/4185757037 learned_addr learned my addr 172.21.15.204:0/4185757037 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 10 monclient(hunting): get_auth_request con 0x7fddb409a730 auth_method 0 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb8fc7640 1 --2- 172.21.15.204:0/4185757037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdda4004de0 0x7fddb4132ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb8fc7640 10 monclient(hunting): get_auth_request con 0x7fdda4004de0 auth_method 0 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb8fc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb8fc7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb8fc7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb8fc7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:49.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb8fc7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb8fc7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 10 monclient(hunting): handle_auth_done global_id 5342 payload 824 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 1 -- 172.21.15.204:0/4185757037 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fddb409b080 msgr2=0x7fddb41481c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 1 --2- 172.21.15.204:0/4185757037 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fddb409b080 0x7fddb41481c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 1 -- 172.21.15.204:0/4185757037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdda4004de0 msgr2=0x7fddb4132ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 1 --2- 172.21.15.204:0/4185757037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdda4004de0 0x7fddb4132ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 1 monclient: found mon.noname-h 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 1 -- 172.21.15.204:0/4185757037 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb409c2e0 con 0x7fddb409a730 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb3fff640 1 --2- 172.21.15.204:0/4185757037 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddb409a730 0x7fddb409ab40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fddb40a2e00 tx=0x7fdda0035000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb0ff9640 1 -- 172.21.15.204:0/4185757037 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdda003d480 con 0x7fddb409a730 2024-09-15T01:34:49.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb0ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb0ff9640 10 monclient: dump: 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:49.845 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb0ff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb0ff9640 1 -- 172.21.15.204:0/4185757037 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda003d5e0 con 0x7fddb409a730 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 5 monclient: authenticate success, global_id 5342 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 10 monclient: _renew_subs 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb0ff9640 1 -- 172.21.15.204:0/4185757037 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdda003dbd0 con 0x7fddb409a730 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 1 -- 172.21.15.204:0/4185757037 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb411da60 con 0x7fddb409a730 2024-09-15T01:34:49.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb2ffd640 4 set_mon_vals no callback set 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb0ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddb0ff9640 10 monclient: dump: 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 10 monclient: _renew_subs 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.819+0000 7fddbb22b640 1 -- 172.21.15.204:0/4185757037 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb409b4b0 con 0x7fddb409a730 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.820+0000 7fddb0ff9640 1 -- 172.21.15.204:0/4185757037 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdda003dd30 con 0x7fddb409a730 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.820+0000 7fddbb22b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.820+0000 7fddbb22b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.820+0000 7fddbb22b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.820+0000 7fddbb22b640 1 -- 172.21.15.204:0/4185757037 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb411de60 con 0x7fddb409a730 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.820+0000 7fddb0ff9640 1 --2- 172.21.15.204:0/4185757037 >> v2:172.21.15.204:6812/28210 conn(0x7fdd980397b0 0x7fdd9803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.820+0000 7fddb0ff9640 1 -- 172.21.15.204:0/4185757037 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5288+0+0 (secure 0 0 0) 0x7fdda006b740 con 0x7fddb409a730 2024-09-15T01:34:49.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.820+0000 7fddb8fc7640 1 --2- 172.21.15.204:0/4185757037 >> v2:172.21.15.204:6812/28210 conn(0x7fdd980397b0 0x7fdd9803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:49.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.820+0000 7fddb8fc7640 10 monclient: get_auth_request con 0x7fdd980397b0 auth_method 0 2024-09-15T01:34:49.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.821+0000 7fddb8fc7640 1 --2- 172.21.15.204:0/4185757037 >> v2:172.21.15.204:6812/28210 conn(0x7fdd980397b0 0x7fdd9803bc60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdda8001a60 tx=0x7fddb40a1df0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:49.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.825+0000 7fddb0ff9640 1 -- 172.21.15.204:0/4185757037 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fdda0046980 con 0x7fddb409a730 2024-09-15T01:34:49.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.825+0000 7fddb0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:49.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.825+0000 7fddb0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:49.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.984+0000 7fddbb22b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-09-15T01:34:49.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.984+0000 7fddbb22b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-09-15T01:34:49.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.984+0000 7fddbb22b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:49.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:49.984+0000 7fddbb22b640 1 -- 172.21.15.204:0/4185757037 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "root-for-test"} v 0) v1 -- 0x7fddb411de60 con 0x7fddb409a730 2024-09-15T01:34:50.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.638+0000 7fddb0ff9640 1 -- 172.21.15.204:0/4185757037 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "root-for-test"}]=0 removed item id -14 name 'root-for-test' from crush map v87) v1 ==== 140+0+0 (secure 0 0 0) 0x7fdda00463c0 con 0x7fddb409a730 2024-09-15T01:34:50.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.638+0000 7fddb0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-09-15T01:34:50.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.638+0000 7fddb0ff9640 10 monclient: _finish_command 2 = system:0 removed item id -14 name 'root-for-test' from crush map 2024-09-15T01:34:50.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.737+0000 7fddbb22b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-09-15T01:34:50.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.737+0000 7fddbb22b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-09-15T01:34:50.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.737+0000 7fddbb22b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:50.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.737+0000 7fddbb22b640 1 -- 172.21.15.204:0/4185757037 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "root-for-test"} v 0) v1 -- 0x7fddb411de60 con 0x7fddb409a730 2024-09-15T01:34:50.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.738+0000 7fddb0ff9640 1 -- 172.21.15.204:0/4185757037 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "root-for-test"}]=0 device 'root-for-test' does not appear in the crush map v87) v1 ==== 140+0+0 (secure 0 0 0) 0x7fdda00463c0 con 0x7fddb409a730 2024-09-15T01:34:50.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.738+0000 7fddb0ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-09-15T01:34:50.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.738+0000 7fddb0ff9640 10 monclient: _finish_command 3 = system:0 device 'root-for-test' does not appear in the crush map 2024-09-15T01:34:50.739 INFO:tasks.workunit.client.0.smithi204.stderr:device 'root-for-test' does not appear in the crush map 2024-09-15T01:34:50.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 1 -- 172.21.15.204:0/4185757037 >> v2:172.21.15.204:6812/28210 conn(0x7fdd980397b0 msgr2=0x7fdd9803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:50.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 1 --2- 172.21.15.204:0/4185757037 >> v2:172.21.15.204:6812/28210 conn(0x7fdd980397b0 0x7fdd9803bc60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdda8001a60 tx=0x7fddb40a1df0 comp rx=0 tx=0).stop 2024-09-15T01:34:50.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 10 monclient: shutdown 2024-09-15T01:34:50.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:50.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 1 -- 172.21.15.204:0/4185757037 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddb409a730 msgr2=0x7fddb409ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:50.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 1 --2- 172.21.15.204:0/4185757037 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddb409a730 0x7fddb409ab40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fddb40a2e00 tx=0x7fdda0035000 comp rx=0 tx=0).stop 2024-09-15T01:34:50.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 1 -- 172.21.15.204:0/4185757037 shutdown_connections 2024-09-15T01:34:50.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 1 --2- 172.21.15.204:0/4185757037 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddb409a730 0x7fddb409ab40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:50.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 1 --2- 172.21.15.204:0/4185757037 >> v2:172.21.15.204:6812/28210 conn(0x7fdd980397b0 0x7fdd9803bc60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:50.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 1 --2- 172.21.15.204:0/4185757037 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fddb409b080 0x7fddb41481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:50.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 1 --2- 172.21.15.204:0/4185757037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdda4004de0 0x7fddb4132ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:50.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 1 -- 172.21.15.204:0/4185757037 >> 172.21.15.204:0/4185757037 conn(0x7fddb406e860 msgr2=0x7fddb409ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:50.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 1 -- 172.21.15.204:0/4185757037 shutdown_connections 2024-09-15T01:34:50.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.740+0000 7fddbb22b640 1 -- 172.21.15.204:0/4185757037 wait complete. 2024-09-15T01:34:50.750 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule create-simple foo-rule foo host firstn 2024-09-15T01:34:50.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.825+0000 7f9505a99640 10 monclient: get_monmap_and_config 2024-09-15T01:34:50.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.825+0000 7f9505a99640 10 monclient: build_initial_monmap 2024-09-15T01:34:50.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.825+0000 7f9505a99640 10 monclient: monmap: 2024-09-15T01:34:50.827 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:50.827 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:50.827 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:50.826052+0000 2024-09-15T01:34:50.827 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:50.826052+0000 2024-09-15T01:34:50.827 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:50.827 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:50.827 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:50.827 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:50.828 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:50.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 1 Processor -- start 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 1 -- start start 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 10 monclient: init 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 10 monclient: _add_conns ranks=[6,0,5,4,10,8,7,1,3,2,9] 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95000a2e30 0x7f9500132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 10 monclient(hunting): picked mon.noname-d con 0x7f95000a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f950009fb80 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f950009ff70 0x7f95000a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 10 monclient(hunting): picked mon.noname-h con 0x7f950009ff70 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f950004b720 con 0x7f950009fb80 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f950009e6c0 con 0x7f950009ff70 2024-09-15T01:34:50.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f95000a3200 con 0x7f95000a2e30 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f9505a99640 10 monclient(hunting): authenticate will time out at 906.468800s 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f94fffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f950009ff70 0x7f95000a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f94ff7fe640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95000a2e30 msgr2=0x7f9500132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f94ff7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95000a2e30 0x7f9500132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f94fffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f950009ff70 0x7f95000a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:53402/0 (socket says 172.21.15.204:53402) 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f94fffff640 1 -- 172.21.15.204:0/2407687057 learned_addr learned my addr 172.21.15.204:0/2407687057 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f94fffff640 10 monclient(hunting): get_auth_request con 0x7f950009ff70 auth_method 0 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f94fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f94fffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.826+0000 7f94fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:50.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 10 monclient(hunting): handle_auth_done global_id 5351 payload 274 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 1 -- 172.21.15.204:0/2407687057 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95000a2e30 msgr2=0x7f9500132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 1 --2- 172.21.15.204:0/2407687057 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95000a2e30 0x7f9500132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 1 -- 172.21.15.204:0/2407687057 >> v1:172.21.15.178:6789/0 conn(0x7f950009fb80 legacy=0x7f95000a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 1 monclient: found mon.noname-h 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 1 -- 172.21.15.204:0/2407687057 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95000a1f10 con 0x7f950009ff70 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.827+0000 7f94fffff640 1 --2- 172.21.15.204:0/2407687057 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f950009ff70 0x7f95000a1150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009f50 tx=0x7f94f8009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d4e9193cd34eed84 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:50.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f94fe7fc640 1 -- 172.21.15.204:0/2407687057 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94f8007880 con 0x7f950009ff70 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f94fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f94fe7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f94fe7fc640 10 monclient: dump: 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:50.839 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f94fe7fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f94fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f94fe7fc640 1 -- 172.21.15.204:0/2407687057 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f802d050 con 0x7f950009ff70 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 5 monclient: authenticate success, global_id 5351 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f94fe7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f94fe7fc640 1 -- 172.21.15.204:0/2407687057 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94f8030360 con 0x7f950009ff70 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 4 set_mon_vals no callback set 2024-09-15T01:34:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f94fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f94fe7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f94fe7fc640 10 monclient: dump: 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:50.841 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 10 monclient: shutdown 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 1 -- 172.21.15.204:0/2407687057 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f950009ff70 msgr2=0x7f95000a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 1 --2- 172.21.15.204:0/2407687057 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f950009ff70 0x7f95000a1150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009f50 tx=0x7f94f8009840 comp rx=0 tx=0).stop 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 1 -- 172.21.15.204:0/2407687057 shutdown_connections 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 1 --2- 172.21.15.204:0/2407687057 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f950009ff70 0x7f95000a1150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 1 --2- 172.21.15.204:0/2407687057 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95000a2e30 0x7f9500132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 1 -- 172.21.15.204:0/2407687057 >> 172.21.15.204:0/2407687057 conn(0x7f950006e860 msgr2=0x7f950006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 1 -- 172.21.15.204:0/2407687057 shutdown_connections 2024-09-15T01:34:50.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.828+0000 7f9505a99640 1 -- 172.21.15.204:0/2407687057 wait complete. 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.829+0000 7f9505a99640 10 monclient: build_initial_monmap 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.829+0000 7f9505a99640 10 monclient: monmap: 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:50.843 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.829+0000 7f9505a99640 1 Processor -- start 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.829+0000 7f9505a99640 1 -- start start 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.829+0000 7f9505a99640 10 monclient: init 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.829+0000 7f9505a99640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient: _add_conns ranks=[2,8,3,7,0,6,1,5,4] 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f950009fb80 0x7f9500141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient(hunting): picked mon.noname-b con 0x7f950009fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f950009ff70 0x7f9500131a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient(hunting): picked mon.noname-e con 0x7f950009ff70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95000a2e30 0x7f950011e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient(hunting): picked mon.noname-g con 0x7f95000a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:50.844 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950009d650 con 0x7f950009fb80 2024-09-15T01:34:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9500142a50 con 0x7f95000a2e30 2024-09-15T01:34:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9500120110 con 0x7f950009ff70 2024-09-15T01:34:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient(hunting): authenticate will time out at 906.472074s 2024-09-15T01:34:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94ff7fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f950009fb80 msgr2=0x7f9500141bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94ff7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f950009fb80 0x7f9500141bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95000a2e30 0x7f950011e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95000a2e30 0x7f950011e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:46284/0 (socket says 172.21.15.204:46284) 2024-09-15T01:34:50.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 1 -- 172.21.15.204:0/1966374335 learned_addr learned my addr 172.21.15.204:0/1966374335 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 10 monclient(hunting): get_auth_request con 0x7f95000a2e30 auth_method 0 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94feffd640 1 --2- 172.21.15.204:0/1966374335 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f950009ff70 0x7f9500131a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94feffd640 10 monclient(hunting): get_auth_request con 0x7f950009ff70 auth_method 0 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94feffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:50.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 10 monclient(hunting): handle_auth_done global_id 14457 payload 824 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 1 -- 172.21.15.204:0/1966374335 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f950009ff70 msgr2=0x7f9500131a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 1 --2- 172.21.15.204:0/1966374335 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f950009ff70 0x7f9500131a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 1 -- 172.21.15.204:0/1966374335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f950009fb80 msgr2=0x7f9500141bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 1 --2- 172.21.15.204:0/1966374335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f950009fb80 0x7f9500141bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 1 monclient: found mon.noname-g 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 1 -- 172.21.15.204:0/1966374335 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f8009ae0 con 0x7f95000a2e30 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fffff640 1 --2- 172.21.15.204:0/1966374335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95000a2e30 0x7f950011e640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f94f802abc0 tx=0x7f94f8036000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94e77fe640 1 -- 172.21.15.204:0/1966374335 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94f8008b20 con 0x7f95000a2e30 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:50.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94e77fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94e77fe640 10 monclient: dump: 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:50.848 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94e77fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94e77fe640 1 -- 172.21.15.204:0/1966374335 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f8008cc0 con 0x7f95000a2e30 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 5 monclient: authenticate success, global_id 14457 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient: _renew_subs 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 1 -- 172.21.15.204:0/1966374335 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9500121bd0 con 0x7f95000a2e30 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94e77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94e77fe640 1 -- 172.21.15.204:0/1966374335 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94f8008e30 con 0x7f95000a2e30 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94e77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:50.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94e77fe640 10 monclient: dump: 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:50.850 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient: _renew_subs 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f9505a99640 1 -- 172.21.15.204:0/1966374335 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9500142100 con 0x7f95000a2e30 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.830+0000 7f94fdffb640 4 set_mon_vals no callback set 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.831+0000 7f94e77fe640 1 -- 172.21.15.204:0/1966374335 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f94f8038050 con 0x7f95000a2e30 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.831+0000 7f94e77fe640 1 --2- 172.21.15.204:0/1966374335 >> v2:172.21.15.204:6812/28210 conn(0x7f94ec0397b0 0x7f94ec03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.831+0000 7f9505a99640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.831+0000 7f9505a99640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.831+0000 7f9505a99640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.831+0000 7f94e77fe640 1 -- 172.21.15.204:0/1966374335 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5129+0+0 (secure 0 0 0) 0x7f94f802ae40 con 0x7f95000a2e30 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.831+0000 7f9505a99640 1 -- 172.21.15.204:0/1966374335 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9500122340 con 0x7f95000a2e30 2024-09-15T01:34:50.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.831+0000 7f94ff7fe640 1 --2- 172.21.15.204:0/1966374335 >> v2:172.21.15.204:6812/28210 conn(0x7f94ec0397b0 0x7f94ec03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.831+0000 7f94ff7fe640 10 monclient: get_auth_request con 0x7f94ec0397b0 auth_method 0 2024-09-15T01:34:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.832+0000 7f94ff7fe640 1 --2- 172.21.15.204:0/1966374335 >> v2:172.21.15.204:6812/28210 conn(0x7f94ec0397b0 0x7f94ec03bc60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f94f0006fd0 tx=0x7f94f0006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.836+0000 7f94e77fe640 1 -- 172.21.15.204:0/1966374335 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9500122340 con 0x7f95000a2e30 2024-09-15T01:34:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.836+0000 7f94e77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:50.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.836+0000 7f94e77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:50.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.994+0000 7f9505a99640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-09-15T01:34:50.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.994+0000 7f9505a99640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-09-15T01:34:50.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.994+0000 7f9505a99640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:50.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:50.994+0000 7f9505a99640 1 -- 172.21.15.204:0/1966374335 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"} v 0) v1 -- 0x7f9500070cc0 con 0x7f95000a2e30 2024-09-15T01:34:51.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.642+0000 7f94e77fe640 1 -- 172.21.15.204:0/1966374335 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}]=0 v88) v1 ==== 145+0+0 (secure 0 0 0) 0x7f94f803f080 con 0x7f95000a2e30 2024-09-15T01:34:51.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.642+0000 7f94e77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-09-15T01:34:51.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.642+0000 7f94e77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.739+0000 7f9505a99640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-09-15T01:34:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.739+0000 7f9505a99640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-09-15T01:34:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.739+0000 7f9505a99640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.739+0000 7f9505a99640 1 -- 172.21.15.204:0/1966374335 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"} v 0) v1 -- 0x7f950009ddb0 con 0x7f95000a2e30 2024-09-15T01:34:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.740+0000 7f94e77fe640 1 -- 172.21.15.204:0/1966374335 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}]=0 rule foo-rule already exists v88) v1 ==== 173+0+0 (secure 0 0 0) 0x7f94f80386a0 con 0x7f95000a2e30 2024-09-15T01:34:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.740+0000 7f94e77fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-09-15T01:34:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.740+0000 7f94e77fe640 10 monclient: _finish_command 3 = system:0 rule foo-rule already exists 2024-09-15T01:34:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:rule foo-rule already exists 2024-09-15T01:34:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 1 -- 172.21.15.204:0/1966374335 >> v2:172.21.15.204:6812/28210 conn(0x7f94ec0397b0 msgr2=0x7f94ec03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 1 --2- 172.21.15.204:0/1966374335 >> v2:172.21.15.204:6812/28210 conn(0x7f94ec0397b0 0x7f94ec03bc60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f94f0006fd0 tx=0x7f94f0006d30 comp rx=0 tx=0).stop 2024-09-15T01:34:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 10 monclient: shutdown 2024-09-15T01:34:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 1 -- 172.21.15.204:0/1966374335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95000a2e30 msgr2=0x7f950011e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 1 --2- 172.21.15.204:0/1966374335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95000a2e30 0x7f950011e640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f94f802abc0 tx=0x7f94f8036000 comp rx=0 tx=0).stop 2024-09-15T01:34:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 1 -- 172.21.15.204:0/1966374335 shutdown_connections 2024-09-15T01:34:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 1 --2- 172.21.15.204:0/1966374335 >> v2:172.21.15.204:6812/28210 conn(0x7f94ec0397b0 0x7f94ec03bc60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 1 --2- 172.21.15.204:0/1966374335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95000a2e30 0x7f950011e640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 1 --2- 172.21.15.204:0/1966374335 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f950009ff70 0x7f9500131a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 1 --2- 172.21.15.204:0/1966374335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f950009fb80 0x7f9500141bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 1 -- 172.21.15.204:0/1966374335 >> 172.21.15.204:0/1966374335 conn(0x7f950006e860 msgr2=0x7f95000a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 1 -- 172.21.15.204:0/1966374335 shutdown_connections 2024-09-15T01:34:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.742+0000 7f9505a99640 1 -- 172.21.15.204:0/1966374335 wait complete. 2024-09-15T01:34:51.753 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush rm foo 2024-09-15T01:34:51.753 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:34:51.753 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm foo 2024-09-15T01:34:51.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.824+0000 7f6f1e6c0640 10 monclient: get_monmap_and_config 2024-09-15T01:34:51.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.824+0000 7f6f1e6c0640 10 monclient: build_initial_monmap 2024-09-15T01:34:51.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.824+0000 7f6f1e6c0640 10 monclient: monmap: 2024-09-15T01:34:51.826 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:51.826 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:51.826 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:51.825394+0000 2024-09-15T01:34:51.826 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:51.825394+0000 2024-09-15T01:34:51.826 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:51.827 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:51.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.825+0000 7f6f1e6c0640 1 Processor -- start 2024-09-15T01:34:51.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.825+0000 7f6f1e6c0640 1 -- start start 2024-09-15T01:34:51.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.825+0000 7f6f1e6c0640 10 monclient: init 2024-09-15T01:34:51.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:51.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 10 monclient: _add_conns ranks=[8,2,4,0,3,10,6,9,1,7,5] 2024-09-15T01:34:51.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f180a2e30 0x7f6f18132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:51.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 10 monclient(hunting): picked mon.noname-e con 0x7f6f180a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:51.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f1809fb80 0x7f6f1809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:51.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 10 monclient(hunting): picked mon.noname-b con 0x7f6f1809fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:51.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f180a0a40 0x7f6f180a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:51.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 10 monclient(hunting): picked mon.noname-c con 0x7f6f180a0a40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:51.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:51.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1809e6c0 con 0x7f6f1809fb80 2024-09-15T01:34:51.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:51.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f180a04b0 con 0x7f6f180a0a40 2024-09-15T01:34:51.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:51.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6f180a3200 con 0x7f6f180a2e30 2024-09-15T01:34:51.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:51.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1e6c0640 10 monclient(hunting): authenticate will time out at 907.468321s 2024-09-15T01:34:51.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f0ffff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f1809fb80 msgr2=0x7f6f1809ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:51.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f0ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f1809fb80 0x7f6f1809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:51.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1cc5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f180a0a40 0x7f6f180a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:51.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f180a2e30 0x7f6f18132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:51.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1cc5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f180a0a40 0x7f6f180a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:36994/0 (socket says 172.21.15.204:36994) 2024-09-15T01:34:51.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1cc5d640 1 -- 172.21.15.204:0/2764317291 learned_addr learned my addr 172.21.15.204:0/2764317291 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:51.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1cc5d640 10 monclient(hunting): get_auth_request con 0x7f6f180a0a40 auth_method 0 2024-09-15T01:34:51.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1cc5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:51.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1cc5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:51.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1cc5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:51.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 10 monclient(hunting): get_auth_request con 0x7f6f180a2e30 auth_method 0 2024-09-15T01:34:51.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:51.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:51.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:51.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1cc5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1cc5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f1cc5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 10 monclient(hunting): handle_auth_done global_id 5408 payload 274 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 1 -- 172.21.15.204:0/2764317291 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f180a0a40 msgr2=0x7f6f180a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 1 --2- 172.21.15.204:0/2764317291 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f180a0a40 0x7f6f180a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 1 -- 172.21.15.204:0/2764317291 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f1809fb80 msgr2=0x7f6f1809ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 1 --2- 172.21.15.204:0/2764317291 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f1809fb80 0x7f6f1809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 1 monclient: found mon.noname-e 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.826+0000 7f6f17fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:51.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f17fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f17fff640 1 -- 172.21.15.204:0/2764317291 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1804b720 con 0x7f6f180a2e30 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f1cc5d640 1 --2- 172.21.15.204:0/2764317291 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f180a0a40 0x7f6f180a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f17fff640 1 --2- 172.21.15.204:0/2764317291 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f180a2e30 0x7f6f18132a60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009880 tx=0x7f6f080090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=513b253bbf27d144 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f177fe640 1 -- 172.21.15.204:0/2764317291 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f0802abd0 con 0x7f6f180a2e30 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f177fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f177fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f177fe640 10 monclient: dump: 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:51.841 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f177fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:51.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f177fe640 1 -- 172.21.15.204:0/2764317291 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0802ad50 con 0x7f6f180a2e30 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f1e6c0640 5 monclient: authenticate success, global_id 5408 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f1e6c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f177fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f1e6c0640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f177fe640 1 -- 172.21.15.204:0/2764317291 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f08031ca0 con 0x7f6f180a2e30 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f177fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f1e6c0640 4 set_mon_vals no callback set 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f177fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.827+0000 7f6f177fe640 10 monclient: dump: 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:51.843 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:51.844 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:51.844 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:51.844 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:51.844 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:51.844 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:51.844 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:51.844 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:51.844 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:51.844 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:51.844 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:51.844 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.828+0000 7f6f1e6c0640 10 monclient: shutdown 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.828+0000 7f6f1e6c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.828+0000 7f6f1e6c0640 1 -- 172.21.15.204:0/2764317291 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f180a2e30 msgr2=0x7f6f18132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.828+0000 7f6f1e6c0640 1 --2- 172.21.15.204:0/2764317291 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f180a2e30 0x7f6f18132a60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009880 tx=0x7f6f080090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.828+0000 7f6f1e6c0640 1 -- 172.21.15.204:0/2764317291 shutdown_connections 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.828+0000 7f6f1e6c0640 1 --2- 172.21.15.204:0/2764317291 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f180a0a40 0x7f6f180a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.828+0000 7f6f1e6c0640 1 --2- 172.21.15.204:0/2764317291 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f1809fb80 0x7f6f1809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.828+0000 7f6f1e6c0640 1 --2- 172.21.15.204:0/2764317291 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f180a2e30 0x7f6f18132a60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.828+0000 7f6f1e6c0640 1 -- 172.21.15.204:0/2764317291 >> 172.21.15.204:0/2764317291 conn(0x7f6f1806e860 msgr2=0x7f6f1806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.828+0000 7f6f1e6c0640 1 -- 172.21.15.204:0/2764317291 shutdown_connections 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.828+0000 7f6f1e6c0640 1 -- 172.21.15.204:0/2764317291 wait complete. 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 10 monclient: build_initial_monmap 2024-09-15T01:34:51.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 10 monclient: monmap: 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:51.846 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 1 Processor -- start 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 1 -- start start 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 10 monclient: init 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 10 monclient: _add_conns ranks=[6,5,0,7,1,8,2,4,3] 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f1809fb80 0x7f6f1809b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 10 monclient(hunting): picked mon.noname-d con 0x7f6f1809fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f180a0a40 0x7f6f1812e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 10 monclient(hunting): picked mon.noname-h con 0x7f6f180a0a40 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:51.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f180a2e30 0x7f6f1811c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 10 monclient(hunting): picked mon.noname-a con 0x7f6f180a2e30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1811e910 con 0x7f6f180a2e30 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1809d650 con 0x7f6f180a0a40 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6f18142990 con 0x7f6f1809fb80 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.829+0000 7f6f1e6c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1e6c0640 10 monclient(hunting): authenticate will time out at 907.472007s 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f17fff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f1809fb80 msgr2=0x7f6f1809b5e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f17fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f1809fb80 0x7f6f1809b5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f180a0a40 0x7f6f1812e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:51.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f180a0a40 0x7f6f1812e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:53410/0 (socket says 172.21.15.204:53410) 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 1 -- 172.21.15.204:0/104239849 learned_addr learned my addr 172.21.15.204:0/104239849 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 10 monclient(hunting): get_auth_request con 0x7f6f180a0a40 auth_method 0 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1cc5d640 1 --2- 172.21.15.204:0/104239849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f180a2e30 0x7f6f1811c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1cc5d640 10 monclient(hunting): get_auth_request con 0x7f6f180a2e30 auth_method 0 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1cc5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1cc5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1cc5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:51.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 10 monclient(hunting): handle_auth_done global_id 5360 payload 824 2024-09-15T01:34:51.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 1 -- 172.21.15.204:0/104239849 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f1809fb80 msgr2=0x7f6f1809b5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:51.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 1 --2- 172.21.15.204:0/104239849 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f1809fb80 0x7f6f1809b5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:51.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 1 -- 172.21.15.204:0/104239849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f180a2e30 msgr2=0x7f6f1811c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:51.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 1 --2- 172.21.15.204:0/104239849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f180a2e30 0x7f6f1811c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:51.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:51.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 1 monclient: found mon.noname-h 2024-09-15T01:34:51.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:51.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:51.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 1 -- 172.21.15.204:0/104239849 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f10009ae0 con 0x7f6f180a0a40 2024-09-15T01:34:51.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1cc5d640 1 --2- 172.21.15.204:0/104239849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f180a2e30 0x7f6f1811c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:51.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f0ffff640 1 --2- 172.21.15.204:0/104239849 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f180a0a40 0x7f6f1812e7c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000b610 tx=0x7f6f00006f90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f14ff9640 1 -- 172.21.15.204:0/104239849 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f00015070 con 0x7f6f180a0a40 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f14ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f14ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f14ff9640 10 monclient: dump: 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:51.851 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f14ff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f14ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1e6c0640 5 monclient: authenticate success, global_id 5360 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f14ff9640 1 -- 172.21.15.204:0/104239849 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f00006150 con 0x7f6f180a0a40 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1e6c0640 10 monclient: _renew_subs 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1e6c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:51.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1e6c0640 1 -- 172.21.15.204:0/104239849 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f08009390 con 0x7f6f180a0a40 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f14ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f14ff9640 1 -- 172.21.15.204:0/104239849 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f00006780 con 0x7f6f180a0a40 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f14ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f14ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f16ffd640 4 set_mon_vals no callback set 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f14ff9640 10 monclient: dump: 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:51.853 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:51.854 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:51.854 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:51.854 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:51.854 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:51.854 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:51.854 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:51.854 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:51.854 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:51.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1e6c0640 10 monclient: _renew_subs 2024-09-15T01:34:51.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1e6c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:51.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.830+0000 7f6f1e6c0640 1 -- 172.21.15.204:0/104239849 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1811da20 con 0x7f6f180a0a40 2024-09-15T01:34:51.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.831+0000 7f6f14ff9640 1 -- 172.21.15.204:0/104239849 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6f00006300 con 0x7f6f180a0a40 2024-09-15T01:34:51.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.831+0000 7f6f1e6c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:51.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.831+0000 7f6f1e6c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:51.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.831+0000 7f6f1e6c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:51.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.831+0000 7f6f1e6c0640 1 -- 172.21.15.204:0/104239849 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1811ed80 con 0x7f6f180a0a40 2024-09-15T01:34:51.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.831+0000 7f6f14ff9640 1 --2- 172.21.15.204:0/104239849 >> v2:172.21.15.204:6812/28210 conn(0x7f6ef80397b0 0x7f6ef803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:51.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.831+0000 7f6f14ff9640 1 -- 172.21.15.204:0/104239849 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5189+0+0 (secure 0 0 0) 0x7f6f0004af90 con 0x7f6f180a0a40 2024-09-15T01:34:51.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.832+0000 7f6f17fff640 1 --2- 172.21.15.204:0/104239849 >> v2:172.21.15.204:6812/28210 conn(0x7f6ef80397b0 0x7f6ef803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:51.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.832+0000 7f6f17fff640 10 monclient: get_auth_request con 0x7f6ef80397b0 auth_method 0 2024-09-15T01:34:51.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.832+0000 7f6f17fff640 1 --2- 172.21.15.204:0/104239849 >> v2:172.21.15.204:6812/28210 conn(0x7f6ef80397b0 0x7f6ef803bc60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6f08034000 tx=0x7f6f08032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:51.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.836+0000 7f6f14ff9640 1 -- 172.21.15.204:0/104239849 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6f00003e00 con 0x7f6f180a0a40 2024-09-15T01:34:51.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.836+0000 7f6f14ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:51.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.836+0000 7f6f14ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:51.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.994+0000 7f6f1e6c0640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:34:51.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.994+0000 7f6f1e6c0640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:34:51.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.994+0000 7f6f1e6c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:51.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.994+0000 7f6f1e6c0640 1 -- 172.21.15.204:0/104239849 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) v1 -- 0x7f6f18070cc0 con 0x7f6f180a0a40 2024-09-15T01:34:51.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.994+0000 7f6f14ff9640 1 -- 172.21.15.204:0/104239849 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=-16 (16) Device or resource busy v88) v1 ==== 103+0+0 (secure 0 0 0) 0x7f6f0000eb30 con 0x7f6f180a0a40 2024-09-15T01:34:51.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.995+0000 7f6f14ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:34:51.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:51.995+0000 7f6f14ff9640 10 monclient: _finish_command 2 = mon:16 (16) Device or resource busy 2024-09-15T01:34:51.997 INFO:tasks.workunit.client.0.smithi204.stderr:Error EBUSY: (16) Device or resource busy 2024-09-15T01:34:52.000 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:34:52.000 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rule rm foo-rule 2024-09-15T01:34:52.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.072+0000 7f7a83680640 10 monclient: get_monmap_and_config 2024-09-15T01:34:52.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.072+0000 7f7a83680640 10 monclient: build_initial_monmap 2024-09-15T01:34:52.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.072+0000 7f7a83680640 10 monclient: monmap: 2024-09-15T01:34:52.073 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:52.073 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:52.074 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:52.072547+0000 2024-09-15T01:34:52.074 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:52.072547+0000 2024-09-15T01:34:52.074 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:52.074 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:52.074 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:52.074 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:52.074 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:52.074 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:52.074 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:52.074 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:52.074 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:52.074 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:52.075 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:52.075 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:52.075 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:52.075 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:52.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.072+0000 7f7a83680640 1 Processor -- start 2024-09-15T01:34:52.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.072+0000 7f7a83680640 1 -- start start 2024-09-15T01:34:52.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.072+0000 7f7a83680640 10 monclient: init 2024-09-15T01:34:52.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:52.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 10 monclient: _add_conns ranks=[6,3,8,4,5,1,2,0,9,7,10] 2024-09-15T01:34:52.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7a7c0a2e30 0x7f7a7c132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 10 monclient(hunting): picked mon.noname-d con 0x7f7a7c0a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:52.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c09fb80 0x7f7a7c09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 10 monclient(hunting): picked mon.noname-g con 0x7f7a7c09fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:52.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7a7c0a0a40 0x7f7a7c0a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 10 monclient(hunting): picked mon.noname-e con 0x7f7a7c0a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:52.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:52.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c09e6c0 con 0x7f7a7c09fb80 2024-09-15T01:34:52.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:52.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c0a04b0 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:52.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c0a3200 con 0x7f7a7c0a0a40 2024-09-15T01:34:52.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:52.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a83680640 10 monclient(hunting): authenticate will time out at 907.715281s 2024-09-15T01:34:52.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a8141c640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7a7c0a2e30 msgr2=0x7f7a7c132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:52.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c09fb80 0x7f7a7c09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a8141c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7a7c0a2e30 0x7f7a7c132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:52.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c09fb80 0x7f7a7c09ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:46294/0 (socket says 172.21.15.204:46294) 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 1 -- 172.21.15.204:0/643045687 learned_addr learned my addr 172.21.15.204:0/643045687 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 10 monclient(hunting): get_auth_request con 0x7f7a7c09fb80 auth_method 0 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a81c1d640 1 --2- 172.21.15.204:0/643045687 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7a7c0a0a40 0x7f7a7c0a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a81c1d640 10 monclient(hunting): get_auth_request con 0x7f7a7c0a0a40 auth_method 0 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a81c1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a81c1d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a81c1d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:52.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 10 monclient(hunting): handle_auth_done global_id 14466 payload 274 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 1 -- 172.21.15.204:0/643045687 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7a7c0a0a40 msgr2=0x7f7a7c0a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 1 --2- 172.21.15.204:0/643045687 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7a7c0a0a40 0x7f7a7c0a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 1 -- 172.21.15.204:0/643045687 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7a7c0a2e30 msgr2=0x7f7a7c132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 1 --2- 172.21.15.204:0/643045687 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7a7c0a2e30 0x7f7a7c132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 10 monclient: _finish_hunting 0 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 1 monclient: found mon.noname-g 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a80c1b640 1 -- 172.21.15.204:0/643045687 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c04b720 con 0x7f7a7c09fb80 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.073+0000 7f7a81c1d640 1 --2- 172.21.15.204:0/643045687 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7a7c0a0a40 0x7f7a7c0a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a80c1b640 1 --2- 172.21.15.204:0/643045687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c09fb80 0x7f7a7c09ff70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009940 tx=0x7f7a6c0091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=766b4e78a42e1a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a73fff640 1 -- 172.21.15.204:0/643045687 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a6c02ac80 con 0x7f7a7c09fb80 2024-09-15T01:34:52.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a73fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a73fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a73fff640 10 monclient: dump: 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:52.087 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a73fff640 10 monclient: _finish_auth 0 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a73fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a73fff640 1 -- 172.21.15.204:0/643045687 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6c02ae00 con 0x7f7a7c09fb80 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a83680640 5 monclient: authenticate success, global_id 14466 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a83680640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a73fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a83680640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a73fff640 1 -- 172.21.15.204:0/643045687 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a6c031ca0 con 0x7f7a7c09fb80 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a83680640 4 set_mon_vals no callback set 2024-09-15T01:34:52.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a73fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a73fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.074+0000 7f7a73fff640 10 monclient: dump: 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:52.089 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 10 monclient: shutdown 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 1 -- 172.21.15.204:0/643045687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c09fb80 msgr2=0x7f7a7c09ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 1 --2- 172.21.15.204:0/643045687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c09fb80 0x7f7a7c09ff70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009940 tx=0x7f7a6c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 1 -- 172.21.15.204:0/643045687 shutdown_connections 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 1 --2- 172.21.15.204:0/643045687 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7a7c0a0a40 0x7f7a7c0a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 1 --2- 172.21.15.204:0/643045687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c09fb80 0x7f7a7c09ff70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 1 --2- 172.21.15.204:0/643045687 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7a7c0a2e30 0x7f7a7c132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 1 -- 172.21.15.204:0/643045687 >> 172.21.15.204:0/643045687 conn(0x7f7a7c06e860 msgr2=0x7f7a7c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:52.090 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 1 -- 172.21.15.204:0/643045687 shutdown_connections 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 1 -- 172.21.15.204:0/643045687 wait complete. 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 10 monclient: build_initial_monmap 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 10 monclient: monmap: 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:52.091 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.075+0000 7f7a83680640 1 Processor -- start 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 1 -- start start 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 10 monclient: init 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 10 monclient: _add_conns ranks=[5,4,3,2,7,6,1,8,0] 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7a7c09fb80 0x7f7a7c09b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 10 monclient(hunting): picked mon.noname-h con 0x7f7a7c09fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:52.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7a7c0a0a40 0x7f7a7c12e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 10 monclient(hunting): picked mon.noname-c con 0x7f7a7c0a0a40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c0a2e30 0x7f7a7c12ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 10 monclient(hunting): picked mon.noname-g con 0x7f7a7c0a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c09d650 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c11ede0 con 0x7f7a7c0a0a40 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c09c040 con 0x7f7a7c09fb80 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a83680640 10 monclient(hunting): authenticate will time out at 907.718619s 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a81c1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c0a2e30 0x7f7a7c12ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a8141c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7a7c09fb80 0x7f7a7c09b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a81c1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c0a2e30 0x7f7a7c12ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:46310/0 (socket says 172.21.15.204:46310) 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a81c1d640 1 -- 172.21.15.204:0/4286800498 learned_addr learned my addr 172.21.15.204:0/4286800498 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a81c1d640 10 monclient(hunting): get_auth_request con 0x7f7a7c0a2e30 auth_method 0 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a81c1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a81c1d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a81c1d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a8141c640 10 monclient(hunting): get_auth_request con 0x7f7a7c09fb80 auth_method 0 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a8141c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a8141c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a8141c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a81c1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a81c1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a81c1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a8141c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:52.094 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a8141c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a8141c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a80c1b640 1 --2- 172.21.15.204:0/4286800498 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7a7c0a0a40 0x7f7a7c12e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a80c1b640 10 monclient(hunting): get_auth_request con 0x7f7a7c0a0a40 auth_method 0 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a80c1b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a80c1b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a80c1b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.076+0000 7f7a81c1d640 10 monclient(hunting): handle_auth_done global_id 14475 payload 824 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a81c1d640 1 -- 172.21.15.204:0/4286800498 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7a7c09fb80 msgr2=0x7f7a7c09b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a81c1d640 1 --2- 172.21.15.204:0/4286800498 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7a7c09fb80 0x7f7a7c09b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a81c1d640 1 -- 172.21.15.204:0/4286800498 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7a7c0a0a40 msgr2=0x7f7a7c12e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a81c1d640 1 --2- 172.21.15.204:0/4286800498 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7a7c0a0a40 0x7f7a7c12e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a81c1d640 10 monclient: _finish_hunting 0 2024-09-15T01:34:52.095 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a81c1d640 1 monclient: found mon.noname-g 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a81c1d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a81c1d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a81c1d640 1 -- 172.21.15.204:0/4286800498 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c009450 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a8141c640 1 --2- 172.21.15.204:0/4286800498 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7a7c09fb80 0x7f7a7c09b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a81c1d640 1 --2- 172.21.15.204:0/4286800498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c0a2e30 0x7f7a7c12ec10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400d180 tx=0x7f7a74009a50 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a717fa640 1 -- 172.21.15.204:0/4286800498 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a74015880 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a717fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a717fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a717fa640 10 monclient: dump: 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.096 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:52.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a717fa640 10 monclient: _finish_auth 0 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a717fa640 1 -- 172.21.15.204:0/4286800498 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7400f040 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a83680640 5 monclient: authenticate success, global_id 14475 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a717fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a83680640 10 monclient: _renew_subs 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a717fa640 1 -- 172.21.15.204:0/4286800498 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a74015e80 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a83680640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a83680640 1 -- 172.21.15.204:0/4286800498 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7c09c990 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a717fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a717fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a717fa640 10 monclient: dump: 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:52.098 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:52.099 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:52.100 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:52.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a737fe640 4 set_mon_vals no callback set 2024-09-15T01:34:52.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a83680640 10 monclient: _renew_subs 2024-09-15T01:34:52.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a83680640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:52.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.077+0000 7f7a83680640 1 -- 172.21.15.204:0/4286800498 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7c11eb00 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.078+0000 7f7a717fa640 1 -- 172.21.15.204:0/4286800498 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7a74015a00 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.078+0000 7f7a83680640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:52.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.078+0000 7f7a83680640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:52.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.078+0000 7f7a83680640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:52.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.078+0000 7f7a717fa640 1 --2- 172.21.15.204:0/4286800498 >> v2:172.21.15.204:6812/28210 conn(0x7f7a600397b0 0x7f7a6003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.078+0000 7f7a83680640 1 -- 172.21.15.204:0/4286800498 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7c070cc0 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.078+0000 7f7a717fa640 1 -- 172.21.15.204:0/4286800498 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5189+0+0 (secure 0 0 0) 0x7f7a7404bf70 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.078+0000 7f7a8141c640 1 --2- 172.21.15.204:0/4286800498 >> v2:172.21.15.204:6812/28210 conn(0x7f7a600397b0 0x7f7a6003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.078+0000 7f7a8141c640 10 monclient: get_auth_request con 0x7f7a600397b0 auth_method 0 2024-09-15T01:34:52.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.078+0000 7f7a8141c640 1 --2- 172.21.15.204:0/4286800498 >> v2:172.21.15.204:6812/28210 conn(0x7f7a600397b0 0x7f7a6003bc60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7a780038f0 tx=0x7f7a780090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:52.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.082+0000 7f7a717fa640 1 -- 172.21.15.204:0/4286800498 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7a7401e400 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.082+0000 7f7a717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:52.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.082+0000 7f7a717fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.241+0000 7f7a83680640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-09-15T01:34:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.241+0000 7f7a83680640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-09-15T01:34:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.241+0000 7f7a83680640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:52.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.241+0000 7f7a83680640 1 -- 172.21.15.204:0/4286800498 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo-rule"} v 0) v1 -- 0x7f7a7c0a1f10 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.651+0000 7f7a717fa640 1 -- 172.21.15.204:0/4286800498 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo-rule"}]=0 v89) v1 ==== 85+0+0 (secure 0 0 0) 0x7f7a74012e40 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.651+0000 7f7a717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-09-15T01:34:52.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.651+0000 7f7a717fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:34:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.749+0000 7f7a83680640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-09-15T01:34:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.749+0000 7f7a83680640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-09-15T01:34:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.749+0000 7f7a83680640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.749+0000 7f7a83680640 1 -- 172.21.15.204:0/4286800498 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo-rule"} v 0) v1 -- 0x7f7a7c11d8c0 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.749+0000 7f7a717fa640 1 -- 172.21.15.204:0/4286800498 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo-rule"}]=0 rule foo-rule does not exist v89) v1 ==== 113+0+0 (secure 0 0 0) 0x7f7a7401d7d0 con 0x7f7a7c0a2e30 2024-09-15T01:34:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.749+0000 7f7a717fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-09-15T01:34:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.749+0000 7f7a717fa640 10 monclient: _finish_command 3 = system:0 rule foo-rule does not exist 2024-09-15T01:34:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:rule foo-rule does not exist 2024-09-15T01:34:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.751+0000 7f7a83680640 1 -- 172.21.15.204:0/4286800498 >> v2:172.21.15.204:6812/28210 conn(0x7f7a600397b0 msgr2=0x7f7a6003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.751+0000 7f7a83680640 1 --2- 172.21.15.204:0/4286800498 >> v2:172.21.15.204:6812/28210 conn(0x7f7a600397b0 0x7f7a6003bc60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7a780038f0 tx=0x7f7a780090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.751+0000 7f7a83680640 10 monclient: shutdown 2024-09-15T01:34:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.751+0000 7f7a83680640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.751+0000 7f7a83680640 1 -- 172.21.15.204:0/4286800498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c0a2e30 msgr2=0x7f7a7c12ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.751+0000 7f7a83680640 1 --2- 172.21.15.204:0/4286800498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c0a2e30 0x7f7a7c12ec10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400d180 tx=0x7f7a74009a50 comp rx=0 tx=0).stop 2024-09-15T01:34:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.752+0000 7f7a83680640 1 -- 172.21.15.204:0/4286800498 shutdown_connections 2024-09-15T01:34:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.752+0000 7f7a83680640 1 --2- 172.21.15.204:0/4286800498 >> v2:172.21.15.204:6812/28210 conn(0x7f7a600397b0 0x7f7a6003bc60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.752+0000 7f7a83680640 1 --2- 172.21.15.204:0/4286800498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a7c0a2e30 0x7f7a7c12ec10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.752+0000 7f7a83680640 1 --2- 172.21.15.204:0/4286800498 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7a7c0a0a40 0x7f7a7c12e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.752+0000 7f7a83680640 1 --2- 172.21.15.204:0/4286800498 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7a7c09fb80 0x7f7a7c09b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.752+0000 7f7a83680640 1 -- 172.21.15.204:0/4286800498 >> 172.21.15.204:0/4286800498 conn(0x7f7a7c06e860 msgr2=0x7f7a7c0a41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.752+0000 7f7a83680640 1 -- 172.21.15.204:0/4286800498 shutdown_connections 2024-09-15T01:34:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.752+0000 7f7a83680640 1 -- 172.21.15.204:0/4286800498 wait complete. 2024-09-15T01:34:52.762 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm bar 2024-09-15T01:34:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.834+0000 7fdcc659d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.834+0000 7fdcc659d640 10 monclient: build_initial_monmap 2024-09-15T01:34:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.834+0000 7fdcc659d640 10 monclient: monmap: 2024-09-15T01:34:52.836 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:52.835414+0000 2024-09-15T01:34:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:52.835414+0000 2024-09-15T01:34:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:52.837 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:52.838 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:52.838 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.835+0000 7fdcc659d640 1 Processor -- start 2024-09-15T01:34:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.835+0000 7fdcc659d640 1 -- start start 2024-09-15T01:34:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.835+0000 7fdcc659d640 10 monclient: init 2024-09-15T01:34:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 10 monclient: _add_conns ranks=[5,3,10,2,9,7,4,0,8,1,6] 2024-09-15T01:34:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdcc00a2e30 0x7fdcc0132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 10 monclient(hunting): picked mon.noname-h con 0x7fdcc00a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc009fb80 0x7fdcc009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 10 monclient(hunting): picked mon.noname-g con 0x7fdcc009fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdcc00a0a40 0x7fdcc00a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 10 monclient(hunting): picked mon.noname-f con 0x7fdcc00a0a40 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:52.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc009e6c0 con 0x7fdcc00a0a40 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc00a04b0 con 0x7fdcc009fb80 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc00a3200 con 0x7fdcc00a2e30 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc659d640 10 monclient(hunting): authenticate will time out at 908.478380s 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdcc00a2e30 0x7fdcc0132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbf7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc009fb80 0x7fdcc009ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc4b3a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdcc00a0a40 0x7fdcc00a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbf7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc009fb80 0x7fdcc009ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:46312/0 (socket says 172.21.15.204:46312) 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 10 monclient(hunting): get_auth_request con 0x7fdcc00a2e30 auth_method 0 2024-09-15T01:34:52.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbf7fe640 1 -- 172.21.15.204:0/272072497 learned_addr learned my addr 172.21.15.204:0/272072497 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc4b3a640 10 monclient(hunting): get_auth_request con 0x7fdcc00a0a40 auth_method 0 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc4b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc4b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc4b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbf7fe640 10 monclient(hunting): get_auth_request con 0x7fdcc009fb80 auth_method 0 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:52.849 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc4b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc4b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc4b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 10 monclient(hunting): handle_auth_done global_id 5378 payload 274 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 1 -- 172.21.15.204:0/272072497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc009fb80 msgr2=0x7fdcc009ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 1 --2- 172.21.15.204:0/272072497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc009fb80 0x7fdcc009ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 1 -- 172.21.15.204:0/272072497 >> v2:172.21.15.204:3300/0 conn(0x7fdcc00a0a40 msgr2=0x7fdcc00a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 1 --2- 172.21.15.204:0/272072497 >> v2:172.21.15.204:3300/0 conn(0x7fdcc00a0a40 0x7fdcc00a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:52.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 1 monclient: found mon.noname-h 2024-09-15T01:34:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbffff640 1 -- 172.21.15.204:0/272072497 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc004b720 con 0x7fdcc00a2e30 2024-09-15T01:34:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcbf7fe640 1 --2- 172.21.15.204:0/272072497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc009fb80 0x7fdcc009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.836+0000 7fdcc4b3a640 1 --2- 172.21.15.204:0/272072497 >> v2:172.21.15.204:3300/0 conn(0x7fdcc00a0a40 0x7fdcc00a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbffff640 1 --2- 172.21.15.204:0/272072497 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdcc00a2e30 0x7fdcc0132a60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009850 tx=0x7fdcb00090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=ab248aa02b82aefd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbeffd640 1 -- 172.21.15.204:0/272072497 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdcb002abc0 con 0x7fdcc00a2e30 2024-09-15T01:34:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbeffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbeffd640 10 monclient: dump: 2024-09-15T01:34:52.851 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:52.852 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:52.853 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbeffd640 10 monclient: _finish_auth 0 2024-09-15T01:34:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcc659d640 5 monclient: authenticate success, global_id 5378 2024-09-15T01:34:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbeffd640 1 -- 172.21.15.204:0/272072497 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb002ad40 con 0x7fdcc00a2e30 2024-09-15T01:34:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcc659d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcc659d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcc659d640 4 set_mon_vals no callback set 2024-09-15T01:34:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbeffd640 1 -- 172.21.15.204:0/272072497 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdcb0031ca0 con 0x7fdcc00a2e30 2024-09-15T01:34:52.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbeffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.837+0000 7fdcbeffd640 10 monclient: dump: 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:52.854 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:52.855 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 10 monclient: shutdown 2024-09-15T01:34:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 1 -- 172.21.15.204:0/272072497 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdcc00a2e30 msgr2=0x7fdcc0132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 1 --2- 172.21.15.204:0/272072497 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdcc00a2e30 0x7fdcc0132a60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009850 tx=0x7fdcb00090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 1 -- 172.21.15.204:0/272072497 shutdown_connections 2024-09-15T01:34:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 1 --2- 172.21.15.204:0/272072497 >> v2:172.21.15.204:3300/0 conn(0x7fdcc00a0a40 0x7fdcc00a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 1 --2- 172.21.15.204:0/272072497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc009fb80 0x7fdcc009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 1 --2- 172.21.15.204:0/272072497 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdcc00a2e30 0x7fdcc0132a60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 1 -- 172.21.15.204:0/272072497 >> 172.21.15.204:0/272072497 conn(0x7fdcc006e860 msgr2=0x7fdcc006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 1 -- 172.21.15.204:0/272072497 shutdown_connections 2024-09-15T01:34:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 1 -- 172.21.15.204:0/272072497 wait complete. 2024-09-15T01:34:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 10 monclient: build_initial_monmap 2024-09-15T01:34:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.838+0000 7fdcc659d640 10 monclient: monmap: 2024-09-15T01:34:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:52.856 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:52.857 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 1 Processor -- start 2024-09-15T01:34:52.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 1 -- start start 2024-09-15T01:34:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 10 monclient: init 2024-09-15T01:34:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 10 monclient: _add_conns ranks=[1,5,3,7,4,0,2,6,8] 2024-09-15T01:34:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc009fb80 0x7fdcc0141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 10 monclient(hunting): picked mon.noname-f con 0x7fdcc009fb80 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdcc00a0a40 0x7fdcc0142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 10 monclient(hunting): picked mon.noname-h con 0x7fdcc00a0a40 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc00a2e30 0x7fdcc01425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 10 monclient(hunting): picked mon.noname-g con 0x7fdcc00a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:52.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc0121760 con 0x7fdcc009fb80 2024-09-15T01:34:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc0121db0 con 0x7fdcc00a2e30 2024-09-15T01:34:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc011fca0 con 0x7fdcc00a0a40 2024-09-15T01:34:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc659d640 10 monclient(hunting): authenticate will time out at 908.481746s 2024-09-15T01:34:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc009fb80 0x7fdcc0141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc4b3a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc00a2e30 0x7fdcc01425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc009fb80 0x7fdcc0141b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:46786/0 (socket says 172.21.15.204:46786) 2024-09-15T01:34:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbffff640 1 -- 172.21.15.204:0/149794422 learned_addr learned my addr 172.21.15.204:0/149794422 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbffff640 10 monclient(hunting): get_auth_request con 0x7fdcc009fb80 auth_method 0 2024-09-15T01:34:52.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbf7fe640 1 --2- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdcc00a0a40 0x7fdcc0142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc4b3a640 10 monclient(hunting): get_auth_request con 0x7fdcc00a2e30 auth_method 0 2024-09-15T01:34:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc4b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc4b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:52.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc4b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbf7fe640 10 monclient(hunting): get_auth_request con 0x7fdcc00a0a40 auth_method 0 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc4b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc4b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:52.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.839+0000 7fdcc4b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbffff640 10 monclient(hunting): handle_auth_done global_id 5851 payload 824 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbffff640 1 -- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdcc00a0a40 msgr2=0x7fdcc0142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbffff640 1 --2- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdcc00a0a40 0x7fdcc0142090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbffff640 1 -- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc00a2e30 msgr2=0x7fdcc01425d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbffff640 1 --2- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc00a2e30 0x7fdcc01425d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbffff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbffff640 1 monclient: found mon.noname-f 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbffff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbffff640 1 -- 172.21.15.204:0/149794422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb4009450 con 0x7fdcc009fb80 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbf7fe640 1 --2- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdcc00a0a40 0x7fdcc0142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcc4b3a640 1 --2- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc00a2e30 0x7fdcc01425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:52.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbffff640 1 --2- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc009fb80 0x7fdcc0141b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009360 tx=0x7fdcb0032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdca7fff640 1 -- 172.21.15.204:0/149794422 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdcb003ca90 con 0x7fdcc009fb80 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdca7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdca7fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdca7fff640 10 monclient: dump: 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:52.863 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:52.864 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:52.864 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:52.864 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:52.864 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:52.864 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:52.864 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:52.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdca7fff640 10 monclient: _finish_auth 0 2024-09-15T01:34:52.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdca7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:52.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcc659d640 5 monclient: authenticate success, global_id 5851 2024-09-15T01:34:52.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdca7fff640 1 -- 172.21.15.204:0/149794422 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb003cc10 con 0x7fdcc009fb80 2024-09-15T01:34:52.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdca7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:52.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdca7fff640 1 -- 172.21.15.204:0/149794422 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdcb003cd80 con 0x7fdcc009fb80 2024-09-15T01:34:52.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcc659d640 10 monclient: _renew_subs 2024-09-15T01:34:52.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcc659d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:52.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcc659d640 1 -- 172.21.15.204:0/149794422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb8009a20 con 0x7fdcc009fb80 2024-09-15T01:34:52.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdca7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:52.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdca7fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:52.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdca7fff640 10 monclient: dump: 2024-09-15T01:34:52.865 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:52.865 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:52.866 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:52.867 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:52.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcbe7fc640 4 set_mon_vals no callback set 2024-09-15T01:34:52.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcc659d640 10 monclient: _renew_subs 2024-09-15T01:34:52.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcc659d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:52.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.840+0000 7fdcc659d640 1 -- 172.21.15.204:0/149794422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb0009390 con 0x7fdcc009fb80 2024-09-15T01:34:52.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.841+0000 7fdca7fff640 1 -- 172.21.15.204:0/149794422 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdcb003c610 con 0x7fdcc009fb80 2024-09-15T01:34:52.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.841+0000 7fdcc659d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:52.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.841+0000 7fdcc659d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:52.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.841+0000 7fdcc659d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:52.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.841+0000 7fdcc659d640 1 -- 172.21.15.204:0/149794422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc0122ad0 con 0x7fdcc009fb80 2024-09-15T01:34:52.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.841+0000 7fdca7fff640 1 --2- 172.21.15.204:0/149794422 >> v2:172.21.15.204:6812/28210 conn(0x7fdcac0397b0 0x7fdcac03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:52.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.841+0000 7fdca7fff640 1 -- 172.21.15.204:0/149794422 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5129+0+0 (secure 0 0 0) 0x7fdcb006b890 con 0x7fdcc009fb80 2024-09-15T01:34:52.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.841+0000 7fdcbf7fe640 1 --2- 172.21.15.204:0/149794422 >> v2:172.21.15.204:6812/28210 conn(0x7fdcac0397b0 0x7fdcac03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:52.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.841+0000 7fdcbf7fe640 10 monclient: get_auth_request con 0x7fdcac0397b0 auth_method 0 2024-09-15T01:34:52.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.841+0000 7fdcbf7fe640 1 --2- 172.21.15.204:0/149794422 >> v2:172.21.15.204:6812/28210 conn(0x7fdcac0397b0 0x7fdcac03bc60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4006cb0 tx=0x7fdcb40091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:52.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.846+0000 7fdca7fff640 1 -- 172.21.15.204:0/149794422 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fdcb004bd50 con 0x7fdcc009fb80 2024-09-15T01:34:52.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.846+0000 7fdca7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:52.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:52.846+0000 7fdca7fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:53.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.005+0000 7fdcc659d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T01:34:53.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.005+0000 7fdcc659d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T01:34:53.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.005+0000 7fdcc659d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:53.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.005+0000 7fdcc659d640 1 -- 172.21.15.204:0/149794422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) v1 -- 0x7fdcc009ca40 con 0x7fdcc009fb80 2024-09-15T01:34:53.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.655+0000 7fdca7fff640 1 -- 172.21.15.204:0/149794422 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=0 removed item id -19 name 'bar' from crush map v90) v1 ==== 120+0+0 (secure 0 0 0) 0x7fdcb0037070 con 0x7fdcc009fb80 2024-09-15T01:34:53.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.655+0000 7fdca7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T01:34:53.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.655+0000 7fdca7fff640 10 monclient: _finish_command 2 = system:0 removed item id -19 name 'bar' from crush map 2024-09-15T01:34:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.754+0000 7fdcc659d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T01:34:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.754+0000 7fdcc659d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T01:34:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.754+0000 7fdcc659d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.754+0000 7fdcc659d640 1 -- 172.21.15.204:0/149794422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) v1 -- 0x7fdcc0070cc0 con 0x7fdcc009fb80 2024-09-15T01:34:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.754+0000 7fdca7fff640 1 -- 172.21.15.204:0/149794422 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=0 device 'bar' does not appear in the crush map v90) v1 ==== 120+0+0 (secure 0 0 0) 0x7fdcb004b770 con 0x7fdcc009fb80 2024-09-15T01:34:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.754+0000 7fdca7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "bar"}] 2024-09-15T01:34:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.754+0000 7fdca7fff640 10 monclient: _finish_command 3 = system:0 device 'bar' does not appear in the crush map 2024-09-15T01:34:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:device 'bar' does not appear in the crush map 2024-09-15T01:34:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 1 -- 172.21.15.204:0/149794422 >> v2:172.21.15.204:6812/28210 conn(0x7fdcac0397b0 msgr2=0x7fdcac03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 1 --2- 172.21.15.204:0/149794422 >> v2:172.21.15.204:6812/28210 conn(0x7fdcac0397b0 0x7fdcac03bc60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4006cb0 tx=0x7fdcb40091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 10 monclient: shutdown 2024-09-15T01:34:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 1 -- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc009fb80 msgr2=0x7fdcc0141b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 1 --2- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc009fb80 0x7fdcc0141b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009360 tx=0x7fdcb0032000 comp rx=0 tx=0).stop 2024-09-15T01:34:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 1 -- 172.21.15.204:0/149794422 shutdown_connections 2024-09-15T01:34:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 1 --2- 172.21.15.204:0/149794422 >> v2:172.21.15.204:6812/28210 conn(0x7fdcac0397b0 0x7fdcac03bc60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 1 --2- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcc00a2e30 0x7fdcc01425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 1 --2- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdcc00a0a40 0x7fdcc0142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 1 --2- 172.21.15.204:0/149794422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc009fb80 0x7fdcc0141b50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 1 -- 172.21.15.204:0/149794422 >> 172.21.15.204:0/149794422 conn(0x7fdcc006e860 msgr2=0x7fdcc00a5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 1 -- 172.21.15.204:0/149794422 shutdown_connections 2024-09-15T01:34:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.756+0000 7fdcc659d640 1 -- 172.21.15.204:0/149794422 wait complete. 2024-09-15T01:34:53.766 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm foo 2024-09-15T01:34:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.838+0000 7f51df646640 10 monclient: get_monmap_and_config 2024-09-15T01:34:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.838+0000 7f51df646640 10 monclient: build_initial_monmap 2024-09-15T01:34:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.838+0000 7f51df646640 10 monclient: monmap: 2024-09-15T01:34:53.839 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:53.838624+0000 2024-09-15T01:34:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:53.838624+0000 2024-09-15T01:34:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:53.841 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.838+0000 7f51df646640 1 Processor -- start 2024-09-15T01:34:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.838+0000 7f51df646640 1 -- start start 2024-09-15T01:34:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.838+0000 7f51df646640 10 monclient: init 2024-09-15T01:34:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 10 monclient: _add_conns ranks=[7,9,10,6,3,8,2,4,5,0,1] 2024-09-15T01:34:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f51d80a4680 0x7f51d809e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 10 monclient(hunting): picked mon.noname-i con 0x7f51d80a4680 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f51d806eea0 0x7f51d809ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 10 monclient(hunting): picked mon.noname-a con 0x7f51d806eea0 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f51d809f2d0 0x7f51d81308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 10 monclient(hunting): picked mon.noname-f con 0x7f51d809f2d0 addr v2:172.21.15.204:3300/0 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d80a4a50 con 0x7f51d806eea0 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d806dc70 con 0x7f51d809f2d0 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f51d80a0710 con 0x7f51d80a4680 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51df646640 10 monclient(hunting): authenticate will time out at 909.481387s 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51dd3e2640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f51d80a4680 msgr2=0x7f51d809e850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51dd3e2640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f51d80a4680 0x7f51d809e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f51d809f2d0 0x7f51d81308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f51d809f2d0 0x7f51d81308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:46788/0 (socket says 172.21.15.204:46788) 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 1 -- 172.21.15.204:0/2962446680 learned_addr learned my addr 172.21.15.204:0/2962446680 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 10 monclient(hunting): get_auth_request con 0x7f51d809f2d0 auth_method 0 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51dcbe1640 1 --2- 172.21.15.204:0/2962446680 >> v2:172.21.15.178:3300/0 conn(0x7f51d806eea0 0x7f51d809ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51dcbe1640 10 monclient(hunting): get_auth_request con 0x7f51d806eea0 auth_method 0 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51dcbe1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51dcbe1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51dcbe1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 10 monclient(hunting): handle_auth_done global_id 5860 payload 274 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 1 -- 172.21.15.204:0/2962446680 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f51d80a4680 msgr2=0x7f51d809e850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 1 --2- 172.21.15.204:0/2962446680 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f51d80a4680 0x7f51d809e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 1 -- 172.21.15.204:0/2962446680 >> v2:172.21.15.178:3300/0 conn(0x7f51d806eea0 msgr2=0x7f51d809ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 1 --2- 172.21.15.204:0/2962446680 >> v2:172.21.15.178:3300/0 conn(0x7f51d806eea0 0x7f51d809ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 10 monclient: _finish_hunting 0 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 1 monclient: found mon.noname-f 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.839+0000 7f51ddbe3640 1 -- 172.21.15.204:0/2962446680 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d804b720 con 0x7f51d809f2d0 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51ddbe3640 1 --2- 172.21.15.204:0/2962446680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d809f2d0 0x7f51d81308a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f51d806f270 tx=0x7f51c80091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d18709f4ab340a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51d7fff640 1 -- 172.21.15.204:0/2962446680 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f51c802acc0 con 0x7f51d809f2d0 2024-09-15T01:34:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51d7fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51d7fff640 10 monclient: dump: 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:53.855 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51d7fff640 10 monclient: _finish_auth 0 2024-09-15T01:34:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51d7fff640 1 -- 172.21.15.204:0/2962446680 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c802ae40 con 0x7f51d809f2d0 2024-09-15T01:34:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51df646640 5 monclient: authenticate success, global_id 5860 2024-09-15T01:34:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51df646640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51d7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51df646640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51d7fff640 1 -- 172.21.15.204:0/2962446680 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f51c8031ca0 con 0x7f51d809f2d0 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51df646640 4 set_mon_vals no callback set 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51d7fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.840+0000 7f51d7fff640 10 monclient: dump: 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 10 monclient: shutdown 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 1 -- 172.21.15.204:0/2962446680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d809f2d0 msgr2=0x7f51d81308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 1 --2- 172.21.15.204:0/2962446680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d809f2d0 0x7f51d81308a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f51d806f270 tx=0x7f51c80091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 1 -- 172.21.15.204:0/2962446680 shutdown_connections 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 1 --2- 172.21.15.204:0/2962446680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d809f2d0 0x7f51d81308a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 1 --2- 172.21.15.204:0/2962446680 >> v2:172.21.15.178:3300/0 conn(0x7f51d806eea0 0x7f51d809ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 1 --2- 172.21.15.204:0/2962446680 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f51d80a4680 0x7f51d809e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 1 -- 172.21.15.204:0/2962446680 >> 172.21.15.204:0/2962446680 conn(0x7f51d8067740 msgr2=0x7f51d80a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 1 -- 172.21.15.204:0/2962446680 shutdown_connections 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 1 -- 172.21.15.204:0/2962446680 wait complete. 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 10 monclient: build_initial_monmap 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.841+0000 7f51df646640 10 monclient: monmap: 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:53.859 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 1 Processor -- start 2024-09-15T01:34:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 1 -- start start 2024-09-15T01:34:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 10 monclient: init 2024-09-15T01:34:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 10 monclient: _add_conns ranks=[3,2,6,8,0,4,7,1,5] 2024-09-15T01:34:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d806eea0 0x7f51d813c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 10 monclient(hunting): picked mon.noname-g con 0x7f51d806eea0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d809f2d0 0x7f51d813b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 10 monclient(hunting): picked mon.noname-b con 0x7f51d809f2d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f51d80a4680 0x7f51d813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 10 monclient(hunting): picked mon.noname-d con 0x7f51d80a4680 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8121130 con 0x7f51d809f2d0 2024-09-15T01:34:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8120b70 con 0x7f51d806eea0 2024-09-15T01:34:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f51d811e6c0 con 0x7f51d80a4680 2024-09-15T01:34:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51df646640 10 monclient(hunting): authenticate will time out at 909.484909s 2024-09-15T01:34:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51dcbe1640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d809f2d0 msgr2=0x7f51d813b7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51dcbe1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d809f2d0 0x7f51d813b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.842+0000 7f51dd3e2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d806eea0 0x7f51d813c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d806eea0 0x7f51d813c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:46330/0 (socket says 172.21.15.204:46330) 2024-09-15T01:34:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51ddbe3640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f51d80a4680 msgr2=0x7f51d813f6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 1 -- 172.21.15.204:0/3325447118 learned_addr learned my addr 172.21.15.204:0/3325447118 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 10 monclient(hunting): get_auth_request con 0x7f51d806eea0 auth_method 0 2024-09-15T01:34:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51ddbe3640 1 --2- 172.21.15.204:0/3325447118 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f51d80a4680 0x7f51d813f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 10 monclient(hunting): handle_auth_done global_id 14502 payload 824 2024-09-15T01:34:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 1 -- 172.21.15.204:0/3325447118 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f51d80a4680 msgr2=0x7f51d813f6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 1 --2- 172.21.15.204:0/3325447118 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f51d80a4680 0x7f51d813f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 1 -- 172.21.15.204:0/3325447118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d809f2d0 msgr2=0x7f51d813b7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 1 --2- 172.21.15.204:0/3325447118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d809f2d0 0x7f51d813b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 10 monclient: _finish_hunting 0 2024-09-15T01:34:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 1 monclient: found mon.noname-g 2024-09-15T01:34:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 1 -- 172.21.15.204:0/3325447118 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8009450 con 0x7f51d806eea0 2024-09-15T01:34:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51dd3e2640 1 --2- 172.21.15.204:0/3325447118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d806eea0 0x7f51d813c140 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f51d000bb80 tx=0x7f51d00097b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d57fa640 1 -- 172.21.15.204:0/3325447118 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f51d000cea0 con 0x7f51d806eea0 2024-09-15T01:34:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d57fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d57fa640 10 monclient: dump: 2024-09-15T01:34:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:53.865 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d57fa640 10 monclient: _finish_auth 0 2024-09-15T01:34:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d57fa640 1 -- 172.21.15.204:0/3325447118 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d0006150 con 0x7f51d806eea0 2024-09-15T01:34:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51df646640 5 monclient: authenticate success, global_id 14502 2024-09-15T01:34:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d57fa640 1 -- 172.21.15.204:0/3325447118 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f51d0006780 con 0x7f51d806eea0 2024-09-15T01:34:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51df646640 10 monclient: _renew_subs 2024-09-15T01:34:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51df646640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51df646640 1 -- 172.21.15.204:0/3325447118 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d8120890 con 0x7f51d806eea0 2024-09-15T01:34:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d57fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d57fa640 10 monclient: dump: 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51d77fe640 4 set_mon_vals no callback set 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51df646640 10 monclient: _renew_subs 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51df646640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.843+0000 7f51df646640 1 -- 172.21.15.204:0/3325447118 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d813bd30 con 0x7f51d806eea0 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.844+0000 7f51d57fa640 1 -- 172.21.15.204:0/3325447118 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f51d0012020 con 0x7f51d806eea0 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.844+0000 7f51df646640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.844+0000 7f51df646640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.844+0000 7f51df646640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.844+0000 7f51df646640 1 -- 172.21.15.204:0/3325447118 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d8070cc0 con 0x7f51d806eea0 2024-09-15T01:34:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.844+0000 7f51d57fa640 1 --2- 172.21.15.204:0/3325447118 >> v2:172.21.15.204:6812/28210 conn(0x7f51c40397b0 0x7f51c403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.844+0000 7f51d57fa640 1 -- 172.21.15.204:0/3325447118 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5000+0+0 (secure 0 0 0) 0x7f51d004b4c0 con 0x7f51d806eea0 2024-09-15T01:34:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.844+0000 7f51dcbe1640 1 --2- 172.21.15.204:0/3325447118 >> v2:172.21.15.204:6812/28210 conn(0x7f51c40397b0 0x7f51c403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.844+0000 7f51dcbe1640 10 monclient: get_auth_request con 0x7f51c40397b0 auth_method 0 2024-09-15T01:34:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.845+0000 7f51dcbe1640 1 --2- 172.21.15.204:0/3325447118 >> v2:172.21.15.204:6812/28210 conn(0x7f51c40397b0 0x7f51c403bc60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006f40 tx=0x7f51cc008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.849+0000 7f51d57fa640 1 -- 172.21.15.204:0/3325447118 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f51d0006300 con 0x7f51d806eea0 2024-09-15T01:34:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.849+0000 7f51d57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:53.849+0000 7f51d57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:54.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.008+0000 7f51df646640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:34:54.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.008+0000 7f51df646640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:34:54.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.008+0000 7f51df646640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:54.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.008+0000 7f51df646640 1 -- 172.21.15.204:0/3325447118 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) v1 -- 0x7f51d806a8b0 con 0x7f51d806eea0 2024-09-15T01:34:54.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.660+0000 7f51d57fa640 1 -- 172.21.15.204:0/3325447118 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 removed item id -10 name 'foo' from crush map v91) v1 ==== 120+0+0 (secure 0 0 0) 0x7f51d0014340 con 0x7f51d806eea0 2024-09-15T01:34:54.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.660+0000 7f51d57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:34:54.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.660+0000 7f51d57fa640 10 monclient: _finish_command 2 = system:0 removed item id -10 name 'foo' from crush map 2024-09-15T01:34:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.758+0000 7f51df646640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:34:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.758+0000 7f51df646640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:34:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.758+0000 7f51df646640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.759+0000 7f51df646640 1 -- 172.21.15.204:0/3325447118 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) v1 -- 0x7f51d811f9c0 con 0x7f51d806eea0 2024-09-15T01:34:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.759+0000 7f51d57fa640 1 -- 172.21.15.204:0/3325447118 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 device 'foo' does not appear in the crush map v91) v1 ==== 120+0+0 (secure 0 0 0) 0x7f51d001d300 con 0x7f51d806eea0 2024-09-15T01:34:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.759+0000 7f51d57fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:34:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.759+0000 7f51d57fa640 10 monclient: _finish_command 3 = system:0 device 'foo' does not appear in the crush map 2024-09-15T01:34:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:device 'foo' does not appear in the crush map 2024-09-15T01:34:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 1 -- 172.21.15.204:0/3325447118 >> v2:172.21.15.204:6812/28210 conn(0x7f51c40397b0 msgr2=0x7f51c403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 1 --2- 172.21.15.204:0/3325447118 >> v2:172.21.15.204:6812/28210 conn(0x7f51c40397b0 0x7f51c403bc60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006f40 tx=0x7f51cc008040 comp rx=0 tx=0).stop 2024-09-15T01:34:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 10 monclient: shutdown 2024-09-15T01:34:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 1 -- 172.21.15.204:0/3325447118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d806eea0 msgr2=0x7f51d813c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 1 --2- 172.21.15.204:0/3325447118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d806eea0 0x7f51d813c140 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f51d000bb80 tx=0x7f51d00097b0 comp rx=0 tx=0).stop 2024-09-15T01:34:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 1 -- 172.21.15.204:0/3325447118 shutdown_connections 2024-09-15T01:34:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 1 --2- 172.21.15.204:0/3325447118 >> v2:172.21.15.204:6812/28210 conn(0x7f51c40397b0 0x7f51c403bc60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 1 --2- 172.21.15.204:0/3325447118 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f51d80a4680 0x7f51d813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 1 --2- 172.21.15.204:0/3325447118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d809f2d0 0x7f51d813b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 1 --2- 172.21.15.204:0/3325447118 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d806eea0 0x7f51d813c140 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 1 -- 172.21.15.204:0/3325447118 >> 172.21.15.204:0/3325447118 conn(0x7f51d8067740 msgr2=0x7f51d80a25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 1 -- 172.21.15.204:0/3325447118 shutdown_connections 2024-09-15T01:34:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.761+0000 7f51df646640 1 -- 172.21.15.204:0/3325447118 wait complete. 2024-09-15T01:34:54.771 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm osd.9 host2 2024-09-15T01:34:54.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.843+0000 7fda77a34640 10 monclient: get_monmap_and_config 2024-09-15T01:34:54.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.843+0000 7fda77a34640 10 monclient: build_initial_monmap 2024-09-15T01:34:54.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.844+0000 7fda77a34640 10 monclient: monmap: 2024-09-15T01:34:54.845 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:54.845 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:54.845 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:54.844479+0000 2024-09-15T01:34:54.845 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:54.844479+0000 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:54.846 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:54.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.844+0000 7fda77a34640 1 Processor -- start 2024-09-15T01:34:54.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.844+0000 7fda77a34640 1 -- start start 2024-09-15T01:34:54.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.844+0000 7fda77a34640 10 monclient: init 2024-09-15T01:34:54.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda77a34640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:54.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda77a34640 10 monclient: _add_conns ranks=[0,1,3,5,9,8,4,6,7,2,10] 2024-09-15T01:34:54.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda77a34640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fda700a2e30 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:54.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda77a34640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fda7009fb80 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:54.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda77a34640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda7009ff70 0x7fda7013b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:54.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda77a34640 10 monclient(hunting): picked mon.noname-g con 0x7fda7009ff70 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:54.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda77a34640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7004b720 con 0x7fda700a2e30 2024-09-15T01:34:54.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda77a34640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda70133170 con 0x7fda7009fb80 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda77a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda77a34640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7009e6c0 con 0x7fda7009ff70 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda77a34640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda77a34640 10 monclient(hunting): authenticate will time out at 910.487283s 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda74fcf640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fda7009fb80 0x7fda70137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:37378/0 (socket says 172.21.15.204:37378) 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda7009ff70 0x7fda7013b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda74fcf640 1 -- 172.21.15.204:0/2004339384 learned_addr learned my addr 172.21.15.204:0/2004339384 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 10 monclient(hunting): get_auth_request con 0x7fda7009ff70 auth_method 0 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:54.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 10 monclient(hunting): handle_auth_done global_id 14511 payload 274 2024-09-15T01:34:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 1 -- 172.21.15.204:0/2004339384 >> v1:172.21.15.204:6789/0 conn(0x7fda7009fb80 legacy=0x7fda70137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda67fff640 1 -- 172.21.15.204:0/2004339384 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395440771 0 0) 0x7fda70133170 con 0x7fda7009fb80 2024-09-15T01:34:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 1 -- 172.21.15.204:0/2004339384 >> v1:172.21.15.178:6789/0 conn(0x7fda700a2e30 legacy=0x7fda70132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 10 monclient: _finish_hunting 0 2024-09-15T01:34:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 1 monclient: found mon.noname-g 2024-09-15T01:34:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda75fd1640 1 -- 172.21.15.204:0/2004339384 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda700a3200 con 0x7fda7009ff70 2024-09-15T01:34:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.845+0000 7fda67fff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:34:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda75fd1640 1 --2- 172.21.15.204:0/2004339384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda7009ff70 0x7fda7013b450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda68009e90 tx=0x7fda68009780 comp rx=0 tx=0).ready entity=mon.3 client_cookie=a06e754a65f57724 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda67fff640 1 -- 172.21.15.204:0/2004339384 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fda680079d0 con 0x7fda7009ff70 2024-09-15T01:34:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda67fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda67fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda67fff640 10 monclient: dump: 2024-09-15T01:34:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda67fff640 10 monclient: _finish_auth 0 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda67fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda67fff640 1 -- 172.21.15.204:0/2004339384 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6802d050 con 0x7fda7009ff70 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda77a34640 5 monclient: authenticate success, global_id 14511 2024-09-15T01:34:54.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda77a34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda67fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda67fff640 1 -- 172.21.15.204:0/2004339384 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fda68007b90 con 0x7fda7009ff70 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda77a34640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda77a34640 4 set_mon_vals no callback set 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda67fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda67fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.846+0000 7fda67fff640 10 monclient: dump: 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:54.859 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:54.860 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:54.860 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:54.860 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:54.860 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:54.860 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:54.860 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:54.860 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:54.860 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:54.860 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:54.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.847+0000 7fda77a34640 10 monclient: shutdown 2024-09-15T01:34:54.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.847+0000 7fda77a34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:54.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.847+0000 7fda77a34640 1 -- 172.21.15.204:0/2004339384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda7009ff70 msgr2=0x7fda7013b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.847+0000 7fda77a34640 1 --2- 172.21.15.204:0/2004339384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda7009ff70 0x7fda7013b450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda68009e90 tx=0x7fda68009780 comp rx=0 tx=0).stop 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.847+0000 7fda77a34640 1 -- 172.21.15.204:0/2004339384 shutdown_connections 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.847+0000 7fda77a34640 1 --2- 172.21.15.204:0/2004339384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fda7009ff70 0x7fda7013b450 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.847+0000 7fda77a34640 1 -- 172.21.15.204:0/2004339384 >> 172.21.15.204:0/2004339384 conn(0x7fda7006e860 msgr2=0x7fda7006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.847+0000 7fda77a34640 1 -- 172.21.15.204:0/2004339384 shutdown_connections 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.847+0000 7fda77a34640 1 -- 172.21.15.204:0/2004339384 wait complete. 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.847+0000 7fda77a34640 10 monclient: build_initial_monmap 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.847+0000 7fda77a34640 10 monclient: monmap: 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:54.861 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 1 Processor -- start 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 1 -- start start 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 10 monclient: init 2024-09-15T01:34:54.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 10 monclient: _add_conns ranks=[5,1,0,6,2,3,4,8,7] 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fda7009fb80 0x7fda70131740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 10 monclient(hunting): picked mon.noname-h con 0x7fda7009fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda7009ff70 0x7fda7011e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 10 monclient(hunting): picked mon.noname-f con 0x7fda7009ff70 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda700a2e30 0x7fda7011e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 10 monclient(hunting): picked mon.noname-a con 0x7fda700a2e30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7009ca40 con 0x7fda700a2e30 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7009d650 con 0x7fda7009ff70 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda70133f40 con 0x7fda7009fb80 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:54.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda77a34640 10 monclient(hunting): authenticate will time out at 910.490679s 2024-09-15T01:34:54.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda74fcf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda7009ff70 0x7fda7011e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:54.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda757d0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fda7009fb80 0x7fda70131740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:54.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda74fcf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda7009ff70 0x7fda7011e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:46800/0 (socket says 172.21.15.204:46800) 2024-09-15T01:34:54.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda74fcf640 1 -- 172.21.15.204:0/1077784452 learned_addr learned my addr 172.21.15.204:0/1077784452 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:54.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda757d0640 1 --2- 172.21.15.204:0/1077784452 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fda7009fb80 0x7fda70131740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:53436/0 (socket says 172.21.15.204:53436) 2024-09-15T01:34:54.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda74fcf640 10 monclient(hunting): get_auth_request con 0x7fda7009ff70 auth_method 0 2024-09-15T01:34:54.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda74fcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:54.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda74fcf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:54.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda74fcf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:54.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda757d0640 10 monclient(hunting): get_auth_request con 0x7fda7009fb80 auth_method 0 2024-09-15T01:34:54.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda757d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:54.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda757d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:54.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda757d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:54.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda74fcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:54.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda74fcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:54.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda74fcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:54.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda757d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:54.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda757d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:54.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda757d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:54.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda75fd1640 1 --2- 172.21.15.204:0/1077784452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda700a2e30 0x7fda7011e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:54.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda75fd1640 10 monclient(hunting): get_auth_request con 0x7fda700a2e30 auth_method 0 2024-09-15T01:34:54.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda75fd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:54.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda75fd1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.848+0000 7fda75fd1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda75fd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda75fd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda75fd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda757d0640 10 monclient(hunting): handle_auth_done global_id 5396 payload 824 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda757d0640 1 -- 172.21.15.204:0/1077784452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda7009ff70 msgr2=0x7fda7011e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda757d0640 1 --2- 172.21.15.204:0/1077784452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda7009ff70 0x7fda7011e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda757d0640 1 -- 172.21.15.204:0/1077784452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda700a2e30 msgr2=0x7fda7011e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda757d0640 1 --2- 172.21.15.204:0/1077784452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda700a2e30 0x7fda7011e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda757d0640 10 monclient: _finish_hunting 0 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda757d0640 1 monclient: found mon.noname-h 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda757d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:54.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda757d0640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda757d0640 1 -- 172.21.15.204:0/1077784452 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda68009a20 con 0x7fda7009fb80 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda74fcf640 1 --2- 172.21.15.204:0/1077784452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda7009ff70 0x7fda7011e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda757d0640 1 --2- 172.21.15.204:0/1077784452 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fda7009fb80 0x7fda70131740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00b490 tx=0x7fda6c006f50 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda657fa640 1 -- 172.21.15.204:0/1077784452 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fda6c00f990 con 0x7fda7009fb80 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda657fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda657fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda657fa640 10 monclient: dump: 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:54.867 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda657fa640 10 monclient: _finish_auth 0 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda657fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda657fa640 1 -- 172.21.15.204:0/1077784452 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c00fb10 con 0x7fda7009fb80 2024-09-15T01:34:54.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda77a34640 5 monclient: authenticate success, global_id 5396 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda657fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda657fa640 1 -- 172.21.15.204:0/1077784452 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fda6c00fc80 con 0x7fda7009fb80 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda77a34640 10 monclient: _renew_subs 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda77a34640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda77a34640 1 -- 172.21.15.204:0/1077784452 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda70120930 con 0x7fda7009fb80 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda657fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda657fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda677fe640 4 set_mon_vals no callback set 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda657fa640 10 monclient: dump: 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:54.869 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda77a34640 10 monclient: _renew_subs 2024-09-15T01:34:54.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda77a34640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:54.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.849+0000 7fda77a34640 1 -- 172.21.15.204:0/1077784452 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7011fd60 con 0x7fda7009fb80 2024-09-15T01:34:54.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.850+0000 7fda657fa640 1 -- 172.21.15.204:0/1077784452 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fda6c00f510 con 0x7fda7009fb80 2024-09-15T01:34:54.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.850+0000 7fda77a34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:54.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.850+0000 7fda77a34640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:54.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.850+0000 7fda77a34640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:54.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.850+0000 7fda77a34640 1 -- 172.21.15.204:0/1077784452 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda70121a30 con 0x7fda7009fb80 2024-09-15T01:34:54.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.850+0000 7fda657fa640 1 --2- 172.21.15.204:0/1077784452 >> v2:172.21.15.204:6812/28210 conn(0x7fda540397b0 0x7fda5403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:54.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.850+0000 7fda657fa640 1 -- 172.21.15.204:0/1077784452 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4871+0+0 (secure 0 0 0) 0x7fda6c04a8f0 con 0x7fda7009fb80 2024-09-15T01:34:54.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.850+0000 7fda74fcf640 1 --2- 172.21.15.204:0/1077784452 >> v2:172.21.15.204:6812/28210 conn(0x7fda540397b0 0x7fda5403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:54.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.850+0000 7fda74fcf640 10 monclient: get_auth_request con 0x7fda540397b0 auth_method 0 2024-09-15T01:34:54.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.851+0000 7fda74fcf640 1 --2- 172.21.15.204:0/1077784452 >> v2:172.21.15.204:6812/28210 conn(0x7fda540397b0 0x7fda5403bc60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fda60001e20 tx=0x7fda60009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:54.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.855+0000 7fda657fa640 1 -- 172.21.15.204:0/1077784452 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fda6c017aa0 con 0x7fda7009fb80 2024-09-15T01:34:54.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.855+0000 7fda657fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:54.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:54.855+0000 7fda657fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:55.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.016+0000 7fda77a34640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-09-15T01:34:55.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.016+0000 7fda77a34640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-09-15T01:34:55.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.016+0000 7fda77a34640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:55.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.016+0000 7fda77a34640 1 -- 172.21.15.204:0/1077784452 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"} v 0) v1 -- 0x7fda70070cc0 con 0x7fda7009fb80 2024-09-15T01:34:55.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.665+0000 7fda657fa640 1 -- 172.21.15.204:0/1077784452 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}]=0 removed item id 9 name 'osd.9' from crush map v92) v1 ==== 143+0+0 (secure 0 0 0) 0x7fda6c017c80 con 0x7fda7009fb80 2024-09-15T01:34:55.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.665+0000 7fda657fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-09-15T01:34:55.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.665+0000 7fda657fa640 10 monclient: _finish_command 2 = system:0 removed item id 9 name 'osd.9' from crush map 2024-09-15T01:34:55.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.765+0000 7fda77a34640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-09-15T01:34:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.765+0000 7fda77a34640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-09-15T01:34:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.765+0000 7fda77a34640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.765+0000 7fda77a34640 1 -- 172.21.15.204:0/1077784452 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"} v 0) v1 -- 0x7fda700a1f10 con 0x7fda7009fb80 2024-09-15T01:34:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.766+0000 7fda657fa640 1 -- 172.21.15.204:0/1077784452 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}]=0 device 'osd.9' does not appear in the crush map v92) v1 ==== 145+0+0 (secure 0 0 0) 0x7fda6c018310 con 0x7fda7009fb80 2024-09-15T01:34:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.766+0000 7fda657fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-09-15T01:34:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.766+0000 7fda657fa640 10 monclient: _finish_command 3 = system:0 device 'osd.9' does not appear in the crush map 2024-09-15T01:34:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:device 'osd.9' does not appear in the crush map 2024-09-15T01:34:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 1 -- 172.21.15.204:0/1077784452 >> v2:172.21.15.204:6812/28210 conn(0x7fda540397b0 msgr2=0x7fda5403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 1 --2- 172.21.15.204:0/1077784452 >> v2:172.21.15.204:6812/28210 conn(0x7fda540397b0 0x7fda5403bc60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fda60001e20 tx=0x7fda60009040 comp rx=0 tx=0).stop 2024-09-15T01:34:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 10 monclient: shutdown 2024-09-15T01:34:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 1 -- 172.21.15.204:0/1077784452 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fda7009fb80 msgr2=0x7fda70131740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 1 --2- 172.21.15.204:0/1077784452 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fda7009fb80 0x7fda70131740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00b490 tx=0x7fda6c006f50 comp rx=0 tx=0).stop 2024-09-15T01:34:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 1 -- 172.21.15.204:0/1077784452 shutdown_connections 2024-09-15T01:34:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 1 --2- 172.21.15.204:0/1077784452 >> v2:172.21.15.204:6812/28210 conn(0x7fda540397b0 0x7fda5403bc60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 1 --2- 172.21.15.204:0/1077784452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda700a2e30 0x7fda7011e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 1 --2- 172.21.15.204:0/1077784452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda7009ff70 0x7fda7011e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 1 --2- 172.21.15.204:0/1077784452 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fda7009fb80 0x7fda70131740 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 1 -- 172.21.15.204:0/1077784452 >> 172.21.15.204:0/1077784452 conn(0x7fda7006e860 msgr2=0x7fda700a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 1 -- 172.21.15.204:0/1077784452 shutdown_connections 2024-09-15T01:34:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.768+0000 7fda77a34640 1 -- 172.21.15.204:0/1077784452 wait complete. 2024-09-15T01:34:55.778 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm host2 2024-09-15T01:34:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.850+0000 7f06bec17640 10 monclient: get_monmap_and_config 2024-09-15T01:34:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.850+0000 7f06bec17640 10 monclient: build_initial_monmap 2024-09-15T01:34:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.850+0000 7f06bec17640 10 monclient: monmap: 2024-09-15T01:34:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:55.850653+0000 2024-09-15T01:34:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:55.850653+0000 2024-09-15T01:34:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.850+0000 7f06bec17640 1 Processor -- start 2024-09-15T01:34:55.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.850+0000 7f06bec17640 1 -- start start 2024-09-15T01:34:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.850+0000 7f06bec17640 10 monclient: init 2024-09-15T01:34:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 10 monclient: _add_conns ranks=[1,3,9,5,6,0,8,10,7,4,2] 2024-09-15T01:34:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f06b80a2e30 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06b809fb80 0x7f06b809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 10 monclient(hunting): picked mon.noname-g con 0x7f06b809fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f06b8137d20 0x7f06b813a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 10 monclient(hunting): picked mon.noname-a con 0x7f06b8137d20 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b804b720 con 0x7f06b80a2e30 2024-09-15T01:34:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b809e6c0 con 0x7f06b8137d20 2024-09-15T01:34:55.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b80a04b0 con 0x7f06b809fb80 2024-09-15T01:34:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bec17640 10 monclient(hunting): authenticate will time out at 911.493483s 2024-09-15T01:34:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06b809fb80 0x7f06b809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bc9b3640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f06b80a2e30 0x7f06b8132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:37382/0 (socket says 172.21.15.204:37382) 2024-09-15T01:34:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bc9b3640 1 -- 172.21.15.204:0/1593188857 learned_addr learned my addr 172.21.15.204:0/1593188857 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:34:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 10 monclient(hunting): get_auth_request con 0x7f06b809fb80 auth_method 0 2024-09-15T01:34:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:55.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bd1b4640 1 --2- 172.21.15.204:0/1593188857 >> v2:172.21.15.178:3300/0 conn(0x7f06b8137d20 0x7f06b813a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b77fe640 1 -- 172.21.15.204:0/1593188857 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321491751 0 0) 0x7f06b804b720 con 0x7f06b80a2e30 2024-09-15T01:34:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b77fe640 10 monclient(hunting): my global_id is 5887 2024-09-15T01:34:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b77fe640 1 -- 172.21.15.204:0/1593188857 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06a00034e0 con 0x7f06b80a2e30 2024-09-15T01:34:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 10 monclient(hunting): handle_auth_done global_id 14520 payload 274 2024-09-15T01:34:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 1 -- 172.21.15.204:0/1593188857 >> v2:172.21.15.178:3300/0 conn(0x7f06b8137d20 msgr2=0x7f06b813a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 1 --2- 172.21.15.204:0/1593188857 >> v2:172.21.15.178:3300/0 conn(0x7f06b8137d20 0x7f06b813a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:55.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 1 -- 172.21.15.204:0/1593188857 >> v1:172.21.15.204:6789/0 conn(0x7f06b80a2e30 legacy=0x7f06b8132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 1 monclient: found mon.noname-g 2024-09-15T01:34:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06b7fff640 1 -- 172.21.15.204:0/1593188857 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b8133170 con 0x7f06b809fb80 2024-09-15T01:34:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bd1b4640 10 monclient: get_auth_request con 0x7f06b8137d20 auth_method 0 2024-09-15T01:34:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.851+0000 7f06bd1b4640 1 --2- 172.21.15.204:0/1593188857 >> v2:172.21.15.178:3300/0 conn(0x7f06b8137d20 0x7f06b813a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:34:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b7fff640 1 --2- 172.21.15.204:0/1593188857 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06b809fb80 0x7f06b809ff70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009850 tx=0x7f06ac0090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=ff031018c82349f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b77fe640 1 -- 172.21.15.204:0/1593188857 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f06ac02ac80 con 0x7f06b809fb80 2024-09-15T01:34:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b77fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:55.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b77fe640 10 monclient: dump: 2024-09-15T01:34:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:55.867 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:55.868 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b77fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b77fe640 1 -- 172.21.15.204:0/1593188857 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ac02ae00 con 0x7f06b809fb80 2024-09-15T01:34:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06bec17640 5 monclient: authenticate success, global_id 14520 2024-09-15T01:34:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06bec17640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:55.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06bec17640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b77fe640 1 -- 172.21.15.204:0/1593188857 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f06ac031ca0 con 0x7f06b809fb80 2024-09-15T01:34:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06bec17640 4 set_mon_vals no callback set 2024-09-15T01:34:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.852+0000 7f06b77fe640 10 monclient: dump: 2024-09-15T01:34:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:55.869 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.853+0000 7f06bec17640 10 monclient: shutdown 2024-09-15T01:34:55.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.853+0000 7f06bec17640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.853+0000 7f06bec17640 1 -- 172.21.15.204:0/1593188857 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06b809fb80 msgr2=0x7f06b809ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.853+0000 7f06bec17640 1 --2- 172.21.15.204:0/1593188857 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06b809fb80 0x7f06b809ff70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009850 tx=0x7f06ac0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.853+0000 7f06bec17640 1 -- 172.21.15.204:0/1593188857 shutdown_connections 2024-09-15T01:34:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.853+0000 7f06bec17640 1 --2- 172.21.15.204:0/1593188857 >> v2:172.21.15.178:3300/0 conn(0x7f06b8137d20 0x7f06b813a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.853+0000 7f06bec17640 1 --2- 172.21.15.204:0/1593188857 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06b809fb80 0x7f06b809ff70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.853+0000 7f06bec17640 1 -- 172.21.15.204:0/1593188857 >> 172.21.15.204:0/1593188857 conn(0x7f06b806e860 msgr2=0x7f06b806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.853+0000 7f06bec17640 1 -- 172.21.15.204:0/1593188857 shutdown_connections 2024-09-15T01:34:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.853+0000 7f06bec17640 1 -- 172.21.15.204:0/1593188857 wait complete. 2024-09-15T01:34:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.853+0000 7f06bec17640 10 monclient: build_initial_monmap 2024-09-15T01:34:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.853+0000 7f06bec17640 10 monclient: monmap: 2024-09-15T01:34:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:55.873 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 1 Processor -- start 2024-09-15T01:34:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 1 -- start start 2024-09-15T01:34:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 10 monclient: init 2024-09-15T01:34:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 10 monclient: _add_conns ranks=[8,2,4,7,0,3,6,1,5] 2024-09-15T01:34:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f06b809fb80 0x7f06b8132ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 10 monclient(hunting): picked mon.noname-e con 0x7f06b809fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06b80a2e30 0x7f06b809a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 10 monclient(hunting): picked mon.noname-b con 0x7f06b80a2e30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f06b8137d20 0x7f06b809ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 10 monclient(hunting): picked mon.noname-c con 0x7f06b8137d20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b809c870 con 0x7f06b80a2e30 2024-09-15T01:34:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06b809ca70 con 0x7f06b8137d20 2024-09-15T01:34:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f06b811dac0 con 0x7f06b809fb80 2024-09-15T01:34:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.854+0000 7f06bec17640 10 monclient(hunting): authenticate will time out at 911.496990s 2024-09-15T01:34:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06b7fff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06b80a2e30 msgr2=0x7f06b809a820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06b7fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06b80a2e30 0x7f06b809a820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f06b809fb80 0x7f06b8132ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bd1b4640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f06b8137d20 0x7f06b809ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f06b809fb80 0x7f06b8132ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:43128/0 (socket says 172.21.15.204:43128) 2024-09-15T01:34:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 1 -- 172.21.15.204:0/2217378249 learned_addr learned my addr 172.21.15.204:0/2217378249 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 10 monclient(hunting): get_auth_request con 0x7f06b809fb80 auth_method 0 2024-09-15T01:34:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bd1b4640 10 monclient(hunting): get_auth_request con 0x7f06b8137d20 auth_method 0 2024-09-15T01:34:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bd1b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bd1b4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bd1b4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bd1b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bd1b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bd1b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 10 monclient(hunting): handle_auth_done global_id 5426 payload 824 2024-09-15T01:34:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 1 -- 172.21.15.204:0/2217378249 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f06b8137d20 msgr2=0x7f06b809ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 1 --2- 172.21.15.204:0/2217378249 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f06b8137d20 0x7f06b809ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 1 -- 172.21.15.204:0/2217378249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06b80a2e30 msgr2=0x7f06b809a820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 1 --2- 172.21.15.204:0/2217378249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06b80a2e30 0x7f06b809a820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 10 monclient: _finish_hunting 0 2024-09-15T01:34:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 1 monclient: found mon.noname-e 2024-09-15T01:34:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 1 -- 172.21.15.204:0/2217378249 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac009390 con 0x7f06b809fb80 2024-09-15T01:34:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bd1b4640 1 --2- 172.21.15.204:0/2217378249 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f06b8137d20 0x7f06b809ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.855+0000 7f06bc9b3640 1 --2- 172.21.15.204:0/2217378249 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f06b809fb80 0x7f06b8132ae0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f06a800c560 tx=0x7f06a8007d60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b4ff9640 1 -- 172.21.15.204:0/2217378249 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f06a80070d0 con 0x7f06b809fb80 2024-09-15T01:34:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b4ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b4ff9640 10 monclient: dump: 2024-09-15T01:34:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:55.879 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:55.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b4ff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:55.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:55.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06bec17640 5 monclient: authenticate success, global_id 5426 2024-09-15T01:34:55.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06bec17640 10 monclient: _renew_subs 2024-09-15T01:34:55.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b4ff9640 1 -- 172.21.15.204:0/2217378249 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a8007290 con 0x7f06b809fb80 2024-09-15T01:34:55.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06bec17640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:55.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06bec17640 1 -- 172.21.15.204:0/2217378249 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b811e050 con 0x7f06b809fb80 2024-09-15T01:34:55.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:55.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b4ff9640 1 -- 172.21.15.204:0/2217378249 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f06a8011360 con 0x7f06b809fb80 2024-09-15T01:34:55.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:55.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b4ff9640 10 monclient: dump: 2024-09-15T01:34:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:55.881 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06bec17640 10 monclient: _renew_subs 2024-09-15T01:34:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06bec17640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06bec17640 1 -- 172.21.15.204:0/2217378249 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b809c2c0 con 0x7f06b809fb80 2024-09-15T01:34:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.856+0000 7f06b6ffd640 4 set_mon_vals no callback set 2024-09-15T01:34:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.857+0000 7f06b4ff9640 1 -- 172.21.15.204:0/2217378249 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f06a8007400 con 0x7f06b809fb80 2024-09-15T01:34:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.857+0000 7f06bec17640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.857+0000 7f06b4ff9640 1 --2- 172.21.15.204:0/2217378249 >> v2:172.21.15.204:6812/28210 conn(0x7f06a40397b0 0x7f06a403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.857+0000 7f06bec17640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.857+0000 7f06bec17640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.857+0000 7f06bec17640 1 -- 172.21.15.204:0/2217378249 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b8070cc0 con 0x7f06b809fb80 2024-09-15T01:34:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.857+0000 7f06b4ff9640 1 -- 172.21.15.204:0/2217378249 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4850+0+0 (secure 0 0 0) 0x7f06a804a600 con 0x7f06b809fb80 2024-09-15T01:34:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.857+0000 7f06b7fff640 1 --2- 172.21.15.204:0/2217378249 >> v2:172.21.15.204:6812/28210 conn(0x7f06a40397b0 0x7f06a403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.857+0000 7f06b7fff640 10 monclient: get_auth_request con 0x7f06a40397b0 auth_method 0 2024-09-15T01:34:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.857+0000 7f06b7fff640 1 --2- 172.21.15.204:0/2217378249 >> v2:172.21.15.204:6812/28210 conn(0x7f06a40397b0 0x7f06a403bc60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f06ac034000 tx=0x7f06ac032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.862+0000 7f06b4ff9640 1 -- 172.21.15.204:0/2217378249 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f06a8017070 con 0x7f06b809fb80 2024-09-15T01:34:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.862+0000 7f06b4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:55.862+0000 7f06b4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.020+0000 7f06bec17640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host2"}] 2024-09-15T01:34:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.021+0000 7f06bec17640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host2"}] 2024-09-15T01:34:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.021+0000 7f06bec17640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.021+0000 7f06bec17640 1 -- 172.21.15.204:0/2217378249 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush rm", "name": "host2"} v 0) v1 -- 0x7f06b80008d0 con 0x7f06b809fb80 2024-09-15T01:34:56.444 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.443+0000 7f06b4ff9640 1 -- 172.21.15.204:0/2217378249 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host2"}]=0 removed item id -16 name 'host2' from crush map v93) v1 ==== 124+0+0 (secure 0 0 0) 0x7f06a80076b0 con 0x7f06b809fb80 2024-09-15T01:34:56.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.443+0000 7f06b4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host2"}] 2024-09-15T01:34:56.445 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.443+0000 7f06b4ff9640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'host2' from crush map 2024-09-15T01:34:56.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.541+0000 7f06bec17640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host2"}] 2024-09-15T01:34:56.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.541+0000 7f06bec17640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "host2"}] 2024-09-15T01:34:56.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.541+0000 7f06bec17640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:34:56.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.541+0000 7f06bec17640 1 -- 172.21.15.204:0/2217378249 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush rm", "name": "host2"} v 0) v1 -- 0x7f06b8067590 con 0x7f06b809fb80 2024-09-15T01:34:56.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.542+0000 7f06b4ff9640 1 -- 172.21.15.204:0/2217378249 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host2"}]=0 device 'host2' does not appear in the crush map v93) v1 ==== 124+0+0 (secure 0 0 0) 0x7f06a800e290 con 0x7f06b809fb80 2024-09-15T01:34:56.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.542+0000 7f06b4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "host2"}] 2024-09-15T01:34:56.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.542+0000 7f06b4ff9640 10 monclient: _finish_command 3 = system:0 device 'host2' does not appear in the crush map 2024-09-15T01:34:56.543 INFO:tasks.workunit.client.0.smithi204.stderr:device 'host2' does not appear in the crush map 2024-09-15T01:34:56.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.543+0000 7f06bec17640 1 -- 172.21.15.204:0/2217378249 >> v2:172.21.15.204:6812/28210 conn(0x7f06a40397b0 msgr2=0x7f06a403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:56.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.543+0000 7f06bec17640 1 --2- 172.21.15.204:0/2217378249 >> v2:172.21.15.204:6812/28210 conn(0x7f06a40397b0 0x7f06a403bc60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f06ac034000 tx=0x7f06ac032000 comp rx=0 tx=0).stop 2024-09-15T01:34:56.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.543+0000 7f06bec17640 10 monclient: shutdown 2024-09-15T01:34:56.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.543+0000 7f06bec17640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:56.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.543+0000 7f06bec17640 1 -- 172.21.15.204:0/2217378249 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f06b809fb80 msgr2=0x7f06b8132ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:56.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.543+0000 7f06bec17640 1 --2- 172.21.15.204:0/2217378249 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f06b809fb80 0x7f06b8132ae0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f06a800c560 tx=0x7f06a8007d60 comp rx=0 tx=0).stop 2024-09-15T01:34:56.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.543+0000 7f06bec17640 1 -- 172.21.15.204:0/2217378249 shutdown_connections 2024-09-15T01:34:56.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.543+0000 7f06bec17640 1 --2- 172.21.15.204:0/2217378249 >> v2:172.21.15.204:6812/28210 conn(0x7f06a40397b0 0x7f06a403bc60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:56.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.543+0000 7f06bec17640 1 --2- 172.21.15.204:0/2217378249 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f06b8137d20 0x7f06b809ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:56.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.543+0000 7f06bec17640 1 --2- 172.21.15.204:0/2217378249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06b80a2e30 0x7f06b809a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:56.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.543+0000 7f06bec17640 1 --2- 172.21.15.204:0/2217378249 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f06b809fb80 0x7f06b8132ae0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:56.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.543+0000 7f06bec17640 1 -- 172.21.15.204:0/2217378249 >> 172.21.15.204:0/2217378249 conn(0x7f06b806e860 msgr2=0x7f06b80a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:56.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.544+0000 7f06bec17640 1 -- 172.21.15.204:0/2217378249 shutdown_connections 2024-09-15T01:34:56.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.544+0000 7f06bec17640 1 -- 172.21.15.204:0/2217378249 wait complete. 2024-09-15T01:34:56.554 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket foo host 2024-09-15T01:34:56.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.628+0000 7f711ae4f640 10 monclient: get_monmap_and_config 2024-09-15T01:34:56.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.628+0000 7f711ae4f640 10 monclient: build_initial_monmap 2024-09-15T01:34:56.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.628+0000 7f711ae4f640 10 monclient: monmap: 2024-09-15T01:34:56.629 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:56.628600+0000 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:56.628600+0000 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:56.630 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:56.631 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:56.631 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:56.631 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:56.631 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:56.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.628+0000 7f711ae4f640 1 Processor -- start 2024-09-15T01:34:56.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.628+0000 7f711ae4f640 1 -- start start 2024-09-15T01:34:56.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.628+0000 7f711ae4f640 10 monclient: init 2024-09-15T01:34:56.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:56.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 10 monclient: _add_conns ranks=[8,7,1,4,9,2,3,10,6,0,5] 2024-09-15T01:34:56.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7114132aa0 0x7f7114132e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:56.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 10 monclient(hunting): picked mon.noname-e con 0x7f7114132aa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:56.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f711406eef0 0x7f71141333b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:56.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 10 monclient(hunting): picked mon.noname-i con 0x7f711406eef0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:56.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f711412f680 addr v1:172.21.15.204:6789/0 2024-09-15T01:34:56.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711404b720 con 0x7f711412f680 2024-09-15T01:34:56.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:56.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f71140692f0 con 0x7f711406eef0 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f71141338f0 con 0x7f7114132aa0 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f711ae4f640 10 monclient(hunting): authenticate will time out at 912.271489s 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f7113fff640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f711406eef0 msgr2=0x7f71141333b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f7113fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f711406eef0 0x7f71141333b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f71193ec640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f711412f680 0x7f711412fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:37398/0 (socket says 172.21.15.204:37398) 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f71193ec640 1 -- 172.21.15.204:0/2941346876 learned_addr learned my addr 172.21.15.204:0/2941346876 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f7118beb640 1 --2- 172.21.15.204:0/2941346876 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7114132aa0 0x7f7114132e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f7118beb640 10 monclient(hunting): get_auth_request con 0x7f7114132aa0 auth_method 0 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f7118beb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f7118beb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f7118beb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:56.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f7118beb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f7118beb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f7118beb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f71137fe640 1 -- 172.21.15.204:0/2941346876 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240069074 0 0) 0x7f711404b720 con 0x7f711412f680 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f71137fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f71137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f71137fe640 10 monclient(hunting): my global_id is 5896 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f71137fe640 1 -- 172.21.15.204:0/2941346876 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71000034e0 con 0x7f711412f680 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f7118beb640 10 monclient(hunting): handle_auth_done global_id 5435 payload 274 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.629+0000 7f7118beb640 1 -- 172.21.15.204:0/2941346876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f711406eef0 msgr2=0x7f71141333b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.630+0000 7f7118beb640 1 --2- 172.21.15.204:0/2941346876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f711406eef0 0x7f71141333b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.630+0000 7f7118beb640 1 -- 172.21.15.204:0/2941346876 >> v1:172.21.15.204:6789/0 conn(0x7f711412f680 legacy=0x7f711412fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.630+0000 7f7118beb640 10 monclient: _finish_hunting 0 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.630+0000 7f7118beb640 1 monclient: found mon.noname-e 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.630+0000 7f7118beb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:56.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.630+0000 7f7118beb640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.630+0000 7f7118beb640 1 -- 172.21.15.204:0/2941346876 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71140a0350 con 0x7f7114132aa0 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.630+0000 7f7118beb640 1 --2- 172.21.15.204:0/2941346876 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7114132aa0 0x7f7114132e70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f710c009880 tx=0x7f710c0090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=868e51b838f71cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f71137fe640 1 -- 172.21.15.204:0/2941346876 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f710c02ad00 con 0x7f7114132aa0 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f71137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f71137fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f71137fe640 10 monclient: dump: 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:56.642 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f71137fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f71137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 5 monclient: authenticate success, global_id 5435 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f71137fe640 1 -- 172.21.15.204:0/2941346876 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c02ae80 con 0x7f7114132aa0 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f71137fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:56.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 4 set_mon_vals no callback set 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f71137fe640 1 -- 172.21.15.204:0/2941346876 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f710c031ca0 con 0x7f7114132aa0 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f71137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f71137fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f71137fe640 10 monclient: dump: 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:56.644 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 10 monclient: shutdown 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 1 -- 172.21.15.204:0/2941346876 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7114132aa0 msgr2=0x7f7114132e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 1 --2- 172.21.15.204:0/2941346876 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7114132aa0 0x7f7114132e70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f710c009880 tx=0x7f710c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 1 -- 172.21.15.204:0/2941346876 shutdown_connections 2024-09-15T01:34:56.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 1 --2- 172.21.15.204:0/2941346876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f711406eef0 0x7f71141333b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 1 --2- 172.21.15.204:0/2941346876 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7114132aa0 0x7f7114132e70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 1 -- 172.21.15.204:0/2941346876 >> 172.21.15.204:0/2941346876 conn(0x7f7114067c00 msgr2=0x7f71140a3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 1 -- 172.21.15.204:0/2941346876 shutdown_connections 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.631+0000 7f711ae4f640 1 -- 172.21.15.204:0/2941346876 wait complete. 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.632+0000 7f711ae4f640 10 monclient: build_initial_monmap 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.632+0000 7f711ae4f640 10 monclient: monmap: 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:56.646 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.632+0000 7f711ae4f640 1 Processor -- start 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.632+0000 7f711ae4f640 1 -- start start 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.632+0000 7f711ae4f640 10 monclient: init 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:56.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient: _add_conns ranks=[8,5,6,1,7,0,3,4,2] 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711406eef0 0x7f711409dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient(hunting): picked mon.noname-e con 0x7f711406eef0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f711412f680 0x7f711411a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient(hunting): picked mon.noname-h con 0x7f711412f680 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7114132aa0 0x7f711411aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient(hunting): picked mon.noname-d con 0x7f7114132aa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f711412f4c0 con 0x7f711412f680 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f711411d300 con 0x7f7114132aa0 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7114068960 con 0x7f711406eef0 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:56.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient(hunting): authenticate will time out at 912.275163s 2024-09-15T01:34:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f71193ec640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7114132aa0 msgr2=0x7f711411aa70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f711412f680 0x7f711411a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f71193ec640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7114132aa0 0x7f711411aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f711412f680 0x7f711411a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:53442/0 (socket says 172.21.15.204:53442) 2024-09-15T01:34:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 1 -- 172.21.15.204:0/1895136869 learned_addr learned my addr 172.21.15.204:0/1895136869 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 10 monclient(hunting): get_auth_request con 0x7f711412f680 auth_method 0 2024-09-15T01:34:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:56.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7118beb640 1 --2- 172.21.15.204:0/1895136869 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711406eef0 0x7f711409dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7118beb640 10 monclient(hunting): get_auth_request con 0x7f711406eef0 auth_method 0 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7118beb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7118beb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7118beb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 10 monclient(hunting): handle_auth_done global_id 5405 payload 824 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 1 -- 172.21.15.204:0/1895136869 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711406eef0 msgr2=0x7f711409dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 1 --2- 172.21.15.204:0/1895136869 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711406eef0 0x7f711409dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 1 -- 172.21.15.204:0/1895136869 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7114132aa0 msgr2=0x7f711411aa70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 1 --2- 172.21.15.204:0/1895136869 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7114132aa0 0x7f711411aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 1 monclient: found mon.noname-h 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:56.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 1 -- 172.21.15.204:0/1895136869 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710c009390 con 0x7f711412f680 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7113fff640 1 --2- 172.21.15.204:0/1895136869 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f711412f680 0x7f711411a530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f710400bb80 tx=0x7f71040097b0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7110ff9640 1 -- 172.21.15.204:0/1895136869 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f710400cea0 con 0x7f711412f680 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7110ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7110ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7110ff9640 10 monclient: dump: 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:56.651 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7110ff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7110ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 5 monclient: authenticate success, global_id 5405 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7110ff9640 1 -- 172.21.15.204:0/1895136869 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7104006150 con 0x7f711412f680 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7110ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:56.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7110ff9640 1 -- 172.21.15.204:0/1895136869 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7104006740 con 0x7f711412f680 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7110ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7110ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7110ff9640 10 monclient: dump: 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:56.653 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f7112ffd640 4 set_mon_vals no callback set 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient: _renew_subs 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 1 -- 172.21.15.204:0/1895136869 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7114068590 con 0x7f711412f680 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient: _renew_subs 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.633+0000 7f711ae4f640 1 -- 172.21.15.204:0/1895136869 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711411dba0 con 0x7f711412f680 2024-09-15T01:34:56.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.634+0000 7f7110ff9640 1 -- 172.21.15.204:0/1895136869 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7104012020 con 0x7f711412f680 2024-09-15T01:34:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.634+0000 7f7110ff9640 1 --2- 172.21.15.204:0/1895136869 >> v2:172.21.15.204:6812/28210 conn(0x7f70f40397b0 0x7f70f403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.634+0000 7f711ae4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.634+0000 7f711ae4f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.634+0000 7f711ae4f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.634+0000 7f7110ff9640 1 -- 172.21.15.204:0/1895136869 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4715+0+0 (secure 0 0 0) 0x7f710404b430 con 0x7f711412f680 2024-09-15T01:34:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.634+0000 7f711ae4f640 1 -- 172.21.15.204:0/1895136869 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7114070cc0 con 0x7f711412f680 2024-09-15T01:34:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.635+0000 7f7118beb640 1 --2- 172.21.15.204:0/1895136869 >> v2:172.21.15.204:6812/28210 conn(0x7f70f40397b0 0x7f70f403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.635+0000 7f7118beb640 10 monclient: get_auth_request con 0x7f70f40397b0 auth_method 0 2024-09-15T01:34:56.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.635+0000 7f7118beb640 1 --2- 172.21.15.204:0/1895136869 >> v2:172.21.15.204:6812/28210 conn(0x7f70f40397b0 0x7f70f403bc60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f710c034000 tx=0x7f710c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.639+0000 7f7110ff9640 1 -- 172.21.15.204:0/1895136869 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f71040153e0 con 0x7f711412f680 2024-09-15T01:34:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.639+0000 7f7110ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:56.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.639+0000 7f7110ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:56.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.802+0000 7f711ae4f640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-09-15T01:34:56.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.802+0000 7f711ae4f640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-09-15T01:34:56.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.802+0000 7f711ae4f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:56.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:56.802+0000 7f711ae4f640 1 -- 172.21.15.204:0/1895136869 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "host"} v 0) v1 -- 0x7f7114069010 con 0x7f711412f680 2024-09-15T01:34:57.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.674+0000 7f7110ff9640 1 -- 172.21.15.204:0/1895136869 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}]=0 added bucket foo type host to crush map v94) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7104024030 con 0x7f711412f680 2024-09-15T01:34:57.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.674+0000 7f7110ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-09-15T01:34:57.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.674+0000 7f7110ff9640 10 monclient: _finish_command 2 = system:0 added bucket foo type host to crush map 2024-09-15T01:34:57.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.776+0000 7f711ae4f640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-09-15T01:34:57.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.776+0000 7f711ae4f640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-09-15T01:34:57.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.776+0000 7f711ae4f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:57.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.776+0000 7f711ae4f640 1 -- 172.21.15.204:0/1895136869 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "host"} v 0) v1 -- 0x7f711411c3c0 con 0x7f711412f680 2024-09-15T01:34:57.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.776+0000 7f7110ff9640 1 -- 172.21.15.204:0/1895136869 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}]=0 bucket 'foo' already exists v94) v1 ==== 126+0+0 (secure 0 0 0) 0x7f7104024070 con 0x7f711412f680 2024-09-15T01:34:57.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.776+0000 7f7110ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-09-15T01:34:57.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.776+0000 7f7110ff9640 10 monclient: _finish_command 3 = system:0 bucket 'foo' already exists 2024-09-15T01:34:57.778 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'foo' already exists 2024-09-15T01:34:57.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 1 -- 172.21.15.204:0/1895136869 >> v2:172.21.15.204:6812/28210 conn(0x7f70f40397b0 msgr2=0x7f70f403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:57.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 1 --2- 172.21.15.204:0/1895136869 >> v2:172.21.15.204:6812/28210 conn(0x7f70f40397b0 0x7f70f403bc60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f710c034000 tx=0x7f710c032000 comp rx=0 tx=0).stop 2024-09-15T01:34:57.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 10 monclient: shutdown 2024-09-15T01:34:57.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:57.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 1 -- 172.21.15.204:0/1895136869 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f711412f680 msgr2=0x7f711411a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:57.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 1 --2- 172.21.15.204:0/1895136869 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f711412f680 0x7f711411a530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f710400bb80 tx=0x7f71040097b0 comp rx=0 tx=0).stop 2024-09-15T01:34:57.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 1 -- 172.21.15.204:0/1895136869 shutdown_connections 2024-09-15T01:34:57.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 1 --2- 172.21.15.204:0/1895136869 >> v2:172.21.15.204:6812/28210 conn(0x7f70f40397b0 0x7f70f403bc60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:57.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 1 --2- 172.21.15.204:0/1895136869 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7114132aa0 0x7f711411aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:57.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 1 --2- 172.21.15.204:0/1895136869 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f711412f680 0x7f711411a530 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:57.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 1 --2- 172.21.15.204:0/1895136869 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711406eef0 0x7f711409dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:57.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 1 -- 172.21.15.204:0/1895136869 >> 172.21.15.204:0/1895136869 conn(0x7f7114067c00 msgr2=0x7f71140a2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:57.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 1 -- 172.21.15.204:0/1895136869 shutdown_connections 2024-09-15T01:34:57.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.778+0000 7f711ae4f640 1 -- 172.21.15.204:0/1895136869 wait complete. 2024-09-15T01:34:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush move foo root=default rack=localrack 2024-09-15T01:34:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.864+0000 7fd5cf11d640 10 monclient: get_monmap_and_config 2024-09-15T01:34:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.864+0000 7fd5cf11d640 10 monclient: build_initial_monmap 2024-09-15T01:34:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.865+0000 7fd5cf11d640 10 monclient: monmap: 2024-09-15T01:34:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:57.866 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:57.865477+0000 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:57.865477+0000 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:57.867 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:57.868 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.865+0000 7fd5cf11d640 1 Processor -- start 2024-09-15T01:34:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.865+0000 7fd5cf11d640 1 -- start start 2024-09-15T01:34:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.865+0000 7fd5cf11d640 10 monclient: init 2024-09-15T01:34:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 10 monclient: _add_conns ranks=[7,9,5,6,1,8,2,0,4,3,10] 2024-09-15T01:34:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd5c80a2cd0 0x7fd5c8132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 10 monclient(hunting): picked mon.noname-i con 0x7fd5c80a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd5c809fa20 0x7fd5c809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 10 monclient(hunting): picked mon.noname-a con 0x7fd5c809fa20 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 0x7fd5c80a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 10 monclient(hunting): picked mon.noname-h con 0x7fd5c80a08e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c80a30a0 con 0x7fd5c809fa20 2024-09-15T01:34:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80a0330 con 0x7fd5c80a08e0 2024-09-15T01:34:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80a1db0 con 0x7fd5c80a2cd0 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cf11d640 10 monclient(hunting): authenticate will time out at 913.508197s 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cceb9640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd5c80a2cd0 msgr2=0x7fd5c8132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cceb9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd5c80a2cd0 0x7fd5c8132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 0x7fd5c80a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 0x7fd5c80a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:41952/0 (socket says 172.21.15.204:41952) 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 1 -- 172.21.15.204:0/2891939809 learned_addr learned my addr 172.21.15.204:0/2891939809 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 10 monclient(hunting): get_auth_request con 0x7fd5c80a08e0 auth_method 0 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5bffff640 1 --2- 172.21.15.204:0/2891939809 >> v2:172.21.15.178:3300/0 conn(0x7fd5c809fa20 0x7fd5c809fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5bffff640 10 monclient(hunting): get_auth_request con 0x7fd5c809fa20 auth_method 0 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5bffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 10 monclient(hunting): handle_auth_done global_id 5414 payload 274 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 1 -- 172.21.15.204:0/2891939809 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd5c80a2cd0 msgr2=0x7fd5c8132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 1 --2- 172.21.15.204:0/2891939809 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd5c80a2cd0 0x7fd5c8132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 1 -- 172.21.15.204:0/2891939809 >> v2:172.21.15.178:3300/0 conn(0x7fd5c809fa20 msgr2=0x7fd5c809fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 1 --2- 172.21.15.204:0/2891939809 >> v2:172.21.15.178:3300/0 conn(0x7fd5c809fa20 0x7fd5c809fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 10 monclient: _finish_hunting 0 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 1 monclient: found mon.noname-h 2024-09-15T01:34:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.866+0000 7fd5cd6ba640 1 -- 172.21.15.204:0/2891939809 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c80648a0 con 0x7fd5c80a08e0 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5cd6ba640 1 --2- 172.21.15.204:0/2891939809 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 0x7fd5c80a0d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009f20 tx=0x7fd5c0009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=599f606ef66c0f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5bf7fe640 1 -- 172.21.15.204:0/2891939809 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5c0007880 con 0x7fd5c80a08e0 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5bf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5bf7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5bf7fe640 10 monclient: dump: 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5bf7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5bf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5bf7fe640 1 -- 172.21.15.204:0/2891939809 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c002d050 con 0x7fd5c80a08e0 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5cf11d640 5 monclient: authenticate success, global_id 5414 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5cf11d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5bf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5cf11d640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5bf7fe640 1 -- 172.21.15.204:0/2891939809 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5c0030360 con 0x7fd5c80a08e0 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5bf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5cf11d640 4 set_mon_vals no callback set 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5bf7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5bf7fe640 10 monclient: dump: 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5cf11d640 10 monclient: shutdown 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5cf11d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5cf11d640 1 -- 172.21.15.204:0/2891939809 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 msgr2=0x7fd5c80a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.867+0000 7fd5cf11d640 1 --2- 172.21.15.204:0/2891939809 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 0x7fd5c80a0d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009f20 tx=0x7fd5c0009840 comp rx=0 tx=0).stop 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.868+0000 7fd5cf11d640 1 -- 172.21.15.204:0/2891939809 shutdown_connections 2024-09-15T01:34:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.868+0000 7fd5cf11d640 1 --2- 172.21.15.204:0/2891939809 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 0x7fd5c80a0d50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.868+0000 7fd5cf11d640 1 --2- 172.21.15.204:0/2891939809 >> v2:172.21.15.178:3300/0 conn(0x7fd5c809fa20 0x7fd5c809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.868+0000 7fd5cf11d640 1 --2- 172.21.15.204:0/2891939809 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd5c80a2cd0 0x7fd5c8132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.868+0000 7fd5cf11d640 1 -- 172.21.15.204:0/2891939809 >> 172.21.15.204:0/2891939809 conn(0x7fd5c806e860 msgr2=0x7fd5c806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.868+0000 7fd5cf11d640 1 -- 172.21.15.204:0/2891939809 shutdown_connections 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.868+0000 7fd5cf11d640 1 -- 172.21.15.204:0/2891939809 wait complete. 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.868+0000 7fd5cf11d640 10 monclient: build_initial_monmap 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.868+0000 7fd5cf11d640 10 monclient: monmap: 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.868+0000 7fd5cf11d640 1 Processor -- start 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 1 -- start start 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 10 monclient: init 2024-09-15T01:34:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 10 monclient: _add_conns ranks=[1,5,7,3,8,6,2,4,0] 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5c809fa20 0x7fd5c8130d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 10 monclient(hunting): picked mon.noname-f con 0x7fd5c809fa20 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 0x7fd5c811d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 10 monclient(hunting): picked mon.noname-h con 0x7fd5c80a08e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd5c80a2cd0 0x7fd5c811de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 10 monclient(hunting): picked mon.noname-i con 0x7fd5c80a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c8120e80 con 0x7fd5c809fa20 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c809cb60 con 0x7fd5c80a08e0 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c809d3d0 con 0x7fd5c80a2cd0 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cf11d640 10 monclient(hunting): authenticate will time out at 913.511430s 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cd6ba640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd5c80a2cd0 msgr2=0x7fd5c811de00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cd6ba640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd5c80a2cd0 0x7fd5c811de00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 0x7fd5c811d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cceb9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5c809fa20 0x7fd5c8130d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 0x7fd5c811d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:41958/0 (socket says 172.21.15.204:41958) 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 1 -- 172.21.15.204:0/1844367625 learned_addr learned my addr 172.21.15.204:0/1844367625 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 10 monclient(hunting): get_auth_request con 0x7fd5c80a08e0 auth_method 0 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cceb9640 10 monclient(hunting): get_auth_request con 0x7fd5c809fa20 auth_method 0 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cceb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cceb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cceb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cceb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cceb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cceb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 10 monclient(hunting): handle_auth_done global_id 5423 payload 824 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 1 -- 172.21.15.204:0/1844367625 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd5c80a2cd0 msgr2=0x7fd5c811de00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 1 --2- 172.21.15.204:0/1844367625 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd5c80a2cd0 0x7fd5c811de00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 1 -- 172.21.15.204:0/1844367625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5c809fa20 msgr2=0x7fd5c8130d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 1 --2- 172.21.15.204:0/1844367625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5c809fa20 0x7fd5c8130d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 10 monclient: _finish_hunting 0 2024-09-15T01:34:57.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 1 monclient: found mon.noname-h 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 1 -- 172.21.15.204:0/1844367625 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c0009ae0 con 0x7fd5c80a08e0 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5cceb9640 1 --2- 172.21.15.204:0/1844367625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5c809fa20 0x7fd5c8130d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.869+0000 7fd5bffff640 1 --2- 172.21.15.204:0/1844367625 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 0x7fd5c811d8c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800d890 tx=0x7fd5b8008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 1 -- 172.21.15.204:0/1844367625 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5b800de20 con 0x7fd5c80a08e0 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 10 monclient: dump: 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:57.888 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 10 monclient: _finish_auth 0 2024-09-15T01:34:57.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 1 -- 172.21.15.204:0/1844367625 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b800bd50 con 0x7fd5c80a08e0 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5cf11d640 5 monclient: authenticate success, global_id 5423 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5cf11d640 10 monclient: _renew_subs 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5cf11d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5cf11d640 1 -- 172.21.15.204:0/1844367625 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c811fb30 con 0x7fd5c80a08e0 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 1 -- 172.21.15.204:0/1844367625 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5b80148b0 con 0x7fd5c80a08e0 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 10 monclient: dump: 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:57.890 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:57.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5cf11d640 10 monclient: _renew_subs 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5cf11d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5cf11d640 1 -- 172.21.15.204:0/1844367625 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c811f3c0 con 0x7fd5c80a08e0 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5beffd640 4 set_mon_vals no callback set 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.870+0000 7fd5bcff9640 1 -- 172.21.15.204:0/1844367625 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd5b8014430 con 0x7fd5c80a08e0 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.871+0000 7fd5cf11d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.871+0000 7fd5cf11d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.871+0000 7fd5cf11d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.871+0000 7fd5bcff9640 1 --2- 172.21.15.204:0/1844367625 >> v2:172.21.15.204:6812/28210 conn(0x7fd5b40397b0 0x7fd5b403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.871+0000 7fd5cf11d640 1 -- 172.21.15.204:0/1844367625 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c8070b80 con 0x7fd5c80a08e0 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.871+0000 7fd5bcff9640 1 -- 172.21.15.204:0/1844367625 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4742+0+0 (secure 0 0 0) 0x7fd5b804b9c0 con 0x7fd5c80a08e0 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.871+0000 7fd5cceb9640 1 --2- 172.21.15.204:0/1844367625 >> v2:172.21.15.204:6812/28210 conn(0x7fd5b40397b0 0x7fd5b403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.871+0000 7fd5cceb9640 10 monclient: get_auth_request con 0x7fd5b40397b0 auth_method 0 2024-09-15T01:34:57.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.871+0000 7fd5cceb9640 1 --2- 172.21.15.204:0/1844367625 >> v2:172.21.15.204:6812/28210 conn(0x7fd5b40397b0 0x7fd5b403bc60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4003850 tx=0x7fd5c40090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:57.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.875+0000 7fd5bcff9640 1 -- 172.21.15.204:0/1844367625 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd5b8014a30 con 0x7fd5c80a08e0 2024-09-15T01:34:57.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.875+0000 7fd5bcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:57.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:57.875+0000 7fd5bcff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:58.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.034+0000 7fd5cf11d640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-09-15T01:34:58.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.034+0000 7fd5cf11d640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-09-15T01:34:58.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.034+0000 7fd5cf11d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:58.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.034+0000 7fd5cf11d640 1 -- 172.21.15.204:0/1844367625 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]} v 0) v1 -- 0x7fd5c8121260 con 0x7fd5c80a08e0 2024-09-15T01:34:58.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.680+0000 7fd5bcff9640 1 -- 172.21.15.204:0/1844367625 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}]=0 moved item id -10 name 'foo' to location {rack=localrack,root=default} in crush map v95) v1 ==== 204+0+0 (secure 0 0 0) 0x7fd5b8011340 con 0x7fd5c80a08e0 2024-09-15T01:34:58.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.680+0000 7fd5bcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-09-15T01:34:58.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.680+0000 7fd5bcff9640 10 monclient: _finish_command 2 = system:0 moved item id -10 name 'foo' to location {rack=localrack,root=default} in crush map 2024-09-15T01:34:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.780+0000 7fd5cf11d640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-09-15T01:34:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.780+0000 7fd5cf11d640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-09-15T01:34:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.780+0000 7fd5cf11d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:34:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.780+0000 7fd5cf11d640 1 -- 172.21.15.204:0/1844367625 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]} v 0) v1 -- 0x7fd5c80008d0 con 0x7fd5c80a08e0 2024-09-15T01:34:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.781+0000 7fd5bcff9640 1 -- 172.21.15.204:0/1844367625 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}]=0 no need to move item id -10 name 'foo' to location {rack=localrack,root=default} in crush map v95) v1 ==== 214+0+0 (secure 0 0 0) 0x7fd5b8011520 con 0x7fd5c80a08e0 2024-09-15T01:34:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.781+0000 7fd5bcff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-09-15T01:34:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.781+0000 7fd5bcff9640 10 monclient: _finish_command 3 = system:0 no need to move item id -10 name 'foo' to location {rack=localrack,root=default} in crush map 2024-09-15T01:34:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:no need to move item id -10 name 'foo' to location {rack=localrack,root=default} in crush map 2024-09-15T01:34:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 1 -- 172.21.15.204:0/1844367625 >> v2:172.21.15.204:6812/28210 conn(0x7fd5b40397b0 msgr2=0x7fd5b403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 1 --2- 172.21.15.204:0/1844367625 >> v2:172.21.15.204:6812/28210 conn(0x7fd5b40397b0 0x7fd5b403bc60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4003850 tx=0x7fd5c40090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 10 monclient: shutdown 2024-09-15T01:34:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 1 -- 172.21.15.204:0/1844367625 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 msgr2=0x7fd5c811d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 1 --2- 172.21.15.204:0/1844367625 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 0x7fd5c811d8c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800d890 tx=0x7fd5b8008040 comp rx=0 tx=0).stop 2024-09-15T01:34:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 1 -- 172.21.15.204:0/1844367625 shutdown_connections 2024-09-15T01:34:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 1 --2- 172.21.15.204:0/1844367625 >> v2:172.21.15.204:6812/28210 conn(0x7fd5b40397b0 0x7fd5b403bc60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 1 --2- 172.21.15.204:0/1844367625 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd5c80a2cd0 0x7fd5c811de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 1 --2- 172.21.15.204:0/1844367625 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd5c80a08e0 0x7fd5c811d8c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 1 --2- 172.21.15.204:0/1844367625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5c809fa20 0x7fd5c8130d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 1 -- 172.21.15.204:0/1844367625 >> 172.21.15.204:0/1844367625 conn(0x7fd5c806e860 msgr2=0x7fd5c80a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 1 -- 172.21.15.204:0/1844367625 shutdown_connections 2024-09-15T01:34:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.783+0000 7fd5cf11d640 1 -- 172.21.15.204:0/1844367625 wait complete. 2024-09-15T01:34:58.793 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush create-or-move osd.7 1.0 root=default 2024-09-15T01:34:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.867+0000 7feafded4640 10 monclient: get_monmap_and_config 2024-09-15T01:34:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.867+0000 7feafded4640 10 monclient: build_initial_monmap 2024-09-15T01:34:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.868+0000 7feafded4640 10 monclient: monmap: 2024-09-15T01:34:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:58.868498+0000 2024-09-15T01:34:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:58.868498+0000 2024-09-15T01:34:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:58.870 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:58.871 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:58.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.868+0000 7feafded4640 1 Processor -- start 2024-09-15T01:34:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.868+0000 7feafded4640 1 -- start start 2024-09-15T01:34:58.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.868+0000 7feafded4640 10 monclient: init 2024-09-15T01:34:58.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 10 monclient: _add_conns ranks=[4,9,0,8,5,6,2,7,10,1,3] 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feaf80a2cd0 0x7feaf8132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 10 monclient(hunting): picked mon.noname-c con 0x7feaf80a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feaf809fa20 0x7feaf809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 10 monclient(hunting): picked mon.noname-a con 0x7feaf809fa20 addr v2:172.21.15.178:3300/0 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feaf80a08e0 addr v1:172.21.15.178:6789/0 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf80648a0 con 0x7feaf80a08e0 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf80a30a0 con 0x7feaf809fa20 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feaf80a0330 con 0x7feaf80a2cd0 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:58.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feafded4640 10 monclient(hunting): authenticate will time out at 914.511320s 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feaf809fa20 0x7feaf809fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feaf80a2cd0 0x7feaf8132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feaf809fa20 0x7feaf809fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:43392/0 (socket says 172.21.15.204:43392) 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf7fff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7feaf80a08e0 0x7feaf80a0d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.204:43590/0 (socket says 172.21.15.204:43590) 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 1 -- 172.21.15.204:0/2217612062 learned_addr learned my addr 172.21.15.204:0/2217612062 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feaf80a2cd0 0x7feaf8132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:54448/0 (socket says 172.21.15.204:54448) 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 10 monclient(hunting): get_auth_request con 0x7feaf809fa20 auth_method 0 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf77fe640 10 monclient(hunting): get_auth_request con 0x7feaf80a2cd0 auth_method 0 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:58.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:58.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:58.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:58.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:58.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:58.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:58.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:58.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 10 monclient(hunting): handle_auth_done global_id 5643 payload 274 2024-09-15T01:34:58.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 1 -- 172.21.15.204:0/2217612062 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feaf80a2cd0 msgr2=0x7feaf8132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:58.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 1 --2- 172.21.15.204:0/2217612062 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feaf80a2cd0 0x7feaf8132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:58.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 1 -- 172.21.15.204:0/2217612062 >> v1:172.21.15.178:6789/0 conn(0x7feaf80a08e0 legacy=0x7feaf80a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:58.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:58.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 1 monclient: found mon.noname-a 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf6ffd640 1 -- 172.21.15.204:0/2217612062 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf80a1db0 con 0x7feaf809fa20 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.869+0000 7feaf77fe640 1 --2- 172.21.15.204:0/2217612062 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feaf80a2cd0 0x7feaf8132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf6ffd640 1 --2- 172.21.15.204:0/2217612062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaf809fa20 0x7feaf809fdf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feaec009910 tx=0x7feaec0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5be79604131320 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf67fc640 1 -- 172.21.15.204:0/2217612062 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feaec02ad20 con 0x7feaf809fa20 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf67fc640 10 monclient: dump: 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:58.884 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf67fc640 10 monclient: _finish_auth 0 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feafded4640 5 monclient: authenticate success, global_id 5643 2024-09-15T01:34:58.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feafded4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:58.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf67fc640 1 -- 172.21.15.204:0/2217612062 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec031820 con 0x7feaf809fa20 2024-09-15T01:34:58.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:58.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feafded4640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:58.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feafded4640 4 set_mon_vals no callback set 2024-09-15T01:34:58.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf67fc640 1 -- 172.21.15.204:0/2217612062 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feaec02aea0 con 0x7feaf809fa20 2024-09-15T01:34:58.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:58.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:34:58.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.870+0000 7feaf67fc640 10 monclient: dump: 2024-09-15T01:34:58.886 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:58.886 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:58.886 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:58.886 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:58.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.871+0000 7feafded4640 10 monclient: shutdown 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.871+0000 7feafded4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.871+0000 7feafded4640 1 -- 172.21.15.204:0/2217612062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaf809fa20 msgr2=0x7feaf809fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.871+0000 7feafded4640 1 --2- 172.21.15.204:0/2217612062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaf809fa20 0x7feaf809fdf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feaec009910 tx=0x7feaec0091d0 comp rx=0 tx=0).stop 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.871+0000 7feafded4640 1 -- 172.21.15.204:0/2217612062 shutdown_connections 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.871+0000 7feafded4640 1 --2- 172.21.15.204:0/2217612062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaf809fa20 0x7feaf809fdf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.871+0000 7feafded4640 1 --2- 172.21.15.204:0/2217612062 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feaf80a2cd0 0x7feaf8132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.871+0000 7feafded4640 1 -- 172.21.15.204:0/2217612062 >> 172.21.15.204:0/2217612062 conn(0x7feaf806e860 msgr2=0x7feaf806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.871+0000 7feafded4640 1 -- 172.21.15.204:0/2217612062 shutdown_connections 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.871+0000 7feafded4640 1 -- 172.21.15.204:0/2217612062 wait complete. 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.871+0000 7feafded4640 10 monclient: build_initial_monmap 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.871+0000 7feafded4640 10 monclient: monmap: 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:58.888 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:58.889 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 1 Processor -- start 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 1 -- start start 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 10 monclient: init 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 10 monclient: _add_conns ranks=[2,3,5,4,8,7,6,1,0] 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf809fa20 0x7feaf8141a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 10 monclient(hunting): picked mon.noname-b con 0x7feaf809fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80a08e0 0x7feaf8130ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 10 monclient(hunting): picked mon.noname-g con 0x7feaf80a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feaf80a2cd0 0x7feaf811db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 10 monclient(hunting): picked mon.noname-h con 0x7feaf80a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf809d400 con 0x7feaf809fa20 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:58.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf809cb90 con 0x7feaf80a08e0 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8121450 con 0x7feaf80a2cd0 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.872+0000 7feafded4640 10 monclient(hunting): authenticate will time out at 914.514977s 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf77fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf809fa20 msgr2=0x7feaf8141a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf809fa20 0x7feaf8141a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80a08e0 0x7feaf8130ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80a08e0 0x7feaf8130ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58818/0 (socket says 172.21.15.204:58818) 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feaf80a2cd0 0x7feaf811db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 1 -- 172.21.15.204:0/3145854827 learned_addr learned my addr 172.21.15.204:0/3145854827 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 10 monclient(hunting): get_auth_request con 0x7feaf80a08e0 auth_method 0 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:58.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf7fff640 10 monclient(hunting): get_auth_request con 0x7feaf80a2cd0 auth_method 0 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 10 monclient(hunting): handle_auth_done global_id 14529 payload 824 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 1 -- 172.21.15.204:0/3145854827 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feaf80a2cd0 msgr2=0x7feaf811db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 1 --2- 172.21.15.204:0/3145854827 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feaf80a2cd0 0x7feaf811db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:58.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 1 -- 172.21.15.204:0/3145854827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf809fa20 msgr2=0x7feaf8141a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 1 --2- 172.21.15.204:0/3145854827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf809fa20 0x7feaf8141a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 1 monclient: found mon.noname-g 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 1 -- 172.21.15.204:0/3145854827 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8009390 con 0x7feaf80a08e0 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf7fff640 1 --2- 172.21.15.204:0/3145854827 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feaf80a2cd0 0x7feaf811db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf6ffd640 1 --2- 172.21.15.204:0/3145854827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80a08e0 0x7feaf8130ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feaec000c00 tx=0x7feaec032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feadf7fe640 1 -- 172.21.15.204:0/3145854827 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feaec033cb0 con 0x7feaf80a08e0 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feadf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feadf7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feadf7fe640 10 monclient: dump: 2024-09-15T01:34:58.893 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:58.894 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:58.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feadf7fe640 10 monclient: _finish_auth 0 2024-09-15T01:34:58.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feadf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:58.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feadf7fe640 1 -- 172.21.15.204:0/3145854827 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec033e70 con 0x7feaf80a08e0 2024-09-15T01:34:58.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feafded4640 5 monclient: authenticate success, global_id 14529 2024-09-15T01:34:58.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feafded4640 10 monclient: _renew_subs 2024-09-15T01:34:58.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feafded4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:58.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feafded4640 1 -- 172.21.15.204:0/3145854827 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec009450 con 0x7feaf80a08e0 2024-09-15T01:34:58.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feadf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:58.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feadf7fe640 1 -- 172.21.15.204:0/3145854827 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feaec03a810 con 0x7feaf80a08e0 2024-09-15T01:34:58.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feadf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:58.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feadf7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:58.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feadf7fe640 10 monclient: dump: 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:58.896 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feafded4640 10 monclient: _renew_subs 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feaf5ffb640 4 set_mon_vals no callback set 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feafded4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.873+0000 7feafded4640 1 -- 172.21.15.204:0/3145854827 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf8121140 con 0x7feaf80a08e0 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.874+0000 7feadf7fe640 1 -- 172.21.15.204:0/3145854827 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7feaec039040 con 0x7feaf80a08e0 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.874+0000 7feafded4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.874+0000 7feafded4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.874+0000 7feafded4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.874+0000 7feafded4640 1 -- 172.21.15.204:0/3145854827 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf8070b80 con 0x7feaf80a08e0 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.874+0000 7feadf7fe640 1 --2- 172.21.15.204:0/3145854827 >> v2:172.21.15.204:6812/28210 conn(0x7feae40397b0 0x7feae403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:58.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.874+0000 7feadf7fe640 1 -- 172.21.15.204:0/3145854827 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5039+0+0 (secure 0 0 0) 0x7feaec06af40 con 0x7feaf80a08e0 2024-09-15T01:34:58.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.874+0000 7feaf77fe640 1 --2- 172.21.15.204:0/3145854827 >> v2:172.21.15.204:6812/28210 conn(0x7feae40397b0 0x7feae403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:58.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.874+0000 7feaf77fe640 10 monclient: get_auth_request con 0x7feae40397b0 auth_method 0 2024-09-15T01:34:58.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.875+0000 7feaf77fe640 1 --2- 172.21.15.204:0/3145854827 >> v2:172.21.15.204:6812/28210 conn(0x7feae40397b0 0x7feae403bc60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feae80095c0 tx=0x7feae80090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:58.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.879+0000 7feadf7fe640 1 -- 172.21.15.204:0/3145854827 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7feaec09ce40 con 0x7feaf80a08e0 2024-09-15T01:34:58.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.879+0000 7feadf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:58.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:58.879+0000 7feadf7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:34:59.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.037+0000 7feafded4640 10 start_mon_command cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-09-15T01:34:59.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.037+0000 7feafded4640 10 monclient: _send_command 2 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-09-15T01:34:59.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.037+0000 7feafded4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:59.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.037+0000 7feafded4640 1 -- 172.21.15.204:0/3145854827 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]} v 0) v1 -- 0x7feaf8067450 con 0x7feaf80a08e0 2024-09-15T01:34:59.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.680+0000 7feadf7fe640 1 -- 172.21.15.204:0/3145854827 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}]=0 create-or-move updating item name 'osd.7' weight 1 at location {root=default} to crush map v96) v1 ==== 212+0+0 (secure 0 0 0) 0x7feaec041020 con 0x7feaf80a08e0 2024-09-15T01:34:59.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.681+0000 7feadf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-09-15T01:34:59.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.681+0000 7feadf7fe640 10 monclient: _finish_command 2 = system:0 create-or-move updating item name 'osd.7' weight 1 at location {root=default} to crush map 2024-09-15T01:34:59.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.782+0000 7feafded4640 10 start_mon_command cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-09-15T01:34:59.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.782+0000 7feafded4640 10 monclient: _send_command 3 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-09-15T01:34:59.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.782+0000 7feafded4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:34:59.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.782+0000 7feafded4640 1 -- 172.21.15.204:0/3145854827 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]} v 0) v1 -- 0x7feaf8142830 con 0x7feaf80a08e0 2024-09-15T01:34:59.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.783+0000 7feadf7fe640 1 -- 172.21.15.204:0/3145854827 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}]=0 create-or-move updated item name 'osd.7' weight 1 at location {root=default} to crush map v96) v1 ==== 211+0+0 (secure 0 0 0) 0x7feaec041030 con 0x7feaf80a08e0 2024-09-15T01:34:59.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.783+0000 7feadf7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-09-15T01:34:59.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.783+0000 7feadf7fe640 10 monclient: _finish_command 3 = system:0 create-or-move updated item name 'osd.7' weight 1 at location {root=default} to crush map 2024-09-15T01:34:59.785 INFO:tasks.workunit.client.0.smithi204.stderr:create-or-move updated item name 'osd.7' weight 1 at location {root=default} to crush map 2024-09-15T01:34:59.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 1 -- 172.21.15.204:0/3145854827 >> v2:172.21.15.204:6812/28210 conn(0x7feae40397b0 msgr2=0x7feae403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:59.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 1 --2- 172.21.15.204:0/3145854827 >> v2:172.21.15.204:6812/28210 conn(0x7feae40397b0 0x7feae403bc60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feae80095c0 tx=0x7feae80090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 10 monclient: shutdown 2024-09-15T01:34:59.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:59.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 1 -- 172.21.15.204:0/3145854827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80a08e0 msgr2=0x7feaf8130ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:59.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 1 --2- 172.21.15.204:0/3145854827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80a08e0 0x7feaf8130ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feaec000c00 tx=0x7feaec032000 comp rx=0 tx=0).stop 2024-09-15T01:34:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 1 -- 172.21.15.204:0/3145854827 shutdown_connections 2024-09-15T01:34:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 1 --2- 172.21.15.204:0/3145854827 >> v2:172.21.15.204:6812/28210 conn(0x7feae40397b0 0x7feae403bc60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 1 --2- 172.21.15.204:0/3145854827 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feaf80a2cd0 0x7feaf811db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 1 --2- 172.21.15.204:0/3145854827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80a08e0 0x7feaf8130ff0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 1 --2- 172.21.15.204:0/3145854827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf809fa20 0x7feaf8141a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 1 -- 172.21.15.204:0/3145854827 >> 172.21.15.204:0/3145854827 conn(0x7feaf806e860 msgr2=0x7feaf80a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 1 -- 172.21.15.204:0/3145854827 shutdown_connections 2024-09-15T01:34:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.785+0000 7feafded4640 1 -- 172.21.15.204:0/3145854827 wait complete. 2024-09-15T01:34:59.796 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush move osd.7 host=foo 2024-09-15T01:34:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.869+0000 7f66a9a4f640 10 monclient: get_monmap_and_config 2024-09-15T01:34:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.869+0000 7f66a9a4f640 10 monclient: build_initial_monmap 2024-09-15T01:34:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.869+0000 7f66a9a4f640 10 monclient: monmap: 2024-09-15T01:34:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:34:59.870572+0000 2024-09-15T01:34:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:34:59.870572+0000 2024-09-15T01:34:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:34:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.869+0000 7f66a9a4f640 1 Processor -- start 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.869+0000 7f66a9a4f640 1 -- start start 2024-09-15T01:34:59.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.869+0000 7f66a9a4f640 10 monclient: init 2024-09-15T01:34:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 10 monclient: _add_conns ranks=[5,8,3,4,2,6,1,0,7,9,10] 2024-09-15T01:34:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a40a3560 0x7f66a409b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 10 monclient(hunting): picked mon.noname-h con 0x7f66a40a3560 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:34:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a409cca0 0x7f66a4067130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 10 monclient(hunting): picked mon.noname-e con 0x7f66a409cca0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:34:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a409d090 0x7f66a4067670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 10 monclient(hunting): picked mon.noname-g con 0x7f66a409d090 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:34:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:59.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a409e120 con 0x7f66a409d090 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66a4099f10 con 0x7f66a40a3560 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f66a409ab00 con 0x7f66a409cca0 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a9a4f640 10 monclient(hunting): authenticate will time out at 915.513265s 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a2ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a40a3560 0x7f66a409b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a409d090 0x7f66a4067670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a2ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a40a3560 0x7f66a409b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:41970/0 (socket says 172.21.15.204:41970) 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a409d090 0x7f66a4067670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58828/0 (socket says 172.21.15.204:58828) 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a2ffd640 1 -- 172.21.15.204:0/2319991454 learned_addr learned my addr 172.21.15.204:0/2319991454 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a2ffd640 10 monclient(hunting): get_auth_request con 0x7f66a40a3560 auth_method 0 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:59.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 10 monclient(hunting): get_auth_request con 0x7f66a409d090 auth_method 0 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a27fc640 1 --2- 172.21.15.204:0/2319991454 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a409cca0 0x7f66a4067130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a27fc640 10 monclient(hunting): get_auth_request con 0x7f66a409cca0 auth_method 0 2024-09-15T01:34:59.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 10 monclient(hunting): handle_auth_done global_id 14538 payload 274 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 1 -- 172.21.15.204:0/2319991454 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a409cca0 msgr2=0x7f66a4067130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 1 --2- 172.21.15.204:0/2319991454 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a409cca0 0x7f66a4067130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 1 -- 172.21.15.204:0/2319991454 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a40a3560 msgr2=0x7f66a409b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 1 --2- 172.21.15.204:0/2319991454 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a40a3560 0x7f66a409b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 1 monclient: found mon.noname-g 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a37fe640 1 -- 172.21.15.204:0/2319991454 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a404b720 con 0x7f66a409d090 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.870+0000 7f66a2ffd640 1 --2- 172.21.15.204:0/2319991454 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a40a3560 0x7f66a409b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:34:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a37fe640 1 --2- 172.21.15.204:0/2319991454 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a409d090 0x7f66a4067670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66940098e0 tx=0x7f66940090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=90e616712000afd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a1ffb640 1 -- 172.21.15.204:0/2319991454 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f669402abc0 con 0x7f66a409d090 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a1ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a1ffb640 10 monclient: dump: 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a1ffb640 1 -- 172.21.15.204:0/2319991454 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669402ad40 con 0x7f66a409d090 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a9a4f640 5 monclient: authenticate success, global_id 14538 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a9a4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a1ffb640 1 -- 172.21.15.204:0/2319991454 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6694031ca0 con 0x7f66a409d090 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a1ffb640 10 monclient: dump: 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:59.889 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a9a4f640 10 monclient: get_monmap_and_config success 2024-09-15T01:34:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.871+0000 7f66a9a4f640 4 set_mon_vals no callback set 2024-09-15T01:34:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 10 monclient: shutdown 2024-09-15T01:34:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:34:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2319991454 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a409d090 msgr2=0x7f66a4067670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 1 --2- 172.21.15.204:0/2319991454 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a409d090 0x7f66a4067670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66940098e0 tx=0x7f66940090f0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2319991454 shutdown_connections 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 1 --2- 172.21.15.204:0/2319991454 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a409d090 0x7f66a4067670 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 1 --2- 172.21.15.204:0/2319991454 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a409cca0 0x7f66a4067130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 1 --2- 172.21.15.204:0/2319991454 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a40a3560 0x7f66a409b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2319991454 >> 172.21.15.204:0/2319991454 conn(0x7f66a40687e0 msgr2=0x7f66a40a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2319991454 shutdown_connections 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2319991454 wait complete. 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 10 monclient: build_initial_monmap 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.872+0000 7f66a9a4f640 10 monclient: monmap: 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:34:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:34:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:34:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:34:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:34:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:34:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:34:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:34:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:34:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:34:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:34:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:34:59.891 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 1 Processor -- start 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 1 -- start start 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 10 monclient: init 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 10 monclient: _reopen_session rank -1 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 10 monclient: _add_conns ranks=[6,1,4,3,2,7,0,8,5] 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a409cca0 0x7f66a411f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 10 monclient(hunting): picked mon.noname-d con 0x7f66a409cca0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a409d090 0x7f66a411f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 10 monclient(hunting): picked mon.noname-f con 0x7f66a409d090 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a40a3560 0x7f66a411fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 10 monclient(hunting): picked mon.noname-c con 0x7f66a40a3560 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a4122df0 con 0x7f66a409d090 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66a406eaa0 con 0x7f66a40a3560 2024-09-15T01:34:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f66a4120d00 con 0x7f66a409cca0 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 10 monclient(hunting): _renew_subs 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a9a4f640 10 monclient(hunting): authenticate will time out at 915.516768s 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a2ffd640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a409cca0 msgr2=0x7f66a411f3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a2ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a409cca0 0x7f66a411f3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a27fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a409d090 0x7f66a411f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a27fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a409d090 0x7f66a411f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:57218/0 (socket says 172.21.15.204:57218) 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a27fc640 1 -- 172.21.15.204:0/2291305614 learned_addr learned my addr 172.21.15.204:0/2291305614 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a27fc640 10 monclient(hunting): get_auth_request con 0x7f66a409d090 auth_method 0 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:34:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:34:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.873+0000 7f66a27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:34:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a37fe640 1 --2- 172.21.15.204:0/2291305614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a40a3560 0x7f66a411fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a37fe640 10 monclient(hunting): get_auth_request con 0x7f66a40a3560 auth_method 0 2024-09-15T01:34:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:34:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:34:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:34:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a27fc640 10 monclient(hunting): handle_auth_done global_id 5914 payload 824 2024-09-15T01:34:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a27fc640 1 -- 172.21.15.204:0/2291305614 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a409cca0 msgr2=0x7f66a411f3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:34:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a27fc640 1 --2- 172.21.15.204:0/2291305614 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a409cca0 0x7f66a411f3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a27fc640 1 -- 172.21.15.204:0/2291305614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a40a3560 msgr2=0x7f66a411fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:34:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a27fc640 1 --2- 172.21.15.204:0/2291305614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a40a3560 0x7f66a411fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:34:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a27fc640 10 monclient: _finish_hunting 0 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a27fc640 1 monclient: found mon.noname-f 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a27fc640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a27fc640 1 -- 172.21.15.204:0/2291305614 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c009ae0 con 0x7f66a409d090 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a37fe640 1 --2- 172.21.15.204:0/2291305614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a40a3560 0x7f66a411fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a27fc640 1 --2- 172.21.15.204:0/2291305614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a409d090 0x7f66a411f910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f669800d8c0 tx=0x7f6698008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f668affd640 1 -- 172.21.15.204:0/2291305614 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f669800de50 con 0x7f66a409d090 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f668affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f668affd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f668affd640 10 monclient: dump: 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f668affd640 10 monclient: _finish_auth 0 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f668affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f668affd640 1 -- 172.21.15.204:0/2291305614 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669800bd50 con 0x7f66a409d090 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f668affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f668affd640 1 -- 172.21.15.204:0/2291305614 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f66980128a0 con 0x7f66a409d090 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a9a4f640 5 monclient: authenticate success, global_id 5914 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f668affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f668affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f668affd640 10 monclient: dump: 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:34:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a17fa640 4 set_mon_vals no callback set 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a9a4f640 10 monclient: _renew_subs 2024-09-15T01:34:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a9a4f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2291305614 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6694009390 con 0x7f66a409d090 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a9a4f640 10 monclient: _renew_subs 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a9a4f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.874+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2291305614 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a4122b10 con 0x7f66a409d090 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.875+0000 7f668affd640 1 -- 172.21.15.204:0/2291305614 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6698012420 con 0x7f66a409d090 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.875+0000 7f66a9a4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.875+0000 7f66a9a4f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.875+0000 7f66a9a4f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.875+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2291305614 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a4070cc0 con 0x7f66a409d090 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.875+0000 7f668affd640 1 --2- 172.21.15.204:0/2291305614 >> v2:172.21.15.204:6812/28210 conn(0x7f66900397b0 0x7f669003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.875+0000 7f668affd640 1 -- 172.21.15.204:0/2291305614 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5060+0+0 (secure 0 0 0) 0x7f669804b940 con 0x7f66a409d090 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.875+0000 7f66a2ffd640 1 --2- 172.21.15.204:0/2291305614 >> v2:172.21.15.204:6812/28210 conn(0x7f66900397b0 0x7f669003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:34:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.875+0000 7f66a2ffd640 10 monclient: get_auth_request con 0x7f66900397b0 auth_method 0 2024-09-15T01:34:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.876+0000 7f66a2ffd640 1 --2- 172.21.15.204:0/2291305614 >> v2:172.21.15.204:6812/28210 conn(0x7f66900397b0 0x7f669003bc60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f669c007250 tx=0x7f669c00b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:34:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.880+0000 7f668affd640 1 -- 172.21.15.204:0/2291305614 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f669800bb60 con 0x7f66a409d090 2024-09-15T01:34:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.880+0000 7f668affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:34:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:34:59.880+0000 7f668affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:00.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.040+0000 7f66a9a4f640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-09-15T01:35:00.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.040+0000 7f66a9a4f640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-09-15T01:35:00.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.040+0000 7f66a9a4f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:00.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.040+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2291305614 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]} v 0) v1 -- 0x7f66a4122520 con 0x7f66a409d090 2024-09-15T01:35:00.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.682+0000 7f668affd640 1 -- 172.21.15.204:0/2291305614 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}]=0 moved item id 7 name 'osd.7' to location {host=foo} in crush map v97) v1 ==== 165+0+0 (secure 0 0 0) 0x7f669801c360 con 0x7f66a409d090 2024-09-15T01:35:00.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.682+0000 7f668affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-09-15T01:35:00.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.682+0000 7f668affd640 10 monclient: _finish_command 2 = system:0 moved item id 7 name 'osd.7' to location {host=foo} in crush map 2024-09-15T01:35:00.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.787+0000 7f66a9a4f640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-09-15T01:35:00.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.787+0000 7f66a9a4f640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-09-15T01:35:00.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.787+0000 7f66a9a4f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:00.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.787+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2291305614 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]} v 0) v1 -- 0x7f66a40008d0 con 0x7f66a409d090 2024-09-15T01:35:00.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.787+0000 7f668affd640 1 -- 172.21.15.204:0/2291305614 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}]=0 no need to move item id 7 name 'osd.7' to location {host=foo} in crush map v97) v1 ==== 175+0+0 (secure 0 0 0) 0x7f669801c540 con 0x7f66a409d090 2024-09-15T01:35:00.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.787+0000 7f668affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-09-15T01:35:00.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.787+0000 7f668affd640 10 monclient: _finish_command 3 = system:0 no need to move item id 7 name 'osd.7' to location {host=foo} in crush map 2024-09-15T01:35:00.790 INFO:tasks.workunit.client.0.smithi204.stderr:no need to move item id 7 name 'osd.7' to location {host=foo} in crush map 2024-09-15T01:35:00.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2291305614 >> v2:172.21.15.204:6812/28210 conn(0x7f66900397b0 msgr2=0x7f669003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:00.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 1 --2- 172.21.15.204:0/2291305614 >> v2:172.21.15.204:6812/28210 conn(0x7f66900397b0 0x7f669003bc60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f669c007250 tx=0x7f669c00b040 comp rx=0 tx=0).stop 2024-09-15T01:35:00.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 10 monclient: shutdown 2024-09-15T01:35:00.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:00.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2291305614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a409d090 msgr2=0x7f66a411f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:00.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 1 --2- 172.21.15.204:0/2291305614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a409d090 0x7f66a411f910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f669800d8c0 tx=0x7f6698008040 comp rx=0 tx=0).stop 2024-09-15T01:35:00.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2291305614 shutdown_connections 2024-09-15T01:35:00.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 1 --2- 172.21.15.204:0/2291305614 >> v2:172.21.15.204:6812/28210 conn(0x7f66900397b0 0x7f669003bc60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:00.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 1 --2- 172.21.15.204:0/2291305614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a40a3560 0x7f66a411fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:00.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 1 --2- 172.21.15.204:0/2291305614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a409d090 0x7f66a411f910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:00.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 1 --2- 172.21.15.204:0/2291305614 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66a409cca0 0x7f66a411f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:00.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2291305614 >> 172.21.15.204:0/2291305614 conn(0x7f66a40687e0 msgr2=0x7f66a40a13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:00.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2291305614 shutdown_connections 2024-09-15T01:35:00.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.789+0000 7f66a9a4f640 1 -- 172.21.15.204:0/2291305614 wait complete. 2024-09-15T01:35:00.801 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd find osd.7 2024-09-15T01:35:00.801 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep host 2024-09-15T01:35:00.801 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep foo 2024-09-15T01:35:00.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.870+0000 7fc3b024f640 10 monclient: get_monmap_and_config 2024-09-15T01:35:00.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.870+0000 7fc3b024f640 10 monclient: build_initial_monmap 2024-09-15T01:35:00.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.871+0000 7fc3b024f640 10 monclient: monmap: 2024-09-15T01:35:00.873 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:00.873 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:00.873 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:00.872483+0000 2024-09-15T01:35:00.873 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:00.872483+0000 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:00.874 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:00.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.871+0000 7fc3b024f640 1 Processor -- start 2024-09-15T01:35:00.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.871+0000 7fc3b024f640 1 -- start start 2024-09-15T01:35:00.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.871+0000 7fc3b024f640 10 monclient: init 2024-09-15T01:35:00.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:00.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 10 monclient: _add_conns ranks=[4,1,3,7,9,10,5,0,2,8,6] 2024-09-15T01:35:00.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc3a80a2400 0x7fc3a80a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:00.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 10 monclient(hunting): picked mon.noname-c con 0x7fc3a80a2400 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:00.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc3a809f990 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:00.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3a8137c60 0x7fc3a813a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:00.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 10 monclient(hunting): picked mon.noname-g con 0x7fc3a8137c60 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:00.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a80648a0 con 0x7fc3a809f990 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a809dc50 con 0x7fc3a8137c60 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a809e1a0 con 0x7fc3a80a2400 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3b024f640 10 monclient(hunting): authenticate will time out at 916.515465s 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ad7ea640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc3a809f990 0x7fc3a809fd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:50422/0 (socket says 172.21.15.204:50422) 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ad7ea640 1 -- 172.21.15.204:0/3810994264 learned_addr learned my addr 172.21.15.204:0/3810994264 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 1 --2- 172.21.15.204:0/3810994264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3a8137c60 0x7fc3a813a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 10 monclient(hunting): get_auth_request con 0x7fc3a8137c60 auth_method 0 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:00.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3adfeb640 1 --2- 172.21.15.204:0/3810994264 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc3a80a2400 0x7fc3a80a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3adfeb640 10 monclient(hunting): get_auth_request con 0x7fc3a80a2400 auth_method 0 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3adfeb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3adfeb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3adfeb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 10 monclient(hunting): handle_auth_done global_id 14547 payload 274 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3acfe9640 1 -- 172.21.15.204:0/3810994264 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091167962 0 0) 0x7fc3a80648a0 con 0x7fc3a809f990 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 1 -- 172.21.15.204:0/3810994264 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc3a80a2400 msgr2=0x7fc3a80a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 1 --2- 172.21.15.204:0/3810994264 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc3a80a2400 0x7fc3a80a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 1 -- 172.21.15.204:0/3810994264 >> v1:172.21.15.204:6789/0 conn(0x7fc3a809f990 legacy=0x7fc3a809fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 10 monclient: _finish_hunting 0 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 1 monclient: found mon.noname-g 2024-09-15T01:35:00.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3ae7ec640 1 -- 172.21.15.204:0/3810994264 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a80a27d0 con 0x7fc3a8137c60 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.872+0000 7fc3acfe9640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3ae7ec640 1 --2- 172.21.15.204:0/3810994264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3a8137c60 0x7fc3a813a040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc398009e90 tx=0x7fc398009780 comp rx=0 tx=0).ready entity=mon.3 client_cookie=8f43e90429e0c5a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3acfe9640 1 -- 172.21.15.204:0/3810994264 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc398031880 con 0x7fc3a8137c60 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3acfe9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3acfe9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3acfe9640 10 monclient: dump: 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:00.885 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3acfe9640 10 monclient: _finish_auth 0 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3acfe9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3b024f640 5 monclient: authenticate success, global_id 14547 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3acfe9640 1 -- 172.21.15.204:0/3810994264 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39802d050 con 0x7fc3a8137c60 2024-09-15T01:35:00.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3b024f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3acfe9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3b024f640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3b024f640 4 set_mon_vals no callback set 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3acfe9640 1 -- 172.21.15.204:0/3810994264 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc3980303a0 con 0x7fc3a8137c60 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3acfe9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3acfe9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.873+0000 7fc3acfe9640 10 monclient: dump: 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:00.887 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.874+0000 7fc3b024f640 10 monclient: shutdown 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.874+0000 7fc3b024f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.874+0000 7fc3b024f640 1 -- 172.21.15.204:0/3810994264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3a8137c60 msgr2=0x7fc3a813a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:00.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.874+0000 7fc3b024f640 1 --2- 172.21.15.204:0/3810994264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3a8137c60 0x7fc3a813a040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc398009e90 tx=0x7fc398009780 comp rx=0 tx=0).stop 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.874+0000 7fc3b024f640 1 -- 172.21.15.204:0/3810994264 shutdown_connections 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.874+0000 7fc3b024f640 1 --2- 172.21.15.204:0/3810994264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3a8137c60 0x7fc3a813a040 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.874+0000 7fc3b024f640 1 --2- 172.21.15.204:0/3810994264 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc3a80a2400 0x7fc3a80a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.874+0000 7fc3b024f640 1 -- 172.21.15.204:0/3810994264 >> 172.21.15.204:0/3810994264 conn(0x7fc3a806e860 msgr2=0x7fc3a806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.874+0000 7fc3b024f640 1 -- 172.21.15.204:0/3810994264 shutdown_connections 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.874+0000 7fc3b024f640 1 -- 172.21.15.204:0/3810994264 wait complete. 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.874+0000 7fc3b024f640 10 monclient: build_initial_monmap 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.874+0000 7fc3b024f640 10 monclient: monmap: 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 1 Processor -- start 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 1 -- start start 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 10 monclient: init 2024-09-15T01:35:00.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 10 monclient: _add_conns ranks=[8,1,2,4,7,5,0,3,6] 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc3a809f990 0x7fc3a809bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 10 monclient(hunting): picked mon.noname-e con 0x7fc3a809f990 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3a80a2400 0x7fc3a812ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 10 monclient(hunting): picked mon.noname-f con 0x7fc3a80a2400 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3a8137c60 0x7fc3a812f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 10 monclient(hunting): picked mon.noname-b con 0x7fc3a8137c60 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a81320e0 con 0x7fc3a80a2400 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a8131420 con 0x7fc3a8137c60 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a809c6f0 con 0x7fc3a809f990 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:00.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3b024f640 10 monclient(hunting): authenticate will time out at 916.518792s 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3ad7ea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3a80a2400 0x7fc3a812ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3ad7ea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3a80a2400 0x7fc3a812ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:57226/0 (socket says 172.21.15.204:57226) 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3ad7ea640 1 -- 172.21.15.204:0/3004466392 learned_addr learned my addr 172.21.15.204:0/3004466392 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3ad7ea640 10 monclient(hunting): get_auth_request con 0x7fc3a80a2400 auth_method 0 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3ad7ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3ad7ea640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3ae7ec640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3a8137c60 msgr2=0x7fc3a812f290 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3ad7ea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3ae7ec640 1 --2- 172.21.15.204:0/3004466392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3a8137c60 0x7fc3a812f290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3ad7ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3ad7ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.875+0000 7fc3ad7ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3adfeb640 1 --2- 172.21.15.204:0/3004466392 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc3a809f990 0x7fc3a809bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:00.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3ad7ea640 10 monclient(hunting): handle_auth_done global_id 5932 payload 824 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3ad7ea640 1 -- 172.21.15.204:0/3004466392 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc3a809f990 msgr2=0x7fc3a809bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3ad7ea640 1 --2- 172.21.15.204:0/3004466392 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc3a809f990 0x7fc3a809bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3ad7ea640 1 -- 172.21.15.204:0/3004466392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3a8137c60 msgr2=0x7fc3a812f290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3ad7ea640 1 --2- 172.21.15.204:0/3004466392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3a8137c60 0x7fc3a812f290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3ad7ea640 10 monclient: _finish_hunting 0 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3ad7ea640 1 monclient: found mon.noname-f 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3ad7ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3ad7ea640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3ad7ea640 1 -- 172.21.15.204:0/3004466392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc398009a20 con 0x7fc3a80a2400 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3adfeb640 10 monclient: get_auth_request con 0x7fc3a809f990 auth_method 0 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3adfeb640 1 --2- 172.21.15.204:0/3004466392 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc3a809f990 0x7fc3a809bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3ad7ea640 1 --2- 172.21.15.204:0/3004466392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3a80a2400 0x7fc3a812ed50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c5e0 tx=0x7fc39c007d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a5ffb640 1 -- 172.21.15.204:0/3004466392 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc39c0070f0 con 0x7fc3a80a2400 2024-09-15T01:35:00.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a5ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a5ffb640 10 monclient: dump: 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:00.894 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3b024f640 5 monclient: authenticate success, global_id 5932 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a5ffb640 1 -- 172.21.15.204:0/3004466392 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c007290 con 0x7fc3a80a2400 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3b024f640 10 monclient: _renew_subs 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a5ffb640 1 -- 172.21.15.204:0/3004466392 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc39c007480 con 0x7fc3a80a2400 2024-09-15T01:35:00.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3b024f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3b024f640 1 -- 172.21.15.204:0/3004466392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a809cad0 con 0x7fc3a80a2400 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a7fff640 4 set_mon_vals no callback set 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3a5ffb640 10 monclient: dump: 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:00.896 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3b024f640 10 monclient: _renew_subs 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3b024f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.876+0000 7fc3b024f640 1 -- 172.21.15.204:0/3004466392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a8136860 con 0x7fc3a80a2400 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.877+0000 7fc3a5ffb640 1 -- 172.21.15.204:0/3004466392 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc39c0075e0 con 0x7fc3a80a2400 2024-09-15T01:35:00.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.877+0000 7fc3b024f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:00.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.877+0000 7fc3b024f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:00.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.877+0000 7fc3b024f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:00.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.877+0000 7fc3a5ffb640 1 --2- 172.21.15.204:0/3004466392 >> v2:172.21.15.204:6812/28210 conn(0x7fc3940397b0 0x7fc39403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:00.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.877+0000 7fc3b024f640 1 -- 172.21.15.204:0/3004466392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a8070b80 con 0x7fc3a80a2400 2024-09-15T01:35:00.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.877+0000 7fc3a5ffb640 1 -- 172.21.15.204:0/3004466392 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5234+0+0 (secure 0 0 0) 0x7fc39c04b070 con 0x7fc3a80a2400 2024-09-15T01:35:00.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.877+0000 7fc3adfeb640 1 --2- 172.21.15.204:0/3004466392 >> v2:172.21.15.204:6812/28210 conn(0x7fc3940397b0 0x7fc39403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:00.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.877+0000 7fc3adfeb640 10 monclient: get_auth_request con 0x7fc3940397b0 auth_method 0 2024-09-15T01:35:00.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.878+0000 7fc3adfeb640 1 --2- 172.21.15.204:0/3004466392 >> v2:172.21.15.204:6812/28210 conn(0x7fc3940397b0 0x7fc39403bc60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00099b0 tx=0x7fc3a0008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:00.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.882+0000 7fc3a5ffb640 1 -- 172.21.15.204:0/3004466392 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc39c005aa0 con 0x7fc3a80a2400 2024-09-15T01:35:00.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.882+0000 7fc3a5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:00.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:00.882+0000 7fc3a5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:01.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.040+0000 7fc3b024f640 10 start_mon_command cmd=[{"prefix": "osd find", "id": 7}] 2024-09-15T01:35:01.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.040+0000 7fc3b024f640 10 monclient: _send_command 2 [{"prefix": "osd find", "id": 7}] 2024-09-15T01:35:01.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.040+0000 7fc3b024f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:01.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.040+0000 7fc3b024f640 1 -- 172.21.15.204:0/3004466392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd find", "id": 7} v 0) v1 -- 0x7fc3a809dec0 con 0x7fc3a80a2400 2024-09-15T01:35:01.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.041+0000 7fc3a5ffb640 1 -- 172.21.15.204:0/3004466392 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd find", "id": 7}]=0 v97) v1 ==== 65+0+226 (secure 0 0 0) 0x7fc39c010af0 con 0x7fc3a80a2400 2024-09-15T01:35:01.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.041+0000 7fc3a5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd find", "id": 7}] 2024-09-15T01:35:01.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.041+0000 7fc3a5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:01.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.136+0000 7fc3b024f640 10 start_mon_command cmd=[{"prefix": "osd find", "id": 7}] 2024-09-15T01:35:01.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.136+0000 7fc3b024f640 10 monclient: _send_command 3 [{"prefix": "osd find", "id": 7}] 2024-09-15T01:35:01.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.136+0000 7fc3b024f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:01.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.136+0000 7fc3b024f640 1 -- 172.21.15.204:0/3004466392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd find", "id": 7} v 0) v1 -- 0x7fc3a80008d0 con 0x7fc3a80a2400 2024-09-15T01:35:01.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.136+0000 7fc3a5ffb640 1 -- 172.21.15.204:0/3004466392 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd find", "id": 7}]=0 v97) v1 ==== 65+0+226 (secure 0 0 0) 0x7fc39c01a360 con 0x7fc3a80a2400 2024-09-15T01:35:01.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.136+0000 7fc3a5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd find", "id": 7}] 2024-09-15T01:35:01.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.136+0000 7fc3a5ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:01.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 1 -- 172.21.15.204:0/3004466392 >> v2:172.21.15.204:6812/28210 conn(0x7fc3940397b0 msgr2=0x7fc39403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:01.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 1 --2- 172.21.15.204:0/3004466392 >> v2:172.21.15.204:6812/28210 conn(0x7fc3940397b0 0x7fc39403bc60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00099b0 tx=0x7fc3a0008040 comp rx=0 tx=0).stop 2024-09-15T01:35:01.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 10 monclient: shutdown 2024-09-15T01:35:01.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:01.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 1 -- 172.21.15.204:0/3004466392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3a80a2400 msgr2=0x7fc3a812ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:01.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 1 --2- 172.21.15.204:0/3004466392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3a80a2400 0x7fc3a812ed50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c5e0 tx=0x7fc39c007d00 comp rx=0 tx=0).stop 2024-09-15T01:35:01.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 1 -- 172.21.15.204:0/3004466392 shutdown_connections 2024-09-15T01:35:01.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 1 --2- 172.21.15.204:0/3004466392 >> v2:172.21.15.204:6812/28210 conn(0x7fc3940397b0 0x7fc39403bc60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 1 --2- 172.21.15.204:0/3004466392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3a80a2400 0x7fc3a812ed50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 1 --2- 172.21.15.204:0/3004466392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3a8137c60 0x7fc3a812f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 1 --2- 172.21.15.204:0/3004466392 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc3a809f990 0x7fc3a809bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.142 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 1 -- 172.21.15.204:0/3004466392 >> 172.21.15.204:0/3004466392 conn(0x7fc3a806e860 msgr2=0x7fc3a80a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:01.142 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 1 -- 172.21.15.204:0/3004466392 shutdown_connections 2024-09-15T01:35:01.142 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.138+0000 7fc3b024f640 1 -- 172.21.15.204:0/3004466392 wait complete. 2024-09-15T01:35:01.149 INFO:tasks.workunit.client.0.smithi204.stdout: "host": "foo", 2024-09-15T01:35:01.149 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm osd.7 2024-09-15T01:35:01.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.221+0000 7f5b28c6f640 10 monclient: get_monmap_and_config 2024-09-15T01:35:01.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.221+0000 7f5b28c6f640 10 monclient: build_initial_monmap 2024-09-15T01:35:01.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.221+0000 7f5b28c6f640 10 monclient: monmap: 2024-09-15T01:35:01.223 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:01.223 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:01.222681+0000 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:01.222681+0000 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:01.224 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:01.225 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:01.225 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:01.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.221+0000 7f5b28c6f640 1 Processor -- start 2024-09-15T01:35:01.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.221+0000 7f5b28c6f640 1 -- start start 2024-09-15T01:35:01.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.221+0000 7f5b28c6f640 10 monclient: init 2024-09-15T01:35:01.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:01.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 10 monclient: _add_conns ranks=[0,5,3,7,8,6,1,4,9,10,2] 2024-09-15T01:35:01.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5b240a2e30 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:01.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 0x7f5b2409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 10 monclient(hunting): picked mon.noname-h con 0x7f5b2409fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:01.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b24137d20 0x7f5b2413a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 10 monclient(hunting): picked mon.noname-g con 0x7f5b24137d20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:01.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2404b720 con 0x7f5b240a2e30 2024-09-15T01:35:01.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:01.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2409e6c0 con 0x7f5b24137d20 2024-09-15T01:35:01.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:01.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240a04b0 con 0x7f5b2409fb80 2024-09-15T01:35:01.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:01.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b28c6f640 10 monclient(hunting): authenticate will time out at 916.865498s 2024-09-15T01:35:01.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 0x7f5b2409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:01.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b22d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b24137d20 0x7f5b2413a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:01.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b22d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b24137d20 0x7f5b2413a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58838/0 (socket says 172.21.15.204:58838) 2024-09-15T01:35:01.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 0x7f5b2409ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:41978/0 (socket says 172.21.15.204:41978) 2024-09-15T01:35:01.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b22d9d640 1 -- 172.21.15.204:0/1122902663 learned_addr learned my addr 172.21.15.204:0/1122902663 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b22d9d640 10 monclient(hunting): get_auth_request con 0x7f5b24137d20 auth_method 0 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b22d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b22d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b22d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 10 monclient(hunting): get_auth_request con 0x7f5b2409fb80 auth_method 0 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b22d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b22d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:01.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b22d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 10 monclient(hunting): handle_auth_done global_id 5450 payload 274 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 1 -- 172.21.15.204:0/1122902663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b24137d20 msgr2=0x7f5b2413a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 1 --2- 172.21.15.204:0/1122902663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b24137d20 0x7f5b2413a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 1 -- 172.21.15.204:0/1122902663 >> v1:172.21.15.178:6789/0 conn(0x7f5b240a2e30 legacy=0x7f5b24132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 1 monclient: found mon.noname-h 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b21d9b640 1 -- 172.21.15.204:0/1122902663 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b24133170 con 0x7f5b2409fb80 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.222+0000 7f5b22d9d640 1 --2- 172.21.15.204:0/1122902663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b24137d20 0x7f5b2413a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b21d9b640 1 --2- 172.21.15.204:0/1122902663 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 0x7f5b2409ff70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009880 tx=0x7f5b180090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=56ff447cc5d722f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b2159a640 1 -- 172.21.15.204:0/1122902663 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b1802ac50 con 0x7f5b2409fb80 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b2159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:01.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b2159a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b2159a640 10 monclient: dump: 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:01.237 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:01.238 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:01.238 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:01.238 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:01.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b2159a640 10 monclient: _finish_auth 0 2024-09-15T01:35:01.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b2159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:01.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b28c6f640 5 monclient: authenticate success, global_id 5450 2024-09-15T01:35:01.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b28c6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:01.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b2159a640 1 -- 172.21.15.204:0/1122902663 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1802add0 con 0x7f5b2409fb80 2024-09-15T01:35:01.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b2159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:01.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b28c6f640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:01.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b28c6f640 4 set_mon_vals no callback set 2024-09-15T01:35:01.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b2159a640 1 -- 172.21.15.204:0/1122902663 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b18031ca0 con 0x7f5b2409fb80 2024-09-15T01:35:01.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b2159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:01.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b2159a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:01.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.223+0000 7f5b2159a640 10 monclient: dump: 2024-09-15T01:35:01.239 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:01.239 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:01.239 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.239 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.239 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:01.239 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:01.239 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:01.239 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:01.239 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:01.240 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:01.240 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:01.240 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:01.240 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:01.240 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:01.240 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:01.240 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:01.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.224+0000 7f5b28c6f640 10 monclient: shutdown 2024-09-15T01:35:01.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.224+0000 7f5b28c6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:01.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.224+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1122902663 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 msgr2=0x7f5b2409ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:01.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.224+0000 7f5b28c6f640 1 --2- 172.21.15.204:0/1122902663 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 0x7f5b2409ff70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009880 tx=0x7f5b180090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.224+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1122902663 shutdown_connections 2024-09-15T01:35:01.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.224+0000 7f5b28c6f640 1 --2- 172.21.15.204:0/1122902663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b24137d20 0x7f5b2413a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.224+0000 7f5b28c6f640 1 --2- 172.21.15.204:0/1122902663 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 0x7f5b2409ff70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.224+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1122902663 >> 172.21.15.204:0/1122902663 conn(0x7f5b2406e860 msgr2=0x7f5b2406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:01.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.224+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1122902663 shutdown_connections 2024-09-15T01:35:01.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.224+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1122902663 wait complete. 2024-09-15T01:35:01.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.224+0000 7f5b28c6f640 10 monclient: build_initial_monmap 2024-09-15T01:35:01.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.224+0000 7f5b28c6f640 10 monclient: monmap: 2024-09-15T01:35:01.241 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:01.241 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:01.241 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:01.241 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:01.241 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 1 Processor -- start 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 1 -- start start 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 10 monclient: init 2024-09-15T01:35:01.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 10 monclient: _add_conns ranks=[5,4,3,6,1,0,8,2,7] 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 0x7f5b24099170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 10 monclient(hunting): picked mon.noname-h con 0x7f5b2409fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5b240a2e30 0x7f5b240996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 10 monclient(hunting): picked mon.noname-c con 0x7f5b240a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b24137d20 0x7f5b24099bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 10 monclient(hunting): picked mon.noname-g con 0x7f5b24137d20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2409d650 con 0x7f5b24137d20 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2409ac60 con 0x7f5b240a2e30 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:01.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2409b170 con 0x7f5b2409fb80 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b28c6f640 10 monclient(hunting): authenticate will time out at 916.868770s 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b2259c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 0x7f5b24099170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b22d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b24137d20 0x7f5b24099bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b2259c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 0x7f5b24099170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:41980/0 (socket says 172.21.15.204:41980) 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b2259c640 1 -- 172.21.15.204:0/1914054984 learned_addr learned my addr 172.21.15.204:0/1914054984 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b2259c640 10 monclient(hunting): get_auth_request con 0x7f5b2409fb80 auth_method 0 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b2259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b2259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b2259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b22d9d640 10 monclient(hunting): get_auth_request con 0x7f5b24137d20 auth_method 0 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b22d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:01.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b22d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b22d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b2259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b2259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b2259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b22d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b22d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b22d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.225+0000 7f5b21d9b640 1 --2- 172.21.15.204:0/1914054984 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5b240a2e30 0x7f5b240996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b21d9b640 10 monclient(hunting): get_auth_request con 0x7f5b240a2e30 auth_method 0 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b21d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b21d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b21d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:01.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b2259c640 10 monclient(hunting): handle_auth_done global_id 5459 payload 824 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b2259c640 1 -- 172.21.15.204:0/1914054984 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5b240a2e30 msgr2=0x7f5b240996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b2259c640 1 --2- 172.21.15.204:0/1914054984 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5b240a2e30 0x7f5b240996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b2259c640 1 -- 172.21.15.204:0/1914054984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b24137d20 msgr2=0x7f5b24099bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b2259c640 1 --2- 172.21.15.204:0/1914054984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b24137d20 0x7f5b24099bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b2259c640 10 monclient: _finish_hunting 0 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b2259c640 1 monclient: found mon.noname-h 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b2259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b2259c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b2259c640 1 -- 172.21.15.204:0/1914054984 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c009ae0 con 0x7f5b2409fb80 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b2259c640 1 --2- 172.21.15.204:0/1914054984 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 0x7f5b24099170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5b2409b340 tx=0x7f5b14007eb0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b127fc640 1 -- 172.21.15.204:0/1914054984 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b14001ea0 con 0x7f5b2409fb80 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:01.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b127fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b127fc640 10 monclient: dump: 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:01.247 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b127fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b127fc640 1 -- 172.21.15.204:0/1914054984 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b14011a10 con 0x7f5b2409fb80 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b28c6f640 5 monclient: authenticate success, global_id 5459 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b28c6f640 10 monclient: _renew_subs 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b28c6f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1914054984 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b18009390 con 0x7f5b2409fb80 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b127fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b127fc640 1 -- 172.21.15.204:0/1914054984 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b140102e0 con 0x7f5b2409fb80 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:01.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b127fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b127fc640 10 monclient: dump: 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:01.249 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b20d99640 4 set_mon_vals no callback set 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b28c6f640 10 monclient: _renew_subs 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b28c6f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.226+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1914054984 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2409c790 con 0x7f5b2409fb80 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.227+0000 7f5b127fc640 1 -- 172.21.15.204:0/1914054984 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5b14011b80 con 0x7f5b2409fb80 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.227+0000 7f5b28c6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.227+0000 7f5b28c6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.227+0000 7f5b28c6f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.227+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1914054984 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b24070cc0 con 0x7f5b2409fb80 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.227+0000 7f5b127fc640 1 --2- 172.21.15.204:0/1914054984 >> v2:172.21.15.204:6812/28210 conn(0x7f5b040396e0 0x7f5b0403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.227+0000 7f5b127fc640 1 -- 172.21.15.204:0/1914054984 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5234+0+0 (secure 0 0 0) 0x7f5b1404a4f0 con 0x7f5b2409fb80 2024-09-15T01:35:01.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.227+0000 7f5b21d9b640 1 --2- 172.21.15.204:0/1914054984 >> v2:172.21.15.204:6812/28210 conn(0x7f5b040396e0 0x7f5b0403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:01.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.227+0000 7f5b21d9b640 10 monclient: get_auth_request con 0x7f5b040396e0 auth_method 0 2024-09-15T01:35:01.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.227+0000 7f5b21d9b640 1 --2- 172.21.15.204:0/1914054984 >> v2:172.21.15.204:6812/28210 conn(0x7f5b040396e0 0x7f5b0403bb90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b18034000 tx=0x7f5b18032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:01.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.231+0000 7f5b127fc640 1 -- 172.21.15.204:0/1914054984 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5b1400f860 con 0x7f5b2409fb80 2024-09-15T01:35:01.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.231+0000 7f5b127fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:01.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.231+0000 7f5b127fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:01.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.389+0000 7f5b28c6f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7"}] 2024-09-15T01:35:01.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.389+0000 7f5b28c6f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-09-15T01:35:01.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.389+0000 7f5b28c6f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.389+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1914054984 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.7"} v 0) v1 -- 0x7f5b240667e0 con 0x7f5b2409fb80 2024-09-15T01:35:01.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.448+0000 7f5b127fc640 1 -- 172.21.15.204:0/1914054984 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7"}]=0 removed item id 7 name 'osd.7' from crush map v98) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5b14017070 con 0x7f5b2409fb80 2024-09-15T01:35:01.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.448+0000 7f5b127fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-09-15T01:35:01.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.448+0000 7f5b127fc640 10 monclient: _finish_command 2 = system:0 removed item id 7 name 'osd.7' from crush map 2024-09-15T01:35:01.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.550+0000 7f5b28c6f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7"}] 2024-09-15T01:35:01.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.550+0000 7f5b28c6f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-09-15T01:35:01.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.550+0000 7f5b28c6f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.550+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1914054984 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.7"} v 0) v1 -- 0x7f5b240008d0 con 0x7f5b2409fb80 2024-09-15T01:35:01.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.551+0000 7f5b127fc640 1 -- 172.21.15.204:0/1914054984 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7"}]=0 device 'osd.7' does not appear in the crush map v98) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5b14017070 con 0x7f5b2409fb80 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.551+0000 7f5b127fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.551+0000 7f5b127fc640 10 monclient: _finish_command 3 = system:0 device 'osd.7' does not appear in the crush map 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:device 'osd.7' does not appear in the crush map 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1914054984 >> v2:172.21.15.204:6812/28210 conn(0x7f5b040396e0 msgr2=0x7f5b0403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 1 --2- 172.21.15.204:0/1914054984 >> v2:172.21.15.204:6812/28210 conn(0x7f5b040396e0 0x7f5b0403bb90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b18034000 tx=0x7f5b18032000 comp rx=0 tx=0).stop 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 10 monclient: shutdown 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1914054984 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 msgr2=0x7f5b24099170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 1 --2- 172.21.15.204:0/1914054984 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 0x7f5b24099170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5b2409b340 tx=0x7f5b14007eb0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1914054984 shutdown_connections 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 1 --2- 172.21.15.204:0/1914054984 >> v2:172.21.15.204:6812/28210 conn(0x7f5b040396e0 0x7f5b0403bb90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 1 --2- 172.21.15.204:0/1914054984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b24137d20 0x7f5b24099bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 1 --2- 172.21.15.204:0/1914054984 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5b240a2e30 0x7f5b240996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 1 --2- 172.21.15.204:0/1914054984 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2409fb80 0x7f5b24099170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1914054984 >> 172.21.15.204:0/1914054984 conn(0x7f5b2406e860 msgr2=0x7f5b240a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:01.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1914054984 shutdown_connections 2024-09-15T01:35:01.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.553+0000 7f5b28c6f640 1 -- 172.21.15.204:0/1914054984 wait complete. 2024-09-15T01:35:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm osd.9 2024-09-15T01:35:01.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.635+0000 7f8f50d1d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:01.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.635+0000 7f8f50d1d640 10 monclient: build_initial_monmap 2024-09-15T01:35:01.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.635+0000 7f8f50d1d640 10 monclient: monmap: 2024-09-15T01:35:01.638 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:01.638 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:01.638 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:01.637232+0000 2024-09-15T01:35:01.638 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:01.637232+0000 2024-09-15T01:35:01.638 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:01.638 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:01.639 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:01.639 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:01.639 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:01.639 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:01.639 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:01.639 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:01.639 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:01.639 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:01.639 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:01.639 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:01.639 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:01.639 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:01.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.636+0000 7f8f50d1d640 1 Processor -- start 2024-09-15T01:35:01.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.636+0000 7f8f50d1d640 1 -- start start 2024-09-15T01:35:01.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.636+0000 7f8f50d1d640 10 monclient: init 2024-09-15T01:35:01.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:01.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 10 monclient: _add_conns ranks=[8,6,9,4,10,0,7,1,3,2,5] 2024-09-15T01:35:01.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f4c0a2e30 0x7f8f4c132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 10 monclient(hunting): picked mon.noname-e con 0x7f8f4c0a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:01.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8f4c09fb80 0x7f8f4c09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 10 monclient(hunting): picked mon.noname-d con 0x7f8f4c09fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:01.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8f4c0a0a40 0x7f8f4c0a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 10 monclient(hunting): picked mon.noname-a con 0x7f8f4c0a0a40 addr v2:172.21.15.178:3300/0 2024-09-15T01:35:01.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:01.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4c09e6c0 con 0x7f8f4c0a0a40 2024-09-15T01:35:01.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:01.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0a04b0 con 0x7f8f4c09fb80 2024-09-15T01:35:01.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:01.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0a3200 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f50d1d640 10 monclient(hunting): authenticate will time out at 917.280174s 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f49d9b640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8f4c09fb80 msgr2=0x7f8f4c09ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f49d9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8f4c09fb80 0x7f8f4c09ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f4c0a2e30 0x7f8f4c132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4ad9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8f4c0a0a40 0x7f8f4c0a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4ad9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8f4c0a0a40 0x7f8f4c0a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:43402/0 (socket says 172.21.15.204:43402) 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f4c0a2e30 0x7f8f4c132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:56498/0 (socket says 172.21.15.204:56498) 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4ad9d640 1 -- 172.21.15.204:0/1095766279 learned_addr learned my addr 172.21.15.204:0/1095766279 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4ad9d640 10 monclient(hunting): get_auth_request con 0x7f8f4c0a0a40 auth_method 0 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:01.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 10 monclient(hunting): get_auth_request con 0x7f8f4c0a2e30 auth_method 0 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 10 monclient(hunting): handle_auth_done global_id 5462 payload 274 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 1 -- 172.21.15.204:0/1095766279 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8f4c09fb80 msgr2=0x7f8f4c09ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:01.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 1 --2- 172.21.15.204:0/1095766279 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8f4c09fb80 0x7f8f4c09ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 1 -- 172.21.15.204:0/1095766279 >> v2:172.21.15.178:3300/0 conn(0x7f8f4c0a0a40 msgr2=0x7f8f4c0a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:01.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 1 --2- 172.21.15.204:0/1095766279 >> v2:172.21.15.178:3300/0 conn(0x7f8f4c0a0a40 0x7f8f4c0a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 10 monclient: _finish_hunting 0 2024-09-15T01:35:01.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 1 monclient: found mon.noname-e 2024-09-15T01:35:01.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:01.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:01.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4a59c640 1 -- 172.21.15.204:0/1095766279 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c04b720 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.637+0000 7f8f4ad9d640 1 --2- 172.21.15.204:0/1095766279 >> v2:172.21.15.178:3300/0 conn(0x7f8f4c0a0a40 0x7f8f4c0a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:01.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4a59c640 1 --2- 172.21.15.204:0/1095766279 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f4c0a2e30 0x7f8f4c132a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009850 tx=0x7f8f3c0090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e3a3e5feac177ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:01.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4959a640 1 -- 172.21.15.204:0/1095766279 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8f3c02abe0 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:01.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:01.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4959a640 10 monclient: dump: 2024-09-15T01:35:01.653 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:01.653 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:01.653 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.653 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.653 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:01.653 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:01.654 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:01.654 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:01.654 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:01.654 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:01.654 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:01.654 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:01.654 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:01.654 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:01.654 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:01.654 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:01.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4959a640 10 monclient: _finish_auth 0 2024-09-15T01:35:01.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f50d1d640 5 monclient: authenticate success, global_id 5462 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f50d1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4959a640 1 -- 172.21.15.204:0/1095766279 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c02ad60 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f50d1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f50d1d640 4 set_mon_vals no callback set 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4959a640 1 -- 172.21.15.204:0/1095766279 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8f3c031ca0 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.638+0000 7f8f4959a640 10 monclient: dump: 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:01.656 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 10 monclient: shutdown 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 1 -- 172.21.15.204:0/1095766279 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f4c0a2e30 msgr2=0x7f8f4c132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 1 --2- 172.21.15.204:0/1095766279 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f4c0a2e30 0x7f8f4c132a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009850 tx=0x7f8f3c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 1 -- 172.21.15.204:0/1095766279 shutdown_connections 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 1 --2- 172.21.15.204:0/1095766279 >> v2:172.21.15.178:3300/0 conn(0x7f8f4c0a0a40 0x7f8f4c0a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 1 --2- 172.21.15.204:0/1095766279 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8f4c09fb80 0x7f8f4c09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 1 --2- 172.21.15.204:0/1095766279 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8f4c0a2e30 0x7f8f4c132a60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 1 -- 172.21.15.204:0/1095766279 >> 172.21.15.204:0/1095766279 conn(0x7f8f4c06e860 msgr2=0x7f8f4c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 1 -- 172.21.15.204:0/1095766279 shutdown_connections 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 1 -- 172.21.15.204:0/1095766279 wait complete. 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 10 monclient: build_initial_monmap 2024-09-15T01:35:01.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.639+0000 7f8f50d1d640 10 monclient: monmap: 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:01.658 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:01.659 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:01.659 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:01.659 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:01.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 1 Processor -- start 2024-09-15T01:35:01.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 1 -- start start 2024-09-15T01:35:01.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 10 monclient: init 2024-09-15T01:35:01.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:01.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 10 monclient: _add_conns ranks=[1,2,5,8,4,7,6,3,0] 2024-09-15T01:35:01.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c09fb80 0x7f8f4c130830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 10 monclient(hunting): picked mon.noname-f con 0x7f8f4c09fb80 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:01.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f4c0a0a40 0x7f8f4c11d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 10 monclient(hunting): picked mon.noname-b con 0x7f8f4c0a0a40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:01.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8f4c0a2e30 0x7f8f4c11d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 10 monclient(hunting): picked mon.noname-h con 0x7f8f4c0a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:01.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:01.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c09d650 con 0x7f8f4c09fb80 2024-09-15T01:35:01.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:01.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c11fa20 con 0x7f8f4c0a0a40 2024-09-15T01:35:01.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:01.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c09ca20 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:01.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f50d1d640 10 monclient(hunting): authenticate will time out at 917.283748s 2024-09-15T01:35:01.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f49d9b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f4c0a0a40 msgr2=0x7f8f4c11d3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:01.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f49d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f4c0a0a40 0x7f8f4c11d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:01.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4ad9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8f4c0a2e30 0x7f8f4c11d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:01.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c09fb80 0x7f8f4c130830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:01.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c09fb80 0x7f8f4c130830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:57236/0 (socket says 172.21.15.204:57236) 2024-09-15T01:35:01.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4ad9d640 10 monclient(hunting): get_auth_request con 0x7f8f4c0a2e30 auth_method 0 2024-09-15T01:35:01.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4a59c640 1 -- 172.21.15.204:0/2206392800 learned_addr learned my addr 172.21.15.204:0/2206392800 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:01.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:01.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:01.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:01.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4a59c640 10 monclient(hunting): get_auth_request con 0x7f8f4c09fb80 auth_method 0 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.640+0000 7f8f4a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4ad9d640 10 monclient(hunting): handle_auth_done global_id 5468 payload 824 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4ad9d640 1 -- 172.21.15.204:0/2206392800 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f4c0a0a40 msgr2=0x7f8f4c11d3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4ad9d640 1 --2- 172.21.15.204:0/2206392800 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f4c0a0a40 0x7f8f4c11d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4ad9d640 1 -- 172.21.15.204:0/2206392800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c09fb80 msgr2=0x7f8f4c130830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:01.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4ad9d640 1 --2- 172.21.15.204:0/2206392800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c09fb80 0x7f8f4c130830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:35:01.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4ad9d640 1 monclient: found mon.noname-h 2024-09-15T01:35:01.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:01.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4ad9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4ad9d640 1 -- 172.21.15.204:0/2206392800 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f44009ae0 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4a59c640 1 --2- 172.21.15.204:0/2206392800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c09fb80 0x7f8f4c130830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:01.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f4ad9d640 1 --2- 172.21.15.204:0/2206392800 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8f4c0a2e30 0x7f8f4c11d8e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c09cbf0 tx=0x7f8f4400b040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:01.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f3a7fc640 1 -- 172.21.15.204:0/2206392800 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8f44018070 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f3a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:01.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f3a7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:01.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f3a7fc640 10 monclient: dump: 2024-09-15T01:35:01.664 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:01.664 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:01.664 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.664 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.664 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:01.664 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:01.664 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:01.664 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:01.664 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:01.664 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:01.664 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:01.664 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:01.665 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:01.665 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:01.665 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:01.665 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:01.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f3a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:01.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f3a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:01.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f50d1d640 5 monclient: authenticate success, global_id 5468 2024-09-15T01:35:01.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f3a7fc640 1 -- 172.21.15.204:0/2206392800 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4400f040 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f50d1d640 10 monclient: _renew_subs 2024-09-15T01:35:01.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f50d1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f50d1d640 1 -- 172.21.15.204:0/2206392800 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c009390 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f3a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:01.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f3a7fc640 1 -- 172.21.15.204:0/2206392800 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8f440092e0 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f3a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:01.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f3a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:01.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f48d99640 4 set_mon_vals no callback set 2024-09-15T01:35:01.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f3a7fc640 10 monclient: dump: 2024-09-15T01:35:01.667 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:01.667 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:01.667 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.667 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:01.667 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:01.667 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:01.667 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:01.667 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:01.667 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:01.668 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:01.668 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:01.668 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:01.668 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:01.668 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:01.668 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:01.668 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:01.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f50d1d640 10 monclient: _renew_subs 2024-09-15T01:35:01.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f50d1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.641+0000 7f8f50d1d640 1 -- 172.21.15.204:0/2206392800 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c11ee50 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.642+0000 7f8f50d1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:01.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.642+0000 7f8f50d1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:01.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.642+0000 7f8f50d1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.642+0000 7f8f3a7fc640 1 -- 172.21.15.204:0/2206392800 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8f44003020 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.642+0000 7f8f50d1d640 1 -- 172.21.15.204:0/2206392800 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4c0668b0 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.642+0000 7f8f3a7fc640 1 --2- 172.21.15.204:0/2206392800 >> v2:172.21.15.204:6812/28210 conn(0x7f8f2c0397e0 0x7f8f2c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:01.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.642+0000 7f8f3a7fc640 1 -- 172.21.15.204:0/2206392800 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5184+0+0 (secure 0 0 0) 0x7f8f4404b870 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.642+0000 7f8f4a59c640 1 --2- 172.21.15.204:0/2206392800 >> v2:172.21.15.204:6812/28210 conn(0x7f8f2c0397e0 0x7f8f2c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:01.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.642+0000 7f8f4a59c640 10 monclient: get_auth_request con 0x7f8f2c0397e0 auth_method 0 2024-09-15T01:35:01.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.643+0000 7f8f4a59c640 1 --2- 172.21.15.204:0/2206392800 >> v2:172.21.15.204:6812/28210 conn(0x7f8f2c0397e0 0x7f8f2c03bc90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009720 tx=0x7f8f3c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:01.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.647+0000 7f8f3a7fc640 1 -- 172.21.15.204:0/2206392800 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8f4400cba0 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.647+0000 7f8f3a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:01.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.647+0000 7f8f3a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.806+0000 7f8f50d1d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9"}] 2024-09-15T01:35:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.806+0000 7f8f50d1d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-09-15T01:35:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.806+0000 7f8f50d1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.806+0000 7f8f50d1d640 1 -- 172.21.15.204:0/2206392800 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.9"} v 0) v1 -- 0x7f8f4c070cc0 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.807+0000 7f8f3a7fc640 1 -- 172.21.15.204:0/2206392800 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9"}]=0 device 'osd.9' does not appear in the crush map v98) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8f440133e0 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.807+0000 7f8f3a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-09-15T01:35:01.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.807+0000 7f8f3a7fc640 10 monclient: _finish_command 2 = system:0 device 'osd.9' does not appear in the crush map 2024-09-15T01:35:01.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.903+0000 7f8f50d1d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9"}] 2024-09-15T01:35:01.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.903+0000 7f8f50d1d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-09-15T01:35:01.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.903+0000 7f8f50d1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:01.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.903+0000 7f8f50d1d640 1 -- 172.21.15.204:0/2206392800 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.9"} v 0) v1 -- 0x7f8f4c133f40 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.903+0000 7f8f3a7fc640 1 -- 172.21.15.204:0/2206392800 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9"}]=0 device 'osd.9' does not appear in the crush map v98) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8f44012340 con 0x7f8f4c0a2e30 2024-09-15T01:35:01.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.904+0000 7f8f3a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-09-15T01:35:01.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.904+0000 7f8f3a7fc640 10 monclient: _finish_command 3 = system:0 device 'osd.9' does not appear in the crush map 2024-09-15T01:35:01.907 INFO:tasks.workunit.client.0.smithi204.stderr:device 'osd.9' does not appear in the crush map 2024-09-15T01:35:01.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 1 -- 172.21.15.204:0/2206392800 >> v2:172.21.15.204:6812/28210 conn(0x7f8f2c0397e0 msgr2=0x7f8f2c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:01.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 1 --2- 172.21.15.204:0/2206392800 >> v2:172.21.15.204:6812/28210 conn(0x7f8f2c0397e0 0x7f8f2c03bc90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009720 tx=0x7f8f3c032000 comp rx=0 tx=0).stop 2024-09-15T01:35:01.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 10 monclient: shutdown 2024-09-15T01:35:01.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:01.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 1 -- 172.21.15.204:0/2206392800 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8f4c0a2e30 msgr2=0x7f8f4c11d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:01.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 1 --2- 172.21.15.204:0/2206392800 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8f4c0a2e30 0x7f8f4c11d8e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c09cbf0 tx=0x7f8f4400b040 comp rx=0 tx=0).stop 2024-09-15T01:35:01.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 1 -- 172.21.15.204:0/2206392800 shutdown_connections 2024-09-15T01:35:01.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 1 --2- 172.21.15.204:0/2206392800 >> v2:172.21.15.204:6812/28210 conn(0x7f8f2c0397e0 0x7f8f2c03bc90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 1 --2- 172.21.15.204:0/2206392800 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8f4c0a2e30 0x7f8f4c11d8e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 1 --2- 172.21.15.204:0/2206392800 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f4c0a0a40 0x7f8f4c11d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 1 --2- 172.21.15.204:0/2206392800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f4c09fb80 0x7f8f4c130830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:01.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 1 -- 172.21.15.204:0/2206392800 >> 172.21.15.204:0/2206392800 conn(0x7f8f4c06e860 msgr2=0x7f8f4c09ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:01.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.905+0000 7f8f50d1d640 1 -- 172.21.15.204:0/2206392800 shutdown_connections 2024-09-15T01:35:01.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.906+0000 7f8f50d1d640 1 -- 172.21.15.204:0/2206392800 wait complete. 2024-09-15T01:35:01.917 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm foo 2024-09-15T01:35:01.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.987+0000 7f4a9ed54640 10 monclient: get_monmap_and_config 2024-09-15T01:35:01.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.987+0000 7f4a9ed54640 10 monclient: build_initial_monmap 2024-09-15T01:35:01.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.987+0000 7f4a9ed54640 10 monclient: monmap: 2024-09-15T01:35:01.990 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:01.990 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:01.990 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:01.988744+0000 2024-09-15T01:35:01.990 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:01.988744+0000 2024-09-15T01:35:01.990 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:01.990 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:01.990 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:01.990 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:01.990 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:01.990 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:01.991 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:01.991 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:01.991 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:01.991 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:01.991 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:01.991 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:01.991 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:01.991 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:02.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 1 Processor -- start 2024-09-15T01:35:02.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 1 -- start start 2024-09-15T01:35:02.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 10 monclient: init 2024-09-15T01:35:02.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:02.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 10 monclient: _add_conns ranks=[4,5,3,2,6,9,7,0,1,10,8] 2024-09-15T01:35:02.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a980a0b40 0x7f4a98130bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 10 monclient(hunting): picked mon.noname-c con 0x7f4a980a0b40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:02.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a98098940 0x7f4a98067130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 10 monclient(hunting): picked mon.noname-h con 0x7f4a98098940 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a98098d50 0x7f4a98067670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 10 monclient(hunting): picked mon.noname-g con 0x7f4a98098d50 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980a0f10 con 0x7f4a98098d50 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9802a7c0 con 0x7f4a980a0b40 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9809c1c0 con 0x7f4a98098940 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9ed54640 10 monclient(hunting): authenticate will time out at 917.631651s 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9d2f1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a98098d50 0x7f4a98067670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a97fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a98098940 0x7f4a98067130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:02.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a97fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a98098940 0x7f4a98067130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:41998/0 (socket says 172.21.15.204:41998) 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9d2f1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a98098d50 0x7f4a98067670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58850/0 (socket says 172.21.15.204:58850) 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a97fff640 1 -- 172.21.15.204:0/922644977 learned_addr learned my addr 172.21.15.204:0/922644977 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a97fff640 10 monclient(hunting): get_auth_request con 0x7f4a98098940 auth_method 0 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a97fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9caf0640 1 --2- 172.21.15.204:0/922644977 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a980a0b40 0x7f4a98130bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9d2f1640 10 monclient(hunting): get_auth_request con 0x7f4a98098d50 auth_method 0 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9d2f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9d2f1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9d2f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9caf0640 10 monclient(hunting): get_auth_request con 0x7f4a980a0b40 auth_method 0 2024-09-15T01:35:02.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9caf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9caf0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9caf0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9d2f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9d2f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9d2f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9caf0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.988+0000 7f4a9caf0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a9caf0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a97fff640 10 monclient(hunting): handle_auth_done global_id 5477 payload 274 2024-09-15T01:35:02.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a97fff640 1 -- 172.21.15.204:0/922644977 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a980a0b40 msgr2=0x7f4a98130bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a97fff640 1 --2- 172.21.15.204:0/922644977 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a980a0b40 0x7f4a98130bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a97fff640 1 -- 172.21.15.204:0/922644977 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a98098d50 msgr2=0x7f4a98067670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a97fff640 1 --2- 172.21.15.204:0/922644977 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a98098d50 0x7f4a98067670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a97fff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a97fff640 1 monclient: found mon.noname-h 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a97fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a97fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a97fff640 1 -- 172.21.15.204:0/922644977 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9804b720 con 0x7f4a98098940 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a9d2f1640 1 --2- 172.21.15.204:0/922644977 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a98098d50 0x7f4a98067670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a97fff640 1 --2- 172.21.15.204:0/922644977 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a98098940 0x7f4a98067130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009940 tx=0x7f4a840091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=6f84c5c8ad988017 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a977fe640 1 -- 172.21.15.204:0/922644977 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a8402ac60 con 0x7f4a98098940 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:02.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a977fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:02.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a977fe640 10 monclient: dump: 2024-09-15T01:35:02.007 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:02.007 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:02.007 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.007 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.007 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:02.007 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:02.007 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:02.007 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:02.007 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:02.007 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:02.007 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:02.008 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:02.008 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:02.008 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:02.008 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:02.008 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:02.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a977fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:02.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:02.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a9ed54640 5 monclient: authenticate success, global_id 5477 2024-09-15T01:35:02.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a9ed54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:02.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a977fe640 1 -- 172.21.15.204:0/922644977 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8402ade0 con 0x7f4a98098940 2024-09-15T01:35:02.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a977fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:02.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a9ed54640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:02.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a977fe640 1 -- 172.21.15.204:0/922644977 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a84031ca0 con 0x7f4a98098940 2024-09-15T01:35:02.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a9ed54640 4 set_mon_vals no callback set 2024-09-15T01:35:02.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:02.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a977fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:02.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.989+0000 7f4a977fe640 10 monclient: dump: 2024-09-15T01:35:02.009 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:02.009 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:02.009 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.009 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.009 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:02.009 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:02.009 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.990+0000 7f4a9ed54640 10 monclient: shutdown 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.990+0000 7f4a9ed54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.990+0000 7f4a9ed54640 1 -- 172.21.15.204:0/922644977 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a98098940 msgr2=0x7f4a98067130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:02.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.990+0000 7f4a9ed54640 1 --2- 172.21.15.204:0/922644977 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a98098940 0x7f4a98067130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009940 tx=0x7f4a840091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.990+0000 7f4a9ed54640 1 -- 172.21.15.204:0/922644977 shutdown_connections 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.990+0000 7f4a9ed54640 1 --2- 172.21.15.204:0/922644977 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4a98098940 0x7f4a98067130 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.990+0000 7f4a9ed54640 1 --2- 172.21.15.204:0/922644977 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a98098d50 0x7f4a98067670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.990+0000 7f4a9ed54640 1 --2- 172.21.15.204:0/922644977 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4a980a0b40 0x7f4a98130bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.990+0000 7f4a9ed54640 1 -- 172.21.15.204:0/922644977 >> 172.21.15.204:0/922644977 conn(0x7f4a980a2e30 msgr2=0x7f4a980a0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.990+0000 7f4a9ed54640 1 -- 172.21.15.204:0/922644977 shutdown_connections 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.990+0000 7f4a9ed54640 1 -- 172.21.15.204:0/922644977 wait complete. 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.990+0000 7f4a9ed54640 10 monclient: build_initial_monmap 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.991+0000 7f4a9ed54640 10 monclient: monmap: 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:02.011 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.991+0000 7f4a9ed54640 1 Processor -- start 2024-09-15T01:35:02.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.991+0000 7f4a9ed54640 1 -- start start 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.991+0000 7f4a9ed54640 10 monclient: init 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.991+0000 7f4a9ed54640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.991+0000 7f4a9ed54640 10 monclient: _add_conns ranks=[8,6,7,5,2,3,1,0,4] 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.991+0000 7f4a9ed54640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4a98098940 0x7f4a98095a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.991+0000 7f4a9ed54640 10 monclient(hunting): picked mon.noname-e con 0x7f4a98098940 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.991+0000 7f4a9ed54640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a98098d50 0x7f4a9806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.991+0000 7f4a9ed54640 10 monclient(hunting): picked mon.noname-d con 0x7f4a98098d50 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9ed54640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4a980a0b40 0x7f4a98129640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9ed54640 10 monclient(hunting): picked mon.noname-i con 0x7f4a980a0b40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9ed54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9ed54640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9809b210 con 0x7f4a98098d50 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9ed54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:02.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9ed54640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9806b870 con 0x7f4a980a0b40 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9ed54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9ed54640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9806f280 con 0x7f4a98098940 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9ed54640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9ed54640 10 monclient(hunting): authenticate will time out at 917.635041s 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9d2f1640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4a980a0b40 msgr2=0x7f4a98129640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9d2f1640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4a980a0b40 0x7f4a98129640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a97fff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a98098d50 msgr2=0x7f4a9806ed40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a97fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a98098d50 0x7f4a9806ed40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4a98098940 0x7f4a98095a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4a98098940 0x7f4a98095a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:56514/0 (socket says 172.21.15.204:56514) 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 1 -- 172.21.15.204:0/777622524 learned_addr learned my addr 172.21.15.204:0/777622524 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 10 monclient(hunting): get_auth_request con 0x7f4a98098940 auth_method 0 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:02.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 10 monclient(hunting): handle_auth_done global_id 5471 payload 824 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 1 -- 172.21.15.204:0/777622524 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4a980a0b40 msgr2=0x7f4a98129640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 1 --2- 172.21.15.204:0/777622524 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4a980a0b40 0x7f4a98129640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 1 -- 172.21.15.204:0/777622524 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a98098d50 msgr2=0x7f4a9806ed40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 1 --2- 172.21.15.204:0/777622524 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a98098d50 0x7f4a9806ed40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 10 monclient: _finish_hunting 0 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 1 monclient: found mon.noname-e 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:02.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 1 -- 172.21.15.204:0/777622524 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a90009a20 con 0x7f4a98098940 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.992+0000 7f4a9caf0640 1 --2- 172.21.15.204:0/777622524 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4a98098940 0x7f4a98095a40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00ea20 tx=0x7f4a8c0090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a94ff9640 1 -- 172.21.15.204:0/777622524 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a8c00ce90 con 0x7f4a98098940 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a94ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a94ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a94ff9640 10 monclient: dump: 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:02.016 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:02.017 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:02.017 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:02.017 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:02.017 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:02.017 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:02.017 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:02.017 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:02.017 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:02.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a94ff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:02.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a94ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a94ff9640 1 -- 172.21.15.204:0/777622524 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c009600 con 0x7f4a98098940 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a9ed54640 5 monclient: authenticate success, global_id 5471 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a9ed54640 10 monclient: _renew_subs 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a9ed54640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a9ed54640 1 -- 172.21.15.204:0/777622524 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a84009450 con 0x7f4a98098940 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a94ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a94ff9640 1 -- 172.21.15.204:0/777622524 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a8c0138f0 con 0x7f4a98098940 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a94ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a94ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a96ffd640 4 set_mon_vals no callback set 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a94ff9640 10 monclient: dump: 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:02.018 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:02.019 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a9ed54640 10 monclient: _renew_subs 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a9ed54640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.993+0000 7f4a9ed54640 1 -- 172.21.15.204:0/777622524 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9809b4f0 con 0x7f4a98098940 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.994+0000 7f4a94ff9640 1 -- 172.21.15.204:0/777622524 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4a8c013470 con 0x7f4a98098940 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.994+0000 7f4a9ed54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.994+0000 7f4a9ed54640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.994+0000 7f4a9ed54640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.994+0000 7f4a9ed54640 1 -- 172.21.15.204:0/777622524 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a98070cc0 con 0x7f4a98098940 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.994+0000 7f4a94ff9640 1 --2- 172.21.15.204:0/777622524 >> v2:172.21.15.204:6812/28210 conn(0x7f4a7c0397b0 0x7f4a7c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.994+0000 7f4a94ff9640 1 -- 172.21.15.204:0/777622524 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5184+0+0 (secure 0 0 0) 0x7f4a8c04b8f0 con 0x7f4a98098940 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.994+0000 7f4a97fff640 1 --2- 172.21.15.204:0/777622524 >> v2:172.21.15.204:6812/28210 conn(0x7f4a7c0397b0 0x7f4a7c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.994+0000 7f4a97fff640 10 monclient: get_auth_request con 0x7f4a7c0397b0 auth_method 0 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.994+0000 7f4a97fff640 1 --2- 172.21.15.204:0/777622524 >> v2:172.21.15.204:6812/28210 conn(0x7f4a7c0397b0 0x7f4a7c03bc60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4a84034000 tx=0x7f4a84032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:02.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.999+0000 7f4a94ff9640 1 -- 172.21.15.204:0/777622524 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4a8c013720 con 0x7f4a98098940 2024-09-15T01:35:02.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.999+0000 7f4a94ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:02.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:01.999+0000 7f4a94ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:02.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.157+0000 7f4a9ed54640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:35:02.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.157+0000 7f4a9ed54640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:35:02.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.157+0000 7f4a9ed54640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:02.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.157+0000 7f4a9ed54640 1 -- 172.21.15.204:0/777622524 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) v1 -- 0x7f4a9809de80 con 0x7f4a98098940 2024-09-15T01:35:02.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.445+0000 7f4a94ff9640 1 -- 172.21.15.204:0/777622524 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 removed item id -10 name 'foo' from crush map v99) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4a8c01bbf0 con 0x7f4a98098940 2024-09-15T01:35:02.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.445+0000 7f4a94ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:35:02.448 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.445+0000 7f4a94ff9640 10 monclient: _finish_command 2 = system:0 removed item id -10 name 'foo' from crush map 2024-09-15T01:35:02.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.545+0000 7f4a9ed54640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:35:02.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.545+0000 7f4a9ed54640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:35:02.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.545+0000 7f4a9ed54640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:02.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.545+0000 7f4a9ed54640 1 -- 172.21.15.204:0/777622524 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) v1 -- 0x7f4a980a10b0 con 0x7f4a98098940 2024-09-15T01:35:02.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.546+0000 7f4a94ff9640 1 -- 172.21.15.204:0/777622524 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 device 'foo' does not appear in the crush map v99) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4a8c01bdd0 con 0x7f4a98098940 2024-09-15T01:35:02.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.546+0000 7f4a94ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-09-15T01:35:02.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.546+0000 7f4a94ff9640 10 monclient: _finish_command 3 = system:0 device 'foo' does not appear in the crush map 2024-09-15T01:35:02.548 INFO:tasks.workunit.client.0.smithi204.stderr:device 'foo' does not appear in the crush map 2024-09-15T01:35:02.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.547+0000 7f4a9ed54640 1 -- 172.21.15.204:0/777622524 >> v2:172.21.15.204:6812/28210 conn(0x7f4a7c0397b0 msgr2=0x7f4a7c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:02.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.547+0000 7f4a9ed54640 1 --2- 172.21.15.204:0/777622524 >> v2:172.21.15.204:6812/28210 conn(0x7f4a7c0397b0 0x7f4a7c03bc60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4a84034000 tx=0x7f4a84032000 comp rx=0 tx=0).stop 2024-09-15T01:35:02.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.547+0000 7f4a9ed54640 10 monclient: shutdown 2024-09-15T01:35:02.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.547+0000 7f4a9ed54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:02.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.547+0000 7f4a9ed54640 1 -- 172.21.15.204:0/777622524 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4a98098940 msgr2=0x7f4a98095a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:02.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.547+0000 7f4a9ed54640 1 --2- 172.21.15.204:0/777622524 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4a98098940 0x7f4a98095a40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00ea20 tx=0x7f4a8c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.547+0000 7f4a9ed54640 1 -- 172.21.15.204:0/777622524 shutdown_connections 2024-09-15T01:35:02.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.548+0000 7f4a9ed54640 1 --2- 172.21.15.204:0/777622524 >> v2:172.21.15.204:6812/28210 conn(0x7f4a7c0397b0 0x7f4a7c03bc60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.548+0000 7f4a9ed54640 1 --2- 172.21.15.204:0/777622524 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4a980a0b40 0x7f4a98129640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.548+0000 7f4a9ed54640 1 --2- 172.21.15.204:0/777622524 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a98098d50 0x7f4a9806ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.548+0000 7f4a9ed54640 1 --2- 172.21.15.204:0/777622524 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4a98098940 0x7f4a98095a40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.548+0000 7f4a9ed54640 1 -- 172.21.15.204:0/777622524 >> 172.21.15.204:0/777622524 conn(0x7f4a980a2e30 msgr2=0x7f4a98067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:02.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.548+0000 7f4a9ed54640 1 -- 172.21.15.204:0/777622524 shutdown_connections 2024-09-15T01:35:02.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.548+0000 7f4a9ed54640 1 -- 172.21.15.204:0/777622524 wait complete. 2024-09-15T01:35:02.560 INFO:tasks.workunit.client.0.smithi204.stderr:++ ceph osd create 2024-09-15T01:35:02.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.627+0000 7fc621a9f640 10 monclient: get_monmap_and_config 2024-09-15T01:35:02.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.627+0000 7fc621a9f640 10 monclient: build_initial_monmap 2024-09-15T01:35:02.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.627+0000 7fc621a9f640 10 monclient: monmap: 2024-09-15T01:35:02.630 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:02.630 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:02.630 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:02.629185+0000 2024-09-15T01:35:02.630 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:02.629185+0000 2024-09-15T01:35:02.630 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:02.630 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:02.631 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:02.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 1 Processor -- start 2024-09-15T01:35:02.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 1 -- start start 2024-09-15T01:35:02.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 10 monclient: init 2024-09-15T01:35:02.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:02.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 10 monclient: _add_conns ranks=[2,0,5,7,3,10,6,9,8,4,1] 2024-09-15T01:35:02.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc61c0a2400 0x7fc61c0a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 10 monclient(hunting): picked mon.noname-b con 0x7fc61c0a2400 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:02.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc61c09f990 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 0x7fc61c13a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 10 monclient(hunting): picked mon.noname-h con 0x7fc61c137c60 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61c0648a0 con 0x7fc61c09f990 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c09dc50 con 0x7fc61c0a2400 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.628+0000 7fc621a9f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c09e1a0 con 0x7fc61c137c60 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc621a9f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc621a9f640 10 monclient(hunting): authenticate will time out at 918.272012s 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61b7fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc61c0a2400 msgr2=0x7fc61c0a0ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 0x7fc61c13a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61b7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc61c0a2400 0x7fc61c0a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 0x7fc61c13a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:42002/0 (socket says 172.21.15.204:42002) 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 1 -- 172.21.15.204:0/3737043586 learned_addr learned my addr 172.21.15.204:0/3737043586 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 10 monclient(hunting): get_auth_request con 0x7fc61c137c60 auth_method 0 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 10 monclient(hunting): handle_auth_done global_id 5486 payload 274 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 1 -- 172.21.15.204:0/3737043586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc61c0a2400 msgr2=0x7fc61c0a0ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 1 --2- 172.21.15.204:0/3737043586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc61c0a2400 0x7fc61c0a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 1 -- 172.21.15.204:0/3737043586 >> v1:172.21.15.178:6789/0 conn(0x7fc61c09f990 legacy=0x7fc61c09fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 1 monclient: found mon.noname-h 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.629+0000 7fc61bfff640 1 -- 172.21.15.204:0/3737043586 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c0a27d0 con 0x7fc61c137c60 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61bfff640 1 --2- 172.21.15.204:0/3737043586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 0x7fc61c13a040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc614009f50 tx=0x7fc614009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=607bf0f7d023f941 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61a7fc640 1 -- 172.21.15.204:0/3737043586 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc614007880 con 0x7fc61c137c60 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61a7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61a7fc640 10 monclient: dump: 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 5 monclient: authenticate success, global_id 5486 2024-09-15T01:35:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61a7fc640 1 -- 172.21.15.204:0/3737043586 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61402d050 con 0x7fc61c137c60 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 4 set_mon_vals no callback set 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61a7fc640 1 -- 172.21.15.204:0/3737043586 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc614030360 con 0x7fc61c137c60 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc61a7fc640 10 monclient: dump: 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 10 monclient: shutdown 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 1 -- 172.21.15.204:0/3737043586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 msgr2=0x7fc61c13a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 1 --2- 172.21.15.204:0/3737043586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 0x7fc61c13a040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc614009f50 tx=0x7fc614009840 comp rx=0 tx=0).stop 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 1 -- 172.21.15.204:0/3737043586 shutdown_connections 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 1 --2- 172.21.15.204:0/3737043586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 0x7fc61c13a040 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 1 --2- 172.21.15.204:0/3737043586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc61c0a2400 0x7fc61c0a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 1 -- 172.21.15.204:0/3737043586 >> 172.21.15.204:0/3737043586 conn(0x7fc61c06e860 msgr2=0x7fc61c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 1 -- 172.21.15.204:0/3737043586 shutdown_connections 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.630+0000 7fc621a9f640 1 -- 172.21.15.204:0/3737043586 wait complete. 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.631+0000 7fc621a9f640 10 monclient: build_initial_monmap 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.631+0000 7fc621a9f640 10 monclient: monmap: 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.631+0000 7fc621a9f640 1 Processor -- start 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.631+0000 7fc621a9f640 1 -- start start 2024-09-15T01:35:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.631+0000 7fc621a9f640 10 monclient: init 2024-09-15T01:35:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 10 monclient: _add_conns ranks=[7,4,5,6,0,3,1,8,2] 2024-09-15T01:35:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc61c09f990 0x7fc61c11f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 10 monclient(hunting): picked mon.noname-i con 0x7fc61c09f990 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc61c0a2400 0x7fc61c11f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 10 monclient(hunting): picked mon.noname-c con 0x7fc61c0a2400 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 0x7fc61c11fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 10 monclient(hunting): picked mon.noname-h con 0x7fc61c137c60 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c09d3d0 con 0x7fc61c0a2400 2024-09-15T01:35:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61b7fe640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc61c09f990 msgr2=0x7fc61c11f120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c09cad0 con 0x7fc61c137c60 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c1228e0 con 0x7fc61c09f990 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61b7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc61c09f990 0x7fc61c11f120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc621a9f640 10 monclient(hunting): authenticate will time out at 918.275381s 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 0x7fc61c11fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 0x7fc61c11fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:42016/0 (socket says 172.21.15.204:42016) 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 1 -- 172.21.15.204:0/3269184221 learned_addr learned my addr 172.21.15.204:0/3269184221 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 10 monclient(hunting): get_auth_request con 0x7fc61c137c60 auth_method 0 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61affd640 1 --2- 172.21.15.204:0/3269184221 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc61c0a2400 0x7fc61c11f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61affd640 10 monclient(hunting): get_auth_request con 0x7fc61c0a2400 auth_method 0 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 10 monclient(hunting): handle_auth_done global_id 5495 payload 824 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 1 -- 172.21.15.204:0/3269184221 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc61c09f990 msgr2=0x7fc61c11f120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 1 --2- 172.21.15.204:0/3269184221 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc61c09f990 0x7fc61c11f120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 1 -- 172.21.15.204:0/3269184221 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc61c0a2400 msgr2=0x7fc61c11f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 1 --2- 172.21.15.204:0/3269184221 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc61c0a2400 0x7fc61c11f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 1 monclient: found mon.noname-h 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 1 -- 172.21.15.204:0/3269184221 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc614009ae0 con 0x7fc61c137c60 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.632+0000 7fc61bfff640 1 --2- 172.21.15.204:0/3269184221 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 0x7fc61c11fba0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc61402abc0 tx=0x7fc614036000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 1 -- 172.21.15.204:0/3269184221 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc61403cab0 con 0x7fc61c137c60 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 10 monclient: dump: 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc621a9f640 5 monclient: authenticate success, global_id 5495 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 1 -- 172.21.15.204:0/3269184221 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61403cc50 con 0x7fc61c137c60 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 1 -- 172.21.15.204:0/3269184221 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc61403cdc0 con 0x7fc61c137c60 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc621a9f640 10 monclient: _renew_subs 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc621a9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc621a9f640 1 -- 172.21.15.204:0/3269184221 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61c1217a0 con 0x7fc61c137c60 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc619ffb640 4 set_mon_vals no callback set 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 10 monclient: dump: 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc621a9f640 10 monclient: _renew_subs 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc621a9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc621a9f640 1 -- 172.21.15.204:0/3269184221 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61c120ad0 con 0x7fc61c137c60 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.633+0000 7fc6037fe640 1 -- 172.21.15.204:0/3269184221 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc614003040 con 0x7fc61c137c60 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.634+0000 7fc621a9f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.634+0000 7fc621a9f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.634+0000 7fc621a9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.634+0000 7fc621a9f640 1 -- 172.21.15.204:0/3269184221 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61c070b80 con 0x7fc61c137c60 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.634+0000 7fc6037fe640 1 --2- 172.21.15.204:0/3269184221 >> v2:172.21.15.204:6812/28210 conn(0x7fc6080397b0 0x7fc60803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.634+0000 7fc6037fe640 1 -- 172.21.15.204:0/3269184221 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4886+0+0 (secure 0 0 0) 0x7fc61402ae40 con 0x7fc61c137c60 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.634+0000 7fc61b7fe640 1 --2- 172.21.15.204:0/3269184221 >> v2:172.21.15.204:6812/28210 conn(0x7fc6080397b0 0x7fc60803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.634+0000 7fc61b7fe640 10 monclient: get_auth_request con 0x7fc6080397b0 auth_method 0 2024-09-15T01:35:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.634+0000 7fc61b7fe640 1 --2- 172.21.15.204:0/3269184221 >> v2:172.21.15.204:6812/28210 conn(0x7fc6080397b0 0x7fc60803bc60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc60c006fd0 tx=0x7fc60c006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.638+0000 7fc6037fe640 1 -- 172.21.15.204:0/3269184221 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc61403b380 con 0x7fc61c137c60 2024-09-15T01:35:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.638+0000 7fc6037fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.638+0000 7fc6037fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:02.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.808+0000 7fc621a9f640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T01:35:02.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.808+0000 7fc621a9f640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-09-15T01:35:02.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.808+0000 7fc621a9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:02.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:02.808+0000 7fc621a9f640 1 -- 172.21.15.204:0/3269184221 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7fc61c1217a0 con 0x7fc61c137c60 2024-09-15T01:35:03.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:03.450+0000 7fc6037fe640 1 -- 172.21.15.204:0/3269184221 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v100) v1 ==== 58+0+3 (secure 0 0 0) 0x7fc61c1217a0 con 0x7fc61c137c60 2024-09-15T01:35:03.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:03.450+0000 7fc6037fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-09-15T01:35:03.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:03.450+0000 7fc6037fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:03.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:03.556+0000 7fc621a9f640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T01:35:03.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:03.556+0000 7fc621a9f640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-09-15T01:35:03.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:03.556+0000 7fc621a9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:03.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:03.556+0000 7fc621a9f640 1 -- 172.21.15.204:0/3269184221 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7fc61c1217a0 con 0x7fc61c137c60 2024-09-15T01:35:04.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.449+0000 7fc6037fe640 1 -- 172.21.15.204:0/3269184221 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v101) v1 ==== 58+0+3 (secure 0 0 0) 0x7fc61c1217a0 con 0x7fc61c137c60 2024-09-15T01:35:04.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.449+0000 7fc6037fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-09-15T01:35:04.452 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.449+0000 7fc6037fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:04.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.451+0000 7fc621a9f640 1 -- 172.21.15.204:0/3269184221 >> v2:172.21.15.204:6812/28210 conn(0x7fc6080397b0 msgr2=0x7fc60803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:04.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.451+0000 7fc621a9f640 1 --2- 172.21.15.204:0/3269184221 >> v2:172.21.15.204:6812/28210 conn(0x7fc6080397b0 0x7fc60803bc60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc60c006fd0 tx=0x7fc60c006d30 comp rx=0 tx=0).stop 2024-09-15T01:35:04.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.451+0000 7fc621a9f640 10 monclient: shutdown 2024-09-15T01:35:04.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.451+0000 7fc621a9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:04.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.451+0000 7fc621a9f640 1 -- 172.21.15.204:0/3269184221 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 msgr2=0x7fc61c11fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:04.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.451+0000 7fc621a9f640 1 --2- 172.21.15.204:0/3269184221 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 0x7fc61c11fba0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc61402abc0 tx=0x7fc614036000 comp rx=0 tx=0).stop 2024-09-15T01:35:04.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.451+0000 7fc621a9f640 1 -- 172.21.15.204:0/3269184221 shutdown_connections 2024-09-15T01:35:04.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.451+0000 7fc621a9f640 1 --2- 172.21.15.204:0/3269184221 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc61c137c60 0x7fc61c11fba0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:04.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.452+0000 7fc621a9f640 1 --2- 172.21.15.204:0/3269184221 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc61c0a2400 0x7fc61c11f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:04.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.452+0000 7fc621a9f640 1 --2- 172.21.15.204:0/3269184221 >> v2:172.21.15.204:6812/28210 conn(0x7fc6080397b0 0x7fc60803bc60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:04.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.452+0000 7fc621a9f640 1 --2- 172.21.15.204:0/3269184221 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc61c09f990 0x7fc61c11f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:04.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.452+0000 7fc621a9f640 1 -- 172.21.15.204:0/3269184221 >> 172.21.15.204:0/3269184221 conn(0x7fc61c06e860 msgr2=0x7fc61c0a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:04.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.452+0000 7fc621a9f640 1 -- 172.21.15.204:0/3269184221 shutdown_connections 2024-09-15T01:35:04.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.452+0000 7fc621a9f640 1 -- 172.21.15.204:0/3269184221 wait complete. 2024-09-15T01:35:04.464 INFO:tasks.workunit.client.0.smithi204.stderr:+ o3=11 2024-09-15T01:35:04.464 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add 11 123 root=default 2024-09-15T01:35:04.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.540+0000 7f568a460640 10 monclient: get_monmap_and_config 2024-09-15T01:35:04.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.540+0000 7f568a460640 10 monclient: build_initial_monmap 2024-09-15T01:35:04.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.540+0000 7f568a460640 10 monclient: monmap: 2024-09-15T01:35:04.543 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:04.543 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:04.543 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:04.542187+0000 2024-09-15T01:35:04.543 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:04.542187+0000 2024-09-15T01:35:04.543 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:04.543 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:04.543 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:04.543 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:04.544 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:04.544 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:04.544 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:04.544 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:04.544 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:04.544 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:04.544 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:04.544 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:04.544 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:04.544 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:04.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.541+0000 7f568a460640 1 Processor -- start 2024-09-15T01:35:04.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.541+0000 7f568a460640 1 -- start start 2024-09-15T01:35:04.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.541+0000 7f568a460640 10 monclient: init 2024-09-15T01:35:04.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.541+0000 7f568a460640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:04.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.541+0000 7f568a460640 10 monclient: _add_conns ranks=[5,2,4,9,7,3,0,1,6,8,10] 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.541+0000 7f568a460640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 0x7f5684132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 10 monclient(hunting): picked mon.noname-h con 0x7f56840a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568409fa20 0x7f568409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 10 monclient(hunting): picked mon.noname-b con 0x7f568409fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56840a08e0 0x7f56840a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 10 monclient(hunting): picked mon.noname-c con 0x7f56840a08e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56840a30a0 con 0x7f568409fa20 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56840a0330 con 0x7f56840a08e0 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56840a1db0 con 0x7f56840a2cd0 2024-09-15T01:35:04.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f568a460640 10 monclient(hunting): authenticate will time out at 920.185066s 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 0x7f5684132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 0x7f5684132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:42026/0 (socket says 172.21.15.204:42026) 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 1 -- 172.21.15.204:0/2752747494 learned_addr learned my addr 172.21.15.204:0/2752747494 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 10 monclient(hunting): get_auth_request con 0x7f56840a2cd0 auth_method 0 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f56837fe640 1 -- 172.21.15.204:0/2752747494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568409fa20 msgr2=0x7f568409fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f56837fe640 1 --2- 172.21.15.204:0/2752747494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568409fa20 0x7f568409fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:04.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f56889fd640 1 --2- 172.21.15.204:0/2752747494 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56840a08e0 0x7f56840a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 10 monclient(hunting): handle_auth_done global_id 5504 payload 274 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 1 -- 172.21.15.204:0/2752747494 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56840a08e0 msgr2=0x7f56840a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 1 --2- 172.21.15.204:0/2752747494 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56840a08e0 0x7f56840a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 1 -- 172.21.15.204:0/2752747494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568409fa20 msgr2=0x7f568409fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 1 --2- 172.21.15.204:0/2752747494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568409fa20 0x7f568409fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 1 monclient: found mon.noname-h 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f5683fff640 1 -- 172.21.15.204:0/2752747494 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56840648a0 con 0x7f56840a2cd0 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f56889fd640 10 monclient: get_auth_request con 0x7f56840a08e0 auth_method 0 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.542+0000 7f56889fd640 1 --2- 172.21.15.204:0/2752747494 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56840a08e0 0x7f56840a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:35:04.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5683fff640 1 --2- 172.21.15.204:0/2752747494 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 0x7f5684132c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5674009940 tx=0x7f56740091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=59d4ebf4ec4bf690 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5682ffd640 1 -- 172.21.15.204:0/2752747494 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f567402ac80 con 0x7f56840a2cd0 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5682ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5682ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5682ffd640 10 monclient: dump: 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:04.556 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:04.557 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:04.557 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:04.557 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:04.557 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:04.557 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:04.557 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:04.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5682ffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:04.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5682ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:04.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 5 monclient: authenticate success, global_id 5504 2024-09-15T01:35:04.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:04.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5682ffd640 1 -- 172.21.15.204:0/2752747494 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567402ae00 con 0x7f56840a2cd0 2024-09-15T01:35:04.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5682ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 4 set_mon_vals no callback set 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5682ffd640 1 -- 172.21.15.204:0/2752747494 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5674031ca0 con 0x7f56840a2cd0 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5682ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5682ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f5682ffd640 10 monclient: dump: 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:04.558 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 10 monclient: shutdown 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 1 -- 172.21.15.204:0/2752747494 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 msgr2=0x7f5684132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 1 --2- 172.21.15.204:0/2752747494 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 0x7f5684132c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5674009940 tx=0x7f56740091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 1 -- 172.21.15.204:0/2752747494 shutdown_connections 2024-09-15T01:35:04.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 1 --2- 172.21.15.204:0/2752747494 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56840a08e0 0x7f56840a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 1 --2- 172.21.15.204:0/2752747494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568409fa20 0x7f568409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 1 --2- 172.21.15.204:0/2752747494 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 0x7f5684132c00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.543+0000 7f568a460640 1 -- 172.21.15.204:0/2752747494 >> 172.21.15.204:0/2752747494 conn(0x7f568406e860 msgr2=0x7f568406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.544+0000 7f568a460640 1 -- 172.21.15.204:0/2752747494 shutdown_connections 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.544+0000 7f568a460640 1 -- 172.21.15.204:0/2752747494 wait complete. 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.544+0000 7f568a460640 10 monclient: build_initial_monmap 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.544+0000 7f568a460640 10 monclient: monmap: 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:04.560 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.544+0000 7f568a460640 1 Processor -- start 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 1 -- start start 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 10 monclient: init 2024-09-15T01:35:04.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 10 monclient: _add_conns ranks=[2,7,5,4,0,3,6,1,8] 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568409fa20 0x7f5684130d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 10 monclient(hunting): picked mon.noname-b con 0x7f568409fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f56840a08e0 0x7f568411d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 10 monclient(hunting): picked mon.noname-i con 0x7f56840a08e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 0x7f568411ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 10 monclient(hunting): picked mon.noname-h con 0x7f56840a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5684133e40 con 0x7f568409fa20 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f568409d3d0 con 0x7f56840a2cd0 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f568409cb60 con 0x7f56840a08e0 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:04.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f568a460640 10 monclient(hunting): authenticate will time out at 920.188486s 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56837fe640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f56840a08e0 msgr2=0x7f568411d890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56837fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f56840a08e0 0x7f568411d890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f5683fff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568409fa20 msgr2=0x7f5684130d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f5683fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568409fa20 0x7f5684130d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 0x7f568411ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 0x7f568411ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:42036/0 (socket says 172.21.15.204:42036) 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 1 -- 172.21.15.204:0/1747583018 learned_addr learned my addr 172.21.15.204:0/1747583018 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 10 monclient(hunting): get_auth_request con 0x7f56840a2cd0 auth_method 0 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:04.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 10 monclient(hunting): handle_auth_done global_id 5513 payload 824 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 1 -- 172.21.15.204:0/1747583018 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f56840a08e0 msgr2=0x7f568411d890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 1 --2- 172.21.15.204:0/1747583018 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f56840a08e0 0x7f568411d890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 1 -- 172.21.15.204:0/1747583018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568409fa20 msgr2=0x7f5684130d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 1 --2- 172.21.15.204:0/1747583018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568409fa20 0x7f5684130d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 10 monclient: _finish_hunting 0 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 1 monclient: found mon.noname-h 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.545+0000 7f56889fd640 1 -- 172.21.15.204:0/1747583018 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5674009450 con 0x7f56840a2cd0 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f56889fd640 1 --2- 172.21.15.204:0/1747583018 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 0x7f568411ddd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f567c00a380 tx=0x7f567c0078f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:04.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f566bfff640 1 -- 172.21.15.204:0/1747583018 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f567c00dad0 con 0x7f56840a2cd0 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f566bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f566bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f566bfff640 10 monclient: dump: 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:04.565 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f566bfff640 10 monclient: _finish_auth 0 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f566bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f568a460640 5 monclient: authenticate success, global_id 5513 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f566bfff640 1 -- 172.21.15.204:0/1747583018 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c00dc70 con 0x7f56840a2cd0 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f568a460640 10 monclient: _renew_subs 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f568a460640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f568a460640 1 -- 172.21.15.204:0/1747583018 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5684121470 con 0x7f56840a2cd0 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f566bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f566bfff640 1 -- 172.21.15.204:0/1747583018 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f567c00dde0 con 0x7f56840a2cd0 2024-09-15T01:35:04.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f566bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f566bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f566bfff640 10 monclient: dump: 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:04.567 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f568a460640 10 monclient: _renew_subs 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f56827fc640 4 set_mon_vals no callback set 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f568a460640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.546+0000 7f568a460640 1 -- 172.21.15.204:0/1747583018 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56841427b0 con 0x7f56840a2cd0 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.547+0000 7f566bfff640 1 -- 172.21.15.204:0/1747583018 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f567c00d650 con 0x7f56840a2cd0 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.547+0000 7f568a460640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.547+0000 7f568a460640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.547+0000 7f568a460640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:04.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.547+0000 7f566bfff640 1 --2- 172.21.15.204:0/1747583018 >> v2:172.21.15.204:6812/28210 conn(0x7f56700397b0 0x7f567003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:04.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.547+0000 7f568a460640 1 -- 172.21.15.204:0/1747583018 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568411ffb0 con 0x7f56840a2cd0 2024-09-15T01:35:04.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.547+0000 7f566bfff640 1 -- 172.21.15.204:0/1747583018 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5116+0+0 (secure 0 0 0) 0x7f567c04ba70 con 0x7f56840a2cd0 2024-09-15T01:35:04.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.547+0000 7f5683fff640 1 --2- 172.21.15.204:0/1747583018 >> v2:172.21.15.204:6812/28210 conn(0x7f56700397b0 0x7f567003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:04.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.547+0000 7f5683fff640 10 monclient: get_auth_request con 0x7f56700397b0 auth_method 0 2024-09-15T01:35:04.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.547+0000 7f5683fff640 1 --2- 172.21.15.204:0/1747583018 >> v2:172.21.15.204:6812/28210 conn(0x7f56700397b0 0x7f567003bc60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5674034000 tx=0x7f5674032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:04.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.551+0000 7f566bfff640 1 -- 172.21.15.204:0/1747583018 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f567c01a070 con 0x7f56840a2cd0 2024-09-15T01:35:04.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.551+0000 7f566bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:04.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.551+0000 7f566bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.711+0000 7f568a460640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-09-15T01:35:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.711+0000 7f568a460640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-09-15T01:35:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.711+0000 7f568a460640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:04.711+0000 7f568a460640 1 -- 172.21.15.204:0/1747583018 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]} v 0) v1 -- 0x7f5684070b80 con 0x7f56840a2cd0 2024-09-15T01:35:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.707+0000 7f566bfff640 1 -- 172.21.15.204:0/1747583018 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}]=0 add item id 11 name 'osd.11' weight 123 at location {root=default} to crush map v102) v1 ==== 193+0+0 (secure 0 0 0) 0x7f567c00d900 con 0x7f56840a2cd0 2024-09-15T01:35:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.708+0000 7f566bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-09-15T01:35:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.708+0000 7f566bfff640 10 monclient: _finish_command 2 = system:0 add item id 11 name 'osd.11' weight 123 at location {root=default} to crush map 2024-09-15T01:35:05.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.806+0000 7f568a460640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-09-15T01:35:05.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.806+0000 7f568a460640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-09-15T01:35:05.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.806+0000 7f568a460640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:05.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.806+0000 7f568a460640 1 -- 172.21.15.204:0/1747583018 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]} v 0) v1 -- 0x7f5684121200 con 0x7f56840a2cd0 2024-09-15T01:35:05.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.807+0000 7f566bfff640 1 -- 172.21.15.204:0/1747583018 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}]=0 set item id 11 name 'osd.11' weight 123 at location {root=default}: no change v102) v1 ==== 191+0+0 (secure 0 0 0) 0x7f567c01c310 con 0x7f56840a2cd0 2024-09-15T01:35:05.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.807+0000 7f566bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-09-15T01:35:05.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.807+0000 7f566bfff640 10 monclient: _finish_command 3 = system:0 set item id 11 name 'osd.11' weight 123 at location {root=default}: no change 2024-09-15T01:35:05.810 INFO:tasks.workunit.client.0.smithi204.stderr:set item id 11 name 'osd.11' weight 123 at location {root=default}: no change 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 1 -- 172.21.15.204:0/1747583018 >> v2:172.21.15.204:6812/28210 conn(0x7f56700397b0 msgr2=0x7f567003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 1 --2- 172.21.15.204:0/1747583018 >> v2:172.21.15.204:6812/28210 conn(0x7f56700397b0 0x7f567003bc60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5674034000 tx=0x7f5674032000 comp rx=0 tx=0).stop 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 10 monclient: shutdown 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 1 -- 172.21.15.204:0/1747583018 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 msgr2=0x7f568411ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 1 --2- 172.21.15.204:0/1747583018 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 0x7f568411ddd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f567c00a380 tx=0x7f567c0078f0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 1 -- 172.21.15.204:0/1747583018 shutdown_connections 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 1 --2- 172.21.15.204:0/1747583018 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56840a2cd0 0x7f568411ddd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 1 --2- 172.21.15.204:0/1747583018 >> v2:172.21.15.204:6812/28210 conn(0x7f56700397b0 0x7f567003bc60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 1 --2- 172.21.15.204:0/1747583018 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f56840a08e0 0x7f568411d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 1 --2- 172.21.15.204:0/1747583018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f568409fa20 0x7f5684130d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 1 -- 172.21.15.204:0/1747583018 >> 172.21.15.204:0/1747583018 conn(0x7f568406e860 msgr2=0x7f56840a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:05.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 1 -- 172.21.15.204:0/1747583018 shutdown_connections 2024-09-15T01:35:05.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.809+0000 7f568a460640 1 -- 172.21.15.204:0/1747583018 wait complete. 2024-09-15T01:35:05.821 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:35:05.821 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep osd.11 2024-09-15T01:35:05.821 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 123 2024-09-15T01:35:05.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.889+0000 7f89b255d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:05.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.889+0000 7f89b255d640 10 monclient: build_initial_monmap 2024-09-15T01:35:05.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.890+0000 7f89b255d640 10 monclient: monmap: 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:05.891484+0000 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:05.891484+0000 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:05.893 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:05.894 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:05.894 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:05.894 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:05.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.890+0000 7f89b255d640 1 Processor -- start 2024-09-15T01:35:05.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.890+0000 7f89b255d640 1 -- start start 2024-09-15T01:35:05.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.890+0000 7f89b255d640 10 monclient: init 2024-09-15T01:35:05.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 10 monclient: _add_conns ranks=[5,4,9,10,3,2,7,1,0,6,8] 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f89ac0a2400 0x7f89ac0a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 10 monclient(hunting): picked mon.noname-h con 0x7f89ac0a2400 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f89ac09f990 0x7f89ac09fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 10 monclient(hunting): picked mon.noname-c con 0x7f89ac09f990 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f89ac132930 0x7f89ac132d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 10 monclient(hunting): picked mon.noname-a con 0x7f89ac132930 addr v2:172.21.15.178:3300/0 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ac09e1a0 con 0x7f89ac132930 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac0a27d0 con 0x7f89ac09f990 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac09dec0 con 0x7f89ac0a2400 2024-09-15T01:35:05.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b255d640 10 monclient(hunting): authenticate will time out at 921.534473s 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f89ac0a2400 0x7f89ac0a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f89ac0a2400 0x7f89ac0a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:42042/0 (socket says 172.21.15.204:42042) 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 1 -- 172.21.15.204:0/1815688219 learned_addr learned my addr 172.21.15.204:0/1815688219 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 10 monclient(hunting): get_auth_request con 0x7f89ac0a2400 auth_method 0 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b0afa640 1 --2- 172.21.15.204:0/1815688219 >> v2:172.21.15.178:3300/0 conn(0x7f89ac132930 0x7f89ac132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:05.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89ab7fe640 1 --2- 172.21.15.204:0/1815688219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f89ac09f990 0x7f89ac09fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b0afa640 10 monclient(hunting): get_auth_request con 0x7f89ac132930 auth_method 0 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b0afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b0afa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89b0afa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89ab7fe640 10 monclient(hunting): get_auth_request con 0x7f89ac09f990 auth_method 0 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 10 monclient(hunting): handle_auth_done global_id 5522 payload 274 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 1 -- 172.21.15.204:0/1815688219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f89ac09f990 msgr2=0x7f89ac09fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 1 --2- 172.21.15.204:0/1815688219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f89ac09f990 0x7f89ac09fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 1 -- 172.21.15.204:0/1815688219 >> v2:172.21.15.178:3300/0 conn(0x7f89ac132930 msgr2=0x7f89ac132d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:05.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 1 --2- 172.21.15.204:0/1815688219 >> v2:172.21.15.178:3300/0 conn(0x7f89ac132930 0x7f89ac132d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 1 monclient: found mon.noname-h 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.891+0000 7f89abfff640 1 -- 172.21.15.204:0/1815688219 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac0648a0 con 0x7f89ac0a2400 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89abfff640 1 --2- 172.21.15.204:0/1815688219 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f89ac0a2400 0x7f89ac0a0ae0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f899c009910 tx=0x7f899c0091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a3c57d0e66170539 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89aaffd640 1 -- 172.21.15.204:0/1815688219 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f899c02ac80 con 0x7f89ac0a2400 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89aaffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89aaffd640 10 monclient: dump: 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:05.905 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89aaffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:05.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89aaffd640 1 -- 172.21.15.204:0/1815688219 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c02ae00 con 0x7f89ac0a2400 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89b255d640 5 monclient: authenticate success, global_id 5522 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89b255d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89aaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89b255d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89aaffd640 1 -- 172.21.15.204:0/1815688219 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f899c031ca0 con 0x7f89ac0a2400 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89b255d640 4 set_mon_vals no callback set 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89aaffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.892+0000 7f89aaffd640 10 monclient: dump: 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:05.907 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 10 monclient: shutdown 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 1 -- 172.21.15.204:0/1815688219 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f89ac0a2400 msgr2=0x7f89ac0a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 1 --2- 172.21.15.204:0/1815688219 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f89ac0a2400 0x7f89ac0a0ae0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f899c009910 tx=0x7f899c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 1 -- 172.21.15.204:0/1815688219 shutdown_connections 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 1 --2- 172.21.15.204:0/1815688219 >> v2:172.21.15.178:3300/0 conn(0x7f89ac132930 0x7f89ac132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 1 --2- 172.21.15.204:0/1815688219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f89ac09f990 0x7f89ac09fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 1 --2- 172.21.15.204:0/1815688219 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f89ac0a2400 0x7f89ac0a0ae0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 1 -- 172.21.15.204:0/1815688219 >> 172.21.15.204:0/1815688219 conn(0x7f89ac06e860 msgr2=0x7f89ac06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 1 -- 172.21.15.204:0/1815688219 shutdown_connections 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 1 -- 172.21.15.204:0/1815688219 wait complete. 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 10 monclient: build_initial_monmap 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.893+0000 7f89b255d640 10 monclient: monmap: 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:05.909 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:05.910 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:05.910 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:05.910 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:05.910 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:05.910 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:05.910 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:05.910 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:05.910 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:05.910 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:05.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 1 Processor -- start 2024-09-15T01:35:05.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 1 -- start start 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 10 monclient: init 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 10 monclient: _add_conns ranks=[6,1,2,0,7,4,8,5,3] 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89ac09f990 0x7f89ac11c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 10 monclient(hunting): picked mon.noname-d con 0x7f89ac09f990 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89ac0a2400 0x7f89ac11ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 10 monclient(hunting): picked mon.noname-f con 0x7f89ac0a2400 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89ac132930 0x7f89ac11cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 10 monclient(hunting): picked mon.noname-b con 0x7f89ac132930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac11f660 con 0x7f89ac0a2400 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac09d3d0 con 0x7f89ac132930 2024-09-15T01:35:05.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac09cab0 con 0x7f89ac09f990 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b255d640 10 monclient(hunting): authenticate will time out at 921.537772s 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b0afa640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89ac132930 msgr2=0x7f89ac11cfd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89ab7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89ac0a2400 0x7f89ac11ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89abfff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89ac09f990 msgr2=0x7f89ac11c550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89b0afa640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89ac132930 0x7f89ac11cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89abfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89ac09f990 0x7f89ac11c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89ab7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89ac0a2400 0x7f89ac11ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:57250/0 (socket says 172.21.15.204:57250) 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89ab7fe640 1 -- 172.21.15.204:0/4102827768 learned_addr learned my addr 172.21.15.204:0/4102827768 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89ab7fe640 10 monclient(hunting): get_auth_request con 0x7f89ac0a2400 auth_method 0 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.894+0000 7f89ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89ab7fe640 10 monclient(hunting): handle_auth_done global_id 5950 payload 824 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89ab7fe640 1 -- 172.21.15.204:0/4102827768 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89ac09f990 msgr2=0x7f89ac11c550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89ab7fe640 1 --2- 172.21.15.204:0/4102827768 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89ac09f990 0x7f89ac11c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89ab7fe640 1 -- 172.21.15.204:0/4102827768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89ac132930 msgr2=0x7f89ac11cfd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89ab7fe640 1 --2- 172.21.15.204:0/4102827768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89ac132930 0x7f89ac11cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89ab7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89ab7fe640 1 monclient: found mon.noname-f 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89ab7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89ab7fe640 1 -- 172.21.15.204:0/4102827768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c009450 con 0x7f89ac0a2400 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89ab7fe640 1 --2- 172.21.15.204:0/4102827768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89ac0a2400 0x7f89ac11ca90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89a000d890 tx=0x7f89a0008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f8993fff640 1 -- 172.21.15.204:0/4102827768 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89a000de20 con 0x7f89ac0a2400 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f8993fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f8993fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f8993fff640 10 monclient: dump: 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:05.914 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f8993fff640 10 monclient: _finish_auth 0 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f8993fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89b255d640 5 monclient: authenticate success, global_id 5950 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f8993fff640 1 -- 172.21.15.204:0/4102827768 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a000bd50 con 0x7f89ac0a2400 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89b255d640 10 monclient: _renew_subs 2024-09-15T01:35:05.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89b255d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89b255d640 1 -- 172.21.15.204:0/4102827768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ac11f380 con 0x7f89ac0a2400 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f8993fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f8993fff640 1 -- 172.21.15.204:0/4102827768 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89a00148b0 con 0x7f89ac0a2400 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f8993fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f8993fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f8993fff640 10 monclient: dump: 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:05.916 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89b255d640 10 monclient: _renew_subs 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89aa7fc640 4 set_mon_vals no callback set 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89b255d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:05.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.895+0000 7f89b255d640 1 -- 172.21.15.204:0/4102827768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ac11e490 con 0x7f89ac0a2400 2024-09-15T01:35:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.896+0000 7f8993fff640 1 -- 172.21.15.204:0/4102827768 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f89a0014430 con 0x7f89ac0a2400 2024-09-15T01:35:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.896+0000 7f8993fff640 1 --2- 172.21.15.204:0/4102827768 >> v2:172.21.15.204:6812/28210 conn(0x7f89980397b0 0x7f899803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.896+0000 7f8993fff640 1 -- 172.21.15.204:0/4102827768 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5138+0+0 (secure 0 0 0) 0x7f89a004bb40 con 0x7f89ac0a2400 2024-09-15T01:35:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.896+0000 7f89b255d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.896+0000 7f89b255d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.896+0000 7f89b255d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.896+0000 7f89abfff640 1 --2- 172.21.15.204:0/4102827768 >> v2:172.21.15.204:6812/28210 conn(0x7f89980397b0 0x7f899803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.896+0000 7f89b255d640 1 -- 172.21.15.204:0/4102827768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ac070b80 con 0x7f89ac0a2400 2024-09-15T01:35:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.896+0000 7f89abfff640 10 monclient: get_auth_request con 0x7f89980397b0 auth_method 0 2024-09-15T01:35:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.896+0000 7f89abfff640 1 --2- 172.21.15.204:0/4102827768 >> v2:172.21.15.204:6812/28210 conn(0x7f89980397b0 0x7f899803bc60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f899c034000 tx=0x7f899c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:05.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.901+0000 7f8993fff640 1 -- 172.21.15.204:0/4102827768 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f89a0014db0 con 0x7f89ac0a2400 2024-09-15T01:35:05.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.901+0000 7f8993fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:05.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:05.901+0000 7f8993fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:06.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.056+0000 7f89b255d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:35:06.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.056+0000 7f89b255d640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:35:06.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.056+0000 7f89b255d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:06.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.056+0000 7f89b255d640 1 -- 172.21.15.204:0/4102827768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f89ac09e480 con 0x7f89ac0a2400 2024-09-15T01:35:06.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.056+0000 7f8993fff640 1 -- 172.21.15.204:0/4102827768 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v102) v1 ==== 56+0+1173 (secure 0 0 0) 0x7f89a002c070 con 0x7f89ac0a2400 2024-09-15T01:35:06.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.056+0000 7f8993fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:35:06.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.056+0000 7f8993fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:06.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.150+0000 7f89b255d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:35:06.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.150+0000 7f89b255d640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:35:06.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.150+0000 7f89b255d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:06.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.150+0000 7f89b255d640 1 -- 172.21.15.204:0/4102827768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f89ac1326c0 con 0x7f89ac0a2400 2024-09-15T01:35:06.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.151+0000 7f8993fff640 1 -- 172.21.15.204:0/4102827768 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v102) v1 ==== 56+0+1173 (secure 0 0 0) 0x7f89a0011340 con 0x7f89ac0a2400 2024-09-15T01:35:06.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.151+0000 7f8993fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:35:06.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.151+0000 7f8993fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:06.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.152+0000 7f89b255d640 1 -- 172.21.15.204:0/4102827768 >> v2:172.21.15.204:6812/28210 conn(0x7f89980397b0 msgr2=0x7f899803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:06.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.152+0000 7f89b255d640 1 --2- 172.21.15.204:0/4102827768 >> v2:172.21.15.204:6812/28210 conn(0x7f89980397b0 0x7f899803bc60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f899c034000 tx=0x7f899c032000 comp rx=0 tx=0).stop 2024-09-15T01:35:06.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.153+0000 7f89b255d640 10 monclient: shutdown 2024-09-15T01:35:06.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.153+0000 7f89b255d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:06.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.153+0000 7f89b255d640 1 -- 172.21.15.204:0/4102827768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89ac0a2400 msgr2=0x7f89ac11ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:06.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.153+0000 7f89b255d640 1 --2- 172.21.15.204:0/4102827768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89ac0a2400 0x7f89ac11ca90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89a000d890 tx=0x7f89a0008040 comp rx=0 tx=0).stop 2024-09-15T01:35:06.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.153+0000 7f89b255d640 1 -- 172.21.15.204:0/4102827768 shutdown_connections 2024-09-15T01:35:06.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.153+0000 7f89b255d640 1 --2- 172.21.15.204:0/4102827768 >> v2:172.21.15.204:6812/28210 conn(0x7f89980397b0 0x7f899803bc60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:06.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.153+0000 7f89b255d640 1 --2- 172.21.15.204:0/4102827768 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89ac132930 0x7f89ac11cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:06.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.153+0000 7f89b255d640 1 --2- 172.21.15.204:0/4102827768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89ac0a2400 0x7f89ac11ca90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:06.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.153+0000 7f89b255d640 1 --2- 172.21.15.204:0/4102827768 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89ac09f990 0x7f89ac11c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:06.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.153+0000 7f89b255d640 1 -- 172.21.15.204:0/4102827768 >> 172.21.15.204:0/4102827768 conn(0x7f89ac06e860 msgr2=0x7f89ac130020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:06.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.153+0000 7f89b255d640 1 -- 172.21.15.204:0/4102827768 shutdown_connections 2024-09-15T01:35:06.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.153+0000 7f89b255d640 1 -- 172.21.15.204:0/4102827768 wait complete. 2024-09-15T01:35:06.164 INFO:tasks.workunit.client.0.smithi204.stdout: 11 123.00000 osd.11 down 1.00000 1.00000 2024-09-15T01:35:06.164 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush reweight osd.11 113 2024-09-15T01:35:06.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.237+0000 7efcee91d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:06.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.237+0000 7efcee91d640 10 monclient: build_initial_monmap 2024-09-15T01:35:06.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.237+0000 7efcee91d640 10 monclient: monmap: 2024-09-15T01:35:06.240 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:06.240 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:06.240 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:06.238924+0000 2024-09-15T01:35:06.240 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:06.238924+0000 2024-09-15T01:35:06.240 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:06.240 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:06.240 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:06.240 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:06.240 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:06.241 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:06.241 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:06.241 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:06.241 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:06.241 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:06.241 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:06.241 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:06.241 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:06.241 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:06.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 1 Processor -- start 2024-09-15T01:35:06.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 1 -- start start 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 10 monclient: init 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 10 monclient: _add_conns ranks=[1,10,3,7,8,2,9,0,4,5,6] 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7efce80a2e30 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efce809fb80 0x7efce809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 10 monclient(hunting): picked mon.noname-f con 0x7efce809fb80 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efce8137d20 0x7efce813a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 10 monclient(hunting): picked mon.noname-g con 0x7efce8137d20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce804b720 con 0x7efce80a2e30 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce809e6c0 con 0x7efce809fb80 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:06.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce80a04b0 con 0x7efce8137d20 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efcee91d640 10 monclient(hunting): authenticate will time out at 921.881760s 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efce77fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efce809fb80 0x7efce809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efceceba640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efce8137d20 0x7efce813a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efce77fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efce809fb80 0x7efce809ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:57254/0 (socket says 172.21.15.204:57254) 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efce77fe640 1 -- 172.21.15.204:0/632420902 learned_addr learned my addr 172.21.15.204:0/632420902 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efceceba640 10 monclient(hunting): get_auth_request con 0x7efce8137d20 auth_method 0 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efceceba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efceceba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efceceba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efce7fff640 1 --1- 172.21.15.204:0/632420902 >> v1:172.21.15.204:6789/0 conn(0x7efce80a2e30 0x7efce8132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:50432/0 (socket says 172.21.15.204:50432) 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efce77fe640 10 monclient(hunting): get_auth_request con 0x7efce809fb80 auth_method 0 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efce77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efce77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:06.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efce77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efceceba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efceceba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.238+0000 7efceceba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 10 monclient(hunting): handle_auth_done global_id 5959 payload 274 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 1 -- 172.21.15.204:0/632420902 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efce8137d20 msgr2=0x7efce813a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 1 --2- 172.21.15.204:0/632420902 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efce8137d20 0x7efce813a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 1 -- 172.21.15.204:0/632420902 >> v1:172.21.15.204:6789/0 conn(0x7efce80a2e30 legacy=0x7efce8132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 1 monclient: found mon.noname-f 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:06.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 1 -- 172.21.15.204:0/632420902 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce8133170 con 0x7efce809fb80 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efceceba640 1 --2- 172.21.15.204:0/632420902 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efce8137d20 0x7efce813a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.239+0000 7efce77fe640 1 --2- 172.21.15.204:0/632420902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efce809fb80 0x7efce809ff70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009910 tx=0x7efcdc0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cad117521f63143b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efce6ffd640 1 -- 172.21.15.204:0/632420902 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcdc02ad20 con 0x7efce809fb80 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efce6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efce6ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efce6ffd640 10 monclient: dump: 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:06.252 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efce6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efce6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efce6ffd640 1 -- 172.21.15.204:0/632420902 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcdc031820 con 0x7efce809fb80 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efce6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efce6ffd640 1 -- 172.21.15.204:0/632420902 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcdc02aea0 con 0x7efce809fb80 2024-09-15T01:35:06.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efce6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efce6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efce6ffd640 10 monclient: dump: 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:06.254 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 5 monclient: authenticate success, global_id 5959 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 4 set_mon_vals no callback set 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 10 monclient: shutdown 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 1 -- 172.21.15.204:0/632420902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efce809fb80 msgr2=0x7efce809ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 1 --2- 172.21.15.204:0/632420902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efce809fb80 0x7efce809ff70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009910 tx=0x7efcdc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 1 -- 172.21.15.204:0/632420902 shutdown_connections 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 1 --2- 172.21.15.204:0/632420902 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efce8137d20 0x7efce813a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 1 --2- 172.21.15.204:0/632420902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efce809fb80 0x7efce809ff70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:06.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 1 -- 172.21.15.204:0/632420902 >> 172.21.15.204:0/632420902 conn(0x7efce806e860 msgr2=0x7efce806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 1 -- 172.21.15.204:0/632420902 shutdown_connections 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.240+0000 7efcee91d640 1 -- 172.21.15.204:0/632420902 wait complete. 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.241+0000 7efcee91d640 10 monclient: build_initial_monmap 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.241+0000 7efcee91d640 10 monclient: monmap: 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:06.256 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.241+0000 7efcee91d640 1 Processor -- start 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.241+0000 7efcee91d640 1 -- start start 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.241+0000 7efcee91d640 10 monclient: init 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient: _add_conns ranks=[5,6,2,3,1,8,7,0,4] 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efce809fb80 0x7efce80991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient(hunting): picked mon.noname-h con 0x7efce809fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:06.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efce80a2e30 0x7efce8099730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient(hunting): picked mon.noname-d con 0x7efce80a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efce8137d20 0x7efce8099c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient(hunting): picked mon.noname-b con 0x7efce8137d20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce809b820 con 0x7efce8137d20 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efce809b1d0 con 0x7efce809fb80 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efce809ca40 con 0x7efce80a2e30 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient(hunting): authenticate will time out at 921.885114s 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efce809fb80 0x7efce80991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce77fe640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efce80a2e30 msgr2=0x7efce8099730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:06.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efce809fb80 0x7efce80991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:42052/0 (socket says 172.21.15.204:42052) 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 1 -- 172.21.15.204:0/3474919823 learned_addr learned my addr 172.21.15.204:0/3474919823 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efceceba640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efce8137d20 msgr2=0x7efce8099c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 10 monclient(hunting): get_auth_request con 0x7efce809fb80 auth_method 0 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce77fe640 1 --2- 172.21.15.204:0/3474919823 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efce80a2e30 0x7efce8099730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efceceba640 1 --2- 172.21.15.204:0/3474919823 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efce8137d20 0x7efce8099c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 10 monclient(hunting): handle_auth_done global_id 5531 payload 824 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 1 -- 172.21.15.204:0/3474919823 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efce80a2e30 msgr2=0x7efce8099730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:06.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 1 --2- 172.21.15.204:0/3474919823 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efce80a2e30 0x7efce8099730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 1 -- 172.21.15.204:0/3474919823 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efce8137d20 msgr2=0x7efce8099c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 1 --2- 172.21.15.204:0/3474919823 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efce8137d20 0x7efce8099c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 1 monclient: found mon.noname-h 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 1 -- 172.21.15.204:0/3474919823 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce0009a20 con 0x7efce809fb80 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce7fff640 1 --2- 172.21.15.204:0/3474919823 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efce809fb80 0x7efce80991f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efcd800c5c0 tx=0x7efcd8007ea0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efccffff640 1 -- 172.21.15.204:0/3474919823 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcd8001120 con 0x7efce809fb80 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efccffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efccffff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efccffff640 10 monclient: dump: 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:06.260 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:06.261 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:06.261 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:06.261 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:06.261 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:06.261 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:06.261 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:06.261 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:06.261 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:06.261 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:06.261 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:06.261 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efccffff640 10 monclient: _finish_auth 0 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efccffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 5 monclient: authenticate success, global_id 5531 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efccffff640 1 -- 172.21.15.204:0/3474919823 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd80047a0 con 0x7efce809fb80 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient: _renew_subs 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 1 -- 172.21.15.204:0/3474919823 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcdc009450 con 0x7efce809fb80 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efccffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efccffff640 1 -- 172.21.15.204:0/3474919823 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcd8004d90 con 0x7efce809fb80 2024-09-15T01:35:06.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efccffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efccffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efccffff640 10 monclient: dump: 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:06.263 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient: _renew_subs 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efce67fc640 4 set_mon_vals no callback set 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.242+0000 7efcee91d640 1 -- 172.21.15.204:0/3474919823 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce809c780 con 0x7efce809fb80 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.243+0000 7efccffff640 1 -- 172.21.15.204:0/3474919823 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7efcd8004910 con 0x7efce809fb80 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.243+0000 7efccffff640 1 --2- 172.21.15.204:0/3474919823 >> v2:172.21.15.204:6812/28210 conn(0x7efcd40397b0 0x7efcd403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.243+0000 7efccffff640 1 -- 172.21.15.204:0/3474919823 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5138+0+0 (secure 0 0 0) 0x7efcd804a970 con 0x7efce809fb80 2024-09-15T01:35:06.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.243+0000 7efce77fe640 1 --2- 172.21.15.204:0/3474919823 >> v2:172.21.15.204:6812/28210 conn(0x7efcd40397b0 0x7efcd403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:06.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.243+0000 7efce77fe640 10 monclient: get_auth_request con 0x7efcd40397b0 auth_method 0 2024-09-15T01:35:06.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.244+0000 7efcee91d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:06.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.244+0000 7efcee91d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:06.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.244+0000 7efcee91d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:06.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.244+0000 7efcee91d640 1 -- 172.21.15.204:0/3474919823 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce811c8e0 con 0x7efce809fb80 2024-09-15T01:35:06.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.244+0000 7efce77fe640 1 --2- 172.21.15.204:0/3474919823 >> v2:172.21.15.204:6812/28210 conn(0x7efcd40397b0 0x7efcd403bc60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efcdc034000 tx=0x7efcdc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:06.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.248+0000 7efccffff640 1 -- 172.21.15.204:0/3474919823 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7efcd800e470 con 0x7efce809fb80 2024-09-15T01:35:06.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.248+0000 7efccffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:06.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.248+0000 7efccffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:06.411 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.408+0000 7efcee91d640 10 start_mon_command cmd=[{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-09-15T01:35:06.411 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.408+0000 7efcee91d640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-09-15T01:35:06.411 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.408+0000 7efcee91d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:06.411 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.408+0000 7efcee91d640 1 -- 172.21.15.204:0/3474919823 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0} v 0) v1 -- 0x7efce8070cc0 con 0x7efce809fb80 2024-09-15T01:35:06.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.445+0000 7efccffff640 1 -- 172.21.15.204:0/3474919823 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}]=0 reweighted item id 11 name 'osd.11' to 113 in crush map v103) v1 ==== 156+0+0 (secure 0 0 0) 0x7efcd800e290 con 0x7efce809fb80 2024-09-15T01:35:06.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.445+0000 7efccffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-09-15T01:35:06.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.445+0000 7efccffff640 10 monclient: _finish_command 2 = system:0 reweighted item id 11 name 'osd.11' to 113 in crush map 2024-09-15T01:35:06.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.543+0000 7efcee91d640 10 start_mon_command cmd=[{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-09-15T01:35:06.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.543+0000 7efcee91d640 10 monclient: _send_command 3 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-09-15T01:35:06.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.543+0000 7efcee91d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:06.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:06.543+0000 7efcee91d640 1 -- 172.21.15.204:0/3474919823 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0} v 0) v1 -- 0x7efce80008d0 con 0x7efce809fb80 2024-09-15T01:35:07.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.447+0000 7efccffff640 1 -- 172.21.15.204:0/3474919823 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}]=0 reweighted item id 11 name 'osd.11' to 113 in crush map v104) v1 ==== 156+0+0 (secure 0 0 0) 0x7efcd800e630 con 0x7efce809fb80 2024-09-15T01:35:07.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.447+0000 7efccffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-09-15T01:35:07.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.447+0000 7efccffff640 10 monclient: _finish_command 3 = system:0 reweighted item id 11 name 'osd.11' to 113 in crush map 2024-09-15T01:35:07.451 INFO:tasks.workunit.client.0.smithi204.stderr:reweighted item id 11 name 'osd.11' to 113 in crush map 2024-09-15T01:35:07.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 1 -- 172.21.15.204:0/3474919823 >> v2:172.21.15.204:6812/28210 conn(0x7efcd40397b0 msgr2=0x7efcd403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:07.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 1 --2- 172.21.15.204:0/3474919823 >> v2:172.21.15.204:6812/28210 conn(0x7efcd40397b0 0x7efcd403bc60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efcdc034000 tx=0x7efcdc032000 comp rx=0 tx=0).stop 2024-09-15T01:35:07.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 10 monclient: shutdown 2024-09-15T01:35:07.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:07.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 1 -- 172.21.15.204:0/3474919823 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efce809fb80 msgr2=0x7efce80991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:07.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 1 --2- 172.21.15.204:0/3474919823 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efce809fb80 0x7efce80991f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efcd800c5c0 tx=0x7efcd8007ea0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 1 -- 172.21.15.204:0/3474919823 shutdown_connections 2024-09-15T01:35:07.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 1 --2- 172.21.15.204:0/3474919823 >> v2:172.21.15.204:6812/28210 conn(0x7efcd40397b0 0x7efcd403bc60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 1 --2- 172.21.15.204:0/3474919823 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efce8137d20 0x7efce8099c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.453 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 1 --2- 172.21.15.204:0/3474919823 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efce80a2e30 0x7efce8099730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 1 --2- 172.21.15.204:0/3474919823 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efce809fb80 0x7efce80991f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 1 -- 172.21.15.204:0/3474919823 >> 172.21.15.204:0/3474919823 conn(0x7efce806e860 msgr2=0x7efce80a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:07.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 1 -- 172.21.15.204:0/3474919823 shutdown_connections 2024-09-15T01:35:07.454 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.450+0000 7efcee91d640 1 -- 172.21.15.204:0/3474919823 wait complete. 2024-09-15T01:35:07.464 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush reweight osd.11 123456 2024-09-15T01:35:07.464 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:35:07.464 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush reweight osd.11 123456 2024-09-15T01:35:07.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.542+0000 7fa288b1d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:07.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.542+0000 7fa288b1d640 10 monclient: build_initial_monmap 2024-09-15T01:35:07.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.542+0000 7fa288b1d640 10 monclient: monmap: 2024-09-15T01:35:07.545 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:07.545 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:07.545 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:07.543650+0000 2024-09-15T01:35:07.545 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:07.543650+0000 2024-09-15T01:35:07.545 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:07.545 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:07.545 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.542+0000 7fa288b1d640 1 Processor -- start 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.542+0000 7fa288b1d640 1 -- start start 2024-09-15T01:35:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.542+0000 7fa288b1d640 10 monclient: init 2024-09-15T01:35:07.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:07.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 10 monclient: _add_conns ranks=[8,3,4,9,6,0,5,2,1,10,7] 2024-09-15T01:35:07.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa2840a4720 0x7fa2841308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 10 monclient(hunting): picked mon.noname-e con 0x7fa2840a4720 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:07.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa284130de0 0x7fa2841311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 10 monclient(hunting): picked mon.noname-g con 0x7fa284130de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:07.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa28409e850 0x7fa28409ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 10 monclient(hunting): picked mon.noname-c con 0x7fa28409e850 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:07.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:07.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa284131730 con 0x7fa284130de0 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840a4af0 con 0x7fa28409e850 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840a0630 con 0x7fa2840a4720 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa288b1d640 10 monclient(hunting): authenticate will time out at 923.186496s 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa284130de0 0x7fa2841311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa284130de0 0x7fa2841311f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:42680/0 (socket says 172.21.15.204:42680) 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 1 -- 172.21.15.204:0/2927910926 learned_addr learned my addr 172.21.15.204:0/2927910926 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 10 monclient(hunting): get_auth_request con 0x7fa284130de0 auth_method 0 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:07.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa282d9d640 1 --2- 172.21.15.204:0/2927910926 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa28409e850 0x7fa28409ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa28259c640 1 --2- 172.21.15.204:0/2927910926 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa2840a4720 0x7fa2841308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 10 monclient(hunting): handle_auth_done global_id 14592 payload 274 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 1 -- 172.21.15.204:0/2927910926 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa2840a4720 msgr2=0x7fa2841308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 1 --2- 172.21.15.204:0/2927910926 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa2840a4720 0x7fa2841308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 1 -- 172.21.15.204:0/2927910926 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa28409e850 msgr2=0x7fa28409ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 1 --2- 172.21.15.204:0/2927910926 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa28409e850 0x7fa28409ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 1 monclient: found mon.noname-g 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:07.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa281d9b640 1 -- 172.21.15.204:0/2927910926 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28404b720 con 0x7fa284130de0 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa282d9d640 10 monclient: get_auth_request con 0x7fa28409e850 auth_method 0 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa28259c640 10 monclient: get_auth_request con 0x7fa2840a4720 auth_method 0 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa282d9d640 1 --2- 172.21.15.204:0/2927910926 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa28409e850 0x7fa28409ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.543+0000 7fa28259c640 1 --2- 172.21.15.204:0/2927910926 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa2840a4720 0x7fa2841308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa281d9b640 1 --2- 172.21.15.204:0/2927910926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa284130de0 0x7fa2841311f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009940 tx=0x7fa26c0091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=50a989b31279c014 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa28159a640 1 -- 172.21.15.204:0/2927910926 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa26c02ac60 con 0x7fa284130de0 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa28159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa28159a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa28159a640 10 monclient: dump: 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:07.557 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:07.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa28159a640 10 monclient: _finish_auth 0 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa28159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa288b1d640 5 monclient: authenticate success, global_id 14592 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa288b1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa28159a640 1 -- 172.21.15.204:0/2927910926 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c02ade0 con 0x7fa284130de0 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa28159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa288b1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa288b1d640 4 set_mon_vals no callback set 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa28159a640 1 -- 172.21.15.204:0/2927910926 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa26c031ca0 con 0x7fa284130de0 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa28159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa28159a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.544+0000 7fa28159a640 10 monclient: dump: 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:07.559 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:07.560 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.560 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.560 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:07.560 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:07.560 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:07.560 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:07.560 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:07.560 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:07.560 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:07.560 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:07.560 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 10 monclient: shutdown 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 1 -- 172.21.15.204:0/2927910926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa284130de0 msgr2=0x7fa2841311f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 1 --2- 172.21.15.204:0/2927910926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa284130de0 0x7fa2841311f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009940 tx=0x7fa26c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 1 -- 172.21.15.204:0/2927910926 shutdown_connections 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 1 --2- 172.21.15.204:0/2927910926 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa28409e850 0x7fa28409ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 1 --2- 172.21.15.204:0/2927910926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa284130de0 0x7fa2841311f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 1 --2- 172.21.15.204:0/2927910926 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa2840a4720 0x7fa2841308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 1 -- 172.21.15.204:0/2927910926 >> 172.21.15.204:0/2927910926 conn(0x7fa2840677e0 msgr2=0x7fa2840a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:07.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 1 -- 172.21.15.204:0/2927910926 shutdown_connections 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 1 -- 172.21.15.204:0/2927910926 wait complete. 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 10 monclient: build_initial_monmap 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.545+0000 7fa288b1d640 10 monclient: monmap: 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:07.562 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 1 Processor -- start 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 1 -- start start 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 10 monclient: init 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 10 monclient: _add_conns ranks=[1,5,6,7,8,4,2,0,3] 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa28409e850 0x7fa284141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 10 monclient(hunting): picked mon.noname-f con 0x7fa28409e850 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2840a4720 0x7fa284142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 10 monclient(hunting): picked mon.noname-h con 0x7fa2840a4720 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa284130de0 0x7fa284068240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 10 monclient(hunting): picked mon.noname-d con 0x7fa284130de0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa284069d60 con 0x7fa28409e850 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa28411e8d0 con 0x7fa2840a4720 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa28406ce70 con 0x7fa284130de0 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa288b1d640 10 monclient(hunting): authenticate will time out at 923.189758s 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa28259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa28409e850 0x7fa284141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa282d9d640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa284130de0 msgr2=0x7fa284068240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:07.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa281d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2840a4720 0x7fa284142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa28259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa28409e850 0x7fa284141e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48434/0 (socket says 172.21.15.204:48434) 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa282d9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa284130de0 0x7fa284068240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa28259c640 1 -- 172.21.15.204:0/3384792439 learned_addr learned my addr 172.21.15.204:0/3384792439 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa281d9b640 10 monclient(hunting): get_auth_request con 0x7fa2840a4720 auth_method 0 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa281d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa281d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa281d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa28259c640 10 monclient(hunting): get_auth_request con 0x7fa28409e850 auth_method 0 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa28259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa28259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa28259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa281d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:07.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa281d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa281d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa28259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa28259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.546+0000 7fa28259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa281d9b640 10 monclient(hunting): handle_auth_done global_id 5540 payload 824 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa281d9b640 1 -- 172.21.15.204:0/3384792439 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa284130de0 msgr2=0x7fa284068240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa281d9b640 1 --2- 172.21.15.204:0/3384792439 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa284130de0 0x7fa284068240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa281d9b640 1 -- 172.21.15.204:0/3384792439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa28409e850 msgr2=0x7fa284141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa281d9b640 1 --2- 172.21.15.204:0/3384792439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa28409e850 0x7fa284141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa281d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa281d9b640 1 monclient: found mon.noname-h 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa281d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa281d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:07.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa281d9b640 1 -- 172.21.15.204:0/3384792439 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c009450 con 0x7fa2840a4720 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa28259c640 1 --2- 172.21.15.204:0/3384792439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa28409e850 0x7fa284141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa281d9b640 1 --2- 172.21.15.204:0/3384792439 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2840a4720 0x7fa284142350 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa26c031000 tx=0x7fa26c032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa27a7fc640 1 -- 172.21.15.204:0/3384792439 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa26c03b870 con 0x7fa2840a4720 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa27a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa27a7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa27a7fc640 10 monclient: dump: 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:07.567 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa27a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa27a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa27a7fc640 1 -- 172.21.15.204:0/3384792439 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c03b9f0 con 0x7fa2840a4720 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa288b1d640 5 monclient: authenticate success, global_id 5540 2024-09-15T01:35:07.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa288b1d640 10 monclient: _renew_subs 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa288b1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa288b1d640 1 -- 172.21.15.204:0/3384792439 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28406d220 con 0x7fa2840a4720 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa27a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa27a7fc640 1 -- 172.21.15.204:0/3384792439 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa26c0313a0 con 0x7fa2840a4720 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa27a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa27a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa27a7fc640 10 monclient: dump: 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:07.569 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa280d99640 4 set_mon_vals no callback set 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa288b1d640 10 monclient: _renew_subs 2024-09-15T01:35:07.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa288b1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:07.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.547+0000 7fa288b1d640 1 -- 172.21.15.204:0/3384792439 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa284142890 con 0x7fa2840a4720 2024-09-15T01:35:07.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.548+0000 7fa27a7fc640 1 -- 172.21.15.204:0/3384792439 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa26c039040 con 0x7fa2840a4720 2024-09-15T01:35:07.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.548+0000 7fa288b1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:07.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.548+0000 7fa288b1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:07.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.548+0000 7fa288b1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:07.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.548+0000 7fa288b1d640 1 -- 172.21.15.204:0/3384792439 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa284070cc0 con 0x7fa2840a4720 2024-09-15T01:35:07.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.548+0000 7fa27a7fc640 1 --2- 172.21.15.204:0/3384792439 >> v2:172.21.15.204:6812/28210 conn(0x7fa2640397b0 0x7fa26403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.548+0000 7fa27a7fc640 1 -- 172.21.15.204:0/3384792439 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5167+0+0 (secure 0 0 0) 0x7fa26c06acd0 con 0x7fa2840a4720 2024-09-15T01:35:07.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.548+0000 7fa28259c640 1 --2- 172.21.15.204:0/3384792439 >> v2:172.21.15.204:6812/28210 conn(0x7fa2640397b0 0x7fa26403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:07.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.548+0000 7fa28259c640 10 monclient: get_auth_request con 0x7fa2640397b0 auth_method 0 2024-09-15T01:35:07.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.549+0000 7fa28259c640 1 --2- 172.21.15.204:0/3384792439 >> v2:172.21.15.204:6812/28210 conn(0x7fa2640397b0 0x7fa26403bc60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00f360 tx=0x7fa27c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:07.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.553+0000 7fa27a7fc640 1 -- 172.21.15.204:0/3384792439 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa26c044ae0 con 0x7fa2840a4720 2024-09-15T01:35:07.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.553+0000 7fa27a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:07.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.553+0000 7fa27a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.713+0000 7fa288b1d640 10 start_mon_command cmd=[{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}] 2024-09-15T01:35:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.713+0000 7fa288b1d640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}] 2024-09-15T01:35:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.713+0000 7fa288b1d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:07.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.713+0000 7fa288b1d640 1 -- 172.21.15.204:0/3384792439 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0} v 0) v1 -- 0x7fa28406a8f0 con 0x7fa2840a4720 2024-09-15T01:35:07.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.714+0000 7fa27a7fc640 1 -- 172.21.15.204:0/3384792439 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}]=-75 (75) Value too large for defined data type v104) v1 ==== 146+0+0 (secure 0 0 0) 0x7fa26c0370a0 con 0x7fa2840a4720 2024-09-15T01:35:07.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.714+0000 7fa27a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}] 2024-09-15T01:35:07.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.714+0000 7fa27a7fc640 10 monclient: _finish_command 2 = mon:75 (75) Value too large for defined data type 2024-09-15T01:35:07.717 INFO:tasks.workunit.client.0.smithi204.stderr:Error EOVERFLOW: (75) Value too large for defined data type 2024-09-15T01:35:07.720 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:35:07.720 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:35:07.720 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep osd.11 2024-09-15T01:35:07.721 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 113 2024-09-15T01:35:07.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.790+0000 7f97fbf1d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:07.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.790+0000 7f97fbf1d640 10 monclient: build_initial_monmap 2024-09-15T01:35:07.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.790+0000 7f97fbf1d640 10 monclient: monmap: 2024-09-15T01:35:07.793 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:07.793 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:07.793 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:07.792104+0000 2024-09-15T01:35:07.793 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:07.792104+0000 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:07.794 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:07.795 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:07.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 1 Processor -- start 2024-09-15T01:35:07.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 1 -- start start 2024-09-15T01:35:07.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 10 monclient: init 2024-09-15T01:35:07.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:07.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 10 monclient: _add_conns ranks=[4,9,7,1,5,10,3,2,0,8,6] 2024-09-15T01:35:07.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f97f40a2400 0x7f97f40a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 10 monclient(hunting): picked mon.noname-c con 0x7f97f40a2400 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97f409f990 0x7f97f409fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 10 monclient(hunting): picked mon.noname-a con 0x7f97f409f990 addr v2:172.21.15.178:3300/0 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97f4132930 0x7f97f4132d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 10 monclient(hunting): picked mon.noname-i con 0x7f97f4132930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f409e1a0 con 0x7f97f409f990 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97f40a27d0 con 0x7f97f40a2400 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f97f409dec0 con 0x7f97f4132930 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:07.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fbf1d640 10 monclient(hunting): authenticate will time out at 923.434892s 2024-09-15T01:35:07.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fa4ba640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97f4132930 msgr2=0x7f97f4132d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:07.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.791+0000 7f97fa4ba640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97f4132930 0x7f97f4132d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:07.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f97f40a2400 0x7f97f40a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:07.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f94b8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97f409f990 0x7f97f409fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:07.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f94b8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97f409f990 0x7f97f409fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:34592/0 (socket says 172.21.15.204:34592) 2024-09-15T01:35:07.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f94b8640 1 -- 172.21.15.204:0/1034421797 learned_addr learned my addr 172.21.15.204:0/1034421797 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:07.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f94b8640 10 monclient(hunting): get_auth_request con 0x7f97f409f990 auth_method 0 2024-09-15T01:35:07.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f94b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:07.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f94b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:07.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f94b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:07.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 10 monclient(hunting): get_auth_request con 0x7f97f40a2400 auth_method 0 2024-09-15T01:35:07.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f94b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f94b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f94b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 10 monclient(hunting): handle_auth_done global_id 5440 payload 274 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 1 -- 172.21.15.204:0/1034421797 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97f4132930 msgr2=0x7f97f4132d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 1 --2- 172.21.15.204:0/1034421797 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97f4132930 0x7f97f4132d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 1 -- 172.21.15.204:0/1034421797 >> v2:172.21.15.178:3300/0 conn(0x7f97f409f990 msgr2=0x7f97f409fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 1 --2- 172.21.15.204:0/1034421797 >> v2:172.21.15.178:3300/0 conn(0x7f97f409f990 0x7f97f409fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 10 monclient: _finish_hunting 0 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 1 monclient: found mon.noname-c 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f9cb9640 1 -- 172.21.15.204:0/1034421797 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f40648a0 con 0x7f97f40a2400 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.792+0000 7f97f94b8640 1 --2- 172.21.15.204:0/1034421797 >> v2:172.21.15.178:3300/0 conn(0x7f97f409f990 0x7f97f409fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f9cb9640 1 --2- 172.21.15.204:0/1034421797 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f97f40a2400 0x7f97f40a0ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009850 tx=0x7f97f00090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d9b98dd7633bb69d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f8cb7640 1 -- 172.21.15.204:0/1034421797 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f97f002abe0 con 0x7f97f40a2400 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f8cb7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f8cb7640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f8cb7640 10 monclient: dump: 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.807 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f8cb7640 10 monclient: _finish_auth 0 2024-09-15T01:35:07.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f8cb7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 5 monclient: authenticate success, global_id 5440 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f8cb7640 1 -- 172.21.15.204:0/1034421797 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f002ad60 con 0x7f97f40a2400 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f8cb7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f8cb7640 1 -- 172.21.15.204:0/1034421797 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f97f0031ca0 con 0x7f97f40a2400 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 4 set_mon_vals no callback set 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f8cb7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f8cb7640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97f8cb7640 10 monclient: dump: 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:07.809 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:07.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 10 monclient: shutdown 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 1 -- 172.21.15.204:0/1034421797 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f97f40a2400 msgr2=0x7f97f40a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 1 --2- 172.21.15.204:0/1034421797 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f97f40a2400 0x7f97f40a0ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009850 tx=0x7f97f00090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 1 -- 172.21.15.204:0/1034421797 shutdown_connections 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 1 --2- 172.21.15.204:0/1034421797 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97f4132930 0x7f97f4132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 1 --2- 172.21.15.204:0/1034421797 >> v2:172.21.15.178:3300/0 conn(0x7f97f409f990 0x7f97f409fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 1 --2- 172.21.15.204:0/1034421797 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f97f40a2400 0x7f97f40a0ae0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.793+0000 7f97fbf1d640 1 -- 172.21.15.204:0/1034421797 >> 172.21.15.204:0/1034421797 conn(0x7f97f406e860 msgr2=0x7f97f406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.794+0000 7f97fbf1d640 1 -- 172.21.15.204:0/1034421797 shutdown_connections 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.794+0000 7f97fbf1d640 1 -- 172.21.15.204:0/1034421797 wait complete. 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.794+0000 7f97fbf1d640 10 monclient: build_initial_monmap 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.794+0000 7f97fbf1d640 10 monclient: monmap: 2024-09-15T01:35:07.811 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:07.812 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.794+0000 7f97fbf1d640 1 Processor -- start 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.794+0000 7f97fbf1d640 1 -- start start 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.794+0000 7f97fbf1d640 10 monclient: init 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 10 monclient: _add_conns ranks=[8,1,3,0,4,2,7,5,6] 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97f409f990 0x7f97f411c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 10 monclient(hunting): picked mon.noname-e con 0x7f97f409f990 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f40a2400 0x7f97f411ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 10 monclient(hunting): picked mon.noname-f con 0x7f97f40a2400 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97f4132930 0x7f97f411cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 10 monclient(hunting): picked mon.noname-g con 0x7f97f4132930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:07.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f409d3d0 con 0x7f97f40a2400 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f411e930 con 0x7f97f4132930 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f97f409c060 con 0x7f97f409f990 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fbf1d640 10 monclient(hunting): authenticate will time out at 923.438368s 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f94b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f40a2400 0x7f97f411ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97f4132930 0x7f97f411cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f94b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f40a2400 0x7f97f411ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48448/0 (socket says 172.21.15.204:48448) 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f94b8640 1 -- 172.21.15.204:0/3118457504 learned_addr learned my addr 172.21.15.204:0/3118457504 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f94b8640 10 monclient(hunting): get_auth_request con 0x7f97f40a2400 auth_method 0 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f94b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:07.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f94b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f94b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 10 monclient(hunting): get_auth_request con 0x7f97f4132930 auth_method 0 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f9cb9640 1 --2- 172.21.15.204:0/3118457504 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97f409f990 0x7f97f411c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f9cb9640 10 monclient(hunting): get_auth_request con 0x7f97f409f990 auth_method 0 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f9cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f9cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f9cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f94b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f94b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f94b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:07.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f9cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f9cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f9cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 10 monclient(hunting): handle_auth_done global_id 14601 payload 824 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 1 -- 172.21.15.204:0/3118457504 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97f409f990 msgr2=0x7f97f411c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 1 --2- 172.21.15.204:0/3118457504 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97f409f990 0x7f97f411c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 1 -- 172.21.15.204:0/3118457504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f40a2400 msgr2=0x7f97f411ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 1 --2- 172.21.15.204:0/3118457504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f40a2400 0x7f97f411ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 10 monclient: _finish_hunting 0 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 1 monclient: found mon.noname-g 2024-09-15T01:35:07.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 1 -- 172.21.15.204:0/3118457504 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e4009450 con 0x7f97f4132930 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97f94b8640 1 --2- 172.21.15.204:0/3118457504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f40a2400 0x7f97f411ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.795+0000 7f97fa4ba640 1 --2- 172.21.15.204:0/3118457504 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97f4132930 0x7f97f411cfd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00bdf0 tx=0x7f97ec009650 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97e9ffb640 1 -- 172.21.15.204:0/3118457504 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f97ec0067e0 con 0x7f97f4132930 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97e9ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97e9ffb640 10 monclient: dump: 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:07.817 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97e9ffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:07.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97e9ffb640 1 -- 172.21.15.204:0/3118457504 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec00f070 con 0x7f97f4132930 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97fbf1d640 5 monclient: authenticate success, global_id 14601 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97fbf1d640 10 monclient: _renew_subs 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97fbf1d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97fbf1d640 1 -- 172.21.15.204:0/3118457504 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f0009390 con 0x7f97f4132930 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97e9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97e9ffb640 1 -- 172.21.15.204:0/3118457504 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f97ec006a00 con 0x7f97f4132930 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97e9ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97e9ffb640 10 monclient: dump: 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.819 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:07.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97fbf1d640 10 monclient: _renew_subs 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97ebfff640 4 set_mon_vals no callback set 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97fbf1d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.796+0000 7f97fbf1d640 1 -- 172.21.15.204:0/3118457504 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f411ec10 con 0x7f97f4132930 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.797+0000 7f97e9ffb640 1 -- 172.21.15.204:0/3118457504 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f97ec006360 con 0x7f97f4132930 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.797+0000 7f97e9ffb640 1 --2- 172.21.15.204:0/3118457504 >> v2:172.21.15.204:6812/28210 conn(0x7f97d80396e0 0x7f97d803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.797+0000 7f97e9ffb640 1 -- 172.21.15.204:0/3118457504 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5167+0+0 (secure 0 0 0) 0x7f97ec04c110 con 0x7f97f4132930 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.797+0000 7f97fbf1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.797+0000 7f97fbf1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.797+0000 7f97f9cb9640 1 --2- 172.21.15.204:0/3118457504 >> v2:172.21.15.204:6812/28210 conn(0x7f97d80396e0 0x7f97d803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.797+0000 7f97fbf1d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.797+0000 7f97fbf1d640 1 -- 172.21.15.204:0/3118457504 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f4070b80 con 0x7f97f4132930 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.797+0000 7f97f9cb9640 10 monclient: get_auth_request con 0x7f97d80396e0 auth_method 0 2024-09-15T01:35:07.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.797+0000 7f97f9cb9640 1 --2- 172.21.15.204:0/3118457504 >> v2:172.21.15.204:6812/28210 conn(0x7f97d80396e0 0x7f97d803bb90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f97f0034000 tx=0x7f97f0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:07.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.801+0000 7f97e9ffb640 1 -- 172.21.15.204:0/3118457504 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f97ec011510 con 0x7f97f4132930 2024-09-15T01:35:07.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.801+0000 7f97e9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:07.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.801+0000 7f97e9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:07.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.960+0000 7f97fbf1d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:35:07.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.960+0000 7f97fbf1d640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:35:07.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.960+0000 7f97fbf1d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:07.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.960+0000 7f97fbf1d640 1 -- 172.21.15.204:0/3118457504 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f97f409e480 con 0x7f97f4132930 2024-09-15T01:35:07.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.960+0000 7f97e9ffb640 1 -- 172.21.15.204:0/3118457504 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v104) v1 ==== 56+0+1173 (secure 0 0 0) 0x7f97ec041070 con 0x7f97f4132930 2024-09-15T01:35:07.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.960+0000 7f97e9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:35:07.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:07.960+0000 7f97e9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:08.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.053+0000 7f97fbf1d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:35:08.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.053+0000 7f97fbf1d640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:35:08.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.053+0000 7f97fbf1d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:08.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.053+0000 7f97fbf1d640 1 -- 172.21.15.204:0/3118457504 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f97f40008d0 con 0x7f97f4132930 2024-09-15T01:35:08.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.053+0000 7f97e9ffb640 1 -- 172.21.15.204:0/3118457504 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v104) v1 ==== 56+0+1173 (secure 0 0 0) 0x7f97ec012b20 con 0x7f97f4132930 2024-09-15T01:35:08.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.053+0000 7f97e9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:35:08.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.053+0000 7f97e9ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:08.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.055+0000 7f97fbf1d640 1 -- 172.21.15.204:0/3118457504 >> v2:172.21.15.204:6812/28210 conn(0x7f97d80396e0 msgr2=0x7f97d803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:08.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.055+0000 7f97fbf1d640 1 --2- 172.21.15.204:0/3118457504 >> v2:172.21.15.204:6812/28210 conn(0x7f97d80396e0 0x7f97d803bb90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f97f0034000 tx=0x7f97f0032000 comp rx=0 tx=0).stop 2024-09-15T01:35:08.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.055+0000 7f97fbf1d640 10 monclient: shutdown 2024-09-15T01:35:08.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.055+0000 7f97fbf1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:08.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.055+0000 7f97fbf1d640 1 -- 172.21.15.204:0/3118457504 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97f4132930 msgr2=0x7f97f411cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:08.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.055+0000 7f97fbf1d640 1 --2- 172.21.15.204:0/3118457504 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97f4132930 0x7f97f411cfd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00bdf0 tx=0x7f97ec009650 comp rx=0 tx=0).stop 2024-09-15T01:35:08.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.056+0000 7f97fbf1d640 1 -- 172.21.15.204:0/3118457504 shutdown_connections 2024-09-15T01:35:08.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.056+0000 7f97fbf1d640 1 --2- 172.21.15.204:0/3118457504 >> v2:172.21.15.204:6812/28210 conn(0x7f97d80396e0 0x7f97d803bb90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.056+0000 7f97fbf1d640 1 --2- 172.21.15.204:0/3118457504 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97f4132930 0x7f97f411cfd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.056+0000 7f97fbf1d640 1 --2- 172.21.15.204:0/3118457504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f40a2400 0x7f97f411ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.056+0000 7f97fbf1d640 1 --2- 172.21.15.204:0/3118457504 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97f409f990 0x7f97f411c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.056+0000 7f97fbf1d640 1 -- 172.21.15.204:0/3118457504 >> 172.21.15.204:0/3118457504 conn(0x7f97f406e860 msgr2=0x7f97f4132d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:08.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.056+0000 7f97fbf1d640 1 -- 172.21.15.204:0/3118457504 shutdown_connections 2024-09-15T01:35:08.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.056+0000 7f97fbf1d640 1 -- 172.21.15.204:0/3118457504 wait complete. 2024-09-15T01:35:08.067 INFO:tasks.workunit.client.0.smithi204.stdout: 11 113.00000 osd.11 down 1.00000 1.00000 2024-09-15T01:35:08.067 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm osd.11 2024-09-15T01:35:08.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.139+0000 7f974bd9d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:08.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.139+0000 7f974bd9d640 10 monclient: build_initial_monmap 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.139+0000 7f974bd9d640 10 monclient: monmap: 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:08.140644+0000 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:08.140644+0000 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:08.142 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:08.143 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:08.143 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:08.143 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:08.143 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:08.143 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:08.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.139+0000 7f974bd9d640 1 Processor -- start 2024-09-15T01:35:08.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 1 -- start start 2024-09-15T01:35:08.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 10 monclient: init 2024-09-15T01:35:08.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:08.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 10 monclient: _add_conns ranks=[7,10,6,4,5,8,2,0,1,3,9] 2024-09-15T01:35:08.150 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97440683e0 0x7f974412f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 10 monclient(hunting): picked mon.noname-i con 0x7f97440683e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:08.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9744099870 0x7f974412fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 10 monclient(hunting): picked mon.noname-f con 0x7f9744099870 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:08.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97441301e0 0x7f9744095040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 10 monclient(hunting): picked mon.noname-d con 0x7f97441301e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:08.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:08.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974402a3a0 con 0x7f9744099870 2024-09-15T01:35:08.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:08.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f974409b480 con 0x7f97441301e0 2024-09-15T01:35:08.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:08.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f974409c770 con 0x7f97440683e0 2024-09-15T01:35:08.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:08.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974bd9d640 10 monclient(hunting): authenticate will time out at 923.783442s 2024-09-15T01:35:08.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749b39640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97440683e0 msgr2=0x7f974412f760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:08.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749b39640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97440683e0 0x7f974412f760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:08.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9744099870 0x7f974412fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:08.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974a33a640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97441301e0 msgr2=0x7f9744095040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:08.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9744099870 0x7f974412fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48454/0 (socket says 172.21.15.204:48454) 2024-09-15T01:35:08.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f974a33a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97441301e0 0x7f9744095040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:08.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 1 -- 172.21.15.204:0/272759 learned_addr learned my addr 172.21.15.204:0/272759 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:08.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 10 monclient(hunting): get_auth_request con 0x7f9744099870 auth_method 0 2024-09-15T01:35:08.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:08.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 10 monclient(hunting): handle_auth_done global_id 5995 payload 274 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 1 -- 172.21.15.204:0/272759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97440683e0 msgr2=0x7f974412f760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 1 --2- 172.21.15.204:0/272759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97440683e0 0x7f974412f760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 1 -- 172.21.15.204:0/272759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97441301e0 msgr2=0x7f9744095040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 1 --2- 172.21.15.204:0/272759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97441301e0 0x7f9744095040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 10 monclient: _finish_hunting 0 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 1 monclient: found mon.noname-f 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:08.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.140+0000 7f9749338640 1 -- 172.21.15.204:0/272759 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974404b720 con 0x7f9744099870 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9749338640 1 --2- 172.21.15.204:0/272759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9744099870 0x7f974412fca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f973400bbf0 tx=0x7f973400b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d45c7904b4abf4c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9748b37640 1 -- 172.21.15.204:0/272759 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f97340348a0 con 0x7f9744099870 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9748b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9748b37640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9748b37640 10 monclient: dump: 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:08.154 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:08.155 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:08.155 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:08.155 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:08.155 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:08.155 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:08.155 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:08.155 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:08.155 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:08.155 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:08.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9748b37640 10 monclient: _finish_auth 0 2024-09-15T01:35:08.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9748b37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:08.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f974bd9d640 5 monclient: authenticate success, global_id 5995 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9748b37640 1 -- 172.21.15.204:0/272759 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9734034a20 con 0x7f9744099870 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f974bd9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9748b37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f974bd9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9748b37640 1 -- 172.21.15.204:0/272759 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9734034c10 con 0x7f9744099870 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f974bd9d640 4 set_mon_vals no callback set 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9748b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9748b37640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.141+0000 7f9748b37640 10 monclient: dump: 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.156 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 10 monclient: shutdown 2024-09-15T01:35:08.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 1 -- 172.21.15.204:0/272759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9744099870 msgr2=0x7f974412fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 1 --2- 172.21.15.204:0/272759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9744099870 0x7f974412fca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f973400bbf0 tx=0x7f973400b460 comp rx=0 tx=0).stop 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 1 -- 172.21.15.204:0/272759 shutdown_connections 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 1 --2- 172.21.15.204:0/272759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97441301e0 0x7f9744095040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 1 --2- 172.21.15.204:0/272759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9744099870 0x7f974412fca0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 1 --2- 172.21.15.204:0/272759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97440683e0 0x7f974412f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 1 -- 172.21.15.204:0/272759 >> 172.21.15.204:0/272759 conn(0x7f97440a2e30 msgr2=0x7f97440a0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 1 -- 172.21.15.204:0/272759 shutdown_connections 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 1 -- 172.21.15.204:0/272759 wait complete. 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 10 monclient: build_initial_monmap 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.142+0000 7f974bd9d640 10 monclient: monmap: 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:08.158 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:08.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 1 Processor -- start 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 1 -- start start 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 10 monclient: init 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 10 monclient: _add_conns ranks=[6,5,3,7,0,4,1,8,2] 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97440683e0 0x7f974412c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 10 monclient(hunting): picked mon.noname-d con 0x7f97440683e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9744099870 0x7f974412e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 10 monclient(hunting): picked mon.noname-h con 0x7f9744099870 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97441301e0 0x7f974412e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 10 monclient(hunting): picked mon.noname-g con 0x7f97441301e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974412bfa0 con 0x7f97441301e0 2024-09-15T01:35:08.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f974412b670 con 0x7f9744099870 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f974412bc30 con 0x7f97440683e0 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974bd9d640 10 monclient(hunting): authenticate will time out at 923.786521s 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f9749b39640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97440683e0 msgr2=0x7f974412c900 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97441301e0 0x7f974412e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f9749b39640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97440683e0 0x7f974412c900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f9749338640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9744099870 0x7f974412e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97441301e0 0x7f974412e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:42696/0 (socket says 172.21.15.204:42696) 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f9749338640 10 monclient(hunting): get_auth_request con 0x7f9744099870 auth_method 0 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 1 -- 172.21.15.204:0/3277365146 learned_addr learned my addr 172.21.15.204:0/3277365146 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:08.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f9749338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f9749338640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f9749338640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 10 monclient(hunting): get_auth_request con 0x7f97441301e0 auth_method 0 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f9749338640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f9749338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f9749338640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:08.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 10 monclient(hunting): handle_auth_done global_id 14610 payload 824 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 1 -- 172.21.15.204:0/3277365146 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97440683e0 msgr2=0x7f974412c900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 1 --2- 172.21.15.204:0/3277365146 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97440683e0 0x7f974412c900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 1 -- 172.21.15.204:0/3277365146 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9744099870 msgr2=0x7f974412e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 1 --2- 172.21.15.204:0/3277365146 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9744099870 0x7f974412e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 10 monclient: _finish_hunting 0 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 1 monclient: found mon.noname-g 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f974a33a640 1 -- 172.21.15.204:0/3277365146 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973400b700 con 0x7f97441301e0 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.143+0000 7f9749338640 1 --2- 172.21.15.204:0/3277365146 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9744099870 0x7f974412e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f974a33a640 1 --2- 172.21.15.204:0/3277365146 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97441301e0 0x7f974412e8f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f973c009fc0 tx=0x7f973c009730 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f9739ffb640 1 -- 172.21.15.204:0/3277365146 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f973c00cd00 con 0x7f97441301e0 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f9739ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:08.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f9739ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f9739ffb640 10 monclient: dump: 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:08.164 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f9739ffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f9739ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f974bd9d640 5 monclient: authenticate success, global_id 14610 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f974bd9d640 10 monclient: _renew_subs 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f9739ffb640 1 -- 172.21.15.204:0/3277365146 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973c0065d0 con 0x7f97441301e0 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f974bd9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f974bd9d640 1 -- 172.21.15.204:0/3277365146 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974412b950 con 0x7f97441301e0 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f9739ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f9739ffb640 1 -- 172.21.15.204:0/3277365146 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f973c00cea0 con 0x7f97441301e0 2024-09-15T01:35:08.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f9739ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f9739ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f9739ffb640 10 monclient: dump: 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:08.166 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f974bd9d640 10 monclient: _renew_subs 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f974bd9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f974bd9d640 1 -- 172.21.15.204:0/3277365146 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9744128430 con 0x7f97441301e0 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.144+0000 7f973bfff640 4 set_mon_vals no callback set 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.145+0000 7f9739ffb640 1 -- 172.21.15.204:0/3277365146 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f973c012020 con 0x7f97441301e0 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.145+0000 7f9739ffb640 1 --2- 172.21.15.204:0/3277365146 >> v2:172.21.15.204:6812/28210 conn(0x7f97280397b0 0x7f972803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.145+0000 7f9739ffb640 1 -- 172.21.15.204:0/3277365146 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5167+0+0 (secure 0 0 0) 0x7f973c04b870 con 0x7f97441301e0 2024-09-15T01:35:08.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.145+0000 7f974bd9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:08.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.145+0000 7f974bd9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:08.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.145+0000 7f974bd9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:08.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.145+0000 7f9749b39640 1 --2- 172.21.15.204:0/3277365146 >> v2:172.21.15.204:6812/28210 conn(0x7f97280397b0 0x7f972803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:08.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.145+0000 7f974bd9d640 1 -- 172.21.15.204:0/3277365146 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9744070cc0 con 0x7f97441301e0 2024-09-15T01:35:08.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.145+0000 7f9749b39640 10 monclient: get_auth_request con 0x7f97280397b0 auth_method 0 2024-09-15T01:35:08.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.145+0000 7f9749b39640 1 --2- 172.21.15.204:0/3277365146 >> v2:172.21.15.204:6812/28210 conn(0x7f97280397b0 0x7f972803bc60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9740006fd0 tx=0x7f9740006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:08.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.149+0000 7f9739ffb640 1 -- 172.21.15.204:0/3277365146 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f973c006740 con 0x7f97441301e0 2024-09-15T01:35:08.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.149+0000 7f9739ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:08.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.149+0000 7f9739ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:08.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.310+0000 7f974bd9d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.11"}] 2024-09-15T01:35:08.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.310+0000 7f974bd9d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-09-15T01:35:08.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.310+0000 7f974bd9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:08.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.310+0000 7f974bd9d640 1 -- 172.21.15.204:0/3277365146 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.11"} v 0) v1 -- 0x7f974412a730 con 0x7f97441301e0 2024-09-15T01:35:08.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.460+0000 7f9739ffb640 1 -- 172.21.15.204:0/3277365146 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.11"}]=0 removed item id 11 name 'osd.11' from crush map v105) v1 ==== 125+0+0 (secure 0 0 0) 0x7f973c0072f0 con 0x7f97441301e0 2024-09-15T01:35:08.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.461+0000 7f9739ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-09-15T01:35:08.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.461+0000 7f9739ffb640 10 monclient: _finish_command 2 = system:0 removed item id 11 name 'osd.11' from crush map 2024-09-15T01:35:08.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.558+0000 7f974bd9d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.11"}] 2024-09-15T01:35:08.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.558+0000 7f974bd9d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-09-15T01:35:08.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.558+0000 7f974bd9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:08.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.558+0000 7f974bd9d640 1 -- 172.21.15.204:0/3277365146 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.11"} v 0) v1 -- 0x7f97440008d0 con 0x7f97441301e0 2024-09-15T01:35:08.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.559+0000 7f9739ffb640 1 -- 172.21.15.204:0/3277365146 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.11"}]=0 device 'osd.11' does not appear in the crush map v105) v1 ==== 126+0+0 (secure 0 0 0) 0x7f973c0074d0 con 0x7f97441301e0 2024-09-15T01:35:08.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.559+0000 7f9739ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-09-15T01:35:08.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.559+0000 7f9739ffb640 10 monclient: _finish_command 3 = system:0 device 'osd.11' does not appear in the crush map 2024-09-15T01:35:08.562 INFO:tasks.workunit.client.0.smithi204.stderr:device 'osd.11' does not appear in the crush map 2024-09-15T01:35:08.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 1 -- 172.21.15.204:0/3277365146 >> v2:172.21.15.204:6812/28210 conn(0x7f97280397b0 msgr2=0x7f972803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:08.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 1 --2- 172.21.15.204:0/3277365146 >> v2:172.21.15.204:6812/28210 conn(0x7f97280397b0 0x7f972803bc60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9740006fd0 tx=0x7f9740006d30 comp rx=0 tx=0).stop 2024-09-15T01:35:08.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 10 monclient: shutdown 2024-09-15T01:35:08.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:08.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 1 -- 172.21.15.204:0/3277365146 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97441301e0 msgr2=0x7f974412e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:08.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 1 --2- 172.21.15.204:0/3277365146 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97441301e0 0x7f974412e8f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f973c009fc0 tx=0x7f973c009730 comp rx=0 tx=0).stop 2024-09-15T01:35:08.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 1 -- 172.21.15.204:0/3277365146 shutdown_connections 2024-09-15T01:35:08.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 1 --2- 172.21.15.204:0/3277365146 >> v2:172.21.15.204:6812/28210 conn(0x7f97280397b0 0x7f972803bc60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 1 --2- 172.21.15.204:0/3277365146 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97441301e0 0x7f974412e8f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 1 --2- 172.21.15.204:0/3277365146 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9744099870 0x7f974412e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 1 --2- 172.21.15.204:0/3277365146 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f97440683e0 0x7f974412c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 1 -- 172.21.15.204:0/3277365146 >> 172.21.15.204:0/3277365146 conn(0x7f97440a2e30 msgr2=0x7f9744098270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:08.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 1 -- 172.21.15.204:0/3277365146 shutdown_connections 2024-09-15T01:35:08.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.561+0000 7f974bd9d640 1 -- 172.21.15.204:0/3277365146 wait complete. 2024-09-15T01:35:08.572 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd rm osd.11 2024-09-15T01:35:08.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.642+0000 7f932df14640 10 monclient: get_monmap_and_config 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.642+0000 7f932df14640 10 monclient: build_initial_monmap 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.642+0000 7f932df14640 10 monclient: monmap: 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:08.643802+0000 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:08.643802+0000 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:08.645 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:08.646 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:08.646 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:08.646 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:08.646 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:08.646 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:08.646 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:08.646 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:08.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 1 Processor -- start 2024-09-15T01:35:08.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 1 -- start start 2024-09-15T01:35:08.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 10 monclient: init 2024-09-15T01:35:08.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:08.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 10 monclient: _add_conns ranks=[1,5,3,6,10,4,2,7,8,9,0] 2024-09-15T01:35:08.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f93280a2400 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:08.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f932809f990 0x7f932809fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 10 monclient(hunting): picked mon.noname-h con 0x7f932809f990 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:08.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9328137c60 0x7f932813a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 10 monclient(hunting): picked mon.noname-g con 0x7f9328137c60 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:08.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93280648a0 con 0x7f93280a2400 2024-09-15T01:35:08.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:08.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932809e1a0 con 0x7f9328137c60 2024-09-15T01:35:08.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:08.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93280a11f0 con 0x7f932809f990 2024-09-15T01:35:08.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:08.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f932df14640 10 monclient(hunting): authenticate will time out at 924.286738s 2024-09-15T01:35:08.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f93277fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f93280a2400 0x7f93280a0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:55916/0 (socket says 172.21.15.204:55916) 2024-09-15T01:35:08.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f93277fe640 1 -- 172.21.15.204:0/3246477248 learned_addr learned my addr 172.21.15.204:0/3246477248 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9326ffd640 1 --2- 172.21.15.204:0/3246477248 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f932809f990 0x7f932809fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9327fff640 1 --2- 172.21.15.204:0/3246477248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9328137c60 0x7f932813a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9326ffd640 10 monclient(hunting): get_auth_request con 0x7f932809f990 auth_method 0 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9326ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9326ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9326ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9327fff640 10 monclient(hunting): get_auth_request con 0x7f9328137c60 auth_method 0 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9327fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9327fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9327fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9326ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9326ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:08.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.643+0000 7f9326ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9327fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9327fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9327fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f93267fc640 1 -- 172.21.15.204:0/3246477248 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371619696 0 0) 0x7f93280648a0 con 0x7f93280a2400 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f93267fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f93267fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f93267fc640 10 monclient(hunting): my global_id is 6004 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f93267fc640 1 -- 172.21.15.204:0/3246477248 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93100034e0 con 0x7f93280a2400 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9326ffd640 10 monclient(hunting): handle_auth_done global_id 5558 payload 274 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9326ffd640 1 -- 172.21.15.204:0/3246477248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9328137c60 msgr2=0x7f932813a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9326ffd640 1 --2- 172.21.15.204:0/3246477248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9328137c60 0x7f932813a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9326ffd640 1 -- 172.21.15.204:0/3246477248 >> v1:172.21.15.204:6789/0 conn(0x7f93280a2400 legacy=0x7f93280a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:08.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9326ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9326ffd640 1 monclient: found mon.noname-h 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9326ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9326ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9326ffd640 1 -- 172.21.15.204:0/3246477248 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280a27d0 con 0x7f932809f990 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9327fff640 1 --2- 172.21.15.204:0/3246477248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9328137c60 0x7f932813a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.644+0000 7f9326ffd640 1 --2- 172.21.15.204:0/3246477248 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f932809f990 0x7f932809fd60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f931c009e90 tx=0x7f931c009780 comp rx=0 tx=0).ready entity=mon.5 client_cookie=fdc495fbf3972548 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f93267fc640 1 -- 172.21.15.204:0/3246477248 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f931c031880 con 0x7f932809f990 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f93267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f93267fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f93267fc640 10 monclient: dump: 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.658 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f93267fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:08.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f93267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f93267fc640 1 -- 172.21.15.204:0/3246477248 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c02d050 con 0x7f932809f990 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 5 monclient: authenticate success, global_id 5558 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f93267fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f93267fc640 1 -- 172.21.15.204:0/3246477248 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f931c0303a0 con 0x7f932809f990 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f93267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 4 set_mon_vals no callback set 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f93267fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f93267fc640 10 monclient: dump: 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:08.660 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:08.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 10 monclient: shutdown 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 1 -- 172.21.15.204:0/3246477248 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f932809f990 msgr2=0x7f932809fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 1 --2- 172.21.15.204:0/3246477248 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f932809f990 0x7f932809fd60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f931c009e90 tx=0x7f931c009780 comp rx=0 tx=0).stop 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 1 -- 172.21.15.204:0/3246477248 shutdown_connections 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 1 --2- 172.21.15.204:0/3246477248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9328137c60 0x7f932813a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 1 --2- 172.21.15.204:0/3246477248 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f932809f990 0x7f932809fd60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 1 -- 172.21.15.204:0/3246477248 >> 172.21.15.204:0/3246477248 conn(0x7f932806e860 msgr2=0x7f932806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 1 -- 172.21.15.204:0/3246477248 shutdown_connections 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.645+0000 7f932df14640 1 -- 172.21.15.204:0/3246477248 wait complete. 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.646+0000 7f932df14640 10 monclient: build_initial_monmap 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.646+0000 7f932df14640 10 monclient: monmap: 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:08.662 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:08.663 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.646+0000 7f932df14640 1 Processor -- start 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.646+0000 7f932df14640 1 -- start start 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.646+0000 7f932df14640 10 monclient: init 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 10 monclient: _add_conns ranks=[7,8,6,3,1,0,4,2,5] 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f932809f990 0x7f9328098b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 10 monclient(hunting): picked mon.noname-i con 0x7f932809f990 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93280a2400 0x7f9328099040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 10 monclient(hunting): picked mon.noname-e con 0x7f93280a2400 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9328137c60 0x7f9328099580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 10 monclient(hunting): picked mon.noname-d con 0x7f9328137c60 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:08.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f932809b0d0 con 0x7f9328137c60 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f932809a6d0 con 0x7f932809f990 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f932809d3d0 con 0x7f93280a2400 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f93277fe640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f932809f990 msgr2=0x7f9328098b00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f932df14640 10 monclient(hunting): authenticate will time out at 924.290106s 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f93277fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f932809f990 0x7f9328098b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9327fff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9328137c60 msgr2=0x7f9328099580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9327fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9328137c60 0x7f9328099580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93280a2400 0x7f9328099040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93280a2400 0x7f9328099040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:50926/0 (socket says 172.21.15.204:50926) 2024-09-15T01:35:08.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 1 -- 172.21.15.204:0/4190735970 learned_addr learned my addr 172.21.15.204:0/4190735970 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 10 monclient(hunting): get_auth_request con 0x7f93280a2400 auth_method 0 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 10 monclient(hunting): handle_auth_done global_id 5489 payload 824 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 1 -- 172.21.15.204:0/4190735970 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f932809f990 msgr2=0x7f9328098b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 1 --2- 172.21.15.204:0/4190735970 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f932809f990 0x7f9328098b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 1 -- 172.21.15.204:0/4190735970 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9328137c60 msgr2=0x7f9328099580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 1 --2- 172.21.15.204:0/4190735970 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9328137c60 0x7f9328099580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:35:08.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 1 monclient: found mon.noname-e 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.647+0000 7f9326ffd640 1 -- 172.21.15.204:0/4190735970 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9320009390 con 0x7f93280a2400 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f9326ffd640 1 --2- 172.21.15.204:0/4190735970 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93280a2400 0x7f9328099040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f931c0099f0 tx=0x7f931c037000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f930f7fe640 1 -- 172.21.15.204:0/4190735970 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f931c03dac0 con 0x7f93280a2400 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f930f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f930f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f930f7fe640 10 monclient: dump: 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.667 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f930f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f930f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:08.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f932df14640 5 monclient: authenticate success, global_id 5489 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f930f7fe640 1 -- 172.21.15.204:0/4190735970 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c03dc40 con 0x7f93280a2400 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f930f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f932df14640 10 monclient: _renew_subs 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f930f7fe640 1 -- 172.21.15.204:0/4190735970 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f931c03ddb0 con 0x7f93280a2400 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f932df14640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f932df14640 1 -- 172.21.15.204:0/4190735970 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c009a20 con 0x7f93280a2400 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f930f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f930f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f930f7fe640 10 monclient: dump: 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.669 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f932df14640 10 monclient: _renew_subs 2024-09-15T01:35:08.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f932df14640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:08.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f932df14640 1 -- 172.21.15.204:0/4190735970 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932809c190 con 0x7f93280a2400 2024-09-15T01:35:08.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.648+0000 7f9325ffb640 4 set_mon_vals no callback set 2024-09-15T01:35:08.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.649+0000 7f932df14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:08.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.649+0000 7f932df14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:08.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.649+0000 7f932df14640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:08.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.649+0000 7f932df14640 1 -- 172.21.15.204:0/4190735970 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93280667d0 con 0x7f93280a2400 2024-09-15T01:35:08.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.649+0000 7f930f7fe640 1 -- 172.21.15.204:0/4190735970 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f931c003040 con 0x7f93280a2400 2024-09-15T01:35:08.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.649+0000 7f930f7fe640 1 --2- 172.21.15.204:0/4190735970 >> v2:172.21.15.204:6812/28210 conn(0x7f93140397b0 0x7f931403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:08.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.649+0000 7f930f7fe640 1 -- 172.21.15.204:0/4190735970 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5145+0+0 (secure 0 0 0) 0x7f931c06ca30 con 0x7f93280a2400 2024-09-15T01:35:08.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.649+0000 7f93277fe640 1 --2- 172.21.15.204:0/4190735970 >> v2:172.21.15.204:6812/28210 conn(0x7f93140397b0 0x7f931403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:08.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.649+0000 7f93277fe640 10 monclient: get_auth_request con 0x7f93140397b0 auth_method 0 2024-09-15T01:35:08.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.649+0000 7f93277fe640 1 --2- 172.21.15.204:0/4190735970 >> v2:172.21.15.204:6812/28210 conn(0x7f93140397b0 0x7f931403bc60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9318000c00 tx=0x7f9318007a10 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:08.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.654+0000 7f930f7fe640 1 -- 172.21.15.204:0/4190735970 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f931c03d640 con 0x7f93280a2400 2024-09-15T01:35:08.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.654+0000 7f930f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:08.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.654+0000 7f930f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:08.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.819+0000 7f932df14640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-09-15T01:35:08.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.819+0000 7f932df14640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-09-15T01:35:08.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.819+0000 7f932df14640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:08.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:08.819+0000 7f932df14640 1 -- 172.21.15.204:0/4190735970 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.11"]} v 0) v1 -- 0x7f9328070b80 con 0x7f93280a2400 2024-09-15T01:35:08.822 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:35:08.820+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e105 definitely_dead 0 2024-09-15T01:35:09.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.456+0000 7f930f7fe640 1 -- 172.21.15.204:0/4190735970 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.11"]}]=0 removed osd.11 v106) v1 ==== 87+0+0 (secure 0 0 0) 0x7f931c039ba0 con 0x7f93280a2400 2024-09-15T01:35:09.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.456+0000 7f930f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-09-15T01:35:09.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.456+0000 7f930f7fe640 10 monclient: _finish_command 2 = system:0 removed osd.11 2024-09-15T01:35:09.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.560+0000 7f932df14640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-09-15T01:35:09.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.560+0000 7f932df14640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-09-15T01:35:09.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.560+0000 7f932df14640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:09.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.560+0000 7f932df14640 1 -- 172.21.15.204:0/4190735970 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.11"]} v 0) v1 -- 0x7f932809dec0 con 0x7f93280a2400 2024-09-15T01:35:09.563 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:35:09.561+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e106 definitely_dead 0 2024-09-15T01:35:09.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.562+0000 7f930f7fe640 1 -- 172.21.15.204:0/4190735970 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.11"]}]=0 osd.11 does not exist. v106) v1 ==== 96+0+0 (secure 0 0 0) 0x7f931c061090 con 0x7f93280a2400 2024-09-15T01:35:09.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.562+0000 7f930f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-09-15T01:35:09.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.562+0000 7f930f7fe640 10 monclient: _finish_command 3 = system:0 osd.11 does not exist. 2024-09-15T01:35:09.564 INFO:tasks.workunit.client.0.smithi204.stderr:osd.11 does not exist. 2024-09-15T01:35:09.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 1 -- 172.21.15.204:0/4190735970 >> v2:172.21.15.204:6812/28210 conn(0x7f93140397b0 msgr2=0x7f931403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:09.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 1 --2- 172.21.15.204:0/4190735970 >> v2:172.21.15.204:6812/28210 conn(0x7f93140397b0 0x7f931403bc60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9318000c00 tx=0x7f9318007a10 comp rx=0 tx=0).stop 2024-09-15T01:35:09.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 10 monclient: shutdown 2024-09-15T01:35:09.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:09.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 1 -- 172.21.15.204:0/4190735970 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93280a2400 msgr2=0x7f9328099040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:09.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 1 --2- 172.21.15.204:0/4190735970 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93280a2400 0x7f9328099040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f931c0099f0 tx=0x7f931c037000 comp rx=0 tx=0).stop 2024-09-15T01:35:09.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 1 -- 172.21.15.204:0/4190735970 shutdown_connections 2024-09-15T01:35:09.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 1 --2- 172.21.15.204:0/4190735970 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9328137c60 0x7f9328099580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:09.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 1 --2- 172.21.15.204:0/4190735970 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93280a2400 0x7f9328099040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:09.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 1 --2- 172.21.15.204:0/4190735970 >> v2:172.21.15.204:6812/28210 conn(0x7f93140397b0 0x7f931403bc60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:09.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 1 --2- 172.21.15.204:0/4190735970 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f932809f990 0x7f9328098b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:09.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 1 -- 172.21.15.204:0/4190735970 >> 172.21.15.204:0/4190735970 conn(0x7f932806e860 msgr2=0x7f93280a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:09.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 1 -- 172.21.15.204:0/4190735970 shutdown_connections 2024-09-15T01:35:09.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.563+0000 7f932df14640 1 -- 172.21.15.204:0/4190735970 wait complete. 2024-09-15T01:35:09.575 INFO:tasks.workunit.client.0.smithi204.stderr:++ ceph osd create 2024-09-15T01:35:09.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.644+0000 7fd6f5909640 10 monclient: get_monmap_and_config 2024-09-15T01:35:09.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.644+0000 7fd6f5909640 10 monclient: build_initial_monmap 2024-09-15T01:35:09.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.644+0000 7fd6f5909640 10 monclient: monmap: 2024-09-15T01:35:09.646 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:09.646 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:09.645605+0000 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:09.645605+0000 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:09.647 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:09.648 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:09.648 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:09.648 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:09.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.644+0000 7fd6f5909640 1 Processor -- start 2024-09-15T01:35:09.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.644+0000 7fd6f5909640 1 -- start start 2024-09-15T01:35:09.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.644+0000 7fd6f5909640 10 monclient: init 2024-09-15T01:35:09.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:09.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 10 monclient: _add_conns ranks=[8,5,7,9,2,10,1,0,4,3,6] 2024-09-15T01:35:09.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6f0132c20 0x7fd6f0132ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:09.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 10 monclient(hunting): picked mon.noname-e con 0x7fd6f0132c20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:09.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6f006eda0 0x7fd6f0133530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:09.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 10 monclient(hunting): picked mon.noname-h con 0x7fd6f006eda0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:09.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd6f009e750 0x7fd6f009ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:09.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 10 monclient(hunting): picked mon.noname-i con 0x7fd6f009e750 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:09.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:09.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f0132a90 con 0x7fd6f006eda0 2024-09-15T01:35:09.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f009ffc0 con 0x7fd6f009e750 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00a03d0 con 0x7fd6f0132c20 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6f5909640 10 monclient(hunting): authenticate will time out at 925.288252s 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ef7fe640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd6f009e750 msgr2=0x7fd6f009ebc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd6f009e750 0x7fd6f009ebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6f006eda0 0x7fd6f0133530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6f006eda0 0x7fd6f0133530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:50048/0 (socket says 172.21.15.204:50048) 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 1 -- 172.21.15.204:0/2365313562 learned_addr learned my addr 172.21.15.204:0/2365313562 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 10 monclient(hunting): get_auth_request con 0x7fd6f006eda0 auth_method 0 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:09.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:09.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:09.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:09.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:09.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:09.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6eeffd640 1 --2- 172.21.15.204:0/2365313562 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6f0132c20 0x7fd6f0132ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:09.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6eeffd640 10 monclient(hunting): get_auth_request con 0x7fd6f0132c20 auth_method 0 2024-09-15T01:35:09.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:09.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6eeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:09.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6eeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:09.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 10 monclient(hunting): handle_auth_done global_id 5567 payload 274 2024-09-15T01:35:09.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 1 -- 172.21.15.204:0/2365313562 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6f0132c20 msgr2=0x7fd6f0132ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:09.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 1 --2- 172.21.15.204:0/2365313562 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6f0132c20 0x7fd6f0132ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 1 -- 172.21.15.204:0/2365313562 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd6f009e750 msgr2=0x7fd6f009ebc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 1 --2- 172.21.15.204:0/2365313562 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd6f009e750 0x7fd6f009ebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 1 monclient: found mon.noname-h 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.645+0000 7fd6ee7fc640 1 -- 172.21.15.204:0/2365313562 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f00648a0 con 0x7fd6f006eda0 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6ee7fc640 1 --2- 172.21.15.204:0/2365313562 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6f006eda0 0x7fd6f0133530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009940 tx=0x7fd6e40091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=4f7590b8210e692 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6edffb640 1 -- 172.21.15.204:0/2365313562 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6e402ad20 con 0x7fd6f006eda0 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6edffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6edffb640 10 monclient: dump: 2024-09-15T01:35:09.660 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:09.661 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:09.661 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:09.661 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:09.661 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:09.661 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:09.661 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:09.661 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:09.661 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:09.661 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:09.661 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:09.661 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:09.661 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6edffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6edffb640 1 -- 172.21.15.204:0/2365313562 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e4031820 con 0x7fd6f006eda0 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6f5909640 5 monclient: authenticate success, global_id 5567 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6f5909640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6edffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6f5909640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6edffb640 1 -- 172.21.15.204:0/2365313562 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6e402aea0 con 0x7fd6f006eda0 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6f5909640 4 set_mon_vals no callback set 2024-09-15T01:35:09.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6edffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.646+0000 7fd6edffb640 10 monclient: dump: 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:09.663 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 10 monclient: shutdown 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 1 -- 172.21.15.204:0/2365313562 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6f006eda0 msgr2=0x7fd6f0133530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 1 --2- 172.21.15.204:0/2365313562 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6f006eda0 0x7fd6f0133530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009940 tx=0x7fd6e40091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 1 -- 172.21.15.204:0/2365313562 shutdown_connections 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 1 --2- 172.21.15.204:0/2365313562 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd6f009e750 0x7fd6f009ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 1 --2- 172.21.15.204:0/2365313562 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6f006eda0 0x7fd6f0133530 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 1 --2- 172.21.15.204:0/2365313562 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6f0132c20 0x7fd6f0132ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:09.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 1 -- 172.21.15.204:0/2365313562 >> 172.21.15.204:0/2365313562 conn(0x7fd6f0067bb0 msgr2=0x7fd6f00a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 1 -- 172.21.15.204:0/2365313562 shutdown_connections 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 1 -- 172.21.15.204:0/2365313562 wait complete. 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 10 monclient: build_initial_monmap 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.647+0000 7fd6f5909640 10 monclient: monmap: 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:09.665 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 1 Processor -- start 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 1 -- start start 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 10 monclient: init 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 10 monclient: _add_conns ranks=[8,3,0,4,1,7,5,2,6] 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6f006eda0 0x7fd6f012f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:09.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 10 monclient(hunting): picked mon.noname-e con 0x7fd6f006eda0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f009e750 0x7fd6f011c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 10 monclient(hunting): picked mon.noname-g con 0x7fd6f009e750 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6f0132c20 0x7fd6f011c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 10 monclient(hunting): picked mon.noname-a con 0x7fd6f0132c20 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f0068aa0 con 0x7fd6f0132c20 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00685c0 con 0x7fd6f009e750 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f0133fa0 con 0x7fd6f006eda0 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6f5909640 10 monclient(hunting): authenticate will time out at 925.291594s 2024-09-15T01:35:09.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f009e750 0x7fd6f011c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:09.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f009e750 0x7fd6f011c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:42710/0 (socket says 172.21.15.204:42710) 2024-09-15T01:35:09.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 1 -- 172.21.15.204:0/3807943493 learned_addr learned my addr 172.21.15.204:0/3807943493 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:09.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 10 monclient(hunting): get_auth_request con 0x7fd6f009e750 auth_method 0 2024-09-15T01:35:09.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:09.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:09.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:09.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6eeffd640 1 --2- 172.21.15.204:0/3807943493 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6f006eda0 0x7fd6f012f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:09.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:09.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:09.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:09.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6eeffd640 10 monclient(hunting): get_auth_request con 0x7fd6f006eda0 auth_method 0 2024-09-15T01:35:09.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6eeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6eeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ef7fe640 1 --2- 172.21.15.204:0/3807943493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6f0132c20 0x7fd6f011c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ef7fe640 10 monclient(hunting): get_auth_request con 0x7fd6f0132c20 auth_method 0 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 10 monclient(hunting): handle_auth_done global_id 14628 payload 824 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 1 -- 172.21.15.204:0/3807943493 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6f006eda0 msgr2=0x7fd6f012f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 1 --2- 172.21.15.204:0/3807943493 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6f006eda0 0x7fd6f012f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:09.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 1 -- 172.21.15.204:0/3807943493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6f0132c20 msgr2=0x7fd6f011c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 1 --2- 172.21.15.204:0/3807943493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6f0132c20 0x7fd6f011c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 1 monclient: found mon.noname-g 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.648+0000 7fd6ee7fc640 1 -- 172.21.15.204:0/3807943493 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e4009450 con 0x7fd6f009e750 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6ee7fc640 1 --2- 172.21.15.204:0/3807943493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f009e750 0x7fd6f011c0b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4000c00 tx=0x7fd6e4032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6d6ffd640 1 -- 172.21.15.204:0/3807943493 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6e4033db0 con 0x7fd6f009e750 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6d6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6d6ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6d6ffd640 10 monclient: dump: 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:09.670 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:09.671 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6d6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6f5909640 5 monclient: authenticate success, global_id 14628 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6d6ffd640 1 -- 172.21.15.204:0/3807943493 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e403b3f0 con 0x7fd6f009e750 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6f5909640 10 monclient: _renew_subs 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6f5909640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6f5909640 1 -- 172.21.15.204:0/3807943493 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f011fc00 con 0x7fd6f009e750 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6d6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6d6ffd640 1 -- 172.21.15.204:0/3807943493 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6e403b9e0 con 0x7fd6f009e750 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6d6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6d6ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:09.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6d6ffd640 10 monclient: dump: 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:09.673 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6ed7fa640 4 set_mon_vals no callback set 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6f5909640 10 monclient: _renew_subs 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6f5909640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.649+0000 7fd6f5909640 1 -- 172.21.15.204:0/3807943493 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f0142970 con 0x7fd6f009e750 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.650+0000 7fd6d6ffd640 1 -- 172.21.15.204:0/3807943493 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd6e4039040 con 0x7fd6f009e750 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.650+0000 7fd6f5909640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.650+0000 7fd6f5909640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.650+0000 7fd6f5909640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.650+0000 7fd6d6ffd640 1 --2- 172.21.15.204:0/3807943493 >> v2:172.21.15.204:6812/28210 conn(0x7fd6dc0397b0 0x7fd6dc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.650+0000 7fd6f5909640 1 -- 172.21.15.204:0/3807943493 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f0069440 con 0x7fd6f009e750 2024-09-15T01:35:09.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.650+0000 7fd6d6ffd640 1 -- 172.21.15.204:0/3807943493 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5164+0+0 (secure 0 0 0) 0x7fd6e406b2b0 con 0x7fd6f009e750 2024-09-15T01:35:09.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.650+0000 7fd6eeffd640 1 --2- 172.21.15.204:0/3807943493 >> v2:172.21.15.204:6812/28210 conn(0x7fd6dc0397b0 0x7fd6dc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:09.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.650+0000 7fd6eeffd640 10 monclient: get_auth_request con 0x7fd6dc0397b0 auth_method 0 2024-09-15T01:35:09.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.650+0000 7fd6eeffd640 1 --2- 172.21.15.204:0/3807943493 >> v2:172.21.15.204:6812/28210 conn(0x7fd6dc0397b0 0x7fd6dc03bc60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00075a0 tx=0x7fd6e00097e0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:09.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.654+0000 7fd6d6ffd640 1 -- 172.21.15.204:0/3807943493 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd6e4045580 con 0x7fd6f009e750 2024-09-15T01:35:09.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.654+0000 7fd6d6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:09.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.654+0000 7fd6d6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:09.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.822+0000 7fd6f5909640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T01:35:09.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.822+0000 7fd6f5909640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-09-15T01:35:09.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.822+0000 7fd6f5909640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:09.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:09.822+0000 7fd6f5909640 1 -- 172.21.15.204:0/3807943493 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7fd6f0070b80 con 0x7fd6f009e750 2024-09-15T01:35:10.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:10.458+0000 7fd6d6ffd640 1 -- 172.21.15.204:0/3807943493 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v107) v1 ==== 58+0+3 (secure 0 0 0) 0x7fd6e4042080 con 0x7fd6f009e750 2024-09-15T01:35:10.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:10.458+0000 7fd6d6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-09-15T01:35:10.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:10.458+0000 7fd6d6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:10.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:10.567+0000 7fd6f5909640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T01:35:10.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:10.567+0000 7fd6f5909640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-09-15T01:35:10.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:10.567+0000 7fd6f5909640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:10.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:10.567+0000 7fd6f5909640 1 -- 172.21.15.204:0/3807943493 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7fd6f011db10 con 0x7fd6f009e750 2024-09-15T01:35:11.475 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.472+0000 7fd6d6ffd640 1 -- 172.21.15.204:0/3807943493 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v108) v1 ==== 58+0+3 (secure 0 0 0) 0x7fd6e4030ae0 con 0x7fd6f009e750 2024-09-15T01:35:11.475 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.472+0000 7fd6d6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-09-15T01:35:11.475 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.472+0000 7fd6d6ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:11.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 1 -- 172.21.15.204:0/3807943493 >> v2:172.21.15.204:6812/28210 conn(0x7fd6dc0397b0 msgr2=0x7fd6dc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:11.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 1 --2- 172.21.15.204:0/3807943493 >> v2:172.21.15.204:6812/28210 conn(0x7fd6dc0397b0 0x7fd6dc03bc60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00075a0 tx=0x7fd6e00097e0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 10 monclient: shutdown 2024-09-15T01:35:11.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:11.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 1 -- 172.21.15.204:0/3807943493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f009e750 msgr2=0x7fd6f011c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:11.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 1 --2- 172.21.15.204:0/3807943493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f009e750 0x7fd6f011c0b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4000c00 tx=0x7fd6e4032000 comp rx=0 tx=0).stop 2024-09-15T01:35:11.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 1 -- 172.21.15.204:0/3807943493 shutdown_connections 2024-09-15T01:35:11.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 1 --2- 172.21.15.204:0/3807943493 >> v2:172.21.15.204:6812/28210 conn(0x7fd6dc0397b0 0x7fd6dc03bc60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 1 --2- 172.21.15.204:0/3807943493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6f0132c20 0x7fd6f011c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 1 --2- 172.21.15.204:0/3807943493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f009e750 0x7fd6f011c0b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 1 --2- 172.21.15.204:0/3807943493 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6f006eda0 0x7fd6f012f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 1 -- 172.21.15.204:0/3807943493 >> 172.21.15.204:0/3807943493 conn(0x7fd6f0067bb0 msgr2=0x7fd6f0131580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:11.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 1 -- 172.21.15.204:0/3807943493 shutdown_connections 2024-09-15T01:35:11.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.475+0000 7fd6f5909640 1 -- 172.21.15.204:0/3807943493 wait complete. 2024-09-15T01:35:11.489 INFO:tasks.workunit.client.0.smithi204.stderr:+ o4=12 2024-09-15T01:35:11.489 INFO:tasks.workunit.client.0.smithi204.stderr:++ ceph osd create 2024-09-15T01:35:11.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.562+0000 7f2a4ecd4640 10 monclient: get_monmap_and_config 2024-09-15T01:35:11.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.562+0000 7f2a4ecd4640 10 monclient: build_initial_monmap 2024-09-15T01:35:11.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.562+0000 7f2a4ecd4640 10 monclient: monmap: 2024-09-15T01:35:11.565 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:11.565 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:11.565 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:11.563789+0000 2024-09-15T01:35:11.565 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:11.563789+0000 2024-09-15T01:35:11.565 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:11.565 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:11.565 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:11.565 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:11.565 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:11.565 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:11.565 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:11.566 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:11.566 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:11.566 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:11.566 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:11.566 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:11.566 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:11.566 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:11.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 1 Processor -- start 2024-09-15T01:35:11.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 1 -- start start 2024-09-15T01:35:11.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 10 monclient: init 2024-09-15T01:35:11.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:11.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 10 monclient: _add_conns ranks=[7,4,5,6,10,9,0,3,8,1,2] 2024-09-15T01:35:11.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2a480a2400 0x7f2a480a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:11.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 10 monclient(hunting): picked mon.noname-i con 0x7f2a480a2400 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:11.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2a4809f990 0x7f2a4809fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:11.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 10 monclient(hunting): picked mon.noname-c con 0x7f2a4809f990 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:11.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2a48132930 0x7f2a48132d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:11.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 10 monclient(hunting): picked mon.noname-h con 0x7f2a48132930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:11.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:11.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4809e1a0 con 0x7f2a4809f990 2024-09-15T01:35:11.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:11.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a480a27d0 con 0x7f2a48132930 2024-09-15T01:35:11.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:11.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4809dec0 con 0x7f2a480a2400 2024-09-15T01:35:11.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:11.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ecd4640 10 monclient(hunting): authenticate will time out at 927.206685s 2024-09-15T01:35:11.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ca70640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2a480a2400 msgr2=0x7f2a480a0ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:11.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4ca70640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2a480a2400 0x7f2a480a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:11.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4d271640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2a48132930 0x7f2a48132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:11.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4d271640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2a48132930 0x7f2a48132d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:50050/0 (socket says 172.21.15.204:50050) 2024-09-15T01:35:11.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4d271640 1 -- 172.21.15.204:0/1342208872 learned_addr learned my addr 172.21.15.204:0/1342208872 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4d271640 10 monclient(hunting): get_auth_request con 0x7f2a48132930 auth_method 0 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4d271640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4d271640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4d271640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4d271640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4d271640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.563+0000 7f2a4d271640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a47fff640 1 --2- 172.21.15.204:0/1342208872 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2a4809f990 0x7f2a4809fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a47fff640 10 monclient(hunting): get_auth_request con 0x7f2a4809f990 auth_method 0 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a47fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a47fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:11.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a4d271640 10 monclient(hunting): handle_auth_done global_id 5576 payload 274 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a4d271640 1 -- 172.21.15.204:0/1342208872 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2a480a2400 msgr2=0x7f2a480a0ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a4d271640 1 --2- 172.21.15.204:0/1342208872 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2a480a2400 0x7f2a480a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a4d271640 1 -- 172.21.15.204:0/1342208872 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2a4809f990 msgr2=0x7f2a4809fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a4d271640 1 --2- 172.21.15.204:0/1342208872 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2a4809f990 0x7f2a4809fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a4d271640 10 monclient: _finish_hunting 0 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a4d271640 1 monclient: found mon.noname-h 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a4d271640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a4d271640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a4d271640 1 -- 172.21.15.204:0/1342208872 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a480648a0 con 0x7f2a48132930 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a4d271640 1 --2- 172.21.15.204:0/1342208872 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2a48132930 0x7f2a48132d60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009940 tx=0x7f2a300091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=376d60e2fe32e6c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.564+0000 7f2a477fe640 1 -- 172.21.15.204:0/1342208872 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2a3002ac80 con 0x7f2a48132930 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:11.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a477fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a477fe640 10 monclient: dump: 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:11.589 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a477fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 5 monclient: authenticate success, global_id 5576 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a477fe640 1 -- 172.21.15.204:0/1342208872 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3002ae00 con 0x7f2a48132930 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a477fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 4 set_mon_vals no callback set 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a477fe640 1 -- 172.21.15.204:0/1342208872 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2a30031ca0 con 0x7f2a48132930 2024-09-15T01:35:11.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a477fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a477fe640 10 monclient: dump: 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:11.591 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:11.592 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:11.592 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:11.592 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:11.592 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:11.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 10 monclient: shutdown 2024-09-15T01:35:11.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:11.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/1342208872 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2a48132930 msgr2=0x7f2a48132d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:11.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 1 --2- 172.21.15.204:0/1342208872 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2a48132930 0x7f2a48132d60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009940 tx=0x7f2a300091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/1342208872 shutdown_connections 2024-09-15T01:35:11.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 1 --2- 172.21.15.204:0/1342208872 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2a48132930 0x7f2a48132d60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 1 --2- 172.21.15.204:0/1342208872 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2a4809f990 0x7f2a4809fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 1 --2- 172.21.15.204:0/1342208872 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2a480a2400 0x7f2a480a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/1342208872 >> 172.21.15.204:0/1342208872 conn(0x7f2a4806e860 msgr2=0x7f2a4806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/1342208872 shutdown_connections 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.565+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/1342208872 wait complete. 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.566+0000 7f2a4ecd4640 10 monclient: build_initial_monmap 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.566+0000 7f2a4ecd4640 10 monclient: monmap: 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:11.593 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.566+0000 7f2a4ecd4640 1 Processor -- start 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.566+0000 7f2a4ecd4640 1 -- start start 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.566+0000 7f2a4ecd4640 10 monclient: init 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 10 monclient: _add_conns ranks=[4,0,8,1,5,7,2,6,3] 2024-09-15T01:35:11.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2a4809f990 0x7f2a48141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 10 monclient(hunting): picked mon.noname-c con 0x7f2a4809f990 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a480a2400 0x7f2a48142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 10 monclient(hunting): picked mon.noname-a con 0x7f2a480a2400 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2a48132930 0x7f2a4811d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 10 monclient(hunting): picked mon.noname-e con 0x7f2a48132930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4809d3d0 con 0x7f2a480a2400 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4809cad0 con 0x7f2a4809f990 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4811ece0 con 0x7f2a48132930 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ecd4640 10 monclient(hunting): authenticate will time out at 927.210103s 2024-09-15T01:35:11.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a480a2400 0x7f2a48142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4d271640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2a48132930 0x7f2a4811d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ca70640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2a4809f990 0x7f2a48141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a480a2400 0x7f2a48142500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:34604/0 (socket says 172.21.15.204:34604) 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 1 -- 172.21.15.204:0/4013504286 learned_addr learned my addr 172.21.15.204:0/4013504286 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4d271640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2a48132930 0x7f2a4811d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:50954/0 (socket says 172.21.15.204:50954) 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 10 monclient(hunting): get_auth_request con 0x7f2a480a2400 auth_method 0 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4d271640 10 monclient(hunting): get_auth_request con 0x7f2a48132930 auth_method 0 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4d271640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4d271640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:11.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4d271640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ca70640 10 monclient(hunting): get_auth_request con 0x7f2a4809f990 auth_method 0 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ca70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ca70640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ca70640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4d271640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4d271640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4d271640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ca70640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ca70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:11.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4ca70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 10 monclient(hunting): handle_auth_done global_id 5706 payload 824 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 1 -- 172.21.15.204:0/4013504286 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2a48132930 msgr2=0x7f2a4811d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 1 --2- 172.21.15.204:0/4013504286 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2a48132930 0x7f2a4811d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 1 -- 172.21.15.204:0/4013504286 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2a4809f990 msgr2=0x7f2a48141fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 1 --2- 172.21.15.204:0/4013504286 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2a4809f990 0x7f2a48141fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 1 monclient: found mon.noname-a 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a47fff640 1 -- 172.21.15.204:0/4013504286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a30009450 con 0x7f2a480a2400 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.567+0000 7f2a4d271640 1 --2- 172.21.15.204:0/4013504286 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2a48132930 0x7f2a4811d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a47fff640 1 --2- 172.21.15.204:0/4013504286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a480a2400 0x7f2a48142500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800e190 tx=0x7f2a38009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:11.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a44ff9640 1 -- 172.21.15.204:0/4013504286 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2a38009a60 con 0x7f2a480a2400 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a44ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a44ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a44ff9640 10 monclient: dump: 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:11.599 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a44ff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a44ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a44ff9640 1 -- 172.21.15.204:0/4013504286 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a380148a0 con 0x7f2a480a2400 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a4ecd4640 5 monclient: authenticate success, global_id 5706 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a4ecd4640 10 monclient: _renew_subs 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a4ecd4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/4013504286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4811f630 con 0x7f2a480a2400 2024-09-15T01:35:11.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a44ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a44ff9640 1 -- 172.21.15.204:0/4013504286 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2a38014e90 con 0x7f2a480a2400 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a44ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a44ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a44ff9640 10 monclient: dump: 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:11.601 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a4ecd4640 10 monclient: _renew_subs 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a4ecd4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a46ffd640 4 set_mon_vals no callback set 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.568+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/4013504286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4811f8b0 con 0x7f2a480a2400 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.569+0000 7f2a44ff9640 1 -- 172.21.15.204:0/4013504286 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2a38003020 con 0x7f2a480a2400 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.569+0000 7f2a4ecd4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:11.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.569+0000 7f2a4ecd4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:11.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.569+0000 7f2a4ecd4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:11.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.569+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/4013504286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a48070b80 con 0x7f2a480a2400 2024-09-15T01:35:11.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.569+0000 7f2a44ff9640 1 --2- 172.21.15.204:0/4013504286 >> v2:172.21.15.204:6812/28210 conn(0x7f2a280397b0 0x7f2a2803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:11.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.569+0000 7f2a44ff9640 1 -- 172.21.15.204:0/4013504286 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5283+0+0 (secure 0 0 0) 0x7f2a3804bcc0 con 0x7f2a480a2400 2024-09-15T01:35:11.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.569+0000 7f2a4ca70640 1 --2- 172.21.15.204:0/4013504286 >> v2:172.21.15.204:6812/28210 conn(0x7f2a280397b0 0x7f2a2803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:11.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.569+0000 7f2a4ca70640 10 monclient: get_auth_request con 0x7f2a280397b0 auth_method 0 2024-09-15T01:35:11.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.569+0000 7f2a4ca70640 1 --2- 172.21.15.204:0/4013504286 >> v2:172.21.15.204:6812/28210 conn(0x7f2a280397b0 0x7f2a2803bc60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2a40003850 tx=0x7f2a400090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:11.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.573+0000 7f2a44ff9640 1 -- 172.21.15.204:0/4013504286 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2a38052080 con 0x7f2a480a2400 2024-09-15T01:35:11.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.574+0000 7f2a44ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:11.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.574+0000 7f2a44ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:11.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.741+0000 7f2a4ecd4640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T01:35:11.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.741+0000 7f2a4ecd4640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-09-15T01:35:11.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.741+0000 7f2a4ecd4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:11.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:11.741+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/4013504286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7f2a4809e480 con 0x7f2a480a2400 2024-09-15T01:35:12.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:12.482+0000 7f2a44ff9640 1 -- 172.21.15.204:0/4013504286 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v109) v1 ==== 58+0+3 (secure 0 0 0) 0x7f2a3801b960 con 0x7f2a480a2400 2024-09-15T01:35:12.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:12.482+0000 7f2a44ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-09-15T01:35:12.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:12.482+0000 7f2a44ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:12.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:12.589+0000 7f2a4ecd4640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-09-15T01:35:12.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:12.589+0000 7f2a4ecd4640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-09-15T01:35:12.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:12.589+0000 7f2a4ecd4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:12.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:12.589+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/4013504286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7f2a480008d0 con 0x7f2a480a2400 2024-09-15T01:35:13.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.482+0000 7f2a44ff9640 1 -- 172.21.15.204:0/4013504286 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v110) v1 ==== 58+0+3 (secure 0 0 0) 0x7f2a3802b020 con 0x7f2a480a2400 2024-09-15T01:35:13.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.482+0000 7f2a44ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-09-15T01:35:13.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.482+0000 7f2a44ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:13.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.485+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/4013504286 >> v2:172.21.15.204:6812/28210 conn(0x7f2a280397b0 msgr2=0x7f2a2803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:13.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.485+0000 7f2a4ecd4640 1 --2- 172.21.15.204:0/4013504286 >> v2:172.21.15.204:6812/28210 conn(0x7f2a280397b0 0x7f2a2803bc60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2a40003850 tx=0x7f2a400090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:13.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.485+0000 7f2a4ecd4640 10 monclient: shutdown 2024-09-15T01:35:13.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.485+0000 7f2a4ecd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:13.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.485+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/4013504286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a480a2400 msgr2=0x7f2a48142500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:13.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.485+0000 7f2a4ecd4640 1 --2- 172.21.15.204:0/4013504286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a480a2400 0x7f2a48142500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800e190 tx=0x7f2a38009920 comp rx=0 tx=0).stop 2024-09-15T01:35:13.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.485+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/4013504286 shutdown_connections 2024-09-15T01:35:13.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.485+0000 7f2a4ecd4640 1 --2- 172.21.15.204:0/4013504286 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2a48132930 0x7f2a4811d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:13.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.485+0000 7f2a4ecd4640 1 --2- 172.21.15.204:0/4013504286 >> v2:172.21.15.204:6812/28210 conn(0x7f2a280397b0 0x7f2a2803bc60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:13.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.485+0000 7f2a4ecd4640 1 --2- 172.21.15.204:0/4013504286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a480a2400 0x7f2a48142500 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:13.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.485+0000 7f2a4ecd4640 1 --2- 172.21.15.204:0/4013504286 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2a4809f990 0x7f2a48141fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:13.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.485+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/4013504286 >> 172.21.15.204:0/4013504286 conn(0x7f2a4806e860 msgr2=0x7f2a48130f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:13.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.486+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/4013504286 shutdown_connections 2024-09-15T01:35:13.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.486+0000 7f2a4ecd4640 1 -- 172.21.15.204:0/4013504286 wait complete. 2024-09-15T01:35:13.500 INFO:tasks.workunit.client.0.smithi204.stderr:+ o5=14 2024-09-15T01:35:13.500 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add 12 123 root=default host=foobaz 2024-09-15T01:35:13.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.579+0000 7fa1d846f640 10 monclient: get_monmap_and_config 2024-09-15T01:35:13.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.579+0000 7fa1d846f640 10 monclient: build_initial_monmap 2024-09-15T01:35:13.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.579+0000 7fa1d846f640 10 monclient: monmap: 2024-09-15T01:35:13.582 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:13.582 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:13.582 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:13.581195+0000 2024-09-15T01:35:13.582 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:13.581195+0000 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:13.583 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:13.584 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:13.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 1 Processor -- start 2024-09-15T01:35:13.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 1 -- start start 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 10 monclient: init 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 10 monclient: _add_conns ranks=[7,5,1,4,9,10,3,0,2,6,8] 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa1d006e890 0x7fa1d0132cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 10 monclient(hunting): picked mon.noname-i con 0x7fa1d006e890 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa1d009fb10 0x7fa1d009ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 10 monclient(hunting): picked mon.noname-h con 0x7fa1d009fb10 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa1d00a09c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d00648a0 con 0x7fa1d00a09c0 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d00a1d30 con 0x7fa1d009fb10 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:13.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d006ec60 con 0x7fa1d006e890 2024-09-15T01:35:13.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:13.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.580+0000 7fa1d846f640 10 monclient(hunting): authenticate will time out at 929.223994s 2024-09-15T01:35:13.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d620b640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa1d006e890 msgr2=0x7fa1d0132cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:13.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d620b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa1d006e890 0x7fa1d0132cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:13.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d6a0c640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa1d00a09c0 0x7fa1d00a0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:55930/0 (socket says 172.21.15.204:55930) 2024-09-15T01:35:13.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d6a0c640 1 -- 172.21.15.204:0/262577557 learned_addr learned my addr 172.21.15.204:0/262577557 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:13.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 1 --2- 172.21.15.204:0/262577557 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa1d009fb10 0x7fa1d009ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:13.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 10 monclient(hunting): get_auth_request con 0x7fa1d009fb10 auth_method 0 2024-09-15T01:35:13.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:13.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:13.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:13.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:13.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:13.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:13.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5209640 1 -- 172.21.15.204:0/262577557 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916160353 0 0) 0x7fa1d00648a0 con 0x7fa1d00a09c0 2024-09-15T01:35:13.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5209640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:13.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5209640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:13.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5209640 10 monclient(hunting): my global_id is 6013 2024-09-15T01:35:13.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5209640 1 -- 172.21.15.204:0/262577557 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1bc0034e0 con 0x7fa1d00a09c0 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 10 monclient(hunting): handle_auth_done global_id 5585 payload 274 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 1 -- 172.21.15.204:0/262577557 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa1d006e890 msgr2=0x7fa1d0132cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 1 --2- 172.21.15.204:0/262577557 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa1d006e890 0x7fa1d0132cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 1 -- 172.21.15.204:0/262577557 >> v1:172.21.15.204:6789/0 conn(0x7fa1d00a09c0 legacy=0x7fa1d00a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 10 monclient: _finish_hunting 0 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 1 monclient: found mon.noname-h 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.581+0000 7fa1d5a0a640 1 -- 172.21.15.204:0/262577557 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d009e020 con 0x7fa1d009fb10 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5a0a640 1 --2- 172.21.15.204:0/262577557 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa1d009fb10 0x7fa1d009ff20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009880 tx=0x7fa1c00090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=45cb9882dbe62b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5209640 1 -- 172.21.15.204:0/262577557 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1c002ac80 con 0x7fa1d009fb10 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5209640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:13.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5209640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5209640 10 monclient: dump: 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:13.596 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5209640 10 monclient: _finish_auth 0 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5209640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5209640 1 -- 172.21.15.204:0/262577557 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c002ae00 con 0x7fa1d009fb10 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 5 monclient: authenticate success, global_id 5585 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5209640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5209640 1 -- 172.21.15.204:0/262577557 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1c0031ca0 con 0x7fa1d009fb10 2024-09-15T01:35:13.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5209640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 4 set_mon_vals no callback set 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5209640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d5209640 10 monclient: dump: 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:13.598 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 10 monclient: shutdown 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 1 -- 172.21.15.204:0/262577557 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa1d009fb10 msgr2=0x7fa1d009ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 1 --2- 172.21.15.204:0/262577557 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa1d009fb10 0x7fa1d009ff20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009880 tx=0x7fa1c00090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 1 -- 172.21.15.204:0/262577557 shutdown_connections 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 1 --2- 172.21.15.204:0/262577557 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa1d009fb10 0x7fa1d009ff20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 1 --2- 172.21.15.204:0/262577557 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa1d006e890 0x7fa1d0132cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 1 -- 172.21.15.204:0/262577557 >> 172.21.15.204:0/262577557 conn(0x7fa1d00a5fd0 msgr2=0x7fa1d00a63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 1 -- 172.21.15.204:0/262577557 shutdown_connections 2024-09-15T01:35:13.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.582+0000 7fa1d846f640 1 -- 172.21.15.204:0/262577557 wait complete. 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.583+0000 7fa1d846f640 10 monclient: build_initial_monmap 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.583+0000 7fa1d846f640 10 monclient: monmap: 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:13.600 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.583+0000 7fa1d846f640 1 Processor -- start 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.583+0000 7fa1d846f640 1 -- start start 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.583+0000 7fa1d846f640 10 monclient: init 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient: _add_conns ranks=[6,1,4,8,2,0,7,3,5] 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa1d006e890 0x7fa1d011cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient(hunting): picked mon.noname-d con 0x7fa1d006e890 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1d009fb10 0x7fa1d009b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:13.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient(hunting): picked mon.noname-f con 0x7fa1d009fb10 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa1d00a09c0 0x7fa1d0131a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient(hunting): picked mon.noname-c con 0x7fa1d00a09c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d011f0e0 con 0x7fa1d009fb10 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d011d720 con 0x7fa1d00a09c0 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d011ee00 con 0x7fa1d006e890 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient(hunting): authenticate will time out at 929.227224s 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1d009fb10 0x7fa1d009b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1d009fb10 0x7fa1d009b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48464/0 (socket says 172.21.15.204:48464) 2024-09-15T01:35:13.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 1 -- 172.21.15.204:0/4251363058 learned_addr learned my addr 172.21.15.204:0/4251363058 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:13.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 10 monclient(hunting): get_auth_request con 0x7fa1d009fb10 auth_method 0 2024-09-15T01:35:13.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:13.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:13.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:13.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d620b640 1 -- 172.21.15.204:0/4251363058 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa1d006e890 msgr2=0x7fa1d011cb00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:13.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d620b640 1 --2- 172.21.15.204:0/4251363058 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa1d006e890 0x7fa1d011cb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:13.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:13.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:13.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:13.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 10 monclient(hunting): handle_auth_done global_id 6022 payload 824 2024-09-15T01:35:13.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 1 -- 172.21.15.204:0/4251363058 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa1d006e890 msgr2=0x7fa1d011cb00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:13.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 1 --2- 172.21.15.204:0/4251363058 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa1d006e890 0x7fa1d011cb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:13.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 1 -- 172.21.15.204:0/4251363058 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa1d00a09c0 msgr2=0x7fa1d0131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:13.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 1 --2- 172.21.15.204:0/4251363058 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa1d00a09c0 0x7fa1d0131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:13.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 10 monclient: _finish_hunting 0 2024-09-15T01:35:13.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 1 monclient: found mon.noname-f 2024-09-15T01:35:13.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:13.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:13.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 1 -- 172.21.15.204:0/4251363058 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c0009390 con 0x7fa1d009fb10 2024-09-15T01:35:13.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d5a0a640 1 --2- 172.21.15.204:0/4251363058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1d009fb10 0x7fa1d009b100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0000c00 tx=0x7fa1c0032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:13.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1c67fc640 1 -- 172.21.15.204:0/4251363058 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1c0031820 con 0x7fa1d009fb10 2024-09-15T01:35:13.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1c67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:13.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1c67fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:13.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1c67fc640 10 monclient: dump: 2024-09-15T01:35:13.605 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:13.605 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:13.605 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:13.605 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:13.605 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:13.605 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:13.605 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:13.605 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:13.605 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:13.605 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1c67fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1c67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1c67fc640 1 -- 172.21.15.204:0/4251363058 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c00319c0 con 0x7fa1d009fb10 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 5 monclient: authenticate success, global_id 6022 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient: _renew_subs 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:13.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 1 -- 172.21.15.204:0/4251363058 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d009cbf0 con 0x7fa1d009fb10 2024-09-15T01:35:13.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1c67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:13.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1c67fc640 1 -- 172.21.15.204:0/4251363058 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1c003a390 con 0x7fa1d009fb10 2024-09-15T01:35:13.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1c67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:13.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1c67fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:13.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1c67fc640 10 monclient: dump: 2024-09-15T01:35:13.607 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:13.607 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:13.607 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:13.607 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d4a08640 4 set_mon_vals no callback set 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient: _renew_subs 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.584+0000 7fa1d846f640 1 -- 172.21.15.204:0/4251363058 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d011de40 con 0x7fa1d009fb10 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.585+0000 7fa1c67fc640 1 -- 172.21.15.204:0/4251363058 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa1c0039040 con 0x7fa1d009fb10 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.585+0000 7fa1c67fc640 1 --2- 172.21.15.204:0/4251363058 >> v2:172.21.15.204:6812/28210 conn(0x7fa1b40397b0 0x7fa1b403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.585+0000 7fa1c67fc640 1 -- 172.21.15.204:0/4251363058 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fa1c006ad90 con 0x7fa1d009fb10 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.585+0000 7fa1d846f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.585+0000 7fa1d846f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.585+0000 7fa1d846f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:13.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.585+0000 7fa1d846f640 1 -- 172.21.15.204:0/4251363058 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d009bc50 con 0x7fa1d009fb10 2024-09-15T01:35:13.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.586+0000 7fa1d620b640 1 --2- 172.21.15.204:0/4251363058 >> v2:172.21.15.204:6812/28210 conn(0x7fa1b40397b0 0x7fa1b403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:13.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.586+0000 7fa1d620b640 10 monclient: get_auth_request con 0x7fa1b40397b0 auth_method 0 2024-09-15T01:35:13.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.586+0000 7fa1d620b640 1 --2- 172.21.15.204:0/4251363058 >> v2:172.21.15.204:6812/28210 conn(0x7fa1b40397b0 0x7fa1b403bc60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc006fd0 tx=0x7fa1cc006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:13.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.590+0000 7fa1c67fc640 1 -- 172.21.15.204:0/4251363058 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa1c0041080 con 0x7fa1d009fb10 2024-09-15T01:35:13.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.590+0000 7fa1c67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:13.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.590+0000 7fa1c67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:13.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.751+0000 7fa1d846f640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T01:35:13.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.751+0000 7fa1d846f640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T01:35:13.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.751+0000 7fa1d846f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:13.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:13.751+0000 7fa1d846f640 1 -- 172.21.15.204:0/4251363058 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) v1 -- 0x7fa1d0133370 con 0x7fa1d009fb10 2024-09-15T01:35:14.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.493+0000 7fa1c67fc640 1 -- 172.21.15.204:0/4251363058 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 add item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default} to crush map v111) v1 ==== 220+0+0 (secure 0 0 0) 0x7fa1c0037020 con 0x7fa1d009fb10 2024-09-15T01:35:14.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.493+0000 7fa1c67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T01:35:14.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.493+0000 7fa1c67fc640 10 monclient: _finish_command 2 = system:0 add item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default} to crush map 2024-09-15T01:35:14.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.594+0000 7fa1d846f640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T01:35:14.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.594+0000 7fa1d846f640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T01:35:14.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.594+0000 7fa1d846f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:14.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.594+0000 7fa1d846f640 1 -- 172.21.15.204:0/4251363058 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) v1 -- 0x7fa1d00008d0 con 0x7fa1d009fb10 2024-09-15T01:35:14.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.594+0000 7fa1c67fc640 1 -- 172.21.15.204:0/4251363058 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 set item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default}: no change v111) v1 ==== 218+0+0 (secure 0 0 0) 0x7fa1c002fa80 con 0x7fa1d009fb10 2024-09-15T01:35:14.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.594+0000 7fa1c67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T01:35:14.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.594+0000 7fa1c67fc640 10 monclient: _finish_command 3 = system:0 set item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default}: no change 2024-09-15T01:35:14.597 INFO:tasks.workunit.client.0.smithi204.stderr:set item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default}: no change 2024-09-15T01:35:14.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 1 -- 172.21.15.204:0/4251363058 >> v2:172.21.15.204:6812/28210 conn(0x7fa1b40397b0 msgr2=0x7fa1b403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 1 --2- 172.21.15.204:0/4251363058 >> v2:172.21.15.204:6812/28210 conn(0x7fa1b40397b0 0x7fa1b403bc60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc006fd0 tx=0x7fa1cc006d30 comp rx=0 tx=0).stop 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 10 monclient: shutdown 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 1 -- 172.21.15.204:0/4251363058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1d009fb10 msgr2=0x7fa1d009b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 1 --2- 172.21.15.204:0/4251363058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1d009fb10 0x7fa1d009b100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0000c00 tx=0x7fa1c0032000 comp rx=0 tx=0).stop 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 1 -- 172.21.15.204:0/4251363058 shutdown_connections 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 1 --2- 172.21.15.204:0/4251363058 >> v2:172.21.15.204:6812/28210 conn(0x7fa1b40397b0 0x7fa1b403bc60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 1 --2- 172.21.15.204:0/4251363058 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa1d00a09c0 0x7fa1d0131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 1 --2- 172.21.15.204:0/4251363058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1d009fb10 0x7fa1d009b100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 1 --2- 172.21.15.204:0/4251363058 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa1d006e890 0x7fa1d011cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 1 -- 172.21.15.204:0/4251363058 >> 172.21.15.204:0/4251363058 conn(0x7fa1d00a5fd0 msgr2=0x7fa1d0135600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 1 -- 172.21.15.204:0/4251363058 shutdown_connections 2024-09-15T01:35:14.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.596+0000 7fa1d846f640 1 -- 172.21.15.204:0/4251363058 wait complete. 2024-09-15T01:35:14.607 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add 14 123 root=default host=foobaz 2024-09-15T01:35:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.681+0000 7fc946c60640 10 monclient: get_monmap_and_config 2024-09-15T01:35:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.681+0000 7fc946c60640 10 monclient: build_initial_monmap 2024-09-15T01:35:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.681+0000 7fc946c60640 10 monclient: monmap: 2024-09-15T01:35:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:14.682991+0000 2024-09-15T01:35:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:14.682991+0000 2024-09-15T01:35:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:14.685 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 1 Processor -- start 2024-09-15T01:35:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 1 -- start start 2024-09-15T01:35:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 10 monclient: init 2024-09-15T01:35:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 10 monclient: _add_conns ranks=[1,3,6,9,5,8,0,2,7,10,4] 2024-09-15T01:35:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc9400a2cd0 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 0x7fc94009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 10 monclient(hunting): picked mon.noname-g con 0x7fc94009fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc940137c80 0x7fc94013a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 10 monclient(hunting): picked mon.noname-d con 0x7fc940137c80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9400648a0 con 0x7fc9400a2cd0 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc940133310 con 0x7fc94009fa20 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400a30a0 con 0x7fc940137c80 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc946c60640 10 monclient(hunting): authenticate will time out at 930.325791s 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc93ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 0x7fc94009fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc9449fc640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc9400a2cd0 0x7fc940132c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:55942/0 (socket says 172.21.15.204:55942) 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc9449fc640 1 -- 172.21.15.204:0/1190941097 learned_addr learned my addr 172.21.15.204:0/1190941097 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc93ffff640 10 monclient(hunting): get_auth_request con 0x7fc94009fa20 auth_method 0 2024-09-15T01:35:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc93ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc93ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc93ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc9451fd640 1 -- 172.21.15.204:0/1190941097 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc940137c80 msgr2=0x7fc94013a060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.682+0000 7fc9451fd640 1 --2- 172.21.15.204:0/1190941097 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc940137c80 0x7fc94013a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 10 monclient(hunting): handle_auth_done global_id 14637 payload 274 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93f7fe640 1 -- 172.21.15.204:0/1190941097 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857399608 0 0) 0x7fc9400648a0 con 0x7fc9400a2cd0 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 1 -- 172.21.15.204:0/1190941097 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc940137c80 msgr2=0x7fc94013a060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 1 --2- 172.21.15.204:0/1190941097 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc940137c80 0x7fc94013a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 1 -- 172.21.15.204:0/1190941097 >> v1:172.21.15.204:6789/0 conn(0x7fc9400a2cd0 legacy=0x7fc940132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 1 monclient: found mon.noname-g 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 1 -- 172.21.15.204:0/1190941097 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9400a0330 con 0x7fc94009fa20 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93f7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.683+0000 7fc93ffff640 1 --2- 172.21.15.204:0/1190941097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 0x7fc94009fdf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009880 tx=0x7fc92c0090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=4c2ca374dc8d738e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc93f7fe640 1 -- 172.21.15.204:0/1190941097 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc92c02ac80 con 0x7fc94009fa20 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc93f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc93f7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc93f7fe640 10 monclient: dump: 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:14.698 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc93f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc93f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc93f7fe640 1 -- 172.21.15.204:0/1190941097 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c02ae00 con 0x7fc94009fa20 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 5 monclient: authenticate success, global_id 14637 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc93f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc93f7fe640 1 -- 172.21.15.204:0/1190941097 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc92c031ca0 con 0x7fc94009fa20 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc93f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc93f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc93f7fe640 10 monclient: dump: 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:14.700 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 4 set_mon_vals no callback set 2024-09-15T01:35:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 10 monclient: shutdown 2024-09-15T01:35:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 1 -- 172.21.15.204:0/1190941097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 msgr2=0x7fc94009fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 1 --2- 172.21.15.204:0/1190941097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 0x7fc94009fdf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009880 tx=0x7fc92c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 1 -- 172.21.15.204:0/1190941097 shutdown_connections 2024-09-15T01:35:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 1 --2- 172.21.15.204:0/1190941097 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc940137c80 0x7fc94013a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 1 --2- 172.21.15.204:0/1190941097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 0x7fc94009fdf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 1 -- 172.21.15.204:0/1190941097 >> 172.21.15.204:0/1190941097 conn(0x7fc94006e860 msgr2=0x7fc94006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 1 -- 172.21.15.204:0/1190941097 shutdown_connections 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.684+0000 7fc946c60640 1 -- 172.21.15.204:0/1190941097 wait complete. 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.685+0000 7fc946c60640 10 monclient: build_initial_monmap 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.685+0000 7fc946c60640 10 monclient: monmap: 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.685+0000 7fc946c60640 1 Processor -- start 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.685+0000 7fc946c60640 1 -- start start 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.685+0000 7fc946c60640 10 monclient: init 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient: _add_conns ranks=[3,5,4,7,1,0,6,8,2] 2024-09-15T01:35:14.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 0x7fc940134350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient(hunting): picked mon.noname-g con 0x7fc94009fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc9400a2cd0 0x7fc940134890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient(hunting): picked mon.noname-h con 0x7fc9400a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc940137c80 0x7fc940134dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient(hunting): picked mon.noname-c con 0x7fc940137c80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94009c690 con 0x7fc94009fa20 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc94009cb60 con 0x7fc940137c80 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc940136540 con 0x7fc9400a2cd0 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient(hunting): authenticate will time out at 930.329225s 2024-09-15T01:35:14.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 0x7fc940134350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:14.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 0x7fc940134350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:42720/0 (socket says 172.21.15.204:42720) 2024-09-15T01:35:14.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 1 -- 172.21.15.204:0/3250309747 learned_addr learned my addr 172.21.15.204:0/3250309747 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:14.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 10 monclient(hunting): get_auth_request con 0x7fc94009fa20 auth_method 0 2024-09-15T01:35:14.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:14.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:14.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:14.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93ffff640 1 --2- 172.21.15.204:0/3250309747 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc9400a2cd0 0x7fc940134890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:14.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93ffff640 10 monclient(hunting): get_auth_request con 0x7fc9400a2cd0 auth_method 0 2024-09-15T01:35:14.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:14.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:14.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:14.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 10 monclient(hunting): handle_auth_done global_id 14646 payload 824 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 1 -- 172.21.15.204:0/3250309747 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc9400a2cd0 msgr2=0x7fc940134890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 1 --2- 172.21.15.204:0/3250309747 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc9400a2cd0 0x7fc940134890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 1 -- 172.21.15.204:0/3250309747 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc940137c80 msgr2=0x7fc940134dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 1 --2- 172.21.15.204:0/3250309747 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc940137c80 0x7fc940134dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 10 monclient: _finish_hunting 0 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 1 monclient: found mon.noname-g 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:14.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 1 -- 172.21.15.204:0/3250309747 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c009390 con 0x7fc94009fa20 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc9449fc640 1 --2- 172.21.15.204:0/3250309747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 0x7fc940134350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c5e0 tx=0x7fc934007d00 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93cff9640 1 -- 172.21.15.204:0/3250309747 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc9340070f0 con 0x7fc94009fa20 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93cff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93cff9640 10 monclient: dump: 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:14.707 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:14.708 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:14.708 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:14.708 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:14.708 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:14.708 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:14.708 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:14.708 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:14.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93cff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:14.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:14.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 5 monclient: authenticate success, global_id 14646 2024-09-15T01:35:14.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93cff9640 1 -- 172.21.15.204:0/3250309747 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc934007290 con 0x7fc94009fa20 2024-09-15T01:35:14.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient: _renew_subs 2024-09-15T01:35:14.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:14.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 1 -- 172.21.15.204:0/3250309747 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94009d3d0 con 0x7fc94009fa20 2024-09-15T01:35:14.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:14.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93cff9640 1 -- 172.21.15.204:0/3250309747 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc934007480 con 0x7fc94009fa20 2024-09-15T01:35:14.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:14.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93cff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:14.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93cff9640 10 monclient: dump: 2024-09-15T01:35:14.709 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:14.709 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:14.709 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:14.710 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:14.710 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:14.710 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:14.710 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:14.710 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:14.710 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:14.710 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:14.710 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:14.710 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:14.710 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:14.710 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:14.710 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc93effd640 4 set_mon_vals no callback set 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient: _renew_subs 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.686+0000 7fc946c60640 1 -- 172.21.15.204:0/3250309747 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc940135d80 con 0x7fc94009fa20 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.687+0000 7fc93cff9640 1 -- 172.21.15.204:0/3250309747 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc9340075e0 con 0x7fc94009fa20 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.687+0000 7fc946c60640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.687+0000 7fc946c60640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.687+0000 7fc946c60640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.687+0000 7fc946c60640 1 -- 172.21.15.204:0/3250309747 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc940070b80 con 0x7fc94009fa20 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.687+0000 7fc93cff9640 1 --2- 172.21.15.204:0/3250309747 >> v2:172.21.15.204:6812/28210 conn(0x7fc9240397b0 0x7fc92403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.688+0000 7fc93cff9640 1 -- 172.21.15.204:0/3250309747 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc93404b1d0 con 0x7fc94009fa20 2024-09-15T01:35:14.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.688+0000 7fc93ffff640 1 --2- 172.21.15.204:0/3250309747 >> v2:172.21.15.204:6812/28210 conn(0x7fc9240397b0 0x7fc92403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:14.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.688+0000 7fc93ffff640 10 monclient: get_auth_request con 0x7fc9240397b0 auth_method 0 2024-09-15T01:35:14.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.688+0000 7fc93ffff640 1 --2- 172.21.15.204:0/3250309747 >> v2:172.21.15.204:6812/28210 conn(0x7fc9240397b0 0x7fc92403bc60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc92c034000 tx=0x7fc92c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:14.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.692+0000 7fc93cff9640 1 -- 172.21.15.204:0/3250309747 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc934005aa0 con 0x7fc94009fa20 2024-09-15T01:35:14.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.692+0000 7fc93cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:14.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.692+0000 7fc93cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:14.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.852+0000 7fc946c60640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T01:35:14.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.852+0000 7fc946c60640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T01:35:14.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.852+0000 7fc946c60640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:14.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:14.852+0000 7fc946c60640 1 -- 172.21.15.204:0/3250309747 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) v1 -- 0x7fc940067450 con 0x7fc94009fa20 2024-09-15T01:35:15.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.502+0000 7fc93cff9640 1 -- 172.21.15.204:0/3250309747 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 add item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default} to crush map v112) v1 ==== 220+0+0 (secure 0 0 0) 0x7fc934010af0 con 0x7fc94009fa20 2024-09-15T01:35:15.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.502+0000 7fc93cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T01:35:15.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.502+0000 7fc93cff9640 10 monclient: _finish_command 2 = system:0 add item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default} to crush map 2024-09-15T01:35:15.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.605+0000 7fc946c60640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T01:35:15.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.605+0000 7fc946c60640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T01:35:15.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.605+0000 7fc946c60640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:15.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.605+0000 7fc946c60640 1 -- 172.21.15.204:0/3250309747 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) v1 -- 0x7fc940136950 con 0x7fc94009fa20 2024-09-15T01:35:15.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.606+0000 7fc93cff9640 1 -- 172.21.15.204:0/3250309747 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 set item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default}: no change v112) v1 ==== 218+0+0 (secure 0 0 0) 0x7fc93401a360 con 0x7fc94009fa20 2024-09-15T01:35:15.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.606+0000 7fc93cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-09-15T01:35:15.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.606+0000 7fc93cff9640 10 monclient: _finish_command 3 = system:0 set item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default}: no change 2024-09-15T01:35:15.608 INFO:tasks.workunit.client.0.smithi204.stderr:set item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default}: no change 2024-09-15T01:35:15.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 1 -- 172.21.15.204:0/3250309747 >> v2:172.21.15.204:6812/28210 conn(0x7fc9240397b0 msgr2=0x7fc92403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:15.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 1 --2- 172.21.15.204:0/3250309747 >> v2:172.21.15.204:6812/28210 conn(0x7fc9240397b0 0x7fc92403bc60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc92c034000 tx=0x7fc92c032000 comp rx=0 tx=0).stop 2024-09-15T01:35:15.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 10 monclient: shutdown 2024-09-15T01:35:15.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:15.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 1 -- 172.21.15.204:0/3250309747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 msgr2=0x7fc940134350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:15.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 1 --2- 172.21.15.204:0/3250309747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 0x7fc940134350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c5e0 tx=0x7fc934007d00 comp rx=0 tx=0).stop 2024-09-15T01:35:15.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 1 -- 172.21.15.204:0/3250309747 shutdown_connections 2024-09-15T01:35:15.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 1 --2- 172.21.15.204:0/3250309747 >> v2:172.21.15.204:6812/28210 conn(0x7fc9240397b0 0x7fc92403bc60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 1 --2- 172.21.15.204:0/3250309747 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc9400a2cd0 0x7fc940134890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 1 --2- 172.21.15.204:0/3250309747 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc940137c80 0x7fc940134dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 1 --2- 172.21.15.204:0/3250309747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc94009fa20 0x7fc940134350 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 1 -- 172.21.15.204:0/3250309747 >> 172.21.15.204:0/3250309747 conn(0x7fc94006e860 msgr2=0x7fc94012db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:15.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 1 -- 172.21.15.204:0/3250309747 shutdown_connections 2024-09-15T01:35:15.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.608+0000 7fc946c60640 1 -- 172.21.15.204:0/3250309747 wait complete. 2024-09-15T01:35:15.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:35:15.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep osd.12 2024-09-15T01:35:15.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 123 2024-09-15T01:35:15.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.690+0000 7f4c65e2b640 10 monclient: get_monmap_and_config 2024-09-15T01:35:15.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.690+0000 7f4c65e2b640 10 monclient: build_initial_monmap 2024-09-15T01:35:15.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.690+0000 7f4c65e2b640 10 monclient: monmap: 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:15.691604+0000 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:15.691604+0000 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:15.693 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:15.694 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:15.694 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:15.694 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:15.694 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.690+0000 7f4c65e2b640 1 Processor -- start 2024-09-15T01:35:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.690+0000 7f4c65e2b640 1 -- start start 2024-09-15T01:35:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.690+0000 7f4c65e2b640 10 monclient: init 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 10 monclient: _add_conns ranks=[10,6,4,5,8,7,3,9,1,2,0] 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4c6006ea10 0x7f4c6009c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 10 monclient(hunting): picked mon.noname-f con 0x7f4c6006ea10 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c60067130 0x7f4c60067540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 10 monclient(hunting): picked mon.noname-d con 0x7f4c60067130 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4c60067a80 0x7f4c601374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 10 monclient(hunting): picked mon.noname-c con 0x7f4c60067a80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6009cf30 con 0x7f4c6006ea10 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6009d130 con 0x7f4c60067a80 2024-09-15T01:35:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4c60096d80 con 0x7f4c60067130 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c65e2b640 10 monclient(hunting): authenticate will time out at 931.334498s 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5effd640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c60067130 msgr2=0x7f4c60067540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c60067130 0x7f4c60067540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4c6006ea10 0x7f4c6009c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4c6006ea10 0x7f4c6009c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48468/0 (socket says 172.21.15.204:48468) 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 1 -- 172.21.15.204:0/1275395404 learned_addr learned my addr 172.21.15.204:0/1275395404 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 10 monclient(hunting): get_auth_request con 0x7f4c6006ea10 auth_method 0 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5ffff640 1 --2- 172.21.15.204:0/1275395404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4c60067a80 0x7f4c601374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5ffff640 10 monclient(hunting): get_auth_request con 0x7f4c60067a80 auth_method 0 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 10 monclient(hunting): handle_auth_done global_id 6040 payload 274 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 1 -- 172.21.15.204:0/1275395404 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c60067130 msgr2=0x7f4c60067540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 1 --2- 172.21.15.204:0/1275395404 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c60067130 0x7f4c60067540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 1 -- 172.21.15.204:0/1275395404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4c60067a80 msgr2=0x7f4c601374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 1 --2- 172.21.15.204:0/1275395404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4c60067a80 0x7f4c601374c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 1 monclient: found mon.noname-f 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.691+0000 7f4c5f7fe640 1 -- 172.21.15.204:0/1275395404 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600648a0 con 0x7f4c6006ea10 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5f7fe640 1 --2- 172.21.15.204:0/1275395404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c6006ea10 0x7f4c6009c3a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009f20 tx=0x7f4c50009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4301237af3b7727 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5e7fc640 1 -- 172.21.15.204:0/1275395404 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c50007880 con 0x7f4c6006ea10 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5e7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5e7fc640 10 monclient: dump: 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5e7fc640 1 -- 172.21.15.204:0/1275395404 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5002d050 con 0x7f4c6006ea10 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c65e2b640 5 monclient: authenticate success, global_id 6040 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c65e2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5e7fc640 1 -- 172.21.15.204:0/1275395404 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c50030360 con 0x7f4c6006ea10 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c65e2b640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c65e2b640 4 set_mon_vals no callback set 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.692+0000 7f4c5e7fc640 10 monclient: dump: 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 10 monclient: shutdown 2024-09-15T01:35:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1275395404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c6006ea10 msgr2=0x7f4c6009c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 1 --2- 172.21.15.204:0/1275395404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c6006ea10 0x7f4c6009c3a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009f20 tx=0x7f4c50009840 comp rx=0 tx=0).stop 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1275395404 shutdown_connections 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 1 --2- 172.21.15.204:0/1275395404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4c60067a80 0x7f4c601374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 1 --2- 172.21.15.204:0/1275395404 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c60067130 0x7f4c60067540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 1 --2- 172.21.15.204:0/1275395404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c6006ea10 0x7f4c6009c3a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1275395404 >> 172.21.15.204:0/1275395404 conn(0x7f4c60068b30 msgr2=0x7f4c600a2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1275395404 shutdown_connections 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1275395404 wait complete. 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 10 monclient: build_initial_monmap 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.693+0000 7f4c65e2b640 10 monclient: monmap: 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:15.711 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 1 Processor -- start 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 1 -- start start 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 10 monclient: init 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 10 monclient: _add_conns ranks=[6,4,3,1,0,5,2,7,8] 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c60067130 0x7f4c6013fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 10 monclient(hunting): picked mon.noname-d con 0x7f4c60067130 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4c60067a80 0x7f4c60133840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 10 monclient(hunting): picked mon.noname-c con 0x7f4c60067a80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4c6006ea10 0x7f4c6011d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 10 monclient(hunting): picked mon.noname-g con 0x7f4c6006ea10 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c60140960 con 0x7f4c6006ea10 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c60120960 con 0x7f4c60067a80 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6006ef00 con 0x7f4c60067130 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.694+0000 7f4c65e2b640 10 monclient(hunting): authenticate will time out at 931.337989s 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5f7fe640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c60067130 msgr2=0x7f4c6013fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5f7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c60067130 0x7f4c6013fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4c6006ea10 0x7f4c6011d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4c6006ea10 0x7f4c6011d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:42728/0 (socket says 172.21.15.204:42728) 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 1 -- 172.21.15.204:0/1833577442 learned_addr learned my addr 172.21.15.204:0/1833577442 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 10 monclient(hunting): get_auth_request con 0x7f4c6006ea10 auth_method 0 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5effd640 1 --2- 172.21.15.204:0/1833577442 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4c60067a80 0x7f4c60133840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5effd640 10 monclient(hunting): get_auth_request con 0x7f4c60067a80 auth_method 0 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_done global_id 14655 payload 824 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 1 -- 172.21.15.204:0/1833577442 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c60067130 msgr2=0x7f4c6013fa50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 1 --2- 172.21.15.204:0/1833577442 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c60067130 0x7f4c6013fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 1 -- 172.21.15.204:0/1833577442 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4c60067a80 msgr2=0x7f4c60133840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 1 --2- 172.21.15.204:0/1833577442 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4c60067a80 0x7f4c60133840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 1 monclient: found mon.noname-g 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 1 -- 172.21.15.204:0/1833577442 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c50009ae0 con 0x7f4c6006ea10 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5ffff640 1 --2- 172.21.15.204:0/1833577442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4c6006ea10 0x7f4c6011d2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800ea80 tx=0x7f4c580090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c477fe640 1 -- 172.21.15.204:0/1833577442 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c5800ce90 con 0x7f4c6006ea10 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c477fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c477fe640 10 monclient: dump: 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:15.716 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c477fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c65e2b640 5 monclient: authenticate success, global_id 14655 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c477fe640 1 -- 172.21.15.204:0/1833577442 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c58009620 con 0x7f4c6006ea10 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c65e2b640 10 monclient: _renew_subs 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c65e2b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1833577442 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6011f340 con 0x7f4c6006ea10 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c477fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:15.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c477fe640 1 -- 172.21.15.204:0/1833577442 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c580138f0 con 0x7f4c6006ea10 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c477fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c477fe640 10 monclient: dump: 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:15.718 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c65e2b640 10 monclient: _renew_subs 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c65e2b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1833577442 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6011fa00 con 0x7f4c6006ea10 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.695+0000 7f4c5dffb640 4 set_mon_vals no callback set 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.696+0000 7f4c477fe640 1 -- 172.21.15.204:0/1833577442 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4c58013470 con 0x7f4c6006ea10 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.696+0000 7f4c65e2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.696+0000 7f4c65e2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:15.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.696+0000 7f4c65e2b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:15.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.696+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1833577442 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c60070b80 con 0x7f4c6006ea10 2024-09-15T01:35:15.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.696+0000 7f4c477fe640 1 --2- 172.21.15.204:0/1833577442 >> v2:172.21.15.204:6812/28210 conn(0x7f4c4c0396e0 0x7f4c4c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:15.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.696+0000 7f4c477fe640 1 -- 172.21.15.204:0/1833577442 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5727+0+0 (secure 0 0 0) 0x7f4c5804bab0 con 0x7f4c6006ea10 2024-09-15T01:35:15.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.696+0000 7f4c5f7fe640 1 --2- 172.21.15.204:0/1833577442 >> v2:172.21.15.204:6812/28210 conn(0x7f4c4c0396e0 0x7f4c4c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:15.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.696+0000 7f4c5f7fe640 10 monclient: get_auth_request con 0x7f4c4c0396e0 auth_method 0 2024-09-15T01:35:15.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.697+0000 7f4c5f7fe640 1 --2- 172.21.15.204:0/1833577442 >> v2:172.21.15.204:6812/28210 conn(0x7f4c4c0396e0 0x7f4c4c03bb90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009ab0 tx=0x7f4c50036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:15.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.701+0000 7f4c477fe640 1 -- 172.21.15.204:0/1833577442 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4c58013720 con 0x7f4c6006ea10 2024-09-15T01:35:15.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.701+0000 7f4c477fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:15.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.701+0000 7f4c477fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:15.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.857+0000 7f4c65e2b640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:35:15.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.857+0000 7f4c65e2b640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:35:15.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.857+0000 7f4c65e2b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:15.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.857+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1833577442 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f4c6006f750 con 0x7f4c6006ea10 2024-09-15T01:35:15.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.858+0000 7f4c477fe640 1 -- 172.21.15.204:0/1833577442 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v112) v1 ==== 56+0+1449 (secure 0 0 0) 0x7f4c5801bb60 con 0x7f4c6006ea10 2024-09-15T01:35:15.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.858+0000 7f4c477fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:35:15.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.858+0000 7f4c477fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:15.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.949+0000 7f4c65e2b640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:35:15.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.949+0000 7f4c65e2b640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:35:15.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.949+0000 7f4c65e2b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:15.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.949+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1833577442 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f4c6011ee30 con 0x7f4c6006ea10 2024-09-15T01:35:15.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.950+0000 7f4c477fe640 1 -- 172.21.15.204:0/1833577442 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v112) v1 ==== 56+0+1449 (secure 0 0 0) 0x7f4c5801bd40 con 0x7f4c6006ea10 2024-09-15T01:35:15.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.950+0000 7f4c477fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:35:15.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.950+0000 7f4c477fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:15.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.951+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1833577442 >> v2:172.21.15.204:6812/28210 conn(0x7f4c4c0396e0 msgr2=0x7f4c4c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:15.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.951+0000 7f4c65e2b640 1 --2- 172.21.15.204:0/1833577442 >> v2:172.21.15.204:6812/28210 conn(0x7f4c4c0396e0 0x7f4c4c03bb90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009ab0 tx=0x7f4c50036000 comp rx=0 tx=0).stop 2024-09-15T01:35:15.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.951+0000 7f4c65e2b640 10 monclient: shutdown 2024-09-15T01:35:15.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.951+0000 7f4c65e2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:15.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.951+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1833577442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4c6006ea10 msgr2=0x7f4c6011d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:15.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.951+0000 7f4c65e2b640 1 --2- 172.21.15.204:0/1833577442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4c6006ea10 0x7f4c6011d2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800ea80 tx=0x7f4c580090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.951+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1833577442 shutdown_connections 2024-09-15T01:35:15.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.951+0000 7f4c65e2b640 1 --2- 172.21.15.204:0/1833577442 >> v2:172.21.15.204:6812/28210 conn(0x7f4c4c0396e0 0x7f4c4c03bb90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.951+0000 7f4c65e2b640 1 --2- 172.21.15.204:0/1833577442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4c6006ea10 0x7f4c6011d2f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.951+0000 7f4c65e2b640 1 --2- 172.21.15.204:0/1833577442 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4c60067a80 0x7f4c60133840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.951+0000 7f4c65e2b640 1 --2- 172.21.15.204:0/1833577442 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4c60067130 0x7f4c6013fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:15.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.951+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1833577442 >> 172.21.15.204:0/1833577442 conn(0x7f4c60068b30 msgr2=0x7f4c60097810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:15.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.952+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1833577442 shutdown_connections 2024-09-15T01:35:15.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:15.952+0000 7f4c65e2b640 1 -- 172.21.15.204:0/1833577442 wait complete. 2024-09-15T01:35:15.963 INFO:tasks.workunit.client.0.smithi204.stdout: 12 123.00000 osd.12 down 1.00000 1.00000 2024-09-15T01:35:15.964 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:35:15.964 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep osd.14 2024-09-15T01:35:15.964 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 123 2024-09-15T01:35:16.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.031+0000 7fc58759d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:16.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.031+0000 7fc58759d640 10 monclient: build_initial_monmap 2024-09-15T01:35:16.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.032+0000 7fc58759d640 10 monclient: monmap: 2024-09-15T01:35:16.034 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:16.033463+0000 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:16.033463+0000 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:16.035 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:16.036 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:16.036 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:16.036 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:16.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.032+0000 7fc58759d640 1 Processor -- start 2024-09-15T01:35:16.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.032+0000 7fc58759d640 1 -- start start 2024-09-15T01:35:16.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.032+0000 7fc58759d640 10 monclient: init 2024-09-15T01:35:16.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:16.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 10 monclient: _add_conns ranks=[8,10,1,4,3,5,9,7,6,2,0] 2024-09-15T01:35:16.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc5800a2400 0x7fc5800a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 10 monclient(hunting): picked mon.noname-e con 0x7fc5800a2400 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:16.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc58009f990 0x7fc58009fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 10 monclient(hunting): picked mon.noname-f con 0x7fc58009f990 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:16.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc580132930 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:16.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5800648a0 con 0x7fc580132930 2024-09-15T01:35:16.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58009e1a0 con 0x7fc58009f990 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc5800a27d0 con 0x7fc5800a2400 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc58759d640 10 monclient(hunting): authenticate will time out at 931.676307s 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc58009f990 0x7fc58009fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc58009f990 0x7fc58009fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48476/0 (socket says 172.21.15.204:48476) 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 1 -- 172.21.15.204:0/2967961197 learned_addr learned my addr 172.21.15.204:0/2967961197 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 10 monclient(hunting): get_auth_request con 0x7fc58009f990 auth_method 0 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:16.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc585b3a640 1 --1- 172.21.15.204:0/2967961197 >> v1:172.21.15.204:6789/0 conn(0x7fc580132930 0x7fc580132db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:55944/0 (socket says 172.21.15.204:55944) 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc585339640 1 --2- 172.21.15.204:0/2967961197 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc5800a2400 0x7fc5800a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc585339640 10 monclient(hunting): get_auth_request con 0x7fc5800a2400 auth_method 0 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc585339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc585339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc585339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 10 monclient(hunting): handle_auth_done global_id 6049 payload 274 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 1 -- 172.21.15.204:0/2967961197 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc5800a2400 msgr2=0x7fc5800a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 1 --2- 172.21.15.204:0/2967961197 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc5800a2400 0x7fc5800a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 1 -- 172.21.15.204:0/2967961197 >> v1:172.21.15.204:6789/0 conn(0x7fc580132930 legacy=0x7fc580132db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 10 monclient: _finish_hunting 0 2024-09-15T01:35:16.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 1 monclient: found mon.noname-f 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.033+0000 7fc584b38640 1 -- 172.21.15.204:0/2967961197 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58009dec0 con 0x7fc58009f990 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc584b38640 1 --2- 172.21.15.204:0/2967961197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc58009f990 0x7fc58009fd60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc570009910 tx=0x7fc5700091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e44a775418ba44b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc577fff640 1 -- 172.21.15.204:0/2967961197 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc57002ad20 con 0x7fc58009f990 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc577fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc577fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc577fff640 10 monclient: dump: 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:16.263 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc577fff640 10 monclient: _finish_auth 0 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc577fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc58759d640 5 monclient: authenticate success, global_id 6049 2024-09-15T01:35:16.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc58759d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:16.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc577fff640 1 -- 172.21.15.204:0/2967961197 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc570031820 con 0x7fc58009f990 2024-09-15T01:35:16.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc577fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:16.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc58759d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:16.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc58759d640 4 set_mon_vals no callback set 2024-09-15T01:35:16.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc577fff640 1 -- 172.21.15.204:0/2967961197 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc57002aea0 con 0x7fc58009f990 2024-09-15T01:35:16.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc577fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:16.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc577fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:16.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.034+0000 7fc577fff640 10 monclient: dump: 2024-09-15T01:35:16.265 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:16.265 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:16.265 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.265 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.266 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:16.266 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:16.266 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:16.266 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:16.266 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:16.266 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:16.266 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:16.266 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:16.266 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:16.266 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.035+0000 7fc58759d640 10 monclient: shutdown 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.035+0000 7fc58759d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.035+0000 7fc58759d640 1 -- 172.21.15.204:0/2967961197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc58009f990 msgr2=0x7fc58009fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.035+0000 7fc58759d640 1 --2- 172.21.15.204:0/2967961197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc58009f990 0x7fc58009fd60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc570009910 tx=0x7fc5700091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.035+0000 7fc58759d640 1 -- 172.21.15.204:0/2967961197 shutdown_connections 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.035+0000 7fc58759d640 1 --2- 172.21.15.204:0/2967961197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc58009f990 0x7fc58009fd60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.035+0000 7fc58759d640 1 --2- 172.21.15.204:0/2967961197 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc5800a2400 0x7fc5800a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.035+0000 7fc58759d640 1 -- 172.21.15.204:0/2967961197 >> 172.21.15.204:0/2967961197 conn(0x7fc58006e860 msgr2=0x7fc58006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.035+0000 7fc58759d640 1 -- 172.21.15.204:0/2967961197 shutdown_connections 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.035+0000 7fc58759d640 1 -- 172.21.15.204:0/2967961197 wait complete. 2024-09-15T01:35:16.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.035+0000 7fc58759d640 10 monclient: build_initial_monmap 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.035+0000 7fc58759d640 10 monclient: monmap: 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:16.268 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 1 Processor -- start 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 1 -- start start 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 10 monclient: init 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 10 monclient: _add_conns ranks=[6,8,3,2,0,5,1,7,4] 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc58009f990 0x7fc580141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 10 monclient(hunting): picked mon.noname-d con 0x7fc58009f990 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc5800a2400 0x7fc5801423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 10 monclient(hunting): picked mon.noname-e con 0x7fc5800a2400 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:16.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc580132930 0x7fc580142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 10 monclient(hunting): picked mon.noname-g con 0x7fc580132930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58009d3d0 con 0x7fc580132930 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc58009cad0 con 0x7fc58009f990 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc580122230 con 0x7fc5800a2400 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc58759d640 10 monclient(hunting): authenticate will time out at 931.679738s 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc585339640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc58009f990 msgr2=0x7fc580141e80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc585339640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc58009f990 0x7fc580141e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc585b3a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc580132930 0x7fc580142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc585b3a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc580132930 0x7fc580142900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:42740/0 (socket says 172.21.15.204:42740) 2024-09-15T01:35:16.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc585b3a640 1 -- 172.21.15.204:0/2024721209 learned_addr learned my addr 172.21.15.204:0/2024721209 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc585b3a640 10 monclient(hunting): get_auth_request con 0x7fc580132930 auth_method 0 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc585b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc585b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc585b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc585b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc585b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc584b38640 1 --2- 172.21.15.204:0/2024721209 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc5800a2400 0x7fc5801423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc585b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc584b38640 10 monclient(hunting): get_auth_request con 0x7fc5800a2400 auth_method 0 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc584b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc584b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.036+0000 7fc584b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:16.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc584b38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc584b38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc584b38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc585b3a640 10 monclient(hunting): handle_auth_done global_id 14664 payload 824 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc585b3a640 1 -- 172.21.15.204:0/2024721209 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc5800a2400 msgr2=0x7fc5801423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc585b3a640 1 --2- 172.21.15.204:0/2024721209 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc5800a2400 0x7fc5801423c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc585b3a640 1 -- 172.21.15.204:0/2024721209 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc58009f990 msgr2=0x7fc580141e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc585b3a640 1 --2- 172.21.15.204:0/2024721209 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc58009f990 0x7fc580141e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc585b3a640 10 monclient: _finish_hunting 0 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc585b3a640 1 monclient: found mon.noname-g 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc585b3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc585b3a640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc585b3a640 1 -- 172.21.15.204:0/2024721209 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc570009450 con 0x7fc580132930 2024-09-15T01:35:16.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc585b3a640 1 --2- 172.21.15.204:0/2024721209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc580132930 0x7fc580142900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc57800c840 tx=0x7fc5780015f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5757fa640 1 -- 172.21.15.204:0/2024721209 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc57800ce30 con 0x7fc580132930 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5757fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5757fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5757fa640 10 monclient: dump: 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:16.273 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5757fa640 10 monclient: _finish_auth 0 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5757fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5757fa640 1 -- 172.21.15.204:0/2024721209 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc578005d60 con 0x7fc580132930 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc58759d640 5 monclient: authenticate success, global_id 14664 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc58759d640 10 monclient: _renew_subs 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc58759d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:16.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc58759d640 1 -- 172.21.15.204:0/2024721209 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58011f3b0 con 0x7fc580132930 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5757fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5757fa640 1 -- 172.21.15.204:0/2024721209 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc5780070f0 con 0x7fc580132930 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5757fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5757fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5777fe640 4 set_mon_vals no callback set 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc5757fa640 10 monclient: dump: 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:16.275 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc58759d640 10 monclient: _renew_subs 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc58759d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.037+0000 7fc58759d640 1 -- 172.21.15.204:0/2024721209 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc580120ee0 con 0x7fc580132930 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.038+0000 7fc5757fa640 1 -- 172.21.15.204:0/2024721209 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc578007250 con 0x7fc580132930 2024-09-15T01:35:16.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.038+0000 7fc58759d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:16.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.038+0000 7fc58759d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:16.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.038+0000 7fc58759d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:16.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.038+0000 7fc58759d640 1 -- 172.21.15.204:0/2024721209 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc580070b80 con 0x7fc580132930 2024-09-15T01:35:16.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.038+0000 7fc5757fa640 1 --2- 172.21.15.204:0/2024721209 >> v2:172.21.15.204:6812/28210 conn(0x7fc5640397b0 0x7fc56403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.038+0000 7fc5757fa640 1 -- 172.21.15.204:0/2024721209 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5727+0+0 (secure 0 0 0) 0x7fc57804b360 con 0x7fc580132930 2024-09-15T01:35:16.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.038+0000 7fc585339640 1 --2- 172.21.15.204:0/2024721209 >> v2:172.21.15.204:6812/28210 conn(0x7fc5640397b0 0x7fc56403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:16.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.038+0000 7fc585339640 10 monclient: get_auth_request con 0x7fc5640397b0 auth_method 0 2024-09-15T01:35:16.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.038+0000 7fc585339640 1 --2- 172.21.15.204:0/2024721209 >> v2:172.21.15.204:6812/28210 conn(0x7fc5640397b0 0x7fc56403bc60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc57c006f40 tx=0x7fc57c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:16.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.042+0000 7fc5757fa640 1 -- 172.21.15.204:0/2024721209 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc57800a5e0 con 0x7fc580132930 2024-09-15T01:35:16.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.042+0000 7fc5757fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:16.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.042+0000 7fc5757fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:16.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.197+0000 7fc58759d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:35:16.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.197+0000 7fc58759d640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:35:16.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.197+0000 7fc58759d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:16.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.197+0000 7fc58759d640 1 -- 172.21.15.204:0/2024721209 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7fc58009e480 con 0x7fc580132930 2024-09-15T01:35:16.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.198+0000 7fc5757fa640 1 -- 172.21.15.204:0/2024721209 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v112) v1 ==== 56+0+1449 (secure 0 0 0) 0x7fc578010300 con 0x7fc580132930 2024-09-15T01:35:16.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.198+0000 7fc5757fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:35:16.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.198+0000 7fc5757fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:16.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.290+0000 7fc58759d640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:35:16.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.290+0000 7fc58759d640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:35:16.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.290+0000 7fc58759d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:16.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.290+0000 7fc58759d640 1 -- 172.21.15.204:0/2024721209 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7fc5800008d0 con 0x7fc580132930 2024-09-15T01:35:16.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.291+0000 7fc5757fa640 1 -- 172.21.15.204:0/2024721209 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v112) v1 ==== 56+0+1449 (secure 0 0 0) 0x7fc57801a360 con 0x7fc580132930 2024-09-15T01:35:16.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.291+0000 7fc5757fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:35:16.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.291+0000 7fc5757fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:16.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.292+0000 7fc58759d640 1 -- 172.21.15.204:0/2024721209 >> v2:172.21.15.204:6812/28210 conn(0x7fc5640397b0 msgr2=0x7fc56403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:16.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.292+0000 7fc58759d640 1 --2- 172.21.15.204:0/2024721209 >> v2:172.21.15.204:6812/28210 conn(0x7fc5640397b0 0x7fc56403bc60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc57c006f40 tx=0x7fc57c008040 comp rx=0 tx=0).stop 2024-09-15T01:35:16.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.292+0000 7fc58759d640 10 monclient: shutdown 2024-09-15T01:35:16.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.292+0000 7fc58759d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:16.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.292+0000 7fc58759d640 1 -- 172.21.15.204:0/2024721209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc580132930 msgr2=0x7fc580142900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:16.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.292+0000 7fc58759d640 1 --2- 172.21.15.204:0/2024721209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc580132930 0x7fc580142900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc57800c840 tx=0x7fc5780015f0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.292+0000 7fc58759d640 1 -- 172.21.15.204:0/2024721209 shutdown_connections 2024-09-15T01:35:16.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.292+0000 7fc58759d640 1 --2- 172.21.15.204:0/2024721209 >> v2:172.21.15.204:6812/28210 conn(0x7fc5640397b0 0x7fc56403bc60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.292+0000 7fc58759d640 1 --2- 172.21.15.204:0/2024721209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc580132930 0x7fc580142900 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.292+0000 7fc58759d640 1 --2- 172.21.15.204:0/2024721209 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc5800a2400 0x7fc5801423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.292+0000 7fc58759d640 1 --2- 172.21.15.204:0/2024721209 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc58009f990 0x7fc580141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.292+0000 7fc58759d640 1 -- 172.21.15.204:0/2024721209 >> 172.21.15.204:0/2024721209 conn(0x7fc58006e860 msgr2=0x7fc580131260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:16.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.293+0000 7fc58759d640 1 -- 172.21.15.204:0/2024721209 shutdown_connections 2024-09-15T01:35:16.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.293+0000 7fc58759d640 1 -- 172.21.15.204:0/2024721209 wait complete. 2024-09-15T01:35:16.304 INFO:tasks.workunit.client.0.smithi204.stdout: 14 123.00000 osd.14 down 1.00000 1.00000 2024-09-15T01:35:16.304 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush reweight-subtree foobaz 155 2024-09-15T01:35:16.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.375+0000 7ff8ed99d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:16.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.375+0000 7ff8ed99d640 10 monclient: build_initial_monmap 2024-09-15T01:35:16.378 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.375+0000 7ff8ed99d640 10 monclient: monmap: 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:16.377367+0000 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:16.377367+0000 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:16.379 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:16.380 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:16.380 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:16.380 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:16.380 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:16.380 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:16.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 1 Processor -- start 2024-09-15T01:35:16.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 1 -- start start 2024-09-15T01:35:16.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 10 monclient: init 2024-09-15T01:35:16.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:16.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 10 monclient: _add_conns ranks=[4,2,8,9,7,10,6,1,0,3,5] 2024-09-15T01:35:16.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff8e80a2e30 0x7ff8e8132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 10 monclient(hunting): picked mon.noname-c con 0x7ff8e80a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:16.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8e809fb80 0x7ff8e809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 10 monclient(hunting): picked mon.noname-b con 0x7ff8e809fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:16.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8e80a0a40 0x7ff8e80a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 10 monclient(hunting): picked mon.noname-e con 0x7ff8e80a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:16.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:16.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e809e6c0 con 0x7ff8e809fb80 2024-09-15T01:35:16.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:16.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e80a04b0 con 0x7ff8e80a2e30 2024-09-15T01:35:16.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:16.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e80a3200 con 0x7ff8e80a0a40 2024-09-15T01:35:16.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:16.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.376+0000 7ff8ed99d640 10 monclient(hunting): authenticate will time out at 932.019989s 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e67fc640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8e809fb80 msgr2=0x7ff8e809ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e67fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8e809fb80 0x7ff8e809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8e80a0a40 0x7ff8e80a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8e80a0a40 0x7ff8e80a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:50980/0 (socket says 172.21.15.204:50980) 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 1 -- 172.21.15.204:0/1489306708 learned_addr learned my addr 172.21.15.204:0/1489306708 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 10 monclient(hunting): get_auth_request con 0x7ff8e80a0a40 auth_method 0 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e6ffd640 1 --2- 172.21.15.204:0/1489306708 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff8e80a2e30 0x7ff8e8132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e6ffd640 10 monclient(hunting): get_auth_request con 0x7ff8e80a2e30 auth_method 0 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:16.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 10 monclient(hunting): handle_auth_done global_id 5543 payload 274 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 1 -- 172.21.15.204:0/1489306708 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff8e80a2e30 msgr2=0x7ff8e8132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 1 --2- 172.21.15.204:0/1489306708 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff8e80a2e30 0x7ff8e8132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 1 -- 172.21.15.204:0/1489306708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8e809fb80 msgr2=0x7ff8e809ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 1 --2- 172.21.15.204:0/1489306708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8e809fb80 0x7ff8e809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 1 monclient: found mon.noname-e 2024-09-15T01:35:16.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.377+0000 7ff8e77fe640 1 -- 172.21.15.204:0/1489306708 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e804b720 con 0x7ff8e80a0a40 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e77fe640 1 --2- 172.21.15.204:0/1489306708 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8e80a0a40 0x7ff8e80a0e90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009940 tx=0x7ff8d80091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=232c184dbb2f5647 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e5ffb640 1 -- 172.21.15.204:0/1489306708 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff8d802ac80 con 0x7ff8e80a0a40 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e5ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e5ffb640 10 monclient: dump: 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:16.392 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e5ffb640 1 -- 172.21.15.204:0/1489306708 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d802ae00 con 0x7ff8e80a0a40 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8ed99d640 5 monclient: authenticate success, global_id 5543 2024-09-15T01:35:16.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8ed99d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8ed99d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e5ffb640 1 -- 172.21.15.204:0/1489306708 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff8d8031ca0 con 0x7ff8e80a0a40 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8ed99d640 4 set_mon_vals no callback set 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e5ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.378+0000 7ff8e5ffb640 10 monclient: dump: 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:16.394 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 10 monclient: shutdown 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 1 -- 172.21.15.204:0/1489306708 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8e80a0a40 msgr2=0x7ff8e80a0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:16.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 1 --2- 172.21.15.204:0/1489306708 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8e80a0a40 0x7ff8e80a0e90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009940 tx=0x7ff8d80091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 1 -- 172.21.15.204:0/1489306708 shutdown_connections 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 1 --2- 172.21.15.204:0/1489306708 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff8e80a0a40 0x7ff8e80a0e90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 1 --2- 172.21.15.204:0/1489306708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8e809fb80 0x7ff8e809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 1 --2- 172.21.15.204:0/1489306708 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff8e80a2e30 0x7ff8e8132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 1 -- 172.21.15.204:0/1489306708 >> 172.21.15.204:0/1489306708 conn(0x7ff8e806e860 msgr2=0x7ff8e806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 1 -- 172.21.15.204:0/1489306708 shutdown_connections 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 1 -- 172.21.15.204:0/1489306708 wait complete. 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 10 monclient: build_initial_monmap 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.379+0000 7ff8ed99d640 10 monclient: monmap: 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:16.396 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 1 Processor -- start 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 1 -- start start 2024-09-15T01:35:16.397 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 10 monclient: init 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 10 monclient: _add_conns ranks=[1,7,5,8,2,0,3,6,4] 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e809fb80 0x7ff8e8131890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 10 monclient(hunting): picked mon.noname-f con 0x7ff8e809fb80 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8e80a0a40 0x7ff8e811e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 10 monclient(hunting): picked mon.noname-i con 0x7ff8e80a0a40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8e80a2e30 0x7ff8e811ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 10 monclient(hunting): picked mon.noname-h con 0x7ff8e80a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e809d650 con 0x7ff8e809fb80 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8121cc0 con 0x7ff8e80a2e30 2024-09-15T01:35:16.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e81219e0 con 0x7ff8e80a0a40 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8ed99d640 10 monclient(hunting): authenticate will time out at 932.023644s 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e67fc640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8e80a0a40 msgr2=0x7ff8e811e540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e67fc640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8e80a0a40 0x7ff8e811e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e809fb80 0x7ff8e8131890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e77fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8e80a2e30 0x7ff8e811ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e77fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8e80a2e30 0x7ff8e811ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:50070/0 (socket says 172.21.15.204:50070) 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e809fb80 0x7ff8e8131890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48486/0 (socket says 172.21.15.204:48486) 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e77fe640 1 -- 172.21.15.204:0/263485690 learned_addr learned my addr 172.21.15.204:0/263485690 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e77fe640 10 monclient(hunting): get_auth_request con 0x7ff8e80a2e30 auth_method 0 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:16.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e6ffd640 10 monclient(hunting): get_auth_request con 0x7ff8e809fb80 auth_method 0 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.380+0000 7ff8e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e77fe640 10 monclient(hunting): handle_auth_done global_id 5603 payload 824 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e77fe640 1 -- 172.21.15.204:0/263485690 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8e80a0a40 msgr2=0x7ff8e811e540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:16.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e77fe640 1 --2- 172.21.15.204:0/263485690 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8e80a0a40 0x7ff8e811e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e77fe640 1 -- 172.21.15.204:0/263485690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e809fb80 msgr2=0x7ff8e8131890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e77fe640 1 --2- 172.21.15.204:0/263485690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e809fb80 0x7ff8e8131890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e77fe640 1 monclient: found mon.noname-h 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e77fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e77fe640 1 -- 172.21.15.204:0/263485690 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d8009450 con 0x7ff8e80a2e30 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e6ffd640 1 --2- 172.21.15.204:0/263485690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e809fb80 0x7ff8e8131890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e77fe640 1 --2- 172.21.15.204:0/263485690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8e80a2e30 0x7ff8e811ea80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8035040 tx=0x7ff8d8032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8deffd640 1 -- 172.21.15.204:0/263485690 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff8d803b870 con 0x7ff8e80a2e30 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8deffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:16.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8deffd640 10 monclient: dump: 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:16.402 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8deffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8deffd640 1 -- 172.21.15.204:0/263485690 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d803ba30 con 0x7ff8e80a2e30 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8ed99d640 5 monclient: authenticate success, global_id 5603 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8ed99d640 10 monclient: _renew_subs 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8ed99d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8ed99d640 1 -- 172.21.15.204:0/263485690 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e811feb0 con 0x7ff8e80a2e30 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8deffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8deffd640 1 -- 172.21.15.204:0/263485690 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff8d803bba0 con 0x7ff8e80a2e30 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8deffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8deffd640 10 monclient: dump: 2024-09-15T01:35:16.403 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:16.404 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8e57fa640 4 set_mon_vals no callback set 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8ed99d640 10 monclient: _renew_subs 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8ed99d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.381+0000 7ff8ed99d640 1 -- 172.21.15.204:0/263485690 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e8133f50 con 0x7ff8e80a2e30 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.382+0000 7ff8deffd640 1 -- 172.21.15.204:0/263485690 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff8d8039040 con 0x7ff8e80a2e30 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.382+0000 7ff8ed99d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.382+0000 7ff8ed99d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.382+0000 7ff8ed99d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.382+0000 7ff8ed99d640 1 -- 172.21.15.204:0/263485690 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e8070cc0 con 0x7ff8e80a2e30 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.382+0000 7ff8deffd640 1 --2- 172.21.15.204:0/263485690 >> v2:172.21.15.204:6812/28210 conn(0x7ff8c80397b0 0x7ff8c803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.382+0000 7ff8deffd640 1 -- 172.21.15.204:0/263485690 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5727+0+0 (secure 0 0 0) 0x7ff8d806b020 con 0x7ff8e80a2e30 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.382+0000 7ff8e6ffd640 1 --2- 172.21.15.204:0/263485690 >> v2:172.21.15.204:6812/28210 conn(0x7ff8c80397b0 0x7ff8c803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:16.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.382+0000 7ff8e6ffd640 10 monclient: get_auth_request con 0x7ff8c80397b0 auth_method 0 2024-09-15T01:35:16.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.382+0000 7ff8e6ffd640 1 --2- 172.21.15.204:0/263485690 >> v2:172.21.15.204:6812/28210 conn(0x7ff8c80397b0 0x7ff8c803bc60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000f8c0 tx=0x7ff8e000b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:16.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.386+0000 7ff8deffd640 1 -- 172.21.15.204:0/263485690 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff8d803b3f0 con 0x7ff8e80a2e30 2024-09-15T01:35:16.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.387+0000 7ff8deffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:16.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.387+0000 7ff8deffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.553+0000 7ff8ed99d640 10 start_mon_command cmd=[{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-09-15T01:35:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.553+0000 7ff8ed99d640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-09-15T01:35:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.553+0000 7ff8ed99d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:16.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:16.553+0000 7ff8ed99d640 1 -- 172.21.15.204:0/263485690 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0} v 0) v1 -- 0x7ff8e80a1f10 con 0x7ff8e80a2e30 2024-09-15T01:35:17.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:17.518+0000 7ff8deffd640 1 -- 172.21.15.204:0/263485690 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}]=0 reweighted subtree id -10 name 'foobaz' to 155 in crush map v114) v1 ==== 168+0+0 (secure 0 0 0) 0x7ff8d80420b0 con 0x7ff8e80a2e30 2024-09-15T01:35:17.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:17.519+0000 7ff8deffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-09-15T01:35:17.521 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:17.519+0000 7ff8deffd640 10 monclient: _finish_command 2 = system:0 reweighted subtree id -10 name 'foobaz' to 155 in crush map 2024-09-15T01:35:17.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:17.620+0000 7ff8ed99d640 10 start_mon_command cmd=[{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-09-15T01:35:17.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:17.620+0000 7ff8ed99d640 10 monclient: _send_command 3 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-09-15T01:35:17.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:17.620+0000 7ff8ed99d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:17.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:17.620+0000 7ff8ed99d640 1 -- 172.21.15.204:0/263485690 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0} v 0) v1 -- 0x7ff8e80008d0 con 0x7ff8e80a2e30 2024-09-15T01:35:18.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.532+0000 7ff8deffd640 1 -- 172.21.15.204:0/263485690 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}]=0 reweighted subtree id -10 name 'foobaz' to 155 in crush map v115) v1 ==== 168+0+0 (secure 0 0 0) 0x7ff8d80307b0 con 0x7ff8e80a2e30 2024-09-15T01:35:18.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.532+0000 7ff8deffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-09-15T01:35:18.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.532+0000 7ff8deffd640 10 monclient: _finish_command 3 = system:0 reweighted subtree id -10 name 'foobaz' to 155 in crush map 2024-09-15T01:35:18.535 INFO:tasks.workunit.client.0.smithi204.stderr:reweighted subtree id -10 name 'foobaz' to 155 in crush map 2024-09-15T01:35:18.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 1 -- 172.21.15.204:0/263485690 >> v2:172.21.15.204:6812/28210 conn(0x7ff8c80397b0 msgr2=0x7ff8c803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:18.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 1 --2- 172.21.15.204:0/263485690 >> v2:172.21.15.204:6812/28210 conn(0x7ff8c80397b0 0x7ff8c803bc60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000f8c0 tx=0x7ff8e000b040 comp rx=0 tx=0).stop 2024-09-15T01:35:18.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 10 monclient: shutdown 2024-09-15T01:35:18.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:18.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 1 -- 172.21.15.204:0/263485690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8e80a2e30 msgr2=0x7ff8e811ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:18.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 1 --2- 172.21.15.204:0/263485690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8e80a2e30 0x7ff8e811ea80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8035040 tx=0x7ff8d8032000 comp rx=0 tx=0).stop 2024-09-15T01:35:18.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 1 -- 172.21.15.204:0/263485690 shutdown_connections 2024-09-15T01:35:18.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 1 --2- 172.21.15.204:0/263485690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff8e80a2e30 0x7ff8e811ea80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 1 --2- 172.21.15.204:0/263485690 >> v2:172.21.15.204:6812/28210 conn(0x7ff8c80397b0 0x7ff8c803bc60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 1 --2- 172.21.15.204:0/263485690 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff8e80a0a40 0x7ff8e811e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 1 --2- 172.21.15.204:0/263485690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e809fb80 0x7ff8e8131890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 1 -- 172.21.15.204:0/263485690 >> 172.21.15.204:0/263485690 conn(0x7ff8e806e860 msgr2=0x7ff8e809bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:18.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 1 -- 172.21.15.204:0/263485690 shutdown_connections 2024-09-15T01:35:18.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.535+0000 7ff8ed99d640 1 -- 172.21.15.204:0/263485690 wait complete. 2024-09-15T01:35:18.549 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush reweight-subtree foobaz 123456 2024-09-15T01:35:18.549 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:35:18.549 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush reweight-subtree foobaz 123456 2024-09-15T01:35:18.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.626+0000 7ff02491d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:18.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.626+0000 7ff02491d640 10 monclient: build_initial_monmap 2024-09-15T01:35:18.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.627+0000 7ff02491d640 10 monclient: monmap: 2024-09-15T01:35:18.630 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:18.630 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:18.630 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:18.628508+0000 2024-09-15T01:35:18.630 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:18.628508+0000 2024-09-15T01:35:18.630 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:18.630 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:18.630 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:18.630 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:18.630 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:18.631 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:18.631 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:18.631 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:18.631 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:18.631 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:18.631 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:18.631 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:18.631 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:18.631 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:18.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.627+0000 7ff02491d640 1 Processor -- start 2024-09-15T01:35:18.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.627+0000 7ff02491d640 1 -- start start 2024-09-15T01:35:18.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.627+0000 7ff02491d640 10 monclient: init 2024-09-15T01:35:18.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:18.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 10 monclient: _add_conns ranks=[10,4,2,6,9,3,1,0,8,7,5] 2024-09-15T01:35:18.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff0200a2e30 0x7ff020133280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 10 monclient(hunting): picked mon.noname-f con 0x7ff0200a2e30 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:18.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff02009fd80 0x7ff0200a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 10 monclient(hunting): picked mon.noname-c con 0x7ff02009fd80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:18.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0200a09c0 0x7ff0200a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 10 monclient(hunting): picked mon.noname-b con 0x7ff0200a09c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:18.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:18.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02009e290 con 0x7ff0200a2e30 2024-09-15T01:35:18.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:18.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0200a2450 con 0x7ff0200a09c0 2024-09-15T01:35:18.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:18.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0200a3200 con 0x7ff02009fd80 2024-09-15T01:35:18.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:18.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff02491d640 10 monclient(hunting): authenticate will time out at 934.271311s 2024-09-15T01:35:18.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff0200a2e30 0x7ff020133280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:18.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01ed9d640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0200a09c0 msgr2=0x7ff0200a0e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff0200a2e30 0x7ff020133280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36000/0 (socket says 172.21.15.204:36000) 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 1 -- 172.21.15.204:0/2935317615 learned_addr learned my addr 172.21.15.204:0/2935317615 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 10 monclient(hunting): get_auth_request con 0x7ff0200a2e30 auth_method 0 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01ed9d640 1 --2- 172.21.15.204:0/2935317615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0200a09c0 0x7ff0200a0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01dd9b640 1 --2- 172.21.15.204:0/2935317615 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff02009fd80 0x7ff0200a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01dd9b640 10 monclient(hunting): get_auth_request con 0x7ff02009fd80 auth_method 0 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:18.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 10 monclient(hunting): handle_auth_done global_id 6076 payload 274 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 1 -- 172.21.15.204:0/2935317615 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff02009fd80 msgr2=0x7ff0200a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 1 --2- 172.21.15.204:0/2935317615 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff02009fd80 0x7ff0200a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 1 -- 172.21.15.204:0/2935317615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0200a09c0 msgr2=0x7ff0200a0e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 1 --2- 172.21.15.204:0/2935317615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0200a09c0 0x7ff0200a0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 1 monclient: found mon.noname-f 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.628+0000 7ff01e59c640 1 -- 172.21.15.204:0/2935317615 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02004b720 con 0x7ff0200a2e30 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01e59c640 1 --2- 172.21.15.204:0/2935317615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0200a2e30 0x7ff020133280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff018009f50 tx=0x7ff018009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff73e10a23bbae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01d59a640 1 -- 172.21.15.204:0/2935317615 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff018007880 con 0x7ff0200a2e30 2024-09-15T01:35:18.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01d59a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01d59a640 10 monclient: dump: 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:18.643 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:18.644 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:18.644 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:18.644 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:18.644 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:18.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01d59a640 10 monclient: _finish_auth 0 2024-09-15T01:35:18.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:18.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01d59a640 1 -- 172.21.15.204:0/2935317615 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01802d050 con 0x7ff0200a2e30 2024-09-15T01:35:18.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff02491d640 5 monclient: authenticate success, global_id 6076 2024-09-15T01:35:18.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff02491d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:18.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:18.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff02491d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:18.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff02491d640 4 set_mon_vals no callback set 2024-09-15T01:35:18.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01d59a640 1 -- 172.21.15.204:0/2935317615 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff018030360 con 0x7ff0200a2e30 2024-09-15T01:35:18.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:18.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01d59a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:18.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.629+0000 7ff01d59a640 10 monclient: dump: 2024-09-15T01:35:18.645 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:18.645 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:18.645 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.645 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 10 monclient: shutdown 2024-09-15T01:35:18.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:18.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 1 -- 172.21.15.204:0/2935317615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0200a2e30 msgr2=0x7ff020133280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:18.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 1 --2- 172.21.15.204:0/2935317615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0200a2e30 0x7ff020133280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff018009f50 tx=0x7ff018009840 comp rx=0 tx=0).stop 2024-09-15T01:35:18.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 1 -- 172.21.15.204:0/2935317615 shutdown_connections 2024-09-15T01:35:18.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 1 --2- 172.21.15.204:0/2935317615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0200a09c0 0x7ff0200a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 1 --2- 172.21.15.204:0/2935317615 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff02009fd80 0x7ff0200a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 1 --2- 172.21.15.204:0/2935317615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0200a2e30 0x7ff020133280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 1 -- 172.21.15.204:0/2935317615 >> 172.21.15.204:0/2935317615 conn(0x7ff02006e860 msgr2=0x7ff02006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:18.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 1 -- 172.21.15.204:0/2935317615 shutdown_connections 2024-09-15T01:35:18.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 1 -- 172.21.15.204:0/2935317615 wait complete. 2024-09-15T01:35:18.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 10 monclient: build_initial_monmap 2024-09-15T01:35:18.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.630+0000 7ff02491d640 10 monclient: monmap: 2024-09-15T01:35:18.647 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:18.648 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:18.648 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:18.648 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:18.648 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:18.648 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:18.648 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:18.648 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:18.648 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:18.648 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:18.648 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:18.648 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:18.648 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:18.649 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:18.649 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:18.649 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:18.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 1 Processor -- start 2024-09-15T01:35:18.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 1 -- start start 2024-09-15T01:35:18.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 10 monclient: init 2024-09-15T01:35:18.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:18.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 10 monclient: _add_conns ranks=[2,8,7,6,5,1,3,0,4] 2024-09-15T01:35:18.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff02009fd80 0x7ff02012f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 10 monclient(hunting): picked mon.noname-b con 0x7ff02009fd80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:18.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff0200a09c0 0x7ff02011c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 10 monclient(hunting): picked mon.noname-e con 0x7ff0200a09c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff0200a2e30 0x7ff02011cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 10 monclient(hunting): picked mon.noname-i con 0x7ff0200a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02009d9b0 con 0x7ff02009fd80 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff02009d4c0 con 0x7ff0200a2e30 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff02011e590 con 0x7ff0200a09c0 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff02491d640 10 monclient(hunting): authenticate will time out at 934.274612s 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01ed9d640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff0200a2e30 msgr2=0x7ff02011cb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01ed9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff0200a2e30 0x7ff02011cb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01e59c640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff02009fd80 msgr2=0x7ff02012f740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01e59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff02009fd80 0x7ff02012f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:18.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01dd9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff0200a09c0 0x7ff02011c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01dd9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff0200a09c0 0x7ff02011c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:33008/0 (socket says 172.21.15.204:33008) 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01dd9b640 1 -- 172.21.15.204:0/4209101308 learned_addr learned my addr 172.21.15.204:0/4209101308 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01dd9b640 10 monclient(hunting): get_auth_request con 0x7ff0200a09c0 auth_method 0 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.631+0000 7ff01dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff01dd9b640 10 monclient(hunting): handle_auth_done global_id 5552 payload 824 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff01dd9b640 1 -- 172.21.15.204:0/4209101308 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff0200a2e30 msgr2=0x7ff02011cb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff01dd9b640 1 --2- 172.21.15.204:0/4209101308 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff0200a2e30 0x7ff02011cb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff01dd9b640 1 -- 172.21.15.204:0/4209101308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff02009fd80 msgr2=0x7ff02012f740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:18.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff01dd9b640 1 --2- 172.21.15.204:0/4209101308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff02009fd80 0x7ff02012f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff01dd9b640 10 monclient: _finish_hunting 0 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff01dd9b640 1 monclient: found mon.noname-e 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff01dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff01dd9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff01dd9b640 1 -- 172.21.15.204:0/4209101308 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff018009ae0 con 0x7ff0200a09c0 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff01dd9b640 1 --2- 172.21.15.204:0/4209101308 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff0200a09c0 0x7ff02011c630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff01400d890 tx=0x7ff014008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff00e7fc640 1 -- 172.21.15.204:0/4209101308 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff01400de20 con 0x7ff0200a09c0 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff00e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff00e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff00e7fc640 10 monclient: dump: 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.652 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff00e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:18.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff00e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff00e7fc640 1 -- 172.21.15.204:0/4209101308 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01400bd50 con 0x7ff0200a09c0 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff02491d640 5 monclient: authenticate success, global_id 5552 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff02491d640 10 monclient: _renew_subs 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff02491d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff02491d640 1 -- 172.21.15.204:0/4209101308 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff020142be0 con 0x7ff0200a09c0 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff00e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff00e7fc640 1 -- 172.21.15.204:0/4209101308 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff0140148b0 con 0x7ff0200a09c0 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff00e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff00e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff00e7fc640 10 monclient: dump: 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.654 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff01cd99640 4 set_mon_vals no callback set 2024-09-15T01:35:18.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff02491d640 10 monclient: _renew_subs 2024-09-15T01:35:18.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff02491d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:18.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.632+0000 7ff02491d640 1 -- 172.21.15.204:0/4209101308 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02011f890 con 0x7ff0200a09c0 2024-09-15T01:35:18.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.633+0000 7ff00e7fc640 1 -- 172.21.15.204:0/4209101308 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff014014430 con 0x7ff0200a09c0 2024-09-15T01:35:18.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.633+0000 7ff02491d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:18.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.633+0000 7ff02491d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:18.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.633+0000 7ff02491d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:18.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.633+0000 7ff02491d640 1 -- 172.21.15.204:0/4209101308 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff020070d80 con 0x7ff0200a09c0 2024-09-15T01:35:18.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.633+0000 7ff00e7fc640 1 --2- 172.21.15.204:0/4209101308 >> v2:172.21.15.204:6812/28210 conn(0x7ff0000397b0 0x7ff00003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.633+0000 7ff00e7fc640 1 -- 172.21.15.204:0/4209101308 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5727+0+0 (secure 0 0 0) 0x7ff01404bd80 con 0x7ff0200a09c0 2024-09-15T01:35:18.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.633+0000 7ff01e59c640 1 --2- 172.21.15.204:0/4209101308 >> v2:172.21.15.204:6812/28210 conn(0x7ff0000397b0 0x7ff00003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:18.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.633+0000 7ff01e59c640 10 monclient: get_auth_request con 0x7ff0000397b0 auth_method 0 2024-09-15T01:35:18.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.634+0000 7ff01e59c640 1 --2- 172.21.15.204:0/4209101308 >> v2:172.21.15.204:6812/28210 conn(0x7ff0000397b0 0x7ff00003bc60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff018033000 tx=0x7ff018036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:18.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.638+0000 7ff00e7fc640 1 -- 172.21.15.204:0/4209101308 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff014014a30 con 0x7ff0200a09c0 2024-09-15T01:35:18.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.638+0000 7ff00e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:18.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.638+0000 7ff00e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:18.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.799+0000 7ff02491d640 10 start_mon_command cmd=[{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}] 2024-09-15T01:35:18.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.799+0000 7ff02491d640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}] 2024-09-15T01:35:18.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.799+0000 7ff02491d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:18.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.799+0000 7ff02491d640 1 -- 172.21.15.204:0/4209101308 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0} v 0) v1 -- 0x7ff02009dfb0 con 0x7ff0200a09c0 2024-09-15T01:35:18.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.800+0000 7ff00e7fc640 1 -- 172.21.15.204:0/4209101308 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}]=-75 (75) Value too large for defined data type v115) v1 ==== 154+0+0 (secure 0 0 0) 0x7ff01404d030 con 0x7ff0200a09c0 2024-09-15T01:35:18.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.800+0000 7ff00e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}] 2024-09-15T01:35:18.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.800+0000 7ff00e7fc640 10 monclient: _finish_command 2 = mon:75 (75) Value too large for defined data type 2024-09-15T01:35:18.803 INFO:tasks.workunit.client.0.smithi204.stderr:Error EOVERFLOW: (75) Value too large for defined data type 2024-09-15T01:35:18.807 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:35:18.807 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:35:18.807 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep osd.12 2024-09-15T01:35:18.807 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 155 2024-09-15T01:35:18.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.876+0000 7f9cd9c4f640 10 monclient: get_monmap_and_config 2024-09-15T01:35:18.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.876+0000 7f9cd9c4f640 10 monclient: build_initial_monmap 2024-09-15T01:35:18.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.877+0000 7f9cd9c4f640 10 monclient: monmap: 2024-09-15T01:35:18.879 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:18.878468+0000 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:18.878468+0000 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:18.880 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:18.881 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:18.881 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:18.881 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:18.881 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.877+0000 7f9cd9c4f640 1 Processor -- start 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.877+0000 7f9cd9c4f640 1 -- start start 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.877+0000 7f9cd9c4f640 10 monclient: init 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 10 monclient: _add_conns ranks=[2,4,7,9,8,10,0,6,3,1,5] 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9cd40a2400 0x7f9cd40a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 10 monclient(hunting): picked mon.noname-b con 0x7f9cd40a2400 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd409f990 0x7f9cd409fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 10 monclient(hunting): picked mon.noname-c con 0x7f9cd409f990 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9cd4132930 0x7f9cd4132d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 10 monclient(hunting): picked mon.noname-i con 0x7f9cd4132930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:18.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd409e1a0 con 0x7f9cd40a2400 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd40a27d0 con 0x7f9cd409f990 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd409dec0 con 0x7f9cd4132930 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd9c4f640 10 monclient(hunting): authenticate will time out at 934.521391s 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd3fff640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9cd4132930 msgr2=0x7f9cd4132d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9cd4132930 0x7f9cd4132d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd37fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9cd40a2400 msgr2=0x7f9cd40a0ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd37fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9cd40a2400 0x7f9cd40a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd409f990 0x7f9cd409fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd409f990 0x7f9cd409fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:46330/0 (socket says 172.21.15.204:46330) 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 1 -- 172.21.15.204:0/951195395 learned_addr learned my addr 172.21.15.204:0/951195395 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:18.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 10 monclient(hunting): get_auth_request con 0x7f9cd409f990 auth_method 0 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 10 monclient(hunting): handle_auth_done global_id 5503 payload 274 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 1 -- 172.21.15.204:0/951195395 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9cd4132930 msgr2=0x7f9cd4132d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 1 --2- 172.21.15.204:0/951195395 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9cd4132930 0x7f9cd4132d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.878+0000 7f9cd2ffd640 1 -- 172.21.15.204:0/951195395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9cd40a2400 msgr2=0x7f9cd40a0ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.879+0000 7f9cd2ffd640 1 --2- 172.21.15.204:0/951195395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9cd40a2400 0x7f9cd40a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.879+0000 7f9cd2ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.879+0000 7f9cd2ffd640 1 monclient: found mon.noname-c 2024-09-15T01:35:18.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.879+0000 7f9cd2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.879+0000 7f9cd2ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.879+0000 7f9cd2ffd640 1 -- 172.21.15.204:0/951195395 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd40648a0 con 0x7f9cd409f990 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.879+0000 7f9cd2ffd640 1 --2- 172.21.15.204:0/951195395 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd409f990 0x7f9cd409fd60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009940 tx=0x7f9cbc0091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=b20b1ed127fe9f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.879+0000 7f9cd27fc640 1 -- 172.21.15.204:0/951195395 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9cbc02acb0 con 0x7f9cd409f990 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd27fc640 10 monclient: dump: 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:18.892 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:18.893 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:18.893 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:18.893 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:18.893 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:18.893 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:18.893 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:18.893 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:18.893 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:18.893 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:18.893 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:18.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd27fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:18.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:18.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd27fc640 1 -- 172.21.15.204:0/951195395 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc02ae30 con 0x7f9cd409f990 2024-09-15T01:35:18.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 5 monclient: authenticate success, global_id 5503 2024-09-15T01:35:18.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:18.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:18.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:18.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd27fc640 1 -- 172.21.15.204:0/951195395 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9cbc031ca0 con 0x7f9cd409f990 2024-09-15T01:35:18.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:18.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 4 set_mon_vals no callback set 2024-09-15T01:35:18.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:18.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd27fc640 10 monclient: dump: 2024-09-15T01:35:18.894 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:18.895 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:18.895 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.895 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.895 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:18.895 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:18.895 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:18.895 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:18.895 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:18.895 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:18.895 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 10 monclient: shutdown 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/951195395 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd409f990 msgr2=0x7f9cd409fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 1 --2- 172.21.15.204:0/951195395 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd409f990 0x7f9cd409fd60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009940 tx=0x7f9cbc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/951195395 shutdown_connections 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 1 --2- 172.21.15.204:0/951195395 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9cd4132930 0x7f9cd4132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 1 --2- 172.21.15.204:0/951195395 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd409f990 0x7f9cd409fd60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 1 --2- 172.21.15.204:0/951195395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9cd40a2400 0x7f9cd40a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/951195395 >> 172.21.15.204:0/951195395 conn(0x7f9cd406e860 msgr2=0x7f9cd406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/951195395 shutdown_connections 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.880+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/951195395 wait complete. 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.881+0000 7f9cd9c4f640 10 monclient: build_initial_monmap 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.881+0000 7f9cd9c4f640 10 monclient: monmap: 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:18.897 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.881+0000 7f9cd9c4f640 1 Processor -- start 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.881+0000 7f9cd9c4f640 1 -- start start 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.881+0000 7f9cd9c4f640 10 monclient: init 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.881+0000 7f9cd9c4f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.881+0000 7f9cd9c4f640 10 monclient: _add_conns ranks=[0,1,3,4,6,5,8,2,7] 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.881+0000 7f9cd9c4f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cd409f990 0x7f9cd411c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 10 monclient(hunting): picked mon.noname-a con 0x7f9cd409f990 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:18.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd40a2400 0x7f9cd411ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 10 monclient(hunting): picked mon.noname-f con 0x7f9cd40a2400 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cd4132930 0x7f9cd411cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 10 monclient(hunting): picked mon.noname-g con 0x7f9cd4132930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd411e650 con 0x7f9cd409f990 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd411f8e0 con 0x7f9cd40a2400 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd411e930 con 0x7f9cd4132930 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 10 monclient(hunting): authenticate will time out at 934.525060s 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd40a2400 0x7f9cd411ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd3fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cd4132930 0x7f9cd411cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd40a2400 0x7f9cd411ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36010/0 (socket says 172.21.15.204:36010) 2024-09-15T01:35:18.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 1 -- 172.21.15.204:0/4096009378 learned_addr learned my addr 172.21.15.204:0/4096009378 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 10 monclient(hunting): get_auth_request con 0x7f9cd40a2400 auth_method 0 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd3fff640 10 monclient(hunting): get_auth_request con 0x7f9cd4132930 auth_method 0 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:18.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd37fe640 1 --2- 172.21.15.204:0/4096009378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cd409f990 0x7f9cd411c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd37fe640 10 monclient(hunting): get_auth_request con 0x7f9cd409f990 auth_method 0 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 10 monclient(hunting): handle_auth_done global_id 6085 payload 824 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 1 -- 172.21.15.204:0/4096009378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cd4132930 msgr2=0x7f9cd411cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 1 --2- 172.21.15.204:0/4096009378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cd4132930 0x7f9cd411cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 1 -- 172.21.15.204:0/4096009378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cd409f990 msgr2=0x7f9cd411c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:18.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 1 --2- 172.21.15.204:0/4096009378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cd409f990 0x7f9cd411c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:18.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:35:18.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 1 monclient: found mon.noname-f 2024-09-15T01:35:18.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:18.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:18.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 1 -- 172.21.15.204:0/4096009378 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc009450 con 0x7f9cd40a2400 2024-09-15T01:35:18.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd3fff640 1 --2- 172.21.15.204:0/4096009378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cd4132930 0x7f9cd411cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:18.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd2ffd640 1 --2- 172.21.15.204:0/4096009378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd40a2400 0x7f9cd411ca90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0097e0 tx=0x7f9cbc032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:18.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9ccb7fe640 1 -- 172.21.15.204:0/4096009378 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9cbc02acb0 con 0x7f9cd40a2400 2024-09-15T01:35:18.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9ccb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:18.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9ccb7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:18.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9ccb7fe640 10 monclient: dump: 2024-09-15T01:35:18.902 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:18.903 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:18.903 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.903 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.903 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:18.903 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:18.903 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:18.903 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:18.903 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:18.903 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:18.903 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:18.903 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:18.903 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9ccb7fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9ccb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 5 monclient: authenticate success, global_id 6085 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9ccb7fe640 1 -- 172.21.15.204:0/4096009378 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc032e20 con 0x7f9cd40a2400 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9ccb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9ccb7fe640 1 -- 172.21.15.204:0/4096009378 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9cbc03a810 con 0x7f9cd40a2400 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 10 monclient: _renew_subs 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/4096009378 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd411ec10 con 0x7f9cd40a2400 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd1ffb640 4 set_mon_vals no callback set 2024-09-15T01:35:18.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9ccb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9ccb7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9ccb7fe640 10 monclient: dump: 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:18.905 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:18.906 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:18.906 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:18.906 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:18.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 10 monclient: _renew_subs 2024-09-15T01:35:18.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:18.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.882+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/4096009378 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd411e340 con 0x7f9cd40a2400 2024-09-15T01:35:18.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.883+0000 7f9ccb7fe640 1 -- 172.21.15.204:0/4096009378 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9cbc039040 con 0x7f9cd40a2400 2024-09-15T01:35:18.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.883+0000 7f9cd9c4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:18.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.883+0000 7f9cd9c4f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:18.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.883+0000 7f9cd9c4f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:18.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.883+0000 7f9ccb7fe640 1 --2- 172.21.15.204:0/4096009378 >> v2:172.21.15.204:6812/28210 conn(0x7f9cb40397a0 0x7f9cb403bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:18.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.883+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/4096009378 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd4070b80 con 0x7f9cd40a2400 2024-09-15T01:35:18.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.883+0000 7f9ccb7fe640 1 -- 172.21.15.204:0/4096009378 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5727+0+0 (secure 0 0 0) 0x7f9cbc06b2a0 con 0x7f9cd40a2400 2024-09-15T01:35:18.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.884+0000 7f9cd37fe640 1 --2- 172.21.15.204:0/4096009378 >> v2:172.21.15.204:6812/28210 conn(0x7f9cb40397a0 0x7f9cb403bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:18.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.884+0000 7f9cd37fe640 10 monclient: get_auth_request con 0x7f9cb40397a0 auth_method 0 2024-09-15T01:35:18.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.884+0000 7f9cd37fe640 1 --2- 172.21.15.204:0/4096009378 >> v2:172.21.15.204:6812/28210 conn(0x7f9cb40397a0 0x7f9cb403bc50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009fd0 tx=0x7f9cc4009760 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:18.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.888+0000 7f9ccb7fe640 1 -- 172.21.15.204:0/4096009378 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9cbc02fa40 con 0x7f9cd40a2400 2024-09-15T01:35:18.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.888+0000 7f9ccb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:18.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:18.888+0000 7f9ccb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.044+0000 7f9cd9c4f640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:35:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.044+0000 7f9cd9c4f640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:35:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.044+0000 7f9cd9c4f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.044+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/4096009378 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f9cd409e480 con 0x7f9cd40a2400 2024-09-15T01:35:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.045+0000 7f9ccb7fe640 1 -- 172.21.15.204:0/4096009378 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v115) v1 ==== 56+0+1449 (secure 0 0 0) 0x7f9cbc0375c0 con 0x7f9cd40a2400 2024-09-15T01:35:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.045+0000 7f9ccb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:35:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.045+0000 7f9ccb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:19.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.138+0000 7f9cd9c4f640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:35:19.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.138+0000 7f9cd9c4f640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:35:19.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.138+0000 7f9cd9c4f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:19.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.138+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/4096009378 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f9cd40008d0 con 0x7f9cd40a2400 2024-09-15T01:35:19.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.138+0000 7f9ccb7fe640 1 -- 172.21.15.204:0/4096009378 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v115) v1 ==== 56+0+1449 (secure 0 0 0) 0x7f9cbc037070 con 0x7f9cd40a2400 2024-09-15T01:35:19.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.138+0000 7f9ccb7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:35:19.141 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.138+0000 7f9ccb7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:19.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/4096009378 >> v2:172.21.15.204:6812/28210 conn(0x7f9cb40397a0 msgr2=0x7f9cb403bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:19.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 1 --2- 172.21.15.204:0/4096009378 >> v2:172.21.15.204:6812/28210 conn(0x7f9cb40397a0 0x7f9cb403bc50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009fd0 tx=0x7f9cc4009760 comp rx=0 tx=0).stop 2024-09-15T01:35:19.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 10 monclient: shutdown 2024-09-15T01:35:19.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:19.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/4096009378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd40a2400 msgr2=0x7f9cd411ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:19.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 1 --2- 172.21.15.204:0/4096009378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd40a2400 0x7f9cd411ca90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0097e0 tx=0x7f9cbc032000 comp rx=0 tx=0).stop 2024-09-15T01:35:19.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/4096009378 shutdown_connections 2024-09-15T01:35:19.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 1 --2- 172.21.15.204:0/4096009378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cd4132930 0x7f9cd411cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 1 --2- 172.21.15.204:0/4096009378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cd40a2400 0x7f9cd411ca90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 1 --2- 172.21.15.204:0/4096009378 >> v2:172.21.15.204:6812/28210 conn(0x7f9cb40397a0 0x7f9cb403bc50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 1 --2- 172.21.15.204:0/4096009378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cd409f990 0x7f9cd411c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/4096009378 >> 172.21.15.204:0/4096009378 conn(0x7f9cd406e860 msgr2=0x7f9cd4133640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:19.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/4096009378 shutdown_connections 2024-09-15T01:35:19.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.140+0000 7f9cd9c4f640 1 -- 172.21.15.204:0/4096009378 wait complete. 2024-09-15T01:35:19.152 INFO:tasks.workunit.client.0.smithi204.stdout: 12 155.00000 osd.12 down 1.00000 1.00000 2024-09-15T01:35:19.152 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd tree 2024-09-15T01:35:19.153 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep osd.14 2024-09-15T01:35:19.153 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 155 2024-09-15T01:35:19.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.222+0000 7f8e428f8640 10 monclient: get_monmap_and_config 2024-09-15T01:35:19.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.222+0000 7f8e428f8640 10 monclient: build_initial_monmap 2024-09-15T01:35:19.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.222+0000 7f8e428f8640 10 monclient: monmap: 2024-09-15T01:35:19.225 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:19.225 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:19.225 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:19.223692+0000 2024-09-15T01:35:19.225 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:19.223692+0000 2024-09-15T01:35:19.225 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:19.225 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:19.225 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:19.225 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:19.225 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:19.225 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:19.226 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:19.226 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:19.226 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:19.226 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:19.226 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:19.226 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:19.226 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:19.226 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:19.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.222+0000 7f8e428f8640 1 Processor -- start 2024-09-15T01:35:19.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.222+0000 7f8e428f8640 1 -- start start 2024-09-15T01:35:19.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 10 monclient: init 2024-09-15T01:35:19.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:19.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 10 monclient: _add_conns ranks=[10,2,9,6,3,5,7,1,0,4,8] 2024-09-15T01:35:19.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8e3c0a2400 0x7f8e3c0a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 10 monclient(hunting): picked mon.noname-f con 0x7f8e3c0a2400 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e3c09f990 0x7f8e3c09fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 10 monclient(hunting): picked mon.noname-b con 0x7f8e3c09f990 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8e3c132930 0x7f8e3c132db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 10 monclient(hunting): picked mon.noname-a con 0x7f8e3c132930 addr v2:172.21.15.178:3300/0 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3c09e1a0 con 0x7f8e3c132930 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3c0a27d0 con 0x7f8e3c0a2400 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c09dec0 con 0x7f8e3c09f990 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:19.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e428f8640 10 monclient(hunting): authenticate will time out at 934.866477s 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3b7fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e3c09f990 msgr2=0x7f8e3c09fd60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3b7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e3c09f990 0x7f8e3c09fd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8e3c0a2400 0x7f8e3c0a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8e3c0a2400 0x7f8e3c0a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36022/0 (socket says 172.21.15.204:36022) 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 1 -- 172.21.15.204:0/3657296810 learned_addr learned my addr 172.21.15.204:0/3657296810 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 10 monclient(hunting): get_auth_request con 0x7f8e3c0a2400 auth_method 0 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e40e95640 1 --2- 172.21.15.204:0/3657296810 >> v2:172.21.15.178:3300/0 conn(0x7f8e3c132930 0x7f8e3c132db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:19.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e40e95640 10 monclient(hunting): get_auth_request con 0x7f8e3c132930 auth_method 0 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e40e95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e40e95640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e40e95640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e40e95640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e40e95640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e40e95640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 10 monclient(hunting): handle_auth_done global_id 6094 payload 274 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 1 -- 172.21.15.204:0/3657296810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e3c09f990 msgr2=0x7f8e3c09fd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 1 --2- 172.21.15.204:0/3657296810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e3c09f990 0x7f8e3c09fd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 1 -- 172.21.15.204:0/3657296810 >> v2:172.21.15.178:3300/0 conn(0x7f8e3c132930 msgr2=0x7f8e3c132db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 1 --2- 172.21.15.204:0/3657296810 >> v2:172.21.15.178:3300/0 conn(0x7f8e3c132930 0x7f8e3c132db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 1 monclient: found mon.noname-f 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.223+0000 7f8e3bfff640 1 -- 172.21.15.204:0/3657296810 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c0648a0 con 0x7f8e3c0a2400 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3bfff640 1 --2- 172.21.15.204:0/3657296810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e3c0a2400 0x7f8e3c0a0ae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c068040 tx=0x7f8e2c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f5704b5569207d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3affd640 1 -- 172.21.15.204:0/3657296810 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e2c02acc0 con 0x7f8e3c0a2400 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3affd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3affd640 10 monclient: dump: 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.238 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:19.239 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:19.239 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:19.239 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:19.239 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:19.239 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:19.239 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:19.239 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:19.239 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:19.239 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:19.239 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3affd640 10 monclient: _finish_auth 0 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3affd640 1 -- 172.21.15.204:0/3657296810 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c02ae40 con 0x7f8e3c0a2400 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e428f8640 5 monclient: authenticate success, global_id 6094 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e428f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e428f8640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3affd640 1 -- 172.21.15.204:0/3657296810 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e2c031ca0 con 0x7f8e3c0a2400 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e428f8640 4 set_mon_vals no callback set 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:19.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.224+0000 7f8e3affd640 10 monclient: dump: 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:19.241 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 10 monclient: shutdown 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 1 -- 172.21.15.204:0/3657296810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e3c0a2400 msgr2=0x7f8e3c0a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 1 --2- 172.21.15.204:0/3657296810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e3c0a2400 0x7f8e3c0a0ae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c068040 tx=0x7f8e2c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 1 -- 172.21.15.204:0/3657296810 shutdown_connections 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 1 --2- 172.21.15.204:0/3657296810 >> v2:172.21.15.178:3300/0 conn(0x7f8e3c132930 0x7f8e3c132db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 1 --2- 172.21.15.204:0/3657296810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e3c09f990 0x7f8e3c09fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 1 --2- 172.21.15.204:0/3657296810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e3c0a2400 0x7f8e3c0a0ae0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 1 -- 172.21.15.204:0/3657296810 >> 172.21.15.204:0/3657296810 conn(0x7f8e3c06e860 msgr2=0x7f8e3c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 1 -- 172.21.15.204:0/3657296810 shutdown_connections 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 1 -- 172.21.15.204:0/3657296810 wait complete. 2024-09-15T01:35:19.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 10 monclient: build_initial_monmap 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.225+0000 7f8e428f8640 10 monclient: monmap: 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:19.243 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 1 Processor -- start 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 1 -- start start 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 10 monclient: init 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 10 monclient: _add_conns ranks=[1,5,8,7,4,0,2,6,3] 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e3c09f990 0x7f8e3c11d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 10 monclient(hunting): picked mon.noname-f con 0x7f8e3c09f990 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8e3c0a2400 0x7f8e3c11d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 10 monclient(hunting): picked mon.noname-h con 0x7f8e3c0a2400 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8e3c132930 0x7f8e3c11dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 10 monclient(hunting): picked mon.noname-e con 0x7f8e3c132930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c09cad0 con 0x7f8e3c09f990 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c09d3d0 con 0x7f8e3c0a2400 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c120c00 con 0x7f8e3c132930 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e428f8640 10 monclient(hunting): authenticate will time out at 934.869764s 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e3c09f990 0x7f8e3c11d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e3c09f990 0x7f8e3c11d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36024/0 (socket says 172.21.15.204:36024) 2024-09-15T01:35:19.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3bfff640 1 -- 172.21.15.204:0/2038574644 learned_addr learned my addr 172.21.15.204:0/2038574644 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3bfff640 10 monclient(hunting): get_auth_request con 0x7f8e3c09f990 auth_method 0 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3b7fe640 1 --2- 172.21.15.204:0/2038574644 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8e3c0a2400 0x7f8e3c11d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3b7fe640 10 monclient(hunting): get_auth_request con 0x7f8e3c0a2400 auth_method 0 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:19.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.226+0000 7f8e3b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e40e95640 1 --2- 172.21.15.204:0/2038574644 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8e3c132930 0x7f8e3c11dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3bfff640 10 monclient(hunting): handle_auth_done global_id 6103 payload 824 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3bfff640 1 -- 172.21.15.204:0/2038574644 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8e3c132930 msgr2=0x7f8e3c11dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3bfff640 1 --2- 172.21.15.204:0/2038574644 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8e3c132930 0x7f8e3c11dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3bfff640 1 -- 172.21.15.204:0/2038574644 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8e3c0a2400 msgr2=0x7f8e3c11d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3bfff640 1 --2- 172.21.15.204:0/2038574644 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8e3c0a2400 0x7f8e3c11d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3bfff640 1 monclient: found mon.noname-f 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3bfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3bfff640 1 -- 172.21.15.204:0/2038574644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c009450 con 0x7f8e3c09f990 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e40e95640 10 monclient: get_auth_request con 0x7f8e3c132930 auth_method 0 2024-09-15T01:35:19.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e40e95640 1 --2- 172.21.15.204:0/2038574644 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8e3c132930 0x7f8e3c11dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3b7fe640 1 --2- 172.21.15.204:0/2038574644 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8e3c0a2400 0x7f8e3c11d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3bfff640 1 --2- 172.21.15.204:0/2038574644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e3c09f990 0x7f8e3c11d2e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c035040 tx=0x7f8e2c032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e33fff640 1 -- 172.21.15.204:0/2038574644 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e2c031820 con 0x7f8e3c09f990 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e33fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e33fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e33fff640 10 monclient: dump: 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:19.248 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e33fff640 10 monclient: _finish_auth 0 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e33fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e33fff640 1 -- 172.21.15.204:0/2038574644 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c031980 con 0x7f8e3c09f990 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e428f8640 5 monclient: authenticate success, global_id 6103 2024-09-15T01:35:19.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e33fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e33fff640 1 -- 172.21.15.204:0/2038574644 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e2c03a390 con 0x7f8e3c09f990 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e33fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e33fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e33fff640 10 monclient: dump: 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:19.250 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e3a7fc640 4 set_mon_vals no callback set 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e428f8640 10 monclient: _renew_subs 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e428f8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e428f8640 1 -- 172.21.15.204:0/2038574644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c120640 con 0x7f8e3c09f990 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e428f8640 10 monclient: _renew_subs 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e428f8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.227+0000 7f8e428f8640 1 -- 172.21.15.204:0/2038574644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c11f490 con 0x7f8e3c09f990 2024-09-15T01:35:19.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.228+0000 7f8e33fff640 1 -- 172.21.15.204:0/2038574644 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8e2c039040 con 0x7f8e3c09f990 2024-09-15T01:35:19.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.228+0000 7f8e428f8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:19.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.228+0000 7f8e428f8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:19.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.228+0000 7f8e428f8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:19.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.228+0000 7f8e428f8640 1 -- 172.21.15.204:0/2038574644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c1207d0 con 0x7f8e3c09f990 2024-09-15T01:35:19.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.228+0000 7f8e33fff640 1 --2- 172.21.15.204:0/2038574644 >> v2:172.21.15.204:6812/28210 conn(0x7f8e1c0397b0 0x7f8e1c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.228+0000 7f8e33fff640 1 -- 172.21.15.204:0/2038574644 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5727+0+0 (secure 0 0 0) 0x7f8e2c06ad20 con 0x7f8e3c09f990 2024-09-15T01:35:19.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.228+0000 7f8e3b7fe640 1 --2- 172.21.15.204:0/2038574644 >> v2:172.21.15.204:6812/28210 conn(0x7f8e1c0397b0 0x7f8e1c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:19.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.228+0000 7f8e3b7fe640 10 monclient: get_auth_request con 0x7f8e1c0397b0 auth_method 0 2024-09-15T01:35:19.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.228+0000 7f8e3b7fe640 1 --2- 172.21.15.204:0/2038574644 >> v2:172.21.15.204:6812/28210 conn(0x7f8e1c0397b0 0x7f8e1c03bc60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c11cfd0 tx=0x7f8e240090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:19.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.232+0000 7f8e33fff640 1 -- 172.21.15.204:0/2038574644 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8e2c02f8a0 con 0x7f8e3c09f990 2024-09-15T01:35:19.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.232+0000 7f8e33fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:19.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.232+0000 7f8e33fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.387+0000 7f8e428f8640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:35:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.387+0000 7f8e428f8640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-09-15T01:35:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.387+0000 7f8e428f8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:19.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.387+0000 7f8e428f8640 1 -- 172.21.15.204:0/2038574644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f8e3c070b80 con 0x7f8e3c09f990 2024-09-15T01:35:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.388+0000 7f8e33fff640 1 -- 172.21.15.204:0/2038574644 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v115) v1 ==== 56+0+1449 (secure 0 0 0) 0x7f8e2c041070 con 0x7f8e3c09f990 2024-09-15T01:35:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.388+0000 7f8e33fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-09-15T01:35:19.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.388+0000 7f8e33fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:19.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.482+0000 7f8e428f8640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-09-15T01:35:19.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.482+0000 7f8e428f8640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-09-15T01:35:19.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.482+0000 7f8e428f8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:19.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.482+0000 7f8e428f8640 1 -- 172.21.15.204:0/2038574644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f8e3c09e480 con 0x7f8e3c09f990 2024-09-15T01:35:19.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.483+0000 7f8e33fff640 1 -- 172.21.15.204:0/2038574644 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v115) v1 ==== 56+0+1449 (secure 0 0 0) 0x7f8e2c037020 con 0x7f8e3c09f990 2024-09-15T01:35:19.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.483+0000 7f8e33fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-09-15T01:35:19.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.483+0000 7f8e33fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:19.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.484+0000 7f8e428f8640 1 -- 172.21.15.204:0/2038574644 >> v2:172.21.15.204:6812/28210 conn(0x7f8e1c0397b0 msgr2=0x7f8e1c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:19.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.484+0000 7f8e428f8640 1 --2- 172.21.15.204:0/2038574644 >> v2:172.21.15.204:6812/28210 conn(0x7f8e1c0397b0 0x7f8e1c03bc60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c11cfd0 tx=0x7f8e240090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.484+0000 7f8e428f8640 10 monclient: shutdown 2024-09-15T01:35:19.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.484+0000 7f8e428f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:19.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.484+0000 7f8e428f8640 1 -- 172.21.15.204:0/2038574644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e3c09f990 msgr2=0x7f8e3c11d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:19.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.484+0000 7f8e428f8640 1 --2- 172.21.15.204:0/2038574644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e3c09f990 0x7f8e3c11d2e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c035040 tx=0x7f8e2c032000 comp rx=0 tx=0).stop 2024-09-15T01:35:19.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.485+0000 7f8e428f8640 1 -- 172.21.15.204:0/2038574644 shutdown_connections 2024-09-15T01:35:19.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.485+0000 7f8e428f8640 1 --2- 172.21.15.204:0/2038574644 >> v2:172.21.15.204:6812/28210 conn(0x7f8e1c0397b0 0x7f8e1c03bc60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.485+0000 7f8e428f8640 1 --2- 172.21.15.204:0/2038574644 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8e3c132930 0x7f8e3c11dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.485+0000 7f8e428f8640 1 --2- 172.21.15.204:0/2038574644 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8e3c0a2400 0x7f8e3c11d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.485+0000 7f8e428f8640 1 --2- 172.21.15.204:0/2038574644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e3c09f990 0x7f8e3c11d2e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.485+0000 7f8e428f8640 1 -- 172.21.15.204:0/2038574644 >> 172.21.15.204:0/2038574644 conn(0x7f8e3c06e860 msgr2=0x7f8e3c0a3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:19.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.485+0000 7f8e428f8640 1 -- 172.21.15.204:0/2038574644 shutdown_connections 2024-09-15T01:35:19.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.485+0000 7f8e428f8640 1 -- 172.21.15.204:0/2038574644 wait complete. 2024-09-15T01:35:19.496 INFO:tasks.workunit.client.0.smithi204.stdout: 14 155.00000 osd.14 down 1.00000 1.00000 2024-09-15T01:35:19.496 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm osd.12 2024-09-15T01:35:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.570+0000 7fac242dd640 10 monclient: get_monmap_and_config 2024-09-15T01:35:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.570+0000 7fac242dd640 10 monclient: build_initial_monmap 2024-09-15T01:35:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.570+0000 7fac242dd640 10 monclient: monmap: 2024-09-15T01:35:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:19.571676+0000 2024-09-15T01:35:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:19.571676+0000 2024-09-15T01:35:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:19.573 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.570+0000 7fac242dd640 1 Processor -- start 2024-09-15T01:35:19.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.570+0000 7fac242dd640 1 -- start start 2024-09-15T01:35:19.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.570+0000 7fac242dd640 10 monclient: init 2024-09-15T01:35:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 10 monclient: _add_conns ranks=[5,8,2,3,9,1,10,6,0,7,4] 2024-09-15T01:35:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac1c0a35e0 0x7fac1c099a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 10 monclient(hunting): picked mon.noname-h con 0x7fac1c0a35e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fac1c09c560 0x7fac1c09d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 10 monclient(hunting): picked mon.noname-e con 0x7fac1c09c560 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c09c950 0x7fac1c09dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 10 monclient(hunting): picked mon.noname-b con 0x7fac1c09c950 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:19.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c099f50 con 0x7fac1c09c950 2024-09-15T01:35:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c09d340 con 0x7fac1c0a35e0 2024-09-15T01:35:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c0a39b0 con 0x7fac1c09c560 2024-09-15T01:35:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac242dd640 10 monclient(hunting): authenticate will time out at 935.214476s 2024-09-15T01:35:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac1c0a35e0 0x7fac1c099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac1c0a35e0 0x7fac1c099a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49724/0 (socket says 172.21.15.204:49724) 2024-09-15T01:35:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 1 -- 172.21.15.204:0/2638535219 learned_addr learned my addr 172.21.15.204:0/2638535219 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 10 monclient(hunting): get_auth_request con 0x7fac1c0a35e0 auth_method 0 2024-09-15T01:35:19.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac2287a640 1 -- 172.21.15.204:0/2638535219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c09c950 msgr2=0x7fac1c09dd80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac2287a640 1 --2- 172.21.15.204:0/2638535219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c09c950 0x7fac1c09dd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 10 monclient(hunting): handle_auth_done global_id 5621 payload 274 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 1 -- 172.21.15.204:0/2638535219 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fac1c09c560 msgr2=0x7fac1c09d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 1 --2- 172.21.15.204:0/2638535219 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fac1c09c560 0x7fac1c09d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 1 -- 172.21.15.204:0/2638535219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c09c950 msgr2=0x7fac1c09dd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 1 --2- 172.21.15.204:0/2638535219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c09c950 0x7fac1c09dd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 10 monclient: _finish_hunting 0 2024-09-15T01:35:19.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 1 monclient: found mon.noname-h 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.571+0000 7fac22079640 1 -- 172.21.15.204:0/2638535219 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c04b720 con 0x7fac1c0a35e0 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac22079640 1 --2- 172.21.15.204:0/2638535219 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac1c0a35e0 0x7fac1c099a10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fac18009970 tx=0x7fac180091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=db609a6c58f59a61 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac21077640 1 -- 172.21.15.204:0/2638535219 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac1802ce80 con 0x7fac1c0a35e0 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac21077640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac21077640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac21077640 10 monclient: dump: 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:19.586 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac21077640 10 monclient: _finish_auth 0 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac21077640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac242dd640 5 monclient: authenticate success, global_id 5621 2024-09-15T01:35:19.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac242dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac21077640 1 -- 172.21.15.204:0/2638535219 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac18033820 con 0x7fac1c0a35e0 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac21077640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac242dd640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac242dd640 4 set_mon_vals no callback set 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac21077640 1 -- 172.21.15.204:0/2638535219 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac18033e90 con 0x7fac1c0a35e0 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac21077640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac21077640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.572+0000 7fac21077640 10 monclient: dump: 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.588 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 10 monclient: shutdown 2024-09-15T01:35:19.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 1 -- 172.21.15.204:0/2638535219 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac1c0a35e0 msgr2=0x7fac1c099a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 1 --2- 172.21.15.204:0/2638535219 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac1c0a35e0 0x7fac1c099a10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fac18009970 tx=0x7fac180091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 1 -- 172.21.15.204:0/2638535219 shutdown_connections 2024-09-15T01:35:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 1 --2- 172.21.15.204:0/2638535219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c09c950 0x7fac1c09dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 1 --2- 172.21.15.204:0/2638535219 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fac1c09c560 0x7fac1c09d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 1 --2- 172.21.15.204:0/2638535219 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac1c0a35e0 0x7fac1c099a10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 1 -- 172.21.15.204:0/2638535219 >> 172.21.15.204:0/2638535219 conn(0x7fac1c0687e0 msgr2=0x7fac1c0a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 1 -- 172.21.15.204:0/2638535219 shutdown_connections 2024-09-15T01:35:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 1 -- 172.21.15.204:0/2638535219 wait complete. 2024-09-15T01:35:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 10 monclient: build_initial_monmap 2024-09-15T01:35:19.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.573+0000 7fac242dd640 10 monclient: monmap: 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:19.591 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 1 Processor -- start 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 1 -- start start 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 10 monclient: init 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 10 monclient: _add_conns ranks=[0,7,2,4,5,3,1,8,6] 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac1c09c560 0x7fac1c1315d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 10 monclient(hunting): picked mon.noname-a con 0x7fac1c09c560 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fac1c09c950 0x7fac1c11b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 10 monclient(hunting): picked mon.noname-i con 0x7fac1c09c950 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c0a35e0 0x7fac1c11b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 10 monclient(hunting): picked mon.noname-b con 0x7fac1c0a35e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c06f000 con 0x7fac1c09c560 2024-09-15T01:35:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c06ea80 con 0x7fac1c0a35e0 2024-09-15T01:35:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c13e750 con 0x7fac1c09c950 2024-09-15T01:35:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac242dd640 10 monclient(hunting): authenticate will time out at 935.217637s 2024-09-15T01:35:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac21878640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fac1c09c950 msgr2=0x7fac1c11b130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac21878640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fac1c09c950 0x7fac1c11b130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac2287a640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c0a35e0 msgr2=0x7fac1c11b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:19.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac2287a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c0a35e0 0x7fac1c11b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac22079640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac1c09c560 0x7fac1c1315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.574+0000 7fac22079640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac1c09c560 0x7fac1c1315d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:43912/0 (socket says 172.21.15.204:43912) 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 1 -- 172.21.15.204:0/2642682605 learned_addr learned my addr 172.21.15.204:0/2642682605 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 10 monclient(hunting): get_auth_request con 0x7fac1c09c560 auth_method 0 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 10 monclient(hunting): handle_auth_done global_id 5733 payload 824 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 1 -- 172.21.15.204:0/2642682605 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fac1c09c950 msgr2=0x7fac1c11b130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 1 --2- 172.21.15.204:0/2642682605 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fac1c09c950 0x7fac1c11b130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 1 -- 172.21.15.204:0/2642682605 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c0a35e0 msgr2=0x7fac1c11b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 1 --2- 172.21.15.204:0/2642682605 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c0a35e0 0x7fac1c11b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 10 monclient: _finish_hunting 0 2024-09-15T01:35:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 1 monclient: found mon.noname-a 2024-09-15T01:35:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:35:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 1 -- 172.21.15.204:0/2642682605 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac18009450 con 0x7fac1c09c560 2024-09-15T01:35:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac22079640 1 --2- 172.21.15.204:0/2642682605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac1c09c560 0x7fac1c1315d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fac180097e0 tx=0x7fac18034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac127fc640 1 -- 172.21.15.204:0/2642682605 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac18035db0 con 0x7fac1c09c560 2024-09-15T01:35:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac127fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:35:19.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac127fc640 10 monclient: dump: 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:19.596 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:19.597 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac127fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac127fc640 1 -- 172.21.15.204:0/2642682605 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1803d3f0 con 0x7fac1c09c560 2024-09-15T01:35:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac242dd640 5 monclient: authenticate success, global_id 5733 2024-09-15T01:35:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac242dd640 10 monclient: _renew_subs 2024-09-15T01:35:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac242dd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac242dd640 1 -- 172.21.15.204:0/2642682605 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c11d140 con 0x7fac1c09c560 2024-09-15T01:35:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac127fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:19.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac127fc640 1 -- 172.21.15.204:0/2642682605 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac1803d9e0 con 0x7fac1c09c560 2024-09-15T01:35:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac127fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:35:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac127fc640 10 monclient: dump: 2024-09-15T01:35:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:19.598 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac20876640 4 set_mon_vals no callback set 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac242dd640 10 monclient: _renew_subs 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac242dd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.575+0000 7fac242dd640 1 -- 172.21.15.204:0/2642682605 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c11e6f0 con 0x7fac1c09c560 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.576+0000 7fac127fc640 1 -- 172.21.15.204:0/2642682605 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fac1803b040 con 0x7fac1c09c560 2024-09-15T01:35:19.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.576+0000 7fac242dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:19.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.576+0000 7fac242dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:19.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.576+0000 7fac242dd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:19.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.576+0000 7fac242dd640 1 -- 172.21.15.204:0/2642682605 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1c070cc0 con 0x7fac1c09c560 2024-09-15T01:35:19.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.576+0000 7fac127fc640 1 --2- 172.21.15.204:0/2642682605 >> v2:172.21.15.204:6812/28210 conn(0x7fac000397b0 0x7fac0003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:19.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.576+0000 7fac127fc640 1 -- 172.21.15.204:0/2642682605 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5727+0+0 (secure 0 0 0) 0x7fac1806d540 con 0x7fac1c09c560 2024-09-15T01:35:19.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.576+0000 7fac21878640 1 --2- 172.21.15.204:0/2642682605 >> v2:172.21.15.204:6812/28210 conn(0x7fac000397b0 0x7fac0003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:19.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.576+0000 7fac21878640 10 monclient: get_auth_request con 0x7fac000397b0 auth_method 0 2024-09-15T01:35:19.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.577+0000 7fac21878640 1 --2- 172.21.15.204:0/2642682605 >> v2:172.21.15.204:6812/28210 conn(0x7fac000397b0 0x7fac0003bc60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0a3c00 tx=0x7fac0c0029e0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:19.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.581+0000 7fac127fc640 1 -- 172.21.15.204:0/2642682605 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fac18047580 con 0x7fac1c09c560 2024-09-15T01:35:19.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.581+0000 7fac127fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:19.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.581+0000 7fac127fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:19.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.741+0000 7fac242dd640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.12"}] 2024-09-15T01:35:19.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.741+0000 7fac242dd640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-09-15T01:35:19.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.741+0000 7fac242dd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:19.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:19.741+0000 7fac242dd640 1 -- 172.21.15.204:0/2642682605 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.12"} v 0) v1 -- 0x7fac1c069880 con 0x7fac1c09c560 2024-09-15T01:35:20.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.532+0000 7fac127fc640 1 -- 172.21.15.204:0/2642682605 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.12"}]=0 removed item id 12 name 'osd.12' from crush map v116) v1 ==== 125+0+0 (secure 0 0 0) 0x7fac18032b30 con 0x7fac1c09c560 2024-09-15T01:35:20.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.532+0000 7fac127fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-09-15T01:35:20.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.532+0000 7fac127fc640 10 monclient: _finish_command 2 = system:0 removed item id 12 name 'osd.12' from crush map 2024-09-15T01:35:20.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.635+0000 7fac242dd640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.12"}] 2024-09-15T01:35:20.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.635+0000 7fac242dd640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-09-15T01:35:20.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.635+0000 7fac242dd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:20.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.635+0000 7fac242dd640 1 -- 172.21.15.204:0/2642682605 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.12"} v 0) v1 -- 0x7fac1c11d800 con 0x7fac1c09c560 2024-09-15T01:35:20.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.636+0000 7fac127fc640 1 -- 172.21.15.204:0/2642682605 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.12"}]=0 device 'osd.12' does not appear in the crush map v116) v1 ==== 126+0+0 (secure 0 0 0) 0x7fac18039070 con 0x7fac1c09c560 2024-09-15T01:35:20.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.636+0000 7fac127fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-09-15T01:35:20.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.636+0000 7fac127fc640 10 monclient: _finish_command 3 = system:0 device 'osd.12' does not appear in the crush map 2024-09-15T01:35:20.639 INFO:tasks.workunit.client.0.smithi204.stderr:device 'osd.12' does not appear in the crush map 2024-09-15T01:35:20.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 1 -- 172.21.15.204:0/2642682605 >> v2:172.21.15.204:6812/28210 conn(0x7fac000397b0 msgr2=0x7fac0003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:20.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 1 --2- 172.21.15.204:0/2642682605 >> v2:172.21.15.204:6812/28210 conn(0x7fac000397b0 0x7fac0003bc60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0a3c00 tx=0x7fac0c0029e0 comp rx=0 tx=0).stop 2024-09-15T01:35:20.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 10 monclient: shutdown 2024-09-15T01:35:20.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:20.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 1 -- 172.21.15.204:0/2642682605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac1c09c560 msgr2=0x7fac1c1315d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:20.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 1 --2- 172.21.15.204:0/2642682605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac1c09c560 0x7fac1c1315d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fac180097e0 tx=0x7fac18034000 comp rx=0 tx=0).stop 2024-09-15T01:35:20.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 1 -- 172.21.15.204:0/2642682605 shutdown_connections 2024-09-15T01:35:20.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 1 --2- 172.21.15.204:0/2642682605 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c0a35e0 0x7fac1c11b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:20.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 1 --2- 172.21.15.204:0/2642682605 >> v2:172.21.15.204:6812/28210 conn(0x7fac000397b0 0x7fac0003bc60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:20.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 1 --2- 172.21.15.204:0/2642682605 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fac1c09c950 0x7fac1c11b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:20.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 1 --2- 172.21.15.204:0/2642682605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac1c09c560 0x7fac1c1315d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:20.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 1 -- 172.21.15.204:0/2642682605 >> 172.21.15.204:0/2642682605 conn(0x7fac1c0687e0 msgr2=0x7fac1c06d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:20.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 1 -- 172.21.15.204:0/2642682605 shutdown_connections 2024-09-15T01:35:20.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.638+0000 7fac242dd640 1 -- 172.21.15.204:0/2642682605 wait complete. 2024-09-15T01:35:20.650 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm osd.14 2024-09-15T01:35:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.722+0000 7f787b554640 10 monclient: get_monmap_and_config 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.722+0000 7f787b554640 10 monclient: build_initial_monmap 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.722+0000 7f787b554640 10 monclient: monmap: 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:20.723760+0000 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:20.723760+0000 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:20.726 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:20.726 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:20.726 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:20.726 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:20.726 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:20.726 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:20.726 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.722+0000 7f787b554640 1 Processor -- start 2024-09-15T01:35:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 1 -- start start 2024-09-15T01:35:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 10 monclient: init 2024-09-15T01:35:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 10 monclient: _add_conns ranks=[1,6,7,8,2,5,9,0,3,10,4] 2024-09-15T01:35:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f78740a2e30 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f787409fb80 0x7f787409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 10 monclient(hunting): picked mon.noname-d con 0x7f787409fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7874137d20 0x7f787413a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 10 monclient(hunting): picked mon.noname-i con 0x7f7874137d20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787404b720 con 0x7f78740a2e30 2024-09-15T01:35:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f787409e6c0 con 0x7f787409fb80 2024-09-15T01:35:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f78740a04b0 con 0x7f7874137d20 2024-09-15T01:35:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f787b554640 10 monclient(hunting): authenticate will time out at 936.366554s 2024-09-15T01:35:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f7879af1640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7874137d20 msgr2=0x7f787413a100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f7879af1640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7874137d20 0x7f787413a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f7878aef640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f787409fb80 msgr2=0x7f787409ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f78792f0640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f78740a2e30 0x7f7874132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:34178/0 (socket says 172.21.15.204:34178) 2024-09-15T01:35:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f7878aef640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f787409fb80 0x7f787409ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f78792f0640 1 -- 172.21.15.204:0/438772225 learned_addr learned my addr 172.21.15.204:0/438772225 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355546723 0 0) 0x7f787404b720 con 0x7f78740a2e30 2024-09-15T01:35:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f786bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f786bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f786bfff640 10 monclient(hunting): my global_id is 6112 2024-09-15T01:35:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.723+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78600034e0 con 0x7f78740a2e30 2024-09-15T01:35:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1165881431 0 0) 0x7f78600034e0 con 0x7f78740a2e30 2024-09-15T01:35:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7874137d20 msgr2=0x7f787413a100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 --2- 172.21.15.204:0/438772225 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7874137d20 0x7f787413a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f787409fb80 msgr2=0x7f787409ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 --2- 172.21.15.204:0/438772225 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f787409fb80 0x7f787409ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7874133170 con 0x7f78740a2e30 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient: _finish_auth 0 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f78700018f0 con 0x7f78740a2e30 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f787b554640 5 monclient: authenticate success, global_id 6112 2024-09-15T01:35:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f787b554640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:35:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient: dump: 2024-09-15T01:35:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 >> v1:172.21.15.204:6789/0 conn(0x7f78740a2e30 legacy=0x7f7874132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient: _add_conns ranks=[2,7,8,1,5,6,0,4,3] 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 --2- 172.21.15.204:0/438772225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7860004250 0x7f7860004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient(hunting): picked mon.b con 0x7f7860004250 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 --2- 172.21.15.204:0/438772225 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7874137d20 0x7f7860004f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient(hunting): picked mon.i con 0x7f7874137d20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 --2- 172.21.15.204:0/438772225 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78600099c0 0x7f786000bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient(hunting): picked mon.e con 0x7f78600099c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787409e6c0 con 0x7f7860004250 2024-09-15T01:35:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f78740a04b0 con 0x7f7874137d20 2024-09-15T01:35:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f786000c5b0 con 0x7f78600099c0 2024-09-15T01:35:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f786bfff640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f787b554640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7878aef640 1 -- 172.21.15.204:0/438772225 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7874137d20 msgr2=0x7f7860004f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f78792f0640 1 -- 172.21.15.204:0/438772225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7860004250 msgr2=0x7f7860004780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f78792f0640 1 --2- 172.21.15.204:0/438772225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7860004250 0x7f7860004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7878aef640 1 --2- 172.21.15.204:0/438772225 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7874137d20 0x7f7860004f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 1 --2- 172.21.15.204:0/438772225 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78600099c0 0x7f786000bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 10 monclient(hunting): get_auth_request con 0x7f78600099c0 auth_method 0 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 10 monclient(hunting): handle_auth_done global_id 6112 payload 293 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 1 -- 172.21.15.204:0/438772225 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7874137d20 msgr2=0x7f7860004f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 1 --2- 172.21.15.204:0/438772225 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7874137d20 0x7f7860004f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 1 -- 172.21.15.204:0/438772225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7860004250 msgr2=0x7f7860004780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 1 --2- 172.21.15.204:0/438772225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7860004250 0x7f7860004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 10 monclient: _finish_hunting 0 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 1 monclient: found mon.e 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 1 -- 172.21.15.204:0/438772225 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78600034e0 con 0x7f78600099c0 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 10 monclient: _finish_auth 0 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.724+0000 7f7879af1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f7879af1640 1 --2- 172.21.15.204:0/438772225 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78600099c0 0x7f786000bda0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f78740a2290 tx=0x7f786c0095f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=5ae9b0215caf7209 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f786c02c940 con 0x7f78600099c0 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f786bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f786bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f786bfff640 10 monclient: dump: 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f787b554640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c02cae0 con 0x7f78600099c0 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f786bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f787b554640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f787b554640 4 set_mon_vals no callback set 2024-09-15T01:35:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f786bfff640 1 -- 172.21.15.204:0/438772225 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f786c02cc50 con 0x7f78600099c0 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f786bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f786bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.725+0000 7f786bfff640 10 monclient: dump: 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:20.748 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.726+0000 7f787b554640 10 monclient: shutdown 2024-09-15T01:35:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.726+0000 7f787b554640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.726+0000 7f787b554640 1 -- 172.21.15.204:0/438772225 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78600099c0 msgr2=0x7f786000bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.726+0000 7f787b554640 1 --2- 172.21.15.204:0/438772225 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f78600099c0 0x7f786000bda0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f78740a2290 tx=0x7f786c0095f0 comp rx=0 tx=0).stop 2024-09-15T01:35:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.726+0000 7f7879af1640 1 -- 172.21.15.204:0/438772225 reap_dead start 2024-09-15T01:35:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.726+0000 7f787b554640 1 -- 172.21.15.204:0/438772225 shutdown_connections 2024-09-15T01:35:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.726+0000 7f787b554640 1 -- 172.21.15.204:0/438772225 >> 172.21.15.204:0/438772225 conn(0x7f787406e860 msgr2=0x7f787406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.726+0000 7f787b554640 1 -- 172.21.15.204:0/438772225 shutdown_connections 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.726+0000 7f787b554640 1 -- 172.21.15.204:0/438772225 wait complete. 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.726+0000 7f787b554640 10 monclient: build_initial_monmap 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.726+0000 7f787b554640 10 monclient: monmap: 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 1 Processor -- start 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 1 -- start start 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 10 monclient: init 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 10 monclient: _add_conns ranks=[4,7,2,3,5,1,6,8,0] 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78600099c0 0x7f7874098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 10 monclient(hunting): picked mon.noname-c con 0x7f78600099c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f787409b650 0x7f7874099410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 10 monclient(hunting): picked mon.noname-i con 0x7f787409b650 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7874099950 0x7f7874099d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 10 monclient(hunting): picked mon.noname-b con 0x7f7874099950 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787409d650 con 0x7f7874099950 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f787409b250 con 0x7f78600099c0 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f787411c750 con 0x7f787409b650 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f787b554640 10 monclient(hunting): authenticate will time out at 936.370818s 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f7878aef640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f787409b650 msgr2=0x7f7874099410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f7878aef640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f787409b650 0x7f7874099410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f7879af1640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7874099950 msgr2=0x7f7874099d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.727+0000 7f7879af1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7874099950 0x7f7874099d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78600099c0 0x7f7874098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78600099c0 0x7f7874098ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:46338/0 (socket says 172.21.15.204:46338) 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 1 -- 172.21.15.204:0/4096807564 learned_addr learned my addr 172.21.15.204:0/4096807564 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 10 monclient(hunting): get_auth_request con 0x7f78600099c0 auth_method 0 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 10 monclient(hunting): handle_auth_done global_id 5512 payload 824 2024-09-15T01:35:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 1 -- 172.21.15.204:0/4096807564 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f787409b650 msgr2=0x7f7874099410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 1 --2- 172.21.15.204:0/4096807564 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f787409b650 0x7f7874099410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 1 -- 172.21.15.204:0/4096807564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7874099950 msgr2=0x7f7874099d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 1 --2- 172.21.15.204:0/4096807564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7874099950 0x7f7874099d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 10 monclient: _finish_hunting 0 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 1 monclient: found mon.noname-c 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 1 -- 172.21.15.204:0/4096807564 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787409a2a0 con 0x7f78600099c0 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78792f0640 1 --2- 172.21.15.204:0/4096807564 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78600099c0 0x7f7874098ed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f787409a490 tx=0x7f7870001e90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78697fa640 1 -- 172.21.15.204:0/4096807564 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78700075d0 con 0x7f78600099c0 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78697fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78697fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78697fa640 10 monclient: dump: 2024-09-15T01:35:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78697fa640 10 monclient: _finish_auth 0 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78697fa640 1 -- 172.21.15.204:0/4096807564 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78700047a0 con 0x7f78600099c0 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f787b554640 5 monclient: authenticate success, global_id 5512 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f787b554640 10 monclient: _renew_subs 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f787b554640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f787b554640 1 -- 172.21.15.204:0/4096807564 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787409c540 con 0x7f78600099c0 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78697fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78697fa640 1 -- 172.21.15.204:0/4096807564 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7870004910 con 0x7f78600099c0 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78697fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78697fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f78697fa640 10 monclient: dump: 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f786b7fe640 4 set_mon_vals no callback set 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f787b554640 10 monclient: _renew_subs 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f787b554640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.728+0000 7f787b554640 1 -- 172.21.15.204:0/4096807564 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787409c920 con 0x7f78600099c0 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.729+0000 7f78697fa640 1 -- 172.21.15.204:0/4096807564 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7870007150 con 0x7f78600099c0 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.729+0000 7f787b554640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.729+0000 7f787b554640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.729+0000 7f787b554640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.729+0000 7f787b554640 1 -- 172.21.15.204:0/4096807564 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78740008d0 con 0x7f78600099c0 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.729+0000 7f78697fa640 1 --2- 172.21.15.204:0/4096807564 >> v2:172.21.15.204:6812/28210 conn(0x7f78580397b0 0x7f785803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.729+0000 7f78697fa640 1 -- 172.21.15.204:0/4096807564 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f787004ac00 con 0x7f78600099c0 2024-09-15T01:35:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.729+0000 7f7878aef640 1 --2- 172.21.15.204:0/4096807564 >> v2:172.21.15.204:6812/28210 conn(0x7f78580397b0 0x7f785803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:20.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.730+0000 7f7878aef640 10 monclient: get_auth_request con 0x7f78580397b0 auth_method 0 2024-09-15T01:35:20.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.730+0000 7f7878aef640 1 --2- 172.21.15.204:0/4096807564 >> v2:172.21.15.204:6812/28210 conn(0x7f78580397b0 0x7f785803bc60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7860004700 tx=0x7f7864006d10 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:20.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.735+0000 7f78697fa640 1 -- 172.21.15.204:0/4096807564 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7870004a70 con 0x7f78600099c0 2024-09-15T01:35:20.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.735+0000 7f78697fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:20.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.735+0000 7f78697fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:20.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.896+0000 7f787b554640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.14"}] 2024-09-15T01:35:20.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.896+0000 7f787b554640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-09-15T01:35:20.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.896+0000 7f787b554640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:20.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:20.896+0000 7f787b554640 1 -- 172.21.15.204:0/4096807564 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.14"} v 0) v1 -- 0x7f7874070b50 con 0x7f78600099c0 2024-09-15T01:35:21.450 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.448+0000 7f78697fa640 1 -- 172.21.15.204:0/4096807564 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.14"}]=0 removed item id 14 name 'osd.14' from crush map v117) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7870018020 con 0x7f78600099c0 2024-09-15T01:35:21.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.448+0000 7f78697fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-09-15T01:35:21.451 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.448+0000 7f78697fa640 10 monclient: _finish_command 2 = system:0 removed item id 14 name 'osd.14' from crush map 2024-09-15T01:35:21.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.557+0000 7f787b554640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.14"}] 2024-09-15T01:35:21.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.557+0000 7f787b554640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-09-15T01:35:21.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.557+0000 7f787b554640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:21.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.557+0000 7f787b554640 1 -- 172.21.15.204:0/4096807564 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.14"} v 0) v1 -- 0x7f7874070ce0 con 0x7f78600099c0 2024-09-15T01:35:21.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.558+0000 7f78697fa640 1 -- 172.21.15.204:0/4096807564 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.14"}]=0 device 'osd.14' does not appear in the crush map v117) v1 ==== 126+0+0 (secure 0 0 0) 0x7f7870010300 con 0x7f78600099c0 2024-09-15T01:35:21.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.558+0000 7f78697fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-09-15T01:35:21.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.558+0000 7f78697fa640 10 monclient: _finish_command 3 = system:0 device 'osd.14' does not appear in the crush map 2024-09-15T01:35:21.561 INFO:tasks.workunit.client.0.smithi204.stderr:device 'osd.14' does not appear in the crush map 2024-09-15T01:35:21.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 1 -- 172.21.15.204:0/4096807564 >> v2:172.21.15.204:6812/28210 conn(0x7f78580397b0 msgr2=0x7f785803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:21.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 1 --2- 172.21.15.204:0/4096807564 >> v2:172.21.15.204:6812/28210 conn(0x7f78580397b0 0x7f785803bc60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7860004700 tx=0x7f7864006d10 comp rx=0 tx=0).stop 2024-09-15T01:35:21.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 10 monclient: shutdown 2024-09-15T01:35:21.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:21.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 1 -- 172.21.15.204:0/4096807564 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78600099c0 msgr2=0x7f7874098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:21.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 1 --2- 172.21.15.204:0/4096807564 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78600099c0 0x7f7874098ed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f787409a490 tx=0x7f7870001e90 comp rx=0 tx=0).stop 2024-09-15T01:35:21.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 1 -- 172.21.15.204:0/4096807564 shutdown_connections 2024-09-15T01:35:21.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 1 --2- 172.21.15.204:0/4096807564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7874099950 0x7f7874099d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:21.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 1 --2- 172.21.15.204:0/4096807564 >> v2:172.21.15.204:6812/28210 conn(0x7f78580397b0 0x7f785803bc60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:21.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 1 --2- 172.21.15.204:0/4096807564 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f787409b650 0x7f7874099410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:21.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 1 --2- 172.21.15.204:0/4096807564 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78600099c0 0x7f7874098ed0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:21.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 1 -- 172.21.15.204:0/4096807564 >> 172.21.15.204:0/4096807564 conn(0x7f787406e860 msgr2=0x7f787409ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:21.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 1 -- 172.21.15.204:0/4096807564 shutdown_connections 2024-09-15T01:35:21.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.560+0000 7f787b554640 1 -- 172.21.15.204:0/4096807564 wait complete. 2024-09-15T01:35:21.572 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd rm osd.12 2024-09-15T01:35:21.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.642+0000 7f40b555d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:21.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.642+0000 7f40b555d640 10 monclient: build_initial_monmap 2024-09-15T01:35:21.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.642+0000 7f40b555d640 10 monclient: monmap: 2024-09-15T01:35:21.657 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:21.657 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:21.657 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:21.643888+0000 2024-09-15T01:35:21.657 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:21.643888+0000 2024-09-15T01:35:21.657 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:21.657 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:21.657 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:21.657 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:21.658 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:21.658 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:21.658 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:21.658 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:21.658 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:21.658 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:21.658 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:21.658 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:21.658 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 1 Processor -- start 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 1 -- start start 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 10 monclient: init 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 10 monclient: _add_conns ranks=[7,8,1,10,4,6,0,3,9,2,5] 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f40b00a2400 0x7f40b00a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 10 monclient(hunting): picked mon.noname-i con 0x7f40b00a2400 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f40b009f990 0x7f40b009fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 10 monclient(hunting): picked mon.noname-e con 0x7f40b009f990 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f40b0132930 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b00648a0 con 0x7f40b0132930 2024-09-15T01:35:21.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f40b009e1a0 con 0x7f40b00a2400 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00a27d0 con 0x7f40b009f990 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40b555d640 10 monclient(hunting): authenticate will time out at 937.286665s 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40aeffd640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f40b00a2400 msgr2=0x7f40b00a0ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f40b00a2400 0x7f40b00a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40af7fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f40b0132930 0x7f40b0132d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:34182/0 (socket says 172.21.15.204:34182) 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.643+0000 7f40af7fe640 1 -- 172.21.15.204:0/4070558518 learned_addr learned my addr 172.21.15.204:0/4070558518 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f40b009f990 0x7f40b009fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933031700 0 0) 0x7f40b00648a0 con 0x7f40b0132930 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:21.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:21.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient(hunting): my global_id is 6121 2024-09-15T01:35:21.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f409c0034e0 con 0x7f40b0132930 2024-09-15T01:35:21.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): get_auth_request con 0x7f40b009f990 auth_method 0 2024-09-15T01:35:21.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:21.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:21.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:21.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:21.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:21.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1005055875 0 0) 0x7f409c0034e0 con 0x7f40b0132930 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f40b009f990 msgr2=0x7f40b009fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f40b009f990 0x7f40b009fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f40b00a2400 msgr2=0x7f40b00a0ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f40b00a2400 0x7f40b00a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b009dec0 con 0x7f40b0132930 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:21.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40b555d640 5 monclient: authenticate success, global_id 6121 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40b555d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f40a8001c70 con 0x7f40b0132930 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient: dump: 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:21.663 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:21.664 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:21.664 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:21.664 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:21.664 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:21.664 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:21.664 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:21.664 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:21.664 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:21.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:35:21.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 >> v1:172.21.15.204:6789/0 conn(0x7f40b0132930 legacy=0x7f40b0132d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient: _add_conns ranks=[4,0,6,8,2,3,7,5,1] 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f409c004220 0x7f409c004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient(hunting): picked mon.c con 0x7f409c004220 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409c004d10 0x7f409c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient(hunting): picked mon.a con 0x7f409c004d10 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f409c009c40 0x7f409c00c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient(hunting): picked mon.d con 0x7f409c009c40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b009e1a0 con 0x7f409c004d10 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00a27d0 con 0x7f409c004220 2024-09-15T01:35:21.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f409c00c9b0 con 0x7f409c009c40 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40ae7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40b555d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40af7fe640 1 -- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f409c009c40 msgr2=0x7f409c00c0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40af7fe640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f409c009c40 0x7f409c00c0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f409c004220 0x7f409c004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 10 monclient(hunting): get_auth_request con 0x7f409c004220 auth_method 0 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409c004d10 0x7f409c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:21.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): get_auth_request con 0x7f409c004d10 auth_method 0 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 10 monclient(hunting): handle_auth_done global_id 6121 payload 293 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 1 -- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f409c009c40 msgr2=0x7f409c00c0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f409c009c40 0x7f409c00c0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:21.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 1 -- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409c004d10 msgr2=0x7f409c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:21.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409c004d10 0x7f409c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:21.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 10 monclient: _finish_hunting 0 2024-09-15T01:35:21.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 1 monclient: found mon.c 2024-09-15T01:35:21.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40af7fe640 1 -- 172.21.15.204:0/4070558518 reap_dead start 2024-09-15T01:35:21.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:21.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:21.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 1 -- 172.21.15.204:0/4070558518 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c0034e0 con 0x7f409c004220 2024-09-15T01:35:21.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:21.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.644+0000 7f40aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:21.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40aeffd640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f409c004220 0x7f409c004780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40a00099c0 tx=0x7f40a0009060 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2978d924540cde3e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:21.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40a0003df0 con 0x7f409c004220 2024-09-15T01:35:21.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40ae7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40ae7fc640 10 monclient: dump: 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:21.669 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:21.670 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:21.670 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:21.670 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:21.670 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:21.670 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:21.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a002a930 con 0x7f409c004220 2024-09-15T01:35:21.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40b555d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:21.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40ae7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:21.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40b555d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:21.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40ae7fc640 1 -- 172.21.15.204:0/4070558518 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40a00050f0 con 0x7f409c004220 2024-09-15T01:35:21.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40b555d640 4 set_mon_vals no callback set 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40ae7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.645+0000 7f40ae7fc640 10 monclient: dump: 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:21.671 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:21.672 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:21.672 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:21.672 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:21.672 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:21.672 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:21.672 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:21.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.646+0000 7f40b555d640 10 monclient: shutdown 2024-09-15T01:35:21.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.646+0000 7f40b555d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:21.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.646+0000 7f40b555d640 1 -- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f409c004220 msgr2=0x7f409c004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:21.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.646+0000 7f40b555d640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f409c004220 0x7f409c004780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40a00099c0 tx=0x7f40a0009060 comp rx=0 tx=0).stop 2024-09-15T01:35:21.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.646+0000 7f40b555d640 1 -- 172.21.15.204:0/4070558518 shutdown_connections 2024-09-15T01:35:21.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.646+0000 7f40b555d640 1 --2- 172.21.15.204:0/4070558518 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f409c004220 0x7f409c004780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:21.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.646+0000 7f40b555d640 1 -- 172.21.15.204:0/4070558518 >> 172.21.15.204:0/4070558518 conn(0x7f40b006e860 msgr2=0x7f40b006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:21.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.646+0000 7f40b555d640 1 -- 172.21.15.204:0/4070558518 shutdown_connections 2024-09-15T01:35:21.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.646+0000 7f40b555d640 1 -- 172.21.15.204:0/4070558518 wait complete. 2024-09-15T01:35:21.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.646+0000 7f40b555d640 10 monclient: build_initial_monmap 2024-09-15T01:35:21.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.646+0000 7f40b555d640 10 monclient: monmap: 2024-09-15T01:35:21.673 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:21.673 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:21.673 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:21.673 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:21.673 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:21.673 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 1 Processor -- start 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 1 -- start start 2024-09-15T01:35:21.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 10 monclient: init 2024-09-15T01:35:21.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:21.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 10 monclient: _add_conns ranks=[0,4,2,1,6,7,5,3,8] 2024-09-15T01:35:21.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409c004220 0x7f40b0141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:21.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 10 monclient(hunting): picked mon.noname-a con 0x7f409c004220 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:21.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f40b0142350 0x7f40b0142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:21.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 10 monclient(hunting): picked mon.noname-c con 0x7f40b0142350 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:21.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b011d0e0 0x7f40b011d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:21.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 10 monclient(hunting): picked mon.noname-b con 0x7f40b011d0e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:21.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:21.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b0120d40 con 0x7f409c004220 2024-09-15T01:35:21.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:21.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b0142ca0 con 0x7f40b011d0e0 2024-09-15T01:35:21.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:21.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40b011ec30 con 0x7f40b0142350 2024-09-15T01:35:21.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:21.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40b555d640 10 monclient(hunting): authenticate will time out at 937.290829s 2024-09-15T01:35:21.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40af7fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b011d0e0 msgr2=0x7f40b011d4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:21.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.647+0000 7f40af7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b011d0e0 0x7f40b011d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:21.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f40b0142350 0x7f40b0142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:21.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409c004220 0x7f40b0141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:21.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f40b0142350 0x7f40b0142760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:46366/0 (socket says 172.21.15.204:46366) 2024-09-15T01:35:21.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409c004220 0x7f40b0141e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:43928/0 (socket says 172.21.15.204:43928) 2024-09-15T01:35:21.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 1 -- 172.21.15.204:0/2519099534 learned_addr learned my addr 172.21.15.204:0/2519099534 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:21.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 10 monclient(hunting): get_auth_request con 0x7f40b0142350 auth_method 0 2024-09-15T01:35:21.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:21.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:21.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40aeffd640 10 monclient(hunting): get_auth_request con 0x7f409c004220 auth_method 0 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 10 monclient(hunting): handle_auth_done global_id 5521 payload 824 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 1 -- 172.21.15.204:0/2519099534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b011d0e0 msgr2=0x7f40b011d4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:21.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 1 --2- 172.21.15.204:0/2519099534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b011d0e0 0x7f40b011d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 1 -- 172.21.15.204:0/2519099534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409c004220 msgr2=0x7f40b0141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 1 --2- 172.21.15.204:0/2519099534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409c004220 0x7f40b0141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 1 monclient: found mon.noname-c 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 1 -- 172.21.15.204:0/2519099534 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b009d3d0 con 0x7f40b0142350 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40aeffd640 1 --2- 172.21.15.204:0/2519099534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409c004220 0x7f40b0141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a7fff640 1 --2- 172.21.15.204:0/2519099534 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f40b0142350 0x7f40b0142760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4098011a00 tx=0x7f40980090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a6ffd640 1 -- 172.21.15.204:0/2519099534 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4098013b00 con 0x7f40b0142350 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:35:21.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a6ffd640 10 monclient: dump: 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:21.680 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:21.681 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:21.681 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:21.681 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:21.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:21.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:21.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40b555d640 5 monclient: authenticate success, global_id 5521 2024-09-15T01:35:21.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a6ffd640 1 -- 172.21.15.204:0/2519099534 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098013c60 con 0x7f40b0142350 2024-09-15T01:35:21.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40b555d640 10 monclient: _renew_subs 2024-09-15T01:35:21.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40b555d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:21.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40b555d640 1 -- 172.21.15.204:0/2519099534 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b009caf0 con 0x7f40b0142350 2024-09-15T01:35:21.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:21.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a6ffd640 1 -- 172.21.15.204:0/2519099534 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4098013dd0 con 0x7f40b0142350 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40a6ffd640 10 monclient: dump: 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:21.682 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40adffb640 4 set_mon_vals no callback set 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40b555d640 10 monclient: _renew_subs 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40b555d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.648+0000 7f40b555d640 1 -- 172.21.15.204:0/2519099534 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b01206f0 con 0x7f40b0142350 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.649+0000 7f40a6ffd640 1 -- 172.21.15.204:0/2519099534 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4098013680 con 0x7f40b0142350 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.649+0000 7f40b555d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.649+0000 7f40b555d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:21.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.649+0000 7f40b555d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:21.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.649+0000 7f40b555d640 1 -- 172.21.15.204:0/2519099534 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b006ed10 con 0x7f40b0142350 2024-09-15T01:35:21.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.649+0000 7f40a6ffd640 1 --2- 172.21.15.204:0/2519099534 >> v2:172.21.15.204:6812/28210 conn(0x7f40900397b0 0x7f409003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:21.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.649+0000 7f40a6ffd640 1 -- 172.21.15.204:0/2519099534 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5683+0+0 (secure 0 0 0) 0x7f4098018030 con 0x7f40b0142350 2024-09-15T01:35:21.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.649+0000 7f40aeffd640 1 --2- 172.21.15.204:0/2519099534 >> v2:172.21.15.204:6812/28210 conn(0x7f40900397b0 0x7f409003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:21.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.649+0000 7f40aeffd640 10 monclient: get_auth_request con 0x7f40900397b0 auth_method 0 2024-09-15T01:35:21.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.650+0000 7f40aeffd640 1 --2- 172.21.15.204:0/2519099534 >> v2:172.21.15.204:6812/28210 conn(0x7f40900397b0 0x7f409003bc60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f40a0032000 tx=0x7f40a0030000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:21.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.654+0000 7f40a6ffd640 1 -- 172.21.15.204:0/2519099534 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4098024300 con 0x7f40b0142350 2024-09-15T01:35:21.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.654+0000 7f40a6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:21.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.654+0000 7f40a6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.820+0000 7f40b555d640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-09-15T01:35:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.820+0000 7f40b555d640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-09-15T01:35:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.820+0000 7f40b555d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:21.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:21.820+0000 7f40b555d640 1 -- 172.21.15.204:0/2519099534 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.12"]} v 0) v1 -- 0x7f40b0121a40 con 0x7f40b0142350 2024-09-15T01:35:21.824 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:35:21.821+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e117 definitely_dead 0 2024-09-15T01:35:22.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.452+0000 7f40a6ffd640 1 -- 172.21.15.204:0/2519099534 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.12"]}]=0 removed osd.12 v118) v1 ==== 87+0+0 (secure 0 0 0) 0x7f409801b340 con 0x7f40b0142350 2024-09-15T01:35:22.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.452+0000 7f40a6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-09-15T01:35:22.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.452+0000 7f40a6ffd640 10 monclient: _finish_command 2 = system:0 removed osd.12 2024-09-15T01:35:22.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.560+0000 7f40b555d640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-09-15T01:35:22.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.560+0000 7f40b555d640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-09-15T01:35:22.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.560+0000 7f40b555d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:22.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.560+0000 7f40b555d640 1 -- 172.21.15.204:0/2519099534 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.12"]} v 0) v1 -- 0x7f40b0070b80 con 0x7f40b0142350 2024-09-15T01:35:22.563 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:35:22.561+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e118 definitely_dead 0 2024-09-15T01:35:22.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.561+0000 7f40a6ffd640 1 -- 172.21.15.204:0/2519099534 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.12"]}]=0 osd.12 does not exist. v118) v1 ==== 96+0+0 (secure 0 0 0) 0x7f409801b990 con 0x7f40b0142350 2024-09-15T01:35:22.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.561+0000 7f40a6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-09-15T01:35:22.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.561+0000 7f40a6ffd640 10 monclient: _finish_command 3 = system:0 osd.12 does not exist. 2024-09-15T01:35:22.564 INFO:tasks.workunit.client.0.smithi204.stderr:osd.12 does not exist. 2024-09-15T01:35:22.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 1 -- 172.21.15.204:0/2519099534 >> v2:172.21.15.204:6812/28210 conn(0x7f40900397b0 msgr2=0x7f409003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:22.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 1 --2- 172.21.15.204:0/2519099534 >> v2:172.21.15.204:6812/28210 conn(0x7f40900397b0 0x7f409003bc60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f40a0032000 tx=0x7f40a0030000 comp rx=0 tx=0).stop 2024-09-15T01:35:22.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 10 monclient: shutdown 2024-09-15T01:35:22.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:22.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 1 -- 172.21.15.204:0/2519099534 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f40b0142350 msgr2=0x7f40b0142760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:22.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 1 --2- 172.21.15.204:0/2519099534 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f40b0142350 0x7f40b0142760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4098011a00 tx=0x7f40980090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 1 -- 172.21.15.204:0/2519099534 shutdown_connections 2024-09-15T01:35:22.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 1 --2- 172.21.15.204:0/2519099534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40b011d0e0 0x7f40b011d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 1 --2- 172.21.15.204:0/2519099534 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f40b0142350 0x7f40b0142760 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 1 --2- 172.21.15.204:0/2519099534 >> v2:172.21.15.204:6812/28210 conn(0x7f40900397b0 0x7f409003bc60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 1 --2- 172.21.15.204:0/2519099534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f409c004220 0x7f40b0141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 1 -- 172.21.15.204:0/2519099534 >> 172.21.15.204:0/2519099534 conn(0x7f40b006e860 msgr2=0x7f40b00a3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:22.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 1 -- 172.21.15.204:0/2519099534 shutdown_connections 2024-09-15T01:35:22.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.564+0000 7f40b555d640 1 -- 172.21.15.204:0/2519099534 wait complete. 2024-09-15T01:35:22.575 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd rm osd.14 2024-09-15T01:35:22.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.645+0000 7f791ab14640 10 monclient: get_monmap_and_config 2024-09-15T01:35:22.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.645+0000 7f791ab14640 10 monclient: build_initial_monmap 2024-09-15T01:35:22.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.645+0000 7f791ab14640 10 monclient: monmap: 2024-09-15T01:35:22.648 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:22.648 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:22.648 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:22.647080+0000 2024-09-15T01:35:22.648 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:22.647080+0000 2024-09-15T01:35:22.648 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:22.649 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:22.650 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:22.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 1 Processor -- start 2024-09-15T01:35:22.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 1 -- start start 2024-09-15T01:35:22.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 10 monclient: init 2024-09-15T01:35:22.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:22.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 10 monclient: _add_conns ranks=[5,3,7,8,6,2,0,1,9,10,4] 2024-09-15T01:35:22.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a2400 0x7f79140a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:22.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 10 monclient(hunting): picked mon.noname-h con 0x7f79140a2400 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:22.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791409f990 0x7f791409fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:22.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 10 monclient(hunting): picked mon.noname-g con 0x7f791409f990 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:22.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914132930 0x7f7914132d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:22.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 10 monclient(hunting): picked mon.noname-i con 0x7f7914132930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:22.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:22.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791409e1a0 con 0x7f791409f990 2024-09-15T01:35:22.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:22.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79140a27d0 con 0x7f79140a2400 2024-09-15T01:35:22.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:22.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f791409dec0 con 0x7f7914132930 2024-09-15T01:35:22.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:22.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f791ab14640 10 monclient(hunting): authenticate will time out at 938.289822s 2024-09-15T01:35:22.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f79190b1640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914132930 msgr2=0x7f7914132d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:22.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f79190b1640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914132930 0x7f7914132d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:22.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f7913fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791409f990 0x7f791409fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f79188b0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a2400 0x7f79140a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f79188b0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a2400 0x7f79140a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49730/0 (socket says 172.21.15.204:49730) 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f79188b0640 1 -- 172.21.15.204:0/1971200887 learned_addr learned my addr 172.21.15.204:0/1971200887 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f79188b0640 10 monclient(hunting): get_auth_request con 0x7f79140a2400 auth_method 0 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.646+0000 7f7913fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791409f990 0x7f791409fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:40256/0 (socket says 172.21.15.204:40256) 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f79188b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f79188b0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f79188b0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 10 monclient(hunting): get_auth_request con 0x7f791409f990 auth_method 0 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:22.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f79188b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f79188b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f79188b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 10 monclient(hunting): handle_auth_done global_id 14682 payload 274 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 1 -- 172.21.15.204:0/1971200887 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914132930 msgr2=0x7f7914132d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 1 --2- 172.21.15.204:0/1971200887 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914132930 0x7f7914132d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 1 -- 172.21.15.204:0/1971200887 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a2400 msgr2=0x7f79140a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 1 --2- 172.21.15.204:0/1971200887 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a2400 0x7f79140a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 1 monclient: found mon.noname-g 2024-09-15T01:35:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 1 -- 172.21.15.204:0/1971200887 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79140648a0 con 0x7f791409f990 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f79188b0640 1 --2- 172.21.15.204:0/1971200887 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a2400 0x7f79140a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.647+0000 7f7913fff640 1 --2- 172.21.15.204:0/1971200887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791409f990 0x7f791409fd60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7900009880 tx=0x7f79000090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=8785245a4e18e691 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f79137fe640 1 -- 172.21.15.204:0/1971200887 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f790002abd0 con 0x7f791409f990 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f79137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f79137fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f79137fe640 10 monclient: dump: 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:22.662 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f79137fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:22.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f79137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:22.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f79137fe640 1 -- 172.21.15.204:0/1971200887 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790002ad50 con 0x7f791409f990 2024-09-15T01:35:22.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 5 monclient: authenticate success, global_id 14682 2024-09-15T01:35:22.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:22.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f79137fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:22.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f79137fe640 1 -- 172.21.15.204:0/1971200887 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7900031ca0 con 0x7f791409f990 2024-09-15T01:35:22.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:22.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 4 set_mon_vals no callback set 2024-09-15T01:35:22.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f79137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:22.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f79137fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:22.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f79137fe640 10 monclient: dump: 2024-09-15T01:35:22.664 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:22.664 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:22.665 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:22.666 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:22.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 10 monclient: shutdown 2024-09-15T01:35:22.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:22.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 1 -- 172.21.15.204:0/1971200887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791409f990 msgr2=0x7f791409fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:22.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 1 --2- 172.21.15.204:0/1971200887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791409f990 0x7f791409fd60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7900009880 tx=0x7f79000090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 1 -- 172.21.15.204:0/1971200887 shutdown_connections 2024-09-15T01:35:22.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 1 --2- 172.21.15.204:0/1971200887 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914132930 0x7f7914132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 1 --2- 172.21.15.204:0/1971200887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791409f990 0x7f791409fd60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 1 --2- 172.21.15.204:0/1971200887 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a2400 0x7f79140a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 1 -- 172.21.15.204:0/1971200887 >> 172.21.15.204:0/1971200887 conn(0x7f791406e860 msgr2=0x7f791406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:22.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 1 -- 172.21.15.204:0/1971200887 shutdown_connections 2024-09-15T01:35:22.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.648+0000 7f791ab14640 1 -- 172.21.15.204:0/1971200887 wait complete. 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.649+0000 7f791ab14640 10 monclient: build_initial_monmap 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.649+0000 7f791ab14640 10 monclient: monmap: 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:22.667 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:22.668 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:22.668 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:22.668 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:22.668 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:22.668 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:22.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.649+0000 7f791ab14640 1 Processor -- start 2024-09-15T01:35:22.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.649+0000 7f791ab14640 1 -- start start 2024-09-15T01:35:22.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.649+0000 7f791ab14640 10 monclient: init 2024-09-15T01:35:22.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:22.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 10 monclient: _add_conns ranks=[8,1,4,0,6,7,2,5,3] 2024-09-15T01:35:22.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f791409f990 0x7f791411d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 10 monclient(hunting): picked mon.noname-e con 0x7f791409f990 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79140a2400 0x7f791411d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 10 monclient(hunting): picked mon.noname-f con 0x7f79140a2400 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7914132930 0x7f791411dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 10 monclient(hunting): picked mon.noname-c con 0x7f7914132930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79141217d0 con 0x7f79140a2400 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7914120920 con 0x7f7914132930 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f791409cab0 con 0x7f791409f990 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:22.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 10 monclient(hunting): authenticate will time out at 938.293239s 2024-09-15T01:35:22.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79140a2400 0x7f791411d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:22.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79140a2400 0x7f791411d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36034/0 (socket says 172.21.15.204:36034) 2024-09-15T01:35:22.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 1 -- 172.21.15.204:0/2144671428 learned_addr learned my addr 172.21.15.204:0/2144671428 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:22.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 10 monclient(hunting): get_auth_request con 0x7f79140a2400 auth_method 0 2024-09-15T01:35:22.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:22.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:22.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:22.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:22.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:22.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 10 monclient(hunting): handle_auth_done global_id 6130 payload 824 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f79188b0640 1 --2- 172.21.15.204:0/2144671428 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f791409f990 0x7f791411d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f79190b1640 1 --2- 172.21.15.204:0/2144671428 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7914132930 0x7f791411dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 1 -- 172.21.15.204:0/2144671428 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f791409f990 msgr2=0x7f791411d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 1 --2- 172.21.15.204:0/2144671428 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f791409f990 0x7f791411d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 1 -- 172.21.15.204:0/2144671428 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7914132930 msgr2=0x7f791411dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 1 --2- 172.21.15.204:0/2144671428 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7914132930 0x7f791411dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 1 monclient: found mon.noname-f 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 1 -- 172.21.15.204:0/2144671428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7908009450 con 0x7f79140a2400 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f79188b0640 10 monclient: get_auth_request con 0x7f791409f990 auth_method 0 2024-09-15T01:35:22.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f79190b1640 10 monclient: get_auth_request con 0x7f7914132930 auth_method 0 2024-09-15T01:35:22.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f79188b0640 1 --2- 172.21.15.204:0/2144671428 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f791409f990 0x7f791411d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:35:22.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f79190b1640 1 --2- 172.21.15.204:0/2144671428 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7914132930 0x7f791411dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:35:22.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7913fff640 1 --2- 172.21.15.204:0/2144671428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79140a2400 0x7f791411d840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7900009360 tx=0x7f7900032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:22.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7910ff9640 1 -- 172.21.15.204:0/2144671428 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f790003b870 con 0x7f79140a2400 2024-09-15T01:35:22.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7910ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:22.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7910ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:22.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7910ff9640 10 monclient: dump: 2024-09-15T01:35:22.672 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:22.672 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:22.673 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7910ff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7910ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7910ff9640 1 -- 172.21.15.204:0/2144671428 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790003b9f0 con 0x7f79140a2400 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 5 monclient: authenticate success, global_id 6130 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7910ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 10 monclient: _renew_subs 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7910ff9640 1 -- 172.21.15.204:0/2144671428 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79000313a0 con 0x7f79140a2400 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f791ab14640 1 -- 172.21.15.204:0/2144671428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7900009390 con 0x7f79140a2400 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7912ffd640 4 set_mon_vals no callback set 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7910ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7910ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:22.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.650+0000 7f7910ff9640 10 monclient: dump: 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:22.675 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:22.676 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:22.676 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:22.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.651+0000 7f791ab14640 10 monclient: _renew_subs 2024-09-15T01:35:22.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.651+0000 7f791ab14640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:22.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.651+0000 7f791ab14640 1 -- 172.21.15.204:0/2144671428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7914120640 con 0x7f79140a2400 2024-09-15T01:35:22.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.651+0000 7f7910ff9640 1 -- 172.21.15.204:0/2144671428 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7900039040 con 0x7f79140a2400 2024-09-15T01:35:22.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.651+0000 7f791ab14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:22.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.651+0000 7f7910ff9640 1 --2- 172.21.15.204:0/2144671428 >> v2:172.21.15.204:6812/28210 conn(0x7f78f80397b0 0x7f78f803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:22.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.651+0000 7f791ab14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:22.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.651+0000 7f791ab14640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:22.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.651+0000 7f791ab14640 1 -- 172.21.15.204:0/2144671428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7914070b80 con 0x7f79140a2400 2024-09-15T01:35:22.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.651+0000 7f7910ff9640 1 -- 172.21.15.204:0/2144671428 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5683+0+0 (secure 0 0 0) 0x7f790006b460 con 0x7f79140a2400 2024-09-15T01:35:22.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.652+0000 7f79188b0640 1 --2- 172.21.15.204:0/2144671428 >> v2:172.21.15.204:6812/28210 conn(0x7f78f80397b0 0x7f78f803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:22.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.652+0000 7f79188b0640 10 monclient: get_auth_request con 0x7f78f80397b0 auth_method 0 2024-09-15T01:35:22.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.652+0000 7f79188b0640 1 --2- 172.21.15.204:0/2144671428 >> v2:172.21.15.204:6812/28210 conn(0x7f78f80397b0 0x7f78f803bc60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7908009420 tx=0x7f7908008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:22.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.656+0000 7f7910ff9640 1 -- 172.21.15.204:0/2144671428 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7900031520 con 0x7f79140a2400 2024-09-15T01:35:22.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.656+0000 7f7910ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:22.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.656+0000 7f7910ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:22.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.821+0000 7f791ab14640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-09-15T01:35:22.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.821+0000 7f791ab14640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-09-15T01:35:22.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.821+0000 7f791ab14640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:22.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:22.821+0000 7f791ab14640 1 -- 172.21.15.204:0/2144671428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.14"]} v 0) v1 -- 0x7f791409e480 con 0x7f79140a2400 2024-09-15T01:35:22.824 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:35:22.821+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e118 definitely_dead 0 2024-09-15T01:35:23.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.548+0000 7f7910ff9640 1 -- 172.21.15.204:0/2144671428 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.14"]}]=0 removed osd.14 v119) v1 ==== 87+0+0 (secure 0 0 0) 0x7f7900030340 con 0x7f79140a2400 2024-09-15T01:35:23.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.548+0000 7f7910ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-09-15T01:35:23.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.548+0000 7f7910ff9640 10 monclient: _finish_command 2 = system:0 removed osd.14 2024-09-15T01:35:23.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.652+0000 7f791ab14640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-09-15T01:35:23.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.652+0000 7f791ab14640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-09-15T01:35:23.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.652+0000 7f791ab14640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:23.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.652+0000 7f791ab14640 1 -- 172.21.15.204:0/2144671428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.14"]} v 0) v1 -- 0x7f7914120b60 con 0x7f79140a2400 2024-09-15T01:35:23.656 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:35:23.653+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e119 definitely_dead 0 2024-09-15T01:35:23.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.653+0000 7f7910ff9640 1 -- 172.21.15.204:0/2144671428 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.14"]}]=0 osd.14 does not exist. v119) v1 ==== 96+0+0 (secure 0 0 0) 0x7f7900042070 con 0x7f79140a2400 2024-09-15T01:35:23.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.653+0000 7f7910ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-09-15T01:35:23.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.653+0000 7f7910ff9640 10 monclient: _finish_command 3 = system:0 osd.14 does not exist. 2024-09-15T01:35:23.656 INFO:tasks.workunit.client.0.smithi204.stderr:osd.14 does not exist. 2024-09-15T01:35:23.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 1 -- 172.21.15.204:0/2144671428 >> v2:172.21.15.204:6812/28210 conn(0x7f78f80397b0 msgr2=0x7f78f803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:23.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 1 --2- 172.21.15.204:0/2144671428 >> v2:172.21.15.204:6812/28210 conn(0x7f78f80397b0 0x7f78f803bc60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7908009420 tx=0x7f7908008040 comp rx=0 tx=0).stop 2024-09-15T01:35:23.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 10 monclient: shutdown 2024-09-15T01:35:23.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:23.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 1 -- 172.21.15.204:0/2144671428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79140a2400 msgr2=0x7f791411d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:23.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 1 --2- 172.21.15.204:0/2144671428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79140a2400 0x7f791411d840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7900009360 tx=0x7f7900032000 comp rx=0 tx=0).stop 2024-09-15T01:35:23.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 1 -- 172.21.15.204:0/2144671428 shutdown_connections 2024-09-15T01:35:23.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 1 --2- 172.21.15.204:0/2144671428 >> v2:172.21.15.204:6812/28210 conn(0x7f78f80397b0 0x7f78f803bc60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:23.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 1 --2- 172.21.15.204:0/2144671428 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7914132930 0x7f791411dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:23.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 1 --2- 172.21.15.204:0/2144671428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79140a2400 0x7f791411d840 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:23.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 1 --2- 172.21.15.204:0/2144671428 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f791409f990 0x7f791411d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:23.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 1 -- 172.21.15.204:0/2144671428 >> 172.21.15.204:0/2144671428 conn(0x7f791406e860 msgr2=0x7f791409fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:23.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 1 -- 172.21.15.204:0/2144671428 shutdown_connections 2024-09-15T01:35:23.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.655+0000 7f791ab14640 1 -- 172.21.15.204:0/2144671428 wait complete. 2024-09-15T01:35:23.666 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd set-require-min-compat-client luminous 2024-09-15T01:35:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.738+0000 7f5b3215d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.738+0000 7f5b3215d640 10 monclient: build_initial_monmap 2024-09-15T01:35:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.738+0000 7f5b3215d640 10 monclient: monmap: 2024-09-15T01:35:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:23.740 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:23.739574+0000 2024-09-15T01:35:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:23.739574+0000 2024-09-15T01:35:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:23.741 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:23.742 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:23.742 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.738+0000 7f5b3215d640 1 Processor -- start 2024-09-15T01:35:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.738+0000 7f5b3215d640 1 -- start start 2024-09-15T01:35:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.738+0000 7f5b3215d640 10 monclient: init 2024-09-15T01:35:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 10 monclient: _add_conns ranks=[2,8,10,4,3,9,7,6,0,1,5] 2024-09-15T01:35:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c067130 0x7f5b2c09a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 10 monclient(hunting): picked mon.noname-b con 0x7f5b2c067130 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5b2c067500 0x7f5b2c0678d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:23.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 10 monclient(hunting): picked mon.noname-e con 0x7f5b2c067500 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5b2c130a70 0x7f5b2c130ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 10 monclient(hunting): picked mon.noname-f con 0x7f5b2c130a70 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2c098f20 con 0x7f5b2c130a70 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c099e00 con 0x7f5b2c067130 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c099810 con 0x7f5b2c067500 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b3215d640 10 monclient(hunting): authenticate will time out at 939.382504s 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5b2c130a70 0x7f5b2c130ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2b7fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c067130 msgr2=0x7f5b2c09a760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2b7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c067130 0x7f5b2c09a760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:23.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5b2c130a70 0x7f5b2c130ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36048/0 (socket says 172.21.15.204:36048) 2024-09-15T01:35:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 1 -- 172.21.15.204:0/1806433272 learned_addr learned my addr 172.21.15.204:0/1806433272 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 10 monclient(hunting): get_auth_request con 0x7f5b2c130a70 auth_method 0 2024-09-15T01:35:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b23fff640 1 --2- 172.21.15.204:0/1806433272 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5b2c067500 0x7f5b2c0678d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b23fff640 10 monclient(hunting): get_auth_request con 0x7f5b2c067500 auth_method 0 2024-09-15T01:35:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b23fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:23.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b23fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b23fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b23fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b23fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b23fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 10 monclient(hunting): handle_auth_done global_id 6139 payload 274 2024-09-15T01:35:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 1 -- 172.21.15.204:0/1806433272 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5b2c067500 msgr2=0x7f5b2c0678d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 1 --2- 172.21.15.204:0/1806433272 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5b2c067500 0x7f5b2c0678d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 1 -- 172.21.15.204:0/1806433272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c067130 msgr2=0x7f5b2c09a760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 1 --2- 172.21.15.204:0/1806433272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c067130 0x7f5b2c09a760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:23.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 1 monclient: found mon.noname-f 2024-09-15T01:35:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.739+0000 7f5b2bfff640 1 -- 172.21.15.204:0/1806433272 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0648a0 con 0x7f5b2c130a70 2024-09-15T01:35:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2bfff640 1 --2- 172.21.15.204:0/1806433272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b2c130a70 0x7f5b2c130ec0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009f20 tx=0x7f5b24009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e02592b204fcbfb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2affd640 1 -- 172.21.15.204:0/1806433272 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b24007880 con 0x7f5b2c130a70 2024-09-15T01:35:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2affd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2affd640 10 monclient: dump: 2024-09-15T01:35:23.755 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:23.756 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:23.757 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2affd640 10 monclient: _finish_auth 0 2024-09-15T01:35:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:23.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b3215d640 5 monclient: authenticate success, global_id 6139 2024-09-15T01:35:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2affd640 1 -- 172.21.15.204:0/1806433272 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2402d050 con 0x7f5b2c130a70 2024-09-15T01:35:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b3215d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b3215d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b3215d640 4 set_mon_vals no callback set 2024-09-15T01:35:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2affd640 1 -- 172.21.15.204:0/1806433272 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b24030360 con 0x7f5b2c130a70 2024-09-15T01:35:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.740+0000 7f5b2affd640 10 monclient: dump: 2024-09-15T01:35:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:23.758 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:23.759 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:23.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 10 monclient: shutdown 2024-09-15T01:35:23.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:23.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 1 -- 172.21.15.204:0/1806433272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b2c130a70 msgr2=0x7f5b2c130ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:23.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 1 --2- 172.21.15.204:0/1806433272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b2c130a70 0x7f5b2c130ec0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009f20 tx=0x7f5b24009840 comp rx=0 tx=0).stop 2024-09-15T01:35:23.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 1 -- 172.21.15.204:0/1806433272 shutdown_connections 2024-09-15T01:35:23.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 1 --2- 172.21.15.204:0/1806433272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b2c130a70 0x7f5b2c130ec0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:23.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 1 --2- 172.21.15.204:0/1806433272 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5b2c067500 0x7f5b2c0678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:23.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 1 --2- 172.21.15.204:0/1806433272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c067130 0x7f5b2c09a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:23.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 1 -- 172.21.15.204:0/1806433272 >> 172.21.15.204:0/1806433272 conn(0x7f5b2c0a2ce0 msgr2=0x7f5b2c0a0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:23.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 1 -- 172.21.15.204:0/1806433272 shutdown_connections 2024-09-15T01:35:23.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 1 -- 172.21.15.204:0/1806433272 wait complete. 2024-09-15T01:35:23.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 10 monclient: build_initial_monmap 2024-09-15T01:35:23.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.741+0000 7f5b3215d640 10 monclient: monmap: 2024-09-15T01:35:23.761 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:23.761 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:23.761 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:23.761 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:23.761 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:23.761 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:23.761 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:23.761 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:23.761 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:23.761 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:23.761 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:23.762 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:23.762 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:23.762 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:23.762 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:23.762 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:23.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 1 Processor -- start 2024-09-15T01:35:23.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 1 -- start start 2024-09-15T01:35:23.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 10 monclient: init 2024-09-15T01:35:23.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:23.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 10 monclient: _add_conns ranks=[4,2,5,3,1,6,7,8,0] 2024-09-15T01:35:23.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5b2c067130 0x7f5b2c140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:23.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 10 monclient(hunting): picked mon.noname-c con 0x7f5b2c067130 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:23.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c067500 0x7f5b2c140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:23.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 10 monclient(hunting): picked mon.noname-b con 0x7f5b2c067500 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:23.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2c130a70 0x7f5b2c11d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:23.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 10 monclient(hunting): picked mon.noname-h con 0x7f5b2c130a70 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:23.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:23.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c09bda0 con 0x7f5b2c067500 2024-09-15T01:35:23.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:23.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c02a6b0 con 0x7f5b2c067130 2024-09-15T01:35:23.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:23.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c1208a0 con 0x7f5b2c130a70 2024-09-15T01:35:23.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:23.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b3215d640 10 monclient(hunting): authenticate will time out at 939.385706s 2024-09-15T01:35:23.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b23fff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c067500 msgr2=0x7f5b2c140640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:23.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b23fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c067500 0x7f5b2c140640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:23.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2c130a70 0x7f5b2c11d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:23.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2c130a70 0x7f5b2c11d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49744/0 (socket says 172.21.15.204:49744) 2024-09-15T01:35:23.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2bfff640 1 -- 172.21.15.204:0/3726660424 learned_addr learned my addr 172.21.15.204:0/3726660424 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:23.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2bfff640 10 monclient(hunting): get_auth_request con 0x7f5b2c130a70 auth_method 0 2024-09-15T01:35:23.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:23.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:23.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:23.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2b7fe640 1 --2- 172.21.15.204:0/3726660424 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5b2c067130 0x7f5b2c140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2b7fe640 10 monclient(hunting): get_auth_request con 0x7f5b2c067130 auth_method 0 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.742+0000 7f5b2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2bfff640 10 monclient(hunting): handle_auth_done global_id 5639 payload 824 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2bfff640 1 -- 172.21.15.204:0/3726660424 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5b2c067130 msgr2=0x7f5b2c140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2bfff640 1 --2- 172.21.15.204:0/3726660424 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5b2c067130 0x7f5b2c140100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2bfff640 1 -- 172.21.15.204:0/3726660424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c067500 msgr2=0x7f5b2c140640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2bfff640 1 --2- 172.21.15.204:0/3726660424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c067500 0x7f5b2c140640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:23.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2bfff640 1 monclient: found mon.noname-h 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2bfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2bfff640 1 -- 172.21.15.204:0/3726660424 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b24009ae0 con 0x7f5b2c130a70 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2b7fe640 1 --2- 172.21.15.204:0/3726660424 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5b2c067130 0x7f5b2c140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2bfff640 1 --2- 172.21.15.204:0/3726660424 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2c130a70 0x7f5b2c11d360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009880 tx=0x7f5b24036000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b237fe640 1 -- 172.21.15.204:0/3726660424 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b240094e0 con 0x7f5b2c130a70 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b237fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b237fe640 10 monclient: dump: 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:23.766 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:23.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b237fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b237fe640 1 -- 172.21.15.204:0/3726660424 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2403cab0 con 0x7f5b2c130a70 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b3215d640 5 monclient: authenticate success, global_id 5639 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b3215d640 10 monclient: _renew_subs 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b3215d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b3215d640 1 -- 172.21.15.204:0/3726660424 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c120b80 con 0x7f5b2c130a70 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b237fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b237fe640 1 -- 172.21.15.204:0/3726660424 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b2403cc60 con 0x7f5b2c130a70 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b237fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b237fe640 10 monclient: dump: 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:23.768 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:23.769 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:23.769 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:23.769 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:23.769 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:23.769 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:23.769 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:23.769 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:23.769 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:23.769 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:23.770 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:23.770 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:23.770 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:23.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b3215d640 10 monclient: _renew_subs 2024-09-15T01:35:23.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b2a7fc640 4 set_mon_vals no callback set 2024-09-15T01:35:23.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b3215d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:23.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.743+0000 7f5b3215d640 1 -- 172.21.15.204:0/3726660424 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c11f940 con 0x7f5b2c130a70 2024-09-15T01:35:23.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.744+0000 7f5b237fe640 1 -- 172.21.15.204:0/3726660424 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5b24003040 con 0x7f5b2c130a70 2024-09-15T01:35:23.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.744+0000 7f5b3215d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:23.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.744+0000 7f5b3215d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:23.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.744+0000 7f5b3215d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:23.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.744+0000 7f5b3215d640 1 -- 172.21.15.204:0/3726660424 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2c070b80 con 0x7f5b2c130a70 2024-09-15T01:35:23.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.744+0000 7f5b237fe640 1 --2- 172.21.15.204:0/3726660424 >> v2:172.21.15.204:6812/28210 conn(0x7f5b0c0397b0 0x7f5b0c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:23.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.744+0000 7f5b237fe640 1 -- 172.21.15.204:0/3726660424 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5683+0+0 (secure 0 0 0) 0x7f5b2406bc60 con 0x7f5b2c130a70 2024-09-15T01:35:23.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.744+0000 7f5b2b7fe640 1 --2- 172.21.15.204:0/3726660424 >> v2:172.21.15.204:6812/28210 conn(0x7f5b0c0397b0 0x7f5b0c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:23.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.744+0000 7f5b2b7fe640 10 monclient: get_auth_request con 0x7f5b0c0397b0 auth_method 0 2024-09-15T01:35:23.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.744+0000 7f5b2b7fe640 1 --2- 172.21.15.204:0/3726660424 >> v2:172.21.15.204:6812/28210 conn(0x7f5b0c0397b0 0x7f5b0c03bc60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c004fd0 tx=0x7f5b1c004980 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:23.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.748+0000 7f5b237fe640 1 -- 172.21.15.204:0/3726660424 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5b2403c630 con 0x7f5b2c130a70 2024-09-15T01:35:23.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.748+0000 7f5b237fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:23.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.748+0000 7f5b237fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:23.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.913+0000 7f5b3215d640 10 start_mon_command cmd=[{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-09-15T01:35:23.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.913+0000 7f5b3215d640 10 monclient: _send_command 2 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-09-15T01:35:23.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.913+0000 7f5b3215d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:23.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:23.913+0000 7f5b3215d640 1 -- 172.21.15.204:0/3726660424 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "luminous"} v 0) v1 -- 0x7f5b2c099ab0 con 0x7f5b2c130a70 2024-09-15T01:35:24.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:24.559+0000 7f5b237fe640 1 -- 172.21.15.204:0/3726660424 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "luminous"}]=0 set require_min_compat_client to luminous v120) v1 ==== 145+0+0 (secure 0 0 0) 0x7f5b24042310 con 0x7f5b2c130a70 2024-09-15T01:35:24.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:24.559+0000 7f5b237fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-09-15T01:35:24.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:24.559+0000 7f5b237fe640 10 monclient: _finish_command 2 = system:0 set require_min_compat_client to luminous 2024-09-15T01:35:24.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:24.662+0000 7f5b3215d640 10 start_mon_command cmd=[{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-09-15T01:35:24.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:24.662+0000 7f5b3215d640 10 monclient: _send_command 3 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-09-15T01:35:24.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:24.662+0000 7f5b3215d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:24.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:24.662+0000 7f5b3215d640 1 -- 172.21.15.204:0/3726660424 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "luminous"} v 0) v1 -- 0x7f5b2c02a8e0 con 0x7f5b2c130a70 2024-09-15T01:35:25.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.557+0000 7f5b237fe640 1 -- 172.21.15.204:0/3726660424 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "luminous"}]=0 set require_min_compat_client to luminous v121) v1 ==== 145+0+0 (secure 0 0 0) 0x7f5b24040030 con 0x7f5b2c130a70 2024-09-15T01:35:25.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.557+0000 7f5b237fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-09-15T01:35:25.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.557+0000 7f5b237fe640 10 monclient: _finish_command 3 = system:0 set require_min_compat_client to luminous 2024-09-15T01:35:25.561 INFO:tasks.workunit.client.0.smithi204.stderr:set require_min_compat_client to luminous 2024-09-15T01:35:25.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.560+0000 7f5b3215d640 1 -- 172.21.15.204:0/3726660424 >> v2:172.21.15.204:6812/28210 conn(0x7f5b0c0397b0 msgr2=0x7f5b0c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:25.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.560+0000 7f5b3215d640 1 --2- 172.21.15.204:0/3726660424 >> v2:172.21.15.204:6812/28210 conn(0x7f5b0c0397b0 0x7f5b0c03bc60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c004fd0 tx=0x7f5b1c004980 comp rx=0 tx=0).stop 2024-09-15T01:35:25.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.561+0000 7f5b3215d640 10 monclient: shutdown 2024-09-15T01:35:25.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.561+0000 7f5b3215d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:25.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.561+0000 7f5b3215d640 1 -- 172.21.15.204:0/3726660424 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2c130a70 msgr2=0x7f5b2c11d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:25.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.561+0000 7f5b3215d640 1 --2- 172.21.15.204:0/3726660424 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2c130a70 0x7f5b2c11d360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009880 tx=0x7f5b24036000 comp rx=0 tx=0).stop 2024-09-15T01:35:25.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.561+0000 7f5b3215d640 1 -- 172.21.15.204:0/3726660424 shutdown_connections 2024-09-15T01:35:25.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.561+0000 7f5b3215d640 1 --2- 172.21.15.204:0/3726660424 >> v2:172.21.15.204:6812/28210 conn(0x7f5b0c0397b0 0x7f5b0c03bc60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.561+0000 7f5b3215d640 1 --2- 172.21.15.204:0/3726660424 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5b2c130a70 0x7f5b2c11d360 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.561+0000 7f5b3215d640 1 --2- 172.21.15.204:0/3726660424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b2c067500 0x7f5b2c140640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.561+0000 7f5b3215d640 1 --2- 172.21.15.204:0/3726660424 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5b2c067130 0x7f5b2c140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.561+0000 7f5b3215d640 1 -- 172.21.15.204:0/3726660424 >> 172.21.15.204:0/3726660424 conn(0x7f5b2c0a2ce0 msgr2=0x7f5b2c06e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:25.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.561+0000 7f5b3215d640 1 -- 172.21.15.204:0/3726660424 shutdown_connections 2024-09-15T01:35:25.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.561+0000 7f5b3215d640 1 -- 172.21.15.204:0/3726660424 wait complete. 2024-09-15T01:35:25.575 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set dump 2024-09-15T01:35:25.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.652+0000 7feb964d4640 10 monclient: get_monmap_and_config 2024-09-15T01:35:25.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.652+0000 7feb964d4640 10 monclient: build_initial_monmap 2024-09-15T01:35:25.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.652+0000 7feb964d4640 10 monclient: monmap: 2024-09-15T01:35:25.655 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:25.655 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:25.655 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:25.653650+0000 2024-09-15T01:35:25.655 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:25.653650+0000 2024-09-15T01:35:25.655 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:25.657 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:25.657 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:25.657 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:25.657 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:25.657 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:25.657 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:25.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.652+0000 7feb964d4640 1 Processor -- start 2024-09-15T01:35:25.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 1 -- start start 2024-09-15T01:35:25.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 10 monclient: init 2024-09-15T01:35:25.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:25.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 10 monclient: _add_conns ranks=[7,9,1,2,8,3,6,4,5,10,0] 2024-09-15T01:35:25.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feb9006ed20 0x7feb90130800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:25.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 10 monclient(hunting): picked mon.noname-i con 0x7feb9006ed20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:25.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feb90130d40 0x7feb90131110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:25.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 10 monclient(hunting): picked mon.noname-a con 0x7feb90130d40 addr v2:172.21.15.178:3300/0 2024-09-15T01:35:25.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7feb9009e950 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:25.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb9004b720 con 0x7feb9009e950 2024-09-15T01:35:25.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:25.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb9006e520 con 0x7feb90130d40 2024-09-15T01:35:25.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:25.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feb9006ce30 con 0x7feb9006ed20 2024-09-15T01:35:25.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:25.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb964d4640 10 monclient(hunting): authenticate will time out at 941.296533s 2024-09-15T01:35:25.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8ffff640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feb9006ed20 msgr2=0x7feb90130800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:25.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8ffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feb9006ed20 0x7feb90130800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:25.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb94a71640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7feb9009e950 0x7feb9009edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:34184/0 (socket says 172.21.15.204:34184) 2024-09-15T01:35:25.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb94a71640 1 -- 172.21.15.204:0/2511770037 learned_addr learned my addr 172.21.15.204:0/2511770037 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:25.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8f7fe640 1 --2- 172.21.15.204:0/2511770037 >> v2:172.21.15.178:3300/0 conn(0x7feb90130d40 0x7feb90131110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:25.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8f7fe640 10 monclient(hunting): get_auth_request con 0x7feb90130d40 auth_method 0 2024-09-15T01:35:25.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:25.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:25.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:25.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:25.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:25.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8effd640 1 -- 172.21.15.204:0/2511770037 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415141311 0 0) 0x7feb9004b720 con 0x7feb9009e950 2024-09-15T01:35:25.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:25.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:25.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:25.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8effd640 10 monclient(hunting): my global_id is 6148 2024-09-15T01:35:25.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.653+0000 7feb8effd640 1 -- 172.21.15.204:0/2511770037 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb780034e0 con 0x7feb9009e950 2024-09-15T01:35:25.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.654+0000 7feb8f7fe640 10 monclient(hunting): handle_auth_done global_id 5751 payload 274 2024-09-15T01:35:25.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.654+0000 7feb8f7fe640 1 -- 172.21.15.204:0/2511770037 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feb9006ed20 msgr2=0x7feb90130800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:25.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.654+0000 7feb8f7fe640 1 --2- 172.21.15.204:0/2511770037 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feb9006ed20 0x7feb90130800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.654+0000 7feb8f7fe640 1 -- 172.21.15.204:0/2511770037 >> v1:172.21.15.204:6789/0 conn(0x7feb9009e950 legacy=0x7feb9009edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:25.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.654+0000 7feb8effd640 1 -- 172.21.15.204:0/2511770037 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3038381316 0 0) 0x7feb780034e0 con 0x7feb9009e950 2024-09-15T01:35:25.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.654+0000 7feb8f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:25.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.654+0000 7feb8f7fe640 1 monclient: found mon.noname-a 2024-09-15T01:35:25.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.654+0000 7feb8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:25.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.654+0000 7feb8f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:35:25.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.654+0000 7feb8f7fe640 1 -- 172.21.15.204:0/2511770037 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb90131650 con 0x7feb90130d40 2024-09-15T01:35:25.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.654+0000 7feb8effd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:35:25.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.654+0000 7feb8f7fe640 1 --2- 172.21.15.204:0/2511770037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb90130d40 0x7feb90131110 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feb84009880 tx=0x7feb840090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c1e9263c5f2e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb8effd640 1 -- 172.21.15.204:0/2511770037 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb8402ad00 con 0x7feb90130d40 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb8effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb8effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb8effd640 10 monclient: dump: 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:25.673 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:25.674 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:25.674 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:25.674 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:25.674 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:25.674 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:25.674 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:25.674 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:25.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb8effd640 10 monclient: _finish_auth 0 2024-09-15T01:35:25.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb8effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:25.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb8effd640 1 -- 172.21.15.204:0/2511770037 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8402ae80 con 0x7feb90130d40 2024-09-15T01:35:25.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 5 monclient: authenticate success, global_id 5751 2024-09-15T01:35:25.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:25.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb8effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:25.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:25.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb8effd640 1 -- 172.21.15.204:0/2511770037 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb84031ca0 con 0x7feb90130d40 2024-09-15T01:35:25.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 4 set_mon_vals no callback set 2024-09-15T01:35:25.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb8effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:25.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb8effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:35:25.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb8effd640 10 monclient: dump: 2024-09-15T01:35:25.675 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:25.675 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:25.675 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:25.676 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:25.676 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:25.676 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:25.676 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:25.676 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:25.676 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:25.676 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:25.676 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:25.676 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:25.676 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:25.677 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:25.677 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:25.677 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:25.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 10 monclient: shutdown 2024-09-15T01:35:25.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:25.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 1 -- 172.21.15.204:0/2511770037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb90130d40 msgr2=0x7feb90131110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:25.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 1 --2- 172.21.15.204:0/2511770037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb90130d40 0x7feb90131110 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feb84009880 tx=0x7feb840090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 1 -- 172.21.15.204:0/2511770037 shutdown_connections 2024-09-15T01:35:25.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 1 --2- 172.21.15.204:0/2511770037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb90130d40 0x7feb90131110 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 1 --2- 172.21.15.204:0/2511770037 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feb9006ed20 0x7feb90130800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 1 -- 172.21.15.204:0/2511770037 >> 172.21.15.204:0/2511770037 conn(0x7feb900a3e60 msgr2=0x7feb900a4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:25.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 1 -- 172.21.15.204:0/2511770037 shutdown_connections 2024-09-15T01:35:25.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.655+0000 7feb964d4640 1 -- 172.21.15.204:0/2511770037 wait complete. 2024-09-15T01:35:25.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.656+0000 7feb964d4640 10 monclient: build_initial_monmap 2024-09-15T01:35:25.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.656+0000 7feb964d4640 10 monclient: monmap: 2024-09-15T01:35:25.678 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:25.679 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:25.680 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:25.680 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:25.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.656+0000 7feb964d4640 1 Processor -- start 2024-09-15T01:35:25.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.656+0000 7feb964d4640 1 -- start start 2024-09-15T01:35:25.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.656+0000 7feb964d4640 10 monclient: init 2024-09-15T01:35:25.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:25.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 10 monclient: _add_conns ranks=[5,0,3,6,7,2,4,8,1] 2024-09-15T01:35:25.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb9006ed20 0x7feb90141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:25.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 10 monclient(hunting): picked mon.noname-h con 0x7feb9006ed20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:25.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb9009e950 0x7feb901421c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:25.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 10 monclient(hunting): picked mon.noname-a con 0x7feb9009e950 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:25.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb90130d40 0x7feb900682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:25.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 10 monclient(hunting): picked mon.noname-g con 0x7feb90130d40 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:25.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:25.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb90121890 con 0x7feb9009e950 2024-09-15T01:35:25.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:25.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb90121dc0 con 0x7feb90130d40 2024-09-15T01:35:25.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:25.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb9006d610 con 0x7feb9006ed20 2024-09-15T01:35:25.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:25.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 10 monclient(hunting): authenticate will time out at 941.300139s 2024-09-15T01:35:25.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb9006ed20 0x7feb90141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:25.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb94a71640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb90130d40 0x7feb900682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:25.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb9006ed20 0x7feb90141c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49748/0 (socket says 172.21.15.204:49748) 2024-09-15T01:35:25.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 1 -- 172.21.15.204:0/3512517694 learned_addr learned my addr 172.21.15.204:0/3512517694 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:25.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 10 monclient(hunting): get_auth_request con 0x7feb9006ed20 auth_method 0 2024-09-15T01:35:25.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:25.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:25.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:25.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb94a71640 10 monclient(hunting): get_auth_request con 0x7feb90130d40 auth_method 0 2024-09-15T01:35:25.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb94a71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:25.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb94a71640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:25.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb94a71640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:25.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:25.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:25.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:25.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb94a71640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:25.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb94a71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:25.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb94a71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:25.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8f7fe640 1 --2- 172.21.15.204:0/3512517694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb9009e950 0x7feb901421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:25.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8f7fe640 10 monclient(hunting): get_auth_request con 0x7feb9009e950 auth_method 0 2024-09-15T01:35:25.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:25.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:25.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:25.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:25.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:25.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:25.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 10 monclient(hunting): handle_auth_done global_id 5648 payload 824 2024-09-15T01:35:25.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 1 -- 172.21.15.204:0/3512517694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb90130d40 msgr2=0x7feb900682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:25.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 1 --2- 172.21.15.204:0/3512517694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb90130d40 0x7feb900682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 1 -- 172.21.15.204:0/3512517694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb9009e950 msgr2=0x7feb901421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 1 --2- 172.21.15.204:0/3512517694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb9009e950 0x7feb901421c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 1 monclient: found mon.noname-h 2024-09-15T01:35:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 1 -- 172.21.15.204:0/3512517694 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb84009390 con 0x7feb9006ed20 2024-09-15T01:35:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb94a71640 1 --2- 172.21.15.204:0/3512517694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb90130d40 0x7feb900682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8ffff640 1 --2- 172.21.15.204:0/3512517694 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb9006ed20 0x7feb90141c80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feb8000b620 tx=0x7feb80009120 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:25.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb77fff640 1 -- 172.21.15.204:0/3512517694 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb80015070 con 0x7feb9006ed20 2024-09-15T01:35:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb77fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb77fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb77fff640 10 monclient: dump: 2024-09-15T01:35:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:25.687 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb77fff640 10 monclient: _finish_auth 0 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb77fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb77fff640 1 -- 172.21.15.204:0/3512517694 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb80006150 con 0x7feb9006ed20 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 5 monclient: authenticate success, global_id 5648 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 10 monclient: _renew_subs 2024-09-15T01:35:25.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:25.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb964d4640 1 -- 172.21.15.204:0/3512517694 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9006d2b0 con 0x7feb9006ed20 2024-09-15T01:35:25.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb77fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:25.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb77fff640 1 -- 172.21.15.204:0/3512517694 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb80006740 con 0x7feb9006ed20 2024-09-15T01:35:25.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb77fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:25.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb77fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:25.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb77fff640 10 monclient: dump: 2024-09-15T01:35:25.689 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:25.689 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:25.689 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:25.689 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:25.689 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:25.689 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.657+0000 7feb8e7fc640 4 set_mon_vals no callback set 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.658+0000 7feb964d4640 10 monclient: _renew_subs 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.658+0000 7feb964d4640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:25.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.658+0000 7feb964d4640 1 -- 172.21.15.204:0/3512517694 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb90120420 con 0x7feb9006ed20 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.658+0000 7feb77fff640 1 -- 172.21.15.204:0/3512517694 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7feb800062c0 con 0x7feb9006ed20 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.659+0000 7feb964d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.659+0000 7feb964d4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.659+0000 7feb964d4640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.659+0000 7feb964d4640 1 -- 172.21.15.204:0/3512517694 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb90070d80 con 0x7feb9006ed20 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.659+0000 7feb77fff640 1 --2- 172.21.15.204:0/3512517694 >> v2:172.21.15.204:6812/28210 conn(0x7feb7c0397e0 0x7feb7c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.659+0000 7feb77fff640 1 -- 172.21.15.204:0/3512517694 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5683+0+0 (secure 0 0 0) 0x7feb8004b1b0 con 0x7feb9006ed20 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.659+0000 7feb8f7fe640 1 --2- 172.21.15.204:0/3512517694 >> v2:172.21.15.204:6812/28210 conn(0x7feb7c0397e0 0x7feb7c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.659+0000 7feb8f7fe640 10 monclient: get_auth_request con 0x7feb7c0397e0 auth_method 0 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.659+0000 7feb8f7fe640 1 --2- 172.21.15.204:0/3512517694 >> v2:172.21.15.204:6812/28210 conn(0x7feb7c0397e0 0x7feb7c03bc90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feb84034000 tx=0x7feb84032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.663+0000 7feb77fff640 1 -- 172.21.15.204:0/3512517694 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7feb80010440 con 0x7feb9006ed20 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.663+0000 7feb77fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:25.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.663+0000 7feb77fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:25.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.826+0000 7feb964d4640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-09-15T01:35:25.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.826+0000 7feb964d4640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T01:35:25.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.826+0000 7feb964d4640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:25.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.826+0000 7feb964d4640 1 -- 172.21.15.204:0/3512517694 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush weight-set dump"} v 0) v1 -- 0x7feb9006aa20 con 0x7feb9006ed20 2024-09-15T01:35:25.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.826+0000 7feb77fff640 1 -- 172.21.15.204:0/3512517694 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v121) v1 ==== 73+0+3 (secure 0 0 0) 0x7feb8000d370 con 0x7feb9006ed20 2024-09-15T01:35:25.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.826+0000 7feb77fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T01:35:25.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.826+0000 7feb77fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:25.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.924+0000 7feb964d4640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-09-15T01:35:25.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.924+0000 7feb964d4640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T01:35:25.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.925+0000 7feb964d4640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:25.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.925+0000 7feb964d4640 1 -- 172.21.15.204:0/3512517694 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush weight-set dump"} v 0) v1 -- 0x7feb901208a0 con 0x7feb9006ed20 2024-09-15T01:35:25.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.925+0000 7feb77fff640 1 -- 172.21.15.204:0/3512517694 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v121) v1 ==== 73+0+3 (secure 0 0 0) 0x7feb8000eb30 con 0x7feb9006ed20 2024-09-15T01:35:25.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.925+0000 7feb77fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T01:35:25.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.925+0000 7feb77fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:25.928 INFO:tasks.workunit.client.0.smithi204.stdout:{} 2024-09-15T01:35:25.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 1 -- 172.21.15.204:0/3512517694 >> v2:172.21.15.204:6812/28210 conn(0x7feb7c0397e0 msgr2=0x7feb7c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:25.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 1 --2- 172.21.15.204:0/3512517694 >> v2:172.21.15.204:6812/28210 conn(0x7feb7c0397e0 0x7feb7c03bc90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feb84034000 tx=0x7feb84032000 comp rx=0 tx=0).stop 2024-09-15T01:35:25.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 10 monclient: shutdown 2024-09-15T01:35:25.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:25.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 1 -- 172.21.15.204:0/3512517694 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb9006ed20 msgr2=0x7feb90141c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:25.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 1 --2- 172.21.15.204:0/3512517694 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb9006ed20 0x7feb90141c80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feb8000b620 tx=0x7feb80009120 comp rx=0 tx=0).stop 2024-09-15T01:35:25.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 1 -- 172.21.15.204:0/3512517694 shutdown_connections 2024-09-15T01:35:25.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 1 --2- 172.21.15.204:0/3512517694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb90130d40 0x7feb900682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 1 --2- 172.21.15.204:0/3512517694 >> v2:172.21.15.204:6812/28210 conn(0x7feb7c0397e0 0x7feb7c03bc90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 1 --2- 172.21.15.204:0/3512517694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb9009e950 0x7feb901421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 1 --2- 172.21.15.204:0/3512517694 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb9006ed20 0x7feb90141c80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:25.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 1 -- 172.21.15.204:0/3512517694 >> 172.21.15.204:0/3512517694 conn(0x7feb900a3e60 msgr2=0x7feb9009b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:25.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 1 -- 172.21.15.204:0/3512517694 shutdown_connections 2024-09-15T01:35:25.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:25.927+0000 7feb964d4640 1 -- 172.21.15.204:0/3512517694 wait complete. 2024-09-15T01:35:25.938 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set ls 2024-09-15T01:35:26.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.010+0000 7fef9579d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:26.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.010+0000 7fef9579d640 10 monclient: build_initial_monmap 2024-09-15T01:35:26.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.010+0000 7fef9579d640 10 monclient: monmap: 2024-09-15T01:35:26.014 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:26.014 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.014 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:26.012019+0000 2024-09-15T01:35:26.014 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:26.012019+0000 2024-09-15T01:35:26.014 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:26.014 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.014 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:26.014 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:26.014 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:26.014 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:26.014 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:26.015 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:26.015 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:26.015 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:26.015 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:26.015 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:26.015 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:26.015 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 1 Processor -- start 2024-09-15T01:35:26.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 1 -- start start 2024-09-15T01:35:26.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 10 monclient: init 2024-09-15T01:35:26.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:26.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 10 monclient: _add_conns ranks=[8,4,0,1,5,7,9,2,6,10,3] 2024-09-15T01:35:26.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fef900a2e30 0x7fef90133280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 10 monclient(hunting): picked mon.noname-e con 0x7fef900a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:26.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 0x7fef900a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 10 monclient(hunting): picked mon.noname-c con 0x7fef9009fd80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:26.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fef900a09c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9004b720 con 0x7fef900a09c0 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef9009e290 con 0x7fef9009fd80 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fef900a2450 con 0x7fef900a2e30 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.011+0000 7fef9579d640 10 monclient(hunting): authenticate will time out at 941.654915s 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fef900a2e30 0x7fef90133280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef87fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 0x7fef900a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fef900a2e30 0x7fef90133280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:33050/0 (socket says 172.21.15.204:33050) 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 1 -- 172.21.15.204:0/2586297995 learned_addr learned my addr 172.21.15.204:0/2586297995 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 10 monclient(hunting): get_auth_request con 0x7fef900a2e30 auth_method 0 2024-09-15T01:35:26.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8f7fe640 1 --1- 172.21.15.204:0/2586297995 >> v1:172.21.15.178:6789/0 conn(0x7fef900a09c0 0x7fef900a0df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.204:60826/0 (socket says 172.21.15.204:60826) 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef87fff640 10 monclient(hunting): get_auth_request con 0x7fef9009fd80 auth_method 0 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef87fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef87fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:26.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 10 monclient(hunting): handle_auth_done global_id 5579 payload 274 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 1 -- 172.21.15.204:0/2586297995 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 msgr2=0x7fef900a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 1 --2- 172.21.15.204:0/2586297995 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 0x7fef900a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 1 -- 172.21.15.204:0/2586297995 >> v1:172.21.15.178:6789/0 conn(0x7fef900a09c0 legacy=0x7fef900a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8e7fc640 1 -- 172.21.15.204:0/2586297995 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080276200 0 0) 0x7fef9004b720 con 0x7fef900a09c0 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 10 monclient: _finish_hunting 0 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 1 monclient: found mon.noname-e 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8effd640 1 -- 172.21.15.204:0/2586297995 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef900a3200 con 0x7fef900a2e30 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef8e7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.012+0000 7fef87fff640 1 --2- 172.21.15.204:0/2586297995 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 0x7fef900a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:26.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8effd640 1 --2- 172.21.15.204:0/2586297995 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fef900a2e30 0x7fef90133280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fef80009940 tx=0x7fef800091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=3ae79a020ac24a4c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8e7fc640 1 -- 172.21.15.204:0/2586297995 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef8002ad00 con 0x7fef900a2e30 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8e7fc640 10 monclient: dump: 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:26.028 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8e7fc640 1 -- 172.21.15.204:0/2586297995 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8002ae80 con 0x7fef900a2e30 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8e7fc640 1 -- 172.21.15.204:0/2586297995 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef80031ca0 con 0x7fef900a2e30 2024-09-15T01:35:26.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef9579d640 5 monclient: authenticate success, global_id 5579 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef9579d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef9579d640 4 set_mon_vals no callback set 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.013+0000 7fef8e7fc640 10 monclient: dump: 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:26.030 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:26.031 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:26.031 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:26.031 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:26.031 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:26.031 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:26.031 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:26.031 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 10 monclient: shutdown 2024-09-15T01:35:26.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:26.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 1 -- 172.21.15.204:0/2586297995 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fef900a2e30 msgr2=0x7fef90133280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:26.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 1 --2- 172.21.15.204:0/2586297995 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fef900a2e30 0x7fef90133280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fef80009940 tx=0x7fef800091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 1 -- 172.21.15.204:0/2586297995 shutdown_connections 2024-09-15T01:35:26.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 1 --2- 172.21.15.204:0/2586297995 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 0x7fef900a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 1 --2- 172.21.15.204:0/2586297995 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fef900a2e30 0x7fef90133280 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 1 -- 172.21.15.204:0/2586297995 >> 172.21.15.204:0/2586297995 conn(0x7fef9006e860 msgr2=0x7fef9006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:26.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 1 -- 172.21.15.204:0/2586297995 shutdown_connections 2024-09-15T01:35:26.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 1 -- 172.21.15.204:0/2586297995 wait complete. 2024-09-15T01:35:26.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 10 monclient: build_initial_monmap 2024-09-15T01:35:26.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 10 monclient: monmap: 2024-09-15T01:35:26.032 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:26.032 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:26.032 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:26.032 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:26.033 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:26.033 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:26.033 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:26.033 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:26.033 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:26.033 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:26.033 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:26.033 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:26.033 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:26.033 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:26.033 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:26.034 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.014+0000 7fef9579d640 1 Processor -- start 2024-09-15T01:35:26.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 1 -- start start 2024-09-15T01:35:26.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 10 monclient: init 2024-09-15T01:35:26.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:26.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 10 monclient: _add_conns ranks=[4,2,1,6,0,8,7,3,5] 2024-09-15T01:35:26.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 0x7fef90141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 10 monclient(hunting): picked mon.noname-c con 0x7fef9009fd80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:26.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef900a09c0 0x7fef9012f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 10 monclient(hunting): picked mon.noname-b con 0x7fef900a09c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:26.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef900a2e30 0x7fef9011c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 10 monclient(hunting): picked mon.noname-f con 0x7fef900a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:26.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9009d9b0 con 0x7fef900a2e30 2024-09-15T01:35:26.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9009d4c0 con 0x7fef900a09c0 2024-09-15T01:35:26.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef90142480 con 0x7fef9009fd80 2024-09-15T01:35:26.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:26.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef9579d640 10 monclient(hunting): authenticate will time out at 941.658547s 2024-09-15T01:35:26.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef87fff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef900a09c0 msgr2=0x7fef9012f990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:26.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef87fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef900a09c0 0x7fef9012f990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:26.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef900a2e30 0x7fef9011c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8effd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 0x7fef90141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef900a2e30 0x7fef9011c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36060/0 (socket says 172.21.15.204:36060) 2024-09-15T01:35:26.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8effd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 0x7fef90141f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:46408/0 (socket says 172.21.15.204:46408) 2024-09-15T01:35:26.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 1 -- 172.21.15.204:0/1106182107 learned_addr learned my addr 172.21.15.204:0/1106182107 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:26.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 10 monclient(hunting): get_auth_request con 0x7fef900a2e30 auth_method 0 2024-09-15T01:35:26.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8effd640 10 monclient(hunting): get_auth_request con 0x7fef9009fd80 auth_method 0 2024-09-15T01:35:26.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.036 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.037 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:26.037 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:26.037 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:26.037 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:26.037 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:26.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:26.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 10 monclient(hunting): handle_auth_done global_id 6157 payload 824 2024-09-15T01:35:26.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 1 -- 172.21.15.204:0/1106182107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 msgr2=0x7fef90141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:26.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 1 --2- 172.21.15.204:0/1106182107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 0x7fef90141f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 1 -- 172.21.15.204:0/1106182107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef900a09c0 msgr2=0x7fef9012f990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:26.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 1 --2- 172.21.15.204:0/1106182107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef900a09c0 0x7fef9012f990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:26.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 1 monclient: found mon.noname-f 2024-09-15T01:35:26.038 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8f7fe640 1 -- 172.21.15.204:0/1106182107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef78009390 con 0x7fef900a2e30 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.015+0000 7fef8effd640 1 --2- 172.21.15.204:0/1106182107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 0x7fef90141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef8f7fe640 1 --2- 172.21.15.204:0/1106182107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef900a2e30 0x7fef9011c630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef8800c870 tx=0x7fef88001630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef86ffd640 1 -- 172.21.15.204:0/1106182107 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef8800ce60 con 0x7fef900a2e30 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef86ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef86ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef86ffd640 10 monclient: dump: 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:26.039 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef86ffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef86ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef86ffd640 1 -- 172.21.15.204:0/1106182107 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef88005d60 con 0x7fef900a2e30 2024-09-15T01:35:26.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef86ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:26.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef86ffd640 1 -- 172.21.15.204:0/1106182107 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef880070f0 con 0x7fef900a2e30 2024-09-15T01:35:26.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef9579d640 5 monclient: authenticate success, global_id 6157 2024-09-15T01:35:26.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef86ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:26.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef86ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:26.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef8dffb640 4 set_mon_vals no callback set 2024-09-15T01:35:26.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef86ffd640 10 monclient: dump: 2024-09-15T01:35:26.041 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:26.041 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.041 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.041 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.042 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:26.042 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.042 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:26.043 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:26.043 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:26.043 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:26.043 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:26.043 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:26.044 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:26.044 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:26.044 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:26.044 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef9579d640 10 monclient: _renew_subs 2024-09-15T01:35:26.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef9579d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:26.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef9579d640 1 -- 172.21.15.204:0/1106182107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef80009450 con 0x7fef900a2e30 2024-09-15T01:35:26.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef9579d640 10 monclient: _renew_subs 2024-09-15T01:35:26.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef9579d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:26.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.016+0000 7fef9579d640 1 -- 172.21.15.204:0/1106182107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9011e040 con 0x7fef900a2e30 2024-09-15T01:35:26.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.017+0000 7fef86ffd640 1 -- 172.21.15.204:0/1106182107 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fef88007250 con 0x7fef900a2e30 2024-09-15T01:35:26.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.017+0000 7fef9579d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:26.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.017+0000 7fef9579d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:26.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.017+0000 7fef9579d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:26.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.017+0000 7fef9579d640 1 -- 172.21.15.204:0/1106182107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9011ff60 con 0x7fef900a2e30 2024-09-15T01:35:26.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.017+0000 7fef86ffd640 1 --2- 172.21.15.204:0/1106182107 >> v2:172.21.15.204:6812/28210 conn(0x7fef700397e0 0x7fef7003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.017+0000 7fef86ffd640 1 -- 172.21.15.204:0/1106182107 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5683+0+0 (secure 0 0 0) 0x7fef8804b3c0 con 0x7fef900a2e30 2024-09-15T01:35:26.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.017+0000 7fef8effd640 1 --2- 172.21.15.204:0/1106182107 >> v2:172.21.15.204:6812/28210 conn(0x7fef700397e0 0x7fef7003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.017+0000 7fef8effd640 10 monclient: get_auth_request con 0x7fef700397e0 auth_method 0 2024-09-15T01:35:26.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.018+0000 7fef8effd640 1 --2- 172.21.15.204:0/1106182107 >> v2:172.21.15.204:6812/28210 conn(0x7fef700397e0 0x7fef7003bc90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fef80009420 tx=0x7fef80032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:26.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.021+0000 7fef86ffd640 1 -- 172.21.15.204:0/1106182107 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fef88010900 con 0x7fef900a2e30 2024-09-15T01:35:26.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.022+0000 7fef86ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:26.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.022+0000 7fef86ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:26.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.183+0000 7fef9579d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:26.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.183+0000 7fef9579d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:26.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.183+0000 7fef9579d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:26.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.183+0000 7fef9579d640 1 -- 172.21.15.204:0/1106182107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7fef90070d80 con 0x7fef900a2e30 2024-09-15T01:35:26.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.184+0000 7fef86ffd640 1 -- 172.21.15.204:0/1106182107 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v121) v1 ==== 71+0+0 (secure 0 0 0) 0x7fef88019300 con 0x7fef900a2e30 2024-09-15T01:35:26.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.184+0000 7fef86ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:26.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.184+0000 7fef86ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:26.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.285+0000 7fef9579d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:26.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.285+0000 7fef9579d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:26.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.285+0000 7fef9579d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:26.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.285+0000 7fef9579d640 1 -- 172.21.15.204:0/1106182107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7fef9009dfb0 con 0x7fef900a2e30 2024-09-15T01:35:26.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.285+0000 7fef86ffd640 1 -- 172.21.15.204:0/1106182107 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v121) v1 ==== 71+0+0 (secure 0 0 0) 0x7fef88010300 con 0x7fef900a2e30 2024-09-15T01:35:26.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.285+0000 7fef86ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:26.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.285+0000 7fef86ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:26.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 1 -- 172.21.15.204:0/1106182107 >> v2:172.21.15.204:6812/28210 conn(0x7fef700397e0 msgr2=0x7fef7003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:26.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 1 --2- 172.21.15.204:0/1106182107 >> v2:172.21.15.204:6812/28210 conn(0x7fef700397e0 0x7fef7003bc90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fef80009420 tx=0x7fef80032000 comp rx=0 tx=0).stop 2024-09-15T01:35:26.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 10 monclient: shutdown 2024-09-15T01:35:26.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:26.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 1 -- 172.21.15.204:0/1106182107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef900a2e30 msgr2=0x7fef9011c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:26.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 1 --2- 172.21.15.204:0/1106182107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef900a2e30 0x7fef9011c630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef8800c870 tx=0x7fef88001630 comp rx=0 tx=0).stop 2024-09-15T01:35:26.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 1 -- 172.21.15.204:0/1106182107 shutdown_connections 2024-09-15T01:35:26.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 1 --2- 172.21.15.204:0/1106182107 >> v2:172.21.15.204:6812/28210 conn(0x7fef700397e0 0x7fef7003bc90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 1 --2- 172.21.15.204:0/1106182107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef900a2e30 0x7fef9011c630 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 1 --2- 172.21.15.204:0/1106182107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef900a09c0 0x7fef9012f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 1 --2- 172.21.15.204:0/1106182107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef9009fd80 0x7fef90141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 1 -- 172.21.15.204:0/1106182107 >> 172.21.15.204:0/1106182107 conn(0x7fef9006e860 msgr2=0x7fef900a47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:26.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 1 -- 172.21.15.204:0/1106182107 shutdown_connections 2024-09-15T01:35:26.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.287+0000 7fef9579d640 1 -- 172.21.15.204:0/1106182107 wait complete. 2024-09-15T01:35:26.299 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush weight-set reweight fooset osd.0 .9 2024-09-15T01:35:26.299 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:35:26.299 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 2024-09-15T01:35:26.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.374+0000 7fb43999d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:26.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.374+0000 7fb43999d640 10 monclient: build_initial_monmap 2024-09-15T01:35:26.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.375+0000 7fb43999d640 10 monclient: monmap: 2024-09-15T01:35:26.377 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:26.377 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.377 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:26.376478+0000 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:26.376478+0000 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:26.378 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:26.379 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:26.379 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.375+0000 7fb43999d640 1 Processor -- start 2024-09-15T01:35:26.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.375+0000 7fb43999d640 1 -- start start 2024-09-15T01:35:26.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.375+0000 7fb43999d640 10 monclient: init 2024-09-15T01:35:26.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:26.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 10 monclient: _add_conns ranks=[6,3,10,4,0,2,5,1,9,7,8] 2024-09-15T01:35:26.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb4340a2cd0 0x7fb434132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 10 monclient(hunting): picked mon.noname-d con 0x7fb4340a2cd0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43409fa20 0x7fb43409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 10 monclient(hunting): picked mon.noname-g con 0x7fb43409fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb4340a08e0 0x7fb4340a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 10 monclient(hunting): picked mon.noname-f con 0x7fb4340a08e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4340a30a0 con 0x7fb4340a08e0 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4340a0330 con 0x7fb43409fa20 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb4340a1db0 con 0x7fb4340a2cd0 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb43999d640 10 monclient(hunting): authenticate will time out at 942.019275s 2024-09-15T01:35:26.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb432ffd640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb4340a2cd0 msgr2=0x7fb434132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:26.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb432ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb4340a2cd0 0x7fb434132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:26.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb4340a08e0 0x7fb4340a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb4340a08e0 0x7fb4340a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36074/0 (socket says 172.21.15.204:36074) 2024-09-15T01:35:26.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 1 -- 172.21.15.204:0/15405513 learned_addr learned my addr 172.21.15.204:0/15405513 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:26.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 10 monclient(hunting): get_auth_request con 0x7fb4340a08e0 auth_method 0 2024-09-15T01:35:26.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb42a7fc640 1 --2- 172.21.15.204:0/15405513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43409fa20 0x7fb43409fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb42a7fc640 10 monclient(hunting): get_auth_request con 0x7fb43409fa20 auth_method 0 2024-09-15T01:35:26.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb42a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb42a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb42a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:26.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:26.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:26.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb42a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:26.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb42a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:26.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb42a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:26.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 10 monclient(hunting): handle_auth_done global_id 6166 payload 274 2024-09-15T01:35:26.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 1 -- 172.21.15.204:0/15405513 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb4340a2cd0 msgr2=0x7fb434132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:26.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 1 --2- 172.21.15.204:0/15405513 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb4340a2cd0 0x7fb434132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 1 -- 172.21.15.204:0/15405513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43409fa20 msgr2=0x7fb43409fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:26.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 1 --2- 172.21.15.204:0/15405513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43409fa20 0x7fb43409fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:26.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 1 monclient: found mon.noname-f 2024-09-15T01:35:26.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:26.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:26.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb4337fe640 1 -- 172.21.15.204:0/15405513 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4340648a0 con 0x7fb4340a08e0 2024-09-15T01:35:26.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.376+0000 7fb42a7fc640 1 --2- 172.21.15.204:0/15405513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43409fa20 0x7fb43409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:26.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4337fe640 1 --2- 172.21.15.204:0/15405513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4340a08e0 0x7fb4340a0d50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009f20 tx=0x7fb42c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d77fe162bd6cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:26.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4327fc640 1 -- 172.21.15.204:0/15405513 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb42c007880 con 0x7fb4340a08e0 2024-09-15T01:35:26.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:26.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4327fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:26.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4327fc640 10 monclient: dump: 2024-09-15T01:35:26.391 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:26.392 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4327fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb43999d640 5 monclient: authenticate success, global_id 6166 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb43999d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4327fc640 1 -- 172.21.15.204:0/15405513 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c02d050 con 0x7fb4340a08e0 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4327fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb43999d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb43999d640 4 set_mon_vals no callback set 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4327fc640 1 -- 172.21.15.204:0/15405513 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb42c030360 con 0x7fb4340a08e0 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:26.393 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4327fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.377+0000 7fb4327fc640 10 monclient: dump: 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:26.394 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 10 monclient: shutdown 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 1 -- 172.21.15.204:0/15405513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4340a08e0 msgr2=0x7fb4340a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 1 --2- 172.21.15.204:0/15405513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4340a08e0 0x7fb4340a0d50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009f20 tx=0x7fb42c009840 comp rx=0 tx=0).stop 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 1 -- 172.21.15.204:0/15405513 shutdown_connections 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 1 --2- 172.21.15.204:0/15405513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb43409fa20 0x7fb43409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 1 --2- 172.21.15.204:0/15405513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4340a08e0 0x7fb4340a0d50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 1 --2- 172.21.15.204:0/15405513 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb4340a2cd0 0x7fb434132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 1 -- 172.21.15.204:0/15405513 >> 172.21.15.204:0/15405513 conn(0x7fb43406e860 msgr2=0x7fb43406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:26.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 1 -- 172.21.15.204:0/15405513 shutdown_connections 2024-09-15T01:35:26.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 1 -- 172.21.15.204:0/15405513 wait complete. 2024-09-15T01:35:26.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 10 monclient: build_initial_monmap 2024-09-15T01:35:26.396 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.378+0000 7fb43999d640 10 monclient: monmap: 2024-09-15T01:35:26.396 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:26.396 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:26.396 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:26.396 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:26.396 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:26.396 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:26.396 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:26.396 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 1 Processor -- start 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 1 -- start start 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 10 monclient: init 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:26.397 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 10 monclient: _add_conns ranks=[0,3,2,4,1,5,7,6,8] 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb43409fa20 0x7fb434130b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 10 monclient(hunting): picked mon.noname-a con 0x7fb43409fa20 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4340a08e0 0x7fb43411d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 10 monclient(hunting): picked mon.noname-g con 0x7fb4340a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340a2cd0 0x7fb43411dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 10 monclient(hunting): picked mon.noname-b con 0x7fb4340a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4341212f0 con 0x7fb43409fa20 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43409d3d0 con 0x7fb4340a2cd0 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43409cb60 con 0x7fb4340a08e0 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:26.398 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb43999d640 10 monclient(hunting): authenticate will time out at 942.022639s 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4340a08e0 0x7fb43411d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb4337fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340a2cd0 msgr2=0x7fb43411dc20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4340a08e0 0x7fb43411d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:40294/0 (socket says 172.21.15.204:40294) 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb4337fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340a2cd0 0x7fb43411dc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 1 -- 172.21.15.204:0/870426310 learned_addr learned my addr 172.21.15.204:0/870426310 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 10 monclient(hunting): get_auth_request con 0x7fb4340a08e0 auth_method 0 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:26.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 10 monclient(hunting): handle_auth_done global_id 14709 payload 824 2024-09-15T01:35:26.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 1 -- 172.21.15.204:0/870426310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340a2cd0 msgr2=0x7fb43411dc20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:26.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb432ffd640 1 --2- 172.21.15.204:0/870426310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb43409fa20 0x7fb434130b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 1 --2- 172.21.15.204:0/870426310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340a2cd0 0x7fb43411dc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.379+0000 7fb42a7fc640 1 -- 172.21.15.204:0/870426310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb43409fa20 msgr2=0x7fb434130b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:26.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42a7fc640 1 --2- 172.21.15.204:0/870426310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb43409fa20 0x7fb434130b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42a7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:35:26.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42a7fc640 1 monclient: found mon.noname-g 2024-09-15T01:35:26.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:26.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42a7fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:26.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42a7fc640 1 -- 172.21.15.204:0/870426310 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c009390 con 0x7fb4340a08e0 2024-09-15T01:35:26.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42a7fc640 1 --2- 172.21.15.204:0/870426310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4340a08e0 0x7fb43411d6e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00eaa0 tx=0x7fb41c0090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:26.400 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42b7fe640 1 -- 172.21.15.204:0/870426310 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb41c00ce90 con 0x7fb4340a08e0 2024-09-15T01:35:26.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:26.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42b7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:26.401 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42b7fe640 10 monclient: dump: 2024-09-15T01:35:26.401 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:26.401 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.401 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.401 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.401 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:26.401 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.401 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42b7fe640 1 -- 172.21.15.204:0/870426310 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c013470 con 0x7fb4340a08e0 2024-09-15T01:35:26.402 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42b7fe640 1 -- 172.21.15.204:0/870426310 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb41c013a60 con 0x7fb4340a08e0 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb43999d640 5 monclient: authenticate success, global_id 14709 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb42b7fe640 10 monclient: dump: 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:26.403 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb431ffb640 4 set_mon_vals no callback set 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb43999d640 10 monclient: _renew_subs 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb43999d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb43999d640 1 -- 172.21.15.204:0/870426310 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42c009ae0 con 0x7fb4340a08e0 2024-09-15T01:35:26.404 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb43999d640 10 monclient: _renew_subs 2024-09-15T01:35:26.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb43999d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:26.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.380+0000 7fb43999d640 1 -- 172.21.15.204:0/870426310 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43411f930 con 0x7fb4340a08e0 2024-09-15T01:35:26.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.381+0000 7fb42b7fe640 1 -- 172.21.15.204:0/870426310 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb41c013bc0 con 0x7fb4340a08e0 2024-09-15T01:35:26.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.381+0000 7fb43999d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:26.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.381+0000 7fb43999d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:26.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.381+0000 7fb43999d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:26.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.381+0000 7fb42b7fe640 1 --2- 172.21.15.204:0/870426310 >> v2:172.21.15.204:6812/28210 conn(0x7fb4140397e0 0x7fb41403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.381+0000 7fb43999d640 1 -- 172.21.15.204:0/870426310 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb434121010 con 0x7fb4340a08e0 2024-09-15T01:35:26.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.381+0000 7fb42b7fe640 1 -- 172.21.15.204:0/870426310 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5683+0+0 (secure 0 0 0) 0x7fb41c04bb10 con 0x7fb4340a08e0 2024-09-15T01:35:26.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.381+0000 7fb432ffd640 1 --2- 172.21.15.204:0/870426310 >> v2:172.21.15.204:6812/28210 conn(0x7fb4140397e0 0x7fb41403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.381+0000 7fb432ffd640 10 monclient: get_auth_request con 0x7fb4140397e0 auth_method 0 2024-09-15T01:35:26.405 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.381+0000 7fb432ffd640 1 --2- 172.21.15.204:0/870426310 >> v2:172.21.15.204:6812/28210 conn(0x7fb4140397e0 0x7fb41403bc90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb424006fd0 tx=0x7fb424008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:26.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.385+0000 7fb42b7fe640 1 -- 172.21.15.204:0/870426310 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb41c00ca10 con 0x7fb4340a08e0 2024-09-15T01:35:26.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.385+0000 7fb42b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:26.406 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.385+0000 7fb42b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:26.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.548+0000 7fb43999d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:26.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.548+0000 7fb43999d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:26.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.548+0000 7fb43999d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:26.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.548+0000 7fb43999d640 1 -- 172.21.15.204:0/870426310 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) v1 -- 0x7fb434070b80 con 0x7fb4340a08e0 2024-09-15T01:35:26.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.548+0000 7fb42b7fe640 1 -- 172.21.15.204:0/870426310 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=-2 pool 'fooset' not found v121) v1 ==== 152+0+0 (secure 0 0 0) 0x7fb41c0192c0 con 0x7fb4340a08e0 2024-09-15T01:35:26.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.548+0000 7fb42b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:26.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.548+0000 7fb42b7fe640 10 monclient: _finish_command 2 = mon:2 pool 'fooset' not found 2024-09-15T01:35:26.551 INFO:tasks.workunit.client.0.smithi204.stderr:Error ENOENT: pool 'fooset' not found 2024-09-15T01:35:26.555 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:35:26.555 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool create fooset 8 2024-09-15T01:35:26.632 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.629+0000 7f2104e9f640 10 monclient: get_monmap_and_config 2024-09-15T01:35:26.632 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.629+0000 7f2104e9f640 10 monclient: build_initial_monmap 2024-09-15T01:35:26.632 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.629+0000 7f2104e9f640 10 monclient: monmap: 2024-09-15T01:35:26.632 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:26.632 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.632 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:26.631026+0000 2024-09-15T01:35:26.632 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:26.631026+0000 2024-09-15T01:35:26.632 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:26.632 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:26.633 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 1 Processor -- start 2024-09-15T01:35:26.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 1 -- start start 2024-09-15T01:35:26.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 10 monclient: init 2024-09-15T01:35:26.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:26.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 10 monclient: _add_conns ranks=[8,4,10,3,0,9,6,7,5,2,1] 2024-09-15T01:35:26.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f21000a2e30 0x7f2100132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 10 monclient(hunting): picked mon.noname-e con 0x7f21000a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:26.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f210009fb80 0x7f210009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 10 monclient(hunting): picked mon.noname-c con 0x7f210009fb80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:26.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f21000a0a40 0x7f21000a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 10 monclient(hunting): picked mon.noname-f con 0x7f21000a0a40 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:26.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210009e6c0 con 0x7f21000a0a40 2024-09-15T01:35:26.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21000a04b0 con 0x7f210009fb80 2024-09-15T01:35:26.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f21000a3200 con 0x7f21000a2e30 2024-09-15T01:35:26.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:26.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f2104e9f640 10 monclient(hunting): authenticate will time out at 942.273768s 2024-09-15T01:35:26.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f20fed9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f21000a0a40 0x7f21000a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f20fed9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f21000a0a40 0x7f21000a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36084/0 (socket says 172.21.15.204:36084) 2024-09-15T01:35:26.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f20fed9d640 1 -- 172.21.15.204:0/2695998839 learned_addr learned my addr 172.21.15.204:0/2695998839 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:26.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f20fed9d640 10 monclient(hunting): get_auth_request con 0x7f21000a0a40 auth_method 0 2024-09-15T01:35:26.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f20fed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f20fed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.630+0000 7f20fed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fe59c640 1 --2- 172.21.15.204:0/2695998839 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f21000a2e30 0x7f2100132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20f5d9b640 1 --2- 172.21.15.204:0/2695998839 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f210009fb80 0x7f210009ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fe59c640 10 monclient(hunting): get_auth_request con 0x7f21000a2e30 auth_method 0 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fe59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fe59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20f5d9b640 10 monclient(hunting): get_auth_request con 0x7f210009fb80 auth_method 0 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20f5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20f5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20f5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 10 monclient(hunting): handle_auth_done global_id 6175 payload 274 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 1 -- 172.21.15.204:0/2695998839 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f21000a2e30 msgr2=0x7f2100132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 1 --2- 172.21.15.204:0/2695998839 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f21000a2e30 0x7f2100132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 1 -- 172.21.15.204:0/2695998839 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f210009fb80 msgr2=0x7f210009ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 1 --2- 172.21.15.204:0/2695998839 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f210009fb80 0x7f210009ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 1 monclient: found mon.noname-f 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 1 -- 172.21.15.204:0/2695998839 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210004b720 con 0x7f21000a0a40 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.631+0000 7f20fed9d640 1 --2- 172.21.15.204:0/2695998839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21000a0a40 0x7f21000a0ee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009f20 tx=0x7f20f8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5c8cc122c7bf10a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:26.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f20fdd9b640 1 -- 172.21.15.204:0/2695998839 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20f8007880 con 0x7f21000a0a40 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f20fdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f20fdd9b640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f20fdd9b640 10 monclient: dump: 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:26.647 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f20fdd9b640 10 monclient: _finish_auth 0 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f20fdd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 5 monclient: authenticate success, global_id 6175 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f20fdd9b640 1 -- 172.21.15.204:0/2695998839 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f802d050 con 0x7f21000a0a40 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f20fdd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:26.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 4 set_mon_vals no callback set 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f20fdd9b640 1 -- 172.21.15.204:0/2695998839 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20f8030360 con 0x7f21000a0a40 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f20fdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f20fdd9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f20fdd9b640 10 monclient: dump: 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:26.649 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 10 monclient: shutdown 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 1 -- 172.21.15.204:0/2695998839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21000a0a40 msgr2=0x7f21000a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 1 --2- 172.21.15.204:0/2695998839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21000a0a40 0x7f21000a0ee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009f20 tx=0x7f20f8009840 comp rx=0 tx=0).stop 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 1 -- 172.21.15.204:0/2695998839 shutdown_connections 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 1 --2- 172.21.15.204:0/2695998839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21000a0a40 0x7f21000a0ee0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 1 --2- 172.21.15.204:0/2695998839 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f210009fb80 0x7f210009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 1 --2- 172.21.15.204:0/2695998839 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f21000a2e30 0x7f2100132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 1 -- 172.21.15.204:0/2695998839 >> 172.21.15.204:0/2695998839 conn(0x7f210006e860 msgr2=0x7f210006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 1 -- 172.21.15.204:0/2695998839 shutdown_connections 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.632+0000 7f2104e9f640 1 -- 172.21.15.204:0/2695998839 wait complete. 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.633+0000 7f2104e9f640 10 monclient: build_initial_monmap 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.633+0000 7f2104e9f640 10 monclient: monmap: 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:26.651 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.633+0000 7f2104e9f640 1 Processor -- start 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.633+0000 7f2104e9f640 1 -- start start 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.633+0000 7f2104e9f640 10 monclient: init 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.633+0000 7f2104e9f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:26.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.633+0000 7f2104e9f640 10 monclient: _add_conns ranks=[0,5,8,4,2,7,6,1,3] 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.633+0000 7f2104e9f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f210009fb80 0x7f21001318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 10 monclient(hunting): picked mon.noname-a con 0x7f210009fb80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f21000a0a40 0x7f210011e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 10 monclient(hunting): picked mon.noname-h con 0x7f21000a0a40 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f21000a2e30 0x7f210011e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 10 monclient(hunting): picked mon.noname-e con 0x7f21000a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2100142a20 con 0x7f210009fb80 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2100121a00 con 0x7f21000a0a40 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f210009ca40 con 0x7f21000a2e30 2024-09-15T01:35:26.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 10 monclient(hunting): authenticate will time out at 942.277047s 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f21000a0a40 0x7f210011e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f21000a0a40 0x7f210011e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:49750/0 (socket says 172.21.15.204:49750) 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 1 -- 172.21.15.204:0/194977048 learned_addr learned my addr 172.21.15.204:0/194977048 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 10 monclient(hunting): get_auth_request con 0x7f21000a0a40 auth_method 0 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fed9d640 1 --2- 172.21.15.204:0/194977048 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f21000a2e30 0x7f210011e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fe59c640 1 --2- 172.21.15.204:0/194977048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f210009fb80 0x7f21001318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fed9d640 10 monclient(hunting): get_auth_request con 0x7f21000a2e30 auth_method 0 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fe59c640 10 monclient(hunting): get_auth_request con 0x7f210009fb80 auth_method 0 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fe59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fe59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fe59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:26.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fe59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:26.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fe59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:26.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 10 monclient(hunting): handle_auth_done global_id 5657 payload 824 2024-09-15T01:35:26.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 1 -- 172.21.15.204:0/194977048 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f21000a2e30 msgr2=0x7f210011e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:26.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 1 --2- 172.21.15.204:0/194977048 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f21000a2e30 0x7f210011e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 1 -- 172.21.15.204:0/194977048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f210009fb80 msgr2=0x7f21001318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:26.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 1 --2- 172.21.15.204:0/194977048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f210009fb80 0x7f21001318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:26.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 1 monclient: found mon.noname-h 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 1 -- 172.21.15.204:0/194977048 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f8009ae0 con 0x7f21000a0a40 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f5d9b640 1 --2- 172.21.15.204:0/194977048 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f21000a0a40 0x7f210011e3b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f20e800d8c0 tx=0x7f20e8008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f6ffd640 1 -- 172.21.15.204:0/194977048 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20e800de50 con 0x7f21000a0a40 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f6ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f6ffd640 10 monclient: dump: 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.657 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 5 monclient: authenticate success, global_id 5657 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f6ffd640 1 -- 172.21.15.204:0/194977048 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e800bd50 con 0x7f21000a0a40 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f6ffd640 1 -- 172.21.15.204:0/194977048 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20e80128a0 con 0x7f21000a0a40 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 10 monclient: _renew_subs 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 1 -- 172.21.15.204:0/194977048 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210009d650 con 0x7f21000a0a40 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20fd59a640 4 set_mon_vals no callback set 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f20f6ffd640 10 monclient: dump: 2024-09-15T01:35:26.659 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 10 monclient: _renew_subs 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.634+0000 7f2104e9f640 1 -- 172.21.15.204:0/194977048 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2100120b10 con 0x7f21000a0a40 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.635+0000 7f20f6ffd640 1 -- 172.21.15.204:0/194977048 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f20e8012420 con 0x7f21000a0a40 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.635+0000 7f2104e9f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.635+0000 7f2104e9f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.635+0000 7f2104e9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.635+0000 7f2104e9f640 1 -- 172.21.15.204:0/194977048 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2100070cc0 con 0x7f21000a0a40 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.635+0000 7f20f6ffd640 1 --2- 172.21.15.204:0/194977048 >> v2:172.21.15.204:6812/28210 conn(0x7f20e00397b0 0x7f20e003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.635+0000 7f20f6ffd640 1 -- 172.21.15.204:0/194977048 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5683+0+0 (secure 0 0 0) 0x7f20e804bac0 con 0x7f21000a0a40 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.636+0000 7f20fe59c640 1 --2- 172.21.15.204:0/194977048 >> v2:172.21.15.204:6812/28210 conn(0x7f20e00397b0 0x7f20e003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.636+0000 7f20fe59c640 10 monclient: get_auth_request con 0x7f20e00397b0 auth_method 0 2024-09-15T01:35:26.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.636+0000 7f20fe59c640 1 --2- 172.21.15.204:0/194977048 >> v2:172.21.15.204:6812/28210 conn(0x7f20e00397b0 0x7f20e003bc60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f20f000bd40 tx=0x7f20f0008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:26.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.640+0000 7f20f6ffd640 1 -- 172.21.15.204:0/194977048 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f20e8012a20 con 0x7f21000a0a40 2024-09-15T01:35:26.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.640+0000 7f20f6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:26.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.640+0000 7f20f6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:26.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.797+0000 7f2104e9f640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-09-15T01:35:26.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.797+0000 7f2104e9f640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-09-15T01:35:26.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.797+0000 7f2104e9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:26.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:26.797+0000 7f2104e9f640 1 -- 172.21.15.204:0/194977048 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "fooset", "pg_num": 8} v 0) v1 -- 0x7f210009c820 con 0x7f21000a0a40 2024-09-15T01:35:27.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.576+0000 7f20f6ffd640 1 -- 172.21.15.204:0/194977048 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}]=0 pool 'fooset' created v122) v1 ==== 115+0+0 (secure 0 0 0) 0x7f20e801c360 con 0x7f21000a0a40 2024-09-15T01:35:27.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.576+0000 7f20f6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-09-15T01:35:27.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.576+0000 7f20f6ffd640 10 monclient: _finish_command 2 = system:0 pool 'fooset' created 2024-09-15T01:35:27.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.676+0000 7f2104e9f640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-09-15T01:35:27.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.676+0000 7f2104e9f640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-09-15T01:35:27.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.676+0000 7f2104e9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.676+0000 7f2104e9f640 1 -- 172.21.15.204:0/194977048 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "fooset", "pg_num": 8} v 0) v1 -- 0x7f210011ff40 con 0x7f21000a0a40 2024-09-15T01:35:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.676+0000 7f20f6ffd640 1 -- 172.21.15.204:0/194977048 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}]=0 pool 'fooset' already exists v122) v1 ==== 122+0+0 (secure 0 0 0) 0x7f20e801c540 con 0x7f21000a0a40 2024-09-15T01:35:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.676+0000 7f20f6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-09-15T01:35:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.676+0000 7f20f6ffd640 10 monclient: _finish_command 3 = system:0 pool 'fooset' already exists 2024-09-15T01:35:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'fooset' already exists 2024-09-15T01:35:27.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 1 -- 172.21.15.204:0/194977048 >> v2:172.21.15.204:6812/28210 conn(0x7f20e00397b0 msgr2=0x7f20e003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 1 --2- 172.21.15.204:0/194977048 >> v2:172.21.15.204:6812/28210 conn(0x7f20e00397b0 0x7f20e003bc60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f20f000bd40 tx=0x7f20f0008040 comp rx=0 tx=0).stop 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 10 monclient: shutdown 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 1 -- 172.21.15.204:0/194977048 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f21000a0a40 msgr2=0x7f210011e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 1 --2- 172.21.15.204:0/194977048 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f21000a0a40 0x7f210011e3b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f20e800d8c0 tx=0x7f20e8008040 comp rx=0 tx=0).stop 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 1 -- 172.21.15.204:0/194977048 shutdown_connections 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 1 --2- 172.21.15.204:0/194977048 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f21000a2e30 0x7f210011e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 1 --2- 172.21.15.204:0/194977048 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f21000a0a40 0x7f210011e3b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 1 --2- 172.21.15.204:0/194977048 >> v2:172.21.15.204:6812/28210 conn(0x7f20e00397b0 0x7f20e003bc60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 1 --2- 172.21.15.204:0/194977048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f210009fb80 0x7f21001318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 1 -- 172.21.15.204:0/194977048 >> 172.21.15.204:0/194977048 conn(0x7f210006e860 msgr2=0x7f210009ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 1 -- 172.21.15.204:0/194977048 shutdown_connections 2024-09-15T01:35:27.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.679+0000 7f2104e9f640 1 -- 172.21.15.204:0/194977048 wait complete. 2024-09-15T01:35:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool create barset 8 2024-09-15T01:35:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.766+0000 7f0f8526f640 10 monclient: get_monmap_and_config 2024-09-15T01:35:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.766+0000 7f0f8526f640 10 monclient: build_initial_monmap 2024-09-15T01:35:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.766+0000 7f0f8526f640 10 monclient: monmap: 2024-09-15T01:35:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:27.767618+0000 2024-09-15T01:35:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:27.767618+0000 2024-09-15T01:35:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:27.770 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.766+0000 7f0f8526f640 1 Processor -- start 2024-09-15T01:35:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.766+0000 7f0f8526f640 1 -- start start 2024-09-15T01:35:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.766+0000 7f0f8526f640 10 monclient: init 2024-09-15T01:35:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 10 monclient: _add_conns ranks=[2,6,10,0,4,5,9,3,8,7,1] 2024-09-15T01:35:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f800a4680 0x7f0f8009e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 10 monclient(hunting): picked mon.noname-b con 0x7f0f800a4680 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0f8006eea0 0x7f0f8009ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 10 monclient(hunting): picked mon.noname-d con 0x7f0f8006eea0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0f8009f2d0 0x7f0f801308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 10 monclient(hunting): picked mon.noname-f con 0x7f0f8009f2d0 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8009f6e0 con 0x7f0f8009f2d0 2024-09-15T01:35:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f800a4a50 con 0x7f0f800a4680 2024-09-15T01:35:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8006dc70 con 0x7f0f8006eea0 2024-09-15T01:35:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f8526f640 10 monclient(hunting): authenticate will time out at 943.410220s 2024-09-15T01:35:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0f8009f2d0 0x7f0f801308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7effd640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f800a4680 msgr2=0x7f0f8009e850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7e7fc640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0f8006eea0 msgr2=0x7f0f8009ed90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7effd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f800a4680 0x7f0f8009e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0f8009f2d0 0x7f0f801308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47278/0 (socket says 172.21.15.204:47278) 2024-09-15T01:35:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 1 -- 172.21.15.204:0/2161501594 learned_addr learned my addr 172.21.15.204:0/2161501594 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 10 monclient(hunting): get_auth_request con 0x7f0f8009f2d0 auth_method 0 2024-09-15T01:35:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7e7fc640 1 --2- 172.21.15.204:0/2161501594 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0f8006eea0 0x7f0f8009ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:27.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:27.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_done global_id 6184 payload 274 2024-09-15T01:35:27.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 1 -- 172.21.15.204:0/2161501594 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0f8006eea0 msgr2=0x7f0f8009ed90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:27.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 1 --2- 172.21.15.204:0/2161501594 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0f8006eea0 0x7f0f8009ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:27.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 1 -- 172.21.15.204:0/2161501594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f800a4680 msgr2=0x7f0f8009e850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:27.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 1 --2- 172.21.15.204:0/2161501594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f800a4680 0x7f0f8009e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:27.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:27.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 1 monclient: found mon.noname-f 2024-09-15T01:35:27.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:27.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:27.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.767+0000 7f0f7f7fe640 1 -- 172.21.15.204:0/2161501594 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8004b720 con 0x7f0f8009f2d0 2024-09-15T01:35:27.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7f7fe640 1 --2- 172.21.15.204:0/2161501594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f8009f2d0 0x7f0f801308a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009940 tx=0x7f0f700091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=941f0419b012e0ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:27.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7dffb640 1 -- 172.21.15.204:0/2161501594 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f7002acc0 con 0x7f0f8009f2d0 2024-09-15T01:35:27.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:27.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7dffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:27.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7dffb640 10 monclient: dump: 2024-09-15T01:35:27.783 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:27.783 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:27.783 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:27.783 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:27.783 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7dffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:27.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7dffb640 1 -- 172.21.15.204:0/2161501594 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7002ae40 con 0x7f0f8009f2d0 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7dffb640 1 -- 172.21.15.204:0/2161501594 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f70031ca0 con 0x7f0f8009f2d0 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f8526f640 5 monclient: authenticate success, global_id 6184 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f8526f640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f8526f640 4 set_mon_vals no callback set 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f7dffb640 10 monclient: dump: 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:27.785 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:27.786 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:27.786 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:27.786 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:27.786 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:27.786 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:27.786 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:27.786 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:27.786 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:27.786 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:27.786 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:27.786 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:27.786 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f8526f640 10 monclient: shutdown 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f8526f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f8526f640 1 -- 172.21.15.204:0/2161501594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f8009f2d0 msgr2=0x7f0f801308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.768+0000 7f0f8526f640 1 --2- 172.21.15.204:0/2161501594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f8009f2d0 0x7f0f801308a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009940 tx=0x7f0f700091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.769+0000 7f0f8526f640 1 -- 172.21.15.204:0/2161501594 shutdown_connections 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.769+0000 7f0f8526f640 1 --2- 172.21.15.204:0/2161501594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f8009f2d0 0x7f0f801308a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.769+0000 7f0f8526f640 1 --2- 172.21.15.204:0/2161501594 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0f8006eea0 0x7f0f8009ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.769+0000 7f0f8526f640 1 --2- 172.21.15.204:0/2161501594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f800a4680 0x7f0f8009e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.769+0000 7f0f8526f640 1 -- 172.21.15.204:0/2161501594 >> 172.21.15.204:0/2161501594 conn(0x7f0f80067740 msgr2=0x7f0f800a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.769+0000 7f0f8526f640 1 -- 172.21.15.204:0/2161501594 shutdown_connections 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.769+0000 7f0f8526f640 1 -- 172.21.15.204:0/2161501594 wait complete. 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.769+0000 7f0f8526f640 10 monclient: build_initial_monmap 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.769+0000 7f0f8526f640 10 monclient: monmap: 2024-09-15T01:35:27.787 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:27.788 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:27.788 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:27.788 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:27.788 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:27.788 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:27.788 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:27.788 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:27.788 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:27.788 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:27.788 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:27.788 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:27.788 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.769+0000 7f0f8526f640 1 Processor -- start 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 1 -- start start 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 10 monclient: init 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 10 monclient: _add_conns ranks=[6,3,1,7,8,2,4,0,5] 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0f8006eea0 0x7f0f8013c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 10 monclient(hunting): picked mon.noname-d con 0x7f0f8006eea0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8009f2d0 0x7f0f8013b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 10 monclient(hunting): picked mon.noname-g con 0x7f0f8009f2d0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f800a4680 0x7f0f8013f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:27.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 10 monclient(hunting): picked mon.noname-f con 0x7f0f800a4680 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80121120 con 0x7f0f800a4680 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80120b60 con 0x7f0f8009f2d0 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8011e620 con 0x7f0f8006eea0 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f8526f640 10 monclient(hunting): authenticate will time out at 943.413478s 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f800a4680 0x7f0f8013f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8009f2d0 0x7f0f8013b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7effd640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0f8006eea0 msgr2=0x7f0f8013c130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f800a4680 0x7f0f8013f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47284/0 (socket says 172.21.15.204:47284) 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7f7fe640 1 -- 172.21.15.204:0/2298259868 learned_addr learned my addr 172.21.15.204:0/2298259868 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:27.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7f7fe640 10 monclient(hunting): get_auth_request con 0x7f0f800a4680 auth_method 0 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7effd640 1 --2- 172.21.15.204:0/2298259868 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0f8006eea0 0x7f0f8013c130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 10 monclient(hunting): get_auth_request con 0x7f0f8009f2d0 auth_method 0 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:27.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 10 monclient(hunting): handle_auth_done global_id 14718 payload 824 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 1 -- 172.21.15.204:0/2298259868 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0f8006eea0 msgr2=0x7f0f8013c130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 1 --2- 172.21.15.204:0/2298259868 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0f8006eea0 0x7f0f8013c130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 1 -- 172.21.15.204:0/2298259868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f800a4680 msgr2=0x7f0f8013f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 1 --2- 172.21.15.204:0/2298259868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f800a4680 0x7f0f8013f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 1 monclient: found mon.noname-g 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7e7fc640 1 -- 172.21.15.204:0/2298259868 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f70009450 con 0x7f0f8009f2d0 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.770+0000 7f0f7f7fe640 1 --2- 172.21.15.204:0/2298259868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f800a4680 0x7f0f8013f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f7e7fc640 1 --2- 172.21.15.204:0/2298259868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8009f2d0 0x7f0f8013b7e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400b5b0 tx=0x7f0f740090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f66ffd640 1 -- 172.21.15.204:0/2298259868 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f74006150 con 0x7f0f8009f2d0 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f66ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f66ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:27.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f66ffd640 10 monclient: dump: 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:27.793 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f66ffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f66ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f8526f640 5 monclient: authenticate success, global_id 14718 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f66ffd640 1 -- 172.21.15.204:0/2298259868 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f740062f0 con 0x7f0f8009f2d0 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f66ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f66ffd640 1 -- 172.21.15.204:0/2298259868 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f740103e0 con 0x7f0f8009f2d0 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f8526f640 10 monclient: _renew_subs 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f8526f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f8526f640 1 -- 172.21.15.204:0/2298259868 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f80120880 con 0x7f0f8009f2d0 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f66ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f66ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f66ffd640 10 monclient: dump: 2024-09-15T01:35:27.794 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:27.795 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f7d7fa640 4 set_mon_vals no callback set 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f8526f640 10 monclient: _renew_subs 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f8526f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.771+0000 7f0f8526f640 1 -- 172.21.15.204:0/2298259868 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8013bd20 con 0x7f0f8009f2d0 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.772+0000 7f0f66ffd640 1 -- 172.21.15.204:0/2298259868 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0f74003b80 con 0x7f0f8009f2d0 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.772+0000 7f0f8526f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.772+0000 7f0f8526f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.772+0000 7f0f8526f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.772+0000 7f0f8526f640 1 -- 172.21.15.204:0/2298259868 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f80070cc0 con 0x7f0f8009f2d0 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.772+0000 7f0f66ffd640 1 --2- 172.21.15.204:0/2298259868 >> v2:172.21.15.204:6812/28210 conn(0x7f0f6c0397b0 0x7f0f6c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.772+0000 7f0f66ffd640 1 -- 172.21.15.204:0/2298259868 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6028+0+0 (secure 0 0 0) 0x7f0f7404ac70 con 0x7f0f8009f2d0 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.772+0000 7f0f7effd640 1 --2- 172.21.15.204:0/2298259868 >> v2:172.21.15.204:6812/28210 conn(0x7f0f6c0397b0 0x7f0f6c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.772+0000 7f0f7effd640 10 monclient: get_auth_request con 0x7f0f6c0397b0 auth_method 0 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.772+0000 7f0f7effd640 1 --2- 172.21.15.204:0/2298259868 >> v2:172.21.15.204:6812/28210 conn(0x7f0f6c0397b0 0x7f0f6c03bc60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0f78003c70 tx=0x7f0f78009310 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:27.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.776+0000 7f0f66ffd640 1 -- 172.21.15.204:0/2298259868 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0f7400daf0 con 0x7f0f8009f2d0 2024-09-15T01:35:27.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.776+0000 7f0f66ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:27.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.776+0000 7f0f66ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:27.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.931+0000 7f0f8526f640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-09-15T01:35:27.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.931+0000 7f0f8526f640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-09-15T01:35:27.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.931+0000 7f0f8526f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:27.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:27.931+0000 7f0f8526f640 1 -- 172.21.15.204:0/2298259868 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "barset", "pg_num": 8} v 0) v1 -- 0x7f0f8006a8b0 con 0x7f0f8009f2d0 2024-09-15T01:35:28.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.585+0000 7f0f66ffd640 1 -- 172.21.15.204:0/2298259868 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}]=0 pool 'barset' created v123) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0f74016070 con 0x7f0f8009f2d0 2024-09-15T01:35:28.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.586+0000 7f0f66ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-09-15T01:35:28.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.586+0000 7f0f66ffd640 10 monclient: _finish_command 2 = system:0 pool 'barset' created 2024-09-15T01:35:28.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.685+0000 7f0f8526f640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-09-15T01:35:28.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.685+0000 7f0f8526f640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-09-15T01:35:28.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.685+0000 7f0f8526f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:28.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.685+0000 7f0f8526f640 1 -- 172.21.15.204:0/2298259868 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "barset", "pg_num": 8} v 0) v1 -- 0x7f0f8011f9b0 con 0x7f0f8009f2d0 2024-09-15T01:35:28.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.685+0000 7f0f66ffd640 1 -- 172.21.15.204:0/2298259868 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}]=0 pool 'barset' already exists v123) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0f7400d310 con 0x7f0f8009f2d0 2024-09-15T01:35:28.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.685+0000 7f0f66ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-09-15T01:35:28.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.685+0000 7f0f66ffd640 10 monclient: _finish_command 3 = system:0 pool 'barset' already exists 2024-09-15T01:35:28.688 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'barset' already exists 2024-09-15T01:35:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 1 -- 172.21.15.204:0/2298259868 >> v2:172.21.15.204:6812/28210 conn(0x7f0f6c0397b0 msgr2=0x7f0f6c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 1 --2- 172.21.15.204:0/2298259868 >> v2:172.21.15.204:6812/28210 conn(0x7f0f6c0397b0 0x7f0f6c03bc60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0f78003c70 tx=0x7f0f78009310 comp rx=0 tx=0).stop 2024-09-15T01:35:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 10 monclient: shutdown 2024-09-15T01:35:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 1 -- 172.21.15.204:0/2298259868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8009f2d0 msgr2=0x7f0f8013b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 1 --2- 172.21.15.204:0/2298259868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8009f2d0 0x7f0f8013b7e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400b5b0 tx=0x7f0f740090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 1 -- 172.21.15.204:0/2298259868 shutdown_connections 2024-09-15T01:35:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 1 --2- 172.21.15.204:0/2298259868 >> v2:172.21.15.204:6812/28210 conn(0x7f0f6c0397b0 0x7f0f6c03bc60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 1 --2- 172.21.15.204:0/2298259868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f800a4680 0x7f0f8013f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 1 --2- 172.21.15.204:0/2298259868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f8009f2d0 0x7f0f8013b7e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 1 --2- 172.21.15.204:0/2298259868 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0f8006eea0 0x7f0f8013c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 1 -- 172.21.15.204:0/2298259868 >> 172.21.15.204:0/2298259868 conn(0x7f0f80067740 msgr2=0x7f0f800a25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 1 -- 172.21.15.204:0/2298259868 shutdown_connections 2024-09-15T01:35:28.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.687+0000 7f0f8526f640 1 -- 172.21.15.204:0/2298259868 wait complete. 2024-09-15T01:35:28.699 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set barset size 3 2024-09-15T01:35:28.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.772+0000 7f01b46a8640 10 monclient: get_monmap_and_config 2024-09-15T01:35:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.772+0000 7f01b46a8640 10 monclient: build_initial_monmap 2024-09-15T01:35:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.772+0000 7f01b46a8640 10 monclient: monmap: 2024-09-15T01:35:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:28.773817+0000 2024-09-15T01:35:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:28.773817+0000 2024-09-15T01:35:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:28.776 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.772+0000 7f01b46a8640 1 Processor -- start 2024-09-15T01:35:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.772+0000 7f01b46a8640 1 -- start start 2024-09-15T01:35:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.772+0000 7f01b46a8640 10 monclient: init 2024-09-15T01:35:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 10 monclient: _add_conns ranks=[0,5,9,4,2,8,7,6,1,3,10] 2024-09-15T01:35:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f01ac0a2cd0 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f01ac09fa20 0x7f01ac09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 10 monclient(hunting): picked mon.noname-h con 0x7f01ac09fa20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f01ac137c80 0x7f01ac13a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 10 monclient(hunting): picked mon.noname-a con 0x7f01ac137c80 addr v2:172.21.15.178:3300/0 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ac0648a0 con 0x7f01ac0a2cd0 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac133310 con 0x7f01ac137c80 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0a30a0 con 0x7f01ac09fa20 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b46a8640 10 monclient(hunting): authenticate will time out at 944.416606s 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f01ac09fa20 0x7f01ac09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f01ac09fa20 0x7f01ac09fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:45894/0 (socket says 172.21.15.204:45894) 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 1 -- 172.21.15.204:0/2229621114 learned_addr learned my addr 172.21.15.204:0/2229621114 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 10 monclient(hunting): get_auth_request con 0x7f01ac09fa20 auth_method 0 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 10 monclient(hunting): handle_auth_done global_id 5666 payload 274 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 1 -- 172.21.15.204:0/2229621114 >> v2:172.21.15.178:3300/0 conn(0x7f01ac137c80 msgr2=0x7f01ac13a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 1 --2- 172.21.15.204:0/2229621114 >> v2:172.21.15.178:3300/0 conn(0x7f01ac137c80 0x7f01ac13a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 1 -- 172.21.15.204:0/2229621114 >> v1:172.21.15.178:6789/0 conn(0x7f01ac0a2cd0 legacy=0x7f01ac132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 10 monclient: _finish_hunting 0 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 1 monclient: found mon.noname-h 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.773+0000 7f01b1c43640 1 -- 172.21.15.204:0/2229621114 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac0a0330 con 0x7f01ac09fa20 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1c43640 1 --2- 172.21.15.204:0/2229621114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f01ac09fa20 0x7f01ac09fdf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f019c009910 tx=0x7f019c0091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=7e39553fd3342e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1442640 1 -- 172.21.15.204:0/2229621114 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f019c02ac90 con 0x7f01ac09fa20 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1442640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1442640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1442640 10 monclient: dump: 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1442640 10 monclient: _finish_auth 0 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1442640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1442640 1 -- 172.21.15.204:0/2229621114 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c02ae10 con 0x7f01ac09fa20 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b46a8640 5 monclient: authenticate success, global_id 5666 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b46a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1442640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b46a8640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b46a8640 4 set_mon_vals no callback set 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1442640 1 -- 172.21.15.204:0/2229621114 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f019c031ca0 con 0x7f01ac09fa20 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1442640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1442640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.774+0000 7f01b1442640 10 monclient: dump: 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.775+0000 7f01b46a8640 10 monclient: shutdown 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.775+0000 7f01b46a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.775+0000 7f01b46a8640 1 -- 172.21.15.204:0/2229621114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f01ac09fa20 msgr2=0x7f01ac09fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.775+0000 7f01b46a8640 1 --2- 172.21.15.204:0/2229621114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f01ac09fa20 0x7f01ac09fdf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f019c009910 tx=0x7f019c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.775+0000 7f01b46a8640 1 -- 172.21.15.204:0/2229621114 shutdown_connections 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.775+0000 7f01b46a8640 1 --2- 172.21.15.204:0/2229621114 >> v2:172.21.15.178:3300/0 conn(0x7f01ac137c80 0x7f01ac13a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.775+0000 7f01b46a8640 1 --2- 172.21.15.204:0/2229621114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f01ac09fa20 0x7f01ac09fdf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.775+0000 7f01b46a8640 1 -- 172.21.15.204:0/2229621114 >> 172.21.15.204:0/2229621114 conn(0x7f01ac06e860 msgr2=0x7f01ac06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.775+0000 7f01b46a8640 1 -- 172.21.15.204:0/2229621114 shutdown_connections 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.775+0000 7f01b46a8640 1 -- 172.21.15.204:0/2229621114 wait complete. 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.775+0000 7f01b46a8640 10 monclient: build_initial_monmap 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.775+0000 7f01b46a8640 10 monclient: monmap: 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 1 Processor -- start 2024-09-15T01:35:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 1 -- start start 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 10 monclient: init 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 10 monclient: _add_conns ranks=[0,7,4,6,8,2,1,5,3] 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01ac09fa20 0x7f01ac1422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 10 monclient(hunting): picked mon.noname-a con 0x7f01ac09fa20 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01ac0a2cd0 0x7f01ac142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 10 monclient(hunting): picked mon.noname-i con 0x7f01ac0a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac137c80 0x7f01ac0998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 10 monclient(hunting): picked mon.noname-c con 0x7f01ac137c80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac09bdf0 con 0x7f01ac09fa20 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac09cb60 con 0x7f01ac137c80 2024-09-15T01:35:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac09b900 con 0x7f01ac0a2cd0 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b46a8640 10 monclient(hunting): authenticate will time out at 944.419847s 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b1c43640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01ac0a2cd0 msgr2=0x7f01ac142830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.776+0000 7f01b1c43640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01ac0a2cd0 0x7f01ac142830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01ac09fa20 0x7f01ac1422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01ac09fa20 0x7f01ac1422f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:40172/0 (socket says 172.21.15.204:40172) 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 1 -- 172.21.15.204:0/1370834163 learned_addr learned my addr 172.21.15.204:0/1370834163 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 10 monclient(hunting): get_auth_request con 0x7f01ac09fa20 auth_method 0 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2c45640 1 --2- 172.21.15.204:0/1370834163 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac137c80 0x7f01ac0998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2c45640 10 monclient(hunting): get_auth_request con 0x7f01ac137c80 auth_method 0 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2c45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2c45640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2c45640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2c45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2c45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2c45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 10 monclient(hunting): handle_auth_done global_id 5787 payload 824 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 1 -- 172.21.15.204:0/1370834163 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01ac0a2cd0 msgr2=0x7f01ac142830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 1 --2- 172.21.15.204:0/1370834163 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01ac0a2cd0 0x7f01ac142830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:28.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 1 -- 172.21.15.204:0/1370834163 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac137c80 msgr2=0x7f01ac0998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 1 --2- 172.21.15.204:0/1370834163 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac137c80 0x7f01ac0998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 10 monclient: _finish_hunting 0 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 1 monclient: found mon.noname-a 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 1 -- 172.21.15.204:0/1370834163 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c009450 con 0x7f01ac09fa20 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2c45640 1 --2- 172.21.15.204:0/1370834163 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac137c80 0x7f01ac0998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b2444640 1 --2- 172.21.15.204:0/1370834163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01ac09fa20 0x7f01ac1422f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f01a800b5e0 tx=0x7f01a80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01a27fc640 1 -- 172.21.15.204:0/1370834163 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01a800fa10 con 0x7f01ac09fa20 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01a27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01a27fc640 10 monclient: dump: 2024-09-15T01:35:28.796 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:28.797 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:28.798 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:28.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01a27fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:28.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:28.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b46a8640 5 monclient: authenticate success, global_id 5787 2024-09-15T01:35:28.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01a27fc640 1 -- 172.21.15.204:0/1370834163 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a800fbd0 con 0x7f01ac09fa20 2024-09-15T01:35:28.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b46a8640 10 monclient: _renew_subs 2024-09-15T01:35:28.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b46a8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:28.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b46a8640 1 -- 172.21.15.204:0/1370834163 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac09d3d0 con 0x7f01ac09fa20 2024-09-15T01:35:28.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01a27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:28.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01a27fc640 1 -- 172.21.15.204:0/1370834163 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01a800fd80 con 0x7f01ac09fa20 2024-09-15T01:35:28.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:28.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01a27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01a27fc640 10 monclient: dump: 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:28.799 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b0c41640 4 set_mon_vals no callback set 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b46a8640 10 monclient: _renew_subs 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b46a8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.777+0000 7f01b46a8640 1 -- 172.21.15.204:0/1370834163 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac09bfc0 con 0x7f01ac09fa20 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.778+0000 7f01b46a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.779+0000 7f01b46a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.779+0000 7f01b46a8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.779+0000 7f01b46a8640 1 -- 172.21.15.204:0/1370834163 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ac070b80 con 0x7f01ac09fa20 2024-09-15T01:35:28.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.779+0000 7f01a27fc640 1 -- 172.21.15.204:0/1370834163 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f01a800f590 con 0x7f01ac09fa20 2024-09-15T01:35:28.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.779+0000 7f01a27fc640 1 --2- 172.21.15.204:0/1370834163 >> v2:172.21.15.204:6812/28210 conn(0x7f01900397b0 0x7f019003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:28.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.779+0000 7f01a27fc640 1 -- 172.21.15.204:0/1370834163 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f01a804b530 con 0x7f01ac09fa20 2024-09-15T01:35:28.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.779+0000 7f01b1c43640 1 --2- 172.21.15.204:0/1370834163 >> v2:172.21.15.204:6812/28210 conn(0x7f01900397b0 0x7f019003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:28.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.779+0000 7f01b1c43640 10 monclient: get_auth_request con 0x7f01900397b0 auth_method 0 2024-09-15T01:35:28.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.779+0000 7f01b1c43640 1 --2- 172.21.15.204:0/1370834163 >> v2:172.21.15.204:6812/28210 conn(0x7f01900397b0 0x7f019003bc60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f019c034000 tx=0x7f019c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:28.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.784+0000 7f01a27fc640 1 -- 172.21.15.204:0/1370834163 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f01a800f840 con 0x7f01ac09fa20 2024-09-15T01:35:28.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.784+0000 7f01a27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:28.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.784+0000 7f01a27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:28.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.936+0000 7f01b46a8640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-09-15T01:35:28.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.937+0000 7f01b46a8640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-09-15T01:35:28.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.937+0000 7f01b46a8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:28.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:28.937+0000 7f01b46a8640 1 -- 172.21.15.204:0/1370834163 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"} v 0) v1 -- 0x7f01ac137a60 con 0x7f01ac09fa20 2024-09-15T01:35:29.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:29.590+0000 7f01a27fc640 1 -- 172.21.15.204:0/1370834163 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}]=0 set pool 10 size to 3 v124) v1 ==== 126+0+0 (secure 0 0 0) 0x7f01a800d310 con 0x7f01ac09fa20 2024-09-15T01:35:29.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:29.592+0000 7f01a27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-09-15T01:35:29.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:29.592+0000 7f01a27fc640 10 monclient: _finish_command 2 = system:0 set pool 10 size to 3 2024-09-15T01:35:29.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:29.687+0000 7f01b46a8640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-09-15T01:35:29.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:29.687+0000 7f01b46a8640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-09-15T01:35:29.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:29.687+0000 7f01b46a8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:29.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:29.687+0000 7f01b46a8640 1 -- 172.21.15.204:0/1370834163 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"} v 0) v1 -- 0x7f01ac0008d0 con 0x7f01ac09fa20 2024-09-15T01:35:30.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.594+0000 7f01a27fc640 1 -- 172.21.15.204:0/1370834163 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}]=0 set pool 10 size to 3 v125) v1 ==== 126+0+0 (secure 0 0 0) 0x7f01a8017310 con 0x7f01ac09fa20 2024-09-15T01:35:30.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.594+0000 7f01a27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-09-15T01:35:30.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.594+0000 7f01a27fc640 10 monclient: _finish_command 3 = system:0 set pool 10 size to 3 2024-09-15T01:35:30.597 INFO:tasks.workunit.client.0.smithi204.stderr:set pool 10 size to 3 2024-09-15T01:35:30.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 1 -- 172.21.15.204:0/1370834163 >> v2:172.21.15.204:6812/28210 conn(0x7f01900397b0 msgr2=0x7f019003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:30.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 1 --2- 172.21.15.204:0/1370834163 >> v2:172.21.15.204:6812/28210 conn(0x7f01900397b0 0x7f019003bc60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f019c034000 tx=0x7f019c032000 comp rx=0 tx=0).stop 2024-09-15T01:35:30.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 10 monclient: shutdown 2024-09-15T01:35:30.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:30.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 1 -- 172.21.15.204:0/1370834163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01ac09fa20 msgr2=0x7f01ac1422f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:30.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 1 --2- 172.21.15.204:0/1370834163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01ac09fa20 0x7f01ac1422f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f01a800b5e0 tx=0x7f01a80090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 1 -- 172.21.15.204:0/1370834163 shutdown_connections 2024-09-15T01:35:30.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 1 --2- 172.21.15.204:0/1370834163 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f01ac137c80 0x7f01ac0998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 1 --2- 172.21.15.204:0/1370834163 >> v2:172.21.15.204:6812/28210 conn(0x7f01900397b0 0x7f019003bc60 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f019c034000 tx=0x7f019c032000 comp rx=0 tx=0).stop 2024-09-15T01:35:30.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 1 --2- 172.21.15.204:0/1370834163 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f01ac0a2cd0 0x7f01ac142830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 1 --2- 172.21.15.204:0/1370834163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01ac09fa20 0x7f01ac1422f0 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f01a800b5e0 tx=0x7f01a80090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 1 -- 172.21.15.204:0/1370834163 >> 172.21.15.204:0/1370834163 conn(0x7f01ac06e860 msgr2=0x7f01ac09f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:30.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 1 -- 172.21.15.204:0/1370834163 shutdown_connections 2024-09-15T01:35:30.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.601+0000 7f01b46a8640 1 -- 172.21.15.204:0/1370834163 wait complete. 2024-09-15T01:35:30.616 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush weight-set reweight fooset osd.0 .9 2024-09-15T01:35:30.617 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:35:30.617 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 2024-09-15T01:35:30.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.695+0000 7f904986f640 10 monclient: get_monmap_and_config 2024-09-15T01:35:30.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.695+0000 7f904986f640 10 monclient: build_initial_monmap 2024-09-15T01:35:30.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.695+0000 7f904986f640 10 monclient: monmap: 2024-09-15T01:35:30.698 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:30.697435+0000 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:30.697435+0000 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:30.699 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:30.700 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:30.700 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:30.700 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:30.700 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:30.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.696+0000 7f904986f640 1 Processor -- start 2024-09-15T01:35:30.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.696+0000 7f904986f640 1 -- start start 2024-09-15T01:35:30.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.696+0000 7f904986f640 10 monclient: init 2024-09-15T01:35:30.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.696+0000 7f904986f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:30.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.696+0000 7f904986f640 10 monclient: _add_conns ranks=[8,5,3,6,7,1,9,10,4,2,0] 2024-09-15T01:35:30.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f90440a2cd0 0x7f9044132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 10 monclient(hunting): picked mon.noname-e con 0x7f90440a2cd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:30.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f904409fa20 0x7f904409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 10 monclient(hunting): picked mon.noname-h con 0x7f904409fa20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:30.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90440a08e0 0x7f90440a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 10 monclient(hunting): picked mon.noname-g con 0x7f90440a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:30.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:30.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90440a30a0 con 0x7f90440a08e0 2024-09-15T01:35:30.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:30.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90440a0330 con 0x7f904409fa20 2024-09-15T01:35:30.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:30.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f90440a1db0 con 0x7f90440a2cd0 2024-09-15T01:35:30.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:30.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f904986f640 10 monclient(hunting): authenticate will time out at 946.340073s 2024-09-15T01:35:30.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90437fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90440a08e0 0x7f90440a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:30.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f904409fa20 0x7f904409fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90437fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90440a08e0 0x7f90440a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:50730/0 (socket says 172.21.15.204:50730) 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f904409fa20 0x7f904409fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:45898/0 (socket says 172.21.15.204:45898) 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90437fe640 1 -- 172.21.15.204:0/1422191010 learned_addr learned my addr 172.21.15.204:0/1422191010 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90437fe640 10 monclient(hunting): get_auth_request con 0x7f90440a08e0 auth_method 0 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 10 monclient(hunting): get_auth_request con 0x7f904409fa20 auth_method 0 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:30.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f9042ffd640 1 --2- 172.21.15.204:0/1422191010 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f90440a2cd0 0x7f9044132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f9042ffd640 10 monclient(hunting): get_auth_request con 0x7f90440a2cd0 auth_method 0 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f9042ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f9042ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f9042ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 10 monclient(hunting): handle_auth_done global_id 5675 payload 274 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 1 -- 172.21.15.204:0/1422191010 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f90440a2cd0 msgr2=0x7f9044132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 1 --2- 172.21.15.204:0/1422191010 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f90440a2cd0 0x7f9044132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 1 -- 172.21.15.204:0/1422191010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90440a08e0 msgr2=0x7f90440a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 1 --2- 172.21.15.204:0/1422191010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90440a08e0 0x7f90440a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 10 monclient: _finish_hunting 0 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 1 monclient: found mon.noname-h 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90427fc640 1 -- 172.21.15.204:0/1422191010 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440648a0 con 0x7f904409fa20 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.697+0000 7f90437fe640 1 --2- 172.21.15.204:0/1422191010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90440a08e0 0x7f90440a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f90427fc640 1 --2- 172.21.15.204:0/1422191010 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f904409fa20 0x7f904409fdf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f902c009880 tx=0x7f902c0090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=bc14f21295f0809c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f9041ffb640 1 -- 172.21.15.204:0/1422191010 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f902c02abb0 con 0x7f904409fa20 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f9041ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f9041ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f9041ffb640 10 monclient: dump: 2024-09-15T01:35:30.712 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:30.713 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:30.714 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:30.714 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:30.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f9041ffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:30.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f9041ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:30.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f904986f640 5 monclient: authenticate success, global_id 5675 2024-09-15T01:35:30.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f904986f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:30.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f9041ffb640 1 -- 172.21.15.204:0/1422191010 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c02ad30 con 0x7f904409fa20 2024-09-15T01:35:30.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f9041ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:30.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f904986f640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:30.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f904986f640 4 set_mon_vals no callback set 2024-09-15T01:35:30.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f9041ffb640 1 -- 172.21.15.204:0/1422191010 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f902c031ca0 con 0x7f904409fa20 2024-09-15T01:35:30.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f9041ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f9041ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f9041ffb640 10 monclient: dump: 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:30.715 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f904986f640 10 monclient: shutdown 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f904986f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f904986f640 1 -- 172.21.15.204:0/1422191010 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f904409fa20 msgr2=0x7f904409fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.698+0000 7f904986f640 1 --2- 172.21.15.204:0/1422191010 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f904409fa20 0x7f904409fdf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f902c009880 tx=0x7f902c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.699+0000 7f904986f640 1 -- 172.21.15.204:0/1422191010 shutdown_connections 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.699+0000 7f904986f640 1 --2- 172.21.15.204:0/1422191010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90440a08e0 0x7f90440a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.699+0000 7f904986f640 1 --2- 172.21.15.204:0/1422191010 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f904409fa20 0x7f904409fdf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.699+0000 7f904986f640 1 --2- 172.21.15.204:0/1422191010 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f90440a2cd0 0x7f9044132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.699+0000 7f904986f640 1 -- 172.21.15.204:0/1422191010 >> 172.21.15.204:0/1422191010 conn(0x7f904406e860 msgr2=0x7f904406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.699+0000 7f904986f640 1 -- 172.21.15.204:0/1422191010 shutdown_connections 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.699+0000 7f904986f640 1 -- 172.21.15.204:0/1422191010 wait complete. 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.699+0000 7f904986f640 10 monclient: build_initial_monmap 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.699+0000 7f904986f640 10 monclient: monmap: 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:30.717 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.699+0000 7f904986f640 1 Processor -- start 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 1 -- start start 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 10 monclient: init 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:30.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 10 monclient: _add_conns ranks=[8,6,4,7,2,3,1,5,0] 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f904409fa20 0x7f9044130c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 10 monclient(hunting): picked mon.noname-e con 0x7f904409fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f90440a08e0 0x7f904411d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 10 monclient(hunting): picked mon.noname-d con 0x7f90440a08e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f90440a2cd0 0x7f904411dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 10 monclient(hunting): picked mon.noname-c con 0x7f90440a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9044121430 con 0x7f90440a2cd0 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f904409d3d0 con 0x7f90440a08e0 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f904409cb60 con 0x7f904409fa20 2024-09-15T01:35:30.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:30.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f904986f640 10 monclient(hunting): authenticate will time out at 946.343494s 2024-09-15T01:35:30.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f90427fc640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f90440a08e0 msgr2=0x7f904411d740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:30.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f90427fc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f90440a08e0 0x7f904411d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:30.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f9042ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f904409fa20 0x7f9044130c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:30.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f90437fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f90440a2cd0 0x7f904411dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:30.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f9042ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f904409fa20 0x7f9044130c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:50910/0 (socket says 172.21.15.204:50910) 2024-09-15T01:35:30.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f9042ffd640 1 -- 172.21.15.204:0/1670477079 learned_addr learned my addr 172.21.15.204:0/1670477079 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:30.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f9042ffd640 10 monclient(hunting): get_auth_request con 0x7f904409fa20 auth_method 0 2024-09-15T01:35:30.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f9042ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:30.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f9042ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:30.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f9042ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:30.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f90437fe640 10 monclient(hunting): get_auth_request con 0x7f90440a2cd0 auth_method 0 2024-09-15T01:35:30.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f90437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:30.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f90437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:30.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f90437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:30.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f90437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:30.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f90437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:30.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f90437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f9042ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f9042ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f9042ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.700+0000 7f90437fe640 10 monclient(hunting): handle_auth_done global_id 5575 payload 824 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f90437fe640 1 -- 172.21.15.204:0/1670477079 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f904409fa20 msgr2=0x7f9044130c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f90437fe640 1 --2- 172.21.15.204:0/1670477079 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f904409fa20 0x7f9044130c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f90437fe640 1 -- 172.21.15.204:0/1670477079 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f90440a08e0 msgr2=0x7f904411d740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f90437fe640 1 --2- 172.21.15.204:0/1670477079 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f90440a08e0 0x7f904411d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f90437fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f90437fe640 1 monclient: found mon.noname-c 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f90437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f90437fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:35:30.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f90437fe640 1 -- 172.21.15.204:0/1670477079 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9034009ae0 con 0x7f90440a2cd0 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f90437fe640 1 --2- 172.21.15.204:0/1670477079 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f90440a2cd0 0x7f904411dc80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9034009ee0 tx=0x7f903400b040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f903affd640 1 -- 172.21.15.204:0/1670477079 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f903400cc00 con 0x7f90440a2cd0 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f903affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f903affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f903affd640 10 monclient: dump: 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:30.723 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f903affd640 10 monclient: _finish_auth 0 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f903affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f904986f640 5 monclient: authenticate success, global_id 5575 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f903affd640 1 -- 172.21.15.204:0/1670477079 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903400f040 con 0x7f90440a2cd0 2024-09-15T01:35:30.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f904986f640 10 monclient: _renew_subs 2024-09-15T01:35:30.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f904986f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:30.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f904986f640 1 -- 172.21.15.204:0/1670477079 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902c009390 con 0x7f90440a2cd0 2024-09-15T01:35:30.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f903affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:30.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f903affd640 1 -- 172.21.15.204:0/1670477079 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f903400cda0 con 0x7f90440a2cd0 2024-09-15T01:35:30.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f903affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:30.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f903affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:30.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f903affd640 10 monclient: dump: 2024-09-15T01:35:30.725 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:30.725 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:30.725 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.725 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.725 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:30.726 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:30.726 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f904986f640 10 monclient: _renew_subs 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f90417fa640 4 set_mon_vals no callback set 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f904986f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.701+0000 7f904986f640 1 -- 172.21.15.204:0/1670477079 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904411fa20 con 0x7f90440a2cd0 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.702+0000 7f903affd640 1 -- 172.21.15.204:0/1670477079 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9034014480 con 0x7f90440a2cd0 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.702+0000 7f904986f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.702+0000 7f904986f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.702+0000 7f904986f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.702+0000 7f904986f640 1 -- 172.21.15.204:0/1670477079 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9044121110 con 0x7f90440a2cd0 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.702+0000 7f903affd640 1 --2- 172.21.15.204:0/1670477079 >> v2:172.21.15.204:6812/28210 conn(0x7f90240396e0 0x7f902403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.702+0000 7f903affd640 1 -- 172.21.15.204:0/1670477079 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f903404c3c0 con 0x7f90440a2cd0 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.702+0000 7f9042ffd640 1 --2- 172.21.15.204:0/1670477079 >> v2:172.21.15.204:6812/28210 conn(0x7f90240396e0 0x7f902403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.702+0000 7f9042ffd640 10 monclient: get_auth_request con 0x7f90240396e0 auth_method 0 2024-09-15T01:35:30.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.702+0000 7f9042ffd640 1 --2- 172.21.15.204:0/1670477079 >> v2:172.21.15.204:6812/28210 conn(0x7f90240396e0 0x7f902403bb90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f903c00bc30 tx=0x7f903c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:30.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.707+0000 7f903affd640 1 -- 172.21.15.204:0/1670477079 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f903401c7e0 con 0x7f90440a2cd0 2024-09-15T01:35:30.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.707+0000 7f903affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:30.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.707+0000 7f903affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:30.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.870+0000 7f904986f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:30.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.870+0000 7f904986f640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:30.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.870+0000 7f904986f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:30.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.870+0000 7f904986f640 1 -- 172.21.15.204:0/1670477079 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) v1 -- 0x7f9044070b80 con 0x7f90440a2cd0 2024-09-15T01:35:30.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.872+0000 7f903affd640 1 -- 172.21.15.204:0/1670477079 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=-2 no weight-set for pool 'fooset' v125) v1 ==== 160+0+0 (secure 0 0 0) 0x7f903401c7e0 con 0x7f90440a2cd0 2024-09-15T01:35:30.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.872+0000 7f903affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:30.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.872+0000 7f903affd640 10 monclient: _finish_command 2 = mon:2 no weight-set for pool 'fooset' 2024-09-15T01:35:30.874 INFO:tasks.workunit.client.0.smithi204.stderr:Error ENOENT: no weight-set for pool 'fooset' 2024-09-15T01:35:30.878 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:35:30.878 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set create fooset flat 2024-09-15T01:35:30.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.950+0000 7f286b59d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:30.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.950+0000 7f286b59d640 10 monclient: build_initial_monmap 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.950+0000 7f286b59d640 10 monclient: monmap: 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:30.952300+0000 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:30.952300+0000 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:30.954 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:30.955 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:30.955 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:30.955 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:30.955 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:30.955 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:30.955 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:30.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.951+0000 7f286b59d640 1 Processor -- start 2024-09-15T01:35:30.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.951+0000 7f286b59d640 1 -- start start 2024-09-15T01:35:30.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.951+0000 7f286b59d640 10 monclient: init 2024-09-15T01:35:30.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:30.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 10 monclient: _add_conns ranks=[0,4,8,5,7,1,9,2,3,10,6] 2024-09-15T01:35:30.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f28640a2cd0 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:30.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f286409fa20 0x7f286409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 10 monclient(hunting): picked mon.noname-c con 0x7f286409fa20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:30.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2864137c80 0x7f286413a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 10 monclient(hunting): picked mon.noname-e con 0x7f2864137c80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:30.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28640648a0 con 0x7f28640a2cd0 2024-09-15T01:35:30.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:30.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2864133310 con 0x7f286409fa20 2024-09-15T01:35:30.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:30.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f28640a30a0 con 0x7f2864137c80 2024-09-15T01:35:30.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:30.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f286b59d640 10 monclient(hunting): authenticate will time out at 946.595095s 2024-09-15T01:35:30.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2868b38640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f286409fa20 0x7f286409fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:30.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869339640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f28640a2cd0 0x7f2864132c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.204:51902/0 (socket says 172.21.15.204:51902) 2024-09-15T01:35:30.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869339640 1 -- 172.21.15.204:0/2862050586 learned_addr learned my addr 172.21.15.204:0/2862050586 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:30.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 1 --2- 172.21.15.204:0/2862050586 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2864137c80 0x7f286413a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:30.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 10 monclient(hunting): get_auth_request con 0x7f2864137c80 auth_method 0 2024-09-15T01:35:30.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:30.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:30.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:30.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2868b38640 10 monclient(hunting): get_auth_request con 0x7f286409fa20 auth_method 0 2024-09-15T01:35:30.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2868b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:30.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2868b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:30.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2868b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:30.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:30.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:30.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:30.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2868b38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:30.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2868b38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:30.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2868b38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:30.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2863fff640 1 -- 172.21.15.204:0/2862050586 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424167131 0 0) 0x7f28640648a0 con 0x7f28640a2cd0 2024-09-15T01:35:30.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2863fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:30.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2863fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:30.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2863fff640 10 monclient(hunting): my global_id is 5796 2024-09-15T01:35:30.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2863fff640 1 -- 172.21.15.204:0/2862050586 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f284c0034e0 con 0x7f28640a2cd0 2024-09-15T01:35:30.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 10 monclient(hunting): handle_auth_done global_id 5624 payload 274 2024-09-15T01:35:30.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 1 -- 172.21.15.204:0/2862050586 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f286409fa20 msgr2=0x7f286409fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:30.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 1 --2- 172.21.15.204:0/2862050586 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f286409fa20 0x7f286409fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 1 -- 172.21.15.204:0/2862050586 >> v1:172.21.15.178:6789/0 conn(0x7f28640a2cd0 legacy=0x7f2864132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:30.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 10 monclient: _finish_hunting 0 2024-09-15T01:35:30.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 1 monclient: found mon.noname-e 2024-09-15T01:35:30.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:30.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:30.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2869b3a640 1 -- 172.21.15.204:0/2862050586 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28640a0330 con 0x7f2864137c80 2024-09-15T01:35:30.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.952+0000 7f2868b38640 1 --2- 172.21.15.204:0/2862050586 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f286409fa20 0x7f286409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:30.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2869b3a640 1 --2- 172.21.15.204:0/2862050586 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2864137c80 0x7f286413a060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f285c009880 tx=0x7f285c0090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=b4f5841cf195ce31 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:30.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2863fff640 1 -- 172.21.15.204:0/2862050586 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f285c02ac50 con 0x7f2864137c80 2024-09-15T01:35:30.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2863fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:30.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2863fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:30.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2863fff640 10 monclient: dump: 2024-09-15T01:35:30.968 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:30.969 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:30.969 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.969 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.969 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:30.969 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:30.969 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:30.972 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:30.972 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:30.972 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:30.972 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:30.973 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:30.973 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:30.973 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:30.973 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:30.973 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:30.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2863fff640 10 monclient: _finish_auth 0 2024-09-15T01:35:30.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2863fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:30.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2863fff640 1 -- 172.21.15.204:0/2862050586 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c02add0 con 0x7f2864137c80 2024-09-15T01:35:30.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f286b59d640 5 monclient: authenticate success, global_id 5624 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f286b59d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2863fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f286b59d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f286b59d640 4 set_mon_vals no callback set 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2863fff640 1 -- 172.21.15.204:0/2862050586 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f285c031820 con 0x7f2864137c80 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2863fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2863fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.953+0000 7f2863fff640 10 monclient: dump: 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:30.974 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.954+0000 7f286b59d640 10 monclient: shutdown 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.954+0000 7f286b59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.954+0000 7f286b59d640 1 -- 172.21.15.204:0/2862050586 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2864137c80 msgr2=0x7f286413a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:30.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.954+0000 7f286b59d640 1 --2- 172.21.15.204:0/2862050586 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2864137c80 0x7f286413a060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f285c009880 tx=0x7f285c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.954+0000 7f286b59d640 1 -- 172.21.15.204:0/2862050586 shutdown_connections 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.954+0000 7f286b59d640 1 --2- 172.21.15.204:0/2862050586 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2864137c80 0x7f286413a060 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.954+0000 7f286b59d640 1 --2- 172.21.15.204:0/2862050586 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f286409fa20 0x7f286409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.954+0000 7f286b59d640 1 -- 172.21.15.204:0/2862050586 >> 172.21.15.204:0/2862050586 conn(0x7f286406e860 msgr2=0x7f286406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.954+0000 7f286b59d640 1 -- 172.21.15.204:0/2862050586 shutdown_connections 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.954+0000 7f286b59d640 1 -- 172.21.15.204:0/2862050586 wait complete. 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.954+0000 7f286b59d640 10 monclient: build_initial_monmap 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.954+0000 7f286b59d640 10 monclient: monmap: 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:30.976 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 1 Processor -- start 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 1 -- start start 2024-09-15T01:35:30.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 10 monclient: init 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 10 monclient: _add_conns ranks=[6,3,2,8,4,7,5,0,1] 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f286409fa20 0x7f286409c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 10 monclient(hunting): picked mon.noname-d con 0x7f286409fa20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28640a2cd0 0x7f2864133f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 10 monclient(hunting): picked mon.noname-g con 0x7f28640a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2864137c80 0x7f28641344c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 10 monclient(hunting): picked mon.noname-b con 0x7f2864137c80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286409d3d0 con 0x7f2864137c80 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286409cb60 con 0x7f28640a2cd0 2024-09-15T01:35:30.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f28641360e0 con 0x7f286409fa20 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f286b59d640 10 monclient(hunting): authenticate will time out at 946.598697s 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2869339640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f286409fa20 msgr2=0x7f286409c3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2869b3a640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2864137c80 msgr2=0x7f28641344c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2869339640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f286409fa20 0x7f286409c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2869b3a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2864137c80 0x7f28641344c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2868b38640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28640a2cd0 0x7f2864133f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2868b38640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28640a2cd0 0x7f2864133f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:50740/0 (socket says 172.21.15.204:50740) 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2868b38640 1 -- 172.21.15.204:0/4045379023 learned_addr learned my addr 172.21.15.204:0/4045379023 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2868b38640 10 monclient(hunting): get_auth_request con 0x7f28640a2cd0 auth_method 0 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2868b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:30.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2868b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2868b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2868b38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2868b38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.955+0000 7f2868b38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f2868b38640 10 monclient(hunting): handle_auth_done global_id 14736 payload 824 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f2868b38640 1 -- 172.21.15.204:0/4045379023 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f286409fa20 msgr2=0x7f286409c3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f2868b38640 1 --2- 172.21.15.204:0/4045379023 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f286409fa20 0x7f286409c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f2868b38640 1 -- 172.21.15.204:0/4045379023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2864137c80 msgr2=0x7f28641344c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f2868b38640 1 --2- 172.21.15.204:0/4045379023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2864137c80 0x7f28641344c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f2868b38640 10 monclient: _finish_hunting 0 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f2868b38640 1 monclient: found mon.noname-g 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f2868b38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:30.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f2868b38640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f2868b38640 1 -- 172.21.15.204:0/4045379023 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2858009390 con 0x7f28640a2cd0 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f2868b38640 1 --2- 172.21.15.204:0/4045379023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28640a2cd0 0x7f2864133f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f285800eaa0 tx=0x7f28580090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28617fa640 1 -- 172.21.15.204:0/4045379023 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f285800ce90 con 0x7f28640a2cd0 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28617fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28617fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28617fa640 10 monclient: dump: 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:30.981 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:30.982 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:30.982 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:30.982 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:30.982 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:30.982 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:30.982 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:30.982 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:30.982 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:30.982 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:30.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28617fa640 10 monclient: _finish_auth 0 2024-09-15T01:35:30.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28617fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:30.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f286b59d640 5 monclient: authenticate success, global_id 14736 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28617fa640 1 -- 172.21.15.204:0/4045379023 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2858013470 con 0x7f28640a2cd0 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28617fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28617fa640 1 -- 172.21.15.204:0/4045379023 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2858013a60 con 0x7f28640a2cd0 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f286b59d640 10 monclient: _renew_subs 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f286b59d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f286b59d640 1 -- 172.21.15.204:0/4045379023 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285c009390 con 0x7f28640a2cd0 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28617fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28617fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28617fa640 10 monclient: dump: 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.983 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f28637fe640 4 set_mon_vals no callback set 2024-09-15T01:35:30.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f286b59d640 10 monclient: _renew_subs 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f286b59d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.956+0000 7f286b59d640 1 -- 172.21.15.204:0/4045379023 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2864135900 con 0x7f28640a2cd0 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.957+0000 7f28617fa640 1 -- 172.21.15.204:0/4045379023 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2858013bc0 con 0x7f28640a2cd0 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.957+0000 7f28617fa640 1 --2- 172.21.15.204:0/4045379023 >> v2:172.21.15.204:6812/28210 conn(0x7f28500397b0 0x7f285003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.957+0000 7f28617fa640 1 -- 172.21.15.204:0/4045379023 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f285804be00 con 0x7f28640a2cd0 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.957+0000 7f2869339640 1 --2- 172.21.15.204:0/4045379023 >> v2:172.21.15.204:6812/28210 conn(0x7f28500397b0 0x7f285003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.957+0000 7f286b59d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.957+0000 7f286b59d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.957+0000 7f286b59d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.957+0000 7f286b59d640 1 -- 172.21.15.204:0/4045379023 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2864137810 con 0x7f28640a2cd0 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.957+0000 7f2869339640 10 monclient: get_auth_request con 0x7f28500397b0 auth_method 0 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.958+0000 7f2869339640 1 --2- 172.21.15.204:0/4045379023 >> v2:172.21.15.204:6812/28210 conn(0x7f28500397b0 0x7f285003bc60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2854000f80 tx=0x7f28540017b0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:30.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.962+0000 7f28617fa640 1 -- 172.21.15.204:0/4045379023 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f285800ca10 con 0x7f28640a2cd0 2024-09-15T01:35:30.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.962+0000 7f28617fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:30.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:30.962+0000 7f28617fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:31.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.124+0000 7f286b59d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-09-15T01:35:31.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.125+0000 7f286b59d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-09-15T01:35:31.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.125+0000 7f286b59d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:31.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.125+0000 7f286b59d640 1 -- 172.21.15.204:0/4045379023 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"} v 0) v1 -- 0x7f2864070b80 con 0x7f28640a2cd0 2024-09-15T01:35:31.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.603+0000 7f28617fa640 1 -- 172.21.15.204:0/4045379023 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}]=0 v126) v1 ==== 109+0+0 (secure 0 0 0) 0x7f28580192c0 con 0x7f28640a2cd0 2024-09-15T01:35:31.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.603+0000 7f28617fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-09-15T01:35:31.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.603+0000 7f28617fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:31.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.706+0000 7f286b59d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-09-15T01:35:31.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.706+0000 7f286b59d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-09-15T01:35:31.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.706+0000 7f286b59d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:31.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.706+0000 7f286b59d640 1 -- 172.21.15.204:0/4045379023 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"} v 0) v1 -- 0x7f28640008d0 con 0x7f28640a2cd0 2024-09-15T01:35:31.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.707+0000 7f28617fa640 1 -- 172.21.15.204:0/4045379023 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}]=0 weight-set for pool 'fooset' already created v126) v1 ==== 153+0+0 (secure 0 0 0) 0x7f28580194a0 con 0x7f28640a2cd0 2024-09-15T01:35:31.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.707+0000 7f28617fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-09-15T01:35:31.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.707+0000 7f28617fa640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'fooset' already created 2024-09-15T01:35:31.710 INFO:tasks.workunit.client.0.smithi204.stderr:weight-set for pool 'fooset' already created 2024-09-15T01:35:31.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 1 -- 172.21.15.204:0/4045379023 >> v2:172.21.15.204:6812/28210 conn(0x7f28500397b0 msgr2=0x7f285003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:31.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 1 --2- 172.21.15.204:0/4045379023 >> v2:172.21.15.204:6812/28210 conn(0x7f28500397b0 0x7f285003bc60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2854000f80 tx=0x7f28540017b0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 10 monclient: shutdown 2024-09-15T01:35:31.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:31.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 1 -- 172.21.15.204:0/4045379023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28640a2cd0 msgr2=0x7f2864133f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:31.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 1 --2- 172.21.15.204:0/4045379023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28640a2cd0 0x7f2864133f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f285800eaa0 tx=0x7f28580090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 1 -- 172.21.15.204:0/4045379023 shutdown_connections 2024-09-15T01:35:31.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 1 --2- 172.21.15.204:0/4045379023 >> v2:172.21.15.204:6812/28210 conn(0x7f28500397b0 0x7f285003bc60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 1 --2- 172.21.15.204:0/4045379023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2864137c80 0x7f28641344c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 1 --2- 172.21.15.204:0/4045379023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28640a2cd0 0x7f2864133f80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 1 --2- 172.21.15.204:0/4045379023 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f286409fa20 0x7f286409c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 1 -- 172.21.15.204:0/4045379023 >> 172.21.15.204:0/4045379023 conn(0x7f286406e860 msgr2=0x7f28640a3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:31.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 1 -- 172.21.15.204:0/4045379023 shutdown_connections 2024-09-15T01:35:31.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.709+0000 7f286b59d640 1 -- 172.21.15.204:0/4045379023 wait complete. 2024-09-15T01:35:31.721 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set create barset positional 2024-09-15T01:35:31.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.794+0000 7ff40fe80640 10 monclient: get_monmap_and_config 2024-09-15T01:35:31.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.794+0000 7ff40fe80640 10 monclient: build_initial_monmap 2024-09-15T01:35:31.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.794+0000 7ff40fe80640 10 monclient: monmap: 2024-09-15T01:35:31.797 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:31.797 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:31.797 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:31.795912+0000 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:31.795912+0000 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:31.798 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:31.799 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:31.799 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:31.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 1 Processor -- start 2024-09-15T01:35:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 1 -- start start 2024-09-15T01:35:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 10 monclient: init 2024-09-15T01:35:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 10 monclient: _add_conns ranks=[2,7,1,6,5,9,3,10,4,8,0] 2024-09-15T01:35:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4080a2cd0 0x7ff408132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 10 monclient(hunting): picked mon.noname-b con 0x7ff4080a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff40809fa20 0x7ff40809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 10 monclient(hunting): picked mon.noname-i con 0x7ff40809fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:31.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff4080a08e0 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4080648a0 con 0x7ff4080a08e0 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080a30a0 con 0x7ff4080a2cd0 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080a0330 con 0x7ff40809fa20 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40fe80640 10 monclient(hunting): authenticate will time out at 947.438776s 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40d41b640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff40809fa20 msgr2=0x7ff40809fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40d41b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff40809fa20 0x7ff40809fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40e41d640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7ff4080a08e0 0x7ff4080a0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:40120/0 (socket says 172.21.15.204:40120) 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40e41d640 1 -- 172.21.15.204:0/3984053012 learned_addr learned my addr 172.21.15.204:0/3984053012 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40dc1c640 1 -- 172.21.15.204:0/3984053012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4080a2cd0 msgr2=0x7ff408132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:31.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.795+0000 7ff40dc1c640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4080a2cd0 0x7ff408132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449759078 0 0) 0x7ff4080648a0 con 0x7ff4080a08e0 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient(hunting): my global_id is 6202 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3f40034e0 con 0x7ff4080a08e0 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 849727032 0 0) 0x7ff3f40034e0 con 0x7ff4080a08e0 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff40809fa20 msgr2=0x7ff40809fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff40809fa20 0x7ff40809fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4080a2cd0 msgr2=0x7ff408132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4080a2cd0 0x7ff408132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient: _finish_hunting 0 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:31.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4080a1db0 con 0x7ff4080a08e0 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient: _finish_auth 0 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7ff400001c70 con 0x7ff4080a08e0 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40fe80640 5 monclient: authenticate success, global_id 6202 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40fe80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient: dump: 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:31.812 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:35:31.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 >> v1:172.21.15.204:6789/0 conn(0x7ff4080a08e0 legacy=0x7ff4080a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient: _add_conns ranks=[5,3,1,4,6,0,2,7,8] 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3f4004220 0x7ff3f4004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient(hunting): picked mon.h con 0x7ff3f4004220 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3f4004ec0 0x7ff3f40099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient(hunting): picked mon.g con 0x7ff3f4004ec0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f4009f00 0x7ff3f400c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient(hunting): picked mon.f con 0x7ff3f4009f00 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080a30a0 con 0x7ff3f4009f00 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080a0330 con 0x7ff3f4004ec0 2024-09-15T01:35:31.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f400cb30 con 0x7ff3f4004220 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40cc1a640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40fe80640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40dc1c640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3f4004220 0x7ff3f4004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40d41b640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3f4004ec0 0x7ff3f40099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f4009f00 0x7ff3f400c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40dc1c640 10 monclient(hunting): get_auth_request con 0x7ff3f4004220 auth_method 0 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40dc1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40dc1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40dc1c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 10 monclient(hunting): get_auth_request con 0x7ff3f4009f00 auth_method 0 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:31.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:35:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40d41b640 10 monclient(hunting): get_auth_request con 0x7ff3f4004ec0 auth_method 0 2024-09-15T01:35:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40d41b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40d41b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40d41b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:35:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:35:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40dc1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40dc1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40dc1c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:35:31.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40d41b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40d41b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40d41b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 10 monclient(hunting): handle_auth_done global_id 6202 payload 293 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 1 -- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3f4004220 msgr2=0x7ff3f4004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3f4004220 0x7ff3f4004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 1 -- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3f4004ec0 msgr2=0x7ff3f40099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3f4004ec0 0x7ff3f40099c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 10 monclient: _finish_hunting 0 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 1 monclient: found mon.f 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 1 -- 172.21.15.204:0/3984053012 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f40034e0 con 0x7ff3f4009f00 2024-09-15T01:35:31.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 10 monclient: _finish_auth 0 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40e41d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40dc1c640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3f4004220 0x7ff3f4004780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.796+0000 7ff40d41b640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3f4004ec0 0x7ff3f40099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40e41d640 1 -- 172.21.15.204:0/3984053012 reap_dead start 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40e41d640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f4009f00 0x7ff3f400c2e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff40000c730 tx=0x7ff40809db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86928c17e40edadf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff400007620 con 0x7ff3f4009f00 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40cc1a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40cc1a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40cc1a640 10 monclient: dump: 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:31.818 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:31.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40fe80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40002c040 con 0x7ff3f4009f00 2024-09-15T01:35:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40cc1a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40fe80640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40cc1a640 1 -- 172.21.15.204:0/3984053012 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4000077c0 con 0x7ff3f4009f00 2024-09-15T01:35:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40fe80640 4 set_mon_vals no callback set 2024-09-15T01:35:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40cc1a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40cc1a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.797+0000 7ff40cc1a640 10 monclient: dump: 2024-09-15T01:35:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:31.820 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:31.821 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.798+0000 7ff40fe80640 10 monclient: shutdown 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.798+0000 7ff40fe80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.798+0000 7ff40fe80640 1 -- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f4009f00 msgr2=0x7ff3f400c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.798+0000 7ff40fe80640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f4009f00 0x7ff3f400c2e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff40000c730 tx=0x7ff40809db10 comp rx=0 tx=0).stop 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.798+0000 7ff40fe80640 1 -- 172.21.15.204:0/3984053012 shutdown_connections 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.798+0000 7ff40fe80640 1 --2- 172.21.15.204:0/3984053012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f4009f00 0x7ff3f400c2e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.798+0000 7ff40fe80640 1 -- 172.21.15.204:0/3984053012 >> 172.21.15.204:0/3984053012 conn(0x7ff40806e860 msgr2=0x7ff40806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.798+0000 7ff40fe80640 1 -- 172.21.15.204:0/3984053012 shutdown_connections 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.798+0000 7ff40fe80640 1 -- 172.21.15.204:0/3984053012 wait complete. 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.798+0000 7ff40fe80640 10 monclient: build_initial_monmap 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.798+0000 7ff40fe80640 10 monclient: monmap: 2024-09-15T01:35:31.822 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:31.823 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:31.823 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:31.823 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:31.823 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:31.823 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:31.823 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:31.823 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:31.823 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:31.823 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:31.823 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:31.823 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:31.823 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:31.824 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:31.824 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:31.824 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:31.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 1 Processor -- start 2024-09-15T01:35:31.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 1 -- start start 2024-09-15T01:35:31.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 10 monclient: init 2024-09-15T01:35:31.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:31.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 10 monclient: _add_conns ranks=[7,2,8,1,6,0,4,3,5] 2024-09-15T01:35:31.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f4009f00 0x7ff40811d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:31.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 10 monclient(hunting): picked mon.noname-i con 0x7ff3f4009f00 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:31.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff40811d900 0x7ff40811dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:31.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 10 monclient(hunting): picked mon.noname-b con 0x7ff40811d900 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:31.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff40811e210 0x7ff408147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:31.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 10 monclient(hunting): picked mon.noname-e con 0x7ff40811e210 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:31.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:31.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff408120890 con 0x7ff40811d900 2024-09-15T01:35:31.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:31.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff40811e680 con 0x7ff3f4009f00 2024-09-15T01:35:31.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:31.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff408120ec0 con 0x7ff40811e210 2024-09-15T01:35:31.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:31.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40fe80640 10 monclient(hunting): authenticate will time out at 947.442620s 2024-09-15T01:35:31.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40dc1c640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f4009f00 msgr2=0x7ff40811d3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:31.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40dc1c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f4009f00 0x7ff40811d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:31.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40d41b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff40811d900 msgr2=0x7ff40811dcd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:31.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40d41b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff40811d900 0x7ff40811dcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:31.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40e41d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff40811e210 0x7ff408147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:31.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40e41d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff40811e210 0x7ff408147ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:50936/0 (socket says 172.21.15.204:50936) 2024-09-15T01:35:31.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40e41d640 1 -- 172.21.15.204:0/3714094463 learned_addr learned my addr 172.21.15.204:0/3714094463 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:31.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.799+0000 7ff40e41d640 10 monclient(hunting): get_auth_request con 0x7ff40811e210 auth_method 0 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 10 monclient(hunting): handle_auth_done global_id 5633 payload 824 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 1 -- 172.21.15.204:0/3714094463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f4009f00 msgr2=0x7ff40811d3c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 1 --2- 172.21.15.204:0/3714094463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f4009f00 0x7ff40811d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 1 -- 172.21.15.204:0/3714094463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff40811d900 msgr2=0x7ff40811dcd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 1 --2- 172.21.15.204:0/3714094463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff40811d900 0x7ff40811dcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 10 monclient: _finish_hunting 0 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 1 monclient: found mon.noname-e 2024-09-15T01:35:31.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 1 -- 172.21.15.204:0/3714094463 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40811f520 con 0x7ff40811e210 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40e41d640 1 --2- 172.21.15.204:0/3714094463 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff40811e210 0x7ff408147ac0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff40000c760 tx=0x7ff400034000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fdffb640 1 -- 172.21.15.204:0/3714094463 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff400004d30 con 0x7ff40811e210 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fdffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fdffb640 10 monclient: dump: 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:31.828 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fdffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40fe80640 5 monclient: authenticate success, global_id 5633 2024-09-15T01:35:31.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fdffb640 1 -- 172.21.15.204:0/3714094463 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40003d8b0 con 0x7ff40811e210 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fdffb640 1 -- 172.21.15.204:0/3714094463 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff40003dea0 con 0x7ff40811e210 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fdffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fdffb640 10 monclient: dump: 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:31.830 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff3fffff640 4 set_mon_vals no callback set 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40fe80640 10 monclient: _renew_subs 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40fe80640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40fe80640 1 -- 172.21.15.204:0/3714094463 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40809d3d0 con 0x7ff40811e210 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40fe80640 10 monclient: _renew_subs 2024-09-15T01:35:31.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40fe80640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.800+0000 7ff40fe80640 1 -- 172.21.15.204:0/3714094463 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40811ed60 con 0x7ff40811e210 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.801+0000 7ff40fe80640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.801+0000 7ff40fe80640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.801+0000 7ff40fe80640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.801+0000 7ff3fdffb640 1 -- 172.21.15.204:0/3714094463 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff40003da20 con 0x7ff40811e210 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.801+0000 7ff40fe80640 1 -- 172.21.15.204:0/3714094463 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff408070b80 con 0x7ff40811e210 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.802+0000 7ff3fdffb640 1 --2- 172.21.15.204:0/3714094463 >> v2:172.21.15.204:6812/28210 conn(0x7ff3ec0397b0 0x7ff3ec03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.802+0000 7ff3fdffb640 1 -- 172.21.15.204:0/3714094463 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6705+0+0 (secure 0 0 0) 0x7ff40006c220 con 0x7ff40811e210 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.802+0000 7ff40dc1c640 1 --2- 172.21.15.204:0/3714094463 >> v2:172.21.15.204:6812/28210 conn(0x7ff3ec0397b0 0x7ff3ec03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.802+0000 7ff40dc1c640 10 monclient: get_auth_request con 0x7ff3ec0397b0 auth_method 0 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.802+0000 7ff40dc1c640 1 --2- 172.21.15.204:0/3714094463 >> v2:172.21.15.204:6812/28210 conn(0x7ff3ec0397b0 0x7ff3ec03bc60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff404009460 tx=0x7ff404009060 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.806+0000 7ff3fdffb640 1 -- 172.21.15.204:0/3714094463 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff40002fd60 con 0x7ff40811e210 2024-09-15T01:35:31.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.806+0000 7ff3fdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:31.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.806+0000 7ff3fdffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:31.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.969+0000 7ff40fe80640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-09-15T01:35:31.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.969+0000 7ff40fe80640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-09-15T01:35:31.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.969+0000 7ff40fe80640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:31.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:31.969+0000 7ff40fe80640 1 -- 172.21.15.204:0/3714094463 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"} v 0) v1 -- 0x7ff4081204f0 con 0x7ff40811e210 2024-09-15T01:35:32.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.614+0000 7ff3fdffb640 1 -- 172.21.15.204:0/3714094463 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}]=0 v127) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff40002fd60 con 0x7ff40811e210 2024-09-15T01:35:32.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.614+0000 7ff3fdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-09-15T01:35:32.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.614+0000 7ff3fdffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:32.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.719+0000 7ff40fe80640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-09-15T01:35:32.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.719+0000 7ff40fe80640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-09-15T01:35:32.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.719+0000 7ff40fe80640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:32.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.719+0000 7ff40fe80640 1 -- 172.21.15.204:0/3714094463 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"} v 0) v1 -- 0x7ff4080008d0 con 0x7ff40811e210 2024-09-15T01:35:32.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.720+0000 7ff3fdffb640 1 -- 172.21.15.204:0/3714094463 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}]=0 weight-set for pool 'barset' already created v127) v1 ==== 159+0+0 (secure 0 0 0) 0x7ff400036030 con 0x7ff40811e210 2024-09-15T01:35:32.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.720+0000 7ff3fdffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-09-15T01:35:32.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.720+0000 7ff3fdffb640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'barset' already created 2024-09-15T01:35:32.723 INFO:tasks.workunit.client.0.smithi204.stderr:weight-set for pool 'barset' already created 2024-09-15T01:35:32.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 1 -- 172.21.15.204:0/3714094463 >> v2:172.21.15.204:6812/28210 conn(0x7ff3ec0397b0 msgr2=0x7ff3ec03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:32.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 1 --2- 172.21.15.204:0/3714094463 >> v2:172.21.15.204:6812/28210 conn(0x7ff3ec0397b0 0x7ff3ec03bc60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff404009460 tx=0x7ff404009060 comp rx=0 tx=0).stop 2024-09-15T01:35:32.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 10 monclient: shutdown 2024-09-15T01:35:32.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:32.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 1 -- 172.21.15.204:0/3714094463 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff40811e210 msgr2=0x7ff408147ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:32.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 1 --2- 172.21.15.204:0/3714094463 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff40811e210 0x7ff408147ac0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff40000c760 tx=0x7ff400034000 comp rx=0 tx=0).stop 2024-09-15T01:35:32.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 1 -- 172.21.15.204:0/3714094463 shutdown_connections 2024-09-15T01:35:32.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 1 --2- 172.21.15.204:0/3714094463 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff40811e210 0x7ff408147ac0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:32.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 1 --2- 172.21.15.204:0/3714094463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff40811d900 0x7ff40811dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:32.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 1 --2- 172.21.15.204:0/3714094463 >> v2:172.21.15.204:6812/28210 conn(0x7ff3ec0397b0 0x7ff3ec03bc60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:32.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 1 --2- 172.21.15.204:0/3714094463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f4009f00 0x7ff40811d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:32.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 1 -- 172.21.15.204:0/3714094463 >> 172.21.15.204:0/3714094463 conn(0x7ff40806e860 msgr2=0x7ff4080a6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:32.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 1 -- 172.21.15.204:0/3714094463 shutdown_connections 2024-09-15T01:35:32.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.722+0000 7ff40fe80640 1 -- 172.21.15.204:0/3714094463 wait complete. 2024-09-15T01:35:32.734 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set ls 2024-09-15T01:35:32.734 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep fooset 2024-09-15T01:35:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.805+0000 7f26c539d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.805+0000 7f26c539d640 10 monclient: build_initial_monmap 2024-09-15T01:35:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.805+0000 7f26c539d640 10 monclient: monmap: 2024-09-15T01:35:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:32.806677+0000 2024-09-15T01:35:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:32.806677+0000 2024-09-15T01:35:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:32.810 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:32.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.805+0000 7f26c539d640 1 Processor -- start 2024-09-15T01:35:32.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.805+0000 7f26c539d640 1 -- start start 2024-09-15T01:35:32.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.805+0000 7f26c539d640 10 monclient: init 2024-09-15T01:35:32.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:32.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 10 monclient: _add_conns ranks=[2,6,4,9,10,0,1,5,8,3,7] 2024-09-15T01:35:32.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c002a470 0x7f26c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:32.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 10 monclient(hunting): picked mon.noname-b con 0x7f26c002a470 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:32.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f26c00969e0 0x7f26c0096db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:32.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 10 monclient(hunting): picked mon.noname-d con 0x7f26c00969e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:32.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f26c00972f0 0x7f26c0132df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 10 monclient(hunting): picked mon.noname-c con 0x7f26c00972f0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c0097ec0 con 0x7f26c002a470 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26c002a840 con 0x7f26c00972f0 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f26c006d900 con 0x7f26c00969e0 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26c539d640 10 monclient(hunting): authenticate will time out at 948.449428s 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26be7fc640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f26c00969e0 msgr2=0x7f26c0096db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26beffd640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c002a470 msgr2=0x7f26c006d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26beffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c002a470 0x7f26c006d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:32.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26be7fc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f26c00969e0 0x7f26c0096db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26bf7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f26c00972f0 0x7f26c0132df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26bf7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f26c00972f0 0x7f26c0132df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:38434/0 (socket says 172.21.15.204:38434) 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26bf7fe640 1 -- 172.21.15.204:0/3834443954 learned_addr learned my addr 172.21.15.204:0/3834443954 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26bf7fe640 10 monclient(hunting): get_auth_request con 0x7f26c00972f0 auth_method 0 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.806+0000 7f26bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.807+0000 7f26bf7fe640 10 monclient(hunting): handle_auth_done global_id 5593 payload 274 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.807+0000 7f26bf7fe640 1 -- 172.21.15.204:0/3834443954 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f26c00969e0 msgr2=0x7f26c0096db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:32.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.807+0000 7f26bf7fe640 1 --2- 172.21.15.204:0/3834443954 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f26c00969e0 0x7f26c0096db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.807+0000 7f26bf7fe640 1 -- 172.21.15.204:0/3834443954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c002a470 msgr2=0x7f26c006d3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.807+0000 7f26bf7fe640 1 --2- 172.21.15.204:0/3834443954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c002a470 0x7f26c006d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.807+0000 7f26bf7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.807+0000 7f26bf7fe640 1 monclient: found mon.noname-c 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.807+0000 7f26bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.807+0000 7f26bf7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.807+0000 7f26bf7fe640 1 -- 172.21.15.204:0/3834443954 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c004b720 con 0x7f26c00972f0 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.807+0000 7f26bf7fe640 1 --2- 172.21.15.204:0/3834443954 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f26c00972f0 0x7f26c0132df0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009f50 tx=0x7f26b8009840 comp rx=0 tx=0).ready entity=mon.4 client_cookie=fb4a88c1478a6055 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26bdffb640 1 -- 172.21.15.204:0/3834443954 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f26b8007880 con 0x7f26c00972f0 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26bdffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26bdffb640 10 monclient: dump: 2024-09-15T01:35:32.822 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:32.823 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:32.824 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:32.824 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:32.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26bdffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:32.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:32.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26bdffb640 1 -- 172.21.15.204:0/3834443954 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b802d050 con 0x7f26c00972f0 2024-09-15T01:35:32.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 5 monclient: authenticate success, global_id 5593 2024-09-15T01:35:32.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:32.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26bdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:32.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:32.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26bdffb640 1 -- 172.21.15.204:0/3834443954 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f26b8030360 con 0x7f26c00972f0 2024-09-15T01:35:32.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:32.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 4 set_mon_vals no callback set 2024-09-15T01:35:32.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26bdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:32.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26bdffb640 10 monclient: dump: 2024-09-15T01:35:32.825 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:32.825 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:32.825 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:32.825 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:32.825 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:32.825 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 10 monclient: shutdown 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:32.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 1 -- 172.21.15.204:0/3834443954 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f26c00972f0 msgr2=0x7f26c0132df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 1 --2- 172.21.15.204:0/3834443954 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f26c00972f0 0x7f26c0132df0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009f50 tx=0x7f26b8009840 comp rx=0 tx=0).stop 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 1 -- 172.21.15.204:0/3834443954 shutdown_connections 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 1 --2- 172.21.15.204:0/3834443954 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f26c00972f0 0x7f26c0132df0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 1 --2- 172.21.15.204:0/3834443954 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f26c00969e0 0x7f26c0096db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 1 --2- 172.21.15.204:0/3834443954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c002a470 0x7f26c006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 1 -- 172.21.15.204:0/3834443954 >> 172.21.15.204:0/3834443954 conn(0x7f26c00672d0 msgr2=0x7f26c00676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 1 -- 172.21.15.204:0/3834443954 shutdown_connections 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.808+0000 7f26c539d640 1 -- 172.21.15.204:0/3834443954 wait complete. 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.809+0000 7f26c539d640 10 monclient: build_initial_monmap 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.809+0000 7f26c539d640 10 monclient: monmap: 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:32.827 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:32.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.809+0000 7f26c539d640 1 Processor -- start 2024-09-15T01:35:32.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.809+0000 7f26c539d640 1 -- start start 2024-09-15T01:35:32.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.809+0000 7f26c539d640 10 monclient: init 2024-09-15T01:35:32.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:32.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 10 monclient: _add_conns ranks=[1,8,6,0,7,4,5,3,2] 2024-09-15T01:35:32.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26c002a470 0x7f26c0141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:32.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 10 monclient(hunting): picked mon.noname-f con 0x7f26c002a470 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:32.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f26c00969e0 0x7f26c012efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:32.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 10 monclient(hunting): picked mon.noname-e con 0x7f26c00969e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:32.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f26c00972f0 0x7f26c0118c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:32.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 10 monclient(hunting): picked mon.noname-d con 0x7f26c00972f0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c00987b0 con 0x7f26c002a470 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f26c011a5a0 con 0x7f26c00972f0 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f26c011c0d0 con 0x7f26c00969e0 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 10 monclient(hunting): authenticate will time out at 948.453186s 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26c002a470 0x7f26c0141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26bf7fe640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f26c00972f0 msgr2=0x7f26c0118c50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26bf7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f26c00972f0 0x7f26c0118c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26c002a470 0x7f26c0141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47306/0 (socket says 172.21.15.204:47306) 2024-09-15T01:35:32.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 1 -- 172.21.15.204:0/1599690525 learned_addr learned my addr 172.21.15.204:0/1599690525 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:32.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 10 monclient(hunting): get_auth_request con 0x7f26c002a470 auth_method 0 2024-09-15T01:35:32.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:32.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:32.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:32.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:32.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:32.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:32.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26be7fc640 1 --2- 172.21.15.204:0/1599690525 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f26c00969e0 0x7f26c012efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:32.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26be7fc640 10 monclient(hunting): get_auth_request con 0x7f26c00969e0 auth_method 0 2024-09-15T01:35:32.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:32.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:32.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:32.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 10 monclient(hunting): handle_auth_done global_id 6211 payload 824 2024-09-15T01:35:32.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 1 -- 172.21.15.204:0/1599690525 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f26c00969e0 msgr2=0x7f26c012efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:32.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 1 --2- 172.21.15.204:0/1599690525 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f26c00969e0 0x7f26c012efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:32.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 1 -- 172.21.15.204:0/1599690525 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f26c00972f0 msgr2=0x7f26c0118c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:32.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 1 --2- 172.21.15.204:0/1599690525 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f26c00972f0 0x7f26c0118c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:32.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 10 monclient: _finish_hunting 0 2024-09-15T01:35:32.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 1 monclient: found mon.noname-f 2024-09-15T01:35:32.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:32.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:32.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 1 -- 172.21.15.204:0/1599690525 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b8009ae0 con 0x7f26c002a470 2024-09-15T01:35:32.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26beffd640 1 --2- 172.21.15.204:0/1599690525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26c002a470 0x7f26c0141aa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f26b000b5c0 tx=0x7f26b0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:32.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26a6ffd640 1 -- 172.21.15.204:0/1599690525 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f26b0009280 con 0x7f26c002a470 2024-09-15T01:35:32.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26a6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:32.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26a6ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:32.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26a6ffd640 10 monclient: dump: 2024-09-15T01:35:32.833 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:32.833 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:32.833 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:32.833 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:32.833 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:32.833 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26a6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:32.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 5 monclient: authenticate success, global_id 6211 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26a6ffd640 1 -- 172.21.15.204:0/1599690525 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b00050f0 con 0x7f26c002a470 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26a6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26a6ffd640 1 -- 172.21.15.204:0/1599690525 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f26b0005720 con 0x7f26c002a470 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26a6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26a6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26bd7fa640 4 set_mon_vals no callback set 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26a6ffd640 10 monclient: dump: 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:32.835 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:32.836 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:32.836 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:32.836 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:32.836 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:32.836 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:32.836 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:32.836 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:32.836 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:32.836 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:32.836 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:32.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 10 monclient: _renew_subs 2024-09-15T01:35:32.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.810+0000 7f26c539d640 1 -- 172.21.15.204:0/1599690525 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c011c3b0 con 0x7f26c002a470 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.811+0000 7f26c539d640 10 monclient: _renew_subs 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.811+0000 7f26c539d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.811+0000 7f26c539d640 1 -- 172.21.15.204:0/1599690525 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c0141fe0 con 0x7f26c002a470 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.811+0000 7f26a6ffd640 1 -- 172.21.15.204:0/1599690525 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f26b00052a0 con 0x7f26c002a470 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.811+0000 7f26a6ffd640 1 --2- 172.21.15.204:0/1599690525 >> v2:172.21.15.204:6812/28210 conn(0x7f26ac0397b0 0x7f26ac03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.811+0000 7f26c539d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.811+0000 7f26c539d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.811+0000 7f26c539d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.811+0000 7f26a6ffd640 1 -- 172.21.15.204:0/1599690525 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 7317+0+0 (secure 0 0 0) 0x7f26b004b920 con 0x7f26c002a470 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.811+0000 7f26c539d640 1 -- 172.21.15.204:0/1599690525 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c0070d80 con 0x7f26c002a470 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.812+0000 7f26be7fc640 1 --2- 172.21.15.204:0/1599690525 >> v2:172.21.15.204:6812/28210 conn(0x7f26ac0397b0 0x7f26ac03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:32.837 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.812+0000 7f26be7fc640 10 monclient: get_auth_request con 0x7f26ac0397b0 auth_method 0 2024-09-15T01:35:32.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.812+0000 7f26be7fc640 1 --2- 172.21.15.204:0/1599690525 >> v2:172.21.15.204:6812/28210 conn(0x7f26ac0397b0 0x7f26ac03bc60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26b40038f0 tx=0x7f26b4006ef0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:32.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.816+0000 7f26a6ffd640 1 -- 172.21.15.204:0/1599690525 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f26b001a460 con 0x7f26c002a470 2024-09-15T01:35:32.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.816+0000 7f26a6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:32.838 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.816+0000 7f26a6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:32.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.979+0000 7f26c539d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:32.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.979+0000 7f26c539d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:32.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.979+0000 7f26c539d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:32.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.979+0000 7f26c539d640 1 -- 172.21.15.204:0/1599690525 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f26c0069070 con 0x7f26c002a470 2024-09-15T01:35:32.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.979+0000 7f26a6ffd640 1 -- 172.21.15.204:0/1599690525 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v127) v1 ==== 71+0+14 (secure 0 0 0) 0x7f26b000f360 con 0x7f26c002a470 2024-09-15T01:35:32.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.979+0000 7f26a6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:32.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:32.979+0000 7f26a6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:33.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.081+0000 7f26c539d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:33.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.081+0000 7f26c539d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:33.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.081+0000 7f26c539d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:33.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.081+0000 7f26c539d640 1 -- 172.21.15.204:0/1599690525 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f26c011b170 con 0x7f26c002a470 2024-09-15T01:35:33.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.081+0000 7f26a6ffd640 1 -- 172.21.15.204:0/1599690525 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v127) v1 ==== 71+0+14 (secure 0 0 0) 0x7f26b0019bb0 con 0x7f26c002a470 2024-09-15T01:35:33.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.081+0000 7f26a6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:33.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.081+0000 7f26a6ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:33.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 1 -- 172.21.15.204:0/1599690525 >> v2:172.21.15.204:6812/28210 conn(0x7f26ac0397b0 msgr2=0x7f26ac03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 1 --2- 172.21.15.204:0/1599690525 >> v2:172.21.15.204:6812/28210 conn(0x7f26ac0397b0 0x7f26ac03bc60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26b40038f0 tx=0x7f26b4006ef0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 10 monclient: shutdown 2024-09-15T01:35:33.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:33.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 1 -- 172.21.15.204:0/1599690525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26c002a470 msgr2=0x7f26c0141aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 1 --2- 172.21.15.204:0/1599690525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26c002a470 0x7f26c0141aa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f26b000b5c0 tx=0x7f26b0009120 comp rx=0 tx=0).stop 2024-09-15T01:35:33.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 1 -- 172.21.15.204:0/1599690525 shutdown_connections 2024-09-15T01:35:33.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 1 --2- 172.21.15.204:0/1599690525 >> v2:172.21.15.204:6812/28210 conn(0x7f26ac0397b0 0x7f26ac03bc60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 1 --2- 172.21.15.204:0/1599690525 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f26c00972f0 0x7f26c0118c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 1 --2- 172.21.15.204:0/1599690525 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f26c00969e0 0x7f26c012efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 1 --2- 172.21.15.204:0/1599690525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26c002a470 0x7f26c0141aa0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 1 -- 172.21.15.204:0/1599690525 >> 172.21.15.204:0/1599690525 conn(0x7f26c00672d0 msgr2=0x7f26c009d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:33.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 1 -- 172.21.15.204:0/1599690525 shutdown_connections 2024-09-15T01:35:33.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.083+0000 7f26c539d640 1 -- 172.21.15.204:0/1599690525 wait complete. 2024-09-15T01:35:33.094 INFO:tasks.workunit.client.0.smithi204.stdout:fooset 2024-09-15T01:35:33.094 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set ls 2024-09-15T01:35:33.094 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep barset 2024-09-15T01:35:33.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.165+0000 7f791bc2b640 10 monclient: get_monmap_and_config 2024-09-15T01:35:33.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.165+0000 7f791bc2b640 10 monclient: build_initial_monmap 2024-09-15T01:35:33.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.165+0000 7f791bc2b640 10 monclient: monmap: 2024-09-15T01:35:33.169 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:33.169 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.169 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:33.167424+0000 2024-09-15T01:35:33.169 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:33.167424+0000 2024-09-15T01:35:33.169 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:33.169 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.169 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:33.169 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:33.170 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:33.170 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:33.170 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:33.170 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:33.170 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:33.170 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:33.170 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:33.170 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:33.170 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:33.170 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.166+0000 7f791bc2b640 1 Processor -- start 2024-09-15T01:35:33.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.166+0000 7f791bc2b640 1 -- start start 2024-09-15T01:35:33.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.166+0000 7f791bc2b640 10 monclient: init 2024-09-15T01:35:33.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:33.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 10 monclient: _add_conns ranks=[6,5,1,4,3,9,10,2,8,0,7] 2024-09-15T01:35:33.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79140a2e30 0x7f7914133280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 10 monclient(hunting): picked mon.noname-d con 0x7f79140a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:33.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f791409fd80 0x7f79140a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 10 monclient(hunting): picked mon.noname-h con 0x7f791409fd80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:33.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f79140a09c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:33.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791404b720 con 0x7f79140a09c0 2024-09-15T01:35:33.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f791409e290 con 0x7f791409fd80 2024-09-15T01:35:33.179 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f79140a2450 con 0x7f79140a2e30 2024-09-15T01:35:33.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:33.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791bc2b640 10 monclient(hunting): authenticate will time out at 948.810195s 2024-09-15T01:35:33.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79199c7640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79140a2e30 msgr2=0x7f7914133280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:33.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f791409fd80 0x7f79140a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791a1c8640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f79140a09c0 0x7f79140a0df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:40126/0 (socket says 172.21.15.204:40126) 2024-09-15T01:35:33.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f791a1c8640 1 -- 172.21.15.204:0/4020989547 learned_addr learned my addr 172.21.15.204:0/4020989547 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:33.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79199c7640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79140a2e30 0x7f7914133280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:33.180 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 10 monclient(hunting): get_auth_request con 0x7f791409fd80 auth_method 0 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 10 monclient(hunting): handle_auth_done global_id 5684 payload 274 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 1 -- 172.21.15.204:0/4020989547 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79140a2e30 msgr2=0x7f7914133280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 1 --2- 172.21.15.204:0/4020989547 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79140a2e30 0x7f7914133280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79189c5640 1 -- 172.21.15.204:0/4020989547 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076280249 0 0) 0x7f791404b720 con 0x7f79140a09c0 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 1 -- 172.21.15.204:0/4020989547 >> v1:172.21.15.204:6789/0 conn(0x7f79140a09c0 legacy=0x7f79140a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 10 monclient: _finish_hunting 0 2024-09-15T01:35:33.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 1 monclient: found mon.noname-h 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79191c6640 1 -- 172.21.15.204:0/4020989547 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79140a3200 con 0x7f791409fd80 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.167+0000 7f79189c5640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79191c6640 1 --2- 172.21.15.204:0/4020989547 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f791409fd80 0x7f79140a0150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f79100098b0 tx=0x7f79100090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a9db9797f80f555c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79189c5640 1 -- 172.21.15.204:0/4020989547 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f791002ac50 con 0x7f791409fd80 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79189c5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79189c5640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79189c5640 10 monclient: dump: 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.182 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79189c5640 10 monclient: _finish_auth 0 2024-09-15T01:35:33.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79189c5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f791bc2b640 5 monclient: authenticate success, global_id 5684 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79189c5640 1 -- 172.21.15.204:0/4020989547 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791002add0 con 0x7f791409fd80 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f791bc2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79189c5640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f791bc2b640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f791bc2b640 4 set_mon_vals no callback set 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79189c5640 1 -- 172.21.15.204:0/4020989547 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7910031ca0 con 0x7f791409fd80 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79189c5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79189c5640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f79189c5640 10 monclient: dump: 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.184 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.185 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.185 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:33.185 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.185 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:33.185 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:33.185 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:33.185 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:33.185 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:33.185 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:33.185 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:33.185 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:33.185 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:33.186 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f791bc2b640 10 monclient: shutdown 2024-09-15T01:35:33.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f791bc2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:33.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f791bc2b640 1 -- 172.21.15.204:0/4020989547 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f791409fd80 msgr2=0x7f79140a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.168+0000 7f791bc2b640 1 --2- 172.21.15.204:0/4020989547 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f791409fd80 0x7f79140a0150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f79100098b0 tx=0x7f79100090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.169+0000 7f791bc2b640 1 -- 172.21.15.204:0/4020989547 shutdown_connections 2024-09-15T01:35:33.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.169+0000 7f791bc2b640 1 --2- 172.21.15.204:0/4020989547 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f791409fd80 0x7f79140a0150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.169+0000 7f791bc2b640 1 --2- 172.21.15.204:0/4020989547 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79140a2e30 0x7f7914133280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.169+0000 7f791bc2b640 1 -- 172.21.15.204:0/4020989547 >> 172.21.15.204:0/4020989547 conn(0x7f791406e860 msgr2=0x7f791406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:33.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.169+0000 7f791bc2b640 1 -- 172.21.15.204:0/4020989547 shutdown_connections 2024-09-15T01:35:33.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.169+0000 7f791bc2b640 1 -- 172.21.15.204:0/4020989547 wait complete. 2024-09-15T01:35:33.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.169+0000 7f791bc2b640 10 monclient: build_initial_monmap 2024-09-15T01:35:33.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.169+0000 7f791bc2b640 10 monclient: monmap: 2024-09-15T01:35:33.187 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:33.187 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:33.187 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:33.187 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:33.187 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:33.187 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:33.187 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:33.187 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:33.188 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:33.188 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:33.188 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:33.188 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:33.188 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:33.188 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:33.188 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:33.188 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.169+0000 7f791bc2b640 1 Processor -- start 2024-09-15T01:35:33.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.169+0000 7f791bc2b640 1 -- start start 2024-09-15T01:35:33.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.169+0000 7f791bc2b640 10 monclient: init 2024-09-15T01:35:33.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 10 monclient: _add_conns ranks=[0,5,8,7,6,4,2,3,1] 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f791409fd80 0x7f791412f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 10 monclient(hunting): picked mon.noname-a con 0x7f791409fd80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a09c0 0x7f791411c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 10 monclient(hunting): picked mon.noname-h con 0x7f79140a09c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f79140a2e30 0x7f791411cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 10 monclient(hunting): picked mon.noname-e con 0x7f79140a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791409d9b0 con 0x7f791409fd80 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f791409d4c0 con 0x7f79140a09c0 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f79141341c0 con 0x7f79140a2e30 2024-09-15T01:35:33.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:33.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791bc2b640 10 monclient(hunting): authenticate will time out at 948.813387s 2024-09-15T01:35:33.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a09c0 0x7f791411c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a09c0 0x7f791411c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:45934/0 (socket says 172.21.15.204:45934) 2024-09-15T01:35:33.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 1 -- 172.21.15.204:0/4111482970 learned_addr learned my addr 172.21.15.204:0/4111482970 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:33.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 10 monclient(hunting): get_auth_request con 0x7f79140a09c0 auth_method 0 2024-09-15T01:35:33.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:33.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:33.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:33.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:33.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:33.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:33.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 10 monclient(hunting): handle_auth_done global_id 5693 payload 824 2024-09-15T01:35:33.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 1 -- 172.21.15.204:0/4111482970 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f79140a2e30 msgr2=0x7f791411cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79199c7640 1 --2- 172.21.15.204:0/4111482970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f791409fd80 0x7f791412f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f791a1c8640 1 --2- 172.21.15.204:0/4111482970 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f79140a2e30 0x7f791411cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 1 --2- 172.21.15.204:0/4111482970 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f79140a2e30 0x7f791411cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 1 -- 172.21.15.204:0/4111482970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f791409fd80 msgr2=0x7f791412f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 1 --2- 172.21.15.204:0/4111482970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f791409fd80 0x7f791412f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 10 monclient: _finish_hunting 0 2024-09-15T01:35:33.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 1 monclient: found mon.noname-h 2024-09-15T01:35:33.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:33.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:33.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 1 -- 172.21.15.204:0/4111482970 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7910009390 con 0x7f79140a09c0 2024-09-15T01:35:33.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.170+0000 7f79191c6640 1 --2- 172.21.15.204:0/4111482970 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a09c0 0x7f791411c630 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7910009910 tx=0x7f7910032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:33.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f7909ffb640 1 -- 172.21.15.204:0/4111482970 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7910033e30 con 0x7f79140a09c0 2024-09-15T01:35:33.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f7909ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:33.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f7909ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:33.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f7909ffb640 10 monclient: dump: 2024-09-15T01:35:33.192 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:33.192 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.192 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.192 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.193 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:33.193 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.193 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:33.193 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:33.193 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:33.193 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:33.193 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:33.193 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:33.193 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:33.193 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f7909ffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f7909ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f7909ffb640 1 -- 172.21.15.204:0/4111482970 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791003b3f0 con 0x7f79140a09c0 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f791bc2b640 5 monclient: authenticate success, global_id 5693 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f7909ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f7909ffb640 1 -- 172.21.15.204:0/4111482970 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f791003b9e0 con 0x7f79140a09c0 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f791bc2b640 10 monclient: _renew_subs 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f791bc2b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f791bc2b640 1 -- 172.21.15.204:0/4111482970 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791411fe20 con 0x7f79140a09c0 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f7909ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:33.194 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f7909ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:33.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f790bfff640 4 set_mon_vals no callback set 2024-09-15T01:35:33.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f7909ffb640 10 monclient: dump: 2024-09-15T01:35:33.195 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:33.195 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.195 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.195 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.195 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:33.195 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.195 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:33.195 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:33.195 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:33.195 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f791bc2b640 10 monclient: _renew_subs 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f791bc2b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.171+0000 7f791bc2b640 1 -- 172.21.15.204:0/4111482970 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791411e900 con 0x7f79140a09c0 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.172+0000 7f7909ffb640 1 -- 172.21.15.204:0/4111482970 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7910039040 con 0x7f79140a09c0 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.172+0000 7f791bc2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.172+0000 7f791bc2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:33.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.172+0000 7f791bc2b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.172+0000 7f791bc2b640 1 -- 172.21.15.204:0/4111482970 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7914070d80 con 0x7f79140a09c0 2024-09-15T01:35:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.172+0000 7f7909ffb640 1 --2- 172.21.15.204:0/4111482970 >> v2:172.21.15.204:6812/28210 conn(0x7f78f80397b0 0x7f78f803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.172+0000 7f7909ffb640 1 -- 172.21.15.204:0/4111482970 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 7317+0+0 (secure 0 0 0) 0x7f791006b6d0 con 0x7f79140a09c0 2024-09-15T01:35:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.172+0000 7f79199c7640 1 --2- 172.21.15.204:0/4111482970 >> v2:172.21.15.204:6812/28210 conn(0x7f78f80397b0 0x7f78f803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.172+0000 7f79199c7640 10 monclient: get_auth_request con 0x7f78f80397b0 auth_method 0 2024-09-15T01:35:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.172+0000 7f79199c7640 1 --2- 172.21.15.204:0/4111482970 >> v2:172.21.15.204:6812/28210 conn(0x7f78f80397b0 0x7f78f803bc60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7904006fd0 tx=0x7f7904008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.176+0000 7f7909ffb640 1 -- 172.21.15.204:0/4111482970 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7910044b30 con 0x7f79140a09c0 2024-09-15T01:35:33.197 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.176+0000 7f7909ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:33.198 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.176+0000 7f7909ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:33.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.337+0000 7f791bc2b640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:33.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.337+0000 7f791bc2b640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:33.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.337+0000 7f791bc2b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:33.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.337+0000 7f791bc2b640 1 -- 172.21.15.204:0/4111482970 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f791409dfb0 con 0x7f79140a09c0 2024-09-15T01:35:33.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.337+0000 7f7909ffb640 1 -- 172.21.15.204:0/4111482970 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v127) v1 ==== 71+0+14 (secure 0 0 0) 0x7f7910044300 con 0x7f79140a09c0 2024-09-15T01:35:33.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.337+0000 7f7909ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:33.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.337+0000 7f7909ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:33.439 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.437+0000 7f791bc2b640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:33.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.437+0000 7f791bc2b640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:33.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.437+0000 7f791bc2b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:33.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.437+0000 7f791bc2b640 1 -- 172.21.15.204:0/4111482970 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f79140a3920 con 0x7f79140a09c0 2024-09-15T01:35:33.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.437+0000 7f7909ffb640 1 -- 172.21.15.204:0/4111482970 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v127) v1 ==== 71+0+14 (secure 0 0 0) 0x7f7910044950 con 0x7f79140a09c0 2024-09-15T01:35:33.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.437+0000 7f7909ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:33.440 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.437+0000 7f7909ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:33.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 1 -- 172.21.15.204:0/4111482970 >> v2:172.21.15.204:6812/28210 conn(0x7f78f80397b0 msgr2=0x7f78f803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 1 --2- 172.21.15.204:0/4111482970 >> v2:172.21.15.204:6812/28210 conn(0x7f78f80397b0 0x7f78f803bc60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7904006fd0 tx=0x7f7904008040 comp rx=0 tx=0).stop 2024-09-15T01:35:33.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 10 monclient: shutdown 2024-09-15T01:35:33.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:33.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 1 -- 172.21.15.204:0/4111482970 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a09c0 msgr2=0x7f791411c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 1 --2- 172.21.15.204:0/4111482970 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a09c0 0x7f791411c630 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7910009910 tx=0x7f7910032000 comp rx=0 tx=0).stop 2024-09-15T01:35:33.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 1 -- 172.21.15.204:0/4111482970 shutdown_connections 2024-09-15T01:35:33.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 1 --2- 172.21.15.204:0/4111482970 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f79140a2e30 0x7f791411cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 1 --2- 172.21.15.204:0/4111482970 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79140a09c0 0x7f791411c630 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 1 --2- 172.21.15.204:0/4111482970 >> v2:172.21.15.204:6812/28210 conn(0x7f78f80397b0 0x7f78f803bc60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 1 --2- 172.21.15.204:0/4111482970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f791409fd80 0x7f791412f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 1 -- 172.21.15.204:0/4111482970 >> 172.21.15.204:0/4111482970 conn(0x7f791406e860 msgr2=0x7f79140a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:33.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 1 -- 172.21.15.204:0/4111482970 shutdown_connections 2024-09-15T01:35:33.443 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.439+0000 7f791bc2b640 1 -- 172.21.15.204:0/4111482970 wait complete. 2024-09-15T01:35:33.451 INFO:tasks.workunit.client.0.smithi204.stdout:barset 2024-09-15T01:35:33.451 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set dump 2024-09-15T01:35:33.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.522+0000 7fd02a99d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:33.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.522+0000 7fd02a99d640 10 monclient: build_initial_monmap 2024-09-15T01:35:33.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.522+0000 7fd02a99d640 10 monclient: monmap: 2024-09-15T01:35:33.525 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:33.525 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.525 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:33.524021+0000 2024-09-15T01:35:33.525 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:33.524021+0000 2024-09-15T01:35:33.525 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:33.525 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.525 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:33.525 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:33.525 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:33.526 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:33.526 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:33.526 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:33.526 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:33.526 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:33.526 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:33.526 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:33.526 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:33.526 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 1 Processor -- start 2024-09-15T01:35:33.534 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 1 -- start start 2024-09-15T01:35:33.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 10 monclient: init 2024-09-15T01:35:33.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:33.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 10 monclient: _add_conns ranks=[4,7,5,6,3,2,10,8,1,0,9] 2024-09-15T01:35:33.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd0240a2e30 0x7fd024133280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 10 monclient(hunting): picked mon.noname-c con 0x7fd0240a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:33.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd02409fd80 0x7fd0240a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 10 monclient(hunting): picked mon.noname-i con 0x7fd02409fd80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:33.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd0240a09c0 0x7fd0240a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 10 monclient(hunting): picked mon.noname-h con 0x7fd0240a09c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:33.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.535 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd02409e290 con 0x7fd0240a2e30 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240a2450 con 0x7fd0240a09c0 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240a3200 con 0x7fd02409fd80 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd02a99d640 10 monclient(hunting): authenticate will time out at 949.166834s 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd0237fe640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd02409fd80 msgr2=0x7fd0240a0150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd0237fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd02409fd80 0x7fd0240a0150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.523+0000 7fd028f3a640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd0240a09c0 0x7fd0240a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd0240a09c0 0x7fd0240a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:45942/0 (socket says 172.21.15.204:45942) 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 1 -- 172.21.15.204:0/484878411 learned_addr learned my addr 172.21.15.204:0/484878411 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 10 monclient(hunting): get_auth_request con 0x7fd0240a09c0 auth_method 0 2024-09-15T01:35:33.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:33.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:33.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:33.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:33.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:33.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd023fff640 1 --2- 172.21.15.204:0/484878411 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd0240a2e30 0x7fd024133280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd023fff640 10 monclient(hunting): get_auth_request con 0x7fd0240a2e30 auth_method 0 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd023fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd023fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd023fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 10 monclient(hunting): handle_auth_done global_id 5702 payload 274 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 1 -- 172.21.15.204:0/484878411 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd02409fd80 msgr2=0x7fd0240a0150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 1 --2- 172.21.15.204:0/484878411 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd02409fd80 0x7fd0240a0150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 1 -- 172.21.15.204:0/484878411 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd0240a2e30 msgr2=0x7fd024133280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 1 --2- 172.21.15.204:0/484878411 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd0240a2e30 0x7fd024133280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 10 monclient: _finish_hunting 0 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 1 monclient: found mon.noname-h 2024-09-15T01:35:33.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 1 -- 172.21.15.204:0/484878411 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02404b720 con 0x7fd0240a09c0 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.524+0000 7fd028f3a640 1 --2- 172.21.15.204:0/484878411 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd0240a09c0 0x7fd0240a0dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd01c009f50 tx=0x7fd01c009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=39fbdf8ea6358b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd022ffd640 1 -- 172.21.15.204:0/484878411 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd01c007880 con 0x7fd0240a09c0 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd022ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd022ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd022ffd640 10 monclient: dump: 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:33.539 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd022ffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd022ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:33.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 5 monclient: authenticate success, global_id 5702 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd022ffd640 1 -- 172.21.15.204:0/484878411 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01c02d050 con 0x7fd0240a09c0 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd022ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 4 set_mon_vals no callback set 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd022ffd640 1 -- 172.21.15.204:0/484878411 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd01c030360 con 0x7fd0240a09c0 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd022ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd022ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd022ffd640 10 monclient: dump: 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.541 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 10 monclient: shutdown 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:33.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 1 -- 172.21.15.204:0/484878411 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd0240a09c0 msgr2=0x7fd0240a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 1 --2- 172.21.15.204:0/484878411 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd0240a09c0 0x7fd0240a0dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd01c009f50 tx=0x7fd01c009840 comp rx=0 tx=0).stop 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 1 -- 172.21.15.204:0/484878411 shutdown_connections 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 1 --2- 172.21.15.204:0/484878411 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd0240a09c0 0x7fd0240a0dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 1 --2- 172.21.15.204:0/484878411 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd02409fd80 0x7fd0240a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 1 --2- 172.21.15.204:0/484878411 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd0240a2e30 0x7fd024133280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 1 -- 172.21.15.204:0/484878411 >> 172.21.15.204:0/484878411 conn(0x7fd02406e860 msgr2=0x7fd02406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 1 -- 172.21.15.204:0/484878411 shutdown_connections 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.525+0000 7fd02a99d640 1 -- 172.21.15.204:0/484878411 wait complete. 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.526+0000 7fd02a99d640 10 monclient: build_initial_monmap 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.526+0000 7fd02a99d640 10 monclient: monmap: 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:33.543 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.544 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.526+0000 7fd02a99d640 1 Processor -- start 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.526+0000 7fd02a99d640 1 -- start start 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.526+0000 7fd02a99d640 10 monclient: init 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.526+0000 7fd02a99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.526+0000 7fd02a99d640 10 monclient: _add_conns ranks=[0,6,2,3,5,1,8,7,4] 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd02409fd80 0x7fd02412f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 10 monclient(hunting): picked mon.noname-a con 0x7fd02409fd80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd0240a09c0 0x7fd02411c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 10 monclient(hunting): picked mon.noname-d con 0x7fd0240a09c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0240a2e30 0x7fd02411cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 10 monclient(hunting): picked mon.noname-b con 0x7fd0240a2e30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02411f930 con 0x7fd02409fd80 2024-09-15T01:35:33.545 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02409d9b0 con 0x7fd0240a2e30 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd02409d4c0 con 0x7fd0240a09c0 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 10 monclient(hunting): authenticate will time out at 949.170076s 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd0237fe640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd0240a09c0 msgr2=0x7fd02411c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd028f3a640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0240a2e30 msgr2=0x7fd02411cb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd028f3a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0240a2e30 0x7fd02411cb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd0237fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd0240a09c0 0x7fd02411c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd02409fd80 0x7fd02412f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd02409fd80 0x7fd02412f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:40194/0 (socket says 172.21.15.204:40194) 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 1 -- 172.21.15.204:0/313965461 learned_addr learned my addr 172.21.15.204:0/313965461 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:33.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 10 monclient(hunting): get_auth_request con 0x7fd02409fd80 auth_method 0 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 10 monclient(hunting): handle_auth_done global_id 5805 payload 824 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 1 -- 172.21.15.204:0/313965461 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd0240a09c0 msgr2=0x7fd02411c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 1 --2- 172.21.15.204:0/313965461 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd0240a09c0 0x7fd02411c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 1 -- 172.21.15.204:0/313965461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0240a2e30 msgr2=0x7fd02411cb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 1 --2- 172.21.15.204:0/313965461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0240a2e30 0x7fd02411cb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:33.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 1 monclient: found mon.noname-a 2024-09-15T01:35:33.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:33.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:35:33.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 1 -- 172.21.15.204:0/313965461 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01c009ae0 con 0x7fd02409fd80 2024-09-15T01:35:33.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd023fff640 1 --2- 172.21.15.204:0/313965461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd02409fd80 0x7fd02412f7e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd01400d890 tx=0x7fd014008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:33.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd00bfff640 1 -- 172.21.15.204:0/313965461 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd01400de20 con 0x7fd02409fd80 2024-09-15T01:35:33.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd00bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:33.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd00bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:35:33.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd00bfff640 10 monclient: dump: 2024-09-15T01:35:33.548 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:33.548 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.548 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:33.549 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd00bfff640 10 monclient: _finish_auth 0 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd00bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd00bfff640 1 -- 172.21.15.204:0/313965461 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd014009d30 con 0x7fd02409fd80 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 5 monclient: authenticate success, global_id 5805 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.527+0000 7fd02a99d640 10 monclient: _renew_subs 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd02a99d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd02a99d640 1 -- 172.21.15.204:0/313965461 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02411e630 con 0x7fd02409fd80 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd00bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd00bfff640 1 -- 172.21.15.204:0/313965461 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd014009ea0 con 0x7fd02409fd80 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd00bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd00bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd00bfff640 10 monclient: dump: 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:33.550 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:33.551 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd0227fc640 4 set_mon_vals no callback set 2024-09-15T01:35:33.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd02a99d640 10 monclient: _renew_subs 2024-09-15T01:35:33.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd02a99d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:33.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd02a99d640 1 -- 172.21.15.204:0/313965461 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd024142c10 con 0x7fd02409fd80 2024-09-15T01:35:33.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd02a99d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:33.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd02a99d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:33.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd02a99d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:33.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd00bfff640 1 -- 172.21.15.204:0/313965461 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd0140098b0 con 0x7fd02409fd80 2024-09-15T01:35:33.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.528+0000 7fd02a99d640 1 -- 172.21.15.204:0/313965461 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd024070d80 con 0x7fd02409fd80 2024-09-15T01:35:33.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.529+0000 7fd00bfff640 1 --2- 172.21.15.204:0/313965461 >> v2:172.21.15.204:6812/28210 conn(0x7fd0100397b0 0x7fd01003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.529+0000 7fd00bfff640 1 -- 172.21.15.204:0/313965461 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 7317+0+0 (secure 0 0 0) 0x7fd01404c0c0 con 0x7fd02409fd80 2024-09-15T01:35:33.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.529+0000 7fd0237fe640 1 --2- 172.21.15.204:0/313965461 >> v2:172.21.15.204:6812/28210 conn(0x7fd0100397b0 0x7fd01003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.529+0000 7fd0237fe640 10 monclient: get_auth_request con 0x7fd0100397b0 auth_method 0 2024-09-15T01:35:33.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.529+0000 7fd0237fe640 1 --2- 172.21.15.204:0/313965461 >> v2:172.21.15.204:6812/28210 conn(0x7fd0100397b0 0x7fd01003bc60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd018006fd0 tx=0x7fd018006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:33.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.533+0000 7fd00bfff640 1 -- 172.21.15.204:0/313965461 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd01401b540 con 0x7fd02409fd80 2024-09-15T01:35:33.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.533+0000 7fd00bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:33.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.533+0000 7fd00bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:33.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.697+0000 7fd02a99d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-09-15T01:35:33.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.697+0000 7fd02a99d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T01:35:33.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.697+0000 7fd02a99d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:33.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.697+0000 7fd02a99d640 1 -- 172.21.15.204:0/313965461 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush weight-set dump"} v 0) v1 -- 0x7fd02411fb70 con 0x7fd02409fd80 2024-09-15T01:35:33.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.698+0000 7fd00bfff640 1 -- 172.21.15.204:0/313965461 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v127) v1 ==== 73+0+7036 (secure 0 0 0) 0x7fd014018070 con 0x7fd02409fd80 2024-09-15T01:35:33.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.698+0000 7fd00bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T01:35:33.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.698+0000 7fd00bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:33.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.797+0000 7fd02a99d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-09-15T01:35:33.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.798+0000 7fd02a99d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T01:35:33.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.798+0000 7fd02a99d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:33.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.798+0000 7fd02a99d640 1 -- 172.21.15.204:0/313965461 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush weight-set dump"} v 0) v1 -- 0x7fd02409dfb0 con 0x7fd02409fd80 2024-09-15T01:35:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.798+0000 7fd00bfff640 1 -- 172.21.15.204:0/313965461 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v127) v1 ==== 73+0+7036 (secure 0 0 0) 0x7fd014011af0 con 0x7fd02409fd80 2024-09-15T01:35:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.798+0000 7fd00bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set dump"}] 2024-09-15T01:35:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.798+0000 7fd00bfff640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:33.801 INFO:tasks.workunit.client.0.smithi204.stdout:{ 2024-09-15T01:35:33.801 INFO:tasks.workunit.client.0.smithi204.stdout: "9": [ 2024-09-15T01:35:33.801 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.802 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -1, 2024-09-15T01:35:33.802 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.802 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.802 INFO:tasks.workunit.client.0.smithi204.stdout: 0.261566162109375, 2024-09-15T01:35:33.802 INFO:tasks.workunit.client.0.smithi204.stdout: 0.261566162109375, 2024-09-15T01:35:33.802 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.802 INFO:tasks.workunit.client.0.smithi204.stdout: 0 2024-09-15T01:35:33.802 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.802 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.802 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -2, 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -3, 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.803 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -4, 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.804 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -5, 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -6, 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.805 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: 0 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -7, 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.806 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -8, 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -9, 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.807 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: 0 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -10, 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -11, 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.808 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.809 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.809 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.809 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.809 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -12, 2024-09-15T01:35:33.809 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.809 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.809 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.809 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.809 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.809 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -13, 2024-09-15T01:35:33.809 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.810 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.810 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.810 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.810 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.810 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -14, 2024-09-15T01:35:33.810 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.810 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.810 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.810 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.810 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.810 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -15, 2024-09-15T01:35:33.810 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: "10": [ 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -1, 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: 0.261566162109375, 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: 0.261566162109375, 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.811 INFO:tasks.workunit.client.0.smithi204.stdout: 0 2024-09-15T01:35:33.812 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.812 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.812 INFO:tasks.workunit.client.0.smithi204.stdout: 0.261566162109375, 2024-09-15T01:35:33.812 INFO:tasks.workunit.client.0.smithi204.stdout: 0.261566162109375, 2024-09-15T01:35:33.812 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.812 INFO:tasks.workunit.client.0.smithi204.stdout: 0 2024-09-15T01:35:33.812 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.812 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.812 INFO:tasks.workunit.client.0.smithi204.stdout: 0.261566162109375, 2024-09-15T01:35:33.812 INFO:tasks.workunit.client.0.smithi204.stdout: 0.261566162109375, 2024-09-15T01:35:33.812 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.812 INFO:tasks.workunit.client.0.smithi204.stdout: 0 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -2, 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -3, 2024-09-15T01:35:33.813 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.814 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.814 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.814 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.814 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.814 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.814 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.814 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.814 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.814 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.814 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.814 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.814 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.815 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.815 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.815 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.815 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.815 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.815 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.815 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -4, 2024-09-15T01:35:33.815 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.815 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.815 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.815 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.815 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -5, 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.816 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.817 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.817 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.817 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.817 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.817 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.817 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.817 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.817 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.817 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.817 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.817 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -6, 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: 0 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.818 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: 0 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: 0 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -7, 2024-09-15T01:35:33.819 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.820 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.820 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.820 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.820 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.820 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.820 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.820 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -8, 2024-09-15T01:35:33.820 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.821 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -9, 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: 0 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: 0 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: ], 2024-09-15T01:35:33.822 INFO:tasks.workunit.client.0.smithi204.stdout: [ 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: 0.087188720703125, 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: 0, 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: 0 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -10, 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.823 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -11, 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -12, 2024-09-15T01:35:33.824 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -13, 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.825 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.826 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.826 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -14, 2024-09-15T01:35:33.826 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.826 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.826 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.826 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.826 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.826 INFO:tasks.workunit.client.0.smithi204.stdout: }, 2024-09-15T01:35:33.826 INFO:tasks.workunit.client.0.smithi204.stdout: { 2024-09-15T01:35:33.826 INFO:tasks.workunit.client.0.smithi204.stdout: "bucket_id": -15, 2024-09-15T01:35:33.826 INFO:tasks.workunit.client.0.smithi204.stdout: "weight_set": [ 2024-09-15T01:35:33.826 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.827 INFO:tasks.workunit.client.0.smithi204.stdout: [], 2024-09-15T01:35:33.827 INFO:tasks.workunit.client.0.smithi204.stdout: [] 2024-09-15T01:35:33.827 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.827 INFO:tasks.workunit.client.0.smithi204.stdout: } 2024-09-15T01:35:33.827 INFO:tasks.workunit.client.0.smithi204.stdout: ] 2024-09-15T01:35:33.827 INFO:tasks.workunit.client.0.smithi204.stdout:} 2024-09-15T01:35:33.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.800+0000 7fd02a99d640 1 -- 172.21.15.204:0/313965461 >> v2:172.21.15.204:6812/28210 conn(0x7fd0100397b0 msgr2=0x7fd01003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.800+0000 7fd02a99d640 1 --2- 172.21.15.204:0/313965461 >> v2:172.21.15.204:6812/28210 conn(0x7fd0100397b0 0x7fd01003bc60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd018006fd0 tx=0x7fd018006d30 comp rx=0 tx=0).stop 2024-09-15T01:35:33.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.800+0000 7fd02a99d640 10 monclient: shutdown 2024-09-15T01:35:33.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.800+0000 7fd02a99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.800+0000 7fd02a99d640 1 -- 172.21.15.204:0/313965461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd02409fd80 msgr2=0x7fd02412f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.800+0000 7fd02a99d640 1 --2- 172.21.15.204:0/313965461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd02409fd80 0x7fd02412f7e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd01400d890 tx=0x7fd014008040 comp rx=0 tx=0).stop 2024-09-15T01:35:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.800+0000 7fd02a99d640 1 -- 172.21.15.204:0/313965461 shutdown_connections 2024-09-15T01:35:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.800+0000 7fd02a99d640 1 --2- 172.21.15.204:0/313965461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0240a2e30 0x7fd02411cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.800+0000 7fd02a99d640 1 --2- 172.21.15.204:0/313965461 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd0240a09c0 0x7fd02411c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.800+0000 7fd02a99d640 1 --2- 172.21.15.204:0/313965461 >> v2:172.21.15.204:6812/28210 conn(0x7fd0100397b0 0x7fd01003bc60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.800+0000 7fd02a99d640 1 --2- 172.21.15.204:0/313965461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd02409fd80 0x7fd02412f7e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.800+0000 7fd02a99d640 1 -- 172.21.15.204:0/313965461 >> 172.21.15.204:0/313965461 conn(0x7fd02406e860 msgr2=0x7fd0240a4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:33.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.801+0000 7fd02a99d640 1 -- 172.21.15.204:0/313965461 shutdown_connections 2024-09-15T01:35:33.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.801+0000 7fd02a99d640 1 -- 172.21.15.204:0/313965461 wait complete. 2024-09-15T01:35:33.830 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 2024-09-15T01:35:33.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.885+0000 7f721d91d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:33.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.885+0000 7f721d91d640 10 monclient: build_initial_monmap 2024-09-15T01:35:33.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.885+0000 7f721d91d640 10 monclient: monmap: 2024-09-15T01:35:33.888 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:33.888 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:33.887343+0000 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:33.887343+0000 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:33.889 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:33.890 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:33.890 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:33.890 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.886+0000 7f721d91d640 1 Processor -- start 2024-09-15T01:35:33.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.886+0000 7f721d91d640 1 -- start start 2024-09-15T01:35:33.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.886+0000 7f721d91d640 10 monclient: init 2024-09-15T01:35:33.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.886+0000 7f721d91d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:33.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 10 monclient: _add_conns ranks=[6,8,3,9,1,0,7,5,4,2,10] 2024-09-15T01:35:33.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f72180a2cd0 0x7f7218132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 10 monclient(hunting): picked mon.noname-d con 0x7f72180a2cd0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:33.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f721809fa20 0x7f721809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 10 monclient(hunting): picked mon.noname-e con 0x7f721809fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:33.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72180a08e0 0x7f72180a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 10 monclient(hunting): picked mon.noname-g con 0x7f72180a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:33.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72180a30a0 con 0x7f72180a08e0 2024-09-15T01:35:33.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f72180a0330 con 0x7f72180a2cd0 2024-09-15T01:35:33.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f72180a1db0 con 0x7f721809fa20 2024-09-15T01:35:33.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:33.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f721d91d640 10 monclient(hunting): authenticate will time out at 949.530075s 2024-09-15T01:35:33.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f7216ffd640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f72180a2cd0 msgr2=0x7f7218132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72180a08e0 0x7f72180a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f7216ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f72180a2cd0 0x7f7218132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72180a08e0 0x7f72180a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:50764/0 (socket says 172.21.15.204:50764) 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 1 -- 172.21.15.204:0/962037962 learned_addr learned my addr 172.21.15.204:0/962037962 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 10 monclient(hunting): get_auth_request con 0x7f72180a08e0 auth_method 0 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f720ffff640 1 --2- 172.21.15.204:0/962037962 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f721809fa20 0x7f721809fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 10 monclient(hunting): handle_auth_done global_id 14745 payload 274 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 1 -- 172.21.15.204:0/962037962 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f721809fa20 msgr2=0x7f721809fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 1 --2- 172.21.15.204:0/962037962 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f721809fa20 0x7f721809fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 1 -- 172.21.15.204:0/962037962 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f72180a2cd0 msgr2=0x7f7218132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 1 --2- 172.21.15.204:0/962037962 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f72180a2cd0 0x7f7218132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 1 monclient: found mon.noname-g 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f72177fe640 1 -- 172.21.15.204:0/962037962 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72180648a0 con 0x7f72180a08e0 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f720ffff640 10 monclient: get_auth_request con 0x7f721809fa20 auth_method 0 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.887+0000 7f720ffff640 1 --2- 172.21.15.204:0/962037962 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f721809fa20 0x7f721809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72177fe640 1 --2- 172.21.15.204:0/962037962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72180a08e0 0x7f72180a0d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7210009f50 tx=0x7f7210009840 comp rx=0 tx=0).ready entity=mon.3 client_cookie=f2cab9286b5d0a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72167fc640 1 -- 172.21.15.204:0/962037962 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7210007880 con 0x7f72180a08e0 2024-09-15T01:35:33.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72167fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72167fc640 10 monclient: dump: 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:33.902 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:33.903 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:33.903 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:33.903 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:33.903 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:33.903 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72167fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:33.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:33.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f721d91d640 5 monclient: authenticate success, global_id 14745 2024-09-15T01:35:33.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f721d91d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:33.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72167fc640 1 -- 172.21.15.204:0/962037962 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721002d050 con 0x7f72180a08e0 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72167fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f721d91d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f721d91d640 4 set_mon_vals no callback set 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72167fc640 1 -- 172.21.15.204:0/962037962 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7210030360 con 0x7f72180a08e0 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72167fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f72167fc640 10 monclient: dump: 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:33.904 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f721d91d640 10 monclient: shutdown 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f721d91d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f721d91d640 1 -- 172.21.15.204:0/962037962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72180a08e0 msgr2=0x7f72180a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.888+0000 7f721d91d640 1 --2- 172.21.15.204:0/962037962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72180a08e0 0x7f72180a0d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7210009f50 tx=0x7f7210009840 comp rx=0 tx=0).stop 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.889+0000 7f721d91d640 1 -- 172.21.15.204:0/962037962 shutdown_connections 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.889+0000 7f721d91d640 1 --2- 172.21.15.204:0/962037962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f72180a08e0 0x7f72180a0d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.889+0000 7f721d91d640 1 --2- 172.21.15.204:0/962037962 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f721809fa20 0x7f721809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.889+0000 7f721d91d640 1 --2- 172.21.15.204:0/962037962 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f72180a2cd0 0x7f7218132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.889+0000 7f721d91d640 1 -- 172.21.15.204:0/962037962 >> 172.21.15.204:0/962037962 conn(0x7f721806e860 msgr2=0x7f721806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.889+0000 7f721d91d640 1 -- 172.21.15.204:0/962037962 shutdown_connections 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.889+0000 7f721d91d640 1 -- 172.21.15.204:0/962037962 wait complete. 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.889+0000 7f721d91d640 10 monclient: build_initial_monmap 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.889+0000 7f721d91d640 10 monclient: monmap: 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:33.906 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:33.907 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:33.907 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:33.907 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:33.907 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:33.907 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:33.907 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:33.907 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:33.907 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:33.907 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:33.907 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:33.907 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:33.907 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.889+0000 7f721d91d640 1 Processor -- start 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.889+0000 7f721d91d640 1 -- start start 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.889+0000 7f721d91d640 10 monclient: init 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 10 monclient: _add_conns ranks=[1,2,5,3,6,8,7,4,0] 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f721809fa20 0x7f7218130ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 10 monclient(hunting): picked mon.noname-f con 0x7f721809fa20 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72180a08e0 0x7f721811d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 10 monclient(hunting): picked mon.noname-b con 0x7f72180a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f72180a2cd0 0x7f721811db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 10 monclient(hunting): picked mon.noname-h con 0x7f72180a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721809d3d0 con 0x7f721809fa20 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721809cb60 con 0x7f72180a08e0 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f721811fd20 con 0x7f72180a2cd0 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f721d91d640 10 monclient(hunting): authenticate will time out at 949.533323s 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f721809fa20 0x7f7218130ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f720ffff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72180a08e0 msgr2=0x7f721811d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f721809fa20 0x7f7218130ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47314/0 (socket says 172.21.15.204:47314) 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f72177fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f72180a2cd0 0x7f721811db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 1 -- 172.21.15.204:0/4142935308 learned_addr learned my addr 172.21.15.204:0/4142935308 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 10 monclient(hunting): get_auth_request con 0x7f721809fa20 auth_method 0 2024-09-15T01:35:33.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f720ffff640 1 --2- 172.21.15.204:0/4142935308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72180a08e0 0x7f721811d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f72177fe640 10 monclient(hunting): get_auth_request con 0x7f72180a2cd0 auth_method 0 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f72177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f72177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f72177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f72177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f72177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:33.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f72177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:33.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 10 monclient(hunting): handle_auth_done global_id 6229 payload 824 2024-09-15T01:35:33.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 1 -- 172.21.15.204:0/4142935308 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f72180a2cd0 msgr2=0x7f721811db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:33.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 1 --2- 172.21.15.204:0/4142935308 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f72180a2cd0 0x7f721811db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 1 -- 172.21.15.204:0/4142935308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72180a08e0 msgr2=0x7f721811d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:33.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 1 --2- 172.21.15.204:0/4142935308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72180a08e0 0x7f721811d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:33.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:35:33.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 1 monclient: found mon.noname-f 2024-09-15T01:35:33.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:33.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:33.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 1 -- 172.21.15.204:0/4142935308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7210009ae0 con 0x7f721809fa20 2024-09-15T01:35:33.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f72177fe640 1 --2- 172.21.15.204:0/4142935308 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f72180a2cd0 0x7f721811db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:33.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.890+0000 7f7216ffd640 1 --2- 172.21.15.204:0/4142935308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f721809fa20 0x7f7218130ac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f720800b600 tx=0x7f72080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:33.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f720effd640 1 -- 172.21.15.204:0/4142935308 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7208015070 con 0x7f721809fa20 2024-09-15T01:35:33.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f720effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:33.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f720effd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:33.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f720effd640 10 monclient: dump: 2024-09-15T01:35:33.912 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:33.912 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.912 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.912 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.912 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:33.912 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.912 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:33.913 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:33.913 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:33.913 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:33.913 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:33.913 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:33.913 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:33.913 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:33.913 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f720effd640 10 monclient: _finish_auth 0 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f720effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f721d91d640 5 monclient: authenticate success, global_id 6229 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f720effd640 1 -- 172.21.15.204:0/4142935308 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7208006150 con 0x7f721809fa20 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f720effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f720effd640 1 -- 172.21.15.204:0/4142935308 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7208006740 con 0x7f721809fa20 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f721d91d640 10 monclient: _renew_subs 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f721d91d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f721d91d640 1 -- 172.21.15.204:0/4142935308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7218120c10 con 0x7f721809fa20 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f720effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f720effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f7215ffb640 4 set_mon_vals no callback set 2024-09-15T01:35:33.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f720effd640 10 monclient: dump: 2024-09-15T01:35:33.915 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:33.915 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:33.915 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.915 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:33.915 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:33.915 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:33.915 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:33.915 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:33.915 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:33.915 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f721d91d640 10 monclient: _renew_subs 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f721d91d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.891+0000 7f721d91d640 1 -- 172.21.15.204:0/4142935308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721811f150 con 0x7f721809fa20 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.892+0000 7f720effd640 1 -- 172.21.15.204:0/4142935308 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f72080062c0 con 0x7f721809fa20 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.892+0000 7f721d91d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.892+0000 7f721d91d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:33.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.892+0000 7f721d91d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:33.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.892+0000 7f721d91d640 1 -- 172.21.15.204:0/4142935308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7218070b80 con 0x7f721809fa20 2024-09-15T01:35:33.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.892+0000 7f720effd640 1 --2- 172.21.15.204:0/4142935308 >> v2:172.21.15.204:6812/28210 conn(0x7f71f80397b0 0x7f71f803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:33.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.892+0000 7f720effd640 1 -- 172.21.15.204:0/4142935308 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 7317+0+0 (secure 0 0 0) 0x7f720804b7c0 con 0x7f721809fa20 2024-09-15T01:35:33.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.892+0000 7f720ffff640 1 --2- 172.21.15.204:0/4142935308 >> v2:172.21.15.204:6812/28210 conn(0x7f71f80397b0 0x7f71f803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:33.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.892+0000 7f720ffff640 10 monclient: get_auth_request con 0x7f71f80397b0 auth_method 0 2024-09-15T01:35:33.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.892+0000 7f720ffff640 1 --2- 172.21.15.204:0/4142935308 >> v2:172.21.15.204:6812/28210 conn(0x7f71f80397b0 0x7f71f803bc60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7200006ca0 tx=0x7f7200008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:33.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.896+0000 7f720effd640 1 -- 172.21.15.204:0/4142935308 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7208003e10 con 0x7f721809fa20 2024-09-15T01:35:33.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.896+0000 7f720effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:33.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:33.896+0000 7f720effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:34.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:34.055+0000 7f721d91d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:34.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:34.055+0000 7f721d91d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:34.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:34.055+0000 7f721d91d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:34.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:34.055+0000 7f721d91d640 1 -- 172.21.15.204:0/4142935308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) v1 -- 0x7f7218067450 con 0x7f721809fa20 2024-09-15T01:35:34.626 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:34.624+0000 7f720effd640 1 -- 172.21.15.204:0/4142935308 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=0 v128) v1 ==== 129+0+0 (secure 0 0 0) 0x7f720800d370 con 0x7f721809fa20 2024-09-15T01:35:34.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:34.624+0000 7f720effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:34.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:34.624+0000 7f720effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:34.727+0000 7f721d91d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:34.727+0000 7f721d91d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:34.727+0000 7f721d91d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:34.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:34.727+0000 7f721d91d640 1 -- 172.21.15.204:0/4142935308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) v1 -- 0x7f72181427b0 con 0x7f721809fa20 2024-09-15T01:35:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.622+0000 7f720effd640 1 -- 172.21.15.204:0/4142935308 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=0 v129) v1 ==== 129+0+0 (secure 0 0 0) 0x7f720800eb30 con 0x7f721809fa20 2024-09-15T01:35:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.622+0000 7f720effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:35.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.622+0000 7f720effd640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:35.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.625+0000 7f721d91d640 1 -- 172.21.15.204:0/4142935308 >> v2:172.21.15.204:6812/28210 conn(0x7f71f80397b0 msgr2=0x7f71f803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:35.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.626+0000 7f721d91d640 1 --2- 172.21.15.204:0/4142935308 >> v2:172.21.15.204:6812/28210 conn(0x7f71f80397b0 0x7f71f803bc60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7200006ca0 tx=0x7f7200008040 comp rx=0 tx=0).stop 2024-09-15T01:35:35.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.626+0000 7f721d91d640 10 monclient: shutdown 2024-09-15T01:35:35.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.626+0000 7f721d91d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:35.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.626+0000 7f721d91d640 1 -- 172.21.15.204:0/4142935308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f721809fa20 msgr2=0x7f7218130ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:35.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.626+0000 7f721d91d640 1 --2- 172.21.15.204:0/4142935308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f721809fa20 0x7f7218130ac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f720800b600 tx=0x7f72080091a0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.626+0000 7f721d91d640 1 -- 172.21.15.204:0/4142935308 shutdown_connections 2024-09-15T01:35:35.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.626+0000 7f721d91d640 1 --2- 172.21.15.204:0/4142935308 >> v2:172.21.15.204:6812/28210 conn(0x7f71f80397b0 0x7f71f803bc60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.626+0000 7f721d91d640 1 --2- 172.21.15.204:0/4142935308 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f72180a2cd0 0x7f721811db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.626+0000 7f721d91d640 1 --2- 172.21.15.204:0/4142935308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72180a08e0 0x7f721811d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.626+0000 7f721d91d640 1 --2- 172.21.15.204:0/4142935308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f721809fa20 0x7f7218130ac0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.626+0000 7f721d91d640 1 -- 172.21.15.204:0/4142935308 >> 172.21.15.204:0/4142935308 conn(0x7f721806e860 msgr2=0x7f72180a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:35.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.627+0000 7f721d91d640 1 -- 172.21.15.204:0/4142935308 shutdown_connections 2024-09-15T01:35:35.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.627+0000 7f721d91d640 1 -- 172.21.15.204:0/4142935308 wait complete. 2024-09-15T01:35:35.640 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush weight-set reweight fooset osd.0 .9 .9 2024-09-15T01:35:35.640 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:35:35.640 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 .9 2024-09-15T01:35:35.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.720+0000 7f4d1ef9d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.720+0000 7f4d1ef9d640 10 monclient: build_initial_monmap 2024-09-15T01:35:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.720+0000 7f4d1ef9d640 10 monclient: monmap: 2024-09-15T01:35:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:35.721850+0000 2024-09-15T01:35:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:35.721850+0000 2024-09-15T01:35:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:35.723 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:35.724 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:35.724 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 1 Processor -- start 2024-09-15T01:35:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 1 -- start start 2024-09-15T01:35:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 10 monclient: init 2024-09-15T01:35:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 10 monclient: _add_conns ranks=[5,0,2,6,3,7,4,10,8,1,9] 2024-09-15T01:35:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d180a2e30 0x7f4d18132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 10 monclient(hunting): picked mon.noname-h con 0x7f4d180a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4d1809fb80 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d1809ff70 0x7f4d180a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:35.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 10 monclient(hunting): picked mon.noname-b con 0x7f4d1809ff70 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1804b720 con 0x7f4d1809fb80 2024-09-15T01:35:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1809e6c0 con 0x7f4d1809ff70 2024-09-15T01:35:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180a3200 con 0x7f4d180a2e30 2024-09-15T01:35:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1ef9d640 10 monclient(hunting): authenticate will time out at 951.364836s 2024-09-15T01:35:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1cd39640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d180a2e30 0x7f4d18132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1cd39640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d180a2e30 0x7f4d18132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:45962/0 (socket says 172.21.15.204:45962) 2024-09-15T01:35:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1cd39640 1 -- 172.21.15.204:0/4061614435 learned_addr learned my addr 172.21.15.204:0/4061614435 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1cd39640 10 monclient(hunting): get_auth_request con 0x7f4d180a2e30 auth_method 0 2024-09-15T01:35:35.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1cd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1cd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1cd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1d53a640 1 -- 172.21.15.204:0/4061614435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d1809ff70 msgr2=0x7f4d180a1150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.721+0000 7f4d1d53a640 1 --2- 172.21.15.204:0/4061614435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d1809ff70 0x7f4d180a1150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 10 monclient(hunting): handle_auth_done global_id 5720 payload 274 2024-09-15T01:35:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 1 -- 172.21.15.204:0/4061614435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d1809ff70 msgr2=0x7f4d180a1150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 1 --2- 172.21.15.204:0/4061614435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d1809ff70 0x7f4d180a1150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 1 -- 172.21.15.204:0/4061614435 >> v1:172.21.15.178:6789/0 conn(0x7f4d1809fb80 legacy=0x7f4d180a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:35.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 10 monclient: _finish_hunting 0 2024-09-15T01:35:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 1 monclient: found mon.noname-h 2024-09-15T01:35:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 1 -- 172.21.15.204:0/4061614435 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180a1f10 con 0x7f4d180a2e30 2024-09-15T01:35:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.722+0000 7f4d1cd39640 1 --2- 172.21.15.204:0/4061614435 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d180a2e30 0x7f4d18132a60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009940 tx=0x7f4d0c0091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=5eaf6847a1126489 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d177fe640 1 -- 172.21.15.204:0/4061614435 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d0c02ad20 con 0x7f4d180a2e30 2024-09-15T01:35:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d177fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:35.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d177fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d177fe640 10 monclient: dump: 2024-09-15T01:35:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:35.737 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:35.738 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d177fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 5 monclient: authenticate success, global_id 5720 2024-09-15T01:35:35.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d177fe640 1 -- 172.21.15.204:0/4061614435 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c031820 con 0x7f4d180a2e30 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d177fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 4 set_mon_vals no callback set 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d177fe640 1 -- 172.21.15.204:0/4061614435 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d0c02aea0 con 0x7f4d180a2e30 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d177fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d177fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d177fe640 10 monclient: dump: 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:35.739 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 10 monclient: shutdown 2024-09-15T01:35:35.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 1 -- 172.21.15.204:0/4061614435 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d180a2e30 msgr2=0x7f4d18132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 1 --2- 172.21.15.204:0/4061614435 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d180a2e30 0x7f4d18132a60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009940 tx=0x7f4d0c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 1 -- 172.21.15.204:0/4061614435 shutdown_connections 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 1 --2- 172.21.15.204:0/4061614435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d1809ff70 0x7f4d180a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 1 --2- 172.21.15.204:0/4061614435 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d180a2e30 0x7f4d18132a60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 1 -- 172.21.15.204:0/4061614435 >> 172.21.15.204:0/4061614435 conn(0x7f4d1806e860 msgr2=0x7f4d1806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 1 -- 172.21.15.204:0/4061614435 shutdown_connections 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.723+0000 7f4d1ef9d640 1 -- 172.21.15.204:0/4061614435 wait complete. 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.724+0000 7f4d1ef9d640 10 monclient: build_initial_monmap 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.724+0000 7f4d1ef9d640 10 monclient: monmap: 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:35.741 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:35.742 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:35.742 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:35.742 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:35.742 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:35.742 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:35.742 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:35.742 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:35.742 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:35.742 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:35.742 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:35.742 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:35.742 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.724+0000 7f4d1ef9d640 1 Processor -- start 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.724+0000 7f4d1ef9d640 1 -- start start 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.724+0000 7f4d1ef9d640 10 monclient: init 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 10 monclient: _add_conns ranks=[3,4,0,5,1,7,6,2,8] 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d1809fb80 0x7f4d18141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 10 monclient(hunting): picked mon.noname-g con 0x7f4d1809fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4d1809ff70 0x7f4d18132830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 10 monclient(hunting): picked mon.noname-c con 0x7f4d1809ff70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d180a2e30 0x7f4d1811f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 10 monclient(hunting): picked mon.noname-a con 0x7f4d180a2e30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:35.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d18121680 con 0x7f4d180a2e30 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1809ca40 con 0x7f4d1809fb80 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1809d650 con 0x7f4d1809ff70 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1ef9d640 10 monclient(hunting): authenticate will time out at 951.368243s 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d1809fb80 0x7f4d18141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d1809fb80 0x7f4d18141dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:50770/0 (socket says 172.21.15.204:50770) 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 1 -- 172.21.15.204:0/3945506428 learned_addr learned my addr 172.21.15.204:0/3945506428 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 10 monclient(hunting): get_auth_request con 0x7f4d1809fb80 auth_method 0 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:35.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:35.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:35.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:35.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:35.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:35.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1d53a640 1 --2- 172.21.15.204:0/3945506428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d180a2e30 0x7f4d1811f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:35.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d17fff640 1 --2- 172.21.15.204:0/3945506428 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4d1809ff70 0x7f4d18132830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:35.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1d53a640 10 monclient(hunting): get_auth_request con 0x7f4d180a2e30 auth_method 0 2024-09-15T01:35:35.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1d53a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:35.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1d53a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1d53a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 10 monclient(hunting): handle_auth_done global_id 14754 payload 824 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 1 -- 172.21.15.204:0/3945506428 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4d1809ff70 msgr2=0x7f4d18132830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 1 --2- 172.21.15.204:0/3945506428 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4d1809ff70 0x7f4d18132830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 1 -- 172.21.15.204:0/3945506428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d180a2e30 msgr2=0x7f4d1811f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 1 --2- 172.21.15.204:0/3945506428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d180a2e30 0x7f4d1811f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 10 monclient: _finish_hunting 0 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 1 monclient: found mon.noname-g 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 1 -- 172.21.15.204:0/3945506428 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c009450 con 0x7f4d1809fb80 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d17fff640 10 monclient: get_auth_request con 0x7f4d1809ff70 auth_method 0 2024-09-15T01:35:35.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d17fff640 1 --2- 172.21.15.204:0/3945506428 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4d1809ff70 0x7f4d18132830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.725+0000 7f4d1cd39640 1 --2- 172.21.15.204:0/3945506428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d1809fb80 0x7f4d18141dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c000c00 tx=0x7f4d0c032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 1 -- 172.21.15.204:0/3945506428 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d0c033e30 con 0x7f4d1809fb80 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 10 monclient: dump: 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:35.747 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:35.748 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:35.748 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:35.748 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:35.748 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:35.748 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:35.748 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:35.748 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:35.748 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:35.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:35.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:35.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 1 -- 172.21.15.204:0/3945506428 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c03a390 con 0x7f4d1809fb80 2024-09-15T01:35:35.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d1ef9d640 5 monclient: authenticate success, global_id 14754 2024-09-15T01:35:35.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:35.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d1ef9d640 10 monclient: _renew_subs 2024-09-15T01:35:35.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 1 -- 172.21.15.204:0/3945506428 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d0c03a980 con 0x7f4d1809fb80 2024-09-15T01:35:35.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d1ef9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:35.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d1ef9d640 1 -- 172.21.15.204:0/3945506428 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d18122cf0 con 0x7f4d1809fb80 2024-09-15T01:35:35.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:35.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:35.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d16ffd640 4 set_mon_vals no callback set 2024-09-15T01:35:35.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 10 monclient: dump: 2024-09-15T01:35:35.749 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:35.749 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:35.750 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:35.751 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:35.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d1ef9d640 10 monclient: _renew_subs 2024-09-15T01:35:35.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d1ef9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:35.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d1ef9d640 1 -- 172.21.15.204:0/3945506428 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d18122980 con 0x7f4d1809fb80 2024-09-15T01:35:35.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.726+0000 7f4d14ff9640 1 -- 172.21.15.204:0/3945506428 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4d0c039040 con 0x7f4d1809fb80 2024-09-15T01:35:35.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.727+0000 7f4d1ef9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:35.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.727+0000 7f4d1ef9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:35.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.727+0000 7f4d1ef9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:35.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.727+0000 7f4d1ef9d640 1 -- 172.21.15.204:0/3945506428 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d18070cc0 con 0x7f4d1809fb80 2024-09-15T01:35:35.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.727+0000 7f4d14ff9640 1 --2- 172.21.15.204:0/3945506428 >> v2:172.21.15.204:6812/28210 conn(0x7f4cfc039810 0x7f4cfc03bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:35.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.727+0000 7f4d14ff9640 1 -- 172.21.15.204:0/3945506428 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 7317+0+0 (secure 0 0 0) 0x7f4d0c06bb10 con 0x7f4d1809fb80 2024-09-15T01:35:35.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.727+0000 7f4d17fff640 1 --2- 172.21.15.204:0/3945506428 >> v2:172.21.15.204:6812/28210 conn(0x7f4cfc039810 0x7f4cfc03bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:35.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.727+0000 7f4d17fff640 10 monclient: get_auth_request con 0x7f4cfc039810 auth_method 0 2024-09-15T01:35:35.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.727+0000 7f4d17fff640 1 --2- 172.21.15.204:0/3945506428 >> v2:172.21.15.204:6812/28210 conn(0x7f4cfc039810 0x7f4cfc03bcc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4d181209a0 tx=0x7f4d04009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:35.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.731+0000 7f4d14ff9640 1 -- 172.21.15.204:0/3945506428 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4d0c02fd10 con 0x7f4d1809fb80 2024-09-15T01:35:35.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.731+0000 7f4d14ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:35.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.731+0000 7f4d14ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:35.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.896+0000 7f4d1ef9d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}] 2024-09-15T01:35:35.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.896+0000 7f4d1ef9d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}] 2024-09-15T01:35:35.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.896+0000 7f4d1ef9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:35.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.896+0000 7f4d1ef9d640 1 -- 172.21.15.204:0/3945506428 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]} v 0) v1 -- 0x7f4d1809ddb0 con 0x7f4d1809fb80 2024-09-15T01:35:35.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.897+0000 7f4d14ff9640 1 -- 172.21.15.204:0/3945506428 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}]=-22 must specify exact 1 weight values v129) v1 ==== 168+0+0 (secure 0 0 0) 0x7f4d0c037070 con 0x7f4d1809fb80 2024-09-15T01:35:35.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.897+0000 7f4d14ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}] 2024-09-15T01:35:35.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.897+0000 7f4d14ff9640 10 monclient: _finish_command 2 = mon:22 must specify exact 1 weight values 2024-09-15T01:35:35.900 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: must specify exact 1 weight values 2024-09-15T01:35:35.903 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:35:35.903 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect_false ceph osd crush weight-set reweight barset osd.0 .9 2024-09-15T01:35:35.903 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -x 2024-09-15T01:35:35.903 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set reweight barset osd.0 .9 2024-09-15T01:35:35.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.978+0000 7fa92479d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:35.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.978+0000 7fa92479d640 10 monclient: build_initial_monmap 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.978+0000 7fa92479d640 10 monclient: monmap: 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:35.979661+0000 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:35.979661+0000 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:35.981 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:35.982 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:35.982 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:35.982 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:35.982 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:35.982 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:35.982 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:35.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.978+0000 7fa92479d640 1 Processor -- start 2024-09-15T01:35:35.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.978+0000 7fa92479d640 1 -- start start 2024-09-15T01:35:35.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.978+0000 7fa92479d640 10 monclient: init 2024-09-15T01:35:35.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:35.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 10 monclient: _add_conns ranks=[10,3,7,6,8,0,2,1,5,9,4] 2024-09-15T01:35:35.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa91c0a2cd0 0x7fa91c132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:35.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 10 monclient(hunting): picked mon.noname-f con 0x7fa91c0a2cd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:35.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa91c09fa20 0x7fa91c09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:35.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 10 monclient(hunting): picked mon.noname-g con 0x7fa91c09fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:35.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa91c0a08e0 0x7fa91c0a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:35.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 10 monclient(hunting): picked mon.noname-i con 0x7fa91c0a08e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:35.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:35.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91c0a30a0 con 0x7fa91c0a2cd0 2024-09-15T01:35:35.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:35.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0a0330 con 0x7fa91c09fa20 2024-09-15T01:35:35.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:35.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0a1db0 con 0x7fa91c0a08e0 2024-09-15T01:35:35.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:35.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa92479d640 10 monclient(hunting): authenticate will time out at 951.622492s 2024-09-15T01:35:35.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922d3a640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa91c0a08e0 msgr2=0x7fa91c0a0d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:35.991 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922d3a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa91c0a08e0 0x7fa91c0a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:35.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa91c0a2cd0 0x7fa91c132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:35.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa921d38640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa91c09fa20 0x7fa91c09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:35.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa921d38640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa91c09fa20 0x7fa91c09fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:50774/0 (socket says 172.21.15.204:50774) 2024-09-15T01:35:35.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa91c0a2cd0 0x7fa91c132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47316/0 (socket says 172.21.15.204:47316) 2024-09-15T01:35:35.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa921d38640 1 -- 172.21.15.204:0/1864943743 learned_addr learned my addr 172.21.15.204:0/1864943743 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:35.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa921d38640 10 monclient(hunting): get_auth_request con 0x7fa91c09fa20 auth_method 0 2024-09-15T01:35:35.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa921d38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:35.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa921d38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:35.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa921d38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:35.992 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 10 monclient(hunting): get_auth_request con 0x7fa91c0a2cd0 auth_method 0 2024-09-15T01:35:35.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:35.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:35.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:35.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa921d38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:35.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa921d38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:35.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa921d38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:35.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:35.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:35.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:35.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 10 monclient(hunting): handle_auth_done global_id 6238 payload 274 2024-09-15T01:35:35.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 1 -- 172.21.15.204:0/1864943743 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa91c0a08e0 msgr2=0x7fa91c0a0d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:35.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 1 --2- 172.21.15.204:0/1864943743 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa91c0a08e0 0x7fa91c0a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 1 -- 172.21.15.204:0/1864943743 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa91c09fa20 msgr2=0x7fa91c09fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:35.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 1 --2- 172.21.15.204:0/1864943743 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa91c09fa20 0x7fa91c09fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:35.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 10 monclient: _finish_hunting 0 2024-09-15T01:35:35.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 1 monclient: found mon.noname-f 2024-09-15T01:35:35.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:35.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:35.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa922539640 1 -- 172.21.15.204:0/1864943743 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c0648a0 con 0x7fa91c0a2cd0 2024-09-15T01:35:35.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.979+0000 7fa921d38640 1 --2- 172.21.15.204:0/1864943743 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa91c09fa20 0x7fa91c09fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:35.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa922539640 1 --2- 172.21.15.204:0/1864943743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa91c0a2cd0 0x7fa91c132c00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa918009850 tx=0x7fa9180090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ad59bf0108f9ca server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:35.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa921537640 1 -- 172.21.15.204:0/1864943743 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa91802abe0 con 0x7fa91c0a2cd0 2024-09-15T01:35:35.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa921537640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:35.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa921537640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:35.996 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa921537640 10 monclient: dump: 2024-09-15T01:35:35.996 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:35.996 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:35.996 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:35.996 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:35.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa921537640 10 monclient: _finish_auth 0 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa921537640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa921537640 1 -- 172.21.15.204:0/1864943743 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91802ad60 con 0x7fa91c0a2cd0 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa92479d640 5 monclient: authenticate success, global_id 6238 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa92479d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa921537640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa92479d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa921537640 1 -- 172.21.15.204:0/1864943743 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa918031ca0 con 0x7fa91c0a2cd0 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa92479d640 4 set_mon_vals no callback set 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa921537640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa921537640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.980+0000 7fa921537640 10 monclient: dump: 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:35.998 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:35.999 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:36.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 10 monclient: shutdown 2024-09-15T01:35:36.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:36.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 1 -- 172.21.15.204:0/1864943743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa91c0a2cd0 msgr2=0x7fa91c132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:36.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 1 --2- 172.21.15.204:0/1864943743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa91c0a2cd0 0x7fa91c132c00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa918009850 tx=0x7fa9180090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:36.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 1 -- 172.21.15.204:0/1864943743 shutdown_connections 2024-09-15T01:35:36.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 1 --2- 172.21.15.204:0/1864943743 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa91c0a08e0 0x7fa91c0a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:36.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 1 --2- 172.21.15.204:0/1864943743 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa91c09fa20 0x7fa91c09fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:36.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 1 --2- 172.21.15.204:0/1864943743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa91c0a2cd0 0x7fa91c132c00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:36.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 1 -- 172.21.15.204:0/1864943743 >> 172.21.15.204:0/1864943743 conn(0x7fa91c06e860 msgr2=0x7fa91c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:36.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 1 -- 172.21.15.204:0/1864943743 shutdown_connections 2024-09-15T01:35:36.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 1 -- 172.21.15.204:0/1864943743 wait complete. 2024-09-15T01:35:36.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 10 monclient: build_initial_monmap 2024-09-15T01:35:36.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.981+0000 7fa92479d640 10 monclient: monmap: 2024-09-15T01:35:36.001 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:36.001 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:36.001 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:36.001 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:36.001 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:36.001 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:36.001 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:36.001 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:36.001 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:36.001 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:36.001 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 1 Processor -- start 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 1 -- start start 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 10 monclient: init 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 10 monclient: _add_conns ranks=[1,8,7,2,3,4,6,0,5] 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa91c09fa20 0x7fa91c1310d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 10 monclient(hunting): picked mon.noname-f con 0x7fa91c09fa20 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa91c0a08e0 0x7fa91c11dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:36.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 10 monclient(hunting): picked mon.noname-e con 0x7fa91c0a08e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:36.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa91c0a2cd0 0x7fa91c11e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:36.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 10 monclient(hunting): picked mon.noname-i con 0x7fa91c0a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:36.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:36.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c09cb60 con 0x7fa91c09fa20 2024-09-15T01:35:36.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:36.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c121500 con 0x7fa91c0a2cd0 2024-09-15T01:35:36.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:36.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c121220 con 0x7fa91c0a08e0 2024-09-15T01:35:36.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:36.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa92479d640 10 monclient(hunting): authenticate will time out at 951.625687s 2024-09-15T01:35:36.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922d3a640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa91c0a2cd0 msgr2=0x7fa91c11e270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:36.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922d3a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa91c0a2cd0 0x7fa91c11e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa91c09fa20 0x7fa91c1310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa91c09fa20 0x7fa91c1310d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47318/0 (socket says 172.21.15.204:47318) 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 1 -- 172.21.15.204:0/1156012520 learned_addr learned my addr 172.21.15.204:0/1156012520 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 10 monclient(hunting): get_auth_request con 0x7fa91c09fa20 auth_method 0 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 10 monclient(hunting): handle_auth_done global_id 6247 payload 824 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 1 -- 172.21.15.204:0/1156012520 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa91c0a08e0 msgr2=0x7fa91c11dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 1 --2- 172.21.15.204:0/1156012520 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa91c0a08e0 0x7fa91c11dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:36.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 1 -- 172.21.15.204:0/1156012520 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa91c0a2cd0 msgr2=0x7fa91c11e270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:36.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 1 --2- 172.21.15.204:0/1156012520 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa91c0a2cd0 0x7fa91c11e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:36.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 10 monclient: _finish_hunting 0 2024-09-15T01:35:36.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 1 monclient: found mon.noname-f 2024-09-15T01:35:36.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:36.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:36.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.982+0000 7fa922539640 1 -- 172.21.15.204:0/1156012520 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c009450 con 0x7fa91c09fa20 2024-09-15T01:35:36.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa922539640 1 --2- 172.21.15.204:0/1156012520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa91c09fa20 0x7fa91c1310d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa918009360 tx=0x7fa918032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:36.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa9127fc640 1 -- 172.21.15.204:0/1156012520 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa91803b870 con 0x7fa91c09fa20 2024-09-15T01:35:36.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa9127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:36.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa9127fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:36.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa9127fc640 10 monclient: dump: 2024-09-15T01:35:36.005 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:36.006 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa9127fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa9127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa9127fc640 1 -- 172.21.15.204:0/1156012520 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91803b9d0 con 0x7fa91c09fa20 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa92479d640 5 monclient: authenticate success, global_id 6247 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa92479d640 10 monclient: _renew_subs 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa92479d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa92479d640 1 -- 172.21.15.204:0/1156012520 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa918009390 con 0x7fa91c09fa20 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa9127fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa9127fc640 1 -- 172.21.15.204:0/1156012520 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa9180313a0 con 0x7fa91c09fa20 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa9127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa9127fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa9127fc640 10 monclient: dump: 2024-09-15T01:35:36.007 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:36.008 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa920d36640 4 set_mon_vals no callback set 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa92479d640 10 monclient: _renew_subs 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa92479d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.983+0000 7fa92479d640 1 -- 172.21.15.204:0/1156012520 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c1428a0 con 0x7fa91c09fa20 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.984+0000 7fa9127fc640 1 -- 172.21.15.204:0/1156012520 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa918039040 con 0x7fa91c09fa20 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.984+0000 7fa92479d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.984+0000 7fa92479d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.984+0000 7fa92479d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.984+0000 7fa92479d640 1 -- 172.21.15.204:0/1156012520 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91c070b80 con 0x7fa91c09fa20 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.984+0000 7fa9127fc640 1 --2- 172.21.15.204:0/1156012520 >> v2:172.21.15.204:6812/28210 conn(0x7fa9000397b0 0x7fa90003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.984+0000 7fa9127fc640 1 -- 172.21.15.204:0/1156012520 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 7317+0+0 (secure 0 0 0) 0x7fa91806b6c0 con 0x7fa91c09fa20 2024-09-15T01:35:36.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.984+0000 7fa921d38640 1 --2- 172.21.15.204:0/1156012520 >> v2:172.21.15.204:6812/28210 conn(0x7fa9000397b0 0x7fa90003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:36.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.984+0000 7fa921d38640 10 monclient: get_auth_request con 0x7fa9000397b0 auth_method 0 2024-09-15T01:35:36.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.984+0000 7fa921d38640 1 --2- 172.21.15.204:0/1156012520 >> v2:172.21.15.204:6812/28210 conn(0x7fa9000397b0 0x7fa90003bc60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009cd0 tx=0x7fa90c0091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:36.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.988+0000 7fa9127fc640 1 -- 172.21.15.204:0/1156012520 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa918030b20 con 0x7fa91c09fa20 2024-09-15T01:35:36.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.988+0000 7fa9127fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:36.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:35.988+0000 7fa9127fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:36.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.149+0000 7fa92479d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:36.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.149+0000 7fa92479d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:36.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.149+0000 7fa92479d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:36.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.149+0000 7fa92479d640 1 -- 172.21.15.204:0/1156012520 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]} v 0) v1 -- 0x7fa91c067450 con 0x7fa91c09fa20 2024-09-15T01:35:36.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.149+0000 7fa9127fc640 1 -- 172.21.15.204:0/1156012520 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}]=-22 must specify exact 3 weight values v129) v1 ==== 163+0+0 (secure 0 0 0) 0x7fa918042020 con 0x7fa91c09fa20 2024-09-15T01:35:36.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.149+0000 7fa9127fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}] 2024-09-15T01:35:36.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.149+0000 7fa9127fc640 10 monclient: _finish_command 2 = mon:22 must specify exact 3 weight values 2024-09-15T01:35:36.152 INFO:tasks.workunit.client.0.smithi204.stderr:Error EINVAL: must specify exact 3 weight values 2024-09-15T01:35:36.155 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:35:36.155 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set reweight barset osd.0 .9 .9 .9 2024-09-15T01:35:36.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.232+0000 7fb267ac0640 10 monclient: get_monmap_and_config 2024-09-15T01:35:36.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.232+0000 7fb267ac0640 10 monclient: build_initial_monmap 2024-09-15T01:35:36.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.232+0000 7fb267ac0640 10 monclient: monmap: 2024-09-15T01:35:36.235 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:36.235 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:36.235 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:36.234002+0000 2024-09-15T01:35:36.235 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:36.234002+0000 2024-09-15T01:35:36.235 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:36.235 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:36.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 1 Processor -- start 2024-09-15T01:35:36.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 1 -- start start 2024-09-15T01:35:36.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 10 monclient: init 2024-09-15T01:35:36.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:36.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 10 monclient: _add_conns ranks=[0,3,5,2,10,7,8,4,1,6,9] 2024-09-15T01:35:36.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb2600a29f0 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:36.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2600a0140 0x7fb2600a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 10 monclient(hunting): picked mon.noname-g con 0x7fb2600a0140 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb260137d00 0x7fb26013a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 10 monclient(hunting): picked mon.noname-h con 0x7fb260137d00 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26004b720 con 0x7fb2600a29f0 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26009e320 con 0x7fb2600a0140 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2600a1370 con 0x7fb260137d00 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb267ac0640 10 monclient(hunting): authenticate will time out at 951.876776s 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb26605d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb260137d00 0x7fb26013a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb26605d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb260137d00 0x7fb26013a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:45978/0 (socket says 172.21.15.204:45978) 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb26605d640 1 -- 172.21.15.204:0/3215089113 learned_addr learned my addr 172.21.15.204:0/3215089113 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:36.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb26605d640 10 monclient(hunting): get_auth_request con 0x7fb260137d00 auth_method 0 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb26605d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb26605d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb26605d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.233+0000 7fb26505b640 1 --2- 172.21.15.204:0/3215089113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2600a0140 0x7fb2600a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26505b640 10 monclient(hunting): get_auth_request con 0x7fb2600a0140 auth_method 0 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26505b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26505b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26505b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26505b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:36.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26505b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26505b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 10 monclient(hunting): handle_auth_done global_id 5729 payload 274 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 1 -- 172.21.15.204:0/3215089113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2600a0140 msgr2=0x7fb2600a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 1 --2- 172.21.15.204:0/3215089113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2600a0140 0x7fb2600a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 1 -- 172.21.15.204:0/3215089113 >> v1:172.21.15.178:6789/0 conn(0x7fb2600a29f0 legacy=0x7fb26009fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 10 monclient: _finish_hunting 0 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 1 monclient: found mon.noname-h 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 1 -- 172.21.15.204:0/3215089113 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2600a2dc0 con 0x7fb260137d00 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.234+0000 7fb26605d640 1 --2- 172.21.15.204:0/3215089113 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb260137d00 0x7fb26013a0e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb258009f50 tx=0x7fb258009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=4ff3fbaef652a632 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb26485a640 1 -- 172.21.15.204:0/3215089113 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb258007880 con 0x7fb260137d00 2024-09-15T01:35:36.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb26485a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb26485a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb26485a640 10 monclient: dump: 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:36.248 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb26485a640 10 monclient: _finish_auth 0 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb26485a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 5 monclient: authenticate success, global_id 5729 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb26485a640 1 -- 172.21.15.204:0/3215089113 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25802d050 con 0x7fb260137d00 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb26485a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb26485a640 1 -- 172.21.15.204:0/3215089113 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb258030360 con 0x7fb260137d00 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb26485a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:36.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb26485a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb26485a640 10 monclient: dump: 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:36.250 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 4 set_mon_vals no callback set 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 10 monclient: shutdown 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 1 -- 172.21.15.204:0/3215089113 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb260137d00 msgr2=0x7fb26013a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 1 --2- 172.21.15.204:0/3215089113 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb260137d00 0x7fb26013a0e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb258009f50 tx=0x7fb258009840 comp rx=0 tx=0).stop 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 1 -- 172.21.15.204:0/3215089113 shutdown_connections 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 1 --2- 172.21.15.204:0/3215089113 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb260137d00 0x7fb26013a0e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 1 --2- 172.21.15.204:0/3215089113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2600a0140 0x7fb2600a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:36.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 1 -- 172.21.15.204:0/3215089113 >> 172.21.15.204:0/3215089113 conn(0x7fb26006e860 msgr2=0x7fb26006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 1 -- 172.21.15.204:0/3215089113 shutdown_connections 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.235+0000 7fb267ac0640 1 -- 172.21.15.204:0/3215089113 wait complete. 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 10 monclient: build_initial_monmap 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 10 monclient: monmap: 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:36.252 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 1 Processor -- start 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 1 -- start start 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 10 monclient: init 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 10 monclient: _add_conns ranks=[1,0,7,8,6,4,5,2,3] 2024-09-15T01:35:36.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2600a0140 0x7fb26009bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 10 monclient(hunting): picked mon.noname-f con 0x7fb2600a0140 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2600a29f0 0x7fb26012ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 10 monclient(hunting): picked mon.noname-a con 0x7fb2600a29f0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb260137d00 0x7fb26012f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 10 monclient(hunting): picked mon.noname-i con 0x7fb260137d00 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.236+0000 7fb267ac0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb267ac0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb260130820 con 0x7fb2600a29f0 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb267ac0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb267ac0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26009cad0 con 0x7fb2600a0140 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb267ac0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb267ac0640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb260131a60 con 0x7fb260137d00 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb267ac0640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb267ac0640 10 monclient(hunting): authenticate will time out at 951.880015s 2024-09-15T01:35:36.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26605d640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb260137d00 msgr2=0x7fb26012f4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26605d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb260137d00 0x7fb26012f4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2600a0140 0x7fb26009bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2600a0140 0x7fb26009bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47320/0 (socket says 172.21.15.204:47320) 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 1 -- 172.21.15.204:0/3918881354 learned_addr learned my addr 172.21.15.204:0/3918881354 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 10 monclient(hunting): get_auth_request con 0x7fb2600a0140 auth_method 0 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 10 monclient(hunting): handle_auth_done global_id 6256 payload 824 2024-09-15T01:35:36.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26505b640 1 --2- 172.21.15.204:0/3918881354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2600a29f0 0x7fb26012ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 1 -- 172.21.15.204:0/3918881354 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb260137d00 msgr2=0x7fb26012f4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 1 --2- 172.21.15.204:0/3918881354 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb260137d00 0x7fb26012f4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 1 -- 172.21.15.204:0/3918881354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2600a29f0 msgr2=0x7fb26012ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 1 --2- 172.21.15.204:0/3918881354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2600a29f0 0x7fb26012ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 10 monclient: _finish_hunting 0 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 1 monclient: found mon.noname-f 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 1 -- 172.21.15.204:0/3918881354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb258009ae0 con 0x7fb2600a0140 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb26585c640 1 --2- 172.21.15.204:0/3918881354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2600a0140 0x7fb26009bd60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00c560 tx=0x7fb25c007eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb255ffb640 1 -- 172.21.15.204:0/3918881354 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb25c011a10 con 0x7fb2600a0140 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb255ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:36.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb255ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb255ffb640 10 monclient: dump: 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:36.257 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb255ffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb255ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb255ffb640 1 -- 172.21.15.204:0/3918881354 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c011bd0 con 0x7fb2600a0140 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb267ac0640 5 monclient: authenticate success, global_id 6256 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb267ac0640 10 monclient: _renew_subs 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb267ac0640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb267ac0640 1 -- 172.21.15.204:0/3918881354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26009c780 con 0x7fb2600a0140 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb255ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:36.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb255ffb640 1 -- 172.21.15.204:0/3918881354 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb25c011dc0 con 0x7fb2600a0140 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb255ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb255ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb255ffb640 10 monclient: dump: 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:36.259 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb267ac0640 10 monclient: _renew_subs 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.237+0000 7fb257fff640 4 set_mon_vals no callback set 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.238+0000 7fb267ac0640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.238+0000 7fb267ac0640 1 -- 172.21.15.204:0/3918881354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb260137670 con 0x7fb2600a0140 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.238+0000 7fb255ffb640 1 -- 172.21.15.204:0/3918881354 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb25c011590 con 0x7fb2600a0140 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.238+0000 7fb255ffb640 1 --2- 172.21.15.204:0/3918881354 >> v2:172.21.15.204:6812/28210 conn(0x7fb244039710 0x7fb24403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.238+0000 7fb255ffb640 1 -- 172.21.15.204:0/3918881354 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 7317+0+0 (secure 0 0 0) 0x7fb25c04ada0 con 0x7fb2600a0140 2024-09-15T01:35:36.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.238+0000 7fb26505b640 1 --2- 172.21.15.204:0/3918881354 >> v2:172.21.15.204:6812/28210 conn(0x7fb244039710 0x7fb24403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:36.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.238+0000 7fb26505b640 10 monclient: get_auth_request con 0x7fb244039710 auth_method 0 2024-09-15T01:35:36.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.239+0000 7fb267ac0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:36.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.239+0000 7fb267ac0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:36.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.239+0000 7fb267ac0640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:36.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.239+0000 7fb267ac0640 1 -- 172.21.15.204:0/3918881354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb260070cc0 con 0x7fb2600a0140 2024-09-15T01:35:36.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.239+0000 7fb26505b640 1 --2- 172.21.15.204:0/3918881354 >> v2:172.21.15.204:6812/28210 conn(0x7fb244039710 0x7fb24403bbc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb250009360 tx=0x7fb250008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:36.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.243+0000 7fb255ffb640 1 -- 172.21.15.204:0/3918881354 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb25c017030 con 0x7fb2600a0140 2024-09-15T01:35:36.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.243+0000 7fb255ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:36.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.243+0000 7fb255ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:36.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.405+0000 7fb267ac0640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-09-15T01:35:36.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.405+0000 7fb267ac0640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-09-15T01:35:36.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.405+0000 7fb267ac0640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:36.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.405+0000 7fb267ac0640 1 -- 172.21.15.204:0/3918881354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]} v 0) v1 -- 0x7fb260131c20 con 0x7fb2600a0140 2024-09-15T01:35:36.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.634+0000 7fb255ffb640 1 -- 172.21.15.204:0/3918881354 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}]=0 v130) v1 ==== 139+0+0 (secure 0 0 0) 0x7fb25c017020 con 0x7fb2600a0140 2024-09-15T01:35:36.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.634+0000 7fb255ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-09-15T01:35:36.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.634+0000 7fb255ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:36.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.738+0000 7fb267ac0640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-09-15T01:35:36.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.738+0000 7fb267ac0640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-09-15T01:35:36.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.738+0000 7fb267ac0640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:36.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:36.738+0000 7fb267ac0640 1 -- 172.21.15.204:0/3918881354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]} v 0) v1 -- 0x7fb2600008d0 con 0x7fb2600a0140 2024-09-15T01:35:37.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.635+0000 7fb255ffb640 1 -- 172.21.15.204:0/3918881354 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}]=0 v131) v1 ==== 139+0+0 (secure 0 0 0) 0x7fb25c017020 con 0x7fb2600a0140 2024-09-15T01:35:37.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.635+0000 7fb255ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-09-15T01:35:37.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.635+0000 7fb255ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.639+0000 7fb267ac0640 1 -- 172.21.15.204:0/3918881354 >> v2:172.21.15.204:6812/28210 conn(0x7fb244039710 msgr2=0x7fb24403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.639+0000 7fb267ac0640 1 --2- 172.21.15.204:0/3918881354 >> v2:172.21.15.204:6812/28210 conn(0x7fb244039710 0x7fb24403bbc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb250009360 tx=0x7fb250008040 comp rx=0 tx=0).stop 2024-09-15T01:35:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.639+0000 7fb267ac0640 10 monclient: shutdown 2024-09-15T01:35:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.639+0000 7fb267ac0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.639+0000 7fb267ac0640 1 -- 172.21.15.204:0/3918881354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2600a0140 msgr2=0x7fb26009bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.639+0000 7fb267ac0640 1 --2- 172.21.15.204:0/3918881354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2600a0140 0x7fb26009bd60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00c560 tx=0x7fb25c007eb0 comp rx=0 tx=0).stop 2024-09-15T01:35:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.640+0000 7fb267ac0640 1 -- 172.21.15.204:0/3918881354 shutdown_connections 2024-09-15T01:35:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.640+0000 7fb267ac0640 1 --2- 172.21.15.204:0/3918881354 >> v2:172.21.15.204:6812/28210 conn(0x7fb244039710 0x7fb24403bbc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.640+0000 7fb267ac0640 1 --2- 172.21.15.204:0/3918881354 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb260137d00 0x7fb26012f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.640+0000 7fb267ac0640 1 --2- 172.21.15.204:0/3918881354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2600a29f0 0x7fb26012ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:37.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.640+0000 7fb267ac0640 1 --2- 172.21.15.204:0/3918881354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2600a0140 0x7fb26009bd60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.640+0000 7fb267ac0640 1 -- 172.21.15.204:0/3918881354 >> 172.21.15.204:0/3918881354 conn(0x7fb26006e860 msgr2=0x7fb26009e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.640+0000 7fb267ac0640 1 -- 172.21.15.204:0/3918881354 shutdown_connections 2024-09-15T01:35:37.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.640+0000 7fb267ac0640 1 -- 172.21.15.204:0/3918881354 wait complete. 2024-09-15T01:35:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set ls 2024-09-15T01:35:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -c fooset 2024-09-15T01:35:37.654 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -q 1 2024-09-15T01:35:37.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.731+0000 7f94dad5d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:37.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.731+0000 7f94dad5d640 10 monclient: build_initial_monmap 2024-09-15T01:35:37.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.731+0000 7f94dad5d640 10 monclient: monmap: 2024-09-15T01:35:37.734 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:37.734 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:37.734 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:37.732941+0000 2024-09-15T01:35:37.734 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:37.732941+0000 2024-09-15T01:35:37.734 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:37.734 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:37.734 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:37.734 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:37.735 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:37.735 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:37.735 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:37.735 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:37.735 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:37.735 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:37.735 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:37.735 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:37.735 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:37.735 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:37.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 1 Processor -- start 2024-09-15T01:35:37.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 1 -- start start 2024-09-15T01:35:37.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 10 monclient: init 2024-09-15T01:35:37.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:37.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 10 monclient: _add_conns ranks=[5,0,3,9,4,10,7,6,8,2,1] 2024-09-15T01:35:37.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f94d40a2e30 0x7f94d4133280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:37.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 10 monclient(hunting): picked mon.noname-h con 0x7f94d40a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:37.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f94d409fd80 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:37.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d40a0150 0x7f94d40a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:37.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 10 monclient(hunting): picked mon.noname-g con 0x7f94d40a0150 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:37.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d404b720 con 0x7f94d409fd80 2024-09-15T01:35:37.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:37.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d409e290 con 0x7f94d40a0150 2024-09-15T01:35:37.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:37.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40a2450 con 0x7f94d40a2e30 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94dad5d640 10 monclient(hunting): authenticate will time out at 953.375906s 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94d8af9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f94d40a2e30 0x7f94d4133280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94d8af9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f94d40a2e30 0x7f94d4133280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:58930/0 (socket says 172.21.15.204:58930) 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94d92fa640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d40a0150 0x7f94d40a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94d8af9640 1 -- 172.21.15.204:0/2214195379 learned_addr learned my addr 172.21.15.204:0/2214195379 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94d8af9640 10 monclient(hunting): get_auth_request con 0x7f94d40a2e30 auth_method 0 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94d8af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94d8af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.732+0000 7f94d8af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d92fa640 10 monclient(hunting): get_auth_request con 0x7f94d40a0150 auth_method 0 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d92fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:37.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d92fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d92fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d92fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d92fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d92fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 10 monclient(hunting): handle_auth_done global_id 5738 payload 274 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 1 -- 172.21.15.204:0/2214195379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d40a0150 msgr2=0x7f94d40a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 1 --2- 172.21.15.204:0/2214195379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d40a0150 0x7f94d40a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 1 -- 172.21.15.204:0/2214195379 >> v1:172.21.15.178:6789/0 conn(0x7f94d409fd80 legacy=0x7f94d40a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 10 monclient: _finish_hunting 0 2024-09-15T01:35:37.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 1 monclient: found mon.noname-h 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 1 -- 172.21.15.204:0/2214195379 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d40a3200 con 0x7f94d40a2e30 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d92fa640 1 --2- 172.21.15.204:0/2214195379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d40a0150 0x7f94d40a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.733+0000 7f94d8af9640 1 --2- 172.21.15.204:0/2214195379 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f94d40a2e30 0x7f94d4133280 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009940 tx=0x7f94c80091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=aefce0b17c16887e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94d37fe640 1 -- 172.21.15.204:0/2214195379 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94c802ac70 con 0x7f94d40a2e30 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94d37fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94d37fe640 10 monclient: dump: 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:37.750 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94d37fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:37.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94d37fe640 1 -- 172.21.15.204:0/2214195379 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c802adf0 con 0x7f94d40a2e30 2024-09-15T01:35:37.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94d37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:37.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94d37fe640 1 -- 172.21.15.204:0/2214195379 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94c8031ca0 con 0x7f94d40a2e30 2024-09-15T01:35:37.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 5 monclient: authenticate success, global_id 5738 2024-09-15T01:35:37.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:37.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 4 set_mon_vals no callback set 2024-09-15T01:35:37.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:37.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94d37fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:37.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94d37fe640 10 monclient: dump: 2024-09-15T01:35:37.752 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:37.752 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:37.752 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:37.752 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 10 monclient: shutdown 2024-09-15T01:35:37.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 1 -- 172.21.15.204:0/2214195379 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f94d40a2e30 msgr2=0x7f94d4133280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 1 --2- 172.21.15.204:0/2214195379 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f94d40a2e30 0x7f94d4133280 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009940 tx=0x7f94c80091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 1 -- 172.21.15.204:0/2214195379 shutdown_connections 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 1 --2- 172.21.15.204:0/2214195379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d40a0150 0x7f94d40a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 1 --2- 172.21.15.204:0/2214195379 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f94d40a2e30 0x7f94d4133280 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 1 -- 172.21.15.204:0/2214195379 >> 172.21.15.204:0/2214195379 conn(0x7f94d406e860 msgr2=0x7f94d406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 1 -- 172.21.15.204:0/2214195379 shutdown_connections 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.734+0000 7f94dad5d640 1 -- 172.21.15.204:0/2214195379 wait complete. 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.735+0000 7f94dad5d640 10 monclient: build_initial_monmap 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.735+0000 7f94dad5d640 10 monclient: monmap: 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:37.754 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.735+0000 7f94dad5d640 1 Processor -- start 2024-09-15T01:35:37.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 1 -- start start 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 10 monclient: init 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 10 monclient: _add_conns ranks=[7,2,4,0,3,1,6,8,5] 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f94d409fd80 0x7f94d41420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 10 monclient(hunting): picked mon.noname-i con 0x7f94d409fd80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94d40a0150 0x7f94d412fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 10 monclient(hunting): picked mon.noname-b con 0x7f94d40a0150 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f94d409c480 0x7f94d4099d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 10 monclient(hunting): picked mon.noname-c con 0x7f94d409c480 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d409d4c0 con 0x7f94d40a0150 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:37.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94d411d070 con 0x7f94d409c480 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d8af9640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f94d409fd80 msgr2=0x7f94d41420c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f94d4142f00 con 0x7f94d409fd80 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94dad5d640 10 monclient(hunting): authenticate will time out at 953.379476s 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d8af9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f94d409fd80 0x7f94d41420c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d3fff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94d40a0150 msgr2=0x7f94d412fc70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94d40a0150 0x7f94d412fc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d92fa640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f94d409c480 0x7f94d4099d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d92fa640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f94d409c480 0x7f94d4099d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:58574/0 (socket says 172.21.15.204:58574) 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d92fa640 1 -- 172.21.15.204:0/945730683 learned_addr learned my addr 172.21.15.204:0/945730683 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d92fa640 10 monclient(hunting): get_auth_request con 0x7f94d409c480 auth_method 0 2024-09-15T01:35:37.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d92fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d92fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d92fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d92fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d92fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.736+0000 7f94d92fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d92fa640 10 monclient(hunting): handle_auth_done global_id 5611 payload 824 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d92fa640 1 -- 172.21.15.204:0/945730683 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f94d409fd80 msgr2=0x7f94d41420c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d92fa640 1 --2- 172.21.15.204:0/945730683 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f94d409fd80 0x7f94d41420c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d92fa640 1 -- 172.21.15.204:0/945730683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94d40a0150 msgr2=0x7f94d412fc70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d92fa640 1 --2- 172.21.15.204:0/945730683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94d40a0150 0x7f94d412fc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d92fa640 10 monclient: _finish_hunting 0 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d92fa640 1 monclient: found mon.noname-c 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d92fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:37.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d92fa640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d92fa640 1 -- 172.21.15.204:0/945730683 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc009a20 con 0x7f94d409c480 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d92fa640 1 --2- 172.21.15.204:0/945730683 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f94d409c480 0x7f94d4099d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009f50 tx=0x7f94cc00b040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d0ff9640 1 -- 172.21.15.204:0/945730683 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94cc009c80 con 0x7f94d409c480 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d0ff9640 10 monclient: dump: 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d0ff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94dad5d640 5 monclient: authenticate success, global_id 5611 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d0ff9640 1 -- 172.21.15.204:0/945730683 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc00f040 con 0x7f94d409c480 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94dad5d640 10 monclient: _renew_subs 2024-09-15T01:35:37.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94dad5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94dad5d640 1 -- 172.21.15.204:0/945730683 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c8009450 con 0x7f94d409c480 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d0ff9640 1 -- 172.21.15.204:0/945730683 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94cc00cdf0 con 0x7f94d409c480 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d2ffd640 4 set_mon_vals no callback set 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94d0ff9640 10 monclient: dump: 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:37.761 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94dad5d640 10 monclient: _renew_subs 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94dad5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.737+0000 7f94dad5d640 1 -- 172.21.15.204:0/945730683 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d411d350 con 0x7f94d409c480 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.738+0000 7f94d0ff9640 1 -- 172.21.15.204:0/945730683 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f94cc003020 con 0x7f94d409c480 2024-09-15T01:35:37.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.738+0000 7f94dad5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:37.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.738+0000 7f94dad5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:37.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.738+0000 7f94dad5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:37.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.738+0000 7f94dad5d640 1 -- 172.21.15.204:0/945730683 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d4070d80 con 0x7f94d409c480 2024-09-15T01:35:37.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.738+0000 7f94d0ff9640 1 --2- 172.21.15.204:0/945730683 >> v2:172.21.15.204:6812/28210 conn(0x7f94b80397b0 0x7f94b803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:37.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.738+0000 7f94d0ff9640 1 -- 172.21.15.204:0/945730683 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 7317+0+0 (secure 0 0 0) 0x7f94cc04c970 con 0x7f94d409c480 2024-09-15T01:35:37.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.738+0000 7f94d8af9640 1 --2- 172.21.15.204:0/945730683 >> v2:172.21.15.204:6812/28210 conn(0x7f94b80397b0 0x7f94b803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:37.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.738+0000 7f94d8af9640 10 monclient: get_auth_request con 0x7f94b80397b0 auth_method 0 2024-09-15T01:35:37.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.739+0000 7f94d8af9640 1 --2- 172.21.15.204:0/945730683 >> v2:172.21.15.204:6812/28210 conn(0x7f94b80397b0 0x7f94b803bc60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009420 tx=0x7f94c8032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:37.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.743+0000 7f94d0ff9640 1 -- 172.21.15.204:0/945730683 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f94cc0133e0 con 0x7f94d409c480 2024-09-15T01:35:37.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.743+0000 7f94d0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:37.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.743+0000 7f94d0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:37.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.906+0000 7f94dad5d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:37.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.906+0000 7f94dad5d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:37.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.906+0000 7f94dad5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:37.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.906+0000 7f94dad5d640 1 -- 172.21.15.204:0/945730683 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f94d409dfb0 con 0x7f94d409c480 2024-09-15T01:35:37.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.907+0000 7f94d0ff9640 1 -- 172.21.15.204:0/945730683 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v131) v1 ==== 71+0+14 (secure 0 0 0) 0x7f94cc018030 con 0x7f94d409c480 2024-09-15T01:35:37.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.907+0000 7f94d0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:37.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:37.907+0000 7f94d0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:38.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.006+0000 7f94dad5d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:38.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.006+0000 7f94dad5d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:38.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.006+0000 7f94dad5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:38.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.006+0000 7f94dad5d640 1 -- 172.21.15.204:0/945730683 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f94d4132d80 con 0x7f94d409c480 2024-09-15T01:35:38.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.006+0000 7f94d0ff9640 1 -- 172.21.15.204:0/945730683 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v131) v1 ==== 71+0+14 (secure 0 0 0) 0x7f94cc018020 con 0x7f94d409c480 2024-09-15T01:35:38.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.006+0000 7f94d0ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:38.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.006+0000 7f94d0ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:38.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 1 -- 172.21.15.204:0/945730683 >> v2:172.21.15.204:6812/28210 conn(0x7f94b80397b0 msgr2=0x7f94b803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:38.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 1 --2- 172.21.15.204:0/945730683 >> v2:172.21.15.204:6812/28210 conn(0x7f94b80397b0 0x7f94b803bc60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009420 tx=0x7f94c8032000 comp rx=0 tx=0).stop 2024-09-15T01:35:38.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 10 monclient: shutdown 2024-09-15T01:35:38.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:38.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 1 -- 172.21.15.204:0/945730683 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f94d409c480 msgr2=0x7f94d4099d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:38.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 1 --2- 172.21.15.204:0/945730683 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f94d409c480 0x7f94d4099d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009f50 tx=0x7f94cc00b040 comp rx=0 tx=0).stop 2024-09-15T01:35:38.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 1 -- 172.21.15.204:0/945730683 shutdown_connections 2024-09-15T01:35:38.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 1 --2- 172.21.15.204:0/945730683 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f94d409c480 0x7f94d4099d90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:38.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 1 --2- 172.21.15.204:0/945730683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94d40a0150 0x7f94d412fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:38.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 1 --2- 172.21.15.204:0/945730683 >> v2:172.21.15.204:6812/28210 conn(0x7f94b80397b0 0x7f94b803bc60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:38.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 1 --2- 172.21.15.204:0/945730683 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f94d409fd80 0x7f94d41420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:38.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 1 -- 172.21.15.204:0/945730683 >> 172.21.15.204:0/945730683 conn(0x7f94d406e860 msgr2=0x7f94d4135d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:38.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 1 -- 172.21.15.204:0/945730683 shutdown_connections 2024-09-15T01:35:38.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.008+0000 7f94dad5d640 1 -- 172.21.15.204:0/945730683 wait complete. 2024-09-15T01:35:38.019 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set rm fooset 2024-09-15T01:35:38.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.090+0000 7fcc02d9d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:38.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.090+0000 7fcc02d9d640 10 monclient: build_initial_monmap 2024-09-15T01:35:38.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.091+0000 7fcc02d9d640 10 monclient: monmap: 2024-09-15T01:35:38.093 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:38.094 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:38.094 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:38.092453+0000 2024-09-15T01:35:38.094 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:38.092453+0000 2024-09-15T01:35:38.094 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:38.094 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:38.094 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:38.094 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:38.094 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:38.094 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:38.094 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:38.094 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:38.094 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:38.095 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:38.095 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:38.095 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:38.095 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:38.095 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:38.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.091+0000 7fcc02d9d640 1 Processor -- start 2024-09-15T01:35:38.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.091+0000 7fcc02d9d640 1 -- start start 2024-09-15T01:35:38.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.091+0000 7fcc02d9d640 10 monclient: init 2024-09-15T01:35:38.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:38.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 10 monclient: _add_conns ranks=[4,7,9,5,0,10,3,8,6,2,1] 2024-09-15T01:35:38.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 0x7fcbfc132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:38.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 10 monclient(hunting): picked mon.noname-c con 0x7fcbfc0a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:38.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcbfc09fb80 0x7fcbfc09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:38.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 10 monclient(hunting): picked mon.noname-i con 0x7fcbfc09fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:38.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcbfc0a0a40 0x7fcbfc0a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 10 monclient(hunting): picked mon.noname-a con 0x7fcbfc0a0a40 addr v2:172.21.15.178:3300/0 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbfc09e6c0 con 0x7fcbfc0a0a40 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0a04b0 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0a3200 con 0x7fcbfc09fb80 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc02d9d640 10 monclient(hunting): authenticate will time out at 953.735277s 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcbfbfff640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcbfc09fb80 msgr2=0x7fcbfc09ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcbfbfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcbfc09fb80 0x7fcbfc09ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 0x7fcbfc132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:38.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 0x7fcbfc132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:58576/0 (socket says 172.21.15.204:58576) 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 1 -- 172.21.15.204:0/2509829459 learned_addr learned my addr 172.21.15.204:0/2509829459 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 10 monclient(hunting): get_auth_request con 0x7fcbfc0a2e30 auth_method 0 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc0133a640 1 --2- 172.21.15.204:0/2509829459 >> v2:172.21.15.178:3300/0 conn(0x7fcbfc0a0a40 0x7fcbfc0a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc0133a640 10 monclient(hunting): get_auth_request con 0x7fcbfc0a0a40 auth_method 0 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc0133a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc0133a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc0133a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:38.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc0133a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc0133a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc0133a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 10 monclient(hunting): handle_auth_done global_id 5620 payload 274 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 1 -- 172.21.15.204:0/2509829459 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcbfc09fb80 msgr2=0x7fcbfc09ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 1 --2- 172.21.15.204:0/2509829459 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcbfc09fb80 0x7fcbfc09ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 1 -- 172.21.15.204:0/2509829459 >> v2:172.21.15.178:3300/0 conn(0x7fcbfc0a0a40 msgr2=0x7fcbfc0a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 1 --2- 172.21.15.204:0/2509829459 >> v2:172.21.15.178:3300/0 conn(0x7fcbfc0a0a40 0x7fcbfc0a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 10 monclient: _finish_hunting 0 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 1 monclient: found mon.noname-c 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.092+0000 7fcc00b39640 1 -- 172.21.15.204:0/2509829459 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc04b720 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcc00b39640 1 --2- 172.21.15.204:0/2509829459 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 0x7fcbfc132a60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009910 tx=0x7fcbec0091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8ba19ec990fb5d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcbfb7fe640 1 -- 172.21.15.204:0/2509829459 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbec02acb0 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcbfb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcbfb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcbfb7fe640 10 monclient: dump: 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:38.108 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:38.109 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:38.109 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:38.109 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:38.110 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:38.110 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:38.110 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:38.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcbfb7fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:38.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcbfb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:38.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcbfb7fe640 1 -- 172.21.15.204:0/2509829459 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec02ae30 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcc02d9d640 5 monclient: authenticate success, global_id 5620 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcc02d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcbfb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcc02d9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcc02d9d640 4 set_mon_vals no callback set 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcbfb7fe640 1 -- 172.21.15.204:0/2509829459 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbec031ca0 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcbfb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcbfb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.093+0000 7fcbfb7fe640 10 monclient: dump: 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:38.111 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:38.112 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:38.112 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:38.112 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:38.112 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:38.112 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:38.112 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:38.112 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:38.112 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:38.112 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:38.112 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:38.112 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:38.112 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 10 monclient: shutdown 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 1 -- 172.21.15.204:0/2509829459 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 msgr2=0x7fcbfc132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 1 --2- 172.21.15.204:0/2509829459 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 0x7fcbfc132a60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009910 tx=0x7fcbec0091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 1 -- 172.21.15.204:0/2509829459 shutdown_connections 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 1 --2- 172.21.15.204:0/2509829459 >> v2:172.21.15.178:3300/0 conn(0x7fcbfc0a0a40 0x7fcbfc0a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 1 --2- 172.21.15.204:0/2509829459 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcbfc09fb80 0x7fcbfc09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 1 --2- 172.21.15.204:0/2509829459 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 0x7fcbfc132a60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 1 -- 172.21.15.204:0/2509829459 >> 172.21.15.204:0/2509829459 conn(0x7fcbfc06e860 msgr2=0x7fcbfc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 1 -- 172.21.15.204:0/2509829459 shutdown_connections 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 1 -- 172.21.15.204:0/2509829459 wait complete. 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 10 monclient: build_initial_monmap 2024-09-15T01:35:38.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.094+0000 7fcc02d9d640 10 monclient: monmap: 2024-09-15T01:35:38.114 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:38.114 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:38.114 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:38.114 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:38.114 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:38.114 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:38.114 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:38.114 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:38.114 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:38.114 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:38.114 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:38.114 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 1 Processor -- start 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 1 -- start start 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 10 monclient: init 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 10 monclient: _add_conns ranks=[7,8,4,3,2,1,0,5,6] 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcbfc09fb80 0x7fcbfc09b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 10 monclient(hunting): picked mon.noname-i con 0x7fcbfc09fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcbfc0a0a40 0x7fcbfc12e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 10 monclient(hunting): picked mon.noname-e con 0x7fcbfc0a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:38.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 0x7fcbfc11c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:38.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 10 monclient(hunting): picked mon.noname-c con 0x7fcbfc0a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:38.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:38.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc11ebf0 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:38.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc11d9b0 con 0x7fcbfc09fb80 2024-09-15T01:35:38.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:38.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc11e910 con 0x7fcbfc0a0a40 2024-09-15T01:35:38.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:38.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc02d9d640 10 monclient(hunting): authenticate will time out at 953.738873s 2024-09-15T01:35:38.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc00b39640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcbfc09fb80 msgr2=0x7fcbfc09b710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:38.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.095+0000 7fcc00b39640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcbfc09fb80 0x7fcbfc09b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbfbfff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcbfc0a0a40 0x7fcbfc12e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 0x7fcbfc11c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbfbfff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcbfc0a0a40 0x7fcbfc12e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:33078/0 (socket says 172.21.15.204:33078) 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbfbfff640 1 -- 172.21.15.204:0/3148700047 learned_addr learned my addr 172.21.15.204:0/3148700047 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbfbfff640 10 monclient(hunting): get_auth_request con 0x7fcbfc0a0a40 auth_method 0 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbfbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbfbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbfbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 10 monclient(hunting): get_auth_request con 0x7fcbfc0a2e30 auth_method 0 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:38.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:38.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:38.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:38.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbfbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:38.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbfbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:38.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbfbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:38.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 10 monclient(hunting): handle_auth_done global_id 5629 payload 824 2024-09-15T01:35:38.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 1 -- 172.21.15.204:0/3148700047 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcbfc0a0a40 msgr2=0x7fcbfc12e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:38.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 1 --2- 172.21.15.204:0/3148700047 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcbfc0a0a40 0x7fcbfc12e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:38.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 1 -- 172.21.15.204:0/3148700047 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcbfc09fb80 msgr2=0x7fcbfc09b710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:38.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 1 --2- 172.21.15.204:0/3148700047 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcbfc09fb80 0x7fcbfc09b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:38.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 10 monclient: _finish_hunting 0 2024-09-15T01:35:38.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 1 monclient: found mon.noname-c 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 1 -- 172.21.15.204:0/3148700047 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec009450 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc0133a640 1 --2- 172.21.15.204:0/3148700047 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 0x7fcbfc11c6b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009e50 tx=0x7fcbf400b040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbf8ff9640 1 -- 172.21.15.204:0/3148700047 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbf400cd40 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbf8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbf8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbf8ff9640 10 monclient: dump: 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:38.119 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbf8ff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:38.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbf8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:38.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbf8ff9640 1 -- 172.21.15.204:0/3148700047 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf400f040 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc02d9d640 5 monclient: authenticate success, global_id 5629 2024-09-15T01:35:38.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbf8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:38.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbf8ff9640 1 -- 172.21.15.204:0/3148700047 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbf4014d80 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc02d9d640 10 monclient: _renew_subs 2024-09-15T01:35:38.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc02d9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:38.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcc02d9d640 1 -- 172.21.15.204:0/3148700047 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbfc11eed0 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbf8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:38.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbf8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:38.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbfaffd640 4 set_mon_vals no callback set 2024-09-15T01:35:38.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.096+0000 7fcbf8ff9640 10 monclient: dump: 2024-09-15T01:35:38.121 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:38.122 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.097+0000 7fcc02d9d640 10 monclient: _renew_subs 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.097+0000 7fcc02d9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.097+0000 7fcc02d9d640 1 -- 172.21.15.204:0/3148700047 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbfc133f40 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.097+0000 7fcc02d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.097+0000 7fcc02d9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.097+0000 7fcc02d9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.097+0000 7fcc02d9d640 1 -- 172.21.15.204:0/3148700047 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbfc09d6d0 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.097+0000 7fcbf8ff9640 1 -- 172.21.15.204:0/3148700047 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fcbf4014480 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.098+0000 7fcbf8ff9640 1 --2- 172.21.15.204:0/3148700047 >> v2:172.21.15.204:6812/28210 conn(0x7fcbe80397b0 0x7fcbe803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.098+0000 7fcbf8ff9640 1 -- 172.21.15.204:0/3148700047 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 7317+0+0 (secure 0 0 0) 0x7fcbf404ca10 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.098+0000 7fcc00b39640 1 --2- 172.21.15.204:0/3148700047 >> v2:172.21.15.204:6812/28210 conn(0x7fcbe80397b0 0x7fcbe803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:38.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.098+0000 7fcc00b39640 10 monclient: get_auth_request con 0x7fcbe80397b0 auth_method 0 2024-09-15T01:35:38.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.098+0000 7fcc00b39640 1 --2- 172.21.15.204:0/3148700047 >> v2:172.21.15.204:6812/28210 conn(0x7fcbe80397b0 0x7fcbe803bc60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcbec034000 tx=0x7fcbec032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:38.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.102+0000 7fcbf8ff9640 1 -- 172.21.15.204:0/3148700047 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcbf4011aa0 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.102+0000 7fcbf8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:38.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.102+0000 7fcbf8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:38.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.265+0000 7fcc02d9d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-09-15T01:35:38.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.265+0000 7fcc02d9d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-09-15T01:35:38.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.265+0000 7fcc02d9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:38.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.265+0000 7fcc02d9d640 1 -- 172.21.15.204:0/3148700047 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush weight-set rm", "pool": "fooset"} v 0) v1 -- 0x7fcbfc09c120 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.652+0000 7fcbf8ff9640 1 -- 172.21.15.204:0/3148700047 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "fooset"}]=0 v132) v1 ==== 89+0+0 (secure 0 0 0) 0x7fcbf4018020 con 0x7fcbfc0a2e30 2024-09-15T01:35:38.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.652+0000 7fcbf8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-09-15T01:35:38.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.652+0000 7fcbf8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:38.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.754+0000 7fcc02d9d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-09-15T01:35:38.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.754+0000 7fcc02d9d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-09-15T01:35:38.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.754+0000 7fcc02d9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:38.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:38.754+0000 7fcc02d9d640 1 -- 172.21.15.204:0/3148700047 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush weight-set rm", "pool": "fooset"} v 0) v1 -- 0x7fcbfc070cc0 con 0x7fcbfc0a2e30 2024-09-15T01:35:39.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.657+0000 7fcbf8ff9640 1 -- 172.21.15.204:0/3148700047 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "fooset"}]=0 v133) v1 ==== 89+0+0 (secure 0 0 0) 0x7fcbf40112c0 con 0x7fcbfc0a2e30 2024-09-15T01:35:39.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.657+0000 7fcbf8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-09-15T01:35:39.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.657+0000 7fcbf8ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:39.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.660+0000 7fcc02d9d640 1 -- 172.21.15.204:0/3148700047 >> v2:172.21.15.204:6812/28210 conn(0x7fcbe80397b0 msgr2=0x7fcbe803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:39.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.660+0000 7fcc02d9d640 1 --2- 172.21.15.204:0/3148700047 >> v2:172.21.15.204:6812/28210 conn(0x7fcbe80397b0 0x7fcbe803bc60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcbec034000 tx=0x7fcbec032000 comp rx=0 tx=0).stop 2024-09-15T01:35:39.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.660+0000 7fcc02d9d640 10 monclient: shutdown 2024-09-15T01:35:39.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.660+0000 7fcc02d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:39.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.660+0000 7fcc02d9d640 1 -- 172.21.15.204:0/3148700047 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 msgr2=0x7fcbfc11c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:39.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.661+0000 7fcc02d9d640 1 --2- 172.21.15.204:0/3148700047 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 0x7fcbfc11c6b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009e50 tx=0x7fcbf400b040 comp rx=0 tx=0).stop 2024-09-15T01:35:39.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.661+0000 7fcc02d9d640 1 -- 172.21.15.204:0/3148700047 shutdown_connections 2024-09-15T01:35:39.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.661+0000 7fcc02d9d640 1 --2- 172.21.15.204:0/3148700047 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcbfc0a2e30 0x7fcbfc11c6b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:39.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.661+0000 7fcc02d9d640 1 --2- 172.21.15.204:0/3148700047 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcbfc0a0a40 0x7fcbfc12e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:39.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.661+0000 7fcc02d9d640 1 --2- 172.21.15.204:0/3148700047 >> v2:172.21.15.204:6812/28210 conn(0x7fcbe80397b0 0x7fcbe803bc60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:39.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.661+0000 7fcc02d9d640 1 --2- 172.21.15.204:0/3148700047 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcbfc09fb80 0x7fcbfc09b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:39.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.661+0000 7fcc02d9d640 1 -- 172.21.15.204:0/3148700047 >> 172.21.15.204:0/3148700047 conn(0x7fcbfc06e860 msgr2=0x7fcbfc09ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:39.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.661+0000 7fcc02d9d640 1 -- 172.21.15.204:0/3148700047 shutdown_connections 2024-09-15T01:35:39.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.661+0000 7fcc02d9d640 1 -- 172.21.15.204:0/3148700047 wait complete. 2024-09-15T01:35:39.674 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set ls 2024-09-15T01:35:39.675 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -c fooset 2024-09-15T01:35:39.675 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -q 0 2024-09-15T01:35:39.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.750+0000 7f2804446640 10 monclient: get_monmap_and_config 2024-09-15T01:35:39.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.750+0000 7f2804446640 10 monclient: build_initial_monmap 2024-09-15T01:35:39.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.750+0000 7f2804446640 10 monclient: monmap: 2024-09-15T01:35:39.754 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:39.754 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:39.754 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:39.752345+0000 2024-09-15T01:35:39.754 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:39.752345+0000 2024-09-15T01:35:39.754 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:39.754 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:39.754 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:39.754 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:39.754 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:39.754 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:39.755 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:39.755 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:39.755 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:39.755 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:39.755 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:39.755 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:39.755 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:39.755 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:39.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.751+0000 7f2804446640 1 Processor -- start 2024-09-15T01:35:39.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.751+0000 7f2804446640 1 -- start start 2024-09-15T01:35:39.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.751+0000 7f2804446640 10 monclient: init 2024-09-15T01:35:39.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:39.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 10 monclient: _add_conns ranks=[2,10,3,7,5,0,1,4,6,9,8] 2024-09-15T01:35:39.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27fc0a2e30 0x7f27fc133280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:39.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 10 monclient(hunting): picked mon.noname-b con 0x7f27fc0a2e30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:39.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f27fc09fd80 0x7f27fc0a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:39.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 10 monclient(hunting): picked mon.noname-f con 0x7f27fc09fd80 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27fc0a09c0 0x7f27fc0a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 10 monclient(hunting): picked mon.noname-g con 0x7f27fc0a09c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27fc09e290 con 0x7f27fc09fd80 2024-09-15T01:35:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc0a2450 con 0x7f27fc0a2e30 2024-09-15T01:35:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc0a3200 con 0x7f27fc0a09c0 2024-09-15T01:35:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f2804446640 10 monclient(hunting): authenticate will time out at 955.395268s 2024-09-15T01:35:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28019e1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f27fc09fd80 0x7f27fc0a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:39.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27fc0a09c0 0x7f27fc0a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:39.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28021e2640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27fc0a2e30 msgr2=0x7f27fc133280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:39.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28019e1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f27fc09fd80 0x7f27fc0a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:56310/0 (socket says 172.21.15.204:56310) 2024-09-15T01:35:39.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28019e1640 1 -- 172.21.15.204:0/2378514277 learned_addr learned my addr 172.21.15.204:0/2378514277 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:39.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28019e1640 10 monclient(hunting): get_auth_request con 0x7f27fc09fd80 auth_method 0 2024-09-15T01:35:39.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28019e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:39.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28019e1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:39.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28019e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:39.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28021e2640 1 --2- 172.21.15.204:0/2378514277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27fc0a2e30 0x7f27fc133280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:39.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 10 monclient(hunting): get_auth_request con 0x7f27fc0a09c0 auth_method 0 2024-09-15T01:35:39.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:39.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:39.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:39.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:39.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:39.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:39.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28019e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:39.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28019e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:39.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28019e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:39.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 10 monclient(hunting): handle_auth_done global_id 14790 payload 274 2024-09-15T01:35:39.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 1 -- 172.21.15.204:0/2378514277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27fc0a2e30 msgr2=0x7f27fc133280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:39.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 1 --2- 172.21.15.204:0/2378514277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27fc0a2e30 0x7f27fc133280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:39.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 1 -- 172.21.15.204:0/2378514277 >> v2:172.21.15.204:3300/0 conn(0x7f27fc09fd80 msgr2=0x7f27fc0a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:39.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 1 --2- 172.21.15.204:0/2378514277 >> v2:172.21.15.204:3300/0 conn(0x7f27fc09fd80 0x7f27fc0a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:39.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 10 monclient: _finish_hunting 0 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 1 monclient: found mon.noname-g 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28029e3640 1 -- 172.21.15.204:0/2378514277 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc04b720 con 0x7f27fc0a09c0 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.752+0000 7f28019e1640 1 --2- 172.21.15.204:0/2378514277 >> v2:172.21.15.204:3300/0 conn(0x7f27fc09fd80 0x7f27fc0a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28029e3640 1 --2- 172.21.15.204:0/2378514277 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27fc0a09c0 0x7f27fc0a0e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009e90 tx=0x7f27f4009780 comp rx=0 tx=0).ready entity=mon.3 client_cookie=9052fb89aeb671d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28011e0640 1 -- 172.21.15.204:0/2378514277 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f27f40079d0 con 0x7f27fc0a09c0 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28011e0640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28011e0640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28011e0640 10 monclient: dump: 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:39.768 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:39.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28011e0640 10 monclient: _finish_auth 0 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28011e0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28011e0640 1 -- 172.21.15.204:0/2378514277 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f402d050 con 0x7f27fc0a09c0 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f2804446640 5 monclient: authenticate success, global_id 14790 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f2804446640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28011e0640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f2804446640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28011e0640 1 -- 172.21.15.204:0/2378514277 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f27f4007bd0 con 0x7f27fc0a09c0 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f2804446640 4 set_mon_vals no callback set 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28011e0640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28011e0640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.753+0000 7f28011e0640 10 monclient: dump: 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:39.770 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:39.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 10 monclient: shutdown 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 1 -- 172.21.15.204:0/2378514277 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27fc0a09c0 msgr2=0x7f27fc0a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 1 --2- 172.21.15.204:0/2378514277 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27fc0a09c0 0x7f27fc0a0e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009e90 tx=0x7f27f4009780 comp rx=0 tx=0).stop 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 1 -- 172.21.15.204:0/2378514277 shutdown_connections 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 1 --2- 172.21.15.204:0/2378514277 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27fc0a09c0 0x7f27fc0a0e10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 1 --2- 172.21.15.204:0/2378514277 >> v2:172.21.15.204:3300/0 conn(0x7f27fc09fd80 0x7f27fc0a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 1 --2- 172.21.15.204:0/2378514277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27fc0a2e30 0x7f27fc133280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 1 -- 172.21.15.204:0/2378514277 >> 172.21.15.204:0/2378514277 conn(0x7f27fc06e860 msgr2=0x7f27fc06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 1 -- 172.21.15.204:0/2378514277 shutdown_connections 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 1 -- 172.21.15.204:0/2378514277 wait complete. 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 10 monclient: build_initial_monmap 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.754+0000 7f2804446640 10 monclient: monmap: 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:39.772 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:39.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 1 Processor -- start 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 1 -- start start 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 10 monclient: init 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 10 monclient: _add_conns ranks=[7,2,8,0,4,1,3,6,5] 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f27fc09fd80 0x7f27fc12f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 10 monclient(hunting): picked mon.noname-i con 0x7f27fc09fd80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27fc0a09c0 0x7f27fc11c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 10 monclient(hunting): picked mon.noname-b con 0x7f27fc0a09c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f27fc0a2e30 0x7f27fc11cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 10 monclient(hunting): picked mon.noname-e con 0x7f27fc0a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc134200 con 0x7f27fc0a09c0 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:39.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc09d9b0 con 0x7f27fc09fd80 2024-09-15T01:35:39.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:39.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc09d4c0 con 0x7f27fc0a2e30 2024-09-15T01:35:39.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:39.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f2804446640 10 monclient(hunting): authenticate will time out at 955.398648s 2024-09-15T01:35:39.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f28021e2640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f27fc09fd80 msgr2=0x7f27fc12f6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:39.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f28021e2640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f27fc09fd80 0x7f27fc12f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:39.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f28019e1640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27fc0a09c0 msgr2=0x7f27fc11c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:39.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f28019e1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27fc0a09c0 0x7f27fc11c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:39.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f28029e3640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f27fc0a2e30 0x7f27fc11cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:39.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.755+0000 7f28029e3640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f27fc0a2e30 0x7f27fc11cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:33094/0 (socket says 172.21.15.204:33094) 2024-09-15T01:35:39.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 1 -- 172.21.15.204:0/3245243925 learned_addr learned my addr 172.21.15.204:0/3245243925 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:39.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 10 monclient(hunting): get_auth_request con 0x7f27fc0a2e30 auth_method 0 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 10 monclient(hunting): handle_auth_done global_id 5660 payload 824 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 1 -- 172.21.15.204:0/3245243925 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f27fc09fd80 msgr2=0x7f27fc12f6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 1 --2- 172.21.15.204:0/3245243925 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f27fc09fd80 0x7f27fc12f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 1 -- 172.21.15.204:0/3245243925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27fc0a09c0 msgr2=0x7f27fc11c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 1 --2- 172.21.15.204:0/3245243925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27fc0a09c0 0x7f27fc11c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 10 monclient: _finish_hunting 0 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 1 monclient: found mon.noname-e 2024-09-15T01:35:39.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 1 -- 172.21.15.204:0/3245243925 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec009450 con 0x7f27fc0a2e30 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28029e3640 1 --2- 172.21.15.204:0/3245243925 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f27fc0a2e30 0x7f27fc11cb70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f27f40099f0 tx=0x7f27f4037000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f27f27fc640 1 -- 172.21.15.204:0/3245243925 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f27f4007eb0 con 0x7f27fc0a2e30 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f27f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f27f27fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f27f27fc640 10 monclient: dump: 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:39.777 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:39.778 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:39.778 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:39.778 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:39.778 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:39.778 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:39.778 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:39.778 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:39.778 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:39.778 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:39.778 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:39.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f27f27fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:39.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f27f27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:39.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f27f27fc640 1 -- 172.21.15.204:0/3245243925 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f403d430 con 0x7f27fc0a2e30 2024-09-15T01:35:39.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f2804446640 5 monclient: authenticate success, global_id 5660 2024-09-15T01:35:39.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f27f27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:39.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f27f27fc640 1 -- 172.21.15.204:0/3245243925 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f27f403da20 con 0x7f27fc0a2e30 2024-09-15T01:35:39.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f2804446640 10 monclient: _renew_subs 2024-09-15T01:35:39.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f2804446640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:39.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f2804446640 1 -- 172.21.15.204:0/3245243925 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f4009a20 con 0x7f27fc0a2e30 2024-09-15T01:35:39.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f27f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f27f27fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f27f27fc640 10 monclient: dump: 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:39.780 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:39.781 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:39.781 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:39.781 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:39.781 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:39.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f28009df640 4 set_mon_vals no callback set 2024-09-15T01:35:39.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f2804446640 10 monclient: _renew_subs 2024-09-15T01:35:39.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f2804446640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:39.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.756+0000 7f2804446640 1 -- 172.21.15.204:0/3245243925 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27fc11ddc0 con 0x7f27fc0a2e30 2024-09-15T01:35:39.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.757+0000 7f2804446640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:39.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.757+0000 7f2804446640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:39.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.757+0000 7f2804446640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:39.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.757+0000 7f27f27fc640 1 -- 172.21.15.204:0/3245243925 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f27f4003040 con 0x7f27fc0a2e30 2024-09-15T01:35:39.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.757+0000 7f2804446640 1 -- 172.21.15.204:0/3245243925 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27fc070d80 con 0x7f27fc0a2e30 2024-09-15T01:35:39.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.757+0000 7f27f27fc640 1 --2- 172.21.15.204:0/3245243925 >> v2:172.21.15.204:6812/28210 conn(0x7f27e00397b0 0x7f27e003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:39.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.757+0000 7f27f27fc640 1 -- 172.21.15.204:0/3245243925 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6985+0+0 (secure 0 0 0) 0x7f27f406d460 con 0x7f27fc0a2e30 2024-09-15T01:35:39.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.758+0000 7f28021e2640 1 --2- 172.21.15.204:0/3245243925 >> v2:172.21.15.204:6812/28210 conn(0x7f27e00397b0 0x7f27e003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:39.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.758+0000 7f28021e2640 10 monclient: get_auth_request con 0x7f27e00397b0 auth_method 0 2024-09-15T01:35:39.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.758+0000 7f28021e2640 1 --2- 172.21.15.204:0/3245243925 >> v2:172.21.15.204:6812/28210 conn(0x7f27e00397b0 0x7f27e003bc60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006fd0 tx=0x7f27f8006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:39.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.763+0000 7f27f27fc640 1 -- 172.21.15.204:0/3245243925 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f27f40443b0 con 0x7f27fc0a2e30 2024-09-15T01:35:39.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.763+0000 7f27f27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:39.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.763+0000 7f27f27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:39.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.925+0000 7f2804446640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:39.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.925+0000 7f2804446640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:39.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.925+0000 7f2804446640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:39.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.925+0000 7f2804446640 1 -- 172.21.15.204:0/3245243925 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f27fc09dfb0 con 0x7f27fc0a2e30 2024-09-15T01:35:39.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.927+0000 7f27f27fc640 1 -- 172.21.15.204:0/3245243925 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v133) v1 ==== 71+0+7 (secure 0 0 0) 0x7f27f4030b90 con 0x7f27fc0a2e30 2024-09-15T01:35:39.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.927+0000 7f27f27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:39.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:39.927+0000 7f27f27fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:40.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.025+0000 7f2804446640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:40.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.025+0000 7f2804446640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:40.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.025+0000 7f2804446640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:40.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.025+0000 7f2804446640 1 -- 172.21.15.204:0/3245243925 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f27fc0a4220 con 0x7f27fc0a2e30 2024-09-15T01:35:40.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.025+0000 7f27f27fc640 1 -- 172.21.15.204:0/3245243925 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v133) v1 ==== 71+0+7 (secure 0 0 0) 0x7f27f40443b0 con 0x7f27fc0a2e30 2024-09-15T01:35:40.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.025+0000 7f27f27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:40.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.025+0000 7f27f27fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:40.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 1 -- 172.21.15.204:0/3245243925 >> v2:172.21.15.204:6812/28210 conn(0x7f27e00397b0 msgr2=0x7f27e003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:40.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 1 --2- 172.21.15.204:0/3245243925 >> v2:172.21.15.204:6812/28210 conn(0x7f27e00397b0 0x7f27e003bc60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006fd0 tx=0x7f27f8006d30 comp rx=0 tx=0).stop 2024-09-15T01:35:40.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 10 monclient: shutdown 2024-09-15T01:35:40.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:40.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 1 -- 172.21.15.204:0/3245243925 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f27fc0a2e30 msgr2=0x7f27fc11cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:40.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 1 --2- 172.21.15.204:0/3245243925 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f27fc0a2e30 0x7f27fc11cb70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f27f40099f0 tx=0x7f27f4037000 comp rx=0 tx=0).stop 2024-09-15T01:35:40.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 1 -- 172.21.15.204:0/3245243925 shutdown_connections 2024-09-15T01:35:40.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 1 --2- 172.21.15.204:0/3245243925 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f27fc0a2e30 0x7f27fc11cb70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 1 --2- 172.21.15.204:0/3245243925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27fc0a09c0 0x7f27fc11c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 1 --2- 172.21.15.204:0/3245243925 >> v2:172.21.15.204:6812/28210 conn(0x7f27e00397b0 0x7f27e003bc60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 1 --2- 172.21.15.204:0/3245243925 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f27fc09fd80 0x7f27fc12f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 1 -- 172.21.15.204:0/3245243925 >> 172.21.15.204:0/3245243925 conn(0x7f27fc06e860 msgr2=0x7f27fc0a67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:40.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 1 -- 172.21.15.204:0/3245243925 shutdown_connections 2024-09-15T01:35:40.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.028+0000 7f2804446640 1 -- 172.21.15.204:0/3245243925 wait complete. 2024-09-15T01:35:40.039 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set ls 2024-09-15T01:35:40.039 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep barset 2024-09-15T01:35:40.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.110+0000 7ffb05d1d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:40.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.110+0000 7ffb05d1d640 10 monclient: build_initial_monmap 2024-09-15T01:35:40.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.110+0000 7ffb05d1d640 10 monclient: monmap: 2024-09-15T01:35:40.113 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:40.113 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:40.113 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:40.111827+0000 2024-09-15T01:35:40.113 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:40.111827+0000 2024-09-15T01:35:40.113 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:40.113 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:40.113 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:40.113 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:40.113 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:40.114 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:40.114 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:40.114 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:40.114 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:40.114 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:40.114 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:40.114 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:40.114 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:40.114 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:40.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 1 Processor -- start 2024-09-15T01:35:40.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 1 -- start start 2024-09-15T01:35:40.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 10 monclient: init 2024-09-15T01:35:40.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:40.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 10 monclient: _add_conns ranks=[9,2,0,1,4,8,3,6,10,5,7] 2024-09-15T01:35:40.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffb000a2e30 0x7ffb00133280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 10 monclient(hunting): picked mon.noname-a con 0x7ffb000a2e30 addr v2:172.21.15.178:3300/0 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb0009fd80 0x7ffb000a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 10 monclient(hunting): picked mon.noname-b con 0x7ffb0009fd80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffb000a09c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0004b720 con 0x7ffb000a09c0 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0009e290 con 0x7ffb000a2e30 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000a2450 con 0x7ffb0009fd80 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffb05d1d640 10 monclient(hunting): authenticate will time out at 955.754591s 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffafeffd640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb0009fd80 msgr2=0x7ffb000a0150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:40.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffafeffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb0009fd80 0x7ffb000a0150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffaff7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffb000a2e30 0x7ffb00133280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffaff7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffb000a2e30 0x7ffb00133280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:39496/0 (socket says 172.21.15.204:39496) 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffaff7fe640 1 -- 172.21.15.204:0/2956536196 learned_addr learned my addr 172.21.15.204:0/2956536196 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffaff7fe640 10 monclient(hunting): get_auth_request con 0x7ffb000a2e30 auth_method 0 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffafffff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ffb000a09c0 0x7ffb000a0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.204:48330/0 (socket says 172.21.15.204:48330) 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffaff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffaff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.111+0000 7ffaff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 10 monclient(hunting): handle_auth_done global_id 5850 payload 274 2024-09-15T01:35:40.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 1 -- 172.21.15.204:0/2956536196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb0009fd80 msgr2=0x7ffb000a0150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:40.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 1 --2- 172.21.15.204:0/2956536196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb0009fd80 0x7ffb000a0150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 1 -- 172.21.15.204:0/2956536196 >> v1:172.21.15.178:6789/0 conn(0x7ffb000a09c0 legacy=0x7ffb000a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:40.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:40.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 1 monclient: found mon.noname-a 2024-09-15T01:35:40.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:40.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:35:40.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 1 -- 172.21.15.204:0/2956536196 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb000a3200 con 0x7ffb000a2e30 2024-09-15T01:35:40.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.112+0000 7ffaff7fe640 1 --2- 172.21.15.204:0/2956536196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb000a2e30 0x7ffb00133280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009940 tx=0x7ffaf00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14a4c75e49d9138 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:40.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffafe7fc640 1 -- 172.21.15.204:0/2956536196 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffaf002ad40 con 0x7ffb000a2e30 2024-09-15T01:35:40.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffafe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:40.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffafe7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffafe7fc640 10 monclient: dump: 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:40.126 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffafe7fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffafe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffafe7fc640 1 -- 172.21.15.204:0/2956536196 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf0031820 con 0x7ffb000a2e30 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 5 monclient: authenticate success, global_id 5850 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffafe7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffafe7fc640 1 -- 172.21.15.204:0/2956536196 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffaf0031e90 con 0x7ffb000a2e30 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 4 set_mon_vals no callback set 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffafe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffafe7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffafe7fc640 10 monclient: dump: 2024-09-15T01:35:40.127 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:40.128 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:40.129 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:40.129 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:40.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 10 monclient: shutdown 2024-09-15T01:35:40.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:40.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2956536196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb000a2e30 msgr2=0x7ffb00133280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:40.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 1 --2- 172.21.15.204:0/2956536196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb000a2e30 0x7ffb00133280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009940 tx=0x7ffaf00091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2956536196 shutdown_connections 2024-09-15T01:35:40.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 1 --2- 172.21.15.204:0/2956536196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb0009fd80 0x7ffb000a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 1 --2- 172.21.15.204:0/2956536196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb000a2e30 0x7ffb00133280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2956536196 >> 172.21.15.204:0/2956536196 conn(0x7ffb0006e860 msgr2=0x7ffb0006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:40.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2956536196 shutdown_connections 2024-09-15T01:35:40.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.113+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2956536196 wait complete. 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.114+0000 7ffb05d1d640 10 monclient: build_initial_monmap 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.114+0000 7ffb05d1d640 10 monclient: monmap: 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:40.130 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.114+0000 7ffb05d1d640 1 Processor -- start 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.114+0000 7ffb05d1d640 1 -- start start 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.114+0000 7ffb05d1d640 10 monclient: init 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 10 monclient: _add_conns ranks=[0,6,7,4,1,3,8,2,5] 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb0009fd80 0x7ffb0011c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 10 monclient(hunting): picked mon.noname-a con 0x7ffb0009fd80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb000a09c0 0x7ffb0011cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:40.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 10 monclient(hunting): picked mon.noname-d con 0x7ffb000a09c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffb000a2e30 0x7ffb0011d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 10 monclient(hunting): picked mon.noname-i con 0x7ffb000a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0011fc00 con 0x7ffb0009fd80 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0009d9e0 con 0x7ffb000a09c0 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0009d4f0 con 0x7ffb000a2e30 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffb05d1d640 10 monclient(hunting): authenticate will time out at 955.758168s 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffafffff640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffb000a2e30 msgr2=0x7ffb0011d0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffafffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffb000a2e30 0x7ffb0011d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:40.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffafeffd640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb000a09c0 msgr2=0x7ffb0011cb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffafeffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb000a09c0 0x7ffb0011cb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb0009fd80 0x7ffb0011c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb0009fd80 0x7ffb0011c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:39500/0 (socket says 172.21.15.204:39500) 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 1 -- 172.21.15.204:0/2013129816 learned_addr learned my addr 172.21.15.204:0/2013129816 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 10 monclient(hunting): get_auth_request con 0x7ffb0009fd80 auth_method 0 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 10 monclient(hunting): handle_auth_done global_id 5868 payload 824 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 1 -- 172.21.15.204:0/2013129816 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffb000a2e30 msgr2=0x7ffb0011d0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:40.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 1 --2- 172.21.15.204:0/2013129816 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffb000a2e30 0x7ffb0011d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 1 -- 172.21.15.204:0/2013129816 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb000a09c0 msgr2=0x7ffb0011cb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 1 --2- 172.21.15.204:0/2013129816 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb000a09c0 0x7ffb0011cb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 1 monclient: found mon.noname-a 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 1 -- 172.21.15.204:0/2013129816 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf0009450 con 0x7ffb0009fd80 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.115+0000 7ffaff7fe640 1 --2- 172.21.15.204:0/2013129816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb0009fd80 0x7ffb0011c630 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0035040 tx=0x7ffaf0032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffaf77fe640 1 -- 172.21.15.204:0/2013129816 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffaf003a390 con 0x7ffb0009fd80 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffaf77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffaf77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffaf77fe640 10 monclient: dump: 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:40.134 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:40.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffaf77fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffaf77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffaf77fe640 1 -- 172.21.15.204:0/2013129816 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf003a530 con 0x7ffb0009fd80 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffb05d1d640 5 monclient: authenticate success, global_id 5868 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffb05d1d640 10 monclient: _renew_subs 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffb05d1d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2013129816 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0011e990 con 0x7ffb0009fd80 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffaf77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffaf77fe640 1 -- 172.21.15.204:0/2013129816 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffaf003ab20 con 0x7ffb0009fd80 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffaf77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffaf77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffaf77fe640 10 monclient: dump: 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:40.136 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:40.137 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffafdffb640 4 set_mon_vals no callback set 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffb05d1d640 10 monclient: _renew_subs 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffb05d1d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.116+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2013129816 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0009c3d0 con 0x7ffb0009fd80 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.117+0000 7ffb05d1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.117+0000 7ffb05d1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.117+0000 7ffb05d1d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.117+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2013129816 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0011ee10 con 0x7ffb0009fd80 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.117+0000 7ffaf77fe640 1 -- 172.21.15.204:0/2013129816 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ffaf0039040 con 0x7ffb0009fd80 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.117+0000 7ffaf77fe640 1 --2- 172.21.15.204:0/2013129816 >> v2:172.21.15.204:6812/28210 conn(0x7ffae00397e0 0x7ffae003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.117+0000 7ffaf77fe640 1 -- 172.21.15.204:0/2013129816 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6985+0+0 (secure 0 0 0) 0x7ffaf006ba60 con 0x7ffb0009fd80 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.117+0000 7ffafeffd640 1 --2- 172.21.15.204:0/2013129816 >> v2:172.21.15.204:6812/28210 conn(0x7ffae00397e0 0x7ffae003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:40.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.117+0000 7ffafeffd640 10 monclient: get_auth_request con 0x7ffae00397e0 auth_method 0 2024-09-15T01:35:40.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.117+0000 7ffafeffd640 1 --2- 172.21.15.204:0/2013129816 >> v2:172.21.15.204:6812/28210 conn(0x7ffae00397e0 0x7ffae003bc90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffae8006fd0 tx=0x7ffae8006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:40.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.122+0000 7ffaf77fe640 1 -- 172.21.15.204:0/2013129816 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ffaf00392f0 con 0x7ffb0009fd80 2024-09-15T01:35:40.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.122+0000 7ffaf77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:40.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.122+0000 7ffaf77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:40.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.283+0000 7ffb05d1d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:40.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.283+0000 7ffb05d1d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:40.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.283+0000 7ffb05d1d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:40.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.283+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2013129816 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7ffb00070d80 con 0x7ffb0009fd80 2024-09-15T01:35:40.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.284+0000 7ffaf77fe640 1 -- 172.21.15.204:0/2013129816 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v133) v1 ==== 71+0+7 (secure 0 0 0) 0x7ffaf0037030 con 0x7ffb0009fd80 2024-09-15T01:35:40.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.284+0000 7ffaf77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:40.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.284+0000 7ffaf77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:40.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.384+0000 7ffb05d1d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:40.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.384+0000 7ffb05d1d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:40.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.384+0000 7ffb05d1d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:40.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.384+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2013129816 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7ffb0009dfb0 con 0x7ffb0009fd80 2024-09-15T01:35:40.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.385+0000 7ffaf77fe640 1 -- 172.21.15.204:0/2013129816 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v133) v1 ==== 71+0+7 (secure 0 0 0) 0x7ffaf0041070 con 0x7ffb0009fd80 2024-09-15T01:35:40.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.385+0000 7ffaf77fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:40.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.385+0000 7ffaf77fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:40.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2013129816 >> v2:172.21.15.204:6812/28210 conn(0x7ffae00397e0 msgr2=0x7ffae003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:40.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 1 --2- 172.21.15.204:0/2013129816 >> v2:172.21.15.204:6812/28210 conn(0x7ffae00397e0 0x7ffae003bc90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffae8006fd0 tx=0x7ffae8006d30 comp rx=0 tx=0).stop 2024-09-15T01:35:40.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 10 monclient: shutdown 2024-09-15T01:35:40.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:40.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2013129816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb0009fd80 msgr2=0x7ffb0011c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:40.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 1 --2- 172.21.15.204:0/2013129816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb0009fd80 0x7ffb0011c630 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0035040 tx=0x7ffaf0032000 comp rx=0 tx=0).stop 2024-09-15T01:35:40.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2013129816 shutdown_connections 2024-09-15T01:35:40.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 1 --2- 172.21.15.204:0/2013129816 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb000a09c0 0x7ffb0011cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.390 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 1 --2- 172.21.15.204:0/2013129816 >> v2:172.21.15.204:6812/28210 conn(0x7ffae00397e0 0x7ffae003bc90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 1 --2- 172.21.15.204:0/2013129816 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffb000a2e30 0x7ffb0011d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 1 --2- 172.21.15.204:0/2013129816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb0009fd80 0x7ffb0011c630 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2013129816 >> 172.21.15.204:0/2013129816 conn(0x7ffb0006e860 msgr2=0x7ffb00131060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:40.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2013129816 shutdown_connections 2024-09-15T01:35:40.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.388+0000 7ffb05d1d640 1 -- 172.21.15.204:0/2013129816 wait complete. 2024-09-15T01:35:40.399 INFO:tasks.workunit.client.0.smithi204.stdout:barset 2024-09-15T01:35:40.399 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set rm barset 2024-09-15T01:35:40.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:40.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient: build_initial_monmap 2024-09-15T01:35:40.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient: monmap: 2024-09-15T01:35:40.474 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:40.474 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:40.474 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:40.472783+0000 2024-09-15T01:35:40.474 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:40.472783+0000 2024-09-15T01:35:40.474 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:40.474 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:40.474 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:40.474 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:40.474 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:40.474 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:40.475 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:40.475 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:40.475 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:40.475 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:40.475 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:40.475 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:40.475 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:40.475 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:40.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 1 Processor -- start 2024-09-15T01:35:40.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 1 -- start start 2024-09-15T01:35:40.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient: init 2024-09-15T01:35:40.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:40.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient: _add_conns ranks=[6,10,0,4,9,2,3,5,1,7,8] 2024-09-15T01:35:40.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19b009ab40 0x7f19b009af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:40.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient(hunting): picked mon.noname-d con 0x7f19b009ab40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:40.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f19b009c200 0x7f19b0099a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:40.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient(hunting): picked mon.noname-f con 0x7f19b009c200 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:40.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f19b009d7b0 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:40.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b004b720 con 0x7f19b009d7b0 2024-09-15T01:35:40.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:40.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b0099f50 con 0x7f19b009c200 2024-09-15T01:35:40.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:40.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f19b009c650 con 0x7f19b009ab40 2024-09-15T01:35:40.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:40.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19b4b5d640 10 monclient(hunting): authenticate will time out at 956.115584s 2024-09-15T01:35:40.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f19b009c200 0x7f19b0099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:40.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19ae59c640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19b009ab40 msgr2=0x7f19b009af10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:40.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19ae59c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19b009ab40 0x7f19b009af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:40.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f19b009c200 0x7f19b0099a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:56324/0 (socket says 172.21.15.204:56324) 2024-09-15T01:35:40.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 1 -- 172.21.15.204:0/3456443122 learned_addr learned my addr 172.21.15.204:0/3456443122 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:40.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 10 monclient(hunting): get_auth_request con 0x7f19b009c200 auth_method 0 2024-09-15T01:35:40.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:40.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:40.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:40.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:40.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 10 monclient(hunting): handle_auth_done global_id 6274 payload 274 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 1 -- 172.21.15.204:0/3456443122 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19b009ab40 msgr2=0x7f19b009af10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 1 --2- 172.21.15.204:0/3456443122 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19b009ab40 0x7f19b009af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 1 -- 172.21.15.204:0/3456443122 >> v1:172.21.15.178:6789/0 conn(0x7f19b009d7b0 legacy=0x7f19b009dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 10 monclient: _finish_hunting 0 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 1 monclient: found mon.noname-f 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.472+0000 7f19add9b640 1 -- 172.21.15.204:0/3456443122 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b009bb00 con 0x7f19b009c200 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19add9b640 1 --2- 172.21.15.204:0/3456443122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19b009c200 0x7f19b0099a10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009910 tx=0x7f19a00091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cef4701068043b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19ad59a640 1 -- 172.21.15.204:0/3456443122 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19a002ad40 con 0x7f19b009c200 2024-09-15T01:35:40.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19ad59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19ad59a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19ad59a640 10 monclient: dump: 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:40.487 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19ad59a640 10 monclient: _finish_auth 0 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19ad59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19b4b5d640 5 monclient: authenticate success, global_id 6274 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19b4b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19ad59a640 1 -- 172.21.15.204:0/3456443122 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a0031820 con 0x7f19b009c200 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19ad59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19b4b5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:40.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19ad59a640 1 -- 172.21.15.204:0/3456443122 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19a0031e50 con 0x7f19b009c200 2024-09-15T01:35:40.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19b4b5d640 4 set_mon_vals no callback set 2024-09-15T01:35:40.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19ad59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:40.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19ad59a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:40.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.473+0000 7f19ad59a640 10 monclient: dump: 2024-09-15T01:35:40.489 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:40.489 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:40.489 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.489 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.489 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:40.489 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:40.489 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:40.489 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:40.490 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:40.490 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:40.490 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:40.490 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:40.490 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:40.490 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:40.490 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:40.490 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:40.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.474+0000 7f19b4b5d640 10 monclient: shutdown 2024-09-15T01:35:40.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.474+0000 7f19b4b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:40.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.474+0000 7f19b4b5d640 1 -- 172.21.15.204:0/3456443122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19b009c200 msgr2=0x7f19b0099a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:40.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.474+0000 7f19b4b5d640 1 --2- 172.21.15.204:0/3456443122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19b009c200 0x7f19b0099a10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009910 tx=0x7f19a00091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.474+0000 7f19b4b5d640 1 -- 172.21.15.204:0/3456443122 shutdown_connections 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.474+0000 7f19b4b5d640 1 --2- 172.21.15.204:0/3456443122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19b009c200 0x7f19b0099a10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.474+0000 7f19b4b5d640 1 --2- 172.21.15.204:0/3456443122 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19b009ab40 0x7f19b009af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.474+0000 7f19b4b5d640 1 -- 172.21.15.204:0/3456443122 >> 172.21.15.204:0/3456443122 conn(0x7f19b0068ba0 msgr2=0x7f19b00a2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.474+0000 7f19b4b5d640 1 -- 172.21.15.204:0/3456443122 shutdown_connections 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.474+0000 7f19b4b5d640 1 -- 172.21.15.204:0/3456443122 wait complete. 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.474+0000 7f19b4b5d640 10 monclient: build_initial_monmap 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.474+0000 7f19b4b5d640 10 monclient: monmap: 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:40.491 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 1 Processor -- start 2024-09-15T01:35:40.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 1 -- start start 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 10 monclient: init 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 10 monclient: _add_conns ranks=[3,5,0,1,8,2,7,6,4] 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b009ab40 0x7f19b0067150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 10 monclient(hunting): picked mon.noname-g con 0x7f19b009ab40 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f19b009c200 0x7f19b0132600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 10 monclient(hunting): picked mon.noname-h con 0x7f19b009c200 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19b009d7b0 0x7f19b011e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 10 monclient(hunting): picked mon.noname-a con 0x7f19b009d7b0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b0120d80 con 0x7f19b009d7b0 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:40.493 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b006e980 con 0x7f19b009ab40 2024-09-15T01:35:40.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:40.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19b01333a0 con 0x7f19b009c200 2024-09-15T01:35:40.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:40.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19b4b5d640 10 monclient(hunting): authenticate will time out at 956.118689s 2024-09-15T01:35:40.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19ae59c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b009ab40 0x7f19b0067150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:40.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19add9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f19b009c200 0x7f19b0132600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:40.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19ae59c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b009ab40 0x7f19b0067150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:47342/0 (socket says 172.21.15.204:47342) 2024-09-15T01:35:40.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19ae59c640 1 -- 172.21.15.204:0/1723943960 learned_addr learned my addr 172.21.15.204:0/1723943960 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:40.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19ae59c640 10 monclient(hunting): get_auth_request con 0x7f19b009ab40 auth_method 0 2024-09-15T01:35:40.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19ae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:40.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19ae59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:40.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19ae59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:40.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19add9b640 10 monclient(hunting): get_auth_request con 0x7f19b009c200 auth_method 0 2024-09-15T01:35:40.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19add9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:40.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19add9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:40.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19add9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:40.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19aed9d640 1 --2- 172.21.15.204:0/1723943960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19b009d7b0 0x7f19b011e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:40.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19ae59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:40.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19ae59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:40.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19ae59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:40.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19add9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:40.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19add9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:40.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19add9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:40.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19aed9d640 10 monclient(hunting): get_auth_request con 0x7f19b009d7b0 auth_method 0 2024-09-15T01:35:40.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19aed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:40.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19aed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:40.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.475+0000 7f19aed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:40.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19aed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:40.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19aed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:40.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19aed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:40.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19ae59c640 10 monclient(hunting): handle_auth_done global_id 14799 payload 824 2024-09-15T01:35:40.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19ae59c640 1 -- 172.21.15.204:0/1723943960 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f19b009c200 msgr2=0x7f19b0132600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:40.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19ae59c640 1 --2- 172.21.15.204:0/1723943960 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f19b009c200 0x7f19b0132600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19ae59c640 1 -- 172.21.15.204:0/1723943960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19b009d7b0 msgr2=0x7f19b011e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:40.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19ae59c640 1 --2- 172.21.15.204:0/1723943960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19b009d7b0 0x7f19b011e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:40.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19ae59c640 10 monclient: _finish_hunting 0 2024-09-15T01:35:40.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19ae59c640 1 monclient: found mon.noname-g 2024-09-15T01:35:40.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19ae59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:40.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19ae59c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:40.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19ae59c640 1 -- 172.21.15.204:0/1723943960 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a0009450 con 0x7f19b009ab40 2024-09-15T01:35:40.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19ae59c640 1 --2- 172.21.15.204:0/1723943960 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b009ab40 0x7f19b0067150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f199800b600 tx=0x7f19980091a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:40.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19a67fc640 1 -- 172.21.15.204:0/1723943960 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1998015070 con 0x7f19b009ab40 2024-09-15T01:35:40.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:40.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19a67fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:40.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19a67fc640 10 monclient: dump: 2024-09-15T01:35:40.497 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:40.497 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:40.498 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.498 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.498 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:40.498 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:40.498 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:40.498 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:40.498 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:40.498 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:40.498 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:40.498 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19a67fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19a67fc640 1 -- 172.21.15.204:0/1723943960 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1998006150 con 0x7f19b009ab40 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19b4b5d640 5 monclient: authenticate success, global_id 14799 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19b4b5d640 10 monclient: _renew_subs 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19b4b5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19b4b5d640 1 -- 172.21.15.204:0/1723943960 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b009cd70 con 0x7f19b009ab40 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19a67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:40.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19a67fc640 1 -- 172.21.15.204:0/1723943960 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1998006740 con 0x7f19b009ab40 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19a67fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19a67fc640 10 monclient: dump: 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:40.500 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19acd99640 4 set_mon_vals no callback set 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19b4b5d640 10 monclient: _renew_subs 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19b4b5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.476+0000 7f19b4b5d640 1 -- 172.21.15.204:0/1723943960 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b0067c80 con 0x7f19b009ab40 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.477+0000 7f19a67fc640 1 -- 172.21.15.204:0/1723943960 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f19980062c0 con 0x7f19b009ab40 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.477+0000 7f19b4b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.477+0000 7f19b4b5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.477+0000 7f19b4b5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:40.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.477+0000 7f19b4b5d640 1 -- 172.21.15.204:0/1723943960 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b0070cc0 con 0x7f19b009ab40 2024-09-15T01:35:40.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.477+0000 7f19a67fc640 1 --2- 172.21.15.204:0/1723943960 >> v2:172.21.15.204:6812/28210 conn(0x7f19900397b0 0x7f199003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:40.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.477+0000 7f19a67fc640 1 -- 172.21.15.204:0/1723943960 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6985+0+0 (secure 0 0 0) 0x7f199804b7b0 con 0x7f19b009ab40 2024-09-15T01:35:40.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.477+0000 7f19add9b640 1 --2- 172.21.15.204:0/1723943960 >> v2:172.21.15.204:6812/28210 conn(0x7f19900397b0 0x7f199003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:40.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.477+0000 7f19add9b640 10 monclient: get_auth_request con 0x7f19900397b0 auth_method 0 2024-09-15T01:35:40.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.477+0000 7f19add9b640 1 --2- 172.21.15.204:0/1723943960 >> v2:172.21.15.204:6812/28210 conn(0x7f19900397b0 0x7f199003bc60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009420 tx=0x7f19a0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:40.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.481+0000 7f19a67fc640 1 -- 172.21.15.204:0/1723943960 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1998010440 con 0x7f19b009ab40 2024-09-15T01:35:40.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.482+0000 7f19a67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:40.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.482+0000 7f19a67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:40.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.642+0000 7f19b4b5d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-09-15T01:35:40.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.642+0000 7f19b4b5d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-09-15T01:35:40.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.642+0000 7f19b4b5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:40.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.642+0000 7f19b4b5d640 1 -- 172.21.15.204:0/1723943960 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set rm", "pool": "barset"} v 0) v1 -- 0x7f19b0069880 con 0x7f19b009ab40 2024-09-15T01:35:40.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.661+0000 7f19a67fc640 1 -- 172.21.15.204:0/1723943960 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "barset"}]=0 v134) v1 ==== 89+0+0 (secure 0 0 0) 0x7f199800eae0 con 0x7f19b009ab40 2024-09-15T01:35:40.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.661+0000 7f19a67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-09-15T01:35:40.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.661+0000 7f19a67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:40.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.763+0000 7f19b4b5d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-09-15T01:35:40.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.763+0000 7f19b4b5d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-09-15T01:35:40.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.763+0000 7f19b4b5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:40.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:40.763+0000 7f19b4b5d640 1 -- 172.21.15.204:0/1723943960 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set rm", "pool": "barset"} v 0) v1 -- 0x7f19b011f840 con 0x7f19b009ab40 2024-09-15T01:35:41.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.454+0000 7f19a67fc640 1 -- 172.21.15.204:0/1723943960 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "barset"}]=0 v135) v1 ==== 89+0+0 (secure 0 0 0) 0x7f199800ecc0 con 0x7f19b009ab40 2024-09-15T01:35:41.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.454+0000 7f19a67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-09-15T01:35:41.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.454+0000 7f19a67fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:41.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.460+0000 7f19b4b5d640 1 -- 172.21.15.204:0/1723943960 >> v2:172.21.15.204:6812/28210 conn(0x7f19900397b0 msgr2=0x7f199003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:41.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.460+0000 7f19b4b5d640 1 --2- 172.21.15.204:0/1723943960 >> v2:172.21.15.204:6812/28210 conn(0x7f19900397b0 0x7f199003bc60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009420 tx=0x7f19a0032000 comp rx=0 tx=0).stop 2024-09-15T01:35:41.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.461+0000 7f19b4b5d640 10 monclient: shutdown 2024-09-15T01:35:41.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.461+0000 7f19b4b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:41.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.461+0000 7f19b4b5d640 1 -- 172.21.15.204:0/1723943960 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b009ab40 msgr2=0x7f19b0067150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:41.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.461+0000 7f19b4b5d640 1 --2- 172.21.15.204:0/1723943960 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b009ab40 0x7f19b0067150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f199800b600 tx=0x7f19980091a0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.461+0000 7f19b4b5d640 1 -- 172.21.15.204:0/1723943960 shutdown_connections 2024-09-15T01:35:41.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.461+0000 7f19b4b5d640 1 --2- 172.21.15.204:0/1723943960 >> v2:172.21.15.204:6812/28210 conn(0x7f19900397b0 0x7f199003bc60 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009420 tx=0x7f19a0032000 comp rx=0 tx=0).stop 2024-09-15T01:35:41.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.461+0000 7f19b4b5d640 1 --2- 172.21.15.204:0/1723943960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19b009d7b0 0x7f19b011e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.461+0000 7f19b4b5d640 1 --2- 172.21.15.204:0/1723943960 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f19b009c200 0x7f19b0132600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.461+0000 7f19b4b5d640 1 --2- 172.21.15.204:0/1723943960 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b009ab40 0x7f19b0067150 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.461+0000 7f19b4b5d640 1 -- 172.21.15.204:0/1723943960 >> 172.21.15.204:0/1723943960 conn(0x7f19b0068ba0 msgr2=0x7f19b00969e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:41.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.461+0000 7f19b4b5d640 1 -- 172.21.15.204:0/1723943960 shutdown_connections 2024-09-15T01:35:41.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.461+0000 7f19b4b5d640 1 -- 172.21.15.204:0/1723943960 wait complete. 2024-09-15T01:35:41.478 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set ls 2024-09-15T01:35:41.478 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -c barset 2024-09-15T01:35:41.478 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep -q 0 2024-09-15T01:35:41.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.552+0000 7f6ac4cd4640 10 monclient: get_monmap_and_config 2024-09-15T01:35:41.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.552+0000 7f6ac4cd4640 10 monclient: build_initial_monmap 2024-09-15T01:35:41.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.553+0000 7f6ac4cd4640 10 monclient: monmap: 2024-09-15T01:35:41.554 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:41.554 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:41.555 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:41.553505+0000 2024-09-15T01:35:41.555 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:41.553505+0000 2024-09-15T01:35:41.555 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:41.555 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:41.555 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:41.555 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:41.555 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:41.555 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:41.555 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:41.555 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:41.555 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:41.555 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:41.556 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:41.556 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:41.556 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:41.556 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:41.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.553+0000 7f6ac4cd4640 1 Processor -- start 2024-09-15T01:35:41.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.553+0000 7f6ac4cd4640 1 -- start start 2024-09-15T01:35:41.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.553+0000 7f6ac4cd4640 10 monclient: init 2024-09-15T01:35:41.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:41.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 10 monclient: _add_conns ranks=[8,10,0,9,4,6,2,3,1,7,5] 2024-09-15T01:35:41.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ac00a2e30 0x7f6ac0133280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 10 monclient(hunting): picked mon.noname-e con 0x7f6ac00a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:41.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6ac009fd80 0x7f6ac00a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 10 monclient(hunting): picked mon.noname-f con 0x7f6ac009fd80 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:41.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6ac00a09c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac004b720 con 0x7f6ac00a09c0 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac009e290 con 0x7f6ac009fd80 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac00a2450 con 0x7f6ac00a2e30 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6ac4cd4640 10 monclient(hunting): authenticate will time out at 957.196309s 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6ac009fd80 0x7f6ac00a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6ac009fd80 0x7f6ac00a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:56328/0 (socket says 172.21.15.204:56328) 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 1 -- 172.21.15.204:0/3176226103 learned_addr learned my addr 172.21.15.204:0/3176226103 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 10 monclient(hunting): get_auth_request con 0x7f6ac009fd80 auth_method 0 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:41.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 1 -- 172.21.15.204:0/3176226103 >> v2:172.21.15.204:3300/0 conn(0x7f6ac009fd80 msgr2=0x7f6ac00a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 1 -- 172.21.15.204:0/3176226103 >> v2:172.21.15.204:3300/0 conn(0x7f6ac009fd80 msgr2=0x7f6ac00a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 1 --2- 172.21.15.204:0/3176226103 >> v2:172.21.15.204:3300/0 conn(0x7f6ac009fd80 0x7f6ac00a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abdd9b640 1 --2- 172.21.15.204:0/3176226103 >> v2:172.21.15.204:3300/0 conn(0x7f6ac009fd80 0x7f6ac00a0150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 1 --2- 172.21.15.204:0/3176226103 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ac00a2e30 0x7f6ac0133280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 10 monclient(hunting): get_auth_request con 0x7f6ac00a2e30 auth_method 0 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:41.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 10 monclient(hunting): handle_auth_done global_id 5669 payload 274 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 1 -- 172.21.15.204:0/3176226103 >> v2:172.21.15.204:3300/0 conn(0x7f6ac009fd80 msgr2=0x7f6ac00a0150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 1 --2- 172.21.15.204:0/3176226103 >> v2:172.21.15.204:3300/0 conn(0x7f6ac009fd80 0x7f6ac00a0150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 1 -- 172.21.15.204:0/3176226103 >> v1:172.21.15.178:6789/0 conn(0x7f6ac00a09c0 legacy=0x7f6ac00a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 10 monclient: _finish_hunting 0 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 1 monclient: found mon.noname-e 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.554+0000 7f6abe59c640 1 -- 172.21.15.204:0/3176226103 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac00a3200 con 0x7f6ac00a2e30 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abe59c640 1 --2- 172.21.15.204:0/3176226103 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ac00a2e30 0x7f6ac0133280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009880 tx=0x7f6ab00090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=fa747c55814f8441 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:41.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abd59a640 1 -- 172.21.15.204:0/3176226103 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ab002ac50 con 0x7f6ac00a2e30 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abd59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abd59a640 10 monclient: dump: 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:41.571 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abd59a640 10 monclient: _finish_auth 0 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6ac4cd4640 5 monclient: authenticate success, global_id 5669 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abd59a640 1 -- 172.21.15.204:0/3176226103 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab002add0 con 0x7f6ac00a2e30 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6ac4cd4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abd59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:41.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6ac4cd4640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:41.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abd59a640 1 -- 172.21.15.204:0/3176226103 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ab0031ca0 con 0x7f6ac00a2e30 2024-09-15T01:35:41.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:41.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6ac4cd4640 4 set_mon_vals no callback set 2024-09-15T01:35:41.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abd59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:41.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.555+0000 7f6abd59a640 10 monclient: dump: 2024-09-15T01:35:41.573 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:41.573 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:41.573 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.573 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.573 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:41.573 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:41.573 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:41.574 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:41.574 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:41.574 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:41.574 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:41.574 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:41.574 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:41.574 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:41.574 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:41.574 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:41.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.556+0000 7f6ac4cd4640 10 monclient: shutdown 2024-09-15T01:35:41.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.556+0000 7f6ac4cd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:41.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.556+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3176226103 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ac00a2e30 msgr2=0x7f6ac0133280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:41.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.556+0000 7f6ac4cd4640 1 --2- 172.21.15.204:0/3176226103 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ac00a2e30 0x7f6ac0133280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009880 tx=0x7f6ab00090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.556+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3176226103 shutdown_connections 2024-09-15T01:35:41.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.556+0000 7f6ac4cd4640 1 --2- 172.21.15.204:0/3176226103 >> v2:172.21.15.204:3300/0 conn(0x7f6ac009fd80 0x7f6ac00a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.556+0000 7f6ac4cd4640 1 --2- 172.21.15.204:0/3176226103 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ac00a2e30 0x7f6ac0133280 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.556+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3176226103 >> 172.21.15.204:0/3176226103 conn(0x7f6ac006e860 msgr2=0x7f6ac006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:41.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.556+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3176226103 shutdown_connections 2024-09-15T01:35:41.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.556+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3176226103 wait complete. 2024-09-15T01:35:41.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.556+0000 7f6ac4cd4640 10 monclient: build_initial_monmap 2024-09-15T01:35:41.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.556+0000 7f6ac4cd4640 10 monclient: monmap: 2024-09-15T01:35:41.575 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:41.576 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:41.576 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:41.576 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:41.576 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:41.576 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:41.576 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:41.576 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:41.576 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:41.576 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:41.576 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:41.576 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:41.576 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:41.577 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:41.577 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:41.577 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:41.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 1 Processor -- start 2024-09-15T01:35:41.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 1 -- start start 2024-09-15T01:35:41.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 10 monclient: init 2024-09-15T01:35:41.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:41.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 10 monclient: _add_conns ranks=[2,6,3,8,7,4,5,0,1] 2024-09-15T01:35:41.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ac009fd80 0x7f6ac009b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 10 monclient(hunting): picked mon.noname-b con 0x7f6ac009fd80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:41.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ac00a09c0 0x7f6ac011c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 10 monclient(hunting): picked mon.noname-d con 0x7f6ac00a09c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:41.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ac00a2e30 0x7f6ac011cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 10 monclient(hunting): picked mon.noname-g con 0x7f6ac00a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:41.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:41.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac011edb0 con 0x7f6ac009fd80 2024-09-15T01:35:41.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:41.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac009d9b0 con 0x7f6ac00a2e30 2024-09-15T01:35:41.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:41.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac011ead0 con 0x7f6ac00a09c0 2024-09-15T01:35:41.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:41.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6ac4cd4640 10 monclient(hunting): authenticate will time out at 957.199726s 2024-09-15T01:35:41.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abe59c640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ac009fd80 msgr2=0x7f6ac009b9f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:41.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abdd9b640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ac00a09c0 msgr2=0x7f6ac011c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:41.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abe59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ac009fd80 0x7f6ac009b9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:41.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abdd9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ac00a09c0 0x7f6ac011c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:41.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ac00a2e30 0x7f6ac011cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:41.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ac00a2e30 0x7f6ac011cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:47350/0 (socket says 172.21.15.204:47350) 2024-09-15T01:35:41.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abed9d640 1 -- 172.21.15.204:0/3457301034 learned_addr learned my addr 172.21.15.204:0/3457301034 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:41.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abed9d640 10 monclient(hunting): get_auth_request con 0x7f6ac00a2e30 auth_method 0 2024-09-15T01:35:41.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:41.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.557+0000 7f6abed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6abed9d640 10 monclient(hunting): handle_auth_done global_id 14808 payload 824 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6abed9d640 1 -- 172.21.15.204:0/3457301034 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ac00a09c0 msgr2=0x7f6ac011c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6abed9d640 1 --2- 172.21.15.204:0/3457301034 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ac00a09c0 0x7f6ac011c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6abed9d640 1 -- 172.21.15.204:0/3457301034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ac009fd80 msgr2=0x7f6ac009b9f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6abed9d640 1 --2- 172.21.15.204:0/3457301034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ac009fd80 0x7f6ac009b9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6abed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6abed9d640 1 monclient: found mon.noname-g 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6abed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:41.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6abed9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:41.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6abed9d640 1 -- 172.21.15.204:0/3457301034 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab0009390 con 0x7f6ac00a2e30 2024-09-15T01:35:41.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6abed9d640 1 --2- 172.21.15.204:0/3457301034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ac00a2e30 0x7f6ac011cb70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800c860 tx=0x7f6ab8001690 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:41.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6aae7fc640 1 -- 172.21.15.204:0/3457301034 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ab800ce50 con 0x7f6ac00a2e30 2024-09-15T01:35:41.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6aae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:41.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6aae7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:41.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6aae7fc640 10 monclient: dump: 2024-09-15T01:35:41.583 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:41.583 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:41.583 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.583 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.583 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:41.583 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:41.584 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:41.584 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:41.584 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:41.584 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:41.584 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:41.584 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:41.584 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:41.584 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:41.584 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:41.584 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:41.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6aae7fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:41.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6aae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:41.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6ac4cd4640 5 monclient: authenticate success, global_id 14808 2024-09-15T01:35:41.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6aae7fc640 1 -- 172.21.15.204:0/3457301034 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab800acd0 con 0x7f6ac00a2e30 2024-09-15T01:35:41.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6ac4cd4640 10 monclient: _renew_subs 2024-09-15T01:35:41.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6ac4cd4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:41.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3457301034 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac011d790 con 0x7f6ac00a2e30 2024-09-15T01:35:41.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6aae7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:41.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6aae7fc640 1 -- 172.21.15.204:0/3457301034 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ab800ae40 con 0x7f6ac00a2e30 2024-09-15T01:35:41.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6aae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:41.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6aae7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:41.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6abcd99640 4 set_mon_vals no callback set 2024-09-15T01:35:41.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6aae7fc640 10 monclient: dump: 2024-09-15T01:35:41.585 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:41.586 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6ac4cd4640 10 monclient: _renew_subs 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6ac4cd4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.558+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3457301034 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac011f370 con 0x7f6ac00a2e30 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.559+0000 7f6aae7fc640 1 -- 172.21.15.204:0/3457301034 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6ab800a850 con 0x7f6ac00a2e30 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.559+0000 7f6ac4cd4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.559+0000 7f6ac4cd4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.559+0000 7f6ac4cd4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.559+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3457301034 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac011efd0 con 0x7f6ac00a2e30 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.559+0000 7f6aae7fc640 1 --2- 172.21.15.204:0/3457301034 >> v2:172.21.15.204:6812/28210 conn(0x7f6aa00397b0 0x7f6aa003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.559+0000 7f6aae7fc640 1 -- 172.21.15.204:0/3457301034 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f6ab804bd70 con 0x7f6ac00a2e30 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.559+0000 7f6abe59c640 1 --2- 172.21.15.204:0/3457301034 >> v2:172.21.15.204:6812/28210 conn(0x7f6aa00397b0 0x7f6aa003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.559+0000 7f6abe59c640 10 monclient: get_auth_request con 0x7f6aa00397b0 auth_method 0 2024-09-15T01:35:41.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.560+0000 7f6abe59c640 1 --2- 172.21.15.204:0/3457301034 >> v2:172.21.15.204:6812/28210 conn(0x7f6aa00397b0 0x7f6aa003bc60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0034000 tx=0x7f6ab0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:41.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.563+0000 7f6aae7fc640 1 -- 172.21.15.204:0/3457301034 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6ac011efd0 con 0x7f6ac00a2e30 2024-09-15T01:35:41.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.563+0000 7f6aae7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:41.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.563+0000 7f6aae7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:41.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.729+0000 7f6ac4cd4640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:41.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.729+0000 7f6ac4cd4640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:41.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.729+0000 7f6ac4cd4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:41.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.729+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3457301034 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f6ac0066cb0 con 0x7f6ac00a2e30 2024-09-15T01:35:41.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.729+0000 7f6aae7fc640 1 -- 172.21.15.204:0/3457301034 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v135) v1 ==== 71+0+0 (secure 0 0 0) 0x7f6ab800ab00 con 0x7f6ac00a2e30 2024-09-15T01:35:41.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.729+0000 7f6aae7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:41.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.729+0000 7f6aae7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:41.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.828+0000 7f6ac4cd4640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:41.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.828+0000 7f6ac4cd4640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:41.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.828+0000 7f6ac4cd4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:41.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.828+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3457301034 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f6ac011efd0 con 0x7f6ac00a2e30 2024-09-15T01:35:41.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.828+0000 7f6aae7fc640 1 -- 172.21.15.204:0/3457301034 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v135) v1 ==== 71+0+0 (secure 0 0 0) 0x7f6ac011efd0 con 0x7f6ac00a2e30 2024-09-15T01:35:41.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.828+0000 7f6aae7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:41.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.828+0000 7f6aae7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:41.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3457301034 >> v2:172.21.15.204:6812/28210 conn(0x7f6aa00397b0 msgr2=0x7f6aa003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:41.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 1 --2- 172.21.15.204:0/3457301034 >> v2:172.21.15.204:6812/28210 conn(0x7f6aa00397b0 0x7f6aa003bc60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0034000 tx=0x7f6ab0032000 comp rx=0 tx=0).stop 2024-09-15T01:35:41.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 10 monclient: shutdown 2024-09-15T01:35:41.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:41.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3457301034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ac00a2e30 msgr2=0x7f6ac011cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:41.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 1 --2- 172.21.15.204:0/3457301034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ac00a2e30 0x7f6ac011cb70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800c860 tx=0x7f6ab8001690 comp rx=0 tx=0).stop 2024-09-15T01:35:41.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3457301034 shutdown_connections 2024-09-15T01:35:41.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 1 --2- 172.21.15.204:0/3457301034 >> v2:172.21.15.204:6812/28210 conn(0x7f6aa00397b0 0x7f6aa003bc60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 1 --2- 172.21.15.204:0/3457301034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ac00a2e30 0x7f6ac011cb70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 1 --2- 172.21.15.204:0/3457301034 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ac00a09c0 0x7f6ac011c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 1 --2- 172.21.15.204:0/3457301034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ac009fd80 0x7f6ac009b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3457301034 >> 172.21.15.204:0/3457301034 conn(0x7f6ac006e860 msgr2=0x7f6ac0131090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:41.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3457301034 shutdown_connections 2024-09-15T01:35:41.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.830+0000 7f6ac4cd4640 1 -- 172.21.15.204:0/3457301034 wait complete. 2024-09-15T01:35:41.841 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set create-compat 2024-09-15T01:35:41.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.913+0000 7f0aeaf1d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:41.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.913+0000 7f0aeaf1d640 10 monclient: build_initial_monmap 2024-09-15T01:35:41.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.913+0000 7f0aeaf1d640 10 monclient: monmap: 2024-09-15T01:35:41.915 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:41.915 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:41.915 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:41.913908+0000 2024-09-15T01:35:41.915 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:41.913908+0000 2024-09-15T01:35:41.915 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:41.915 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:41.915 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:41.915 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:41.915 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:41.915 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:41.916 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:41.916 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:41.916 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:41.916 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:41.916 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:41.916 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:41.916 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:41.916 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:41.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 1 Processor -- start 2024-09-15T01:35:41.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 1 -- start start 2024-09-15T01:35:41.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 10 monclient: init 2024-09-15T01:35:41.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:41.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 10 monclient: _add_conns ranks=[2,7,8,9,6,10,0,4,1,3,5] 2024-09-15T01:35:41.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ae40a2c60 0x7f0ae4132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 10 monclient(hunting): picked mon.noname-b con 0x7f0ae40a2c60 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:41.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0ae409fc60 0x7f0ae40a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 10 monclient(hunting): picked mon.noname-i con 0x7f0ae409fc60 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:41.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ae40a0ab0 0x7f0ae40a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 10 monclient(hunting): picked mon.noname-e con 0x7f0ae40a0ab0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:41.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:41.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae40a3030 con 0x7f0ae40a2c60 2024-09-15T01:35:41.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae409d5a0 con 0x7f0ae409fc60 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae41338d0 con 0x7f0ae40a0ab0 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0aeaf1d640 10 monclient(hunting): authenticate will time out at 957.556665s 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0ae3fff640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0ae409fc60 msgr2=0x7f0ae40a0050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0ae3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0ae409fc60 0x7f0ae40a0050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0ae8cb9640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ae40a2c60 msgr2=0x7f0ae4132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.914+0000 7f0ae8cb9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ae40a2c60 0x7f0ae4132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ae40a0ab0 0x7f0ae40a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ae40a0ab0 0x7f0ae40a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:33118/0 (socket says 172.21.15.204:33118) 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 1 -- 172.21.15.204:0/1227925744 learned_addr learned my addr 172.21.15.204:0/1227925744 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:41.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 10 monclient(hunting): get_auth_request con 0x7f0ae40a0ab0 auth_method 0 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 10 monclient(hunting): handle_auth_done global_id 5678 payload 274 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 1 -- 172.21.15.204:0/1227925744 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0ae409fc60 msgr2=0x7f0ae40a0050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 1 --2- 172.21.15.204:0/1227925744 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0ae409fc60 0x7f0ae40a0050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 1 -- 172.21.15.204:0/1227925744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ae40a2c60 msgr2=0x7f0ae4132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 1 --2- 172.21.15.204:0/1227925744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ae40a2c60 0x7f0ae4132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 10 monclient: _finish_hunting 0 2024-09-15T01:35:41.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 1 monclient: found mon.noname-e 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.915+0000 7f0ae94ba640 1 -- 172.21.15.204:0/1227925744 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae404b720 con 0x7f0ae40a0ab0 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae94ba640 1 --2- 172.21.15.204:0/1227925744 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ae40a0ab0 0x7f0ae40a0f00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009f50 tx=0x7f0adc009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f97fa5ea70d0c758 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae37fe640 1 -- 172.21.15.204:0/1227925744 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0adc007880 con 0x7f0ae40a0ab0 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae37fe640 10 monclient: dump: 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:41.927 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae37fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:41.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae37fe640 1 -- 172.21.15.204:0/1227925744 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0adc02d050 con 0x7f0ae40a0ab0 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 5 monclient: authenticate success, global_id 5678 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae37fe640 1 -- 172.21.15.204:0/1227925744 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0adc030360 con 0x7f0ae40a0ab0 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 4 set_mon_vals no callback set 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0ae37fe640 10 monclient: dump: 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.929 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:41.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 10 monclient: shutdown 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1227925744 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ae40a0ab0 msgr2=0x7f0ae40a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 1 --2- 172.21.15.204:0/1227925744 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ae40a0ab0 0x7f0ae40a0f00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009f50 tx=0x7f0adc009840 comp rx=0 tx=0).stop 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1227925744 shutdown_connections 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 1 --2- 172.21.15.204:0/1227925744 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0ae409fc60 0x7f0ae40a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 1 --2- 172.21.15.204:0/1227925744 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0ae40a0ab0 0x7f0ae40a0f00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 1 --2- 172.21.15.204:0/1227925744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ae40a2c60 0x7f0ae4132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1227925744 >> 172.21.15.204:0/1227925744 conn(0x7f0ae40a60c0 msgr2=0x7f0ae40a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.916+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1227925744 shutdown_connections 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.917+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1227925744 wait complete. 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.917+0000 7f0aeaf1d640 10 monclient: build_initial_monmap 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.917+0000 7f0aeaf1d640 10 monclient: monmap: 2024-09-15T01:35:41.931 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:41.932 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.917+0000 7f0aeaf1d640 1 Processor -- start 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 1 -- start start 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 10 monclient: init 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 10 monclient: _add_conns ranks=[1,6,7,4,0,8,3,5,2] 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ae409fc60 0x7f0ae41307e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 10 monclient(hunting): picked mon.noname-f con 0x7f0ae409fc60 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0ae40a0ab0 0x7f0ae411d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 10 monclient(hunting): picked mon.noname-d con 0x7f0ae40a0ab0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0ae40a2c60 0x7f0ae411d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 10 monclient(hunting): picked mon.noname-i con 0x7f0ae40a2c60 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:41.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:41.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae4120f80 con 0x7f0ae409fc60 2024-09-15T01:35:41.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:41.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae409ca40 con 0x7f0ae40a0ab0 2024-09-15T01:35:41.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:41.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae409d7d0 con 0x7f0ae40a2c60 2024-09-15T01:35:41.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:41.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0aeaf1d640 10 monclient(hunting): authenticate will time out at 957.560499s 2024-09-15T01:35:41.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae94ba640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0ae40a2c60 msgr2=0x7f0ae411d8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:41.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae94ba640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0ae40a2c60 0x7f0ae411d8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:41.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae3fff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0ae40a0ab0 msgr2=0x7f0ae411d3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:41.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae3fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0ae40a0ab0 0x7f0ae411d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ae409fc60 0x7f0ae41307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ae409fc60 0x7f0ae41307e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:56336/0 (socket says 172.21.15.204:56336) 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 1 -- 172.21.15.204:0/1441156027 learned_addr learned my addr 172.21.15.204:0/1441156027 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 10 monclient(hunting): get_auth_request con 0x7f0ae409fc60 auth_method 0 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 10 monclient(hunting): handle_auth_done global_id 6292 payload 824 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 1 -- 172.21.15.204:0/1441156027 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0ae40a2c60 msgr2=0x7f0ae411d8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:41.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 1 --2- 172.21.15.204:0/1441156027 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0ae40a2c60 0x7f0ae411d8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 1 -- 172.21.15.204:0/1441156027 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0ae40a0ab0 msgr2=0x7f0ae411d3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:41.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 1 --2- 172.21.15.204:0/1441156027 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0ae40a0ab0 0x7f0ae411d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:41.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 10 monclient: _finish_hunting 0 2024-09-15T01:35:41.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 1 monclient: found mon.noname-f 2024-09-15T01:35:41.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:41.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:41.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.918+0000 7f0ae8cb9640 1 -- 172.21.15.204:0/1441156027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0adc009ae0 con 0x7f0ae409fc60 2024-09-15T01:35:41.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae8cb9640 1 --2- 172.21.15.204:0/1441156027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ae409fc60 0x7f0ae41307e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800b630 tx=0x7f0ad80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:41.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae0ff9640 1 -- 172.21.15.204:0/1441156027 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ad8006150 con 0x7f0ae409fc60 2024-09-15T01:35:41.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:41.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae0ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:41.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae0ff9640 10 monclient: dump: 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:41.937 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae0ff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0aeaf1d640 5 monclient: authenticate success, global_id 6292 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0aeaf1d640 10 monclient: _renew_subs 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae0ff9640 1 -- 172.21.15.204:0/1441156027 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad80062f0 con 0x7f0ae409fc60 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0aeaf1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1441156027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae411ee00 con 0x7f0ae409fc60 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae0ff9640 1 -- 172.21.15.204:0/1441156027 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ad80103e0 con 0x7f0ae409fc60 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:41.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae0ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae2ffd640 4 set_mon_vals no callback set 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0ae0ff9640 10 monclient: dump: 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:41.939 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0aeaf1d640 10 monclient: _renew_subs 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0aeaf1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.919+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1441156027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae411f9d0 con 0x7f0ae409fc60 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.920+0000 7f0ae0ff9640 1 -- 172.21.15.204:0/1441156027 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0ad8003b80 con 0x7f0ae409fc60 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.920+0000 7f0aeaf1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.920+0000 7f0aeaf1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.920+0000 7f0aeaf1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.920+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1441156027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae4120a50 con 0x7f0ae409fc60 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.920+0000 7f0ae0ff9640 1 --2- 172.21.15.204:0/1441156027 >> v2:172.21.15.204:6812/28210 conn(0x7f0ac80397b0 0x7f0ac803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:41.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.920+0000 7f0ae0ff9640 1 -- 172.21.15.204:0/1441156027 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f0ad804ada0 con 0x7f0ae409fc60 2024-09-15T01:35:41.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.920+0000 7f0ae3fff640 1 --2- 172.21.15.204:0/1441156027 >> v2:172.21.15.204:6812/28210 conn(0x7f0ac80397b0 0x7f0ac803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:41.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.920+0000 7f0ae3fff640 10 monclient: get_auth_request con 0x7f0ac80397b0 auth_method 0 2024-09-15T01:35:41.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.920+0000 7f0ae3fff640 1 --2- 172.21.15.204:0/1441156027 >> v2:172.21.15.204:6812/28210 conn(0x7f0ac80397b0 0x7f0ac803bc60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0ae411d2f0 tx=0x7f0ad0006d00 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:41.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.924+0000 7f0ae0ff9640 1 -- 172.21.15.204:0/1441156027 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0ad800daf0 con 0x7f0ae409fc60 2024-09-15T01:35:41.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.924+0000 7f0ae0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:41.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:41.924+0000 7f0ae0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:42.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.087+0000 7f0aeaf1d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:35:42.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.087+0000 7f0aeaf1d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:35:42.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.087+0000 7f0aeaf1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:42.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.087+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1441156027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7f0ae4071100 con 0x7f0ae409fc60 2024-09-15T01:35:42.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.670+0000 7f0ae0ff9640 1 -- 172.21.15.204:0/1441156027 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v136) v1 ==== 82+0+0 (secure 0 0 0) 0x7f0ad8017030 con 0x7f0ae409fc60 2024-09-15T01:35:42.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.670+0000 7f0ae0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:35:42.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.670+0000 7f0ae0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:42.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.772+0000 7f0aeaf1d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:35:42.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.772+0000 7f0aeaf1d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:35:42.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.772+0000 7f0aeaf1d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:42.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.772+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1441156027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7f0ae40008d0 con 0x7f0ae409fc60 2024-09-15T01:35:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.773+0000 7f0ae0ff9640 1 -- 172.21.15.204:0/1441156027 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v136) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0ad8017020 con 0x7f0ae409fc60 2024-09-15T01:35:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.773+0000 7f0ae0ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:35:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.773+0000 7f0ae0ff9640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-09-15T01:35:42.775 INFO:tasks.workunit.client.0.smithi204.stderr:compat weight-set already created 2024-09-15T01:35:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1441156027 >> v2:172.21.15.204:6812/28210 conn(0x7f0ac80397b0 msgr2=0x7f0ac803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 1 --2- 172.21.15.204:0/1441156027 >> v2:172.21.15.204:6812/28210 conn(0x7f0ac80397b0 0x7f0ac803bc60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0ae411d2f0 tx=0x7f0ad0006d00 comp rx=0 tx=0).stop 2024-09-15T01:35:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 10 monclient: shutdown 2024-09-15T01:35:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1441156027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ae409fc60 msgr2=0x7f0ae41307e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 1 --2- 172.21.15.204:0/1441156027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ae409fc60 0x7f0ae41307e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800b630 tx=0x7f0ad80091a0 comp rx=0 tx=0).stop 2024-09-15T01:35:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1441156027 shutdown_connections 2024-09-15T01:35:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 1 --2- 172.21.15.204:0/1441156027 >> v2:172.21.15.204:6812/28210 conn(0x7f0ac80397b0 0x7f0ac803bc60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:42.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 1 --2- 172.21.15.204:0/1441156027 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0ae40a2c60 0x7f0ae411d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 1 --2- 172.21.15.204:0/1441156027 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0ae40a0ab0 0x7f0ae411d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 1 --2- 172.21.15.204:0/1441156027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ae409fc60 0x7f0ae41307e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1441156027 >> 172.21.15.204:0/1441156027 conn(0x7f0ae40a60c0 msgr2=0x7f0ae40298c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1441156027 shutdown_connections 2024-09-15T01:35:42.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.775+0000 7f0aeaf1d640 1 -- 172.21.15.204:0/1441156027 wait complete. 2024-09-15T01:35:42.785 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set ls 2024-09-15T01:35:42.786 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep '(compat)' 2024-09-15T01:35:42.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.858+0000 7efdf4217640 10 monclient: get_monmap_and_config 2024-09-15T01:35:42.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.858+0000 7efdf4217640 10 monclient: build_initial_monmap 2024-09-15T01:35:42.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.858+0000 7efdf4217640 10 monclient: monmap: 2024-09-15T01:35:42.860 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:42.860 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:42.860 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:42.858519+0000 2024-09-15T01:35:42.860 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:42.858519+0000 2024-09-15T01:35:42.860 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:42.860 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:42.860 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:42.860 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:42.860 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:42.860 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:42.860 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:42.861 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:42.861 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:42.861 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:42.861 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:42.861 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:42.861 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:42.861 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:42.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.858+0000 7efdf4217640 1 Processor -- start 2024-09-15T01:35:42.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.858+0000 7efdf4217640 1 -- start start 2024-09-15T01:35:42.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.858+0000 7efdf4217640 10 monclient: init 2024-09-15T01:35:42.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:42.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 10 monclient: _add_conns ranks=[1,8,10,7,0,4,9,6,3,2,5] 2024-09-15T01:35:42.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7efdec0a2e30 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:42.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdec09fd80 0x7efdec0a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:42.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 10 monclient(hunting): picked mon.noname-e con 0x7efdec09fd80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:42.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efdec137fa0 0x7efdec13a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:42.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 10 monclient(hunting): picked mon.noname-f con 0x7efdec137fa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:42.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdec04b720 con 0x7efdec0a2e30 2024-09-15T01:35:42.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:42.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdec09e290 con 0x7efdec137fa0 2024-09-15T01:35:42.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:42.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0a2450 con 0x7efdec09fd80 2024-09-15T01:35:42.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:42.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf4217640 10 monclient(hunting): authenticate will time out at 958.501438s 2024-09-15T01:35:42.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efdec137fa0 0x7efdec13a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:42.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efdec137fa0 0x7efdec13a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:56350/0 (socket says 172.21.15.204:56350) 2024-09-15T01:35:42.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 1 -- 172.21.15.204:0/4107786921 learned_addr learned my addr 172.21.15.204:0/4107786921 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:42.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 10 monclient(hunting): get_auth_request con 0x7efdec137fa0 auth_method 0 2024-09-15T01:35:42.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:42.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:42.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:42.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf1fb3640 1 --1- 172.21.15.204:0/4107786921 >> v1:172.21.15.204:6789/0 conn(0x7efdec0a2e30 0x7efdec133280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:47982/0 (socket says 172.21.15.204:47982) 2024-09-15T01:35:42.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:42.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:42.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:42.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf17b2640 1 --2- 172.21.15.204:0/4107786921 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdec09fd80 0x7efdec0a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:42.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf17b2640 10 monclient(hunting): get_auth_request con 0x7efdec09fd80 auth_method 0 2024-09-15T01:35:42.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf17b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:42.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf17b2640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:42.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf17b2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:42.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 10 monclient(hunting): handle_auth_done global_id 6301 payload 274 2024-09-15T01:35:42.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 1 -- 172.21.15.204:0/4107786921 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdec09fd80 msgr2=0x7efdec0a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 1 --2- 172.21.15.204:0/4107786921 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdec09fd80 0x7efdec0a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 1 -- 172.21.15.204:0/4107786921 >> v1:172.21.15.204:6789/0 conn(0x7efdec0a2e30 legacy=0x7efdec133280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 10 monclient: _finish_hunting 0 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 1 monclient: found mon.noname-f 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.859+0000 7efdf27b4640 1 -- 172.21.15.204:0/4107786921 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec133990 con 0x7efdec137fa0 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf27b4640 1 --2- 172.21.15.204:0/4107786921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdec137fa0 0x7efdec13a380 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efde4009f20 tx=0x7efde4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e697b5e64a0be51 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf0fb1640 1 -- 172.21.15.204:0/4107786921 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efde4007880 con 0x7efdec137fa0 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf0fb1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf0fb1640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf0fb1640 10 monclient: dump: 2024-09-15T01:35:42.873 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:42.874 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf0fb1640 10 monclient: _finish_auth 0 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf0fb1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf0fb1640 1 -- 172.21.15.204:0/4107786921 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde402d050 con 0x7efdec137fa0 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf4217640 5 monclient: authenticate success, global_id 6301 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf4217640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf0fb1640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf4217640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf0fb1640 1 -- 172.21.15.204:0/4107786921 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efde4030360 con 0x7efdec137fa0 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf4217640 4 set_mon_vals no callback set 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf0fb1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:42.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf0fb1640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:42.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.860+0000 7efdf0fb1640 10 monclient: dump: 2024-09-15T01:35:42.876 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:42.876 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:42.876 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:42.876 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:42.876 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:42.876 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:42.876 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:42.876 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:42.876 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:42.876 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.861+0000 7efdf4217640 10 monclient: shutdown 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.861+0000 7efdf4217640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.861+0000 7efdf4217640 1 -- 172.21.15.204:0/4107786921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdec137fa0 msgr2=0x7efdec13a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.861+0000 7efdf4217640 1 --2- 172.21.15.204:0/4107786921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdec137fa0 0x7efdec13a380 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efde4009f20 tx=0x7efde4009840 comp rx=0 tx=0).stop 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.861+0000 7efdf4217640 1 -- 172.21.15.204:0/4107786921 shutdown_connections 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.861+0000 7efdf4217640 1 --2- 172.21.15.204:0/4107786921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdec137fa0 0x7efdec13a380 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:42.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.861+0000 7efdf4217640 1 --2- 172.21.15.204:0/4107786921 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdec09fd80 0x7efdec0a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.861+0000 7efdf4217640 1 -- 172.21.15.204:0/4107786921 >> 172.21.15.204:0/4107786921 conn(0x7efdec06e860 msgr2=0x7efdec06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.861+0000 7efdf4217640 1 -- 172.21.15.204:0/4107786921 shutdown_connections 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.861+0000 7efdf4217640 1 -- 172.21.15.204:0/4107786921 wait complete. 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.861+0000 7efdf4217640 10 monclient: build_initial_monmap 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.861+0000 7efdf4217640 10 monclient: monmap: 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:42.878 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 1 Processor -- start 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 1 -- start start 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 10 monclient: init 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 10 monclient: _add_conns ranks=[4,8,3,0,5,1,2,7,6] 2024-09-15T01:35:42.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efdec09fd80 0x7efdec099440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 10 monclient(hunting): picked mon.noname-c con 0x7efdec09fd80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdec0a2e30 0x7efdec099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 10 monclient(hunting): picked mon.noname-e con 0x7efdec0a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdec137fa0 0x7efdec099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 10 monclient(hunting): picked mon.noname-g con 0x7efdec137fa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec09b940 con 0x7efdec137fa0 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efdec09d9b0 con 0x7efdec09fd80 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7efdec09d4c0 con 0x7efdec0a2e30 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:42.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf4217640 10 monclient(hunting): authenticate will time out at 958.504622s 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf27b4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdec137fa0 0x7efdec099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf27b4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdec137fa0 0x7efdec099ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:47354/0 (socket says 172.21.15.204:47354) 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf27b4640 1 -- 172.21.15.204:0/1324285885 learned_addr learned my addr 172.21.15.204:0/1324285885 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf27b4640 10 monclient(hunting): get_auth_request con 0x7efdec137fa0 auth_method 0 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf27b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf27b4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf27b4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf27b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf27b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf27b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf1fb3640 1 --2- 172.21.15.204:0/1324285885 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efdec09fd80 0x7efdec099440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf17b2640 1 --2- 172.21.15.204:0/1324285885 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdec0a2e30 0x7efdec099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:42.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf1fb3640 10 monclient(hunting): get_auth_request con 0x7efdec09fd80 auth_method 0 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf1fb3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf1fb3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf1fb3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf17b2640 10 monclient(hunting): get_auth_request con 0x7efdec0a2e30 auth_method 0 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf17b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf17b2640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf17b2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.862+0000 7efdf27b4640 10 monclient(hunting): handle_auth_done global_id 14817 payload 824 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf27b4640 1 -- 172.21.15.204:0/1324285885 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdec0a2e30 msgr2=0x7efdec099980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf27b4640 1 --2- 172.21.15.204:0/1324285885 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdec0a2e30 0x7efdec099980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf27b4640 1 -- 172.21.15.204:0/1324285885 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efdec09fd80 msgr2=0x7efdec099440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf27b4640 1 --2- 172.21.15.204:0/1324285885 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efdec09fd80 0x7efdec099440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf27b4640 10 monclient: _finish_hunting 0 2024-09-15T01:35:42.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf27b4640 1 monclient: found mon.noname-g 2024-09-15T01:35:42.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf27b4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:42.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf27b4640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:42.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf27b4640 1 -- 172.21.15.204:0/1324285885 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde4009ae0 con 0x7efdec137fa0 2024-09-15T01:35:42.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf27b4640 1 --2- 172.21.15.204:0/1324285885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdec137fa0 0x7efdec099ec0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efde4009880 tx=0x7efde4036000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:42.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde1ffb640 1 -- 172.21.15.204:0/1324285885 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efde403cab0 con 0x7efdec137fa0 2024-09-15T01:35:42.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:42.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde1ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:42.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde1ffb640 10 monclient: dump: 2024-09-15T01:35:42.883 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:42.883 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:42.883 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:42.883 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:42.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf4217640 5 monclient: authenticate success, global_id 14817 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf4217640 10 monclient: _renew_subs 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde1ffb640 1 -- 172.21.15.204:0/1324285885 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde403cc50 con 0x7efdec137fa0 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf4217640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf4217640 1 -- 172.21.15.204:0/1324285885 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdec130b40 con 0x7efdec137fa0 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde1ffb640 1 -- 172.21.15.204:0/1324285885 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efde403cdc0 con 0x7efdec137fa0 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde1ffb640 10 monclient: dump: 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:42.885 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:42.886 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf4217640 10 monclient: _renew_subs 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf4217640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efde3fff640 4 set_mon_vals no callback set 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.863+0000 7efdf4217640 1 -- 172.21.15.204:0/1324285885 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec09af40 con 0x7efdec137fa0 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.864+0000 7efde1ffb640 1 -- 172.21.15.204:0/1324285885 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7efde4003040 con 0x7efdec137fa0 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.864+0000 7efdf4217640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.864+0000 7efdf4217640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.864+0000 7efdf4217640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.864+0000 7efdf4217640 1 -- 172.21.15.204:0/1324285885 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec11c820 con 0x7efdec137fa0 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.864+0000 7efde1ffb640 1 --2- 172.21.15.204:0/1324285885 >> v2:172.21.15.204:6812/28210 conn(0x7efdd00397b0 0x7efdd003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.864+0000 7efde1ffb640 1 -- 172.21.15.204:0/1324285885 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6705+0+0 (secure 0 0 0) 0x7efde406c0a0 con 0x7efdec137fa0 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.865+0000 7efdf1fb3640 1 --2- 172.21.15.204:0/1324285885 >> v2:172.21.15.204:6812/28210 conn(0x7efdd00397b0 0x7efdd003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.865+0000 7efdf1fb3640 10 monclient: get_auth_request con 0x7efdd00397b0 auth_method 0 2024-09-15T01:35:42.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.865+0000 7efdf1fb3640 1 --2- 172.21.15.204:0/1324285885 >> v2:172.21.15.204:6812/28210 conn(0x7efdd00397b0 0x7efdd003bc60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efdec0a1cb0 tx=0x7efde8000bc0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:42.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.869+0000 7efde1ffb640 1 -- 172.21.15.204:0/1324285885 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7efde403c630 con 0x7efdec137fa0 2024-09-15T01:35:42.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.869+0000 7efde1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:42.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:42.869+0000 7efde1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:43.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.029+0000 7efdf4217640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:43.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.029+0000 7efdf4217640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:43.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.029+0000 7efdf4217640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:43.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.029+0000 7efdf4217640 1 -- 172.21.15.204:0/1324285885 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7efdec130b40 con 0x7efdec137fa0 2024-09-15T01:35:43.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.030+0000 7efde1ffb640 1 -- 172.21.15.204:0/1324285885 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v136) v1 ==== 71+0+9 (secure 0 0 0) 0x7efdec130b40 con 0x7efdec137fa0 2024-09-15T01:35:43.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.030+0000 7efde1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:43.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.030+0000 7efde1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:43.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.127+0000 7efdf4217640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:43.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.127+0000 7efdf4217640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:43.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.127+0000 7efdf4217640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:43.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.127+0000 7efdf4217640 1 -- 172.21.15.204:0/1324285885 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7efdec130b40 con 0x7efdec137fa0 2024-09-15T01:35:43.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.128+0000 7efde1ffb640 1 -- 172.21.15.204:0/1324285885 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v136) v1 ==== 71+0+9 (secure 0 0 0) 0x7efdec130b40 con 0x7efdec137fa0 2024-09-15T01:35:43.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.128+0000 7efde1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-09-15T01:35:43.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.128+0000 7efde1ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:43.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 1 -- 172.21.15.204:0/1324285885 >> v2:172.21.15.204:6812/28210 conn(0x7efdd00397b0 msgr2=0x7efdd003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:43.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 1 --2- 172.21.15.204:0/1324285885 >> v2:172.21.15.204:6812/28210 conn(0x7efdd00397b0 0x7efdd003bc60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efdec0a1cb0 tx=0x7efde8000bc0 comp rx=0 tx=0).stop 2024-09-15T01:35:43.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 10 monclient: shutdown 2024-09-15T01:35:43.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:43.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 1 -- 172.21.15.204:0/1324285885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdec137fa0 msgr2=0x7efdec099ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:43.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 1 --2- 172.21.15.204:0/1324285885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdec137fa0 0x7efdec099ec0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efde4009880 tx=0x7efde4036000 comp rx=0 tx=0).stop 2024-09-15T01:35:43.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 1 -- 172.21.15.204:0/1324285885 shutdown_connections 2024-09-15T01:35:43.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 1 --2- 172.21.15.204:0/1324285885 >> v2:172.21.15.204:6812/28210 conn(0x7efdd00397b0 0x7efdd003bc60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:43.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 1 --2- 172.21.15.204:0/1324285885 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdec0a2e30 0x7efdec099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:43.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 1 --2- 172.21.15.204:0/1324285885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdec137fa0 0x7efdec099ec0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:43.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 1 --2- 172.21.15.204:0/1324285885 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efdec09fd80 0x7efdec099440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:43.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 1 -- 172.21.15.204:0/1324285885 >> 172.21.15.204:0/1324285885 conn(0x7efdec06e860 msgr2=0x7efdec0a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:43.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 1 -- 172.21.15.204:0/1324285885 shutdown_connections 2024-09-15T01:35:43.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.130+0000 7efdf4217640 1 -- 172.21.15.204:0/1324285885 wait complete. 2024-09-15T01:35:43.140 INFO:tasks.workunit.client.0.smithi204.stdout:(compat) 2024-09-15T01:35:43.140 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set rm-compat 2024-09-15T01:35:43.214 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.212+0000 7f2086e57640 10 monclient: get_monmap_and_config 2024-09-15T01:35:43.214 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.212+0000 7f2086e57640 10 monclient: build_initial_monmap 2024-09-15T01:35:43.214 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.212+0000 7f2086e57640 10 monclient: monmap: 2024-09-15T01:35:43.214 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:43.213279+0000 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:43.213279+0000 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:43.215 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:43.216 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:43.216 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:43.216 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:43.216 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:43.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.213+0000 7f2086e57640 1 Processor -- start 2024-09-15T01:35:43.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.213+0000 7f2086e57640 1 -- start start 2024-09-15T01:35:43.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.213+0000 7f2086e57640 10 monclient: init 2024-09-15T01:35:43.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:43.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 10 monclient: _add_conns ranks=[1,6,3,7,0,9,8,2,4,10,5] 2024-09-15T01:35:43.224 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f20800a2e30 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f208009fb80 0x7f208009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 10 monclient(hunting): picked mon.noname-d con 0x7f208009fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2080137d20 0x7f208013a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 10 monclient(hunting): picked mon.noname-g con 0x7f2080137d20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208004b720 con 0x7f20800a2e30 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208009e6c0 con 0x7f2080137d20 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f20800a04b0 con 0x7f208009fb80 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2086e57640 10 monclient(hunting): authenticate will time out at 958.856246s 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2084bf3640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f20800a2e30 0x7f2080132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:47988/0 (socket says 172.21.15.204:47988) 2024-09-15T01:35:43.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f2084bf3640 1 -- 172.21.15.204:0/3513542691 learned_addr learned my addr 172.21.15.204:0/3513542691 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:43.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 1 --2- 172.21.15.204:0/3513542691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2080137d20 0x7f208013a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:43.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f207ffff640 1 -- 172.21.15.204:0/3513542691 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f208009fb80 msgr2=0x7f208009ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:43.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 10 monclient(hunting): get_auth_request con 0x7f2080137d20 auth_method 0 2024-09-15T01:35:43.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:43.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:43.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:43.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f207ffff640 1 --2- 172.21.15.204:0/3513542691 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f208009fb80 0x7f208009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:43.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:43.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:43.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:43.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 10 monclient(hunting): handle_auth_done global_id 14826 payload 274 2024-09-15T01:35:43.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f207f7fe640 1 -- 172.21.15.204:0/3513542691 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247003709 0 0) 0x7f208004b720 con 0x7f20800a2e30 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 1 -- 172.21.15.204:0/3513542691 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f208009fb80 msgr2=0x7f208009ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 1 --2- 172.21.15.204:0/3513542691 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f208009fb80 0x7f208009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 1 -- 172.21.15.204:0/3513542691 >> v1:172.21.15.204:6789/0 conn(0x7f20800a2e30 legacy=0x7f2080132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 10 monclient: _finish_hunting 0 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 1 monclient: found mon.noname-g 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f20853f4640 1 -- 172.21.15.204:0/3513542691 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2080133170 con 0x7f2080137d20 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.214+0000 7f207f7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f20853f4640 1 --2- 172.21.15.204:0/3513542691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2080137d20 0x7f208013a100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2078009e90 tx=0x7f2078009780 comp rx=0 tx=0).ready entity=mon.3 client_cookie=baea782b03478a59 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f207f7fe640 1 -- 172.21.15.204:0/3513542691 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20780079d0 con 0x7f2080137d20 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f207f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:43.227 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f207f7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f207f7fe640 10 monclient: dump: 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:43.228 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f207f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f207f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f207f7fe640 1 -- 172.21.15.204:0/3513542691 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207802d050 con 0x7f2080137d20 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f2086e57640 5 monclient: authenticate success, global_id 14826 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f2086e57640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f207f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f207f7fe640 1 -- 172.21.15.204:0/3513542691 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2078031400 con 0x7f2080137d20 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f2086e57640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:43.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f2086e57640 4 set_mon_vals no callback set 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f207f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f207f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.215+0000 7f207f7fe640 10 monclient: dump: 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:43.230 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 10 monclient: shutdown 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 1 -- 172.21.15.204:0/3513542691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2080137d20 msgr2=0x7f208013a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 1 --2- 172.21.15.204:0/3513542691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2080137d20 0x7f208013a100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2078009e90 tx=0x7f2078009780 comp rx=0 tx=0).stop 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 1 -- 172.21.15.204:0/3513542691 shutdown_connections 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 1 --2- 172.21.15.204:0/3513542691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2080137d20 0x7f208013a100 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 1 --2- 172.21.15.204:0/3513542691 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f208009fb80 0x7f208009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 1 -- 172.21.15.204:0/3513542691 >> 172.21.15.204:0/3513542691 conn(0x7f208006e860 msgr2=0x7f208006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:43.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 1 -- 172.21.15.204:0/3513542691 shutdown_connections 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 1 -- 172.21.15.204:0/3513542691 wait complete. 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 10 monclient: build_initial_monmap 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 10 monclient: monmap: 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:43.232 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.216+0000 7f2086e57640 1 Processor -- start 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 1 -- start start 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 10 monclient: init 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 10 monclient: _add_conns ranks=[7,6,4,0,5,8,3,1,2] 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f208009fb80 0x7f2080098ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:43.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 10 monclient(hunting): picked mon.noname-i con 0x7f208009fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f20800a2e30 0x7f2080099530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 10 monclient(hunting): picked mon.noname-d con 0x7f20800a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2080137d20 0x7f2080099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 10 monclient(hunting): picked mon.noname-c con 0x7f2080137d20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20801309a0 con 0x7f2080137d20 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f208009d650 con 0x7f20800a2e30 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f208011c850 con 0x7f208009fb80 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2084bf3640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f208009fb80 msgr2=0x7f2080098ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:43.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2086e57640 10 monclient(hunting): authenticate will time out at 958.859557s 2024-09-15T01:35:43.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f2084bf3640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f208009fb80 0x7f2080098ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:43.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f207ffff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f20800a2e30 msgr2=0x7f2080099530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:43.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f207ffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f20800a2e30 0x7f2080099530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:43.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f20853f4640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2080137d20 0x7f2080099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:43.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f20853f4640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2080137d20 0x7f2080099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:58616/0 (socket says 172.21.15.204:58616) 2024-09-15T01:35:43.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f20853f4640 1 -- 172.21.15.204:0/3339439293 learned_addr learned my addr 172.21.15.204:0/3339439293 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:43.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f20853f4640 10 monclient(hunting): get_auth_request con 0x7f2080137d20 auth_method 0 2024-09-15T01:35:43.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f20853f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:43.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f20853f4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:43.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.217+0000 7f20853f4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:43.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:43.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 10 monclient(hunting): handle_auth_done global_id 5647 payload 824 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 1 -- 172.21.15.204:0/3339439293 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f208009fb80 msgr2=0x7f2080098ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 1 --2- 172.21.15.204:0/3339439293 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f208009fb80 0x7f2080098ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 1 -- 172.21.15.204:0/3339439293 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f20800a2e30 msgr2=0x7f2080099530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 1 --2- 172.21.15.204:0/3339439293 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f20800a2e30 0x7f2080099530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 10 monclient: _finish_hunting 0 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 1 monclient: found mon.noname-c 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 1 -- 172.21.15.204:0/3339439293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078009a20 con 0x7f2080137d20 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f20853f4640 1 --2- 172.21.15.204:0/3339439293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2080137d20 0x7f2080099a70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2078009ef0 tx=0x7f2078037000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:43.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207cff9640 1 -- 172.21.15.204:0/3339439293 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f207803c420 con 0x7f2080137d20 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207cff9640 10 monclient: dump: 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:43.237 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207cff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207cff9640 1 -- 172.21.15.204:0/3339439293 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207803c5a0 con 0x7f2080137d20 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f2086e57640 5 monclient: authenticate success, global_id 5647 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f2086e57640 10 monclient: _renew_subs 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207cff9640 1 -- 172.21.15.204:0/3339439293 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f207803cb90 con 0x7f2080137d20 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f2086e57640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:43.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f2086e57640 1 -- 172.21.15.204:0/3339439293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208009c610 con 0x7f2080137d20 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207cff9640 10 monclient: dump: 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:43.239 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f207effd640 4 set_mon_vals no callback set 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f2086e57640 10 monclient: _renew_subs 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f2086e57640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.218+0000 7f2086e57640 1 -- 172.21.15.204:0/3339439293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208009ca90 con 0x7f2080137d20 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.219+0000 7f2086e57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.219+0000 7f2086e57640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.219+0000 7f2086e57640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.219+0000 7f2086e57640 1 -- 172.21.15.204:0/3339439293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208009aae0 con 0x7f2080137d20 2024-09-15T01:35:43.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.219+0000 7f207cff9640 1 -- 172.21.15.204:0/3339439293 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2078003040 con 0x7f2080137d20 2024-09-15T01:35:43.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.220+0000 7f207cff9640 1 --2- 172.21.15.204:0/3339439293 >> v2:172.21.15.204:6812/28210 conn(0x7f206c0396e0 0x7f206c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:43.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.220+0000 7f207cff9640 1 -- 172.21.15.204:0/3339439293 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6705+0+0 (secure 0 0 0) 0x7f207806d040 con 0x7f2080137d20 2024-09-15T01:35:43.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.220+0000 7f2084bf3640 1 --2- 172.21.15.204:0/3339439293 >> v2:172.21.15.204:6812/28210 conn(0x7f206c0396e0 0x7f206c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:43.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.220+0000 7f2084bf3640 10 monclient: get_auth_request con 0x7f206c0396e0 auth_method 0 2024-09-15T01:35:43.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.220+0000 7f2084bf3640 1 --2- 172.21.15.204:0/3339439293 >> v2:172.21.15.204:6812/28210 conn(0x7f206c0396e0 0x7f206c03bb90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2070007fd0 tx=0x7f2070007a90 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:43.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.224+0000 7f207cff9640 1 -- 172.21.15.204:0/3339439293 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2078031ac0 con 0x7f2080137d20 2024-09-15T01:35:43.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.224+0000 7f207cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:43.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.224+0000 7f207cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:43.387 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.386+0000 7f2086e57640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:35:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.386+0000 7f2086e57640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:35:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.386+0000 7f2086e57640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:43.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.386+0000 7f2086e57640 1 -- 172.21.15.204:0/3339439293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7f208009b050 con 0x7f2080137d20 2024-09-15T01:35:43.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.668+0000 7f207cff9640 1 -- 172.21.15.204:0/3339439293 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v137) v1 ==== 78+0+0 (secure 0 0 0) 0x7f208009b050 con 0x7f2080137d20 2024-09-15T01:35:43.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.668+0000 7f207cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:35:43.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.668+0000 7f207cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:43.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.770+0000 7f2086e57640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:35:43.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.770+0000 7f2086e57640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:35:43.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.770+0000 7f2086e57640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:43.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:43.770+0000 7f2086e57640 1 -- 172.21.15.204:0/3339439293 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7f208009b050 con 0x7f2080137d20 2024-09-15T01:35:44.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.686+0000 7f207cff9640 1 -- 172.21.15.204:0/3339439293 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v138) v1 ==== 78+0+0 (secure 0 0 0) 0x7f208009b050 con 0x7f2080137d20 2024-09-15T01:35:44.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.686+0000 7f207cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:35:44.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.686+0000 7f207cff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:44.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 1 -- 172.21.15.204:0/3339439293 >> v2:172.21.15.204:6812/28210 conn(0x7f206c0396e0 msgr2=0x7f206c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:44.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 1 --2- 172.21.15.204:0/3339439293 >> v2:172.21.15.204:6812/28210 conn(0x7f206c0396e0 0x7f206c03bb90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2070007fd0 tx=0x7f2070007a90 comp rx=0 tx=0).stop 2024-09-15T01:35:44.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 10 monclient: shutdown 2024-09-15T01:35:44.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:44.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 1 -- 172.21.15.204:0/3339439293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2080137d20 msgr2=0x7f2080099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:44.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 1 --2- 172.21.15.204:0/3339439293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2080137d20 0x7f2080099a70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2078009ef0 tx=0x7f2078037000 comp rx=0 tx=0).stop 2024-09-15T01:35:44.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 1 -- 172.21.15.204:0/3339439293 shutdown_connections 2024-09-15T01:35:44.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 1 --2- 172.21.15.204:0/3339439293 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2080137d20 0x7f2080099a70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:44.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 1 --2- 172.21.15.204:0/3339439293 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f20800a2e30 0x7f2080099530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:44.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 1 --2- 172.21.15.204:0/3339439293 >> v2:172.21.15.204:6812/28210 conn(0x7f206c0396e0 0x7f206c03bb90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:44.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 1 --2- 172.21.15.204:0/3339439293 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f208009fb80 0x7f2080098ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:44.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 1 -- 172.21.15.204:0/3339439293 >> 172.21.15.204:0/3339439293 conn(0x7f208006e860 msgr2=0x7f20800a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:44.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 1 -- 172.21.15.204:0/3339439293 shutdown_connections 2024-09-15T01:35:44.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.688+0000 7f2086e57640 1 -- 172.21.15.204:0/3339439293 wait complete. 2024-09-15T01:35:44.700 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool create cool 2 2024-09-15T01:35:44.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.779+0000 7f0d8fc2b640 10 monclient: get_monmap_and_config 2024-09-15T01:35:44.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.779+0000 7f0d8fc2b640 10 monclient: build_initial_monmap 2024-09-15T01:35:44.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.779+0000 7f0d8fc2b640 10 monclient: monmap: 2024-09-15T01:35:44.781 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:44.781 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:44.781 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:44.780045+0000 2024-09-15T01:35:44.781 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:44.780045+0000 2024-09-15T01:35:44.781 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:44.781 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:44.781 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:44.782 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:44.782 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:44.782 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:44.782 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:44.782 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:44.782 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:44.782 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:44.782 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:44.782 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:44.782 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:44.782 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:44.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 1 Processor -- start 2024-09-15T01:35:44.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 1 -- start start 2024-09-15T01:35:44.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 10 monclient: init 2024-09-15T01:35:44.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:44.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 10 monclient: _add_conns ranks=[0,8,3,1,4,10,9,6,2,7,5] 2024-09-15T01:35:44.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0d880a2e30 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:44.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8809fb80 0x7f0d8809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:44.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 10 monclient(hunting): picked mon.noname-e con 0x7f0d8809fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d88137d20 0x7f0d8813a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 10 monclient(hunting): picked mon.noname-g con 0x7f0d88137d20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8804b720 con 0x7f0d880a2e30 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8809e6c0 con 0x7f0d88137d20 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880a04b0 con 0x7f0d8809fb80 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8fc2b640 10 monclient(hunting): authenticate will time out at 960.422817s 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8e1c8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d88137d20 0x7f0d8813a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8e1c8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d88137d20 0x7f0d8813a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:47378/0 (socket says 172.21.15.204:47378) 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8e1c8640 1 -- 172.21.15.204:0/3542253196 learned_addr learned my addr 172.21.15.204:0/3542253196 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:44.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8e1c8640 10 monclient(hunting): get_auth_request con 0x7f0d88137d20 auth_method 0 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8e1c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8e1c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.780+0000 7f0d8e1c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8d1c6640 1 --2- 172.21.15.204:0/3542253196 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8809fb80 0x7f0d8809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8d1c6640 10 monclient(hunting): get_auth_request con 0x7f0d8809fb80 auth_method 0 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8d1c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8d1c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8d1c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 10 monclient(hunting): handle_auth_done global_id 14835 payload 274 2024-09-15T01:35:44.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 1 -- 172.21.15.204:0/3542253196 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8809fb80 msgr2=0x7f0d8809ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 1 --2- 172.21.15.204:0/3542253196 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8809fb80 0x7f0d8809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 1 -- 172.21.15.204:0/3542253196 >> v1:172.21.15.178:6789/0 conn(0x7f0d880a2e30 legacy=0x7f0d88132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 10 monclient: _finish_hunting 0 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 1 monclient: found mon.noname-g 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 1 -- 172.21.15.204:0/3542253196 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88133170 con 0x7f0d88137d20 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.781+0000 7f0d8e1c8640 1 --2- 172.21.15.204:0/3542253196 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d88137d20 0x7f0d8813a100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009f50 tx=0x7f0d80009840 comp rx=0 tx=0).ready entity=mon.3 client_cookie=24905648ede0f4e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8c9c5640 1 -- 172.21.15.204:0/3542253196 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d80007880 con 0x7f0d88137d20 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8c9c5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8c9c5640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8c9c5640 10 monclient: dump: 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:44.793 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:44.794 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:44.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8c9c5640 10 monclient: _finish_auth 0 2024-09-15T01:35:44.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8c9c5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:44.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8c9c5640 1 -- 172.21.15.204:0/3542253196 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8002d050 con 0x7f0d88137d20 2024-09-15T01:35:44.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 5 monclient: authenticate success, global_id 14835 2024-09-15T01:35:44.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:44.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8c9c5640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:44.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8c9c5640 1 -- 172.21.15.204:0/3542253196 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d80030360 con 0x7f0d88137d20 2024-09-15T01:35:44.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8c9c5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8c9c5640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8c9c5640 10 monclient: dump: 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:44.796 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 4 set_mon_vals no callback set 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 10 monclient: shutdown 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/3542253196 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d88137d20 msgr2=0x7f0d8813a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 1 --2- 172.21.15.204:0/3542253196 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d88137d20 0x7f0d8813a100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009f50 tx=0x7f0d80009840 comp rx=0 tx=0).stop 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/3542253196 shutdown_connections 2024-09-15T01:35:44.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 1 --2- 172.21.15.204:0/3542253196 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d88137d20 0x7f0d8813a100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:44.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 1 --2- 172.21.15.204:0/3542253196 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8809fb80 0x7f0d8809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:44.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/3542253196 >> 172.21.15.204:0/3542253196 conn(0x7f0d8806e860 msgr2=0x7f0d8806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:44.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/3542253196 shutdown_connections 2024-09-15T01:35:44.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.782+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/3542253196 wait complete. 2024-09-15T01:35:44.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.783+0000 7f0d8fc2b640 10 monclient: build_initial_monmap 2024-09-15T01:35:44.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.783+0000 7f0d8fc2b640 10 monclient: monmap: 2024-09-15T01:35:44.798 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:44.798 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:44.798 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:44.798 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.783+0000 7f0d8fc2b640 1 Processor -- start 2024-09-15T01:35:44.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.783+0000 7f0d8fc2b640 1 -- start start 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.783+0000 7f0d8fc2b640 10 monclient: init 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 10 monclient: _add_conns ranks=[8,1,4,0,3,7,2,6,5] 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8809fb80 0x7f0d8811d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 10 monclient(hunting): picked mon.noname-e con 0x7f0d8809fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d880a2e30 0x7f0d88099140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 10 monclient(hunting): picked mon.noname-f con 0x7f0d880a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0d88137d20 0x7f0d88099680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 10 monclient(hunting): picked mon.noname-c con 0x7f0d88137d20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8811d5d0 con 0x7f0d880a2e30 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:44.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8811ca20 con 0x7f0d88137d20 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8809b860 con 0x7f0d8809fb80 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 10 monclient(hunting): authenticate will time out at 960.426111s 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d880a2e30 0x7f0d88099140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d880a2e30 0x7f0d88099140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:56358/0 (socket says 172.21.15.204:56358) 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 1 -- 172.21.15.204:0/2465971881 learned_addr learned my addr 172.21.15.204:0/2465971881 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 10 monclient(hunting): get_auth_request con 0x7f0d880a2e30 auth_method 0 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d9c7640 1 --2- 172.21.15.204:0/2465971881 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8809fb80 0x7f0d8811d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:44.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8e1c8640 1 --2- 172.21.15.204:0/2465971881 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0d88137d20 0x7f0d88099680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d9c7640 10 monclient(hunting): get_auth_request con 0x7f0d8809fb80 auth_method 0 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d9c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d9c7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d9c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8e1c8640 10 monclient(hunting): get_auth_request con 0x7f0d88137d20 auth_method 0 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8e1c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8e1c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8e1c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d9c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d9c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:44.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d9c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8e1c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8e1c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8e1c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 10 monclient(hunting): handle_auth_done global_id 6328 payload 824 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 1 -- 172.21.15.204:0/2465971881 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8809fb80 msgr2=0x7f0d8811d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 1 --2- 172.21.15.204:0/2465971881 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8809fb80 0x7f0d8811d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 1 -- 172.21.15.204:0/2465971881 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0d88137d20 msgr2=0x7f0d88099680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 1 --2- 172.21.15.204:0/2465971881 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0d88137d20 0x7f0d88099680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 10 monclient: _finish_hunting 0 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 1 monclient: found mon.noname-f 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 1 -- 172.21.15.204:0/2465971881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d80009ae0 con 0x7f0d880a2e30 2024-09-15T01:35:44.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8d1c6640 1 --2- 172.21.15.204:0/2465971881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d880a2e30 0x7f0d88099140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800d8c0 tx=0x7f0d78008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7dffb640 1 -- 172.21.15.204:0/2465971881 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d7800de50 con 0x7f0d880a2e30 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7dffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7dffb640 10 monclient: dump: 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:44.804 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7dffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7dffb640 1 -- 172.21.15.204:0/2465971881 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7800bd50 con 0x7f0d880a2e30 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 5 monclient: authenticate success, global_id 6328 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 10 monclient: _renew_subs 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:44.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/2465971881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8809d650 con 0x7f0d880a2e30 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7dffb640 1 -- 172.21.15.204:0/2465971881 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d7800bec0 con 0x7f0d880a2e30 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7dffb640 10 monclient: dump: 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:44.806 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.785+0000 7f0d8fc2b640 10 monclient: _renew_subs 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.784+0000 7f0d7ffff640 4 set_mon_vals no callback set 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.785+0000 7f0d8fc2b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.785+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/2465971881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8809ac90 con 0x7f0d880a2e30 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.785+0000 7f0d7dffb640 1 -- 172.21.15.204:0/2465971881 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0d7800b8d0 con 0x7f0d880a2e30 2024-09-15T01:35:44.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.786+0000 7f0d8fc2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:44.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.786+0000 7f0d8fc2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:44.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.786+0000 7f0d8fc2b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:44.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.786+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/2465971881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8811c6b0 con 0x7f0d880a2e30 2024-09-15T01:35:44.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.786+0000 7f0d7dffb640 1 --2- 172.21.15.204:0/2465971881 >> v2:172.21.15.204:6812/28210 conn(0x7f0d6c0396e0 0x7f0d6c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:44.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.786+0000 7f0d7dffb640 1 -- 172.21.15.204:0/2465971881 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6373+0+0 (secure 0 0 0) 0x7f0d7804bcf0 con 0x7f0d880a2e30 2024-09-15T01:35:44.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.786+0000 7f0d8d9c7640 1 --2- 172.21.15.204:0/2465971881 >> v2:172.21.15.204:6812/28210 conn(0x7f0d6c0396e0 0x7f0d6c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:44.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.786+0000 7f0d8d9c7640 10 monclient: get_auth_request con 0x7f0d6c0396e0 auth_method 0 2024-09-15T01:35:44.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.786+0000 7f0d8d9c7640 1 --2- 172.21.15.204:0/2465971881 >> v2:172.21.15.204:6812/28210 conn(0x7f0d6c0396e0 0x7f0d6c03bb90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d84007fd0 tx=0x7f0d84000bc0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:44.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.790+0000 7f0d7dffb640 1 -- 172.21.15.204:0/2465971881 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0d7800bb80 con 0x7f0d880a2e30 2024-09-15T01:35:44.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.790+0000 7f0d7dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:44.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.790+0000 7f0d7dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:44.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.945+0000 7f0d8fc2b640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-09-15T01:35:44.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.945+0000 7f0d8fc2b640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-09-15T01:35:44.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.945+0000 7f0d8fc2b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:44.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:44.945+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/2465971881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cool", "pg_num": 2} v 0) v1 -- 0x7f0d88070cc0 con 0x7f0d880a2e30 2024-09-15T01:35:45.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.680+0000 7f0d7dffb640 1 -- 172.21.15.204:0/2465971881 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}]=0 pool 'cool' created v139) v1 ==== 111+0+0 (secure 0 0 0) 0x7f0d78011340 con 0x7f0d880a2e30 2024-09-15T01:35:45.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.680+0000 7f0d7dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-09-15T01:35:45.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.680+0000 7f0d7dffb640 10 monclient: _finish_command 2 = system:0 pool 'cool' created 2024-09-15T01:35:45.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.778+0000 7f0d8fc2b640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-09-15T01:35:45.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.778+0000 7f0d8fc2b640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-09-15T01:35:45.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.778+0000 7f0d8fc2b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:45.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.778+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/2465971881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cool", "pg_num": 2} v 0) v1 -- 0x7f0d88070cc0 con 0x7f0d880a2e30 2024-09-15T01:35:45.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.778+0000 7f0d7dffb640 1 -- 172.21.15.204:0/2465971881 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}]=0 pool 'cool' already exists v139) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0d7801c370 con 0x7f0d880a2e30 2024-09-15T01:35:45.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.778+0000 7f0d7dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-09-15T01:35:45.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.778+0000 7f0d7dffb640 10 monclient: _finish_command 3 = system:0 pool 'cool' already exists 2024-09-15T01:35:45.780 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'cool' already exists 2024-09-15T01:35:45.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.780+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/2465971881 >> v2:172.21.15.204:6812/28210 conn(0x7f0d6c0396e0 msgr2=0x7f0d6c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:45.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.780+0000 7f0d8fc2b640 1 --2- 172.21.15.204:0/2465971881 >> v2:172.21.15.204:6812/28210 conn(0x7f0d6c0396e0 0x7f0d6c03bb90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d84007fd0 tx=0x7f0d84000bc0 comp rx=0 tx=0).stop 2024-09-15T01:35:45.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.780+0000 7f0d8fc2b640 10 monclient: shutdown 2024-09-15T01:35:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.780+0000 7f0d8fc2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.780+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/2465971881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d880a2e30 msgr2=0x7f0d88099140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.780+0000 7f0d8fc2b640 1 --2- 172.21.15.204:0/2465971881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d880a2e30 0x7f0d88099140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800d8c0 tx=0x7f0d78008040 comp rx=0 tx=0).stop 2024-09-15T01:35:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.780+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/2465971881 shutdown_connections 2024-09-15T01:35:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.780+0000 7f0d8fc2b640 1 --2- 172.21.15.204:0/2465971881 >> v2:172.21.15.204:6812/28210 conn(0x7f0d6c0396e0 0x7f0d6c03bb90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.780+0000 7f0d8fc2b640 1 --2- 172.21.15.204:0/2465971881 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0d88137d20 0x7f0d88099680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.781+0000 7f0d8fc2b640 1 --2- 172.21.15.204:0/2465971881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d880a2e30 0x7f0d88099140 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.781+0000 7f0d8fc2b640 1 --2- 172.21.15.204:0/2465971881 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8809fb80 0x7f0d8811d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.781+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/2465971881 >> 172.21.15.204:0/2465971881 conn(0x7f0d8806e860 msgr2=0x7f0d880a55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.781+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/2465971881 shutdown_connections 2024-09-15T01:35:45.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.781+0000 7f0d8fc2b640 1 -- 172.21.15.204:0/2465971881 wait complete. 2024-09-15T01:35:45.791 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool create cold 2 2024-09-15T01:35:45.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.865+0000 7fc7bcb5d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:45.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.865+0000 7fc7bcb5d640 10 monclient: build_initial_monmap 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.865+0000 7fc7bcb5d640 10 monclient: monmap: 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:45.865542+0000 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:45.865542+0000 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:45.867 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:45.868 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:45.868 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:45.868 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:45.868 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:45.868 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:45.868 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:45.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.865+0000 7fc7bcb5d640 1 Processor -- start 2024-09-15T01:35:45.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.865+0000 7fc7bcb5d640 1 -- start start 2024-09-15T01:35:45.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.865+0000 7fc7bcb5d640 10 monclient: init 2024-09-15T01:35:45.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:45.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 10 monclient: _add_conns ranks=[8,2,0,5,1,7,9,10,4,3,6] 2024-09-15T01:35:45.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7b806e840 0x7fc7b8130870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:45.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 10 monclient(hunting): picked mon.noname-e con 0x7fc7b806e840 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:45.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b8130db0 0x7fc7b8131180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:45.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 10 monclient(hunting): picked mon.noname-b con 0x7fc7b8130db0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:45.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc7b809e9c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b804b720 con 0x7fc7b809e9c0 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b806dd60 con 0x7fc7b8130db0 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b806da80 con 0x7fc7b806e840 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7bcb5d640 10 monclient(hunting): authenticate will time out at 961.508304s 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7add9b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b8130db0 msgr2=0x7fc7b8131180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7add9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b8130db0 0x7fc7b8131180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7b806e840 0x7fc7b8130870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b6d9d640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc7b809e9c0 0x7fc7b809edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.204:48364/0 (socket says 172.21.15.204:48364) 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7b806e840 0x7fc7b8130870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:33154/0 (socket says 172.21.15.204:33154) 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b6d9d640 1 -- 172.21.15.204:0/2222035614 learned_addr learned my addr 172.21.15.204:0/2222035614 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:45.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 10 monclient(hunting): get_auth_request con 0x7fc7b806e840 auth_method 0 2024-09-15T01:35:45.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:45.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:45.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:45.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:45.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:45.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:45.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 10 monclient(hunting): handle_auth_done global_id 5723 payload 274 2024-09-15T01:35:45.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b5d9b640 1 -- 172.21.15.204:0/2222035614 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749744315 0 0) 0x7fc7b804b720 con 0x7fc7b809e9c0 2024-09-15T01:35:45.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 1 -- 172.21.15.204:0/2222035614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b8130db0 msgr2=0x7fc7b8131180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:45.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 1 --2- 172.21.15.204:0/2222035614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b8130db0 0x7fc7b8131180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:45.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 1 -- 172.21.15.204:0/2222035614 >> v1:172.21.15.178:6789/0 conn(0x7fc7b809e9c0 legacy=0x7fc7b809edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:45.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 10 monclient: _finish_hunting 0 2024-09-15T01:35:45.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 1 monclient: found mon.noname-e 2024-09-15T01:35:45.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:45.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:45.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b659c640 1 -- 172.21.15.204:0/2222035614 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b81316c0 con 0x7fc7b806e840 2024-09-15T01:35:45.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.866+0000 7fc7b5d9b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:35:45.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b659c640 1 --2- 172.21.15.204:0/2222035614 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7b806e840 0x7fc7b8130870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8131850 tx=0x7fc7a80090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6213f0030d9696b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:45.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b5d9b640 1 -- 172.21.15.204:0/2222035614 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7a802ace0 con 0x7fc7b806e840 2024-09-15T01:35:45.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:45.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b5d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b5d9b640 10 monclient: dump: 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:45.882 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b5d9b640 10 monclient: _finish_auth 0 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7bcb5d640 5 monclient: authenticate success, global_id 5723 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7bcb5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b5d9b640 1 -- 172.21.15.204:0/2222035614 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a802ae60 con 0x7fc7b806e840 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b5d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7bcb5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7bcb5d640 4 set_mon_vals no callback set 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b5d9b640 1 -- 172.21.15.204:0/2222035614 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7a8031ca0 con 0x7fc7b806e840 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:45.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b5d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.867+0000 7fc7b5d9b640 10 monclient: dump: 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:45.884 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:45.885 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:45.885 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:45.885 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:45.885 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:45.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.868+0000 7fc7bcb5d640 10 monclient: shutdown 2024-09-15T01:35:45.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.868+0000 7fc7bcb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:45.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.868+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/2222035614 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7b806e840 msgr2=0x7fc7b8130870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:45.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.868+0000 7fc7bcb5d640 1 --2- 172.21.15.204:0/2222035614 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7b806e840 0x7fc7b8130870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8131850 tx=0x7fc7a80090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:45.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.868+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/2222035614 shutdown_connections 2024-09-15T01:35:45.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.868+0000 7fc7bcb5d640 1 --2- 172.21.15.204:0/2222035614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b8130db0 0x7fc7b8131180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:45.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.868+0000 7fc7bcb5d640 1 --2- 172.21.15.204:0/2222035614 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7b806e840 0x7fc7b8130870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:45.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.868+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/2222035614 >> 172.21.15.204:0/2222035614 conn(0x7fc7b8067c00 msgr2=0x7fc7b80a3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:45.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.868+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/2222035614 shutdown_connections 2024-09-15T01:35:45.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.868+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/2222035614 wait complete. 2024-09-15T01:35:45.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient: build_initial_monmap 2024-09-15T01:35:45.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient: monmap: 2024-09-15T01:35:45.886 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:45.886 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:45.887 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:45.888 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:45.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 1 Processor -- start 2024-09-15T01:35:45.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 1 -- start start 2024-09-15T01:35:45.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient: init 2024-09-15T01:35:45.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:45.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient: _add_conns ranks=[0,2,3,5,8,6,4,1,7] 2024-09-15T01:35:45.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b806e840 0x7fc7b811c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:45.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient(hunting): picked mon.noname-a con 0x7fc7b806e840 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:45.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b809e9c0 0x7fc7b811c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:45.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient(hunting): picked mon.noname-b con 0x7fc7b809e9c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:45.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7b8130db0 0x7fc7b811ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient(hunting): picked mon.noname-g con 0x7fc7b8130db0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b811fbf0 con 0x7fc7b806e840 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b806d1a0 con 0x7fc7b809e9c0 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b806cdd0 con 0x7fc7b8130db0 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7bcb5d640 10 monclient(hunting): authenticate will time out at 961.511914s 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.869+0000 7fc7add9b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b809e9c0 msgr2=0x7fc7b811c8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7b8130db0 0x7fc7b811ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7add9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b809e9c0 0x7fc7b811c8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7b8130db0 0x7fc7b811ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:47392/0 (socket says 172.21.15.204:47392) 2024-09-15T01:35:45.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 1 -- 172.21.15.204:0/1146966381 learned_addr learned my addr 172.21.15.204:0/1146966381 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 10 monclient(hunting): get_auth_request con 0x7fc7b8130db0 auth_method 0 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b659c640 1 --2- 172.21.15.204:0/1146966381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b806e840 0x7fc7b811c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b659c640 10 monclient(hunting): get_auth_request con 0x7fc7b806e840 auth_method 0 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:45.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 10 monclient(hunting): handle_auth_done global_id 14844 payload 824 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 1 -- 172.21.15.204:0/1146966381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b809e9c0 msgr2=0x7fc7b811c8c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 1 --2- 172.21.15.204:0/1146966381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b809e9c0 0x7fc7b811c8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 1 -- 172.21.15.204:0/1146966381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b806e840 msgr2=0x7fc7b811c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 1 --2- 172.21.15.204:0/1146966381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b806e840 0x7fc7b811c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 1 monclient: found mon.noname-g 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 1 -- 172.21.15.204:0/1146966381 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a8009390 con 0x7fc7b8130db0 2024-09-15T01:35:45.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b6d9d640 1 --2- 172.21.15.204:0/1146966381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7b8130db0 0x7fc7b811ce00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000ccc0 tx=0x7fc7b0001a20 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:45.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7aeffd640 1 -- 172.21.15.204:0/1146966381 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7b000acd0 con 0x7fc7b8130db0 2024-09-15T01:35:45.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:45.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7aeffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:45.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7aeffd640 10 monclient: dump: 2024-09-15T01:35:45.892 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:45.892 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:45.892 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:45.892 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:45.892 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:45.892 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:45.892 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:45.892 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7aeffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7bcb5d640 5 monclient: authenticate success, global_id 14844 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7aeffd640 1 -- 172.21.15.204:0/1146966381 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b000ae50 con 0x7fc7b8130db0 2024-09-15T01:35:45.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7bcb5d640 10 monclient: _renew_subs 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7bcb5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/1146966381 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b811f300 con 0x7fc7b8130db0 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7aeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7aeffd640 1 -- 172.21.15.204:0/1146966381 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7b0007570 con 0x7fc7b8130db0 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7aeffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7b559a640 4 set_mon_vals no callback set 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7aeffd640 10 monclient: dump: 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:45.894 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7bcb5d640 10 monclient: _renew_subs 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7bcb5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:45.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.870+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/1146966381 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b811da70 con 0x7fc7b8130db0 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.871+0000 7fc7aeffd640 1 -- 172.21.15.204:0/1146966381 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc7b00070f0 con 0x7fc7b8130db0 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.871+0000 7fc7bcb5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.871+0000 7fc7bcb5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.871+0000 7fc7bcb5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.871+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/1146966381 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b811e730 con 0x7fc7b8130db0 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.871+0000 7fc7aeffd640 1 --2- 172.21.15.204:0/1146966381 >> v2:172.21.15.204:6812/28210 conn(0x7fc7980397b0 0x7fc79803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.871+0000 7fc7aeffd640 1 -- 172.21.15.204:0/1146966381 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6716+0+0 (secure 0 0 0) 0x7fc7b004bd10 con 0x7fc7b8130db0 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.871+0000 7fc7b659c640 1 --2- 172.21.15.204:0/1146966381 >> v2:172.21.15.204:6812/28210 conn(0x7fc7980397b0 0x7fc79803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.871+0000 7fc7b659c640 10 monclient: get_auth_request con 0x7fc7980397b0 auth_method 0 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.872+0000 7fc7b659c640 1 --2- 172.21.15.204:0/1146966381 >> v2:172.21.15.204:6812/28210 conn(0x7fc7980397b0 0x7fc79803bc60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8034000 tx=0x7fc7a8032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.876+0000 7fc7aeffd640 1 -- 172.21.15.204:0/1146966381 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc7b00076d0 con 0x7fc7b8130db0 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.876+0000 7fc7aeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:45.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:45.876+0000 7fc7aeffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:46.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.033+0000 7fc7bcb5d640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-09-15T01:35:46.034 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.033+0000 7fc7bcb5d640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-09-15T01:35:46.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.033+0000 7fc7bcb5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:46.035 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.033+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/1146966381 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cold", "pg_num": 2} v 0) v1 -- 0x7fc7b8070cc0 con 0x7fc7b8130db0 2024-09-15T01:35:46.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.697+0000 7fc7aeffd640 1 -- 172.21.15.204:0/1146966381 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}]=0 pool 'cold' created v140) v1 ==== 111+0+0 (secure 0 0 0) 0x7fc7b0010300 con 0x7fc7b8130db0 2024-09-15T01:35:46.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.697+0000 7fc7aeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-09-15T01:35:46.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.697+0000 7fc7aeffd640 10 monclient: _finish_command 2 = system:0 pool 'cold' created 2024-09-15T01:35:46.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.795+0000 7fc7bcb5d640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-09-15T01:35:46.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.795+0000 7fc7bcb5d640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-09-15T01:35:46.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.795+0000 7fc7bcb5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:46.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.795+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/1146966381 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cold", "pg_num": 2} v 0) v1 -- 0x7fc7b8120830 con 0x7fc7b8130db0 2024-09-15T01:35:46.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.796+0000 7fc7aeffd640 1 -- 172.21.15.204:0/1146966381 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}]=0 pool 'cold' already exists v140) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc7b00104e0 con 0x7fc7b8130db0 2024-09-15T01:35:46.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.796+0000 7fc7aeffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-09-15T01:35:46.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.796+0000 7fc7aeffd640 10 monclient: _finish_command 3 = system:0 pool 'cold' already exists 2024-09-15T01:35:46.798 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'cold' already exists 2024-09-15T01:35:46.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/1146966381 >> v2:172.21.15.204:6812/28210 conn(0x7fc7980397b0 msgr2=0x7fc79803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:46.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 1 --2- 172.21.15.204:0/1146966381 >> v2:172.21.15.204:6812/28210 conn(0x7fc7980397b0 0x7fc79803bc60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8034000 tx=0x7fc7a8032000 comp rx=0 tx=0).stop 2024-09-15T01:35:46.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 10 monclient: shutdown 2024-09-15T01:35:46.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:46.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/1146966381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7b8130db0 msgr2=0x7fc7b811ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:46.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 1 --2- 172.21.15.204:0/1146966381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7b8130db0 0x7fc7b811ce00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000ccc0 tx=0x7fc7b0001a20 comp rx=0 tx=0).stop 2024-09-15T01:35:46.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/1146966381 shutdown_connections 2024-09-15T01:35:46.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 1 --2- 172.21.15.204:0/1146966381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7b8130db0 0x7fc7b811ce00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:46.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 1 --2- 172.21.15.204:0/1146966381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b809e9c0 0x7fc7b811c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:46.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 1 --2- 172.21.15.204:0/1146966381 >> v2:172.21.15.204:6812/28210 conn(0x7fc7980397b0 0x7fc79803bc60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:46.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 1 --2- 172.21.15.204:0/1146966381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b806e840 0x7fc7b811c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:46.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/1146966381 >> 172.21.15.204:0/1146966381 conn(0x7fc7b8067c00 msgr2=0x7fc7b80969e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:46.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/1146966381 shutdown_connections 2024-09-15T01:35:46.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.798+0000 7fc7bcb5d640 1 -- 172.21.15.204:0/1146966381 wait complete. 2024-09-15T01:35:46.808 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool set cold size 2 2024-09-15T01:35:46.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.883+0000 7ff7c4154640 10 monclient: get_monmap_and_config 2024-09-15T01:35:46.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.883+0000 7ff7c4154640 10 monclient: build_initial_monmap 2024-09-15T01:35:46.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.883+0000 7ff7c4154640 10 monclient: monmap: 2024-09-15T01:35:46.885 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:46.884069+0000 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:46.884069+0000 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:46.886 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:46.887 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:46.887 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:46.887 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:46.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 1 Processor -- start 2024-09-15T01:35:46.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 1 -- start start 2024-09-15T01:35:46.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 10 monclient: init 2024-09-15T01:35:46.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:46.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 10 monclient: _add_conns ranks=[2,4,10,8,5,7,3,9,6,1,0] 2024-09-15T01:35:46.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7bc0a2cd0 0x7ff7bc132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:46.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 10 monclient(hunting): picked mon.noname-b con 0x7ff7bc0a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:46.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff7bc09fa20 0x7ff7bc09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:46.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 10 monclient(hunting): picked mon.noname-c con 0x7ff7bc09fa20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:46.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff7bc0a08e0 0x7ff7bc0a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:46.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 10 monclient(hunting): picked mon.noname-f con 0x7ff7bc0a08e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:46.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:46.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7bc0a30a0 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc0a0330 con 0x7ff7bc0a2cd0 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc0a1db0 con 0x7ff7bc09fa20 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c4154640 10 monclient(hunting): authenticate will time out at 962.526834s 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c26f1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff7bc0a08e0 0x7ff7bc0a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.884+0000 7ff7c1ef0640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7bc0a2cd0 msgr2=0x7ff7bc132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c1ef0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7bc0a2cd0 0x7ff7bc132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff7bc0a08e0 0x7ff7bc0a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:56374/0 (socket says 172.21.15.204:56374) 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 1 -- 172.21.15.204:0/798131643 learned_addr learned my addr 172.21.15.204:0/798131643 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 10 monclient(hunting): get_auth_request con 0x7ff7bc0a08e0 auth_method 0 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:46.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c16ef640 1 --2- 172.21.15.204:0/798131643 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff7bc09fa20 0x7ff7bc09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c16ef640 10 monclient(hunting): get_auth_request con 0x7ff7bc09fa20 auth_method 0 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c16ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c16ef640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c16ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 10 monclient(hunting): handle_auth_done global_id 6337 payload 274 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 1 -- 172.21.15.204:0/798131643 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff7bc09fa20 msgr2=0x7ff7bc09fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 1 --2- 172.21.15.204:0/798131643 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff7bc09fa20 0x7ff7bc09fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:46.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 1 -- 172.21.15.204:0/798131643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7bc0a2cd0 msgr2=0x7ff7bc132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 1 --2- 172.21.15.204:0/798131643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7bc0a2cd0 0x7ff7bc132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 10 monclient: _finish_hunting 0 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 1 monclient: found mon.noname-f 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 1 -- 172.21.15.204:0/798131643 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc0648a0 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.885+0000 7ff7c26f1640 1 --2- 172.21.15.204:0/798131643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7bc0a08e0 0x7ff7bc0a0d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009f20 tx=0x7ff7b4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64caa5e6d57a97c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c0eee640 1 -- 172.21.15.204:0/798131643 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7b4007880 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c0eee640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c0eee640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c0eee640 10 monclient: dump: 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:46.899 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:46.900 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:46.901 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:46.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c0eee640 10 monclient: _finish_auth 0 2024-09-15T01:35:46.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c0eee640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:46.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 5 monclient: authenticate success, global_id 6337 2024-09-15T01:35:46.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c0eee640 1 -- 172.21.15.204:0/798131643 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b402d050 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:46.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c0eee640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:46.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:46.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 4 set_mon_vals no callback set 2024-09-15T01:35:46.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c0eee640 1 -- 172.21.15.204:0/798131643 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7b4030360 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c0eee640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c0eee640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c0eee640 10 monclient: dump: 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:46.902 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 10 monclient: shutdown 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 1 -- 172.21.15.204:0/798131643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7bc0a08e0 msgr2=0x7ff7bc0a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 1 --2- 172.21.15.204:0/798131643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7bc0a08e0 0x7ff7bc0a0d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009f20 tx=0x7ff7b4009840 comp rx=0 tx=0).stop 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 1 -- 172.21.15.204:0/798131643 shutdown_connections 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 1 --2- 172.21.15.204:0/798131643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7bc0a08e0 0x7ff7bc0a0d50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 1 --2- 172.21.15.204:0/798131643 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff7bc09fa20 0x7ff7bc09fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:46.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 1 --2- 172.21.15.204:0/798131643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7bc0a2cd0 0x7ff7bc132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 1 -- 172.21.15.204:0/798131643 >> 172.21.15.204:0/798131643 conn(0x7ff7bc06e860 msgr2=0x7ff7bc06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 1 -- 172.21.15.204:0/798131643 shutdown_connections 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.886+0000 7ff7c4154640 1 -- 172.21.15.204:0/798131643 wait complete. 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 10 monclient: build_initial_monmap 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 10 monclient: monmap: 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:46.904 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:46.905 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:46.905 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:46.905 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:46.905 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:46.905 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:46.905 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:46.905 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:46.905 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:46.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 1 Processor -- start 2024-09-15T01:35:46.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 1 -- start start 2024-09-15T01:35:46.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 10 monclient: init 2024-09-15T01:35:46.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 10 monclient: _add_conns ranks=[8,1,0,2,4,5,3,6,7] 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff7bc09fa20 0x7ff7bc1319c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 10 monclient(hunting): picked mon.noname-e con 0x7ff7bc09fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7bc0a08e0 0x7ff7bc11d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 10 monclient(hunting): picked mon.noname-f con 0x7ff7bc0a08e0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7bc0a2cd0 0x7ff7bc11dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 10 monclient(hunting): picked mon.noname-a con 0x7ff7bc0a2cd0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc120b00 con 0x7ff7bc0a2cd0 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.887+0000 7ff7c4154640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc11f5c0 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c4154640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:46.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c4154640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc121130 con 0x7ff7bc09fa20 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c4154640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c4154640 10 monclient(hunting): authenticate will time out at 962.530015s 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7bc0a08e0 0x7ff7bc11d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7bc0a08e0 0x7ff7bc11d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:56384/0 (socket says 172.21.15.204:56384) 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 1 -- 172.21.15.204:0/1041614665 learned_addr learned my addr 172.21.15.204:0/1041614665 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 10 monclient(hunting): get_auth_request con 0x7ff7bc0a08e0 auth_method 0 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:46.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c1ef0640 1 --2- 172.21.15.204:0/1041614665 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff7bc09fa20 0x7ff7bc1319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c26f1640 1 --2- 172.21.15.204:0/1041614665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7bc0a2cd0 0x7ff7bc11dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c1ef0640 10 monclient(hunting): get_auth_request con 0x7ff7bc09fa20 auth_method 0 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c1ef0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c1ef0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c1ef0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c26f1640 10 monclient(hunting): get_auth_request con 0x7ff7bc0a2cd0 auth_method 0 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c26f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c26f1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c26f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 10 monclient(hunting): handle_auth_done global_id 6346 payload 824 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 1 -- 172.21.15.204:0/1041614665 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff7bc09fa20 msgr2=0x7ff7bc1319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 1 --2- 172.21.15.204:0/1041614665 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff7bc09fa20 0x7ff7bc1319c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:46.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 1 -- 172.21.15.204:0/1041614665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7bc0a2cd0 msgr2=0x7ff7bc11dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 1 --2- 172.21.15.204:0/1041614665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7bc0a2cd0 0x7ff7bc11dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 10 monclient: _finish_hunting 0 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 1 monclient: found mon.noname-f 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 1 -- 172.21.15.204:0/1041614665 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b4009ae0 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c16ef640 1 --2- 172.21.15.204:0/1041614665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7bc0a08e0 0x7ff7bc11d570 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00d8c0 tx=0x7ff7ac008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b1ffb640 1 -- 172.21.15.204:0/1041614665 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7ac00de50 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b1ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b1ffb640 10 monclient: dump: 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:46.909 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:46.910 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b1ffb640 1 -- 172.21.15.204:0/1041614665 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac009cd0 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c4154640 5 monclient: authenticate success, global_id 6346 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c4154640 10 monclient: _renew_subs 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c4154640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c4154640 1 -- 172.21.15.204:0/1041614665 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc142b50 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b1ffb640 1 -- 172.21.15.204:0/1041614665 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7ac009e40 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b1ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b1ffb640 10 monclient: dump: 2024-09-15T01:35:46.911 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:46.912 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7b3fff640 4 set_mon_vals no callback set 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c4154640 10 monclient: _renew_subs 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c4154640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.888+0000 7ff7c4154640 1 -- 172.21.15.204:0/1041614665 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7bc11f040 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.889+0000 7ff7b1ffb640 1 -- 172.21.15.204:0/1041614665 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff7ac009850 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.889+0000 7ff7c4154640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.889+0000 7ff7c4154640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.889+0000 7ff7c4154640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.889+0000 7ff7c4154640 1 -- 172.21.15.204:0/1041614665 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7bc070b80 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.890+0000 7ff7b1ffb640 1 --2- 172.21.15.204:0/1041614665 >> v2:172.21.15.204:6812/28210 conn(0x7ff7a00397b0 0x7ff7a003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.890+0000 7ff7b1ffb640 1 -- 172.21.15.204:0/1041614665 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 7059+0+0 (secure 0 0 0) 0x7ff7ac04bf90 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.890+0000 7ff7c1ef0640 1 --2- 172.21.15.204:0/1041614665 >> v2:172.21.15.204:6812/28210 conn(0x7ff7a00397b0 0x7ff7a003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:46.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.890+0000 7ff7c1ef0640 10 monclient: get_auth_request con 0x7ff7a00397b0 auth_method 0 2024-09-15T01:35:46.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.890+0000 7ff7c1ef0640 1 --2- 172.21.15.204:0/1041614665 >> v2:172.21.15.204:6812/28210 conn(0x7ff7a00397b0 0x7ff7a003bc60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80038f0 tx=0x7ff7b8006ef0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:46.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.894+0000 7ff7b1ffb640 1 -- 172.21.15.204:0/1041614665 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff7ac01b540 con 0x7ff7bc0a08e0 2024-09-15T01:35:46.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.894+0000 7ff7b1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:46.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:46.894+0000 7ff7b1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:47.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:47.050+0000 7ff7c4154640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T01:35:47.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:47.050+0000 7ff7c4154640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T01:35:47.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:47.050+0000 7ff7c4154640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:47.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:47.050+0000 7ff7c4154640 1 -- 172.21.15.204:0/1041614665 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"} v 0) v1 -- 0x7ff7bc135160 con 0x7ff7bc0a08e0 2024-09-15T01:35:47.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:47.703+0000 7ff7b1ffb640 1 -- 172.21.15.204:0/1041614665 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}]=0 set pool 12 size to 2 v141) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff7ac018070 con 0x7ff7bc0a08e0 2024-09-15T01:35:47.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:47.703+0000 7ff7b1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T01:35:47.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:47.703+0000 7ff7b1ffb640 10 monclient: _finish_command 2 = system:0 set pool 12 size to 2 2024-09-15T01:35:47.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:47.798+0000 7ff7c4154640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T01:35:47.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:47.798+0000 7ff7c4154640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T01:35:47.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:47.798+0000 7ff7c4154640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:47.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:47.798+0000 7ff7c4154640 1 -- 172.21.15.204:0/1041614665 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"} v 0) v1 -- 0x7ff7bc0008d0 con 0x7ff7bc0a08e0 2024-09-15T01:35:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.709+0000 7ff7b1ffb640 1 -- 172.21.15.204:0/1041614665 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}]=0 set pool 12 size to 2 v142) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff7ac011b30 con 0x7ff7bc0a08e0 2024-09-15T01:35:48.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.709+0000 7ff7b1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-09-15T01:35:48.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.709+0000 7ff7b1ffb640 10 monclient: _finish_command 3 = system:0 set pool 12 size to 2 2024-09-15T01:35:48.711 INFO:tasks.workunit.client.0.smithi204.stderr:set pool 12 size to 2 2024-09-15T01:35:48.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.711+0000 7ff7c4154640 1 -- 172.21.15.204:0/1041614665 >> v2:172.21.15.204:6812/28210 conn(0x7ff7a00397b0 msgr2=0x7ff7a003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:48.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.711+0000 7ff7c4154640 1 --2- 172.21.15.204:0/1041614665 >> v2:172.21.15.204:6812/28210 conn(0x7ff7a00397b0 0x7ff7a003bc60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80038f0 tx=0x7ff7b8006ef0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.711+0000 7ff7c4154640 10 monclient: shutdown 2024-09-15T01:35:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.711+0000 7ff7c4154640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.711+0000 7ff7c4154640 1 -- 172.21.15.204:0/1041614665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7bc0a08e0 msgr2=0x7ff7bc11d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.711+0000 7ff7c4154640 1 --2- 172.21.15.204:0/1041614665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7bc0a08e0 0x7ff7bc11d570 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00d8c0 tx=0x7ff7ac008040 comp rx=0 tx=0).stop 2024-09-15T01:35:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.711+0000 7ff7c4154640 1 -- 172.21.15.204:0/1041614665 shutdown_connections 2024-09-15T01:35:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.711+0000 7ff7c4154640 1 --2- 172.21.15.204:0/1041614665 >> v2:172.21.15.204:6812/28210 conn(0x7ff7a00397b0 0x7ff7a003bc60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.712+0000 7ff7c4154640 1 --2- 172.21.15.204:0/1041614665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7bc0a2cd0 0x7ff7bc11dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.712+0000 7ff7c4154640 1 --2- 172.21.15.204:0/1041614665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7bc0a08e0 0x7ff7bc11d570 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.712+0000 7ff7c4154640 1 --2- 172.21.15.204:0/1041614665 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff7bc09fa20 0x7ff7bc1319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.712+0000 7ff7c4154640 1 -- 172.21.15.204:0/1041614665 >> 172.21.15.204:0/1041614665 conn(0x7ff7bc06e860 msgr2=0x7ff7bc0a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:48.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.712+0000 7ff7c4154640 1 -- 172.21.15.204:0/1041614665 shutdown_connections 2024-09-15T01:35:48.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.712+0000 7ff7c4154640 1 -- 172.21.15.204:0/1041614665 wait complete. 2024-09-15T01:35:48.724 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set create-compat 2024-09-15T01:35:48.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.801+0000 7fdd26cc9640 10 monclient: get_monmap_and_config 2024-09-15T01:35:48.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.802+0000 7fdd26cc9640 10 monclient: build_initial_monmap 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.802+0000 7fdd26cc9640 10 monclient: monmap: 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:48.802511+0000 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:48.802511+0000 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:48.804 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:48.805 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:48.805 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:48.805 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:48.805 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:48.805 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:48.805 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.802+0000 7fdd26cc9640 1 Processor -- start 2024-09-15T01:35:48.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.802+0000 7fdd26cc9640 1 -- start start 2024-09-15T01:35:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.802+0000 7fdd26cc9640 10 monclient: init 2024-09-15T01:35:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 10 monclient: _add_conns ranks=[8,9,2,1,6,0,4,7,3,5,10] 2024-09-15T01:35:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a2e30 0x7fdd20132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 10 monclient(hunting): picked mon.noname-e con 0x7fdd200a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdd2009fb80 0x7fdd2009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 10 monclient(hunting): picked mon.noname-a con 0x7fdd2009fb80 addr v2:172.21.15.178:3300/0 2024-09-15T01:35:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd200a0a40 0x7fdd200a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 10 monclient(hunting): picked mon.noname-b con 0x7fdd200a0a40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2009e6c0 con 0x7fdd2009fb80 2024-09-15T01:35:48.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd200a04b0 con 0x7fdd200a0a40 2024-09-15T01:35:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdd200a3200 con 0x7fdd200a2e30 2024-09-15T01:35:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd26cc9640 10 monclient(hunting): authenticate will time out at 964.445555s 2024-09-15T01:35:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd25266640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd200a0a40 msgr2=0x7fdd200a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd25266640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd200a0a40 0x7fdd200a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd24a65640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a2e30 0x7fdd20132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd1ffff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdd2009fb80 0x7fdd2009ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd24a65640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a2e30 0x7fdd20132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:60606/0 (socket says 172.21.15.204:60606) 2024-09-15T01:35:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd24a65640 1 -- 172.21.15.204:0/3894739913 learned_addr learned my addr 172.21.15.204:0/3894739913 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:48.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd24a65640 10 monclient(hunting): get_auth_request con 0x7fdd200a2e30 auth_method 0 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd24a65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd24a65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd24a65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd1ffff640 10 monclient(hunting): get_auth_request con 0x7fdd2009fb80 auth_method 0 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd1ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd1ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd1ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd24a65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd24a65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd24a65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd1ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd1ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:48.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.803+0000 7fdd1ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd24a65640 10 monclient(hunting): handle_auth_done global_id 5741 payload 274 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd24a65640 1 -- 172.21.15.204:0/3894739913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd200a0a40 msgr2=0x7fdd200a0ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd24a65640 1 --2- 172.21.15.204:0/3894739913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd200a0a40 0x7fdd200a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd24a65640 1 -- 172.21.15.204:0/3894739913 >> v2:172.21.15.178:3300/0 conn(0x7fdd2009fb80 msgr2=0x7fdd2009ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd24a65640 1 --2- 172.21.15.204:0/3894739913 >> v2:172.21.15.178:3300/0 conn(0x7fdd2009fb80 0x7fdd2009ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd24a65640 10 monclient: _finish_hunting 0 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd24a65640 1 monclient: found mon.noname-e 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd24a65640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd24a65640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd24a65640 1 -- 172.21.15.204:0/3894739913 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2004b720 con 0x7fdd200a2e30 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd1ffff640 1 --2- 172.21.15.204:0/3894739913 >> v2:172.21.15.178:3300/0 conn(0x7fdd2009fb80 0x7fdd2009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd24a65640 1 --2- 172.21.15.204:0/3894739913 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a2e30 0x7fdd20132a60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009910 tx=0x7fdd100091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1e7d61ad793a6594 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:48.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.804+0000 7fdd1f7fe640 1 -- 172.21.15.204:0/3894739913 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd1002acb0 con 0x7fdd200a2e30 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd1f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd1f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd1f7fe640 10 monclient: dump: 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:48.818 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd1f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd1f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 5 monclient: authenticate success, global_id 5741 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd1f7fe640 1 -- 172.21.15.204:0/3894739913 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1002ae30 con 0x7fdd200a2e30 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd1f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:48.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 4 set_mon_vals no callback set 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd1f7fe640 1 -- 172.21.15.204:0/3894739913 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd10031ca0 con 0x7fdd200a2e30 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd1f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd1f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd1f7fe640 10 monclient: dump: 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:48.820 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 10 monclient: shutdown 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 1 -- 172.21.15.204:0/3894739913 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a2e30 msgr2=0x7fdd20132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 1 --2- 172.21.15.204:0/3894739913 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a2e30 0x7fdd20132a60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009910 tx=0x7fdd100091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 1 -- 172.21.15.204:0/3894739913 shutdown_connections 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 1 --2- 172.21.15.204:0/3894739913 >> v2:172.21.15.178:3300/0 conn(0x7fdd2009fb80 0x7fdd2009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 1 --2- 172.21.15.204:0/3894739913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd200a0a40 0x7fdd200a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 1 --2- 172.21.15.204:0/3894739913 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a2e30 0x7fdd20132a60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 1 -- 172.21.15.204:0/3894739913 >> 172.21.15.204:0/3894739913 conn(0x7fdd2006e860 msgr2=0x7fdd2006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 1 -- 172.21.15.204:0/3894739913 shutdown_connections 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.805+0000 7fdd26cc9640 1 -- 172.21.15.204:0/3894739913 wait complete. 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.806+0000 7fdd26cc9640 10 monclient: build_initial_monmap 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.806+0000 7fdd26cc9640 10 monclient: monmap: 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:48.822 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.806+0000 7fdd26cc9640 1 Processor -- start 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.806+0000 7fdd26cc9640 1 -- start start 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.806+0000 7fdd26cc9640 10 monclient: init 2024-09-15T01:35:48.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.806+0000 7fdd26cc9640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.806+0000 7fdd26cc9640 10 monclient: _add_conns ranks=[0,8,1,4,6,2,3,7,5] 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.806+0000 7fdd26cc9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2009fb80 0x7fdd20131830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.806+0000 7fdd26cc9640 10 monclient(hunting): picked mon.noname-a con 0x7fdd2009fb80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.806+0000 7fdd26cc9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a0a40 0x7fdd2011e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 10 monclient(hunting): picked mon.noname-e con 0x7fdd200a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd200a2e30 0x7fdd2011e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 10 monclient(hunting): picked mon.noname-f con 0x7fdd200a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2009ca90 con 0x7fdd2009fb80 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2009d650 con 0x7fdd200a2e30 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdd20121c60 con 0x7fdd200a0a40 2024-09-15T01:35:48.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 10 monclient(hunting): authenticate will time out at 964.449034s 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd200a2e30 0x7fdd2011e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd24a65640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2009fb80 0x7fdd20131830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd200a2e30 0x7fdd2011e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:50718/0 (socket says 172.21.15.204:50718) 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd24a65640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2009fb80 0x7fdd20131830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:58782/0 (socket says 172.21.15.204:58782) 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 1 -- 172.21.15.204:0/2683809473 learned_addr learned my addr 172.21.15.204:0/2683809473 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 10 monclient(hunting): get_auth_request con 0x7fdd200a2e30 auth_method 0 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd24a65640 10 monclient(hunting): get_auth_request con 0x7fdd2009fb80 auth_method 0 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd24a65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:48.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd24a65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd24a65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1ffff640 1 --2- 172.21.15.204:0/2683809473 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a0a40 0x7fdd2011e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1ffff640 10 monclient(hunting): get_auth_request con 0x7fdd200a0a40 auth_method 0 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd24a65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:48.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd24a65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd24a65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 10 monclient(hunting): handle_auth_done global_id 6355 payload 824 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 1 -- 172.21.15.204:0/2683809473 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a0a40 msgr2=0x7fdd2011e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 1 --2- 172.21.15.204:0/2683809473 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a0a40 0x7fdd2011e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 1 -- 172.21.15.204:0/2683809473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2009fb80 msgr2=0x7fdd20131830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 1 --2- 172.21.15.204:0/2683809473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2009fb80 0x7fdd20131830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 10 monclient: _finish_hunting 0 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 1 monclient: found mon.noname-f 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 1 -- 172.21.15.204:0/2683809473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14009390 con 0x7fdd200a2e30 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1ffff640 1 --2- 172.21.15.204:0/2683809473 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a0a40 0x7fdd2011e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:48.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd25266640 1 --2- 172.21.15.204:0/2683809473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd200a2e30 0x7fdd2011e980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009fc0 tx=0x7fdd18009730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1cff9640 1 -- 172.21.15.204:0/2683809473 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd1800cdc0 con 0x7fdd200a2e30 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1cff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1cff9640 10 monclient: dump: 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:48.828 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1cff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1cff9640 1 -- 172.21.15.204:0/2683809473 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd180065d0 con 0x7fdd200a2e30 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 5 monclient: authenticate success, global_id 6355 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 10 monclient: _renew_subs 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:48.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd26cc9640 1 -- 172.21.15.204:0/2683809473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd10009450 con 0x7fdd200a2e30 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1cff9640 1 -- 172.21.15.204:0/2683809473 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd180153e0 con 0x7fdd200a2e30 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1cff9640 10 monclient: dump: 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:48.830 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.808+0000 7fdd26cc9640 10 monclient: _renew_subs 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.807+0000 7fdd1effd640 4 set_mon_vals no callback set 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.808+0000 7fdd26cc9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.808+0000 7fdd26cc9640 1 -- 172.21.15.204:0/2683809473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2011fe50 con 0x7fdd200a2e30 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.808+0000 7fdd1cff9640 1 -- 172.21.15.204:0/2683809473 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdd18012020 con 0x7fdd200a2e30 2024-09-15T01:35:48.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.808+0000 7fdd26cc9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.808+0000 7fdd26cc9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.808+0000 7fdd26cc9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.808+0000 7fdd26cc9640 1 -- 172.21.15.204:0/2683809473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd20121e10 con 0x7fdd200a2e30 2024-09-15T01:35:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.809+0000 7fdd1cff9640 1 --2- 172.21.15.204:0/2683809473 >> v2:172.21.15.204:6812/28210 conn(0x7fdd0c0397b0 0x7fdd0c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.809+0000 7fdd1cff9640 1 -- 172.21.15.204:0/2683809473 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 7059+0+0 (secure 0 0 0) 0x7fdd1804c050 con 0x7fdd200a2e30 2024-09-15T01:35:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.809+0000 7fdd24a65640 1 --2- 172.21.15.204:0/2683809473 >> v2:172.21.15.204:6812/28210 conn(0x7fdd0c0397b0 0x7fdd0c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.809+0000 7fdd24a65640 10 monclient: get_auth_request con 0x7fdd0c0397b0 auth_method 0 2024-09-15T01:35:48.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.809+0000 7fdd24a65640 1 --2- 172.21.15.204:0/2683809473 >> v2:172.21.15.204:6812/28210 conn(0x7fdd0c0397b0 0x7fdd0c03bc60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdd10034000 tx=0x7fdd10032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.813+0000 7fdd1cff9640 1 -- 172.21.15.204:0/2683809473 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fdd20121e10 con 0x7fdd200a2e30 2024-09-15T01:35:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.813+0000 7fdd1cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:48.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.813+0000 7fdd1cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:48.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.973+0000 7fdd26cc9640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:35:48.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.973+0000 7fdd26cc9640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:35:48.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.973+0000 7fdd26cc9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:48.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:48.973+0000 7fdd26cc9640 1 -- 172.21.15.204:0/2683809473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7fdd20121e10 con 0x7fdd200a2e30 2024-09-15T01:35:49.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.709+0000 7fdd1cff9640 1 -- 172.21.15.204:0/2683809473 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v143) v1 ==== 82+0+0 (secure 0 0 0) 0x7fdd20121e10 con 0x7fdd200a2e30 2024-09-15T01:35:49.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.709+0000 7fdd1cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:35:49.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.709+0000 7fdd1cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:49.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.814+0000 7fdd26cc9640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:35:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.814+0000 7fdd26cc9640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:35:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.814+0000 7fdd26cc9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.814+0000 7fdd26cc9640 1 -- 172.21.15.204:0/2683809473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7fdd20121e10 con 0x7fdd200a2e30 2024-09-15T01:35:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.814+0000 7fdd1cff9640 1 -- 172.21.15.204:0/2683809473 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v143) v1 ==== 115+0+0 (secure 0 0 0) 0x7fdd20121e10 con 0x7fdd200a2e30 2024-09-15T01:35:49.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.814+0000 7fdd1cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:35:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.814+0000 7fdd1cff9640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-09-15T01:35:49.817 INFO:tasks.workunit.client.0.smithi204.stderr:compat weight-set already created 2024-09-15T01:35:49.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.816+0000 7fdd26cc9640 1 -- 172.21.15.204:0/2683809473 >> v2:172.21.15.204:6812/28210 conn(0x7fdd0c0397b0 msgr2=0x7fdd0c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:49.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.816+0000 7fdd26cc9640 1 --2- 172.21.15.204:0/2683809473 >> v2:172.21.15.204:6812/28210 conn(0x7fdd0c0397b0 0x7fdd0c03bc60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdd10034000 tx=0x7fdd10032000 comp rx=0 tx=0).stop 2024-09-15T01:35:49.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.816+0000 7fdd26cc9640 10 monclient: shutdown 2024-09-15T01:35:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.816+0000 7fdd26cc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.816+0000 7fdd26cc9640 1 -- 172.21.15.204:0/2683809473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd200a2e30 msgr2=0x7fdd2011e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.816+0000 7fdd26cc9640 1 --2- 172.21.15.204:0/2683809473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd200a2e30 0x7fdd2011e980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009fc0 tx=0x7fdd18009730 comp rx=0 tx=0).stop 2024-09-15T01:35:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.816+0000 7fdd26cc9640 1 -- 172.21.15.204:0/2683809473 shutdown_connections 2024-09-15T01:35:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.816+0000 7fdd26cc9640 1 --2- 172.21.15.204:0/2683809473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd200a2e30 0x7fdd2011e980 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.816+0000 7fdd26cc9640 1 --2- 172.21.15.204:0/2683809473 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd200a0a40 0x7fdd2011e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.816+0000 7fdd26cc9640 1 --2- 172.21.15.204:0/2683809473 >> v2:172.21.15.204:6812/28210 conn(0x7fdd0c0397b0 0x7fdd0c03bc60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.816+0000 7fdd26cc9640 1 --2- 172.21.15.204:0/2683809473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd2009fb80 0x7fdd20131830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.816+0000 7fdd26cc9640 1 -- 172.21.15.204:0/2683809473 >> 172.21.15.204:0/2683809473 conn(0x7fdd2006e860 msgr2=0x7fdd2009ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.817+0000 7fdd26cc9640 1 -- 172.21.15.204:0/2683809473 shutdown_connections 2024-09-15T01:35:49.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.817+0000 7fdd26cc9640 1 -- 172.21.15.204:0/2683809473 wait complete. 2024-09-15T01:35:49.827 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set create cool flat 2024-09-15T01:35:49.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.902+0000 7fedc8714640 10 monclient: get_monmap_and_config 2024-09-15T01:35:49.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.902+0000 7fedc8714640 10 monclient: build_initial_monmap 2024-09-15T01:35:49.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.902+0000 7fedc8714640 10 monclient: monmap: 2024-09-15T01:35:49.904 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:49.904 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:49.904 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:49.903040+0000 2024-09-15T01:35:49.904 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:49.903040+0000 2024-09-15T01:35:49.904 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:49.904 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:49.904 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:49.905 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:49.905 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:49.905 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:49.905 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:49.905 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:49.905 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:49.905 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:49.905 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:49.905 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:49.905 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:49.905 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:49.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 1 Processor -- start 2024-09-15T01:35:49.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 1 -- start start 2024-09-15T01:35:49.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 10 monclient: init 2024-09-15T01:35:49.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:49.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 10 monclient: _add_conns ranks=[5,3,9,7,10,6,8,2,1,4,0] 2024-09-15T01:35:49.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fedc00a2cd0 0x7fedc0132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:49.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 10 monclient(hunting): picked mon.noname-h con 0x7fedc00a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:49.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedc009fa20 0x7fedc009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:49.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 10 monclient(hunting): picked mon.noname-g con 0x7fedc009fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:49.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fedc00a08e0 0x7fedc00a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:49.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 10 monclient(hunting): picked mon.noname-a con 0x7fedc00a08e0 addr v2:172.21.15.178:3300/0 2024-09-15T01:35:49.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:49.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc00a30a0 con 0x7fedc00a08e0 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00a0330 con 0x7fedc009fa20 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00a1db0 con 0x7fedc00a2cd0 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc8714640 10 monclient(hunting): authenticate will time out at 965.545813s 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc5caf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedc009fa20 0x7fedc009fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc64b0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fedc00a2cd0 0x7fedc0132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc5caf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedc009fa20 0x7fedc009fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:51522/0 (socket says 172.21.15.204:51522) 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc5caf640 1 -- 172.21.15.204:0/2017011612 learned_addr learned my addr 172.21.15.204:0/2017011612 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc5caf640 10 monclient(hunting): get_auth_request con 0x7fedc009fa20 auth_method 0 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc5caf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:49.914 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc5caf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc5caf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc64b0640 1 --2- 172.21.15.204:0/2017011612 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fedc00a2cd0 0x7fedc0132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:44352/0 (socket says 172.21.15.204:44352) 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc64b0640 10 monclient(hunting): get_auth_request con 0x7fedc00a2cd0 auth_method 0 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc64b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc64b0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.903+0000 7fedc64b0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc64b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc64b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc64b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:49.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 10 monclient(hunting): handle_auth_done global_id 14853 payload 274 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 1 -- 172.21.15.204:0/2017011612 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fedc00a2cd0 msgr2=0x7fedc0132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 1 --2- 172.21.15.204:0/2017011612 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fedc00a2cd0 0x7fedc0132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 1 -- 172.21.15.204:0/2017011612 >> v2:172.21.15.178:3300/0 conn(0x7fedc00a08e0 msgr2=0x7fedc00a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 1 --2- 172.21.15.204:0/2017011612 >> v2:172.21.15.178:3300/0 conn(0x7fedc00a08e0 0x7fedc00a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 10 monclient: _finish_hunting 0 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 1 monclient: found mon.noname-g 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 1 -- 172.21.15.204:0/2017011612 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00648a0 con 0x7fedc009fa20 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc64b0640 1 --2- 172.21.15.204:0/2017011612 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fedc00a2cd0 0x7fedc0132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.904+0000 7fedc5caf640 1 --2- 172.21.15.204:0/2017011612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedc009fa20 0x7fedc009fdf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009910 tx=0x7fedb00091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=b04c55c205f5d21b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc54ae640 1 -- 172.21.15.204:0/2017011612 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedb002ac80 con 0x7fedc009fa20 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc54ae640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:49.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc54ae640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:49.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc54ae640 10 monclient: dump: 2024-09-15T01:35:49.917 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:49.917 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:49.917 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:49.917 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:49.917 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:49.917 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:49.917 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:49.917 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:49.917 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:49.917 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:49.917 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc54ae640 10 monclient: _finish_auth 0 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc54ae640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc54ae640 1 -- 172.21.15.204:0/2017011612 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb002ae00 con 0x7fedc009fa20 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 5 monclient: authenticate success, global_id 14853 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc54ae640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:49.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc54ae640 1 -- 172.21.15.204:0/2017011612 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedb0031ca0 con 0x7fedc009fa20 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 4 set_mon_vals no callback set 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc54ae640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc54ae640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc54ae640 10 monclient: dump: 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:49.919 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 10 monclient: shutdown 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 1 -- 172.21.15.204:0/2017011612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedc009fa20 msgr2=0x7fedc009fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 1 --2- 172.21.15.204:0/2017011612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedc009fa20 0x7fedc009fdf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009910 tx=0x7fedb00091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 1 -- 172.21.15.204:0/2017011612 shutdown_connections 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 1 --2- 172.21.15.204:0/2017011612 >> v2:172.21.15.178:3300/0 conn(0x7fedc00a08e0 0x7fedc00a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 1 --2- 172.21.15.204:0/2017011612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fedc009fa20 0x7fedc009fdf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:49.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 1 --2- 172.21.15.204:0/2017011612 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fedc00a2cd0 0x7fedc0132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 1 -- 172.21.15.204:0/2017011612 >> 172.21.15.204:0/2017011612 conn(0x7fedc006e860 msgr2=0x7fedc006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 1 -- 172.21.15.204:0/2017011612 shutdown_connections 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.905+0000 7fedc8714640 1 -- 172.21.15.204:0/2017011612 wait complete. 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient: build_initial_monmap 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient: monmap: 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:49.921 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 1 Processor -- start 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 1 -- start start 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient: init 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:49.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient: _add_conns ranks=[6,4,8,2,7,0,1,5,3] 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedc009fa20 0x7fedc0141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient(hunting): picked mon.noname-d con 0x7fedc009fa20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedc00a08e0 0x7fedc0142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient(hunting): picked mon.noname-c con 0x7fedc00a08e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedc00a2cd0 0x7fedc011d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient(hunting): picked mon.noname-e con 0x7fedc00a2cd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedc009d3d0 con 0x7fedc00a08e0 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fedc009cb60 con 0x7fedc009fa20 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fedc011f140 con 0x7fedc00a2cd0 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:49.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.906+0000 7fedc8714640 10 monclient(hunting): authenticate will time out at 965.548974s 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc64b0640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedc009fa20 msgr2=0x7fedc0141ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc64b0640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedc009fa20 0x7fedc0141ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedc00a08e0 0x7fedc0142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc6cb1640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedc00a2cd0 0x7fedc011d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedc00a08e0 0x7fedc0142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:58600/0 (socket says 172.21.15.204:58600) 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 1 -- 172.21.15.204:0/3628318326 learned_addr learned my addr 172.21.15.204:0/3628318326 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 10 monclient(hunting): get_auth_request con 0x7fedc00a08e0 auth_method 0 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc6cb1640 10 monclient(hunting): get_auth_request con 0x7fedc00a2cd0 auth_method 0 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc6cb1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:49.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc6cb1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc6cb1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc6cb1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc6cb1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc6cb1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 10 monclient(hunting): handle_auth_done global_id 5674 payload 824 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 1 -- 172.21.15.204:0/3628318326 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedc00a2cd0 msgr2=0x7fedc011d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 1 --2- 172.21.15.204:0/3628318326 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedc00a2cd0 0x7fedc011d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 1 -- 172.21.15.204:0/3628318326 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedc009fa20 msgr2=0x7fedc0141ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 1 --2- 172.21.15.204:0/3628318326 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedc009fa20 0x7fedc0141ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 10 monclient: _finish_hunting 0 2024-09-15T01:35:49.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 1 monclient: found mon.noname-c 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 1 -- 172.21.15.204:0/3628318326 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc009a20 con 0x7fedc00a08e0 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc5caf640 1 --2- 172.21.15.204:0/3628318326 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedc00a08e0 0x7fedc0142430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fedb0000c00 tx=0x7fedb0032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedb67fc640 1 -- 172.21.15.204:0/3628318326 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedb0031820 con 0x7fedc00a08e0 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedb67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedb67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedb67fc640 10 monclient: dump: 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:49.926 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedb67fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedb67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc8714640 5 monclient: authenticate success, global_id 5674 2024-09-15T01:35:49.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc8714640 10 monclient: _renew_subs 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedb67fc640 1 -- 172.21.15.204:0/3628318326 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb0031980 con 0x7fedc00a08e0 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc8714640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc8714640 1 -- 172.21.15.204:0/3628318326 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb0009450 con 0x7fedc00a08e0 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedb67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedb67fc640 1 -- 172.21.15.204:0/3628318326 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedb003a390 con 0x7fedc00a08e0 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedb67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.908+0000 7fedb67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.908+0000 7fedb67fc640 10 monclient: dump: 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:49.928 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.907+0000 7fedc4cad640 4 set_mon_vals no callback set 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.908+0000 7fedc8714640 10 monclient: _renew_subs 2024-09-15T01:35:49.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.908+0000 7fedc8714640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:49.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.908+0000 7fedc8714640 1 -- 172.21.15.204:0/3628318326 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc0120cb0 con 0x7fedc00a08e0 2024-09-15T01:35:49.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.908+0000 7fedc8714640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:49.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.908+0000 7fedc8714640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:49.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.909+0000 7fedc8714640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:49.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.909+0000 7fedc8714640 1 -- 172.21.15.204:0/3628318326 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc0070b80 con 0x7fedc00a08e0 2024-09-15T01:35:49.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.909+0000 7fedb67fc640 1 -- 172.21.15.204:0/3628318326 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fedb0039040 con 0x7fedc00a08e0 2024-09-15T01:35:49.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.909+0000 7fedb67fc640 1 --2- 172.21.15.204:0/3628318326 >> v2:172.21.15.204:6812/28210 conn(0x7feda40397d0 0x7feda403bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:49.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.909+0000 7fedb67fc640 1 -- 172.21.15.204:0/3628318326 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 7391+0+0 (secure 0 0 0) 0x7fedb006b070 con 0x7fedc00a08e0 2024-09-15T01:35:49.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.909+0000 7fedc64b0640 1 --2- 172.21.15.204:0/3628318326 >> v2:172.21.15.204:6812/28210 conn(0x7feda40397d0 0x7feda403bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:49.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.909+0000 7fedc64b0640 10 monclient: get_auth_request con 0x7feda40397d0 auth_method 0 2024-09-15T01:35:49.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.909+0000 7fedc64b0640 1 --2- 172.21.15.204:0/3628318326 >> v2:172.21.15.204:6812/28210 conn(0x7feda40397d0 0x7feda403bc80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009c50 tx=0x7fedbc00b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:49.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.913+0000 7fedb67fc640 1 -- 172.21.15.204:0/3628318326 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fedb0031e10 con 0x7fedc00a08e0 2024-09-15T01:35:49.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.913+0000 7fedb67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:49.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:49.913+0000 7fedb67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:50.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.072+0000 7fedc8714640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-09-15T01:35:50.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.072+0000 7fedc8714640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-09-15T01:35:50.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.072+0000 7fedc8714640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:50.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.072+0000 7fedc8714640 1 -- 172.21.15.204:0/3628318326 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"} v 0) v1 -- 0x7fedc011ec30 con 0x7fedc00a08e0 2024-09-15T01:35:50.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.726+0000 7fedb67fc640 1 -- 172.21.15.204:0/3628318326 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}]=0 v144) v1 ==== 107+0+0 (secure 0 0 0) 0x7fedb00370b0 con 0x7fedc00a08e0 2024-09-15T01:35:50.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.726+0000 7fedb67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-09-15T01:35:50.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.726+0000 7fedb67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.826+0000 7fedc8714640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-09-15T01:35:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.826+0000 7fedc8714640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-09-15T01:35:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.826+0000 7fedc8714640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:50.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.826+0000 7fedc8714640 1 -- 172.21.15.204:0/3628318326 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"} v 0) v1 -- 0x7fedc011ec30 con 0x7fedc00a08e0 2024-09-15T01:35:50.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.828+0000 7fedb67fc640 1 -- 172.21.15.204:0/3628318326 <== mon.4 v2:172.21.15.178:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}]=0 weight-set for pool 'cool' already created v144) v1 ==== 149+0+0 (secure 0 0 0) 0x7fedb00392f0 con 0x7fedc00a08e0 2024-09-15T01:35:50.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.828+0000 7fedb67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-09-15T01:35:50.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.828+0000 7fedb67fc640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'cool' already created 2024-09-15T01:35:50.829 INFO:tasks.workunit.client.0.smithi204.stderr:weight-set for pool 'cool' already created 2024-09-15T01:35:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 1 -- 172.21.15.204:0/3628318326 >> v2:172.21.15.204:6812/28210 conn(0x7feda40397d0 msgr2=0x7feda403bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 1 --2- 172.21.15.204:0/3628318326 >> v2:172.21.15.204:6812/28210 conn(0x7feda40397d0 0x7feda403bc80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009c50 tx=0x7fedbc00b040 comp rx=0 tx=0).stop 2024-09-15T01:35:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 10 monclient: shutdown 2024-09-15T01:35:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 1 -- 172.21.15.204:0/3628318326 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedc00a08e0 msgr2=0x7fedc0142430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 1 --2- 172.21.15.204:0/3628318326 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedc00a08e0 0x7fedc0142430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fedb0000c00 tx=0x7fedb0032000 comp rx=0 tx=0).stop 2024-09-15T01:35:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 1 -- 172.21.15.204:0/3628318326 shutdown_connections 2024-09-15T01:35:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 1 --2- 172.21.15.204:0/3628318326 >> v2:172.21.15.204:6812/28210 conn(0x7feda40397d0 0x7feda403bc80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 1 --2- 172.21.15.204:0/3628318326 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedc00a2cd0 0x7fedc011d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 1 --2- 172.21.15.204:0/3628318326 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedc00a08e0 0x7fedc0142430 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 1 --2- 172.21.15.204:0/3628318326 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedc009fa20 0x7fedc0141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:50.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 1 -- 172.21.15.204:0/3628318326 >> 172.21.15.204:0/3628318326 conn(0x7fedc006e860 msgr2=0x7fedc00a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:50.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 1 -- 172.21.15.204:0/3628318326 shutdown_connections 2024-09-15T01:35:50.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.830+0000 7fedc8714640 1 -- 172.21.15.204:0/3628318326 wait complete. 2024-09-15T01:35:50.840 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set create cold positional 2024-09-15T01:35:50.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.914+0000 7f677eb09640 10 monclient: get_monmap_and_config 2024-09-15T01:35:50.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.914+0000 7f677eb09640 10 monclient: build_initial_monmap 2024-09-15T01:35:50.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.914+0000 7f677eb09640 10 monclient: monmap: 2024-09-15T01:35:50.916 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:50.916 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:50.916 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:50.915027+0000 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:50.915027+0000 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:50.917 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:50.918 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:50.918 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:50.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 1 Processor -- start 2024-09-15T01:35:50.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 1 -- start start 2024-09-15T01:35:50.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 10 monclient: init 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 10 monclient: _add_conns ranks=[10,5,4,9,0,1,8,2,7,3,6] 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f67780a2cd0 0x7f6778132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 10 monclient(hunting): picked mon.noname-f con 0x7f67780a2cd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f677809fa20 0x7f677809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 10 monclient(hunting): picked mon.noname-h con 0x7f677809fa20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f67780a08e0 0x7f67780a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 10 monclient(hunting): picked mon.noname-c con 0x7f67780a08e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67780a30a0 con 0x7f67780a2cd0 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67780a0330 con 0x7f67780a08e0 2024-09-15T01:35:50.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:50.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67780a1db0 con 0x7f677809fa20 2024-09-15T01:35:50.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:50.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677eb09640 10 monclient(hunting): authenticate will time out at 966.557811s 2024-09-15T01:35:50.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f6777fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f677809fa20 0x7f677809fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:50.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677c8a5640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f67780a2cd0 0x7f6778132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:50.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f6777fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f677809fa20 0x7f677809fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:44358/0 (socket says 172.21.15.204:44358) 2024-09-15T01:35:50.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677c8a5640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f67780a2cd0 0x7f6778132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:50720/0 (socket says 172.21.15.204:50720) 2024-09-15T01:35:50.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f6777fff640 1 -- 172.21.15.204:0/1531316330 learned_addr learned my addr 172.21.15.204:0/1531316330 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:50.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f6777fff640 10 monclient(hunting): get_auth_request con 0x7f677809fa20 auth_method 0 2024-09-15T01:35:50.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f6777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:50.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f6777fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:50.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f6777fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677c8a5640 10 monclient(hunting): get_auth_request con 0x7f67780a2cd0 auth_method 0 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677c8a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677c8a5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.915+0000 7f677c8a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f6777fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f6777fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f6777fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677d0a6640 1 --2- 172.21.15.204:0/1531316330 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f67780a08e0 0x7f67780a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677d0a6640 10 monclient(hunting): get_auth_request con 0x7f67780a08e0 auth_method 0 2024-09-15T01:35:50.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677d0a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677d0a6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677d0a6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 10 monclient(hunting): handle_auth_done global_id 6364 payload 274 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 1 -- 172.21.15.204:0/1531316330 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f677809fa20 msgr2=0x7f677809fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 1 --2- 172.21.15.204:0/1531316330 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f677809fa20 0x7f677809fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 1 -- 172.21.15.204:0/1531316330 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f67780a08e0 msgr2=0x7f67780a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 1 --2- 172.21.15.204:0/1531316330 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f67780a08e0 0x7f67780a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 10 monclient: _finish_hunting 0 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 1 monclient: found mon.noname-f 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 1 -- 172.21.15.204:0/1531316330 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67780648a0 con 0x7f67780a2cd0 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f6777fff640 1 --2- 172.21.15.204:0/1531316330 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f677809fa20 0x7f677809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:50.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.916+0000 7f677c8a5640 1 --2- 172.21.15.204:0/1531316330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67780a2cd0 0x7f6778132c00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f676c009850 tx=0x7f676c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c40c0187d0442140 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f67777fe640 1 -- 172.21.15.204:0/1531316330 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f676c02abc0 con 0x7f67780a2cd0 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f67777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f67777fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f67777fe640 10 monclient: dump: 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:50.931 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f67777fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f67777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f67777fe640 1 -- 172.21.15.204:0/1531316330 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c02ad40 con 0x7f67780a2cd0 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 5 monclient: authenticate success, global_id 6364 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:50.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f67777fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f67777fe640 1 -- 172.21.15.204:0/1531316330 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f676c031ca0 con 0x7f67780a2cd0 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f67777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 4 set_mon_vals no callback set 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f67777fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f67777fe640 10 monclient: dump: 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:50.933 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 10 monclient: shutdown 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 1 -- 172.21.15.204:0/1531316330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67780a2cd0 msgr2=0x7f6778132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:50.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 1 --2- 172.21.15.204:0/1531316330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67780a2cd0 0x7f6778132c00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f676c009850 tx=0x7f676c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 1 -- 172.21.15.204:0/1531316330 shutdown_connections 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 1 --2- 172.21.15.204:0/1531316330 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f67780a08e0 0x7f67780a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 1 --2- 172.21.15.204:0/1531316330 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f677809fa20 0x7f677809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 1 --2- 172.21.15.204:0/1531316330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67780a2cd0 0x7f6778132c00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 1 -- 172.21.15.204:0/1531316330 >> 172.21.15.204:0/1531316330 conn(0x7f677806e860 msgr2=0x7f677806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 1 -- 172.21.15.204:0/1531316330 shutdown_connections 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.917+0000 7f677eb09640 1 -- 172.21.15.204:0/1531316330 wait complete. 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.918+0000 7f677eb09640 10 monclient: build_initial_monmap 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.918+0000 7f677eb09640 10 monclient: monmap: 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:50.935 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.918+0000 7f677eb09640 1 Processor -- start 2024-09-15T01:35:50.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.918+0000 7f677eb09640 1 -- start start 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.918+0000 7f677eb09640 10 monclient: init 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 10 monclient: _add_conns ranks=[2,8,6,4,5,7,1,0,3] 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f677809fa20 0x7f6778141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 10 monclient(hunting): picked mon.noname-b con 0x7f677809fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f67780a08e0 0x7f67781421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 10 monclient(hunting): picked mon.noname-e con 0x7f67780a08e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f67780a2cd0 0x7f6778142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 10 monclient(hunting): picked mon.noname-d con 0x7f67780a2cd0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677811ecd0 con 0x7f677809fa20 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:50.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f677809d3d0 con 0x7f67780a2cd0 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f677809cb60 con 0x7f67780a08e0 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677eb09640 10 monclient(hunting): authenticate will time out at 966.561139s 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677d0a6640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f67780a2cd0 msgr2=0x7f6778142720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677c8a5640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f677809fa20 msgr2=0x7f6778141ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677c8a5640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f677809fa20 0x7f6778141ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f677d0a6640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f67780a2cd0 0x7f6778142720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f67780a08e0 0x7f67781421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f67780a08e0 0x7f67781421e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:60640/0 (socket says 172.21.15.204:60640) 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 1 -- 172.21.15.204:0/1289579569 learned_addr learned my addr 172.21.15.204:0/1289579569 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 10 monclient(hunting): get_auth_request con 0x7f67780a08e0 auth_method 0 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:50.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 10 monclient(hunting): handle_auth_done global_id 5768 payload 824 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 1 -- 172.21.15.204:0/1289579569 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f67780a2cd0 msgr2=0x7f6778142720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 1 --2- 172.21.15.204:0/1289579569 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f67780a2cd0 0x7f6778142720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 1 -- 172.21.15.204:0/1289579569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f677809fa20 msgr2=0x7f6778141ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 1 --2- 172.21.15.204:0/1289579569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f677809fa20 0x7f6778141ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 1 monclient: found mon.noname-e 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 1 -- 172.21.15.204:0/1289579569 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6764009450 con 0x7f67780a08e0 2024-09-15T01:35:50.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.919+0000 7f6777fff640 1 --2- 172.21.15.204:0/1289579569 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f67780a08e0 0x7f67781421e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f676400ece0 tx=0x7f67640091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6774ff9640 1 -- 172.21.15.204:0/1289579569 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6764009730 con 0x7f67780a08e0 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6774ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6774ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6774ff9640 10 monclient: dump: 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:50.940 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6774ff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6774ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6774ff9640 1 -- 172.21.15.204:0/1289579569 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6764013470 con 0x7f67780a08e0 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f677eb09640 5 monclient: authenticate success, global_id 5768 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6774ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:50.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6774ff9640 1 -- 172.21.15.204:0/1289579569 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6764013a60 con 0x7f67780a08e0 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f677eb09640 10 monclient: _renew_subs 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f677eb09640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f677eb09640 1 -- 172.21.15.204:0/1289579569 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c009390 con 0x7f67780a08e0 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6774ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6776ffd640 4 set_mon_vals no callback set 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6774ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f6774ff9640 10 monclient: dump: 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:50.942 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f677eb09640 10 monclient: _renew_subs 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f677eb09640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.920+0000 7f677eb09640 1 -- 172.21.15.204:0/1289579569 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677811f1e0 con 0x7f67780a08e0 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.921+0000 7f6774ff9640 1 -- 172.21.15.204:0/1289579569 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6764013be0 con 0x7f67780a08e0 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.921+0000 7f677eb09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:50.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.921+0000 7f677eb09640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:50.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.921+0000 7f677eb09640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:50.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.921+0000 7f677eb09640 1 -- 172.21.15.204:0/1289579569 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6778070b80 con 0x7f67780a08e0 2024-09-15T01:35:50.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.921+0000 7f6774ff9640 1 --2- 172.21.15.204:0/1289579569 >> v2:172.21.15.204:6812/28210 conn(0x7f675c0397b0 0x7f675c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:50.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.921+0000 7f6774ff9640 1 -- 172.21.15.204:0/1289579569 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 7723+0+0 (secure 0 0 0) 0x7f676404c430 con 0x7f67780a08e0 2024-09-15T01:35:50.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.921+0000 7f677c8a5640 1 --2- 172.21.15.204:0/1289579569 >> v2:172.21.15.204:6812/28210 conn(0x7f675c0397b0 0x7f675c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:50.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.921+0000 7f677c8a5640 10 monclient: get_auth_request con 0x7f675c0397b0 auth_method 0 2024-09-15T01:35:50.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.921+0000 7f677c8a5640 1 --2- 172.21.15.204:0/1289579569 >> v2:172.21.15.204:6812/28210 conn(0x7f675c0397b0 0x7f675c03bc60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f676c034000 tx=0x7f676c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:50.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.926+0000 7f6774ff9640 1 -- 172.21.15.204:0/1289579569 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f67640138c0 con 0x7f67780a08e0 2024-09-15T01:35:50.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.926+0000 7f6774ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:50.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:50.926+0000 7f6774ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:51.091 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.090+0000 7f677eb09640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-09-15T01:35:51.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.090+0000 7f677eb09640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-09-15T01:35:51.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.090+0000 7f677eb09640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:51.092 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.090+0000 7f677eb09640 1 -- 172.21.15.204:0/1289579569 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"} v 0) v1 -- 0x7f6778067450 con 0x7f67780a08e0 2024-09-15T01:35:51.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.459+0000 7f6774ff9640 1 -- 172.21.15.204:0/1289579569 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}]=0 v145) v1 ==== 113+0+0 (secure 0 0 0) 0x7f67640192c0 con 0x7f67780a08e0 2024-09-15T01:35:51.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.459+0000 7f6774ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-09-15T01:35:51.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.459+0000 7f6774ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:51.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.563+0000 7f677eb09640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-09-15T01:35:51.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.563+0000 7f677eb09640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-09-15T01:35:51.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.563+0000 7f677eb09640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:51.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.563+0000 7f677eb09640 1 -- 172.21.15.204:0/1289579569 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"} v 0) v1 -- 0x7f677811f8a0 con 0x7f67780a08e0 2024-09-15T01:35:51.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.564+0000 7f6774ff9640 1 -- 172.21.15.204:0/1289579569 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}]=0 weight-set for pool 'cold' already created v145) v1 ==== 155+0+0 (secure 0 0 0) 0x7f67640194a0 con 0x7f67780a08e0 2024-09-15T01:35:51.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.564+0000 7f6774ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-09-15T01:35:51.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.564+0000 7f6774ff9640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'cold' already created 2024-09-15T01:35:51.566 INFO:tasks.workunit.client.0.smithi204.stderr:weight-set for pool 'cold' already created 2024-09-15T01:35:51.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 1 -- 172.21.15.204:0/1289579569 >> v2:172.21.15.204:6812/28210 conn(0x7f675c0397b0 msgr2=0x7f675c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:51.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 1 --2- 172.21.15.204:0/1289579569 >> v2:172.21.15.204:6812/28210 conn(0x7f675c0397b0 0x7f675c03bc60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f676c034000 tx=0x7f676c032000 comp rx=0 tx=0).stop 2024-09-15T01:35:51.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 10 monclient: shutdown 2024-09-15T01:35:51.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:51.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 1 -- 172.21.15.204:0/1289579569 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f67780a08e0 msgr2=0x7f67781421e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:51.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 1 --2- 172.21.15.204:0/1289579569 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f67780a08e0 0x7f67781421e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f676400ece0 tx=0x7f67640091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 1 -- 172.21.15.204:0/1289579569 shutdown_connections 2024-09-15T01:35:51.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 1 --2- 172.21.15.204:0/1289579569 >> v2:172.21.15.204:6812/28210 conn(0x7f675c0397b0 0x7f675c03bc60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 1 --2- 172.21.15.204:0/1289579569 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f67780a2cd0 0x7f6778142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 1 --2- 172.21.15.204:0/1289579569 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f67780a08e0 0x7f67781421e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 1 --2- 172.21.15.204:0/1289579569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f677809fa20 0x7f6778141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 1 -- 172.21.15.204:0/1289579569 >> 172.21.15.204:0/1289579569 conn(0x7f677806e860 msgr2=0x7f67780a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:51.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 1 -- 172.21.15.204:0/1289579569 shutdown_connections 2024-09-15T01:35:51.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.566+0000 7f677eb09640 1 -- 172.21.15.204:0/1289579569 wait complete. 2024-09-15T01:35:51.577 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm-device-class osd.0 2024-09-15T01:35:51.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.649+0000 7fd746b5d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.649+0000 7fd746b5d640 10 monclient: build_initial_monmap 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.649+0000 7fd746b5d640 10 monclient: monmap: 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:51.649652+0000 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:51.649652+0000 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:51.651 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:51.652 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:51.652 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:51.652 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:51.652 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:51.652 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:51.652 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:51.652 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:51.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.649+0000 7fd746b5d640 1 Processor -- start 2024-09-15T01:35:51.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.649+0000 7fd746b5d640 1 -- start start 2024-09-15T01:35:51.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.649+0000 7fd746b5d640 10 monclient: init 2024-09-15T01:35:51.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:51.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 10 monclient: _add_conns ranks=[3,8,2,6,4,1,0,5,7,9,10] 2024-09-15T01:35:51.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7400a4680 0x7fd74009e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:51.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 10 monclient(hunting): picked mon.noname-g con 0x7fd7400a4680 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:51.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd74006eea0 0x7fd74009ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:51.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 10 monclient(hunting): picked mon.noname-e con 0x7fd74006eea0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:51.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd74009f2d0 0x7fd7401308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:51.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 10 monclient(hunting): picked mon.noname-b con 0x7fd74009f2d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:51.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:51.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74009f6e0 con 0x7fd74009f2d0 2024-09-15T01:35:51.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:51.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400a4a50 con 0x7fd7400a4680 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd74006dc70 con 0x7fd74006eea0 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd746b5d640 10 monclient(hunting): authenticate will time out at 967.292428s 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7400a4680 0x7fd74009e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7400a4680 0x7fd74009e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:51524/0 (socket says 172.21.15.204:51524) 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 1 -- 172.21.15.204:0/3672005683 learned_addr learned my addr 172.21.15.204:0/3672005683 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 10 monclient(hunting): get_auth_request con 0x7fd7400a4680 auth_method 0 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7450fa640 1 -- 172.21.15.204:0/3672005683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd74009f2d0 msgr2=0x7fd7401308a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:51.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7450fa640 1 --2- 172.21.15.204:0/3672005683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd74009f2d0 0x7fd7401308a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 10 monclient(hunting): handle_auth_done global_id 14862 payload 274 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd737fff640 1 --2- 172.21.15.204:0/3672005683 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd74006eea0 0x7fd74009ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 1 -- 172.21.15.204:0/3672005683 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd74006eea0 msgr2=0x7fd74009ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 1 --2- 172.21.15.204:0/3672005683 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd74006eea0 0x7fd74009ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 1 -- 172.21.15.204:0/3672005683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd74009f2d0 msgr2=0x7fd7401308a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 1 --2- 172.21.15.204:0/3672005683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd74009f2d0 0x7fd7401308a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 10 monclient: _finish_hunting 0 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 1 monclient: found mon.noname-g 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:51.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd7448f9640 1 -- 172.21.15.204:0/3672005683 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74004b720 con 0x7fd7400a4680 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd737fff640 10 monclient: get_auth_request con 0x7fd74006eea0 auth_method 0 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.650+0000 7fd737fff640 1 --2- 172.21.15.204:0/3672005683 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd74006eea0 0x7fd74009ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7448f9640 1 --2- 172.21.15.204:0/3672005683 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7400a4680 0x7fd74009e850 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0099a0 tx=0x7fd73c0091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=2fe311f009d1b3cb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7377fe640 1 -- 172.21.15.204:0/3672005683 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd73c02acc0 con 0x7fd7400a4680 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7377fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7377fe640 10 monclient: dump: 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:51.667 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:51.668 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:51.668 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:51.668 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:51.668 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:51.668 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:51.668 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:51.668 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:51.668 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:51.668 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:51.668 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:51.668 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:51.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7377fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:51.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:51.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7377fe640 1 -- 172.21.15.204:0/3672005683 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c02ae40 con 0x7fd7400a4680 2024-09-15T01:35:51.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd746b5d640 5 monclient: authenticate success, global_id 14862 2024-09-15T01:35:51.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd746b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:51.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7377fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:51.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7377fe640 1 -- 172.21.15.204:0/3672005683 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd73c031ca0 con 0x7fd7400a4680 2024-09-15T01:35:51.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7377fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd7377fe640 10 monclient: dump: 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd746b5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.651+0000 7fd746b5d640 4 set_mon_vals no callback set 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 10 monclient: shutdown 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 1 -- 172.21.15.204:0/3672005683 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7400a4680 msgr2=0x7fd74009e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 1 --2- 172.21.15.204:0/3672005683 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7400a4680 0x7fd74009e850 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0099a0 tx=0x7fd73c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 1 -- 172.21.15.204:0/3672005683 shutdown_connections 2024-09-15T01:35:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 1 --2- 172.21.15.204:0/3672005683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd74009f2d0 0x7fd7401308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 1 --2- 172.21.15.204:0/3672005683 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd74006eea0 0x7fd74009ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 1 --2- 172.21.15.204:0/3672005683 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7400a4680 0x7fd74009e850 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 1 -- 172.21.15.204:0/3672005683 >> 172.21.15.204:0/3672005683 conn(0x7fd740067740 msgr2=0x7fd7400a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 1 -- 172.21.15.204:0/3672005683 shutdown_connections 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 1 -- 172.21.15.204:0/3672005683 wait complete. 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 10 monclient: build_initial_monmap 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.652+0000 7fd746b5d640 10 monclient: monmap: 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:51.673 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 1 Processor -- start 2024-09-15T01:35:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 1 -- start start 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 10 monclient: init 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 10 monclient: _add_conns ranks=[7,4,2,5,3,1,6,0,8] 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd74006eea0 0x7fd74013c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 10 monclient(hunting): picked mon.noname-i con 0x7fd74006eea0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd74009f2d0 0x7fd74013b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 10 monclient(hunting): picked mon.noname-c con 0x7fd74009f2d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7400a4680 0x7fd74013f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 10 monclient(hunting): picked mon.noname-b con 0x7fd7400a4680 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd740121150 con 0x7fd7400a4680 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd740120b90 con 0x7fd74009f2d0 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd74011e6a0 con 0x7fd74006eea0 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd746b5d640 10 monclient(hunting): authenticate will time out at 967.295809s 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd7448f9640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd74006eea0 msgr2=0x7fd74013c160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd7448f9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd74006eea0 0x7fd74013c160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd7450fa640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7400a4680 msgr2=0x7fd74013f6d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.653+0000 7fd7450fa640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7400a4680 0x7fd74013f6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd74009f2d0 0x7fd74013b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd74009f2d0 0x7fd74013b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:58612/0 (socket says 172.21.15.204:58612) 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 1 -- 172.21.15.204:0/2443716521 learned_addr learned my addr 172.21.15.204:0/2443716521 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 10 monclient(hunting): get_auth_request con 0x7fd74009f2d0 auth_method 0 2024-09-15T01:35:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 10 monclient(hunting): handle_auth_done global_id 5692 payload 824 2024-09-15T01:35:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 1 -- 172.21.15.204:0/2443716521 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd74006eea0 msgr2=0x7fd74013c160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd74006eea0 0x7fd74013c160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 1 -- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7400a4680 msgr2=0x7fd74013f6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7400a4680 0x7fd74013f6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 1 monclient: found mon.noname-c 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 1 -- 172.21.15.204:0/2443716521 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c009450 con 0x7fd74009f2d0 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd737fff640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd74009f2d0 0x7fd74013b810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c880 tx=0x7fd730008040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd73000ce10 con 0x7fd74009f2d0 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.654+0000 7fd734ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient: dump: 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd730007d10 con 0x7fd74009f2d0 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd746b5d640 5 monclient: authenticate success, global_id 5692 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd746b5d640 10 monclient: _renew_subs 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd746b5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd746b5d640 1 -- 172.21.15.204:0/2443716521 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7401208b0 con 0x7fd74009f2d0 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd730007e80 con 0x7fd74009f2d0 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient: dump: 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:51.680 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd746b5d640 10 monclient: _renew_subs 2024-09-15T01:35:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd746b5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:35:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd746b5d640 1 -- 172.21.15.204:0/2443716521 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74013bd50 con 0x7fd74009f2d0 2024-09-15T01:35:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd736ffd640 4 set_mon_vals no callback set 2024-09-15T01:35:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 1 -- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd74009f2d0 msgr2=0x7fd74013b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:35:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 1 -- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd74009f2d0 msgr2=0x7fd74013b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:35:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd74009f2d0 0x7fd74013b810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c880 tx=0x7fd730008040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:35:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd74009f2d0 0x7fd74013b810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c880 tx=0x7fd730008040 comp rx=0 tx=0).stop 2024-09-15T01:35:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient: ms_handle_reset current mon [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd74009f2d0 msgr2=0x7fd74013b810 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:35:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd74009f2d0 0x7fd74013b810 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient: _add_conns ranks=[2,3,6,4,7,8,1,5,0] 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7400a4680 0x7fd72c006c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient(hunting): picked mon.b con 0x7fd7400a4680 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd72c0073e0 0x7fd72c0097c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient(hunting): picked mon.g con 0x7fd72c0073e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd72c00a170 0x7fd72c00c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient(hunting): picked mon.d con 0x7fd72c00a170 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c00d1a0 con 0x7fd7400a4680 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c00d310 con 0x7fd72c0073e0 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c00d480 con 0x7fd72c00a170 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd734ff9640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd7448f9640 1 -- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7400a4680 msgr2=0x7fd72c006c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd7448f9640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7400a4680 0x7fd72c006c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd7450fa640 1 -- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd72c00a170 msgr2=0x7fd72c00c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd7450fa640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd72c00a170 0x7fd72c00c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd72c0073e0 0x7fd72c0097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 10 monclient(hunting): get_auth_request con 0x7fd72c0073e0 auth_method 0 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:35:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 10 monclient(hunting): handle_auth_done global_id 5692 payload 293 2024-09-15T01:35:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 1 -- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd72c00a170 msgr2=0x7fd72c00c5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd72c00a170 0x7fd72c00c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 1 -- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7400a4680 msgr2=0x7fd72c006c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7400a4680 0x7fd72c006c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 1 monclient: found mon.g 2024-09-15T01:35:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 1 -- 172.21.15.204:0/2443716521 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) v3 -- 0x7fd72c003f00 con 0x7fd72c0073e0 2024-09-15T01:35:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 10 monclient: _finish_auth 0 2024-09-15T01:35:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.655+0000 7fd737fff640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd72c0073e0 0x7fd72c0097c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd740120a40 tx=0x7fd730019000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.656+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd730028070 con 0x7fd72c0073e0 2024-09-15T01:35:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.656+0000 7fd734ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.656+0000 7fd734ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.656+0000 7fd734ff9640 10 monclient: dump: 2024-09-15T01:35:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.656+0000 7fd746b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.656+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd730007ac0 con 0x7fd72c0073e0 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.656+0000 7fd746b5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.656+0000 7fd746b5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.656+0000 7fd746b5d640 1 -- 172.21.15.204:0/2443716521 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd740070cc0 con 0x7fd72c0073e0 2024-09-15T01:35:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.656+0000 7fd734ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.656+0000 7fd7367fc640 4 set_mon_vals no callback set 2024-09-15T01:35:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.656+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd730023460 con 0x7fd72c0073e0 2024-09-15T01:35:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.657+0000 7fd734ff9640 1 --2- 172.21.15.204:0/2443716521 >> v2:172.21.15.204:6812/28210 conn(0x7fd72c0429e0 0x7fd72c044e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.657+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 <== mon.3 v2:172.21.15.204:3301/0 4 ==== osd_map(145..145 src has 1..145) v4 ==== 8195+0+0 (secure 0 0 0) 0x7fd73004dff0 con 0x7fd72c0073e0 2024-09-15T01:35:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.657+0000 7fd7448f9640 1 --2- 172.21.15.204:0/2443716521 >> v2:172.21.15.204:6812/28210 conn(0x7fd72c0429e0 0x7fd72c044e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.657+0000 7fd7448f9640 10 monclient: get_auth_request con 0x7fd72c0429e0 auth_method 0 2024-09-15T01:35:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.657+0000 7fd7448f9640 1 --2- 172.21.15.204:0/2443716521 >> v2:172.21.15.204:6812/28210 conn(0x7fd72c0429e0 0x7fd72c044e90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd73c034000 tx=0x7fd73c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.660+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 <== mon.3 v2:172.21.15.204:3301/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd730021b30 con 0x7fd72c0073e0 2024-09-15T01:35:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.660+0000 7fd734ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.660+0000 7fd734ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:51.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.822+0000 7fd746b5d640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:35:51.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.822+0000 7fd746b5d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:35:51.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.822+0000 7fd746b5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:51.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:51.822+0000 7fd746b5d640 1 -- 172.21.15.204:0/2443716521 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) v1 -- 0x7fd74006eb50 con 0x7fd72c0073e0 2024-09-15T01:35:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:52.733+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 done removing class of osd(s): 0 v146) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd730021350 con 0x7fd72c0073e0 2024-09-15T01:35:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:52.733+0000 7fd734ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:35:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:52.733+0000 7fd734ff9640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0 2024-09-15T01:35:52.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:52.832+0000 7fd746b5d640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:35:52.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:52.832+0000 7fd746b5d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:35:52.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:52.832+0000 7fd746b5d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:52.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:52.832+0000 7fd746b5d640 1 -- 172.21.15.204:0/2443716521 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) v1 -- 0x7fd740121290 con 0x7fd72c0073e0 2024-09-15T01:35:53.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.736+0000 7fd734ff9640 1 -- 172.21.15.204:0/2443716521 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 osd.0 belongs to no class, done removing class of osd(s): v147) v1 ==== 149+0+0 (secure 0 0 0) 0x7fd740121290 con 0x7fd72c0073e0 2024-09-15T01:35:53.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.736+0000 7fd734ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:35:53.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.736+0000 7fd734ff9640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T01:35:53.739 INFO:tasks.workunit.client.0.smithi204.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T01:35:53.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.740+0000 7fd746b5d640 1 -- 172.21.15.204:0/2443716521 >> v2:172.21.15.204:6812/28210 conn(0x7fd72c0429e0 msgr2=0x7fd72c044e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:53.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.740+0000 7fd746b5d640 1 --2- 172.21.15.204:0/2443716521 >> v2:172.21.15.204:6812/28210 conn(0x7fd72c0429e0 0x7fd72c044e90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd73c034000 tx=0x7fd73c032000 comp rx=0 tx=0).stop 2024-09-15T01:35:53.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.740+0000 7fd7450fa640 1 -- 172.21.15.204:0/2443716521 reap_dead start 2024-09-15T01:35:53.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.740+0000 7fd746b5d640 10 monclient: shutdown 2024-09-15T01:35:53.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.740+0000 7fd746b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:53.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.740+0000 7fd746b5d640 1 -- 172.21.15.204:0/2443716521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd72c0073e0 msgr2=0x7fd72c0097c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:53.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.740+0000 7fd746b5d640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd72c0073e0 0x7fd72c0097c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd740120a40 tx=0x7fd730019000 comp rx=0 tx=0).stop 2024-09-15T01:35:53.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.740+0000 7fd746b5d640 1 -- 172.21.15.204:0/2443716521 shutdown_connections 2024-09-15T01:35:53.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.740+0000 7fd746b5d640 1 --2- 172.21.15.204:0/2443716521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd72c0073e0 0x7fd72c0097c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:53.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.740+0000 7fd746b5d640 1 -- 172.21.15.204:0/2443716521 >> 172.21.15.204:0/2443716521 conn(0x7fd740067740 msgr2=0x7fd74009d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:53.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.741+0000 7fd746b5d640 1 -- 172.21.15.204:0/2443716521 shutdown_connections 2024-09-15T01:35:53.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.741+0000 7fd746b5d640 1 -- 172.21.15.204:0/2443716521 wait complete. 2024-09-15T01:35:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set reweight-compat osd.0 10.5 2024-09-15T01:35:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.833+0000 7f966711d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.833+0000 7f966711d640 10 monclient: build_initial_monmap 2024-09-15T01:35:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.834+0000 7f966711d640 10 monclient: monmap: 2024-09-15T01:35:53.835 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:53.834458+0000 2024-09-15T01:35:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:53.834458+0000 2024-09-15T01:35:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:53.840 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:53.841 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:53.841 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.834+0000 7f966711d640 1 Processor -- start 2024-09-15T01:35:53.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.834+0000 7f966711d640 1 -- start start 2024-09-15T01:35:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.834+0000 7f966711d640 10 monclient: init 2024-09-15T01:35:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 10 monclient: _add_conns ranks=[10,9,6,8,5,1,7,2,3,4,0] 2024-09-15T01:35:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f96600a2cd0 0x7f9660132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 10 monclient(hunting): picked mon.noname-f con 0x7f96600a2cd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f966009fa20 0x7f966009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 10 monclient(hunting): picked mon.noname-a con 0x7f966009fa20 addr v2:172.21.15.178:3300/0 2024-09-15T01:35:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f96600a08e0 0x7f96600a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 10 monclient(hunting): picked mon.noname-d con 0x7f96600a08e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96600a30a0 con 0x7f966009fa20 2024-09-15T01:35:53.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96600a0330 con 0x7f96600a2cd0 2024-09-15T01:35:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f96600a1db0 con 0x7f96600a08e0 2024-09-15T01:35:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f966711d640 10 monclient(hunting): authenticate will time out at 969.477398s 2024-09-15T01:35:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f96600a2cd0 0x7f9660132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f96656ba640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f96600a08e0 msgr2=0x7f96600a0d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f96600a2cd0 0x7f9660132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:50728/0 (socket says 172.21.15.204:50728) 2024-09-15T01:35:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f96656ba640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f96600a08e0 0x7f96600a0d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 1 -- 172.21.15.204:0/1661775842 learned_addr learned my addr 172.21.15.204:0/1661775842 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 10 monclient(hunting): get_auth_request con 0x7f96600a2cd0 auth_method 0 2024-09-15T01:35:53.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f965ffff640 1 --2- 172.21.15.204:0/1661775842 >> v2:172.21.15.178:3300/0 conn(0x7f966009fa20 0x7f966009fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f965ffff640 10 monclient(hunting): get_auth_request con 0x7f966009fa20 auth_method 0 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f965ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f965ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f965ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 10 monclient(hunting): handle_auth_done global_id 6373 payload 274 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 1 -- 172.21.15.204:0/1661775842 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f96600a08e0 msgr2=0x7f96600a0d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:53.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 1 --2- 172.21.15.204:0/1661775842 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f96600a08e0 0x7f96600a0d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 1 -- 172.21.15.204:0/1661775842 >> v2:172.21.15.178:3300/0 conn(0x7f966009fa20 msgr2=0x7f966009fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 1 --2- 172.21.15.204:0/1661775842 >> v2:172.21.15.178:3300/0 conn(0x7f966009fa20 0x7f966009fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 10 monclient: _finish_hunting 0 2024-09-15T01:35:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 1 monclient: found mon.noname-f 2024-09-15T01:35:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.835+0000 7f9664eb9640 1 -- 172.21.15.204:0/1661775842 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96600648a0 con 0x7f96600a2cd0 2024-09-15T01:35:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f9664eb9640 1 --2- 172.21.15.204:0/1661775842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96600a2cd0 0x7f9660132c00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f96600a3a30 tx=0x7f96500091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf663ad5a3ab5561 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:53.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f965f7fe640 1 -- 172.21.15.204:0/1661775842 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f965002acc0 con 0x7f96600a2cd0 2024-09-15T01:35:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f965f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f965f7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f965f7fe640 10 monclient: dump: 2024-09-15T01:35:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:53.855 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:53.856 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f965f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:35:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f965f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:53.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f966711d640 5 monclient: authenticate success, global_id 6373 2024-09-15T01:35:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f966711d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f965f7fe640 1 -- 172.21.15.204:0/1661775842 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965002ae40 con 0x7f96600a2cd0 2024-09-15T01:35:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f965f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f966711d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f965f7fe640 1 -- 172.21.15.204:0/1661775842 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9650031ca0 con 0x7f96600a2cd0 2024-09-15T01:35:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f966711d640 4 set_mon_vals no callback set 2024-09-15T01:35:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f965f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f965f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.836+0000 7f965f7fe640 10 monclient: dump: 2024-09-15T01:35:53.857 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:53.858 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:53.859 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 10 monclient: shutdown 2024-09-15T01:35:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 1 -- 172.21.15.204:0/1661775842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96600a2cd0 msgr2=0x7f9660132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 1 --2- 172.21.15.204:0/1661775842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96600a2cd0 0x7f9660132c00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f96600a3a30 tx=0x7f96500091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 1 -- 172.21.15.204:0/1661775842 shutdown_connections 2024-09-15T01:35:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 1 --2- 172.21.15.204:0/1661775842 >> v2:172.21.15.178:3300/0 conn(0x7f966009fa20 0x7f966009fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:53.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 1 --2- 172.21.15.204:0/1661775842 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f96600a08e0 0x7f96600a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 1 --2- 172.21.15.204:0/1661775842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96600a2cd0 0x7f9660132c00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 1 -- 172.21.15.204:0/1661775842 >> 172.21.15.204:0/1661775842 conn(0x7f966006e860 msgr2=0x7f966006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 1 -- 172.21.15.204:0/1661775842 shutdown_connections 2024-09-15T01:35:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 1 -- 172.21.15.204:0/1661775842 wait complete. 2024-09-15T01:35:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 10 monclient: build_initial_monmap 2024-09-15T01:35:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.837+0000 7f966711d640 10 monclient: monmap: 2024-09-15T01:35:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:53.860 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:53.861 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:53.862 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 1 Processor -- start 2024-09-15T01:35:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 1 -- start start 2024-09-15T01:35:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 10 monclient: init 2024-09-15T01:35:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 10 monclient: _add_conns ranks=[7,2,1,0,6,3,5,4,8] 2024-09-15T01:35:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f966009fa20 0x7f9660141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 10 monclient(hunting): picked mon.noname-i con 0x7f966009fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:53.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96600a08e0 0x7f96601311b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 10 monclient(hunting): picked mon.noname-b con 0x7f96600a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96600a2cd0 0x7f966011dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 10 monclient(hunting): picked mon.noname-f con 0x7f96600a2cd0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f9664eb9640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f966009fa20 msgr2=0x7f9660141b80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9660121300 con 0x7f96600a2cd0 2024-09-15T01:35:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966009cb60 con 0x7f96600a08e0 2024-09-15T01:35:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:53.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9660142a60 con 0x7f966009fa20 2024-09-15T01:35:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f9664eb9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f966009fa20 0x7f9660141b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f966711d640 10 monclient(hunting): authenticate will time out at 969.480851s 2024-09-15T01:35:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f96656ba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96600a2cd0 0x7f966011dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f965ffff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96600a08e0 msgr2=0x7f96601311b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f965ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96600a08e0 0x7f96601311b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f96656ba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96600a2cd0 0x7f966011dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:50740/0 (socket says 172.21.15.204:50740) 2024-09-15T01:35:53.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f96656ba640 1 -- 172.21.15.204:0/2670431705 learned_addr learned my addr 172.21.15.204:0/2670431705 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f96656ba640 10 monclient(hunting): get_auth_request con 0x7f96600a2cd0 auth_method 0 2024-09-15T01:35:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f96656ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f96656ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.838+0000 7f96656ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 10 monclient(hunting): handle_auth_done global_id 6382 payload 824 2024-09-15T01:35:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 1 -- 172.21.15.204:0/2670431705 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f966009fa20 msgr2=0x7f9660141b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:53.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 1 --2- 172.21.15.204:0/2670431705 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f966009fa20 0x7f9660141b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 1 -- 172.21.15.204:0/2670431705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96600a08e0 msgr2=0x7f96601311b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 1 --2- 172.21.15.204:0/2670431705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96600a08e0 0x7f96601311b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 10 monclient: _finish_hunting 0 2024-09-15T01:35:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 1 monclient: found mon.noname-f 2024-09-15T01:35:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:53.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 1 -- 172.21.15.204:0/2670431705 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9650009450 con 0x7f96600a2cd0 2024-09-15T01:35:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f96656ba640 1 --2- 172.21.15.204:0/2670431705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96600a2cd0 0x7f966011dd20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f965800bf80 tx=0x7f9658009730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965cff9640 1 -- 172.21.15.204:0/2670431705 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f965800cd90 con 0x7f96600a2cd0 2024-09-15T01:35:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965cff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:53.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965cff9640 10 monclient: dump: 2024-09-15T01:35:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:53.868 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:53.869 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:53.869 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:53.869 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:53.869 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:53.869 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:53.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965cff9640 10 monclient: _finish_auth 0 2024-09-15T01:35:53.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:53.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965cff9640 1 -- 172.21.15.204:0/2670431705 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96580065d0 con 0x7f96600a2cd0 2024-09-15T01:35:53.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f966711d640 5 monclient: authenticate success, global_id 6382 2024-09-15T01:35:53.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f966711d640 10 monclient: _renew_subs 2024-09-15T01:35:53.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f966711d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:53.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f966711d640 1 -- 172.21.15.204:0/2670431705 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966009d3d0 con 0x7f96600a2cd0 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965cff9640 1 -- 172.21.15.204:0/2670431705 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96580153e0 con 0x7f96600a2cd0 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965cff9640 10 monclient: dump: 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:53.870 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f966711d640 10 monclient: _renew_subs 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f966711d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f965effd640 4 set_mon_vals no callback set 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.839+0000 7f966711d640 1 -- 172.21.15.204:0/2670431705 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966011f7d0 con 0x7f96600a2cd0 2024-09-15T01:35:53.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.840+0000 7f965cff9640 1 -- 172.21.15.204:0/2670431705 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9658012020 con 0x7f96600a2cd0 2024-09-15T01:35:53.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.840+0000 7f965cff9640 1 --2- 172.21.15.204:0/2670431705 >> v2:172.21.15.204:6812/28210 conn(0x7f964c0397b0 0x7f964c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:53.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.840+0000 7f966711d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:53.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.840+0000 7f966711d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:53.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.840+0000 7f966711d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:53.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.840+0000 7f965cff9640 1 -- 172.21.15.204:0/2670431705 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 8163+0+0 (secure 0 0 0) 0x7f965804c470 con 0x7f96600a2cd0 2024-09-15T01:35:53.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.840+0000 7f966711d640 1 -- 172.21.15.204:0/2670431705 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9660070b80 con 0x7f96600a2cd0 2024-09-15T01:35:53.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.840+0000 7f9664eb9640 1 --2- 172.21.15.204:0/2670431705 >> v2:172.21.15.204:6812/28210 conn(0x7f964c0397b0 0x7f964c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:53.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.840+0000 7f9664eb9640 10 monclient: get_auth_request con 0x7f964c0397b0 auth_method 0 2024-09-15T01:35:53.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.841+0000 7f9664eb9640 1 --2- 172.21.15.204:0/2670431705 >> v2:172.21.15.204:6812/28210 conn(0x7f964c0397b0 0x7f964c03bc60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9650034000 tx=0x7f9650032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:53.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.845+0000 7f965cff9640 1 -- 172.21.15.204:0/2670431705 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f965801b030 con 0x7f96600a2cd0 2024-09-15T01:35:53.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.845+0000 7f965cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:53.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:53.845+0000 7f965cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:54.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:54.006+0000 7f966711d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-09-15T01:35:54.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:54.006+0000 7f966711d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-09-15T01:35:54.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:54.006+0000 7f966711d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:54.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:54.006+0000 7f966711d640 1 -- 172.21.15.204:0/2670431705 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]} v 0) v1 -- 0x7f9660067450 con 0x7f96600a2cd0 2024-09-15T01:35:54.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:54.754+0000 7f965cff9640 1 -- 172.21.15.204:0/2670431705 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}]=0 v148) v1 ==== 119+0+0 (secure 0 0 0) 0x7f965801b020 con 0x7f96600a2cd0 2024-09-15T01:35:54.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:54.754+0000 7f965cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-09-15T01:35:54.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:54.754+0000 7f965cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:54.855+0000 7f966711d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-09-15T01:35:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:54.855+0000 7f966711d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-09-15T01:35:54.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:54.855+0000 7f966711d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:54.857 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:54.855+0000 7f966711d640 1 -- 172.21.15.204:0/2670431705 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]} v 0) v1 -- 0x7f966011fec0 con 0x7f96600a2cd0 2024-09-15T01:35:55.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.758+0000 7f965cff9640 1 -- 172.21.15.204:0/2670431705 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}]=0 v149) v1 ==== 119+0+0 (secure 0 0 0) 0x7f965801b020 con 0x7f96600a2cd0 2024-09-15T01:35:55.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.758+0000 7f965cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-09-15T01:35:55.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.758+0000 7f965cff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:55.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.761+0000 7f966711d640 1 -- 172.21.15.204:0/2670431705 >> v2:172.21.15.204:6812/28210 conn(0x7f964c0397b0 msgr2=0x7f964c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:55.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.761+0000 7f966711d640 1 --2- 172.21.15.204:0/2670431705 >> v2:172.21.15.204:6812/28210 conn(0x7f964c0397b0 0x7f964c03bc60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9650034000 tx=0x7f9650032000 comp rx=0 tx=0).stop 2024-09-15T01:35:55.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.761+0000 7f966711d640 10 monclient: shutdown 2024-09-15T01:35:55.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.761+0000 7f966711d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:55.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.761+0000 7f966711d640 1 -- 172.21.15.204:0/2670431705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96600a2cd0 msgr2=0x7f966011dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:55.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.761+0000 7f966711d640 1 --2- 172.21.15.204:0/2670431705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96600a2cd0 0x7f966011dd20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f965800bf80 tx=0x7f9658009730 comp rx=0 tx=0).stop 2024-09-15T01:35:55.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.761+0000 7f966711d640 1 -- 172.21.15.204:0/2670431705 shutdown_connections 2024-09-15T01:35:55.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.761+0000 7f966711d640 1 --2- 172.21.15.204:0/2670431705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96600a2cd0 0x7f966011dd20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:55.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.761+0000 7f966711d640 1 --2- 172.21.15.204:0/2670431705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96600a08e0 0x7f96601311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:55.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.761+0000 7f966711d640 1 --2- 172.21.15.204:0/2670431705 >> v2:172.21.15.204:6812/28210 conn(0x7f964c0397b0 0x7f964c03bc60 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9650034000 tx=0x7f9650032000 comp rx=0 tx=0).stop 2024-09-15T01:35:55.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.761+0000 7f966711d640 1 --2- 172.21.15.204:0/2670431705 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f966009fa20 0x7f9660141b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:55.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.761+0000 7f966711d640 1 -- 172.21.15.204:0/2670431705 >> 172.21.15.204:0/2670431705 conn(0x7f966006e860 msgr2=0x7f96600a51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:55.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.762+0000 7f966711d640 1 -- 172.21.15.204:0/2670431705 shutdown_connections 2024-09-15T01:35:55.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.762+0000 7f966711d640 1 -- 172.21.15.204:0/2670431705 wait complete. 2024-09-15T01:35:55.778 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set reweight cool osd.0 11.5 2024-09-15T01:35:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.857+0000 7efd99954640 10 monclient: get_monmap_and_config 2024-09-15T01:35:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.857+0000 7efd99954640 10 monclient: build_initial_monmap 2024-09-15T01:35:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.857+0000 7efd99954640 10 monclient: monmap: 2024-09-15T01:35:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:55.859 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:55.858389+0000 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:55.858389+0000 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:55.860 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:55.861 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:55.861 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 1 Processor -- start 2024-09-15T01:35:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 1 -- start start 2024-09-15T01:35:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 10 monclient: init 2024-09-15T01:35:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 10 monclient: _add_conns ranks=[1,0,6,2,10,3,8,7,9,4,5] 2024-09-15T01:35:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7efd940a2cd0 addr v1:172.21.15.204:6789/0 2024-09-15T01:35:55.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efd9409fa20 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd94138c90 0x7efd9413b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 10 monclient(hunting): picked mon.noname-d con 0x7efd94138c90 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd940648a0 con 0x7efd9409fa20 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd94133310 con 0x7efd940a2cd0 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efd940a30a0 con 0x7efd94138c90 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.858+0000 7efd99954640 10 monclient(hunting): authenticate will time out at 971.500986s 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd92ffd640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7efd940a2cd0 0x7efd94132c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:44588/0 (socket says 172.21.15.204:44588) 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd92ffd640 1 -- 172.21.15.204:0/1923336778 learned_addr learned my addr 172.21.15.204:0/1923336778 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd937fe640 1 -- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd94138c90 msgr2=0x7efd9413b070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd937fe640 1 --2- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd94138c90 0x7efd9413b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:55.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538148848 0 0) 0x7efd94133310 con 0x7efd940a2cd0 2024-09-15T01:35:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient(hunting): my global_id is 6391 2024-09-15T01:35:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd7c0034e0 con 0x7efd940a2cd0 2024-09-15T01:35:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1929247600 0 0) 0x7efd7c0034e0 con 0x7efd940a2cd0 2024-09-15T01:35:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd94138c90 msgr2=0x7efd9413b070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 --2- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd94138c90 0x7efd9413b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 >> v1:172.21.15.178:6789/0 conn(0x7efd9409fa20 legacy=0x7efd9409fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient: _finish_hunting 0 2024-09-15T01:35:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:35:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:55.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd940a1db0 con 0x7efd940a2cd0 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient: _finish_auth 0 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7efd840018f0 con 0x7efd940a2cd0 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd99954640 5 monclient: authenticate success, global_id 6391 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd99954640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient: dump: 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:55.874 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:35:55.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 >> v1:172.21.15.204:6789/0 conn(0x7efd940a2cd0 legacy=0x7efd94132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient: _add_conns ranks=[7,0,6,2,3,1,4,5,8] 2024-09-15T01:35:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 --2- 172.21.15.204:0/1923336778 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efd7c004220 0x7efd7c004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient(hunting): picked mon.i con 0x7efd7c004220 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 --2- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd7c004ec0 0x7efd7c0099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient(hunting): picked mon.a con 0x7efd7c004ec0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 --2- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd94138c90 0x7efd7c00bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient(hunting): picked mon.d con 0x7efd94138c90 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:35:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd940a30a0 con 0x7efd7c004ec0 2024-09-15T01:35:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:55.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c00c5e0 con 0x7efd94138c90 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c00c750 con 0x7efd7c004220 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd92ffd640 1 -- 172.21.15.204:0/1923336778 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efd7c004220 msgr2=0x7efd7c004780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd91ffb640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd92ffd640 1 --2- 172.21.15.204:0/1923336778 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efd7c004220 0x7efd7c004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd99954640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd937fe640 1 -- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd94138c90 msgr2=0x7efd7c00bf10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd937fe640 1 --2- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd94138c90 0x7efd7c00bf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd927fc640 1 --2- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd7c004ec0 0x7efd7c0099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd927fc640 10 monclient(hunting): get_auth_request con 0x7efd7c004ec0 auth_method 0 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd927fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:55.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd927fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd927fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd927fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd927fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.859+0000 7efd927fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 10 monclient(hunting): handle_auth_done global_id 6391 payload 293 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 1 -- 172.21.15.204:0/1923336778 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efd7c004220 msgr2=0x7efd7c004780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 1 --2- 172.21.15.204:0/1923336778 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efd7c004220 0x7efd7c004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 1 -- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd94138c90 msgr2=0x7efd7c00bf10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 1 --2- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd94138c90 0x7efd7c00bf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 10 monclient: _finish_hunting 0 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 1 monclient: found mon.a 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:55.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:35:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 1 -- 172.21.15.204:0/1923336778 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c0034e0 con 0x7efd7c004ec0 2024-09-15T01:35:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:55.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.860+0000 7efd927fc640 1 --2- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd7c004ec0 0x7efd7c0099c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efd88001800 tx=0x7efd88007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434570637a96b2dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd8802ad80 con 0x7efd7c004ec0 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd91ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd91ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd91ffb640 10 monclient: dump: 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:55.881 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd88031390 con 0x7efd7c004ec0 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd99954640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd91ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd99954640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd91ffb640 1 -- 172.21.15.204:0/1923336778 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd88031980 con 0x7efd7c004ec0 2024-09-15T01:35:55.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd91ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd99954640 4 set_mon_vals no callback set 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd91ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd91ffb640 10 monclient: dump: 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:55.883 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:55.884 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:55.884 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:55.884 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:55.884 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:55.884 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:55.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd99954640 10 monclient: shutdown 2024-09-15T01:35:55.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd99954640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:55.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd99954640 1 -- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd7c004ec0 msgr2=0x7efd7c0099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:55.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd99954640 1 --2- 172.21.15.204:0/1923336778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd7c004ec0 0x7efd7c0099c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efd88001800 tx=0x7efd88007ca0 comp rx=0 tx=0).stop 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd937fe640 1 -- 172.21.15.204:0/1923336778 reap_dead start 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.861+0000 7efd99954640 1 -- 172.21.15.204:0/1923336778 shutdown_connections 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.862+0000 7efd99954640 1 -- 172.21.15.204:0/1923336778 >> 172.21.15.204:0/1923336778 conn(0x7efd9406e860 msgr2=0x7efd9406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.862+0000 7efd99954640 1 -- 172.21.15.204:0/1923336778 shutdown_connections 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.862+0000 7efd99954640 1 -- 172.21.15.204:0/1923336778 wait complete. 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.862+0000 7efd99954640 10 monclient: build_initial_monmap 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.862+0000 7efd99954640 10 monclient: monmap: 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:55.885 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 1 Processor -- start 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 1 -- start start 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 10 monclient: init 2024-09-15T01:35:55.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 10 monclient: _add_conns ranks=[5,3,0,2,4,7,8,6,1] 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efd7c004ec0 0x7efd9412f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 10 monclient(hunting): picked mon.noname-h con 0x7efd7c004ec0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd9412f6f0 0x7efd9412fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 10 monclient(hunting): picked mon.noname-g con 0x7efd9412f6f0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd94130000 0x7efd94148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 10 monclient(hunting): picked mon.noname-a con 0x7efd94130000 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd94135d40 con 0x7efd94130000 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9409cb60 con 0x7efd9412f6f0 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:55.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd94135780 con 0x7efd7c004ec0 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd99954640 10 monclient(hunting): authenticate will time out at 971.505530s 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efd7c004ec0 0x7efd9412f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd927fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd9412f6f0 0x7efd9412fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efd7c004ec0 0x7efd9412f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:44376/0 (socket says 172.21.15.204:44376) 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 1 -- 172.21.15.204:0/2736310513 learned_addr learned my addr 172.21.15.204:0/2736310513 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 10 monclient(hunting): get_auth_request con 0x7efd7c004ec0 auth_method 0 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd927fc640 10 monclient(hunting): get_auth_request con 0x7efd9412f6f0 auth_method 0 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd927fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:55.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd927fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd927fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd937fe640 1 --2- 172.21.15.204:0/2736310513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd94130000 0x7efd94148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd927fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd927fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd927fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd937fe640 10 monclient(hunting): get_auth_request con 0x7efd94130000 auth_method 0 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd937fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:55.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:55.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:55.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 10 monclient(hunting): handle_auth_done global_id 5774 payload 824 2024-09-15T01:35:55.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 1 -- 172.21.15.204:0/2736310513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd9412f6f0 msgr2=0x7efd9412fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:55.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 1 --2- 172.21.15.204:0/2736310513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd9412f6f0 0x7efd9412fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:55.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 1 -- 172.21.15.204:0/2736310513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd94130000 msgr2=0x7efd94148120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:55.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 1 --2- 172.21.15.204:0/2736310513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd94130000 0x7efd94148120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:55.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:35:55.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 1 monclient: found mon.noname-h 2024-09-15T01:35:55.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:55.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:35:55.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd92ffd640 1 -- 172.21.15.204:0/2736310513 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd94138620 con 0x7efd7c004ec0 2024-09-15T01:35:55.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.863+0000 7efd927fc640 1 --2- 172.21.15.204:0/2736310513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd9412f6f0 0x7efd9412fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd92ffd640 1 --2- 172.21.15.204:0/2736310513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efd7c004ec0 0x7efd9412f1b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efd8400c790 tx=0x7efd840015b0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd7affd640 1 -- 172.21.15.204:0/2736310513 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd840075d0 con 0x7efd7c004ec0 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd7affd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd7affd640 10 monclient: dump: 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:55.891 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd7affd640 10 monclient: _finish_auth 0 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd7affd640 1 -- 172.21.15.204:0/2736310513 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd840047a0 con 0x7efd7c004ec0 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd99954640 5 monclient: authenticate success, global_id 5774 2024-09-15T01:35:55.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd7affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd7affd640 1 -- 172.21.15.204:0/2736310513 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd84004dd0 con 0x7efd7c004ec0 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd99954640 10 monclient: _renew_subs 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd99954640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd99954640 1 -- 172.21.15.204:0/2736310513 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd94135a60 con 0x7efd7c004ec0 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd7affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd7affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd7affd640 10 monclient: dump: 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:55.893 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd917fa640 4 set_mon_vals no callback set 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd99954640 10 monclient: _renew_subs 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd99954640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:55.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.864+0000 7efd99954640 1 -- 172.21.15.204:0/2736310513 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd94130430 con 0x7efd7c004ec0 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.865+0000 7efd7affd640 1 -- 172.21.15.204:0/2736310513 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7efd84004950 con 0x7efd7c004ec0 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.865+0000 7efd99954640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.865+0000 7efd99954640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.865+0000 7efd99954640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.865+0000 7efd99954640 1 -- 172.21.15.204:0/2736310513 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd94135f40 con 0x7efd7c004ec0 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.865+0000 7efd7affd640 1 --2- 172.21.15.204:0/2736310513 >> v2:172.21.15.204:6812/28210 conn(0x7efd800396e0 0x7efd8003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.865+0000 7efd7affd640 1 -- 172.21.15.204:0/2736310513 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 8349+0+0 (secure 0 0 0) 0x7efd8404b730 con 0x7efd7c004ec0 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.865+0000 7efd927fc640 1 --2- 172.21.15.204:0/2736310513 >> v2:172.21.15.204:6812/28210 conn(0x7efd800396e0 0x7efd8003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.865+0000 7efd927fc640 10 monclient: get_auth_request con 0x7efd800396e0 auth_method 0 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.866+0000 7efd927fc640 1 --2- 172.21.15.204:0/2736310513 >> v2:172.21.15.204:6812/28210 conn(0x7efd800396e0 0x7efd8003bb90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efd88032000 tx=0x7efd88003000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.870+0000 7efd7affd640 1 -- 172.21.15.204:0/2736310513 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7efd84004c00 con 0x7efd7c004ec0 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.870+0000 7efd7affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:55.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:55.870+0000 7efd7affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:56.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:56.030+0000 7efd99954640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-09-15T01:35:56.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:56.030+0000 7efd99954640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-09-15T01:35:56.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:56.030+0000 7efd99954640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:56.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:56.030+0000 7efd99954640 1 -- 172.21.15.204:0/2736310513 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]} v 0) v1 -- 0x7efd94070b80 con 0x7efd7c004ec0 2024-09-15T01:35:56.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:56.759+0000 7efd7affd640 1 -- 172.21.15.204:0/2736310513 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}]=0 v150) v1 ==== 128+0+0 (secure 0 0 0) 0x7efd84018020 con 0x7efd7c004ec0 2024-09-15T01:35:56.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:56.759+0000 7efd7affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-09-15T01:35:56.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:56.759+0000 7efd7affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:56.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:56.862+0000 7efd99954640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-09-15T01:35:56.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:56.862+0000 7efd99954640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-09-15T01:35:56.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:56.862+0000 7efd99954640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:35:56.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:56.862+0000 7efd99954640 1 -- 172.21.15.204:0/2736310513 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]} v 0) v1 -- 0x7efd940008d0 con 0x7efd7c004ec0 2024-09-15T01:35:57.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.761+0000 7efd7affd640 1 -- 172.21.15.204:0/2736310513 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}]=0 v151) v1 ==== 128+0+0 (secure 0 0 0) 0x7efd84018030 con 0x7efd7c004ec0 2024-09-15T01:35:57.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.761+0000 7efd7affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-09-15T01:35:57.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.761+0000 7efd7affd640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:57.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 1 -- 172.21.15.204:0/2736310513 >> v2:172.21.15.204:6812/28210 conn(0x7efd800396e0 msgr2=0x7efd8003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:57.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 1 --2- 172.21.15.204:0/2736310513 >> v2:172.21.15.204:6812/28210 conn(0x7efd800396e0 0x7efd8003bb90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efd88032000 tx=0x7efd88003000 comp rx=0 tx=0).stop 2024-09-15T01:35:57.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 10 monclient: shutdown 2024-09-15T01:35:57.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:57.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 1 -- 172.21.15.204:0/2736310513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efd7c004ec0 msgr2=0x7efd9412f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:57.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 1 --2- 172.21.15.204:0/2736310513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efd7c004ec0 0x7efd9412f1b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efd8400c790 tx=0x7efd840015b0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 1 -- 172.21.15.204:0/2736310513 shutdown_connections 2024-09-15T01:35:57.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 1 --2- 172.21.15.204:0/2736310513 >> v2:172.21.15.204:6812/28210 conn(0x7efd800396e0 0x7efd8003bb90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 1 --2- 172.21.15.204:0/2736310513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd94130000 0x7efd94148120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 1 --2- 172.21.15.204:0/2736310513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd9412f6f0 0x7efd9412fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 1 --2- 172.21.15.204:0/2736310513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efd7c004ec0 0x7efd9412f1b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 1 -- 172.21.15.204:0/2736310513 >> 172.21.15.204:0/2736310513 conn(0x7efd9406e860 msgr2=0x7efd9412cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:57.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.764+0000 7efd99954640 1 -- 172.21.15.204:0/2736310513 shutdown_connections 2024-09-15T01:35:57.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.765+0000 7efd99954640 1 -- 172.21.15.204:0/2736310513 wait complete. 2024-09-15T01:35:57.777 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set reweight cold osd.0 12.5 12.4 2024-09-15T01:35:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.856+0000 7f71d295d640 10 monclient: get_monmap_and_config 2024-09-15T01:35:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.856+0000 7f71d295d640 10 monclient: build_initial_monmap 2024-09-15T01:35:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.856+0000 7f71d295d640 10 monclient: monmap: 2024-09-15T01:35:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:57.857232+0000 2024-09-15T01:35:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:57.857232+0000 2024-09-15T01:35:57.858 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:57.859 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.857+0000 7f71d295d640 1 Processor -- start 2024-09-15T01:35:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.857+0000 7f71d295d640 1 -- start start 2024-09-15T01:35:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.857+0000 7f71d295d640 10 monclient: init 2024-09-15T01:35:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.857+0000 7f71d295d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.857+0000 7f71d295d640 10 monclient: _add_conns ranks=[10,9,7,4,0,2,5,8,3,6,1] 2024-09-15T01:35:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.857+0000 7f71d295d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f71cc0a2e30 0x7f71cc132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 10 monclient(hunting): picked mon.noname-f con 0x7f71cc0a2e30 addr v2:172.21.15.204:3300/0 2024-09-15T01:35:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f71cc09fb80 0x7f71cc09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 10 monclient(hunting): picked mon.noname-a con 0x7f71cc09fb80 addr v2:172.21.15.178:3300/0 2024-09-15T01:35:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71cc0a0a40 0x7f71cc0a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 10 monclient(hunting): picked mon.noname-i con 0x7f71cc0a0a40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:57.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71cc09e6c0 con 0x7f71cc09fb80 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71cc0a04b0 con 0x7f71cc0a2e30 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0a3200 con 0x7f71cc0a0a40 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d295d640 10 monclient(hunting): authenticate will time out at 973.500073s 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d0efa640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71cc0a0a40 msgr2=0x7f71cc0a0e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71d0efa640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71cc0a0a40 0x7f71cc0a0e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f71cc0a2e30 0x7f71cc132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f71cc0a2e30 0x7f71cc132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43418/0 (socket says 172.21.15.204:43418) 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 1 -- 172.21.15.204:0/3454948081 learned_addr learned my addr 172.21.15.204:0/3454948081 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:57.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 10 monclient(hunting): get_auth_request con 0x7f71cc0a2e30 auth_method 0 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cb7fe640 1 --2- 172.21.15.204:0/3454948081 >> v2:172.21.15.178:3300/0 conn(0x7f71cc09fb80 0x7f71cc09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cb7fe640 10 monclient(hunting): get_auth_request con 0x7f71cc09fb80 auth_method 0 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 10 monclient(hunting): handle_auth_done global_id 6400 payload 274 2024-09-15T01:35:57.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 1 -- 172.21.15.204:0/3454948081 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71cc0a0a40 msgr2=0x7f71cc0a0e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:57.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 1 --2- 172.21.15.204:0/3454948081 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71cc0a0a40 0x7f71cc0a0e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 1 -- 172.21.15.204:0/3454948081 >> v2:172.21.15.178:3300/0 conn(0x7f71cc09fb80 msgr2=0x7f71cc09ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:35:57.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 1 --2- 172.21.15.204:0/3454948081 >> v2:172.21.15.178:3300/0 conn(0x7f71cc09fb80 0x7f71cc09ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:57.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 1 monclient: found mon.noname-f 2024-09-15T01:35:57.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:57.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:35:57.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cbfff640 1 -- 172.21.15.204:0/3454948081 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc04b720 con 0x7f71cc0a2e30 2024-09-15T01:35:57.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.858+0000 7f71cb7fe640 1 --2- 172.21.15.204:0/3454948081 >> v2:172.21.15.178:3300/0 conn(0x7f71cc09fb80 0x7f71cc09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71cbfff640 1 --2- 172.21.15.204:0/3454948081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71cc0a2e30 0x7f71cc132a60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f71cc06f710 tx=0x7f71bc0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2deca413de5e5c30 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71caffd640 1 -- 172.21.15.204:0/3454948081 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71bc02acc0 con 0x7f71cc0a2e30 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71caffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71caffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71caffd640 10 monclient: dump: 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:57.872 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71caffd640 10 monclient: _finish_auth 0 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71caffd640 1 -- 172.21.15.204:0/3454948081 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc02ae40 con 0x7f71cc0a2e30 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 5 monclient: authenticate success, global_id 6400 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:57.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71caffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71caffd640 1 -- 172.21.15.204:0/3454948081 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71bc031ca0 con 0x7f71cc0a2e30 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71caffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 4 set_mon_vals no callback set 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71caffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71caffd640 10 monclient: dump: 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:57.874 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 10 monclient: shutdown 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 1 -- 172.21.15.204:0/3454948081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71cc0a2e30 msgr2=0x7f71cc132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 1 --2- 172.21.15.204:0/3454948081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71cc0a2e30 0x7f71cc132a60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f71cc06f710 tx=0x7f71bc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 1 -- 172.21.15.204:0/3454948081 shutdown_connections 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 1 --2- 172.21.15.204:0/3454948081 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71cc0a0a40 0x7f71cc0a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 1 --2- 172.21.15.204:0/3454948081 >> v2:172.21.15.178:3300/0 conn(0x7f71cc09fb80 0x7f71cc09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 1 --2- 172.21.15.204:0/3454948081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71cc0a2e30 0x7f71cc132a60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 1 -- 172.21.15.204:0/3454948081 >> 172.21.15.204:0/3454948081 conn(0x7f71cc06e860 msgr2=0x7f71cc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.859+0000 7f71d295d640 1 -- 172.21.15.204:0/3454948081 shutdown_connections 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.860+0000 7f71d295d640 1 -- 172.21.15.204:0/3454948081 wait complete. 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.860+0000 7f71d295d640 10 monclient: build_initial_monmap 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.860+0000 7f71d295d640 10 monclient: monmap: 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:57.876 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.860+0000 7f71d295d640 1 Processor -- start 2024-09-15T01:35:57.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 1 -- start start 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 10 monclient: init 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 10 monclient: _add_conns ranks=[8,7,2,6,5,4,3,0,1] 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71cc09fb80 0x7f71cc131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 10 monclient(hunting): picked mon.noname-e con 0x7f71cc09fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71cc0a0a40 0x7f71cc11e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 10 monclient(hunting): picked mon.noname-i con 0x7f71cc0a0a40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71cc0a2e30 0x7f71cc11e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 10 monclient(hunting): picked mon.noname-b con 0x7f71cc0a2e30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc09d650 con 0x7f71cc0a2e30 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc142980 con 0x7f71cc0a0a40 2024-09-15T01:35:57.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc11fed0 con 0x7f71cc09fb80 2024-09-15T01:35:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d295d640 10 monclient(hunting): authenticate will time out at 973.503500s 2024-09-15T01:35:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cb7fe640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71cc0a0a40 msgr2=0x7f71cc11e3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:35:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71cc0a0a40 0x7f71cc11e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d0efa640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71cc0a2e30 msgr2=0x7f71cc11e910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71d0efa640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71cc0a2e30 0x7f71cc11e910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71cc09fb80 0x7f71cc131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71cc09fb80 0x7f71cc131840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:60408/0 (socket says 172.21.15.204:60408) 2024-09-15T01:35:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 1 -- 172.21.15.204:0/858791842 learned_addr learned my addr 172.21.15.204:0/858791842 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 10 monclient(hunting): get_auth_request con 0x7f71cc09fb80 auth_method 0 2024-09-15T01:35:57.879 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 10 monclient(hunting): handle_auth_done global_id 5777 payload 824 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 1 -- 172.21.15.204:0/858791842 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71cc0a0a40 msgr2=0x7f71cc11e3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 1 --2- 172.21.15.204:0/858791842 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71cc0a0a40 0x7f71cc11e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 1 -- 172.21.15.204:0/858791842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71cc0a2e30 msgr2=0x7f71cc11e910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 1 --2- 172.21.15.204:0/858791842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71cc0a2e30 0x7f71cc11e910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 1 monclient: found mon.noname-e 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:57.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.861+0000 7f71cbfff640 1 -- 172.21.15.204:0/858791842 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc009450 con 0x7f71cc09fb80 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71cbfff640 1 --2- 172.21.15.204:0/858791842 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71cc09fb80 0x7f71cc131840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000c00 tx=0x7f71bc032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71b3fff640 1 -- 172.21.15.204:0/858791842 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71bc006b30 con 0x7f71cc09fb80 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71b3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71b3fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71b3fff640 10 monclient: dump: 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:57.881 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71b3fff640 10 monclient: _finish_auth 0 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71d295d640 5 monclient: authenticate success, global_id 5777 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71b3fff640 1 -- 172.21.15.204:0/858791842 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc02acc0 con 0x7f71cc09fb80 2024-09-15T01:35:57.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71d295d640 10 monclient: _renew_subs 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71d295d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71d295d640 1 -- 172.21.15.204:0/858791842 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc121990 con 0x7f71cc09fb80 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71b3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71b3fff640 1 -- 172.21.15.204:0/858791842 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71bc033e30 con 0x7f71cc09fb80 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71b3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71b3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71b3fff640 10 monclient: dump: 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:57.883 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71d295d640 10 monclient: _renew_subs 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71ca7fc640 4 set_mon_vals no callback set 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71d295d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:57.884 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.862+0000 7f71d295d640 1 -- 172.21.15.204:0/858791842 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc120aa0 con 0x7f71cc09fb80 2024-09-15T01:35:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.863+0000 7f71b3fff640 1 -- 172.21.15.204:0/858791842 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f71bc039040 con 0x7f71cc09fb80 2024-09-15T01:35:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.863+0000 7f71d295d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.863+0000 7f71d295d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.863+0000 7f71d295d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.863+0000 7f71d295d640 1 -- 172.21.15.204:0/858791842 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc121e10 con 0x7f71cc09fb80 2024-09-15T01:35:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.863+0000 7f71b3fff640 1 --2- 172.21.15.204:0/858791842 >> v2:172.21.15.204:6812/28210 conn(0x7f71b80397b0 0x7f71b803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.863+0000 7f71b3fff640 1 -- 172.21.15.204:0/858791842 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 8407+0+0 (secure 0 0 0) 0x7f71bc06bd10 con 0x7f71cc09fb80 2024-09-15T01:35:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.863+0000 7f71cb7fe640 1 --2- 172.21.15.204:0/858791842 >> v2:172.21.15.204:6812/28210 conn(0x7f71b80397b0 0x7f71b803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.863+0000 7f71cb7fe640 10 monclient: get_auth_request con 0x7f71b80397b0 auth_method 0 2024-09-15T01:35:57.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.863+0000 7f71cb7fe640 1 --2- 172.21.15.204:0/858791842 >> v2:172.21.15.204:6812/28210 conn(0x7f71b80397b0 0x7f71b803bc60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f71c0006e70 tx=0x7f71c0008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.868+0000 7f71b3fff640 1 -- 172.21.15.204:0/858791842 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f71bc02fcf0 con 0x7f71cc09fb80 2024-09-15T01:35:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.868+0000 7f71b3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:57.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:57.868+0000 7f71b3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:35:58.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:58.030+0000 7f71d295d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-09-15T01:35:58.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:58.030+0000 7f71d295d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-09-15T01:35:58.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:58.030+0000 7f71d295d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:58.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:58.030+0000 7f71d295d640 1 -- 172.21.15.204:0/858791842 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]} v 0) v1 -- 0x7f71cc070cc0 con 0x7f71cc09fb80 2024-09-15T01:35:58.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:58.771+0000 7f71b3fff640 1 -- 172.21.15.204:0/858791842 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}]=0 v152) v1 ==== 134+0+0 (secure 0 0 0) 0x7f71bc041070 con 0x7f71cc09fb80 2024-09-15T01:35:58.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:58.771+0000 7f71b3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-09-15T01:35:58.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:58.771+0000 7f71b3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:35:58.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:58.874+0000 7f71d295d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-09-15T01:35:58.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:58.874+0000 7f71d295d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-09-15T01:35:58.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:58.874+0000 7f71d295d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:35:58.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:58.874+0000 7f71d295d640 1 -- 172.21.15.204:0/858791842 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]} v 0) v1 -- 0x7f71cc0a1f10 con 0x7f71cc09fb80 2024-09-15T01:35:59.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.772+0000 7f71b3fff640 1 -- 172.21.15.204:0/858791842 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}]=0 v153) v1 ==== 134+0+0 (secure 0 0 0) 0x7f71bc037030 con 0x7f71cc09fb80 2024-09-15T01:35:59.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.772+0000 7f71b3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-09-15T01:35:59.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.772+0000 7f71b3fff640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:35:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.776+0000 7f71d295d640 1 -- 172.21.15.204:0/858791842 >> v2:172.21.15.204:6812/28210 conn(0x7f71b80397b0 msgr2=0x7f71b803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.776+0000 7f71d295d640 1 --2- 172.21.15.204:0/858791842 >> v2:172.21.15.204:6812/28210 conn(0x7f71b80397b0 0x7f71b803bc60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f71c0006e70 tx=0x7f71c0008040 comp rx=0 tx=0).stop 2024-09-15T01:35:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.776+0000 7f71d295d640 10 monclient: shutdown 2024-09-15T01:35:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.776+0000 7f71d295d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.776+0000 7f71d295d640 1 -- 172.21.15.204:0/858791842 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71cc09fb80 msgr2=0x7f71cc131840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.776+0000 7f71d295d640 1 --2- 172.21.15.204:0/858791842 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71cc09fb80 0x7f71cc131840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000c00 tx=0x7f71bc032000 comp rx=0 tx=0).stop 2024-09-15T01:35:59.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.776+0000 7f71d295d640 1 -- 172.21.15.204:0/858791842 shutdown_connections 2024-09-15T01:35:59.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.776+0000 7f71d295d640 1 --2- 172.21.15.204:0/858791842 >> v2:172.21.15.204:6812/28210 conn(0x7f71b80397b0 0x7f71b803bc60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:59.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.776+0000 7f71d295d640 1 --2- 172.21.15.204:0/858791842 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f71cc0a0a40 0x7f71cc11e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:59.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.776+0000 7f71d295d640 1 --2- 172.21.15.204:0/858791842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71cc0a2e30 0x7f71cc11e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:59.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.776+0000 7f71d295d640 1 --2- 172.21.15.204:0/858791842 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71cc09fb80 0x7f71cc131840 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:59.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.776+0000 7f71d295d640 1 -- 172.21.15.204:0/858791842 >> 172.21.15.204:0/858791842 conn(0x7f71cc06e860 msgr2=0x7f71cc09bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:59.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.777+0000 7f71d295d640 1 -- 172.21.15.204:0/858791842 shutdown_connections 2024-09-15T01:35:59.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.777+0000 7f71d295d640 1 -- 172.21.15.204:0/858791842 wait complete. 2024-09-15T01:35:59.791 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush set-device-class fish osd.0 2024-09-15T01:35:59.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.869+0000 7f3db4b14640 10 monclient: get_monmap_and_config 2024-09-15T01:35:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.869+0000 7f3db4b14640 10 monclient: build_initial_monmap 2024-09-15T01:35:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.869+0000 7f3db4b14640 10 monclient: monmap: 2024-09-15T01:35:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:35:59.869628+0000 2024-09-15T01:35:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:35:59.869628+0000 2024-09-15T01:35:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:35:59.871 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:35:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:35:59.872 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:35:59.873 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.869+0000 7f3db4b14640 1 Processor -- start 2024-09-15T01:35:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.869+0000 7f3db4b14640 1 -- start start 2024-09-15T01:35:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.869+0000 7f3db4b14640 10 monclient: init 2024-09-15T01:35:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 10 monclient: _add_conns ranks=[2,0,8,4,1,6,7,9,3,10,5] 2024-09-15T01:35:59.885 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3db00a4680 0x7f3db009e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 10 monclient(hunting): picked mon.noname-b con 0x7f3db00a4680 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3db006eea0 addr v1:172.21.15.178:6789/0 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3db0135bd0 0x7f3db0137fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 10 monclient(hunting): picked mon.noname-e con 0x7f3db0135bd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db004b720 con 0x7f3db006eea0 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db00a4a50 con 0x7f3db00a4680 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3db006dc70 con 0x7f3db0135bd0 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3db4b14640 10 monclient(hunting): authenticate will time out at 975.512468s 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3dae59c640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3db00a4680 msgr2=0x7f3db009e850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:59.886 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3dae59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3db00a4680 0x7f3db009e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3daed9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3db0135bd0 0x7f3db0137fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3da7fff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f3db006eea0 0x7f3db009ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.204:57830/0 (socket says 172.21.15.204:57830) 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3da7fff640 1 -- 172.21.15.204:0/47895566 learned_addr learned my addr 172.21.15.204:0/47895566 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3daed9d640 10 monclient(hunting): get_auth_request con 0x7f3db0135bd0 auth_method 0 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3daed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3daed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3daed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3daed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.870+0000 7f3daed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3daed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3dadd9b640 1 -- 172.21.15.204:0/47895566 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3770593139 0 0) 0x7f3db004b720 con 0x7f3db006eea0 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3dadd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:59.887 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3dadd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3dadd9b640 10 monclient(hunting): my global_id is 5985 2024-09-15T01:35:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3dadd9b640 1 -- 172.21.15.204:0/47895566 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d9c0034e0 con 0x7f3db006eea0 2024-09-15T01:35:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3daed9d640 10 monclient(hunting): handle_auth_done global_id 5786 payload 274 2024-09-15T01:35:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3daed9d640 1 -- 172.21.15.204:0/47895566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3db00a4680 msgr2=0x7f3db009e850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:35:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3daed9d640 1 --2- 172.21.15.204:0/47895566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3db00a4680 0x7f3db009e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3daed9d640 1 -- 172.21.15.204:0/47895566 >> v1:172.21.15.178:6789/0 conn(0x7f3db006eea0 legacy=0x7f3db009ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3daed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:35:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3daed9d640 1 monclient: found mon.noname-e 2024-09-15T01:35:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3daed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3daed9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:35:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3daed9d640 1 -- 172.21.15.204:0/47895566 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db00a0710 con 0x7f3db0135bd0 2024-09-15T01:35:59.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.871+0000 7f3daed9d640 1 --2- 172.21.15.204:0/47895566 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3db0135bd0 0x7f3db0137fb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009e90 tx=0x7f3da8009780 comp rx=0 tx=0).ready entity=mon.8 client_cookie=b6e0aaf6d470992d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3dadd9b640 1 -- 172.21.15.204:0/47895566 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3da80079d0 con 0x7f3db0135bd0 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3dadd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3dadd9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3dadd9b640 10 monclient: dump: 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:59.889 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3dadd9b640 10 monclient: _finish_auth 0 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3dadd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 5 monclient: authenticate success, global_id 5786 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3dadd9b640 1 -- 172.21.15.204:0/47895566 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da802d050 con 0x7f3db0135bd0 2024-09-15T01:35:59.890 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3dadd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 10 monclient: get_monmap_and_config success 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 4 set_mon_vals no callback set 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3dadd9b640 1 -- 172.21.15.204:0/47895566 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3da8007b90 con 0x7f3db0135bd0 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3dadd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3dadd9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3dadd9b640 10 monclient: dump: 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:59.891 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 10 monclient: shutdown 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 1 -- 172.21.15.204:0/47895566 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3db0135bd0 msgr2=0x7f3db0137fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:59.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 1 --2- 172.21.15.204:0/47895566 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3db0135bd0 0x7f3db0137fb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009e90 tx=0x7f3da8009780 comp rx=0 tx=0).stop 2024-09-15T01:35:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 1 -- 172.21.15.204:0/47895566 shutdown_connections 2024-09-15T01:35:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 1 --2- 172.21.15.204:0/47895566 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3db0135bd0 0x7f3db0137fb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 1 --2- 172.21.15.204:0/47895566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3db00a4680 0x7f3db009e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 1 -- 172.21.15.204:0/47895566 >> 172.21.15.204:0/47895566 conn(0x7f3db0067740 msgr2=0x7f3db00a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:35:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 1 -- 172.21.15.204:0/47895566 shutdown_connections 2024-09-15T01:35:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.872+0000 7f3db4b14640 1 -- 172.21.15.204:0/47895566 wait complete. 2024-09-15T01:35:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient: build_initial_monmap 2024-09-15T01:35:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient: monmap: 2024-09-15T01:35:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:35:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:35:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:35:59.893 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 1 Processor -- start 2024-09-15T01:35:59.894 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 1 -- start start 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient: init 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient: _add_conns ranks=[1,4,3,0,6,7,5,2,8] 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db006eea0 0x7f3db009cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient(hunting): picked mon.noname-f con 0x7f3db006eea0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3db00a4680 0x7f3db00969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient(hunting): picked mon.noname-c con 0x7f3db00a4680 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db0135bd0 0x7f3db0096f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient(hunting): picked mon.noname-g con 0x7f3db0135bd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db0098a80 con 0x7f3db006eea0 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:59.895 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db006d140 con 0x7f3db0135bd0 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3db006cce0 con 0x7f3db00a4680 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.873+0000 7f3db4b14640 10 monclient(hunting): authenticate will time out at 975.515999s 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db0135bd0 0x7f3db0096f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3dae59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db006eea0 0x7f3db009cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db0135bd0 0x7f3db0096f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54908/0 (socket says 172.21.15.204:54908) 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 -- 172.21.15.204:0/1720385383 learned_addr learned my addr 172.21.15.204:0/1720385383 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 10 monclient(hunting): get_auth_request con 0x7f3db0135bd0 auth_method 0 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:59.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3dae59c640 10 monclient(hunting): get_auth_request con 0x7f3db006eea0 auth_method 0 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3dae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3dae59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3dae59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da7fff640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3db00a4680 0x7f3db00969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3dae59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3dae59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3dae59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da7fff640 10 monclient(hunting): get_auth_request con 0x7f3db00a4680 auth_method 0 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:59.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 10 monclient(hunting): handle_auth_done global_id 14880 payload 824 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 -- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3db00a4680 msgr2=0x7f3db00969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3db00a4680 0x7f3db00969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 -- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db006eea0 msgr2=0x7f3db009cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db006eea0 0x7f3db009cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 monclient: found mon.noname-g 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 -- 172.21.15.204:0/1720385383 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da8009a20 con 0x7f3db0135bd0 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da7fff640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3db00a4680 0x7f3db00969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3dae59c640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db006eea0 0x7f3db009cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:35:59.898 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db0135bd0 0x7f3db0096f20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009d30 tx=0x7f3da8037000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3da803c420 con 0x7f3db0135bd0 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 10 monclient: dump: 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:59.899 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 10 monclient: _finish_auth 0 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da803c5a0 con 0x7f3db0135bd0 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3db4b14640 5 monclient: authenticate success, global_id 14880 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3da803cb90 con 0x7f3db0135bd0 2024-09-15T01:35:59.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3db4b14640 10 monclient: _renew_subs 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3db4b14640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3db4b14640 1 -- 172.21.15.204:0/1720385383 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db009a6f0 con 0x7f3db0135bd0 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 10 monclient: dump: 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:59.901 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3dad59a640 4 set_mon_vals no callback set 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3db4b14640 10 monclient: _renew_subs 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3db4b14640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3db4b14640 1 -- 172.21.15.204:0/1720385383 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db009d560 con 0x7f3db0135bd0 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 -- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db0135bd0 msgr2=0x7f3db0096f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:35:59.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 -- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db0135bd0 msgr2=0x7f3db0096f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db0135bd0 0x7f3db0096f20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009d30 tx=0x7f3da8037000 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3daed9d640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db0135bd0 0x7f3db0096f20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009d30 tx=0x7f3da8037000 comp rx=0 tx=0).stop 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 10 monclient: ms_handle_reset current mon [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 10 monclient: _reopen_session rank -1 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db0135bd0 msgr2=0x7f3db0096f20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db0135bd0 0x7f3db0096f20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 10 monclient: _add_conns ranks=[2,1,8,0,7,4,3,5,6] 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d90004c80 0x7f3d90007060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.874+0000 7f3da67fc640 10 monclient(hunting): picked mon.b con 0x7f3d90004c80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db006eea0 0x7f3d900097c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 10 monclient(hunting): picked mon.f con 0x7f3db006eea0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3d9000a100 0x7f3d9000c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:59.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 10 monclient(hunting): picked mon.e con 0x7f3d9000a100 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9000d030 con 0x7f3db006eea0 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9000d170 con 0x7f3d90004c80 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9000d2e0 con 0x7f3d9000a100 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 10 monclient(hunting): _renew_subs 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3dae59c640 1 -- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d90004c80 msgr2=0x7f3d90007060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3dae59c640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d90004c80 0x7f3d90007060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db006eea0 0x7f3d900097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 10 monclient(hunting): get_auth_request con 0x7f3db006eea0 auth_method 0 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:35:59.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 10 monclient(hunting): handle_auth_done global_id 14880 payload 293 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 1 -- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3d9000a100 msgr2=0x7f3d9000c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3daed9d640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3d9000a100 0x7f3d9000c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3d9000a100 0x7f3d9000c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 1 -- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d90004c80 msgr2=0x7f3d90007060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d90004c80 0x7f3d90007060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 1 monclient: found mon.f 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:35:59.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 1 -- 172.21.15.204:0/1720385383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) v3 -- 0x7f3d90003f00 con 0x7f3db006eea0 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 10 monclient: _finish_auth 0 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da7fff640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db006eea0 0x7f3d900097c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3d98005840 tx=0x7f3d98009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3d98005dc0 con 0x7f3db006eea0 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 10 monclient: dump: 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:35:59.906 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d980070d0 con 0x7f3db006eea0 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3da67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:35:59.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.875+0000 7f3dacd99640 4 set_mon_vals no callback set 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.876+0000 7f3db4b14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.876+0000 7f3db4b14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.876+0000 7f3db4b14640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.876+0000 7f3db4b14640 1 -- 172.21.15.204:0/1720385383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d88005010 con 0x7f3db006eea0 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.876+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3d98005940 con 0x7f3db006eea0 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.876+0000 7f3da67fc640 1 --2- 172.21.15.204:0/1720385383 >> v2:172.21.15.204:6812/28210 conn(0x7f3d90042be0 0x7f3d90045090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.876+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 <== mon.1 v2:172.21.15.204:3300/0 4 ==== osd_map(153..153 src has 1..153) v4 ==== 8407+0+0 (secure 0 0 0) 0x7f3d980499e0 con 0x7f3db006eea0 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.876+0000 7f3dae59c640 1 --2- 172.21.15.204:0/1720385383 >> v2:172.21.15.204:6812/28210 conn(0x7f3d90042be0 0x7f3d90045090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.876+0000 7f3dae59c640 10 monclient: get_auth_request con 0x7f3d90042be0 auth_method 0 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.877+0000 7f3dae59c640 1 --2- 172.21.15.204:0/1720385383 >> v2:172.21.15.204:6812/28210 conn(0x7f3d90042be0 0x7f3d90045090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3da00038f0 tx=0x7f3da00090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.881+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 <== mon.1 v2:172.21.15.204:3300/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f3d98005bf0 con 0x7f3db006eea0 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.881+0000 7f3da67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:35:59.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:35:59.881+0000 7f3da67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:00.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:00.039+0000 7f3db4b14640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-09-15T01:36:00.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:00.039+0000 7f3db4b14640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-09-15T01:36:00.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:00.039+0000 7f3db4b14640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:00.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:00.039+0000 7f3db4b14640 1 -- 172.21.15.204:0/1720385383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]} v 0) v1 -- 0x7f3d88005330 con 0x7f3db006eea0 2024-09-15T01:36:00.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:00.791+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}]=0 set osd(s) 0 to class 'fish' v154) v1 ==== 137+0+0 (secure 0 0 0) 0x7f3d98010850 con 0x7f3db006eea0 2024-09-15T01:36:00.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:00.791+0000 7f3da67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-09-15T01:36:00.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:00.791+0000 7f3da67fc640 10 monclient: _finish_command 2 = system:0 set osd(s) 0 to class 'fish' 2024-09-15T01:36:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:00.887+0000 7f3db4b14640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-09-15T01:36:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:00.887+0000 7f3db4b14640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-09-15T01:36:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:00.887+0000 7f3db4b14640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:00.889 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:00.887+0000 7f3db4b14640 1 -- 172.21.15.204:0/1720385383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]} v 0) v1 -- 0x7f3d88005a00 con 0x7f3db006eea0 2024-09-15T01:36:01.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.458+0000 7f3da67fc640 1 -- 172.21.15.204:0/1720385383 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}]=0 osd.0 already set to class fish. set-device-class item id 0 name 'osd.0' device_class 'fish': no change. set osd(s) to class 'fish' v155) v1 ==== 241+0+0 (secure 0 0 0) 0x7f3d98010a30 con 0x7f3db006eea0 2024-09-15T01:36:01.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.458+0000 7f3da67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-09-15T01:36:01.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.458+0000 7f3da67fc640 10 monclient: _finish_command 3 = system:0 osd.0 already set to class fish. set-device-class item id 0 name 'osd.0' device_class 'fish': no change. set osd(s) to class 'fish' 2024-09-15T01:36:01.460 INFO:tasks.workunit.client.0.smithi204.stderr:osd.0 already set to class fish. set-device-class item id 0 name 'osd.0' device_class 'fish': no change. set osd(s) to class 'fish' 2024-09-15T01:36:01.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.461+0000 7f3db4b14640 1 -- 172.21.15.204:0/1720385383 >> v2:172.21.15.204:6812/28210 conn(0x7f3d90042be0 msgr2=0x7f3d90045090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:01.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.461+0000 7f3db4b14640 1 --2- 172.21.15.204:0/1720385383 >> v2:172.21.15.204:6812/28210 conn(0x7f3d90042be0 0x7f3d90045090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3da00038f0 tx=0x7f3da00090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.461+0000 7f3db4b14640 10 monclient: shutdown 2024-09-15T01:36:01.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.461+0000 7f3db4b14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:01.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.461+0000 7f3db4b14640 1 -- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db006eea0 msgr2=0x7f3d900097c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:01.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.461+0000 7f3db4b14640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db006eea0 0x7f3d900097c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3d98005840 tx=0x7f3d98009040 comp rx=0 tx=0).stop 2024-09-15T01:36:01.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.461+0000 7f3daed9d640 1 -- 172.21.15.204:0/1720385383 reap_dead start 2024-09-15T01:36:01.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.461+0000 7f3db4b14640 1 -- 172.21.15.204:0/1720385383 shutdown_connections 2024-09-15T01:36:01.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.461+0000 7f3db4b14640 1 --2- 172.21.15.204:0/1720385383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db006eea0 0x7f3d900097c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.461+0000 7f3db4b14640 1 -- 172.21.15.204:0/1720385383 >> 172.21.15.204:0/1720385383 conn(0x7f3db0067740 msgr2=0x7f3db00a1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:01.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.461+0000 7f3db4b14640 1 -- 172.21.15.204:0/1720385383 shutdown_connections 2024-09-15T01:36:01.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.461+0000 7f3db4b14640 1 -- 172.21.15.204:0/1720385383 wait complete. 2024-09-15T01:36:01.475 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T01:36:01.475 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 'osd\.0' 2024-09-15T01:36:01.476 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep fish 2024-09-15T01:36:01.476 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep '10\.' 2024-09-15T01:36:01.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.550+0000 7f36e9909640 10 monclient: get_monmap_and_config 2024-09-15T01:36:01.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.550+0000 7f36e9909640 10 monclient: build_initial_monmap 2024-09-15T01:36:01.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.550+0000 7f36e9909640 10 monclient: monmap: 2024-09-15T01:36:01.551 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:01.551 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:01.550602+0000 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:01.550602+0000 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:01.552 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:01.553 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:01.553 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:01.553 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:01.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.550+0000 7f36e9909640 1 Processor -- start 2024-09-15T01:36:01.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.550+0000 7f36e9909640 1 -- start start 2024-09-15T01:36:01.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.550+0000 7f36e9909640 10 monclient: init 2024-09-15T01:36:01.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:01.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 10 monclient: _add_conns ranks=[6,8,1,9,2,7,10,0,3,5,4] 2024-09-15T01:36:01.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f36e40a0b40 0x7f36e409aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 10 monclient(hunting): picked mon.noname-d con 0x7f36e40a0b40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:01.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f36e4067130 0x7f36e4067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 10 monclient(hunting): picked mon.noname-e con 0x7f36e4067130 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:01.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f36e4067a40 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:01.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e404b720 con 0x7f36e4067a40 2024-09-15T01:36:01.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:01.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40a0f10 con 0x7f36e40a0b40 2024-09-15T01:36:01.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:01.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4068ca0 con 0x7f36e4067130 2024-09-15T01:36:01.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:01.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e9909640 10 monclient(hunting): authenticate will time out at 977.193318s 2024-09-15T01:36:01.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e2ffd640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f36e40a0b40 msgr2=0x7f36e409aa60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:01.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e2ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f36e40a0b40 0x7f36e409aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:01.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e37fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f36e4067a40 0x7f36e4139550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:50902/0 (socket says 172.21.15.204:50902) 2024-09-15T01:36:01.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e37fe640 1 -- 172.21.15.204:0/55991341 learned_addr learned my addr 172.21.15.204:0/55991341 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:36:01.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 1 --2- 172.21.15.204:0/55991341 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f36e4067130 0x7f36e4067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:01.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 10 monclient(hunting): get_auth_request con 0x7f36e4067130 auth_method 0 2024-09-15T01:36:01.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:01.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:01.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:01.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:01.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:01.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:01.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e1ffb640 1 -- 172.21.15.204:0/55991341 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766973868 0 0) 0x7f36e404b720 con 0x7f36e4067a40 2024-09-15T01:36:01.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e1ffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:01.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e1ffb640 10 monclient(hunting): my global_id is 6418 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e1ffb640 1 -- 172.21.15.204:0/55991341 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d00034e0 con 0x7f36e4067a40 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 10 monclient(hunting): handle_auth_done global_id 5795 payload 274 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 1 -- 172.21.15.204:0/55991341 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f36e40a0b40 msgr2=0x7f36e409aa60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 1 --2- 172.21.15.204:0/55991341 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f36e40a0b40 0x7f36e409aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 1 -- 172.21.15.204:0/55991341 >> v1:172.21.15.204:6789/0 conn(0x7f36e4067a40 legacy=0x7f36e4139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 10 monclient: _finish_hunting 0 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 1 monclient: found mon.noname-e 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.551+0000 7f36e27fc640 1 -- 172.21.15.204:0/55991341 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e4096050 con 0x7f36e4067130 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e27fc640 1 --2- 172.21.15.204:0/55991341 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f36e4067130 0x7f36e4067500 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009880 tx=0x7f36cc0090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=16e3531e713efb72 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:01.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e1ffb640 1 -- 172.21.15.204:0/55991341 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36cc02ac80 con 0x7f36e4067130 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e1ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e1ffb640 10 monclient: dump: 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:01.566 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e9909640 5 monclient: authenticate success, global_id 5795 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e9909640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e1ffb640 1 -- 172.21.15.204:0/55991341 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36cc02ae00 con 0x7f36e4067130 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e9909640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e9909640 4 set_mon_vals no callback set 2024-09-15T01:36:01.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e1ffb640 1 -- 172.21.15.204:0/55991341 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36cc031ca0 con 0x7f36e4067130 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e1ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.552+0000 7f36e1ffb640 10 monclient: dump: 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:01.568 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.553+0000 7f36e9909640 10 monclient: shutdown 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.553+0000 7f36e9909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.553+0000 7f36e9909640 1 -- 172.21.15.204:0/55991341 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f36e4067130 msgr2=0x7f36e4067500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.553+0000 7f36e9909640 1 --2- 172.21.15.204:0/55991341 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f36e4067130 0x7f36e4067500 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009880 tx=0x7f36cc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.553+0000 7f36e9909640 1 -- 172.21.15.204:0/55991341 shutdown_connections 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.553+0000 7f36e9909640 1 --2- 172.21.15.204:0/55991341 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f36e4067130 0x7f36e4067500 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.553+0000 7f36e9909640 1 --2- 172.21.15.204:0/55991341 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f36e40a0b40 0x7f36e409aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.553+0000 7f36e9909640 1 -- 172.21.15.204:0/55991341 >> 172.21.15.204:0/55991341 conn(0x7f36e40a2e30 msgr2=0x7f36e40a0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.553+0000 7f36e9909640 1 -- 172.21.15.204:0/55991341 shutdown_connections 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.553+0000 7f36e9909640 1 -- 172.21.15.204:0/55991341 wait complete. 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.553+0000 7f36e9909640 10 monclient: build_initial_monmap 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.553+0000 7f36e9909640 10 monclient: monmap: 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:01.570 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 1 Processor -- start 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 1 -- start start 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 10 monclient: init 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 10 monclient: _add_conns ranks=[0,1,6,5,3,7,2,4,8] 2024-09-15T01:36:01.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36e4067130 0x7f36e4141970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 10 monclient(hunting): picked mon.noname-a con 0x7f36e4067130 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36e4067a40 0x7f36e4094ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 10 monclient(hunting): picked mon.noname-f con 0x7f36e4067a40 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f36e40a0b40 0x7f36e408d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 10 monclient(hunting): picked mon.noname-d con 0x7f36e40a0b40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e402a5b0 con 0x7f36e4067130 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4090fb0 con 0x7f36e4067a40 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4090cd0 con 0x7f36e40a0b40 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e9909640 10 monclient(hunting): authenticate will time out at 977.196820s 2024-09-15T01:36:01.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e37fe640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f36e40a0b40 msgr2=0x7f36e408d750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:01.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e37fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f36e40a0b40 0x7f36e408d750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:01.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e27fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36e4067a40 0x7f36e4094ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:01.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e27fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36e4067a40 0x7f36e4094ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43454/0 (socket says 172.21.15.204:43454) 2024-09-15T01:36:01.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e27fc640 1 -- 172.21.15.204:0/931079736 learned_addr learned my addr 172.21.15.204:0/931079736 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:01.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e27fc640 10 monclient(hunting): get_auth_request con 0x7f36e4067a40 auth_method 0 2024-09-15T01:36:01.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:01.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:01.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:01.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e2ffd640 1 --2- 172.21.15.204:0/931079736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36e4067130 0x7f36e4141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:01.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:01.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.554+0000 7f36e27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:01.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e2ffd640 10 monclient(hunting): get_auth_request con 0x7f36e4067130 auth_method 0 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e27fc640 10 monclient(hunting): handle_auth_done global_id 6427 payload 824 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e27fc640 1 -- 172.21.15.204:0/931079736 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f36e40a0b40 msgr2=0x7f36e408d750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e27fc640 1 --2- 172.21.15.204:0/931079736 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f36e40a0b40 0x7f36e408d750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e27fc640 1 -- 172.21.15.204:0/931079736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36e4067130 msgr2=0x7f36e4141970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e27fc640 1 --2- 172.21.15.204:0/931079736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36e4067130 0x7f36e4141970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e27fc640 10 monclient: _finish_hunting 0 2024-09-15T01:36:01.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e27fc640 1 monclient: found mon.noname-f 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e27fc640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e27fc640 1 -- 172.21.15.204:0/931079736 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc009390 con 0x7f36e4067a40 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e27fc640 1 --2- 172.21.15.204:0/931079736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36e4067a40 0x7f36e4094ac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0098e0 tx=0x7f36cc032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36daffd640 1 -- 172.21.15.204:0/931079736 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36cc03b870 con 0x7f36e4067a40 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36daffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36daffd640 10 monclient: dump: 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:01.575 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36daffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:01.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36daffd640 1 -- 172.21.15.204:0/931079736 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36cc03b9d0 con 0x7f36e4067a40 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e9909640 5 monclient: authenticate success, global_id 6427 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e9909640 10 monclient: _renew_subs 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e9909640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e9909640 1 -- 172.21.15.204:0/931079736 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e4141eb0 con 0x7f36e4067a40 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36daffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36daffd640 1 -- 172.21.15.204:0/931079736 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36cc0313a0 con 0x7f36e4067a40 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36daffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36daffd640 10 monclient: dump: 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.577 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e17fa640 4 set_mon_vals no callback set 2024-09-15T01:36:01.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e9909640 10 monclient: _renew_subs 2024-09-15T01:36:01.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e9909640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:01.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.555+0000 7f36e9909640 1 -- 172.21.15.204:0/931079736 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e41426d0 con 0x7f36e4067a40 2024-09-15T01:36:01.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.556+0000 7f36daffd640 1 -- 172.21.15.204:0/931079736 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f36cc039040 con 0x7f36e4067a40 2024-09-15T01:36:01.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.556+0000 7f36e9909640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:01.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.556+0000 7f36e9909640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:01.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.556+0000 7f36e9909640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:01.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.556+0000 7f36e9909640 1 -- 172.21.15.204:0/931079736 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e4091170 con 0x7f36e4067a40 2024-09-15T01:36:01.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.556+0000 7f36daffd640 1 --2- 172.21.15.204:0/931079736 >> v2:172.21.15.204:6812/28210 conn(0x7f36c40396e0 0x7f36c403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.556+0000 7f36daffd640 1 -- 172.21.15.204:0/931079736 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 9072+0+0 (secure 0 0 0) 0x7f36cc06bf10 con 0x7f36e4067a40 2024-09-15T01:36:01.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.556+0000 7f36e2ffd640 1 --2- 172.21.15.204:0/931079736 >> v2:172.21.15.204:6812/28210 conn(0x7f36c40396e0 0x7f36c403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:01.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.556+0000 7f36e2ffd640 10 monclient: get_auth_request con 0x7f36c40396e0 auth_method 0 2024-09-15T01:36:01.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.557+0000 7f36e2ffd640 1 --2- 172.21.15.204:0/931079736 >> v2:172.21.15.204:6812/28210 conn(0x7f36c40396e0 0x7f36c403bb90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0038f0 tx=0x7f36dc0090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:01.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.561+0000 7f36daffd640 1 -- 172.21.15.204:0/931079736 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f36cc030b20 con 0x7f36e4067a40 2024-09-15T01:36:01.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.561+0000 7f36daffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:01.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.561+0000 7f36daffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:01.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.724+0000 7f36e9909640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:01.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.724+0000 7f36e9909640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:01.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.724+0000 7f36e9909640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:01.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.724+0000 7f36e9909640 1 -- 172.21.15.204:0/931079736 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f36e4070cc0 con 0x7f36e4067a40 2024-09-15T01:36:01.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.725+0000 7f36daffd640 1 -- 172.21.15.204:0/931079736 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v155) v1 ==== 83+0+2175 (secure 0 0 0) 0x7f36cc042890 con 0x7f36e4067a40 2024-09-15T01:36:01.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.725+0000 7f36daffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:01.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.725+0000 7f36daffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:01.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.826+0000 7f36e9909640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:01.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.826+0000 7f36e9909640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:01.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.826+0000 7f36e9909640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:01.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.826+0000 7f36e9909640 1 -- 172.21.15.204:0/931079736 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f36e4068f00 con 0x7f36e4067a40 2024-09-15T01:36:01.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.827+0000 7f36daffd640 1 -- 172.21.15.204:0/931079736 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v155) v1 ==== 83+0+2175 (secure 0 0 0) 0x7f36cc042070 con 0x7f36e4067a40 2024-09-15T01:36:01.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.827+0000 7f36daffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:01.829 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.827+0000 7f36daffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:01.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 1 -- 172.21.15.204:0/931079736 >> v2:172.21.15.204:6812/28210 conn(0x7f36c40396e0 msgr2=0x7f36c403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:01.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 1 --2- 172.21.15.204:0/931079736 >> v2:172.21.15.204:6812/28210 conn(0x7f36c40396e0 0x7f36c403bb90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0038f0 tx=0x7f36dc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 10 monclient: shutdown 2024-09-15T01:36:01.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:01.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 1 -- 172.21.15.204:0/931079736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36e4067a40 msgr2=0x7f36e4094ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:01.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 1 --2- 172.21.15.204:0/931079736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36e4067a40 0x7f36e4094ac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0098e0 tx=0x7f36cc032000 comp rx=0 tx=0).stop 2024-09-15T01:36:01.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 1 -- 172.21.15.204:0/931079736 shutdown_connections 2024-09-15T01:36:01.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 1 --2- 172.21.15.204:0/931079736 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f36e40a0b40 0x7f36e408d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 1 --2- 172.21.15.204:0/931079736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36e4067a40 0x7f36e4094ac0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 1 --2- 172.21.15.204:0/931079736 >> v2:172.21.15.204:6812/28210 conn(0x7f36c40396e0 0x7f36c403bb90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 1 --2- 172.21.15.204:0/931079736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36e4067130 0x7f36e4141970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 1 -- 172.21.15.204:0/931079736 >> 172.21.15.204:0/931079736 conn(0x7f36e40a2e30 msgr2=0x7f36e409cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:01.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 1 -- 172.21.15.204:0/931079736 shutdown_connections 2024-09-15T01:36:01.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.829+0000 7f36e9909640 1 -- 172.21.15.204:0/931079736 wait complete. 2024-09-15T01:36:01.840 INFO:tasks.workunit.client.0.smithi204.stdout: 0 fish 0.08719 10.50000 11.50000 12.50000 osd.0 2024-09-15T01:36:01.840 INFO:tasks.workunit.client.0.smithi204.stdout: 0 fish 0.08719 10.50000 11.50000 12.50000 osd.0 2024-09-15T01:36:01.841 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T01:36:01.841 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 'osd\.0' 2024-09-15T01:36:01.841 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep fish 2024-09-15T01:36:01.841 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep '11\.' 2024-09-15T01:36:01.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.911+0000 7f7ea3c68640 10 monclient: get_monmap_and_config 2024-09-15T01:36:01.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.911+0000 7f7ea3c68640 10 monclient: build_initial_monmap 2024-09-15T01:36:01.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.911+0000 7f7ea3c68640 10 monclient: monmap: 2024-09-15T01:36:01.913 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:01.913 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:01.913 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:01.912072+0000 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:01.912072+0000 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:01.914 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:01.915 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:01.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 1 Processor -- start 2024-09-15T01:36:01.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 1 -- start start 2024-09-15T01:36:01.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 10 monclient: init 2024-09-15T01:36:01.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:01.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 10 monclient: _add_conns ranks=[3,2,6,0,5,4,9,1,8,10,7] 2024-09-15T01:36:01.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a2c60 0x7f7e9c132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 10 monclient(hunting): picked mon.noname-g con 0x7f7e9c0a2c60 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:01.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e9c09fc60 0x7f7e9c0a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 10 monclient(hunting): picked mon.noname-b con 0x7f7e9c09fc60 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:01.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a0ab0 0x7f7e9c0a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 10 monclient(hunting): picked mon.noname-d con 0x7f7e9c0a0ab0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:01.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:01.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0a3030 con 0x7f7e9c09fc60 2024-09-15T01:36:01.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:01.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c09d5a0 con 0x7f7e9c0a2c60 2024-09-15T01:36:01.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:01.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1338d0 con 0x7f7e9c0a0ab0 2024-09-15T01:36:01.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:01.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea3c68640 10 monclient(hunting): authenticate will time out at 977.554824s 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea1203640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e9c09fc60 msgr2=0x7f7e9c0a0050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea2205640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a0ab0 msgr2=0x7f7e9c0a0f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.912+0000 7f7ea1203640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e9c09fc60 0x7f7e9c0a0050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a2c60 0x7f7e9c132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea2205640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a0ab0 0x7f7e9c0a0f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a2c60 0x7f7e9c132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54910/0 (socket says 172.21.15.204:54910) 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 1 -- 172.21.15.204:0/3363638643 learned_addr learned my addr 172.21.15.204:0/3363638643 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 10 monclient(hunting): get_auth_request con 0x7f7e9c0a2c60 auth_method 0 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:01.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 10 monclient(hunting): handle_auth_done global_id 14889 payload 274 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 1 -- 172.21.15.204:0/3363638643 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a0ab0 msgr2=0x7f7e9c0a0f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 1 --2- 172.21.15.204:0/3363638643 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a0ab0 0x7f7e9c0a0f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 1 -- 172.21.15.204:0/3363638643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e9c09fc60 msgr2=0x7f7e9c0a0050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 1 --2- 172.21.15.204:0/3363638643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e9c09fc60 0x7f7e9c0a0050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 10 monclient: _finish_hunting 0 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 1 monclient: found mon.noname-g 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.913+0000 7f7ea1a04640 1 -- 172.21.15.204:0/3363638643 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c04b720 con 0x7f7e9c0a2c60 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea1a04640 1 --2- 172.21.15.204:0/3363638643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a2c60 0x7f7e9c132a60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009940 tx=0x7f7e980091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=69e3659678b915dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea0a02640 1 -- 172.21.15.204:0/3363638643 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e9802acb0 con 0x7f7e9c0a2c60 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea0a02640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:01.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea0a02640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea0a02640 10 monclient: dump: 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:01.927 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea0a02640 10 monclient: _finish_auth 0 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea0a02640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 5 monclient: authenticate success, global_id 14889 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea0a02640 1 -- 172.21.15.204:0/3363638643 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9802ae30 con 0x7f7e9c0a2c60 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea0a02640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 4 set_mon_vals no callback set 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea0a02640 1 -- 172.21.15.204:0/3363638643 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e98031ca0 con 0x7f7e9c0a2c60 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea0a02640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:01.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea0a02640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea0a02640 10 monclient: dump: 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:01.929 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 10 monclient: shutdown 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 1 -- 172.21.15.204:0/3363638643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a2c60 msgr2=0x7f7e9c132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 1 --2- 172.21.15.204:0/3363638643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a2c60 0x7f7e9c132a60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009940 tx=0x7f7e980091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 1 -- 172.21.15.204:0/3363638643 shutdown_connections 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 1 --2- 172.21.15.204:0/3363638643 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a0ab0 0x7f7e9c0a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 1 --2- 172.21.15.204:0/3363638643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e9c09fc60 0x7f7e9c0a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 1 --2- 172.21.15.204:0/3363638643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c0a2c60 0x7f7e9c132a60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 1 -- 172.21.15.204:0/3363638643 >> 172.21.15.204:0/3363638643 conn(0x7f7e9c0a60c0 msgr2=0x7f7e9c0a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 1 -- 172.21.15.204:0/3363638643 shutdown_connections 2024-09-15T01:36:01.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.914+0000 7f7ea3c68640 1 -- 172.21.15.204:0/3363638643 wait complete. 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.915+0000 7f7ea3c68640 10 monclient: build_initial_monmap 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.915+0000 7f7ea3c68640 10 monclient: monmap: 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:01.931 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.915+0000 7f7ea3c68640 1 Processor -- start 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.915+0000 7f7ea3c68640 1 -- start start 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.915+0000 7f7ea3c68640 10 monclient: init 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 10 monclient: _add_conns ranks=[3,6,7,5,8,4,2,0,1] 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c09fc60 0x7f7e9c1319c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 10 monclient(hunting): picked mon.noname-g con 0x7f7e9c09fc60 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a0ab0 0x7f7e9c11e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 10 monclient(hunting): picked mon.noname-d con 0x7f7e9c0a0ab0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:01.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7e9c0a2c60 0x7f7e9c11ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 10 monclient(hunting): picked mon.noname-i con 0x7f7e9c0a2c60 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c09ca40 con 0x7f7e9c09fc60 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c09d7d0 con 0x7f7e9c0a0ab0 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1207b0 con 0x7f7e9c0a2c60 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 10 monclient(hunting): authenticate will time out at 977.558216s 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea2205640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7e9c0a2c60 msgr2=0x7f7e9c11ebb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c09fc60 0x7f7e9c1319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea2205640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7e9c0a2c60 0x7f7e9c11ebb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:01.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c09fc60 0x7f7e9c1319c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54912/0 (socket says 172.21.15.204:54912) 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 1 -- 172.21.15.204:0/1126173667 learned_addr learned my addr 172.21.15.204:0/1126173667 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 10 monclient(hunting): get_auth_request con 0x7f7e9c09fc60 auth_method 0 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1203640 1 -- 172.21.15.204:0/1126173667 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a0ab0 msgr2=0x7f7e9c11e670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1203640 1 --2- 172.21.15.204:0/1126173667 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a0ab0 0x7f7e9c11e670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 10 monclient(hunting): handle_auth_done global_id 14898 payload 824 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 1 -- 172.21.15.204:0/1126173667 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7e9c0a2c60 msgr2=0x7f7e9c11ebb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:01.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 1 --2- 172.21.15.204:0/1126173667 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7e9c0a2c60 0x7f7e9c11ebb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 1 -- 172.21.15.204:0/1126173667 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a0ab0 msgr2=0x7f7e9c11e670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 1 --2- 172.21.15.204:0/1126173667 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a0ab0 0x7f7e9c11e670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 10 monclient: _finish_hunting 0 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 1 monclient: found mon.noname-g 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 1 -- 172.21.15.204:0/1126173667 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98009450 con 0x7f7e9c09fc60 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea1a04640 1 --2- 172.21.15.204:0/1126173667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c09fc60 0x7f7e9c1319c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e980097e0 tx=0x7f7e98032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e91ffb640 1 -- 172.21.15.204:0/1126173667 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e9802acb0 con 0x7f7e9c09fc60 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e91ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e91ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e91ffb640 10 monclient: dump: 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:01.935 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:01.936 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e91ffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e91ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 5 monclient: authenticate success, global_id 14898 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e91ffb640 1 -- 172.21.15.204:0/1126173667 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e98032e20 con 0x7f7e9c09fc60 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 10 monclient: _renew_subs 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7ea3c68640 1 -- 172.21.15.204:0/1126173667 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c121b10 con 0x7f7e9c09fc60 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e91ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e91ffb640 1 -- 172.21.15.204:0/1126173667 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e9803a810 con 0x7f7e9c09fc60 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e91ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e91ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:01.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e91ffb640 10 monclient: dump: 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:01.938 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:01.939 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:01.939 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:01.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.917+0000 7f7ea3c68640 10 monclient: _renew_subs 2024-09-15T01:36:01.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.916+0000 7f7e93fff640 4 set_mon_vals no callback set 2024-09-15T01:36:01.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.917+0000 7f7ea3c68640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:01.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.917+0000 7f7ea3c68640 1 -- 172.21.15.204:0/1126173667 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c133fb0 con 0x7f7e9c09fc60 2024-09-15T01:36:01.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.917+0000 7f7e91ffb640 1 -- 172.21.15.204:0/1126173667 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7e98039040 con 0x7f7e9c09fc60 2024-09-15T01:36:01.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.917+0000 7f7ea3c68640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:01.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.917+0000 7f7ea3c68640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:01.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.917+0000 7f7ea3c68640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:01.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.917+0000 7f7ea3c68640 1 -- 172.21.15.204:0/1126173667 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c142a00 con 0x7f7e9c09fc60 2024-09-15T01:36:01.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.918+0000 7f7e91ffb640 1 --2- 172.21.15.204:0/1126173667 >> v2:172.21.15.204:6812/28210 conn(0x7f7e800396e0 0x7f7e8003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:01.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.918+0000 7f7e91ffb640 1 -- 172.21.15.204:0/1126173667 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 9072+0+0 (secure 0 0 0) 0x7f7e9806bf10 con 0x7f7e9c09fc60 2024-09-15T01:36:01.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.918+0000 7f7ea1203640 1 --2- 172.21.15.204:0/1126173667 >> v2:172.21.15.204:6812/28210 conn(0x7f7e800396e0 0x7f7e8003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:01.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.918+0000 7f7ea1203640 10 monclient: get_auth_request con 0x7f7e800396e0 auth_method 0 2024-09-15T01:36:01.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.918+0000 7f7ea1203640 1 --2- 172.21.15.204:0/1126173667 >> v2:172.21.15.204:6812/28210 conn(0x7f7e800396e0 0x7f7e8003bb90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0a2c30 tx=0x7f7e8c006d00 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:01.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.922+0000 7f7e91ffb640 1 -- 172.21.15.204:0/1126173667 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7e9802fa80 con 0x7f7e9c09fc60 2024-09-15T01:36:01.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.922+0000 7f7e91ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:01.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:01.922+0000 7f7e91ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:02.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.086+0000 7f7ea3c68640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:02.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.086+0000 7f7ea3c68640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:02.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.086+0000 7f7ea3c68640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:02.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.086+0000 7f7ea3c68640 1 -- 172.21.15.204:0/1126173667 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f7e9c071100 con 0x7f7e9c09fc60 2024-09-15T01:36:02.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.086+0000 7f7e91ffb640 1 -- 172.21.15.204:0/1126173667 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v155) v1 ==== 83+0+2175 (secure 0 0 0) 0x7f7e98037020 con 0x7f7e9c09fc60 2024-09-15T01:36:02.088 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.086+0000 7f7e91ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:02.089 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.086+0000 7f7e91ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:02.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.186+0000 7f7ea3c68640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:02.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.186+0000 7f7ea3c68640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:02.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.186+0000 7f7ea3c68640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:02.189 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.186+0000 7f7ea3c68640 1 -- 172.21.15.204:0/1126173667 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f7e9c0a04b0 con 0x7f7e9c09fc60 2024-09-15T01:36:02.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.187+0000 7f7e91ffb640 1 -- 172.21.15.204:0/1126173667 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v155) v1 ==== 83+0+2175 (secure 0 0 0) 0x7f7e98041070 con 0x7f7e9c09fc60 2024-09-15T01:36:02.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.187+0000 7f7e91ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:02.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.187+0000 7f7e91ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:02.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 1 -- 172.21.15.204:0/1126173667 >> v2:172.21.15.204:6812/28210 conn(0x7f7e800396e0 msgr2=0x7f7e8003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:02.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 1 --2- 172.21.15.204:0/1126173667 >> v2:172.21.15.204:6812/28210 conn(0x7f7e800396e0 0x7f7e8003bb90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0a2c30 tx=0x7f7e8c006d00 comp rx=0 tx=0).stop 2024-09-15T01:36:02.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 10 monclient: shutdown 2024-09-15T01:36:02.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:02.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 1 -- 172.21.15.204:0/1126173667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c09fc60 msgr2=0x7f7e9c1319c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:02.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 1 --2- 172.21.15.204:0/1126173667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c09fc60 0x7f7e9c1319c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e980097e0 tx=0x7f7e98032000 comp rx=0 tx=0).stop 2024-09-15T01:36:02.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 1 -- 172.21.15.204:0/1126173667 shutdown_connections 2024-09-15T01:36:02.192 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 1 --2- 172.21.15.204:0/1126173667 >> v2:172.21.15.204:6812/28210 conn(0x7f7e800396e0 0x7f7e8003bb90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 1 --2- 172.21.15.204:0/1126173667 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7e9c0a2c60 0x7f7e9c11ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 1 --2- 172.21.15.204:0/1126173667 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7e9c0a0ab0 0x7f7e9c11e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 1 --2- 172.21.15.204:0/1126173667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e9c09fc60 0x7f7e9c1319c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 1 -- 172.21.15.204:0/1126173667 >> 172.21.15.204:0/1126173667 conn(0x7f7e9c0a60c0 msgr2=0x7f7e9c09bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:02.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 1 -- 172.21.15.204:0/1126173667 shutdown_connections 2024-09-15T01:36:02.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.189+0000 7f7ea3c68640 1 -- 172.21.15.204:0/1126173667 wait complete. 2024-09-15T01:36:02.199 INFO:tasks.workunit.client.0.smithi204.stdout: 0 fish 0.08719 10.50000 11.50000 12.50000 osd.0 2024-09-15T01:36:02.200 INFO:tasks.workunit.client.0.smithi204.stdout: 0 fish 0.08719 10.50000 11.50000 12.50000 osd.0 2024-09-15T01:36:02.200 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T01:36:02.200 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 'osd\.0' 2024-09-15T01:36:02.200 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep fish 2024-09-15T01:36:02.201 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep '12\.' 2024-09-15T01:36:02.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.271+0000 7fe66b06f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:02.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.271+0000 7fe66b06f640 10 monclient: build_initial_monmap 2024-09-15T01:36:02.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.272+0000 7fe66b06f640 10 monclient: monmap: 2024-09-15T01:36:02.273 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:02.273 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:02.272499+0000 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:02.272499+0000 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:02.274 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:02.275 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:02.275 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:02.275 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:02.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.272+0000 7fe66b06f640 1 Processor -- start 2024-09-15T01:36:02.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.272+0000 7fe66b06f640 1 -- start start 2024-09-15T01:36:02.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.272+0000 7fe66b06f640 10 monclient: init 2024-09-15T01:36:02.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:02.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 10 monclient: _add_conns ranks=[0,8,6,9,1,2,3,10,4,5,7] 2024-09-15T01:36:02.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe6640a2e30 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:02.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe66409fb80 0x7fe66409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 10 monclient(hunting): picked mon.noname-e con 0x7fe66409fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:02.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe664137d20 0x7fe66413a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 10 monclient(hunting): picked mon.noname-d con 0x7fe664137d20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:02.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66404b720 con 0x7fe6640a2e30 2024-09-15T01:36:02.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:02.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe66409e6c0 con 0x7fe664137d20 2024-09-15T01:36:02.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:02.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640a04b0 con 0x7fe66409fb80 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66b06f640 10 monclient(hunting): authenticate will time out at 977.915432s 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66960c640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe664137d20 msgr2=0x7fe66413a100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe66960c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe664137d20 0x7fe66413a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe663fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe66409fb80 0x7fe66409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe668e0b640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe6640a2e30 0x7fe664132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.204:57832/0 (socket says 172.21.15.204:57832) 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe668e0b640 1 -- 172.21.15.204:0/1736094801 learned_addr learned my addr 172.21.15.204:0/1736094801 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe663fff640 10 monclient(hunting): get_auth_request con 0x7fe66409fb80 auth_method 0 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe663fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe663fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe663fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe663fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:02.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe663fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.273+0000 7fe663fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe6637fe640 1 -- 172.21.15.204:0/1736094801 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065613784 0 0) 0x7fe66404b720 con 0x7fe6640a2e30 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe6637fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe6637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe6637fe640 10 monclient(hunting): my global_id is 6003 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe6637fe640 1 -- 172.21.15.204:0/1736094801 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6500034e0 con 0x7fe6640a2e30 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe663fff640 10 monclient(hunting): handle_auth_done global_id 5804 payload 274 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe663fff640 1 -- 172.21.15.204:0/1736094801 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe664137d20 msgr2=0x7fe66413a100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe663fff640 1 --2- 172.21.15.204:0/1736094801 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe664137d20 0x7fe66413a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe663fff640 1 -- 172.21.15.204:0/1736094801 >> v1:172.21.15.178:6789/0 conn(0x7fe6640a2e30 legacy=0x7fe664132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe663fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe663fff640 1 monclient: found mon.noname-e 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe663fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:02.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe663fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe663fff640 1 -- 172.21.15.204:0/1736094801 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe664133170 con 0x7fe66409fb80 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.274+0000 7fe663fff640 1 --2- 172.21.15.204:0/1736094801 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe66409fb80 0x7fe66409ff70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe654009880 tx=0x7fe6540090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c742236e71432652 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe6637fe640 1 -- 172.21.15.204:0/1736094801 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe65402ac80 con 0x7fe66409fb80 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe6637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe6637fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe6637fe640 10 monclient: dump: 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:02.286 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe6637fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe6637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 5 monclient: authenticate success, global_id 5804 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe6637fe640 1 -- 172.21.15.204:0/1736094801 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65402ae00 con 0x7fe66409fb80 2024-09-15T01:36:02.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe6637fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 4 set_mon_vals no callback set 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe6637fe640 1 -- 172.21.15.204:0/1736094801 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe654031ca0 con 0x7fe66409fb80 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe6637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe6637fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe6637fe640 10 monclient: dump: 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:02.288 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 10 monclient: shutdown 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 1 -- 172.21.15.204:0/1736094801 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe66409fb80 msgr2=0x7fe66409ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 1 --2- 172.21.15.204:0/1736094801 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe66409fb80 0x7fe66409ff70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe654009880 tx=0x7fe6540090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 1 -- 172.21.15.204:0/1736094801 shutdown_connections 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 1 --2- 172.21.15.204:0/1736094801 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe664137d20 0x7fe66413a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 1 --2- 172.21.15.204:0/1736094801 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe66409fb80 0x7fe66409ff70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 1 -- 172.21.15.204:0/1736094801 >> 172.21.15.204:0/1736094801 conn(0x7fe66406e860 msgr2=0x7fe66406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 1 -- 172.21.15.204:0/1736094801 shutdown_connections 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.275+0000 7fe66b06f640 1 -- 172.21.15.204:0/1736094801 wait complete. 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.276+0000 7fe66b06f640 10 monclient: build_initial_monmap 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.276+0000 7fe66b06f640 10 monclient: monmap: 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:02.290 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.276+0000 7fe66b06f640 1 Processor -- start 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.276+0000 7fe66b06f640 1 -- start start 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.276+0000 7fe66b06f640 10 monclient: init 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.276+0000 7fe66b06f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:02.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 10 monclient: _add_conns ranks=[7,6,0,8,5,3,1,2,4] 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe66409fb80 0x7fe6640994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 10 monclient(hunting): picked mon.noname-i con 0x7fe66409fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6640a2e30 0x7fe6640999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 10 monclient(hunting): picked mon.noname-d con 0x7fe6640a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe664137d20 0x7fe664099f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 10 monclient(hunting): picked mon.noname-a con 0x7fe664137d20 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe664131690 con 0x7fe664137d20 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe66409d650 con 0x7fe6640a2e30 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe66411ca30 con 0x7fe66409fb80 2024-09-15T01:36:02.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66b06f640 10 monclient(hunting): authenticate will time out at 977.919074s 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe668e0b640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe66409fb80 msgr2=0x7fe6640994a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe668e0b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe66409fb80 0x7fe6640994a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe663fff640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6640a2e30 msgr2=0x7fe6640999e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe663fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6640a2e30 0x7fe6640999e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe664137d20 0x7fe664099f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe664137d20 0x7fe664099f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:35000/0 (socket says 172.21.15.204:35000) 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 1 -- 172.21.15.204:0/1964247185 learned_addr learned my addr 172.21.15.204:0/1964247185 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 10 monclient(hunting): get_auth_request con 0x7fe664137d20 auth_method 0 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:02.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:02.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:02.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:02.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 10 monclient(hunting): handle_auth_done global_id 6012 payload 824 2024-09-15T01:36:02.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 1 -- 172.21.15.204:0/1964247185 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe66409fb80 msgr2=0x7fe6640994a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:02.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 1 --2- 172.21.15.204:0/1964247185 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe66409fb80 0x7fe6640994a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 1 -- 172.21.15.204:0/1964247185 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6640a2e30 msgr2=0x7fe6640999e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:02.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 1 --2- 172.21.15.204:0/1964247185 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6640a2e30 0x7fe6640999e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:02.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 1 monclient: found mon.noname-a 2024-09-15T01:36:02.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:02.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 1 -- 172.21.15.204:0/1964247185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe654009390 con 0x7fe664137d20 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.277+0000 7fe66960c640 1 --2- 172.21.15.204:0/1964247185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe664137d20 0x7fe664099f20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00bf80 tx=0x7fe65c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe660ff9640 1 -- 172.21.15.204:0/1964247185 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe65c00ce10 con 0x7fe664137d20 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe660ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe660ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe660ff9640 10 monclient: dump: 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:02.295 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe660ff9640 10 monclient: _finish_auth 0 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe660ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe660ff9640 1 -- 172.21.15.204:0/1964247185 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c006150 con 0x7fe664137d20 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe660ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:02.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe660ff9640 1 -- 172.21.15.204:0/1964247185 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe65c006780 con 0x7fe664137d20 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe66b06f640 5 monclient: authenticate success, global_id 6012 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe660ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe660ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe660ff9640 10 monclient: dump: 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:02.297 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe662ffd640 4 set_mon_vals no callback set 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe66b06f640 10 monclient: _renew_subs 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe66b06f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe66b06f640 1 -- 172.21.15.204:0/1964247185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66409acc0 con 0x7fe664137d20 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe66b06f640 10 monclient: _renew_subs 2024-09-15T01:36:02.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe66b06f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:02.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.278+0000 7fe66b06f640 1 -- 172.21.15.204:0/1964247185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66409c7f0 con 0x7fe664137d20 2024-09-15T01:36:02.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.279+0000 7fe660ff9640 1 -- 172.21.15.204:0/1964247185 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe65c012020 con 0x7fe664137d20 2024-09-15T01:36:02.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.279+0000 7fe66b06f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:02.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.279+0000 7fe66b06f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:02.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.279+0000 7fe66b06f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:02.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.279+0000 7fe66b06f640 1 -- 172.21.15.204:0/1964247185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe664070cc0 con 0x7fe664137d20 2024-09-15T01:36:02.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.279+0000 7fe660ff9640 1 --2- 172.21.15.204:0/1964247185 >> v2:172.21.15.204:6812/28210 conn(0x7fe6440397b0 0x7fe64403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.279+0000 7fe660ff9640 1 -- 172.21.15.204:0/1964247185 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 9072+0+0 (secure 0 0 0) 0x7fe65c04c4e0 con 0x7fe664137d20 2024-09-15T01:36:02.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.279+0000 7fe668e0b640 1 --2- 172.21.15.204:0/1964247185 >> v2:172.21.15.204:6812/28210 conn(0x7fe6440397b0 0x7fe64403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:02.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.279+0000 7fe668e0b640 10 monclient: get_auth_request con 0x7fe6440397b0 auth_method 0 2024-09-15T01:36:02.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.279+0000 7fe668e0b640 1 --2- 172.21.15.204:0/1964247185 >> v2:172.21.15.204:6812/28210 conn(0x7fe6440397b0 0x7fe64403bc60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe64c000c00 tx=0x7fe64c007a10 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:02.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.284+0000 7fe660ff9640 1 -- 172.21.15.204:0/1964247185 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe65c014940 con 0x7fe664137d20 2024-09-15T01:36:02.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.284+0000 7fe660ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:02.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.284+0000 7fe660ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:02.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.445+0000 7fe66b06f640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:02.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.445+0000 7fe66b06f640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:02.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.445+0000 7fe66b06f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:02.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.445+0000 7fe66b06f640 1 -- 172.21.15.204:0/1964247185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7fe664067590 con 0x7fe664137d20 2024-09-15T01:36:02.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.446+0000 7fe660ff9640 1 -- 172.21.15.204:0/1964247185 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v155) v1 ==== 83+0+2175 (secure 0 0 0) 0x7fe65c0132e0 con 0x7fe664137d20 2024-09-15T01:36:02.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.446+0000 7fe660ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:02.447 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.446+0000 7fe660ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:02.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.544+0000 7fe66b06f640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:02.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.544+0000 7fe66b06f640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:02.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.544+0000 7fe66b06f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:02.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.544+0000 7fe66b06f640 1 -- 172.21.15.204:0/1964247185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7fe66409b890 con 0x7fe664137d20 2024-09-15T01:36:02.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.545+0000 7fe660ff9640 1 -- 172.21.15.204:0/1964247185 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v155) v1 ==== 83+0+2175 (secure 0 0 0) 0x7fe65c014340 con 0x7fe664137d20 2024-09-15T01:36:02.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.545+0000 7fe660ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:02.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.545+0000 7fe660ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:02.548 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 1 -- 172.21.15.204:0/1964247185 >> v2:172.21.15.204:6812/28210 conn(0x7fe6440397b0 msgr2=0x7fe64403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:02.549 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 1 --2- 172.21.15.204:0/1964247185 >> v2:172.21.15.204:6812/28210 conn(0x7fe6440397b0 0x7fe64403bc60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe64c000c00 tx=0x7fe64c007a10 comp rx=0 tx=0).stop 2024-09-15T01:36:02.549 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 10 monclient: shutdown 2024-09-15T01:36:02.549 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:02.549 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 1 -- 172.21.15.204:0/1964247185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe664137d20 msgr2=0x7fe664099f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:02.549 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 1 --2- 172.21.15.204:0/1964247185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe664137d20 0x7fe664099f20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00bf80 tx=0x7fe65c009700 comp rx=0 tx=0).stop 2024-09-15T01:36:02.549 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 1 -- 172.21.15.204:0/1964247185 shutdown_connections 2024-09-15T01:36:02.549 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 1 --2- 172.21.15.204:0/1964247185 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6640a2e30 0x7fe6640999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.549 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 1 --2- 172.21.15.204:0/1964247185 >> v2:172.21.15.204:6812/28210 conn(0x7fe6440397b0 0x7fe64403bc60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.549 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 1 --2- 172.21.15.204:0/1964247185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe664137d20 0x7fe664099f20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.549 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 1 --2- 172.21.15.204:0/1964247185 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe66409fb80 0x7fe6640994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.549 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 1 -- 172.21.15.204:0/1964247185 >> 172.21.15.204:0/1964247185 conn(0x7fe66406e860 msgr2=0x7fe6640a54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:02.549 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 1 -- 172.21.15.204:0/1964247185 shutdown_connections 2024-09-15T01:36:02.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.547+0000 7fe66b06f640 1 -- 172.21.15.204:0/1964247185 wait complete. 2024-09-15T01:36:02.557 INFO:tasks.workunit.client.0.smithi204.stdout: 0 fish 0.08719 10.50000 11.50000 12.50000 osd.0 2024-09-15T01:36:02.557 INFO:tasks.workunit.client.0.smithi204.stdout: 0 fish 0.08719 10.50000 11.50000 12.50000 osd.0 2024-09-15T01:36:02.558 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm-device-class osd.0 2024-09-15T01:36:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.629+0000 7fcd8d300640 10 monclient: get_monmap_and_config 2024-09-15T01:36:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.629+0000 7fcd8d300640 10 monclient: build_initial_monmap 2024-09-15T01:36:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.629+0000 7fcd8d300640 10 monclient: monmap: 2024-09-15T01:36:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:02.631 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:02.630377+0000 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:02.630377+0000 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:02.632 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:02.633 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:02.633 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:02.633 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.630+0000 7fcd8d300640 1 Processor -- start 2024-09-15T01:36:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.630+0000 7fcd8d300640 1 -- start start 2024-09-15T01:36:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.630+0000 7fcd8d300640 10 monclient: init 2024-09-15T01:36:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.630+0000 7fcd8d300640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 10 monclient: _add_conns ranks=[7,6,9,1,3,5,2,8,0,4,10] 2024-09-15T01:36:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcd880a2e30 0x7fcd88132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 10 monclient(hunting): picked mon.noname-i con 0x7fcd880a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcd8809fb80 0x7fcd8809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 10 monclient(hunting): picked mon.noname-d con 0x7fcd8809fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcd880a0a40 0x7fcd880a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 10 monclient(hunting): picked mon.noname-a con 0x7fcd880a0a40 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8809e6c0 con 0x7fcd880a0a40 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcd880a04b0 con 0x7fcd8809fb80 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcd880a3200 con 0x7fcd880a2e30 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd8d300640 10 monclient(hunting): authenticate will time out at 978.273079s 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd86ffd640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcd880a2e30 msgr2=0x7fcd88132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd86ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcd880a2e30 0x7fcd88132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:02.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd7e7fc640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcd8809fb80 msgr2=0x7fcd8809ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd7e7fc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcd8809fb80 0x7fcd8809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcd880a0a40 0x7fcd880a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcd880a0a40 0x7fcd880a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:35016/0 (socket says 172.21.15.204:35016) 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 1 -- 172.21.15.204:0/2997854609 learned_addr learned my addr 172.21.15.204:0/2997854609 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 10 monclient(hunting): get_auth_request con 0x7fcd880a0a40 auth_method 0 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 10 monclient(hunting): handle_auth_done global_id 6021 payload 274 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 1 -- 172.21.15.204:0/2997854609 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcd880a2e30 msgr2=0x7fcd88132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:02.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 1 --2- 172.21.15.204:0/2997854609 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcd880a2e30 0x7fcd88132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 1 -- 172.21.15.204:0/2997854609 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcd8809fb80 msgr2=0x7fcd8809ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 1 --2- 172.21.15.204:0/2997854609 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcd8809fb80 0x7fcd8809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 1 monclient: found mon.noname-a 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.631+0000 7fcd877fe640 1 -- 172.21.15.204:0/2997854609 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8804b720 con 0x7fcd880a0a40 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd877fe640 1 --2- 172.21.15.204:0/2997854609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd880a0a40 0x7fcd880a0ee0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009f20 tx=0x7fcd80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4513a41e04aa1820 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd867fc640 1 -- 172.21.15.204:0/2997854609 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd80007880 con 0x7fcd880a0a40 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd867fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd867fc640 10 monclient: dump: 2024-09-15T01:36:02.644 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:02.645 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd867fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd867fc640 1 -- 172.21.15.204:0/2997854609 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8002d050 con 0x7fcd880a0a40 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd8d300640 5 monclient: authenticate success, global_id 6021 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd8d300640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd867fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd8d300640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd8d300640 4 set_mon_vals no callback set 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd867fc640 1 -- 172.21.15.204:0/2997854609 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd80030360 con 0x7fcd880a0a40 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd867fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.632+0000 7fcd867fc640 10 monclient: dump: 2024-09-15T01:36:02.646 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:02.647 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 10 monclient: shutdown 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 1 -- 172.21.15.204:0/2997854609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd880a0a40 msgr2=0x7fcd880a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 1 --2- 172.21.15.204:0/2997854609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd880a0a40 0x7fcd880a0ee0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009f20 tx=0x7fcd80009840 comp rx=0 tx=0).stop 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 1 -- 172.21.15.204:0/2997854609 shutdown_connections 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 1 --2- 172.21.15.204:0/2997854609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd880a0a40 0x7fcd880a0ee0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 1 --2- 172.21.15.204:0/2997854609 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcd8809fb80 0x7fcd8809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 1 --2- 172.21.15.204:0/2997854609 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcd880a2e30 0x7fcd88132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 1 -- 172.21.15.204:0/2997854609 >> 172.21.15.204:0/2997854609 conn(0x7fcd8806e860 msgr2=0x7fcd8806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 1 -- 172.21.15.204:0/2997854609 shutdown_connections 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 1 -- 172.21.15.204:0/2997854609 wait complete. 2024-09-15T01:36:02.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 10 monclient: build_initial_monmap 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.633+0000 7fcd8d300640 10 monclient: monmap: 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:02.649 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 1 Processor -- start 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 1 -- start start 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 10 monclient: init 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 10 monclient: _add_conns ranks=[5,1,7,3,4,8,0,2,6] 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd8809fb80 0x7fcd881317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 10 monclient(hunting): picked mon.noname-h con 0x7fcd8809fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd880a0a40 0x7fcd8811e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 10 monclient(hunting): picked mon.noname-f con 0x7fcd880a0a40 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:02.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcd880a2e30 0x7fcd8811e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 10 monclient(hunting): picked mon.noname-i con 0x7fcd880a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd88121f60 con 0x7fcd880a0a40 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8809ca40 con 0x7fcd8809fb80 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8809d650 con 0x7fcd880a2e30 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd8d300640 10 monclient(hunting): authenticate will time out at 978.276573s 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd877fe640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcd880a2e30 msgr2=0x7fcd8811e840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd877fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcd880a2e30 0x7fcd8811e840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd8809fb80 0x7fcd881317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:02.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd7e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd880a0a40 0x7fcd8811e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd8809fb80 0x7fcd881317c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:34750/0 (socket says 172.21.15.204:34750) 2024-09-15T01:36:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 1 -- 172.21.15.204:0/3613072054 learned_addr learned my addr 172.21.15.204:0/3613072054 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 10 monclient(hunting): get_auth_request con 0x7fcd8809fb80 auth_method 0 2024-09-15T01:36:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd7e7fc640 10 monclient(hunting): get_auth_request con 0x7fcd880a0a40 auth_method 0 2024-09-15T01:36:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd7e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd7e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:02.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd7e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd7e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd7e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 10 monclient(hunting): handle_auth_done global_id 5783 payload 824 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 1 -- 172.21.15.204:0/3613072054 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcd880a2e30 msgr2=0x7fcd8811e840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 1 --2- 172.21.15.204:0/3613072054 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcd880a2e30 0x7fcd8811e840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 1 -- 172.21.15.204:0/3613072054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd880a0a40 msgr2=0x7fcd8811e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 1 --2- 172.21.15.204:0/3613072054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd880a0a40 0x7fcd8811e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:36:02.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 1 monclient: found mon.noname-h 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.634+0000 7fcd86ffd640 1 -- 172.21.15.204:0/3613072054 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd80009ae0 con 0x7fcd8809fb80 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7e7fc640 1 --2- 172.21.15.204:0/3613072054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd880a0a40 0x7fcd8811e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd86ffd640 1 --2- 172.21.15.204:0/3613072054 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd8809fb80 0x7fcd881317c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800b620 tx=0x7fcd78009120 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7f7fe640 1 -- 172.21.15.204:0/3613072054 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd78015070 con 0x7fcd8809fb80 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7f7fe640 10 monclient: dump: 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.654 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:02.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd8d300640 5 monclient: authenticate success, global_id 5783 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7f7fe640 1 -- 172.21.15.204:0/3613072054 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd78006150 con 0x7fcd8809fb80 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7f7fe640 1 -- 172.21.15.204:0/3613072054 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd78006740 con 0x7fcd8809fb80 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd8d300640 10 monclient: _renew_subs 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd8d300640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd8d300640 1 -- 172.21.15.204:0/3613072054 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd88142970 con 0x7fcd8809fb80 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd85ffb640 4 set_mon_vals no callback set 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd7f7fe640 10 monclient: dump: 2024-09-15T01:36:02.656 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:02.657 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd8d300640 10 monclient: _renew_subs 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd8d300640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.635+0000 7fcd8d300640 1 -- 172.21.15.204:0/3613072054 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd88120550 con 0x7fcd8809fb80 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.636+0000 7fcd8d300640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.636+0000 7fcd7f7fe640 1 -- 172.21.15.204:0/3613072054 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fcd780062c0 con 0x7fcd8809fb80 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.636+0000 7fcd8d300640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.636+0000 7fcd8d300640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.636+0000 7fcd8d300640 1 -- 172.21.15.204:0/3613072054 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd880668b0 con 0x7fcd8809fb80 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.636+0000 7fcd7f7fe640 1 --2- 172.21.15.204:0/3613072054 >> v2:172.21.15.204:6812/28210 conn(0x7fcd680397b0 0x7fcd6803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.636+0000 7fcd7f7fe640 1 -- 172.21.15.204:0/3613072054 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 9072+0+0 (secure 0 0 0) 0x7fcd7804bf10 con 0x7fcd8809fb80 2024-09-15T01:36:02.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.636+0000 7fcd7e7fc640 1 --2- 172.21.15.204:0/3613072054 >> v2:172.21.15.204:6812/28210 conn(0x7fcd680397b0 0x7fcd6803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:02.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.636+0000 7fcd7e7fc640 10 monclient: get_auth_request con 0x7fcd680397b0 auth_method 0 2024-09-15T01:36:02.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.637+0000 7fcd7e7fc640 1 --2- 172.21.15.204:0/3613072054 >> v2:172.21.15.204:6812/28210 conn(0x7fcd680397b0 0x7fcd6803bc60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd70006fd0 tx=0x7fcd700090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:02.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.640+0000 7fcd7f7fe640 1 -- 172.21.15.204:0/3613072054 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcd78010440 con 0x7fcd8809fb80 2024-09-15T01:36:02.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.640+0000 7fcd7f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:02.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.640+0000 7fcd7f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.804+0000 7fcd8d300640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:36:02.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.804+0000 7fcd8d300640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:36:02.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.804+0000 7fcd8d300640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:02.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:02.804+0000 7fcd8d300640 1 -- 172.21.15.204:0/3613072054 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) v1 -- 0x7fcd88070cc0 con 0x7fcd8809fb80 2024-09-15T01:36:03.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:03.800+0000 7fcd7f7fe640 1 -- 172.21.15.204:0/3613072054 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 done removing class of osd(s): 0 v156) v1 ==== 123+0+0 (secure 0 0 0) 0x7fcd7800d370 con 0x7fcd8809fb80 2024-09-15T01:36:03.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:03.800+0000 7fcd7f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:36:03.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:03.800+0000 7fcd7f7fe640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0 2024-09-15T01:36:03.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:03.900+0000 7fcd8d300640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:36:03.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:03.900+0000 7fcd8d300640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:36:03.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:03.900+0000 7fcd8d300640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:03.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:03.900+0000 7fcd8d300640 1 -- 172.21.15.204:0/3613072054 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) v1 -- 0x7fcd880a1f10 con 0x7fcd8809fb80 2024-09-15T01:36:04.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.814+0000 7fcd7f7fe640 1 -- 172.21.15.204:0/3613072054 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 osd.0 belongs to no class, done removing class of osd(s): v157) v1 ==== 149+0+0 (secure 0 0 0) 0x7fcd780290a0 con 0x7fcd8809fb80 2024-09-15T01:36:04.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.814+0000 7fcd7f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:36:04.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.814+0000 7fcd7f7fe640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T01:36:04.816 INFO:tasks.workunit.client.0.smithi204.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T01:36:04.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 1 -- 172.21.15.204:0/3613072054 >> v2:172.21.15.204:6812/28210 conn(0x7fcd680397b0 msgr2=0x7fcd6803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:04.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 1 --2- 172.21.15.204:0/3613072054 >> v2:172.21.15.204:6812/28210 conn(0x7fcd680397b0 0x7fcd6803bc60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd70006fd0 tx=0x7fcd700090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 10 monclient: shutdown 2024-09-15T01:36:04.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:04.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 1 -- 172.21.15.204:0/3613072054 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd8809fb80 msgr2=0x7fcd881317c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:04.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 1 --2- 172.21.15.204:0/3613072054 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd8809fb80 0x7fcd881317c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800b620 tx=0x7fcd78009120 comp rx=0 tx=0).stop 2024-09-15T01:36:04.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 1 -- 172.21.15.204:0/3613072054 shutdown_connections 2024-09-15T01:36:04.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 1 --2- 172.21.15.204:0/3613072054 >> v2:172.21.15.204:6812/28210 conn(0x7fcd680397b0 0x7fcd6803bc60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 1 --2- 172.21.15.204:0/3613072054 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcd880a2e30 0x7fcd8811e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 1 --2- 172.21.15.204:0/3613072054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd880a0a40 0x7fcd8811e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 1 --2- 172.21.15.204:0/3613072054 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd8809fb80 0x7fcd881317c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 1 -- 172.21.15.204:0/3613072054 >> 172.21.15.204:0/3613072054 conn(0x7fcd8806e860 msgr2=0x7fcd8809ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:04.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 1 -- 172.21.15.204:0/3613072054 shutdown_connections 2024-09-15T01:36:04.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.816+0000 7fcd8d300640 1 -- 172.21.15.204:0/3613072054 wait complete. 2024-09-15T01:36:04.828 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush set-device-class globster osd.0 2024-09-15T01:36:04.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.903+0000 7f852b86f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:04.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.903+0000 7f852b86f640 10 monclient: build_initial_monmap 2024-09-15T01:36:04.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.903+0000 7f852b86f640 10 monclient: monmap: 2024-09-15T01:36:04.905 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:04.905 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:04.905 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:04.903524+0000 2024-09-15T01:36:04.905 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:04.903524+0000 2024-09-15T01:36:04.905 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:04.905 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:04.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.903+0000 7f852b86f640 1 Processor -- start 2024-09-15T01:36:04.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.903+0000 7f852b86f640 1 -- start start 2024-09-15T01:36:04.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.903+0000 7f852b86f640 10 monclient: init 2024-09-15T01:36:04.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:04.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 10 monclient: _add_conns ranks=[4,2,8,7,0,6,9,3,5,10,1] 2024-09-15T01:36:04.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f85240a2e30 0x7f8524132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:04.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 10 monclient(hunting): picked mon.noname-c con 0x7f85240a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:04.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f852409fb80 0x7f852409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:04.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 10 monclient(hunting): picked mon.noname-b con 0x7f852409fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:04.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f85240a0a40 0x7f85240a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:04.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 10 monclient(hunting): picked mon.noname-e con 0x7f85240a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:04.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:04.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852409e6c0 con 0x7f852409fb80 2024-09-15T01:36:04.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:04.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85240a04b0 con 0x7f85240a2e30 2024-09-15T01:36:04.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:04.916 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f85240a3200 con 0x7f85240a0a40 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852b86f640 10 monclient(hunting): authenticate will time out at 980.546345s 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f8528e0a640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f852409fb80 msgr2=0x7f852409ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f8528e0a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f852409fb80 0x7f852409ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852960b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f85240a2e30 0x7f8524132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f8529e0c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f85240a0a40 0x7f85240a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852960b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f85240a2e30 0x7f8524132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.204:39970/0 (socket says 172.21.15.204:39970) 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f8529e0c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f85240a0a40 0x7f85240a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:60460/0 (socket says 172.21.15.204:60460) 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852960b640 1 -- 172.21.15.204:0/3553899805 learned_addr learned my addr 172.21.15.204:0/3553899805 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852960b640 10 monclient(hunting): get_auth_request con 0x7f85240a2e30 auth_method 0 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852960b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852960b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:04.917 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852960b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f8529e0c640 10 monclient(hunting): get_auth_request con 0x7f85240a0a40 auth_method 0 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f8529e0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f8529e0c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f8529e0c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852960b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852960b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f852960b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f8529e0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f8529e0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.904+0000 7f8529e0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.905+0000 7f852960b640 10 monclient(hunting): handle_auth_done global_id 5710 payload 274 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.905+0000 7f852960b640 1 -- 172.21.15.204:0/3553899805 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f85240a0a40 msgr2=0x7f85240a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.905+0000 7f852960b640 1 --2- 172.21.15.204:0/3553899805 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f85240a0a40 0x7f85240a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.905+0000 7f852960b640 1 -- 172.21.15.204:0/3553899805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f852409fb80 msgr2=0x7f852409ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.905+0000 7f852960b640 1 --2- 172.21.15.204:0/3553899805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f852409fb80 0x7f852409ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.905+0000 7f852960b640 10 monclient: _finish_hunting 0 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.905+0000 7f852960b640 1 monclient: found mon.noname-c 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.905+0000 7f852960b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.905+0000 7f852960b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.905+0000 7f852960b640 1 -- 172.21.15.204:0/3553899805 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852404b720 con 0x7f85240a2e30 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.905+0000 7f8529e0c640 1 --2- 172.21.15.204:0/3553899805 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f85240a0a40 0x7f85240a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.905+0000 7f852960b640 1 --2- 172.21.15.204:0/3553899805 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f85240a2e30 0x7f8524132a60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8520009940 tx=0x7f85200091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=9b80f999f3045f91 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f851bfff640 1 -- 172.21.15.204:0/3553899805 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f852002ac80 con 0x7f85240a2e30 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f851bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f851bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f851bfff640 10 monclient: dump: 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:04.919 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:04.920 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f851bfff640 10 monclient: _finish_auth 0 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f851bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f851bfff640 1 -- 172.21.15.204:0/3553899805 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852002ae00 con 0x7f85240a2e30 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 5 monclient: authenticate success, global_id 5710 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f851bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f851bfff640 1 -- 172.21.15.204:0/3553899805 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8520031ca0 con 0x7f85240a2e30 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 4 set_mon_vals no callback set 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f851bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f851bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:04.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f851bfff640 10 monclient: dump: 2024-09-15T01:36:04.922 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:04.922 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:04.922 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:04.922 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:04.922 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:04.922 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:04.922 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:04.922 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:04.922 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:04.922 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:04.922 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:04.922 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:04.923 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:04.923 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:04.923 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:04.923 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:04.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 10 monclient: shutdown 2024-09-15T01:36:04.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:04.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 1 -- 172.21.15.204:0/3553899805 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f85240a2e30 msgr2=0x7f8524132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:04.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 1 --2- 172.21.15.204:0/3553899805 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f85240a2e30 0x7f8524132a60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8520009940 tx=0x7f85200091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 1 -- 172.21.15.204:0/3553899805 shutdown_connections 2024-09-15T01:36:04.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 1 --2- 172.21.15.204:0/3553899805 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f85240a0a40 0x7f85240a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 1 --2- 172.21.15.204:0/3553899805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f852409fb80 0x7f852409ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 1 --2- 172.21.15.204:0/3553899805 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f85240a2e30 0x7f8524132a60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 1 -- 172.21.15.204:0/3553899805 >> 172.21.15.204:0/3553899805 conn(0x7f852406e860 msgr2=0x7f852406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 1 -- 172.21.15.204:0/3553899805 shutdown_connections 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.906+0000 7f852b86f640 1 -- 172.21.15.204:0/3553899805 wait complete. 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.907+0000 7f852b86f640 10 monclient: build_initial_monmap 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.907+0000 7f852b86f640 10 monclient: monmap: 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:04.924 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:04.925 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:04.925 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:04.925 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:04.925 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:04.925 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:04.925 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:04.925 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:04.925 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:04.925 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.907+0000 7f852b86f640 1 Processor -- start 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.907+0000 7f852b86f640 1 -- start start 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.907+0000 7f852b86f640 10 monclient: init 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient: _add_conns ranks=[2,0,5,8,6,4,7,1,3] 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f852409fb80 0x7f8524130830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient(hunting): picked mon.noname-b con 0x7f852409fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85240a0a40 0x7f852411d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient(hunting): picked mon.noname-a con 0x7f85240a0a40 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f85240a2e30 0x7f852411da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient(hunting): picked mon.noname-h con 0x7f85240a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:04.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852409ca40 con 0x7f85240a0a40 2024-09-15T01:36:04.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:04.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852409d650 con 0x7f852409fb80 2024-09-15T01:36:04.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:04.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8524142930 con 0x7f85240a2e30 2024-09-15T01:36:04.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:04.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient(hunting): authenticate will time out at 980.550147s 2024-09-15T01:36:04.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852960b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f852409fb80 msgr2=0x7f8524130830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:04.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852960b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f852409fb80 0x7f8524130830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:04.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f85240a2e30 0x7f852411da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:04.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f85240a2e30 0x7f852411da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:34754/0 (socket says 172.21.15.204:34754) 2024-09-15T01:36:04.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 1 -- 172.21.15.204:0/3630869948 learned_addr learned my addr 172.21.15.204:0/3630869948 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:04.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 10 monclient(hunting): get_auth_request con 0x7f85240a2e30 auth_method 0 2024-09-15T01:36:04.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:04.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:04.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:04.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8528e0a640 1 --2- 172.21.15.204:0/3630869948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85240a0a40 0x7f852411d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:04.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:04.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:04.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:04.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8528e0a640 10 monclient(hunting): get_auth_request con 0x7f85240a0a40 auth_method 0 2024-09-15T01:36:04.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8528e0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:04.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8528e0a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:04.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8528e0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:04.928 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8528e0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8528e0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8528e0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 10 monclient(hunting): handle_auth_done global_id 5792 payload 824 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 1 -- 172.21.15.204:0/3630869948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f852409fb80 msgr2=0x7f8524130830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 1 --2- 172.21.15.204:0/3630869948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f852409fb80 0x7f8524130830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 1 -- 172.21.15.204:0/3630869948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85240a0a40 msgr2=0x7f852411d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 1 --2- 172.21.15.204:0/3630869948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85240a0a40 0x7f852411d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 1 monclient: found mon.noname-h 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 1 -- 172.21.15.204:0/3630869948 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c009a20 con 0x7f85240a2e30 2024-09-15T01:36:04.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f8529e0c640 1 --2- 172.21.15.204:0/3630869948 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f85240a2e30 0x7f852411da20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f851c009f90 tx=0x7f851c00b040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f85197fa640 1 -- 172.21.15.204:0/3630869948 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f851c00ce70 con 0x7f85240a2e30 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f85197fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f85197fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f85197fa640 10 monclient: dump: 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:04.930 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f85197fa640 10 monclient: _finish_auth 0 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f85197fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 5 monclient: authenticate success, global_id 5792 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f85197fa640 1 -- 172.21.15.204:0/3630869948 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c00f040 con 0x7f85240a2e30 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient: _renew_subs 2024-09-15T01:36:04.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 1 -- 172.21.15.204:0/3630869948 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8520009450 con 0x7f85240a2e30 2024-09-15T01:36:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f85197fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f85197fa640 1 -- 172.21.15.204:0/3630869948 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f851c014d80 con 0x7f85240a2e30 2024-09-15T01:36:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f85197fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f85197fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:04.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f85197fa640 10 monclient: dump: 2024-09-15T01:36:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:04.933 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:04.934 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:04.935 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f851b7fe640 4 set_mon_vals no callback set 2024-09-15T01:36:04.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient: _renew_subs 2024-09-15T01:36:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.908+0000 7f852b86f640 1 -- 172.21.15.204:0/3630869948 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852411f5a0 con 0x7f85240a2e30 2024-09-15T01:36:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.909+0000 7f85197fa640 1 -- 172.21.15.204:0/3630869948 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f851c014480 con 0x7f85240a2e30 2024-09-15T01:36:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.909+0000 7f852b86f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.909+0000 7f852b86f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:04.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.909+0000 7f852b86f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.909+0000 7f852b86f640 1 -- 172.21.15.204:0/3630869948 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8524070cc0 con 0x7f85240a2e30 2024-09-15T01:36:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.909+0000 7f85197fa640 1 --2- 172.21.15.204:0/3630869948 >> v2:172.21.15.204:6812/28210 conn(0x7f85080397b0 0x7f850803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.910+0000 7f85197fa640 1 -- 172.21.15.204:0/3630869948 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 8407+0+0 (secure 0 0 0) 0x7f851c04d240 con 0x7f85240a2e30 2024-09-15T01:36:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.910+0000 7f852960b640 1 --2- 172.21.15.204:0/3630869948 >> v2:172.21.15.204:6812/28210 conn(0x7f85080397b0 0x7f850803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:04.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.910+0000 7f852960b640 10 monclient: get_auth_request con 0x7f85080397b0 auth_method 0 2024-09-15T01:36:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.910+0000 7f852960b640 1 --2- 172.21.15.204:0/3630869948 >> v2:172.21.15.204:6812/28210 conn(0x7f85080397b0 0x7f850803bc60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8520034000 tx=0x7f8520032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.914+0000 7f85197fa640 1 -- 172.21.15.204:0/3630869948 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f851c012b20 con 0x7f85240a2e30 2024-09-15T01:36:04.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.914+0000 7f85197fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:04.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:04.914+0000 7f85197fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:05.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:05.071+0000 7f852b86f640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-09-15T01:36:05.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:05.071+0000 7f852b86f640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-09-15T01:36:05.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:05.071+0000 7f852b86f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:05.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:05.071+0000 7f852b86f640 1 -- 172.21.15.204:0/3630869948 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]} v 0) v1 -- 0x7f85240a1f10 con 0x7f85240a2e30 2024-09-15T01:36:05.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:05.812+0000 7f85197fa640 1 -- 172.21.15.204:0/3630869948 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}]=0 set osd(s) 0 to class 'globster' v158) v1 ==== 145+0+0 (secure 0 0 0) 0x7f851c018030 con 0x7f85240a2e30 2024-09-15T01:36:05.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:05.812+0000 7f85197fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-09-15T01:36:05.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:05.812+0000 7f85197fa640 10 monclient: _finish_command 2 = system:0 set osd(s) 0 to class 'globster' 2024-09-15T01:36:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:05.911+0000 7f852b86f640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-09-15T01:36:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:05.911+0000 7f852b86f640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-09-15T01:36:05.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:05.911+0000 7f852b86f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:05.913 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:05.911+0000 7f852b86f640 1 -- 172.21.15.204:0/3630869948 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]} v 0) v1 -- 0x7f8524067590 con 0x7f85240a2e30 2024-09-15T01:36:06.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.459+0000 7f85197fa640 1 -- 172.21.15.204:0/3630869948 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}]=0 osd.0 already set to class globster. set-device-class item id 0 name 'osd.0' device_class 'globster': no change. set osd(s) to class 'globster' v159) v1 ==== 257+0+0 (secure 0 0 0) 0x7f851c018020 con 0x7f85240a2e30 2024-09-15T01:36:06.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.459+0000 7f85197fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-09-15T01:36:06.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.459+0000 7f85197fa640 10 monclient: _finish_command 3 = system:0 osd.0 already set to class globster. set-device-class item id 0 name 'osd.0' device_class 'globster': no change. set osd(s) to class 'globster' 2024-09-15T01:36:06.461 INFO:tasks.workunit.client.0.smithi204.stderr:osd.0 already set to class globster. set-device-class item id 0 name 'osd.0' device_class 'globster': no change. set osd(s) to class 'globster' 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 1 -- 172.21.15.204:0/3630869948 >> v2:172.21.15.204:6812/28210 conn(0x7f85080397b0 msgr2=0x7f850803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 1 --2- 172.21.15.204:0/3630869948 >> v2:172.21.15.204:6812/28210 conn(0x7f85080397b0 0x7f850803bc60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8520034000 tx=0x7f8520032000 comp rx=0 tx=0).stop 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 10 monclient: shutdown 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 1 -- 172.21.15.204:0/3630869948 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f85240a2e30 msgr2=0x7f852411da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 1 --2- 172.21.15.204:0/3630869948 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f85240a2e30 0x7f852411da20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f851c009f90 tx=0x7f851c00b040 comp rx=0 tx=0).stop 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 1 -- 172.21.15.204:0/3630869948 shutdown_connections 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 1 --2- 172.21.15.204:0/3630869948 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f85240a2e30 0x7f852411da20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 1 --2- 172.21.15.204:0/3630869948 >> v2:172.21.15.204:6812/28210 conn(0x7f85080397b0 0x7f850803bc60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 1 --2- 172.21.15.204:0/3630869948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85240a0a40 0x7f852411d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 1 --2- 172.21.15.204:0/3630869948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f852409fb80 0x7f8524130830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 1 -- 172.21.15.204:0/3630869948 >> 172.21.15.204:0/3630869948 conn(0x7f852406e860 msgr2=0x7f852409ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 1 -- 172.21.15.204:0/3630869948 shutdown_connections 2024-09-15T01:36:06.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.462+0000 7f852b86f640 1 -- 172.21.15.204:0/3630869948 wait complete. 2024-09-15T01:36:06.475 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T01:36:06.475 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 'osd\.0' 2024-09-15T01:36:06.476 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep globster 2024-09-15T01:36:06.476 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep '10\.' 2024-09-15T01:36:06.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.553+0000 7f2210f14640 10 monclient: get_monmap_and_config 2024-09-15T01:36:06.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.553+0000 7f2210f14640 10 monclient: build_initial_monmap 2024-09-15T01:36:06.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.553+0000 7f2210f14640 10 monclient: monmap: 2024-09-15T01:36:06.555 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:06.555 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:06.555 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:06.553885+0000 2024-09-15T01:36:06.555 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:06.553885+0000 2024-09-15T01:36:06.555 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:06.555 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:06.555 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:06.555 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:06.555 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:06.556 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:06.556 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:06.556 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:06.556 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:06.556 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:06.556 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:06.556 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:06.556 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:06.556 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:06.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.553+0000 7f2210f14640 1 Processor -- start 2024-09-15T01:36:06.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 1 -- start start 2024-09-15T01:36:06.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 10 monclient: init 2024-09-15T01:36:06.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 10 monclient: _add_conns ranks=[6,8,3,2,4,10,9,1,5,0,7] 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f220c0a2c60 0x7f220c132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 10 monclient(hunting): picked mon.noname-d con 0x7f220c0a2c60 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f220c09fc60 0x7f220c0a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 10 monclient(hunting): picked mon.noname-e con 0x7f220c09fc60 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f220c0a0ab0 0x7f220c0a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 10 monclient(hunting): picked mon.noname-g con 0x7f220c0a0ab0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c0a3030 con 0x7f220c0a0ab0 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f220c09d5a0 con 0x7f220c0a2c60 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f220c1338d0 con 0x7f220c09fc60 2024-09-15T01:36:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f2210f14640 10 monclient(hunting): authenticate will time out at 982.196737s 2024-09-15T01:36:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f220a59c640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f220c0a2c60 msgr2=0x7f220c132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f220ad9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f220c0a0ab0 0x7f220c0a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f220a59c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f220c0a2c60 0x7f220c132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f220ad9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f220c0a0ab0 0x7f220c0a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:54922/0 (socket says 172.21.15.204:54922) 2024-09-15T01:36:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f220ad9d640 1 -- 172.21.15.204:0/2593495200 learned_addr learned my addr 172.21.15.204:0/2593495200 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f220ad9d640 10 monclient(hunting): get_auth_request con 0x7f220c0a0ab0 auth_method 0 2024-09-15T01:36:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f220ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:06.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f220ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.554+0000 7f220ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f2209d9b640 1 --2- 172.21.15.204:0/2593495200 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f220c09fc60 0x7f220c0a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f2209d9b640 10 monclient(hunting): get_auth_request con 0x7f220c09fc60 auth_method 0 2024-09-15T01:36:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f2209d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f2209d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f2209d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f2209d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f2209d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:06.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f2209d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:06.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 10 monclient(hunting): handle_auth_done global_id 14907 payload 274 2024-09-15T01:36:06.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 1 -- 172.21.15.204:0/2593495200 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f220c09fc60 msgr2=0x7f220c0a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:06.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 1 --2- 172.21.15.204:0/2593495200 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f220c09fc60 0x7f220c0a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 1 -- 172.21.15.204:0/2593495200 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f220c0a2c60 msgr2=0x7f220c132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:06.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 1 --2- 172.21.15.204:0/2593495200 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f220c0a2c60 0x7f220c132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:36:06.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 1 monclient: found mon.noname-g 2024-09-15T01:36:06.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:06.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:06.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 1 -- 172.21.15.204:0/2593495200 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c04b720 con 0x7f220c0a0ab0 2024-09-15T01:36:06.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.555+0000 7f220ad9d640 1 --2- 172.21.15.204:0/2593495200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f220c0a0ab0 0x7f220c0a0f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2204009f50 tx=0x7f2204009840 comp rx=0 tx=0).ready entity=mon.3 client_cookie=dff8191bb2940c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:06.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f220959a640 1 -- 172.21.15.204:0/2593495200 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2204007880 con 0x7f220c0a0ab0 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f220959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f220959a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f220959a640 10 monclient: dump: 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:06.569 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f220959a640 10 monclient: _finish_auth 0 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f220959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 5 monclient: authenticate success, global_id 14907 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f220959a640 1 -- 172.21.15.204:0/2593495200 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220402d050 con 0x7f220c0a0ab0 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f220959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:06.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 4 set_mon_vals no callback set 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f220959a640 1 -- 172.21.15.204:0/2593495200 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2204030360 con 0x7f220c0a0ab0 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f220959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f220959a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f220959a640 10 monclient: dump: 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:06.571 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 10 monclient: shutdown 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 1 -- 172.21.15.204:0/2593495200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f220c0a0ab0 msgr2=0x7f220c0a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 1 --2- 172.21.15.204:0/2593495200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f220c0a0ab0 0x7f220c0a0f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2204009f50 tx=0x7f2204009840 comp rx=0 tx=0).stop 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 1 -- 172.21.15.204:0/2593495200 shutdown_connections 2024-09-15T01:36:06.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 1 --2- 172.21.15.204:0/2593495200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f220c0a0ab0 0x7f220c0a0f00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 1 --2- 172.21.15.204:0/2593495200 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f220c09fc60 0x7f220c0a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 1 --2- 172.21.15.204:0/2593495200 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f220c0a2c60 0x7f220c132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 1 -- 172.21.15.204:0/2593495200 >> 172.21.15.204:0/2593495200 conn(0x7f220c0a60c0 msgr2=0x7f220c0a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 1 -- 172.21.15.204:0/2593495200 shutdown_connections 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.556+0000 7f2210f14640 1 -- 172.21.15.204:0/2593495200 wait complete. 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient: build_initial_monmap 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient: monmap: 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:06.573 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 1 Processor -- start 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 1 -- start start 2024-09-15T01:36:06.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient: init 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient: _add_conns ranks=[6,5,4,0,8,2,3,7,1] 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f220c09fc60 0x7f220c09b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient(hunting): picked mon.noname-d con 0x7f220c09fc60 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f220c0a0ab0 0x7f220c12e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient(hunting): picked mon.noname-h con 0x7f220c0a0ab0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f220c0a2c60 0x7f220c11c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient(hunting): picked mon.noname-c con 0x7f220c0a2c60 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f220c09d7d0 con 0x7f220c0a2c60 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f220c133fc0 con 0x7f220c0a0ab0 2024-09-15T01:36:06.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f220c09ca10 con 0x7f220c09fc60 2024-09-15T01:36:06.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:06.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.557+0000 7f2210f14640 10 monclient(hunting): authenticate will time out at 982.199984s 2024-09-15T01:36:06.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f220a59c640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f220c09fc60 msgr2=0x7f220c09b6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:06.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f220c0a0ab0 0x7f220c12e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:06.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f220a59c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f220c09fc60 0x7f220c09b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:06.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f220c0a0ab0 0x7f220c12e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:34762/0 (socket says 172.21.15.204:34762) 2024-09-15T01:36:06.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 1 -- 172.21.15.204:0/1875166207 learned_addr learned my addr 172.21.15.204:0/1875166207 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:06.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 10 monclient(hunting): get_auth_request con 0x7f220c0a0ab0 auth_method 0 2024-09-15T01:36:06.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:06.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:06.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f220ad9d640 1 --2- 172.21.15.204:0/1875166207 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f220c0a2c60 0x7f220c11c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 10 monclient(hunting): handle_auth_done global_id 5801 payload 824 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 1 -- 172.21.15.204:0/1875166207 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f220c09fc60 msgr2=0x7f220c09b6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 1 --2- 172.21.15.204:0/1875166207 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f220c09fc60 0x7f220c09b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 1 -- 172.21.15.204:0/1875166207 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f220c0a2c60 msgr2=0x7f220c11c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 1 --2- 172.21.15.204:0/1875166207 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f220c0a2c60 0x7f220c11c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 1 monclient: found mon.noname-h 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:06.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 1 -- 172.21.15.204:0/1875166207 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2204009ae0 con 0x7f220c0a0ab0 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2209d9b640 1 --2- 172.21.15.204:0/1875166207 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f220c0a0ab0 0x7f220c12e8a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f220000ea20 tx=0x7f22000090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f21fa7fc640 1 -- 172.21.15.204:0/1875166207 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f220000ce90 con 0x7f220c0a0ab0 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f21fa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f21fa7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f21fa7fc640 10 monclient: dump: 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:06.578 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:06.579 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:06.579 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:06.580 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:06.580 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:06.580 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:06.580 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:06.580 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:06.580 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:06.580 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:06.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f21fa7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:06.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f21fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:06.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2210f14640 5 monclient: authenticate success, global_id 5801 2024-09-15T01:36:06.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f21fa7fc640 1 -- 172.21.15.204:0/1875166207 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200009600 con 0x7f220c0a0ab0 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2210f14640 10 monclient: _renew_subs 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2210f14640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2210f14640 1 -- 172.21.15.204:0/1875166207 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220c11da90 con 0x7f220c0a0ab0 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f21fa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f21fa7fc640 1 -- 172.21.15.204:0/1875166207 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22000138f0 con 0x7f220c0a0ab0 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f21fa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f21fa7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f21fa7fc640 10 monclient: dump: 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.581 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2208d99640 4 set_mon_vals no callback set 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2210f14640 10 monclient: _renew_subs 2024-09-15T01:36:06.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2210f14640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.558+0000 7f2210f14640 1 -- 172.21.15.204:0/1875166207 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220c11e9f0 con 0x7f220c0a0ab0 2024-09-15T01:36:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.559+0000 7f21fa7fc640 1 -- 172.21.15.204:0/1875166207 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2200013470 con 0x7f220c0a0ab0 2024-09-15T01:36:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.559+0000 7f21fa7fc640 1 --2- 172.21.15.204:0/1875166207 >> v2:172.21.15.204:6812/28210 conn(0x7f21ec0397b0 0x7f21ec03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.559+0000 7f2210f14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.559+0000 7f2210f14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.559+0000 7f2210f14640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.559+0000 7f21fa7fc640 1 -- 172.21.15.204:0/1875166207 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 9096+0+0 (secure 0 0 0) 0x7f220004c850 con 0x7f220c0a0ab0 2024-09-15T01:36:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.559+0000 7f2210f14640 1 -- 172.21.15.204:0/1875166207 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220c11f3c0 con 0x7f220c0a0ab0 2024-09-15T01:36:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.559+0000 7f220a59c640 1 --2- 172.21.15.204:0/1875166207 >> v2:172.21.15.204:6812/28210 conn(0x7f21ec0397b0 0x7f21ec03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.559+0000 7f220a59c640 10 monclient: get_auth_request con 0x7f21ec0397b0 auth_method 0 2024-09-15T01:36:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.560+0000 7f220a59c640 1 --2- 172.21.15.204:0/1875166207 >> v2:172.21.15.204:6812/28210 conn(0x7f21ec0397b0 0x7f21ec03bc60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f21fc006fd0 tx=0x7f21fc006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:06.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.564+0000 7f21fa7fc640 1 -- 172.21.15.204:0/1875166207 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2200013720 con 0x7f220c0a0ab0 2024-09-15T01:36:06.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.564+0000 7f21fa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:06.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.564+0000 7f21fa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.725+0000 7f2210f14640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.725+0000 7f2210f14640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.725+0000 7f2210f14640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.725+0000 7f2210f14640 1 -- 172.21.15.204:0/1875166207 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f220c11ebe0 con 0x7f220c0a0ab0 2024-09-15T01:36:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.725+0000 7f21fa7fc640 1 -- 172.21.15.204:0/1875166207 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v159) v1 ==== 83+0+2378 (secure 0 0 0) 0x7f220001bb60 con 0x7f220c0a0ab0 2024-09-15T01:36:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.725+0000 7f21fa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.725+0000 7f21fa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:06.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.825+0000 7f2210f14640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:06.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.825+0000 7f2210f14640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:06.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.825+0000 7f2210f14640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:06.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.825+0000 7f2210f14640 1 -- 172.21.15.204:0/1875166207 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f220c071100 con 0x7f220c0a0ab0 2024-09-15T01:36:06.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.826+0000 7f21fa7fc640 1 -- 172.21.15.204:0/1875166207 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v159) v1 ==== 83+0+2378 (secure 0 0 0) 0x7f22000232f0 con 0x7f220c0a0ab0 2024-09-15T01:36:06.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.826+0000 7f21fa7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:06.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.826+0000 7f21fa7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:06.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 1 -- 172.21.15.204:0/1875166207 >> v2:172.21.15.204:6812/28210 conn(0x7f21ec0397b0 msgr2=0x7f21ec03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:06.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 1 --2- 172.21.15.204:0/1875166207 >> v2:172.21.15.204:6812/28210 conn(0x7f21ec0397b0 0x7f21ec03bc60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f21fc006fd0 tx=0x7f21fc006d30 comp rx=0 tx=0).stop 2024-09-15T01:36:06.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 10 monclient: shutdown 2024-09-15T01:36:06.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:06.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 1 -- 172.21.15.204:0/1875166207 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f220c0a0ab0 msgr2=0x7f220c12e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:06.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 1 --2- 172.21.15.204:0/1875166207 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f220c0a0ab0 0x7f220c12e8a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f220000ea20 tx=0x7f22000090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 1 -- 172.21.15.204:0/1875166207 shutdown_connections 2024-09-15T01:36:06.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 1 --2- 172.21.15.204:0/1875166207 >> v2:172.21.15.204:6812/28210 conn(0x7f21ec0397b0 0x7f21ec03bc60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 1 --2- 172.21.15.204:0/1875166207 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f220c0a2c60 0x7f220c11c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 1 --2- 172.21.15.204:0/1875166207 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f220c0a0ab0 0x7f220c12e8a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 1 --2- 172.21.15.204:0/1875166207 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f220c09fc60 0x7f220c09b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 1 -- 172.21.15.204:0/1875166207 >> 172.21.15.204:0/1875166207 conn(0x7f220c0a60c0 msgr2=0x7f220c0298c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:06.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 1 -- 172.21.15.204:0/1875166207 shutdown_connections 2024-09-15T01:36:06.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.828+0000 7f2210f14640 1 -- 172.21.15.204:0/1875166207 wait complete. 2024-09-15T01:36:06.838 INFO:tasks.workunit.client.0.smithi204.stdout: 0 globster 0.08719 10.50000 11.50000 12.50000 osd.0 2024-09-15T01:36:06.838 INFO:tasks.workunit.client.0.smithi204.stdout: 0 globster 0.08719 10.50000 11.50000 12.50000 osd.0 2024-09-15T01:36:06.839 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T01:36:06.839 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 'osd\.0' 2024-09-15T01:36:06.839 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep globster 2024-09-15T01:36:06.839 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep '11\.' 2024-09-15T01:36:06.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.910+0000 7fe10f35d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:06.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.910+0000 7fe10f35d640 10 monclient: build_initial_monmap 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.910+0000 7fe10f35d640 10 monclient: monmap: 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:06.910689+0000 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:06.910689+0000 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:06.912 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:06.913 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:06.913 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:06.913 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:06.913 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:06.913 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:06.913 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:06.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.910+0000 7fe10f35d640 1 Processor -- start 2024-09-15T01:36:06.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 1 -- start start 2024-09-15T01:36:06.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 10 monclient: init 2024-09-15T01:36:06.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:06.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 10 monclient: _add_conns ranks=[10,6,4,5,3,0,9,7,2,8,1] 2024-09-15T01:36:06.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1080a2e30 0x7fe108132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 10 monclient(hunting): picked mon.noname-f con 0x7fe1080a2e30 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:06.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe10809fb80 0x7fe10809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 10 monclient(hunting): picked mon.noname-d con 0x7fe10809fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:06.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1080a0a40 0x7fe1080a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 10 monclient(hunting): picked mon.noname-c con 0x7fe1080a0a40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:06.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10809e6c0 con 0x7fe1080a2e30 2024-09-15T01:36:06.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080a04b0 con 0x7fe1080a0a40 2024-09-15T01:36:06.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080a3200 con 0x7fe10809fb80 2024-09-15T01:36:06.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:06.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10f35d640 10 monclient(hunting): authenticate will time out at 982.553525s 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10c8f8640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe10809fb80 msgr2=0x7fe10809ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1080a2e30 0x7fe108132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10c8f8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe10809fb80 0x7fe10809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1080a2e30 0x7fe108132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43482/0 (socket says 172.21.15.204:43482) 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 1 -- 172.21.15.204:0/1793858978 learned_addr learned my addr 172.21.15.204:0/1793858978 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 10 monclient(hunting): get_auth_request con 0x7fe1080a2e30 auth_method 0 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d8fa640 1 --2- 172.21.15.204:0/1793858978 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1080a0a40 0x7fe1080a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:06.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d8fa640 10 monclient(hunting): get_auth_request con 0x7fe1080a0a40 auth_method 0 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d8fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d8fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d8fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d8fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d8fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d8fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 10 monclient(hunting): handle_auth_done global_id 6445 payload 274 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 1 -- 172.21.15.204:0/1793858978 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe10809fb80 msgr2=0x7fe10809ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 1 --2- 172.21.15.204:0/1793858978 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe10809fb80 0x7fe10809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 1 -- 172.21.15.204:0/1793858978 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1080a0a40 msgr2=0x7fe1080a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 1 --2- 172.21.15.204:0/1793858978 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1080a0a40 0x7fe1080a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 10 monclient: _finish_hunting 0 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 1 monclient: found mon.noname-f 2024-09-15T01:36:06.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:06.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.911+0000 7fe10d0f9640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:06.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe10d0f9640 1 -- 172.21.15.204:0/1793858978 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10804b720 con 0x7fe1080a2e30 2024-09-15T01:36:06.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe10d0f9640 1 --2- 172.21.15.204:0/1793858978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1080a2e30 0x7fe108132a60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009f20 tx=0x7fe0f8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed9c805044e9ccfd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:06.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe107fff640 1 -- 172.21.15.204:0/1793858978 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0f8007880 con 0x7fe1080a2e30 2024-09-15T01:36:06.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe107fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:06.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe107fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:06.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe107fff640 10 monclient: dump: 2024-09-15T01:36:06.927 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:06.927 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:06.927 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.927 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.927 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:06.928 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:06.928 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:06.928 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:06.928 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:06.928 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:06.928 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:06.928 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:06.928 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:06.928 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:06.928 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:06.928 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe107fff640 10 monclient: _finish_auth 0 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe107fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe107fff640 1 -- 172.21.15.204:0/1793858978 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f802d050 con 0x7fe1080a2e30 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe10f35d640 5 monclient: authenticate success, global_id 6445 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe10f35d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe107fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe10f35d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe107fff640 1 -- 172.21.15.204:0/1793858978 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0f8030360 con 0x7fe1080a2e30 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe107fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe10f35d640 4 set_mon_vals no callback set 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe107fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.912+0000 7fe107fff640 10 monclient: dump: 2024-09-15T01:36:06.929 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:06.930 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:06.930 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.930 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.930 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:06.930 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:06.930 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:06.930 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:06.930 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:06.930 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:06.930 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:06.930 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:06.930 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.913+0000 7fe10f35d640 10 monclient: shutdown 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.913+0000 7fe10f35d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.913+0000 7fe10f35d640 1 -- 172.21.15.204:0/1793858978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1080a2e30 msgr2=0x7fe108132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.913+0000 7fe10f35d640 1 --2- 172.21.15.204:0/1793858978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1080a2e30 0x7fe108132a60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009f20 tx=0x7fe0f8009840 comp rx=0 tx=0).stop 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.913+0000 7fe10f35d640 1 -- 172.21.15.204:0/1793858978 shutdown_connections 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.913+0000 7fe10f35d640 1 --2- 172.21.15.204:0/1793858978 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1080a0a40 0x7fe1080a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.913+0000 7fe10f35d640 1 --2- 172.21.15.204:0/1793858978 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe10809fb80 0x7fe10809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.913+0000 7fe10f35d640 1 --2- 172.21.15.204:0/1793858978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1080a2e30 0x7fe108132a60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.913+0000 7fe10f35d640 1 -- 172.21.15.204:0/1793858978 >> 172.21.15.204:0/1793858978 conn(0x7fe10806e860 msgr2=0x7fe10806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:06.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.913+0000 7fe10f35d640 1 -- 172.21.15.204:0/1793858978 shutdown_connections 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.913+0000 7fe10f35d640 1 -- 172.21.15.204:0/1793858978 wait complete. 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.914+0000 7fe10f35d640 10 monclient: build_initial_monmap 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.914+0000 7fe10f35d640 10 monclient: monmap: 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:06.932 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.914+0000 7fe10f35d640 1 Processor -- start 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.914+0000 7fe10f35d640 1 -- start start 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.914+0000 7fe10f35d640 10 monclient: init 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 10 monclient: _add_conns ranks=[7,6,8,3,4,2,5,0,1] 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe10809fb80 0x7fe1081317b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 10 monclient(hunting): picked mon.noname-i con 0x7fe10809fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:06.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1080a0a40 0x7fe10811e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 10 monclient(hunting): picked mon.noname-d con 0x7fe1080a0a40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1080a2e30 0x7fe10811e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d0f9640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe10809fb80 msgr2=0x7fe1081317b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 10 monclient(hunting): picked mon.noname-e con 0x7fe1080a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe10809ca40 con 0x7fe1080a0a40 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe10809d650 con 0x7fe10809fb80 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d0f9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe10809fb80 0x7fe1081317b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe108120a10 con 0x7fe1080a2e30 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10f35d640 10 monclient(hunting): authenticate will time out at 982.557133s 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10c8f8640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1080a0a40 msgr2=0x7fe10811e3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:06.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10c8f8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1080a0a40 0x7fe10811e3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1080a2e30 0x7fe10811e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1080a2e30 0x7fe10811e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:60480/0 (socket says 172.21.15.204:60480) 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 1 -- 172.21.15.204:0/1621147803 learned_addr learned my addr 172.21.15.204:0/1621147803 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 10 monclient(hunting): get_auth_request con 0x7fe1080a2e30 auth_method 0 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 10 monclient(hunting): handle_auth_done global_id 5831 payload 824 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 1 -- 172.21.15.204:0/1621147803 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe10809fb80 msgr2=0x7fe1081317b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:06.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe10809fb80 0x7fe1081317b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 1 -- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1080a0a40 msgr2=0x7fe10811e3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:06.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1080a0a40 0x7fe10811e3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 10 monclient: _finish_hunting 0 2024-09-15T01:36:06.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 1 monclient: found mon.noname-e 2024-09-15T01:36:06.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:06.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:06.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.915+0000 7fe10d8fa640 1 -- 172.21.15.204:0/1621147803 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f8009ae0 con 0x7fe1080a2e30 2024-09-15T01:36:06.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d8fa640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1080a2e30 0x7fe10811e920 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe108120be0 tx=0x7fe1000090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:06.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe10000ce90 con 0x7fe1080a2e30 2024-09-15T01:36:06.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:06.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:06.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient: dump: 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:06.937 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d8fa640 1 -- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1080a2e30 msgr2=0x7fe10811e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d8fa640 1 -- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1080a2e30 msgr2=0x7fe10811e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient: _finish_auth 0 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100009620 con 0x7fe1080a2e30 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10f35d640 5 monclient: authenticate success, global_id 5831 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d8fa640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1080a2e30 0x7fe10811e920 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe108120be0 tx=0x7fe1000090f0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10f35d640 10 monclient: _renew_subs 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d8fa640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1080a2e30 0x7fe10811e920 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe108120be0 tx=0x7fe1000090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10f35d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:06.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10f35d640 1 -- 172.21.15.204:0/1621147803 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe108133f80 con 0x7fe1080a2e30 2024-09-15T01:36:06.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:06.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient: ms_handle_reset current mon [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:06.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:06.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1080a2e30 msgr2=0x7fe10811e920 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:06.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1080a2e30 0x7fe10811e920 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient: _add_conns ranks=[0,6,7,4,5,3,2,8,1] 2024-09-15T01:36:06.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0f40044e0 0x7fe0f40068e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient(hunting): picked mon.a con 0x7fe0f40044e0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:06.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1077fe640 4 set_mon_vals no callback set 2024-09-15T01:36:06.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1080a0a40 0x7fe0f4009080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient(hunting): picked mon.d con 0x7fe1080a0a40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe10809fb80 0x7fe0f400bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient(hunting): picked mon.i con 0x7fe10809fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f400c530 con 0x7fe0f40044e0 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f400c670 con 0x7fe1080a0a40 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f400c7b0 con 0x7fe10809fb80 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe1057fa640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10f35d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d8fa640 1 -- 172.21.15.204:0/1621147803 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe10809fb80 msgr2=0x7fe0f400bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d8fa640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe10809fb80 0x7fe0f400bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:06.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10c8f8640 1 -- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1080a0a40 msgr2=0x7fe0f4009080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10c8f8640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1080a0a40 0x7fe0f4009080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0f40044e0 0x7fe0f40068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 10 monclient(hunting): get_auth_request con 0x7fe0f40044e0 auth_method 0 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 10 monclient(hunting): handle_auth_done global_id 5831 payload 293 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 1 -- 172.21.15.204:0/1621147803 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe10809fb80 msgr2=0x7fe0f400bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe10809fb80 0x7fe0f400bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.941 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 1 -- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1080a0a40 msgr2=0x7fe0f4009080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1080a0a40 0x7fe0f4009080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 10 monclient: _finish_hunting 0 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 1 monclient: found mon.a 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 1 -- 172.21.15.204:0/1621147803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) v3 -- 0x7fe0f4003e30 con 0x7fe0f40044e0 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 1 -- 172.21.15.204:0/1621147803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe108133f80 con 0x7fe0f40044e0 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 10 monclient: _finish_auth 0 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.916+0000 7fe10d0f9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe10d0f9640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0f40044e0 0x7fe0f40068e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe10809e800 tx=0x7fe0f8036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe10f35d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe10f35d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:06.942 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0f8007300 con 0x7fe0f40044e0 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe10f35d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe10f35d640 1 -- 172.21.15.204:0/1621147803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe108070cc0 con 0x7fe0f40044e0 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe1057fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe1057fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe1057fa640 10 monclient: dump: 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:06.943 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:06.944 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:06.944 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:06.944 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:06.944 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:06.944 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:06.944 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:06.944 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:06.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8007480 con 0x7fe0f40044e0 2024-09-15T01:36:06.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe1057fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:06.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe106ffd640 4 set_mon_vals no callback set 2024-09-15T01:36:06.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.917+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe0f8038070 con 0x7fe0f40044e0 2024-09-15T01:36:06.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.918+0000 7fe1057fa640 1 --2- 172.21.15.204:0/1621147803 >> v2:172.21.15.204:6812/28210 conn(0x7fe0f4041da0 0x7fe0f4044250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:06.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.918+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 <== mon.0 v2:172.21.15.178:3300/0 4 ==== osd_map(159..159 src has 1..159) v4 ==== 9096+0+0 (secure 0 0 0) 0x7fe0f806bdb0 con 0x7fe0f40044e0 2024-09-15T01:36:06.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.918+0000 7fe10c8f8640 1 --2- 172.21.15.204:0/1621147803 >> v2:172.21.15.204:6812/28210 conn(0x7fe0f4041da0 0x7fe0f4044250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:06.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.918+0000 7fe10c8f8640 10 monclient: get_auth_request con 0x7fe0f4041da0 auth_method 0 2024-09-15T01:36:06.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.918+0000 7fe10c8f8640 1 --2- 172.21.15.204:0/1621147803 >> v2:172.21.15.204:6812/28210 conn(0x7fe0f4041da0 0x7fe0f4044250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe1080a2e00 tx=0x7fe0fc006da0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:06.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.922+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 <== mon.0 v2:172.21.15.178:3300/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe0f8007da0 con 0x7fe0f40044e0 2024-09-15T01:36:06.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.922+0000 7fe1057fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:06.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:06.922+0000 7fe1057fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:07.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.081+0000 7fe10f35d640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:07.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.081+0000 7fe10f35d640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:07.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.081+0000 7fe10f35d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:07.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.081+0000 7fe10f35d640 1 -- 172.21.15.204:0/1621147803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7fe1080a1f10 con 0x7fe0f40044e0 2024-09-15T01:36:07.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.082+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v159) v1 ==== 83+0+2378 (secure 0 0 0) 0x7fe0f803a370 con 0x7fe0f40044e0 2024-09-15T01:36:07.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.082+0000 7fe1057fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:07.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.082+0000 7fe1057fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:07.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.181+0000 7fe10f35d640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:07.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.181+0000 7fe10f35d640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:07.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.181+0000 7fe10f35d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:07.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.181+0000 7fe10f35d640 1 -- 172.21.15.204:0/1621147803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7fe10811f7e0 con 0x7fe0f40044e0 2024-09-15T01:36:07.183 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.182+0000 7fe1057fa640 1 -- 172.21.15.204:0/1621147803 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v159) v1 ==== 83+0+2378 (secure 0 0 0) 0x7fe0f803a9c0 con 0x7fe0f40044e0 2024-09-15T01:36:07.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.182+0000 7fe1057fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:07.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.182+0000 7fe1057fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:07.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.184+0000 7fe10f35d640 1 -- 172.21.15.204:0/1621147803 >> v2:172.21.15.204:6812/28210 conn(0x7fe0f4041da0 msgr2=0x7fe0f4044250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:07.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.184+0000 7fe10f35d640 1 --2- 172.21.15.204:0/1621147803 >> v2:172.21.15.204:6812/28210 conn(0x7fe0f4041da0 0x7fe0f4044250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe1080a2e00 tx=0x7fe0fc006da0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.184+0000 7fe10f35d640 10 monclient: shutdown 2024-09-15T01:36:07.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.184+0000 7fe10f35d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:07.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.184+0000 7fe10f35d640 1 -- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0f40044e0 msgr2=0x7fe0f40068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:07.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.184+0000 7fe10f35d640 1 --2- 172.21.15.204:0/1621147803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0f40044e0 0x7fe0f40068e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe10809e800 tx=0x7fe0f8036000 comp rx=0 tx=0).stop 2024-09-15T01:36:07.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.184+0000 7fe10d8fa640 1 -- 172.21.15.204:0/1621147803 reap_dead start 2024-09-15T01:36:07.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.184+0000 7fe10f35d640 1 -- 172.21.15.204:0/1621147803 shutdown_connections 2024-09-15T01:36:07.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.184+0000 7fe10f35d640 1 -- 172.21.15.204:0/1621147803 >> 172.21.15.204:0/1621147803 conn(0x7fe10806e860 msgr2=0x7fe10809bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:07.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.184+0000 7fe10f35d640 1 -- 172.21.15.204:0/1621147803 shutdown_connections 2024-09-15T01:36:07.186 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.184+0000 7fe10f35d640 1 -- 172.21.15.204:0/1621147803 wait complete. 2024-09-15T01:36:07.194 INFO:tasks.workunit.client.0.smithi204.stdout: 0 globster 0.08719 10.50000 11.50000 12.50000 osd.0 2024-09-15T01:36:07.194 INFO:tasks.workunit.client.0.smithi204.stdout: 0 globster 0.08719 10.50000 11.50000 12.50000 osd.0 2024-09-15T01:36:07.195 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T01:36:07.195 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 'osd\.0' 2024-09-15T01:36:07.195 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep globster 2024-09-15T01:36:07.195 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep '12\.' 2024-09-15T01:36:07.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.267+0000 7f23ad26f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:07.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.267+0000 7f23ad26f640 10 monclient: build_initial_monmap 2024-09-15T01:36:07.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.267+0000 7f23ad26f640 10 monclient: monmap: 2024-09-15T01:36:07.269 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:07.269 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:07.269 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:07.268359+0000 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:07.268359+0000 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:07.270 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:07.271 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:07.271 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.268+0000 7f23ad26f640 1 Processor -- start 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.268+0000 7f23ad26f640 1 -- start start 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.268+0000 7f23ad26f640 10 monclient: init 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.268+0000 7f23ad26f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.268+0000 7f23ad26f640 10 monclient: _add_conns ranks=[2,9,1,0,4,10,5,7,8,6,3] 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.268+0000 7f23ad26f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23a80a2e30 0x7f23a8132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 10 monclient(hunting): picked mon.noname-b con 0x7f23a80a2e30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f23a809fb80 0x7f23a809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 10 monclient(hunting): picked mon.noname-a con 0x7f23a809fb80 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f23a80a0a40 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a804b720 con 0x7f23a80a0a40 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a809e6c0 con 0x7f23a809fb80 2024-09-15T01:36:07.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80a04b0 con 0x7f23a80a2e30 2024-09-15T01:36:07.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:07.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 10 monclient(hunting): authenticate will time out at 982.911057s 2024-09-15T01:36:07.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a6ffd640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23a80a2e30 msgr2=0x7f23a8132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:07.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a77fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f23a80a0a40 0x7f23a80a0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43790/0 (socket says 172.21.15.204:43790) 2024-09-15T01:36:07.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a77fe640 1 -- 172.21.15.204:0/3334133313 learned_addr learned my addr 172.21.15.204:0/3334133313 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:36:07.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a6ffd640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23a80a2e30 0x7f23a8132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:07.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992613018 0 0) 0x7f23a804b720 con 0x7f23a80a0a40 2024-09-15T01:36:07.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:07.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:07.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient(hunting): my global_id is 6454 2024-09-15T01:36:07.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23940034e0 con 0x7f23a80a0a40 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f239e7fc640 1 --2- 172.21.15.204:0/3334133313 >> v2:172.21.15.178:3300/0 conn(0x7f23a809fb80 0x7f23a809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f239e7fc640 10 monclient(hunting): get_auth_request con 0x7f23a809fb80 auth_method 0 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f239e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f239e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f239e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1975363364 0 0) 0x7f23940034e0 con 0x7f23a80a0a40 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23a80a2e30 msgr2=0x7f23a8132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23a80a2e30 0x7f23a8132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 >> v2:172.21.15.178:3300/0 conn(0x7f23a809fb80 msgr2=0x7f23a809ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 --2- 172.21.15.204:0/3334133313 >> v2:172.21.15.178:3300/0 conn(0x7f23a809fb80 0x7f23a809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient: _finish_hunting 0 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:07.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a80a3200 con 0x7f23a80a0a40 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 5 monclient: authenticate success, global_id 6454 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f23a0001c70 con 0x7f23a80a0a40 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient: dump: 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.284 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:36:07.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 >> v1:172.21.15.204:6789/0 conn(0x7f23a80a0a40 legacy=0x7f23a80a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient: _add_conns ranks=[5,3,2,4,8,1,6,0,7] 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2394004220 0x7f2394004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient(hunting): picked mon.h con 0x7f2394004220 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2394004de0 0x7f2394005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient(hunting): picked mon.g con 0x7f2394004de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23a80a2e30 0x7f239400bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient(hunting): picked mon.b con 0x7f23a80a2e30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a809e6c0 con 0x7f23a80a2e30 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80a04b0 con 0x7f2394004de0 2024-09-15T01:36:07.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f239400c490 con 0x7f2394004220 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a67fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23ad26f640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a77fe640 1 -- 172.21.15.204:0/3334133313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23a80a2e30 msgr2=0x7f239400bd50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a77fe640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23a80a2e30 0x7f239400bd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a6ffd640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2394004220 0x7f2394004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f239e7fc640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2394004de0 0x7f2394005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a6ffd640 10 monclient(hunting): get_auth_request con 0x7f2394004220 auth_method 0 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a6ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f239e7fc640 10 monclient(hunting): get_auth_request con 0x7f2394004de0 auth_method 0 2024-09-15T01:36:07.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f239e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f239e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f239e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f23a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f239e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.269+0000 7f239e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f239e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 10 monclient(hunting): handle_auth_done global_id 6454 payload 293 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 1 -- 172.21.15.204:0/3334133313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2394004de0 msgr2=0x7f2394005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2394004de0 0x7f2394005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 1 -- 172.21.15.204:0/3334133313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23a80a2e30 msgr2=0x7f239400bd50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23a80a2e30 0x7f239400bd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 1 monclient: found mon.h 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 1 -- 172.21.15.204:0/3334133313 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23940034e0 con 0x7f2394004220 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f239e7fc640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2394004de0 0x7f2394005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a6ffd640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2394004220 0x7f2394004780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23980099c0 tx=0x7f2398009060 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a89854dc2f77d5b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2398003df0 con 0x7f2394004220 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a67fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a67fc640 10 monclient: dump: 2024-09-15T01:36:07.289 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:07.290 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:07.290 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.290 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.290 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:07.290 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:07.290 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:07.290 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:07.290 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:07.290 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:07.290 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:07.290 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:07.290 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:07.291 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:07.291 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:07.291 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23ad26f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:07.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239802a930 con 0x7f2394004220 2024-09-15T01:36:07.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:07.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23ad26f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:07.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a67fc640 1 -- 172.21.15.204:0/3334133313 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f23980050f0 con 0x7f2394004220 2024-09-15T01:36:07.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23ad26f640 4 set_mon_vals no callback set 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a67fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.270+0000 7f23a67fc640 10 monclient: dump: 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:07.292 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.271+0000 7f23ad26f640 10 monclient: shutdown 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.271+0000 7f23ad26f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.271+0000 7f23ad26f640 1 -- 172.21.15.204:0/3334133313 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2394004220 msgr2=0x7f2394004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.271+0000 7f23ad26f640 1 --2- 172.21.15.204:0/3334133313 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2394004220 0x7f2394004780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23980099c0 tx=0x7f2398009060 comp rx=0 tx=0).stop 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.271+0000 7f23a77fe640 1 -- 172.21.15.204:0/3334133313 reap_dead start 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.271+0000 7f23ad26f640 1 -- 172.21.15.204:0/3334133313 shutdown_connections 2024-09-15T01:36:07.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.271+0000 7f23ad26f640 1 -- 172.21.15.204:0/3334133313 >> 172.21.15.204:0/3334133313 conn(0x7f23a806e860 msgr2=0x7f23a806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:07.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.271+0000 7f23ad26f640 1 -- 172.21.15.204:0/3334133313 shutdown_connections 2024-09-15T01:36:07.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.271+0000 7f23ad26f640 1 -- 172.21.15.204:0/3334133313 wait complete. 2024-09-15T01:36:07.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.271+0000 7f23ad26f640 10 monclient: build_initial_monmap 2024-09-15T01:36:07.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.271+0000 7f23ad26f640 10 monclient: monmap: 2024-09-15T01:36:07.294 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:07.294 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:07.294 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:07.294 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:07.294 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:07.294 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:07.294 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:07.294 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 1 Processor -- start 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 1 -- start start 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 10 monclient: init 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 10 monclient: _add_conns ranks=[5,7,0,1,6,2,3,8,4] 2024-09-15T01:36:07.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2394004220 0x7f23a8141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 10 monclient(hunting): picked mon.noname-h con 0x7f2394004220 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f23a81422c0 0x7f23a811e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 10 monclient(hunting): picked mon.noname-i con 0x7f23a81422c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23a8142690 0x7f23a811e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 10 monclient(hunting): picked mon.noname-a con 0x7f23a8142690 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a809ca40 con 0x7f23a8142690 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23a809d650 con 0x7f2394004220 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f23a8121d70 con 0x7f23a81422c0 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:07.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f23ad26f640 10 monclient(hunting): authenticate will time out at 982.914941s 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f239e7fc640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f23a81422c0 msgr2=0x7f23a811e270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.272+0000 7f239e7fc640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f23a81422c0 0x7f23a811e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2394004220 0x7f23a8141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2394004220 0x7f23a8141d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:46012/0 (socket says 172.21.15.204:46012) 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 1 -- 172.21.15.204:0/3157775278 learned_addr learned my addr 172.21.15.204:0/3157775278 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 10 monclient(hunting): get_auth_request con 0x7f2394004220 auth_method 0 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:07.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 10 monclient(hunting): handle_auth_done global_id 5810 payload 824 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 1 -- 172.21.15.204:0/3157775278 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f23a81422c0 msgr2=0x7f23a811e270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 1 --2- 172.21.15.204:0/3157775278 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f23a81422c0 0x7f23a811e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 1 -- 172.21.15.204:0/3157775278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23a8142690 msgr2=0x7f23a811e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 1 --2- 172.21.15.204:0/3157775278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23a8142690 0x7f23a811e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 1 monclient: found mon.noname-h 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 1 -- 172.21.15.204:0/3157775278 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a8120850 con 0x7f2394004220 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a6ffd640 1 --2- 172.21.15.204:0/3157775278 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2394004220 0x7f23a8141d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2398003fd0 tx=0x7f2398030000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f239f7fe640 1 -- 172.21.15.204:0/3157775278 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f239803b070 con 0x7f2394004220 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f239f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:07.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f239f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f239f7fe640 10 monclient: dump: 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:07.299 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:07.300 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:07.300 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:07.300 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:07.300 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f239f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:07.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f239f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:07.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f239f7fe640 1 -- 172.21.15.204:0/3157775278 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398005700 con 0x7f2394004220 2024-09-15T01:36:07.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23ad26f640 5 monclient: authenticate success, global_id 5810 2024-09-15T01:36:07.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23ad26f640 10 monclient: _renew_subs 2024-09-15T01:36:07.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23ad26f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:07.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23ad26f640 1 -- 172.21.15.204:0/3157775278 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a811fc80 con 0x7f2394004220 2024-09-15T01:36:07.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f239f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f239f7fe640 1 -- 172.21.15.204:0/3157775278 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f23980338f0 con 0x7f2394004220 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f239f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f239f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f239f7fe640 10 monclient: dump: 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:07.301 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:07.302 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:07.302 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:07.302 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:07.302 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:07.302 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:07.302 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:07.302 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23a5ffb640 4 set_mon_vals no callback set 2024-09-15T01:36:07.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23ad26f640 10 monclient: _renew_subs 2024-09-15T01:36:07.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23ad26f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:07.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.273+0000 7f23ad26f640 1 -- 172.21.15.204:0/3157775278 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a8122960 con 0x7f2394004220 2024-09-15T01:36:07.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.274+0000 7f239f7fe640 1 -- 172.21.15.204:0/3157775278 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2398030070 con 0x7f2394004220 2024-09-15T01:36:07.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.274+0000 7f23ad26f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:07.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.274+0000 7f23ad26f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:07.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.274+0000 7f23ad26f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:07.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.274+0000 7f23ad26f640 1 -- 172.21.15.204:0/3157775278 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a8121900 con 0x7f2394004220 2024-09-15T01:36:07.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.274+0000 7f239f7fe640 1 --2- 172.21.15.204:0/3157775278 >> v2:172.21.15.204:6812/28210 conn(0x7f23880396e0 0x7f238803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.274+0000 7f239f7fe640 1 -- 172.21.15.204:0/3157775278 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 9096+0+0 (secure 0 0 0) 0x7f239806b980 con 0x7f2394004220 2024-09-15T01:36:07.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.274+0000 7f239e7fc640 1 --2- 172.21.15.204:0/3157775278 >> v2:172.21.15.204:6812/28210 conn(0x7f23880396e0 0x7f238803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:07.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.274+0000 7f239e7fc640 10 monclient: get_auth_request con 0x7f23880396e0 auth_method 0 2024-09-15T01:36:07.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.275+0000 7f239e7fc640 1 --2- 172.21.15.204:0/3157775278 >> v2:172.21.15.204:6812/28210 conn(0x7f23880396e0 0x7f238803bb90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2390006fd0 tx=0x7f2390008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:07.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.278+0000 7f239f7fe640 1 -- 172.21.15.204:0/3157775278 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f23980479b0 con 0x7f2394004220 2024-09-15T01:36:07.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.278+0000 7f239f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:07.303 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.278+0000 7f239f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:07.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.440+0000 7f23ad26f640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:07.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.440+0000 7f23ad26f640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:07.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.440+0000 7f23ad26f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:07.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.440+0000 7f23ad26f640 1 -- 172.21.15.204:0/3157775278 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f23a8070cc0 con 0x7f2394004220 2024-09-15T01:36:07.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.441+0000 7f239f7fe640 1 -- 172.21.15.204:0/3157775278 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v159) v1 ==== 83+0+2378 (secure 0 0 0) 0x7f23980323d0 con 0x7f2394004220 2024-09-15T01:36:07.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.441+0000 7f239f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:07.442 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.441+0000 7f239f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:07.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.540+0000 7f23ad26f640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:07.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.540+0000 7f23ad26f640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:07.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.540+0000 7f23ad26f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:07.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.540+0000 7f23ad26f640 1 -- 172.21.15.204:0/3157775278 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f23a80a1f10 con 0x7f2394004220 2024-09-15T01:36:07.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.541+0000 7f239f7fe640 1 -- 172.21.15.204:0/3157775278 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v159) v1 ==== 83+0+2378 (secure 0 0 0) 0x7f23980473b0 con 0x7f2394004220 2024-09-15T01:36:07.542 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.541+0000 7f239f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:07.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.541+0000 7f239f7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 1 -- 172.21.15.204:0/3157775278 >> v2:172.21.15.204:6812/28210 conn(0x7f23880396e0 msgr2=0x7f238803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 1 --2- 172.21.15.204:0/3157775278 >> v2:172.21.15.204:6812/28210 conn(0x7f23880396e0 0x7f238803bb90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2390006fd0 tx=0x7f2390008040 comp rx=0 tx=0).stop 2024-09-15T01:36:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 10 monclient: shutdown 2024-09-15T01:36:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 1 -- 172.21.15.204:0/3157775278 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2394004220 msgr2=0x7f23a8141d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 1 --2- 172.21.15.204:0/3157775278 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2394004220 0x7f23a8141d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2398003fd0 tx=0x7f2398030000 comp rx=0 tx=0).stop 2024-09-15T01:36:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 1 -- 172.21.15.204:0/3157775278 shutdown_connections 2024-09-15T01:36:07.546 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 1 --2- 172.21.15.204:0/3157775278 >> v2:172.21.15.204:6812/28210 conn(0x7f23880396e0 0x7f238803bb90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 1 --2- 172.21.15.204:0/3157775278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23a8142690 0x7f23a811e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 1 --2- 172.21.15.204:0/3157775278 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f23a81422c0 0x7f23a811e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 1 --2- 172.21.15.204:0/3157775278 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2394004220 0x7f23a8141d80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 1 -- 172.21.15.204:0/3157775278 >> 172.21.15.204:0/3157775278 conn(0x7f23a806e860 msgr2=0x7f23a80a3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:07.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 1 -- 172.21.15.204:0/3157775278 shutdown_connections 2024-09-15T01:36:07.547 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.543+0000 7f23ad26f640 1 -- 172.21.15.204:0/3157775278 wait complete. 2024-09-15T01:36:07.554 INFO:tasks.workunit.client.0.smithi204.stdout: 0 globster 0.08719 10.50000 11.50000 12.50000 osd.0 2024-09-15T01:36:07.554 INFO:tasks.workunit.client.0.smithi204.stdout: 0 globster 0.08719 10.50000 11.50000 12.50000 osd.0 2024-09-15T01:36:07.554 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set reweight-compat osd.0 7.5 2024-09-15T01:36:07.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.627+0000 7f958fa3f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:07.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.627+0000 7f958fa3f640 10 monclient: build_initial_monmap 2024-09-15T01:36:07.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.627+0000 7f958fa3f640 10 monclient: monmap: 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:07.628223+0000 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:07.628223+0000 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:07.630 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:07.631 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:07.631 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:07.631 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:07.631 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:07.631 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 1 Processor -- start 2024-09-15T01:36:07.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 1 -- start start 2024-09-15T01:36:07.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 10 monclient: init 2024-09-15T01:36:07.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:07.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 10 monclient: _add_conns ranks=[7,6,5,4,8,1,10,2,9,3,0] 2024-09-15T01:36:07.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95880a2cd0 0x7f9588132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 10 monclient(hunting): picked mon.noname-i con 0x7f95880a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:07.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f958809fa20 0x7f958809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 10 monclient(hunting): picked mon.noname-d con 0x7f958809fa20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95880a08e0 0x7f95880a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 10 monclient(hunting): picked mon.noname-h con 0x7f95880a08e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95880a30a0 con 0x7f95880a08e0 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f95880a0330 con 0x7f958809fa20 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f95880a1db0 con 0x7f95880a2cd0 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958fa3f640 10 monclient(hunting): authenticate will time out at 983.270941s 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.628+0000 7f958d7db640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95880a2cd0 msgr2=0x7f9588132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958d7db640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95880a2cd0 0x7f9588132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:07.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95880a08e0 0x7f95880a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958cfda640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f958809fa20 msgr2=0x7f958809fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95880a08e0 0x7f95880a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:46022/0 (socket says 172.21.15.204:46022) 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958cfda640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f958809fa20 0x7f958809fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 1 -- 172.21.15.204:0/2425862224 learned_addr learned my addr 172.21.15.204:0/2425862224 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 10 monclient(hunting): get_auth_request con 0x7f95880a08e0 auth_method 0 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 10 monclient(hunting): handle_auth_done global_id 5819 payload 274 2024-09-15T01:36:07.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 1 -- 172.21.15.204:0/2425862224 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95880a2cd0 msgr2=0x7f9588132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:07.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 1 --2- 172.21.15.204:0/2425862224 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95880a2cd0 0x7f9588132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 1 -- 172.21.15.204:0/2425862224 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f958809fa20 msgr2=0x7f958809fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:07.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 1 --2- 172.21.15.204:0/2425862224 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f958809fa20 0x7f958809fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 10 monclient: _finish_hunting 0 2024-09-15T01:36:07.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 1 monclient: found mon.noname-h 2024-09-15T01:36:07.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:07.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:07.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 1 -- 172.21.15.204:0/2425862224 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880648a0 con 0x7f95880a08e0 2024-09-15T01:36:07.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.629+0000 7f958dfdc640 1 --2- 172.21.15.204:0/2425862224 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95880a08e0 0x7f95880a0d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9580009f50 tx=0x7f9580009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=40fd6e7278c78c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:07.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f957ffff640 1 -- 172.21.15.204:0/2425862224 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9580007880 con 0x7f95880a08e0 2024-09-15T01:36:07.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f957ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:07.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f957ffff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f957ffff640 10 monclient: dump: 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:07.642 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:07.643 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:07.643 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:07.643 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f957ffff640 10 monclient: _finish_auth 0 2024-09-15T01:36:07.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f957ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:07.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f957ffff640 1 -- 172.21.15.204:0/2425862224 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958002d050 con 0x7f95880a08e0 2024-09-15T01:36:07.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 5 monclient: authenticate success, global_id 5819 2024-09-15T01:36:07.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:07.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f957ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:07.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:07.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f957ffff640 1 -- 172.21.15.204:0/2425862224 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9580030360 con 0x7f95880a08e0 2024-09-15T01:36:07.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 4 set_mon_vals no callback set 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f957ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f957ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f957ffff640 10 monclient: dump: 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:07.644 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 10 monclient: shutdown 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 1 -- 172.21.15.204:0/2425862224 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95880a08e0 msgr2=0x7f95880a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 1 --2- 172.21.15.204:0/2425862224 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95880a08e0 0x7f95880a0d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9580009f50 tx=0x7f9580009840 comp rx=0 tx=0).stop 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 1 -- 172.21.15.204:0/2425862224 shutdown_connections 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 1 --2- 172.21.15.204:0/2425862224 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95880a08e0 0x7f95880a0d50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.645 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 1 --2- 172.21.15.204:0/2425862224 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f958809fa20 0x7f958809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 1 --2- 172.21.15.204:0/2425862224 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95880a2cd0 0x7f9588132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 1 -- 172.21.15.204:0/2425862224 >> 172.21.15.204:0/2425862224 conn(0x7f958806e860 msgr2=0x7f958806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 1 -- 172.21.15.204:0/2425862224 shutdown_connections 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.630+0000 7f958fa3f640 1 -- 172.21.15.204:0/2425862224 wait complete. 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.631+0000 7f958fa3f640 10 monclient: build_initial_monmap 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.631+0000 7f958fa3f640 10 monclient: monmap: 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:07.646 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.631+0000 7f958fa3f640 1 Processor -- start 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.631+0000 7f958fa3f640 1 -- start start 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.631+0000 7f958fa3f640 10 monclient: init 2024-09-15T01:36:07.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 10 monclient: _add_conns ranks=[6,3,1,4,0,8,2,7,5] 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f958809fa20 0x7f9588130fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 10 monclient(hunting): picked mon.noname-d con 0x7f958809fa20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95880a08e0 0x7f958809b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 10 monclient(hunting): picked mon.noname-g con 0x7f95880a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95880a2cd0 0x7f958809b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 10 monclient(hunting): picked mon.noname-f con 0x7f95880a2cd0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958809d3d0 con 0x7f95880a2cd0 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958809cb60 con 0x7f95880a08e0 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:07.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f95881370c0 con 0x7f958809fa20 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958fa3f640 10 monclient(hunting): authenticate will time out at 983.274293s 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95880a08e0 0x7f958809b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958d7db640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f958809fa20 msgr2=0x7f9588130fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958dfdc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95880a2cd0 0x7f958809b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958d7db640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f958809fa20 0x7f9588130fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95880a08e0 0x7f958809b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:33458/0 (socket says 172.21.15.204:33458) 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 1 -- 172.21.15.204:0/117298401 learned_addr learned my addr 172.21.15.204:0/117298401 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958dfdc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95880a2cd0 0x7f958809b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39922/0 (socket says 172.21.15.204:39922) 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 10 monclient(hunting): get_auth_request con 0x7f95880a08e0 auth_method 0 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:07.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958dfdc640 10 monclient(hunting): get_auth_request con 0x7f95880a2cd0 auth_method 0 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958dfdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958dfdc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958dfdc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958dfdc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958dfdc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958dfdc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 10 monclient(hunting): handle_auth_done global_id 14916 payload 824 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 1 -- 172.21.15.204:0/117298401 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f958809fa20 msgr2=0x7f9588130fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:07.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 1 --2- 172.21.15.204:0/117298401 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f958809fa20 0x7f9588130fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 1 -- 172.21.15.204:0/117298401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95880a2cd0 msgr2=0x7f958809b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 1 --2- 172.21.15.204:0/117298401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95880a2cd0 0x7f958809b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 10 monclient: _finish_hunting 0 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 1 monclient: found mon.noname-g 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 1 -- 172.21.15.204:0/117298401 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9580009ae0 con 0x7f95880a08e0 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958dfdc640 1 --2- 172.21.15.204:0/117298401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95880a2cd0 0x7f958809b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.632+0000 7f958cfda640 1 --2- 172.21.15.204:0/117298401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95880a08e0 0x7f958809b020 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f957800b600 tx=0x7f95780091a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 1 -- 172.21.15.204:0/117298401 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9578015070 con 0x7f95880a08e0 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:07.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 10 monclient: dump: 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:07.652 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 10 monclient: _finish_auth 0 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 1 -- 172.21.15.204:0/117298401 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9578006150 con 0x7f95880a08e0 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f958fa3f640 5 monclient: authenticate success, global_id 14916 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 1 -- 172.21.15.204:0/117298401 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9578006740 con 0x7f95880a08e0 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f958fa3f640 10 monclient: _renew_subs 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f958fa3f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f958fa3f640 1 -- 172.21.15.204:0/117298401 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958811e6b0 con 0x7f95880a08e0 2024-09-15T01:36:07.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957f7fe640 4 set_mon_vals no callback set 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 10 monclient: dump: 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:07.654 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f958fa3f640 10 monclient: _renew_subs 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f958fa3f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f958fa3f640 1 -- 172.21.15.204:0/117298401 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958811d200 con 0x7f95880a08e0 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.633+0000 7f957d7fa640 1 -- 172.21.15.204:0/117298401 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f95780062c0 con 0x7f95880a08e0 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.634+0000 7f958fa3f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.634+0000 7f957d7fa640 1 --2- 172.21.15.204:0/117298401 >> v2:172.21.15.204:6812/28210 conn(0x7f956c0397b0 0x7f956c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:07.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.634+0000 7f958fa3f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:07.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.634+0000 7f958fa3f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:07.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.634+0000 7f958fa3f640 1 -- 172.21.15.204:0/117298401 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9588070b80 con 0x7f95880a08e0 2024-09-15T01:36:07.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.634+0000 7f957d7fa640 1 -- 172.21.15.204:0/117298401 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 9096+0+0 (secure 0 0 0) 0x7f957804bf50 con 0x7f95880a08e0 2024-09-15T01:36:07.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.634+0000 7f958d7db640 1 --2- 172.21.15.204:0/117298401 >> v2:172.21.15.204:6812/28210 conn(0x7f956c0397b0 0x7f956c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:07.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.634+0000 7f958d7db640 10 monclient: get_auth_request con 0x7f956c0397b0 auth_method 0 2024-09-15T01:36:07.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.634+0000 7f958d7db640 1 --2- 172.21.15.204:0/117298401 >> v2:172.21.15.204:6812/28210 conn(0x7f956c0397b0 0x7f956c03bc60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9584006fd0 tx=0x7f9584006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:07.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.638+0000 7f957d7fa640 1 -- 172.21.15.204:0/117298401 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9578010440 con 0x7f95880a08e0 2024-09-15T01:36:07.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.638+0000 7f957d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:07.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.638+0000 7f957d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:07.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.800+0000 7f958fa3f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-09-15T01:36:07.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.800+0000 7f958fa3f640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-09-15T01:36:07.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.800+0000 7f958fa3f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:07.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.800+0000 7f958fa3f640 1 -- 172.21.15.204:0/117298401 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]} v 0) v1 -- 0x7f9588067450 con 0x7f95880a08e0 2024-09-15T01:36:07.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.825+0000 7f957d7fa640 1 -- 172.21.15.204:0/117298401 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}]=0 v160) v1 ==== 118+0+0 (secure 0 0 0) 0x7f957800d370 con 0x7f95880a08e0 2024-09-15T01:36:07.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.825+0000 7f957d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-09-15T01:36:07.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.825+0000 7f957d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:07.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.928+0000 7f958fa3f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-09-15T01:36:07.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.928+0000 7f958fa3f640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-09-15T01:36:07.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.928+0000 7f958fa3f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:07.929 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:07.928+0000 7f958fa3f640 1 -- 172.21.15.204:0/117298401 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]} v 0) v1 -- 0x7f9588142a00 con 0x7f95880a08e0 2024-09-15T01:36:08.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.839+0000 7f957d7fa640 1 -- 172.21.15.204:0/117298401 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}]=0 v161) v1 ==== 118+0+0 (secure 0 0 0) 0x7f957800eb30 con 0x7f95880a08e0 2024-09-15T01:36:08.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.839+0000 7f957d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-09-15T01:36:08.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.839+0000 7f957d7fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:08.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 1 -- 172.21.15.204:0/117298401 >> v2:172.21.15.204:6812/28210 conn(0x7f956c0397b0 msgr2=0x7f956c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:08.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 1 --2- 172.21.15.204:0/117298401 >> v2:172.21.15.204:6812/28210 conn(0x7f956c0397b0 0x7f956c03bc60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9584006fd0 tx=0x7f9584006d30 comp rx=0 tx=0).stop 2024-09-15T01:36:08.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 10 monclient: shutdown 2024-09-15T01:36:08.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:08.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 1 -- 172.21.15.204:0/117298401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95880a08e0 msgr2=0x7f958809b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:08.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 1 --2- 172.21.15.204:0/117298401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95880a08e0 0x7f958809b020 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f957800b600 tx=0x7f95780091a0 comp rx=0 tx=0).stop 2024-09-15T01:36:08.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 1 -- 172.21.15.204:0/117298401 shutdown_connections 2024-09-15T01:36:08.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 1 --2- 172.21.15.204:0/117298401 >> v2:172.21.15.204:6812/28210 conn(0x7f956c0397b0 0x7f956c03bc60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:08.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 1 --2- 172.21.15.204:0/117298401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95880a2cd0 0x7f958809b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:08.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 1 --2- 172.21.15.204:0/117298401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95880a08e0 0x7f958809b020 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:08.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 1 --2- 172.21.15.204:0/117298401 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f958809fa20 0x7f9588130fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:08.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 1 -- 172.21.15.204:0/117298401 >> 172.21.15.204:0/117298401 conn(0x7f958806e860 msgr2=0x7f95880a5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:08.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 1 -- 172.21.15.204:0/117298401 shutdown_connections 2024-09-15T01:36:08.846 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.843+0000 7f958fa3f640 1 -- 172.21.15.204:0/117298401 wait complete. 2024-09-15T01:36:08.855 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set reweight cool osd.0 8.5 2024-09-15T01:36:08.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.933+0000 7f4b4080f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:08.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.933+0000 7f4b4080f640 10 monclient: build_initial_monmap 2024-09-15T01:36:08.935 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.933+0000 7f4b4080f640 10 monclient: monmap: 2024-09-15T01:36:08.935 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:08.935 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:08.935 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:08.934045+0000 2024-09-15T01:36:08.935 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:08.934045+0000 2024-09-15T01:36:08.935 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:08.935 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:08.936 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:08.936 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:08.936 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:08.936 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:08.936 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:08.936 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:08.936 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:08.936 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:08.936 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:08.936 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:08.936 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:08.936 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:08.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 1 Processor -- start 2024-09-15T01:36:08.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 1 -- start start 2024-09-15T01:36:08.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 10 monclient: init 2024-09-15T01:36:08.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:08.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 10 monclient: _add_conns ranks=[10,0,7,2,1,9,6,3,4,5,8] 2024-09-15T01:36:08.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4b3c0a2cd0 0x7f4b3c132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:08.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 10 monclient(hunting): picked mon.noname-f con 0x7f4b3c0a2cd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:08.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4b3c09fa20 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:08.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b3c137c80 0x7f4b3c13a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:08.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 10 monclient(hunting): picked mon.noname-i con 0x7f4b3c137c80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:08.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3c0648a0 con 0x7f4b3c09fa20 2024-09-15T01:36:08.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:08.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3c0a30a0 con 0x7f4b3c0a2cd0 2024-09-15T01:36:08.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c0a1db0 con 0x7f4b3c137c80 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b4080f640 10 monclient(hunting): authenticate will time out at 984.576807s 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b3ad9d640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b3c137c80 msgr2=0x7f4b3c13a060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b3ad9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b3c137c80 0x7f4b3c13a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b3a59c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4b3c0a2cd0 0x7f4b3c132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b3a59c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4b3c0a2cd0 0x7f4b3c132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39936/0 (socket says 172.21.15.204:39936) 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b3a59c640 1 -- 172.21.15.204:0/3835949868 learned_addr learned my addr 172.21.15.204:0/3835949868 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b3a59c640 10 monclient(hunting): get_auth_request con 0x7f4b3c0a2cd0 auth_method 0 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b3a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b3a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.934+0000 7f4b3a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:08.946 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:08.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:08.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:08.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 10 monclient(hunting): handle_auth_done global_id 6472 payload 274 2024-09-15T01:36:08.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 1 -- 172.21.15.204:0/3835949868 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b3c137c80 msgr2=0x7f4b3c13a060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:08.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 1 --2- 172.21.15.204:0/3835949868 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b3c137c80 0x7f4b3c13a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:08.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 1 -- 172.21.15.204:0/3835949868 >> v1:172.21.15.178:6789/0 conn(0x7f4b3c09fa20 legacy=0x7f4b3c09fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:08.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:08.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 1 monclient: found mon.noname-f 2024-09-15T01:36:08.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:08.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:08.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 1 -- 172.21.15.204:0/3835949868 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c09dc50 con 0x7f4b3c0a2cd0 2024-09-15T01:36:08.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3a59c640 1 --2- 172.21.15.204:0/3835949868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b3c0a2cd0 0x7f4b3c132c00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009910 tx=0x7f4b2c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9aa3c29c830de server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3959a640 1 -- 172.21.15.204:0/3835949868 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b2c02acc0 con 0x7f4b3c0a2cd0 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.935+0000 7f4b3959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b3959a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b3959a640 10 monclient: dump: 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:08.948 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b3959a640 10 monclient: _finish_auth 0 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b3959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b3959a640 1 -- 172.21.15.204:0/3835949868 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2c02ae40 con 0x7f4b3c0a2cd0 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 5 monclient: authenticate success, global_id 6472 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:08.949 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b3959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b3959a640 1 -- 172.21.15.204:0/3835949868 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b2c031ca0 con 0x7f4b3c0a2cd0 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 4 set_mon_vals no callback set 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b3959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b3959a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b3959a640 10 monclient: dump: 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:08.950 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 10 monclient: shutdown 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 1 -- 172.21.15.204:0/3835949868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b3c0a2cd0 msgr2=0x7f4b3c132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 1 --2- 172.21.15.204:0/3835949868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b3c0a2cd0 0x7f4b3c132c00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009910 tx=0x7f4b2c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:08.951 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 1 -- 172.21.15.204:0/3835949868 shutdown_connections 2024-09-15T01:36:08.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 1 --2- 172.21.15.204:0/3835949868 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b3c137c80 0x7f4b3c13a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:08.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 1 --2- 172.21.15.204:0/3835949868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b3c0a2cd0 0x7f4b3c132c00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:08.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 1 -- 172.21.15.204:0/3835949868 >> 172.21.15.204:0/3835949868 conn(0x7f4b3c06e860 msgr2=0x7f4b3c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:08.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 1 -- 172.21.15.204:0/3835949868 shutdown_connections 2024-09-15T01:36:08.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.936+0000 7f4b4080f640 1 -- 172.21.15.204:0/3835949868 wait complete. 2024-09-15T01:36:08.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient: build_initial_monmap 2024-09-15T01:36:08.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient: monmap: 2024-09-15T01:36:08.952 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:08.952 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:08.952 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:08.952 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:08.952 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:08.953 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:08.953 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:08.953 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:08.953 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:08.953 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:08.953 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:08.953 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:08.953 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:08.953 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:08.953 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:08.953 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:08.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 1 Processor -- start 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 1 -- start start 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient: init 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient: _add_conns ranks=[7,0,8,1,3,4,2,6,5] 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b3c09fa20 0x7f4b3c1420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient(hunting): picked mon.noname-i con 0x7f4b3c09fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b3c0a2cd0 0x7f4b3c142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient(hunting): picked mon.noname-a con 0x7f4b3c0a2cd0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b3c137c80 0x7f4b3c142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient(hunting): picked mon.noname-e con 0x7f4b3c137c80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c09b600 con 0x7f4b3c0a2cd0 2024-09-15T01:36:08.954 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c09cb60 con 0x7f4b3c09fa20 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c09c830 con 0x7f4b3c137c80 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b3a59c640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b3c09fa20 msgr2=0x7f4b3c1420f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b4080f640 10 monclient(hunting): authenticate will time out at 984.579857s 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.937+0000 7f4b3a59c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b3c09fa20 0x7f4b3c1420f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b3c137c80 0x7f4b3c142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b39d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b3c0a2cd0 0x7f4b3c142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b3c137c80 0x7f4b3c142b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:37540/0 (socket says 172.21.15.204:37540) 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 1 -- 172.21.15.204:0/3532391393 learned_addr learned my addr 172.21.15.204:0/3532391393 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b39d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b3c0a2cd0 0x7f4b3c142630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:48128/0 (socket says 172.21.15.204:48128) 2024-09-15T01:36:08.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 10 monclient(hunting): get_auth_request con 0x7f4b3c137c80 auth_method 0 2024-09-15T01:36:08.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:08.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:08.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:08.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b39d9b640 10 monclient(hunting): get_auth_request con 0x7f4b3c0a2cd0 auth_method 0 2024-09-15T01:36:08.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b39d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:08.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b39d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:08.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b39d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:08.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:08.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:08.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:08.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b39d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:08.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b39d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b39d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 10 monclient(hunting): handle_auth_done global_id 5840 payload 824 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 1 -- 172.21.15.204:0/3532391393 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b3c09fa20 msgr2=0x7f4b3c1420f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 1 --2- 172.21.15.204:0/3532391393 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b3c09fa20 0x7f4b3c1420f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 1 -- 172.21.15.204:0/3532391393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b3c0a2cd0 msgr2=0x7f4b3c142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 1 --2- 172.21.15.204:0/3532391393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b3c0a2cd0 0x7f4b3c142630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 1 monclient: found mon.noname-e 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 1 -- 172.21.15.204:0/3532391393 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c009450 con 0x7f4b3c137c80 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b39d9b640 1 --2- 172.21.15.204:0/3532391393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b3c0a2cd0 0x7f4b3c142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:08.957 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b3ad9d640 1 --2- 172.21.15.204:0/3532391393 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b3c137c80 0x7f4b3c142b70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400beb0 tx=0x7f4b340096e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b2a7fc640 1 -- 172.21.15.204:0/3532391393 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b3400cd00 con 0x7f4b3c137c80 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b2a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b2a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b2a7fc640 10 monclient: dump: 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:08.958 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b2a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b2a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b2a7fc640 1 -- 172.21.15.204:0/3532391393 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b340065d0 con 0x7f4b3c137c80 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b4080f640 5 monclient: authenticate success, global_id 5840 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b2a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:08.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b4080f640 10 monclient: _renew_subs 2024-09-15T01:36:08.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b4080f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:08.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b2a7fc640 1 -- 172.21.15.204:0/3532391393 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b34006780 con 0x7f4b3c137c80 2024-09-15T01:36:08.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b4080f640 1 -- 172.21.15.204:0/3532391393 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3c09d3d0 con 0x7f4b3c137c80 2024-09-15T01:36:08.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b2a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:08.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.939+0000 7f4b2a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:08.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.939+0000 7f4b2a7fc640 10 monclient: dump: 2024-09-15T01:36:08.960 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:08.960 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:08.960 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:08.960 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:08.960 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:08.960 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.938+0000 7f4b38d99640 4 set_mon_vals no callback set 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.939+0000 7f4b4080f640 10 monclient: _renew_subs 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.939+0000 7f4b4080f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:08.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.939+0000 7f4b4080f640 1 -- 172.21.15.204:0/3532391393 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3c11c680 con 0x7f4b3c137c80 2024-09-15T01:36:08.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.939+0000 7f4b4080f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:08.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.939+0000 7f4b4080f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:08.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.939+0000 7f4b4080f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:08.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.939+0000 7f4b4080f640 1 -- 172.21.15.204:0/3532391393 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3c070b80 con 0x7f4b3c137c80 2024-09-15T01:36:08.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.939+0000 7f4b2a7fc640 1 -- 172.21.15.204:0/3532391393 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4b34012020 con 0x7f4b3c137c80 2024-09-15T01:36:08.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.940+0000 7f4b2a7fc640 1 --2- 172.21.15.204:0/3532391393 >> v2:172.21.15.204:6812/28210 conn(0x7f4b1c0396e0 0x7f4b1c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:08.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.940+0000 7f4b2a7fc640 1 -- 172.21.15.204:0/3532391393 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 9034+0+0 (secure 0 0 0) 0x7f4b3404c7e0 con 0x7f4b3c137c80 2024-09-15T01:36:08.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.940+0000 7f4b3a59c640 1 --2- 172.21.15.204:0/3532391393 >> v2:172.21.15.204:6812/28210 conn(0x7f4b1c0396e0 0x7f4b1c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:08.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.940+0000 7f4b3a59c640 10 monclient: get_auth_request con 0x7f4b1c0396e0 auth_method 0 2024-09-15T01:36:08.962 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.940+0000 7f4b3a59c640 1 --2- 172.21.15.204:0/3532391393 >> v2:172.21.15.204:6812/28210 conn(0x7f4b1c0396e0 0x7f4b1c03bb90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c034000 tx=0x7f4b2c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:08.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.944+0000 7f4b2a7fc640 1 -- 172.21.15.204:0/3532391393 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4b34013ac0 con 0x7f4b3c137c80 2024-09-15T01:36:08.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.944+0000 7f4b2a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:08.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:08.944+0000 7f4b2a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:09.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:09.106+0000 7f4b4080f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-09-15T01:36:09.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:09.106+0000 7f4b4080f640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-09-15T01:36:09.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:09.106+0000 7f4b4080f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:09.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:09.106+0000 7f4b4080f640 1 -- 172.21.15.204:0/3532391393 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]} v 0) v1 -- 0x7f4b3c067450 con 0x7f4b3c137c80 2024-09-15T01:36:09.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:09.839+0000 7f4b2a7fc640 1 -- 172.21.15.204:0/3532391393 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}]=0 v162) v1 ==== 127+0+0 (secure 0 0 0) 0x7f4b340132e0 con 0x7f4b3c137c80 2024-09-15T01:36:09.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:09.839+0000 7f4b2a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-09-15T01:36:09.841 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:09.839+0000 7f4b2a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:09.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:09.946+0000 7f4b4080f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-09-15T01:36:09.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:09.946+0000 7f4b4080f640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-09-15T01:36:09.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:09.946+0000 7f4b4080f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:09.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:09.946+0000 7f4b4080f640 1 -- 172.21.15.204:0/3532391393 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]} v 0) v1 -- 0x7f4b3c099640 con 0x7f4b3c137c80 2024-09-15T01:36:10.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.854+0000 7f4b2a7fc640 1 -- 172.21.15.204:0/3532391393 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}]=0 v163) v1 ==== 127+0+0 (secure 0 0 0) 0x7f4b340134c0 con 0x7f4b3c137c80 2024-09-15T01:36:10.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.854+0000 7f4b2a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-09-15T01:36:10.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.854+0000 7f4b2a7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:10.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 1 -- 172.21.15.204:0/3532391393 >> v2:172.21.15.204:6812/28210 conn(0x7f4b1c0396e0 msgr2=0x7f4b1c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:10.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 1 --2- 172.21.15.204:0/3532391393 >> v2:172.21.15.204:6812/28210 conn(0x7f4b1c0396e0 0x7f4b1c03bb90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c034000 tx=0x7f4b2c032000 comp rx=0 tx=0).stop 2024-09-15T01:36:10.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 10 monclient: shutdown 2024-09-15T01:36:10.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:10.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 1 -- 172.21.15.204:0/3532391393 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b3c137c80 msgr2=0x7f4b3c142b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:10.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 1 --2- 172.21.15.204:0/3532391393 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b3c137c80 0x7f4b3c142b70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400beb0 tx=0x7f4b340096e0 comp rx=0 tx=0).stop 2024-09-15T01:36:10.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 1 -- 172.21.15.204:0/3532391393 shutdown_connections 2024-09-15T01:36:10.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 1 --2- 172.21.15.204:0/3532391393 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b3c137c80 0x7f4b3c142b70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:10.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 1 --2- 172.21.15.204:0/3532391393 >> v2:172.21.15.204:6812/28210 conn(0x7f4b1c0396e0 0x7f4b1c03bb90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:10.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 1 --2- 172.21.15.204:0/3532391393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b3c0a2cd0 0x7f4b3c142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:10.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 1 --2- 172.21.15.204:0/3532391393 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b3c09fa20 0x7f4b3c1420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:10.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 1 -- 172.21.15.204:0/3532391393 >> 172.21.15.204:0/3532391393 conn(0x7f4b3c06e860 msgr2=0x7f4b3c0a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:10.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 1 -- 172.21.15.204:0/3532391393 shutdown_connections 2024-09-15T01:36:10.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.857+0000 7f4b4080f640 1 -- 172.21.15.204:0/3532391393 wait complete. 2024-09-15T01:36:10.868 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set reweight cold osd.0 6.5 6.6 2024-09-15T01:36:10.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.946+0000 7f05aab5d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:10.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.946+0000 7f05aab5d640 10 monclient: build_initial_monmap 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.946+0000 7f05aab5d640 10 monclient: monmap: 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:10.946643+0000 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:10.946643+0000 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:10.948 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:10.949 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:10.949 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:10.949 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:10.949 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:10.949 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:10.949 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:10.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.946+0000 7f05aab5d640 1 Processor -- start 2024-09-15T01:36:10.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.946+0000 7f05aab5d640 1 -- start start 2024-09-15T01:36:10.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.946+0000 7f05aab5d640 10 monclient: init 2024-09-15T01:36:10.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:10.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 10 monclient: _add_conns ranks=[6,1,10,3,0,8,5,4,7,2,9] 2024-09-15T01:36:10.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f05a40a2e30 0x7f05a4132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:10.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 10 monclient(hunting): picked mon.noname-d con 0x7f05a40a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:10.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f05a409fb80 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:10.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f05a409ff70 0x7f05a40a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:10.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 10 monclient(hunting): picked mon.noname-f con 0x7f05a409ff70 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:10.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a404b720 con 0x7f05a409fb80 2024-09-15T01:36:10.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:10.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a409e6c0 con 0x7f05a409ff70 2024-09-15T01:36:10.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:10.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f05a40a3200 con 0x7f05a40a2e30 2024-09-15T01:36:10.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:10.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05aab5d640 10 monclient(hunting): authenticate will time out at 986.589455s 2024-09-15T01:36:10.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a88f9640 1 -- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f05a40a2e30 msgr2=0x7f05a4132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3303/0 2024-09-15T01:36:10.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a88f9640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f05a40a2e30 0x7f05a4132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:10.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f05a409ff70 0x7f05a40a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a3fff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f05a409fb80 0x7f05a40a0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43794/0 (socket says 172.21.15.204:43794) 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a3fff640 1 -- 172.21.15.204:0/3634248996 learned_addr learned my addr 172.21.15.204:0/3634248996 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 10 monclient(hunting): get_auth_request con 0x7f05a409ff70 auth_method 0 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 10 monclient(hunting): handle_auth_done global_id 6481 payload 274 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 1 -- 172.21.15.204:0/3634248996 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f05a40a2e30 msgr2=0x7f05a4132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 1 --2- 172.21.15.204:0/3634248996 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f05a40a2e30 0x7f05a4132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:10.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 1 -- 172.21.15.204:0/3634248996 >> v1:172.21.15.204:6789/0 conn(0x7f05a409fb80 legacy=0x7f05a40a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 10 monclient: _finish_hunting 0 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 1 monclient: found mon.noname-f 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a90fa640 1 -- 172.21.15.204:0/3634248996 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a40a1f10 con 0x7f05a409ff70 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a37fe640 1 -- 172.21.15.204:0/3634248996 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216266913 0 0) 0x7f05a404b720 con 0x7f05a409fb80 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.947+0000 7f05a37fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a90fa640 1 --2- 172.21.15.204:0/3634248996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a409ff70 0x7f05a40a1150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f059c009e60 tx=0x7f059c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4e43f039673afa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a37fe640 1 -- 172.21.15.204:0/3634248996 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f059c0079d0 con 0x7f05a409ff70 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a37fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a37fe640 10 monclient: dump: 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:10.961 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:10.962 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:10.962 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:10.962 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:10.962 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:10.962 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:10.962 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:10.962 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:10.962 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:10.962 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:10.962 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:10.962 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:10.962 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a37fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05aab5d640 5 monclient: authenticate success, global_id 6481 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a37fe640 1 -- 172.21.15.204:0/3634248996 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059c02d050 con 0x7f05a409ff70 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05aab5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05aab5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05aab5d640 4 set_mon_vals no callback set 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a37fe640 1 -- 172.21.15.204:0/3634248996 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f059c031400 con 0x7f05a409ff70 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:10.963 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.948+0000 7f05a37fe640 10 monclient: dump: 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:10.964 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:10.965 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:10.965 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:10.965 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:10.965 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:10.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.949+0000 7f05aab5d640 10 monclient: shutdown 2024-09-15T01:36:10.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.949+0000 7f05aab5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:10.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.949+0000 7f05aab5d640 1 -- 172.21.15.204:0/3634248996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a409ff70 msgr2=0x7f05a40a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:10.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.949+0000 7f05aab5d640 1 --2- 172.21.15.204:0/3634248996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a409ff70 0x7f05a40a1150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f059c009e60 tx=0x7f059c009780 comp rx=0 tx=0).stop 2024-09-15T01:36:10.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.949+0000 7f05aab5d640 1 -- 172.21.15.204:0/3634248996 shutdown_connections 2024-09-15T01:36:10.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.949+0000 7f05aab5d640 1 --2- 172.21.15.204:0/3634248996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a409ff70 0x7f05a40a1150 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:10.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.949+0000 7f05aab5d640 1 --2- 172.21.15.204:0/3634248996 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f05a40a2e30 0x7f05a4132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:10.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.949+0000 7f05aab5d640 1 -- 172.21.15.204:0/3634248996 >> 172.21.15.204:0/3634248996 conn(0x7f05a406e860 msgr2=0x7f05a406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.949+0000 7f05aab5d640 1 -- 172.21.15.204:0/3634248996 shutdown_connections 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.949+0000 7f05aab5d640 1 -- 172.21.15.204:0/3634248996 wait complete. 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.949+0000 7f05aab5d640 10 monclient: build_initial_monmap 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.949+0000 7f05aab5d640 10 monclient: monmap: 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:10.966 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:10.967 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:10.967 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:10.967 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:10.967 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:10.967 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:10.967 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:10.967 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:10.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 1 Processor -- start 2024-09-15T01:36:10.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 1 -- start start 2024-09-15T01:36:10.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 10 monclient: init 2024-09-15T01:36:10.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:10.967 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 10 monclient: _add_conns ranks=[2,1,0,8,4,7,3,6,5] 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05a409fb80 0x7f05a4131780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 10 monclient(hunting): picked mon.noname-b con 0x7f05a409fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a409ff70 0x7f05a411e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 10 monclient(hunting): picked mon.noname-f con 0x7f05a409ff70 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05a40a2e30 0x7f05a411e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 10 monclient(hunting): picked mon.noname-a con 0x7f05a40a2e30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a41218d0 con 0x7f05a40a2e30 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a409d650 con 0x7f05a409ff70 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a409ca40 con 0x7f05a409fb80 2024-09-15T01:36:10.968 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:10.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05aab5d640 10 monclient(hunting): authenticate will time out at 986.592778s 2024-09-15T01:36:10.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05a3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a409ff70 0x7f05a411e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:10.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05a88f9640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05a409fb80 msgr2=0x7f05a4131780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:10.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05a3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a409ff70 0x7f05a411e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39954/0 (socket says 172.21.15.204:39954) 2024-09-15T01:36:10.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05a3fff640 1 -- 172.21.15.204:0/55671570 learned_addr learned my addr 172.21.15.204:0/55671570 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:10.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05a3fff640 10 monclient(hunting): get_auth_request con 0x7f05a409ff70 auth_method 0 2024-09-15T01:36:10.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:10.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:10.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:10.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05a88f9640 1 --2- 172.21.15.204:0/55671570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05a409fb80 0x7f05a4131780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:10.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:10.969 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:10.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.950+0000 7f05a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:10.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a3fff640 10 monclient(hunting): handle_auth_done global_id 6499 payload 824 2024-09-15T01:36:10.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a3fff640 1 -- 172.21.15.204:0/55671570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05a409fb80 msgr2=0x7f05a4131780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:10.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a90fa640 1 --2- 172.21.15.204:0/55671570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05a40a2e30 0x7f05a411e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:10.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a3fff640 1 --2- 172.21.15.204:0/55671570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05a409fb80 0x7f05a4131780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:10.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a3fff640 1 -- 172.21.15.204:0/55671570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05a40a2e30 msgr2=0x7f05a411e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:10.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a3fff640 1 --2- 172.21.15.204:0/55671570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05a40a2e30 0x7f05a411e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:10.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:10.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a3fff640 1 monclient: found mon.noname-f 2024-09-15T01:36:10.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:10.970 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a3fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a3fff640 1 -- 172.21.15.204:0/55671570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c009a20 con 0x7f05a409ff70 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a3fff640 1 --2- 172.21.15.204:0/55671570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a409ff70 0x7f05a411e230 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0598001eb0 tx=0x7f0598007ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a0ff9640 1 -- 172.21.15.204:0/55671570 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f05980070f0 con 0x7f05a409ff70 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a0ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a0ff9640 10 monclient: dump: 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:10.971 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a0ff9640 10 monclient: _finish_auth 0 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a0ff9640 1 -- 172.21.15.204:0/55671570 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0598007290 con 0x7f05a409ff70 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05aab5d640 5 monclient: authenticate success, global_id 6499 2024-09-15T01:36:10.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a0ff9640 1 -- 172.21.15.204:0/55671570 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0598012400 con 0x7f05a409ff70 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05aab5d640 10 monclient: _renew_subs 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05aab5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05aab5d640 1 -- 172.21.15.204:0/55671570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a41209e0 con 0x7f05a409ff70 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a0ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a0ff9640 10 monclient: dump: 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:10.973 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05a2ffd640 4 set_mon_vals no callback set 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05aab5d640 10 monclient: _renew_subs 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05aab5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:10.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.951+0000 7f05aab5d640 1 -- 172.21.15.204:0/55671570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a411fe10 con 0x7f05a409ff70 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.952+0000 7f05a0ff9640 1 -- 172.21.15.204:0/55671570 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0598007400 con 0x7f05a409ff70 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.952+0000 7f05aab5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.952+0000 7f05aab5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.952+0000 7f05aab5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.952+0000 7f05aab5d640 1 -- 172.21.15.204:0/55671570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a4121d40 con 0x7f05a409ff70 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.952+0000 7f05a0ff9640 1 --2- 172.21.15.204:0/55671570 >> v2:172.21.15.204:6812/28210 conn(0x7f05900397b0 0x7f059003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.952+0000 7f05a0ff9640 1 -- 172.21.15.204:0/55671570 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 9034+0+0 (secure 0 0 0) 0x7f059804bf90 con 0x7f05a409ff70 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.952+0000 7f05a88f9640 1 --2- 172.21.15.204:0/55671570 >> v2:172.21.15.204:6812/28210 conn(0x7f05900397b0 0x7f059003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.952+0000 7f05a88f9640 10 monclient: get_auth_request con 0x7f05900397b0 auth_method 0 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.953+0000 7f05a88f9640 1 --2- 172.21.15.204:0/55671570 >> v2:172.21.15.204:6812/28210 conn(0x7f05900397b0 0x7f059003bc60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0594006fd0 tx=0x7f0594006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.956+0000 7f05a0ff9640 1 -- 172.21.15.204:0/55671570 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f05980076b0 con 0x7f05a409ff70 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.956+0000 7f05a0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:10.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:10.956+0000 7f05a0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:11.119+0000 7f05aab5d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-09-15T01:36:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:11.119+0000 7f05aab5d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-09-15T01:36:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:11.119+0000 7f05aab5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:11.119+0000 7f05aab5d640 1 -- 172.21.15.204:0/55671570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]} v 0) v1 -- 0x7f05a4070cc0 con 0x7f05a409ff70 2024-09-15T01:36:11.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:11.854+0000 7f05a0ff9640 1 -- 172.21.15.204:0/55671570 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}]=0 v164) v1 ==== 132+0+0 (secure 0 0 0) 0x7f0598019300 con 0x7f05a409ff70 2024-09-15T01:36:11.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:11.854+0000 7f05a0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-09-15T01:36:11.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:11.854+0000 7f05a0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:11.958 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:11.957+0000 7f05aab5d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-09-15T01:36:11.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:11.957+0000 7f05aab5d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-09-15T01:36:11.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:11.957+0000 7f05aab5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:11.959 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:11.957+0000 7f05aab5d640 1 -- 172.21.15.204:0/55671570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]} v 0) v1 -- 0x7f05a409ddb0 con 0x7f05a409ff70 2024-09-15T01:36:12.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.871+0000 7f05a0ff9640 1 -- 172.21.15.204:0/55671570 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}]=0 v165) v1 ==== 132+0+0 (secure 0 0 0) 0x7f059802a0c0 con 0x7f05a409ff70 2024-09-15T01:36:12.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.871+0000 7f05a0ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-09-15T01:36:12.872 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.871+0000 7f05a0ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:12.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 1 -- 172.21.15.204:0/55671570 >> v2:172.21.15.204:6812/28210 conn(0x7f05900397b0 msgr2=0x7f059003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:12.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 1 --2- 172.21.15.204:0/55671570 >> v2:172.21.15.204:6812/28210 conn(0x7f05900397b0 0x7f059003bc60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0594006fd0 tx=0x7f0594006d30 comp rx=0 tx=0).stop 2024-09-15T01:36:12.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 10 monclient: shutdown 2024-09-15T01:36:12.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:12.875 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 1 -- 172.21.15.204:0/55671570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a409ff70 msgr2=0x7f05a411e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:12.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 1 --2- 172.21.15.204:0/55671570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a409ff70 0x7f05a411e230 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0598001eb0 tx=0x7f0598007ce0 comp rx=0 tx=0).stop 2024-09-15T01:36:12.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 1 -- 172.21.15.204:0/55671570 shutdown_connections 2024-09-15T01:36:12.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 1 --2- 172.21.15.204:0/55671570 >> v2:172.21.15.204:6812/28210 conn(0x7f05900397b0 0x7f059003bc60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:12.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 1 --2- 172.21.15.204:0/55671570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05a40a2e30 0x7f05a411e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:12.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 1 --2- 172.21.15.204:0/55671570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05a409ff70 0x7f05a411e230 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:12.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 1 --2- 172.21.15.204:0/55671570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05a409fb80 0x7f05a4131780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:12.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 1 -- 172.21.15.204:0/55671570 >> 172.21.15.204:0/55671570 conn(0x7f05a406e860 msgr2=0x7f05a40a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:12.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 1 -- 172.21.15.204:0/55671570 shutdown_connections 2024-09-15T01:36:12.876 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.873+0000 7f05aab5d640 1 -- 172.21.15.204:0/55671570 wait complete. 2024-09-15T01:36:12.885 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T01:36:12.885 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 'osd\.0' 2024-09-15T01:36:12.885 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep globster 2024-09-15T01:36:12.886 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep '7\.' 2024-09-15T01:36:12.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.959+0000 7f39a18a8640 10 monclient: get_monmap_and_config 2024-09-15T01:36:12.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.959+0000 7f39a18a8640 10 monclient: build_initial_monmap 2024-09-15T01:36:12.961 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.959+0000 7f39a18a8640 10 monclient: monmap: 2024-09-15T01:36:12.961 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:12.961 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:12.961 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:12.959833+0000 2024-09-15T01:36:12.961 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:12.959833+0000 2024-09-15T01:36:12.961 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:12.961 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:12.961 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:12.961 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:12.961 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:12.961 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:12.962 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:12.962 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:12.962 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:12.962 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:12.962 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:12.962 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:12.962 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:12.962 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:12.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 1 Processor -- start 2024-09-15T01:36:12.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 1 -- start start 2024-09-15T01:36:12.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 10 monclient: init 2024-09-15T01:36:12.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:12.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 10 monclient: _add_conns ranks=[8,7,3,6,10,0,9,5,1,2,4] 2024-09-15T01:36:12.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f399c0a2fc0 0x7f399c0a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:12.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 10 monclient(hunting): picked mon.noname-e con 0x7f399c0a2fc0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:12.971 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f399c0a14b0 0x7f399c09fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 10 monclient(hunting): picked mon.noname-i con 0x7f399c0a14b0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 0x7f399c132af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 10 monclient(hunting): picked mon.noname-g con 0x7f399c0a02a0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0a3390 con 0x7f399c0a02a0 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0a1c20 con 0x7f399c0a14b0 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f399c09e270 con 0x7f399c0a2fc0 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f39a18a8640 10 monclient(hunting): authenticate will time out at 988.602625s 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f3993fff640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f399c0a14b0 msgr2=0x7f399c09fd60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:36:12.972 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f3993fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f399c0a14b0 0x7f399c09fd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:12.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f399b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 0x7f399c132af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:12.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f399b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 0x7f399c132af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:33470/0 (socket says 172.21.15.204:33470) 2024-09-15T01:36:12.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f399b7fe640 1 -- 172.21.15.204:0/2513842886 learned_addr learned my addr 172.21.15.204:0/2513842886 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:12.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f399b7fe640 10 monclient(hunting): get_auth_request con 0x7f399c0a02a0 auth_method 0 2024-09-15T01:36:12.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f399b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:12.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f399b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:12.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f399b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:12.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f399b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:12.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f399b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:12.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.960+0000 7f399b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:12.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399affd640 1 --2- 172.21.15.204:0/2513842886 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f399c0a2fc0 0x7f399c0a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:12.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399b7fe640 10 monclient(hunting): handle_auth_done global_id 14925 payload 274 2024-09-15T01:36:12.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399b7fe640 1 -- 172.21.15.204:0/2513842886 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f399c0a2fc0 msgr2=0x7f399c0a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:12.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399b7fe640 1 --2- 172.21.15.204:0/2513842886 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f399c0a2fc0 0x7f399c0a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:12.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399b7fe640 1 -- 172.21.15.204:0/2513842886 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f399c0a14b0 msgr2=0x7f399c09fd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:12.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399b7fe640 1 --2- 172.21.15.204:0/2513842886 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f399c0a14b0 0x7f399c09fd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:12.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:12.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399b7fe640 1 monclient: found mon.noname-g 2024-09-15T01:36:12.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:12.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399b7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:12.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399b7fe640 1 -- 172.21.15.204:0/2513842886 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c04b720 con 0x7f399c0a02a0 2024-09-15T01:36:12.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399affd640 10 monclient: get_auth_request con 0x7f399c0a2fc0 auth_method 0 2024-09-15T01:36:12.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399affd640 1 --2- 172.21.15.204:0/2513842886 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f399c0a2fc0 0x7f399c0a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:36:12.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399b7fe640 1 --2- 172.21.15.204:0/2513842886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 0x7f399c132af0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3994009f50 tx=0x7f3994009840 comp rx=0 tx=0).ready entity=mon.3 client_cookie=a8f581f17878decf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399a7fc640 1 -- 172.21.15.204:0/2513842886 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3994007880 con 0x7f399c0a02a0 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399a7fc640 10 monclient: dump: 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:12.975 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f39a18a8640 5 monclient: authenticate success, global_id 14925 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399a7fc640 1 -- 172.21.15.204:0/2513842886 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399402d050 con 0x7f399c0a02a0 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f39a18a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:12.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:12.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399a7fc640 1 -- 172.21.15.204:0/2513842886 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3994030360 con 0x7f399c0a02a0 2024-09-15T01:36:12.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f39a18a8640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:12.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f39a18a8640 4 set_mon_vals no callback set 2024-09-15T01:36:12.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:12.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:12.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.961+0000 7f399a7fc640 10 monclient: dump: 2024-09-15T01:36:12.977 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:12.977 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:12.977 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:12.977 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:12.977 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:12.977 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.962+0000 7f39a18a8640 10 monclient: shutdown 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.962+0000 7f39a18a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.962+0000 7f39a18a8640 1 -- 172.21.15.204:0/2513842886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 msgr2=0x7f399c132af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.962+0000 7f39a18a8640 1 --2- 172.21.15.204:0/2513842886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 0x7f399c132af0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3994009f50 tx=0x7f3994009840 comp rx=0 tx=0).stop 2024-09-15T01:36:12.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.962+0000 7f39a18a8640 1 -- 172.21.15.204:0/2513842886 shutdown_connections 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.962+0000 7f39a18a8640 1 --2- 172.21.15.204:0/2513842886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 0x7f399c132af0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.962+0000 7f39a18a8640 1 --2- 172.21.15.204:0/2513842886 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f399c0a14b0 0x7f399c09fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.962+0000 7f39a18a8640 1 --2- 172.21.15.204:0/2513842886 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f399c0a2fc0 0x7f399c0a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.962+0000 7f39a18a8640 1 -- 172.21.15.204:0/2513842886 >> 172.21.15.204:0/2513842886 conn(0x7f399c0a62f0 msgr2=0x7f399c0a66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.962+0000 7f39a18a8640 1 -- 172.21.15.204:0/2513842886 shutdown_connections 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.962+0000 7f39a18a8640 1 -- 172.21.15.204:0/2513842886 wait complete. 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient: build_initial_monmap 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient: monmap: 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:12.979 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 1 Processor -- start 2024-09-15T01:36:12.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 1 -- start start 2024-09-15T01:36:12.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient: init 2024-09-15T01:36:12.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:12.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient: _add_conns ranks=[3,5,1,6,8,7,2,0,4] 2024-09-15T01:36:12.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 0x7f399c132830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:12.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient(hunting): picked mon.noname-g con 0x7f399c0a02a0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:12.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f399c0a14b0 0x7f399c11f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:12.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient(hunting): picked mon.noname-h con 0x7f399c0a14b0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:12.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f399c0a2fc0 0x7f399c11f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:12.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient(hunting): picked mon.noname-f con 0x7f399c0a2fc0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:12.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:12.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c09d6d0 con 0x7f399c0a2fc0 2024-09-15T01:36:12.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c099e50 con 0x7f399c0a02a0 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f399c121630 con 0x7f399c0a14b0 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f39a18a8640 10 monclient(hunting): authenticate will time out at 988.605825s 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f399affd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 0x7f399c132830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f399affd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 0x7f399c132830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:33482/0 (socket says 172.21.15.204:33482) 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f399affd640 1 -- 172.21.15.204:0/1147797655 learned_addr learned my addr 172.21.15.204:0/1147797655 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f399affd640 10 monclient(hunting): get_auth_request con 0x7f399c0a02a0 auth_method 0 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f399affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f399affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f399b7fe640 1 --2- 172.21.15.204:0/1147797655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f399c0a2fc0 0x7f399c11f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:12.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f399affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f399b7fe640 10 monclient(hunting): get_auth_request con 0x7f399c0a2fc0 auth_method 0 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f399b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f399b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f399b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.963+0000 7f3993fff640 1 --2- 172.21.15.204:0/1147797655 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f399c0a14b0 0x7f399c11f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3993fff640 10 monclient(hunting): get_auth_request con 0x7f399c0a14b0 auth_method 0 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3993fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3993fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3993fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:12.983 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3993fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3993fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3993fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 10 monclient(hunting): handle_auth_done global_id 14934 payload 824 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 1 -- 172.21.15.204:0/1147797655 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f399c0a14b0 msgr2=0x7f399c11f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 1 --2- 172.21.15.204:0/1147797655 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f399c0a14b0 0x7f399c11f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 1 -- 172.21.15.204:0/1147797655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f399c0a2fc0 msgr2=0x7f399c11f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 1 --2- 172.21.15.204:0/1147797655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f399c0a2fc0 0x7f399c11f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 10 monclient: _finish_hunting 0 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 1 monclient: found mon.noname-g 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:12.984 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 1 -- 172.21.15.204:0/1147797655 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3994009ae0 con 0x7f399c0a02a0 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f399affd640 1 --2- 172.21.15.204:0/1147797655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 0x7f399c132830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f398c00c880 tx=0x7f398c008040 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3992ffd640 1 -- 172.21.15.204:0/1147797655 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f398c00ce10 con 0x7f399c0a02a0 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3992ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3992ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3992ffd640 10 monclient: dump: 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:12.985 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3992ffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3992ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3992ffd640 1 -- 172.21.15.204:0/1147797655 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c007d10 con 0x7f399c0a02a0 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f39a18a8640 5 monclient: authenticate success, global_id 14934 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3992ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:12.986 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f39a18a8640 10 monclient: _renew_subs 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3992ffd640 1 -- 172.21.15.204:0/1147797655 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f398c007e80 con 0x7f399c0a02a0 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f39a18a8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f39a18a8640 1 -- 172.21.15.204:0/1147797655 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c120e50 con 0x7f399c0a02a0 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3992ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3992ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3999ffb640 4 set_mon_vals no callback set 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f3992ffd640 10 monclient: dump: 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:12.987 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:12.988 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:12.988 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:12.988 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:12.988 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:12.988 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:12.988 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:12.988 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:12.988 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:12.988 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:12.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f39a18a8640 10 monclient: _renew_subs 2024-09-15T01:36:12.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f39a18a8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:12.988 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.964+0000 7f39a18a8640 1 -- 172.21.15.204:0/1147797655 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c122620 con 0x7f399c0a02a0 2024-09-15T01:36:12.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.965+0000 7f3992ffd640 1 -- 172.21.15.204:0/1147797655 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f398c007890 con 0x7f399c0a02a0 2024-09-15T01:36:12.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.965+0000 7f39a18a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:12.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.965+0000 7f39a18a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:12.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.965+0000 7f39a18a8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:12.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.965+0000 7f3992ffd640 1 --2- 172.21.15.204:0/1147797655 >> v2:172.21.15.204:6812/28210 conn(0x7f397c0397b0 0x7f397c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:12.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.965+0000 7f39a18a8640 1 -- 172.21.15.204:0/1147797655 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399c122cf0 con 0x7f399c0a02a0 2024-09-15T01:36:12.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.965+0000 7f3992ffd640 1 -- 172.21.15.204:0/1147797655 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 9034+0+0 (secure 0 0 0) 0x7f398c04c5c0 con 0x7f399c0a02a0 2024-09-15T01:36:12.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.965+0000 7f3993fff640 1 --2- 172.21.15.204:0/1147797655 >> v2:172.21.15.204:6812/28210 conn(0x7f397c0397b0 0x7f397c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:12.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.965+0000 7f3993fff640 10 monclient: get_auth_request con 0x7f397c0397b0 auth_method 0 2024-09-15T01:36:12.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.966+0000 7f3993fff640 1 --2- 172.21.15.204:0/1147797655 >> v2:172.21.15.204:6812/28210 conn(0x7f397c0397b0 0x7f397c03bc60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3984006fd0 tx=0x7f39840090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:12.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.970+0000 7f3992ffd640 1 -- 172.21.15.204:0/1147797655 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f398c01e070 con 0x7f399c0a02a0 2024-09-15T01:36:12.989 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.970+0000 7f3992ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:12.990 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:12.970+0000 7f3992ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:13.124 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-15T01:36:13.125 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-15T01:36:13.125 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-15T01:36:13.125 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-15T01:36:13.127 INFO:tasks.ceph.mon.i:Started 2024-09-15T01:36:13.128 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T01:36:13.128 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T01:36:13.128 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T01:36:13.128 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T01:36:13.131 INFO:tasks.ceph.mon.b:Started 2024-09-15T01:36:13.131 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-15T01:36:13.131 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.d 2024-09-15T01:36:13.131 INFO:tasks.ceph.mon.d:Restarting daemon 2024-09-15T01:36:13.131 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-09-15T01:36:13.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.130+0000 7f39a18a8640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.130+0000 7f39a18a8640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.130+0000 7f39a18a8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:13.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.130+0000 7f39a18a8640 1 -- 172.21.15.204:0/1147797655 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f399c071100 con 0x7f399c0a02a0 2024-09-15T01:36:13.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.130+0000 7f3992ffd640 1 -- 172.21.15.204:0/1147797655 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v165) v1 ==== 83+0+2320 (secure 0 0 0) 0x7f398c007b40 con 0x7f399c0a02a0 2024-09-15T01:36:13.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.130+0000 7f3992ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.130+0000 7f3992ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:13.134 INFO:tasks.ceph.mon.d:Started 2024-09-15T01:36:13.134 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:36:13.134 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:36:13.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.235+0000 7f39a18a8640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.235+0000 7f39a18a8640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.235+0000 7f39a18a8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:13.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.235+0000 7f39a18a8640 1 -- 172.21.15.204:0/1147797655 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f399c0008d0 con 0x7f399c0a02a0 2024-09-15T01:36:13.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.235+0000 7f3992ffd640 1 -- 172.21.15.204:0/1147797655 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v165) v1 ==== 83+0+2320 (secure 0 0 0) 0x7f398c020300 con 0x7f399c0a02a0 2024-09-15T01:36:13.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.235+0000 7f3992ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.235+0000 7f3992ffd640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:13.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 1 -- 172.21.15.204:0/1147797655 >> v2:172.21.15.204:6812/28210 conn(0x7f397c0397b0 msgr2=0x7f397c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:13.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 1 --2- 172.21.15.204:0/1147797655 >> v2:172.21.15.204:6812/28210 conn(0x7f397c0397b0 0x7f397c03bc60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3984006fd0 tx=0x7f39840090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 10 monclient: shutdown 2024-09-15T01:36:13.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:13.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 1 -- 172.21.15.204:0/1147797655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 msgr2=0x7f399c132830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:13.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 1 --2- 172.21.15.204:0/1147797655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 0x7f399c132830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f398c00c880 tx=0x7f398c008040 comp rx=0 tx=0).stop 2024-09-15T01:36:13.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 1 -- 172.21.15.204:0/1147797655 shutdown_connections 2024-09-15T01:36:13.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 1 --2- 172.21.15.204:0/1147797655 >> v2:172.21.15.204:6812/28210 conn(0x7f397c0397b0 0x7f397c03bc60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 1 --2- 172.21.15.204:0/1147797655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f399c0a2fc0 0x7f399c11f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 1 --2- 172.21.15.204:0/1147797655 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f399c0a14b0 0x7f399c11f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 1 --2- 172.21.15.204:0/1147797655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f399c0a02a0 0x7f399c132830 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 1 -- 172.21.15.204:0/1147797655 >> 172.21.15.204:0/1147797655 conn(0x7f399c0a62f0 msgr2=0x7f399c09a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:13.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 1 -- 172.21.15.204:0/1147797655 shutdown_connections 2024-09-15T01:36:13.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.237+0000 7f39a18a8640 1 -- 172.21.15.204:0/1147797655 wait complete. 2024-09-15T01:36:13.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.240+0000 7f5945100640 10 monclient: get_monmap_and_config 2024-09-15T01:36:13.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.240+0000 7f5945100640 10 monclient: build_initial_monmap 2024-09-15T01:36:13.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.240+0000 7f5945100640 10 monclient: monmap: 2024-09-15T01:36:13.242 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:13.242 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:13.241600+0000 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:13.241600+0000 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:13.243 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:13.244 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:13.244 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:13.244 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:13.250 INFO:tasks.workunit.client.0.smithi204.stdout: 0 globster 0.08719 7.50000 8.50000 6.50000 osd.0 2024-09-15T01:36:13.250 INFO:tasks.workunit.client.0.smithi204.stdout: 0 globster 0.08719 7.50000 8.50000 6.50000 osd.0 2024-09-15T01:36:13.251 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T01:36:13.251 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 'osd\.0' 2024-09-15T01:36:13.251 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep globster 2024-09-15T01:36:13.251 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep '8\.' 2024-09-15T01:36:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.241+0000 7f5945100640 1 Processor -- start 2024-09-15T01:36:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.241+0000 7f5945100640 1 -- start start 2024-09-15T01:36:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.241+0000 7f5945100640 10 monclient: init 2024-09-15T01:36:13.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.241+0000 7f5945100640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.241+0000 7f5945100640 10 monclient: _add_conns ranks=[9,2,6,10,8,5,3,0,7,4,1] 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.241+0000 7f5945100640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f59400678d0 0x7f594009e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.241+0000 7f5945100640 10 monclient(hunting): picked mon.noname-a con 0x7f59400678d0 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.241+0000 7f5945100640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594006ea30 0x7f594009ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f5945100640 10 monclient(hunting): picked mon.noname-b con 0x7f594006ea30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f5945100640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f594009f180 0x7f59401306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f5945100640 10 monclient(hunting): picked mon.noname-d con 0x7f594009f180 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f5945100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f5945100640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f594009f5c0 con 0x7f59400678d0 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f5945100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f5945100640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594006d5b0 con 0x7f594006ea30 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f5945100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f5945100640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5940067ca0 con 0x7f594009f180 2024-09-15T01:36:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f59400678d0 0x7f594009e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f5945100640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f5945100640 10 monclient(hunting): authenticate will time out at 977.992165s 2024-09-15T01:36:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f59400678d0 0x7f594009e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39288/0 (socket says 172.21.15.178:39288) 2024-09-15T01:36:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 1 -- 172.21.15.178:0/1644807849 learned_addr learned my addr 172.21.15.178:0/1644807849 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 10 monclient(hunting): get_auth_request con 0x7f59400678d0 auth_method 0 2024-09-15T01:36:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593e59c640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594006ea30 0x7f594009ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593f59e640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f594009f180 0x7f59401306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593e59c640 10 monclient(hunting): get_auth_request con 0x7f594006ea30 auth_method 0 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593f59e640 10 monclient(hunting): get_auth_request con 0x7f594009f180 auth_method 0 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593f59e640 1 -- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f594009f180 msgr2=0x7f59401306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593f59e640 1 -- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f594009f180 msgr2=0x7f59401306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:13.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593f59e640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f594009f180 0x7f59401306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593e59c640 1 -- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594006ea30 msgr2=0x7f594009ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:36:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593f59e640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f594009f180 0x7f59401306b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593e59c640 1 -- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594006ea30 msgr2=0x7f594009ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593e59c640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594006ea30 0x7f594009ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 10 monclient(hunting): handle_auth_done global_id 6057 payload 274 2024-09-15T01:36:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593e59c640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594006ea30 0x7f594009ec40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 1 -- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f594009f180 msgr2=0x7f59401306b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f594009f180 0x7f59401306b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 1 -- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594006ea30 msgr2=0x7f594009ec40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594006ea30 0x7f594009ec40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 1 monclient: found mon.noname-a 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.242+0000 7f593ed9d640 1 -- 172.21.15.178:0/1644807849 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594004bf60 con 0x7f59400678d0 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593ed9d640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59400678d0 0x7f594009e700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5930009910 tx=0x7f59300091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde3c489916c8fed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593dd9b640 1 -- 172.21.15.178:0/1644807849 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f593002ad20 con 0x7f59400678d0 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593dd9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593dd9b640 10 monclient: dump: 2024-09-15T01:36:13.258 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593dd9b640 10 monclient: _finish_auth 0 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f5945100640 5 monclient: authenticate success, global_id 6057 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f5945100640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593dd9b640 1 -- 172.21.15.178:0/1644807849 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593002ae80 con 0x7f59400678d0 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593dd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f5945100640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f5945100640 4 set_mon_vals no callback set 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593dd9b640 1 -- 172.21.15.178:0/1644807849 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f593002ad20 con 0x7f59400678d0 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:13.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593dd9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.243+0000 7f593dd9b640 10 monclient: dump: 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:13.261 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 10 monclient: shutdown 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 1 -- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59400678d0 msgr2=0x7f594009e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59400678d0 0x7f594009e700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5930009910 tx=0x7f59300091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 1 -- 172.21.15.178:0/1644807849 shutdown_connections 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f594009f180 0x7f59401306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594006ea30 0x7f594009ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 1 --2- 172.21.15.178:0/1644807849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59400678d0 0x7f594009e700 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 1 -- 172.21.15.178:0/1644807849 >> 172.21.15.178:0/1644807849 conn(0x7f59400a4390 msgr2=0x7f5940067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:13.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 1 -- 172.21.15.178:0/1644807849 shutdown_connections 2024-09-15T01:36:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 1 -- 172.21.15.178:0/1644807849 wait complete. 2024-09-15T01:36:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 10 monclient: build_initial_monmap 2024-09-15T01:36:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.244+0000 7f5945100640 10 monclient: monmap: 2024-09-15T01:36:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:13.263 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:13.264 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 1 Processor -- start 2024-09-15T01:36:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 1 -- start start 2024-09-15T01:36:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 10 monclient: init 2024-09-15T01:36:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:13.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 10 monclient: _add_conns ranks=[4,7,0,1,8,2,3,5,6] 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f59400678d0 0x7f594013c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 10 monclient(hunting): picked mon.noname-c con 0x7f59400678d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f594006ea30 0x7f594013b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 10 monclient(hunting): picked mon.noname-i con 0x7f594006ea30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f594009f180 0x7f594013f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 10 monclient(hunting): picked mon.noname-a con 0x7f594009f180 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5940121100 con 0x7f594009f180 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5940120b40 con 0x7f59400678d0 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f594011e650 con 0x7f594006ea30 2024-09-15T01:36:13.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f5945100640 10 monclient(hunting): authenticate will time out at 977.995646s 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593ed9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f59400678d0 0x7f594013c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593ed9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f59400678d0 0x7f594013c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:34152/0 (socket says 172.21.15.178:34152) 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593ed9d640 1 -- 172.21.15.178:0/769840106 learned_addr learned my addr 172.21.15.178:0/769840106 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593ed9d640 10 monclient(hunting): get_auth_request con 0x7f59400678d0 auth_method 0 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593f59e640 1 --2- 172.21.15.178:0/769840106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f594009f180 0x7f594013f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593f59e640 10 monclient(hunting): get_auth_request con 0x7f594009f180 auth_method 0 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593e59c640 1 --2- 172.21.15.178:0/769840106 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f594006ea30 0x7f594013b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593e59c640 10 monclient(hunting): get_auth_request con 0x7f594006ea30 auth_method 0 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.245+0000 7f593ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593e59c640 1 -- 172.21.15.178:0/769840106 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f594006ea30 msgr2=0x7f594013b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:36:13.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593e59c640 1 -- 172.21.15.178:0/769840106 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f594006ea30 msgr2=0x7f594013b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593e59c640 1 --2- 172.21.15.178:0/769840106 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f594006ea30 0x7f594013b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593f59e640 10 monclient(hunting): handle_auth_done global_id 6066 payload 824 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593e59c640 1 --2- 172.21.15.178:0/769840106 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f594006ea30 0x7f594013b7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593f59e640 1 -- 172.21.15.178:0/769840106 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f594006ea30 msgr2=0x7f594013b7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593f59e640 1 --2- 172.21.15.178:0/769840106 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f594006ea30 0x7f594013b7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593f59e640 1 -- 172.21.15.178:0/769840106 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f59400678d0 msgr2=0x7f594013c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593f59e640 1 --2- 172.21.15.178:0/769840106 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f59400678d0 0x7f594013c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593f59e640 10 monclient: _finish_hunting 0 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593f59e640 1 monclient: found mon.noname-a 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593f59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593f59e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593f59e640 1 -- 172.21.15.178:0/769840106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5930009450 con 0x7f594009f180 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593f59e640 1 --2- 172.21.15.178:0/769840106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f594009f180 0x7f594013f680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5938009fd0 tx=0x7f5938009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:13.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5926ffd640 1 -- 172.21.15.178:0/769840106 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5938015070 con 0x7f594009f180 2024-09-15T01:36:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5926ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5926ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:36:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5926ffd640 10 monclient: dump: 2024-09-15T01:36:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:13.269 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5926ffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5926ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5926ffd640 1 -- 172.21.15.178:0/769840106 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5938007e80 con 0x7f594009f180 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5945100640 5 monclient: authenticate success, global_id 6066 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5926ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5945100640 10 monclient: _renew_subs 2024-09-15T01:36:13.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5926ffd640 1 -- 172.21.15.178:0/769840106 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f59380102e0 con 0x7f594009f180 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5945100640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5945100640 1 -- 172.21.15.178:0/769840106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5940120860 con 0x7f594009f180 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5926ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5926ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5926ffd640 10 monclient: dump: 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:13.271 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f593d59a640 4 set_mon_vals no callback set 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5945100640 10 monclient: _renew_subs 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5945100640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.246+0000 7f5945100640 1 -- 172.21.15.178:0/769840106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594013bd00 con 0x7f594009f180 2024-09-15T01:36:13.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.247+0000 7f5926ffd640 1 -- 172.21.15.178:0/769840106 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f593800e450 con 0x7f594009f180 2024-09-15T01:36:13.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.247+0000 7f5945100640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:13.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.247+0000 7f5945100640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:13.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.248+0000 7f5945100640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:13.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.248+0000 7f5945100640 1 -- 172.21.15.178:0/769840106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594005ed70 con 0x7f594009f180 2024-09-15T01:36:13.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.248+0000 7f5926ffd640 1 --2- 172.21.15.178:0/769840106 >> v2:172.21.15.204:6812/28210 conn(0x7f592c0397b0 0x7f592c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.248+0000 7f5926ffd640 1 -- 172.21.15.178:0/769840106 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 9034+0+0 (secure 0 0 0) 0x7f59380507e0 con 0x7f594009f180 2024-09-15T01:36:13.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.248+0000 7f593ed9d640 1 --2- 172.21.15.178:0/769840106 >> v2:172.21.15.204:6812/28210 conn(0x7f592c0397b0 0x7f592c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.248+0000 7f593ed9d640 10 monclient: get_auth_request con 0x7f592c0397b0 auth_method 0 2024-09-15T01:36:13.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.248+0000 7f593ed9d640 1 --2- 172.21.15.178:0/769840106 >> v2:172.21.15.204:6812/28210 conn(0x7f592c0397b0 0x7f592c03bc60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5930000c00 tx=0x7f5930032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:13.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.252+0000 7f5926ffd640 1 -- 172.21.15.178:0/769840106 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5938017af0 con 0x7f594009f180 2024-09-15T01:36:13.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.252+0000 7f5926ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:13.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.252+0000 7f5926ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:13.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.320+0000 7f034e15d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:13.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.320+0000 7f034e15d640 10 monclient: build_initial_monmap 2024-09-15T01:36:13.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.320+0000 7f034e15d640 10 monclient: monmap: 2024-09-15T01:36:13.321 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:13.320688+0000 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:13.320688+0000 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:13.323 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:13.331 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.320+0000 7f034e15d640 1 Processor -- start 2024-09-15T01:36:13.331 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.320+0000 7f034e15d640 1 -- start start 2024-09-15T01:36:13.331 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.320+0000 7f034e15d640 10 monclient: init 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 10 monclient: _add_conns ranks=[6,4,10,9,2,1,0,7,3,8,5] 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03480a4680 0x7f034809e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 10 monclient(hunting): picked mon.noname-d con 0x7f03480a4680 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f034806eea0 0x7f034809ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 10 monclient(hunting): picked mon.noname-c con 0x7f034806eea0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f034809f2d0 0x7f03481308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 10 monclient(hunting): picked mon.noname-f con 0x7f034809f2d0 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034809f6e0 con 0x7f034809f2d0 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03480a4a50 con 0x7f034806eea0 2024-09-15T01:36:13.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f034806dc70 con 0x7f03480a4680 2024-09-15T01:36:13.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:13.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f034e15d640 10 monclient(hunting): authenticate will time out at 988.963468s 2024-09-15T01:36:13.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f034809f2d0 0x7f03481308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f034809f2d0 0x7f03481308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39966/0 (socket says 172.21.15.204:39966) 2024-09-15T01:36:13.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 1 -- 172.21.15.204:0/2667515144 learned_addr learned my addr 172.21.15.204:0/2667515144 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:13.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 10 monclient(hunting): get_auth_request con 0x7f034809f2d0 auth_method 0 2024-09-15T01:36:13.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:13.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:13.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0346ffd640 1 --2- 172.21.15.204:0/2667515144 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f034806eea0 0x7f034809ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f03477fe640 1 --2- 172.21.15.204:0/2667515144 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03480a4680 0x7f034809e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:13.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0346ffd640 10 monclient(hunting): get_auth_request con 0x7f034806eea0 auth_method 0 2024-09-15T01:36:13.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0346ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0346ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0346ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f03477fe640 10 monclient(hunting): get_auth_request con 0x7f03480a4680 auth_method 0 2024-09-15T01:36:13.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f03477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f03477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f03477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0346ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0346ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f03477fe640 1 -- 172.21.15.204:0/2667515144 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03480a4680 msgr2=0x7f034809e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f03477fe640 1 -- 172.21.15.204:0/2667515144 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03480a4680 msgr2=0x7f034809e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0346ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 10 monclient(hunting): handle_auth_done global_id 6517 payload 274 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f03477fe640 1 --2- 172.21.15.204:0/2667515144 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03480a4680 0x7f034809e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 1 -- 172.21.15.204:0/2667515144 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03480a4680 msgr2=0x7f034809e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f03477fe640 1 --2- 172.21.15.204:0/2667515144 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03480a4680 0x7f034809e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 1 --2- 172.21.15.204:0/2667515144 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03480a4680 0x7f034809e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 1 -- 172.21.15.204:0/2667515144 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f034806eea0 msgr2=0x7f034809ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:13.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 1 --2- 172.21.15.204:0/2667515144 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f034806eea0 0x7f034809ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 1 monclient: found mon.noname-f 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.321+0000 7f0347fff640 1 -- 172.21.15.204:0/2667515144 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034804b720 con 0x7f034809f2d0 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f0347fff640 1 --2- 172.21.15.204:0/2667515144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f034809f2d0 0x7f03481308a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0340009f50 tx=0x7f0340009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d4b9b2a8842efab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f03467fc640 1 -- 172.21.15.204:0/2667515144 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0340007880 con 0x7f034809f2d0 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f03467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f03467fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f03467fc640 10 monclient: dump: 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.336 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:13.337 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f03467fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f03467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f034e15d640 5 monclient: authenticate success, global_id 6517 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f03467fc640 1 -- 172.21.15.204:0/2667515144 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034002d050 con 0x7f034809f2d0 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f034e15d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f03467fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f034e15d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f03467fc640 1 -- 172.21.15.204:0/2667515144 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0340030360 con 0x7f034809f2d0 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f034e15d640 4 set_mon_vals no callback set 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f03467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f03467fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.322+0000 7f03467fc640 10 monclient: dump: 2024-09-15T01:36:13.338 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:13.339 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 10 monclient: shutdown 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 1 -- 172.21.15.204:0/2667515144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f034809f2d0 msgr2=0x7f03481308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 1 --2- 172.21.15.204:0/2667515144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f034809f2d0 0x7f03481308a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0340009f50 tx=0x7f0340009840 comp rx=0 tx=0).stop 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 1 -- 172.21.15.204:0/2667515144 shutdown_connections 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 1 --2- 172.21.15.204:0/2667515144 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f034806eea0 0x7f034809ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 1 --2- 172.21.15.204:0/2667515144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f034809f2d0 0x7f03481308a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 1 --2- 172.21.15.204:0/2667515144 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f03480a4680 0x7f034809e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 1 -- 172.21.15.204:0/2667515144 >> 172.21.15.204:0/2667515144 conn(0x7f0348067740 msgr2=0x7f03480a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 1 -- 172.21.15.204:0/2667515144 shutdown_connections 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 1 -- 172.21.15.204:0/2667515144 wait complete. 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 10 monclient: build_initial_monmap 2024-09-15T01:36:13.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.323+0000 7f034e15d640 10 monclient: monmap: 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:13.341 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 1 Processor -- start 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 1 -- start start 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 10 monclient: init 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 10 monclient: _add_conns ranks=[3,7,4,2,5,8,1,6,0] 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f034806eea0 0x7f034813c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 10 monclient(hunting): picked mon.noname-g con 0x7f034806eea0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f034809f2d0 0x7f034813b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 10 monclient(hunting): picked mon.noname-i con 0x7f034809f2d0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f03480a4680 0x7f034813f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.342 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 10 monclient(hunting): picked mon.noname-c con 0x7f03480a4680 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:13.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0348120ff0 con 0x7f034806eea0 2024-09-15T01:36:13.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0348120a30 con 0x7f03480a4680 2024-09-15T01:36:13.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f034811e540 con 0x7f034809f2d0 2024-09-15T01:36:13.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:13.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f034e15d640 10 monclient(hunting): authenticate will time out at 988.966905s 2024-09-15T01:36:13.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f03477fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f034806eea0 0x7f034813c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.324+0000 7f0346ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f034809f2d0 0x7f034813b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f034806eea0 0x7f034813c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:33496/0 (socket says 172.21.15.204:33496) 2024-09-15T01:36:13.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 1 -- 172.21.15.204:0/566454533 learned_addr learned my addr 172.21.15.204:0/566454533 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 10 monclient(hunting): get_auth_request con 0x7f034806eea0 auth_method 0 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0346ffd640 10 monclient(hunting): get_auth_request con 0x7f034809f2d0 auth_method 0 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0346ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0346ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0346ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0347fff640 1 --2- 172.21.15.204:0/566454533 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f03480a4680 0x7f034813f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:13.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0346ffd640 1 -- 172.21.15.204:0/566454533 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f034809f2d0 msgr2=0x7f034813b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:36:13.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0346ffd640 1 -- 172.21.15.204:0/566454533 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f034809f2d0 msgr2=0x7f034813b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:36:13.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0347fff640 10 monclient(hunting): get_auth_request con 0x7f03480a4680 auth_method 0 2024-09-15T01:36:13.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0347fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0347fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0347fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0346ffd640 1 --2- 172.21.15.204:0/566454533 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f034809f2d0 0x7f034813b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:13.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0346ffd640 1 --2- 172.21.15.204:0/566454533 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f034809f2d0 0x7f034813b6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:13.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0347fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:13.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0347fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:13.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0347fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:13.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 10 monclient(hunting): handle_auth_done global_id 14943 payload 824 2024-09-15T01:36:13.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 1 -- 172.21.15.204:0/566454533 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f034809f2d0 msgr2=0x7f034813b6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:13.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 1 --2- 172.21.15.204:0/566454533 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f034809f2d0 0x7f034813b6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 1 -- 172.21.15.204:0/566454533 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f03480a4680 msgr2=0x7f034813f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:13.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 1 --2- 172.21.15.204:0/566454533 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f03480a4680 0x7f034813f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:13.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 1 monclient: found mon.noname-g 2024-09-15T01:36:13.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:13.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:13.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 1 -- 172.21.15.204:0/566454533 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0340009ae0 con 0x7f034806eea0 2024-09-15T01:36:13.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f03477fe640 1 --2- 172.21.15.204:0/566454533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f034806eea0 0x7f034813c000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f033800d890 tx=0x7f0338008040 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:13.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f033f7fe640 1 -- 172.21.15.204:0/566454533 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f033800de20 con 0x7f034806eea0 2024-09-15T01:36:13.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f033f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:13.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f033f7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f033f7fe640 10 monclient: dump: 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:13.347 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:13.348 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:13.348 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:13.348 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:13.348 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:13.348 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f033f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:13.348 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f033f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:13.348 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f033f7fe640 1 -- 172.21.15.204:0/566454533 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033800bd50 con 0x7f034806eea0 2024-09-15T01:36:13.348 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f034e15d640 5 monclient: authenticate success, global_id 14943 2024-09-15T01:36:13.348 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f034e15d640 10 monclient: _renew_subs 2024-09-15T01:36:13.348 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f034e15d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f034e15d640 1 -- 172.21.15.204:0/566454533 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0348120750 con 0x7f034806eea0 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f033f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f033f7fe640 1 -- 172.21.15.204:0/566454533 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03380148b0 con 0x7f034806eea0 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f033f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f033f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f033f7fe640 10 monclient: dump: 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:13.349 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f034e15d640 10 monclient: _renew_subs 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f0345ffb640 4 set_mon_vals no callback set 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f034e15d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.325+0000 7f034e15d640 1 -- 172.21.15.204:0/566454533 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034813bbf0 con 0x7f034806eea0 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.326+0000 7f033f7fe640 1 -- 172.21.15.204:0/566454533 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0338014430 con 0x7f034806eea0 2024-09-15T01:36:13.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.326+0000 7f034e15d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:13.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.326+0000 7f034e15d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:13.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.326+0000 7f034e15d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:13.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.326+0000 7f034e15d640 1 -- 172.21.15.204:0/566454533 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0348070cc0 con 0x7f034806eea0 2024-09-15T01:36:13.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.326+0000 7f033f7fe640 1 --2- 172.21.15.204:0/566454533 >> v2:172.21.15.204:6812/28210 conn(0x7f03280396e0 0x7f032803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.326+0000 7f033f7fe640 1 -- 172.21.15.204:0/566454533 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 9034+0+0 (secure 0 0 0) 0x7f033804cbc0 con 0x7f034806eea0 2024-09-15T01:36:13.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.326+0000 7f0346ffd640 1 --2- 172.21.15.204:0/566454533 >> v2:172.21.15.204:6812/28210 conn(0x7f03280396e0 0x7f032803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.326+0000 7f0346ffd640 10 monclient: get_auth_request con 0x7f03280396e0 auth_method 0 2024-09-15T01:36:13.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.327+0000 7f0346ffd640 1 --2- 172.21.15.204:0/566454533 >> v2:172.21.15.204:6812/28210 conn(0x7f03280396e0 0x7f032803bb90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0330009310 tx=0x7f0330008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:13.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.331+0000 7f033f7fe640 1 -- 172.21.15.204:0/566454533 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0338014a30 con 0x7f034806eea0 2024-09-15T01:36:13.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.331+0000 7f033f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:13.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.331+0000 7f033f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:13.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.479+0000 7f5945100640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:36:13.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.479+0000 7f5945100640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:13.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.479+0000 7f5945100640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:13.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.479+0000 7f5945100640 1 -- 172.21.15.178:0/769840106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f594004b720 con 0x7f594009f180 2024-09-15T01:36:13.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.479+0000 7f5926ffd640 1 -- 172.21.15.178:0/769840106 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3166 (secure 0 0 0) 0x7f593801e020 con 0x7f594009f180 2024-09-15T01:36:13.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.479+0000 7f5926ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:13.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.479+0000 7f5926ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:13.482 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":44,"quorum":[0,1,3,4,5,8],"quorum_names":["a","f","g","c","h","e"],"quorum_leader_name":"f","quorum_age":91,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:36:13.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 1 -- 172.21.15.178:0/769840106 >> v2:172.21.15.204:6812/28210 conn(0x7f592c0397b0 msgr2=0x7f592c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:13.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 1 --2- 172.21.15.178:0/769840106 >> v2:172.21.15.204:6812/28210 conn(0x7f592c0397b0 0x7f592c03bc60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5930000c00 tx=0x7f5930032000 comp rx=0 tx=0).stop 2024-09-15T01:36:13.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 10 monclient: shutdown 2024-09-15T01:36:13.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:13.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 1 -- 172.21.15.178:0/769840106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f594009f180 msgr2=0x7f594013f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:13.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 1 --2- 172.21.15.178:0/769840106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f594009f180 0x7f594013f680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5938009fd0 tx=0x7f5938009870 comp rx=0 tx=0).stop 2024-09-15T01:36:13.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 1 -- 172.21.15.178:0/769840106 shutdown_connections 2024-09-15T01:36:13.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 1 --2- 172.21.15.178:0/769840106 >> v2:172.21.15.204:6812/28210 conn(0x7f592c0397b0 0x7f592c03bc60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 1 --2- 172.21.15.178:0/769840106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f594009f180 0x7f594013f680 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 1 --2- 172.21.15.178:0/769840106 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f594006ea30 0x7f594013b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 1 --2- 172.21.15.178:0/769840106 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f59400678d0 0x7f594013c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 1 -- 172.21.15.178:0/769840106 >> 172.21.15.178:0/769840106 conn(0x7f59400a4390 msgr2=0x7f594009d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:13.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.481+0000 7f5945100640 1 -- 172.21.15.178:0/769840106 shutdown_connections 2024-09-15T01:36:13.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:13.482+0000 7f5945100640 1 -- 172.21.15.178:0/769840106 wait complete. 2024-09-15T01:36:13.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.508+0000 7f034e15d640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.508+0000 7f034e15d640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.508+0000 7f034e15d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:13.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.508+0000 7f034e15d640 1 -- 172.21.15.204:0/566454533 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f034806a8b0 con 0x7f034806eea0 2024-09-15T01:36:13.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.509+0000 7f033f7fe640 1 -- 172.21.15.204:0/566454533 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v165) v1 ==== 83+0+2320 (secure 0 0 0) 0x7f0338011340 con 0x7f034806eea0 2024-09-15T01:36:13.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.509+0000 7f033f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.509+0000 7f033f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.606+0000 7f034e15d640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.606+0000 7f034e15d640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.606+0000 7f034e15d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:13.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.606+0000 7f034e15d640 1 -- 172.21.15.204:0/566454533 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f034811f880 con 0x7f034806eea0 2024-09-15T01:36:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.607+0000 7f033f7fe640 1 -- 172.21.15.204:0/566454533 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v165) v1 ==== 83+0+2320 (secure 0 0 0) 0x7f033802c090 con 0x7f034806eea0 2024-09-15T01:36:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.607+0000 7f033f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.607+0000 7f033f7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:13.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 1 -- 172.21.15.204:0/566454533 >> v2:172.21.15.204:6812/28210 conn(0x7f03280396e0 msgr2=0x7f032803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:13.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 1 --2- 172.21.15.204:0/566454533 >> v2:172.21.15.204:6812/28210 conn(0x7f03280396e0 0x7f032803bb90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0330009310 tx=0x7f0330008040 comp rx=0 tx=0).stop 2024-09-15T01:36:13.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 10 monclient: shutdown 2024-09-15T01:36:13.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:13.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 1 -- 172.21.15.204:0/566454533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f034806eea0 msgr2=0x7f034813c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:13.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 1 --2- 172.21.15.204:0/566454533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f034806eea0 0x7f034813c000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f033800d890 tx=0x7f0338008040 comp rx=0 tx=0).stop 2024-09-15T01:36:13.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 1 -- 172.21.15.204:0/566454533 shutdown_connections 2024-09-15T01:36:13.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 1 --2- 172.21.15.204:0/566454533 >> v2:172.21.15.204:6812/28210 conn(0x7f03280396e0 0x7f032803bb90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 1 --2- 172.21.15.204:0/566454533 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f034809f2d0 0x7f034813b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 1 --2- 172.21.15.204:0/566454533 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f03480a4680 0x7f034813f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 1 --2- 172.21.15.204:0/566454533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f034806eea0 0x7f034813c000 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 1 -- 172.21.15.204:0/566454533 >> 172.21.15.204:0/566454533 conn(0x7f0348067740 msgr2=0x7f034809cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:13.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 1 -- 172.21.15.204:0/566454533 shutdown_connections 2024-09-15T01:36:13.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.609+0000 7f034e15d640 1 -- 172.21.15.204:0/566454533 wait complete. 2024-09-15T01:36:13.619 INFO:tasks.workunit.client.0.smithi204.stdout: 0 globster 0.08719 7.50000 8.50000 6.50000 osd.0 2024-09-15T01:36:13.619 INFO:tasks.workunit.client.0.smithi204.stdout: 0 globster 0.08719 7.50000 8.50000 6.50000 osd.0 2024-09-15T01:36:13.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush tree --show-shadow 2024-09-15T01:36:13.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep 'osd\.0' 2024-09-15T01:36:13.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep globster 2024-09-15T01:36:13.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep '6\.' 2024-09-15T01:36:13.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.692+0000 7f798e19d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:13.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.692+0000 7f798e19d640 10 monclient: build_initial_monmap 2024-09-15T01:36:13.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.692+0000 7f798e19d640 10 monclient: monmap: 2024-09-15T01:36:13.694 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:13.694 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.694 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:13.692835+0000 2024-09-15T01:36:13.694 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:13.692835+0000 2024-09-15T01:36:13.694 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:13.694 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:13.694 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:13.694 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:13.694 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:13.694 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:13.695 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:13.695 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:13.695 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:13.695 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:13.695 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:13.695 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:13.695 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:13.695 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:13.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 1 Processor -- start 2024-09-15T01:36:13.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 1 -- start start 2024-09-15T01:36:13.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 10 monclient: init 2024-09-15T01:36:13.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 10 monclient: _add_conns ranks=[5,7,10,9,1,4,8,6,3,0,2] 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79880a2e30 0x7f7988132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 10 monclient(hunting): picked mon.noname-h con 0x7f79880a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798809fb80 0x7f798809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 10 monclient(hunting): picked mon.noname-i con 0x7f798809fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f79880a0a40 0x7f79880a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 10 monclient(hunting): picked mon.noname-f con 0x7f79880a0a40 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798809e6c0 con 0x7f79880a0a40 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79880a04b0 con 0x7f79880a2e30 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f79880a3200 con 0x7f798809fb80 2024-09-15T01:36:13.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:13.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f798e19d640 10 monclient(hunting): authenticate will time out at 989.335584s 2024-09-15T01:36:13.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7987fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f79880a0a40 0x7f79880a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7987fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f79880a0a40 0x7f79880a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39984/0 (socket says 172.21.15.204:39984) 2024-09-15T01:36:13.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7987fff640 1 -- 172.21.15.204:0/3045120769 learned_addr learned my addr 172.21.15.204:0/3045120769 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:13.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7987fff640 10 monclient(hunting): get_auth_request con 0x7f79880a0a40 auth_method 0 2024-09-15T01:36:13.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7987fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7987fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7987fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7986ffd640 1 --2- 172.21.15.204:0/3045120769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798809fb80 0x7f798809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f79877fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79880a2e30 0x7f7988132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7986ffd640 10 monclient(hunting): get_auth_request con 0x7f798809fb80 auth_method 0 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7986ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7986ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7986ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f79877fe640 10 monclient(hunting): get_auth_request con 0x7f79880a2e30 auth_method 0 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f79877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f79877fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f79877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7987fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7987fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7987fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f79877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f79877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:13.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f79877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7986ffd640 1 -- 172.21.15.204:0/3045120769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798809fb80 msgr2=0x7f798809ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7986ffd640 1 -- 172.21.15.204:0/3045120769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798809fb80 msgr2=0x7f798809ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7986ffd640 1 --2- 172.21.15.204:0/3045120769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798809fb80 0x7f798809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.693+0000 7f7986ffd640 1 --2- 172.21.15.204:0/3045120769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798809fb80 0x7f798809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79877fe640 10 monclient(hunting): handle_auth_done global_id 5837 payload 274 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79877fe640 1 -- 172.21.15.204:0/3045120769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798809fb80 msgr2=0x7f798809ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79877fe640 1 --2- 172.21.15.204:0/3045120769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798809fb80 0x7f798809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79877fe640 1 -- 172.21.15.204:0/3045120769 >> v2:172.21.15.204:3300/0 conn(0x7f79880a0a40 msgr2=0x7f79880a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79877fe640 1 --2- 172.21.15.204:0/3045120769 >> v2:172.21.15.204:3300/0 conn(0x7f79880a0a40 0x7f79880a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79877fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79877fe640 1 monclient: found mon.noname-h 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79877fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79877fe640 1 -- 172.21.15.204:0/3045120769 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798804b720 con 0x7f79880a2e30 2024-09-15T01:36:13.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f7987fff640 1 --2- 172.21.15.204:0/3045120769 >> v2:172.21.15.204:3300/0 conn(0x7f79880a0a40 0x7f79880a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79877fe640 1 --2- 172.21.15.204:0/3045120769 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79880a2e30 0x7f7988132a60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7978009850 tx=0x7f79780090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=cf76b8afb8d69686 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79867fc640 1 -- 172.21.15.204:0/3045120769 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f797802abc0 con 0x7f79880a2e30 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79867fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79867fc640 10 monclient: dump: 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:13.708 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79867fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79867fc640 1 -- 172.21.15.204:0/3045120769 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797802ad40 con 0x7f79880a2e30 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f798e19d640 5 monclient: authenticate success, global_id 5837 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f798e19d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79867fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f798e19d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:13.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79867fc640 1 -- 172.21.15.204:0/3045120769 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7978031ca0 con 0x7f79880a2e30 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f798e19d640 4 set_mon_vals no callback set 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79867fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.694+0000 7f79867fc640 10 monclient: dump: 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:13.710 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 10 monclient: shutdown 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 1 -- 172.21.15.204:0/3045120769 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79880a2e30 msgr2=0x7f7988132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 1 --2- 172.21.15.204:0/3045120769 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79880a2e30 0x7f7988132a60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7978009850 tx=0x7f79780090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 1 -- 172.21.15.204:0/3045120769 shutdown_connections 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 1 --2- 172.21.15.204:0/3045120769 >> v2:172.21.15.204:3300/0 conn(0x7f79880a0a40 0x7f79880a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 1 --2- 172.21.15.204:0/3045120769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f798809fb80 0x7f798809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 1 --2- 172.21.15.204:0/3045120769 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79880a2e30 0x7f7988132a60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 1 -- 172.21.15.204:0/3045120769 >> 172.21.15.204:0/3045120769 conn(0x7f798806e860 msgr2=0x7f798806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 1 -- 172.21.15.204:0/3045120769 shutdown_connections 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 1 -- 172.21.15.204:0/3045120769 wait complete. 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 10 monclient: build_initial_monmap 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.695+0000 7f798e19d640 10 monclient: monmap: 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:13.712 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 1 Processor -- start 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 1 -- start start 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 10 monclient: init 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 10 monclient: _add_conns ranks=[3,6,5,1,0,8,7,2,4] 2024-09-15T01:36:13.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f798809fb80 0x7f7988130680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 10 monclient(hunting): picked mon.noname-g con 0x7f798809fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:13.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79880a0a40 0x7f798811d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 10 monclient(hunting): picked mon.noname-d con 0x7f79880a0a40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:13.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79880a2e30 0x7f798811d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 10 monclient(hunting): picked mon.noname-h con 0x7f79880a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:13.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798809d650 con 0x7f798809fb80 2024-09-15T01:36:13.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7988133f40 con 0x7f79880a2e30 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7988120ab0 con 0x7f79880a0a40 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.696+0000 7f798e19d640 10 monclient(hunting): authenticate will time out at 989.338914s 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f79877fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f798809fb80 0x7f7988130680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79880a2e30 0x7f798811d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f79877fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f798809fb80 0x7f7988130680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:33508/0 (socket says 172.21.15.204:33508) 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 10 monclient(hunting): get_auth_request con 0x7f79880a2e30 auth_method 0 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f79877fe640 1 -- 172.21.15.204:0/3058902359 learned_addr learned my addr 172.21.15.204:0/3058902359 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f79877fe640 10 monclient(hunting): get_auth_request con 0x7f798809fb80 auth_method 0 2024-09-15T01:36:13.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f79877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f79877fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f79877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:13.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:13.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:13.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f79877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:13.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f79877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:13.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f79877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:13.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7986ffd640 1 --2- 172.21.15.204:0/3058902359 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79880a0a40 0x7f798811d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7986ffd640 10 monclient(hunting): get_auth_request con 0x7f79880a0a40 auth_method 0 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7986ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7986ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7986ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7986ffd640 1 -- 172.21.15.204:0/3058902359 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79880a0a40 msgr2=0x7f798811d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7986ffd640 1 -- 172.21.15.204:0/3058902359 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79880a0a40 msgr2=0x7f798811d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 10 monclient(hunting): handle_auth_done global_id 5846 payload 824 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7986ffd640 1 --2- 172.21.15.204:0/3058902359 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79880a0a40 0x7f798811d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7986ffd640 1 --2- 172.21.15.204:0/3058902359 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79880a0a40 0x7f798811d130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 1 -- 172.21.15.204:0/3058902359 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79880a0a40 msgr2=0x7f798811d130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 1 --2- 172.21.15.204:0/3058902359 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79880a0a40 0x7f798811d130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 1 -- 172.21.15.204:0/3058902359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f798809fb80 msgr2=0x7f7988130680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 1 --2- 172.21.15.204:0/3058902359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f798809fb80 0x7f7988130680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:13.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 1 monclient: found mon.noname-h 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 1 -- 172.21.15.204:0/3058902359 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7980009ae0 con 0x7f79880a2e30 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f79877fe640 1 --2- 172.21.15.204:0/3058902359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f798809fb80 0x7f7988130680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7987fff640 1 --2- 172.21.15.204:0/3058902359 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79880a2e30 0x7f798811d670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7980009f10 tx=0x7f798000b040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f796f7fe640 1 -- 172.21.15.204:0/3058902359 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7980018070 con 0x7f79880a2e30 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f796f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f796f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f796f7fe640 10 monclient: dump: 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.718 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:13.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f796f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f796f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f796f7fe640 1 -- 172.21.15.204:0/3058902359 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798000f040 con 0x7f79880a2e30 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f798e19d640 5 monclient: authenticate success, global_id 5846 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f798e19d640 10 monclient: _renew_subs 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f798e19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f798e19d640 1 -- 172.21.15.204:0/3058902359 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7978009390 con 0x7f79880a2e30 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f796f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f796f7fe640 1 -- 172.21.15.204:0/3058902359 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79800092e0 con 0x7f79880a2e30 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f796f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f796f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f796f7fe640 10 monclient: dump: 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:13.720 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:13.721 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f798e19d640 10 monclient: _renew_subs 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f798e19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f7985ffb640 4 set_mon_vals no callback set 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.697+0000 7f798e19d640 1 -- 172.21.15.204:0/3058902359 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798811ed10 con 0x7f79880a2e30 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.698+0000 7f796f7fe640 1 -- 172.21.15.204:0/3058902359 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7980003020 con 0x7f79880a2e30 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.698+0000 7f798e19d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.698+0000 7f798e19d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.698+0000 7f798e19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.698+0000 7f796f7fe640 1 --2- 172.21.15.204:0/3058902359 >> v2:172.21.15.204:6812/28210 conn(0x7f79740397b0 0x7f797403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.698+0000 7f798e19d640 1 -- 172.21.15.204:0/3058902359 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7988120cd0 con 0x7f79880a2e30 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.698+0000 7f796f7fe640 1 -- 172.21.15.204:0/3058902359 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 9034+0+0 (secure 0 0 0) 0x7f798004c640 con 0x7f79880a2e30 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.698+0000 7f79877fe640 1 --2- 172.21.15.204:0/3058902359 >> v2:172.21.15.204:6812/28210 conn(0x7f79740397b0 0x7f797403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.698+0000 7f79877fe640 10 monclient: get_auth_request con 0x7f79740397b0 auth_method 0 2024-09-15T01:36:13.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.699+0000 7f79877fe640 1 --2- 172.21.15.204:0/3058902359 >> v2:172.21.15.204:6812/28210 conn(0x7f79740397b0 0x7f797403bc60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7978009720 tx=0x7f7978032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:13.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.703+0000 7f796f7fe640 1 -- 172.21.15.204:0/3058902359 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7988120cd0 con 0x7f79880a2e30 2024-09-15T01:36:13.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.703+0000 7f796f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:13.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.703+0000 7f796f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:13.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.868+0000 7f798e19d640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.868+0000 7f798e19d640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:13.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.868+0000 7f798e19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:13.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:13.868+0000 7f798e19d640 1 -- 172.21.15.204:0/3058902359 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f7988120cd0 con 0x7f79880a2e30 2024-09-15T01:36:16.494 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:36:16.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.596+0000 7fcc80634640 10 monclient: get_monmap_and_config 2024-09-15T01:36:16.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.596+0000 7fcc80634640 10 monclient: build_initial_monmap 2024-09-15T01:36:16.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.596+0000 7fcc80634640 10 monclient: monmap: 2024-09-15T01:36:16.598 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:16.598 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:16.598 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:16.596994+0000 2024-09-15T01:36:16.598 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:16.596994+0000 2024-09-15T01:36:16.598 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:16.598 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:16.598 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:16.598 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:16.598 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:16.598 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:16.599 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:16.599 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:16.599 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:16.599 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:16.599 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:16.599 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:16.599 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:16.599 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:16.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.596+0000 7fcc80634640 1 Processor -- start 2024-09-15T01:36:16.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.596+0000 7fcc80634640 1 -- start start 2024-09-15T01:36:16.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.596+0000 7fcc80634640 10 monclient: init 2024-09-15T01:36:16.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:16.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 10 monclient: _add_conns ranks=[3,7,0,10,9,1,2,4,8,6,5] 2024-09-15T01:36:16.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:16.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 10 monclient(hunting): picked mon.noname-g con 0x7fcc78067130 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:16.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:16.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 10 monclient(hunting): picked mon.noname-i con 0x7fcc7809d580 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:16.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcc7809f000 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:16.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7804bf60 con 0x7fcc7809f000 2024-09-15T01:36:16.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:16.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7809e0c0 con 0x7fcc78067130 2024-09-15T01:36:16.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:16.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78067f80 con 0x7fcc7809d580 2024-09-15T01:36:16.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:16.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc80634640 10 monclient(hunting): authenticate will time out at 981.347575s 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7ebd1640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fcc7809f000 0x7fcc7812d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:50412/0 (socket says 172.21.15.178:50412) 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7ebd1640 1 -- 172.21.15.178:0/3167020153 learned_addr learned my addr 172.21.15.178:0/3167020153 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request con 0x7fcc7809d580 auth_method 0 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request con 0x7fcc78067130 auth_method 0 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:16.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7dbcf640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 msgr2=0x7fcc78067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:36:16.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7dbcf640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 msgr2=0x7fcc78067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:16.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:16.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.597+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 msgr2=0x7fcc78067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:36:16.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.598+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 msgr2=0x7fcc78067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:16.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.598+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:16.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.598+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:16.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.598+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:16.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:16.696+0000 7f796e7fc640 10 monclient: tick 2024-09-15T01:36:16.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:16.696+0000 7f796e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:16.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:16.696+0000 7f796e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:16.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:16.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request con 0x7fcc7809d580 auth_method 0 2024-09-15T01:36:16.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:16.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:16.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:16.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:16.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request con 0x7fcc78067130 auth_method 0 2024-09-15T01:36:16.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:16.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:16.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:16.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7dbcf640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 msgr2=0x7fcc78067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:36:16.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7dbcf640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 msgr2=0x7fcc78067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:16.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:16.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:16.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 msgr2=0x7fcc78067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:36:16.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 msgr2=0x7fcc78067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:16.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:16.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:16.799+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:17.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:17.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request con 0x7fcc78067130 auth_method 0 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request con 0x7fcc7809d580 auth_method 0 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 msgr2=0x7fcc78067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7dbcf640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 msgr2=0x7fcc78067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 msgr2=0x7fcc78067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7dbcf640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 msgr2=0x7fcc78067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:17.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:17.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:17.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:17.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:17.200+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:17.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:17.696+0000 7f796e7fc640 10 monclient: tick 2024-09-15T01:36:17.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:17.696+0000 7f796e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request con 0x7fcc7809d580 auth_method 0 2024-09-15T01:36:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request con 0x7fcc78067130 auth_method 0 2024-09-15T01:36:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7dbcf640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 msgr2=0x7fcc78067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:36:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7dbcf640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 msgr2=0x7fcc78067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 msgr2=0x7fcc78067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:36:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 msgr2=0x7fcc78067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.002+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.696+0000 7f796e7fc640 10 monclient: tick 2024-09-15T01:36:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.696+0000 7f796e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:18.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120102085 0 0) 0x7fcc7804bf60 con 0x7fcc7809f000 2024-09-15T01:36:18.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:18.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:18.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient(hunting): my global_id is 6075 2024-09-15T01:36:18.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc600034e0 con 0x7fcc7809f000 2024-09-15T01:36:18.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3692415272 0 0) 0x7fcc600034e0 con 0x7fcc7809f000 2024-09-15T01:36:18.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 msgr2=0x7fcc78067a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:18.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc7809d580 0x7fcc78067a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:18.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 msgr2=0x7fcc78067500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:18.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc78067130 0x7fcc78067500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:18.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient: _finish_hunting 0 2024-09-15T01:36:18.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:36:18.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:18.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:36:18.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7809dde0 con 0x7fcc7809f000 2024-09-15T01:36:18.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient: _finish_auth 0 2024-09-15T01:36:18.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:18.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fcc68001c70 con 0x7fcc7809f000 2024-09-15T01:36:18.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc80634640 5 monclient: authenticate success, global_id 6075 2024-09-15T01:36:18.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc80634640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:18.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:18.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:36:18.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient: dump: 2024-09-15T01:36:18.833 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:18.833 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:18.833 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:18.833 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:18.833 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:18.833 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:18.833 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:18.834 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:18.834 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:18.834 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:18.834 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:18.834 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:18.834 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:18.834 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:18.834 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:18.834 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:18.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:36:18.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:18.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 >> v1:172.21.15.178:6789/0 conn(0x7fcc7809f000 legacy=0x7fcc7812d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient: _add_conns ranks=[2,6,4,1,3,7,0,5,8] 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60004220 0x7fcc60004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient(hunting): picked mon.b con 0x7fcc60004220 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004de0 0x7fcc60005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient(hunting): picked mon.d con 0x7fcc60004de0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcc60009c00 0x7fcc6000c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient(hunting): picked mon.c con 0x7fcc60009c00 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7809e0c0 con 0x7fcc60004220 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78067f80 con 0x7fcc60009c00 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:18.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6000c970 con 0x7fcc60004de0 2024-09-15T01:36:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc7d3ce640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:36:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.825+0000 7fcc80634640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7ebd1640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcc60009c00 0x7fcc6000c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60004220 0x7fcc60004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7ebd1640 10 monclient(hunting): get_auth_request con 0x7fcc60009c00 auth_method 0 2024-09-15T01:36:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7ebd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7ebd1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7ebd1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request con 0x7fcc60004220 auth_method 0 2024-09-15T01:36:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004de0 0x7fcc60005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request con 0x7fcc60004de0 auth_method 0 2024-09-15T01:36:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7ebd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7ebd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7ebd1640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:18.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:18.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7dbcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:18.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7dbcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:18.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7dbcf640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:18.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 10 monclient(hunting): handle_auth_done global_id 6075 payload 293 2024-09-15T01:36:18.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004de0 msgr2=0x7fcc60005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:18.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004de0 0x7fcc60005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:18.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcc60009c00 msgr2=0x7fcc6000c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:18.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcc60009c00 0x7fcc6000c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:18.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 10 monclient: _finish_hunting 0 2024-09-15T01:36:18.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 1 monclient: found mon.b 2024-09-15T01:36:18.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:18.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:18.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc600034e0 con 0x7fcc60004220 2024-09-15T01:36:18.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 10 monclient: _finish_auth 0 2024-09-15T01:36:18.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7e3d0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:18.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7ebd1640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcc60009c00 0x7fcc6000c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:18.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.826+0000 7fcc7ebd1640 1 -- 172.21.15.178:0/3167020153 reap_dead start 2024-09-15T01:36:18.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.827+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60004220 msgr2=0x7fcc60004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:36:18.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.827+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60004220 msgr2=0x7fcc60004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:18.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.827+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60004220 0x7fcc60004780 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7fcc70006ea0 tx=0x7fcc70008040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:18.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:18.827+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60004220 0x7fcc60004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:18.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.827+0000 7f796f7fe640 1 -- 172.21.15.204:0/3058902359 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v165) v1 ==== 83+0+2320 (secure 0 0 0) 0x7f7988120cd0 con 0x7f79880a2e30 2024-09-15T01:36:18.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.827+0000 7f796f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:18.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.827+0000 7f796f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:18.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.934+0000 7f798e19d640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:18.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.934+0000 7f798e19d640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:18.936 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.934+0000 7f798e19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:18.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.934+0000 7f798e19d640 1 -- 172.21.15.204:0/3058902359 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f7988120c70 con 0x7f79880a2e30 2024-09-15T01:36:18.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.934+0000 7f796f7fe640 1 -- 172.21.15.204:0/3058902359 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v165) v1 ==== 83+0+2320 (secure 0 0 0) 0x7f798000ce30 con 0x7f79880a2e30 2024-09-15T01:36:18.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.934+0000 7f796f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-09-15T01:36:18.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.934+0000 7f796f7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:18.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 1 -- 172.21.15.204:0/3058902359 >> v2:172.21.15.204:6812/28210 conn(0x7f79740397b0 msgr2=0x7f797403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:18.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 1 --2- 172.21.15.204:0/3058902359 >> v2:172.21.15.204:6812/28210 conn(0x7f79740397b0 0x7f797403bc60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7978009720 tx=0x7f7978032000 comp rx=0 tx=0).stop 2024-09-15T01:36:18.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 10 monclient: shutdown 2024-09-15T01:36:18.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:18.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 1 -- 172.21.15.204:0/3058902359 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79880a2e30 msgr2=0x7f798811d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:18.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 1 --2- 172.21.15.204:0/3058902359 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79880a2e30 0x7f798811d670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7980009f10 tx=0x7f798000b040 comp rx=0 tx=0).stop 2024-09-15T01:36:18.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 1 -- 172.21.15.204:0/3058902359 shutdown_connections 2024-09-15T01:36:18.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 1 --2- 172.21.15.204:0/3058902359 >> v2:172.21.15.204:6812/28210 conn(0x7f79740397b0 0x7f797403bc60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:18.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 1 --2- 172.21.15.204:0/3058902359 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f79880a2e30 0x7f798811d670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:18.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 1 --2- 172.21.15.204:0/3058902359 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79880a0a40 0x7f798811d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:18.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 1 --2- 172.21.15.204:0/3058902359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f798809fb80 0x7f7988130680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:18.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 1 -- 172.21.15.204:0/3058902359 >> 172.21.15.204:0/3058902359 conn(0x7f798806e860 msgr2=0x7f79880a5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:18.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 1 -- 172.21.15.204:0/3058902359 shutdown_connections 2024-09-15T01:36:18.940 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:18.936+0000 7f798e19d640 1 -- 172.21.15.204:0/3058902359 wait complete. 2024-09-15T01:36:18.948 INFO:tasks.workunit.client.0.smithi204.stdout: 0 globster 0.08719 7.50000 8.50000 6.50000 osd.0 2024-09-15T01:36:18.948 INFO:tasks.workunit.client.0.smithi204.stdout: 0 globster 0.08719 7.50000 8.50000 6.50000 osd.0 2024-09-15T01:36:18.948 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm-device-class osd.0 2024-09-15T01:36:19.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.019+0000 7f0c7fd9d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:19.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.019+0000 7f0c7fd9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:19.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.019+0000 7f0c7fd9d640 10 monclient: monmap: 2024-09-15T01:36:19.022 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:19.022 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:19.022 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:19.021101+0000 2024-09-15T01:36:19.022 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:19.021101+0000 2024-09-15T01:36:19.022 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:19.022 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:19.022 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:19.023 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:19.023 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 1 Processor -- start 2024-09-15T01:36:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 1 -- start start 2024-09-15T01:36:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 10 monclient: init 2024-09-15T01:36:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 10 monclient: _add_conns ranks=[4,1,0,7,2,10,9,8,5,6,3] 2024-09-15T01:36:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c780a2e30 0x7f0c78132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 10 monclient(hunting): picked mon.noname-c con 0x7f0c780a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0c7809fb80 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0c7809ff70 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7804b720 con 0x7f0c7809ff70 2024-09-15T01:36:19.040 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c780a3200 con 0x7f0c7809fb80 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7809e6c0 con 0x7f0c780a2e30 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.020+0000 7f0c7fd9d640 10 monclient(hunting): authenticate will time out at 994.663937s 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7d338640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f0c7809fb80 0x7f0c780a0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:45958/0 (socket says 172.21.15.204:45958) 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7d338640 1 -- 172.21.15.204:0/2541159752 learned_addr learned my addr 172.21.15.204:0/2541159752 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698909000 0 0) 0x7f0c780a3200 con 0x7f0c7809fb80 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c780a2e30 0x7f0c78132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient(hunting): my global_id is 6535 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c640034e0 con 0x7f0c7809fb80 2024-09-15T01:36:19.041 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 10 monclient(hunting): get_auth_request con 0x7f0c780a2e30 auth_method 0 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1160121300 0 0) 0x7f0c640034e0 con 0x7f0c7809fb80 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c780a2e30 msgr2=0x7f0c78132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c780a2e30 0x7f0c78132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 >> v1:172.21.15.178:6789/0 conn(0x7f0c7809ff70 legacy=0x7f0c780a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient: _finish_hunting 0 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:19.042 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c780a1860 con 0x7f0c7809fb80 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient: _finish_auth 0 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7fd9d640 5 monclient: authenticate success, global_id 6535 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f0c680018f0 con 0x7f0c7809fb80 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient: dump: 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.043 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:19.044 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 >> v1:172.21.15.204:6789/0 conn(0x7f0c7809fb80 legacy=0x7f0c780a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient: _add_conns ranks=[0,7,2,8,3,6,1,5,4] 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c640041d0 0x7f0c64004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient(hunting): picked mon.a con 0x7f0c640041d0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c64004cc0 0x7f0c64005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient(hunting): picked mon.i con 0x7f0c64004cc0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c64009ba0 0x7f0c6400bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient(hunting): picked mon.b con 0x7f0c64009ba0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7809e6c0 con 0x7f0c640041d0 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6400c8c0 con 0x7f0c64009ba0 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.045 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6400ca30 con 0x7f0c64004cc0 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7cb37640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7fd9d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7d338640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c64004cc0 0x7f0c64005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7d338640 10 monclient(hunting): get_auth_request con 0x7f0c64004cc0 auth_method 0 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7d338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7d338640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7d338640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c640041d0 0x7f0c64004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 10 monclient(hunting): get_auth_request con 0x7f0c640041d0 auth_method 0 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.046 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7db39640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7d338640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7d338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7d338640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7e33a640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c64009ba0 0x7f0c6400bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7e33a640 10 monclient(hunting): get_auth_request con 0x7f0c64009ba0 auth_method 0 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7e33a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7e33a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.021+0000 7f0c7e33a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7db39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7db39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7db39640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7e33a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7e33a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7e33a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:19.047 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 10 monclient(hunting): handle_auth_done global_id 6535 payload 293 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 1 -- 172.21.15.204:0/2541159752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c64009ba0 msgr2=0x7f0c6400bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c64009ba0 0x7f0c6400bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 1 -- 172.21.15.204:0/2541159752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c640041d0 msgr2=0x7f0c64004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c640041d0 0x7f0c64004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 10 monclient: _finish_hunting 0 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 1 monclient: found mon.i 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7e33a640 1 -- 172.21.15.204:0/2541159752 reap_dead start 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 1 -- 172.21.15.204:0/2541159752 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c640034e0 con 0x7f0c64004cc0 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 10 monclient: _finish_auth 0 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:19.048 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7d338640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c64004cc0 0x7f0c64005130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0c780a2e00 tx=0x7f0c68004860 comp rx=0 tx=0).ready entity=mon.7 client_cookie=ad54b3cbf2801bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c680085d0 con 0x7f0c64004cc0 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7cb37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7cb37640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.022+0000 7f0c7cb37640 10 monclient: dump: 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:19.049 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7fd9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c68008750 con 0x7f0c64004cc0 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7cb37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7fd9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7fd9d640 4 set_mon_vals no callback set 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7cb37640 1 -- 172.21.15.204:0/2541159752 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c68005d10 con 0x7f0c64004cc0 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7cb37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:19.050 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7cb37640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7cb37640 10 monclient: dump: 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:19.051 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:19.052 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:19.052 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:19.052 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:19.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7fd9d640 10 monclient: shutdown 2024-09-15T01:36:19.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7fd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:19.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/2541159752 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c64004cc0 msgr2=0x7f0c64005130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:19.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7fd9d640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c64004cc0 0x7f0c64005130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0c780a2e00 tx=0x7f0c68004860 comp rx=0 tx=0).stop 2024-09-15T01:36:19.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/2541159752 shutdown_connections 2024-09-15T01:36:19.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7fd9d640 1 --2- 172.21.15.204:0/2541159752 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c64004cc0 0x7f0c64005130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/2541159752 >> 172.21.15.204:0/2541159752 conn(0x7f0c7806e860 msgr2=0x7f0c7806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:19.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/2541159752 shutdown_connections 2024-09-15T01:36:19.052 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.023+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/2541159752 wait complete. 2024-09-15T01:36:19.053 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:19.053 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient: monmap: 2024-09-15T01:36:19.053 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:19.053 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:19.053 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:19.053 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:19.053 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:19.053 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:19.053 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 1 Processor -- start 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 1 -- start start 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient: init 2024-09-15T01:36:19.054 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient: _add_conns ranks=[4,5,1,6,8,2,0,3,7] 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c64004cc0 0x7f0c7811de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient(hunting): picked mon.noname-c con 0x7f0c64004cc0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c78120530 0x7f0c7811e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient(hunting): picked mon.noname-h con 0x7f0c78120530 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c7811e8f0 0x7f0c7811ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient(hunting): picked mon.noname-f con 0x7f0c7811e8f0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c78121a50 con 0x7f0c7811e8f0 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7809d650 con 0x7f0c64004cc0 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.055 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7811f280 con 0x7f0c78120530 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.024+0000 7f0c7fd9d640 10 monclient(hunting): authenticate will time out at 994.667937s 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c7811e8f0 0x7f0c7811ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7d338640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c78120530 0x7f0c7811e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c7811e8f0 0x7f0c7811ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:52790/0 (socket says 172.21.15.204:52790) 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 1 -- 172.21.15.204:0/1771525616 learned_addr learned my addr 172.21.15.204:0/1771525616 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 10 monclient(hunting): get_auth_request con 0x7f0c7811e8f0 auth_method 0 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7d338640 10 monclient(hunting): get_auth_request con 0x7f0c78120530 auth_method 0 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7d338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:19.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7d338640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7d338640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7d338640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7d338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7d338640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7db39640 1 --2- 172.21.15.204:0/1771525616 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c64004cc0 0x7f0c7811de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 10 monclient(hunting): handle_auth_done global_id 6544 payload 824 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 1 -- 172.21.15.204:0/1771525616 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c78120530 msgr2=0x7f0c7811e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 1 --2- 172.21.15.204:0/1771525616 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c78120530 0x7f0c7811e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 1 -- 172.21.15.204:0/1771525616 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c64004cc0 msgr2=0x7f0c7811de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 1 --2- 172.21.15.204:0/1771525616 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c64004cc0 0x7f0c7811de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 10 monclient: _finish_hunting 0 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 1 monclient: found mon.noname-f 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 1 -- 172.21.15.204:0/1771525616 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c78132640 con 0x7f0c7811e8f0 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7db39640 10 monclient: get_auth_request con 0x7f0c64004cc0 auth_method 0 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7db39640 1 --2- 172.21.15.204:0/1771525616 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c64004cc0 0x7f0c7811de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7e33a640 1 --2- 172.21.15.204:0/1771525616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c7811e8f0 0x7f0c7811ed40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0c7806f710 tx=0x7f0c70007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6dffb640 1 -- 172.21.15.204:0/1771525616 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c70006d20 con 0x7f0c7811e8f0 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6dffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6dffb640 10 monclient: dump: 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:19.058 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:19.059 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6dffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6dffb640 1 -- 172.21.15.204:0/1771525616 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c700074b0 con 0x7f0c7811e8f0 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7fd9d640 5 monclient: authenticate success, global_id 6544 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7fd9d640 10 monclient: _renew_subs 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7fd9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/1771525616 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c78121490 con 0x7f0c7811e8f0 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6dffb640 1 -- 172.21.15.204:0/1771525616 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c70007aa0 con 0x7f0c7811e8f0 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6dffb640 10 monclient: dump: 2024-09-15T01:36:19.060 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:19.061 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c6ffff640 4 set_mon_vals no callback set 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7fd9d640 10 monclient: _renew_subs 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7fd9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.025+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/1771525616 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c78147b80 con 0x7f0c7811e8f0 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.026+0000 7f0c6dffb640 1 -- 172.21.15.204:0/1771525616 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0c70007620 con 0x7f0c7811e8f0 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.026+0000 7f0c7fd9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.026+0000 7f0c7fd9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.026+0000 7f0c7fd9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.026+0000 7f0c6dffb640 1 --2- 172.21.15.204:0/1771525616 >> v2:172.21.15.204:6812/28210 conn(0x7f0c5c0397e0 0x7f0c5c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.026+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/1771525616 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c78070cc0 con 0x7f0c7811e8f0 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.027+0000 7f0c6dffb640 1 -- 172.21.15.204:0/1771525616 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 9034+0+0 (secure 0 0 0) 0x7f0c7004b500 con 0x7f0c7811e8f0 2024-09-15T01:36:19.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.027+0000 7f0c7db39640 1 --2- 172.21.15.204:0/1771525616 >> v2:172.21.15.204:6812/28210 conn(0x7f0c5c0397e0 0x7f0c5c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.027+0000 7f0c7db39640 10 monclient: get_auth_request con 0x7f0c5c0397e0 auth_method 0 2024-09-15T01:36:19.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.027+0000 7f0c7db39640 1 --2- 172.21.15.204:0/1771525616 >> v2:172.21.15.204:6812/28210 conn(0x7f0c5c0397e0 0x7f0c5c03bc90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009360 tx=0x7f0c74008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:19.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.031+0000 7f0c6dffb640 1 -- 172.21.15.204:0/1771525616 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0c700078d0 con 0x7f0c7811e8f0 2024-09-15T01:36:19.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.031+0000 7f0c6dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:19.063 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.031+0000 7f0c6dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:19.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60004220 0x7fcc60004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 10 monclient: get_auth_request con 0x7fcc60004220 auth_method 0 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 0 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60004220 0x7fcc60004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60004220 0x7fcc60004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 10 monclient: ms_handle_reset current mon [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60004220 msgr2=0x7fcc60004780 unknown :-1 s=STATE_CLOSED l=0).mark_down 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60004220 0x7fcc60004780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 10 monclient: _add_conns ranks=[8,5,1,3,0,7] 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc60005250 0x7fcc6000a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 10 monclient(hunting): picked mon.e con 0x7fcc60005250 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc6000a640 0x7fcc6000aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 10 monclient(hunting): picked mon.h con 0x7fcc6000a640 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:19.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc6000b0e0 0x7fcc6000b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 10 monclient(hunting): picked mon.f con 0x7fcc6000b0e0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6000be10 con 0x7fcc6000b0e0 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6000bf50 con 0x7fcc6000a640 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6000c110 con 0x7fcc60005250 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7d3ce640 10 monclient(hunting): ms_handle_reset stray mon [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc60005250 0x7fcc6000a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc6000a640 0x7fcc6000aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request con 0x7fcc60005250 auth_method 0 2024-09-15T01:36:19.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request con 0x7fcc6000a640 auth_method 0 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7ebd1640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc6000b0e0 0x7fcc6000b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7ebd1640 10 monclient(hunting): get_auth_request con 0x7fcc6000b0e0 auth_method 0 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7ebd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7ebd1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7ebd1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:19.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7dbcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7dbcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7dbcf640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7ebd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7ebd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7ebd1640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 10 monclient(hunting): handle_auth_done global_id 6075 payload 293 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc6000a640 msgr2=0x7fcc6000aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc6000a640 0x7fcc6000aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc6000b0e0 msgr2=0x7fcc6000b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc6000b0e0 0x7fcc6000b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 10 monclient: _finish_hunting 0 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 1 monclient: found mon.e 2024-09-15T01:36:19.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3167020153 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=2+}) v3 -- 0x7fcc7804bf60 con 0x7fcc60005250 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 10 monclient: _finish_auth 0 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7e3d0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc6000a640 0x7fcc6000aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.028+0000 7fcc7ebd1640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc6000b0e0 0x7fcc6000b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.029+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc60005250 0x7fcc6000a050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000c910 tx=0x7fcc70008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1cbe22919aa71357 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.029+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc7002bc70 con 0x7fcc60005250 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.029+0000 7fcc7d3ce640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.029+0000 7fcc7d3ce640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.029+0000 7fcc7d3ce640 10 monclient: dump: 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:19.071 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:19.072 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.029+0000 7fcc7d3ce640 1 -- 172.21.15.178:0/3167020153 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7002bdd0 con 0x7fcc60005250 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.029+0000 7fcc80634640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.029+0000 7fcc7d3ce640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.029+0000 7fcc80634640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.029+0000 7fcc80634640 4 set_mon_vals no callback set 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 10 monclient: shutdown 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 1 -- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc60005250 msgr2=0x7fcc6000a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc60005250 0x7fcc6000a050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000c910 tx=0x7fcc70008040 comp rx=0 tx=0).stop 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 1 -- 172.21.15.178:0/3167020153 shutdown_connections 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc6000b0e0 0x7fcc6000b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc6000a640 0x7fcc6000aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc60005250 0x7fcc6000a050 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 1 --2- 172.21.15.178:0/3167020153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60004220 0x7fcc60004780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 1 -- 172.21.15.178:0/3167020153 >> 172.21.15.178:0/3167020153 conn(0x7fcc78068a60 msgr2=0x7fcc780a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 1 -- 172.21.15.178:0/3167020153 shutdown_connections 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 1 -- 172.21.15.178:0/3167020153 wait complete. 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 10 monclient: build_initial_monmap 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.030+0000 7fcc80634640 10 monclient: monmap: 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:19.074 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 1 Processor -- start 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 1 -- start start 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 10 monclient: init 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 10 monclient: _add_conns ranks=[6,2,0,3,7,4,1,5,8] 2024-09-15T01:36:19.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004220 0x7fcc78133540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 10 monclient(hunting): picked mon.noname-d con 0x7fcc60004220 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60005250 0x7fcc7811d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 10 monclient(hunting): picked mon.noname-b con 0x7fcc60005250 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc6000a640 0x7fcc7811d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 10 monclient(hunting): picked mon.noname-a con 0x7fcc6000a640 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78120cb0 con 0x7fcc6000a640 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7809cb50 con 0x7fcc60005250 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7809d9e0 con 0x7fcc60004220 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:19.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc80634640 10 monclient(hunting): authenticate will time out at 983.781844s 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc7e3d0640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004220 0x7fcc78133540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc7dbcf640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60005250 0x7fcc7811d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc7ebd1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc6000a640 0x7fcc7811d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc7e3d0640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004220 0x7fcc78133540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:60064/0 (socket says 172.21.15.178:60064) 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request con 0x7fcc60005250 auth_method 0 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.031+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3424200867 learned_addr learned my addr 172.21.15.178:0/3424200867 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7dbcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7dbcf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request con 0x7fcc60004220 auth_method 0 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7e3d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7e3d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 10 monclient(hunting): get_auth_request con 0x7fcc6000a640 auth_method 0 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7dbcf640 1 -- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60005250 msgr2=0x7fcc7811d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7dbcf640 1 -- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60005250 msgr2=0x7fcc7811d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60005250 0x7fcc7811d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004220 msgr2=0x7fcc78133540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7e3d0640 1 -- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004220 msgr2=0x7fcc78133540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004220 0x7fcc78133540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:19.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7dbcf640 1 --2- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60005250 0x7fcc7811d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004220 0x7fcc78133540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 10 monclient(hunting): handle_auth_done global_id 6093 payload 824 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 1 -- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004220 msgr2=0x7fcc78133540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 1 --2- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004220 0x7fcc78133540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 1 -- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60005250 msgr2=0x7fcc7811d010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 1 --2- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60005250 0x7fcc7811d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 10 monclient: _finish_hunting 0 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 1 monclient: found mon.noname-a 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 1 -- 172.21.15.178:0/3424200867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7811f770 con 0x7fcc6000a640 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7ebd1640 1 --2- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc6000a640 0x7fcc7811d550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc780a37e0 tx=0x7fcc68009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:19.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc767fc640 1 -- 172.21.15.178:0/3424200867 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc680168d0 con 0x7fcc6000a640 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc767fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc767fc640 10 monclient: dump: 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:19.080 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc767fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc767fc640 1 -- 172.21.15.178:0/3424200867 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc68016ab0 con 0x7fcc6000a640 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc80634640 5 monclient: authenticate success, global_id 6093 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc80634640 10 monclient: _renew_subs 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc80634640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc80634640 1 -- 172.21.15.178:0/3424200867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7811f0b0 con 0x7fcc6000a640 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc767fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:19.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc767fc640 1 -- 172.21.15.178:0/3424200867 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc68016c20 con 0x7fcc6000a640 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc767fc640 10 monclient: dump: 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:19.082 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc80634640 10 monclient: _renew_subs 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc80634640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc7cbcd640 4 set_mon_vals no callback set 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.032+0000 7fcc80634640 1 -- 172.21.15.178:0/3424200867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc78120660 con 0x7fcc6000a640 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.033+0000 7fcc767fc640 1 -- 172.21.15.178:0/3424200867 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fcc68016450 con 0x7fcc6000a640 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.033+0000 7fcc80634640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.033+0000 7fcc80634640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.034+0000 7fcc80634640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:19.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.034+0000 7fcc767fc640 1 --2- 172.21.15.178:0/3424200867 >> v2:172.21.15.204:6812/28210 conn(0x7fcc640397e0 0x7fcc6403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:19.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.034+0000 7fcc80634640 1 -- 172.21.15.178:0/3424200867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7804b720 con 0x7fcc6000a640 2024-09-15T01:36:19.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.034+0000 7fcc767fc640 1 -- 172.21.15.178:0/3424200867 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 9034+0+0 (secure 0 0 0) 0x7fcc6804c560 con 0x7fcc6000a640 2024-09-15T01:36:19.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.034+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3424200867 >> v2:172.21.15.204:6812/28210 conn(0x7fcc640397e0 0x7fcc6403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:19.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.034+0000 7fcc7e3d0640 10 monclient: get_auth_request con 0x7fcc640397e0 auth_method 0 2024-09-15T01:36:19.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.034+0000 7fcc7e3d0640 1 --2- 172.21.15.178:0/3424200867 >> v2:172.21.15.204:6812/28210 conn(0x7fcc640397e0 0x7fcc6403bc90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcc70000c00 tx=0x7fcc70038000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:19.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.038+0000 7fcc767fc640 1 -- 172.21.15.178:0/3424200867 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcc6801a080 con 0x7fcc6000a640 2024-09-15T01:36:19.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.038+0000 7fcc767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:19.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.038+0000 7fcc767fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:19.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.190+0000 7f0c7fd9d640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:36:19.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.190+0000 7f0c7fd9d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:36:19.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.190+0000 7f0c7fd9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:19.193 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.190+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/1771525616 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) v1 -- 0x7f0c780a1f10 con 0x7f0c7811e8f0 2024-09-15T01:36:19.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.254+0000 7fcc80634640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:36:19.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.254+0000 7fcc80634640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:19.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.254+0000 7fcc80634640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:19.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.254+0000 7fcc80634640 1 -- 172.21.15.178:0/3424200867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcc78070770 con 0x7fcc6000a640 2024-09-15T01:36:19.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.254+0000 7fcc767fc640 1 -- 172.21.15.178:0/3424200867 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3177 (secure 0 0 0) 0x7fcc68015a90 con 0x7fcc6000a640 2024-09-15T01:36:19.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.254+0000 7fcc767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:19.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.254+0000 7fcc767fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:19.256 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":48,"quorum":[0,1,2,3,4,5,7,8],"quorum_names":["a","f","b","g","c","h","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:36:19.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 1 -- 172.21.15.178:0/3424200867 >> v2:172.21.15.204:6812/28210 conn(0x7fcc640397e0 msgr2=0x7fcc6403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:19.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 1 --2- 172.21.15.178:0/3424200867 >> v2:172.21.15.204:6812/28210 conn(0x7fcc640397e0 0x7fcc6403bc90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcc70000c00 tx=0x7fcc70038000 comp rx=0 tx=0).stop 2024-09-15T01:36:19.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 10 monclient: shutdown 2024-09-15T01:36:19.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:19.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 1 -- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc6000a640 msgr2=0x7fcc7811d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:19.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 1 --2- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc6000a640 0x7fcc7811d550 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc780a37e0 tx=0x7fcc68009040 comp rx=0 tx=0).stop 2024-09-15T01:36:19.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 1 -- 172.21.15.178:0/3424200867 shutdown_connections 2024-09-15T01:36:19.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 1 --2- 172.21.15.178:0/3424200867 >> v2:172.21.15.204:6812/28210 conn(0x7fcc640397e0 0x7fcc6403bc90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 1 --2- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc6000a640 0x7fcc7811d550 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 1 --2- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc60005250 0x7fcc7811d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 1 --2- 172.21.15.178:0/3424200867 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc60004220 0x7fcc78133540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:19.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 1 -- 172.21.15.178:0/3424200867 >> 172.21.15.178:0/3424200867 conn(0x7fcc78068a60 msgr2=0x7fcc780a0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:19.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 1 -- 172.21.15.178:0/3424200867 shutdown_connections 2024-09-15T01:36:19.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:19.256+0000 7fcc80634640 1 -- 172.21.15.178:0/3424200867 wait complete. 2024-09-15T01:36:19.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.898+0000 7f0c6dffb640 1 -- 172.21.15.204:0/1771525616 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 done removing class of osd(s): 0 v166) v1 ==== 123+0+0 (secure 0 0 0) 0x7f0c70019020 con 0x7f0c7811e8f0 2024-09-15T01:36:19.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.898+0000 7f0c6dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:36:19.901 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.898+0000 7f0c6dffb640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0 2024-09-15T01:36:19.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.995+0000 7f0c7fd9d640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:36:19.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.995+0000 7f0c7fd9d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:36:19.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.995+0000 7f0c7fd9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:19.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:19.995+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/1771525616 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) v1 -- 0x7f0c7809a230 con 0x7f0c7811e8f0 2024-09-15T01:36:20.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.905+0000 7f0c6dffb640 1 -- 172.21.15.204:0/1771525616 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 osd.0 belongs to no class, done removing class of osd(s): v167) v1 ==== 149+0+0 (secure 0 0 0) 0x7f0c70019030 con 0x7f0c7811e8f0 2024-09-15T01:36:20.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.905+0000 7f0c6dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-09-15T01:36:20.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.905+0000 7f0c6dffb640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T01:36:20.907 INFO:tasks.workunit.client.0.smithi204.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-09-15T01:36:20.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/1771525616 >> v2:172.21.15.204:6812/28210 conn(0x7f0c5c0397e0 msgr2=0x7f0c5c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:20.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 1 --2- 172.21.15.204:0/1771525616 >> v2:172.21.15.204:6812/28210 conn(0x7f0c5c0397e0 0x7f0c5c03bc90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009360 tx=0x7f0c74008040 comp rx=0 tx=0).stop 2024-09-15T01:36:20.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 10 monclient: shutdown 2024-09-15T01:36:20.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:20.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/1771525616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c7811e8f0 msgr2=0x7f0c7811ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:20.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 1 --2- 172.21.15.204:0/1771525616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c7811e8f0 0x7f0c7811ed40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0c7806f710 tx=0x7f0c70007f90 comp rx=0 tx=0).stop 2024-09-15T01:36:20.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/1771525616 shutdown_connections 2024-09-15T01:36:20.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 1 --2- 172.21.15.204:0/1771525616 >> v2:172.21.15.204:6812/28210 conn(0x7f0c5c0397e0 0x7f0c5c03bc90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:20.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 1 --2- 172.21.15.204:0/1771525616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c7811e8f0 0x7f0c7811ed40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:20.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 1 --2- 172.21.15.204:0/1771525616 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c78120530 0x7f0c7811e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:20.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 1 --2- 172.21.15.204:0/1771525616 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c64004cc0 0x7f0c7811de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:20.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/1771525616 >> 172.21.15.204:0/1771525616 conn(0x7f0c7806e860 msgr2=0x7f0c78132ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:20.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/1771525616 shutdown_connections 2024-09-15T01:36:20.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.907+0000 7f0c7fd9d640 1 -- 172.21.15.204:0/1771525616 wait complete. 2024-09-15T01:36:20.919 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool rm cool cool --yes-i-really-really-mean-it 2024-09-15T01:36:20.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.992+0000 7fe00c68b640 10 monclient: get_monmap_and_config 2024-09-15T01:36:20.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.992+0000 7fe00c68b640 10 monclient: build_initial_monmap 2024-09-15T01:36:20.995 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.992+0000 7fe00c68b640 10 monclient: monmap: 2024-09-15T01:36:20.995 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:20.995 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:20.995 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:20.993991+0000 2024-09-15T01:36:20.995 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:20.993991+0000 2024-09-15T01:36:20.995 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:20.996 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:20.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 1 Processor -- start 2024-09-15T01:36:20.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 1 -- start start 2024-09-15T01:36:20.997 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 10 monclient: init 2024-09-15T01:36:21.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:21.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 10 monclient: _add_conns ranks=[9,8,2,5,6,0,10,4,7,3,1] 2024-09-15T01:36:21.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe0040a2cd0 0x7fe004132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:21.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 10 monclient(hunting): picked mon.noname-a con 0x7fe0040a2cd0 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:21.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe00409fa20 0x7fe00409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:21.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 10 monclient(hunting): picked mon.noname-e con 0x7fe00409fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:21.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0040a08e0 0x7fe0040a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:21.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 10 monclient(hunting): picked mon.noname-b con 0x7fe0040a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:21.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:21.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0040a30a0 con 0x7fe0040a2cd0 2024-09-15T01:36:21.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:21.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0040a0330 con 0x7fe0040a08e0 2024-09-15T01:36:21.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:21.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe0040a1db0 con 0x7fe00409fa20 2024-09-15T01:36:21.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:21.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.993+0000 7fe00c68b640 10 monclient(hunting): authenticate will time out at 996.636751s 2024-09-15T01:36:21.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe00409fa20 0x7fe00409fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:21.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00a427640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe0040a2cd0 0x7fe004132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:21.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00ac28640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0040a08e0 0x7fe0040a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:21.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe00409fa20 0x7fe00409fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:47482/0 (socket says 172.21.15.204:47482) 2024-09-15T01:36:21.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00a427640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe0040a2cd0 0x7fe004132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.204:54678/0 (socket says 172.21.15.204:54678) 2024-09-15T01:36:21.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00ac28640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0040a08e0 0x7fe0040a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:48152/0 (socket says 172.21.15.204:48152) 2024-09-15T01:36:21.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 1 -- 172.21.15.204:0/3208372037 learned_addr learned my addr 172.21.15.204:0/3208372037 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:21.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 10 monclient(hunting): get_auth_request con 0x7fe00409fa20 auth_method 0 2024-09-15T01:36:21.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:21.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:21.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:21.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00ac28640 10 monclient(hunting): get_auth_request con 0x7fe0040a08e0 auth_method 0 2024-09-15T01:36:21.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00ac28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:21.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00ac28640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:21.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00ac28640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:21.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00a427640 10 monclient(hunting): get_auth_request con 0x7fe0040a2cd0 auth_method 0 2024-09-15T01:36:21.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00a427640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:21.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00a427640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:21.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00a427640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:21.008 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00ac28640 1 -- 172.21.15.204:0/3208372037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0040a08e0 msgr2=0x7fe0040a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00a427640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00a427640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00ac28640 1 -- 172.21.15.204:0/3208372037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0040a08e0 msgr2=0x7fe0040a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00ac28640 1 --2- 172.21.15.204:0/3208372037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0040a08e0 0x7fe0040a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00a427640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00ac28640 1 --2- 172.21.15.204:0/3208372037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0040a08e0 0x7fe0040a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 10 monclient(hunting): handle_auth_done global_id 5849 payload 274 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 1 -- 172.21.15.204:0/3208372037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0040a08e0 msgr2=0x7fe0040a0d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 1 --2- 172.21.15.204:0/3208372037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0040a08e0 0x7fe0040a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 1 -- 172.21.15.204:0/3208372037 >> v2:172.21.15.178:3300/0 conn(0x7fe0040a2cd0 msgr2=0x7fe004132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:21.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 1 --2- 172.21.15.204:0/3208372037 >> v2:172.21.15.178:3300/0 conn(0x7fe0040a2cd0 0x7fe004132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 10 monclient: _finish_hunting 0 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 1 monclient: found mon.noname-e 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe009c26640 1 -- 172.21.15.204:0/3208372037 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0040648a0 con 0x7fe00409fa20 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.994+0000 7fe00a427640 1 --2- 172.21.15.204:0/3208372037 >> v2:172.21.15.178:3300/0 conn(0x7fe0040a2cd0 0x7fe004132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009c26640 1 --2- 172.21.15.204:0/3208372037 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe00409fa20 0x7fe00409fdf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009910 tx=0x7fdff40091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e46dfd3bd0646225 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009425640 1 -- 172.21.15.204:0/3208372037 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdff402ac80 con 0x7fe00409fa20 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009425640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009425640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009425640 10 monclient: dump: 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:21.010 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:21.011 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:21.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009425640 10 monclient: _finish_auth 0 2024-09-15T01:36:21.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009425640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:21.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009425640 1 -- 172.21.15.204:0/3208372037 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff402ae00 con 0x7fe00409fa20 2024-09-15T01:36:21.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe00c68b640 5 monclient: authenticate success, global_id 5849 2024-09-15T01:36:21.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe00c68b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:21.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009425640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:21.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009425640 1 -- 172.21.15.204:0/3208372037 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdff4031ca0 con 0x7fe00409fa20 2024-09-15T01:36:21.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009425640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:21.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009425640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:21.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe009425640 10 monclient: dump: 2024-09-15T01:36:21.012 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:21.012 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:21.013 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe00c68b640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe00c68b640 4 set_mon_vals no callback set 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe00c68b640 10 monclient: shutdown 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe00c68b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe00c68b640 1 -- 172.21.15.204:0/3208372037 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe00409fa20 msgr2=0x7fe00409fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.995+0000 7fe00c68b640 1 --2- 172.21.15.204:0/3208372037 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe00409fa20 0x7fe00409fdf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009910 tx=0x7fdff40091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.996+0000 7fe00c68b640 1 -- 172.21.15.204:0/3208372037 shutdown_connections 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.996+0000 7fe00c68b640 1 --2- 172.21.15.204:0/3208372037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe0040a08e0 0x7fe0040a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.996+0000 7fe00c68b640 1 --2- 172.21.15.204:0/3208372037 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe00409fa20 0x7fe00409fdf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.996+0000 7fe00c68b640 1 --2- 172.21.15.204:0/3208372037 >> v2:172.21.15.178:3300/0 conn(0x7fe0040a2cd0 0x7fe004132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.996+0000 7fe00c68b640 1 -- 172.21.15.204:0/3208372037 >> 172.21.15.204:0/3208372037 conn(0x7fe00406e860 msgr2=0x7fe00406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.996+0000 7fe00c68b640 1 -- 172.21.15.204:0/3208372037 shutdown_connections 2024-09-15T01:36:21.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.996+0000 7fe00c68b640 1 -- 172.21.15.204:0/3208372037 wait complete. 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.996+0000 7fe00c68b640 10 monclient: build_initial_monmap 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.996+0000 7fe00c68b640 10 monclient: monmap: 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:21.015 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:21.016 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.996+0000 7fe00c68b640 1 Processor -- start 2024-09-15T01:36:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.996+0000 7fe00c68b640 1 -- start start 2024-09-15T01:36:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.996+0000 7fe00c68b640 10 monclient: init 2024-09-15T01:36:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 10 monclient: _add_conns ranks=[5,4,7,2,0,1,6,8,3] 2024-09-15T01:36:21.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe00409fa20 0x7fe004141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 10 monclient(hunting): picked mon.noname-h con 0x7fe00409fa20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe0040a08e0 0x7fe0041421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 10 monclient(hunting): picked mon.noname-c con 0x7fe0040a08e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0040a2cd0 0x7fe004142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 10 monclient(hunting): picked mon.noname-i con 0x7fe0040a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe00411cd20 con 0x7fe0040a08e0 2024-09-15T01:36:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe00409cb60 con 0x7fe00409fa20 2024-09-15T01:36:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe00411dec0 con 0x7fe0040a2cd0 2024-09-15T01:36:21.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:21.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00c68b640 10 monclient(hunting): authenticate will time out at 996.640342s 2024-09-15T01:36:21.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe00409fa20 0x7fe004141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:21.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe00409fa20 0x7fe004141ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:36940/0 (socket says 172.21.15.204:36940) 2024-09-15T01:36:21.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 1 -- 172.21.15.204:0/917537138 learned_addr learned my addr 172.21.15.204:0/917537138 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:21.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 10 monclient(hunting): get_auth_request con 0x7fe00409fa20 auth_method 0 2024-09-15T01:36:21.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:21.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:21.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:21.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00ac28640 1 --2- 172.21.15.204:0/917537138 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0040a2cd0 0x7fe004142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00ac28640 10 monclient(hunting): get_auth_request con 0x7fe0040a2cd0 auth_method 0 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00ac28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00ac28640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00ac28640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe009c26640 1 --2- 172.21.15.204:0/917537138 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe0040a08e0 0x7fe0041421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe009c26640 10 monclient(hunting): get_auth_request con 0x7fe0040a08e0 auth_method 0 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe009c26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe009c26640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe009c26640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00ac28640 1 -- 172.21.15.204:0/917537138 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0040a2cd0 msgr2=0x7fe004142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:36:21.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe009c26640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:21.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe009c26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:21.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00ac28640 1 -- 172.21.15.204:0/917537138 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0040a2cd0 msgr2=0x7fe004142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:36:21.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00ac28640 1 --2- 172.21.15.204:0/917537138 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0040a2cd0 0x7fe004142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:21.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe009c26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:21.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00ac28640 1 --2- 172.21.15.204:0/917537138 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0040a2cd0 0x7fe004142720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:21.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 10 monclient(hunting): handle_auth_done global_id 5864 payload 824 2024-09-15T01:36:21.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 1 -- 172.21.15.204:0/917537138 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0040a2cd0 msgr2=0x7fe004142720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:21.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 1 --2- 172.21.15.204:0/917537138 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0040a2cd0 0x7fe004142720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:21.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 1 -- 172.21.15.204:0/917537138 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe0040a08e0 msgr2=0x7fe0041421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:21.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 1 --2- 172.21.15.204:0/917537138 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe0040a08e0 0x7fe0041421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:21.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 10 monclient: _finish_hunting 0 2024-09-15T01:36:21.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 1 monclient: found mon.noname-h 2024-09-15T01:36:21.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:21.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:21.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 1 -- 172.21.15.204:0/917537138 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe000009390 con 0x7fe00409fa20 2024-09-15T01:36:21.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe009c26640 1 --2- 172.21.15.204:0/917537138 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe0040a08e0 0x7fe0041421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:21.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.997+0000 7fe00a427640 1 --2- 172.21.15.204:0/917537138 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe00409fa20 0x7fe004141ca0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe00000eaa0 tx=0x7fe0000090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:21.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 1 -- 172.21.15.204:0/917537138 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe00000ce90 con 0x7fe00409fa20 2024-09-15T01:36:21.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:21.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:21.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 10 monclient: dump: 2024-09-15T01:36:21.021 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:21.021 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:21.021 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:21.022 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:21.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:21.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:21.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 1 -- 172.21.15.204:0/917537138 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe000013470 con 0x7fe00409fa20 2024-09-15T01:36:21.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fe00c68b640 5 monclient: authenticate success, global_id 5864 2024-09-15T01:36:21.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fe00c68b640 10 monclient: _renew_subs 2024-09-15T01:36:21.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fe00c68b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:21.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fe00c68b640 1 -- 172.21.15.204:0/917537138 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff4009450 con 0x7fe00409fa20 2024-09-15T01:36:21.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 1 -- 172.21.15.204:0/917537138 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe000013a60 con 0x7fe00409fa20 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 10 monclient: dump: 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:21.024 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fe008c24640 4 set_mon_vals no callback set 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fe00c68b640 10 monclient: _renew_subs 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fe00c68b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fe00c68b640 1 -- 172.21.15.204:0/917537138 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00409c5a0 con 0x7fe00409fa20 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.998+0000 7fdffa7fc640 1 -- 172.21.15.204:0/917537138 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe000013bc0 con 0x7fe00409fa20 2024-09-15T01:36:21.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.999+0000 7fe00c68b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:21.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.999+0000 7fe00c68b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:21.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.999+0000 7fe00c68b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:21.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.999+0000 7fe00c68b640 1 -- 172.21.15.204:0/917537138 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe004070b80 con 0x7fe00409fa20 2024-09-15T01:36:21.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.999+0000 7fdffa7fc640 1 --2- 172.21.15.204:0/917537138 >> v2:172.21.15.204:6812/28210 conn(0x7fdfe80397b0 0x7fdfe803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:21.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.999+0000 7fdffa7fc640 1 -- 172.21.15.204:0/917537138 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 8345+0+0 (secure 0 0 0) 0x7fe00004c8a0 con 0x7fe00409fa20 2024-09-15T01:36:21.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.999+0000 7fe009c26640 1 --2- 172.21.15.204:0/917537138 >> v2:172.21.15.204:6812/28210 conn(0x7fdfe80397b0 0x7fdfe803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:21.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.999+0000 7fe009c26640 10 monclient: get_auth_request con 0x7fdfe80397b0 auth_method 0 2024-09-15T01:36:21.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:20.999+0000 7fe009c26640 1 --2- 172.21.15.204:0/917537138 >> v2:172.21.15.204:6812/28210 conn(0x7fdfe80397b0 0x7fdfe803bc60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdff4034000 tx=0x7fdff4032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:21.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:21.003+0000 7fdffa7fc640 1 -- 172.21.15.204:0/917537138 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe0000138c0 con 0x7fe00409fa20 2024-09-15T01:36:21.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:21.003+0000 7fdffa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:21.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:21.003+0000 7fdffa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:21.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:21.157+0000 7fe00c68b640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-09-15T01:36:21.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:21.157+0000 7fe00c68b640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-09-15T01:36:21.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:21.157+0000 7fe00c68b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:21.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:21.157+0000 7fe00c68b640 1 -- 172.21.15.204:0/917537138 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe00411e080 con 0x7fe00409fa20 2024-09-15T01:36:21.911 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:21.909+0000 7fdffa7fc640 1 -- 172.21.15.204:0/917537138 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}]=0 pool 'cool' removed v168) v1 ==== 148+0+0 (secure 0 0 0) 0x7fe00001b820 con 0x7fe00409fa20 2024-09-15T01:36:21.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:21.909+0000 7fdffa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-09-15T01:36:21.912 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:21.909+0000 7fdffa7fc640 10 monclient: _finish_command 2 = system:0 pool 'cool' removed 2024-09-15T01:36:22.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.004+0000 7fe00c68b640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-09-15T01:36:22.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.004+0000 7fe00c68b640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-09-15T01:36:22.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.004+0000 7fe00c68b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:22.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.004+0000 7fe00c68b640 1 -- 172.21.15.204:0/917537138 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe0040008d0 con 0x7fe00409fa20 2024-09-15T01:36:22.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.004+0000 7fdffa7fc640 1 -- 172.21.15.204:0/917537138 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}]=0 pool 'cool' does not exist v168) v1 ==== 155+0+0 (secure 0 0 0) 0x7fe00001ba00 con 0x7fe00409fa20 2024-09-15T01:36:22.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.004+0000 7fdffa7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-09-15T01:36:22.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.004+0000 7fdffa7fc640 10 monclient: _finish_command 3 = system:0 pool 'cool' does not exist 2024-09-15T01:36:22.007 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'cool' does not exist 2024-09-15T01:36:22.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 1 -- 172.21.15.204:0/917537138 >> v2:172.21.15.204:6812/28210 conn(0x7fdfe80397b0 msgr2=0x7fdfe803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:22.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 1 --2- 172.21.15.204:0/917537138 >> v2:172.21.15.204:6812/28210 conn(0x7fdfe80397b0 0x7fdfe803bc60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdff4034000 tx=0x7fdff4032000 comp rx=0 tx=0).stop 2024-09-15T01:36:22.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 10 monclient: shutdown 2024-09-15T01:36:22.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:22.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 1 -- 172.21.15.204:0/917537138 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe00409fa20 msgr2=0x7fe004141ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:22.009 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 1 --2- 172.21.15.204:0/917537138 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe00409fa20 0x7fe004141ca0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe00000eaa0 tx=0x7fe0000090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 1 -- 172.21.15.204:0/917537138 shutdown_connections 2024-09-15T01:36:22.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 1 --2- 172.21.15.204:0/917537138 >> v2:172.21.15.204:6812/28210 conn(0x7fdfe80397b0 0x7fdfe803bc60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 1 --2- 172.21.15.204:0/917537138 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe0040a2cd0 0x7fe004142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 1 --2- 172.21.15.204:0/917537138 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe0040a08e0 0x7fe0041421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 1 --2- 172.21.15.204:0/917537138 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe00409fa20 0x7fe004141ca0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 1 -- 172.21.15.204:0/917537138 >> 172.21.15.204:0/917537138 conn(0x7fe00406e860 msgr2=0x7fe0040a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:22.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 1 -- 172.21.15.204:0/917537138 shutdown_connections 2024-09-15T01:36:22.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.006+0000 7fe00c68b640 1 -- 172.21.15.204:0/917537138 wait complete. 2024-09-15T01:36:22.018 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd pool rm cold cold --yes-i-really-really-mean-it 2024-09-15T01:36:22.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.091+0000 7f12fc59d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:22.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.091+0000 7f12fc59d640 10 monclient: build_initial_monmap 2024-09-15T01:36:22.093 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.091+0000 7f12fc59d640 10 monclient: monmap: 2024-09-15T01:36:22.093 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:22.093 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:22.092522+0000 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:22.092522+0000 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:22.094 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:22.095 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:22.095 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.091+0000 7f12fc59d640 1 Processor -- start 2024-09-15T01:36:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.091+0000 7f12fc59d640 1 -- start start 2024-09-15T01:36:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.091+0000 7f12fc59d640 10 monclient: init 2024-09-15T01:36:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 10 monclient: _add_conns ranks=[7,3,4,5,1,2,0,6,10,8,9] 2024-09-15T01:36:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12f40a2cd0 0x7f12f4132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 10 monclient(hunting): picked mon.noname-i con 0x7f12f40a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12f409fa20 0x7f12f409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 10 monclient(hunting): picked mon.noname-g con 0x7f12f409fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f12f40a08e0 0x7f12f40a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 10 monclient(hunting): picked mon.noname-c con 0x7f12f40a08e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40a30a0 con 0x7f12f409fa20 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40a0330 con 0x7f12f40a08e0 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40a1db0 con 0x7f12f40a2cd0 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fc59d640 10 monclient(hunting): authenticate will time out at 997.735299s 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fa339640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12f40a2cd0 0x7f12f4132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12f409fa20 0x7f12f409fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12f409fa20 0x7f12f409fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:56024/0 (socket says 172.21.15.204:56024) 2024-09-15T01:36:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fa339640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12f40a2cd0 0x7f12f4132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:47598/0 (socket says 172.21.15.204:47598) 2024-09-15T01:36:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 1 -- 172.21.15.204:0/4294689645 learned_addr learned my addr 172.21.15.204:0/4294689645 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 10 monclient(hunting): get_auth_request con 0x7f12f409fa20 auth_method 0 2024-09-15T01:36:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fa339640 10 monclient(hunting): get_auth_request con 0x7f12f40a2cd0 auth_method 0 2024-09-15T01:36:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fa339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fa339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fa339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fa339640 1 -- 172.21.15.204:0/4294689645 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12f40a2cd0 msgr2=0x7f12f4132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fa339640 1 -- 172.21.15.204:0/4294689645 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12f40a2cd0 msgr2=0x7f12f4132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fa339640 1 --2- 172.21.15.204:0/4294689645 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12f40a2cd0 0x7f12f4132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12fa339640 1 --2- 172.21.15.204:0/4294689645 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12f40a2cd0 0x7f12f4132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 10 monclient(hunting): handle_auth_done global_id 14961 payload 274 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 1 -- 172.21.15.204:0/4294689645 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12f40a2cd0 msgr2=0x7f12f4132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 1 --2- 172.21.15.204:0/4294689645 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12f40a2cd0 0x7f12f4132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 1 -- 172.21.15.204:0/4294689645 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f12f40a08e0 msgr2=0x7f12f40a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 1 --2- 172.21.15.204:0/4294689645 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f12f40a08e0 0x7f12f40a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 10 monclient: _finish_hunting 0 2024-09-15T01:36:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 1 monclient: found mon.noname-g 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.092+0000 7f12f9b38640 1 -- 172.21.15.204:0/4294689645 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f40648a0 con 0x7f12f409fa20 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9b38640 1 --2- 172.21.15.204:0/4294689645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12f409fa20 0x7f12f409fdf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009940 tx=0x7f12e40091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=72d76aebedfd6a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9337640 1 -- 172.21.15.204:0/4294689645 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12e402ac60 con 0x7f12f409fa20 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9337640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9337640 10 monclient: dump: 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9337640 10 monclient: _finish_auth 0 2024-09-15T01:36:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9337640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12fc59d640 5 monclient: authenticate success, global_id 14961 2024-09-15T01:36:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12fc59d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9337640 1 -- 172.21.15.204:0/4294689645 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e402ade0 con 0x7f12f409fa20 2024-09-15T01:36:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9337640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12fc59d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12fc59d640 4 set_mon_vals no callback set 2024-09-15T01:36:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9337640 1 -- 172.21.15.204:0/4294689645 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12e4031ca0 con 0x7f12f409fa20 2024-09-15T01:36:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9337640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.093+0000 7f12f9337640 10 monclient: dump: 2024-09-15T01:36:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 10 monclient: shutdown 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 1 -- 172.21.15.204:0/4294689645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12f409fa20 msgr2=0x7f12f409fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 1 --2- 172.21.15.204:0/4294689645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12f409fa20 0x7f12f409fdf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009940 tx=0x7f12e40091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 1 -- 172.21.15.204:0/4294689645 shutdown_connections 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 1 --2- 172.21.15.204:0/4294689645 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f12f40a08e0 0x7f12f40a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 1 --2- 172.21.15.204:0/4294689645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12f409fa20 0x7f12f409fdf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 1 --2- 172.21.15.204:0/4294689645 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f12f40a2cd0 0x7f12f4132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 1 -- 172.21.15.204:0/4294689645 >> 172.21.15.204:0/4294689645 conn(0x7f12f406e860 msgr2=0x7f12f406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 1 -- 172.21.15.204:0/4294689645 shutdown_connections 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 1 -- 172.21.15.204:0/4294689645 wait complete. 2024-09-15T01:36:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 10 monclient: build_initial_monmap 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.094+0000 7f12fc59d640 10 monclient: monmap: 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 1 Processor -- start 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 1 -- start start 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 10 monclient: init 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 10 monclient: _add_conns ranks=[4,2,1,3,0,8,5,6,7] 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f12f409fa20 0x7f12f4141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 10 monclient(hunting): picked mon.noname-c con 0x7f12f409fa20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f40a08e0 0x7f12f41310e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 10 monclient(hunting): picked mon.noname-b con 0x7f12f40a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f40a2cd0 0x7f12f411dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 10 monclient(hunting): picked mon.noname-f con 0x7f12f40a2cd0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f4121510 con 0x7f12f40a2cd0 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f409d3d0 con 0x7f12f40a08e0 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12f409cb60 con 0x7f12f409fa20 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fc59d640 10 monclient(hunting): authenticate will time out at 997.738542s 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f40a2cd0 0x7f12f411dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f40a2cd0 0x7f12f411dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:52792/0 (socket says 172.21.15.204:52792) 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 1 -- 172.21.15.204:0/1965971000 learned_addr learned my addr 172.21.15.204:0/1965971000 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 10 monclient(hunting): get_auth_request con 0x7f12f40a2cd0 auth_method 0 2024-09-15T01:36:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fa339640 1 --2- 172.21.15.204:0/1965971000 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f12f409fa20 0x7f12f4141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12f9b38640 1 --2- 172.21.15.204:0/1965971000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f40a08e0 0x7f12f41310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fa339640 10 monclient(hunting): get_auth_request con 0x7f12f409fa20 auth_method 0 2024-09-15T01:36:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fa339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fa339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fa339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12f9b38640 10 monclient(hunting): get_auth_request con 0x7f12f40a08e0 auth_method 0 2024-09-15T01:36:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12f9b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12f9b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12f9b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fa339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fa339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12f9b38640 1 -- 172.21.15.204:0/1965971000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f40a08e0 msgr2=0x7f12f41310e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12f9b38640 1 -- 172.21.15.204:0/1965971000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f40a08e0 msgr2=0x7f12f41310e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12f9b38640 1 --2- 172.21.15.204:0/1965971000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f40a08e0 0x7f12f41310e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fa339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12f9b38640 1 --2- 172.21.15.204:0/1965971000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f40a08e0 0x7f12f41310e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 10 monclient(hunting): handle_auth_done global_id 6553 payload 824 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 1 -- 172.21.15.204:0/1965971000 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f12f409fa20 msgr2=0x7f12f4141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 1 --2- 172.21.15.204:0/1965971000 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f12f409fa20 0x7f12f4141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 1 -- 172.21.15.204:0/1965971000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f40a08e0 msgr2=0x7f12f41310e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 1 --2- 172.21.15.204:0/1965971000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f40a08e0 0x7f12f41310e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 10 monclient: _finish_hunting 0 2024-09-15T01:36:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.095+0000 7f12fab3a640 1 monclient: found mon.noname-f 2024-09-15T01:36:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12fab3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12fab3a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12fab3a640 1 -- 172.21.15.204:0/1965971000 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e4009450 con 0x7f12f40a2cd0 2024-09-15T01:36:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12fab3a640 1 --2- 172.21.15.204:0/1965971000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f40a2cd0 0x7f12f411dca0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00bbc0 tx=0x7f12ec009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12ea7fc640 1 -- 172.21.15.204:0/1965971000 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12ec014410 con 0x7f12f40a2cd0 2024-09-15T01:36:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12ea7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12ea7fc640 10 monclient: dump: 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:22.120 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12ea7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12ea7fc640 1 -- 172.21.15.204:0/1965971000 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec0145b0 con 0x7f12f40a2cd0 2024-09-15T01:36:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12fc59d640 5 monclient: authenticate success, global_id 6553 2024-09-15T01:36:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12fc59d640 10 monclient: _renew_subs 2024-09-15T01:36:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12fc59d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12fc59d640 1 -- 172.21.15.204:0/1965971000 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f4141fc0 con 0x7f12f40a2cd0 2024-09-15T01:36:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12ea7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12ea7fc640 1 -- 172.21.15.204:0/1965971000 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12ec014ba0 con 0x7f12f40a2cd0 2024-09-15T01:36:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12ea7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12ea7fc640 10 monclient: dump: 2024-09-15T01:36:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:22.122 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:22.122 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:22.122 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:22.122 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:22.122 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:22.122 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:22.122 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:22.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12f8b36640 4 set_mon_vals no callback set 2024-09-15T01:36:22.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12fc59d640 10 monclient: _renew_subs 2024-09-15T01:36:22.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12fc59d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:22.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.096+0000 7f12fc59d640 1 -- 172.21.15.204:0/1965971000 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f41427a0 con 0x7f12f40a2cd0 2024-09-15T01:36:22.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.097+0000 7f12ea7fc640 1 -- 172.21.15.204:0/1965971000 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f12ec014d20 con 0x7f12f40a2cd0 2024-09-15T01:36:22.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.097+0000 7f12fc59d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:22.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.097+0000 7f12fc59d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:22.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.097+0000 7f12fc59d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:22.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.097+0000 7f12fc59d640 1 -- 172.21.15.204:0/1965971000 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f4142960 con 0x7f12f40a2cd0 2024-09-15T01:36:22.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.097+0000 7f12ea7fc640 1 --2- 172.21.15.204:0/1965971000 >> v2:172.21.15.204:6812/28210 conn(0x7f12d80397b0 0x7f12d803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.097+0000 7f12ea7fc640 1 -- 172.21.15.204:0/1965971000 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 7674+0+0 (secure 0 0 0) 0x7f12ec04c090 con 0x7f12f40a2cd0 2024-09-15T01:36:22.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.097+0000 7f12fa339640 1 --2- 172.21.15.204:0/1965971000 >> v2:172.21.15.204:6812/28210 conn(0x7f12d80397b0 0x7f12d803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.097+0000 7f12fa339640 10 monclient: get_auth_request con 0x7f12d80397b0 auth_method 0 2024-09-15T01:36:22.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.097+0000 7f12fa339640 1 --2- 172.21.15.204:0/1965971000 >> v2:172.21.15.204:6812/28210 conn(0x7f12d80397b0 0x7f12d803bc60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f12f0006fd0 tx=0x7f12f0008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:22.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.101+0000 7f12ea7fc640 1 -- 172.21.15.204:0/1965971000 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f12f4142960 con 0x7f12f40a2cd0 2024-09-15T01:36:22.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.101+0000 7f12ea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:22.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.101+0000 7f12ea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:22.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.260+0000 7f12fc59d640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-09-15T01:36:22.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.260+0000 7f12fc59d640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-09-15T01:36:22.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.260+0000 7f12fc59d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:22.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.260+0000 7f12fc59d640 1 -- 172.21.15.204:0/1965971000 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f12f4142960 con 0x7f12f40a2cd0 2024-09-15T01:36:22.269 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:36:22.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.369+0000 7f37b5f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:22.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.369+0000 7f37b5f9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:22.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.369+0000 7f37b5f9d640 10 monclient: monmap: 2024-09-15T01:36:22.373 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:22.373 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:22.373 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:22.371763+0000 2024-09-15T01:36:22.373 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:22.371763+0000 2024-09-15T01:36:22.373 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:22.373 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:22.373 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:22.373 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:22.373 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:22.373 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:22.374 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:22.374 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:22.374 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:22.374 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:22.374 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:22.374 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:22.374 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:22.374 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.370+0000 7f37b5f9d640 1 Processor -- start 2024-09-15T01:36:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.370+0000 7f37b5f9d640 1 -- start start 2024-09-15T01:36:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.370+0000 7f37b5f9d640 10 monclient: init 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 10 monclient: _add_conns ranks=[7,4,3,10,2,9,1,6,8,5,0] 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f37b00a2930 0x7f37b0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 10 monclient(hunting): picked mon.noname-i con 0x7f37b00a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b009faa0 0x7f37b009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 10 monclient(hunting): picked mon.noname-c con 0x7f37b009faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37b00a08c0 0x7f37b00a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 10 monclient(hunting): picked mon.noname-g con 0x7f37b00a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b009e0f0 con 0x7f37b00a08c0 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37b009dd80 con 0x7f37b009faa0 2024-09-15T01:36:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00a2d00 con 0x7f37b00a2930 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37b5f9d640 10 monclient(hunting): authenticate will time out at 987.122500s 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b009faa0 0x7f37b009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b009faa0 0x7f37b009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44072/0 (socket says 172.21.15.178:44072) 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 1 -- 172.21.15.178:0/3436799529 learned_addr learned my addr 172.21.15.178:0/3436799529 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 10 monclient(hunting): get_auth_request con 0x7f37b009faa0 auth_method 0 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37affff640 1 --2- 172.21.15.178:0/3436799529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37b00a08c0 0x7f37b00a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37af7fe640 1 --2- 172.21.15.178:0/3436799529 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f37b00a2930 0x7f37b0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37af7fe640 10 monclient(hunting): get_auth_request con 0x7f37b00a2930 auth_method 0 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37affff640 10 monclient(hunting): get_auth_request con 0x7f37b00a08c0 auth_method 0 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37af7fe640 1 -- 172.21.15.178:0/3436799529 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f37b00a2930 msgr2=0x7f37b0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37af7fe640 1 -- 172.21.15.178:0/3436799529 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f37b00a2930 msgr2=0x7f37b0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37af7fe640 1 --2- 172.21.15.178:0/3436799529 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f37b00a2930 0x7f37b0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37af7fe640 1 --2- 172.21.15.178:0/3436799529 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f37b00a2930 0x7f37b0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 10 monclient(hunting): handle_auth_done global_id 5782 payload 274 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 1 -- 172.21.15.178:0/3436799529 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f37b00a2930 msgr2=0x7f37b0132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 1 --2- 172.21.15.178:0/3436799529 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f37b00a2930 0x7f37b0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 1 -- 172.21.15.178:0/3436799529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37b00a08c0 msgr2=0x7f37b00a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 1 --2- 172.21.15.178:0/3436799529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37b00a08c0 0x7f37b00a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 1 monclient: found mon.noname-c 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.371+0000 7f37a7fff640 1 -- 172.21.15.178:0/3436799529 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b004bf60 con 0x7f37b009faa0 2024-09-15T01:36:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37a7fff640 1 --2- 172.21.15.178:0/3436799529 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b009faa0 0x7f37b009fe70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3798009940 tx=0x7f37980091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c81bb3c9050530da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37aeffd640 1 -- 172.21.15.178:0/3436799529 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f379802ac60 con 0x7f37b009faa0 2024-09-15T01:36:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37aeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37aeffd640 10 monclient: dump: 2024-09-15T01:36:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37aeffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37aeffd640 1 -- 172.21.15.178:0/3436799529 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379802ade0 con 0x7f37b009faa0 2024-09-15T01:36:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37b5f9d640 5 monclient: authenticate success, global_id 5782 2024-09-15T01:36:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37b5f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37aeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37b5f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37aeffd640 1 -- 172.21.15.178:0/3436799529 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3798031ca0 con 0x7f37b009faa0 2024-09-15T01:36:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37b5f9d640 4 set_mon_vals no callback set 2024-09-15T01:36:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37aeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.372+0000 7f37aeffd640 10 monclient: dump: 2024-09-15T01:36:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.373+0000 7f37b5f9d640 10 monclient: shutdown 2024-09-15T01:36:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.373+0000 7f37b5f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.373+0000 7f37b5f9d640 1 -- 172.21.15.178:0/3436799529 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b009faa0 msgr2=0x7f37b009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.373+0000 7f37b5f9d640 1 --2- 172.21.15.178:0/3436799529 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b009faa0 0x7f37b009fe70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3798009940 tx=0x7f37980091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.373+0000 7f37b5f9d640 1 -- 172.21.15.178:0/3436799529 shutdown_connections 2024-09-15T01:36:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.373+0000 7f37b5f9d640 1 --2- 172.21.15.178:0/3436799529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37b00a08c0 0x7f37b00a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.373+0000 7f37b5f9d640 1 --2- 172.21.15.178:0/3436799529 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b009faa0 0x7f37b009fe70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.373+0000 7f37b5f9d640 1 --2- 172.21.15.178:0/3436799529 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f37b00a2930 0x7f37b0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.373+0000 7f37b5f9d640 1 -- 172.21.15.178:0/3436799529 >> 172.21.15.178:0/3436799529 conn(0x7f37b00a5ed0 msgr2=0x7f37b00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.373+0000 7f37b5f9d640 1 -- 172.21.15.178:0/3436799529 shutdown_connections 2024-09-15T01:36:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.373+0000 7f37b5f9d640 1 -- 172.21.15.178:0/3436799529 wait complete. 2024-09-15T01:36:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient: monmap: 2024-09-15T01:36:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 1 Processor -- start 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 1 -- start start 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient: init 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient: _add_conns ranks=[6,4,5,8,2,7,1,0,3] 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f37b009faa0 0x7f37b009b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient(hunting): picked mon.noname-d con 0x7f37b009faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b00a08c0 0x7f37b0131a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient(hunting): picked mon.noname-c con 0x7f37b00a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f37b00a2930 0x7f37b011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient(hunting): picked mon.noname-h con 0x7f37b00a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37b011d9f0 con 0x7f37b00a08c0 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37b009d630 con 0x7f37b00a2930 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00a2390 con 0x7f37b009faa0 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.374+0000 7f37b5f9d640 10 monclient(hunting): authenticate will time out at 987.125938s 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b00a08c0 0x7f37b0131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37af7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f37b009faa0 0x7f37b009b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b00a08c0 0x7f37b0131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44074/0 (socket says 172.21.15.178:44074) 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37af7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f37b009faa0 0x7f37b009b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:60070/0 (socket says 172.21.15.178:60070) 2024-09-15T01:36:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 1 -- 172.21.15.178:0/349970000 learned_addr learned my addr 172.21.15.178:0/349970000 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 10 monclient(hunting): get_auth_request con 0x7f37b00a08c0 auth_method 0 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37af7fe640 10 monclient(hunting): get_auth_request con 0x7f37b009faa0 auth_method 0 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37affff640 1 --2- 172.21.15.178:0/349970000 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f37b00a2930 0x7f37b011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37affff640 10 monclient(hunting): get_auth_request con 0x7f37b00a2930 auth_method 0 2024-09-15T01:36:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37af7fe640 1 -- 172.21.15.178:0/349970000 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f37b009faa0 msgr2=0x7f37b009b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37af7fe640 1 -- 172.21.15.178:0/349970000 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f37b009faa0 msgr2=0x7f37b009b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37af7fe640 1 --2- 172.21.15.178:0/349970000 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f37b009faa0 0x7f37b009b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37af7fe640 1 --2- 172.21.15.178:0/349970000 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f37b009faa0 0x7f37b009b3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 10 monclient(hunting): handle_auth_done global_id 5791 payload 824 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 1 -- 172.21.15.178:0/349970000 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f37b009faa0 msgr2=0x7f37b009b3c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 1 --2- 172.21.15.178:0/349970000 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f37b009faa0 0x7f37b009b3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 1 -- 172.21.15.178:0/349970000 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f37b00a2930 msgr2=0x7f37b011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 1 --2- 172.21.15.178:0/349970000 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f37b00a2930 0x7f37b011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 1 monclient: found mon.noname-c 2024-09-15T01:36:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 1 -- 172.21.15.178:0/349970000 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798009450 con 0x7f37b00a08c0 2024-09-15T01:36:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a7fff640 1 --2- 172.21.15.178:0/349970000 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b00a08c0 0x7f37b0131a00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3798000c00 tx=0x7f3798032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a77fe640 1 -- 172.21.15.178:0/349970000 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3798031820 con 0x7f37b00a08c0 2024-09-15T01:36:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a77fe640 10 monclient: dump: 2024-09-15T01:36:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:22.398 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a77fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a77fe640 1 -- 172.21.15.178:0/349970000 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37980319a0 con 0x7f37b00a08c0 2024-09-15T01:36:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37b5f9d640 5 monclient: authenticate success, global_id 5791 2024-09-15T01:36:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37b5f9d640 10 monclient: _renew_subs 2024-09-15T01:36:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37b5f9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:36:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37b5f9d640 1 -- 172.21.15.178:0/349970000 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b011dbc0 con 0x7f37b00a08c0 2024-09-15T01:36:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a77fe640 1 -- 172.21.15.178:0/349970000 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f379803a390 con 0x7f37b00a08c0 2024-09-15T01:36:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37a77fe640 10 monclient: dump: 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:22.401 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37b5f9d640 10 monclient: _renew_subs 2024-09-15T01:36:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37b5f9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:36:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37ae7fc640 4 set_mon_vals no callback set 2024-09-15T01:36:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.375+0000 7f37b5f9d640 1 -- 172.21.15.178:0/349970000 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b009bf10 con 0x7f37b00a08c0 2024-09-15T01:36:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.376+0000 7f37a77fe640 1 -- 172.21.15.178:0/349970000 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3798039040 con 0x7f37b00a08c0 2024-09-15T01:36:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.376+0000 7f37b5f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.376+0000 7f37b5f9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.376+0000 7f37b5f9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:36:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.376+0000 7f37b5f9d640 1 -- 172.21.15.178:0/349970000 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b011ee30 con 0x7f37b00a08c0 2024-09-15T01:36:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.377+0000 7f37a77fe640 1 --2- 172.21.15.178:0/349970000 >> v2:172.21.15.204:6812/28210 conn(0x7f37900397b0 0x7f379003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:22.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.377+0000 7f37a77fe640 1 -- 172.21.15.178:0/349970000 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 7674+0+0 (secure 0 0 0) 0x7f379806b340 con 0x7f37b00a08c0 2024-09-15T01:36:22.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.377+0000 7f37af7fe640 1 --2- 172.21.15.178:0/349970000 >> v2:172.21.15.204:6812/28210 conn(0x7f37900397b0 0x7f379003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:22.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.377+0000 7f37af7fe640 10 monclient: get_auth_request con 0x7f37900397b0 auth_method 0 2024-09-15T01:36:22.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.377+0000 7f37af7fe640 1 --2- 172.21.15.178:0/349970000 >> v2:172.21.15.204:6812/28210 conn(0x7f37900397b0 0x7f379003bc60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f37a0006fd0 tx=0x7f37a0008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:22.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.381+0000 7f37a77fe640 1 -- 172.21.15.178:0/349970000 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f379802fa80 con 0x7f37b00a08c0 2024-09-15T01:36:22.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.381+0000 7f37a77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:22.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.381+0000 7f37a77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:22.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.598+0000 7f37b5f9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:36:22.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.598+0000 7f37b5f9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:22.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.598+0000 7f37b5f9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:36:22.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.598+0000 7f37b5f9d640 1 -- 172.21.15.178:0/349970000 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f37b004b720 con 0x7f37b00a08c0 2024-09-15T01:36:22.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.599+0000 7f37a77fe640 1 -- 172.21.15.178:0/349970000 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3177 (secure 0 0 0) 0x7f3798041020 con 0x7f37b00a08c0 2024-09-15T01:36:22.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.599+0000 7f37a77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:22.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.599+0000 7f37a77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:22.602 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":48,"quorum":[0,1,2,3,4,5,7,8],"quorum_names":["a","f","b","g","c","h","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:36:22.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 1 -- 172.21.15.178:0/349970000 >> v2:172.21.15.204:6812/28210 conn(0x7f37900397b0 msgr2=0x7f379003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:22.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 1 --2- 172.21.15.178:0/349970000 >> v2:172.21.15.204:6812/28210 conn(0x7f37900397b0 0x7f379003bc60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f37a0006fd0 tx=0x7f37a0008040 comp rx=0 tx=0).stop 2024-09-15T01:36:22.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 10 monclient: shutdown 2024-09-15T01:36:22.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:22.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 1 -- 172.21.15.178:0/349970000 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b00a08c0 msgr2=0x7f37b0131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:22.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 1 --2- 172.21.15.178:0/349970000 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b00a08c0 0x7f37b0131a00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3798000c00 tx=0x7f3798032000 comp rx=0 tx=0).stop 2024-09-15T01:36:22.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 1 -- 172.21.15.178:0/349970000 shutdown_connections 2024-09-15T01:36:22.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 1 --2- 172.21.15.178:0/349970000 >> v2:172.21.15.204:6812/28210 conn(0x7f37900397b0 0x7f379003bc60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 1 --2- 172.21.15.178:0/349970000 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f37b00a2930 0x7f37b011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 1 --2- 172.21.15.178:0/349970000 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f37b00a08c0 0x7f37b0131a00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 1 --2- 172.21.15.178:0/349970000 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f37b009faa0 0x7f37b009b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:22.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 1 -- 172.21.15.178:0/349970000 >> 172.21.15.178:0/349970000 conn(0x7f37b00a5ed0 msgr2=0x7f37b009e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:22.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 1 -- 172.21.15.178:0/349970000 shutdown_connections 2024-09-15T01:36:22.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:22.601+0000 7f37b5f9d640 1 -- 172.21.15.178:0/349970000 wait complete. 2024-09-15T01:36:22.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.923+0000 7f12ea7fc640 1 -- 172.21.15.204:0/1965971000 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}]=0 pool 'cold' removed v169) v1 ==== 148+0+0 (secure 0 0 0) 0x7f12f4142960 con 0x7f12f40a2cd0 2024-09-15T01:36:22.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.923+0000 7f12ea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-09-15T01:36:22.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:22.923+0000 7f12ea7fc640 10 monclient: _finish_command 2 = system:0 pool 'cold' removed 2024-09-15T01:36:23.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.017+0000 7f12fc59d640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-09-15T01:36:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.017+0000 7f12fc59d640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-09-15T01:36:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.017+0000 7f12fc59d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.017+0000 7f12fc59d640 1 -- 172.21.15.204:0/1965971000 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f12f4142930 con 0x7f12f40a2cd0 2024-09-15T01:36:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.018+0000 7f12ea7fc640 1 -- 172.21.15.204:0/1965971000 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}]=0 pool 'cold' does not exist v169) v1 ==== 155+0+0 (secure 0 0 0) 0x7f12ec0149b0 con 0x7f12f40a2cd0 2024-09-15T01:36:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.018+0000 7f12ea7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-09-15T01:36:23.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.018+0000 7f12ea7fc640 10 monclient: _finish_command 3 = system:0 pool 'cold' does not exist 2024-09-15T01:36:23.021 INFO:tasks.workunit.client.0.smithi204.stderr:pool 'cold' does not exist 2024-09-15T01:36:23.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 1 -- 172.21.15.204:0/1965971000 >> v2:172.21.15.204:6812/28210 conn(0x7f12d80397b0 msgr2=0x7f12d803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:23.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 1 --2- 172.21.15.204:0/1965971000 >> v2:172.21.15.204:6812/28210 conn(0x7f12d80397b0 0x7f12d803bc60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f12f0006fd0 tx=0x7f12f0008040 comp rx=0 tx=0).stop 2024-09-15T01:36:23.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 10 monclient: shutdown 2024-09-15T01:36:23.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:23.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 1 -- 172.21.15.204:0/1965971000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f40a2cd0 msgr2=0x7f12f411dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:23.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 1 --2- 172.21.15.204:0/1965971000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f40a2cd0 0x7f12f411dca0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00bbc0 tx=0x7f12ec009810 comp rx=0 tx=0).stop 2024-09-15T01:36:23.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 1 -- 172.21.15.204:0/1965971000 shutdown_connections 2024-09-15T01:36:23.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 1 --2- 172.21.15.204:0/1965971000 >> v2:172.21.15.204:6812/28210 conn(0x7f12d80397b0 0x7f12d803bc60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:23.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 1 --2- 172.21.15.204:0/1965971000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f40a2cd0 0x7f12f411dca0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:23.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 1 --2- 172.21.15.204:0/1965971000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f40a08e0 0x7f12f41310e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:23.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 1 --2- 172.21.15.204:0/1965971000 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f12f409fa20 0x7f12f4141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:23.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 1 -- 172.21.15.204:0/1965971000 >> 172.21.15.204:0/1965971000 conn(0x7f12f406e860 msgr2=0x7f12f40a5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:23.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 1 -- 172.21.15.204:0/1965971000 shutdown_connections 2024-09-15T01:36:23.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.020+0000 7f12fc59d640 1 -- 172.21.15.204:0/1965971000 wait complete. 2024-09-15T01:36:23.031 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set rm-compat 2024-09-15T01:36:23.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.103+0000 7fef7ba94640 10 monclient: get_monmap_and_config 2024-09-15T01:36:23.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.103+0000 7fef7ba94640 10 monclient: build_initial_monmap 2024-09-15T01:36:23.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.103+0000 7fef7ba94640 10 monclient: monmap: 2024-09-15T01:36:23.106 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:23.106 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:23.106 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:23.104910+0000 2024-09-15T01:36:23.106 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:23.104910+0000 2024-09-15T01:36:23.106 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:23.106 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:23.106 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:23.106 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:23.106 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:23.107 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:23.107 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:23.107 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:23.107 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:23.107 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:23.107 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:23.107 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:23.107 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:23.107 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:23.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.103+0000 7fef7ba94640 1 Processor -- start 2024-09-15T01:36:23.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 1 -- start start 2024-09-15T01:36:23.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 10 monclient: init 2024-09-15T01:36:23.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:23.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 10 monclient: _add_conns ranks=[0,4,6,2,7,8,10,5,3,1,9] 2024-09-15T01:36:23.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fef7406e840 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:23.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef7409f9d0 0x7fef74130f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:23.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 10 monclient(hunting): picked mon.noname-c con 0x7fef7409f9d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:23.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fef74135ba0 0x7fef74137f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:23.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 10 monclient(hunting): picked mon.noname-d con 0x7fef74135ba0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:23.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7404b720 con 0x7fef7406e840 2024-09-15T01:36:23.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:23.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef7406dd60 con 0x7fef7409f9d0 2024-09-15T01:36:23.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:23.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fef7406da80 con 0x7fef74135ba0 2024-09-15T01:36:23.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:23.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7ba94640 10 monclient(hunting): authenticate will time out at 998.747734s 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7902f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef7409f9d0 0x7fef74130f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef79830640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fef7406e840 0x7fef74130870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.204:48404/0 (socket says 172.21.15.204:48404) 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef79830640 1 -- 172.21.15.204:0/132094944 learned_addr learned my addr 172.21.15.204:0/132094944 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7902f640 10 monclient(hunting): get_auth_request con 0x7fef7409f9d0 auth_method 0 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7902f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7902f640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7902f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7a031640 1 --2- 172.21.15.204:0/132094944 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fef74135ba0 0x7fef74137f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7a031640 10 monclient(hunting): get_auth_request con 0x7fef74135ba0 auth_method 0 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7a031640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7a031640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.104+0000 7fef7a031640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:23.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7902f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:23.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7902f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:23.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7902f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:23.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7a031640 1 -- 172.21.15.204:0/132094944 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fef74135ba0 msgr2=0x7fef74137f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:36:23.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7a031640 1 -- 172.21.15.204:0/132094944 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fef74135ba0 msgr2=0x7fef74137f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:23.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7a031640 1 --2- 172.21.15.204:0/132094944 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fef74135ba0 0x7fef74137f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:23.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7a031640 1 --2- 172.21.15.204:0/132094944 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fef74135ba0 0x7fef74137f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:23.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7882e640 1 -- 172.21.15.204:0/132094944 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953126837 0 0) 0x7fef7404b720 con 0x7fef7406e840 2024-09-15T01:36:23.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7902f640 10 monclient(hunting): handle_auth_done global_id 5800 payload 274 2024-09-15T01:36:23.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7902f640 1 -- 172.21.15.204:0/132094944 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fef74135ba0 msgr2=0x7fef74137f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:23.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7902f640 1 --2- 172.21.15.204:0/132094944 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fef74135ba0 0x7fef74137f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:23.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7902f640 1 -- 172.21.15.204:0/132094944 >> v1:172.21.15.178:6789/0 conn(0x7fef7406e840 legacy=0x7fef74130870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:23.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7902f640 10 monclient: _finish_hunting 0 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7902f640 1 monclient: found mon.noname-c 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7902f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7902f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7902f640 1 -- 172.21.15.204:0/132094944 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7406ec10 con 0x7fef7409f9d0 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.105+0000 7fef7882e640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7902f640 1 --2- 172.21.15.204:0/132094944 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef7409f9d0 0x7fef74130f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fef70009880 tx=0x7fef700090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=a69c1d1e0c097cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7882e640 1 -- 172.21.15.204:0/132094944 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef7002acd0 con 0x7fef7409f9d0 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7882e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7882e640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7882e640 10 monclient: dump: 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:23.119 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:23.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7882e640 10 monclient: _finish_auth 0 2024-09-15T01:36:23.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7882e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:23.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7882e640 1 -- 172.21.15.204:0/132094944 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7002ae50 con 0x7fef7409f9d0 2024-09-15T01:36:23.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 5 monclient: authenticate success, global_id 5800 2024-09-15T01:36:23.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:23.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7882e640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:23.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:23.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7882e640 1 -- 172.21.15.204:0/132094944 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef70031ca0 con 0x7fef7409f9d0 2024-09-15T01:36:23.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 4 set_mon_vals no callback set 2024-09-15T01:36:23.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7882e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:23.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7882e640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:23.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7882e640 10 monclient: dump: 2024-09-15T01:36:23.121 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:23.122 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:23.123 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:23.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 10 monclient: shutdown 2024-09-15T01:36:23.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:23.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 1 -- 172.21.15.204:0/132094944 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef7409f9d0 msgr2=0x7fef74130f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:23.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 1 --2- 172.21.15.204:0/132094944 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef7409f9d0 0x7fef74130f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fef70009880 tx=0x7fef700090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:23.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 1 -- 172.21.15.204:0/132094944 shutdown_connections 2024-09-15T01:36:23.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 1 --2- 172.21.15.204:0/132094944 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fef74135ba0 0x7fef74137f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:23.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 1 --2- 172.21.15.204:0/132094944 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef7409f9d0 0x7fef74130f80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:23.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 1 -- 172.21.15.204:0/132094944 >> 172.21.15.204:0/132094944 conn(0x7fef74067c00 msgr2=0x7fef740a3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:23.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 1 -- 172.21.15.204:0/132094944 shutdown_connections 2024-09-15T01:36:23.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.106+0000 7fef7ba94640 1 -- 172.21.15.204:0/132094944 wait complete. 2024-09-15T01:36:23.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.107+0000 7fef7ba94640 10 monclient: build_initial_monmap 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.107+0000 7fef7ba94640 10 monclient: monmap: 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:23.124 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:23.125 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:23.125 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:23.125 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:23.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.107+0000 7fef7ba94640 1 Processor -- start 2024-09-15T01:36:23.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.107+0000 7fef7ba94640 1 -- start start 2024-09-15T01:36:23.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.107+0000 7fef7ba94640 10 monclient: init 2024-09-15T01:36:23.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:23.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 10 monclient: _add_conns ranks=[1,4,7,6,0,2,8,3,5] 2024-09-15T01:36:23.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef7406e840 0x7fef74133130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 10 monclient(hunting): picked mon.noname-f con 0x7fef7406e840 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef7409f9d0 0x7fef74133670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 10 monclient(hunting): picked mon.noname-c con 0x7fef7409f9d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fef74135ba0 0x7fef74133bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 10 monclient(hunting): picked mon.noname-i con 0x7fef74135ba0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7411b4e0 con 0x7fef7406e840 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef7411b730 con 0x7fef7409f9d0 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fef7406d1a0 con 0x7fef74135ba0 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:23.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 10 monclient(hunting): authenticate will time out at 998.751253s 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef7406e840 0x7fef74133130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef7406e840 0x7fef74133130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:52808/0 (socket says 172.21.15.204:52808) 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 1 -- 172.21.15.204:0/2598979697 learned_addr learned my addr 172.21.15.204:0/2598979697 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 10 monclient(hunting): get_auth_request con 0x7fef7406e840 auth_method 0 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7a031640 1 --2- 172.21.15.204:0/2598979697 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fef74135ba0 0x7fef74133bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7a031640 10 monclient(hunting): get_auth_request con 0x7fef74135ba0 auth_method 0 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7a031640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7a031640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7a031640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7902f640 1 --2- 172.21.15.204:0/2598979697 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef7409f9d0 0x7fef74133670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7902f640 10 monclient(hunting): get_auth_request con 0x7fef7409f9d0 auth_method 0 2024-09-15T01:36:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7902f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7902f640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7902f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7a031640 1 -- 172.21.15.204:0/2598979697 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fef74135ba0 msgr2=0x7fef74133bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:36:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7a031640 1 -- 172.21.15.204:0/2598979697 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fef74135ba0 msgr2=0x7fef74133bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:36:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7902f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7902f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7a031640 1 --2- 172.21.15.204:0/2598979697 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fef74135ba0 0x7fef74133bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7902f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7a031640 1 --2- 172.21.15.204:0/2598979697 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fef74135ba0 0x7fef74133bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 10 monclient(hunting): handle_auth_done global_id 6562 payload 824 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 1 -- 172.21.15.204:0/2598979697 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fef74135ba0 msgr2=0x7fef74133bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 1 --2- 172.21.15.204:0/2598979697 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fef74135ba0 0x7fef74133bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 1 -- 172.21.15.204:0/2598979697 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef7409f9d0 msgr2=0x7fef74133670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 1 --2- 172.21.15.204:0/2598979697 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef7409f9d0 0x7fef74133670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 10 monclient: _finish_hunting 0 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 1 monclient: found mon.noname-f 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:23.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 1 -- 172.21.15.204:0/2598979697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef70009390 con 0x7fef7406e840 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef79830640 1 --2- 172.21.15.204:0/2598979697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef7406e840 0x7fef74133130 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c610 tx=0x7fef68007d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef65ffb640 1 -- 172.21.15.204:0/2598979697 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef680070f0 con 0x7fef7406e840 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef65ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef65ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef65ffb640 10 monclient: dump: 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:23.130 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef65ffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef65ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 5 monclient: authenticate success, global_id 6562 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef65ffb640 1 -- 172.21.15.204:0/2598979697 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef68007290 con 0x7fef7406e840 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 10 monclient: _renew_subs 2024-09-15T01:36:23.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef7ba94640 1 -- 172.21.15.204:0/2598979697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef74135200 con 0x7fef7406e840 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef65ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef65ffb640 1 -- 172.21.15.204:0/2598979697 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef68007440 con 0x7fef7406e840 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef65ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef65ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef65ffb640 10 monclient: dump: 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:23.132 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.109+0000 7fef7ba94640 10 monclient: _renew_subs 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.109+0000 7fef7ba94640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.109+0000 7fef7ba94640 1 -- 172.21.15.204:0/2598979697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef74135620 con 0x7fef7406e840 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.108+0000 7fef67fff640 4 set_mon_vals no callback set 2024-09-15T01:36:23.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.109+0000 7fef65ffb640 1 -- 172.21.15.204:0/2598979697 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fef680075a0 con 0x7fef7406e840 2024-09-15T01:36:23.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.109+0000 7fef65ffb640 1 --2- 172.21.15.204:0/2598979697 >> v2:172.21.15.204:6812/28210 conn(0x7fef580397b0 0x7fef5803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:23.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.110+0000 7fef65ffb640 1 -- 172.21.15.204:0/2598979697 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6867+0+0 (secure 0 0 0) 0x7fef6804b5e0 con 0x7fef7406e840 2024-09-15T01:36:23.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.110+0000 7fef7902f640 1 --2- 172.21.15.204:0/2598979697 >> v2:172.21.15.204:6812/28210 conn(0x7fef580397b0 0x7fef5803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:23.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.110+0000 7fef7902f640 10 monclient: get_auth_request con 0x7fef580397b0 auth_method 0 2024-09-15T01:36:23.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.110+0000 7fef7ba94640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:23.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.110+0000 7fef7ba94640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:23.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.110+0000 7fef7ba94640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:23.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.110+0000 7fef7ba94640 1 -- 172.21.15.204:0/2598979697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7411b870 con 0x7fef7406e840 2024-09-15T01:36:23.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.110+0000 7fef7902f640 1 --2- 172.21.15.204:0/2598979697 >> v2:172.21.15.204:6812/28210 conn(0x7fef580397b0 0x7fef5803bc60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fef70034000 tx=0x7fef70032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:23.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.114+0000 7fef65ffb640 1 -- 172.21.15.204:0/2598979697 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fef7411b870 con 0x7fef7406e840 2024-09-15T01:36:23.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.114+0000 7fef65ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:23.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.114+0000 7fef65ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.277+0000 7fef7ba94640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:36:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.277+0000 7fef7ba94640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:36:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.277+0000 7fef7ba94640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:23.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:23.277+0000 7fef7ba94640 1 -- 172.21.15.204:0/2598979697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7fef7411b870 con 0x7fef7406e840 2024-09-15T01:36:24.221 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:24.219+0000 7fef65ffb640 1 -- 172.21.15.204:0/2598979697 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v170) v1 ==== 78+0+0 (secure 0 0 0) 0x7fef7411b870 con 0x7fef7406e840 2024-09-15T01:36:24.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:24.219+0000 7fef65ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:36:24.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:24.219+0000 7fef65ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:24.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:24.325+0000 7fef7ba94640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:36:24.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:24.325+0000 7fef7ba94640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:36:24.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:24.325+0000 7fef7ba94640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:24.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:24.325+0000 7fef7ba94640 1 -- 172.21.15.204:0/2598979697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7fef7411b870 con 0x7fef7406e840 2024-09-15T01:36:25.228 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.222+0000 7fef65ffb640 1 -- 172.21.15.204:0/2598979697 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v171) v1 ==== 78+0+0 (secure 0 0 0) 0x7fef7411b870 con 0x7fef7406e840 2024-09-15T01:36:25.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.225+0000 7fef65ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:36:25.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.225+0000 7fef65ffb640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:25.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 1 -- 172.21.15.204:0/2598979697 >> v2:172.21.15.204:6812/28210 conn(0x7fef580397b0 msgr2=0x7fef5803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:25.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 1 --2- 172.21.15.204:0/2598979697 >> v2:172.21.15.204:6812/28210 conn(0x7fef580397b0 0x7fef5803bc60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fef70034000 tx=0x7fef70032000 comp rx=0 tx=0).stop 2024-09-15T01:36:25.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 10 monclient: shutdown 2024-09-15T01:36:25.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:25.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 1 -- 172.21.15.204:0/2598979697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef7406e840 msgr2=0x7fef74133130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:25.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 1 --2- 172.21.15.204:0/2598979697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef7406e840 0x7fef74133130 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c610 tx=0x7fef68007d00 comp rx=0 tx=0).stop 2024-09-15T01:36:25.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 1 -- 172.21.15.204:0/2598979697 shutdown_connections 2024-09-15T01:36:25.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 1 --2- 172.21.15.204:0/2598979697 >> v2:172.21.15.204:6812/28210 conn(0x7fef580397b0 0x7fef5803bc60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 1 --2- 172.21.15.204:0/2598979697 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fef74135ba0 0x7fef74133bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 1 --2- 172.21.15.204:0/2598979697 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fef7409f9d0 0x7fef74133670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 1 --2- 172.21.15.204:0/2598979697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef7406e840 0x7fef74133130 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 1 -- 172.21.15.204:0/2598979697 >> 172.21.15.204:0/2598979697 conn(0x7fef74067c00 msgr2=0x7fef7409dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:25.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 1 -- 172.21.15.204:0/2598979697 shutdown_connections 2024-09-15T01:36:25.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.230+0000 7fef7ba94640 1 -- 172.21.15.204:0/2598979697 wait complete. 2024-09-15T01:36:25.244 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set create-compat 2024-09-15T01:36:25.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.316+0000 7f51b171d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:25.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.316+0000 7f51b171d640 10 monclient: build_initial_monmap 2024-09-15T01:36:25.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.316+0000 7f51b171d640 10 monclient: monmap: 2024-09-15T01:36:25.319 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:25.319 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:25.319 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:25.317864+0000 2024-09-15T01:36:25.320 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:25.317864+0000 2024-09-15T01:36:25.320 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:25.320 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:25.320 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:25.320 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:25.320 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:25.320 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:25.320 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:25.320 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:25.320 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:25.320 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:25.321 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:25.321 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:25.321 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:25.321 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:25.330 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.316+0000 7f51b171d640 1 Processor -- start 2024-09-15T01:36:25.330 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 1 -- start start 2024-09-15T01:36:25.330 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 10 monclient: init 2024-09-15T01:36:25.330 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:25.331 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 10 monclient: _add_conns ranks=[8,0,3,10,7,5,4,2,6,1,9] 2024-09-15T01:36:25.331 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f51ac0a2e30 0x7f51ac132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.331 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 10 monclient(hunting): picked mon.noname-e con 0x7f51ac0a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:25.331 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f51ac09fb80 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:25.331 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 0x7f51ac0a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.331 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 10 monclient(hunting): picked mon.noname-g con 0x7f51ac09ff70 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:25.331 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac04b720 con 0x7f51ac09fb80 2024-09-15T01:36:25.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:25.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac09e6c0 con 0x7f51ac09ff70 2024-09-15T01:36:25.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:25.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0a3200 con 0x7f51ac0a2e30 2024-09-15T01:36:25.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:25.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51b171d640 10 monclient(hunting): authenticate will time out at 1000.960600s 2024-09-15T01:36:25.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51ab7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 0x7f51ac0a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51ab7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 0x7f51ac0a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:56040/0 (socket says 172.21.15.204:56040) 2024-09-15T01:36:25.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51ab7fe640 1 -- 172.21.15.204:0/1202872596 learned_addr learned my addr 172.21.15.204:0/1202872596 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:25.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51ab7fe640 10 monclient(hunting): get_auth_request con 0x7f51ac09ff70 auth_method 0 2024-09-15T01:36:25.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:25.332 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:25.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:25.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:25.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:25.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:25.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51aaffd640 1 --2- 172.21.15.204:0/1202872596 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f51ac0a2e30 0x7f51ac132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51aaffd640 10 monclient(hunting): get_auth_request con 0x7f51ac0a2e30 auth_method 0 2024-09-15T01:36:25.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:25.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:25.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.317+0000 7f51aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:25.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:25.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:25.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:25.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51ab7fe640 10 monclient(hunting): handle_auth_done global_id 14979 payload 274 2024-09-15T01:36:25.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51ab7fe640 1 -- 172.21.15.204:0/1202872596 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f51ac0a2e30 msgr2=0x7f51ac132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:25.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51ab7fe640 1 --2- 172.21.15.204:0/1202872596 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f51ac0a2e30 0x7f51ac132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51ab7fe640 1 -- 172.21.15.204:0/1202872596 >> v1:172.21.15.178:6789/0 conn(0x7f51ac09fb80 legacy=0x7f51ac0a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:25.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51ab7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:25.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51ab7fe640 1 monclient: found mon.noname-g 2024-09-15T01:36:25.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:25.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51ab7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:25.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51ab7fe640 1 -- 172.21.15.204:0/1202872596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac0a1f10 con 0x7f51ac09ff70 2024-09-15T01:36:25.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51ab7fe640 1 --2- 172.21.15.204:0/1202872596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 0x7f51ac0a1150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009f50 tx=0x7f51a4009840 comp rx=0 tx=0).ready entity=mon.3 client_cookie=6df33bf2bffe249f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:25.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51a9ffb640 1 -- 172.21.15.204:0/1202872596 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f51a4007880 con 0x7f51ac09ff70 2024-09-15T01:36:25.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51a9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:25.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51a9ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:25.336 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51a9ffb640 10 monclient: dump: 2024-09-15T01:36:25.336 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:25.336 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:25.336 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:25.337 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:25.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51a9ffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:25.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:25.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51a9ffb640 1 -- 172.21.15.204:0/1202872596 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a402d050 con 0x7f51ac09ff70 2024-09-15T01:36:25.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51b171d640 5 monclient: authenticate success, global_id 14979 2024-09-15T01:36:25.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51b171d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:25.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51a9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:25.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51b171d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:25.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51a9ffb640 1 -- 172.21.15.204:0/1202872596 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f51a4030360 con 0x7f51ac09ff70 2024-09-15T01:36:25.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51b171d640 4 set_mon_vals no callback set 2024-09-15T01:36:25.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51a9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:25.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51a9ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:25.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.318+0000 7f51a9ffb640 10 monclient: dump: 2024-09-15T01:36:25.339 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:25.339 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:25.339 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.339 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.339 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:25.339 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:25.339 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:25.339 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:25.339 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.319+0000 7f51b171d640 10 monclient: shutdown 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.319+0000 7f51b171d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.319+0000 7f51b171d640 1 -- 172.21.15.204:0/1202872596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 msgr2=0x7f51ac0a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.319+0000 7f51b171d640 1 --2- 172.21.15.204:0/1202872596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 0x7f51ac0a1150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009f50 tx=0x7f51a4009840 comp rx=0 tx=0).stop 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.319+0000 7f51b171d640 1 -- 172.21.15.204:0/1202872596 shutdown_connections 2024-09-15T01:36:25.340 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.319+0000 7f51b171d640 1 --2- 172.21.15.204:0/1202872596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 0x7f51ac0a1150 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.341 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.319+0000 7f51b171d640 1 --2- 172.21.15.204:0/1202872596 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f51ac0a2e30 0x7f51ac132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.341 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.319+0000 7f51b171d640 1 -- 172.21.15.204:0/1202872596 >> 172.21.15.204:0/1202872596 conn(0x7f51ac06e860 msgr2=0x7f51ac06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:25.341 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.319+0000 7f51b171d640 1 -- 172.21.15.204:0/1202872596 shutdown_connections 2024-09-15T01:36:25.341 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.319+0000 7f51b171d640 1 -- 172.21.15.204:0/1202872596 wait complete. 2024-09-15T01:36:25.341 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient: build_initial_monmap 2024-09-15T01:36:25.341 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient: monmap: 2024-09-15T01:36:25.341 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:25.341 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:25.341 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:25.341 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:25.341 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:25.341 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:25.342 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:25.342 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:25.342 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:25.342 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:25.342 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:25.342 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:25.342 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:25.342 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:25.342 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:25.342 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:25.342 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 1 Processor -- start 2024-09-15T01:36:25.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 1 -- start start 2024-09-15T01:36:25.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient: init 2024-09-15T01:36:25.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:25.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient: _add_conns ranks=[2,3,6,0,5,7,4,8,1] 2024-09-15T01:36:25.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51ac09fb80 0x7f51ac1319c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient(hunting): picked mon.noname-b con 0x7f51ac09fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:25.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 0x7f51ac11e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient(hunting): picked mon.noname-g con 0x7f51ac09ff70 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:25.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f51ac0a2e30 0x7f51ac11ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient(hunting): picked mon.noname-d con 0x7f51ac0a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:25.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:25.343 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac120070 con 0x7f51ac09fb80 2024-09-15T01:36:25.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:25.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac09ca40 con 0x7f51ac09ff70 2024-09-15T01:36:25.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:25.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac09d650 con 0x7f51ac0a2e30 2024-09-15T01:36:25.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:25.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51b171d640 10 monclient(hunting): authenticate will time out at 1000.963898s 2024-09-15T01:36:25.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.320+0000 7f51aa7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 0x7f51ac11e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.344 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 0x7f51ac11e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:56054/0 (socket says 172.21.15.204:56054) 2024-09-15T01:36:25.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 1 -- 172.21.15.204:0/415194648 learned_addr learned my addr 172.21.15.204:0/415194648 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:25.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 10 monclient(hunting): get_auth_request con 0x7f51ac09ff70 auth_method 0 2024-09-15T01:36:25.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:25.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:25.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:25.345 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aaffd640 1 --2- 172.21.15.204:0/415194648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51ac09fb80 0x7f51ac1319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:25.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:25.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:25.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aaffd640 10 monclient(hunting): get_auth_request con 0x7f51ac09fb80 auth_method 0 2024-09-15T01:36:25.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:25.346 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:25.347 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:25.347 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51ab7fe640 1 --2- 172.21.15.204:0/415194648 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f51ac0a2e30 0x7f51ac11ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.347 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51ab7fe640 10 monclient(hunting): get_auth_request con 0x7f51ac0a2e30 auth_method 0 2024-09-15T01:36:25.347 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:25.347 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:25.347 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:25.348 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:25.348 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:25.348 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:25.348 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:25.348 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:25.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:25.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 10 monclient(hunting): handle_auth_done global_id 14988 payload 824 2024-09-15T01:36:25.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 1 -- 172.21.15.204:0/415194648 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f51ac0a2e30 msgr2=0x7f51ac11ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:25.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 1 --2- 172.21.15.204:0/415194648 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f51ac0a2e30 0x7f51ac11ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 1 -- 172.21.15.204:0/415194648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51ac09fb80 msgr2=0x7f51ac1319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:25.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 1 --2- 172.21.15.204:0/415194648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51ac09fb80 0x7f51ac1319c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:36:25.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 1 monclient: found mon.noname-g 2024-09-15T01:36:25.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:25.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:25.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 1 -- 172.21.15.204:0/415194648 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4009ae0 con 0x7f51ac09ff70 2024-09-15T01:36:25.349 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51aa7fc640 1 --2- 172.21.15.204:0/415194648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 0x7f51ac11e530 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f519400c5c0 tx=0x7f5194007e80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:25.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a2ffd640 1 -- 172.21.15.204:0/415194648 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5194011a10 con 0x7f51ac09ff70 2024-09-15T01:36:25.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:25.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a2ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:25.350 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a2ffd640 10 monclient: dump: 2024-09-15T01:36:25.350 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:25.350 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:25.350 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.350 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.350 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:25.350 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:25.350 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:25.350 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:25.351 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:25.351 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:25.351 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:25.351 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:25.351 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:25.351 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:25.351 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:25.351 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:25.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:25.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:25.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a2ffd640 1 -- 172.21.15.204:0/415194648 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5194011bd0 con 0x7f51ac09ff70 2024-09-15T01:36:25.351 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51b171d640 5 monclient: authenticate success, global_id 14988 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51b171d640 10 monclient: _renew_subs 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a2ffd640 1 -- 172.21.15.204:0/415194648 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5194011d40 con 0x7f51ac09ff70 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51b171d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51b171d640 1 -- 172.21.15.204:0/415194648 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac1429c0 con 0x7f51ac09ff70 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a2ffd640 10 monclient: dump: 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.352 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:25.353 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:25.353 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:25.353 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:25.353 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:25.353 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:25.353 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:25.353 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:25.353 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:25.353 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:25.354 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:25.354 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:25.354 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51a97fa640 4 set_mon_vals no callback set 2024-09-15T01:36:25.354 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51b171d640 10 monclient: _renew_subs 2024-09-15T01:36:25.354 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51b171d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:25.354 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.321+0000 7f51b171d640 1 -- 172.21.15.204:0/415194648 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac133f40 con 0x7f51ac09ff70 2024-09-15T01:36:25.354 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.322+0000 7f51a2ffd640 1 -- 172.21.15.204:0/415194648 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5194011590 con 0x7f51ac09ff70 2024-09-15T01:36:25.354 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.322+0000 7f51b171d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:25.354 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.322+0000 7f51b171d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:25.354 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.322+0000 7f51b171d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:25.355 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.322+0000 7f51b171d640 1 -- 172.21.15.204:0/415194648 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac070cc0 con 0x7f51ac09ff70 2024-09-15T01:36:25.355 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.322+0000 7f51a2ffd640 1 --2- 172.21.15.204:0/415194648 >> v2:172.21.15.204:6812/28210 conn(0x7f518c0396e0 0x7f518c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.355 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.322+0000 7f51a2ffd640 1 -- 172.21.15.204:0/415194648 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6415+0+0 (secure 0 0 0) 0x7f519404aea0 con 0x7f51ac09ff70 2024-09-15T01:36:25.355 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.322+0000 7f51aaffd640 1 --2- 172.21.15.204:0/415194648 >> v2:172.21.15.204:6812/28210 conn(0x7f518c0396e0 0x7f518c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.355 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.322+0000 7f51aaffd640 10 monclient: get_auth_request con 0x7f518c0396e0 auth_method 0 2024-09-15T01:36:25.355 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.323+0000 7f51aaffd640 1 --2- 172.21.15.204:0/415194648 >> v2:172.21.15.204:6812/28210 conn(0x7f518c0396e0 0x7f518c03bb90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f519c009c80 tx=0x7f519c0090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:25.355 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.327+0000 7f51a2ffd640 1 -- 172.21.15.204:0/415194648 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5194012420 con 0x7f51ac09ff70 2024-09-15T01:36:25.356 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.327+0000 7f51a2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:25.356 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.327+0000 7f51a2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:25.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.492+0000 7f51b171d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:36:25.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.492+0000 7f51b171d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:36:25.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.492+0000 7f51b171d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:25.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:25.492+0000 7f51b171d640 1 -- 172.21.15.204:0/415194648 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7f51ac09ddb0 con 0x7f51ac09ff70 2024-09-15T01:36:25.614 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:36:25.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.708+0000 7fc4d34f8640 10 monclient: get_monmap_and_config 2024-09-15T01:36:25.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.708+0000 7fc4d34f8640 10 monclient: build_initial_monmap 2024-09-15T01:36:25.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.708+0000 7fc4d34f8640 10 monclient: monmap: 2024-09-15T01:36:25.711 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:25.711 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:25.711 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:25.710004+0000 2024-09-15T01:36:25.711 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:25.710004+0000 2024-09-15T01:36:25.711 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:25.711 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:25.711 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:25.712 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:25.712 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:25.712 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:25.712 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:25.712 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:25.712 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:25.712 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:25.712 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:25.712 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:25.712 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:25.712 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:25.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.708+0000 7fc4d34f8640 1 Processor -- start 2024-09-15T01:36:25.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.708+0000 7fc4d34f8640 1 -- start start 2024-09-15T01:36:25.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 10 monclient: init 2024-09-15T01:36:25.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:25.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 10 monclient: _add_conns ranks=[5,8,4,6,9,1,3,10,7,2,0] 2024-09-15T01:36:25.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc4cc0a2930 0x7fc4cc132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 10 monclient(hunting): picked mon.noname-h con 0x7fc4cc0a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:25.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc4cc09faa0 0x7fc4cc09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 10 monclient(hunting): picked mon.noname-e con 0x7fc4cc09faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:25.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc4cc0a08c0 0x7fc4cc0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 10 monclient(hunting): picked mon.noname-c con 0x7fc4cc0a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:25.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:25.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc09e0f0 con 0x7fc4cc0a08c0 2024-09-15T01:36:25.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:25.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc09dd80 con 0x7fc4cc0a2930 2024-09-15T01:36:25.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:25.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc0a2d00 con 0x7fc4cc09faa0 2024-09-15T01:36:25.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:25.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d34f8640 10 monclient(hunting): authenticate will time out at 990.460504s 2024-09-15T01:36:25.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1a95640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc4cc0a08c0 0x7fc4cc0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc4cc09faa0 0x7fc4cc09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc4cc09faa0 0x7fc4cc09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:56034/0 (socket says 172.21.15.178:56034) 2024-09-15T01:36:25.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1a95640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc4cc0a08c0 0x7fc4cc0a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44088/0 (socket says 172.21.15.178:44088) 2024-09-15T01:36:25.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 1 -- 172.21.15.178:0/2535945566 learned_addr learned my addr 172.21.15.178:0/2535945566 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:25.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 10 monclient(hunting): get_auth_request con 0x7fc4cc09faa0 auth_method 0 2024-09-15T01:36:25.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:25.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:25.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:25.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1a95640 10 monclient(hunting): get_auth_request con 0x7fc4cc0a08c0 auth_method 0 2024-09-15T01:36:25.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1a95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:25.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1a95640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:25.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1a95640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:25.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:25.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:25.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:25.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1a95640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:25.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1a95640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:25.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1a95640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:25.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1294640 1 --2- 172.21.15.178:0/2535945566 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc4cc0a2930 0x7fc4cc132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1294640 10 monclient(hunting): get_auth_request con 0x7fc4cc0a2930 auth_method 0 2024-09-15T01:36:25.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1294640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:25.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1294640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:25.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1294640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:25.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1294640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:25.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1294640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:25.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1294640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:25.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 10 monclient(hunting): handle_auth_done global_id 5867 payload 274 2024-09-15T01:36:25.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 1 -- 172.21.15.178:0/2535945566 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc4cc0a2930 msgr2=0x7fc4cc132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:25.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 1 --2- 172.21.15.178:0/2535945566 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc4cc0a2930 0x7fc4cc132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 1 -- 172.21.15.178:0/2535945566 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc4cc0a08c0 msgr2=0x7fc4cc0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:25.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 1 --2- 172.21.15.178:0/2535945566 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc4cc0a08c0 0x7fc4cc0a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 10 monclient: _finish_hunting 0 2024-09-15T01:36:25.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 1 monclient: found mon.noname-e 2024-09-15T01:36:25.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:25.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:25.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d0a93640 1 -- 172.21.15.178:0/2535945566 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc04bf60 con 0x7fc4cc09faa0 2024-09-15T01:36:25.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.709+0000 7fc4d1a95640 1 --2- 172.21.15.178:0/2535945566 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc4cc0a08c0 0x7fc4cc0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:25.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4d0a93640 1 --2- 172.21.15.178:0/2535945566 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc4cc09faa0 0x7fc4cc09fe70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009940 tx=0x7fc4bc0091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=acef4f8ef2e5eaea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:25.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4c3fff640 1 -- 172.21.15.178:0/2535945566 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4bc02ac60 con 0x7fc4cc09faa0 2024-09-15T01:36:25.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:25.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4c3fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:25.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4c3fff640 10 monclient: dump: 2024-09-15T01:36:25.731 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:25.731 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:25.731 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.731 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.731 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:25.732 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:25.732 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:25.732 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:25.732 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:25.732 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:25.732 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:25.732 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:25.732 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:25.732 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:25.732 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:25.732 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:25.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4c3fff640 10 monclient: _finish_auth 0 2024-09-15T01:36:25.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:25.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4d34f8640 5 monclient: authenticate success, global_id 5867 2024-09-15T01:36:25.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4d34f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:25.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4c3fff640 1 -- 172.21.15.178:0/2535945566 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc02ade0 con 0x7fc4cc09faa0 2024-09-15T01:36:25.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4c3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:25.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4d34f8640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:25.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4c3fff640 1 -- 172.21.15.178:0/2535945566 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4bc031ca0 con 0x7fc4cc09faa0 2024-09-15T01:36:25.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4d34f8640 4 set_mon_vals no callback set 2024-09-15T01:36:25.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:25.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4c3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:25.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.710+0000 7fc4c3fff640 10 monclient: dump: 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 10 monclient: shutdown 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 1 -- 172.21.15.178:0/2535945566 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc4cc09faa0 msgr2=0x7fc4cc09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 1 --2- 172.21.15.178:0/2535945566 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc4cc09faa0 0x7fc4cc09fe70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009940 tx=0x7fc4bc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 1 -- 172.21.15.178:0/2535945566 shutdown_connections 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 1 --2- 172.21.15.178:0/2535945566 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc4cc0a08c0 0x7fc4cc0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 1 --2- 172.21.15.178:0/2535945566 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc4cc09faa0 0x7fc4cc09fe70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 1 --2- 172.21.15.178:0/2535945566 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc4cc0a2930 0x7fc4cc132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 1 -- 172.21.15.178:0/2535945566 >> 172.21.15.178:0/2535945566 conn(0x7fc4cc0a5ed0 msgr2=0x7fc4cc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 1 -- 172.21.15.178:0/2535945566 shutdown_connections 2024-09-15T01:36:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 1 -- 172.21.15.178:0/2535945566 wait complete. 2024-09-15T01:36:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 10 monclient: build_initial_monmap 2024-09-15T01:36:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.711+0000 7fc4d34f8640 10 monclient: monmap: 2024-09-15T01:36:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:25.737 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 1 Processor -- start 2024-09-15T01:36:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 1 -- start start 2024-09-15T01:36:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 10 monclient: init 2024-09-15T01:36:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 10 monclient: _add_conns ranks=[2,7,5,4,6,8,1,0,3] 2024-09-15T01:36:25.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4cc09faa0 0x7fc4cc09b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 10 monclient(hunting): picked mon.noname-b con 0x7fc4cc09faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:25.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc4cc0a08c0 0x7fc4cc131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 10 monclient(hunting): picked mon.noname-i con 0x7fc4cc0a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:25.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc4cc0a2930 0x7fc4cc141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 10 monclient(hunting): picked mon.noname-h con 0x7fc4cc0a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:25.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:25.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc11d760 con 0x7fc4cc09faa0 2024-09-15T01:36:25.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:25.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc11ed20 con 0x7fc4cc0a2930 2024-09-15T01:36:25.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:25.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc11ea40 con 0x7fc4cc0a08c0 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.712+0000 7fc4d34f8640 10 monclient(hunting): authenticate will time out at 990.463944s 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4cc09faa0 0x7fc4cc09b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4cc09faa0 0x7fc4cc09b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:37980/0 (socket says 172.21.15.178:37980) 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 1 -- 172.21.15.178:0/428809405 learned_addr learned my addr 172.21.15.178:0/428809405 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 10 monclient(hunting): get_auth_request con 0x7fc4cc09faa0 auth_method 0 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1a95640 1 --2- 172.21.15.178:0/428809405 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc4cc0a2930 0x7fc4cc141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d0a93640 1 --2- 172.21.15.178:0/428809405 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc4cc0a08c0 0x7fc4cc131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1a95640 10 monclient(hunting): get_auth_request con 0x7fc4cc0a2930 auth_method 0 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1a95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:25.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1a95640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1a95640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d0a93640 10 monclient(hunting): get_auth_request con 0x7fc4cc0a08c0 auth_method 0 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d0a93640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d0a93640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d0a93640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1a95640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1a95640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1a95640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d0a93640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:25.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d0a93640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:25.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d0a93640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:25.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 10 monclient(hunting): handle_auth_done global_id 24113 payload 824 2024-09-15T01:36:25.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 1 -- 172.21.15.178:0/428809405 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc4cc0a08c0 msgr2=0x7fc4cc131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:25.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 1 --2- 172.21.15.178:0/428809405 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc4cc0a08c0 0x7fc4cc131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 1 -- 172.21.15.178:0/428809405 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc4cc0a2930 msgr2=0x7fc4cc141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:25.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 1 --2- 172.21.15.178:0/428809405 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc4cc0a2930 0x7fc4cc141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 10 monclient: _finish_hunting 0 2024-09-15T01:36:25.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 1 monclient: found mon.noname-b 2024-09-15T01:36:25.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:25.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:36:25.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 1 -- 172.21.15.178:0/428809405 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c4009a20 con 0x7fc4cc09faa0 2024-09-15T01:36:25.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d1294640 1 --2- 172.21.15.178:0/428809405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4cc09faa0 0x7fc4cc09b2f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800ea20 tx=0x7fc4c80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:25.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c17fa640 1 -- 172.21.15.178:0/428809405 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4c800ce90 con 0x7fc4cc09faa0 2024-09-15T01:36:25.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:25.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:36:25.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c17fa640 10 monclient: dump: 2024-09-15T01:36:25.742 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:25.742 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:25.742 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.742 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.742 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:25.742 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c17fa640 10 monclient: _finish_auth 0 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:25.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c17fa640 1 -- 172.21.15.178:0/428809405 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c8009600 con 0x7fc4cc09faa0 2024-09-15T01:36:25.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d34f8640 5 monclient: authenticate success, global_id 24113 2024-09-15T01:36:25.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d34f8640 10 monclient: _renew_subs 2024-09-15T01:36:25.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d34f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:25.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d34f8640 1 -- 172.21.15.178:0/428809405 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc009450 con 0x7fc4cc09faa0 2024-09-15T01:36:25.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:25.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c17fa640 1 -- 172.21.15.178:0/428809405 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4c80138f0 con 0x7fc4cc09faa0 2024-09-15T01:36:25.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:25.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:25.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c17fa640 10 monclient: dump: 2024-09-15T01:36:25.744 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:25.744 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:25.744 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:25.745 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4c37fe640 4 set_mon_vals no callback set 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d34f8640 10 monclient: _renew_subs 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d34f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.713+0000 7fc4d34f8640 1 -- 172.21.15.178:0/428809405 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4cc11db50 con 0x7fc4cc09faa0 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.714+0000 7fc4c17fa640 1 -- 172.21.15.178:0/428809405 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc4c8013470 con 0x7fc4cc09faa0 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.714+0000 7fc4d34f8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.714+0000 7fc4d34f8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.714+0000 7fc4d34f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.714+0000 7fc4c17fa640 1 --2- 172.21.15.178:0/428809405 >> v2:172.21.15.204:6812/28210 conn(0x7fc4b00396e0 0x7fc4b003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.714+0000 7fc4d34f8640 1 -- 172.21.15.178:0/428809405 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4cc04b720 con 0x7fc4cc09faa0 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.714+0000 7fc4c17fa640 1 -- 172.21.15.178:0/428809405 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6415+0+0 (secure 0 0 0) 0x7fc4c804c1a0 con 0x7fc4cc09faa0 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.714+0000 7fc4d0a93640 1 --2- 172.21.15.178:0/428809405 >> v2:172.21.15.204:6812/28210 conn(0x7fc4b00396e0 0x7fc4b003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.714+0000 7fc4d0a93640 10 monclient: get_auth_request con 0x7fc4b00396e0 auth_method 0 2024-09-15T01:36:25.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.715+0000 7fc4d0a93640 1 --2- 172.21.15.178:0/428809405 >> v2:172.21.15.204:6812/28210 conn(0x7fc4b00396e0 0x7fc4b003bb90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc034000 tx=0x7fc4bc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:25.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.719+0000 7fc4c17fa640 1 -- 172.21.15.178:0/428809405 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc4c8039e60 con 0x7fc4cc09faa0 2024-09-15T01:36:25.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.719+0000 7fc4c17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:25.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.719+0000 7fc4c17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.932+0000 7fc4d34f8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:36:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.932+0000 7fc4d34f8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.932+0000 7fc4d34f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.932+0000 7fc4d34f8640 1 -- 172.21.15.178:0/428809405 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc4cc09cef0 con 0x7fc4cc09faa0 2024-09-15T01:36:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.933+0000 7fc4c17fa640 1 -- 172.21.15.178:0/428809405 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7fc4c8013720 con 0x7fc4cc09faa0 2024-09-15T01:36:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.933+0000 7fc4c17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.933+0000 7fc4c17fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:25.936 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":2,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:36:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 1 -- 172.21.15.178:0/428809405 >> v2:172.21.15.204:6812/28210 conn(0x7fc4b00396e0 msgr2=0x7fc4b003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 1 --2- 172.21.15.178:0/428809405 >> v2:172.21.15.204:6812/28210 conn(0x7fc4b00396e0 0x7fc4b003bb90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc034000 tx=0x7fc4bc032000 comp rx=0 tx=0).stop 2024-09-15T01:36:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 10 monclient: shutdown 2024-09-15T01:36:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 1 -- 172.21.15.178:0/428809405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4cc09faa0 msgr2=0x7fc4cc09b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 1 --2- 172.21.15.178:0/428809405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4cc09faa0 0x7fc4cc09b2f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800ea20 tx=0x7fc4c80090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 1 -- 172.21.15.178:0/428809405 shutdown_connections 2024-09-15T01:36:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 1 --2- 172.21.15.178:0/428809405 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc4cc0a2930 0x7fc4cc141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 1 --2- 172.21.15.178:0/428809405 >> v2:172.21.15.204:6812/28210 conn(0x7fc4b00396e0 0x7fc4b003bb90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 1 --2- 172.21.15.178:0/428809405 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc4cc0a08c0 0x7fc4cc131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 1 --2- 172.21.15.178:0/428809405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4cc09faa0 0x7fc4cc09b2f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:25.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 1 -- 172.21.15.178:0/428809405 >> 172.21.15.178:0/428809405 conn(0x7fc4cc0a5ed0 msgr2=0x7fc4cc0a4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:25.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 1 -- 172.21.15.178:0/428809405 shutdown_connections 2024-09-15T01:36:25.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:25.935+0000 7fc4d34f8640 1 -- 172.21.15.178:0/428809405 wait complete. 2024-09-15T01:36:25.948 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:36:25.948 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:36:25.948 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:36:26.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.039+0000 7f1b10a4f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:26.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.039+0000 7f1b10a4f640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.039+0000 7f1b10a4f640 10 monclient: monmap: 2024-09-15T01:36:26.042 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:26.042 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.042 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:26.040967+0000 2024-09-15T01:36:26.042 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:26.040967+0000 2024-09-15T01:36:26.042 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:26.043 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.039+0000 7f1b10a4f640 1 Processor -- start 2024-09-15T01:36:26.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.039+0000 7f1b10a4f640 1 -- start start 2024-09-15T01:36:26.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.039+0000 7f1b10a4f640 10 monclient: init 2024-09-15T01:36:26.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 10 monclient: _add_conns ranks=[9,3,10,2,7,0,1,4,8,6,5] 2024-09-15T01:36:26.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1b0c0678d0 0x7f1b0c09e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 10 monclient(hunting): picked mon.noname-a con 0x7f1b0c0678d0 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:26.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b0c06ea30 0x7f1b0c09ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 10 monclient(hunting): picked mon.noname-g con 0x7f1b0c06ea30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:26.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1b0c09f180 0x7f1b0c1306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 10 monclient(hunting): picked mon.noname-f con 0x7f1b0c09f180 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:26.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c09f5a0 con 0x7f1b0c0678d0 2024-09-15T01:36:26.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c06d5b0 con 0x7f1b0c09f180 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c067ca0 con 0x7f1b0c06ea30 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b10a4f640 10 monclient(hunting): authenticate will time out at 990.791361s 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1b0c0678d0 0x7f1b0c09e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1b0c0678d0 0x7f1b0c09e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60898/0 (socket says 172.21.15.178:60898) 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 1 -- 172.21.15.178:0/1473415966 learned_addr learned my addr 172.21.15.178:0/1473415966 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 10 monclient(hunting): get_auth_request con 0x7f1b0c0678d0 auth_method 0 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_done global_id 6129 payload 274 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 1 -- 172.21.15.178:0/1473415966 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b0c06ea30 msgr2=0x7f1b0c09ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0ad9d640 1 --2- 172.21.15.178:0/1473415966 >> v2:172.21.15.204:3300/0 conn(0x7f1b0c09f180 0x7f1b0c1306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b09d9b640 1 --2- 172.21.15.178:0/1473415966 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b0c06ea30 0x7f1b0c09ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 1 --2- 172.21.15.178:0/1473415966 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b0c06ea30 0x7f1b0c09ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 1 -- 172.21.15.178:0/1473415966 >> v2:172.21.15.204:3300/0 conn(0x7f1b0c09f180 msgr2=0x7f1b0c1306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 1 --2- 172.21.15.178:0/1473415966 >> v2:172.21.15.204:3300/0 conn(0x7f1b0c09f180 0x7f1b0c1306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 1 monclient: found mon.noname-a 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:36:26.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0a59c640 1 -- 172.21.15.178:0/1473415966 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c04bf60 con 0x7f1b0c0678d0 2024-09-15T01:36:26.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0ad9d640 10 monclient: get_auth_request con 0x7f1b0c09f180 auth_method 0 2024-09-15T01:36:26.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b09d9b640 10 monclient: get_auth_request con 0x7f1b0c06ea30 auth_method 0 2024-09-15T01:36:26.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b0ad9d640 1 --2- 172.21.15.178:0/1473415966 >> v2:172.21.15.204:3300/0 conn(0x7f1b0c09f180 0x7f1b0c1306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:36:26.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.040+0000 7f1b09d9b640 1 --2- 172.21.15.178:0/1473415966 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b0c06ea30 0x7f1b0c09ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:36:26.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0a59c640 1 --2- 172.21.15.178:0/1473415966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c0678d0 0x7f1b0c09e700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c06ee00 tx=0x7f1afc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff2efce0cf05f9e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0959a640 1 -- 172.21.15.178:0/1473415966 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1afc02ad40 con 0x7f1b0c0678d0 2024-09-15T01:36:26.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:36:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0959a640 10 monclient: dump: 2024-09-15T01:36:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.059 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0959a640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0959a640 1 -- 172.21.15.178:0/1473415966 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc031820 con 0x7f1b0c0678d0 2024-09-15T01:36:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b10a4f640 5 monclient: authenticate success, global_id 6129 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b10a4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b10a4f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0959a640 1 -- 172.21.15.178:0/1473415966 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1afc031e90 con 0x7f1b0c0678d0 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b10a4f640 4 set_mon_vals no callback set 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.041+0000 7f1b0959a640 10 monclient: dump: 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.061 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 10 monclient: shutdown 2024-09-15T01:36:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1473415966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c0678d0 msgr2=0x7f1b0c09e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 1 --2- 172.21.15.178:0/1473415966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c0678d0 0x7f1b0c09e700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c06ee00 tx=0x7f1afc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1473415966 shutdown_connections 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 1 --2- 172.21.15.178:0/1473415966 >> v2:172.21.15.204:3300/0 conn(0x7f1b0c09f180 0x7f1b0c1306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 1 --2- 172.21.15.178:0/1473415966 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b0c06ea30 0x7f1b0c09ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 1 --2- 172.21.15.178:0/1473415966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c0678d0 0x7f1b0c09e700 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1473415966 >> 172.21.15.178:0/1473415966 conn(0x7f1b0c0a4390 msgr2=0x7f1b0c067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1473415966 shutdown_connections 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1473415966 wait complete. 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.042+0000 7f1b10a4f640 10 monclient: monmap: 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:26.063 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:26.063 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:26.063 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.063 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:26.063 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:26.063 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:26.063 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.063 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.063 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.064 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.064 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.064 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.064 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.064 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 1 Processor -- start 2024-09-15T01:36:26.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 1 -- start start 2024-09-15T01:36:26.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 10 monclient: init 2024-09-15T01:36:26.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 10 monclient: _add_conns ranks=[7,6,5,1,8,2,0,3,4] 2024-09-15T01:36:26.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1b0c0678d0 0x7f1b0c13c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 10 monclient(hunting): picked mon.noname-i con 0x7f1b0c0678d0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:26.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1b0c06ea30 0x7f1b0c13b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 10 monclient(hunting): picked mon.noname-d con 0x7f1b0c06ea30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:26.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1b0c09f180 0x7f1b0c13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 10 monclient(hunting): picked mon.noname-h con 0x7f1b0c09f180 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:26.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c06ccd0 con 0x7f1b0c09f180 2024-09-15T01:36:26.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c06c900 con 0x7f1b0c06ea30 2024-09-15T01:36:26.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c13cc20 con 0x7f1b0c0678d0 2024-09-15T01:36:26.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b10a4f640 10 monclient(hunting): authenticate will time out at 990.794601s 2024-09-15T01:36:26.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1b0c06ea30 0x7f1b0c13b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1b0c06ea30 0x7f1b0c13b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:60100/0 (socket says 172.21.15.178:60100) 2024-09-15T01:36:26.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 1 -- 172.21.15.178:0/1967713986 learned_addr learned my addr 172.21.15.178:0/1967713986 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:26.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 10 monclient(hunting): get_auth_request con 0x7f1b0c06ea30 auth_method 0 2024-09-15T01:36:26.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0a59c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1b0c0678d0 0x7f1b0c13c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0ad9d640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1b0c09f180 0x7f1b0c13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0a59c640 10 monclient(hunting): get_auth_request con 0x7f1b0c0678d0 auth_method 0 2024-09-15T01:36:26.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0ad9d640 10 monclient(hunting): get_auth_request con 0x7f1b0c09f180 auth_method 0 2024-09-15T01:36:26.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b0ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 10 monclient(hunting): handle_auth_done global_id 24117 payload 824 2024-09-15T01:36:26.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 1 -- 172.21.15.178:0/1967713986 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1b0c0678d0 msgr2=0x7f1b0c13c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.043+0000 7f1b09d9b640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1b0c0678d0 0x7f1b0c13c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b09d9b640 1 -- 172.21.15.178:0/1967713986 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1b0c09f180 msgr2=0x7f1b0c13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b09d9b640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1b0c09f180 0x7f1b0c13f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b09d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b09d9b640 1 monclient: found mon.noname-d 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b09d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b09d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b09d9b640 1 -- 172.21.15.178:0/1967713986 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1afc009450 con 0x7f1b0c06ea30 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b0ad9d640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1b0c09f180 0x7f1b0c13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b09d9b640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1b0c06ea30 0x7f1b0c13b700 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000c880 tx=0x7f1b00008040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b00007aa0 con 0x7f1b0c06ea30 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1afa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1afa7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1afa7fc640 10 monclient: dump: 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.071 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1afa7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1afa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b00007c40 con 0x7f1b0c06ea30 2024-09-15T01:36:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b10a4f640 5 monclient: authenticate success, global_id 24117 2024-09-15T01:36:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1afa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b00007db0 con 0x7f1b0c06ea30 2024-09-15T01:36:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1afa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1afa7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1afa7fc640 10 monclient: dump: 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.072 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b08d99640 4 set_mon_vals no callback set 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b10a4f640 10 monclient: _renew_subs 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b10a4f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1967713986 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0c1207a0 con 0x7f1b0c06ea30 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b10a4f640 10 monclient: _renew_subs 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b10a4f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.044+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1967713986 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0c13bc40 con 0x7f1b0c06ea30 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1b00007620 con 0x7f1b0c06ea30 2024-09-15T01:36:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b10a4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b10a4f640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11f880 0x7f1b0c13fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b10a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1967713986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c120c20 con 0x7f1b0c11f880 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1afa7fc640 1 --2- 172.21.15.178:0/1967713986 >> v2:172.21.15.204:6812/28210 conn(0x7f1aec0397b0 0x7f1aec03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6415+0+0 (secure 0 0 0) 0x7f1b0004baf0 con 0x7f1b0c06ea30 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0a59c640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11f880 0x7f1b0c13fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0a59c640 10 monclient: get_auth_request con 0x7f1b0c11f880 auth_method 0 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0ad9d640 1 --2- 172.21.15.178:0/1967713986 >> v2:172.21.15.204:6812/28210 conn(0x7f1aec0397b0 0x7f1aec03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0ad9d640 10 monclient: get_auth_request con 0x7f1aec0397b0 auth_method 0 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_done global_id 6138 payload 824 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0a59c640 1 -- 172.21.15.178:0/1967713986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b0c04c4a0 con 0x7f1b0c11f880 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0a59c640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11f880 0x7f1b0c13fae0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c068080 tx=0x7f1afc032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1b0ad9d640 1 --2- 172.21.15.178:0/1967713986 >> v2:172.21.15.204:6812/28210 conn(0x7f1aec0397b0 0x7f1aec03bc60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1b04007fd0 tx=0x7f1b040077a0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1afc03d480 con 0x7f1b0c11f880 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1afa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1afa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.045+0000 7f1afa7fc640 10 monclient: dump: 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.076 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.076 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.076 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.076 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.076 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.076 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.076 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.076 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.076 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.076 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.076 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.077 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.046+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f1afc032ae0 con 0x7f1b0c11f880 2024-09-15T01:36:26.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.046+0000 7f1afa7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.046+0000 7f1afa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:26.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.046+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11f880 msgr2=0x7f1b0c13fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.046+0000 7f1afa7fc640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11f880 0x7f1b0c13fae0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c068080 tx=0x7f1afc032000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.046+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11f880 msgr2=0x7f1b0c13fae0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:26.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.046+0000 7f1afa7fc640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11f880 0x7f1b0c13fae0 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c068080 tx=0x7f1afc032000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.051+0000 7f1b10a4f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:26.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.051+0000 7f1b10a4f640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11e450 0x7f1b0c140020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.051+0000 7f1b10a4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.051+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1967713986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c06c4a0 con 0x7f1b0c11e450 2024-09-15T01:36:26.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.051+0000 7f1b0a59c640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11e450 0x7f1b0c140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.051+0000 7f1b0a59c640 10 monclient: get_auth_request con 0x7f1b0c11e450 auth_method 0 2024-09-15T01:36:26.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.051+0000 7f1b0a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.051+0000 7f1b0a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.051+0000 7f1b0a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1b0a59c640 10 monclient(hunting): handle_auth_done global_id 6147 payload 824 2024-09-15T01:36:26.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1b0a59c640 1 -- 172.21.15.178:0/1967713986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1b0c04bc80 con 0x7f1b0c11e450 2024-09-15T01:36:26.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1b0a59c640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11e450 0x7f1b0c140020 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1afc000c00 tx=0x7f1afc03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1afc04b070 con 0x7f1b0c11e450 2024-09-15T01:36:26.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1afa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1afa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1afa7fc640 10 monclient: dump: 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.082 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f1afc03c3e0 con 0x7f1b0c11e450 2024-09-15T01:36:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1afa7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1afa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11e450 msgr2=0x7f1b0c140020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1afa7fc640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11e450 0x7f1b0c140020 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1afc000c00 tx=0x7f1afc03d000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1afa7fc640 1 -- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11e450 msgr2=0x7f1b0c140020 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.052+0000 7f1afa7fc640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b0c11e450 0x7f1b0c140020 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:26.083 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.084 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.085 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.085 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.085 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.085 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.085 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:26.085 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:26.085 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.085 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.085 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.085 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:26.086 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.087 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:26.088 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:26.090 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:26.090 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.090 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.090 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.090 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:26.090 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:26.090 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:26.091 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:26.092 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.092 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.092 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.092 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:26.092 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:26.092 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.092 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:26.093 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.093 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:26.093 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.093 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.093 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:26.093 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.095 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:26.095 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.095 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.095 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:26.095 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:26.095 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.096 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.096 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.096 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:26.096 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:26.096 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:26.096 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.097 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.098 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:26.099 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.101 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:26.102 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.104 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.105 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:26.106 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.106 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.106 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:26.106 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:26.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.053+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1967713986 >> v2:172.21.15.204:6812/28210 conn(0x7f1aec0397b0 msgr2=0x7f1aec03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.053+0000 7f1b10a4f640 1 --2- 172.21.15.178:0/1967713986 >> v2:172.21.15.204:6812/28210 conn(0x7f1aec0397b0 0x7f1aec03bc60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1b04007fd0 tx=0x7f1b040077a0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.053+0000 7f1b10a4f640 10 monclient: shutdown 2024-09-15T01:36:26.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.053+0000 7f1b10a4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.053+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1b0c06ea30 msgr2=0x7f1b0c13b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.053+0000 7f1b10a4f640 1 --2- 172.21.15.178:0/1967713986 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1b0c06ea30 0x7f1b0c13b700 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000c880 tx=0x7f1b00008040 comp rx=0 tx=0).stop 2024-09-15T01:36:26.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.053+0000 7f1b0ad9d640 1 -- 172.21.15.178:0/1967713986 reap_dead start 2024-09-15T01:36:26.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.053+0000 7f1b0ad9d640 1 -- 172.21.15.178:0/1967713986 reap_dead start 2024-09-15T01:36:26.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.053+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1967713986 shutdown_connections 2024-09-15T01:36:26.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.053+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1967713986 >> 172.21.15.178:0/1967713986 conn(0x7f1b0c0a4390 msgr2=0x7f1b0c09c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.053+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1967713986 shutdown_connections 2024-09-15T01:36:26.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.053+0000 7f1b10a4f640 1 -- 172.21.15.178:0/1967713986 wait complete. 2024-09-15T01:36:26.108 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:26.109 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:36:26.109 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:36:26.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.198+0000 7ff779e20640 10 monclient: get_monmap_and_config 2024-09-15T01:36:26.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.198+0000 7ff779e20640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.198+0000 7ff779e20640 10 monclient: monmap: 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:26.200611+0000 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:26.200611+0000 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.202 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.203 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.203 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:26.203 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:26.203 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.199+0000 7ff779e20640 1 Processor -- start 2024-09-15T01:36:26.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.199+0000 7ff779e20640 1 -- start start 2024-09-15T01:36:26.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.199+0000 7ff779e20640 10 monclient: init 2024-09-15T01:36:26.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.199+0000 7ff779e20640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.199+0000 7ff779e20640 10 monclient: _add_conns ranks=[2,3,9,4,7,10,6,8,5,1,0] 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.199+0000 7ff779e20640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 0x7ff774132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.199+0000 7ff779e20640 10 monclient(hunting): picked mon.noname-b con 0x7ff7740a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.199+0000 7ff779e20640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff77409faa0 0x7ff77409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.199+0000 7ff779e20640 10 monclient(hunting): picked mon.noname-g con 0x7ff77409faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff779e20640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff7740a08c0 0x7ff7740a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff779e20640 10 monclient(hunting): picked mon.noname-a con 0x7ff7740a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff779e20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff779e20640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77409e0f0 con 0x7ff7740a08c0 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff779e20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff779e20640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77409dd80 con 0x7ff7740a2930 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff779e20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff779e20640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740a2d00 con 0x7ff77409faa0 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff779e20640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff779e20640 10 monclient(hunting): authenticate will time out at 990.951042s 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 0x7ff774132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff773fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff7740a08c0 0x7ff7740a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 0x7ff774132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:37988/0 (socket says 172.21.15.178:37988) 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 1 -- 172.21.15.178:0/1851127987 learned_addr learned my addr 172.21.15.178:0/1851127987 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 10 monclient(hunting): get_auth_request con 0x7ff7740a2930 auth_method 0 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff773fff640 10 monclient(hunting): get_auth_request con 0x7ff7740a08c0 auth_method 0 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff773fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff773fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff773fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff773fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff773fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff773fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff772ffd640 1 --2- 172.21.15.178:0/1851127987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff77409faa0 0x7ff77409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff772ffd640 10 monclient(hunting): get_auth_request con 0x7ff77409faa0 auth_method 0 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff772ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff772ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff772ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 10 monclient(hunting): handle_auth_done global_id 24122 payload 274 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 1 -- 172.21.15.178:0/1851127987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff77409faa0 msgr2=0x7ff77409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 1 --2- 172.21.15.178:0/1851127987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff77409faa0 0x7ff77409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 1 -- 172.21.15.178:0/1851127987 >> v2:172.21.15.178:3300/0 conn(0x7ff7740a08c0 msgr2=0x7ff7740a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 1 --2- 172.21.15.178:0/1851127987 >> v2:172.21.15.178:3300/0 conn(0x7ff7740a08c0 0x7ff7740a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 1 monclient: found mon.noname-b 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff7737fe640 1 -- 172.21.15.178:0/1851127987 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77404bf60 con 0x7ff7740a2930 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.200+0000 7ff773fff640 1 --2- 172.21.15.178:0/1851127987 >> v2:172.21.15.178:3300/0 conn(0x7ff7740a08c0 0x7ff7740a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7737fe640 1 --2- 172.21.15.178:0/1851127987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 0x7ff774132de0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff764009f20 tx=0x7ff764009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=953a84f714f73903 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7727fc640 1 -- 172.21.15.178:0/1851127987 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff764007880 con 0x7ff7740a2930 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7727fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:36:26.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7727fc640 10 monclient: dump: 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.217 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7727fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 5 monclient: authenticate success, global_id 24122 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7727fc640 1 -- 172.21.15.178:0/1851127987 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76402d050 con 0x7ff7740a2930 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7727fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 4 set_mon_vals no callback set 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7727fc640 1 -- 172.21.15.178:0/1851127987 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff764030360 con 0x7ff7740a2930 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7727fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff7727fc640 10 monclient: dump: 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.219 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 10 monclient: shutdown 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 1 -- 172.21.15.178:0/1851127987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 msgr2=0x7ff774132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 1 --2- 172.21.15.178:0/1851127987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 0x7ff774132de0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff764009f20 tx=0x7ff764009840 comp rx=0 tx=0).stop 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 1 -- 172.21.15.178:0/1851127987 shutdown_connections 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 1 --2- 172.21.15.178:0/1851127987 >> v2:172.21.15.178:3300/0 conn(0x7ff7740a08c0 0x7ff7740a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 1 --2- 172.21.15.178:0/1851127987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff77409faa0 0x7ff77409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 1 --2- 172.21.15.178:0/1851127987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 0x7ff774132de0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.201+0000 7ff779e20640 1 -- 172.21.15.178:0/1851127987 >> 172.21.15.178:0/1851127987 conn(0x7ff7740a5ed0 msgr2=0x7ff7740a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.202+0000 7ff779e20640 1 -- 172.21.15.178:0/1851127987 shutdown_connections 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.202+0000 7ff779e20640 1 -- 172.21.15.178:0/1851127987 wait complete. 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.202+0000 7ff779e20640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.202+0000 7ff779e20640 10 monclient: monmap: 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.221 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.202+0000 7ff779e20640 1 Processor -- start 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 1 -- start start 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 10 monclient: init 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 10 monclient: _add_conns ranks=[5,0,2,6,3,4,1,8,7] 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff77409faa0 0x7ff77409b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 10 monclient(hunting): picked mon.noname-h con 0x7ff77409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:26.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7740a08c0 0x7ff774131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 10 monclient(hunting): picked mon.noname-a con 0x7ff7740a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:26.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 0x7ff77411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 10 monclient(hunting): picked mon.noname-b con 0x7ff7740a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:26.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77411ed20 con 0x7ff7740a08c0 2024-09-15T01:36:26.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77409bdc0 con 0x7ff7740a2930 2024-09-15T01:36:26.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff77409d630 con 0x7ff77409faa0 2024-09-15T01:36:26.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff779e20640 10 monclient(hunting): authenticate will time out at 990.954446s 2024-09-15T01:36:26.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 0x7ff77411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff772ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7740a08c0 0x7ff774131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff772ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7740a08c0 0x7ff774131820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60938/0 (socket says 172.21.15.178:60938) 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff772ffd640 1 -- 172.21.15.178:0/1615210555 learned_addr learned my addr 172.21.15.178:0/1615210555 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff772ffd640 10 monclient(hunting): get_auth_request con 0x7ff7740a08c0 auth_method 0 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 0x7ff77411c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:38002/0 (socket says 172.21.15.178:38002) 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff772ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff772ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff772ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 10 monclient(hunting): get_auth_request con 0x7ff7740a2930 auth_method 0 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff772ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff772ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff772ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff7737fe640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff77409faa0 0x7ff77409b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff7737fe640 10 monclient(hunting): get_auth_request con 0x7ff77409faa0 auth_method 0 2024-09-15T01:36:26.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff7737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff7737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff7737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff7737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff7737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff7737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 10 monclient(hunting): handle_auth_done global_id 24131 payload 824 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 1 -- 172.21.15.178:0/1615210555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff77409faa0 msgr2=0x7ff77409b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff77409faa0 0x7ff77409b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 1 -- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7740a08c0 msgr2=0x7ff774131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7740a08c0 0x7ff774131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 1 monclient: found mon.noname-b 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff773fff640 1 -- 172.21.15.178:0/1615210555 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c009390 con 0x7ff7740a2930 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.203+0000 7ff772ffd640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7740a08c0 0x7ff774131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff773fff640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 0x7ff77411c530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00e980 tx=0x7ff76c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff76c0095e0 con 0x7ff7740a2930 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff75b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff75b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff75b7fe640 10 monclient: dump: 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.227 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff75b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff75b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c00cdc0 con 0x7ff7740a2930 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff779e20640 5 monclient: authenticate success, global_id 24131 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff779e20640 10 monclient: _renew_subs 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff779e20640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff779e20640 1 -- 172.21.15.178:0/1615210555 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff764009ae0 con 0x7ff7740a2930 2024-09-15T01:36:26.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff75b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff76c0138f0 con 0x7ff7740a2930 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff75b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff75b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff75b7fe640 10 monclient: dump: 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.229 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff771ffb640 4 set_mon_vals no callback set 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff779e20640 10 monclient: _renew_subs 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff779e20640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.204+0000 7ff779e20640 1 -- 172.21.15.178:0/1615210555 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77411e940 con 0x7ff7740a2930 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff779e20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff779e20640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7741422c0 0x7ff77411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff779e20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff76c013470 con 0x7ff7740a2930 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff779e20640 1 -- 172.21.15.178:0/1615210555 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77404c4a0 con 0x7ff7741422c0 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff7737fe640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7741422c0 0x7ff77411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff7737fe640 10 monclient: get_auth_request con 0x7ff7741422c0 auth_method 0 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff7737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff7737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff7737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff7737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff7737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff7737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 1 --2- 172.21.15.178:0/1615210555 >> v2:172.21.15.204:6812/28210 conn(0x7ff7600397e0 0x7ff76003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6415+0+0 (secure 0 0 0) 0x7ff76c04ba60 con 0x7ff7740a2930 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff7737fe640 10 monclient(hunting): handle_auth_done global_id 24140 payload 824 2024-09-15T01:36:26.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff7737fe640 1 -- 172.21.15.178:0/1615210555 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff77405ed70 con 0x7ff7741422c0 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff772ffd640 1 --2- 172.21.15.178:0/1615210555 >> v2:172.21.15.204:6812/28210 conn(0x7ff7600397e0 0x7ff76003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff772ffd640 10 monclient: get_auth_request con 0x7ff7600397e0 auth_method 0 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff7737fe640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7741422c0 0x7ff77411ca70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff7740a3290 tx=0x7ff764036000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff76403d880 con 0x7ff7741422c0 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 10 monclient: dump: 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.232 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff772ffd640 1 --2- 172.21.15.178:0/1615210555 >> v2:172.21.15.204:6812/28210 conn(0x7ff7600397e0 0x7ff76003bc90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff768010370 tx=0x7ff768008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7ff76403d400 con 0x7ff7741422c0 2024-09-15T01:36:26.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:26.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7741422c0 msgr2=0x7ff77411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7741422c0 0x7ff77411ca70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff7740a3290 tx=0x7ff764036000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7741422c0 msgr2=0x7ff77411ca70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:26.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.205+0000 7ff75b7fe640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7741422c0 0x7ff77411ca70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.226+0000 7f51a2ffd640 1 -- 172.21.15.204:0/415194648 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v172) v1 ==== 82+0+0 (secure 0 0 0) 0x7f51940102e0 con 0x7f51ac09ff70 2024-09-15T01:36:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.226+0000 7f51a2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:36:26.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.226+0000 7f51a2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:26.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff779e20640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:26.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff779e20640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff774131d60 0x7ff77411cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff779e20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff779e20640 1 -- 172.21.15.178:0/1615210555 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77404bc80 con 0x7ff774131d60 2024-09-15T01:36:26.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff7737fe640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff774131d60 0x7ff77411cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff7737fe640 10 monclient: get_auth_request con 0x7ff774131d60 auth_method 0 2024-09-15T01:36:26.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff7737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff7737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff7737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff7737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff7737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff7737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff7737fe640 10 monclient(hunting): handle_auth_done global_id 24149 payload 824 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.211+0000 7ff7737fe640 1 -- 172.21.15.178:0/1615210555 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff77411eb30 con 0x7ff774131d60 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff7737fe640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff774131d60 0x7ff77411cfb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff77404c5e0 tx=0x7ff76403f070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff76403de70 con 0x7ff774131d60 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff75b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff75b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff75b7fe640 10 monclient: dump: 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.238 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7ff76404b040 con 0x7ff774131d60 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff75b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff75b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff774131d60 msgr2=0x7ff77411cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff75b7fe640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff774131d60 0x7ff77411cfb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff77404c5e0 tx=0x7ff76403f070 comp rx=0 tx=0).stop 2024-09-15T01:36:26.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff75b7fe640 1 -- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff774131d60 msgr2=0x7ff77411cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:26.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff75b7fe640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff774131d60 0x7ff77411cfb0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff77404c5e0 tx=0x7ff76403f070 comp rx=0 tx=0).stop 2024-09-15T01:36:26.240 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:26.240 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:26.240 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:26.240 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:26.240 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:26.240 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:26.240 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:26.240 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:26.241 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.242 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:26.243 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.244 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.245 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.246 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:26.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:26.247 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.247 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.247 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.247 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:26.247 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:26.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:26.248 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:26.249 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:26.252 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:26.255 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.256 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.256 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.256 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:26.256 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:26.256 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.256 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.257 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.258 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.258 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:26.258 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:26.258 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.258 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:26.258 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.259 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:26.259 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.259 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.259 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:26.259 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:26.259 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.259 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.259 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.260 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.261 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.262 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.263 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:26.263 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:26.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff779e20640 1 -- 172.21.15.178:0/1615210555 >> v2:172.21.15.204:6812/28210 conn(0x7ff7600397e0 msgr2=0x7ff76003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff779e20640 1 --2- 172.21.15.178:0/1615210555 >> v2:172.21.15.204:6812/28210 conn(0x7ff7600397e0 0x7ff76003bc90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff768010370 tx=0x7ff768008040 comp rx=0 tx=0).stop 2024-09-15T01:36:26.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff779e20640 10 monclient: shutdown 2024-09-15T01:36:26.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff779e20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff779e20640 1 -- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 msgr2=0x7ff77411c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff779e20640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 0x7ff77411c530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00e980 tx=0x7ff76c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff773fff640 1 -- 172.21.15.178:0/1615210555 reap_dead start 2024-09-15T01:36:26.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff779e20640 1 -- 172.21.15.178:0/1615210555 shutdown_connections 2024-09-15T01:36:26.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff779e20640 1 --2- 172.21.15.178:0/1615210555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7740a2930 0x7ff77411c530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff779e20640 1 -- 172.21.15.178:0/1615210555 >> 172.21.15.178:0/1615210555 conn(0x7ff7740a5ed0 msgr2=0x7ff7740a53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff779e20640 1 -- 172.21.15.178:0/1615210555 shutdown_connections 2024-09-15T01:36:26.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.212+0000 7ff779e20640 1 -- 172.21.15.178:0/1615210555 wait complete. 2024-09-15T01:36:26.265 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:26.265 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:36:26.266 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:36:26.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.331+0000 7f51b171d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:36:26.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.331+0000 7f51b171d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:36:26.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.331+0000 7f51b171d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:26.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.331+0000 7f51b171d640 1 -- 172.21.15.204:0/415194648 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7f51ac0008d0 con 0x7f51ac09ff70 2024-09-15T01:36:26.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.332+0000 7f51a2ffd640 1 -- 172.21.15.204:0/415194648 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v172) v1 ==== 115+0+0 (secure 0 0 0) 0x7f519400e290 con 0x7f51ac09ff70 2024-09-15T01:36:26.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.332+0000 7f51a2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:36:26.335 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.332+0000 7f51a2ffd640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-09-15T01:36:26.335 INFO:tasks.workunit.client.0.smithi204.stderr:compat weight-set already created 2024-09-15T01:36:26.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.333+0000 7f51b171d640 1 -- 172.21.15.204:0/415194648 >> v2:172.21.15.204:6812/28210 conn(0x7f518c0396e0 msgr2=0x7f518c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.333+0000 7f51b171d640 1 --2- 172.21.15.204:0/415194648 >> v2:172.21.15.204:6812/28210 conn(0x7f518c0396e0 0x7f518c03bb90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f519c009c80 tx=0x7f519c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.333+0000 7f51b171d640 10 monclient: shutdown 2024-09-15T01:36:26.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.333+0000 7f51b171d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.333+0000 7f51b171d640 1 -- 172.21.15.204:0/415194648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 msgr2=0x7f51ac11e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.333+0000 7f51b171d640 1 --2- 172.21.15.204:0/415194648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 0x7f51ac11e530 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f519400c5c0 tx=0x7f5194007e80 comp rx=0 tx=0).stop 2024-09-15T01:36:26.337 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.334+0000 7f51b171d640 1 -- 172.21.15.204:0/415194648 shutdown_connections 2024-09-15T01:36:26.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.334+0000 7f51b171d640 1 --2- 172.21.15.204:0/415194648 >> v2:172.21.15.204:6812/28210 conn(0x7f518c0396e0 0x7f518c03bb90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.334+0000 7f51b171d640 1 --2- 172.21.15.204:0/415194648 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f51ac0a2e30 0x7f51ac11ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.334+0000 7f51b171d640 1 --2- 172.21.15.204:0/415194648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51ac09ff70 0x7f51ac11e530 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.334+0000 7f51b171d640 1 --2- 172.21.15.204:0/415194648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51ac09fb80 0x7f51ac1319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.334+0000 7f51b171d640 1 -- 172.21.15.204:0/415194648 >> 172.21.15.204:0/415194648 conn(0x7f51ac06e860 msgr2=0x7f51ac0a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.334+0000 7f51b171d640 1 -- 172.21.15.204:0/415194648 shutdown_connections 2024-09-15T01:36:26.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.334+0000 7f51b171d640 1 -- 172.21.15.204:0/415194648 wait complete. 2024-09-15T01:36:26.345 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket fooo host 2024-09-15T01:36:26.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.356+0000 7fd185c99640 10 monclient: get_monmap_and_config 2024-09-15T01:36:26.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.356+0000 7fd185c99640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.356+0000 7fd185c99640 10 monclient: monmap: 2024-09-15T01:36:26.360 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:26.360 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.360 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:26.358817+0000 2024-09-15T01:36:26.360 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:26.358817+0000 2024-09-15T01:36:26.360 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.361 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.361 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:26.361 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:26.361 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.361 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.361 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.362 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.362 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.362 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.362 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.362 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:26.362 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:26.362 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.357+0000 7fd185c99640 1 Processor -- start 2024-09-15T01:36:26.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.357+0000 7fd185c99640 1 -- start start 2024-09-15T01:36:26.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.357+0000 7fd185c99640 10 monclient: init 2024-09-15T01:36:26.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 10 monclient: _add_conns ranks=[3,9,8,5,2,10,7,0,4,6,1] 2024-09-15T01:36:26.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1800a2930 0x7fd180132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 10 monclient(hunting): picked mon.noname-g con 0x7fd1800a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:26.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd18009faa0 0x7fd18009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 10 monclient(hunting): picked mon.noname-a con 0x7fd18009faa0 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:26.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd1800a08c0 0x7fd1800a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 10 monclient(hunting): picked mon.noname-e con 0x7fd1800a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:26.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18009e0f0 con 0x7fd18009faa0 2024-09-15T01:36:26.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18009dd80 con 0x7fd1800a2930 2024-09-15T01:36:26.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd1800a2d00 con 0x7fd1800a08c0 2024-09-15T01:36:26.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd185c99640 10 monclient(hunting): authenticate will time out at 991.109464s 2024-09-15T01:36:26.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd1800a08c0 0x7fd1800a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd177fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd18009faa0 0x7fd18009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd1800a08c0 0x7fd1800a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:56060/0 (socket says 172.21.15.178:56060) 2024-09-15T01:36:26.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 1 -- 172.21.15.178:0/1318701025 learned_addr learned my addr 172.21.15.178:0/1318701025 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:26.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd177fff640 10 monclient(hunting): get_auth_request con 0x7fd18009faa0 auth_method 0 2024-09-15T01:36:26.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd177fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd177fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd177fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 10 monclient(hunting): get_auth_request con 0x7fd1800a08c0 auth_method 0 2024-09-15T01:36:26.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17f7fe640 1 --2- 172.21.15.178:0/1318701025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1800a2930 0x7fd180132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd177fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd177fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd177fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17f7fe640 10 monclient(hunting): get_auth_request con 0x7fd1800a2930 auth_method 0 2024-09-15T01:36:26.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 10 monclient(hunting): handle_auth_done global_id 5876 payload 274 2024-09-15T01:36:26.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 1 -- 172.21.15.178:0/1318701025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1800a2930 msgr2=0x7fd180132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 1 --2- 172.21.15.178:0/1318701025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1800a2930 0x7fd180132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 1 -- 172.21.15.178:0/1318701025 >> v2:172.21.15.178:3300/0 conn(0x7fd18009faa0 msgr2=0x7fd18009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 1 --2- 172.21.15.178:0/1318701025 >> v2:172.21.15.178:3300/0 conn(0x7fd18009faa0 0x7fd18009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 1 monclient: found mon.noname-e 2024-09-15T01:36:26.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:26.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd17ffff640 1 -- 172.21.15.178:0/1318701025 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18004bf60 con 0x7fd1800a08c0 2024-09-15T01:36:26.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.358+0000 7fd177fff640 1 --2- 172.21.15.178:0/1318701025 >> v2:172.21.15.178:3300/0 conn(0x7fd18009faa0 0x7fd18009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:26.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17ffff640 1 --2- 172.21.15.178:0/1318701025 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd1800a08c0 0x7fd1800a0d60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd178009f20 tx=0x7fd178009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=b42103bc76774ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17effd640 1 -- 172.21.15.178:0/1318701025 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd178007880 con 0x7fd1800a08c0 2024-09-15T01:36:26.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17effd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:26.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17effd640 10 monclient: dump: 2024-09-15T01:36:26.390 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.390 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.390 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17effd640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17effd640 1 -- 172.21.15.178:0/1318701025 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17802d050 con 0x7fd1800a08c0 2024-09-15T01:36:26.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd185c99640 5 monclient: authenticate success, global_id 5876 2024-09-15T01:36:26.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd185c99640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:26.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd185c99640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:26.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17effd640 1 -- 172.21.15.178:0/1318701025 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd178030360 con 0x7fd1800a08c0 2024-09-15T01:36:26.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd185c99640 4 set_mon_vals no callback set 2024-09-15T01:36:26.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:26.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.359+0000 7fd17effd640 10 monclient: dump: 2024-09-15T01:36:26.392 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.393 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.393 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.393 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.393 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.393 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.393 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.393 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.393 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.393 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.393 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.393 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.393 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 10 monclient: shutdown 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 1 -- 172.21.15.178:0/1318701025 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd1800a08c0 msgr2=0x7fd1800a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 1 --2- 172.21.15.178:0/1318701025 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd1800a08c0 0x7fd1800a0d60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd178009f20 tx=0x7fd178009840 comp rx=0 tx=0).stop 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 1 -- 172.21.15.178:0/1318701025 shutdown_connections 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 1 --2- 172.21.15.178:0/1318701025 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd1800a08c0 0x7fd1800a0d60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 1 --2- 172.21.15.178:0/1318701025 >> v2:172.21.15.178:3300/0 conn(0x7fd18009faa0 0x7fd18009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 1 --2- 172.21.15.178:0/1318701025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1800a2930 0x7fd180132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 1 -- 172.21.15.178:0/1318701025 >> 172.21.15.178:0/1318701025 conn(0x7fd1800a5ed0 msgr2=0x7fd1800a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 1 -- 172.21.15.178:0/1318701025 shutdown_connections 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 1 -- 172.21.15.178:0/1318701025 wait complete. 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.360+0000 7fd185c99640 10 monclient: monmap: 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.395 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 1 Processor -- start 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 1 -- start start 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 10 monclient: init 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 10 monclient: _add_conns ranks=[3,4,5,0,8,1,6,2,7] 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd18009faa0 0x7fd18009af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 10 monclient(hunting): picked mon.noname-g con 0x7fd18009faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:26.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1800a08c0 0x7fd180131770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 10 monclient(hunting): picked mon.noname-c con 0x7fd1800a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd1800a2930 0x7fd18011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 10 monclient(hunting): picked mon.noname-h con 0x7fd1800a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18011e890 con 0x7fd18009faa0 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd18009d630 con 0x7fd1800a08c0 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd18011d9a0 con 0x7fd1800a2930 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd185c99640 10 monclient(hunting): authenticate will time out at 991.112830s 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd177fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1800a08c0 0x7fd180131770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd18009faa0 0x7fd18009af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd18009faa0 0x7fd18009af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:57860/0 (socket says 172.21.15.178:57860) 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd177fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1800a08c0 0x7fd180131770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44108/0 (socket says 172.21.15.178:44108) 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd17f7fe640 1 -- 172.21.15.178:0/682883366 learned_addr learned my addr 172.21.15.178:0/682883366 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd17f7fe640 10 monclient(hunting): get_auth_request con 0x7fd18009faa0 auth_method 0 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd17f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd17f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd17f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd177fff640 10 monclient(hunting): get_auth_request con 0x7fd1800a08c0 auth_method 0 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd177fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd177fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd177fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd17ffff640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd1800a2930 0x7fd18011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd17ffff640 10 monclient(hunting): get_auth_request con 0x7fd1800a2930 auth_method 0 2024-09-15T01:36:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd17ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd17ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.361+0000 7fd17ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd177fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd177fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd177fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 10 monclient(hunting): handle_auth_done global_id 15015 payload 824 2024-09-15T01:36:26.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 1 -- 172.21.15.178:0/682883366 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd1800a2930 msgr2=0x7fd18011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd1800a2930 0x7fd18011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 1 -- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1800a08c0 msgr2=0x7fd180131770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1800a08c0 0x7fd180131770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 1 monclient: found mon.noname-g 2024-09-15T01:36:26.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:26.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 1 -- 172.21.15.178:0/682883366 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009390 con 0x7fd18009faa0 2024-09-15T01:36:26.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17f7fe640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd18009faa0 0x7fd18009af70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd17000ea20 tx=0x7fd1700090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd17000ce90 con 0x7fd18009faa0 2024-09-15T01:36:26.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd1777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd1777fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd1777fe640 10 monclient: dump: 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.401 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd1777fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd1777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd170009600 con 0x7fd18009faa0 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd185c99640 5 monclient: authenticate success, global_id 15015 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd1777fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1700138f0 con 0x7fd18009faa0 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd1777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd1777fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd1777fe640 10 monclient: dump: 2024-09-15T01:36:26.402 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.403 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.404 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.404 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd17e7fc640 4 set_mon_vals no callback set 2024-09-15T01:36:26.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd185c99640 10 monclient: _renew_subs 2024-09-15T01:36:26.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd185c99640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:26.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd185c99640 1 -- 172.21.15.178:0/682883366 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd178009ae0 con 0x7fd18009faa0 2024-09-15T01:36:26.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd185c99640 10 monclient: _renew_subs 2024-09-15T01:36:26.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd185c99640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:26.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.362+0000 7fd185c99640 1 -- 172.21.15.178:0/682883366 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18011d700 con 0x7fd18009faa0 2024-09-15T01:36:26.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd185c99640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd185c99640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd180131cb0 0x7fd18011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd185c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd185c99640 1 -- 172.21.15.178:0/682883366 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd18009f940 con 0x7fd180131cb0 2024-09-15T01:36:26.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd170013470 con 0x7fd18009faa0 2024-09-15T01:36:26.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd177fff640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd180131cb0 0x7fd18011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd177fff640 10 monclient: get_auth_request con 0x7fd180131cb0 auth_method 0 2024-09-15T01:36:26.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd177fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd177fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd177fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd177fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd177fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd177fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd1777fe640 1 --2- 172.21.15.178:0/682883366 >> v2:172.21.15.204:6812/28210 conn(0x7fd1600396e0 0x7fd16003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fd17004bf80 con 0x7fd18009faa0 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd177fff640 10 monclient(hunting): handle_auth_done global_id 5836 payload 824 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.363+0000 7fd177fff640 1 -- 172.21.15.178:0/682883366 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd18004c4a0 con 0x7fd180131cb0 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd17ffff640 1 --2- 172.21.15.178:0/682883366 >> v2:172.21.15.204:6812/28210 conn(0x7fd1600396e0 0x7fd16003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd17ffff640 10 monclient: get_auth_request con 0x7fd1600396e0 auth_method 0 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd177fff640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd180131cb0 0x7fd18011ca70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd168011bf0 tx=0x7fd1680090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd16800fd50 con 0x7fd180131cb0 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd1777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd1777fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd1777fe640 10 monclient: dump: 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.407 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd17ffff640 1 --2- 172.21.15.178:0/682883366 >> v2:172.21.15.204:6812/28210 conn(0x7fd1600396e0 0x7fd16003bb90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd178033000 tx=0x7fd178036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fd16800feb0 con 0x7fd180131cb0 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd1777fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd1777fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd180131cb0 msgr2=0x7fd18011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd1777fe640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd180131cb0 0x7fd18011ca70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd168011bf0 tx=0x7fd1680090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd180131cb0 msgr2=0x7fd18011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.364+0000 7fd1777fe640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd180131cb0 0x7fd18011ca70 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd168011bf0 tx=0x7fd1680090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd185c99640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:26.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd185c99640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd18011cfb0 0x7fd180096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd185c99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd185c99640 1 -- 172.21.15.178:0/682883366 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd180132080 con 0x7fd18011cfb0 2024-09-15T01:36:26.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd177fff640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd18011cfb0 0x7fd180096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd177fff640 10 monclient: get_auth_request con 0x7fd18011cfb0 auth_method 0 2024-09-15T01:36:26.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd177fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd177fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd177fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd177fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd177fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd177fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd177fff640 10 monclient(hunting): handle_auth_done global_id 5845 payload 824 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd177fff640 1 -- 172.21.15.178:0/682883366 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd18004bc80 con 0x7fd18011cfb0 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd177fff640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd18011cfb0 0x7fd180096a30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd16801c000 tx=0x7fd16801d000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd168025040 con 0x7fd18011cfb0 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd1777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd1777fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd1777fe640 10 monclient: dump: 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7fd16801c3e0 con 0x7fd18011cfb0 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd1777fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.370+0000 7fd1777fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd18011cfb0 msgr2=0x7fd180096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd1777fe640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd18011cfb0 0x7fd180096a30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd16801c000 tx=0x7fd16801d000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd1777fe640 1 -- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd18011cfb0 msgr2=0x7fd180096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:26.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd1777fe640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd18011cfb0 0x7fd180096a30 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd16801c000 tx=0x7fd16801d000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.412 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:26.412 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:26.412 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:26.412 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:26.412 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:26.412 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:26.412 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:26.413 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:26.414 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.415 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.415 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.415 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.415 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.415 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.415 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.415 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.415 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.416 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.416 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:26.416 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:26.416 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:26.416 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:26.417 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:26.417 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:26.417 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:26.417 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:26.417 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:26.417 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:26.418 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:26.418 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:26.418 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:26.418 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:26.418 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:26.418 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:26.418 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:26.419 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.419 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.419 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.419 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.419 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.419 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.419 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.420 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.420 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.420 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.420 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:26.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.420 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:26.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.421 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:26.421 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:26.421 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:26.422 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.422 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:26.422 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.423 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.423 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:26.423 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:26.423 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.423 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.423 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.424 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:26.424 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:26.424 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.425 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.425 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:26.425 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.426 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:26.426 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.426 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.426 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.427 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:26.427 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:26.427 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.427 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.427 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.428 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:26.428 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:26.428 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.428 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.428 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.428 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:26.429 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.429 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.429 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:26.429 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.429 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.429 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.430 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:26.430 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:26.430 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.430 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.430 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.431 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:26.431 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:26.431 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.431 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.431 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.431 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:26.432 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.432 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.432 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:26.432 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.433 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.433 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.433 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:26.433 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:26.433 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.434 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.434 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.434 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:26.434 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:26.434 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.434 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.434 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.434 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:26.434 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.435 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.436 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:26.437 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.438 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.439 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.440 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:26.440 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.440 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.440 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:26.440 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.440 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.440 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:26.442 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.443 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.444 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:26.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.444 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.444 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.444 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:26.444 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.444 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.445 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:26.445 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:26.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd185c99640 1 -- 172.21.15.178:0/682883366 >> v2:172.21.15.204:6812/28210 conn(0x7fd1600396e0 msgr2=0x7fd16003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd185c99640 1 --2- 172.21.15.178:0/682883366 >> v2:172.21.15.204:6812/28210 conn(0x7fd1600396e0 0x7fd16003bb90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd178033000 tx=0x7fd178036000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd185c99640 10 monclient: shutdown 2024-09-15T01:36:26.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd185c99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd185c99640 1 -- 172.21.15.178:0/682883366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd18009faa0 msgr2=0x7fd18009af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd185c99640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd18009faa0 0x7fd18009af70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd17000ea20 tx=0x7fd1700090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd17ffff640 1 -- 172.21.15.178:0/682883366 reap_dead start 2024-09-15T01:36:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd185c99640 1 -- 172.21.15.178:0/682883366 shutdown_connections 2024-09-15T01:36:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd185c99640 1 --2- 172.21.15.178:0/682883366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd18009faa0 0x7fd18009af70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd185c99640 1 -- 172.21.15.178:0/682883366 >> 172.21.15.178:0/682883366 conn(0x7fd1800a5ed0 msgr2=0x7fd18009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd185c99640 1 -- 172.21.15.178:0/682883366 shutdown_connections 2024-09-15T01:36:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.371+0000 7fd185c99640 1 -- 172.21.15.178:0/682883366 wait complete. 2024-09-15T01:36:26.447 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:26.447 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:36:26.448 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:36:26.458 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.418+0000 7fe9b8f5d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.418+0000 7fe9b8f5d640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.418+0000 7fe9b8f5d640 10 monclient: monmap: 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:26.419640+0000 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:26.419640+0000 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.459 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.418+0000 7fe9b8f5d640 1 Processor -- start 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.418+0000 7fe9b8f5d640 1 -- start start 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.418+0000 7fe9b8f5d640 10 monclient: init 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 10 monclient: _add_conns ranks=[4,9,1,8,2,10,7,0,6,5,3] 2024-09-15T01:36:26.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9b4069d60 0x7fe9b409f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 10 monclient(hunting): picked mon.noname-c con 0x7fe9b4069d60 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe9b406ee50 0x7fe9b409f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 10 monclient(hunting): picked mon.noname-a con 0x7fe9b406ee50 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe9b4132aa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b404b720 con 0x7fe9b4132aa0 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b40698d0 con 0x7fe9b406ee50 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b4069280 con 0x7fe9b4069d60 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b8f5d640 10 monclient(hunting): authenticate will time out at 1002.062465s 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b2d9d640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fe9b4132aa0 0x7fe9b4132f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:45960/0 (socket says 172.21.15.204:45960) 2024-09-15T01:36:26.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b2d9d640 1 -- 172.21.15.204:0/2210288219 learned_addr learned my addr 172.21.15.204:0/2210288219 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b1d9b640 1 --2- 172.21.15.204:0/2210288219 >> v2:172.21.15.178:3300/0 conn(0x7fe9b406ee50 0x7fe9b409f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b259c640 1 --2- 172.21.15.204:0/2210288219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9b4069d60 0x7fe9b409f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b1d9b640 10 monclient(hunting): get_auth_request con 0x7fe9b406ee50 auth_method 0 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b259c640 10 monclient(hunting): get_auth_request con 0x7fe9b4069d60 auth_method 0 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b159a640 1 -- 172.21.15.204:0/2210288219 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443199176 0 0) 0x7fe9b404b720 con 0x7fe9b4132aa0 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b159a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b159a640 10 monclient(hunting): my global_id is 6571 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b159a640 1 -- 172.21.15.204:0/2210288219 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a00034e0 con 0x7fe9b4132aa0 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.419+0000 7fe9b1d9b640 10 monclient(hunting): handle_auth_done global_id 6183 payload 274 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b1d9b640 1 -- 172.21.15.204:0/2210288219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9b4069d60 msgr2=0x7fe9b409f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b1d9b640 1 --2- 172.21.15.204:0/2210288219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9b4069d60 0x7fe9b409f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b1d9b640 1 -- 172.21.15.204:0/2210288219 >> v1:172.21.15.204:6789/0 conn(0x7fe9b4132aa0 legacy=0x7fe9b4132f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b1d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b1d9b640 1 monclient: found mon.noname-a 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b1d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b1d9b640 1 -- 172.21.15.204:0/2210288219 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b406a130 con 0x7fe9b406ee50 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b259c640 1 --2- 172.21.15.204:0/2210288219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9b4069d60 0x7fe9b409f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b1d9b640 1 --2- 172.21.15.204:0/2210288219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9b406ee50 0x7fe9b409f870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009850 tx=0x7fe99c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810df00e6fe0ddf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b159a640 1 -- 172.21.15.204:0/2210288219 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe99c02ac80 con 0x7fe9b406ee50 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b159a640 10 monclient: dump: 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.464 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.465 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.465 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:26.465 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.465 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.465 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.465 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.465 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.465 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.465 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.465 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b159a640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b159a640 1 -- 172.21.15.204:0/2210288219 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99c02ae00 con 0x7fe9b406ee50 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b8f5d640 5 monclient: authenticate success, global_id 6183 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b8f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b159a640 1 -- 172.21.15.204:0/2210288219 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe99c031ca0 con 0x7fe9b406ee50 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.420+0000 7fe9b159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b159a640 10 monclient: dump: 2024-09-15T01:36:26.466 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.467 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b8f5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b8f5d640 4 set_mon_vals no callback set 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b8f5d640 10 monclient: shutdown 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b8f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2210288219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9b406ee50 msgr2=0x7fe9b409f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b8f5d640 1 --2- 172.21.15.204:0/2210288219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9b406ee50 0x7fe9b409f870 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009850 tx=0x7fe99c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2210288219 shutdown_connections 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b8f5d640 1 --2- 172.21.15.204:0/2210288219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9b406ee50 0x7fe9b409f870 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b8f5d640 1 --2- 172.21.15.204:0/2210288219 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9b4069d60 0x7fe9b409f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2210288219 >> 172.21.15.204:0/2210288219 conn(0x7fe9b4067bb0 msgr2=0x7fe9b40a3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2210288219 shutdown_connections 2024-09-15T01:36:26.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.421+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2210288219 wait complete. 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient: monmap: 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.469 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 1 Processor -- start 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 1 -- start start 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient: init 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient: _add_conns ranks=[1,8,4,3,5,7,2,6,0] 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b4069d60 0x7fe9b411e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient(hunting): picked mon.noname-f con 0x7fe9b4069d60 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9b406ee50 0x7fe9b411ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient(hunting): picked mon.noname-e con 0x7fe9b406ee50 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:26.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9b4132aa0 0x7fe9b411f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient(hunting): picked mon.noname-c con 0x7fe9b4132aa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b4068590 con 0x7fe9b4069d60 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b4121e90 con 0x7fe9b4132aa0 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b4120080 con 0x7fe9b406ee50 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.422+0000 7fe9b8f5d640 10 monclient(hunting): authenticate will time out at 1002.065975s 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b4069d60 0x7fe9b411e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b4069d60 0x7fe9b411e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:52810/0 (socket says 172.21.15.204:52810) 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 1 -- 172.21.15.204:0/2803170416 learned_addr learned my addr 172.21.15.204:0/2803170416 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:26.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 10 monclient(hunting): get_auth_request con 0x7fe9b4069d60 auth_method 0 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b2d9d640 1 --2- 172.21.15.204:0/2803170416 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9b4132aa0 0x7fe9b411f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b2d9d640 10 monclient(hunting): get_auth_request con 0x7fe9b4132aa0 auth_method 0 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b1d9b640 1 --2- 172.21.15.204:0/2803170416 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9b406ee50 0x7fe9b411ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b1d9b640 10 monclient(hunting): get_auth_request con 0x7fe9b406ee50 auth_method 0 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 10 monclient(hunting): handle_auth_done global_id 6580 payload 824 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 1 -- 172.21.15.204:0/2803170416 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9b406ee50 msgr2=0x7fe9b411ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 1 --2- 172.21.15.204:0/2803170416 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9b406ee50 0x7fe9b411ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 1 -- 172.21.15.204:0/2803170416 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9b4132aa0 msgr2=0x7fe9b411f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 1 --2- 172.21.15.204:0/2803170416 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9b4132aa0 0x7fe9b411f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 1 monclient: found mon.noname-f 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:26.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 1 -- 172.21.15.204:0/2803170416 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac009a20 con 0x7fe9b4069d60 2024-09-15T01:36:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b2d9d640 1 --2- 172.21.15.204:0/2803170416 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9b4132aa0 0x7fe9b411f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:36:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b259c640 1 --2- 172.21.15.204:0/2803170416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b4069d60 0x7fe9b411e730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00c0c0 tx=0x7fe9ac00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9aa7fc640 1 -- 172.21.15.204:0/2803170416 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9ac00cea0 con 0x7fe9b4069d60 2024-09-15T01:36:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9aa7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9aa7fc640 10 monclient: dump: 2024-09-15T01:36:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.474 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.475 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.476 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9aa7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9aa7fc640 1 -- 172.21.15.204:0/2803170416 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac00f040 con 0x7fe9b4069d60 2024-09-15T01:36:26.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b8f5d640 5 monclient: authenticate success, global_id 6580 2024-09-15T01:36:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b8f5d640 10 monclient: _renew_subs 2024-09-15T01:36:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b8f5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2803170416 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99c009390 con 0x7fe9b4069d60 2024-09-15T01:36:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9aa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9aa7fc640 1 -- 172.21.15.204:0/2803170416 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9ac014d80 con 0x7fe9b4069d60 2024-09-15T01:36:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9aa7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9aa7fc640 10 monclient: dump: 2024-09-15T01:36:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.477 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.478 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b0d99640 4 set_mon_vals no callback set 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b8f5d640 10 monclient: _renew_subs 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b8f5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.423+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2803170416 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b41218a0 con 0x7fe9b4069d60 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.424+0000 7fe9aa7fc640 1 -- 172.21.15.204:0/2803170416 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe9ac003020 con 0x7fe9b4069d60 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.424+0000 7fe9b8f5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.424+0000 7fe9b8f5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.424+0000 7fe9b8f5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.424+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2803170416 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b4070cc0 con 0x7fe9b4069d60 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.424+0000 7fe9aa7fc640 1 --2- 172.21.15.204:0/2803170416 >> v2:172.21.15.204:6812/28210 conn(0x7fe9940397b0 0x7fe99403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.424+0000 7fe9aa7fc640 1 -- 172.21.15.204:0/2803170416 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fe9ac04c620 con 0x7fe9b4069d60 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.424+0000 7fe9b1d9b640 1 --2- 172.21.15.204:0/2803170416 >> v2:172.21.15.204:6812/28210 conn(0x7fe9940397b0 0x7fe99403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.424+0000 7fe9b1d9b640 10 monclient: get_auth_request con 0x7fe9940397b0 auth_method 0 2024-09-15T01:36:26.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.425+0000 7fe9b1d9b640 1 --2- 172.21.15.204:0/2803170416 >> v2:172.21.15.204:6812/28210 conn(0x7fe9940397b0 0x7fe99403bc60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe99c034000 tx=0x7fe99c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.428+0000 7fe9aa7fc640 1 -- 172.21.15.204:0/2803170416 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe9ac01a870 con 0x7fe9b4069d60 2024-09-15T01:36:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.428+0000 7fe9aa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.428+0000 7fe9aa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:26.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.555+0000 7fbeccb5d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:26.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.555+0000 7fbeccb5d640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.555+0000 7fbeccb5d640 10 monclient: monmap: 2024-09-15T01:36:26.558 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:26.558 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.558 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:26.557061+0000 2024-09-15T01:36:26.558 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:26.557061+0000 2024-09-15T01:36:26.558 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:26.559 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.555+0000 7fbeccb5d640 1 Processor -- start 2024-09-15T01:36:26.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.555+0000 7fbeccb5d640 1 -- start start 2024-09-15T01:36:26.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.555+0000 7fbeccb5d640 10 monclient: init 2024-09-15T01:36:26.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 10 monclient: _add_conns ranks=[6,3,10,2,7,9,0,4,1,5,8] 2024-09-15T01:36:26.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec80a2930 0x7fbec8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 10 monclient(hunting): picked mon.noname-d con 0x7fbec80a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:26.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec809faa0 0x7fbec809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 10 monclient(hunting): picked mon.noname-g con 0x7fbec809faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:26.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbec80a08c0 0x7fbec80a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 10 monclient(hunting): picked mon.noname-f con 0x7fbec80a08c0 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec809e0f0 con 0x7fbec80a08c0 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec809dd80 con 0x7fbec809faa0 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbec80a2d00 con 0x7fbec80a2930 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbeccb5d640 10 monclient(hunting): authenticate will time out at 991.307503s 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec80a2930 0x7fbec8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec80a2930 0x7fbec8132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:60108/0 (socket says 172.21.15.178:60108) 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 1 -- 172.21.15.178:0/3913966715 learned_addr learned my addr 172.21.15.178:0/3913966715 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 10 monclient(hunting): get_auth_request con 0x7fbec80a2930 auth_method 0 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec5d9b640 1 --2- 172.21.15.178:0/3913966715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec809faa0 0x7fbec809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec6d9d640 1 --2- 172.21.15.178:0/3913966715 >> v2:172.21.15.204:3300/0 conn(0x7fbec80a08c0 0x7fbec80a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec5d9b640 10 monclient(hunting): get_auth_request con 0x7fbec809faa0 auth_method 0 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec6d9d640 10 monclient(hunting): get_auth_request con 0x7fbec80a08c0 auth_method 0 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 10 monclient(hunting): handle_auth_done global_id 24126 payload 274 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 1 -- 172.21.15.178:0/3913966715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec809faa0 msgr2=0x7fbec809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 1 --2- 172.21.15.178:0/3913966715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec809faa0 0x7fbec809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 1 -- 172.21.15.178:0/3913966715 >> v2:172.21.15.204:3300/0 conn(0x7fbec80a08c0 msgr2=0x7fbec80a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 1 --2- 172.21.15.178:0/3913966715 >> v2:172.21.15.204:3300/0 conn(0x7fbec80a08c0 0x7fbec80a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 1 monclient: found mon.noname-d 2024-09-15T01:36:26.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:26.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.556+0000 7fbec659c640 1 -- 172.21.15.178:0/3913966715 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec804bf60 con 0x7fbec80a2930 2024-09-15T01:36:26.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec659c640 1 --2- 172.21.15.178:0/3913966715 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec80a2930 0x7fbec8132de0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009910 tx=0x7fbeb00091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=444c16c6ada77588 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec559a640 1 -- 172.21.15.178:0/3913966715 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbeb002ac80 con 0x7fbec80a2930 2024-09-15T01:36:26.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec559a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:26.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec559a640 10 monclient: dump: 2024-09-15T01:36:26.576 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.576 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.576 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.576 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.577 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.577 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.577 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.577 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.577 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.577 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.577 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.577 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.577 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.577 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.578 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.578 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec559a640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbeccb5d640 5 monclient: authenticate success, global_id 24126 2024-09-15T01:36:26.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec559a640 1 -- 172.21.15.178:0/3913966715 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb002ae00 con 0x7fbec80a2930 2024-09-15T01:36:26.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec559a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec559a640 1 -- 172.21.15.178:0/3913966715 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbeb0031ca0 con 0x7fbec80a2930 2024-09-15T01:36:26.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbeccb5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:26.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbeccb5d640 4 set_mon_vals no callback set 2024-09-15T01:36:26.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec559a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:26.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.557+0000 7fbec559a640 10 monclient: dump: 2024-09-15T01:36:26.579 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.579 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.579 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.579 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.579 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.579 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.579 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.580 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.580 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.580 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.580 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.580 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.580 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.580 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.580 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.580 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 10 monclient: shutdown 2024-09-15T01:36:26.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3913966715 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec80a2930 msgr2=0x7fbec8132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 1 --2- 172.21.15.178:0/3913966715 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec80a2930 0x7fbec8132de0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009910 tx=0x7fbeb00091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3913966715 shutdown_connections 2024-09-15T01:36:26.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 1 --2- 172.21.15.178:0/3913966715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec809faa0 0x7fbec809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 1 --2- 172.21.15.178:0/3913966715 >> v2:172.21.15.204:3300/0 conn(0x7fbec80a08c0 0x7fbec80a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 1 --2- 172.21.15.178:0/3913966715 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec80a2930 0x7fbec8132de0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3913966715 >> 172.21.15.178:0/3913966715 conn(0x7fbec80a5ed0 msgr2=0x7fbec80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3913966715 shutdown_connections 2024-09-15T01:36:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3913966715 wait complete. 2024-09-15T01:36:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.558+0000 7fbeccb5d640 10 monclient: monmap: 2024-09-15T01:36:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 1 Processor -- start 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 1 -- start start 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 10 monclient: init 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 10 monclient: _add_conns ranks=[4,3,7,0,6,2,8,5,1] 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbec809faa0 0x7fbec809b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 10 monclient(hunting): picked mon.noname-c con 0x7fbec809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec80a08c0 0x7fbec8131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 10 monclient(hunting): picked mon.noname-g con 0x7fbec80a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbec80a2930 0x7fbec8141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 10 monclient(hunting): picked mon.noname-i con 0x7fbec80a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec80a23e0 con 0x7fbec80a08c0 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbec809d630 con 0x7fbec809faa0 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbec809bdf0 con 0x7fbec80a2930 2024-09-15T01:36:26.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbeccb5d640 10 monclient(hunting): authenticate will time out at 991.310825s 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbec809faa0 0x7fbec809b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbec809faa0 0x7fbec809b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44152/0 (socket says 172.21.15.178:44152) 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec659c640 1 -- 172.21.15.178:0/3609806041 learned_addr learned my addr 172.21.15.178:0/3609806041 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec659c640 10 monclient(hunting): get_auth_request con 0x7fbec809faa0 auth_method 0 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec5d9b640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec80a08c0 0x7fbec8131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec6d9d640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbec80a2930 0x7fbec8141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec5d9b640 10 monclient(hunting): get_auth_request con 0x7fbec80a08c0 auth_method 0 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec6d9d640 10 monclient(hunting): get_auth_request con 0x7fbec80a2930 auth_method 0 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.559+0000 7fbec659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec659c640 10 monclient(hunting): handle_auth_done global_id 5872 payload 824 2024-09-15T01:36:26.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec659c640 1 -- 172.21.15.178:0/3609806041 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbec80a2930 msgr2=0x7fbec8141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec659c640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbec80a2930 0x7fbec8141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec659c640 1 -- 172.21.15.178:0/3609806041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec80a08c0 msgr2=0x7fbec8131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec659c640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec80a08c0 0x7fbec8131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec659c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec659c640 1 monclient: found mon.noname-c 2024-09-15T01:36:26.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec659c640 1 -- 172.21.15.178:0/3609806041 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb0009450 con 0x7fbec809faa0 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec5d9b640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbec80a08c0 0x7fbec8131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec659c640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbec809faa0 0x7fbec809b240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0000c00 tx=0x7fbeb0032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbeb0031820 con 0x7fbec809faa0 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbebe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbebe7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbebe7fc640 10 monclient: dump: 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.588 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbebe7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbebe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb00319e0 con 0x7fbec809faa0 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbeccb5d640 5 monclient: authenticate success, global_id 5872 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbebe7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbeb003a390 con 0x7fbec809faa0 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbeccb5d640 10 monclient: _renew_subs 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbeccb5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3609806041 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec811ead0 con 0x7fbec809faa0 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbebe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbebe7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbec4d99640 4 set_mon_vals no callback set 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbebe7fc640 10 monclient: dump: 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.590 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.591 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.591 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.591 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.591 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.591 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.591 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.591 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.591 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.591 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.591 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.591 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.592 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.592 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbeccb5d640 10 monclient: _renew_subs 2024-09-15T01:36:26.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbeccb5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:36:26.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.560+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3609806041 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec811db50 con 0x7fbec809faa0 2024-09-15T01:36:26.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbeccb5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbeccb5d640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec811ca40 0x7fbec8142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbeccb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3609806041 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbec811eec0 con 0x7fbec811ca40 2024-09-15T01:36:26.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fbeb0039040 con 0x7fbec809faa0 2024-09-15T01:36:26.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec5d9b640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec811ca40 0x7fbec8142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec5d9b640 10 monclient: get_auth_request con 0x7fbec811ca40 auth_method 0 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_done global_id 24135 payload 824 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbebe7fc640 1 --2- 172.21.15.178:0/3609806041 >> v2:172.21.15.204:6812/28210 conn(0x7fbea80397b0 0x7fbea803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec5d9b640 1 -- 172.21.15.178:0/3609806041 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbec804c4a0 con 0x7fbec811ca40 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fbeb006b050 con 0x7fbec809faa0 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec5d9b640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec811ca40 0x7fbec8142260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8011d60 tx=0x7fbeb80090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec6d9d640 1 --2- 172.21.15.178:0/3609806041 >> v2:172.21.15.204:6812/28210 conn(0x7fbea80397b0 0x7fbea803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbec6d9d640 10 monclient: get_auth_request con 0x7fbea80397b0 auth_method 0 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbeb80148c0 con 0x7fbec811ca40 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbebe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbebe7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.561+0000 7fbebe7fc640 10 monclient: dump: 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.594 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.595 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.595 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.595 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.595 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.595 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.595 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.595 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.562+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fbeb8014440 con 0x7fbec811ca40 2024-09-15T01:36:26.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.562+0000 7fbebe7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.562+0000 7fbebe7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:26.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.562+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec811ca40 msgr2=0x7fbec8142260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.562+0000 7fbebe7fc640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec811ca40 0x7fbec8142260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8011d60 tx=0x7fbeb80090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.562+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec811ca40 msgr2=0x7fbec8142260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.562+0000 7fbebe7fc640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec811ca40 0x7fbec8142260 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8011d60 tx=0x7fbeb80090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.562+0000 7fbec6d9d640 1 --2- 172.21.15.178:0/3609806041 >> v2:172.21.15.204:6812/28210 conn(0x7fbea80397b0 0x7fbea803bc60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009fd0 tx=0x7fbec000b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.567+0000 7fbeccb5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.567+0000 7fbeccb5d640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec8131e60 0x7fbec81427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.567+0000 7fbeccb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.567+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3609806041 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbec804bc80 con 0x7fbec8131e60 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbec5d9b640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec8131e60 0x7fbec81427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbec5d9b640 10 monclient: get_auth_request con 0x7fbec8131e60 auth_method 0 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbec5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbec5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbec5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbec5d9b640 10 monclient(hunting): handle_auth_done global_id 24144 payload 824 2024-09-15T01:36:26.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbec5d9b640 1 -- 172.21.15.178:0/3609806041 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbec805ed70 con 0x7fbec8131e60 2024-09-15T01:36:26.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbec5d9b640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec8131e60 0x7fbec81427a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8011d60 tx=0x7fbeb801c000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbeb8024040 con 0x7fbec8131e60 2024-09-15T01:36:26.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbebe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbebe7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:26.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbebe7fc640 10 monclient: dump: 2024-09-15T01:36:26.597 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.597 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.598 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.598 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.598 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.598 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.598 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.598 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.598 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.598 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.598 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.598 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.598 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.598 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.599 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.599 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7fbeb80133a0 con 0x7fbec8131e60 2024-09-15T01:36:26.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbebe7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:26.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbebe7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:26.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec8131e60 msgr2=0x7fbec81427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbebe7fc640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec8131e60 0x7fbec81427a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8011d60 tx=0x7fbeb801c000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbebe7fc640 1 -- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec8131e60 msgr2=0x7fbec81427a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:26.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.568+0000 7fbebe7fc640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbec8131e60 0x7fbec81427a0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8011d60 tx=0x7fbeb801c000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.599 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:26.600 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:26.600 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:26.600 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:26.600 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:26.600 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:26.600 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:26.600 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:26.600 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:26.601 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:26.601 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:26.601 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:26.601 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:26.601 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:26.601 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:26.601 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.601 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:36:26.601 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:26.602 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:26.602 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:26.602 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.602 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.602 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.602 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.602 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.602 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.602 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.602 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.602 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.604 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:26.604 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:26.604 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:26.604 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:26.604 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:26.604 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:26.604 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:26.604 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:26.604 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:26.604 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:26.604 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:26.604 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.605 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.607 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:26.607 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.607 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.607 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:26.607 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:26.607 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.607 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.607 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.608 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:26.608 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:26.608 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.608 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.611 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:26.611 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.611 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.611 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:26.611 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:26.611 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.611 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.611 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.612 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:26.612 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:26.612 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.612 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.612 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:26.613 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.613 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:26.613 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.613 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.613 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.614 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:26.614 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:26.614 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.614 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.614 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.614 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:26.614 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:26.614 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.615 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:26.615 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.617 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:26.617 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:26.617 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.617 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.617 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.617 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:26.617 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.618 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.618 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:26.618 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.618 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.618 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.618 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:26.618 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:26.618 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.619 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.619 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.619 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:26.619 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:26.620 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.620 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:26.620 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.621 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.621 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.621 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:26.621 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.621 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.622 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.622 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:26.622 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:26.622 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.622 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.622 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.623 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:26.623 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:26.623 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.623 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.624 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.624 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:26.624 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.624 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.624 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.625 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:26.625 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.625 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.625 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.625 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.625 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:26.625 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:26.625 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.625 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.625 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.625 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.626 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:26.626 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:26.626 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.626 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.626 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.626 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:26.626 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.626 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.627 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.627 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:26.627 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.627 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.627 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.627 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.627 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:26.627 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:26.627 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.627 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.627 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.628 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.628 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.628 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.628 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:26.628 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:26.628 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.629 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.629 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.629 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:26.629 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.629 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.629 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:26.629 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.630 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.630 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.630 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:26.630 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.630 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.630 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.631 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:26.631 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:26.632 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.602+0000 7fe9b8f5d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-09-15T01:36:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.603+0000 7fe9b8f5d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-09-15T01:36:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.603+0000 7fe9b8f5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:26.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:26.603+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2803170416 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"} v 0) v1 -- 0x7fe9b406ea20 con 0x7fe9b4069d60 2024-09-15T01:36:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.569+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3609806041 >> v2:172.21.15.204:6812/28210 conn(0x7fbea80397b0 msgr2=0x7fbea803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.569+0000 7fbeccb5d640 1 --2- 172.21.15.178:0/3609806041 >> v2:172.21.15.204:6812/28210 conn(0x7fbea80397b0 0x7fbea803bc60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009fd0 tx=0x7fbec000b040 comp rx=0 tx=0).stop 2024-09-15T01:36:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.569+0000 7fbeccb5d640 10 monclient: shutdown 2024-09-15T01:36:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.569+0000 7fbeccb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.569+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbec809faa0 msgr2=0x7fbec809b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.569+0000 7fbeccb5d640 1 --2- 172.21.15.178:0/3609806041 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbec809faa0 0x7fbec809b240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0000c00 tx=0x7fbeb0032000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.569+0000 7fbec6d9d640 1 -- 172.21.15.178:0/3609806041 reap_dead start 2024-09-15T01:36:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.569+0000 7fbec6d9d640 1 -- 172.21.15.178:0/3609806041 reap_dead start 2024-09-15T01:36:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.569+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3609806041 shutdown_connections 2024-09-15T01:36:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.569+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3609806041 >> 172.21.15.178:0/3609806041 conn(0x7fbec80a5ed0 msgr2=0x7fbec81358d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.569+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3609806041 shutdown_connections 2024-09-15T01:36:26.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.569+0000 7fbeccb5d640 1 -- 172.21.15.178:0/3609806041 wait complete. 2024-09-15T01:36:26.637 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:26.637 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:36:26.637 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:36:26.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.728+0000 7f6f3eec9640 10 monclient: get_monmap_and_config 2024-09-15T01:36:26.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.728+0000 7f6f3eec9640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.728+0000 7f6f3eec9640 10 monclient: monmap: 2024-09-15T01:36:26.731 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:26.731 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:26.730395+0000 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:26.730395+0000 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:26.733 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:26.733 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 1 Processor -- start 2024-09-15T01:36:26.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 1 -- start start 2024-09-15T01:36:26.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 10 monclient: init 2024-09-15T01:36:26.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 10 monclient: _add_conns ranks=[7,4,5,6,3,9,0,8,10,1,2] 2024-09-15T01:36:26.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f380a2930 0x7f6f38132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 10 monclient(hunting): picked mon.noname-i con 0x7f6f380a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:26.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f3809faa0 0x7f6f3809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 10 monclient(hunting): picked mon.noname-c con 0x7f6f3809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f380a08c0 0x7f6f380a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 10 monclient(hunting): picked mon.noname-h con 0x7f6f380a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3809e0f0 con 0x7f6f3809faa0 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3809dd80 con 0x7f6f380a08c0 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6f380a2d00 con 0x7f6f380a2930 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f3eec9640 10 monclient(hunting): authenticate will time out at 991.480748s 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f37fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f3809faa0 0x7f6f3809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f37fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f3809faa0 0x7f6f3809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44162/0 (socket says 172.21.15.178:44162) 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f37fff640 1 -- 172.21.15.178:0/2451016544 learned_addr learned my addr 172.21.15.178:0/2451016544 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:26.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f37fff640 10 monclient(hunting): get_auth_request con 0x7f6f3809faa0 auth_method 0 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f37fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f37fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.729+0000 7f6f37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f3cc65640 1 --2- 172.21.15.178:0/2451016544 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f380a2930 0x7f6f38132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f3d466640 1 --2- 172.21.15.178:0/2451016544 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f380a08c0 0x7f6f380a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f3cc65640 10 monclient(hunting): get_auth_request con 0x7f6f380a2930 auth_method 0 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f3cc65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f3cc65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f3cc65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f3d466640 10 monclient(hunting): get_auth_request con 0x7f6f380a08c0 auth_method 0 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f3d466640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f3d466640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f3d466640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f37fff640 10 monclient(hunting): handle_auth_done global_id 5881 payload 274 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f37fff640 1 -- 172.21.15.178:0/2451016544 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f380a2930 msgr2=0x7f6f38132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f37fff640 1 --2- 172.21.15.178:0/2451016544 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f380a2930 0x7f6f38132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f37fff640 1 -- 172.21.15.178:0/2451016544 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f380a08c0 msgr2=0x7f6f380a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f37fff640 1 --2- 172.21.15.178:0/2451016544 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f380a08c0 0x7f6f380a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f37fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f37fff640 1 monclient: found mon.noname-c 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f37fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f37fff640 1 -- 172.21.15.178:0/2451016544 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3804bf60 con 0x7f6f3809faa0 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f37fff640 1 --2- 172.21.15.178:0/2451016544 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f3809faa0 0x7f6f3809fe70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009940 tx=0x7f6f240091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=b07d82ac3e28cd97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f377fe640 1 -- 172.21.15.178:0/2451016544 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f2402ac60 con 0x7f6f3809faa0 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.730+0000 7f6f377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f377fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f377fe640 10 monclient: dump: 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f377fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f377fe640 1 -- 172.21.15.178:0/2451016544 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2402ade0 con 0x7f6f3809faa0 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 5 monclient: authenticate success, global_id 5881 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f377fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f377fe640 1 -- 172.21.15.178:0/2451016544 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f24031ca0 con 0x7f6f3809faa0 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 4 set_mon_vals no callback set 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f377fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f377fe640 10 monclient: dump: 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.747 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 10 monclient: shutdown 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 1 -- 172.21.15.178:0/2451016544 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f3809faa0 msgr2=0x7f6f3809fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 1 --2- 172.21.15.178:0/2451016544 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f3809faa0 0x7f6f3809fe70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009940 tx=0x7f6f240091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 1 -- 172.21.15.178:0/2451016544 shutdown_connections 2024-09-15T01:36:26.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 1 --2- 172.21.15.178:0/2451016544 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f380a08c0 0x7f6f380a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 1 --2- 172.21.15.178:0/2451016544 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f3809faa0 0x7f6f3809fe70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 1 --2- 172.21.15.178:0/2451016544 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f380a2930 0x7f6f38132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 1 -- 172.21.15.178:0/2451016544 >> 172.21.15.178:0/2451016544 conn(0x7f6f380a5ed0 msgr2=0x7f6f380a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 1 -- 172.21.15.178:0/2451016544 shutdown_connections 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 1 -- 172.21.15.178:0/2451016544 wait complete. 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.731+0000 7f6f3eec9640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.732+0000 7f6f3eec9640 10 monclient: monmap: 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:26.749 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.732+0000 7f6f3eec9640 1 Processor -- start 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.732+0000 7f6f3eec9640 1 -- start start 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.732+0000 7f6f3eec9640 10 monclient: init 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 10 monclient: _add_conns ranks=[6,2,8,5,1,7,4,0,3] 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f3809faa0 0x7f6f3809b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 10 monclient(hunting): picked mon.noname-d con 0x7f6f3809faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f380a08c0 0x7f6f381316c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 10 monclient(hunting): picked mon.noname-b con 0x7f6f380a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f380a2930 0x7f6f3811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 10 monclient(hunting): picked mon.noname-e con 0x7f6f380a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3809c2c0 con 0x7f6f380a08c0 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6f380a2390 con 0x7f6f3809faa0 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3809d630 con 0x7f6f380a2930 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 10 monclient(hunting): authenticate will time out at 991.484089s 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3cc65640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f3809faa0 0x7f6f3809b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f380a08c0 0x7f6f381316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3d466640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f380a2930 0x7f6f3811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3cc65640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f3809faa0 0x7f6f3809b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:60134/0 (socket says 172.21.15.178:60134) 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3d466640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f380a2930 0x7f6f3811c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:56068/0 (socket says 172.21.15.178:56068) 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3cc65640 1 -- 172.21.15.178:0/1934549696 learned_addr learned my addr 172.21.15.178:0/1934549696 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 10 monclient(hunting): get_auth_request con 0x7f6f380a08c0 auth_method 0 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3cc65640 10 monclient(hunting): get_auth_request con 0x7f6f3809faa0 auth_method 0 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3cc65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3cc65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3cc65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3d466640 10 monclient(hunting): get_auth_request con 0x7f6f380a2930 auth_method 0 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3d466640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3d466640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3d466640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3cc65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3cc65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3cc65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3d466640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3d466640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3d466640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 10 monclient(hunting): handle_auth_done global_id 24158 payload 824 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 1 -- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f380a2930 msgr2=0x7f6f3811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f380a2930 0x7f6f3811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 1 -- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f3809faa0 msgr2=0x7f6f3809b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f3809faa0 0x7f6f3809b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 1 monclient: found mon.noname-b 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 1 -- 172.21.15.178:0/1934549696 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f24009450 con 0x7f6f380a08c0 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3cc65640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f3809faa0 0x7f6f3809b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3d466640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f380a2930 0x7f6f3811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:26.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f37fff640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f380a08c0 0x7f6f381316c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6f24031000 tx=0x7f6f24032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f24006b30 con 0x7f6f380a08c0 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f34ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f34ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f34ff9640 10 monclient: dump: 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.755 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f34ff9640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 5 monclient: authenticate success, global_id 24158 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2402ac60 con 0x7f6f380a08c0 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 10 monclient: _renew_subs 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f3eec9640 1 -- 172.21.15.178:0/1934549696 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f38142c90 con 0x7f6f380a08c0 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f34ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f24033e00 con 0x7f6f380a08c0 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f34ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f34ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f34ff9640 10 monclient: dump: 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.757 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.733+0000 7f6f36ffd640 4 set_mon_vals no callback set 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3eec9640 10 monclient: _renew_subs 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3eec9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3eec9640 1 -- 172.21.15.178:0/1934549696 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3809bd40 con 0x7f6f380a08c0 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3eec9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6f24039040 con 0x7f6f380a08c0 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3eec9640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f38131c00 0x7f6f3811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3eec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3eec9640 1 -- 172.21.15.178:0/1934549696 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38131fd0 con 0x7f6f38131c00 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3cc65640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f38131c00 0x7f6f3811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3cc65640 10 monclient: get_auth_request con 0x7f6f38131c00 auth_method 0 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3cc65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3cc65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3cc65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3cc65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3cc65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f3cc65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.734+0000 7f6f34ff9640 1 --2- 172.21.15.178:0/1934549696 >> v2:172.21.15.204:6812/28210 conn(0x7f6f1c0396d0 0x7f6f1c03bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f6f2406b420 con 0x7f6f380a08c0 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f3cc65640 10 monclient(hunting): handle_auth_done global_id 5903 payload 824 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f3cc65640 1 -- 172.21.15.178:0/1934549696 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f3804c4a0 con 0x7f6f38131c00 2024-09-15T01:36:26.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f3d466640 1 --2- 172.21.15.178:0/1934549696 >> v2:172.21.15.204:6812/28210 conn(0x7f6f1c0396d0 0x7f6f1c03bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f3d466640 10 monclient: get_auth_request con 0x7f6f1c0396d0 auth_method 0 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f3cc65640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f38131c00 0x7f6f3811ca70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0027b0 tx=0x7f6f2c009920 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f2c010d40 con 0x7f6f38131c00 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f34ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f34ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f34ff9640 10 monclient: dump: 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.760 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f3d466640 1 --2- 172.21.15.178:0/1934549696 >> v2:172.21.15.204:6812/28210 conn(0x7f6f1c0396d0 0x7f6f1c03bb80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6f3809b900 tx=0x7f6f30008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f6f2c010ea0 con 0x7f6f38131c00 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f34ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f34ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:26.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f38131c00 msgr2=0x7f6f3811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f34ff9640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f38131c00 0x7f6f3811ca70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0027b0 tx=0x7f6f2c009920 comp rx=0 tx=0).stop 2024-09-15T01:36:26.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f38131c00 msgr2=0x7f6f3811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:26.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.735+0000 7f6f34ff9640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f38131c00 0x7f6f3811ca70 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0027b0 tx=0x7f6f2c009920 comp rx=0 tx=0).stop 2024-09-15T01:36:26.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3eec9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:26.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3eec9640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f3811cfb0 0x7f6f38096b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3eec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3eec9640 1 -- 172.21.15.178:0/1934549696 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3804bc80 con 0x7f6f3811cfb0 2024-09-15T01:36:26.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3cc65640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f3811cfb0 0x7f6f38096b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3cc65640 10 monclient: get_auth_request con 0x7f6f3811cfb0 auth_method 0 2024-09-15T01:36:26.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3cc65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3cc65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3cc65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3cc65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3cc65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3cc65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3cc65640 10 monclient(hunting): handle_auth_done global_id 5912 payload 824 2024-09-15T01:36:26.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3cc65640 1 -- 172.21.15.178:0/1934549696 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6f3805ed70 con 0x7f6f3811cfb0 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f3cc65640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f3811cfb0 0x7f6f38096b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c011010 tx=0x7f6f2c01b000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f2c015c90 con 0x7f6f3811cfb0 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f34ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f34ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.741+0000 7f6f34ff9640 10 monclient: dump: 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.765 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f6f2c010650 con 0x7f6f3811cfb0 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f34ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f34ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f3811cfb0 msgr2=0x7f6f38096b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f34ff9640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f3811cfb0 0x7f6f38096b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c011010 tx=0x7f6f2c01b000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f34ff9640 1 -- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f3811cfb0 msgr2=0x7f6f38096b30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:26.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f34ff9640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f3811cfb0 0x7f6f38096b30 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c011010 tx=0x7f6f2c01b000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.767 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:26.767 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:26.767 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:26.767 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:26.767 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:26.767 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:26.767 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:26.767 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:26.767 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.768 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.769 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:26.770 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.771 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:26.772 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.775 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.776 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:26.777 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.778 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.779 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:26.780 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.782 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.784 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:26.785 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:36:26.788 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.789 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.789 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:26.789 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:26.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f3eec9640 1 -- 172.21.15.178:0/1934549696 >> v2:172.21.15.204:6812/28210 conn(0x7f6f1c0396d0 msgr2=0x7f6f1c03bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f3eec9640 1 --2- 172.21.15.178:0/1934549696 >> v2:172.21.15.204:6812/28210 conn(0x7f6f1c0396d0 0x7f6f1c03bb80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6f3809b900 tx=0x7f6f30008040 comp rx=0 tx=0).stop 2024-09-15T01:36:26.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f3eec9640 10 monclient: shutdown 2024-09-15T01:36:26.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f3eec9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f3eec9640 1 -- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f380a08c0 msgr2=0x7f6f381316c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f3eec9640 1 --2- 172.21.15.178:0/1934549696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f380a08c0 0x7f6f381316c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6f24031000 tx=0x7f6f24032000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f3d466640 1 -- 172.21.15.178:0/1934549696 reap_dead start 2024-09-15T01:36:26.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f3d466640 1 -- 172.21.15.178:0/1934549696 reap_dead start 2024-09-15T01:36:26.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f3eec9640 1 -- 172.21.15.178:0/1934549696 shutdown_connections 2024-09-15T01:36:26.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f3eec9640 1 -- 172.21.15.178:0/1934549696 >> 172.21.15.178:0/1934549696 conn(0x7f6f380a5ed0 msgr2=0x7f6f3809e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f3eec9640 1 -- 172.21.15.178:0/1934549696 shutdown_connections 2024-09-15T01:36:26.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.742+0000 7f6f3eec9640 1 -- 172.21.15.178:0/1934549696 wait complete. 2024-09-15T01:36:26.791 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:26.791 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:36:26.792 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:36:26.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.883+0000 7f176571d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:26.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.883+0000 7f176571d640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.883+0000 7f176571d640 10 monclient: monmap: 2024-09-15T01:36:26.886 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:26.886 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.886 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:26.884954+0000 2024-09-15T01:36:26.886 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:26.884954+0000 2024-09-15T01:36:26.886 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.886 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.886 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:26.886 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:26.886 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.887 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.887 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.887 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.887 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.887 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.887 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.887 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:26.887 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:26.887 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.883+0000 7f176571d640 1 Processor -- start 2024-09-15T01:36:26.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.883+0000 7f176571d640 1 -- start start 2024-09-15T01:36:26.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.883+0000 7f176571d640 10 monclient: init 2024-09-15T01:36:26.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 10 monclient: _add_conns ranks=[3,4,8,0,6,7,10,2,5,9,1] 2024-09-15T01:36:26.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17600678d0 0x7f176009e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 10 monclient(hunting): picked mon.noname-g con 0x7f17600678d0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:26.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f176006ea30 0x7f176009ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 10 monclient(hunting): picked mon.noname-c con 0x7f176006ea30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176009f180 0x7f17601306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 10 monclient(hunting): picked mon.noname-e con 0x7f176009f180 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176009f570 con 0x7f17600678d0 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f176006d5b0 con 0x7f176006ea30 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1760067ca0 con 0x7f176009f180 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f176571d640 10 monclient(hunting): authenticate will time out at 991.635238s 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176009f180 0x7f17601306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175e7fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f176006ea30 0x7f176009ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176009f180 0x7f17601306b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:56088/0 (socket says 172.21.15.178:56088) 2024-09-15T01:36:26.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 1 -- 172.21.15.178:0/2762979819 learned_addr learned my addr 172.21.15.178:0/2762979819 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 10 monclient(hunting): get_auth_request con 0x7f176009f180 auth_method 0 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175e7fc640 10 monclient(hunting): get_auth_request con 0x7f176006ea30 auth_method 0 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175effd640 1 --2- 172.21.15.178:0/2762979819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17600678d0 0x7f176009e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175effd640 10 monclient(hunting): get_auth_request con 0x7f17600678d0 auth_method 0 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 10 monclient(hunting): handle_auth_done global_id 5921 payload 274 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 1 -- 172.21.15.178:0/2762979819 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f176006ea30 msgr2=0x7f176009ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 1 --2- 172.21.15.178:0/2762979819 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f176006ea30 0x7f176009ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 1 -- 172.21.15.178:0/2762979819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17600678d0 msgr2=0x7f176009e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 1 --2- 172.21.15.178:0/2762979819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17600678d0 0x7f176009e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 1 monclient: found mon.noname-e 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175f7fe640 1 -- 172.21.15.178:0/2762979819 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176004bf60 con 0x7f176009f180 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.884+0000 7f175e7fc640 1 --2- 172.21.15.178:0/2762979819 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f176006ea30 0x7f176009ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175f7fe640 1 --2- 172.21.15.178:0/2762979819 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176009f180 0x7f17601306b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1750009940 tx=0x7f17500091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=4451580f5f946e02 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175dffb640 1 -- 172.21.15.178:0/2762979819 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f175002ad00 con 0x7f176009f180 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175dffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175dffb640 10 monclient: dump: 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.901 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175dffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175dffb640 1 -- 172.21.15.178:0/2762979819 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175002ae80 con 0x7f176009f180 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f176571d640 5 monclient: authenticate success, global_id 5921 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f176571d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f176571d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175dffb640 1 -- 172.21.15.178:0/2762979819 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1750031ca0 con 0x7f176009f180 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f176571d640 4 set_mon_vals no callback set 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.885+0000 7f175dffb640 10 monclient: dump: 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 10 monclient: shutdown 2024-09-15T01:36:26.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 1 -- 172.21.15.178:0/2762979819 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176009f180 msgr2=0x7f17601306b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 1 --2- 172.21.15.178:0/2762979819 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176009f180 0x7f17601306b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1750009940 tx=0x7f17500091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 1 -- 172.21.15.178:0/2762979819 shutdown_connections 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 1 --2- 172.21.15.178:0/2762979819 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176009f180 0x7f17601306b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 1 --2- 172.21.15.178:0/2762979819 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f176006ea30 0x7f176009ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 1 --2- 172.21.15.178:0/2762979819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17600678d0 0x7f176009e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 1 -- 172.21.15.178:0/2762979819 >> 172.21.15.178:0/2762979819 conn(0x7f17600a4390 msgr2=0x7f1760067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 1 -- 172.21.15.178:0/2762979819 shutdown_connections 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 1 -- 172.21.15.178:0/2762979819 wait complete. 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 10 monclient: build_initial_monmap 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.886+0000 7f176571d640 10 monclient: monmap: 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:26.905 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 1 Processor -- start 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 1 -- start start 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 10 monclient: init 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 10 monclient: _add_conns ranks=[5,8,2,6,3,4,0,1,7] 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f17600678d0 0x7f176013e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 10 monclient(hunting): picked mon.noname-h con 0x7f17600678d0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176006ea30 0x7f176013d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 10 monclient(hunting): picked mon.noname-e con 0x7f176006ea30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f176009f180 0x7f1760141750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 10 monclient(hunting): picked mon.noname-b con 0x7f176009f180 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176006c900 con 0x7f176009f180 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1760069e00 con 0x7f17600678d0 2024-09-15T01:36:26.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1760122950 con 0x7f176006ea30 2024-09-15T01:36:26.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:26.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f176571d640 10 monclient(hunting): authenticate will time out at 991.638622s 2024-09-15T01:36:26.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176006ea30 0x7f176013d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176006ea30 0x7f176013d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:56090/0 (socket says 172.21.15.178:56090) 2024-09-15T01:36:26.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175f7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f176009f180 0x7f1760141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 1 -- 172.21.15.178:0/42498190 learned_addr learned my addr 172.21.15.178:0/42498190 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:26.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 10 monclient(hunting): get_auth_request con 0x7f176006ea30 auth_method 0 2024-09-15T01:36:26.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175f7fe640 10 monclient(hunting): get_auth_request con 0x7f176009f180 auth_method 0 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175effd640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f17600678d0 0x7f176013e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175effd640 10 monclient(hunting): get_auth_request con 0x7f17600678d0 auth_method 0 2024-09-15T01:36:26.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 10 monclient(hunting): handle_auth_done global_id 5930 payload 824 2024-09-15T01:36:26.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 1 -- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f17600678d0 msgr2=0x7f176013e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.887+0000 7f175e7fc640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f17600678d0 0x7f176013e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f175e7fc640 1 -- 172.21.15.178:0/42498190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f176009f180 msgr2=0x7f1760141750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f175e7fc640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f176009f180 0x7f1760141750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f175e7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:36:26.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f175e7fc640 1 monclient: found mon.noname-e 2024-09-15T01:36:26.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f175e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:26.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f175e7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:26.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f175e7fc640 1 -- 172.21.15.178:0/42498190 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1748009390 con 0x7f176006ea30 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f175f7fe640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f176009f180 0x7f1760141750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f175e7fc640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176006ea30 0x7f176013d8b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f174800ecb0 tx=0x7f17480090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1748013470 con 0x7f176006ea30 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f1756ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f1756ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f1756ffd640 10 monclient: dump: 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.911 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f1756ffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f1756ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f176571d640 5 monclient: authenticate success, global_id 5930 2024-09-15T01:36:26.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17480135f0 con 0x7f176006ea30 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f176571d640 10 monclient: _renew_subs 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f176571d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f176571d640 1 -- 172.21.15.178:0/42498190 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1750009450 con 0x7f176006ea30 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f1756ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1748013be0 con 0x7f176006ea30 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f1756ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f1756ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f175d7fa640 4 set_mon_vals no callback set 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f1756ffd640 10 monclient: dump: 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.913 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f176571d640 10 monclient: _renew_subs 2024-09-15T01:36:26.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f176571d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:26.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.888+0000 7f176571d640 1 -- 172.21.15.178:0/42498190 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176013ed10 con 0x7f176006ea30 2024-09-15T01:36:26.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1748013d40 con 0x7f176006ea30 2024-09-15T01:36:26.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f176571d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f176571d640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760121a30 0x7f1760141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f176571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f176571d640 1 -- 172.21.15.178:0/42498190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1760122dd0 con 0x7f1760121a30 2024-09-15T01:36:26.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f1756ffd640 1 --2- 172.21.15.178:0/42498190 >> v2:172.21.15.204:6812/28210 conn(0x7f17400397b0 0x7f174003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f174804c2c0 con 0x7f176006ea30 2024-09-15T01:36:26.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f175effd640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760121a30 0x7f1760141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f175effd640 10 monclient: get_auth_request con 0x7f1760121a30 auth_method 0 2024-09-15T01:36:26.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f175effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f175effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f175effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f175f7fe640 1 --2- 172.21.15.178:0/42498190 >> v2:172.21.15.204:6812/28210 conn(0x7f17400397b0 0x7f174003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f175f7fe640 10 monclient: get_auth_request con 0x7f17400397b0 auth_method 0 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f175effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f175effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f175effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f175effd640 10 monclient(hunting): handle_auth_done global_id 6598 payload 824 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.889+0000 7f175effd640 1 -- 172.21.15.178:0/42498190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f176004bc80 con 0x7f1760121a30 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f175effd640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760121a30 0x7f1760141c90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1758011430 tx=0x7f175800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f175f7fe640 1 --2- 172.21.15.178:0/42498190 >> v2:172.21.15.204:6812/28210 conn(0x7f17400397b0 0x7f174003bc60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1750034000 tx=0x7f1750032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f175800e070 con 0x7f1760121a30 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f1756ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f1756ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:26.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f1756ffd640 10 monclient: dump: 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.917 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.918 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.918 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.918 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f17580155e0 con 0x7f1760121a30 2024-09-15T01:36:26.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f1756ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:26.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f1756ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:26.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760121a30 msgr2=0x7f1760141c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f1756ffd640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760121a30 0x7f1760141c90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1758011430 tx=0x7f175800b040 comp rx=0 tx=0).stop 2024-09-15T01:36:26.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760121a30 msgr2=0x7f1760141c90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:26.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.890+0000 7f1756ffd640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760121a30 0x7f1760141c90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f176571d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:26.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f176571d640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760069530 0x7f17601421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:26.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f176571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:26.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f176571d640 1 -- 172.21.15.178:0/42498190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176005ed70 con 0x7f1760069530 2024-09-15T01:36:26.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f175effd640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760069530 0x7f17601421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f175effd640 10 monclient: get_auth_request con 0x7f1760069530 auth_method 0 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f175effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f175effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f175effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f175effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f175effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f175effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f175effd640 10 monclient(hunting): handle_auth_done global_id 6607 payload 824 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f175effd640 1 -- 172.21.15.178:0/42498190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1760122f60 con 0x7f1760069530 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.896+0000 7f175effd640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760069530 0x7f17601421d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f175801a000 tx=0x7f175801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1758027070 con 0x7f1760069530 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f1756ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:26.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f1756ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f1756ffd640 10 monclient: dump: 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:26.922 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:26.923 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:26.923 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:26.923 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:26.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9481 (secure 0 0 0) 0x7f1758023690 con 0x7f1760069530 2024-09-15T01:36:26.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f1756ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:26.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f1756ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:26.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760069530 msgr2=0x7f17601421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f1756ffd640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760069530 0x7f17601421d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f175801a000 tx=0x7f175801b000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f1756ffd640 1 -- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760069530 msgr2=0x7f17601421d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:26.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f1756ffd640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1760069530 0x7f17601421d0 secure :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f175801a000 tx=0x7f175801b000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.923 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.925 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.926 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:26.927 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:26.928 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:26.929 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:26.929 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:26.929 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.932 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:26.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.932 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:26.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.932 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:26.932 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:26.932 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.932 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.932 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.932 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:26.932 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.934 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.936 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:26.938 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:26.939 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.941 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:26.942 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.943 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:26.944 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.945 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:26.946 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:26.947 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.948 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:26.949 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:26.950 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:26.950 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:26.950 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:26.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f176571d640 1 -- 172.21.15.178:0/42498190 >> v2:172.21.15.204:6812/28210 conn(0x7f17400397b0 msgr2=0x7f174003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f176571d640 1 --2- 172.21.15.178:0/42498190 >> v2:172.21.15.204:6812/28210 conn(0x7f17400397b0 0x7f174003bc60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1750034000 tx=0x7f1750032000 comp rx=0 tx=0).stop 2024-09-15T01:36:26.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f176571d640 10 monclient: shutdown 2024-09-15T01:36:26.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f176571d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f176571d640 1 -- 172.21.15.178:0/42498190 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176006ea30 msgr2=0x7f176013d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f176571d640 1 --2- 172.21.15.178:0/42498190 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f176006ea30 0x7f176013d8b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f174800ecb0 tx=0x7f17480090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f175f7fe640 1 -- 172.21.15.178:0/42498190 reap_dead start 2024-09-15T01:36:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f175f7fe640 1 -- 172.21.15.178:0/42498190 reap_dead start 2024-09-15T01:36:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f176571d640 1 -- 172.21.15.178:0/42498190 shutdown_connections 2024-09-15T01:36:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f176571d640 1 -- 172.21.15.178:0/42498190 >> 172.21.15.178:0/42498190 conn(0x7f17600a4390 msgr2=0x7f176009d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f176571d640 1 -- 172.21.15.178:0/42498190 shutdown_connections 2024-09-15T01:36:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:26.897+0000 7f176571d640 1 -- 172.21.15.178:0/42498190 wait complete. 2024-09-15T01:36:26.952 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:26.953 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:36:26.953 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:36:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.043+0000 7f4bd72c0640 10 monclient: get_monmap_and_config 2024-09-15T01:36:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.043+0000 7f4bd72c0640 10 monclient: build_initial_monmap 2024-09-15T01:36:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.043+0000 7f4bd72c0640 10 monclient: monmap: 2024-09-15T01:36:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:27.044944+0000 2024-09-15T01:36:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:27.044944+0000 2024-09-15T01:36:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.043+0000 7f4bd72c0640 1 Processor -- start 2024-09-15T01:36:27.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.043+0000 7f4bd72c0640 1 -- start start 2024-09-15T01:36:27.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.043+0000 7f4bd72c0640 10 monclient: init 2024-09-15T01:36:27.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:27.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 10 monclient: _add_conns ranks=[9,1,8,6,5,3,7,0,10,4,2] 2024-09-15T01:36:27.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4bd00678d0 0x7f4bd009e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 10 monclient(hunting): picked mon.noname-a con 0x7f4bd00678d0 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:27.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4bd006ea30 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd01359e0 0x7f4bd0137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 10 monclient(hunting): picked mon.noname-e con 0x7f4bd01359e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd004bf60 con 0x7f4bd006ea30 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd006d5b0 con 0x7f4bd00678d0 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0067ca0 con 0x7f4bd01359e0 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd72c0640 10 monclient(hunting): authenticate will time out at 991.795315s 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd01359e0 0x7f4bd0137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd505c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4bd00678d0 0x7f4bd009e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd01359e0 0x7f4bd0137dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:56096/0 (socket says 172.21.15.178:56096) 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 1 -- 172.21.15.178:0/2768369259 learned_addr learned my addr 172.21.15.178:0/2768369259 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:27.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 10 monclient(hunting): get_auth_request con 0x7f4bd01359e0 auth_method 0 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd505c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4bd00678d0 0x7f4bd009e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60946/0 (socket says 172.21.15.178:60946) 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd505c640 10 monclient(hunting): get_auth_request con 0x7f4bd00678d0 auth_method 0 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd505c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd505c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd505c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd505c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd505c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd505c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 10 monclient(hunting): handle_auth_done global_id 5939 payload 274 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 1 -- 172.21.15.178:0/2768369259 >> v2:172.21.15.178:3300/0 conn(0x7f4bd00678d0 msgr2=0x7f4bd009e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 1 --2- 172.21.15.178:0/2768369259 >> v2:172.21.15.178:3300/0 conn(0x7f4bd00678d0 0x7f4bd009e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 1 -- 172.21.15.178:0/2768369259 >> v1:172.21.15.204:6789/0 conn(0x7f4bd006ea30 legacy=0x7f4bd009ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 10 monclient: _finish_hunting 0 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 1 monclient: found mon.noname-e 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd585d640 1 -- 172.21.15.178:0/2768369259 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd006d890 con 0x7f4bd01359e0 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.044+0000 7f4bd505c640 1 --2- 172.21.15.178:0/2768369259 >> v2:172.21.15.178:3300/0 conn(0x7f4bd00678d0 0x7f4bd009e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bd585d640 1 --2- 172.21.15.178:0/2768369259 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd01359e0 0x7f4bd0137dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009f20 tx=0x7f4bc0009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=2ab90ce10b189f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bcffff640 1 -- 172.21.15.178:0/2768369259 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4bc0007880 con 0x7f4bd01359e0 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bcffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bcffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bcffff640 10 monclient: dump: 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.060 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bcffff640 10 monclient: _finish_auth 0 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bcffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bcffff640 1 -- 172.21.15.178:0/2768369259 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc002d050 con 0x7f4bd01359e0 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bcffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bcffff640 1 -- 172.21.15.178:0/2768369259 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4bc0030360 con 0x7f4bd01359e0 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bd72c0640 5 monclient: authenticate success, global_id 5939 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bd72c0640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:27.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bd72c0640 4 set_mon_vals no callback set 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bcffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bcffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.045+0000 7f4bcffff640 10 monclient: dump: 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.062 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.046+0000 7f4bd72c0640 10 monclient: shutdown 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.046+0000 7f4bd72c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.046+0000 7f4bd72c0640 1 -- 172.21.15.178:0/2768369259 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd01359e0 msgr2=0x7f4bd0137dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.046+0000 7f4bd72c0640 1 --2- 172.21.15.178:0/2768369259 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd01359e0 0x7f4bd0137dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009f20 tx=0x7f4bc0009840 comp rx=0 tx=0).stop 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.046+0000 7f4bd72c0640 1 -- 172.21.15.178:0/2768369259 shutdown_connections 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.046+0000 7f4bd72c0640 1 --2- 172.21.15.178:0/2768369259 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd01359e0 0x7f4bd0137dc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.046+0000 7f4bd72c0640 1 --2- 172.21.15.178:0/2768369259 >> v2:172.21.15.178:3300/0 conn(0x7f4bd00678d0 0x7f4bd009e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.046+0000 7f4bd72c0640 1 -- 172.21.15.178:0/2768369259 >> 172.21.15.178:0/2768369259 conn(0x7f4bd00a4390 msgr2=0x7f4bd0067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.046+0000 7f4bd72c0640 1 -- 172.21.15.178:0/2768369259 shutdown_connections 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.046+0000 7f4bd72c0640 1 -- 172.21.15.178:0/2768369259 wait complete. 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.046+0000 7f4bd72c0640 10 monclient: build_initial_monmap 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.046+0000 7f4bd72c0640 10 monclient: monmap: 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:27.064 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 1 Processor -- start 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 1 -- start start 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 10 monclient: init 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 10 monclient: _add_conns ranks=[2,8,5,4,6,3,0,1,7] 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4bd00678d0 0x7f4bd013fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 10 monclient(hunting): picked mon.noname-b con 0x7f4bd00678d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd006ea30 0x7f4bd0140390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 10 monclient(hunting): picked mon.noname-e con 0x7f4bd006ea30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4bd01359e0 0x7f4bd01408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 10 monclient(hunting): picked mon.noname-h con 0x7f4bd01359e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0120560 con 0x7f4bd00678d0 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd006ccd0 con 0x7f4bd01359e0 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd006c900 con 0x7f4bd006ea30 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd72c0640 10 monclient(hunting): authenticate will time out at 991.798857s 2024-09-15T01:36:27.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd485b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd006ea30 0x7f4bd0140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd505c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4bd00678d0 0x7f4bd013fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd505c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4bd00678d0 0x7f4bd013fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:38044/0 (socket says 172.21.15.178:38044) 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd485b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd006ea30 0x7f4bd0140390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:56106/0 (socket says 172.21.15.178:56106) 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd505c640 1 -- 172.21.15.178:0/741885096 learned_addr learned my addr 172.21.15.178:0/741885096 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd505c640 10 monclient(hunting): get_auth_request con 0x7f4bd00678d0 auth_method 0 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd505c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd505c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd505c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd485b640 10 monclient(hunting): get_auth_request con 0x7f4bd006ea30 auth_method 0 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd485b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd485b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.047+0000 7f4bd485b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd505c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd505c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd505c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd585d640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4bd01359e0 0x7f4bd01408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd585d640 10 monclient(hunting): get_auth_request con 0x7f4bd01359e0 auth_method 0 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd585d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd585d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd585d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd585d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd585d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd585d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 10 monclient(hunting): handle_auth_done global_id 5948 payload 824 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 1 -- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4bd01359e0 msgr2=0x7f4bd01408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4bd01359e0 0x7f4bd01408d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 1 -- 172.21.15.178:0/741885096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4bd00678d0 msgr2=0x7f4bd013fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4bd00678d0 0x7f4bd013fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 10 monclient: _finish_hunting 0 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 1 monclient: found mon.noname-e 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 1 -- 172.21.15.178:0/741885096 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8009390 con 0x7f4bd006ea30 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd505c640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4bd00678d0 0x7f4bd013fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd485b640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd006ea30 0x7f4bd0140390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400c4e0 tx=0x7f4bc4007e10 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4bc4011a10 con 0x7f4bd006ea30 2024-09-15T01:36:27.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcd7fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcd7fa640 10 monclient: dump: 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.070 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcd7fa640 10 monclient: _finish_auth 0 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc4011bb0 con 0x7f4bd006ea30 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd72c0640 5 monclient: authenticate success, global_id 5948 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd72c0640 10 monclient: _renew_subs 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd72c0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd72c0640 1 -- 172.21.15.178:0/741885096 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc0009ae0 con 0x7f4bd006ea30 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcd7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:27.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4bc4011d60 con 0x7f4bd006ea30 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcd7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcd7fa640 10 monclient: dump: 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.072 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd72c0640 10 monclient: _renew_subs 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd72c0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bcf7fe640 4 set_mon_vals no callback set 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.048+0000 7f4bd72c0640 1 -- 172.21.15.178:0/741885096 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd011f290 con 0x7f4bd006ea30 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd72c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4bc4011590 con 0x7f4bd006ea30 2024-09-15T01:36:27.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd72c0640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd0134720 0x7f4bd0134af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd72c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd72c0640 1 -- 172.21.15.178:0/741885096 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd004c4a0 con 0x7f4bd0134720 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bcd7fa640 1 --2- 172.21.15.178:0/741885096 >> v2:172.21.15.204:6812/28210 conn(0x7f4bbc0397b0 0x7f4bbc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f4bc404b6d0 con 0x7f4bd006ea30 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd505c640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd0134720 0x7f4bd0134af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd505c640 10 monclient: get_auth_request con 0x7f4bd0134720 auth_method 0 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd505c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd505c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd505c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd585d640 1 --2- 172.21.15.178:0/741885096 >> v2:172.21.15.204:6812/28210 conn(0x7f4bbc0397b0 0x7f4bbc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd585d640 10 monclient: get_auth_request con 0x7f4bbc0397b0 auth_method 0 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd505c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd505c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.049+0000 7f4bd505c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bd505c640 10 monclient(hunting): handle_auth_done global_id 15051 payload 824 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bd505c640 1 -- 172.21.15.178:0/741885096 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4bd004bc80 con 0x7f4bd0134720 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bd585d640 1 --2- 172.21.15.178:0/741885096 >> v2:172.21.15.204:6812/28210 conn(0x7f4bbc0397b0 0x7f4bbc03bc60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0033000 tx=0x7f4bc0036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bd505c640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd0134720 0x7f4bd0134af0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800f4a0 tx=0x7f4bc80090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4bc8013bd0 con 0x7f4bd0134720 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bcd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bcd7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bcd7fa640 10 monclient: dump: 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.075 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f4bc8013d30 con 0x7f4bd0134720 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bcd7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:27.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bcd7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:27.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd0134720 msgr2=0x7f4bd0134af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bcd7fa640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd0134720 0x7f4bd0134af0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800f4a0 tx=0x7f4bc80090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd0134720 msgr2=0x7f4bd0134af0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:27.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.050+0000 7f4bcd7fa640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd0134720 0x7f4bd0134af0 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800f4a0 tx=0x7f4bc80090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:27.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd011f5d0 0x7f4bd0135030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 1 -- 172.21.15.178:0/741885096 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd005ed70 con 0x7f4bd011f5d0 2024-09-15T01:36:27.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd505c640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd011f5d0 0x7f4bd0135030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd505c640 10 monclient: get_auth_request con 0x7f4bd011f5d0 auth_method 0 2024-09-15T01:36:27.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd505c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd505c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd505c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd505c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd505c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd505c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd505c640 10 monclient(hunting): handle_auth_done global_id 15060 payload 824 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd505c640 1 -- 172.21.15.178:0/741885096 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4bd004b720 con 0x7f4bd011f5d0 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd505c640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd011f5d0 0x7f4bd0135030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800f4a0 tx=0x7f4bc801c000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4bc8024040 con 0x7f4bd011f5d0 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bcd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bcd7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bcd7fa640 10 monclient: dump: 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.080 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f4bc800fed0 con 0x7f4bd011f5d0 2024-09-15T01:36:27.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bcd7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:27.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bcd7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:27.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd011f5d0 msgr2=0x7f4bd0135030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bcd7fa640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd011f5d0 0x7f4bd0135030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800f4a0 tx=0x7f4bc801c000 comp rx=0 tx=0).stop 2024-09-15T01:36:27.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bcd7fa640 1 -- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd011f5d0 msgr2=0x7f4bd0135030 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:27.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bcd7fa640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4bd011f5d0 0x7f4bd0135030 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.082 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:27.082 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:27.082 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:27.082 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:36:27.083 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:27.084 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:27.086 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.087 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:27.089 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:27.091 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.092 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:27.094 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.097 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.098 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:27.099 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.101 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.102 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.103 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:27.104 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:27.104 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:27.104 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.104 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.104 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:27.104 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:27.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 1 -- 172.21.15.178:0/741885096 >> v2:172.21.15.204:6812/28210 conn(0x7f4bbc0397b0 msgr2=0x7f4bbc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 1 --2- 172.21.15.178:0/741885096 >> v2:172.21.15.204:6812/28210 conn(0x7f4bbc0397b0 0x7f4bbc03bc60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0033000 tx=0x7f4bc0036000 comp rx=0 tx=0).stop 2024-09-15T01:36:27.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 10 monclient: shutdown 2024-09-15T01:36:27.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:27.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 1 -- 172.21.15.178:0/741885096 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd006ea30 msgr2=0x7f4bd0140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 1 --2- 172.21.15.178:0/741885096 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4bd006ea30 0x7f4bd0140390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400c4e0 tx=0x7f4bc4007e10 comp rx=0 tx=0).stop 2024-09-15T01:36:27.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd585d640 1 -- 172.21.15.178:0/741885096 reap_dead start 2024-09-15T01:36:27.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd585d640 1 -- 172.21.15.178:0/741885096 reap_dead start 2024-09-15T01:36:27.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 1 -- 172.21.15.178:0/741885096 shutdown_connections 2024-09-15T01:36:27.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 1 -- 172.21.15.178:0/741885096 >> 172.21.15.178:0/741885096 conn(0x7f4bd00a4390 msgr2=0x7f4bd00a23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:27.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 1 -- 172.21.15.178:0/741885096 shutdown_connections 2024-09-15T01:36:27.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.057+0000 7f4bd72c0640 1 -- 172.21.15.178:0/741885096 wait complete. 2024-09-15T01:36:27.106 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:27.107 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:36:27.107 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:36:27.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.199+0000 7efdd1909640 10 monclient: get_monmap_and_config 2024-09-15T01:36:27.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.199+0000 7efdd1909640 10 monclient: build_initial_monmap 2024-09-15T01:36:27.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.199+0000 7efdd1909640 10 monclient: monmap: 2024-09-15T01:36:27.202 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:27.202 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.202 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:27.201251+0000 2024-09-15T01:36:27.202 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:27.201251+0000 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:27.203 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.199+0000 7efdd1909640 1 Processor -- start 2024-09-15T01:36:27.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.199+0000 7efdd1909640 1 -- start start 2024-09-15T01:36:27.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.199+0000 7efdd1909640 10 monclient: init 2024-09-15T01:36:27.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:27.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 10 monclient: _add_conns ranks=[9,8,3,2,5,6,0,7,4,1,10] 2024-09-15T01:36:27.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efdcc0a2930 0x7efdcc132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 10 monclient(hunting): picked mon.noname-a con 0x7efdcc0a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdcc09faa0 0x7efdcc09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 10 monclient(hunting): picked mon.noname-e con 0x7efdcc09faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdcc0a08c0 0x7efdcc0a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 10 monclient(hunting): picked mon.noname-g con 0x7efdcc0a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc09e0f0 con 0x7efdcc0a2930 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc09dd80 con 0x7efdcc0a08c0 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc0a2d00 con 0x7efdcc09faa0 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdd1909640 10 monclient(hunting): authenticate will time out at 991.951460s 2024-09-15T01:36:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdcc09faa0 0x7efdcc09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcaffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efdcc0a2930 0x7efdcc132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdcc09faa0 0x7efdcc09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:56112/0 (socket says 172.21.15.178:56112) 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 1 -- 172.21.15.178:0/466150995 learned_addr learned my addr 172.21.15.178:0/466150995 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 10 monclient(hunting): get_auth_request con 0x7efdcc09faa0 auth_method 0 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcaffd640 10 monclient(hunting): get_auth_request con 0x7efdcc0a2930 auth_method 0 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcb7fe640 1 --2- 172.21.15.178:0/466150995 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdcc0a08c0 0x7efdcc0a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcb7fe640 10 monclient(hunting): get_auth_request con 0x7efdcc0a08c0 auth_method 0 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 10 monclient(hunting): handle_auth_done global_id 5957 payload 274 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 1 -- 172.21.15.178:0/466150995 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdcc0a08c0 msgr2=0x7efdcc0a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 1 --2- 172.21.15.178:0/466150995 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdcc0a08c0 0x7efdcc0a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 1 -- 172.21.15.178:0/466150995 >> v2:172.21.15.178:3300/0 conn(0x7efdcc0a2930 msgr2=0x7efdcc132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 1 --2- 172.21.15.178:0/466150995 >> v2:172.21.15.178:3300/0 conn(0x7efdcc0a2930 0x7efdcc132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 1 monclient: found mon.noname-e 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdca7fc640 1 -- 172.21.15.178:0/466150995 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc04bf60 con 0x7efdcc09faa0 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.200+0000 7efdcaffd640 1 --2- 172.21.15.178:0/466150995 >> v2:172.21.15.178:3300/0 conn(0x7efdcc0a2930 0x7efdcc132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdca7fc640 1 --2- 172.21.15.178:0/466150995 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdcc09faa0 0x7efdcc09fe70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009910 tx=0x7efdc00091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=27355e4024a8aa04 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdc9ffb640 1 -- 172.21.15.178:0/466150995 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efdc002ac80 con 0x7efdcc09faa0 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdc9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdc9ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdc9ffb640 10 monclient: dump: 2024-09-15T01:36:27.216 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.217 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdc9ffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdc9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdd1909640 5 monclient: authenticate success, global_id 5957 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdd1909640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdc9ffb640 1 -- 172.21.15.178:0/466150995 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc002ae00 con 0x7efdcc09faa0 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdc9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdd1909640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdc9ffb640 1 -- 172.21.15.178:0/466150995 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efdc0031ca0 con 0x7efdcc09faa0 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdd1909640 4 set_mon_vals no callback set 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdc9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdc9ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.201+0000 7efdc9ffb640 10 monclient: dump: 2024-09-15T01:36:27.218 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.202+0000 7efdd1909640 10 monclient: shutdown 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.202+0000 7efdd1909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.202+0000 7efdd1909640 1 -- 172.21.15.178:0/466150995 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdcc09faa0 msgr2=0x7efdcc09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.202+0000 7efdd1909640 1 --2- 172.21.15.178:0/466150995 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdcc09faa0 0x7efdcc09fe70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009910 tx=0x7efdc00091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.202+0000 7efdd1909640 1 -- 172.21.15.178:0/466150995 shutdown_connections 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.202+0000 7efdd1909640 1 --2- 172.21.15.178:0/466150995 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdcc0a08c0 0x7efdcc0a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.202+0000 7efdd1909640 1 --2- 172.21.15.178:0/466150995 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efdcc09faa0 0x7efdcc09fe70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.202+0000 7efdd1909640 1 --2- 172.21.15.178:0/466150995 >> v2:172.21.15.178:3300/0 conn(0x7efdcc0a2930 0x7efdcc132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.202+0000 7efdd1909640 1 -- 172.21.15.178:0/466150995 >> 172.21.15.178:0/466150995 conn(0x7efdcc0a5ed0 msgr2=0x7efdcc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.202+0000 7efdd1909640 1 -- 172.21.15.178:0/466150995 shutdown_connections 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.202+0000 7efdd1909640 1 -- 172.21.15.178:0/466150995 wait complete. 2024-09-15T01:36:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient: build_initial_monmap 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient: monmap: 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 1 Processor -- start 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 1 -- start start 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient: init 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient: _add_conns ranks=[4,1,3,6,7,5,8,0,2] 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efdcc09faa0 0x7efdcc09b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient(hunting): picked mon.noname-c con 0x7efdcc09faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdcc0a08c0 0x7efdcc131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient(hunting): picked mon.noname-f con 0x7efdcc0a08c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdcc0a2930 0x7efdcc11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient(hunting): picked mon.noname-g con 0x7efdcc0a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc09d630 con 0x7efdcc0a08c0 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc11ec40 con 0x7efdcc0a2930 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc1429e0 con 0x7efdcc09faa0 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.203+0000 7efdd1909640 10 monclient(hunting): authenticate will time out at 991.954989s 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efdcc09faa0 0x7efdcc09b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efdcc09faa0 0x7efdcc09b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44174/0 (socket says 172.21.15.178:44174) 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 1 -- 172.21.15.178:0/456248152 learned_addr learned my addr 172.21.15.178:0/456248152 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 10 monclient(hunting): get_auth_request con 0x7efdcc09faa0 auth_method 0 2024-09-15T01:36:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdca7fc640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdcc0a08c0 0x7efdcc131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdca7fc640 10 monclient(hunting): get_auth_request con 0x7efdcc0a08c0 auth_method 0 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdca7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcb7fe640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdcc0a2930 0x7efdcc11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcb7fe640 10 monclient(hunting): get_auth_request con 0x7efdcc0a2930 auth_method 0 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdca7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdca7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdca7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 10 monclient(hunting): handle_auth_done global_id 5899 payload 824 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 1 -- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdcc0a2930 msgr2=0x7efdcc11c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdcc0a2930 0x7efdcc11c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 1 -- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdcc0a08c0 msgr2=0x7efdcc131840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdcc0a08c0 0x7efdcc131840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 10 monclient: _finish_hunting 0 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 1 monclient: found mon.noname-c 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 1 -- 172.21.15.178:0/456248152 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc4009390 con 0x7efdcc09faa0 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdcaffd640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efdcc09faa0 0x7efdcc09b090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efdc400ecb0 tx=0x7efdc40090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efdc4013470 con 0x7efdcc09faa0 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdb2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdb2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdb2ffd640 10 monclient: dump: 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdb2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdb2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc40135d0 con 0x7efdcc09faa0 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdd1909640 5 monclient: authenticate success, global_id 5899 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdd1909640 10 monclient: _renew_subs 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdd1909640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdd1909640 1 -- 172.21.15.178:0/456248152 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc0009450 con 0x7efdcc09faa0 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdb2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efdc4013bc0 con 0x7efdcc09faa0 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdb2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdb2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdb2ffd640 10 monclient: dump: 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdc97fa640 4 set_mon_vals no callback set 2024-09-15T01:36:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdd1909640 10 monclient: _renew_subs 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdd1909640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.204+0000 7efdd1909640 1 -- 172.21.15.178:0/456248152 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdcc11da70 con 0x7efdcc09faa0 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdd1909640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdd1909640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc131d80 0x7efdcc11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdd1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdd1909640 1 -- 172.21.15.178:0/456248152 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc04c4a0 con 0x7efdcc131d80 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7efdc4013d20 con 0x7efdcc09faa0 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdca7fc640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc131d80 0x7efdcc11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdca7fc640 10 monclient: get_auth_request con 0x7efdcc131d80 auth_method 0 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdca7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdca7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdca7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdca7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdb2ffd640 1 --2- 172.21.15.178:0/456248152 >> v2:172.21.15.204:6812/28210 conn(0x7efdb80397b0 0x7efdb803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.205+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6743+0+0 (secure 0 0 0) 0x7efdc404c2f0 con 0x7efdcc09faa0 2024-09-15T01:36:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdcb7fe640 1 --2- 172.21.15.178:0/456248152 >> v2:172.21.15.204:6812/28210 conn(0x7efdb80397b0 0x7efdb803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdca7fc640 10 monclient(hunting): handle_auth_done global_id 5954 payload 824 2024-09-15T01:36:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdca7fc640 1 -- 172.21.15.178:0/456248152 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efdcc05ed70 con 0x7efdcc131d80 2024-09-15T01:36:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdcb7fe640 10 monclient: get_auth_request con 0x7efdb80397b0 auth_method 0 2024-09-15T01:36:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdca7fc640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc131d80 0x7efdcc11ca70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0a2f70 tx=0x7efdc0032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efdc002ac80 con 0x7efdcc131d80 2024-09-15T01:36:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdb2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdcb7fe640 1 --2- 172.21.15.178:0/456248152 >> v2:172.21.15.204:6812/28210 conn(0x7efdb80397b0 0x7efdb803bc60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efdcc09ba70 tx=0x7efdbc009850 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdb2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdb2ffd640 10 monclient: dump: 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.233 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7efdc003c3e0 con 0x7efdcc131d80 2024-09-15T01:36:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdb2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdb2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc131d80 msgr2=0x7efdcc11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdb2ffd640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc131d80 0x7efdcc11ca70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0a2f70 tx=0x7efdc0032000 comp rx=0 tx=0).stop 2024-09-15T01:36:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc131d80 msgr2=0x7efdcc11ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.206+0000 7efdb2ffd640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc131d80 0x7efdcc11ca70 secure :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0a2f70 tx=0x7efdc0032000 comp rx=0 tx=0).stop 2024-09-15T01:36:27.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:27.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc11cfb0 0x7efdcc096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 1 -- 172.21.15.178:0/456248152 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc04bc80 con 0x7efdcc11cfb0 2024-09-15T01:36:27.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdca7fc640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc11cfb0 0x7efdcc096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdca7fc640 10 monclient: get_auth_request con 0x7efdcc11cfb0 auth_method 0 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdca7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdca7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdca7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdca7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdca7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdca7fc640 10 monclient(hunting): handle_auth_done global_id 5963 payload 824 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdca7fc640 1 -- 172.21.15.178:0/456248152 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efdcc11d3b0 con 0x7efdcc11cfb0 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdca7fc640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc11cfb0 0x7efdcc096a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009910 tx=0x7efdc003d000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efdc003cd80 con 0x7efdcc11cfb0 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdb2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdb2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdb2ffd640 10 monclient: dump: 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.238 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9313 (secure 0 0 0) 0x7efdc0044390 con 0x7efdcc11cfb0 2024-09-15T01:36:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdb2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdb2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc11cfb0 msgr2=0x7efdcc096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdb2ffd640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc11cfb0 0x7efdcc096a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009910 tx=0x7efdc003d000 comp rx=0 tx=0).stop 2024-09-15T01:36:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdb2ffd640 1 -- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc11cfb0 msgr2=0x7efdcc096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdb2ffd640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efdcc11cfb0 0x7efdcc096a30 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009910 tx=0x7efdc003d000 comp rx=0 tx=0).stop 2024-09-15T01:36:27.238 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:27.239 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:27.239 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:27.239 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:36:27.239 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:27.239 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:27.239 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:27.239 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:27.239 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:27.239 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:27.239 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:27.239 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:27.239 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:27.240 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:27.241 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:27.242 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:27.243 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.244 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.245 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.247 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.248 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.250 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:27.253 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:27.254 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.256 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:27.257 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.260 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:27.260 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:27.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.260 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:27.260 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:27.260 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:27.260 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.260 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.260 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:36:27.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 4 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.261 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:27.262 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:27.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 1 -- 172.21.15.178:0/456248152 >> v2:172.21.15.204:6812/28210 conn(0x7efdb80397b0 msgr2=0x7efdb803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 1 --2- 172.21.15.178:0/456248152 >> v2:172.21.15.204:6812/28210 conn(0x7efdb80397b0 0x7efdb803bc60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efdcc09ba70 tx=0x7efdbc009850 comp rx=0 tx=0).stop 2024-09-15T01:36:27.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 10 monclient: shutdown 2024-09-15T01:36:27.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:27.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 1 -- 172.21.15.178:0/456248152 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efdcc09faa0 msgr2=0x7efdcc09b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 1 --2- 172.21.15.178:0/456248152 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efdcc09faa0 0x7efdcc09b090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efdc400ecb0 tx=0x7efdc40090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdcb7fe640 1 -- 172.21.15.178:0/456248152 reap_dead start 2024-09-15T01:36:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdcb7fe640 1 -- 172.21.15.178:0/456248152 reap_dead start 2024-09-15T01:36:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 1 -- 172.21.15.178:0/456248152 shutdown_connections 2024-09-15T01:36:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 1 -- 172.21.15.178:0/456248152 >> 172.21.15.178:0/456248152 conn(0x7efdcc0a5ed0 msgr2=0x7efdcc135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 1 -- 172.21.15.178:0/456248152 shutdown_connections 2024-09-15T01:36:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.213+0000 7efdd1909640 1 -- 172.21.15.178:0/456248152 wait complete. 2024-09-15T01:36:27.264 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:27.264 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:36:27.264 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:36:27.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient: get_monmap_and_config 2024-09-15T01:36:27.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient: build_initial_monmap 2024-09-15T01:36:27.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient: monmap: 2024-09-15T01:36:27.370 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:27.370 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.370 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:27.357382+0000 2024-09-15T01:36:27.370 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:27.357382+0000 2024-09-15T01:36:27.370 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:27.370 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.370 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:27.370 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:27.370 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:27.370 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:27.370 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:27.371 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:27.371 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:27.371 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:27.371 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:27.371 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:27.371 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:27.371 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 1 Processor -- start 2024-09-15T01:36:27.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 1 -- start start 2024-09-15T01:36:27.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient: init 2024-09-15T01:36:27.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:27.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient: _add_conns ranks=[5,3,0,6,4,10,9,1,7,2,8] 2024-09-15T01:36:27.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2f200a2930 0x7f2f20132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient(hunting): picked mon.noname-h con 0x7f2f200a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:27.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f2009faa0 0x7f2f2009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient(hunting): picked mon.noname-g con 0x7f2f2009faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:27.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2f200a08c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:27.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2004bf60 con 0x7f2f200a08c0 2024-09-15T01:36:27.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2009e0f0 con 0x7f2f2009faa0 2024-09-15T01:36:27.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2009dd80 con 0x7f2f200a2930 2024-09-15T01:36:27.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:27.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f25900640 10 monclient(hunting): authenticate will time out at 992.107587s 2024-09-15T01:36:27.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1f7fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2f200a08c0 0x7f2f200a0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:46454/0 (socket says 172.21.15.178:46454) 2024-09-15T01:36:27.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1f7fe640 1 -- 172.21.15.178:0/1346678439 learned_addr learned my addr 172.21.15.178:0/1346678439 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:36:27.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1e7fc640 1 --2- 172.21.15.178:0/1346678439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f2009faa0 0x7f2f2009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1effd640 1 --2- 172.21.15.178:0/1346678439 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2f200a2930 0x7f2f20132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1e7fc640 10 monclient(hunting): get_auth_request con 0x7f2f2009faa0 auth_method 0 2024-09-15T01:36:27.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1effd640 10 monclient(hunting): get_auth_request con 0x7f2f200a2930 auth_method 0 2024-09-15T01:36:27.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.356+0000 7f2f1dffb640 1 -- 172.21.15.178:0/1346678439 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887605524 0 0) 0x7f2f2004bf60 con 0x7f2f200a08c0 2024-09-15T01:36:27.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1dffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1dffb640 10 monclient(hunting): my global_id is 6210 2024-09-15T01:36:27.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1dffb640 1 -- 172.21.15.178:0/1346678439 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f080034e0 con 0x7f2f200a08c0 2024-09-15T01:36:27.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_done global_id 15087 payload 274 2024-09-15T01:36:27.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1e7fc640 1 -- 172.21.15.178:0/1346678439 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2f200a2930 msgr2=0x7f2f20132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:27.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1e7fc640 1 --2- 172.21.15.178:0/1346678439 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2f200a2930 0x7f2f20132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1e7fc640 1 -- 172.21.15.178:0/1346678439 >> v1:172.21.15.178:6789/0 conn(0x7f2f200a08c0 legacy=0x7f2f200a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1e7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:36:27.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1e7fc640 1 monclient: found mon.noname-g 2024-09-15T01:36:27.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:27.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1e7fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:27.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1e7fc640 1 -- 172.21.15.178:0/1346678439 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f200a2d00 con 0x7f2f2009faa0 2024-09-15T01:36:27.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1effd640 1 --2- 172.21.15.178:0/1346678439 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2f200a2930 0x7f2f20132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:27.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.357+0000 7f2f1e7fc640 1 --2- 172.21.15.178:0/1346678439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f2009faa0 0x7f2f2009fe70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f14009880 tx=0x7f2f140090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=5303beb569ad5cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f1dffb640 1 -- 172.21.15.178:0/1346678439 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f1402ac50 con 0x7f2f2009faa0 2024-09-15T01:36:27.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f1dffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:27.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f1dffb640 10 monclient: dump: 2024-09-15T01:36:27.376 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.376 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.376 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.377 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.377 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.377 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.377 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.377 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.377 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.377 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.377 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.377 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.377 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.377 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.378 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.378 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f1dffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:27.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:27.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f1dffb640 1 -- 172.21.15.178:0/1346678439 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1402add0 con 0x7f2f2009faa0 2024-09-15T01:36:27.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 5 monclient: authenticate success, global_id 15087 2024-09-15T01:36:27.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:27.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f1dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:27.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:27.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f1dffb640 1 -- 172.21.15.178:0/1346678439 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f14031ca0 con 0x7f2f2009faa0 2024-09-15T01:36:27.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 4 set_mon_vals no callback set 2024-09-15T01:36:27.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f1dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:27.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f1dffb640 10 monclient: dump: 2024-09-15T01:36:27.379 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.379 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.379 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.379 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.379 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.379 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.379 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.379 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.379 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.380 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.380 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.380 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.380 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.380 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.380 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.380 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 10 monclient: shutdown 2024-09-15T01:36:27.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:27.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 1 -- 172.21.15.178:0/1346678439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f2009faa0 msgr2=0x7f2f2009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 1 --2- 172.21.15.178:0/1346678439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f2009faa0 0x7f2f2009fe70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f14009880 tx=0x7f2f140090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 1 -- 172.21.15.178:0/1346678439 shutdown_connections 2024-09-15T01:36:27.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 1 --2- 172.21.15.178:0/1346678439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f2009faa0 0x7f2f2009fe70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 1 --2- 172.21.15.178:0/1346678439 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2f200a2930 0x7f2f20132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 1 -- 172.21.15.178:0/1346678439 >> 172.21.15.178:0/1346678439 conn(0x7f2f200a5ed0 msgr2=0x7f2f200a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:27.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 1 -- 172.21.15.178:0/1346678439 shutdown_connections 2024-09-15T01:36:27.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.358+0000 7f2f25900640 1 -- 172.21.15.178:0/1346678439 wait complete. 2024-09-15T01:36:27.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.359+0000 7f2f25900640 10 monclient: build_initial_monmap 2024-09-15T01:36:27.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.359+0000 7f2f25900640 10 monclient: monmap: 2024-09-15T01:36:27.381 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:27.381 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:27.381 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:27.382 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:27.382 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:27.382 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:27.382 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:27.382 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:27.382 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:27.382 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:27.382 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:27.382 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:27.382 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:27.382 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:27.383 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:27.383 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.359+0000 7f2f25900640 1 Processor -- start 2024-09-15T01:36:27.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.359+0000 7f2f25900640 1 -- start start 2024-09-15T01:36:27.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.359+0000 7f2f25900640 10 monclient: init 2024-09-15T01:36:27.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:27.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient: _add_conns ranks=[2,8,7,5,4,6,0,1,3] 2024-09-15T01:36:27.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f2009faa0 0x7f2f2009b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient(hunting): picked mon.noname-b con 0x7f2f2009faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:27.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f200a08c0 0x7f2f201318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient(hunting): picked mon.noname-e con 0x7f2f200a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:27.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f200a2930 0x7f2f2011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient(hunting): picked mon.noname-i con 0x7f2f200a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:27.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f20142af0 con 0x7f2f2009faa0 2024-09-15T01:36:27.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2f200a2390 con 0x7f2f200a2930 2024-09-15T01:36:27.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2009d630 con 0x7f2f200a08c0 2024-09-15T01:36:27.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:27.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient(hunting): authenticate will time out at 992.111122s 2024-09-15T01:36:27.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f2009faa0 0x7f2f2009b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f2009faa0 0x7f2f2009b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:38046/0 (socket says 172.21.15.178:38046) 2024-09-15T01:36:27.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1e7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f200a08c0 0x7f2f201318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 1 -- 172.21.15.178:0/462191939 learned_addr learned my addr 172.21.15.178:0/462191939 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:27.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 10 monclient(hunting): get_auth_request con 0x7f2f2009faa0 auth_method 0 2024-09-15T01:36:27.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1e7fc640 10 monclient(hunting): get_auth_request con 0x7f2f200a08c0 auth_method 0 2024-09-15T01:36:27.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1f7fe640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f200a2930 0x7f2f2011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1f7fe640 10 monclient(hunting): get_auth_request con 0x7f2f200a2930 auth_method 0 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 10 monclient(hunting): handle_auth_done global_id 24185 payload 824 2024-09-15T01:36:27.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 1 -- 172.21.15.178:0/462191939 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f200a08c0 msgr2=0x7f2f201318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f200a08c0 0x7f2f201318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 1 -- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f200a2930 msgr2=0x7f2f2011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f200a2930 0x7f2f2011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 10 monclient: _finish_hunting 0 2024-09-15T01:36:27.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 1 monclient: found mon.noname-b 2024-09-15T01:36:27.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:27.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:36:27.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 1 -- 172.21.15.178:0/462191939 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f10009390 con 0x7f2f2009faa0 2024-09-15T01:36:27.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1effd640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f2009faa0 0x7f2f2009b240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000ecb0 tx=0x7f2f100090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f10013470 con 0x7f2f2009faa0 2024-09-15T01:36:27.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f06ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f06ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:36:27.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f06ffd640 10 monclient: dump: 2024-09-15T01:36:27.388 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.388 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.388 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.388 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.388 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.388 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.389 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.389 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.389 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.389 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.389 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.389 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.389 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.389 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.389 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.389 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f06ffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 5 monclient: authenticate success, global_id 24185 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f100135f0 con 0x7f2f2009faa0 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient: _renew_subs 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 1 -- 172.21.15.178:0/462191939 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f14009390 con 0x7f2f2009faa0 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f06ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f10013be0 con 0x7f2f2009faa0 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f06ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f06ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f06ffd640 10 monclient: dump: 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.391 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.391 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.391 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.391 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.391 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.391 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.391 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.391 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.391 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.391 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.391 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.392 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.392 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.392 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient: _renew_subs 2024-09-15T01:36:27.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:27.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f25900640 1 -- 172.21.15.178:0/462191939 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2011e9d0 con 0x7f2f2009faa0 2024-09-15T01:36:27.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.360+0000 7f2f1d7fa640 4 set_mon_vals no callback set 2024-09-15T01:36:27.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2f10013d40 con 0x7f2f2009faa0 2024-09-15T01:36:27.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f25900640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:27.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f25900640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f20131df0 0x7f2f2011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f25900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f25900640 1 -- 172.21.15.178:0/462191939 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2f201321c0 con 0x7f2f20131df0 2024-09-15T01:36:27.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f06ffd640 1 --2- 172.21.15.178:0/462191939 >> v2:172.21.15.204:6812/28210 conn(0x7f2f0c0397b0 0x7f2f0c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f1e7fc640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f20131df0 0x7f2f2011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f1e7fc640 10 monclient: get_auth_request con 0x7f2f20131df0 auth_method 0 2024-09-15T01:36:27.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f2f1004c320 con 0x7f2f2009faa0 2024-09-15T01:36:27.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.361+0000 7f2f1f7fe640 1 --2- 172.21.15.178:0/462191939 >> v2:172.21.15.204:6812/28210 conn(0x7f2f0c0397b0 0x7f2f0c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f1f7fe640 10 monclient: get_auth_request con 0x7f2f0c0397b0 auth_method 0 2024-09-15T01:36:27.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_done global_id 24154 payload 824 2024-09-15T01:36:27.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f1e7fc640 1 -- 172.21.15.178:0/462191939 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f2004c4a0 con 0x7f2f20131df0 2024-09-15T01:36:27.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f1e7fc640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f20131df0 0x7f2f2011ca70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2f200a3290 tx=0x7f2f14032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f1f7fe640 1 --2- 172.21.15.178:0/462191939 >> v2:172.21.15.204:6812/28210 conn(0x7f2f0c0397b0 0x7f2f0c03bc60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2f2009b190 tx=0x7f2f18009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f1402ac50 con 0x7f2f20131df0 2024-09-15T01:36:27.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f06ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f06ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:27.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f06ffd640 10 monclient: dump: 2024-09-15T01:36:27.394 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.395 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.395 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.395 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.395 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.395 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.395 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.395 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.395 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.395 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.395 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.395 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.396 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.396 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.396 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.396 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f2f1403d690 con 0x7f2f20131df0 2024-09-15T01:36:27.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f06ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:27.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f06ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:27.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f20131df0 msgr2=0x7f2f2011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f06ffd640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f20131df0 0x7f2f2011ca70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2f200a3290 tx=0x7f2f14032000 comp rx=0 tx=0).stop 2024-09-15T01:36:27.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f20131df0 msgr2=0x7f2f2011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:27.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.362+0000 7f2f06ffd640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f20131df0 0x7f2f2011ca70 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2f200a3290 tx=0x7f2f14032000 comp rx=0 tx=0).stop 2024-09-15T01:36:27.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f25900640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:27.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f25900640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f2011cfb0 0x7f2f20096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f25900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f25900640 1 -- 172.21.15.178:0/462191939 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2004bc80 con 0x7f2f2011cfb0 2024-09-15T01:36:27.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f1e7fc640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f2011cfb0 0x7f2f20096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f1e7fc640 10 monclient: get_auth_request con 0x7f2f2011cfb0 auth_method 0 2024-09-15T01:36:27.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f1e7fc640 10 monclient(hunting): handle_auth_done global_id 24163 payload 824 2024-09-15T01:36:27.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.368+0000 7f2f1e7fc640 1 -- 172.21.15.178:0/462191939 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2f2005ed70 con 0x7f2f2011cfb0 2024-09-15T01:36:27.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.369+0000 7f2f1e7fc640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f2011cfb0 0x7f2f20096a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f1403f000 tx=0x7f2f1403d000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.369+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f14046d70 con 0x7f2f2011cfb0 2024-09-15T01:36:27.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.369+0000 7f2f06ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.369+0000 7f2f06ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:27.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.369+0000 7f2f06ffd640 10 monclient: dump: 2024-09-15T01:36:27.400 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.400 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.400 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.400 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.400 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.400 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.401 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.401 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.401 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.401 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.401 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.401 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.401 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.401 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.401 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.401 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.369+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f2f14052410 con 0x7f2f2011cfb0 2024-09-15T01:36:27.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.369+0000 7f2f06ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:27.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.369+0000 7f2f06ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:27.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.369+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f2011cfb0 msgr2=0x7f2f20096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.369+0000 7f2f06ffd640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f2011cfb0 0x7f2f20096a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f1403f000 tx=0x7f2f1403d000 comp rx=0 tx=0).stop 2024-09-15T01:36:27.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.369+0000 7f2f06ffd640 1 -- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f2011cfb0 msgr2=0x7f2f20096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:27.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.369+0000 7f2f06ffd640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f2011cfb0 0x7f2f20096a30 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f1403f000 tx=0x7f2f1403d000 comp rx=0 tx=0).stop 2024-09-15T01:36:27.402 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:27.403 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:27.404 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:27.404 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:27.404 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.404 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:36:27.404 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:27.404 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:27.404 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:27.404 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:27.404 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:27.404 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:27.405 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:27.405 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:27.405 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:27.405 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:27.405 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:27.405 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:27.405 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.405 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:27.405 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:27.405 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:27.405 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:27.406 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:27.406 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:27.406 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:27.406 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:27.406 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:27.406 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:27.406 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:27.406 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.406 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:27.406 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:27.406 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:27.407 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:27.407 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:27.407 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:27.407 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:27.407 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:27.407 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:27.407 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:27.407 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:27.407 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:27.407 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:27.407 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:27.408 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:27.408 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:27.408 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:27.408 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:27.408 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:27.408 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:27.408 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:27.408 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:27.408 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:27.408 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:27.408 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:27.409 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:27.409 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.409 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:27.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.409 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:27.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.409 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:27.409 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:27.409 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:27.410 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.410 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:27.410 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.410 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.410 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:27.411 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:27.411 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.411 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.411 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.411 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:27.411 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:27.411 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.412 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:27.412 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.412 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:27.412 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.412 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.413 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:27.413 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:27.413 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.413 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.413 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.413 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:27.413 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:27.413 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:27.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:27.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.414 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.415 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:27.415 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:27.415 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.415 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.415 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.415 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:27.415 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:27.416 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:27.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:27.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.416 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:27.418 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:27.418 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:27.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:27.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.419 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.419 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:27.419 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:27.419 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.419 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.419 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.419 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:27.420 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:27.420 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.420 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:27.420 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.420 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:27.421 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.421 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.421 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:27.421 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:27.421 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.421 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.421 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.421 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:27.422 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:27.422 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.422 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:27.422 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.422 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:27.423 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.423 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.423 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:27.423 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:27.423 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.423 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.423 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.423 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:27.424 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:27.424 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.424 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:27.424 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.424 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:27.425 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.425 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.425 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.425 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:27.425 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:27.425 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.425 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.425 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.426 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:27.426 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:27.426 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:27.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:27.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.426 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:27.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:27.426 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.426 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:27.427 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:27.427 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:27.427 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.427 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.427 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:27.427 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:27.427 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:27.427 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:27.427 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:27.427 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.427 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.428 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:27.428 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:27.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.428 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:27.428 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:27.428 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:27.428 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.428 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:27.428 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:27.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:27.428 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:27.429 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:27.429 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:27.429 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:27.429 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:27.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:27.429 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:27.429 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:27.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.370+0000 7f2f25900640 1 -- 172.21.15.178:0/462191939 >> v2:172.21.15.204:6812/28210 conn(0x7f2f0c0397b0 msgr2=0x7f2f0c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.370+0000 7f2f25900640 1 --2- 172.21.15.178:0/462191939 >> v2:172.21.15.204:6812/28210 conn(0x7f2f0c0397b0 0x7f2f0c03bc60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2f2009b190 tx=0x7f2f18009040 comp rx=0 tx=0).stop 2024-09-15T01:36:27.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.370+0000 7f2f25900640 10 monclient: shutdown 2024-09-15T01:36:27.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.370+0000 7f2f25900640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:27.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.370+0000 7f2f25900640 1 -- 172.21.15.178:0/462191939 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f2009faa0 msgr2=0x7f2f2009b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.370+0000 7f2f25900640 1 --2- 172.21.15.178:0/462191939 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f2009faa0 0x7f2f2009b240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000ecb0 tx=0x7f2f100090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.370+0000 7f2f1f7fe640 1 -- 172.21.15.178:0/462191939 reap_dead start 2024-09-15T01:36:27.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.370+0000 7f2f1f7fe640 1 -- 172.21.15.178:0/462191939 reap_dead start 2024-09-15T01:36:27.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.370+0000 7f2f25900640 1 -- 172.21.15.178:0/462191939 shutdown_connections 2024-09-15T01:36:27.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.370+0000 7f2f25900640 1 -- 172.21.15.178:0/462191939 >> 172.21.15.178:0/462191939 conn(0x7f2f200a5ed0 msgr2=0x7f2f20135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:27.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.370+0000 7f2f25900640 1 -- 172.21.15.178:0/462191939 shutdown_connections 2024-09-15T01:36:27.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.370+0000 7f2f25900640 1 -- 172.21.15.178:0/462191939 wait complete. 2024-09-15T01:36:27.432 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:27.432 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T01:36:27.432 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T01:36:27.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.491+0000 7fe9aa7fc640 1 -- 172.21.15.204:0/2803170416 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}]=0 added bucket fooo type host to crush map v174) v1 ==== 140+0+0 (secure 0 0 0) 0x7fe9ac0180a0 con 0x7fe9b4069d60 2024-09-15T01:36:27.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.491+0000 7fe9aa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-09-15T01:36:27.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.491+0000 7fe9aa7fc640 10 monclient: _finish_command 2 = system:0 added bucket fooo type host to crush map 2024-09-15T01:36:27.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.524+0000 7f0c1dc34640 10 monclient: get_monmap_and_config 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.524+0000 7f0c1dc34640 10 monclient: build_initial_monmap 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.524+0000 7f0c1dc34640 10 monclient: monmap: 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:27.525928+0000 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:27.525928+0000 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:27.527 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:27.528 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:27.528 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:27.528 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:27.528 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:27.528 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:27.528 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.524+0000 7f0c1dc34640 1 Processor -- start 2024-09-15T01:36:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 1 -- start start 2024-09-15T01:36:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 10 monclient: init 2024-09-15T01:36:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 10 monclient: _add_conns ranks=[5,6,2,8,7,1,10,3,0,4,9] 2024-09-15T01:36:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c18096a30 0x7f0c18098d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 10 monclient(hunting): picked mon.noname-h con 0x7f0c18096a30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:27.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c18096e00 0x7f0c180971d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 10 monclient(hunting): picked mon.noname-d con 0x7f0c18096e00 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18133ad0 0x7f0c18135eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 10 monclient(hunting): picked mon.noname-b con 0x7f0c18133ad0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c18099260 con 0x7f0c18133ad0 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180997c0 con 0x7f0c18096a30 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0c18098060 con 0x7f0c18096e00 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c1dc34640 10 monclient(hunting): authenticate will time out at 992.276599s 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c17fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18133ad0 0x7f0c18135eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c0ffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c18096e00 0x7f0c180971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c0ffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c18096e00 0x7f0c180971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:60144/0 (socket says 172.21.15.178:60144) 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c17fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18133ad0 0x7f0c18135eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:38064/0 (socket says 172.21.15.178:38064) 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c0ffff640 1 -- 172.21.15.178:0/272082249 learned_addr learned my addr 172.21.15.178:0/272082249 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c0ffff640 10 monclient(hunting): get_auth_request con 0x7f0c18096e00 auth_method 0 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c0ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c17fff640 10 monclient(hunting): get_auth_request con 0x7f0c18133ad0 auth_method 0 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c17fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c17fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c17fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c177fe640 1 --2- 172.21.15.178:0/272082249 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c18096a30 0x7f0c18098d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c177fe640 10 monclient(hunting): get_auth_request con 0x7f0c18096a30 auth_method 0 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c17fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c17fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c17fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.525+0000 7f0c177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c17fff640 10 monclient(hunting): handle_auth_done global_id 24194 payload 274 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c17fff640 1 -- 172.21.15.178:0/272082249 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c18096e00 msgr2=0x7f0c180971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:27.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c17fff640 1 --2- 172.21.15.178:0/272082249 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c18096e00 0x7f0c180971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c17fff640 1 -- 172.21.15.178:0/272082249 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c18096a30 msgr2=0x7f0c18098d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c17fff640 1 --2- 172.21.15.178:0/272082249 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c18096a30 0x7f0c18098d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c17fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c17fff640 1 monclient: found mon.noname-b 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c17fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c17fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c17fff640 1 -- 172.21.15.178:0/272082249 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1804bf60 con 0x7f0c18133ad0 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c0ffff640 1 --2- 172.21.15.178:0/272082249 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c18096e00 0x7f0c180971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c17fff640 1 --2- 172.21.15.178:0/272082249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18133ad0 0x7f0c18135eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000d100 tx=0x7f0c1000b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b1aae9b1d43c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c16ffd640 1 -- 172.21.15.178:0/272082249 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c1000b2e0 con 0x7f0c18133ad0 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c16ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c16ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c16ffd640 10 monclient: dump: 2024-09-15T01:36:27.540 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.541 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c16ffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c16ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c16ffd640 1 -- 172.21.15.178:0/272082249 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1002f050 con 0x7f0c18133ad0 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c1dc34640 5 monclient: authenticate success, global_id 24194 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c1dc34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c16ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c1dc34640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c16ffd640 1 -- 172.21.15.178:0/272082249 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c10009e50 con 0x7f0c18133ad0 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c1dc34640 4 set_mon_vals no callback set 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c16ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c16ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.526+0000 7f0c16ffd640 10 monclient: dump: 2024-09-15T01:36:27.542 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.544 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.527+0000 7f0c1dc34640 10 monclient: shutdown 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.527+0000 7f0c1dc34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.527+0000 7f0c1dc34640 1 -- 172.21.15.178:0/272082249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18133ad0 msgr2=0x7f0c18135eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.527+0000 7f0c1dc34640 1 --2- 172.21.15.178:0/272082249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18133ad0 0x7f0c18135eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000d100 tx=0x7f0c1000b990 comp rx=0 tx=0).stop 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.527+0000 7f0c1dc34640 1 -- 172.21.15.178:0/272082249 shutdown_connections 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.527+0000 7f0c1dc34640 1 --2- 172.21.15.178:0/272082249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18133ad0 0x7f0c18135eb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.527+0000 7f0c1dc34640 1 --2- 172.21.15.178:0/272082249 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c18096e00 0x7f0c180971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.527+0000 7f0c1dc34640 1 --2- 172.21.15.178:0/272082249 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c18096a30 0x7f0c18098d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.527+0000 7f0c1dc34640 1 -- 172.21.15.178:0/272082249 >> 172.21.15.178:0/272082249 conn(0x7f0c180a22e0 msgr2=0x7f0c180a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.527+0000 7f0c1dc34640 1 -- 172.21.15.178:0/272082249 shutdown_connections 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.527+0000 7f0c1dc34640 1 -- 172.21.15.178:0/272082249 wait complete. 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient: build_initial_monmap 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient: monmap: 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:27.546 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 1 Processor -- start 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 1 -- start start 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient: init 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient: _add_conns ranks=[1,2,7,4,6,8,0,5,3] 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c18096a30 0x7f0c1806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient(hunting): picked mon.noname-f con 0x7f0c18096a30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18096e00 0x7f0c1806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient(hunting): picked mon.noname-b con 0x7f0c18096e00 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c18133ad0 0x7f0c1806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient(hunting): picked mon.noname-i con 0x7f0c18133ad0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1809bc30 con 0x7f0c18096a30 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c18119680 con 0x7f0c18096e00 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0c18119bb0 con 0x7f0c18133ad0 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.528+0000 7f0c1dc34640 10 monclient(hunting): authenticate will time out at 992.279915s 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18096e00 0x7f0c1806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18096e00 0x7f0c1806d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:38080/0 (socket says 172.21.15.178:38080) 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 1 -- 172.21.15.178:0/3910455127 learned_addr learned my addr 172.21.15.178:0/3910455127 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 10 monclient(hunting): get_auth_request con 0x7f0c18096e00 auth_method 0 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c177fe640 1 --2- 172.21.15.178:0/3910455127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c18096a30 0x7f0c1806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c177fe640 10 monclient(hunting): get_auth_request con 0x7f0c18096a30 auth_method 0 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c17fff640 1 --2- 172.21.15.178:0/3910455127 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c18133ad0 0x7f0c1806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c17fff640 10 monclient(hunting): get_auth_request con 0x7f0c18133ad0 auth_method 0 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c17fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c17fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c17fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c17fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c17fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c17fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 10 monclient(hunting): handle_auth_done global_id 24203 payload 824 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 1 -- 172.21.15.178:0/3910455127 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c18133ad0 msgr2=0x7f0c1806d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 1 --2- 172.21.15.178:0/3910455127 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c18133ad0 0x7f0c1806d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 1 -- 172.21.15.178:0/3910455127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c18096a30 msgr2=0x7f0c1806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 1 --2- 172.21.15.178:0/3910455127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c18096a30 0x7f0c1806cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 1 monclient: found mon.noname-b 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 1 -- 172.21.15.178:0/3910455127 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c00009450 con 0x7f0c18096e00 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0ffff640 1 --2- 172.21.15.178:0/3910455127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18096e00 0x7f0c1806d270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000ecb0 tx=0x7f0c000091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0f7fe640 1 -- 172.21.15.178:0/3910455127 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c00013470 con 0x7f0c18096e00 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0f7fe640 10 monclient: dump: 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.551 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.552 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c1dc34640 5 monclient: authenticate success, global_id 24203 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0f7fe640 1 -- 172.21.15.178:0/3910455127 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c00013610 con 0x7f0c18096e00 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c1dc34640 10 monclient: _renew_subs 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c1dc34640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c1dc34640 1 -- 172.21.15.178:0/3910455127 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1000bc30 con 0x7f0c18096e00 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0f7fe640 1 -- 172.21.15.178:0/3910455127 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c00013c00 con 0x7f0c18096e00 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c167fc640 4 set_mon_vals no callback set 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c0f7fe640 10 monclient: dump: 2024-09-15T01:36:27.553 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.554 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c1dc34640 10 monclient: _renew_subs 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c1dc34640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.529+0000 7f0c1dc34640 1 -- 172.21.15.178:0/3910455127 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1809e940 con 0x7f0c18096e00 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.530+0000 7f0c0f7fe640 1 -- 172.21.15.178:0/3910455127 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0c00013780 con 0x7f0c18096e00 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.530+0000 7f0c0f7fe640 1 --2- 172.21.15.178:0/3910455127 >> v2:172.21.15.204:6812/28210 conn(0x7f0bf80397b0 0x7f0bf803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.530+0000 7f0c0f7fe640 1 -- 172.21.15.178:0/3910455127 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6771+0+0 (secure 0 0 0) 0x7f0c0004c2c0 con 0x7f0c18096e00 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.530+0000 7f0c1dc34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.530+0000 7f0c1dc34640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.530+0000 7f0c1dc34640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.530+0000 7f0c1dc34640 1 -- 172.21.15.178:0/3910455127 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1804b720 con 0x7f0c18096e00 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.530+0000 7f0c177fe640 1 --2- 172.21.15.178:0/3910455127 >> v2:172.21.15.204:6812/28210 conn(0x7f0bf80397b0 0x7f0bf803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.530+0000 7f0c177fe640 10 monclient: get_auth_request con 0x7f0bf80397b0 auth_method 0 2024-09-15T01:36:27.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.531+0000 7f0c177fe640 1 --2- 172.21.15.178:0/3910455127 >> v2:172.21.15.204:6812/28210 conn(0x7f0bf80397b0 0x7f0bf803bc60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009d20 tx=0x7f0c080090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.535+0000 7f0c0f7fe640 1 -- 172.21.15.178:0/3910455127 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0c0000cb60 con 0x7f0c18096e00 2024-09-15T01:36:27.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.535+0000 7f0c0f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:27.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.535+0000 7f0c0f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:27.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.590+0000 7fe9b8f5d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-09-15T01:36:27.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.590+0000 7fe9b8f5d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-09-15T01:36:27.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.590+0000 7fe9b8f5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:27.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.590+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2803170416 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"} v 0) v1 -- 0x7fe9b40008d0 con 0x7fe9b4069d60 2024-09-15T01:36:27.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.590+0000 7fe9aa7fc640 1 -- 172.21.15.204:0/2803170416 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}]=0 bucket 'fooo' already exists v174) v1 ==== 128+0+0 (secure 0 0 0) 0x7fe9ac01a2c0 con 0x7fe9b4069d60 2024-09-15T01:36:27.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.590+0000 7fe9aa7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-09-15T01:36:27.593 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.590+0000 7fe9aa7fc640 10 monclient: _finish_command 3 = system:0 bucket 'fooo' already exists 2024-09-15T01:36:27.593 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'fooo' already exists 2024-09-15T01:36:27.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.592+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2803170416 >> v2:172.21.15.204:6812/28210 conn(0x7fe9940397b0 msgr2=0x7fe99403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.592+0000 7fe9b8f5d640 1 --2- 172.21.15.204:0/2803170416 >> v2:172.21.15.204:6812/28210 conn(0x7fe9940397b0 0x7fe99403bc60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe99c034000 tx=0x7fe99c032000 comp rx=0 tx=0).stop 2024-09-15T01:36:27.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.592+0000 7fe9b8f5d640 10 monclient: shutdown 2024-09-15T01:36:27.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.592+0000 7fe9b8f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:27.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.592+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2803170416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b4069d60 msgr2=0x7fe9b411e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.592+0000 7fe9b8f5d640 1 --2- 172.21.15.204:0/2803170416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b4069d60 0x7fe9b411e730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00c0c0 tx=0x7fe9ac00b040 comp rx=0 tx=0).stop 2024-09-15T01:36:27.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.593+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2803170416 shutdown_connections 2024-09-15T01:36:27.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.593+0000 7fe9b8f5d640 1 --2- 172.21.15.204:0/2803170416 >> v2:172.21.15.204:6812/28210 conn(0x7fe9940397b0 0x7fe99403bc60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.593+0000 7fe9b8f5d640 1 --2- 172.21.15.204:0/2803170416 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9b4132aa0 0x7fe9b411f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.593+0000 7fe9b8f5d640 1 --2- 172.21.15.204:0/2803170416 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9b406ee50 0x7fe9b411ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.593+0000 7fe9b8f5d640 1 --2- 172.21.15.204:0/2803170416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9b4069d60 0x7fe9b411e730 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.593+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2803170416 >> 172.21.15.204:0/2803170416 conn(0x7fe9b4067bb0 msgr2=0x7fe9b406d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:27.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.593+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2803170416 shutdown_connections 2024-09-15T01:36:27.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.593+0000 7fe9b8f5d640 1 -- 172.21.15.204:0/2803170416 wait complete. 2024-09-15T01:36:27.604 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush move fooo root=default 2024-09-15T01:36:27.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.676+0000 7f93d7c6f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:27.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.676+0000 7f93d7c6f640 10 monclient: build_initial_monmap 2024-09-15T01:36:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.676+0000 7f93d7c6f640 10 monclient: monmap: 2024-09-15T01:36:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:27.677676+0000 2024-09-15T01:36:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:27.677676+0000 2024-09-15T01:36:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:27.679 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:27.680 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:27.680 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:27.680 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:27.680 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:27.680 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:27.680 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:27.680 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:27.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.676+0000 7f93d7c6f640 1 Processor -- start 2024-09-15T01:36:27.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.676+0000 7f93d7c6f640 1 -- start start 2024-09-15T01:36:27.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 10 monclient: init 2024-09-15T01:36:27.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:27.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 10 monclient: _add_conns ranks=[10,6,4,5,0,3,8,7,1,9,2] 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f93d00a3560 0x7f93d009b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 10 monclient(hunting): picked mon.noname-f con 0x7f93d00a3560 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f93d009cca0 0x7f93d0067130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 10 monclient(hunting): picked mon.noname-d con 0x7f93d009cca0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f93d009d090 0x7f93d0067670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 10 monclient(hunting): picked mon.noname-c con 0x7f93d009d090 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d009e120 con 0x7f93d00a3560 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93d0099f10 con 0x7f93d009d090 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f93d009ab00 con 0x7f93d009cca0 2024-09-15T01:36:27.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d7c6f640 10 monclient(hunting): authenticate will time out at 1003.320499s 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f93d00a3560 0x7f93d009b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f93d00a3560 0x7f93d009b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39388/0 (socket says 172.21.15.204:39388) 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 1 -- 172.21.15.204:0/3183281015 learned_addr learned my addr 172.21.15.204:0/3183281015 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 10 monclient(hunting): get_auth_request con 0x7f93d00a3560 auth_method 0 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d620c640 1 --2- 172.21.15.204:0/3183281015 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f93d009d090 0x7f93d0067670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d520a640 1 --2- 172.21.15.204:0/3183281015 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f93d009cca0 0x7f93d0067130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d620c640 10 monclient(hunting): get_auth_request con 0x7f93d009d090 auth_method 0 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d620c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d620c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d620c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d520a640 10 monclient(hunting): get_auth_request con 0x7f93d009cca0 auth_method 0 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d520a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d520a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d520a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d620c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d620c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d620c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 10 monclient(hunting): handle_auth_done global_id 6634 payload 274 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 1 -- 172.21.15.204:0/3183281015 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f93d009cca0 msgr2=0x7f93d0067130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 1 --2- 172.21.15.204:0/3183281015 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f93d009cca0 0x7f93d0067130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 1 -- 172.21.15.204:0/3183281015 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f93d009d090 msgr2=0x7f93d0067670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:27.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 1 --2- 172.21.15.204:0/3183281015 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f93d009d090 0x7f93d0067670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 10 monclient: _finish_hunting 0 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 1 monclient: found mon.noname-f 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d5a0b640 1 -- 172.21.15.204:0/3183281015 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d004b720 con 0x7f93d00a3560 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.677+0000 7f93d520a640 1 --2- 172.21.15.204:0/3183281015 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f93d009cca0 0x7f93d0067130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d5a0b640 1 --2- 172.21.15.204:0/3183281015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93d00a3560 0x7f93d009b480 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009970 tx=0x7f93cc0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75fe0dee9c92d3b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d4a09640 1 -- 172.21.15.204:0/3183281015 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93cc02aca0 con 0x7f93d00a3560 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d4a09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d4a09640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d4a09640 10 monclient: dump: 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:27.692 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.693 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d4a09640 10 monclient: _finish_auth 0 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d4a09640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d7c6f640 5 monclient: authenticate success, global_id 6634 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d7c6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d4a09640 1 -- 172.21.15.204:0/3183281015 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93cc02ae00 con 0x7f93d00a3560 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d4a09640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d4a09640 1 -- 172.21.15.204:0/3183281015 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93cc031820 con 0x7f93d00a3560 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d4a09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d4a09640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d4a09640 10 monclient: dump: 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.694 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.695 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.695 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d7c6f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.678+0000 7f93d7c6f640 4 set_mon_vals no callback set 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.679+0000 7f93d7c6f640 10 monclient: shutdown 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.679+0000 7f93d7c6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.679+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3183281015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93d00a3560 msgr2=0x7f93d009b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.679+0000 7f93d7c6f640 1 --2- 172.21.15.204:0/3183281015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93d00a3560 0x7f93d009b480 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009970 tx=0x7f93cc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.679+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3183281015 shutdown_connections 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.679+0000 7f93d7c6f640 1 --2- 172.21.15.204:0/3183281015 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f93d009d090 0x7f93d0067670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.679+0000 7f93d7c6f640 1 --2- 172.21.15.204:0/3183281015 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f93d009cca0 0x7f93d0067130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.679+0000 7f93d7c6f640 1 --2- 172.21.15.204:0/3183281015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93d00a3560 0x7f93d009b480 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.679+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3183281015 >> 172.21.15.204:0/3183281015 conn(0x7f93d00687e0 msgr2=0x7f93d00a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.679+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3183281015 shutdown_connections 2024-09-15T01:36:27.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.679+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3183281015 wait complete. 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient: build_initial_monmap 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient: monmap: 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:27.697 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 1 Processor -- start 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 1 -- start start 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient: init 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient: _add_conns ranks=[8,0,5,1,2,3,6,4,7] 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93d009cca0 0x7f93d0141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient(hunting): picked mon.noname-e con 0x7f93d009cca0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:27.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93d009d090 0x7f93d0142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient(hunting): picked mon.noname-a con 0x7f93d009d090 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f93d00a3560 0x7f93d0142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient(hunting): picked mon.noname-h con 0x7f93d00a3560 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d0121920 con 0x7f93d009d090 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93d006f000 con 0x7f93d00a3560 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f93d006eaa0 con 0x7f93d009cca0 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.680+0000 7f93d7c6f640 10 monclient(hunting): authenticate will time out at 1003.323965s 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f93d00a3560 0x7f93d0142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d5a0b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93d009cca0 0x7f93d0141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f93d00a3560 0x7f93d0142790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:58166/0 (socket says 172.21.15.204:58166) 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 1 -- 172.21.15.204:0/3098336722 learned_addr learned my addr 172.21.15.204:0/3098336722 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 10 monclient(hunting): get_auth_request con 0x7f93d00a3560 auth_method 0 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d5a0b640 10 monclient(hunting): get_auth_request con 0x7f93d009cca0 auth_method 0 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d5a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d5a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d5a0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d520a640 1 --2- 172.21.15.204:0/3098336722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93d009d090 0x7f93d0142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d520a640 10 monclient(hunting): get_auth_request con 0x7f93d009d090 auth_method 0 2024-09-15T01:36:27.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d520a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d520a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d520a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d5a0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d5a0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d5a0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d520a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d520a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d520a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 10 monclient(hunting): handle_auth_done global_id 5990 payload 824 2024-09-15T01:36:27.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 1 -- 172.21.15.204:0/3098336722 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93d009cca0 msgr2=0x7f93d0141d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 1 --2- 172.21.15.204:0/3098336722 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93d009cca0 0x7f93d0141d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 1 -- 172.21.15.204:0/3098336722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93d009d090 msgr2=0x7f93d0142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 1 --2- 172.21.15.204:0/3098336722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93d009d090 0x7f93d0142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 1 monclient: found mon.noname-h 2024-09-15T01:36:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 1 -- 172.21.15.204:0/3098336722 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93cc009450 con 0x7f93d00a3560 2024-09-15T01:36:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d5a0b640 1 --2- 172.21.15.204:0/3098336722 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93d009cca0 0x7f93d0141d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d620c640 1 --2- 172.21.15.204:0/3098336722 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f93d00a3560 0x7f93d0142790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009f20 tx=0x7f93c800b040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c5ffb640 1 -- 172.21.15.204:0/3098336722 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93c800ce10 con 0x7f93d00a3560 2024-09-15T01:36:27.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c5ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c5ffb640 10 monclient: dump: 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.703 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.704 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c5ffb640 1 -- 172.21.15.204:0/3098336722 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c800f040 con 0x7f93d00a3560 2024-09-15T01:36:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d7c6f640 5 monclient: authenticate success, global_id 5990 2024-09-15T01:36:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c5ffb640 1 -- 172.21.15.204:0/3098336722 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93c8014d80 con 0x7f93d00a3560 2024-09-15T01:36:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:27.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c5ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c7fff640 4 set_mon_vals no callback set 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93c5ffb640 10 monclient: dump: 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:27.705 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d7c6f640 10 monclient: _renew_subs 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d7c6f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3098336722 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d0122e40 con 0x7f93d00a3560 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d7c6f640 10 monclient: _renew_subs 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d7c6f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.681+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3098336722 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d0122880 con 0x7f93d00a3560 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.682+0000 7f93c5ffb640 1 -- 172.21.15.204:0/3098336722 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f93c8014480 con 0x7f93d00a3560 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.682+0000 7f93d7c6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:27.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.682+0000 7f93d7c6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.682+0000 7f93d7c6f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.682+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3098336722 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d0122a70 con 0x7f93d00a3560 2024-09-15T01:36:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.682+0000 7f93c5ffb640 1 --2- 172.21.15.204:0/3098336722 >> v2:172.21.15.204:6812/28210 conn(0x7f93b40396e0 0x7f93b403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.682+0000 7f93c5ffb640 1 -- 172.21.15.204:0/3098336722 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6771+0+0 (secure 0 0 0) 0x7f93c804cae0 con 0x7f93d00a3560 2024-09-15T01:36:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.682+0000 7f93d5a0b640 1 --2- 172.21.15.204:0/3098336722 >> v2:172.21.15.204:6812/28210 conn(0x7f93b40396e0 0x7f93b403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.683+0000 7f93d5a0b640 10 monclient: get_auth_request con 0x7f93b40396e0 auth_method 0 2024-09-15T01:36:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.683+0000 7f93d5a0b640 1 --2- 172.21.15.204:0/3098336722 >> v2:172.21.15.204:6812/28210 conn(0x7f93b40396e0 0x7f93b403bb90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f93cc034000 tx=0x7f93cc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:27.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.687+0000 7f93c5ffb640 1 -- 172.21.15.204:0/3098336722 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f93c8012b20 con 0x7f93d00a3560 2024-09-15T01:36:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.687+0000 7f93c5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:27.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.687+0000 7f93c5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:27.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.750+0000 7f0c1dc34640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T01:36:27.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.750+0000 7f0c1dc34640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T01:36:27.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.750+0000 7f0c1dc34640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:27.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.750+0000 7f0c1dc34640 1 -- 172.21.15.178:0/3910455127 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f0c181197f0 con 0x7f0c18096e00 2024-09-15T01:36:27.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.752+0000 7f0c0f7fe640 1 -- 172.21.15.178:0/3910455127 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f0c0001b820 con 0x7f0c18096e00 2024-09-15T01:36:27.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.752+0000 7f0c0f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T01:36:27.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.752+0000 7f0c0f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:27.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 1 -- 172.21.15.178:0/3910455127 >> v2:172.21.15.204:6812/28210 conn(0x7f0bf80397b0 msgr2=0x7f0bf803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 1 --2- 172.21.15.178:0/3910455127 >> v2:172.21.15.204:6812/28210 conn(0x7f0bf80397b0 0x7f0bf803bc60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009d20 tx=0x7f0c080090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 10 monclient: shutdown 2024-09-15T01:36:27.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:27.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 1 -- 172.21.15.178:0/3910455127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18096e00 msgr2=0x7f0c1806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:27.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 1 --2- 172.21.15.178:0/3910455127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18096e00 0x7f0c1806d270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000ecb0 tx=0x7f0c000091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 1 -- 172.21.15.178:0/3910455127 shutdown_connections 2024-09-15T01:36:27.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 1 --2- 172.21.15.178:0/3910455127 >> v2:172.21.15.204:6812/28210 conn(0x7f0bf80397b0 0x7f0bf803bc60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 1 --2- 172.21.15.178:0/3910455127 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c18133ad0 0x7f0c1806d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 1 --2- 172.21.15.178:0/3910455127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c18096e00 0x7f0c1806d270 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 1 --2- 172.21.15.178:0/3910455127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c18096a30 0x7f0c1806cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:27.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 1 -- 172.21.15.178:0/3910455127 >> 172.21.15.178:0/3910455127 conn(0x7f0c180a22e0 msgr2=0x7f0c180a0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:27.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.754+0000 7f0c1dc34640 1 -- 172.21.15.178:0/3910455127 shutdown_connections 2024-09-15T01:36:27.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:27.755+0000 7f0c1dc34640 1 -- 172.21.15.178:0/3910455127 wait complete. 2024-09-15T01:36:27.775 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T01:36:27.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=9,config=2,health=11,kv=8,logm=70} crc {auth=3865675262,config=3140514009,health=2256599861,kv=706962213,logm=968727092}) 2024-09-15T01:36:27.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=1,health=11,kv=8,logm=71} crc {auth=3865675262,config=1228700967,health=2256599861,kv=706962213,logm=1503994886}) 2024-09-15T01:36:27.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=9,config=2,health=11,kv=8,logm=70} crc {auth=3865675262,config=3140514009,health=2256599861,kv=706962213,logm=968727092}) 2024-09-15T01:36:27.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=8,config=2,health=12,kv=7,logm=71} crc {auth=1249981521,config=3140514010,health=3245091846,kv=2951374574,logm=2372049208}) 2024-09-15T01:36:27.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=9,config=2,health=11,kv=8,logm=70} crc {auth=3865675262,config=3140514009,health=2256599861,kv=706962213,logm=968727092}) 2024-09-15T01:36:27.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=9,config=2,health=11,kv=8,logm=70} crc {auth=3865675262,config=3140514009,health=3752708998,kv=706962213,logm=2566468169}) 2024-09-15T01:36:27.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=9,config=2,health=11,kv=8,logm=70} crc {auth=3865675262,config=3140514009,health=2256599861,kv=706962213,logm=968727092}) 2024-09-15T01:36:27.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=8,config=2,health=12,kv=7,logm=71} crc {auth=1970646714,config=3140514009,health=3245091846,kv=2951374574,logm=1477829273}) 2024-09-15T01:36:27.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=9,config=2,health=11,kv=8,logm=70} crc {auth=3865675262,config=3140514009,health=2256599861,kv=706962213,logm=968727092}) 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=9,config=2,health=12,kv=8,logm=69} crc {auth=3865675262,config=3140514009,health=3245091846,kv=706962213,logm=3939382050}) 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=9,config=2,health=11,kv=8,logm=70} crc {auth=3865675262,config=3140514009,health=2256599861,kv=706962213,logm=968727092}) 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=9,config=2,health=11,kv=8,logm=70} crc {auth=3865675262,config=3140514009,health=2035481767,kv=706962213,logm=2796803265}) 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=9,config=2,health=11,kv=8,logm=70} crc {auth=3865675262,config=3140514009,health=2256599861,kv=706962213,logm=968727092}) 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=9,config=2,health=12,kv=8,logm=69} crc {auth=3865675262,config=3140514009,health=3245091846,kv=706962213,logm=2496150622}) 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=9,config=2,health=11,kv=8,logm=70} crc {auth=3865675262,config=3140514009,health=2256599861,kv=706962213,logm=968727092}) 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=9,config=2,health=12,kv=8,logm=69} crc {auth=3865675262,config=3140514009,health=3245091846,kv=706962213,logm=2125459052}) 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=34752516}) 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586763109}) 2024-09-15T01:36:27.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=34752516}) 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1806439682}) 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=34752516}) 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2964781509}) 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=34752516}) 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=791172143}) 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=34752516}) 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3047553345}) 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=34752516}) 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2316178802}) 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=34752516}) 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2374478399}) 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=34752516}) 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=671092190}) 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=700184545}) 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1372761501}) 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=700184545}) 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2283943581}) 2024-09-15T01:36:27.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=700184545}) 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3494888318}) 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=700184545}) 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=860852821}) 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=700184545}) 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1061858425}) 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=700184545}) 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3878464180}) 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=700184545}) 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3805728521}) 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=700184545}) 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=449692142}) 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=125622592}) 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2450895765}) 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=125622592}) 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1010823856}) 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=125622592}) 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3636820343}) 2024-09-15T01:36:27.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=125622592}) 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3301483221}) 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=125622592}) 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2709011586}) 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=125622592}) 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4053184438}) 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=125622592}) 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3680205818}) 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=125622592}) 2024-09-15T01:36:27.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3746588763}) 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=180142285}) 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1394890821}) 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=180142285}) 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3240993668}) 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=180142285}) 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1739864814}) 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=180142285}) 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=493042193}) 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=180142285}) 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4102203443}) 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=180142285}) 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4269079521}) 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=180142285}) 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2653022939}) 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=180142285}) 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=276221592}) 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2771988474}) 2024-09-15T01:36:27.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3844318349}) 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2771988474}) 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=181782801}) 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2771988474}) 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1637525116}) 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2771988474}) 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2974174750}) 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2771988474}) 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=120972849}) 2024-09-15T01:36:27.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2771988474}) 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1865658937}) 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2771988474}) 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3320683784}) 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2771988474}) 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4163458634}) 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1854845042}) 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=463950280}) 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1854845042}) 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1132366590}) 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1854845042}) 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=782096736}) 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1854845042}) 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3556956054}) 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1854845042}) 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=296237162}) 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1854845042}) 2024-09-15T01:36:27.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1810208482}) 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1854845042}) 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2335057098}) 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1854845042}) 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3583758570}) 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1545251875}) 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=761063685}) 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1545251875}) 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=727455871}) 2024-09-15T01:36:27.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1545251875}) 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=306538248}) 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1545251875}) 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3064318151}) 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1545251875}) 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2988374266}) 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1545251875}) 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1184510280}) 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1545251875}) 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3274941745}) 2024-09-15T01:36:27.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1545251875}) 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2295607079}) 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=273670188}) 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2836537506}) 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=273670188}) 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=801264995}) 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=273670188}) 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4133180215}) 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=273670188}) 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=659225663}) 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=273670188}) 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1663129679}) 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=273670188}) 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2394465847}) 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=273670188}) 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1136833514}) 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=273670188}) 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3614257545}) 2024-09-15T01:36:27.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3297559171}) 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2489891303}) 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3297559171}) 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3320553346}) 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3297559171}) 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1919345649}) 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3297559171}) 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1999553128}) 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3297559171}) 2024-09-15T01:36:27.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1055704216}) 2024-09-15T01:36:27.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3297559171}) 2024-09-15T01:36:27.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=372249717}) 2024-09-15T01:36:27.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3297559171}) 2024-09-15T01:36:27.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3891689333}) 2024-09-15T01:36:27.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3297559171}) 2024-09-15T01:36:27.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1816469325}) 2024-09-15T01:36:27.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3318517866}) 2024-09-15T01:36:27.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2108069295}) 2024-09-15T01:36:27.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3318517866}) 2024-09-15T01:36:27.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1814959055}) 2024-09-15T01:36:27.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3318517866}) 2024-09-15T01:36:27.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=225421619}) 2024-09-15T01:36:27.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3318517866}) 2024-09-15T01:36:27.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3700698890}) 2024-09-15T01:36:27.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3318517866}) 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1923050039}) 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3318517866}) 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1269177635}) 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3318517866}) 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2820246971}) 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3318517866}) 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3002428534}) 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1482232647}) 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1124594623}) 2024-09-15T01:36:27.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1482232647}) 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4245090988}) 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1482232647}) 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3297010886}) 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1482232647}) 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3833331182}) 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1482232647}) 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=755232414}) 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1482232647}) 2024-09-15T01:36:27.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1780088119}) 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1482232647}) 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3648958583}) 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1482232647}) 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=94869797}) 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=535527508}) 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=534956898}) 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=535527508}) 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1616569282}) 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=535527508}) 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=460709621}) 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=535527508}) 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=575228701}) 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=535527508}) 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1189345176}) 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=535527508}) 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=57474561}) 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=535527508}) 2024-09-15T01:36:27.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4291506527}) 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=535527508}) 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=544408075}) 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=305194768}) 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=836997482}) 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=305194768}) 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2237947269}) 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=305194768}) 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2646175046}) 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=305194768}) 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3780038322}) 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=305194768}) 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2178085298}) 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=305194768}) 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1322186570}) 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=305194768}) 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=714571990}) 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=305194768}) 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3562368500}) 2024-09-15T01:36:27.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3301395752}) 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488631108}) 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3301395752}) 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=380348371}) 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3301395752}) 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1602406476}) 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3301395752}) 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=423558593}) 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3301395752}) 2024-09-15T01:36:27.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3047189805}) 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3301395752}) 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1539236005}) 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3301395752}) 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=62766673}) 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3301395752}) 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2748599521}) 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=952401}) 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1369403264}) 2024-09-15T01:36:27.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=952401}) 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4060202291}) 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=952401}) 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=293564511}) 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=952401}) 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2688871939}) 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=952401}) 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2341166364}) 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=952401}) 2024-09-15T01:36:27.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2910472804}) 2024-09-15T01:36:27.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=952401}) 2024-09-15T01:36:27.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=189443288}) 2024-09-15T01:36:27.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=952401}) 2024-09-15T01:36:27.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1947799492}) 2024-09-15T01:36:27.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=694163995}) 2024-09-15T01:36:27.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1942234876}) 2024-09-15T01:36:27.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=694163995}) 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1214241237}) 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=694163995}) 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4219030441}) 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=694163995}) 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=29764300}) 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=694163995}) 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3590666413}) 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=694163995}) 2024-09-15T01:36:27.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1866197944}) 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=694163995}) 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2666588842}) 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=694163995}) 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1963517310}) 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2619713831}) 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4271159060}) 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2619713831}) 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1708660898}) 2024-09-15T01:36:27.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2619713831}) 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1484316346}) 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2619713831}) 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3158109964}) 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2619713831}) 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2679855339}) 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2619713831}) 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=650383370}) 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2619713831}) 2024-09-15T01:36:27.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1543638047}) 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2619713831}) 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2657916157}) 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2789481875}) 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=364077073}) 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2789481875}) 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=398845707}) 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2789481875}) 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3597823424}) 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2789481875}) 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=980958644}) 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2789481875}) 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=641475227}) 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2789481875}) 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=923395019}) 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2789481875}) 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2562268388}) 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2789481875}) 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4188012638}) 2024-09-15T01:36:27.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=440262046}) 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1971673589}) 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=440262046}) 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2941426556}) 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=440262046}) 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2747214781}) 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=440262046}) 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=426065068}) 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=440262046}) 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=424178227}) 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=440262046}) 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4136002445}) 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=440262046}) 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1964216744}) 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=440262046}) 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1408639504}) 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3338594578}) 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1936819272}) 2024-09-15T01:36:27.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3338594578}) 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3667987403}) 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3338594578}) 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1277947316}) 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3338594578}) 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3264526234}) 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3338594578}) 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3004116002}) 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3338594578}) 2024-09-15T01:36:27.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3240447629}) 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.781+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3338594578}) 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.781+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2412788751}) 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.781+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.781+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3338594578}) 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.781+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2495782733}) 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.781+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.781+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565473167}) 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.781+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1947784076}) 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565473167}) 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=167281394}) 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565473167}) 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=831891353}) 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565473167}) 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4147585440}) 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565473167}) 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2184537435}) 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565473167}) 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3175042039}) 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565473167}) 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=154901745}) 2024-09-15T01:36:27.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565473167}) 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.782+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=71810355}) 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1752644161}) 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=174385785}) 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1752644161}) 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1285532811}) 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1752644161}) 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1145576197}) 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1752644161}) 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1313127812}) 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1752644161}) 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1553839828}) 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1752644161}) 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3461499515}) 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1752644161}) 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1317536180}) 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1752644161}) 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=921940571}) 2024-09-15T01:36:27.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619499128}) 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3652736685}) 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619499128}) 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=314875150}) 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619499128}) 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=61512491}) 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619499128}) 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4217280664}) 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619499128}) 2024-09-15T01:36:27.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=228397787}) 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619499128}) 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=326028142}) 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619499128}) 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.784+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2325206908}) 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619499128}) 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1858085111}) 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=916029406}) 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2098827369}) 2024-09-15T01:36:27.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=916029406}) 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2301238162}) 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=916029406}) 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3521886889}) 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=916029406}) 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2382046985}) 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=916029406}) 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3138221198}) 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=916029406}) 2024-09-15T01:36:27.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4141726105}) 2024-09-15T01:36:27.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=916029406}) 2024-09-15T01:36:27.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2911710978}) 2024-09-15T01:36:27.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=916029406}) 2024-09-15T01:36:27.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.786+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3399288224}) 2024-09-15T01:36:27.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=2874073272,mdsmap=3585761405}) 2024-09-15T01:36:27.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=2040602289,mdsmap=1531151339}) 2024-09-15T01:36:27.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=2874073272,mdsmap=3585761405}) 2024-09-15T01:36:27.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=96,mdsmap=3,mgr=1} crc {logm=3881972410,mdsmap=1531151339,mgr=3533299129}) 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=2874073272,mdsmap=3585761405}) 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=3494905308,mdsmap=1228700967}) 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=2874073272,mdsmap=3585761405}) 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=1227046900,mdsmap=1531151339}) 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=2874073272,mdsmap=3585761405}) 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=2856896924,mdsmap=1228700967}) 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=2874073272,mdsmap=3585761405}) 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1539321982}) 2024-09-15T01:36:27.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=2874073272,mdsmap=3585761405}) 2024-09-15T01:36:27.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=381044666,mdsmap=3585761405}) 2024-09-15T01:36:27.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=2874073272,mdsmap=3585761405}) 2024-09-15T01:36:27.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.787+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=2033325139,mdsmap=2826658659}) 2024-09-15T01:36:27.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=65} crc {mdsmap=1228700967,mgr=2983835071,mgr_command_descs=3545737492,mgrstat=2023978892,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=2595779131}) 2024-09-15T01:36:27.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=66} crc {mdsmap=1228700967,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3210797403,mon_config_key=1841931312,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=2123813043}) 2024-09-15T01:36:27.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=65} crc {mdsmap=1228700967,mgr=2983835071,mgr_command_descs=3545737492,mgrstat=2023978892,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=2595779131}) 2024-09-15T01:36:27.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=65} crc {mdsmap=1228700967,mgr=3506364876,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2023978892,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=748692828}) 2024-09-15T01:36:27.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=65} crc {mdsmap=1228700967,mgr=2983835071,mgr_command_descs=3545737492,mgrstat=2023978892,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=2595779131}) 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mdsmap=1,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=66} crc {mdsmap=1228700967,mgr=4228988328,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2023978892,mon_config_key=1111662797,monmap=3140514009,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=1156983089}) 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=65} crc {mdsmap=1228700967,mgr=2983835071,mgr_command_descs=3545737492,mgrstat=2023978892,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=2595779131}) 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=2,osdmap=66} crc {mdsmap=1228700967,mgr=2983835072,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1083197897,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_snap=3178633393,osdmap=4194821938}) 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=65} crc {mdsmap=1228700967,mgr=2983835071,mgr_command_descs=3545737492,mgrstat=2023978892,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=2595779131}) 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mdsmap=1,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=65} crc {mdsmap=1228700967,mgr=3506364876,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2023978892,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=3896549967}) 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=65} crc {mdsmap=1228700967,mgr=2983835071,mgr_command_descs=3545737492,mgrstat=2023978892,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=2595779131}) 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mdsmap=1228700967,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2023978892,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=3130571679}) 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=65} crc {mdsmap=1228700967,mgr=2983835071,mgr_command_descs=3545737492,mgrstat=2023978892,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=2595779131}) 2024-09-15T01:36:27.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=4,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=66} crc {mdsmap=1228700967,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2023978892,mon_config_key=1763697598,monmap=2850442478,osd_metadata=4292802864,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=242098495}) 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=65} crc {mdsmap=1228700967,mgr=2983835071,mgr_command_descs=3545737492,mgrstat=2023978892,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=2595779131}) 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.789+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mdsmap=1228700967,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2023978892,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=1050671081,osd_snap=3178633393,osdmap=1186824609}) 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4095892756}) 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=81023344}) 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4095892756}) 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3569534213}) 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4095892756}) 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3194808186}) 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4095892756}) 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2027810712}) 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4095892756}) 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1014065518}) 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4095892756}) 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3677126062}) 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4095892756}) 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=427138516}) 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4095892756}) 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.790+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3442614156}) 2024-09-15T01:36:27.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=12441562}) 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=989644011}) 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=12441562}) 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1432382308}) 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=12441562}) 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3279969498}) 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=12441562}) 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4093900854}) 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=12441562}) 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=837326796}) 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=12441562}) 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2627295145}) 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=12441562}) 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=97} crc {osdmap=2286539841}) 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=12441562}) 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2126913883}) 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=4} crc {osdmap=2516154207}) 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=4} crc {osdmap=615908469}) 2024-09-15T01:36:27.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:36:27.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=4} crc {osdmap=2516154207}) 2024-09-15T01:36:27.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:36:27.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=3} crc {osdmap=2026225854}) 2024-09-15T01:36:27.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.863+0000 7f93d7c6f640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-09-15T01:36:27.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.863+0000 7f93d7c6f640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-09-15T01:36:27.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.863+0000 7f93d7c6f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:27.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:27.863+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3098336722 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]} v 0) v1 -- 0x7f93d0070cc0 con 0x7f93d00a3560 2024-09-15T01:36:28.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.500+0000 7f93c5ffb640 1 -- 172.21.15.204:0/3098336722 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}]=0 moved item id -16 name 'fooo' to location {root=default} in crush map v175) v1 ==== 173+0+0 (secure 0 0 0) 0x7f93c8018020 con 0x7f93d00a3560 2024-09-15T01:36:28.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.500+0000 7f93c5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-09-15T01:36:28.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.500+0000 7f93c5ffb640 10 monclient: _finish_command 2 = system:0 moved item id -16 name 'fooo' to location {root=default} in crush map 2024-09-15T01:36:28.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.599+0000 7f93d7c6f640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-09-15T01:36:28.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.599+0000 7f93d7c6f640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-09-15T01:36:28.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.599+0000 7f93d7c6f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:28.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.599+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3098336722 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]} v 0) v1 -- 0x7f93d0120d50 con 0x7f93d00a3560 2024-09-15T01:36:28.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.600+0000 7f93c5ffb640 1 -- 172.21.15.204:0/3098336722 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}]=0 no need to move item id -16 name 'fooo' to location {root=default} in crush map v175) v1 ==== 183+0+0 (secure 0 0 0) 0x7f93c8012340 con 0x7f93d00a3560 2024-09-15T01:36:28.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.600+0000 7f93c5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-09-15T01:36:28.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.600+0000 7f93c5ffb640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'fooo' to location {root=default} in crush map 2024-09-15T01:36:28.603 INFO:tasks.workunit.client.0.smithi204.stderr:no need to move item id -16 name 'fooo' to location {root=default} in crush map 2024-09-15T01:36:28.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3098336722 >> v2:172.21.15.204:6812/28210 conn(0x7f93b40396e0 msgr2=0x7f93b403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:28.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 1 --2- 172.21.15.204:0/3098336722 >> v2:172.21.15.204:6812/28210 conn(0x7f93b40396e0 0x7f93b403bb90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f93cc034000 tx=0x7f93cc032000 comp rx=0 tx=0).stop 2024-09-15T01:36:28.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 10 monclient: shutdown 2024-09-15T01:36:28.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:28.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3098336722 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f93d00a3560 msgr2=0x7f93d0142790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:28.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 1 --2- 172.21.15.204:0/3098336722 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f93d00a3560 0x7f93d0142790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009f20 tx=0x7f93c800b040 comp rx=0 tx=0).stop 2024-09-15T01:36:28.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3098336722 shutdown_connections 2024-09-15T01:36:28.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 1 --2- 172.21.15.204:0/3098336722 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f93d00a3560 0x7f93d0142790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 1 --2- 172.21.15.204:0/3098336722 >> v2:172.21.15.204:6812/28210 conn(0x7f93b40396e0 0x7f93b403bb90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 1 --2- 172.21.15.204:0/3098336722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93d009d090 0x7f93d0142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 1 --2- 172.21.15.204:0/3098336722 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f93d009cca0 0x7f93d0141d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3098336722 >> 172.21.15.204:0/3098336722 conn(0x7f93d00687e0 msgr2=0x7f93d00a2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:28.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3098336722 shutdown_connections 2024-09-15T01:36:28.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.602+0000 7f93d7c6f640 1 -- 172.21.15.204:0/3098336722 wait complete. 2024-09-15T01:36:28.613 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket barr rack 2024-09-15T01:36:28.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.686+0000 7f6d1e19d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:28.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.686+0000 7f6d1e19d640 10 monclient: build_initial_monmap 2024-09-15T01:36:28.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.686+0000 7f6d1e19d640 10 monclient: monmap: 2024-09-15T01:36:28.689 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:28.689 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:28.689 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:28.687627+0000 2024-09-15T01:36:28.689 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:28.687627+0000 2024-09-15T01:36:28.689 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:28.689 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:28.689 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:28.689 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:28.689 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:28.690 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:28.690 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.686+0000 7f6d1e19d640 1 Processor -- start 2024-09-15T01:36:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 1 -- start start 2024-09-15T01:36:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 10 monclient: init 2024-09-15T01:36:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 10 monclient: _add_conns ranks=[3,9,6,2,10,8,0,1,4,5,7] 2024-09-15T01:36:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d180a35e0 0x7f6d18099a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 10 monclient(hunting): picked mon.noname-g con 0x7f6d180a35e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6d1809c560 0x7f6d1809d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 10 monclient(hunting): picked mon.noname-a con 0x7f6d1809c560 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6d1809c950 0x7f6d1809dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 10 monclient(hunting): picked mon.noname-d con 0x7f6d1809c950 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d18099f50 con 0x7f6d1809c560 2024-09-15T01:36:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1809d340 con 0x7f6d180a35e0 2024-09-15T01:36:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6d180a39b0 con 0x7f6d1809c950 2024-09-15T01:36:28.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d1e19d640 10 monclient(hunting): authenticate will time out at 1004.330575s 2024-09-15T01:36:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d177fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d180a35e0 0x7f6d18099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d177fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d180a35e0 0x7f6d18099a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:41578/0 (socket says 172.21.15.204:41578) 2024-09-15T01:36:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d177fe640 1 -- 172.21.15.204:0/3402979959 learned_addr learned my addr 172.21.15.204:0/3402979959 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d177fe640 10 monclient(hunting): get_auth_request con 0x7f6d180a35e0 auth_method 0 2024-09-15T01:36:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d0ffff640 1 --2- 172.21.15.204:0/3402979959 >> v2:172.21.15.178:3300/0 conn(0x7f6d1809c560 0x7f6d1809d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:28.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d0ffff640 10 monclient(hunting): get_auth_request con 0x7f6d1809c560 auth_method 0 2024-09-15T01:36:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d0ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d17fff640 1 --2- 172.21.15.204:0/3402979959 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6d1809c950 0x7f6d1809dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d17fff640 10 monclient(hunting): get_auth_request con 0x7f6d1809c950 auth_method 0 2024-09-15T01:36:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d17fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d17fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d17fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:28.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d17fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d17fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.687+0000 7f6d17fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d177fe640 10 monclient(hunting): handle_auth_done global_id 15096 payload 274 2024-09-15T01:36:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d177fe640 1 -- 172.21.15.204:0/3402979959 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6d1809c950 msgr2=0x7f6d1809dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d177fe640 1 --2- 172.21.15.204:0/3402979959 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6d1809c950 0x7f6d1809dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d177fe640 1 -- 172.21.15.204:0/3402979959 >> v2:172.21.15.178:3300/0 conn(0x7f6d1809c560 msgr2=0x7f6d1809d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d177fe640 1 --2- 172.21.15.204:0/3402979959 >> v2:172.21.15.178:3300/0 conn(0x7f6d1809c560 0x7f6d1809d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d177fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:28.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d177fe640 1 monclient: found mon.noname-g 2024-09-15T01:36:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d177fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d177fe640 1 -- 172.21.15.204:0/3402979959 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1804b720 con 0x7f6d180a35e0 2024-09-15T01:36:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d177fe640 1 --2- 172.21.15.204:0/3402979959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d180a35e0 0x7f6d18099a10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800bbf0 tx=0x7f6d0800b460 comp rx=0 tx=0).ready entity=mon.3 client_cookie=24b4fdd63a368acf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d16ffd640 1 -- 172.21.15.204:0/3402979959 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6d08031990 con 0x7f6d180a35e0 2024-09-15T01:36:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d16ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d16ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d16ffd640 10 monclient: dump: 2024-09-15T01:36:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:28.705 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:28.706 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:28.707 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d16ffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d16ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d16ffd640 1 -- 172.21.15.204:0/3402979959 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0802f050 con 0x7f6d180a35e0 2024-09-15T01:36:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d1e19d640 5 monclient: authenticate success, global_id 15096 2024-09-15T01:36:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d1e19d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.688+0000 7f6d16ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d16ffd640 1 -- 172.21.15.204:0/3402979959 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6d08031b90 con 0x7f6d180a35e0 2024-09-15T01:36:28.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 4 set_mon_vals no callback set 2024-09-15T01:36:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d16ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d16ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d16ffd640 10 monclient: dump: 2024-09-15T01:36:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:28.708 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:28.709 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:28.709 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:28.709 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:28.709 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:28.709 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:28.709 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:28.709 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:28.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 10 monclient: shutdown 2024-09-15T01:36:28.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:28.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 1 -- 172.21.15.204:0/3402979959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d180a35e0 msgr2=0x7f6d18099a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:28.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 1 --2- 172.21.15.204:0/3402979959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d180a35e0 0x7f6d18099a10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800bbf0 tx=0x7f6d0800b460 comp rx=0 tx=0).stop 2024-09-15T01:36:28.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 1 -- 172.21.15.204:0/3402979959 shutdown_connections 2024-09-15T01:36:28.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 1 --2- 172.21.15.204:0/3402979959 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6d1809c950 0x7f6d1809dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 1 --2- 172.21.15.204:0/3402979959 >> v2:172.21.15.178:3300/0 conn(0x7f6d1809c560 0x7f6d1809d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 1 --2- 172.21.15.204:0/3402979959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d180a35e0 0x7f6d18099a10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 1 -- 172.21.15.204:0/3402979959 >> 172.21.15.204:0/3402979959 conn(0x7f6d180687e0 msgr2=0x7f6d180a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:28.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 1 -- 172.21.15.204:0/3402979959 shutdown_connections 2024-09-15T01:36:28.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 1 -- 172.21.15.204:0/3402979959 wait complete. 2024-09-15T01:36:28.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.689+0000 7f6d1e19d640 10 monclient: build_initial_monmap 2024-09-15T01:36:28.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 10 monclient: monmap: 2024-09-15T01:36:28.710 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:28.710 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:28.711 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:28.711 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:28.711 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:28.711 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:28.711 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:28.711 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:28.711 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:28.711 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:28.711 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:28.711 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:28.711 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:28.711 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:28.712 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:28.712 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:28.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 1 Processor -- start 2024-09-15T01:36:28.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 1 -- start start 2024-09-15T01:36:28.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 10 monclient: init 2024-09-15T01:36:28.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:28.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 10 monclient: _add_conns ranks=[6,8,4,7,2,5,3,1,0] 2024-09-15T01:36:28.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6d1809c560 0x7f6d1813e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 10 monclient(hunting): picked mon.noname-d con 0x7f6d1809c560 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:28.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6d1809c950 0x7f6d1813d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 10 monclient(hunting): picked mon.noname-e con 0x7f6d1809c950 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:28.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6d180a35e0 0x7f6d18141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 10 monclient(hunting): picked mon.noname-c con 0x7f6d180a35e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:28.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:28.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d18122d10 con 0x7f6d180a35e0 2024-09-15T01:36:28.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:28.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1806f000 con 0x7f6d1809c560 2024-09-15T01:36:28.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:28.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1806ea80 con 0x7f6d1809c950 2024-09-15T01:36:28.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:28.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.690+0000 7f6d1e19d640 10 monclient(hunting): authenticate will time out at 1004.333827s 2024-09-15T01:36:28.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d177fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6d1809c560 0x7f6d1813e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6d1809c950 0x7f6d1813d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d17fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6d180a35e0 0x7f6d18141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6d1809c950 0x7f6d1813d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:33522/0 (socket says 172.21.15.204:33522) 2024-09-15T01:36:28.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d177fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6d1809c560 0x7f6d1813e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:46424/0 (socket says 172.21.15.204:46424) 2024-09-15T01:36:28.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 1 -- 172.21.15.204:0/2215725134 learned_addr learned my addr 172.21.15.204:0/2215725134 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:28.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d17fff640 10 monclient(hunting): get_auth_request con 0x7f6d180a35e0 auth_method 0 2024-09-15T01:36:28.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d17fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:28.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d17fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:28.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d17fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:28.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 10 monclient(hunting): get_auth_request con 0x7f6d1809c950 auth_method 0 2024-09-15T01:36:28.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:28.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:28.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:28.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d177fe640 10 monclient(hunting): get_auth_request con 0x7f6d1809c560 auth_method 0 2024-09-15T01:36:28.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:28.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d177fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:28.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:28.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d17fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:28.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d17fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:28.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d17fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:28.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:28.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:28.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:28.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:28.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:28.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:28.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 10 monclient(hunting): handle_auth_done global_id 5984 payload 824 2024-09-15T01:36:28.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 1 -- 172.21.15.204:0/2215725134 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6d1809c560 msgr2=0x7f6d1813e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:28.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 1 --2- 172.21.15.204:0/2215725134 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6d1809c560 0x7f6d1813e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 1 -- 172.21.15.204:0/2215725134 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6d180a35e0 msgr2=0x7f6d18141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:28.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 1 --2- 172.21.15.204:0/2215725134 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6d180a35e0 0x7f6d18141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:28.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 1 monclient: found mon.noname-e 2024-09-15T01:36:28.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:28.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:28.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 1 -- 172.21.15.204:0/2215725134 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0800b700 con 0x7f6d1809c950 2024-09-15T01:36:28.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d17fff640 1 --2- 172.21.15.204:0/2215725134 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6d180a35e0 0x7f6d18141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:28.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0ffff640 1 --2- 172.21.15.204:0/2215725134 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6d1809c950 0x7f6d1813d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009fd0 tx=0x7f6d000090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:28.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0f7fe640 1 -- 172.21.15.204:0/2215725134 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6d0000cdc0 con 0x7f6d1809c950 2024-09-15T01:36:28.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:28.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:28.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0f7fe640 10 monclient: dump: 2024-09-15T01:36:28.717 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:28.717 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:28.718 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.718 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.718 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:28.718 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:28.718 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:28.718 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:28.718 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:28.718 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:28.718 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:28.718 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:28.718 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:28.718 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:28.719 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:28.719 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:28.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:28.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:28.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0f7fe640 1 -- 172.21.15.204:0/2215725134 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d00013470 con 0x7f6d1809c950 2024-09-15T01:36:28.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d1e19d640 5 monclient: authenticate success, global_id 5984 2024-09-15T01:36:28.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d1e19d640 10 monclient: _renew_subs 2024-09-15T01:36:28.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d1e19d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:28.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d1e19d640 1 -- 172.21.15.204:0/2215725134 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1813eeb0 con 0x7f6d1809c950 2024-09-15T01:36:28.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:28.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0f7fe640 1 -- 172.21.15.204:0/2215725134 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6d00013a60 con 0x7f6d1809c950 2024-09-15T01:36:28.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:28.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:28.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d0f7fe640 10 monclient: dump: 2024-09-15T01:36:28.720 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:28.720 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:28.720 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.720 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.720 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:28.720 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:28.720 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:28.720 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:28.721 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:28.721 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:28.721 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:28.721 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:28.721 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:28.721 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:28.721 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:28.721 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:28.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d167fc640 4 set_mon_vals no callback set 2024-09-15T01:36:28.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d1e19d640 10 monclient: _renew_subs 2024-09-15T01:36:28.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d1e19d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:28.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.691+0000 7f6d1e19d640 1 -- 172.21.15.204:0/2215725134 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1813ded0 con 0x7f6d1809c950 2024-09-15T01:36:28.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.692+0000 7f6d0f7fe640 1 -- 172.21.15.204:0/2215725134 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6d00013bc0 con 0x7f6d1809c950 2024-09-15T01:36:28.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.692+0000 7f6d0f7fe640 1 --2- 172.21.15.204:0/2215725134 >> v2:172.21.15.204:6812/28210 conn(0x7f6cf80397b0 0x7f6cf803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.692+0000 7f6d1e19d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:28.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.692+0000 7f6d1e19d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:28.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.692+0000 7f6d1e19d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:28.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.692+0000 7f6d0f7fe640 1 -- 172.21.15.204:0/2215725134 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6959+0+0 (secure 0 0 0) 0x7f6d0004bfd0 con 0x7f6d1809c950 2024-09-15T01:36:28.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.692+0000 7f6d1e19d640 1 -- 172.21.15.204:0/2215725134 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d18123050 con 0x7f6d1809c950 2024-09-15T01:36:28.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.692+0000 7f6d177fe640 1 --2- 172.21.15.204:0/2215725134 >> v2:172.21.15.204:6812/28210 conn(0x7f6cf80397b0 0x7f6cf803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.692+0000 7f6d177fe640 10 monclient: get_auth_request con 0x7f6cf80397b0 auth_method 0 2024-09-15T01:36:28.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.693+0000 7f6d177fe640 1 --2- 172.21.15.204:0/2215725134 >> v2:172.21.15.204:6812/28210 conn(0x7f6cf80397b0 0x7f6cf803bc60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6d08036000 tx=0x7f6d08039000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:28.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.697+0000 7f6d0f7fe640 1 -- 172.21.15.204:0/2215725134 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6d18123050 con 0x7f6d1809c950 2024-09-15T01:36:28.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.697+0000 7f6d0f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:28.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.697+0000 7f6d0f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:28.776 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:36:28.777 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:36:28.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.858+0000 7f6d1e19d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-09-15T01:36:28.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.858+0000 7f6d1e19d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-09-15T01:36:28.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.858+0000 7f6d1e19d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:28.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:28.858+0000 7f6d1e19d640 1 -- 172.21.15.204:0/2215725134 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"} v 0) v1 -- 0x7f6d18123050 con 0x7f6d1809c950 2024-09-15T01:36:28.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.872+0000 7f4a5975d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.872+0000 7f4a5975d640 10 monclient: build_initial_monmap 2024-09-15T01:36:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.872+0000 7f4a5975d640 10 monclient: monmap: 2024-09-15T01:36:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:28.873080+0000 2024-09-15T01:36:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:28.873080+0000 2024-09-15T01:36:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:28.875 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:28.875 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:28.875 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:28.875 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:28.875 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:28.875 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:28.875 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:28.875 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:28.875 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:28.876 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:28.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.872+0000 7f4a5975d640 1 Processor -- start 2024-09-15T01:36:28.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.872+0000 7f4a5975d640 1 -- start start 2024-09-15T01:36:28.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.872+0000 7f4a5975d640 10 monclient: init 2024-09-15T01:36:28.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:28.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 10 monclient: _add_conns ranks=[9,10,2,1,4,5,3,8,0,7,6] 2024-09-15T01:36:28.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4a54067130 0x7f4a54067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 10 monclient(hunting): picked mon.noname-a con 0x7f4a54067130 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:28.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4a5409d580 0x7f4a54067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 10 monclient(hunting): picked mon.noname-f con 0x7f4a5409d580 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:28.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a5409f000 0x7f4a5412d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 10 monclient(hunting): picked mon.noname-b con 0x7f4a5409f000 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:28.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:28.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5409e0c0 con 0x7f4a54067130 2024-09-15T01:36:28.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:28.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a54067f80 con 0x7f4a5409d580 2024-09-15T01:36:28.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:28.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5409dde0 con 0x7f4a5409f000 2024-09-15T01:36:28.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:28.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a5975d640 10 monclient(hunting): authenticate will time out at 993.623440s 2024-09-15T01:36:28.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a52ffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4a54067130 0x7f4a54067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a5409f000 0x7f4a5412d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a52ffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4a54067130 0x7f4a54067500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53890/0 (socket says 172.21.15.178:53890) 2024-09-15T01:36:28.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a52ffd640 1 -- 172.21.15.178:0/437799906 learned_addr learned my addr 172.21.15.178:0/437799906 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:28.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a52ffd640 10 monclient(hunting): get_auth_request con 0x7f4a54067130 auth_method 0 2024-09-15T01:36:28.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:28.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a52ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:28.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a5409f000 0x7f4a5412d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:60196/0 (socket says 172.21.15.178:60196) 2024-09-15T01:36:28.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:28.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 10 monclient(hunting): get_auth_request con 0x7f4a5409f000 auth_method 0 2024-09-15T01:36:28.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:28.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:28.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:28.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:28.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:28.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:28.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:28.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:28.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:28.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a4a7fc640 1 --2- 172.21.15.178:0/437799906 >> v2:172.21.15.204:3300/0 conn(0x7f4a5409d580 0x7f4a54067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a4a7fc640 10 monclient(hunting): get_auth_request con 0x7f4a5409d580 auth_method 0 2024-09-15T01:36:28.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a4a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:28.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:28.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:28.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 10 monclient(hunting): handle_auth_done global_id 24212 payload 274 2024-09-15T01:36:28.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 1 -- 172.21.15.178:0/437799906 >> v2:172.21.15.204:3300/0 conn(0x7f4a5409d580 msgr2=0x7f4a54067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:28.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 1 --2- 172.21.15.178:0/437799906 >> v2:172.21.15.204:3300/0 conn(0x7f4a5409d580 0x7f4a54067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 1 -- 172.21.15.178:0/437799906 >> v2:172.21.15.178:3300/0 conn(0x7f4a54067130 msgr2=0x7f4a54067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:28.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 1 --2- 172.21.15.178:0/437799906 >> v2:172.21.15.178:3300/0 conn(0x7f4a54067130 0x7f4a54067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:28.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 1 monclient: found mon.noname-b 2024-09-15T01:36:28.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:28.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:36:28.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a537fe640 1 -- 172.21.15.178:0/437799906 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5404bf60 con 0x7f4a5409f000 2024-09-15T01:36:28.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.873+0000 7f4a52ffd640 1 --2- 172.21.15.178:0/437799906 >> v2:172.21.15.178:3300/0 conn(0x7f4a54067130 0x7f4a54067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:28.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a537fe640 1 --2- 172.21.15.178:0/437799906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a5409f000 0x7f4a5412d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4a540a3520 tx=0x7f4a4c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a253ae973e8a7058 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:28.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a527fc640 1 -- 172.21.15.178:0/437799906 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a4c0079d0 con 0x7f4a5409f000 2024-09-15T01:36:28.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:28.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a527fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:36:28.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a527fc640 10 monclient: dump: 2024-09-15T01:36:28.891 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:28.891 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:28.891 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.891 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.891 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:28.891 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:28.891 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:28.891 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:28.891 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:28.891 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:28.891 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:28.891 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a527fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a527fc640 1 -- 172.21.15.178:0/437799906 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4c02d050 con 0x7f4a5409f000 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a5975d640 5 monclient: authenticate success, global_id 24212 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a5975d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a527fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a527fc640 1 -- 172.21.15.178:0/437799906 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a4c007bd0 con 0x7f4a5409f000 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:28.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a527fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:28.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a527fc640 10 monclient: dump: 2024-09-15T01:36:28.893 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:28.893 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:28.893 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.893 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.893 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:28.894 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:28.894 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:28.894 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:28.894 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:28.894 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:28.894 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:28.895 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:28.895 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:28.895 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:28.895 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:28.895 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:28.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a5975d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:28.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.874+0000 7f4a5975d640 4 set_mon_vals no callback set 2024-09-15T01:36:28.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 10 monclient: shutdown 2024-09-15T01:36:28.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:28.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 1 -- 172.21.15.178:0/437799906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a5409f000 msgr2=0x7f4a5412d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:28.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 1 --2- 172.21.15.178:0/437799906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a5409f000 0x7f4a5412d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4a540a3520 tx=0x7f4a4c009780 comp rx=0 tx=0).stop 2024-09-15T01:36:28.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 1 -- 172.21.15.178:0/437799906 shutdown_connections 2024-09-15T01:36:28.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 1 --2- 172.21.15.178:0/437799906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a5409f000 0x7f4a5412d330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 1 --2- 172.21.15.178:0/437799906 >> v2:172.21.15.204:3300/0 conn(0x7f4a5409d580 0x7f4a54067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 1 --2- 172.21.15.178:0/437799906 >> v2:172.21.15.178:3300/0 conn(0x7f4a54067130 0x7f4a54067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 1 -- 172.21.15.178:0/437799906 >> 172.21.15.178:0/437799906 conn(0x7f4a54068a60 msgr2=0x7f4a540a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:28.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 1 -- 172.21.15.178:0/437799906 shutdown_connections 2024-09-15T01:36:28.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 1 -- 172.21.15.178:0/437799906 wait complete. 2024-09-15T01:36:28.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 10 monclient: build_initial_monmap 2024-09-15T01:36:28.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.875+0000 7f4a5975d640 10 monclient: monmap: 2024-09-15T01:36:28.896 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:28.897 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 1 Processor -- start 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 1 -- start start 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 10 monclient: init 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 10 monclient: _add_conns ranks=[6,0,3,2,7,4,5,1,8] 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a54067130 0x7f4a5413fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 10 monclient(hunting): picked mon.noname-d con 0x7f4a54067130 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a5409d580 0x7f4a5413ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 10 monclient(hunting): picked mon.noname-a con 0x7f4a5409d580 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a5409f000 0x7f4a54140490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 10 monclient(hunting): picked mon.noname-g con 0x7f4a5409f000 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:28.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5411ca90 con 0x7f4a5409d580 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5411a9a0 con 0x7f4a5409f000 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5409cb50 con 0x7f4a54067130 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a5975d640 10 monclient(hunting): authenticate will time out at 993.626831s 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a4a7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a5409d580 0x7f4a5413ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a52ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a54067130 0x7f4a5413fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a4a7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a5409d580 0x7f4a5413ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53896/0 (socket says 172.21.15.178:53896) 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a4a7fc640 1 -- 172.21.15.178:0/1114274754 learned_addr learned my addr 172.21.15.178:0/1114274754 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a4a7fc640 10 monclient(hunting): get_auth_request con 0x7f4a5409d580 auth_method 0 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a52ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a54067130 0x7f4a5413fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:56260/0 (socket says 172.21.15.178:56260) 2024-09-15T01:36:28.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a4a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:28.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:28.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:28.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a52ffd640 10 monclient(hunting): get_auth_request con 0x7f4a54067130 auth_method 0 2024-09-15T01:36:28.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:28.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a52ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:28.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.876+0000 7f4a52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:28.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:28.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:28.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a537fe640 1 --2- 172.21.15.178:0/1114274754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a5409f000 0x7f4a54140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a537fe640 10 monclient(hunting): get_auth_request con 0x7f4a5409f000 auth_method 0 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 10 monclient(hunting): handle_auth_done global_id 6246 payload 824 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 1 -- 172.21.15.178:0/1114274754 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a54067130 msgr2=0x7f4a5413fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:28.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 1 --2- 172.21.15.178:0/1114274754 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a54067130 0x7f4a5413fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 1 -- 172.21.15.178:0/1114274754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a5409f000 msgr2=0x7f4a54140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 1 --2- 172.21.15.178:0/1114274754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a5409f000 0x7f4a54140490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 1 monclient: found mon.noname-a 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 1 -- 172.21.15.178:0/1114274754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a44009450 con 0x7f4a5409d580 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a52ffd640 1 --2- 172.21.15.178:0/1114274754 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a54067130 0x7f4a5413fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4a7fc640 1 --2- 172.21.15.178:0/1114274754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a5409d580 0x7f4a5413ff50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00d8c0 tx=0x7f4a3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4b7fe640 1 -- 172.21.15.178:0/1114274754 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a3c00de50 con 0x7f4a5409d580 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:36:28.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4b7fe640 10 monclient: dump: 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4b7fe640 1 -- 172.21.15.178:0/1114274754 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c00bcf0 con 0x7f4a5409d580 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a5975d640 5 monclient: authenticate success, global_id 6246 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4b7fe640 1 -- 172.21.15.178:0/1114274754 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a3c00be60 con 0x7f4a5409d580 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a5975d640 10 monclient: _renew_subs 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a5975d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a5975d640 1 -- 172.21.15.178:0/1114274754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c009a20 con 0x7f4a5409d580 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:28.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a51ffb640 4 set_mon_vals no callback set 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a4b7fe640 10 monclient: dump: 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:28.905 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a5975d640 10 monclient: _renew_subs 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a5975d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.877+0000 7f4a5975d640 1 -- 172.21.15.178:0/1114274754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5411ae90 con 0x7f4a5409d580 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.878+0000 7f4a4b7fe640 1 -- 172.21.15.178:0/1114274754 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4a3c00b870 con 0x7f4a5409d580 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.878+0000 7f4a5975d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.878+0000 7f4a5975d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.878+0000 7f4a5975d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.878+0000 7f4a5975d640 1 -- 172.21.15.178:0/1114274754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5404b720 con 0x7f4a5409d580 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.878+0000 7f4a4b7fe640 1 --2- 172.21.15.178:0/1114274754 >> v2:172.21.15.204:6812/28210 conn(0x7f4a340397b0 0x7f4a3403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.878+0000 7f4a4b7fe640 1 -- 172.21.15.178:0/1114274754 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6959+0+0 (secure 0 0 0) 0x7f4a3c04c190 con 0x7f4a5409d580 2024-09-15T01:36:28.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.878+0000 7f4a52ffd640 1 --2- 172.21.15.178:0/1114274754 >> v2:172.21.15.204:6812/28210 conn(0x7f4a340397b0 0x7f4a3403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:28.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.878+0000 7f4a52ffd640 10 monclient: get_auth_request con 0x7f4a340397b0 auth_method 0 2024-09-15T01:36:28.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.879+0000 7f4a52ffd640 1 --2- 172.21.15.178:0/1114274754 >> v2:172.21.15.204:6812/28210 conn(0x7f4a340397b0 0x7f4a3403bc60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4a44006c90 tx=0x7f4a440091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:28.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.883+0000 7f4a4b7fe640 1 -- 172.21.15.178:0/1114274754 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4a3c00bb20 con 0x7f4a5409d580 2024-09-15T01:36:28.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.883+0000 7f4a4b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:28.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:28.883+0000 7f4a4b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:29.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.093+0000 7f4a5975d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:36:29.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.093+0000 7f4a5975d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:29.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.093+0000 7f4a5975d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:29.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.093+0000 7f4a5975d640 1 -- 172.21.15.178:0/1114274754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4a54070770 con 0x7f4a5409d580 2024-09-15T01:36:29.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.094+0000 7f4a4b7fe640 1 -- 172.21.15.178:0/1114274754 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7f4a3c01c360 con 0x7f4a5409d580 2024-09-15T01:36:29.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.094+0000 7f4a4b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:29.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.094+0000 7f4a4b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:29.097 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":5,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:36:29.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 1 -- 172.21.15.178:0/1114274754 >> v2:172.21.15.204:6812/28210 conn(0x7f4a340397b0 msgr2=0x7f4a3403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 1 --2- 172.21.15.178:0/1114274754 >> v2:172.21.15.204:6812/28210 conn(0x7f4a340397b0 0x7f4a3403bc60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4a44006c90 tx=0x7f4a440091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 10 monclient: shutdown 2024-09-15T01:36:29.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 1 -- 172.21.15.178:0/1114274754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a5409d580 msgr2=0x7f4a5413ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 1 --2- 172.21.15.178:0/1114274754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a5409d580 0x7f4a5413ff50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00d8c0 tx=0x7f4a3c008040 comp rx=0 tx=0).stop 2024-09-15T01:36:29.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 1 -- 172.21.15.178:0/1114274754 shutdown_connections 2024-09-15T01:36:29.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 1 --2- 172.21.15.178:0/1114274754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a5409f000 0x7f4a54140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 1 --2- 172.21.15.178:0/1114274754 >> v2:172.21.15.204:6812/28210 conn(0x7f4a340397b0 0x7f4a3403bc60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 1 --2- 172.21.15.178:0/1114274754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a5409d580 0x7f4a5413ff50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 1 --2- 172.21.15.178:0/1114274754 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4a54067130 0x7f4a5413fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 1 -- 172.21.15.178:0/1114274754 >> 172.21.15.178:0/1114274754 conn(0x7f4a54068a60 msgr2=0x7f4a54131e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 1 -- 172.21.15.178:0/1114274754 shutdown_connections 2024-09-15T01:36:29.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.096+0000 7f4a5975d640 1 -- 172.21.15.178:0/1114274754 wait complete. 2024-09-15T01:36:29.108 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:36:29.108 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T01:36:29.108 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:36:29.108 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:36:29.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.199+0000 7feee0c60640 10 monclient: get_monmap_and_config 2024-09-15T01:36:29.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.199+0000 7feee0c60640 10 monclient: build_initial_monmap 2024-09-15T01:36:29.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.199+0000 7feee0c60640 10 monclient: monmap: 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:29.200474+0000 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:29.200474+0000 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:29.203 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:29.203 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:29.203 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:29.203 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:29.203 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 1 Processor -- start 2024-09-15T01:36:29.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 1 -- start start 2024-09-15T01:36:29.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 10 monclient: init 2024-09-15T01:36:29.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:29.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 10 monclient: _add_conns ranks=[6,0,9,3,7,8,5,10,1,2,4] 2024-09-15T01:36:29.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7feedc06e7c0 0x7feedc1305b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 10 monclient(hunting): picked mon.noname-d con 0x7feedc06e7c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feedc130af0 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feedc130ee0 0x7feedc09d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 10 monclient(hunting): picked mon.noname-a con 0x7feedc130ee0 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feedc04bf60 con 0x7feedc130af0 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc06e2c0 con 0x7feedc130ee0 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feedc06dd00 con 0x7feedc06e7c0 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feee0c60640 10 monclient(hunting): authenticate will time out at 993.950851s 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feeda59c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7feedc06e7c0 0x7feedc1305b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.200+0000 7feedad9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feedc130ee0 0x7feedc09d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feeda59c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7feedc06e7c0 0x7feedc1305b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:56276/0 (socket says 172.21.15.178:56276) 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feeda59c640 1 -- 172.21.15.178:0/3860854743 learned_addr learned my addr 172.21.15.178:0/3860854743 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feedc130ee0 0x7feedc09d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53908/0 (socket says 172.21.15.178:53908) 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feeda59c640 10 monclient(hunting): get_auth_request con 0x7feedc06e7c0 auth_method 0 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feeda59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feeda59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feeda59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 10 monclient(hunting): get_auth_request con 0x7feedc130ee0 auth_method 0 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feeda59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feeda59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feeda59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 10 monclient(hunting): handle_auth_done global_id 6255 payload 274 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 1 -- 172.21.15.178:0/3860854743 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7feedc06e7c0 msgr2=0x7feedc1305b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 1 --2- 172.21.15.178:0/3860854743 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7feedc06e7c0 0x7feedc1305b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 1 -- 172.21.15.178:0/3860854743 >> v1:172.21.15.178:6789/0 conn(0x7feedc130af0 legacy=0x7feedc09cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 1 monclient: found mon.noname-a 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feedad9d640 1 -- 172.21.15.178:0/3860854743 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc06eb90 con 0x7feedc130ee0 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.201+0000 7feeda59c640 1 --2- 172.21.15.178:0/3860854743 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7feedc06e7c0 0x7feedc1305b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feedad9d640 1 --2- 172.21.15.178:0/3860854743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc130ee0 0x7feedc09d1d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feed4009e60 tx=0x7feed4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177215f9a839a1f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feed9d9b640 1 -- 172.21.15.178:0/3860854743 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feed40079d0 con 0x7feedc130ee0 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feed9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feed9d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feed9d9b640 10 monclient: dump: 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feed9d9b640 10 monclient: _finish_auth 0 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feed9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feed9d9b640 1 -- 172.21.15.178:0/3860854743 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed402d050 con 0x7feedc130ee0 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 5 monclient: authenticate success, global_id 6255 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feed9d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feed9d9b640 1 -- 172.21.15.178:0/3860854743 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feed4007bd0 con 0x7feedc130ee0 2024-09-15T01:36:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feed9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 4 set_mon_vals no callback set 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feed9d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feed9d9b640 10 monclient: dump: 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.218 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 10 monclient: shutdown 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 1 -- 172.21.15.178:0/3860854743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc130ee0 msgr2=0x7feedc09d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 1 --2- 172.21.15.178:0/3860854743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc130ee0 0x7feedc09d1d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feed4009e60 tx=0x7feed4009780 comp rx=0 tx=0).stop 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 1 -- 172.21.15.178:0/3860854743 shutdown_connections 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 1 --2- 172.21.15.178:0/3860854743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc130ee0 0x7feedc09d1d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 1 --2- 172.21.15.178:0/3860854743 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7feedc06e7c0 0x7feedc1305b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 1 -- 172.21.15.178:0/3860854743 >> 172.21.15.178:0/3860854743 conn(0x7feedc067d20 msgr2=0x7feedc0a1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 1 -- 172.21.15.178:0/3860854743 shutdown_connections 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.202+0000 7feee0c60640 1 -- 172.21.15.178:0/3860854743 wait complete. 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.203+0000 7feee0c60640 10 monclient: build_initial_monmap 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.203+0000 7feee0c60640 10 monclient: monmap: 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.203+0000 7feee0c60640 1 Processor -- start 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.203+0000 7feee0c60640 1 -- start start 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.203+0000 7feee0c60640 10 monclient: init 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.203+0000 7feee0c60640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.203+0000 7feee0c60640 10 monclient: _add_conns ranks=[0,4,3,8,2,6,5,1,7] 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.203+0000 7feee0c60640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc06e7c0 0x7feedc12f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 10 monclient(hunting): picked mon.noname-a con 0x7feedc06e7c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feedc130af0 0x7feedc118fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 10 monclient(hunting): picked mon.noname-c con 0x7feedc130af0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feedc130ee0 0x7feedc119520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 10 monclient(hunting): picked mon.noname-g con 0x7feedc130ee0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc06d120 con 0x7feedc06e7c0 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc06cde0 con 0x7feedc130ee0 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feedc140720 con 0x7feedc130af0 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 10 monclient(hunting): authenticate will time out at 993.954050s 2024-09-15T01:36:29.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc06e7c0 0x7feedc12f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc06e7c0 0x7feedc12f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53918/0 (socket says 172.21.15.178:53918) 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 1 -- 172.21.15.178:0/2329104356 learned_addr learned my addr 172.21.15.178:0/2329104356 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 10 monclient(hunting): get_auth_request con 0x7feedc06e7c0 auth_method 0 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed3fff640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feedc130af0 0x7feedc118fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed3fff640 10 monclient(hunting): get_auth_request con 0x7feedc130af0 auth_method 0 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feedad9d640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feedc130ee0 0x7feedc119520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feedad9d640 10 monclient(hunting): get_auth_request con 0x7feedc130ee0 auth_method 0 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feedad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feedad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feedad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 10 monclient(hunting): handle_auth_done global_id 6273 payload 824 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 1 -- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feedc130af0 msgr2=0x7feedc118fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7feedc130af0 0x7feedc118fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 1 -- 172.21.15.178:0/2329104356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feedc130ee0 msgr2=0x7feedc119520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feedc130ee0 0x7feedc119520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 1 monclient: found mon.noname-a 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 1 -- 172.21.15.178:0/2329104356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc009450 con 0x7feedc06e7c0 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feeda59c640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc06e7c0 0x7feedc12f480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feecc00ec80 tx=0x7feecc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feecc009730 con 0x7feedc06e7c0 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed27fc640 10 monclient: dump: 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed27fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 5 monclient: authenticate success, global_id 6273 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc00eec0 con 0x7feedc06e7c0 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 10 monclient: _renew_subs 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 1 -- 172.21.15.178:0/2329104356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed4009a20 con 0x7feedc06e7c0 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feecc013d70 con 0x7feedc06e7c0 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed959a640 4 set_mon_vals no callback set 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feed27fc640 10 monclient: dump: 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.227 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 10 monclient: _renew_subs 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.204+0000 7feee0c60640 1 -- 172.21.15.178:0/2329104356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc11c5a0 con 0x7feedc06e7c0 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feee0c60640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7feecc00cbc0 con 0x7feedc06e7c0 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feee0c60640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc12f9c0 0x7feedc119a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feee0c60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feee0c60640 1 -- 172.21.15.178:0/2329104356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc064250 con 0x7feedc12f9c0 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed3fff640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc12f9c0 0x7feedc119a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed3fff640 10 monclient: get_auth_request con 0x7feedc12f9c0 auth_method 0 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed27fc640 1 --2- 172.21.15.178:0/2329104356 >> v2:172.21.15.204:6812/28210 conn(0x7feebc0397b0 0x7feebc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6959+0+0 (secure 0 0 0) 0x7feecc04c220 con 0x7feedc06e7c0 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed3fff640 10 monclient(hunting): handle_auth_done global_id 6282 payload 824 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.205+0000 7feed3fff640 1 -- 172.21.15.178:0/2329104356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feedc04c4a0 con 0x7feedc12f9c0 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feedad9d640 1 --2- 172.21.15.178:0/2329104356 >> v2:172.21.15.204:6812/28210 conn(0x7feebc0397b0 0x7feebc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feedad9d640 10 monclient: get_auth_request con 0x7feebc0397b0 auth_method 0 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feed3fff640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc12f9c0 0x7feedc119a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feec40027a0 tx=0x7feec4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feec400fd60 con 0x7feedc12f9c0 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feed27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feed27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feed27fc640 10 monclient: dump: 2024-09-15T01:36:29.230 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.231 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.232 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feedad9d640 1 --2- 172.21.15.178:0/2329104356 >> v2:172.21.15.204:6812/28210 conn(0x7feebc0397b0 0x7feebc03bc60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feed4034000 tx=0x7feed4037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7feec400fec0 con 0x7feedc12f9c0 2024-09-15T01:36:29.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feed27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feed27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:29.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc12f9c0 msgr2=0x7feedc119a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feed27fc640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc12f9c0 0x7feedc119a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feec40027a0 tx=0x7feec4000bc0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc12f9c0 msgr2=0x7feedc119a60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:29.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.206+0000 7feed27fc640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc12f9c0 0x7feedc119a60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feee0c60640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:29.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feee0c60640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc119fa0 0x7feedc097060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feee0c60640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feee0c60640 1 -- 172.21.15.178:0/2329104356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc12fd90 con 0x7feedc119fa0 2024-09-15T01:36:29.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed3fff640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc119fa0 0x7feedc097060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed3fff640 10 monclient: get_auth_request con 0x7feedc119fa0 auth_method 0 2024-09-15T01:36:29.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed3fff640 10 monclient(hunting): handle_auth_done global_id 6291 payload 824 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed3fff640 1 -- 172.21.15.178:0/2329104356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7feedc04bc80 con 0x7feedc119fa0 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed3fff640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc119fa0 0x7feedc097060 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feec40027a0 tx=0x7feec4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feec4013aa0 con 0x7feedc119fa0 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.212+0000 7feed27fc640 10 monclient: dump: 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.235 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7feec4025070 con 0x7feedc119fa0 2024-09-15T01:36:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feed27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:29.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feed27fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:29.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc119fa0 msgr2=0x7feedc097060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feed27fc640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc119fa0 0x7feedc097060 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feec40027a0 tx=0x7feec4014000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feed27fc640 1 -- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc119fa0 msgr2=0x7feedc097060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:29.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feed27fc640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc119fa0 0x7feedc097060 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feec40027a0 tx=0x7feec4014000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.237 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:29.237 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:29.238 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 5, 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.239 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.240 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.240 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:29.240 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:29.240 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.240 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.240 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.240 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.240 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.240 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.240 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.240 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.240 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:29.241 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.242 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:29.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:29.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.249 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:29.251 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.254 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:29.256 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:29.258 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:29.259 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.259 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.259 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:29.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.259 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:29.259 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:29.259 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:36:29.259 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.259 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.259 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:29.259 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:29.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feee0c60640 1 -- 172.21.15.178:0/2329104356 >> v2:172.21.15.204:6812/28210 conn(0x7feebc0397b0 msgr2=0x7feebc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feee0c60640 1 --2- 172.21.15.178:0/2329104356 >> v2:172.21.15.204:6812/28210 conn(0x7feebc0397b0 0x7feebc03bc60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feed4034000 tx=0x7feed4037000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feee0c60640 10 monclient: shutdown 2024-09-15T01:36:29.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feee0c60640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feee0c60640 1 -- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc06e7c0 msgr2=0x7feedc12f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feee0c60640 1 --2- 172.21.15.178:0/2329104356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feedc06e7c0 0x7feedc12f480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feecc00ec80 tx=0x7feecc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feedad9d640 1 -- 172.21.15.178:0/2329104356 reap_dead start 2024-09-15T01:36:29.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feedad9d640 1 -- 172.21.15.178:0/2329104356 reap_dead start 2024-09-15T01:36:29.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feee0c60640 1 -- 172.21.15.178:0/2329104356 shutdown_connections 2024-09-15T01:36:29.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feee0c60640 1 -- 172.21.15.178:0/2329104356 >> 172.21.15.178:0/2329104356 conn(0x7feedc067d20 msgr2=0x7feedc09c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feee0c60640 1 -- 172.21.15.178:0/2329104356 shutdown_connections 2024-09-15T01:36:29.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.213+0000 7feee0c60640 1 -- 172.21.15.178:0/2329104356 wait complete. 2024-09-15T01:36:29.262 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:29.262 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:36:29.262 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:36:29.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.353+0000 7f945569f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:29.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.353+0000 7f945569f640 10 monclient: build_initial_monmap 2024-09-15T01:36:29.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.353+0000 7f945569f640 10 monclient: monmap: 2024-09-15T01:36:29.356 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:29.356 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.356 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:29.354581+0000 2024-09-15T01:36:29.356 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:29.354581+0000 2024-09-15T01:36:29.356 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:29.357 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.357 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:29.357 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:29.357 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:29.357 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:29.357 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:29.357 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:29.357 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:29.357 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:29.357 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:29.358 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:29.358 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:29.358 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 1 Processor -- start 2024-09-15T01:36:29.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 1 -- start start 2024-09-15T01:36:29.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 10 monclient: init 2024-09-15T01:36:29.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:29.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 10 monclient: _add_conns ranks=[3,4,7,9,8,10,6,0,5,2,1] 2024-09-15T01:36:29.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94500a2930 0x7f9450132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 10 monclient(hunting): picked mon.noname-g con 0x7f94500a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:29.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f945009faa0 0x7f945009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 10 monclient(hunting): picked mon.noname-c con 0x7f945009faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:29.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f94500a08c0 0x7f94500a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 10 monclient(hunting): picked mon.noname-i con 0x7f94500a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:29.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945009e0f0 con 0x7f94500a2930 2024-09-15T01:36:29.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f945009dd80 con 0x7f945009faa0 2024-09-15T01:36:29.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f94500a2d00 con 0x7f94500a08c0 2024-09-15T01:36:29.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:29.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f945569f640 10 monclient(hunting): authenticate will time out at 994.104839s 2024-09-15T01:36:29.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.354+0000 7f944e7fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f945009faa0 0x7f945009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f945009faa0 0x7f945009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:48440/0 (socket says 172.21.15.178:48440) 2024-09-15T01:36:29.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 1 -- 172.21.15.178:0/1592745855 learned_addr learned my addr 172.21.15.178:0/1592745855 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:29.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 10 monclient(hunting): get_auth_request con 0x7f945009faa0 auth_method 0 2024-09-15T01:36:29.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944f7fe640 1 --2- 172.21.15.178:0/1592745855 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f94500a08c0 0x7f94500a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944effd640 1 --2- 172.21.15.178:0/1592745855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94500a2930 0x7f9450132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944f7fe640 10 monclient(hunting): get_auth_request con 0x7f94500a08c0 auth_method 0 2024-09-15T01:36:29.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944effd640 10 monclient(hunting): get_auth_request con 0x7f94500a2930 auth_method 0 2024-09-15T01:36:29.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 10 monclient(hunting): handle_auth_done global_id 5935 payload 274 2024-09-15T01:36:29.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 1 -- 172.21.15.178:0/1592745855 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f94500a08c0 msgr2=0x7f94500a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:29.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 1 --2- 172.21.15.178:0/1592745855 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f94500a08c0 0x7f94500a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 1 -- 172.21.15.178:0/1592745855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94500a2930 msgr2=0x7f9450132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:29.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 1 --2- 172.21.15.178:0/1592745855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94500a2930 0x7f9450132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:36:29.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 1 monclient: found mon.noname-c 2024-09-15T01:36:29.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:29.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:29.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 1 -- 172.21.15.178:0/1592745855 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945004bf60 con 0x7f945009faa0 2024-09-15T01:36:29.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.355+0000 7f944e7fc640 1 --2- 172.21.15.178:0/1592745855 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f945009faa0 0x7f945009fe70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9444009940 tx=0x7f94440091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c6684af83885efb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f944dffb640 1 -- 172.21.15.178:0/1592745855 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f944402ac60 con 0x7f945009faa0 2024-09-15T01:36:29.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f944dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f944dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:29.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f944dffb640 10 monclient: dump: 2024-09-15T01:36:29.375 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.375 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.376 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.376 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.376 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.376 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.376 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.376 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.376 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.376 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.376 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.376 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.376 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.377 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.377 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.377 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f944dffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:29.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f944dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:29.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 5 monclient: authenticate success, global_id 5935 2024-09-15T01:36:29.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f944dffb640 1 -- 172.21.15.178:0/1592745855 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944402ade0 con 0x7f945009faa0 2024-09-15T01:36:29.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f944dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:29.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:29.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f944dffb640 1 -- 172.21.15.178:0/1592745855 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9444031ca0 con 0x7f945009faa0 2024-09-15T01:36:29.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f944dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 4 set_mon_vals no callback set 2024-09-15T01:36:29.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f944dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:29.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f944dffb640 10 monclient: dump: 2024-09-15T01:36:29.378 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.378 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.378 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.378 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.378 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.378 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.378 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.378 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.379 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.379 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.379 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.379 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.379 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.379 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.379 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.379 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 10 monclient: shutdown 2024-09-15T01:36:29.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 1 -- 172.21.15.178:0/1592745855 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f945009faa0 msgr2=0x7f945009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 1 --2- 172.21.15.178:0/1592745855 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f945009faa0 0x7f945009fe70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9444009940 tx=0x7f94440091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 1 -- 172.21.15.178:0/1592745855 shutdown_connections 2024-09-15T01:36:29.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 1 --2- 172.21.15.178:0/1592745855 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f94500a08c0 0x7f94500a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 1 --2- 172.21.15.178:0/1592745855 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f945009faa0 0x7f945009fe70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 1 --2- 172.21.15.178:0/1592745855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94500a2930 0x7f9450132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 1 -- 172.21.15.178:0/1592745855 >> 172.21.15.178:0/1592745855 conn(0x7f94500a5ed0 msgr2=0x7f94500a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 1 -- 172.21.15.178:0/1592745855 shutdown_connections 2024-09-15T01:36:29.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.356+0000 7f945569f640 1 -- 172.21.15.178:0/1592745855 wait complete. 2024-09-15T01:36:29.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.357+0000 7f945569f640 10 monclient: build_initial_monmap 2024-09-15T01:36:29.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.357+0000 7f945569f640 10 monclient: monmap: 2024-09-15T01:36:29.381 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:29.381 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:29.381 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:29.381 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:29.381 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:29.381 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:29.381 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:29.381 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:29.381 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:29.381 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:29.381 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:29.382 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:29.382 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:29.382 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:29.382 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:29.382 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.357+0000 7f945569f640 1 Processor -- start 2024-09-15T01:36:29.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.357+0000 7f945569f640 1 -- start start 2024-09-15T01:36:29.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.357+0000 7f945569f640 10 monclient: init 2024-09-15T01:36:29.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:29.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 10 monclient: _add_conns ranks=[1,3,5,7,8,2,6,0,4] 2024-09-15T01:36:29.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f945009faa0 0x7f945011c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 10 monclient(hunting): picked mon.noname-f con 0x7f945009faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:29.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94500a08c0 0x7f945009b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 10 monclient(hunting): picked mon.noname-g con 0x7f94500a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:29.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f94500a2930 0x7f9450131950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 10 monclient(hunting): picked mon.noname-h con 0x7f94500a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:29.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945011f030 con 0x7f945009faa0 2024-09-15T01:36:29.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945009d630 con 0x7f94500a08c0 2024-09-15T01:36:29.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f945011db80 con 0x7f94500a2930 2024-09-15T01:36:29.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:29.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f945569f640 10 monclient(hunting): authenticate will time out at 994.108129s 2024-09-15T01:36:29.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94500a08c0 0x7f945009b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f945009faa0 0x7f945011c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944f7fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f94500a2930 0x7f9450131950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f945009faa0 0x7f945011c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:39408/0 (socket says 172.21.15.178:39408) 2024-09-15T01:36:29.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944effd640 1 -- 172.21.15.178:0/4205197743 learned_addr learned my addr 172.21.15.178:0/4205197743 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:29.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 10 monclient(hunting): get_auth_request con 0x7f94500a08c0 auth_method 0 2024-09-15T01:36:29.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944effd640 10 monclient(hunting): get_auth_request con 0x7f945009faa0 auth_method 0 2024-09-15T01:36:29.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944f7fe640 10 monclient(hunting): get_auth_request con 0x7f94500a2930 auth_method 0 2024-09-15T01:36:29.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 10 monclient(hunting): handle_auth_done global_id 15132 payload 824 2024-09-15T01:36:29.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 1 -- 172.21.15.178:0/4205197743 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f94500a2930 msgr2=0x7f9450131950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f94500a2930 0x7f9450131950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 1 -- 172.21.15.178:0/4205197743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f945009faa0 msgr2=0x7f945011c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f945009faa0 0x7f945011c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:36:29.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 1 monclient: found mon.noname-g 2024-09-15T01:36:29.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:29.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:29.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944e7fc640 1 -- 172.21.15.178:0/4205197743 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9444009450 con 0x7f94500a08c0 2024-09-15T01:36:29.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.358+0000 7f944f7fe640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f94500a2930 0x7f9450131950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:29.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f944e7fc640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94500a08c0 0x7f945009b350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9444031000 tx=0x7f9444032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9444006b30 con 0x7f94500a08c0 2024-09-15T01:36:29.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f9436ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f9436ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:29.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f9436ffd640 10 monclient: dump: 2024-09-15T01:36:29.388 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.388 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.388 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.388 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.388 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.388 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.388 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.388 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.388 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.388 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.389 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.389 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.389 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.389 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.389 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.389 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f9436ffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:29.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f9436ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:29.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944402ac60 con 0x7f94500a08c0 2024-09-15T01:36:29.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f9436ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:29.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f945569f640 5 monclient: authenticate success, global_id 15132 2024-09-15T01:36:29.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f945569f640 10 monclient: _renew_subs 2024-09-15T01:36:29.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f945569f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:29.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f945569f640 1 -- 172.21.15.178:0/4205197743 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945011ea70 con 0x7f94500a08c0 2024-09-15T01:36:29.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f944d7fa640 4 set_mon_vals no callback set 2024-09-15T01:36:29.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f945569f640 10 monclient: _renew_subs 2024-09-15T01:36:29.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f945569f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:29.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f945569f640 1 -- 172.21.15.178:0/4205197743 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945011d4a0 con 0x7f94500a08c0 2024-09-15T01:36:29.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9444033e00 con 0x7f94500a08c0 2024-09-15T01:36:29.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f9436ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f9436ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:29.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.359+0000 7f9436ffd640 10 monclient: dump: 2024-09-15T01:36:29.390 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.391 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.391 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.391 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.391 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.391 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.391 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.391 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.391 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.391 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.391 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.391 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.391 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.392 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.392 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.392 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f945569f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f945569f640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9450131e90 0x7f9450142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f945569f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f945569f640 1 -- 172.21.15.178:0/4205197743 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945011ec00 con 0x7f9450131e90 2024-09-15T01:36:29.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944effd640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9450131e90 0x7f9450142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9444039040 con 0x7f94500a08c0 2024-09-15T01:36:29.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944effd640 10 monclient: get_auth_request con 0x7f9450131e90 auth_method 0 2024-09-15T01:36:29.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944effd640 10 monclient(hunting): handle_auth_done global_id 24221 payload 824 2024-09-15T01:36:29.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f9436ffd640 1 --2- 172.21.15.178:0/4205197743 >> v2:172.21.15.204:6812/28210 conn(0x7f943c0396e0 0x7f943c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944effd640 1 -- 172.21.15.178:0/4205197743 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f945004c4a0 con 0x7f9450131e90 2024-09-15T01:36:29.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6959+0+0 (secure 0 0 0) 0x7f944406b260 con 0x7f94500a08c0 2024-09-15T01:36:29.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944effd640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9450131e90 0x7f9450142090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9440010980 tx=0x7f9440008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944f7fe640 1 --2- 172.21.15.178:0/4205197743 >> v2:172.21.15.204:6812/28210 conn(0x7f943c0396e0 0x7f943c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f944f7fe640 10 monclient: get_auth_request con 0x7f943c0396e0 auth_method 0 2024-09-15T01:36:29.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9440014430 con 0x7f9450131e90 2024-09-15T01:36:29.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f9436ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f9436ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:29.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.360+0000 7f9436ffd640 10 monclient: dump: 2024-09-15T01:36:29.394 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.394 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.394 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.394 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.394 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.394 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.395 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.395 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.395 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.395 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.395 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.395 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.395 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.395 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.395 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.395 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.361+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f944000eb40 con 0x7f9450131e90 2024-09-15T01:36:29.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.361+0000 7f9436ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.361+0000 7f9436ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:29.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.361+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9450131e90 msgr2=0x7f9450142090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.361+0000 7f9436ffd640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9450131e90 0x7f9450142090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9440010980 tx=0x7f9440008040 comp rx=0 tx=0).stop 2024-09-15T01:36:29.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.361+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9450131e90 msgr2=0x7f9450142090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:29.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.361+0000 7f9436ffd640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9450131e90 0x7f9450142090 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9440010980 tx=0x7f9440008040 comp rx=0 tx=0).stop 2024-09-15T01:36:29.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.361+0000 7f944f7fe640 1 --2- 172.21.15.178:0/4205197743 >> v2:172.21.15.204:6812/28210 conn(0x7f943c0396e0 0x7f943c03bb90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f94500a3290 tx=0x7f9448009850 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.366+0000 7f945569f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:29.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.366+0000 7f945569f640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94501425d0 0x7f94501429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.366+0000 7f945569f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.366+0000 7f945569f640 1 -- 172.21.15.178:0/4205197743 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945011f1a0 con 0x7f94501425d0 2024-09-15T01:36:29.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.366+0000 7f944effd640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94501425d0 0x7f94501429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.366+0000 7f944effd640 10 monclient: get_auth_request con 0x7f94501425d0 auth_method 0 2024-09-15T01:36:29.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.366+0000 7f944effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.366+0000 7f944effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.366+0000 7f944effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f944effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f944effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f944effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f944effd640 10 monclient(hunting): handle_auth_done global_id 24230 payload 824 2024-09-15T01:36:29.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f944effd640 1 -- 172.21.15.178:0/4205197743 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f945004bc80 con 0x7f94501425d0 2024-09-15T01:36:29.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f944effd640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94501425d0 0x7f94501429d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9440010980 tx=0x7f944001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f944002b070 con 0x7f94501425d0 2024-09-15T01:36:29.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f9436ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f9436ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:29.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f9436ffd640 10 monclient: dump: 2024-09-15T01:36:29.398 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.398 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.398 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.398 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.398 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.398 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.398 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.398 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.398 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.399 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.399 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.399 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.399 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.399 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.399 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.399 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f9440023040 con 0x7f94501425d0 2024-09-15T01:36:29.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f9436ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:29.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f9436ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:29.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94501425d0 msgr2=0x7f94501429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f9436ffd640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94501425d0 0x7f94501429d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9440010980 tx=0x7f944001d000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f9436ffd640 1 -- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94501425d0 msgr2=0x7f94501429d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:29.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.367+0000 7f9436ffd640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f94501425d0 0x7f94501429d0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9440010980 tx=0x7f944001d000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.400 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:29.400 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:29.400 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:29.400 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:29.401 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:29.401 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:29.401 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:29.401 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:29.401 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:29.401 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:29.401 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:29.401 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:29.401 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:29.401 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:29.401 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:29.402 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.402 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 5, 2024-09-15T01:36:29.402 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:29.402 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:29.402 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:29.402 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.402 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.402 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.402 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.402 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.402 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.402 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.404 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.404 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.404 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.404 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:29.404 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:29.404 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:29.404 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:29.404 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:29.404 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:29.404 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:29.405 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:29.405 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:29.405 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:29.405 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:29.405 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:29.405 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:29.405 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:29.405 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:29.405 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:29.405 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:29.405 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.405 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.406 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.406 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.406 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.406 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.406 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.406 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.406 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.406 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.406 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:29.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.406 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:29.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.407 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:29.407 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:29.407 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.407 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:29.407 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.407 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:29.408 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.408 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.408 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:29.408 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:29.408 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.408 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.408 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.409 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:29.409 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:29.409 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:29.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:29.410 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.410 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.410 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:29.410 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:29.410 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.410 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.410 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.411 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:29.411 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:29.411 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.411 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:29.411 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.411 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:29.412 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.412 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.412 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:29.412 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:29.412 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.412 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.412 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.413 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:29.413 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:29.413 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.413 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:29.413 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.413 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:29.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.414 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.414 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:29.414 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:29.414 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.414 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.414 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.414 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:29.415 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:29.415 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.415 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:29.415 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.415 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:29.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.416 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.416 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:29.416 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:29.416 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.416 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.416 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.417 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:29.417 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:29.417 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.417 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:29.417 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.417 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:29.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.418 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:29.418 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:29.418 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.418 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.418 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.418 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:29.419 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:29.419 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.419 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:29.419 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.419 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:29.419 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.420 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.420 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:29.420 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:29.420 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.420 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.420 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.420 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:29.421 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:29.421 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:29.421 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:29.422 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.422 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.422 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:29.422 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:29.422 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.422 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.422 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.423 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:29.423 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:29.423 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.423 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:29.423 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.423 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:29.424 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.424 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.424 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:29.424 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:29.424 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.424 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.424 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.424 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.424 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.425 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:29.425 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:29.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.425 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:29.425 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:29.425 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:29.425 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.425 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.425 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:29.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.425 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:29.426 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:29.426 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:29.426 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.426 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.426 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:29.426 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:29.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.368+0000 7f945569f640 1 -- 172.21.15.178:0/4205197743 >> v2:172.21.15.204:6812/28210 conn(0x7f943c0396e0 msgr2=0x7f943c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.368+0000 7f945569f640 1 --2- 172.21.15.178:0/4205197743 >> v2:172.21.15.204:6812/28210 conn(0x7f943c0396e0 0x7f943c03bb90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f94500a3290 tx=0x7f9448009850 comp rx=0 tx=0).stop 2024-09-15T01:36:29.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.368+0000 7f945569f640 10 monclient: shutdown 2024-09-15T01:36:29.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.368+0000 7f945569f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.368+0000 7f945569f640 1 -- 172.21.15.178:0/4205197743 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94500a08c0 msgr2=0x7f945009b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.368+0000 7f945569f640 1 --2- 172.21.15.178:0/4205197743 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94500a08c0 0x7f945009b350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9444031000 tx=0x7f9444032000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.368+0000 7f944f7fe640 1 -- 172.21.15.178:0/4205197743 reap_dead start 2024-09-15T01:36:29.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.368+0000 7f944f7fe640 1 -- 172.21.15.178:0/4205197743 reap_dead start 2024-09-15T01:36:29.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.368+0000 7f945569f640 1 -- 172.21.15.178:0/4205197743 shutdown_connections 2024-09-15T01:36:29.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.368+0000 7f945569f640 1 -- 172.21.15.178:0/4205197743 >> 172.21.15.178:0/4205197743 conn(0x7f94500a5ed0 msgr2=0x7f945009e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.368+0000 7f945569f640 1 -- 172.21.15.178:0/4205197743 shutdown_connections 2024-09-15T01:36:29.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.368+0000 7f945569f640 1 -- 172.21.15.178:0/4205197743 wait complete. 2024-09-15T01:36:29.429 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:29.429 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:36:29.429 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:36:29.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.500+0000 7f6d0f7fe640 1 -- 172.21.15.204:0/2215725134 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}]=0 added bucket barr type rack to crush map v176) v1 ==== 140+0+0 (secure 0 0 0) 0x7f6d18123050 con 0x7f6d1809c950 2024-09-15T01:36:29.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.500+0000 7f6d0f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-09-15T01:36:29.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.500+0000 7f6d0f7fe640 10 monclient: _finish_command 2 = system:0 added bucket barr type rack to crush map 2024-09-15T01:36:29.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.523+0000 7f7eb699d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:29.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.523+0000 7f7eb699d640 10 monclient: build_initial_monmap 2024-09-15T01:36:29.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.523+0000 7f7eb699d640 10 monclient: monmap: 2024-09-15T01:36:29.526 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:29.526 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.526 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:29.524809+0000 2024-09-15T01:36:29.526 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:29.524809+0000 2024-09-15T01:36:29.526 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:29.526 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.526 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:29.526 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:29.526 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:29.526 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:29.527 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:29.527 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:29.527 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:29.527 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:29.527 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:29.527 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:29.527 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:29.527 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.524+0000 7f7eb699d640 1 Processor -- start 2024-09-15T01:36:29.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.524+0000 7f7eb699d640 1 -- start start 2024-09-15T01:36:29.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.524+0000 7f7eb699d640 10 monclient: init 2024-09-15T01:36:29.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:29.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 10 monclient: _add_conns ranks=[6,2,0,4,9,7,1,3,5,10,8] 2024-09-15T01:36:29.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7eb00a2930 0x7f7eb0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 10 monclient(hunting): picked mon.noname-d con 0x7f7eb00a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:29.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb009faa0 0x7f7eb009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 10 monclient(hunting): picked mon.noname-b con 0x7f7eb009faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:29.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7eb00a08c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:29.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb004bf60 con 0x7f7eb00a08c0 2024-09-15T01:36:29.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb009e0f0 con 0x7f7eb009faa0 2024-09-15T01:36:29.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb009dd80 con 0x7f7eb00a2930 2024-09-15T01:36:29.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:29.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb699d640 10 monclient(hunting): authenticate will time out at 994.275374s 2024-09-15T01:36:29.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7eb00a2930 0x7f7eb0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb009faa0 0x7f7eb009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb4f3a640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f7eb00a08c0 0x7f7eb00a0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:47922/0 (socket says 172.21.15.178:47922) 2024-09-15T01:36:29.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eb4f3a640 1 -- 172.21.15.178:0/2180528701 learned_addr learned my addr 172.21.15.178:0/2180528701 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:36:29.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaffff640 10 monclient(hunting): get_auth_request con 0x7f7eb00a2930 auth_method 0 2024-09-15T01:36:29.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 10 monclient(hunting): get_auth_request con 0x7f7eb009faa0 auth_method 0 2024-09-15T01:36:29.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_done global_id 24239 payload 274 2024-09-15T01:36:29.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaeffd640 1 -- 172.21.15.178:0/2180528701 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230484775 0 0) 0x7f7eb004bf60 con 0x7f7eb00a08c0 2024-09-15T01:36:29.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 1 -- 172.21.15.178:0/2180528701 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7eb00a2930 msgr2=0x7f7eb0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:29.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 1 --2- 172.21.15.178:0/2180528701 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7eb00a2930 0x7f7eb0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 1 -- 172.21.15.178:0/2180528701 >> v1:172.21.15.178:6789/0 conn(0x7f7eb00a08c0 legacy=0x7f7eb00a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:29.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 1 monclient: found mon.noname-b 2024-09-15T01:36:29.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:29.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:36:29.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaf7fe640 1 -- 172.21.15.178:0/2180528701 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb00a2d00 con 0x7f7eb009faa0 2024-09-15T01:36:29.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaffff640 1 --2- 172.21.15.178:0/2180528701 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7eb00a2930 0x7f7eb0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:29.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.525+0000 7f7eaeffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:36:29.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaf7fe640 1 --2- 172.21.15.178:0/2180528701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb009faa0 0x7f7eb009fe70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009880 tx=0x7f7ea40090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad3c80a00951cd23 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaeffd640 1 -- 172.21.15.178:0/2180528701 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ea402ac50 con 0x7f7eb009faa0 2024-09-15T01:36:29.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:36:29.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaeffd640 10 monclient: dump: 2024-09-15T01:36:29.545 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.545 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.545 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.545 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.545 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.546 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.546 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.546 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.546 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.546 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.546 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.546 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.546 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.546 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.546 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.547 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaeffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:29.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:29.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaeffd640 1 -- 172.21.15.178:0/2180528701 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea402add0 con 0x7f7eb009faa0 2024-09-15T01:36:29.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eb699d640 5 monclient: authenticate success, global_id 24239 2024-09-15T01:36:29.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eb699d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:29.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:29.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eb699d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:29.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaeffd640 1 -- 172.21.15.178:0/2180528701 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ea4031ca0 con 0x7f7eb009faa0 2024-09-15T01:36:29.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eb699d640 4 set_mon_vals no callback set 2024-09-15T01:36:29.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:29.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.526+0000 7f7eaeffd640 10 monclient: dump: 2024-09-15T01:36:29.548 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.548 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.548 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.548 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.548 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.548 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.548 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.548 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.548 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.549 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.549 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.549 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.549 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.549 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.549 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.549 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.527+0000 7f7eb699d640 10 monclient: shutdown 2024-09-15T01:36:29.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.527+0000 7f7eb699d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.527+0000 7f7eb699d640 1 -- 172.21.15.178:0/2180528701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb009faa0 msgr2=0x7f7eb009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.527+0000 7f7eb699d640 1 --2- 172.21.15.178:0/2180528701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb009faa0 0x7f7eb009fe70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009880 tx=0x7f7ea40090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.527+0000 7f7eb699d640 1 -- 172.21.15.178:0/2180528701 shutdown_connections 2024-09-15T01:36:29.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.527+0000 7f7eb699d640 1 --2- 172.21.15.178:0/2180528701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb009faa0 0x7f7eb009fe70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.527+0000 7f7eb699d640 1 --2- 172.21.15.178:0/2180528701 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7eb00a2930 0x7f7eb0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.527+0000 7f7eb699d640 1 -- 172.21.15.178:0/2180528701 >> 172.21.15.178:0/2180528701 conn(0x7f7eb00a5ed0 msgr2=0x7f7eb00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.527+0000 7f7eb699d640 1 -- 172.21.15.178:0/2180528701 shutdown_connections 2024-09-15T01:36:29.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.527+0000 7f7eb699d640 1 -- 172.21.15.178:0/2180528701 wait complete. 2024-09-15T01:36:29.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.527+0000 7f7eb699d640 10 monclient: build_initial_monmap 2024-09-15T01:36:29.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.527+0000 7f7eb699d640 10 monclient: monmap: 2024-09-15T01:36:29.550 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:29.550 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:29.550 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:29.550 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:29.550 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:29.551 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:29.551 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:29.551 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:29.551 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:29.551 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:29.551 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:29.551 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:29.551 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:29.551 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:29.551 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:29.551 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 1 Processor -- start 2024-09-15T01:36:29.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 1 -- start start 2024-09-15T01:36:29.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 10 monclient: init 2024-09-15T01:36:29.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:29.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 10 monclient: _add_conns ranks=[4,7,3,2,1,0,6,8,5] 2024-09-15T01:36:29.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb009faa0 0x7f7eb0141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 10 monclient(hunting): picked mon.noname-c con 0x7f7eb009faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:29.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7eb00a08c0 0x7f7eb0131b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 10 monclient(hunting): picked mon.noname-i con 0x7f7eb00a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:29.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7eb00a2930 0x7f7eb011e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 10 monclient(hunting): picked mon.noname-g con 0x7f7eb00a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:29.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb0142b70 con 0x7f7eb00a2930 2024-09-15T01:36:29.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb00a2390 con 0x7f7eb009faa0 2024-09-15T01:36:29.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb009d630 con 0x7f7eb00a08c0 2024-09-15T01:36:29.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:29.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb699d640 10 monclient(hunting): authenticate will time out at 994.278709s 2024-09-15T01:36:29.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaffff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb009faa0 0x7f7eb0141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaffff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb009faa0 0x7f7eb0141c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:48442/0 (socket says 172.21.15.178:48442) 2024-09-15T01:36:29.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaffff640 1 -- 172.21.15.178:0/2402108757 learned_addr learned my addr 172.21.15.178:0/2402108757 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:29.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaffff640 10 monclient(hunting): get_auth_request con 0x7f7eb009faa0 auth_method 0 2024-09-15T01:36:29.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaf7fe640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7eb00a08c0 0x7f7eb0131b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb4f3a640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7eb00a2930 0x7f7eb011e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaf7fe640 10 monclient(hunting): get_auth_request con 0x7f7eb00a08c0 auth_method 0 2024-09-15T01:36:29.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb4f3a640 10 monclient(hunting): get_auth_request con 0x7f7eb00a2930 auth_method 0 2024-09-15T01:36:29.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb4f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb4f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.528+0000 7f7eb4f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eb4f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eb4f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eb4f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaffff640 10 monclient(hunting): handle_auth_done global_id 5944 payload 824 2024-09-15T01:36:29.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaffff640 1 -- 172.21.15.178:0/2402108757 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7eb00a08c0 msgr2=0x7f7eb0131b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaffff640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7eb00a08c0 0x7f7eb0131b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaffff640 1 -- 172.21.15.178:0/2402108757 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7eb00a2930 msgr2=0x7f7eb011e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaffff640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7eb00a2930 0x7f7eb011e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaffff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:29.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaffff640 1 monclient: found mon.noname-c 2024-09-15T01:36:29.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:29.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:29.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaffff640 1 -- 172.21.15.178:0/2402108757 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0009390 con 0x7f7eb009faa0 2024-09-15T01:36:29.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eaffff640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb009faa0 0x7f7eb0141c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000ec80 tx=0x7f7ea00090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ea000eec0 con 0x7f7eb009faa0 2024-09-15T01:36:29.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7e97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7e97fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:29.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7e97fff640 10 monclient: dump: 2024-09-15T01:36:29.557 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.557 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.557 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.557 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.557 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.557 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.557 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.558 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.558 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.558 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.558 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.558 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.558 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.558 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.558 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.558 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7e97fff640 10 monclient: _finish_auth 0 2024-09-15T01:36:29.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7e97fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:29.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea0013470 con 0x7f7eb009faa0 2024-09-15T01:36:29.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eb699d640 5 monclient: authenticate success, global_id 5944 2024-09-15T01:36:29.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eb699d640 10 monclient: _renew_subs 2024-09-15T01:36:29.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eb699d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:36:29.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eb699d640 1 -- 172.21.15.178:0/2402108757 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea4009390 con 0x7f7eb009faa0 2024-09-15T01:36:29.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7e97fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:29.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ea0013a60 con 0x7f7eb009faa0 2024-09-15T01:36:29.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7e97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7e97fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:29.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7e97fff640 10 monclient: dump: 2024-09-15T01:36:29.559 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.559 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.560 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.560 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.560 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.560 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.560 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.560 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.560 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.560 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.560 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.560 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.560 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.560 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.561 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.561 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eb699d640 10 monclient: _renew_subs 2024-09-15T01:36:29.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eae7fc640 4 set_mon_vals no callback set 2024-09-15T01:36:29.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eb699d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:36:29.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.529+0000 7f7eb699d640 1 -- 172.21.15.178:0/2402108757 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb0120140 con 0x7f7eb009faa0 2024-09-15T01:36:29.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eb699d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7ea0013bc0 con 0x7f7eb009faa0 2024-09-15T01:36:29.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eb699d640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb0132060 0x7f7eb011eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eb699d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eb699d640 1 -- 172.21.15.178:0/2402108757 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb0121e10 con 0x7f7eb0132060 2024-09-15T01:36:29.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eaf7fe640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb0132060 0x7f7eb011eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eaf7fe640 10 monclient: get_auth_request con 0x7f7eb0132060 auth_method 0 2024-09-15T01:36:29.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eaf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7e97fff640 1 --2- 172.21.15.178:0/2402108757 >> v2:172.21.15.204:6812/28210 conn(0x7f7e9c0397b0 0x7f7e9c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6987+0+0 (secure 0 0 0) 0x7f7ea004c3b0 con 0x7f7eb009faa0 2024-09-15T01:36:29.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_done global_id 5953 payload 824 2024-09-15T01:36:29.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eb4f3a640 1 --2- 172.21.15.178:0/2402108757 >> v2:172.21.15.204:6812/28210 conn(0x7f7e9c0397b0 0x7f7e9c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eaf7fe640 1 -- 172.21.15.178:0/2402108757 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7eb004c4a0 con 0x7f7eb0132060 2024-09-15T01:36:29.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eb4f3a640 10 monclient: get_auth_request con 0x7f7e9c0397b0 auth_method 0 2024-09-15T01:36:29.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7eaf7fe640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb0132060 0x7f7eb011eb90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00a3290 tx=0x7f7ea4032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ea402ac50 con 0x7f7eb0132060 2024-09-15T01:36:29.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.530+0000 7f7e97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.531+0000 7f7e97fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:29.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.531+0000 7f7e97fff640 10 monclient: dump: 2024-09-15T01:36:29.563 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.563 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.564 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.564 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.564 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.564 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.564 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.564 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.564 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.564 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.564 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.564 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.564 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.565 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.565 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.565 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.531+0000 7f7eb4f3a640 1 --2- 172.21.15.178:0/2402108757 >> v2:172.21.15.204:6812/28210 conn(0x7f7e9c0397b0 0x7f7e9c03bc60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8001760 tx=0x7f7ea8009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.531+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f7ea4032e20 con 0x7f7eb0132060 2024-09-15T01:36:29.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.531+0000 7f7e97fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.531+0000 7f7e97fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:29.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.531+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb0132060 msgr2=0x7f7eb011eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.531+0000 7f7e97fff640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb0132060 0x7f7eb011eb90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00a3290 tx=0x7f7ea4032000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.531+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb0132060 msgr2=0x7f7eb011eb90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:29.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.531+0000 7f7e97fff640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb0132060 0x7f7eb011eb90 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00a3290 tx=0x7f7ea4032000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.536+0000 7f7eb699d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:29.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.536+0000 7f7eb699d640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb011f0d0 0x7f7eb0099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.536+0000 7f7eb699d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.536+0000 7f7eb699d640 1 -- 172.21.15.178:0/2402108757 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb0132430 con 0x7f7eb011f0d0 2024-09-15T01:36:29.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7eaf7fe640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb011f0d0 0x7f7eb0099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7eaf7fe640 10 monclient: get_auth_request con 0x7f7eb011f0d0 auth_method 0 2024-09-15T01:36:29.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7eaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7eaf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7eaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7eaf7fe640 10 monclient(hunting): handle_auth_done global_id 5962 payload 824 2024-09-15T01:36:29.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7eaf7fe640 1 -- 172.21.15.178:0/2402108757 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7eb004bc80 con 0x7f7eb011f0d0 2024-09-15T01:36:29.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7eaf7fe640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb011f0d0 0x7f7eb0099a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4041000 tx=0x7f7ea403d000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ea403dda0 con 0x7f7eb011f0d0 2024-09-15T01:36:29.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7e97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7e97fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:29.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7e97fff640 10 monclient: dump: 2024-09-15T01:36:29.567 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.567 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.567 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.568 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.568 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.568 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.568 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.568 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.568 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.568 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.568 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.568 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.568 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.568 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.569 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.569 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f7ea403c3e0 con 0x7f7eb011f0d0 2024-09-15T01:36:29.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7e97fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:29.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7e97fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:29.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb011f0d0 msgr2=0x7f7eb0099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7e97fff640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb011f0d0 0x7f7eb0099a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4041000 tx=0x7f7ea403d000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7e97fff640 1 -- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb011f0d0 msgr2=0x7f7eb0099a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:29.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.537+0000 7f7e97fff640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb011f0d0 0x7f7eb0099a50 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4041000 tx=0x7f7ea403d000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.570 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:29.570 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:29.570 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:29.570 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:29.570 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:29.570 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:29.570 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:29.570 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:29.570 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:29.570 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:29.570 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:29.571 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:29.571 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:29.571 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:29.571 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:29.571 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.571 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:36:29.571 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:29.571 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:29.571 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:29.571 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.571 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.571 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.572 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.572 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.572 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.572 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.572 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.572 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.572 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.572 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:29.572 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:29.572 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.572 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.573 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.573 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.573 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.573 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.573 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.573 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.573 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.573 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.573 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:29.574 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:29.574 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:29.574 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:29.574 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:29.574 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:29.574 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:29.574 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:29.574 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:29.574 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:29.574 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:29.574 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:29.574 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:29.575 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:29.575 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:29.575 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:29.575 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:29.575 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.575 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.575 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.575 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.575 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.575 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.575 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.576 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.576 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.576 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.576 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:29.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.576 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:29.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.576 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:29.576 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:29.576 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.577 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:29.577 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.577 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:29.577 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.577 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.577 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.578 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:29.578 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:29.578 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.578 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.578 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.578 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:29.578 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:29.578 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.579 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:29.579 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.579 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:29.579 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.579 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.579 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.580 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:29.580 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:29.580 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.580 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.580 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.580 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:29.580 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:29.580 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.581 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:29.581 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.581 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:29.581 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.581 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.581 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.582 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:29.582 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:29.582 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.582 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.582 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.582 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:29.582 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:29.582 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.583 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.583 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.583 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:29.583 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.583 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.583 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:29.583 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.583 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.583 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.584 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:29.584 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:29.584 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.584 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.584 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.584 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:29.584 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:29.584 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.585 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.586 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:29.586 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:29.586 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.586 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.586 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.586 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:29.586 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:29.586 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.587 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.587 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.587 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.587 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:29.587 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.587 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.587 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.587 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:29.587 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.587 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.587 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.588 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:29.588 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:29.588 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.588 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.588 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.588 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.588 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:29.588 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:29.588 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.589 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.589 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.589 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:29.589 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.589 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.589 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:29.589 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.589 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.589 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:29.590 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:29.590 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.590 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.590 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.590 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:29.590 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:29.590 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.591 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.591 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:29.591 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.591 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.591 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:29.591 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.591 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.591 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.592 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:29.592 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:29.592 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.592 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.592 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.592 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:29.592 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:29.592 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.592 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.593 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.593 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:29.593 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.593 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.593 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:29.593 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.593 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.593 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.594 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:29.594 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:29.594 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.594 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.594 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.594 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.594 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.594 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:29.594 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:29.594 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.595 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:29.595 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:29.595 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:29.595 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.595 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.595 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:29.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.595 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:29.595 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:29.595 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:36:29.595 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.595 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.596 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:29.596 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:29.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.538+0000 7f7eb699d640 1 -- 172.21.15.178:0/2402108757 >> v2:172.21.15.204:6812/28210 conn(0x7f7e9c0397b0 msgr2=0x7f7e9c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.538+0000 7f7eb699d640 1 --2- 172.21.15.178:0/2402108757 >> v2:172.21.15.204:6812/28210 conn(0x7f7e9c0397b0 0x7f7e9c03bc60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8001760 tx=0x7f7ea8009040 comp rx=0 tx=0).stop 2024-09-15T01:36:29.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.538+0000 7f7eb699d640 10 monclient: shutdown 2024-09-15T01:36:29.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.538+0000 7f7eb699d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.538+0000 7f7eb699d640 1 -- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb009faa0 msgr2=0x7f7eb0141c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.538+0000 7f7eb699d640 1 --2- 172.21.15.178:0/2402108757 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7eb009faa0 0x7f7eb0141c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000ec80 tx=0x7f7ea00090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.538+0000 7f7eb4f3a640 1 -- 172.21.15.178:0/2402108757 reap_dead start 2024-09-15T01:36:29.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.538+0000 7f7eb4f3a640 1 -- 172.21.15.178:0/2402108757 reap_dead start 2024-09-15T01:36:29.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.538+0000 7f7eb699d640 1 -- 172.21.15.178:0/2402108757 shutdown_connections 2024-09-15T01:36:29.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.538+0000 7f7eb699d640 1 -- 172.21.15.178:0/2402108757 >> 172.21.15.178:0/2402108757 conn(0x7f7eb00a5ed0 msgr2=0x7f7eb009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.538+0000 7f7eb699d640 1 -- 172.21.15.178:0/2402108757 shutdown_connections 2024-09-15T01:36:29.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.538+0000 7f7eb699d640 1 -- 172.21.15.178:0/2402108757 wait complete. 2024-09-15T01:36:29.599 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:29.599 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:36:29.599 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:36:29.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.602+0000 7f6d1e19d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-09-15T01:36:29.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.602+0000 7f6d1e19d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-09-15T01:36:29.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.602+0000 7f6d1e19d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:29.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.602+0000 7f6d1e19d640 1 -- 172.21.15.204:0/2215725134 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"} v 0) v1 -- 0x7f6d18123050 con 0x7f6d1809c950 2024-09-15T01:36:29.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.603+0000 7f6d0f7fe640 1 -- 172.21.15.204:0/2215725134 <== mon.8 v2:172.21.15.178:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}]=0 bucket 'barr' already exists v176) v1 ==== 128+0+0 (secure 0 0 0) 0x7f6d18123050 con 0x7f6d1809c950 2024-09-15T01:36:29.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.603+0000 7f6d0f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-09-15T01:36:29.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.603+0000 7f6d0f7fe640 10 monclient: _finish_command 3 = system:0 bucket 'barr' already exists 2024-09-15T01:36:29.606 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'barr' already exists 2024-09-15T01:36:29.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 1 -- 172.21.15.204:0/2215725134 >> v2:172.21.15.204:6812/28210 conn(0x7f6cf80397b0 msgr2=0x7f6cf803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 1 --2- 172.21.15.204:0/2215725134 >> v2:172.21.15.204:6812/28210 conn(0x7f6cf80397b0 0x7f6cf803bc60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6d08036000 tx=0x7f6d08039000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 10 monclient: shutdown 2024-09-15T01:36:29.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 1 -- 172.21.15.204:0/2215725134 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6d1809c950 msgr2=0x7f6d1813d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 1 --2- 172.21.15.204:0/2215725134 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6d1809c950 0x7f6d1813d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009fd0 tx=0x7f6d000090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 1 -- 172.21.15.204:0/2215725134 shutdown_connections 2024-09-15T01:36:29.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 1 --2- 172.21.15.204:0/2215725134 >> v2:172.21.15.204:6812/28210 conn(0x7f6cf80397b0 0x7f6cf803bc60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 1 --2- 172.21.15.204:0/2215725134 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6d180a35e0 0x7f6d18141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 1 --2- 172.21.15.204:0/2215725134 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6d1809c950 0x7f6d1813d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 1 --2- 172.21.15.204:0/2215725134 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6d1809c560 0x7f6d1813e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 1 -- 172.21.15.204:0/2215725134 >> 172.21.15.204:0/2215725134 conn(0x7f6d180687e0 msgr2=0x7f6d180a1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 1 -- 172.21.15.204:0/2215725134 shutdown_connections 2024-09-15T01:36:29.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.605+0000 7f6d1e19d640 1 -- 172.21.15.204:0/2215725134 wait complete. 2024-09-15T01:36:29.616 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush move barr root=default 2024-09-15T01:36:29.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.688+0000 7f70e1d14640 10 monclient: get_monmap_and_config 2024-09-15T01:36:29.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.689+0000 7f70e1d14640 10 monclient: build_initial_monmap 2024-09-15T01:36:29.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.689+0000 7f70e1d14640 10 monclient: monmap: 2024-09-15T01:36:29.692 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:29.692 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.692 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:29.690511+0000 2024-09-15T01:36:29.692 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:29.690511+0000 2024-09-15T01:36:29.692 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:29.692 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:29.692 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:29.692 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:29.692 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:29.692 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:29.692 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:29.693 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:29.693 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:29.693 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:29.693 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:29.693 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:29.693 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:29.693 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.689+0000 7f70e1d14640 1 Processor -- start 2024-09-15T01:36:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.689+0000 7f70e1d14640 1 -- start start 2024-09-15T01:36:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.689+0000 7f70e1d14640 10 monclient: init 2024-09-15T01:36:29.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 10 monclient: _add_conns ranks=[6,5,9,10,1,7,8,4,2,3,0] 2024-09-15T01:36:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f70dc0a2e30 0x7f70dc132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 10 monclient(hunting): picked mon.noname-d con 0x7f70dc0a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc09fb80 0x7f70dc09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 10 monclient(hunting): picked mon.noname-h con 0x7f70dc09fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f70dc0a0a40 0x7f70dc0a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 10 monclient(hunting): picked mon.noname-a con 0x7f70dc0a0a40 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc09e6c0 con 0x7f70dc0a0a40 2024-09-15T01:36:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0a04b0 con 0x7f70dc09fb80 2024-09-15T01:36:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0a3200 con 0x7f70dc0a2e30 2024-09-15T01:36:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70e1d14640 10 monclient(hunting): authenticate will time out at 1005.333398s 2024-09-15T01:36:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc09fb80 0x7f70dc09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc09fb80 0x7f70dc09ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:58178/0 (socket says 172.21.15.204:58178) 2024-09-15T01:36:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 1 -- 172.21.15.204:0/2869990361 learned_addr learned my addr 172.21.15.204:0/2869990361 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 10 monclient(hunting): get_auth_request con 0x7f70dc09fb80 auth_method 0 2024-09-15T01:36:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70db7fe640 1 --2- 172.21.15.204:0/2869990361 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f70dc0a2e30 0x7f70dc132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70dbfff640 1 --2- 172.21.15.204:0/2869990361 >> v2:172.21.15.178:3300/0 conn(0x7f70dc0a0a40 0x7f70dc0a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70db7fe640 10 monclient(hunting): get_auth_request con 0x7f70dc0a2e30 auth_method 0 2024-09-15T01:36:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70dbfff640 10 monclient(hunting): get_auth_request con 0x7f70dc0a0a40 auth_method 0 2024-09-15T01:36:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 10 monclient(hunting): handle_auth_done global_id 6008 payload 274 2024-09-15T01:36:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 1 -- 172.21.15.204:0/2869990361 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f70dc0a2e30 msgr2=0x7f70dc132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:29.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 1 --2- 172.21.15.204:0/2869990361 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f70dc0a2e30 0x7f70dc132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 1 -- 172.21.15.204:0/2869990361 >> v2:172.21.15.178:3300/0 conn(0x7f70dc0a0a40 msgr2=0x7f70dc0a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 1 --2- 172.21.15.204:0/2869990361 >> v2:172.21.15.178:3300/0 conn(0x7f70dc0a0a40 0x7f70dc0a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 10 monclient: _finish_hunting 0 2024-09-15T01:36:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 1 monclient: found mon.noname-h 2024-09-15T01:36:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.690+0000 7f70daffd640 1 -- 172.21.15.204:0/2869990361 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc04b720 con 0x7f70dc09fb80 2024-09-15T01:36:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70daffd640 1 --2- 172.21.15.204:0/2869990361 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc09fb80 0x7f70dc09ff70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009910 tx=0x7f70d00091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=5aea1488102ceb2a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70da7fc640 1 -- 172.21.15.204:0/2869990361 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f70d002ac80 con 0x7f70dc09fb80 2024-09-15T01:36:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70da7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70da7fc640 10 monclient: dump: 2024-09-15T01:36:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.718 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70da7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70da7fc640 1 -- 172.21.15.204:0/2869990361 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d002ae00 con 0x7f70dc09fb80 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70e1d14640 5 monclient: authenticate success, global_id 6008 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70e1d14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70da7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70da7fc640 1 -- 172.21.15.204:0/2869990361 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f70d0031ca0 con 0x7f70dc09fb80 2024-09-15T01:36:29.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70da7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70da7fc640 10 monclient: dump: 2024-09-15T01:36:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.720 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.721 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70e1d14640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.691+0000 7f70e1d14640 4 set_mon_vals no callback set 2024-09-15T01:36:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 10 monclient: shutdown 2024-09-15T01:36:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 1 -- 172.21.15.204:0/2869990361 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc09fb80 msgr2=0x7f70dc09ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 1 --2- 172.21.15.204:0/2869990361 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc09fb80 0x7f70dc09ff70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009910 tx=0x7f70d00091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 1 -- 172.21.15.204:0/2869990361 shutdown_connections 2024-09-15T01:36:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 1 --2- 172.21.15.204:0/2869990361 >> v2:172.21.15.178:3300/0 conn(0x7f70dc0a0a40 0x7f70dc0a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 1 --2- 172.21.15.204:0/2869990361 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc09fb80 0x7f70dc09ff70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 1 --2- 172.21.15.204:0/2869990361 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f70dc0a2e30 0x7f70dc132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 1 -- 172.21.15.204:0/2869990361 >> 172.21.15.204:0/2869990361 conn(0x7f70dc06e860 msgr2=0x7f70dc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 1 -- 172.21.15.204:0/2869990361 shutdown_connections 2024-09-15T01:36:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 1 -- 172.21.15.204:0/2869990361 wait complete. 2024-09-15T01:36:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 10 monclient: build_initial_monmap 2024-09-15T01:36:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.692+0000 7f70e1d14640 10 monclient: monmap: 2024-09-15T01:36:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:29.722 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:29.723 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:29.723 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:29.723 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:29.723 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:29.723 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:29.723 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:29.723 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:29.723 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:29.723 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:29.723 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:29.723 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:29.724 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:29.724 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:29.724 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:29.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 1 Processor -- start 2024-09-15T01:36:29.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 1 -- start start 2024-09-15T01:36:29.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 10 monclient: init 2024-09-15T01:36:29.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:29.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 10 monclient: _add_conns ranks=[0,1,5,2,3,7,8,4,6] 2024-09-15T01:36:29.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70dc09fb80 0x7f70dc130700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 10 monclient(hunting): picked mon.noname-a con 0x7f70dc09fb80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:29.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70dc0a0a40 0x7f70dc11d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 10 monclient(hunting): picked mon.noname-f con 0x7f70dc0a0a40 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:29.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc0a2e30 0x7f70dc11d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 10 monclient(hunting): picked mon.noname-h con 0x7f70dc0a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:29.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc09ca40 con 0x7f70dc09fb80 2024-09-15T01:36:29.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc09d650 con 0x7f70dc0a0a40 2024-09-15T01:36:29.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc120850 con 0x7f70dc0a2e30 2024-09-15T01:36:29.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:29.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70e1d14640 10 monclient(hunting): authenticate will time out at 1005.336805s 2024-09-15T01:36:29.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70daffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70dc0a0a40 0x7f70dc11d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70dbfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc0a2e30 0x7f70dc11d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70daffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70dc0a0a40 0x7f70dc11d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39402/0 (socket says 172.21.15.204:39402) 2024-09-15T01:36:29.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70db7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70dc09fb80 0x7f70dc130700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70dbfff640 10 monclient(hunting): get_auth_request con 0x7f70dc0a2e30 auth_method 0 2024-09-15T01:36:29.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70daffd640 1 -- 172.21.15.204:0/4051040635 learned_addr learned my addr 172.21.15.204:0/4051040635 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:29.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70daffd640 10 monclient(hunting): get_auth_request con 0x7f70dc0a0a40 auth_method 0 2024-09-15T01:36:29.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70daffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70daffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70db7fe640 10 monclient(hunting): get_auth_request con 0x7f70dc09fb80 auth_method 0 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.693+0000 7f70daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70dbfff640 10 monclient(hunting): handle_auth_done global_id 6017 payload 824 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70dbfff640 1 -- 172.21.15.204:0/4051040635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70dc0a0a40 msgr2=0x7f70dc11d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70dbfff640 1 --2- 172.21.15.204:0/4051040635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70dc0a0a40 0x7f70dc11d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70dbfff640 1 -- 172.21.15.204:0/4051040635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70dc09fb80 msgr2=0x7f70dc130700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70dbfff640 1 --2- 172.21.15.204:0/4051040635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70dc09fb80 0x7f70dc130700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70dbfff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70dbfff640 1 monclient: found mon.noname-h 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70dbfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70dbfff640 1 -- 172.21.15.204:0/4051040635 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d0009450 con 0x7f70dc0a2e30 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70dbfff640 1 --2- 172.21.15.204:0/4051040635 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc0a2e30 0x7f70dc11d720 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f70d400e930 tx=0x7f70d40090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70c37fe640 1 -- 172.21.15.204:0/4051040635 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f70d4009520 con 0x7f70dc0a2e30 2024-09-15T01:36:29.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70c37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70c37fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70c37fe640 10 monclient: dump: 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.729 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70c37fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70c37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70c37fe640 1 -- 172.21.15.204:0/4051040635 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d400cdc0 con 0x7f70dc0a2e30 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70e1d14640 5 monclient: authenticate success, global_id 6017 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70e1d14640 10 monclient: _renew_subs 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70e1d14640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70e1d14640 1 -- 172.21.15.204:0/4051040635 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc120b30 con 0x7f70dc0a2e30 2024-09-15T01:36:29.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70c37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70c37fe640 1 -- 172.21.15.204:0/4051040635 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f70d40138f0 con 0x7f70dc0a2e30 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70c37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70c37fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70c37fe640 10 monclient: dump: 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.731 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.732 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.732 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.732 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.732 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.732 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.732 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.732 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:29.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70e1d14640 10 monclient: _renew_subs 2024-09-15T01:36:29.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70e1d14640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:29.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70e1d14640 1 -- 172.21.15.204:0/4051040635 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc1204d0 con 0x7f70dc0a2e30 2024-09-15T01:36:29.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.694+0000 7f70d9ffb640 4 set_mon_vals no callback set 2024-09-15T01:36:29.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.695+0000 7f70c37fe640 1 -- 172.21.15.204:0/4051040635 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f70d4013470 con 0x7f70dc0a2e30 2024-09-15T01:36:29.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.695+0000 7f70e1d14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.695+0000 7f70e1d14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.695+0000 7f70e1d14640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:29.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.695+0000 7f70c37fe640 1 --2- 172.21.15.204:0/4051040635 >> v2:172.21.15.204:6812/28210 conn(0x7f70c80397b0 0x7f70c803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.695+0000 7f70e1d14640 1 -- 172.21.15.204:0/4051040635 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc070cc0 con 0x7f70dc0a2e30 2024-09-15T01:36:29.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.695+0000 7f70c37fe640 1 -- 172.21.15.204:0/4051040635 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6987+0+0 (secure 0 0 0) 0x7f70d404bec0 con 0x7f70dc0a2e30 2024-09-15T01:36:29.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.695+0000 7f70db7fe640 1 --2- 172.21.15.204:0/4051040635 >> v2:172.21.15.204:6812/28210 conn(0x7f70c80397b0 0x7f70c803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.695+0000 7f70db7fe640 10 monclient: get_auth_request con 0x7f70c80397b0 auth_method 0 2024-09-15T01:36:29.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.696+0000 7f70db7fe640 1 --2- 172.21.15.204:0/4051040635 >> v2:172.21.15.204:6812/28210 conn(0x7f70c80397b0 0x7f70c803bc60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009fd0 tx=0x7f70cc00b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.700+0000 7f70c37fe640 1 -- 172.21.15.204:0/4051040635 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f70d40194f0 con 0x7f70dc0a2e30 2024-09-15T01:36:29.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.700+0000 7f70c37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.700+0000 7f70c37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:29.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.690+0000 7ffb51b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:29.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.690+0000 7ffb51b9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:29.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.690+0000 7ffb51b9d640 10 monclient: monmap: 2024-09-15T01:36:29.737 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:29.737 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.737 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:29.691832+0000 2024-09-15T01:36:29.737 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:29.691832+0000 2024-09-15T01:36:29.737 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:29.737 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.737 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:29.737 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:29.737 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:29.737 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:29.737 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.691+0000 7ffb51b9d640 1 Processor -- start 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.691+0000 7ffb51b9d640 1 -- start start 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.691+0000 7ffb51b9d640 10 monclient: init 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 10 monclient: _add_conns ranks=[3,4,9,7,10,1,6,5,8,2,0] 2024-09-15T01:36:29.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb4c0a2930 0x7ffb4c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 10 monclient(hunting): picked mon.noname-g con 0x7ffb4c0a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 0x7ffb4c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 10 monclient(hunting): picked mon.noname-c con 0x7ffb4c09faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffb4c0a08c0 0x7ffb4c0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 10 monclient(hunting): picked mon.noname-a con 0x7ffb4c0a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c09e0f0 con 0x7ffb4c0a08c0 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c09dd80 con 0x7ffb4c0a2930 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0a2d00 con 0x7ffb4c09faa0 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:29.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb51b9d640 10 monclient(hunting): authenticate will time out at 994.442454s 2024-09-15T01:36:29.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 0x7ffb4c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4bfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffb4c0a08c0 0x7ffb4c0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb4c0a2930 0x7ffb4c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 0x7ffb4c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:48466/0 (socket says 172.21.15.178:48466) 2024-09-15T01:36:29.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 1 -- 172.21.15.178:0/809433117 learned_addr learned my addr 172.21.15.178:0/809433117 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:29.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4bfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffb4c0a08c0 0x7ffb4c0a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53940/0 (socket says 172.21.15.178:53940) 2024-09-15T01:36:29.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 10 monclient(hunting): get_auth_request con 0x7ffb4c09faa0 auth_method 0 2024-09-15T01:36:29.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4bfff640 10 monclient(hunting): get_auth_request con 0x7ffb4c0a08c0 auth_method 0 2024-09-15T01:36:29.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4b7fe640 10 monclient(hunting): get_auth_request con 0x7ffb4c0a2930 auth_method 0 2024-09-15T01:36:29.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 10 monclient(hunting): handle_auth_done global_id 5971 payload 274 2024-09-15T01:36:29.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 1 -- 172.21.15.178:0/809433117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb4c0a2930 msgr2=0x7ffb4c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:29.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 1 --2- 172.21.15.178:0/809433117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb4c0a2930 0x7ffb4c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 1 -- 172.21.15.178:0/809433117 >> v2:172.21.15.178:3300/0 conn(0x7ffb4c0a08c0 msgr2=0x7ffb4c0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:29.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 1 --2- 172.21.15.178:0/809433117 >> v2:172.21.15.178:3300/0 conn(0x7ffb4c0a08c0 0x7ffb4c0a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 10 monclient: _finish_hunting 0 2024-09-15T01:36:29.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 1 monclient: found mon.noname-c 2024-09-15T01:36:29.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:29.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:29.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4affd640 1 -- 172.21.15.178:0/809433117 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c04bf60 con 0x7ffb4c09faa0 2024-09-15T01:36:29.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.692+0000 7ffb4bfff640 1 --2- 172.21.15.178:0/809433117 >> v2:172.21.15.178:3300/0 conn(0x7ffb4c0a08c0 0x7ffb4c0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:29.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4affd640 1 --2- 172.21.15.178:0/809433117 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 0x7ffb4c09fe70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009910 tx=0x7ffb400091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=ceb960e89e30613e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4a7fc640 1 -- 172.21.15.178:0/809433117 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb4002ac90 con 0x7ffb4c09faa0 2024-09-15T01:36:29.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:29.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4a7fc640 10 monclient: dump: 2024-09-15T01:36:29.743 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.744 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.744 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.744 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.744 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.744 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.744 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.744 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.744 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.744 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.744 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.744 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.744 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.745 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.745 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.745 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:29.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:29.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4a7fc640 1 -- 172.21.15.178:0/809433117 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4002ae10 con 0x7ffb4c09faa0 2024-09-15T01:36:29.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb51b9d640 5 monclient: authenticate success, global_id 5971 2024-09-15T01:36:29.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb51b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:29.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:29.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb51b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:29.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4a7fc640 1 -- 172.21.15.178:0/809433117 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb40031ca0 con 0x7ffb4c09faa0 2024-09-15T01:36:29.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb51b9d640 4 set_mon_vals no callback set 2024-09-15T01:36:29.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:29.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.693+0000 7ffb4a7fc640 10 monclient: dump: 2024-09-15T01:36:29.746 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.746 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.746 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.746 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.746 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.746 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.746 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.747 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.747 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.747 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.747 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.747 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.747 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.747 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.747 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.747 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 10 monclient: shutdown 2024-09-15T01:36:29.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 1 -- 172.21.15.178:0/809433117 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 msgr2=0x7ffb4c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 1 --2- 172.21.15.178:0/809433117 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 0x7ffb4c09fe70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009910 tx=0x7ffb400091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 1 -- 172.21.15.178:0/809433117 shutdown_connections 2024-09-15T01:36:29.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 1 --2- 172.21.15.178:0/809433117 >> v2:172.21.15.178:3300/0 conn(0x7ffb4c0a08c0 0x7ffb4c0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 1 --2- 172.21.15.178:0/809433117 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 0x7ffb4c09fe70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 1 --2- 172.21.15.178:0/809433117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb4c0a2930 0x7ffb4c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 1 -- 172.21.15.178:0/809433117 >> 172.21.15.178:0/809433117 conn(0x7ffb4c0a5ed0 msgr2=0x7ffb4c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 1 -- 172.21.15.178:0/809433117 shutdown_connections 2024-09-15T01:36:29.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 1 -- 172.21.15.178:0/809433117 wait complete. 2024-09-15T01:36:29.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:29.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.694+0000 7ffb51b9d640 10 monclient: monmap: 2024-09-15T01:36:29.748 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:29.748 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:29.749 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:29.749 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:29.749 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:29.749 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:29.749 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:29.749 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:29.749 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:29.749 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:29.749 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:29.749 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:29.749 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:29.749 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:29.750 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:29.750 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 1 Processor -- start 2024-09-15T01:36:29.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 1 -- start start 2024-09-15T01:36:29.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 10 monclient: init 2024-09-15T01:36:29.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:29.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 10 monclient: _add_conns ranks=[4,2,8,7,0,6,1,3,5] 2024-09-15T01:36:29.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 0x7ffb4c09b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 10 monclient(hunting): picked mon.noname-c con 0x7ffb4c09faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:29.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb4c0a08c0 0x7ffb4c131790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 10 monclient(hunting): picked mon.noname-b con 0x7ffb4c0a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:29.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffb4c0a2930 0x7ffb4c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 10 monclient(hunting): picked mon.noname-e con 0x7ffb4c0a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:29.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c09bc30 con 0x7ffb4c0a08c0 2024-09-15T01:36:29.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c09d630 con 0x7ffb4c09faa0 2024-09-15T01:36:29.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0a2390 con 0x7ffb4c0a2930 2024-09-15T01:36:29.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:29.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb51b9d640 10 monclient(hunting): authenticate will time out at 994.445872s 2024-09-15T01:36:29.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb4b7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 0x7ffb4c09b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb4b7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 0x7ffb4c09b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:48476/0 (socket says 172.21.15.178:48476) 2024-09-15T01:36:29.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb4b7fe640 1 -- 172.21.15.178:0/833413541 learned_addr learned my addr 172.21.15.178:0/833413541 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:29.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb4b7fe640 10 monclient(hunting): get_auth_request con 0x7ffb4c09faa0 auth_method 0 2024-09-15T01:36:29.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb4affd640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb4c0a08c0 0x7ffb4c131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.695+0000 7ffb4bfff640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffb4c0a2930 0x7ffb4c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4affd640 10 monclient(hunting): get_auth_request con 0x7ffb4c0a08c0 auth_method 0 2024-09-15T01:36:29.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4bfff640 10 monclient(hunting): get_auth_request con 0x7ffb4c0a2930 auth_method 0 2024-09-15T01:36:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 10 monclient(hunting): handle_auth_done global_id 5980 payload 824 2024-09-15T01:36:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 1 -- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffb4c0a2930 msgr2=0x7ffb4c11c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffb4c0a2930 0x7ffb4c11c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 1 -- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb4c0a08c0 msgr2=0x7ffb4c131790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb4c0a08c0 0x7ffb4c131790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 1 monclient: found mon.noname-c 2024-09-15T01:36:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 1 -- 172.21.15.178:0/833413541 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb44009a20 con 0x7ffb4c09faa0 2024-09-15T01:36:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4affd640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffb4c0a08c0 0x7ffb4c131790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb4b7fe640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 0x7ffb4c09b000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00ea80 tx=0x7ffb3c0090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb3c00cdc0 con 0x7ffb4c09faa0 2024-09-15T01:36:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb337fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb337fe640 10 monclient: dump: 2024-09-15T01:36:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.756 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.756 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.756 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.756 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.756 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.756 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.756 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.756 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.756 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb337fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:29.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:29.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c013470 con 0x7ffb4c09faa0 2024-09-15T01:36:29.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb51b9d640 5 monclient: authenticate success, global_id 5980 2024-09-15T01:36:29.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb51b9d640 10 monclient: _renew_subs 2024-09-15T01:36:29.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb51b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:36:29.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb51b9d640 1 -- 172.21.15.178:0/833413541 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb40009450 con 0x7ffb4c09faa0 2024-09-15T01:36:29.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb337fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:29.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb3c013a60 con 0x7ffb4c09faa0 2024-09-15T01:36:29.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb337fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:29.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb337fe640 10 monclient: dump: 2024-09-15T01:36:29.757 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.758 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.758 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.758 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.758 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.758 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.758 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.758 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.758 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.758 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.758 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.758 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.758 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb51b9d640 10 monclient: _renew_subs 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb51b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb51b9d640 1 -- 172.21.15.178:0/833413541 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c11eb90 con 0x7ffb4c09faa0 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.696+0000 7ffb49ffb640 4 set_mon_vals no callback set 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb51b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb51b9d640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c1370c0 0x7ffb4c11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb51b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb51b9d640 1 -- 172.21.15.178:0/833413541 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c04c4a0 con 0x7ffb4c1370c0 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ffb3c013bc0 con 0x7ffb4c09faa0 2024-09-15T01:36:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb4affd640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c1370c0 0x7ffb4c11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb4affd640 10 monclient: get_auth_request con 0x7ffb4c1370c0 auth_method 0 2024-09-15T01:36:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb4affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb337fe640 1 --2- 172.21.15.178:0/833413541 >> v2:172.21.15.204:6812/28210 conn(0x7ffb380397b0 0x7ffb3803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6987+0+0 (secure 0 0 0) 0x7ffb3c04c220 con 0x7ffb4c09faa0 2024-09-15T01:36:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb4affd640 10 monclient(hunting): handle_auth_done global_id 24234 payload 824 2024-09-15T01:36:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb4affd640 1 -- 172.21.15.178:0/833413541 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb4c05ed70 con 0x7ffb4c1370c0 2024-09-15T01:36:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb4bfff640 1 --2- 172.21.15.178:0/833413541 >> v2:172.21.15.204:6812/28210 conn(0x7ffb380397b0 0x7ffb3803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.697+0000 7ffb4bfff640 10 monclient: get_auth_request con 0x7ffb380397b0 auth_method 0 2024-09-15T01:36:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb4affd640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c1370c0 0x7ffb4c11ca70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0a3290 tx=0x7ffb40032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb40031ea0 con 0x7ffb4c1370c0 2024-09-15T01:36:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb337fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb337fe640 10 monclient: dump: 2024-09-15T01:36:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.762 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb4bfff640 1 --2- 172.21.15.178:0/833413541 >> v2:172.21.15.204:6812/28210 conn(0x7ffb380397b0 0x7ffb3803bc60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400f700 tx=0x7ffb4400b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7ffb4002ac90 con 0x7ffb4c1370c0 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb337fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb337fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c1370c0 msgr2=0x7ffb4c11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb337fe640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c1370c0 0x7ffb4c11ca70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0a3290 tx=0x7ffb40032000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c1370c0 msgr2=0x7ffb4c11ca70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.698+0000 7ffb337fe640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c1370c0 0x7ffb4c11ca70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb51b9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb51b9d640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c131cd0 0x7ffb4c11cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb51b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb51b9d640 1 -- 172.21.15.178:0/833413541 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c04bc80 con 0x7ffb4c131cd0 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb4affd640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c131cd0 0x7ffb4c11cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb4affd640 10 monclient: get_auth_request con 0x7ffb4c131cd0 auth_method 0 2024-09-15T01:36:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb4affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb4affd640 10 monclient(hunting): handle_auth_done global_id 24243 payload 824 2024-09-15T01:36:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb4affd640 1 -- 172.21.15.178:0/833413541 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffb4c04b720 con 0x7ffb4c131cd0 2024-09-15T01:36:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb4affd640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c131cd0 0x7ffb4c11cfb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009910 tx=0x7ffb4003d000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb4004c070 con 0x7ffb4c131cd0 2024-09-15T01:36:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb337fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb337fe640 10 monclient: dump: 2024-09-15T01:36:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.766 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7ffb4003c440 con 0x7ffb4c131cd0 2024-09-15T01:36:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb337fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb337fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c131cd0 msgr2=0x7ffb4c11cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb337fe640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c131cd0 0x7ffb4c11cfb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009910 tx=0x7ffb4003d000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb337fe640 1 -- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c131cd0 msgr2=0x7ffb4c11cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.704+0000 7ffb337fe640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffb4c131cd0 0x7ffb4c11cfb0 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009910 tx=0x7ffb4003d000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.767 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:29.767 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:29.767 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:29.767 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:29.767 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:29.767 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:29.767 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:29.767 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:29.767 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:29.767 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:29.768 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:29.768 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:29.768 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:29.768 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:29.768 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:29.768 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.768 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:36:29.768 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:29.768 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:29.768 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:29.768 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.769 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.769 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.769 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.769 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.769 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.769 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.769 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.769 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.769 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.769 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:29.769 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:29.770 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.770 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.770 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.770 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.770 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.770 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.770 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.770 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.770 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.770 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:29.771 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:29.771 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:29.771 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:29.771 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:29.771 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:29.771 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:29.771 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:29.771 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:29.771 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:29.771 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:29.771 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:29.771 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:29.772 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:29.772 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:29.772 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:29.772 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:29.772 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.772 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.772 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.772 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.772 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.772 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.772 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.773 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.773 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.773 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.773 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:29.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.773 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:29.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.773 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:29.773 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:29.773 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:29.774 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:29.774 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.774 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.774 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:29.775 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:29.775 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.775 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.775 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.775 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:29.775 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:29.775 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.776 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.776 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:29.776 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.776 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.776 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:29.776 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.776 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.776 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:29.777 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:29.777 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.777 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.777 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.777 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:29.777 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:29.777 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.778 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:29.778 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.778 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:29.778 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.778 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.778 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.780 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.780 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:29.780 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.780 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.780 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:29.780 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.780 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.780 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.780 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:29.781 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:29.781 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.781 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.781 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.781 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:29.781 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:29.781 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.781 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.782 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.782 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:29.782 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.782 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.782 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:29.782 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.782 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.782 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.783 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.784 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:29.784 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.784 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.784 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.784 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:29.784 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.784 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.784 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.784 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:29.785 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:29.785 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.785 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.785 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.785 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.785 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.785 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:29.785 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:29.785 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.785 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.785 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.786 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.786 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:29.786 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.786 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.786 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:29.786 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.786 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.786 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.787 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.788 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:29.788 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.788 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.788 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.788 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.788 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:29.788 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.788 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.788 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.788 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.788 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:29.789 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:29.789 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.789 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.789 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.789 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:29.789 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:29.789 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.789 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.789 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:29.791 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.791 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.791 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.791 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.791 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.791 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:29.791 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:29.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.791 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:29.791 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:29.791 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:29.792 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.792 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.792 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:29.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.792 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:29.792 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:29.792 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:29.792 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.792 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.792 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:29.792 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:29.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.705+0000 7ffb51b9d640 1 -- 172.21.15.178:0/833413541 >> v2:172.21.15.204:6812/28210 conn(0x7ffb380397b0 msgr2=0x7ffb3803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.705+0000 7ffb51b9d640 1 --2- 172.21.15.178:0/833413541 >> v2:172.21.15.204:6812/28210 conn(0x7ffb380397b0 0x7ffb3803bc60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400f700 tx=0x7ffb4400b040 comp rx=0 tx=0).stop 2024-09-15T01:36:29.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.705+0000 7ffb51b9d640 10 monclient: shutdown 2024-09-15T01:36:29.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.705+0000 7ffb51b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.705+0000 7ffb51b9d640 1 -- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 msgr2=0x7ffb4c09b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.705+0000 7ffb51b9d640 1 --2- 172.21.15.178:0/833413541 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ffb4c09faa0 0x7ffb4c09b000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00ea80 tx=0x7ffb3c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.705+0000 7ffb4bfff640 1 -- 172.21.15.178:0/833413541 reap_dead start 2024-09-15T01:36:29.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.705+0000 7ffb4bfff640 1 -- 172.21.15.178:0/833413541 reap_dead start 2024-09-15T01:36:29.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.705+0000 7ffb51b9d640 1 -- 172.21.15.178:0/833413541 shutdown_connections 2024-09-15T01:36:29.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.705+0000 7ffb51b9d640 1 -- 172.21.15.178:0/833413541 >> 172.21.15.178:0/833413541 conn(0x7ffb4c0a5ed0 msgr2=0x7ffb4c09e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.705+0000 7ffb51b9d640 1 -- 172.21.15.178:0/833413541 shutdown_connections 2024-09-15T01:36:29.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.705+0000 7ffb51b9d640 1 -- 172.21.15.178:0/833413541 wait complete. 2024-09-15T01:36:29.795 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:29.795 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:36:29.795 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:36:29.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.860+0000 7f70e1d14640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-09-15T01:36:29.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.860+0000 7f70e1d14640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-09-15T01:36:29.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.860+0000 7f70e1d14640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:29.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:29.860+0000 7f70e1d14640 1 -- 172.21.15.204:0/4051040635 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush move", "name": "barr", "args": ["root=default"]} v 0) v1 -- 0x7f70dc0a1f10 con 0x7f70dc0a2e30 2024-09-15T01:36:29.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.887+0000 7fe1d666f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:29.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.887+0000 7fe1d666f640 10 monclient: build_initial_monmap 2024-09-15T01:36:29.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.887+0000 7fe1d666f640 10 monclient: monmap: 2024-09-15T01:36:29.889 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:29.889 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.889 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:29.888694+0000 2024-09-15T01:36:29.890 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:29.888694+0000 2024-09-15T01:36:29.890 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:29.890 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.890 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:29.890 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:29.890 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:29.890 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:29.890 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:29.890 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:29.890 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:29.891 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:29.891 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:29.891 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:29.891 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:29.891 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.888+0000 7fe1d666f640 1 Processor -- start 2024-09-15T01:36:29.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.888+0000 7fe1d666f640 1 -- start start 2024-09-15T01:36:29.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.888+0000 7fe1d666f640 10 monclient: init 2024-09-15T01:36:29.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:29.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 10 monclient: _add_conns ranks=[7,1,6,3,0,9,2,8,4,10,5] 2024-09-15T01:36:29.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe1d00a2930 0x7fe1d0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 10 monclient(hunting): picked mon.noname-i con 0x7fe1d00a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:29.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe1d009faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:29.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1d009fe70 0x7fe1d00a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 10 monclient(hunting): picked mon.noname-d con 0x7fe1d009fe70 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d004bf60 con 0x7fe1d009faa0 2024-09-15T01:36:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d009dd80 con 0x7fe1d009fe70 2024-09-15T01:36:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d00a2d00 con 0x7fe1d00a2930 2024-09-15T01:36:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d666f640 10 monclient(hunting): authenticate will time out at 994.639268s 2024-09-15T01:36:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1d009fe70 0x7fe1d00a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1d009fe70 0x7fe1d00a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:56298/0 (socket says 172.21.15.178:56298) 2024-09-15T01:36:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 1 -- 172.21.15.178:0/698610105 learned_addr learned my addr 172.21.15.178:0/698610105 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 10 monclient(hunting): get_auth_request con 0x7fe1d009fe70 auth_method 0 2024-09-15T01:36:29.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1cffff640 1 --2- 172.21.15.178:0/698610105 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe1d00a2930 0x7fe1d0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1cffff640 10 monclient(hunting): get_auth_request con 0x7fe1d00a2930 auth_method 0 2024-09-15T01:36:29.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 10 monclient(hunting): handle_auth_done global_id 24252 payload 274 2024-09-15T01:36:29.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 1 -- 172.21.15.178:0/698610105 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe1d00a2930 msgr2=0x7fe1d0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:29.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 1 --2- 172.21.15.178:0/698610105 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe1d00a2930 0x7fe1d0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 1 -- 172.21.15.178:0/698610105 >> v1:172.21.15.204:6789/0 conn(0x7fe1d009faa0 legacy=0x7fe1d00a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:29.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:29.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 1 monclient: found mon.noname-d 2024-09-15T01:36:29.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:29.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:29.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.889+0000 7fe1d4c0c640 1 -- 172.21.15.178:0/698610105 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d00a0310 con 0x7fe1d009fe70 2024-09-15T01:36:29.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1d4c0c640 1 --2- 172.21.15.178:0/698610105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1d009fe70 0x7fe1d00a0fd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009f50 tx=0x7fe1c8009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=def4117cd7837a51 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1cf7fe640 1 -- 172.21.15.178:0/698610105 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1c8007880 con 0x7fe1d009fe70 2024-09-15T01:36:29.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1cf7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:29.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1cf7fe640 10 monclient: dump: 2024-09-15T01:36:29.906 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.906 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.906 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.906 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.906 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.906 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.906 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.906 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.907 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1cf7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1cf7fe640 1 -- 172.21.15.178:0/698610105 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c802d050 con 0x7fe1d009fe70 2024-09-15T01:36:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1d666f640 5 monclient: authenticate success, global_id 24252 2024-09-15T01:36:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1d666f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1cf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1d666f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1cf7fe640 1 -- 172.21.15.178:0/698610105 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1c8030360 con 0x7fe1d009fe70 2024-09-15T01:36:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1d666f640 4 set_mon_vals no callback set 2024-09-15T01:36:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1cf7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.890+0000 7fe1cf7fe640 10 monclient: dump: 2024-09-15T01:36:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.910 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.910 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.891+0000 7fe1d666f640 10 monclient: shutdown 2024-09-15T01:36:29.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.891+0000 7fe1d666f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.891+0000 7fe1d666f640 1 -- 172.21.15.178:0/698610105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1d009fe70 msgr2=0x7fe1d00a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.891+0000 7fe1d666f640 1 --2- 172.21.15.178:0/698610105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1d009fe70 0x7fe1d00a0fd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009f50 tx=0x7fe1c8009840 comp rx=0 tx=0).stop 2024-09-15T01:36:29.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.891+0000 7fe1d666f640 1 -- 172.21.15.178:0/698610105 shutdown_connections 2024-09-15T01:36:29.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.891+0000 7fe1d666f640 1 --2- 172.21.15.178:0/698610105 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe1d009fe70 0x7fe1d00a0fd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.891+0000 7fe1d666f640 1 --2- 172.21.15.178:0/698610105 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe1d00a2930 0x7fe1d0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.891+0000 7fe1d666f640 1 -- 172.21.15.178:0/698610105 >> 172.21.15.178:0/698610105 conn(0x7fe1d00a5ed0 msgr2=0x7fe1d00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.891+0000 7fe1d666f640 1 -- 172.21.15.178:0/698610105 shutdown_connections 2024-09-15T01:36:29.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.891+0000 7fe1d666f640 1 -- 172.21.15.178:0/698610105 wait complete. 2024-09-15T01:36:29.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.891+0000 7fe1d666f640 10 monclient: build_initial_monmap 2024-09-15T01:36:29.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.891+0000 7fe1d666f640 10 monclient: monmap: 2024-09-15T01:36:29.911 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:29.911 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:29.911 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:29.911 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:29.911 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:29.911 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:29.911 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:29.911 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:29.911 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:29.912 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:29.912 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:29.912 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:29.912 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:29.912 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:29.912 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:29.912 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 1 Processor -- start 2024-09-15T01:36:29.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 1 -- start start 2024-09-15T01:36:29.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 10 monclient: init 2024-09-15T01:36:29.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:29.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 10 monclient: _add_conns ranks=[0,3,4,7,1,2,8,6,5] 2024-09-15T01:36:29.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1d009faa0 0x7fe1d009b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 10 monclient(hunting): picked mon.noname-a con 0x7fe1d009faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:29.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d009fe70 0x7fe1d0131990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 10 monclient(hunting): picked mon.noname-g con 0x7fe1d009fe70 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:29.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1d00a2930 0x7fe1d0141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 10 monclient(hunting): picked mon.noname-c con 0x7fe1d00a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:29.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d00a2390 con 0x7fe1d009faa0 2024-09-15T01:36:29.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d009d630 con 0x7fe1d009fe70 2024-09-15T01:36:29.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d011ee20 con 0x7fe1d00a2930 2024-09-15T01:36:29.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:29.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d666f640 10 monclient(hunting): authenticate will time out at 994.642612s 2024-09-15T01:36:29.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1d009faa0 0x7fe1d009b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1d009faa0 0x7fe1d009b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53950/0 (socket says 172.21.15.178:53950) 2024-09-15T01:36:29.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 1 -- 172.21.15.178:0/1453380637 learned_addr learned my addr 172.21.15.178:0/1453380637 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:29.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 10 monclient(hunting): get_auth_request con 0x7fe1d009faa0 auth_method 0 2024-09-15T01:36:29.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d4c0c640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1d00a2930 0x7fe1d0141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d4c0c640 10 monclient(hunting): get_auth_request con 0x7fe1d00a2930 auth_method 0 2024-09-15T01:36:29.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d4c0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d4c0c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d4c0c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d4c0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d4c0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1d4c0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1c7fff640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d009fe70 0x7fe1d0131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 10 monclient(hunting): handle_auth_done global_id 6336 payload 824 2024-09-15T01:36:29.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 1 -- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1d00a2930 msgr2=0x7fe1d0141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1d00a2930 0x7fe1d0141eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 1 -- 172.21.15.178:0/1453380637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d009fe70 msgr2=0x7fe1d0131990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.892+0000 7fe1cffff640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d009fe70 0x7fe1d0131990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1cffff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:29.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1cffff640 1 monclient: found mon.noname-a 2024-09-15T01:36:29.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:29.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1cffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:36:29.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1cffff640 1 -- 172.21.15.178:0/1453380637 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c8009ae0 con 0x7fe1d009faa0 2024-09-15T01:36:29.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1c7fff640 10 monclient: get_auth_request con 0x7fe1d009fe70 auth_method 0 2024-09-15T01:36:29.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1c7fff640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d009fe70 0x7fe1d0131990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:36:29.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1d4c0c640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1d00a2930 0x7fe1d0141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:29.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1cffff640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1d009faa0 0x7fe1d009b2a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000d890 tx=0x7fe1c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1c000de20 con 0x7fe1d009faa0 2024-09-15T01:36:29.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ccff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ccff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:36:29.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ccff9640 10 monclient: dump: 2024-09-15T01:36:29.917 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.917 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.917 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.917 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.917 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.917 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.918 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.918 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.918 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.918 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.918 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.918 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.918 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.918 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.918 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.918 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ccff9640 10 monclient: _finish_auth 0 2024-09-15T01:36:29.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ccff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:29.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000bd50 con 0x7fe1d009faa0 2024-09-15T01:36:29.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1d666f640 5 monclient: authenticate success, global_id 6336 2024-09-15T01:36:29.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1d666f640 10 monclient: _renew_subs 2024-09-15T01:36:29.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1d666f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:29.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1d666f640 1 -- 172.21.15.178:0/1453380637 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d009bdd0 con 0x7fe1d009faa0 2024-09-15T01:36:29.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ccff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:29.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1c00148b0 con 0x7fe1d009faa0 2024-09-15T01:36:29.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ccff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ccff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:29.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ccff9640 10 monclient: dump: 2024-09-15T01:36:29.920 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.920 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.920 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.920 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.920 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.920 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.920 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.920 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.920 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.920 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.920 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.921 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.921 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.921 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.921 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.921 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1d666f640 10 monclient: _renew_subs 2024-09-15T01:36:29.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1ceffd640 4 set_mon_vals no callback set 2024-09-15T01:36:29.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1d666f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:36:29.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.893+0000 7fe1d666f640 1 -- 172.21.15.178:0/1453380637 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d011eab0 con 0x7fe1d009faa0 2024-09-15T01:36:29.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe1c0014430 con 0x7fe1d009faa0 2024-09-15T01:36:29.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1d666f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1d666f640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d011ca00 0x7fe1d01423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1d666f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1d666f640 1 -- 172.21.15.178:0/1453380637 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d011ef90 con 0x7fe1d011ca00 2024-09-15T01:36:29.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1ccff9640 1 --2- 172.21.15.178:0/1453380637 >> v2:172.21.15.204:6812/28210 conn(0x7fe1b00397b0 0x7fe1b003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6987+0+0 (secure 0 0 0) 0x7fe1c004c2a0 con 0x7fe1d009faa0 2024-09-15T01:36:29.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1c7fff640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d011ca00 0x7fe1d01423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1c7fff640 10 monclient: get_auth_request con 0x7fe1d011ca00 auth_method 0 2024-09-15T01:36:29.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1c7fff640 10 monclient(hunting): handle_auth_done global_id 6002 payload 824 2024-09-15T01:36:29.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1c7fff640 1 -- 172.21.15.178:0/1453380637 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1d004c4a0 con 0x7fe1d011ca00 2024-09-15T01:36:29.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1d4c0c640 1 --2- 172.21.15.178:0/1453380637 >> v2:172.21.15.204:6812/28210 conn(0x7fe1b00397b0 0x7fe1b003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1d4c0c640 10 monclient: get_auth_request con 0x7fe1b00397b0 auth_method 0 2024-09-15T01:36:29.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1c7fff640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d011ca00 0x7fe1d01423f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe1d009f910 tx=0x7fe1b8008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1b8010a00 con 0x7fe1d011ca00 2024-09-15T01:36:29.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.894+0000 7fe1ccff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.895+0000 7fe1ccff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:29.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.895+0000 7fe1ccff9640 10 monclient: dump: 2024-09-15T01:36:29.924 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.924 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.924 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.924 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.924 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.924 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.924 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.924 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.924 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.924 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.924 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.925 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.925 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.925 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.925 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.925 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.895+0000 7fe1d4c0c640 1 --2- 172.21.15.178:0/1453380637 >> v2:172.21.15.204:6812/28210 conn(0x7fe1b00397b0 0x7fe1b003bc60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8033000 tx=0x7fe1c8036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.895+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fe1b8010580 con 0x7fe1d011ca00 2024-09-15T01:36:29.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.895+0000 7fe1ccff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:29.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.895+0000 7fe1ccff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:29.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.895+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d011ca00 msgr2=0x7fe1d01423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.895+0000 7fe1ccff9640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d011ca00 0x7fe1d01423f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe1d009f910 tx=0x7fe1b8008040 comp rx=0 tx=0).stop 2024-09-15T01:36:29.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.895+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d011ca00 msgr2=0x7fe1d01423f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:29.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.895+0000 7fe1ccff9640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d011ca00 0x7fe1d01423f0 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe1d009f910 tx=0x7fe1b8008040 comp rx=0 tx=0).stop 2024-09-15T01:36:29.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.900+0000 7fe1d666f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:29.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.900+0000 7fe1d666f640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d0131ed0 0x7fe1d0142930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:29.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.900+0000 7fe1d666f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:29.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.900+0000 7fe1d666f640 1 -- 172.21.15.178:0/1453380637 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d011f550 con 0x7fe1d0131ed0 2024-09-15T01:36:29.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1c7fff640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d0131ed0 0x7fe1d0142930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:29.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1c7fff640 10 monclient: get_auth_request con 0x7fe1d0131ed0 auth_method 0 2024-09-15T01:36:29.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:29.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:29.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:29.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:29.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1c7fff640 10 monclient(hunting): handle_auth_done global_id 6011 payload 824 2024-09-15T01:36:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1c7fff640 1 -- 172.21.15.178:0/1453380637 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe1d004bc80 con 0x7fe1d0131ed0 2024-09-15T01:36:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1c7fff640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d0131ed0 0x7fe1d0142930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8015000 tx=0x7fe1b8018000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1b8008070 con 0x7fe1d0131ed0 2024-09-15T01:36:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1ccff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1ccff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1ccff9640 10 monclient: dump: 2024-09-15T01:36:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:29.931 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:29.931 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:29.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7fe1b8011c90 con 0x7fe1d0131ed0 2024-09-15T01:36:29.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1ccff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:29.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1ccff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:29.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d0131ed0 msgr2=0x7fe1d0142930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1ccff9640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d0131ed0 0x7fe1d0142930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8015000 tx=0x7fe1b8018000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1ccff9640 1 -- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d0131ed0 msgr2=0x7fe1d0142930 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:29.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.901+0000 7fe1ccff9640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1d0131ed0 0x7fe1d0142930 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:29.932 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:29.932 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:29.932 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:29.932 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:29.932 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:29.932 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:29.932 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:29.932 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:29.932 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:29.932 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:29.932 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.933 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.934 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.934 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.934 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.934 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.934 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.934 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.934 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:29.934 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:29.934 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.934 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.934 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.935 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.935 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.935 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.935 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.935 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.935 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.935 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.935 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:29.935 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:29.936 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:29.937 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:29.937 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:29.937 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:29.937 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:29.937 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:29.937 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:29.937 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:29.937 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:29.937 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:29.937 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:29.937 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:29.937 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.938 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:29.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.938 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:29.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.938 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:29.938 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:29.938 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.938 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:29.938 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.939 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.939 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.939 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.939 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:29.939 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.939 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.939 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.939 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.939 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:29.939 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:29.939 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.939 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.940 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.940 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:29.940 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:29.940 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.940 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.940 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.940 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:29.940 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.941 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.941 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.941 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:29.941 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.941 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.941 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.941 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:29.941 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:29.941 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.941 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.942 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.942 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.942 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:29.942 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:29.942 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.942 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.942 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.942 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.942 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:29.942 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.943 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.943 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.943 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:29.943 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.943 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.943 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.943 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.943 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:29.943 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:29.943 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.943 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.943 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.944 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.944 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:29.944 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:29.944 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.944 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.944 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.944 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.944 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:29.944 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.945 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.945 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.945 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:29.945 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.945 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.945 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.945 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.945 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:29.945 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:29.945 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.945 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.946 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.946 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:29.946 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:29.946 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.946 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.946 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.946 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:29.946 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.947 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.947 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.947 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:29.947 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.947 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.947 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.947 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:29.947 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:29.947 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.947 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.948 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.948 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.948 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:29.948 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:29.948 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.948 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.948 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.948 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:29.948 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.948 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.949 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.949 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:29.949 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.949 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.949 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.949 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.949 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:29.949 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:29.949 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.949 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.950 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.950 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.950 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.950 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:29.950 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:29.950 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.950 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.950 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.950 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.950 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:29.950 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.950 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.951 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.951 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.951 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:29.951 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.951 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.951 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.951 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.951 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:29.951 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:29.951 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.951 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.952 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.952 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:29.952 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:29.952 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.952 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:29.952 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.953 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.953 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:29.953 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.953 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.953 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.953 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:29.953 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:29.953 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.953 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.953 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.954 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.954 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:29.954 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:29.954 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:29.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:29.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.954 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:29.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:29.954 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.954 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.955 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.955 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:29.955 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:29.955 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:29.955 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.955 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.955 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:29.955 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:29.955 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:29.955 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:29.955 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:29.956 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.956 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.956 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.956 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:29.956 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:29.956 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.956 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:29.956 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:29.956 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:29.956 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.957 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:29.957 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:29.957 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:29.957 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:29.957 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:29.957 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:36:29.957 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:29.957 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:29.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:29.957 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:29.958 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:29.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.902+0000 7fe1d666f640 1 -- 172.21.15.178:0/1453380637 >> v2:172.21.15.204:6812/28210 conn(0x7fe1b00397b0 msgr2=0x7fe1b003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.902+0000 7fe1d666f640 1 --2- 172.21.15.178:0/1453380637 >> v2:172.21.15.204:6812/28210 conn(0x7fe1b00397b0 0x7fe1b003bc60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8033000 tx=0x7fe1c8036000 comp rx=0 tx=0).stop 2024-09-15T01:36:29.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.902+0000 7fe1d666f640 10 monclient: shutdown 2024-09-15T01:36:29.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.902+0000 7fe1d666f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:29.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.902+0000 7fe1d666f640 1 -- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1d009faa0 msgr2=0x7fe1d009b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:29.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.902+0000 7fe1d666f640 1 --2- 172.21.15.178:0/1453380637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1d009faa0 0x7fe1d009b2a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000d890 tx=0x7fe1c0008040 comp rx=0 tx=0).stop 2024-09-15T01:36:29.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.902+0000 7fe1d4c0c640 1 -- 172.21.15.178:0/1453380637 reap_dead start 2024-09-15T01:36:29.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.902+0000 7fe1d4c0c640 1 -- 172.21.15.178:0/1453380637 reap_dead start 2024-09-15T01:36:29.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.902+0000 7fe1d666f640 1 -- 172.21.15.178:0/1453380637 shutdown_connections 2024-09-15T01:36:29.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.902+0000 7fe1d666f640 1 -- 172.21.15.178:0/1453380637 >> 172.21.15.178:0/1453380637 conn(0x7fe1d00a5ed0 msgr2=0x7fe1d009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:29.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.902+0000 7fe1d666f640 1 -- 172.21.15.178:0/1453380637 shutdown_connections 2024-09-15T01:36:29.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:29.902+0000 7fe1d666f640 1 -- 172.21.15.178:0/1453380637 wait complete. 2024-09-15T01:36:29.960 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:29.960 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:36:29.961 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:36:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.052+0000 7f9a84f5d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.052+0000 7f9a84f5d640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.052+0000 7f9a84f5d640 10 monclient: monmap: 2024-09-15T01:36:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:30.053028+0000 2024-09-15T01:36:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:30.053028+0000 2024-09-15T01:36:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.054 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:30.055 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:30.055 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.052+0000 7f9a84f5d640 1 Processor -- start 2024-09-15T01:36:30.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.052+0000 7f9a84f5d640 1 -- start start 2024-09-15T01:36:30.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.052+0000 7f9a84f5d640 10 monclient: init 2024-09-15T01:36:30.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 10 monclient: _add_conns ranks=[5,7,1,6,2,3,0,10,4,8,9] 2024-09-15T01:36:30.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a80067130 0x7f9a80067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 10 monclient(hunting): picked mon.noname-h con 0x7f9a80067130 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:30.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9a8009d580 0x7f9a80067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 10 monclient(hunting): picked mon.noname-i con 0x7f9a8009d580 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9a8009f000 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8004bf60 con 0x7f9a8009f000 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8009e0c0 con 0x7f9a80067130 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9a80067f80 con 0x7f9a8009d580 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a84f5d640 10 monclient(hunting): authenticate will time out at 994.803555s 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7ed9d640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f9a8009f000 0x7f9a8012d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:45956/0 (socket says 172.21.15.178:45956) 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7dd9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9a8009d580 0x7f9a80067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7ed9d640 1 -- 172.21.15.178:0/3440904517 learned_addr learned my addr 172.21.15.178:0/3440904517 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:36:30.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7e59c640 1 --2- 172.21.15.178:0/3440904517 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a80067130 0x7f9a80067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7dd9b640 10 monclient(hunting): get_auth_request con 0x7f9a8009d580 auth_method 0 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7e59c640 10 monclient(hunting): get_auth_request con 0x7f9a80067130 auth_method 0 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7dd9b640 10 monclient(hunting): handle_auth_done global_id 24208 payload 274 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.053+0000 7f9a7d59a640 1 -- 172.21.15.178:0/3440904517 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749059304 0 0) 0x7f9a8004bf60 con 0x7f9a8009f000 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7dd9b640 1 -- 172.21.15.178:0/3440904517 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a80067130 msgr2=0x7f9a80067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7dd9b640 1 --2- 172.21.15.178:0/3440904517 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a80067130 0x7f9a80067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7dd9b640 1 -- 172.21.15.178:0/3440904517 >> v1:172.21.15.204:6789/0 conn(0x7f9a8009f000 legacy=0x7f9a8012d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7dd9b640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7dd9b640 1 monclient: found mon.noname-i 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7dd9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7dd9b640 1 -- 172.21.15.178:0/3440904517 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8009dde0 con 0x7f9a8009d580 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:36:30.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7e59c640 1 --2- 172.21.15.178:0/3440904517 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a80067130 0x7f9a80067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7dd9b640 1 --2- 172.21.15.178:0/3440904517 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9a8009d580 0x7f9a80067a40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009880 tx=0x7f9a740090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=24b7e0d835362f66 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 1 -- 172.21.15.178:0/3440904517 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a7402ac50 con 0x7f9a8009d580 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 10 monclient: dump: 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.069 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 1 -- 172.21.15.178:0/3440904517 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7402add0 con 0x7f9a8009d580 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a84f5d640 5 monclient: authenticate success, global_id 24208 2024-09-15T01:36:30.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a84f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:30.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 1 -- 172.21.15.178:0/3440904517 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a74031ca0 con 0x7f9a8009d580 2024-09-15T01:36:30.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a84f5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:30.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a84f5d640 4 set_mon_vals no callback set 2024-09-15T01:36:30.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:30.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.054+0000 7f9a7d59a640 10 monclient: dump: 2024-09-15T01:36:30.071 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.071 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.071 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.071 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.071 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.055+0000 7f9a84f5d640 10 monclient: shutdown 2024-09-15T01:36:30.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.055+0000 7f9a84f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.055+0000 7f9a84f5d640 1 -- 172.21.15.178:0/3440904517 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9a8009d580 msgr2=0x7f9a80067a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.055+0000 7f9a84f5d640 1 --2- 172.21.15.178:0/3440904517 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9a8009d580 0x7f9a80067a40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009880 tx=0x7f9a740090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.055+0000 7f9a84f5d640 1 -- 172.21.15.178:0/3440904517 shutdown_connections 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.055+0000 7f9a84f5d640 1 --2- 172.21.15.178:0/3440904517 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9a8009d580 0x7f9a80067a40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.055+0000 7f9a84f5d640 1 --2- 172.21.15.178:0/3440904517 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a80067130 0x7f9a80067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.055+0000 7f9a84f5d640 1 -- 172.21.15.178:0/3440904517 >> 172.21.15.178:0/3440904517 conn(0x7f9a80068a60 msgr2=0x7f9a800a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.055+0000 7f9a84f5d640 1 -- 172.21.15.178:0/3440904517 shutdown_connections 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.055+0000 7f9a84f5d640 1 -- 172.21.15.178:0/3440904517 wait complete. 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient: monmap: 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:30.073 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:30.074 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:30.074 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.074 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:30.074 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:30.074 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:30.074 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.074 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.074 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.074 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.074 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.074 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.074 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.075 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 1 Processor -- start 2024-09-15T01:36:30.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 1 -- start start 2024-09-15T01:36:30.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient: init 2024-09-15T01:36:30.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient: _add_conns ranks=[8,6,1,7,4,0,2,5,3] 2024-09-15T01:36:30.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a80067130 0x7f9a8012d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient(hunting): picked mon.noname-e con 0x7f9a80067130 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:30.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a8009d580 0x7f9a801191f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient(hunting): picked mon.noname-d con 0x7f9a8009d580 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:30.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8009f000 0x7f9a80119730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient(hunting): picked mon.noname-f con 0x7f9a8009f000 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8009cb50 con 0x7f9a8009f000 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8009d9e0 con 0x7f9a8009d580 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8011c950 con 0x7f9a80067130 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.056+0000 7f9a84f5d640 10 monclient(hunting): authenticate will time out at 994.806960s 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7e59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a80067130 0x7f9a8012d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a8009d580 0x7f9a801191f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a8009d580 0x7f9a801191f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:56314/0 (socket says 172.21.15.178:56314) 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 1 -- 172.21.15.178:0/4173235524 learned_addr learned my addr 172.21.15.178:0/4173235524 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:30.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7e59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a80067130 0x7f9a8012d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:59942/0 (socket says 172.21.15.178:59942) 2024-09-15T01:36:30.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 10 monclient(hunting): get_auth_request con 0x7f9a8009d580 auth_method 0 2024-09-15T01:36:30.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7e59c640 10 monclient(hunting): get_auth_request con 0x7f9a80067130 auth_method 0 2024-09-15T01:36:30.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7ed9d640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8009f000 0x7f9a80119730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7ed9d640 10 monclient(hunting): get_auth_request con 0x7f9a8009f000 auth_method 0 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 10 monclient(hunting): handle_auth_done global_id 24261 payload 824 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 1 -- 172.21.15.178:0/4173235524 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a80067130 msgr2=0x7f9a8012d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a80067130 0x7f9a8012d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 1 -- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8009f000 msgr2=0x7f9a80119730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8009f000 0x7f9a80119730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 1 monclient: found mon.noname-d 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 1 -- 172.21.15.178:0/4173235524 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a70009a20 con 0x7f9a8009d580 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7ed9d640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8009f000 0x7f9a80119730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7e59c640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a80067130 0x7f9a8012d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7dd9b640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a8009d580 0x7f9a801191f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9a74035040 tx=0x7f9a74032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a7403b870 con 0x7f9a8009d580 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a6e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a6e7fc640 10 monclient: dump: 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a6e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7403b9d0 con 0x7f9a8009d580 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a84f5d640 5 monclient: authenticate success, global_id 24261 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a6e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a84f5d640 10 monclient: _renew_subs 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a740313a0 con 0x7f9a8009d580 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a84f5d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a84f5d640 1 -- 172.21.15.178:0/4173235524 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a74009390 con 0x7f9a8009d580 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a6e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:30.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a6e7fc640 10 monclient: dump: 2024-09-15T01:36:30.082 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.082 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.082 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.082 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.082 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.082 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.082 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.082 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.082 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.082 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.082 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.082 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.083 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.083 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.083 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.083 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a7cd99640 4 set_mon_vals no callback set 2024-09-15T01:36:30.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a84f5d640 10 monclient: _renew_subs 2024-09-15T01:36:30.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a84f5d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:30.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.057+0000 7f9a84f5d640 1 -- 172.21.15.178:0/4173235524 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8011c670 con 0x7f9a8009d580 2024-09-15T01:36:30.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a84f5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9a74039040 con 0x7f9a8009d580 2024-09-15T01:36:30.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a84f5d640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8012da70 0x7f9a80119c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a84f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a84f5d640 1 -- 172.21.15.178:0/4173235524 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8011c800 con 0x7f9a8012da70 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a6e7fc640 1 --2- 172.21.15.178:0/4173235524 >> v2:172.21.15.204:6812/28210 conn(0x7f9a600397b0 0x7f9a6003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a7e59c640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8012da70 0x7f9a80119c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a7e59c640 10 monclient: get_auth_request con 0x7f9a8012da70 auth_method 0 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a7e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a7e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a7e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6987+0+0 (secure 0 0 0) 0x7f9a7406b520 con 0x7f9a8009d580 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a7ed9d640 1 --2- 172.21.15.178:0/4173235524 >> v2:172.21.15.204:6812/28210 conn(0x7f9a600397b0 0x7f9a6003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.058+0000 7f9a7ed9d640 10 monclient: get_auth_request con 0x7f9a600397b0 auth_method 0 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_done global_id 6688 payload 824 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a7e59c640 1 -- 172.21.15.178:0/4173235524 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a8004c4a0 con 0x7f9a8012da70 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a7e59c640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8012da70 0x7f9a80119c70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9a70002860 tx=0x7f9a7000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a7ed9d640 1 --2- 172.21.15.178:0/4173235524 >> v2:172.21.15.204:6812/28210 conn(0x7f9a600397b0 0x7f9a6003bc60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9a8009c100 tx=0x7f9a78009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a70011e10 con 0x7f9a8012da70 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a6e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a6e7fc640 10 monclient: dump: 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.085 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f9a7000f040 con 0x7f9a8012da70 2024-09-15T01:36:30.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a6e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a6e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8012da70 msgr2=0x7f9a80119c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a6e7fc640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8012da70 0x7f9a80119c70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9a70002860 tx=0x7f9a7000b040 comp rx=0 tx=0).stop 2024-09-15T01:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8012da70 msgr2=0x7f9a80119c70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:30.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.059+0000 7f9a6e7fc640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8012da70 0x7f9a80119c70 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9a70002860 tx=0x7f9a7000b040 comp rx=0 tx=0).stop 2024-09-15T01:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a84f5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a84f5d640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8011a1b0 0x7f9a80097160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a84f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a84f5d640 1 -- 172.21.15.178:0/4173235524 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a800641f0 con 0x7f9a8011a1b0 2024-09-15T01:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a7e59c640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8011a1b0 0x7f9a80097160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a7e59c640 10 monclient: get_auth_request con 0x7f9a8011a1b0 auth_method 0 2024-09-15T01:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a7e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a7e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a7e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a7e59c640 10 monclient(hunting): handle_auth_done global_id 6697 payload 824 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a7e59c640 1 -- 172.21.15.178:0/4173235524 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9a8004bc80 con 0x7f9a8011a1b0 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.065+0000 7f9a7e59c640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8011a1b0 0x7f9a80097160 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9a70002860 tx=0x7f9a7001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a70015e50 con 0x7f9a8011a1b0 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a6e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a6e7fc640 10 monclient: dump: 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.090 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9481 (secure 0 0 0) 0x7f9a70011720 con 0x7f9a8011a1b0 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a6e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:30.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a6e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:30.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8011a1b0 msgr2=0x7f9a80097160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a6e7fc640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8011a1b0 0x7f9a80097160 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9a70002860 tx=0x7f9a7001d000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a6e7fc640 1 -- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8011a1b0 msgr2=0x7f9a80097160 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:30.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a6e7fc640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8011a1b0 0x7f9a80097160 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9a70002860 tx=0x7f9a7001d000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.092 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:30.092 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:30.092 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:30.092 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:30.092 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:30.093 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:30.093 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:30.093 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:30.093 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:30.093 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:30.093 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:30.093 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:30.093 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:30.093 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:30.093 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:30.093 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.093 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.094 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:30.095 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:30.095 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:30.095 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:30.095 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:30.095 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:30.095 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:30.095 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:30.095 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:30.095 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:30.095 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:30.095 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:30.096 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:30.097 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.099 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.100 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.101 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:30.102 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.104 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:30.105 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:30.105 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.105 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.105 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.105 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:30.105 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.105 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.105 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:30.105 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.106 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.106 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.106 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:30.106 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:30.106 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.106 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.106 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.106 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:30.106 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.108 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.109 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.110 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.111 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.112 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.113 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.114 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:30.115 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:30.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a84f5d640 1 -- 172.21.15.178:0/4173235524 >> v2:172.21.15.204:6812/28210 conn(0x7f9a600397b0 msgr2=0x7f9a6003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a84f5d640 1 --2- 172.21.15.178:0/4173235524 >> v2:172.21.15.204:6812/28210 conn(0x7f9a600397b0 0x7f9a6003bc60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9a8009c100 tx=0x7f9a78009040 comp rx=0 tx=0).stop 2024-09-15T01:36:30.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a84f5d640 10 monclient: shutdown 2024-09-15T01:36:30.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a84f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a84f5d640 1 -- 172.21.15.178:0/4173235524 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a8009d580 msgr2=0x7f9a801191f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a84f5d640 1 --2- 172.21.15.178:0/4173235524 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a8009d580 0x7f9a801191f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9a74035040 tx=0x7f9a74032000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a7ed9d640 1 -- 172.21.15.178:0/4173235524 reap_dead start 2024-09-15T01:36:30.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a7ed9d640 1 -- 172.21.15.178:0/4173235524 reap_dead start 2024-09-15T01:36:30.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a84f5d640 1 -- 172.21.15.178:0/4173235524 shutdown_connections 2024-09-15T01:36:30.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a84f5d640 1 -- 172.21.15.178:0/4173235524 >> 172.21.15.178:0/4173235524 conn(0x7f9a80068a60 msgr2=0x7f9a80131ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a84f5d640 1 -- 172.21.15.178:0/4173235524 shutdown_connections 2024-09-15T01:36:30.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.066+0000 7f9a84f5d640 1 -- 172.21.15.178:0/4173235524 wait complete. 2024-09-15T01:36:30.118 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:30.118 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:36:30.118 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:36:30.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.210+0000 7f52a9f54640 10 monclient: get_monmap_and_config 2024-09-15T01:36:30.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.210+0000 7f52a9f54640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.210+0000 7f52a9f54640 10 monclient: monmap: 2024-09-15T01:36:30.212 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:30.212 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.212 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:30.211066+0000 2024-09-15T01:36:30.212 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:30.211066+0000 2024-09-15T01:36:30.212 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.212 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.212 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:30.213 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:30.213 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.213 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.213 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.213 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.213 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.213 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.213 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.213 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:30.213 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:30.213 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.210+0000 7f52a9f54640 1 Processor -- start 2024-09-15T01:36:30.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.210+0000 7f52a9f54640 1 -- start start 2024-09-15T01:36:30.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.210+0000 7f52a9f54640 10 monclient: init 2024-09-15T01:36:30.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 10 monclient: _add_conns ranks=[5,3,8,2,4,0,10,1,9,7,6] 2024-09-15T01:36:30.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f52a40a2930 0x7f52a4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 10 monclient(hunting): picked mon.noname-h con 0x7f52a40a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:30.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a409faa0 0x7f52a409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 10 monclient(hunting): picked mon.noname-g con 0x7f52a409faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:30.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a40a08c0 0x7f52a40a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 10 monclient(hunting): picked mon.noname-e con 0x7f52a40a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a409e0f0 con 0x7f52a409faa0 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52a409dd80 con 0x7f52a40a2930 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40a2d00 con 0x7f52a40a08c0 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a9f54640 10 monclient(hunting): authenticate will time out at 994.961480s 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a40a08c0 0x7f52a40a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a40a08c0 0x7f52a40a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:59946/0 (socket says 172.21.15.178:59946) 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 1 -- 172.21.15.178:0/1248897011 learned_addr learned my addr 172.21.15.178:0/1248897011 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 10 monclient(hunting): get_auth_request con 0x7f52a40a08c0 auth_method 0 2024-09-15T01:36:30.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a37fe640 1 --2- 172.21.15.178:0/1248897011 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f52a40a2930 0x7f52a4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a2ffd640 1 --2- 172.21.15.178:0/1248897011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a409faa0 0x7f52a409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a37fe640 10 monclient(hunting): get_auth_request con 0x7f52a40a2930 auth_method 0 2024-09-15T01:36:30.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a2ffd640 10 monclient(hunting): get_auth_request con 0x7f52a409faa0 auth_method 0 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 10 monclient(hunting): handle_auth_done global_id 6029 payload 274 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 1 -- 172.21.15.178:0/1248897011 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f52a40a2930 msgr2=0x7f52a4132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 1 --2- 172.21.15.178:0/1248897011 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f52a40a2930 0x7f52a4132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 1 -- 172.21.15.178:0/1248897011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a409faa0 msgr2=0x7f52a409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 1 --2- 172.21.15.178:0/1248897011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a409faa0 0x7f52a409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 1 monclient: found mon.noname-e 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.211+0000 7f52a3fff640 1 -- 172.21.15.178:0/1248897011 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a404bf60 con 0x7f52a40a08c0 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a3fff640 1 --2- 172.21.15.178:0/1248897011 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a40a08c0 0x7f52a40a0d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f529c009f50 tx=0x7f529c009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=da53ba72f4a02eae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a27fc640 1 -- 172.21.15.178:0/1248897011 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f529c007880 con 0x7f52a40a08c0 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a27fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a27fc640 10 monclient: dump: 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.228 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a27fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a27fc640 1 -- 172.21.15.178:0/1248897011 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c02d050 con 0x7f52a40a08c0 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a9f54640 5 monclient: authenticate success, global_id 6029 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a9f54640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a27fc640 1 -- 172.21.15.178:0/1248897011 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f529c030360 con 0x7f52a40a08c0 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a9f54640 4 set_mon_vals no callback set 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a27fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.212+0000 7f52a27fc640 10 monclient: dump: 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.229 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.230 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 10 monclient: shutdown 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 1 -- 172.21.15.178:0/1248897011 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a40a08c0 msgr2=0x7f52a40a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 1 --2- 172.21.15.178:0/1248897011 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a40a08c0 0x7f52a40a0d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f529c009f50 tx=0x7f529c009840 comp rx=0 tx=0).stop 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 1 -- 172.21.15.178:0/1248897011 shutdown_connections 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 1 --2- 172.21.15.178:0/1248897011 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a40a08c0 0x7f52a40a0d30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 1 --2- 172.21.15.178:0/1248897011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a409faa0 0x7f52a409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 1 --2- 172.21.15.178:0/1248897011 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f52a40a2930 0x7f52a4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 1 -- 172.21.15.178:0/1248897011 >> 172.21.15.178:0/1248897011 conn(0x7f52a40a5ed0 msgr2=0x7f52a40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 1 -- 172.21.15.178:0/1248897011 shutdown_connections 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 1 -- 172.21.15.178:0/1248897011 wait complete. 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.213+0000 7f52a9f54640 10 monclient: monmap: 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.232 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 1 Processor -- start 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 1 -- start start 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 10 monclient: init 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 10 monclient: _add_conns ranks=[8,5,7,0,6,2,1,4,3] 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a409faa0 0x7f52a409b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 10 monclient(hunting): picked mon.noname-e con 0x7f52a409faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f52a40a08c0 0x7f52a4131990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 10 monclient(hunting): picked mon.noname-h con 0x7f52a40a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f52a40a2930 0x7f52a4141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 10 monclient(hunting): picked mon.noname-i con 0x7f52a40a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40a2390 con 0x7f52a40a08c0 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f52a409d630 con 0x7f52a40a2930 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f52a409bea0 con 0x7f52a409faa0 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.214+0000 7f52a9f54640 10 monclient(hunting): authenticate will time out at 994.964993s 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a409faa0 0x7f52a409b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a409faa0 0x7f52a409b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:59948/0 (socket says 172.21.15.178:59948) 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 1 -- 172.21.15.178:0/1876596445 learned_addr learned my addr 172.21.15.178:0/1876596445 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 10 monclient(hunting): get_auth_request con 0x7f52a409faa0 auth_method 0 2024-09-15T01:36:30.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a2ffd640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f52a40a08c0 0x7f52a4131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a3fff640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f52a40a2930 0x7f52a4141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a2ffd640 10 monclient(hunting): get_auth_request con 0x7f52a40a08c0 auth_method 0 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a3fff640 10 monclient(hunting): get_auth_request con 0x7f52a40a2930 auth_method 0 2024-09-15T01:36:30.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 10 monclient(hunting): handle_auth_done global_id 6038 payload 824 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 1 -- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f52a40a2930 msgr2=0x7f52a4141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f52a40a2930 0x7f52a4141e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 1 -- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f52a40a08c0 msgr2=0x7f52a4131990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f52a40a08c0 0x7f52a4131990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 1 monclient: found mon.noname-e 2024-09-15T01:36:30.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:36:30.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 1 -- 172.21.15.178:0/1876596445 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c009ae0 con 0x7f52a409faa0 2024-09-15T01:36:30.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a37fe640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a409faa0 0x7f52a409b450 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f529400ea20 tx=0x7f52940090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f529400ce90 con 0x7f52a409faa0 2024-09-15T01:36:30.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f528b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f528b7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:36:30.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f528b7fe640 10 monclient: dump: 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.238 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f528b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f528b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5294009600 con 0x7f52a409faa0 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a9f54640 5 monclient: authenticate success, global_id 6038 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f528b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f52940138f0 con 0x7f52a409faa0 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a9f54640 10 monclient: _renew_subs 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a9f54640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a9f54640 1 -- 172.21.15.178:0/1876596445 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a411eab0 con 0x7f52a409faa0 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f528b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f528b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f528b7fe640 10 monclient: dump: 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.240 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a1ffb640 4 set_mon_vals no callback set 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a9f54640 10 monclient: _renew_subs 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a9f54640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.215+0000 7f52a9f54640 1 -- 172.21.15.178:0/1876596445 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a411d580 con 0x7f52a409faa0 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f52a9f54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f52a9f54640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a4131ed0 0x7f52a4142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f52a9f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f52a9f54640 1 -- 172.21.15.178:0/1876596445 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a411d7d0 con 0x7f52a4131ed0 2024-09-15T01:36:30.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5294013470 con 0x7f52a409faa0 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f52a2ffd640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a4131ed0 0x7f52a4142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f52a2ffd640 10 monclient: get_auth_request con 0x7f52a4131ed0 auth_method 0 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f52a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f52a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f52a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f528b7fe640 1 --2- 172.21.15.178:0/1876596445 >> v2:172.21.15.204:6812/28210 conn(0x7f52900397b0 0x7f529003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.216+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6987+0+0 (secure 0 0 0) 0x7f529404c0b0 con 0x7f52a409faa0 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_done global_id 15168 payload 824 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f52a2ffd640 1 -- 172.21.15.178:0/1876596445 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52a404c4a0 con 0x7f52a4131ed0 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f52a3fff640 1 --2- 172.21.15.178:0/1876596445 >> v2:172.21.15.204:6812/28210 conn(0x7f52900397b0 0x7f529003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f52a3fff640 10 monclient: get_auth_request con 0x7f52900397b0 auth_method 0 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f52a2ffd640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a4131ed0 0x7f52a4142380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5298011bf0 tx=0x7f52980090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f529800fc80 con 0x7f52a4131ed0 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f528b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f528b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f528b7fe640 10 monclient: dump: 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f52a3fff640 1 --2- 172.21.15.178:0/1876596445 >> v2:172.21.15.204:6812/28210 conn(0x7f52900397b0 0x7f529003bc60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f529c033000 tx=0x7f529c036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f529800fde0 con 0x7f52a4131ed0 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f528b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f528b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a4131ed0 msgr2=0x7f52a4142380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f528b7fe640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a4131ed0 0x7f52a4142380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5298011bf0 tx=0x7f52980090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a4131ed0 msgr2=0x7f52a4142380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:30.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.217+0000 7f528b7fe640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a4131ed0 0x7f52a4142380 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a9f54640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a9f54640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a41428c0 0x7f52a4096b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a9f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a9f54640 1 -- 172.21.15.178:0/1876596445 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a409b990 con 0x7f52a41428c0 2024-09-15T01:36:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a2ffd640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a41428c0 0x7f52a4096b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a2ffd640 10 monclient: get_auth_request con 0x7f52a41428c0 auth_method 0 2024-09-15T01:36:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a2ffd640 10 monclient(hunting): handle_auth_done global_id 15177 payload 824 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a2ffd640 1 -- 172.21.15.178:0/1876596445 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f52a404bc80 con 0x7f52a41428c0 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f52a2ffd640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a41428c0 0x7f52a4096b30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5298011bf0 tx=0x7f529801c000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5298024040 con 0x7f52a41428c0 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f528b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f528b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.223+0000 7f528b7fe640 10 monclient: dump: 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.248 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f52980133a0 con 0x7f52a41428c0 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f528b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f528b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a41428c0 msgr2=0x7f52a4096b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f528b7fe640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a41428c0 0x7f52a4096b30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5298011bf0 tx=0x7f529801c000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f528b7fe640 1 -- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a41428c0 msgr2=0x7f52a4096b30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:30.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f528b7fe640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52a41428c0 0x7f52a4096b30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.250 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:30.250 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:30.250 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:30.250 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:30.250 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:30.250 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:30.250 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:30.251 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:30.252 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:30.253 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:30.254 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.255 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.256 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:30.259 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.260 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.261 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:30.262 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.263 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:30.265 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.267 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.268 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.269 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:30.270 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.271 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.272 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:30.272 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.272 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:30.272 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:30.272 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:30.272 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.272 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.272 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.272 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:30.272 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:30.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f52a9f54640 1 -- 172.21.15.178:0/1876596445 >> v2:172.21.15.204:6812/28210 conn(0x7f52900397b0 msgr2=0x7f529003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f52a9f54640 1 --2- 172.21.15.178:0/1876596445 >> v2:172.21.15.204:6812/28210 conn(0x7f52900397b0 0x7f529003bc60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f529c033000 tx=0x7f529c036000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f52a9f54640 10 monclient: shutdown 2024-09-15T01:36:30.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f52a9f54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f52a9f54640 1 -- 172.21.15.178:0/1876596445 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a409faa0 msgr2=0x7f52a409b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f52a9f54640 1 --2- 172.21.15.178:0/1876596445 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f52a409faa0 0x7f52a409b450 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f529400ea20 tx=0x7f52940090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f52a3fff640 1 -- 172.21.15.178:0/1876596445 reap_dead start 2024-09-15T01:36:30.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f52a3fff640 1 -- 172.21.15.178:0/1876596445 reap_dead start 2024-09-15T01:36:30.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f52a9f54640 1 -- 172.21.15.178:0/1876596445 shutdown_connections 2024-09-15T01:36:30.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f52a9f54640 1 -- 172.21.15.178:0/1876596445 >> 172.21.15.178:0/1876596445 conn(0x7f52a40a5ed0 msgr2=0x7f52a4135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f52a9f54640 1 -- 172.21.15.178:0/1876596445 shutdown_connections 2024-09-15T01:36:30.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.224+0000 7f52a9f54640 1 -- 172.21.15.178:0/1876596445 wait complete. 2024-09-15T01:36:30.275 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:30.275 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:36:30.275 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:36:30.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.366+0000 7fac7fc80640 10 monclient: get_monmap_and_config 2024-09-15T01:36:30.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.366+0000 7fac7fc80640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.366+0000 7fac7fc80640 10 monclient: monmap: 2024-09-15T01:36:30.369 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:30.369 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.369 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:30.367752+0000 2024-09-15T01:36:30.369 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:30.367752+0000 2024-09-15T01:36:30.369 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.369 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.369 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:30.369 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:30.370 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.370 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.370 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.370 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.370 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.370 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.370 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.370 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:30.370 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:30.370 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.367+0000 7fac7fc80640 1 Processor -- start 2024-09-15T01:36:30.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.367+0000 7fac7fc80640 1 -- start start 2024-09-15T01:36:30.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.367+0000 7fac7fc80640 10 monclient: init 2024-09-15T01:36:30.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 10 monclient: _add_conns ranks=[1,9,8,4,5,7,10,3,2,0,6] 2024-09-15T01:36:30.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fac780a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:30.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fac78132de0 0x7fac78137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 10 monclient(hunting): picked mon.noname-a con 0x7fac78132de0 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:30.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fac781380e0 0x7fac7813a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 10 monclient(hunting): picked mon.noname-e con 0x7fac781380e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:30.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7804bf60 con 0x7fac780a2930 2024-09-15T01:36:30.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac7809dd80 con 0x7fac78132de0 2024-09-15T01:36:30.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fac780a2d00 con 0x7fac781380e0 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7fc80640 10 monclient(hunting): authenticate will time out at 995.118194s 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fac78132de0 0x7fac78137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fac78132de0 0x7fac78137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53966/0 (socket says 172.21.15.178:53966) 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7e21d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fac781380e0 0x7fac7813a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 1 -- 172.21.15.178:0/1456310251 learned_addr learned my addr 172.21.15.178:0/1456310251 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 10 monclient(hunting): get_auth_request con 0x7fac78132de0 auth_method 0 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7e21d640 10 monclient(hunting): get_auth_request con 0x7fac781380e0 auth_method 0 2024-09-15T01:36:30.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7e21d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7e21d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7e21d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7e21d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7e21d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7e21d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 10 monclient(hunting): handle_auth_done global_id 6345 payload 274 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 1 -- 172.21.15.178:0/1456310251 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fac781380e0 msgr2=0x7fac7813a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 1 --2- 172.21.15.178:0/1456310251 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fac781380e0 0x7fac7813a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 1 -- 172.21.15.178:0/1456310251 >> v1:172.21.15.204:6789/0 conn(0x7fac780a2930 legacy=0x7fac7809faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 1 monclient: found mon.noname-a 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7d21b640 1 -- 172.21.15.178:0/1456310251 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac781331b0 con 0x7fac78132de0 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.368+0000 7fac7e21d640 1 --2- 172.21.15.178:0/1456310251 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fac781380e0 0x7fac7813a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7d21b640 1 --2- 172.21.15.178:0/1456310251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac78132de0 0x7fac78137ba0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fac68009910 tx=0x7fac680091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71be7723d8079953 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7ca1a640 1 -- 172.21.15.178:0/1456310251 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac6802ad20 con 0x7fac78132de0 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7ca1a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7ca1a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7ca1a640 10 monclient: dump: 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.383 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7ca1a640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7ca1a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7ca1a640 1 -- 172.21.15.178:0/1456310251 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac68031820 con 0x7fac78132de0 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7fc80640 5 monclient: authenticate success, global_id 6345 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7fc80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7ca1a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7fc80640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7ca1a640 1 -- 172.21.15.178:0/1456310251 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac6802aea0 con 0x7fac78132de0 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7ca1a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7fc80640 4 set_mon_vals no callback set 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7ca1a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.369+0000 7fac7ca1a640 10 monclient: dump: 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.386 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 10 monclient: shutdown 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 1 -- 172.21.15.178:0/1456310251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac78132de0 msgr2=0x7fac78137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 1 --2- 172.21.15.178:0/1456310251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac78132de0 0x7fac78137ba0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fac68009910 tx=0x7fac680091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 1 -- 172.21.15.178:0/1456310251 shutdown_connections 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 1 --2- 172.21.15.178:0/1456310251 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fac781380e0 0x7fac7813a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 1 --2- 172.21.15.178:0/1456310251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac78132de0 0x7fac78137ba0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 1 -- 172.21.15.178:0/1456310251 >> 172.21.15.178:0/1456310251 conn(0x7fac780a5ed0 msgr2=0x7fac780a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 1 -- 172.21.15.178:0/1456310251 shutdown_connections 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 1 -- 172.21.15.178:0/1456310251 wait complete. 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 10 monclient: monmap: 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.388 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 1 Processor -- start 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.370+0000 7fac7fc80640 1 -- start start 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 10 monclient: init 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 10 monclient: _add_conns ranks=[2,1,0,3,8,7,6,4,5] 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac780a2930 0x7fac78132530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 10 monclient(hunting): picked mon.noname-b con 0x7fac780a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac78132de0 0x7fac7809a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 10 monclient(hunting): picked mon.noname-f con 0x7fac78132de0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac781380e0 0x7fac7809aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 10 monclient(hunting): picked mon.noname-a con 0x7fac781380e0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7809d630 con 0x7fac781380e0 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7809c900 con 0x7fac78132de0 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac780a2390 con 0x7fac780a2930 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7fc80640 10 monclient(hunting): authenticate will time out at 995.121410s 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7e21d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac781380e0 0x7fac7809aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac780a2930 0x7fac78132530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7e21d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac781380e0 0x7fac7809aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53978/0 (socket says 172.21.15.178:53978) 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac780a2930 0x7fac78132530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:60236/0 (socket says 172.21.15.178:60236) 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7e21d640 1 -- 172.21.15.178:0/1909441051 learned_addr learned my addr 172.21.15.178:0/1909441051 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7e21d640 10 monclient(hunting): get_auth_request con 0x7fac781380e0 auth_method 0 2024-09-15T01:36:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7e21d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7e21d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7e21d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 10 monclient(hunting): get_auth_request con 0x7fac780a2930 auth_method 0 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7d21b640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac78132de0 0x7fac7809a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7d21b640 10 monclient(hunting): get_auth_request con 0x7fac78132de0 auth_method 0 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7d21b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7d21b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7d21b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7e21d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7e21d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7e21d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7d21b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7d21b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7d21b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 10 monclient(hunting): handle_auth_done global_id 24257 payload 824 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 1 -- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac78132de0 msgr2=0x7fac7809a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac78132de0 0x7fac7809a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 1 -- 172.21.15.178:0/1909441051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac781380e0 msgr2=0x7fac7809aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac781380e0 0x7fac7809aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 1 monclient: found mon.noname-b 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 1 -- 172.21.15.178:0/1909441051 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac70009a20 con 0x7fac780a2930 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7e21d640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac781380e0 0x7fac7809aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.371+0000 7fac7da1c640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac780a2930 0x7fac78132530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fac74001eb0 tx=0x7fac74007e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac74011a10 con 0x7fac780a2930 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 10 monclient: dump: 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.393 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac74011bd0 con 0x7fac780a2930 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac7fc80640 5 monclient: authenticate success, global_id 24257 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac7fc80640 10 monclient: _renew_subs 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac7fc80640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac7fc80640 1 -- 172.21.15.178:0/1909441051 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac68009450 con 0x7fac780a2930 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac74011d40 con 0x7fac780a2930 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 10 monclient: dump: 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.395 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac7fc80640 10 monclient: _renew_subs 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac7fc80640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6ffff640 4 set_mon_vals no callback set 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac7fc80640 1 -- 172.21.15.178:0/1909441051 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7809c140 con 0x7fac780a2930 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.372+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fac74011590 con 0x7fac780a2930 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7fc80640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac6dffb640 1 --2- 172.21.15.178:0/1909441051 >> v2:172.21.15.204:6812/28210 conn(0x7fac5c0397b0 0x7fac5c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7fc80640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac7809afc0 0x7fac78096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7fc80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7fc80640 1 -- 172.21.15.178:0/1909441051 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac7804c4a0 con 0x7fac7809afc0 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6987+0+0 (secure 0 0 0) 0x7fac7404abe0 con 0x7fac780a2930 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7d21b640 1 --2- 172.21.15.178:0/1909441051 >> v2:172.21.15.204:6812/28210 conn(0x7fac5c0397b0 0x7fac5c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7e21d640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac7809afc0 0x7fac78096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7d21b640 10 monclient: get_auth_request con 0x7fac5c0397b0 auth_method 0 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7e21d640 10 monclient: get_auth_request con 0x7fac7809afc0 auth_method 0 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7e21d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7e21d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7e21d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7e21d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7e21d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7e21d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7e21d640 10 monclient(hunting): handle_auth_done global_id 6053 payload 824 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7e21d640 1 -- 172.21.15.178:0/1909441051 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac7805ed70 con 0x7fac7809afc0 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7e21d640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac7809afc0 0x7fac78096a30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fac7811e2e0 tx=0x7fac7000b040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac7d21b640 1 --2- 172.21.15.178:0/1909441051 >> v2:172.21.15.204:6812/28210 conn(0x7fac5c0397b0 0x7fac5c03bc60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fac68034000 tx=0x7fac68032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac70011e10 con 0x7fac7809afc0 2024-09-15T01:36:30.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac6dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac6dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.373+0000 7fac6dffb640 10 monclient: dump: 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.399 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.400 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.400 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.400 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.374+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fac7000f040 con 0x7fac7809afc0 2024-09-15T01:36:30.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.374+0000 7fac6dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.374+0000 7fac6dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:30.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.374+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac7809afc0 msgr2=0x7fac78096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.374+0000 7fac6dffb640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac7809afc0 0x7fac78096a30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fac7811e2e0 tx=0x7fac7000b040 comp rx=0 tx=0).stop 2024-09-15T01:36:30.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.374+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac7809afc0 msgr2=0x7fac78096a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:30.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.374+0000 7fac6dffb640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac7809afc0 0x7fac78096a30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.379+0000 7fac7fc80640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:30.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.379+0000 7fac7fc80640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac78096f70 0x7fac78097370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.379+0000 7fac7fc80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.379+0000 7fac7fc80640 1 -- 172.21.15.178:0/1909441051 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac7804bc80 con 0x7fac78096f70 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac7e21d640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac78096f70 0x7fac78097370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac7e21d640 10 monclient: get_auth_request con 0x7fac78096f70 auth_method 0 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac7e21d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac7e21d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac7e21d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac7e21d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac7e21d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac7e21d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac7e21d640 10 monclient(hunting): handle_auth_done global_id 6062 payload 824 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac7e21d640 1 -- 172.21.15.178:0/1909441051 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fac7811d900 con 0x7fac78096f70 2024-09-15T01:36:30.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac7e21d640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac78096f70 0x7fac78097370 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fac7001c000 tx=0x7fac7001d000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac7001da90 con 0x7fac78096f70 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac6dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac6dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac6dffb640 10 monclient: dump: 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.404 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9313 (secure 0 0 0) 0x7fac7001dbf0 con 0x7fac78096f70 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac6dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac6dffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac78096f70 msgr2=0x7fac78097370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac6dffb640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac78096f70 0x7fac78097370 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fac7001c000 tx=0x7fac7001d000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac6dffb640 1 -- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac78096f70 msgr2=0x7fac78097370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:30.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.380+0000 7fac6dffb640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fac78096f70 0x7fac78097370 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.406 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:30.406 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:30.406 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:30.406 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:36:30.406 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:30.406 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:30.406 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:30.406 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:30.406 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:30.407 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:30.408 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:30.409 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:30.410 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:30.411 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.412 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:30.414 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:30.417 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:30.418 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:30.420 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:30.422 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:30.425 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.426 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:30.427 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:36:30.428 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.428 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.428 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:30.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.428 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:30.428 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:30.428 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 5 2024-09-15T01:36:30.428 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.428 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.428 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:30.428 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:30.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.381+0000 7fac7fc80640 1 -- 172.21.15.178:0/1909441051 >> v2:172.21.15.204:6812/28210 conn(0x7fac5c0397b0 msgr2=0x7fac5c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.381+0000 7fac7fc80640 1 --2- 172.21.15.178:0/1909441051 >> v2:172.21.15.204:6812/28210 conn(0x7fac5c0397b0 0x7fac5c03bc60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fac68034000 tx=0x7fac68032000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.381+0000 7fac7fc80640 10 monclient: shutdown 2024-09-15T01:36:30.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.381+0000 7fac7fc80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.381+0000 7fac7fc80640 1 -- 172.21.15.178:0/1909441051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac780a2930 msgr2=0x7fac78132530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.381+0000 7fac7e21d640 1 -- 172.21.15.178:0/1909441051 reap_dead start 2024-09-15T01:36:30.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.381+0000 7fac7fc80640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac780a2930 0x7fac78132530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fac74001eb0 tx=0x7fac74007e10 comp rx=0 tx=0).stop 2024-09-15T01:36:30.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.381+0000 7fac7fc80640 1 -- 172.21.15.178:0/1909441051 shutdown_connections 2024-09-15T01:36:30.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.381+0000 7fac7fc80640 1 --2- 172.21.15.178:0/1909441051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac780a2930 0x7fac78132530 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.381+0000 7fac7fc80640 1 -- 172.21.15.178:0/1909441051 >> 172.21.15.178:0/1909441051 conn(0x7fac780a5ed0 msgr2=0x7fac781358c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.381+0000 7fac7fc80640 1 -- 172.21.15.178:0/1909441051 shutdown_connections 2024-09-15T01:36:30.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.381+0000 7fac7fc80640 1 -- 172.21.15.178:0/1909441051 wait complete. 2024-09-15T01:36:30.431 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:30.431 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:36:30.431 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:36:30.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.506+0000 7f70c37fe640 1 -- 172.21.15.204:0/4051040635 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}]=0 moved item id -19 name 'barr' to location {root=default} in crush map v177) v1 ==== 173+0+0 (secure 0 0 0) 0x7f70d401bba0 con 0x7f70dc0a2e30 2024-09-15T01:36:30.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.506+0000 7f70c37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-09-15T01:36:30.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.506+0000 7f70c37fe640 10 monclient: _finish_command 2 = system:0 moved item id -19 name 'barr' to location {root=default} in crush map 2024-09-15T01:36:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.526+0000 7fee4960f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.526+0000 7fee4960f640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.526+0000 7fee4960f640 10 monclient: monmap: 2024-09-15T01:36:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:30.526947+0000 2024-09-15T01:36:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:30.526947+0000 2024-09-15T01:36:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.529 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.529 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.529 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.529 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.529 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.529 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.529 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:30.529 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:30.529 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.526+0000 7fee4960f640 1 Processor -- start 2024-09-15T01:36:30.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.526+0000 7fee4960f640 1 -- start start 2024-09-15T01:36:30.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.526+0000 7fee4960f640 10 monclient: init 2024-09-15T01:36:30.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 10 monclient: _add_conns ranks=[10,0,4,9,6,2,3,8,5,1,7] 2024-09-15T01:36:30.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fee440678d0 0x7fee4409e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 10 monclient(hunting): picked mon.noname-f con 0x7fee440678d0 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:30.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fee4406ea30 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:30.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee441359e0 0x7fee44137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 10 monclient(hunting): picked mon.noname-c con 0x7fee441359e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4404bf60 con 0x7fee4406ea30 2024-09-15T01:36:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4406d5b0 con 0x7fee440678d0 2024-09-15T01:36:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee44067ca0 con 0x7fee441359e0 2024-09-15T01:36:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee4960f640 10 monclient(hunting): authenticate will time out at 995.277251s 2024-09-15T01:36:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee3a7fc640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fee4406ea30 0x7fee4409ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:47934/0 (socket says 172.21.15.178:47934) 2024-09-15T01:36:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee3a7fc640 1 -- 172.21.15.178:0/821771481 learned_addr learned my addr 172.21.15.178:0/821771481 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:36:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 1 --2- 172.21.15.178:0/821771481 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee441359e0 0x7fee44137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 10 monclient(hunting): get_auth_request con 0x7fee441359e0 auth_method 0 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee42ffd640 1 --2- 172.21.15.178:0/821771481 >> v2:172.21.15.204:3300/0 conn(0x7fee440678d0 0x7fee4409e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee42ffd640 10 monclient(hunting): get_auth_request con 0x7fee440678d0 auth_method 0 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee42ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee42ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee42ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee42ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 10 monclient(hunting): handle_auth_done global_id 5998 payload 274 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee427fc640 1 -- 172.21.15.178:0/821771481 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092511527 0 0) 0x7fee4404bf60 con 0x7fee4406ea30 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 1 -- 172.21.15.178:0/821771481 >> v2:172.21.15.204:3300/0 conn(0x7fee440678d0 msgr2=0x7fee4409e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 1 --2- 172.21.15.178:0/821771481 >> v2:172.21.15.204:3300/0 conn(0x7fee440678d0 0x7fee4409e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 1 -- 172.21.15.178:0/821771481 >> v1:172.21.15.178:6789/0 conn(0x7fee4406ea30 legacy=0x7fee4409ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 1 monclient: found mon.noname-c 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee437fe640 1 -- 172.21.15.178:0/821771481 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4406d890 con 0x7fee441359e0 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.527+0000 7fee427fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:36:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee437fe640 1 --2- 172.21.15.178:0/821771481 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee441359e0 0x7fee44137dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009e60 tx=0x7fee3c009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=52d4da75dd52e3ac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee427fc640 1 -- 172.21.15.178:0/821771481 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee3c0079d0 con 0x7fee441359e0 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee427fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee427fc640 10 monclient: dump: 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee427fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee4960f640 5 monclient: authenticate success, global_id 5998 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee427fc640 1 -- 172.21.15.178:0/821771481 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c02d050 con 0x7fee441359e0 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee4960f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee427fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee4960f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee4960f640 4 set_mon_vals no callback set 2024-09-15T01:36:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee427fc640 1 -- 172.21.15.178:0/821771481 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee3c007b90 con 0x7fee441359e0 2024-09-15T01:36:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee427fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.528+0000 7fee427fc640 10 monclient: dump: 2024-09-15T01:36:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.529+0000 7fee4960f640 10 monclient: shutdown 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.529+0000 7fee4960f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.529+0000 7fee4960f640 1 -- 172.21.15.178:0/821771481 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee441359e0 msgr2=0x7fee44137dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.529+0000 7fee4960f640 1 --2- 172.21.15.178:0/821771481 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee441359e0 0x7fee44137dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009e60 tx=0x7fee3c009780 comp rx=0 tx=0).stop 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.529+0000 7fee4960f640 1 -- 172.21.15.178:0/821771481 shutdown_connections 2024-09-15T01:36:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.529+0000 7fee4960f640 1 --2- 172.21.15.178:0/821771481 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee441359e0 0x7fee44137dc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.529+0000 7fee4960f640 1 --2- 172.21.15.178:0/821771481 >> v2:172.21.15.204:3300/0 conn(0x7fee440678d0 0x7fee4409e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.529+0000 7fee4960f640 1 -- 172.21.15.178:0/821771481 >> 172.21.15.178:0/821771481 conn(0x7fee440a4390 msgr2=0x7fee44067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.529+0000 7fee4960f640 1 -- 172.21.15.178:0/821771481 shutdown_connections 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.529+0000 7fee4960f640 1 -- 172.21.15.178:0/821771481 wait complete. 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.529+0000 7fee4960f640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.529+0000 7fee4960f640 10 monclient: monmap: 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:30.546 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 1 Processor -- start 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 1 -- start start 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 10 monclient: init 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 10 monclient: _add_conns ranks=[6,2,7,5,4,3,1,0,8] 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee440678d0 0x7fee4411cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 10 monclient(hunting): picked mon.noname-d con 0x7fee440678d0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee4406ea30 0x7fee4411d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 10 monclient(hunting): picked mon.noname-b con 0x7fee4406ea30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee441359e0 0x7fee4411d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 10 monclient(hunting): picked mon.noname-i con 0x7fee441359e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4411ef60 con 0x7fee4406ea30 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fee44134ad0 con 0x7fee440678d0 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fee4406ccd0 con 0x7fee441359e0 2024-09-15T01:36:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee4960f640 10 monclient(hunting): authenticate will time out at 995.280637s 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee42ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee440678d0 0x7fee4411cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee3a7fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee4406ea30 0x7fee4411d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee3a7fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee4406ea30 0x7fee4411d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:60248/0 (socket says 172.21.15.178:60248) 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee42ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee440678d0 0x7fee4411cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:56324/0 (socket says 172.21.15.178:56324) 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee3a7fc640 1 -- 172.21.15.178:0/3985809745 learned_addr learned my addr 172.21.15.178:0/3985809745 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee3a7fc640 10 monclient(hunting): get_auth_request con 0x7fee4406ea30 auth_method 0 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee3a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee3a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee3a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee42ffd640 10 monclient(hunting): get_auth_request con 0x7fee440678d0 auth_method 0 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee42ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee3a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee3a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee437fe640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee441359e0 0x7fee4411d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee3a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee42ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee42ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee42ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee437fe640 10 monclient(hunting): get_auth_request con 0x7fee441359e0 auth_method 0 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.530+0000 7fee3a7fc640 10 monclient(hunting): handle_auth_done global_id 24266 payload 824 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3a7fc640 1 -- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee441359e0 msgr2=0x7fee4411d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3a7fc640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee441359e0 0x7fee4411d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3a7fc640 1 -- 172.21.15.178:0/3985809745 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee440678d0 msgr2=0x7fee4411cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3a7fc640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee440678d0 0x7fee4411cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3a7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3a7fc640 1 monclient: found mon.noname-b 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3a7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3a7fc640 1 -- 172.21.15.178:0/3985809745 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c009a20 con 0x7fee4406ea30 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee42ffd640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee440678d0 0x7fee4411cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:30.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3a7fc640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee4406ea30 0x7fee4411d420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c5e0 tx=0x7fee2c007d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee2c0070f0 con 0x7fee4406ea30 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3b7fe640 10 monclient: dump: 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.552 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee4960f640 5 monclient: authenticate success, global_id 24266 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c007290 con 0x7fee4406ea30 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee4960f640 10 monclient: _renew_subs 2024-09-15T01:36:30.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee2c007440 con 0x7fee4406ea30 2024-09-15T01:36:30.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee4960f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:30.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee4960f640 1 -- 172.21.15.178:0/3985809745 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4406c900 con 0x7fee4406ea30 2024-09-15T01:36:30.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee41ffb640 4 set_mon_vals no callback set 2024-09-15T01:36:30.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:30.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee3b7fe640 10 monclient: dump: 2024-09-15T01:36:30.554 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.554 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.554 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.554 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee4960f640 10 monclient: _renew_subs 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee4960f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.531+0000 7fee4960f640 1 -- 172.21.15.178:0/3985809745 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4411e9e0 con 0x7fee4406ea30 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee4960f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee4960f640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee4411f5b0 0x7fee4411dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee4960f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee4960f640 1 -- 172.21.15.178:0/3985809745 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fee440a3400 con 0x7fee4411f5b0 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fee2c0075a0 con 0x7fee4406ea30 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee42ffd640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee4411f5b0 0x7fee4411dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee42ffd640 10 monclient: get_auth_request con 0x7fee4411f5b0 auth_method 0 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee42ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee42ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee42ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee3b7fe640 1 --2- 172.21.15.178:0/3985809745 >> v2:172.21.15.204:6812/28210 conn(0x7fee24039810 0x7fee2403bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee42ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fee2c04b720 con 0x7fee4406ea30 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee42ffd640 10 monclient(hunting): handle_auth_done global_id 24235 payload 824 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee42ffd640 1 -- 172.21.15.178:0/3985809745 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee4404c4a0 con 0x7fee4411f5b0 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee437fe640 1 --2- 172.21.15.178:0/3985809745 >> v2:172.21.15.204:6812/28210 conn(0x7fee24039810 0x7fee2403bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.532+0000 7fee437fe640 10 monclient: get_auth_request con 0x7fee24039810 auth_method 0 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee42ffd640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee4411f5b0 0x7fee4411dea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee34009f30 tx=0x7fee340090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee34014ad0 con 0x7fee4411f5b0 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee3b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee3b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee3b7fe640 10 monclient: dump: 2024-09-15T01:36:30.557 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.558 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.559 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.559 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee437fe640 1 --2- 172.21.15.178:0/3985809745 >> v2:172.21.15.204:6812/28210 conn(0x7fee24039810 0x7fee2403bcc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fee3c034000 tx=0x7fee3c037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fee34014c30 con 0x7fee4411f5b0 2024-09-15T01:36:30.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee3b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee3b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:30.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee4411f5b0 msgr2=0x7fee4411dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee3b7fe640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee4411f5b0 0x7fee4411dea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee34009f30 tx=0x7fee340090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee4411f5b0 msgr2=0x7fee4411dea0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:30.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.533+0000 7fee3b7fe640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee4411f5b0 0x7fee4411dea0 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee34009f30 tx=0x7fee340090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee4960f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:30.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee4960f640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee44098a40 0x7fee44098e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee4960f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee4960f640 1 -- 172.21.15.178:0/3985809745 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fee44120810 con 0x7fee44098a40 2024-09-15T01:36:30.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee42ffd640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee44098a40 0x7fee44098e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee42ffd640 10 monclient: get_auth_request con 0x7fee44098a40 auth_method 0 2024-09-15T01:36:30.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee42ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee42ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee42ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee42ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee42ffd640 10 monclient(hunting): handle_auth_done global_id 24244 payload 824 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee42ffd640 1 -- 172.21.15.178:0/3985809745 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fee4404bc80 con 0x7fee44098a40 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee42ffd640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee44098a40 0x7fee44098e40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee34009f30 tx=0x7fee3401c000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee34024040 con 0x7fee44098a40 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee3b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee3b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.540+0000 7fee3b7fe640 10 monclient: dump: 2024-09-15T01:36:30.562 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.563 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.564 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7fee3401caa0 con 0x7fee44098a40 2024-09-15T01:36:30.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee3b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:30.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee3b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:30.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee44098a40 msgr2=0x7fee44098e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee3b7fe640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee44098a40 0x7fee44098e40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee34009f30 tx=0x7fee3401c000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee3b7fe640 1 -- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee44098a40 msgr2=0x7fee44098e40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:30.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee3b7fe640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee44098a40 0x7fee44098e40 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee34009f30 tx=0x7fee3401c000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.564 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:30.564 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 58, 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:36:30.565 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 7, 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:30.566 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:30.567 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:30.568 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:30.569 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:30.570 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.571 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.572 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:30.573 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:30.574 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.574 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.574 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.574 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:30.574 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.574 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.574 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:30.574 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:30.575 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.578 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.580 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.581 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:30.583 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:30.585 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:30.586 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:30.587 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:30.587 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:30.587 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:30.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:30.587 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:30.587 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:30.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee4960f640 1 -- 172.21.15.178:0/3985809745 >> v2:172.21.15.204:6812/28210 conn(0x7fee24039810 msgr2=0x7fee2403bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee4960f640 1 --2- 172.21.15.178:0/3985809745 >> v2:172.21.15.204:6812/28210 conn(0x7fee24039810 0x7fee2403bcc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fee3c034000 tx=0x7fee3c037000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee4960f640 10 monclient: shutdown 2024-09-15T01:36:30.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee4960f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee4960f640 1 -- 172.21.15.178:0/3985809745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee4406ea30 msgr2=0x7fee4411d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee4960f640 1 --2- 172.21.15.178:0/3985809745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee4406ea30 0x7fee4411d420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c5e0 tx=0x7fee2c007d00 comp rx=0 tx=0).stop 2024-09-15T01:36:30.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee437fe640 1 -- 172.21.15.178:0/3985809745 reap_dead start 2024-09-15T01:36:30.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee437fe640 1 -- 172.21.15.178:0/3985809745 reap_dead start 2024-09-15T01:36:30.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee4960f640 1 -- 172.21.15.178:0/3985809745 shutdown_connections 2024-09-15T01:36:30.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee4960f640 1 -- 172.21.15.178:0/3985809745 >> 172.21.15.178:0/3985809745 conn(0x7fee440a4390 msgr2=0x7fee4409cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee4960f640 1 -- 172.21.15.178:0/3985809745 shutdown_connections 2024-09-15T01:36:30.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.541+0000 7fee4960f640 1 -- 172.21.15.178:0/3985809745 wait complete. 2024-09-15T01:36:30.590 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:36:30.590 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c', 'e', 'a', 'b'] 2024-09-15T01:36:30.590 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T01:36:30.590 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-15T01:36:30.590 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c store 2024-09-15T01:36:30.590 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c sync_force --yes-i-really-mean-it 2024-09-15T01:36:30.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.608+0000 7f70e1d14640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-09-15T01:36:30.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.608+0000 7f70e1d14640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-09-15T01:36:30.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.608+0000 7f70e1d14640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:30.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.608+0000 7f70e1d14640 1 -- 172.21.15.204:0/4051040635 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush move", "name": "barr", "args": ["root=default"]} v 0) v1 -- 0x7f70dc0008d0 con 0x7f70dc0a2e30 2024-09-15T01:36:30.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.608+0000 7f70c37fe640 1 -- 172.21.15.204:0/4051040635 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}]=0 no need to move item id -19 name 'barr' to location {root=default} in crush map v177) v1 ==== 183+0+0 (secure 0 0 0) 0x7f70d40192c0 con 0x7f70dc0a2e30 2024-09-15T01:36:30.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.608+0000 7f70c37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-09-15T01:36:30.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.608+0000 7f70c37fe640 10 monclient: _finish_command 3 = system:0 no need to move item id -19 name 'barr' to location {root=default} in crush map 2024-09-15T01:36:30.611 INFO:tasks.workunit.client.0.smithi204.stderr:no need to move item id -19 name 'barr' to location {root=default} in crush map 2024-09-15T01:36:30.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 1 -- 172.21.15.204:0/4051040635 >> v2:172.21.15.204:6812/28210 conn(0x7f70c80397b0 msgr2=0x7f70c803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 1 --2- 172.21.15.204:0/4051040635 >> v2:172.21.15.204:6812/28210 conn(0x7f70c80397b0 0x7f70c803bc60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009fd0 tx=0x7f70cc00b040 comp rx=0 tx=0).stop 2024-09-15T01:36:30.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 10 monclient: shutdown 2024-09-15T01:36:30.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 1 -- 172.21.15.204:0/4051040635 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc0a2e30 msgr2=0x7f70dc11d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 1 --2- 172.21.15.204:0/4051040635 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc0a2e30 0x7f70dc11d720 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f70d400e930 tx=0x7f70d40090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 1 -- 172.21.15.204:0/4051040635 shutdown_connections 2024-09-15T01:36:30.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 1 --2- 172.21.15.204:0/4051040635 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f70dc0a2e30 0x7f70dc11d720 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 1 --2- 172.21.15.204:0/4051040635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70dc0a0a40 0x7f70dc11d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 1 --2- 172.21.15.204:0/4051040635 >> v2:172.21.15.204:6812/28210 conn(0x7f70c80397b0 0x7f70c803bc60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 1 --2- 172.21.15.204:0/4051040635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70dc09fb80 0x7f70dc130700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 1 -- 172.21.15.204:0/4051040635 >> 172.21.15.204:0/4051040635 conn(0x7f70dc06e860 msgr2=0x7f70dc09ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 1 -- 172.21.15.204:0/4051040635 shutdown_connections 2024-09-15T01:36:30.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.610+0000 7f70e1d14640 1 -- 172.21.15.204:0/4051040635 wait complete. 2024-09-15T01:36:30.621 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush move fooo rack=barr 2024-09-15T01:36:30.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.683+0000 7fee62c80640 10 monclient: get_monmap_and_config 2024-09-15T01:36:30.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.683+0000 7fee62c80640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.683+0000 7fee62c80640 10 monclient: monmap: 2024-09-15T01:36:30.685 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:30.685 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.685 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:30.684246+0000 2024-09-15T01:36:30.685 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:30.684246+0000 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:30.686 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:30.687 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 1 Processor -- start 2024-09-15T01:36:30.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 1 -- start start 2024-09-15T01:36:30.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 10 monclient: init 2024-09-15T01:36:30.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 10 monclient: _add_conns ranks=[10,4,7,8,2,6,1,9,5,0,3] 2024-09-15T01:36:30.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fee5c12f5d0 0x7fee5c12f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 10 monclient(hunting): picked mon.noname-f con 0x7fee5c12f5d0 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c06e510 0x7fee5c09d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 10 monclient(hunting): picked mon.noname-c con 0x7fee5c06e510 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee5c06e8e0 0x7fee5c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 10 monclient(hunting): picked mon.noname-i con 0x7fee5c06e8e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5c09f2c0 con 0x7fee5c12f5d0 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c06b850 con 0x7fee5c06e510 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c06c740 con 0x7fee5c06e8e0 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee62c80640 10 monclient(hunting): authenticate will time out at 995.434641s 2024-09-15T01:36:30.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee5bfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c06e510 0x7fee5c09d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee5bfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c06e510 0x7fee5c09d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:48502/0 (socket says 172.21.15.178:48502) 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee5bfff640 1 -- 172.21.15.178:0/229395474 learned_addr learned my addr 172.21.15.178:0/229395474 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee5bfff640 10 monclient(hunting): get_auth_request con 0x7fee5c06e510 auth_method 0 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee5bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee60a1c640 1 --2- 172.21.15.178:0/229395474 >> v2:172.21.15.204:3300/0 conn(0x7fee5c12f5d0 0x7fee5c12f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee5bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee5bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee6121d640 1 --2- 172.21.15.178:0/229395474 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee5c06e8e0 0x7fee5c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee5bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee60a1c640 10 monclient(hunting): get_auth_request con 0x7fee5c12f5d0 auth_method 0 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee60a1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee60a1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee60a1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee6121d640 10 monclient(hunting): get_auth_request con 0x7fee5c06e8e0 auth_method 0 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee6121d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee6121d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee6121d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee60a1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee60a1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee60a1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee6121d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee6121d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee6121d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.684+0000 7fee5bfff640 10 monclient(hunting): handle_auth_done global_id 6007 payload 274 2024-09-15T01:36:30.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5bfff640 1 -- 172.21.15.178:0/229395474 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee5c06e8e0 msgr2=0x7fee5c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5bfff640 1 --2- 172.21.15.178:0/229395474 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee5c06e8e0 0x7fee5c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5bfff640 1 -- 172.21.15.178:0/229395474 >> v2:172.21.15.204:3300/0 conn(0x7fee5c12f5d0 msgr2=0x7fee5c12f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5bfff640 1 --2- 172.21.15.178:0/229395474 >> v2:172.21.15.204:3300/0 conn(0x7fee5c12f5d0 0x7fee5c12f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5bfff640 1 monclient: found mon.noname-c 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5bfff640 1 -- 172.21.15.178:0/229395474 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c04bf60 con 0x7fee5c06e510 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5bfff640 1 --2- 172.21.15.178:0/229395474 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c06e510 0x7fee5c09d580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fee48009910 tx=0x7fee480091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=41d282e8bfd81ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5b7fe640 1 -- 172.21.15.178:0/229395474 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee4802ac80 con 0x7fee5c06e510 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:36:30.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5b7fe640 10 monclient: dump: 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.701 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5b7fe640 1 -- 172.21.15.178:0/229395474 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4802ae00 con 0x7fee5c06e510 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee62c80640 5 monclient: authenticate success, global_id 6007 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee62c80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee62c80640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5b7fe640 1 -- 172.21.15.178:0/229395474 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee48031ca0 con 0x7fee5c06e510 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee62c80640 4 set_mon_vals no callback set 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.685+0000 7fee5b7fe640 10 monclient: dump: 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.703 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 10 monclient: shutdown 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 1 -- 172.21.15.178:0/229395474 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c06e510 msgr2=0x7fee5c09d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 1 --2- 172.21.15.178:0/229395474 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c06e510 0x7fee5c09d580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fee48009910 tx=0x7fee480091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 1 -- 172.21.15.178:0/229395474 shutdown_connections 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 1 --2- 172.21.15.178:0/229395474 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee5c06e8e0 0x7fee5c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 1 --2- 172.21.15.178:0/229395474 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c06e510 0x7fee5c09d580 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 1 --2- 172.21.15.178:0/229395474 >> v2:172.21.15.204:3300/0 conn(0x7fee5c12f5d0 0x7fee5c12f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 1 -- 172.21.15.178:0/229395474 >> 172.21.15.178:0/229395474 conn(0x7fee5c068a60 msgr2=0x7fee5c0a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 1 -- 172.21.15.178:0/229395474 shutdown_connections 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 1 -- 172.21.15.178:0/229395474 wait complete. 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.686+0000 7fee62c80640 10 monclient: monmap: 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.705 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 1 Processor -- start 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 1 -- start start 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 10 monclient: init 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 10 monclient: _add_conns ranks=[2,3,8,4,1,6,0,7,5] 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee5c06e510 0x7fee5c141b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 10 monclient(hunting): picked mon.noname-b con 0x7fee5c06e510 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:30.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5c06e8e0 0x7fee5c09be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 10 monclient(hunting): picked mon.noname-g con 0x7fee5c06e8e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fee5c12f5d0 0x7fee5c096a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 10 monclient(hunting): picked mon.noname-e con 0x7fee5c12f5d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c06bb00 con 0x7fee5c06e510 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c09ccc0 con 0x7fee5c06e8e0 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c098a20 con 0x7fee5c12f5d0 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.687+0000 7fee62c80640 10 monclient(hunting): authenticate will time out at 995.437957s 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee5c06e510 0x7fee5c141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee6121d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fee5c12f5d0 0x7fee5c096a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee5c06e510 0x7fee5c141b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:60256/0 (socket says 172.21.15.178:60256) 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 1 -- 172.21.15.178:0/349889420 learned_addr learned my addr 172.21.15.178:0/349889420 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 10 monclient(hunting): get_auth_request con 0x7fee5c06e510 auth_method 0 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee6121d640 10 monclient(hunting): get_auth_request con 0x7fee5c12f5d0 auth_method 0 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee6121d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee6121d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee6121d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee5bfff640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5c06e8e0 0x7fee5c09be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee5bfff640 10 monclient(hunting): get_auth_request con 0x7fee5c06e8e0 auth_method 0 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee5bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee6121d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee6121d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee6121d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee5bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee5bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee5bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 10 monclient(hunting): handle_auth_done global_id 24275 payload 824 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 1 -- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fee5c12f5d0 msgr2=0x7fee5c096a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fee5c12f5d0 0x7fee5c096a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 1 -- 172.21.15.178:0/349889420 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5c06e8e0 msgr2=0x7fee5c09be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5c06e8e0 0x7fee5c09be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 1 monclient: found mon.noname-b 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 1 -- 172.21.15.178:0/349889420 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48009450 con 0x7fee5c06e510 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee6121d640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fee5c12f5d0 0x7fee5c096a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee5bfff640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5c06e8e0 0x7fee5c09be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee60a1c640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee5c06e510 0x7fee5c141b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fee5c098bf0 tx=0x7fee500090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee50009480 con 0x7fee5c06e510 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee58ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee58ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee58ff9640 10 monclient: dump: 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.711 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee58ff9640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee58ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee62c80640 5 monclient: authenticate success, global_id 24275 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5000cd60 con 0x7fee5c06e510 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee58ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee62c80640 10 monclient: _renew_subs 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee5000ced0 con 0x7fee5c06e510 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee62c80640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee62c80640 1 -- 172.21.15.178:0/349889420 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c142070 con 0x7fee5c06e510 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee58ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee58ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee5affd640 4 set_mon_vals no callback set 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee58ff9640 10 monclient: dump: 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.713 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee62c80640 10 monclient: _renew_subs 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee62c80640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.688+0000 7fee62c80640 1 -- 172.21.15.178:0/349889420 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c09a600 con 0x7fee5c06e510 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee62c80640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fee5000c8e0 con 0x7fee5c06e510 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee62c80640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c09c350 0x7fee5c096f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee62c80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee62c80640 1 -- 172.21.15.178:0/349889420 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c070cd0 con 0x7fee5c09c350 2024-09-15T01:36:30.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee5bfff640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c09c350 0x7fee5c096f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee5bfff640 10 monclient: get_auth_request con 0x7fee5c09c350 auth_method 0 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee5bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee5bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee5bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee5bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee58ff9640 1 --2- 172.21.15.178:0/349889420 >> v2:172.21.15.204:6812/28210 conn(0x7fee400397b0 0x7fee4003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fee5004bf00 con 0x7fee5c06e510 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee5bfff640 10 monclient(hunting): handle_auth_done global_id 6016 payload 824 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.689+0000 7fee5bfff640 1 -- 172.21.15.178:0/349889420 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee5c04c4a0 con 0x7fee5c09c350 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee6121d640 1 --2- 172.21.15.178:0/349889420 >> v2:172.21.15.204:6812/28210 conn(0x7fee400397b0 0x7fee4003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee6121d640 10 monclient: get_auth_request con 0x7fee400397b0 auth_method 0 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee5bfff640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c09c350 0x7fee5c096f90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0a3860 tx=0x7fee48032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee4802ac80 con 0x7fee5c09c350 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee58ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee58ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee58ff9640 10 monclient: dump: 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.716 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee6121d640 1 --2- 172.21.15.178:0/349889420 >> v2:172.21.15.204:6812/28210 conn(0x7fee400397b0 0x7fee4003bc60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fee5c1365f0 tx=0x7fee5400b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fee48031990 con 0x7fee5c09c350 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee58ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee58ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:30.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c09c350 msgr2=0x7fee5c096f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee58ff9640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c09c350 0x7fee5c096f90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0a3860 tx=0x7fee48032000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c09c350 msgr2=0x7fee5c096f90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:30.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.690+0000 7fee58ff9640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c09c350 0x7fee5c096f90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee62c80640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:36:30.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee62c80640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c0974d0 0x7fee5c12d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee62c80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee62c80640 1 -- 172.21.15.178:0/349889420 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0641f0 con 0x7fee5c0974d0 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee5bfff640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c0974d0 0x7fee5c12d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee5bfff640 10 monclient: get_auth_request con 0x7fee5c0974d0 auth_method 0 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee5bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee5bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee5bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee5bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee5bfff640 10 monclient(hunting): handle_auth_done global_id 6025 payload 824 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee5bfff640 1 -- 172.21.15.178:0/349889420 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7fee5c04bc80 con 0x7fee5c0974d0 2024-09-15T01:36:30.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee5bfff640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c0974d0 0x7fee5c12d6d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fee48009910 tx=0x7fee4803d000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee4803c990 con 0x7fee5c0974d0 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee58ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee58ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.697+0000 7fee58ff9640 10 monclient: dump: 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.723 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7fee4803c510 con 0x7fee5c0974d0 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee58ff9640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee58ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c0974d0 msgr2=0x7fee5c12d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee58ff9640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c0974d0 0x7fee5c12d6d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fee48009910 tx=0x7fee4803d000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee58ff9640 1 -- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c0974d0 msgr2=0x7fee5c12d6d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:30.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee58ff9640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee5c0974d0 0x7fee5c12d6d0 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fee48009910 tx=0x7fee4803d000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.725 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:30.725 INFO:teuthology.orchestra.run.smithi178.stdout: "ret": 0, 2024-09-15T01:36:30.725 INFO:teuthology.orchestra.run.smithi178.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-15T01:36:30.725 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:30.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee62c80640 1 -- 172.21.15.178:0/349889420 >> v2:172.21.15.204:6812/28210 conn(0x7fee400397b0 msgr2=0x7fee4003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee62c80640 1 --2- 172.21.15.178:0/349889420 >> v2:172.21.15.204:6812/28210 conn(0x7fee400397b0 0x7fee4003bc60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fee5c1365f0 tx=0x7fee5400b040 comp rx=0 tx=0).stop 2024-09-15T01:36:30.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee62c80640 10 monclient: shutdown 2024-09-15T01:36:30.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee62c80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee62c80640 1 -- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee5c06e510 msgr2=0x7fee5c141b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee62c80640 1 --2- 172.21.15.178:0/349889420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee5c06e510 0x7fee5c141b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fee5c098bf0 tx=0x7fee500090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee6121d640 1 -- 172.21.15.178:0/349889420 reap_dead start 2024-09-15T01:36:30.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee6121d640 1 -- 172.21.15.178:0/349889420 reap_dead start 2024-09-15T01:36:30.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee62c80640 1 -- 172.21.15.178:0/349889420 shutdown_connections 2024-09-15T01:36:30.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee62c80640 1 -- 172.21.15.178:0/349889420 >> 172.21.15.178:0/349889420 conn(0x7fee5c068a60 msgr2=0x7fee5c0a0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee62c80640 1 -- 172.21.15.178:0/349889420 shutdown_connections 2024-09-15T01:36:30.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.698+0000 7fee62c80640 1 -- 172.21.15.178:0/349889420 wait complete. 2024-09-15T01:36:30.729 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T01:36:30.729 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T01:36:30.730 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:36:30.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.692+0000 7fc59d59d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:30.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.692+0000 7fc59d59d640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.692+0000 7fc59d59d640 10 monclient: monmap: 2024-09-15T01:36:30.736 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:30.736 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.736 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:30.694066+0000 2024-09-15T01:36:30.736 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:30.694066+0000 2024-09-15T01:36:30.736 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:30.737 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 1 Processor -- start 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 1 -- start start 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 10 monclient: init 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 10 monclient: _add_conns ranks=[7,10,9,6,1,4,2,5,0,3,8] 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc5980a2e30 0x7fc598132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 10 monclient(hunting): picked mon.noname-i con 0x7fc5980a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc59809fb80 0x7fc59809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 10 monclient(hunting): picked mon.noname-f con 0x7fc59809fb80 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc5980a0a40 0x7fc5980a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 10 monclient(hunting): picked mon.noname-a con 0x7fc5980a0a40 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59809e6c0 con 0x7fc5980a0a40 2024-09-15T01:36:30.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5980a04b0 con 0x7fc59809fb80 2024-09-15T01:36:30.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc5980a3200 con 0x7fc5980a2e30 2024-09-15T01:36:30.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc59d59d640 10 monclient(hunting): authenticate will time out at 1006.336740s 2024-09-15T01:36:30.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc596ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc5980a2e30 0x7fc598132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc596ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc5980a2e30 0x7fc598132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:35578/0 (socket says 172.21.15.204:35578) 2024-09-15T01:36:30.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc596ffd640 1 -- 172.21.15.204:0/948591876 learned_addr learned my addr 172.21.15.204:0/948591876 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:30.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5967fc640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc59809fb80 0x7fc59809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc596ffd640 10 monclient(hunting): get_auth_request con 0x7fc5980a2e30 auth_method 0 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc596ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc596ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc596ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5967fc640 10 monclient(hunting): get_auth_request con 0x7fc59809fb80 auth_method 0 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5967fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc596ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc596ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5977fe640 1 --2- 172.21.15.204:0/948591876 >> v2:172.21.15.178:3300/0 conn(0x7fc5980a0a40 0x7fc5980a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc596ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5977fe640 10 monclient(hunting): get_auth_request con 0x7fc5980a0a40 auth_method 0 2024-09-15T01:36:30.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5977fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5967fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5967fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.693+0000 7fc5967fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc5977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc5977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc5977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc596ffd640 10 monclient(hunting): handle_auth_done global_id 24262 payload 274 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc596ffd640 1 -- 172.21.15.204:0/948591876 >> v2:172.21.15.204:3300/0 conn(0x7fc59809fb80 msgr2=0x7fc59809ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc596ffd640 1 --2- 172.21.15.204:0/948591876 >> v2:172.21.15.204:3300/0 conn(0x7fc59809fb80 0x7fc59809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc596ffd640 1 -- 172.21.15.204:0/948591876 >> v2:172.21.15.178:3300/0 conn(0x7fc5980a0a40 msgr2=0x7fc5980a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:30.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc596ffd640 1 --2- 172.21.15.204:0/948591876 >> v2:172.21.15.178:3300/0 conn(0x7fc5980a0a40 0x7fc5980a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc596ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc596ffd640 1 monclient: found mon.noname-i 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc596ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc596ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc596ffd640 1 -- 172.21.15.204:0/948591876 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59804b720 con 0x7fc5980a2e30 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc596ffd640 1 --2- 172.21.15.204:0/948591876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc5980a2e30 0x7fc598132a60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc598133920 tx=0x7fc5880091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=915b74fa514e608e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc595ffb640 1 -- 172.21.15.204:0/948591876 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc58802acb0 con 0x7fc5980a2e30 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc595ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc595ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc595ffb640 10 monclient: dump: 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.742 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:30.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc595ffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc595ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc595ffb640 1 -- 172.21.15.204:0/948591876 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58802ae30 con 0x7fc5980a2e30 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc59d59d640 5 monclient: authenticate success, global_id 24262 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc59d59d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc595ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc59d59d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc595ffb640 1 -- 172.21.15.204:0/948591876 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc588031ca0 con 0x7fc5980a2e30 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc59d59d640 4 set_mon_vals no callback set 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc595ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc595ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.694+0000 7fc595ffb640 10 monclient: dump: 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:30.744 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.745 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.695+0000 7fc59d59d640 10 monclient: shutdown 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.695+0000 7fc59d59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.695+0000 7fc59d59d640 1 -- 172.21.15.204:0/948591876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc5980a2e30 msgr2=0x7fc598132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.695+0000 7fc59d59d640 1 --2- 172.21.15.204:0/948591876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc5980a2e30 0x7fc598132a60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc598133920 tx=0x7fc5880091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.695+0000 7fc59d59d640 1 -- 172.21.15.204:0/948591876 shutdown_connections 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.695+0000 7fc59d59d640 1 --2- 172.21.15.204:0/948591876 >> v2:172.21.15.178:3300/0 conn(0x7fc5980a0a40 0x7fc5980a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.695+0000 7fc59d59d640 1 --2- 172.21.15.204:0/948591876 >> v2:172.21.15.204:3300/0 conn(0x7fc59809fb80 0x7fc59809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.695+0000 7fc59d59d640 1 --2- 172.21.15.204:0/948591876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc5980a2e30 0x7fc598132a60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.695+0000 7fc59d59d640 1 -- 172.21.15.204:0/948591876 >> 172.21.15.204:0/948591876 conn(0x7fc59806e860 msgr2=0x7fc59806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.695+0000 7fc59d59d640 1 -- 172.21.15.204:0/948591876 shutdown_connections 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.695+0000 7fc59d59d640 1 -- 172.21.15.204:0/948591876 wait complete. 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient: monmap: 2024-09-15T01:36:30.746 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.747 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 1 Processor -- start 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 1 -- start start 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient: init 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient: _add_conns ranks=[7,6,3,5,1,2,0,4,8] 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc59809fb80 0x7fc598131650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient(hunting): picked mon.noname-i con 0x7fc59809fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc5980a0a40 0x7fc59811e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient(hunting): picked mon.noname-d con 0x7fc5980a0a40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5980a2e30 0x7fc59811e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient(hunting): picked mon.noname-g con 0x7fc5980a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59809ca40 con 0x7fc5980a2e30 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc59809d650 con 0x7fc5980a0a40 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc598121a80 con 0x7fc59809fb80 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.696+0000 7fc59d59d640 10 monclient(hunting): authenticate will time out at 1006.339997s 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc59809fb80 0x7fc598131650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc59809fb80 0x7fc598131650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:35584/0 (socket says 172.21.15.204:35584) 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 1 -- 172.21.15.204:0/2831552544 learned_addr learned my addr 172.21.15.204:0/2831552544 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 10 monclient(hunting): get_auth_request con 0x7fc59809fb80 auth_method 0 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5977fe640 1 --2- 172.21.15.204:0/2831552544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5980a2e30 0x7fc59811e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5977fe640 10 monclient(hunting): get_auth_request con 0x7fc5980a2e30 auth_method 0 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5977fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5967fc640 1 --2- 172.21.15.204:0/2831552544 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc5980a0a40 0x7fc59811e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5967fc640 10 monclient(hunting): get_auth_request con 0x7fc5980a0a40 auth_method 0 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5967fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5967fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5967fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5967fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 10 monclient(hunting): handle_auth_done global_id 24271 payload 824 2024-09-15T01:36:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 1 -- 172.21.15.204:0/2831552544 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc5980a0a40 msgr2=0x7fc59811e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 1 --2- 172.21.15.204:0/2831552544 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc5980a0a40 0x7fc59811e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 1 -- 172.21.15.204:0/2831552544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5980a2e30 msgr2=0x7fc59811e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 1 --2- 172.21.15.204:0/2831552544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5980a2e30 0x7fc59811e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 1 monclient: found mon.noname-i 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 1 -- 172.21.15.204:0/2831552544 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc588009450 con 0x7fc59809fb80 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5977fe640 1 --2- 172.21.15.204:0/2831552544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5980a2e30 0x7fc59811e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc596ffd640 1 --2- 172.21.15.204:0/2831552544 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc59809fb80 0x7fc598131650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc588035040 tx=0x7fc588032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc57effd640 1 -- 172.21.15.204:0/2831552544 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc588031820 con 0x7fc59809fb80 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc57effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc57effd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc57effd640 10 monclient: dump: 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.754 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.755 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.756 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc57effd640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc57effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc57effd640 1 -- 172.21.15.204:0/2831552544 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5880319e0 con 0x7fc59809fb80 2024-09-15T01:36:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc57effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc57effd640 1 -- 172.21.15.204:0/2831552544 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc58803a390 con 0x7fc59809fb80 2024-09-15T01:36:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc59d59d640 5 monclient: authenticate success, global_id 24271 2024-09-15T01:36:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc59d59d640 10 monclient: _renew_subs 2024-09-15T01:36:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc59d59d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc59d59d640 1 -- 172.21.15.204:0/2831552544 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59811fce0 con 0x7fc59809fb80 2024-09-15T01:36:30.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc57effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc57effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc57effd640 10 monclient: dump: 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.757 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc5957fa640 4 set_mon_vals no callback set 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc59d59d640 10 monclient: _renew_subs 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc59d59d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.697+0000 7fc59d59d640 1 -- 172.21.15.204:0/2831552544 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc598121490 con 0x7fc59809fb80 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.698+0000 7fc57effd640 1 -- 172.21.15.204:0/2831552544 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc588039040 con 0x7fc59809fb80 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.698+0000 7fc59d59d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.698+0000 7fc59d59d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.698+0000 7fc59d59d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:30.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.698+0000 7fc59d59d640 1 -- 172.21.15.204:0/2831552544 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5981204c0 con 0x7fc59809fb80 2024-09-15T01:36:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.698+0000 7fc57effd640 1 --2- 172.21.15.204:0/2831552544 >> v2:172.21.15.204:6812/28210 conn(0x7fc5840397e0 0x7fc58403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.698+0000 7fc57effd640 1 -- 172.21.15.204:0/2831552544 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fc58806b090 con 0x7fc59809fb80 2024-09-15T01:36:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.698+0000 7fc5967fc640 1 --2- 172.21.15.204:0/2831552544 >> v2:172.21.15.204:6812/28210 conn(0x7fc5840397e0 0x7fc58403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.698+0000 7fc5967fc640 10 monclient: get_auth_request con 0x7fc5840397e0 auth_method 0 2024-09-15T01:36:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.699+0000 7fc5967fc640 1 --2- 172.21.15.204:0/2831552544 >> v2:172.21.15.204:6812/28210 conn(0x7fc5840397e0 0x7fc58403bc90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc5980a2e00 tx=0x7fc58c0090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.703+0000 7fc57effd640 1 -- 172.21.15.204:0/2831552544 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc58802fa80 con 0x7fc59809fb80 2024-09-15T01:36:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.703+0000 7fc57effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.703+0000 7fc57effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:30.760 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T01:36:30.760 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-09-15T01:36:30.760 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-15T01:36:30.760 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-15T01:36:30.760 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:36:30.812 INFO:tasks.ceph.mon.e:Stopped 2024-09-15T01:36:30.812 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-15T01:36:30.812 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a store 2024-09-15T01:36:30.813 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a sync_force --yes-i-really-mean-it 2024-09-15T01:36:30.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.865+0000 7fc59d59d640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-09-15T01:36:30.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.865+0000 7fc59d59d640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-09-15T01:36:30.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.865+0000 7fc59d59d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:30.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:30.865+0000 7fc59d59d640 1 -- 172.21.15.204:0/2831552544 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]} v 0) v1 -- 0x7fc598121cc0 con 0x7fc59809fb80 2024-09-15T01:36:30.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.945+0000 7f722d15d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:30.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.945+0000 7f722d15d640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.945+0000 7f722d15d640 10 monclient: monmap: 2024-09-15T01:36:30.947 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:30.947 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.947 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:30.946089+0000 2024-09-15T01:36:30.947 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:30.946089+0000 2024-09-15T01:36:30.947 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.947 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.947 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:30.948 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:30.948 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.948 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.948 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.948 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.948 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.948 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.948 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.948 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:30.948 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:30.948 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.945+0000 7f722d15d640 1 Processor -- start 2024-09-15T01:36:30.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.945+0000 7f722d15d640 1 -- start start 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.945+0000 7f722d15d640 10 monclient: init 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 10 monclient: _add_conns ranks=[10,5,4,8,3,9,6,1,2,0,7] 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f72280678d0 0x7f722809e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 10 monclient(hunting): picked mon.noname-f con 0x7f72280678d0 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f722806ea30 0x7f722809ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 10 monclient(hunting): picked mon.noname-h con 0x7f722806ea30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f722809f180 0x7f72281306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 10 monclient(hunting): picked mon.noname-c con 0x7f722809f180 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722809f5c0 con 0x7f72280678d0 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f722806d5b0 con 0x7f722809f180 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7228067ca0 con 0x7f722806ea30 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722d15d640 10 monclient(hunting): authenticate will time out at 995.696328s 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722759e640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f722809f180 msgr2=0x7f72281306b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722759e640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f722809f180 0x7f72281306b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f7226d9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f72280678d0 0x7f722809e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f722806ea30 0x7f722809ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f7226d9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f72280678d0 0x7f722809e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:39488/0 (socket says 172.21.15.178:39488) 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f722806ea30 0x7f722809ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:58576/0 (socket says 172.21.15.178:58576) 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f7226d9d640 1 -- 172.21.15.178:0/2143171798 learned_addr learned my addr 172.21.15.178:0/2143171798 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f7226d9d640 10 monclient(hunting): get_auth_request con 0x7f72280678d0 auth_method 0 2024-09-15T01:36:30.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f7226d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f7226d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f7226d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 10 monclient(hunting): get_auth_request con 0x7f722806ea30 auth_method 0 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f7226d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f7226d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f7226d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 10 monclient(hunting): handle_auth_done global_id 6071 payload 274 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 1 -- 172.21.15.178:0/2143171798 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f722809f180 msgr2=0x7f72281306b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 1 --2- 172.21.15.178:0/2143171798 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f722809f180 0x7f72281306b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 1 -- 172.21.15.178:0/2143171798 >> v2:172.21.15.204:3300/0 conn(0x7f72280678d0 msgr2=0x7f722809e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 1 --2- 172.21.15.178:0/2143171798 >> v2:172.21.15.204:3300/0 conn(0x7f72280678d0 0x7f722809e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 1 monclient: found mon.noname-h 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f722659c640 1 -- 172.21.15.178:0/2143171798 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722804bf60 con 0x7f722806ea30 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.946+0000 7f7226d9d640 1 --2- 172.21.15.178:0/2143171798 >> v2:172.21.15.204:3300/0 conn(0x7f72280678d0 0x7f722809e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f722659c640 1 --2- 172.21.15.178:0/2143171798 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f722806ea30 0x7f722809ec40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f721c009880 tx=0x7f721c0090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=8864f36e15c1ff3f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f7225d9b640 1 -- 172.21.15.178:0/2143171798 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f721c02ac70 con 0x7f722806ea30 2024-09-15T01:36:30.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f7225d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f7225d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f7225d9b640 10 monclient: dump: 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.962 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f7225d9b640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f7225d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f7225d9b640 1 -- 172.21.15.178:0/2143171798 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c02adf0 con 0x7f722806ea30 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f722d15d640 5 monclient: authenticate success, global_id 6071 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f722d15d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f7225d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f722d15d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f7225d9b640 1 -- 172.21.15.178:0/2143171798 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f721c031ca0 con 0x7f722806ea30 2024-09-15T01:36:30.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f7225d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f722d15d640 4 set_mon_vals no callback set 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f7225d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.947+0000 7f7225d9b640 10 monclient: dump: 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.964 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.948+0000 7f722d15d640 10 monclient: shutdown 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.948+0000 7f722d15d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.948+0000 7f722d15d640 1 -- 172.21.15.178:0/2143171798 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f722806ea30 msgr2=0x7f722809ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.948+0000 7f722d15d640 1 --2- 172.21.15.178:0/2143171798 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f722806ea30 0x7f722809ec40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f721c009880 tx=0x7f721c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.948+0000 7f722d15d640 1 -- 172.21.15.178:0/2143171798 shutdown_connections 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.948+0000 7f722d15d640 1 --2- 172.21.15.178:0/2143171798 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f722809f180 0x7f72281306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.948+0000 7f722d15d640 1 --2- 172.21.15.178:0/2143171798 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f722806ea30 0x7f722809ec40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.948+0000 7f722d15d640 1 --2- 172.21.15.178:0/2143171798 >> v2:172.21.15.204:3300/0 conn(0x7f72280678d0 0x7f722809e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.948+0000 7f722d15d640 1 -- 172.21.15.178:0/2143171798 >> 172.21.15.178:0/2143171798 conn(0x7f72280a4390 msgr2=0x7f7228067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.948+0000 7f722d15d640 1 -- 172.21.15.178:0/2143171798 shutdown_connections 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.948+0000 7f722d15d640 1 -- 172.21.15.178:0/2143171798 wait complete. 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient: build_initial_monmap 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient: monmap: 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:30.966 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 1 Processor -- start 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 1 -- start start 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient: init 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient: _add_conns ranks=[7,4,3,5,1,2,6,0,8] 2024-09-15T01:36:30.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f72280678d0 0x7f7228133670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient(hunting): picked mon.noname-i con 0x7f72280678d0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f722806ea30 0x7f722811d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient(hunting): picked mon.noname-c con 0x7f722806ea30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f722809f180 0x7f722811d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient(hunting): picked mon.noname-g con 0x7f722809f180 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722806cd20 con 0x7f722809f180 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f722806c900 con 0x7f722806ea30 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7228120a70 con 0x7f72280678d0 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:30.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722d15d640 10 monclient(hunting): authenticate will time out at 995.699942s 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722659c640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f722806ea30 msgr2=0x7f722811d1b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.949+0000 7f722659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f722806ea30 0x7f722811d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f72280678d0 0x7f7228133670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722759e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f722809f180 0x7f722811d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f72280678d0 0x7f7228133670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:54736/0 (socket says 172.21.15.178:54736) 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 1 -- 172.21.15.178:0/3463931578 learned_addr learned my addr 172.21.15.178:0/3463931578 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 10 monclient(hunting): get_auth_request con 0x7f72280678d0 auth_method 0 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722759e640 10 monclient(hunting): get_auth_request con 0x7f722809f180 auth_method 0 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 10 monclient(hunting): handle_auth_done global_id 24280 payload 824 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 1 -- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f722806ea30 msgr2=0x7f722811d1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f722806ea30 0x7f722811d1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 1 -- 172.21.15.178:0/3463931578 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f722809f180 msgr2=0x7f722811d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f722809f180 0x7f722811d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 1 monclient: found mon.noname-i 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 1 -- 172.21.15.178:0/3463931578 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7218009450 con 0x7f72280678d0 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f7226d9d640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f72280678d0 0x7f7228133670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f721800ece0 tx=0x7f72180091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7218009730 con 0x7f72280678d0 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f720effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f720effd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f720effd640 10 monclient: dump: 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.971 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f720effd640 10 monclient: _finish_auth 0 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f720effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722d15d640 5 monclient: authenticate success, global_id 24280 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7218013470 con 0x7f72280678d0 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722d15d640 10 monclient: _renew_subs 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722d15d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722d15d640 1 -- 172.21.15.178:0/3463931578 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c009390 con 0x7f72280678d0 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f720effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7218013a60 con 0x7f72280678d0 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f720effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f720effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f720effd640 10 monclient: dump: 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.973 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.974 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.975 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722d15d640 10 monclient: _renew_subs 2024-09-15T01:36:30.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722d15d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:30.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722d15d640 1 -- 172.21.15.178:0/3463931578 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7228120790 con 0x7f72280678d0 2024-09-15T01:36:30.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.950+0000 7f722559a640 4 set_mon_vals no callback set 2024-09-15T01:36:30.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f722d15d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f722d15d640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f722811e3f0 0x7f72280298c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f722d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f722d15d640 1 -- 172.21.15.178:0/3463931578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722804c4a0 con 0x7f722811e3f0 2024-09-15T01:36:30.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7218013be0 con 0x7f72280678d0 2024-09-15T01:36:30.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f722659c640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f722811e3f0 0x7f72280298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f722659c640 10 monclient: get_auth_request con 0x7f722811e3f0 auth_method 0 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f722659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f722659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f722659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f722659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f722659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.951+0000 7f722659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 1 --2- 172.21.15.178:0/3463931578 >> v2:172.21.15.204:6812/28210 conn(0x7f72140397b0 0x7f721403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f722659c640 10 monclient(hunting): handle_auth_done global_id 6381 payload 824 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f721804c480 con 0x7f72280678d0 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f722659c640 1 -- 172.21.15.178:0/3463931578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f722804bc80 con 0x7f722811e3f0 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f722759e640 1 --2- 172.21.15.178:0/3463931578 >> v2:172.21.15.204:6812/28210 conn(0x7f72140397b0 0x7f721403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f722759e640 10 monclient: get_auth_request con 0x7f72140397b0 auth_method 0 2024-09-15T01:36:30.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f722659c640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f722811e3f0 0x7f72280298c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7228068080 tx=0x7f721c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f721c031ea0 con 0x7f722811e3f0 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 10 monclient: dump: 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.977 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f722759e640 1 --2- 172.21.15.178:0/3463931578 >> v2:172.21.15.204:6812/28210 conn(0x7f72140397b0 0x7f721403bc60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f722811d100 tx=0x7f7220009650 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f721c02ac70 con 0x7f722811e3f0 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f722811e3f0 msgr2=0x7f72280298c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f722811e3f0 0x7f72280298c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7228068080 tx=0x7f721c032000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f722811e3f0 msgr2=0x7f72280298c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:30.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.952+0000 7f720effd640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f722811e3f0 0x7f72280298c0 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7228068080 tx=0x7f721c032000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722d15d640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:36:30.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722d15d640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7228133bb0 0x7f7228029e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:30.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722d15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:30.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722d15d640 1 -- 172.21.15.178:0/3463931578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722806c6d0 con 0x7f7228133bb0 2024-09-15T01:36:30.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722659c640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7228133bb0 0x7f7228029e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:30.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722659c640 10 monclient: get_auth_request con 0x7f7228133bb0 auth_method 0 2024-09-15T01:36:30.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:30.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:30.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:30.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:30.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722659c640 10 monclient(hunting): handle_auth_done global_id 6390 payload 824 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722659c640 1 -- 172.21.15.178:0/3463931578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7f7228140750 con 0x7f7228133bb0 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.959+0000 7f722659c640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7228133bb0 0x7f7228029e00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f721c009720 tx=0x7f721c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f721c03cd50 con 0x7f7228133bb0 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f720effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f720effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f720effd640 10 monclient: dump: 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:30.982 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7f721c044430 con 0x7f7228133bb0 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f720effd640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:36:30.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f720effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:30.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7228133bb0 msgr2=0x7f7228029e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f720effd640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7228133bb0 0x7f7228029e00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f721c009720 tx=0x7f721c03c000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f720effd640 1 -- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7228133bb0 msgr2=0x7f7228029e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:30.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f720effd640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7228133bb0 0x7f7228029e00 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f721c009720 tx=0x7f721c03c000 comp rx=0 tx=0).stop 2024-09-15T01:36:30.984 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:30.984 INFO:teuthology.orchestra.run.smithi178.stdout: "ret": 0, 2024-09-15T01:36:30.984 INFO:teuthology.orchestra.run.smithi178.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-15T01:36:30.984 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:30.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f722d15d640 1 -- 172.21.15.178:0/3463931578 >> v2:172.21.15.204:6812/28210 conn(0x7f72140397b0 msgr2=0x7f721403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f722d15d640 1 --2- 172.21.15.178:0/3463931578 >> v2:172.21.15.204:6812/28210 conn(0x7f72140397b0 0x7f721403bc60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f722811d100 tx=0x7f7220009650 comp rx=0 tx=0).stop 2024-09-15T01:36:30.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f722d15d640 10 monclient: shutdown 2024-09-15T01:36:30.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f722d15d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:30.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f722d15d640 1 -- 172.21.15.178:0/3463931578 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f72280678d0 msgr2=0x7f7228133670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:30.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f722d15d640 1 --2- 172.21.15.178:0/3463931578 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f72280678d0 0x7f7228133670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f721800ece0 tx=0x7f72180091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:30.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f722759e640 1 -- 172.21.15.178:0/3463931578 reap_dead start 2024-09-15T01:36:30.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f722759e640 1 -- 172.21.15.178:0/3463931578 reap_dead start 2024-09-15T01:36:30.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f722d15d640 1 -- 172.21.15.178:0/3463931578 shutdown_connections 2024-09-15T01:36:30.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f722d15d640 1 -- 172.21.15.178:0/3463931578 >> 172.21.15.178:0/3463931578 conn(0x7f72280a4390 msgr2=0x7f722809d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:30.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f722d15d640 1 -- 172.21.15.178:0/3463931578 shutdown_connections 2024-09-15T01:36:30.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:30.960+0000 7f722d15d640 1 -- 172.21.15.178:0/3463931578 wait complete. 2024-09-15T01:36:30.987 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T01:36:30.987 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T01:36:30.987 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:36:30.998 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T01:36:30.998 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T01:36:30.999 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b store 2024-09-15T01:36:30.999 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b sync_force --yes-i-really-mean-it 2024-09-15T01:36:31.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.131+0000 7fbfb779d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.131+0000 7fbfb779d640 10 monclient: build_initial_monmap 2024-09-15T01:36:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.131+0000 7fbfb779d640 10 monclient: monmap: 2024-09-15T01:36:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:31.132546+0000 2024-09-15T01:36:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:31.132546+0000 2024-09-15T01:36:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:31.135 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:31.135 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:31.135 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:31.135 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:31.135 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:31.135 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:31.135 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:31.135 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:31.135 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 1 Processor -- start 2024-09-15T01:36:31.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 1 -- start start 2024-09-15T01:36:31.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 10 monclient: init 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 10 monclient: _add_conns ranks=[4,0,7,3,9,8,5,2,6,10,1] 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfb00a2930 0x7fbfb0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 10 monclient(hunting): picked mon.noname-c con 0x7fbfb00a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbfb009faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfb009fe70 0x7fbfb00a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 10 monclient(hunting): picked mon.noname-i con 0x7fbfb009fe70 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb004bf60 con 0x7fbfb009faa0 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb009dd80 con 0x7fbfb00a2930 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb00a2d00 con 0x7fbfb009fe70 2024-09-15T01:36:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb779d640 10 monclient(hunting): authenticate will time out at 995.882921s 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb4d38640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7fbfb009faa0 legacy=0x7fbfb00a08c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb5539640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfb00a2930 msgr2=0x7fbfb0132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.132+0000 7fbfb5539640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfb00a2930 0x7fbfb0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfb009fe70 0x7fbfb00a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfb009fe70 0x7fbfb00a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:54742/0 (socket says 172.21.15.178:54742) 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 1 -- 172.21.15.178:0/3069393018 learned_addr learned my addr 172.21.15.178:0/3069393018 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 10 monclient(hunting): get_auth_request con 0x7fbfb009fe70 auth_method 0 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:31.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 10 monclient(hunting): handle_auth_done global_id 24289 payload 274 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 1 -- 172.21.15.178:0/3069393018 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfb00a2930 msgr2=0x7fbfb0132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 1 --2- 172.21.15.178:0/3069393018 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfb00a2930 0x7fbfb0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 1 -- 172.21.15.178:0/3069393018 >> v1:172.21.15.178:6789/0 conn(0x7fbfb009faa0 legacy=0x7fbfb00a08c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 10 monclient: _finish_hunting 0 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 1 monclient: found mon.noname-i 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.133+0000 7fbfb5d3a640 1 -- 172.21.15.178:0/3069393018 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb00a0310 con 0x7fbfb009fe70 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb5d3a640 1 --2- 172.21.15.178:0/3069393018 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfb009fe70 0x7fbfb00a0fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009f50 tx=0x7fbfa8009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=b598d242b1c7d7d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfa7fff640 1 -- 172.21.15.178:0/3069393018 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfa8007880 con 0x7fbfb009fe70 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfa7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfa7fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfa7fff640 10 monclient: dump: 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfa7fff640 10 monclient: _finish_auth 0 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfa7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfa7fff640 1 -- 172.21.15.178:0/3069393018 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa802d050 con 0x7fbfb009fe70 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb779d640 5 monclient: authenticate success, global_id 24289 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb779d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfa7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb779d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfa7fff640 1 -- 172.21.15.178:0/3069393018 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfa8030360 con 0x7fbfb009fe70 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb779d640 4 set_mon_vals no callback set 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfa7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:31.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfa7fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfa7fff640 10 monclient: dump: 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:31.150 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb779d640 10 monclient: shutdown 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb779d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb779d640 1 -- 172.21.15.178:0/3069393018 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfb009fe70 msgr2=0x7fbfb00a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb779d640 1 --2- 172.21.15.178:0/3069393018 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfb009fe70 0x7fbfb00a0fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009f50 tx=0x7fbfa8009840 comp rx=0 tx=0).stop 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb779d640 1 -- 172.21.15.178:0/3069393018 shutdown_connections 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb779d640 1 --2- 172.21.15.178:0/3069393018 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfb009fe70 0x7fbfb00a0fd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb779d640 1 --2- 172.21.15.178:0/3069393018 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfb00a2930 0x7fbfb0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.134+0000 7fbfb779d640 1 -- 172.21.15.178:0/3069393018 >> 172.21.15.178:0/3069393018 conn(0x7fbfb00a5ed0 msgr2=0x7fbfb00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.135+0000 7fbfb779d640 1 -- 172.21.15.178:0/3069393018 shutdown_connections 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.135+0000 7fbfb779d640 1 -- 172.21.15.178:0/3069393018 wait complete. 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.135+0000 7fbfb779d640 10 monclient: build_initial_monmap 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.135+0000 7fbfb779d640 10 monclient: monmap: 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.135+0000 7fbfb779d640 1 Processor -- start 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.135+0000 7fbfb779d640 1 -- start start 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.135+0000 7fbfb779d640 10 monclient: init 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 10 monclient: _add_conns ranks=[3,0,6,1,2,7,8,4,5] 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfb009faa0 0x7fbfb009b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 10 monclient(hunting): picked mon.noname-g con 0x7fbfb009faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb009fe70 0x7fbfb0131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 10 monclient(hunting): picked mon.noname-a con 0x7fbfb009fe70 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfb00a2930 0x7fbfb0141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 10 monclient(hunting): picked mon.noname-d con 0x7fbfb00a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb009d630 con 0x7fbfb009fe70 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb009bbd0 con 0x7fbfb009faa0 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb011ea40 con 0x7fbfb00a2930 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb779d640 10 monclient(hunting): authenticate will time out at 995.886463s 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb4d38640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb009fe70 msgr2=0x7fbfb0131920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb4d38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb009fe70 0x7fbfb0131920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:31.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfb00a2930 0x7fbfb0141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:31.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfb00a2930 0x7fbfb0141dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:56332/0 (socket says 172.21.15.178:56332) 2024-09-15T01:36:31.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 1 -- 172.21.15.178:0/2714919365 learned_addr learned my addr 172.21.15.178:0/2714919365 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:31.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 10 monclient(hunting): get_auth_request con 0x7fbfb00a2930 auth_method 0 2024-09-15T01:36:31.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:31.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:31.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:31.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:31.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:31.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:31.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5539640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfb009faa0 0x7fbfb009b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:31.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5539640 10 monclient(hunting): get_auth_request con 0x7fbfb009faa0 auth_method 0 2024-09-15T01:36:31.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:31.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5539640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:31.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5539640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:31.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5539640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:31.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5539640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:31.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5539640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:31.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 10 monclient(hunting): handle_auth_done global_id 24288 payload 824 2024-09-15T01:36:31.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 1 -- 172.21.15.178:0/2714919365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfb009faa0 msgr2=0x7fbfb009b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:31.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfb009faa0 0x7fbfb009b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 1 -- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb009fe70 msgr2=0x7fbfb0131920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:31.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb009fe70 0x7fbfb0131920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 10 monclient: _finish_hunting 0 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 1 monclient: found mon.noname-d 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.136+0000 7fbfb5d3a640 1 -- 172.21.15.178:0/2714919365 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa8009ae0 con 0x7fbfb00a2930 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfb5d3a640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfb00a2930 0x7fbfb0141dd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009fb0 tx=0x7fbfa8036000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfa803bb70 con 0x7fbfb00a2930 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa57fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa57fa640 10 monclient: dump: 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:31.158 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa57fa640 10 monclient: _finish_auth 0 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa803bcf0 con 0x7fbfb00a2930 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfb779d640 5 monclient: authenticate success, global_id 24288 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfb779d640 10 monclient: _renew_subs 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfb779d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfb779d640 1 -- 172.21.15.178:0/2714919365 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb011ed20 con 0x7fbfb00a2930 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfa803be60 con 0x7fbfb00a2930 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa57fa640 10 monclient: dump: 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:31.159 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:31.160 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfa77fe640 4 set_mon_vals no callback set 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfb779d640 10 monclient: _renew_subs 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfb779d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.137+0000 7fbfb779d640 1 -- 172.21.15.178:0/2714919365 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb011d580 con 0x7fbfb00a2930 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fbfa8038070 con 0x7fbfb00a2930 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb779d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb779d640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb011ca00 0x7fbfb0142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb779d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb779d640 1 -- 172.21.15.178:0/2714919365 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb011d740 con 0x7fbfb011ca00 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb5539640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb011ca00 0x7fbfb0142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb5539640 10 monclient: get_auth_request con 0x7fbfb011ca00 auth_method 0 2024-09-15T01:36:31.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfa57fa640 1 --2- 172.21.15.178:0/2714919365 >> v2:172.21.15.204:6812/28210 conn(0x7fbf940397b0 0x7fbf9403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb5539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb5539640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb5539640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fbfa802ae40 con 0x7fbfb00a2930 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb5539640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb5539640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb5539640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb5539640 10 monclient(hunting): handle_auth_done global_id 24284 payload 824 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb4d38640 1 --2- 172.21.15.178:0/2714919365 >> v2:172.21.15.204:6812/28210 conn(0x7fbf940397b0 0x7fbf9403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb5539640 1 -- 172.21.15.178:0/2714919365 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfb004c4a0 con 0x7fbfb011ca00 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb4d38640 10 monclient: get_auth_request con 0x7fbf940397b0 auth_method 0 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb5539640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb011ca00 0x7fbfb0142310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009590 tx=0x7fbfac006f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:31.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfac00eb20 con 0x7fbfb011ca00 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfa57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfa57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfa57fa640 10 monclient: dump: 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:31.163 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.138+0000 7fbfb4d38640 1 --2- 172.21.15.178:0/2714919365 >> v2:172.21.15.204:6812/28210 conn(0x7fbf940397b0 0x7fbf9403bc60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbfb009aff0 tx=0x7fbfa0006cc0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.139+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fbfac00ec80 con 0x7fbfb011ca00 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.139+0000 7fbfa57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.139+0000 7fbfa57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.139+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb011ca00 msgr2=0x7fbfb0142310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.139+0000 7fbfa57fa640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb011ca00 0x7fbfb0142310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009590 tx=0x7fbfac006f90 comp rx=0 tx=0).stop 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.139+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb011ca00 msgr2=0x7fbfb0142310 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:31.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.139+0000 7fbfa57fa640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb011ca00 0x7fbfb0142310 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009590 tx=0x7fbfac006f90 comp rx=0 tx=0).stop 2024-09-15T01:36:31.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb779d640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:36:31.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb779d640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb0131e60 0x7fbfb0142850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb779d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb779d640 1 -- 172.21.15.178:0/2714919365 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb009f940 con 0x7fbfb0131e60 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb5539640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb0131e60 0x7fbfb0142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb5539640 10 monclient: get_auth_request con 0x7fbfb0131e60 auth_method 0 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb5539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb5539640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb5539640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb5539640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb5539640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb5539640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb5539640 10 monclient(hunting): handle_auth_done global_id 24293 payload 824 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.145+0000 7fbfb5539640 1 -- 172.21.15.178:0/2714919365 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7fbfb004bc80 con 0x7fbfb0131e60 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfb5539640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb0131e60 0x7fbfb0142850 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbfac000c00 tx=0x7fbfac019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfac022070 con 0x7fbfb0131e60 2024-09-15T01:36:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfa57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfa57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfa57fa640 10 monclient: dump: 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:31.168 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:31.169 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:31.169 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:31.169 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:31.169 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7fbfac0066a0 con 0x7fbfb0131e60 2024-09-15T01:36:31.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfa57fa640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:36:31.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfa57fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:31.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb0131e60 msgr2=0x7fbfb0142850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:31.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfa57fa640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb0131e60 0x7fbfb0142850 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbfac000c00 tx=0x7fbfac019000 comp rx=0 tx=0).stop 2024-09-15T01:36:31.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfa57fa640 1 -- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb0131e60 msgr2=0x7fbfb0142850 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:31.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfa57fa640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb0131e60 0x7fbfb0142850 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.170 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:31.170 INFO:teuthology.orchestra.run.smithi178.stdout: "ret": 0, 2024-09-15T01:36:31.170 INFO:teuthology.orchestra.run.smithi178.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-15T01:36:31.170 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:31.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfb779d640 1 -- 172.21.15.178:0/2714919365 >> v2:172.21.15.204:6812/28210 conn(0x7fbf940397b0 msgr2=0x7fbf9403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:31.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfb779d640 1 --2- 172.21.15.178:0/2714919365 >> v2:172.21.15.204:6812/28210 conn(0x7fbf940397b0 0x7fbf9403bc60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbfb009aff0 tx=0x7fbfa0006cc0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfb779d640 10 monclient: shutdown 2024-09-15T01:36:31.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfb779d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:31.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfb779d640 1 -- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfb00a2930 msgr2=0x7fbfb0141dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:31.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfb5d3a640 1 -- 172.21.15.178:0/2714919365 reap_dead start 2024-09-15T01:36:31.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.146+0000 7fbfb779d640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfb00a2930 0x7fbfb0141dd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009fb0 tx=0x7fbfa8036000 comp rx=0 tx=0).stop 2024-09-15T01:36:31.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.147+0000 7fbfb779d640 1 -- 172.21.15.178:0/2714919365 shutdown_connections 2024-09-15T01:36:31.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.147+0000 7fbfb779d640 1 --2- 172.21.15.178:0/2714919365 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfb00a2930 0x7fbfb0141dd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.147+0000 7fbfb779d640 1 -- 172.21.15.178:0/2714919365 >> 172.21.15.178:0/2714919365 conn(0x7fbfb00a5ed0 msgr2=0x7fbfb0135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:31.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.147+0000 7fbfb779d640 1 -- 172.21.15.178:0/2714919365 shutdown_connections 2024-09-15T01:36:31.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.147+0000 7fbfb779d640 1 -- 172.21.15.178:0/2714919365 wait complete. 2024-09-15T01:36:31.172 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T01:36:31.172 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T01:36:31.172 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:36:31.183 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T01:36:31.183 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 5 2024-09-15T01:36:31.183 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:36:31.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.313+0000 7f697b31d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:31.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.313+0000 7f697b31d640 10 monclient: build_initial_monmap 2024-09-15T01:36:31.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.313+0000 7f697b31d640 10 monclient: monmap: 2024-09-15T01:36:31.316 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:31.316 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:31.316 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:31.314572+0000 2024-09-15T01:36:31.316 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:31.314572+0000 2024-09-15T01:36:31.316 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:31.316 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:31.316 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:31.316 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:31.316 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:31.316 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:31.316 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:31.317 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:31.317 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:31.317 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:31.317 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:31.317 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:31.317 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:31.317 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 1 Processor -- start 2024-09-15T01:36:31.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 1 -- start start 2024-09-15T01:36:31.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 10 monclient: init 2024-09-15T01:36:31.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 10 monclient: _add_conns ranks=[4,3,2,1,5,9,6,8,7,10,0] 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f69740a2930 0x7f6974132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 10 monclient(hunting): picked mon.noname-c con 0x7f69740a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f697409faa0 0x7f697409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 10 monclient(hunting): picked mon.noname-g con 0x7f697409faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69740a08c0 0x7f69740a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 10 monclient(hunting): picked mon.noname-b con 0x7f69740a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697409e0f0 con 0x7f69740a08c0 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697409dd80 con 0x7f697409faa0 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69740a2d00 con 0x7f69740a2930 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f697b31d640 10 monclient(hunting): authenticate will time out at 996.064948s 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f69798ba640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69740a08c0 msgr2=0x7f69740a0d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.314+0000 7f69790b9640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f69740a2930 msgr2=0x7f6974132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69798ba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69740a08c0 0x7f69740a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69790b9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f69740a2930 0x7f6974132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f697409faa0 0x7f697409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f697409faa0 0x7f697409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:46350/0 (socket says 172.21.15.178:46350) 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 1 -- 172.21.15.178:0/2792273250 learned_addr learned my addr 172.21.15.178:0/2792273250 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 10 monclient(hunting): get_auth_request con 0x7f697409faa0 auth_method 0 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:31.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 10 monclient(hunting): handle_auth_done global_id 15222 payload 274 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 1 -- 172.21.15.178:0/2792273250 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f69740a2930 msgr2=0x7f6974132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 1 --2- 172.21.15.178:0/2792273250 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f69740a2930 0x7f6974132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 1 -- 172.21.15.178:0/2792273250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69740a08c0 msgr2=0x7f69740a0d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 1 --2- 172.21.15.178:0/2792273250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69740a08c0 0x7f69740a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 10 monclient: _finish_hunting 0 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 1 monclient: found mon.noname-g 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:31.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.315+0000 7f69788b8640 1 -- 172.21.15.178:0/2792273250 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697404bf60 con 0x7f697409faa0 2024-09-15T01:36:31.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f69788b8640 1 --2- 172.21.15.178:0/2792273250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f697409faa0 0x7f697409fe70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6964009940 tx=0x7f69640091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c887f8a98b8ffbae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:31.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f696bfff640 1 -- 172.21.15.178:0/2792273250 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f696402acb0 con 0x7f697409faa0 2024-09-15T01:36:31.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f696bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:31.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f696bfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:31.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f696bfff640 10 monclient: dump: 2024-09-15T01:36:31.329 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:31.329 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:31.329 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.329 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.329 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:31.329 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:31.329 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f696bfff640 10 monclient: _finish_auth 0 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f696bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f697b31d640 5 monclient: authenticate success, global_id 15222 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f696bfff640 1 -- 172.21.15.178:0/2792273250 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696402ae30 con 0x7f697409faa0 2024-09-15T01:36:31.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f696bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f697b31d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f696bfff640 1 -- 172.21.15.178:0/2792273250 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6964031ca0 con 0x7f697409faa0 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f697b31d640 4 set_mon_vals no callback set 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f696bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f696bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f696bfff640 10 monclient: dump: 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:31.331 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f697b31d640 10 monclient: shutdown 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f697b31d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f697b31d640 1 -- 172.21.15.178:0/2792273250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f697409faa0 msgr2=0x7f697409fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:31.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f697b31d640 1 --2- 172.21.15.178:0/2792273250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f697409faa0 0x7f697409fe70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6964009940 tx=0x7f69640091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f697b31d640 1 -- 172.21.15.178:0/2792273250 shutdown_connections 2024-09-15T01:36:31.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f697b31d640 1 --2- 172.21.15.178:0/2792273250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69740a08c0 0x7f69740a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f697b31d640 1 --2- 172.21.15.178:0/2792273250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f697409faa0 0x7f697409fe70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f697b31d640 1 --2- 172.21.15.178:0/2792273250 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f69740a2930 0x7f6974132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.316+0000 7f697b31d640 1 -- 172.21.15.178:0/2792273250 >> 172.21.15.178:0/2792273250 conn(0x7f69740a5ed0 msgr2=0x7f69740a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:31.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.317+0000 7f697b31d640 1 -- 172.21.15.178:0/2792273250 shutdown_connections 2024-09-15T01:36:31.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.317+0000 7f697b31d640 1 -- 172.21.15.178:0/2792273250 wait complete. 2024-09-15T01:36:31.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.317+0000 7f697b31d640 10 monclient: build_initial_monmap 2024-09-15T01:36:31.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.317+0000 7f697b31d640 10 monclient: monmap: 2024-09-15T01:36:31.333 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:31.333 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:31.334 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.317+0000 7f697b31d640 1 Processor -- start 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.317+0000 7f697b31d640 1 -- start start 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.317+0000 7f697b31d640 10 monclient: init 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 10 monclient: _add_conns ranks=[0,6,4,5,2,8,3,7,1] 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f697409faa0 0x7f697409b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 10 monclient(hunting): picked mon.noname-a con 0x7f697409faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f69740a08c0 0x7f69741318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 10 monclient(hunting): picked mon.noname-d con 0x7f69740a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f69740a2930 0x7f697411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 10 monclient(hunting): picked mon.noname-c con 0x7f69740a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697411f270 con 0x7f697409faa0 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69740a2400 con 0x7f69740a2930 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f697409d630 con 0x7f69740a08c0 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f697b31d640 10 monclient(hunting): authenticate will time out at 996.068468s 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69790b9640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f697409faa0 msgr2=0x7f697409b300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69798ba640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f69740a2930 msgr2=0x7f697411c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69790b9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f697409faa0 0x7f697409b300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69798ba640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f69740a2930 0x7f697411c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f69740a08c0 0x7f69741318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f69740a08c0 0x7f69741318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:56338/0 (socket says 172.21.15.178:56338) 2024-09-15T01:36:31.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 1 -- 172.21.15.178:0/93754883 learned_addr learned my addr 172.21.15.178:0/93754883 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 10 monclient(hunting): get_auth_request con 0x7f69740a08c0 auth_method 0 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 10 monclient(hunting): handle_auth_done global_id 24297 payload 824 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 1 -- 172.21.15.178:0/93754883 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f69740a2930 msgr2=0x7f697411c530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 1 --2- 172.21.15.178:0/93754883 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f69740a2930 0x7f697411c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 1 -- 172.21.15.178:0/93754883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f697409faa0 msgr2=0x7f697409b300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 1 --2- 172.21.15.178:0/93754883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f697409faa0 0x7f697409b300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 10 monclient: _finish_hunting 0 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 1 monclient: found mon.noname-d 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.318+0000 7f69788b8640 1 -- 172.21.15.178:0/93754883 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6964009450 con 0x7f69740a08c0 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69788b8640 1 --2- 172.21.15.178:0/93754883 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f69740a08c0 0x7f69741318b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f697409d7c0 tx=0x7f6964032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 1 -- 172.21.15.178:0/93754883 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f696402acb0 con 0x7f69740a08c0 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 10 monclient: dump: 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.338 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:31.339 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:31.339 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:31.339 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:31.339 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:31.339 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:31.339 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:31.339 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:31.339 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:31.339 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:31.339 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:31.339 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 10 monclient: _finish_auth 0 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f697b31d640 5 monclient: authenticate success, global_id 24297 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 1 -- 172.21.15.178:0/93754883 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6964032e20 con 0x7f69740a08c0 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 1 -- 172.21.15.178:0/93754883 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f696403a810 con 0x7f69740a08c0 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f697b31d640 10 monclient: _renew_subs 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f697b31d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f697b31d640 1 -- 172.21.15.178:0/93754883 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697411ecb0 con 0x7f69740a08c0 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f696b7fe640 4 set_mon_vals no callback set 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 10 monclient: dump: 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:31.340 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:31.341 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f697b31d640 10 monclient: _renew_subs 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f697b31d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f697b31d640 1 -- 172.21.15.178:0/93754883 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697411da70 con 0x7f69740a08c0 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.319+0000 7f69697fa640 1 -- 172.21.15.178:0/93754883 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6964039040 con 0x7f69740a08c0 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.320+0000 7f69697fa640 1 --2- 172.21.15.178:0/93754883 >> v2:172.21.15.204:6812/28210 conn(0x7f69580397b0 0x7f695803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.320+0000 7f69697fa640 1 -- 172.21.15.178:0/93754883 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f696406b6b0 con 0x7f69740a08c0 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.320+0000 7f697b31d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.320+0000 7f697b31d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.320+0000 7f697b31d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.320+0000 7f697b31d640 1 -- 172.21.15.178:0/93754883 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697411f020 con 0x7f69740a08c0 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.320+0000 7f69790b9640 1 --2- 172.21.15.178:0/93754883 >> v2:172.21.15.204:6812/28210 conn(0x7f69580397b0 0x7f695803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:31.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.320+0000 7f69790b9640 10 monclient: get_auth_request con 0x7f69580397b0 auth_method 0 2024-09-15T01:36:31.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.320+0000 7f69790b9640 1 --2- 172.21.15.178:0/93754883 >> v2:172.21.15.204:6812/28210 conn(0x7f69580397b0 0x7f695803bc60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6970003870 tx=0x7f6970006ce0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:31.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.324+0000 7f69697fa640 1 -- 172.21.15.178:0/93754883 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6964041020 con 0x7f69740a08c0 2024-09-15T01:36:31.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.324+0000 7f69697fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:31.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.324+0000 7f69697fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:31.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.539+0000 7f697b31d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:36:31.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.539+0000 7f697b31d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:31.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.539+0000 7f697b31d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:31.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.539+0000 7f697b31d640 1 -- 172.21.15.178:0/93754883 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f697404b720 con 0x7f69740a08c0 2024-09-15T01:36:31.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.539+0000 7f69697fa640 1 -- 172.21.15.178:0/93754883 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7f6964037070 con 0x7f69740a08c0 2024-09-15T01:36:31.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.539+0000 7f69697fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:31.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.539+0000 7f69697fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:31.542 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":8,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:36:31.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 1 -- 172.21.15.178:0/93754883 >> v2:172.21.15.204:6812/28210 conn(0x7f69580397b0 msgr2=0x7f695803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:31.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 1 --2- 172.21.15.178:0/93754883 >> v2:172.21.15.204:6812/28210 conn(0x7f69580397b0 0x7f695803bc60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6970003870 tx=0x7f6970006ce0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 10 monclient: shutdown 2024-09-15T01:36:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 1 -- 172.21.15.178:0/93754883 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f69740a08c0 msgr2=0x7f69741318b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 1 --2- 172.21.15.178:0/93754883 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f69740a08c0 0x7f69741318b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f697409d7c0 tx=0x7f6964032000 comp rx=0 tx=0).stop 2024-09-15T01:36:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 1 -- 172.21.15.178:0/93754883 shutdown_connections 2024-09-15T01:36:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 1 --2- 172.21.15.178:0/93754883 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f69740a2930 0x7f697411c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 1 --2- 172.21.15.178:0/93754883 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f69740a08c0 0x7f69741318b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 1 --2- 172.21.15.178:0/93754883 >> v2:172.21.15.204:6812/28210 conn(0x7f69580397b0 0x7f695803bc60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 1 --2- 172.21.15.178:0/93754883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f697409faa0 0x7f697409b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 1 -- 172.21.15.178:0/93754883 >> 172.21.15.178:0/93754883 conn(0x7f69740a5ed0 msgr2=0x7f69740a5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 1 -- 172.21.15.178:0/93754883 shutdown_connections 2024-09-15T01:36:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:31.541+0000 7f697b31d640 1 -- 172.21.15.178:0/93754883 wait complete. 2024-09-15T01:36:33.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:33.697+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:33.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:33.697+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:33.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:33.697+0000 7fc57dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:34.554 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:36:34.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.656+0000 7efc34b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:34.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.656+0000 7efc34b9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:34.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.656+0000 7efc34b9d640 10 monclient: monmap: 2024-09-15T01:36:34.659 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:34.659 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:34.659 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:34.657646+0000 2024-09-15T01:36:34.659 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:34.657646+0000 2024-09-15T01:36:34.660 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:34.660 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:34.660 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:34.660 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:34.660 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:34.660 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:34.660 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:34.660 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:34.660 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:34.660 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:34.660 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:34.661 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:34.661 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:34.661 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:34.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.657+0000 7efc34b9d640 1 Processor -- start 2024-09-15T01:36:34.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.657+0000 7efc34b9d640 1 -- start start 2024-09-15T01:36:34.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.657+0000 7efc34b9d640 10 monclient: init 2024-09-15T01:36:34.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:34.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 10 monclient: _add_conns ranks=[4,8,1,9,3,6,7,2,0,5,10] 2024-09-15T01:36:34.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc300a2930 0x7efc30132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:34.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 10 monclient(hunting): picked mon.noname-c con 0x7efc300a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:34.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc3009faa0 0x7efc3009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:34.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 10 monclient(hunting): picked mon.noname-e con 0x7efc3009faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:34.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7efc300a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:34.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3004bf60 con 0x7efc300a08c0 2024-09-15T01:36:34.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:34.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc3009e0f0 con 0x7efc300a2930 2024-09-15T01:36:34.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:34.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7efc3009dd80 con 0x7efc3009faa0 2024-09-15T01:36:34.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:34.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc34b9d640 10 monclient(hunting): authenticate will time out at 999.408133s 2024-09-15T01:36:34.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc2e59c640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc300a2930 msgr2=0x7efc30132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:34.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc2dd9b640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc3009faa0 msgr2=0x7efc3009fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:34.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc2e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc300a2930 0x7efc30132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:34.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc2dd9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc3009faa0 0x7efc3009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:34.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc2ed9d640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7efc300a08c0 0x7efc300a0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:45972/0 (socket says 172.21.15.178:45972) 2024-09-15T01:36:34.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.658+0000 7efc2ed9d640 1 -- 172.21.15.178:0/795773107 learned_addr learned my addr 172.21.15.178:0/795773107 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:36:34.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:34.697+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:34.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:34.697+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:34.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.859+0000 7efc2e59c640 1 -- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc300a2930 msgr2=0x7efc30132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:34.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.859+0000 7efc2dd9b640 1 -- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc3009faa0 msgr2=0x7efc3009fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:34.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.859+0000 7efc2e59c640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc300a2930 0x7efc30132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:34.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:34.859+0000 7efc2dd9b640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc3009faa0 0x7efc3009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:35.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:35.260+0000 7efc2dd9b640 1 -- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc3009faa0 msgr2=0x7efc3009fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:35.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:35.260+0000 7efc2e59c640 1 -- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc300a2930 msgr2=0x7efc30132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:35.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:35.260+0000 7efc2dd9b640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc3009faa0 0x7efc3009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:35.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:35.260+0000 7efc2e59c640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc300a2930 0x7efc30132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:35.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:35.698+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:35.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:35.698+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:36.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:36.061+0000 7efc2dd9b640 1 -- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc3009faa0 msgr2=0x7efc3009fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:36.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:36.061+0000 7efc2e59c640 1 -- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc300a2930 msgr2=0x7efc30132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:36.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:36.061+0000 7efc2dd9b640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc3009faa0 0x7efc3009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:36.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:36.061+0000 7efc2e59c640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc300a2930 0x7efc30132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:36.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:36.698+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:36.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:36.698+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 10 monclient(hunting): tick 2024-09-15T01:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 1 monclient(hunting): continuing hunt 2024-09-15T01:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 1 -- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc3009faa0 msgr2=0x7efc3009fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc3009faa0 0x7efc3009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 1 -- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc300a2930 msgr2=0x7efc30132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efc300a2930 0x7efc30132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 1 -- 172.21.15.178:0/795773107 >> v1:172.21.15.204:6789/0 conn(0x7efc300a08c0 legacy=0x7efc300a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 10 monclient: _add_conns ranks=[3,0,2,9,10,5,6,7] 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc10003800 0x7efc10005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 10 monclient(hunting): picked mon.noname-g con 0x7efc10003800 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efc10006240 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc10009f30 0x7efc1000c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 10 monclient(hunting): picked mon.noname-b con 0x7efc10009f30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 1 -- 172.21.15.178:0/795773107 --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc300a2d00 con 0x7efc10006240 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 1 -- 172.21.15.178:0/795773107 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3009e0f0 con 0x7efc10009f30 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 1 -- 172.21.15.178:0/795773107 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3009dd80 con 0x7efc10003800 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc27fff640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc2dd9b640 1 -- 172.21.15.178:0/795773107 >> v1:172.21.15.178:6789/0 conn(0x7efc10006240 legacy=0x7efc100086a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc2ed9d640 1 -- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc10009f30 msgr2=0x7efc1000c3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc2ed9d640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc10009f30 0x7efc1000c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc2e59c640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc10003800 0x7efc10005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc2e59c640 10 monclient(hunting): get_auth_request con 0x7efc10003800 auth_method 0 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc2e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc2e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc2e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc2e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc2e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.658+0000 7efc2e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.659+0000 7efc2e59c640 10 monclient(hunting): handle_auth_done global_id 15231 payload 274 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.659+0000 7efc2e59c640 1 -- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc10009f30 msgr2=0x7efc1000c3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.659+0000 7efc2e59c640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc10009f30 0x7efc1000c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.659+0000 7efc2e59c640 1 -- 172.21.15.178:0/795773107 >> v1:172.21.15.178:6789/0 conn(0x7efc10006240 legacy=0x7efc100086a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.659+0000 7efc2e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.659+0000 7efc2e59c640 1 monclient: found mon.noname-g 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.659+0000 7efc2ed9d640 1 -- 172.21.15.178:0/795773107 reap_dead start 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.659+0000 7efc2e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.659+0000 7efc2e59c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.659+0000 7efc2e59c640 1 -- 172.21.15.178:0/795773107 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1000dcc0 con 0x7efc10003800 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.659+0000 7efc2e59c640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc10003800 0x7efc10005c40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efc28003cc0 tx=0x7efc280076d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=dd0013efec3d068e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc2d59a640 1 -- 172.21.15.178:0/795773107 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc280328b0 con 0x7efc10003800 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc2d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc2d59a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc2d59a640 10 monclient: dump: 2024-09-15T01:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc2d59a640 10 monclient: _finish_auth 0 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc2d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc2d59a640 1 -- 172.21.15.178:0/795773107 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2802d070 con 0x7efc10003800 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc34b9d640 5 monclient: authenticate success, global_id 15231 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc34b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc2d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc2d59a640 1 -- 172.21.15.178:0/795773107 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc280313d0 con 0x7efc10003800 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc34b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc34b9d640 4 set_mon_vals no callback set 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc2d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc2d59a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.660+0000 7efc2d59a640 10 monclient: dump: 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:37.668 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:37.669 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:37.669 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.661+0000 7efc34b9d640 10 monclient: shutdown 2024-09-15T01:36:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.661+0000 7efc34b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.661+0000 7efc34b9d640 1 -- 172.21.15.178:0/795773107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc10003800 msgr2=0x7efc10005c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.661+0000 7efc34b9d640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc10003800 0x7efc10005c40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efc28003cc0 tx=0x7efc280076d0 comp rx=0 tx=0).stop 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.661+0000 7efc34b9d640 1 -- 172.21.15.178:0/795773107 shutdown_connections 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.661+0000 7efc34b9d640 1 --2- 172.21.15.178:0/795773107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc10003800 0x7efc10005c40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.661+0000 7efc34b9d640 1 -- 172.21.15.178:0/795773107 >> 172.21.15.178:0/795773107 conn(0x7efc300a5ed0 msgr2=0x7efc300a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.661+0000 7efc34b9d640 1 -- 172.21.15.178:0/795773107 shutdown_connections 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.661+0000 7efc34b9d640 1 -- 172.21.15.178:0/795773107 wait complete. 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.661+0000 7efc34b9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.661+0000 7efc34b9d640 10 monclient: monmap: 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 1 Processor -- start 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 1 -- start start 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 10 monclient: init 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 10 monclient: _add_conns ranks=[7,3,8,1,2,6,0,4,5] 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efc10003800 0x7efc3009b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 10 monclient(hunting): picked mon.noname-i con 0x7efc10003800 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc3011ca00 0x7efc30131900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 10 monclient(hunting): picked mon.noname-g con 0x7efc3011ca00 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc30131e40 0x7efc30141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 10 monclient(hunting): picked mon.noname-e con 0x7efc30131e40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc300a2390 con 0x7efc3011ca00 2024-09-15T01:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efc3009d600 con 0x7efc10003800 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7efc3009bd10 con 0x7efc30131e40 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.662+0000 7efc34b9d640 10 monclient(hunting): authenticate will time out at 1002.412988s 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2ed9d640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc30131e40 msgr2=0x7efc30141dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2ed9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc30131e40 0x7efc30141dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efc10003800 0x7efc3009b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efc10003800 0x7efc3009b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:54744/0 (socket says 172.21.15.178:54744) 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 1 -- 172.21.15.178:0/1234368917 learned_addr learned my addr 172.21.15.178:0/1234368917 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 10 monclient(hunting): get_auth_request con 0x7efc10003800 auth_method 0 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2dd9b640 1 --2- 172.21.15.178:0/1234368917 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc3011ca00 0x7efc30131900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2dd9b640 10 monclient(hunting): get_auth_request con 0x7efc3011ca00 auth_method 0 2024-09-15T01:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 10 monclient(hunting): handle_auth_done global_id 24298 payload 824 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 1 -- 172.21.15.178:0/1234368917 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc30131e40 msgr2=0x7efc30141dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 1 --2- 172.21.15.178:0/1234368917 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc30131e40 0x7efc30141dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 1 -- 172.21.15.178:0/1234368917 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc3011ca00 msgr2=0x7efc30131900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 1 --2- 172.21.15.178:0/1234368917 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc3011ca00 0x7efc30131900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 1 monclient: found mon.noname-i 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 1 -- 172.21.15.178:0/1234368917 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc280079b0 con 0x7efc10003800 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.663+0000 7efc2e59c640 1 --2- 172.21.15.178:0/1234368917 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efc10003800 0x7efc3009b310 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efc28000f80 tx=0x7efc28035000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc267fc640 1 -- 172.21.15.178:0/1234368917 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc2803e880 con 0x7efc10003800 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc267fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc267fc640 10 monclient: dump: 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc267fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc267fc640 1 -- 172.21.15.178:0/1234368917 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2803ea40 con 0x7efc10003800 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc34b9d640 5 monclient: authenticate success, global_id 24298 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc267fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc267fc640 1 -- 172.21.15.178:0/1234368917 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc2803ebb0 con 0x7efc10003800 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc34b9d640 10 monclient: _renew_subs 2024-09-15T01:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc34b9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc34b9d640 1 -- 172.21.15.178:0/1234368917 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3011ea20 con 0x7efc10003800 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc267fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc267fc640 10 monclient: dump: 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc2cd99640 4 set_mon_vals no callback set 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc34b9d640 10 monclient: _renew_subs 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc34b9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.664+0000 7efc34b9d640 1 -- 172.21.15.178:0/1234368917 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3011dac0 con 0x7efc10003800 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.665+0000 7efc267fc640 1 -- 172.21.15.178:0/1234368917 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7efc2803c040 con 0x7efc10003800 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.665+0000 7efc34b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.665+0000 7efc34b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.665+0000 7efc34b9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.665+0000 7efc267fc640 1 --2- 172.21.15.178:0/1234368917 >> v2:172.21.15.204:6812/28210 conn(0x7efc1003aab0 0x7efc1003cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.665+0000 7efc34b9d640 1 -- 172.21.15.178:0/1234368917 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3011eea0 con 0x7efc10003800 2024-09-15T01:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.665+0000 7efc267fc640 1 -- 172.21.15.178:0/1234368917 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7175+0+0 (secure 0 0 0) 0x7efc2806c960 con 0x7efc10003800 2024-09-15T01:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.665+0000 7efc2dd9b640 1 --2- 172.21.15.178:0/1234368917 >> v2:172.21.15.204:6812/28210 conn(0x7efc1003aab0 0x7efc1003cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.665+0000 7efc2dd9b640 10 monclient: get_auth_request con 0x7efc1003aab0 auth_method 0 2024-09-15T01:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.666+0000 7efc2dd9b640 1 --2- 172.21.15.178:0/1234368917 >> v2:172.21.15.204:6812/28210 conn(0x7efc1003aab0 0x7efc1003cf60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efc18003fd0 tx=0x7efc18009520 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.670+0000 7efc267fc640 1 -- 172.21.15.178:0/1234368917 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7efc2803e400 con 0x7efc10003800 2024-09-15T01:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.670+0000 7efc267fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.670+0000 7efc267fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:37.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:37.697+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:37.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:37.697+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:37.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.886+0000 7efc34b9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:36:37.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.886+0000 7efc34b9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:37.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.886+0000 7efc34b9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:37.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.886+0000 7efc34b9d640 1 -- 172.21.15.178:0/1234368917 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7efc3011eea0 con 0x7efc10003800 2024-09-15T01:36:37.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.887+0000 7efc267fc640 1 -- 172.21.15.178:0/1234368917 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7efc2804b100 con 0x7efc10003800 2024-09-15T01:36:37.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.887+0000 7efc267fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:37.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.887+0000 7efc267fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:37.889 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":14,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:36:37.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.888+0000 7efc34b9d640 1 -- 172.21.15.178:0/1234368917 >> v2:172.21.15.204:6812/28210 conn(0x7efc1003aab0 msgr2=0x7efc1003cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:37.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.888+0000 7efc34b9d640 1 --2- 172.21.15.178:0/1234368917 >> v2:172.21.15.204:6812/28210 conn(0x7efc1003aab0 0x7efc1003cf60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efc18003fd0 tx=0x7efc18009520 comp rx=0 tx=0).stop 2024-09-15T01:36:37.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.888+0000 7efc34b9d640 10 monclient: shutdown 2024-09-15T01:36:37.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.888+0000 7efc34b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:37.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.888+0000 7efc34b9d640 1 -- 172.21.15.178:0/1234368917 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efc10003800 msgr2=0x7efc3009b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:37.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.888+0000 7efc34b9d640 1 --2- 172.21.15.178:0/1234368917 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efc10003800 0x7efc3009b310 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efc28000f80 tx=0x7efc28035000 comp rx=0 tx=0).stop 2024-09-15T01:36:37.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.889+0000 7efc34b9d640 1 -- 172.21.15.178:0/1234368917 shutdown_connections 2024-09-15T01:36:37.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.889+0000 7efc34b9d640 1 --2- 172.21.15.178:0/1234368917 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efc30131e40 0x7efc30141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:37.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.889+0000 7efc34b9d640 1 --2- 172.21.15.178:0/1234368917 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc3011ca00 0x7efc30131900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:37.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.889+0000 7efc34b9d640 1 --2- 172.21.15.178:0/1234368917 >> v2:172.21.15.204:6812/28210 conn(0x7efc1003aab0 0x7efc1003cf60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:37.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.889+0000 7efc34b9d640 1 --2- 172.21.15.178:0/1234368917 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efc10003800 0x7efc3009b310 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:37.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.889+0000 7efc34b9d640 1 -- 172.21.15.178:0/1234368917 >> 172.21.15.178:0/1234368917 conn(0x7efc300a5ed0 msgr2=0x7efc300a3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:37.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.889+0000 7efc34b9d640 1 -- 172.21.15.178:0/1234368917 shutdown_connections 2024-09-15T01:36:37.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:37.889+0000 7efc34b9d640 1 -- 172.21.15.178:0/1234368917 wait complete. 2024-09-15T01:36:38.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:38.698+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:38.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:38.698+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:39.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:39.698+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:39.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:39.698+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:40.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:40.698+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:40.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:40.698+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:40.902 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:36:41.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.004+0000 7fea00c6f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:41.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.004+0000 7fea00c6f640 10 monclient: build_initial_monmap 2024-09-15T01:36:41.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.004+0000 7fea00c6f640 10 monclient: monmap: 2024-09-15T01:36:41.006 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:41.007 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:41.007 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:41.005541+0000 2024-09-15T01:36:41.007 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:41.005541+0000 2024-09-15T01:36:41.007 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:41.007 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:41.007 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:41.007 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:41.007 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:41.007 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:41.007 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:41.007 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:41.008 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:41.008 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:41.008 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:41.008 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:41.008 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:41.008 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:41.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 1 Processor -- start 2024-09-15T01:36:41.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 1 -- start start 2024-09-15T01:36:41.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 10 monclient: init 2024-09-15T01:36:41.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:41.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 10 monclient: _add_conns ranks=[4,2,6,5,9,8,0,3,10,7,1] 2024-09-15T01:36:41.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9fc09df80 0x7fe9fc06aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:41.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 10 monclient(hunting): picked mon.noname-c con 0x7fe9fc09df80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:41.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc06e760 0x7fe9fc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:41.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 10 monclient(hunting): picked mon.noname-b con 0x7fe9fc06e760 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:41.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:41.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 10 monclient(hunting): picked mon.noname-d con 0x7fe9fc12f5a0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:41.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:41.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.005+0000 7fea00c6f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc06a6e0 con 0x7fe9fc06e760 2024-09-15T01:36:41.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fea00c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:41.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fea00c6f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc06cfe0 con 0x7fe9fc09df80 2024-09-15T01:36:41.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fea00c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:41.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fea00c6f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc09e350 con 0x7fe9fc12f5a0 2024-09-15T01:36:41.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fea00c6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:41.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fea00c6f640 10 monclient(hunting): authenticate will time out at 1005.756019s 2024-09-15T01:36:41.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9f9d9b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc06e760 msgr2=0x7fe9fc06eb30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:41.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fa59c640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9fc09df80 msgr2=0x7fe9fc06aa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:41.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9f9d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc06e760 0x7fe9fc06eb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:41.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fa59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9fc09df80 0x7fe9fc06aa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:41.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fad9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:41.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fad9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:40798/0 (socket says 172.21.15.178:40798) 2024-09-15T01:36:41.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 learned_addr learned my addr 172.21.15.178:0/876767759 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:41.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fad9d640 10 monclient(hunting): get_auth_request con 0x7fe9fc12f5a0 auth_method 0 2024-09-15T01:36:41.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:41.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:41.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:41.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 msgr2=0x7fe9fc12f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:36:41.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 msgr2=0x7fe9fc12f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:41.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:41.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.006+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:41.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9f9d9b640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc06e760 msgr2=0x7fe9fc06eb30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9fa59c640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9fc09df80 msgr2=0x7fe9fc06aa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc06e760 0x7fe9fc06eb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9fc09df80 0x7fe9fc06aa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9fad9d640 10 monclient(hunting): get_auth_request con 0x7fe9fc12f5a0 auth_method 0 2024-09-15T01:36:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9fad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9fad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9fad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 msgr2=0x7fe9fc12f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:36:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 msgr2=0x7fe9fc12f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.207+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:41.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9f9d9b640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc06e760 msgr2=0x7fe9fc06eb30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:41.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9fa59c640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9fc09df80 msgr2=0x7fe9fc06aa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:41.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc06e760 0x7fe9fc06eb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:41.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9fc09df80 0x7fe9fc06aa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9fad9d640 10 monclient(hunting): get_auth_request con 0x7fe9fc12f5a0 auth_method 0 2024-09-15T01:36:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9fad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9fad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9fad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 msgr2=0x7fe9fc12f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:36:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 msgr2=0x7fe9fc12f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:41.608+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:41.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:41.698+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:41.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:41.698+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:42.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9f9d9b640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc06e760 msgr2=0x7fe9fc06eb30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:42.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9fa59c640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9fc09df80 msgr2=0x7fe9fc06aa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:42.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc06e760 0x7fe9fc06eb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:42.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9fc09df80 0x7fe9fc06aa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:42.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:42.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9fad9d640 10 monclient(hunting): get_auth_request con 0x7fe9fc12f5a0 auth_method 0 2024-09-15T01:36:42.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9fad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:42.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9fad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:42.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9fad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:42.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 msgr2=0x7fe9fc12f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:36:42.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 msgr2=0x7fe9fc12f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:42.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:42.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:42.409+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:42.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:42.698+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:42.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:42.698+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:43.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:43.698+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:43.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:43.698+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:43.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:43.698+0000 7fc57dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:44.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 10 monclient(hunting): tick 2024-09-15T01:36:44.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 1 monclient(hunting): continuing hunt 2024-09-15T01:36:44.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:36:44.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 msgr2=0x7fe9fc12f9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:44.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9fc12f5a0 0x7fe9fc12f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:44.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9fc09df80 msgr2=0x7fe9fc06aa20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:44.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe9fc09df80 0x7fe9fc06aa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:44.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc06e760 msgr2=0x7fe9fc06eb30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:44.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9fc06e760 0x7fe9fc06eb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 10 monclient: _add_conns ranks=[7,5,1,0,9,10,3,8] 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 10 monclient(hunting): picked mon.noname-i con 0x7fe9dc003860 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 10 monclient(hunting): picked mon.noname-h con 0x7fe9dc006260 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe9dc008d60 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 1 -- 172.21.15.178:0/876767759 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9fc04bf60 con 0x7fe9dc008d60 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 1 -- 172.21.15.178:0/876767759 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc06a6e0 con 0x7fe9dc006260 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 1 -- 172.21.15.178:0/876767759 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc06cfe0 con 0x7fe9dc003860 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9ebfff640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:44.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9f9d9b640 10 monclient(hunting): get_auth_request con 0x7fe9dc006260 auth_method 0 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9f9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9f9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9f9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9fa59c640 10 monclient(hunting): get_auth_request con 0x7fe9dc003860 auth_method 0 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9fa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9fa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9f9d9b640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 msgr2=0x7fe9dc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9f9d9b640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 msgr2=0x7fe9dc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:44.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9fa59c640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 msgr2=0x7fe9dc005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:36:44.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9fa59c640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 msgr2=0x7fe9dc005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:44.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:44.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:44.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.006+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:44.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:44.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:44.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9fa59c640 10 monclient(hunting): get_auth_request con 0x7fe9dc003860 auth_method 0 2024-09-15T01:36:44.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:44.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9fa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:44.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9fa59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:44.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9f9d9b640 10 monclient(hunting): get_auth_request con 0x7fe9dc006260 auth_method 0 2024-09-15T01:36:44.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9f9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:44.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9f9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:44.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9f9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:44.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9fa59c640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 msgr2=0x7fe9dc005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:36:44.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9fa59c640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 msgr2=0x7fe9dc005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:44.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:44.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:44.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9f9d9b640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 msgr2=0x7fe9dc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:36:44.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9f9d9b640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 msgr2=0x7fe9dc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:44.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:44.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.207+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:44.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:44.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9fa59c640 10 monclient(hunting): get_auth_request con 0x7fe9dc003860 auth_method 0 2024-09-15T01:36:44.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:44.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9fa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:44.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9fa59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:44.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:44.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9f9d9b640 10 monclient(hunting): get_auth_request con 0x7fe9dc006260 auth_method 0 2024-09-15T01:36:44.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9f9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:44.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9f9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:44.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9f9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:44.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9fa59c640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 msgr2=0x7fe9dc005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:36:44.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9fa59c640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 msgr2=0x7fe9dc005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:44.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:44.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:44.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9f9d9b640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 msgr2=0x7fe9dc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:36:44.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9f9d9b640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 msgr2=0x7fe9dc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:44.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:44.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:44.608+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:44.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:44.698+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:44.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:44.698+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:45.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:45.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9fa59c640 10 monclient(hunting): get_auth_request con 0x7fe9dc003860 auth_method 0 2024-09-15T01:36:45.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:45.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:45.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9fa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:45.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9fa59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:45.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9f9d9b640 10 monclient(hunting): get_auth_request con 0x7fe9dc006260 auth_method 0 2024-09-15T01:36:45.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9f9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:45.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9f9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:45.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9f9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:45.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9fa59c640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 msgr2=0x7fe9dc005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:36:45.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9fa59c640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 msgr2=0x7fe9dc005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:45.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:45.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9f9d9b640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 msgr2=0x7fe9dc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:36:45.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9f9d9b640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 msgr2=0x7fe9dc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:45.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:45.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:45.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.409+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:45.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.527+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046592783 0 0) 0x7fe9fc04bf60 con 0x7fe9dc008d60 2024-09-15T01:36:45.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.527+0000 7fe9f959a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:45.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.527+0000 7fe9f959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:45.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.527+0000 7fe9f959a640 10 monclient(hunting): my global_id is 6751 2024-09-15T01:36:45.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.527+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e40034e0 con 0x7fe9dc008d60 2024-09-15T01:36:45.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 355521485 0 0) 0x7fe9e40034e0 con 0x7fe9dc008d60 2024-09-15T01:36:45.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 msgr2=0x7fe9dc005c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:45.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9dc003860 0x7fe9dc005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 msgr2=0x7fe9dc0086e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:45.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe9dc006260 0x7fe9dc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient: _finish_hunting 0 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 reap_dead start 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc00db80 con 0x7fe9dc008d60 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient: _finish_auth 0 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fe9f4009610 con 0x7fe9dc008d60 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fea00c6f640 5 monclient: authenticate success, global_id 6751 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fea00c6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:36:45.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient: dump: 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:45.534 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 >> v1:172.21.15.204:6789/0 conn(0x7fe9dc008d60 legacy=0x7fe9dc00b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient: _add_conns ranks=[8,2,7,0,4,6,1,5,3] 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9e4004220 0x7fe9e4004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient(hunting): picked mon.e con 0x7fe9e4004220 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9e4004de0 0x7fe9e4005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient(hunting): picked mon.b con 0x7fe9e4004de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9e4009c00 0x7fe9e400c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient(hunting): picked mon.i con 0x7fe9e4009c00 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:45.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:45.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc06a6e0 con 0x7fe9e4004de0 2024-09-15T01:36:45.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:45.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc06cfe0 con 0x7fe9e4009c00 2024-09-15T01:36:45.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:45.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e400c970 con 0x7fe9e4004220 2024-09-15T01:36:45.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:45.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f959a640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:36:45.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fa59c640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9e4004220 msgr2=0x7fe9e4004780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:45.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fea00c6f640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:45.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f9d9b640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9e4004de0 msgr2=0x7fe9e4005250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:45.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9e4004220 0x7fe9e4004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:45.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9e4004de0 0x7fe9e4005250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:45.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9e4009c00 0x7fe9e400c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:45.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 10 monclient(hunting): get_auth_request con 0x7fe9e4009c00 auth_method 0 2024-09-15T01:36:45.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:45.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:45.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:45.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:45.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:45.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:45.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 10 monclient(hunting): handle_auth_done global_id 6751 payload 293 2024-09-15T01:36:45.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9e4004220 msgr2=0x7fe9e4004780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:45.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9e4004220 0x7fe9e4004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9e4004de0 msgr2=0x7fe9e4005250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:45.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9e4004de0 0x7fe9e4005250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:36:45.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 1 monclient: found mon.i 2024-09-15T01:36:45.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:45.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:45.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 1 -- 172.21.15.178:0/876767759 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e40034e0 con 0x7fe9e4009c00 2024-09-15T01:36:45.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 10 monclient: _finish_auth 0 2024-09-15T01:36:45.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.528+0000 7fe9fad9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:45.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.529+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9e4009c00 0x7fe9e400c060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0034b0 tx=0x7fe9f400b040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=1da2bb0456bdbcda server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:45.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.532+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9f402ac90 con 0x7fe9e4009c00 2024-09-15T01:36:45.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.532+0000 7fe9f959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:45.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.532+0000 7fe9f959a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:45.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.532+0000 7fe9f959a640 10 monclient: dump: 2024-09-15T01:36:45.544 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:45.544 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:45.544 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.544 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.544 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:45.544 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:45.544 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:45.544 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:45.544 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.532+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f402e070 con 0x7fe9e4009c00 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.532+0000 7fe9f959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.532+0000 7fe9f959a640 1 -- 172.21.15.178:0/876767759 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9f402ae10 con 0x7fe9e4009c00 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.532+0000 7fe9f959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.532+0000 7fe9f959a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.532+0000 7fe9f959a640 10 monclient: dump: 2024-09-15T01:36:45.545 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:45.546 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.532+0000 7fea00c6f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.532+0000 7fea00c6f640 4 set_mon_vals no callback set 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.533+0000 7fea00c6f640 10 monclient: shutdown 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.533+0000 7fea00c6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.533+0000 7fea00c6f640 1 -- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9e4009c00 msgr2=0x7fe9e400c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.533+0000 7fea00c6f640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9e4009c00 0x7fe9e400c060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0034b0 tx=0x7fe9f400b040 comp rx=0 tx=0).stop 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.533+0000 7fea00c6f640 1 -- 172.21.15.178:0/876767759 shutdown_connections 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.533+0000 7fea00c6f640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe9e4009c00 0x7fe9e400c060 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.533+0000 7fea00c6f640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9e4004de0 0x7fe9e4005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.533+0000 7fea00c6f640 1 --2- 172.21.15.178:0/876767759 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe9e4004220 0x7fe9e4004780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.533+0000 7fea00c6f640 1 -- 172.21.15.178:0/876767759 >> 172.21.15.178:0/876767759 conn(0x7fe9fc067d20 msgr2=0x7fe9fc0a1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:45.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.534+0000 7fea00c6f640 1 -- 172.21.15.178:0/876767759 shutdown_connections 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.534+0000 7fea00c6f640 1 -- 172.21.15.178:0/876767759 wait complete. 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.534+0000 7fea00c6f640 10 monclient: build_initial_monmap 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.534+0000 7fea00c6f640 10 monclient: monmap: 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:45.548 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 1 Processor -- start 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 1 -- start start 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 10 monclient: init 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 10 monclient: _add_conns ranks=[6,1,3,2,8,4,0,5,7] 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9dc008d60 0x7fe9fc139a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 10 monclient(hunting): picked mon.noname-d con 0x7fe9dc008d60 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:45.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e4004220 0x7fe9fc12da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 10 monclient(hunting): picked mon.noname-f con 0x7fe9e4004220 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4004de0 0x7fe9fc13cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 10 monclient(hunting): picked mon.noname-g con 0x7fe9e4004de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc11d180 con 0x7fe9e4004220 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc11a990 con 0x7fe9e4004de0 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc11cea0 con 0x7fe9dc008d60 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fea00c6f640 10 monclient(hunting): authenticate will time out at 1010.285737s 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fe9fa59c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9dc008d60 0x7fe9fc139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:45.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fe9fa59c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9dc008d60 0x7fe9fc139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:40830/0 (socket says 172.21.15.178:40830) 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fe9fa59c640 1 -- 172.21.15.178:0/3212163771 learned_addr learned my addr 172.21.15.178:0/3212163771 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fe9fa59c640 10 monclient(hunting): get_auth_request con 0x7fe9dc008d60 auth_method 0 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fe9fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fe9fa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fe9fa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/3212163771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e4004220 0x7fe9fc12da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fe9fa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.535+0000 7fe9fa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9f9d9b640 10 monclient(hunting): get_auth_request con 0x7fe9e4004220 auth_method 0 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9f9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9f9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9f9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:45.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fad9d640 1 --2- 172.21.15.178:0/3212163771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4004de0 0x7fe9fc13cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fad9d640 10 monclient(hunting): get_auth_request con 0x7fe9e4004de0 auth_method 0 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9f9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9f9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9f9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fa59c640 10 monclient(hunting): handle_auth_done global_id 24306 payload 824 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fa59c640 1 -- 172.21.15.178:0/3212163771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4004de0 msgr2=0x7fe9fc13cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:45.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/3212163771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4004de0 0x7fe9fc13cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fa59c640 1 -- 172.21.15.178:0/3212163771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e4004220 msgr2=0x7fe9fc12da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/3212163771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e4004220 0x7fe9fc12da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fa59c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fa59c640 1 monclient: found mon.noname-d 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fa59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fa59c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fa59c640 1 -- 172.21.15.178:0/3212163771 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc11cbc0 con 0x7fe9dc008d60 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9fa59c640 1 --2- 172.21.15.178:0/3212163771 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9dc008d60 0x7fe9fc139a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec003fd0 tx=0x7fe9ec003cf0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9ea7fc640 1 -- 172.21.15.178:0/3212163771 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9ec007e30 con 0x7fe9dc008d60 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9ea7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9ea7fc640 10 monclient: dump: 2024-09-15T01:36:45.553 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:45.554 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9ea7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9ea7fc640 1 -- 172.21.15.178:0/3212163771 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ec00f040 con 0x7fe9dc008d60 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fea00c6f640 5 monclient: authenticate success, global_id 24306 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fea00c6f640 10 monclient: _renew_subs 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fea00c6f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fea00c6f640 1 -- 172.21.15.178:0/3212163771 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc13a500 con 0x7fe9dc008d60 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9ea7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9ea7fc640 1 -- 172.21.15.178:0/3212163771 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9ec00b2e0 con 0x7fe9dc008d60 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9ea7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:45.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9ea7fc640 10 monclient: dump: 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:45.556 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fea00c6f640 10 monclient: _renew_subs 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fe9f8d99640 4 set_mon_vals no callback set 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fea00c6f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.536+0000 7fea00c6f640 1 -- 172.21.15.178:0/3212163771 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc11be70 con 0x7fe9dc008d60 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.537+0000 7fe9ea7fc640 1 -- 172.21.15.178:0/3212163771 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe9ec0079b0 con 0x7fe9dc008d60 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.537+0000 7fea00c6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.537+0000 7fea00c6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.537+0000 7fea00c6f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.537+0000 7fea00c6f640 1 -- 172.21.15.178:0/3212163771 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9fc09c070 con 0x7fe9dc008d60 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.538+0000 7fe9ea7fc640 1 --2- 172.21.15.178:0/3212163771 >> v2:172.21.15.204:6812/28210 conn(0x7fe9dc03a8f0 0x7fe9dc03cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.538+0000 7fe9ea7fc640 1 -- 172.21.15.178:0/3212163771 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fe9ec04cae0 con 0x7fe9dc008d60 2024-09-15T01:36:45.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.538+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/3212163771 >> v2:172.21.15.204:6812/28210 conn(0x7fe9dc03a8f0 0x7fe9dc03cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:45.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.538+0000 7fe9f9d9b640 10 monclient: get_auth_request con 0x7fe9dc03a8f0 auth_method 0 2024-09-15T01:36:45.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.539+0000 7fe9f9d9b640 1 --2- 172.21.15.178:0/3212163771 >> v2:172.21.15.204:6812/28210 conn(0x7fe9dc03a8f0 0x7fe9dc03cda0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0000ea0 tx=0x7fe9f0016040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:45.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.542+0000 7fe9ea7fc640 1 -- 172.21.15.178:0/3212163771 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe9fc09c070 con 0x7fe9dc008d60 2024-09-15T01:36:45.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.542+0000 7fe9ea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:45.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.542+0000 7fe9ea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:45.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:45.698+0000 7fc57dffb640 10 monclient: tick 2024-09-15T01:36:45.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:45.698+0000 7fc57dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:45.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.758+0000 7fea00c6f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:36:45.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.758+0000 7fea00c6f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:45.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.758+0000 7fea00c6f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:45.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.758+0000 7fea00c6f640 1 -- 172.21.15.178:0/3212163771 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe9fc09c070 con 0x7fe9dc008d60 2024-09-15T01:36:45.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.759+0000 7fe9ea7fc640 1 -- 172.21.15.178:0/3212163771 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3159 (secure 0 0 0) 0x7fe9fc09c070 con 0x7fe9dc008d60 2024-09-15T01:36:45.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.759+0000 7fe9ea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:36:45.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.759+0000 7fe9ea7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:45.761 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":64,"quorum":[1,3,5,6,7],"quorum_names":["f","g","h","d","i"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:36:45.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 1 -- 172.21.15.178:0/3212163771 >> v2:172.21.15.204:6812/28210 conn(0x7fe9dc03a8f0 msgr2=0x7fe9dc03cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:45.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 1 --2- 172.21.15.178:0/3212163771 >> v2:172.21.15.204:6812/28210 conn(0x7fe9dc03a8f0 0x7fe9dc03cda0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0000ea0 tx=0x7fe9f0016040 comp rx=0 tx=0).stop 2024-09-15T01:36:45.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 10 monclient: shutdown 2024-09-15T01:36:45.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:45.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 1 -- 172.21.15.178:0/3212163771 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9dc008d60 msgr2=0x7fe9fc139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:45.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 1 --2- 172.21.15.178:0/3212163771 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9dc008d60 0x7fe9fc139a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec003fd0 tx=0x7fe9ec003cf0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 1 -- 172.21.15.178:0/3212163771 shutdown_connections 2024-09-15T01:36:45.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 1 --2- 172.21.15.178:0/3212163771 >> v2:172.21.15.204:6812/28210 conn(0x7fe9dc03a8f0 0x7fe9dc03cda0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 1 --2- 172.21.15.178:0/3212163771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4004de0 0x7fe9fc13cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 1 --2- 172.21.15.178:0/3212163771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e4004220 0x7fe9fc12da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 1 --2- 172.21.15.178:0/3212163771 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe9dc008d60 0x7fe9fc139a80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 1 -- 172.21.15.178:0/3212163771 >> 172.21.15.178:0/3212163771 conn(0x7fe9fc067d20 msgr2=0x7fe9fc0a0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:45.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 1 -- 172.21.15.178:0/3212163771 shutdown_connections 2024-09-15T01:36:45.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.761+0000 7fea00c6f640 1 -- 172.21.15.178:0/3212163771 wait complete. 2024-09-15T01:36:45.774 INFO:tasks.mon_thrash.ceph_manager:quorum is size 5 2024-09-15T01:36:45.774 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon d 2024-09-15T01:36:45.774 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:36:45.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.865+0000 7fe6ca290640 10 monclient: get_monmap_and_config 2024-09-15T01:36:45.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.865+0000 7fe6ca290640 10 monclient: build_initial_monmap 2024-09-15T01:36:45.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.865+0000 7fe6ca290640 10 monclient: monmap: 2024-09-15T01:36:45.868 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:45.868 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:45.868 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:45.866355+0000 2024-09-15T01:36:45.868 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:45.866355+0000 2024-09-15T01:36:45.868 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:45.868 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:45.868 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:45.868 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:45.868 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:45.868 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:45.869 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:45.869 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:45.869 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:45.869 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:45.869 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:45.869 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:45.869 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:45.869 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:45.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 1 Processor -- start 2024-09-15T01:36:45.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 1 -- start start 2024-09-15T01:36:45.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 10 monclient: init 2024-09-15T01:36:45.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:45.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 10 monclient: _add_conns ranks=[6,7,3,10,0,1,8,5,2,4,9] 2024-09-15T01:36:45.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c40a2930 0x7fe6c4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 10 monclient(hunting): picked mon.noname-d con 0x7fe6c40a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6c409faa0 0x7fe6c409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 10 monclient(hunting): picked mon.noname-i con 0x7fe6c409faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c40a08c0 0x7fe6c40a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 10 monclient(hunting): picked mon.noname-g con 0x7fe6c40a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c409e0f0 con 0x7fe6c40a08c0 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c409dd80 con 0x7fe6c40a2930 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40a2d00 con 0x7fe6c409faa0 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:45.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.866+0000 7fe6ca290640 10 monclient(hunting): authenticate will time out at 1010.616899s 2024-09-15T01:36:45.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c40a2930 0x7fe6c4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:45.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c40a2930 0x7fe6c4132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:40832/0 (socket says 172.21.15.178:40832) 2024-09-15T01:36:45.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 1 -- 172.21.15.178:0/16662759 learned_addr learned my addr 172.21.15.178:0/16662759 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:45.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 10 monclient(hunting): get_auth_request con 0x7fe6c40a2930 auth_method 0 2024-09-15T01:36:45.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:45.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:45.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:45.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:45.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:45.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c882d640 1 --2- 172.21.15.178:0/16662759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c40a08c0 0x7fe6c40a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:45.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:45.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c882d640 10 monclient(hunting): get_auth_request con 0x7fe6c40a08c0 auth_method 0 2024-09-15T01:36:45.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6bbfff640 1 --2- 172.21.15.178:0/16662759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6c409faa0 0x7fe6c409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:45.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c882d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:45.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c882d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:45.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c882d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:45.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6bbfff640 10 monclient(hunting): get_auth_request con 0x7fe6c409faa0 auth_method 0 2024-09-15T01:36:45.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:45.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6bbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:45.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:45.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c882d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:45.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c882d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:45.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c882d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:45.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 10 monclient(hunting): handle_auth_done global_id 24315 payload 274 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 1 -- 172.21.15.178:0/16662759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6c409faa0 msgr2=0x7fe6c409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 1 --2- 172.21.15.178:0/16662759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6c409faa0 0x7fe6c409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 1 -- 172.21.15.178:0/16662759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c40a08c0 msgr2=0x7fe6c40a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 1 --2- 172.21.15.178:0/16662759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c40a08c0 0x7fe6c40a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 1 monclient: found mon.noname-d 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6c3fff640 1 -- 172.21.15.178:0/16662759 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c404bf60 con 0x7fe6c40a2930 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.867+0000 7fe6bbfff640 1 --2- 172.21.15.178:0/16662759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6c409faa0 0x7fe6c409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c3fff640 1 --2- 172.21.15.178:0/16662759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c40a2930 0x7fe6c4132de0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009940 tx=0x7fe6b40091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=6055306970676468 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c37fe640 1 -- 172.21.15.178:0/16662759 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6b402ac60 con 0x7fe6c40a2930 2024-09-15T01:36:45.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c37fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c37fe640 10 monclient: dump: 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:45.880 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:45.881 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:45.881 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:45.881 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:45.881 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:45.881 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:45.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c37fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:45.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:45.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 5 monclient: authenticate success, global_id 24315 2024-09-15T01:36:45.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:45.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c37fe640 1 -- 172.21.15.178:0/16662759 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b402ade0 con 0x7fe6c40a2930 2024-09-15T01:36:45.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:45.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 4 set_mon_vals no callback set 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c37fe640 1 -- 172.21.15.178:0/16662759 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6b4031ca0 con 0x7fe6c40a2930 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6c37fe640 10 monclient: dump: 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:45.882 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 10 monclient: shutdown 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 1 -- 172.21.15.178:0/16662759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c40a2930 msgr2=0x7fe6c4132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 1 --2- 172.21.15.178:0/16662759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c40a2930 0x7fe6c4132de0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009940 tx=0x7fe6b40091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 1 -- 172.21.15.178:0/16662759 shutdown_connections 2024-09-15T01:36:45.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 1 --2- 172.21.15.178:0/16662759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c40a08c0 0x7fe6c40a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 1 --2- 172.21.15.178:0/16662759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6c409faa0 0x7fe6c409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 1 --2- 172.21.15.178:0/16662759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c40a2930 0x7fe6c4132de0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 1 -- 172.21.15.178:0/16662759 >> 172.21.15.178:0/16662759 conn(0x7fe6c40a5ed0 msgr2=0x7fe6c40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 1 -- 172.21.15.178:0/16662759 shutdown_connections 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.868+0000 7fe6ca290640 1 -- 172.21.15.178:0/16662759 wait complete. 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.869+0000 7fe6ca290640 10 monclient: build_initial_monmap 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.869+0000 7fe6ca290640 10 monclient: monmap: 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:45.884 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.869+0000 7fe6ca290640 1 Processor -- start 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.869+0000 7fe6ca290640 1 -- start start 2024-09-15T01:36:45.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.869+0000 7fe6ca290640 10 monclient: init 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 10 monclient: _add_conns ranks=[0,2,4,1,8,7,6,5,3] 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6c409faa0 0x7fe6c409b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 10 monclient(hunting): picked mon.noname-a con 0x7fe6c409faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6c40a08c0 0x7fe6c4131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 10 monclient(hunting): picked mon.noname-b con 0x7fe6c40a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe6c40a2930 0x7fe6c411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 10 monclient(hunting): picked mon.noname-c con 0x7fe6c40a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c411ec40 con 0x7fe6c409faa0 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c409d630 con 0x7fe6c40a08c0 2024-09-15T01:36:45.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:45.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c409bc20 con 0x7fe6c40a2930 2024-09-15T01:36:45.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:45.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6c3fff640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6c409faa0 msgr2=0x7fe6c409b2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:45.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6ca290640 10 monclient(hunting): authenticate will time out at 1010.620102s 2024-09-15T01:36:45.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6bbfff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6c40a08c0 msgr2=0x7fe6c4131840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:45.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6c409faa0 0x7fe6c409b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:45.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6c882d640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe6c40a2930 msgr2=0x7fe6c411c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:45.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6bbfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6c40a08c0 0x7fe6c4131840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:45.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:45.870+0000 7fe6c882d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe6c40a2930 0x7fe6c411c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:46.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:46.071+0000 7fe6bbfff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6c40a08c0 msgr2=0x7fe6c4131840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:46.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:46.071+0000 7fe6c882d640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe6c40a2930 msgr2=0x7fe6c411c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:46.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:46.071+0000 7fe6c3fff640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6c409faa0 msgr2=0x7fe6c409b2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:46.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:46.071+0000 7fe6bbfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6c40a08c0 0x7fe6c4131840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:46.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:46.071+0000 7fe6c882d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe6c40a2930 0x7fe6c411c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:46.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:46.071+0000 7fe6c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6c409faa0 0x7fe6c409b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:46.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:46.472+0000 7fe6bbfff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6c40a08c0 msgr2=0x7fe6c4131840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:46.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:46.472+0000 7fe6c3fff640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6c409faa0 msgr2=0x7fe6c409b2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:46.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:46.472+0000 7fe6c882d640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe6c40a2930 msgr2=0x7fe6c411c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:46.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:46.472+0000 7fe6bbfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6c40a08c0 0x7fe6c4131840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:46.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:46.472+0000 7fe6c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6c409faa0 0x7fe6c409b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:46.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:46.472+0000 7fe6c882d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe6c40a2930 0x7fe6c411c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:46.529 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.527+0000 7fc57effd640 1 -- 172.21.15.204:0/2831552544 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}]=0 moved item id -16 name 'fooo' to location {rack=barr} in crush map v178) v1 ==== 167+0+0 (secure 0 0 0) 0x7fc588041070 con 0x7fc59809fb80 2024-09-15T01:36:46.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.527+0000 7fc57effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-09-15T01:36:46.530 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.527+0000 7fc57effd640 10 monclient: _finish_command 2 = system:0 moved item id -16 name 'fooo' to location {rack=barr} in crush map 2024-09-15T01:36:46.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.628+0000 7fc59d59d640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-09-15T01:36:46.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.628+0000 7fc59d59d640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-09-15T01:36:46.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.628+0000 7fc59d59d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:46.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.628+0000 7fc59d59d640 1 -- 172.21.15.204:0/2831552544 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]} v 0) v1 -- 0x7fc598070cc0 con 0x7fc59809fb80 2024-09-15T01:36:46.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.629+0000 7fc57effd640 1 -- 172.21.15.204:0/2831552544 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}]=0 no need to move item id -16 name 'fooo' to location {rack=barr} in crush map v178) v1 ==== 177+0+0 (secure 0 0 0) 0x7fc588037020 con 0x7fc59809fb80 2024-09-15T01:36:46.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.629+0000 7fc57effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-09-15T01:36:46.631 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.629+0000 7fc57effd640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'fooo' to location {rack=barr} in crush map 2024-09-15T01:36:46.632 INFO:tasks.workunit.client.0.smithi204.stderr:no need to move item id -16 name 'fooo' to location {rack=barr} in crush map 2024-09-15T01:36:46.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 1 -- 172.21.15.204:0/2831552544 >> v2:172.21.15.204:6812/28210 conn(0x7fc5840397e0 msgr2=0x7fc58403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:46.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 1 --2- 172.21.15.204:0/2831552544 >> v2:172.21.15.204:6812/28210 conn(0x7fc5840397e0 0x7fc58403bc90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc5980a2e00 tx=0x7fc58c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:46.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 10 monclient: shutdown 2024-09-15T01:36:46.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:46.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 1 -- 172.21.15.204:0/2831552544 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc59809fb80 msgr2=0x7fc598131650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:46.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 1 --2- 172.21.15.204:0/2831552544 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc59809fb80 0x7fc598131650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc588035040 tx=0x7fc588032000 comp rx=0 tx=0).stop 2024-09-15T01:36:46.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 1 -- 172.21.15.204:0/2831552544 shutdown_connections 2024-09-15T01:36:46.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 1 --2- 172.21.15.204:0/2831552544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5980a2e30 0x7fc59811e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:46.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 1 --2- 172.21.15.204:0/2831552544 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc5980a0a40 0x7fc59811e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:46.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 1 --2- 172.21.15.204:0/2831552544 >> v2:172.21.15.204:6812/28210 conn(0x7fc5840397e0 0x7fc58403bc90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:46.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 1 --2- 172.21.15.204:0/2831552544 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc59809fb80 0x7fc598131650 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:46.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 1 -- 172.21.15.204:0/2831552544 >> 172.21.15.204:0/2831552544 conn(0x7fc59806e860 msgr2=0x7fc59809ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:46.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 1 -- 172.21.15.204:0/2831552544 shutdown_connections 2024-09-15T01:36:46.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.631+0000 7fc59d59d640 1 -- 172.21.15.204:0/2831552544 wait complete. 2024-09-15T01:36:46.643 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm fooo 2024-09-15T01:36:46.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.714+0000 7fcef28c0640 10 monclient: get_monmap_and_config 2024-09-15T01:36:46.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.714+0000 7fcef28c0640 10 monclient: build_initial_monmap 2024-09-15T01:36:46.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.714+0000 7fcef28c0640 10 monclient: monmap: 2024-09-15T01:36:46.718 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:46.718 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:46.716405+0000 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:46.716405+0000 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:46.719 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:46.720 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:46.720 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:46.720 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:46.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.715+0000 7fcef28c0640 1 Processor -- start 2024-09-15T01:36:46.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.715+0000 7fcef28c0640 1 -- start start 2024-09-15T01:36:46.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.715+0000 7fcef28c0640 10 monclient: init 2024-09-15T01:36:46.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:46.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 10 monclient: _add_conns ranks=[1,3,5,7,10,4,6,2,0,8,9] 2024-09-15T01:36:46.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fceec0a2e30 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fceec09fb80 0x7fceec09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 10 monclient(hunting): picked mon.noname-g con 0x7fceec09fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fceec137d20 0x7fceec13a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 10 monclient(hunting): picked mon.noname-h con 0x7fceec137d20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceec04b720 con 0x7fceec0a2e30 2024-09-15T01:36:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceec09e6c0 con 0x7fceec09fb80 2024-09-15T01:36:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fceec0a04b0 con 0x7fceec137d20 2024-09-15T01:36:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef28c0640 10 monclient(hunting): authenticate will time out at 1022.359426s 2024-09-15T01:36:46.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fceebfff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fceec0a2e30 0x7fceec132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:46144/0 (socket says 172.21.15.204:46144) 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fceebfff640 1 -- 172.21.15.204:0/3217832619 learned_addr learned my addr 172.21.15.204:0/3217832619 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 1 --2- 172.21.15.204:0/3217832619 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fceec137d20 0x7fceec13a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fceeb7fe640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fceec09fb80 msgr2=0x7fceec09ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fceeb7fe640 1 -- 172.21.15.204:0/3217832619 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fceec09fb80 msgr2=0x7fceec09ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fceeb7fe640 1 --2- 172.21.15.204:0/3217832619 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fceec09fb80 0x7fceec09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 10 monclient(hunting): get_auth_request con 0x7fceec137d20 auth_method 0 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fceeb7fe640 1 --2- 172.21.15.204:0/3217832619 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fceec09fb80 0x7fceec09ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:46.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 10 monclient(hunting): handle_auth_done global_id 6080 payload 274 2024-09-15T01:36:46.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fceeaffd640 1 -- 172.21.15.204:0/3217832619 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655447057 0 0) 0x7fceec04b720 con 0x7fceec0a2e30 2024-09-15T01:36:46.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 1 -- 172.21.15.204:0/3217832619 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fceec09fb80 msgr2=0x7fceec09ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:46.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 1 --2- 172.21.15.204:0/3217832619 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fceec09fb80 0x7fceec09ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:46.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 1 -- 172.21.15.204:0/3217832619 >> v1:172.21.15.204:6789/0 conn(0x7fceec0a2e30 legacy=0x7fceec132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:46.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 10 monclient: _finish_hunting 0 2024-09-15T01:36:46.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 1 monclient: found mon.noname-h 2024-09-15T01:36:46.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:46.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:46.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fcef0e5d640 1 -- 172.21.15.204:0/3217832619 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec133170 con 0x7fceec137d20 2024-09-15T01:36:46.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.716+0000 7fceeaffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:36:46.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fcef0e5d640 1 --2- 172.21.15.204:0/3217832619 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fceec137d20 0x7fceec13a100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009e90 tx=0x7fcee4009780 comp rx=0 tx=0).ready entity=mon.5 client_cookie=30697213f3dd74b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:46.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fceeaffd640 1 -- 172.21.15.204:0/3217832619 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcee4031880 con 0x7fceec137d20 2024-09-15T01:36:46.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fceeaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:46.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fceeaffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:46.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fceeaffd640 10 monclient: dump: 2024-09-15T01:36:46.732 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:46.732 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:46.732 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:46.732 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:46.732 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:46.732 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:46.732 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:46.732 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:46.732 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:46.733 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:46.733 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:46.733 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:46.733 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:46.733 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:46.733 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:46.733 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:46.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fceeaffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:46.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fceeaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:46.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fceeaffd640 1 -- 172.21.15.204:0/3217832619 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee402d050 con 0x7fceec137d20 2024-09-15T01:36:46.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fcef28c0640 5 monclient: authenticate success, global_id 6080 2024-09-15T01:36:46.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fcef28c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:46.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fceeaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:46.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fceeaffd640 1 -- 172.21.15.204:0/3217832619 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcee40303a0 con 0x7fceec137d20 2024-09-15T01:36:46.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fcef28c0640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:46.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fcef28c0640 4 set_mon_vals no callback set 2024-09-15T01:36:46.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fceeaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:46.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fceeaffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:46.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.717+0000 7fceeaffd640 10 monclient: dump: 2024-09-15T01:36:46.734 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:46.734 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:46.734 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:46.734 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:46.734 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.718+0000 7fcef28c0640 10 monclient: shutdown 2024-09-15T01:36:46.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.718+0000 7fcef28c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.718+0000 7fcef28c0640 1 -- 172.21.15.204:0/3217832619 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fceec137d20 msgr2=0x7fceec13a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.718+0000 7fcef28c0640 1 --2- 172.21.15.204:0/3217832619 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fceec137d20 0x7fceec13a100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009e90 tx=0x7fcee4009780 comp rx=0 tx=0).stop 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.718+0000 7fcef28c0640 1 -- 172.21.15.204:0/3217832619 shutdown_connections 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.718+0000 7fcef28c0640 1 --2- 172.21.15.204:0/3217832619 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fceec137d20 0x7fceec13a100 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.718+0000 7fcef28c0640 1 --2- 172.21.15.204:0/3217832619 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fceec09fb80 0x7fceec09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.718+0000 7fcef28c0640 1 -- 172.21.15.204:0/3217832619 >> 172.21.15.204:0/3217832619 conn(0x7fceec06e860 msgr2=0x7fceec06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.718+0000 7fcef28c0640 1 -- 172.21.15.204:0/3217832619 shutdown_connections 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.718+0000 7fcef28c0640 1 -- 172.21.15.204:0/3217832619 wait complete. 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.718+0000 7fcef28c0640 10 monclient: build_initial_monmap 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.718+0000 7fcef28c0640 10 monclient: monmap: 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:46.736 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:46.737 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 1 Processor -- start 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 1 -- start start 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 10 monclient: init 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 10 monclient: _add_conns ranks=[6,1,5,8,0,4,3,7,2] 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fceec09fb80 0x7fceec132bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 10 monclient(hunting): picked mon.noname-d con 0x7fceec09fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceec0a2e30 0x7fceec09a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 10 monclient(hunting): picked mon.noname-f con 0x7fceec0a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fceec137d20 0x7fceec09ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 10 monclient(hunting): picked mon.noname-h con 0x7fceec137d20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:46.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec11db40 con 0x7fceec0a2e30 2024-09-15T01:36:46.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:46.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fceec11e090 con 0x7fceec137d20 2024-09-15T01:36:46.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:46.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fceec11ddb0 con 0x7fceec09fb80 2024-09-15T01:36:46.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:46.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef28c0640 10 monclient(hunting): authenticate will time out at 1022.362808s 2024-09-15T01:36:46.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fceeb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceec0a2e30 0x7fceec09a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:46.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef0e5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fceec137d20 0x7fceec09ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:46.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fceeb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceec0a2e30 0x7fceec09a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:54214/0 (socket says 172.21.15.204:54214) 2024-09-15T01:36:46.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fceeb7fe640 1 -- 172.21.15.204:0/3473033759 learned_addr learned my addr 172.21.15.204:0/3473033759 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:46.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fceeb7fe640 10 monclient(hunting): get_auth_request con 0x7fceec0a2e30 auth_method 0 2024-09-15T01:36:46.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fceeb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:46.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fceeb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:46.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fceeb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:46.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef0e5d640 10 monclient(hunting): get_auth_request con 0x7fceec137d20 auth_method 0 2024-09-15T01:36:46.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef0e5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:46.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef0e5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:46.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef0e5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:46.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef0e5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:46.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.719+0000 7fcef0e5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:46.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef0e5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fceeb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fceeb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fceeb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fceebfff640 1 --2- 172.21.15.204:0/3473033759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fceec09fb80 0x7fceec132bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fceebfff640 10 monclient(hunting): get_auth_request con 0x7fceec09fb80 auth_method 0 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fceebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fceebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fceebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef0e5d640 10 monclient(hunting): handle_auth_done global_id 6089 payload 824 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef0e5d640 1 -- 172.21.15.204:0/3473033759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fceec09fb80 msgr2=0x7fceec132bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef0e5d640 1 --2- 172.21.15.204:0/3473033759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fceec09fb80 0x7fceec132bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef0e5d640 1 -- 172.21.15.204:0/3473033759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceec0a2e30 msgr2=0x7fceec09a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef0e5d640 1 --2- 172.21.15.204:0/3473033759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceec0a2e30 0x7fceec09a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:46.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef0e5d640 10 monclient: _finish_hunting 0 2024-09-15T01:36:46.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef0e5d640 1 monclient: found mon.noname-h 2024-09-15T01:36:46.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef0e5d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:46.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef0e5d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:46.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef0e5d640 1 -- 172.21.15.204:0/3473033759 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee4009a20 con 0x7fceec137d20 2024-09-15T01:36:46.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fceeb7fe640 1 --2- 172.21.15.204:0/3473033759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceec0a2e30 0x7fceec09a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:46.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef0e5d640 1 --2- 172.21.15.204:0/3473033759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fceec137d20 0x7fceec09ac40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcee4031000 tx=0x7fcee4037000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:46.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fced3fff640 1 -- 172.21.15.204:0/3473033759 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcee4009460 con 0x7fceec137d20 2024-09-15T01:36:46.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fced3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:46.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fced3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fced3fff640 10 monclient: dump: 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:46.743 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:46.744 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:46.744 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:46.744 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:46.744 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:46.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fced3fff640 10 monclient: _finish_auth 0 2024-09-15T01:36:46.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fced3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:46.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef28c0640 5 monclient: authenticate success, global_id 6089 2024-09-15T01:36:46.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef28c0640 10 monclient: _renew_subs 2024-09-15T01:36:46.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fced3fff640 1 -- 172.21.15.204:0/3473033759 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee403c420 con 0x7fceec137d20 2024-09-15T01:36:46.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef28c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef28c0640 1 -- 172.21.15.204:0/3473033759 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceec11e370 con 0x7fceec137d20 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fced3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fced3fff640 1 -- 172.21.15.204:0/3473033759 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcee403ca10 con 0x7fceec137d20 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fced3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fced3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fced3fff640 10 monclient: dump: 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:46.745 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:46.746 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:46.746 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:46.746 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:46.746 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:46.746 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:46.746 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:46.746 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:46.746 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:46.746 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:46.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef28c0640 10 monclient: _renew_subs 2024-09-15T01:36:46.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef28c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:46.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fcef28c0640 1 -- 172.21.15.204:0/3473033759 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceec11e790 con 0x7fceec137d20 2024-09-15T01:36:46.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.720+0000 7fceea7fc640 4 set_mon_vals no callback set 2024-09-15T01:36:46.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.721+0000 7fced3fff640 1 -- 172.21.15.204:0/3473033759 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fcee4003040 con 0x7fceec137d20 2024-09-15T01:36:46.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.721+0000 7fcef28c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:46.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.721+0000 7fcef28c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:46.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.721+0000 7fcef28c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:46.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.721+0000 7fcef28c0640 1 -- 172.21.15.204:0/3473033759 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceec070cc0 con 0x7fceec137d20 2024-09-15T01:36:46.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.721+0000 7fced3fff640 1 --2- 172.21.15.204:0/3473033759 >> v2:172.21.15.204:6812/28210 conn(0x7fced80396d0 0x7fced803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:46.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.721+0000 7fced3fff640 1 -- 172.21.15.204:0/3473033759 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fcee406cf30 con 0x7fceec137d20 2024-09-15T01:36:46.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.721+0000 7fceebfff640 1 --2- 172.21.15.204:0/3473033759 >> v2:172.21.15.204:6812/28210 conn(0x7fced80396d0 0x7fced803bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:46.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.721+0000 7fceebfff640 10 monclient: get_auth_request con 0x7fced80396d0 auth_method 0 2024-09-15T01:36:46.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.721+0000 7fceebfff640 1 --2- 172.21.15.204:0/3473033759 >> v2:172.21.15.204:6812/28210 conn(0x7fced80396d0 0x7fced803bb80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0011e0 tx=0x7fcedc009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:46.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.725+0000 7fced3fff640 1 -- 172.21.15.204:0/3473033759 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcee409ee40 con 0x7fceec137d20 2024-09-15T01:36:46.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.726+0000 7fced3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:46.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.726+0000 7fced3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:46.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.885+0000 7fcef28c0640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "fooo"}] 2024-09-15T01:36:46.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.885+0000 7fcef28c0640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-09-15T01:36:46.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.885+0000 7fcef28c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:46.888 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:46.885+0000 7fcef28c0640 1 -- 172.21.15.204:0/3473033759 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "fooo"} v 0) v1 -- 0x7fceec067590 con 0x7fceec137d20 2024-09-15T01:36:47.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:47.273+0000 7fe6bbfff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6c40a08c0 msgr2=0x7fe6c4131840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:47.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:47.273+0000 7fe6c3fff640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6c409faa0 msgr2=0x7fe6c409b2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:47.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:47.273+0000 7fe6c882d640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe6c40a2930 msgr2=0x7fe6c411c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:47.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:47.273+0000 7fe6bbfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6c40a08c0 0x7fe6c4131840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:47.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:47.273+0000 7fe6c882d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe6c40a2930 0x7fe6c411c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:47.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:47.273+0000 7fe6c3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6c409faa0 0x7fe6c409b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:47.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.538+0000 7fced3fff640 1 -- 172.21.15.204:0/3473033759 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "fooo"}]=0 removed item id -16 name 'fooo' from crush map v179) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcee403c770 con 0x7fceec137d20 2024-09-15T01:36:47.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.538+0000 7fced3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-09-15T01:36:47.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.538+0000 7fced3fff640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'fooo' from crush map 2024-09-15T01:36:47.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.636+0000 7fcef28c0640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "fooo"}] 2024-09-15T01:36:47.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.636+0000 7fcef28c0640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-09-15T01:36:47.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.636+0000 7fcef28c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:47.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.636+0000 7fcef28c0640 1 -- 172.21.15.204:0/3473033759 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "fooo"} v 0) v1 -- 0x7fceec11cbe0 con 0x7fceec137d20 2024-09-15T01:36:47.639 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.637+0000 7fced3fff640 1 -- 172.21.15.204:0/3473033759 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "fooo"}]=0 device 'fooo' does not appear in the crush map v179) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcee4038350 con 0x7fceec137d20 2024-09-15T01:36:47.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.637+0000 7fced3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-09-15T01:36:47.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.637+0000 7fced3fff640 10 monclient: _finish_command 3 = system:0 device 'fooo' does not appear in the crush map 2024-09-15T01:36:47.640 INFO:tasks.workunit.client.0.smithi204.stderr:device 'fooo' does not appear in the crush map 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 1 -- 172.21.15.204:0/3473033759 >> v2:172.21.15.204:6812/28210 conn(0x7fced80396d0 msgr2=0x7fced803bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 1 --2- 172.21.15.204:0/3473033759 >> v2:172.21.15.204:6812/28210 conn(0x7fced80396d0 0x7fced803bb80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0011e0 tx=0x7fcedc009040 comp rx=0 tx=0).stop 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 10 monclient: shutdown 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 1 -- 172.21.15.204:0/3473033759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fceec137d20 msgr2=0x7fceec09ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 1 --2- 172.21.15.204:0/3473033759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fceec137d20 0x7fceec09ac40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcee4031000 tx=0x7fcee4037000 comp rx=0 tx=0).stop 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 1 -- 172.21.15.204:0/3473033759 shutdown_connections 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 1 --2- 172.21.15.204:0/3473033759 >> v2:172.21.15.204:6812/28210 conn(0x7fced80396d0 0x7fced803bb80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 1 --2- 172.21.15.204:0/3473033759 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fceec137d20 0x7fceec09ac40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 1 --2- 172.21.15.204:0/3473033759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fceec0a2e30 0x7fceec09a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 1 --2- 172.21.15.204:0/3473033759 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fceec09fb80 0x7fceec132bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 1 -- 172.21.15.204:0/3473033759 >> 172.21.15.204:0/3473033759 conn(0x7fceec06e860 msgr2=0x7fceec0a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:47.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 1 -- 172.21.15.204:0/3473033759 shutdown_connections 2024-09-15T01:36:47.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.639+0000 7fcef28c0640 1 -- 172.21.15.204:0/3473033759 wait complete. 2024-09-15T01:36:47.650 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm barr 2024-09-15T01:36:47.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.722+0000 7fa50fc6f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:47.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.722+0000 7fa50fc6f640 10 monclient: build_initial_monmap 2024-09-15T01:36:47.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.723+0000 7fa50fc6f640 10 monclient: monmap: 2024-09-15T01:36:47.725 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:47.726 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:47.726 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:47.724479+0000 2024-09-15T01:36:47.726 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:47.724479+0000 2024-09-15T01:36:47.726 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:47.726 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:47.726 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:47.726 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:47.726 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:47.726 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:47.726 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:47.727 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:47.727 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:47.727 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:47.727 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:47.727 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:47.727 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:47.727 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:47.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.723+0000 7fa50fc6f640 1 Processor -- start 2024-09-15T01:36:47.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.723+0000 7fa50fc6f640 1 -- start start 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.723+0000 7fa50fc6f640 10 monclient: init 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 10 monclient: _add_conns ranks=[10,7,5,6,4,9,0,1,2,3,8] 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa5080a2e30 0x7fa508132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 10 monclient(hunting): picked mon.noname-f con 0x7fa5080a2e30 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa50809fb80 0x7fa50809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 10 monclient(hunting): picked mon.noname-i con 0x7fa50809fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa5080a0a40 0x7fa5080a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 10 monclient(hunting): picked mon.noname-h con 0x7fa5080a0a40 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50809e6c0 con 0x7fa5080a2e30 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:47.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080a04b0 con 0x7fa5080a0a40 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080a3200 con 0x7fa50809fb80 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50fc6f640 10 monclient(hunting): authenticate will time out at 1023.367403s 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50d20a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa50809fb80 0x7fa50809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa5080a2e30 0x7fa508132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50e20c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa5080a0a40 0x7fa5080a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50d20a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa50809fb80 0x7fa50809ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:60390/0 (socket says 172.21.15.204:60390) 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa5080a2e30 0x7fa508132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:50612/0 (socket says 172.21.15.204:50612) 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50d20a640 1 -- 172.21.15.204:0/822436154 learned_addr learned my addr 172.21.15.204:0/822436154 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50e20c640 10 monclient(hunting): get_auth_request con 0x7fa5080a0a40 auth_method 0 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50e20c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:47.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50e20c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50e20c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 10 monclient(hunting): get_auth_request con 0x7fa5080a2e30 auth_method 0 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50d20a640 10 monclient(hunting): get_auth_request con 0x7fa50809fb80 auth_method 0 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50d20a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50d20a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50d20a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:47.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50e20c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50e20c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50e20c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50d20a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50d20a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50d20a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 10 monclient(hunting): handle_auth_done global_id 6787 payload 274 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 1 -- 172.21.15.204:0/822436154 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa50809fb80 msgr2=0x7fa50809ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 1 --2- 172.21.15.204:0/822436154 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa50809fb80 0x7fa50809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 1 -- 172.21.15.204:0/822436154 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa5080a0a40 msgr2=0x7fa5080a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 1 --2- 172.21.15.204:0/822436154 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa5080a0a40 0x7fa5080a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 10 monclient: _finish_hunting 0 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 1 monclient: found mon.noname-f 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:47.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:47.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50da0b640 1 -- 172.21.15.204:0/822436154 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50804b720 con 0x7fa5080a2e30 2024-09-15T01:36:47.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.724+0000 7fa50e20c640 1 --2- 172.21.15.204:0/822436154 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa5080a0a40 0x7fa5080a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:47.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50da0b640 1 --2- 172.21.15.204:0/822436154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5080a2e30 0x7fa508132a60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa504009850 tx=0x7fa5040090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d7edf3832e73236 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:47.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50ca09640 1 -- 172.21.15.204:0/822436154 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa50402abc0 con 0x7fa5080a2e30 2024-09-15T01:36:47.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50ca09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:47.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50ca09640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:47.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50ca09640 10 monclient: dump: 2024-09-15T01:36:47.741 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:47.741 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:47.741 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:47.741 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:47.741 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50ca09640 10 monclient: _finish_auth 0 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50ca09640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:47.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50fc6f640 5 monclient: authenticate success, global_id 6787 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50ca09640 1 -- 172.21.15.204:0/822436154 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50402ad40 con 0x7fa5080a2e30 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50fc6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50ca09640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50fc6f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50fc6f640 4 set_mon_vals no callback set 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50ca09640 1 -- 172.21.15.204:0/822436154 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa504031ca0 con 0x7fa5080a2e30 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50ca09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50ca09640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.725+0000 7fa50ca09640 10 monclient: dump: 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:47.743 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 10 monclient: shutdown 2024-09-15T01:36:47.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 1 -- 172.21.15.204:0/822436154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5080a2e30 msgr2=0x7fa508132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 1 --2- 172.21.15.204:0/822436154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5080a2e30 0x7fa508132a60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa504009850 tx=0x7fa5040090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 1 -- 172.21.15.204:0/822436154 shutdown_connections 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 1 --2- 172.21.15.204:0/822436154 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa5080a0a40 0x7fa5080a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 1 --2- 172.21.15.204:0/822436154 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa50809fb80 0x7fa50809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 1 --2- 172.21.15.204:0/822436154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5080a2e30 0x7fa508132a60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 1 -- 172.21.15.204:0/822436154 >> 172.21.15.204:0/822436154 conn(0x7fa50806e860 msgr2=0x7fa50806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 1 -- 172.21.15.204:0/822436154 shutdown_connections 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 1 -- 172.21.15.204:0/822436154 wait complete. 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 10 monclient: build_initial_monmap 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.726+0000 7fa50fc6f640 10 monclient: monmap: 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:47.745 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:47.746 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 1 Processor -- start 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 1 -- start start 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 10 monclient: init 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 10 monclient: _add_conns ranks=[8,4,1,2,3,7,5,0,6] 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa50809fb80 0x7fa50809b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 10 monclient(hunting): picked mon.noname-e con 0x7fa50809fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa5080a0a40 0x7fa50812e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 10 monclient(hunting): picked mon.noname-c con 0x7fa5080a0a40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5080a2e30 0x7fa50811c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 10 monclient(hunting): picked mon.noname-f con 0x7fa5080a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50809ca70 con 0x7fa5080a2e30 2024-09-15T01:36:47.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa50809d680 con 0x7fa5080a0a40 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa508133f60 con 0x7fa50809fb80 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50fc6f640 10 monclient(hunting): authenticate will time out at 1023.370648s 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5080a2e30 0x7fa50811c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50d20a640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa5080a0a40 msgr2=0x7fa50812e750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50da0b640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa50809fb80 msgr2=0x7fa50809b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5080a2e30 0x7fa50811c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:50616/0 (socket says 172.21.15.204:50616) 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50d20a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa5080a0a40 0x7fa50812e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 1 -- 172.21.15.204:0/3541060727 learned_addr learned my addr 172.21.15.204:0/3541060727 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 10 monclient(hunting): get_auth_request con 0x7fa5080a2e30 auth_method 0 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:47.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50da0b640 1 --2- 172.21.15.204:0/3541060727 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa50809fb80 0x7fa50809b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 10 monclient(hunting): handle_auth_done global_id 6796 payload 824 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 1 -- 172.21.15.204:0/3541060727 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa50809fb80 msgr2=0x7fa50809b5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 1 --2- 172.21.15.204:0/3541060727 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa50809fb80 0x7fa50809b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 1 -- 172.21.15.204:0/3541060727 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa5080a0a40 msgr2=0x7fa50812e750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 1 --2- 172.21.15.204:0/3541060727 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa5080a0a40 0x7fa50812e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 1 monclient: found mon.noname-f 2024-09-15T01:36:47.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.727+0000 7fa50e20c640 1 -- 172.21.15.204:0/3541060727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa500009a20 con 0x7fa5080a2e30 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa50e20c640 1 --2- 172.21.15.204:0/3541060727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5080a2e30 0x7fa50811c6b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa500009fc0 tx=0x7fa50000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fdffb640 1 -- 172.21.15.204:0/3541060727 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa50000ce70 con 0x7fa5080a2e30 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fdffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fdffb640 10 monclient: dump: 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:47.750 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fdffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa50fc6f640 5 monclient: authenticate success, global_id 6796 2024-09-15T01:36:47.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fdffb640 1 -- 172.21.15.204:0/3541060727 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50000f040 con 0x7fa5080a2e30 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fdffb640 1 -- 172.21.15.204:0/3541060727 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa500009d60 con 0x7fa5080a2e30 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fdffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fdffb640 10 monclient: dump: 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:47.752 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa4fffff640 4 set_mon_vals no callback set 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa50fc6f640 10 monclient: _renew_subs 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa50fc6f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa50fc6f640 1 -- 172.21.15.204:0/3541060727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa504009390 con 0x7fa5080a2e30 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa50fc6f640 10 monclient: _renew_subs 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa50fc6f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:47.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.728+0000 7fa50fc6f640 1 -- 172.21.15.204:0/3541060727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50811d940 con 0x7fa5080a2e30 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.729+0000 7fa4fdffb640 1 -- 172.21.15.204:0/3541060727 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa500003020 con 0x7fa5080a2e30 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.729+0000 7fa50fc6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.729+0000 7fa4fdffb640 1 --2- 172.21.15.204:0/3541060727 >> v2:172.21.15.204:6812/28210 conn(0x7fa4ec0397b0 0x7fa4ec03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.729+0000 7fa50fc6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.729+0000 7fa50fc6f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.729+0000 7fa50fc6f640 1 -- 172.21.15.204:0/3541060727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa508070cc0 con 0x7fa5080a2e30 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.729+0000 7fa4fdffb640 1 -- 172.21.15.204:0/3541060727 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6959+0+0 (secure 0 0 0) 0x7fa50004c7e0 con 0x7fa5080a2e30 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.729+0000 7fa50da0b640 1 --2- 172.21.15.204:0/3541060727 >> v2:172.21.15.204:6812/28210 conn(0x7fa4ec0397b0 0x7fa4ec03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.729+0000 7fa50da0b640 10 monclient: get_auth_request con 0x7fa4ec0397b0 auth_method 0 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.730+0000 7fa50da0b640 1 --2- 172.21.15.204:0/3541060727 >> v2:172.21.15.204:6812/28210 conn(0x7fa4ec0397b0 0x7fa4ec03bc60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa504034000 tx=0x7fa504032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.734+0000 7fa4fdffb640 1 -- 172.21.15.204:0/3541060727 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa500018070 con 0x7fa5080a2e30 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.734+0000 7fa4fdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:47.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.734+0000 7fa4fdffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:47.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.893+0000 7fa50fc6f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "barr"}] 2024-09-15T01:36:47.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.893+0000 7fa50fc6f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "barr"}] 2024-09-15T01:36:47.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.893+0000 7fa50fc6f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:47.896 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:47.893+0000 7fa50fc6f640 1 -- 172.21.15.204:0/3541060727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "barr"} v 0) v1 -- 0x7fa5080a1f10 con 0x7fa5080a2e30 2024-09-15T01:36:48.543 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.541+0000 7fa4fdffb640 1 -- 172.21.15.204:0/3541060727 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "barr"}]=0 removed item id -19 name 'barr' from crush map v180) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa500012940 con 0x7fa5080a2e30 2024-09-15T01:36:48.544 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.541+0000 7fa4fdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "barr"}] 2024-09-15T01:36:48.544 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.541+0000 7fa4fdffb640 10 monclient: _finish_command 2 = system:0 removed item id -19 name 'barr' from crush map 2024-09-15T01:36:48.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.638+0000 7fa50fc6f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "barr"}] 2024-09-15T01:36:48.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.638+0000 7fa50fc6f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "barr"}] 2024-09-15T01:36:48.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.638+0000 7fa50fc6f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:48.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.638+0000 7fa50fc6f640 1 -- 172.21.15.204:0/3541060727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "barr"} v 0) v1 -- 0x7fa50809c0b0 con 0x7fa5080a2e30 2024-09-15T01:36:48.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.638+0000 7fa4fdffb640 1 -- 172.21.15.204:0/3541060727 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "barr"}]=0 device 'barr' does not appear in the crush map v180) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa50001b360 con 0x7fa5080a2e30 2024-09-15T01:36:48.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.638+0000 7fa4fdffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "barr"}] 2024-09-15T01:36:48.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.638+0000 7fa4fdffb640 10 monclient: _finish_command 3 = system:0 device 'barr' does not appear in the crush map 2024-09-15T01:36:48.642 INFO:tasks.workunit.client.0.smithi204.stderr:device 'barr' does not appear in the crush map 2024-09-15T01:36:48.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 1 -- 172.21.15.204:0/3541060727 >> v2:172.21.15.204:6812/28210 conn(0x7fa4ec0397b0 msgr2=0x7fa4ec03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:48.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 1 --2- 172.21.15.204:0/3541060727 >> v2:172.21.15.204:6812/28210 conn(0x7fa4ec0397b0 0x7fa4ec03bc60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa504034000 tx=0x7fa504032000 comp rx=0 tx=0).stop 2024-09-15T01:36:48.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 10 monclient: shutdown 2024-09-15T01:36:48.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:48.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 1 -- 172.21.15.204:0/3541060727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5080a2e30 msgr2=0x7fa50811c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:48.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 1 --2- 172.21.15.204:0/3541060727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5080a2e30 0x7fa50811c6b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa500009fc0 tx=0x7fa50000b040 comp rx=0 tx=0).stop 2024-09-15T01:36:48.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 1 -- 172.21.15.204:0/3541060727 shutdown_connections 2024-09-15T01:36:48.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 1 --2- 172.21.15.204:0/3541060727 >> v2:172.21.15.204:6812/28210 conn(0x7fa4ec0397b0 0x7fa4ec03bc60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 1 --2- 172.21.15.204:0/3541060727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5080a2e30 0x7fa50811c6b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 1 --2- 172.21.15.204:0/3541060727 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa5080a0a40 0x7fa50812e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 1 --2- 172.21.15.204:0/3541060727 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa50809fb80 0x7fa50809b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 1 -- 172.21.15.204:0/3541060727 >> 172.21.15.204:0/3541060727 conn(0x7fa50806e860 msgr2=0x7fa5080a5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:48.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 1 -- 172.21.15.204:0/3541060727 shutdown_connections 2024-09-15T01:36:48.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.640+0000 7fa50fc6f640 1 -- 172.21.15.204:0/3541060727 wait complete. 2024-09-15T01:36:48.651 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set rm-compat 2024-09-15T01:36:48.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.722+0000 7f2fe6e9f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:48.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.722+0000 7f2fe6e9f640 10 monclient: build_initial_monmap 2024-09-15T01:36:48.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.722+0000 7f2fe6e9f640 10 monclient: monmap: 2024-09-15T01:36:48.725 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:48.726 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:48.726 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:48.724337+0000 2024-09-15T01:36:48.726 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:48.724337+0000 2024-09-15T01:36:48.726 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:48.726 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:48.726 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:48.726 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:48.726 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:48.726 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:48.726 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:48.726 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:48.727 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:48.727 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:48.727 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:48.727 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:48.727 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:48.727 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:48.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.723+0000 7f2fe6e9f640 1 Processor -- start 2024-09-15T01:36:48.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.723+0000 7f2fe6e9f640 1 -- start start 2024-09-15T01:36:48.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.723+0000 7f2fe6e9f640 10 monclient: init 2024-09-15T01:36:48.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:48.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 10 monclient: _add_conns ranks=[0,10,2,1,8,5,4,7,3,9,6] 2024-09-15T01:36:48.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2fe00a2e30 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:48.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2fe009fb80 0x7f2fe009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:48.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 10 monclient(hunting): picked mon.noname-f con 0x7f2fe009fb80 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:48.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fe0137d20 0x7f2fe013a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:48.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 10 monclient(hunting): picked mon.noname-b con 0x7f2fe0137d20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:48.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe004b720 con 0x7f2fe00a2e30 2024-09-15T01:36:48.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:48.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe009e6c0 con 0x7f2fe009fb80 2024-09-15T01:36:48.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:48.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00a04b0 con 0x7f2fe0137d20 2024-09-15T01:36:48.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:48.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe6e9f640 10 monclient(hunting): authenticate will time out at 1024.367180s 2024-09-15T01:36:48.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2fe009fb80 0x7f2fe009ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:48.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe4c3b640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f2fe00a2e30 legacy=0x7f2fe0132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:36:48.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2fe009fb80 0x7f2fe009ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:50630/0 (socket says 172.21.15.204:50630) 2024-09-15T01:36:48.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 1 -- 172.21.15.204:0/1347822124 learned_addr learned my addr 172.21.15.204:0/1347822124 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:48.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 10 monclient(hunting): get_auth_request con 0x7f2fe009fb80 auth_method 0 2024-09-15T01:36:48.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:48.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:48.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:48.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe543c640 1 -- 172.21.15.204:0/1347822124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fe0137d20 msgr2=0x7f2fe013a100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:48.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fe543c640 1 --2- 172.21.15.204:0/1347822124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fe0137d20 0x7f2fe013a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:48.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:48.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:48.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:48.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_done global_id 6805 payload 274 2024-09-15T01:36:48.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 1 -- 172.21.15.204:0/1347822124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fe0137d20 msgr2=0x7f2fe013a100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:48.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 1 --2- 172.21.15.204:0/1347822124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fe0137d20 0x7f2fe013a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 1 -- 172.21.15.204:0/1347822124 >> v1:172.21.15.178:6789/0 conn(0x7f2fe00a2e30 legacy=0x7f2fe0132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:48.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:48.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 1 monclient: found mon.noname-f 2024-09-15T01:36:48.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:48.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:48.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.724+0000 7f2fd7fff640 1 -- 172.21.15.204:0/1347822124 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe0133170 con 0x7f2fe009fb80 2024-09-15T01:36:48.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd7fff640 1 --2- 172.21.15.204:0/1347822124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe009fb80 0x7f2fe009ff70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009910 tx=0x7f2fd00091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc6a68cbea657d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:48.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd77fe640 1 -- 172.21.15.204:0/1347822124 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fd002acc0 con 0x7f2fe009fb80 2024-09-15T01:36:48.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:48.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd77fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:48.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd77fe640 10 monclient: dump: 2024-09-15T01:36:48.740 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:48.741 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:48.741 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.741 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.741 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:48.741 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:48.741 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:48.741 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:48.741 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:48.741 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:48.741 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:48.741 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:48.742 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:48.742 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:48.742 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:48.742 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:48.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd77fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:48.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:48.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd77fe640 1 -- 172.21.15.204:0/1347822124 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd002ae40 con 0x7f2fe009fb80 2024-09-15T01:36:48.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fe6e9f640 5 monclient: authenticate success, global_id 6805 2024-09-15T01:36:48.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fe6e9f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:48.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:48.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fe6e9f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:48.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd77fe640 1 -- 172.21.15.204:0/1347822124 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fd0031ca0 con 0x7f2fe009fb80 2024-09-15T01:36:48.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:48.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fe6e9f640 4 set_mon_vals no callback set 2024-09-15T01:36:48.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:48.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fd77fe640 10 monclient: dump: 2024-09-15T01:36:48.743 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:48.743 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:48.743 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.743 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.743 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:48.743 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:48.743 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:48.743 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:48.744 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:48.744 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:48.744 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:48.744 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:48.744 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:48.744 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:48.744 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:48.744 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:48.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fe6e9f640 10 monclient: shutdown 2024-09-15T01:36:48.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fe6e9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:48.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/1347822124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe009fb80 msgr2=0x7f2fe009ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:48.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fe6e9f640 1 --2- 172.21.15.204:0/1347822124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe009fb80 0x7f2fe009ff70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009910 tx=0x7f2fd00091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/1347822124 shutdown_connections 2024-09-15T01:36:48.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fe6e9f640 1 --2- 172.21.15.204:0/1347822124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fe0137d20 0x7f2fe013a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fe6e9f640 1 --2- 172.21.15.204:0/1347822124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe009fb80 0x7f2fe009ff70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.725+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/1347822124 >> 172.21.15.204:0/1347822124 conn(0x7f2fe006e860 msgr2=0x7f2fe006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:48.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.726+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/1347822124 shutdown_connections 2024-09-15T01:36:48.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.726+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/1347822124 wait complete. 2024-09-15T01:36:48.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.726+0000 7f2fe6e9f640 10 monclient: build_initial_monmap 2024-09-15T01:36:48.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.726+0000 7f2fe6e9f640 10 monclient: monmap: 2024-09-15T01:36:48.745 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:48.745 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:48.746 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:48.746 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:48.746 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:48.746 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:48.746 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:48.746 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:48.746 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:48.746 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:48.746 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:48.746 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:48.746 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:48.746 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:48.747 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:48.747 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:48.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.726+0000 7f2fe6e9f640 1 Processor -- start 2024-09-15T01:36:48.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 1 -- start start 2024-09-15T01:36:48.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 10 monclient: init 2024-09-15T01:36:48.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:48.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 10 monclient: _add_conns ranks=[6,3,7,8,2,0,5,1,4] 2024-09-15T01:36:48.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2fe009fb80 0x7f2fe00991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:48.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 10 monclient(hunting): picked mon.noname-d con 0x7f2fe009fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:48.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fe00a2e30 0x7f2fe0099710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:48.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 10 monclient(hunting): picked mon.noname-g con 0x7f2fe00a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:48.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2fe0137d20 0x7f2fe0099c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:48.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 10 monclient(hunting): picked mon.noname-i con 0x7f2fe0137d20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:48.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:48.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe009d650 con 0x7f2fe00a2e30 2024-09-15T01:36:48.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:48.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe009ad50 con 0x7f2fe009fb80 2024-09-15T01:36:48.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:48.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe011ca30 con 0x7f2fe0137d20 2024-09-15T01:36:48.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:48.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe6e9f640 10 monclient(hunting): authenticate will time out at 1024.370435s 2024-09-15T01:36:48.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fd7fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fe00a2e30 0x7f2fe0099710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:48.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2fe0137d20 0x7f2fe0099c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:48.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fd7fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fe00a2e30 0x7f2fe0099710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:43970/0 (socket says 172.21.15.204:43970) 2024-09-15T01:36:48.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fd7fff640 1 -- 172.21.15.204:0/4150085593 learned_addr learned my addr 172.21.15.204:0/4150085593 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:48.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 10 monclient(hunting): get_auth_request con 0x7f2fe0137d20 auth_method 0 2024-09-15T01:36:48.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:48.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:48.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:48.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fd7fff640 10 monclient(hunting): get_auth_request con 0x7f2fe00a2e30 auth_method 0 2024-09-15T01:36:48.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:48.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fd7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:48.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:48.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:48.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:48.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:48.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:48.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:48.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fd7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:48.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 10 monclient(hunting): handle_auth_done global_id 24325 payload 824 2024-09-15T01:36:48.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 1 -- 172.21.15.204:0/4150085593 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2fe009fb80 msgr2=0x7f2fe00991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:48.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe4c3b640 1 --2- 172.21.15.204:0/4150085593 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2fe009fb80 0x7f2fe00991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:48.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 1 --2- 172.21.15.204:0/4150085593 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2fe009fb80 0x7f2fe00991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 1 -- 172.21.15.204:0/4150085593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fe00a2e30 msgr2=0x7f2fe0099710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:48.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 1 --2- 172.21.15.204:0/4150085593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fe00a2e30 0x7f2fe0099710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:48.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 1 monclient: found mon.noname-i 2024-09-15T01:36:48.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:48.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:36:48.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fe543c640 1 -- 172.21.15.204:0/4150085593 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd0009450 con 0x7f2fe0137d20 2024-09-15T01:36:48.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.727+0000 7f2fd7fff640 1 --2- 172.21.15.204:0/4150085593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fe00a2e30 0x7f2fe0099710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:48.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fe543c640 1 --2- 172.21.15.204:0/4150085593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2fe0137d20 0x7f2fe0099c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800bdf0 tx=0x7f2fd80096a0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:48.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd4ff9640 1 -- 172.21.15.204:0/4150085593 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fd80067e0 con 0x7f2fe0137d20 2024-09-15T01:36:48.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:48.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd4ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:36:48.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd4ff9640 10 monclient: dump: 2024-09-15T01:36:48.751 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:48.751 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:48.752 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.752 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.752 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:48.752 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:48.752 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:48.752 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:48.752 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:48.752 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:48.752 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:48.752 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:48.752 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:48.753 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:48.753 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:48.753 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:48.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd4ff9640 10 monclient: _finish_auth 0 2024-09-15T01:36:48.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:48.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd4ff9640 1 -- 172.21.15.204:0/4150085593 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd800f070 con 0x7f2fe0137d20 2024-09-15T01:36:48.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fe6e9f640 5 monclient: authenticate success, global_id 24325 2024-09-15T01:36:48.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:48.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fe6e9f640 10 monclient: _renew_subs 2024-09-15T01:36:48.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd4ff9640 1 -- 172.21.15.204:0/4150085593 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fd80069e0 con 0x7f2fe0137d20 2024-09-15T01:36:48.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fe6e9f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:48.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/4150085593 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe009c970 con 0x7f2fe0137d20 2024-09-15T01:36:48.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:48.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:48.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd4ff9640 10 monclient: dump: 2024-09-15T01:36:48.754 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:48.754 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:48.754 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.754 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.754 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:48.754 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:48.754 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:48.754 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:48.755 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:48.755 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:48.755 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:48.755 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:48.755 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:48.755 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:48.755 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:48.755 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:48.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fd6ffd640 4 set_mon_vals no callback set 2024-09-15T01:36:48.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fe6e9f640 10 monclient: _renew_subs 2024-09-15T01:36:48.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fe6e9f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:48.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.728+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/4150085593 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe009b890 con 0x7f2fe0137d20 2024-09-15T01:36:48.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.729+0000 7f2fd4ff9640 1 -- 172.21.15.204:0/4150085593 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2fd8006360 con 0x7f2fe0137d20 2024-09-15T01:36:48.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.729+0000 7f2fe6e9f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:48.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.729+0000 7f2fe6e9f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:48.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.729+0000 7f2fe6e9f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:48.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.729+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/4150085593 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe009bb10 con 0x7f2fe0137d20 2024-09-15T01:36:48.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.729+0000 7f2fd4ff9640 1 --2- 172.21.15.204:0/4150085593 >> v2:172.21.15.204:6812/28210 conn(0x7f2fcc0397b0 0x7f2fcc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:48.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.729+0000 7f2fd4ff9640 1 -- 172.21.15.204:0/4150085593 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f2fd804c860 con 0x7f2fe0137d20 2024-09-15T01:36:48.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.729+0000 7f2fe4c3b640 1 --2- 172.21.15.204:0/4150085593 >> v2:172.21.15.204:6812/28210 conn(0x7f2fcc0397b0 0x7f2fcc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:48.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.729+0000 7f2fe4c3b640 10 monclient: get_auth_request con 0x7f2fcc0397b0 auth_method 0 2024-09-15T01:36:48.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.729+0000 7f2fe4c3b640 1 --2- 172.21.15.204:0/4150085593 >> v2:172.21.15.204:6812/28210 conn(0x7f2fcc0397b0 0x7f2fcc03bc60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0038f0 tx=0x7f2fdc008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:48.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.734+0000 7f2fd4ff9640 1 -- 172.21.15.204:0/4150085593 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2fd8012b60 con 0x7f2fe0137d20 2024-09-15T01:36:48.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.734+0000 7f2fd4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:48.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.734+0000 7f2fd4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:48.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 10 monclient(hunting): tick 2024-09-15T01:36:48.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 monclient(hunting): continuing hunt 2024-09-15T01:36:48.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:36:48.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe6c40a2930 msgr2=0x7fe6c411c530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:48.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe6c40a2930 0x7fe6c411c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6c40a08c0 msgr2=0x7fe6c4131840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:48.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6c40a08c0 0x7fe6c4131840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6c409faa0 msgr2=0x7fe6c409b2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6c409faa0 0x7fe6c409b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 10 monclient: _add_conns ranks=[8,7,6,5,3,1] 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe6a80037b0 0x7fe6a8005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 10 monclient(hunting): picked mon.noname-e con 0x7fe6a80037b0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6a80061d0 0x7fe6a8008630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 10 monclient(hunting): picked mon.noname-i con 0x7fe6a80061d0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6a8008cd0 0x7fe6a800b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 10 monclient(hunting): picked mon.noname-d con 0x7fe6a8008cd0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c411ec40 con 0x7fe6a8008cd0 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c409d630 con 0x7fe6a80061d0 2024-09-15T01:36:48.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:48.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c409bc20 con 0x7fe6a80037b0 2024-09-15T01:36:48.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6baffd640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:48.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c3fff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe6a80037b0 msgr2=0x7fe6a8005bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:48.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe6a80037b0 0x7fe6a8005bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:48.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c882d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6a8008cd0 0x7fe6a800b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:48.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c882d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6a8008cd0 0x7fe6a800b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:50034/0 (socket says 172.21.15.178:50034) 2024-09-15T01:36:48.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c882d640 1 -- 172.21.15.178:0/893908492 learned_addr learned my addr 172.21.15.178:0/893908492 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:48.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c882d640 10 monclient(hunting): get_auth_request con 0x7fe6a8008cd0 auth_method 0 2024-09-15T01:36:48.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c882d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c882d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c882d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6bbfff640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6a80061d0 0x7fe6a8008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6bbfff640 10 monclient(hunting): get_auth_request con 0x7fe6a80061d0 auth_method 0 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6bbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c882d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c882d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c882d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:48.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.870+0000 7fe6c882d640 10 monclient(hunting): handle_auth_done global_id 24333 payload 824 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c882d640 1 -- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe6a80037b0 msgr2=0x7fe6a8005bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c882d640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe6a80037b0 0x7fe6a8005bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c882d640 1 -- 172.21.15.178:0/893908492 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6a80061d0 msgr2=0x7fe6a8008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c882d640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6a80061d0 0x7fe6a8008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c882d640 10 monclient: _finish_hunting 0 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c882d640 1 monclient: found mon.noname-d 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c882d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c882d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c882d640 1 -- 172.21.15.178:0/893908492 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b4009450 con 0x7fe6a8008cd0 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6bbfff640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe6a80061d0 0x7fe6a8008630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c882d640 1 -- 172.21.15.178:0/893908492 reap_dead start 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c882d640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6a8008cd0 0x7fe6a800b180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009f00 tx=0x7fe6bc00b040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:48.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6bc00ce10 con 0x7fe6a8008cd0 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c0ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c0ff9640 10 monclient: dump: 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:48.882 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c0ff9640 10 monclient: _finish_auth 0 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6ca290640 5 monclient: authenticate success, global_id 24333 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc00f040 con 0x7fe6a8008cd0 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6bc014d80 con 0x7fe6a8008cd0 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6ca290640 10 monclient: _renew_subs 2024-09-15T01:36:48.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6ca290640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6ca290640 1 -- 172.21.15.178:0/893908492 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c411e960 con 0x7fe6a8008cd0 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c0ff9640 10 monclient: dump: 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:48.884 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6c2ffd640 4 set_mon_vals no callback set 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6ca290640 10 monclient: _renew_subs 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6ca290640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.871+0000 7fe6ca290640 1 -- 172.21.15.178:0/893908492 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c411dba0 con 0x7fe6a8008cd0 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.872+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe6bc003020 con 0x7fe6a8008cd0 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.872+0000 7fe6ca290640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:48.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.872+0000 7fe6ca290640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4137020 0x7fe6c409b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.872+0000 7fe6ca290640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.872+0000 7fe6ca290640 1 -- 172.21.15.178:0/893908492 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c409f940 con 0x7fe6c4137020 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.872+0000 7fe6c0ff9640 1 --2- 172.21.15.178:0/893908492 >> v2:172.21.15.204:6812/28210 conn(0x7fe6a4039720 0x7fe6a403bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.872+0000 7fe6c3fff640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4137020 0x7fe6c409b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.872+0000 7fe6c3fff640 10 monclient: get_auth_request con 0x7fe6c4137020 auth_method 0 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.872+0000 7fe6c3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.872+0000 7fe6c3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.872+0000 7fe6c3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.872+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fe6bc04bfc0 con 0x7fe6a8008cd0 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6bbfff640 1 --2- 172.21.15.178:0/893908492 >> v2:172.21.15.204:6812/28210 conn(0x7fe6a4039720 0x7fe6a403bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:48.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6bbfff640 10 monclient: get_auth_request con 0x7fe6a4039720 auth_method 0 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c3fff640 10 monclient(hunting): handle_auth_done global_id 24342 payload 824 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c3fff640 1 -- 172.21.15.178:0/893908492 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe6c404c4a0 con 0x7fe6c4137020 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c3fff640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4137020 0x7fe6c409b2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009910 tx=0x7fe6b4032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6bbfff640 1 --2- 172.21.15.178:0/893908492 >> v2:172.21.15.204:6812/28210 conn(0x7fe6a4039720 0x7fe6a403bbd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac001640 tx=0x7fe6ac008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6b4031ea0 con 0x7fe6c4137020 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c0ff9640 10 monclient: dump: 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.887 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fe6b402ac60 con 0x7fe6c4137020 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:48.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:48.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4137020 msgr2=0x7fe6c409b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:48.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c0ff9640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4137020 0x7fe6c409b2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009910 tx=0x7fe6b4032000 comp rx=0 tx=0).stop 2024-09-15T01:36:48.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4137020 msgr2=0x7fe6c409b2c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:48.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.873+0000 7fe6c0ff9640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4137020 0x7fe6c409b2c0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009910 tx=0x7fe6b4032000 comp rx=0 tx=0).stop 2024-09-15T01:36:48.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6ca290640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:48.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6ca290640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4131840 0x7fe6c4131c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:48.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6ca290640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:48.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6ca290640 1 -- 172.21.15.178:0/893908492 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c404bc80 con 0x7fe6c4131840 2024-09-15T01:36:48.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6c3fff640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4131840 0x7fe6c4131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:48.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6c3fff640 10 monclient: get_auth_request con 0x7fe6c4131840 auth_method 0 2024-09-15T01:36:48.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6c3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:48.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6c3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:48.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6c3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:48.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6c3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:48.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6c3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:48.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6c3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6c3fff640 10 monclient(hunting): handle_auth_done global_id 24351 payload 824 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6c3fff640 1 -- 172.21.15.178:0/893908492 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe6c405ed70 con 0x7fe6c4131840 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.880+0000 7fe6c3fff640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4131840 0x7fe6c4131c40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009940 tx=0x7fe6b403b000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6b403cd20 con 0x7fe6c4131840 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6c0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6c0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6c0ff9640 10 monclient: dump: 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:48.892 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9097 (secure 0 0 0) 0x7fe6b4032760 con 0x7fe6c4131840 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6c0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6c0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:48.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4131840 msgr2=0x7fe6c4131c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:48.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6c0ff9640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4131840 0x7fe6c4131c40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009940 tx=0x7fe6b403b000 comp rx=0 tx=0).stop 2024-09-15T01:36:48.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6c0ff9640 1 -- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4131840 msgr2=0x7fe6c4131c40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:48.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6c0ff9640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4131840 0x7fe6c4131c40 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009940 tx=0x7fe6b403b000 comp rx=0 tx=0).stop 2024-09-15T01:36:48.894 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:48.894 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:48.894 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:48.894 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:48.894 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 64, 2024-09-15T01:36:48.894 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: 7 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:48.895 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:48.896 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:48.897 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:48.897 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:48.897 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:48.897 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:48.897 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:48.897 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:48.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.897 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:48.897 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:48.897 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:48.897 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:48.897 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:48.900 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:48.900 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:48.901 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:48.902 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.903 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:48.904 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:48.905 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.906 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:48.907 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:48.908 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:48.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:48.910 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.912 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.913 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:48.915 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:48.916 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:48.917 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:48.918 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:48.919 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:36:48.919 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:48.919 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:48.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:48.919 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:48.919 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:48.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.897+0000 7f2fe6e9f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:36:48.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.897+0000 7f2fe6e9f640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:36:48.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.897+0000 7f2fe6e9f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:48.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:48.897+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/4150085593 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7f2fe0070cc0 con 0x7f2fe0137d20 2024-09-15T01:36:48.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6ca290640 1 -- 172.21.15.178:0/893908492 >> v2:172.21.15.204:6812/28210 conn(0x7fe6a4039720 msgr2=0x7fe6a403bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:48.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6ca290640 1 --2- 172.21.15.178:0/893908492 >> v2:172.21.15.204:6812/28210 conn(0x7fe6a4039720 0x7fe6a403bbd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac001640 tx=0x7fe6ac008040 comp rx=0 tx=0).stop 2024-09-15T01:36:48.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6ca290640 10 monclient: shutdown 2024-09-15T01:36:48.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6ca290640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:48.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6ca290640 1 -- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6a8008cd0 msgr2=0x7fe6a800b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:48.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6ca290640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6a8008cd0 0x7fe6a800b180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009f00 tx=0x7fe6bc00b040 comp rx=0 tx=0).stop 2024-09-15T01:36:48.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6ca290640 1 -- 172.21.15.178:0/893908492 shutdown_connections 2024-09-15T01:36:48.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6ca290640 1 --2- 172.21.15.178:0/893908492 >> v2:172.21.15.204:6812/28210 conn(0x7fe6a4039720 0x7fe6a403bbd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6ca290640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6a8008cd0 0x7fe6a800b180 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6ca290640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4131840 0x7fe6c4131c40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6ca290640 1 --2- 172.21.15.178:0/893908492 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe6c4137020 0x7fe6c409b2c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:48.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.881+0000 7fe6ca290640 1 -- 172.21.15.178:0/893908492 >> 172.21.15.178:0/893908492 conn(0x7fe6c40a5ed0 msgr2=0x7fe6c4135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:48.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.882+0000 7fe6ca290640 1 -- 172.21.15.178:0/893908492 shutdown_connections 2024-09-15T01:36:48.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:48.882+0000 7fe6ca290640 1 -- 172.21.15.178:0/893908492 wait complete. 2024-09-15T01:36:48.923 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T01:36:48.923 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon f 2024-09-15T01:36:48.923 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.024+0000 7f578fe80640 10 monclient: get_monmap_and_config 2024-09-15T01:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.024+0000 7f578fe80640 10 monclient: build_initial_monmap 2024-09-15T01:36:49.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.024+0000 7f578fe80640 10 monclient: monmap: 2024-09-15T01:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:49.025682+0000 2024-09-15T01:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:49.025682+0000 2024-09-15T01:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.025+0000 7f578fe80640 1 Processor -- start 2024-09-15T01:36:49.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.025+0000 7f578fe80640 1 -- start start 2024-09-15T01:36:49.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.025+0000 7f578fe80640 10 monclient: init 2024-09-15T01:36:49.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:49.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 10 monclient: _add_conns ranks=[5,7,6,10,4,8,9,1,3,0,2] 2024-09-15T01:36:49.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f57880a2930 0x7f5788132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 10 monclient(hunting): picked mon.noname-h con 0x7f57880a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:49.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f578809faa0 0x7f578809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 10 monclient(hunting): picked mon.noname-i con 0x7f578809faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:49.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57880a08c0 0x7f57880a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 10 monclient(hunting): picked mon.noname-d con 0x7f57880a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:49.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f578809e0f0 con 0x7f57880a2930 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f578809dd80 con 0x7f57880a08c0 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f57880a2d00 con 0x7f578809faa0 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578fe80640 10 monclient(hunting): authenticate will time out at 1013.776198s 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57880a08c0 0x7f57880a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57880a08c0 0x7f57880a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:50064/0 (socket says 172.21.15.178:50064) 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 1 -- 172.21.15.178:0/2231988428 learned_addr learned my addr 172.21.15.178:0/2231988428 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 10 monclient(hunting): get_auth_request con 0x7f57880a08c0 auth_method 0 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:49.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:49.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:49.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578dc1c640 1 --2- 172.21.15.178:0/2231988428 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f57880a2930 0x7f5788132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578d41b640 1 --2- 172.21.15.178:0/2231988428 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f578809faa0 0x7f578809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578dc1c640 10 monclient(hunting): get_auth_request con 0x7f57880a2930 auth_method 0 2024-09-15T01:36:49.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578dc1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578dc1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578dc1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578d41b640 10 monclient(hunting): get_auth_request con 0x7f578809faa0 auth_method 0 2024-09-15T01:36:49.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578d41b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578d41b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578d41b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 10 monclient(hunting): handle_auth_done global_id 24360 payload 274 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 1 -- 172.21.15.178:0/2231988428 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f578809faa0 msgr2=0x7f578809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 1 --2- 172.21.15.178:0/2231988428 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f578809faa0 0x7f578809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 1 -- 172.21.15.178:0/2231988428 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f57880a2930 msgr2=0x7f5788132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 1 --2- 172.21.15.178:0/2231988428 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f57880a2930 0x7f5788132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 10 monclient: _finish_hunting 0 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 1 monclient: found mon.noname-d 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.026+0000 7f578e41d640 1 -- 172.21.15.178:0/2231988428 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578804bf60 con 0x7f57880a08c0 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578e41d640 1 --2- 172.21.15.178:0/2231988428 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57880a08c0 0x7f57880a0d30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5780009f50 tx=0x7f5780009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=196e5e275415c9a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:49.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578cc1a640 1 -- 172.21.15.178:0/2231988428 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5780007880 con 0x7f57880a08c0 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578cc1a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578cc1a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578cc1a640 10 monclient: dump: 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.047 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.048 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.048 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.048 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.048 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.048 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578cc1a640 10 monclient: _finish_auth 0 2024-09-15T01:36:49.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578cc1a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:49.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578cc1a640 1 -- 172.21.15.178:0/2231988428 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578002d050 con 0x7f57880a08c0 2024-09-15T01:36:49.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578fe80640 5 monclient: authenticate success, global_id 24360 2024-09-15T01:36:49.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578fe80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:49.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578cc1a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:49.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578fe80640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578cc1a640 1 -- 172.21.15.178:0/2231988428 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5780030360 con 0x7f57880a08c0 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578cc1a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578fe80640 4 set_mon_vals no callback set 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578cc1a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.027+0000 7f578cc1a640 10 monclient: dump: 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.049 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 10 monclient: shutdown 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 1 -- 172.21.15.178:0/2231988428 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57880a08c0 msgr2=0x7f57880a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 1 --2- 172.21.15.178:0/2231988428 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57880a08c0 0x7f57880a0d30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5780009f50 tx=0x7f5780009840 comp rx=0 tx=0).stop 2024-09-15T01:36:49.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 1 -- 172.21.15.178:0/2231988428 shutdown_connections 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 1 --2- 172.21.15.178:0/2231988428 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f57880a08c0 0x7f57880a0d30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 1 --2- 172.21.15.178:0/2231988428 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f578809faa0 0x7f578809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 1 --2- 172.21.15.178:0/2231988428 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f57880a2930 0x7f5788132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 1 -- 172.21.15.178:0/2231988428 >> 172.21.15.178:0/2231988428 conn(0x7f57880a5ed0 msgr2=0x7f57880a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 1 -- 172.21.15.178:0/2231988428 shutdown_connections 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 1 -- 172.21.15.178:0/2231988428 wait complete. 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 10 monclient: build_initial_monmap 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.028+0000 7f578fe80640 10 monclient: monmap: 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:49.051 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:49.052 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:49.052 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:49.052 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:49.052 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:49.052 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:49.052 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:49.052 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:49.052 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:49.052 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:49.052 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:49.053 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 1 Processor -- start 2024-09-15T01:36:49.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 1 -- start start 2024-09-15T01:36:49.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 10 monclient: init 2024-09-15T01:36:49.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:49.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 10 monclient: _add_conns ranks=[6,8,3,1,5,4,2,7,0] 2024-09-15T01:36:49.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f578809faa0 0x7f578809b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 10 monclient(hunting): picked mon.noname-d con 0x7f578809faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:49.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f57880a08c0 0x7f5788131b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 10 monclient(hunting): picked mon.noname-e con 0x7f57880a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:49.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57880a2930 0x7f57881320a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 10 monclient(hunting): picked mon.noname-g con 0x7f57880a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:49.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578809c100 con 0x7f57880a2930 2024-09-15T01:36:49.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f57880a2390 con 0x7f578809faa0 2024-09-15T01:36:49.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f578809d630 con 0x7f57880a08c0 2024-09-15T01:36:49.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:49.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578fe80640 10 monclient(hunting): authenticate will time out at 1013.779626s 2024-09-15T01:36:49.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578d41b640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f57880a08c0 msgr2=0x7f5788131b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:49.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578d41b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f57880a08c0 0x7f5788131b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:49.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f578809faa0 0x7f578809b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f578809faa0 0x7f578809b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:50080/0 (socket says 172.21.15.178:50080) 2024-09-15T01:36:49.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 1 -- 172.21.15.178:0/3642242251 learned_addr learned my addr 172.21.15.178:0/3642242251 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:49.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 10 monclient(hunting): get_auth_request con 0x7f578809faa0 auth_method 0 2024-09-15T01:36:49.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:49.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:49.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:49.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578e41d640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57880a2930 0x7f57881320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578e41d640 10 monclient(hunting): get_auth_request con 0x7f57880a2930 auth_method 0 2024-09-15T01:36:49.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578e41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578e41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578e41d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 10 monclient(hunting): handle_auth_done global_id 24369 payload 824 2024-09-15T01:36:49.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 1 -- 172.21.15.178:0/3642242251 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f57880a08c0 msgr2=0x7f5788131b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:49.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.029+0000 7f578dc1c640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f57880a08c0 0x7f5788131b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578dc1c640 1 -- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57880a2930 msgr2=0x7f57881320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578dc1c640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57880a2930 0x7f57881320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578dc1c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:49.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578dc1c640 1 monclient: found mon.noname-d 2024-09-15T01:36:49.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578dc1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578dc1c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578dc1c640 1 -- 172.21.15.178:0/3642242251 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5780009ae0 con 0x7f578809faa0 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578e41d640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57880a2930 0x7f57881320a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578dc1c640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f578809faa0 0x7f578809b670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f578400d8f0 tx=0x7f5784008040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f578400de80 con 0x7f578809faa0 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577dffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577dffb640 10 monclient: dump: 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.058 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.059 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.059 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.059 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.059 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.059 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.059 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.059 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.059 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.059 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.060 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.060 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577dffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:49.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:49.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578fe80640 5 monclient: authenticate success, global_id 24369 2024-09-15T01:36:49.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578400bd50 con 0x7f578809faa0 2024-09-15T01:36:49.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:49.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578fe80640 10 monclient: _renew_subs 2024-09-15T01:36:49.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f578400bec0 con 0x7f578809faa0 2024-09-15T01:36:49.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578fe80640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:49.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578fe80640 1 -- 172.21.15.178:0/3642242251 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5788142f80 con 0x7f578809faa0 2024-09-15T01:36:49.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577ffff640 4 set_mon_vals no callback set 2024-09-15T01:36:49.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:49.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f577dffb640 10 monclient: dump: 2024-09-15T01:36:49.061 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.061 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.061 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.061 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.061 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.062 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.062 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.062 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.062 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.062 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.062 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.062 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.062 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.062 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.062 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.063 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578fe80640 10 monclient: _renew_subs 2024-09-15T01:36:49.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578fe80640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:49.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.030+0000 7f578fe80640 1 -- 172.21.15.178:0/3642242251 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578811ef60 con 0x7f578809faa0 2024-09-15T01:36:49.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f578400b8d0 con 0x7f578809faa0 2024-09-15T01:36:49.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578fe80640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:49.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f577dffb640 1 --2- 172.21.15.178:0/3642242251 >> v2:172.21.15.204:6812/28210 conn(0x7f576c0397e0 0x7f576c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578fe80640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5788142520 0x7f578811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578fe80640 1 -- 172.21.15.178:0/3642242251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578811ee20 con 0x7f5788142520 2024-09-15T01:36:49.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f578404c160 con 0x7f578809faa0 2024-09-15T01:36:49.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578e41d640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5788142520 0x7f578811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578d41b640 1 --2- 172.21.15.178:0/3642242251 >> v2:172.21.15.204:6812/28210 conn(0x7f576c0397e0 0x7f576c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578e41d640 10 monclient: get_auth_request con 0x7f5788142520 auth_method 0 2024-09-15T01:36:49.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578e41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578e41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578e41d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578d41b640 10 monclient: get_auth_request con 0x7f576c0397e0 auth_method 0 2024-09-15T01:36:49.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578e41d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:49.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578e41d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:49.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578e41d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:49.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578e41d640 10 monclient(hunting): handle_auth_done global_id 6814 payload 824 2024-09-15T01:36:49.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.031+0000 7f578e41d640 1 -- 172.21.15.178:0/3642242251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f578804c4a0 con 0x7f5788142520 2024-09-15T01:36:49.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f578d41b640 1 --2- 172.21.15.178:0/3642242251 >> v2:172.21.15.204:6812/28210 conn(0x7f576c0397e0 0x7f576c03bc90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f578809f870 tx=0x7f5778008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:49.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f578e41d640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5788142520 0x7f578811c530 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5780009f20 tx=0x7f5780036000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:49.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f578003ca80 con 0x7f5788142520 2024-09-15T01:36:49.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f577dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f577dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:49.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f577dffb640 10 monclient: dump: 2024-09-15T01:36:49.065 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.066 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f578003cbe0 con 0x7f5788142520 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f577dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f577dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5788142520 msgr2=0x7f578811c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f577dffb640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5788142520 0x7f578811c530 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5780009f20 tx=0x7f5780036000 comp rx=0 tx=0).stop 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5788142520 msgr2=0x7f578811c530 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.032+0000 7f577dffb640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5788142520 0x7f578811c530 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578fe80640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578fe80640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f578811ca70 0x7f578811ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578fe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578fe80640 1 -- 172.21.15.178:0/3642242251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578811f0f0 con 0x7f578811ca70 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578e41d640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f578811ca70 0x7f578811ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578e41d640 10 monclient: get_auth_request con 0x7f578811ca70 auth_method 0 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578e41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578e41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578e41d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578e41d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578e41d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578e41d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578e41d640 10 monclient(hunting): handle_auth_done global_id 6823 payload 824 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578e41d640 1 -- 172.21.15.178:0/3642242251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f578804bc80 con 0x7f578811ca70 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.038+0000 7f578e41d640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f578811ca70 0x7f578811ce70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5780009f50 tx=0x7f578003d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f578004b070 con 0x7f578811ca70 2024-09-15T01:36:49.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f577dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f577dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f577dffb640 10 monclient: dump: 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.069 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.070 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.070 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.070 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.070 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.070 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9439 (secure 0 0 0) 0x7f5780044400 con 0x7f578811ca70 2024-09-15T01:36:49.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f577dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:49.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f577dffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:49.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f578811ca70 msgr2=0x7f578811ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f577dffb640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f578811ca70 0x7f578811ce70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5780009f50 tx=0x7f578003d000 comp rx=0 tx=0).stop 2024-09-15T01:36:49.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f577dffb640 1 -- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f578811ca70 msgr2=0x7f578811ce70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:49.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f577dffb640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f578811ca70 0x7f578811ce70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.071 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:49.071 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:49.071 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:49.071 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:36:49.071 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 64, 2024-09-15T01:36:49.071 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:49.071 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:49.071 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:49.071 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:49.071 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: 7 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:49.072 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:49.073 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:49.073 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:49.073 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:49.073 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:49.073 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:49.073 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:49.073 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:49.073 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:49.073 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:49.073 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:49.073 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:49.073 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:49.074 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:49.074 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:49.074 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.074 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:49.074 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:49.074 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:49.074 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:49.074 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:49.074 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:49.074 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:49.074 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:49.075 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:49.076 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:49.076 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:49.076 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:49.076 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:49.076 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:49.076 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:49.076 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:49.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.076 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:49.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.076 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.077 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.078 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:49.080 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:49.080 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.080 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.080 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.080 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:49.080 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:49.080 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.080 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.080 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:49.081 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:49.085 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.085 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.085 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.085 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:49.085 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:49.085 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.085 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.085 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.086 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:49.086 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:49.086 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.086 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.086 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.086 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:49.086 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.088 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:49.088 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:49.088 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.088 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:49.088 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.090 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:49.090 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:49.090 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.090 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:49.090 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:49.091 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.091 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.091 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:49.091 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:49.091 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.091 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.091 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.092 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:49.092 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:49.092 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.092 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:49.093 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.093 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:49.093 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.093 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.093 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:49.093 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:49.094 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.094 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.094 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.094 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.094 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.094 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:49.094 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:49.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.094 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:49.095 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:49.095 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:49.095 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.095 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:49.095 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:36:49.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.095 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:49.095 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:49.095 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:36:49.095 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.096 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:49.096 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:49.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.096 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:49.096 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:49.096 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:49.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.096 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:49.096 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:36:49.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.097 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:49.097 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:49.097 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:49.097 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.097 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.097 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:49.097 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:49.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f578fe80640 1 -- 172.21.15.178:0/3642242251 >> v2:172.21.15.204:6812/28210 conn(0x7f576c0397e0 msgr2=0x7f576c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f578fe80640 1 --2- 172.21.15.178:0/3642242251 >> v2:172.21.15.204:6812/28210 conn(0x7f576c0397e0 0x7f576c03bc90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f578809f870 tx=0x7f5778008040 comp rx=0 tx=0).stop 2024-09-15T01:36:49.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f578fe80640 10 monclient: shutdown 2024-09-15T01:36:49.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f578fe80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:49.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f578fe80640 1 -- 172.21.15.178:0/3642242251 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f578809faa0 msgr2=0x7f578809b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f578fe80640 1 --2- 172.21.15.178:0/3642242251 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f578809faa0 0x7f578809b670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f578400d8f0 tx=0x7f5784008040 comp rx=0 tx=0).stop 2024-09-15T01:36:49.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f578e41d640 1 -- 172.21.15.178:0/3642242251 reap_dead start 2024-09-15T01:36:49.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f578e41d640 1 -- 172.21.15.178:0/3642242251 reap_dead start 2024-09-15T01:36:49.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f578fe80640 1 -- 172.21.15.178:0/3642242251 shutdown_connections 2024-09-15T01:36:49.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f578fe80640 1 -- 172.21.15.178:0/3642242251 >> 172.21.15.178:0/3642242251 conn(0x7f57880a5ed0 msgr2=0x7f57880a53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:49.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f578fe80640 1 -- 172.21.15.178:0/3642242251 shutdown_connections 2024-09-15T01:36:49.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.039+0000 7f578fe80640 1 -- 172.21.15.178:0/3642242251 wait complete. 2024-09-15T01:36:49.100 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T01:36:49.100 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon g 2024-09-15T01:36:49.100 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:36:49.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.193+0000 7f0b90f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:49.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.193+0000 7f0b90f9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:49.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.193+0000 7f0b90f9d640 10 monclient: monmap: 2024-09-15T01:36:49.196 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:49.196 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.196 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:49.194182+0000 2024-09-15T01:36:49.196 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:49.194182+0000 2024-09-15T01:36:49.196 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:49.196 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.196 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:49.196 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:49.196 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:49.197 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:49.197 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:49.197 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:49.197 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:49.197 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:49.197 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:49.197 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:49.197 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:49.197 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.193+0000 7f0b90f9d640 1 Processor -- start 2024-09-15T01:36:49.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.193+0000 7f0b90f9d640 1 -- start start 2024-09-15T01:36:49.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.193+0000 7f0b90f9d640 10 monclient: init 2024-09-15T01:36:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 10 monclient: _add_conns ranks=[5,7,2,6,10,4,0,1,3,9,8] 2024-09-15T01:36:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b8c0a2930 0x7f0b8c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 10 monclient(hunting): picked mon.noname-h con 0x7f0b8c0a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b8c09faa0 0x7f0b8c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 10 monclient(hunting): picked mon.noname-i con 0x7f0b8c09faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:49.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b8c0a08c0 0x7f0b8c0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f0b8c0a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:49.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c09e0f0 con 0x7f0b8c0a08c0 2024-09-15T01:36:49.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c09dd80 con 0x7f0b8c0a2930 2024-09-15T01:36:49.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0a2d00 con 0x7f0b8c09faa0 2024-09-15T01:36:49.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:49.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b90f9d640 10 monclient(hunting): authenticate will time out at 1013.944587s 2024-09-15T01:36:49.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b8ad9d640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b8c0a08c0 msgr2=0x7f0b8c0a0d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:49.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b8ad9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b8c0a08c0 0x7f0b8c0a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:49.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b8a59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b8c0a2930 0x7f0b8c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b8a59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b8c0a2930 0x7f0b8c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:55114/0 (socket says 172.21.15.178:55114) 2024-09-15T01:36:49.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b8a59c640 1 -- 172.21.15.178:0/466112597 learned_addr learned my addr 172.21.15.178:0/466112597 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:49.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b8a59c640 10 monclient(hunting): get_auth_request con 0x7f0b8c0a2930 auth_method 0 2024-09-15T01:36:49.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b8a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b8a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b8a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.194+0000 7f0b89d9b640 1 --2- 172.21.15.178:0/466112597 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b8c09faa0 0x7f0b8c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b89d9b640 10 monclient(hunting): get_auth_request con 0x7f0b8c09faa0 auth_method 0 2024-09-15T01:36:49.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b89d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b89d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b89d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:49.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:49.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:49.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b89d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:49.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b89d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:49.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b89d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:49.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_done global_id 6116 payload 274 2024-09-15T01:36:49.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 1 -- 172.21.15.178:0/466112597 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b8c09faa0 msgr2=0x7f0b8c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:49.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 1 --2- 172.21.15.178:0/466112597 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b8c09faa0 0x7f0b8c09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 1 -- 172.21.15.178:0/466112597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b8c0a08c0 msgr2=0x7f0b8c0a0d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:49.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 1 --2- 172.21.15.178:0/466112597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b8c0a08c0 0x7f0b8c0a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:49.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 1 monclient: found mon.noname-h 2024-09-15T01:36:49.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:49.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:49.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 1 -- 172.21.15.178:0/466112597 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c04bf60 con 0x7f0b8c0a2930 2024-09-15T01:36:49.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b89d9b640 1 --2- 172.21.15.178:0/466112597 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b8c09faa0 0x7f0b8c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:49.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.195+0000 7f0b8a59c640 1 --2- 172.21.15.178:0/466112597 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b8c0a2930 0x7f0b8c132de0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009940 tx=0x7f0b7c0091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d4c26efeff03fd70 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:49.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b8959a640 1 -- 172.21.15.178:0/466112597 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b7c02ac80 con 0x7f0b8c0a2930 2024-09-15T01:36:49.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b8959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b8959a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:49.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b8959a640 10 monclient: dump: 2024-09-15T01:36:49.212 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.212 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.212 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.213 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.213 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.213 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.213 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.213 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.213 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.213 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.213 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.256 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.256 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.256 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.256 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.256 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b8959a640 10 monclient: _finish_auth 0 2024-09-15T01:36:49.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b8959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:49.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b8959a640 1 -- 172.21.15.178:0/466112597 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7c02ae00 con 0x7f0b8c0a2930 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 5 monclient: authenticate success, global_id 6116 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b8959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b8959a640 1 -- 172.21.15.178:0/466112597 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b7c031ca0 con 0x7f0b8c0a2930 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b8959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 4 set_mon_vals no callback set 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b8959a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b8959a640 10 monclient: dump: 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.257 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 10 monclient: shutdown 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 1 -- 172.21.15.178:0/466112597 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b8c0a2930 msgr2=0x7f0b8c132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 1 --2- 172.21.15.178:0/466112597 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b8c0a2930 0x7f0b8c132de0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009940 tx=0x7f0b7c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 1 -- 172.21.15.178:0/466112597 shutdown_connections 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 1 --2- 172.21.15.178:0/466112597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b8c0a08c0 0x7f0b8c0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 1 --2- 172.21.15.178:0/466112597 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b8c09faa0 0x7f0b8c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 1 --2- 172.21.15.178:0/466112597 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0b8c0a2930 0x7f0b8c132de0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 1 -- 172.21.15.178:0/466112597 >> 172.21.15.178:0/466112597 conn(0x7f0b8c0a5ed0 msgr2=0x7f0b8c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 1 -- 172.21.15.178:0/466112597 shutdown_connections 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.196+0000 7f0b90f9d640 1 -- 172.21.15.178:0/466112597 wait complete. 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient: monmap: 2024-09-15T01:36:49.259 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:49.260 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 1 Processor -- start 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 1 -- start start 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient: init 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient: _add_conns ranks=[7,6,3,2,1,0,8,4,5] 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b8c09faa0 0x7f0b8c09b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient(hunting): picked mon.noname-i con 0x7f0b8c09faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b8c0a08c0 0x7f0b8c131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient(hunting): picked mon.noname-d con 0x7f0b8c0a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c0a2930 0x7f0b8c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient(hunting): picked mon.noname-g con 0x7f0b8c0a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c11e960 con 0x7f0b8c0a2930 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c11ef90 con 0x7f0b8c0a08c0 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c11d8a0 con 0x7f0b8c09faa0 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.197+0000 7f0b90f9d640 10 monclient(hunting): authenticate will time out at 1013.947983s 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b8c0a08c0 0x7f0b8c131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b8c0a08c0 0x7f0b8c131840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:50096/0 (socket says 172.21.15.178:50096) 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 1 -- 172.21.15.178:0/3511550659 learned_addr learned my addr 172.21.15.178:0/3511550659 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 10 monclient(hunting): get_auth_request con 0x7f0b8c0a08c0 auth_method 0 2024-09-15T01:36:49.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8a59c640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b8c09faa0 0x7f0b8c09b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8a59c640 10 monclient(hunting): get_auth_request con 0x7f0b8c09faa0 auth_method 0 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8ad9d640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c0a2930 0x7f0b8c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:49.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8ad9d640 10 monclient(hunting): get_auth_request con 0x7f0b8c0a2930 auth_method 0 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b8ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 10 monclient(hunting): handle_auth_done global_id 24378 payload 824 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 1 -- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b8c09faa0 msgr2=0x7f0b8c09b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b8c09faa0 0x7f0b8c09b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 1 -- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c0a2930 msgr2=0x7f0b8c11c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c0a2930 0x7f0b8c11c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 1 monclient: found mon.noname-d 2024-09-15T01:36:49.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 1 -- 172.21.15.178:0/3511550659 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b74009390 con 0x7f0b8c0a08c0 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b89d9b640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b8c0a08c0 0x7f0b8c131840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400ea80 tx=0x7f0b740090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b7400ce90 con 0x7f0b8c0a08c0 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b827fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b827fc640 10 monclient: dump: 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.265 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b827fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:49.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b74013470 con 0x7f0b8c0a08c0 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b90f9d640 5 monclient: authenticate success, global_id 24378 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b827fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b90f9d640 10 monclient: _renew_subs 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b74013a60 con 0x7f0b8c0a08c0 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b90f9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b90f9d640 1 -- 172.21.15.178:0/3511550659 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7c009450 con 0x7f0b8c0a08c0 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b827fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b827fc640 10 monclient: dump: 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.267 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b88d99640 4 set_mon_vals no callback set 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b90f9d640 10 monclient: _renew_subs 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b90f9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.198+0000 7f0b90f9d640 1 -- 172.21.15.178:0/3511550659 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8c09bc70 con 0x7f0b8c0a08c0 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b90f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b90f9d640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c131d80 0x7f0b8c11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b90f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b90f9d640 1 -- 172.21.15.178:0/3511550659 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0a2450 con 0x7f0b8c131d80 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0b74013bc0 con 0x7f0b8c0a08c0 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b8a59c640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c131d80 0x7f0b8c11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b8a59c640 10 monclient: get_auth_request con 0x7f0b8c131d80 auth_method 0 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b8a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b8a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b8a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b827fc640 1 --2- 172.21.15.178:0/3511550659 >> v2:172.21.15.204:6812/28210 conn(0x7f0b6c0397b0 0x7f0b6c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f0b7404bed0 con 0x7f0b8c0a08c0 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b8ad9d640 1 --2- 172.21.15.178:0/3511550659 >> v2:172.21.15.204:6812/28210 conn(0x7f0b6c0397b0 0x7f0b6c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_done global_id 15294 payload 824 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b8a59c640 1 -- 172.21.15.178:0/3511550659 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b8c04c4a0 con 0x7f0b8c131d80 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.199+0000 7f0b8ad9d640 10 monclient: get_auth_request con 0x7f0b6c0397b0 auth_method 0 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b8a59c640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c131d80 0x7f0b8c11ca70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009910 tx=0x7f0b7c032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b7c02ac80 con 0x7f0b8c131d80 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b827fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b827fc640 10 monclient: dump: 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.271 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.272 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.272 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.272 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b8ad9d640 1 --2- 172.21.15.178:0/3511550659 >> v2:172.21.15.204:6812/28210 conn(0x7f0b6c0397b0 0x7f0b6c03bc60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c11eb70 tx=0x7f0b840094f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:49.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f0b7c032e20 con 0x7f0b8c131d80 2024-09-15T01:36:49.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b827fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:49.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b827fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:49.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c131d80 msgr2=0x7f0b8c11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b827fc640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c131d80 0x7f0b8c11ca70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009910 tx=0x7f0b7c032000 comp rx=0 tx=0).stop 2024-09-15T01:36:49.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c131d80 msgr2=0x7f0b8c11ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:49.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.200+0000 7f0b827fc640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c131d80 0x7f0b8c11ca70 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009910 tx=0x7f0b7c032000 comp rx=0 tx=0).stop 2024-09-15T01:36:49.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b90f9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:49.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b90f9d640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c11cfb0 0x7f0b8c096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b90f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b90f9d640 1 -- 172.21.15.178:0/3511550659 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0a63a0 con 0x7f0b8c11cfb0 2024-09-15T01:36:49.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b8a59c640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c11cfb0 0x7f0b8c096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b8a59c640 10 monclient: get_auth_request con 0x7f0b8c11cfb0 auth_method 0 2024-09-15T01:36:49.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b8a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b8a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b8a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:49.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:49.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:49.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b8a59c640 10 monclient(hunting): handle_auth_done global_id 15303 payload 824 2024-09-15T01:36:49.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b8a59c640 1 -- 172.21.15.178:0/3511550659 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0b8c04bc80 con 0x7f0b8c11cfb0 2024-09-15T01:36:49.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.206+0000 7f0b8a59c640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c11cfb0 0x7f0b8c096a30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009940 tx=0x7f0b7c03d000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:49.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b7c03c860 con 0x7f0b8c11cfb0 2024-09-15T01:36:49.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b827fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:49.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b827fc640 10 monclient: dump: 2024-09-15T01:36:49.277 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.277 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.277 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.277 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.277 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9097 (secure 0 0 0) 0x7f0b7c03c3e0 con 0x7f0b8c11cfb0 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b827fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:49.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b827fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:49.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c11cfb0 msgr2=0x7f0b8c096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b827fc640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c11cfb0 0x7f0b8c096a30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009940 tx=0x7f0b7c03d000 comp rx=0 tx=0).stop 2024-09-15T01:36:49.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b827fc640 1 -- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c11cfb0 msgr2=0x7f0b8c096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:49.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b827fc640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b8c11cfb0 0x7f0b8c096a30 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009940 tx=0x7f0b7c03d000 comp rx=0 tx=0).stop 2024-09-15T01:36:49.279 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:49.279 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:49.279 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:49.279 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:49.279 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 64, 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: 7 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:49.280 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:49.281 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:49.281 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:49.281 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:49.281 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:49.281 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:49.281 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:49.281 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:49.281 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:49.281 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:49.281 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:49.281 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:49.282 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:49.282 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:49.282 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:49.282 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:49.282 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:49.282 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:49.282 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:49.282 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:49.282 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.282 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.282 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:49.282 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:49.283 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:49.283 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:49.283 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:49.283 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:49.283 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:49.283 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:49.283 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:49.283 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:49.283 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:49.283 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:49.283 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:49.283 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:49.284 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:49.284 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:49.284 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:49.284 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:49.284 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:49.284 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:49.284 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:49.284 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:49.284 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:49.284 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:49.284 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:49.285 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:49.285 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:49.285 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:49.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.285 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:49.285 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.285 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:49.285 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:49.285 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.285 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.286 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.286 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.286 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:49.286 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.286 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.286 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.286 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.286 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:49.286 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.287 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.287 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.287 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:49.287 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:49.287 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.287 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.287 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.288 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.288 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:49.288 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:49.288 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.288 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.288 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.288 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.288 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:49.288 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.289 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.289 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.289 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:49.289 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.289 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.289 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.289 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:49.289 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.291 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.292 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:49.292 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:49.292 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.292 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.292 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.292 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.292 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:49.292 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.292 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.292 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.292 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:49.293 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.293 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.293 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.293 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:49.293 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:49.293 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.293 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.293 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.293 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.294 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.295 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.295 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.295 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:49.295 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:49.295 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.295 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.295 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.295 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.295 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.295 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:49.295 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:49.295 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:49.297 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:49.297 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.297 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.297 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.297 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.297 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:49.297 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:49.297 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:49.298 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.299 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.299 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.299 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.299 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:49.299 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:49.299 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.299 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.299 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.299 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.299 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:49.299 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:49.301 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:49.302 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:49.303 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:49.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b90f9d640 1 -- 172.21.15.178:0/3511550659 >> v2:172.21.15.204:6812/28210 conn(0x7f0b6c0397b0 msgr2=0x7f0b6c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b90f9d640 1 --2- 172.21.15.178:0/3511550659 >> v2:172.21.15.204:6812/28210 conn(0x7f0b6c0397b0 0x7f0b6c03bc60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c11eb70 tx=0x7f0b840094f0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b90f9d640 10 monclient: shutdown 2024-09-15T01:36:49.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b90f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:49.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b90f9d640 1 -- 172.21.15.178:0/3511550659 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b8c0a08c0 msgr2=0x7f0b8c131840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b90f9d640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b8c0a08c0 0x7f0b8c131840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400ea80 tx=0x7f0b740090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b8ad9d640 1 -- 172.21.15.178:0/3511550659 reap_dead start 2024-09-15T01:36:49.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b90f9d640 1 -- 172.21.15.178:0/3511550659 shutdown_connections 2024-09-15T01:36:49.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b90f9d640 1 --2- 172.21.15.178:0/3511550659 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b8c0a08c0 0x7f0b8c131840 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b90f9d640 1 -- 172.21.15.178:0/3511550659 >> 172.21.15.178:0/3511550659 conn(0x7f0b8c0a5ed0 msgr2=0x7f0b8c0a51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:49.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b90f9d640 1 -- 172.21.15.178:0/3511550659 shutdown_connections 2024-09-15T01:36:49.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.207+0000 7f0b90f9d640 1 -- 172.21.15.178:0/3511550659 wait complete. 2024-09-15T01:36:49.306 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T01:36:49.306 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon h 2024-09-15T01:36:49.306 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:36:49.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.398+0000 7f6fa1f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:49.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.398+0000 7f6fa1f9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:49.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.398+0000 7f6fa1f9d640 10 monclient: monmap: 2024-09-15T01:36:49.400 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:49.400 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.400 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:49.399059+0000 2024-09-15T01:36:49.400 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:49.399059+0000 2024-09-15T01:36:49.400 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:49.401 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 1 Processor -- start 2024-09-15T01:36:49.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 1 -- start start 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 10 monclient: init 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 10 monclient: _add_conns ranks=[7,1,6,3,10,8,5,0,9,2,4] 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f9c0a2930 0x7f6f9c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 10 monclient(hunting): picked mon.noname-i con 0x7f6f9c0a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6f9c09faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f9c09fe70 0x7f6f9c0a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 10 monclient(hunting): picked mon.noname-d con 0x7f6f9c09fe70 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c04bf60 con 0x7f6f9c09faa0 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c09dd80 con 0x7f6f9c09fe70 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0a2d00 con 0x7f6f9c0a2930 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6fa1f9d640 10 monclient(hunting): authenticate will time out at 1014.149666s 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9bfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f9c09fe70 0x7f6f9c0a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9bfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f9c09fe70 0x7f6f9c0a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:50104/0 (socket says 172.21.15.178:50104) 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9bfff640 1 -- 172.21.15.178:0/4092869115 learned_addr learned my addr 172.21.15.178:0/4092869115 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9bfff640 10 monclient(hunting): get_auth_request con 0x7f6f9c09fe70 auth_method 0 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9b7fe640 1 --2- 172.21.15.178:0/4092869115 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f9c0a2930 0x7f6f9c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:49.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9b7fe640 10 monclient(hunting): get_auth_request con 0x7f6f9c0a2930 auth_method 0 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.399+0000 7f6f9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9bfff640 10 monclient(hunting): handle_auth_done global_id 24387 payload 274 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9bfff640 1 -- 172.21.15.178:0/4092869115 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f9c0a2930 msgr2=0x7f6f9c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9bfff640 1 --2- 172.21.15.178:0/4092869115 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f9c0a2930 0x7f6f9c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9bfff640 1 -- 172.21.15.178:0/4092869115 >> v1:172.21.15.204:6789/0 conn(0x7f6f9c09faa0 legacy=0x7f6f9c0a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9bfff640 1 monclient: found mon.noname-d 2024-09-15T01:36:49.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9bfff640 1 -- 172.21.15.178:0/4092869115 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c0a0310 con 0x7f6f9c09fe70 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9bfff640 1 --2- 172.21.15.178:0/4092869115 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f9c09fe70 0x7f6f9c0a0fd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009f50 tx=0x7f6f94009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=fd1076d513956fef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9affd640 1 -- 172.21.15.178:0/4092869115 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f94007880 con 0x7f6f9c09fe70 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9affd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9affd640 10 monclient: dump: 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.410 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9affd640 10 monclient: _finish_auth 0 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:49.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9affd640 1 -- 172.21.15.178:0/4092869115 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9402d050 con 0x7f6f9c09fe70 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6fa1f9d640 5 monclient: authenticate success, global_id 24387 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6fa1f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6fa1f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9affd640 1 -- 172.21.15.178:0/4092869115 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f94030360 con 0x7f6f9c09fe70 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6fa1f9d640 4 set_mon_vals no callback set 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.400+0000 7f6f9affd640 10 monclient: dump: 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.412 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:49.413 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:49.413 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:49.413 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:49.413 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:49.413 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:49.413 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:49.413 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:49.413 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:49.413 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:49.413 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:49.413 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:49.413 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.401+0000 7f6fa1f9d640 10 monclient: shutdown 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.401+0000 7f6fa1f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.401+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/4092869115 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f9c09fe70 msgr2=0x7f6f9c0a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.401+0000 7f6fa1f9d640 1 --2- 172.21.15.178:0/4092869115 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f9c09fe70 0x7f6f9c0a0fd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009f50 tx=0x7f6f94009840 comp rx=0 tx=0).stop 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.401+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/4092869115 shutdown_connections 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.401+0000 7f6fa1f9d640 1 --2- 172.21.15.178:0/4092869115 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f9c09fe70 0x7f6f9c0a0fd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.401+0000 7f6fa1f9d640 1 --2- 172.21.15.178:0/4092869115 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f9c0a2930 0x7f6f9c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.401+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/4092869115 >> 172.21.15.178:0/4092869115 conn(0x7f6f9c0a5ed0 msgr2=0x7f6f9c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.401+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/4092869115 shutdown_connections 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.401+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/4092869115 wait complete. 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient: monmap: 2024-09-15T01:36:49.414 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:49.415 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:49.416 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:49.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 1 Processor -- start 2024-09-15T01:36:49.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 1 -- start start 2024-09-15T01:36:49.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient: init 2024-09-15T01:36:49.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:49.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient: _add_conns ranks=[8,2,0,4,6,3,7,1,5] 2024-09-15T01:36:49.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f9c09faa0 0x7f6f9c09b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f6f9c09faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:49.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f9c09fe70 0x7f6f9c131800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f6f9c09fe70 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:49.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f9c0a2930 0x7f6f9c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:49.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient(hunting): picked mon.noname-a con 0x7f6f9c0a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c11d640 con 0x7f6f9c0a2930 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0a2390 con 0x7f6f9c09fe70 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c09d630 con 0x7f6f9c09faa0 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6fa1f9d640 10 monclient(hunting): authenticate will time out at 1014.152818s 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6f9b7fe640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f9c09faa0 msgr2=0x7f6f9c09b100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f9c09faa0 0x7f6f9c09b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6f9bfff640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f9c0a2930 msgr2=0x7f6f9c11c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6f93fff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f9c09fe70 msgr2=0x7f6f9c131800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6f9bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f9c0a2930 0x7f6f9c11c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:49.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.402+0000 7f6f93fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f9c09fe70 0x7f6f9c131800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:49.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:49.550+0000 7f2fd4ff9640 1 -- 172.21.15.204:0/4150085593 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v181) v1 ==== 78+0+0 (secure 0 0 0) 0x7f2fd801a020 con 0x7f2fe0137d20 2024-09-15T01:36:49.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:49.550+0000 7f2fd4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:36:49.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:49.550+0000 7f2fd4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:49.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.603+0000 7f6f93fff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f9c09fe70 msgr2=0x7f6f9c131800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:49.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.603+0000 7f6f9bfff640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f9c0a2930 msgr2=0x7f6f9c11c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:49.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.603+0000 7f6f9b7fe640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f9c09faa0 msgr2=0x7f6f9c09b100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:49.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.603+0000 7f6f93fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f9c09fe70 0x7f6f9c131800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:49.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.603+0000 7f6f9bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f9c0a2930 0x7f6f9c11c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:49.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:49.603+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f9c09faa0 0x7f6f9c09b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:49.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:49.651+0000 7f2fe6e9f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:36:49.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:49.651+0000 7f2fe6e9f640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:36:49.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:49.651+0000 7f2fe6e9f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:49.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:49.651+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/4150085593 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7f2fe011cbf0 con 0x7f2fe0137d20 2024-09-15T01:36:50.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:50.004+0000 7f6f93fff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f9c09fe70 msgr2=0x7f6f9c131800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:50.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:50.004+0000 7f6f9b7fe640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f9c09faa0 msgr2=0x7f6f9c09b100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:50.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:50.004+0000 7f6f9bfff640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f9c0a2930 msgr2=0x7f6f9c11c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:50.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:50.004+0000 7f6f93fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f9c09fe70 0x7f6f9c131800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:50.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:50.004+0000 7f6f9bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f9c0a2930 0x7f6f9c11c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:50.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:50.004+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f9c09faa0 0x7f6f9c09b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:50.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.562+0000 7f2fd4ff9640 1 -- 172.21.15.204:0/4150085593 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v182) v1 ==== 78+0+0 (secure 0 0 0) 0x7f2fd8012330 con 0x7f2fe0137d20 2024-09-15T01:36:50.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.562+0000 7f2fd4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:36:50.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.562+0000 7f2fd4ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:36:50.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/4150085593 >> v2:172.21.15.204:6812/28210 conn(0x7f2fcc0397b0 msgr2=0x7f2fcc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:50.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 1 --2- 172.21.15.204:0/4150085593 >> v2:172.21.15.204:6812/28210 conn(0x7f2fcc0397b0 0x7f2fcc03bc60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0038f0 tx=0x7f2fdc008040 comp rx=0 tx=0).stop 2024-09-15T01:36:50.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 10 monclient: shutdown 2024-09-15T01:36:50.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:50.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/4150085593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2fe0137d20 msgr2=0x7f2fe0099c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:50.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 1 --2- 172.21.15.204:0/4150085593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2fe0137d20 0x7f2fe0099c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800bdf0 tx=0x7f2fd80096a0 comp rx=0 tx=0).stop 2024-09-15T01:36:50.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/4150085593 shutdown_connections 2024-09-15T01:36:50.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 1 --2- 172.21.15.204:0/4150085593 >> v2:172.21.15.204:6812/28210 conn(0x7f2fcc0397b0 0x7f2fcc03bc60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:50.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 1 --2- 172.21.15.204:0/4150085593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2fe0137d20 0x7f2fe0099c50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:50.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 1 --2- 172.21.15.204:0/4150085593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fe00a2e30 0x7f2fe0099710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:50.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 1 --2- 172.21.15.204:0/4150085593 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2fe009fb80 0x7f2fe00991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:50.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/4150085593 >> 172.21.15.204:0/4150085593 conn(0x7f2fe006e860 msgr2=0x7f2fe00a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:50.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.565+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/4150085593 shutdown_connections 2024-09-15T01:36:50.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.566+0000 7f2fe6e9f640 1 -- 172.21.15.204:0/4150085593 wait complete. 2024-09-15T01:36:50.578 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set create-compat 2024-09-15T01:36:50.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.654+0000 7f84eaac9640 10 monclient: get_monmap_and_config 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.654+0000 7f84eaac9640 10 monclient: build_initial_monmap 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.654+0000 7f84eaac9640 10 monclient: monmap: 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:50.656272+0000 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:50.656272+0000 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:50.658 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:50.659 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:50.659 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:50.659 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:50.659 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:50.659 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:50.659 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:50.659 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:50.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.655+0000 7f84eaac9640 1 Processor -- start 2024-09-15T01:36:50.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.655+0000 7f84eaac9640 1 -- start start 2024-09-15T01:36:50.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.655+0000 7f84eaac9640 10 monclient: init 2024-09-15T01:36:50.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.655+0000 7f84eaac9640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:50.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.655+0000 7f84eaac9640 10 monclient: _add_conns ranks=[3,7,0,9,5,4,10,8,6,1,2] 2024-09-15T01:36:50.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84eaac9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84e40a2e30 0x7f84e4132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:50.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84eaac9640 10 monclient(hunting): picked mon.noname-g con 0x7f84e40a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:50.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84eaac9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e409fb80 0x7f84e409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:50.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84eaac9640 10 monclient(hunting): picked mon.noname-i con 0x7f84e409fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84eaac9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f84e40a0a40 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84eaac9640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e404b720 con 0x7f84e40a0a40 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84eaac9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84eaac9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e409e6c0 con 0x7f84e40a2e30 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84eaac9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84eaac9640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40a04b0 con 0x7f84e409fb80 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84eaac9640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84eaac9640 10 monclient(hunting): authenticate will time out at 1026.299078s 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e409fb80 0x7f84e409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84e8865640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84e40a2e30 0x7f84e4132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e409fb80 0x7f84e409ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:60408/0 (socket says 172.21.15.204:60408) 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84e9066640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f84e40a0a40 legacy=0x7f84e40a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 1 -- 172.21.15.204:0/1085684286 learned_addr learned my addr 172.21.15.204:0/1085684286 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:50.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84e8865640 10 monclient(hunting): get_auth_request con 0x7f84e40a2e30 auth_method 0 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84e8865640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84e8865640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84e8865640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 10 monclient(hunting): get_auth_request con 0x7f84e409fb80 auth_method 0 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84e8865640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84e8865640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84e8865640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:50.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:50.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 10 monclient(hunting): handle_auth_done global_id 24379 payload 274 2024-09-15T01:36:50.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 1 -- 172.21.15.204:0/1085684286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84e40a2e30 msgr2=0x7f84e4132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:50.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 1 --2- 172.21.15.204:0/1085684286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84e40a2e30 0x7f84e4132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:50.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 1 -- 172.21.15.204:0/1085684286 >> v1:172.21.15.178:6789/0 conn(0x7f84e40a0a40 legacy=0x7f84e40a0ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:50.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:50.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 1 monclient: found mon.noname-i 2024-09-15T01:36:50.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:50.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:36:50.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84dbfff640 1 -- 172.21.15.204:0/1085684286 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e40a3200 con 0x7f84e409fb80 2024-09-15T01:36:50.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.656+0000 7f84e8865640 1 --2- 172.21.15.204:0/1085684286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84e40a2e30 0x7f84e4132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:50.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84dbfff640 1 --2- 172.21.15.204:0/1085684286 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e409fb80 0x7f84e409ff70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009940 tx=0x7f84d40091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=249af7bd6191fc7a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:50.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84db7fe640 1 -- 172.21.15.204:0/1085684286 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84d402ac70 con 0x7f84e409fb80 2024-09-15T01:36:50.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84db7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:50.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84db7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:36:50.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84db7fe640 10 monclient: dump: 2024-09-15T01:36:50.673 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:50.673 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:50.673 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:50.673 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:50.673 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:50.673 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84db7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84eaac9640 5 monclient: authenticate success, global_id 24379 2024-09-15T01:36:50.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84eaac9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:50.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84db7fe640 1 -- 172.21.15.204:0/1085684286 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d402adf0 con 0x7f84e409fb80 2024-09-15T01:36:50.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84db7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:50.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84eaac9640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:50.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84eaac9640 4 set_mon_vals no callback set 2024-09-15T01:36:50.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84db7fe640 1 -- 172.21.15.204:0/1085684286 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84d4031ca0 con 0x7f84e409fb80 2024-09-15T01:36:50.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84db7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:50.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84db7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:50.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84db7fe640 10 monclient: dump: 2024-09-15T01:36:50.675 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:50.676 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:50.677 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:50.677 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:50.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84eaac9640 10 monclient: shutdown 2024-09-15T01:36:50.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84eaac9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:50.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84eaac9640 1 -- 172.21.15.204:0/1085684286 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e409fb80 msgr2=0x7f84e409ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:50.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84eaac9640 1 --2- 172.21.15.204:0/1085684286 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e409fb80 0x7f84e409ff70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009940 tx=0x7f84d40091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:50.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84eaac9640 1 -- 172.21.15.204:0/1085684286 shutdown_connections 2024-09-15T01:36:50.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84eaac9640 1 --2- 172.21.15.204:0/1085684286 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e409fb80 0x7f84e409ff70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:50.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84eaac9640 1 --2- 172.21.15.204:0/1085684286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84e40a2e30 0x7f84e4132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:50.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.657+0000 7f84eaac9640 1 -- 172.21.15.204:0/1085684286 >> 172.21.15.204:0/1085684286 conn(0x7f84e406e860 msgr2=0x7f84e406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.658+0000 7f84eaac9640 1 -- 172.21.15.204:0/1085684286 shutdown_connections 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.658+0000 7f84eaac9640 1 -- 172.21.15.204:0/1085684286 wait complete. 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.658+0000 7f84eaac9640 10 monclient: build_initial_monmap 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.658+0000 7f84eaac9640 10 monclient: monmap: 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:50.678 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:50.679 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:50.679 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:50.679 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:50.679 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:50.679 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:50.679 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:50.679 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:50.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.658+0000 7f84eaac9640 1 Processor -- start 2024-09-15T01:36:50.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 1 -- start start 2024-09-15T01:36:50.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 10 monclient: init 2024-09-15T01:36:50.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:50.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 10 monclient: _add_conns ranks=[1,7,6,5,2,8,3,4,0] 2024-09-15T01:36:50.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84e409fb80 0x7f84e4141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:50.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 10 monclient(hunting): picked mon.noname-f con 0x7f84e409fb80 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:50.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e40a0a40 0x7f84e4131a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:50.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 10 monclient(hunting): picked mon.noname-i con 0x7f84e40a0a40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:50.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f84e40a2e30 0x7f84e411e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:50.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 10 monclient(hunting): picked mon.noname-d con 0x7f84e40a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:50.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:50.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e4120170 con 0x7f84e409fb80 2024-09-15T01:36:50.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:50.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f84e409d650 con 0x7f84e40a2e30 2024-09-15T01:36:50.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:50.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f84e4121b50 con 0x7f84e40a0a40 2024-09-15T01:36:50.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:50.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84eaac9640 10 monclient(hunting): authenticate will time out at 1026.302478s 2024-09-15T01:36:50.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e40a0a40 0x7f84e4131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:50.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84e8865640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84e409fb80 0x7f84e4141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:50.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e40a0a40 0x7f84e4131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:60414/0 (socket says 172.21.15.204:60414) 2024-09-15T01:36:50.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 1 -- 172.21.15.204:0/3925706074 learned_addr learned my addr 172.21.15.204:0/3925706074 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:50.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 10 monclient(hunting): get_auth_request con 0x7f84e40a0a40 auth_method 0 2024-09-15T01:36:50.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:50.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:50.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:50.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84e8865640 10 monclient(hunting): get_auth_request con 0x7f84e409fb80 auth_method 0 2024-09-15T01:36:50.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84e8865640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:50.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84e8865640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:50.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84e8865640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:50.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:50.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:50.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:50.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84e8865640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:50.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84e8865640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:50.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84e8865640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:50.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84e9066640 1 --2- 172.21.15.204:0/3925706074 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f84e40a2e30 0x7f84e411e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:50.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 10 monclient(hunting): handle_auth_done global_id 24388 payload 824 2024-09-15T01:36:50.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 1 -- 172.21.15.204:0/3925706074 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f84e40a2e30 msgr2=0x7f84e411e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:50.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 1 --2- 172.21.15.204:0/3925706074 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f84e40a2e30 0x7f84e411e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:50.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 1 -- 172.21.15.204:0/3925706074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84e409fb80 msgr2=0x7f84e4141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:50.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 1 --2- 172.21.15.204:0/3925706074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84e409fb80 0x7f84e4141c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:50.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:50.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 1 monclient: found mon.noname-i 2024-09-15T01:36:50.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:50.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:36:50.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84dbfff640 1 -- 172.21.15.204:0/3925706074 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e0009390 con 0x7f84e40a0a40 2024-09-15T01:36:50.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84e9066640 10 monclient: get_auth_request con 0x7f84e40a2e30 auth_method 0 2024-09-15T01:36:50.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84e9066640 1 --2- 172.21.15.204:0/3925706074 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f84e40a2e30 0x7f84e411e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:36:50.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.659+0000 7f84e8865640 1 --2- 172.21.15.204:0/3925706074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84e409fb80 0x7f84e4141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:50.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84dbfff640 1 --2- 172.21.15.204:0/3925706074 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e40a0a40 0x7f84e4131a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f84d40097e0 tx=0x7f84d4032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:50.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84d8ff9640 1 -- 172.21.15.204:0/3925706074 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84d403b870 con 0x7f84e40a0a40 2024-09-15T01:36:50.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84d8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:50.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84d8ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:36:50.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84d8ff9640 10 monclient: dump: 2024-09-15T01:36:50.685 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:50.685 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:50.685 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:50.685 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:50.685 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:50.685 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:50.685 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:50.685 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:50.685 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:50.685 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:50.686 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:50.686 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:50.686 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:50.686 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:50.686 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:50.686 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:50.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84d8ff9640 10 monclient: _finish_auth 0 2024-09-15T01:36:50.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84d8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:50.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84eaac9640 5 monclient: authenticate success, global_id 24388 2024-09-15T01:36:50.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84eaac9640 10 monclient: _renew_subs 2024-09-15T01:36:50.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84d8ff9640 1 -- 172.21.15.204:0/3925706074 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d403ba10 con 0x7f84e40a0a40 2024-09-15T01:36:50.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84eaac9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:50.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84eaac9640 1 -- 172.21.15.204:0/3925706074 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d4009450 con 0x7f84e40a0a40 2024-09-15T01:36:50.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84d8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:50.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84d8ff9640 1 -- 172.21.15.204:0/3925706074 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84d40313a0 con 0x7f84e40a0a40 2024-09-15T01:36:50.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84d8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:50.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84d8ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:50.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84d8ff9640 10 monclient: dump: 2024-09-15T01:36:50.687 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:50.687 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:50.688 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84eaac9640 10 monclient: _renew_subs 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84eaac9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84eaac9640 1 -- 172.21.15.204:0/3925706074 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e4142170 con 0x7f84e40a0a40 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.660+0000 7f84daffd640 4 set_mon_vals no callback set 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.661+0000 7f84d8ff9640 1 -- 172.21.15.204:0/3925706074 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f84d4039040 con 0x7f84e40a0a40 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.661+0000 7f84eaac9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.661+0000 7f84eaac9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.661+0000 7f84eaac9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.661+0000 7f84d8ff9640 1 --2- 172.21.15.204:0/3925706074 >> v2:172.21.15.204:6812/28210 conn(0x7f84d00397e0 0x7f84d003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.661+0000 7f84eaac9640 1 -- 172.21.15.204:0/3925706074 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e409cab0 con 0x7f84e40a0a40 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.661+0000 7f84d8ff9640 1 -- 172.21.15.204:0/3925706074 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6415+0+0 (secure 0 0 0) 0x7f84d406b220 con 0x7f84e40a0a40 2024-09-15T01:36:50.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.661+0000 7f84e8865640 1 --2- 172.21.15.204:0/3925706074 >> v2:172.21.15.204:6812/28210 conn(0x7f84d00397e0 0x7f84d003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:50.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.661+0000 7f84e8865640 10 monclient: get_auth_request con 0x7f84d00397e0 auth_method 0 2024-09-15T01:36:50.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.661+0000 7f84e8865640 1 --2- 172.21.15.204:0/3925706074 >> v2:172.21.15.204:6812/28210 conn(0x7f84d00397e0 0x7f84d003bc90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009fd0 tx=0x7f84e00090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:50.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.665+0000 7f84d8ff9640 1 -- 172.21.15.204:0/3925706074 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f84e409cab0 con 0x7f84e40a0a40 2024-09-15T01:36:50.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.665+0000 7f84d8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:50.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.665+0000 7f84d8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:50.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:50.805+0000 7f6f93fff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f9c09fe70 msgr2=0x7f6f9c131800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:50.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:50.805+0000 7f6f9bfff640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f9c0a2930 msgr2=0x7f6f9c11c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:50.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:50.805+0000 7f6f9b7fe640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f9c09faa0 msgr2=0x7f6f9c09b100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:50.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:50.805+0000 7f6f93fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f9c09fe70 0x7f6f9c131800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:50.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:50.805+0000 7f6f9b7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f9c09faa0 0x7f6f9c09b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:50.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:50.805+0000 7f6f9bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f9c0a2930 0x7f6f9c11c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:50.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.827+0000 7f84eaac9640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:36:50.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.828+0000 7f84eaac9640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:36:50.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.828+0000 7f84eaac9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:50.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:50.828+0000 7f84eaac9640 1 -- 172.21.15.204:0/3925706074 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7f84e409cab0 con 0x7f84e40a0a40 2024-09-15T01:36:51.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.462+0000 7f84d8ff9640 1 -- 172.21.15.204:0/3925706074 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v183) v1 ==== 82+0+0 (secure 0 0 0) 0x7f84e409cab0 con 0x7f84e40a0a40 2024-09-15T01:36:51.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.462+0000 7f84d8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:36:51.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.462+0000 7f84d8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:51.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.569+0000 7f84eaac9640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:36:51.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.570+0000 7f84eaac9640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:36:51.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.570+0000 7f84eaac9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:36:51.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.570+0000 7f84eaac9640 1 -- 172.21.15.204:0/3925706074 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7f84e40667e0 con 0x7f84e40a0a40 2024-09-15T01:36:51.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.570+0000 7f84d8ff9640 1 -- 172.21.15.204:0/3925706074 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v183) v1 ==== 115+0+0 (secure 0 0 0) 0x7f84d4042070 con 0x7f84e40a0a40 2024-09-15T01:36:51.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.570+0000 7f84d8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-09-15T01:36:51.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.570+0000 7f84d8ff9640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-09-15T01:36:51.573 INFO:tasks.workunit.client.0.smithi204.stderr:compat weight-set already created 2024-09-15T01:36:51.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 1 -- 172.21.15.204:0/3925706074 >> v2:172.21.15.204:6812/28210 conn(0x7f84d00397e0 msgr2=0x7f84d003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:51.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 1 --2- 172.21.15.204:0/3925706074 >> v2:172.21.15.204:6812/28210 conn(0x7f84d00397e0 0x7f84d003bc90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009fd0 tx=0x7f84e00090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:51.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 10 monclient: shutdown 2024-09-15T01:36:51.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:51.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 1 -- 172.21.15.204:0/3925706074 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e40a0a40 msgr2=0x7f84e4131a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:51.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 1 --2- 172.21.15.204:0/3925706074 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e40a0a40 0x7f84e4131a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f84d40097e0 tx=0x7f84d4032000 comp rx=0 tx=0).stop 2024-09-15T01:36:51.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 1 -- 172.21.15.204:0/3925706074 shutdown_connections 2024-09-15T01:36:51.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 1 --2- 172.21.15.204:0/3925706074 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f84e40a2e30 0x7f84e411e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:51.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 1 --2- 172.21.15.204:0/3925706074 >> v2:172.21.15.204:6812/28210 conn(0x7f84d00397e0 0x7f84d003bc90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:51.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 1 --2- 172.21.15.204:0/3925706074 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84e40a0a40 0x7f84e4131a00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:51.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 1 --2- 172.21.15.204:0/3925706074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84e409fb80 0x7f84e4141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:51.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 1 -- 172.21.15.204:0/3925706074 >> 172.21.15.204:0/3925706074 conn(0x7f84e406e860 msgr2=0x7f84e409ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:51.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 1 -- 172.21.15.204:0/3925706074 shutdown_connections 2024-09-15T01:36:51.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.572+0000 7f84eaac9640 1 -- 172.21.15.204:0/3925706074 wait complete. 2024-09-15T01:36:51.584 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket r1 rack root=default 2024-09-15T01:36:51.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.657+0000 7f9f86d09640 10 monclient: get_monmap_and_config 2024-09-15T01:36:51.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.657+0000 7f9f86d09640 10 monclient: build_initial_monmap 2024-09-15T01:36:51.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.657+0000 7f9f86d09640 10 monclient: monmap: 2024-09-15T01:36:51.660 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:51.660 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:51.659272+0000 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:51.659272+0000 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:51.661 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:51.662 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:51.662 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 1 Processor -- start 2024-09-15T01:36:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 1 -- start start 2024-09-15T01:36:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 10 monclient: init 2024-09-15T01:36:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 10 monclient: _add_conns ranks=[4,2,10,7,1,5,8,3,0,9,6] 2024-09-15T01:36:51.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9f800a2cd0 0x7f9f80132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 10 monclient(hunting): picked mon.noname-c con 0x7f9f800a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f8009fa20 0x7f9f8009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 10 monclient(hunting): picked mon.noname-b con 0x7f9f8009fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9f800a08e0 0x7f9f800a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 10 monclient(hunting): picked mon.noname-f con 0x7f9f800a08e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.658+0000 7f9f86d09640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f800a30a0 con 0x7f9f800a08e0 2024-09-15T01:36:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f86d09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:51.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f86d09640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f800a0330 con 0x7f9f8009fa20 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f86d09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f86d09640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f800a1db0 con 0x7f9f800a2cd0 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f86d09640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f86d09640 10 monclient(hunting): authenticate will time out at 1027.302019s 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f7ffff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f8009fa20 msgr2=0x7f9f8009fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f84aa5640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9f800a2cd0 msgr2=0x7f9f80132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f7ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f8009fa20 0x7f9f8009fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f84aa5640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9f800a2cd0 0x7f9f80132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9f800a08e0 0x7f9f800a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9f800a08e0 0x7f9f800a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:50646/0 (socket says 172.21.15.204:50646) 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 1 -- 172.21.15.204:0/3808335783 learned_addr learned my addr 172.21.15.204:0/3808335783 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 10 monclient(hunting): get_auth_request con 0x7f9f800a08e0 auth_method 0 2024-09-15T01:36:51.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 10 monclient(hunting): handle_auth_done global_id 6850 payload 274 2024-09-15T01:36:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 1 -- 172.21.15.204:0/3808335783 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9f800a2cd0 msgr2=0x7f9f80132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 1 --2- 172.21.15.204:0/3808335783 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9f800a2cd0 0x7f9f80132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 1 -- 172.21.15.204:0/3808335783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f8009fa20 msgr2=0x7f9f8009fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:51.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 1 --2- 172.21.15.204:0/3808335783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f8009fa20 0x7f9f8009fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 10 monclient: _finish_hunting 0 2024-09-15T01:36:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 1 monclient: found mon.noname-f 2024-09-15T01:36:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.659+0000 7f9f852a6640 1 -- 172.21.15.204:0/3808335783 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f800648a0 con 0x7f9f800a08e0 2024-09-15T01:36:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f852a6640 1 --2- 172.21.15.204:0/3808335783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f800a08e0 0x7f9f800a0d50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009f20 tx=0x7f9f78009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=980a5f02c1a0675e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f7f7fe640 1 -- 172.21.15.204:0/3808335783 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f78007880 con 0x7f9f800a08e0 2024-09-15T01:36:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f7f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f7f7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f7f7fe640 10 monclient: dump: 2024-09-15T01:36:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:51.674 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:51.675 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:51.676 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f7f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f7f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f7f7fe640 1 -- 172.21.15.204:0/3808335783 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7802d050 con 0x7f9f800a08e0 2024-09-15T01:36:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 5 monclient: authenticate success, global_id 6850 2024-09-15T01:36:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:51.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f7f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f7f7fe640 1 -- 172.21.15.204:0/3808335783 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f78030360 con 0x7f9f800a08e0 2024-09-15T01:36:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f7f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 4 set_mon_vals no callback set 2024-09-15T01:36:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f7f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f7f7fe640 10 monclient: dump: 2024-09-15T01:36:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:51.677 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:51.678 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:51.679 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 10 monclient: shutdown 2024-09-15T01:36:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 1 -- 172.21.15.204:0/3808335783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f800a08e0 msgr2=0x7f9f800a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 1 --2- 172.21.15.204:0/3808335783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f800a08e0 0x7f9f800a0d50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009f20 tx=0x7f9f78009840 comp rx=0 tx=0).stop 2024-09-15T01:36:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 1 -- 172.21.15.204:0/3808335783 shutdown_connections 2024-09-15T01:36:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 1 --2- 172.21.15.204:0/3808335783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f800a08e0 0x7f9f800a0d50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 1 --2- 172.21.15.204:0/3808335783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f8009fa20 0x7f9f8009fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:51.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 1 --2- 172.21.15.204:0/3808335783 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9f800a2cd0 0x7f9f80132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 1 -- 172.21.15.204:0/3808335783 >> 172.21.15.204:0/3808335783 conn(0x7f9f8006e860 msgr2=0x7f9f8006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 1 -- 172.21.15.204:0/3808335783 shutdown_connections 2024-09-15T01:36:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.660+0000 7f9f86d09640 1 -- 172.21.15.204:0/3808335783 wait complete. 2024-09-15T01:36:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.661+0000 7f9f86d09640 10 monclient: build_initial_monmap 2024-09-15T01:36:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.661+0000 7f9f86d09640 10 monclient: monmap: 2024-09-15T01:36:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:51.680 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:51.681 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.661+0000 7f9f86d09640 1 Processor -- start 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.661+0000 7f9f86d09640 1 -- start start 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.661+0000 7f9f86d09640 10 monclient: init 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 10 monclient: _add_conns ranks=[6,4,8,2,0,7,3,1,5] 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9f8009fa20 0x7f9f80131780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 10 monclient(hunting): picked mon.noname-d con 0x7f9f8009fa20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9f800a08e0 0x7f9f8011d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 10 monclient(hunting): picked mon.noname-c con 0x7f9f800a08e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9f800a2cd0 0x7f9f8011d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 10 monclient(hunting): picked mon.noname-e con 0x7f9f800a2cd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:51.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8009d3d0 con 0x7f9f800a08e0 2024-09-15T01:36:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8009cb60 con 0x7f9f8009fa20 2024-09-15T01:36:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9f801350a0 con 0x7f9f800a2cd0 2024-09-15T01:36:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f86d09640 10 monclient(hunting): authenticate will time out at 1027.305284s 2024-09-15T01:36:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f852a6640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9f800a2cd0 msgr2=0x7f9f8011d840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f852a6640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9f800a2cd0 0x7f9f8011d840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f7ffff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9f800a08e0 msgr2=0x7f9f8011d300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f7ffff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9f800a08e0 0x7f9f8011d300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:51.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9f8009fa20 0x7f9f80131780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9f8009fa20 0x7f9f80131780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:43534/0 (socket says 172.21.15.204:43534) 2024-09-15T01:36:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 1 -- 172.21.15.204:0/2686295875 learned_addr learned my addr 172.21.15.204:0/2686295875 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 10 monclient(hunting): get_auth_request con 0x7f9f8009fa20 auth_method 0 2024-09-15T01:36:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 10 monclient(hunting): handle_auth_done global_id 24396 payload 824 2024-09-15T01:36:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 1 -- 172.21.15.204:0/2686295875 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9f800a2cd0 msgr2=0x7f9f8011d840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:51.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.662+0000 7f9f84aa5640 1 --2- 172.21.15.204:0/2686295875 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9f800a2cd0 0x7f9f8011d840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f84aa5640 1 -- 172.21.15.204:0/2686295875 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9f800a08e0 msgr2=0x7f9f8011d300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f84aa5640 1 --2- 172.21.15.204:0/2686295875 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9f800a08e0 0x7f9f8011d300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f84aa5640 10 monclient: _finish_hunting 0 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f84aa5640 1 monclient: found mon.noname-d 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f84aa5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f84aa5640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f84aa5640 1 -- 172.21.15.204:0/2686295875 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f78009ae0 con 0x7f9f8009fa20 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f84aa5640 1 --2- 172.21.15.204:0/2686295875 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9f8009fa20 0x7f9f80131780 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400b5f0 tx=0x7f9f74009120 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7cff9640 1 -- 172.21.15.204:0/2686295875 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f74009280 con 0x7f9f8009fa20 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7cff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7cff9640 10 monclient: dump: 2024-09-15T01:36:51.685 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:51.686 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7cff9640 10 monclient: _finish_auth 0 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7cff9640 1 -- 172.21.15.204:0/2686295875 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f740050f0 con 0x7f9f8009fa20 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7cff9640 1 -- 172.21.15.204:0/2686295875 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f740052a0 con 0x7f9f8009fa20 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7effd640 4 set_mon_vals no callback set 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f7cff9640 10 monclient: dump: 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:51.687 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:51.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f86d09640 5 monclient: authenticate success, global_id 24396 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f86d09640 10 monclient: _renew_subs 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f86d09640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f86d09640 1 -- 172.21.15.204:0/2686295875 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8011ee00 con 0x7f9f8009fa20 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f86d09640 10 monclient: _renew_subs 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f86d09640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.663+0000 7f9f86d09640 1 -- 172.21.15.204:0/2686295875 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f801208c0 con 0x7f9f8009fa20 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.664+0000 7f9f7cff9640 1 -- 172.21.15.204:0/2686295875 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9f74005400 con 0x7f9f8009fa20 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.664+0000 7f9f86d09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.664+0000 7f9f86d09640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.664+0000 7f9f86d09640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.664+0000 7f9f86d09640 1 -- 172.21.15.204:0/2686295875 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f80070b80 con 0x7f9f8009fa20 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.664+0000 7f9f7cff9640 1 --2- 172.21.15.204:0/2686295875 >> v2:172.21.15.204:6812/28210 conn(0x7f9f640397b0 0x7f9f6403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.664+0000 7f9f7cff9640 1 -- 172.21.15.204:0/2686295875 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f9f7404ba70 con 0x7f9f8009fa20 2024-09-15T01:36:51.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.664+0000 7f9f7ffff640 1 --2- 172.21.15.204:0/2686295875 >> v2:172.21.15.204:6812/28210 conn(0x7f9f640397b0 0x7f9f6403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:51.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.664+0000 7f9f7ffff640 10 monclient: get_auth_request con 0x7f9f640397b0 auth_method 0 2024-09-15T01:36:51.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.664+0000 7f9f7ffff640 1 --2- 172.21.15.204:0/2686295875 >> v2:172.21.15.204:6812/28210 conn(0x7f9f640397b0 0x7f9f6403bc60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c006fd0 tx=0x7f9f6c006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:51.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.669+0000 7f9f7cff9640 1 -- 172.21.15.204:0/2686295875 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9f74010480 con 0x7f9f8009fa20 2024-09-15T01:36:51.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.669+0000 7f9f7cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:51.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.669+0000 7f9f7cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:51.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.828+0000 7f9f86d09640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-09-15T01:36:51.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.828+0000 7f9f86d09640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-09-15T01:36:51.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.828+0000 7f9f86d09640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:51.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:51.828+0000 7f9f86d09640 1 -- 172.21.15.204:0/2686295875 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]} v 0) v1 -- 0x7f9f80120590 con 0x7f9f8009fa20 2024-09-15T01:36:52.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 10 monclient(hunting): tick 2024-09-15T01:36:52.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 monclient(hunting): continuing hunt 2024-09-15T01:36:52.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:36:52.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f9c09faa0 msgr2=0x7f6f9c09b100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:52.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f9c09faa0 0x7f6f9c09b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f9c09fe70 msgr2=0x7f6f9c131800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:52.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f9c09fe70 0x7f6f9c131800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f9c0a2930 msgr2=0x7f6f9c11c530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:52.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f9c0a2930 0x7f6f9c11c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 10 monclient: _add_conns ranks=[7,6,1,4,5,3] 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f800037a0 0x7f6f80005b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 10 monclient(hunting): picked mon.noname-i con 0x7f6f800037a0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f80006180 0x7f6f80008600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 10 monclient(hunting): picked mon.noname-d con 0x7f6f80006180 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f80008ca0 0x7f6f8000b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 10 monclient(hunting): picked mon.noname-f con 0x7f6f80008ca0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c11d640 con 0x7f6f80008ca0 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0a2390 con 0x7f6f80006180 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c09d630 con 0x7f6f800037a0 2024-09-15T01:36:52.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f927fc640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f80006180 0x7f6f80008600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f80006180 0x7f6f80008600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:50112/0 (socket says 172.21.15.178:50112) 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 1 -- 172.21.15.178:0/589294513 learned_addr learned my addr 172.21.15.178:0/589294513 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 10 monclient(hunting): get_auth_request con 0x7f6f80006180 auth_method 0 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9bfff640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f80008ca0 0x7f6f8000b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9b7fe640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f800037a0 0x7f6f80005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9bfff640 10 monclient(hunting): get_auth_request con 0x7f6f80008ca0 auth_method 0 2024-09-15T01:36:52.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9b7fe640 10 monclient(hunting): get_auth_request con 0x7f6f800037a0 auth_method 0 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 10 monclient(hunting): handle_auth_done global_id 24405 payload 824 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 1 -- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f800037a0 msgr2=0x7f6f80005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f800037a0 0x7f6f80005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 1 -- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f80008ca0 msgr2=0x7f6f8000b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f80008ca0 0x7f6f8000b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:52.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 1 monclient: found mon.noname-d 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 1 -- 172.21.15.178:0/589294513 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f94009ae0 con 0x7f6f80006180 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9b7fe640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f800037a0 0x7f6f80005b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9bfff640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f80008ca0 0x7f6f8000b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f9bfff640 1 -- 172.21.15.178:0/589294513 reap_dead start 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.403+0000 7f6f93fff640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f80006180 0x7f6f80008600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400c790 tx=0x7f6f84007ea0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f84007570 con 0x7f6f80006180 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f937fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f937fe640 10 monclient: dump: 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:52.413 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:52.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f937fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f84005780 con 0x7f6f80006180 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f937fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f84005d70 con 0x7f6f80006180 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6fa1f9d640 5 monclient: authenticate success, global_id 24405 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6fa1f9d640 10 monclient: _renew_subs 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6fa1f9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/589294513 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c09bc40 con 0x7f6f80006180 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f937fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f937fe640 10 monclient: dump: 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:52.415 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.416 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6f9a7fc640 4 set_mon_vals no callback set 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6fa1f9d640 10 monclient: _renew_subs 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6fa1f9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.404+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/589294513 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c142b60 con 0x7f6f80006180 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6f840058f0 con 0x7f6f80006180 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6fa1f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6fa1f9d640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c09b100 0x7f6f9c131800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6fa1f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/589294513 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c11eac0 con 0x7f6f9c09b100 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f937fe640 1 --2- 172.21.15.178:0/589294513 >> v2:172.21.15.204:6812/28210 conn(0x7f6f7c039750 0x7f6f7c03bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f6f8404b560 con 0x7f6f80006180 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f9b7fe640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c09b100 0x7f6f9c131800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f9b7fe640 10 monclient: get_auth_request con 0x7f6f9c09b100 auth_method 0 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f9b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f9bfff640 1 --2- 172.21.15.178:0/589294513 >> v2:172.21.15.204:6812/28210 conn(0x7f6f7c039750 0x7f6f7c03bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f9bfff640 10 monclient: get_auth_request con 0x7f6f7c039750 auth_method 0 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.405+0000 7f6f9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f9b7fe640 10 monclient(hunting): handle_auth_done global_id 6125 payload 824 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f9b7fe640 1 -- 172.21.15.178:0/589294513 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f9c04c4a0 con 0x7f6f9c09b100 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f9b7fe640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c09b100 0x7f6f9c131800 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f80003640 tx=0x7f6f8c0090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f9bfff640 1 --2- 172.21.15.178:0/589294513 >> v2:172.21.15.204:6812/28210 conn(0x7f6f7c039750 0x7f6f7c03bc00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f94033000 tx=0x7f6f94036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f8c013ad0 con 0x7f6f9c09b100 2024-09-15T01:36:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f937fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f937fe640 10 monclient: dump: 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.420 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.420 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.420 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.420 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:52.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f6f8c013c30 con 0x7f6f9c09b100 2024-09-15T01:36:52.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f937fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:52.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f937fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:52.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c09b100 msgr2=0x7f6f9c131800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f937fe640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c09b100 0x7f6f9c131800 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f80003640 tx=0x7f6f8c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c09b100 msgr2=0x7f6f9c131800 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:52.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.406+0000 7f6f937fe640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c09b100 0x7f6f9c131800 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f80003640 tx=0x7f6f8c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6fa1f9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:52.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6fa1f9d640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c131d40 0x7f6f9c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6fa1f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/589294513 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c11f5a0 con 0x7f6f9c131d40 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6f9b7fe640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c131d40 0x7f6f9c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6f9b7fe640 10 monclient: get_auth_request con 0x7f6f9c131d40 auth_method 0 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6f9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6f9b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6f9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6f9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6f9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6f9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6f9b7fe640 10 monclient(hunting): handle_auth_done global_id 6134 payload 824 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.413+0000 7f6f9b7fe640 1 -- 172.21.15.178:0/589294513 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6f9c09b4d0 con 0x7f6f9c131d40 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6f9b7fe640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c131d40 0x7f6f9c11c530 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c01a000 tx=0x7f6f8c01b000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f8c023040 con 0x7f6f9c131d40 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6f937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6f937fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6f937fe640 10 monclient: dump: 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.424 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.425 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.425 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.425 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.425 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:52.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9267 (secure 0 0 0) 0x7f6f8c00fda0 con 0x7f6f9c131d40 2024-09-15T01:36:52.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6f937fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:52.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6f937fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:52.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c131d40 msgr2=0x7f6f9c11c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6f937fe640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c131d40 0x7f6f9c11c530 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c01a000 tx=0x7f6f8c01b000 comp rx=0 tx=0).stop 2024-09-15T01:36:52.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6f937fe640 1 -- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c131d40 msgr2=0x7f6f9c11c530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:52.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6f937fe640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c131d40 0x7f6f9c11c530 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.425 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:52.426 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:52.426 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:52.426 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:52.426 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 64, 2024-09-15T01:36:52.426 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:52.426 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:52.426 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:52.426 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:52.426 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:52.426 INFO:teuthology.orchestra.run.smithi178.stdout: 7 2024-09-15T01:36:52.426 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:52.426 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:52.427 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.428 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:52.429 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:52.430 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.431 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.432 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:52.432 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.432 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.432 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.432 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:52.432 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:52.432 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.432 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.432 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.433 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:52.440 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:52.443 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:52.444 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.445 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:52.446 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 4 2024-09-15T01:36:52.447 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.448 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.448 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:52.448 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.414+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/589294513 >> v2:172.21.15.204:6812/28210 conn(0x7f6f7c039750 msgr2=0x7f6f7c03bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 1 --2- 172.21.15.178:0/589294513 >> v2:172.21.15.204:6812/28210 conn(0x7f6f7c039750 0x7f6f7c03bc00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f94033000 tx=0x7f6f94036000 comp rx=0 tx=0).stop 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 10 monclient: shutdown 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/589294513 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f80006180 msgr2=0x7f6f80008600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f80006180 0x7f6f80008600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400c790 tx=0x7f6f84007ea0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/589294513 shutdown_connections 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 1 --2- 172.21.15.178:0/589294513 >> v2:172.21.15.204:6812/28210 conn(0x7f6f7c039750 0x7f6f7c03bc00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6f80006180 0x7f6f80008600 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c09b100 0x7f6f9c131800 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 1 --2- 172.21.15.178:0/589294513 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f9c131d40 0x7f6f9c11c530 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/589294513 >> 172.21.15.178:0/589294513 conn(0x7f6f9c0a5ed0 msgr2=0x7f6f9c09e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/589294513 shutdown_connections 2024-09-15T01:36:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.415+0000 7f6fa1f9d640 1 -- 172.21.15.178:0/589294513 wait complete. 2024-09-15T01:36:52.451 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T01:36:52.451 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon i 2024-09-15T01:36:52.451 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:36:52.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.551+0000 7fedfa19d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:52.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.551+0000 7fedfa19d640 10 monclient: build_initial_monmap 2024-09-15T01:36:52.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.551+0000 7fedfa19d640 10 monclient: monmap: 2024-09-15T01:36:52.553 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:52.553 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.553 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:36:52.552104+0000 2024-09-15T01:36:52.553 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:36:52.552104+0000 2024-09-15T01:36:52.553 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:52.554 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:52.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 1 Processor -- start 2024-09-15T01:36:52.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 1 -- start start 2024-09-15T01:36:52.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 10 monclient: init 2024-09-15T01:36:52.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:52.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 10 monclient: _add_conns ranks=[0,4,7,3,1,6,8,5,2,10,9] 2024-09-15T01:36:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fedf40a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedf4132de0 0x7fedf4137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 10 monclient(hunting): picked mon.noname-c con 0x7fedf4132de0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf41380e0 0x7fedf413a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 10 monclient(hunting): picked mon.noname-i con 0x7fedf41380e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf404bf60 con 0x7fedf40a2930 2024-09-15T01:36:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedf409dd80 con 0x7fedf4132de0 2024-09-15T01:36:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedf37fe640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7fedf40a2930 legacy=0x7fedf409faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:36:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40a2d00 con 0x7fedf41380e0 2024-09-15T01:36:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedebfff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedf4132de0 msgr2=0x7fedf4137ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedfa19d640 10 monclient(hunting): authenticate will time out at 1017.302820s 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.552+0000 7fedebfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedf4132de0 0x7fedf4137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf41380e0 0x7fedf413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf41380e0 0x7fedf413a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:44610/0 (socket says 172.21.15.178:44610) 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 1 -- 172.21.15.178:0/4080536064 learned_addr learned my addr 172.21.15.178:0/4080536064 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 10 monclient(hunting): get_auth_request con 0x7fedf41380e0 auth_method 0 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 10 monclient(hunting): handle_auth_done global_id 24406 payload 274 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 1 -- 172.21.15.178:0/4080536064 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedf4132de0 msgr2=0x7fedf4137ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 1 --2- 172.21.15.178:0/4080536064 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedf4132de0 0x7fedf4137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 1 -- 172.21.15.178:0/4080536064 >> v1:172.21.15.178:6789/0 conn(0x7fedf40a2930 legacy=0x7fedf409faa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 1 monclient: found mon.noname-i 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.553+0000 7fedf3fff640 1 -- 172.21.15.178:0/4080536064 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf41331b0 con 0x7fedf41380e0 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf3fff640 1 --2- 172.21.15.178:0/4080536064 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf41380e0 0x7fedf413a4c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fedec009f50 tx=0x7fedec009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=cf14e53e0d5c62d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf2ffd640 1 -- 172.21.15.178:0/4080536064 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedec007880 con 0x7fedf41380e0 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf2ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf2ffd640 10 monclient: dump: 2024-09-15T01:36:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf2ffd640 1 -- 172.21.15.178:0/4080536064 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec02d050 con 0x7fedf41380e0 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf2ffd640 1 -- 172.21.15.178:0/4080536064 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedec030360 con 0x7fedf41380e0 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedfa19d640 5 monclient: authenticate success, global_id 24406 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedfa19d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedfa19d640 4 set_mon_vals no callback set 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:52.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.554+0000 7fedf2ffd640 10 monclient: dump: 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.569 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.555+0000 7fedfa19d640 10 monclient: shutdown 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.555+0000 7fedfa19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.555+0000 7fedfa19d640 1 -- 172.21.15.178:0/4080536064 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf41380e0 msgr2=0x7fedf413a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.555+0000 7fedfa19d640 1 --2- 172.21.15.178:0/4080536064 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf41380e0 0x7fedf413a4c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fedec009f50 tx=0x7fedec009840 comp rx=0 tx=0).stop 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.555+0000 7fedfa19d640 1 -- 172.21.15.178:0/4080536064 shutdown_connections 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.555+0000 7fedfa19d640 1 --2- 172.21.15.178:0/4080536064 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf41380e0 0x7fedf413a4c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.555+0000 7fedfa19d640 1 --2- 172.21.15.178:0/4080536064 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fedf4132de0 0x7fedf4137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.555+0000 7fedfa19d640 1 -- 172.21.15.178:0/4080536064 >> 172.21.15.178:0/4080536064 conn(0x7fedf40a5ed0 msgr2=0x7fedf40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.555+0000 7fedfa19d640 1 -- 172.21.15.178:0/4080536064 shutdown_connections 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.555+0000 7fedfa19d640 1 -- 172.21.15.178:0/4080536064 wait complete. 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.555+0000 7fedfa19d640 10 monclient: build_initial_monmap 2024-09-15T01:36:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.555+0000 7fedfa19d640 10 monclient: monmap: 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:52.571 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 1 Processor -- start 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 1 -- start start 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 10 monclient: init 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 10 monclient: _add_conns ranks=[1,8,2,7,4,3,0,6,5] 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedf40a2930 0x7fedf409aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 10 monclient(hunting): picked mon.noname-f con 0x7fedf40a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedf4132de0 0x7fedf409b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 10 monclient(hunting): picked mon.noname-e con 0x7fedf4132de0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedf41380e0 0x7fedf409b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 10 monclient(hunting): picked mon.noname-b con 0x7fedf41380e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf409d630 con 0x7fedf40a2930 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf411d760 con 0x7fedf41380e0 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fedf411da40 con 0x7fedf4132de0 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedebfff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedf4132de0 msgr2=0x7fedf409b000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedfa19d640 10 monclient(hunting): authenticate will time out at 1017.306603s 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedf3fff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedf41380e0 msgr2=0x7fedf409b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedebfff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedf4132de0 0x7fedf409b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedf3fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedf41380e0 0x7fedf409b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:52.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedf37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedf40a2930 0x7fedf409aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedf37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedf40a2930 0x7fedf409aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:37716/0 (socket says 172.21.15.178:37716) 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedf37fe640 1 -- 172.21.15.178:0/2743383681 learned_addr learned my addr 172.21.15.178:0/2743383681 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedf37fe640 10 monclient(hunting): get_auth_request con 0x7fedf40a2930 auth_method 0 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedf37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedf37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedf37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedf37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.556+0000 7fedf37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedf37fe640 10 monclient(hunting): handle_auth_done global_id 6868 payload 824 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedf37fe640 1 -- 172.21.15.178:0/2743383681 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedf4132de0 msgr2=0x7fedf409b000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedf37fe640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedf4132de0 0x7fedf409b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedf37fe640 1 -- 172.21.15.178:0/2743383681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedf41380e0 msgr2=0x7fedf409b540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedf37fe640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedf41380e0 0x7fedf409b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedf37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedf37fe640 1 monclient: found mon.noname-f 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedf37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedf37fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedf37fe640 1 -- 172.21.15.178:0/2743383681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec009ae0 con 0x7fedf40a2930 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedf37fe640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedf40a2930 0x7fedf409aac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fede400b5c0 tx=0x7fede4009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fede4006150 con 0x7fedf40a2930 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedeb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedeb7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedeb7fe640 10 monclient: dump: 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedeb7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedeb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede40062d0 con 0x7fedf40a2930 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedfa19d640 5 monclient: authenticate success, global_id 6868 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedfa19d640 10 monclient: _renew_subs 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedfa19d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedfa19d640 1 -- 172.21.15.178:0/2743383681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf41327b0 con 0x7fedf40a2930 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedeb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fede40103c0 con 0x7fedf40a2930 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedeb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedeb7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedeb7fe640 10 monclient: dump: 2024-09-15T01:36:52.577 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.578 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.579 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedf27fc640 4 set_mon_vals no callback set 2024-09-15T01:36:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedfa19d640 10 monclient: _renew_subs 2024-09-15T01:36:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedfa19d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.557+0000 7fedfa19d640 1 -- 172.21.15.178:0/2743383681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf409bc20 con 0x7fedf40a2930 2024-09-15T01:36:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedfa19d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedfa19d640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf411c800 0x7fedf4096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedfa19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedfa19d640 1 -- 172.21.15.178:0/2743383681 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40a62f0 con 0x7fedf411c800 2024-09-15T01:36:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fede4003b80 con 0x7fedf40a2930 2024-09-15T01:36:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedebfff640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf411c800 0x7fedf4096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedebfff640 10 monclient: get_auth_request con 0x7fedf411c800 auth_method 0 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedeb7fe640 1 --2- 172.21.15.178:0/2743383681 >> v2:172.21.15.204:6812/28210 conn(0x7fedd40397b0 0x7fedd403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fede404aee0 con 0x7fedf40a2930 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedebfff640 10 monclient(hunting): handle_auth_done global_id 24415 payload 824 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedebfff640 1 -- 172.21.15.178:0/2743383681 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fedf411d8d0 con 0x7fedf411c800 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedf3fff640 1 --2- 172.21.15.178:0/2743383681 >> v2:172.21.15.204:6812/28210 conn(0x7fedd40397b0 0x7fedd403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.558+0000 7fedf3fff640 10 monclient: get_auth_request con 0x7fedd40397b0 auth_method 0 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedebfff640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf411c800 0x7fedf4096a30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedf40a4460 tx=0x7feddc0090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feddc003850 con 0x7fedf411c800 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedeb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedeb7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedeb7fe640 10 monclient: dump: 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.581 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedf3fff640 1 --2- 172.21.15.178:0/2743383681 >> v2:172.21.15.204:6812/28210 conn(0x7fedd40397b0 0x7fedd403bc60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fedec033000 tx=0x7fedec036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7feddc005300 con 0x7fedf411c800 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedeb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedeb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf411c800 msgr2=0x7fedf4096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedeb7fe640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf411c800 0x7fedf4096a30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedf40a4460 tx=0x7feddc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf411c800 msgr2=0x7fedf4096a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:52.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.559+0000 7fedeb7fe640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf411c800 0x7fedf4096a30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.572+0000 7f9f7cff9640 1 -- 172.21.15.204:0/2686295875 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}]=0 added bucket r1 type rack to location {root=default} v184) v1 ==== 176+0+0 (secure 0 0 0) 0x7f9f740193c0 con 0x7f9f8009fa20 2024-09-15T01:36:52.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.572+0000 7f9f7cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-09-15T01:36:52.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.572+0000 7f9f7cff9640 10 monclient: _finish_command 2 = system:0 added bucket r1 type rack to location {root=default} 2024-09-15T01:36:52.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedfa19d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:36:52.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedfa19d640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf4096f70 0x7fedf4097370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedfa19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedfa19d640 1 -- 172.21.15.178:0/2743383681 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fedf411deb0 con 0x7fedf4096f70 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedebfff640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf4096f70 0x7fedf4097370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedebfff640 10 monclient: get_auth_request con 0x7fedf4096f70 auth_method 0 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedebfff640 10 monclient(hunting): handle_auth_done global_id 24424 payload 824 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.565+0000 7fedebfff640 1 -- 172.21.15.178:0/2743383681 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fedf404bc80 con 0x7fedf4096f70 2024-09-15T01:36:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedebfff640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf4096f70 0x7fedf4097370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fedf40a4460 tx=0x7feddc015000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feddc01f890 con 0x7fedf4096f70 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedeb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedeb7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedeb7fe640 10 monclient: dump: 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9097 (secure 0 0 0) 0x7feddc01f410 con 0x7fedf4096f70 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedeb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedeb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf4096f70 msgr2=0x7fedf4097370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedeb7fe640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf4096f70 0x7fedf4097370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fedf40a4460 tx=0x7feddc015000 comp rx=0 tx=0).stop 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedeb7fe640 1 -- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf4096f70 msgr2=0x7fedf4097370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:36:52.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedeb7fe640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fedf4096f70 0x7fedf4097370 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.589 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:36:52.589 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:52.589 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:52.589 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:36:52.589 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 64, 2024-09-15T01:36:52.589 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:36:52.589 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:36:52.589 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:36:52.589 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: 7 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 7, 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:52.590 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:52.591 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:36:52.592 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:36:52.593 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:36:52.594 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.596 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:36:52.597 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:36:52.599 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.600 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:36:52.602 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:36:52.604 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:36:52.604 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.604 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.604 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.604 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:36:52.604 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:36:52.604 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:36:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:36:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:36:52.606 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.607 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:36:52.607 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.607 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.607 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:36:52.607 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:36:52.607 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.607 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.608 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:36:52.610 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:36:52.611 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:36:52.611 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:36:52.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:36:52.611 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:36:52.611 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:36:52.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedfa19d640 1 -- 172.21.15.178:0/2743383681 >> v2:172.21.15.204:6812/28210 conn(0x7fedd40397b0 msgr2=0x7fedd403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedfa19d640 1 --2- 172.21.15.178:0/2743383681 >> v2:172.21.15.204:6812/28210 conn(0x7fedd40397b0 0x7fedd403bc60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fedec033000 tx=0x7fedec036000 comp rx=0 tx=0).stop 2024-09-15T01:36:52.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedfa19d640 10 monclient: shutdown 2024-09-15T01:36:52.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedfa19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:52.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedfa19d640 1 -- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedf40a2930 msgr2=0x7fedf409aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.566+0000 7fedfa19d640 1 --2- 172.21.15.178:0/2743383681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedf40a2930 0x7fedf409aac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fede400b5c0 tx=0x7fede4009120 comp rx=0 tx=0).stop 2024-09-15T01:36:52.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.567+0000 7fedf3fff640 1 -- 172.21.15.178:0/2743383681 reap_dead start 2024-09-15T01:36:52.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.567+0000 7fedf3fff640 1 -- 172.21.15.178:0/2743383681 reap_dead start 2024-09-15T01:36:52.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.567+0000 7fedfa19d640 1 -- 172.21.15.178:0/2743383681 shutdown_connections 2024-09-15T01:36:52.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.567+0000 7fedfa19d640 1 -- 172.21.15.178:0/2743383681 >> 172.21.15.178:0/2743383681 conn(0x7fedf40a5ed0 msgr2=0x7fedf409f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:52.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.567+0000 7fedfa19d640 1 -- 172.21.15.178:0/2743383681 shutdown_connections 2024-09-15T01:36:52.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:36:52.567+0000 7fedfa19d640 1 -- 172.21.15.178:0/2743383681 wait complete. 2024-09-15T01:36:52.613 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T01:36:52.614 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T01:36:52.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.670+0000 7f9f86d09640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-09-15T01:36:52.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.670+0000 7f9f86d09640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-09-15T01:36:52.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.670+0000 7f9f86d09640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:52.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.670+0000 7f9f86d09640 1 -- 172.21.15.204:0/2686295875 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]} v 0) v1 -- 0x7f9f800008d0 con 0x7f9f8009fa20 2024-09-15T01:36:52.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.671+0000 7f9f7cff9640 1 -- 172.21.15.204:0/2686295875 <== mon.6 v2:172.21.15.178:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}]=0 bucket 'r1' already exists v184) v1 ==== 150+0+0 (secure 0 0 0) 0x7f9f7400e340 con 0x7f9f8009fa20 2024-09-15T01:36:52.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.671+0000 7f9f7cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-09-15T01:36:52.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.671+0000 7f9f7cff9640 10 monclient: _finish_command 3 = system:0 bucket 'r1' already exists 2024-09-15T01:36:52.674 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'r1' already exists 2024-09-15T01:36:52.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 1 -- 172.21.15.204:0/2686295875 >> v2:172.21.15.204:6812/28210 conn(0x7f9f640397b0 msgr2=0x7f9f6403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 1 --2- 172.21.15.204:0/2686295875 >> v2:172.21.15.204:6812/28210 conn(0x7f9f640397b0 0x7f9f6403bc60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c006fd0 tx=0x7f9f6c006d30 comp rx=0 tx=0).stop 2024-09-15T01:36:52.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 10 monclient: shutdown 2024-09-15T01:36:52.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:52.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 1 -- 172.21.15.204:0/2686295875 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9f8009fa20 msgr2=0x7f9f80131780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 1 --2- 172.21.15.204:0/2686295875 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9f8009fa20 0x7f9f80131780 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400b5f0 tx=0x7f9f74009120 comp rx=0 tx=0).stop 2024-09-15T01:36:52.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 1 -- 172.21.15.204:0/2686295875 shutdown_connections 2024-09-15T01:36:52.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 1 --2- 172.21.15.204:0/2686295875 >> v2:172.21.15.204:6812/28210 conn(0x7f9f640397b0 0x7f9f6403bc60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 1 --2- 172.21.15.204:0/2686295875 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9f800a2cd0 0x7f9f8011d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 1 --2- 172.21.15.204:0/2686295875 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9f800a08e0 0x7f9f8011d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 1 --2- 172.21.15.204:0/2686295875 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9f8009fa20 0x7f9f80131780 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 1 -- 172.21.15.204:0/2686295875 >> 172.21.15.204:0/2686295875 conn(0x7f9f8006e860 msgr2=0x7f9f800a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:52.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 1 -- 172.21.15.204:0/2686295875 shutdown_connections 2024-09-15T01:36:52.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.673+0000 7f9f86d09640 1 -- 172.21.15.204:0/2686295875 wait complete. 2024-09-15T01:36:52.685 INFO:tasks.workunit.client.0.smithi204.stderr:++ seq 1 32 2024-09-15T01:36:52.699 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:36:52.699 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h1 host rack=r1 2024-09-15T01:36:52.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.773+0000 7f4d51a68640 10 monclient: get_monmap_and_config 2024-09-15T01:36:52.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.773+0000 7f4d51a68640 10 monclient: build_initial_monmap 2024-09-15T01:36:52.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.773+0000 7f4d51a68640 10 monclient: monmap: 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:52.774573+0000 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:52.774573+0000 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:52.776 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:52.777 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:52.777 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:52.777 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:52.777 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:52.777 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:52.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.773+0000 7f4d51a68640 1 Processor -- start 2024-09-15T01:36:52.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.773+0000 7f4d51a68640 1 -- start start 2024-09-15T01:36:52.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.773+0000 7f4d51a68640 10 monclient: init 2024-09-15T01:36:52.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:52.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 10 monclient: _add_conns ranks=[3,6,0,8,2,9,10,7,4,1,5] 2024-09-15T01:36:52.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d4c067130 0x7f4d4c09d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 10 monclient(hunting): picked mon.noname-g con 0x7f4d4c067130 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d4c09c650 0x7f4d4c09ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 10 monclient(hunting): picked mon.noname-d con 0x7f4d4c09c650 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4d4c132970 addr v1:172.21.15.178:6789/0 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4c0648a0 con 0x7f4d4c132970 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c09abd0 con 0x7f4d4c067130 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c067500 con 0x7f4d4c09c650 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d51a68640 10 monclient(hunting): authenticate will time out at 1028.417388s 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4bfff640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f4d4c132970 legacy=0x7f4d4c132e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d4c067130 0x7f4d4c09d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d4c067130 0x7f4d4c09d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:43984/0 (socket says 172.21.15.204:43984) 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 1 -- 172.21.15.204:0/146023418 learned_addr learned my addr 172.21.15.204:0/146023418 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 10 monclient(hunting): get_auth_request con 0x7f4d4c067130 auth_method 0 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4affd640 1 --2- 172.21.15.204:0/146023418 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d4c09c650 0x7f4d4c09ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4affd640 10 monclient(hunting): get_auth_request con 0x7f4d4c09c650 auth_method 0 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:52.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:52.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:52.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:52.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:52.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_done global_id 15321 payload 274 2024-09-15T01:36:52.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 1 -- 172.21.15.204:0/146023418 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d4c09c650 msgr2=0x7f4d4c09ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:52.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 1 --2- 172.21.15.204:0/146023418 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d4c09c650 0x7f4d4c09ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 1 -- 172.21.15.204:0/146023418 >> v1:172.21.15.178:6789/0 conn(0x7f4d4c132970 legacy=0x7f4d4c132e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:52.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:36:52.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 1 monclient: found mon.noname-g 2024-09-15T01:36:52.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:52.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:52.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.774+0000 7f4d4b7fe640 1 -- 172.21.15.204:0/146023418 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c09ae40 con 0x7f4d4c067130 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4b7fe640 1 --2- 172.21.15.204:0/146023418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d4c067130 0x7f4d4c09d900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009940 tx=0x7f4d3c0091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=b1a9b926b4d0fc0e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4a7fc640 1 -- 172.21.15.204:0/146023418 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d3c02ac70 con 0x7f4d4c067130 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4a7fc640 10 monclient: dump: 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.791 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d51a68640 5 monclient: authenticate success, global_id 15321 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d51a68640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4a7fc640 1 -- 172.21.15.204:0/146023418 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3c02adf0 con 0x7f4d4c067130 2024-09-15T01:36:52.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d51a68640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d51a68640 4 set_mon_vals no callback set 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4a7fc640 1 -- 172.21.15.204:0/146023418 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d3c031ca0 con 0x7f4d4c067130 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.775+0000 7f4d4a7fc640 10 monclient: dump: 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:52.793 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.776+0000 7f4d51a68640 10 monclient: shutdown 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.776+0000 7f4d51a68640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.776+0000 7f4d51a68640 1 -- 172.21.15.204:0/146023418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d4c067130 msgr2=0x7f4d4c09d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.776+0000 7f4d51a68640 1 --2- 172.21.15.204:0/146023418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d4c067130 0x7f4d4c09d900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009940 tx=0x7f4d3c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.776+0000 7f4d51a68640 1 -- 172.21.15.204:0/146023418 shutdown_connections 2024-09-15T01:36:52.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.776+0000 7f4d51a68640 1 --2- 172.21.15.204:0/146023418 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d4c09c650 0x7f4d4c09ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.776+0000 7f4d51a68640 1 --2- 172.21.15.204:0/146023418 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d4c067130 0x7f4d4c09d900 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.776+0000 7f4d51a68640 1 -- 172.21.15.204:0/146023418 >> 172.21.15.204:0/146023418 conn(0x7f4d4c0a2c40 msgr2=0x7f4d4c0a3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:52.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.776+0000 7f4d51a68640 1 -- 172.21.15.204:0/146023418 shutdown_connections 2024-09-15T01:36:52.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.776+0000 7f4d51a68640 1 -- 172.21.15.204:0/146023418 wait complete. 2024-09-15T01:36:52.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.776+0000 7f4d51a68640 10 monclient: build_initial_monmap 2024-09-15T01:36:52.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.776+0000 7f4d51a68640 10 monclient: monmap: 2024-09-15T01:36:52.795 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:52.795 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:52.795 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:52.795 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:52.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 1 Processor -- start 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 1 -- start start 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 10 monclient: init 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 10 monclient: _add_conns ranks=[0,5,6,2,8,4,3,1,7] 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d4c067130 0x7f4d4c141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 10 monclient(hunting): picked mon.noname-a con 0x7f4d4c067130 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d4c09c650 0x7f4d4c131d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 10 monclient(hunting): picked mon.noname-h con 0x7f4d4c09c650 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d4c132970 0x7f4d4c1322c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 10 monclient(hunting): picked mon.noname-d con 0x7f4d4c132970 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c099a10 con 0x7f4d4c067130 2024-09-15T01:36:52.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c11f460 con 0x7f4d4c09c650 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c11f180 con 0x7f4d4c132970 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d51a68640 10 monclient(hunting): authenticate will time out at 1028.420705s 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4b7fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d4c067130 msgr2=0x7f4d4c141fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d4c067130 0x7f4d4c141fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4affd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d4c09c650 0x7f4d4c131d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4affd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d4c09c650 0x7f4d4c131d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:43546/0 (socket says 172.21.15.204:43546) 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4bfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d4c132970 0x7f4d4c1322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4affd640 1 -- 172.21.15.204:0/488414035 learned_addr learned my addr 172.21.15.204:0/488414035 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4affd640 10 monclient(hunting): get_auth_request con 0x7f4d4c09c650 auth_method 0 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4bfff640 10 monclient(hunting): get_auth_request con 0x7f4d4c132970 auth_method 0 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.777+0000 7f4d4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d4bfff640 10 monclient(hunting): handle_auth_done global_id 24423 payload 824 2024-09-15T01:36:52.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d4bfff640 1 -- 172.21.15.204:0/488414035 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d4c09c650 msgr2=0x7f4d4c131d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d4bfff640 1 --2- 172.21.15.204:0/488414035 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d4c09c650 0x7f4d4c131d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d4bfff640 1 -- 172.21.15.204:0/488414035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d4c067130 msgr2=0x7f4d4c141fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d4bfff640 1 --2- 172.21.15.204:0/488414035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d4c067130 0x7f4d4c141fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d4bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d4bfff640 1 monclient: found mon.noname-d 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d4bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d4bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d4bfff640 1 -- 172.21.15.204:0/488414035 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c009450 con 0x7f4d4c132970 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d4affd640 1 --2- 172.21.15.204:0/488414035 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d4c09c650 0x7f4d4c131d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d4bfff640 1 --2- 172.21.15.204:0/488414035 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d4c132970 0x7f4d4c1322c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009fd0 tx=0x7f4d44009740 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d337fe640 1 -- 172.21.15.204:0/488414035 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d4400cd00 con 0x7f4d4c132970 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d337fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d337fe640 10 monclient: dump: 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.801 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d337fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d337fe640 1 -- 172.21.15.204:0/488414035 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d440065d0 con 0x7f4d4c132970 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d51a68640 5 monclient: authenticate success, global_id 24423 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d51a68640 10 monclient: _renew_subs 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d51a68640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d51a68640 1 -- 172.21.15.204:0/488414035 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4c11e010 con 0x7f4d4c132970 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d337fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:52.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d337fe640 1 -- 172.21.15.204:0/488414035 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d4400cea0 con 0x7f4d4c132970 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d337fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d337fe640 10 monclient: dump: 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:52.803 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:52.804 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:52.804 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:52.804 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:52.804 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:52.804 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:52.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d51a68640 10 monclient: _renew_subs 2024-09-15T01:36:52.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d49ffb640 4 set_mon_vals no callback set 2024-09-15T01:36:52.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d51a68640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:52.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.778+0000 7f4d51a68640 1 -- 172.21.15.204:0/488414035 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4c142d90 con 0x7f4d4c132970 2024-09-15T01:36:52.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.779+0000 7f4d337fe640 1 -- 172.21.15.204:0/488414035 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4d44012020 con 0x7f4d4c132970 2024-09-15T01:36:52.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.779+0000 7f4d51a68640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:52.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.779+0000 7f4d51a68640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:52.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.779+0000 7f4d51a68640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:52.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.779+0000 7f4d51a68640 1 -- 172.21.15.204:0/488414035 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4c069250 con 0x7f4d4c132970 2024-09-15T01:36:52.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.779+0000 7f4d337fe640 1 --2- 172.21.15.204:0/488414035 >> v2:172.21.15.204:6812/28210 conn(0x7f4d380397b0 0x7f4d3803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:52.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.779+0000 7f4d337fe640 1 -- 172.21.15.204:0/488414035 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6953+0+0 (secure 0 0 0) 0x7f4d4404c020 con 0x7f4d4c132970 2024-09-15T01:36:52.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.779+0000 7f4d4b7fe640 1 --2- 172.21.15.204:0/488414035 >> v2:172.21.15.204:6812/28210 conn(0x7f4d380397b0 0x7f4d3803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:52.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.779+0000 7f4d4b7fe640 10 monclient: get_auth_request con 0x7f4d380397b0 auth_method 0 2024-09-15T01:36:52.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.780+0000 7f4d4b7fe640 1 --2- 172.21.15.204:0/488414035 >> v2:172.21.15.204:6812/28210 conn(0x7f4d380397b0 0x7f4d3803bc60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c034000 tx=0x7f4d3c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:52.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.784+0000 7f4d337fe640 1 -- 172.21.15.204:0/488414035 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4d440153e0 con 0x7f4d4c132970 2024-09-15T01:36:52.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.784+0000 7f4d337fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:52.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.784+0000 7f4d337fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:52.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.941+0000 7f4d51a68640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:52.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.941+0000 7f4d51a68640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:52.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.941+0000 7f4d51a68640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:52.944 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:52.941+0000 7f4d51a68640 1 -- 172.21.15.204:0/488414035 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f4d4c11f5d0 con 0x7f4d4c132970 2024-09-15T01:36:53.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.577+0000 7f4d337fe640 1 -- 172.21.15.204:0/488414035 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}]=0 added bucket h1 type host to location {rack=r1} v185) v1 ==== 166+0+0 (secure 0 0 0) 0x7f4d44007300 con 0x7f4d4c132970 2024-09-15T01:36:53.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.577+0000 7f4d337fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:53.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.577+0000 7f4d337fe640 10 monclient: _finish_command 2 = system:0 added bucket h1 type host to location {rack=r1} 2024-09-15T01:36:53.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.673+0000 7f4d51a68640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:53.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.673+0000 7f4d51a68640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:53.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.673+0000 7f4d51a68640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:36:53.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.673+0000 7f4d51a68640 1 -- 172.21.15.204:0/488414035 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f4d4c070b80 con 0x7f4d4c132970 2024-09-15T01:36:53.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.674+0000 7f4d337fe640 1 -- 172.21.15.204:0/488414035 <== mon.6 v2:172.21.15.178:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h1' already exists v185) v1 ==== 145+0+0 (secure 0 0 0) 0x7f4d440250b0 con 0x7f4d4c132970 2024-09-15T01:36:53.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.674+0000 7f4d337fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:53.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.674+0000 7f4d337fe640 10 monclient: _finish_command 3 = system:0 bucket 'h1' already exists 2024-09-15T01:36:53.677 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h1' already exists 2024-09-15T01:36:53.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 1 -- 172.21.15.204:0/488414035 >> v2:172.21.15.204:6812/28210 conn(0x7f4d380397b0 msgr2=0x7f4d3803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:53.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 1 --2- 172.21.15.204:0/488414035 >> v2:172.21.15.204:6812/28210 conn(0x7f4d380397b0 0x7f4d3803bc60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c034000 tx=0x7f4d3c032000 comp rx=0 tx=0).stop 2024-09-15T01:36:53.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 10 monclient: shutdown 2024-09-15T01:36:53.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:53.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 1 -- 172.21.15.204:0/488414035 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d4c132970 msgr2=0x7f4d4c1322c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:53.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 1 --2- 172.21.15.204:0/488414035 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d4c132970 0x7f4d4c1322c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009fd0 tx=0x7f4d44009740 comp rx=0 tx=0).stop 2024-09-15T01:36:53.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 1 -- 172.21.15.204:0/488414035 shutdown_connections 2024-09-15T01:36:53.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 1 --2- 172.21.15.204:0/488414035 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d4c132970 0x7f4d4c1322c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:53.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 1 --2- 172.21.15.204:0/488414035 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d4c09c650 0x7f4d4c131d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:53.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 1 --2- 172.21.15.204:0/488414035 >> v2:172.21.15.204:6812/28210 conn(0x7f4d380397b0 0x7f4d3803bc60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:53.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 1 --2- 172.21.15.204:0/488414035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d4c067130 0x7f4d4c141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:53.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 1 -- 172.21.15.204:0/488414035 >> 172.21.15.204:0/488414035 conn(0x7f4d4c0a2c40 msgr2=0x7f4d4c0a23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:53.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 1 -- 172.21.15.204:0/488414035 shutdown_connections 2024-09-15T01:36:53.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.676+0000 7f4d51a68640 1 -- 172.21.15.204:0/488414035 wait complete. 2024-09-15T01:36:53.688 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:36:53.688 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h2 host rack=r1 2024-09-15T01:36:53.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.761+0000 7f1487b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:53.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.761+0000 7f1487b9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:53.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.761+0000 7f1487b9d640 10 monclient: monmap: 2024-09-15T01:36:53.764 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:53.764 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:53.764 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:53.763106+0000 2024-09-15T01:36:53.764 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:53.763106+0000 2024-09-15T01:36:53.764 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:53.764 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:53.765 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:53.765 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:53.765 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:53.765 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:53.765 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:53.765 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:53.765 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:53.765 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:53.765 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:53.765 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:53.765 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:53.765 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:53.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 1 Processor -- start 2024-09-15T01:36:53.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 1 -- start start 2024-09-15T01:36:53.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 10 monclient: init 2024-09-15T01:36:53.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:53.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 10 monclient: _add_conns ranks=[1,2,8,6,0,9,7,10,5,4,3] 2024-09-15T01:36:53.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f14800a2cd0 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:53.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f148009fa20 0x7f148009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:53.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 10 monclient(hunting): picked mon.noname-b con 0x7f148009fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:53.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1480137c80 0x7f148013a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:53.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 10 monclient(hunting): picked mon.noname-e con 0x7f1480137c80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:53.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14800648a0 con 0x7f14800a2cd0 2024-09-15T01:36:53.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:53.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1480133310 con 0x7f148009fa20 2024-09-15T01:36:53.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:53.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f14800a30a0 con 0x7f1480137c80 2024-09-15T01:36:53.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:53.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1487b9d640 10 monclient(hunting): authenticate will time out at 1029.405876s 2024-09-15T01:36:53.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1485939640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f14800a2cd0 0x7f1480132c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:50906/0 (socket says 172.21.15.204:50906) 2024-09-15T01:36:53.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.762+0000 7f1485939640 1 -- 172.21.15.204:0/2514407429 learned_addr learned my addr 172.21.15.204:0/2514407429 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:36:53.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f148613a640 1 -- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1480137c80 msgr2=0x7f148013a060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:53.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 1 -- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f148009fa20 msgr2=0x7f148009fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:53.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f148613a640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1480137c80 0x7f148013a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:53.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f148009fa20 0x7f148009fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:53.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293759653 0 0) 0x7f14800648a0 con 0x7f14800a2cd0 2024-09-15T01:36:53.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:53.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:53.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient(hunting): my global_id is 6877 2024-09-15T01:36:53.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f146c0034e0 con 0x7f14800a2cd0 2024-09-15T01:36:53.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 17730456 0 0) 0x7f146c0034e0 con 0x7f14800a2cd0 2024-09-15T01:36:53.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1480137c80 msgr2=0x7f148013a060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:53.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1480137c80 0x7f148013a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:53.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f148009fa20 msgr2=0x7f148009fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:53.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f148009fa20 0x7f148009fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient: _finish_hunting 0 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14800a0330 con 0x7f14800a2cd0 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient: _finish_auth 0 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f147c001820 con 0x7f14800a2cd0 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient: dump: 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:53.779 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:53.780 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:53.781 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:53.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:36:53.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:53.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 >> v1:172.21.15.204:6789/0 conn(0x7f14800a2cd0 legacy=0x7f1480132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:53.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient: _add_conns ranks=[4,5,8,7,3,6,0,1,2] 2024-09-15T01:36:53.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f146c004220 0x7f146c004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:53.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient(hunting): picked mon.c con 0x7f146c004220 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:53.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f146c004ec0 0x7f146c0099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:53.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient(hunting): picked mon.h con 0x7f146c004ec0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:53.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1480137c80 0x7f146c00bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:53.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient(hunting): picked mon.e con 0x7f1480137c80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:53.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1480133310 con 0x7f146c004220 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14800a30a0 con 0x7f146c004ec0 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f146c00c630 con 0x7f1480137c80 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1484937640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f146c004ec0 0x7f146c0099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485939640 1 -- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f146c004220 msgr2=0x7f146c004780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f148613a640 1 -- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1480137c80 msgr2=0x7f146c00bf10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485939640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f146c004220 0x7f146c004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 10 monclient(hunting): get_auth_request con 0x7f146c004ec0 auth_method 0 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:53.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f148613a640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1480137c80 0x7f146c00bf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 10 monclient(hunting): handle_auth_done global_id 6877 payload 293 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 1 -- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1480137c80 msgr2=0x7f146c00bf10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1480137c80 0x7f146c00bf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 1 -- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f146c004220 msgr2=0x7f146c004780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f146c004220 0x7f146c004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 10 monclient: _finish_hunting 0 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 1 monclient: found mon.h 2024-09-15T01:36:53.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 1 -- 172.21.15.204:0/2514407429 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0034e0 con 0x7f146c004ec0 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 10 monclient: _finish_auth 0 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1485138640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1487b9d640 5 monclient: authenticate success, global_id 6877 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.763+0000 7f1487b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1485138640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f146c004ec0 0x7f146c0099c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f14700038f0 tx=0x7f1470009060 comp rx=0 tx=0).ready entity=mon.5 client_cookie=1f657f6703892c47 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1470003df0 con 0x7f146c004ec0 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1484937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1484937640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1484937640 10 monclient: dump: 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:53.784 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:53.785 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147002a900 con 0x7f146c004ec0 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1487b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1484937640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1487b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1484937640 1 -- 172.21.15.204:0/2514407429 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f14700050f0 con 0x7f146c004ec0 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1487b9d640 4 set_mon_vals no callback set 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1484937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1484937640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.764+0000 7f1484937640 10 monclient: dump: 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:53.786 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.765+0000 7f1487b9d640 10 monclient: shutdown 2024-09-15T01:36:53.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.765+0000 7f1487b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.765+0000 7f1487b9d640 1 -- 172.21.15.204:0/2514407429 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f146c004ec0 msgr2=0x7f146c0099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.765+0000 7f1487b9d640 1 --2- 172.21.15.204:0/2514407429 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f146c004ec0 0x7f146c0099c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f14700038f0 tx=0x7f1470009060 comp rx=0 tx=0).stop 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.765+0000 7f148613a640 1 -- 172.21.15.204:0/2514407429 reap_dead start 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.765+0000 7f1487b9d640 1 -- 172.21.15.204:0/2514407429 shutdown_connections 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.765+0000 7f1487b9d640 1 -- 172.21.15.204:0/2514407429 >> 172.21.15.204:0/2514407429 conn(0x7f148006e860 msgr2=0x7f148006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.765+0000 7f1487b9d640 1 -- 172.21.15.204:0/2514407429 shutdown_connections 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.765+0000 7f1487b9d640 1 -- 172.21.15.204:0/2514407429 wait complete. 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.765+0000 7f1487b9d640 10 monclient: build_initial_monmap 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.765+0000 7f1487b9d640 10 monclient: monmap: 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:53.788 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:53.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 1 Processor -- start 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 1 -- start start 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 10 monclient: init 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 10 monclient: _add_conns ranks=[2,1,0,6,5,8,7,3,4] 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f146c004ec0 0x7f14800998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 10 monclient(hunting): picked mon.noname-b con 0x7f146c004ec0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1480099e10 0x7f148009a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 10 monclient(hunting): picked mon.noname-f con 0x7f1480099e10 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f148009a720 0x7f1480148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 10 monclient(hunting): picked mon.noname-a con 0x7f148009a720 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1480130ea0 con 0x7f148009a720 2024-09-15T01:36:53.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1480131810 con 0x7f1480099e10 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148009cb60 con 0x7f146c004ec0 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1487b9d640 10 monclient(hunting): authenticate will time out at 1029.409557s 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485939640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f146c004ec0 msgr2=0x7f14800998d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485939640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f146c004ec0 0x7f14800998d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1480099e10 0x7f148009a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f148613a640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f148009a720 msgr2=0x7f1480148120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f148613a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f148009a720 0x7f1480148120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1480099e10 0x7f148009a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:50648/0 (socket says 172.21.15.204:50648) 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 1 -- 172.21.15.204:0/445541850 learned_addr learned my addr 172.21.15.204:0/445541850 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:53.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 10 monclient(hunting): get_auth_request con 0x7f1480099e10 auth_method 0 2024-09-15T01:36:53.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:53.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:53.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:53.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:53.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:53.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:53.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 10 monclient(hunting): handle_auth_done global_id 6886 payload 824 2024-09-15T01:36:53.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 1 -- 172.21.15.204:0/445541850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f146c004ec0 msgr2=0x7f14800998d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:53.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 1 --2- 172.21.15.204:0/445541850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f146c004ec0 0x7f14800998d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:53.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 1 -- 172.21.15.204:0/445541850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f148009a720 msgr2=0x7f1480148120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:53.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 1 --2- 172.21.15.204:0/445541850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f148009a720 0x7f1480148120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:53.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 10 monclient: _finish_hunting 0 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 1 monclient: found mon.noname-f 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.766+0000 7f1485138640 1 -- 172.21.15.204:0/445541850 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148009b640 con 0x7f1480099e10 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1485138640 1 --2- 172.21.15.204:0/445541850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1480099e10 0x7f148009a1e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1470009890 tx=0x7f1470030000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1475ffb640 1 -- 172.21.15.204:0/445541850 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f147003b070 con 0x7f1480099e10 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1475ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1475ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1475ffb640 10 monclient: dump: 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:53.793 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:53.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1475ffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1475ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1475ffb640 1 -- 172.21.15.204:0/445541850 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1470006ac0 con 0x7f1480099e10 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1487b9d640 5 monclient: authenticate success, global_id 6886 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1475ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1475ffb640 1 -- 172.21.15.204:0/445541850 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1470005700 con 0x7f1480099e10 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1487b9d640 10 monclient: _renew_subs 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1487b9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1487b9d640 1 -- 172.21.15.204:0/445541850 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148009d3d0 con 0x7f1480099e10 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1475ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1477fff640 4 set_mon_vals no callback set 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1475ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1475ffb640 10 monclient: dump: 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:53.795 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:53.796 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1487b9d640 10 monclient: _renew_subs 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1487b9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.767+0000 7f1487b9d640 1 -- 172.21.15.204:0/445541850 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148009b130 con 0x7f1480099e10 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.768+0000 7f1475ffb640 1 -- 172.21.15.204:0/445541850 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1470030070 con 0x7f1480099e10 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.768+0000 7f1487b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.768+0000 7f1475ffb640 1 --2- 172.21.15.204:0/445541850 >> v2:172.21.15.204:6812/28210 conn(0x7f14640397e0 0x7f146403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.768+0000 7f1487b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.768+0000 7f1487b9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.768+0000 7f1487b9d640 1 -- 172.21.15.204:0/445541850 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148011c800 con 0x7f1480099e10 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.768+0000 7f1475ffb640 1 -- 172.21.15.204:0/445541850 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 7163+0+0 (secure 0 0 0) 0x7f147006b3d0 con 0x7f1480099e10 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.768+0000 7f1485939640 1 --2- 172.21.15.204:0/445541850 >> v2:172.21.15.204:6812/28210 conn(0x7f14640397e0 0x7f146403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:53.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.768+0000 7f1485939640 10 monclient: get_auth_request con 0x7f14640397e0 auth_method 0 2024-09-15T01:36:53.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.768+0000 7f1485939640 1 --2- 172.21.15.204:0/445541850 >> v2:172.21.15.204:6812/28210 conn(0x7f14640397e0 0x7f146403bc90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f147c0047d0 tx=0x7f14800a04c0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:53.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.773+0000 7f1475ffb640 1 -- 172.21.15.204:0/445541850 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f148011c800 con 0x7f1480099e10 2024-09-15T01:36:53.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.773+0000 7f1475ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:53.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.773+0000 7f1475ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:53.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.932+0000 7f1487b9d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:53.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.932+0000 7f1487b9d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:53.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.932+0000 7f1487b9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:53.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:53.932+0000 7f1487b9d640 1 -- 172.21.15.204:0/445541850 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f148011c800 con 0x7f1480099e10 2024-09-15T01:36:54.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.581+0000 7f1475ffb640 1 -- 172.21.15.204:0/445541850 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}]=0 added bucket h2 type host to location {rack=r1} v186) v1 ==== 166+0+0 (secure 0 0 0) 0x7f148011c800 con 0x7f1480099e10 2024-09-15T01:36:54.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.581+0000 7f1475ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:54.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.581+0000 7f1475ffb640 10 monclient: _finish_command 2 = system:0 added bucket h2 type host to location {rack=r1} 2024-09-15T01:36:54.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.678+0000 7f1487b9d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:54.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.678+0000 7f1487b9d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:54.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.678+0000 7f1487b9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:36:54.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.678+0000 7f1487b9d640 1 -- 172.21.15.204:0/445541850 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f148011c800 con 0x7f1480099e10 2024-09-15T01:36:54.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.679+0000 7f1475ffb640 1 -- 172.21.15.204:0/445541850 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h2' already exists v186) v1 ==== 145+0+0 (secure 0 0 0) 0x7f148011c800 con 0x7f1480099e10 2024-09-15T01:36:54.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.679+0000 7f1475ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:54.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.679+0000 7f1475ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h2' already exists 2024-09-15T01:36:54.682 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h2' already exists 2024-09-15T01:36:54.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 1 -- 172.21.15.204:0/445541850 >> v2:172.21.15.204:6812/28210 conn(0x7f14640397e0 msgr2=0x7f146403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:54.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 1 --2- 172.21.15.204:0/445541850 >> v2:172.21.15.204:6812/28210 conn(0x7f14640397e0 0x7f146403bc90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f147c0047d0 tx=0x7f14800a04c0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 10 monclient: shutdown 2024-09-15T01:36:54.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:54.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 1 -- 172.21.15.204:0/445541850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1480099e10 msgr2=0x7f148009a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:54.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 1 --2- 172.21.15.204:0/445541850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1480099e10 0x7f148009a1e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1470009890 tx=0x7f1470030000 comp rx=0 tx=0).stop 2024-09-15T01:36:54.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 1 -- 172.21.15.204:0/445541850 shutdown_connections 2024-09-15T01:36:54.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 1 --2- 172.21.15.204:0/445541850 >> v2:172.21.15.204:6812/28210 conn(0x7f14640397e0 0x7f146403bc90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 1 --2- 172.21.15.204:0/445541850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f148009a720 0x7f1480148120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 1 --2- 172.21.15.204:0/445541850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1480099e10 0x7f148009a1e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 1 --2- 172.21.15.204:0/445541850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f146c004ec0 0x7f14800998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 1 -- 172.21.15.204:0/445541850 >> 172.21.15.204:0/445541850 conn(0x7f148006e860 msgr2=0x7f14800a4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:54.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 1 -- 172.21.15.204:0/445541850 shutdown_connections 2024-09-15T01:36:54.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.681+0000 7f1487b9d640 1 -- 172.21.15.204:0/445541850 wait complete. 2024-09-15T01:36:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:36:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h3 host rack=r1 2024-09-15T01:36:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.766+0000 7fe69d19d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.766+0000 7fe69d19d640 10 monclient: build_initial_monmap 2024-09-15T01:36:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.766+0000 7fe69d19d640 10 monclient: monmap: 2024-09-15T01:36:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:54.767542+0000 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:54.767542+0000 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:54.770 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:54.770 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:54.770 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:54.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.766+0000 7fe69d19d640 1 Processor -- start 2024-09-15T01:36:54.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.766+0000 7fe69d19d640 1 -- start start 2024-09-15T01:36:54.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.766+0000 7fe69d19d640 10 monclient: init 2024-09-15T01:36:54.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:54.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 10 monclient: _add_conns ranks=[5,9,8,0,3,4,6,10,2,7,1] 2024-09-15T01:36:54.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe69806e890 0x7fe698132cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 10 monclient(hunting): picked mon.noname-h con 0x7fe69806e890 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe69809fb10 0x7fe69809ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 10 monclient(hunting): picked mon.noname-a con 0x7fe69809fb10 addr v2:172.21.15.178:3300/0 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe6980a09c0 0x7fe6980a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 10 monclient(hunting): picked mon.noname-e con 0x7fe6980a09c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6980a1d30 con 0x7fe69809fb10 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe69806ec60 con 0x7fe69806e890 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe69809e020 con 0x7fe6980a09c0 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:54.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69d19d640 10 monclient(hunting): authenticate will time out at 1030.410186s 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe69806e890 0x7fe698132cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe69806e890 0x7fe698132cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:43574/0 (socket says 172.21.15.204:43574) 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 1 -- 172.21.15.204:0/2999891150 learned_addr learned my addr 172.21.15.204:0/2999891150 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 10 monclient(hunting): get_auth_request con 0x7fe69806e890 auth_method 0 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69759e640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe6980a09c0 msgr2=0x7fe6980a0e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe68e59c640 1 -- 172.21.15.204:0/2999891150 >> v2:172.21.15.178:3300/0 conn(0x7fe69809fb10 msgr2=0x7fe69809ff20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe69759e640 1 --2- 172.21.15.204:0/2999891150 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe6980a09c0 0x7fe6980a0e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe68e59c640 1 --2- 172.21.15.204:0/2999891150 >> v2:172.21.15.178:3300/0 conn(0x7fe69809fb10 0x7fe69809ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:54.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 10 monclient(hunting): handle_auth_done global_id 6152 payload 274 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 1 -- 172.21.15.204:0/2999891150 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe6980a09c0 msgr2=0x7fe6980a0e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 1 --2- 172.21.15.204:0/2999891150 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe6980a09c0 0x7fe6980a0e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 1 -- 172.21.15.204:0/2999891150 >> v2:172.21.15.178:3300/0 conn(0x7fe69809fb10 msgr2=0x7fe69809ff20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 1 --2- 172.21.15.204:0/2999891150 >> v2:172.21.15.178:3300/0 conn(0x7fe69809fb10 0x7fe69809ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 1 monclient: found mon.noname-h 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.767+0000 7fe696d9d640 1 -- 172.21.15.204:0/2999891150 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6980648a0 con 0x7fe69806e890 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe696d9d640 1 --2- 172.21.15.204:0/2999891150 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe69806e890 0x7fe698132cb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe688009910 tx=0x7fe6880091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=c0aca4a28a2a1499 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:54.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69659c640 1 -- 172.21.15.204:0/2999891150 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe68802acd0 con 0x7fe69806e890 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69659c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69659c640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69659c640 10 monclient: dump: 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:54.781 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69659c640 10 monclient: _finish_auth 0 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69659c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69659c640 1 -- 172.21.15.204:0/2999891150 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68802ae50 con 0x7fe69806e890 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 5 monclient: authenticate success, global_id 6152 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69659c640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:54.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69659c640 1 -- 172.21.15.204:0/2999891150 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe688031ca0 con 0x7fe69806e890 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 4 set_mon_vals no callback set 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69659c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69659c640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69659c640 10 monclient: dump: 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:54.783 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 10 monclient: shutdown 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 1 -- 172.21.15.204:0/2999891150 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe69806e890 msgr2=0x7fe698132cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 1 --2- 172.21.15.204:0/2999891150 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe69806e890 0x7fe698132cb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe688009910 tx=0x7fe6880091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 1 -- 172.21.15.204:0/2999891150 shutdown_connections 2024-09-15T01:36:54.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 1 --2- 172.21.15.204:0/2999891150 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe6980a09c0 0x7fe6980a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 1 --2- 172.21.15.204:0/2999891150 >> v2:172.21.15.178:3300/0 conn(0x7fe69809fb10 0x7fe69809ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 1 --2- 172.21.15.204:0/2999891150 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe69806e890 0x7fe698132cb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.768+0000 7fe69d19d640 1 -- 172.21.15.204:0/2999891150 >> 172.21.15.204:0/2999891150 conn(0x7fe6980a5fd0 msgr2=0x7fe6980a63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.769+0000 7fe69d19d640 1 -- 172.21.15.204:0/2999891150 shutdown_connections 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.769+0000 7fe69d19d640 1 -- 172.21.15.204:0/2999891150 wait complete. 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.769+0000 7fe69d19d640 10 monclient: build_initial_monmap 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.769+0000 7fe69d19d640 10 monclient: monmap: 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:54.785 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.769+0000 7fe69d19d640 1 Processor -- start 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 1 -- start start 2024-09-15T01:36:54.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 10 monclient: init 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 10 monclient: _add_conns ranks=[7,6,5,4,3,1,8,2,0] 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe69806e890 0x7fe69809b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 10 monclient(hunting): picked mon.noname-i con 0x7fe69806e890 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe69809fb10 0x7fe6981319b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 10 monclient(hunting): picked mon.noname-d con 0x7fe69809fb10 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe6980a09c0 0x7fe698131ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 10 monclient(hunting): picked mon.noname-h con 0x7fe6980a09c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe698142a60 con 0x7fe6980a09c0 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe69811edb0 con 0x7fe69809fb10 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:54.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe69809bd20 con 0x7fe69806e890 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69d19d640 10 monclient(hunting): authenticate will time out at 1030.413510s 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe696d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe69806e890 0x7fe69809b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe696d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe69806e890 0x7fe69809b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:60426/0 (socket says 172.21.15.204:60426) 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe6980a09c0 0x7fe698131ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe696d9d640 1 -- 172.21.15.204:0/3509246432 learned_addr learned my addr 172.21.15.204:0/3509246432 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe696d9d640 10 monclient(hunting): get_auth_request con 0x7fe69806e890 auth_method 0 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe696d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe696d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe696d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 10 monclient(hunting): get_auth_request con 0x7fe6980a09c0 auth_method 0 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:54.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe696d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe696d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe696d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 10 monclient(hunting): handle_auth_done global_id 6161 payload 824 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 1 -- 172.21.15.204:0/3509246432 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe69806e890 msgr2=0x7fe69809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 1 --2- 172.21.15.204:0/3509246432 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe69806e890 0x7fe69809b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 1 -- 172.21.15.204:0/3509246432 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe69809fb10 msgr2=0x7fe6981319b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 1 --2- 172.21.15.204:0/3509246432 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe69809fb10 0x7fe6981319b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 10 monclient: _finish_hunting 0 2024-09-15T01:36:54.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 1 monclient: found mon.noname-h 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.770+0000 7fe69759e640 1 -- 172.21.15.204:0/3509246432 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe688009450 con 0x7fe6980a09c0 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe69759e640 1 --2- 172.21.15.204:0/3509246432 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe6980a09c0 0x7fe698131ef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe69000bdf0 tx=0x7fe6900096a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe68f7fe640 1 -- 172.21.15.204:0/3509246432 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6900067e0 con 0x7fe6980a09c0 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe68f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe68f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe68f7fe640 10 monclient: dump: 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:54.790 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe68f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe68f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:54.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe69d19d640 5 monclient: authenticate success, global_id 6161 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe68f7fe640 1 -- 172.21.15.204:0/3509246432 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69000f070 con 0x7fe6980a09c0 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe69d19d640 10 monclient: _renew_subs 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe69d19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe69d19d640 1 -- 172.21.15.204:0/3509246432 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69809cc70 con 0x7fe6980a09c0 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe68f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe68f7fe640 1 -- 172.21.15.204:0/3509246432 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6900069e0 con 0x7fe6980a09c0 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe68f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe68f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe68f7fe640 10 monclient: dump: 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:54.792 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:54.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe695d9b640 4 set_mon_vals no callback set 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe69d19d640 10 monclient: _renew_subs 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe69d19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.771+0000 7fe69d19d640 1 -- 172.21.15.204:0/3509246432 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe698142060 con 0x7fe6980a09c0 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.772+0000 7fe68f7fe640 1 -- 172.21.15.204:0/3509246432 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe690006360 con 0x7fe6980a09c0 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.772+0000 7fe69d19d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.772+0000 7fe69d19d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.772+0000 7fe69d19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.772+0000 7fe69d19d640 1 -- 172.21.15.204:0/3509246432 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe698133370 con 0x7fe6980a09c0 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.772+0000 7fe68f7fe640 1 --2- 172.21.15.204:0/3509246432 >> v2:172.21.15.204:6812/28210 conn(0x7fe6780396e0 0x7fe67803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.772+0000 7fe68f7fe640 1 -- 172.21.15.204:0/3509246432 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 7373+0+0 (secure 0 0 0) 0x7fe69004ca20 con 0x7fe6980a09c0 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.772+0000 7fe696d9d640 1 --2- 172.21.15.204:0/3509246432 >> v2:172.21.15.204:6812/28210 conn(0x7fe6780396e0 0x7fe67803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.772+0000 7fe696d9d640 10 monclient: get_auth_request con 0x7fe6780396e0 auth_method 0 2024-09-15T01:36:54.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.772+0000 7fe696d9d640 1 --2- 172.21.15.204:0/3509246432 >> v2:172.21.15.204:6812/28210 conn(0x7fe6780396e0 0x7fe67803bb90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe68802bfd0 tx=0x7fe688032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:54.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.776+0000 7fe68f7fe640 1 -- 172.21.15.204:0/3509246432 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe690006610 con 0x7fe6980a09c0 2024-09-15T01:36:54.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.776+0000 7fe68f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:54.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.776+0000 7fe68f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:54.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.937+0000 7fe69d19d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:54.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.937+0000 7fe69d19d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:54.938 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.937+0000 7fe69d19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:54.939 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:54.937+0000 7fe69d19d640 1 -- 172.21.15.204:0/3509246432 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fe698067540 con 0x7fe6980a09c0 2024-09-15T01:36:55.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.588+0000 7fe68f7fe640 1 -- 172.21.15.204:0/3509246432 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}]=0 added bucket h3 type host to location {rack=r1} v187) v1 ==== 166+0+0 (secure 0 0 0) 0x7fe690012330 con 0x7fe6980a09c0 2024-09-15T01:36:55.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.588+0000 7fe68f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:55.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.588+0000 7fe68f7fe640 10 monclient: _finish_command 2 = system:0 added bucket h3 type host to location {rack=r1} 2024-09-15T01:36:55.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.685+0000 7fe69d19d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:55.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.685+0000 7fe69d19d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:55.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.685+0000 7fe69d19d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:55.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.685+0000 7fe69d19d640 1 -- 172.21.15.204:0/3509246432 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fe698070fc0 con 0x7fe6980a09c0 2024-09-15T01:36:55.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.686+0000 7fe68f7fe640 1 -- 172.21.15.204:0/3509246432 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h3' already exists v187) v1 ==== 145+0+0 (secure 0 0 0) 0x7fe6900112e0 con 0x7fe6980a09c0 2024-09-15T01:36:55.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.686+0000 7fe68f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:55.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.686+0000 7fe68f7fe640 10 monclient: _finish_command 3 = system:0 bucket 'h3' already exists 2024-09-15T01:36:55.688 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h3' already exists 2024-09-15T01:36:55.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.687+0000 7fe69d19d640 1 -- 172.21.15.204:0/3509246432 >> v2:172.21.15.204:6812/28210 conn(0x7fe6780396e0 msgr2=0x7fe67803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:55.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.687+0000 7fe69d19d640 1 --2- 172.21.15.204:0/3509246432 >> v2:172.21.15.204:6812/28210 conn(0x7fe6780396e0 0x7fe67803bb90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe68802bfd0 tx=0x7fe688032000 comp rx=0 tx=0).stop 2024-09-15T01:36:55.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.688+0000 7fe69d19d640 10 monclient: shutdown 2024-09-15T01:36:55.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.688+0000 7fe69d19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:55.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.688+0000 7fe69d19d640 1 -- 172.21.15.204:0/3509246432 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe6980a09c0 msgr2=0x7fe698131ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:55.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.688+0000 7fe69d19d640 1 --2- 172.21.15.204:0/3509246432 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe6980a09c0 0x7fe698131ef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe69000bdf0 tx=0x7fe6900096a0 comp rx=0 tx=0).stop 2024-09-15T01:36:55.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.688+0000 7fe69d19d640 1 -- 172.21.15.204:0/3509246432 shutdown_connections 2024-09-15T01:36:55.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.688+0000 7fe69d19d640 1 --2- 172.21.15.204:0/3509246432 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe6980a09c0 0x7fe698131ef0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:55.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.688+0000 7fe69d19d640 1 --2- 172.21.15.204:0/3509246432 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe69809fb10 0x7fe6981319b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:55.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.688+0000 7fe69d19d640 1 --2- 172.21.15.204:0/3509246432 >> v2:172.21.15.204:6812/28210 conn(0x7fe6780396e0 0x7fe67803bb90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:55.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.688+0000 7fe69d19d640 1 --2- 172.21.15.204:0/3509246432 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe69806e890 0x7fe69809b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:55.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.688+0000 7fe69d19d640 1 -- 172.21.15.204:0/3509246432 >> 172.21.15.204:0/3509246432 conn(0x7fe6980a5fd0 msgr2=0x7fe698135750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:55.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.688+0000 7fe69d19d640 1 -- 172.21.15.204:0/3509246432 shutdown_connections 2024-09-15T01:36:55.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.688+0000 7fe69d19d640 1 -- 172.21.15.204:0/3509246432 wait complete. 2024-09-15T01:36:55.698 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:36:55.698 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h4 host rack=r1 2024-09-15T01:36:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.772+0000 7f017499d640 10 monclient: get_monmap_and_config 2024-09-15T01:36:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.772+0000 7f017499d640 10 monclient: build_initial_monmap 2024-09-15T01:36:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.772+0000 7f017499d640 10 monclient: monmap: 2024-09-15T01:36:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:55.773186+0000 2024-09-15T01:36:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:55.773186+0000 2024-09-15T01:36:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:55.775 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 1 Processor -- start 2024-09-15T01:36:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 1 -- start start 2024-09-15T01:36:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 10 monclient: init 2024-09-15T01:36:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 10 monclient: _add_conns ranks=[10,5,3,4,0,8,6,7,9,2,1] 2024-09-15T01:36:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f01700a2cd0 0x7f0170132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 10 monclient(hunting): picked mon.noname-f con 0x7f01700a2cd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f017009fa20 0x7f017009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 10 monclient(hunting): picked mon.noname-h con 0x7f017009fa20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01700a08e0 0x7f01700a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 10 monclient(hunting): picked mon.noname-g con 0x7f01700a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01700a30a0 con 0x7f01700a2cd0 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01700a0330 con 0x7f01700a08e0 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01700a1db0 con 0x7f017009fa20 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f017499d640 10 monclient(hunting): authenticate will time out at 1031.415837s 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f016e59c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f01700a2cd0 0x7f0170132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f016ed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01700a08e0 0x7f01700a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f016e59c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f01700a2cd0 0x7f0170132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:50658/0 (socket says 172.21.15.204:50658) 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f016ed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01700a08e0 0x7f01700a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:43990/0 (socket says 172.21.15.204:43990) 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f016e59c640 1 -- 172.21.15.204:0/3417148829 learned_addr learned my addr 172.21.15.204:0/3417148829 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.773+0000 7f016e59c640 10 monclient(hunting): get_auth_request con 0x7f01700a2cd0 auth_method 0 2024-09-15T01:36:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016ed9d640 10 monclient(hunting): get_auth_request con 0x7f01700a08e0 auth_method 0 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f0165d9b640 1 --2- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f017009fa20 0x7f017009fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f0165d9b640 10 monclient(hunting): get_auth_request con 0x7f017009fa20 auth_method 0 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f0165d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f0165d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f0165d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f0165d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f0165d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f0165d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 10 monclient(hunting): handle_auth_done global_id 6895 payload 274 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 1 -- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f017009fa20 msgr2=0x7f017009fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 1 --2- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f017009fa20 0x7f017009fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 1 -- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01700a08e0 msgr2=0x7f01700a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 1 --2- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01700a08e0 0x7f01700a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:55.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:36:55.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 1 monclient: found mon.noname-f 2024-09-15T01:36:55.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:55.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:55.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 1 -- 172.21.15.204:0/3417148829 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01700648a0 con 0x7f01700a2cd0 2024-09-15T01:36:55.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016ed9d640 1 --2- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01700a08e0 0x7f01700a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:55.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f0165d9b640 1 --2- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f017009fa20 0x7f017009fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:55.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.774+0000 7f016e59c640 1 --2- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01700a2cd0 0x7f0170132c00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0160009910 tx=0x7f01600091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc9015e03cdc0999 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:55.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f016dd9b640 1 -- 172.21.15.204:0/3417148829 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f016002ac90 con 0x7f01700a2cd0 2024-09-15T01:36:55.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f016dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:55.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f016dd9b640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:55.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f016dd9b640 10 monclient: dump: 2024-09-15T01:36:55.791 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:55.791 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:55.791 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:55.791 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:55.791 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:55.791 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:55.791 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:55.791 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:55.791 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:55.791 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:55.791 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f016dd9b640 10 monclient: _finish_auth 0 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f016dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f016dd9b640 1 -- 172.21.15.204:0/3417148829 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016002ae10 con 0x7f01700a2cd0 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 5 monclient: authenticate success, global_id 6895 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f016dd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:55.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f016dd9b640 1 -- 172.21.15.204:0/3417148829 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0160031ca0 con 0x7f01700a2cd0 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f016dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 4 set_mon_vals no callback set 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f016dd9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f016dd9b640 10 monclient: dump: 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:55.793 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 10 monclient: shutdown 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 1 -- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01700a2cd0 msgr2=0x7f0170132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 1 --2- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01700a2cd0 0x7f0170132c00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0160009910 tx=0x7f01600091d0 comp rx=0 tx=0).stop 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 1 -- 172.21.15.204:0/3417148829 shutdown_connections 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 1 --2- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01700a08e0 0x7f01700a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:55.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 1 --2- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f017009fa20 0x7f017009fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 1 --2- 172.21.15.204:0/3417148829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01700a2cd0 0x7f0170132c00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 1 -- 172.21.15.204:0/3417148829 >> 172.21.15.204:0/3417148829 conn(0x7f017006e860 msgr2=0x7f017006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 1 -- 172.21.15.204:0/3417148829 shutdown_connections 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.775+0000 7f017499d640 1 -- 172.21.15.204:0/3417148829 wait complete. 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.776+0000 7f017499d640 10 monclient: build_initial_monmap 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.776+0000 7f017499d640 10 monclient: monmap: 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:55.795 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:55.796 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:55.796 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:55.796 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:55.796 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:55.796 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:55.796 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:55.796 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:55.796 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:55.796 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:55.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.776+0000 7f017499d640 1 Processor -- start 2024-09-15T01:36:55.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.776+0000 7f017499d640 1 -- start start 2024-09-15T01:36:55.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.776+0000 7f017499d640 10 monclient: init 2024-09-15T01:36:55.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:55.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 10 monclient: _add_conns ranks=[4,2,0,7,5,1,6,8,3] 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f017009fa20 0x7f0170141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 10 monclient(hunting): picked mon.noname-c con 0x7f017009fa20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700a08e0 0x7f0170142410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 10 monclient(hunting): picked mon.noname-b con 0x7f01700a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01700a2cd0 0x7f017011d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 10 monclient(hunting): picked mon.noname-a con 0x7f01700a2cd0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017009d3d0 con 0x7f01700a2cd0 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01701207f0 con 0x7f01700a08e0 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0170120d20 con 0x7f017009fa20 2024-09-15T01:36:55.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:55.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f017499d640 10 monclient(hunting): authenticate will time out at 1031.419139s 2024-09-15T01:36:55.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f0165d9b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700a08e0 msgr2=0x7f0170142410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:55.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f016e59c640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f017009fa20 msgr2=0x7f0170141ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:55.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f016ed9d640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01700a2cd0 msgr2=0x7f017011d140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:55.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f016ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01700a2cd0 0x7f017011d140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:55.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f0165d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700a08e0 0x7f0170142410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:55.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.777+0000 7f016e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f017009fa20 0x7f0170141ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:55.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.978+0000 7f016e59c640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f017009fa20 msgr2=0x7f0170141ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:55.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.978+0000 7f0165d9b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700a08e0 msgr2=0x7f0170142410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:55.981 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.978+0000 7f016ed9d640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01700a2cd0 msgr2=0x7f017011d140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:55.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.978+0000 7f016e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f017009fa20 0x7f0170141ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:55.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.978+0000 7f0165d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700a08e0 0x7f0170142410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:55.982 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:55.978+0000 7f016ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01700a2cd0 0x7f017011d140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:36:56.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:56.379+0000 7f016e59c640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f017009fa20 msgr2=0x7f0170141ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:56.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:56.379+0000 7f0165d9b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700a08e0 msgr2=0x7f0170142410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:56.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:56.379+0000 7f016ed9d640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01700a2cd0 msgr2=0x7f017011d140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:56.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:56.379+0000 7f016e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f017009fa20 0x7f0170141ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:56.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:56.379+0000 7f0165d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700a08e0 0x7f0170142410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:56.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:56.379+0000 7f016ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01700a2cd0 0x7f017011d140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:36:57.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:57.180+0000 7f016e59c640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f017009fa20 msgr2=0x7f0170141ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:57.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:57.180+0000 7f0165d9b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700a08e0 msgr2=0x7f0170142410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:36:57.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:57.180+0000 7f016ed9d640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01700a2cd0 msgr2=0x7f017011d140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:57.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:57.180+0000 7f016e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f017009fa20 0x7f0170141ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:57.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:57.180+0000 7f016ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01700a2cd0 0x7f017011d140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:57.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:57.180+0000 7f0165d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700a08e0 0x7f0170142410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:36:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 10 monclient(hunting): tick 2024-09-15T01:36:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 monclient(hunting): continuing hunt 2024-09-15T01:36:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:36:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f017009fa20 msgr2=0x7f0170141ed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f017009fa20 0x7f0170141ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700a08e0 msgr2=0x7f0170142410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700a08e0 0x7f0170142410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01700a2cd0 msgr2=0x7f017011d140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01700a2cd0 0x7f017011d140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:58.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 10 monclient: _add_conns ranks=[8,3,1,5,7,6] 2024-09-15T01:36:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0154003780 0x7f0154005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 10 monclient(hunting): picked mon.noname-e con 0x7f0154003780 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:36:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0154006180 0x7f0154008600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 10 monclient(hunting): picked mon.noname-g con 0x7f0154006180 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0154008ca0 0x7f015400b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 10 monclient(hunting): picked mon.noname-f con 0x7f0154008ca0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:36:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017009d3d0 con 0x7f0154008ca0 2024-09-15T01:36:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01701207f0 con 0x7f0154006180 2024-09-15T01:36:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:58.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0170120d20 con 0x7f0154003780 2024-09-15T01:36:58.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016559a640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:58.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f0165d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0154006180 0x7f0154008600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:58.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0154008ca0 0x7f015400b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:58.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016e59c640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0154003780 msgr2=0x7f0154005ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:36:58.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0154008ca0 0x7f015400b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44784/0 (socket says 172.21.15.204:44784) 2024-09-15T01:36:58.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016e59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0154003780 0x7f0154005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:58.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f0165d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0154006180 0x7f0154008600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:55292/0 (socket says 172.21.15.204:55292) 2024-09-15T01:36:58.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016ed9d640 1 -- 172.21.15.204:0/1078931081 learned_addr learned my addr 172.21.15.204:0/1078931081 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:58.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016ed9d640 10 monclient(hunting): get_auth_request con 0x7f0154008ca0 auth_method 0 2024-09-15T01:36:58.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:58.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:58.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f0165d9b640 10 monclient(hunting): get_auth_request con 0x7f0154006180 auth_method 0 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f0165d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f0165d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f0165d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f016ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f0165d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f0165d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.777+0000 7f0165d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0165d9b640 10 monclient(hunting): handle_auth_done global_id 15339 payload 824 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0165d9b640 1 -- 172.21.15.204:0/1078931081 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0154003780 msgr2=0x7f0154005ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0165d9b640 1 --2- 172.21.15.204:0/1078931081 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0154003780 0x7f0154005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0165d9b640 1 -- 172.21.15.204:0/1078931081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0154008ca0 msgr2=0x7f015400b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0165d9b640 1 --2- 172.21.15.204:0/1078931081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0154008ca0 0x7f015400b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0165d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:36:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0165d9b640 1 monclient: found mon.noname-g 2024-09-15T01:36:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0165d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0165d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:36:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0165d9b640 1 -- 172.21.15.204:0/1078931081 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0158009390 con 0x7f0154006180 2024-09-15T01:36:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f016ed9d640 1 --2- 172.21.15.204:0/1078931081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0154008ca0 0x7f015400b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f016ed9d640 1 -- 172.21.15.204:0/1078931081 reap_dead start 2024-09-15T01:36:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0165d9b640 1 --2- 172.21.15.204:0/1078931081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0154006180 0x7f0154008600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f01580095c0 tx=0x7f01580090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0166ffd640 1 -- 172.21.15.204:0/1078931081 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f015800ede0 con 0x7f0154006180 2024-09-15T01:36:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0166ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0166ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0166ffd640 10 monclient: dump: 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0166ffd640 10 monclient: _finish_auth 0 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0166ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0166ffd640 1 -- 172.21.15.204:0/1078931081 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015800cbc0 con 0x7f0154006180 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f017499d640 5 monclient: authenticate success, global_id 15339 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f017499d640 10 monclient: _renew_subs 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f017499d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f017499d640 1 -- 172.21.15.204:0/1078931081 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0168009a20 con 0x7f0154006180 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0166ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0166ffd640 1 -- 172.21.15.204:0/1078931081 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0158013d70 con 0x7f0154006180 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0166ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0166ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f0166ffd640 10 monclient: dump: 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f017499d640 10 monclient: _renew_subs 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f017499d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f016d59a640 4 set_mon_vals no callback set 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.778+0000 7f017499d640 1 -- 172.21.15.204:0/1078931081 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0160009450 con 0x7f0154006180 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.779+0000 7f0166ffd640 1 -- 172.21.15.204:0/1078931081 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f015800cd30 con 0x7f0154006180 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.779+0000 7f017499d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.779+0000 7f017499d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.779+0000 7f017499d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.779+0000 7f017499d640 1 -- 172.21.15.204:0/1078931081 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0170065b80 con 0x7f0154006180 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.779+0000 7f0166ffd640 1 --2- 172.21.15.204:0/1078931081 >> v2:172.21.15.204:6812/28210 conn(0x7f0150039720 0x7f015003bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:58.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.780+0000 7f0166ffd640 1 -- 172.21.15.204:0/1078931081 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 7583+0+0 (secure 0 0 0) 0x7f015804bf90 con 0x7f0154006180 2024-09-15T01:36:58.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.780+0000 7f016e59c640 1 --2- 172.21.15.204:0/1078931081 >> v2:172.21.15.204:6812/28210 conn(0x7f0150039720 0x7f015003bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:58.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.780+0000 7f016e59c640 10 monclient: get_auth_request con 0x7f0150039720 auth_method 0 2024-09-15T01:36:58.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.780+0000 7f016e59c640 1 --2- 172.21.15.204:0/1078931081 >> v2:172.21.15.204:6812/28210 conn(0x7f0150039720 0x7f015003bbd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0160034000 tx=0x7f0160032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:58.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.785+0000 7f0166ffd640 1 -- 172.21.15.204:0/1078931081 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f015807de40 con 0x7f0154006180 2024-09-15T01:36:58.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.785+0000 7f0166ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:58.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.785+0000 7f0166ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:58.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.946+0000 7f017499d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:58.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.946+0000 7f017499d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:58.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.946+0000 7f017499d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:58.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:58.946+0000 7f017499d640 1 -- 172.21.15.204:0/1078931081 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f017009cb60 con 0x7f0154006180 2024-09-15T01:36:59.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.273+0000 7f0166ffd640 1 -- 172.21.15.204:0/1078931081 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}]=0 added bucket h4 type host to location {rack=r1} v188) v1 ==== 166+0+0 (secure 0 0 0) 0x7f0158013470 con 0x7f0154006180 2024-09-15T01:36:59.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.273+0000 7f0166ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:59.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.273+0000 7f0166ffd640 10 monclient: _finish_command 2 = system:0 added bucket h4 type host to location {rack=r1} 2024-09-15T01:36:59.368 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.367+0000 7f017499d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:59.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.367+0000 7f017499d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:59.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.367+0000 7f017499d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:36:59.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.367+0000 7f017499d640 1 -- 172.21.15.204:0/1078931081 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f0170070b80 con 0x7f0154006180 2024-09-15T01:36:59.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.368+0000 7f0166ffd640 1 -- 172.21.15.204:0/1078931081 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h4' already exists v188) v1 ==== 145+0+0 (secure 0 0 0) 0x7f0158005970 con 0x7f0154006180 2024-09-15T01:36:59.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.368+0000 7f0166ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:59.369 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.368+0000 7f0166ffd640 10 monclient: _finish_command 3 = system:0 bucket 'h4' already exists 2024-09-15T01:36:59.369 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h4' already exists 2024-09-15T01:36:59.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.369+0000 7f017499d640 1 -- 172.21.15.204:0/1078931081 >> v2:172.21.15.204:6812/28210 conn(0x7f0150039720 msgr2=0x7f015003bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:59.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.369+0000 7f017499d640 1 --2- 172.21.15.204:0/1078931081 >> v2:172.21.15.204:6812/28210 conn(0x7f0150039720 0x7f015003bbd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0160034000 tx=0x7f0160032000 comp rx=0 tx=0).stop 2024-09-15T01:36:59.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.369+0000 7f017499d640 10 monclient: shutdown 2024-09-15T01:36:59.371 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.369+0000 7f017499d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:59.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.369+0000 7f017499d640 1 -- 172.21.15.204:0/1078931081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0154006180 msgr2=0x7f0154008600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:59.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.369+0000 7f017499d640 1 --2- 172.21.15.204:0/1078931081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0154006180 0x7f0154008600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f01580095c0 tx=0x7f01580090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:59.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.369+0000 7f017499d640 1 -- 172.21.15.204:0/1078931081 shutdown_connections 2024-09-15T01:36:59.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.369+0000 7f017499d640 1 --2- 172.21.15.204:0/1078931081 >> v2:172.21.15.204:6812/28210 conn(0x7f0150039720 0x7f015003bbd0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:59.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.369+0000 7f017499d640 1 --2- 172.21.15.204:0/1078931081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0154006180 0x7f0154008600 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:59.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.369+0000 7f017499d640 1 -- 172.21.15.204:0/1078931081 >> 172.21.15.204:0/1078931081 conn(0x7f017006e860 msgr2=0x7f01700a4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:59.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.370+0000 7f017499d640 1 -- 172.21.15.204:0/1078931081 shutdown_connections 2024-09-15T01:36:59.372 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.370+0000 7f017499d640 1 -- 172.21.15.204:0/1078931081 wait complete. 2024-09-15T01:36:59.380 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:36:59.380 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h5 host rack=r1 2024-09-15T01:36:59.455 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.454+0000 7f755424f640 10 monclient: get_monmap_and_config 2024-09-15T01:36:59.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.454+0000 7f755424f640 10 monclient: build_initial_monmap 2024-09-15T01:36:59.456 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.454+0000 7f755424f640 10 monclient: monmap: 2024-09-15T01:36:59.456 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:59.456 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:59.456 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:36:59.454633+0000 2024-09-15T01:36:59.456 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:36:59.454633+0000 2024-09-15T01:36:59.456 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:59.456 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:59.456 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:36:59.456 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:36:59.456 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:59.456 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:59.457 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:59.457 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:59.457 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:59.457 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:59.457 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:59.457 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:36:59.457 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:36:59.457 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:59.457 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.454+0000 7f755424f640 1 Processor -- start 2024-09-15T01:36:59.465 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.454+0000 7f755424f640 1 -- start start 2024-09-15T01:36:59.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.454+0000 7f755424f640 10 monclient: init 2024-09-15T01:36:59.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:59.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 10 monclient: _add_conns ranks=[10,4,1,7,3,6,9,8,5,2,0] 2024-09-15T01:36:59.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f754c0a4590 0x7f754c09e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:59.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 10 monclient(hunting): picked mon.noname-f con 0x7f754c0a4590 addr v2:172.21.15.204:3300/0 2024-09-15T01:36:59.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f754c06edd0 0x7f754c09ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:59.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 10 monclient(hunting): picked mon.noname-c con 0x7f754c06edd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:36:59.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f754c09f1d0 addr v1:172.21.15.204:6789/0 2024-09-15T01:36:59.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c0648a0 con 0x7f754c09f1d0 2024-09-15T01:36:59.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:59.466 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754c0a4960 con 0x7f754c0a4590 2024-09-15T01:36:59.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:59.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f754c06dba0 con 0x7f754c06edd0 2024-09-15T01:36:59.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:59.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f755424f640 10 monclient(hunting): authenticate will time out at 1035.097501s 2024-09-15T01:36:59.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f75517ea640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f754c06edd0 msgr2=0x7f754c09ec90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:36:59.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f75517ea640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f754c06edd0 0x7f754c09ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:59.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f754c0a4590 0x7f754c09e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:59.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f75527ec640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f754c09f1d0 0x7f754c1307a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:59912/0 (socket says 172.21.15.204:59912) 2024-09-15T01:36:59.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f75527ec640 1 -- 172.21.15.204:0/157293371 learned_addr learned my addr 172.21.15.204:0/157293371 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:36:59.467 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 10 monclient(hunting): get_auth_request con 0x7f754c0a4590 auth_method 0 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 10 monclient(hunting): handle_auth_done global_id 6913 payload 274 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 1 -- 172.21.15.204:0/157293371 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f754c06edd0 msgr2=0x7f754c09ec90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 1 --2- 172.21.15.204:0/157293371 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f754c06edd0 0x7f754c09ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 1 -- 172.21.15.204:0/157293371 >> v1:172.21.15.204:6789/0 conn(0x7f754c09f1d0 legacy=0x7f754c1307a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 10 monclient: _finish_hunting 0 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 1 monclient: found mon.noname-f 2024-09-15T01:36:59.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7551feb640 1 -- 172.21.15.204:0/157293371 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0a04c0 con 0x7f754c0a4590 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7550fe9640 1 -- 172.21.15.204:0/157293371 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622274378 0 0) 0x7f754c0648a0 con 0x7f754c09f1d0 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.455+0000 7f7550fe9640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7551feb640 1 --2- 172.21.15.204:0/157293371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f754c0a4590 0x7f754c09e750 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7544009850 tx=0x7f75440090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c8435a285ab6007 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7550fe9640 1 -- 172.21.15.204:0/157293371 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f754402ac90 con 0x7f754c0a4590 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7550fe9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7550fe9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7550fe9640 10 monclient: dump: 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:59.469 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:59.470 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:59.470 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:59.470 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:59.470 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:59.470 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:59.470 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:59.470 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:59.470 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:59.470 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:59.471 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:59.471 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:59.471 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:59.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7550fe9640 10 monclient: _finish_auth 0 2024-09-15T01:36:59.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7550fe9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:59.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f755424f640 5 monclient: authenticate success, global_id 6913 2024-09-15T01:36:59.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f755424f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:36:59.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7550fe9640 1 -- 172.21.15.204:0/157293371 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754402ae10 con 0x7f754c0a4590 2024-09-15T01:36:59.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7550fe9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:59.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f755424f640 10 monclient: get_monmap_and_config success 2024-09-15T01:36:59.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f755424f640 4 set_mon_vals no callback set 2024-09-15T01:36:59.471 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7550fe9640 1 -- 172.21.15.204:0/157293371 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7544031ca0 con 0x7f754c0a4590 2024-09-15T01:36:59.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7550fe9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:59.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7550fe9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:36:59.472 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.456+0000 7f7550fe9640 10 monclient: dump: 2024-09-15T01:36:59.472 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:59.472 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:59.472 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:59.472 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:59.472 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:59.472 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:59.472 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:59.472 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.457+0000 7f755424f640 10 monclient: shutdown 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.457+0000 7f755424f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.457+0000 7f755424f640 1 -- 172.21.15.204:0/157293371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f754c0a4590 msgr2=0x7f754c09e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.457+0000 7f755424f640 1 --2- 172.21.15.204:0/157293371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f754c0a4590 0x7f754c09e750 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7544009850 tx=0x7f75440090f0 comp rx=0 tx=0).stop 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.457+0000 7f755424f640 1 -- 172.21.15.204:0/157293371 shutdown_connections 2024-09-15T01:36:59.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.457+0000 7f755424f640 1 --2- 172.21.15.204:0/157293371 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f754c06edd0 0x7f754c09ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.457+0000 7f755424f640 1 --2- 172.21.15.204:0/157293371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f754c0a4590 0x7f754c09e750 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.457+0000 7f755424f640 1 -- 172.21.15.204:0/157293371 >> 172.21.15.204:0/157293371 conn(0x7f754c067700 msgr2=0x7f754c0a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.457+0000 7f755424f640 1 -- 172.21.15.204:0/157293371 shutdown_connections 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.457+0000 7f755424f640 1 -- 172.21.15.204:0/157293371 wait complete. 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.457+0000 7f755424f640 10 monclient: build_initial_monmap 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.457+0000 7f755424f640 10 monclient: monmap: 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:36:59.474 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 1 Processor -- start 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 1 -- start start 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 10 monclient: init 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 10 monclient: _reopen_session rank -1 2024-09-15T01:36:59.475 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 10 monclient: _add_conns ranks=[5,3,0,4,1,2,7,8,6] 2024-09-15T01:36:59.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f754c06edd0 0x7f754c13bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:59.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 10 monclient(hunting): picked mon.noname-h con 0x7f754c06edd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:36:59.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754c09f1d0 0x7f754c13b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:59.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 10 monclient(hunting): picked mon.noname-g con 0x7f754c09f1d0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:36:59.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f754c0a4590 0x7f754c13f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:59.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 10 monclient(hunting): picked mon.noname-a con 0x7f754c0a4590 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:36:59.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:59.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c11ecb0 con 0x7f754c0a4590 2024-09-15T01:36:59.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:59.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c06d070 con 0x7f754c09f1d0 2024-09-15T01:36:59.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:36:59.476 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f754c06cc80 con 0x7f754c06edd0 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 10 monclient(hunting): _renew_subs 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f755424f640 10 monclient(hunting): authenticate will time out at 1035.100820s 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f7551feb640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f754c06edd0 0x7f754c13bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f75517ea640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754c09f1d0 0x7f754c13b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f7551feb640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f754c06edd0 0x7f754c13bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:50010/0 (socket says 172.21.15.204:50010) 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f7551feb640 1 -- 172.21.15.204:0/1540915459 learned_addr learned my addr 172.21.15.204:0/1540915459 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f7551feb640 10 monclient(hunting): get_auth_request con 0x7f754c06edd0 auth_method 0 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f7551feb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f7551feb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f7551feb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f75517ea640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754c09f1d0 0x7f754c13b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:55296/0 (socket says 172.21.15.204:55296) 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f75527ec640 1 -- 172.21.15.204:0/1540915459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f754c0a4590 msgr2=0x7f754c13f510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:36:59.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f75517ea640 10 monclient(hunting): get_auth_request con 0x7f754c09f1d0 auth_method 0 2024-09-15T01:36:59.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f75517ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:36:59.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f75517ea640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:36:59.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f75517ea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:36:59.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.458+0000 7f75527ec640 1 --2- 172.21.15.204:0/1540915459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f754c0a4590 0x7f754c13f510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:36:59.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:59.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:59.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:59.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f75517ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:36:59.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f75517ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:36:59.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f75517ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 10 monclient(hunting): handle_auth_done global_id 6179 payload 824 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 1 -- 172.21.15.204:0/1540915459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754c09f1d0 msgr2=0x7f754c13b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 1 --2- 172.21.15.204:0/1540915459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754c09f1d0 0x7f754c13b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 1 -- 172.21.15.204:0/1540915459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f754c0a4590 msgr2=0x7f754c13f510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 1 --2- 172.21.15.204:0/1540915459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f754c0a4590 0x7f754c13f510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 10 monclient: _finish_hunting 0 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 1 monclient: found mon.noname-h 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 1 -- 172.21.15.204:0/1540915459 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7544009390 con 0x7f754c06edd0 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f75517ea640 1 --2- 172.21.15.204:0/1540915459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754c09f1d0 0x7f754c13b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7551feb640 1 --2- 172.21.15.204:0/1540915459 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f754c06edd0 0x7f754c13bfc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7544035040 tx=0x7f7544032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:59.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7549ffb640 1 -- 172.21.15.204:0/1540915459 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7544031c00 con 0x7f754c06edd0 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7549ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7549ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7549ffb640 10 monclient: dump: 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:59.480 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7549ffb640 10 monclient: _finish_auth 0 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7549ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f755424f640 5 monclient: authenticate success, global_id 6179 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7549ffb640 1 -- 172.21.15.204:0/1540915459 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7544031d60 con 0x7f754c06edd0 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f755424f640 10 monclient: _renew_subs 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f755424f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f755424f640 1 -- 172.21.15.204:0/1540915459 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c1207a0 con 0x7f754c06edd0 2024-09-15T01:36:59.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7549ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:36:59.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7549ffb640 1 -- 172.21.15.204:0/1540915459 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f754403a810 con 0x7f754c06edd0 2024-09-15T01:36:59.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7549ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:36:59.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7549ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:36:59.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f7549ffb640 10 monclient: dump: 2024-09-15T01:36:59.482 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:36:59.482 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:36:59.483 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:59.483 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:36:59.483 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:36:59.483 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:36:59.483 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:36:59.483 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:36:59.483 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:36:59.483 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:36:59.483 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f755424f640 10 monclient: _renew_subs 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f754bfff640 4 set_mon_vals no callback set 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f755424f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.459+0000 7f755424f640 1 -- 172.21.15.204:0/1540915459 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c13cad0 con 0x7f754c06edd0 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.460+0000 7f7549ffb640 1 -- 172.21.15.204:0/1540915459 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7544039040 con 0x7f754c06edd0 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.460+0000 7f755424f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.460+0000 7f755424f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:59.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.460+0000 7f755424f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:59.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.460+0000 7f7549ffb640 1 --2- 172.21.15.204:0/1540915459 >> v2:172.21.15.204:6812/28210 conn(0x7f75380397b0 0x7f753803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:36:59.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.460+0000 7f755424f640 1 -- 172.21.15.204:0/1540915459 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c120df0 con 0x7f754c06edd0 2024-09-15T01:36:59.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.460+0000 7f7549ffb640 1 -- 172.21.15.204:0/1540915459 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 7793+0+0 (secure 0 0 0) 0x7f754406ba00 con 0x7f754c06edd0 2024-09-15T01:36:59.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.460+0000 7f75517ea640 1 --2- 172.21.15.204:0/1540915459 >> v2:172.21.15.204:6812/28210 conn(0x7f75380397b0 0x7f753803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:36:59.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.460+0000 7f75517ea640 10 monclient: get_auth_request con 0x7f75380397b0 auth_method 0 2024-09-15T01:36:59.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.461+0000 7f75517ea640 1 --2- 172.21.15.204:0/1540915459 >> v2:172.21.15.204:6812/28210 conn(0x7f75380397b0 0x7f753803bc60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f754c11d120 tx=0x7f75400090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:36:59.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.465+0000 7f7549ffb640 1 -- 172.21.15.204:0/1540915459 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7544037020 con 0x7f754c06edd0 2024-09-15T01:36:59.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.465+0000 7f7549ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:36:59.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.465+0000 7f7549ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:36:59.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.622+0000 7f755424f640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:59.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.622+0000 7f755424f640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:36:59.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.622+0000 7f755424f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:36:59.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:36:59.622+0000 7f755424f640 1 -- 172.21.15.204:0/1540915459 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f754c0665e0 con 0x7f754c06edd0 2024-09-15T01:37:00.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.281+0000 7f7549ffb640 1 -- 172.21.15.204:0/1540915459 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}]=0 added bucket h5 type host to location {rack=r1} v189) v1 ==== 166+0+0 (secure 0 0 0) 0x7f7544041070 con 0x7f754c06edd0 2024-09-15T01:37:00.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.282+0000 7f7549ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:00.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.282+0000 7f7549ffb640 10 monclient: _finish_command 2 = system:0 added bucket h5 type host to location {rack=r1} 2024-09-15T01:37:00.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.379+0000 7f755424f640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:00.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.379+0000 7f755424f640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:00.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.379+0000 7f755424f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:00.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.379+0000 7f755424f640 1 -- 172.21.15.204:0/1540915459 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f754c070b80 con 0x7f754c06edd0 2024-09-15T01:37:00.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.380+0000 7f7549ffb640 1 -- 172.21.15.204:0/1540915459 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h5' already exists v189) v1 ==== 145+0+0 (secure 0 0 0) 0x7f754403a990 con 0x7f754c06edd0 2024-09-15T01:37:00.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.380+0000 7f7549ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:00.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.380+0000 7f7549ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h5' already exists 2024-09-15T01:37:00.381 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h5' already exists 2024-09-15T01:37:00.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 1 -- 172.21.15.204:0/1540915459 >> v2:172.21.15.204:6812/28210 conn(0x7f75380397b0 msgr2=0x7f753803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:00.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 1 --2- 172.21.15.204:0/1540915459 >> v2:172.21.15.204:6812/28210 conn(0x7f75380397b0 0x7f753803bc60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f754c11d120 tx=0x7f75400090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 10 monclient: shutdown 2024-09-15T01:37:00.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:00.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 1 -- 172.21.15.204:0/1540915459 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f754c06edd0 msgr2=0x7f754c13bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:00.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 1 --2- 172.21.15.204:0/1540915459 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f754c06edd0 0x7f754c13bfc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7544035040 tx=0x7f7544032000 comp rx=0 tx=0).stop 2024-09-15T01:37:00.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 1 -- 172.21.15.204:0/1540915459 shutdown_connections 2024-09-15T01:37:00.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 1 --2- 172.21.15.204:0/1540915459 >> v2:172.21.15.204:6812/28210 conn(0x7f75380397b0 0x7f753803bc60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 1 --2- 172.21.15.204:0/1540915459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f754c0a4590 0x7f754c13f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 1 --2- 172.21.15.204:0/1540915459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754c09f1d0 0x7f754c13b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 1 --2- 172.21.15.204:0/1540915459 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f754c06edd0 0x7f754c13bfc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 1 -- 172.21.15.204:0/1540915459 >> 172.21.15.204:0/1540915459 conn(0x7f754c067700 msgr2=0x7f754c0a29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:00.384 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 1 -- 172.21.15.204:0/1540915459 shutdown_connections 2024-09-15T01:37:00.385 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.382+0000 7f755424f640 1 -- 172.21.15.204:0/1540915459 wait complete. 2024-09-15T01:37:00.392 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:00.392 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h6 host rack=r1 2024-09-15T01:37:00.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.467+0000 7f9ffa39d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:00.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.467+0000 7f9ffa39d640 10 monclient: build_initial_monmap 2024-09-15T01:37:00.468 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.467+0000 7f9ffa39d640 10 monclient: monmap: 2024-09-15T01:37:00.468 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:00.467559+0000 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:00.467559+0000 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:00.469 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:00.470 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:00.470 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:00.470 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:00.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.467+0000 7f9ffa39d640 1 Processor -- start 2024-09-15T01:37:00.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.467+0000 7f9ffa39d640 1 -- start start 2024-09-15T01:37:00.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.467+0000 7f9ffa39d640 10 monclient: init 2024-09-15T01:37:00.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:00.478 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 10 monclient: _add_conns ranks=[8,7,6,4,0,3,10,1,9,5,2] 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9ff409f760 0x7f9ff4131a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 10 monclient(hunting): picked mon.noname-e con 0x7f9ff409f760 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 0x7f9ff4132340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 10 monclient(hunting): picked mon.noname-i con 0x7f9ff4131f70 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff4130920 0x7f9ff4130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 10 monclient(hunting): picked mon.noname-d con 0x7f9ff4130920 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4132880 con 0x7f9ff4130920 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4069360 con 0x7f9ff4131f70 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff406ed90 con 0x7f9ff409f760 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:00.479 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ffa39d640 10 monclient(hunting): authenticate will time out at 1036.110471s 2024-09-15T01:37:00.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff3fff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9ff409f760 msgr2=0x7f9ff4131a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:00.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 0x7f9ff4132340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:00.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9ff409f760 0x7f9ff4131a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:00.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 0x7f9ff4132340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:55816/0 (socket says 172.21.15.204:55816) 2024-09-15T01:37:00.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 1 -- 172.21.15.204:0/317745371 learned_addr learned my addr 172.21.15.204:0/317745371 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:00.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 10 monclient(hunting): get_auth_request con 0x7f9ff4131f70 auth_method 0 2024-09-15T01:37:00.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:00.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:00.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:00.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:00.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:00.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:00.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff893a640 1 --2- 172.21.15.204:0/317745371 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff4130920 0x7f9ff4130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:00.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 10 monclient(hunting): handle_auth_done global_id 24442 payload 274 2024-09-15T01:37:00.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 1 -- 172.21.15.204:0/317745371 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9ff409f760 msgr2=0x7f9ff4131a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:00.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 1 --2- 172.21.15.204:0/317745371 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9ff409f760 0x7f9ff4131a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 1 -- 172.21.15.204:0/317745371 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff4130920 msgr2=0x7f9ff4130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:00.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 1 --2- 172.21.15.204:0/317745371 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff4130920 0x7f9ff4130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:00.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 1 monclient: found mon.noname-i 2024-09-15T01:37:00.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:00.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:00.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff37fe640 1 -- 172.21.15.204:0/317745371 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff40648a0 con 0x7f9ff4131f70 2024-09-15T01:37:00.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff893a640 10 monclient: get_auth_request con 0x7f9ff4130920 auth_method 0 2024-09-15T01:37:00.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.468+0000 7f9ff893a640 1 --2- 172.21.15.204:0/317745371 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff4130920 0x7f9ff4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:37:00.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff37fe640 1 --2- 172.21.15.204:0/317745371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 0x7f9ff4132340 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009940 tx=0x7f9fe80091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=635745b05fe4e304 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:00.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff2ffd640 1 -- 172.21.15.204:0/317745371 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9fe802ac80 con 0x7f9ff4131f70 2024-09-15T01:37:00.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:00.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff2ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:00.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff2ffd640 10 monclient: dump: 2024-09-15T01:37:00.482 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:00.482 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:00.482 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:00.482 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:00.483 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:00.483 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:00.483 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:00.483 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:00.483 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:00.483 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:00.483 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:00.483 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:00.483 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:00.483 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ffa39d640 5 monclient: authenticate success, global_id 24442 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff2ffd640 1 -- 172.21.15.204:0/317745371 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe802ae00 con 0x7f9ff4131f70 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff2ffd640 1 -- 172.21.15.204:0/317745371 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9fe8031ca0 con 0x7f9ff4131f70 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ffa39d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ffa39d640 4 set_mon_vals no callback set 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:00.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.469+0000 7f9ff2ffd640 10 monclient: dump: 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:00.485 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 10 monclient: shutdown 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 1 -- 172.21.15.204:0/317745371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 msgr2=0x7f9ff4132340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 1 --2- 172.21.15.204:0/317745371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 0x7f9ff4132340 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009940 tx=0x7f9fe80091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 1 -- 172.21.15.204:0/317745371 shutdown_connections 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 1 --2- 172.21.15.204:0/317745371 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff4130920 0x7f9ff4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 1 --2- 172.21.15.204:0/317745371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 0x7f9ff4132340 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 1 --2- 172.21.15.204:0/317745371 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9ff409f760 0x7f9ff4131a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 1 -- 172.21.15.204:0/317745371 >> 172.21.15.204:0/317745371 conn(0x7f9ff4067c00 msgr2=0x7f9ff40a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 1 -- 172.21.15.204:0/317745371 shutdown_connections 2024-09-15T01:37:00.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 1 -- 172.21.15.204:0/317745371 wait complete. 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 10 monclient: build_initial_monmap 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.470+0000 7f9ffa39d640 10 monclient: monmap: 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:00.487 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 1 Processor -- start 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 1 -- start start 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 10 monclient: init 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 10 monclient: _add_conns ranks=[4,5,7,8,2,0,3,6,1] 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9ff409f760 0x7f9ff4141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 10 monclient(hunting): picked mon.noname-c con 0x7f9ff409f760 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9ff4130920 0x7f9ff41422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:00.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 10 monclient(hunting): picked mon.noname-h con 0x7f9ff4130920 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 0x7f9ff411cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 10 monclient(hunting): picked mon.noname-i con 0x7f9ff4131f70 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4068ad0 con 0x7f9ff409f760 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40688a0 con 0x7f9ff4130920 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4120970 con 0x7f9ff4131f70 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.471+0000 7f9ffa39d640 10 monclient(hunting): authenticate will time out at 1036.113987s 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 0x7f9ff411cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff3fff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9ff409f760 msgr2=0x7f9ff4141da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:00.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 0x7f9ff411cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:55822/0 (socket says 172.21.15.204:55822) 2024-09-15T01:37:00.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff3fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9ff409f760 0x7f9ff4141da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:00.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 1 -- 172.21.15.204:0/4121503715 learned_addr learned my addr 172.21.15.204:0/4121503715 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:00.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 10 monclient(hunting): get_auth_request con 0x7f9ff4131f70 auth_method 0 2024-09-15T01:37:00.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:00.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:00.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:00.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff37fe640 1 --2- 172.21.15.204:0/4121503715 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9ff4130920 0x7f9ff41422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:00.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff37fe640 10 monclient(hunting): get_auth_request con 0x7f9ff4130920 auth_method 0 2024-09-15T01:37:00.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:00.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:00.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:00.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:00.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:00.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:00.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:00.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:00.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:00.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 10 monclient(hunting): handle_auth_done global_id 24451 payload 824 2024-09-15T01:37:00.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 1 -- 172.21.15.204:0/4121503715 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9ff4130920 msgr2=0x7f9ff41422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:00.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 1 --2- 172.21.15.204:0/4121503715 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9ff4130920 0x7f9ff41422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 1 -- 172.21.15.204:0/4121503715 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9ff409f760 msgr2=0x7f9ff4141da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:00.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 1 --2- 172.21.15.204:0/4121503715 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9ff409f760 0x7f9ff4141da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:00.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 10 monclient: _finish_hunting 0 2024-09-15T01:37:00.491 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 1 monclient: found mon.noname-i 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 1 -- 172.21.15.204:0/4121503715 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe8009450 con 0x7f9ff4131f70 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff37fe640 1 --2- 172.21.15.204:0/4121503715 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9ff4130920 0x7f9ff41422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff893a640 1 --2- 172.21.15.204:0/4121503715 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 0x7f9ff411cff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00a3b0 tx=0x7f9fec0078f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9fdbfff640 1 -- 172.21.15.204:0/4121503715 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9fec00dad0 con 0x7f9ff4131f70 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9fdbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9fdbfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9fdbfff640 10 monclient: dump: 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:00.492 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:00.493 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:00.493 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:00.493 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:00.493 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:00.493 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:00.493 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:00.493 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:00.494 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:00.494 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:00.494 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:00.494 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:00.494 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:00.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9fdbfff640 10 monclient: _finish_auth 0 2024-09-15T01:37:00.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9fdbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:00.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ffa39d640 5 monclient: authenticate success, global_id 24451 2024-09-15T01:37:00.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9fdbfff640 1 -- 172.21.15.204:0/4121503715 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec00dc70 con 0x7f9ff4131f70 2024-09-15T01:37:00.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ffa39d640 10 monclient: _renew_subs 2024-09-15T01:37:00.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ffa39d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ffa39d640 1 -- 172.21.15.204:0/4121503715 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff411f820 con 0x7f9ff4131f70 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9fdbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9fdbfff640 1 -- 172.21.15.204:0/4121503715 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9fec00dde0 con 0x7f9ff4131f70 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9fdbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9fdbfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ff27fc640 4 set_mon_vals no callback set 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9fdbfff640 10 monclient: dump: 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:00.495 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ffa39d640 10 monclient: _renew_subs 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ffa39d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.472+0000 7f9ffa39d640 1 -- 172.21.15.204:0/4121503715 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff411ebd0 con 0x7f9ff4131f70 2024-09-15T01:37:00.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.473+0000 7f9fdbfff640 1 -- 172.21.15.204:0/4121503715 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9fec00d650 con 0x7f9ff4131f70 2024-09-15T01:37:00.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.473+0000 7f9fdbfff640 1 --2- 172.21.15.204:0/4121503715 >> v2:172.21.15.204:6812/28210 conn(0x7f9fe00397b0 0x7f9fe003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:00.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.473+0000 7f9ffa39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:00.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.473+0000 7f9ffa39d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:00.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.473+0000 7f9fdbfff640 1 -- 172.21.15.204:0/4121503715 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 8131+0+0 (secure 0 0 0) 0x7f9fec04c680 con 0x7f9ff4131f70 2024-09-15T01:37:00.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.473+0000 7f9ffa39d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:00.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.473+0000 7f9ffa39d640 1 -- 172.21.15.204:0/4121503715 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff4070b80 con 0x7f9ff4131f70 2024-09-15T01:37:00.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.473+0000 7f9ff3fff640 1 --2- 172.21.15.204:0/4121503715 >> v2:172.21.15.204:6812/28210 conn(0x7f9fe00397b0 0x7f9fe003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:00.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.473+0000 7f9ff3fff640 10 monclient: get_auth_request con 0x7f9fe00397b0 auth_method 0 2024-09-15T01:37:00.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.474+0000 7f9ff3fff640 1 --2- 172.21.15.204:0/4121503715 >> v2:172.21.15.204:6812/28210 conn(0x7f9fe00397b0 0x7f9fe003bc60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40038f0 tx=0x7f9fe4006d20 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:00.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.478+0000 7f9fdbfff640 1 -- 172.21.15.204:0/4121503715 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9fec01a070 con 0x7f9ff4131f70 2024-09-15T01:37:00.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.478+0000 7f9fdbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:00.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.478+0000 7f9fdbfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:00.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.635+0000 7f9ffa39d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:00.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.635+0000 7f9ffa39d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:00.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.635+0000 7f9ffa39d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:00.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:00.635+0000 7f9ffa39d640 1 -- 172.21.15.204:0/4121503715 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f9ff4069080 con 0x7f9ff4131f70 2024-09-15T01:37:01.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.287+0000 7f9fdbfff640 1 -- 172.21.15.204:0/4121503715 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}]=0 added bucket h6 type host to location {rack=r1} v190) v1 ==== 166+0+0 (secure 0 0 0) 0x7f9fec00d900 con 0x7f9ff4131f70 2024-09-15T01:37:01.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.287+0000 7f9fdbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:01.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.287+0000 7f9fdbfff640 10 monclient: _finish_command 2 = system:0 added bucket h6 type host to location {rack=r1} 2024-09-15T01:37:01.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.387+0000 7f9ffa39d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:01.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.387+0000 7f9ffa39d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:01.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.387+0000 7f9ffa39d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:01.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.387+0000 7f9ffa39d640 1 -- 172.21.15.204:0/4121503715 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f9ff40008d0 con 0x7f9ff4131f70 2024-09-15T01:37:01.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.387+0000 7f9fdbfff640 1 -- 172.21.15.204:0/4121503715 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h6' already exists v190) v1 ==== 145+0+0 (secure 0 0 0) 0x7f9fec01cac0 con 0x7f9ff4131f70 2024-09-15T01:37:01.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.387+0000 7f9fdbfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:01.389 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.387+0000 7f9fdbfff640 10 monclient: _finish_command 3 = system:0 bucket 'h6' already exists 2024-09-15T01:37:01.389 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h6' already exists 2024-09-15T01:37:01.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 1 -- 172.21.15.204:0/4121503715 >> v2:172.21.15.204:6812/28210 conn(0x7f9fe00397b0 msgr2=0x7f9fe003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:01.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 1 --2- 172.21.15.204:0/4121503715 >> v2:172.21.15.204:6812/28210 conn(0x7f9fe00397b0 0x7f9fe003bc60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40038f0 tx=0x7f9fe4006d20 comp rx=0 tx=0).stop 2024-09-15T01:37:01.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 10 monclient: shutdown 2024-09-15T01:37:01.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:01.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 1 -- 172.21.15.204:0/4121503715 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 msgr2=0x7f9ff411cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:01.391 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 1 --2- 172.21.15.204:0/4121503715 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 0x7f9ff411cff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00a3b0 tx=0x7f9fec0078f0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 1 -- 172.21.15.204:0/4121503715 shutdown_connections 2024-09-15T01:37:01.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 1 --2- 172.21.15.204:0/4121503715 >> v2:172.21.15.204:6812/28210 conn(0x7f9fe00397b0 0x7f9fe003bc60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 1 --2- 172.21.15.204:0/4121503715 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9ff4131f70 0x7f9ff411cff0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 1 --2- 172.21.15.204:0/4121503715 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9ff4130920 0x7f9ff41422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 1 --2- 172.21.15.204:0/4121503715 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9ff409f760 0x7f9ff4141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 1 -- 172.21.15.204:0/4121503715 >> 172.21.15.204:0/4121503715 conn(0x7f9ff4067c00 msgr2=0x7f9ff40a3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:01.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.389+0000 7f9ffa39d640 1 -- 172.21.15.204:0/4121503715 shutdown_connections 2024-09-15T01:37:01.392 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.390+0000 7f9ffa39d640 1 -- 172.21.15.204:0/4121503715 wait complete. 2024-09-15T01:37:01.400 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:01.400 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h7 host rack=r1 2024-09-15T01:37:01.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.480+0000 7fb51fb9d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:01.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.480+0000 7fb51fb9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:01.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.480+0000 7fb51fb9d640 10 monclient: monmap: 2024-09-15T01:37:01.482 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:01.482 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:01.481364+0000 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:01.481364+0000 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:01.483 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:01.484 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:01.484 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:01.484 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:01.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.481+0000 7fb51fb9d640 1 Processor -- start 2024-09-15T01:37:01.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.482+0000 7fb51fb9d640 1 -- start start 2024-09-15T01:37:01.486 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.482+0000 7fb51fb9d640 10 monclient: init 2024-09-15T01:37:01.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:01.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 10 monclient: _add_conns ranks=[7,4,2,0,6,10,3,1,5,9,8] 2024-09-15T01:37:01.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb518067130 0x7fb518097c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:01.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 10 monclient(hunting): picked mon.noname-i con 0x7fb518067130 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:01.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb518067500 0x7fb5180678d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:01.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 10 monclient(hunting): picked mon.noname-c con 0x7fb518067500 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:01.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb518136e80 0x7fb518139260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:01.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 10 monclient(hunting): picked mon.noname-b con 0x7fb518136e80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:01.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:01.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51d138640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb518067500 msgr2=0x7fb5180678d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:01.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51d138640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb518067500 0x7fb5180678d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:01.487 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb518097850 con 0x7fb518136e80 2024-09-15T01:37:01.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:01.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb518096ef0 con 0x7fb518067500 2024-09-15T01:37:01.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:01.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb5180971d0 con 0x7fb518067130 2024-09-15T01:37:01.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:01.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51fb9d640 10 monclient(hunting): authenticate will time out at 1037.125501s 2024-09-15T01:37:01.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51e13a640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb518136e80 msgr2=0x7fb518139260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:01.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51e13a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb518136e80 0x7fb518139260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:01.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51d939640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb518067130 0x7fb518097c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:01.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51d939640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb518067130 0x7fb518097c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:55830/0 (socket says 172.21.15.204:55830) 2024-09-15T01:37:01.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51d939640 1 -- 172.21.15.204:0/668000066 learned_addr learned my addr 172.21.15.204:0/668000066 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:01.488 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51d939640 10 monclient(hunting): get_auth_request con 0x7fb518067130 auth_method 0 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51d939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51d939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51d939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51d939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51d939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.483+0000 7fb51d939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.484+0000 7fb51d939640 10 monclient(hunting): handle_auth_done global_id 24460 payload 274 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.484+0000 7fb51d939640 1 -- 172.21.15.204:0/668000066 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb518067500 msgr2=0x7fb5180678d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.484+0000 7fb51d939640 1 --2- 172.21.15.204:0/668000066 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb518067500 0x7fb5180678d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.484+0000 7fb51d939640 1 -- 172.21.15.204:0/668000066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb518136e80 msgr2=0x7fb518139260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.484+0000 7fb51d939640 1 --2- 172.21.15.204:0/668000066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb518136e80 0x7fb518139260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.484+0000 7fb51d939640 10 monclient: _finish_hunting 0 2024-09-15T01:37:01.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.484+0000 7fb51d939640 1 monclient: found mon.noname-i 2024-09-15T01:37:01.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.484+0000 7fb51d939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:01.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.484+0000 7fb51d939640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:01.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.484+0000 7fb51d939640 1 -- 172.21.15.204:0/668000066 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180648a0 con 0x7fb518067130 2024-09-15T01:37:01.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51d939640 1 --2- 172.21.15.204:0/668000066 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb518067130 0x7fb518097c20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb514009bd0 tx=0x7fb5140091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=4138a65c57827417 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:01.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51c937640 1 -- 172.21.15.204:0/668000066 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb514031880 con 0x7fb518067130 2024-09-15T01:37:01.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51c937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:01.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51c937640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:01.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51c937640 10 monclient: dump: 2024-09-15T01:37:01.496 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:01.496 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:01.496 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:01.496 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:01.497 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:01.497 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:01.497 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:01.497 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:01.497 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:01.497 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:01.497 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:01.497 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:01.497 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:01.497 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:01.497 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:01.498 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:01.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51c937640 10 monclient: _finish_auth 0 2024-09-15T01:37:01.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51c937640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:01.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51c937640 1 -- 172.21.15.204:0/668000066 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51402d050 con 0x7fb518067130 2024-09-15T01:37:01.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51fb9d640 5 monclient: authenticate success, global_id 24460 2024-09-15T01:37:01.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51fb9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:01.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51c937640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:01.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51fb9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:01.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51fb9d640 4 set_mon_vals no callback set 2024-09-15T01:37:01.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51c937640 1 -- 172.21.15.204:0/668000066 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5140303a0 con 0x7fb518067130 2024-09-15T01:37:01.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51c937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51c937640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.485+0000 7fb51c937640 10 monclient: dump: 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:01.499 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.486+0000 7fb51fb9d640 10 monclient: shutdown 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.486+0000 7fb51fb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.486+0000 7fb51fb9d640 1 -- 172.21.15.204:0/668000066 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb518067130 msgr2=0x7fb518097c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.486+0000 7fb51fb9d640 1 --2- 172.21.15.204:0/668000066 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb518067130 0x7fb518097c20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb514009bd0 tx=0x7fb5140091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.486+0000 7fb51fb9d640 1 -- 172.21.15.204:0/668000066 shutdown_connections 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.486+0000 7fb51fb9d640 1 --2- 172.21.15.204:0/668000066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb518136e80 0x7fb518139260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.486+0000 7fb51fb9d640 1 --2- 172.21.15.204:0/668000066 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb518067500 0x7fb5180678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.486+0000 7fb51fb9d640 1 --2- 172.21.15.204:0/668000066 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb518067130 0x7fb518097c20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.486+0000 7fb51fb9d640 1 -- 172.21.15.204:0/668000066 >> 172.21.15.204:0/668000066 conn(0x7fb5180a2ce0 msgr2=0x7fb5180a0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.486+0000 7fb51fb9d640 1 -- 172.21.15.204:0/668000066 shutdown_connections 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.486+0000 7fb51fb9d640 1 -- 172.21.15.204:0/668000066 wait complete. 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.487+0000 7fb51fb9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.487+0000 7fb51fb9d640 10 monclient: monmap: 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:01.501 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.487+0000 7fb51fb9d640 1 Processor -- start 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.487+0000 7fb51fb9d640 1 -- start start 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.487+0000 7fb51fb9d640 10 monclient: init 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:01.502 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 10 monclient: _add_conns ranks=[1,3,4,8,6,5,7,2,0] 2024-09-15T01:37:01.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb518067130 0x7fb51806ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:01.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 10 monclient(hunting): picked mon.noname-f con 0x7fb518067130 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:01.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb518067500 0x7fb51806e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:01.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 10 monclient(hunting): picked mon.noname-g con 0x7fb518067500 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:01.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb518136e80 0x7fb51806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:01.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 10 monclient(hunting): picked mon.noname-c con 0x7fb518136e80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:01.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:01.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb518131fc0 con 0x7fb518067130 2024-09-15T01:37:01.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:01.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51811eb90 con 0x7fb518067500 2024-09-15T01:37:01.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:01.503 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb518098590 con 0x7fb518136e80 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51fb9d640 10 monclient(hunting): authenticate will time out at 1037.130256s 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51e13a640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb518136e80 msgr2=0x7fb51806e870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d939640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb518067130 0x7fb51806ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb518067500 0x7fb51806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51e13a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb518136e80 0x7fb51806e870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d939640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb518067130 0x7fb51806ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44798/0 (socket says 172.21.15.204:44798) 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb518067500 0x7fb51806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:55298/0 (socket says 172.21.15.204:55298) 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d939640 1 -- 172.21.15.204:0/3259362528 learned_addr learned my addr 172.21.15.204:0/3259362528 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d939640 10 monclient(hunting): get_auth_request con 0x7fb518067130 auth_method 0 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:01.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:01.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 10 monclient(hunting): get_auth_request con 0x7fb518067500 auth_method 0 2024-09-15T01:37:01.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:01.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:01.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:01.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:01.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:01.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:01.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:01.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:01.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:01.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 10 monclient(hunting): handle_auth_done global_id 15357 payload 824 2024-09-15T01:37:01.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 1 -- 172.21.15.204:0/3259362528 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb518136e80 msgr2=0x7fb51806e870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 1 --2- 172.21.15.204:0/3259362528 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb518136e80 0x7fb51806e870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 1 -- 172.21.15.204:0/3259362528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb518067130 msgr2=0x7fb51806ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 1 --2- 172.21.15.204:0/3259362528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb518067130 0x7fb51806ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 10 monclient: _finish_hunting 0 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 1 monclient: found mon.noname-g 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 1 -- 172.21.15.204:0/3259362528 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb514009450 con 0x7fb518067500 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d939640 1 --2- 172.21.15.204:0/3259362528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb518067130 0x7fb51806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb51d138640 1 --2- 172.21.15.204:0/3259362528 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb518067500 0x7fb51806e330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb50800b5b0 tx=0x7fb5080090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.488+0000 7fb50dffb640 1 -- 172.21.15.204:0/3259362528 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb508006150 con 0x7fb518067500 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50dffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:01.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50dffb640 10 monclient: dump: 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:01.507 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50dffb640 10 monclient: _finish_auth 0 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50dffb640 1 -- 172.21.15.204:0/3259362528 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5080062d0 con 0x7fb518067500 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb51fb9d640 5 monclient: authenticate success, global_id 15357 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb51fb9d640 10 monclient: _renew_subs 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50dffb640 1 -- 172.21.15.204:0/3259362528 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb508010360 con 0x7fb518067500 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb51fb9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb51fb9d640 1 -- 172.21.15.204:0/3259362528 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51811f650 con 0x7fb518067500 2024-09-15T01:37:01.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:01.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:01.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50ffff640 4 set_mon_vals no callback set 2024-09-15T01:37:01.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50dffb640 10 monclient: dump: 2024-09-15T01:37:01.509 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:01.509 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:01.509 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:01.509 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:01.509 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:01.509 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:01.509 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:01.509 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:01.509 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb51fb9d640 10 monclient: _renew_subs 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb51fb9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb51fb9d640 1 -- 172.21.15.204:0/3259362528 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51811cda0 con 0x7fb518067500 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.489+0000 7fb50dffb640 1 -- 172.21.15.204:0/3259362528 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb508003b70 con 0x7fb518067500 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.490+0000 7fb50dffb640 1 --2- 172.21.15.204:0/3259362528 >> v2:172.21.15.204:6812/28210 conn(0x7fb4fc0397b0 0x7fb4fc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.490+0000 7fb50dffb640 1 -- 172.21.15.204:0/3259362528 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 8341+0+0 (secure 0 0 0) 0x7fb50804b680 con 0x7fb518067500 2024-09-15T01:37:01.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.490+0000 7fb51d939640 1 --2- 172.21.15.204:0/3259362528 >> v2:172.21.15.204:6812/28210 conn(0x7fb4fc0397b0 0x7fb4fc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:01.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.490+0000 7fb51fb9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:01.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.490+0000 7fb51fb9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:01.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.490+0000 7fb51fb9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:01.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.490+0000 7fb51fb9d640 1 -- 172.21.15.204:0/3259362528 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5180987c0 con 0x7fb518067500 2024-09-15T01:37:01.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.490+0000 7fb51d939640 10 monclient: get_auth_request con 0x7fb4fc0397b0 auth_method 0 2024-09-15T01:37:01.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.490+0000 7fb51d939640 1 --2- 172.21.15.204:0/3259362528 >> v2:172.21.15.204:6812/28210 conn(0x7fb4fc0397b0 0x7fb4fc03bc60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb514009bd0 tx=0x7fb514035000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:01.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.494+0000 7fb50dffb640 1 -- 172.21.15.204:0/3259362528 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb508017070 con 0x7fb518067500 2024-09-15T01:37:01.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.494+0000 7fb50dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:01.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.494+0000 7fb50dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.653+0000 7fb51fb9d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.653+0000 7fb51fb9d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.653+0000 7fb51fb9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:01.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:01.653+0000 7fb51fb9d640 1 -- 172.21.15.204:0/3259362528 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fb5180a0e20 con 0x7fb518067500 2024-09-15T01:37:02.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.481+0000 7fb50dffb640 1 -- 172.21.15.204:0/3259362528 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}]=0 added bucket h7 type host to location {rack=r1} v192) v1 ==== 166+0+0 (secure 0 0 0) 0x7fb5080182b0 con 0x7fb518067500 2024-09-15T01:37:02.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.481+0000 7fb50dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:02.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.481+0000 7fb50dffb640 10 monclient: _finish_command 2 = system:0 added bucket h7 type host to location {rack=r1} 2024-09-15T01:37:02.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.578+0000 7fb51fb9d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:02.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.578+0000 7fb51fb9d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:02.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.578+0000 7fb51fb9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:02.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.578+0000 7fb51fb9d640 1 -- 172.21.15.204:0/3259362528 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fb518070b80 con 0x7fb518067500 2024-09-15T01:37:02.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.579+0000 7fb50dffb640 1 -- 172.21.15.204:0/3259362528 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h7' already exists v192) v1 ==== 145+0+0 (secure 0 0 0) 0x7fb50800f300 con 0x7fb518067500 2024-09-15T01:37:02.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.579+0000 7fb50dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:02.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.579+0000 7fb50dffb640 10 monclient: _finish_command 3 = system:0 bucket 'h7' already exists 2024-09-15T01:37:02.581 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h7' already exists 2024-09-15T01:37:02.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 1 -- 172.21.15.204:0/3259362528 >> v2:172.21.15.204:6812/28210 conn(0x7fb4fc0397b0 msgr2=0x7fb4fc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:02.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 1 --2- 172.21.15.204:0/3259362528 >> v2:172.21.15.204:6812/28210 conn(0x7fb4fc0397b0 0x7fb4fc03bc60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb514009bd0 tx=0x7fb514035000 comp rx=0 tx=0).stop 2024-09-15T01:37:02.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 10 monclient: shutdown 2024-09-15T01:37:02.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:02.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 1 -- 172.21.15.204:0/3259362528 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb518067500 msgr2=0x7fb51806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:02.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 1 --2- 172.21.15.204:0/3259362528 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb518067500 0x7fb51806e330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb50800b5b0 tx=0x7fb5080090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:02.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 1 -- 172.21.15.204:0/3259362528 shutdown_connections 2024-09-15T01:37:02.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 1 --2- 172.21.15.204:0/3259362528 >> v2:172.21.15.204:6812/28210 conn(0x7fb4fc0397b0 0x7fb4fc03bc60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:02.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 1 --2- 172.21.15.204:0/3259362528 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb518136e80 0x7fb51806e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:02.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 1 --2- 172.21.15.204:0/3259362528 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb518067500 0x7fb51806e330 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:02.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 1 --2- 172.21.15.204:0/3259362528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb518067130 0x7fb51806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:02.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 1 -- 172.21.15.204:0/3259362528 >> 172.21.15.204:0/3259362528 conn(0x7fb5180a2ce0 msgr2=0x7fb518136590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:02.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 1 -- 172.21.15.204:0/3259362528 shutdown_connections 2024-09-15T01:37:02.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.581+0000 7fb51fb9d640 1 -- 172.21.15.204:0/3259362528 wait complete. 2024-09-15T01:37:02.592 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:02.592 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h8 host rack=r1 2024-09-15T01:37:02.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.665+0000 7f9a5dd54640 10 monclient: get_monmap_and_config 2024-09-15T01:37:02.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.665+0000 7f9a5dd54640 10 monclient: build_initial_monmap 2024-09-15T01:37:02.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.665+0000 7f9a5dd54640 10 monclient: monmap: 2024-09-15T01:37:02.667 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:02.667 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:02.667 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:02.665671+0000 2024-09-15T01:37:02.667 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:02.665671+0000 2024-09-15T01:37:02.667 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:02.667 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:02.667 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:02.667 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:02.667 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:02.667 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:02.667 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:02.668 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:02.668 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:02.668 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:02.668 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:02.668 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:02.668 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:02.668 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:02.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 1 Processor -- start 2024-09-15T01:37:02.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 1 -- start start 2024-09-15T01:37:02.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 10 monclient: init 2024-09-15T01:37:02.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:02.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 10 monclient: _add_conns ranks=[0,1,10,8,5,9,7,2,3,4,6] 2024-09-15T01:37:02.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9a580a2cd0 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9a5809fa20 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9a58138c90 0x7f9a5813b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 10 monclient(hunting): picked mon.noname-f con 0x7f9a58138c90 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a580648a0 con 0x7f9a580a2cd0 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a58133310 con 0x7f9a5809fa20 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a580a30a0 con 0x7f9a58138c90 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a5dd54640 10 monclient(hunting): authenticate will time out at 1038.308650s 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a57fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9a58138c90 0x7f9a5813b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a577fe640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f9a580a2cd0 legacy=0x7f9a58132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a4ffff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f9a5809fa20 0x7f9a5809fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:59924/0 (socket says 172.21.15.204:59924) 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a57fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9a58138c90 0x7f9a5813b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44806/0 (socket says 172.21.15.204:44806) 2024-09-15T01:37:02.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a4ffff640 1 -- 172.21.15.204:0/3385041613 learned_addr learned my addr 172.21.15.204:0/3385041613 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:37:02.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a57fff640 10 monclient(hunting): get_auth_request con 0x7f9a58138c90 auth_method 0 2024-09-15T01:37:02.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a57fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:02.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a57fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:02.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a57fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a57fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a57fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.666+0000 7f9a57fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a57fff640 10 monclient(hunting): handle_auth_done global_id 6940 payload 274 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a57fff640 1 -- 172.21.15.204:0/3385041613 >> v1:172.21.15.204:6789/0 conn(0x7f9a5809fa20 legacy=0x7f9a5809fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a57fff640 1 -- 172.21.15.204:0/3385041613 >> v1:172.21.15.178:6789/0 conn(0x7f9a580a2cd0 legacy=0x7f9a58132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a57fff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a57fff640 1 monclient: found mon.noname-f 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a57fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a57fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a57fff640 1 -- 172.21.15.204:0/3385041613 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a580a1db0 con 0x7f9a58138c90 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a57fff640 1 --2- 172.21.15.204:0/3385041613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a58138c90 0x7f9a5813b070 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009e60 tx=0x7f9a50009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81dffd2b294e8031 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:02.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a56ffd640 1 -- 172.21.15.204:0/3385041613 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a500079d0 con 0x7f9a58138c90 2024-09-15T01:37:02.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:02.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a56ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:02.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a56ffd640 10 monclient: dump: 2024-09-15T01:37:02.679 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:02.679 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:02.679 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:02.679 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:02.679 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:02.679 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:02.679 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:02.680 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:02.680 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:02.680 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:02.680 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:02.680 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:02.680 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:02.680 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:02.680 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:02.680 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:02.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a56ffd640 10 monclient: _finish_auth 0 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a5dd54640 5 monclient: authenticate success, global_id 6940 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a56ffd640 1 -- 172.21.15.204:0/3385041613 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5002d050 con 0x7f9a58138c90 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a5dd54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a56ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a5dd54640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a5dd54640 4 set_mon_vals no callback set 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a56ffd640 1 -- 172.21.15.204:0/3385041613 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a50007bd0 con 0x7f9a58138c90 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a56ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.667+0000 7f9a56ffd640 10 monclient: dump: 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:02.681 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:02.682 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.668+0000 7f9a5dd54640 10 monclient: shutdown 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.668+0000 7f9a5dd54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.668+0000 7f9a5dd54640 1 -- 172.21.15.204:0/3385041613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a58138c90 msgr2=0x7f9a5813b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.668+0000 7f9a5dd54640 1 --2- 172.21.15.204:0/3385041613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a58138c90 0x7f9a5813b070 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009e60 tx=0x7f9a50009780 comp rx=0 tx=0).stop 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.668+0000 7f9a5dd54640 1 -- 172.21.15.204:0/3385041613 shutdown_connections 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.668+0000 7f9a5dd54640 1 --2- 172.21.15.204:0/3385041613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a58138c90 0x7f9a5813b070 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.668+0000 7f9a5dd54640 1 -- 172.21.15.204:0/3385041613 >> 172.21.15.204:0/3385041613 conn(0x7f9a5806e860 msgr2=0x7f9a5806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.668+0000 7f9a5dd54640 1 -- 172.21.15.204:0/3385041613 shutdown_connections 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.668+0000 7f9a5dd54640 1 -- 172.21.15.204:0/3385041613 wait complete. 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.668+0000 7f9a5dd54640 10 monclient: build_initial_monmap 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 10 monclient: monmap: 2024-09-15T01:37:02.683 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:02.684 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:02.684 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:02.684 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:02.684 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:02.684 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:02.684 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:02.684 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:02.684 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:02.684 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:02.684 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:02.684 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:02.684 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 1 Processor -- start 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 1 -- start start 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 10 monclient: init 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 10 monclient: _add_conns ranks=[3,2,5,7,0,6,4,8,1] 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a5809fa20 0x7f9a581320e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 10 monclient(hunting): picked mon.noname-g con 0x7f9a5809fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a580a2cd0 0x7f9a581339b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 10 monclient(hunting): picked mon.noname-b con 0x7f9a580a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a58138c90 0x7f9a58133ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:02.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 10 monclient(hunting): picked mon.noname-h con 0x7f9a58138c90 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a58136e60 con 0x7f9a580a2cd0 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5809cb60 con 0x7f9a5809fa20 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a58136af0 con 0x7f9a58138c90 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a5dd54640 10 monclient(hunting): authenticate will time out at 1038.311824s 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a577fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a5809fa20 0x7f9a581320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a57fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a58138c90 0x7f9a58133ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a577fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a5809fa20 0x7f9a581320e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:55306/0 (socket says 172.21.15.204:55306) 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a4ffff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a580a2cd0 msgr2=0x7f9a581339b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a577fe640 1 -- 172.21.15.204:0/2364010234 learned_addr learned my addr 172.21.15.204:0/2364010234 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:02.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a577fe640 10 monclient(hunting): get_auth_request con 0x7f9a5809fa20 auth_method 0 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a577fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a57fff640 10 monclient(hunting): get_auth_request con 0x7f9a58138c90 auth_method 0 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a4ffff640 1 --2- 172.21.15.204:0/2364010234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a580a2cd0 0x7f9a581339b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a57fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a57fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.669+0000 7f9a57fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:02.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 10 monclient(hunting): handle_auth_done global_id 6197 payload 824 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 1 -- 172.21.15.204:0/2364010234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a5809fa20 msgr2=0x7f9a581320e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 1 --2- 172.21.15.204:0/2364010234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a5809fa20 0x7f9a581320e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 1 -- 172.21.15.204:0/2364010234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a580a2cd0 msgr2=0x7f9a581339b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 1 --2- 172.21.15.204:0/2364010234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a580a2cd0 0x7f9a581339b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 1 monclient: found mon.noname-h 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 1 -- 172.21.15.204:0/2364010234 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a50009a20 con 0x7f9a58138c90 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a577fe640 1 --2- 172.21.15.204:0/2364010234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a5809fa20 0x7f9a581320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:02.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a57fff640 1 --2- 172.21.15.204:0/2364010234 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a58138c90 0x7f9a58133ef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9a50031000 tx=0x7f9a50037000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:02.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a4f7fe640 1 -- 172.21.15.204:0/2364010234 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a5003c420 con 0x7f9a58138c90 2024-09-15T01:37:02.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a4f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:02.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a4f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:02.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a4f7fe640 10 monclient: dump: 2024-09-15T01:37:02.689 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:02.689 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:02.689 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:02.689 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:02.689 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:02.689 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:02.689 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a4f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a4f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a4f7fe640 1 -- 172.21.15.204:0/2364010234 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5003c5a0 con 0x7f9a58138c90 2024-09-15T01:37:02.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a5dd54640 5 monclient: authenticate success, global_id 6197 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a4f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a5dd54640 10 monclient: _renew_subs 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a4f7fe640 1 -- 172.21.15.204:0/2364010234 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a5003cbd0 con 0x7f9a58138c90 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a5dd54640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a5dd54640 1 -- 172.21.15.204:0/2364010234 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a58137bf0 con 0x7f9a58138c90 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a4f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a4f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a4f7fe640 10 monclient: dump: 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:02.691 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a567fc640 4 set_mon_vals no callback set 2024-09-15T01:37:02.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a5dd54640 10 monclient: _renew_subs 2024-09-15T01:37:02.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a5dd54640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:02.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.670+0000 7f9a5dd54640 1 -- 172.21.15.204:0/2364010234 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a58136810 con 0x7f9a58138c90 2024-09-15T01:37:02.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.671+0000 7f9a4f7fe640 1 -- 172.21.15.204:0/2364010234 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9a50003040 con 0x7f9a58138c90 2024-09-15T01:37:02.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.671+0000 7f9a5dd54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:02.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.671+0000 7f9a5dd54640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:02.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.671+0000 7f9a5dd54640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:02.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.671+0000 7f9a5dd54640 1 -- 172.21.15.204:0/2364010234 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a58136fd0 con 0x7f9a58138c90 2024-09-15T01:37:02.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.671+0000 7f9a4f7fe640 1 --2- 172.21.15.204:0/2364010234 >> v2:172.21.15.204:6812/28210 conn(0x7f9a380397b0 0x7f9a3803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:02.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.671+0000 7f9a4f7fe640 1 -- 172.21.15.204:0/2364010234 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 8551+0+0 (secure 0 0 0) 0x7f9a5006d890 con 0x7f9a58138c90 2024-09-15T01:37:02.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.671+0000 7f9a577fe640 1 --2- 172.21.15.204:0/2364010234 >> v2:172.21.15.204:6812/28210 conn(0x7f9a380397b0 0x7f9a3803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:02.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.671+0000 7f9a577fe640 10 monclient: get_auth_request con 0x7f9a380397b0 auth_method 0 2024-09-15T01:37:02.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.672+0000 7f9a577fe640 1 --2- 172.21.15.204:0/2364010234 >> v2:172.21.15.204:6812/28210 conn(0x7f9a380397b0 0x7f9a3803bc60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9a480098e0 tx=0x7f9a48009120 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:02.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.676+0000 7f9a4f7fe640 1 -- 172.21.15.204:0/2364010234 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9a50031ac0 con 0x7f9a58138c90 2024-09-15T01:37:02.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.676+0000 7f9a4f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:02.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.676+0000 7f9a4f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:02.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.834+0000 7f9a5dd54640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:02.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.834+0000 7f9a5dd54640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:02.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.834+0000 7f9a5dd54640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:02.836 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:02.834+0000 7f9a5dd54640 1 -- 172.21.15.204:0/2364010234 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f9a58070b80 con 0x7f9a58138c90 2024-09-15T01:37:03.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.483+0000 7f9a4f7fe640 1 -- 172.21.15.204:0/2364010234 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}]=0 added bucket h8 type host to location {rack=r1} v193) v1 ==== 166+0+0 (secure 0 0 0) 0x7f9a50038350 con 0x7f9a58138c90 2024-09-15T01:37:03.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.483+0000 7f9a4f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:03.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.483+0000 7f9a4f7fe640 10 monclient: _finish_command 2 = system:0 added bucket h8 type host to location {rack=r1} 2024-09-15T01:37:03.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.582+0000 7f9a5dd54640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:03.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.582+0000 7f9a5dd54640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:03.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.582+0000 7f9a5dd54640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:03.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.582+0000 7f9a5dd54640 1 -- 172.21.15.204:0/2364010234 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f9a580008d0 con 0x7f9a58138c90 2024-09-15T01:37:03.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.583+0000 7f9a4f7fe640 1 -- 172.21.15.204:0/2364010234 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h8' already exists v193) v1 ==== 145+0+0 (secure 0 0 0) 0x7f9a50038530 con 0x7f9a58138c90 2024-09-15T01:37:03.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.583+0000 7f9a4f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:03.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.583+0000 7f9a4f7fe640 10 monclient: _finish_command 3 = system:0 bucket 'h8' already exists 2024-09-15T01:37:03.585 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h8' already exists 2024-09-15T01:37:03.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 1 -- 172.21.15.204:0/2364010234 >> v2:172.21.15.204:6812/28210 conn(0x7f9a380397b0 msgr2=0x7f9a3803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:03.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 1 --2- 172.21.15.204:0/2364010234 >> v2:172.21.15.204:6812/28210 conn(0x7f9a380397b0 0x7f9a3803bc60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9a480098e0 tx=0x7f9a48009120 comp rx=0 tx=0).stop 2024-09-15T01:37:03.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 10 monclient: shutdown 2024-09-15T01:37:03.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:03.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 1 -- 172.21.15.204:0/2364010234 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a58138c90 msgr2=0x7f9a58133ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:03.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 1 --2- 172.21.15.204:0/2364010234 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a58138c90 0x7f9a58133ef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9a50031000 tx=0x7f9a50037000 comp rx=0 tx=0).stop 2024-09-15T01:37:03.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 1 -- 172.21.15.204:0/2364010234 shutdown_connections 2024-09-15T01:37:03.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 1 --2- 172.21.15.204:0/2364010234 >> v2:172.21.15.204:6812/28210 conn(0x7f9a380397b0 0x7f9a3803bc60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:03.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 1 --2- 172.21.15.204:0/2364010234 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a58138c90 0x7f9a58133ef0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:03.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 1 --2- 172.21.15.204:0/2364010234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a580a2cd0 0x7f9a581339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:03.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 1 --2- 172.21.15.204:0/2364010234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a5809fa20 0x7f9a581320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:03.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 1 -- 172.21.15.204:0/2364010234 >> 172.21.15.204:0/2364010234 conn(0x7f9a5806e860 msgr2=0x7f9a5812cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:03.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 1 -- 172.21.15.204:0/2364010234 shutdown_connections 2024-09-15T01:37:03.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.585+0000 7f9a5dd54640 1 -- 172.21.15.204:0/2364010234 wait complete. 2024-09-15T01:37:03.596 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:03.596 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h9 host rack=r1 2024-09-15T01:37:03.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.671+0000 7fd217ec0640 10 monclient: get_monmap_and_config 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.671+0000 7fd217ec0640 10 monclient: build_initial_monmap 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.671+0000 7fd217ec0640 10 monclient: monmap: 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:03.671603+0000 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:03.671603+0000 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:03.673 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:03.674 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:03.674 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:03.674 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:03.674 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:03.674 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:03.674 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:03.674 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:03.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.671+0000 7fd217ec0640 1 Processor -- start 2024-09-15T01:37:03.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.671+0000 7fd217ec0640 1 -- start start 2024-09-15T01:37:03.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.671+0000 7fd217ec0640 10 monclient: init 2024-09-15T01:37:03.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:03.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 10 monclient: _add_conns ranks=[3,10,8,0,4,2,5,1,6,9,7] 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2100a2cd0 0x7fd210132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 10 monclient(hunting): picked mon.noname-g con 0x7fd2100a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd21009fa20 0x7fd21009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 10 monclient(hunting): picked mon.noname-f con 0x7fd21009fa20 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2100a08e0 0x7fd2100a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 10 monclient(hunting): picked mon.noname-e con 0x7fd2100a08e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2100a30a0 con 0x7fd21009fa20 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2100a0330 con 0x7fd2100a2cd0 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd2100a1db0 con 0x7fd2100a08e0 2024-09-15T01:37:03.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:03.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd217ec0640 10 monclient(hunting): authenticate will time out at 1039.314402s 2024-09-15T01:37:03.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd215c5c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2100a2cd0 0x7fd210132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:03.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd21009fa20 0x7fd21009fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:03.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd21009fa20 0x7fd21009fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44810/0 (socket says 172.21.15.204:44810) 2024-09-15T01:37:03.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21645d640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2100a08e0 msgr2=0x7fd2100a0d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:03.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd215c5c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2100a2cd0 0x7fd210132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:55318/0 (socket says 172.21.15.204:55318) 2024-09-15T01:37:03.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 1 -- 172.21.15.204:0/502017157 learned_addr learned my addr 172.21.15.204:0/502017157 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:03.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 10 monclient(hunting): get_auth_request con 0x7fd21009fa20 auth_method 0 2024-09-15T01:37:03.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:03.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:03.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:03.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21645d640 1 --2- 172.21.15.204:0/502017157 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2100a08e0 0x7fd2100a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd215c5c640 10 monclient(hunting): get_auth_request con 0x7fd2100a2cd0 auth_method 0 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd215c5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd215c5c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd215c5c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd215c5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd215c5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd215c5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 10 monclient(hunting): handle_auth_done global_id 6958 payload 274 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 1 -- 172.21.15.204:0/502017157 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2100a08e0 msgr2=0x7fd2100a0d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:03.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 1 --2- 172.21.15.204:0/502017157 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2100a08e0 0x7fd2100a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 1 -- 172.21.15.204:0/502017157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2100a2cd0 msgr2=0x7fd210132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 1 --2- 172.21.15.204:0/502017157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2100a2cd0 0x7fd210132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 10 monclient: _finish_hunting 0 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 1 monclient: found mon.noname-f 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd21545b640 1 -- 172.21.15.204:0/502017157 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2100648a0 con 0x7fd21009fa20 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.672+0000 7fd215c5c640 1 --2- 172.21.15.204:0/502017157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2100a2cd0 0x7fd210132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd21545b640 1 --2- 172.21.15.204:0/502017157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd21009fa20 0x7fd21009fdf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd200009910 tx=0x7fd2000091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a55bd203c910c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd214c5a640 1 -- 172.21.15.204:0/502017157 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd20002acb0 con 0x7fd21009fa20 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd214c5a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd214c5a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd214c5a640 10 monclient: dump: 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd214c5a640 10 monclient: _finish_auth 0 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd214c5a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd214c5a640 1 -- 172.21.15.204:0/502017157 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20002ae30 con 0x7fd21009fa20 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd217ec0640 5 monclient: authenticate success, global_id 6958 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd217ec0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd214c5a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd217ec0640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd214c5a640 1 -- 172.21.15.204:0/502017157 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd200031ca0 con 0x7fd21009fa20 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd217ec0640 4 set_mon_vals no callback set 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd214c5a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd214c5a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:03.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.673+0000 7fd214c5a640 10 monclient: dump: 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:03.688 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 10 monclient: shutdown 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 1 -- 172.21.15.204:0/502017157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd21009fa20 msgr2=0x7fd21009fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 1 --2- 172.21.15.204:0/502017157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd21009fa20 0x7fd21009fdf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd200009910 tx=0x7fd2000091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 1 -- 172.21.15.204:0/502017157 shutdown_connections 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 1 --2- 172.21.15.204:0/502017157 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2100a08e0 0x7fd2100a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 1 --2- 172.21.15.204:0/502017157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd21009fa20 0x7fd21009fdf0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 1 --2- 172.21.15.204:0/502017157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2100a2cd0 0x7fd210132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 1 -- 172.21.15.204:0/502017157 >> 172.21.15.204:0/502017157 conn(0x7fd21006e860 msgr2=0x7fd21006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:03.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 1 -- 172.21.15.204:0/502017157 shutdown_connections 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 1 -- 172.21.15.204:0/502017157 wait complete. 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 10 monclient: build_initial_monmap 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.674+0000 7fd217ec0640 10 monclient: monmap: 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:03.690 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 1 Processor -- start 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 1 -- start start 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 10 monclient: init 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 10 monclient: _add_conns ranks=[5,0,2,3,1,7,6,8,4] 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd21009fa20 0x7fd210131850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 10 monclient(hunting): picked mon.noname-h con 0x7fd21009fa20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2100a08e0 0x7fd21011d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:03.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 10 monclient(hunting): picked mon.noname-a con 0x7fd2100a08e0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:03.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2100a2cd0 0x7fd21011d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:03.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 10 monclient(hunting): picked mon.noname-b con 0x7fd2100a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:03.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:03.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21009d3d0 con 0x7fd2100a08e0 2024-09-15T01:37:03.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:03.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21009cb60 con 0x7fd2100a2cd0 2024-09-15T01:37:03.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:03.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd21011f3e0 con 0x7fd21009fa20 2024-09-15T01:37:03.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:03.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd217ec0640 10 monclient(hunting): authenticate will time out at 1039.317612s 2024-09-15T01:37:03.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd21009fa20 0x7fd210131850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd21009fa20 0x7fd210131850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:50034/0 (socket says 172.21.15.204:50034) 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd21545b640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2100a08e0 msgr2=0x7fd21011d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd21645d640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2100a2cd0 msgr2=0x7fd21011d8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 1 -- 172.21.15.204:0/324718724 learned_addr learned my addr 172.21.15.204:0/324718724 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 10 monclient(hunting): get_auth_request con 0x7fd21009fa20 auth_method 0 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd21545b640 1 --2- 172.21.15.204:0/324718724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2100a08e0 0x7fd21011d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd21645d640 1 --2- 172.21.15.204:0/324718724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2100a2cd0 0x7fd21011d8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:03.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 10 monclient(hunting): handle_auth_done global_id 6206 payload 824 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 1 -- 172.21.15.204:0/324718724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2100a2cd0 msgr2=0x7fd21011d8c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 1 --2- 172.21.15.204:0/324718724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2100a2cd0 0x7fd21011d8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 1 -- 172.21.15.204:0/324718724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2100a08e0 msgr2=0x7fd21011d380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 1 --2- 172.21.15.204:0/324718724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2100a08e0 0x7fd21011d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 10 monclient: _finish_hunting 0 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 1 monclient: found mon.noname-h 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.675+0000 7fd215c5c640 1 -- 172.21.15.204:0/324718724 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20c009390 con 0x7fd21009fa20 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd215c5c640 1 --2- 172.21.15.204:0/324718724 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd21009fa20 0x7fd210131850 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00ea70 tx=0x7fd20c0090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd205ffb640 1 -- 172.21.15.204:0/324718724 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd20c00ce90 con 0x7fd21009fa20 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd205ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd205ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd205ffb640 10 monclient: dump: 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:03.696 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd205ffb640 10 monclient: _finish_auth 0 2024-09-15T01:37:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd205ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd205ffb640 1 -- 172.21.15.204:0/324718724 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20c013470 con 0x7fd21009fa20 2024-09-15T01:37:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd217ec0640 5 monclient: authenticate success, global_id 6206 2024-09-15T01:37:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd217ec0640 10 monclient: _renew_subs 2024-09-15T01:37:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd217ec0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd217ec0640 1 -- 172.21.15.204:0/324718724 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd200009450 con 0x7fd21009fa20 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd205ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd205ffb640 1 -- 172.21.15.204:0/324718724 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd20c00ca10 con 0x7fd21009fa20 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd205ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd205ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd205ffb640 10 monclient: dump: 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd217ec0640 10 monclient: _renew_subs 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd207fff640 4 set_mon_vals no callback set 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd217ec0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.676+0000 7fd217ec0640 1 -- 172.21.15.204:0/324718724 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd210120990 con 0x7fd21009fa20 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.677+0000 7fd205ffb640 1 -- 172.21.15.204:0/324718724 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd20c00cb70 con 0x7fd21009fa20 2024-09-15T01:37:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.677+0000 7fd217ec0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.677+0000 7fd217ec0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.677+0000 7fd217ec0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.677+0000 7fd205ffb640 1 --2- 172.21.15.204:0/324718724 >> v2:172.21.15.204:6812/28210 conn(0x7fd1f40397b0 0x7fd1f403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.677+0000 7fd217ec0640 1 -- 172.21.15.204:0/324718724 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd210120e00 con 0x7fd21009fa20 2024-09-15T01:37:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.677+0000 7fd205ffb640 1 -- 172.21.15.204:0/324718724 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 8761+0+0 (secure 0 0 0) 0x7fd20c04c850 con 0x7fd21009fa20 2024-09-15T01:37:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.677+0000 7fd21545b640 1 --2- 172.21.15.204:0/324718724 >> v2:172.21.15.204:6812/28210 conn(0x7fd1f40397b0 0x7fd1f403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.677+0000 7fd21545b640 10 monclient: get_auth_request con 0x7fd1f40397b0 auth_method 0 2024-09-15T01:37:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.677+0000 7fd21545b640 1 --2- 172.21.15.204:0/324718724 >> v2:172.21.15.204:6812/28210 conn(0x7fd1f40397b0 0x7fd1f403bc60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd200034000 tx=0x7fd200032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.681+0000 7fd205ffb640 1 -- 172.21.15.204:0/324718724 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd20c013e30 con 0x7fd21009fa20 2024-09-15T01:37:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.681+0000 7fd205ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.681+0000 7fd205ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:03.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.840+0000 7fd217ec0640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:03.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.840+0000 7fd217ec0640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:03.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.840+0000 7fd217ec0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:03.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:03.840+0000 7fd217ec0640 1 -- 172.21.15.204:0/324718724 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fd210070b80 con 0x7fd21009fa20 2024-09-15T01:37:04.499 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.498+0000 7fd205ffb640 1 -- 172.21.15.204:0/324718724 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}]=0 added bucket h9 type host to location {rack=r1} v194) v1 ==== 166+0+0 (secure 0 0 0) 0x7fd20c0192c0 con 0x7fd21009fa20 2024-09-15T01:37:04.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.498+0000 7fd205ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:04.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.498+0000 7fd205ffb640 10 monclient: _finish_command 2 = system:0 added bucket h9 type host to location {rack=r1} 2024-09-15T01:37:04.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.596+0000 7fd217ec0640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:04.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.596+0000 7fd217ec0640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:04.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.596+0000 7fd217ec0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:04.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.596+0000 7fd217ec0640 1 -- 172.21.15.204:0/324718724 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fd210067450 con 0x7fd21009fa20 2024-09-15T01:37:04.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.597+0000 7fd205ffb640 1 -- 172.21.15.204:0/324718724 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h9' already exists v194) v1 ==== 145+0+0 (secure 0 0 0) 0x7fd20c0194a0 con 0x7fd21009fa20 2024-09-15T01:37:04.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.597+0000 7fd205ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:04.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.597+0000 7fd205ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h9' already exists 2024-09-15T01:37:04.600 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h9' already exists 2024-09-15T01:37:04.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 1 -- 172.21.15.204:0/324718724 >> v2:172.21.15.204:6812/28210 conn(0x7fd1f40397b0 msgr2=0x7fd1f403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:04.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 1 --2- 172.21.15.204:0/324718724 >> v2:172.21.15.204:6812/28210 conn(0x7fd1f40397b0 0x7fd1f403bc60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd200034000 tx=0x7fd200032000 comp rx=0 tx=0).stop 2024-09-15T01:37:04.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 10 monclient: shutdown 2024-09-15T01:37:04.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:04.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 1 -- 172.21.15.204:0/324718724 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd21009fa20 msgr2=0x7fd210131850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:04.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 1 --2- 172.21.15.204:0/324718724 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd21009fa20 0x7fd210131850 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00ea70 tx=0x7fd20c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 1 -- 172.21.15.204:0/324718724 shutdown_connections 2024-09-15T01:37:04.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 1 --2- 172.21.15.204:0/324718724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2100a2cd0 0x7fd21011d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 1 --2- 172.21.15.204:0/324718724 >> v2:172.21.15.204:6812/28210 conn(0x7fd1f40397b0 0x7fd1f403bc60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 1 --2- 172.21.15.204:0/324718724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2100a08e0 0x7fd21011d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 1 --2- 172.21.15.204:0/324718724 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd21009fa20 0x7fd210131850 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 1 -- 172.21.15.204:0/324718724 >> 172.21.15.204:0/324718724 conn(0x7fd21006e860 msgr2=0x7fd2100a4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:04.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 1 -- 172.21.15.204:0/324718724 shutdown_connections 2024-09-15T01:37:04.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.599+0000 7fd217ec0640 1 -- 172.21.15.204:0/324718724 wait complete. 2024-09-15T01:37:04.610 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:04.610 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h10 host rack=r1 2024-09-15T01:37:04.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.685+0000 7f4d48954640 10 monclient: get_monmap_and_config 2024-09-15T01:37:04.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.685+0000 7f4d48954640 10 monclient: build_initial_monmap 2024-09-15T01:37:04.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.685+0000 7f4d48954640 10 monclient: monmap: 2024-09-15T01:37:04.687 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:04.687 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:04.687 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:04.685620+0000 2024-09-15T01:37:04.687 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:04.685620+0000 2024-09-15T01:37:04.687 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:04.688 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:04.688 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:04.688 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:04.688 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:04.688 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:04.688 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:04.689 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:04.689 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:04.689 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:04.689 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:04.689 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:04.689 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:04.690 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:04.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.685+0000 7f4d48954640 1 Processor -- start 2024-09-15T01:37:04.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.685+0000 7f4d48954640 1 -- start start 2024-09-15T01:37:04.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.685+0000 7f4d48954640 10 monclient: init 2024-09-15T01:37:04.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:04.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 10 monclient: _add_conns ranks=[8,10,2,4,1,5,3,7,9,0,6] 2024-09-15T01:37:04.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4d440a2440 0x7f4d44132990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:04.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 10 monclient(hunting): picked mon.noname-e con 0x7f4d440a2440 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:04.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4d4409f950 0x7f4d4409fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:04.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 10 monclient(hunting): picked mon.noname-f con 0x7f4d4409f950 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:04.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d440a08e0 0x7f4d440a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:04.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 10 monclient(hunting): picked mon.noname-b con 0x7f4d440a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:04.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:04.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d440a1e20 con 0x7f4d4409f950 2024-09-15T01:37:04.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:04.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d440a2810 con 0x7f4d440a08e0 2024-09-15T01:37:04.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:04.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4409db80 con 0x7f4d440a2440 2024-09-15T01:37:04.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:04.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d48954640 10 monclient(hunting): authenticate will time out at 1040.328461s 2024-09-15T01:37:04.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4d4409f950 0x7f4d4409fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:04.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4d4409f950 0x7f4d4409fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44818/0 (socket says 172.21.15.204:44818) 2024-09-15T01:37:04.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 1 -- 172.21.15.204:0/406389071 learned_addr learned my addr 172.21.15.204:0/406389071 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:04.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d42d9d640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d440a08e0 msgr2=0x7f4d440a0d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:04.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 10 monclient(hunting): get_auth_request con 0x7f4d4409f950 auth_method 0 2024-09-15T01:37:04.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d4259c640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4d440a2440 msgr2=0x7f4d44132990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:04.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:04.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:04.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:04.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d42d9d640 1 --2- 172.21.15.204:0/406389071 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d440a08e0 0x7f4d440a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:04.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d4259c640 1 --2- 172.21.15.204:0/406389071 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4d440a2440 0x7f4d44132990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:04.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:04.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:04.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:04.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 10 monclient(hunting): handle_auth_done global_id 6967 payload 274 2024-09-15T01:37:04.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 1 -- 172.21.15.204:0/406389071 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4d440a2440 msgr2=0x7f4d44132990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:04.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 1 --2- 172.21.15.204:0/406389071 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4d440a2440 0x7f4d44132990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 1 -- 172.21.15.204:0/406389071 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d440a08e0 msgr2=0x7f4d440a0d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:04.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 1 --2- 172.21.15.204:0/406389071 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d440a08e0 0x7f4d440a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:04.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 1 monclient: found mon.noname-f 2024-09-15T01:37:04.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:04.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:04.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.686+0000 7f4d3bfff640 1 -- 172.21.15.204:0/406389071 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d440648a0 con 0x7f4d4409f950 2024-09-15T01:37:04.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d3bfff640 1 --2- 172.21.15.204:0/406389071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d4409f950 0x7f4d4409fd60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009910 tx=0x7f4d2c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b484a4340e60d97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:04.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d41d9b640 1 -- 172.21.15.204:0/406389071 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d2c02ad60 con 0x7f4d4409f950 2024-09-15T01:37:04.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d41d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:04.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d41d9b640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:04.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d41d9b640 10 monclient: dump: 2024-09-15T01:37:04.702 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:04.702 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:04.702 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:04.702 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:04.703 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:04.703 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:04.703 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:04.703 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:04.703 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:04.703 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:04.703 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:04.703 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:04.703 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:04.703 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:04.703 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:04.704 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:04.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d41d9b640 10 monclient: _finish_auth 0 2024-09-15T01:37:04.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d41d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:04.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d41d9b640 1 -- 172.21.15.204:0/406389071 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c031820 con 0x7f4d4409f950 2024-09-15T01:37:04.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d48954640 5 monclient: authenticate success, global_id 6967 2024-09-15T01:37:04.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d48954640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:04.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d41d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:04.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d48954640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:04.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d48954640 4 set_mon_vals no callback set 2024-09-15T01:37:04.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d41d9b640 1 -- 172.21.15.204:0/406389071 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d2c031e90 con 0x7f4d4409f950 2024-09-15T01:37:04.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d41d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d41d9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.687+0000 7f4d41d9b640 10 monclient: dump: 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 10 monclient: shutdown 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 1 -- 172.21.15.204:0/406389071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d4409f950 msgr2=0x7f4d4409fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 1 --2- 172.21.15.204:0/406389071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d4409f950 0x7f4d4409fd60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009910 tx=0x7f4d2c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 1 -- 172.21.15.204:0/406389071 shutdown_connections 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 1 --2- 172.21.15.204:0/406389071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d4409f950 0x7f4d4409fd60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 1 --2- 172.21.15.204:0/406389071 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d440a08e0 0x7f4d440a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 1 --2- 172.21.15.204:0/406389071 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4d440a2440 0x7f4d44132990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 1 -- 172.21.15.204:0/406389071 >> 172.21.15.204:0/406389071 conn(0x7f4d4406e860 msgr2=0x7f4d4406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 1 -- 172.21.15.204:0/406389071 shutdown_connections 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 1 -- 172.21.15.204:0/406389071 wait complete. 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 10 monclient: build_initial_monmap 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.688+0000 7f4d48954640 10 monclient: monmap: 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 1 Processor -- start 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 1 -- start start 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 10 monclient: init 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 10 monclient: _add_conns ranks=[1,7,8,6,4,3,0,5,2] 2024-09-15T01:37:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d4409f950 0x7f4d441317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 10 monclient(hunting): picked mon.noname-f con 0x7f4d4409f950 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4d440a08e0 0x7f4d4411e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 10 monclient(hunting): picked mon.noname-i con 0x7f4d440a08e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4d440a2440 0x7f4d4411e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 10 monclient(hunting): picked mon.noname-e con 0x7f4d440a2440 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d44120680 con 0x7f4d4409f950 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4409ca40 con 0x7f4d440a08e0 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4409d4d0 con 0x7f4d440a2440 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d48954640 10 monclient(hunting): authenticate will time out at 1040.331723s 2024-09-15T01:37:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d4259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d4409f950 0x7f4d441317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d4259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d4409f950 0x7f4d441317c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44830/0 (socket says 172.21.15.204:44830) 2024-09-15T01:37:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d4259c640 1 -- 172.21.15.204:0/2885134666 learned_addr learned my addr 172.21.15.204:0/2885134666 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d4259c640 10 monclient(hunting): get_auth_request con 0x7f4d4409f950 auth_method 0 2024-09-15T01:37:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d3bfff640 1 --2- 172.21.15.204:0/2885134666 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4d440a08e0 0x7f4d4411e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d4259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d4259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d4259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d42d9d640 1 -- 172.21.15.204:0/2885134666 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4d440a2440 msgr2=0x7f4d4411e930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d3bfff640 10 monclient(hunting): get_auth_request con 0x7f4d440a08e0 auth_method 0 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d42d9d640 1 --2- 172.21.15.204:0/2885134666 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4d440a2440 0x7f4d4411e930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d4259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d4259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d4259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.689+0000 7f4d3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d4259c640 10 monclient(hunting): handle_auth_done global_id 6976 payload 824 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d4259c640 1 -- 172.21.15.204:0/2885134666 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4d440a2440 msgr2=0x7f4d4411e930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d4259c640 1 --2- 172.21.15.204:0/2885134666 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4d440a2440 0x7f4d4411e930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d4259c640 1 -- 172.21.15.204:0/2885134666 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4d440a08e0 msgr2=0x7f4d4411e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d4259c640 1 --2- 172.21.15.204:0/2885134666 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4d440a08e0 0x7f4d4411e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d4259c640 10 monclient: _finish_hunting 0 2024-09-15T01:37:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d4259c640 1 monclient: found mon.noname-f 2024-09-15T01:37:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d4259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d4259c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d4259c640 1 -- 172.21.15.204:0/2885134666 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c009450 con 0x7f4d4409f950 2024-09-15T01:37:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3bfff640 1 --2- 172.21.15.204:0/2885134666 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4d440a08e0 0x7f4d4411e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d4259c640 1 --2- 172.21.15.204:0/2885134666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d4409f950 0x7f4d441317c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400b5c0 tx=0x7f4d34009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3a7fc640 1 -- 172.21.15.204:0/2885134666 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d34009280 con 0x7f4d4409f950 2024-09-15T01:37:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3a7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3a7fc640 10 monclient: dump: 2024-09-15T01:37:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:04.714 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d48954640 5 monclient: authenticate success, global_id 6976 2024-09-15T01:37:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3a7fc640 1 -- 172.21.15.204:0/2885134666 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d340050f0 con 0x7f4d4409f950 2024-09-15T01:37:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d48954640 10 monclient: _renew_subs 2024-09-15T01:37:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d48954640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d48954640 1 -- 172.21.15.204:0/2885134666 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4411fe50 con 0x7f4d4409f950 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3a7fc640 1 -- 172.21.15.204:0/2885134666 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d34005720 con 0x7f4d4409f950 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d4159a640 4 set_mon_vals no callback set 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d3a7fc640 10 monclient: dump: 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:04.716 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d48954640 10 monclient: _renew_subs 2024-09-15T01:37:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d48954640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.690+0000 7f4d48954640 1 -- 172.21.15.204:0/2885134666 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d44120a20 con 0x7f4d4409f950 2024-09-15T01:37:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.691+0000 7f4d3a7fc640 1 -- 172.21.15.204:0/2885134666 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4d340052a0 con 0x7f4d4409f950 2024-09-15T01:37:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.691+0000 7f4d48954640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.691+0000 7f4d48954640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.691+0000 7f4d48954640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.691+0000 7f4d48954640 1 -- 172.21.15.204:0/2885134666 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d44121c50 con 0x7f4d4409f950 2024-09-15T01:37:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.691+0000 7f4d3a7fc640 1 --2- 172.21.15.204:0/2885134666 >> v2:172.21.15.204:6812/28210 conn(0x7f4d240397b0 0x7f4d2403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.691+0000 7f4d3a7fc640 1 -- 172.21.15.204:0/2885134666 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 8971+0+0 (secure 0 0 0) 0x7f4d3404bf60 con 0x7f4d4409f950 2024-09-15T01:37:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.691+0000 7f4d3bfff640 1 --2- 172.21.15.204:0/2885134666 >> v2:172.21.15.204:6812/28210 conn(0x7f4d240397b0 0x7f4d2403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.691+0000 7f4d3bfff640 10 monclient: get_auth_request con 0x7f4d240397b0 auth_method 0 2024-09-15T01:37:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.692+0000 7f4d3bfff640 1 --2- 172.21.15.204:0/2885134666 >> v2:172.21.15.204:6812/28210 conn(0x7f4d240397b0 0x7f4d2403bc60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c034000 tx=0x7f4d2c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.696+0000 7f4d3a7fc640 1 -- 172.21.15.204:0/2885134666 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4d34005550 con 0x7f4d4409f950 2024-09-15T01:37:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.696+0000 7f4d3a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:04.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.696+0000 7f4d3a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:04.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.853+0000 7f4d48954640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:04.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.853+0000 7f4d48954640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:04.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.853+0000 7f4d48954640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:04.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:04.853+0000 7f4d48954640 1 -- 172.21.15.204:0/2885134666 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f4d44070b80 con 0x7f4d4409f950 2024-09-15T01:37:05.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.498+0000 7f4d3a7fc640 1 -- 172.21.15.204:0/2885134666 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}]=0 added bucket h10 type host to location {rack=r1} v195) v1 ==== 168+0+0 (secure 0 0 0) 0x7f4d3400f360 con 0x7f4d4409f950 2024-09-15T01:37:05.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.499+0000 7f4d3a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:05.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.499+0000 7f4d3a7fc640 10 monclient: _finish_command 2 = system:0 added bucket h10 type host to location {rack=r1} 2024-09-15T01:37:05.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.597+0000 7f4d48954640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:05.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.597+0000 7f4d48954640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:05.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.597+0000 7f4d48954640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:05.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.597+0000 7f4d48954640 1 -- 172.21.15.204:0/2885134666 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f4d4409de60 con 0x7f4d4409f950 2024-09-15T01:37:05.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.598+0000 7f4d3a7fc640 1 -- 172.21.15.204:0/2885134666 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h10' already exists v195) v1 ==== 147+0+0 (secure 0 0 0) 0x7f4d34019bb0 con 0x7f4d4409f950 2024-09-15T01:37:05.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.598+0000 7f4d3a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:05.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.598+0000 7f4d3a7fc640 10 monclient: _finish_command 3 = system:0 bucket 'h10' already exists 2024-09-15T01:37:05.600 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h10' already exists 2024-09-15T01:37:05.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.599+0000 7f4d48954640 1 -- 172.21.15.204:0/2885134666 >> v2:172.21.15.204:6812/28210 conn(0x7f4d240397b0 msgr2=0x7f4d2403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:05.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.599+0000 7f4d48954640 1 --2- 172.21.15.204:0/2885134666 >> v2:172.21.15.204:6812/28210 conn(0x7f4d240397b0 0x7f4d2403bc60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c034000 tx=0x7f4d2c032000 comp rx=0 tx=0).stop 2024-09-15T01:37:05.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.599+0000 7f4d48954640 10 monclient: shutdown 2024-09-15T01:37:05.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.599+0000 7f4d48954640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:05.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.599+0000 7f4d48954640 1 -- 172.21.15.204:0/2885134666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d4409f950 msgr2=0x7f4d441317c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:05.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.599+0000 7f4d48954640 1 --2- 172.21.15.204:0/2885134666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d4409f950 0x7f4d441317c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400b5c0 tx=0x7f4d34009120 comp rx=0 tx=0).stop 2024-09-15T01:37:05.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.599+0000 7f4d48954640 1 -- 172.21.15.204:0/2885134666 shutdown_connections 2024-09-15T01:37:05.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.599+0000 7f4d48954640 1 --2- 172.21.15.204:0/2885134666 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4d440a2440 0x7f4d4411e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:05.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.599+0000 7f4d48954640 1 --2- 172.21.15.204:0/2885134666 >> v2:172.21.15.204:6812/28210 conn(0x7f4d240397b0 0x7f4d2403bc60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:05.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.599+0000 7f4d48954640 1 --2- 172.21.15.204:0/2885134666 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4d440a08e0 0x7f4d4411e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:05.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.599+0000 7f4d48954640 1 --2- 172.21.15.204:0/2885134666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d4409f950 0x7f4d441317c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:05.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.599+0000 7f4d48954640 1 -- 172.21.15.204:0/2885134666 >> 172.21.15.204:0/2885134666 conn(0x7f4d4406e860 msgr2=0x7f4d440a3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:05.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.600+0000 7f4d48954640 1 -- 172.21.15.204:0/2885134666 shutdown_connections 2024-09-15T01:37:05.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.600+0000 7f4d48954640 1 -- 172.21.15.204:0/2885134666 wait complete. 2024-09-15T01:37:05.610 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:05.610 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h11 host rack=r1 2024-09-15T01:37:05.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.684+0000 7f56d831d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:05.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.684+0000 7f56d831d640 10 monclient: build_initial_monmap 2024-09-15T01:37:05.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.684+0000 7f56d831d640 10 monclient: monmap: 2024-09-15T01:37:05.686 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:05.686 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:05.686 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:05.684666+0000 2024-09-15T01:37:05.686 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:05.684666+0000 2024-09-15T01:37:05.686 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:05.686 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:05.686 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:05.686 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:05.686 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:05.687 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:05.687 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:05.687 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:05.687 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:05.687 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:05.687 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:05.687 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:05.687 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:05.687 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:05.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.684+0000 7f56d831d640 1 Processor -- start 2024-09-15T01:37:05.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.684+0000 7f56d831d640 1 -- start start 2024-09-15T01:37:05.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.684+0000 7f56d831d640 10 monclient: init 2024-09-15T01:37:05.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:05.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 10 monclient: _add_conns ranks=[4,1,5,0,3,6,7,10,9,2,8] 2024-09-15T01:37:05.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56d00a24a0 0x7f56d0132b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:05.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 10 monclient(hunting): picked mon.noname-c con 0x7f56d00a24a0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:05.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f56d009f9b0 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:05.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56d0137c60 0x7f56d013a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 10 monclient(hunting): picked mon.noname-h con 0x7f56d0137c60 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d00648a0 con 0x7f56d009f9b0 2024-09-15T01:37:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00a0490 con 0x7f56d00a24a0 2024-09-15T01:37:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00a2870 con 0x7f56d0137c60 2024-09-15T01:37:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d831d640 10 monclient(hunting): authenticate will time out at 1041.327490s 2024-09-15T01:37:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d58b8640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f56d009f9b0 0x7f56d009fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:59926/0 (socket says 172.21.15.204:59926) 2024-09-15T01:37:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d58b8640 1 -- 172.21.15.204:0/2115472815 learned_addr learned my addr 172.21.15.204:0/2115472815 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:37:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 1 --2- 172.21.15.204:0/2115472815 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56d0137c60 0x7f56d013a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 10 monclient(hunting): get_auth_request con 0x7f56d0137c60 auth_method 0 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d60b9640 1 -- 172.21.15.204:0/2115472815 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56d00a24a0 msgr2=0x7f56d0132b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d60b9640 1 --2- 172.21.15.204:0/2115472815 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56d00a24a0 0x7f56d0132b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d50b7640 1 -- 172.21.15.204:0/2115472815 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812149395 0 0) 0x7f56d00648a0 con 0x7f56d009f9b0 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d50b7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d50b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d50b7640 10 monclient(hunting): my global_id is 6985 2024-09-15T01:37:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d50b7640 1 -- 172.21.15.204:0/2115472815 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56bc0034e0 con 0x7f56d009f9b0 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 10 monclient(hunting): handle_auth_done global_id 6215 payload 274 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 1 -- 172.21.15.204:0/2115472815 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56d00a24a0 msgr2=0x7f56d0132b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 1 --2- 172.21.15.204:0/2115472815 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56d00a24a0 0x7f56d0132b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 1 -- 172.21.15.204:0/2115472815 >> v1:172.21.15.204:6789/0 conn(0x7f56d009f9b0 legacy=0x7f56d009fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 10 monclient: _finish_hunting 0 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 1 monclient: found mon.noname-h 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.685+0000 7f56d68ba640 1 -- 172.21.15.204:0/2115472815 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d00a1bd0 con 0x7f56d0137c60 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d68ba640 1 --2- 172.21.15.204:0/2115472815 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56d0137c60 0x7f56d013a040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009e90 tx=0x7f56c8009780 comp rx=0 tx=0).ready entity=mon.5 client_cookie=aa4f75e7e45088f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d50b7640 1 -- 172.21.15.204:0/2115472815 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f56c80079d0 con 0x7f56d0137c60 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d50b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d50b7640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d50b7640 10 monclient: dump: 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:05.699 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:05.700 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:05.700 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:05.700 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:05.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d50b7640 10 monclient: _finish_auth 0 2024-09-15T01:37:05.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d50b7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:05.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d831d640 5 monclient: authenticate success, global_id 6215 2024-09-15T01:37:05.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d50b7640 1 -- 172.21.15.204:0/2115472815 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c802d050 con 0x7f56d0137c60 2024-09-15T01:37:05.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d831d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:05.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d50b7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:05.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d831d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:05.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d831d640 4 set_mon_vals no callback set 2024-09-15T01:37:05.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d50b7640 1 -- 172.21.15.204:0/2115472815 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f56c8007b90 con 0x7f56d0137c60 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d50b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d50b7640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.686+0000 7f56d50b7640 10 monclient: dump: 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:05.701 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.687+0000 7f56d831d640 10 monclient: shutdown 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.687+0000 7f56d831d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.687+0000 7f56d831d640 1 -- 172.21.15.204:0/2115472815 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56d0137c60 msgr2=0x7f56d013a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.687+0000 7f56d831d640 1 --2- 172.21.15.204:0/2115472815 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56d0137c60 0x7f56d013a040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009e90 tx=0x7f56c8009780 comp rx=0 tx=0).stop 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.687+0000 7f56d831d640 1 -- 172.21.15.204:0/2115472815 shutdown_connections 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.687+0000 7f56d831d640 1 --2- 172.21.15.204:0/2115472815 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f56d0137c60 0x7f56d013a040 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:05.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.687+0000 7f56d831d640 1 --2- 172.21.15.204:0/2115472815 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56d00a24a0 0x7f56d0132b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.687+0000 7f56d831d640 1 -- 172.21.15.204:0/2115472815 >> 172.21.15.204:0/2115472815 conn(0x7f56d00a5f70 msgr2=0x7f56d00a6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.687+0000 7f56d831d640 1 -- 172.21.15.204:0/2115472815 shutdown_connections 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.687+0000 7f56d831d640 1 -- 172.21.15.204:0/2115472815 wait complete. 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.687+0000 7f56d831d640 10 monclient: build_initial_monmap 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.687+0000 7f56d831d640 10 monclient: monmap: 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:05.703 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 1 Processor -- start 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 1 -- start start 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 10 monclient: init 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:05.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 10 monclient: _add_conns ranks=[2,0,6,1,7,8,5,3,4] 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d009f9b0 0x7f56d009c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 10 monclient(hunting): picked mon.noname-b con 0x7f56d009f9b0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56d00a24a0 0x7f56d0135f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 10 monclient(hunting): picked mon.noname-a con 0x7f56d00a24a0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f56d0137c60 0x7f56d0136490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 10 monclient(hunting): picked mon.noname-d con 0x7f56d0137c60 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d009d470 con 0x7f56d00a24a0 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d0096a70 con 0x7f56d009f9b0 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f56d0096e50 con 0x7f56d0137c60 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:05.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d831d640 10 monclient(hunting): authenticate will time out at 1041.330706s 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d60b9640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d009f9b0 msgr2=0x7f56d009c3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d58b8640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56d00a24a0 msgr2=0x7f56d0135f50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d60b9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d009f9b0 0x7f56d009c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d58b8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56d00a24a0 0x7f56d0135f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.688+0000 7f56d68ba640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f56d0137c60 0x7f56d0136490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f56d0137c60 0x7f56d0136490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:47888/0 (socket says 172.21.15.204:47888) 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 1 -- 172.21.15.204:0/3615657379 learned_addr learned my addr 172.21.15.204:0/3615657379 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 10 monclient(hunting): get_auth_request con 0x7f56d0137c60 auth_method 0 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 10 monclient(hunting): handle_auth_done global_id 24432 payload 824 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 1 -- 172.21.15.204:0/3615657379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d009f9b0 msgr2=0x7f56d009c3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 1 --2- 172.21.15.204:0/3615657379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d009f9b0 0x7f56d009c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 1 -- 172.21.15.204:0/3615657379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56d00a24a0 msgr2=0x7f56d0135f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 1 --2- 172.21.15.204:0/3615657379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56d00a24a0 0x7f56d0135f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 10 monclient: _finish_hunting 0 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 1 monclient: found mon.noname-d 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 1 -- 172.21.15.204:0/3615657379 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c8009a20 con 0x7f56d0137c60 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d68ba640 1 --2- 172.21.15.204:0/3615657379 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f56d0137c60 0x7f56d0136490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009d30 tx=0x7f56c8037000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56c67fc640 1 -- 172.21.15.204:0/3615657379 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f56c803d430 con 0x7f56d0137c60 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56c67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56c67fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56c67fc640 10 monclient: dump: 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56c67fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56c67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56c67fc640 1 -- 172.21.15.204:0/3615657379 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c803d5b0 con 0x7f56d0137c60 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d831d640 5 monclient: authenticate success, global_id 24432 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56c67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d831d640 10 monclient: _renew_subs 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56c67fc640 1 -- 172.21.15.204:0/3615657379 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f56c803dba0 con 0x7f56d0137c60 2024-09-15T01:37:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d831d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d831d640 1 -- 172.21.15.204:0/3615657379 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d0098470 con 0x7f56d0137c60 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56c67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d48b6640 4 set_mon_vals no callback set 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56c67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56c67fc640 10 monclient: dump: 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d831d640 10 monclient: _renew_subs 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d831d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.689+0000 7f56d831d640 1 -- 172.21.15.204:0/3615657379 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d009c910 con 0x7f56d0137c60 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.690+0000 7f56c67fc640 1 -- 172.21.15.204:0/3615657379 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f56c8003040 con 0x7f56d0137c60 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.690+0000 7f56d831d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.690+0000 7f56d831d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.690+0000 7f56d831d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.690+0000 7f56d831d640 1 -- 172.21.15.204:0/3615657379 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d0098870 con 0x7f56d0137c60 2024-09-15T01:37:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.691+0000 7f56c67fc640 1 --2- 172.21.15.204:0/3615657379 >> v2:172.21.15.204:6812/28210 conn(0x7f56b40397b0 0x7f56b403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.691+0000 7f56c67fc640 1 -- 172.21.15.204:0/3615657379 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 9184+0+0 (secure 0 0 0) 0x7f56c806dbd0 con 0x7f56d0137c60 2024-09-15T01:37:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.691+0000 7f56d60b9640 1 --2- 172.21.15.204:0/3615657379 >> v2:172.21.15.204:6812/28210 conn(0x7f56b40397b0 0x7f56b403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.691+0000 7f56d60b9640 10 monclient: get_auth_request con 0x7f56b40397b0 auth_method 0 2024-09-15T01:37:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.691+0000 7f56d60b9640 1 --2- 172.21.15.204:0/3615657379 >> v2:172.21.15.204:6812/28210 conn(0x7f56b40397b0 0x7f56b403bc60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f56cc006fd0 tx=0x7f56cc006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.695+0000 7f56c67fc640 1 -- 172.21.15.204:0/3615657379 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f56c8031ac0 con 0x7f56d0137c60 2024-09-15T01:37:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.696+0000 7f56c67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.696+0000 7f56c67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:05.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.853+0000 7f56d831d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:05.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.854+0000 7f56d831d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:05.855 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.854+0000 7f56d831d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:05.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:05.854+0000 7f56d831d640 1 -- 172.21.15.204:0/3615657379 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f56d00298c0 con 0x7f56d0137c60 2024-09-15T01:37:06.523 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.521+0000 7f56c67fc640 1 -- 172.21.15.204:0/3615657379 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}]=0 added bucket h11 type host to location {rack=r1} v196) v1 ==== 168+0+0 (secure 0 0 0) 0x7f56c80303a0 con 0x7f56d0137c60 2024-09-15T01:37:06.523 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.521+0000 7f56c67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:06.523 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.521+0000 7f56c67fc640 10 monclient: _finish_command 2 = system:0 added bucket h11 type host to location {rack=r1} 2024-09-15T01:37:06.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.617+0000 7f56d831d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:06.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.618+0000 7f56d831d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:06.619 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.618+0000 7f56d831d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:06.620 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.618+0000 7f56d831d640 1 -- 172.21.15.204:0/3615657379 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f56d009caa0 con 0x7f56d0137c60 2024-09-15T01:37:06.620 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.619+0000 7f56c67fc640 1 -- 172.21.15.204:0/3615657379 <== mon.6 v2:172.21.15.178:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h11' already exists v196) v1 ==== 147+0+0 (secure 0 0 0) 0x7f56c8030580 con 0x7f56d0137c60 2024-09-15T01:37:06.620 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.619+0000 7f56c67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:06.620 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.619+0000 7f56c67fc640 10 monclient: _finish_command 3 = system:0 bucket 'h11' already exists 2024-09-15T01:37:06.620 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h11' already exists 2024-09-15T01:37:06.622 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 1 -- 172.21.15.204:0/3615657379 >> v2:172.21.15.204:6812/28210 conn(0x7f56b40397b0 msgr2=0x7f56b403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:06.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 1 --2- 172.21.15.204:0/3615657379 >> v2:172.21.15.204:6812/28210 conn(0x7f56b40397b0 0x7f56b403bc60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f56cc006fd0 tx=0x7f56cc006d30 comp rx=0 tx=0).stop 2024-09-15T01:37:06.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 10 monclient: shutdown 2024-09-15T01:37:06.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:06.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 1 -- 172.21.15.204:0/3615657379 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f56d0137c60 msgr2=0x7f56d0136490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:06.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 1 --2- 172.21.15.204:0/3615657379 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f56d0137c60 0x7f56d0136490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009d30 tx=0x7f56c8037000 comp rx=0 tx=0).stop 2024-09-15T01:37:06.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 1 -- 172.21.15.204:0/3615657379 shutdown_connections 2024-09-15T01:37:06.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 1 --2- 172.21.15.204:0/3615657379 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f56d0137c60 0x7f56d0136490 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:06.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 1 --2- 172.21.15.204:0/3615657379 >> v2:172.21.15.204:6812/28210 conn(0x7f56b40397b0 0x7f56b403bc60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:06.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 1 --2- 172.21.15.204:0/3615657379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56d00a24a0 0x7f56d0135f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:06.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 1 --2- 172.21.15.204:0/3615657379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d009f9b0 0x7f56d009c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:06.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 1 -- 172.21.15.204:0/3615657379 >> 172.21.15.204:0/3615657379 conn(0x7f56d00a5f70 msgr2=0x7f56d012fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:06.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 1 -- 172.21.15.204:0/3615657379 shutdown_connections 2024-09-15T01:37:06.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.621+0000 7f56d831d640 1 -- 172.21.15.204:0/3615657379 wait complete. 2024-09-15T01:37:06.630 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:06.630 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h12 host rack=r1 2024-09-15T01:37:06.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.705+0000 7fe6624c0640 10 monclient: get_monmap_and_config 2024-09-15T01:37:06.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.705+0000 7fe6624c0640 10 monclient: build_initial_monmap 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.705+0000 7fe6624c0640 10 monclient: monmap: 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:06.705686+0000 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:06.705686+0000 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:06.707 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:06.708 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:06.708 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:06.708 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:06.708 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:06.708 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:06.708 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:06.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.705+0000 7fe6624c0640 1 Processor -- start 2024-09-15T01:37:06.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 1 -- start start 2024-09-15T01:37:06.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 10 monclient: init 2024-09-15T01:37:06.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:06.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 10 monclient: _add_conns ranks=[7,2,6,9,10,1,5,8,0,4,3] 2024-09-15T01:37:06.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe65c067130 0x7fe65c130e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:06.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 10 monclient(hunting): picked mon.noname-i con 0x7fe65c067130 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:06.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe65c067500 0x7fe65c0678d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:06.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 10 monclient(hunting): picked mon.noname-b con 0x7fe65c067500 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:06.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe65c12d450 0x7fe65c12d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:06.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 10 monclient(hunting): picked mon.noname-d con 0x7fe65c12d450 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:06.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:06.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c09f150 con 0x7fe65c067500 2024-09-15T01:37:06.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:06.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c09ee70 con 0x7fe65c12d450 2024-09-15T01:37:06.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:06.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c09f430 con 0x7fe65c067130 2024-09-15T01:37:06.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:06.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe6624c0640 10 monclient(hunting): authenticate will time out at 1042.348567s 2024-09-15T01:37:06.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65b7fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe65c067500 msgr2=0x7fe65c0678d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:06.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe65c067130 0x7fe65c130e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:06.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65b7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe65c067500 0x7fe65c0678d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe65c067130 0x7fe65c130e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:55846/0 (socket says 172.21.15.204:55846) 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 1 -- 172.21.15.204:0/3531924273 learned_addr learned my addr 172.21.15.204:0/3531924273 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 10 monclient(hunting): get_auth_request con 0x7fe65c067130 auth_method 0 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe660a5d640 1 --2- 172.21.15.204:0/3531924273 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe65c12d450 0x7fe65c12d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe660a5d640 10 monclient(hunting): get_auth_request con 0x7fe65c12d450 auth_method 0 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe660a5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe660a5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:06.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe660a5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 10 monclient(hunting): handle_auth_done global_id 24478 payload 274 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 1 -- 172.21.15.204:0/3531924273 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe65c12d450 msgr2=0x7fe65c12d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 1 --2- 172.21.15.204:0/3531924273 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe65c12d450 0x7fe65c12d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 1 -- 172.21.15.204:0/3531924273 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe65c067500 msgr2=0x7fe65c0678d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 1 --2- 172.21.15.204:0/3531924273 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe65c067500 0x7fe65c0678d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 1 monclient: found mon.noname-i 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.706+0000 7fe65bfff640 1 -- 172.21.15.204:0/3531924273 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c0648a0 con 0x7fe65c067130 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65bfff640 1 --2- 172.21.15.204:0/3531924273 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe65c067130 0x7fe65c130e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009940 tx=0x7fe64c0091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=eb7c9550526f1e75 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65affd640 1 -- 172.21.15.204:0/3531924273 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe64c02ac80 con 0x7fe65c067130 2024-09-15T01:37:06.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65affd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65affd640 10 monclient: dump: 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:06.720 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65affd640 10 monclient: _finish_auth 0 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65affd640 1 -- 172.21.15.204:0/3531924273 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c02ae00 con 0x7fe65c067130 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe6624c0640 5 monclient: authenticate success, global_id 24478 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe6624c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe6624c0640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65affd640 1 -- 172.21.15.204:0/3531924273 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe64c031ca0 con 0x7fe65c067130 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe6624c0640 4 set_mon_vals no callback set 2024-09-15T01:37:06.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.707+0000 7fe65affd640 10 monclient: dump: 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:06.722 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 10 monclient: shutdown 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 1 -- 172.21.15.204:0/3531924273 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe65c067130 msgr2=0x7fe65c130e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 1 --2- 172.21.15.204:0/3531924273 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe65c067130 0x7fe65c130e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009940 tx=0x7fe64c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 1 -- 172.21.15.204:0/3531924273 shutdown_connections 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 1 --2- 172.21.15.204:0/3531924273 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe65c12d450 0x7fe65c12d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 1 --2- 172.21.15.204:0/3531924273 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe65c067500 0x7fe65c0678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 1 --2- 172.21.15.204:0/3531924273 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe65c067130 0x7fe65c130e10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:06.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 1 -- 172.21.15.204:0/3531924273 >> 172.21.15.204:0/3531924273 conn(0x7fe65c0a2cb0 msgr2=0x7fe65c0a30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 1 -- 172.21.15.204:0/3531924273 shutdown_connections 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 1 -- 172.21.15.204:0/3531924273 wait complete. 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 10 monclient: build_initial_monmap 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.708+0000 7fe6624c0640 10 monclient: monmap: 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:06.724 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 1 Processor -- start 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 1 -- start start 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 10 monclient: init 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 10 monclient: _add_conns ranks=[5,1,4,7,0,8,6,2,3] 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe65c067130 0x7fe65c119d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 10 monclient(hunting): picked mon.noname-h con 0x7fe65c067130 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:06.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c067500 0x7fe65c11a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 10 monclient(hunting): picked mon.noname-f con 0x7fe65c067500 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe65c12d450 0x7fe65c11a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 10 monclient(hunting): picked mon.noname-c con 0x7fe65c12d450 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c11b630 con 0x7fe65c067500 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c09e5f0 con 0x7fe65c12d450 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c09e230 con 0x7fe65c067130 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.709+0000 7fe6624c0640 10 monclient(hunting): authenticate will time out at 1042.351994s 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe65c067130 0x7fe65c119d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe660a5d640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe65c12d450 msgr2=0x7fe65c11a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:06.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c067500 0x7fe65c11a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe660a5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe65c12d450 0x7fe65c11a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe65c067130 0x7fe65c119d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:50040/0 (socket says 172.21.15.204:50040) 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65b7fe640 10 monclient(hunting): get_auth_request con 0x7fe65c067500 auth_method 0 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 1 -- 172.21.15.204:0/2055066593 learned_addr learned my addr 172.21.15.204:0/2055066593 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 10 monclient(hunting): get_auth_request con 0x7fe65c067130 auth_method 0 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:06.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 10 monclient(hunting): handle_auth_done global_id 6224 payload 824 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 1 -- 172.21.15.204:0/2055066593 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe65c12d450 msgr2=0x7fe65c11a7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 1 --2- 172.21.15.204:0/2055066593 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe65c12d450 0x7fe65c11a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 1 -- 172.21.15.204:0/2055066593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c067500 msgr2=0x7fe65c11a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 1 --2- 172.21.15.204:0/2055066593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c067500 0x7fe65c11a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 1 monclient: found mon.noname-h 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:06.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 1 -- 172.21.15.204:0/2055066593 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c009450 con 0x7fe65c067130 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65b7fe640 1 --2- 172.21.15.204:0/2055066593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c067500 0x7fe65c11a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65bfff640 1 --2- 172.21.15.204:0/2055066593 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe65c067130 0x7fe65c119d70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0097e0 tx=0x7fe64c032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe653fff640 1 -- 172.21.15.204:0/2055066593 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe64c02ac80 con 0x7fe65c067130 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe653fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe653fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe653fff640 10 monclient: dump: 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:06.729 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe653fff640 10 monclient: _finish_auth 0 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe653fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe653fff640 1 -- 172.21.15.204:0/2055066593 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c033e30 con 0x7fe65c067130 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe6624c0640 5 monclient: authenticate success, global_id 6224 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe653fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:06.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe653fff640 1 -- 172.21.15.204:0/2055066593 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe64c03a810 con 0x7fe65c067130 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe653fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe653fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe653fff640 10 monclient: dump: 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:06.731 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe65a7fc640 4 set_mon_vals no callback set 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe6624c0640 10 monclient: _renew_subs 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe6624c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe6624c0640 1 -- 172.21.15.204:0/2055066593 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c11d0d0 con 0x7fe65c067130 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe6624c0640 10 monclient: _renew_subs 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe6624c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.710+0000 7fe6624c0640 1 -- 172.21.15.204:0/2055066593 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c11c170 con 0x7fe65c067130 2024-09-15T01:37:06.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.711+0000 7fe653fff640 1 -- 172.21.15.204:0/2055066593 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe64c039040 con 0x7fe65c067130 2024-09-15T01:37:06.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.711+0000 7fe6624c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:06.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.711+0000 7fe6624c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:06.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.711+0000 7fe6624c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:06.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.711+0000 7fe653fff640 1 --2- 172.21.15.204:0/2055066593 >> v2:172.21.15.204:6812/28210 conn(0x7fe63c0397b0 0x7fe63c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:06.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.711+0000 7fe6624c0640 1 -- 172.21.15.204:0/2055066593 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65c11d5f0 con 0x7fe65c067130 2024-09-15T01:37:06.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.711+0000 7fe653fff640 1 -- 172.21.15.204:0/2055066593 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 9397+0+0 (secure 0 0 0) 0x7fe64c06bfc0 con 0x7fe65c067130 2024-09-15T01:37:06.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.711+0000 7fe65b7fe640 1 --2- 172.21.15.204:0/2055066593 >> v2:172.21.15.204:6812/28210 conn(0x7fe63c0397b0 0x7fe63c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:06.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.711+0000 7fe65b7fe640 10 monclient: get_auth_request con 0x7fe63c0397b0 auth_method 0 2024-09-15T01:37:06.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.712+0000 7fe65b7fe640 1 --2- 172.21.15.204:0/2055066593 >> v2:172.21.15.204:6812/28210 conn(0x7fe63c0397b0 0x7fe63c03bc60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe6440038f0 tx=0x7fe6440090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:06.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.716+0000 7fe653fff640 1 -- 172.21.15.204:0/2055066593 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe65c11d5f0 con 0x7fe65c067130 2024-09-15T01:37:06.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.716+0000 7fe653fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:06.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.716+0000 7fe653fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:06.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.875+0000 7fe6624c0640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:06.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.875+0000 7fe6624c0640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:06.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.875+0000 7fe6624c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:06.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:06.875+0000 7fe6624c0640 1 -- 172.21.15.204:0/2055066593 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fe65c11d5f0 con 0x7fe65c067130 2024-09-15T01:37:07.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.527+0000 7fe653fff640 1 -- 172.21.15.204:0/2055066593 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}]=0 added bucket h12 type host to location {rack=r1} v197) v1 ==== 168+0+0 (secure 0 0 0) 0x7fe65c11d5f0 con 0x7fe65c067130 2024-09-15T01:37:07.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.527+0000 7fe653fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:07.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.527+0000 7fe653fff640 10 monclient: _finish_command 2 = system:0 added bucket h12 type host to location {rack=r1} 2024-09-15T01:37:07.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.622+0000 7fe6624c0640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:07.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.622+0000 7fe6624c0640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:07.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.622+0000 7fe6624c0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:07.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.622+0000 7fe6624c0640 1 -- 172.21.15.204:0/2055066593 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fe65c11d260 con 0x7fe65c067130 2024-09-15T01:37:07.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.623+0000 7fe653fff640 1 -- 172.21.15.204:0/2055066593 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h12' already exists v197) v1 ==== 147+0+0 (secure 0 0 0) 0x7fe65c11d260 con 0x7fe65c067130 2024-09-15T01:37:07.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.623+0000 7fe653fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:07.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.623+0000 7fe653fff640 10 monclient: _finish_command 3 = system:0 bucket 'h12' already exists 2024-09-15T01:37:07.625 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h12' already exists 2024-09-15T01:37:07.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 1 -- 172.21.15.204:0/2055066593 >> v2:172.21.15.204:6812/28210 conn(0x7fe63c0397b0 msgr2=0x7fe63c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:07.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 1 --2- 172.21.15.204:0/2055066593 >> v2:172.21.15.204:6812/28210 conn(0x7fe63c0397b0 0x7fe63c03bc60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe6440038f0 tx=0x7fe6440090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 10 monclient: shutdown 2024-09-15T01:37:07.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:07.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 1 -- 172.21.15.204:0/2055066593 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe65c067130 msgr2=0x7fe65c119d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:07.627 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 1 --2- 172.21.15.204:0/2055066593 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe65c067130 0x7fe65c119d70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0097e0 tx=0x7fe64c032000 comp rx=0 tx=0).stop 2024-09-15T01:37:07.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 1 -- 172.21.15.204:0/2055066593 shutdown_connections 2024-09-15T01:37:07.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 1 --2- 172.21.15.204:0/2055066593 >> v2:172.21.15.204:6812/28210 conn(0x7fe63c0397b0 0x7fe63c03bc60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 1 --2- 172.21.15.204:0/2055066593 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe65c12d450 0x7fe65c11a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 1 --2- 172.21.15.204:0/2055066593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c067500 0x7fe65c11a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 1 --2- 172.21.15.204:0/2055066593 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe65c067130 0x7fe65c119d70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 1 -- 172.21.15.204:0/2055066593 >> 172.21.15.204:0/2055066593 conn(0x7fe65c0a2cb0 msgr2=0x7fe65c0a1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:07.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 1 -- 172.21.15.204:0/2055066593 shutdown_connections 2024-09-15T01:37:07.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.625+0000 7fe6624c0640 1 -- 172.21.15.204:0/2055066593 wait complete. 2024-09-15T01:37:07.635 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:07.635 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h13 host rack=r1 2024-09-15T01:37:07.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.709+0000 7f8b9c99d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.709+0000 7f8b9c99d640 10 monclient: build_initial_monmap 2024-09-15T01:37:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.709+0000 7f8b9c99d640 10 monclient: monmap: 2024-09-15T01:37:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:07.710327+0000 2024-09-15T01:37:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:07.710327+0000 2024-09-15T01:37:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:07.712 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:07.713 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:07.713 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.710+0000 7f8b9c99d640 1 Processor -- start 2024-09-15T01:37:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.710+0000 7f8b9c99d640 1 -- start start 2024-09-15T01:37:07.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.710+0000 7f8b9c99d640 10 monclient: init 2024-09-15T01:37:07.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:07.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 10 monclient: _add_conns ranks=[5,7,10,4,1,6,2,9,3,8,0] 2024-09-15T01:37:07.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 0x7f8b98132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:07.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 10 monclient(hunting): picked mon.noname-h con 0x7f8b980a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:07.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b9809fa20 0x7f8b9809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 10 monclient(hunting): picked mon.noname-i con 0x7f8b9809fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8b980a08e0 0x7f8b980a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 10 monclient(hunting): picked mon.noname-f con 0x7f8b980a08e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b980a30a0 con 0x7f8b980a08e0 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980a0330 con 0x7f8b980a2cd0 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980a1db0 con 0x7f8b9809fa20 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9c99d640 10 monclient(hunting): authenticate will time out at 1043.353139s 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b95d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b9809fa20 0x7f8b9809fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:07.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 0x7f8b98132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:07.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b96d9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8b980a08e0 0x7f8b980a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:07.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 0x7f8b98132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:40888/0 (socket says 172.21.15.204:40888) 2024-09-15T01:37:07.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b96d9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8b980a08e0 0x7f8b980a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41812/0 (socket says 172.21.15.204:41812) 2024-09-15T01:37:07.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 1 -- 172.21.15.204:0/2804756110 learned_addr learned my addr 172.21.15.204:0/2804756110 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:07.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b95d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b9809fa20 0x7f8b9809fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:55120/0 (socket says 172.21.15.204:55120) 2024-09-15T01:37:07.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b95d9b640 10 monclient(hunting): get_auth_request con 0x7f8b9809fa20 auth_method 0 2024-09-15T01:37:07.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b95d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:07.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b95d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:07.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b95d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:07.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 10 monclient(hunting): get_auth_request con 0x7f8b980a2cd0 auth_method 0 2024-09-15T01:37:07.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:07.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:07.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:07.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b96d9d640 10 monclient(hunting): get_auth_request con 0x7f8b980a08e0 auth_method 0 2024-09-15T01:37:07.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b96d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:07.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b96d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:07.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b96d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:07.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b95d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:07.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b95d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:07.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b95d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:07.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:07.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:07.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b96d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b96d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b96d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 10 monclient(hunting): handle_auth_done global_id 6233 payload 274 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 1 -- 172.21.15.204:0/2804756110 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b9809fa20 msgr2=0x7f8b9809fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 1 --2- 172.21.15.204:0/2804756110 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b9809fa20 0x7f8b9809fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 1 -- 172.21.15.204:0/2804756110 >> v2:172.21.15.204:3300/0 conn(0x7f8b980a08e0 msgr2=0x7f8b980a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 1 --2- 172.21.15.204:0/2804756110 >> v2:172.21.15.204:3300/0 conn(0x7f8b980a08e0 0x7f8b980a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 10 monclient: _finish_hunting 0 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 1 monclient: found mon.noname-h 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:07.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b9659c640 1 -- 172.21.15.204:0/2804756110 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b980648a0 con 0x7f8b980a2cd0 2024-09-15T01:37:07.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b95d9b640 1 --2- 172.21.15.204:0/2804756110 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b9809fa20 0x7f8b9809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:07.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.711+0000 7f8b96d9d640 1 --2- 172.21.15.204:0/2804756110 >> v2:172.21.15.204:3300/0 conn(0x7f8b980a08e0 0x7f8b980a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:07.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9659c640 1 --2- 172.21.15.204:0/2804756110 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 0x7f8b98132c00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009910 tx=0x7f8b880091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=3561da0d00de68d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:07.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9559a640 1 -- 172.21.15.204:0/2804756110 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b8802ac80 con 0x7f8b980a2cd0 2024-09-15T01:37:07.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:07.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9559a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:07.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9559a640 10 monclient: dump: 2024-09-15T01:37:07.727 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:07.727 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:07.727 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:07.727 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:07.727 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9559a640 10 monclient: _finish_auth 0 2024-09-15T01:37:07.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:07.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9c99d640 5 monclient: authenticate success, global_id 6233 2024-09-15T01:37:07.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9c99d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:07.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9559a640 1 -- 172.21.15.204:0/2804756110 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8802ae00 con 0x7f8b980a2cd0 2024-09-15T01:37:07.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9559a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:07.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9c99d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:07.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9c99d640 4 set_mon_vals no callback set 2024-09-15T01:37:07.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9559a640 1 -- 172.21.15.204:0/2804756110 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b88031ca0 con 0x7f8b980a2cd0 2024-09-15T01:37:07.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:07.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9559a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:07.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9559a640 10 monclient: dump: 2024-09-15T01:37:07.729 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:07.730 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:07.730 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:07.730 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:07.730 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:07.730 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:07.730 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:07.730 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:07.730 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:07.730 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:07.730 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:07.730 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:07.730 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9c99d640 10 monclient: shutdown 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9c99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9c99d640 1 -- 172.21.15.204:0/2804756110 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 msgr2=0x7f8b98132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.712+0000 7f8b9c99d640 1 --2- 172.21.15.204:0/2804756110 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 0x7f8b98132c00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009910 tx=0x7f8b880091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.713+0000 7f8b9c99d640 1 -- 172.21.15.204:0/2804756110 shutdown_connections 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.713+0000 7f8b9c99d640 1 --2- 172.21.15.204:0/2804756110 >> v2:172.21.15.204:3300/0 conn(0x7f8b980a08e0 0x7f8b980a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.713+0000 7f8b9c99d640 1 --2- 172.21.15.204:0/2804756110 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b9809fa20 0x7f8b9809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.713+0000 7f8b9c99d640 1 --2- 172.21.15.204:0/2804756110 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 0x7f8b98132c00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.713+0000 7f8b9c99d640 1 -- 172.21.15.204:0/2804756110 >> 172.21.15.204:0/2804756110 conn(0x7f8b9806e860 msgr2=0x7f8b9806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:07.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.713+0000 7f8b9c99d640 1 -- 172.21.15.204:0/2804756110 shutdown_connections 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.713+0000 7f8b9c99d640 1 -- 172.21.15.204:0/2804756110 wait complete. 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.713+0000 7f8b9c99d640 10 monclient: build_initial_monmap 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.713+0000 7f8b9c99d640 10 monclient: monmap: 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:07.732 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:07.733 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:07.733 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:07.733 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:07.733 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:07.733 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:07.733 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:07.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.713+0000 7f8b9c99d640 1 Processor -- start 2024-09-15T01:37:07.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.713+0000 7f8b9c99d640 1 -- start start 2024-09-15T01:37:07.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.713+0000 7f8b9c99d640 10 monclient: init 2024-09-15T01:37:07.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:07.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 10 monclient: _add_conns ranks=[3,7,5,2,4,1,8,0,6] 2024-09-15T01:37:07.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b9809fa20 0x7f8b98130a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:07.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 10 monclient(hunting): picked mon.noname-g con 0x7f8b9809fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:07.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b980a08e0 0x7f8b9811d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:07.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 10 monclient(hunting): picked mon.noname-i con 0x7f8b980a08e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:07.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 0x7f8b9811dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:07.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 10 monclient(hunting): picked mon.noname-h con 0x7f8b980a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:07.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:07.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9809d3d0 con 0x7f8b9809fa20 2024-09-15T01:37:07.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:07.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9809cb60 con 0x7f8b980a2cd0 2024-09-15T01:37:07.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:07.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b981208c0 con 0x7f8b980a08e0 2024-09-15T01:37:07.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9c99d640 10 monclient(hunting): authenticate will time out at 1043.356334s 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b980a08e0 0x7f8b9811d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9659c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b9809fa20 0x7f8b98130a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b980a08e0 0x7f8b9811d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:55124/0 (socket says 172.21.15.204:55124) 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b96d9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 0x7f8b9811dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 1 -- 172.21.15.204:0/1425558524 learned_addr learned my addr 172.21.15.204:0/1425558524 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 10 monclient(hunting): get_auth_request con 0x7f8b980a08e0 auth_method 0 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b96d9d640 10 monclient(hunting): get_auth_request con 0x7f8b980a2cd0 auth_method 0 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b96d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:07.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b96d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:07.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b96d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:07.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9659c640 10 monclient(hunting): get_auth_request con 0x7f8b9809fa20 auth_method 0 2024-09-15T01:37:07.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:07.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:07.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:07.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:07.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:07.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:07.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b96d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:07.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b96d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:07.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b96d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:07.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 10 monclient(hunting): handle_auth_done global_id 24496 payload 824 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 1 -- 172.21.15.204:0/1425558524 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 msgr2=0x7f8b9811dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 1 --2- 172.21.15.204:0/1425558524 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 0x7f8b9811dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 1 -- 172.21.15.204:0/1425558524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b9809fa20 msgr2=0x7f8b98130a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 1 --2- 172.21.15.204:0/1425558524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b9809fa20 0x7f8b98130a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 1 monclient: found mon.noname-i 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 1 -- 172.21.15.204:0/1425558524 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c009390 con 0x7f8b980a08e0 2024-09-15T01:37:07.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b96d9d640 1 --2- 172.21.15.204:0/1425558524 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 0x7f8b9811dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b9659c640 1 --2- 172.21.15.204:0/1425558524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b9809fa20 0x7f8b98130a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.714+0000 7f8b95d9b640 1 --2- 172.21.15.204:0/1425558524 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b980a08e0 0x7f8b9811d580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00ea70 tx=0x7f8b8c0090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 1 -- 172.21.15.204:0/1425558524 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b8c00cdc0 con 0x7f8b980a08e0 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 10 monclient: dump: 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:07.738 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 1 -- 172.21.15.204:0/1425558524 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c013470 con 0x7f8b980a08e0 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b9c99d640 5 monclient: authenticate success, global_id 24496 2024-09-15T01:37:07.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b9c99d640 10 monclient: _renew_subs 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b9c99d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b9c99d640 1 -- 172.21.15.204:0/1425558524 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b90009a20 con 0x7f8b980a08e0 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 1 -- 172.21.15.204:0/1425558524 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b8c013a60 con 0x7f8b980a08e0 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 10 monclient: dump: 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:07.740 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:07.741 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:07.741 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:07.741 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:07.741 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:07.741 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:07.741 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:07.741 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:07.741 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:07.741 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:07.741 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:07.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b94d99640 4 set_mon_vals no callback set 2024-09-15T01:37:07.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b9c99d640 10 monclient: _renew_subs 2024-09-15T01:37:07.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b9c99d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:07.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b9c99d640 1 -- 172.21.15.204:0/1425558524 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b88009450 con 0x7f8b980a08e0 2024-09-15T01:37:07.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.715+0000 7f8b867fc640 1 -- 172.21.15.204:0/1425558524 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8b8c013bc0 con 0x7f8b980a08e0 2024-09-15T01:37:07.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.716+0000 7f8b9c99d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:07.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.716+0000 7f8b9c99d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:07.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.716+0000 7f8b9c99d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:07.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.716+0000 7f8b867fc640 1 --2- 172.21.15.204:0/1425558524 >> v2:172.21.15.204:6812/28210 conn(0x7f8b780397b0 0x7f8b7803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:07.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.716+0000 7f8b9c99d640 1 -- 172.21.15.204:0/1425558524 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b98070b80 con 0x7f8b980a08e0 2024-09-15T01:37:07.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.716+0000 7f8b867fc640 1 -- 172.21.15.204:0/1425558524 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 9610+0+0 (secure 0 0 0) 0x7f8b8c04cd60 con 0x7f8b980a08e0 2024-09-15T01:37:07.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.716+0000 7f8b9659c640 1 --2- 172.21.15.204:0/1425558524 >> v2:172.21.15.204:6812/28210 conn(0x7f8b780397b0 0x7f8b7803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:07.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.716+0000 7f8b9659c640 10 monclient: get_auth_request con 0x7f8b780397b0 auth_method 0 2024-09-15T01:37:07.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.716+0000 7f8b9659c640 1 --2- 172.21.15.204:0/1425558524 >> v2:172.21.15.204:6812/28210 conn(0x7f8b780397b0 0x7f8b7803bc60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b88034000 tx=0x7f8b88032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:07.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.720+0000 7f8b867fc640 1 -- 172.21.15.204:0/1425558524 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8b8c00c940 con 0x7f8b980a08e0 2024-09-15T01:37:07.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.720+0000 7f8b867fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:07.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.720+0000 7f8b867fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:07.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.878+0000 7f8b9c99d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:07.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.878+0000 7f8b9c99d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:07.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.878+0000 7f8b9c99d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:07.880 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:07.878+0000 7f8b9c99d640 1 -- 172.21.15.204:0/1425558524 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8b9811f110 con 0x7f8b980a08e0 2024-09-15T01:37:08.536 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.535+0000 7f8b867fc640 1 -- 172.21.15.204:0/1425558524 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}]=0 added bucket h13 type host to location {rack=r1} v198) v1 ==== 168+0+0 (secure 0 0 0) 0x7f8b8c01b820 con 0x7f8b980a08e0 2024-09-15T01:37:08.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.535+0000 7f8b867fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:08.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.535+0000 7f8b867fc640 10 monclient: _finish_command 2 = system:0 added bucket h13 type host to location {rack=r1} 2024-09-15T01:37:08.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.631+0000 7f8b9c99d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:08.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.631+0000 7f8b9c99d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:08.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.631+0000 7f8b9c99d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:08.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.631+0000 7f8b9c99d640 1 -- 172.21.15.204:0/1425558524 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8b9811fce0 con 0x7f8b980a08e0 2024-09-15T01:37:08.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.632+0000 7f8b867fc640 1 -- 172.21.15.204:0/1425558524 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h13' already exists v198) v1 ==== 147+0+0 (secure 0 0 0) 0x7f8b8c01ba00 con 0x7f8b980a08e0 2024-09-15T01:37:08.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.632+0000 7f8b867fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:08.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.632+0000 7f8b867fc640 10 monclient: _finish_command 3 = system:0 bucket 'h13' already exists 2024-09-15T01:37:08.634 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h13' already exists 2024-09-15T01:37:08.635 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 1 -- 172.21.15.204:0/1425558524 >> v2:172.21.15.204:6812/28210 conn(0x7f8b780397b0 msgr2=0x7f8b7803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:08.635 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 1 --2- 172.21.15.204:0/1425558524 >> v2:172.21.15.204:6812/28210 conn(0x7f8b780397b0 0x7f8b7803bc60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b88034000 tx=0x7f8b88032000 comp rx=0 tx=0).stop 2024-09-15T01:37:08.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 10 monclient: shutdown 2024-09-15T01:37:08.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:08.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 1 -- 172.21.15.204:0/1425558524 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b980a08e0 msgr2=0x7f8b9811d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:08.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 1 --2- 172.21.15.204:0/1425558524 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b980a08e0 0x7f8b9811d580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00ea70 tx=0x7f8b8c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:08.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 1 -- 172.21.15.204:0/1425558524 shutdown_connections 2024-09-15T01:37:08.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 1 --2- 172.21.15.204:0/1425558524 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b980a2cd0 0x7f8b9811dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:08.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 1 --2- 172.21.15.204:0/1425558524 >> v2:172.21.15.204:6812/28210 conn(0x7f8b780397b0 0x7f8b7803bc60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:08.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 1 --2- 172.21.15.204:0/1425558524 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b980a08e0 0x7f8b9811d580 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:08.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 1 --2- 172.21.15.204:0/1425558524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b9809fa20 0x7f8b98130a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:08.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 1 -- 172.21.15.204:0/1425558524 >> 172.21.15.204:0/1425558524 conn(0x7f8b9806e860 msgr2=0x7f8b980a5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:08.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 1 -- 172.21.15.204:0/1425558524 shutdown_connections 2024-09-15T01:37:08.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.634+0000 7f8b9c99d640 1 -- 172.21.15.204:0/1425558524 wait complete. 2024-09-15T01:37:08.644 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:08.644 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h14 host rack=r1 2024-09-15T01:37:08.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.719+0000 7fd134457640 10 monclient: get_monmap_and_config 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.719+0000 7fd134457640 10 monclient: build_initial_monmap 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.719+0000 7fd134457640 10 monclient: monmap: 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:08.719643+0000 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:08.719643+0000 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:08.721 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:08.722 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:08.722 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:08.722 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:08.722 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:08.722 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:08.722 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:08.722 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:08.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.719+0000 7fd134457640 1 Processor -- start 2024-09-15T01:37:08.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.719+0000 7fd134457640 1 -- start start 2024-09-15T01:37:08.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.719+0000 7fd134457640 10 monclient: init 2024-09-15T01:37:08.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:08.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 10 monclient: _add_conns ranks=[8,2,10,9,4,0,7,5,1,3,6] 2024-09-15T01:37:08.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd12c0a2cd0 0x7fd12c132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:08.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 10 monclient(hunting): picked mon.noname-e con 0x7fd12c0a2cd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:08.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c09fa20 0x7fd12c09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 10 monclient(hunting): picked mon.noname-b con 0x7fd12c09fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd12c0a08e0 0x7fd12c0a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 10 monclient(hunting): picked mon.noname-f con 0x7fd12c0a08e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c0a30a0 con 0x7fd12c0a08e0 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0a0330 con 0x7fd12c09fa20 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0a1db0 con 0x7fd12c0a2cd0 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd134457640 10 monclient(hunting): authenticate will time out at 1044.362590s 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd12c0a08e0 0x7fd12c0a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:08.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd12c0a08e0 0x7fd12c0a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41818/0 (socket says 172.21.15.204:41818) 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 1 -- 172.21.15.204:0/2392286222 learned_addr learned my addr 172.21.15.204:0/2392286222 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 10 monclient(hunting): get_auth_request con 0x7fd12c0a08e0 auth_method 0 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1319f2640 1 -- 172.21.15.204:0/2392286222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c09fa20 msgr2=0x7fd12c09fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1321f3640 1 -- 172.21.15.204:0/2392286222 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd12c0a2cd0 msgr2=0x7fd12c132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1319f2640 1 --2- 172.21.15.204:0/2392286222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c09fa20 0x7fd12c09fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1321f3640 1 --2- 172.21.15.204:0/2392286222 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd12c0a2cd0 0x7fd12c132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:08.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 10 monclient(hunting): handle_auth_done global_id 7012 payload 274 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 1 -- 172.21.15.204:0/2392286222 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd12c0a2cd0 msgr2=0x7fd12c132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 1 --2- 172.21.15.204:0/2392286222 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd12c0a2cd0 0x7fd12c132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 1 -- 172.21.15.204:0/2392286222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c09fa20 msgr2=0x7fd12c09fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 1 --2- 172.21.15.204:0/2392286222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c09fa20 0x7fd12c09fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 10 monclient: _finish_hunting 0 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 1 monclient: found mon.noname-f 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.720+0000 7fd1329f4640 1 -- 172.21.15.204:0/2392286222 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c0648a0 con 0x7fd12c0a08e0 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1329f4640 1 --2- 172.21.15.204:0/2392286222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd12c0a08e0 0x7fd12c0a0d50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd124009f20 tx=0x7fd124009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=735c26ea008ebc37 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1311f1640 1 -- 172.21.15.204:0/2392286222 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd124007880 con 0x7fd12c0a08e0 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1311f1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:08.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1311f1640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1311f1640 10 monclient: dump: 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:08.734 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1311f1640 10 monclient: _finish_auth 0 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1311f1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd134457640 5 monclient: authenticate success, global_id 7012 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1311f1640 1 -- 172.21.15.204:0/2392286222 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12402d050 con 0x7fd12c0a08e0 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd134457640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1311f1640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd134457640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd134457640 4 set_mon_vals no callback set 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1311f1640 1 -- 172.21.15.204:0/2392286222 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd124030360 con 0x7fd12c0a08e0 2024-09-15T01:37:08.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1311f1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1311f1640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.721+0000 7fd1311f1640 10 monclient: dump: 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:08.736 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 10 monclient: shutdown 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 1 -- 172.21.15.204:0/2392286222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd12c0a08e0 msgr2=0x7fd12c0a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 1 --2- 172.21.15.204:0/2392286222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd12c0a08e0 0x7fd12c0a0d50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd124009f20 tx=0x7fd124009840 comp rx=0 tx=0).stop 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 1 -- 172.21.15.204:0/2392286222 shutdown_connections 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 1 --2- 172.21.15.204:0/2392286222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd12c0a08e0 0x7fd12c0a0d50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 1 --2- 172.21.15.204:0/2392286222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c09fa20 0x7fd12c09fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:08.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 1 --2- 172.21.15.204:0/2392286222 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd12c0a2cd0 0x7fd12c132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 1 -- 172.21.15.204:0/2392286222 >> 172.21.15.204:0/2392286222 conn(0x7fd12c06e860 msgr2=0x7fd12c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 1 -- 172.21.15.204:0/2392286222 shutdown_connections 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 1 -- 172.21.15.204:0/2392286222 wait complete. 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 10 monclient: build_initial_monmap 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.722+0000 7fd134457640 10 monclient: monmap: 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:08.738 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 1 Processor -- start 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 1 -- start start 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 10 monclient: init 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 10 monclient: _add_conns ranks=[2,3,4,0,1,6,5,7,8] 2024-09-15T01:37:08.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c09fa20 0x7fd12c131910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 10 monclient(hunting): picked mon.noname-b con 0x7fd12c09fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd12c0a08e0 0x7fd12c11d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 10 monclient(hunting): picked mon.noname-g con 0x7fd12c0a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd12c0a2cd0 0x7fd12c11daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 10 monclient(hunting): picked mon.noname-c con 0x7fd12c0a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c09d3d0 con 0x7fd12c09fa20 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c09cb60 con 0x7fd12c0a08e0 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c11faf0 con 0x7fd12c0a2cd0 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:08.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd134457640 10 monclient(hunting): authenticate will time out at 1044.365764s 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1319f2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd12c0a08e0 0x7fd12c11d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1329f4640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd12c0a2cd0 msgr2=0x7fd12c11daa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1319f2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd12c0a08e0 0x7fd12c11d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:40862/0 (socket says 172.21.15.204:40862) 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1321f3640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c09fa20 msgr2=0x7fd12c131910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1319f2640 1 -- 172.21.15.204:0/2722604532 learned_addr learned my addr 172.21.15.204:0/2722604532 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1319f2640 10 monclient(hunting): get_auth_request con 0x7fd12c0a08e0 auth_method 0 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1319f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1319f2640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1319f2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1329f4640 1 --2- 172.21.15.204:0/2722604532 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd12c0a2cd0 0x7fd12c11daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1321f3640 1 --2- 172.21.15.204:0/2722604532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c09fa20 0x7fd12c131910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1319f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:08.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1319f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.723+0000 7fd1319f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1319f2640 10 monclient(hunting): handle_auth_done global_id 15393 payload 824 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1319f2640 1 -- 172.21.15.204:0/2722604532 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd12c0a2cd0 msgr2=0x7fd12c11daa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1319f2640 1 --2- 172.21.15.204:0/2722604532 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd12c0a2cd0 0x7fd12c11daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1319f2640 1 -- 172.21.15.204:0/2722604532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c09fa20 msgr2=0x7fd12c131910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1319f2640 1 --2- 172.21.15.204:0/2722604532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c09fa20 0x7fd12c131910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1319f2640 10 monclient: _finish_hunting 0 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1319f2640 1 monclient: found mon.noname-g 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1319f2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1319f2640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1319f2640 1 -- 172.21.15.204:0/2722604532 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd124009ae0 con 0x7fd12c0a08e0 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1319f2640 1 --2- 172.21.15.204:0/2722604532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd12c0a08e0 0x7fd12c11d560 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00b600 tx=0x7fd11c0091a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:08.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1227fc640 1 -- 172.21.15.204:0/2722604532 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd11c015070 con 0x7fd12c0a08e0 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1227fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1227fc640 10 monclient: dump: 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:08.743 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1227fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1227fc640 1 -- 172.21.15.204:0/2722604532 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c006150 con 0x7fd12c0a08e0 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd134457640 5 monclient: authenticate success, global_id 15393 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1227fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1227fc640 1 -- 172.21.15.204:0/2722604532 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd11c006740 con 0x7fd12c0a08e0 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd134457640 10 monclient: _renew_subs 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd134457640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:08.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd134457640 1 -- 172.21.15.204:0/2722604532 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c11f430 con 0x7fd12c0a08e0 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1227fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1309f0640 4 set_mon_vals no callback set 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd1227fc640 10 monclient: dump: 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:08.745 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd134457640 10 monclient: _renew_subs 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd134457640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.724+0000 7fd134457640 1 -- 172.21.15.204:0/2722604532 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c11ef20 con 0x7fd12c0a08e0 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.725+0000 7fd1227fc640 1 -- 172.21.15.204:0/2722604532 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd11c0062c0 con 0x7fd12c0a08e0 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.725+0000 7fd134457640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.725+0000 7fd134457640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:08.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.725+0000 7fd134457640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:08.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.725+0000 7fd134457640 1 -- 172.21.15.204:0/2722604532 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c070b80 con 0x7fd12c0a08e0 2024-09-15T01:37:08.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.725+0000 7fd1227fc640 1 --2- 172.21.15.204:0/2722604532 >> v2:172.21.15.204:6812/28210 conn(0x7fd1100397b0 0x7fd11003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:08.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.725+0000 7fd1227fc640 1 -- 172.21.15.204:0/2722604532 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 9823+0+0 (secure 0 0 0) 0x7fd11c04c1b0 con 0x7fd12c0a08e0 2024-09-15T01:37:08.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.725+0000 7fd1321f3640 1 --2- 172.21.15.204:0/2722604532 >> v2:172.21.15.204:6812/28210 conn(0x7fd1100397b0 0x7fd11003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:08.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.725+0000 7fd1321f3640 10 monclient: get_auth_request con 0x7fd1100397b0 auth_method 0 2024-09-15T01:37:08.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.726+0000 7fd1321f3640 1 --2- 172.21.15.204:0/2722604532 >> v2:172.21.15.204:6812/28210 conn(0x7fd1100397b0 0x7fd11003bc60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd1280038f0 tx=0x7fd128006cb0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:08.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.730+0000 7fd1227fc640 1 -- 172.21.15.204:0/2722604532 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd11c003e10 con 0x7fd12c0a08e0 2024-09-15T01:37:08.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.730+0000 7fd1227fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:08.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.730+0000 7fd1227fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:08.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.890+0000 7fd134457640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:08.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.890+0000 7fd134457640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:08.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.890+0000 7fd134457640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:08.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:08.890+0000 7fd134457640 1 -- 172.21.15.204:0/2722604532 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fd12c120dc0 con 0x7fd12c0a08e0 2024-09-15T01:37:09.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.539+0000 7fd1227fc640 1 -- 172.21.15.204:0/2722604532 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}]=0 added bucket h14 type host to location {rack=r1} v199) v1 ==== 168+0+0 (secure 0 0 0) 0x7fd11c00d370 con 0x7fd12c0a08e0 2024-09-15T01:37:09.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.539+0000 7fd1227fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:09.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.539+0000 7fd1227fc640 10 monclient: _finish_command 2 = system:0 added bucket h14 type host to location {rack=r1} 2024-09-15T01:37:09.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.638+0000 7fd134457640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:09.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.638+0000 7fd134457640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:09.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.638+0000 7fd134457640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:09.640 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.638+0000 7fd134457640 1 -- 172.21.15.204:0/2722604532 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fd12c1350a0 con 0x7fd12c0a08e0 2024-09-15T01:37:09.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.639+0000 7fd1227fc640 1 -- 172.21.15.204:0/2722604532 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h14' already exists v199) v1 ==== 147+0+0 (secure 0 0 0) 0x7fd11c00eb30 con 0x7fd12c0a08e0 2024-09-15T01:37:09.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.639+0000 7fd1227fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:09.641 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.639+0000 7fd1227fc640 10 monclient: _finish_command 3 = system:0 bucket 'h14' already exists 2024-09-15T01:37:09.641 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h14' already exists 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 1 -- 172.21.15.204:0/2722604532 >> v2:172.21.15.204:6812/28210 conn(0x7fd1100397b0 msgr2=0x7fd11003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 1 --2- 172.21.15.204:0/2722604532 >> v2:172.21.15.204:6812/28210 conn(0x7fd1100397b0 0x7fd11003bc60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd1280038f0 tx=0x7fd128006cb0 comp rx=0 tx=0).stop 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 10 monclient: shutdown 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 1 -- 172.21.15.204:0/2722604532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd12c0a08e0 msgr2=0x7fd12c11d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 1 --2- 172.21.15.204:0/2722604532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd12c0a08e0 0x7fd12c11d560 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00b600 tx=0x7fd11c0091a0 comp rx=0 tx=0).stop 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 1 -- 172.21.15.204:0/2722604532 shutdown_connections 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 1 --2- 172.21.15.204:0/2722604532 >> v2:172.21.15.204:6812/28210 conn(0x7fd1100397b0 0x7fd11003bc60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 1 --2- 172.21.15.204:0/2722604532 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd12c0a2cd0 0x7fd12c11daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 1 --2- 172.21.15.204:0/2722604532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd12c0a08e0 0x7fd12c11d560 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 1 --2- 172.21.15.204:0/2722604532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd12c09fa20 0x7fd12c131910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 1 -- 172.21.15.204:0/2722604532 >> 172.21.15.204:0/2722604532 conn(0x7fd12c06e860 msgr2=0x7fd12c0a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 1 -- 172.21.15.204:0/2722604532 shutdown_connections 2024-09-15T01:37:09.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.641+0000 7fd134457640 1 -- 172.21.15.204:0/2722604532 wait complete. 2024-09-15T01:37:09.651 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:09.651 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h15 host rack=r1 2024-09-15T01:37:09.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.725+0000 7f58aa99d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:09.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.725+0000 7f58aa99d640 10 monclient: build_initial_monmap 2024-09-15T01:37:09.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.725+0000 7f58aa99d640 10 monclient: monmap: 2024-09-15T01:37:09.727 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:09.727 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:09.727 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:09.726069+0000 2024-09-15T01:37:09.727 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:09.726069+0000 2024-09-15T01:37:09.727 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:09.727 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:09.727 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:09.728 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:09.728 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:09.728 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:09.728 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:09.728 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:09.728 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:09.728 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:09.728 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:09.728 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:09.728 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:09.728 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:09.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 1 Processor -- start 2024-09-15T01:37:09.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 1 -- start start 2024-09-15T01:37:09.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 10 monclient: init 2024-09-15T01:37:09.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 10 monclient: _add_conns ranks=[1,5,10,3,9,6,8,7,2,0,4] 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f58a40a2cd0 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58a409fa20 0x7f58a409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 10 monclient(hunting): picked mon.noname-h con 0x7f58a409fa20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f58a4137c80 0x7f58a413a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 10 monclient(hunting): picked mon.noname-f con 0x7f58a4137c80 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a40648a0 con 0x7f58a40a2cd0 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a4133310 con 0x7f58a4137c80 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40a30a0 con 0x7f58a409fa20 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:09.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58aa99d640 10 monclient(hunting): authenticate will time out at 1045.368848s 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58a37fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58a409fa20 0x7f58a409fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58a8f3a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f58a4137c80 0x7f58a413a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58a3fff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f58a40a2cd0 0x7f58a4132c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:41982/0 (socket says 172.21.15.204:41982) 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.726+0000 7f58a37fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58a409fa20 0x7f58a409fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:40902/0 (socket says 172.21.15.204:40902) 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a8f3a640 10 monclient(hunting): get_auth_request con 0x7f58a4137c80 auth_method 0 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a3fff640 1 -- 172.21.15.204:0/2249303549 learned_addr learned my addr 172.21.15.204:0/2249303549 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a8f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a8f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a8f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 10 monclient(hunting): get_auth_request con 0x7f58a409fa20 auth_method 0 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:09.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a8f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a8f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a8f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 10 monclient(hunting): handle_auth_done global_id 6251 payload 274 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 1 -- 172.21.15.204:0/2249303549 >> v2:172.21.15.204:3300/0 conn(0x7f58a4137c80 msgr2=0x7f58a413a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 1 --2- 172.21.15.204:0/2249303549 >> v2:172.21.15.204:3300/0 conn(0x7f58a4137c80 0x7f58a413a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 1 -- 172.21.15.204:0/2249303549 >> v1:172.21.15.204:6789/0 conn(0x7f58a40a2cd0 legacy=0x7f58a4132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 1 monclient: found mon.noname-h 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:09.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a37fe640 1 -- 172.21.15.204:0/2249303549 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a40a0330 con 0x7f58a409fa20 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a2ffd640 1 -- 172.21.15.204:0/2249303549 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091020170 0 0) 0x7f58a40648a0 con 0x7f58a40a2cd0 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a2ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.727+0000 7f58a8f3a640 1 --2- 172.21.15.204:0/2249303549 >> v2:172.21.15.204:3300/0 conn(0x7f58a4137c80 0x7f58a413a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a37fe640 1 --2- 172.21.15.204:0/2249303549 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58a409fa20 0x7f58a409fdf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5898009850 tx=0x7f58980090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=dcbc0a9d87cd1cef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a2ffd640 1 -- 172.21.15.204:0/2249303549 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f589802ac80 con 0x7f58a409fa20 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a2ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a2ffd640 10 monclient: dump: 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:09.737 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:09.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:37:09.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:09.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 5 monclient: authenticate success, global_id 6251 2024-09-15T01:37:09.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:09.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a2ffd640 1 -- 172.21.15.204:0/2249303549 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589802ae00 con 0x7f58a409fa20 2024-09-15T01:37:09.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:09.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:09.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 4 set_mon_vals no callback set 2024-09-15T01:37:09.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a2ffd640 1 -- 172.21.15.204:0/2249303549 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5898031ca0 con 0x7f58a409fa20 2024-09-15T01:37:09.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:09.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:09.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58a2ffd640 10 monclient: dump: 2024-09-15T01:37:09.739 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:09.740 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:09.741 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 10 monclient: shutdown 2024-09-15T01:37:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 1 -- 172.21.15.204:0/2249303549 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58a409fa20 msgr2=0x7f58a409fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 1 --2- 172.21.15.204:0/2249303549 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58a409fa20 0x7f58a409fdf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5898009850 tx=0x7f58980090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 1 -- 172.21.15.204:0/2249303549 shutdown_connections 2024-09-15T01:37:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 1 --2- 172.21.15.204:0/2249303549 >> v2:172.21.15.204:3300/0 conn(0x7f58a4137c80 0x7f58a413a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 1 --2- 172.21.15.204:0/2249303549 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58a409fa20 0x7f58a409fdf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 1 -- 172.21.15.204:0/2249303549 >> 172.21.15.204:0/2249303549 conn(0x7f58a406e860 msgr2=0x7f58a406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:09.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 1 -- 172.21.15.204:0/2249303549 shutdown_connections 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.728+0000 7f58aa99d640 1 -- 172.21.15.204:0/2249303549 wait complete. 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.729+0000 7f58aa99d640 10 monclient: build_initial_monmap 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.729+0000 7f58aa99d640 10 monclient: monmap: 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:09.742 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:09.743 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:09.743 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:09.743 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:09.743 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:09.743 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:09.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.729+0000 7f58aa99d640 1 Processor -- start 2024-09-15T01:37:09.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.729+0000 7f58aa99d640 1 -- start start 2024-09-15T01:37:09.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.729+0000 7f58aa99d640 10 monclient: init 2024-09-15T01:37:09.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:09.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 10 monclient: _add_conns ranks=[8,0,4,7,6,2,1,5,3] 2024-09-15T01:37:09.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58a409fa20 0x7f58a409c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 10 monclient(hunting): picked mon.noname-e con 0x7f58a409fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a40a2cd0 0x7f58a4134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 10 monclient(hunting): picked mon.noname-a con 0x7f58a40a2cd0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58a4137c80 0x7f58a4134760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 10 monclient(hunting): picked mon.noname-c con 0x7f58a4137c80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a409d3d0 con 0x7f58a40a2cd0 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4096b10 con 0x7f58a4137c80 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4136910 con 0x7f58a409fa20 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58aa99d640 10 monclient(hunting): authenticate will time out at 1045.372274s 2024-09-15T01:37:09.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58a3fff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58a409fa20 msgr2=0x7f58a409c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:09.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58a8f3a640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58a4137c80 msgr2=0x7f58a4134760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:09.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58a409fa20 0x7f58a409c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:09.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58a8f3a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58a4137c80 0x7f58a4134760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:09.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58a37fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a40a2cd0 msgr2=0x7f58a4134220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:09.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.730+0000 7f58a37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a40a2cd0 0x7f58a4134220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:09.932 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.931+0000 7f58a37fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a40a2cd0 msgr2=0x7f58a4134220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:09.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.931+0000 7f58a3fff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58a409fa20 msgr2=0x7f58a409c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:09.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.931+0000 7f58a8f3a640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58a4137c80 msgr2=0x7f58a4134760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:09.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.931+0000 7f58a37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a40a2cd0 0x7f58a4134220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:37:09.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.931+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58a409fa20 0x7f58a409c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:37:09.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:09.931+0000 7f58a8f3a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58a4137c80 0x7f58a4134760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:37:10.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:10.332+0000 7f58a3fff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58a409fa20 msgr2=0x7f58a409c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:10.333 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:10.332+0000 7f58a8f3a640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58a4137c80 msgr2=0x7f58a4134760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:10.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:10.332+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58a409fa20 0x7f58a409c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:37:10.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:10.332+0000 7f58a37fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a40a2cd0 msgr2=0x7f58a4134220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:10.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:10.332+0000 7f58a8f3a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58a4137c80 0x7f58a4134760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:37:10.334 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:10.332+0000 7f58a37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a40a2cd0 0x7f58a4134220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:37:11.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:11.133+0000 7f58a8f3a640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58a4137c80 msgr2=0x7f58a4134760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:11.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:11.133+0000 7f58a3fff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58a409fa20 msgr2=0x7f58a409c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:11.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:11.133+0000 7f58a37fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a40a2cd0 msgr2=0x7f58a4134220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:11.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:11.133+0000 7f58a37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a40a2cd0 0x7f58a4134220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:37:11.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:11.133+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58a409fa20 0x7f58a409c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:37:11.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:11.133+0000 7f58a8f3a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58a4137c80 0x7f58a4134760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:37:12.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 10 monclient(hunting): tick 2024-09-15T01:37:12.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 monclient(hunting): continuing hunt 2024-09-15T01:37:12.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:37:12.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58a409fa20 msgr2=0x7f58a409c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:12.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58a409fa20 0x7f58a409c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:12.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58a4137c80 msgr2=0x7f58a4134760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:12.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58a4137c80 0x7f58a4134760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:12.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a40a2cd0 msgr2=0x7f58a4134220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:12.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58a40a2cd0 0x7f58a4134220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:12.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 10 monclient: _add_conns ranks=[5,2,3,1,7,6] 2024-09-15T01:37:12.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58840037d0 0x7f5884005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:12.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 10 monclient(hunting): picked mon.noname-h con 0x7f58840037d0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:12.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58840061d0 0x7f5884008650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:12.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 10 monclient(hunting): picked mon.noname-b con 0x7f58840061d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:12.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5884008cf0 0x7f588400b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:12.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 10 monclient(hunting): picked mon.noname-g con 0x7f5884008cf0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:12.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:12.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a409d3d0 con 0x7f58840061d0 2024-09-15T01:37:12.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:12.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4096b10 con 0x7f5884008cf0 2024-09-15T01:37:12.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:12.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4136910 con 0x7f58840037d0 2024-09-15T01:37:12.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f588affd640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:12.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58840037d0 0x7f5884005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:12.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a37fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58840061d0 msgr2=0x7f5884008650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:12.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a8f3a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5884008cf0 0x7f588400b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:12.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a37fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58840061d0 0x7f5884008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:12.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58840037d0 0x7f5884005bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:40906/0 (socket says 172.21.15.204:40906) 2024-09-15T01:37:12.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a3fff640 1 -- 172.21.15.204:0/2841002272 learned_addr learned my addr 172.21.15.204:0/2841002272 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:12.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a8f3a640 10 monclient(hunting): get_auth_request con 0x7f5884008cf0 auth_method 0 2024-09-15T01:37:12.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a8f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:12.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a8f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:12.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a8f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:12.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a3fff640 10 monclient(hunting): get_auth_request con 0x7f58840037d0 auth_method 0 2024-09-15T01:37:12.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:12.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:12.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:12.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a8f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:12.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a8f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:12.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a8f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:12.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:12.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:12.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.730+0000 7f58a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:12.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a8f3a640 10 monclient(hunting): handle_auth_done global_id 15402 payload 824 2024-09-15T01:37:12.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a8f3a640 1 -- 172.21.15.204:0/2841002272 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58840037d0 msgr2=0x7f5884005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:12.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a8f3a640 1 --2- 172.21.15.204:0/2841002272 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58840037d0 0x7f5884005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:12.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a8f3a640 1 -- 172.21.15.204:0/2841002272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58840061d0 msgr2=0x7f5884008650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:12.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a8f3a640 1 --2- 172.21.15.204:0/2841002272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58840061d0 0x7f5884008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:12.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a8f3a640 10 monclient: _finish_hunting 0 2024-09-15T01:37:12.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a8f3a640 1 monclient: found mon.noname-g 2024-09-15T01:37:12.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a8f3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:12.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a8f3a640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:12.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a8f3a640 1 -- 172.21.15.204:0/2841002272 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c009a20 con 0x7f5884008cf0 2024-09-15T01:37:12.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a3fff640 1 --2- 172.21.15.204:0/2841002272 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58840037d0 0x7f5884005bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:12.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a8f3a640 1 -- 172.21.15.204:0/2841002272 reap_dead start 2024-09-15T01:37:12.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a8f3a640 1 --2- 172.21.15.204:0/2841002272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5884008cf0 0x7f588400b1a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f58a4135930 tx=0x7f589c00b040 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:12.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f588bfff640 1 -- 172.21.15.204:0/2841002272 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f589c00cd10 con 0x7f5884008cf0 2024-09-15T01:37:12.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f588bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:12.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f588bfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:12.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f588bfff640 10 monclient: dump: 2024-09-15T01:37:12.741 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:12.741 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:12.741 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:12.741 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:12.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f588bfff640 10 monclient: _finish_auth 0 2024-09-15T01:37:12.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f588bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:12.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f588bfff640 1 -- 172.21.15.204:0/2841002272 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c00f040 con 0x7f5884008cf0 2024-09-15T01:37:12.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58aa99d640 5 monclient: authenticate success, global_id 15402 2024-09-15T01:37:12.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58aa99d640 10 monclient: _renew_subs 2024-09-15T01:37:12.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58aa99d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:12.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58aa99d640 1 -- 172.21.15.204:0/2841002272 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5898009390 con 0x7f5884008cf0 2024-09-15T01:37:12.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f588bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:12.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f588bfff640 1 -- 172.21.15.204:0/2841002272 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f589c00ce70 con 0x7f5884008cf0 2024-09-15T01:37:12.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f588bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:12.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f588bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:12.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f588bfff640 10 monclient: dump: 2024-09-15T01:37:12.743 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:12.744 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:12.745 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:12.745 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:12.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58a27fc640 4 set_mon_vals no callback set 2024-09-15T01:37:12.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58aa99d640 10 monclient: _renew_subs 2024-09-15T01:37:12.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58aa99d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:12.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.731+0000 7f58aa99d640 1 -- 172.21.15.204:0/2841002272 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a40974b0 con 0x7f5884008cf0 2024-09-15T01:37:12.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.732+0000 7f588bfff640 1 -- 172.21.15.204:0/2841002272 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f589c003030 con 0x7f5884008cf0 2024-09-15T01:37:12.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.732+0000 7f588bfff640 1 --2- 172.21.15.204:0/2841002272 >> v2:172.21.15.204:6812/28210 conn(0x7f5890039740 0x7f589003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:12.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.732+0000 7f588bfff640 1 -- 172.21.15.204:0/2841002272 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 10036+0+0 (secure 0 0 0) 0x7f589c04cca0 con 0x7f5884008cf0 2024-09-15T01:37:12.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.732+0000 7f58aa99d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:12.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.732+0000 7f58aa99d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:12.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.732+0000 7f58aa99d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:12.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.732+0000 7f58aa99d640 1 -- 172.21.15.204:0/2841002272 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a4096f40 con 0x7f5884008cf0 2024-09-15T01:37:12.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.732+0000 7f58a3fff640 1 --2- 172.21.15.204:0/2841002272 >> v2:172.21.15.204:6812/28210 conn(0x7f5890039740 0x7f589003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:12.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.733+0000 7f58a3fff640 10 monclient: get_auth_request con 0x7f5890039740 auth_method 0 2024-09-15T01:37:12.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.733+0000 7f58a3fff640 1 --2- 172.21.15.204:0/2841002272 >> v2:172.21.15.204:6812/28210 conn(0x7f5890039740 0x7f589003bbf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5894000c00 tx=0x7f5894007f90 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:12.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.737+0000 7f588bfff640 1 -- 172.21.15.204:0/2841002272 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f589c0133e0 con 0x7f5884008cf0 2024-09-15T01:37:12.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.737+0000 7f588bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:12.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.737+0000 7f588bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:12.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.900+0000 7f58aa99d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:12.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.900+0000 7f58aa99d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:12.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.900+0000 7f58aa99d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:12.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:12.900+0000 7f58aa99d640 1 -- 172.21.15.204:0/2841002272 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f58a4070b80 con 0x7f5884008cf0 2024-09-15T01:37:13.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.471+0000 7f588bfff640 1 -- 172.21.15.204:0/2841002272 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}]=0 added bucket h15 type host to location {rack=r1} v200) v1 ==== 168+0+0 (secure 0 0 0) 0x7f589c007340 con 0x7f5884008cf0 2024-09-15T01:37:13.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.471+0000 7f588bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:13.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.471+0000 7f588bfff640 10 monclient: _finish_command 2 = system:0 added bucket h15 type host to location {rack=r1} 2024-09-15T01:37:13.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.568+0000 7f58aa99d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:13.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.568+0000 7f58aa99d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:13.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.568+0000 7f58aa99d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:13.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.568+0000 7f58aa99d640 1 -- 172.21.15.204:0/2841002272 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f58a40976a0 con 0x7f5884008cf0 2024-09-15T01:37:13.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.569+0000 7f588bfff640 1 -- 172.21.15.204:0/2841002272 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h15' already exists v200) v1 ==== 147+0+0 (secure 0 0 0) 0x7f589c007520 con 0x7f5884008cf0 2024-09-15T01:37:13.570 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.569+0000 7f588bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:13.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.569+0000 7f588bfff640 10 monclient: _finish_command 3 = system:0 bucket 'h15' already exists 2024-09-15T01:37:13.571 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h15' already exists 2024-09-15T01:37:13.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.571+0000 7f58aa99d640 1 -- 172.21.15.204:0/2841002272 >> v2:172.21.15.204:6812/28210 conn(0x7f5890039740 msgr2=0x7f589003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:13.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.571+0000 7f58aa99d640 1 --2- 172.21.15.204:0/2841002272 >> v2:172.21.15.204:6812/28210 conn(0x7f5890039740 0x7f589003bbf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5894000c00 tx=0x7f5894007f90 comp rx=0 tx=0).stop 2024-09-15T01:37:13.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.571+0000 7f58aa99d640 10 monclient: shutdown 2024-09-15T01:37:13.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.571+0000 7f58aa99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:13.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.571+0000 7f58aa99d640 1 -- 172.21.15.204:0/2841002272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5884008cf0 msgr2=0x7f588400b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:13.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.571+0000 7f58aa99d640 1 --2- 172.21.15.204:0/2841002272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5884008cf0 0x7f588400b1a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f58a4135930 tx=0x7f589c00b040 comp rx=0 tx=0).stop 2024-09-15T01:37:13.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.571+0000 7f58aa99d640 1 -- 172.21.15.204:0/2841002272 shutdown_connections 2024-09-15T01:37:13.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.571+0000 7f58aa99d640 1 --2- 172.21.15.204:0/2841002272 >> v2:172.21.15.204:6812/28210 conn(0x7f5890039740 0x7f589003bbf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:13.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.571+0000 7f58aa99d640 1 --2- 172.21.15.204:0/2841002272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5884008cf0 0x7f588400b1a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:13.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.571+0000 7f58aa99d640 1 -- 172.21.15.204:0/2841002272 >> 172.21.15.204:0/2841002272 conn(0x7f58a406e860 msgr2=0x7f58a412dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:13.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.571+0000 7f58aa99d640 1 -- 172.21.15.204:0/2841002272 shutdown_connections 2024-09-15T01:37:13.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.571+0000 7f58aa99d640 1 -- 172.21.15.204:0/2841002272 wait complete. 2024-09-15T01:37:13.581 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:13.581 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h16 host rack=r1 2024-09-15T01:37:13.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.655+0000 7f1d1266f640 10 monclient: get_monmap_and_config 2024-09-15T01:37:13.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.655+0000 7f1d1266f640 10 monclient: build_initial_monmap 2024-09-15T01:37:13.657 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.655+0000 7f1d1266f640 10 monclient: monmap: 2024-09-15T01:37:13.657 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:13.657 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:13.657 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:13.655834+0000 2024-09-15T01:37:13.657 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:13.655834+0000 2024-09-15T01:37:13.657 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:13.658 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:13.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 1 Processor -- start 2024-09-15T01:37:13.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 1 -- start start 2024-09-15T01:37:13.659 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 10 monclient: init 2024-09-15T01:37:13.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:13.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 10 monclient: _add_conns ranks=[4,3,10,2,0,5,1,7,6,8,9] 2024-09-15T01:37:13.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d0c0a2cd0 0x7f1d0c132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:13.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 10 monclient(hunting): picked mon.noname-c con 0x7f1d0c0a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:13.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d0c09fa20 0x7f1d0c09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:13.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 10 monclient(hunting): picked mon.noname-g con 0x7f1d0c09fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:13.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1d0c0a08e0 0x7f1d0c0a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:13.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 10 monclient(hunting): picked mon.noname-f con 0x7f1d0c0a08e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:13.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:13.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0c0a30a0 con 0x7f1d0c0a08e0 2024-09-15T01:37:13.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:13.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0a0330 con 0x7f1d0c09fa20 2024-09-15T01:37:13.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:13.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0a1db0 con 0x7f1d0c0a2cd0 2024-09-15T01:37:13.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:13.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d1266f640 10 monclient(hunting): authenticate will time out at 1049.298608s 2024-09-15T01:37:13.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d10c0c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1d0c0a08e0 0x7f1d0c0a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:13.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d0b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d0c09fa20 0x7f1d0c09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:13.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d0bfff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d0c0a2cd0 msgr2=0x7f1d0c132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:13.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d10c0c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1d0c0a08e0 0x7f1d0c0a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41836/0 (socket says 172.21.15.204:41836) 2024-09-15T01:37:13.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d10c0c640 1 -- 172.21.15.204:0/3210974881 learned_addr learned my addr 172.21.15.204:0/3210974881 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:13.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d0bfff640 1 --2- 172.21.15.204:0/3210974881 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d0c0a2cd0 0x7f1d0c132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:13.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d0b7fe640 10 monclient(hunting): get_auth_request con 0x7f1d0c09fa20 auth_method 0 2024-09-15T01:37:13.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:13.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d0b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:13.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d0b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:13.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d10c0c640 10 monclient(hunting): get_auth_request con 0x7f1d0c0a08e0 auth_method 0 2024-09-15T01:37:13.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d10c0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:13.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d10c0c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:13.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d10c0c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:13.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:13.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:13.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d10c0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d10c0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.656+0000 7f1d10c0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0b7fe640 10 monclient(hunting): handle_auth_done global_id 15411 payload 274 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0b7fe640 1 -- 172.21.15.204:0/3210974881 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d0c0a2cd0 msgr2=0x7f1d0c132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0b7fe640 1 --2- 172.21.15.204:0/3210974881 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d0c0a2cd0 0x7f1d0c132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0b7fe640 1 -- 172.21.15.204:0/3210974881 >> v2:172.21.15.204:3300/0 conn(0x7f1d0c0a08e0 msgr2=0x7f1d0c0a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0b7fe640 1 --2- 172.21.15.204:0/3210974881 >> v2:172.21.15.204:3300/0 conn(0x7f1d0c0a08e0 0x7f1d0c0a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0b7fe640 1 monclient: found mon.noname-g 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0b7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:13.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0b7fe640 1 -- 172.21.15.204:0/3210974881 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c0648a0 con 0x7f1d0c09fa20 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d10c0c640 1 --2- 172.21.15.204:0/3210974881 >> v2:172.21.15.204:3300/0 conn(0x7f1d0c0a08e0 0x7f1d0c0a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0b7fe640 1 --2- 172.21.15.204:0/3210974881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d0c09fa20 0x7f1d0c09fdf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009850 tx=0x7f1d000090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=be5c07229e43c992 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0affd640 1 -- 172.21.15.204:0/3210974881 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1d0002abe0 con 0x7f1d0c09fa20 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0affd640 10 monclient: dump: 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:13.671 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:13.672 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:13.672 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:13.672 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:13.672 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:13.672 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:13.672 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:13.672 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:13.672 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:13.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0affd640 10 monclient: _finish_auth 0 2024-09-15T01:37:13.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:13.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0affd640 1 -- 172.21.15.204:0/3210974881 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0002ad60 con 0x7f1d0c09fa20 2024-09-15T01:37:13.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d1266f640 5 monclient: authenticate success, global_id 15411 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d1266f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d1266f640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0affd640 1 -- 172.21.15.204:0/3210974881 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1d00031ca0 con 0x7f1d0c09fa20 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d1266f640 4 set_mon_vals no callback set 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.657+0000 7f1d0affd640 10 monclient: dump: 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:13.673 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 10 monclient: shutdown 2024-09-15T01:37:13.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 1 -- 172.21.15.204:0/3210974881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d0c09fa20 msgr2=0x7f1d0c09fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 1 --2- 172.21.15.204:0/3210974881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d0c09fa20 0x7f1d0c09fdf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009850 tx=0x7f1d000090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 1 -- 172.21.15.204:0/3210974881 shutdown_connections 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 1 --2- 172.21.15.204:0/3210974881 >> v2:172.21.15.204:3300/0 conn(0x7f1d0c0a08e0 0x7f1d0c0a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 1 --2- 172.21.15.204:0/3210974881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d0c09fa20 0x7f1d0c09fdf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 1 --2- 172.21.15.204:0/3210974881 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d0c0a2cd0 0x7f1d0c132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 1 -- 172.21.15.204:0/3210974881 >> 172.21.15.204:0/3210974881 conn(0x7f1d0c06e860 msgr2=0x7f1d0c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 1 -- 172.21.15.204:0/3210974881 shutdown_connections 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 1 -- 172.21.15.204:0/3210974881 wait complete. 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 10 monclient: build_initial_monmap 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.658+0000 7f1d1266f640 10 monclient: monmap: 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:13.675 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:13.676 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 1 Processor -- start 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 1 -- start start 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 10 monclient: init 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 10 monclient: _add_conns ranks=[0,5,4,6,8,2,1,7,3] 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d0c09fa20 0x7f1d0c131070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 10 monclient(hunting): picked mon.noname-a con 0x7f1d0c09fa20 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d0c0a08e0 0x7f1d0c09b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 10 monclient(hunting): picked mon.noname-h con 0x7f1d0c0a08e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d0c0a2cd0 0x7f1d0c09b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 10 monclient(hunting): picked mon.noname-c con 0x7f1d0c0a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c09d3d0 con 0x7f1d0c09fa20 2024-09-15T01:37:13.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:13.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c09cb60 con 0x7f1d0c0a2cd0 2024-09-15T01:37:13.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:13.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c11cbe0 con 0x7f1d0c0a08e0 2024-09-15T01:37:13.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:13.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d1266f640 10 monclient(hunting): authenticate will time out at 1049.301862s 2024-09-15T01:37:13.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d0b7fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d0c0a08e0 0x7f1d0c09b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:13.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d10c0c640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d0c0a2cd0 msgr2=0x7f1d0c09b770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:13.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d10c0c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d0c0a2cd0 0x7f1d0c09b770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:13.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d0bfff640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d0c09fa20 msgr2=0x7f1d0c131070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:13.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.659+0000 7f1d0b7fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d0c0a08e0 0x7f1d0c09b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:40912/0 (socket says 172.21.15.204:40912) 2024-09-15T01:37:13.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 1 -- 172.21.15.204:0/2192297659 learned_addr learned my addr 172.21.15.204:0/2192297659 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:13.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 10 monclient(hunting): get_auth_request con 0x7f1d0c0a08e0 auth_method 0 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0bfff640 1 --2- 172.21.15.204:0/2192297659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d0c09fa20 0x7f1d0c131070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 10 monclient(hunting): handle_auth_done global_id 6269 payload 824 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 1 -- 172.21.15.204:0/2192297659 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d0c0a2cd0 msgr2=0x7f1d0c09b770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 1 --2- 172.21.15.204:0/2192297659 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d0c0a2cd0 0x7f1d0c09b770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 1 -- 172.21.15.204:0/2192297659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d0c09fa20 msgr2=0x7f1d0c131070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 1 --2- 172.21.15.204:0/2192297659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d0c09fa20 0x7f1d0c131070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:13.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 1 monclient: found mon.noname-h 2024-09-15T01:37:13.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:13.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:13.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 1 -- 172.21.15.204:0/2192297659 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04009ae0 con 0x7f1d0c0a08e0 2024-09-15T01:37:13.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0b7fe640 1 --2- 172.21.15.204:0/2192297659 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d0c0a08e0 0x7f1d0c09b230 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009360 tx=0x7f1d00032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:13.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1cf3fff640 1 -- 172.21.15.204:0/2192297659 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1d0003b870 con 0x7f1d0c0a08e0 2024-09-15T01:37:13.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1cf3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:13.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1cf3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:13.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1cf3fff640 10 monclient: dump: 2024-09-15T01:37:13.680 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:13.680 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:13.680 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:13.680 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:13.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1cf3fff640 10 monclient: _finish_auth 0 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1cf3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1cf3fff640 1 -- 172.21.15.204:0/2192297659 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0003b9d0 con 0x7f1d0c0a08e0 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d1266f640 5 monclient: authenticate success, global_id 6269 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1cf3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1cf3fff640 1 -- 172.21.15.204:0/2192297659 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1d000313a0 con 0x7f1d0c0a08e0 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d1266f640 10 monclient: _renew_subs 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d1266f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d1266f640 1 -- 172.21.15.204:0/2192297659 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d00009390 con 0x7f1d0c0a08e0 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1cf3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1cf3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d0a7fc640 4 set_mon_vals no callback set 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1cf3fff640 10 monclient: dump: 2024-09-15T01:37:13.682 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:13.683 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d1266f640 10 monclient: _renew_subs 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d1266f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.660+0000 7f1d1266f640 1 -- 172.21.15.204:0/2192297659 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0c11e750 con 0x7f1d0c0a08e0 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.661+0000 7f1cf3fff640 1 -- 172.21.15.204:0/2192297659 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1d00039040 con 0x7f1d0c0a08e0 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.661+0000 7f1d1266f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.661+0000 7f1d1266f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.661+0000 7f1d1266f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.661+0000 7f1cf3fff640 1 --2- 172.21.15.204:0/2192297659 >> v2:172.21.15.204:6812/28210 conn(0x7f1cf80397b0 0x7f1cf803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.661+0000 7f1d1266f640 1 -- 172.21.15.204:0/2192297659 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0c11e8e0 con 0x7f1d0c0a08e0 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.661+0000 7f1cf3fff640 1 -- 172.21.15.204:0/2192297659 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 10249+0+0 (secure 0 0 0) 0x7f1d0006c4d0 con 0x7f1d0c0a08e0 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.661+0000 7f1d0bfff640 1 --2- 172.21.15.204:0/2192297659 >> v2:172.21.15.204:6812/28210 conn(0x7f1cf80397b0 0x7f1cf803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:13.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.661+0000 7f1d0bfff640 10 monclient: get_auth_request con 0x7f1cf80397b0 auth_method 0 2024-09-15T01:37:13.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.662+0000 7f1d0bfff640 1 --2- 172.21.15.204:0/2192297659 >> v2:172.21.15.204:6812/28210 conn(0x7f1cf80397b0 0x7f1cf803bc60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc006fd0 tx=0x7f1cfc006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:13.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.666+0000 7f1cf3fff640 1 -- 172.21.15.204:0/2192297659 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1d00031500 con 0x7f1d0c0a08e0 2024-09-15T01:37:13.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.666+0000 7f1cf3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:13.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.666+0000 7f1cf3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:13.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.824+0000 7f1d1266f640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:13.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.824+0000 7f1d1266f640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:13.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.824+0000 7f1d1266f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:13.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:13.824+0000 7f1d1266f640 1 -- 172.21.15.204:0/2192297659 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f1d0c070b80 con 0x7f1d0c0a08e0 2024-09-15T01:37:14.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.482+0000 7f1cf3fff640 1 -- 172.21.15.204:0/2192297659 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}]=0 added bucket h16 type host to location {rack=r1} v201) v1 ==== 168+0+0 (secure 0 0 0) 0x7f1d00037030 con 0x7f1d0c0a08e0 2024-09-15T01:37:14.483 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.482+0000 7f1cf3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:14.484 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.482+0000 7f1cf3fff640 10 monclient: _finish_command 2 = system:0 added bucket h16 type host to location {rack=r1} 2024-09-15T01:37:14.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.577+0000 7f1d1266f640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:14.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.577+0000 7f1d1266f640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:14.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.577+0000 7f1d1266f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:14.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.577+0000 7f1d1266f640 1 -- 172.21.15.204:0/2192297659 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f1d0c11c6d0 con 0x7f1d0c0a08e0 2024-09-15T01:37:14.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.578+0000 7f1cf3fff640 1 -- 172.21.15.204:0/2192297659 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h16' already exists v201) v1 ==== 147+0+0 (secure 0 0 0) 0x7f1d00037070 con 0x7f1d0c0a08e0 2024-09-15T01:37:14.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.578+0000 7f1cf3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:14.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.578+0000 7f1cf3fff640 10 monclient: _finish_command 3 = system:0 bucket 'h16' already exists 2024-09-15T01:37:14.580 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h16' already exists 2024-09-15T01:37:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 1 -- 172.21.15.204:0/2192297659 >> v2:172.21.15.204:6812/28210 conn(0x7f1cf80397b0 msgr2=0x7f1cf803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 1 --2- 172.21.15.204:0/2192297659 >> v2:172.21.15.204:6812/28210 conn(0x7f1cf80397b0 0x7f1cf803bc60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc006fd0 tx=0x7f1cfc006d30 comp rx=0 tx=0).stop 2024-09-15T01:37:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 10 monclient: shutdown 2024-09-15T01:37:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 1 -- 172.21.15.204:0/2192297659 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d0c0a08e0 msgr2=0x7f1d0c09b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 1 --2- 172.21.15.204:0/2192297659 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d0c0a08e0 0x7f1d0c09b230 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009360 tx=0x7f1d00032000 comp rx=0 tx=0).stop 2024-09-15T01:37:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 1 -- 172.21.15.204:0/2192297659 shutdown_connections 2024-09-15T01:37:14.582 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 1 --2- 172.21.15.204:0/2192297659 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1d0c0a2cd0 0x7f1d0c09b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:14.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 1 --2- 172.21.15.204:0/2192297659 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1d0c0a08e0 0x7f1d0c09b230 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:14.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 1 --2- 172.21.15.204:0/2192297659 >> v2:172.21.15.204:6812/28210 conn(0x7f1cf80397b0 0x7f1cf803bc60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:14.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 1 --2- 172.21.15.204:0/2192297659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d0c09fa20 0x7f1d0c131070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:14.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 1 -- 172.21.15.204:0/2192297659 >> 172.21.15.204:0/2192297659 conn(0x7f1d0c06e860 msgr2=0x7f1d0c0a5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:14.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 1 -- 172.21.15.204:0/2192297659 shutdown_connections 2024-09-15T01:37:14.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.580+0000 7f1d1266f640 1 -- 172.21.15.204:0/2192297659 wait complete. 2024-09-15T01:37:14.590 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:14.591 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h17 host rack=r1 2024-09-15T01:37:14.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.663+0000 7f8479add640 10 monclient: get_monmap_and_config 2024-09-15T01:37:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.663+0000 7f8479add640 10 monclient: build_initial_monmap 2024-09-15T01:37:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.663+0000 7f8479add640 10 monclient: monmap: 2024-09-15T01:37:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:14.664201+0000 2024-09-15T01:37:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:14.664201+0000 2024-09-15T01:37:14.680 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:14.681 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.664+0000 7f8479add640 1 Processor -- start 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.664+0000 7f8479add640 1 -- start start 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.664+0000 7f8479add640 10 monclient: init 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.664+0000 7f8479add640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.664+0000 7f8479add640 10 monclient: _add_conns ranks=[0,3,10,4,2,7,6,5,1,8,9] 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.664+0000 7f8479add640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f84740a2cd0 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.664+0000 7f8479add640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847409fa20 0x7f847409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.664+0000 7f8479add640 10 monclient(hunting): picked mon.noname-g con 0x7f847409fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.664+0000 7f8479add640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8474137c80 0x7f847413a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8479add640 10 monclient(hunting): picked mon.noname-f con 0x7f8474137c80 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8479add640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84740648a0 con 0x7f84740a2cd0 2024-09-15T01:37:14.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8479add640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8479add640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8474133310 con 0x7f8474137c80 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8479add640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8479add640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84740a30a0 con 0x7f847409fa20 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8479add640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8479add640 10 monclient(hunting): authenticate will time out at 1050.307029s 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f84737fe640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f84740a2cd0 legacy=0x7f8474132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847409fa20 0x7f847409fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8474137c80 0x7f847413a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8474137c80 0x7f847413a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41850/0 (socket says 172.21.15.204:41850) 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847409fa20 0x7f847409fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:40888/0 (socket says 172.21.15.204:40888) 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 1 -- 172.21.15.204:0/3026043773 learned_addr learned my addr 172.21.15.204:0/3026043773 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 10 monclient(hunting): get_auth_request con 0x7f8474137c80 auth_method 0 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:14.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8472ffd640 10 monclient(hunting): get_auth_request con 0x7f847409fa20 auth_method 0 2024-09-15T01:37:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8472ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8472ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8472ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8472ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8472ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8472ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:14.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 10 monclient(hunting): handle_auth_done global_id 7048 payload 274 2024-09-15T01:37:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 1 -- 172.21.15.204:0/3026043773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847409fa20 msgr2=0x7f847409fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 1 --2- 172.21.15.204:0/3026043773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847409fa20 0x7f847409fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 1 -- 172.21.15.204:0/3026043773 >> v1:172.21.15.178:6789/0 conn(0x7f84740a2cd0 legacy=0x7f8474132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 1 monclient: found mon.noname-f 2024-09-15T01:37:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8473fff640 1 -- 172.21.15.204:0/3026043773 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84740a0330 con 0x7f8474137c80 2024-09-15T01:37:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.665+0000 7f8472ffd640 1 --2- 172.21.15.204:0/3026043773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847409fa20 0x7f847409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8473fff640 1 --2- 172.21.15.204:0/3026043773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8474137c80 0x7f847413a060 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f846c009f20 tx=0x7f846c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24715b200a665fee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:14.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f84727fc640 1 -- 172.21.15.204:0/3026043773 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f846c007880 con 0x7f8474137c80 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f84727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f84727fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f84727fc640 10 monclient: dump: 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:14.686 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f84727fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f84727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 5 monclient: authenticate success, global_id 7048 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f84727fc640 1 -- 172.21.15.204:0/3026043773 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c02d050 con 0x7f8474137c80 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f84727fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:14.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 4 set_mon_vals no callback set 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f84727fc640 1 -- 172.21.15.204:0/3026043773 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f846c030360 con 0x7f8474137c80 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f84727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f84727fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f84727fc640 10 monclient: dump: 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:14.688 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 10 monclient: shutdown 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 1 -- 172.21.15.204:0/3026043773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8474137c80 msgr2=0x7f847413a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 1 --2- 172.21.15.204:0/3026043773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8474137c80 0x7f847413a060 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f846c009f20 tx=0x7f846c009840 comp rx=0 tx=0).stop 2024-09-15T01:37:14.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 1 -- 172.21.15.204:0/3026043773 shutdown_connections 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 1 --2- 172.21.15.204:0/3026043773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8474137c80 0x7f847413a060 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 1 --2- 172.21.15.204:0/3026043773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847409fa20 0x7f847409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 1 -- 172.21.15.204:0/3026043773 >> 172.21.15.204:0/3026043773 conn(0x7f847406e860 msgr2=0x7f847406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 1 -- 172.21.15.204:0/3026043773 shutdown_connections 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.666+0000 7f8479add640 1 -- 172.21.15.204:0/3026043773 wait complete. 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.667+0000 7f8479add640 10 monclient: build_initial_monmap 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.667+0000 7f8479add640 10 monclient: monmap: 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:14.690 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.667+0000 7f8479add640 1 Processor -- start 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.667+0000 7f8479add640 1 -- start start 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.667+0000 7f8479add640 10 monclient: init 2024-09-15T01:37:14.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 10 monclient: _add_conns ranks=[2,7,3,6,0,1,4,8,5] 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f847409fa20 0x7f84740997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 10 monclient(hunting): picked mon.noname-b con 0x7f847409fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84740a2cd0 0x7f8474099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 10 monclient(hunting): picked mon.noname-i con 0x7f84740a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8474137c80 0x7f847409a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 10 monclient(hunting): picked mon.noname-g con 0x7f8474137c80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847409d3d0 con 0x7f847409fa20 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847409cb60 con 0x7f8474137c80 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f847411c6c0 con 0x7f84740a2cd0 2024-09-15T01:37:14.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:14.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8479add640 10 monclient(hunting): authenticate will time out at 1050.310311s 2024-09-15T01:37:14.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8474137c80 0x7f847409a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:14.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f84737fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f847409fa20 msgr2=0x7f84740997f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:14.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84740a2cd0 0x7f8474099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:14.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f84737fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f847409fa20 0x7f84740997f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:14.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84740a2cd0 0x7f8474099d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:55140/0 (socket says 172.21.15.204:55140) 2024-09-15T01:37:14.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8472ffd640 1 -- 172.21.15.204:0/2209967728 learned_addr learned my addr 172.21.15.204:0/2209967728 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:14.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8472ffd640 10 monclient(hunting): get_auth_request con 0x7f84740a2cd0 auth_method 0 2024-09-15T01:37:14.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8472ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:14.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8472ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:14.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8472ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:14.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 10 monclient(hunting): get_auth_request con 0x7f8474137c80 auth_method 0 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8472ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8472ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8472ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 10 monclient(hunting): handle_auth_done global_id 15429 payload 824 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 1 -- 172.21.15.204:0/2209967728 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84740a2cd0 msgr2=0x7f8474099d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 1 --2- 172.21.15.204:0/2209967728 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84740a2cd0 0x7f8474099d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:14.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 1 -- 172.21.15.204:0/2209967728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f847409fa20 msgr2=0x7f84740997f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 1 --2- 172.21.15.204:0/2209967728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f847409fa20 0x7f84740997f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 1 monclient: found mon.noname-g 2024-09-15T01:37:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 1 -- 172.21.15.204:0/2209967728 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8468009390 con 0x7f8474137c80 2024-09-15T01:37:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8472ffd640 1 --2- 172.21.15.204:0/2209967728 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84740a2cd0 0x7f8474099d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f8473fff640 1 --2- 172.21.15.204:0/2209967728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8474137c80 0x7f847409a270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8474133450 tx=0x7f846c036000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.668+0000 7f845b7fe640 1 -- 172.21.15.204:0/2209967728 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f846c03cab0 con 0x7f8474137c80 2024-09-15T01:37:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f845b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f845b7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:14.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f845b7fe640 10 monclient: dump: 2024-09-15T01:37:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:14.696 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f845b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f845b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f8479add640 5 monclient: authenticate success, global_id 15429 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f8479add640 10 monclient: _renew_subs 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f845b7fe640 1 -- 172.21.15.204:0/2209967728 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c03cc10 con 0x7f8474137c80 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f8479add640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f8479add640 1 -- 172.21.15.204:0/2209967728 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846c009ae0 con 0x7f8474137c80 2024-09-15T01:37:14.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f845b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f845b7fe640 1 -- 172.21.15.204:0/2209967728 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f846c03cd80 con 0x7f8474137c80 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f845b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f845b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f845b7fe640 10 monclient: dump: 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:14.698 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:14.699 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:14.699 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f8471ffb640 4 set_mon_vals no callback set 2024-09-15T01:37:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f8479add640 10 monclient: _renew_subs 2024-09-15T01:37:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f8479add640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f8479add640 1 -- 172.21.15.204:0/2209967728 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847409b600 con 0x7f8474137c80 2024-09-15T01:37:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f845b7fe640 1 -- 172.21.15.204:0/2209967728 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f846c003040 con 0x7f8474137c80 2024-09-15T01:37:14.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.669+0000 7f8479add640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.670+0000 7f8479add640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.670+0000 7f8479add640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.670+0000 7f8479add640 1 -- 172.21.15.204:0/2209967728 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847411c830 con 0x7f8474137c80 2024-09-15T01:37:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.670+0000 7f845b7fe640 1 --2- 172.21.15.204:0/2209967728 >> v2:172.21.15.204:6812/28210 conn(0x7f84600397e0 0x7f846003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.670+0000 7f845b7fe640 1 -- 172.21.15.204:0/2209967728 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 10718+0+0 (secure 0 0 0) 0x7f846c02ae40 con 0x7f8474137c80 2024-09-15T01:37:14.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.670+0000 7f84737fe640 1 --2- 172.21.15.204:0/2209967728 >> v2:172.21.15.204:6812/28210 conn(0x7f84600397e0 0x7f846003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.670+0000 7f84737fe640 10 monclient: get_auth_request con 0x7f84600397e0 auth_method 0 2024-09-15T01:37:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.670+0000 7f84737fe640 1 --2- 172.21.15.204:0/2209967728 >> v2:172.21.15.204:6812/28210 conn(0x7f84600397e0 0x7f846003bc90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f84640038f0 tx=0x7f8464006ce0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.674+0000 7f845b7fe640 1 -- 172.21.15.204:0/2209967728 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f846c042af0 con 0x7f8474137c80 2024-09-15T01:37:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.674+0000 7f845b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:14.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.674+0000 7f845b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:14.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.831+0000 7f8479add640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:14.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.831+0000 7f8479add640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:14.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.831+0000 7f8479add640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:14.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:14.831+0000 7f8479add640 1 -- 172.21.15.204:0/2209967728 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8474070b80 con 0x7f8474137c80 2024-09-15T01:37:15.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.489+0000 7f845b7fe640 1 -- 172.21.15.204:0/2209967728 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}]=0 added bucket h17 type host to location {rack=r1} v202) v1 ==== 168+0+0 (secure 0 0 0) 0x7f846c040070 con 0x7f8474137c80 2024-09-15T01:37:15.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.489+0000 7f845b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:15.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.489+0000 7f845b7fe640 10 monclient: _finish_command 2 = system:0 added bucket h17 type host to location {rack=r1} 2024-09-15T01:37:15.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.587+0000 7f8479add640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:15.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.587+0000 7f8479add640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:15.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.587+0000 7f8479add640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:15.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.587+0000 7f8479add640 1 -- 172.21.15.204:0/2209967728 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f84740008d0 con 0x7f8474137c80 2024-09-15T01:37:15.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.587+0000 7f845b7fe640 1 -- 172.21.15.204:0/2209967728 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h17' already exists v202) v1 ==== 147+0+0 (secure 0 0 0) 0x7f846c042310 con 0x7f8474137c80 2024-09-15T01:37:15.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.587+0000 7f845b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:15.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.587+0000 7f845b7fe640 10 monclient: _finish_command 3 = system:0 bucket 'h17' already exists 2024-09-15T01:37:15.589 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h17' already exists 2024-09-15T01:37:15.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.589+0000 7f8479add640 1 -- 172.21.15.204:0/2209967728 >> v2:172.21.15.204:6812/28210 conn(0x7f84600397e0 msgr2=0x7f846003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:15.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.589+0000 7f8479add640 1 --2- 172.21.15.204:0/2209967728 >> v2:172.21.15.204:6812/28210 conn(0x7f84600397e0 0x7f846003bc90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f84640038f0 tx=0x7f8464006ce0 comp rx=0 tx=0).stop 2024-09-15T01:37:15.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.589+0000 7f8479add640 10 monclient: shutdown 2024-09-15T01:37:15.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.589+0000 7f8479add640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:15.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.589+0000 7f8479add640 1 -- 172.21.15.204:0/2209967728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8474137c80 msgr2=0x7f847409a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:15.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.589+0000 7f8479add640 1 --2- 172.21.15.204:0/2209967728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8474137c80 0x7f847409a270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8474133450 tx=0x7f846c036000 comp rx=0 tx=0).stop 2024-09-15T01:37:15.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.589+0000 7f8479add640 1 -- 172.21.15.204:0/2209967728 shutdown_connections 2024-09-15T01:37:15.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.589+0000 7f8479add640 1 --2- 172.21.15.204:0/2209967728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8474137c80 0x7f847409a270 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:15.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.589+0000 7f8479add640 1 --2- 172.21.15.204:0/2209967728 >> v2:172.21.15.204:6812/28210 conn(0x7f84600397e0 0x7f846003bc90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:15.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.589+0000 7f8479add640 1 --2- 172.21.15.204:0/2209967728 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84740a2cd0 0x7f8474099d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:15.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.589+0000 7f8479add640 1 --2- 172.21.15.204:0/2209967728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f847409fa20 0x7f84740997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:15.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.589+0000 7f8479add640 1 -- 172.21.15.204:0/2209967728 >> 172.21.15.204:0/2209967728 conn(0x7f847406e860 msgr2=0x7f84740a14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:15.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.590+0000 7f8479add640 1 -- 172.21.15.204:0/2209967728 shutdown_connections 2024-09-15T01:37:15.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.590+0000 7f8479add640 1 -- 172.21.15.204:0/2209967728 wait complete. 2024-09-15T01:37:15.600 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:15.600 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h18 host rack=r1 2024-09-15T01:37:15.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.673+0000 7f486d709640 10 monclient: get_monmap_and_config 2024-09-15T01:37:15.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.673+0000 7f486d709640 10 monclient: build_initial_monmap 2024-09-15T01:37:15.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.673+0000 7f486d709640 10 monclient: monmap: 2024-09-15T01:37:15.676 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:15.676 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:15.676 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:15.674121+0000 2024-09-15T01:37:15.676 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:15.674121+0000 2024-09-15T01:37:15.677 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:15.677 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:15.677 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:15.677 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:15.677 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:15.677 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:15.678 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:15.678 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:15.678 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:15.678 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:15.678 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:15.678 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:15.679 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:15.679 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:15.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 1 Processor -- start 2024-09-15T01:37:15.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 1 -- start start 2024-09-15T01:37:15.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 10 monclient: init 2024-09-15T01:37:15.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:15.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 10 monclient: _add_conns ranks=[0,5,3,1,7,8,2,6,10,4,9] 2024-09-15T01:37:15.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f48680a2cd0 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:15.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f486809fa20 0x7f486809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:15.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 10 monclient(hunting): picked mon.noname-h con 0x7f486809fa20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:15.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4868137c80 0x7f486813a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:15.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 10 monclient(hunting): picked mon.noname-g con 0x7f4868137c80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:15.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48680648a0 con 0x7f48680a2cd0 2024-09-15T01:37:15.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:15.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4868133310 con 0x7f4868137c80 2024-09-15T01:37:15.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:15.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48680a30a0 con 0x7f486809fa20 2024-09-15T01:37:15.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:15.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f486d709640 10 monclient(hunting): authenticate will time out at 1051.316883s 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.674+0000 7f4866ffd640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f48680a2cd0 legacy=0x7f4868132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f48677fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4868137c80 0x7f486813a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f486809fa20 0x7f486809fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f48677fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4868137c80 0x7f486813a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:40914/0 (socket says 172.21.15.204:40914) 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f486809fa20 0x7f486809fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:40920/0 (socket says 172.21.15.204:40920) 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f48677fe640 1 -- 172.21.15.204:0/1784833401 learned_addr learned my addr 172.21.15.204:0/1784833401 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f48677fe640 10 monclient(hunting): get_auth_request con 0x7f4868137c80 auth_method 0 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f48677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f48677fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f48677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 10 monclient(hunting): get_auth_request con 0x7f486809fa20 auth_method 0 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:15.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:15.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f48677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:15.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f48677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:15.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f48677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:15.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:15.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:15.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:15.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 10 monclient(hunting): handle_auth_done global_id 6278 payload 274 2024-09-15T01:37:15.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 1 -- 172.21.15.204:0/1784833401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4868137c80 msgr2=0x7f486813a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:15.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 1 --2- 172.21.15.204:0/1784833401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4868137c80 0x7f486813a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:15.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 1 -- 172.21.15.204:0/1784833401 >> v1:172.21.15.178:6789/0 conn(0x7f48680a2cd0 legacy=0x7f4868132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:15.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:15.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 1 monclient: found mon.noname-h 2024-09-15T01:37:15.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:15.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:15.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 1 -- 172.21.15.204:0/1784833401 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48680a0330 con 0x7f486809fa20 2024-09-15T01:37:15.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f48677fe640 1 --2- 172.21.15.204:0/1784833401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4868137c80 0x7f486813a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:15.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.675+0000 7f485ffff640 1 --2- 172.21.15.204:0/1784833401 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f486809fa20 0x7f486809fdf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4850009880 tx=0x7f48500090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=f40125079e4b5717 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:15.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f48667fc640 1 -- 172.21.15.204:0/1784833401 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f485002abc0 con 0x7f486809fa20 2024-09-15T01:37:15.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f48667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f48667fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f48667fc640 10 monclient: dump: 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:15.701 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:15.702 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f48667fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f48667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f48667fc640 1 -- 172.21.15.204:0/1784833401 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485002ad40 con 0x7f486809fa20 2024-09-15T01:37:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 5 monclient: authenticate success, global_id 6278 2024-09-15T01:37:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f48667fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:15.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f48667fc640 1 -- 172.21.15.204:0/1784833401 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4850031ca0 con 0x7f486809fa20 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f48667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 4 set_mon_vals no callback set 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f48667fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f48667fc640 10 monclient: dump: 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:15.703 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 10 monclient: shutdown 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 1 -- 172.21.15.204:0/1784833401 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f486809fa20 msgr2=0x7f486809fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 1 --2- 172.21.15.204:0/1784833401 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f486809fa20 0x7f486809fdf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4850009880 tx=0x7f48500090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 1 -- 172.21.15.204:0/1784833401 shutdown_connections 2024-09-15T01:37:15.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 1 --2- 172.21.15.204:0/1784833401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4868137c80 0x7f486813a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 1 --2- 172.21.15.204:0/1784833401 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f486809fa20 0x7f486809fdf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 1 -- 172.21.15.204:0/1784833401 >> 172.21.15.204:0/1784833401 conn(0x7f486806e860 msgr2=0x7f486806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 1 -- 172.21.15.204:0/1784833401 shutdown_connections 2024-09-15T01:37:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.676+0000 7f486d709640 1 -- 172.21.15.204:0/1784833401 wait complete. 2024-09-15T01:37:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient: build_initial_monmap 2024-09-15T01:37:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient: monmap: 2024-09-15T01:37:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:15.705 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:15.706 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 1 Processor -- start 2024-09-15T01:37:15.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 1 -- start start 2024-09-15T01:37:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient: init 2024-09-15T01:37:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient: _add_conns ranks=[2,7,6,1,4,3,8,0,5] 2024-09-15T01:37:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f486809fa20 0x7f48681340b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient(hunting): picked mon.noname-b con 0x7f486809fa20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f48680a2cd0 0x7f48681345f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient(hunting): picked mon.noname-i con 0x7f48680a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4868137c80 0x7f4868134b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient(hunting): picked mon.noname-d con 0x7f4868137c80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486809bc00 con 0x7f486809fa20 2024-09-15T01:37:15.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f48681360e0 con 0x7f4868137c80 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4868135ba0 con 0x7f48680a2cd0 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.677+0000 7f486d709640 10 monclient(hunting): authenticate will time out at 1051.319990s 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f48680a2cd0 0x7f48681345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f4866ffd640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f486809fa20 msgr2=0x7f48681340b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f48680a2cd0 0x7f48681345f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:55142/0 (socket says 172.21.15.204:55142) 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 1 -- 172.21.15.204:0/2682282954 learned_addr learned my addr 172.21.15.204:0/2682282954 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 10 monclient(hunting): get_auth_request con 0x7f48680a2cd0 auth_method 0 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f4866ffd640 1 --2- 172.21.15.204:0/2682282954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f486809fa20 0x7f48681340b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:15.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 10 monclient(hunting): handle_auth_done global_id 24514 payload 824 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f48677fe640 1 --2- 172.21.15.204:0/2682282954 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4868137c80 0x7f4868134b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 1 -- 172.21.15.204:0/2682282954 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4868137c80 msgr2=0x7f4868134b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 1 --2- 172.21.15.204:0/2682282954 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4868137c80 0x7f4868134b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 1 -- 172.21.15.204:0/2682282954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f486809fa20 msgr2=0x7f48681340b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 1 --2- 172.21.15.204:0/2682282954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f486809fa20 0x7f48681340b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 1 monclient: found mon.noname-i 2024-09-15T01:37:15.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 1 -- 172.21.15.204:0/2682282954 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4860009ae0 con 0x7f48680a2cd0 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485ffff640 1 --2- 172.21.15.204:0/2682282954 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f48680a2cd0 0x7f48681345f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4850035040 tx=0x7f4850032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485effd640 1 -- 172.21.15.204:0/2682282954 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f485003b870 con 0x7f48680a2cd0 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485effd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485effd640 10 monclient: dump: 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:15.710 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485effd640 10 monclient: _finish_auth 0 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485effd640 1 -- 172.21.15.204:0/2682282954 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485003b9d0 con 0x7f48680a2cd0 2024-09-15T01:37:15.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f486d709640 5 monclient: authenticate success, global_id 24514 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485effd640 1 -- 172.21.15.204:0/2682282954 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48500313a0 con 0x7f48680a2cd0 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f486d709640 10 monclient: _renew_subs 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f486d709640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f486d709640 1 -- 172.21.15.204:0/2682282954 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4850009390 con 0x7f48680a2cd0 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f4865ffb640 4 set_mon_vals no callback set 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f485effd640 10 monclient: dump: 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:15.712 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:15.713 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f486d709640 10 monclient: _renew_subs 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f486d709640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.678+0000 7f486d709640 1 -- 172.21.15.204:0/2682282954 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f486809cb60 con 0x7f48680a2cd0 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.679+0000 7f485effd640 1 -- 172.21.15.204:0/2682282954 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4850039040 con 0x7f48680a2cd0 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.679+0000 7f485effd640 1 --2- 172.21.15.204:0/2682282954 >> v2:172.21.15.204:6812/28210 conn(0x7f48480397e0 0x7f484803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.679+0000 7f485effd640 1 -- 172.21.15.204:0/2682282954 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 10931+0+0 (secure 0 0 0) 0x7f485006c3c0 con 0x7f48680a2cd0 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.679+0000 7f486d709640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.679+0000 7f486d709640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.679+0000 7f486d709640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.679+0000 7f486d709640 1 -- 172.21.15.204:0/2682282954 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4868070b80 con 0x7f48680a2cd0 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.679+0000 7f4866ffd640 1 --2- 172.21.15.204:0/2682282954 >> v2:172.21.15.204:6812/28210 conn(0x7f48480397e0 0x7f484803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:15.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.679+0000 7f4866ffd640 10 monclient: get_auth_request con 0x7f48480397e0 auth_method 0 2024-09-15T01:37:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.680+0000 7f4866ffd640 1 --2- 172.21.15.204:0/2682282954 >> v2:172.21.15.204:6812/28210 conn(0x7f48480397e0 0x7f484803bc90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f48580038f0 tx=0x7f4858006ce0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.684+0000 7f485effd640 1 -- 172.21.15.204:0/2682282954 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4850037080 con 0x7f48680a2cd0 2024-09-15T01:37:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.684+0000 7f485effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:15.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.684+0000 7f485effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:15.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.840+0000 7f486d709640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:15.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.840+0000 7f486d709640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:15.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.840+0000 7f486d709640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:15.842 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:15.840+0000 7f486d709640 1 -- 172.21.15.204:0/2682282954 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f48680008d0 con 0x7f48680a2cd0 2024-09-15T01:37:16.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.469+0000 7f485effd640 1 -- 172.21.15.204:0/2682282954 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}]=0 added bucket h18 type host to location {rack=r1} v203) v1 ==== 168+0+0 (secure 0 0 0) 0x7f4850042070 con 0x7f48680a2cd0 2024-09-15T01:37:16.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.469+0000 7f485effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:16.470 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.469+0000 7f485effd640 10 monclient: _finish_command 2 = system:0 added bucket h18 type host to location {rack=r1} 2024-09-15T01:37:16.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.573+0000 7f486d709640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:16.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.573+0000 7f486d709640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:16.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.573+0000 7f486d709640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:16.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.573+0000 7f486d709640 1 -- 172.21.15.204:0/2682282954 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f48680008d0 con 0x7f48680a2cd0 2024-09-15T01:37:16.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.574+0000 7f485effd640 1 -- 172.21.15.204:0/2682282954 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h18' already exists v203) v1 ==== 147+0+0 (secure 0 0 0) 0x7f4850044ae0 con 0x7f48680a2cd0 2024-09-15T01:37:16.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.574+0000 7f485effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:16.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.574+0000 7f485effd640 10 monclient: _finish_command 3 = system:0 bucket 'h18' already exists 2024-09-15T01:37:16.576 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h18' already exists 2024-09-15T01:37:16.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 1 -- 172.21.15.204:0/2682282954 >> v2:172.21.15.204:6812/28210 conn(0x7f48480397e0 msgr2=0x7f484803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:16.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 1 --2- 172.21.15.204:0/2682282954 >> v2:172.21.15.204:6812/28210 conn(0x7f48480397e0 0x7f484803bc90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f48580038f0 tx=0x7f4858006ce0 comp rx=0 tx=0).stop 2024-09-15T01:37:16.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 10 monclient: shutdown 2024-09-15T01:37:16.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:16.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 1 -- 172.21.15.204:0/2682282954 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f48680a2cd0 msgr2=0x7f48681345f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:16.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 1 --2- 172.21.15.204:0/2682282954 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f48680a2cd0 0x7f48681345f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4850035040 tx=0x7f4850032000 comp rx=0 tx=0).stop 2024-09-15T01:37:16.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 1 -- 172.21.15.204:0/2682282954 shutdown_connections 2024-09-15T01:37:16.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 1 --2- 172.21.15.204:0/2682282954 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4868137c80 0x7f4868134b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:16.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 1 --2- 172.21.15.204:0/2682282954 >> v2:172.21.15.204:6812/28210 conn(0x7f48480397e0 0x7f484803bc90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:16.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 1 --2- 172.21.15.204:0/2682282954 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f48680a2cd0 0x7f48681345f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:16.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 1 --2- 172.21.15.204:0/2682282954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f486809fa20 0x7f48681340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:16.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 1 -- 172.21.15.204:0/2682282954 >> 172.21.15.204:0/2682282954 conn(0x7f486806e860 msgr2=0x7f486812da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:16.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 1 -- 172.21.15.204:0/2682282954 shutdown_connections 2024-09-15T01:37:16.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.576+0000 7f486d709640 1 -- 172.21.15.204:0/2682282954 wait complete. 2024-09-15T01:37:16.586 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:16.586 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h19 host rack=r1 2024-09-15T01:37:16.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.660+0000 7f87c615d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:16.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.660+0000 7f87c615d640 10 monclient: build_initial_monmap 2024-09-15T01:37:16.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.660+0000 7f87c615d640 10 monclient: monmap: 2024-09-15T01:37:16.662 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:16.662 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:16.662 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:16.661220+0000 2024-09-15T01:37:16.662 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:16.661220+0000 2024-09-15T01:37:16.663 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:16.663 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:16.664 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:16.665 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:16.665 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:16.665 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:16.665 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:16.665 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:16.665 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:16.665 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:16.665 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:16.665 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:16.665 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:16.665 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:16.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.661+0000 7f87c615d640 1 Processor -- start 2024-09-15T01:37:16.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.661+0000 7f87c615d640 1 -- start start 2024-09-15T01:37:16.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.661+0000 7f87c615d640 10 monclient: init 2024-09-15T01:37:16.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.661+0000 7f87c615d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.661+0000 7f87c615d640 10 monclient: _add_conns ranks=[7,0,4,8,3,1,2,9,6,5,10] 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.661+0000 7f87c615d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87c00a2cd0 0x7f87c0132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.661+0000 7f87c615d640 10 monclient(hunting): picked mon.noname-i con 0x7f87c00a2cd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.661+0000 7f87c615d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f87c009fa20 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.661+0000 7f87c615d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87c0137c80 0x7f87c013a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.661+0000 7f87c615d640 10 monclient(hunting): picked mon.noname-c con 0x7f87c0137c80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87c615d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c00648a0 con 0x7f87c009fa20 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87c615d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87c615d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00a30a0 con 0x7f87c0137c80 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87c615d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87c615d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00a1db0 con 0x7f87c00a2cd0 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87c615d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87c615d640 10 monclient(hunting): authenticate will time out at 1052.304024s 2024-09-15T01:37:16.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87c00a2cd0 0x7f87c0132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:16.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87c00a2cd0 0x7f87c0132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:55144/0 (socket says 172.21.15.204:55144) 2024-09-15T01:37:16.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 1 -- 172.21.15.204:0/2763065734 learned_addr learned my addr 172.21.15.204:0/2763065734 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:16.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 10 monclient(hunting): get_auth_request con 0x7f87c00a2cd0 auth_method 0 2024-09-15T01:37:16.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87beffd640 1 -- 172.21.15.204:0/2763065734 >> v1:172.21.15.178:6789/0 conn(0x7f87c009fa20 legacy=0x7f87c009fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:16.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bffff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87c0137c80 msgr2=0x7f87c013a060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:16.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:16.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:16.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:16.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bffff640 1 --2- 172.21.15.204:0/2763065734 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87c0137c80 0x7f87c013a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:16.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:16.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:16.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 10 monclient(hunting): handle_auth_done global_id 24523 payload 274 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 1 -- 172.21.15.204:0/2763065734 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87c0137c80 msgr2=0x7f87c013a060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 1 --2- 172.21.15.204:0/2763065734 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87c0137c80 0x7f87c013a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 1 -- 172.21.15.204:0/2763065734 >> v1:172.21.15.178:6789/0 conn(0x7f87c009fa20 legacy=0x7f87c009fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 1 monclient: found mon.noname-i 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.662+0000 7f87bf7fe640 1 -- 172.21.15.204:0/2763065734 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c009dc50 con 0x7f87c00a2cd0 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87bf7fe640 1 --2- 172.21.15.204:0/2763065734 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87c00a2cd0 0x7f87c0132c00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009940 tx=0x7f87b00091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=2f3f51970f0e8e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87be7fc640 1 -- 172.21.15.204:0/2763065734 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87b002ac90 con 0x7f87c00a2cd0 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87be7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:16.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87be7fc640 10 monclient: dump: 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:16.677 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:16.678 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:16.678 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:16.678 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:16.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87be7fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:16.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:16.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87be7fc640 1 -- 172.21.15.204:0/2763065734 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b002ae10 con 0x7f87c00a2cd0 2024-09-15T01:37:16.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 5 monclient: authenticate success, global_id 24523 2024-09-15T01:37:16.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:16.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87be7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:16.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:16.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87be7fc640 1 -- 172.21.15.204:0/2763065734 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87b0031ca0 con 0x7f87c00a2cd0 2024-09-15T01:37:16.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 4 set_mon_vals no callback set 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87be7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87be7fc640 10 monclient: dump: 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:16.679 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 10 monclient: shutdown 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 1 -- 172.21.15.204:0/2763065734 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87c00a2cd0 msgr2=0x7f87c0132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 1 --2- 172.21.15.204:0/2763065734 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87c00a2cd0 0x7f87c0132c00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009940 tx=0x7f87b00091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 1 -- 172.21.15.204:0/2763065734 shutdown_connections 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 1 --2- 172.21.15.204:0/2763065734 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f87c0137c80 0x7f87c013a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:16.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 1 --2- 172.21.15.204:0/2763065734 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f87c00a2cd0 0x7f87c0132c00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 1 -- 172.21.15.204:0/2763065734 >> 172.21.15.204:0/2763065734 conn(0x7f87c006e860 msgr2=0x7f87c006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 1 -- 172.21.15.204:0/2763065734 shutdown_connections 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.663+0000 7f87c615d640 1 -- 172.21.15.204:0/2763065734 wait complete. 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.664+0000 7f87c615d640 10 monclient: build_initial_monmap 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.664+0000 7f87c615d640 10 monclient: monmap: 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:16.681 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.664+0000 7f87c615d640 1 Processor -- start 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.664+0000 7f87c615d640 1 -- start start 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.664+0000 7f87c615d640 10 monclient: init 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 10 monclient: _add_conns ranks=[8,5,6,2,1,4,7,3,0] 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f87c009fa20 0x7f87c0099870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:16.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 10 monclient(hunting): picked mon.noname-e con 0x7f87c009fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f87c00a2cd0 0x7f87c0099db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 10 monclient(hunting): picked mon.noname-h con 0x7f87c00a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87c0137c80 0x7f87c009a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 10 monclient(hunting): picked mon.noname-d con 0x7f87c0137c80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0130ec0 con 0x7f87c00a2cd0 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f87c009d440 con 0x7f87c0137c80 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f87c009c890 con 0x7f87c009fa20 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87c615d640 10 monclient(hunting): authenticate will time out at 1052.307393s 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f87c00a2cd0 0x7f87c0099db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:16.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87bf7fe640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f87c009fa20 msgr2=0x7f87c0099870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f87c00a2cd0 0x7f87c0099db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:40946/0 (socket says 172.21.15.204:40946) 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 1 -- 172.21.15.204:0/1972055729 learned_addr learned my addr 172.21.15.204:0/1972055729 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 10 monclient(hunting): get_auth_request con 0x7f87c00a2cd0 auth_method 0 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87bf7fe640 1 --2- 172.21.15.204:0/1972055729 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f87c009fa20 0x7f87c0099870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 10 monclient(hunting): handle_auth_done global_id 6287 payload 824 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87bffff640 1 --2- 172.21.15.204:0/1972055729 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87c0137c80 0x7f87c009a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 1 -- 172.21.15.204:0/1972055729 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f87c009fa20 msgr2=0x7f87c0099870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:16.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 1 --2- 172.21.15.204:0/1972055729 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f87c009fa20 0x7f87c0099870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 1 -- 172.21.15.204:0/1972055729 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87c0137c80 msgr2=0x7f87c009a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 1 --2- 172.21.15.204:0/1972055729 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87c0137c80 0x7f87c009a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 10 monclient: _finish_hunting 0 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 1 monclient: found mon.noname-h 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 1 -- 172.21.15.204:0/1972055729 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0009450 con 0x7f87c00a2cd0 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.665+0000 7f87beffd640 1 --2- 172.21.15.204:0/1972055729 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f87c00a2cd0 0x7f87c0099db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f87b400b5c0 tx=0x7f87b4009120 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 1 -- 172.21.15.204:0/1972055729 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87b4006150 con 0x7f87c00a2cd0 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 10 monclient: dump: 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:16.685 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:16.686 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87c615d640 5 monclient: authenticate success, global_id 6287 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 1 -- 172.21.15.204:0/1972055729 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b40062f0 con 0x7f87c00a2cd0 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87c615d640 10 monclient: _renew_subs 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87c615d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87c615d640 1 -- 172.21.15.204:0/1972055729 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c009cb60 con 0x7f87c00a2cd0 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 1 -- 172.21.15.204:0/1972055729 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87b40103c0 con 0x7f87c00a2cd0 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 10 monclient: dump: 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:16.687 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:16.688 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87c615d640 10 monclient: _renew_subs 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87bdffb640 4 set_mon_vals no callback set 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87c615d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87c615d640 1 -- 172.21.15.204:0/1972055729 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c011c730 con 0x7f87c00a2cd0 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.666+0000 7f87a77fe640 1 -- 172.21.15.204:0/1972055729 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f87b40064a0 con 0x7f87c00a2cd0 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.667+0000 7f87c615d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.667+0000 7f87c615d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.667+0000 7f87c615d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.667+0000 7f87c615d640 1 -- 172.21.15.204:0/1972055729 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c0070b80 con 0x7f87c00a2cd0 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.667+0000 7f87a77fe640 1 --2- 172.21.15.204:0/1972055729 >> v2:172.21.15.204:6812/28210 conn(0x7f87ac0397b0 0x7f87ac03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.667+0000 7f87a77fe640 1 -- 172.21.15.204:0/1972055729 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 11144+0+0 (secure 0 0 0) 0x7f87b404c0f0 con 0x7f87c00a2cd0 2024-09-15T01:37:16.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.667+0000 7f87bf7fe640 1 --2- 172.21.15.204:0/1972055729 >> v2:172.21.15.204:6812/28210 conn(0x7f87ac0397b0 0x7f87ac03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:16.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.667+0000 7f87bf7fe640 10 monclient: get_auth_request con 0x7f87ac0397b0 auth_method 0 2024-09-15T01:37:16.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.667+0000 7f87bf7fe640 1 --2- 172.21.15.204:0/1972055729 >> v2:172.21.15.204:6812/28210 conn(0x7f87ac0397b0 0x7f87ac03bc60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f87b0034000 tx=0x7f87b0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:16.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.671+0000 7f87a77fe640 1 -- 172.21.15.204:0/1972055729 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f87b400dad0 con 0x7f87c00a2cd0 2024-09-15T01:37:16.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.671+0000 7f87a77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:16.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.671+0000 7f87a77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:16.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.833+0000 7f87c615d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:16.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.833+0000 7f87c615d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:16.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.833+0000 7f87c615d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:16.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:16.833+0000 7f87c615d640 1 -- 172.21.15.204:0/1972055729 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f87c011cb10 con 0x7f87c00a2cd0 2024-09-15T01:37:17.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.504+0000 7f87a77fe640 1 -- 172.21.15.204:0/1972055729 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}]=0 added bucket h19 type host to location {rack=r1} v204) v1 ==== 168+0+0 (secure 0 0 0) 0x7f87b4017070 con 0x7f87c00a2cd0 2024-09-15T01:37:17.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.504+0000 7f87a77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:17.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.504+0000 7f87a77fe640 10 monclient: _finish_command 2 = system:0 added bucket h19 type host to location {rack=r1} 2024-09-15T01:37:17.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.602+0000 7f87c615d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:17.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.602+0000 7f87c615d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:17.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.602+0000 7f87c615d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:17.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.602+0000 7f87c615d640 1 -- 172.21.15.204:0/1972055729 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f87c00008d0 con 0x7f87c00a2cd0 2024-09-15T01:37:17.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.603+0000 7f87a77fe640 1 -- 172.21.15.204:0/1972055729 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h19' already exists v204) v1 ==== 147+0+0 (secure 0 0 0) 0x7f87b400d2f0 con 0x7f87c00a2cd0 2024-09-15T01:37:17.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.603+0000 7f87a77fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:17.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.603+0000 7f87a77fe640 10 monclient: _finish_command 3 = system:0 bucket 'h19' already exists 2024-09-15T01:37:17.605 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h19' already exists 2024-09-15T01:37:17.606 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.604+0000 7f87c615d640 1 -- 172.21.15.204:0/1972055729 >> v2:172.21.15.204:6812/28210 conn(0x7f87ac0397b0 msgr2=0x7f87ac03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.604+0000 7f87c615d640 1 --2- 172.21.15.204:0/1972055729 >> v2:172.21.15.204:6812/28210 conn(0x7f87ac0397b0 0x7f87ac03bc60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f87b0034000 tx=0x7f87b0032000 comp rx=0 tx=0).stop 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.604+0000 7f87c615d640 10 monclient: shutdown 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.604+0000 7f87c615d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.604+0000 7f87c615d640 1 -- 172.21.15.204:0/1972055729 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f87c00a2cd0 msgr2=0x7f87c0099db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.604+0000 7f87c615d640 1 --2- 172.21.15.204:0/1972055729 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f87c00a2cd0 0x7f87c0099db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f87b400b5c0 tx=0x7f87b4009120 comp rx=0 tx=0).stop 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.605+0000 7f87c615d640 1 -- 172.21.15.204:0/1972055729 shutdown_connections 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.605+0000 7f87c615d640 1 --2- 172.21.15.204:0/1972055729 >> v2:172.21.15.204:6812/28210 conn(0x7f87ac0397b0 0x7f87ac03bc60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.605+0000 7f87c615d640 1 --2- 172.21.15.204:0/1972055729 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f87c0137c80 0x7f87c009a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.605+0000 7f87c615d640 1 --2- 172.21.15.204:0/1972055729 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f87c00a2cd0 0x7f87c0099db0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.605+0000 7f87c615d640 1 --2- 172.21.15.204:0/1972055729 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f87c009fa20 0x7f87c0099870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.605+0000 7f87c615d640 1 -- 172.21.15.204:0/1972055729 >> 172.21.15.204:0/1972055729 conn(0x7f87c006e860 msgr2=0x7f87c00a5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.605+0000 7f87c615d640 1 -- 172.21.15.204:0/1972055729 shutdown_connections 2024-09-15T01:37:17.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.605+0000 7f87c615d640 1 -- 172.21.15.204:0/1972055729 wait complete. 2024-09-15T01:37:17.615 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:17.615 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h20 host rack=r1 2024-09-15T01:37:17.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.689+0000 7f842831d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:17.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.689+0000 7f842831d640 10 monclient: build_initial_monmap 2024-09-15T01:37:17.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.689+0000 7f842831d640 10 monclient: monmap: 2024-09-15T01:37:17.691 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:17.690239+0000 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:17.690239+0000 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:17.692 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:17.693 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:17.693 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:17.693 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:17.693 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:17.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 1 Processor -- start 2024-09-15T01:37:17.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 1 -- start start 2024-09-15T01:37:17.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 10 monclient: init 2024-09-15T01:37:17.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:17.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 10 monclient: _add_conns ranks=[10,3,9,7,1,0,8,2,6,4,5] 2024-09-15T01:37:17.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f84200a2cd0 0x7f8420132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:17.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 10 monclient(hunting): picked mon.noname-f con 0x7f84200a2cd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:17.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f842009fa20 0x7f842009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:17.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 10 monclient(hunting): picked mon.noname-g con 0x7f842009fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:17.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f84200a08e0 0x7f84200a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:17.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 10 monclient(hunting): picked mon.noname-a con 0x7f84200a08e0 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:17.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:17.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84200a30a0 con 0x7f84200a08e0 2024-09-15T01:37:17.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:17.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84200a0330 con 0x7f84200a2cd0 2024-09-15T01:37:17.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:17.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84200a1db0 con 0x7f842009fa20 2024-09-15T01:37:17.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:17.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.690+0000 7f842831d640 10 monclient(hunting): authenticate will time out at 1053.332975s 2024-09-15T01:37:17.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84260b9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f84200a2cd0 0x7f8420132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:17.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f842009fa20 0x7f842009fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:17.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84268ba640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7f84200a08e0 msgr2=0x7f84200a0d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:17.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84260b9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f84200a2cd0 0x7f8420132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48724/0 (socket says 172.21.15.204:48724) 2024-09-15T01:37:17.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84260b9640 1 -- 172.21.15.204:0/1706276663 learned_addr learned my addr 172.21.15.204:0/1706276663 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:17.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84260b9640 10 monclient(hunting): get_auth_request con 0x7f84200a2cd0 auth_method 0 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84260b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84260b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84260b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84268ba640 1 --2- 172.21.15.204:0/1706276663 >> v2:172.21.15.178:3300/0 conn(0x7f84200a08e0 0x7f84200a0d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 10 monclient(hunting): get_auth_request con 0x7f842009fa20 auth_method 0 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84260b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84260b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84260b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:17.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 10 monclient(hunting): handle_auth_done global_id 15447 payload 274 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 1 -- 172.21.15.204:0/1706276663 >> v2:172.21.15.204:3300/0 conn(0x7f84200a2cd0 msgr2=0x7f8420132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 1 --2- 172.21.15.204:0/1706276663 >> v2:172.21.15.204:3300/0 conn(0x7f84200a2cd0 0x7f8420132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 1 -- 172.21.15.204:0/1706276663 >> v2:172.21.15.178:3300/0 conn(0x7f84200a08e0 msgr2=0x7f84200a0d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 1 --2- 172.21.15.204:0/1706276663 >> v2:172.21.15.178:3300/0 conn(0x7f84200a08e0 0x7f84200a0d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 10 monclient: _finish_hunting 0 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 1 monclient: found mon.noname-g 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84258b8640 1 -- 172.21.15.204:0/1706276663 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200648a0 con 0x7f842009fa20 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.691+0000 7f84260b9640 1 --2- 172.21.15.204:0/1706276663 >> v2:172.21.15.204:3300/0 conn(0x7f84200a2cd0 0x7f8420132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84258b8640 1 --2- 172.21.15.204:0/1706276663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f842009fa20 0x7f842009fdf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f84200a3a30 tx=0x7f84100090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=2cfdf00b733df746 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:17.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84250b7640 1 -- 172.21.15.204:0/1706276663 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f841002ac00 con 0x7f842009fa20 2024-09-15T01:37:17.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84250b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:17.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84250b7640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:17.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84250b7640 10 monclient: dump: 2024-09-15T01:37:17.705 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:17.705 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:17.705 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:17.705 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:17.705 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:17.705 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:17.705 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:17.705 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:17.705 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84250b7640 10 monclient: _finish_auth 0 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84250b7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84250b7640 1 -- 172.21.15.204:0/1706276663 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841002ad80 con 0x7f842009fa20 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 5 monclient: authenticate success, global_id 15447 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:17.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84250b7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84250b7640 1 -- 172.21.15.204:0/1706276663 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8410031ca0 con 0x7f842009fa20 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84250b7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 4 set_mon_vals no callback set 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84250b7640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f84250b7640 10 monclient: dump: 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:17.707 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 10 monclient: shutdown 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 1 -- 172.21.15.204:0/1706276663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f842009fa20 msgr2=0x7f842009fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 1 --2- 172.21.15.204:0/1706276663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f842009fa20 0x7f842009fdf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f84200a3a30 tx=0x7f84100090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:17.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 1 -- 172.21.15.204:0/1706276663 shutdown_connections 2024-09-15T01:37:17.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 1 --2- 172.21.15.204:0/1706276663 >> v2:172.21.15.178:3300/0 conn(0x7f84200a08e0 0x7f84200a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:17.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 1 --2- 172.21.15.204:0/1706276663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f842009fa20 0x7f842009fdf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:17.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 1 --2- 172.21.15.204:0/1706276663 >> v2:172.21.15.204:3300/0 conn(0x7f84200a2cd0 0x7f8420132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:17.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 1 -- 172.21.15.204:0/1706276663 >> 172.21.15.204:0/1706276663 conn(0x7f842006e860 msgr2=0x7f842006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:17.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 1 -- 172.21.15.204:0/1706276663 shutdown_connections 2024-09-15T01:37:17.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.692+0000 7f842831d640 1 -- 172.21.15.204:0/1706276663 wait complete. 2024-09-15T01:37:17.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.693+0000 7f842831d640 10 monclient: build_initial_monmap 2024-09-15T01:37:17.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.693+0000 7f842831d640 10 monclient: monmap: 2024-09-15T01:37:17.709 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:17.709 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:17.709 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:17.709 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.693+0000 7f842831d640 1 Processor -- start 2024-09-15T01:37:17.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.693+0000 7f842831d640 1 -- start start 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.693+0000 7f842831d640 10 monclient: init 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 10 monclient: _add_conns ranks=[7,3,1,4,6,2,8,0,5] 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f842009fa20 0x7f8420141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 10 monclient(hunting): picked mon.noname-i con 0x7f842009fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84200a08e0 0x7f8420130f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 10 monclient(hunting): picked mon.noname-g con 0x7f84200a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84200a2cd0 0x7f842011dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 10 monclient(hunting): picked mon.noname-f con 0x7f84200a2cd0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84201213b0 con 0x7f84200a2cd0 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:17.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842009d3d0 con 0x7f84200a08e0 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f842009cb60 con 0x7f842009fa20 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f842831d640 10 monclient(hunting): authenticate will time out at 1053.336264s 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84200a08e0 0x7f8420130f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84260b9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f842009fa20 0x7f8420141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84200a08e0 0x7f8420130f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48460/0 (socket says 172.21.15.204:48460) 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 1 -- 172.21.15.204:0/3888580569 learned_addr learned my addr 172.21.15.204:0/3888580569 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 10 monclient(hunting): get_auth_request con 0x7f84200a08e0 auth_method 0 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:17.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84260b9640 10 monclient(hunting): get_auth_request con 0x7f842009fa20 auth_method 0 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84260b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84260b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84260b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84268ba640 1 --2- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84200a2cd0 0x7f842011dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84268ba640 10 monclient(hunting): get_auth_request con 0x7f84200a2cd0 auth_method 0 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84268ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84268ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84268ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84260b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84260b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:17.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84260b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84268ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84268ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84268ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 10 monclient(hunting): handle_auth_done global_id 15456 payload 824 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 1 -- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f842009fa20 msgr2=0x7f8420141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 1 --2- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f842009fa20 0x7f8420141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 1 -- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84200a2cd0 msgr2=0x7f842011dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 1 --2- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84200a2cd0 0x7f842011dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 10 monclient: _finish_hunting 0 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 1 monclient: found mon.noname-g 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:17.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 1 -- 172.21.15.204:0/3888580569 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c009450 con 0x7f84200a08e0 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84260b9640 1 --2- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f842009fa20 0x7f8420141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84268ba640 1 --2- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84200a2cd0 0x7f842011dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84258b8640 1 --2- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84200a08e0 0x7f8420130f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8410000c00 tx=0x7f8410032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84167fc640 1 -- 172.21.15.204:0/3888580569 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f841003b870 con 0x7f84200a08e0 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84167fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84167fc640 10 monclient: dump: 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:17.715 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84167fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.694+0000 7f84167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:17.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f842831d640 5 monclient: authenticate success, global_id 15456 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f84167fc640 1 -- 172.21.15.204:0/3888580569 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841003b9d0 con 0x7f84200a08e0 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f84167fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f84167fc640 1 -- 172.21.15.204:0/3888580569 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84100313a0 con 0x7f84200a08e0 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f842831d640 10 monclient: _renew_subs 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f842831d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f842831d640 1 -- 172.21.15.204:0/3888580569 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8410009390 con 0x7f84200a08e0 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f84167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f84248b6640 4 set_mon_vals no callback set 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f84167fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f84167fc640 10 monclient: dump: 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:17.717 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:17.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f842831d640 10 monclient: _renew_subs 2024-09-15T01:37:17.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f842831d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:17.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f842831d640 1 -- 172.21.15.204:0/3888580569 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842011f5a0 con 0x7f84200a08e0 2024-09-15T01:37:17.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f84167fc640 1 -- 172.21.15.204:0/3888580569 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8410039040 con 0x7f84200a08e0 2024-09-15T01:37:17.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.695+0000 7f84167fc640 1 --2- 172.21.15.204:0/3888580569 >> v2:172.21.15.204:6812/28210 conn(0x7f84040397b0 0x7f840403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:17.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.696+0000 7f842831d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:17.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.696+0000 7f842831d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:17.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.696+0000 7f842831d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:17.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.696+0000 7f84167fc640 1 -- 172.21.15.204:0/3888580569 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 11357+0+0 (secure 0 0 0) 0x7f841006c740 con 0x7f84200a08e0 2024-09-15T01:37:17.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.696+0000 7f842831d640 1 -- 172.21.15.204:0/3888580569 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8420070b80 con 0x7f84200a08e0 2024-09-15T01:37:17.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.696+0000 7f84260b9640 1 --2- 172.21.15.204:0/3888580569 >> v2:172.21.15.204:6812/28210 conn(0x7f84040397b0 0x7f840403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:17.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.696+0000 7f84260b9640 10 monclient: get_auth_request con 0x7f84040397b0 auth_method 0 2024-09-15T01:37:17.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.696+0000 7f84260b9640 1 --2- 172.21.15.204:0/3888580569 >> v2:172.21.15.204:6812/28210 conn(0x7f84040397b0 0x7f840403bc60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f841c006c90 tx=0x7f841c0091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:17.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.700+0000 7f84167fc640 1 -- 172.21.15.204:0/3888580569 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8410042020 con 0x7f84200a08e0 2024-09-15T01:37:17.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.700+0000 7f84167fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:17.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.700+0000 7f84167fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:17.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.859+0000 7f842831d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:17.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.859+0000 7f842831d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:17.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.859+0000 7f842831d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:17.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:17.859+0000 7f842831d640 1 -- 172.21.15.204:0/3888580569 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8420121190 con 0x7f84200a08e0 2024-09-15T01:37:18.516 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.514+0000 7f84167fc640 1 -- 172.21.15.204:0/3888580569 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}]=0 added bucket h20 type host to location {rack=r1} v205) v1 ==== 168+0+0 (secure 0 0 0) 0x7f8410031500 con 0x7f84200a08e0 2024-09-15T01:37:18.516 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.514+0000 7f84167fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:18.516 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.514+0000 7f84167fc640 10 monclient: _finish_command 2 = system:0 added bucket h20 type host to location {rack=r1} 2024-09-15T01:37:18.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.612+0000 7f842831d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:18.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.612+0000 7f842831d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:18.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.612+0000 7f842831d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:18.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.612+0000 7f842831d640 1 -- 172.21.15.204:0/3888580569 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f84200008d0 con 0x7f84200a08e0 2024-09-15T01:37:18.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.613+0000 7f84167fc640 1 -- 172.21.15.204:0/3888580569 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h20' already exists v205) v1 ==== 147+0+0 (secure 0 0 0) 0x7f8410032440 con 0x7f84200a08e0 2024-09-15T01:37:18.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.613+0000 7f84167fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:18.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.613+0000 7f84167fc640 10 monclient: _finish_command 3 = system:0 bucket 'h20' already exists 2024-09-15T01:37:18.615 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h20' already exists 2024-09-15T01:37:18.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 1 -- 172.21.15.204:0/3888580569 >> v2:172.21.15.204:6812/28210 conn(0x7f84040397b0 msgr2=0x7f840403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:18.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 1 --2- 172.21.15.204:0/3888580569 >> v2:172.21.15.204:6812/28210 conn(0x7f84040397b0 0x7f840403bc60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f841c006c90 tx=0x7f841c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:18.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 10 monclient: shutdown 2024-09-15T01:37:18.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:18.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 1 -- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84200a08e0 msgr2=0x7f8420130f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:18.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 1 --2- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84200a08e0 0x7f8420130f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8410000c00 tx=0x7f8410032000 comp rx=0 tx=0).stop 2024-09-15T01:37:18.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 1 -- 172.21.15.204:0/3888580569 shutdown_connections 2024-09-15T01:37:18.618 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 1 --2- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84200a2cd0 0x7f842011dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:18.618 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 1 --2- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84200a08e0 0x7f8420130f80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:18.618 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 1 --2- 172.21.15.204:0/3888580569 >> v2:172.21.15.204:6812/28210 conn(0x7f84040397b0 0x7f840403bc60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:18.618 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 1 --2- 172.21.15.204:0/3888580569 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f842009fa20 0x7f8420141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:18.618 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 1 -- 172.21.15.204:0/3888580569 >> 172.21.15.204:0/3888580569 conn(0x7f842006e860 msgr2=0x7f84200a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:18.618 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 1 -- 172.21.15.204:0/3888580569 shutdown_connections 2024-09-15T01:37:18.618 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.615+0000 7f842831d640 1 -- 172.21.15.204:0/3888580569 wait complete. 2024-09-15T01:37:18.626 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:18.626 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h21 host rack=r1 2024-09-15T01:37:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.699+0000 7f082de94640 10 monclient: get_monmap_and_config 2024-09-15T01:37:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.699+0000 7f082de94640 10 monclient: build_initial_monmap 2024-09-15T01:37:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.699+0000 7f082de94640 10 monclient: monmap: 2024-09-15T01:37:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:18.700145+0000 2024-09-15T01:37:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:18.700145+0000 2024-09-15T01:37:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:18.702 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:18.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 1 Processor -- start 2024-09-15T01:37:18.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 1 -- start start 2024-09-15T01:37:18.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 10 monclient: init 2024-09-15T01:37:18.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:18.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 10 monclient: _add_conns ranks=[4,8,5,6,2,0,9,7,10,3,1] 2024-09-15T01:37:18.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f08280a2cd0 0x7f0828132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:18.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 10 monclient(hunting): picked mon.noname-c con 0x7f08280a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:18.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f082809fa20 0x7f082809fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:18.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 10 monclient(hunting): picked mon.noname-e con 0x7f082809fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:18.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f08280a08e0 0x7f08280a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:18.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 10 monclient(hunting): picked mon.noname-h con 0x7f08280a08e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:18.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:18.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08280a30a0 con 0x7f08280a2cd0 2024-09-15T01:37:18.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:18.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08280a0330 con 0x7f08280a08e0 2024-09-15T01:37:18.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:18.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f08280a1db0 con 0x7f082809fa20 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.700+0000 7f082de94640 10 monclient(hunting): authenticate will time out at 1054.342939s 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f08280a08e0 0x7f08280a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f08277fe640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f08280a2cd0 msgr2=0x7f0828132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0826ffd640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f082809fa20 msgr2=0x7f082809fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f08280a08e0 0x7f08280a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:38476/0 (socket says 172.21.15.204:38476) 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f08277fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f08280a2cd0 0x7f0828132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 1 -- 172.21.15.204:0/2775763406 learned_addr learned my addr 172.21.15.204:0/2775763406 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 10 monclient(hunting): get_auth_request con 0x7f08280a08e0 auth_method 0 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0826ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f082809fa20 0x7f082809fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:18.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 10 monclient(hunting): handle_auth_done global_id 6296 payload 274 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 1 -- 172.21.15.204:0/2775763406 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f082809fa20 msgr2=0x7f082809fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 1 --2- 172.21.15.204:0/2775763406 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f082809fa20 0x7f082809fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 1 -- 172.21.15.204:0/2775763406 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f08280a2cd0 msgr2=0x7f0828132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 1 --2- 172.21.15.204:0/2775763406 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f08280a2cd0 0x7f0828132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 1 monclient: found mon.noname-h 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 1 -- 172.21.15.204:0/2775763406 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08280648a0 con 0x7f08280a08e0 2024-09-15T01:37:18.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.701+0000 7f0827fff640 1 --2- 172.21.15.204:0/2775763406 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f08280a08e0 0x7f08280a0d50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0820009f50 tx=0x7f0820009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=4e5dd06028c0b2bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f08267fc640 1 -- 172.21.15.204:0/2775763406 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0820007880 con 0x7f08280a08e0 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f08267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f08267fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f08267fc640 10 monclient: dump: 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:18.719 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f08267fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f08267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f08267fc640 1 -- 172.21.15.204:0/2775763406 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082002d050 con 0x7f08280a08e0 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 5 monclient: authenticate success, global_id 6296 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f08267fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:18.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f08267fc640 1 -- 172.21.15.204:0/2775763406 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0820007400 con 0x7f08280a08e0 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 4 set_mon_vals no callback set 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f08267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f08267fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f08267fc640 10 monclient: dump: 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:18.721 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 10 monclient: shutdown 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 1 -- 172.21.15.204:0/2775763406 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f08280a08e0 msgr2=0x7f08280a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 1 --2- 172.21.15.204:0/2775763406 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f08280a08e0 0x7f08280a0d50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0820009f50 tx=0x7f0820009840 comp rx=0 tx=0).stop 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 1 -- 172.21.15.204:0/2775763406 shutdown_connections 2024-09-15T01:37:18.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 1 --2- 172.21.15.204:0/2775763406 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f08280a08e0 0x7f08280a0d50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:18.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 1 --2- 172.21.15.204:0/2775763406 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f082809fa20 0x7f082809fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:18.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 1 --2- 172.21.15.204:0/2775763406 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f08280a2cd0 0x7f0828132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:18.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 1 -- 172.21.15.204:0/2775763406 >> 172.21.15.204:0/2775763406 conn(0x7f082806e860 msgr2=0x7f082806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:18.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 1 -- 172.21.15.204:0/2775763406 shutdown_connections 2024-09-15T01:37:18.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.702+0000 7f082de94640 1 -- 172.21.15.204:0/2775763406 wait complete. 2024-09-15T01:37:18.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.703+0000 7f082de94640 10 monclient: build_initial_monmap 2024-09-15T01:37:18.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.703+0000 7f082de94640 10 monclient: monmap: 2024-09-15T01:37:18.723 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:18.723 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:18.723 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:18.723 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:18.723 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.703+0000 7f082de94640 1 Processor -- start 2024-09-15T01:37:18.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.703+0000 7f082de94640 1 -- start start 2024-09-15T01:37:18.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.703+0000 7f082de94640 10 monclient: init 2024-09-15T01:37:18.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:18.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient: _add_conns ranks=[6,7,2,1,5,4,3,8,0] 2024-09-15T01:37:18.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f082809fa20 0x7f0828130e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:18.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient(hunting): picked mon.noname-d con 0x7f082809fa20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:18.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08280a08e0 0x7f082811d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:18.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient(hunting): picked mon.noname-i con 0x7f08280a08e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:18.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08280a2cd0 0x7f082811de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:18.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient(hunting): picked mon.noname-b con 0x7f08280a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:18.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:18.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0828133e80 con 0x7f08280a2cd0 2024-09-15T01:37:18.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0828121510 con 0x7f082809fa20 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f082809cb40 con 0x7f08280a08e0 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient(hunting): authenticate will time out at 1054.346130s 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0827fff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08280a2cd0 msgr2=0x7f082811de80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08280a08e0 0x7f082811d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0827fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08280a2cd0 0x7f082811de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08280a08e0 0x7f082811d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:42938/0 (socket says 172.21.15.204:42938) 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 1 -- 172.21.15.204:0/2468185246 learned_addr learned my addr 172.21.15.204:0/2468185246 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 10 monclient(hunting): get_auth_request con 0x7f08280a08e0 auth_method 0 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:18.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f08277fe640 1 --2- 172.21.15.204:0/2468185246 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f082809fa20 0x7f0828130e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 10 monclient(hunting): handle_auth_done global_id 24541 payload 824 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 1 -- 172.21.15.204:0/2468185246 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f082809fa20 msgr2=0x7f0828130e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 1 --2- 172.21.15.204:0/2468185246 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f082809fa20 0x7f0828130e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 1 -- 172.21.15.204:0/2468185246 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08280a2cd0 msgr2=0x7f082811de80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 1 --2- 172.21.15.204:0/2468185246 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08280a2cd0 0x7f082811de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 1 monclient: found mon.noname-i 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:18.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 1 -- 172.21.15.204:0/2468185246 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0820009ae0 con 0x7f08280a08e0 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f08277fe640 10 monclient: get_auth_request con 0x7f082809fa20 auth_method 0 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f08277fe640 1 --2- 172.21.15.204:0/2468185246 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f082809fa20 0x7f0828130e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0826ffd640 1 --2- 172.21.15.204:0/2468185246 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08280a08e0 0x7f082811d940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f081c00b5c0 tx=0x7f081c009120 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f080f7fe640 1 -- 172.21.15.204:0/2468185246 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f081c009280 con 0x7f08280a08e0 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f080f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f080f7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f080f7fe640 10 monclient: dump: 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:18.728 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f080f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:18.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f080f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 5 monclient: authenticate success, global_id 24541 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f080f7fe640 1 -- 172.21.15.204:0/2468185246 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081c0050f0 con 0x7f08280a08e0 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient: _renew_subs 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 1 -- 172.21.15.204:0/2468185246 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082811fff0 con 0x7f08280a08e0 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f080f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f080f7fe640 1 -- 172.21.15.204:0/2468185246 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f081c005760 con 0x7f08280a08e0 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f080f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f080f7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f080f7fe640 10 monclient: dump: 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:18.730 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:18.731 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:18.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient: _renew_subs 2024-09-15T01:37:18.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:18.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f0825ffb640 4 set_mon_vals no callback set 2024-09-15T01:37:18.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.704+0000 7f082de94640 1 -- 172.21.15.204:0/2468185246 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082811f420 con 0x7f08280a08e0 2024-09-15T01:37:18.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.705+0000 7f080f7fe640 1 -- 172.21.15.204:0/2468185246 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f081c0052e0 con 0x7f08280a08e0 2024-09-15T01:37:18.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.705+0000 7f080f7fe640 1 --2- 172.21.15.204:0/2468185246 >> v2:172.21.15.204:6812/28210 conn(0x7f08140397b0 0x7f081403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:18.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.705+0000 7f082de94640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:18.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.705+0000 7f082de94640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:18.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.705+0000 7f082de94640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:18.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.705+0000 7f082de94640 1 -- 172.21.15.204:0/2468185246 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0828070b80 con 0x7f08280a08e0 2024-09-15T01:37:18.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.706+0000 7f080f7fe640 1 -- 172.21.15.204:0/2468185246 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 11570+0+0 (secure 0 0 0) 0x7f081c04ca70 con 0x7f08280a08e0 2024-09-15T01:37:18.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.706+0000 7f08277fe640 1 --2- 172.21.15.204:0/2468185246 >> v2:172.21.15.204:6812/28210 conn(0x7f08140397b0 0x7f081403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:18.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.706+0000 7f08277fe640 10 monclient: get_auth_request con 0x7f08140397b0 auth_method 0 2024-09-15T01:37:18.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.706+0000 7f08277fe640 1 --2- 172.21.15.204:0/2468185246 >> v2:172.21.15.204:6812/28210 conn(0x7f08140397b0 0x7f081403bc60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f08180038f0 tx=0x7f0818008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:18.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.710+0000 7f080f7fe640 1 -- 172.21.15.204:0/2468185246 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f081c01a460 con 0x7f08280a08e0 2024-09-15T01:37:18.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.710+0000 7f080f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:18.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.710+0000 7f080f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:18.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.869+0000 7f082de94640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:18.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.869+0000 7f082de94640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:18.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.869+0000 7f082de94640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:18.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:18.869+0000 7f082de94640 1 -- 172.21.15.204:0/2468185246 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f08280008d0 con 0x7f08280a08e0 2024-09-15T01:37:19.524 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.523+0000 7f080f7fe640 1 -- 172.21.15.204:0/2468185246 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}]=0 added bucket h21 type host to location {rack=r1} v206) v1 ==== 168+0+0 (secure 0 0 0) 0x7f081c019bb0 con 0x7f08280a08e0 2024-09-15T01:37:19.524 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.523+0000 7f080f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:19.525 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.523+0000 7f080f7fe640 10 monclient: _finish_command 2 = system:0 added bucket h21 type host to location {rack=r1} 2024-09-15T01:37:19.620 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.619+0000 7f082de94640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:19.621 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.619+0000 7f082de94640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:19.621 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.619+0000 7f082de94640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:19.621 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.619+0000 7f082de94640 1 -- 172.21.15.204:0/2468185246 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f0828067450 con 0x7f08280a08e0 2024-09-15T01:37:19.621 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.620+0000 7f080f7fe640 1 -- 172.21.15.204:0/2468185246 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h21' already exists v206) v1 ==== 147+0+0 (secure 0 0 0) 0x7f081c019d90 con 0x7f08280a08e0 2024-09-15T01:37:19.621 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.620+0000 7f080f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:19.622 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.620+0000 7f080f7fe640 10 monclient: _finish_command 3 = system:0 bucket 'h21' already exists 2024-09-15T01:37:19.622 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h21' already exists 2024-09-15T01:37:19.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 1 -- 172.21.15.204:0/2468185246 >> v2:172.21.15.204:6812/28210 conn(0x7f08140397b0 msgr2=0x7f081403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:19.623 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 1 --2- 172.21.15.204:0/2468185246 >> v2:172.21.15.204:6812/28210 conn(0x7f08140397b0 0x7f081403bc60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f08180038f0 tx=0x7f0818008040 comp rx=0 tx=0).stop 2024-09-15T01:37:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 10 monclient: shutdown 2024-09-15T01:37:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 1 -- 172.21.15.204:0/2468185246 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08280a08e0 msgr2=0x7f082811d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 1 --2- 172.21.15.204:0/2468185246 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08280a08e0 0x7f082811d940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f081c00b5c0 tx=0x7f081c009120 comp rx=0 tx=0).stop 2024-09-15T01:37:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 1 -- 172.21.15.204:0/2468185246 shutdown_connections 2024-09-15T01:37:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 1 --2- 172.21.15.204:0/2468185246 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08280a2cd0 0x7f082811de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 1 --2- 172.21.15.204:0/2468185246 >> v2:172.21.15.204:6812/28210 conn(0x7f08140397b0 0x7f081403bc60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 1 --2- 172.21.15.204:0/2468185246 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08280a08e0 0x7f082811d940 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 1 --2- 172.21.15.204:0/2468185246 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f082809fa20 0x7f0828130e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 1 -- 172.21.15.204:0/2468185246 >> 172.21.15.204:0/2468185246 conn(0x7f082806e860 msgr2=0x7f08280a5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 1 -- 172.21.15.204:0/2468185246 shutdown_connections 2024-09-15T01:37:19.624 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.622+0000 7f082de94640 1 -- 172.21.15.204:0/2468185246 wait complete. 2024-09-15T01:37:19.632 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:19.632 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h22 host rack=r1 2024-09-15T01:37:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.707+0000 7f8ab2e80640 10 monclient: get_monmap_and_config 2024-09-15T01:37:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.707+0000 7f8ab2e80640 10 monclient: build_initial_monmap 2024-09-15T01:37:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.707+0000 7f8ab2e80640 10 monclient: monmap: 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:19.707564+0000 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:19.707564+0000 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:19.710 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.707+0000 7f8ab2e80640 1 Processor -- start 2024-09-15T01:37:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.707+0000 7f8ab2e80640 1 -- start start 2024-09-15T01:37:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.707+0000 7f8ab2e80640 10 monclient: init 2024-09-15T01:37:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 10 monclient: _add_conns ranks=[10,9,6,4,1,3,7,2,8,5,0] 2024-09-15T01:37:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8aac0a4590 0x7f8aac09e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 10 monclient(hunting): picked mon.noname-f con 0x7f8aac0a4590 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8aac06edd0 0x7f8aac09ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 10 monclient(hunting): picked mon.noname-a con 0x7f8aac06edd0 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:19.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8aac09f1d0 0x7f8aac1307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 10 monclient(hunting): picked mon.noname-d con 0x7f8aac09f1d0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac09f610 con 0x7f8aac06edd0 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac0a4960 con 0x7f8aac0a4590 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac06dba0 con 0x7f8aac09f1d0 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab2e80640 10 monclient(hunting): authenticate will time out at 1055.350315s 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8aac0a4590 0x7f8aac09e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8aac0a4590 0x7f8aac09e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48746/0 (socket says 172.21.15.204:48746) 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 1 -- 172.21.15.204:0/2418944169 learned_addr learned my addr 172.21.15.204:0/2418944169 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:19.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 10 monclient(hunting): get_auth_request con 0x7f8aac0a4590 auth_method 0 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8aabfff640 1 -- 172.21.15.204:0/2418944169 >> v2:172.21.15.178:3300/0 conn(0x7f8aac06edd0 msgr2=0x7f8aac09ec90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8aabfff640 1 --2- 172.21.15.204:0/2418944169 >> v2:172.21.15.178:3300/0 conn(0x7f8aac06edd0 0x7f8aac09ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 10 monclient(hunting): handle_auth_done global_id 7075 payload 274 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 1 -- 172.21.15.204:0/2418944169 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8aac09f1d0 msgr2=0x7f8aac1307a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 1 --2- 172.21.15.204:0/2418944169 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8aac09f1d0 0x7f8aac1307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 1 -- 172.21.15.204:0/2418944169 >> v2:172.21.15.178:3300/0 conn(0x7f8aac06edd0 msgr2=0x7f8aac09ec90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:19.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 1 --2- 172.21.15.204:0/2418944169 >> v2:172.21.15.178:3300/0 conn(0x7f8aac06edd0 0x7f8aac09ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 10 monclient: _finish_hunting 0 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 1 monclient: found mon.noname-f 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.708+0000 7f8ab0c1c640 1 -- 172.21.15.204:0/2418944169 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0648a0 con 0x7f8aac0a4590 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8ab0c1c640 1 --2- 172.21.15.204:0/2418944169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aac0a4590 0x7f8aac09e750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0a4db0 tx=0x7f8a9c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbc3d094ba5d0fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8aab7fe640 1 -- 172.21.15.204:0/2418944169 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a9c02acc0 con 0x7f8aac0a4590 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8aab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8aab7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8aab7fe640 10 monclient: dump: 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:19.722 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:19.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8aab7fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8aab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8aab7fe640 1 -- 172.21.15.204:0/2418944169 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c02ae40 con 0x7f8aac0a4590 2024-09-15T01:37:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8ab2e80640 5 monclient: authenticate success, global_id 7075 2024-09-15T01:37:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8ab2e80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8aab7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8ab2e80640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8aab7fe640 1 -- 172.21.15.204:0/2418944169 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a9c031ca0 con 0x7f8aac0a4590 2024-09-15T01:37:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8aab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8ab2e80640 4 set_mon_vals no callback set 2024-09-15T01:37:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8aab7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.709+0000 7f8aab7fe640 10 monclient: dump: 2024-09-15T01:37:19.724 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:19.725 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 10 monclient: shutdown 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 1 -- 172.21.15.204:0/2418944169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aac0a4590 msgr2=0x7f8aac09e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 1 --2- 172.21.15.204:0/2418944169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aac0a4590 0x7f8aac09e750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0a4db0 tx=0x7f8a9c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 1 -- 172.21.15.204:0/2418944169 shutdown_connections 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 1 --2- 172.21.15.204:0/2418944169 >> v2:172.21.15.178:3300/0 conn(0x7f8aac06edd0 0x7f8aac09ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 1 --2- 172.21.15.204:0/2418944169 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8aac09f1d0 0x7f8aac1307a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 1 --2- 172.21.15.204:0/2418944169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aac0a4590 0x7f8aac09e750 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 1 -- 172.21.15.204:0/2418944169 >> 172.21.15.204:0/2418944169 conn(0x7f8aac067700 msgr2=0x7f8aac0a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 1 -- 172.21.15.204:0/2418944169 shutdown_connections 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 1 -- 172.21.15.204:0/2418944169 wait complete. 2024-09-15T01:37:19.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 10 monclient: build_initial_monmap 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 10 monclient: monmap: 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:19.727 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:19.728 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.710+0000 7f8ab2e80640 1 Processor -- start 2024-09-15T01:37:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 1 -- start start 2024-09-15T01:37:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 10 monclient: init 2024-09-15T01:37:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 10 monclient: _add_conns ranks=[3,2,5,1,0,7,6,8,4] 2024-09-15T01:37:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aac06edd0 0x7f8aac133660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:19.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 10 monclient(hunting): picked mon.noname-g con 0x7f8aac06edd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aac09f1d0 0x7f8aac11d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 10 monclient(hunting): picked mon.noname-b con 0x7f8aac09f1d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8aac0a4590 0x7f8aac11d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 10 monclient(hunting): picked mon.noname-h con 0x7f8aac0a4590 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac06cc80 con 0x7f8aac09f1d0 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac120490 con 0x7f8aac06edd0 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac120d40 con 0x7f8aac0a4590 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab2e80640 10 monclient(hunting): authenticate will time out at 1055.353597s 2024-09-15T01:37:19.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8aabfff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aac09f1d0 msgr2=0x7f8aac11d280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8aabfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aac09f1d0 0x7f8aac11d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab0c1c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aac06edd0 0x7f8aac133660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab0c1c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aac06edd0 0x7f8aac133660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48462/0 (socket says 172.21.15.204:48462) 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab0c1c640 1 -- 172.21.15.204:0/1099438142 learned_addr learned my addr 172.21.15.204:0/1099438142 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab0c1c640 10 monclient(hunting): get_auth_request con 0x7f8aac06edd0 auth_method 0 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab141d640 1 --2- 172.21.15.204:0/1099438142 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8aac0a4590 0x7f8aac11d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab0c1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab0c1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab0c1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab141d640 10 monclient(hunting): get_auth_request con 0x7f8aac0a4590 auth_method 0 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab141d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab141d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab141d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:19.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab0c1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab0c1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab0c1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab141d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab141d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab141d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab141d640 10 monclient(hunting): handle_auth_done global_id 6305 payload 824 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.711+0000 7f8ab141d640 1 -- 172.21.15.204:0/1099438142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aac06edd0 msgr2=0x7f8aac133660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab141d640 1 --2- 172.21.15.204:0/1099438142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aac06edd0 0x7f8aac133660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab141d640 1 -- 172.21.15.204:0/1099438142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aac09f1d0 msgr2=0x7f8aac11d280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab141d640 1 --2- 172.21.15.204:0/1099438142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aac09f1d0 0x7f8aac11d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab141d640 10 monclient: _finish_hunting 0 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab141d640 1 monclient: found mon.noname-h 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab141d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab141d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:19.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab141d640 1 -- 172.21.15.204:0/1099438142 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c009450 con 0x7f8aac0a4590 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab0c1c640 1 --2- 172.21.15.204:0/1099438142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aac06edd0 0x7f8aac133660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab141d640 1 --2- 172.21.15.204:0/1099438142 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8aac0a4590 0x7f8aac11d7c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400bad0 tx=0x7f8aa4009780 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aa8ff9640 1 -- 172.21.15.204:0/1099438142 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8aa400ce30 con 0x7f8aac0a4590 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aa8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aa8ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aa8ff9640 10 monclient: dump: 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:19.732 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aa8ff9640 10 monclient: _finish_auth 0 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aa8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aa8ff9640 1 -- 172.21.15.204:0/1099438142 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa4014410 con 0x7f8aac0a4590 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab2e80640 5 monclient: authenticate success, global_id 6305 2024-09-15T01:37:19.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab2e80640 10 monclient: _renew_subs 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab2e80640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab2e80640 1 -- 172.21.15.204:0/1099438142 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac11f820 con 0x7f8aac0a4590 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aa8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aa8ff9640 1 -- 172.21.15.204:0/1099438142 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8aa4014a00 con 0x7f8aac0a4590 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aa8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aa8ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aa8ff9640 10 monclient: dump: 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:19.734 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:19.735 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8aaaffd640 4 set_mon_vals no callback set 2024-09-15T01:37:19.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab2e80640 10 monclient: _renew_subs 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab2e80640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.712+0000 7f8ab2e80640 1 -- 172.21.15.204:0/1099438142 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac120a60 con 0x7f8aac0a4590 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.713+0000 7f8aa8ff9640 1 -- 172.21.15.204:0/1099438142 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8aa4014b80 con 0x7f8aac0a4590 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.713+0000 7f8ab2e80640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.713+0000 7f8aa8ff9640 1 --2- 172.21.15.204:0/1099438142 >> v2:172.21.15.204:6812/28210 conn(0x7f8a980397b0 0x7f8a9803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.713+0000 7f8ab2e80640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.713+0000 7f8ab2e80640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.713+0000 7f8ab2e80640 1 -- 172.21.15.204:0/1099438142 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac070b80 con 0x7f8aac0a4590 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.713+0000 7f8aa8ff9640 1 -- 172.21.15.204:0/1099438142 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 11783+0+0 (secure 0 0 0) 0x7f8aa404cf30 con 0x7f8aac0a4590 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.713+0000 7f8ab0c1c640 1 --2- 172.21.15.204:0/1099438142 >> v2:172.21.15.204:6812/28210 conn(0x7f8a980397b0 0x7f8a9803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.713+0000 7f8ab0c1c640 10 monclient: get_auth_request con 0x7f8a980397b0 auth_method 0 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.713+0000 7f8ab0c1c640 1 --2- 172.21.15.204:0/1099438142 >> v2:172.21.15.204:6812/28210 conn(0x7f8a980397b0 0x7f8a9803bc60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0097e0 tx=0x7f8a9c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.717+0000 7f8aa8ff9640 1 -- 172.21.15.204:0/1099438142 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8aa40133c0 con 0x7f8aac0a4590 2024-09-15T01:37:19.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.717+0000 7f8aa8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:19.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.717+0000 7f8aa8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:19.877 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.876+0000 7f8ab2e80640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:19.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.876+0000 7f8ab2e80640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:19.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.876+0000 7f8ab2e80640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:19.878 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:19.876+0000 7f8ab2e80640 1 -- 172.21.15.204:0/1099438142 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8aac0a04c0 con 0x7f8aac0a4590 2024-09-15T01:37:20.529 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.527+0000 7f8aa8ff9640 1 -- 172.21.15.204:0/1099438142 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}]=0 added bucket h22 type host to location {rack=r1} v207) v1 ==== 168+0+0 (secure 0 0 0) 0x7f8aa404c070 con 0x7f8aac0a4590 2024-09-15T01:37:20.529 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.527+0000 7f8aa8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:20.529 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.527+0000 7f8aa8ff9640 10 monclient: _finish_command 2 = system:0 added bucket h22 type host to location {rack=r1} 2024-09-15T01:37:20.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.623+0000 7f8ab2e80640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:20.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.623+0000 7f8ab2e80640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:20.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.623+0000 7f8ab2e80640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:20.625 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.623+0000 7f8ab2e80640 1 -- 172.21.15.204:0/1099438142 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8aac120f80 con 0x7f8aac0a4590 2024-09-15T01:37:20.626 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.624+0000 7f8aa8ff9640 1 -- 172.21.15.204:0/1099438142 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h22' already exists v207) v1 ==== 147+0+0 (secure 0 0 0) 0x7f8aa4012320 con 0x7f8aac0a4590 2024-09-15T01:37:20.626 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.624+0000 7f8aa8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:20.626 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.624+0000 7f8aa8ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h22' already exists 2024-09-15T01:37:20.626 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h22' already exists 2024-09-15T01:37:20.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 1 -- 172.21.15.204:0/1099438142 >> v2:172.21.15.204:6812/28210 conn(0x7f8a980397b0 msgr2=0x7f8a9803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:20.628 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 1 --2- 172.21.15.204:0/1099438142 >> v2:172.21.15.204:6812/28210 conn(0x7f8a980397b0 0x7f8a9803bc60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0097e0 tx=0x7f8a9c032000 comp rx=0 tx=0).stop 2024-09-15T01:37:20.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 10 monclient: shutdown 2024-09-15T01:37:20.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:20.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 1 -- 172.21.15.204:0/1099438142 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8aac0a4590 msgr2=0x7f8aac11d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:20.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 1 --2- 172.21.15.204:0/1099438142 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8aac0a4590 0x7f8aac11d7c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400bad0 tx=0x7f8aa4009780 comp rx=0 tx=0).stop 2024-09-15T01:37:20.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 1 -- 172.21.15.204:0/1099438142 shutdown_connections 2024-09-15T01:37:20.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 1 --2- 172.21.15.204:0/1099438142 >> v2:172.21.15.204:6812/28210 conn(0x7f8a980397b0 0x7f8a9803bc60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:20.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 1 --2- 172.21.15.204:0/1099438142 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8aac0a4590 0x7f8aac11d7c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:20.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 1 --2- 172.21.15.204:0/1099438142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aac09f1d0 0x7f8aac11d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:20.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 1 --2- 172.21.15.204:0/1099438142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aac06edd0 0x7f8aac133660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:20.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 1 -- 172.21.15.204:0/1099438142 >> 172.21.15.204:0/1099438142 conn(0x7f8aac067700 msgr2=0x7f8aac0a24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:20.629 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 1 -- 172.21.15.204:0/1099438142 shutdown_connections 2024-09-15T01:37:20.630 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.626+0000 7f8ab2e80640 1 -- 172.21.15.204:0/1099438142 wait complete. 2024-09-15T01:37:20.637 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:20.637 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h23 host rack=r1 2024-09-15T01:37:20.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.710+0000 7f2f39d1d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:20.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.710+0000 7f2f39d1d640 10 monclient: build_initial_monmap 2024-09-15T01:37:20.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.710+0000 7f2f39d1d640 10 monclient: monmap: 2024-09-15T01:37:20.728 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:20.728 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:20.728 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:20.711229+0000 2024-09-15T01:37:20.728 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:20.711229+0000 2024-09-15T01:37:20.729 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:20.729 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:20.729 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:20.729 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:20.729 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:20.729 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:20.729 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:20.730 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:20.730 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:20.730 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:20.730 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:20.730 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:20.730 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:20.730 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:20.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.711+0000 7f2f39d1d640 1 Processor -- start 2024-09-15T01:37:20.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.711+0000 7f2f39d1d640 1 -- start start 2024-09-15T01:37:20.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.711+0000 7f2f39d1d640 10 monclient: init 2024-09-15T01:37:20.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.711+0000 7f2f39d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:20.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.711+0000 7f2f39d1d640 10 monclient: _add_conns ranks=[5,3,1,0,9,2,6,8,7,10,4] 2024-09-15T01:37:20.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.711+0000 7f2f39d1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2f340a2cd0 0x7f2f34132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:20.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.711+0000 7f2f39d1d640 10 monclient(hunting): picked mon.noname-h con 0x7f2f340a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:20.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f39d1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f3409fa20 0x7f2f3409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:20.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f39d1d640 10 monclient(hunting): picked mon.noname-g con 0x7f2f3409fa20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:20.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f39d1d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2f340a08e0 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:20.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f39d1d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f340648a0 con 0x7f2f340a08e0 2024-09-15T01:37:20.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f39d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:20.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f39d1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340a30a0 con 0x7f2f3409fa20 2024-09-15T01:37:20.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f39d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:20.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f39d1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340a0330 con 0x7f2f340a2cd0 2024-09-15T01:37:20.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f39d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:20.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f39d1d640 10 monclient(hunting): authenticate will time out at 1056.354053s 2024-09-15T01:37:20.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f337fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2f340a2cd0 0x7f2f34132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:20.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f3409fa20 0x7f2f3409fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:20.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f33fff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f2f340a08e0 0x7f2f340a0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:57032/0 (socket says 172.21.15.204:57032) 2024-09-15T01:37:20.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f33fff640 1 -- 172.21.15.204:0/2400033519 learned_addr learned my addr 172.21.15.204:0/2400033519 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:37:20.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f337fe640 10 monclient(hunting): get_auth_request con 0x7f2f340a2cd0 auth_method 0 2024-09-15T01:37:20.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:20.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:20.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:20.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 10 monclient(hunting): get_auth_request con 0x7f2f3409fa20 auth_method 0 2024-09-15T01:37:20.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:20.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:20.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:20.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:20.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:20.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:20.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:20.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:20.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:20.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 10 monclient(hunting): handle_auth_done global_id 15474 payload 274 2024-09-15T01:37:20.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f327fc640 1 -- 172.21.15.204:0/2400033519 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822824574 0 0) 0x7f2f340648a0 con 0x7f2f340a08e0 2024-09-15T01:37:20.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 1 -- 172.21.15.204:0/2400033519 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2f340a2cd0 msgr2=0x7f2f34132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:20.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 1 --2- 172.21.15.204:0/2400033519 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2f340a2cd0 0x7f2f34132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:20.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 1 -- 172.21.15.204:0/2400033519 >> v1:172.21.15.204:6789/0 conn(0x7f2f340a08e0 legacy=0x7f2f340a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:20.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:37:20.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 1 monclient: found mon.noname-g 2024-09-15T01:37:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f32ffd640 1 -- 172.21.15.204:0/2400033519 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f340a1db0 con 0x7f2f3409fa20 2024-09-15T01:37:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f327fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:37:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.712+0000 7f2f337fe640 1 --2- 172.21.15.204:0/2400033519 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2f340a2cd0 0x7f2f34132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f32ffd640 1 --2- 172.21.15.204:0/2400033519 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f3409fa20 0x7f2f3409fdf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009880 tx=0x7f2f280090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=190f7aa22a3d866 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f327fc640 1 -- 172.21.15.204:0/2400033519 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f2802ac50 con 0x7f2f3409fa20 2024-09-15T01:37:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:20.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f327fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f327fc640 10 monclient: dump: 2024-09-15T01:37:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:20.737 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:20.738 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f327fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 5 monclient: authenticate success, global_id 15474 2024-09-15T01:37:20.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f327fc640 1 -- 172.21.15.204:0/2400033519 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2802add0 con 0x7f2f3409fa20 2024-09-15T01:37:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f327fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 4 set_mon_vals no callback set 2024-09-15T01:37:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f327fc640 1 -- 172.21.15.204:0/2400033519 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f28031ca0 con 0x7f2f3409fa20 2024-09-15T01:37:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f327fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f327fc640 10 monclient: dump: 2024-09-15T01:37:20.739 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:20.740 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:20.741 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:20.742 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 10 monclient: shutdown 2024-09-15T01:37:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 1 -- 172.21.15.204:0/2400033519 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f3409fa20 msgr2=0x7f2f3409fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 1 --2- 172.21.15.204:0/2400033519 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f3409fa20 0x7f2f3409fdf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009880 tx=0x7f2f280090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 1 -- 172.21.15.204:0/2400033519 shutdown_connections 2024-09-15T01:37:20.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 1 --2- 172.21.15.204:0/2400033519 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f3409fa20 0x7f2f3409fdf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 1 --2- 172.21.15.204:0/2400033519 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2f340a2cd0 0x7f2f34132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 1 -- 172.21.15.204:0/2400033519 >> 172.21.15.204:0/2400033519 conn(0x7f2f3406e860 msgr2=0x7f2f3406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 1 -- 172.21.15.204:0/2400033519 shutdown_connections 2024-09-15T01:37:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.713+0000 7f2f39d1d640 1 -- 172.21.15.204:0/2400033519 wait complete. 2024-09-15T01:37:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.714+0000 7f2f39d1d640 10 monclient: build_initial_monmap 2024-09-15T01:37:20.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.714+0000 7f2f39d1d640 10 monclient: monmap: 2024-09-15T01:37:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:20.744 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:20.745 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:20.746 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:20.746 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.714+0000 7f2f39d1d640 1 Processor -- start 2024-09-15T01:37:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.714+0000 7f2f39d1d640 1 -- start start 2024-09-15T01:37:20.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.714+0000 7f2f39d1d640 10 monclient: init 2024-09-15T01:37:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 10 monclient: _add_conns ranks=[7,8,4,5,1,3,2,6,0] 2024-09-15T01:37:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f3409fa20 0x7f2f34141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:20.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 10 monclient(hunting): picked mon.noname-i con 0x7f2f3409fa20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f340a08e0 0x7f2f341423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 10 monclient(hunting): picked mon.noname-e con 0x7f2f340a08e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2f340a2cd0 0x7f2f3411d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 10 monclient(hunting): picked mon.noname-c con 0x7f2f340a2cd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3409d3d0 con 0x7f2f340a2cd0 2024-09-15T01:37:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:20.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3409cb60 con 0x7f2f3409fa20 2024-09-15T01:37:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3411f230 con 0x7f2f340a08e0 2024-09-15T01:37:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f39d1d640 10 monclient(hunting): authenticate will time out at 1056.357373s 2024-09-15T01:37:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f3409fa20 0x7f2f34141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:20.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f3409fa20 0x7f2f34141e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:42942/0 (socket says 172.21.15.204:42942) 2024-09-15T01:37:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 1 -- 172.21.15.204:0/3917868944 learned_addr learned my addr 172.21.15.204:0/3917868944 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 10 monclient(hunting): get_auth_request con 0x7f2f3409fa20 auth_method 0 2024-09-15T01:37:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f33fff640 1 -- 172.21.15.204:0/3917868944 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2f340a2cd0 msgr2=0x7f2f3411d1e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:20.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f32ffd640 1 -- 172.21.15.204:0/3917868944 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f340a08e0 msgr2=0x7f2f341423b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f33fff640 1 --2- 172.21.15.204:0/3917868944 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2f340a2cd0 0x7f2f3411d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f32ffd640 1 --2- 172.21.15.204:0/3917868944 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f340a08e0 0x7f2f341423b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:20.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 10 monclient(hunting): handle_auth_done global_id 24550 payload 824 2024-09-15T01:37:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 1 -- 172.21.15.204:0/3917868944 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f340a08e0 msgr2=0x7f2f341423b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 1 --2- 172.21.15.204:0/3917868944 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f340a08e0 0x7f2f341423b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 1 -- 172.21.15.204:0/3917868944 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2f340a2cd0 msgr2=0x7f2f3411d1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:20.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 1 --2- 172.21.15.204:0/3917868944 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2f340a2cd0 0x7f2f3411d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 1 monclient: found mon.noname-i 2024-09-15T01:37:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 1 -- 172.21.15.204:0/3917868944 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24009a20 con 0x7f2f3409fa20 2024-09-15T01:37:20.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.715+0000 7f2f337fe640 1 --2- 172.21.15.204:0/3917868944 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f3409fa20 0x7f2f34141e70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400c0f0 tx=0x7f2f2400b040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 1 -- 172.21.15.204:0/3917868944 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f2400ce70 con 0x7f2f3409fa20 2024-09-15T01:37:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 10 monclient: dump: 2024-09-15T01:37:20.755 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:20.756 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:20.757 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:20.757 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:20.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:20.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:20.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 1 -- 172.21.15.204:0/3917868944 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2400f040 con 0x7f2f3409fa20 2024-09-15T01:37:20.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f39d1d640 5 monclient: authenticate success, global_id 24550 2024-09-15T01:37:20.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f39d1d640 10 monclient: _renew_subs 2024-09-15T01:37:20.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f39d1d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:20.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f39d1d640 1 -- 172.21.15.204:0/3917868944 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f28009390 con 0x7f2f3409fa20 2024-09-15T01:37:20.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:20.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 1 -- 172.21.15.204:0/3917868944 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f24014d80 con 0x7f2f3409fa20 2024-09-15T01:37:20.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:20.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:20.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 10 monclient: dump: 2024-09-15T01:37:20.761 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:20.761 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:20.761 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:20.761 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:20.761 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f39d1d640 10 monclient: _renew_subs 2024-09-15T01:37:20.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f39d1d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:20.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f39d1d640 1 -- 172.21.15.204:0/3917868944 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f34121990 con 0x7f2f3409fa20 2024-09-15T01:37:20.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f31ffb640 4 set_mon_vals no callback set 2024-09-15T01:37:20.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.716+0000 7f2f1b7fe640 1 -- 172.21.15.204:0/3917868944 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2f24003020 con 0x7f2f3409fa20 2024-09-15T01:37:20.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.717+0000 7f2f39d1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:20.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.717+0000 7f2f39d1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:20.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.717+0000 7f2f39d1d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:20.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.717+0000 7f2f1b7fe640 1 --2- 172.21.15.204:0/3917868944 >> v2:172.21.15.204:6812/28210 conn(0x7f2f200397b0 0x7f2f2003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:20.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.717+0000 7f2f39d1d640 1 -- 172.21.15.204:0/3917868944 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f34070b80 con 0x7f2f3409fa20 2024-09-15T01:37:20.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.717+0000 7f2f1b7fe640 1 -- 172.21.15.204:0/3917868944 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 11996+0+0 (secure 0 0 0) 0x7f2f2404db70 con 0x7f2f3409fa20 2024-09-15T01:37:20.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.717+0000 7f2f32ffd640 1 --2- 172.21.15.204:0/3917868944 >> v2:172.21.15.204:6812/28210 conn(0x7f2f200397b0 0x7f2f2003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:20.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.717+0000 7f2f32ffd640 10 monclient: get_auth_request con 0x7f2f200397b0 auth_method 0 2024-09-15T01:37:20.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.717+0000 7f2f32ffd640 1 --2- 172.21.15.204:0/3917868944 >> v2:172.21.15.204:6812/28210 conn(0x7f2f200397b0 0x7f2f2003bc60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2f28034000 tx=0x7f2f28032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:20.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.721+0000 7f2f1b7fe640 1 -- 172.21.15.204:0/3917868944 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2f2401a680 con 0x7f2f3409fa20 2024-09-15T01:37:20.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.721+0000 7f2f1b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:20.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.721+0000 7f2f1b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:20.881 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.880+0000 7f2f39d1d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:20.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.880+0000 7f2f39d1d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:20.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.880+0000 7f2f39d1d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:20.882 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:20.880+0000 7f2f39d1d640 1 -- 172.21.15.204:0/3917868944 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f2f3411f8f0 con 0x7f2f3409fa20 2024-09-15T01:37:21.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.535+0000 7f2f1b7fe640 1 -- 172.21.15.204:0/3917868944 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}]=0 added bucket h23 type host to location {rack=r1} v208) v1 ==== 168+0+0 (secure 0 0 0) 0x7f2f24018020 con 0x7f2f3409fa20 2024-09-15T01:37:21.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.535+0000 7f2f1b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:21.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.535+0000 7f2f1b7fe640 10 monclient: _finish_command 2 = system:0 added bucket h23 type host to location {rack=r1} 2024-09-15T01:37:21.632 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.631+0000 7f2f39d1d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:21.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.631+0000 7f2f39d1d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:21.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.631+0000 7f2f39d1d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:21.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.631+0000 7f2f39d1d640 1 -- 172.21.15.204:0/3917868944 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f2f340008d0 con 0x7f2f3409fa20 2024-09-15T01:37:21.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.631+0000 7f2f1b7fe640 1 -- 172.21.15.204:0/3917868944 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h23' already exists v208) v1 ==== 147+0+0 (secure 0 0 0) 0x7f2f2401a2c0 con 0x7f2f3409fa20 2024-09-15T01:37:21.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.631+0000 7f2f1b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:21.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.631+0000 7f2f1b7fe640 10 monclient: _finish_command 3 = system:0 bucket 'h23' already exists 2024-09-15T01:37:21.633 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h23' already exists 2024-09-15T01:37:21.635 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.633+0000 7f2f39d1d640 1 -- 172.21.15.204:0/3917868944 >> v2:172.21.15.204:6812/28210 conn(0x7f2f200397b0 msgr2=0x7f2f2003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:21.635 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.633+0000 7f2f39d1d640 1 --2- 172.21.15.204:0/3917868944 >> v2:172.21.15.204:6812/28210 conn(0x7f2f200397b0 0x7f2f2003bc60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2f28034000 tx=0x7f2f28032000 comp rx=0 tx=0).stop 2024-09-15T01:37:21.635 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.633+0000 7f2f39d1d640 10 monclient: shutdown 2024-09-15T01:37:21.635 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.633+0000 7f2f39d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:21.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.633+0000 7f2f39d1d640 1 -- 172.21.15.204:0/3917868944 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f3409fa20 msgr2=0x7f2f34141e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:21.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.633+0000 7f2f39d1d640 1 --2- 172.21.15.204:0/3917868944 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f3409fa20 0x7f2f34141e70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400c0f0 tx=0x7f2f2400b040 comp rx=0 tx=0).stop 2024-09-15T01:37:21.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.633+0000 7f2f39d1d640 1 -- 172.21.15.204:0/3917868944 shutdown_connections 2024-09-15T01:37:21.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.633+0000 7f2f39d1d640 1 --2- 172.21.15.204:0/3917868944 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2f340a2cd0 0x7f2f3411d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:21.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.633+0000 7f2f39d1d640 1 --2- 172.21.15.204:0/3917868944 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f340a08e0 0x7f2f341423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:21.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.633+0000 7f2f39d1d640 1 --2- 172.21.15.204:0/3917868944 >> v2:172.21.15.204:6812/28210 conn(0x7f2f200397b0 0x7f2f2003bc60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:21.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.633+0000 7f2f39d1d640 1 --2- 172.21.15.204:0/3917868944 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f3409fa20 0x7f2f34141e70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:21.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.633+0000 7f2f39d1d640 1 -- 172.21.15.204:0/3917868944 >> 172.21.15.204:0/3917868944 conn(0x7f2f3406e860 msgr2=0x7f2f340a1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:21.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.634+0000 7f2f39d1d640 1 -- 172.21.15.204:0/3917868944 shutdown_connections 2024-09-15T01:37:21.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.634+0000 7f2f39d1d640 1 -- 172.21.15.204:0/3917868944 wait complete. 2024-09-15T01:37:21.644 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:21.645 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h24 host rack=r1 2024-09-15T01:37:21.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.719+0000 7f6e4ad9d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:21.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.719+0000 7f6e4ad9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:21.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.719+0000 7f6e4ad9d640 10 monclient: monmap: 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:21.719521+0000 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:21.719521+0000 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:21.721 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:21.722 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:21.722 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:21.722 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:21.722 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.719+0000 7f6e4ad9d640 1 Processor -- start 2024-09-15T01:37:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.719+0000 7f6e4ad9d640 1 -- start start 2024-09-15T01:37:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.719+0000 7f6e4ad9d640 10 monclient: init 2024-09-15T01:37:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 10 monclient: _add_conns ranks=[5,8,7,10,4,9,0,2,1,3,6] 2024-09-15T01:37:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e440a2cd0 0x7f6e44132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 10 monclient(hunting): picked mon.noname-h con 0x7f6e440a2cd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e4409fa20 0x7f6e4409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 10 monclient(hunting): picked mon.noname-e con 0x7f6e4409fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6e440a08e0 0x7f6e440a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 10 monclient(hunting): picked mon.noname-i con 0x7f6e440a08e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440a30a0 con 0x7f6e440a2cd0 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440a0330 con 0x7f6e440a08e0 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440a1db0 con 0x7f6e4409fa20 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4ad9d640 10 monclient(hunting): authenticate will time out at 1057.362268s 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e440a2cd0 0x7f6e44132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4933a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6e440a08e0 0x7f6e440a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:21.728 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e43fff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e4409fa20 msgr2=0x7f6e4409fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e440a2cd0 0x7f6e44132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:38494/0 (socket says 172.21.15.204:38494) 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 1 -- 172.21.15.204:0/4116113362 learned_addr learned my addr 172.21.15.204:0/4116113362 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4933a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6e440a08e0 0x7f6e440a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:42950/0 (socket says 172.21.15.204:42950) 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 10 monclient(hunting): get_auth_request con 0x7f6e440a2cd0 auth_method 0 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e43fff640 1 --2- 172.21.15.204:0/4116113362 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e4409fa20 0x7f6e4409fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4933a640 10 monclient(hunting): get_auth_request con 0x7f6e440a08e0 auth_method 0 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4933a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4933a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4933a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:21.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4933a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4933a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e4933a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 10 monclient(hunting): handle_auth_done global_id 6323 payload 274 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 1 -- 172.21.15.204:0/4116113362 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e4409fa20 msgr2=0x7f6e4409fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 1 --2- 172.21.15.204:0/4116113362 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e4409fa20 0x7f6e4409fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 1 -- 172.21.15.204:0/4116113362 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6e440a08e0 msgr2=0x7f6e440a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 1 --2- 172.21.15.204:0/4116113362 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6e440a08e0 0x7f6e440a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 10 monclient: _finish_hunting 0 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 1 monclient: found mon.noname-h 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:21.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.720+0000 7f6e48b39640 1 -- 172.21.15.204:0/4116113362 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440648a0 con 0x7f6e440a2cd0 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e48b39640 1 --2- 172.21.15.204:0/4116113362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e440a2cd0 0x7f6e44132c00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009940 tx=0x7f6e3c0091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=7625878fdb39a10f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e437fe640 1 -- 172.21.15.204:0/4116113362 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6e3c02ac80 con 0x7f6e440a2cd0 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e437fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e437fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e437fe640 10 monclient: dump: 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:21.731 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e437fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e437fe640 1 -- 172.21.15.204:0/4116113362 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c02ae00 con 0x7f6e440a2cd0 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e4ad9d640 5 monclient: authenticate success, global_id 6323 2024-09-15T01:37:21.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e4ad9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:21.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e437fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:21.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e4ad9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:21.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e4ad9d640 4 set_mon_vals no callback set 2024-09-15T01:37:21.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e437fe640 1 -- 172.21.15.204:0/4116113362 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6e3c031ca0 con 0x7f6e440a2cd0 2024-09-15T01:37:21.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e437fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:21.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e437fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:21.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.721+0000 7f6e437fe640 10 monclient: dump: 2024-09-15T01:37:21.733 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:21.733 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:21.733 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:21.733 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:21.733 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 10 monclient: shutdown 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/4116113362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e440a2cd0 msgr2=0x7f6e44132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 1 --2- 172.21.15.204:0/4116113362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e440a2cd0 0x7f6e44132c00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009940 tx=0x7f6e3c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/4116113362 shutdown_connections 2024-09-15T01:37:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 1 --2- 172.21.15.204:0/4116113362 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6e440a08e0 0x7f6e440a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 1 --2- 172.21.15.204:0/4116113362 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e4409fa20 0x7f6e4409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 1 --2- 172.21.15.204:0/4116113362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e440a2cd0 0x7f6e44132c00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/4116113362 >> 172.21.15.204:0/4116113362 conn(0x7f6e4406e860 msgr2=0x7f6e4406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/4116113362 shutdown_connections 2024-09-15T01:37:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/4116113362 wait complete. 2024-09-15T01:37:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 10 monclient: monmap: 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.722+0000 7f6e4ad9d640 1 Processor -- start 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 1 -- start start 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 10 monclient: init 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 10 monclient: _add_conns ranks=[4,8,0,5,2,7,1,3,6] 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6e4409fa20 0x7f6e44130fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 10 monclient(hunting): picked mon.noname-c con 0x7f6e4409fa20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e440a08e0 0x7f6e4409b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 10 monclient(hunting): picked mon.noname-e con 0x7f6e440a08e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e440a2cd0 0x7f6e4409b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 10 monclient(hunting): picked mon.noname-a con 0x7f6e440a2cd0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4411e3b0 con 0x7f6e440a2cd0 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4409cb60 con 0x7f6e4409fa20 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4411e0d0 con 0x7f6e440a08e0 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4ad9d640 10 monclient(hunting): authenticate will time out at 1057.365597s 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e48b39640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6e4409fa20 msgr2=0x7f6e44130fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4933a640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e440a2cd0 msgr2=0x7f6e4409b6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e43fff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e440a08e0 msgr2=0x7f6e4409b1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e4933a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e440a2cd0 0x7f6e4409b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e48b39640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6e4409fa20 0x7f6e44130fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.723+0000 7f6e43fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e440a08e0 0x7f6e4409b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:21.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.924+0000 7f6e48b39640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6e4409fa20 msgr2=0x7f6e44130fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:21.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.924+0000 7f6e4933a640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e440a2cd0 msgr2=0x7f6e4409b6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:21.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.924+0000 7f6e43fff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e440a08e0 msgr2=0x7f6e4409b1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:21.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.924+0000 7f6e48b39640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6e4409fa20 0x7f6e44130fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:37:21.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.924+0000 7f6e43fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e440a08e0 0x7f6e4409b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:37:21.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:21.924+0000 7f6e4933a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e440a2cd0 0x7f6e4409b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:37:22.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:22.325+0000 7f6e48b39640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6e4409fa20 msgr2=0x7f6e44130fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:22.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:22.325+0000 7f6e4933a640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e440a2cd0 msgr2=0x7f6e4409b6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:22.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:22.325+0000 7f6e43fff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e440a08e0 msgr2=0x7f6e4409b1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:22.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:22.325+0000 7f6e48b39640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6e4409fa20 0x7f6e44130fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:37:22.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:22.325+0000 7f6e4933a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e440a2cd0 0x7f6e4409b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:37:22.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:22.325+0000 7f6e43fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e440a08e0 0x7f6e4409b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:37:23.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:23.126+0000 7f6e43fff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e440a08e0 msgr2=0x7f6e4409b1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:23.126+0000 7f6e4933a640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e440a2cd0 msgr2=0x7f6e4409b6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:23.126+0000 7f6e48b39640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6e4409fa20 msgr2=0x7f6e44130fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:23.126+0000 7f6e4933a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e440a2cd0 0x7f6e4409b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:37:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:23.126+0000 7f6e43fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e440a08e0 0x7f6e4409b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:37:23.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:23.126+0000 7f6e48b39640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6e4409fa20 0x7f6e44130fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:37:24.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 10 monclient(hunting): tick 2024-09-15T01:37:24.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 monclient(hunting): continuing hunt 2024-09-15T01:37:24.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:37:24.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e440a08e0 msgr2=0x7f6e4409b1a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:24.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6e440a08e0 0x7f6e4409b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:24.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6e4409fa20 msgr2=0x7f6e44130fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:24.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6e4409fa20 0x7f6e44130fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:24.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e440a2cd0 msgr2=0x7f6e4409b6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:24.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e440a2cd0 0x7f6e4409b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:24.729 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 10 monclient: _add_conns ranks=[3,6,5,7,1,2] 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e28003780 0x7f6e28005b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 10 monclient(hunting): picked mon.noname-g con 0x7f6e28003780 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6e28006180 0x7f6e28008600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 10 monclient(hunting): picked mon.noname-d con 0x7f6e28006180 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e28008ca0 0x7f6e2800b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 10 monclient(hunting): picked mon.noname-h con 0x7f6e28008ca0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4411e3b0 con 0x7f6e28003780 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4409cb60 con 0x7f6e28008ca0 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4411e0d0 con 0x7f6e28006180 2024-09-15T01:37:24.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e3b7fe640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e48b39640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e28003780 0x7f6e28005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e28008ca0 0x7f6e2800b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e48b39640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e28003780 0x7f6e28005b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48492/0 (socket says 172.21.15.204:48492) 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e48b39640 1 -- 172.21.15.204:0/86863272 learned_addr learned my addr 172.21.15.204:0/86863272 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e48b39640 10 monclient(hunting): get_auth_request con 0x7f6e28003780 auth_method 0 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e48b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e48b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e48b39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 10 monclient(hunting): get_auth_request con 0x7f6e28008ca0 auth_method 0 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:24.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e48b39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e48b39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e48b39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e43fff640 1 --2- 172.21.15.204:0/86863272 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6e28006180 0x7f6e28008600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e43fff640 10 monclient(hunting): get_auth_request con 0x7f6e28006180 auth_method 0 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e43fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e43fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e43fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:24.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 10 monclient(hunting): handle_auth_done global_id 6332 payload 824 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 1 -- 172.21.15.204:0/86863272 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6e28006180 msgr2=0x7f6e28008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 1 --2- 172.21.15.204:0/86863272 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6e28006180 0x7f6e28008600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 1 -- 172.21.15.204:0/86863272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e28003780 msgr2=0x7f6e28005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 1 --2- 172.21.15.204:0/86863272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e28003780 0x7f6e28005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 10 monclient: _finish_hunting 0 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 1 monclient: found mon.noname-h 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 1 -- 172.21.15.204:0/86863272 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c009450 con 0x7f6e28008ca0 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e48b39640 1 --2- 172.21.15.204:0/86863272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6e28003780 0x7f6e28005b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 1 -- 172.21.15.204:0/86863272 reap_dead start 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e4933a640 1 --2- 172.21.15.204:0/86863272 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e28008ca0 0x7f6e2800b150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6e440a2ca0 tx=0x7f6e2c0090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:24.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.724+0000 7f6e40ff9640 1 -- 172.21.15.204:0/86863272 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6e2c009570 con 0x7f6e28008ca0 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e40ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e40ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e40ff9640 10 monclient: dump: 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:24.734 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e40ff9640 10 monclient: _finish_auth 0 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e4ad9d640 5 monclient: authenticate success, global_id 6332 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e40ff9640 1 -- 172.21.15.204:0/86863272 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c00cdc0 con 0x7f6e28008ca0 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e40ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e40ff9640 1 -- 172.21.15.204:0/86863272 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6e2c0138f0 con 0x7f6e28008ca0 2024-09-15T01:37:24.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e4ad9d640 10 monclient: _renew_subs 2024-09-15T01:37:24.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e4ad9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:24.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/86863272 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4411e690 con 0x7f6e28008ca0 2024-09-15T01:37:24.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e40ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:24.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e42ffd640 4 set_mon_vals no callback set 2024-09-15T01:37:24.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e40ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:24.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e40ff9640 10 monclient: dump: 2024-09-15T01:37:24.736 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:24.736 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:24.736 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:24.736 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:24.736 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:24.737 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:24.737 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:24.737 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:24.737 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:24.737 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:24.737 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:24.737 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:24.737 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:24.737 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:24.737 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:24.737 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:24.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e4ad9d640 10 monclient: _renew_subs 2024-09-15T01:37:24.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e4ad9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:24.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.725+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/86863272 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e44142920 con 0x7f6e28008ca0 2024-09-15T01:37:24.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.726+0000 7f6e40ff9640 1 -- 172.21.15.204:0/86863272 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6e2c013470 con 0x7f6e28008ca0 2024-09-15T01:37:24.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.726+0000 7f6e4ad9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:24.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.726+0000 7f6e4ad9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:24.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.726+0000 7f6e4ad9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:24.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.726+0000 7f6e40ff9640 1 --2- 172.21.15.204:0/86863272 >> v2:172.21.15.204:6812/28210 conn(0x7f6e24039720 0x7f6e2403bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:24.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.726+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/86863272 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e44065b80 con 0x7f6e28008ca0 2024-09-15T01:37:24.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.726+0000 7f6e40ff9640 1 -- 172.21.15.204:0/86863272 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 12209+0+0 (secure 0 0 0) 0x7f6e2c04cf10 con 0x7f6e28008ca0 2024-09-15T01:37:24.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.726+0000 7f6e48b39640 1 --2- 172.21.15.204:0/86863272 >> v2:172.21.15.204:6812/28210 conn(0x7f6e24039720 0x7f6e2403bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:24.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.726+0000 7f6e48b39640 10 monclient: get_auth_request con 0x7f6e24039720 auth_method 0 2024-09-15T01:37:24.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.727+0000 7f6e48b39640 1 --2- 172.21.15.204:0/86863272 >> v2:172.21.15.204:6812/28210 conn(0x7f6e24039720 0x7f6e2403bbd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c02a650 tx=0x7f6e3c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:24.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.731+0000 7f6e40ff9640 1 -- 172.21.15.204:0/86863272 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6e2c017070 con 0x7f6e28008ca0 2024-09-15T01:37:24.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.731+0000 7f6e40ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:24.739 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.731+0000 7f6e40ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:24.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.895+0000 7f6e4ad9d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:24.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.895+0000 7f6e4ad9d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:24.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.895+0000 7f6e4ad9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:24.897 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:24.895+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/86863272 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f6e44142b10 con 0x7f6e28008ca0 2024-09-15T01:37:25.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.554+0000 7f6e40ff9640 1 -- 172.21.15.204:0/86863272 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}]=0 added bucket h24 type host to location {rack=r1} v209) v1 ==== 168+0+0 (secure 0 0 0) 0x7f6e44142b10 con 0x7f6e28008ca0 2024-09-15T01:37:25.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.554+0000 7f6e40ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:25.556 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.554+0000 7f6e40ff9640 10 monclient: _finish_command 2 = system:0 added bucket h24 type host to location {rack=r1} 2024-09-15T01:37:25.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.651+0000 7f6e4ad9d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:25.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.651+0000 7f6e4ad9d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:25.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.651+0000 7f6e4ad9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:25.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.651+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/86863272 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f6e44142b10 con 0x7f6e28008ca0 2024-09-15T01:37:25.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.652+0000 7f6e40ff9640 1 -- 172.21.15.204:0/86863272 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h24' already exists v209) v1 ==== 147+0+0 (secure 0 0 0) 0x7f6e44142b10 con 0x7f6e28008ca0 2024-09-15T01:37:25.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.652+0000 7f6e40ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:25.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.652+0000 7f6e40ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h24' already exists 2024-09-15T01:37:25.653 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h24' already exists 2024-09-15T01:37:25.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.654+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/86863272 >> v2:172.21.15.204:6812/28210 conn(0x7f6e24039720 msgr2=0x7f6e2403bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:25.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.654+0000 7f6e4ad9d640 1 --2- 172.21.15.204:0/86863272 >> v2:172.21.15.204:6812/28210 conn(0x7f6e24039720 0x7f6e2403bbd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c02a650 tx=0x7f6e3c032000 comp rx=0 tx=0).stop 2024-09-15T01:37:25.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.654+0000 7f6e4ad9d640 10 monclient: shutdown 2024-09-15T01:37:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.654+0000 7f6e4ad9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.654+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/86863272 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e28008ca0 msgr2=0x7f6e2800b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.654+0000 7f6e4ad9d640 1 --2- 172.21.15.204:0/86863272 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e28008ca0 0x7f6e2800b150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6e440a2ca0 tx=0x7f6e2c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.654+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/86863272 shutdown_connections 2024-09-15T01:37:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.654+0000 7f6e4ad9d640 1 --2- 172.21.15.204:0/86863272 >> v2:172.21.15.204:6812/28210 conn(0x7f6e24039720 0x7f6e2403bbd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.654+0000 7f6e4ad9d640 1 --2- 172.21.15.204:0/86863272 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6e28008ca0 0x7f6e2800b150 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.654+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/86863272 >> 172.21.15.204:0/86863272 conn(0x7f6e4406e860 msgr2=0x7f6e440a5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.654+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/86863272 shutdown_connections 2024-09-15T01:37:25.656 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.654+0000 7f6e4ad9d640 1 -- 172.21.15.204:0/86863272 wait complete. 2024-09-15T01:37:25.665 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:25.665 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h25 host rack=r1 2024-09-15T01:37:25.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.739+0000 7f71196d4640 10 monclient: get_monmap_and_config 2024-09-15T01:37:25.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.739+0000 7f71196d4640 10 monclient: build_initial_monmap 2024-09-15T01:37:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.739+0000 7f71196d4640 10 monclient: monmap: 2024-09-15T01:37:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:25.739677+0000 2024-09-15T01:37:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:25.739677+0000 2024-09-15T01:37:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:25.742 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:25.742 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:25.742 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:25.742 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:25.742 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:25.742 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:25.742 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:25.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.739+0000 7f71196d4640 1 Processor -- start 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.739+0000 7f71196d4640 1 -- start start 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.739+0000 7f71196d4640 10 monclient: init 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 10 monclient: _add_conns ranks=[3,8,6,7,2,4,10,1,0,5,9] 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a2cd0 0x7f7114132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 10 monclient(hunting): picked mon.noname-g con 0x7f71140a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711409fa20 0x7f711409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 10 monclient(hunting): picked mon.noname-e con 0x7f711409fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f71140a08e0 0x7f71140a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 10 monclient(hunting): picked mon.noname-d con 0x7f71140a08e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71140a30a0 con 0x7f71140a2cd0 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:25.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f71140a0330 con 0x7f71140a08e0 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f71140a1db0 con 0x7f711409fa20 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71196d4640 10 monclient(hunting): authenticate will time out at 1061.382289s 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a2cd0 0x7f7114132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71127fc640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711409fa20 msgr2=0x7f711409fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a2cd0 0x7f7114132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48506/0 (socket says 172.21.15.204:48506) 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 1 -- 172.21.15.204:0/425565692 learned_addr learned my addr 172.21.15.204:0/425565692 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 10 monclient(hunting): get_auth_request con 0x7f71140a2cd0 auth_method 0 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:25.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71127fc640 1 --2- 172.21.15.204:0/425565692 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711409fa20 0x7f711409fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 10 monclient(hunting): handle_auth_done global_id 15492 payload 274 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 1 -- 172.21.15.204:0/425565692 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711409fa20 msgr2=0x7f711409fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71137fe640 1 --2- 172.21.15.204:0/425565692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f71140a08e0 0x7f71140a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 1 --2- 172.21.15.204:0/425565692 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711409fa20 0x7f711409fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 1 -- 172.21.15.204:0/425565692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f71140a08e0 msgr2=0x7f71140a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 1 --2- 172.21.15.204:0/425565692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f71140a08e0 0x7f71140a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 1 monclient: found mon.noname-g 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:25.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f7112ffd640 1 -- 172.21.15.204:0/425565692 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71140648a0 con 0x7f71140a2cd0 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71137fe640 10 monclient: get_auth_request con 0x7f71140a08e0 auth_method 0 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.740+0000 7f71137fe640 1 --2- 172.21.15.204:0/425565692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f71140a08e0 0x7f71140a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7112ffd640 1 --2- 172.21.15.204:0/425565692 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a2cd0 0x7f7114132c00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7104009940 tx=0x7f71040091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=64f5f3070a7684d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7111ffb640 1 -- 172.21.15.204:0/425565692 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f710402ac80 con 0x7f71140a2cd0 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7111ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7111ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7111ffb640 10 monclient: dump: 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:25.753 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7111ffb640 10 monclient: _finish_auth 0 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7111ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7111ffb640 1 -- 172.21.15.204:0/425565692 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710402ae00 con 0x7f71140a2cd0 2024-09-15T01:37:25.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f71196d4640 5 monclient: authenticate success, global_id 15492 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7111ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f71196d4640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7111ffb640 1 -- 172.21.15.204:0/425565692 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7104031ca0 con 0x7f71140a2cd0 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f71196d4640 4 set_mon_vals no callback set 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7111ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7111ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.741+0000 7f7111ffb640 10 monclient: dump: 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:25.755 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 10 monclient: shutdown 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 1 -- 172.21.15.204:0/425565692 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a2cd0 msgr2=0x7f7114132c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 1 --2- 172.21.15.204:0/425565692 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a2cd0 0x7f7114132c00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7104009940 tx=0x7f71040091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:25.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 1 -- 172.21.15.204:0/425565692 shutdown_connections 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 1 --2- 172.21.15.204:0/425565692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f71140a08e0 0x7f71140a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 1 --2- 172.21.15.204:0/425565692 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711409fa20 0x7f711409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 1 --2- 172.21.15.204:0/425565692 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a2cd0 0x7f7114132c00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 1 -- 172.21.15.204:0/425565692 >> 172.21.15.204:0/425565692 conn(0x7f711406e860 msgr2=0x7f711406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 1 -- 172.21.15.204:0/425565692 shutdown_connections 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 1 -- 172.21.15.204:0/425565692 wait complete. 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 10 monclient: build_initial_monmap 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.742+0000 7f71196d4640 10 monclient: monmap: 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:25.757 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 1 Processor -- start 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 1 -- start start 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 10 monclient: init 2024-09-15T01:37:25.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 10 monclient: _add_conns ranks=[8,3,2,1,7,0,4,5,6] 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711409fa20 0x7f7114130fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 10 monclient(hunting): picked mon.noname-e con 0x7f711409fa20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a08e0 0x7f711411db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 10 monclient(hunting): picked mon.noname-g con 0x7f71140a08e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71140a2cd0 0x7f711411e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 10 monclient(hunting): picked mon.noname-b con 0x7f71140a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71141213e0 con 0x7f71140a2cd0 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711409d3d0 con 0x7f71140a08e0 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:25.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f711409cb60 con 0x7f711409fa20 2024-09-15T01:37:25.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:25.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71196d4640 10 monclient(hunting): authenticate will time out at 1061.385637s 2024-09-15T01:37:25.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a08e0 0x7f711411db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:25.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f7112ffd640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711409fa20 msgr2=0x7f7114130fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:25.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a08e0 0x7f711411db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48510/0 (socket says 172.21.15.204:48510) 2024-09-15T01:37:25.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 1 -- 172.21.15.204:0/1300294392 learned_addr learned my addr 172.21.15.204:0/1300294392 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:25.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 10 monclient(hunting): get_auth_request con 0x7f71140a08e0 auth_method 0 2024-09-15T01:37:25.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:25.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:25.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:25.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f7112ffd640 1 --2- 172.21.15.204:0/1300294392 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711409fa20 0x7f7114130fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71137fe640 1 -- 172.21.15.204:0/1300294392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71140a2cd0 msgr2=0x7f711411e040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71137fe640 1 --2- 172.21.15.204:0/1300294392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71140a2cd0 0x7f711411e040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 10 monclient(hunting): handle_auth_done global_id 15501 payload 824 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 1 -- 172.21.15.204:0/1300294392 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711409fa20 msgr2=0x7f7114130fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 1 --2- 172.21.15.204:0/1300294392 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711409fa20 0x7f7114130fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 1 -- 172.21.15.204:0/1300294392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71140a2cd0 msgr2=0x7f711411e040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 1 --2- 172.21.15.204:0/1300294392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71140a2cd0 0x7f711411e040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 10 monclient: _finish_hunting 0 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 1 monclient: found mon.noname-g 2024-09-15T01:37:25.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.743+0000 7f71127fc640 1 -- 172.21.15.204:0/1300294392 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7104009450 con 0x7f71140a08e0 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f71127fc640 1 --2- 172.21.15.204:0/1300294392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a08e0 0x7f711411db00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f710800b630 tx=0x7f71080091a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f70faffd640 1 -- 172.21.15.204:0/1300294392 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7108015070 con 0x7f71140a08e0 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f70faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f70faffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f70faffd640 10 monclient: dump: 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:25.762 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f70faffd640 10 monclient: _finish_auth 0 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f70faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f70faffd640 1 -- 172.21.15.204:0/1300294392 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108006150 con 0x7f71140a08e0 2024-09-15T01:37:25.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f71196d4640 5 monclient: authenticate success, global_id 15501 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f71196d4640 10 monclient: _renew_subs 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f71196d4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f71196d4640 1 -- 172.21.15.204:0/1300294392 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71141427a0 con 0x7f71140a08e0 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f70faffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f70faffd640 1 -- 172.21.15.204:0/1300294392 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7108006740 con 0x7f71140a08e0 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f70faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f70faffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f70faffd640 10 monclient: dump: 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:25.764 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:25.765 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f71196d4640 10 monclient: _renew_subs 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f71196d4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f71196d4640 1 -- 172.21.15.204:0/1300294392 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711411f610 con 0x7f71140a08e0 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.744+0000 7f71117fa640 4 set_mon_vals no callback set 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.745+0000 7f70faffd640 1 -- 172.21.15.204:0/1300294392 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f71080062c0 con 0x7f71140a08e0 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.745+0000 7f71196d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.745+0000 7f71196d4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.745+0000 7f71196d4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.745+0000 7f70faffd640 1 --2- 172.21.15.204:0/1300294392 >> v2:172.21.15.204:6812/28210 conn(0x7f71000397b0 0x7f710003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.745+0000 7f71196d4640 1 -- 172.21.15.204:0/1300294392 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71141215e0 con 0x7f71140a08e0 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.745+0000 7f70faffd640 1 -- 172.21.15.204:0/1300294392 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 12422+0+0 (secure 0 0 0) 0x7f710804cc10 con 0x7f71140a08e0 2024-09-15T01:37:25.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.745+0000 7f7112ffd640 1 --2- 172.21.15.204:0/1300294392 >> v2:172.21.15.204:6812/28210 conn(0x7f71000397b0 0x7f710003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.745+0000 7f7112ffd640 10 monclient: get_auth_request con 0x7f71000397b0 auth_method 0 2024-09-15T01:37:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.745+0000 7f7112ffd640 1 --2- 172.21.15.204:0/1300294392 >> v2:172.21.15.204:6812/28210 conn(0x7f71000397b0 0x7f710003bc60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7104034000 tx=0x7f7104032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.750+0000 7f70faffd640 1 -- 172.21.15.204:0/1300294392 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7108010440 con 0x7f71140a08e0 2024-09-15T01:37:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.750+0000 7f70faffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:25.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.750+0000 7f70faffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:25.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.907+0000 7f71196d4640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:25.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.907+0000 7f71196d4640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:25.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.907+0000 7f71196d4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:25.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:25.907+0000 7f71196d4640 1 -- 172.21.15.204:0/1300294392 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f7114070b80 con 0x7f71140a08e0 2024-09-15T01:37:26.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.561+0000 7f70faffd640 1 -- 172.21.15.204:0/1300294392 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}]=0 added bucket h25 type host to location {rack=r1} v210) v1 ==== 168+0+0 (secure 0 0 0) 0x7f710800d370 con 0x7f71140a08e0 2024-09-15T01:37:26.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.561+0000 7f70faffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:26.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.561+0000 7f70faffd640 10 monclient: _finish_command 2 = system:0 added bucket h25 type host to location {rack=r1} 2024-09-15T01:37:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.659+0000 7f71196d4640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:26.660 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.659+0000 7f71196d4640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.659+0000 7f71196d4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.659+0000 7f71196d4640 1 -- 172.21.15.204:0/1300294392 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f71141211c0 con 0x7f71140a08e0 2024-09-15T01:37:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.659+0000 7f70faffd640 1 -- 172.21.15.204:0/1300294392 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h25' already exists v210) v1 ==== 147+0+0 (secure 0 0 0) 0x7f710800eb30 con 0x7f71140a08e0 2024-09-15T01:37:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.659+0000 7f70faffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.659+0000 7f70faffd640 10 monclient: _finish_command 3 = system:0 bucket 'h25' already exists 2024-09-15T01:37:26.661 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h25' already exists 2024-09-15T01:37:26.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 1 -- 172.21.15.204:0/1300294392 >> v2:172.21.15.204:6812/28210 conn(0x7f71000397b0 msgr2=0x7f710003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:26.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 1 --2- 172.21.15.204:0/1300294392 >> v2:172.21.15.204:6812/28210 conn(0x7f71000397b0 0x7f710003bc60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7104034000 tx=0x7f7104032000 comp rx=0 tx=0).stop 2024-09-15T01:37:26.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 10 monclient: shutdown 2024-09-15T01:37:26.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:26.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 1 -- 172.21.15.204:0/1300294392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a08e0 msgr2=0x7f711411db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:26.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 1 --2- 172.21.15.204:0/1300294392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a08e0 0x7f711411db00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f710800b630 tx=0x7f71080091a0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 1 -- 172.21.15.204:0/1300294392 shutdown_connections 2024-09-15T01:37:26.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 1 --2- 172.21.15.204:0/1300294392 >> v2:172.21.15.204:6812/28210 conn(0x7f71000397b0 0x7f710003bc60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 1 --2- 172.21.15.204:0/1300294392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71140a08e0 0x7f711411db00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 1 --2- 172.21.15.204:0/1300294392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71140a2cd0 0x7f711411e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 1 --2- 172.21.15.204:0/1300294392 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f711409fa20 0x7f7114130fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 1 -- 172.21.15.204:0/1300294392 >> 172.21.15.204:0/1300294392 conn(0x7f711406e860 msgr2=0x7f71140a5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:26.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 1 -- 172.21.15.204:0/1300294392 shutdown_connections 2024-09-15T01:37:26.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.661+0000 7f71196d4640 1 -- 172.21.15.204:0/1300294392 wait complete. 2024-09-15T01:37:26.672 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:26.673 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h26 host rack=r1 2024-09-15T01:37:26.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.746+0000 7f021579d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:26.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.746+0000 7f021579d640 10 monclient: build_initial_monmap 2024-09-15T01:37:26.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.746+0000 7f021579d640 10 monclient: monmap: 2024-09-15T01:37:26.749 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:26.749 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:26.749 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:26.747386+0000 2024-09-15T01:37:26.749 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:26.747386+0000 2024-09-15T01:37:26.749 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:26.749 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:26.749 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:26.749 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:26.750 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:26.750 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:26.750 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:26.750 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:26.750 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:26.750 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:26.750 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:26.750 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:26.750 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:26.750 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 1 Processor -- start 2024-09-15T01:37:26.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 1 -- start start 2024-09-15T01:37:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 10 monclient: init 2024-09-15T01:37:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 10 monclient: _add_conns ranks=[8,10,2,5,9,4,0,6,3,1,7] 2024-09-15T01:37:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02100a2cd0 0x7f0210132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 10 monclient(hunting): picked mon.noname-e con 0x7f02100a2cd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f021009fa20 0x7f021009fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 10 monclient(hunting): picked mon.noname-f con 0x7f021009fa20 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02100a08e0 0x7f02100a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 10 monclient(hunting): picked mon.noname-b con 0x7f02100a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:26.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:26.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02100a30a0 con 0x7f021009fa20 2024-09-15T01:37:26.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:26.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02100a0330 con 0x7f02100a08e0 2024-09-15T01:37:26.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:26.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f02100a1db0 con 0x7f02100a2cd0 2024-09-15T01:37:26.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:26.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f021579d640 10 monclient(hunting): authenticate will time out at 1062.390616s 2024-09-15T01:37:26.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f02067fc640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f021009fa20 0x7f021009fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:26.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f020effd640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02100a2cd0 msgr2=0x7f0210132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:26.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f020f7fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02100a08e0 msgr2=0x7f02100a0d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:26.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f020effd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02100a2cd0 0x7f0210132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:26.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f020f7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02100a08e0 0x7f02100a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:26.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f02067fc640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f021009fa20 0x7f021009fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48758/0 (socket says 172.21.15.204:48758) 2024-09-15T01:37:26.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f02067fc640 1 -- 172.21.15.204:0/397682718 learned_addr learned my addr 172.21.15.204:0/397682718 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:26.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f02067fc640 10 monclient(hunting): get_auth_request con 0x7f021009fa20 auth_method 0 2024-09-15T01:37:26.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f02067fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:26.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f02067fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:26.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f02067fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:26.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f02067fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:26.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f02067fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:26.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f02067fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.748+0000 7f02067fc640 10 monclient(hunting): handle_auth_done global_id 7093 payload 274 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f02067fc640 1 -- 172.21.15.204:0/397682718 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02100a2cd0 msgr2=0x7f0210132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f02067fc640 1 --2- 172.21.15.204:0/397682718 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02100a2cd0 0x7f0210132c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f02067fc640 1 -- 172.21.15.204:0/397682718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02100a08e0 msgr2=0x7f02100a0d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f02067fc640 1 --2- 172.21.15.204:0/397682718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02100a08e0 0x7f02100a0d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f02067fc640 10 monclient: _finish_hunting 0 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f02067fc640 1 monclient: found mon.noname-f 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f02067fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f02067fc640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f02067fc640 1 -- 172.21.15.204:0/397682718 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02100648a0 con 0x7f021009fa20 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f02067fc640 1 --2- 172.21.15.204:0/397682718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f021009fa20 0x7f021009fdf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009910 tx=0x7f01f80091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f85a5f94890c6f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f020e7fc640 1 -- 172.21.15.204:0/397682718 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01f802acd0 con 0x7f021009fa20 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f020e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f020e7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:26.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f020e7fc640 10 monclient: dump: 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:26.764 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:26.765 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:26.765 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:26.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f020e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:26.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f020e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:26.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f021579d640 5 monclient: authenticate success, global_id 7093 2024-09-15T01:37:26.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f020e7fc640 1 -- 172.21.15.204:0/397682718 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f802ae50 con 0x7f021009fa20 2024-09-15T01:37:26.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f021579d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:26.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f020e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:26.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f021579d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:26.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f020e7fc640 1 -- 172.21.15.204:0/397682718 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01f8031ca0 con 0x7f021009fa20 2024-09-15T01:37:26.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f020e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:26.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f021579d640 4 set_mon_vals no callback set 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f020e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.749+0000 7f020e7fc640 10 monclient: dump: 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:26.766 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:26.767 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:26.767 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:26.767 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:26.767 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:26.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.750+0000 7f021579d640 10 monclient: shutdown 2024-09-15T01:37:26.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.750+0000 7f021579d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:26.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.750+0000 7f021579d640 1 -- 172.21.15.204:0/397682718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f021009fa20 msgr2=0x7f021009fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:26.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.750+0000 7f021579d640 1 --2- 172.21.15.204:0/397682718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f021009fa20 0x7f021009fdf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009910 tx=0x7f01f80091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.750+0000 7f021579d640 1 -- 172.21.15.204:0/397682718 shutdown_connections 2024-09-15T01:37:26.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.750+0000 7f021579d640 1 --2- 172.21.15.204:0/397682718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f021009fa20 0x7f021009fdf0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.750+0000 7f021579d640 1 --2- 172.21.15.204:0/397682718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02100a08e0 0x7f02100a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.750+0000 7f021579d640 1 --2- 172.21.15.204:0/397682718 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02100a2cd0 0x7f0210132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.750+0000 7f021579d640 1 -- 172.21.15.204:0/397682718 >> 172.21.15.204:0/397682718 conn(0x7f021006e860 msgr2=0x7f021006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.750+0000 7f021579d640 1 -- 172.21.15.204:0/397682718 shutdown_connections 2024-09-15T01:37:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.750+0000 7f021579d640 1 -- 172.21.15.204:0/397682718 wait complete. 2024-09-15T01:37:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.751+0000 7f021579d640 10 monclient: build_initial_monmap 2024-09-15T01:37:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.751+0000 7f021579d640 10 monclient: monmap: 2024-09-15T01:37:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:26.768 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:26.769 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.751+0000 7f021579d640 1 Processor -- start 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.751+0000 7f021579d640 1 -- start start 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.751+0000 7f021579d640 10 monclient: init 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient: _add_conns ranks=[5,8,1,2,0,4,6,3,7] 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f021009fa20 0x7f0210130e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient(hunting): picked mon.noname-h con 0x7f021009fa20 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02100a08e0 0x7f021011d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient(hunting): picked mon.noname-e con 0x7f02100a08e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02100a2cd0 0x7f021011de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient(hunting): picked mon.noname-f con 0x7f02100a2cd0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:26.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021011fcc0 con 0x7f02100a2cd0 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f021009d3d0 con 0x7f021009fa20 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f021009cb60 con 0x7f02100a08e0 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient(hunting): authenticate will time out at 1062.394108s 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020effd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f021009fa20 0x7f0210130e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02100a2cd0 0x7f021011de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02100a2cd0 0x7f021011de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48760/0 (socket says 172.21.15.204:48760) 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020effd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f021009fa20 0x7f0210130e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:38506/0 (socket says 172.21.15.204:38506) 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 1 -- 172.21.15.204:0/3767923050 learned_addr learned my addr 172.21.15.204:0/3767923050 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:26.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 10 monclient(hunting): get_auth_request con 0x7f02100a2cd0 auth_method 0 2024-09-15T01:37:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020effd640 10 monclient(hunting): get_auth_request con 0x7f021009fa20 auth_method 0 2024-09-15T01:37:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02067fc640 1 -- 172.21.15.204:0/3767923050 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02100a08e0 msgr2=0x7f021011d940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02067fc640 1 --2- 172.21.15.204:0/3767923050 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02100a08e0 0x7f021011d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:26.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 10 monclient(hunting): handle_auth_done global_id 7102 payload 824 2024-09-15T01:37:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 1 -- 172.21.15.204:0/3767923050 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02100a08e0 msgr2=0x7f021011d940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 1 --2- 172.21.15.204:0/3767923050 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02100a08e0 0x7f021011d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 1 -- 172.21.15.204:0/3767923050 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f021009fa20 msgr2=0x7f0210130e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 1 --2- 172.21.15.204:0/3767923050 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f021009fa20 0x7f0210130e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 1 monclient: found mon.noname-f 2024-09-15T01:37:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:26.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 1 -- 172.21.15.204:0/3767923050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f8009450 con 0x7f02100a2cd0 2024-09-15T01:37:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020effd640 1 --2- 172.21.15.204:0/3767923050 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f021009fa20 0x7f0210130e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020f7fe640 1 --2- 172.21.15.204:0/3767923050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02100a2cd0 0x7f021011de80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f020800bb80 tx=0x7f02080097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02077fe640 1 -- 172.21.15.204:0/3767923050 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f020800cea0 con 0x7f02100a2cd0 2024-09-15T01:37:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02077fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02077fe640 10 monclient: dump: 2024-09-15T01:37:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:26.774 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:26.775 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:26.775 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02077fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 5 monclient: authenticate success, global_id 7102 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02077fe640 1 -- 172.21.15.204:0/3767923050 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0208006150 con 0x7f02100a2cd0 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient: _renew_subs 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 1 -- 172.21.15.204:0/3767923050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0210120f50 con 0x7f02100a2cd0 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02077fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02077fe640 1 -- 172.21.15.204:0/3767923050 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0208006780 con 0x7f02100a2cd0 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02077fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f02077fe640 10 monclient: dump: 2024-09-15T01:37:26.776 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:26.777 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:26.778 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f020dffb640 4 set_mon_vals no callback set 2024-09-15T01:37:26.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient: _renew_subs 2024-09-15T01:37:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.752+0000 7f021579d640 1 -- 172.21.15.204:0/3767923050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021011f490 con 0x7f02100a2cd0 2024-09-15T01:37:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.753+0000 7f02077fe640 1 -- 172.21.15.204:0/3767923050 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0208012020 con 0x7f02100a2cd0 2024-09-15T01:37:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.753+0000 7f021579d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.753+0000 7f021579d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.753+0000 7f021579d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:26.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.753+0000 7f021579d640 1 -- 172.21.15.204:0/3767923050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0210070b80 con 0x7f02100a2cd0 2024-09-15T01:37:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.753+0000 7f02077fe640 1 --2- 172.21.15.204:0/3767923050 >> v2:172.21.15.204:6812/28210 conn(0x7f01f00397b0 0x7f01f003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.754+0000 7f02077fe640 1 -- 172.21.15.204:0/3767923050 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 12635+0+0 (secure 0 0 0) 0x7f020804d430 con 0x7f02100a2cd0 2024-09-15T01:37:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.754+0000 7f020effd640 1 --2- 172.21.15.204:0/3767923050 >> v2:172.21.15.204:6812/28210 conn(0x7f01f00397b0 0x7f01f003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.754+0000 7f020effd640 10 monclient: get_auth_request con 0x7f01f00397b0 auth_method 0 2024-09-15T01:37:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.754+0000 7f020effd640 1 --2- 172.21.15.204:0/3767923050 >> v2:172.21.15.204:6812/28210 conn(0x7f01f00397b0 0x7f01f003bc60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f02000038f0 tx=0x7f02000090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.758+0000 7f02077fe640 1 -- 172.21.15.204:0/3767923050 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f020801d900 con 0x7f02100a2cd0 2024-09-15T01:37:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.758+0000 7f02077fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:26.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.758+0000 7f02077fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:26.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.916+0000 7f021579d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:26.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.917+0000 7f021579d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:26.918 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.917+0000 7f021579d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:26.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:26.917+0000 7f021579d640 1 -- 172.21.15.204:0/3767923050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f0210067450 con 0x7f02100a2cd0 2024-09-15T01:37:27.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.567+0000 7f02077fe640 1 -- 172.21.15.204:0/3767923050 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}]=0 added bucket h26 type host to location {rack=r1} v211) v1 ==== 168+0+0 (secure 0 0 0) 0x7f0208014340 con 0x7f02100a2cd0 2024-09-15T01:37:27.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.567+0000 7f02077fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:27.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.567+0000 7f02077fe640 10 monclient: _finish_command 2 = system:0 added bucket h26 type host to location {rack=r1} 2024-09-15T01:37:27.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.665+0000 7f021579d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:27.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.665+0000 7f021579d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:27.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.665+0000 7f021579d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:27.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.665+0000 7f021579d640 1 -- 172.21.15.204:0/3767923050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f0210121290 con 0x7f02100a2cd0 2024-09-15T01:37:27.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.665+0000 7f02077fe640 1 -- 172.21.15.204:0/3767923050 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h26' already exists v211) v1 ==== 147+0+0 (secure 0 0 0) 0x7f020801d300 con 0x7f02100a2cd0 2024-09-15T01:37:27.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.665+0000 7f02077fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:27.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.665+0000 7f02077fe640 10 monclient: _finish_command 3 = system:0 bucket 'h26' already exists 2024-09-15T01:37:27.667 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h26' already exists 2024-09-15T01:37:27.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 1 -- 172.21.15.204:0/3767923050 >> v2:172.21.15.204:6812/28210 conn(0x7f01f00397b0 msgr2=0x7f01f003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:27.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 1 --2- 172.21.15.204:0/3767923050 >> v2:172.21.15.204:6812/28210 conn(0x7f01f00397b0 0x7f01f003bc60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f02000038f0 tx=0x7f02000090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:27.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 10 monclient: shutdown 2024-09-15T01:37:27.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:27.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 1 -- 172.21.15.204:0/3767923050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02100a2cd0 msgr2=0x7f021011de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:27.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 1 --2- 172.21.15.204:0/3767923050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02100a2cd0 0x7f021011de80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f020800bb80 tx=0x7f02080097b0 comp rx=0 tx=0).stop 2024-09-15T01:37:27.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 1 -- 172.21.15.204:0/3767923050 shutdown_connections 2024-09-15T01:37:27.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 1 --2- 172.21.15.204:0/3767923050 >> v2:172.21.15.204:6812/28210 conn(0x7f01f00397b0 0x7f01f003bc60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:27.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 1 --2- 172.21.15.204:0/3767923050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02100a2cd0 0x7f021011de80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:27.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 1 --2- 172.21.15.204:0/3767923050 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02100a08e0 0x7f021011d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:27.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 1 --2- 172.21.15.204:0/3767923050 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f021009fa20 0x7f0210130e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:27.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 1 -- 172.21.15.204:0/3767923050 >> 172.21.15.204:0/3767923050 conn(0x7f021006e860 msgr2=0x7f02100a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:27.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 1 -- 172.21.15.204:0/3767923050 shutdown_connections 2024-09-15T01:37:27.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.667+0000 7f021579d640 1 -- 172.21.15.204:0/3767923050 wait complete. 2024-09-15T01:37:27.677 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:27.677 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h27 host rack=r1 2024-09-15T01:37:27.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.752+0000 7f74690ef640 10 monclient: get_monmap_and_config 2024-09-15T01:37:27.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.752+0000 7f74690ef640 10 monclient: build_initial_monmap 2024-09-15T01:37:27.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.752+0000 7f74690ef640 10 monclient: monmap: 2024-09-15T01:37:27.754 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:27.754 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:27.754 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:27.752734+0000 2024-09-15T01:37:27.754 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:27.752734+0000 2024-09-15T01:37:27.754 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:27.754 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:27.754 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:27.754 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:27.754 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:27.754 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:27.754 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:27.755 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:27.755 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:27.755 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:27.755 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:27.755 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:27.755 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:27.755 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:27.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.752+0000 7f74690ef640 1 Processor -- start 2024-09-15T01:37:27.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 1 -- start start 2024-09-15T01:37:27.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 10 monclient: init 2024-09-15T01:37:27.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:27.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 10 monclient: _add_conns ranks=[0,10,8,7,4,2,5,3,9,1,6] 2024-09-15T01:37:27.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f74640a4590 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:27.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f746406edd0 0x7f746409ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:27.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 10 monclient(hunting): picked mon.noname-f con 0x7f746406edd0 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:27.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7464135ad0 0x7f7464137eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:27.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 10 monclient(hunting): picked mon.noname-e con 0x7f7464135ad0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:27.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74640648a0 con 0x7f74640a4590 2024-09-15T01:37:27.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:27.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74640a4960 con 0x7f746406edd0 2024-09-15T01:37:27.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:27.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f746406dba0 con 0x7f7464135ad0 2024-09-15T01:37:27.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:27.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f74690ef640 10 monclient(hunting): authenticate will time out at 1063.395503s 2024-09-15T01:37:27.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f746406edd0 0x7f746409ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:27.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f746406edd0 0x7f746409ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36540/0 (socket says 172.21.15.204:36540) 2024-09-15T01:37:27.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 1 -- 172.21.15.204:0/2788086474 learned_addr learned my addr 172.21.15.204:0/2788086474 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:27.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 10 monclient(hunting): get_auth_request con 0x7f746406edd0 auth_method 0 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f746359e640 1 -- 172.21.15.204:0/2788086474 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7464135ad0 msgr2=0x7f7464137eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f7462d9d640 1 -- 172.21.15.204:0/2788086474 >> v1:172.21.15.178:6789/0 conn(0x7f74640a4590 legacy=0x7f746409e750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f746359e640 1 --2- 172.21.15.204:0/2788086474 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7464135ad0 0x7f7464137eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 10 monclient(hunting): handle_auth_done global_id 7111 payload 274 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 1 -- 172.21.15.204:0/2788086474 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7464135ad0 msgr2=0x7f7464137eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 1 --2- 172.21.15.204:0/2788086474 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7464135ad0 0x7f7464137eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:27.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 1 -- 172.21.15.204:0/2788086474 >> v1:172.21.15.178:6789/0 conn(0x7f74640a4590 legacy=0x7f746409e750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 1 monclient: found mon.noname-f 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.753+0000 7f745bfff640 1 -- 172.21.15.204:0/2788086474 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74640a04c0 con 0x7f746406edd0 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f745bfff640 1 --2- 172.21.15.204:0/2788086474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f746406edd0 0x7f746409ee60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f744c009910 tx=0x7f744c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=767f194fcab302eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f746259c640 1 -- 172.21.15.204:0/2788086474 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f744c02acc0 con 0x7f746406edd0 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f746259c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f746259c640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f746259c640 10 monclient: dump: 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:27.766 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f746259c640 10 monclient: _finish_auth 0 2024-09-15T01:37:27.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f746259c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f746259c640 1 -- 172.21.15.204:0/2788086474 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c02ae40 con 0x7f746406edd0 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f74690ef640 5 monclient: authenticate success, global_id 7111 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f74690ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f746259c640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f74690ef640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f746259c640 1 -- 172.21.15.204:0/2788086474 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f744c031ca0 con 0x7f746406edd0 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f746259c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f74690ef640 4 set_mon_vals no callback set 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f746259c640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.754+0000 7f746259c640 10 monclient: dump: 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:27.768 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:27.769 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.755+0000 7f74690ef640 10 monclient: shutdown 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.755+0000 7f74690ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.755+0000 7f74690ef640 1 -- 172.21.15.204:0/2788086474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f746406edd0 msgr2=0x7f746409ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.755+0000 7f74690ef640 1 --2- 172.21.15.204:0/2788086474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f746406edd0 0x7f746409ee60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f744c009910 tx=0x7f744c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.755+0000 7f74690ef640 1 -- 172.21.15.204:0/2788086474 shutdown_connections 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.755+0000 7f74690ef640 1 --2- 172.21.15.204:0/2788086474 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7464135ad0 0x7f7464137eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.755+0000 7f74690ef640 1 --2- 172.21.15.204:0/2788086474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f746406edd0 0x7f746409ee60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.755+0000 7f74690ef640 1 -- 172.21.15.204:0/2788086474 >> 172.21.15.204:0/2788086474 conn(0x7f7464067700 msgr2=0x7f74640a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.755+0000 7f74690ef640 1 -- 172.21.15.204:0/2788086474 shutdown_connections 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.755+0000 7f74690ef640 1 -- 172.21.15.204:0/2788086474 wait complete. 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.755+0000 7f74690ef640 10 monclient: build_initial_monmap 2024-09-15T01:37:27.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.755+0000 7f74690ef640 10 monclient: monmap: 2024-09-15T01:37:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:27.771 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 1 Processor -- start 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 1 -- start start 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 10 monclient: init 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 10 monclient: _add_conns ranks=[2,6,0,7,5,8,1,4,3] 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f746406edd0 0x7f74641347a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 10 monclient(hunting): picked mon.noname-b con 0x7f746406edd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:27.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f74640a4590 0x7f74641307d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 10 monclient(hunting): picked mon.noname-d con 0x7f74640a4590 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7464135ad0 0x7f7464068240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 10 monclient(hunting): picked mon.noname-a con 0x7f7464135ad0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746406d070 con 0x7f7464135ad0 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746406cc80 con 0x7f746406edd0 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f746411e900 con 0x7f74640a4590 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f74690ef640 10 monclient(hunting): authenticate will time out at 1063.398738s 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f746359e640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7464135ad0 msgr2=0x7f7464068240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:27.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f7462d9d640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f746406edd0 msgr2=0x7f74641347a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f746359e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7464135ad0 0x7f7464068240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.756+0000 7f7462d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f746406edd0 0x7f74641347a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f74640a4590 0x7f74641307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f74640a4590 0x7f74641307d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:36348/0 (socket says 172.21.15.204:36348) 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 1 -- 172.21.15.204:0/488698916 learned_addr learned my addr 172.21.15.204:0/488698916 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 10 monclient(hunting): get_auth_request con 0x7f74640a4590 auth_method 0 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:27.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 10 monclient(hunting): handle_auth_done global_id 24459 payload 824 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 1 -- 172.21.15.204:0/488698916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f746406edd0 msgr2=0x7f74641347a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 1 --2- 172.21.15.204:0/488698916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f746406edd0 0x7f74641347a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 1 -- 172.21.15.204:0/488698916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7464135ad0 msgr2=0x7f7464068240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 1 --2- 172.21.15.204:0/488698916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7464135ad0 0x7f7464068240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 1 monclient: found mon.noname-d 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 1 -- 172.21.15.204:0/488698916 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c009450 con 0x7f74640a4590 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745bfff640 1 --2- 172.21.15.204:0/488698916 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f74640a4590 0x7f74641307d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f744c009970 tx=0x7f744c032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745affd640 1 -- 172.21.15.204:0/488698916 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f744c02acc0 con 0x7f74640a4590 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:27.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745affd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745affd640 10 monclient: dump: 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:27.776 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745affd640 10 monclient: _finish_auth 0 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f74690ef640 5 monclient: authenticate success, global_id 24459 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745affd640 1 -- 172.21.15.204:0/488698916 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c03b3f0 con 0x7f74640a4590 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745affd640 1 -- 172.21.15.204:0/488698916 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f744c03b9e0 con 0x7f74640a4590 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:37:27.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f745affd640 10 monclient: dump: 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:27.778 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:27.779 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f7461d9b640 4 set_mon_vals no callback set 2024-09-15T01:37:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f74690ef640 10 monclient: _renew_subs 2024-09-15T01:37:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f74690ef640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f74690ef640 1 -- 172.21.15.204:0/488698916 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7464135720 con 0x7f74640a4590 2024-09-15T01:37:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f74690ef640 10 monclient: _renew_subs 2024-09-15T01:37:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f74690ef640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.757+0000 7f74690ef640 1 -- 172.21.15.204:0/488698916 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7464135210 con 0x7f74640a4590 2024-09-15T01:37:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.758+0000 7f745affd640 1 -- 172.21.15.204:0/488698916 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f744c039040 con 0x7f74640a4590 2024-09-15T01:37:27.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.758+0000 7f74690ef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.758+0000 7f74690ef640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.758+0000 7f74690ef640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.758+0000 7f74690ef640 1 -- 172.21.15.204:0/488698916 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7464070b80 con 0x7f74640a4590 2024-09-15T01:37:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.758+0000 7f745affd640 1 --2- 172.21.15.204:0/488698916 >> v2:172.21.15.204:6812/28210 conn(0x7f74440397b0 0x7f744403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.758+0000 7f745affd640 1 -- 172.21.15.204:0/488698916 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 12848+0+0 (secure 0 0 0) 0x7f744c06d7c0 con 0x7f74640a4590 2024-09-15T01:37:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.759+0000 7f7462d9d640 1 --2- 172.21.15.204:0/488698916 >> v2:172.21.15.204:6812/28210 conn(0x7f74440397b0 0x7f744403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.759+0000 7f7462d9d640 10 monclient: get_auth_request con 0x7f74440397b0 auth_method 0 2024-09-15T01:37:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.759+0000 7f7462d9d640 1 --2- 172.21.15.204:0/488698916 >> v2:172.21.15.204:6812/28210 conn(0x7f74440397b0 0x7f744403bc60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f74540038f0 tx=0x7f7454006c60 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:27.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.763+0000 7f745affd640 1 -- 172.21.15.204:0/488698916 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f744c04ad70 con 0x7f74640a4590 2024-09-15T01:37:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.763+0000 7f745affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:27.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.763+0000 7f745affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:27.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.921+0000 7f74690ef640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:27.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.921+0000 7f74690ef640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:27.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.921+0000 7f74690ef640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:27.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:27.921+0000 7f74690ef640 1 -- 172.21.15.204:0/488698916 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f746406d8c0 con 0x7f74640a4590 2024-09-15T01:37:28.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.575+0000 7f745affd640 1 -- 172.21.15.204:0/488698916 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}]=0 added bucket h27 type host to location {rack=r1} v212) v1 ==== 168+0+0 (secure 0 0 0) 0x7f744c037070 con 0x7f74640a4590 2024-09-15T01:37:28.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.575+0000 7f745affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:28.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.575+0000 7f745affd640 10 monclient: _finish_command 2 = system:0 added bucket h27 type host to location {rack=r1} 2024-09-15T01:37:28.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.673+0000 7f74690ef640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:28.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.673+0000 7f74690ef640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:28.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.673+0000 7f74690ef640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:28.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.673+0000 7f74690ef640 1 -- 172.21.15.204:0/488698916 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f746406d8c0 con 0x7f74640a4590 2024-09-15T01:37:28.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.674+0000 7f745affd640 1 -- 172.21.15.204:0/488698916 <== mon.6 v2:172.21.15.178:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h27' already exists v212) v1 ==== 147+0+0 (secure 0 0 0) 0x7f744c04be40 con 0x7f74640a4590 2024-09-15T01:37:28.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.674+0000 7f745affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:28.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.674+0000 7f745affd640 10 monclient: _finish_command 3 = system:0 bucket 'h27' already exists 2024-09-15T01:37:28.677 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h27' already exists 2024-09-15T01:37:28.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.676+0000 7f74690ef640 1 -- 172.21.15.204:0/488698916 >> v2:172.21.15.204:6812/28210 conn(0x7f74440397b0 msgr2=0x7f744403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:28.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.676+0000 7f74690ef640 1 --2- 172.21.15.204:0/488698916 >> v2:172.21.15.204:6812/28210 conn(0x7f74440397b0 0x7f744403bc60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f74540038f0 tx=0x7f7454006c60 comp rx=0 tx=0).stop 2024-09-15T01:37:28.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.676+0000 7f74690ef640 10 monclient: shutdown 2024-09-15T01:37:28.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.676+0000 7f74690ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:28.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.676+0000 7f74690ef640 1 -- 172.21.15.204:0/488698916 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f74640a4590 msgr2=0x7f74641307d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:28.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.676+0000 7f74690ef640 1 --2- 172.21.15.204:0/488698916 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f74640a4590 0x7f74641307d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f744c009970 tx=0x7f744c032000 comp rx=0 tx=0).stop 2024-09-15T01:37:28.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.676+0000 7f74690ef640 1 -- 172.21.15.204:0/488698916 shutdown_connections 2024-09-15T01:37:28.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.676+0000 7f74690ef640 1 --2- 172.21.15.204:0/488698916 >> v2:172.21.15.204:6812/28210 conn(0x7f74440397b0 0x7f744403bc60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:28.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.676+0000 7f74690ef640 1 --2- 172.21.15.204:0/488698916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7464135ad0 0x7f7464068240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:28.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.676+0000 7f74690ef640 1 --2- 172.21.15.204:0/488698916 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f74640a4590 0x7f74641307d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:28.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.676+0000 7f74690ef640 1 --2- 172.21.15.204:0/488698916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f746406edd0 0x7f74641347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:28.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.676+0000 7f74690ef640 1 -- 172.21.15.204:0/488698916 >> 172.21.15.204:0/488698916 conn(0x7f7464067700 msgr2=0x7f746409cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:28.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.677+0000 7f74690ef640 1 -- 172.21.15.204:0/488698916 shutdown_connections 2024-09-15T01:37:28.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.677+0000 7f74690ef640 1 -- 172.21.15.204:0/488698916 wait complete. 2024-09-15T01:37:28.688 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:28.688 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h28 host rack=r1 2024-09-15T01:37:28.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.762+0000 7f4318caf640 10 monclient: get_monmap_and_config 2024-09-15T01:37:28.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.762+0000 7f4318caf640 10 monclient: build_initial_monmap 2024-09-15T01:37:28.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.762+0000 7f4318caf640 10 monclient: monmap: 2024-09-15T01:37:28.765 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:28.765 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:28.765 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:28.763712+0000 2024-09-15T01:37:28.765 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:28.763712+0000 2024-09-15T01:37:28.765 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:28.765 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:28.765 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:28.765 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:28.765 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:28.766 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:28.766 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:28.766 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:28.766 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:28.766 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:28.766 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:28.766 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:28.766 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:28.766 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.762+0000 7f4318caf640 1 Processor -- start 2024-09-15T01:37:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.762+0000 7f4318caf640 1 -- start start 2024-09-15T01:37:28.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.762+0000 7f4318caf640 10 monclient: init 2024-09-15T01:37:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 10 monclient: _add_conns ranks=[10,7,1,0,9,4,8,5,2,6,3] 2024-09-15T01:37:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f431409f0e0 0x7f431409e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 10 monclient(hunting): picked mon.noname-f con 0x7f431409f0e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f4b0 0x7f4314132930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 10 monclient(hunting): picked mon.noname-i con 0x7f431409f4b0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f431409f880 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43140648a0 con 0x7f431409f880 2024-09-15T01:37:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43140683f0 con 0x7f431409f0e0 2024-09-15T01:37:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:28.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4314069590 con 0x7f431409f4b0 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4318caf640 10 monclient(hunting): authenticate will time out at 1064.406549s 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f431259c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f431409f0e0 0x7f431409e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f4b0 0x7f4314132930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f431259c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f431409f0e0 0x7f431409e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36556/0 (socket says 172.21.15.204:36556) 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f4b0 0x7f4314132930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:33468/0 (socket says 172.21.15.204:33468) 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f431259c640 1 -- 172.21.15.204:0/730917458 learned_addr learned my addr 172.21.15.204:0/730917458 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f431259c640 10 monclient(hunting): get_auth_request con 0x7f431409f0e0 auth_method 0 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f431259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f431259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f431259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4312d9d640 1 --1- 172.21.15.204:0/730917458 >> v1:172.21.15.204:6789/0 conn(0x7f431409f880 0x7f4314132e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:38376/0 (socket says 172.21.15.204:38376) 2024-09-15T01:37:28.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 10 monclient(hunting): get_auth_request con 0x7f431409f4b0 auth_method 0 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f431259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f431259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f431259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 10 monclient(hunting): handle_auth_done global_id 24568 payload 274 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 1 -- 172.21.15.204:0/730917458 >> v2:172.21.15.204:3300/0 conn(0x7f431409f0e0 msgr2=0x7f431409e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 1 --2- 172.21.15.204:0/730917458 >> v2:172.21.15.204:3300/0 conn(0x7f431409f0e0 0x7f431409e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:28.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 1 -- 172.21.15.204:0/730917458 >> v1:172.21.15.204:6789/0 conn(0x7f431409f880 legacy=0x7f4314132e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 1 monclient: found mon.noname-i 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f4311d9b640 1 -- 172.21.15.204:0/730917458 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431409ffc0 con 0x7f431409f4b0 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.763+0000 7f431259c640 1 --2- 172.21.15.204:0/730917458 >> v2:172.21.15.204:3300/0 conn(0x7f431409f0e0 0x7f431409e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f4311d9b640 1 --2- 172.21.15.204:0/730917458 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f4b0 0x7f4314132930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4308009880 tx=0x7f43080090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=305cf3440b5ad290 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f431159a640 1 -- 172.21.15.204:0/730917458 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f430802ac50 con 0x7f431409f4b0 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f431159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f431159a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f431159a640 10 monclient: dump: 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:28.779 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:28.780 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f431159a640 10 monclient: _finish_auth 0 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f431159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f431159a640 1 -- 172.21.15.204:0/730917458 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430802add0 con 0x7f431409f4b0 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f4318caf640 5 monclient: authenticate success, global_id 24568 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f4318caf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f431159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f4318caf640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f431159a640 1 -- 172.21.15.204:0/730917458 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4308031ca0 con 0x7f431409f4b0 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f4318caf640 4 set_mon_vals no callback set 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f431159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f431159a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:28.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.764+0000 7f431159a640 10 monclient: dump: 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:28.782 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.765+0000 7f4318caf640 10 monclient: shutdown 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.765+0000 7f4318caf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.765+0000 7f4318caf640 1 -- 172.21.15.204:0/730917458 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f4b0 msgr2=0x7f4314132930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.765+0000 7f4318caf640 1 --2- 172.21.15.204:0/730917458 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f4b0 0x7f4314132930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4308009880 tx=0x7f43080090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.765+0000 7f4318caf640 1 -- 172.21.15.204:0/730917458 shutdown_connections 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.765+0000 7f4318caf640 1 --2- 172.21.15.204:0/730917458 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f4b0 0x7f4314132930 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.765+0000 7f4318caf640 1 --2- 172.21.15.204:0/730917458 >> v2:172.21.15.204:3300/0 conn(0x7f431409f0e0 0x7f431409e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.765+0000 7f4318caf640 1 -- 172.21.15.204:0/730917458 >> 172.21.15.204:0/730917458 conn(0x7f4314067c00 msgr2=0x7f43140a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.765+0000 7f4318caf640 1 -- 172.21.15.204:0/730917458 shutdown_connections 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.765+0000 7f4318caf640 1 -- 172.21.15.204:0/730917458 wait complete. 2024-09-15T01:37:28.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.765+0000 7f4318caf640 10 monclient: build_initial_monmap 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.765+0000 7f4318caf640 10 monclient: monmap: 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:28.784 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 1 Processor -- start 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 1 -- start start 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 10 monclient: init 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 10 monclient: _add_conns ranks=[5,1,7,8,4,2,6,0,3] 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f431409f0e0 0x7f4314142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 10 monclient(hunting): picked mon.noname-h con 0x7f431409f0e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f431409f4b0 0x7f4314132a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 10 monclient(hunting): picked mon.noname-f con 0x7f431409f4b0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:28.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f880 0x7f431411c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 10 monclient(hunting): picked mon.noname-i con 0x7f431409f880 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431411fb20 con 0x7f431409f4b0 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4314068920 con 0x7f431409f0e0 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f431411e500 con 0x7f431409f880 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4318caf640 10 monclient(hunting): authenticate will time out at 1064.409789s 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4311d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f431409f4b0 0x7f4314132a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4312d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f880 0x7f431411c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f431259c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f431409f0e0 0x7f4314142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4311d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f431409f4b0 0x7f4314132a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36558/0 (socket says 172.21.15.204:36558) 2024-09-15T01:37:28.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4311d9b640 1 -- 172.21.15.204:0/3375926019 learned_addr learned my addr 172.21.15.204:0/3375926019 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4312d9d640 10 monclient(hunting): get_auth_request con 0x7f431409f880 auth_method 0 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f431259c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f431409f0e0 0x7f4314142120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:54454/0 (socket says 172.21.15.204:54454) 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4312d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4312d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4312d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4311d9b640 10 monclient(hunting): get_auth_request con 0x7f431409f4b0 auth_method 0 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4311d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4311d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4311d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f431259c640 10 monclient(hunting): get_auth_request con 0x7f431409f0e0 auth_method 0 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f431259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f431259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:28.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f431259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4312d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4312d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4312d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4311d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.766+0000 7f4311d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4311d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f431259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f431259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f431259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4312d9d640 10 monclient(hunting): handle_auth_done global_id 24577 payload 824 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4312d9d640 1 -- 172.21.15.204:0/3375926019 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f431409f0e0 msgr2=0x7f4314142120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4312d9d640 1 --2- 172.21.15.204:0/3375926019 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f431409f0e0 0x7f4314142120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:28.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4312d9d640 1 -- 172.21.15.204:0/3375926019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f431409f4b0 msgr2=0x7f4314132a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4312d9d640 1 --2- 172.21.15.204:0/3375926019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f431409f4b0 0x7f4314132a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4312d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4312d9d640 1 monclient: found mon.noname-i 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4312d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4312d9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4312d9d640 1 -- 172.21.15.204:0/3375926019 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4304009450 con 0x7f431409f880 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4311d9b640 1 --2- 172.21.15.204:0/3375926019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f431409f4b0 0x7f4314132a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4312d9d640 1 --2- 172.21.15.204:0/3375926019 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f880 0x7f431411c6d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c710 tx=0x7f430c001560 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f43027fc640 1 -- 172.21.15.204:0/3375926019 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f430c005d60 con 0x7f431409f880 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f43027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f43027fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f43027fc640 10 monclient: dump: 2024-09-15T01:37:28.789 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:28.790 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f43027fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f43027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f43027fc640 1 -- 172.21.15.204:0/3375926019 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c007570 con 0x7f431409f880 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4318caf640 5 monclient: authenticate success, global_id 24577 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4318caf640 10 monclient: _renew_subs 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4318caf640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4318caf640 1 -- 172.21.15.204:0/3375926019 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4308009390 con 0x7f431409f880 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f43027fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f43027fc640 1 -- 172.21.15.204:0/3375926019 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f430c012400 con 0x7f431409f880 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f43027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f43027fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:28.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f43027fc640 10 monclient: dump: 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:28.792 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:28.793 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4318caf640 10 monclient: _renew_subs 2024-09-15T01:37:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4310d99640 4 set_mon_vals no callback set 2024-09-15T01:37:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4318caf640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.767+0000 7f4318caf640 1 -- 172.21.15.204:0/3375926019 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431411dff0 con 0x7f431409f880 2024-09-15T01:37:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.768+0000 7f43027fc640 1 -- 172.21.15.204:0/3375926019 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f430c0076e0 con 0x7f431409f880 2024-09-15T01:37:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.768+0000 7f4318caf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.768+0000 7f4318caf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.768+0000 7f4318caf640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:28.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.768+0000 7f4318caf640 1 -- 172.21.15.204:0/3375926019 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4314070b80 con 0x7f431409f880 2024-09-15T01:37:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.768+0000 7f43027fc640 1 --2- 172.21.15.204:0/3375926019 >> v2:172.21.15.204:6812/28210 conn(0x7f42f4039710 0x7f42f403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.768+0000 7f43027fc640 1 -- 172.21.15.204:0/3375926019 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 13061+0+0 (secure 0 0 0) 0x7f430c04d070 con 0x7f431409f880 2024-09-15T01:37:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.768+0000 7f431259c640 1 --2- 172.21.15.204:0/3375926019 >> v2:172.21.15.204:6812/28210 conn(0x7f42f4039710 0x7f42f403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.768+0000 7f431259c640 10 monclient: get_auth_request con 0x7f42f4039710 auth_method 0 2024-09-15T01:37:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.769+0000 7f431259c640 1 --2- 172.21.15.204:0/3375926019 >> v2:172.21.15.204:6812/28210 conn(0x7f42f4039710 0x7f42f403bbc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4304006c90 tx=0x7f43040091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.773+0000 7f43027fc640 1 -- 172.21.15.204:0/3375926019 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f430c007380 con 0x7f431409f880 2024-09-15T01:37:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.773+0000 7f43027fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:28.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.773+0000 7f43027fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:28.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.930+0000 7f4318caf640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:28.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.930+0000 7f4318caf640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:28.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.930+0000 7f4318caf640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:28.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:28.930+0000 7f4318caf640 1 -- 172.21.15.204:0/3375926019 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f431409ec90 con 0x7f431409f880 2024-09-15T01:37:29.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.577+0000 7f43027fc640 1 -- 172.21.15.204:0/3375926019 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}]=0 added bucket h28 type host to location {rack=r1} v213) v1 ==== 168+0+0 (secure 0 0 0) 0x7f430c00a370 con 0x7f431409f880 2024-09-15T01:37:29.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.577+0000 7f43027fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:29.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.577+0000 7f43027fc640 10 monclient: _finish_command 2 = system:0 added bucket h28 type host to location {rack=r1} 2024-09-15T01:37:29.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.675+0000 7f4318caf640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:29.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.675+0000 7f4318caf640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:29.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.675+0000 7f4318caf640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:29.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.675+0000 7f4318caf640 1 -- 172.21.15.204:0/3375926019 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f431406a050 con 0x7f431409f880 2024-09-15T01:37:29.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.676+0000 7f43027fc640 1 -- 172.21.15.204:0/3375926019 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h28' already exists v213) v1 ==== 147+0+0 (secure 0 0 0) 0x7f430c010af0 con 0x7f431409f880 2024-09-15T01:37:29.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.676+0000 7f43027fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:29.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.676+0000 7f43027fc640 10 monclient: _finish_command 3 = system:0 bucket 'h28' already exists 2024-09-15T01:37:29.679 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h28' already exists 2024-09-15T01:37:29.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 1 -- 172.21.15.204:0/3375926019 >> v2:172.21.15.204:6812/28210 conn(0x7f42f4039710 msgr2=0x7f42f403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:29.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 1 --2- 172.21.15.204:0/3375926019 >> v2:172.21.15.204:6812/28210 conn(0x7f42f4039710 0x7f42f403bbc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4304006c90 tx=0x7f43040091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 10 monclient: shutdown 2024-09-15T01:37:29.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:29.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 1 -- 172.21.15.204:0/3375926019 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f880 msgr2=0x7f431411c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:29.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 1 --2- 172.21.15.204:0/3375926019 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f880 0x7f431411c6d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c710 tx=0x7f430c001560 comp rx=0 tx=0).stop 2024-09-15T01:37:29.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 1 -- 172.21.15.204:0/3375926019 shutdown_connections 2024-09-15T01:37:29.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 1 --2- 172.21.15.204:0/3375926019 >> v2:172.21.15.204:6812/28210 conn(0x7f42f4039710 0x7f42f403bbc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 1 --2- 172.21.15.204:0/3375926019 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f431409f880 0x7f431411c6d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 1 --2- 172.21.15.204:0/3375926019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f431409f4b0 0x7f4314132a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 1 --2- 172.21.15.204:0/3375926019 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f431409f0e0 0x7f4314142120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 1 -- 172.21.15.204:0/3375926019 >> 172.21.15.204:0/3375926019 conn(0x7f4314067c00 msgr2=0x7f431406e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:29.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 1 -- 172.21.15.204:0/3375926019 shutdown_connections 2024-09-15T01:37:29.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.678+0000 7f4318caf640 1 -- 172.21.15.204:0/3375926019 wait complete. 2024-09-15T01:37:29.689 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:29.689 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h29 host rack=r1 2024-09-15T01:37:29.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.763+0000 7f24ebb9d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:29.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.763+0000 7f24ebb9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:29.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.763+0000 7f24ebb9d640 10 monclient: monmap: 2024-09-15T01:37:29.766 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:29.766 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:29.766 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:29.764667+0000 2024-09-15T01:37:29.766 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:29.764667+0000 2024-09-15T01:37:29.766 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:29.766 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:29.766 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:29.766 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:29.767 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:29.767 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:29.767 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:29.767 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:29.767 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:29.767 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:29.767 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:29.767 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:29.767 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:29.767 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:29.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.763+0000 7f24ebb9d640 1 Processor -- start 2024-09-15T01:37:29.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 1 -- start start 2024-09-15T01:37:29.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 10 monclient: init 2024-09-15T01:37:29.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:29.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 10 monclient: _add_conns ranks=[10,2,6,1,5,4,3,9,7,0,8] 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f24e4067130 0x7f24e4097c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 10 monclient(hunting): picked mon.noname-f con 0x7f24e4067130 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24e4067500 0x7f24e40678d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 10 monclient(hunting): picked mon.noname-b con 0x7f24e4067500 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f24e4136e80 0x7f24e4139260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 10 monclient(hunting): picked mon.noname-d con 0x7f24e4136e80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e4097820 con 0x7f24e4067130 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e4096ef0 con 0x7f24e4067500 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40971d0 con 0x7f24e4136e80 2024-09-15T01:37:29.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ebb9d640 10 monclient(hunting): authenticate will time out at 1065.407578s 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f24e4067130 0x7f24e4097c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9138640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24e4067500 msgr2=0x7f24e40678d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9138640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24e4067500 0x7f24e40678d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f24e4067130 0x7f24e4097c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36562/0 (socket says 172.21.15.204:36562) 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 1 -- 172.21.15.204:0/3421770316 learned_addr learned my addr 172.21.15.204:0/3421770316 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 10 monclient(hunting): get_auth_request con 0x7f24e4067130 auth_method 0 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:29.778 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:29.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 10 monclient(hunting): handle_auth_done global_id 7147 payload 274 2024-09-15T01:37:29.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24ea13a640 1 --2- 172.21.15.204:0/3421770316 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f24e4136e80 0x7f24e4139260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:29.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 1 -- 172.21.15.204:0/3421770316 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f24e4136e80 msgr2=0x7f24e4139260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:29.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 1 --2- 172.21.15.204:0/3421770316 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f24e4136e80 0x7f24e4139260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 1 -- 172.21.15.204:0/3421770316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24e4067500 msgr2=0x7f24e40678d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:29.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 1 --2- 172.21.15.204:0/3421770316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24e4067500 0x7f24e40678d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 10 monclient: _finish_hunting 0 2024-09-15T01:37:29.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 1 monclient: found mon.noname-f 2024-09-15T01:37:29.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:29.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:29.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.764+0000 7f24e9939640 1 -- 172.21.15.204:0/3421770316 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e40648a0 con 0x7f24e4067130 2024-09-15T01:37:29.779 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e9939640 1 --2- 172.21.15.204:0/3421770316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e4067130 0x7f24e4097c20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009910 tx=0x7f24e00091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=940680041d657089 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e8937640 1 -- 172.21.15.204:0/3421770316 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f24e002ad40 con 0x7f24e4067130 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e8937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e8937640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e8937640 10 monclient: dump: 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:29.780 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:29.781 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:29.781 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:29.781 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:29.781 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:29.781 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:29.781 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:29.781 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:29.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e8937640 10 monclient: _finish_auth 0 2024-09-15T01:37:29.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e8937640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:29.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24ebb9d640 5 monclient: authenticate success, global_id 7147 2024-09-15T01:37:29.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24ebb9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:29.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e8937640 1 -- 172.21.15.204:0/3421770316 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0031820 con 0x7f24e4067130 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e8937640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24ebb9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24ebb9d640 4 set_mon_vals no callback set 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e8937640 1 -- 172.21.15.204:0/3421770316 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f24e0031e90 con 0x7f24e4067130 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e8937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e8937640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.765+0000 7f24e8937640 10 monclient: dump: 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:29.782 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:29.783 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:29.783 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:29.783 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:29.783 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:29.783 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:29.783 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:29.783 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:29.783 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:29.783 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:29.783 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:29.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 10 monclient: shutdown 2024-09-15T01:37:29.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 1 -- 172.21.15.204:0/3421770316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e4067130 msgr2=0x7f24e4097c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 1 --2- 172.21.15.204:0/3421770316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e4067130 0x7f24e4097c20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009910 tx=0x7f24e00091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 1 -- 172.21.15.204:0/3421770316 shutdown_connections 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 1 --2- 172.21.15.204:0/3421770316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24e4067500 0x7f24e40678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 1 --2- 172.21.15.204:0/3421770316 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f24e4136e80 0x7f24e4139260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 1 --2- 172.21.15.204:0/3421770316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24e4067130 0x7f24e4097c20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 1 -- 172.21.15.204:0/3421770316 >> 172.21.15.204:0/3421770316 conn(0x7f24e40a2ce0 msgr2=0x7f24e40a0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 1 -- 172.21.15.204:0/3421770316 shutdown_connections 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 1 -- 172.21.15.204:0/3421770316 wait complete. 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.766+0000 7f24ebb9d640 10 monclient: monmap: 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:29.784 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:29.785 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 1 Processor -- start 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 1 -- start start 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 10 monclient: init 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 10 monclient: _add_conns ranks=[8,6,4,7,3,0,1,2,5] 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f24e4067130 0x7f24e406dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 10 monclient(hunting): picked mon.noname-e con 0x7f24e4067130 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f24e4067500 0x7f24e406e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 10 monclient(hunting): picked mon.noname-d con 0x7f24e4067500 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f24e4136e80 0x7f24e406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 10 monclient(hunting): picked mon.noname-c con 0x7f24e4136e80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:29.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24e411edc0 con 0x7f24e4136e80 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f24e411cf40 con 0x7f24e4067500 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40987c0 con 0x7f24e4067130 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ebb9d640 10 monclient(hunting): authenticate will time out at 1065.410802s 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ea13a640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f24e4136e80 msgr2=0x7f24e406ea40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24e9939640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f24e4067130 msgr2=0x7f24e406dfc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24e9939640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f24e4067130 0x7f24e406dfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.767+0000 7f24ea13a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f24e4136e80 0x7f24e406ea40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f24e4067500 0x7f24e406e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:29.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f24e4067500 0x7f24e406e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:36362/0 (socket says 172.21.15.204:36362) 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 1 -- 172.21.15.204:0/1738051736 learned_addr learned my addr 172.21.15.204:0/1738051736 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 10 monclient(hunting): get_auth_request con 0x7f24e4067500 auth_method 0 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 10 monclient(hunting): handle_auth_done global_id 24468 payload 824 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 1 -- 172.21.15.204:0/1738051736 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f24e4067130 msgr2=0x7f24e406dfc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 1 --2- 172.21.15.204:0/1738051736 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f24e4067130 0x7f24e406dfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 1 -- 172.21.15.204:0/1738051736 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f24e4136e80 msgr2=0x7f24e406ea40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:29.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 1 --2- 172.21.15.204:0/1738051736 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f24e4136e80 0x7f24e406ea40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 10 monclient: _finish_hunting 0 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 1 monclient: found mon.noname-d 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 1 -- 172.21.15.204:0/1738051736 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e0009450 con 0x7f24e4067500 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24e9138640 1 --2- 172.21.15.204:0/1738051736 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f24e4067500 0x7f24e406e500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f24d400b5e0 tx=0x7f24d4006f90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24d9ffb640 1 -- 172.21.15.204:0/1738051736 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f24d4006150 con 0x7f24e4067500 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24d9ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24d9ffb640 10 monclient: dump: 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:29.789 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:29.790 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:29.790 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:29.790 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:29.790 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:29.790 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:29.790 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:29.790 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:29.790 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:29.790 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:29.790 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:29.790 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:29.790 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24d9ffb640 10 monclient: _finish_auth 0 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24d9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24d9ffb640 1 -- 172.21.15.204:0/1738051736 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d40062f0 con 0x7f24e4067500 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24ebb9d640 5 monclient: authenticate success, global_id 24468 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24d9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24ebb9d640 10 monclient: _renew_subs 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24d9ffb640 1 -- 172.21.15.204:0/1738051736 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f24d4006460 con 0x7f24e4067500 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24ebb9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24ebb9d640 1 -- 172.21.15.204:0/1738051736 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e4098590 con 0x7f24e4067500 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24d9ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:37:29.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24dbfff640 4 set_mon_vals no callback set 2024-09-15T01:37:29.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.768+0000 7f24d9ffb640 10 monclient: dump: 2024-09-15T01:37:29.792 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:29.792 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:29.792 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:29.792 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:29.792 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:29.792 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:29.792 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:29.792 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:29.792 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:29.792 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:29.792 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.769+0000 7f24ebb9d640 10 monclient: _renew_subs 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.769+0000 7f24ebb9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.769+0000 7f24ebb9d640 1 -- 172.21.15.204:0/1738051736 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e411da80 con 0x7f24e4067500 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.769+0000 7f24d9ffb640 1 -- 172.21.15.204:0/1738051736 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f24d40065c0 con 0x7f24e4067500 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.769+0000 7f24d9ffb640 1 --2- 172.21.15.204:0/1738051736 >> v2:172.21.15.204:6812/28210 conn(0x7f24c80397e0 0x7f24c803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.769+0000 7f24ebb9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.769+0000 7f24ebb9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.769+0000 7f24ebb9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:29.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.769+0000 7f24d9ffb640 1 -- 172.21.15.204:0/1738051736 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 13274+0+0 (secure 0 0 0) 0x7f24d404c9a0 con 0x7f24e4067500 2024-09-15T01:37:29.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.769+0000 7f24ebb9d640 1 -- 172.21.15.204:0/1738051736 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e4070b80 con 0x7f24e4067500 2024-09-15T01:37:29.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.770+0000 7f24e9939640 1 --2- 172.21.15.204:0/1738051736 >> v2:172.21.15.204:6812/28210 conn(0x7f24c80397e0 0x7f24c803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:29.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.770+0000 7f24e9939640 10 monclient: get_auth_request con 0x7f24c80397e0 auth_method 0 2024-09-15T01:37:29.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.770+0000 7f24e9939640 1 --2- 172.21.15.204:0/1738051736 >> v2:172.21.15.204:6812/28210 conn(0x7f24c80397e0 0x7f24c803bc90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f24e0034000 tx=0x7f24e0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:29.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.775+0000 7f24d9ffb640 1 -- 172.21.15.204:0/1738051736 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f24d4017030 con 0x7f24e4067500 2024-09-15T01:37:29.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.775+0000 7f24d9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:29.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.775+0000 7f24d9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:29.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.934+0000 7f24ebb9d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:29.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.934+0000 7f24ebb9d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:29.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.934+0000 7f24ebb9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:29.937 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:29.934+0000 7f24ebb9d640 1 -- 172.21.15.204:0/1738051736 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f24e40a0db0 con 0x7f24e4067500 2024-09-15T01:37:30.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.593+0000 7f24d9ffb640 1 -- 172.21.15.204:0/1738051736 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}]=0 added bucket h29 type host to location {rack=r1} v214) v1 ==== 168+0+0 (secure 0 0 0) 0x7f24d4017030 con 0x7f24e4067500 2024-09-15T01:37:30.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.593+0000 7f24d9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:30.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.593+0000 7f24d9ffb640 10 monclient: _finish_command 2 = system:0 added bucket h29 type host to location {rack=r1} 2024-09-15T01:37:30.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.689+0000 7f24ebb9d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:30.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.689+0000 7f24ebb9d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:30.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.689+0000 7f24ebb9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:30.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.689+0000 7f24ebb9d640 1 -- 172.21.15.204:0/1738051736 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f24e411f170 con 0x7f24e4067500 2024-09-15T01:37:30.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.690+0000 7f24d9ffb640 1 -- 172.21.15.204:0/1738051736 <== mon.6 v2:172.21.15.178:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h29' already exists v214) v1 ==== 147+0+0 (secure 0 0 0) 0x7f24d4017030 con 0x7f24e4067500 2024-09-15T01:37:30.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.690+0000 7f24d9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:30.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.690+0000 7f24d9ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h29' already exists 2024-09-15T01:37:30.693 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h29' already exists 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 1 -- 172.21.15.204:0/1738051736 >> v2:172.21.15.204:6812/28210 conn(0x7f24c80397e0 msgr2=0x7f24c803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 1 --2- 172.21.15.204:0/1738051736 >> v2:172.21.15.204:6812/28210 conn(0x7f24c80397e0 0x7f24c803bc90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f24e0034000 tx=0x7f24e0032000 comp rx=0 tx=0).stop 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 10 monclient: shutdown 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 1 -- 172.21.15.204:0/1738051736 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f24e4067500 msgr2=0x7f24e406e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 1 --2- 172.21.15.204:0/1738051736 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f24e4067500 0x7f24e406e500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f24d400b5e0 tx=0x7f24d4006f90 comp rx=0 tx=0).stop 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 1 -- 172.21.15.204:0/1738051736 shutdown_connections 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 1 --2- 172.21.15.204:0/1738051736 >> v2:172.21.15.204:6812/28210 conn(0x7f24c80397e0 0x7f24c803bc90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 1 --2- 172.21.15.204:0/1738051736 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f24e4136e80 0x7f24e406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 1 --2- 172.21.15.204:0/1738051736 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f24e4067500 0x7f24e406e500 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 1 --2- 172.21.15.204:0/1738051736 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f24e4067130 0x7f24e406dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 1 -- 172.21.15.204:0/1738051736 >> 172.21.15.204:0/1738051736 conn(0x7f24e40a2ce0 msgr2=0x7f24e41366a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:30.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 1 -- 172.21.15.204:0/1738051736 shutdown_connections 2024-09-15T01:37:30.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.692+0000 7f24ebb9d640 1 -- 172.21.15.204:0/1738051736 wait complete. 2024-09-15T01:37:30.703 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:30.703 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h30 host rack=r1 2024-09-15T01:37:30.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.777+0000 7f627935d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:30.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.777+0000 7f627935d640 10 monclient: build_initial_monmap 2024-09-15T01:37:30.780 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.777+0000 7f627935d640 10 monclient: monmap: 2024-09-15T01:37:30.780 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:30.780 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:30.780 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:30.778925+0000 2024-09-15T01:37:30.780 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:30.778925+0000 2024-09-15T01:37:30.780 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:30.780 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:30.780 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:30.780 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:30.781 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:30.781 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:30.781 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:30.781 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:30.781 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:30.781 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:30.781 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:30.781 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:30.781 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:30.781 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:30.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 1 Processor -- start 2024-09-15T01:37:30.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 1 -- start start 2024-09-15T01:37:30.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 10 monclient: init 2024-09-15T01:37:30.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:30.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 10 monclient: _add_conns ranks=[0,4,3,6,7,1,9,2,5,10,8] 2024-09-15T01:37:30.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f62740a2cd0 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:30.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f627409fa20 0x7f627409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:30.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 10 monclient(hunting): picked mon.noname-c con 0x7f627409fa20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:30.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6274137c80 0x7f627413a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:30.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 10 monclient(hunting): picked mon.noname-g con 0x7f6274137c80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:30.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62740648a0 con 0x7f62740a2cd0 2024-09-15T01:37:30.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:30.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6274133310 con 0x7f6274137c80 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62740a30a0 con 0x7f627409fa20 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f627935d640 10 monclient(hunting): authenticate will time out at 1066.421702s 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f62737fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6274137c80 0x7f627413a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f6272ffd640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f62740a2cd0 legacy=0x7f6274132c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f626bfff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f627409fa20 msgr2=0x7f627409fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f62737fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6274137c80 0x7f627413a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:52690/0 (socket says 172.21.15.204:52690) 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f62737fe640 1 -- 172.21.15.204:0/508405406 learned_addr learned my addr 172.21.15.204:0/508405406 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f62737fe640 10 monclient(hunting): get_auth_request con 0x7f6274137c80 auth_method 0 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f62737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f62737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f62737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:30.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f626bfff640 1 --2- 172.21.15.204:0/508405406 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f627409fa20 0x7f627409fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f62737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f62737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.778+0000 7f62737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62737fe640 10 monclient(hunting): handle_auth_done global_id 15510 payload 274 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62737fe640 1 -- 172.21.15.204:0/508405406 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f627409fa20 msgr2=0x7f627409fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62737fe640 1 --2- 172.21.15.204:0/508405406 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f627409fa20 0x7f627409fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62737fe640 1 -- 172.21.15.204:0/508405406 >> v1:172.21.15.178:6789/0 conn(0x7f62740a2cd0 legacy=0x7f6274132c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62737fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62737fe640 1 monclient: found mon.noname-g 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62737fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62737fe640 1 -- 172.21.15.204:0/508405406 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62740a0330 con 0x7f6274137c80 2024-09-15T01:37:30.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62737fe640 1 --2- 172.21.15.204:0/508405406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6274137c80 0x7f627413a060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f626c009f50 tx=0x7f626c009840 comp rx=0 tx=0).ready entity=mon.3 client_cookie=fe0d4006379bac1c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62727fc640 1 -- 172.21.15.204:0/508405406 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f626c007880 con 0x7f6274137c80 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62727fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62727fc640 10 monclient: dump: 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:30.797 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62727fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62727fc640 1 -- 172.21.15.204:0/508405406 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626c02d050 con 0x7f6274137c80 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f627935d640 5 monclient: authenticate success, global_id 15510 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f627935d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62727fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f627935d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:30.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62727fc640 1 -- 172.21.15.204:0/508405406 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f626c030360 con 0x7f6274137c80 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f627935d640 4 set_mon_vals no callback set 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62727fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.779+0000 7f62727fc640 10 monclient: dump: 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:30.799 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.780+0000 7f627935d640 10 monclient: shutdown 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.780+0000 7f627935d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.780+0000 7f627935d640 1 -- 172.21.15.204:0/508405406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6274137c80 msgr2=0x7f627413a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.780+0000 7f627935d640 1 --2- 172.21.15.204:0/508405406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6274137c80 0x7f627413a060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f626c009f50 tx=0x7f626c009840 comp rx=0 tx=0).stop 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.780+0000 7f627935d640 1 -- 172.21.15.204:0/508405406 shutdown_connections 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.780+0000 7f627935d640 1 --2- 172.21.15.204:0/508405406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6274137c80 0x7f627413a060 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:30.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.780+0000 7f627935d640 1 --2- 172.21.15.204:0/508405406 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f627409fa20 0x7f627409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.780+0000 7f627935d640 1 -- 172.21.15.204:0/508405406 >> 172.21.15.204:0/508405406 conn(0x7f627406e860 msgr2=0x7f627406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.780+0000 7f627935d640 1 -- 172.21.15.204:0/508405406 shutdown_connections 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.780+0000 7f627935d640 1 -- 172.21.15.204:0/508405406 wait complete. 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.780+0000 7f627935d640 10 monclient: build_initial_monmap 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 10 monclient: monmap: 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:30.801 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:30.802 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:30.802 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:30.802 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:30.802 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:30.802 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:30.802 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:30.802 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:30.802 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:30.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 1 Processor -- start 2024-09-15T01:37:30.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 1 -- start start 2024-09-15T01:37:30.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 10 monclient: init 2024-09-15T01:37:30.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:30.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 10 monclient: _add_conns ranks=[4,1,7,3,5,8,6,2,0] 2024-09-15T01:37:30.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f627409fa20 0x7f6274130470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:30.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 10 monclient(hunting): picked mon.noname-c con 0x7f627409fa20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:30.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62740a2cd0 0x7f62741309b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:30.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 10 monclient(hunting): picked mon.noname-f con 0x7f62740a2cd0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:30.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6274137c80 0x7f6274130ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:30.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 10 monclient(hunting): picked mon.noname-i con 0x7f6274137c80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:30.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:30.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6274134b20 con 0x7f62740a2cd0 2024-09-15T01:37:30.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:30.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f627409cb60 con 0x7f627409fa20 2024-09-15T01:37:30.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:30.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6274135a10 con 0x7f6274137c80 2024-09-15T01:37:30.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:30.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f627935d640 10 monclient(hunting): authenticate will time out at 1066.424813s 2024-09-15T01:37:30.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f6272ffd640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f627409fa20 msgr2=0x7f6274130470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:30.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f62737fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6274137c80 0x7f6274130ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:30.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f626bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62740a2cd0 0x7f62741309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:30.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f6272ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f627409fa20 0x7f6274130470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:30.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f62737fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6274137c80 0x7f6274130ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:33486/0 (socket says 172.21.15.204:33486) 2024-09-15T01:37:30.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f62737fe640 1 -- 172.21.15.204:0/2237099377 learned_addr learned my addr 172.21.15.204:0/2237099377 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:30.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f62737fe640 10 monclient(hunting): get_auth_request con 0x7f6274137c80 auth_method 0 2024-09-15T01:37:30.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f626bfff640 1 --2- 172.21.15.204:0/2237099377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62740a2cd0 0x7f62741309b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36568/0 (socket says 172.21.15.204:36568) 2024-09-15T01:37:30.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f62737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:30.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f62737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:30.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f62737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:30.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f626bfff640 10 monclient(hunting): get_auth_request con 0x7f62740a2cd0 auth_method 0 2024-09-15T01:37:30.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f626bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:30.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f626bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:30.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.781+0000 7f626bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:30.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f62737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:30.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f62737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:30.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f62737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:30.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:30.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:30.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:30.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 10 monclient(hunting): handle_auth_done global_id 7156 payload 824 2024-09-15T01:37:30.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 1 -- 172.21.15.204:0/2237099377 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6274137c80 msgr2=0x7f6274130ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:30.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 1 --2- 172.21.15.204:0/2237099377 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6274137c80 0x7f6274130ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:30.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 1 -- 172.21.15.204:0/2237099377 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f627409fa20 msgr2=0x7f6274130470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:30.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 1 --2- 172.21.15.204:0/2237099377 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f627409fa20 0x7f6274130470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:30.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:30.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 1 monclient: found mon.noname-f 2024-09-15T01:37:30.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:30.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:30.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 1 -- 172.21.15.204:0/2237099377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c009ae0 con 0x7f62740a2cd0 2024-09-15T01:37:30.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f62737fe640 1 --2- 172.21.15.204:0/2237099377 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6274137c80 0x7f6274130ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:30.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626bfff640 1 --2- 172.21.15.204:0/2237099377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62740a2cd0 0x7f62741309b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f625c00b500 tx=0x7f625c006f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:30.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626affd640 1 -- 172.21.15.204:0/2237099377 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f625c0050f0 con 0x7f62740a2cd0 2024-09-15T01:37:30.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:30.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626affd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:30.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626affd640 10 monclient: dump: 2024-09-15T01:37:30.811 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:30.811 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:30.811 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:30.811 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:30.811 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:30.812 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:30.812 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:30.812 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:30.812 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:30.812 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:30.812 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:30.812 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:30.812 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:30.812 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:30.812 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:30.813 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:30.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626affd640 10 monclient: _finish_auth 0 2024-09-15T01:37:30.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:30.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626affd640 1 -- 172.21.15.204:0/2237099377 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c005290 con 0x7f62740a2cd0 2024-09-15T01:37:30.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:30.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626affd640 1 -- 172.21.15.204:0/2237099377 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f625c011400 con 0x7f62740a2cd0 2024-09-15T01:37:30.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f627935d640 5 monclient: authenticate success, global_id 7156 2024-09-15T01:37:30.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f627935d640 10 monclient: _renew_subs 2024-09-15T01:37:30.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f627935d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:30.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f627935d640 1 -- 172.21.15.204:0/2237099377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6274131d00 con 0x7f62740a2cd0 2024-09-15T01:37:30.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:30.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f6271ffb640 4 set_mon_vals no callback set 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f626affd640 10 monclient: dump: 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:30.814 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f627935d640 10 monclient: _renew_subs 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f627935d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.782+0000 7f627935d640 1 -- 172.21.15.204:0/2237099377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62741338e0 con 0x7f62740a2cd0 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.783+0000 7f626affd640 1 -- 172.21.15.204:0/2237099377 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f625c005440 con 0x7f62740a2cd0 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.783+0000 7f627935d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.783+0000 7f627935d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.783+0000 7f627935d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.783+0000 7f627935d640 1 -- 172.21.15.204:0/2237099377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6274134d60 con 0x7f62740a2cd0 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.783+0000 7f626affd640 1 --2- 172.21.15.204:0/2237099377 >> v2:172.21.15.204:6812/28210 conn(0x7f62540397e0 0x7f625403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.783+0000 7f626affd640 1 -- 172.21.15.204:0/2237099377 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 13487+0+0 (secure 0 0 0) 0x7f625c04cfc0 con 0x7f62740a2cd0 2024-09-15T01:37:30.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.783+0000 7f6272ffd640 1 --2- 172.21.15.204:0/2237099377 >> v2:172.21.15.204:6812/28210 conn(0x7f62540397e0 0x7f625403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:30.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.783+0000 7f6272ffd640 10 monclient: get_auth_request con 0x7f62540397e0 auth_method 0 2024-09-15T01:37:30.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.784+0000 7f6272ffd640 1 --2- 172.21.15.204:0/2237099377 >> v2:172.21.15.204:6812/28210 conn(0x7f62540397e0 0x7f625403bc90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f62640038f0 tx=0x7f6264006ce0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:30.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.788+0000 7f626affd640 1 -- 172.21.15.204:0/2237099377 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f625c003d40 con 0x7f62740a2cd0 2024-09-15T01:37:30.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.788+0000 7f626affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:30.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.788+0000 7f626affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:30.947 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.945+0000 7f627935d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:30.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.945+0000 7f627935d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:30.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.945+0000 7f627935d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:30.948 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:30.945+0000 7f627935d640 1 -- 172.21.15.204:0/2237099377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f6274070b80 con 0x7f62740a2cd0 2024-09-15T01:37:31.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.470+0000 7f626affd640 1 -- 172.21.15.204:0/2237099377 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}]=0 added bucket h30 type host to location {rack=r1} v215) v1 ==== 168+0+0 (secure 0 0 0) 0x7f625c018300 con 0x7f62740a2cd0 2024-09-15T01:37:31.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.470+0000 7f626affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:31.473 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.470+0000 7f626affd640 10 monclient: _finish_command 2 = system:0 added bucket h30 type host to location {rack=r1} 2024-09-15T01:37:31.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.573+0000 7f627935d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:31.576 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.573+0000 7f627935d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:31.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.573+0000 7f627935d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:31.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.573+0000 7f627935d640 1 -- 172.21.15.204:0/2237099377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f6274132610 con 0x7f62740a2cd0 2024-09-15T01:37:31.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.574+0000 7f626affd640 1 -- 172.21.15.204:0/2237099377 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h30' already exists v215) v1 ==== 147+0+0 (secure 0 0 0) 0x7f625c00faa0 con 0x7f62740a2cd0 2024-09-15T01:37:31.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.574+0000 7f626affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:31.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.574+0000 7f626affd640 10 monclient: _finish_command 3 = system:0 bucket 'h30' already exists 2024-09-15T01:37:31.577 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h30' already exists 2024-09-15T01:37:31.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 1 -- 172.21.15.204:0/2237099377 >> v2:172.21.15.204:6812/28210 conn(0x7f62540397e0 msgr2=0x7f625403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:31.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 1 --2- 172.21.15.204:0/2237099377 >> v2:172.21.15.204:6812/28210 conn(0x7f62540397e0 0x7f625403bc90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f62640038f0 tx=0x7f6264006ce0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 10 monclient: shutdown 2024-09-15T01:37:31.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:31.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 1 -- 172.21.15.204:0/2237099377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62740a2cd0 msgr2=0x7f62741309b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:31.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 1 --2- 172.21.15.204:0/2237099377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62740a2cd0 0x7f62741309b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f625c00b500 tx=0x7f625c006f90 comp rx=0 tx=0).stop 2024-09-15T01:37:31.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 1 -- 172.21.15.204:0/2237099377 shutdown_connections 2024-09-15T01:37:31.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 1 --2- 172.21.15.204:0/2237099377 >> v2:172.21.15.204:6812/28210 conn(0x7f62540397e0 0x7f625403bc90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 1 --2- 172.21.15.204:0/2237099377 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6274137c80 0x7f6274130ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 1 --2- 172.21.15.204:0/2237099377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62740a2cd0 0x7f62741309b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 1 --2- 172.21.15.204:0/2237099377 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f627409fa20 0x7f6274130470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.579 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 1 -- 172.21.15.204:0/2237099377 >> 172.21.15.204:0/2237099377 conn(0x7f627406e860 msgr2=0x7f627412dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:31.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 1 -- 172.21.15.204:0/2237099377 shutdown_connections 2024-09-15T01:37:31.580 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.576+0000 7f627935d640 1 -- 172.21.15.204:0/2237099377 wait complete. 2024-09-15T01:37:31.588 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:31.588 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h31 host rack=r1 2024-09-15T01:37:31.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.661+0000 7f9eb691d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:31.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.661+0000 7f9eb691d640 10 monclient: build_initial_monmap 2024-09-15T01:37:31.664 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.661+0000 7f9eb691d640 10 monclient: monmap: 2024-09-15T01:37:31.664 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:31.664 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:31.664 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:31.663222+0000 2024-09-15T01:37:31.664 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:31.663222+0000 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:31.665 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:31.666 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:31.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.662+0000 7f9eb691d640 1 Processor -- start 2024-09-15T01:37:31.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.662+0000 7f9eb691d640 1 -- start start 2024-09-15T01:37:31.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.662+0000 7f9eb691d640 10 monclient: init 2024-09-15T01:37:31.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.662+0000 7f9eb691d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:31.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 10 monclient: _add_conns ranks=[6,5,4,1,3,10,8,0,7,2,9] 2024-09-15T01:37:31.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9eb006e890 0x7f9eb0132cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:31.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 10 monclient(hunting): picked mon.noname-d con 0x7f9eb006e890 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:31.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9eb009fb10 0x7f9eb009ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:31.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 10 monclient(hunting): picked mon.noname-h con 0x7f9eb009fb10 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:31.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9eb00a09c0 0x7f9eb00a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:31.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 10 monclient(hunting): picked mon.noname-c con 0x7f9eb00a09c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:31.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb00a1d30 con 0x7f9eb00a09c0 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb006ec60 con 0x7f9eb009fb10 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb009e020 con 0x7f9eb006e890 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb691d640 10 monclient(hunting): authenticate will time out at 1067.306115s 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb4eba640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9eb00a09c0 msgr2=0x7f9eb00a0e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9eb009fb10 0x7f9eb009ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eb4eba640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9eb00a09c0 0x7f9eb00a0e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9eb009fb10 0x7f9eb009ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:54470/0 (socket says 172.21.15.204:54470) 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 1 -- 172.21.15.204:0/2269929529 learned_addr learned my addr 172.21.15.204:0/2269929529 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:31.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 10 monclient(hunting): get_auth_request con 0x7f9eb009fb10 auth_method 0 2024-09-15T01:37:31.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:31.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:31.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:31.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:31.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:31.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:31.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaffff640 1 --2- 172.21.15.204:0/2269929529 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9eb006e890 0x7f9eb0132cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:31.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 10 monclient(hunting): handle_auth_done global_id 6359 payload 274 2024-09-15T01:37:31.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 1 -- 172.21.15.204:0/2269929529 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9eb006e890 msgr2=0x7f9eb0132cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:31.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 1 --2- 172.21.15.204:0/2269929529 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9eb006e890 0x7f9eb0132cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 1 -- 172.21.15.204:0/2269929529 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9eb00a09c0 msgr2=0x7f9eb00a0e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:31.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 1 --2- 172.21.15.204:0/2269929529 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9eb00a09c0 0x7f9eb00a0e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:31.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 1 monclient: found mon.noname-h 2024-09-15T01:37:31.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:31.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:31.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaf7fe640 1 -- 172.21.15.204:0/2269929529 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb00648a0 con 0x7f9eb009fb10 2024-09-15T01:37:31.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaffff640 10 monclient: get_auth_request con 0x7f9eb006e890 auth_method 0 2024-09-15T01:37:31.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.663+0000 7f9eaffff640 1 --2- 172.21.15.204:0/2269929529 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9eb006e890 0x7f9eb0132cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:37:31.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaf7fe640 1 --2- 172.21.15.204:0/2269929529 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9eb009fb10 0x7f9eb009ff20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009970 tx=0x7f9ea40091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=66c930dd2a2ae88f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:31.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaeffd640 1 -- 172.21.15.204:0/2269929529 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ea402ac70 con 0x7f9eb009fb10 2024-09-15T01:37:31.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:31.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaeffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:31.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaeffd640 10 monclient: dump: 2024-09-15T01:37:31.679 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:31.679 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:31.679 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:31.679 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:31.680 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:31.680 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:31.680 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:31.680 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:31.680 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:31.680 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:31.680 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:31.680 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:31.681 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:31.681 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:31.681 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:31.681 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:31.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaeffd640 10 monclient: _finish_auth 0 2024-09-15T01:37:31.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:31.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaeffd640 1 -- 172.21.15.204:0/2269929529 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea402adf0 con 0x7f9eb009fb10 2024-09-15T01:37:31.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:31.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaeffd640 1 -- 172.21.15.204:0/2269929529 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ea4031ca0 con 0x7f9eb009fb10 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaeffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eaeffd640 10 monclient: dump: 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:31.682 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eb691d640 5 monclient: authenticate success, global_id 6359 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eb691d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eb691d640 4 set_mon_vals no callback set 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eb691d640 10 monclient: shutdown 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eb691d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eb691d640 1 -- 172.21.15.204:0/2269929529 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9eb009fb10 msgr2=0x7f9eb009ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:31.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.664+0000 7f9eb691d640 1 --2- 172.21.15.204:0/2269929529 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9eb009fb10 0x7f9eb009ff20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009970 tx=0x7f9ea40091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.665+0000 7f9eb691d640 1 -- 172.21.15.204:0/2269929529 shutdown_connections 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.665+0000 7f9eb691d640 1 --2- 172.21.15.204:0/2269929529 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9eb00a09c0 0x7f9eb00a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.665+0000 7f9eb691d640 1 --2- 172.21.15.204:0/2269929529 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9eb009fb10 0x7f9eb009ff20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.665+0000 7f9eb691d640 1 --2- 172.21.15.204:0/2269929529 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9eb006e890 0x7f9eb0132cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.665+0000 7f9eb691d640 1 -- 172.21.15.204:0/2269929529 >> 172.21.15.204:0/2269929529 conn(0x7f9eb00a5fd0 msgr2=0x7f9eb00a63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.665+0000 7f9eb691d640 1 -- 172.21.15.204:0/2269929529 shutdown_connections 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.665+0000 7f9eb691d640 1 -- 172.21.15.204:0/2269929529 wait complete. 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.665+0000 7f9eb691d640 10 monclient: build_initial_monmap 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.665+0000 7f9eb691d640 10 monclient: monmap: 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:31.684 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:31.685 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:31.685 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:31.685 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:31.685 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:31.685 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:31.685 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:31.686 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:31.686 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:31.686 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:31.686 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:31.686 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:31.686 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:31.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.665+0000 7f9eb691d640 1 Processor -- start 2024-09-15T01:37:31.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 1 -- start start 2024-09-15T01:37:31.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 10 monclient: init 2024-09-15T01:37:31.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:31.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 10 monclient: _add_conns ranks=[1,7,3,8,5,6,2,4,0] 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9eb006e890 0x7f9eb009b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 10 monclient(hunting): picked mon.noname-f con 0x7f9eb006e890 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9eb009fb10 0x7f9eb0131a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 10 monclient(hunting): picked mon.noname-i con 0x7f9eb009fb10 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9eb00a09c0 0x7f9eb0141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 10 monclient(hunting): picked mon.noname-g con 0x7f9eb00a09c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb009cc70 con 0x7f9eb006e890 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb011eb40 con 0x7f9eb00a09c0 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb011ee20 con 0x7f9eb009fb10 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:31.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb691d640 10 monclient(hunting): authenticate will time out at 1067.309439s 2024-09-15T01:37:31.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9eb006e890 0x7f9eb009b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:31.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9eb009fb10 0x7f9eb0131a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:31.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9eb006e890 0x7f9eb009b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36576/0 (socket says 172.21.15.204:36576) 2024-09-15T01:37:31.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaffff640 1 -- 172.21.15.204:0/1259355469 learned_addr learned my addr 172.21.15.204:0/1259355469 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:31.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaffff640 10 monclient(hunting): get_auth_request con 0x7f9eb006e890 auth_method 0 2024-09-15T01:37:31.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:31.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:31.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:31.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb4eba640 1 --2- 172.21.15.204:0/1259355469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9eb00a09c0 0x7f9eb0141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:31.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 10 monclient(hunting): get_auth_request con 0x7f9eb009fb10 auth_method 0 2024-09-15T01:37:31.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:31.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:31.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:31.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb4eba640 10 monclient(hunting): get_auth_request con 0x7f9eb00a09c0 auth_method 0 2024-09-15T01:37:31.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb4eba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:31.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb4eba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:31.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb4eba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:31.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:31.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:31.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:31.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:31.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:31.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:31.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb4eba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:31.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb4eba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:31.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eb4eba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:31.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 10 monclient(hunting): handle_auth_done global_id 24595 payload 824 2024-09-15T01:37:31.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 1 -- 172.21.15.204:0/1259355469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9eb00a09c0 msgr2=0x7f9eb0141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:31.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 1 --2- 172.21.15.204:0/1259355469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9eb00a09c0 0x7f9eb0141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 1 -- 172.21.15.204:0/1259355469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9eb006e890 msgr2=0x7f9eb009b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:31.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 1 --2- 172.21.15.204:0/1259355469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9eb006e890 0x7f9eb009b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:31.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:31.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 1 monclient: found mon.noname-i 2024-09-15T01:37:31.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:31.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:31.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaf7fe640 1 -- 172.21.15.204:0/1259355469 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4009450 con 0x7f9eb009fb10 2024-09-15T01:37:31.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.666+0000 7f9eaffff640 1 --2- 172.21.15.204:0/1259355469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9eb006e890 0x7f9eb009b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:31.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9eaf7fe640 1 --2- 172.21.15.204:0/1259355469 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9eb009fb10 0x7f9eb0131a20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4031000 tx=0x7f9ea4032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:31.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9e97fff640 1 -- 172.21.15.204:0/1259355469 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ea4033e90 con 0x7f9eb009fb10 2024-09-15T01:37:31.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9e97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:31.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9e97fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:31.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9e97fff640 10 monclient: dump: 2024-09-15T01:37:31.693 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:31.693 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:31.693 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:31.693 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:31.693 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:31.693 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:31.694 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:31.694 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:31.694 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:31.694 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:31.694 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:31.694 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:31.694 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:31.694 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:31.694 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:31.694 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:31.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9e97fff640 10 monclient: _finish_auth 0 2024-09-15T01:37:31.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9e97fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:31.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9e97fff640 1 -- 172.21.15.204:0/1259355469 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea402ac70 con 0x7f9eb009fb10 2024-09-15T01:37:31.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9eb691d640 5 monclient: authenticate success, global_id 24595 2024-09-15T01:37:31.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9eb691d640 10 monclient: _renew_subs 2024-09-15T01:37:31.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9eb691d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:31.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9eb691d640 1 -- 172.21.15.204:0/1259355469 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb009bbe0 con 0x7f9eb009fb10 2024-09-15T01:37:31.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9e97fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:31.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9e97fff640 1 -- 172.21.15.204:0/1259355469 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ea403a810 con 0x7f9eb009fb10 2024-09-15T01:37:31.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9e97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:31.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9e97fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:31.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9e97fff640 10 monclient: dump: 2024-09-15T01:37:31.696 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:31.696 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:31.696 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:31.696 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:31.696 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:31.696 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:31.696 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:31.696 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:31.696 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:31.696 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9eb691d640 10 monclient: _renew_subs 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9eb691d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9eae7fc640 4 set_mon_vals no callback set 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.667+0000 7f9eb691d640 1 -- 172.21.15.204:0/1259355469 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb011dbe0 con 0x7f9eb009fb10 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.668+0000 7f9e97fff640 1 -- 172.21.15.204:0/1259355469 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9ea4039040 con 0x7f9eb009fb10 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.668+0000 7f9eb691d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:31.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.668+0000 7f9eb691d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:31.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.668+0000 7f9eb691d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:31.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.668+0000 7f9eb691d640 1 -- 172.21.15.204:0/1259355469 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb0133370 con 0x7f9eb009fb10 2024-09-15T01:37:31.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.668+0000 7f9e97fff640 1 --2- 172.21.15.204:0/1259355469 >> v2:172.21.15.204:6812/28210 conn(0x7f9e9c0397b0 0x7f9e9c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:31.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.668+0000 7f9e97fff640 1 -- 172.21.15.204:0/1259355469 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 13700+0+0 (secure 0 0 0) 0x7f9ea406ccb0 con 0x7f9eb009fb10 2024-09-15T01:37:31.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.668+0000 7f9eaffff640 1 --2- 172.21.15.204:0/1259355469 >> v2:172.21.15.204:6812/28210 conn(0x7f9e9c0397b0 0x7f9e9c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:31.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.668+0000 7f9eaffff640 10 monclient: get_auth_request con 0x7f9e9c0397b0 auth_method 0 2024-09-15T01:37:31.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.668+0000 7f9eaffff640 1 --2- 172.21.15.204:0/1259355469 >> v2:172.21.15.204:6812/28210 conn(0x7f9e9c0397b0 0x7f9e9c03bc60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000f360 tx=0x7f9ea0008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:31.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.673+0000 7f9e97fff640 1 -- 172.21.15.204:0/1259355469 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9ea4044da0 con 0x7f9eb009fb10 2024-09-15T01:37:31.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.673+0000 7f9e97fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:31.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.673+0000 7f9e97fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:31.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.832+0000 7f9eb691d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:31.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.832+0000 7f9eb691d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:31.834 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.832+0000 7f9eb691d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:31.835 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:31.832+0000 7f9eb691d640 1 -- 172.21.15.204:0/1259355469 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f9eb0067540 con 0x7f9eb009fb10 2024-09-15T01:37:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.596+0000 7f9e97fff640 1 -- 172.21.15.204:0/1259355469 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}]=0 added bucket h31 type host to location {rack=r1} v216) v1 ==== 168+0+0 (secure 0 0 0) 0x7f9ea4041070 con 0x7f9eb009fb10 2024-09-15T01:37:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.596+0000 7f9e97fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:32.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.596+0000 7f9e97fff640 10 monclient: _finish_command 2 = system:0 added bucket h31 type host to location {rack=r1} 2024-09-15T01:37:32.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.692+0000 7f9eb691d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:32.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.692+0000 7f9eb691d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:32.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.692+0000 7f9eb691d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:32.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.692+0000 7f9eb691d640 1 -- 172.21.15.204:0/1259355469 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f9eb011ec80 con 0x7f9eb009fb10 2024-09-15T01:37:32.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.693+0000 7f9e97fff640 1 -- 172.21.15.204:0/1259355469 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h31' already exists v216) v1 ==== 147+0+0 (secure 0 0 0) 0x7f9eb011ec80 con 0x7f9eb009fb10 2024-09-15T01:37:32.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.693+0000 7f9e97fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:32.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.693+0000 7f9e97fff640 10 monclient: _finish_command 3 = system:0 bucket 'h31' already exists 2024-09-15T01:37:32.696 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h31' already exists 2024-09-15T01:37:32.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 1 -- 172.21.15.204:0/1259355469 >> v2:172.21.15.204:6812/28210 conn(0x7f9e9c0397b0 msgr2=0x7f9e9c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:32.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 1 --2- 172.21.15.204:0/1259355469 >> v2:172.21.15.204:6812/28210 conn(0x7f9e9c0397b0 0x7f9e9c03bc60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000f360 tx=0x7f9ea0008040 comp rx=0 tx=0).stop 2024-09-15T01:37:32.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 10 monclient: shutdown 2024-09-15T01:37:32.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:32.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 1 -- 172.21.15.204:0/1259355469 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9eb009fb10 msgr2=0x7f9eb0131a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:32.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 1 --2- 172.21.15.204:0/1259355469 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9eb009fb10 0x7f9eb0131a20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4031000 tx=0x7f9ea4032000 comp rx=0 tx=0).stop 2024-09-15T01:37:32.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 1 -- 172.21.15.204:0/1259355469 shutdown_connections 2024-09-15T01:37:32.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 1 --2- 172.21.15.204:0/1259355469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9eb00a09c0 0x7f9eb0141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:32.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 1 --2- 172.21.15.204:0/1259355469 >> v2:172.21.15.204:6812/28210 conn(0x7f9e9c0397b0 0x7f9e9c03bc60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:32.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 1 --2- 172.21.15.204:0/1259355469 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9eb009fb10 0x7f9eb0131a20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:32.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 1 --2- 172.21.15.204:0/1259355469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9eb006e890 0x7f9eb009b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:32.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 1 -- 172.21.15.204:0/1259355469 >> 172.21.15.204:0/1259355469 conn(0x7f9eb00a5fd0 msgr2=0x7f9eb01357d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:32.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 1 -- 172.21.15.204:0/1259355469 shutdown_connections 2024-09-15T01:37:32.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.695+0000 7f9eb691d640 1 -- 172.21.15.204:0/1259355469 wait complete. 2024-09-15T01:37:32.706 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:32.707 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush add-bucket h32 host rack=r1 2024-09-15T01:37:32.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.780+0000 7ff3f9017640 10 monclient: get_monmap_and_config 2024-09-15T01:37:32.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.780+0000 7ff3f9017640 10 monclient: build_initial_monmap 2024-09-15T01:37:32.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.780+0000 7ff3f9017640 10 monclient: monmap: 2024-09-15T01:37:32.783 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:32.783 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:32.783 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:32.782100+0000 2024-09-15T01:37:32.783 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:32.782100+0000 2024-09-15T01:37:32.783 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:32.783 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:32.783 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:32.784 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:32.784 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:32.784 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:32.784 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:32.784 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:32.784 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:32.784 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:32.784 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:32.784 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:32.784 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:32.784 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:32.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 1 Processor -- start 2024-09-15T01:37:32.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 1 -- start start 2024-09-15T01:37:32.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 10 monclient: init 2024-09-15T01:37:32.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:32.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 10 monclient: _add_conns ranks=[3,6,7,10,4,2,0,1,9,5,8] 2024-09-15T01:37:32.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3f40a2cd0 0x7ff3f4132c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:32.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 10 monclient(hunting): picked mon.noname-g con 0x7ff3f40a2cd0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:32.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff3f409fa20 0x7ff3f409fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:32.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 10 monclient(hunting): picked mon.noname-d con 0x7ff3f409fa20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:32.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f40a08e0 0x7ff3f40a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:32.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 10 monclient(hunting): picked mon.noname-i con 0x7ff3f40a08e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40a30a0 con 0x7ff3f40a2cd0 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40a0330 con 0x7ff3f409fa20 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40a1db0 con 0x7ff3f40a08e0 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f9017640 10 monclient(hunting): authenticate will time out at 1068.424725s 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f2d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3f40a2cd0 0x7ff3f4132c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f359e640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f40a08e0 0x7ff3f40a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f359e640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f40a08e0 0x7ff3f40a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:33494/0 (socket says 172.21.15.204:33494) 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f2d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3f40a2cd0 0x7ff3f4132c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:52706/0 (socket says 172.21.15.204:52706) 2024-09-15T01:37:32.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f359e640 1 -- 172.21.15.204:0/1347225041 learned_addr learned my addr 172.21.15.204:0/1347225041 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:32.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f359e640 10 monclient(hunting): get_auth_request con 0x7ff3f40a08e0 auth_method 0 2024-09-15T01:37:32.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:32.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:32.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:32.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f2d9d640 10 monclient(hunting): get_auth_request con 0x7ff3f40a2cd0 auth_method 0 2024-09-15T01:37:32.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:32.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:32.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:32.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:32.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:32.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:32.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:32.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.781+0000 7ff3f359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:32.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:32.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3ea59c640 1 --2- 172.21.15.204:0/1347225041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff3f409fa20 0x7ff3f409fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:32.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f359e640 10 monclient(hunting): handle_auth_done global_id 24604 payload 274 2024-09-15T01:37:32.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f359e640 1 -- 172.21.15.204:0/1347225041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff3f409fa20 msgr2=0x7ff3f409fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:32.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f359e640 1 --2- 172.21.15.204:0/1347225041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff3f409fa20 0x7ff3f409fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:32.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f359e640 1 -- 172.21.15.204:0/1347225041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3f40a2cd0 msgr2=0x7ff3f4132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:32.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f359e640 1 --2- 172.21.15.204:0/1347225041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3f40a2cd0 0x7ff3f4132c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:32.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f359e640 10 monclient: _finish_hunting 0 2024-09-15T01:37:32.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f359e640 1 monclient: found mon.noname-i 2024-09-15T01:37:32.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:32.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f359e640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:32.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f359e640 1 -- 172.21.15.204:0/1347225041 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f40648a0 con 0x7ff3f40a08e0 2024-09-15T01:37:32.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3ea59c640 10 monclient: get_auth_request con 0x7ff3f409fa20 auth_method 0 2024-09-15T01:37:32.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f2d9d640 1 --2- 172.21.15.204:0/1347225041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3f40a2cd0 0x7ff3f4132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:32.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3ea59c640 1 --2- 172.21.15.204:0/1347225041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff3f409fa20 0x7ff3f409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:37:32.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f359e640 1 --2- 172.21.15.204:0/1347225041 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f40a08e0 0x7ff3f40a0d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009f50 tx=0x7ff3ec009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=459ca4c34798b75e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:32.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f259c640 1 -- 172.21.15.204:0/1347225041 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3ec007880 con 0x7ff3f40a08e0 2024-09-15T01:37:32.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f259c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:32.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f259c640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:32.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.782+0000 7ff3f259c640 10 monclient: dump: 2024-09-15T01:37:32.797 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:32.797 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:32.797 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:32.798 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f259c640 10 monclient: _finish_auth 0 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f259c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f259c640 1 -- 172.21.15.204:0/1347225041 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec02d050 con 0x7ff3f40a08e0 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 5 monclient: authenticate success, global_id 24604 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f259c640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f259c640 1 -- 172.21.15.204:0/1347225041 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3ec030360 con 0x7ff3f40a08e0 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f259c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 4 set_mon_vals no callback set 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f259c640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f259c640 10 monclient: dump: 2024-09-15T01:37:32.799 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:32.800 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:32.800 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:32.800 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:32.800 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:32.800 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:32.800 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:32.800 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:32.800 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:32.800 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:32.800 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:32.800 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:32.800 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:32.801 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:32.801 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:32.801 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:32.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 10 monclient: shutdown 2024-09-15T01:37:32.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:32.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 1 -- 172.21.15.204:0/1347225041 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f40a08e0 msgr2=0x7ff3f40a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:32.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 1 --2- 172.21.15.204:0/1347225041 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f40a08e0 0x7ff3f40a0d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009f50 tx=0x7ff3ec009840 comp rx=0 tx=0).stop 2024-09-15T01:37:32.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 1 -- 172.21.15.204:0/1347225041 shutdown_connections 2024-09-15T01:37:32.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 1 --2- 172.21.15.204:0/1347225041 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3f40a08e0 0x7ff3f40a0d50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:32.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 1 --2- 172.21.15.204:0/1347225041 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff3f409fa20 0x7ff3f409fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:32.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 1 --2- 172.21.15.204:0/1347225041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3f40a2cd0 0x7ff3f4132c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:32.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 1 -- 172.21.15.204:0/1347225041 >> 172.21.15.204:0/1347225041 conn(0x7ff3f406e860 msgr2=0x7ff3f406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 1 -- 172.21.15.204:0/1347225041 shutdown_connections 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.783+0000 7ff3f9017640 1 -- 172.21.15.204:0/1347225041 wait complete. 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.784+0000 7ff3f9017640 10 monclient: build_initial_monmap 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.784+0000 7ff3f9017640 10 monclient: monmap: 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:32.802 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.784+0000 7ff3f9017640 1 Processor -- start 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.784+0000 7ff3f9017640 1 -- start start 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.784+0000 7ff3f9017640 10 monclient: init 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.784+0000 7ff3f9017640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.784+0000 7ff3f9017640 10 monclient: _add_conns ranks=[0,1,2,7,3,6,4,8,5] 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.784+0000 7ff3f9017640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3f409fa20 0x7ff3f4141f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.784+0000 7ff3f9017640 10 monclient(hunting): picked mon.noname-a con 0x7ff3f409fa20 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:32.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.784+0000 7ff3f9017640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f40a08e0 0x7ff3f4142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:32.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.784+0000 7ff3f9017640 10 monclient(hunting): picked mon.noname-f con 0x7ff3f40a08e0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:32.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3f40a2cd0 0x7ff3f411d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:32.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 10 monclient(hunting): picked mon.noname-b con 0x7ff3f40a2cd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:32.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:32.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f409cb60 con 0x7ff3f409fa20 2024-09-15T01:37:32.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:32.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f41206b0 con 0x7ff3f40a08e0 2024-09-15T01:37:32.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:32.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f409d3d0 con 0x7ff3f40a2cd0 2024-09-15T01:37:32.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:32.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 10 monclient(hunting): authenticate will time out at 1068.428038s 2024-09-15T01:37:32.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f40a08e0 0x7ff3f4142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f2d9d640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3f409fa20 msgr2=0x7ff3f4141f00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f359e640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3f40a2cd0 msgr2=0x7ff3f411d010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f2d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3f409fa20 0x7ff3f4141f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f40a08e0 0x7ff3f4142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36588/0 (socket says 172.21.15.204:36588) 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f359e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3f40a2cd0 0x7ff3f411d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 1 -- 172.21.15.204:0/754094599 learned_addr learned my addr 172.21.15.204:0/754094599 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 10 monclient(hunting): get_auth_request con 0x7ff3f40a08e0 auth_method 0 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:32.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 10 monclient(hunting): handle_auth_done global_id 7174 payload 824 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 1 -- 172.21.15.204:0/754094599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3f40a2cd0 msgr2=0x7ff3f411d010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 1 --2- 172.21.15.204:0/754094599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3f40a2cd0 0x7ff3f411d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 1 -- 172.21.15.204:0/754094599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3f409fa20 msgr2=0x7ff3f4141f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 1 --2- 172.21.15.204:0/754094599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3f409fa20 0x7ff3f4141f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 10 monclient: _finish_hunting 0 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 1 monclient: found mon.noname-f 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 1 -- 172.21.15.204:0/754094599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e4009390 con 0x7ff3f40a08e0 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3ea59c640 1 --2- 172.21.15.204:0/754094599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f40a08e0 0x7ff3f4142440 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00c880 tx=0x7ff3dc008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3eb7fe640 1 -- 172.21.15.204:0/754094599 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3dc00ce10 con 0x7ff3f40a08e0 2024-09-15T01:37:32.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3eb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3eb7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3eb7fe640 10 monclient: dump: 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:32.807 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:32.808 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3eb7fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3eb7fe640 1 -- 172.21.15.204:0/754094599 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc007d10 con 0x7ff3f40a08e0 2024-09-15T01:37:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 5 monclient: authenticate success, global_id 7174 2024-09-15T01:37:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 10 monclient: _renew_subs 2024-09-15T01:37:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:32.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 1 -- 172.21.15.204:0/754094599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec009ae0 con 0x7ff3f40a08e0 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3eb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3eb7fe640 1 -- 172.21.15.204:0/754094599 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3dc007e80 con 0x7ff3f40a08e0 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3eb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3eb7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f1d9b640 4 set_mon_vals no callback set 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3eb7fe640 10 monclient: dump: 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:32.809 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 10 monclient: _renew_subs 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.785+0000 7ff3f9017640 1 -- 172.21.15.204:0/754094599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f411f100 con 0x7ff3f40a08e0 2024-09-15T01:37:32.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.786+0000 7ff3eb7fe640 1 -- 172.21.15.204:0/754094599 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff3dc007890 con 0x7ff3f40a08e0 2024-09-15T01:37:32.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.786+0000 7ff3f9017640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:32.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.786+0000 7ff3f9017640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:32.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.786+0000 7ff3f9017640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:32.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.786+0000 7ff3f9017640 1 -- 172.21.15.204:0/754094599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f4070b80 con 0x7ff3f40a08e0 2024-09-15T01:37:32.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.786+0000 7ff3eb7fe640 1 --2- 172.21.15.204:0/754094599 >> v2:172.21.15.204:6812/28210 conn(0x7ff3d4039710 0x7ff3d403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:32.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.787+0000 7ff3eb7fe640 1 -- 172.21.15.204:0/754094599 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 13913+0+0 (secure 0 0 0) 0x7ff3dc04d840 con 0x7ff3f40a08e0 2024-09-15T01:37:32.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.787+0000 7ff3f2d9d640 1 --2- 172.21.15.204:0/754094599 >> v2:172.21.15.204:6812/28210 conn(0x7ff3d4039710 0x7ff3d403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:32.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.787+0000 7ff3f2d9d640 10 monclient: get_auth_request con 0x7ff3d4039710 auth_method 0 2024-09-15T01:37:32.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.787+0000 7ff3f2d9d640 1 --2- 172.21.15.204:0/754094599 >> v2:172.21.15.204:6812/28210 conn(0x7ff3d4039710 0x7ff3d403bbc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009520 tx=0x7ff3e40090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:32.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.791+0000 7ff3eb7fe640 1 -- 172.21.15.204:0/754094599 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff3dc017300 con 0x7ff3f40a08e0 2024-09-15T01:37:32.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.791+0000 7ff3eb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:32.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.791+0000 7ff3eb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:32.952 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.950+0000 7ff3f9017640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.950+0000 7ff3f9017640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.950+0000 7ff3f9017640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:32.953 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:32.950+0000 7ff3f9017640 1 -- 172.21.15.204:0/754094599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7ff3f411ebf0 con 0x7ff3f40a08e0 2024-09-15T01:37:33.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.598+0000 7ff3eb7fe640 1 -- 172.21.15.204:0/754094599 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}]=0 added bucket h32 type host to location {rack=r1} v217) v1 ==== 168+0+0 (secure 0 0 0) 0x7ff3dc020910 con 0x7ff3f40a08e0 2024-09-15T01:37:33.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.598+0000 7ff3eb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:33.601 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.598+0000 7ff3eb7fe640 10 monclient: _finish_command 2 = system:0 added bucket h32 type host to location {rack=r1} 2024-09-15T01:37:33.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.700+0000 7ff3f9017640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:33.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.700+0000 7ff3f9017640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:33.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.700+0000 7ff3f9017640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:33.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.700+0000 7ff3f9017640 1 -- 172.21.15.204:0/754094599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7ff3f40008d0 con 0x7ff3f40a08e0 2024-09-15T01:37:33.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.701+0000 7ff3eb7fe640 1 -- 172.21.15.204:0/754094599 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h32' already exists v217) v1 ==== 147+0+0 (secure 0 0 0) 0x7ff3dc020af0 con 0x7ff3f40a08e0 2024-09-15T01:37:33.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.701+0000 7ff3eb7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-09-15T01:37:33.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.701+0000 7ff3eb7fe640 10 monclient: _finish_command 3 = system:0 bucket 'h32' already exists 2024-09-15T01:37:33.704 INFO:tasks.workunit.client.0.smithi204.stderr:bucket 'h32' already exists 2024-09-15T01:37:33.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.702+0000 7ff3f9017640 1 -- 172.21.15.204:0/754094599 >> v2:172.21.15.204:6812/28210 conn(0x7ff3d4039710 msgr2=0x7ff3d403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:33.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.702+0000 7ff3f9017640 1 --2- 172.21.15.204:0/754094599 >> v2:172.21.15.204:6812/28210 conn(0x7ff3d4039710 0x7ff3d403bbc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009520 tx=0x7ff3e40090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:33.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.703+0000 7ff3f9017640 10 monclient: shutdown 2024-09-15T01:37:33.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.703+0000 7ff3f9017640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:33.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.703+0000 7ff3f9017640 1 -- 172.21.15.204:0/754094599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f40a08e0 msgr2=0x7ff3f4142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:33.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.703+0000 7ff3f9017640 1 --2- 172.21.15.204:0/754094599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f40a08e0 0x7ff3f4142440 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00c880 tx=0x7ff3dc008040 comp rx=0 tx=0).stop 2024-09-15T01:37:33.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.703+0000 7ff3f9017640 1 -- 172.21.15.204:0/754094599 shutdown_connections 2024-09-15T01:37:33.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.703+0000 7ff3f9017640 1 --2- 172.21.15.204:0/754094599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3f40a2cd0 0x7ff3f411d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:33.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.703+0000 7ff3f9017640 1 --2- 172.21.15.204:0/754094599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3f40a08e0 0x7ff3f4142440 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:33.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.703+0000 7ff3f9017640 1 --2- 172.21.15.204:0/754094599 >> v2:172.21.15.204:6812/28210 conn(0x7ff3d4039710 0x7ff3d403bbc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:33.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.703+0000 7ff3f9017640 1 --2- 172.21.15.204:0/754094599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3f409fa20 0x7ff3f4141f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:33.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.703+0000 7ff3f9017640 1 -- 172.21.15.204:0/754094599 >> 172.21.15.204:0/754094599 conn(0x7ff3f406e860 msgr2=0x7ff3f40a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:33.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.703+0000 7ff3f9017640 1 -- 172.21.15.204:0/754094599 shutdown_connections 2024-09-15T01:37:33.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.703+0000 7ff3f9017640 1 -- 172.21.15.204:0/754094599 wait complete. 2024-09-15T01:37:33.714 INFO:tasks.workunit.client.0.smithi204.stderr:++ seq 1 32 2024-09-15T01:37:33.715 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:33.715 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h1 2024-09-15T01:37:33.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.786+0000 7f451f31d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:33.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.786+0000 7f451f31d640 10 monclient: build_initial_monmap 2024-09-15T01:37:33.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.786+0000 7f451f31d640 10 monclient: monmap: 2024-09-15T01:37:33.789 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:33.790 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:33.790 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:33.788126+0000 2024-09-15T01:37:33.790 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:33.788126+0000 2024-09-15T01:37:33.790 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:33.790 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:33.790 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:33.790 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:33.790 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:33.790 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:33.790 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:33.790 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:33.791 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:33.791 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:33.791 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:33.791 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:33.791 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:33.791 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:33.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 1 Processor -- start 2024-09-15T01:37:33.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 1 -- start start 2024-09-15T01:37:33.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 10 monclient: init 2024-09-15T01:37:33.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:33.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 10 monclient: _add_conns ranks=[8,7,10,4,3,1,0,5,9,2,6] 2024-09-15T01:37:33.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45180a2e30 0x7f4518132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:33.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 10 monclient(hunting): picked mon.noname-e con 0x7f45180a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:33.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f451809fb80 0x7f451809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:33.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 10 monclient(hunting): picked mon.noname-i con 0x7f451809fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:33.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f45180a0a40 0x7f45180a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:33.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 10 monclient(hunting): picked mon.noname-f con 0x7f45180a0a40 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f451809e6c0 con 0x7f45180a0a40 2024-09-15T01:37:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f45180a04b0 con 0x7f451809fb80 2024-09-15T01:37:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f45180a3200 con 0x7f45180a2e30 2024-09-15T01:37:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451f31d640 10 monclient(hunting): authenticate will time out at 1069.430866s 2024-09-15T01:37:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451d8ba640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f45180a0a40 0x7f45180a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.787+0000 7f451c8b8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f451809fb80 0x7f451809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f45180a0a40 0x7f45180a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36594/0 (socket says 172.21.15.204:36594) 2024-09-15T01:37:33.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 1 -- 172.21.15.204:0/2852081673 learned_addr learned my addr 172.21.15.204:0/2852081673 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 10 monclient(hunting): get_auth_request con 0x7f45180a0a40 auth_method 0 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d0b9640 1 -- 172.21.15.204:0/2852081673 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45180a2e30 msgr2=0x7f4518132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451c8b8640 10 monclient(hunting): get_auth_request con 0x7f451809fb80 auth_method 0 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451c8b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451c8b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451c8b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d0b9640 1 --2- 172.21.15.204:0/2852081673 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45180a2e30 0x7f4518132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:33.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451c8b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451c8b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451c8b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 10 monclient(hunting): handle_auth_done global_id 7183 payload 274 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 1 -- 172.21.15.204:0/2852081673 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45180a2e30 msgr2=0x7f4518132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 1 --2- 172.21.15.204:0/2852081673 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45180a2e30 0x7f4518132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 1 -- 172.21.15.204:0/2852081673 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f451809fb80 msgr2=0x7f451809ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 1 --2- 172.21.15.204:0/2852081673 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f451809fb80 0x7f451809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 10 monclient: _finish_hunting 0 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 1 monclient: found mon.noname-f 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 1 -- 172.21.15.204:0/2852081673 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451804b720 con 0x7f45180a0a40 2024-09-15T01:37:33.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451c8b8640 1 --2- 172.21.15.204:0/2852081673 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f451809fb80 0x7f451809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.788+0000 7f451d8ba640 1 --2- 172.21.15.204:0/2852081673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45180a0a40 0x7f45180a0ee0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4510009f20 tx=0x7f4510009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaab9646a3824022 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f450ffff640 1 -- 172.21.15.204:0/2852081673 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4510007880 con 0x7f45180a0a40 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f450ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f450ffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f450ffff640 10 monclient: dump: 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:33.804 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:33.805 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:33.805 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:33.805 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:33.805 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:33.805 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:33.805 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:33.805 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:33.805 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:33.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f450ffff640 10 monclient: _finish_auth 0 2024-09-15T01:37:33.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f450ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:33.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f450ffff640 1 -- 172.21.15.204:0/2852081673 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451002d050 con 0x7f45180a0a40 2024-09-15T01:37:33.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 5 monclient: authenticate success, global_id 7183 2024-09-15T01:37:33.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:33.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f450ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:33.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:33.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f450ffff640 1 -- 172.21.15.204:0/2852081673 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4510030360 con 0x7f45180a0a40 2024-09-15T01:37:33.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f450ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:33.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 4 set_mon_vals no callback set 2024-09-15T01:37:33.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f450ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:33.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f450ffff640 10 monclient: dump: 2024-09-15T01:37:33.807 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:33.807 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:33.807 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:33.807 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:33.807 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:33.807 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:33.807 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:33.807 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:33.807 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:33.807 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:33.807 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:33.808 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:33.808 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:33.808 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:33.808 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:33.808 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:33.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 10 monclient: shutdown 2024-09-15T01:37:33.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:33.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 1 -- 172.21.15.204:0/2852081673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45180a0a40 msgr2=0x7f45180a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:33.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 1 --2- 172.21.15.204:0/2852081673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45180a0a40 0x7f45180a0ee0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4510009f20 tx=0x7f4510009840 comp rx=0 tx=0).stop 2024-09-15T01:37:33.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 1 -- 172.21.15.204:0/2852081673 shutdown_connections 2024-09-15T01:37:33.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 1 --2- 172.21.15.204:0/2852081673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45180a0a40 0x7f45180a0ee0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:33.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 1 --2- 172.21.15.204:0/2852081673 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f451809fb80 0x7f451809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:33.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 1 --2- 172.21.15.204:0/2852081673 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45180a2e30 0x7f4518132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:33.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 1 -- 172.21.15.204:0/2852081673 >> 172.21.15.204:0/2852081673 conn(0x7f451806e860 msgr2=0x7f451806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:33.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 1 -- 172.21.15.204:0/2852081673 shutdown_connections 2024-09-15T01:37:33.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.789+0000 7f451f31d640 1 -- 172.21.15.204:0/2852081673 wait complete. 2024-09-15T01:37:33.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.790+0000 7f451f31d640 10 monclient: build_initial_monmap 2024-09-15T01:37:33.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.790+0000 7f451f31d640 10 monclient: monmap: 2024-09-15T01:37:33.810 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:33.811 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:33.811 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:33.811 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:33.811 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:33.811 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:33.811 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:33.812 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:33.812 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:33.812 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:33.812 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:33.812 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:33.812 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:33.813 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:33.813 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:33.813 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:33.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.790+0000 7f451f31d640 1 Processor -- start 2024-09-15T01:37:33.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.790+0000 7f451f31d640 1 -- start start 2024-09-15T01:37:33.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.790+0000 7f451f31d640 10 monclient: init 2024-09-15T01:37:33.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:33.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 10 monclient: _add_conns ranks=[5,8,1,0,2,4,7,6,3] 2024-09-15T01:37:33.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f451809fb80 0x7f4518131650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:33.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 10 monclient(hunting): picked mon.noname-h con 0x7f451809fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:33.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45180a0a40 0x7f451811e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:33.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 10 monclient(hunting): picked mon.noname-e con 0x7f45180a0a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:33.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45180a2e30 0x7f451811e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:33.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 10 monclient(hunting): picked mon.noname-f con 0x7f45180a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:33.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:33.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4518120510 con 0x7f45180a2e30 2024-09-15T01:37:33.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:33.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4518121a80 con 0x7f451809fb80 2024-09-15T01:37:33.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:33.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f451809ca40 con 0x7f45180a0a40 2024-09-15T01:37:33.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:33.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 10 monclient(hunting): authenticate will time out at 1069.434258s 2024-09-15T01:37:33.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d8ba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45180a2e30 0x7f451811e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:33.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f451809fb80 0x7f4518131650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:33.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d8ba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45180a2e30 0x7f451811e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36596/0 (socket says 172.21.15.204:36596) 2024-09-15T01:37:33.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f451809fb80 0x7f4518131650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:54484/0 (socket says 172.21.15.204:54484) 2024-09-15T01:37:33.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d8ba640 1 -- 172.21.15.204:0/2404396928 learned_addr learned my addr 172.21.15.204:0/2404396928 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:33.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d8ba640 10 monclient(hunting): get_auth_request con 0x7f45180a2e30 auth_method 0 2024-09-15T01:37:33.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d8ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:33.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d8ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:33.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d8ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:33.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451c8b8640 1 -- 172.21.15.204:0/2404396928 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45180a0a40 msgr2=0x7f451811e100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:33.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 10 monclient(hunting): get_auth_request con 0x7f451809fb80 auth_method 0 2024-09-15T01:37:33.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:33.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:33.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:33.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451c8b8640 1 --2- 172.21.15.204:0/2404396928 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45180a0a40 0x7f451811e100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:33.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d8ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:33.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d8ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:33.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d8ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:33.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:33.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:33.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:33.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 10 monclient(hunting): handle_auth_done global_id 6368 payload 824 2024-09-15T01:37:33.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 1 -- 172.21.15.204:0/2404396928 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45180a0a40 msgr2=0x7f451811e100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:33.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 1 --2- 172.21.15.204:0/2404396928 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45180a0a40 0x7f451811e100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:33.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 1 -- 172.21.15.204:0/2404396928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45180a2e30 msgr2=0x7f451811e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:33.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 1 --2- 172.21.15.204:0/2404396928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45180a2e30 0x7f451811e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:33.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 10 monclient: _finish_hunting 0 2024-09-15T01:37:33.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 1 monclient: found mon.noname-h 2024-09-15T01:37:33.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:33.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:33.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 1 -- 172.21.15.204:0/2404396928 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4508009390 con 0x7f451809fb80 2024-09-15T01:37:33.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d8ba640 1 --2- 172.21.15.204:0/2404396928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45180a2e30 0x7f451811e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:33.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451d0b9640 1 --2- 172.21.15.204:0/2404396928 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f451809fb80 0x7f4518131650 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f451400b5b0 tx=0x7f45140090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:33.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f450d7fa640 1 -- 172.21.15.204:0/2404396928 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4514006150 con 0x7f451809fb80 2024-09-15T01:37:33.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f450d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:33.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f450d7fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:33.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f450d7fa640 10 monclient: dump: 2024-09-15T01:37:33.823 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:33.823 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:33.823 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:33.823 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:33.823 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:33.824 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:33.824 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:33.824 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:33.824 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:33.824 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:33.825 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:33.825 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:33.825 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:33.825 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:33.825 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:33.825 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:33.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f450d7fa640 10 monclient: _finish_auth 0 2024-09-15T01:37:33.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f450d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:33.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f450d7fa640 1 -- 172.21.15.204:0/2404396928 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45140062f0 con 0x7f451809fb80 2024-09-15T01:37:33.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 5 monclient: authenticate success, global_id 6368 2024-09-15T01:37:33.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f450d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:33.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f450d7fa640 1 -- 172.21.15.204:0/2404396928 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f45140103e0 con 0x7f451809fb80 2024-09-15T01:37:33.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 10 monclient: _renew_subs 2024-09-15T01:37:33.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:33.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f451f31d640 1 -- 172.21.15.204:0/2404396928 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4510009ae0 con 0x7f451809fb80 2024-09-15T01:37:33.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f450d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:33.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.791+0000 7f450f7fe640 4 set_mon_vals no callback set 2024-09-15T01:37:33.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.792+0000 7f450d7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:33.828 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.792+0000 7f450d7fa640 10 monclient: dump: 2024-09-15T01:37:33.828 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:33.828 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:33.828 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:33.828 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:33.828 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:33.829 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:33.830 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:33.830 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:33.830 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:33.830 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:33.830 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:33.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.792+0000 7f451f31d640 10 monclient: _renew_subs 2024-09-15T01:37:33.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.792+0000 7f451f31d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:33.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.792+0000 7f451f31d640 1 -- 172.21.15.204:0/2404396928 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451809d650 con 0x7f451809fb80 2024-09-15T01:37:33.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.792+0000 7f450d7fa640 1 -- 172.21.15.204:0/2404396928 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4514003b80 con 0x7f451809fb80 2024-09-15T01:37:33.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.792+0000 7f450d7fa640 1 --2- 172.21.15.204:0/2404396928 >> v2:172.21.15.204:6812/28210 conn(0x7f44fc0397b0 0x7f44fc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:33.831 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.793+0000 7f450d7fa640 1 -- 172.21.15.204:0/2404396928 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 14126+0+0 (secure 0 0 0) 0x7f451404cc70 con 0x7f451809fb80 2024-09-15T01:37:33.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.793+0000 7f451f31d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:33.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.793+0000 7f451f31d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:33.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.793+0000 7f451f31d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:33.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.793+0000 7f451f31d640 1 -- 172.21.15.204:0/2404396928 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4518070cc0 con 0x7f451809fb80 2024-09-15T01:37:33.832 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.793+0000 7f451c8b8640 1 --2- 172.21.15.204:0/2404396928 >> v2:172.21.15.204:6812/28210 conn(0x7f44fc0397b0 0x7f44fc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:33.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.793+0000 7f451c8b8640 10 monclient: get_auth_request con 0x7f44fc0397b0 auth_method 0 2024-09-15T01:37:33.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.793+0000 7f451c8b8640 1 --2- 172.21.15.204:0/2404396928 >> v2:172.21.15.204:6812/28210 conn(0x7f44fc0397b0 0x7f44fc03bc60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f45180a2e00 tx=0x7f45080090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:33.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.797+0000 7f450d7fa640 1 -- 172.21.15.204:0/2404396928 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4514016070 con 0x7f451809fb80 2024-09-15T01:37:33.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.797+0000 7f450d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:33.833 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.797+0000 7f450d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:33.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.957+0000 7f451f31d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h1"}] 2024-09-15T01:37:33.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.957+0000 7f451f31d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h1"}] 2024-09-15T01:37:33.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.957+0000 7f451f31d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:33.960 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:33.957+0000 7f451f31d640 1 -- 172.21.15.204:0/2404396928 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h1"} v 0) v1 -- 0x7f4518121490 con 0x7f451809fb80 2024-09-15T01:37:34.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.612+0000 7f450d7fa640 1 -- 172.21.15.204:0/2404396928 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h1"}]=0 removed item id -19 name 'h1' from crush map v218) v1 ==== 118+0+0 (secure 0 0 0) 0x7f451400d6d0 con 0x7f451809fb80 2024-09-15T01:37:34.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.612+0000 7f450d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h1"}] 2024-09-15T01:37:34.615 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.612+0000 7f450d7fa640 10 monclient: _finish_command 2 = system:0 removed item id -19 name 'h1' from crush map 2024-09-15T01:37:34.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.708+0000 7f451f31d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h1"}] 2024-09-15T01:37:34.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.708+0000 7f451f31d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h1"}] 2024-09-15T01:37:34.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.708+0000 7f451f31d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:34.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.708+0000 7f451f31d640 1 -- 172.21.15.204:0/2404396928 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h1"} v 0) v1 -- 0x7f45181208b0 con 0x7f451809fb80 2024-09-15T01:37:34.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.708+0000 7f450d7fa640 1 -- 172.21.15.204:0/2404396928 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h1"}]=0 device 'h1' does not appear in the crush map v218) v1 ==== 118+0+0 (secure 0 0 0) 0x7f451400d310 con 0x7f451809fb80 2024-09-15T01:37:34.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.708+0000 7f450d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h1"}] 2024-09-15T01:37:34.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.708+0000 7f450d7fa640 10 monclient: _finish_command 3 = system:0 device 'h1' does not appear in the crush map 2024-09-15T01:37:34.711 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h1' does not appear in the crush map 2024-09-15T01:37:34.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 1 -- 172.21.15.204:0/2404396928 >> v2:172.21.15.204:6812/28210 conn(0x7f44fc0397b0 msgr2=0x7f44fc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:34.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 1 --2- 172.21.15.204:0/2404396928 >> v2:172.21.15.204:6812/28210 conn(0x7f44fc0397b0 0x7f44fc03bc60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f45180a2e00 tx=0x7f45080090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:34.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 10 monclient: shutdown 2024-09-15T01:37:34.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:34.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 1 -- 172.21.15.204:0/2404396928 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f451809fb80 msgr2=0x7f4518131650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:34.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 1 --2- 172.21.15.204:0/2404396928 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f451809fb80 0x7f4518131650 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f451400b5b0 tx=0x7f45140090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:34.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 1 -- 172.21.15.204:0/2404396928 shutdown_connections 2024-09-15T01:37:34.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 1 --2- 172.21.15.204:0/2404396928 >> v2:172.21.15.204:6812/28210 conn(0x7f44fc0397b0 0x7f44fc03bc60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:34.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 1 --2- 172.21.15.204:0/2404396928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45180a2e30 0x7f451811e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:34.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 1 --2- 172.21.15.204:0/2404396928 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45180a0a40 0x7f451811e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:34.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 1 --2- 172.21.15.204:0/2404396928 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f451809fb80 0x7f4518131650 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:34.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 1 -- 172.21.15.204:0/2404396928 >> 172.21.15.204:0/2404396928 conn(0x7f451806e860 msgr2=0x7f45180a41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:34.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 1 -- 172.21.15.204:0/2404396928 shutdown_connections 2024-09-15T01:37:34.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.710+0000 7f451f31d640 1 -- 172.21.15.204:0/2404396928 wait complete. 2024-09-15T01:37:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:34.722 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h2 2024-09-15T01:37:34.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.793+0000 7fbdf52a8640 10 monclient: get_monmap_and_config 2024-09-15T01:37:34.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.793+0000 7fbdf52a8640 10 monclient: build_initial_monmap 2024-09-15T01:37:34.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.793+0000 7fbdf52a8640 10 monclient: monmap: 2024-09-15T01:37:34.796 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:34.796 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:34.796 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:34.795230+0000 2024-09-15T01:37:34.796 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:34.795230+0000 2024-09-15T01:37:34.796 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:34.797 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:34.797 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:34.797 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:34.797 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:34.797 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:34.797 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:34.797 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:34.797 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:34.797 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:34.797 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:34.797 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:34.797 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:34.798 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:34.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 1 Processor -- start 2024-09-15T01:37:34.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 1 -- start start 2024-09-15T01:37:34.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 10 monclient: init 2024-09-15T01:37:34.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:34.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 10 monclient: _add_conns ranks=[8,4,0,9,10,1,7,2,6,5,3] 2024-09-15T01:37:34.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdf00a2e30 0x7fbdf0132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:34.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 10 monclient(hunting): picked mon.noname-e con 0x7fbdf00a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdf009fb80 0x7fbdf009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 10 monclient(hunting): picked mon.noname-c con 0x7fbdf009fb80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbdf00a0a40 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf004b720 con 0x7fbdf00a0a40 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf009e6c0 con 0x7fbdf009fb80 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf00a04b0 con 0x7fbdf00a2e30 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.794+0000 7fbdf52a8640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.795+0000 7fbdf52a8640 10 monclient(hunting): authenticate will time out at 1070.438013s 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.795+0000 7fbdef7fe640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7fbdf00a0a40 legacy=0x7fbdf00a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.795+0000 7fbdeeffd640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdf00a2e30 msgr2=0x7fbdf0132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.795+0000 7fbde7fff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdf009fb80 msgr2=0x7fbdf009ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:34.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.795+0000 7fbdeeffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdf00a2e30 0x7fbdf0132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:34.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.795+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdf009fb80 0x7fbdf009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:34.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.996+0000 7fbdef7fe640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7fbdf00a0a40 legacy=0x7fbdf00a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:34.998 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.996+0000 7fbde7fff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdf009fb80 msgr2=0x7fbdf009ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:34.999 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.996+0000 7fbdeeffd640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdf00a2e30 msgr2=0x7fbdf0132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:34.999 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.996+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdf009fb80 0x7fbdf009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:37:34.999 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:34.996+0000 7fbdeeffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdf00a2e30 0x7fbdf0132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:37:35.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:35.397+0000 7fbdeeffd640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdf00a2e30 msgr2=0x7fbdf0132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:35.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:35.397+0000 7fbde7fff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdf009fb80 msgr2=0x7fbdf009ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:35.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:35.397+0000 7fbdef7fe640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7fbdf00a0a40 legacy=0x7fbdf00a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:35.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:35.397+0000 7fbdeeffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdf00a2e30 0x7fbdf0132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:37:35.399 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:35.397+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdf009fb80 0x7fbdf009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:37:36.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:36.198+0000 7fbdef7fe640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7fbdf00a0a40 legacy=0x7fbdf00a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:36.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:36.198+0000 7fbde7fff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdf009fb80 msgr2=0x7fbdf009ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:36.200 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:36.198+0000 7fbdeeffd640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdf00a2e30 msgr2=0x7fbdf0132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:36.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:36.198+0000 7fbdeeffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdf00a2e30 0x7fbdf0132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:37:36.201 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:36.198+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdf009fb80 0x7fbdf009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:37:37.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 10 monclient(hunting): tick 2024-09-15T01:37:37.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 1 monclient(hunting): continuing hunt 2024-09-15T01:37:37.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:37:37.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdf00a2e30 msgr2=0x7fbdf0132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:37.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbdf00a2e30 0x7fbdf0132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:37.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdf009fb80 msgr2=0x7fbdf009ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:37.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbdf009fb80 0x7fbdf009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:37.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7fbdf00a0a40 legacy=0x7fbdf00a0ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:37.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 10 monclient: _add_conns ranks=[3,1,9,2,6,5,10,7] 2024-09-15T01:37:37.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdd00037e0 0x7fbdd0005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:37.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 10 monclient(hunting): picked mon.noname-g con 0x7fbdd00037e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:37.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbdd0006220 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:37.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbdd0009f40 0x7fbdd000c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:37.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 10 monclient(hunting): picked mon.noname-a con 0x7fbdd0009f40 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf00a3200 con 0x7fbdd0006220 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf009e6c0 con 0x7fbdd0009f40 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf00a04b0 con 0x7fbdd00037e0 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbded7fa640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdd00037e0 0x7fbdd0005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdef7fe640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7fbdd0009f40 msgr2=0x7fbdd000c3f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbde7fff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fbdd0006220 0x7fbdd00086a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:57474/0 (socket says 172.21.15.204:57474) 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbde7fff640 1 -- 172.21.15.204:0/879293953 learned_addr learned my addr 172.21.15.204:0/879293953 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdef7fe640 1 --2- 172.21.15.204:0/879293953 >> v2:172.21.15.178:3300/0 conn(0x7fbdd0009f40 0x7fbdd000c3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 10 monclient(hunting): get_auth_request con 0x7fbdd00037e0 auth_method 0 2024-09-15T01:37:37.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:37.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:37.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:37.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:37.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:37.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:37.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 10 monclient(hunting): handle_auth_done global_id 15537 payload 274 2024-09-15T01:37:37.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 1 -- 172.21.15.204:0/879293953 >> v2:172.21.15.178:3300/0 conn(0x7fbdd0009f40 msgr2=0x7fbdd000c3f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:37.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 1 --2- 172.21.15.204:0/879293953 >> v2:172.21.15.178:3300/0 conn(0x7fbdd0009f40 0x7fbdd000c3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:37.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 1 -- 172.21.15.204:0/879293953 >> v1:172.21.15.204:6789/0 conn(0x7fbdd0006220 legacy=0x7fbdd00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:37.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 10 monclient: _finish_hunting 0 2024-09-15T01:37:37.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 1 monclient: found mon.noname-g 2024-09-15T01:37:37.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdef7fe640 1 -- 172.21.15.204:0/879293953 reap_dead start 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdee7fc640 1 -- 172.21.15.204:0/879293953 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294043290 0 0) 0x7fbdf00a3200 con 0x7fbdd0006220 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdeeffd640 1 -- 172.21.15.204:0/879293953 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf004b720 con 0x7fbdd00037e0 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.795+0000 7fbdee7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdeeffd640 1 --2- 172.21.15.204:0/879293953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdd00037e0 0x7fbdd0005c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbde0003870 tx=0x7fbde0009220 comp rx=0 tx=0).ready entity=mon.3 client_cookie=83b95033e79ddc0b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdee7fc640 1 -- 172.21.15.204:0/879293953 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbde002ace0 con 0x7fbdd00037e0 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdee7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdee7fc640 10 monclient: dump: 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:37.809 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:37.810 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:37.810 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:37.810 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:37.810 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:37.810 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:37.810 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:37.810 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:37.810 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:37.810 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdee7fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdee7fc640 1 -- 172.21.15.204:0/879293953 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde002ae80 con 0x7fbdd00037e0 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdf52a8640 5 monclient: authenticate success, global_id 15537 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdf52a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdee7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdf52a8640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdee7fc640 1 -- 172.21.15.204:0/879293953 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbde00054f0 con 0x7fbdd00037e0 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:37.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdf52a8640 4 set_mon_vals no callback set 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdee7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.796+0000 7fbdee7fc640 10 monclient: dump: 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:37.812 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.797+0000 7fbdf52a8640 10 monclient: shutdown 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.797+0000 7fbdf52a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.797+0000 7fbdf52a8640 1 -- 172.21.15.204:0/879293953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdd00037e0 msgr2=0x7fbdd0005c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.797+0000 7fbdf52a8640 1 --2- 172.21.15.204:0/879293953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdd00037e0 0x7fbdd0005c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbde0003870 tx=0x7fbde0009220 comp rx=0 tx=0).stop 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.797+0000 7fbdf52a8640 1 -- 172.21.15.204:0/879293953 shutdown_connections 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.797+0000 7fbdf52a8640 1 --2- 172.21.15.204:0/879293953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdd00037e0 0x7fbdd0005c20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.797+0000 7fbdf52a8640 1 -- 172.21.15.204:0/879293953 >> 172.21.15.204:0/879293953 conn(0x7fbdf006e860 msgr2=0x7fbdf006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.797+0000 7fbdf52a8640 1 -- 172.21.15.204:0/879293953 shutdown_connections 2024-09-15T01:37:37.813 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.797+0000 7fbdf52a8640 1 -- 172.21.15.204:0/879293953 wait complete. 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.798+0000 7fbdf52a8640 10 monclient: build_initial_monmap 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.798+0000 7fbdf52a8640 10 monclient: monmap: 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:37.814 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.798+0000 7fbdf52a8640 1 Processor -- start 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 1 -- start start 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 10 monclient: init 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 10 monclient: _add_conns ranks=[6,1,0,2,7,5,4,3,8] 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbdd00037e0 0x7fbdf0131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 10 monclient(hunting): picked mon.noname-d con 0x7fbdd00037e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf0131e60 0x7fbdf011e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 10 monclient(hunting): picked mon.noname-f con 0x7fbdf0131e60 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:37.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf011ea70 0x7fbdf011eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 10 monclient(hunting): picked mon.noname-a con 0x7fbdf011ea70 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf0132230 con 0x7fbdf011ea70 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf011ff40 con 0x7fbdf0131e60 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf0142950 con 0x7fbdd00037e0 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdf52a8640 10 monclient(hunting): authenticate will time out at 1073.442611s 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdef7fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf011ea70 msgr2=0x7fbdf011eea0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf0131e60 0x7fbdf011e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdef7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf011ea70 0x7fbdf011eea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:37.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbde7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf0131e60 0x7fbdf011e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39482/0 (socket says 172.21.15.204:39482) 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbde7fff640 1 -- 172.21.15.204:0/1895546869 learned_addr learned my addr 172.21.15.204:0/1895546869 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbde7fff640 10 monclient(hunting): get_auth_request con 0x7fbdf0131e60 auth_method 0 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbde7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbde7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbde7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbde7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbde7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbde7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdeeffd640 1 --2- 172.21.15.204:0/1895546869 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbdd00037e0 0x7fbdf0131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdeeffd640 10 monclient(hunting): get_auth_request con 0x7fbdd00037e0 auth_method 0 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdeeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdeeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:37.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.799+0000 7fbdeeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:37.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbdeeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:37.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbdeeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:37.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbdeeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:37.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde7fff640 10 monclient(hunting): handle_auth_done global_id 7210 payload 824 2024-09-15T01:37:37.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde7fff640 1 -- 172.21.15.204:0/1895546869 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbdd00037e0 msgr2=0x7fbdf0131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:37.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde7fff640 1 --2- 172.21.15.204:0/1895546869 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbdd00037e0 0x7fbdf0131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:37.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde7fff640 1 -- 172.21.15.204:0/1895546869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf011ea70 msgr2=0x7fbdf011eea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:37.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde7fff640 1 --2- 172.21.15.204:0/1895546869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf011ea70 0x7fbdf011eea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:37.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:37.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde7fff640 1 monclient: found mon.noname-f 2024-09-15T01:37:37.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:37.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde7fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde7fff640 1 -- 172.21.15.204:0/1895546869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde0009500 con 0x7fbdf0131e60 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde7fff640 1 --2- 172.21.15.204:0/1895546869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf0131e60 0x7fbdf011e530 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800b5f0 tx=0x7fbdd8009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde6ffd640 1 -- 172.21.15.204:0/1895546869 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbdd8015070 con 0x7fbdf0131e60 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde6ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde6ffd640 10 monclient: dump: 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:37.819 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde6ffd640 1 -- 172.21.15.204:0/1895546869 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd8003c80 con 0x7fbdf0131e60 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbdf52a8640 5 monclient: authenticate success, global_id 7210 2024-09-15T01:37:37.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbdf52a8640 10 monclient: _renew_subs 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbdf52a8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbdf52a8640 1 -- 172.21.15.204:0/1895546869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf009ca40 con 0x7fbdf0131e60 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde6ffd640 1 -- 172.21.15.204:0/1895546869 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbdd8003e30 con 0x7fbdf0131e60 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbde6ffd640 10 monclient: dump: 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:37.821 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbdf52a8640 10 monclient: _renew_subs 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbdedffb640 4 set_mon_vals no callback set 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbdf52a8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.800+0000 7fbdf52a8640 1 -- 172.21.15.204:0/1895546869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf01206b0 con 0x7fbdf0131e60 2024-09-15T01:37:37.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.801+0000 7fbde6ffd640 1 -- 172.21.15.204:0/1895546869 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fbdd8006150 con 0x7fbdf0131e60 2024-09-15T01:37:37.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.801+0000 7fbdf52a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:37.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.801+0000 7fbdf52a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:37.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.801+0000 7fbdf52a8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:37.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.801+0000 7fbde6ffd640 1 --2- 172.21.15.204:0/1895546869 >> v2:172.21.15.204:6812/28210 conn(0x7fbdd003a440 0x7fbdd003c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:37.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.801+0000 7fbdf52a8640 1 -- 172.21.15.204:0/1895546869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf0070cc0 con 0x7fbdf0131e60 2024-09-15T01:37:37.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.802+0000 7fbde6ffd640 1 -- 172.21.15.204:0/1895546869 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 13916+0+0 (secure 0 0 0) 0x7fbdd804d290 con 0x7fbdf0131e60 2024-09-15T01:37:37.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.802+0000 7fbdeeffd640 1 --2- 172.21.15.204:0/1895546869 >> v2:172.21.15.204:6812/28210 conn(0x7fbdd003a440 0x7fbdd003c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:37.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.802+0000 7fbdeeffd640 10 monclient: get_auth_request con 0x7fbdd003a440 auth_method 0 2024-09-15T01:37:37.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.802+0000 7fbdeeffd640 1 --2- 172.21.15.204:0/1895546869 >> v2:172.21.15.204:6812/28210 conn(0x7fbdd003a440 0x7fbdd003c8f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbde0032000 tx=0x7fbde0030000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:37.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.806+0000 7fbde6ffd640 1 -- 172.21.15.204:0/1895546869 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fbdd8010b40 con 0x7fbdf0131e60 2024-09-15T01:37:37.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.806+0000 7fbde6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:37.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.806+0000 7fbde6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:37.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.970+0000 7fbdf52a8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h2"}] 2024-09-15T01:37:37.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.970+0000 7fbdf52a8640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h2"}] 2024-09-15T01:37:37.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.970+0000 7fbdf52a8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:37.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:37.970+0000 7fbdf52a8640 1 -- 172.21.15.204:0/1895546869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h2"} v 0) v1 -- 0x7fbdf00a1f10 con 0x7fbdf0131e60 2024-09-15T01:37:38.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.028+0000 7fbde6ffd640 1 -- 172.21.15.204:0/1895546869 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h2"}]=0 removed item id -22 name 'h2' from crush map v220) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbdd800eb30 con 0x7fbdf0131e60 2024-09-15T01:37:38.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.028+0000 7fbde6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h2"}] 2024-09-15T01:37:38.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.028+0000 7fbde6ffd640 10 monclient: _finish_command 2 = system:0 removed item id -22 name 'h2' from crush map 2024-09-15T01:37:38.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.129+0000 7fbdf52a8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h2"}] 2024-09-15T01:37:38.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.129+0000 7fbdf52a8640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h2"}] 2024-09-15T01:37:38.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.129+0000 7fbdf52a8640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:38.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.129+0000 7fbdf52a8640 1 -- 172.21.15.204:0/1895546869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h2"} v 0) v1 -- 0x7fbdf006f120 con 0x7fbdf0131e60 2024-09-15T01:37:38.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.129+0000 7fbde6ffd640 1 -- 172.21.15.204:0/1895546869 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h2"}]=0 device 'h2' does not appear in the crush map v220) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbdd8019370 con 0x7fbdf0131e60 2024-09-15T01:37:38.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.129+0000 7fbde6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h2"}] 2024-09-15T01:37:38.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.129+0000 7fbde6ffd640 10 monclient: _finish_command 3 = system:0 device 'h2' does not appear in the crush map 2024-09-15T01:37:38.133 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h2' does not appear in the crush map 2024-09-15T01:37:38.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 1 -- 172.21.15.204:0/1895546869 >> v2:172.21.15.204:6812/28210 conn(0x7fbdd003a440 msgr2=0x7fbdd003c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:38.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 1 --2- 172.21.15.204:0/1895546869 >> v2:172.21.15.204:6812/28210 conn(0x7fbdd003a440 0x7fbdd003c8f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbde0032000 tx=0x7fbde0030000 comp rx=0 tx=0).stop 2024-09-15T01:37:38.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 10 monclient: shutdown 2024-09-15T01:37:38.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:38.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 1 -- 172.21.15.204:0/1895546869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf0131e60 msgr2=0x7fbdf011e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:38.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 1 --2- 172.21.15.204:0/1895546869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf0131e60 0x7fbdf011e530 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800b5f0 tx=0x7fbdd8009120 comp rx=0 tx=0).stop 2024-09-15T01:37:38.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 1 -- 172.21.15.204:0/1895546869 shutdown_connections 2024-09-15T01:37:38.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 1 --2- 172.21.15.204:0/1895546869 >> v2:172.21.15.204:6812/28210 conn(0x7fbdd003a440 0x7fbdd003c8f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:38.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 1 --2- 172.21.15.204:0/1895546869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdf011ea70 0x7fbdf011eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:38.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 1 --2- 172.21.15.204:0/1895546869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf0131e60 0x7fbdf011e530 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:38.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 1 --2- 172.21.15.204:0/1895546869 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbdd00037e0 0x7fbdf0131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:38.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 1 -- 172.21.15.204:0/1895546869 >> 172.21.15.204:0/1895546869 conn(0x7fbdf006e860 msgr2=0x7fbdf009bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:38.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 1 -- 172.21.15.204:0/1895546869 shutdown_connections 2024-09-15T01:37:38.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.131+0000 7fbdf52a8640 1 -- 172.21.15.204:0/1895546869 wait complete. 2024-09-15T01:37:38.143 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:38.143 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h3 2024-09-15T01:37:38.219 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.216+0000 7f0b5f846640 10 monclient: get_monmap_and_config 2024-09-15T01:37:38.219 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.216+0000 7f0b5f846640 10 monclient: build_initial_monmap 2024-09-15T01:37:38.219 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.216+0000 7f0b5f846640 10 monclient: monmap: 2024-09-15T01:37:38.219 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:38.220 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:38.220 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:38.217559+0000 2024-09-15T01:37:38.220 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:38.217559+0000 2024-09-15T01:37:38.220 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:38.220 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:38.220 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:38.220 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:38.220 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:38.220 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:38.221 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:38.221 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:38.221 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:38.221 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:38.221 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:38.221 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:38.221 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:38.221 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:38.221 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.216+0000 7f0b5f846640 1 Processor -- start 2024-09-15T01:37:38.221 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.216+0000 7f0b5f846640 1 -- start start 2024-09-15T01:37:38.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.216+0000 7f0b5f846640 10 monclient: init 2024-09-15T01:37:38.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:38.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 10 monclient: _add_conns ranks=[9,1,6,3,2,5,10,8,7,4,0] 2024-09-15T01:37:38.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0b580a3560 0x7f0b5809b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:38.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 10 monclient(hunting): picked mon.noname-a con 0x7f0b580a3560 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:38.230 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0b5809cca0 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:38.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b5809d090 0x7f0b58067840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:38.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 10 monclient(hunting): picked mon.noname-d con 0x7f0b5809d090 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:38.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5804b720 con 0x7f0b5809cca0 2024-09-15T01:37:38.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:38.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5809e120 con 0x7f0b580a3560 2024-09-15T01:37:38.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:38.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0b58099f10 con 0x7f0b5809d090 2024-09-15T01:37:38.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:38.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5f846640 10 monclient(hunting): authenticate will time out at 1073.860430s 2024-09-15T01:37:38.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5cde1640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f0b5809cca0 0x7f0b58067130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:57490/0 (socket says 172.21.15.204:57490) 2024-09-15T01:37:38.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5d5e2640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7f0b580a3560 msgr2=0x7f0b5809b480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:38.231 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5cde1640 1 -- 172.21.15.204:0/2496875334 learned_addr learned my addr 172.21.15.204:0/2496875334 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:37:38.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5d5e2640 1 --2- 172.21.15.204:0/2496875334 >> v2:172.21.15.178:3300/0 conn(0x7f0b580a3560 0x7f0b5809b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:38.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5dde3640 1 --2- 172.21.15.204:0/2496875334 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b5809d090 0x7f0b58067840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:38.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b4ffff640 1 -- 172.21.15.204:0/2496875334 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188649221 0 0) 0x7f0b5804b720 con 0x7f0b5809cca0 2024-09-15T01:37:38.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5dde3640 10 monclient(hunting): get_auth_request con 0x7f0b5809d090 auth_method 0 2024-09-15T01:37:38.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5dde3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:38.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5dde3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:38.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5dde3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:38.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:38.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:38.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b4ffff640 10 monclient(hunting): my global_id is 7219 2024-09-15T01:37:38.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b4ffff640 1 -- 172.21.15.204:0/2496875334 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b440034e0 con 0x7f0b5809cca0 2024-09-15T01:37:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5dde3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5dde3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.217+0000 7f0b5dde3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.218+0000 7f0b5dde3640 10 monclient(hunting): handle_auth_done global_id 24486 payload 274 2024-09-15T01:37:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.218+0000 7f0b5dde3640 1 -- 172.21.15.204:0/2496875334 >> v2:172.21.15.178:3300/0 conn(0x7f0b580a3560 msgr2=0x7f0b5809b480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.218+0000 7f0b5dde3640 1 --2- 172.21.15.204:0/2496875334 >> v2:172.21.15.178:3300/0 conn(0x7f0b580a3560 0x7f0b5809b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.218+0000 7f0b5dde3640 1 -- 172.21.15.204:0/2496875334 >> v1:172.21.15.204:6789/0 conn(0x7f0b5809cca0 legacy=0x7f0b58067130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.218+0000 7f0b5dde3640 10 monclient: _finish_hunting 0 2024-09-15T01:37:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.218+0000 7f0b5dde3640 1 monclient: found mon.noname-d 2024-09-15T01:37:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.218+0000 7f0b5dde3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.218+0000 7f0b5dde3640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:37:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.218+0000 7f0b5dde3640 1 -- 172.21.15.204:0/2496875334 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5809ab00 con 0x7f0b5809d090 2024-09-15T01:37:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.218+0000 7f0b5dde3640 1 --2- 172.21.15.204:0/2496875334 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b5809d090 0x7f0b58067840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009e60 tx=0x7f0b50009780 comp rx=0 tx=0).ready entity=mon.6 client_cookie=31be93aae6a7e6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b4ffff640 1 -- 172.21.15.204:0/2496875334 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b500079d0 con 0x7f0b5809d090 2024-09-15T01:37:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b4ffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:37:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b4ffff640 10 monclient: dump: 2024-09-15T01:37:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:38.235 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:38.235 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:38.235 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:38.235 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:38.235 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:38.235 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:38.235 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:38.235 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:38.235 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:38.235 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:38.235 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:38.235 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:38.236 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:38.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b4ffff640 10 monclient: _finish_auth 0 2024-09-15T01:37:38.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b4ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:38.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b4ffff640 1 -- 172.21.15.204:0/2496875334 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5002d050 con 0x7f0b5809d090 2024-09-15T01:37:38.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 5 monclient: authenticate success, global_id 24486 2024-09-15T01:37:38.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:38.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b4ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:38.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:38.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b4ffff640 1 -- 172.21.15.204:0/2496875334 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b50031400 con 0x7f0b5809d090 2024-09-15T01:37:38.236 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 4 set_mon_vals no callback set 2024-09-15T01:37:38.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:38.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b4ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:37:38.237 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b4ffff640 10 monclient: dump: 2024-09-15T01:37:38.237 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:38.237 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:38.237 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:38.237 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:38.237 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:38.237 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:38.237 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:38.237 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:38.238 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:38.238 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:38.238 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:38.238 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:38.238 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:38.238 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:38.238 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:38.238 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:38.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 10 monclient: shutdown 2024-09-15T01:37:38.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:38.238 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 1 -- 172.21.15.204:0/2496875334 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b5809d090 msgr2=0x7f0b58067840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:38.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 1 --2- 172.21.15.204:0/2496875334 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b5809d090 0x7f0b58067840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009e60 tx=0x7f0b50009780 comp rx=0 tx=0).stop 2024-09-15T01:37:38.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 1 -- 172.21.15.204:0/2496875334 shutdown_connections 2024-09-15T01:37:38.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 1 --2- 172.21.15.204:0/2496875334 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0b5809d090 0x7f0b58067840 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:38.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 1 --2- 172.21.15.204:0/2496875334 >> v2:172.21.15.178:3300/0 conn(0x7f0b580a3560 0x7f0b5809b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:38.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 1 -- 172.21.15.204:0/2496875334 >> 172.21.15.204:0/2496875334 conn(0x7f0b580687e0 msgr2=0x7f0b580a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:38.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 1 -- 172.21.15.204:0/2496875334 shutdown_connections 2024-09-15T01:37:38.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.219+0000 7f0b5f846640 1 -- 172.21.15.204:0/2496875334 wait complete. 2024-09-15T01:37:38.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.220+0000 7f0b5f846640 10 monclient: build_initial_monmap 2024-09-15T01:37:38.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.220+0000 7f0b5f846640 10 monclient: monmap: 2024-09-15T01:37:38.239 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:38.239 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:38.240 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:38.240 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:38.240 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:38.240 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:38.240 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:38.240 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:38.240 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:38.240 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:38.240 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:38.240 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:38.240 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:38.241 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:38.241 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:38.241 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:38.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.220+0000 7f0b5f846640 1 Processor -- start 2024-09-15T01:37:38.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.220+0000 7f0b5f846640 1 -- start start 2024-09-15T01:37:38.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.220+0000 7f0b5f846640 10 monclient: init 2024-09-15T01:37:38.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:38.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 10 monclient: _add_conns ranks=[8,7,2,1,6,5,3,0,4] 2024-09-15T01:37:38.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0b5809cca0 0x7f0b58120590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:38.241 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 10 monclient(hunting): picked mon.noname-e con 0x7f0b5809cca0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:38.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b5809d090 0x7f0b5813da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:38.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 10 monclient(hunting): picked mon.noname-i con 0x7f0b5809d090 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:38.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b580a3560 0x7f0b58141920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:38.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 10 monclient(hunting): picked mon.noname-b con 0x7f0b580a3560 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:38.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:38.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5806f000 con 0x7f0b580a3560 2024-09-15T01:37:38.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:38.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5806eaa0 con 0x7f0b5809d090 2024-09-15T01:37:38.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:38.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5813ee50 con 0x7f0b5809cca0 2024-09-15T01:37:38.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:38.242 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 10 monclient(hunting): authenticate will time out at 1073.864251s 2024-09-15T01:37:38.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b5809d090 0x7f0b5813da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:38.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5d5e2640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0b5809cca0 msgr2=0x7f0b58120590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:38.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5dde3640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b580a3560 msgr2=0x7f0b58141920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:38.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5d5e2640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0b5809cca0 0x7f0b58120590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:38.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5dde3640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b580a3560 0x7f0b58141920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:38.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b5809d090 0x7f0b5813da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:52672/0 (socket says 172.21.15.204:52672) 2024-09-15T01:37:38.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 1 -- 172.21.15.204:0/3285853875 learned_addr learned my addr 172.21.15.204:0/3285853875 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:38.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 10 monclient(hunting): get_auth_request con 0x7f0b5809d090 auth_method 0 2024-09-15T01:37:38.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:38.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:38.243 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:38.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:38.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:38.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:38.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 10 monclient(hunting): handle_auth_done global_id 24622 payload 824 2024-09-15T01:37:38.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 1 -- 172.21.15.204:0/3285853875 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0b5809cca0 msgr2=0x7f0b58120590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:38.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 1 --2- 172.21.15.204:0/3285853875 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0b5809cca0 0x7f0b58120590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:38.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 1 -- 172.21.15.204:0/3285853875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b580a3560 msgr2=0x7f0b58141920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:38.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 1 --2- 172.21.15.204:0/3285853875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b580a3560 0x7f0b58141920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:38.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 10 monclient: _finish_hunting 0 2024-09-15T01:37:38.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 1 monclient: found mon.noname-i 2024-09-15T01:37:38.244 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:38.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:38.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 1 -- 172.21.15.204:0/3285853875 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b50009a20 con 0x7f0b5809d090 2024-09-15T01:37:38.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5cde1640 1 --2- 172.21.15.204:0/3285853875 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b5809d090 0x7f0b5813da80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800c590 tx=0x7f0b48007d60 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:38.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b4d7fa640 1 -- 172.21.15.204:0/3285853875 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b480070f0 con 0x7f0b5809d090 2024-09-15T01:37:38.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b4d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:38.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b4d7fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:38.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b4d7fa640 10 monclient: dump: 2024-09-15T01:37:38.245 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:38.245 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:38.245 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:38.245 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:38.246 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:38.246 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:38.246 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:38.246 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:38.246 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:38.246 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:38.246 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:38.246 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:38.246 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:38.247 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:38.247 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:38.247 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:38.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b4d7fa640 10 monclient: _finish_auth 0 2024-09-15T01:37:38.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b4d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:38.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b5f846640 5 monclient: authenticate success, global_id 24622 2024-09-15T01:37:38.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b4d7fa640 1 -- 172.21.15.204:0/3285853875 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b480072b0 con 0x7f0b5809d090 2024-09-15T01:37:38.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b4d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:38.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.222+0000 7f0b5f846640 10 monclient: _renew_subs 2024-09-15T01:37:38.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.222+0000 7f0b5f846640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:38.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.221+0000 7f0b4d7fa640 1 -- 172.21.15.204:0/3285853875 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b48012400 con 0x7f0b5809d090 2024-09-15T01:37:38.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.222+0000 7f0b5f846640 1 -- 172.21.15.204:0/3285853875 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b58122b20 con 0x7f0b5809d090 2024-09-15T01:37:38.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.222+0000 7f0b4d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:38.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.222+0000 7f0b4d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:38.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.222+0000 7f0b4f7fe640 4 set_mon_vals no callback set 2024-09-15T01:37:38.248 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.222+0000 7f0b4d7fa640 10 monclient: dump: 2024-09-15T01:37:38.248 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:38.248 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:38.248 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:38.248 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:38.248 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:38.248 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:38.248 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:38.249 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:38.249 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:38.249 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:38.249 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:38.249 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:38.249 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:38.249 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:38.249 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:38.249 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:38.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.222+0000 7f0b5f846640 10 monclient: _renew_subs 2024-09-15T01:37:38.249 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.222+0000 7f0b5f846640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:38.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.222+0000 7f0b5f846640 1 -- 172.21.15.204:0/3285853875 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b58121030 con 0x7f0b5809d090 2024-09-15T01:37:38.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.222+0000 7f0b4d7fa640 1 -- 172.21.15.204:0/3285853875 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0b48007420 con 0x7f0b5809d090 2024-09-15T01:37:38.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.223+0000 7f0b4d7fa640 1 --2- 172.21.15.204:0/3285853875 >> v2:172.21.15.204:6812/28210 conn(0x7f0b3c0397b0 0x7f0b3c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:38.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.223+0000 7f0b4d7fa640 1 -- 172.21.15.204:0/3285853875 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 13706+0+0 (secure 0 0 0) 0x7f0b4804d1e0 con 0x7f0b5809d090 2024-09-15T01:37:38.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.223+0000 7f0b5f846640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:38.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.223+0000 7f0b5f846640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:38.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.223+0000 7f0b5f846640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:38.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.223+0000 7f0b5f846640 1 -- 172.21.15.204:0/3285853875 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b58066b50 con 0x7f0b5809d090 2024-09-15T01:37:38.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.223+0000 7f0b5d5e2640 1 --2- 172.21.15.204:0/3285853875 >> v2:172.21.15.204:6812/28210 conn(0x7f0b3c0397b0 0x7f0b3c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:38.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.223+0000 7f0b5d5e2640 10 monclient: get_auth_request con 0x7f0b3c0397b0 auth_method 0 2024-09-15T01:37:38.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.223+0000 7f0b5d5e2640 1 --2- 172.21.15.204:0/3285853875 >> v2:172.21.15.204:6812/28210 conn(0x7f0b3c0397b0 0x7f0b3c03bc60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0b54006fd0 tx=0x7f0b54006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:38.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.227+0000 7f0b4d7fa640 1 -- 172.21.15.204:0/3285853875 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0b48012560 con 0x7f0b5809d090 2024-09-15T01:37:38.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.227+0000 7f0b4d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:38.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.227+0000 7f0b4d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:38.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.385+0000 7f0b5f846640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h3"}] 2024-09-15T01:37:38.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.385+0000 7f0b5f846640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h3"}] 2024-09-15T01:37:38.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.385+0000 7f0b5f846640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:38.388 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:38.385+0000 7f0b5f846640 1 -- 172.21.15.204:0/3285853875 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h3"} v 0) v1 -- 0x7f0b58070cc0 con 0x7f0b5809d090 2024-09-15T01:37:39.056 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.053+0000 7f0b4d7fa640 1 -- 172.21.15.204:0/3285853875 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h3"}]=0 removed item id -25 name 'h3' from crush map v221) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0b48010af0 con 0x7f0b5809d090 2024-09-15T01:37:39.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.053+0000 7f0b4d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h3"}] 2024-09-15T01:37:39.057 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.053+0000 7f0b4d7fa640 10 monclient: _finish_command 2 = system:0 removed item id -25 name 'h3' from crush map 2024-09-15T01:37:39.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.151+0000 7f0b5f846640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h3"}] 2024-09-15T01:37:39.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.151+0000 7f0b5f846640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h3"}] 2024-09-15T01:37:39.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.151+0000 7f0b5f846640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:39.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.151+0000 7f0b5f846640 1 -- 172.21.15.204:0/3285853875 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h3"} v 0) v1 -- 0x7f0b58071140 con 0x7f0b5809d090 2024-09-15T01:37:39.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.151+0000 7f0b4d7fa640 1 -- 172.21.15.204:0/3285853875 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h3"}]=0 device 'h3' does not appear in the crush map v221) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0b4801a360 con 0x7f0b5809d090 2024-09-15T01:37:39.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.151+0000 7f0b4d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h3"}] 2024-09-15T01:37:39.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.151+0000 7f0b4d7fa640 10 monclient: _finish_command 3 = system:0 device 'h3' does not appear in the crush map 2024-09-15T01:37:39.154 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h3' does not appear in the crush map 2024-09-15T01:37:39.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 1 -- 172.21.15.204:0/3285853875 >> v2:172.21.15.204:6812/28210 conn(0x7f0b3c0397b0 msgr2=0x7f0b3c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:39.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 1 --2- 172.21.15.204:0/3285853875 >> v2:172.21.15.204:6812/28210 conn(0x7f0b3c0397b0 0x7f0b3c03bc60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0b54006fd0 tx=0x7f0b54006d30 comp rx=0 tx=0).stop 2024-09-15T01:37:39.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 10 monclient: shutdown 2024-09-15T01:37:39.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:39.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 1 -- 172.21.15.204:0/3285853875 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b5809d090 msgr2=0x7f0b5813da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:39.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 1 --2- 172.21.15.204:0/3285853875 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b5809d090 0x7f0b5813da80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800c590 tx=0x7f0b48007d60 comp rx=0 tx=0).stop 2024-09-15T01:37:39.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 1 -- 172.21.15.204:0/3285853875 shutdown_connections 2024-09-15T01:37:39.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 1 --2- 172.21.15.204:0/3285853875 >> v2:172.21.15.204:6812/28210 conn(0x7f0b3c0397b0 0x7f0b3c03bc60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:39.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 1 --2- 172.21.15.204:0/3285853875 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0b5809d090 0x7f0b5813da80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:39.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 1 --2- 172.21.15.204:0/3285853875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b580a3560 0x7f0b58141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:39.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 1 --2- 172.21.15.204:0/3285853875 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0b5809cca0 0x7f0b58120590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:39.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 1 -- 172.21.15.204:0/3285853875 >> 172.21.15.204:0/3285853875 conn(0x7f0b580687e0 msgr2=0x7f0b580a1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:39.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 1 -- 172.21.15.204:0/3285853875 shutdown_connections 2024-09-15T01:37:39.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.153+0000 7f0b5f846640 1 -- 172.21.15.204:0/3285853875 wait complete. 2024-09-15T01:37:39.165 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:39.165 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h4 2024-09-15T01:37:39.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.237+0000 7f289899d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:39.239 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.237+0000 7f289899d640 10 monclient: build_initial_monmap 2024-09-15T01:37:39.240 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.237+0000 7f289899d640 10 monclient: monmap: 2024-09-15T01:37:39.240 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:39.240 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:39.240 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:39.238674+0000 2024-09-15T01:37:39.240 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:39.238674+0000 2024-09-15T01:37:39.240 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:39.240 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:39.240 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:39.240 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:39.241 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:39.241 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:39.241 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:39.241 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:39.241 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:39.241 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:39.241 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:39.241 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:39.241 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:39.241 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:39.251 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.237+0000 7f289899d640 1 Processor -- start 2024-09-15T01:37:39.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.237+0000 7f289899d640 1 -- start start 2024-09-15T01:37:39.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.237+0000 7f289899d640 10 monclient: init 2024-09-15T01:37:39.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:39.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 10 monclient: _add_conns ranks=[6,3,4,1,0,9,2,5,8,10,7] 2024-09-15T01:37:39.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f289409f380 0x7f289409e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:39.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 10 monclient(hunting): picked mon.noname-d con 0x7f289409f380 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:39.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 0x7f289409f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:39.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 10 monclient(hunting): picked mon.noname-g con 0x7f289406cc50 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:39.252 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2894130790 0x7f2894130b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:39.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 10 monclient(hunting): picked mon.noname-c con 0x7f2894130790 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:39.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:39.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289406dfc0 con 0x7f289406cc50 2024-09-15T01:37:39.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:39.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f289409ed90 con 0x7f2894130790 2024-09-15T01:37:39.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:39.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f289409fc90 con 0x7f289409f380 2024-09-15T01:37:39.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:39.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289899d640 10 monclient(hunting): authenticate will time out at 1074.881505s 2024-09-15T01:37:39.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2892d9d640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2894130790 msgr2=0x7f2894130b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:39.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2892d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2894130790 0x7f2894130b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:39.253 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 0x7f289409f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:39.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 0x7f289409f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48248/0 (socket says 172.21.15.204:48248) 2024-09-15T01:37:39.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 1 -- 172.21.15.204:0/108247964 learned_addr learned my addr 172.21.15.204:0/108247964 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:39.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 10 monclient(hunting): get_auth_request con 0x7f289406cc50 auth_method 0 2024-09-15T01:37:39.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:39.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:39.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:39.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:39.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:39.254 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:39.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289259c640 1 --2- 172.21.15.204:0/108247964 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f289409f380 0x7f289409e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:39.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289259c640 10 monclient(hunting): get_auth_request con 0x7f289409f380 auth_method 0 2024-09-15T01:37:39.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:39.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:39.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:39.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 10 monclient(hunting): handle_auth_done global_id 15546 payload 274 2024-09-15T01:37:39.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 1 -- 172.21.15.204:0/108247964 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f289409f380 msgr2=0x7f289409e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:39.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 1 --2- 172.21.15.204:0/108247964 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f289409f380 0x7f289409e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:39.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 1 -- 172.21.15.204:0/108247964 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2894130790 msgr2=0x7f2894130b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:39.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 1 --2- 172.21.15.204:0/108247964 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2894130790 0x7f2894130b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:39.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:37:39.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 1 monclient: found mon.noname-g 2024-09-15T01:37:39.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:39.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:39.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f2891d9b640 1 -- 172.21.15.204:0/108247964 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289404b720 con 0x7f289406cc50 2024-09-15T01:37:39.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.238+0000 7f289259c640 1 --2- 172.21.15.204:0/108247964 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f289409f380 0x7f289409e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:37:39.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f2891d9b640 1 --2- 172.21.15.204:0/108247964 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 0x7f289409f750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f287c009940 tx=0x7f287c0091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=cc5644a136119ba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:39.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289159a640 1 -- 172.21.15.204:0/108247964 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f287c02ac80 con 0x7f289406cc50 2024-09-15T01:37:39.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:39.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289159a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:39.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289159a640 10 monclient: dump: 2024-09-15T01:37:39.257 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:39.257 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:39.257 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:39.257 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:39.258 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:39.258 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:39.258 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:39.258 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:39.258 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:39.258 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:39.258 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:39.258 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:39.258 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:39.258 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:39.259 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:39.259 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:39.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289159a640 10 monclient: _finish_auth 0 2024-09-15T01:37:39.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:39.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289159a640 1 -- 172.21.15.204:0/108247964 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c02ae00 con 0x7f289406cc50 2024-09-15T01:37:39.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289899d640 5 monclient: authenticate success, global_id 15546 2024-09-15T01:37:39.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289899d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:39.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:39.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289899d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:39.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289159a640 1 -- 172.21.15.204:0/108247964 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f287c031ca0 con 0x7f289406cc50 2024-09-15T01:37:39.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:39.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289899d640 4 set_mon_vals no callback set 2024-09-15T01:37:39.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289159a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:39.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.239+0000 7f289159a640 10 monclient: dump: 2024-09-15T01:37:39.260 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:39.260 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:39.260 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:39.260 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:39.260 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:39.260 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:39.260 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:39.261 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:39.261 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:39.261 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:39.261 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:39.261 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:39.261 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:39.261 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:39.261 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:39.261 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:39.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 10 monclient: shutdown 2024-09-15T01:37:39.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:39.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 1 -- 172.21.15.204:0/108247964 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 msgr2=0x7f289409f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:39.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 1 --2- 172.21.15.204:0/108247964 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 0x7f289409f750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f287c009940 tx=0x7f287c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:39.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 1 -- 172.21.15.204:0/108247964 shutdown_connections 2024-09-15T01:37:39.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 1 --2- 172.21.15.204:0/108247964 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2894130790 0x7f2894130b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:39.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 1 --2- 172.21.15.204:0/108247964 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 0x7f289409f750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:39.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 1 --2- 172.21.15.204:0/108247964 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f289409f380 0x7f289409e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:39.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 1 -- 172.21.15.204:0/108247964 >> 172.21.15.204:0/108247964 conn(0x7f28940a3e60 msgr2=0x7f28940a4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:39.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 1 -- 172.21.15.204:0/108247964 shutdown_connections 2024-09-15T01:37:39.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 1 -- 172.21.15.204:0/108247964 wait complete. 2024-09-15T01:37:39.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 10 monclient: build_initial_monmap 2024-09-15T01:37:39.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.240+0000 7f289899d640 10 monclient: monmap: 2024-09-15T01:37:39.263 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:39.263 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:39.263 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:39.263 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:39.263 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:39.263 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:39.263 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:39.264 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:39.264 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:39.264 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:39.264 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:39.264 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:39.264 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:39.264 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:39.264 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:39.264 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:39.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 1 Processor -- start 2024-09-15T01:37:39.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 1 -- start start 2024-09-15T01:37:39.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 10 monclient: init 2024-09-15T01:37:39.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:39.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 10 monclient: _add_conns ranks=[3,4,1,2,0,5,6,7,8] 2024-09-15T01:37:39.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 0x7f289413fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:39.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 10 monclient(hunting): picked mon.noname-g con 0x7f289406cc50 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:39.265 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f289409f380 0x7f2894140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:39.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 10 monclient(hunting): picked mon.noname-c con 0x7f289409f380 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:39.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2894130790 0x7f289411d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:39.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 10 monclient(hunting): picked mon.noname-f con 0x7f2894130790 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:39.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:39.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2894120a60 con 0x7f2894130790 2024-09-15T01:37:39.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:39.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289406d630 con 0x7f289406cc50 2024-09-15T01:37:39.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:39.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f289409ac50 con 0x7f289409f380 2024-09-15T01:37:39.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:39.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289899d640 10 monclient(hunting): authenticate will time out at 1074.884768s 2024-09-15T01:37:39.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289259c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 0x7f289413fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:39.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f2891d9b640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f289409f380 msgr2=0x7f2894140330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:39.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f2891d9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f289409f380 0x7f2894140330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:39.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289259c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 0x7f289413fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48256/0 (socket says 172.21.15.204:48256) 2024-09-15T01:37:39.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289259c640 1 -- 172.21.15.204:0/4063698704 learned_addr learned my addr 172.21.15.204:0/4063698704 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:39.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289259c640 10 monclient(hunting): get_auth_request con 0x7f289406cc50 auth_method 0 2024-09-15T01:37:39.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:39.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:39.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:39.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f2892d9d640 1 --2- 172.21.15.204:0/4063698704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2894130790 0x7f289411d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:39.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f2892d9d640 10 monclient(hunting): get_auth_request con 0x7f2894130790 auth_method 0 2024-09-15T01:37:39.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f2892d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:39.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f2892d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:39.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f2892d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:39.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:39.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:39.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f289259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:39.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f2892d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:39.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f2892d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:39.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.241+0000 7f2892d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:39.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289259c640 10 monclient(hunting): handle_auth_done global_id 15555 payload 824 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289259c640 1 -- 172.21.15.204:0/4063698704 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f289409f380 msgr2=0x7f2894140330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289259c640 1 --2- 172.21.15.204:0/4063698704 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f289409f380 0x7f2894140330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289259c640 1 -- 172.21.15.204:0/4063698704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2894130790 msgr2=0x7f289411d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289259c640 1 --2- 172.21.15.204:0/4063698704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2894130790 0x7f289411d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289259c640 10 monclient: _finish_hunting 0 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289259c640 1 monclient: found mon.noname-g 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289259c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289259c640 1 -- 172.21.15.204:0/4063698704 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c009450 con 0x7f289406cc50 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289259c640 1 --2- 172.21.15.204:0/4063698704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 0x7f289413fdf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f288400d1d0 tx=0x7f2884009a90 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f288a7fc640 1 -- 172.21.15.204:0/4063698704 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2884014820 con 0x7f289406cc50 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f288a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:39.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f288a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:39.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f288a7fc640 10 monclient: dump: 2024-09-15T01:37:39.271 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:39.271 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:39.271 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:39.271 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:39.271 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:39.271 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:39.271 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:39.271 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:39.271 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:39.271 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:39.271 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f288a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f288a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f288a7fc640 1 -- 172.21.15.204:0/4063698704 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288400f040 con 0x7f289406cc50 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289899d640 5 monclient: authenticate success, global_id 15555 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289899d640 10 monclient: _renew_subs 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289899d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289899d640 1 -- 172.21.15.204:0/4063698704 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289411ec50 con 0x7f289406cc50 2024-09-15T01:37:39.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f288a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f288a7fc640 1 -- 172.21.15.204:0/4063698704 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2884014e00 con 0x7f289406cc50 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f288a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f288a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f288a7fc640 10 monclient: dump: 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:39.273 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f2890d99640 4 set_mon_vals no callback set 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289899d640 10 monclient: _renew_subs 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289899d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.242+0000 7f289899d640 1 -- 172.21.15.204:0/4063698704 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289411f820 con 0x7f289406cc50 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.243+0000 7f288a7fc640 1 -- 172.21.15.204:0/4063698704 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2884014980 con 0x7f289406cc50 2024-09-15T01:37:39.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.243+0000 7f289899d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:39.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.243+0000 7f289899d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:39.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.243+0000 7f289899d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:39.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.243+0000 7f289899d640 1 -- 172.21.15.204:0/4063698704 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2894070cc0 con 0x7f289406cc50 2024-09-15T01:37:39.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.243+0000 7f288a7fc640 1 --2- 172.21.15.204:0/4063698704 >> v2:172.21.15.204:6812/28210 conn(0x7f28740396e0 0x7f287403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:39.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.243+0000 7f288a7fc640 1 -- 172.21.15.204:0/4063698704 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 13496+0+0 (secure 0 0 0) 0x7f288404d980 con 0x7f289406cc50 2024-09-15T01:37:39.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.243+0000 7f2891d9b640 1 --2- 172.21.15.204:0/4063698704 >> v2:172.21.15.204:6812/28210 conn(0x7f28740396e0 0x7f287403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:39.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.243+0000 7f2891d9b640 10 monclient: get_auth_request con 0x7f28740396e0 auth_method 0 2024-09-15T01:37:39.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.243+0000 7f2891d9b640 1 --2- 172.21.15.204:0/4063698704 >> v2:172.21.15.204:6812/28210 conn(0x7f28740396e0 0x7f287403bb90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f287c034000 tx=0x7f287c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:39.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.247+0000 7f288a7fc640 1 -- 172.21.15.204:0/4063698704 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2884011880 con 0x7f289406cc50 2024-09-15T01:37:39.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.247+0000 7f288a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:39.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.247+0000 7f288a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:39.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.405+0000 7f289899d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h4"}] 2024-09-15T01:37:39.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.405+0000 7f289899d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h4"}] 2024-09-15T01:37:39.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.405+0000 7f289899d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:39.408 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:39.405+0000 7f289899d640 1 -- 172.21.15.204:0/4063698704 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h4"} v 0) v1 -- 0x7f289409ae80 con 0x7f289406cc50 2024-09-15T01:37:40.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.057+0000 7f288a7fc640 1 -- 172.21.15.204:0/4063698704 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h4"}]=0 removed item id -28 name 'h4' from crush map v222) v1 ==== 118+0+0 (secure 0 0 0) 0x7f288401a030 con 0x7f289406cc50 2024-09-15T01:37:40.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.057+0000 7f288a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h4"}] 2024-09-15T01:37:40.060 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.057+0000 7f288a7fc640 10 monclient: _finish_command 2 = system:0 removed item id -28 name 'h4' from crush map 2024-09-15T01:37:40.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.156+0000 7f289899d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h4"}] 2024-09-15T01:37:40.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.156+0000 7f289899d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h4"}] 2024-09-15T01:37:40.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.156+0000 7f289899d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:40.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.156+0000 7f289899d640 1 -- 172.21.15.204:0/4063698704 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h4"} v 0) v1 -- 0x7f28940008d0 con 0x7f289406cc50 2024-09-15T01:37:40.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.157+0000 7f288a7fc640 1 -- 172.21.15.204:0/4063698704 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h4"}]=0 device 'h4' does not appear in the crush map v222) v1 ==== 118+0+0 (secure 0 0 0) 0x7f288401a020 con 0x7f289406cc50 2024-09-15T01:37:40.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.157+0000 7f288a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h4"}] 2024-09-15T01:37:40.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.157+0000 7f288a7fc640 10 monclient: _finish_command 3 = system:0 device 'h4' does not appear in the crush map 2024-09-15T01:37:40.160 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h4' does not appear in the crush map 2024-09-15T01:37:40.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 1 -- 172.21.15.204:0/4063698704 >> v2:172.21.15.204:6812/28210 conn(0x7f28740396e0 msgr2=0x7f287403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:40.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 1 --2- 172.21.15.204:0/4063698704 >> v2:172.21.15.204:6812/28210 conn(0x7f28740396e0 0x7f287403bb90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f287c034000 tx=0x7f287c032000 comp rx=0 tx=0).stop 2024-09-15T01:37:40.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 10 monclient: shutdown 2024-09-15T01:37:40.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:40.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 1 -- 172.21.15.204:0/4063698704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 msgr2=0x7f289413fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:40.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 1 --2- 172.21.15.204:0/4063698704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 0x7f289413fdf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f288400d1d0 tx=0x7f2884009a90 comp rx=0 tx=0).stop 2024-09-15T01:37:40.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 1 -- 172.21.15.204:0/4063698704 shutdown_connections 2024-09-15T01:37:40.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 1 --2- 172.21.15.204:0/4063698704 >> v2:172.21.15.204:6812/28210 conn(0x7f28740396e0 0x7f287403bb90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:40.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 1 --2- 172.21.15.204:0/4063698704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2894130790 0x7f289411d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:40.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 1 --2- 172.21.15.204:0/4063698704 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f289409f380 0x7f2894140330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:40.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 1 --2- 172.21.15.204:0/4063698704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f289406cc50 0x7f289413fdf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:40.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 1 -- 172.21.15.204:0/4063698704 >> 172.21.15.204:0/4063698704 conn(0x7f28940a3e60 msgr2=0x7f289409c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:40.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 1 -- 172.21.15.204:0/4063698704 shutdown_connections 2024-09-15T01:37:40.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.159+0000 7f289899d640 1 -- 172.21.15.204:0/4063698704 wait complete. 2024-09-15T01:37:40.170 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:40.170 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h5 2024-09-15T01:37:40.245 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.242+0000 7fd2dff9d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:40.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.242+0000 7fd2dff9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:40.246 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.242+0000 7fd2dff9d640 10 monclient: monmap: 2024-09-15T01:37:40.246 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:40.246 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:40.246 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:40.244359+0000 2024-09-15T01:37:40.246 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:40.244359+0000 2024-09-15T01:37:40.246 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:40.246 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:40.246 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:40.246 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:40.246 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:40.246 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:40.247 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:40.247 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:40.247 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:40.247 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:40.247 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:40.247 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:40.247 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:40.247 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:40.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.243+0000 7fd2dff9d640 1 Processor -- start 2024-09-15T01:37:40.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.243+0000 7fd2dff9d640 1 -- start start 2024-09-15T01:37:40.247 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.243+0000 7fd2dff9d640 10 monclient: init 2024-09-15T01:37:40.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:40.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 10 monclient: _add_conns ranks=[6,5,10,1,8,7,4,3,9,2,0] 2024-09-15T01:37:40.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd2d80a2e30 0x7fd2d8132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:40.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 10 monclient(hunting): picked mon.noname-d con 0x7fd2d80a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:40.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd2d809fb80 0x7fd2d809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:40.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 10 monclient(hunting): picked mon.noname-h con 0x7fd2d809fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:40.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd2d80a0a40 0x7fd2d80a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:40.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 10 monclient(hunting): picked mon.noname-f con 0x7fd2d80a0a40 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:40.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:40.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d809e6c0 con 0x7fd2d80a0a40 2024-09-15T01:37:40.256 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80a04b0 con 0x7fd2d809fb80 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80a3200 con 0x7fd2d80a2e30 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dff9d640 10 monclient(hunting): authenticate will time out at 1075.887131s 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd2d80a0a40 0x7fd2d80a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dd538640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd2d809fb80 0x7fd2d809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd2d80a0a40 0x7fd2d80a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39514/0 (socket says 172.21.15.204:39514) 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dd538640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd2d809fb80 0x7fd2d809ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:46264/0 (socket says 172.21.15.204:46264) 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 1 -- 172.21.15.204:0/420437776 learned_addr learned my addr 172.21.15.204:0/420437776 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 10 monclient(hunting): get_auth_request con 0x7fd2d80a0a40 auth_method 0 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:40.257 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dd538640 10 monclient(hunting): get_auth_request con 0x7fd2d809fb80 auth_method 0 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dd538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dd538640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dd538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dd538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dd538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dd538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 10 monclient(hunting): handle_auth_done global_id 7237 payload 274 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2ddd39640 1 --2- 172.21.15.204:0/420437776 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd2d80a2e30 0x7fd2d8132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:40.258 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 1 -- 172.21.15.204:0/420437776 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd2d80a2e30 msgr2=0x7fd2d8132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 1 --2- 172.21.15.204:0/420437776 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd2d80a2e30 0x7fd2d8132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 1 -- 172.21.15.204:0/420437776 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd2d809fb80 msgr2=0x7fd2d809ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 1 --2- 172.21.15.204:0/420437776 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd2d809fb80 0x7fd2d809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 10 monclient: _finish_hunting 0 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 1 monclient: found mon.noname-f 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2de53a640 1 -- 172.21.15.204:0/420437776 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d804b720 con 0x7fd2d80a0a40 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2ddd39640 10 monclient: get_auth_request con 0x7fd2d80a2e30 auth_method 0 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2dd538640 1 --2- 172.21.15.204:0/420437776 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd2d809fb80 0x7fd2d809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.244+0000 7fd2ddd39640 1 --2- 172.21.15.204:0/420437776 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd2d80a2e30 0x7fd2d8132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2de53a640 1 --2- 172.21.15.204:0/420437776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d80a0a40 0x7fd2d80a0ee0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009f20 tx=0x7fd2c8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=546bc90c8f0ae04e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dcd37640 1 -- 172.21.15.204:0/420437776 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd2c8007880 con 0x7fd2d80a0a40 2024-09-15T01:37:40.259 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dcd37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dcd37640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dcd37640 10 monclient: dump: 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:40.260 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:40.261 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:40.261 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:40.261 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:40.261 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:40.261 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:40.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dcd37640 10 monclient: _finish_auth 0 2024-09-15T01:37:40.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dcd37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:40.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dcd37640 1 -- 172.21.15.204:0/420437776 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c802d050 con 0x7fd2d80a0a40 2024-09-15T01:37:40.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 5 monclient: authenticate success, global_id 7237 2024-09-15T01:37:40.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:40.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dcd37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:40.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:40.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 4 set_mon_vals no callback set 2024-09-15T01:37:40.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dcd37640 1 -- 172.21.15.204:0/420437776 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd2c8030360 con 0x7fd2d80a0a40 2024-09-15T01:37:40.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dcd37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:40.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dcd37640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:40.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dcd37640 10 monclient: dump: 2024-09-15T01:37:40.262 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:40.262 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:40.262 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:40.262 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:40.262 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:40.262 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:40.262 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 10 monclient: shutdown 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 1 -- 172.21.15.204:0/420437776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d80a0a40 msgr2=0x7fd2d80a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:40.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 1 --2- 172.21.15.204:0/420437776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d80a0a40 0x7fd2d80a0ee0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009f20 tx=0x7fd2c8009840 comp rx=0 tx=0).stop 2024-09-15T01:37:40.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 1 -- 172.21.15.204:0/420437776 shutdown_connections 2024-09-15T01:37:40.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 1 --2- 172.21.15.204:0/420437776 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd2d809fb80 0x7fd2d809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:40.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 1 --2- 172.21.15.204:0/420437776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d80a0a40 0x7fd2d80a0ee0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:40.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 1 --2- 172.21.15.204:0/420437776 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd2d80a2e30 0x7fd2d8132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:40.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.245+0000 7fd2dff9d640 1 -- 172.21.15.204:0/420437776 >> 172.21.15.204:0/420437776 conn(0x7fd2d806e860 msgr2=0x7fd2d806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:40.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.246+0000 7fd2dff9d640 1 -- 172.21.15.204:0/420437776 shutdown_connections 2024-09-15T01:37:40.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.246+0000 7fd2dff9d640 1 -- 172.21.15.204:0/420437776 wait complete. 2024-09-15T01:37:40.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.246+0000 7fd2dff9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:40.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.246+0000 7fd2dff9d640 10 monclient: monmap: 2024-09-15T01:37:40.264 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:40.264 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:40.264 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:40.265 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.246+0000 7fd2dff9d640 1 Processor -- start 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.246+0000 7fd2dff9d640 1 -- start start 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.246+0000 7fd2dff9d640 10 monclient: init 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 10 monclient: _add_conns ranks=[7,6,4,2,8,0,3,5,1] 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd2d809fb80 0x7fd2d81307b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 10 monclient(hunting): picked mon.noname-i con 0x7fd2d809fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd2d80a0a40 0x7fd2d811d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 10 monclient(hunting): picked mon.noname-d con 0x7fd2d80a0a40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd2d80a2e30 0x7fd2d811d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 10 monclient(hunting): picked mon.noname-c con 0x7fd2d80a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:40.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8120f50 con 0x7fd2d80a2e30 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d809ca40 con 0x7fd2d80a0a40 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d809d650 con 0x7fd2d809fb80 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 10 monclient(hunting): authenticate will time out at 1075.890285s 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd2d809fb80 0x7fd2d81307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2de53a640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd2d80a2e30 msgr2=0x7fd2d811d7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd2d809fb80 0x7fd2d81307b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:52688/0 (socket says 172.21.15.204:52688) 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 1 -- 172.21.15.204:0/3000024584 learned_addr learned my addr 172.21.15.204:0/3000024584 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 10 monclient(hunting): get_auth_request con 0x7fd2d809fb80 auth_method 0 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:40.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:40.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:40.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2de53a640 1 --2- 172.21.15.204:0/3000024584 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd2d80a2e30 0x7fd2d811d7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:40.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:40.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:40.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:40.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dd538640 1 --2- 172.21.15.204:0/3000024584 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd2d80a0a40 0x7fd2d811d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:40.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dd538640 10 monclient(hunting): get_auth_request con 0x7fd2d80a0a40 auth_method 0 2024-09-15T01:37:40.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dd538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:40.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dd538640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:40.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dd538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:40.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dd538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:40.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dd538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:40.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dd538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:40.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 10 monclient(hunting): handle_auth_done global_id 24631 payload 824 2024-09-15T01:37:40.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 1 -- 172.21.15.204:0/3000024584 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd2d80a0a40 msgr2=0x7fd2d811d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:40.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 1 --2- 172.21.15.204:0/3000024584 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd2d80a0a40 0x7fd2d811d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:40.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 1 -- 172.21.15.204:0/3000024584 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd2d80a2e30 msgr2=0x7fd2d811d7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:40.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 1 --2- 172.21.15.204:0/3000024584 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd2d80a2e30 0x7fd2d811d7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:40.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 10 monclient: _finish_hunting 0 2024-09-15T01:37:40.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 1 monclient: found mon.noname-i 2024-09-15T01:37:40.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:40.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:40.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 1 -- 172.21.15.204:0/3000024584 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc009390 con 0x7fd2d809fb80 2024-09-15T01:37:40.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2ddd39640 1 --2- 172.21.15.204:0/3000024584 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd2d809fb80 0x7fd2d81307b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000c8e0 tx=0x7fd2d0008040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2d5ffb640 1 -- 172.21.15.204:0/3000024584 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd2d000ce70 con 0x7fd2d809fb80 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2d5ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2d5ffb640 10 monclient: dump: 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:40.270 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2d5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2d5ffb640 1 -- 172.21.15.204:0/3000024584 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d0007cb0 con 0x7fd2d809fb80 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 5 monclient: authenticate success, global_id 24631 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 10 monclient: _renew_subs 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:40.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.247+0000 7fd2dff9d640 1 -- 172.21.15.204:0/3000024584 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c8009ae0 con 0x7fd2d809fb80 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.248+0000 7fd2d5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.248+0000 7fd2d5ffb640 1 -- 172.21.15.204:0/3000024584 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd2d0007e20 con 0x7fd2d809fb80 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.248+0000 7fd2d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.248+0000 7fd2d5ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.248+0000 7fd2d5ffb640 10 monclient: dump: 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:40.272 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.248+0000 7fd2dff9d640 10 monclient: _renew_subs 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.248+0000 7fd2d7fff640 4 set_mon_vals no callback set 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.248+0000 7fd2dff9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.248+0000 7fd2dff9d640 1 -- 172.21.15.204:0/3000024584 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d811ee40 con 0x7fd2d809fb80 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.248+0000 7fd2d5ffb640 1 -- 172.21.15.204:0/3000024584 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd2d0007830 con 0x7fd2d809fb80 2024-09-15T01:37:40.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.249+0000 7fd2dff9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:40.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.249+0000 7fd2dff9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:40.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.249+0000 7fd2dff9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:40.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.249+0000 7fd2dff9d640 1 -- 172.21.15.204:0/3000024584 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d8070cc0 con 0x7fd2d809fb80 2024-09-15T01:37:40.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.249+0000 7fd2d5ffb640 1 --2- 172.21.15.204:0/3000024584 >> v2:172.21.15.204:6812/28210 conn(0x7fd2c40397b0 0x7fd2c403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:40.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.249+0000 7fd2d5ffb640 1 -- 172.21.15.204:0/3000024584 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 13286+0+0 (secure 0 0 0) 0x7fd2d004d650 con 0x7fd2d809fb80 2024-09-15T01:37:40.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.249+0000 7fd2dd538640 1 --2- 172.21.15.204:0/3000024584 >> v2:172.21.15.204:6812/28210 conn(0x7fd2c40397b0 0x7fd2c403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:40.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.249+0000 7fd2dd538640 10 monclient: get_auth_request con 0x7fd2c40397b0 auth_method 0 2024-09-15T01:37:40.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.249+0000 7fd2dd538640 1 --2- 172.21.15.204:0/3000024584 >> v2:172.21.15.204:6812/28210 conn(0x7fd2c40397b0 0x7fd2c403bc60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80a2e00 tx=0x7fd2cc0090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:40.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.253+0000 7fd2d5ffb640 1 -- 172.21.15.204:0/3000024584 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd2d00113f0 con 0x7fd2d809fb80 2024-09-15T01:37:40.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.253+0000 7fd2d5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:40.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.253+0000 7fd2d5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:40.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.413+0000 7fd2dff9d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h5"}] 2024-09-15T01:37:40.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.413+0000 7fd2dff9d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h5"}] 2024-09-15T01:37:40.415 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.413+0000 7fd2dff9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:40.416 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:40.413+0000 7fd2dff9d640 1 -- 172.21.15.204:0/3000024584 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h5"} v 0) v1 -- 0x7fd2d80008d0 con 0x7fd2d809fb80 2024-09-15T01:37:41.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.062+0000 7fd2d5ffb640 1 -- 172.21.15.204:0/3000024584 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h5"}]=0 removed item id -31 name 'h5' from crush map v223) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd2d001a450 con 0x7fd2d809fb80 2024-09-15T01:37:41.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.062+0000 7fd2d5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h5"}] 2024-09-15T01:37:41.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.062+0000 7fd2d5ffb640 10 monclient: _finish_command 2 = system:0 removed item id -31 name 'h5' from crush map 2024-09-15T01:37:41.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.161+0000 7fd2dff9d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h5"}] 2024-09-15T01:37:41.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.161+0000 7fd2dff9d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h5"}] 2024-09-15T01:37:41.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.161+0000 7fd2dff9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:41.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.161+0000 7fd2dff9d640 1 -- 172.21.15.204:0/3000024584 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h5"} v 0) v1 -- 0x7fd2d811f540 con 0x7fd2d809fb80 2024-09-15T01:37:41.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.162+0000 7fd2d5ffb640 1 -- 172.21.15.204:0/3000024584 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h5"}]=0 device 'h5' does not appear in the crush map v223) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd2d00107c0 con 0x7fd2d809fb80 2024-09-15T01:37:41.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.162+0000 7fd2d5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h5"}] 2024-09-15T01:37:41.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.162+0000 7fd2d5ffb640 10 monclient: _finish_command 3 = system:0 device 'h5' does not appear in the crush map 2024-09-15T01:37:41.165 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h5' does not appear in the crush map 2024-09-15T01:37:41.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 1 -- 172.21.15.204:0/3000024584 >> v2:172.21.15.204:6812/28210 conn(0x7fd2c40397b0 msgr2=0x7fd2c403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:41.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 1 --2- 172.21.15.204:0/3000024584 >> v2:172.21.15.204:6812/28210 conn(0x7fd2c40397b0 0x7fd2c403bc60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80a2e00 tx=0x7fd2cc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 10 monclient: shutdown 2024-09-15T01:37:41.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:41.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 1 -- 172.21.15.204:0/3000024584 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd2d809fb80 msgr2=0x7fd2d81307b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:41.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 1 --2- 172.21.15.204:0/3000024584 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd2d809fb80 0x7fd2d81307b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000c8e0 tx=0x7fd2d0008040 comp rx=0 tx=0).stop 2024-09-15T01:37:41.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 1 -- 172.21.15.204:0/3000024584 shutdown_connections 2024-09-15T01:37:41.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 1 --2- 172.21.15.204:0/3000024584 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd2d80a2e30 0x7fd2d811d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 1 --2- 172.21.15.204:0/3000024584 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd2d80a0a40 0x7fd2d811d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 1 --2- 172.21.15.204:0/3000024584 >> v2:172.21.15.204:6812/28210 conn(0x7fd2c40397b0 0x7fd2c403bc60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 1 --2- 172.21.15.204:0/3000024584 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd2d809fb80 0x7fd2d81307b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 1 -- 172.21.15.204:0/3000024584 >> 172.21.15.204:0/3000024584 conn(0x7fd2d806e860 msgr2=0x7fd2d809ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:41.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 1 -- 172.21.15.204:0/3000024584 shutdown_connections 2024-09-15T01:37:41.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.164+0000 7fd2dff9d640 1 -- 172.21.15.204:0/3000024584 wait complete. 2024-09-15T01:37:41.176 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:41.176 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h6 2024-09-15T01:37:41.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.248+0000 7f30ceb9d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:41.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.248+0000 7f30ceb9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:41.250 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.248+0000 7f30ceb9d640 10 monclient: monmap: 2024-09-15T01:37:41.250 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:41.250 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:41.249559+0000 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:41.249559+0000 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:41.251 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:41.252 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:41.252 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:41.252 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:41.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.248+0000 7f30ceb9d640 1 Processor -- start 2024-09-15T01:37:41.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.248+0000 7f30ceb9d640 1 -- start start 2024-09-15T01:37:41.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.248+0000 7f30ceb9d640 10 monclient: init 2024-09-15T01:37:41.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:41.260 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 10 monclient: _add_conns ranks=[10,5,4,6,0,1,7,3,9,2,8] 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f30c80a2e30 0x7f30c8132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 10 monclient(hunting): picked mon.noname-f con 0x7f30c80a2e30 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30c809fb80 0x7f30c809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 10 monclient(hunting): picked mon.noname-h con 0x7f30c809fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30c80a0a40 0x7f30c80a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 10 monclient(hunting): picked mon.noname-c con 0x7f30c80a0a40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c809e6c0 con 0x7f30c80a2e30 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80a04b0 con 0x7f30c80a0a40 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80a3200 con 0x7f30c809fb80 2024-09-15T01:37:41.261 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:41.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30ceb9d640 10 monclient(hunting): authenticate will time out at 1076.892416s 2024-09-15T01:37:41.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cd13a640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30c80a0a40 msgr2=0x7f30c80a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:41.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f30c80a2e30 0x7f30c8132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:41.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cd13a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30c80a0a40 0x7f30c80a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:41.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30c7fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30c809fb80 0x7f30c809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:41.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f30c80a2e30 0x7f30c8132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39520/0 (socket says 172.21.15.204:39520) 2024-09-15T01:37:41.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 1 -- 172.21.15.204:0/1706437571 learned_addr learned my addr 172.21.15.204:0/1706437571 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:41.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 10 monclient(hunting): get_auth_request con 0x7f30c80a2e30 auth_method 0 2024-09-15T01:37:41.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:41.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:41.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:41.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30c7fff640 10 monclient(hunting): get_auth_request con 0x7f30c809fb80 auth_method 0 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 10 monclient(hunting): handle_auth_done global_id 7246 payload 274 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 1 -- 172.21.15.204:0/1706437571 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30c809fb80 msgr2=0x7f30c809ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 1 --2- 172.21.15.204:0/1706437571 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30c809fb80 0x7f30c809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.263 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 1 -- 172.21.15.204:0/1706437571 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30c80a0a40 msgr2=0x7f30c80a0ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 1 --2- 172.21.15.204:0/1706437571 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30c80a0a40 0x7f30c80a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 10 monclient: _finish_hunting 0 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 1 monclient: found mon.noname-f 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30cc939640 1 -- 172.21.15.204:0/1706437571 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c804b720 con 0x7f30c80a2e30 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.249+0000 7f30c7fff640 1 --2- 172.21.15.204:0/1706437571 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30c809fb80 0x7f30c809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30cc939640 1 --2- 172.21.15.204:0/1706437571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c80a2e30 0x7f30c8132a60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009910 tx=0x7f30b80091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25bc2a9e3233d5ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30c77fe640 1 -- 172.21.15.204:0/1706437571 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30b802acb0 con 0x7f30c80a2e30 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30c77fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30c77fe640 10 monclient: dump: 2024-09-15T01:37:41.264 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:41.265 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:41.265 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:41.265 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:41.265 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:41.265 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:41.265 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:41.265 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:41.265 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:41.265 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:41.266 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:41.266 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:41.266 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:41.266 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:41.266 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:41.266 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:41.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30c77fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:41.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:41.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30ceb9d640 5 monclient: authenticate success, global_id 7246 2024-09-15T01:37:41.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30ceb9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:41.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30c77fe640 1 -- 172.21.15.204:0/1706437571 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b802ae30 con 0x7f30c80a2e30 2024-09-15T01:37:41.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30c77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:41.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30ceb9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:41.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30ceb9d640 4 set_mon_vals no callback set 2024-09-15T01:37:41.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30c77fe640 1 -- 172.21.15.204:0/1706437571 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30b8031ca0 con 0x7f30c80a2e30 2024-09-15T01:37:41.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:41.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30c77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:41.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.250+0000 7f30c77fe640 10 monclient: dump: 2024-09-15T01:37:41.267 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:41.267 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:41.267 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:41.267 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:41.267 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:41.267 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:41.268 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:41.268 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:41.268 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:41.268 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:41.268 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:41.268 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:41.268 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:41.268 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:41.268 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:41.268 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:41.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 10 monclient: shutdown 2024-09-15T01:37:41.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 1 -- 172.21.15.204:0/1706437571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c80a2e30 msgr2=0x7f30c8132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 1 --2- 172.21.15.204:0/1706437571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c80a2e30 0x7f30c8132a60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009910 tx=0x7f30b80091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 1 -- 172.21.15.204:0/1706437571 shutdown_connections 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 1 --2- 172.21.15.204:0/1706437571 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30c80a0a40 0x7f30c80a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 1 --2- 172.21.15.204:0/1706437571 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30c809fb80 0x7f30c809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 1 --2- 172.21.15.204:0/1706437571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c80a2e30 0x7f30c8132a60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 1 -- 172.21.15.204:0/1706437571 >> 172.21.15.204:0/1706437571 conn(0x7f30c806e860 msgr2=0x7f30c806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 1 -- 172.21.15.204:0/1706437571 shutdown_connections 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 1 -- 172.21.15.204:0/1706437571 wait complete. 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.251+0000 7f30ceb9d640 10 monclient: monmap: 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:41.269 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:41.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 1 Processor -- start 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 1 -- start start 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 10 monclient: init 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 10 monclient: _add_conns ranks=[1,0,5,8,6,7,4,3,2] 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c809fb80 0x7f30c81318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 10 monclient(hunting): picked mon.noname-f con 0x7f30c809fb80 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c80a0a40 0x7f30c811e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 10 monclient(hunting): picked mon.noname-a con 0x7f30c80a0a40 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30c80a2e30 0x7f30c811e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 10 monclient(hunting): picked mon.noname-h con 0x7f30c80a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c81429e0 con 0x7f30c80a0a40 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:41.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c809d650 con 0x7f30c809fb80 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8121ce0 con 0x7f30c80a2e30 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30ceb9d640 10 monclient(hunting): authenticate will time out at 1076.895720s 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cc939640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c809fb80 0x7f30c81318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cc939640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c809fb80 0x7f30c81318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39534/0 (socket says 172.21.15.204:39534) 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cc939640 1 -- 172.21.15.204:0/2378408922 learned_addr learned my addr 172.21.15.204:0/2378408922 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cc939640 10 monclient(hunting): get_auth_request con 0x7f30c809fb80 auth_method 0 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cc939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cc939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cc939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cd13a640 1 --2- 172.21.15.204:0/2378408922 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30c80a2e30 0x7f30c811e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:41.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cd13a640 10 monclient(hunting): get_auth_request con 0x7f30c80a2e30 auth_method 0 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cd13a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30c7fff640 1 -- 172.21.15.204:0/2378408922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c80a0a40 msgr2=0x7f30c811e3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cd13a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cd13a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30c7fff640 1 --2- 172.21.15.204:0/2378408922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c80a0a40 0x7f30c811e3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cc939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cc939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cc939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cd13a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cd13a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.252+0000 7f30cd13a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30cc939640 10 monclient(hunting): handle_auth_done global_id 7255 payload 824 2024-09-15T01:37:41.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30cc939640 1 -- 172.21.15.204:0/2378408922 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30c80a2e30 msgr2=0x7f30c811e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30cc939640 1 --2- 172.21.15.204:0/2378408922 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30c80a2e30 0x7f30c811e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30cc939640 1 -- 172.21.15.204:0/2378408922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c80a0a40 msgr2=0x7f30c811e3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30cc939640 1 --2- 172.21.15.204:0/2378408922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c80a0a40 0x7f30c811e3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30cc939640 10 monclient: _finish_hunting 0 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30cc939640 1 monclient: found mon.noname-f 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30cc939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30cc939640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30cc939640 1 -- 172.21.15.204:0/2378408922 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc009390 con 0x7f30c809fb80 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30cc939640 1 --2- 172.21.15.204:0/2378408922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c809fb80 0x7f30c81318b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f30b8035040 tx=0x7f30b8032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c4ff9640 1 -- 172.21.15.204:0/2378408922 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30b803bac0 con 0x7f30c809fb80 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c4ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c4ff9640 10 monclient: dump: 2024-09-15T01:37:41.274 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:41.275 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c4ff9640 10 monclient: _finish_auth 0 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c4ff9640 1 -- 172.21.15.204:0/2378408922 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b803bc20 con 0x7f30c809fb80 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30ceb9d640 5 monclient: authenticate success, global_id 7255 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30ceb9d640 10 monclient: _renew_subs 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30ceb9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30ceb9d640 1 -- 172.21.15.204:0/2378408922 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b8009450 con 0x7f30c809fb80 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c4ff9640 1 -- 172.21.15.204:0/2378408922 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30b803bd90 con 0x7f30c809fb80 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c4ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:41.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c4ff9640 10 monclient: dump: 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:41.277 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30ceb9d640 10 monclient: _renew_subs 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30ceb9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30ceb9d640 1 -- 172.21.15.204:0/2378408922 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c8121a00 con 0x7f30c809fb80 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.253+0000 7f30c6ffd640 4 set_mon_vals no callback set 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.254+0000 7f30c4ff9640 1 -- 172.21.15.204:0/2378408922 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f30b8039040 con 0x7f30c809fb80 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.254+0000 7f30ceb9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.254+0000 7f30ceb9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.254+0000 7f30ceb9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.254+0000 7f30ceb9d640 1 -- 172.21.15.204:0/2378408922 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c8070cc0 con 0x7f30c809fb80 2024-09-15T01:37:41.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.254+0000 7f30c4ff9640 1 --2- 172.21.15.204:0/2378408922 >> v2:172.21.15.204:6812/28210 conn(0x7f30b40397b0 0x7f30b403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:41.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.254+0000 7f30c4ff9640 1 -- 172.21.15.204:0/2378408922 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 13076+0+0 (secure 0 0 0) 0x7f30b806cb60 con 0x7f30c809fb80 2024-09-15T01:37:41.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.254+0000 7f30c7fff640 1 --2- 172.21.15.204:0/2378408922 >> v2:172.21.15.204:6812/28210 conn(0x7f30b40397b0 0x7f30b403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:41.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.254+0000 7f30c7fff640 10 monclient: get_auth_request con 0x7f30b40397b0 auth_method 0 2024-09-15T01:37:41.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.254+0000 7f30c7fff640 1 --2- 172.21.15.204:0/2378408922 >> v2:172.21.15.204:6812/28210 conn(0x7f30b40397b0 0x7f30b403bc60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f30c80a2e00 tx=0x7f30bc0090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:41.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.259+0000 7f30c4ff9640 1 -- 172.21.15.204:0/2378408922 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f30b8044ae0 con 0x7f30c809fb80 2024-09-15T01:37:41.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.259+0000 7f30c4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:41.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.259+0000 7f30c4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:41.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.415+0000 7f30ceb9d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h6"}] 2024-09-15T01:37:41.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.415+0000 7f30ceb9d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h6"}] 2024-09-15T01:37:41.417 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.415+0000 7f30ceb9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:41.418 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:41.415+0000 7f30ceb9d640 1 -- 172.21.15.204:0/2378408922 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h6"} v 0) v1 -- 0x7f30c80a1f10 con 0x7f30c809fb80 2024-09-15T01:37:42.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.066+0000 7f30c4ff9640 1 -- 172.21.15.204:0/2378408922 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h6"}]=0 removed item id -34 name 'h6' from crush map v224) v1 ==== 118+0+0 (secure 0 0 0) 0x7f30b8042070 con 0x7f30c809fb80 2024-09-15T01:37:42.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.066+0000 7f30c4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h6"}] 2024-09-15T01:37:42.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.066+0000 7f30c4ff9640 10 monclient: _finish_command 2 = system:0 removed item id -34 name 'h6' from crush map 2024-09-15T01:37:42.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.166+0000 7f30ceb9d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h6"}] 2024-09-15T01:37:42.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.166+0000 7f30ceb9d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h6"}] 2024-09-15T01:37:42.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.166+0000 7f30ceb9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:42.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.166+0000 7f30ceb9d640 1 -- 172.21.15.204:0/2378408922 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h6"} v 0) v1 -- 0x7f30c8120b10 con 0x7f30c809fb80 2024-09-15T01:37:42.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.167+0000 7f30c4ff9640 1 -- 172.21.15.204:0/2378408922 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h6"}]=0 device 'h6' does not appear in the crush map v224) v1 ==== 118+0+0 (secure 0 0 0) 0x7f30b8037030 con 0x7f30c809fb80 2024-09-15T01:37:42.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.167+0000 7f30c4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h6"}] 2024-09-15T01:37:42.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.167+0000 7f30c4ff9640 10 monclient: _finish_command 3 = system:0 device 'h6' does not appear in the crush map 2024-09-15T01:37:42.170 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h6' does not appear in the crush map 2024-09-15T01:37:42.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 1 -- 172.21.15.204:0/2378408922 >> v2:172.21.15.204:6812/28210 conn(0x7f30b40397b0 msgr2=0x7f30b403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:42.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 1 --2- 172.21.15.204:0/2378408922 >> v2:172.21.15.204:6812/28210 conn(0x7f30b40397b0 0x7f30b403bc60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f30c80a2e00 tx=0x7f30bc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 10 monclient: shutdown 2024-09-15T01:37:42.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:42.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 1 -- 172.21.15.204:0/2378408922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c809fb80 msgr2=0x7f30c81318b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:42.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 1 --2- 172.21.15.204:0/2378408922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c809fb80 0x7f30c81318b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f30b8035040 tx=0x7f30b8032000 comp rx=0 tx=0).stop 2024-09-15T01:37:42.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 1 -- 172.21.15.204:0/2378408922 shutdown_connections 2024-09-15T01:37:42.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 1 --2- 172.21.15.204:0/2378408922 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30c80a2e30 0x7f30c811e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 1 --2- 172.21.15.204:0/2378408922 >> v2:172.21.15.204:6812/28210 conn(0x7f30b40397b0 0x7f30b403bc60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 1 --2- 172.21.15.204:0/2378408922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30c80a0a40 0x7f30c811e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 1 --2- 172.21.15.204:0/2378408922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30c809fb80 0x7f30c81318b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 1 -- 172.21.15.204:0/2378408922 >> 172.21.15.204:0/2378408922 conn(0x7f30c806e860 msgr2=0x7f30c809bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:42.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 1 -- 172.21.15.204:0/2378408922 shutdown_connections 2024-09-15T01:37:42.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.169+0000 7f30ceb9d640 1 -- 172.21.15.204:0/2378408922 wait complete. 2024-09-15T01:37:42.180 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:42.180 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h7 2024-09-15T01:37:42.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.253+0000 7fc7f9ca8640 10 monclient: get_monmap_and_config 2024-09-15T01:37:42.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.253+0000 7fc7f9ca8640 10 monclient: build_initial_monmap 2024-09-15T01:37:42.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.253+0000 7fc7f9ca8640 10 monclient: monmap: 2024-09-15T01:37:42.255 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:42.255 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:42.256 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:42.254569+0000 2024-09-15T01:37:42.256 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:42.254569+0000 2024-09-15T01:37:42.256 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:42.256 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:42.256 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:42.256 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:42.256 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:42.256 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:42.256 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:42.256 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:42.256 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:42.257 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:42.257 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:42.257 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:42.257 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:42.257 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:42.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.253+0000 7fc7f9ca8640 1 Processor -- start 2024-09-15T01:37:42.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 1 -- start start 2024-09-15T01:37:42.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 10 monclient: init 2024-09-15T01:37:42.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:42.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 10 monclient: _add_conns ranks=[7,9,10,6,4,1,3,5,0,8,2] 2024-09-15T01:37:42.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc7f406ebb0 0x7fc7f4132f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:42.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 10 monclient(hunting): picked mon.noname-i con 0x7fc7f406ebb0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:42.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc7f406ef80 0x7fc7f406f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:42.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 10 monclient(hunting): picked mon.noname-a con 0x7fc7f406ef80 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:42.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc7f409e850 0x7fc7f409ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:42.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 10 monclient(hunting): picked mon.noname-f con 0x7fc7f409e850 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:42.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:42.267 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f4069e70 con 0x7fc7f406ef80 2024-09-15T01:37:42.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:42.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f40696b0 con 0x7fc7f409e850 2024-09-15T01:37:42.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:42.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f40698b0 con 0x7fc7f406ebb0 2024-09-15T01:37:42.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:42.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f9ca8640 10 monclient(hunting): authenticate will time out at 1077.897542s 2024-09-15T01:37:42.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc7f406ebb0 0x7fc7f4132f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:42.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc7f406ebb0 0x7fc7f4132f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:52700/0 (socket says 172.21.15.204:52700) 2024-09-15T01:37:42.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f3fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc7f409e850 0x7fc7f409ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:42.268 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 1 -- 172.21.15.204:0/3254261863 learned_addr learned my addr 172.21.15.204:0/3254261863 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:42.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 10 monclient(hunting): get_auth_request con 0x7fc7f406ebb0 auth_method 0 2024-09-15T01:37:42.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:42.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:42.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:42.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7ebfff640 1 -- 172.21.15.204:0/3254261863 >> v2:172.21.15.178:3300/0 conn(0x7fc7f406ef80 msgr2=0x7fc7f406f370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:42.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f3fff640 10 monclient(hunting): get_auth_request con 0x7fc7f409e850 auth_method 0 2024-09-15T01:37:42.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:42.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:42.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:42.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7ebfff640 1 --2- 172.21.15.204:0/3254261863 >> v2:172.21.15.178:3300/0 conn(0x7fc7f406ef80 0x7fc7f406f370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:42.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:42.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:42.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:42.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:42.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:42.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:42.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 10 monclient(hunting): handle_auth_done global_id 24640 payload 274 2024-09-15T01:37:42.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 1 -- 172.21.15.204:0/3254261863 >> v2:172.21.15.204:3300/0 conn(0x7fc7f409e850 msgr2=0x7fc7f409ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:42.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 1 --2- 172.21.15.204:0/3254261863 >> v2:172.21.15.204:3300/0 conn(0x7fc7f409e850 0x7fc7f409ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 1 -- 172.21.15.204:0/3254261863 >> v2:172.21.15.178:3300/0 conn(0x7fc7f406ef80 msgr2=0x7fc7f406f370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:42.270 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 1 --2- 172.21.15.204:0/3254261863 >> v2:172.21.15.178:3300/0 conn(0x7fc7f406ef80 0x7fc7f406f370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:42.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 1 monclient: found mon.noname-i 2024-09-15T01:37:42.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:42.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:42.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f37fe640 1 -- 172.21.15.204:0/3254261863 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f404b720 con 0x7fc7f406ebb0 2024-09-15T01:37:42.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.254+0000 7fc7f3fff640 1 --2- 172.21.15.204:0/3254261863 >> v2:172.21.15.204:3300/0 conn(0x7fc7f409e850 0x7fc7f409ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:42.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f37fe640 1 --2- 172.21.15.204:0/3254261863 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc7f406ebb0 0x7fc7f4132f70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40694d0 tx=0x7fc7e40091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=4fdf5537df2ba2b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:42.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f2ffd640 1 -- 172.21.15.204:0/3254261863 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7e402ad60 con 0x7fc7f406ebb0 2024-09-15T01:37:42.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:42.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f2ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:42.271 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f2ffd640 10 monclient: dump: 2024-09-15T01:37:42.271 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:42.272 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:42.272 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:42.272 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:42.272 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:42.272 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:42.272 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:42.272 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:42.272 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:42.272 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:42.272 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:42.272 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:42.273 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:42.273 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:42.273 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:42.273 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:42.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:37:42.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:42.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f2ffd640 1 -- 172.21.15.204:0/3254261863 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e4031820 con 0x7fc7f406ebb0 2024-09-15T01:37:42.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f9ca8640 5 monclient: authenticate success, global_id 24640 2024-09-15T01:37:42.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f9ca8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:42.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:42.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f9ca8640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:42.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f9ca8640 4 set_mon_vals no callback set 2024-09-15T01:37:42.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f2ffd640 1 -- 172.21.15.204:0/3254261863 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7e4031a10 con 0x7fc7f406ebb0 2024-09-15T01:37:42.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:42.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:42.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.255+0000 7fc7f2ffd640 10 monclient: dump: 2024-09-15T01:37:42.274 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:42.274 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:42.274 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:42.274 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:42.274 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:42.274 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:42.275 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:42.275 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:42.275 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:42.275 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:42.275 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:42.275 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:42.275 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:42.275 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:42.275 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:42.275 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:42.275 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 10 monclient: shutdown 2024-09-15T01:37:42.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:42.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/3254261863 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc7f406ebb0 msgr2=0x7fc7f4132f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:42.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 1 --2- 172.21.15.204:0/3254261863 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc7f406ebb0 0x7fc7f4132f70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40694d0 tx=0x7fc7e40091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/3254261863 shutdown_connections 2024-09-15T01:37:42.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 1 --2- 172.21.15.204:0/3254261863 >> v2:172.21.15.204:3300/0 conn(0x7fc7f409e850 0x7fc7f409ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 1 --2- 172.21.15.204:0/3254261863 >> v2:172.21.15.178:3300/0 conn(0x7fc7f406ef80 0x7fc7f406f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 1 --2- 172.21.15.204:0/3254261863 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc7f406ebb0 0x7fc7f4132f70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/3254261863 >> 172.21.15.204:0/3254261863 conn(0x7fc7f40a3e60 msgr2=0x7fc7f40a4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:42.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/3254261863 shutdown_connections 2024-09-15T01:37:42.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/3254261863 wait complete. 2024-09-15T01:37:42.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 10 monclient: build_initial_monmap 2024-09-15T01:37:42.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.256+0000 7fc7f9ca8640 10 monclient: monmap: 2024-09-15T01:37:42.277 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:42.277 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:42.277 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:42.277 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:42.277 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:42.277 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:42.277 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:42.277 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:42.277 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:42.278 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:42.278 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:42.278 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:42.278 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:42.278 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:42.278 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:42.278 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:42.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 1 Processor -- start 2024-09-15T01:37:42.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 1 -- start start 2024-09-15T01:37:42.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 10 monclient: init 2024-09-15T01:37:42.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:42.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 10 monclient: _add_conns ranks=[3,1,5,0,8,7,4,2,6] 2024-09-15T01:37:42.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f406ebb0 0x7fc7f412f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:42.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 10 monclient(hunting): picked mon.noname-g con 0x7fc7f406ebb0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:42.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f406ef80 0x7fc7f411c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:42.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 10 monclient(hunting): picked mon.noname-f con 0x7fc7f406ef80 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:42.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc7f409e850 0x7fc7f411c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:42.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 10 monclient(hunting): picked mon.noname-h con 0x7fc7f409e850 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:42.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:42.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4068db0 con 0x7fc7f406ef80 2024-09-15T01:37:42.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:42.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4068990 con 0x7fc7f406ebb0 2024-09-15T01:37:42.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:42.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4142880 con 0x7fc7f409e850 2024-09-15T01:37:42.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:42.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f9ca8640 10 monclient(hunting): authenticate will time out at 1077.900798s 2024-09-15T01:37:42.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7ebfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f406ef80 0x7fc7f411c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:42.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f37fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f406ebb0 0x7fc7f412f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:42.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f37fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f406ebb0 0x7fc7f412f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48266/0 (socket says 172.21.15.204:48266) 2024-09-15T01:37:42.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f37fe640 1 -- 172.21.15.204:0/4172712894 learned_addr learned my addr 172.21.15.204:0/4172712894 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:42.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f3fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc7f409e850 0x7fc7f411c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:42.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f37fe640 10 monclient(hunting): get_auth_request con 0x7fc7f406ebb0 auth_method 0 2024-09-15T01:37:42.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:42.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:42.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:42.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7ebfff640 10 monclient(hunting): get_auth_request con 0x7fc7f406ef80 auth_method 0 2024-09-15T01:37:42.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:42.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:42.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:42.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f3fff640 10 monclient(hunting): get_auth_request con 0x7fc7f409e850 auth_method 0 2024-09-15T01:37:42.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:42.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:42.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:42.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:42.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.257+0000 7fc7f37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:42.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:42.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:42.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:42.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:42.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:42.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:42.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:42.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f37fe640 10 monclient(hunting): handle_auth_done global_id 15564 payload 824 2024-09-15T01:37:42.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f37fe640 1 -- 172.21.15.204:0/4172712894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc7f409e850 msgr2=0x7fc7f411c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:42.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f37fe640 1 --2- 172.21.15.204:0/4172712894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc7f409e850 0x7fc7f411c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f37fe640 1 -- 172.21.15.204:0/4172712894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f406ef80 msgr2=0x7fc7f411c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:42.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f37fe640 1 --2- 172.21.15.204:0/4172712894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f406ef80 0x7fc7f411c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:42.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:42.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f37fe640 1 monclient: found mon.noname-g 2024-09-15T01:37:42.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:42.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f37fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:42.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f37fe640 1 -- 172.21.15.204:0/4172712894 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec009a20 con 0x7fc7f406ebb0 2024-09-15T01:37:42.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7ebfff640 1 --2- 172.21.15.204:0/4172712894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f406ef80 0x7fc7f411c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:42.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f3fff640 1 --2- 172.21.15.204:0/4172712894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc7f409e850 0x7fc7f411c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:42.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f37fe640 1 --2- 172.21.15.204:0/4172712894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f406ebb0 0x7fc7f412f800 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4035040 tx=0x7fc7e4032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:42.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7eb7fe640 1 -- 172.21.15.204:0/4172712894 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7e4033d30 con 0x7fc7f406ebb0 2024-09-15T01:37:42.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7eb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:42.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7eb7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:42.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7eb7fe640 10 monclient: dump: 2024-09-15T01:37:42.284 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:42.284 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:42.284 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:42.284 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:42.284 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:42.284 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:42.284 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:42.284 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:42.284 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:42.284 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:42.285 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:42.285 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:42.285 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:42.285 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:42.285 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:42.285 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:42.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7eb7fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:42.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:42.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7eb7fe640 1 -- 172.21.15.204:0/4172712894 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e403b3f0 con 0x7fc7f406ebb0 2024-09-15T01:37:42.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f9ca8640 5 monclient: authenticate success, global_id 15564 2024-09-15T01:37:42.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f9ca8640 10 monclient: _renew_subs 2024-09-15T01:37:42.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f9ca8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:42.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/4172712894 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e4009450 con 0x7fc7f406ebb0 2024-09-15T01:37:42.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7eb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:42.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7eb7fe640 1 -- 172.21.15.204:0/4172712894 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7e403b9e0 con 0x7fc7f406ebb0 2024-09-15T01:37:42.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7eb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:42.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7eb7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:42.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7eb7fe640 10 monclient: dump: 2024-09-15T01:37:42.286 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:42.286 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:42.286 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:42.286 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:42.286 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:42.287 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:42.287 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:42.287 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:42.287 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:42.287 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:42.287 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:42.287 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:42.287 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:42.287 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:42.287 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:42.287 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:42.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f27fc640 4 set_mon_vals no callback set 2024-09-15T01:37:42.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f9ca8640 10 monclient: _renew_subs 2024-09-15T01:37:42.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f9ca8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:42.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.258+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/4172712894 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f411f950 con 0x7fc7f406ebb0 2024-09-15T01:37:42.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.259+0000 7fc7eb7fe640 1 -- 172.21.15.204:0/4172712894 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc7e4039040 con 0x7fc7f406ebb0 2024-09-15T01:37:42.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.259+0000 7fc7f9ca8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:42.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.259+0000 7fc7f9ca8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:42.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.259+0000 7fc7f9ca8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:42.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.259+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/4172712894 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f4070cc0 con 0x7fc7f406ebb0 2024-09-15T01:37:42.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.259+0000 7fc7eb7fe640 1 --2- 172.21.15.204:0/4172712894 >> v2:172.21.15.204:6812/28210 conn(0x7fc7d40397b0 0x7fc7d403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:42.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.259+0000 7fc7eb7fe640 1 -- 172.21.15.204:0/4172712894 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 12866+0+0 (secure 0 0 0) 0x7fc7e406ce00 con 0x7fc7f406ebb0 2024-09-15T01:37:42.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.259+0000 7fc7ebfff640 1 --2- 172.21.15.204:0/4172712894 >> v2:172.21.15.204:6812/28210 conn(0x7fc7d40397b0 0x7fc7d403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:42.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.259+0000 7fc7ebfff640 10 monclient: get_auth_request con 0x7fc7d40397b0 auth_method 0 2024-09-15T01:37:42.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.260+0000 7fc7ebfff640 1 --2- 172.21.15.204:0/4172712894 >> v2:172.21.15.204:6812/28210 conn(0x7fc7d40397b0 0x7fc7d403bc60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc006fd0 tx=0x7fc7dc0090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:42.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.264+0000 7fc7eb7fe640 1 -- 172.21.15.204:0/4172712894 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc7e40432b0 con 0x7fc7f406ebb0 2024-09-15T01:37:42.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.264+0000 7fc7eb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:42.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.264+0000 7fc7eb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:42.424 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.422+0000 7fc7f9ca8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h7"}] 2024-09-15T01:37:42.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.422+0000 7fc7f9ca8640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h7"}] 2024-09-15T01:37:42.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.422+0000 7fc7f9ca8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:42.425 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:42.422+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/4172712894 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h7"} v 0) v1 -- 0x7fc7f411de20 con 0x7fc7f406ebb0 2024-09-15T01:37:43.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.082+0000 7fc7eb7fe640 1 -- 172.21.15.204:0/4172712894 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h7"}]=0 removed item id -37 name 'h7' from crush map v225) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc7e4044910 con 0x7fc7f406ebb0 2024-09-15T01:37:43.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.082+0000 7fc7eb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h7"}] 2024-09-15T01:37:43.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.082+0000 7fc7eb7fe640 10 monclient: _finish_command 2 = system:0 removed item id -37 name 'h7' from crush map 2024-09-15T01:37:43.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.178+0000 7fc7f9ca8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h7"}] 2024-09-15T01:37:43.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.178+0000 7fc7f9ca8640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h7"}] 2024-09-15T01:37:43.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.178+0000 7fc7f9ca8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:43.181 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.178+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/4172712894 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h7"} v 0) v1 -- 0x7fc7f4133e20 con 0x7fc7f406ebb0 2024-09-15T01:37:43.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.179+0000 7fc7eb7fe640 1 -- 172.21.15.204:0/4172712894 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h7"}]=0 device 'h7' does not appear in the crush map v225) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc7e4044af0 con 0x7fc7f406ebb0 2024-09-15T01:37:43.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.179+0000 7fc7eb7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h7"}] 2024-09-15T01:37:43.182 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.179+0000 7fc7eb7fe640 10 monclient: _finish_command 3 = system:0 device 'h7' does not appear in the crush map 2024-09-15T01:37:43.182 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h7' does not appear in the crush map 2024-09-15T01:37:43.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/4172712894 >> v2:172.21.15.204:6812/28210 conn(0x7fc7d40397b0 msgr2=0x7fc7d403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:43.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 1 --2- 172.21.15.204:0/4172712894 >> v2:172.21.15.204:6812/28210 conn(0x7fc7d40397b0 0x7fc7d403bc60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc006fd0 tx=0x7fc7dc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:43.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 10 monclient: shutdown 2024-09-15T01:37:43.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:43.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/4172712894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f406ebb0 msgr2=0x7fc7f412f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:43.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 1 --2- 172.21.15.204:0/4172712894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f406ebb0 0x7fc7f412f800 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4035040 tx=0x7fc7e4032000 comp rx=0 tx=0).stop 2024-09-15T01:37:43.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/4172712894 shutdown_connections 2024-09-15T01:37:43.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 1 --2- 172.21.15.204:0/4172712894 >> v2:172.21.15.204:6812/28210 conn(0x7fc7d40397b0 0x7fc7d403bc60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:43.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 1 --2- 172.21.15.204:0/4172712894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc7f409e850 0x7fc7f411c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:43.184 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 1 --2- 172.21.15.204:0/4172712894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f406ef80 0x7fc7f411c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:43.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 1 --2- 172.21.15.204:0/4172712894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f406ebb0 0x7fc7f412f800 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:43.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/4172712894 >> 172.21.15.204:0/4172712894 conn(0x7fc7f40a3e60 msgr2=0x7fc7f406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:43.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/4172712894 shutdown_connections 2024-09-15T01:37:43.185 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.181+0000 7fc7f9ca8640 1 -- 172.21.15.204:0/4172712894 wait complete. 2024-09-15T01:37:43.192 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:43.192 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h8 2024-09-15T01:37:43.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.263+0000 7fcc2ed14640 10 monclient: get_monmap_and_config 2024-09-15T01:37:43.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.263+0000 7fcc2ed14640 10 monclient: build_initial_monmap 2024-09-15T01:37:43.266 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.263+0000 7fcc2ed14640 10 monclient: monmap: 2024-09-15T01:37:43.266 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:43.266 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:43.266 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:43.265313+0000 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:43.265313+0000 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:43.267 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:43.268 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:43.268 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:43.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.264+0000 7fcc2ed14640 1 Processor -- start 2024-09-15T01:37:43.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.264+0000 7fcc2ed14640 1 -- start start 2024-09-15T01:37:43.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.264+0000 7fcc2ed14640 10 monclient: init 2024-09-15T01:37:43.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:43.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 10 monclient: _add_conns ranks=[5,7,0,1,3,8,4,10,9,6,2] 2024-09-15T01:37:43.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc280a2e30 0x7fcc28132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:43.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 10 monclient(hunting): picked mon.noname-h con 0x7fcc280a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:43.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc2809fb80 0x7fcc2809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:43.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 10 monclient(hunting): picked mon.noname-i con 0x7fcc2809fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:43.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcc280a0a40 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:43.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2804b720 con 0x7fcc280a0a40 2024-09-15T01:37:43.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:43.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2809e6c0 con 0x7fcc280a2e30 2024-09-15T01:37:43.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:43.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280a04b0 con 0x7fcc2809fb80 2024-09-15T01:37:43.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:43.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2ed14640 10 monclient(hunting): authenticate will time out at 1078.908212s 2024-09-15T01:37:43.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2cab0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc280a2e30 0x7fcc28132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:43.277 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2d2b1640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7fcc280a0a40 legacy=0x7fcc280a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2cab0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc280a2e30 0x7fcc28132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:46296/0 (socket says 172.21.15.204:46296) 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc2809fb80 0x7fcc2809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2cab0640 1 -- 172.21.15.204:0/2323869586 learned_addr learned my addr 172.21.15.204:0/2323869586 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2cab0640 10 monclient(hunting): get_auth_request con 0x7fcc280a2e30 auth_method 0 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2cab0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2cab0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2cab0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 10 monclient(hunting): get_auth_request con 0x7fcc2809fb80 auth_method 0 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2cab0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:43.278 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2cab0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:43.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2cab0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:43.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:43.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:43.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:43.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 10 monclient(hunting): handle_auth_done global_id 24649 payload 274 2024-09-15T01:37:43.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 1 -- 172.21.15.204:0/2323869586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc280a2e30 msgr2=0x7fcc28132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:43.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 1 --2- 172.21.15.204:0/2323869586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc280a2e30 0x7fcc28132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:43.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 1 -- 172.21.15.204:0/2323869586 >> v1:172.21.15.178:6789/0 conn(0x7fcc280a0a40 legacy=0x7fcc280a0ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:43.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:43.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 1 monclient: found mon.noname-i 2024-09-15T01:37:43.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:43.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc27fff640 1 -- 172.21.15.204:0/2323869586 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc280a3200 con 0x7fcc2809fb80 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.265+0000 7fcc2cab0640 1 --2- 172.21.15.204:0/2323869586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc280a2e30 0x7fcc28132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc27fff640 1 --2- 172.21.15.204:0/2323869586 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc2809fb80 0x7fcc2809ff70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009880 tx=0x7fcc100090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=e74d46d167976c74 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc277fe640 1 -- 172.21.15.204:0/2323869586 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc1002abc0 con 0x7fcc2809fb80 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc277fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc277fe640 10 monclient: dump: 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:43.280 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:43.281 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:43.281 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:43.281 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:43.281 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:43.281 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:43.281 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:43.281 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:43.281 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:43.281 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:43.281 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:43.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc277fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:43.281 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc277fe640 1 -- 172.21.15.204:0/2323869586 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1002ad40 con 0x7fcc2809fb80 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc2ed14640 5 monclient: authenticate success, global_id 24649 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc2ed14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc277fe640 1 -- 172.21.15.204:0/2323869586 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc10031ca0 con 0x7fcc2809fb80 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc277fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc277fe640 10 monclient: dump: 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:43.282 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:43.283 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:43.283 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:43.283 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:43.283 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:43.283 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:43.283 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:43.283 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:43.283 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:43.283 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:43.283 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:43.283 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:43.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc2ed14640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.266+0000 7fcc2ed14640 4 set_mon_vals no callback set 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 10 monclient: shutdown 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 1 -- 172.21.15.204:0/2323869586 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc2809fb80 msgr2=0x7fcc2809ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 1 --2- 172.21.15.204:0/2323869586 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc2809fb80 0x7fcc2809ff70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009880 tx=0x7fcc100090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 1 -- 172.21.15.204:0/2323869586 shutdown_connections 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 1 --2- 172.21.15.204:0/2323869586 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc2809fb80 0x7fcc2809ff70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 1 --2- 172.21.15.204:0/2323869586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc280a2e30 0x7fcc28132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 1 -- 172.21.15.204:0/2323869586 >> 172.21.15.204:0/2323869586 conn(0x7fcc2806e860 msgr2=0x7fcc2806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 1 -- 172.21.15.204:0/2323869586 shutdown_connections 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 1 -- 172.21.15.204:0/2323869586 wait complete. 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 10 monclient: build_initial_monmap 2024-09-15T01:37:43.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 10 monclient: monmap: 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:43.285 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.267+0000 7fcc2ed14640 1 Processor -- start 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 1 -- start start 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 10 monclient: init 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 10 monclient: _add_conns ranks=[0,7,6,8,4,2,5,3,1] 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc2809fb80 0x7fcc28141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 10 monclient(hunting): picked mon.noname-a con 0x7fcc2809fb80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc280a0a40 0x7fcc28131be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 10 monclient(hunting): picked mon.noname-i con 0x7fcc280a0a40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:43.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc280a2e30 0x7fcc2811e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 10 monclient(hunting): picked mon.noname-d con 0x7fcc280a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2809ca40 con 0x7fcc2809fb80 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2809d650 con 0x7fcc280a2e30 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28120dd0 con 0x7fcc280a0a40 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2ed14640 10 monclient(hunting): authenticate will time out at 1078.911694s 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc280a0a40 0x7fcc28131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc280a0a40 0x7fcc28131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:52716/0 (socket says 172.21.15.204:52716) 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 1 -- 172.21.15.204:0/4251010943 learned_addr learned my addr 172.21.15.204:0/4251010943 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 10 monclient(hunting): get_auth_request con 0x7fcc280a0a40 auth_method 0 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:43.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2cab0640 1 -- 172.21.15.204:0/4251010943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc2809fb80 msgr2=0x7fcc28141dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc2cab0640 1 --2- 172.21.15.204:0/4251010943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc2809fb80 0x7fcc28141dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 10 monclient(hunting): handle_auth_done global_id 24658 payload 824 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 1 -- 172.21.15.204:0/4251010943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc280a2e30 msgr2=0x7fcc2811e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.268+0000 7fcc27fff640 1 --2- 172.21.15.204:0/4251010943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc280a2e30 0x7fcc2811e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc27fff640 1 -- 172.21.15.204:0/4251010943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc2809fb80 msgr2=0x7fcc28141dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc27fff640 1 --2- 172.21.15.204:0/4251010943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc2809fb80 0x7fcc28141dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc27fff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:43.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc27fff640 1 monclient: found mon.noname-i 2024-09-15T01:37:43.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:43.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc27fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:43.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc27fff640 1 -- 172.21.15.204:0/4251010943 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc18009ae0 con 0x7fcc280a0a40 2024-09-15T01:37:43.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc27fff640 1 --2- 172.21.15.204:0/4251010943 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc280a0a40 0x7fcc28131be0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009720 tx=0x7fcc10032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:43.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc24ff9640 1 -- 172.21.15.204:0/4251010943 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc1003b870 con 0x7fcc280a0a40 2024-09-15T01:37:43.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc24ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:43.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc24ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:43.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc24ff9640 10 monclient: dump: 2024-09-15T01:37:43.289 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:43.289 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:43.289 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:43.289 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:43.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc24ff9640 10 monclient: _finish_auth 0 2024-09-15T01:37:43.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc24ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:43.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc24ff9640 1 -- 172.21.15.204:0/4251010943 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1003b9f0 con 0x7fcc280a0a40 2024-09-15T01:37:43.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc2ed14640 5 monclient: authenticate success, global_id 24658 2024-09-15T01:37:43.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc2ed14640 10 monclient: _renew_subs 2024-09-15T01:37:43.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc2ed14640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:43.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc2ed14640 1 -- 172.21.15.204:0/4251010943 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc10009390 con 0x7fcc280a0a40 2024-09-15T01:37:43.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc24ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:43.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc24ff9640 1 -- 172.21.15.204:0/4251010943 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc100313a0 con 0x7fcc280a0a40 2024-09-15T01:37:43.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc24ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc24ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc24ff9640 10 monclient: dump: 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:43.293 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc26ffd640 4 set_mon_vals no callback set 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc2ed14640 10 monclient: _renew_subs 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc2ed14640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.269+0000 7fcc2ed14640 1 -- 172.21.15.204:0/4251010943 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc28142c00 con 0x7fcc280a0a40 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.270+0000 7fcc24ff9640 1 -- 172.21.15.204:0/4251010943 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fcc10039040 con 0x7fcc280a0a40 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.270+0000 7fcc2ed14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.270+0000 7fcc2ed14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:43.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.270+0000 7fcc2ed14640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:43.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.270+0000 7fcc2ed14640 1 -- 172.21.15.204:0/4251010943 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc28070cc0 con 0x7fcc280a0a40 2024-09-15T01:37:43.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.270+0000 7fcc24ff9640 1 --2- 172.21.15.204:0/4251010943 >> v2:172.21.15.204:6812/28210 conn(0x7fcc080397b0 0x7fcc0803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:43.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.270+0000 7fcc24ff9640 1 -- 172.21.15.204:0/4251010943 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 12656+0+0 (secure 0 0 0) 0x7fcc1006cd20 con 0x7fcc280a0a40 2024-09-15T01:37:43.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.270+0000 7fcc2cab0640 1 --2- 172.21.15.204:0/4251010943 >> v2:172.21.15.204:6812/28210 conn(0x7fcc080397b0 0x7fcc0803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:43.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.270+0000 7fcc2cab0640 10 monclient: get_auth_request con 0x7fcc080397b0 auth_method 0 2024-09-15T01:37:43.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.270+0000 7fcc2cab0640 1 --2- 172.21.15.204:0/4251010943 >> v2:172.21.15.204:6812/28210 conn(0x7fcc080397b0 0x7fcc0803bc60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcc18007250 tx=0x7fcc1800b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:43.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.274+0000 7fcc24ff9640 1 -- 172.21.15.204:0/4251010943 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcc10030b20 con 0x7fcc280a0a40 2024-09-15T01:37:43.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.274+0000 7fcc24ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:43.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.274+0000 7fcc24ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:43.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.432+0000 7fcc2ed14640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h8"}] 2024-09-15T01:37:43.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.432+0000 7fcc2ed14640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h8"}] 2024-09-15T01:37:43.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.432+0000 7fcc2ed14640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:43.435 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:43.432+0000 7fcc2ed14640 1 -- 172.21.15.204:0/4251010943 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h8"} v 0) v1 -- 0x7fcc28070cc0 con 0x7fcc280a0a40 2024-09-15T01:37:44.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.083+0000 7fcc24ff9640 1 -- 172.21.15.204:0/4251010943 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h8"}]=0 removed item id -40 name 'h8' from crush map v226) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcc10042090 con 0x7fcc280a0a40 2024-09-15T01:37:44.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.083+0000 7fcc24ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h8"}] 2024-09-15T01:37:44.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.083+0000 7fcc24ff9640 10 monclient: _finish_command 2 = system:0 removed item id -40 name 'h8' from crush map 2024-09-15T01:37:44.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.184+0000 7fcc2ed14640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h8"}] 2024-09-15T01:37:44.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.184+0000 7fcc2ed14640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h8"}] 2024-09-15T01:37:44.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.184+0000 7fcc2ed14640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:44.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.184+0000 7fcc2ed14640 1 -- 172.21.15.204:0/4251010943 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h8"} v 0) v1 -- 0x7fcc28070cc0 con 0x7fcc280a0a40 2024-09-15T01:37:44.187 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.185+0000 7fcc24ff9640 1 -- 172.21.15.204:0/4251010943 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h8"}]=0 device 'h8' does not appear in the crush map v226) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcc10042090 con 0x7fcc280a0a40 2024-09-15T01:37:44.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.185+0000 7fcc24ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h8"}] 2024-09-15T01:37:44.188 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.185+0000 7fcc24ff9640 10 monclient: _finish_command 3 = system:0 device 'h8' does not appear in the crush map 2024-09-15T01:37:44.188 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h8' does not appear in the crush map 2024-09-15T01:37:44.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 1 -- 172.21.15.204:0/4251010943 >> v2:172.21.15.204:6812/28210 conn(0x7fcc080397b0 msgr2=0x7fcc0803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:44.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 1 --2- 172.21.15.204:0/4251010943 >> v2:172.21.15.204:6812/28210 conn(0x7fcc080397b0 0x7fcc0803bc60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcc18007250 tx=0x7fcc1800b040 comp rx=0 tx=0).stop 2024-09-15T01:37:44.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 10 monclient: shutdown 2024-09-15T01:37:44.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:44.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 1 -- 172.21.15.204:0/4251010943 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc280a0a40 msgr2=0x7fcc28131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:44.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 1 --2- 172.21.15.204:0/4251010943 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc280a0a40 0x7fcc28131be0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009720 tx=0x7fcc10032000 comp rx=0 tx=0).stop 2024-09-15T01:37:44.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 1 -- 172.21.15.204:0/4251010943 shutdown_connections 2024-09-15T01:37:44.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 1 --2- 172.21.15.204:0/4251010943 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcc280a2e30 0x7fcc2811e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:44.190 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 1 --2- 172.21.15.204:0/4251010943 >> v2:172.21.15.204:6812/28210 conn(0x7fcc080397b0 0x7fcc0803bc60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:44.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 1 --2- 172.21.15.204:0/4251010943 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcc280a0a40 0x7fcc28131be0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:44.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 1 --2- 172.21.15.204:0/4251010943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc2809fb80 0x7fcc28141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:44.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 1 -- 172.21.15.204:0/4251010943 >> 172.21.15.204:0/4251010943 conn(0x7fcc2806e860 msgr2=0x7fcc2809ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:44.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 1 -- 172.21.15.204:0/4251010943 shutdown_connections 2024-09-15T01:37:44.191 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.187+0000 7fcc2ed14640 1 -- 172.21.15.204:0/4251010943 wait complete. 2024-09-15T01:37:44.199 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:44.199 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h9 2024-09-15T01:37:44.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.271+0000 7f8cdb35d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:44.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.271+0000 7f8cdb35d640 10 monclient: build_initial_monmap 2024-09-15T01:37:44.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.271+0000 7f8cdb35d640 10 monclient: monmap: 2024-09-15T01:37:44.274 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:44.274 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:44.274 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:44.272601+0000 2024-09-15T01:37:44.274 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:44.272601+0000 2024-09-15T01:37:44.274 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:44.274 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:44.274 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:44.274 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:44.274 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:44.274 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:44.274 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:44.275 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:44.275 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:44.275 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:44.275 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:44.275 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:44.275 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:44.275 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:44.283 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.271+0000 7f8cdb35d640 1 Processor -- start 2024-09-15T01:37:44.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.271+0000 7f8cdb35d640 1 -- start start 2024-09-15T01:37:44.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.271+0000 7f8cdb35d640 10 monclient: init 2024-09-15T01:37:44.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:44.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 10 monclient: _add_conns ranks=[7,0,5,10,3,1,8,9,2,6,4] 2024-09-15T01:37:44.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8cd40a4680 0x7f8cd409e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:44.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 10 monclient(hunting): picked mon.noname-i con 0x7f8cd40a4680 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:44.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8cd406eea0 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:44.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cd4135bd0 0x7f8cd4137fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:44.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 10 monclient(hunting): picked mon.noname-h con 0x7f8cd4135bd0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:44.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd404b720 con 0x7f8cd406eea0 2024-09-15T01:37:44.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:44.284 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40a4a50 con 0x7f8cd4135bd0 2024-09-15T01:37:44.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:44.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd406dc70 con 0x7f8cd40a4680 2024-09-15T01:37:44.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:44.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cdb35d640 10 monclient(hunting): authenticate will time out at 1079.915426s 2024-09-15T01:37:44.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd88f8640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f8cd406eea0 legacy=0x7f8cd409ed90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:44.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cd4135bd0 0x7f8cd4137fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:44.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd90f9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8cd40a4680 0x7f8cd409e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:44.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cd4135bd0 0x7f8cd4137fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:46310/0 (socket says 172.21.15.204:46310) 2024-09-15T01:37:44.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd90f9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8cd40a4680 0x7f8cd409e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:52732/0 (socket says 172.21.15.204:52732) 2024-09-15T01:37:44.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 1 -- 172.21.15.204:0/1105242556 learned_addr learned my addr 172.21.15.204:0/1105242556 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:44.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 10 monclient(hunting): get_auth_request con 0x7f8cd4135bd0 auth_method 0 2024-09-15T01:37:44.285 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd90f9640 10 monclient(hunting): get_auth_request con 0x7f8cd40a4680 auth_method 0 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd90f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd90f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd90f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd90f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd90f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd90f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:44.286 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 10 monclient(hunting): handle_auth_done global_id 6422 payload 274 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 1 -- 172.21.15.204:0/1105242556 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8cd40a4680 msgr2=0x7f8cd409e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 1 --2- 172.21.15.204:0/1105242556 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8cd40a4680 0x7f8cd409e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 1 -- 172.21.15.204:0/1105242556 >> v1:172.21.15.178:6789/0 conn(0x7f8cd406eea0 legacy=0x7f8cd409ed90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 10 monclient: _finish_hunting 0 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 1 monclient: found mon.noname-h 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd98fa640 1 -- 172.21.15.204:0/1105242556 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd40a0710 con 0x7f8cd4135bd0 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.272+0000 7f8cd90f9640 1 --2- 172.21.15.204:0/1105242556 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8cd40a4680 0x7f8cd409e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8cd98fa640 1 --2- 172.21.15.204:0/1105242556 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cd4135bd0 0x7f8cd4137fb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009f50 tx=0x7f8ccc009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=8c2edf3681c7c497 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8ccbfff640 1 -- 172.21.15.204:0/1105242556 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ccc007880 con 0x7f8cd4135bd0 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8ccbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:44.287 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8ccbfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8ccbfff640 10 monclient: dump: 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:44.288 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8ccbfff640 10 monclient: _finish_auth 0 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8ccbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8ccbfff640 1 -- 172.21.15.204:0/1105242556 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc02d050 con 0x7f8cd4135bd0 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8cdb35d640 5 monclient: authenticate success, global_id 6422 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8cdb35d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8ccbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8cdb35d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8cdb35d640 4 set_mon_vals no callback set 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8ccbfff640 1 -- 172.21.15.204:0/1105242556 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ccc007a80 con 0x7f8cd4135bd0 2024-09-15T01:37:44.289 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8ccbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8ccbfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.273+0000 7f8ccbfff640 10 monclient: dump: 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:44.290 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.274+0000 7f8cdb35d640 10 monclient: shutdown 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.274+0000 7f8cdb35d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.274+0000 7f8cdb35d640 1 -- 172.21.15.204:0/1105242556 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cd4135bd0 msgr2=0x7f8cd4137fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.274+0000 7f8cdb35d640 1 --2- 172.21.15.204:0/1105242556 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cd4135bd0 0x7f8cd4137fb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009f50 tx=0x7f8ccc009840 comp rx=0 tx=0).stop 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.274+0000 7f8cdb35d640 1 -- 172.21.15.204:0/1105242556 shutdown_connections 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.274+0000 7f8cdb35d640 1 --2- 172.21.15.204:0/1105242556 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cd4135bd0 0x7f8cd4137fb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.274+0000 7f8cdb35d640 1 --2- 172.21.15.204:0/1105242556 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8cd40a4680 0x7f8cd409e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.274+0000 7f8cdb35d640 1 -- 172.21.15.204:0/1105242556 >> 172.21.15.204:0/1105242556 conn(0x7f8cd4067740 msgr2=0x7f8cd40a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.274+0000 7f8cdb35d640 1 -- 172.21.15.204:0/1105242556 shutdown_connections 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.274+0000 7f8cdb35d640 1 -- 172.21.15.204:0/1105242556 wait complete. 2024-09-15T01:37:44.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.274+0000 7f8cdb35d640 10 monclient: build_initial_monmap 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.274+0000 7f8cdb35d640 10 monclient: monmap: 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:44.292 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:44.293 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:44.293 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:44.293 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:44.293 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:44.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 1 Processor -- start 2024-09-15T01:37:44.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 1 -- start start 2024-09-15T01:37:44.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 10 monclient: init 2024-09-15T01:37:44.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:44.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 10 monclient: _add_conns ranks=[0,1,8,6,5,2,3,7,4] 2024-09-15T01:37:44.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd406eea0 0x7f8cd409b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:44.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 10 monclient(hunting): picked mon.noname-a con 0x7f8cd406eea0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:44.293 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd40a4680 0x7f8cd409bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 10 monclient(hunting): picked mon.noname-f con 0x7f8cd40a4680 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cd4135bd0 0x7f8cd409c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 10 monclient(hunting): picked mon.noname-e con 0x7f8cd4135bd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd4132be0 con 0x7f8cd406eea0 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41354e0 con 0x7f8cd40a4680 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd406d100 con 0x7f8cd4135bd0 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cdb35d640 10 monclient(hunting): authenticate will time out at 1079.918879s 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cd88f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd40a4680 0x7f8cd409bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cd90f9640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd406eea0 msgr2=0x7f8cd409b9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:44.294 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cd88f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd40a4680 0x7f8cd409bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39568/0 (socket says 172.21.15.204:39568) 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cd88f8640 1 -- 172.21.15.204:0/207039161 learned_addr learned my addr 172.21.15.204:0/207039161 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cd88f8640 10 monclient(hunting): get_auth_request con 0x7f8cd40a4680 auth_method 0 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cd98fa640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cd4135bd0 msgr2=0x7f8cd409c420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cd88f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cd88f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cd88f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.275+0000 7f8cd90f9640 1 --2- 172.21.15.204:0/207039161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd406eea0 0x7f8cd409b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd98fa640 1 --2- 172.21.15.204:0/207039161 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cd4135bd0 0x7f8cd409c420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 10 monclient(hunting): handle_auth_done global_id 7282 payload 824 2024-09-15T01:37:44.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 1 -- 172.21.15.204:0/207039161 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cd4135bd0 msgr2=0x7f8cd409c420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 1 --2- 172.21.15.204:0/207039161 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cd4135bd0 0x7f8cd409c420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 1 -- 172.21.15.204:0/207039161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd406eea0 msgr2=0x7f8cd409b9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 1 --2- 172.21.15.204:0/207039161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd406eea0 0x7f8cd409b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 10 monclient: _finish_hunting 0 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 1 monclient: found mon.noname-f 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 1 -- 172.21.15.204:0/207039161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd0009390 con 0x7f8cd40a4680 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cd88f8640 1 --2- 172.21.15.204:0/207039161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd40a4680 0x7f8cd409bee0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400b5c0 tx=0x7f8cc4009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cc97fa640 1 -- 172.21.15.204:0/207039161 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8cc4006150 con 0x7f8cd40a4680 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cc97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cc97fa640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:44.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cc97fa640 10 monclient: dump: 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:44.297 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cc97fa640 10 monclient: _finish_auth 0 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cc97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cc97fa640 1 -- 172.21.15.204:0/207039161 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc40062f0 con 0x7f8cd40a4680 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cdb35d640 5 monclient: authenticate success, global_id 7282 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cdb35d640 10 monclient: _renew_subs 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cdb35d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cdb35d640 1 -- 172.21.15.204:0/207039161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc009ae0 con 0x7f8cd40a4680 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cc97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cc97fa640 1 -- 172.21.15.204:0/207039161 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8cc40103c0 con 0x7f8cd40a4680 2024-09-15T01:37:44.298 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cc97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cc97fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cc97fa640 10 monclient: dump: 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:44.299 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8ccb7fe640 4 set_mon_vals no callback set 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cdb35d640 10 monclient: _renew_subs 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cdb35d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.276+0000 7f8cdb35d640 1 -- 172.21.15.204:0/207039161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd4132560 con 0x7f8cd40a4680 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.277+0000 7f8cc97fa640 1 -- 172.21.15.204:0/207039161 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8cc40064a0 con 0x7f8cd40a4680 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.277+0000 7f8cc97fa640 1 --2- 172.21.15.204:0/207039161 >> v2:172.21.15.204:6812/28210 conn(0x7f8cb80397b0 0x7f8cb803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.277+0000 7f8cc97fa640 1 -- 172.21.15.204:0/207039161 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 12446+0+0 (secure 0 0 0) 0x7f8cc404c610 con 0x7f8cd40a4680 2024-09-15T01:37:44.300 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.277+0000 7f8cdb35d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:44.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.277+0000 7f8cdb35d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:44.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.277+0000 7f8cdb35d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:44.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.277+0000 7f8cd90f9640 1 --2- 172.21.15.204:0/207039161 >> v2:172.21.15.204:6812/28210 conn(0x7f8cb80397b0 0x7f8cb803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:44.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.277+0000 7f8cdb35d640 1 -- 172.21.15.204:0/207039161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd4132960 con 0x7f8cd40a4680 2024-09-15T01:37:44.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.277+0000 7f8cd90f9640 10 monclient: get_auth_request con 0x7f8cb80397b0 auth_method 0 2024-09-15T01:37:44.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.278+0000 7f8cd90f9640 1 --2- 172.21.15.204:0/207039161 >> v2:172.21.15.204:6812/28210 conn(0x7f8cb80397b0 0x7f8cb803bc60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00095c0 tx=0x7f8cd00090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:44.301 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.282+0000 7f8cc97fa640 1 -- 172.21.15.204:0/207039161 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8cc4017030 con 0x7f8cd40a4680 2024-09-15T01:37:44.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.282+0000 7f8cc97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:44.302 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.282+0000 7f8cc97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:44.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.438+0000 7f8cdb35d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h9"}] 2024-09-15T01:37:44.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.438+0000 7f8cdb35d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h9"}] 2024-09-15T01:37:44.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.438+0000 7f8cdb35d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:44.441 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:44.438+0000 7f8cdb35d640 1 -- 172.21.15.204:0/207039161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h9"} v 0) v1 -- 0x7f8cd4070cc0 con 0x7f8cd40a4680 2024-09-15T01:37:45.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.103+0000 7f8cc97fa640 1 -- 172.21.15.204:0/207039161 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h9"}]=0 removed item id -43 name 'h9' from crush map v227) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8cc4017020 con 0x7f8cd40a4680 2024-09-15T01:37:45.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.103+0000 7f8cc97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h9"}] 2024-09-15T01:37:45.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.103+0000 7f8cc97fa640 10 monclient: _finish_command 2 = system:0 removed item id -43 name 'h9' from crush map 2024-09-15T01:37:45.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.202+0000 7f8cdb35d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h9"}] 2024-09-15T01:37:45.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.202+0000 7f8cdb35d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h9"}] 2024-09-15T01:37:45.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.202+0000 7f8cdb35d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:45.205 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.202+0000 7f8cdb35d640 1 -- 172.21.15.204:0/207039161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h9"} v 0) v1 -- 0x7f8cd409db50 con 0x7f8cd40a4680 2024-09-15T01:37:45.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.203+0000 7f8cc97fa640 1 -- 172.21.15.204:0/207039161 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h9"}]=0 device 'h9' does not appear in the crush map v227) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8cc400d6b0 con 0x7f8cd40a4680 2024-09-15T01:37:45.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.203+0000 7f8cc97fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h9"}] 2024-09-15T01:37:45.206 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.203+0000 7f8cc97fa640 10 monclient: _finish_command 3 = system:0 device 'h9' does not appear in the crush map 2024-09-15T01:37:45.206 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h9' does not appear in the crush map 2024-09-15T01:37:45.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 1 -- 172.21.15.204:0/207039161 >> v2:172.21.15.204:6812/28210 conn(0x7f8cb80397b0 msgr2=0x7f8cb803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:45.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 1 --2- 172.21.15.204:0/207039161 >> v2:172.21.15.204:6812/28210 conn(0x7f8cb80397b0 0x7f8cb803bc60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00095c0 tx=0x7f8cd00090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 10 monclient: shutdown 2024-09-15T01:37:45.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:45.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 1 -- 172.21.15.204:0/207039161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd40a4680 msgr2=0x7f8cd409bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:45.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 1 --2- 172.21.15.204:0/207039161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd40a4680 0x7f8cd409bee0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400b5c0 tx=0x7f8cc4009120 comp rx=0 tx=0).stop 2024-09-15T01:37:45.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 1 -- 172.21.15.204:0/207039161 shutdown_connections 2024-09-15T01:37:45.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 1 --2- 172.21.15.204:0/207039161 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cd4135bd0 0x7f8cd409c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 1 --2- 172.21.15.204:0/207039161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd40a4680 0x7f8cd409bee0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 1 --2- 172.21.15.204:0/207039161 >> v2:172.21.15.204:6812/28210 conn(0x7f8cb80397b0 0x7f8cb803bc60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 1 --2- 172.21.15.204:0/207039161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd406eea0 0x7f8cd409b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 1 -- 172.21.15.204:0/207039161 >> 172.21.15.204:0/207039161 conn(0x7f8cd4067740 msgr2=0x7f8cd40a1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:45.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 1 -- 172.21.15.204:0/207039161 shutdown_connections 2024-09-15T01:37:45.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.205+0000 7f8cdb35d640 1 -- 172.21.15.204:0/207039161 wait complete. 2024-09-15T01:37:45.217 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:45.217 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h10 2024-09-15T01:37:45.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.288+0000 7f710d02b640 10 monclient: get_monmap_and_config 2024-09-15T01:37:45.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.288+0000 7f710d02b640 10 monclient: build_initial_monmap 2024-09-15T01:37:45.291 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.289+0000 7f710d02b640 10 monclient: monmap: 2024-09-15T01:37:45.291 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:45.290461+0000 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:45.290461+0000 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:45.292 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:45.293 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:45.293 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:45.293 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:45.293 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:45.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.289+0000 7f710d02b640 1 Processor -- start 2024-09-15T01:37:45.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.289+0000 7f710d02b640 1 -- start start 2024-09-15T01:37:45.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.289+0000 7f710d02b640 10 monclient: init 2024-09-15T01:37:45.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.289+0000 7f710d02b640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:45.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.289+0000 7f710d02b640 10 monclient: _add_conns ranks=[8,1,4,0,7,10,9,2,6,5,3] 2024-09-15T01:37:45.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.289+0000 7f710d02b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71080a2e30 0x7f7108132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:45.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 10 monclient(hunting): picked mon.noname-e con 0x7f71080a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:45.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f710809fb80 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:45.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f710809ff70 0x7f71080a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:45.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 10 monclient(hunting): picked mon.noname-c con 0x7f710809ff70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710804b720 con 0x7f710809fb80 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f710809e6c0 con 0x7f710809ff70 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f71080a3200 con 0x7f71080a2e30 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 10 monclient(hunting): authenticate will time out at 1080.933061s 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7106d9d640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71080a2e30 msgr2=0x7f7108132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7106d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71080a2e30 0x7f7108132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710759e640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f710809ff70 msgr2=0x7f71080a1150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710659c640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f710809fb80 0x7f71080a0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:57494/0 (socket says 172.21.15.204:57494) 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710659c640 1 -- 172.21.15.204:0/2155607181 learned_addr learned my addr 172.21.15.204:0/2155607181 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:37:45.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710759e640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f710809ff70 0x7f71080a1150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710259346 0 0) 0x7f710804b720 con 0x7f710809fb80 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient(hunting): my global_id is 7291 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70f40034e0 con 0x7f710809fb80 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 820473835 0 0) 0x7f70f40034e0 con 0x7f710809fb80 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71080a2e30 msgr2=0x7f7108132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71080a2e30 0x7f7108132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f710809ff70 msgr2=0x7f71080a1150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f710809ff70 0x7f71080a1150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:45.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:37:45.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71080a1f10 con 0x7f710809fb80 2024-09-15T01:37:45.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient: _finish_auth 0 2024-09-15T01:37:45.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:45.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f70f00018f0 con 0x7f710809fb80 2024-09-15T01:37:45.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 5 monclient: authenticate success, global_id 7291 2024-09-15T01:37:45.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:45.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:45.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:37:45.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient: dump: 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 >> v1:172.21.15.204:6789/0 conn(0x7f710809fb80 legacy=0x7f71080a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient: _add_conns ranks=[1,4,8,5,3,2,7,6,0] 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70f4004220 0x7f70f4004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient(hunting): picked mon.f con 0x7f70f4004220 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f710809ff70 0x7f70f4004f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient(hunting): picked mon.c con 0x7f710809ff70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71080a2e30 0x7f70f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient(hunting): picked mon.e con 0x7f71080a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710809e6c0 con 0x7f70f4004220 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71080a3200 con 0x7f710809ff70 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f70f400c0a0 con 0x7f71080a2e30 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7105d9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710d02b640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710659c640 1 -- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f710809ff70 msgr2=0x7f70f4004f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7106d9d640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70f4004220 0x7f70f4004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710659c640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f710809ff70 0x7f70f4004f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7106d9d640 10 monclient(hunting): get_auth_request con 0x7f70f4004220 auth_method 0 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7106d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7106d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7106d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710759e640 1 -- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71080a2e30 msgr2=0x7f70f400b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f710759e640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71080a2e30 0x7f70f400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7106d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7106d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7106d9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.290+0000 7f7106d9d640 10 monclient(hunting): handle_auth_done global_id 7291 payload 293 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7106d9d640 1 -- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71080a2e30 msgr2=0x7f70f400b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7106d9d640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71080a2e30 0x7f70f400b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7106d9d640 1 -- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f710809ff70 msgr2=0x7f70f4004f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7106d9d640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f710809ff70 0x7f70f4004f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7106d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:37:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7106d9d640 1 monclient: found mon.f 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7106d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7106d9d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7106d9d640 1 -- 172.21.15.204:0/2155607181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f40034e0 con 0x7f70f4004220 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7106d9d640 10 monclient: _finish_auth 0 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7106d9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7106d9d640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70f4004220 0x7f70f4004780 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7100009f90 tx=0x7f71000095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaef28f7cb1c4129 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71000328b0 con 0x7f70f4004220 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7105d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7105d9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7105d9b640 10 monclient: dump: 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710002d070 con 0x7f70f4004220 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f710d02b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7105d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7105d9b640 1 -- 172.21.15.204:0/2155607181 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71000313d0 con 0x7f70f4004220 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f710d02b640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f710d02b640 4 set_mon_vals no callback set 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7105d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7105d9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.291+0000 7f7105d9b640 10 monclient: dump: 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.292+0000 7f710d02b640 10 monclient: shutdown 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.292+0000 7f710d02b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.292+0000 7f710d02b640 1 -- 172.21.15.204:0/2155607181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70f4004220 msgr2=0x7f70f4004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.292+0000 7f710d02b640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70f4004220 0x7f70f4004780 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7100009f90 tx=0x7f71000095d0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.292+0000 7f710d02b640 1 -- 172.21.15.204:0/2155607181 shutdown_connections 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.292+0000 7f710d02b640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f71080a2e30 0x7f70f400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.292+0000 7f710d02b640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f710809ff70 0x7f70f4004f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.292+0000 7f710d02b640 1 --2- 172.21.15.204:0/2155607181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70f4004220 0x7f70f4004780 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.292+0000 7f710d02b640 1 -- 172.21.15.204:0/2155607181 >> 172.21.15.204:0/2155607181 conn(0x7f710806e860 msgr2=0x7f710806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.292+0000 7f710d02b640 1 -- 172.21.15.204:0/2155607181 shutdown_connections 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.292+0000 7f710d02b640 1 -- 172.21.15.204:0/2155607181 wait complete. 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient: build_initial_monmap 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient: monmap: 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 1 Processor -- start 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 1 -- start start 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient: init 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient: _add_conns ranks=[4,3,8,0,6,2,5,7,1] 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f70f4004220 0x7f710811e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient(hunting): picked mon.noname-c con 0x7f70f4004220 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710809fb80 0x7f710811e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient(hunting): picked mon.noname-g con 0x7f710809fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f710809ff70 0x7f710811ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient(hunting): picked mon.noname-e con 0x7f710809ff70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7108131d70 con 0x7f710809fb80 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:45.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7108120290 con 0x7f70f4004220 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f710809d680 con 0x7f710809ff70 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710d02b640 10 monclient(hunting): authenticate will time out at 1080.936837s 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f7106d9d640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f70f4004220 msgr2=0x7f710811e1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710759e640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f710809ff70 msgr2=0x7f710811ec50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f7106d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f70f4004220 0x7f710811e1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710759e640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f710809ff70 0x7f710811ec50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710659c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710809fb80 0x7f710811e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710659c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710809fb80 0x7f710811e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48272/0 (socket says 172.21.15.204:48272) 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710659c640 1 -- 172.21.15.204:0/459075667 learned_addr learned my addr 172.21.15.204:0/459075667 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710659c640 10 monclient(hunting): get_auth_request con 0x7f710809fb80 auth_method 0 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:45.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.293+0000 7f710659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 10 monclient(hunting): handle_auth_done global_id 15573 payload 824 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 1 -- 172.21.15.204:0/459075667 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f710809ff70 msgr2=0x7f710811ec50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 1 --2- 172.21.15.204:0/459075667 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f710809ff70 0x7f710811ec50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 1 -- 172.21.15.204:0/459075667 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f70f4004220 msgr2=0x7f710811e1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 1 --2- 172.21.15.204:0/459075667 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f70f4004220 0x7f710811e1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 10 monclient: _finish_hunting 0 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 1 monclient: found mon.noname-g 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 1 -- 172.21.15.204:0/459075667 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7108121870 con 0x7f710809fb80 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710659c640 1 --2- 172.21.15.204:0/459075667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710809fb80 0x7f710811e710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f70f000c7c0 tx=0x7f70f00014e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f70feffd640 1 -- 172.21.15.204:0/459075667 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f70f000ce90 con 0x7f710809fb80 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f70feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f70feffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f70feffd640 10 monclient: dump: 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f70feffd640 10 monclient: _finish_auth 0 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f70feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710d02b640 5 monclient: authenticate success, global_id 15573 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f70feffd640 1 -- 172.21.15.204:0/459075667 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f00075d0 con 0x7f710809fb80 2024-09-15T01:37:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710d02b640 10 monclient: _renew_subs 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710d02b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710d02b640 1 -- 172.21.15.204:0/459075667 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7108120630 con 0x7f710809fb80 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f70feffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f70feffd640 1 -- 172.21.15.204:0/459075667 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f70f0004c20 con 0x7f710809fb80 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f70feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f70feffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f70feffd640 10 monclient: dump: 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710559a640 4 set_mon_vals no callback set 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710d02b640 10 monclient: _renew_subs 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710d02b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.294+0000 7f710d02b640 1 -- 172.21.15.204:0/459075667 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7108121590 con 0x7f710809fb80 2024-09-15T01:37:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.295+0000 7f70feffd640 1 -- 172.21.15.204:0/459075667 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f70f00047a0 con 0x7f710809fb80 2024-09-15T01:37:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.295+0000 7f710d02b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.295+0000 7f710d02b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.295+0000 7f710d02b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.295+0000 7f70feffd640 1 --2- 172.21.15.204:0/459075667 >> v2:172.21.15.204:6812/28210 conn(0x7f70e80397b0 0x7f70e803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.295+0000 7f710d02b640 1 -- 172.21.15.204:0/459075667 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710811fac0 con 0x7f710809fb80 2024-09-15T01:37:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.295+0000 7f70feffd640 1 -- 172.21.15.204:0/459075667 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 12236+0+0 (secure 0 0 0) 0x7f70f004c460 con 0x7f710809fb80 2024-09-15T01:37:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.295+0000 7f7106d9d640 1 --2- 172.21.15.204:0/459075667 >> v2:172.21.15.204:6812/28210 conn(0x7f70e80397b0 0x7f70e803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.295+0000 7f7106d9d640 10 monclient: get_auth_request con 0x7f70e80397b0 auth_method 0 2024-09-15T01:37:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.296+0000 7f7106d9d640 1 --2- 172.21.15.204:0/459075667 >> v2:172.21.15.204:6812/28210 conn(0x7f70e80397b0 0x7f70e803bc60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7100032000 tx=0x7f7100035000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.300+0000 7f70feffd640 1 -- 172.21.15.204:0/459075667 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f70f0004a50 con 0x7f710809fb80 2024-09-15T01:37:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.300+0000 7f70feffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.300+0000 7f70feffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:45.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.460+0000 7f710d02b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h10"}] 2024-09-15T01:37:45.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.460+0000 7f710d02b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h10"}] 2024-09-15T01:37:45.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.460+0000 7f710d02b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:45.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:45.460+0000 7f710d02b640 1 -- 172.21.15.204:0/459075667 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h10"} v 0) v1 -- 0x7f7108070cc0 con 0x7f710809fb80 2024-09-15T01:37:46.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.105+0000 7f70feffd640 1 -- 172.21.15.204:0/459075667 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h10"}]=0 removed item id -46 name 'h10' from crush map v228) v1 ==== 120+0+0 (secure 0 0 0) 0x7f70f0018020 con 0x7f710809fb80 2024-09-15T01:37:46.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.105+0000 7f70feffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h10"}] 2024-09-15T01:37:46.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.105+0000 7f70feffd640 10 monclient: _finish_command 2 = system:0 removed item id -46 name 'h10' from crush map 2024-09-15T01:37:46.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.204+0000 7f710d02b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h10"}] 2024-09-15T01:37:46.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.204+0000 7f710d02b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h10"}] 2024-09-15T01:37:46.207 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.204+0000 7f710d02b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:46.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.204+0000 7f710d02b640 1 -- 172.21.15.204:0/459075667 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h10"} v 0) v1 -- 0x7f710809ddb0 con 0x7f710809fb80 2024-09-15T01:37:46.208 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.205+0000 7f70feffd640 1 -- 172.21.15.204:0/459075667 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h10"}]=0 device 'h10' does not appear in the crush map v228) v1 ==== 120+0+0 (secure 0 0 0) 0x7f70f0010300 con 0x7f710809fb80 2024-09-15T01:37:46.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.205+0000 7f70feffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h10"}] 2024-09-15T01:37:46.209 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.205+0000 7f70feffd640 10 monclient: _finish_command 3 = system:0 device 'h10' does not appear in the crush map 2024-09-15T01:37:46.209 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h10' does not appear in the crush map 2024-09-15T01:37:46.210 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 1 -- 172.21.15.204:0/459075667 >> v2:172.21.15.204:6812/28210 conn(0x7f70e80397b0 msgr2=0x7f70e803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:46.210 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 1 --2- 172.21.15.204:0/459075667 >> v2:172.21.15.204:6812/28210 conn(0x7f70e80397b0 0x7f70e803bc60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7100032000 tx=0x7f7100035000 comp rx=0 tx=0).stop 2024-09-15T01:37:46.210 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 10 monclient: shutdown 2024-09-15T01:37:46.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:46.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 1 -- 172.21.15.204:0/459075667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710809fb80 msgr2=0x7f710811e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:46.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 1 --2- 172.21.15.204:0/459075667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710809fb80 0x7f710811e710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f70f000c7c0 tx=0x7f70f00014e0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 1 -- 172.21.15.204:0/459075667 shutdown_connections 2024-09-15T01:37:46.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 1 --2- 172.21.15.204:0/459075667 >> v2:172.21.15.204:6812/28210 conn(0x7f70e80397b0 0x7f70e803bc60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 1 --2- 172.21.15.204:0/459075667 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f710809ff70 0x7f710811ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.211 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 1 --2- 172.21.15.204:0/459075667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f710809fb80 0x7f710811e710 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.212 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 1 --2- 172.21.15.204:0/459075667 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f70f4004220 0x7f710811e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.212 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 1 -- 172.21.15.204:0/459075667 >> 172.21.15.204:0/459075667 conn(0x7f710806e860 msgr2=0x7f71080a3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:46.212 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 1 -- 172.21.15.204:0/459075667 shutdown_connections 2024-09-15T01:37:46.212 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.207+0000 7f710d02b640 1 -- 172.21.15.204:0/459075667 wait complete. 2024-09-15T01:37:46.219 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:46.219 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h11 2024-09-15T01:37:46.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.292+0000 7f364799d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:46.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.292+0000 7f364799d640 10 monclient: build_initial_monmap 2024-09-15T01:37:46.295 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.292+0000 7f364799d640 10 monclient: monmap: 2024-09-15T01:37:46.295 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:46.296 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:46.296 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:46.294014+0000 2024-09-15T01:37:46.296 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:46.294014+0000 2024-09-15T01:37:46.296 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:46.296 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:46.296 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:46.296 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:46.296 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:46.296 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:46.296 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:46.297 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:46.297 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:46.297 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:46.297 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:46.297 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:46.297 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:46.297 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:46.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 1 Processor -- start 2024-09-15T01:37:46.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 1 -- start start 2024-09-15T01:37:46.297 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 10 monclient: init 2024-09-15T01:37:46.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 10 monclient: _add_conns ranks=[8,9,7,4,2,3,10,0,5,6,1] 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f36400a2e30 0x7f3640132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 10 monclient(hunting): picked mon.noname-e con 0x7f36400a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f364009fb80 0x7f364009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 10 monclient(hunting): picked mon.noname-a con 0x7f364009fb80 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 0x7f36400a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 10 monclient(hunting): picked mon.noname-i con 0x7f36400a0a40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364009e6c0 con 0x7f364009fb80 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f36400a04b0 con 0x7f36400a0a40 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:46.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f36400a3200 con 0x7f36400a2e30 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f364799d640 10 monclient(hunting): authenticate will time out at 1081.936793s 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f3644f38640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7f364009fb80 msgr2=0x7f364009ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f3645739640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f36400a2e30 msgr2=0x7f3640132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f3645f3a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 0x7f36400a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f3644f38640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f364009fb80 0x7f364009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f3645739640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f36400a2e30 0x7f3640132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f3645f3a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 0x7f36400a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:52736/0 (socket says 172.21.15.204:52736) 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f3645f3a640 1 -- 172.21.15.204:0/2877037002 learned_addr learned my addr 172.21.15.204:0/2877037002 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f3645f3a640 10 monclient(hunting): get_auth_request con 0x7f36400a0a40 auth_method 0 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f3645f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f3645f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:46.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.293+0000 7f3645f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 10 monclient(hunting): handle_auth_done global_id 24676 payload 274 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 1 -- 172.21.15.204:0/2877037002 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f36400a2e30 msgr2=0x7f3640132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 1 --2- 172.21.15.204:0/2877037002 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f36400a2e30 0x7f3640132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 1 -- 172.21.15.204:0/2877037002 >> v2:172.21.15.178:3300/0 conn(0x7f364009fb80 msgr2=0x7f364009ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 1 --2- 172.21.15.204:0/2877037002 >> v2:172.21.15.178:3300/0 conn(0x7f364009fb80 0x7f364009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 10 monclient: _finish_hunting 0 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 1 monclient: found mon.noname-i 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:46.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 1 -- 172.21.15.204:0/2877037002 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364004b720 con 0x7f36400a0a40 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3645f3a640 1 --2- 172.21.15.204:0/2877037002 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 0x7f36400a0ee0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3638009f20 tx=0x7f3638009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=3136746c9751e2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3637fff640 1 -- 172.21.15.204:0/2877037002 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3638007880 con 0x7f36400a0a40 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3637fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.294+0000 7f3637fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f3637fff640 10 monclient: dump: 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:46.310 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f3637fff640 10 monclient: _finish_auth 0 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f3637fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f3637fff640 1 -- 172.21.15.204:0/2877037002 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363802d050 con 0x7f36400a0a40 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 5 monclient: authenticate success, global_id 24676 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f3637fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:46.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f3637fff640 1 -- 172.21.15.204:0/2877037002 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3638030360 con 0x7f36400a0a40 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f3637fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 4 set_mon_vals no callback set 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f3637fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f3637fff640 10 monclient: dump: 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:46.312 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:46.313 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:46.313 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:46.313 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:46.313 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:46.313 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:46.313 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:46.313 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:46.313 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:46.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 10 monclient: shutdown 2024-09-15T01:37:46.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:46.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 1 -- 172.21.15.204:0/2877037002 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 msgr2=0x7f36400a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:46.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 1 --2- 172.21.15.204:0/2877037002 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 0x7f36400a0ee0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3638009f20 tx=0x7f3638009840 comp rx=0 tx=0).stop 2024-09-15T01:37:46.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 1 -- 172.21.15.204:0/2877037002 shutdown_connections 2024-09-15T01:37:46.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 1 --2- 172.21.15.204:0/2877037002 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 0x7f36400a0ee0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 1 --2- 172.21.15.204:0/2877037002 >> v2:172.21.15.178:3300/0 conn(0x7f364009fb80 0x7f364009ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 1 --2- 172.21.15.204:0/2877037002 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f36400a2e30 0x7f3640132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 1 -- 172.21.15.204:0/2877037002 >> 172.21.15.204:0/2877037002 conn(0x7f364006e860 msgr2=0x7f364006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:46.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 1 -- 172.21.15.204:0/2877037002 shutdown_connections 2024-09-15T01:37:46.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.295+0000 7f364799d640 1 -- 172.21.15.204:0/2877037002 wait complete. 2024-09-15T01:37:46.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.296+0000 7f364799d640 10 monclient: build_initial_monmap 2024-09-15T01:37:46.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.296+0000 7f364799d640 10 monclient: monmap: 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:46.315 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:46.316 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:46.316 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:46.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.296+0000 7f364799d640 1 Processor -- start 2024-09-15T01:37:46.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.296+0000 7f364799d640 1 -- start start 2024-09-15T01:37:46.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.296+0000 7f364799d640 10 monclient: init 2024-09-15T01:37:46.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:46.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient: _add_conns ranks=[4,7,2,3,5,8,0,6,1] 2024-09-15T01:37:46.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f364009fb80 0x7f36401317f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient(hunting): picked mon.noname-c con 0x7f364009fb80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:46.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 0x7f364011e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient(hunting): picked mon.noname-i con 0x7f36400a0a40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:46.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36400a2e30 0x7f364011e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient(hunting): picked mon.noname-b con 0x7f36400a2e30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:46.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:46.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364009ca40 con 0x7f36400a2e30 2024-09-15T01:37:46.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:46.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f364009d650 con 0x7f364009fb80 2024-09-15T01:37:46.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:46.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f36401205d0 con 0x7f36400a0a40 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient(hunting): authenticate will time out at 1081.940115s 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 0x7f364011e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3645739640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f364009fb80 msgr2=0x7f36401317f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3645739640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f364009fb80 0x7f36401317f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3645f3a640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36400a2e30 msgr2=0x7f364011e870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 0x7f364011e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:52744/0 (socket says 172.21.15.204:52744) 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 1 -- 172.21.15.204:0/485313437 learned_addr learned my addr 172.21.15.204:0/485313437 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 10 monclient(hunting): get_auth_request con 0x7f36400a0a40 auth_method 0 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:46.318 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3645f3a640 1 --2- 172.21.15.204:0/485313437 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36400a2e30 0x7f364011e870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 10 monclient(hunting): handle_auth_done global_id 24685 payload 824 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 1 -- 172.21.15.204:0/485313437 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f364009fb80 msgr2=0x7f36401317f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 1 --2- 172.21.15.204:0/485313437 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f364009fb80 0x7f36401317f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 1 -- 172.21.15.204:0/485313437 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36400a2e30 msgr2=0x7f364011e870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 1 --2- 172.21.15.204:0/485313437 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36400a2e30 0x7f364011e870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 10 monclient: _finish_hunting 0 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 1 monclient: found mon.noname-i 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:46.319 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 1 -- 172.21.15.204:0/485313437 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3638009ae0 con 0x7f36400a0a40 2024-09-15T01:37:46.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f3644f38640 1 --2- 172.21.15.204:0/485313437 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 0x7f364011e330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f363000b5e0 tx=0x7f3630006f90 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:46.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36357fa640 1 -- 172.21.15.204:0/485313437 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3630006150 con 0x7f36400a0a40 2024-09-15T01:37:46.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:46.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36357fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:46.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36357fa640 10 monclient: dump: 2024-09-15T01:37:46.320 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:46.320 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:46.320 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.320 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.320 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:46.320 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:46.321 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:46.321 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:46.321 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:46.321 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:46.321 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:46.321 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:46.321 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:46.321 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:46.321 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:46.321 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:46.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36357fa640 10 monclient: _finish_auth 0 2024-09-15T01:37:46.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36357fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:46.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 5 monclient: authenticate success, global_id 24685 2024-09-15T01:37:46.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36357fa640 1 -- 172.21.15.204:0/485313437 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36300062f0 con 0x7f36400a0a40 2024-09-15T01:37:46.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient: _renew_subs 2024-09-15T01:37:46.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:46.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 1 -- 172.21.15.204:0/485313437 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3640121940 con 0x7f36400a0a40 2024-09-15T01:37:46.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36357fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:46.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36357fa640 1 -- 172.21.15.204:0/485313437 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36300103e0 con 0x7f36400a0a40 2024-09-15T01:37:46.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:46.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36357fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:46.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36377fe640 4 set_mon_vals no callback set 2024-09-15T01:37:46.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f36357fa640 10 monclient: dump: 2024-09-15T01:37:46.324 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:46.324 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:46.324 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.324 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.325 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:46.325 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:46.325 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:46.325 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:46.325 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:46.325 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:46.326 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:46.326 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:46.326 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:46.326 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:46.326 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:46.326 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:46.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient: _renew_subs 2024-09-15T01:37:46.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:46.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.297+0000 7f364799d640 1 -- 172.21.15.204:0/485313437 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3640120a50 con 0x7f36400a0a40 2024-09-15T01:37:46.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.298+0000 7f36357fa640 1 -- 172.21.15.204:0/485313437 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3630003b70 con 0x7f36400a0a40 2024-09-15T01:37:46.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.298+0000 7f364799d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:46.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.298+0000 7f364799d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:46.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.298+0000 7f364799d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:46.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.298+0000 7f364799d640 1 -- 172.21.15.204:0/485313437 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3640070cc0 con 0x7f36400a0a40 2024-09-15T01:37:46.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.298+0000 7f36357fa640 1 --2- 172.21.15.204:0/485313437 >> v2:172.21.15.204:6812/28210 conn(0x7f36240397b0 0x7f362403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.298+0000 7f36357fa640 1 -- 172.21.15.204:0/485313437 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 12023+0+0 (secure 0 0 0) 0x7f363004c500 con 0x7f36400a0a40 2024-09-15T01:37:46.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.299+0000 7f3645739640 1 --2- 172.21.15.204:0/485313437 >> v2:172.21.15.204:6812/28210 conn(0x7f36240397b0 0x7f362403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:46.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.299+0000 7f3645739640 10 monclient: get_auth_request con 0x7f36240397b0 auth_method 0 2024-09-15T01:37:46.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.299+0000 7f3645739640 1 --2- 172.21.15.204:0/485313437 >> v2:172.21.15.204:6812/28210 conn(0x7f36240397b0 0x7f362403bc60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f363c003870 tx=0x7f363c006ce0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:46.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.303+0000 7f36357fa640 1 -- 172.21.15.204:0/485313437 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f363000daf0 con 0x7f36400a0a40 2024-09-15T01:37:46.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.303+0000 7f36357fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:46.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.303+0000 7f36357fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:46.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.460+0000 7f364799d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h11"}] 2024-09-15T01:37:46.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.460+0000 7f364799d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h11"}] 2024-09-15T01:37:46.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.460+0000 7f364799d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:46.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.460+0000 7f364799d640 1 -- 172.21.15.204:0/485313437 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h11"} v 0) v1 -- 0x7f36400a1f10 con 0x7f36400a0a40 2024-09-15T01:37:46.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.474+0000 7f36357fa640 1 -- 172.21.15.204:0/485313437 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h11"}]=0 removed item id -49 name 'h11' from crush map v229) v1 ==== 120+0+0 (secure 0 0 0) 0x7f3630017020 con 0x7f36400a0a40 2024-09-15T01:37:46.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.474+0000 7f36357fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h11"}] 2024-09-15T01:37:46.477 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.474+0000 7f36357fa640 10 monclient: _finish_command 2 = system:0 removed item id -49 name 'h11' from crush map 2024-09-15T01:37:46.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.571+0000 7f364799d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h11"}] 2024-09-15T01:37:46.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.571+0000 7f364799d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h11"}] 2024-09-15T01:37:46.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.571+0000 7f364799d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:46.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.571+0000 7f364799d640 1 -- 172.21.15.204:0/485313437 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h11"} v 0) v1 -- 0x7f36400008d0 con 0x7f36400a0a40 2024-09-15T01:37:46.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.572+0000 7f36357fa640 1 -- 172.21.15.204:0/485313437 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h11"}]=0 device 'h11' does not appear in the crush map v229) v1 ==== 120+0+0 (secure 0 0 0) 0x7f363000d310 con 0x7f36400a0a40 2024-09-15T01:37:46.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.572+0000 7f36357fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h11"}] 2024-09-15T01:37:46.575 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.572+0000 7f36357fa640 10 monclient: _finish_command 3 = system:0 device 'h11' does not appear in the crush map 2024-09-15T01:37:46.575 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h11' does not appear in the crush map 2024-09-15T01:37:46.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 1 -- 172.21.15.204:0/485313437 >> v2:172.21.15.204:6812/28210 conn(0x7f36240397b0 msgr2=0x7f362403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:46.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 1 --2- 172.21.15.204:0/485313437 >> v2:172.21.15.204:6812/28210 conn(0x7f36240397b0 0x7f362403bc60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f363c003870 tx=0x7f363c006ce0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 10 monclient: shutdown 2024-09-15T01:37:46.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:46.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 1 -- 172.21.15.204:0/485313437 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 msgr2=0x7f364011e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:46.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 1 --2- 172.21.15.204:0/485313437 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 0x7f364011e330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f363000b5e0 tx=0x7f3630006f90 comp rx=0 tx=0).stop 2024-09-15T01:37:46.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 1 -- 172.21.15.204:0/485313437 shutdown_connections 2024-09-15T01:37:46.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 1 --2- 172.21.15.204:0/485313437 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36400a2e30 0x7f364011e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 1 --2- 172.21.15.204:0/485313437 >> v2:172.21.15.204:6812/28210 conn(0x7f36240397b0 0x7f362403bc60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 1 --2- 172.21.15.204:0/485313437 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f36400a0a40 0x7f364011e330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 1 --2- 172.21.15.204:0/485313437 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f364009fb80 0x7f36401317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.577 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 1 -- 172.21.15.204:0/485313437 >> 172.21.15.204:0/485313437 conn(0x7f364006e860 msgr2=0x7f364009ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:46.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 1 -- 172.21.15.204:0/485313437 shutdown_connections 2024-09-15T01:37:46.578 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.574+0000 7f364799d640 1 -- 172.21.15.204:0/485313437 wait complete. 2024-09-15T01:37:46.585 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:46.585 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h12 2024-09-15T01:37:46.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.656+0000 7f6487c4f640 10 monclient: get_monmap_and_config 2024-09-15T01:37:46.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.656+0000 7f6487c4f640 10 monclient: build_initial_monmap 2024-09-15T01:37:46.658 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.656+0000 7f6487c4f640 10 monclient: monmap: 2024-09-15T01:37:46.658 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:46.657719+0000 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:46.657719+0000 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:46.659 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:46.660 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:46.660 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:46.660 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:46.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.656+0000 7f6487c4f640 1 Processor -- start 2024-09-15T01:37:46.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 1 -- start start 2024-09-15T01:37:46.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 10 monclient: init 2024-09-15T01:37:46.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:46.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 10 monclient: _add_conns ranks=[1,10,4,0,7,6,9,2,3,8,5] 2024-09-15T01:37:46.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f64800a2c60 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:46.667 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f648009fc60 0x7f64800a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 10 monclient(hunting): picked mon.noname-f con 0x7f648009fc60 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:46.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6480137d90 0x7f648013a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 10 monclient(hunting): picked mon.noname-c con 0x7f6480137d90 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:46.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648004b720 con 0x7f64800a2c60 2024-09-15T01:37:46.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:46.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6480133170 con 0x7f648009fc60 2024-09-15T01:37:46.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:46.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64800a3030 con 0x7f6480137d90 2024-09-15T01:37:46.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:46.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f6487c4f640 10 monclient(hunting): authenticate will time out at 1082.300715s 2024-09-15T01:37:46.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f64859eb640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f64800a2c60 0x7f6480132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:57506/0 (socket says 172.21.15.204:57506) 2024-09-15T01:37:46.668 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f64859eb640 1 -- 172.21.15.204:0/3546430983 learned_addr learned my addr 172.21.15.204:0/3546430983 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:37:46.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f64851ea640 1 --2- 172.21.15.204:0/3546430983 >> v2:172.21.15.204:3300/0 conn(0x7f648009fc60 0x7f64800a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:46.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f64851ea640 10 monclient(hunting): get_auth_request con 0x7f648009fc60 auth_method 0 2024-09-15T01:37:46.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f64851ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:46.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f64851ea640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:46.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f64851ea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:46.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f64861ec640 1 -- 172.21.15.204:0/3546430983 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6480137d90 msgr2=0x7f648013a170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:46.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f64861ec640 1 --2- 172.21.15.204:0/3546430983 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6480137d90 0x7f648013a170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:46.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f64851ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:46.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f64851ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:46.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.657+0000 7f64851ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64851ea640 10 monclient(hunting): handle_auth_done global_id 7300 payload 274 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64851ea640 1 -- 172.21.15.204:0/3546430983 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6480137d90 msgr2=0x7f648013a170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64851ea640 1 --2- 172.21.15.204:0/3546430983 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6480137d90 0x7f648013a170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64851ea640 1 -- 172.21.15.204:0/3546430983 >> v1:172.21.15.204:6789/0 conn(0x7f64800a2c60 legacy=0x7f6480132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64851ea640 10 monclient: _finish_hunting 0 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64851ea640 1 monclient: found mon.noname-f 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64851ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64851ea640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64851ea640 1 -- 172.21.15.204:0/3546430983 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648009d5a0 con 0x7f648009fc60 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64851ea640 1 --2- 172.21.15.204:0/3546430983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648009fc60 0x7f64800a0050 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6470009850 tx=0x7f64700090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de9dc93417957b84 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64849e9640 1 -- 172.21.15.204:0/3546430983 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f647002ac90 con 0x7f648009fc60 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64849e9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64849e9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:46.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64849e9640 10 monclient: dump: 2024-09-15T01:37:46.671 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:46.671 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:46.671 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.671 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.671 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:46.671 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:46.671 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:46.671 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:46.671 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:46.671 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.658+0000 7f64849e9640 10 monclient: _finish_auth 0 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f64849e9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f64849e9640 1 -- 172.21.15.204:0/3546430983 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647002ae10 con 0x7f648009fc60 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 5 monclient: authenticate success, global_id 7300 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f64849e9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:46.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f64849e9640 1 -- 172.21.15.204:0/3546430983 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6470031ca0 con 0x7f648009fc60 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f64849e9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 4 set_mon_vals no callback set 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f64849e9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f64849e9640 10 monclient: dump: 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:46.673 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 10 monclient: shutdown 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 1 -- 172.21.15.204:0/3546430983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648009fc60 msgr2=0x7f64800a0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 1 --2- 172.21.15.204:0/3546430983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648009fc60 0x7f64800a0050 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6470009850 tx=0x7f64700090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 1 -- 172.21.15.204:0/3546430983 shutdown_connections 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 1 --2- 172.21.15.204:0/3546430983 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6480137d90 0x7f648013a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 1 --2- 172.21.15.204:0/3546430983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648009fc60 0x7f64800a0050 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 1 -- 172.21.15.204:0/3546430983 >> 172.21.15.204:0/3546430983 conn(0x7f64800a60c0 msgr2=0x7f64800a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 1 -- 172.21.15.204:0/3546430983 shutdown_connections 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.659+0000 7f6487c4f640 1 -- 172.21.15.204:0/3546430983 wait complete. 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient: build_initial_monmap 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient: monmap: 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:46.675 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 1 Processor -- start 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 1 -- start start 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient: init 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient: _add_conns ranks=[0,3,5,2,1,6,7,8,4] 2024-09-15T01:37:46.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f648009fc60 0x7f6480099170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient(hunting): picked mon.noname-a con 0x7f648009fc60 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64800a2c60 0x7f64800996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient(hunting): picked mon.noname-g con 0x7f64800a2c60 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6480137d90 0x7f6480099bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient(hunting): picked mon.noname-h con 0x7f6480137d90 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648009ca50 con 0x7f648009fc60 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648009b150 con 0x7f64800a2c60 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f648011c720 con 0x7f6480137d90 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.660+0000 7f6487c4f640 10 monclient(hunting): authenticate will time out at 1082.303962s 2024-09-15T01:37:46.677 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64851ea640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64800a2c60 0x7f64800996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64859eb640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f648009fc60 msgr2=0x7f6480099170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6480137d90 0x7f6480099bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64851ea640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64800a2c60 0x7f64800996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:48278/0 (socket says 172.21.15.204:48278) 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64859eb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f648009fc60 0x7f6480099170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64851ea640 1 -- 172.21.15.204:0/2663963910 learned_addr learned my addr 172.21.15.204:0/2663963910 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64851ea640 10 monclient(hunting): get_auth_request con 0x7f64800a2c60 auth_method 0 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64851ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64851ea640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64851ea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 10 monclient(hunting): get_auth_request con 0x7f6480137d90 auth_method 0 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:46.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64851ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64851ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64851ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 10 monclient(hunting): handle_auth_done global_id 6431 payload 824 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 1 -- 172.21.15.204:0/2663963910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64800a2c60 msgr2=0x7f64800996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 1 --2- 172.21.15.204:0/2663963910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64800a2c60 0x7f64800996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 1 -- 172.21.15.204:0/2663963910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f648009fc60 msgr2=0x7f6480099170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 1 --2- 172.21.15.204:0/2663963910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f648009fc60 0x7f6480099170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 10 monclient: _finish_hunting 0 2024-09-15T01:37:46.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 1 monclient: found mon.noname-h 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 1 -- 172.21.15.204:0/2663963910 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6470009390 con 0x7f6480137d90 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64851ea640 1 --2- 172.21.15.204:0/2663963910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64800a2c60 0x7f64800996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f64861ec640 1 --2- 172.21.15.204:0/2663963910 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6480137d90 0x7f6480099bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f647800be50 tx=0x7f64780096a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6475ffb640 1 -- 172.21.15.204:0/2663963910 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f64780067e0 con 0x7f6480137d90 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6475ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6475ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6475ffb640 10 monclient: dump: 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.680 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6475ffb640 10 monclient: _finish_auth 0 2024-09-15T01:37:46.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6475ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6475ffb640 1 -- 172.21.15.204:0/2663963910 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647800f070 con 0x7f6480137d90 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6487c4f640 5 monclient: authenticate success, global_id 6431 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6487c4f640 10 monclient: _renew_subs 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6487c4f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6487c4f640 1 -- 172.21.15.204:0/2663963910 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648009c700 con 0x7f6480137d90 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6475ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6475ffb640 1 -- 172.21.15.204:0/2663963910 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f64780069c0 con 0x7f6480137d90 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6475ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6475ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6475ffb640 10 monclient: dump: 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.682 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6487c4f640 10 monclient: _renew_subs 2024-09-15T01:37:46.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6477fff640 4 set_mon_vals no callback set 2024-09-15T01:37:46.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6487c4f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:46.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.661+0000 7f6487c4f640 1 -- 172.21.15.204:0/2663963910 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648009b7a0 con 0x7f6480137d90 2024-09-15T01:37:46.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.662+0000 7f6475ffb640 1 -- 172.21.15.204:0/2663963910 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6478006360 con 0x7f6480137d90 2024-09-15T01:37:46.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.662+0000 7f6487c4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:46.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.662+0000 7f6487c4f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:46.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.662+0000 7f6487c4f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:46.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.662+0000 7f6487c4f640 1 -- 172.21.15.204:0/2663963910 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648011ca70 con 0x7f6480137d90 2024-09-15T01:37:46.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.663+0000 7f6475ffb640 1 --2- 172.21.15.204:0/2663963910 >> v2:172.21.15.204:6812/28210 conn(0x7f64640397b0 0x7f646403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:46.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.663+0000 7f6475ffb640 1 -- 172.21.15.204:0/2663963910 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 11810+0+0 (secure 0 0 0) 0x7f647804d960 con 0x7f6480137d90 2024-09-15T01:37:46.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.663+0000 7f64859eb640 1 --2- 172.21.15.204:0/2663963910 >> v2:172.21.15.204:6812/28210 conn(0x7f64640397b0 0x7f646403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:46.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.663+0000 7f64859eb640 10 monclient: get_auth_request con 0x7f64640397b0 auth_method 0 2024-09-15T01:37:46.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.663+0000 7f64859eb640 1 --2- 172.21.15.204:0/2663963910 >> v2:172.21.15.204:6812/28210 conn(0x7f64640397b0 0x7f646403bc60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f647c0047d0 tx=0x7f647c007ba0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:46.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.667+0000 7f6475ffb640 1 -- 172.21.15.204:0/2663963910 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6478006610 con 0x7f6480137d90 2024-09-15T01:37:46.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.667+0000 7f6475ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:46.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.667+0000 7f6475ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.827+0000 7f6487c4f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h12"}] 2024-09-15T01:37:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.827+0000 7f6487c4f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h12"}] 2024-09-15T01:37:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.827+0000 7f6487c4f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:46.830 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:46.827+0000 7f6487c4f640 1 -- 172.21.15.204:0/2663963910 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h12"} v 0) v1 -- 0x7f648009b9c0 con 0x7f6480137d90 2024-09-15T01:37:47.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.482+0000 7f6475ffb640 1 -- 172.21.15.204:0/2663963910 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h12"}]=0 removed item id -52 name 'h12' from crush map v230) v1 ==== 120+0+0 (secure 0 0 0) 0x7f647801a020 con 0x7f6480137d90 2024-09-15T01:37:47.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.482+0000 7f6475ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h12"}] 2024-09-15T01:37:47.485 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.482+0000 7f6475ffb640 10 monclient: _finish_command 2 = system:0 removed item id -52 name 'h12' from crush map 2024-09-15T01:37:47.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.582+0000 7f6487c4f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h12"}] 2024-09-15T01:37:47.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.582+0000 7f6487c4f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h12"}] 2024-09-15T01:37:47.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.582+0000 7f6487c4f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:47.585 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.582+0000 7f6487c4f640 1 -- 172.21.15.204:0/2663963910 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h12"} v 0) v1 -- 0x7f648011c890 con 0x7f6480137d90 2024-09-15T01:37:47.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.583+0000 7f6475ffb640 1 -- 172.21.15.204:0/2663963910 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h12"}]=0 device 'h12' does not appear in the crush map v230) v1 ==== 120+0+0 (secure 0 0 0) 0x7f647801db40 con 0x7f6480137d90 2024-09-15T01:37:47.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.583+0000 7f6475ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h12"}] 2024-09-15T01:37:47.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.583+0000 7f6475ffb640 10 monclient: _finish_command 3 = system:0 device 'h12' does not appear in the crush map 2024-09-15T01:37:47.589 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h12' does not appear in the crush map 2024-09-15T01:37:47.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 1 -- 172.21.15.204:0/2663963910 >> v2:172.21.15.204:6812/28210 conn(0x7f64640397b0 msgr2=0x7f646403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:47.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 1 --2- 172.21.15.204:0/2663963910 >> v2:172.21.15.204:6812/28210 conn(0x7f64640397b0 0x7f646403bc60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f647c0047d0 tx=0x7f647c007ba0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 10 monclient: shutdown 2024-09-15T01:37:47.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:47.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 1 -- 172.21.15.204:0/2663963910 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6480137d90 msgr2=0x7f6480099bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:47.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 1 --2- 172.21.15.204:0/2663963910 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6480137d90 0x7f6480099bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f647800be50 tx=0x7f64780096a0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 1 -- 172.21.15.204:0/2663963910 shutdown_connections 2024-09-15T01:37:47.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 1 --2- 172.21.15.204:0/2663963910 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6480137d90 0x7f6480099bf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 1 --2- 172.21.15.204:0/2663963910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64800a2c60 0x7f64800996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 1 --2- 172.21.15.204:0/2663963910 >> v2:172.21.15.204:6812/28210 conn(0x7f64640397b0 0x7f646403bc60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 1 --2- 172.21.15.204:0/2663963910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f648009fc60 0x7f6480099170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 1 -- 172.21.15.204:0/2663963910 >> 172.21.15.204:0/2663963910 conn(0x7f64800a60c0 msgr2=0x7f64800298c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:47.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 1 -- 172.21.15.204:0/2663963910 shutdown_connections 2024-09-15T01:37:47.590 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.585+0000 7f6487c4f640 1 -- 172.21.15.204:0/2663963910 wait complete. 2024-09-15T01:37:47.596 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:47.596 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h13 2024-09-15T01:37:47.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.667+0000 7f7be12dd640 10 monclient: get_monmap_and_config 2024-09-15T01:37:47.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.667+0000 7f7be12dd640 10 monclient: build_initial_monmap 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.667+0000 7f7be12dd640 10 monclient: monmap: 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:47.668629+0000 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:47.668629+0000 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:47.670 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:47.671 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:47.671 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:47.671 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:47.671 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:47.671 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:47.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.667+0000 7f7be12dd640 1 Processor -- start 2024-09-15T01:37:47.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.667+0000 7f7be12dd640 1 -- start start 2024-09-15T01:37:47.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.667+0000 7f7be12dd640 10 monclient: init 2024-09-15T01:37:47.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:47.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 10 monclient: _add_conns ranks=[9,3,10,5,2,7,0,8,6,1,4] 2024-09-15T01:37:47.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7bdc1329a0 0x7f7bdc09e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:47.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 10 monclient(hunting): picked mon.noname-a con 0x7f7bdc1329a0 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:47.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bdc132d70 0x7f7bdc133140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:47.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 10 monclient(hunting): picked mon.noname-g con 0x7f7bdc132d70 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:47.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7bdc12f680 0x7f7bdc12fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 10 monclient(hunting): picked mon.noname-f con 0x7f7bdc12f680 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bdc06a2a0 con 0x7f7bdc1329a0 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bdc069800 con 0x7f7bdc12f680 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc069520 con 0x7f7bdc132d70 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7be12dd640 10 monclient(hunting): authenticate will time out at 1083.311326s 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdaffd640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7f7bdc1329a0 msgr2=0x7f7bdc09e8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdb7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7bdc12f680 0x7f7bdc12fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bdc132d70 0x7f7bdc133140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:47.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdaffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7bdc1329a0 0x7f7bdc09e8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdb7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7bdc12f680 0x7f7bdc12fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:37524/0 (socket says 172.21.15.204:37524) 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdb7fe640 1 -- 172.21.15.204:0/2909015294 learned_addr learned my addr 172.21.15.204:0/2909015294 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdb7fe640 10 monclient(hunting): get_auth_request con 0x7f7bdc12f680 auth_method 0 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 10 monclient(hunting): get_auth_request con 0x7f7bdc132d70 auth_method 0 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:47.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 10 monclient(hunting): handle_auth_done global_id 15591 payload 274 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 1 -- 172.21.15.204:0/2909015294 >> v2:172.21.15.204:3300/0 conn(0x7f7bdc12f680 msgr2=0x7f7bdc12fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 1 --2- 172.21.15.204:0/2909015294 >> v2:172.21.15.204:3300/0 conn(0x7f7bdc12f680 0x7f7bdc12fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 1 -- 172.21.15.204:0/2909015294 >> v2:172.21.15.178:3300/0 conn(0x7f7bdc1329a0 msgr2=0x7f7bdc09e8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 1 --2- 172.21.15.204:0/2909015294 >> v2:172.21.15.178:3300/0 conn(0x7f7bdc1329a0 0x7f7bdc09e8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 10 monclient: _finish_hunting 0 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 1 monclient: found mon.noname-g 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:47.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bd27fc640 1 -- 172.21.15.204:0/2909015294 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc04b720 con 0x7f7bdc132d70 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.668+0000 7f7bdb7fe640 1 --2- 172.21.15.204:0/2909015294 >> v2:172.21.15.204:3300/0 conn(0x7f7bdc12f680 0x7f7bdc12fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bd27fc640 1 --2- 172.21.15.204:0/2909015294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bdc132d70 0x7f7bdc133140 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc06bab0 tx=0x7f7bc40090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c97c0f9539d91f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bda7fc640 1 -- 172.21.15.204:0/2909015294 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bc402ac00 con 0x7f7bdc132d70 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bda7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bda7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bda7fc640 10 monclient: dump: 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:47.683 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:47.684 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:47.684 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:47.684 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:47.684 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:47.684 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:47.684 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:47.684 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:47.684 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:47.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bda7fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:47.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bda7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:47.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bda7fc640 1 -- 172.21.15.204:0/2909015294 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc402ad80 con 0x7f7bdc132d70 2024-09-15T01:37:47.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7be12dd640 5 monclient: authenticate success, global_id 15591 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7be12dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bda7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7be12dd640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bda7fc640 1 -- 172.21.15.204:0/2909015294 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bc4031ca0 con 0x7f7bdc132d70 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7be12dd640 4 set_mon_vals no callback set 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bda7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bda7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.669+0000 7f7bda7fc640 10 monclient: dump: 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:47.685 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 10 monclient: shutdown 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:47.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 1 -- 172.21.15.204:0/2909015294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bdc132d70 msgr2=0x7f7bdc133140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 1 --2- 172.21.15.204:0/2909015294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bdc132d70 0x7f7bdc133140 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc06bab0 tx=0x7f7bc40090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 1 -- 172.21.15.204:0/2909015294 shutdown_connections 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 1 --2- 172.21.15.204:0/2909015294 >> v2:172.21.15.204:3300/0 conn(0x7f7bdc12f680 0x7f7bdc12fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 1 --2- 172.21.15.204:0/2909015294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bdc132d70 0x7f7bdc133140 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 1 --2- 172.21.15.204:0/2909015294 >> v2:172.21.15.178:3300/0 conn(0x7f7bdc1329a0 0x7f7bdc09e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 1 -- 172.21.15.204:0/2909015294 >> 172.21.15.204:0/2909015294 conn(0x7f7bdc067c00 msgr2=0x7f7bdc0a3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 1 -- 172.21.15.204:0/2909015294 shutdown_connections 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 1 -- 172.21.15.204:0/2909015294 wait complete. 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 10 monclient: build_initial_monmap 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.670+0000 7f7be12dd640 10 monclient: monmap: 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:47.687 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:47.688 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 1 Processor -- start 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 1 -- start start 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 10 monclient: init 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 10 monclient: _add_conns ranks=[2,6,4,1,8,0,3,5,7] 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bdc12f680 0x7f7bdc12e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 10 monclient(hunting): picked mon.noname-b con 0x7f7bdc12f680 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7bdc1329a0 0x7f7bdc09c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 10 monclient(hunting): picked mon.noname-d con 0x7f7bdc1329a0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7bdc132d70 0x7f7bdc09c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 10 monclient(hunting): picked mon.noname-c con 0x7f7bdc132d70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:47.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc11b860 con 0x7f7bdc12f680 2024-09-15T01:37:47.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:47.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc11be90 con 0x7f7bdc132d70 2024-09-15T01:37:47.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:47.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc09e170 con 0x7f7bdc1329a0 2024-09-15T01:37:47.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:47.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7be12dd640 10 monclient(hunting): authenticate will time out at 1083.314658s 2024-09-15T01:37:47.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7bdaffd640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bdc12f680 msgr2=0x7f7bdc12e740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:47.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7bdb7fe640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7bdc132d70 msgr2=0x7f7bdc09c7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:47.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7bdaffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bdc12f680 0x7f7bdc12e740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:47.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7bdc132d70 0x7f7bdc09c7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:47.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7bd27fc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7bdc1329a0 0x7f7bdc09c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:47.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7bd27fc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7bdc1329a0 0x7f7bdc09c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:38004/0 (socket says 172.21.15.204:38004) 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7bd27fc640 1 -- 172.21.15.204:0/3106242477 learned_addr learned my addr 172.21.15.204:0/3106242477 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7bd27fc640 10 monclient(hunting): get_auth_request con 0x7f7bdc1329a0 auth_method 0 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7bd27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7bd27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.671+0000 7f7bd27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 10 monclient(hunting): handle_auth_done global_id 24513 payload 824 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 1 -- 172.21.15.204:0/3106242477 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7bdc132d70 msgr2=0x7f7bdc09c7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 1 --2- 172.21.15.204:0/3106242477 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7bdc132d70 0x7f7bdc09c7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 1 -- 172.21.15.204:0/3106242477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bdc12f680 msgr2=0x7f7bdc12e740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 1 --2- 172.21.15.204:0/3106242477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bdc12f680 0x7f7bdc12e740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:47.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 10 monclient: _finish_hunting 0 2024-09-15T01:37:47.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 1 monclient: found mon.noname-d 2024-09-15T01:37:47.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:47.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:37:47.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 1 -- 172.21.15.204:0/3106242477 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd4009ae0 con 0x7f7bdc1329a0 2024-09-15T01:37:47.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd27fc640 1 --2- 172.21.15.204:0/3106242477 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7bdc1329a0 0x7f7bdc09c280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4000c00 tx=0x7f7bc4032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:47.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd37fe640 1 -- 172.21.15.204:0/3106242477 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bc403b870 con 0x7f7bdc1329a0 2024-09-15T01:37:47.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:47.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd37fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:37:47.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd37fe640 10 monclient: dump: 2024-09-15T01:37:47.692 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:47.692 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:47.692 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:47.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd37fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd37fe640 1 -- 172.21.15.204:0/3106242477 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc403b9d0 con 0x7f7bdc1329a0 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7be12dd640 5 monclient: authenticate success, global_id 24513 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7be12dd640 10 monclient: _renew_subs 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7be12dd640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd37fe640 1 -- 172.21.15.204:0/3106242477 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bc403bb40 con 0x7f7bdc1329a0 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7be12dd640 1 -- 172.21.15.204:0/3106242477 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc4009390 con 0x7f7bdc1329a0 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd9ffb640 4 set_mon_vals no callback set 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7bd37fe640 10 monclient: dump: 2024-09-15T01:37:47.694 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:47.695 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7be12dd640 10 monclient: _renew_subs 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7be12dd640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.672+0000 7f7be12dd640 1 -- 172.21.15.204:0/3106242477 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc068550 con 0x7f7bdc1329a0 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.673+0000 7f7bd37fe640 1 -- 172.21.15.204:0/3106242477 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7bc4039040 con 0x7f7bdc1329a0 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.673+0000 7f7be12dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.673+0000 7f7be12dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.673+0000 7f7be12dd640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.673+0000 7f7be12dd640 1 -- 172.21.15.204:0/3106242477 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bdc070cc0 con 0x7f7bdc1329a0 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.673+0000 7f7bd37fe640 1 --2- 172.21.15.204:0/3106242477 >> v2:172.21.15.204:6812/28210 conn(0x7f7bbc0397e0 0x7f7bbc03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.673+0000 7f7bd37fe640 1 -- 172.21.15.204:0/3106242477 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 11597+0+0 (secure 0 0 0) 0x7f7bc406c6e0 con 0x7f7bdc1329a0 2024-09-15T01:37:47.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.673+0000 7f7bdaffd640 1 --2- 172.21.15.204:0/3106242477 >> v2:172.21.15.204:6812/28210 conn(0x7f7bbc0397e0 0x7f7bbc03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:47.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.673+0000 7f7bdaffd640 10 monclient: get_auth_request con 0x7f7bbc0397e0 auth_method 0 2024-09-15T01:37:47.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.674+0000 7f7bdaffd640 1 --2- 172.21.15.204:0/3106242477 >> v2:172.21.15.204:6812/28210 conn(0x7f7bbc0397e0 0x7f7bbc03bc90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc006fd0 tx=0x7f7bcc006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:47.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.678+0000 7f7bd37fe640 1 -- 172.21.15.204:0/3106242477 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7bc4044900 con 0x7f7bdc1329a0 2024-09-15T01:37:47.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.678+0000 7f7bd37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:47.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.678+0000 7f7bd37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:47.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.837+0000 7f7be12dd640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h13"}] 2024-09-15T01:37:47.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.837+0000 7f7be12dd640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h13"}] 2024-09-15T01:37:47.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.837+0000 7f7be12dd640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:47.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:47.837+0000 7f7be12dd640 1 -- 172.21.15.204:0/3106242477 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h13"} v 0) v1 -- 0x7f7bdc09dbb0 con 0x7f7bdc1329a0 2024-09-15T01:37:48.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.492+0000 7f7bd37fe640 1 -- 172.21.15.204:0/3106242477 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h13"}]=0 removed item id -55 name 'h13' from crush map v231) v1 ==== 120+0+0 (secure 0 0 0) 0x7f7bc4037090 con 0x7f7bdc1329a0 2024-09-15T01:37:48.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.492+0000 7f7bd37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h13"}] 2024-09-15T01:37:48.495 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.492+0000 7f7bd37fe640 10 monclient: _finish_command 2 = system:0 removed item id -55 name 'h13' from crush map 2024-09-15T01:37:48.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.593+0000 7f7be12dd640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h13"}] 2024-09-15T01:37:48.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.593+0000 7f7be12dd640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h13"}] 2024-09-15T01:37:48.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.593+0000 7f7be12dd640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:48.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.593+0000 7f7be12dd640 1 -- 172.21.15.204:0/3106242477 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h13"} v 0) v1 -- 0x7f7bdc0008d0 con 0x7f7bdc1329a0 2024-09-15T01:37:48.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.594+0000 7f7bd37fe640 1 -- 172.21.15.204:0/3106242477 <== mon.6 v2:172.21.15.178:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h13"}]=0 device 'h13' does not appear in the crush map v231) v1 ==== 120+0+0 (secure 0 0 0) 0x7f7bc4042030 con 0x7f7bdc1329a0 2024-09-15T01:37:48.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.594+0000 7f7bd37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h13"}] 2024-09-15T01:37:48.596 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.594+0000 7f7bd37fe640 10 monclient: _finish_command 3 = system:0 device 'h13' does not appear in the crush map 2024-09-15T01:37:48.596 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h13' does not appear in the crush map 2024-09-15T01:37:48.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 1 -- 172.21.15.204:0/3106242477 >> v2:172.21.15.204:6812/28210 conn(0x7f7bbc0397e0 msgr2=0x7f7bbc03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:48.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 1 --2- 172.21.15.204:0/3106242477 >> v2:172.21.15.204:6812/28210 conn(0x7f7bbc0397e0 0x7f7bbc03bc90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc006fd0 tx=0x7f7bcc006d30 comp rx=0 tx=0).stop 2024-09-15T01:37:48.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 10 monclient: shutdown 2024-09-15T01:37:48.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:48.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 1 -- 172.21.15.204:0/3106242477 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7bdc1329a0 msgr2=0x7f7bdc09c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:48.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 1 --2- 172.21.15.204:0/3106242477 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7bdc1329a0 0x7f7bdc09c280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4000c00 tx=0x7f7bc4032000 comp rx=0 tx=0).stop 2024-09-15T01:37:48.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 1 -- 172.21.15.204:0/3106242477 shutdown_connections 2024-09-15T01:37:48.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 1 --2- 172.21.15.204:0/3106242477 >> v2:172.21.15.204:6812/28210 conn(0x7f7bbc0397e0 0x7f7bbc03bc90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:48.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 1 --2- 172.21.15.204:0/3106242477 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7bdc132d70 0x7f7bdc09c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:48.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 1 --2- 172.21.15.204:0/3106242477 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7bdc1329a0 0x7f7bdc09c280 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:48.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 1 --2- 172.21.15.204:0/3106242477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bdc12f680 0x7f7bdc12e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:48.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 1 -- 172.21.15.204:0/3106242477 >> 172.21.15.204:0/3106242477 conn(0x7f7bdc067c00 msgr2=0x7f7bdc130220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:48.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 1 -- 172.21.15.204:0/3106242477 shutdown_connections 2024-09-15T01:37:48.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.596+0000 7f7be12dd640 1 -- 172.21.15.204:0/3106242477 wait complete. 2024-09-15T01:37:48.610 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:48.610 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h14 2024-09-15T01:37:48.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.678+0000 7fc9a0817640 10 monclient: get_monmap_and_config 2024-09-15T01:37:48.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.678+0000 7fc9a0817640 10 monclient: build_initial_monmap 2024-09-15T01:37:48.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.678+0000 7fc9a0817640 10 monclient: monmap: 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:48.680191+0000 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:48.680191+0000 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:48.682 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:48.683 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:48.683 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:48.683 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:48.683 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:48.683 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:48.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.679+0000 7fc9a0817640 1 Processor -- start 2024-09-15T01:37:48.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.679+0000 7fc9a0817640 1 -- start start 2024-09-15T01:37:48.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.679+0000 7fc9a0817640 10 monclient: init 2024-09-15T01:37:48.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.679+0000 7fc9a0817640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:48.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.679+0000 7fc9a0817640 10 monclient: _add_conns ranks=[5,2,7,0,9,4,3,6,8,1,10] 2024-09-15T01:37:48.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.679+0000 7fc9a0817640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc99c0a2e30 0x7fc99c132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:48.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.679+0000 7fc9a0817640 10 monclient(hunting): picked mon.noname-h con 0x7fc99c0a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:48.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.679+0000 7fc9a0817640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc99c09fb80 0x7fc99c09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:48.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.679+0000 7fc9a0817640 10 monclient(hunting): picked mon.noname-b con 0x7fc99c09fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:48.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.679+0000 7fc9a0817640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a0a40 0x7fc99c0a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:48.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.679+0000 7fc9a0817640 10 monclient(hunting): picked mon.noname-i con 0x7fc99c0a0a40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:48.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.679+0000 7fc9a0817640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:48.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc9a0817640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c09e6c0 con 0x7fc99c09fb80 2024-09-15T01:37:48.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc9a0817640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:48.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc9a0817640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0a04b0 con 0x7fc99c0a2e30 2024-09-15T01:37:48.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc9a0817640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:48.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc9a0817640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0a3200 con 0x7fc99c0a0a40 2024-09-15T01:37:48.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc9a0817640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:48.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc9a0817640 10 monclient(hunting): authenticate will time out at 1084.323033s 2024-09-15T01:37:48.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a0a40 0x7fc99c0a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:48.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99a59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc99c0a2e30 0x7fc99c132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:48.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99a59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc99c0a2e30 0x7fc99c132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:60934/0 (socket says 172.21.15.204:60934) 2024-09-15T01:37:48.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a0a40 0x7fc99c0a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:57488/0 (socket says 172.21.15.204:57488) 2024-09-15T01:37:48.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99a59c640 1 -- 172.21.15.204:0/2202276239 learned_addr learned my addr 172.21.15.204:0/2202276239 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:48.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc999d9b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc99c09fb80 msgr2=0x7fc99c09ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:48.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99a59c640 10 monclient(hunting): get_auth_request con 0x7fc99c0a2e30 auth_method 0 2024-09-15T01:37:48.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:48.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:48.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:48.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 10 monclient(hunting): get_auth_request con 0x7fc99c0a0a40 auth_method 0 2024-09-15T01:37:48.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:48.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:48.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:48.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc999d9b640 1 --2- 172.21.15.204:0/2202276239 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc99c09fb80 0x7fc99c09ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:48.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:48.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:48.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:48.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:48.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:48.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:48.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 10 monclient(hunting): handle_auth_done global_id 24694 payload 274 2024-09-15T01:37:48.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 1 -- 172.21.15.204:0/2202276239 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc99c0a2e30 msgr2=0x7fc99c132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:48.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 1 --2- 172.21.15.204:0/2202276239 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc99c0a2e30 0x7fc99c132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:48.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 1 -- 172.21.15.204:0/2202276239 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc99c09fb80 msgr2=0x7fc99c09ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:48.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 1 --2- 172.21.15.204:0/2202276239 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc99c09fb80 0x7fc99c09ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:48.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:37:48.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 1 monclient: found mon.noname-i 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99ad9d640 1 -- 172.21.15.204:0/2202276239 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c04b720 con 0x7fc99c0a0a40 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.680+0000 7fc99a59c640 1 --2- 172.21.15.204:0/2202276239 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc99c0a2e30 0x7fc99c132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99ad9d640 1 --2- 172.21.15.204:0/2202276239 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a0a40 0x7fc99c0a0e90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc994009e90 tx=0x7fc994009780 comp rx=0 tx=0).ready entity=mon.7 client_cookie=6577f02eb494e944 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99959a640 1 -- 172.21.15.204:0/2202276239 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc9940079d0 con 0x7fc99c0a0a40 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99959a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99959a640 10 monclient: dump: 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:48.696 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99959a640 10 monclient: _finish_auth 0 2024-09-15T01:37:48.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99959a640 1 -- 172.21.15.204:0/2202276239 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99402d050 con 0x7fc99c0a0a40 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 5 monclient: authenticate success, global_id 24694 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99959a640 1 -- 172.21.15.204:0/2202276239 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc994007bd0 con 0x7fc99c0a0a40 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 4 set_mon_vals no callback set 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99959a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc99959a640 10 monclient: dump: 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:48.698 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:48.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 10 monclient: shutdown 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 1 -- 172.21.15.204:0/2202276239 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a0a40 msgr2=0x7fc99c0a0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 1 --2- 172.21.15.204:0/2202276239 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a0a40 0x7fc99c0a0e90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc994009e90 tx=0x7fc994009780 comp rx=0 tx=0).stop 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 1 -- 172.21.15.204:0/2202276239 shutdown_connections 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 1 --2- 172.21.15.204:0/2202276239 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a0a40 0x7fc99c0a0e90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 1 --2- 172.21.15.204:0/2202276239 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc99c09fb80 0x7fc99c09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 1 --2- 172.21.15.204:0/2202276239 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc99c0a2e30 0x7fc99c132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 1 -- 172.21.15.204:0/2202276239 >> 172.21.15.204:0/2202276239 conn(0x7fc99c06e860 msgr2=0x7fc99c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.681+0000 7fc9a0817640 1 -- 172.21.15.204:0/2202276239 shutdown_connections 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.682+0000 7fc9a0817640 1 -- 172.21.15.204:0/2202276239 wait complete. 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.682+0000 7fc9a0817640 10 monclient: build_initial_monmap 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.682+0000 7fc9a0817640 10 monclient: monmap: 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:48.700 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:48.701 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.682+0000 7fc9a0817640 1 Processor -- start 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.682+0000 7fc9a0817640 1 -- start start 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.682+0000 7fc9a0817640 10 monclient: init 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 10 monclient: _add_conns ranks=[3,6,7,4,8,0,1,5,2] 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc99c09fb80 0x7fc99c09b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 10 monclient(hunting): picked mon.noname-g con 0x7fc99c09fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc99c0a0a40 0x7fc99c12e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 10 monclient(hunting): picked mon.noname-d con 0x7fc99c0a0a40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a2e30 0x7fc99c11c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 10 monclient(hunting): picked mon.noname-i con 0x7fc99c0a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:48.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c09d650 con 0x7fc99c09fb80 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c1429b0 con 0x7fc99c0a0a40 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c11efd0 con 0x7fc99c0a2e30 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc9a0817640 10 monclient(hunting): authenticate will time out at 1084.326346s 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc99c09fb80 0x7fc99c09b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99ad9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a2e30 0x7fc99c11c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc99c09fb80 0x7fc99c09b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:34404/0 (socket says 172.21.15.204:34404) 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99ad9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a2e30 0x7fc99c11c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:57490/0 (socket says 172.21.15.204:57490) 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 1 -- 172.21.15.204:0/3957433864 learned_addr learned my addr 172.21.15.204:0/3957433864 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 10 monclient(hunting): get_auth_request con 0x7fc99c09fb80 auth_method 0 2024-09-15T01:37:48.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99ad9d640 10 monclient(hunting): get_auth_request con 0x7fc99c0a2e30 auth_method 0 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc999d9b640 1 --2- 172.21.15.204:0/3957433864 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc99c0a0a40 0x7fc99c12e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:48.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc999d9b640 10 monclient(hunting): get_auth_request con 0x7fc99c0a0a40 auth_method 0 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc999d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc999d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc999d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 10 monclient(hunting): handle_auth_done global_id 15600 payload 824 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 1 -- 172.21.15.204:0/3957433864 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a2e30 msgr2=0x7fc99c11c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 1 --2- 172.21.15.204:0/3957433864 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a2e30 0x7fc99c11c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 1 -- 172.21.15.204:0/3957433864 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc99c0a0a40 msgr2=0x7fc99c12e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 1 --2- 172.21.15.204:0/3957433864 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc99c0a0a40 0x7fc99c12e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 10 monclient: _finish_hunting 0 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 1 monclient: found mon.noname-g 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:48.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 1 -- 172.21.15.204:0/3957433864 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c009450 con 0x7fc99c09fb80 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99ad9d640 1 --2- 172.21.15.204:0/3957433864 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a2e30 0x7fc99c11c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.683+0000 7fc99a59c640 1 --2- 172.21.15.204:0/3957433864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc99c09fb80 0x7fc99c09b730 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00ea60 tx=0x7fc98c0091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 1 -- 172.21.15.204:0/3957433864 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc98c009690 con 0x7fc99c09fb80 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 10 monclient: dump: 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:48.706 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc9a0817640 5 monclient: authenticate success, global_id 15600 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc9a0817640 10 monclient: _renew_subs 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 1 -- 172.21.15.204:0/3957433864 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c00ce90 con 0x7fc99c09fb80 2024-09-15T01:37:48.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc9a0817640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:48.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc9a0817640 1 -- 172.21.15.204:0/3957433864 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc994009a20 con 0x7fc99c09fb80 2024-09-15T01:37:48.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:48.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 1 -- 172.21.15.204:0/3957433864 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc98c0138f0 con 0x7fc99c09fb80 2024-09-15T01:37:48.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:48.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:48.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc998d99640 4 set_mon_vals no callback set 2024-09-15T01:37:48.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 10 monclient: dump: 2024-09-15T01:37:48.708 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:48.708 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:48.708 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:48.708 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:48.708 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc9a0817640 10 monclient: _renew_subs 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc9a0817640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:48.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc9a0817640 1 -- 172.21.15.204:0/3957433864 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99c11da20 con 0x7fc99c09fb80 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.684+0000 7fc98a7fc640 1 -- 172.21.15.204:0/3957433864 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc98c013470 con 0x7fc99c09fb80 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.685+0000 7fc9a0817640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.685+0000 7fc9a0817640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.685+0000 7fc9a0817640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.685+0000 7fc9a0817640 1 -- 172.21.15.204:0/3957433864 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99c070cc0 con 0x7fc99c09fb80 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.685+0000 7fc98a7fc640 1 --2- 172.21.15.204:0/3957433864 >> v2:172.21.15.204:6812/28210 conn(0x7fc97c0397b0 0x7fc97c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.685+0000 7fc98a7fc640 1 -- 172.21.15.204:0/3957433864 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 11384+0+0 (secure 0 0 0) 0x7fc98c04d4b0 con 0x7fc99c09fb80 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.685+0000 7fc999d9b640 1 --2- 172.21.15.204:0/3957433864 >> v2:172.21.15.204:6812/28210 conn(0x7fc97c0397b0 0x7fc97c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.685+0000 7fc999d9b640 10 monclient: get_auth_request con 0x7fc97c0397b0 auth_method 0 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.685+0000 7fc999d9b640 1 --2- 172.21.15.204:0/3957433864 >> v2:172.21.15.204:6812/28210 conn(0x7fc97c0397b0 0x7fc97c03bc60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc990004fd0 tx=0x7fc990006ee0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.689+0000 7fc98a7fc640 1 -- 172.21.15.204:0/3957433864 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc98c01b9b0 con 0x7fc99c09fb80 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.689+0000 7fc98a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:48.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.689+0000 7fc98a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:48.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.850+0000 7fc9a0817640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h14"}] 2024-09-15T01:37:48.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.850+0000 7fc9a0817640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h14"}] 2024-09-15T01:37:48.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.850+0000 7fc9a0817640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:48.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:48.850+0000 7fc9a0817640 1 -- 172.21.15.204:0/3957433864 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h14"} v 0) v1 -- 0x7fc99c0a1f10 con 0x7fc99c09fb80 2024-09-15T01:37:49.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.491+0000 7fc98a7fc640 1 -- 172.21.15.204:0/3957433864 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h14"}]=0 removed item id -58 name 'h14' from crush map v232) v1 ==== 120+0+0 (secure 0 0 0) 0x7fc98c011350 con 0x7fc99c09fb80 2024-09-15T01:37:49.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.491+0000 7fc98a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h14"}] 2024-09-15T01:37:49.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.491+0000 7fc98a7fc640 10 monclient: _finish_command 2 = system:0 removed item id -58 name 'h14' from crush map 2024-09-15T01:37:49.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.591+0000 7fc9a0817640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h14"}] 2024-09-15T01:37:49.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.591+0000 7fc9a0817640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h14"}] 2024-09-15T01:37:49.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.591+0000 7fc9a0817640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:49.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.591+0000 7fc9a0817640 1 -- 172.21.15.204:0/3957433864 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h14"} v 0) v1 -- 0x7fc99c0008d0 con 0x7fc99c09fb80 2024-09-15T01:37:49.594 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.592+0000 7fc98a7fc640 1 -- 172.21.15.204:0/3957433864 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h14"}]=0 device 'h14' does not appear in the crush map v232) v1 ==== 120+0+0 (secure 0 0 0) 0x7fc98c01b3b0 con 0x7fc99c09fb80 2024-09-15T01:37:49.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.592+0000 7fc98a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h14"}] 2024-09-15T01:37:49.595 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.592+0000 7fc98a7fc640 10 monclient: _finish_command 3 = system:0 device 'h14' does not appear in the crush map 2024-09-15T01:37:49.595 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h14' does not appear in the crush map 2024-09-15T01:37:49.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 1 -- 172.21.15.204:0/3957433864 >> v2:172.21.15.204:6812/28210 conn(0x7fc97c0397b0 msgr2=0x7fc97c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:49.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 1 --2- 172.21.15.204:0/3957433864 >> v2:172.21.15.204:6812/28210 conn(0x7fc97c0397b0 0x7fc97c03bc60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc990004fd0 tx=0x7fc990006ee0 comp rx=0 tx=0).stop 2024-09-15T01:37:49.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 10 monclient: shutdown 2024-09-15T01:37:49.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:49.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 1 -- 172.21.15.204:0/3957433864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc99c09fb80 msgr2=0x7fc99c09b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:49.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 1 --2- 172.21.15.204:0/3957433864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc99c09fb80 0x7fc99c09b730 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00ea60 tx=0x7fc98c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:49.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 1 -- 172.21.15.204:0/3957433864 shutdown_connections 2024-09-15T01:37:49.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 1 --2- 172.21.15.204:0/3957433864 >> v2:172.21.15.204:6812/28210 conn(0x7fc97c0397b0 0x7fc97c03bc60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:49.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 1 --2- 172.21.15.204:0/3957433864 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc99c0a2e30 0x7fc99c11c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:49.597 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 1 --2- 172.21.15.204:0/3957433864 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc99c0a0a40 0x7fc99c12e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:49.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 1 --2- 172.21.15.204:0/3957433864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc99c09fb80 0x7fc99c09b730 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:49.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 1 -- 172.21.15.204:0/3957433864 >> 172.21.15.204:0/3957433864 conn(0x7fc99c06e860 msgr2=0x7fc99c09ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:49.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 1 -- 172.21.15.204:0/3957433864 shutdown_connections 2024-09-15T01:37:49.598 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.594+0000 7fc9a0817640 1 -- 172.21.15.204:0/3957433864 wait complete. 2024-09-15T01:37:49.605 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:49.605 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h15 2024-09-15T01:37:49.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.675+0000 7feb2eb9d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:49.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.675+0000 7feb2eb9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:49.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.675+0000 7feb2eb9d640 10 monclient: monmap: 2024-09-15T01:37:49.678 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:49.678 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:49.678 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:49.676794+0000 2024-09-15T01:37:49.678 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:49.676794+0000 2024-09-15T01:37:49.678 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:49.678 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:49.679 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:49.679 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:49.679 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:49.679 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:49.679 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:49.679 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:49.679 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:49.679 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:49.679 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:49.679 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:49.679 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:49.679 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:49.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 1 Processor -- start 2024-09-15T01:37:49.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 1 -- start start 2024-09-15T01:37:49.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 10 monclient: init 2024-09-15T01:37:49.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:49.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 10 monclient: _add_conns ranks=[6,1,10,3,2,0,4,7,8,5,9] 2024-09-15T01:37:49.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7feb280a2e30 0x7feb28132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:49.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 10 monclient(hunting): picked mon.noname-d con 0x7feb280a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:49.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7feb2809fb80 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:49.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7feb2809ff70 0x7feb280a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:49.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 10 monclient(hunting): picked mon.noname-f con 0x7feb2809ff70 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:49.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2804b720 con 0x7feb2809fb80 2024-09-15T01:37:49.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:49.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2809e6c0 con 0x7feb2809ff70 2024-09-15T01:37:49.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:49.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feb280a3200 con 0x7feb280a2e30 2024-09-15T01:37:49.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:49.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2eb9d640 10 monclient(hunting): authenticate will time out at 1085.319603s 2024-09-15T01:37:49.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2d13a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7feb2809ff70 0x7feb280a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:49.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2d13a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7feb2809ff70 0x7feb280a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:37526/0 (socket says 172.21.15.204:37526) 2024-09-15T01:37:49.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2d13a640 1 -- 172.21.15.204:0/2106709530 learned_addr learned my addr 172.21.15.204:0/2106709530 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:49.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2d13a640 10 monclient(hunting): get_auth_request con 0x7feb2809ff70 auth_method 0 2024-09-15T01:37:49.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2d13a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:49.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2d13a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:49.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2d13a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:49.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2d13a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:49.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2d13a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:49.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2d13a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:49.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2c939640 1 --2- 172.21.15.204:0/2106709530 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7feb280a2e30 0x7feb28132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:49.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2c939640 10 monclient(hunting): get_auth_request con 0x7feb280a2e30 auth_method 0 2024-09-15T01:37:49.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2c939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:49.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2c939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:49.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2c939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:49.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.676+0000 7feb2d13a640 10 monclient(hunting): handle_auth_done global_id 7327 payload 274 2024-09-15T01:37:49.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2d13a640 1 -- 172.21.15.204:0/2106709530 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7feb280a2e30 msgr2=0x7feb28132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:49.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2d13a640 1 --2- 172.21.15.204:0/2106709530 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7feb280a2e30 0x7feb28132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:49.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2d13a640 1 -- 172.21.15.204:0/2106709530 >> v1:172.21.15.204:6789/0 conn(0x7feb2809fb80 legacy=0x7feb280a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:49.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2d13a640 10 monclient: _finish_hunting 0 2024-09-15T01:37:49.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2d13a640 1 monclient: found mon.noname-f 2024-09-15T01:37:49.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2d13a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:49.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2d13a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:49.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2d13a640 1 -- 172.21.15.204:0/2106709530 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb280a1f10 con 0x7feb2809ff70 2024-09-15T01:37:49.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2d13a640 1 --2- 172.21.15.204:0/2106709530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb2809ff70 0x7feb280a1150 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feb20009f20 tx=0x7feb20009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccba0b40618ca4a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:49.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb277fe640 1 -- 172.21.15.204:0/2106709530 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb20007880 con 0x7feb2809ff70 2024-09-15T01:37:49.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:49.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb277fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:49.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb277fe640 10 monclient: dump: 2024-09-15T01:37:49.692 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:49.692 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:49.692 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:49.692 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:49.692 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:49.692 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:49.692 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:49.692 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:49.692 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:49.693 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:49.693 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:49.693 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:49.693 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:49.693 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:49.693 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:49.693 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:49.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb277fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:49.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2eb9d640 5 monclient: authenticate success, global_id 7327 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2eb9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb277fe640 1 -- 172.21.15.204:0/2106709530 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2002d050 con 0x7feb2809ff70 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2eb9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb2eb9d640 4 set_mon_vals no callback set 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb277fe640 1 -- 172.21.15.204:0/2106709530 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb20030360 con 0x7feb2809ff70 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb277fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.677+0000 7feb277fe640 10 monclient: dump: 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:49.694 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:49.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.678+0000 7feb2eb9d640 10 monclient: shutdown 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.678+0000 7feb2eb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.678+0000 7feb2eb9d640 1 -- 172.21.15.204:0/2106709530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb2809ff70 msgr2=0x7feb280a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.678+0000 7feb2eb9d640 1 --2- 172.21.15.204:0/2106709530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb2809ff70 0x7feb280a1150 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feb20009f20 tx=0x7feb20009840 comp rx=0 tx=0).stop 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.678+0000 7feb2eb9d640 1 -- 172.21.15.204:0/2106709530 shutdown_connections 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.678+0000 7feb2eb9d640 1 --2- 172.21.15.204:0/2106709530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb2809ff70 0x7feb280a1150 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.678+0000 7feb2eb9d640 1 --2- 172.21.15.204:0/2106709530 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7feb280a2e30 0x7feb28132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.678+0000 7feb2eb9d640 1 -- 172.21.15.204:0/2106709530 >> 172.21.15.204:0/2106709530 conn(0x7feb2806e860 msgr2=0x7feb2806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.678+0000 7feb2eb9d640 1 -- 172.21.15.204:0/2106709530 shutdown_connections 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.678+0000 7feb2eb9d640 1 -- 172.21.15.204:0/2106709530 wait complete. 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.678+0000 7feb2eb9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.678+0000 7feb2eb9d640 10 monclient: monmap: 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:49.696 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:49.697 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 1 Processor -- start 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 1 -- start start 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 10 monclient: init 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 10 monclient: _add_conns ranks=[3,5,7,1,4,0,8,2,6] 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb2809fb80 0x7feb28141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 10 monclient(hunting): picked mon.noname-g con 0x7feb2809fb80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb2809ff70 0x7feb28131a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 10 monclient(hunting): picked mon.noname-h con 0x7feb2809ff70 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feb280a2e30 0x7feb2811e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 10 monclient(hunting): picked mon.noname-i con 0x7feb280a2e30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:49.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2809ca40 con 0x7feb2809fb80 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb2809d650 con 0x7feb2809ff70 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feb281200c0 con 0x7feb280a2e30 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2eb9d640 10 monclient(hunting): authenticate will time out at 1085.322862s 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2c939640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb2809fb80 0x7feb28141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2d13a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feb280a2e30 0x7feb2811e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2c939640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb2809fb80 0x7feb28141bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:34412/0 (socket says 172.21.15.204:34412) 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2c939640 1 -- 172.21.15.204:0/3979109362 learned_addr learned my addr 172.21.15.204:0/3979109362 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2c939640 10 monclient(hunting): get_auth_request con 0x7feb2809fb80 auth_method 0 2024-09-15T01:37:49.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2c939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2c939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb2c939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.679+0000 7feb27fff640 1 --2- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb2809ff70 0x7feb28131a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2d13a640 10 monclient(hunting): get_auth_request con 0x7feb280a2e30 auth_method 0 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2d13a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2d13a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2d13a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb27fff640 10 monclient(hunting): get_auth_request con 0x7feb2809ff70 auth_method 0 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb27fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:49.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2d13a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2d13a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2d13a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 10 monclient(hunting): handle_auth_done global_id 15609 payload 824 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 1 -- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feb280a2e30 msgr2=0x7feb2811e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 1 --2- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feb280a2e30 0x7feb2811e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 1 -- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb2809ff70 msgr2=0x7feb28131a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 1 --2- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb2809ff70 0x7feb28131a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 10 monclient: _finish_hunting 0 2024-09-15T01:37:49.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 1 monclient: found mon.noname-g 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 1 -- 172.21.15.204:0/3979109362 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb20009ae0 con 0x7feb2809fb80 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2d13a640 1 --2- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feb280a2e30 0x7feb2811e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb27fff640 1 --2- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb2809ff70 0x7feb28131a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2c939640 1 --2- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb2809fb80 0x7feb28141bd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00d890 tx=0x7feb1c008040 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb24ff9640 1 -- 172.21.15.204:0/3979109362 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb1c00de20 con 0x7feb2809fb80 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb24ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb24ff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb24ff9640 10 monclient: dump: 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:49.702 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:49.703 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:49.703 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:49.703 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:49.703 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:49.703 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:49.703 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:49.703 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:49.703 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:49.703 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:49.704 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:49.704 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:49.704 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:49.704 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:49.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb24ff9640 10 monclient: _finish_auth 0 2024-09-15T01:37:49.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb24ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:49.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb24ff9640 1 -- 172.21.15.204:0/3979109362 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c009d30 con 0x7feb2809fb80 2024-09-15T01:37:49.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2eb9d640 5 monclient: authenticate success, global_id 15609 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2eb9d640 10 monclient: _renew_subs 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2eb9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2eb9d640 1 -- 172.21.15.204:0/3979109362 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb28121be0 con 0x7feb2809fb80 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb24ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb24ff9640 1 -- 172.21.15.204:0/3979109362 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb1c009ea0 con 0x7feb2809fb80 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb24ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb24ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb24ff9640 10 monclient: dump: 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:49.705 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb26ffd640 4 set_mon_vals no callback set 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2eb9d640 10 monclient: _renew_subs 2024-09-15T01:37:49.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2eb9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:49.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.680+0000 7feb2eb9d640 1 -- 172.21.15.204:0/3979109362 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb281429a0 con 0x7feb2809fb80 2024-09-15T01:37:49.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.681+0000 7feb24ff9640 1 -- 172.21.15.204:0/3979109362 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7feb1c0098b0 con 0x7feb2809fb80 2024-09-15T01:37:49.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.681+0000 7feb2eb9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:49.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.681+0000 7feb2eb9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:49.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.681+0000 7feb2eb9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:49.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.681+0000 7feb2eb9d640 1 -- 172.21.15.204:0/3979109362 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb28070cc0 con 0x7feb2809fb80 2024-09-15T01:37:49.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.681+0000 7feb24ff9640 1 --2- 172.21.15.204:0/3979109362 >> v2:172.21.15.204:6812/28210 conn(0x7feb0c0397e0 0x7feb0c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:49.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.682+0000 7feb24ff9640 1 -- 172.21.15.204:0/3979109362 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 11171+0+0 (secure 0 0 0) 0x7feb1c04d480 con 0x7feb2809fb80 2024-09-15T01:37:49.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.682+0000 7feb27fff640 1 --2- 172.21.15.204:0/3979109362 >> v2:172.21.15.204:6812/28210 conn(0x7feb0c0397e0 0x7feb0c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:49.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.682+0000 7feb27fff640 10 monclient: get_auth_request con 0x7feb0c0397e0 auth_method 0 2024-09-15T01:37:49.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.682+0000 7feb27fff640 1 --2- 172.21.15.204:0/3979109362 >> v2:172.21.15.204:6812/28210 conn(0x7feb0c0397e0 0x7feb0c03bc90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7feb14007fd0 tx=0x7feb14009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:49.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.686+0000 7feb24ff9640 1 -- 172.21.15.204:0/3979109362 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7feb1c025580 con 0x7feb2809fb80 2024-09-15T01:37:49.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.686+0000 7feb24ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:49.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.686+0000 7feb24ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.844+0000 7feb2eb9d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h15"}] 2024-09-15T01:37:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.844+0000 7feb2eb9d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h15"}] 2024-09-15T01:37:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.844+0000 7feb2eb9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:49.847 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:49.844+0000 7feb2eb9d640 1 -- 172.21.15.204:0/3979109362 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h15"} v 0) v1 -- 0x7feb2809ddb0 con 0x7feb2809fb80 2024-09-15T01:37:50.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.504+0000 7feb24ff9640 1 -- 172.21.15.204:0/3979109362 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h15"}]=0 removed item id -61 name 'h15' from crush map v233) v1 ==== 120+0+0 (secure 0 0 0) 0x7feb1c018090 con 0x7feb2809fb80 2024-09-15T01:37:50.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.504+0000 7feb24ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h15"}] 2024-09-15T01:37:50.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.504+0000 7feb24ff9640 10 monclient: _finish_command 2 = system:0 removed item id -61 name 'h15' from crush map 2024-09-15T01:37:50.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.605+0000 7feb2eb9d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h15"}] 2024-09-15T01:37:50.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.605+0000 7feb2eb9d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h15"}] 2024-09-15T01:37:50.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.605+0000 7feb2eb9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:50.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.605+0000 7feb2eb9d640 1 -- 172.21.15.204:0/3979109362 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h15"} v 0) v1 -- 0x7feb280008d0 con 0x7feb2809fb80 2024-09-15T01:37:50.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.606+0000 7feb24ff9640 1 -- 172.21.15.204:0/3979109362 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h15"}]=0 device 'h15' does not appear in the crush map v233) v1 ==== 120+0+0 (secure 0 0 0) 0x7feb1c013b60 con 0x7feb2809fb80 2024-09-15T01:37:50.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.606+0000 7feb24ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h15"}] 2024-09-15T01:37:50.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.606+0000 7feb24ff9640 10 monclient: _finish_command 3 = system:0 device 'h15' does not appear in the crush map 2024-09-15T01:37:50.609 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h15' does not appear in the crush map 2024-09-15T01:37:50.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 1 -- 172.21.15.204:0/3979109362 >> v2:172.21.15.204:6812/28210 conn(0x7feb0c0397e0 msgr2=0x7feb0c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:50.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 1 --2- 172.21.15.204:0/3979109362 >> v2:172.21.15.204:6812/28210 conn(0x7feb0c0397e0 0x7feb0c03bc90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7feb14007fd0 tx=0x7feb14009040 comp rx=0 tx=0).stop 2024-09-15T01:37:50.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 10 monclient: shutdown 2024-09-15T01:37:50.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:50.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 1 -- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb2809fb80 msgr2=0x7feb28141bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:50.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 1 --2- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb2809fb80 0x7feb28141bd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00d890 tx=0x7feb1c008040 comp rx=0 tx=0).stop 2024-09-15T01:37:50.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 1 -- 172.21.15.204:0/3979109362 shutdown_connections 2024-09-15T01:37:50.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 1 --2- 172.21.15.204:0/3979109362 >> v2:172.21.15.204:6812/28210 conn(0x7feb0c0397e0 0x7feb0c03bc90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:50.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 1 --2- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7feb280a2e30 0x7feb2811e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:50.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 1 --2- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7feb2809ff70 0x7feb28131a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:50.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 1 --2- 172.21.15.204:0/3979109362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb2809fb80 0x7feb28141bd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:50.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 1 -- 172.21.15.204:0/3979109362 >> 172.21.15.204:0/3979109362 conn(0x7feb2806e860 msgr2=0x7feb280a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:50.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 1 -- 172.21.15.204:0/3979109362 shutdown_connections 2024-09-15T01:37:50.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.608+0000 7feb2eb9d640 1 -- 172.21.15.204:0/3979109362 wait complete. 2024-09-15T01:37:50.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:50.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h16 2024-09-15T01:37:50.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.691+0000 7f7c4ad9d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:50.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.691+0000 7f7c4ad9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:50.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.691+0000 7f7c4ad9d640 10 monclient: monmap: 2024-09-15T01:37:50.694 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:50.694 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:50.694 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:50.693025+0000 2024-09-15T01:37:50.694 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:50.693025+0000 2024-09-15T01:37:50.694 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:50.694 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:50.694 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:50.695 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:50.695 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:50.695 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:50.695 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:50.695 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:50.695 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:50.695 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:50.695 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:50.695 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:50.695 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:50.695 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:50.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 1 Processor -- start 2024-09-15T01:37:50.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 1 -- start start 2024-09-15T01:37:50.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 10 monclient: init 2024-09-15T01:37:50.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:50.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 10 monclient: _add_conns ranks=[9,0,5,6,4,7,1,8,10,2,3] 2024-09-15T01:37:50.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7c440a2e30 0x7f7c44132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:50.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 10 monclient(hunting): picked mon.noname-a con 0x7f7c440a2e30 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:50.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7c4409fb80 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:50.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7c4409ff70 0x7f7c440a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:50.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 10 monclient(hunting): picked mon.noname-h con 0x7f7c4409ff70 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:50.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c4404b720 con 0x7f7c4409fb80 2024-09-15T01:37:50.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:50.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4409e6c0 con 0x7f7c440a2e30 2024-09-15T01:37:50.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:50.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440a3200 con 0x7f7c4409ff70 2024-09-15T01:37:50.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:50.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4ad9d640 10 monclient(hunting): authenticate will time out at 1086.335869s 2024-09-15T01:37:50.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4933a640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7c4409ff70 0x7f7c440a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:50.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c48b39640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7f7c440a2e30 msgr2=0x7f7c44132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:50.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c3bfff640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f7c4409fb80 legacy=0x7f7c440a0a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:50.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4933a640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7c4409ff70 0x7f7c440a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:60956/0 (socket says 172.21.15.204:60956) 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c48b39640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7c440a2e30 0x7f7c44132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.692+0000 7f7c4933a640 1 -- 172.21.15.204:0/1381476781 learned_addr learned my addr 172.21.15.204:0/1381476781 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 10 monclient(hunting): get_auth_request con 0x7f7c4409ff70 auth_method 0 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 10 monclient(hunting): handle_auth_done global_id 6458 payload 274 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 1 -- 172.21.15.204:0/1381476781 >> v2:172.21.15.178:3300/0 conn(0x7f7c440a2e30 msgr2=0x7f7c44132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 1 --2- 172.21.15.204:0/1381476781 >> v2:172.21.15.178:3300/0 conn(0x7f7c440a2e30 0x7f7c44132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:50.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 1 -- 172.21.15.204:0/1381476781 >> v1:172.21.15.178:6789/0 conn(0x7f7c4409fb80 legacy=0x7f7c440a0a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 10 monclient: _finish_hunting 0 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 1 monclient: found mon.noname-h 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 1 -- 172.21.15.204:0/1381476781 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440a1f10 con 0x7f7c4409ff70 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.693+0000 7f7c4933a640 1 --2- 172.21.15.204:0/1381476781 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7c4409ff70 0x7f7c440a1150 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009f20 tx=0x7f7c3c009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=64c94ab4236b63a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c3b7fe640 1 -- 172.21.15.204:0/1381476781 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c3c007880 con 0x7f7c4409ff70 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c3b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c3b7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c3b7fe640 10 monclient: dump: 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:50.707 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:50.708 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:50.708 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:50.708 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:50.708 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:50.708 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:50.708 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:50.708 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:50.708 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:50.708 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:50.708 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:50.708 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:50.708 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c3b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c3b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c3b7fe640 1 -- 172.21.15.204:0/1381476781 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c02d050 con 0x7f7c4409ff70 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 5 monclient: authenticate success, global_id 6458 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c3b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c3b7fe640 1 -- 172.21.15.204:0/1381476781 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c3c030360 con 0x7f7c4409ff70 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c3b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 4 set_mon_vals no callback set 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c3b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:50.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c3b7fe640 10 monclient: dump: 2024-09-15T01:37:50.710 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:50.710 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:50.710 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:50.710 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:50.710 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:50.710 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:50.710 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:50.710 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:50.710 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:50.710 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:50.710 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:50.710 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 10 monclient: shutdown 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1381476781 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7c4409ff70 msgr2=0x7f7c440a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 1 --2- 172.21.15.204:0/1381476781 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7c4409ff70 0x7f7c440a1150 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009f20 tx=0x7f7c3c009840 comp rx=0 tx=0).stop 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1381476781 shutdown_connections 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 1 --2- 172.21.15.204:0/1381476781 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7c4409ff70 0x7f7c440a1150 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 1 --2- 172.21.15.204:0/1381476781 >> v2:172.21.15.178:3300/0 conn(0x7f7c440a2e30 0x7f7c44132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1381476781 >> 172.21.15.204:0/1381476781 conn(0x7f7c4406e860 msgr2=0x7f7c4406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1381476781 shutdown_connections 2024-09-15T01:37:50.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.694+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1381476781 wait complete. 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.695+0000 7f7c4ad9d640 10 monclient: build_initial_monmap 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.695+0000 7f7c4ad9d640 10 monclient: monmap: 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:50.712 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.695+0000 7f7c4ad9d640 1 Processor -- start 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.695+0000 7f7c4ad9d640 1 -- start start 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.695+0000 7f7c4ad9d640 10 monclient: init 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.695+0000 7f7c4ad9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.695+0000 7f7c4ad9d640 10 monclient: _add_conns ranks=[0,8,6,4,2,3,7,5,1] 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c4409fb80 0x7f7c44130730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 10 monclient(hunting): picked mon.noname-a con 0x7f7c4409fb80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c4409ff70 0x7f7c4411d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 10 monclient(hunting): picked mon.noname-e con 0x7f7c4409ff70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:50.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c440a2e30 0x7f7c4411d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 10 monclient(hunting): picked mon.noname-d con 0x7f7c440a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44120880 con 0x7f7c4409fb80 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4409ca40 con 0x7f7c440a2e30 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44142970 con 0x7f7c4409ff70 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4ad9d640 10 monclient(hunting): authenticate will time out at 1086.339048s 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c48b39640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c4409fb80 msgr2=0x7f7c44130730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c3bfff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c4409ff70 msgr2=0x7f7c4411d1e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c48b39640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c4409fb80 0x7f7c44130730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:50.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c3bfff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c4409ff70 0x7f7c4411d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c440a2e30 0x7f7c4411d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c440a2e30 0x7f7c4411d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:38040/0 (socket says 172.21.15.204:38040) 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 1 -- 172.21.15.204:0/1505926586 learned_addr learned my addr 172.21.15.204:0/1505926586 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 10 monclient(hunting): get_auth_request con 0x7f7c440a2e30 auth_method 0 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 10 monclient(hunting): handle_auth_done global_id 24540 payload 824 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 1 -- 172.21.15.204:0/1505926586 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c4409ff70 msgr2=0x7f7c4411d1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 1 --2- 172.21.15.204:0/1505926586 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c4409ff70 0x7f7c4411d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 1 -- 172.21.15.204:0/1505926586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c4409fb80 msgr2=0x7f7c44130730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:50.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 1 --2- 172.21.15.204:0/1505926586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c4409fb80 0x7f7c44130730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 10 monclient: _finish_hunting 0 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 1 monclient: found mon.noname-d 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.696+0000 7f7c4933a640 1 -- 172.21.15.204:0/1505926586 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3c009ae0 con 0x7f7c440a2e30 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c4933a640 1 --2- 172.21.15.204:0/1505926586 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c440a2e30 0x7f7c4411d720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009880 tx=0x7f7c3c036000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c38ff9640 1 -- 172.21.15.204:0/1505926586 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c3c008b20 con 0x7f7c440a2e30 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c38ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c38ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c38ff9640 10 monclient: dump: 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:50.716 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c38ff9640 10 monclient: _finish_auth 0 2024-09-15T01:37:50.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c38ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c38ff9640 1 -- 172.21.15.204:0/1505926586 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c008cc0 con 0x7f7c440a2e30 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c4ad9d640 5 monclient: authenticate success, global_id 24540 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c4ad9d640 10 monclient: _renew_subs 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c4ad9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1505926586 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4409d650 con 0x7f7c440a2e30 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c38ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c38ff9640 1 -- 172.21.15.204:0/1505926586 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c3c008e30 con 0x7f7c440a2e30 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c38ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c38ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c38ff9640 10 monclient: dump: 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:50.718 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c4ad9d640 10 monclient: _renew_subs 2024-09-15T01:37:50.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c4ad9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:50.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1505926586 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4411f4c0 con 0x7f7c440a2e30 2024-09-15T01:37:50.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.697+0000 7f7c3affd640 4 set_mon_vals no callback set 2024-09-15T01:37:50.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.698+0000 7f7c38ff9640 1 -- 172.21.15.204:0/1505926586 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7c3c038050 con 0x7f7c440a2e30 2024-09-15T01:37:50.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.698+0000 7f7c4ad9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:50.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.698+0000 7f7c4ad9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:50.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.698+0000 7f7c4ad9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:50.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.698+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1505926586 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c44070cc0 con 0x7f7c440a2e30 2024-09-15T01:37:50.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.698+0000 7f7c38ff9640 1 --2- 172.21.15.204:0/1505926586 >> v2:172.21.15.204:6812/28210 conn(0x7f7c300397b0 0x7f7c3003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:50.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.698+0000 7f7c38ff9640 1 -- 172.21.15.204:0/1505926586 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 10958+0+0 (secure 0 0 0) 0x7f7c3c06d6c0 con 0x7f7c440a2e30 2024-09-15T01:37:50.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.698+0000 7f7c48b39640 1 --2- 172.21.15.204:0/1505926586 >> v2:172.21.15.204:6812/28210 conn(0x7f7c300397b0 0x7f7c3003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:50.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.698+0000 7f7c48b39640 10 monclient: get_auth_request con 0x7f7c300397b0 auth_method 0 2024-09-15T01:37:50.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.698+0000 7f7c48b39640 1 --2- 172.21.15.204:0/1505926586 >> v2:172.21.15.204:6812/28210 conn(0x7f7c300397b0 0x7f7c3003bc60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006fd0 tx=0x7f7c40006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:50.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.703+0000 7f7c38ff9640 1 -- 172.21.15.204:0/1505926586 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7c3c042960 con 0x7f7c440a2e30 2024-09-15T01:37:50.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.703+0000 7f7c38ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:50.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.703+0000 7f7c38ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:50.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.864+0000 7f7c4ad9d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h16"}] 2024-09-15T01:37:50.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.864+0000 7f7c4ad9d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h16"}] 2024-09-15T01:37:50.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.864+0000 7f7c4ad9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:50.867 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:50.864+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1505926586 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h16"} v 0) v1 -- 0x7f7c4409ddb0 con 0x7f7c440a2e30 2024-09-15T01:37:51.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.528+0000 7f7c38ff9640 1 -- 172.21.15.204:0/1505926586 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h16"}]=0 removed item id -64 name 'h16' from crush map v234) v1 ==== 120+0+0 (secure 0 0 0) 0x7f7c3c049020 con 0x7f7c440a2e30 2024-09-15T01:37:51.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.528+0000 7f7c38ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h16"}] 2024-09-15T01:37:51.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.528+0000 7f7c38ff9640 10 monclient: _finish_command 2 = system:0 removed item id -64 name 'h16' from crush map 2024-09-15T01:37:51.633 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.631+0000 7f7c4ad9d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h16"}] 2024-09-15T01:37:51.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.631+0000 7f7c4ad9d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h16"}] 2024-09-15T01:37:51.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.631+0000 7f7c4ad9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:51.634 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.631+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1505926586 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h16"} v 0) v1 -- 0x7f7c440008d0 con 0x7f7c440a2e30 2024-09-15T01:37:51.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.632+0000 7f7c38ff9640 1 -- 172.21.15.204:0/1505926586 <== mon.6 v2:172.21.15.178:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h16"}]=0 device 'h16' does not appear in the crush map v234) v1 ==== 120+0+0 (secure 0 0 0) 0x7f7c3c03f070 con 0x7f7c440a2e30 2024-09-15T01:37:51.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.632+0000 7f7c38ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h16"}] 2024-09-15T01:37:51.636 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.632+0000 7f7c38ff9640 10 monclient: _finish_command 3 = system:0 device 'h16' does not appear in the crush map 2024-09-15T01:37:51.636 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h16' does not appear in the crush map 2024-09-15T01:37:51.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1505926586 >> v2:172.21.15.204:6812/28210 conn(0x7f7c300397b0 msgr2=0x7f7c3003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:51.637 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 1 --2- 172.21.15.204:0/1505926586 >> v2:172.21.15.204:6812/28210 conn(0x7f7c300397b0 0x7f7c3003bc60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006fd0 tx=0x7f7c40006d30 comp rx=0 tx=0).stop 2024-09-15T01:37:51.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 10 monclient: shutdown 2024-09-15T01:37:51.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:51.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1505926586 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c440a2e30 msgr2=0x7f7c4411d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:51.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 1 --2- 172.21.15.204:0/1505926586 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c440a2e30 0x7f7c4411d720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009880 tx=0x7f7c3c036000 comp rx=0 tx=0).stop 2024-09-15T01:37:51.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1505926586 shutdown_connections 2024-09-15T01:37:51.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 1 --2- 172.21.15.204:0/1505926586 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7c440a2e30 0x7f7c4411d720 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:51.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 1 --2- 172.21.15.204:0/1505926586 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7c4409ff70 0x7f7c4411d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:51.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 1 --2- 172.21.15.204:0/1505926586 >> v2:172.21.15.204:6812/28210 conn(0x7f7c300397b0 0x7f7c3003bc60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:51.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 1 --2- 172.21.15.204:0/1505926586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c4409fb80 0x7f7c44130730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:51.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1505926586 >> 172.21.15.204:0/1505926586 conn(0x7f7c4406e860 msgr2=0x7f7c44131870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:51.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1505926586 shutdown_connections 2024-09-15T01:37:51.638 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.634+0000 7f7c4ad9d640 1 -- 172.21.15.204:0/1505926586 wait complete. 2024-09-15T01:37:51.646 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:51.646 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h17 2024-09-15T01:37:51.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.717+0000 7faf910e8640 10 monclient: get_monmap_and_config 2024-09-15T01:37:51.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.717+0000 7faf910e8640 10 monclient: build_initial_monmap 2024-09-15T01:37:51.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.717+0000 7faf910e8640 10 monclient: monmap: 2024-09-15T01:37:51.720 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:51.720 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:51.720 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:51.718854+0000 2024-09-15T01:37:51.720 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:51.718854+0000 2024-09-15T01:37:51.720 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:51.721 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:51.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.717+0000 7faf910e8640 1 Processor -- start 2024-09-15T01:37:51.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 1 -- start start 2024-09-15T01:37:51.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 10 monclient: init 2024-09-15T01:37:51.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:51.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 10 monclient: _add_conns ranks=[9,0,1,7,2,6,5,10,8,3,4] 2024-09-15T01:37:51.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faf8c0a2e30 0x7faf8c132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:51.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 10 monclient(hunting): picked mon.noname-a con 0x7faf8c0a2e30 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:51.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faf8c09fb80 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:51.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7faf8c09ff70 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:51.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8c04b720 con 0x7faf8c09fb80 2024-09-15T01:37:51.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8c0a3200 con 0x7faf8c09ff70 2024-09-15T01:37:51.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:51.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf8c09e6c0 con 0x7faf8c0a2e30 2024-09-15T01:37:51.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:51.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf910e8640 10 monclient(hunting): authenticate will time out at 1087.361534s 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf8ad9d640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7faf8c0a2e30 msgr2=0x7faf8c132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf8a59c640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7faf8c09fb80 legacy=0x7faf8c0a0a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf8b59e640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7faf8c09ff70 0x7faf8c0a1150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:48354/0 (socket says 172.21.15.204:48354) 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf8b59e640 1 -- 172.21.15.204:0/3617221362 learned_addr learned my addr 172.21.15.204:0/3617221362 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf8ad9d640 1 --2- 172.21.15.204:0/3617221362 >> v2:172.21.15.178:3300/0 conn(0x7faf8c0a2e30 0x7faf8c132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358641057 0 0) 0x7faf8c0a3200 con 0x7faf8c09ff70 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf89d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf89d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf89d9b640 10 monclient(hunting): my global_id is 7336 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.718+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf780034e0 con 0x7faf8c09ff70 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 150736720 0 0) 0x7faf780034e0 con 0x7faf8c09ff70 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 >> v2:172.21.15.178:3300/0 conn(0x7faf8c0a2e30 msgr2=0x7faf8c132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:51.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 --2- 172.21.15.204:0/3617221362 >> v2:172.21.15.178:3300/0 conn(0x7faf8c0a2e30 0x7faf8c132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 >> v1:172.21.15.178:6789/0 conn(0x7faf8c09fb80 legacy=0x7faf8c0a0a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c0a1860 con 0x7faf8c09ff70 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient: _finish_auth 0 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf910e8640 5 monclient: authenticate success, global_id 7336 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7faf84001c70 con 0x7faf8c09ff70 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:37:51.733 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient: dump: 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:51.734 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 >> v1:172.21.15.204:6789/0 conn(0x7faf8c09ff70 legacy=0x7faf8c0a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient: _add_conns ranks=[5,2,6,8,7,4,0,3,1] 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 --2- 172.21.15.204:0/3617221362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 0x7faf78004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient(hunting): picked mon.h con 0x7faf78004220 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 --2- 172.21.15.204:0/3617221362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf78004ec0 0x7faf780099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient(hunting): picked mon.b con 0x7faf78004ec0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 --2- 172.21.15.204:0/3617221362 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7faf78009f00 0x7faf7800c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient(hunting): picked mon.d con 0x7faf78009f00 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:51.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c09e6c0 con 0x7faf78004ec0 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf7800cb00 con 0x7faf78004220 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7faf7800cc70 con 0x7faf78009f00 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf89d9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf910e8640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 1 --2- 172.21.15.204:0/3617221362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 0x7faf78004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8a59c640 1 -- 172.21.15.204:0/3617221362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf78004ec0 msgr2=0x7faf780099c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8a59c640 1 --2- 172.21.15.204:0/3617221362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf78004ec0 0x7faf780099c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 10 monclient(hunting): get_auth_request con 0x7faf78004220 auth_method 0 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:51.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8b59e640 1 --2- 172.21.15.204:0/3617221362 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7faf78009f00 0x7faf7800c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8b59e640 10 monclient(hunting): get_auth_request con 0x7faf78009f00 auth_method 0 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8b59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 10 monclient(hunting): handle_auth_done global_id 7336 payload 293 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 1 -- 172.21.15.204:0/3617221362 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7faf78009f00 msgr2=0x7faf7800c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 1 --2- 172.21.15.204:0/3617221362 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7faf78009f00 0x7faf7800c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:51.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 1 -- 172.21.15.204:0/3617221362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf78004ec0 msgr2=0x7faf780099c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 1 --2- 172.21.15.204:0/3617221362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf78004ec0 0x7faf780099c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 1 monclient: found mon.h 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 1 -- 172.21.15.204:0/3617221362 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf780034e0 con 0x7faf78004220 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 10 monclient: _finish_auth 0 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8ad9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.719+0000 7faf8b59e640 1 -- 172.21.15.204:0/3617221362 reap_dead start 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf8ad9d640 1 --2- 172.21.15.204:0/3617221362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 0x7faf78004780 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faf740099c0 tx=0x7faf74009060 comp rx=0 tx=0).ready entity=mon.5 client_cookie=56ce9b48d1309dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf74003df0 con 0x7faf78004220 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf89d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf89d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:51.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf89d9b640 10 monclient: dump: 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:51.739 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf910e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7402a930 con 0x7faf78004220 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf89d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf910e8640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf89d9b640 1 -- 172.21.15.204:0/3617221362 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf7402aaa0 con 0x7faf78004220 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf89d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf910e8640 4 set_mon_vals no callback set 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf89d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf89d9b640 10 monclient: dump: 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:51.740 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:51.741 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf910e8640 10 monclient: shutdown 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf910e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf910e8640 1 -- 172.21.15.204:0/3617221362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 msgr2=0x7faf78004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf910e8640 1 --2- 172.21.15.204:0/3617221362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 0x7faf78004780 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faf740099c0 tx=0x7faf74009060 comp rx=0 tx=0).stop 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf910e8640 1 -- 172.21.15.204:0/3617221362 shutdown_connections 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf910e8640 1 --2- 172.21.15.204:0/3617221362 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 0x7faf78004780 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf910e8640 1 -- 172.21.15.204:0/3617221362 >> 172.21.15.204:0/3617221362 conn(0x7faf8c06e860 msgr2=0x7faf8c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf910e8640 1 -- 172.21.15.204:0/3617221362 shutdown_connections 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.720+0000 7faf910e8640 1 -- 172.21.15.204:0/3617221362 wait complete. 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.721+0000 7faf910e8640 10 monclient: build_initial_monmap 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.721+0000 7faf910e8640 10 monclient: monmap: 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:51.742 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:51.743 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.721+0000 7faf910e8640 1 Processor -- start 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.721+0000 7faf910e8640 1 -- start start 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.721+0000 7faf910e8640 10 monclient: init 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 10 monclient: _add_conns ranks=[5,8,0,2,6,7,1,3,4] 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 0x7faf8c11de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 10 monclient(hunting): picked mon.noname-h con 0x7faf78004220 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf8c11e390 0x7faf8c11e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 10 monclient(hunting): picked mon.noname-e con 0x7faf8c11e390 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf8c11eca0 0x7faf8c147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 10 monclient(hunting): picked mon.noname-a con 0x7faf8c11eca0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:51.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c11f9d0 con 0x7faf8c11eca0 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c09ca40 con 0x7faf78004220 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c09d650 con 0x7faf8c11e390 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf910e8640 10 monclient(hunting): authenticate will time out at 1087.365328s 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 0x7faf8c11de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8b59e640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf8c11eca0 msgr2=0x7faf8c147b80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8a59c640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf8c11e390 msgr2=0x7faf8c11e760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8b59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf8c11eca0 0x7faf8c147b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8a59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf8c11e390 0x7faf8c11e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 0x7faf8c11de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:60962/0 (socket says 172.21.15.204:60962) 2024-09-15T01:37:51.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 1 -- 172.21.15.204:0/2900578446 learned_addr learned my addr 172.21.15.204:0/2900578446 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 10 monclient(hunting): get_auth_request con 0x7faf78004220 auth_method 0 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 10 monclient(hunting): handle_auth_done global_id 6467 payload 824 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 1 -- 172.21.15.204:0/2900578446 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf8c11e390 msgr2=0x7faf8c11e760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 1 --2- 172.21.15.204:0/2900578446 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf8c11e390 0x7faf8c11e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 1 -- 172.21.15.204:0/2900578446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf8c11eca0 msgr2=0x7faf8c147b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 1 --2- 172.21.15.204:0/2900578446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf8c11eca0 0x7faf8c147b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:51.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 1 monclient: found mon.noname-h 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 1 -- 172.21.15.204:0/2900578446 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c120690 con 0x7faf78004220 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf8ad9d640 1 --2- 172.21.15.204:0/2900578446 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 0x7faf8c11de50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faf8c120880 tx=0x7faf74030000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.722+0000 7faf82ffd640 1 -- 172.21.15.204:0/2900578446 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf74005780 con 0x7faf78004220 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf82ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf82ffd640 10 monclient: dump: 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:51.747 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf82ffd640 10 monclient: _finish_auth 0 2024-09-15T01:37:51.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf82ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf82ffd640 1 -- 172.21.15.204:0/2900578446 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf74033470 con 0x7faf78004220 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf910e8640 5 monclient: authenticate success, global_id 6467 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf910e8640 10 monclient: _renew_subs 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf910e8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf910e8640 1 -- 172.21.15.204:0/2900578446 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8c121470 con 0x7faf78004220 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf82ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf82ffd640 1 -- 172.21.15.204:0/2900578446 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf74033a60 con 0x7faf78004220 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf82ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf82ffd640 10 monclient: dump: 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:51.749 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:51.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf910e8640 10 monclient: _renew_subs 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf910e8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf910e8640 1 -- 172.21.15.204:0/2900578446 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8c11f0b0 con 0x7faf78004220 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf8959a640 4 set_mon_vals no callback set 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf82ffd640 1 -- 172.21.15.204:0/2900578446 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7faf74030070 con 0x7faf78004220 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.723+0000 7faf910e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.724+0000 7faf910e8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.724+0000 7faf910e8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.724+0000 7faf910e8640 1 -- 172.21.15.204:0/2900578446 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8c070cc0 con 0x7faf78004220 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.724+0000 7faf82ffd640 1 --2- 172.21.15.204:0/2900578446 >> v2:172.21.15.204:6812/28210 conn(0x7faf6c0397b0 0x7faf6c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.724+0000 7faf82ffd640 1 -- 172.21.15.204:0/2900578446 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 10745+0+0 (secure 0 0 0) 0x7faf7403a070 con 0x7faf78004220 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.724+0000 7faf8a59c640 1 --2- 172.21.15.204:0/2900578446 >> v2:172.21.15.204:6812/28210 conn(0x7faf6c0397b0 0x7faf6c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.724+0000 7faf8a59c640 10 monclient: get_auth_request con 0x7faf6c0397b0 auth_method 0 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.724+0000 7faf8a59c640 1 --2- 172.21.15.204:0/2900578446 >> v2:172.21.15.204:6812/28210 conn(0x7faf6c0397b0 0x7faf6c03bc60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faf7c003870 tx=0x7faf7c006c60 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:51.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.728+0000 7faf82ffd640 1 -- 172.21.15.204:0/2900578446 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7faf740335e0 con 0x7faf78004220 2024-09-15T01:37:51.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.728+0000 7faf82ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:51.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.728+0000 7faf82ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:51.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.889+0000 7faf910e8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h17"}] 2024-09-15T01:37:51.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.889+0000 7faf910e8640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h17"}] 2024-09-15T01:37:51.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.889+0000 7faf910e8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:51.892 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:51.889+0000 7faf910e8640 1 -- 172.21.15.204:0/2900578446 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h17"} v 0) v1 -- 0x7faf8c121c00 con 0x7faf78004220 2024-09-15T01:37:52.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.547+0000 7faf82ffd640 1 -- 172.21.15.204:0/2900578446 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h17"}]=0 removed item id -67 name 'h17' from crush map v235) v1 ==== 120+0+0 (secure 0 0 0) 0x7faf74043070 con 0x7faf78004220 2024-09-15T01:37:52.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.547+0000 7faf82ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h17"}] 2024-09-15T01:37:52.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.547+0000 7faf82ffd640 10 monclient: _finish_command 2 = system:0 removed item id -67 name 'h17' from crush map 2024-09-15T01:37:52.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.645+0000 7faf910e8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h17"}] 2024-09-15T01:37:52.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.645+0000 7faf910e8640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h17"}] 2024-09-15T01:37:52.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.646+0000 7faf910e8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:52.648 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.646+0000 7faf910e8640 1 -- 172.21.15.204:0/2900578446 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h17"} v 0) v1 -- 0x7faf8c0a1f10 con 0x7faf78004220 2024-09-15T01:37:52.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.646+0000 7faf82ffd640 1 -- 172.21.15.204:0/2900578446 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h17"}]=0 device 'h17' does not appear in the crush map v235) v1 ==== 120+0+0 (secure 0 0 0) 0x7faf740323d0 con 0x7faf78004220 2024-09-15T01:37:52.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.646+0000 7faf82ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h17"}] 2024-09-15T01:37:52.649 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.646+0000 7faf82ffd640 10 monclient: _finish_command 3 = system:0 device 'h17' does not appear in the crush map 2024-09-15T01:37:52.649 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h17' does not appear in the crush map 2024-09-15T01:37:52.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 1 -- 172.21.15.204:0/2900578446 >> v2:172.21.15.204:6812/28210 conn(0x7faf6c0397b0 msgr2=0x7faf6c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:52.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 1 --2- 172.21.15.204:0/2900578446 >> v2:172.21.15.204:6812/28210 conn(0x7faf6c0397b0 0x7faf6c03bc60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faf7c003870 tx=0x7faf7c006c60 comp rx=0 tx=0).stop 2024-09-15T01:37:52.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 10 monclient: shutdown 2024-09-15T01:37:52.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:52.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 1 -- 172.21.15.204:0/2900578446 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 msgr2=0x7faf8c11de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:52.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 1 --2- 172.21.15.204:0/2900578446 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 0x7faf8c11de50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faf8c120880 tx=0x7faf74030000 comp rx=0 tx=0).stop 2024-09-15T01:37:52.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 1 -- 172.21.15.204:0/2900578446 shutdown_connections 2024-09-15T01:37:52.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 1 --2- 172.21.15.204:0/2900578446 >> v2:172.21.15.204:6812/28210 conn(0x7faf6c0397b0 0x7faf6c03bc60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:52.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 1 --2- 172.21.15.204:0/2900578446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf8c11eca0 0x7faf8c147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:52.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 1 --2- 172.21.15.204:0/2900578446 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7faf8c11e390 0x7faf8c11e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:52.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 1 --2- 172.21.15.204:0/2900578446 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7faf78004220 0x7faf8c11de50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:52.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 1 -- 172.21.15.204:0/2900578446 >> 172.21.15.204:0/2900578446 conn(0x7faf8c06e860 msgr2=0x7faf8c09ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:52.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 1 -- 172.21.15.204:0/2900578446 shutdown_connections 2024-09-15T01:37:52.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.648+0000 7faf910e8640 1 -- 172.21.15.204:0/2900578446 wait complete. 2024-09-15T01:37:52.659 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:52.659 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h18 2024-09-15T01:37:52.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.731+0000 7fcd3979d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:52.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.731+0000 7fcd3979d640 10 monclient: build_initial_monmap 2024-09-15T01:37:52.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.731+0000 7fcd3979d640 10 monclient: monmap: 2024-09-15T01:37:52.734 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:52.734 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:52.734 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:52.732801+0000 2024-09-15T01:37:52.734 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:52.732801+0000 2024-09-15T01:37:52.734 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:52.734 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.731+0000 7fcd3979d640 1 Processor -- start 2024-09-15T01:37:52.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.731+0000 7fcd3979d640 1 -- start start 2024-09-15T01:37:52.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.731+0000 7fcd3979d640 10 monclient: init 2024-09-15T01:37:52.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:52.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 10 monclient: _add_conns ranks=[4,9,5,2,1,6,8,7,0,3,10] 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcd340a2e30 0x7fcd34132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 10 monclient(hunting): picked mon.noname-c con 0x7fcd340a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcd3409fb80 0x7fcd3409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 10 monclient(hunting): picked mon.noname-a con 0x7fcd3409fb80 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 0x7fcd340a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 10 monclient(hunting): picked mon.noname-h con 0x7fcd340a0a40 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3409e6c0 con 0x7fcd3409fb80 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd340a04b0 con 0x7fcd340a2e30 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd340a3200 con 0x7fcd340a0a40 2024-09-15T01:37:52.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:52.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd3979d640 10 monclient(hunting): authenticate will time out at 1088.375427s 2024-09-15T01:37:52.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 0x7fcd340a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:52.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd2a7fc640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7fcd3409fb80 msgr2=0x7fcd3409ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:52.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 0x7fcd340a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:60964/0 (socket says 172.21.15.204:60964) 2024-09-15T01:37:52.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd32ffd640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcd340a2e30 msgr2=0x7fcd34132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:52.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 1 -- 172.21.15.204:0/4243282237 learned_addr learned my addr 172.21.15.204:0/4243282237 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:52.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 10 monclient(hunting): get_auth_request con 0x7fcd340a0a40 auth_method 0 2024-09-15T01:37:52.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:52.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:52.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:52.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd2a7fc640 1 --2- 172.21.15.204:0/4243282237 >> v2:172.21.15.178:3300/0 conn(0x7fcd3409fb80 0x7fcd3409ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:52.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd32ffd640 1 --2- 172.21.15.204:0/4243282237 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcd340a2e30 0x7fcd34132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 10 monclient(hunting): handle_auth_done global_id 6476 payload 274 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 1 -- 172.21.15.204:0/4243282237 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcd340a2e30 msgr2=0x7fcd34132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 1 --2- 172.21.15.204:0/4243282237 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcd340a2e30 0x7fcd34132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 1 -- 172.21.15.204:0/4243282237 >> v2:172.21.15.178:3300/0 conn(0x7fcd3409fb80 msgr2=0x7fcd3409ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 1 --2- 172.21.15.204:0/4243282237 >> v2:172.21.15.178:3300/0 conn(0x7fcd3409fb80 0x7fcd3409ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 1 monclient: found mon.noname-h 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.732+0000 7fcd337fe640 1 -- 172.21.15.204:0/4243282237 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3404b720 con 0x7fcd340a0a40 2024-09-15T01:37:52.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd337fe640 1 --2- 172.21.15.204:0/4243282237 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 0x7fcd340a0ee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009f20 tx=0x7fcd2c009840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=8dab1c2f82e4d117 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd327fc640 1 -- 172.21.15.204:0/4243282237 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd2c007880 con 0x7fcd340a0a40 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd327fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd327fc640 10 monclient: dump: 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:52.747 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:52.748 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:52.748 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:52.748 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:52.748 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:52.748 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:52.748 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:52.748 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:52.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd327fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:52.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:52.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd327fc640 1 -- 172.21.15.204:0/4243282237 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c02d050 con 0x7fcd340a0a40 2024-09-15T01:37:52.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd3979d640 5 monclient: authenticate success, global_id 6476 2024-09-15T01:37:52.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd3979d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd327fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd327fc640 1 -- 172.21.15.204:0/4243282237 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd2c030360 con 0x7fcd340a0a40 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd327fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd327fc640 10 monclient: dump: 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:52.749 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd3979d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.733+0000 7fcd3979d640 4 set_mon_vals no callback set 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 10 monclient: shutdown 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 1 -- 172.21.15.204:0/4243282237 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 msgr2=0x7fcd340a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:52.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 1 --2- 172.21.15.204:0/4243282237 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 0x7fcd340a0ee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009f20 tx=0x7fcd2c009840 comp rx=0 tx=0).stop 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 1 -- 172.21.15.204:0/4243282237 shutdown_connections 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 1 --2- 172.21.15.204:0/4243282237 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 0x7fcd340a0ee0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 1 --2- 172.21.15.204:0/4243282237 >> v2:172.21.15.178:3300/0 conn(0x7fcd3409fb80 0x7fcd3409ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 1 --2- 172.21.15.204:0/4243282237 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcd340a2e30 0x7fcd34132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 1 -- 172.21.15.204:0/4243282237 >> 172.21.15.204:0/4243282237 conn(0x7fcd3406e860 msgr2=0x7fcd3406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 1 -- 172.21.15.204:0/4243282237 shutdown_connections 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 1 -- 172.21.15.204:0/4243282237 wait complete. 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 10 monclient: build_initial_monmap 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.734+0000 7fcd3979d640 10 monclient: monmap: 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:52.751 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:52.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 1 Processor -- start 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 1 -- start start 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 10 monclient: init 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 10 monclient: _add_conns ranks=[8,5,0,7,4,3,2,1,6] 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcd3409fb80 0x7fcd341318a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 10 monclient(hunting): picked mon.noname-e con 0x7fcd3409fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 0x7fcd3411e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 10 monclient(hunting): picked mon.noname-h con 0x7fcd340a0a40 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd340a2e30 0x7fcd3411e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 10 monclient(hunting): picked mon.noname-a con 0x7fcd340a2e30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd341206f0 con 0x7fcd340a2e30 2024-09-15T01:37:52.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd34142960 con 0x7fcd340a0a40 2024-09-15T01:37:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3409ca40 con 0x7fcd3409fb80 2024-09-15T01:37:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd3979d640 10 monclient(hunting): authenticate will time out at 1088.378818s 2024-09-15T01:37:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd2a7fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 0x7fcd3411e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd337fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd340a2e30 msgr2=0x7fcd3411e890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd32ffd640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcd3409fb80 msgr2=0x7fcd341318a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd337fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd340a2e30 0x7fcd3411e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd32ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcd3409fb80 0x7fcd341318a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd2a7fc640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 0x7fcd3411e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:60972/0 (socket says 172.21.15.204:60972) 2024-09-15T01:37:52.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd2a7fc640 1 -- 172.21.15.204:0/1638876181 learned_addr learned my addr 172.21.15.204:0/1638876181 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd2a7fc640 10 monclient(hunting): get_auth_request con 0x7fcd340a0a40 auth_method 0 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd2a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd2a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.735+0000 7fcd2a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 10 monclient(hunting): handle_auth_done global_id 6485 payload 824 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 1 -- 172.21.15.204:0/1638876181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcd3409fb80 msgr2=0x7fcd341318a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 1 --2- 172.21.15.204:0/1638876181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcd3409fb80 0x7fcd341318a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 1 -- 172.21.15.204:0/1638876181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd340a2e30 msgr2=0x7fcd3411e890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 1 --2- 172.21.15.204:0/1638876181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd340a2e30 0x7fcd3411e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:52.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 1 monclient: found mon.noname-h 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 1 -- 172.21.15.204:0/1638876181 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c009ae0 con 0x7fcd340a0a40 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2a7fc640 1 --2- 172.21.15.204:0/1638876181 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 0x7fcd3411e350 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00b5e0 tx=0x7fcd1c006f90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2b7fe640 1 -- 172.21.15.204:0/1638876181 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd1c006150 con 0x7fcd340a0a40 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2b7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2b7fe640 10 monclient: dump: 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:52.756 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:52.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd3979d640 5 monclient: authenticate success, global_id 6485 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2b7fe640 1 -- 172.21.15.204:0/1638876181 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c0062f0 con 0x7fcd340a0a40 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2b7fe640 1 -- 172.21.15.204:0/1638876181 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd1c0103e0 con 0x7fcd340a0a40 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd3979d640 10 monclient: _renew_subs 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd3979d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd3979d640 1 -- 172.21.15.204:0/1638876181 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3409d650 con 0x7fcd340a0a40 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd31ffb640 4 set_mon_vals no callback set 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd2b7fe640 10 monclient: dump: 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:52.758 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:52.759 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd3979d640 10 monclient: _renew_subs 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd3979d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.736+0000 7fcd3979d640 1 -- 172.21.15.204:0/1638876181 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd341219f0 con 0x7fcd340a0a40 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.737+0000 7fcd2b7fe640 1 -- 172.21.15.204:0/1638876181 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fcd1c003b70 con 0x7fcd340a0a40 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.737+0000 7fcd3979d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.737+0000 7fcd3979d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.737+0000 7fcd3979d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.737+0000 7fcd3979d640 1 -- 172.21.15.204:0/1638876181 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd34070cc0 con 0x7fcd340a0a40 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.737+0000 7fcd2b7fe640 1 --2- 172.21.15.204:0/1638876181 >> v2:172.21.15.204:6812/28210 conn(0x7fcd140396e0 0x7fcd1403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.737+0000 7fcd2b7fe640 1 -- 172.21.15.204:0/1638876181 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 10532+0+0 (secure 0 0 0) 0x7fcd1c04bf30 con 0x7fcd340a0a40 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.737+0000 7fcd32ffd640 1 --2- 172.21.15.204:0/1638876181 >> v2:172.21.15.204:6812/28210 conn(0x7fcd140396e0 0x7fcd1403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.737+0000 7fcd32ffd640 10 monclient: get_auth_request con 0x7fcd140396e0 auth_method 0 2024-09-15T01:37:52.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.738+0000 7fcd32ffd640 1 --2- 172.21.15.204:0/1638876181 >> v2:172.21.15.204:6812/28210 conn(0x7fcd140396e0 0x7fcd1403bb90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcd24003870 tx=0x7fcd24006c60 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:52.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.742+0000 7fcd2b7fe640 1 -- 172.21.15.204:0/1638876181 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcd1c00daf0 con 0x7fcd340a0a40 2024-09-15T01:37:52.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.742+0000 7fcd2b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:52.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.742+0000 7fcd2b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:52.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.904+0000 7fcd3979d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h18"}] 2024-09-15T01:37:52.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.904+0000 7fcd3979d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h18"}] 2024-09-15T01:37:52.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.904+0000 7fcd3979d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:52.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:52.904+0000 7fcd3979d640 1 -- 172.21.15.204:0/1638876181 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h18"} v 0) v1 -- 0x7fcd340a1f10 con 0x7fcd340a0a40 2024-09-15T01:37:53.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.547+0000 7fcd2b7fe640 1 -- 172.21.15.204:0/1638876181 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h18"}]=0 removed item id -70 name 'h18' from crush map v236) v1 ==== 120+0+0 (secure 0 0 0) 0x7fcd1c00d310 con 0x7fcd340a0a40 2024-09-15T01:37:53.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.547+0000 7fcd2b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h18"}] 2024-09-15T01:37:53.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.547+0000 7fcd2b7fe640 10 monclient: _finish_command 2 = system:0 removed item id -70 name 'h18' from crush map 2024-09-15T01:37:53.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.649+0000 7fcd3979d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h18"}] 2024-09-15T01:37:53.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.649+0000 7fcd3979d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h18"}] 2024-09-15T01:37:53.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.649+0000 7fcd3979d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:53.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.649+0000 7fcd3979d640 1 -- 172.21.15.204:0/1638876181 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h18"} v 0) v1 -- 0x7fcd34121dd0 con 0x7fcd340a0a40 2024-09-15T01:37:53.652 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.650+0000 7fcd2b7fe640 1 -- 172.21.15.204:0/1638876181 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h18"}]=0 device 'h18' does not appear in the crush map v236) v1 ==== 120+0+0 (secure 0 0 0) 0x7fcd1c017030 con 0x7fcd340a0a40 2024-09-15T01:37:53.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.650+0000 7fcd2b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h18"}] 2024-09-15T01:37:53.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.650+0000 7fcd2b7fe640 10 monclient: _finish_command 3 = system:0 device 'h18' does not appear in the crush map 2024-09-15T01:37:53.653 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h18' does not appear in the crush map 2024-09-15T01:37:53.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 1 -- 172.21.15.204:0/1638876181 >> v2:172.21.15.204:6812/28210 conn(0x7fcd140396e0 msgr2=0x7fcd1403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 1 --2- 172.21.15.204:0/1638876181 >> v2:172.21.15.204:6812/28210 conn(0x7fcd140396e0 0x7fcd1403bb90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcd24003870 tx=0x7fcd24006c60 comp rx=0 tx=0).stop 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 10 monclient: shutdown 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 1 -- 172.21.15.204:0/1638876181 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 msgr2=0x7fcd3411e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 1 --2- 172.21.15.204:0/1638876181 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 0x7fcd3411e350 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00b5e0 tx=0x7fcd1c006f90 comp rx=0 tx=0).stop 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 1 -- 172.21.15.204:0/1638876181 shutdown_connections 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 1 --2- 172.21.15.204:0/1638876181 >> v2:172.21.15.204:6812/28210 conn(0x7fcd140396e0 0x7fcd1403bb90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 1 --2- 172.21.15.204:0/1638876181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd340a2e30 0x7fcd3411e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 1 --2- 172.21.15.204:0/1638876181 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcd340a0a40 0x7fcd3411e350 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 1 --2- 172.21.15.204:0/1638876181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcd3409fb80 0x7fcd341318a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 1 -- 172.21.15.204:0/1638876181 >> 172.21.15.204:0/1638876181 conn(0x7fcd3406e860 msgr2=0x7fcd3409ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 1 -- 172.21.15.204:0/1638876181 shutdown_connections 2024-09-15T01:37:53.655 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.652+0000 7fcd3979d640 1 -- 172.21.15.204:0/1638876181 wait complete. 2024-09-15T01:37:53.663 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:53.663 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h19 2024-09-15T01:37:53.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.733+0000 7f32f9860640 10 monclient: get_monmap_and_config 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.733+0000 7f32f9860640 10 monclient: build_initial_monmap 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.734+0000 7f32f9860640 10 monclient: monmap: 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:53.735454+0000 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:53.735454+0000 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:53.744 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.734+0000 7f32f9860640 1 Processor -- start 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.734+0000 7f32f9860640 1 -- start start 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.734+0000 7f32f9860640 10 monclient: init 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.734+0000 7f32f9860640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.734+0000 7f32f9860640 10 monclient: _add_conns ranks=[5,0,9,7,10,4,3,2,8,6,1] 2024-09-15T01:37:53.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.734+0000 7f32f9860640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f40a1a50 0x7f32f409d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f9860640 10 monclient(hunting): picked mon.noname-h con 0x7f32f40a1a50 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f9860640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f32f4069ee0 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f9860640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f32f413c430 0x7f32f413e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f9860640 10 monclient(hunting): picked mon.noname-a con 0x7f32f413c430 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f9860640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f404b720 con 0x7f32f4069ee0 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f9860640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f9860640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f409fc10 con 0x7f32f413c430 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f9860640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f9860640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32f40a0240 con 0x7f32f40a1a50 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f9860640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f9860640 10 monclient(hunting): authenticate will time out at 1089.378055s 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f40a1a50 0x7f32f409d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:53.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f27fc640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f32f4069ee0 legacy=0x7f32f409da70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f37fe640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7f32f413c430 msgr2=0x7f32f413e810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f37fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f32f413c430 0x7f32f413e810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f40a1a50 0x7f32f409d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:60976/0 (socket says 172.21.15.204:60976) 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 1 -- 172.21.15.204:0/254609816 learned_addr learned my addr 172.21.15.204:0/254609816 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 10 monclient(hunting): get_auth_request con 0x7f32f40a1a50 auth_method 0 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 10 monclient(hunting): handle_auth_done global_id 6494 payload 274 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 1 -- 172.21.15.204:0/254609816 >> v2:172.21.15.178:3300/0 conn(0x7f32f413c430 msgr2=0x7f32f413e810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:53.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 1 --2- 172.21.15.204:0/254609816 >> v2:172.21.15.178:3300/0 conn(0x7f32f413c430 0x7f32f413e810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 1 -- 172.21.15.204:0/254609816 >> v1:172.21.15.178:6789/0 conn(0x7f32f4069ee0 legacy=0x7f32f409da70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 1 monclient: found mon.noname-h 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.735+0000 7f32f2ffd640 1 -- 172.21.15.204:0/254609816 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f409e830 con 0x7f32f40a1a50 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f2ffd640 1 --2- 172.21.15.204:0/254609816 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f40a1a50 0x7f32f409d530 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009910 tx=0x7f32e40091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=5ec9f4be3e9fa265 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f1ffb640 1 -- 172.21.15.204:0/254609816 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32e402acc0 con 0x7f32f40a1a50 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f1ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f1ffb640 10 monclient: dump: 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:53.748 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:53.749 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f1ffb640 1 -- 172.21.15.204:0/254609816 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e402ae40 con 0x7f32f40a1a50 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f9860640 5 monclient: authenticate success, global_id 6494 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f9860640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f9860640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f1ffb640 1 -- 172.21.15.204:0/254609816 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32e4031ca0 con 0x7f32f40a1a50 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f9860640 4 set_mon_vals no callback set 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f1ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f1ffb640 10 monclient: dump: 2024-09-15T01:37:53.750 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:53.751 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:53.752 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:53.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f9860640 10 monclient: shutdown 2024-09-15T01:37:53.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f9860640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:53.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f9860640 1 -- 172.21.15.204:0/254609816 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f40a1a50 msgr2=0x7f32f409d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:53.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f9860640 1 --2- 172.21.15.204:0/254609816 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f40a1a50 0x7f32f409d530 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009910 tx=0x7f32e40091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:53.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f9860640 1 -- 172.21.15.204:0/254609816 shutdown_connections 2024-09-15T01:37:53.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f9860640 1 --2- 172.21.15.204:0/254609816 >> v2:172.21.15.178:3300/0 conn(0x7f32f413c430 0x7f32f413e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:53.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f9860640 1 --2- 172.21.15.204:0/254609816 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f40a1a50 0x7f32f409d530 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:53.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.736+0000 7f32f9860640 1 -- 172.21.15.204:0/254609816 >> 172.21.15.204:0/254609816 conn(0x7f32f40a6cc0 msgr2=0x7f32f40a2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:53.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.737+0000 7f32f9860640 1 -- 172.21.15.204:0/254609816 shutdown_connections 2024-09-15T01:37:53.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.737+0000 7f32f9860640 1 -- 172.21.15.204:0/254609816 wait complete. 2024-09-15T01:37:53.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.737+0000 7f32f9860640 10 monclient: build_initial_monmap 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.737+0000 7f32f9860640 10 monclient: monmap: 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:53.753 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.737+0000 7f32f9860640 1 Processor -- start 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 1 -- start start 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 10 monclient: init 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 10 monclient: _add_conns ranks=[5,6,0,1,7,4,8,2,3] 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f4069ee0 0x7f32f413be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 10 monclient(hunting): picked mon.noname-h con 0x7f32f4069ee0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f32f40a1a50 0x7f32f4124ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 10 monclient(hunting): picked mon.noname-d con 0x7f32f40a1a50 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:53.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f413c430 0x7f32f4125400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 10 monclient(hunting): picked mon.noname-a con 0x7f32f413c430 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f409cc70 con 0x7f32f413c430 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32f409c660 con 0x7f32f4069ee0 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4127a20 con 0x7f32f40a1a50 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f9860640 10 monclient(hunting): authenticate will time out at 1089.381438s 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f4069ee0 0x7f32f413be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f4069ee0 0x7f32f413be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:60988/0 (socket says 172.21.15.204:60988) 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 1 -- 172.21.15.204:0/1066536821 learned_addr learned my addr 172.21.15.204:0/1066536821 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 10 monclient(hunting): get_auth_request con 0x7f32f4069ee0 auth_method 0 2024-09-15T01:37:53.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f37fe640 1 -- 172.21.15.204:0/1066536821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f413c430 msgr2=0x7f32f4125400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f37fe640 1 --2- 172.21.15.204:0/1066536821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f413c430 0x7f32f4125400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 10 monclient(hunting): handle_auth_done global_id 6503 payload 824 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 1 -- 172.21.15.204:0/1066536821 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f32f40a1a50 msgr2=0x7f32f4124ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 1 --2- 172.21.15.204:0/1066536821 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f32f40a1a50 0x7f32f4124ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 1 -- 172.21.15.204:0/1066536821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f413c430 msgr2=0x7f32f4125400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 1 --2- 172.21.15.204:0/1066536821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f413c430 0x7f32f4125400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:53.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 1 monclient: found mon.noname-h 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 1 -- 172.21.15.204:0/1066536821 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e4009450 con 0x7f32f4069ee0 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32f2ffd640 1 --2- 172.21.15.204:0/1066536821 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f4069ee0 0x7f32f413be40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f32e4035040 tx=0x7f32e4032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.738+0000 7f32daffd640 1 -- 172.21.15.204:0/1066536821 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32e402acc0 con 0x7f32f4069ee0 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32daffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32daffd640 10 monclient: dump: 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:53.757 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32daffd640 10 monclient: _finish_auth 0 2024-09-15T01:37:53.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32f9860640 5 monclient: authenticate success, global_id 6503 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32daffd640 1 -- 172.21.15.204:0/1066536821 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e4032e40 con 0x7f32f4069ee0 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32daffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32daffd640 1 -- 172.21.15.204:0/1066536821 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32e403a810 con 0x7f32f4069ee0 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32f9860640 10 monclient: _renew_subs 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32f9860640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32f9860640 1 -- 172.21.15.204:0/1066536821 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f409f000 con 0x7f32f4069ee0 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32daffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32f17fa640 4 set_mon_vals no callback set 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32daffd640 10 monclient: dump: 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:53.759 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:53.760 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32f9860640 10 monclient: _renew_subs 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32f9860640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32f9860640 1 -- 172.21.15.204:0/1066536821 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f409bea0 con 0x7f32f4069ee0 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32daffd640 1 -- 172.21.15.204:0/1066536821 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f32e4039040 con 0x7f32f4069ee0 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32f9860640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32f9860640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32f9860640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.739+0000 7f32f9860640 1 -- 172.21.15.204:0/1066536821 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f4070cc0 con 0x7f32f4069ee0 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.740+0000 7f32daffd640 1 --2- 172.21.15.204:0/1066536821 >> v2:172.21.15.204:6812/28210 conn(0x7f32e00397b0 0x7f32e003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.740+0000 7f32daffd640 1 -- 172.21.15.204:0/1066536821 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 10319+0+0 (secure 0 0 0) 0x7f32e406c4b0 con 0x7f32f4069ee0 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.740+0000 7f32f27fc640 1 --2- 172.21.15.204:0/1066536821 >> v2:172.21.15.204:6812/28210 conn(0x7f32e00397b0 0x7f32e003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:53.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.740+0000 7f32f27fc640 10 monclient: get_auth_request con 0x7f32e00397b0 auth_method 0 2024-09-15T01:37:53.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.740+0000 7f32f27fc640 1 --2- 172.21.15.204:0/1066536821 >> v2:172.21.15.204:6812/28210 conn(0x7f32e00397b0 0x7f32e003bc60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f32e80038f0 tx=0x7f32e8006ca0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:53.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.744+0000 7f32daffd640 1 -- 172.21.15.204:0/1066536821 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f32e402f660 con 0x7f32f4069ee0 2024-09-15T01:37:53.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.744+0000 7f32daffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:53.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.744+0000 7f32daffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:53.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.904+0000 7f32f9860640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h19"}] 2024-09-15T01:37:53.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.904+0000 7f32f9860640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h19"}] 2024-09-15T01:37:53.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.904+0000 7f32f9860640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:53.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:53.904+0000 7f32f9860640 1 -- 172.21.15.204:0/1066536821 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h19"} v 0) v1 -- 0x7f32f4067590 con 0x7f32f4069ee0 2024-09-15T01:37:54.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.560+0000 7f32daffd640 1 -- 172.21.15.204:0/1066536821 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h19"}]=0 removed item id -73 name 'h19' from crush map v237) v1 ==== 120+0+0 (secure 0 0 0) 0x7f32e4037020 con 0x7f32f4069ee0 2024-09-15T01:37:54.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.561+0000 7f32daffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h19"}] 2024-09-15T01:37:54.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.561+0000 7f32daffd640 10 monclient: _finish_command 2 = system:0 removed item id -73 name 'h19' from crush map 2024-09-15T01:37:54.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.659+0000 7f32f9860640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h19"}] 2024-09-15T01:37:54.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.659+0000 7f32f9860640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h19"}] 2024-09-15T01:37:54.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.659+0000 7f32f9860640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:54.662 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.659+0000 7f32f9860640 1 -- 172.21.15.204:0/1066536821 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h19"} v 0) v1 -- 0x7f32f4067590 con 0x7f32f4069ee0 2024-09-15T01:37:54.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.660+0000 7f32daffd640 1 -- 172.21.15.204:0/1066536821 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h19"}]=0 device 'h19' does not appear in the crush map v237) v1 ==== 120+0+0 (secure 0 0 0) 0x7f32e4041070 con 0x7f32f4069ee0 2024-09-15T01:37:54.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.660+0000 7f32daffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h19"}] 2024-09-15T01:37:54.663 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.660+0000 7f32daffd640 10 monclient: _finish_command 3 = system:0 device 'h19' does not appear in the crush map 2024-09-15T01:37:54.663 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h19' does not appear in the crush map 2024-09-15T01:37:54.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 1 -- 172.21.15.204:0/1066536821 >> v2:172.21.15.204:6812/28210 conn(0x7f32e00397b0 msgr2=0x7f32e003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:54.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 1 --2- 172.21.15.204:0/1066536821 >> v2:172.21.15.204:6812/28210 conn(0x7f32e00397b0 0x7f32e003bc60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f32e80038f0 tx=0x7f32e8006ca0 comp rx=0 tx=0).stop 2024-09-15T01:37:54.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 10 monclient: shutdown 2024-09-15T01:37:54.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:54.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 1 -- 172.21.15.204:0/1066536821 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f4069ee0 msgr2=0x7f32f413be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:54.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 1 --2- 172.21.15.204:0/1066536821 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f4069ee0 0x7f32f413be40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f32e4035040 tx=0x7f32e4032000 comp rx=0 tx=0).stop 2024-09-15T01:37:54.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 1 -- 172.21.15.204:0/1066536821 shutdown_connections 2024-09-15T01:37:54.665 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 1 --2- 172.21.15.204:0/1066536821 >> v2:172.21.15.204:6812/28210 conn(0x7f32e00397b0 0x7f32e003bc60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:54.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 1 --2- 172.21.15.204:0/1066536821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f413c430 0x7f32f4125400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:54.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 1 --2- 172.21.15.204:0/1066536821 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f32f40a1a50 0x7f32f4124ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:54.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 1 --2- 172.21.15.204:0/1066536821 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f32f4069ee0 0x7f32f413be40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:54.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 1 -- 172.21.15.204:0/1066536821 >> 172.21.15.204:0/1066536821 conn(0x7f32f40a6cc0 msgr2=0x7f32f406b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:54.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 1 -- 172.21.15.204:0/1066536821 shutdown_connections 2024-09-15T01:37:54.666 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.662+0000 7f32f9860640 1 -- 172.21.15.204:0/1066536821 wait complete. 2024-09-15T01:37:54.673 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:54.674 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h20 2024-09-15T01:37:54.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.744+0000 7f89cfaa8640 10 monclient: get_monmap_and_config 2024-09-15T01:37:54.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.744+0000 7f89cfaa8640 10 monclient: build_initial_monmap 2024-09-15T01:37:54.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.744+0000 7f89cfaa8640 10 monclient: monmap: 2024-09-15T01:37:54.747 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:54.747 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:54.747 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:54.745762+0000 2024-09-15T01:37:54.747 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:54.745762+0000 2024-09-15T01:37:54.747 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:54.747 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:54.747 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:54.747 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:54.747 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:54.747 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:54.748 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:54.748 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:54.748 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:54.748 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:54.748 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:54.748 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:54.748 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:54.748 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:54.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.744+0000 7f89cfaa8640 1 Processor -- start 2024-09-15T01:37:54.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 1 -- start start 2024-09-15T01:37:54.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 10 monclient: init 2024-09-15T01:37:54.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:54.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 10 monclient: _add_conns ranks=[1,9,6,7,10,5,8,3,2,0,4] 2024-09-15T01:37:54.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f89c80a0b40 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:54.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f89c809acf0 0x7f89c8067840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:54.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 10 monclient(hunting): picked mon.noname-a con 0x7f89c809acf0 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:54.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89c8138550 0x7f89c813a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:54.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 10 monclient(hunting): picked mon.noname-d con 0x7f89c8138550 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:54.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c804b720 con 0x7f89c80a0b40 2024-09-15T01:37:54.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:54.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c8099d50 con 0x7f89c809acf0 2024-09-15T01:37:54.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f89c809a8c0 con 0x7f89c8138550 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cfaa8640 10 monclient(hunting): authenticate will time out at 1090.388676s 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cd043640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7f89c809acf0 msgr2=0x7f89c8067840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cd043640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f89c809acf0 0x7f89c8067840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cd844640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f89c80a0b40 0x7f89c8067130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:48370/0 (socket says 172.21.15.204:48370) 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89cd844640 1 -- 172.21.15.204:0/3845398582 learned_addr learned my addr 172.21.15.204:0/3845398582 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89ce045640 1 --2- 172.21.15.204:0/3845398582 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89c8138550 0x7f89c813a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89ce045640 10 monclient(hunting): get_auth_request con 0x7f89c8138550 auth_method 0 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89ce045640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89ce045640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.745+0000 7f89ce045640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:54.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89cc842640 1 -- 172.21.15.204:0/3845398582 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967543398 0 0) 0x7f89c804b720 con 0x7f89c80a0b40 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89cc842640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89cc842640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89cc842640 10 monclient(hunting): my global_id is 7345 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89cc842640 1 -- 172.21.15.204:0/3845398582 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89b40034e0 con 0x7f89c80a0b40 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 10 monclient(hunting): handle_auth_done global_id 24549 payload 274 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 1 -- 172.21.15.204:0/3845398582 >> v2:172.21.15.178:3300/0 conn(0x7f89c809acf0 msgr2=0x7f89c8067840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 1 --2- 172.21.15.204:0/3845398582 >> v2:172.21.15.178:3300/0 conn(0x7f89c809acf0 0x7f89c8067840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89cc842640 1 -- 172.21.15.204:0/3845398582 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3553401552 0 0) 0x7f89b40034e0 con 0x7f89c80a0b40 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 1 -- 172.21.15.204:0/3845398582 >> v1:172.21.15.204:6789/0 conn(0x7f89c80a0b40 legacy=0x7f89c8067130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:54.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 10 monclient: _finish_hunting 0 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 1 monclient: found mon.noname-d 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 1 -- 172.21.15.204:0/3845398582 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c80a0f10 con 0x7f89c8138550 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89cc842640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.746+0000 7f89ce045640 1 --2- 172.21.15.204:0/3845398582 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89c8138550 0x7f89c813a930 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009e60 tx=0x7f89c0009780 comp rx=0 tx=0).ready entity=mon.6 client_cookie=bbb933f20920a7b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cc842640 1 -- 172.21.15.204:0/3845398582 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89c40018f0 con 0x7f89c8138550 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cc842640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cc842640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cc842640 10 monclient: dump: 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:54.761 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cc842640 10 monclient: _finish_auth 0 2024-09-15T01:37:54.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cc842640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cc842640 1 -- 172.21.15.204:0/3845398582 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c002d050 con 0x7f89c8138550 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 5 monclient: authenticate success, global_id 24549 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cc842640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cc842640 1 -- 172.21.15.204:0/3845398582 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89c0007e50 con 0x7f89c8138550 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 4 set_mon_vals no callback set 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cc842640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cc842640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cc842640 10 monclient: dump: 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:54.763 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:54.764 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 10 monclient: shutdown 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 1 -- 172.21.15.204:0/3845398582 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89c8138550 msgr2=0x7f89c813a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 1 --2- 172.21.15.204:0/3845398582 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89c8138550 0x7f89c813a930 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009e60 tx=0x7f89c0009780 comp rx=0 tx=0).stop 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 1 -- 172.21.15.204:0/3845398582 shutdown_connections 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 1 --2- 172.21.15.204:0/3845398582 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89c8138550 0x7f89c813a930 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 1 --2- 172.21.15.204:0/3845398582 >> v2:172.21.15.178:3300/0 conn(0x7f89c809acf0 0x7f89c8067840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 1 -- 172.21.15.204:0/3845398582 >> 172.21.15.204:0/3845398582 conn(0x7f89c80a2e30 msgr2=0x7f89c80a0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 1 -- 172.21.15.204:0/3845398582 shutdown_connections 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.747+0000 7f89cfaa8640 1 -- 172.21.15.204:0/3845398582 wait complete. 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.748+0000 7f89cfaa8640 10 monclient: build_initial_monmap 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.748+0000 7f89cfaa8640 10 monclient: monmap: 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:54.765 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:54.766 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.748+0000 7f89cfaa8640 1 Processor -- start 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.748+0000 7f89cfaa8640 1 -- start start 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.748+0000 7f89cfaa8640 10 monclient: init 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 10 monclient: _add_conns ranks=[8,4,3,5,1,0,7,2,6] 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f89c809acf0 0x7f89c806e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 10 monclient(hunting): picked mon.noname-e con 0x7f89c809acf0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f89c80a0b40 0x7f89c8094750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 10 monclient(hunting): picked mon.noname-c con 0x7f89c80a0b40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89c8138550 0x7f89c8141b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 10 monclient(hunting): picked mon.noname-g con 0x7f89c8138550 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:54.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c808ec20 con 0x7f89c8138550 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89c8099b40 con 0x7f89c80a0b40 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f89c8099870 con 0x7f89c809acf0 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 10 monclient(hunting): authenticate will time out at 1090.392198s 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cd844640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f89c809acf0 msgr2=0x7f89c806e800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cd844640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f89c809acf0 0x7f89c806e800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cd043640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f89c80a0b40 msgr2=0x7f89c8094750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89c8138550 0x7f89c8141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cd043640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f89c80a0b40 0x7f89c8094750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89c8138550 0x7f89c8141b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:34420/0 (socket says 172.21.15.204:34420) 2024-09-15T01:37:54.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 1 -- 172.21.15.204:0/400192324 learned_addr learned my addr 172.21.15.204:0/400192324 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 10 monclient(hunting): get_auth_request con 0x7f89c8138550 auth_method 0 2024-09-15T01:37:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:54.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:54.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 10 monclient(hunting): handle_auth_done global_id 15618 payload 824 2024-09-15T01:37:54.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 1 -- 172.21.15.204:0/400192324 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f89c809acf0 msgr2=0x7f89c806e800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:54.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 1 --2- 172.21.15.204:0/400192324 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f89c809acf0 0x7f89c806e800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:54.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 1 -- 172.21.15.204:0/400192324 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f89c80a0b40 msgr2=0x7f89c8094750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:54.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 1 --2- 172.21.15.204:0/400192324 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f89c80a0b40 0x7f89c8094750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:54.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 10 monclient: _finish_hunting 0 2024-09-15T01:37:54.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 1 monclient: found mon.noname-g 2024-09-15T01:37:54.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:54.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:54.770 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 1 -- 172.21.15.204:0/400192324 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c0009a20 con 0x7f89c8138550 2024-09-15T01:37:54.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89ce045640 1 --2- 172.21.15.204:0/400192324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89c8138550 0x7f89c8141b90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009d30 tx=0x7f89c0037000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:54.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89bdffb640 1 -- 172.21.15.204:0/400192324 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89c003c420 con 0x7f89c8138550 2024-09-15T01:37:54.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:54.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89bdffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:54.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89bdffb640 10 monclient: dump: 2024-09-15T01:37:54.771 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:54.771 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:54.771 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:54.771 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:54.771 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:54.771 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89bdffb640 10 monclient: _finish_auth 0 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:54.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 5 monclient: authenticate success, global_id 15618 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89bdffb640 1 -- 172.21.15.204:0/400192324 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c003c5a0 con 0x7f89c8138550 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 10 monclient: _renew_subs 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89cfaa8640 1 -- 172.21.15.204:0/400192324 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c808da50 con 0x7f89c8138550 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89bdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.749+0000 7f89bdffb640 1 -- 172.21.15.204:0/400192324 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89c003cb90 con 0x7f89c8138550 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.750+0000 7f89bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.750+0000 7f89bdffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.750+0000 7f89bdffb640 10 monclient: dump: 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:54.773 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.750+0000 7f89bffff640 4 set_mon_vals no callback set 2024-09-15T01:37:54.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.750+0000 7f89cfaa8640 10 monclient: _renew_subs 2024-09-15T01:37:54.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.750+0000 7f89cfaa8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:54.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.750+0000 7f89cfaa8640 1 -- 172.21.15.204:0/400192324 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c808d2e0 con 0x7f89c8138550 2024-09-15T01:37:54.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.750+0000 7f89bdffb640 1 -- 172.21.15.204:0/400192324 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f89c0003040 con 0x7f89c8138550 2024-09-15T01:37:54.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.751+0000 7f89cfaa8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:54.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.751+0000 7f89cfaa8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:54.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.751+0000 7f89cfaa8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:54.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.751+0000 7f89cfaa8640 1 -- 172.21.15.204:0/400192324 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c8066ba0 con 0x7f89c8138550 2024-09-15T01:37:54.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.751+0000 7f89bdffb640 1 --2- 172.21.15.204:0/400192324 >> v2:172.21.15.204:6812/28210 conn(0x7f89ac0397b0 0x7f89ac03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:54.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.751+0000 7f89bdffb640 1 -- 172.21.15.204:0/400192324 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 10106+0+0 (secure 0 0 0) 0x7f89c006dc20 con 0x7f89c8138550 2024-09-15T01:37:54.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.751+0000 7f89cd844640 1 --2- 172.21.15.204:0/400192324 >> v2:172.21.15.204:6812/28210 conn(0x7f89ac0397b0 0x7f89ac03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:54.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.751+0000 7f89cd844640 10 monclient: get_auth_request con 0x7f89ac0397b0 auth_method 0 2024-09-15T01:37:54.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.751+0000 7f89cd844640 1 --2- 172.21.15.204:0/400192324 >> v2:172.21.15.204:6812/28210 conn(0x7f89ac0397b0 0x7f89ac03bc60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f89c4001a20 tx=0x7f89c4001db0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:54.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.755+0000 7f89bdffb640 1 -- 172.21.15.204:0/400192324 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f89c0038da0 con 0x7f89c8138550 2024-09-15T01:37:54.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.755+0000 7f89bdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:54.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.755+0000 7f89bdffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:54.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.916+0000 7f89cfaa8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h20"}] 2024-09-15T01:37:54.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.916+0000 7f89cfaa8640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h20"}] 2024-09-15T01:37:54.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.916+0000 7f89cfaa8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:54.919 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:54.916+0000 7f89cfaa8640 1 -- 172.21.15.204:0/400192324 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h20"} v 0) v1 -- 0x7f89c8070cc0 con 0x7f89c8138550 2024-09-15T01:37:55.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.569+0000 7f89bdffb640 1 -- 172.21.15.204:0/400192324 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h20"}]=0 removed item id -76 name 'h20' from crush map v238) v1 ==== 120+0+0 (secure 0 0 0) 0x7f89c00387c0 con 0x7f89c8138550 2024-09-15T01:37:55.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.569+0000 7f89bdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h20"}] 2024-09-15T01:37:55.574 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.569+0000 7f89bdffb640 10 monclient: _finish_command 2 = system:0 removed item id -76 name 'h20' from crush map 2024-09-15T01:37:55.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.669+0000 7f89cfaa8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h20"}] 2024-09-15T01:37:55.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.669+0000 7f89cfaa8640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h20"}] 2024-09-15T01:37:55.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.669+0000 7f89cfaa8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:37:55.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.669+0000 7f89cfaa8640 1 -- 172.21.15.204:0/400192324 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h20"} v 0) v1 -- 0x7f89c8099fb0 con 0x7f89c8138550 2024-09-15T01:37:55.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.670+0000 7f89bdffb640 1 -- 172.21.15.204:0/400192324 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h20"}]=0 device 'h20' does not appear in the crush map v238) v1 ==== 120+0+0 (secure 0 0 0) 0x7f89c00387c0 con 0x7f89c8138550 2024-09-15T01:37:55.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.670+0000 7f89bdffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h20"}] 2024-09-15T01:37:55.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.670+0000 7f89bdffb640 10 monclient: _finish_command 3 = system:0 device 'h20' does not appear in the crush map 2024-09-15T01:37:55.673 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h20' does not appear in the crush map 2024-09-15T01:37:55.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.672+0000 7f89cfaa8640 1 -- 172.21.15.204:0/400192324 >> v2:172.21.15.204:6812/28210 conn(0x7f89ac0397b0 msgr2=0x7f89ac03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:55.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.672+0000 7f89cfaa8640 1 --2- 172.21.15.204:0/400192324 >> v2:172.21.15.204:6812/28210 conn(0x7f89ac0397b0 0x7f89ac03bc60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f89c4001a20 tx=0x7f89c4001db0 comp rx=0 tx=0).stop 2024-09-15T01:37:55.675 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.672+0000 7f89cfaa8640 10 monclient: shutdown 2024-09-15T01:37:55.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.672+0000 7f89cfaa8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:55.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.672+0000 7f89cfaa8640 1 -- 172.21.15.204:0/400192324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89c8138550 msgr2=0x7f89c8141b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:55.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.672+0000 7f89cfaa8640 1 --2- 172.21.15.204:0/400192324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89c8138550 0x7f89c8141b90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009d30 tx=0x7f89c0037000 comp rx=0 tx=0).stop 2024-09-15T01:37:55.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.672+0000 7f89cfaa8640 1 -- 172.21.15.204:0/400192324 shutdown_connections 2024-09-15T01:37:55.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.672+0000 7f89cfaa8640 1 --2- 172.21.15.204:0/400192324 >> v2:172.21.15.204:6812/28210 conn(0x7f89ac0397b0 0x7f89ac03bc60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:55.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.672+0000 7f89cfaa8640 1 --2- 172.21.15.204:0/400192324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89c8138550 0x7f89c8141b90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:55.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.672+0000 7f89cfaa8640 1 --2- 172.21.15.204:0/400192324 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f89c80a0b40 0x7f89c8094750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:55.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.672+0000 7f89cfaa8640 1 --2- 172.21.15.204:0/400192324 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f89c809acf0 0x7f89c806e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:55.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.672+0000 7f89cfaa8640 1 -- 172.21.15.204:0/400192324 >> 172.21.15.204:0/400192324 conn(0x7f89c80a2e30 msgr2=0x7f89c809e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:55.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.673+0000 7f89cfaa8640 1 -- 172.21.15.204:0/400192324 shutdown_connections 2024-09-15T01:37:55.676 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.673+0000 7f89cfaa8640 1 -- 172.21.15.204:0/400192324 wait complete. 2024-09-15T01:37:55.683 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:55.684 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h21 2024-09-15T01:37:55.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.754+0000 7fa31c509640 10 monclient: get_monmap_and_config 2024-09-15T01:37:55.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.754+0000 7fa31c509640 10 monclient: build_initial_monmap 2024-09-15T01:37:55.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.754+0000 7fa31c509640 10 monclient: monmap: 2024-09-15T01:37:55.757 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:55.757 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:55.757 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:55.756100+0000 2024-09-15T01:37:55.757 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:55.756100+0000 2024-09-15T01:37:55.757 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:55.758 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:55.758 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:55.758 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:55.758 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:55.758 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:55.758 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:55.758 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:55.758 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:55.758 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:55.758 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:55.758 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:55.758 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:55.759 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:55.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.755+0000 7fa31c509640 1 Processor -- start 2024-09-15T01:37:55.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.755+0000 7fa31c509640 1 -- start start 2024-09-15T01:37:55.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.755+0000 7fa31c509640 10 monclient: init 2024-09-15T01:37:55.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.755+0000 7fa31c509640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:55.766 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.755+0000 7fa31c509640 10 monclient: _add_conns ranks=[8,0,7,1,9,5,4,2,10,3,6] 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.755+0000 7fa31c509640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3140a2e30 0x7fa314132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.755+0000 7fa31c509640 10 monclient(hunting): picked mon.noname-e con 0x7fa3140a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.755+0000 7fa31c509640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa31409fb80 addr v1:172.21.15.178:6789/0 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.755+0000 7fa31c509640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409ff70 0x7fa3140a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.755+0000 7fa31c509640 10 monclient(hunting): picked mon.noname-i con 0x7fa31409ff70 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.755+0000 7fa31c509640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31404b720 con 0x7fa31409fb80 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31c509640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31c509640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa31409e6c0 con 0x7fa31409ff70 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31c509640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31c509640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa3140a3200 con 0x7fa3140a2e30 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31c509640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31c509640 10 monclient(hunting): authenticate will time out at 1091.399025s 2024-09-15T01:37:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa319aa4640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7fa31409fb80 legacy=0x7fa3140a0a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31a2a5640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3140a2e30 msgr2=0x7fa314132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31a2a5640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3140a2e30 0x7fa314132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409ff70 0x7fa3140a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409ff70 0x7fa3140a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:57536/0 (socket says 172.21.15.204:57536) 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 1 -- 172.21.15.204:0/2403773786 learned_addr learned my addr 172.21.15.204:0/2403773786 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 10 monclient(hunting): get_auth_request con 0x7fa31409ff70 auth_method 0 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 10 monclient(hunting): handle_auth_done global_id 24721 payload 274 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 1 -- 172.21.15.204:0/2403773786 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3140a2e30 msgr2=0x7fa314132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 1 --2- 172.21.15.204:0/2403773786 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3140a2e30 0x7fa314132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 1 -- 172.21.15.204:0/2403773786 >> v1:172.21.15.178:6789/0 conn(0x7fa31409fb80 legacy=0x7fa3140a0a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 10 monclient: _finish_hunting 0 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 1 monclient: found mon.noname-i 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.756+0000 7fa31aaa6640 1 -- 172.21.15.204:0/2403773786 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3140a1f10 con 0x7fa31409ff70 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31aaa6640 1 --2- 172.21.15.204:0/2403773786 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409ff70 0x7fa3140a1150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009f50 tx=0x7fa30c009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=2d8eccdf3e1e077d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa3192a3640 1 -- 172.21.15.204:0/2403773786 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa30c007880 con 0x7fa31409ff70 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa3192a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa3192a3640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:55.769 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa3192a3640 10 monclient: dump: 2024-09-15T01:37:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:55.770 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa3192a3640 10 monclient: _finish_auth 0 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa3192a3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa3192a3640 1 -- 172.21.15.204:0/2403773786 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c02d050 con 0x7fa31409ff70 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 5 monclient: authenticate success, global_id 24721 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa3192a3640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa3192a3640 1 -- 172.21.15.204:0/2403773786 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa30c030360 con 0x7fa31409ff70 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa3192a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:55.771 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 4 set_mon_vals no callback set 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa3192a3640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa3192a3640 10 monclient: dump: 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:55.772 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:55.773 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:55.773 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:55.773 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:55.773 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:55.773 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:55.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 10 monclient: shutdown 2024-09-15T01:37:55.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:55.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 1 -- 172.21.15.204:0/2403773786 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409ff70 msgr2=0x7fa3140a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:55.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 1 --2- 172.21.15.204:0/2403773786 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409ff70 0x7fa3140a1150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009f50 tx=0x7fa30c009840 comp rx=0 tx=0).stop 2024-09-15T01:37:55.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 1 -- 172.21.15.204:0/2403773786 shutdown_connections 2024-09-15T01:37:55.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 1 --2- 172.21.15.204:0/2403773786 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409ff70 0x7fa3140a1150 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:55.773 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 1 --2- 172.21.15.204:0/2403773786 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3140a2e30 0x7fa314132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 1 -- 172.21.15.204:0/2403773786 >> 172.21.15.204:0/2403773786 conn(0x7fa31406e860 msgr2=0x7fa31406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 1 -- 172.21.15.204:0/2403773786 shutdown_connections 2024-09-15T01:37:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.757+0000 7fa31c509640 1 -- 172.21.15.204:0/2403773786 wait complete. 2024-09-15T01:37:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.758+0000 7fa31c509640 10 monclient: build_initial_monmap 2024-09-15T01:37:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.758+0000 7fa31c509640 10 monclient: monmap: 2024-09-15T01:37:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:55.774 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:55.775 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.758+0000 7fa31c509640 1 Processor -- start 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.758+0000 7fa31c509640 1 -- start start 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.758+0000 7fa31c509640 10 monclient: init 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient: _add_conns ranks=[7,0,4,2,6,3,5,8,1] 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409fb80 0x7fa314131870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient(hunting): picked mon.noname-i con 0x7fa31409fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa31409ff70 0x7fa31411e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient(hunting): picked mon.noname-a con 0x7fa31409ff70 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3140a2e30 0x7fa31411e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient(hunting): picked mon.noname-c con 0x7fa3140a2e30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:55.776 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31409cab0 con 0x7fa31409ff70 2024-09-15T01:37:55.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:55.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa314121ca0 con 0x7fa3140a2e30 2024-09-15T01:37:55.777 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa31409d650 con 0x7fa31409fb80 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient(hunting): authenticate will time out at 1091.402222s 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409fb80 0x7fa314131870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409fb80 0x7fa314131870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:57538/0 (socket says 172.21.15.204:57538) 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 1 -- 172.21.15.204:0/3839032651 learned_addr learned my addr 172.21.15.204:0/3839032651 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 10 monclient(hunting): get_auth_request con 0x7fa31409fb80 auth_method 0 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31aaa6640 1 -- 172.21.15.204:0/3839032651 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3140a2e30 msgr2=0x7fa31411e8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa319aa4640 1 -- 172.21.15.204:0/3839032651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa31409ff70 msgr2=0x7fa31411e3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:55.781 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:55.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:55.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31aaa6640 1 --2- 172.21.15.204:0/3839032651 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3140a2e30 0x7fa31411e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:55.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa319aa4640 1 --2- 172.21.15.204:0/3839032651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa31409ff70 0x7fa31411e3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:55.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:55.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 10 monclient(hunting): handle_auth_done global_id 24730 payload 824 2024-09-15T01:37:55.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 1 -- 172.21.15.204:0/3839032651 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3140a2e30 msgr2=0x7fa31411e8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:55.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 1 --2- 172.21.15.204:0/3839032651 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3140a2e30 0x7fa31411e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:55.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 1 -- 172.21.15.204:0/3839032651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa31409ff70 msgr2=0x7fa31411e3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:55.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 1 --2- 172.21.15.204:0/3839032651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa31409ff70 0x7fa31411e3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:55.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 10 monclient: _finish_hunting 0 2024-09-15T01:37:55.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 1 monclient: found mon.noname-i 2024-09-15T01:37:55.782 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 1 -- 172.21.15.204:0/3839032651 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c009ae0 con 0x7fa31409fb80 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31a2a5640 1 --2- 172.21.15.204:0/3839032651 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409fb80 0x7fa314131870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa31000b5f0 tx=0x7fa310009120 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa30a7fc640 1 -- 172.21.15.204:0/3839032651 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa310009280 con 0x7fa31409fb80 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa30a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa30a7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa30a7fc640 10 monclient: dump: 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:55.783 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa30a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa30a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 5 monclient: authenticate success, global_id 24730 2024-09-15T01:37:55.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa30a7fc640 1 -- 172.21.15.204:0/3839032651 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3100050f0 con 0x7fa31409fb80 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa30a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa30a7fc640 1 -- 172.21.15.204:0/3839032651 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa310005720 con 0x7fa31409fb80 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa30a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa30a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa30a7fc640 10 monclient: dump: 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:55.785 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa318aa2640 4 set_mon_vals no callback set 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient: _renew_subs 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 1 -- 172.21.15.204:0/3839032651 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3141219c0 con 0x7fa31409fb80 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient: _renew_subs 2024-09-15T01:37:55.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.759+0000 7fa31c509640 1 -- 172.21.15.204:0/3839032651 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31411ff00 con 0x7fa31409fb80 2024-09-15T01:37:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.760+0000 7fa30a7fc640 1 -- 172.21.15.204:0/3839032651 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa3100052a0 con 0x7fa31409fb80 2024-09-15T01:37:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.760+0000 7fa31c509640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.760+0000 7fa31c509640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.760+0000 7fa31c509640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.760+0000 7fa31c509640 1 -- 172.21.15.204:0/3839032651 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa314070cc0 con 0x7fa31409fb80 2024-09-15T01:37:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.760+0000 7fa30a7fc640 1 --2- 172.21.15.204:0/3839032651 >> v2:172.21.15.204:6812/28210 conn(0x7fa2f80397b0 0x7fa2f803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.761+0000 7fa30a7fc640 1 -- 172.21.15.204:0/3839032651 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 9893+0+0 (secure 0 0 0) 0x7fa31004c640 con 0x7fa31409fb80 2024-09-15T01:37:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.761+0000 7fa319aa4640 1 --2- 172.21.15.204:0/3839032651 >> v2:172.21.15.204:6812/28210 conn(0x7fa2f80397b0 0x7fa2f803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.761+0000 7fa319aa4640 10 monclient: get_auth_request con 0x7fa2f80397b0 auth_method 0 2024-09-15T01:37:55.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.761+0000 7fa319aa4640 1 --2- 172.21.15.204:0/3839032651 >> v2:172.21.15.204:6812/28210 conn(0x7fa2f80397b0 0x7fa2f803bc60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa3140a2e00 tx=0x7fa304006c80 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.765+0000 7fa30a7fc640 1 -- 172.21.15.204:0/3839032651 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa310005550 con 0x7fa31409fb80 2024-09-15T01:37:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.765+0000 7fa30a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:55.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.765+0000 7fa30a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:55.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.925+0000 7fa31c509640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h21"}] 2024-09-15T01:37:55.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.925+0000 7fa31c509640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h21"}] 2024-09-15T01:37:55.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.925+0000 7fa31c509640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:55.927 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:55.925+0000 7fa31c509640 1 -- 172.21.15.204:0/3839032651 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h21"} v 0) v1 -- 0x7fa31409ddb0 con 0x7fa31409fb80 2024-09-15T01:37:56.480 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.478+0000 7fa30a7fc640 1 -- 172.21.15.204:0/3839032651 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h21"}]=0 removed item id -79 name 'h21' from crush map v239) v1 ==== 120+0+0 (secure 0 0 0) 0x7fa310018360 con 0x7fa31409fb80 2024-09-15T01:37:56.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.478+0000 7fa30a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h21"}] 2024-09-15T01:37:56.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.478+0000 7fa30a7fc640 10 monclient: _finish_command 2 = system:0 removed item id -79 name 'h21' from crush map 2024-09-15T01:37:56.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.578+0000 7fa31c509640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h21"}] 2024-09-15T01:37:56.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.578+0000 7fa31c509640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h21"}] 2024-09-15T01:37:56.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.578+0000 7fa31c509640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:37:56.581 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.578+0000 7fa31c509640 1 -- 172.21.15.204:0/3839032651 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h21"} v 0) v1 -- 0x7fa3140008d0 con 0x7fa31409fb80 2024-09-15T01:37:56.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.585+0000 7fa30a7fc640 1 -- 172.21.15.204:0/3839032651 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h21"}]=0 device 'h21' does not appear in the crush map v239) v1 ==== 120+0+0 (secure 0 0 0) 0x7fa3100193c0 con 0x7fa31409fb80 2024-09-15T01:37:56.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.585+0000 7fa30a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h21"}] 2024-09-15T01:37:56.588 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.585+0000 7fa30a7fc640 10 monclient: _finish_command 3 = system:0 device 'h21' does not appear in the crush map 2024-09-15T01:37:56.588 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h21' does not appear in the crush map 2024-09-15T01:37:56.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 1 -- 172.21.15.204:0/3839032651 >> v2:172.21.15.204:6812/28210 conn(0x7fa2f80397b0 msgr2=0x7fa2f803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:56.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 1 --2- 172.21.15.204:0/3839032651 >> v2:172.21.15.204:6812/28210 conn(0x7fa2f80397b0 0x7fa2f803bc60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa3140a2e00 tx=0x7fa304006c80 comp rx=0 tx=0).stop 2024-09-15T01:37:56.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 10 monclient: shutdown 2024-09-15T01:37:56.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:56.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 1 -- 172.21.15.204:0/3839032651 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409fb80 msgr2=0x7fa314131870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:56.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 1 --2- 172.21.15.204:0/3839032651 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409fb80 0x7fa314131870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa31000b5f0 tx=0x7fa310009120 comp rx=0 tx=0).stop 2024-09-15T01:37:56.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 1 -- 172.21.15.204:0/3839032651 shutdown_connections 2024-09-15T01:37:56.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 1 --2- 172.21.15.204:0/3839032651 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa3140a2e30 0x7fa31411e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:56.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 1 --2- 172.21.15.204:0/3839032651 >> v2:172.21.15.204:6812/28210 conn(0x7fa2f80397b0 0x7fa2f803bc60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:56.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 1 --2- 172.21.15.204:0/3839032651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa31409ff70 0x7fa31411e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:56.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 1 --2- 172.21.15.204:0/3839032651 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa31409fb80 0x7fa314131870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:56.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 1 -- 172.21.15.204:0/3839032651 >> 172.21.15.204:0/3839032651 conn(0x7fa31406e860 msgr2=0x7fa31409bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:56.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 1 -- 172.21.15.204:0/3839032651 shutdown_connections 2024-09-15T01:37:56.592 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.588+0000 7fa31c509640 1 -- 172.21.15.204:0/3839032651 wait complete. 2024-09-15T01:37:56.599 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:56.599 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h22 2024-09-15T01:37:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.670+0000 7f3b59c9f640 10 monclient: get_monmap_and_config 2024-09-15T01:37:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.670+0000 7f3b59c9f640 10 monclient: build_initial_monmap 2024-09-15T01:37:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.670+0000 7f3b59c9f640 10 monclient: monmap: 2024-09-15T01:37:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:56.672156+0000 2024-09-15T01:37:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:56.672156+0000 2024-09-15T01:37:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:56.673 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:56.674 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:56.674 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:56.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 1 Processor -- start 2024-09-15T01:37:56.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 1 -- start start 2024-09-15T01:37:56.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 10 monclient: init 2024-09-15T01:37:56.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:56.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 10 monclient: _add_conns ranks=[1,9,3,4,10,5,6,2,7,8,0] 2024-09-15T01:37:56.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3b540a2e30 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:56.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3b5409fb80 0x7f3b5409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:56.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 10 monclient(hunting): picked mon.noname-a con 0x7f3b5409fb80 addr v2:172.21.15.178:3300/0 2024-09-15T01:37:56.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b54137d20 0x7f3b5413a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:56.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 10 monclient(hunting): picked mon.noname-g con 0x7f3b54137d20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:37:56.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5404b720 con 0x7f3b540a2e30 2024-09-15T01:37:56.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:56.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5409e6c0 con 0x7f3b5409fb80 2024-09-15T01:37:56.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:56.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b540a04b0 con 0x7f3b54137d20 2024-09-15T01:37:56.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:56.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.671+0000 7f3b59c9f640 10 monclient(hunting): authenticate will time out at 1092.314961s 2024-09-15T01:37:56.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b54137d20 0x7f3b5413a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b537fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f3b540a2e30 0x7f3b54132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:48376/0 (socket says 172.21.15.204:48376) 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b537fe640 1 -- 172.21.15.204:0/4099143336 learned_addr learned my addr 172.21.15.204:0/4099143336 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b52ffd640 1 -- 172.21.15.204:0/4099143336 >> v2:172.21.15.178:3300/0 conn(0x7f3b5409fb80 msgr2=0x7f3b5409ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 10 monclient(hunting): get_auth_request con 0x7f3b54137d20 auth_method 0 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b52ffd640 1 --2- 172.21.15.204:0/4099143336 >> v2:172.21.15.178:3300/0 conn(0x7f3b5409fb80 0x7f3b5409ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 10 monclient(hunting): handle_auth_done global_id 15627 payload 274 2024-09-15T01:37:56.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b527fc640 1 -- 172.21.15.204:0/4099143336 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464704385 0 0) 0x7f3b5404b720 con 0x7f3b540a2e30 2024-09-15T01:37:56.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 1 -- 172.21.15.204:0/4099143336 >> v2:172.21.15.178:3300/0 conn(0x7f3b5409fb80 msgr2=0x7f3b5409ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:56.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 1 --2- 172.21.15.204:0/4099143336 >> v2:172.21.15.178:3300/0 conn(0x7f3b5409fb80 0x7f3b5409ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:56.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 1 -- 172.21.15.204:0/4099143336 >> v1:172.21.15.204:6789/0 conn(0x7f3b540a2e30 legacy=0x7f3b54132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:56.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 10 monclient: _finish_hunting 0 2024-09-15T01:37:56.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 1 monclient: found mon.noname-g 2024-09-15T01:37:56.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:56.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:37:56.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b53fff640 1 -- 172.21.15.204:0/4099143336 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b54133170 con 0x7f3b54137d20 2024-09-15T01:37:56.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.672+0000 7f3b527fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:37:56.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b53fff640 1 --2- 172.21.15.204:0/4099143336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b54137d20 0x7f3b5413a100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009e60 tx=0x7f3b4c009780 comp rx=0 tx=0).ready entity=mon.3 client_cookie=4860b264673e4f64 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:56.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b527fc640 1 -- 172.21.15.204:0/4099143336 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b4c0079d0 con 0x7f3b54137d20 2024-09-15T01:37:56.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b527fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b527fc640 10 monclient: dump: 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:56.687 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:56.688 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:56.688 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:56.688 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:56.688 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:56.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b527fc640 10 monclient: _finish_auth 0 2024-09-15T01:37:56.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:56.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 5 monclient: authenticate success, global_id 15627 2024-09-15T01:37:56.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b527fc640 1 -- 172.21.15.204:0/4099143336 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c02d050 con 0x7f3b54137d20 2024-09-15T01:37:56.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b527fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b527fc640 1 -- 172.21.15.204:0/4099143336 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b4c007b50 con 0x7f3b54137d20 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 4 set_mon_vals no callback set 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b527fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b527fc640 10 monclient: dump: 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:56.689 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 10 monclient: shutdown 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:56.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 1 -- 172.21.15.204:0/4099143336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b54137d20 msgr2=0x7f3b5413a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 1 --2- 172.21.15.204:0/4099143336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b54137d20 0x7f3b5413a100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009e60 tx=0x7f3b4c009780 comp rx=0 tx=0).stop 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 1 -- 172.21.15.204:0/4099143336 shutdown_connections 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 1 --2- 172.21.15.204:0/4099143336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b54137d20 0x7f3b5413a100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 1 --2- 172.21.15.204:0/4099143336 >> v2:172.21.15.178:3300/0 conn(0x7f3b5409fb80 0x7f3b5409ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 1 -- 172.21.15.204:0/4099143336 >> 172.21.15.204:0/4099143336 conn(0x7f3b5406e860 msgr2=0x7f3b5406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 1 -- 172.21.15.204:0/4099143336 shutdown_connections 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.673+0000 7f3b59c9f640 1 -- 172.21.15.204:0/4099143336 wait complete. 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.674+0000 7f3b59c9f640 10 monclient: build_initial_monmap 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.674+0000 7f3b59c9f640 10 monclient: monmap: 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:56.691 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:56.692 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.674+0000 7f3b59c9f640 1 Processor -- start 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.674+0000 7f3b59c9f640 1 -- start start 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.674+0000 7f3b59c9f640 10 monclient: init 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 10 monclient: _add_conns ranks=[8,5,6,4,0,7,3,1,2] 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3b5409fb80 0x7f3b54130eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 10 monclient(hunting): picked mon.noname-e con 0x7f3b5409fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b540a2e30 0x7f3b54141680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 10 monclient(hunting): picked mon.noname-h con 0x7f3b540a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b54137d20 0x7f3b54141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 10 monclient(hunting): picked mon.noname-d con 0x7f3b54137d20 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5409a2c0 con 0x7f3b540a2e30 2024-09-15T01:37:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5409ad50 con 0x7f3b54137d20 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5409ca40 con 0x7f3b5409fb80 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 10 monclient(hunting): authenticate will time out at 1092.318227s 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b540a2e30 0x7f3b54141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b540a2e30 0x7f3b54141680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:32768/0 (socket says 172.21.15.204:32768) 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b537fe640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3b5409fb80 msgr2=0x7f3b54130eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 1 -- 172.21.15.204:0/1788136818 learned_addr learned my addr 172.21.15.204:0/1788136818 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 10 monclient(hunting): get_auth_request con 0x7f3b540a2e30 auth_method 0 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:56.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b537fe640 1 --2- 172.21.15.204:0/1788136818 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3b5409fb80 0x7f3b54130eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b53fff640 1 --2- 172.21.15.204:0/1788136818 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b54137d20 0x7f3b54141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b53fff640 10 monclient(hunting): get_auth_request con 0x7f3b54137d20 auth_method 0 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b53fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b53fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 10 monclient(hunting): handle_auth_done global_id 6512 payload 824 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 1 -- 172.21.15.204:0/1788136818 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3b5409fb80 msgr2=0x7f3b54130eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 1 --2- 172.21.15.204:0/1788136818 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3b5409fb80 0x7f3b54130eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 1 -- 172.21.15.204:0/1788136818 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b54137d20 msgr2=0x7f3b54141bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 1 --2- 172.21.15.204:0/1788136818 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b54137d20 0x7f3b54141bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:37:56.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 1 monclient: found mon.noname-h 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 1 -- 172.21.15.204:0/1788136818 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c009a20 con 0x7f3b540a2e30 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b52ffd640 1 --2- 172.21.15.204:0/1788136818 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b540a2e30 0x7f3b54141680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800b600 tx=0x7f3b480091a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b3b7fe640 1 -- 172.21.15.204:0/1788136818 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b48015070 con 0x7f3b540a2e30 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b3b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b3b7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b3b7fe640 10 monclient: dump: 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:56.696 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:56.697 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:56.697 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:56.697 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:56.697 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:56.697 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:56.697 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:56.697 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:56.697 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:56.697 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:56.697 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:56.697 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:56.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b3b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:37:56.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b3b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:56.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 5 monclient: authenticate success, global_id 6512 2024-09-15T01:37:56.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b3b7fe640 1 -- 172.21.15.204:0/1788136818 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b48006150 con 0x7f3b540a2e30 2024-09-15T01:37:56.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b3b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b3b7fe640 1 -- 172.21.15.204:0/1788136818 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b48006740 con 0x7f3b540a2e30 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 10 monclient: _renew_subs 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b59c9f640 1 -- 172.21.15.204:0/1788136818 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5409d650 con 0x7f3b540a2e30 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.676+0000 7f3b3b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.676+0000 7f3b3b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.675+0000 7f3b51ffb640 4 set_mon_vals no callback set 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.676+0000 7f3b3b7fe640 10 monclient: dump: 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:56.699 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.676+0000 7f3b59c9f640 10 monclient: _renew_subs 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.676+0000 7f3b59c9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.676+0000 7f3b59c9f640 1 -- 172.21.15.204:0/1788136818 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5411c850 con 0x7f3b540a2e30 2024-09-15T01:37:56.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.676+0000 7f3b3b7fe640 1 -- 172.21.15.204:0/1788136818 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3b480062c0 con 0x7f3b540a2e30 2024-09-15T01:37:56.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.676+0000 7f3b59c9f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:56.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.676+0000 7f3b59c9f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:56.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.676+0000 7f3b59c9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:56.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.676+0000 7f3b59c9f640 1 -- 172.21.15.204:0/1788136818 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b54070cc0 con 0x7f3b540a2e30 2024-09-15T01:37:56.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.677+0000 7f3b3b7fe640 1 --2- 172.21.15.204:0/1788136818 >> v2:172.21.15.204:6812/28210 conn(0x7f3b400397b0 0x7f3b4003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:56.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.677+0000 7f3b3b7fe640 1 -- 172.21.15.204:0/1788136818 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 9680+0+0 (secure 0 0 0) 0x7f3b4804c210 con 0x7f3b540a2e30 2024-09-15T01:37:56.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.677+0000 7f3b537fe640 1 --2- 172.21.15.204:0/1788136818 >> v2:172.21.15.204:6812/28210 conn(0x7f3b400397b0 0x7f3b4003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:56.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.677+0000 7f3b537fe640 10 monclient: get_auth_request con 0x7f3b400397b0 auth_method 0 2024-09-15T01:37:56.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.677+0000 7f3b537fe640 1 --2- 172.21.15.204:0/1788136818 >> v2:172.21.15.204:6812/28210 conn(0x7f3b400397b0 0x7f3b4003bc60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3b440010e0 tx=0x7f3b44007ae0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:56.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.681+0000 7f3b3b7fe640 1 -- 172.21.15.204:0/1788136818 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f3b48003e10 con 0x7f3b540a2e30 2024-09-15T01:37:56.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.681+0000 7f3b3b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:56.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.681+0000 7f3b3b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:56.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.842+0000 7f3b59c9f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h22"}] 2024-09-15T01:37:56.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.842+0000 7f3b59c9f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h22"}] 2024-09-15T01:37:56.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.842+0000 7f3b59c9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:56.845 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:56.842+0000 7f3b59c9f640 1 -- 172.21.15.204:0/1788136818 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h22"} v 0) v1 -- 0x7f3b5409b430 con 0x7f3b540a2e30 2024-09-15T01:37:57.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.586+0000 7f3b3b7fe640 1 -- 172.21.15.204:0/1788136818 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h22"}]=0 removed item id -82 name 'h22' from crush map v240) v1 ==== 120+0+0 (secure 0 0 0) 0x7f3b4801acb0 con 0x7f3b540a2e30 2024-09-15T01:37:57.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.586+0000 7f3b3b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h22"}] 2024-09-15T01:37:57.589 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.586+0000 7f3b3b7fe640 10 monclient: _finish_command 2 = system:0 removed item id -82 name 'h22' from crush map 2024-09-15T01:37:57.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.686+0000 7f3b59c9f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h22"}] 2024-09-15T01:37:57.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.686+0000 7f3b59c9f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h22"}] 2024-09-15T01:37:57.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.686+0000 7f3b59c9f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:37:57.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.686+0000 7f3b59c9f640 1 -- 172.21.15.204:0/1788136818 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h22"} v 0) v1 -- 0x7f3b540008d0 con 0x7f3b540a2e30 2024-09-15T01:37:57.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.687+0000 7f3b3b7fe640 1 -- 172.21.15.204:0/1788136818 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h22"}]=0 device 'h22' does not appear in the crush map v240) v1 ==== 120+0+0 (secure 0 0 0) 0x7f3b4800eb30 con 0x7f3b540a2e30 2024-09-15T01:37:57.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.687+0000 7f3b3b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h22"}] 2024-09-15T01:37:57.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.687+0000 7f3b3b7fe640 10 monclient: _finish_command 3 = system:0 device 'h22' does not appear in the crush map 2024-09-15T01:37:57.690 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h22' does not appear in the crush map 2024-09-15T01:37:57.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 1 -- 172.21.15.204:0/1788136818 >> v2:172.21.15.204:6812/28210 conn(0x7f3b400397b0 msgr2=0x7f3b4003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:57.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 1 --2- 172.21.15.204:0/1788136818 >> v2:172.21.15.204:6812/28210 conn(0x7f3b400397b0 0x7f3b4003bc60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3b440010e0 tx=0x7f3b44007ae0 comp rx=0 tx=0).stop 2024-09-15T01:37:57.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 10 monclient: shutdown 2024-09-15T01:37:57.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:57.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 1 -- 172.21.15.204:0/1788136818 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b540a2e30 msgr2=0x7f3b54141680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:57.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 1 --2- 172.21.15.204:0/1788136818 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b540a2e30 0x7f3b54141680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800b600 tx=0x7f3b480091a0 comp rx=0 tx=0).stop 2024-09-15T01:37:57.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 1 -- 172.21.15.204:0/1788136818 shutdown_connections 2024-09-15T01:37:57.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 1 --2- 172.21.15.204:0/1788136818 >> v2:172.21.15.204:6812/28210 conn(0x7f3b400397b0 0x7f3b4003bc60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:57.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 1 --2- 172.21.15.204:0/1788136818 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b54137d20 0x7f3b54141bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:57.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 1 --2- 172.21.15.204:0/1788136818 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b540a2e30 0x7f3b54141680 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:57.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 1 --2- 172.21.15.204:0/1788136818 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3b5409fb80 0x7f3b54130eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:57.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 1 -- 172.21.15.204:0/1788136818 >> 172.21.15.204:0/1788136818 conn(0x7f3b5406e860 msgr2=0x7f3b540a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:57.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 1 -- 172.21.15.204:0/1788136818 shutdown_connections 2024-09-15T01:37:57.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.689+0000 7f3b59c9f640 1 -- 172.21.15.204:0/1788136818 wait complete. 2024-09-15T01:37:57.701 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:57.701 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h23 2024-09-15T01:37:57.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.772+0000 7f190739d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:57.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.772+0000 7f190739d640 10 monclient: build_initial_monmap 2024-09-15T01:37:57.775 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.772+0000 7f190739d640 10 monclient: monmap: 2024-09-15T01:37:57.775 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:57.775 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:57.775 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:57.774015+0000 2024-09-15T01:37:57.775 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:57.774015+0000 2024-09-15T01:37:57.775 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:57.775 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:57.775 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:57.776 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:57.776 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:57.776 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:57.776 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:57.776 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:57.776 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:57.776 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:57.776 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:57.776 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:57.776 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:57.776 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:57.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 1 Processor -- start 2024-09-15T01:37:57.784 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 1 -- start start 2024-09-15T01:37:57.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 10 monclient: init 2024-09-15T01:37:57.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:57.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 10 monclient: _add_conns ranks=[5,1,2,7,4,10,0,9,8,3,6] 2024-09-15T01:37:57.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f19000a2e30 0x7f1900132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:57.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 10 monclient(hunting): picked mon.noname-h con 0x7f19000a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:37:57.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f190009fb80 addr v1:172.21.15.204:6789/0 2024-09-15T01:37:57.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f190009ff70 0x7f19000a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:57.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 10 monclient(hunting): picked mon.noname-b con 0x7f190009ff70 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:57.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190004b720 con 0x7f190009fb80 2024-09-15T01:37:57.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190009e6c0 con 0x7f190009ff70 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19000a3200 con 0x7f19000a2e30 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190739d640 10 monclient(hunting): authenticate will time out at 1093.416865s 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f1904938640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f190009fb80 0x7f19000a0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:46280/0 (socket says 172.21.15.204:46280) 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f1904938640 1 -- 172.21.15.204:0/4049284721 learned_addr learned my addr 172.21.15.204:0/4049284721 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f1905139640 1 --2- 172.21.15.204:0/4049284721 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f19000a2e30 0x7f1900132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f1905139640 10 monclient(hunting): get_auth_request con 0x7f19000a2e30 auth_method 0 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f190593a640 1 -- 172.21.15.204:0/4049284721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f190009ff70 msgr2=0x7f19000a1150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f1905139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f1905139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:57.786 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.773+0000 7f1905139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f190593a640 1 --2- 172.21.15.204:0/4049284721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f190009ff70 0x7f19000a1150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f18f7fff640 1 -- 172.21.15.204:0/4049284721 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555723739 0 0) 0x7f190004b720 con 0x7f190009fb80 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f18f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f18f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f18f7fff640 10 monclient(hunting): my global_id is 7363 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f18f7fff640 1 -- 172.21.15.204:0/4049284721 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18ec0034e0 con 0x7f190009fb80 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 10 monclient(hunting): handle_auth_done global_id 6521 payload 274 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 1 -- 172.21.15.204:0/4049284721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f190009ff70 msgr2=0x7f19000a1150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 1 --2- 172.21.15.204:0/4049284721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f190009ff70 0x7f19000a1150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:57.787 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 1 -- 172.21.15.204:0/4049284721 >> v1:172.21.15.204:6789/0 conn(0x7f190009fb80 legacy=0x7f19000a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:57.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 10 monclient: _finish_hunting 0 2024-09-15T01:37:57.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 1 monclient: found mon.noname-h 2024-09-15T01:37:57.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:57.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:37:57.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 1 -- 172.21.15.204:0/4049284721 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19000a1f10 con 0x7f19000a2e30 2024-09-15T01:37:57.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.774+0000 7f1905139640 1 --2- 172.21.15.204:0/4049284721 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f19000a2e30 0x7f1900132a60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009880 tx=0x7f18fc0090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=77ad5659cb4cc705 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:57.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f18f7fff640 1 -- 172.21.15.204:0/4049284721 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f18fc02ac80 con 0x7f19000a2e30 2024-09-15T01:37:57.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f18f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:57.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f18f7fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:37:57.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f18f7fff640 10 monclient: dump: 2024-09-15T01:37:57.788 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:57.788 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:57.789 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:57.790 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:57.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f18f7fff640 10 monclient: _finish_auth 0 2024-09-15T01:37:57.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f18f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:57.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 5 monclient: authenticate success, global_id 6521 2024-09-15T01:37:57.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f18f7fff640 1 -- 172.21.15.204:0/4049284721 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc02ae00 con 0x7f19000a2e30 2024-09-15T01:37:57.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:57.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f18f7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:57.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:57.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 4 set_mon_vals no callback set 2024-09-15T01:37:57.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f18f7fff640 1 -- 172.21.15.204:0/4049284721 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f18fc031ca0 con 0x7f19000a2e30 2024-09-15T01:37:57.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f18f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:57.790 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f18f7fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f18f7fff640 10 monclient: dump: 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:57.791 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:57.792 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:57.792 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:57.792 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:57.792 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:57.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 10 monclient: shutdown 2024-09-15T01:37:57.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:57.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 1 -- 172.21.15.204:0/4049284721 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f19000a2e30 msgr2=0x7f1900132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:57.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 1 --2- 172.21.15.204:0/4049284721 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f19000a2e30 0x7f1900132a60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009880 tx=0x7f18fc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:57.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 1 -- 172.21.15.204:0/4049284721 shutdown_connections 2024-09-15T01:37:57.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 1 --2- 172.21.15.204:0/4049284721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f190009ff70 0x7f19000a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:57.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 1 --2- 172.21.15.204:0/4049284721 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f19000a2e30 0x7f1900132a60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:57.792 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 1 -- 172.21.15.204:0/4049284721 >> 172.21.15.204:0/4049284721 conn(0x7f190006e860 msgr2=0x7f190006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 1 -- 172.21.15.204:0/4049284721 shutdown_connections 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.775+0000 7f190739d640 1 -- 172.21.15.204:0/4049284721 wait complete. 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.776+0000 7f190739d640 10 monclient: build_initial_monmap 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.776+0000 7f190739d640 10 monclient: monmap: 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:57.793 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.776+0000 7f190739d640 1 Processor -- start 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.776+0000 7f190739d640 1 -- start start 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.776+0000 7f190739d640 10 monclient: init 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 10 monclient: _add_conns ranks=[0,4,6,2,7,3,5,1,8] 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f190009fb80 0x7f190009b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:57.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 10 monclient(hunting): picked mon.noname-a con 0x7f190009fb80 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:57.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f190009ff70 0x7f190012e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:57.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 10 monclient(hunting): picked mon.noname-c con 0x7f190009ff70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:37:57.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19000a2e30 0x7f190011c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:57.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 10 monclient(hunting): picked mon.noname-d con 0x7f19000a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:57.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:57.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190009d650 con 0x7f190009fb80 2024-09-15T01:37:57.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:57.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f190009ca40 con 0x7f190009ff70 2024-09-15T01:37:57.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:57.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f190011da00 con 0x7f19000a2e30 2024-09-15T01:37:57.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:57.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190739d640 10 monclient(hunting): authenticate will time out at 1093.420175s 2024-09-15T01:37:57.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f1905139640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f190009fb80 msgr2=0x7f190009b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:57.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f1904938640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f190009ff70 msgr2=0x7f190012e7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:37:57.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f1905139640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f190009fb80 0x7f190009b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:57.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f1904938640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f190009ff70 0x7f190012e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:57.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19000a2e30 0x7f190011c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:57.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19000a2e30 0x7f190011c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:42924/0 (socket says 172.21.15.204:42924) 2024-09-15T01:37:57.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 1 -- 172.21.15.204:0/949266789 learned_addr learned my addr 172.21.15.204:0/949266789 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:57.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 10 monclient(hunting): get_auth_request con 0x7f19000a2e30 auth_method 0 2024-09-15T01:37:57.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:57.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:57.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:57.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 10 monclient(hunting): handle_auth_done global_id 24567 payload 824 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 1 -- 172.21.15.204:0/949266789 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f190009ff70 msgr2=0x7f190012e7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 1 --2- 172.21.15.204:0/949266789 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f190009ff70 0x7f190012e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 1 -- 172.21.15.204:0/949266789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f190009fb80 msgr2=0x7f190009b5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 1 --2- 172.21.15.204:0/949266789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f190009fb80 0x7f190009b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 10 monclient: _finish_hunting 0 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 1 monclient: found mon.noname-d 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.777+0000 7f190593a640 1 -- 172.21.15.204:0/949266789 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc009390 con 0x7f19000a2e30 2024-09-15T01:37:57.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f190593a640 1 --2- 172.21.15.204:0/949266789 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19000a2e30 0x7f190011c6b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f18f800bf80 tx=0x7f18f8009730 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f57fa640 1 -- 172.21.15.204:0/949266789 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f18f800cdf0 con 0x7f19000a2e30 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f57fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f57fa640 10 monclient: dump: 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:57.798 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f57fa640 10 monclient: _finish_auth 0 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f57fa640 1 -- 172.21.15.204:0/949266789 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f80065d0 con 0x7f19000a2e30 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f190739d640 5 monclient: authenticate success, global_id 24567 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f190739d640 10 monclient: _renew_subs 2024-09-15T01:37:57.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f190739d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:57.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f190739d640 1 -- 172.21.15.204:0/949266789 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190011e8f0 con 0x7f19000a2e30 2024-09-15T01:37:57.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:57.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f57fa640 1 -- 172.21.15.204:0/949266789 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f18f80153e0 con 0x7f19000a2e30 2024-09-15T01:37:57.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:57.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:37:57.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f57fa640 10 monclient: dump: 2024-09-15T01:37:57.800 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:57.800 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:57.800 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:57.800 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:57.800 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:57.800 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:57.801 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:57.801 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:57.801 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:57.801 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:57.801 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:57.801 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:57.801 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:57.801 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:57.801 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:57.801 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:57.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f18f77fe640 4 set_mon_vals no callback set 2024-09-15T01:37:57.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f190739d640 10 monclient: _renew_subs 2024-09-15T01:37:57.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f190739d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:57.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.778+0000 7f190739d640 1 -- 172.21.15.204:0/949266789 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190009c170 con 0x7f19000a2e30 2024-09-15T01:37:57.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.779+0000 7f18f57fa640 1 -- 172.21.15.204:0/949266789 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f18f8012020 con 0x7f19000a2e30 2024-09-15T01:37:57.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.779+0000 7f190739d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:57.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.779+0000 7f190739d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:57.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.779+0000 7f190739d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:57.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.779+0000 7f190739d640 1 -- 172.21.15.204:0/949266789 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1900070cc0 con 0x7f19000a2e30 2024-09-15T01:37:57.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.779+0000 7f18f57fa640 1 --2- 172.21.15.204:0/949266789 >> v2:172.21.15.204:6812/28210 conn(0x7f18e40397b0 0x7f18e403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:57.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.779+0000 7f18f57fa640 1 -- 172.21.15.204:0/949266789 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f18f804cac0 con 0x7f19000a2e30 2024-09-15T01:37:57.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.779+0000 7f1905139640 1 --2- 172.21.15.204:0/949266789 >> v2:172.21.15.204:6812/28210 conn(0x7f18e40397b0 0x7f18e403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:57.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.779+0000 7f1905139640 10 monclient: get_auth_request con 0x7f18e40397b0 auth_method 0 2024-09-15T01:37:57.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.779+0000 7f1905139640 1 --2- 172.21.15.204:0/949266789 >> v2:172.21.15.204:6812/28210 conn(0x7f18e40397b0 0x7f18e403bc60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f18fc034000 tx=0x7f18fc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:57.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.784+0000 7f18f57fa640 1 -- 172.21.15.204:0/949266789 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f18f8006780 con 0x7f19000a2e30 2024-09-15T01:37:57.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.784+0000 7f18f57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:57.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.784+0000 7f18f57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:57.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.942+0000 7f190739d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h23"}] 2024-09-15T01:37:57.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.942+0000 7f190739d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h23"}] 2024-09-15T01:37:57.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.942+0000 7f190739d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:57.945 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:57.942+0000 7f190739d640 1 -- 172.21.15.204:0/949266789 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h23"} v 0) v1 -- 0x7f190009ddb0 con 0x7f19000a2e30 2024-09-15T01:37:58.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.596+0000 7f18f57fa640 1 -- 172.21.15.204:0/949266789 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h23"}]=0 removed item id -85 name 'h23' from crush map v241) v1 ==== 120+0+0 (secure 0 0 0) 0x7f18f80072f0 con 0x7f19000a2e30 2024-09-15T01:37:58.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.596+0000 7f18f57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h23"}] 2024-09-15T01:37:58.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.596+0000 7f18f57fa640 10 monclient: _finish_command 2 = system:0 removed item id -85 name 'h23' from crush map 2024-09-15T01:37:58.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.695+0000 7f190739d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h23"}] 2024-09-15T01:37:58.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.695+0000 7f190739d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h23"}] 2024-09-15T01:37:58.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.695+0000 7f190739d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:37:58.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.695+0000 7f190739d640 1 -- 172.21.15.204:0/949266789 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h23"} v 0) v1 -- 0x7f19000008d0 con 0x7f19000a2e30 2024-09-15T01:37:58.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.696+0000 7f18f57fa640 1 -- 172.21.15.204:0/949266789 <== mon.6 v2:172.21.15.178:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h23"}]=0 device 'h23' does not appear in the crush map v241) v1 ==== 120+0+0 (secure 0 0 0) 0x7f18f80074d0 con 0x7f19000a2e30 2024-09-15T01:37:58.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.697+0000 7f18f57fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h23"}] 2024-09-15T01:37:58.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.697+0000 7f18f57fa640 10 monclient: _finish_command 3 = system:0 device 'h23' does not appear in the crush map 2024-09-15T01:37:58.699 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h23' does not appear in the crush map 2024-09-15T01:37:58.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 1 -- 172.21.15.204:0/949266789 >> v2:172.21.15.204:6812/28210 conn(0x7f18e40397b0 msgr2=0x7f18e403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:58.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 1 --2- 172.21.15.204:0/949266789 >> v2:172.21.15.204:6812/28210 conn(0x7f18e40397b0 0x7f18e403bc60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f18fc034000 tx=0x7f18fc032000 comp rx=0 tx=0).stop 2024-09-15T01:37:58.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 10 monclient: shutdown 2024-09-15T01:37:58.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:58.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 1 -- 172.21.15.204:0/949266789 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19000a2e30 msgr2=0x7f190011c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:58.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 1 --2- 172.21.15.204:0/949266789 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19000a2e30 0x7f190011c6b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f18f800bf80 tx=0x7f18f8009730 comp rx=0 tx=0).stop 2024-09-15T01:37:58.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 1 -- 172.21.15.204:0/949266789 shutdown_connections 2024-09-15T01:37:58.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 1 --2- 172.21.15.204:0/949266789 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f19000a2e30 0x7f190011c6b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:58.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 1 --2- 172.21.15.204:0/949266789 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f190009ff70 0x7f190012e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:58.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 1 --2- 172.21.15.204:0/949266789 >> v2:172.21.15.204:6812/28210 conn(0x7f18e40397b0 0x7f18e403bc60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:58.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 1 --2- 172.21.15.204:0/949266789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f190009fb80 0x7f190009b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:58.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 1 -- 172.21.15.204:0/949266789 >> 172.21.15.204:0/949266789 conn(0x7f190006e860 msgr2=0x7f1900131970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:58.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 1 -- 172.21.15.204:0/949266789 shutdown_connections 2024-09-15T01:37:58.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.699+0000 7f190739d640 1 -- 172.21.15.204:0/949266789 wait complete. 2024-09-15T01:37:58.710 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:58.710 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h24 2024-09-15T01:37:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.782+0000 7fedbf59d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.782+0000 7fedbf59d640 10 monclient: build_initial_monmap 2024-09-15T01:37:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.782+0000 7fedbf59d640 10 monclient: monmap: 2024-09-15T01:37:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:58.785 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:58.784140+0000 2024-09-15T01:37:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:58.784140+0000 2024-09-15T01:37:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:58.786 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:58.787 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:58.788 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 1 Processor -- start 2024-09-15T01:37:58.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 1 -- start start 2024-09-15T01:37:58.789 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 10 monclient: init 2024-09-15T01:37:58.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:58.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 10 monclient: _add_conns ranks=[6,8,2,5,3,1,4,10,7,9,0] 2024-09-15T01:37:58.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedb80a2e30 0x7fedb8132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:58.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 10 monclient(hunting): picked mon.noname-d con 0x7fedb80a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:37:58.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb809fb80 0x7fedb809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:58.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 10 monclient(hunting): picked mon.noname-e con 0x7fedb809fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:58.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb80a0a40 0x7fedb80a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:58.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 10 monclient(hunting): picked mon.noname-b con 0x7fedb80a0a40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:58.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:58.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb809e6c0 con 0x7fedb80a0a40 2024-09-15T01:37:58.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:58.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80a04b0 con 0x7fedb80a2e30 2024-09-15T01:37:58.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:58.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80a3200 con 0x7fedb809fb80 2024-09-15T01:37:58.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:58.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.783+0000 7fedbf59d640 10 monclient(hunting): authenticate will time out at 1094.426910s 2024-09-15T01:37:58.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbdb3a640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb80a0a40 msgr2=0x7fedb80a0e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:58.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbcb38640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb809fb80 msgr2=0x7fedb809ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:58.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbcb38640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb809fb80 0x7fedb809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:58.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbdb3a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb80a0a40 0x7fedb80a0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:58.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedb80a2e30 0x7fedb8132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:58.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedb80a2e30 0x7fedb8132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:42932/0 (socket says 172.21.15.204:42932) 2024-09-15T01:37:58.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 1 -- 172.21.15.204:0/1727682977 learned_addr learned my addr 172.21.15.204:0/1727682977 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:58.808 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 10 monclient(hunting): get_auth_request con 0x7fedb80a2e30 auth_method 0 2024-09-15T01:37:58.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:58.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:58.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:58.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:58.809 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:58.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:58.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 10 monclient(hunting): handle_auth_done global_id 24576 payload 274 2024-09-15T01:37:58.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 1 -- 172.21.15.204:0/1727682977 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb809fb80 msgr2=0x7fedb809ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:58.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 1 --2- 172.21.15.204:0/1727682977 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb809fb80 0x7fedb809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:58.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 1 -- 172.21.15.204:0/1727682977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb80a0a40 msgr2=0x7fedb80a0e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:58.810 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 1 --2- 172.21.15.204:0/1727682977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb80a0a40 0x7fedb80a0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:58.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 10 monclient: _finish_hunting 0 2024-09-15T01:37:58.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 1 monclient: found mon.noname-d 2024-09-15T01:37:58.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:58.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:37:58.811 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.784+0000 7fedbd339640 1 -- 172.21.15.204:0/1727682977 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb804b720 con 0x7fedb80a2e30 2024-09-15T01:37:58.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedbd339640 1 --2- 172.21.15.204:0/1727682977 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedb80a2e30 0x7fedb8132a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009940 tx=0x7fedb40091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e9fe86bb6288a6db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:58.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedaffff640 1 -- 172.21.15.204:0/1727682977 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedb402acb0 con 0x7fedb80a2e30 2024-09-15T01:37:58.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedaffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:58.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedaffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:37:58.812 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedaffff640 10 monclient: dump: 2024-09-15T01:37:58.812 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:58.813 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:58.813 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:58.813 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:58.813 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:58.813 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:58.813 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:58.813 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:58.813 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:58.813 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:58.813 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:58.813 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedaffff640 10 monclient: _finish_auth 0 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedaffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedaffff640 1 -- 172.21.15.204:0/1727682977 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb402ae30 con 0x7fedb80a2e30 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedbf59d640 5 monclient: authenticate success, global_id 24576 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedbf59d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedaffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedbf59d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedaffff640 1 -- 172.21.15.204:0/1727682977 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedb4031ca0 con 0x7fedb80a2e30 2024-09-15T01:37:58.814 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedaffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedbf59d640 4 set_mon_vals no callback set 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedaffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.785+0000 7fedaffff640 10 monclient: dump: 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:58.815 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:58.816 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:58.816 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:58.816 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:58.816 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:58.816 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:58.816 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:58.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 10 monclient: shutdown 2024-09-15T01:37:58.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:58.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 1 -- 172.21.15.204:0/1727682977 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedb80a2e30 msgr2=0x7fedb8132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:58.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 1 --2- 172.21.15.204:0/1727682977 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedb80a2e30 0x7fedb8132a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009940 tx=0x7fedb40091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:58.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 1 -- 172.21.15.204:0/1727682977 shutdown_connections 2024-09-15T01:37:58.816 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 1 --2- 172.21.15.204:0/1727682977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb80a0a40 0x7fedb80a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 1 --2- 172.21.15.204:0/1727682977 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb809fb80 0x7fedb809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 1 --2- 172.21.15.204:0/1727682977 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fedb80a2e30 0x7fedb8132a60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 1 -- 172.21.15.204:0/1727682977 >> 172.21.15.204:0/1727682977 conn(0x7fedb806e860 msgr2=0x7fedb806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 1 -- 172.21.15.204:0/1727682977 shutdown_connections 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 1 -- 172.21.15.204:0/1727682977 wait complete. 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 10 monclient: build_initial_monmap 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.786+0000 7fedbf59d640 10 monclient: monmap: 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:58.817 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 1 Processor -- start 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 1 -- start start 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 10 monclient: init 2024-09-15T01:37:58.818 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 10 monclient: _add_conns ranks=[2,1,8,5,7,6,4,0,3] 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb809fb80 0x7fedb8141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 10 monclient(hunting): picked mon.noname-b con 0x7fedb809fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb80a0a40 0x7fedb8132650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 10 monclient(hunting): picked mon.noname-f con 0x7fedb80a0a40 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb80a2e30 0x7fedb811f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 10 monclient(hunting): picked mon.noname-e con 0x7fedb80a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb8122830 con 0x7fedb80a0a40 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb8122d60 con 0x7fedb809fb80 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fedb8120d50 con 0x7fedb80a2e30 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:58.819 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbf59d640 10 monclient(hunting): authenticate will time out at 1094.430774s 2024-09-15T01:37:58.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbd339640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb809fb80 msgr2=0x7fedb8141cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:58.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbd339640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb809fb80 0x7fedb8141cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:58.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbcb38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb80a0a40 0x7fedb8132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:58.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbdb3a640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb80a2e30 msgr2=0x7fedb811f200 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:58.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbdb3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb80a2e30 0x7fedb811f200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:58.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbcb38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb80a0a40 0x7fedb8132650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48236/0 (socket says 172.21.15.204:48236) 2024-09-15T01:37:58.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbcb38640 1 -- 172.21.15.204:0/472753953 learned_addr learned my addr 172.21.15.204:0/472753953 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:58.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbcb38640 10 monclient(hunting): get_auth_request con 0x7fedb80a0a40 auth_method 0 2024-09-15T01:37:58.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbcb38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:58.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbcb38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:58.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbcb38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:58.820 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbcb38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:58.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.787+0000 7fedbcb38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:58.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbcb38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:58.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbcb38640 10 monclient(hunting): handle_auth_done global_id 7372 payload 824 2024-09-15T01:37:58.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbcb38640 1 -- 172.21.15.204:0/472753953 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb80a2e30 msgr2=0x7fedb811f200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:58.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbcb38640 1 --2- 172.21.15.204:0/472753953 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb80a2e30 0x7fedb811f200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:58.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbcb38640 1 -- 172.21.15.204:0/472753953 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb809fb80 msgr2=0x7fedb8141cc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:37:58.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbcb38640 1 --2- 172.21.15.204:0/472753953 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb809fb80 0x7fedb8141cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:58.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbcb38640 10 monclient: _finish_hunting 0 2024-09-15T01:37:58.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbcb38640 1 monclient: found mon.noname-f 2024-09-15T01:37:58.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbcb38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:58.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbcb38640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:58.821 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbcb38640 1 -- 172.21.15.204:0/472753953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb4009450 con 0x7fedb80a0a40 2024-09-15T01:37:58.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbcb38640 1 --2- 172.21.15.204:0/472753953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb80a0a40 0x7fedb8132650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feda800b590 tx=0x7feda80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:58.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedad7fa640 1 -- 172.21.15.204:0/472753953 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feda8009250 con 0x7fedb80a0a40 2024-09-15T01:37:58.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedad7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:58.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedad7fa640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:58.822 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedad7fa640 10 monclient: dump: 2024-09-15T01:37:58.822 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:58.822 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:58.822 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:58.822 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:58.822 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:58.822 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:58.822 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedad7fa640 10 monclient: _finish_auth 0 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedad7fa640 1 -- 172.21.15.204:0/472753953 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda80050f0 con 0x7fedb80a0a40 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbf59d640 5 monclient: authenticate success, global_id 7372 2024-09-15T01:37:58.823 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbf59d640 10 monclient: _renew_subs 2024-09-15T01:37:58.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbf59d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:58.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbf59d640 1 -- 172.21.15.204:0/472753953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb8142200 con 0x7fedb80a0a40 2024-09-15T01:37:58.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedad7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:58.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedad7fa640 1 -- 172.21.15.204:0/472753953 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feda8005720 con 0x7fedb80a0a40 2024-09-15T01:37:58.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedad7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:58.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedad7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:58.824 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedad7fa640 10 monclient: dump: 2024-09-15T01:37:58.824 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:58.824 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:58.824 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:58.824 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:58.824 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:58.825 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:58.825 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:58.825 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:58.825 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:58.825 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:58.825 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:58.825 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:58.825 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:58.825 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:58.825 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:58.825 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:58.825 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedaf7fe640 4 set_mon_vals no callback set 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbf59d640 10 monclient: _renew_subs 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbf59d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.788+0000 7fedbf59d640 1 -- 172.21.15.204:0/472753953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb81218b0 con 0x7fedb80a0a40 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.789+0000 7fedad7fa640 1 -- 172.21.15.204:0/472753953 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7feda80052a0 con 0x7fedb80a0a40 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.789+0000 7fedbf59d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.789+0000 7fedbf59d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.789+0000 7fedbf59d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.789+0000 7fedbf59d640 1 -- 172.21.15.204:0/472753953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb8070cc0 con 0x7fedb80a0a40 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.789+0000 7fedad7fa640 1 --2- 172.21.15.204:0/472753953 >> v2:172.21.15.204:6812/28210 conn(0x7fed9c0397b0 0x7fed9c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.789+0000 7fedad7fa640 1 -- 172.21.15.204:0/472753953 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 9254+0+0 (secure 0 0 0) 0x7feda804c100 con 0x7fedb80a0a40 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.789+0000 7fedbd339640 1 --2- 172.21.15.204:0/472753953 >> v2:172.21.15.204:6812/28210 conn(0x7fed9c0397b0 0x7fed9c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.789+0000 7fedbd339640 10 monclient: get_auth_request con 0x7fed9c0397b0 auth_method 0 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.790+0000 7fedbd339640 1 --2- 172.21.15.204:0/472753953 >> v2:172.21.15.204:6812/28210 conn(0x7fed9c0397b0 0x7fed9c03bc60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fedb4034000 tx=0x7fedb4032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:58.826 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.794+0000 7fedad7fa640 1 -- 172.21.15.204:0/472753953 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7feda8005550 con 0x7fedb80a0a40 2024-09-15T01:37:58.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.794+0000 7fedad7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:37:58.827 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.794+0000 7fedad7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:37:58.955 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.953+0000 7fedbf59d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h24"}] 2024-09-15T01:37:58.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.953+0000 7fedbf59d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h24"}] 2024-09-15T01:37:58.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.953+0000 7fedbf59d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:58.956 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:58.953+0000 7fedbf59d640 1 -- 172.21.15.204:0/472753953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h24"} v 0) v1 -- 0x7fedb80a1f10 con 0x7fedb80a0a40 2024-09-15T01:37:59.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.600+0000 7fedad7fa640 1 -- 172.21.15.204:0/472753953 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h24"}]=0 removed item id -88 name 'h24' from crush map v242) v1 ==== 120+0+0 (secure 0 0 0) 0x7feda80193c0 con 0x7fedb80a0a40 2024-09-15T01:37:59.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.600+0000 7fedad7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h24"}] 2024-09-15T01:37:59.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.600+0000 7fedad7fa640 10 monclient: _finish_command 2 = system:0 removed item id -88 name 'h24' from crush map 2024-09-15T01:37:59.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.699+0000 7fedbf59d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h24"}] 2024-09-15T01:37:59.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.699+0000 7fedbf59d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h24"}] 2024-09-15T01:37:59.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.699+0000 7fedbf59d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:37:59.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.699+0000 7fedbf59d640 1 -- 172.21.15.204:0/472753953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h24"} v 0) v1 -- 0x7fedb8121ad0 con 0x7fedb80a0a40 2024-09-15T01:37:59.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.700+0000 7fedad7fa640 1 -- 172.21.15.204:0/472753953 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h24"}]=0 device 'h24' does not appear in the crush map v242) v1 ==== 120+0+0 (secure 0 0 0) 0x7feda8018b50 con 0x7fedb80a0a40 2024-09-15T01:37:59.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.700+0000 7fedad7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h24"}] 2024-09-15T01:37:59.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.700+0000 7fedad7fa640 10 monclient: _finish_command 3 = system:0 device 'h24' does not appear in the crush map 2024-09-15T01:37:59.703 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h24' does not appear in the crush map 2024-09-15T01:37:59.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 1 -- 172.21.15.204:0/472753953 >> v2:172.21.15.204:6812/28210 conn(0x7fed9c0397b0 msgr2=0x7fed9c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:59.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 1 --2- 172.21.15.204:0/472753953 >> v2:172.21.15.204:6812/28210 conn(0x7fed9c0397b0 0x7fed9c03bc60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fedb4034000 tx=0x7fedb4032000 comp rx=0 tx=0).stop 2024-09-15T01:37:59.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 10 monclient: shutdown 2024-09-15T01:37:59.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:59.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 1 -- 172.21.15.204:0/472753953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb80a0a40 msgr2=0x7fedb8132650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:59.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 1 --2- 172.21.15.204:0/472753953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb80a0a40 0x7fedb8132650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feda800b590 tx=0x7feda80090f0 comp rx=0 tx=0).stop 2024-09-15T01:37:59.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 1 -- 172.21.15.204:0/472753953 shutdown_connections 2024-09-15T01:37:59.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 1 --2- 172.21.15.204:0/472753953 >> v2:172.21.15.204:6812/28210 conn(0x7fed9c0397b0 0x7fed9c03bc60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:59.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 1 --2- 172.21.15.204:0/472753953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedb80a0a40 0x7fedb8132650 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:59.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 1 --2- 172.21.15.204:0/472753953 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fedb80a2e30 0x7fedb811f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:59.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 1 --2- 172.21.15.204:0/472753953 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fedb809fb80 0x7fedb8141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:59.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 1 -- 172.21.15.204:0/472753953 >> 172.21.15.204:0/472753953 conn(0x7fedb806e860 msgr2=0x7fedb80969e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:59.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 1 -- 172.21.15.204:0/472753953 shutdown_connections 2024-09-15T01:37:59.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.702+0000 7fedbf59d640 1 -- 172.21.15.204:0/472753953 wait complete. 2024-09-15T01:37:59.713 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:37:59.713 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h25 2024-09-15T01:37:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.785+0000 7f5fa259d640 10 monclient: get_monmap_and_config 2024-09-15T01:37:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.785+0000 7f5fa259d640 10 monclient: build_initial_monmap 2024-09-15T01:37:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.785+0000 7f5fa259d640 10 monclient: monmap: 2024-09-15T01:37:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:37:59.787167+0000 2024-09-15T01:37:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:37:59.787167+0000 2024-09-15T01:37:59.788 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:37:59.789 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:59.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.786+0000 7f5fa259d640 1 Processor -- start 2024-09-15T01:37:59.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.786+0000 7f5fa259d640 1 -- start start 2024-09-15T01:37:59.794 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.786+0000 7f5fa259d640 10 monclient: init 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.786+0000 7f5fa259d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.786+0000 7f5fa259d640 10 monclient: _add_conns ranks=[8,10,2,6,1,4,0,9,7,5,3] 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.786+0000 7f5fa259d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 0x7f5f9c132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.786+0000 7f5fa259d640 10 monclient(hunting): picked mon.noname-e con 0x7f5f9c0a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa259d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5f9c09fb80 0x7f5f9c09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa259d640 10 monclient(hunting): picked mon.noname-f con 0x7f5f9c09fb80 addr v2:172.21.15.204:3300/0 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa259d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c0a0a40 0x7f5f9c0a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa259d640 10 monclient(hunting): picked mon.noname-b con 0x7f5f9c0a0a40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa259d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa259d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9c09e6c0 con 0x7f5f9c09fb80 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa259d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa259d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0a04b0 con 0x7f5f9c0a0a40 2024-09-15T01:37:59.795 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa259d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:59.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa259d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0a3200 con 0x7f5f9c0a2e30 2024-09-15T01:37:59.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa259d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:59.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa259d640 10 monclient(hunting): authenticate will time out at 1095.430059s 2024-09-15T01:37:59.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5f9c09fb80 0x7f5f9c09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:37:59.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa0b3a640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c0a0a40 msgr2=0x7f5f9c0a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:59.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9bfff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 msgr2=0x7f5f9c132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:59.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5fa0b3a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c0a0a40 0x7f5f9c0a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:59.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5f9c09fb80 0x7f5f9c09ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48238/0 (socket says 172.21.15.204:48238) 2024-09-15T01:37:59.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 1 -- 172.21.15.204:0/475950850 learned_addr learned my addr 172.21.15.204:0/475950850 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 10 monclient(hunting): get_auth_request con 0x7f5f9c09fb80 auth_method 0 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9bfff640 1 --2- 172.21.15.204:0/475950850 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 0x7f5f9c132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 10 monclient(hunting): handle_auth_done global_id 7381 payload 274 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 1 -- 172.21.15.204:0/475950850 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 msgr2=0x7f5f9c132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 1 --2- 172.21.15.204:0/475950850 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 0x7f5f9c132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 1 -- 172.21.15.204:0/475950850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c0a0a40 msgr2=0x7f5f9c0a0ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:37:59.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 1 --2- 172.21.15.204:0/475950850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c0a0a40 0x7f5f9c0a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 1 monclient: found mon.noname-f 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.787+0000 7f5f9b7fe640 1 -- 172.21.15.204:0/475950850 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c04b720 con 0x7f5f9c09fb80 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9b7fe640 1 --2- 172.21.15.204:0/475950850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9c09fb80 0x7f5f9c09ff70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009910 tx=0x7f5f900091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed2cc21e7682ef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9affd640 1 -- 172.21.15.204:0/475950850 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5f9002acd0 con 0x7f5f9c09fb80 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9affd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9affd640 10 monclient: dump: 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:59.798 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:59.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9affd640 10 monclient: _finish_auth 0 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 5 monclient: authenticate success, global_id 7381 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9affd640 1 -- 172.21.15.204:0/475950850 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9002ae50 con 0x7f5f9c09fb80 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 10 monclient: get_monmap_and_config success 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 4 set_mon_vals no callback set 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9affd640 1 -- 172.21.15.204:0/475950850 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5f90031ca0 con 0x7f5f9c09fb80 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5f9affd640 10 monclient: dump: 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:37:59.800 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:37:59.801 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 10 monclient: shutdown 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 1 -- 172.21.15.204:0/475950850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9c09fb80 msgr2=0x7f5f9c09ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 1 --2- 172.21.15.204:0/475950850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9c09fb80 0x7f5f9c09ff70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009910 tx=0x7f5f900091d0 comp rx=0 tx=0).stop 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 1 -- 172.21.15.204:0/475950850 shutdown_connections 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 1 --2- 172.21.15.204:0/475950850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9c09fb80 0x7f5f9c09ff70 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 1 --2- 172.21.15.204:0/475950850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c0a0a40 0x7f5f9c0a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 1 --2- 172.21.15.204:0/475950850 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 0x7f5f9c132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 1 -- 172.21.15.204:0/475950850 >> 172.21.15.204:0/475950850 conn(0x7f5f9c06e860 msgr2=0x7f5f9c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 1 -- 172.21.15.204:0/475950850 shutdown_connections 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.788+0000 7f5fa259d640 1 -- 172.21.15.204:0/475950850 wait complete. 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.789+0000 7f5fa259d640 10 monclient: build_initial_monmap 2024-09-15T01:37:59.802 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.789+0000 7f5fa259d640 10 monclient: monmap: 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:37:59.803 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.789+0000 7f5fa259d640 1 Processor -- start 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.789+0000 7f5fa259d640 1 -- start start 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.789+0000 7f5fa259d640 10 monclient: init 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 10 monclient: _reopen_session rank -1 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 10 monclient: _add_conns ranks=[2,0,8,5,4,3,7,1,6] 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c09fb80 0x7f5f9c141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 10 monclient(hunting): picked mon.noname-b con 0x7f5f9c09fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f9c0a0a40 0x7f5f9c131b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 10 monclient(hunting): picked mon.noname-a con 0x7f5f9c0a0a40 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:37:59.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 0x7f5f9c11e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 10 monclient(hunting): picked mon.noname-e con 0x7f5f9c0a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c120210 con 0x7f5f9c0a0a40 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c09ca40 con 0x7f5f9c09fb80 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c09d650 con 0x7f5f9c0a2e30 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 10 monclient(hunting): _renew_subs 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa259d640 10 monclient(hunting): authenticate will time out at 1095.433388s 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5f9bfff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c09fb80 msgr2=0x7f5f9c141d30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa0b3a640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 msgr2=0x7f5f9c11e640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5f9b7fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f9c0a0a40 msgr2=0x7f5f9c131b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:59.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5f9bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c09fb80 0x7f5f9c141d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:59.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5fa0b3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 0x7f5f9c11e640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:59.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.790+0000 7f5f9b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f9c0a0a40 0x7f5f9c131b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:37:59.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.991+0000 7f5f9bfff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c09fb80 msgr2=0x7f5f9c141d30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:37:59.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.991+0000 7f5f9b7fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f9c0a0a40 msgr2=0x7f5f9c131b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:37:59.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.991+0000 7f5f9bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c09fb80 0x7f5f9c141d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:37:59.993 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.991+0000 7f5fa0b3a640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 msgr2=0x7f5f9c11e640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:37:59.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.991+0000 7f5f9b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f9c0a0a40 0x7f5f9c131b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:37:59.994 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:37:59.991+0000 7f5fa0b3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 0x7f5f9c11e640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:00.394 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:00.392+0000 7f5f9bfff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c09fb80 msgr2=0x7f5f9c141d30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:00.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:00.392+0000 7f5f9b7fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f9c0a0a40 msgr2=0x7f5f9c131b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:00.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:00.392+0000 7f5fa0b3a640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 msgr2=0x7f5f9c11e640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:00.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:00.392+0000 7f5f9bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c09fb80 0x7f5f9c141d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:00.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:00.392+0000 7f5fa0b3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 0x7f5f9c11e640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:00.395 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:00.392+0000 7f5f9b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f9c0a0a40 0x7f5f9c131b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:01.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:01.193+0000 7f5f9bfff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c09fb80 msgr2=0x7f5f9c141d30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:01.195 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:01.193+0000 7f5fa0b3a640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 msgr2=0x7f5f9c11e640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:01.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:01.193+0000 7f5f9b7fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f9c0a0a40 msgr2=0x7f5f9c131b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:01.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:01.193+0000 7f5f9bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c09fb80 0x7f5f9c141d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:01.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:01.193+0000 7f5fa0b3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 0x7f5f9c11e640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:01.196 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:01.193+0000 7f5f9b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f9c0a0a40 0x7f5f9c131b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:02.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 10 monclient(hunting): tick 2024-09-15T01:38:02.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 monclient(hunting): continuing hunt 2024-09-15T01:38:02.796 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 msgr2=0x7f5f9c11e640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5f9c0a2e30 0x7f5f9c11e640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c09fb80 msgr2=0x7f5f9c141d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f9c09fb80 0x7f5f9c141d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f9c0a0a40 msgr2=0x7f5f9c131b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f9c0a0a40 0x7f5f9c131b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 10 monclient: _add_conns ranks=[6,4,3,5,1,7] 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5f7c003800 0x7f5f7c005c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f5f7c003800 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5f7c006200 0x7f5f7c008660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f5f7c006200 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f7c008d50 0x7f5f7c00b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:02.797 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f5f7c008d50 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c120210 con 0x7f5f7c008d50 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c09ca40 con 0x7f5f7c006200 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c09d650 con 0x7f5f7c003800 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f82ffd640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f9b7fe640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5f7c006200 msgr2=0x7f5f7c008660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f9b7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5f7c006200 0x7f5f7c008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5fa0b3a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f7c008d50 0x7f5f7c00b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5fa0b3a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f7c008d50 0x7f5f7c00b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:51244/0 (socket says 172.21.15.204:51244) 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5fa0b3a640 1 -- 172.21.15.204:0/1781591085 learned_addr learned my addr 172.21.15.204:0/1781591085 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:02.798 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5fa0b3a640 10 monclient(hunting): get_auth_request con 0x7f5f7c008d50 auth_method 0 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5fa0b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5fa0b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5fa0b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5fa0b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5fa0b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f9bfff640 1 --2- 172.21.15.204:0/1781591085 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5f7c003800 0x7f5f7c005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5fa0b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f9bfff640 10 monclient(hunting): get_auth_request con 0x7f5f7c003800 auth_method 0 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.790+0000 7f5f9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:02.799 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:02.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:02.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa0b3a640 10 monclient(hunting): handle_auth_done global_id 15636 payload 824 2024-09-15T01:38:02.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa0b3a640 1 -- 172.21.15.204:0/1781591085 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5f7c003800 msgr2=0x7f5f7c005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:02.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa0b3a640 1 --2- 172.21.15.204:0/1781591085 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5f7c003800 0x7f5f7c005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:02.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa0b3a640 1 -- 172.21.15.204:0/1781591085 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5f7c006200 msgr2=0x7f5f7c008660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:02.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa0b3a640 1 --2- 172.21.15.204:0/1781591085 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5f7c006200 0x7f5f7c008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:02.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa0b3a640 10 monclient: _finish_hunting 0 2024-09-15T01:38:02.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa0b3a640 1 monclient: found mon.noname-g 2024-09-15T01:38:02.800 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa0b3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:02.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa0b3a640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:02.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa0b3a640 1 -- 172.21.15.204:0/1781591085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f90009450 con 0x7f5f7c008d50 2024-09-15T01:38:02.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa0b3a640 1 -- 172.21.15.204:0/1781591085 reap_dead start 2024-09-15T01:38:02.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa0b3a640 1 --2- 172.21.15.204:0/1781591085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f7c008d50 0x7f5f7c00b200 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400bb80 tx=0x7f5f94009780 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:02.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f83fff640 1 -- 172.21.15.204:0/1781591085 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5f9400ce70 con 0x7f5f7c008d50 2024-09-15T01:38:02.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f83fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:02.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f83fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:02.801 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f83fff640 10 monclient: dump: 2024-09-15T01:38:02.801 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:02.801 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:02.802 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:02.802 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:02.802 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:02.802 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:02.802 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:02.802 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:02.802 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:02.802 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:02.802 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:02.802 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f83fff640 10 monclient: _finish_auth 0 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f83fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f83fff640 1 -- 172.21.15.204:0/1781591085 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f94006150 con 0x7f5f7c008d50 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa259d640 5 monclient: authenticate success, global_id 15636 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f83fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f83fff640 1 -- 172.21.15.204:0/1781591085 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5f94006740 con 0x7f5f7c008d50 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa259d640 10 monclient: _renew_subs 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa259d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa259d640 1 -- 172.21.15.204:0/1781591085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9c142bf0 con 0x7f5f7c008d50 2024-09-15T01:38:02.803 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f83fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:02.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f83fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:02.804 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f83fff640 10 monclient: dump: 2024-09-15T01:38:02.804 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:02.804 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:02.804 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:02.804 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:02.804 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:02.804 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:02.804 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:02.804 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:02.804 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:02.804 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5f9a7fc640 4 set_mon_vals no callback set 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa259d640 10 monclient: _renew_subs 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa259d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.791+0000 7f5fa259d640 1 -- 172.21.15.204:0/1781591085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c121f90 con 0x7f5f7c008d50 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.792+0000 7f5f83fff640 1 -- 172.21.15.204:0/1781591085 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5f94012020 con 0x7f5f7c008d50 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.792+0000 7f5f83fff640 1 --2- 172.21.15.204:0/1781591085 >> v2:172.21.15.204:6812/28210 conn(0x7f5f88039720 0x7f5f8803bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.793+0000 7f5f83fff640 1 -- 172.21.15.204:0/1781591085 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 9041+0+0 (secure 0 0 0) 0x7f5f9404c180 con 0x7f5f7c008d50 2024-09-15T01:38:02.805 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.793+0000 7f5f9bfff640 1 --2- 172.21.15.204:0/1781591085 >> v2:172.21.15.204:6812/28210 conn(0x7f5f88039720 0x7f5f8803bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:02.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.793+0000 7f5f9bfff640 10 monclient: get_auth_request con 0x7f5f88039720 auth_method 0 2024-09-15T01:38:02.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.793+0000 7f5fa259d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:02.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.793+0000 7f5fa259d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:02.806 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.793+0000 7f5fa259d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:02.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.793+0000 7f5fa259d640 1 -- 172.21.15.204:0/1781591085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c0a3fc0 con 0x7f5f7c008d50 2024-09-15T01:38:02.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.793+0000 7f5f9bfff640 1 --2- 172.21.15.204:0/1781591085 >> v2:172.21.15.204:6812/28210 conn(0x7f5f88039720 0x7f5f8803bbd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c003bb0 tx=0x7f5f8c009760 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:02.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.798+0000 7f5f83fff640 1 -- 172.21.15.204:0/1781591085 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5f9c0a3fc0 con 0x7f5f7c008d50 2024-09-15T01:38:02.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.798+0000 7f5f83fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:02.807 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.798+0000 7f5f83fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:02.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.962+0000 7f5fa259d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h25"}] 2024-09-15T01:38:02.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.963+0000 7f5fa259d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h25"}] 2024-09-15T01:38:02.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.963+0000 7f5fa259d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:02.965 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:02.963+0000 7f5fa259d640 1 -- 172.21.15.204:0/1781591085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h25"} v 0) v1 -- 0x7f5f9c0a3fc0 con 0x7f5f7c008d50 2024-09-15T01:38:03.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.497+0000 7f5f83fff640 1 -- 172.21.15.204:0/1781591085 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h25"}]=0 removed item id -91 name 'h25' from crush map v244) v1 ==== 120+0+0 (secure 0 0 0) 0x7f5f9c0a3fc0 con 0x7f5f7c008d50 2024-09-15T01:38:03.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.497+0000 7f5f83fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h25"}] 2024-09-15T01:38:03.500 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.497+0000 7f5f83fff640 10 monclient: _finish_command 2 = system:0 removed item id -91 name 'h25' from crush map 2024-09-15T01:38:03.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.596+0000 7f5fa259d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h25"}] 2024-09-15T01:38:03.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.596+0000 7f5fa259d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h25"}] 2024-09-15T01:38:03.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.596+0000 7f5fa259d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:03.599 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.596+0000 7f5fa259d640 1 -- 172.21.15.204:0/1781591085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h25"} v 0) v1 -- 0x7f5f9c0a3fc0 con 0x7f5f7c008d50 2024-09-15T01:38:03.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.597+0000 7f5f83fff640 1 -- 172.21.15.204:0/1781591085 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h25"}]=0 device 'h25' does not appear in the crush map v244) v1 ==== 120+0+0 (secure 0 0 0) 0x7f5f9c0a3fc0 con 0x7f5f7c008d50 2024-09-15T01:38:03.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.597+0000 7f5f83fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h25"}] 2024-09-15T01:38:03.600 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.597+0000 7f5f83fff640 10 monclient: _finish_command 3 = system:0 device 'h25' does not appear in the crush map 2024-09-15T01:38:03.600 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h25' does not appear in the crush map 2024-09-15T01:38:03.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.599+0000 7f5fa259d640 1 -- 172.21.15.204:0/1781591085 >> v2:172.21.15.204:6812/28210 conn(0x7f5f88039720 msgr2=0x7f5f8803bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:03.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.599+0000 7f5fa259d640 1 --2- 172.21.15.204:0/1781591085 >> v2:172.21.15.204:6812/28210 conn(0x7f5f88039720 0x7f5f8803bbd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c003bb0 tx=0x7f5f8c009760 comp rx=0 tx=0).stop 2024-09-15T01:38:03.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.599+0000 7f5fa259d640 10 monclient: shutdown 2024-09-15T01:38:03.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.599+0000 7f5fa259d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:03.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.599+0000 7f5fa259d640 1 -- 172.21.15.204:0/1781591085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f7c008d50 msgr2=0x7f5f7c00b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:03.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.599+0000 7f5fa259d640 1 --2- 172.21.15.204:0/1781591085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f7c008d50 0x7f5f7c00b200 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400bb80 tx=0x7f5f94009780 comp rx=0 tx=0).stop 2024-09-15T01:38:03.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.599+0000 7f5fa259d640 1 -- 172.21.15.204:0/1781591085 shutdown_connections 2024-09-15T01:38:03.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.599+0000 7f5fa259d640 1 --2- 172.21.15.204:0/1781591085 >> v2:172.21.15.204:6812/28210 conn(0x7f5f88039720 0x7f5f8803bbd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:03.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.599+0000 7f5fa259d640 1 --2- 172.21.15.204:0/1781591085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f7c008d50 0x7f5f7c00b200 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:03.602 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.599+0000 7f5fa259d640 1 -- 172.21.15.204:0/1781591085 >> 172.21.15.204:0/1781591085 conn(0x7f5f9c06e860 msgr2=0x7f5f9c0969e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:03.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.599+0000 7f5fa259d640 1 -- 172.21.15.204:0/1781591085 shutdown_connections 2024-09-15T01:38:03.603 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.599+0000 7f5fa259d640 1 -- 172.21.15.204:0/1781591085 wait complete. 2024-09-15T01:38:03.611 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:38:03.611 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h26 2024-09-15T01:38:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.682+0000 7fd3a008b640 10 monclient: get_monmap_and_config 2024-09-15T01:38:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.682+0000 7fd3a008b640 10 monclient: build_initial_monmap 2024-09-15T01:38:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.682+0000 7fd3a008b640 10 monclient: monmap: 2024-09-15T01:38:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:03.683828+0000 2024-09-15T01:38:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:03.683828+0000 2024-09-15T01:38:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:03.685 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:03.686 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:03.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.682+0000 7fd3a008b640 1 Processor -- start 2024-09-15T01:38:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 1 -- start start 2024-09-15T01:38:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 10 monclient: init 2024-09-15T01:38:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 10 monclient: _add_conns ranks=[10,2,1,9,0,4,7,5,8,6,3] 2024-09-15T01:38:03.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd3980a2e30 0x7fd398132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 10 monclient(hunting): picked mon.noname-f con 0x7fd3980a2e30 addr v2:172.21.15.204:3300/0 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd39809fb80 0x7fd39809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 10 monclient(hunting): picked mon.noname-b con 0x7fd39809fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd3980a0a40 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39804b720 con 0x7fd3980a0a40 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39809e6c0 con 0x7fd3980a2e30 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980a04b0 con 0x7fd39809fb80 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd3a008b640 10 monclient(hunting): authenticate will time out at 1099.326657s 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd39de27640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd3980a2e30 0x7fd398132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:03.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd39e628640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fd3980a0a40 0x7fd3980a0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:46294/0 (socket says 172.21.15.204:46294) 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd39e628640 1 -- 172.21.15.204:0/2577210941 learned_addr learned my addr 172.21.15.204:0/2577210941 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd39d626640 1 -- 172.21.15.204:0/2577210941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd39809fb80 msgr2=0x7fd39809ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd39de27640 10 monclient(hunting): get_auth_request con 0x7fd3980a2e30 auth_method 0 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd39de27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd39de27640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd39de27640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd39d626640 1 --2- 172.21.15.204:0/2577210941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd39809fb80 0x7fd39809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd39de27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd39de27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.683+0000 7fd39de27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39de27640 10 monclient(hunting): handle_auth_done global_id 7390 payload 274 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39de27640 1 -- 172.21.15.204:0/2577210941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd39809fb80 msgr2=0x7fd39809ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39de27640 1 --2- 172.21.15.204:0/2577210941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd39809fb80 0x7fd39809ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:03.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39de27640 1 -- 172.21.15.204:0/2577210941 >> v1:172.21.15.204:6789/0 conn(0x7fd3980a0a40 legacy=0x7fd3980a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39de27640 10 monclient: _finish_hunting 0 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39de27640 1 monclient: found mon.noname-f 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39de27640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 1 -- 172.21.15.204:0/2577210941 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389201073 0 0) 0x7fd39804b720 con 0x7fd3980a0a40 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39de27640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39de27640 1 -- 172.21.15.204:0/2577210941 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3980a3200 con 0x7fd3980a2e30 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39de27640 1 --2- 172.21.15.204:0/2577210941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3980a2e30 0x7fd398132a60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd394009850 tx=0x7fd3940090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbde0c600e88155b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 1 -- 172.21.15.204:0/2577210941 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd39402ac90 con 0x7fd3980a2e30 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 10 monclient: dump: 2024-09-15T01:38:03.697 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:03.698 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 10 monclient: _finish_auth 0 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd3a008b640 5 monclient: authenticate success, global_id 7390 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd3a008b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 1 -- 172.21.15.204:0/2577210941 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39402ae10 con 0x7fd3980a2e30 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd3a008b640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd3a008b640 4 set_mon_vals no callback set 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 1 -- 172.21.15.204:0/2577210941 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd394031ca0 con 0x7fd3980a2e30 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.684+0000 7fd39ce25640 10 monclient: dump: 2024-09-15T01:38:03.699 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:03.700 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.685+0000 7fd3a008b640 10 monclient: shutdown 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.685+0000 7fd3a008b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.685+0000 7fd3a008b640 1 -- 172.21.15.204:0/2577210941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3980a2e30 msgr2=0x7fd398132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.685+0000 7fd3a008b640 1 --2- 172.21.15.204:0/2577210941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3980a2e30 0x7fd398132a60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd394009850 tx=0x7fd3940090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.685+0000 7fd3a008b640 1 -- 172.21.15.204:0/2577210941 shutdown_connections 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.685+0000 7fd3a008b640 1 --2- 172.21.15.204:0/2577210941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd39809fb80 0x7fd39809ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.685+0000 7fd3a008b640 1 --2- 172.21.15.204:0/2577210941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3980a2e30 0x7fd398132a60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.685+0000 7fd3a008b640 1 -- 172.21.15.204:0/2577210941 >> 172.21.15.204:0/2577210941 conn(0x7fd39806e860 msgr2=0x7fd39806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.685+0000 7fd3a008b640 1 -- 172.21.15.204:0/2577210941 shutdown_connections 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.685+0000 7fd3a008b640 1 -- 172.21.15.204:0/2577210941 wait complete. 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.685+0000 7fd3a008b640 10 monclient: build_initial_monmap 2024-09-15T01:38:03.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 10 monclient: monmap: 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:03.702 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 1 Processor -- start 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 1 -- start start 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 10 monclient: init 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 10 monclient: _add_conns ranks=[5,1,8,0,6,3,7,2,4] 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd39809fb80 0x7fd398130880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 10 monclient(hunting): picked mon.noname-h con 0x7fd39809fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3980a0a40 0x7fd39811d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 10 monclient(hunting): picked mon.noname-f con 0x7fd3980a0a40 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd3980a2e30 0x7fd39811d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:03.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 10 monclient(hunting): picked mon.noname-e con 0x7fd3980a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd398120ad0 con 0x7fd3980a0a40 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd398121000 con 0x7fd39809fb80 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd39809ca40 con 0x7fd3980a2e30 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd3a008b640 10 monclient(hunting): authenticate will time out at 1099.329823s 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39d626640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3980a0a40 0x7fd39811d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39de27640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd39809fb80 0x7fd398130880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39d626640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3980a0a40 0x7fd39811d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:48264/0 (socket says 172.21.15.204:48264) 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39d626640 1 -- 172.21.15.204:0/3579179497 learned_addr learned my addr 172.21.15.204:0/3579179497 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:03.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39de27640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd39809fb80 0x7fd398130880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:37838/0 (socket says 172.21.15.204:37838) 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39d626640 10 monclient(hunting): get_auth_request con 0x7fd3980a0a40 auth_method 0 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39d626640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39d626640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39d626640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39de27640 10 monclient(hunting): get_auth_request con 0x7fd39809fb80 auth_method 0 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39de27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39de27640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39de27640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39e628640 1 -- 172.21.15.204:0/3579179497 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd3980a2e30 msgr2=0x7fd39811d930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39e628640 1 --2- 172.21.15.204:0/3579179497 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd3980a2e30 0x7fd39811d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.686+0000 7fd39d626640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:03.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39de27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39de27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39de27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 10 monclient(hunting): handle_auth_done global_id 7408 payload 824 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 1 -- 172.21.15.204:0/3579179497 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd3980a2e30 msgr2=0x7fd39811d930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 1 --2- 172.21.15.204:0/3579179497 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd3980a2e30 0x7fd39811d930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 1 -- 172.21.15.204:0/3579179497 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd39809fb80 msgr2=0x7fd398130880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 1 --2- 172.21.15.204:0/3579179497 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd39809fb80 0x7fd398130880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 10 monclient: _finish_hunting 0 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 1 monclient: found mon.noname-f 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:38:03.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 1 -- 172.21.15.204:0/3579179497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd394009390 con 0x7fd3980a0a40 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39de27640 1 --2- 172.21.15.204:0/3579179497 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd39809fb80 0x7fd398130880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd39d626640 1 --2- 172.21.15.204:0/3579179497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3980a0a40 0x7fd39811d3f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd38800b640 tx=0x7fd388006f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38dffb640 1 -- 172.21.15.204:0/3579179497 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd388015070 con 0x7fd3980a0a40 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38dffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38dffb640 10 monclient: dump: 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:03.707 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38dffb640 10 monclient: _finish_auth 0 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38dffb640 1 -- 172.21.15.204:0/3579179497 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd388006150 con 0x7fd3980a0a40 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd3a008b640 5 monclient: authenticate success, global_id 7408 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:03.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd3a008b640 10 monclient: _renew_subs 2024-09-15T01:38:03.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38dffb640 1 -- 172.21.15.204:0/3579179497 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd388006740 con 0x7fd3980a0a40 2024-09-15T01:38:03.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd3a008b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:03.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd3a008b640 1 -- 172.21.15.204:0/3579179497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd398142970 con 0x7fd3980a0a40 2024-09-15T01:38:03.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:03.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:38:03.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38dffb640 10 monclient: dump: 2024-09-15T01:38:03.709 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:03.709 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:03.709 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:03.709 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:03.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd38ffff640 4 set_mon_vals no callback set 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd3a008b640 10 monclient: _renew_subs 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd3a008b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.687+0000 7fd3a008b640 1 -- 172.21.15.204:0/3579179497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39811f5a0 con 0x7fd3980a0a40 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.688+0000 7fd38dffb640 1 -- 172.21.15.204:0/3579179497 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd3880062c0 con 0x7fd3980a0a40 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.688+0000 7fd38dffb640 1 --2- 172.21.15.204:0/3579179497 >> v2:172.21.15.204:6812/28210 conn(0x7fd37c0397b0 0x7fd37c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.688+0000 7fd38dffb640 1 -- 172.21.15.204:0/3579179497 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 8828+0+0 (secure 0 0 0) 0x7fd38804be70 con 0x7fd3980a0a40 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.688+0000 7fd3a008b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.688+0000 7fd3a008b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.688+0000 7fd3a008b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.688+0000 7fd3a008b640 1 -- 172.21.15.204:0/3579179497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39809d6b0 con 0x7fd3980a0a40 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.688+0000 7fd39de27640 1 --2- 172.21.15.204:0/3579179497 >> v2:172.21.15.204:6812/28210 conn(0x7fd37c0397b0 0x7fd37c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.688+0000 7fd39de27640 10 monclient: get_auth_request con 0x7fd37c0397b0 auth_method 0 2024-09-15T01:38:03.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.689+0000 7fd39de27640 1 --2- 172.21.15.204:0/3579179497 >> v2:172.21.15.204:6812/28210 conn(0x7fd37c0397b0 0x7fd37c03bc60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd394034000 tx=0x7fd394032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:03.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.693+0000 7fd38dffb640 1 -- 172.21.15.204:0/3579179497 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd388010440 con 0x7fd3980a0a40 2024-09-15T01:38:03.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.693+0000 7fd38dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:03.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.693+0000 7fd38dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:03.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.856+0000 7fd3a008b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h26"}] 2024-09-15T01:38:03.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.856+0000 7fd3a008b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h26"}] 2024-09-15T01:38:03.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.856+0000 7fd3a008b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:03.859 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:03.856+0000 7fd3a008b640 1 -- 172.21.15.204:0/3579179497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h26"} v 0) v1 -- 0x7fd398070cc0 con 0x7fd3980a0a40 2024-09-15T01:38:04.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.505+0000 7fd38dffb640 1 -- 172.21.15.204:0/3579179497 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h26"}]=0 removed item id -94 name 'h26' from crush map v245) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd38800eb30 con 0x7fd3980a0a40 2024-09-15T01:38:04.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.506+0000 7fd38dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h26"}] 2024-09-15T01:38:04.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.506+0000 7fd38dffb640 10 monclient: _finish_command 2 = system:0 removed item id -94 name 'h26' from crush map 2024-09-15T01:38:04.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.606+0000 7fd3a008b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h26"}] 2024-09-15T01:38:04.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.606+0000 7fd3a008b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h26"}] 2024-09-15T01:38:04.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.606+0000 7fd3a008b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:04.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.606+0000 7fd3a008b640 1 -- 172.21.15.204:0/3579179497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h26"} v 0) v1 -- 0x7fd3980a1f10 con 0x7fd3980a0a40 2024-09-15T01:38:04.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.607+0000 7fd38dffb640 1 -- 172.21.15.204:0/3579179497 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h26"}]=0 device 'h26' does not appear in the crush map v245) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd38800ed10 con 0x7fd3980a0a40 2024-09-15T01:38:04.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.607+0000 7fd38dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h26"}] 2024-09-15T01:38:04.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.607+0000 7fd38dffb640 10 monclient: _finish_command 3 = system:0 device 'h26' does not appear in the crush map 2024-09-15T01:38:04.610 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h26' does not appear in the crush map 2024-09-15T01:38:04.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.608+0000 7fd3a008b640 1 -- 172.21.15.204:0/3579179497 >> v2:172.21.15.204:6812/28210 conn(0x7fd37c0397b0 msgr2=0x7fd37c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:04.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.608+0000 7fd3a008b640 1 --2- 172.21.15.204:0/3579179497 >> v2:172.21.15.204:6812/28210 conn(0x7fd37c0397b0 0x7fd37c03bc60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd394034000 tx=0x7fd394032000 comp rx=0 tx=0).stop 2024-09-15T01:38:04.611 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.608+0000 7fd3a008b640 10 monclient: shutdown 2024-09-15T01:38:04.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.608+0000 7fd3a008b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:04.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.608+0000 7fd3a008b640 1 -- 172.21.15.204:0/3579179497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3980a0a40 msgr2=0x7fd39811d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:04.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.608+0000 7fd3a008b640 1 --2- 172.21.15.204:0/3579179497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3980a0a40 0x7fd39811d3f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd38800b640 tx=0x7fd388006f90 comp rx=0 tx=0).stop 2024-09-15T01:38:04.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.608+0000 7fd3a008b640 1 -- 172.21.15.204:0/3579179497 shutdown_connections 2024-09-15T01:38:04.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.608+0000 7fd3a008b640 1 --2- 172.21.15.204:0/3579179497 >> v2:172.21.15.204:6812/28210 conn(0x7fd37c0397b0 0x7fd37c03bc60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:04.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.608+0000 7fd3a008b640 1 --2- 172.21.15.204:0/3579179497 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd3980a2e30 0x7fd39811d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:04.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.608+0000 7fd3a008b640 1 --2- 172.21.15.204:0/3579179497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3980a0a40 0x7fd39811d3f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:04.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.608+0000 7fd3a008b640 1 --2- 172.21.15.204:0/3579179497 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd39809fb80 0x7fd398130880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:04.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.608+0000 7fd3a008b640 1 -- 172.21.15.204:0/3579179497 >> 172.21.15.204:0/3579179497 conn(0x7fd39806e860 msgr2=0x7fd39809ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:04.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.609+0000 7fd3a008b640 1 -- 172.21.15.204:0/3579179497 shutdown_connections 2024-09-15T01:38:04.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.609+0000 7fd3a008b640 1 -- 172.21.15.204:0/3579179497 wait complete. 2024-09-15T01:38:04.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:38:04.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h27 2024-09-15T01:38:04.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.691+0000 7fdc2295d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:04.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.691+0000 7fdc2295d640 10 monclient: build_initial_monmap 2024-09-15T01:38:04.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.691+0000 7fdc2295d640 10 monclient: monmap: 2024-09-15T01:38:04.694 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:04.694 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:04.694 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:04.693212+0000 2024-09-15T01:38:04.694 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:04.693212+0000 2024-09-15T01:38:04.695 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:04.695 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:04.695 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:04.695 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:04.695 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:04.695 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:04.695 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:04.695 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:04.695 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:04.695 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:04.696 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:04.696 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:04.696 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:04.696 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:04.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.692+0000 7fdc2295d640 1 Processor -- start 2024-09-15T01:38:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.692+0000 7fdc2295d640 1 -- start start 2024-09-15T01:38:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.692+0000 7fdc2295d640 10 monclient: init 2024-09-15T01:38:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.692+0000 7fdc2295d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.692+0000 7fdc2295d640 10 monclient: _add_conns ranks=[3,6,5,1,2,10,4,9,7,8,0] 2024-09-15T01:38:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.692+0000 7fdc2295d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc1c0a2e30 0x7fdc1c132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 10 monclient(hunting): picked mon.noname-g con 0x7fdc1c0a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdc1c09fb80 0x7fdc1c09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 10 monclient(hunting): picked mon.noname-d con 0x7fdc1c09fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdc1c0a0a40 0x7fdc1c0a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 10 monclient(hunting): picked mon.noname-h con 0x7fdc1c0a0a40 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:04.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c09e6c0 con 0x7fdc1c0a2e30 2024-09-15T01:38:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0a04b0 con 0x7fdc1c0a0a40 2024-09-15T01:38:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0a3200 con 0x7fdc1c09fb80 2024-09-15T01:38:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc2295d640 10 monclient(hunting): authenticate will time out at 1100.336052s 2024-09-15T01:38:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc20efa640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdc1c0a0a40 0x7fdc1c0a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc1c0a2e30 0x7fdc1c132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc20efa640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdc1c0a0a40 0x7fdc1c0a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:37846/0 (socket says 172.21.15.204:37846) 2024-09-15T01:38:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc20efa640 1 -- 172.21.15.204:0/1797659130 learned_addr learned my addr 172.21.15.204:0/1797659130 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:04.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 10 monclient(hunting): get_auth_request con 0x7fdc1c0a2e30 auth_method 0 2024-09-15T01:38:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc20efa640 10 monclient(hunting): get_auth_request con 0x7fdc1c0a0a40 auth_method 0 2024-09-15T01:38:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc20efa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc20efa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc20efa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc20efa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:04.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc20efa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc20efa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 10 monclient(hunting): handle_auth_done global_id 15645 payload 274 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 1 -- 172.21.15.204:0/1797659130 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdc1c09fb80 msgr2=0x7fdc1c09ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1b7fe640 1 --2- 172.21.15.204:0/1797659130 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdc1c09fb80 0x7fdc1c09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 1 --2- 172.21.15.204:0/1797659130 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdc1c09fb80 0x7fdc1c09ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 1 -- 172.21.15.204:0/1797659130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdc1c0a0a40 msgr2=0x7fdc1c0a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 1 --2- 172.21.15.204:0/1797659130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdc1c0a0a40 0x7fdc1c0a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 1 monclient: found mon.noname-g 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc1bfff640 1 -- 172.21.15.204:0/1797659130 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c04b720 con 0x7fdc1c0a2e30 2024-09-15T01:38:04.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.693+0000 7fdc20efa640 1 --2- 172.21.15.204:0/1797659130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdc1c0a0a40 0x7fdc1c0a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1bfff640 1 --2- 172.21.15.204:0/1797659130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc1c0a2e30 0x7fdc1c132a60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009880 tx=0x7fdc040090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=47f6ccfb183d01f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1affd640 1 -- 172.21.15.204:0/1797659130 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc0402abb0 con 0x7fdc1c0a2e30 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1affd640 10 monclient: dump: 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:04.709 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1affd640 10 monclient: _finish_auth 0 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1affd640 1 -- 172.21.15.204:0/1797659130 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0402ad30 con 0x7fdc1c0a2e30 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 5 monclient: authenticate success, global_id 15645 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:04.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1affd640 1 -- 172.21.15.204:0/1797659130 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc04031ca0 con 0x7fdc1c0a2e30 2024-09-15T01:38:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 4 set_mon_vals no callback set 2024-09-15T01:38:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc1affd640 10 monclient: dump: 2024-09-15T01:38:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:04.711 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:04.712 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:04.713 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:04.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 10 monclient: shutdown 2024-09-15T01:38:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 1 -- 172.21.15.204:0/1797659130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc1c0a2e30 msgr2=0x7fdc1c132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 1 --2- 172.21.15.204:0/1797659130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc1c0a2e30 0x7fdc1c132a60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009880 tx=0x7fdc040090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 1 -- 172.21.15.204:0/1797659130 shutdown_connections 2024-09-15T01:38:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 1 --2- 172.21.15.204:0/1797659130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdc1c0a0a40 0x7fdc1c0a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:04.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 1 --2- 172.21.15.204:0/1797659130 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdc1c09fb80 0x7fdc1c09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 1 --2- 172.21.15.204:0/1797659130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc1c0a2e30 0x7fdc1c132a60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 1 -- 172.21.15.204:0/1797659130 >> 172.21.15.204:0/1797659130 conn(0x7fdc1c06e860 msgr2=0x7fdc1c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.694+0000 7fdc2295d640 1 -- 172.21.15.204:0/1797659130 shutdown_connections 2024-09-15T01:38:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.695+0000 7fdc2295d640 1 -- 172.21.15.204:0/1797659130 wait complete. 2024-09-15T01:38:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.695+0000 7fdc2295d640 10 monclient: build_initial_monmap 2024-09-15T01:38:04.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.695+0000 7fdc2295d640 10 monclient: monmap: 2024-09-15T01:38:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:04.716 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:04.717 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:04.718 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:04.718 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:04.718 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:04.718 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:04.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.695+0000 7fdc2295d640 1 Processor -- start 2024-09-15T01:38:04.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 1 -- start start 2024-09-15T01:38:04.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 10 monclient: init 2024-09-15T01:38:04.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:04.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 10 monclient: _add_conns ranks=[8,7,2,6,4,0,1,5,3] 2024-09-15T01:38:04.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc1c09fb80 0x7fdc1c09b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:04.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 10 monclient(hunting): picked mon.noname-e con 0x7fdc1c09fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:04.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdc1c0a0a40 0x7fdc1c12e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:04.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 10 monclient(hunting): picked mon.noname-i con 0x7fdc1c0a0a40 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:04.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc1c0a2e30 0x7fdc1c11c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:04.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 10 monclient(hunting): picked mon.noname-b con 0x7fdc1c0a2e30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:04.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:04.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c09d650 con 0x7fdc1c0a2e30 2024-09-15T01:38:04.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:04.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c11efb0 con 0x7fdc1c0a0a40 2024-09-15T01:38:04.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:04.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c11ecd0 con 0x7fdc1c09fb80 2024-09-15T01:38:04.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:04.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc2295d640 10 monclient(hunting): authenticate will time out at 1100.339471s 2024-09-15T01:38:04.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1bfff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc1c09fb80 msgr2=0x7fdc1c09b760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:04.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1bfff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc1c09fb80 0x7fdc1c09b760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:04.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc20efa640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc1c0a2e30 msgr2=0x7fdc1c11c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:04.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc20efa640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc1c0a2e30 0x7fdc1c11c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:04.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdc1c0a0a40 0x7fdc1c12e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:04.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdc1c0a0a40 0x7fdc1c12e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:54832/0 (socket says 172.21.15.204:54832) 2024-09-15T01:38:04.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 1 -- 172.21.15.204:0/1511739230 learned_addr learned my addr 172.21.15.204:0/1511739230 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:04.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 10 monclient(hunting): get_auth_request con 0x7fdc1c0a0a40 auth_method 0 2024-09-15T01:38:04.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:04.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:04.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:04.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:04.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:04.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:04.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 10 monclient(hunting): handle_auth_done global_id 24739 payload 824 2024-09-15T01:38:04.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 1 -- 172.21.15.204:0/1511739230 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc1c09fb80 msgr2=0x7fdc1c09b760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:04.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 1 --2- 172.21.15.204:0/1511739230 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc1c09fb80 0x7fdc1c09b760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:04.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 1 -- 172.21.15.204:0/1511739230 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc1c0a2e30 msgr2=0x7fdc1c11c6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:04.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 1 --2- 172.21.15.204:0/1511739230 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc1c0a2e30 0x7fdc1c11c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:04.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:38:04.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 1 monclient: found mon.noname-i 2024-09-15T01:38:04.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:04.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:38:04.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.696+0000 7fdc1b7fe640 1 -- 172.21.15.204:0/1511739230 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc14009ae0 con 0x7fdc1c0a0a40 2024-09-15T01:38:04.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc1b7fe640 1 --2- 172.21.15.204:0/1511739230 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdc1c0a0a40 0x7fdc1c12e8a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c8b0 tx=0x7fdc0c008040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:04.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc13fff640 1 -- 172.21.15.204:0/1511739230 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc0c00ce40 con 0x7fdc1c0a0a40 2024-09-15T01:38:04.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc13fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:04.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc13fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:38:04.727 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc13fff640 10 monclient: dump: 2024-09-15T01:38:04.727 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:04.727 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:04.727 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:04.727 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:04.727 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:04.728 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:04.728 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:04.728 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:04.728 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:04.728 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:04.728 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:04.729 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:04.729 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:04.729 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:04.729 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:04.729 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:04.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc13fff640 10 monclient: _finish_auth 0 2024-09-15T01:38:04.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc13fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:04.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc2295d640 5 monclient: authenticate success, global_id 24739 2024-09-15T01:38:04.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc13fff640 1 -- 172.21.15.204:0/1511739230 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c007cb0 con 0x7fdc1c0a0a40 2024-09-15T01:38:04.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc2295d640 10 monclient: _renew_subs 2024-09-15T01:38:04.730 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc2295d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:04.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc2295d640 1 -- 172.21.15.204:0/1511739230 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc04009390 con 0x7fdc1c0a0a40 2024-09-15T01:38:04.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc13fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:04.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc13fff640 1 -- 172.21.15.204:0/1511739230 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc0c007e20 con 0x7fdc1c0a0a40 2024-09-15T01:38:04.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc13fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:04.731 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc13fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:04.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc1a7fc640 4 set_mon_vals no callback set 2024-09-15T01:38:04.732 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc13fff640 10 monclient: dump: 2024-09-15T01:38:04.732 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:04.732 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:04.732 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:04.732 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:04.733 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:04.733 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:04.733 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:04.733 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:04.733 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:04.733 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:04.734 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:04.734 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:04.734 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:04.734 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:04.734 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:04.734 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:04.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc2295d640 10 monclient: _renew_subs 2024-09-15T01:38:04.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc2295d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:04.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.697+0000 7fdc2295d640 1 -- 172.21.15.204:0/1511739230 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c11e9f0 con 0x7fdc1c0a0a40 2024-09-15T01:38:04.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.698+0000 7fdc13fff640 1 -- 172.21.15.204:0/1511739230 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdc0c007830 con 0x7fdc1c0a0a40 2024-09-15T01:38:04.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.698+0000 7fdc2295d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:04.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.698+0000 7fdc2295d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:04.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.698+0000 7fdc2295d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:04.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.698+0000 7fdc2295d640 1 -- 172.21.15.204:0/1511739230 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c070cc0 con 0x7fdc1c0a0a40 2024-09-15T01:38:04.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.698+0000 7fdc13fff640 1 --2- 172.21.15.204:0/1511739230 >> v2:172.21.15.204:6812/28210 conn(0x7fdbfc0397b0 0x7fdbfc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:04.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.698+0000 7fdc13fff640 1 -- 172.21.15.204:0/1511739230 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 8615+0+0 (secure 0 0 0) 0x7fdc0c04c280 con 0x7fdc1c0a0a40 2024-09-15T01:38:04.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.698+0000 7fdc1bfff640 1 --2- 172.21.15.204:0/1511739230 >> v2:172.21.15.204:6812/28210 conn(0x7fdbfc0397b0 0x7fdbfc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:04.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.698+0000 7fdc1bfff640 10 monclient: get_auth_request con 0x7fdbfc0397b0 auth_method 0 2024-09-15T01:38:04.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.698+0000 7fdc1bfff640 1 --2- 172.21.15.204:0/1511739230 >> v2:172.21.15.204:6812/28210 conn(0x7fdbfc0397b0 0x7fdbfc03bc60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdc04034000 tx=0x7fdc04032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:04.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.702+0000 7fdc13fff640 1 -- 172.21.15.204:0/1511739230 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fdc0c017300 con 0x7fdc1c0a0a40 2024-09-15T01:38:04.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.702+0000 7fdc13fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:04.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.702+0000 7fdc13fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:04.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.862+0000 7fdc2295d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h27"}] 2024-09-15T01:38:04.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.862+0000 7fdc2295d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h27"}] 2024-09-15T01:38:04.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.862+0000 7fdc2295d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:04.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:04.862+0000 7fdc2295d640 1 -- 172.21.15.204:0/1511739230 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h27"} v 0) v1 -- 0x7fdc1c0a1f10 con 0x7fdc1c0a0a40 2024-09-15T01:38:05.508 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.506+0000 7fdc13fff640 1 -- 172.21.15.204:0/1511739230 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h27"}]=0 removed item id -97 name 'h27' from crush map v246) v1 ==== 120+0+0 (secure 0 0 0) 0x7fdc0c01e030 con 0x7fdc1c0a0a40 2024-09-15T01:38:05.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.506+0000 7fdc13fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h27"}] 2024-09-15T01:38:05.509 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.506+0000 7fdc13fff640 10 monclient: _finish_command 2 = system:0 removed item id -97 name 'h27' from crush map 2024-09-15T01:38:05.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.605+0000 7fdc2295d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h27"}] 2024-09-15T01:38:05.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.605+0000 7fdc2295d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h27"}] 2024-09-15T01:38:05.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.605+0000 7fdc2295d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:05.609 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.605+0000 7fdc2295d640 1 -- 172.21.15.204:0/1511739230 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h27"} v 0) v1 -- 0x7fdc1c1428e0 con 0x7fdc1c0a0a40 2024-09-15T01:38:05.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.606+0000 7fdc13fff640 1 -- 172.21.15.204:0/1511739230 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h27"}]=0 device 'h27' does not appear in the crush map v246) v1 ==== 120+0+0 (secure 0 0 0) 0x7fdc0c01e020 con 0x7fdc1c0a0a40 2024-09-15T01:38:05.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.606+0000 7fdc13fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h27"}] 2024-09-15T01:38:05.610 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.606+0000 7fdc13fff640 10 monclient: _finish_command 3 = system:0 device 'h27' does not appear in the crush map 2024-09-15T01:38:05.610 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h27' does not appear in the crush map 2024-09-15T01:38:05.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 1 -- 172.21.15.204:0/1511739230 >> v2:172.21.15.204:6812/28210 conn(0x7fdbfc0397b0 msgr2=0x7fdbfc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:05.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 1 --2- 172.21.15.204:0/1511739230 >> v2:172.21.15.204:6812/28210 conn(0x7fdbfc0397b0 0x7fdbfc03bc60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdc04034000 tx=0x7fdc04032000 comp rx=0 tx=0).stop 2024-09-15T01:38:05.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 10 monclient: shutdown 2024-09-15T01:38:05.612 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:05.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 1 -- 172.21.15.204:0/1511739230 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdc1c0a0a40 msgr2=0x7fdc1c12e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:05.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 1 --2- 172.21.15.204:0/1511739230 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdc1c0a0a40 0x7fdc1c12e8a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c8b0 tx=0x7fdc0c008040 comp rx=0 tx=0).stop 2024-09-15T01:38:05.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 1 -- 172.21.15.204:0/1511739230 shutdown_connections 2024-09-15T01:38:05.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 1 --2- 172.21.15.204:0/1511739230 >> v2:172.21.15.204:6812/28210 conn(0x7fdbfc0397b0 0x7fdbfc03bc60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:05.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 1 --2- 172.21.15.204:0/1511739230 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdc1c0a0a40 0x7fdc1c12e8a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:05.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 1 --2- 172.21.15.204:0/1511739230 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc1c0a2e30 0x7fdc1c11c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:05.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 1 --2- 172.21.15.204:0/1511739230 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc1c09fb80 0x7fdc1c09b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:05.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 1 -- 172.21.15.204:0/1511739230 >> 172.21.15.204:0/1511739230 conn(0x7fdc1c06e860 msgr2=0x7fdc1c0a5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:05.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 1 -- 172.21.15.204:0/1511739230 shutdown_connections 2024-09-15T01:38:05.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.608+0000 7fdc2295d640 1 -- 172.21.15.204:0/1511739230 wait complete. 2024-09-15T01:38:05.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:38:05.620 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h28 2024-09-15T01:38:05.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.691+0000 7f093c55d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:05.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.691+0000 7f093c55d640 10 monclient: build_initial_monmap 2024-09-15T01:38:05.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.691+0000 7f093c55d640 10 monclient: monmap: 2024-09-15T01:38:05.695 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:05.695 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:05.695 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:05.692617+0000 2024-09-15T01:38:05.695 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:05.692617+0000 2024-09-15T01:38:05.695 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:05.695 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:05.696 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:05.697 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:05.698 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.691+0000 7f093c55d640 1 Processor -- start 2024-09-15T01:38:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.691+0000 7f093c55d640 1 -- start start 2024-09-15T01:38:05.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.691+0000 7f093c55d640 10 monclient: init 2024-09-15T01:38:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 10 monclient: _add_conns ranks=[1,4,3,8,0,2,6,7,9,5,10] 2024-09-15T01:38:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f09340a2e30 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093409fb80 0x7f093409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:05.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 10 monclient(hunting): picked mon.noname-c con 0x7f093409fb80 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0934137d20 0x7f093413a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 10 monclient(hunting): picked mon.noname-g con 0x7f0934137d20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093404b720 con 0x7f09340a2e30 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093409e6c0 con 0x7f0934137d20 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09340a04b0 con 0x7f093409fb80 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093c55d640 10 monclient(hunting): authenticate will time out at 1101.335517s 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093a2f9640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f09340a2e30 0x7f0934132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:46302/0 (socket says 172.21.15.204:46302) 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093a2f9640 1 -- 172.21.15.204:0/3369675533 learned_addr learned my addr 172.21.15.204:0/3369675533 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f0939af8640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093409fb80 msgr2=0x7f093409ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:05.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 1 --2- 172.21.15.204:0/3369675533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0934137d20 0x7f093413a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f0939af8640 1 --2- 172.21.15.204:0/3369675533 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093409fb80 0x7f093409ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 10 monclient(hunting): get_auth_request con 0x7f0934137d20 auth_method 0 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f09392f7640 1 -- 172.21.15.204:0/3369675533 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672861580 0 0) 0x7f093404b720 con 0x7f09340a2e30 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f09392f7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f09392f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f09392f7640 10 monclient(hunting): my global_id is 7417 2024-09-15T01:38:05.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f09392f7640 1 -- 172.21.15.204:0/3369675533 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f091c0034e0 con 0x7f09340a2e30 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 10 monclient(hunting): handle_auth_done global_id 15654 payload 274 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 1 -- 172.21.15.204:0/3369675533 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093409fb80 msgr2=0x7f093409ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 1 --2- 172.21.15.204:0/3369675533 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093409fb80 0x7f093409ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 1 -- 172.21.15.204:0/3369675533 >> v1:172.21.15.204:6789/0 conn(0x7f09340a2e30 legacy=0x7f0934132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 10 monclient: _finish_hunting 0 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 1 monclient: found mon.noname-g 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.692+0000 7f093aafa640 1 -- 172.21.15.204:0/3369675533 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0934133170 con 0x7f0934137d20 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f093aafa640 1 --2- 172.21.15.204:0/3369675533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0934137d20 0x7f093413a100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f092c009880 tx=0x7f092c0090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=f89bf4912b3ef7b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f09392f7640 1 -- 172.21.15.204:0/3369675533 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f092c02ac80 con 0x7f0934137d20 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f09392f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:05.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f09392f7640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f09392f7640 10 monclient: dump: 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:05.710 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f09392f7640 10 monclient: _finish_auth 0 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f09392f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f093c55d640 5 monclient: authenticate success, global_id 15654 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f093c55d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f09392f7640 1 -- 172.21.15.204:0/3369675533 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c02ae00 con 0x7f0934137d20 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f09392f7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f093c55d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f093c55d640 4 set_mon_vals no callback set 2024-09-15T01:38:05.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f09392f7640 1 -- 172.21.15.204:0/3369675533 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f092c031ca0 con 0x7f0934137d20 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f09392f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f09392f7640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.693+0000 7f09392f7640 10 monclient: dump: 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:05.712 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.694+0000 7f093c55d640 10 monclient: shutdown 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.694+0000 7f093c55d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.694+0000 7f093c55d640 1 -- 172.21.15.204:0/3369675533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0934137d20 msgr2=0x7f093413a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.694+0000 7f093c55d640 1 --2- 172.21.15.204:0/3369675533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0934137d20 0x7f093413a100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f092c009880 tx=0x7f092c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.694+0000 7f093c55d640 1 -- 172.21.15.204:0/3369675533 shutdown_connections 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.694+0000 7f093c55d640 1 --2- 172.21.15.204:0/3369675533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0934137d20 0x7f093413a100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.694+0000 7f093c55d640 1 --2- 172.21.15.204:0/3369675533 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f093409fb80 0x7f093409ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:05.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.694+0000 7f093c55d640 1 -- 172.21.15.204:0/3369675533 >> 172.21.15.204:0/3369675533 conn(0x7f093406e860 msgr2=0x7f093406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.694+0000 7f093c55d640 1 -- 172.21.15.204:0/3369675533 shutdown_connections 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.694+0000 7f093c55d640 1 -- 172.21.15.204:0/3369675533 wait complete. 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.694+0000 7f093c55d640 10 monclient: build_initial_monmap 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.694+0000 7f093c55d640 10 monclient: monmap: 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:05.714 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 1 Processor -- start 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 1 -- start start 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 10 monclient: init 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 10 monclient: _add_conns ranks=[7,5,8,0,3,2,6,4,1] 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f093409fb80 0x7f09340990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:05.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 10 monclient(hunting): picked mon.noname-i con 0x7f093409fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09340a2e30 0x7f09340995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 10 monclient(hunting): picked mon.noname-h con 0x7f09340a2e30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0934137d20 0x7f0934099b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 10 monclient(hunting): picked mon.noname-e con 0x7f0934137d20 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09341313b0 con 0x7f09340a2e30 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f093409d650 con 0x7f093409fb80 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f093409b060 con 0x7f0934137d20 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093c55d640 10 monclient(hunting): authenticate will time out at 1101.338699s 2024-09-15T01:38:05.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f0939af8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09340a2e30 0x7f09340995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093a2f9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f093409fb80 0x7f09340990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f0939af8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09340a2e30 0x7f09340995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:37862/0 (socket says 172.21.15.204:37862) 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f0939af8640 1 -- 172.21.15.204:0/1059840691 learned_addr learned my addr 172.21.15.204:0/1059840691 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f0939af8640 10 monclient(hunting): get_auth_request con 0x7f09340a2e30 auth_method 0 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f0939af8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f0939af8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f0939af8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093a2f9640 10 monclient(hunting): get_auth_request con 0x7f093409fb80 auth_method 0 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093a2f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093a2f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093a2f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093aafa640 1 -- 172.21.15.204:0/1059840691 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0934137d20 msgr2=0x7f0934099b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:05.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093aafa640 1 --2- 172.21.15.204:0/1059840691 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0934137d20 0x7f0934099b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f0939af8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f0939af8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f0939af8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093a2f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093a2f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.695+0000 7f093a2f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f0939af8640 10 monclient(hunting): handle_auth_done global_id 6548 payload 824 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f0939af8640 1 -- 172.21.15.204:0/1059840691 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0934137d20 msgr2=0x7f0934099b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f0939af8640 1 --2- 172.21.15.204:0/1059840691 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0934137d20 0x7f0934099b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f0939af8640 1 -- 172.21.15.204:0/1059840691 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f093409fb80 msgr2=0x7f09340990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f0939af8640 1 --2- 172.21.15.204:0/1059840691 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f093409fb80 0x7f09340990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f0939af8640 10 monclient: _finish_hunting 0 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f0939af8640 1 monclient: found mon.noname-h 2024-09-15T01:38:05.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f0939af8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f0939af8640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f0939af8640 1 -- 172.21.15.204:0/1059840691 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c009390 con 0x7f09340a2e30 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f0939af8640 1 --2- 172.21.15.204:0/1059840691 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09340a2e30 0x7f09340995e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f092800b5e0 tx=0x7f0928006f90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f09327fc640 1 -- 172.21.15.204:0/1059840691 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0928006150 con 0x7f09340a2e30 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f09327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f09327fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f09327fc640 10 monclient: dump: 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:05.719 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f09327fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f09327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:05.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f09327fc640 1 -- 172.21.15.204:0/1059840691 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09280062f0 con 0x7f09340a2e30 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f093c55d640 5 monclient: authenticate success, global_id 6548 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f093c55d640 10 monclient: _renew_subs 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f093c55d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f093c55d640 1 -- 172.21.15.204:0/1059840691 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093409b2a0 con 0x7f09340a2e30 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f09327fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f09327fc640 1 -- 172.21.15.204:0/1059840691 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f09280103e0 con 0x7f09340a2e30 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f09327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f09327fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f09327fc640 10 monclient: dump: 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:05.721 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f093c55d640 10 monclient: _renew_subs 2024-09-15T01:38:05.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f0938af6640 4 set_mon_vals no callback set 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f093c55d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.696+0000 7f093c55d640 1 -- 172.21.15.204:0/1059840691 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093409ab50 con 0x7f09340a2e30 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.697+0000 7f09327fc640 1 -- 172.21.15.204:0/1059840691 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0928006460 con 0x7f09340a2e30 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.697+0000 7f093c55d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.697+0000 7f093c55d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.697+0000 7f093c55d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.697+0000 7f093c55d640 1 -- 172.21.15.204:0/1059840691 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093411c7e0 con 0x7f09340a2e30 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.697+0000 7f09327fc640 1 --2- 172.21.15.204:0/1059840691 >> v2:172.21.15.204:6812/28210 conn(0x7f09200397b0 0x7f092003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.697+0000 7f09327fc640 1 -- 172.21.15.204:0/1059840691 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 8402+0+0 (secure 0 0 0) 0x7f092804b6c0 con 0x7f09340a2e30 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.697+0000 7f093a2f9640 1 --2- 172.21.15.204:0/1059840691 >> v2:172.21.15.204:6812/28210 conn(0x7f09200397b0 0x7f092003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.697+0000 7f093a2f9640 10 monclient: get_auth_request con 0x7f09200397b0 auth_method 0 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.697+0000 7f093a2f9640 1 --2- 172.21.15.204:0/1059840691 >> v2:172.21.15.204:6812/28210 conn(0x7f09200397b0 0x7f092003bc60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0924005bc0 tx=0x7f0924009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:05.723 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.701+0000 7f09327fc640 1 -- 172.21.15.204:0/1059840691 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f092800dd30 con 0x7f09340a2e30 2024-09-15T01:38:05.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.701+0000 7f09327fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:05.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.701+0000 7f09327fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:05.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.865+0000 7f093c55d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h28"}] 2024-09-15T01:38:05.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.865+0000 7f093c55d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h28"}] 2024-09-15T01:38:05.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.865+0000 7f093c55d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:05.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:05.865+0000 7f093c55d640 1 -- 172.21.15.204:0/1059840691 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h28"} v 0) v1 -- 0x7f0934070cc0 con 0x7f09340a2e30 2024-09-15T01:38:06.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.478+0000 7f09327fc640 1 -- 172.21.15.204:0/1059840691 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h28"}]=0 removed item id -100 name 'h28' from crush map v247) v1 ==== 121+0+0 (secure 0 0 0) 0x7f0928017030 con 0x7f09340a2e30 2024-09-15T01:38:06.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.478+0000 7f09327fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h28"}] 2024-09-15T01:38:06.481 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.478+0000 7f09327fc640 10 monclient: _finish_command 2 = system:0 removed item id -100 name 'h28' from crush map 2024-09-15T01:38:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.581+0000 7f093c55d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h28"}] 2024-09-15T01:38:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.581+0000 7f093c55d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h28"}] 2024-09-15T01:38:06.583 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.581+0000 7f093c55d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:06.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.581+0000 7f093c55d640 1 -- 172.21.15.204:0/1059840691 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h28"} v 0) v1 -- 0x7f09340008d0 con 0x7f09340a2e30 2024-09-15T01:38:06.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.581+0000 7f09327fc640 1 -- 172.21.15.204:0/1059840691 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h28"}]=0 device 'h28' does not appear in the crush map v247) v1 ==== 120+0+0 (secure 0 0 0) 0x7f0928017020 con 0x7f09340a2e30 2024-09-15T01:38:06.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.581+0000 7f09327fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h28"}] 2024-09-15T01:38:06.584 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.581+0000 7f09327fc640 10 monclient: _finish_command 3 = system:0 device 'h28' does not appear in the crush map 2024-09-15T01:38:06.584 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h28' does not appear in the crush map 2024-09-15T01:38:06.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.583+0000 7f093c55d640 1 -- 172.21.15.204:0/1059840691 >> v2:172.21.15.204:6812/28210 conn(0x7f09200397b0 msgr2=0x7f092003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:06.586 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.583+0000 7f093c55d640 1 --2- 172.21.15.204:0/1059840691 >> v2:172.21.15.204:6812/28210 conn(0x7f09200397b0 0x7f092003bc60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0924005bc0 tx=0x7f0924009040 comp rx=0 tx=0).stop 2024-09-15T01:38:06.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.583+0000 7f093c55d640 10 monclient: shutdown 2024-09-15T01:38:06.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.583+0000 7f093c55d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:06.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.583+0000 7f093c55d640 1 -- 172.21.15.204:0/1059840691 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09340a2e30 msgr2=0x7f09340995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:06.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.583+0000 7f093c55d640 1 --2- 172.21.15.204:0/1059840691 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09340a2e30 0x7f09340995e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f092800b5e0 tx=0x7f0928006f90 comp rx=0 tx=0).stop 2024-09-15T01:38:06.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.584+0000 7f093c55d640 1 -- 172.21.15.204:0/1059840691 shutdown_connections 2024-09-15T01:38:06.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.584+0000 7f093c55d640 1 --2- 172.21.15.204:0/1059840691 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0934137d20 0x7f0934099b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:06.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.584+0000 7f093c55d640 1 --2- 172.21.15.204:0/1059840691 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f09340a2e30 0x7f09340995e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:06.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.584+0000 7f093c55d640 1 --2- 172.21.15.204:0/1059840691 >> v2:172.21.15.204:6812/28210 conn(0x7f09200397b0 0x7f092003bc60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:06.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.584+0000 7f093c55d640 1 --2- 172.21.15.204:0/1059840691 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f093409fb80 0x7f09340990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:06.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.584+0000 7f093c55d640 1 -- 172.21.15.204:0/1059840691 >> 172.21.15.204:0/1059840691 conn(0x7f093406e860 msgr2=0x7f09340a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:06.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.584+0000 7f093c55d640 1 -- 172.21.15.204:0/1059840691 shutdown_connections 2024-09-15T01:38:06.587 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.584+0000 7f093c55d640 1 -- 172.21.15.204:0/1059840691 wait complete. 2024-09-15T01:38:06.595 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:38:06.595 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h29 2024-09-15T01:38:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.666+0000 7ff12819d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.666+0000 7ff12819d640 10 monclient: build_initial_monmap 2024-09-15T01:38:06.669 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.666+0000 7ff12819d640 10 monclient: monmap: 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:06.668434+0000 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:06.668434+0000 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:06.670 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:06.671 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:06.671 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.667+0000 7ff12819d640 1 Processor -- start 2024-09-15T01:38:06.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.667+0000 7ff12819d640 1 -- start start 2024-09-15T01:38:06.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.667+0000 7ff12819d640 10 monclient: init 2024-09-15T01:38:06.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:06.672 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 10 monclient: _add_conns ranks=[8,2,9,7,0,4,10,6,5,1,3] 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1200a2e30 0x7ff120132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 10 monclient(hunting): picked mon.noname-e con 0x7ff1200a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12009fb80 0x7ff12009ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 10 monclient(hunting): picked mon.noname-b con 0x7ff12009fb80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff1200a0a40 0x7ff1200a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 10 monclient(hunting): picked mon.noname-a con 0x7ff1200a0a40 addr v2:172.21.15.178:3300/0 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12009e6c0 con 0x7ff1200a0a40 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1200a04b0 con 0x7ff12009fb80 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff1200a3200 con 0x7ff1200a2e30 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:06.673 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12819d640 10 monclient(hunting): authenticate will time out at 1102.311196s 2024-09-15T01:38:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12673a640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7ff1200a0a40 msgr2=0x7ff1200a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff125f39640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1200a2e30 msgr2=0x7ff120132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff125738640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12009fb80 msgr2=0x7ff12009ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff12673a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff1200a0a40 0x7ff1200a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff125738640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12009fb80 0x7ff12009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:06.674 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.668+0000 7ff125f39640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1200a2e30 0x7ff120132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:06.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.869+0000 7ff12673a640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7ff1200a0a40 msgr2=0x7ff1200a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:06.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.869+0000 7ff125f39640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1200a2e30 msgr2=0x7ff120132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:06.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.869+0000 7ff125738640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12009fb80 msgr2=0x7ff12009ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:06.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.869+0000 7ff12673a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff1200a0a40 0x7ff1200a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:06.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.869+0000 7ff125738640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12009fb80 0x7ff12009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:06.871 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:06.869+0000 7ff125f39640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1200a2e30 0x7ff120132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:07.272 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:07.270+0000 7ff125738640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12009fb80 msgr2=0x7ff12009ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:07.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:07.270+0000 7ff125738640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12009fb80 0x7ff12009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:07.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:07.270+0000 7ff12673a640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7ff1200a0a40 msgr2=0x7ff1200a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:07.273 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:07.270+0000 7ff12673a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff1200a0a40 0x7ff1200a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:07.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:07.270+0000 7ff125f39640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1200a2e30 msgr2=0x7ff120132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:07.274 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:07.270+0000 7ff125f39640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1200a2e30 0x7ff120132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:08.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:08.071+0000 7ff125738640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12009fb80 msgr2=0x7ff12009ff70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:08.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:08.071+0000 7ff12673a640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7ff1200a0a40 msgr2=0x7ff1200a0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:08.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:08.071+0000 7ff125f39640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1200a2e30 msgr2=0x7ff120132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:08.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:08.071+0000 7ff125f39640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1200a2e30 0x7ff120132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:08.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:08.071+0000 7ff125738640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12009fb80 0x7ff12009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:08.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:08.071+0000 7ff12673a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff1200a0a40 0x7ff1200a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:09.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 10 monclient(hunting): tick 2024-09-15T01:38:09.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 1 monclient(hunting): continuing hunt 2024-09-15T01:38:09.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:38:09.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1200a2e30 msgr2=0x7ff120132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:09.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff1200a2e30 0x7ff120132a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:09.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12009fb80 msgr2=0x7ff12009ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:09.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12009fb80 0x7ff12009ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:09.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7ff1200a0a40 msgr2=0x7ff1200a0ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:09.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff1200a0a40 0x7ff1200a0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:09.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 10 monclient: _add_conns ranks=[0,4,1,3,7,5,6,10] 2024-09-15T01:38:09.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff1040037d0 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:09.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff1040073f0 0x7ff104009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:09.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 10 monclient(hunting): picked mon.noname-c con 0x7ff1040073f0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:09.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff104009f30 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:09.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12004b720 con 0x7ff1040037d0 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10400ed00 con 0x7ff104009f30 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff12009e6c0 con 0x7ff1040073f0 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff1177fe640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff125738640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff1040073f0 msgr2=0x7ff104009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff125f39640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7ff1040037d0 legacy=0x7ff104005c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff125738640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff1040073f0 0x7ff104009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff12673a640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7ff104009f30 0x7ff10400c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:56906/0 (socket says 172.21.15.204:56906) 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff12673a640 1 -- 172.21.15.204:0/2919780683 learned_addr learned my addr 172.21.15.204:0/2919780683 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.669+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102148081 0 0) 0x7ff10400ed00 con 0x7ff104009f30 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient(hunting): my global_id is 7426 2024-09-15T01:38:09.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff10c0034e0 con 0x7ff104009f30 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1949524645 0 0) 0x7ff10c0034e0 con 0x7ff104009f30 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff1040073f0 msgr2=0x7ff104009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff1040073f0 0x7ff104009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 >> v1:172.21.15.178:6789/0 conn(0x7ff1040037d0 legacy=0x7ff104005c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient: _finish_hunting 0 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff12673a640 1 -- 172.21.15.204:0/2919780683 reap_dead start 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10400ffb0 con 0x7ff104009f30 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient: _finish_auth 0 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:09.682 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7ff1180046f0 con 0x7ff104009f30 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient: dump: 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:09.683 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 >> v1:172.21.15.204:6789/0 conn(0x7ff104009f30 legacy=0x7ff10400c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient: _add_conns ranks=[2,1,6,5,0,8,3,4,7] 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff10c004220 0x7ff10c004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient(hunting): picked mon.b con 0x7ff10c004220 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff10c004ec0 0x7ff10c0099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:09.684 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient(hunting): picked mon.f con 0x7ff10c004ec0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff10c009f00 0x7ff10c00c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient(hunting): picked mon.d con 0x7ff10c009f00 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12009e6c0 con 0x7ff10c004ec0 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c00cae0 con 0x7ff10c004220 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c00cc50 con 0x7ff10c009f00 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff124f37640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125f39640 1 -- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff10c004220 msgr2=0x7ff10c004780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff10c004ec0 0x7ff10c0099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125f39640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff10c004220 0x7ff10c004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:09.685 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 10 monclient(hunting): get_auth_request con 0x7ff10c004ec0 auth_method 0 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff12673a640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff10c009f00 0x7ff10c00c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff12673a640 10 monclient(hunting): get_auth_request con 0x7ff10c009f00 auth_method 0 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff12673a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff12673a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff12673a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff12673a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:09.686 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff12673a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff12673a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 10 monclient(hunting): handle_auth_done global_id 7426 payload 293 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 1 -- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff10c009f00 msgr2=0x7ff10c00c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff10c009f00 0x7ff10c00c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 1 -- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff10c004220 msgr2=0x7ff10c004780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff10c004220 0x7ff10c004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 10 monclient: _finish_hunting 0 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 1 monclient: found mon.f 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.670+0000 7ff125738640 1 -- 172.21.15.204:0/2919780683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c0034e0 con 0x7ff10c004ec0 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.671+0000 7ff125738640 10 monclient: _finish_auth 0 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.671+0000 7ff125738640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:09.687 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.671+0000 7ff12819d640 5 monclient: authenticate success, global_id 7426 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.671+0000 7ff12819d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.671+0000 7ff125738640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff10c004ec0 0x7ff10c0099c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff110009f90 tx=0x7ff1100095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7cf071212790aef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.671+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1100050d0 con 0x7ff10c004ec0 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.671+0000 7ff124f37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.671+0000 7ff124f37640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.671+0000 7ff124f37640 10 monclient: dump: 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:09.688 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11002d070 con 0x7ff10c004ec0 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff12819d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff124f37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:09.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff124f37640 1 -- 172.21.15.204:0/2919780683 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff110005710 con 0x7ff10c004ec0 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff12819d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff12819d640 4 set_mon_vals no callback set 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff124f37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff124f37640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff124f37640 10 monclient: dump: 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:09.690 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff12819d640 10 monclient: shutdown 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff12819d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff12819d640 1 -- 172.21.15.204:0/2919780683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff10c004ec0 msgr2=0x7ff10c0099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff12819d640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff10c004ec0 0x7ff10c0099c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff110009f90 tx=0x7ff1100095d0 comp rx=0 tx=0).stop 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff12819d640 1 -- 172.21.15.204:0/2919780683 shutdown_connections 2024-09-15T01:38:09.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff12819d640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff10c009f00 0x7ff10c00c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff12819d640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff10c004ec0 0x7ff10c0099c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff12819d640 1 --2- 172.21.15.204:0/2919780683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff10c004220 0x7ff10c004780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.672+0000 7ff12819d640 1 -- 172.21.15.204:0/2919780683 >> 172.21.15.204:0/2919780683 conn(0x7ff12006e860 msgr2=0x7ff12006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.673+0000 7ff12819d640 1 -- 172.21.15.204:0/2919780683 shutdown_connections 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.673+0000 7ff12819d640 1 -- 172.21.15.204:0/2919780683 wait complete. 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.673+0000 7ff12819d640 10 monclient: build_initial_monmap 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.673+0000 7ff12819d640 10 monclient: monmap: 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:09.692 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 1 Processor -- start 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 1 -- start start 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 10 monclient: init 2024-09-15T01:38:09.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 10 monclient: _add_conns ranks=[3,8,5,0,4,2,6,7,1] 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff104009f30 0x7ff12011df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 10 monclient(hunting): picked mon.noname-g con 0x7ff104009f30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff10c004220 0x7ff12011e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 10 monclient(hunting): picked mon.noname-e con 0x7ff10c004220 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff10c004ec0 0x7ff12011e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 10 monclient(hunting): picked mon.noname-h con 0x7ff10c004ec0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff120121720 con 0x7ff104009f30 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff12009ca40 con 0x7ff10c004ec0 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:09.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff12009d650 con 0x7ff10c004220 2024-09-15T01:38:09.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:09.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12819d640 10 monclient(hunting): authenticate will time out at 1105.316808s 2024-09-15T01:38:09.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff125f39640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff104009f30 0x7ff12011df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:09.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff125738640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff10c004220 msgr2=0x7ff12011e440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:09.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12673a640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff10c004ec0 0x7ff12011e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:09.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff125738640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff10c004220 0x7ff12011e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:09.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff125f39640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff104009f30 0x7ff12011df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:34132/0 (socket says 172.21.15.204:34132) 2024-09-15T01:38:09.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff125f39640 1 -- 172.21.15.204:0/870569765 learned_addr learned my addr 172.21.15.204:0/870569765 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:09.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff125f39640 10 monclient(hunting): get_auth_request con 0x7ff104009f30 auth_method 0 2024-09-15T01:38:09.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff125f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:09.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff125f39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:09.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff125f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12673a640 10 monclient(hunting): get_auth_request con 0x7ff10c004ec0 auth_method 0 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12673a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12673a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.674+0000 7ff12673a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff125f39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff125f39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff125f39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 10 monclient(hunting): handle_auth_done global_id 6557 payload 824 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 1 -- 172.21.15.204:0/870569765 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff10c004220 msgr2=0x7ff12011e440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:09.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 1 --2- 172.21.15.204:0/870569765 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff10c004220 0x7ff12011e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 1 -- 172.21.15.204:0/870569765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff104009f30 msgr2=0x7ff12011df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 1 --2- 172.21.15.204:0/870569765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff104009f30 0x7ff12011df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 10 monclient: _finish_hunting 0 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 1 monclient: found mon.noname-h 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 1 -- 172.21.15.204:0/870569765 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff120120580 con 0x7ff10c004ec0 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff125f39640 1 --2- 172.21.15.204:0/870569765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff104009f30 0x7ff12011df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12673a640 1 --2- 172.21.15.204:0/870569765 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff10c004ec0 0x7ff12011e980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff1200a3340 tx=0x7ff118004610 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff115ffb640 1 -- 172.21.15.204:0/870569765 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff11800fec0 con 0x7ff10c004ec0 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff115ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff115ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff115ffb640 10 monclient: dump: 2024-09-15T01:38:09.697 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:09.698 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:09.698 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:09.698 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:09.698 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:09.698 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:09.698 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:09.698 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:09.698 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:09.698 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:09.698 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:09.698 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:09.698 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff115ffb640 10 monclient: _finish_auth 0 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff115ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12819d640 5 monclient: authenticate success, global_id 6557 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff115ffb640 1 -- 172.21.15.204:0/870569765 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1180127d0 con 0x7ff10c004ec0 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12819d640 10 monclient: _renew_subs 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12819d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff12819d640 1 -- 172.21.15.204:0/870569765 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff120121440 con 0x7ff10c004ec0 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff115ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff115ffb640 1 -- 172.21.15.204:0/870569765 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff118012dc0 con 0x7ff10c004ec0 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff115ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:09.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff115ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:38:09.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff115ffb640 10 monclient: dump: 2024-09-15T01:38:09.700 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:09.700 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:09.700 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:09.700 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:09.700 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:09.700 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:09.700 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:09.700 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:09.700 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:09.700 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:09.700 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.676+0000 7ff12819d640 10 monclient: _renew_subs 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.676+0000 7ff12819d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.676+0000 7ff12819d640 1 -- 172.21.15.204:0/870569765 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff120132470 con 0x7ff10c004ec0 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.675+0000 7ff117fff640 4 set_mon_vals no callback set 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.676+0000 7ff115ffb640 1 -- 172.21.15.204:0/870569765 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff118012940 con 0x7ff10c004ec0 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.677+0000 7ff115ffb640 1 --2- 172.21.15.204:0/870569765 >> v2:172.21.15.204:6812/28210 conn(0x7ff10403a7c0 0x7ff10403cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.677+0000 7ff115ffb640 1 -- 172.21.15.204:0/870569765 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 8189+0+0 (secure 0 0 0) 0x7ff11804b090 con 0x7ff10c004ec0 2024-09-15T01:38:09.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.677+0000 7ff125f39640 1 --2- 172.21.15.204:0/870569765 >> v2:172.21.15.204:6812/28210 conn(0x7ff10403a7c0 0x7ff10403cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:09.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.677+0000 7ff125f39640 10 monclient: get_auth_request con 0x7ff10403a7c0 auth_method 0 2024-09-15T01:38:09.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.677+0000 7ff12819d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:09.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.677+0000 7ff12819d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:09.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.677+0000 7ff12819d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:09.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.677+0000 7ff12819d640 1 -- 172.21.15.204:0/870569765 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1200a22c0 con 0x7ff10c004ec0 2024-09-15T01:38:09.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.677+0000 7ff125f39640 1 --2- 172.21.15.204:0/870569765 >> v2:172.21.15.204:6812/28210 conn(0x7ff10403a7c0 0x7ff10403cc70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0099a0 tx=0x7ff11c009120 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:09.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.682+0000 7ff115ffb640 1 -- 172.21.15.204:0/870569765 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff118011a90 con 0x7ff10c004ec0 2024-09-15T01:38:09.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.682+0000 7ff115ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:09.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.682+0000 7ff115ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:09.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.846+0000 7ff12819d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h29"}] 2024-09-15T01:38:09.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.846+0000 7ff12819d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h29"}] 2024-09-15T01:38:09.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.846+0000 7ff12819d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:09.848 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.846+0000 7ff12819d640 1 -- 172.21.15.204:0/870569765 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h29"} v 0) v1 -- 0x7ff120070cc0 con 0x7ff10c004ec0 2024-09-15T01:38:09.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.903+0000 7ff115ffb640 1 -- 172.21.15.204:0/870569765 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h29"}]=0 removed item id -103 name 'h29' from crush map v248) v1 ==== 121+0+0 (secure 0 0 0) 0x7ff1180112b0 con 0x7ff10c004ec0 2024-09-15T01:38:09.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.903+0000 7ff115ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h29"}] 2024-09-15T01:38:09.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.903+0000 7ff115ffb640 10 monclient: _finish_command 2 = system:0 removed item id -103 name 'h29' from crush map 2024-09-15T01:38:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.999+0000 7ff12819d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h29"}] 2024-09-15T01:38:10.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.999+0000 7ff12819d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h29"}] 2024-09-15T01:38:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.999+0000 7ff12819d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.999+0000 7ff12819d640 1 -- 172.21.15.204:0/870569765 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h29"} v 0) v1 -- 0x7ff1200a1f10 con 0x7ff10c004ec0 2024-09-15T01:38:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.999+0000 7ff115ffb640 1 -- 172.21.15.204:0/870569765 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h29"}]=0 device 'h29' does not appear in the crush map v248) v1 ==== 120+0+0 (secure 0 0 0) 0x7ff118011490 con 0x7ff10c004ec0 2024-09-15T01:38:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.999+0000 7ff115ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h29"}] 2024-09-15T01:38:10.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:09.999+0000 7ff115ffb640 10 monclient: _finish_command 3 = system:0 device 'h29' does not appear in the crush map 2024-09-15T01:38:10.002 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h29' does not appear in the crush map 2024-09-15T01:38:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 1 -- 172.21.15.204:0/870569765 >> v2:172.21.15.204:6812/28210 conn(0x7ff10403a7c0 msgr2=0x7ff10403cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 1 --2- 172.21.15.204:0/870569765 >> v2:172.21.15.204:6812/28210 conn(0x7ff10403a7c0 0x7ff10403cc70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0099a0 tx=0x7ff11c009120 comp rx=0 tx=0).stop 2024-09-15T01:38:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 10 monclient: shutdown 2024-09-15T01:38:10.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 1 -- 172.21.15.204:0/870569765 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff10c004ec0 msgr2=0x7ff12011e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 1 --2- 172.21.15.204:0/870569765 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff10c004ec0 0x7ff12011e980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff1200a3340 tx=0x7ff118004610 comp rx=0 tx=0).stop 2024-09-15T01:38:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 1 -- 172.21.15.204:0/870569765 shutdown_connections 2024-09-15T01:38:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 1 --2- 172.21.15.204:0/870569765 >> v2:172.21.15.204:6812/28210 conn(0x7ff10403a7c0 0x7ff10403cc70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 1 --2- 172.21.15.204:0/870569765 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff10c004ec0 0x7ff12011e980 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 1 --2- 172.21.15.204:0/870569765 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff10c004220 0x7ff12011e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 1 --2- 172.21.15.204:0/870569765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff104009f30 0x7ff12011df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 1 -- 172.21.15.204:0/870569765 >> 172.21.15.204:0/870569765 conn(0x7ff12006e860 msgr2=0x7ff120132be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 1 -- 172.21.15.204:0/870569765 shutdown_connections 2024-09-15T01:38:10.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.001+0000 7ff12819d640 1 -- 172.21.15.204:0/870569765 wait complete. 2024-09-15T01:38:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:38:10.012 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h30 2024-09-15T01:38:10.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.082+0000 7fe7c108b640 10 monclient: get_monmap_and_config 2024-09-15T01:38:10.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.082+0000 7fe7c108b640 10 monclient: build_initial_monmap 2024-09-15T01:38:10.085 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.082+0000 7fe7c108b640 10 monclient: monmap: 2024-09-15T01:38:10.085 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:10.085 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:10.085 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:10.083392+0000 2024-09-15T01:38:10.085 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:10.083392+0000 2024-09-15T01:38:10.085 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:10.085 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:10.085 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:10.085 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:10.086 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:10.086 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:10.086 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:10.086 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:10.086 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:10.087 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:10.087 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:10.087 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:10.087 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:10.087 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:10.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.083+0000 7fe7c108b640 1 Processor -- start 2024-09-15T01:38:10.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.083+0000 7fe7c108b640 1 -- start start 2024-09-15T01:38:10.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.083+0000 7fe7c108b640 10 monclient: init 2024-09-15T01:38:10.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.083+0000 7fe7c108b640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:10.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 10 monclient: _add_conns ranks=[0,7,2,9,10,4,6,5,3,1,8] 2024-09-15T01:38:10.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe7bc0a2e30 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:10.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe7bc09fb80 0x7fe7bc09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:10.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 10 monclient(hunting): picked mon.noname-i con 0x7fe7bc09fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:10.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7bc137d20 0x7fe7bc13a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:10.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 10 monclient(hunting): picked mon.noname-b con 0x7fe7bc137d20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:10.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc04b720 con 0x7fe7bc0a2e30 2024-09-15T01:38:10.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:10.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc09e6c0 con 0x7fe7bc137d20 2024-09-15T01:38:10.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:10.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0a04b0 con 0x7fe7bc09fb80 2024-09-15T01:38:10.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:10.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7c108b640 10 monclient(hunting): authenticate will time out at 1105.726083s 2024-09-15T01:38:10.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe7bc09fb80 0x7fe7bc09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:10.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe7bc09fb80 0x7fe7bc09ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:36320/0 (socket says 172.21.15.204:36320) 2024-09-15T01:38:10.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 1 -- 172.21.15.204:0/799470317 learned_addr learned my addr 172.21.15.204:0/799470317 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:10.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 10 monclient(hunting): get_auth_request con 0x7fe7bc09fb80 auth_method 0 2024-09-15T01:38:10.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:10.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:10.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:10.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7bad9d640 1 -- 172.21.15.204:0/799470317 >> v1:172.21.15.178:6789/0 conn(0x7fe7bc0a2e30 legacy=0x7fe7bc132a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:38:10.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7bb59e640 1 -- 172.21.15.204:0/799470317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7bc137d20 msgr2=0x7fe7bc13a100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:10.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7bb59e640 1 --2- 172.21.15.204:0/799470317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7bc137d20 0x7fe7bc13a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:10.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:10.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:10.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:10.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 10 monclient(hunting): handle_auth_done global_id 24757 payload 274 2024-09-15T01:38:10.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 1 -- 172.21.15.204:0/799470317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7bc137d20 msgr2=0x7fe7bc13a100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:10.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 1 --2- 172.21.15.204:0/799470317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7bc137d20 0x7fe7bc13a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:10.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 1 -- 172.21.15.204:0/799470317 >> v1:172.21.15.178:6789/0 conn(0x7fe7bc0a2e30 legacy=0x7fe7bc132a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:10.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 10 monclient: _finish_hunting 0 2024-09-15T01:38:10.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 1 monclient: found mon.noname-i 2024-09-15T01:38:10.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:10.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:38:10.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.084+0000 7fe7b259c640 1 -- 172.21.15.204:0/799470317 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc133170 con 0x7fe7bc09fb80 2024-09-15T01:38:10.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7b259c640 1 --2- 172.21.15.204:0/799470317 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe7bc09fb80 0x7fe7bc09ff70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009940 tx=0x7fe7a40091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=a85d78507f98b2a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:10.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7ba59c640 1 -- 172.21.15.204:0/799470317 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe7a402ac90 con 0x7fe7bc09fb80 2024-09-15T01:38:10.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7ba59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7ba59c640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7ba59c640 10 monclient: dump: 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:10.100 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7ba59c640 10 monclient: _finish_auth 0 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7ba59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 5 monclient: authenticate success, global_id 24757 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7ba59c640 1 -- 172.21.15.204:0/799470317 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a402ae10 con 0x7fe7bc09fb80 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7ba59c640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:10.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 4 set_mon_vals no callback set 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7ba59c640 1 -- 172.21.15.204:0/799470317 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe7a4031ca0 con 0x7fe7bc09fb80 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7ba59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7ba59c640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7ba59c640 10 monclient: dump: 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:10.102 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 10 monclient: shutdown 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 1 -- 172.21.15.204:0/799470317 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe7bc09fb80 msgr2=0x7fe7bc09ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 1 --2- 172.21.15.204:0/799470317 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe7bc09fb80 0x7fe7bc09ff70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009940 tx=0x7fe7a40091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 1 -- 172.21.15.204:0/799470317 shutdown_connections 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 1 --2- 172.21.15.204:0/799470317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7bc137d20 0x7fe7bc13a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:10.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 1 --2- 172.21.15.204:0/799470317 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe7bc09fb80 0x7fe7bc09ff70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 1 -- 172.21.15.204:0/799470317 >> 172.21.15.204:0/799470317 conn(0x7fe7bc06e860 msgr2=0x7fe7bc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 1 -- 172.21.15.204:0/799470317 shutdown_connections 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.085+0000 7fe7c108b640 1 -- 172.21.15.204:0/799470317 wait complete. 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.086+0000 7fe7c108b640 10 monclient: build_initial_monmap 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.086+0000 7fe7c108b640 10 monclient: monmap: 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:10.104 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.086+0000 7fe7c108b640 1 Processor -- start 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.086+0000 7fe7c108b640 1 -- start start 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.086+0000 7fe7c108b640 10 monclient: init 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 10 monclient: _add_conns ranks=[6,8,2,0,7,4,5,1,3] 2024-09-15T01:38:10.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe7bc09fb80 0x7fe7bc0969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 10 monclient(hunting): picked mon.noname-d con 0x7fe7bc09fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe7bc0a2e30 0x7fe7bc096f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 10 monclient(hunting): picked mon.noname-e con 0x7fe7bc0a2e30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7bc137d20 0x7fe7bc097460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 10 monclient(hunting): picked mon.noname-b con 0x7fe7bc137d20 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc099eb0 con 0x7fe7bc137d20 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc098a90 con 0x7fe7bc09fb80 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc099bd0 con 0x7fe7bc0a2e30 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7c108b640 10 monclient(hunting): authenticate will time out at 1105.729330s 2024-09-15T01:38:10.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7b259c640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe7bc0a2e30 msgr2=0x7fe7bc096f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:10.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7bb59e640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7bc137d20 msgr2=0x7fe7bc097460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:10.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7b259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe7bc0a2e30 0x7fe7bc096f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:10.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7bb59e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7bc137d20 0x7fe7bc097460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:10.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7bad9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe7bc09fb80 0x7fe7bc0969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:10.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7bad9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe7bc09fb80 0x7fe7bc0969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:47464/0 (socket says 172.21.15.204:47464) 2024-09-15T01:38:10.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7bad9d640 1 -- 172.21.15.204:0/4141461766 learned_addr learned my addr 172.21.15.204:0/4141461766 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:10.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7bad9d640 10 monclient(hunting): get_auth_request con 0x7fe7bc09fb80 auth_method 0 2024-09-15T01:38:10.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7bad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:10.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7bad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:10.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7bad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:10.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7bad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:10.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7bad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.087+0000 7fe7bad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7bad9d640 10 monclient(hunting): handle_auth_done global_id 24594 payload 824 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7bad9d640 1 -- 172.21.15.204:0/4141461766 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe7bc0a2e30 msgr2=0x7fe7bc096f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7bad9d640 1 --2- 172.21.15.204:0/4141461766 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe7bc0a2e30 0x7fe7bc096f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7bad9d640 1 -- 172.21.15.204:0/4141461766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7bc137d20 msgr2=0x7fe7bc097460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7bad9d640 1 --2- 172.21.15.204:0/4141461766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7bc137d20 0x7fe7bc097460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7bad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7bad9d640 1 monclient: found mon.noname-d 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7bad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7bad9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7bad9d640 1 -- 172.21.15.204:0/4141461766 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a4009450 con 0x7fe7bc09fb80 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7bad9d640 1 --2- 172.21.15.204:0/4141461766 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe7bc09fb80 0x7fe7bc0969e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00b5f0 tx=0x7fe7ac009120 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:10.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b37fe640 1 -- 172.21.15.204:0/4141461766 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe7ac015070 con 0x7fe7bc09fb80 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b37fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b37fe640 10 monclient: dump: 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:10.109 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b37fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b37fe640 1 -- 172.21.15.204:0/4141461766 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ac006150 con 0x7fe7bc09fb80 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7c108b640 5 monclient: authenticate success, global_id 24594 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7c108b640 10 monclient: _renew_subs 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b37fe640 1 -- 172.21.15.204:0/4141461766 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe7ac006740 con 0x7fe7bc09fb80 2024-09-15T01:38:10.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7c108b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7c108b640 1 -- 172.21.15.204:0/4141461766 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc0998f0 con 0x7fe7bc09fb80 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b37fe640 10 monclient: dump: 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:10.111 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7b9d9b640 4 set_mon_vals no callback set 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7c108b640 10 monclient: _renew_subs 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7c108b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.088+0000 7fe7c108b640 1 -- 172.21.15.204:0/4141461766 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc09ab30 con 0x7fe7bc09fb80 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.089+0000 7fe7b37fe640 1 -- 172.21.15.204:0/4141461766 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe7ac0062c0 con 0x7fe7bc09fb80 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.089+0000 7fe7c108b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:10.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.089+0000 7fe7c108b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:10.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.089+0000 7fe7c108b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:10.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.089+0000 7fe7c108b640 1 -- 172.21.15.204:0/4141461766 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7bc070cc0 con 0x7fe7bc09fb80 2024-09-15T01:38:10.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.089+0000 7fe7b37fe640 1 --2- 172.21.15.204:0/4141461766 >> v2:172.21.15.204:6812/28210 conn(0x7fe79c0397b0 0x7fe79c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:10.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.089+0000 7fe7b37fe640 1 -- 172.21.15.204:0/4141461766 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 7976+0+0 (secure 0 0 0) 0x7fe7ac04bb50 con 0x7fe7bc09fb80 2024-09-15T01:38:10.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.089+0000 7fe7b259c640 1 --2- 172.21.15.204:0/4141461766 >> v2:172.21.15.204:6812/28210 conn(0x7fe79c0397b0 0x7fe79c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:10.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.089+0000 7fe7b259c640 10 monclient: get_auth_request con 0x7fe79c0397b0 auth_method 0 2024-09-15T01:38:10.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.090+0000 7fe7b259c640 1 --2- 172.21.15.204:0/4141461766 >> v2:172.21.15.204:6812/28210 conn(0x7fe79c0397b0 0x7fe79c03bc60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4034000 tx=0x7fe7a4032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:10.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.094+0000 7fe7b37fe640 1 -- 172.21.15.204:0/4141461766 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe7ac010440 con 0x7fe7bc09fb80 2024-09-15T01:38:10.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.094+0000 7fe7b37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:10.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.094+0000 7fe7b37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:10.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.253+0000 7fe7c108b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h30"}] 2024-09-15T01:38:10.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.253+0000 7fe7c108b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h30"}] 2024-09-15T01:38:10.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.253+0000 7fe7c108b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:10.255 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.253+0000 7fe7c108b640 1 -- 172.21.15.204:0/4141461766 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h30"} v 0) v1 -- 0x7fe7bc067590 con 0x7fe7bc09fb80 2024-09-15T01:38:10.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.913+0000 7fe7b37fe640 1 -- 172.21.15.204:0/4141461766 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h30"}]=0 removed item id -106 name 'h30' from crush map v249) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe7ac00eae0 con 0x7fe7bc09fb80 2024-09-15T01:38:10.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.913+0000 7fe7b37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h30"}] 2024-09-15T01:38:10.915 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:10.913+0000 7fe7b37fe640 10 monclient: _finish_command 2 = system:0 removed item id -106 name 'h30' from crush map 2024-09-15T01:38:11.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.011+0000 7fe7c108b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h30"}] 2024-09-15T01:38:11.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.011+0000 7fe7c108b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h30"}] 2024-09-15T01:38:11.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.011+0000 7fe7c108b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:11.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.011+0000 7fe7c108b640 1 -- 172.21.15.204:0/4141461766 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h30"} v 0) v1 -- 0x7fe7bc09c950 con 0x7fe7bc09fb80 2024-09-15T01:38:11.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.012+0000 7fe7b37fe640 1 -- 172.21.15.204:0/4141461766 <== mon.6 v2:172.21.15.178:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h30"}]=0 device 'h30' does not appear in the crush map v249) v1 ==== 120+0+0 (secure 0 0 0) 0x7fe7ac00ecc0 con 0x7fe7bc09fb80 2024-09-15T01:38:11.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.012+0000 7fe7b37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h30"}] 2024-09-15T01:38:11.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.012+0000 7fe7b37fe640 10 monclient: _finish_command 3 = system:0 device 'h30' does not appear in the crush map 2024-09-15T01:38:11.014 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h30' does not appear in the crush map 2024-09-15T01:38:11.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 1 -- 172.21.15.204:0/4141461766 >> v2:172.21.15.204:6812/28210 conn(0x7fe79c0397b0 msgr2=0x7fe79c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:11.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 1 --2- 172.21.15.204:0/4141461766 >> v2:172.21.15.204:6812/28210 conn(0x7fe79c0397b0 0x7fe79c03bc60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4034000 tx=0x7fe7a4032000 comp rx=0 tx=0).stop 2024-09-15T01:38:11.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 10 monclient: shutdown 2024-09-15T01:38:11.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:11.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 1 -- 172.21.15.204:0/4141461766 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe7bc09fb80 msgr2=0x7fe7bc0969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:11.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 1 --2- 172.21.15.204:0/4141461766 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe7bc09fb80 0x7fe7bc0969e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00b5f0 tx=0x7fe7ac009120 comp rx=0 tx=0).stop 2024-09-15T01:38:11.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 1 -- 172.21.15.204:0/4141461766 shutdown_connections 2024-09-15T01:38:11.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 1 --2- 172.21.15.204:0/4141461766 >> v2:172.21.15.204:6812/28210 conn(0x7fe79c0397b0 0x7fe79c03bc60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:11.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 1 --2- 172.21.15.204:0/4141461766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7bc137d20 0x7fe7bc097460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:11.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 1 --2- 172.21.15.204:0/4141461766 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe7bc0a2e30 0x7fe7bc096f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:11.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 1 --2- 172.21.15.204:0/4141461766 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe7bc09fb80 0x7fe7bc0969e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:11.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 1 -- 172.21.15.204:0/4141461766 >> 172.21.15.204:0/4141461766 conn(0x7fe7bc06e860 msgr2=0x7fe7bc0a3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:11.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 1 -- 172.21.15.204:0/4141461766 shutdown_connections 2024-09-15T01:38:11.017 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.014+0000 7fe7c108b640 1 -- 172.21.15.204:0/4141461766 wait complete. 2024-09-15T01:38:11.024 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:38:11.024 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h31 2024-09-15T01:38:11.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.096+0000 7f9a54f54640 10 monclient: get_monmap_and_config 2024-09-15T01:38:11.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.096+0000 7f9a54f54640 10 monclient: build_initial_monmap 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.096+0000 7f9a54f54640 10 monclient: monmap: 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:11.097306+0000 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:11.097306+0000 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:11.099 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:11.100 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:11.100 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:11.100 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:11.100 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:11.100 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:11.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.097+0000 7f9a54f54640 1 Processor -- start 2024-09-15T01:38:11.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.097+0000 7f9a54f54640 1 -- start start 2024-09-15T01:38:11.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.097+0000 7f9a54f54640 10 monclient: init 2024-09-15T01:38:11.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.097+0000 7f9a54f54640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.097+0000 7f9a54f54640 10 monclient: _add_conns ranks=[6,1,8,4,0,9,3,2,7,10,5] 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.097+0000 7f9a54f54640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a500a2e30 0x7f9a50132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.097+0000 7f9a54f54640 10 monclient(hunting): picked mon.noname-d con 0x7f9a500a2e30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.097+0000 7f9a54f54640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9a5009fb80 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.097+0000 7f9a54f54640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a5009ff70 0x7f9a500a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.097+0000 7f9a54f54640 10 monclient(hunting): picked mon.noname-e con 0x7f9a5009ff70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a54f54640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5004b720 con 0x7f9a5009fb80 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a54f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a54f54640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5009e6c0 con 0x7f9a500a2e30 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a54f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a54f54640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500a3200 con 0x7f9a5009ff70 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a54f54640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a54f54640 10 monclient(hunting): authenticate will time out at 1106.740028s 2024-09-15T01:38:11.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4ed9d640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a5009ff70 msgr2=0x7f9a500a1150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a45d9b640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f9a5009fb80 0x7f9a500a0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:56916/0 (socket says 172.21.15.204:56916) 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4ed9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a5009ff70 0x7f9a500a1150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a45d9b640 1 -- 172.21.15.204:0/1525361709 learned_addr learned my addr 172.21.15.204:0/1525361709 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016556679 0 0) 0x7f9a5004b720 con 0x7f9a5009fb80 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient(hunting): my global_id is 7435 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a3c0034e0 con 0x7f9a5009fb80 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4e59c640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a500a2e30 0x7f9a50132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4e59c640 10 monclient(hunting): get_auth_request con 0x7f9a500a2e30 auth_method 0 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:11.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1970566249 0 0) 0x7f9a3c0034e0 con 0x7f9a5009fb80 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a5009ff70 msgr2=0x7f9a500a1150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9a5009ff70 0x7f9a500a1150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a500a2e30 msgr2=0x7f9a50132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a500a2e30 0x7f9a50132a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient: _finish_hunting 0 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500a1f10 con 0x7f9a5009fb80 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient: _finish_auth 0 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a54f54640 5 monclient: authenticate success, global_id 7435 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f9a380018f0 con 0x7f9a5009fb80 2024-09-15T01:38:11.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient: dump: 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:11.113 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 >> v1:172.21.15.204:6789/0 conn(0x7f9a5009fb80 legacy=0x7f9a500a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient: _add_conns ranks=[2,4,6,5,1,0,7,3,8] 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a3c004220 0x7f9a3c004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient(hunting): picked mon.b con 0x7f9a3c004220 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9a3c004de0 0x7f9a3c005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient(hunting): picked mon.c con 0x7f9a3c004de0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a3c009ce0 0x7f9a3c00c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient(hunting): picked mon.d con 0x7f9a3c009ce0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:11.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5009e6c0 con 0x7f9a3c004220 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500a3200 con 0x7f9a3c004de0 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c00c9d0 con 0x7f9a3c009ce0 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4dd9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a54f54640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4e59c640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a500a2e30 0x7f9a50132a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a45d9b640 1 -- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9a3c004de0 msgr2=0x7f9a3c005250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a45d9b640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9a3c004de0 0x7f9a3c005250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4e59c640 1 -- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a3c004220 msgr2=0x7f9a3c004780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4e59c640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a3c004220 0x7f9a3c004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:11.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4ed9d640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a3c009ce0 0x7f9a3c00c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4ed9d640 10 monclient(hunting): get_auth_request con 0x7f9a3c009ce0 auth_method 0 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.098+0000 7f9a4ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 10 monclient(hunting): handle_auth_done global_id 7435 payload 293 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 1 -- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9a3c004de0 msgr2=0x7f9a3c005250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9a3c004de0 0x7f9a3c005250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 1 -- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a3c004220 msgr2=0x7f9a3c004780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a3c004220 0x7f9a3c004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:11.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 1 monclient: found mon.d 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 1 -- 172.21.15.204:0/1525361709 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c0034e0 con 0x7f9a3c009ce0 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 10 monclient: _finish_auth 0 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.099+0000 7f9a4ed9d640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a3c009ce0 0x7f9a3c00c140 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009f90 tx=0x7f9a480095d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f1c8dfe3aea5ec68 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a480328b0 con 0x7f9a3c009ce0 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a4dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a4dd9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a4dd9b640 10 monclient: dump: 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:11.117 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:11.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a54f54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4802d070 con 0x7f9a3c009ce0 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a4dd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a54f54640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a4dd9b640 1 -- 172.21.15.204:0/1525361709 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a480313d0 con 0x7f9a3c009ce0 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a4dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a54f54640 4 set_mon_vals no callback set 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a4dd9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a4dd9b640 10 monclient: dump: 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:11.119 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a54f54640 10 monclient: shutdown 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a54f54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a54f54640 1 -- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a3c009ce0 msgr2=0x7f9a3c00c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:11.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a54f54640 1 --2- 172.21.15.204:0/1525361709 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a3c009ce0 0x7f9a3c00c140 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009f90 tx=0x7f9a480095d0 comp rx=0 tx=0).stop 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a4ed9d640 1 -- 172.21.15.204:0/1525361709 reap_dead start 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a54f54640 1 -- 172.21.15.204:0/1525361709 shutdown_connections 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a54f54640 1 -- 172.21.15.204:0/1525361709 >> 172.21.15.204:0/1525361709 conn(0x7f9a5006e860 msgr2=0x7f9a5006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a54f54640 1 -- 172.21.15.204:0/1525361709 shutdown_connections 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.100+0000 7f9a54f54640 1 -- 172.21.15.204:0/1525361709 wait complete. 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.101+0000 7f9a54f54640 10 monclient: build_initial_monmap 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.101+0000 7f9a54f54640 10 monclient: monmap: 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:11.121 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.101+0000 7f9a54f54640 1 Processor -- start 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.101+0000 7f9a54f54640 1 -- start start 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.101+0000 7f9a54f54640 10 monclient: init 2024-09-15T01:38:11.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient: _add_conns ranks=[5,4,6,3,7,2,8,0,1] 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a3c009ce0 0x7f9a50141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient(hunting): picked mon.noname-h con 0x7f9a3c009ce0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9a50142060 0x7f9a50142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient(hunting): picked mon.noname-c con 0x7f9a50142060 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a5011e090 0x7f9a5011e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient(hunting): picked mon.noname-d con 0x7f9a5011e090 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5011fc20 con 0x7f9a50142060 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5009ca40 con 0x7f9a3c009ce0 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:11.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5009d650 con 0x7f9a5011e090 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient(hunting): authenticate will time out at 1106.744196s 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a3c009ce0 0x7f9a50141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a3c009ce0 0x7f9a50141b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:34162/0 (socket says 172.21.15.204:34162) 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a45d9b640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9a50142060 msgr2=0x7f9a50142430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 1 -- 172.21.15.204:0/127301946 learned_addr learned my addr 172.21.15.204:0/127301946 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 10 monclient(hunting): get_auth_request con 0x7f9a3c009ce0 auth_method 0 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a45d9b640 1 --2- 172.21.15.204:0/127301946 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9a50142060 0x7f9a50142430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:11.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:11.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:11.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4ed9d640 1 --2- 172.21.15.204:0/127301946 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a5011e090 0x7f9a5011e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:11.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 10 monclient(hunting): handle_auth_done global_id 6566 payload 824 2024-09-15T01:38:11.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 1 -- 172.21.15.204:0/127301946 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a5011e090 msgr2=0x7f9a5011e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:11.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 1 --2- 172.21.15.204:0/127301946 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a5011e090 0x7f9a5011e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:11.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 1 -- 172.21.15.204:0/127301946 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9a50142060 msgr2=0x7f9a50142430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:11.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 1 --2- 172.21.15.204:0/127301946 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9a50142060 0x7f9a50142430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:11.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:38:11.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 1 monclient: found mon.noname-h 2024-09-15T01:38:11.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:11.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:38:11.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 1 -- 172.21.15.204:0/127301946 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a50142970 con 0x7f9a3c009ce0 2024-09-15T01:38:11.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4e59c640 1 --2- 172.21.15.204:0/127301946 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a3c009ce0 0x7f9a50141b20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000d660 tx=0x7f9a40008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:11.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a46ffd640 1 -- 172.21.15.204:0/127301946 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a4000b320 con 0x7f9a3c009ce0 2024-09-15T01:38:11.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a46ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:11.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a46ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:38:11.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a46ffd640 10 monclient: dump: 2024-09-15T01:38:11.126 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:11.126 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:11.126 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:11.126 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:11.126 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:11.126 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a46ffd640 10 monclient: _finish_auth 0 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a46ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a46ffd640 1 -- 172.21.15.204:0/127301946 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4000de60 con 0x7f9a3c009ce0 2024-09-15T01:38:11.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 5 monclient: authenticate success, global_id 6566 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient: _renew_subs 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 1 -- 172.21.15.204:0/127301946 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a50122a50 con 0x7f9a3c009ce0 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a46ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a46ffd640 1 -- 172.21.15.204:0/127301946 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a40014430 con 0x7f9a3c009ce0 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a46ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a46ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a46ffd640 10 monclient: dump: 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:11.128 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a4d59a640 4 set_mon_vals no callback set 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient: _renew_subs 2024-09-15T01:38:11.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:11.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.102+0000 7f9a54f54640 1 -- 172.21.15.204:0/127301946 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a50120420 con 0x7f9a3c009ce0 2024-09-15T01:38:11.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.103+0000 7f9a46ffd640 1 -- 172.21.15.204:0/127301946 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9a400125f0 con 0x7f9a3c009ce0 2024-09-15T01:38:11.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.103+0000 7f9a54f54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:11.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.103+0000 7f9a54f54640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:11.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.103+0000 7f9a54f54640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:11.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.103+0000 7f9a54f54640 1 -- 172.21.15.204:0/127301946 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a50121b50 con 0x7f9a3c009ce0 2024-09-15T01:38:11.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.103+0000 7f9a46ffd640 1 --2- 172.21.15.204:0/127301946 >> v2:172.21.15.204:6812/28210 conn(0x7f9a300397b0 0x7f9a3003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:11.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.103+0000 7f9a46ffd640 1 -- 172.21.15.204:0/127301946 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 7763+0+0 (secure 0 0 0) 0x7f9a4004bdb0 con 0x7f9a3c009ce0 2024-09-15T01:38:11.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.104+0000 7f9a45d9b640 1 --2- 172.21.15.204:0/127301946 >> v2:172.21.15.204:6812/28210 conn(0x7f9a300397b0 0x7f9a3003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:11.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.104+0000 7f9a45d9b640 10 monclient: get_auth_request con 0x7f9a300397b0 auth_method 0 2024-09-15T01:38:11.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.104+0000 7f9a45d9b640 1 --2- 172.21.15.204:0/127301946 >> v2:172.21.15.204:6812/28210 conn(0x7f9a300397b0 0x7f9a3003bc60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9a38001a20 tx=0x7f9a38001e00 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:11.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.108+0000 7f9a46ffd640 1 -- 172.21.15.204:0/127301946 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9a400128a0 con 0x7f9a3c009ce0 2024-09-15T01:38:11.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.108+0000 7f9a46ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:11.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.108+0000 7f9a46ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:11.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.267+0000 7f9a54f54640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h31"}] 2024-09-15T01:38:11.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.267+0000 7f9a54f54640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h31"}] 2024-09-15T01:38:11.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.267+0000 7f9a54f54640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:11.269 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.267+0000 7f9a54f54640 1 -- 172.21.15.204:0/127301946 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h31"} v 0) v1 -- 0x7f9a50070cc0 con 0x7f9a3c009ce0 2024-09-15T01:38:11.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.924+0000 7f9a46ffd640 1 -- 172.21.15.204:0/127301946 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h31"}]=0 removed item id -109 name 'h31' from crush map v250) v1 ==== 121+0+0 (secure 0 0 0) 0x7f9a40011340 con 0x7f9a3c009ce0 2024-09-15T01:38:11.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.924+0000 7f9a46ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h31"}] 2024-09-15T01:38:11.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:11.924+0000 7f9a46ffd640 10 monclient: _finish_command 2 = system:0 removed item id -109 name 'h31' from crush map 2024-09-15T01:38:12.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.022+0000 7f9a54f54640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h31"}] 2024-09-15T01:38:12.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.022+0000 7f9a54f54640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h31"}] 2024-09-15T01:38:12.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.022+0000 7f9a54f54640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:12.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.022+0000 7f9a54f54640 1 -- 172.21.15.204:0/127301946 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h31"} v 0) v1 -- 0x7f9a5009ddb0 con 0x7f9a3c009ce0 2024-09-15T01:38:12.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.023+0000 7f9a46ffd640 1 -- 172.21.15.204:0/127301946 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h31"}]=0 device 'h31' does not appear in the crush map v250) v1 ==== 120+0+0 (secure 0 0 0) 0x7f9a4001cb60 con 0x7f9a3c009ce0 2024-09-15T01:38:12.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.023+0000 7f9a46ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h31"}] 2024-09-15T01:38:12.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.023+0000 7f9a46ffd640 10 monclient: _finish_command 3 = system:0 device 'h31' does not appear in the crush map 2024-09-15T01:38:12.025 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h31' does not appear in the crush map 2024-09-15T01:38:12.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 1 -- 172.21.15.204:0/127301946 >> v2:172.21.15.204:6812/28210 conn(0x7f9a300397b0 msgr2=0x7f9a3003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:12.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 1 --2- 172.21.15.204:0/127301946 >> v2:172.21.15.204:6812/28210 conn(0x7f9a300397b0 0x7f9a3003bc60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9a38001a20 tx=0x7f9a38001e00 comp rx=0 tx=0).stop 2024-09-15T01:38:12.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 10 monclient: shutdown 2024-09-15T01:38:12.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:12.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 1 -- 172.21.15.204:0/127301946 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a3c009ce0 msgr2=0x7f9a50141b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:12.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 1 --2- 172.21.15.204:0/127301946 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a3c009ce0 0x7f9a50141b20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000d660 tx=0x7f9a40008040 comp rx=0 tx=0).stop 2024-09-15T01:38:12.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 1 -- 172.21.15.204:0/127301946 shutdown_connections 2024-09-15T01:38:12.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 1 --2- 172.21.15.204:0/127301946 >> v2:172.21.15.204:6812/28210 conn(0x7f9a300397b0 0x7f9a3003bc60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:12.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 1 --2- 172.21.15.204:0/127301946 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9a5011e090 0x7f9a5011e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:12.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 1 --2- 172.21.15.204:0/127301946 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9a50142060 0x7f9a50142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:12.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 1 --2- 172.21.15.204:0/127301946 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9a3c009ce0 0x7f9a50141b20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:12.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 1 -- 172.21.15.204:0/127301946 >> 172.21.15.204:0/127301946 conn(0x7f9a5006e860 msgr2=0x7f9a5009ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:12.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 1 -- 172.21.15.204:0/127301946 shutdown_connections 2024-09-15T01:38:12.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.025+0000 7f9a54f54640 1 -- 172.21.15.204:0/127301946 wait complete. 2024-09-15T01:38:12.035 INFO:tasks.workunit.client.0.smithi204.stderr:+ for f in `seq 1 32` 2024-09-15T01:38:12.035 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm h32 2024-09-15T01:38:12.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.106+0000 7f4450c6f640 10 monclient: get_monmap_and_config 2024-09-15T01:38:12.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.106+0000 7f4450c6f640 10 monclient: build_initial_monmap 2024-09-15T01:38:12.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.106+0000 7f4450c6f640 10 monclient: monmap: 2024-09-15T01:38:12.108 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:12.108 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:12.108 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:12.107000+0000 2024-09-15T01:38:12.108 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:12.107000+0000 2024-09-15T01:38:12.108 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:12.108 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:12.108 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:12.109 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:12.109 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:12.109 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:12.109 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:12.109 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:12.109 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:12.109 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:12.109 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:12.109 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:12.109 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:12.109 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 1 Processor -- start 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 1 -- start start 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 10 monclient: init 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 10 monclient: _add_conns ranks=[3,7,4,6,2,10,9,5,1,8,0] 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a2e30 0x7f444c132a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 10 monclient(hunting): picked mon.noname-g con 0x7f444c0a2e30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f444c09fb80 0x7f444c09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 10 monclient(hunting): picked mon.noname-i con 0x7f444c09fb80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f444c0a0a40 0x7f444c0a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 10 monclient(hunting): picked mon.noname-c con 0x7f444c0a0a40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:12.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c09e6c0 con 0x7f444c0a2e30 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0a04b0 con 0x7f444c0a0a40 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0a3200 con 0x7f444c09fb80 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4450c6f640 10 monclient(hunting): authenticate will time out at 1107.749844s 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f444a59c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a2e30 0x7f444c132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4449d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f444c09fb80 0x7f444c09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f444a59c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a2e30 0x7f444c132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:34146/0 (socket says 172.21.15.204:34146) 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f444ad9d640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f444c0a0a40 msgr2=0x7f444c0a0e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f4449d9b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f444c09fb80 0x7f444c09ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:36346/0 (socket says 172.21.15.204:36346) 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f444a59c640 1 -- 172.21.15.204:0/3617805855 learned_addr learned my addr 172.21.15.204:0/3617805855 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:12.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.107+0000 7f444a59c640 10 monclient(hunting): get_auth_request con 0x7f444c0a2e30 auth_method 0 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f4449d9b640 10 monclient(hunting): get_auth_request con 0x7f444c09fb80 auth_method 0 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f4449d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f4449d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f4449d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444ad9d640 1 --2- 172.21.15.204:0/3617805855 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f444c0a0a40 0x7f444c0a0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f4449d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f4449d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f4449d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:12.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:12.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:12.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 10 monclient(hunting): handle_auth_done global_id 15672 payload 274 2024-09-15T01:38:12.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 1 -- 172.21.15.204:0/3617805855 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f444c09fb80 msgr2=0x7f444c09ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:12.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 1 --2- 172.21.15.204:0/3617805855 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f444c09fb80 0x7f444c09ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:12.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 1 -- 172.21.15.204:0/3617805855 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f444c0a0a40 msgr2=0x7f444c0a0e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:12.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 1 --2- 172.21.15.204:0/3617805855 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f444c0a0a40 0x7f444c0a0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:12.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 10 monclient: _finish_hunting 0 2024-09-15T01:38:12.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 1 monclient: found mon.noname-g 2024-09-15T01:38:12.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:12.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:12.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 1 -- 172.21.15.204:0/3617805855 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c04b720 con 0x7f444c0a2e30 2024-09-15T01:38:12.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f4449d9b640 1 --2- 172.21.15.204:0/3617805855 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f444c09fb80 0x7f444c09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:12.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.108+0000 7f444a59c640 1 --2- 172.21.15.204:0/3617805855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a2e30 0x7f444c132a60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f443c009940 tx=0x7f443c0091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=ff0ed6af5229b4fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:12.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f444959a640 1 -- 172.21.15.204:0/3617805855 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f443c02ac80 con 0x7f444c0a2e30 2024-09-15T01:38:12.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f444959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:12.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f444959a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:12.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f444959a640 10 monclient: dump: 2024-09-15T01:38:12.122 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:12.122 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:12.123 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f444959a640 10 monclient: _finish_auth 0 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f444959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f444959a640 1 -- 172.21.15.204:0/3617805855 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c02ae00 con 0x7f444c0a2e30 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 5 monclient: authenticate success, global_id 15672 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f444959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f444959a640 1 -- 172.21.15.204:0/3617805855 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f443c031ca0 con 0x7f444c0a2e30 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f444959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 4 set_mon_vals no callback set 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f444959a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f444959a640 10 monclient: dump: 2024-09-15T01:38:12.124 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:12.125 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 10 monclient: shutdown 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 1 -- 172.21.15.204:0/3617805855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a2e30 msgr2=0x7f444c132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 1 --2- 172.21.15.204:0/3617805855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a2e30 0x7f444c132a60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f443c009940 tx=0x7f443c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 1 -- 172.21.15.204:0/3617805855 shutdown_connections 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 1 --2- 172.21.15.204:0/3617805855 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f444c09fb80 0x7f444c09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 1 --2- 172.21.15.204:0/3617805855 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f444c0a0a40 0x7f444c0a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 1 --2- 172.21.15.204:0/3617805855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a2e30 0x7f444c132a60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 1 -- 172.21.15.204:0/3617805855 >> 172.21.15.204:0/3617805855 conn(0x7f444c06e860 msgr2=0x7f444c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 1 -- 172.21.15.204:0/3617805855 shutdown_connections 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.109+0000 7f4450c6f640 1 -- 172.21.15.204:0/3617805855 wait complete. 2024-09-15T01:38:12.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.110+0000 7f4450c6f640 10 monclient: build_initial_monmap 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.110+0000 7f4450c6f640 10 monclient: monmap: 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:12.127 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.110+0000 7f4450c6f640 1 Processor -- start 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.110+0000 7f4450c6f640 1 -- start start 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.110+0000 7f4450c6f640 10 monclient: init 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 10 monclient: _add_conns ranks=[6,3,1,2,4,7,8,5,0] 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f444c09fb80 0x7f444c141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 10 monclient(hunting): picked mon.noname-d con 0x7f444c09fb80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a0a40 0x7f444c131ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 10 monclient(hunting): picked mon.noname-g con 0x7f444c0a0a40 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f444c0a2e30 0x7f444c11e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:12.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 10 monclient(hunting): picked mon.noname-f con 0x7f444c0a2e30 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c121c00 con 0x7f444c0a2e30 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c09ca40 con 0x7f444c0a0a40 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f444c09d650 con 0x7f444c09fb80 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 10 monclient(hunting): authenticate will time out at 1107.753207s 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a0a40 0x7f444c131ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a0a40 0x7f444c131ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:34148/0 (socket says 172.21.15.204:34148) 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 1 -- 172.21.15.204:0/3043232167 learned_addr learned my addr 172.21.15.204:0/3043232167 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 10 monclient(hunting): get_auth_request con 0x7f444c0a0a40 auth_method 0 2024-09-15T01:38:12.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444ad9d640 1 --2- 172.21.15.204:0/3043232167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f444c0a2e30 0x7f444c11e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444ad9d640 10 monclient(hunting): get_auth_request con 0x7f444c0a2e30 auth_method 0 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:12.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444a59c640 1 --2- 172.21.15.204:0/3043232167 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f444c09fb80 0x7f444c141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444a59c640 10 monclient(hunting): get_auth_request con 0x7f444c09fb80 auth_method 0 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 10 monclient(hunting): handle_auth_done global_id 15681 payload 824 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 1 -- 172.21.15.204:0/3043232167 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f444c09fb80 msgr2=0x7f444c141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 1 --2- 172.21.15.204:0/3043232167 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f444c09fb80 0x7f444c141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 1 -- 172.21.15.204:0/3043232167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f444c0a2e30 msgr2=0x7f444c11e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 1 --2- 172.21.15.204:0/3043232167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f444c0a2e30 0x7f444c11e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:12.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 1 monclient: found mon.noname-g 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 1 -- 172.21.15.204:0/3043232167 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4440009390 con 0x7f444c0a0a40 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f444ad9d640 1 --2- 172.21.15.204:0/3043232167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f444c0a2e30 0x7f444c11e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4449d9b640 1 --2- 172.21.15.204:0/3043232167 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a0a40 0x7f444c131ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f444000ec50 tx=0x7f44400090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f443a7fc640 1 -- 172.21.15.204:0/3043232167 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f444000ee90 con 0x7f444c0a0a40 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f443a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f443a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f443a7fc640 10 monclient: dump: 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:12.132 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f443a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:12.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f443a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f443a7fc640 1 -- 172.21.15.204:0/3043232167 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444000cbc0 con 0x7f444c0a0a40 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f443a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f443a7fc640 1 -- 172.21.15.204:0/3043232167 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4440013d70 con 0x7f444c0a0a40 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4450c6f640 5 monclient: authenticate success, global_id 15681 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f443a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f443a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f4448d99640 4 set_mon_vals no callback set 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.111+0000 7f443a7fc640 10 monclient: dump: 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:12.134 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.112+0000 7f4450c6f640 10 monclient: _renew_subs 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.112+0000 7f4450c6f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:12.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.112+0000 7f4450c6f640 1 -- 172.21.15.204:0/3043232167 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443c009450 con 0x7f444c0a0a40 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.112+0000 7f4450c6f640 10 monclient: _renew_subs 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.112+0000 7f4450c6f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.112+0000 7f4450c6f640 1 -- 172.21.15.204:0/3043232167 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c1200d0 con 0x7f444c0a0a40 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.113+0000 7f443a7fc640 1 -- 172.21.15.204:0/3043232167 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f444000cd30 con 0x7f444c0a0a40 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.113+0000 7f4450c6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.113+0000 7f4450c6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.113+0000 7f4450c6f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.113+0000 7f4450c6f640 1 -- 172.21.15.204:0/3043232167 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c121f40 con 0x7f444c0a0a40 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.113+0000 7f443a7fc640 1 --2- 172.21.15.204:0/3043232167 >> v2:172.21.15.204:6812/28210 conn(0x7f442c0397b0 0x7f442c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.113+0000 7f443a7fc640 1 -- 172.21.15.204:0/3043232167 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 7550+0+0 (secure 0 0 0) 0x7f444004c290 con 0x7f444c0a0a40 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.113+0000 7f444a59c640 1 --2- 172.21.15.204:0/3043232167 >> v2:172.21.15.204:6812/28210 conn(0x7f442c0397b0 0x7f442c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.113+0000 7f444a59c640 10 monclient: get_auth_request con 0x7f442c0397b0 auth_method 0 2024-09-15T01:38:12.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.113+0000 7f444a59c640 1 --2- 172.21.15.204:0/3043232167 >> v2:172.21.15.204:6812/28210 conn(0x7f442c0397b0 0x7f442c03bc60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f443c034000 tx=0x7f443c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:12.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.117+0000 7f443a7fc640 1 -- 172.21.15.204:0/3043232167 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f44400238f0 con 0x7f444c0a0a40 2024-09-15T01:38:12.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.117+0000 7f443a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:12.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.117+0000 7f443a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:12.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.278+0000 7f4450c6f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h32"}] 2024-09-15T01:38:12.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.278+0000 7f4450c6f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h32"}] 2024-09-15T01:38:12.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.278+0000 7f4450c6f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:12.279 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.278+0000 7f4450c6f640 1 -- 172.21.15.204:0/3043232167 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h32"} v 0) v1 -- 0x7f444c070cc0 con 0x7f444c0a0a40 2024-09-15T01:38:12.930 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.929+0000 7f443a7fc640 1 -- 172.21.15.204:0/3043232167 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h32"}]=0 removed item id -112 name 'h32' from crush map v251) v1 ==== 121+0+0 (secure 0 0 0) 0x7f444001b3b0 con 0x7f444c0a0a40 2024-09-15T01:38:12.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.929+0000 7f443a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h32"}] 2024-09-15T01:38:12.931 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:12.929+0000 7f443a7fc640 10 monclient: _finish_command 2 = system:0 removed item id -112 name 'h32' from crush map 2024-09-15T01:38:13.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.028+0000 7f4450c6f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h32"}] 2024-09-15T01:38:13.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.028+0000 7f4450c6f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h32"}] 2024-09-15T01:38:13.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.028+0000 7f4450c6f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:13.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.028+0000 7f4450c6f640 1 -- 172.21.15.204:0/3043232167 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h32"} v 0) v1 -- 0x7f444c0a1f10 con 0x7f444c0a0a40 2024-09-15T01:38:13.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.029+0000 7f443a7fc640 1 -- 172.21.15.204:0/3043232167 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h32"}]=0 device 'h32' does not appear in the crush map v251) v1 ==== 120+0+0 (secure 0 0 0) 0x7f44400232f0 con 0x7f444c0a0a40 2024-09-15T01:38:13.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.029+0000 7f443a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h32"}] 2024-09-15T01:38:13.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.029+0000 7f443a7fc640 10 monclient: _finish_command 3 = system:0 device 'h32' does not appear in the crush map 2024-09-15T01:38:13.030 INFO:tasks.workunit.client.0.smithi204.stderr:device 'h32' does not appear in the crush map 2024-09-15T01:38:13.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.030+0000 7f4450c6f640 1 -- 172.21.15.204:0/3043232167 >> v2:172.21.15.204:6812/28210 conn(0x7f442c0397b0 msgr2=0x7f442c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:13.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.030+0000 7f4450c6f640 1 --2- 172.21.15.204:0/3043232167 >> v2:172.21.15.204:6812/28210 conn(0x7f442c0397b0 0x7f442c03bc60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f443c034000 tx=0x7f443c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:13.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.030+0000 7f4450c6f640 10 monclient: shutdown 2024-09-15T01:38:13.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.030+0000 7f4450c6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:13.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.030+0000 7f4450c6f640 1 -- 172.21.15.204:0/3043232167 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a0a40 msgr2=0x7f444c131ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:13.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.030+0000 7f4450c6f640 1 --2- 172.21.15.204:0/3043232167 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a0a40 0x7f444c131ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f444000ec50 tx=0x7f44400090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:13.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.030+0000 7f4450c6f640 1 -- 172.21.15.204:0/3043232167 shutdown_connections 2024-09-15T01:38:13.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.030+0000 7f4450c6f640 1 --2- 172.21.15.204:0/3043232167 >> v2:172.21.15.204:6812/28210 conn(0x7f442c0397b0 0x7f442c03bc60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:13.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.030+0000 7f4450c6f640 1 --2- 172.21.15.204:0/3043232167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f444c0a2e30 0x7f444c11e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:13.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.030+0000 7f4450c6f640 1 --2- 172.21.15.204:0/3043232167 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c0a0a40 0x7f444c131ab0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:13.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.031+0000 7f4450c6f640 1 --2- 172.21.15.204:0/3043232167 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f444c09fb80 0x7f444c141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:13.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.031+0000 7f4450c6f640 1 -- 172.21.15.204:0/3043232167 >> 172.21.15.204:0/3043232167 conn(0x7f444c06e860 msgr2=0x7f444c09bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:13.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.031+0000 7f4450c6f640 1 -- 172.21.15.204:0/3043232167 shutdown_connections 2024-09-15T01:38:13.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.031+0000 7f4450c6f640 1 -- 172.21.15.204:0/3043232167 wait complete. 2024-09-15T01:38:13.041 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush rm r1 2024-09-15T01:38:13.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.112+0000 7fd6f0100640 10 monclient: get_monmap_and_config 2024-09-15T01:38:13.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.112+0000 7fd6f0100640 10 monclient: build_initial_monmap 2024-09-15T01:38:13.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.112+0000 7fd6f0100640 10 monclient: monmap: 2024-09-15T01:38:13.114 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:13.114 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:13.113264+0000 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:13.113264+0000 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:13.115 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:13.116 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:13.116 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:13.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 1 Processor -- start 2024-09-15T01:38:13.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 1 -- start start 2024-09-15T01:38:13.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 10 monclient: init 2024-09-15T01:38:13.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:13.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 10 monclient: _add_conns ranks=[1,5,4,10,0,7,8,6,9,2,3] 2024-09-15T01:38:13.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd6e80a2e30 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:13.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6e809fb80 0x7fd6e809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:13.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 10 monclient(hunting): picked mon.noname-h con 0x7fd6e809fb80 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd6e8137d20 0x7fd6e813a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 10 monclient(hunting): picked mon.noname-c con 0x7fd6e8137d20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e804b720 con 0x7fd6e80a2e30 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e809e6c0 con 0x7fd6e8137d20 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.113+0000 7fd6f0100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6f0100640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e80a04b0 con 0x7fd6e809fb80 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6f0100640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6f0100640 10 monclient(hunting): authenticate will time out at 1108.756013s 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6e809fb80 0x7fd6e809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ede9c640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fd6e80a2e30 0x7fd6e8132a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:56926/0 (socket says 172.21.15.204:56926) 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ede9c640 1 -- 172.21.15.204:0/582546615 learned_addr learned my addr 172.21.15.204:0/582546615 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:38:13.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 10 monclient(hunting): get_auth_request con 0x7fd6e809fb80 auth_method 0 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ee69d640 1 -- 172.21.15.204:0/582546615 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd6e8137d20 msgr2=0x7fd6e813a100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ee69d640 1 --2- 172.21.15.204:0/582546615 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd6e8137d20 0x7fd6e813a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ece9a640 1 -- 172.21.15.204:0/582546615 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072216737 0 0) 0x7fd6e804b720 con 0x7fd6e80a2e30 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ece9a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ece9a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ece9a640 10 monclient(hunting): my global_id is 7453 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ece9a640 1 -- 172.21.15.204:0/582546615 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6d40034e0 con 0x7fd6e80a2e30 2024-09-15T01:38:13.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 10 monclient(hunting): handle_auth_done global_id 6575 payload 274 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 1 -- 172.21.15.204:0/582546615 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd6e8137d20 msgr2=0x7fd6e813a100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 1 --2- 172.21.15.204:0/582546615 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd6e8137d20 0x7fd6e813a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 1 -- 172.21.15.204:0/582546615 >> v1:172.21.15.204:6789/0 conn(0x7fd6e80a2e30 legacy=0x7fd6e8132a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 10 monclient: _finish_hunting 0 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 1 monclient: found mon.noname-h 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.114+0000 7fd6ed69b640 1 -- 172.21.15.204:0/582546615 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8133170 con 0x7fd6e809fb80 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ed69b640 1 --2- 172.21.15.204:0/582546615 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6e809fb80 0x7fd6e809ff70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009880 tx=0x7fd6d80090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=51cc8ec3d230eb29 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ece9a640 1 -- 172.21.15.204:0/582546615 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6d802ac80 con 0x7fd6e809fb80 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ece9a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ece9a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:38:13.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ece9a640 10 monclient: dump: 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:13.125 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ece9a640 10 monclient: _finish_auth 0 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ece9a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ece9a640 1 -- 172.21.15.204:0/582546615 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d802ae00 con 0x7fd6e809fb80 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 5 monclient: authenticate success, global_id 6575 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ece9a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ece9a640 1 -- 172.21.15.204:0/582546615 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6d8031ca0 con 0x7fd6e809fb80 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 4 set_mon_vals no callback set 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ece9a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:13.126 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ece9a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6ece9a640 10 monclient: dump: 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:13.127 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 10 monclient: shutdown 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 1 -- 172.21.15.204:0/582546615 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6e809fb80 msgr2=0x7fd6e809ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 1 --2- 172.21.15.204:0/582546615 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6e809fb80 0x7fd6e809ff70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009880 tx=0x7fd6d80090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 1 -- 172.21.15.204:0/582546615 shutdown_connections 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 1 --2- 172.21.15.204:0/582546615 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd6e8137d20 0x7fd6e813a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 1 --2- 172.21.15.204:0/582546615 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd6e809fb80 0x7fd6e809ff70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 1 -- 172.21.15.204:0/582546615 >> 172.21.15.204:0/582546615 conn(0x7fd6e806e860 msgr2=0x7fd6e806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 1 -- 172.21.15.204:0/582546615 shutdown_connections 2024-09-15T01:38:13.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.115+0000 7fd6f0100640 1 -- 172.21.15.204:0/582546615 wait complete. 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.116+0000 7fd6f0100640 10 monclient: build_initial_monmap 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.116+0000 7fd6f0100640 10 monclient: monmap: 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:13.129 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.116+0000 7fd6f0100640 1 Processor -- start 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.116+0000 7fd6f0100640 1 -- start start 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.116+0000 7fd6f0100640 10 monclient: init 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 10 monclient: _add_conns ranks=[8,0,1,2,7,5,4,6,3] 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6e809fb80 0x7fd6e8142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 10 monclient(hunting): picked mon.noname-e con 0x7fd6e809fb80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6e80a2e30 0x7fd6e81426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 10 monclient(hunting): picked mon.noname-a con 0x7fd6e80a2e30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6e8137d20 0x7fd6e8142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:13.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 10 monclient(hunting): picked mon.noname-f con 0x7fd6e8137d20 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e811cdf0 con 0x7fd6e80a2e30 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e809d650 con 0x7fd6e8137d20 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e811cb10 con 0x7fd6e809fb80 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6f0100640 10 monclient(hunting): authenticate will time out at 1108.759140s 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6e8137d20 0x7fd6e8142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ede9c640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6e809fb80 msgr2=0x7fd6e8142190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6e8137d20 0x7fd6e8142c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:46230/0 (socket says 172.21.15.204:46230) 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ed69b640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6e80a2e30 msgr2=0x7fd6e81426d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 1 -- 172.21.15.204:0/598330223 learned_addr learned my addr 172.21.15.204:0/598330223 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:13.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 10 monclient(hunting): get_auth_request con 0x7fd6e8137d20 auth_method 0 2024-09-15T01:38:13.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ede9c640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6e809fb80 0x7fd6e8142190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:13.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:13.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:13.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:13.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ed69b640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6e80a2e30 0x7fd6e81426d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:13.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:13.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:13.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:13.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 1 -- 172.21.15.204:0/598330223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6e8137d20 msgr2=0x7fd6e8142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:38:13.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 1 -- 172.21.15.204:0/598330223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6e8137d20 msgr2=0x7fd6e8142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:13.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6e8137d20 0x7fd6e8142c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:13.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.117+0000 7fd6ee69d640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6e8137d20 0x7fd6e8142c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6e8137d20 0x7fd6e8142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 10 monclient(hunting): get_auth_request con 0x7fd6e8137d20 auth_method 0 2024-09-15T01:38:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ede9c640 1 -- 172.21.15.204:0/598330223 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6e809fb80 msgr2=0x7fd6e8142190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:13.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ed69b640 1 -- 172.21.15.204:0/598330223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6e80a2e30 msgr2=0x7fd6e81426d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ede9c640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6e809fb80 0x7fd6e8142190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ed69b640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6e80a2e30 0x7fd6e81426d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 10 monclient(hunting): handle_auth_done global_id 7471 payload 824 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 1 -- 172.21.15.204:0/598330223 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6e809fb80 msgr2=0x7fd6e8142190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6e809fb80 0x7fd6e8142190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 1 -- 172.21.15.204:0/598330223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6e80a2e30 msgr2=0x7fd6e81426d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6e80a2e30 0x7fd6e81426d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 1 monclient: found mon.noname-f 2024-09-15T01:38:13.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:13.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:38:13.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 1 -- 172.21.15.204:0/598330223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d8009390 con 0x7fd6e8137d20 2024-09-15T01:38:13.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ee69d640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6e8137d20 0x7fd6e8142c10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00099e0 tx=0x7fd6e000d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:13.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ddffb640 1 -- 172.21.15.204:0/598330223 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6e0010040 con 0x7fd6e8137d20 2024-09-15T01:38:13.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:13.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ddffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:38:13.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ddffb640 10 monclient: dump: 2024-09-15T01:38:13.324 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:13.324 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:13.324 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:13.324 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:13.324 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:13.325 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:13.325 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:13.325 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:13.325 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:13.325 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:13.325 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:13.325 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:13.325 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:13.325 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:13.325 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:13.325 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:13.325 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ddffb640 10 monclient: _finish_auth 0 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ddffb640 1 -- 172.21.15.204:0/598330223 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e00069b0 con 0x7fd6e8137d20 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6f0100640 5 monclient: authenticate success, global_id 7471 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6f0100640 10 monclient: _renew_subs 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6f0100640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6f0100640 1 -- 172.21.15.204:0/598330223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e811d900 con 0x7fd6e8137d20 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ddffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ddffb640 1 -- 172.21.15.204:0/598330223 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6e0006b20 con 0x7fd6e8137d20 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ddffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6ddffb640 10 monclient: dump: 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:13.326 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:13.327 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.318+0000 7fd6dffff640 4 set_mon_vals no callback set 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.319+0000 7fd6f0100640 10 monclient: _renew_subs 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.319+0000 7fd6f0100640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.319+0000 7fd6f0100640 1 -- 172.21.15.204:0/598330223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e809c920 con 0x7fd6e8137d20 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.319+0000 7fd6ddffb640 1 -- 172.21.15.204:0/598330223 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd6e0013380 con 0x7fd6e8137d20 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.319+0000 7fd6ddffb640 1 --2- 172.21.15.204:0/598330223 >> v2:172.21.15.204:6812/28210 conn(0x7fd6cc0397e0 0x7fd6cc03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.319+0000 7fd6ddffb640 1 -- 172.21.15.204:0/598330223 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 7337+0+0 (secure 0 0 0) 0x7fd6e004bca0 con 0x7fd6e8137d20 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.319+0000 7fd6f0100640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.319+0000 7fd6f0100640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.319+0000 7fd6f0100640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.319+0000 7fd6f0100640 1 -- 172.21.15.204:0/598330223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e8070cc0 con 0x7fd6e8137d20 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.320+0000 7fd6ede9c640 1 --2- 172.21.15.204:0/598330223 >> v2:172.21.15.204:6812/28210 conn(0x7fd6cc0397e0 0x7fd6cc03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:13.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.320+0000 7fd6ede9c640 10 monclient: get_auth_request con 0x7fd6cc0397e0 auth_method 0 2024-09-15T01:38:13.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.320+0000 7fd6ede9c640 1 --2- 172.21.15.204:0/598330223 >> v2:172.21.15.204:6812/28210 conn(0x7fd6cc0397e0 0x7fd6cc03bc90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4007fd0 tx=0x7fd6e4007b00 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:13.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.324+0000 7fd6ddffb640 1 -- 172.21.15.204:0/598330223 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd6e001b070 con 0x7fd6e8137d20 2024-09-15T01:38:13.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.324+0000 7fd6ddffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:13.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.324+0000 7fd6ddffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:13.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.480+0000 7fd6f0100640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "r1"}] 2024-09-15T01:38:13.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.480+0000 7fd6f0100640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "r1"}] 2024-09-15T01:38:13.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.480+0000 7fd6f0100640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:13.482 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.480+0000 7fd6f0100640 1 -- 172.21.15.204:0/598330223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "r1"} v 0) v1 -- 0x7fd6e809b9e0 con 0x7fd6e8137d20 2024-09-15T01:38:13.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.932+0000 7fd6ddffb640 1 -- 172.21.15.204:0/598330223 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "r1"}]=0 removed item id -16 name 'r1' from crush map v252) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd6e0007aa0 con 0x7fd6e8137d20 2024-09-15T01:38:13.933 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.932+0000 7fd6ddffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "r1"}] 2024-09-15T01:38:13.934 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:13.932+0000 7fd6ddffb640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'r1' from crush map 2024-09-15T01:38:14.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.028+0000 7fd6f0100640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "r1"}] 2024-09-15T01:38:14.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.028+0000 7fd6f0100640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "r1"}] 2024-09-15T01:38:14.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.028+0000 7fd6f0100640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:14.030 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.028+0000 7fd6f0100640 1 -- 172.21.15.204:0/598330223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "r1"} v 0) v1 -- 0x7fd6e8070f60 con 0x7fd6e8137d20 2024-09-15T01:38:14.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.029+0000 7fd6ddffb640 1 -- 172.21.15.204:0/598330223 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "r1"}]=0 device 'r1' does not appear in the crush map v252) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd6e00072c0 con 0x7fd6e8137d20 2024-09-15T01:38:14.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.029+0000 7fd6ddffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "r1"}] 2024-09-15T01:38:14.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.029+0000 7fd6ddffb640 10 monclient: _finish_command 3 = system:0 device 'r1' does not appear in the crush map 2024-09-15T01:38:14.031 INFO:tasks.workunit.client.0.smithi204.stderr:device 'r1' does not appear in the crush map 2024-09-15T01:38:14.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 1 -- 172.21.15.204:0/598330223 >> v2:172.21.15.204:6812/28210 conn(0x7fd6cc0397e0 msgr2=0x7fd6cc03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 1 --2- 172.21.15.204:0/598330223 >> v2:172.21.15.204:6812/28210 conn(0x7fd6cc0397e0 0x7fd6cc03bc90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4007fd0 tx=0x7fd6e4007b00 comp rx=0 tx=0).stop 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 10 monclient: shutdown 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 1 -- 172.21.15.204:0/598330223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6e8137d20 msgr2=0x7fd6e8142c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6e8137d20 0x7fd6e8142c10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00099e0 tx=0x7fd6e000d040 comp rx=0 tx=0).stop 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 1 -- 172.21.15.204:0/598330223 shutdown_connections 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6e8137d20 0x7fd6e8142c10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 1 --2- 172.21.15.204:0/598330223 >> v2:172.21.15.204:6812/28210 conn(0x7fd6cc0397e0 0x7fd6cc03bc90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6e80a2e30 0x7fd6e81426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 1 --2- 172.21.15.204:0/598330223 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd6e809fb80 0x7fd6e8142190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 1 -- 172.21.15.204:0/598330223 >> 172.21.15.204:0/598330223 conn(0x7fd6e806e860 msgr2=0x7fd6e80a0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 1 -- 172.21.15.204:0/598330223 shutdown_connections 2024-09-15T01:38:14.033 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.031+0000 7fd6f0100640 1 -- 172.21.15.204:0/598330223 wait complete. 2024-09-15T01:38:14.041 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd crush weight-set rm-compat 2024-09-15T01:38:14.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.112+0000 7f8734f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:14.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.112+0000 7f8734f9d640 10 monclient: build_initial_monmap 2024-09-15T01:38:14.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.112+0000 7f8734f9d640 10 monclient: monmap: 2024-09-15T01:38:14.114 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:14.114 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:14.114 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:14.112678+0000 2024-09-15T01:38:14.114 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:14.112678+0000 2024-09-15T01:38:14.114 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:14.114 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:14.114 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:14.114 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:14.114 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:14.114 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:14.114 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:14.114 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:14.115 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:14.115 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:14.115 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:14.115 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:14.115 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:14.115 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:14.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.112+0000 7f8734f9d640 1 Processor -- start 2024-09-15T01:38:14.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.112+0000 7f8734f9d640 1 -- start start 2024-09-15T01:38:14.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.112+0000 7f8734f9d640 10 monclient: init 2024-09-15T01:38:14.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:14.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 10 monclient: _add_conns ranks=[2,9,8,6,1,5,10,7,0,3,4] 2024-09-15T01:38:14.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87300a3560 0x7f873009b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:14.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f87300a3560 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:14.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f873009cca0 0x7f8730067130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:14.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 10 monclient(hunting): picked mon.noname-a con 0x7f873009cca0 addr v2:172.21.15.178:3300/0 2024-09-15T01:38:14.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f873009d090 0x7f8730067670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:14.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f873009d090 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:14.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:14.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873009e120 con 0x7f873009cca0 2024-09-15T01:38:14.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:14.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8730099f10 con 0x7f87300a3560 2024-09-15T01:38:14.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:14.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f873009ab00 con 0x7f873009d090 2024-09-15T01:38:14.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:14.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f8734f9d640 10 monclient(hunting): authenticate will time out at 1109.755459s 2024-09-15T01:38:14.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f872ed9d640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f873009d090 msgr2=0x7f8730067670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:14.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f872dd9b640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7f873009cca0 msgr2=0x7f8730067130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:14.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f872e59c640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87300a3560 msgr2=0x7f873009b480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:14.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f872dd9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f873009cca0 0x7f8730067130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:14.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f872e59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87300a3560 0x7f873009b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:14.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.113+0000 7f872ed9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f873009d090 0x7f8730067670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:14.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.314+0000 7f872dd9b640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7f873009cca0 msgr2=0x7f8730067130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:14.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.314+0000 7f872e59c640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87300a3560 msgr2=0x7f873009b480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:14.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.314+0000 7f872ed9d640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f873009d090 msgr2=0x7f8730067670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:14.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.314+0000 7f872dd9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f873009cca0 0x7f8730067130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:14.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.314+0000 7f872ed9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f873009d090 0x7f8730067670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:14.316 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.314+0000 7f872e59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87300a3560 0x7f873009b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:14.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.715+0000 7f872ed9d640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f873009d090 msgr2=0x7f8730067670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:14.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.715+0000 7f872dd9b640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7f873009cca0 msgr2=0x7f8730067130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:14.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.715+0000 7f872e59c640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87300a3560 msgr2=0x7f873009b480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:14.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.715+0000 7f872ed9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f873009d090 0x7f8730067670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:14.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.715+0000 7f872dd9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f873009cca0 0x7f8730067130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:14.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:14.715+0000 7f872e59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87300a3560 0x7f873009b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:15.517 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:15.516+0000 7f872dd9b640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7f873009cca0 msgr2=0x7f8730067130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:15.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:15.516+0000 7f872e59c640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87300a3560 msgr2=0x7f873009b480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:15.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:15.516+0000 7f872ed9d640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f873009d090 msgr2=0x7f8730067670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:15.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:15.516+0000 7f872dd9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f873009cca0 0x7f8730067130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:15.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:15.516+0000 7f872e59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87300a3560 0x7f873009b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:15.518 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:15.516+0000 7f872ed9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f873009d090 0x7f8730067670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:17.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 10 monclient(hunting): tick 2024-09-15T01:38:17.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 1 monclient(hunting): continuing hunt 2024-09-15T01:38:17.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:38:17.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f873009d090 msgr2=0x7f8730067670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:17.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f873009d090 0x7f8730067670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:17.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87300a3560 msgr2=0x7f873009b480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:17.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87300a3560 0x7f873009b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:17.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7f873009cca0 msgr2=0x7f8730067130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:17.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f873009cca0 0x7f8730067130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 10 monclient: _add_conns ranks=[5,4,0,10,1,6,7,3] 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8710003830 0x7f8710005c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 10 monclient(hunting): picked mon.noname-h con 0x7f8710003830 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8710006270 0x7f87100086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 10 monclient(hunting): picked mon.noname-c con 0x7f8710006270 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8710008dc0 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873004b720 con 0x7f8710008dc0 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f873009e120 con 0x7f8710006270 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8730099f10 con 0x7f8710003830 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f871ffff640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:17.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f872dd9b640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8710006270 msgr2=0x7f87100086f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f872ed9d640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f8710008dc0 legacy=0x7f871000b270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f872e59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8710003830 0x7f8710005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.113+0000 7f872dd9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8710006270 0x7f87100086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8710003830 0x7f8710005c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:34196/0 (socket says 172.21.15.204:34196) 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 1 -- 172.21.15.204:0/79696384 learned_addr learned my addr 172.21.15.204:0/79696384 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 10 monclient(hunting): get_auth_request con 0x7f8710003830 auth_method 0 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:17.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 10 monclient(hunting): handle_auth_done global_id 6584 payload 274 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 1 -- 172.21.15.204:0/79696384 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8710006270 msgr2=0x7f87100086f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 1 --2- 172.21.15.204:0/79696384 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8710006270 0x7f87100086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 1 -- 172.21.15.204:0/79696384 >> v1:172.21.15.178:6789/0 conn(0x7f8710008dc0 legacy=0x7f871000b270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 1 monclient: found mon.noname-h 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872ed9d640 1 -- 172.21.15.204:0/79696384 reap_dead start 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.114+0000 7f872e59c640 1 -- 172.21.15.204:0/79696384 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871000dca0 con 0x7f8710003830 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872e59c640 1 --2- 172.21.15.204:0/79696384 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8710003830 0x7f8710005c70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f87240098d0 tx=0x7f87240091a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=8c5b5563cf1b366c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872d59a640 1 -- 172.21.15.204:0/79696384 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8724003d60 con 0x7f8710003830 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872d59a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:38:17.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872d59a640 10 monclient: dump: 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:17.122 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872d59a640 10 monclient: _finish_auth 0 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872d59a640 1 -- 172.21.15.204:0/79696384 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872402abe0 con 0x7f8710003830 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872d59a640 1 -- 172.21.15.204:0/79696384 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f872402ad50 con 0x7f8710003830 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872d59a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f872d59a640 10 monclient: dump: 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:17.123 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:17.124 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:17.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f8734f9d640 5 monclient: authenticate success, global_id 6584 2024-09-15T01:38:17.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f8734f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:17.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.115+0000 7f8734f9d640 4 set_mon_vals no callback set 2024-09-15T01:38:17.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.116+0000 7f8734f9d640 10 monclient: shutdown 2024-09-15T01:38:17.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.116+0000 7f8734f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:17.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.116+0000 7f8734f9d640 1 -- 172.21.15.204:0/79696384 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8710003830 msgr2=0x7f8710005c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:17.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.116+0000 7f8734f9d640 1 --2- 172.21.15.204:0/79696384 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8710003830 0x7f8710005c70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f87240098d0 tx=0x7f87240091a0 comp rx=0 tx=0).stop 2024-09-15T01:38:17.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.116+0000 7f8734f9d640 1 -- 172.21.15.204:0/79696384 shutdown_connections 2024-09-15T01:38:17.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.116+0000 7f8734f9d640 1 --2- 172.21.15.204:0/79696384 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8710003830 0x7f8710005c70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:17.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.116+0000 7f8734f9d640 1 -- 172.21.15.204:0/79696384 >> 172.21.15.204:0/79696384 conn(0x7f87300687e0 msgr2=0x7f87300a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:17.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.116+0000 7f8734f9d640 1 -- 172.21.15.204:0/79696384 shutdown_connections 2024-09-15T01:38:17.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.116+0000 7f8734f9d640 1 -- 172.21.15.204:0/79696384 wait complete. 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.117+0000 7f8734f9d640 10 monclient: build_initial_monmap 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.117+0000 7f8734f9d640 10 monclient: monmap: 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:17.130 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.117+0000 7f8734f9d640 1 Processor -- start 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.117+0000 7f8734f9d640 1 -- start start 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.117+0000 7f8734f9d640 10 monclient: init 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 10 monclient: _add_conns ranks=[6,2,7,3,0,8,1,5,4] 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8710003830 0x7f873011e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 10 monclient(hunting): picked mon.noname-d con 0x7f8710003830 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:17.131 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f873011eb90 0x7f873011ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f873011eb90 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f873011f4a0 0x7f8730147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 10 monclient(hunting): picked mon.noname-i con 0x7f873011f4a0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8730121f70 con 0x7f873011eb90 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f873006f000 con 0x7f8710003830 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f873006eaa0 con 0x7f873011f4a0 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f8734f9d640 10 monclient(hunting): authenticate will time out at 1112.760294s 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872dd9b640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f873011eb90 msgr2=0x7f873011ef60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:17.132 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f873011f4a0 0x7f8730147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872dd9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f873011eb90 0x7f873011ef60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f873011f4a0 0x7f8730147f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:39496/0 (socket says 172.21.15.204:39496) 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 1 -- 172.21.15.204:0/2907791372 learned_addr learned my addr 172.21.15.204:0/2907791372 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 10 monclient(hunting): get_auth_request con 0x7f873011f4a0 auth_method 0 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872e59c640 1 --2- 172.21.15.204:0/2907791372 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8710003830 0x7f873011e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872e59c640 10 monclient(hunting): get_auth_request con 0x7f8710003830 auth_method 0 2024-09-15T01:38:17.133 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 10 monclient(hunting): handle_auth_done global_id 24775 payload 824 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 1 -- 172.21.15.204:0/2907791372 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8710003830 msgr2=0x7f873011e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 1 --2- 172.21.15.204:0/2907791372 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8710003830 0x7f873011e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 1 -- 172.21.15.204:0/2907791372 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f873011eb90 msgr2=0x7f873011ef60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 1 --2- 172.21.15.204:0/2907791372 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f873011eb90 0x7f873011ef60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 1 monclient: found mon.noname-i 2024-09-15T01:38:17.134 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.118+0000 7f872ed9d640 1 -- 172.21.15.204:0/2907791372 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8724009480 con 0x7f873011f4a0 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f872ed9d640 1 --2- 172.21.15.204:0/2907791372 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f873011f4a0 0x7f8730147f10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8728003c70 tx=0x7f8728009420 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f871e7fc640 1 -- 172.21.15.204:0/2907791372 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8728015070 con 0x7f873011f4a0 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f871e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f871e7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f871e7fc640 10 monclient: dump: 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:17.135 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f871e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f871e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:17.136 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f871e7fc640 1 -- 172.21.15.204:0/2907791372 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872800a040 con 0x7f873011f4a0 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f8734f9d640 5 monclient: authenticate success, global_id 24775 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f871e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f871e7fc640 1 -- 172.21.15.204:0/2907791372 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8728010d40 con 0x7f873011f4a0 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f8734f9d640 10 monclient: _renew_subs 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f8734f9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f8734f9d640 1 -- 172.21.15.204:0/2907791372 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8730120300 con 0x7f873011f4a0 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f871e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f872cd99640 4 set_mon_vals no callback set 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f871e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f871e7fc640 10 monclient: dump: 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:17.137 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:17.138 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f8734f9d640 10 monclient: _renew_subs 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f8734f9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.119+0000 7f8734f9d640 1 -- 172.21.15.204:0/2907791372 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8730121920 con 0x7f873011f4a0 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.120+0000 7f871e7fc640 1 -- 172.21.15.204:0/2907791372 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8728010440 con 0x7f873011f4a0 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.120+0000 7f871e7fc640 1 --2- 172.21.15.204:0/2907791372 >> v2:172.21.15.204:6812/28210 conn(0x7f871003aaa0 0x7f871003cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.120+0000 7f8734f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.120+0000 7f8734f9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.120+0000 7f8734f9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.120+0000 7f8734f9d640 1 -- 172.21.15.204:0/2907791372 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8730120a80 con 0x7f873011f4a0 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.120+0000 7f871e7fc640 1 -- 172.21.15.204:0/2907791372 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f872804b490 con 0x7f873011f4a0 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.121+0000 7f872e59c640 1 --2- 172.21.15.204:0/2907791372 >> v2:172.21.15.204:6812/28210 conn(0x7f871003aaa0 0x7f871003cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.121+0000 7f872e59c640 10 monclient: get_auth_request con 0x7f871003aaa0 auth_method 0 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.121+0000 7f872e59c640 1 --2- 172.21.15.204:0/2907791372 >> v2:172.21.15.204:6812/28210 conn(0x7f871003aaa0 0x7f871003cf50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8724032000 tx=0x7f8724030000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.125+0000 7f871e7fc640 1 -- 172.21.15.204:0/2907791372 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f872801a410 con 0x7f873011f4a0 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.125+0000 7f871e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:17.139 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.125+0000 7f871e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:17.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.294+0000 7f8734f9d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:38:17.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.294+0000 7f8734f9d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:38:17.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.294+0000 7f8734f9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:17.296 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.294+0000 7f8734f9d640 1 -- 172.21.15.204:0/2907791372 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7f873009a660 con 0x7f873011f4a0 2024-09-15T01:38:17.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.488+0000 7f871e7fc640 1 -- 172.21.15.204:0/2907791372 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v254) v1 ==== 78+0+0 (secure 0 0 0) 0x7f8728018310 con 0x7f873011f4a0 2024-09-15T01:38:17.489 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.488+0000 7f871e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:38:17.490 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.488+0000 7f871e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:17.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.590+0000 7f8734f9d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:38:17.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.590+0000 7f8734f9d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:38:17.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.590+0000 7f8734f9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:17.591 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:17.590+0000 7f8734f9d640 1 -- 172.21.15.204:0/2907791372 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7f8730070cc0 con 0x7f873011f4a0 2024-09-15T01:38:18.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.492+0000 7f871e7fc640 1 -- 172.21.15.204:0/2907791372 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v255) v1 ==== 78+0+0 (secure 0 0 0) 0x7f8728018960 con 0x7f873011f4a0 2024-09-15T01:38:18.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.492+0000 7f871e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-09-15T01:38:18.494 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.492+0000 7f871e7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:38:18.496 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 1 -- 172.21.15.204:0/2907791372 >> v2:172.21.15.204:6812/28210 conn(0x7f871003aaa0 msgr2=0x7f871003cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:18.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 1 --2- 172.21.15.204:0/2907791372 >> v2:172.21.15.204:6812/28210 conn(0x7f871003aaa0 0x7f871003cf50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8724032000 tx=0x7f8724030000 comp rx=0 tx=0).stop 2024-09-15T01:38:18.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 10 monclient: shutdown 2024-09-15T01:38:18.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:18.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 1 -- 172.21.15.204:0/2907791372 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f873011f4a0 msgr2=0x7f8730147f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:18.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 1 --2- 172.21.15.204:0/2907791372 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f873011f4a0 0x7f8730147f10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8728003c70 tx=0x7f8728009420 comp rx=0 tx=0).stop 2024-09-15T01:38:18.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 1 -- 172.21.15.204:0/2907791372 shutdown_connections 2024-09-15T01:38:18.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 1 --2- 172.21.15.204:0/2907791372 >> v2:172.21.15.204:6812/28210 conn(0x7f871003aaa0 0x7f871003cf50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:18.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 1 --2- 172.21.15.204:0/2907791372 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f873011f4a0 0x7f8730147f10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:18.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 1 --2- 172.21.15.204:0/2907791372 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f873011eb90 0x7f873011ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:18.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 1 --2- 172.21.15.204:0/2907791372 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8710003830 0x7f873011e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:18.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 1 -- 172.21.15.204:0/2907791372 >> 172.21.15.204:0/2907791372 conn(0x7f87300687e0 msgr2=0x7f87300a1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:18.497 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 1 -- 172.21.15.204:0/2907791372 shutdown_connections 2024-09-15T01:38:18.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.495+0000 7f8734f9d640 1 -- 172.21.15.204:0/2907791372 wait complete. 2024-09-15T01:38:18.507 INFO:tasks.workunit.client.0.smithi204.stdout:OK 2024-09-15T01:38:18.508 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo OK 2024-09-15T01:38:18.508 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T01:38:18.508 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T01:38:18.535 INFO:tasks.workunit:Running workunits matching mon/osd.sh on client.0... 2024-09-15T01:38:18.536 INFO:tasks.workunit:Running workunit mon/osd.sh... 2024-09-15T01:38:18.536 DEBUG:teuthology.orchestra.run.smithi204:workunit test mon/osd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d75f23820806da622f561ee5ee7551015285993f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/osd.sh 2024-09-15T01:38:18.594 INFO:tasks.workunit.client.0.smithi204.stderr:+ set -e 2024-09-15T01:38:18.594 INFO:tasks.workunit.client.0.smithi204.stderr:++ uuidgen 2024-09-15T01:38:18.602 INFO:tasks.workunit.client.0.smithi204.stderr:+ ua=e5de064c-679b-416c-b9b7-f61aa58c84ce 2024-09-15T01:38:18.602 INFO:tasks.workunit.client.0.smithi204.stderr:++ uuidgen 2024-09-15T01:38:18.602 INFO:tasks.workunit.client.0.smithi204.stderr:+ ub=fa8c99c4-ab34-4381-a4b1-0acedd0271cf 2024-09-15T01:38:18.603 INFO:tasks.workunit.client.0.smithi204.stderr:++ ceph osd create e5de064c-679b-416c-b9b7-f61aa58c84ce 2024-09-15T01:38:18.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.675+0000 7f184909f640 10 monclient: get_monmap_and_config 2024-09-15T01:38:18.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.675+0000 7f184909f640 10 monclient: build_initial_monmap 2024-09-15T01:38:18.678 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.675+0000 7f184909f640 10 monclient: monmap: 2024-09-15T01:38:18.678 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:18.678 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:18.678 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:18.676414+0000 2024-09-15T01:38:18.678 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:18.676414+0000 2024-09-15T01:38:18.678 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:18.678 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:18.678 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:18.678 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.676+0000 7f184909f640 1 Processor -- start 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.676+0000 7f184909f640 1 -- start start 2024-09-15T01:38:18.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.676+0000 7f184909f640 10 monclient: init 2024-09-15T01:38:18.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:18.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 10 monclient: _add_conns ranks=[10,6,5,4,1,2,0,8,3,7,9] 2024-09-15T01:38:18.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f18440a2400 0x7f18440a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:18.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 10 monclient(hunting): picked mon.noname-f con 0x7f18440a2400 addr v2:172.21.15.204:3300/0 2024-09-15T01:38:18.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f184409f990 0x7f184409fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:18.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 10 monclient(hunting): picked mon.noname-d con 0x7f184409f990 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:18.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 0x7f1844132d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 10 monclient(hunting): picked mon.noname-h con 0x7f1844132930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184409e1a0 con 0x7f18440a2400 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18440a27d0 con 0x7f1844132930 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f184409dec0 con 0x7f184409f990 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184909f640 10 monclient(hunting): authenticate will time out at 1114.319100s 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f1842d9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f18440a2400 0x7f18440a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 0x7f1844132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f1842d9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f18440a2400 0x7f18440a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:52728/0 (socket says 172.21.15.204:52728) 2024-09-15T01:38:18.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f1842d9d640 1 -- 172.21.15.204:0/1444638293 learned_addr learned my addr 172.21.15.204:0/1444638293 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 10 monclient(hunting): get_auth_request con 0x7f1844132930 auth_method 0 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f1842d9d640 10 monclient(hunting): get_auth_request con 0x7f18440a2400 auth_method 0 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f1842d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f1842d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f1842d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f1842d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:18.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f1842d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f1842d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f183a59c640 1 --2- 172.21.15.204:0/1444638293 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f184409f990 0x7f184409fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f183a59c640 10 monclient(hunting): get_auth_request con 0x7f184409f990 auth_method 0 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f183a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f183a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f183a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 10 monclient(hunting): handle_auth_done global_id 6593 payload 274 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 1 -- 172.21.15.204:0/1444638293 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f184409f990 msgr2=0x7f184409fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 1 --2- 172.21.15.204:0/1444638293 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f184409f990 0x7f184409fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 1 -- 172.21.15.204:0/1444638293 >> v2:172.21.15.204:3300/0 conn(0x7f18440a2400 msgr2=0x7f18440a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 1 --2- 172.21.15.204:0/1444638293 >> v2:172.21.15.204:3300/0 conn(0x7f18440a2400 0x7f18440a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 10 monclient: _finish_hunting 0 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 1 monclient: found mon.noname-h 2024-09-15T01:38:18.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f184359e640 1 -- 172.21.15.204:0/1444638293 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440648a0 con 0x7f1844132930 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.677+0000 7f1842d9d640 1 --2- 172.21.15.204:0/1444638293 >> v2:172.21.15.204:3300/0 conn(0x7f18440a2400 0x7f18440a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184359e640 1 --2- 172.21.15.204:0/1444638293 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 0x7f1844132d80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f183c009e60 tx=0x7f183c009780 comp rx=0 tx=0).ready entity=mon.5 client_cookie=b9dd0df1580b6826 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184259c640 1 -- 172.21.15.204:0/1444638293 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f183c0079d0 con 0x7f1844132930 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184259c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184259c640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184259c640 10 monclient: dump: 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:18.692 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184259c640 10 monclient: _finish_auth 0 2024-09-15T01:38:18.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184259c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:18.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184259c640 1 -- 172.21.15.204:0/1444638293 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c02d050 con 0x7f1844132930 2024-09-15T01:38:18.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 5 monclient: authenticate success, global_id 6593 2024-09-15T01:38:18.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:18.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184259c640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:18.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:18.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184259c640 1 -- 172.21.15.204:0/1444638293 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f183c007bd0 con 0x7f1844132930 2024-09-15T01:38:18.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 4 set_mon_vals no callback set 2024-09-15T01:38:18.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184259c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:18.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184259c640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:38:18.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184259c640 10 monclient: dump: 2024-09-15T01:38:18.694 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:18.694 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:18.695 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:18.696 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:18.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 10 monclient: shutdown 2024-09-15T01:38:18.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:18.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 1 -- 172.21.15.204:0/1444638293 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 msgr2=0x7f1844132d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:18.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 1 --2- 172.21.15.204:0/1444638293 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 0x7f1844132d80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f183c009e60 tx=0x7f183c009780 comp rx=0 tx=0).stop 2024-09-15T01:38:18.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 1 -- 172.21.15.204:0/1444638293 shutdown_connections 2024-09-15T01:38:18.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 1 --2- 172.21.15.204:0/1444638293 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 0x7f1844132d80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:18.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 1 --2- 172.21.15.204:0/1444638293 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f184409f990 0x7f184409fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:18.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 1 --2- 172.21.15.204:0/1444638293 >> v2:172.21.15.204:3300/0 conn(0x7f18440a2400 0x7f18440a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:18.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.678+0000 7f184909f640 1 -- 172.21.15.204:0/1444638293 >> 172.21.15.204:0/1444638293 conn(0x7f184406e860 msgr2=0x7f184406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:18.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.679+0000 7f184909f640 1 -- 172.21.15.204:0/1444638293 shutdown_connections 2024-09-15T01:38:18.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.679+0000 7f184909f640 1 -- 172.21.15.204:0/1444638293 wait complete. 2024-09-15T01:38:18.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.679+0000 7f184909f640 10 monclient: build_initial_monmap 2024-09-15T01:38:18.697 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.679+0000 7f184909f640 10 monclient: monmap: 2024-09-15T01:38:18.697 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:18.697 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:18.697 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:18.697 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:18.697 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:18.697 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:18.697 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:18.697 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:18.697 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.679+0000 7f184909f640 1 Processor -- start 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.679+0000 7f184909f640 1 -- start start 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.679+0000 7f184909f640 10 monclient: init 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 10 monclient: _add_conns ranks=[8,7,5,1,3,0,4,2,6] 2024-09-15T01:38:18.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f184409f990 0x7f184411d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 10 monclient(hunting): picked mon.noname-e con 0x7f184409f990 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f18440a2400 0x7f184411d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 10 monclient(hunting): picked mon.noname-i con 0x7f18440a2400 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 0x7f184411dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 10 monclient(hunting): picked mon.noname-h con 0x7f1844132930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f184409cad0 con 0x7f1844132930 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1844120c80 con 0x7f18440a2400 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f18441209a0 con 0x7f184409f990 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:18.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184909f640 10 monclient(hunting): authenticate will time out at 1114.322406s 2024-09-15T01:38:18.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f1842d9d640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f184409f990 msgr2=0x7f184411d2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:18.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f1842d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f184409f990 0x7f184411d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:18.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 0x7f184411dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:18.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f183a59c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f18440a2400 0x7f184411d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:18.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 0x7f184411dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:52448/0 (socket says 172.21.15.204:52448) 2024-09-15T01:38:18.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 1 -- 172.21.15.204:0/3324971620 learned_addr learned my addr 172.21.15.204:0/3324971620 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:18.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 10 monclient(hunting): get_auth_request con 0x7f1844132930 auth_method 0 2024-09-15T01:38:18.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:18.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:18.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:18.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f183a59c640 10 monclient(hunting): get_auth_request con 0x7f18440a2400 auth_method 0 2024-09-15T01:38:18.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f183a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f183a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f183a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f183a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f183a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f183a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 10 monclient(hunting): handle_auth_done global_id 6602 payload 824 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 1 -- 172.21.15.204:0/3324971620 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f184409f990 msgr2=0x7f184411d2b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 1 --2- 172.21.15.204:0/3324971620 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f184409f990 0x7f184411d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 1 -- 172.21.15.204:0/3324971620 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f18440a2400 msgr2=0x7f184411d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:18.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 1 --2- 172.21.15.204:0/3324971620 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f18440a2400 0x7f184411d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 10 monclient: _finish_hunting 0 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 1 monclient: found mon.noname-h 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 1 -- 172.21.15.204:0/3324971620 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1834009450 con 0x7f1844132930 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f183a59c640 1 --2- 172.21.15.204:0/3324971620 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f18440a2400 0x7f184411d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.680+0000 7f184359e640 1 --2- 172.21.15.204:0/3324971620 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 0x7f184411dd30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f183c0099f0 tx=0x7f183c037000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 1 -- 172.21.15.204:0/3324971620 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f183c03c420 con 0x7f1844132930 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 10 monclient: dump: 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:18.702 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:18.703 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 1 -- 172.21.15.204:0/3324971620 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c03c5a0 con 0x7f1844132930 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f184909f640 5 monclient: authenticate success, global_id 6602 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f184909f640 10 monclient: _renew_subs 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f184909f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f184909f640 1 -- 172.21.15.204:0/3324971620 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183c009a20 con 0x7f1844132930 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 1 -- 172.21.15.204:0/3324971620 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f183c03cb90 con 0x7f1844132930 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:38:18.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 10 monclient: dump: 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:18.705 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f1841d9b640 4 set_mon_vals no callback set 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f184909f640 10 monclient: _renew_subs 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f184909f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f184909f640 1 -- 172.21.15.204:0/3324971620 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184411eb90 con 0x7f1844132930 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.681+0000 7f183b7fe640 1 -- 172.21.15.204:0/3324971620 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f183c003040 con 0x7f1844132930 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.682+0000 7f183b7fe640 1 --2- 172.21.15.204:0/3324971620 >> v2:172.21.15.204:6812/28210 conn(0x7f18240397b0 0x7f182403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.682+0000 7f183b7fe640 1 -- 172.21.15.204:0/3324971620 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f183c06ce40 con 0x7f1844132930 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.682+0000 7f184909f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.682+0000 7f184909f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.682+0000 7f184909f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:18.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.682+0000 7f1842d9d640 1 --2- 172.21.15.204:0/3324971620 >> v2:172.21.15.204:6812/28210 conn(0x7f18240397b0 0x7f182403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:18.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.682+0000 7f184909f640 1 -- 172.21.15.204:0/3324971620 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1844070b80 con 0x7f1844132930 2024-09-15T01:38:18.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.682+0000 7f1842d9d640 10 monclient: get_auth_request con 0x7f18240397b0 auth_method 0 2024-09-15T01:38:18.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.682+0000 7f1842d9d640 1 --2- 172.21.15.204:0/3324971620 >> v2:172.21.15.204:6812/28210 conn(0x7f18240397b0 0x7f182403bc60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1834006c10 tx=0x7f18340091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:18.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.686+0000 7f183b7fe640 1 -- 172.21.15.204:0/3324971620 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f183c05ce60 con 0x7f1844132930 2024-09-15T01:38:18.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.686+0000 7f183b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:18.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.686+0000 7f183b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:18.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.852+0000 7f184909f640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:18.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.852+0000 7f184909f640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:18.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.852+0000 7f184909f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:18.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:18.852+0000 7f184909f640 1 -- 172.21.15.204:0/3324971620 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"} v 0) v1 -- 0x7f184409e480 con 0x7f1844132930 2024-09-15T01:38:19.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.496+0000 7f183b7fe640 1 -- 172.21.15.204:0/3324971620 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}]=0 v256) v1 ==== 106+0+3 (secure 0 0 0) 0x7f183c031ac0 con 0x7f1844132930 2024-09-15T01:38:19.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.496+0000 7f183b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:19.498 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.496+0000 7f183b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:19.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.602+0000 7f184909f640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:19.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.602+0000 7f184909f640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:19.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.602+0000 7f184909f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:38:19.604 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.602+0000 7f184909f640 1 -- 172.21.15.204:0/3324971620 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"} v 0) v1 -- 0x7f1844120ae0 con 0x7f1844132930 2024-09-15T01:38:19.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.603+0000 7f183b7fe640 1 -- 172.21.15.204:0/3324971620 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}]=0 v256) v1 ==== 106+0+3 (secure 0 0 0) 0x7f183c05ce60 con 0x7f1844132930 2024-09-15T01:38:19.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.603+0000 7f183b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:19.605 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.603+0000 7f183b7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:38:19.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 1 -- 172.21.15.204:0/3324971620 >> v2:172.21.15.204:6812/28210 conn(0x7f18240397b0 msgr2=0x7f182403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:19.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 1 --2- 172.21.15.204:0/3324971620 >> v2:172.21.15.204:6812/28210 conn(0x7f18240397b0 0x7f182403bc60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1834006c10 tx=0x7f18340091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 10 monclient: shutdown 2024-09-15T01:38:19.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:19.607 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 1 -- 172.21.15.204:0/3324971620 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 msgr2=0x7f184411dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:19.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 1 --2- 172.21.15.204:0/3324971620 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 0x7f184411dd30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f183c0099f0 tx=0x7f183c037000 comp rx=0 tx=0).stop 2024-09-15T01:38:19.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 1 -- 172.21.15.204:0/3324971620 shutdown_connections 2024-09-15T01:38:19.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 1 --2- 172.21.15.204:0/3324971620 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1844132930 0x7f184411dd30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 1 --2- 172.21.15.204:0/3324971620 >> v2:172.21.15.204:6812/28210 conn(0x7f18240397b0 0x7f182403bc60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 1 --2- 172.21.15.204:0/3324971620 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f18440a2400 0x7f184411d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 1 --2- 172.21.15.204:0/3324971620 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f184409f990 0x7f184411d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 1 -- 172.21.15.204:0/3324971620 >> 172.21.15.204:0/3324971620 conn(0x7f184406e860 msgr2=0x7f18440a45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:19.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 1 -- 172.21.15.204:0/3324971620 shutdown_connections 2024-09-15T01:38:19.608 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.605+0000 7f184909f640 1 -- 172.21.15.204:0/3324971620 wait complete. 2024-09-15T01:38:19.616 INFO:tasks.workunit.client.0.smithi204.stderr:+ na=12 2024-09-15T01:38:19.617 INFO:tasks.workunit.client.0.smithi204.stderr:++ ceph osd create e5de064c-679b-416c-b9b7-f61aa58c84ce 2024-09-15T01:38:19.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.688+0000 7f988d499640 10 monclient: get_monmap_and_config 2024-09-15T01:38:19.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.688+0000 7f988d499640 10 monclient: build_initial_monmap 2024-09-15T01:38:19.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.688+0000 7f988d499640 10 monclient: monmap: 2024-09-15T01:38:19.690 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:19.690 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:19.690 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:19.688871+0000 2024-09-15T01:38:19.690 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:19.688871+0000 2024-09-15T01:38:19.690 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:19.690 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:19.690 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:19.690 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:19.690 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:19.690 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:19.691 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:19.691 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:19.691 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:19.691 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:19.691 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:19.691 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:19.691 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:19.691 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:19.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.688+0000 7f988d499640 1 Processor -- start 2024-09-15T01:38:19.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 1 -- start start 2024-09-15T01:38:19.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 10 monclient: init 2024-09-15T01:38:19.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:19.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 10 monclient: _add_conns ranks=[4,3,5,10,8,7,1,2,6,0,9] 2024-09-15T01:38:19.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98880a2400 0x7f98880a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:19.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 10 monclient(hunting): picked mon.noname-c con 0x7f98880a2400 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:19.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 0x7f988809fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:19.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 10 monclient(hunting): picked mon.noname-g con 0x7f988809f990 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:19.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9888132930 0x7f9888132d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 10 monclient(hunting): picked mon.noname-h con 0x7f9888132930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988809e1a0 con 0x7f988809f990 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98880a27d0 con 0x7f98880a2400 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f988809dec0 con 0x7f9888132930 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f988d499640 10 monclient(hunting): authenticate will time out at 1115.331548s 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 0x7f988809fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f98877fe640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9888132930 0x7f9888132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 0x7f988809fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:44412/0 (socket says 172.21.15.204:44412) 2024-09-15T01:38:19.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f9886ffd640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98880a2400 msgr2=0x7f98880a0ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 1 -- 172.21.15.204:0/146357828 learned_addr learned my addr 172.21.15.204:0/146357828 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 10 monclient(hunting): get_auth_request con 0x7f988809f990 auth_method 0 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f9886ffd640 1 --2- 172.21.15.204:0/146357828 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98880a2400 0x7f98880a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f98877fe640 10 monclient(hunting): get_auth_request con 0x7f9888132930 auth_method 0 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f98877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f98877fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f98877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:19.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f98877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f98877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f98877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 10 monclient(hunting): handle_auth_done global_id 15690 payload 274 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 1 -- 172.21.15.204:0/146357828 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9888132930 msgr2=0x7f9888132d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 1 --2- 172.21.15.204:0/146357828 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9888132930 0x7f9888132d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 1 -- 172.21.15.204:0/146357828 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98880a2400 msgr2=0x7f98880a0ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 1 --2- 172.21.15.204:0/146357828 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98880a2400 0x7f98880a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 1 monclient: found mon.noname-g 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f987e7fc640 1 -- 172.21.15.204:0/146357828 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98880648a0 con 0x7f988809f990 2024-09-15T01:38:19.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.689+0000 7f98877fe640 1 --2- 172.21.15.204:0/146357828 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9888132930 0x7f9888132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:19.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f987e7fc640 1 --2- 172.21.15.204:0/146357828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 0x7f988809fd60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9870009940 tx=0x7f98700091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c2ee9187ed71c01b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:19.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f98867fc640 1 -- 172.21.15.204:0/146357828 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f987002ac80 con 0x7f988809f990 2024-09-15T01:38:19.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f98867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:19.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f98867fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:19.703 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f98867fc640 10 monclient: dump: 2024-09-15T01:38:19.703 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:19.703 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:19.703 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:19.703 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:19.703 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:19.703 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:19.703 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f98867fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f98867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f98867fc640 1 -- 172.21.15.204:0/146357828 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987002ae00 con 0x7f988809f990 2024-09-15T01:38:19.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f988d499640 5 monclient: authenticate success, global_id 15690 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f988d499640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f98867fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f988d499640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f98867fc640 1 -- 172.21.15.204:0/146357828 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9870031ca0 con 0x7f988809f990 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f98867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f988d499640 4 set_mon_vals no callback set 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f98867fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.690+0000 7f98867fc640 10 monclient: dump: 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:19.705 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 10 monclient: shutdown 2024-09-15T01:38:19.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 1 -- 172.21.15.204:0/146357828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 msgr2=0x7f988809fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 1 --2- 172.21.15.204:0/146357828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 0x7f988809fd60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9870009940 tx=0x7f98700091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 1 -- 172.21.15.204:0/146357828 shutdown_connections 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 1 --2- 172.21.15.204:0/146357828 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9888132930 0x7f9888132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 1 --2- 172.21.15.204:0/146357828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 0x7f988809fd60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 1 --2- 172.21.15.204:0/146357828 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98880a2400 0x7f98880a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 1 -- 172.21.15.204:0/146357828 >> 172.21.15.204:0/146357828 conn(0x7f988806e860 msgr2=0x7f988806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 1 -- 172.21.15.204:0/146357828 shutdown_connections 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 1 -- 172.21.15.204:0/146357828 wait complete. 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 10 monclient: build_initial_monmap 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.691+0000 7f988d499640 10 monclient: monmap: 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:19.707 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:19.708 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 1 Processor -- start 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 1 -- start start 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 10 monclient: init 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 10 monclient: _add_conns ranks=[3,2,5,1,6,8,0,7,4] 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 0x7f9888141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 10 monclient(hunting): picked mon.noname-g con 0x7f988809f990 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98880a2400 0x7f9888142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 10 monclient(hunting): picked mon.noname-b con 0x7f98880a2400 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9888132930 0x7f98881428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:19.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 10 monclient(hunting): picked mon.noname-h con 0x7f9888132930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98881218f0 con 0x7f98880a2400 2024-09-15T01:38:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988809d3d0 con 0x7f988809f990 2024-09-15T01:38:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f988809cab0 con 0x7f9888132930 2024-09-15T01:38:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f988d499640 10 monclient(hunting): authenticate will time out at 1115.334881s 2024-09-15T01:38:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f9886ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 0x7f9888141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:19.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f9886ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 0x7f9888141e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:44416/0 (socket says 172.21.15.204:44416) 2024-09-15T01:38:19.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f9886ffd640 1 -- 172.21.15.204:0/4254717775 learned_addr learned my addr 172.21.15.204:0/4254717775 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:19.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f9886ffd640 10 monclient(hunting): get_auth_request con 0x7f988809f990 auth_method 0 2024-09-15T01:38:19.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f9886ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:19.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f9886ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:19.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f9886ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:19.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f98877fe640 1 --2- 172.21.15.204:0/4254717775 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9888132930 0x7f98881428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:19.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f98877fe640 10 monclient(hunting): get_auth_request con 0x7f9888132930 auth_method 0 2024-09-15T01:38:19.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f98877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:19.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f98877fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:19.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f98877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:19.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.692+0000 7f987e7fc640 1 -- 172.21.15.204:0/4254717775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98880a2400 msgr2=0x7f9888142370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:19.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987e7fc640 1 --2- 172.21.15.204:0/4254717775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98880a2400 0x7f9888142370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:19.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:19.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:19.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:19.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f98877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:19.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f98877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:19.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f98877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:19.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 10 monclient(hunting): handle_auth_done global_id 15699 payload 824 2024-09-15T01:38:19.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 1 -- 172.21.15.204:0/4254717775 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9888132930 msgr2=0x7f98881428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:19.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 1 --2- 172.21.15.204:0/4254717775 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9888132930 0x7f98881428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 1 -- 172.21.15.204:0/4254717775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98880a2400 msgr2=0x7f9888142370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:19.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 1 --2- 172.21.15.204:0/4254717775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98880a2400 0x7f9888142370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 1 monclient: found mon.noname-g 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 1 -- 172.21.15.204:0/4254717775 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9880009a20 con 0x7f988809f990 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9886ffd640 1 --2- 172.21.15.204:0/4254717775 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 0x7f9888141e30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f987800b630 tx=0x7f98780091a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987f7fe640 1 -- 172.21.15.204:0/4254717775 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9878015070 con 0x7f988809f990 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987f7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987f7fe640 10 monclient: dump: 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:19.713 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:19.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987f7fe640 1 -- 172.21.15.204:0/4254717775 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9878006150 con 0x7f988809f990 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f988d499640 5 monclient: authenticate success, global_id 15699 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f988d499640 10 monclient: _renew_subs 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f988d499640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f988d499640 1 -- 172.21.15.204:0/4254717775 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9870009450 con 0x7f988809f990 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987f7fe640 1 -- 172.21.15.204:0/4254717775 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9878006740 con 0x7f988809f990 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f987f7fe640 10 monclient: dump: 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:19.715 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:19.716 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f9885ffb640 4 set_mon_vals no callback set 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f988d499640 10 monclient: _renew_subs 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f988d499640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.693+0000 7f988d499640 1 -- 172.21.15.204:0/4254717775 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98881212c0 con 0x7f988809f990 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.694+0000 7f987f7fe640 1 -- 172.21.15.204:0/4254717775 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f98780062c0 con 0x7f988809f990 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.694+0000 7f987f7fe640 1 --2- 172.21.15.204:0/4254717775 >> v2:172.21.15.204:6812/28210 conn(0x7f98680397e0 0x7f986803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.694+0000 7f987f7fe640 1 -- 172.21.15.204:0/4254717775 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f987804b510 con 0x7f988809f990 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.694+0000 7f988d499640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.694+0000 7f987e7fc640 1 --2- 172.21.15.204:0/4254717775 >> v2:172.21.15.204:6812/28210 conn(0x7f98680397e0 0x7f986803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.694+0000 7f988d499640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.694+0000 7f988d499640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.694+0000 7f988d499640 1 -- 172.21.15.204:0/4254717775 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9888070b80 con 0x7f988809f990 2024-09-15T01:38:19.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.694+0000 7f987e7fc640 10 monclient: get_auth_request con 0x7f98680397e0 auth_method 0 2024-09-15T01:38:19.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.695+0000 7f987e7fc640 1 --2- 172.21.15.204:0/4254717775 >> v2:172.21.15.204:6812/28210 conn(0x7f98680397e0 0x7f986803bc90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9870034000 tx=0x7f9870032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:19.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.699+0000 7f987f7fe640 1 -- 172.21.15.204:0/4254717775 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9878010440 con 0x7f988809f990 2024-09-15T01:38:19.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.699+0000 7f987f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:19.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.699+0000 7f987f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:19.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.868+0000 7f988d499640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:19.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.868+0000 7f988d499640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:19.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.868+0000 7f988d499640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:19.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.868+0000 7f988d499640 1 -- 172.21.15.204:0/4254717775 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"} v 0) v1 -- 0x7f988809e480 con 0x7f988809f990 2024-09-15T01:38:19.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.868+0000 7f987f7fe640 1 -- 172.21.15.204:0/4254717775 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}]=0 v256) v1 ==== 106+0+3 (secure 0 0 0) 0x7f987800d370 con 0x7f988809f990 2024-09-15T01:38:19.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.868+0000 7f987f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:19.870 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.868+0000 7f987f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.972+0000 7f988d499640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.972+0000 7f988d499640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.972+0000 7f988d499640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:19.973 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.972+0000 7f988d499640 1 -- 172.21.15.204:0/4254717775 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"} v 0) v1 -- 0x7f98881225b0 con 0x7f988809f990 2024-09-15T01:38:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.972+0000 7f987f7fe640 1 -- 172.21.15.204:0/4254717775 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}]=0 v256) v1 ==== 106+0+3 (secure 0 0 0) 0x7f987800eb30 con 0x7f988809f990 2024-09-15T01:38:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.972+0000 7f987f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:19.974 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.972+0000 7f987f7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:38:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 1 -- 172.21.15.204:0/4254717775 >> v2:172.21.15.204:6812/28210 conn(0x7f98680397e0 msgr2=0x7f986803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 1 --2- 172.21.15.204:0/4254717775 >> v2:172.21.15.204:6812/28210 conn(0x7f98680397e0 0x7f986803bc90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9870034000 tx=0x7f9870032000 comp rx=0 tx=0).stop 2024-09-15T01:38:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 10 monclient: shutdown 2024-09-15T01:38:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 1 -- 172.21.15.204:0/4254717775 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 msgr2=0x7f9888141e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:19.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 1 --2- 172.21.15.204:0/4254717775 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 0x7f9888141e30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f987800b630 tx=0x7f98780091a0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 1 -- 172.21.15.204:0/4254717775 shutdown_connections 2024-09-15T01:38:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 1 --2- 172.21.15.204:0/4254717775 >> v2:172.21.15.204:6812/28210 conn(0x7f98680397e0 0x7f986803bc90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 1 --2- 172.21.15.204:0/4254717775 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9888132930 0x7f98881428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 1 --2- 172.21.15.204:0/4254717775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f98880a2400 0x7f9888142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 1 --2- 172.21.15.204:0/4254717775 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f988809f990 0x7f9888141e30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 1 -- 172.21.15.204:0/4254717775 >> 172.21.15.204:0/4254717775 conn(0x7f988806e860 msgr2=0x7f9888131400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 1 -- 172.21.15.204:0/4254717775 shutdown_connections 2024-09-15T01:38:19.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:19.974+0000 7f988d499640 1 -- 172.21.15.204:0/4254717775 wait complete. 2024-09-15T01:38:19.985 INFO:tasks.workunit.client.0.smithi204.stderr:+ test 12 -eq 12 2024-09-15T01:38:19.985 INFO:tasks.workunit.client.0.smithi204.stderr:++ ceph osd create fa8c99c4-ab34-4381-a4b1-0acedd0271cf 2024-09-15T01:38:20.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.056+0000 7f0227217640 10 monclient: get_monmap_and_config 2024-09-15T01:38:20.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.056+0000 7f0227217640 10 monclient: build_initial_monmap 2024-09-15T01:38:20.058 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.056+0000 7f0227217640 10 monclient: monmap: 2024-09-15T01:38:20.058 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:20.058 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:20.058 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:20.057065+0000 2024-09-15T01:38:20.058 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:20.057065+0000 2024-09-15T01:38:20.058 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:20.059 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:20.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 1 Processor -- start 2024-09-15T01:38:20.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 1 -- start start 2024-09-15T01:38:20.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 10 monclient: init 2024-09-15T01:38:20.066 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 10 monclient: _add_conns ranks=[8,7,2,5,10,1,9,0,3,6,4] 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02200a2400 0x7f02200a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 10 monclient(hunting): picked mon.noname-e con 0x7f02200a2400 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f022009f990 0x7f022009fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 10 monclient(hunting): picked mon.noname-i con 0x7f022009f990 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0220132930 0x7f0220132d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 10 monclient(hunting): picked mon.noname-b con 0x7f0220132930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022009e1a0 con 0x7f0220132930 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f02200a27d0 con 0x7f022009f990 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:20.067 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f022009dec0 con 0x7f02200a2400 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0227217640 10 monclient(hunting): authenticate will time out at 1115.699818s 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f021ffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f022009f990 0x7f022009fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0224fb3640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02200a2400 msgr2=0x7f02200a0ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f02257b4640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0220132930 msgr2=0x7f0220132d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f02257b4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0220132930 0x7f0220132d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f0224fb3640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02200a2400 0x7f02200a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f021ffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f022009f990 0x7f022009fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:39516/0 (socket says 172.21.15.204:39516) 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.057+0000 7f021ffff640 1 -- 172.21.15.204:0/4019815108 learned_addr learned my addr 172.21.15.204:0/4019815108 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 10 monclient(hunting): get_auth_request con 0x7f022009f990 auth_method 0 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:20.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 10 monclient(hunting): handle_auth_done global_id 24793 payload 274 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 1 -- 172.21.15.204:0/4019815108 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02200a2400 msgr2=0x7f02200a0ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 1 --2- 172.21.15.204:0/4019815108 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02200a2400 0x7f02200a0ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 1 -- 172.21.15.204:0/4019815108 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0220132930 msgr2=0x7f0220132d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 1 --2- 172.21.15.204:0/4019815108 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0220132930 0x7f0220132d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 1 monclient: found mon.noname-i 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:38:20.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 1 -- 172.21.15.204:0/4019815108 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02200648a0 con 0x7f022009f990 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.058+0000 7f021ffff640 1 --2- 172.21.15.204:0/4019815108 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f022009f990 0x7f022009fd60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f020c009940 tx=0x7f020c0091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=acc7632a79642834 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f021f7fe640 1 -- 172.21.15.204:0/4019815108 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f020c02acb0 con 0x7f022009f990 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f021f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f021f7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f021f7fe640 10 monclient: dump: 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:20.070 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f021f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f021f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f021f7fe640 1 -- 172.21.15.204:0/4019815108 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c02ae30 con 0x7f022009f990 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 5 monclient: authenticate success, global_id 24793 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:20.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f021f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f021f7fe640 1 -- 172.21.15.204:0/4019815108 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f020c031ca0 con 0x7f022009f990 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f021f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 4 set_mon_vals no callback set 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f021f7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f021f7fe640 10 monclient: dump: 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:20.072 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 10 monclient: shutdown 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 1 -- 172.21.15.204:0/4019815108 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f022009f990 msgr2=0x7f022009fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:20.073 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 1 --2- 172.21.15.204:0/4019815108 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f022009f990 0x7f022009fd60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f020c009940 tx=0x7f020c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 1 -- 172.21.15.204:0/4019815108 shutdown_connections 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 1 --2- 172.21.15.204:0/4019815108 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f022009f990 0x7f022009fd60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 1 --2- 172.21.15.204:0/4019815108 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0220132930 0x7f0220132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 1 --2- 172.21.15.204:0/4019815108 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f02200a2400 0x7f02200a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 1 -- 172.21.15.204:0/4019815108 >> 172.21.15.204:0/4019815108 conn(0x7f022006e860 msgr2=0x7f022006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 1 -- 172.21.15.204:0/4019815108 shutdown_connections 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.059+0000 7f0227217640 1 -- 172.21.15.204:0/4019815108 wait complete. 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.060+0000 7f0227217640 10 monclient: build_initial_monmap 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.060+0000 7f0227217640 10 monclient: monmap: 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:20.074 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.060+0000 7f0227217640 1 Processor -- start 2024-09-15T01:38:20.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.060+0000 7f0227217640 1 -- start start 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.060+0000 7f0227217640 10 monclient: init 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient: _add_conns ranks=[3,2,8,4,1,0,7,5,6] 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f022009f990 0x7f022011c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient(hunting): picked mon.noname-g con 0x7f022009f990 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02200a2400 0x7f022011ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient(hunting): picked mon.noname-b con 0x7f02200a2400 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0220132930 0x7f022011cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient(hunting): picked mon.noname-e con 0x7f0220132930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022011ed70 con 0x7f02200a2400 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022011ea00 con 0x7f022009f990 2024-09-15T01:38:20.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f022011d7c0 con 0x7f0220132930 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient(hunting): authenticate will time out at 1115.703101s 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f022009f990 0x7f022011c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f022009f990 0x7f022011c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:44430/0 (socket says 172.21.15.204:44430) 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 1 -- 172.21.15.204:0/3865384049 learned_addr learned my addr 172.21.15.204:0/3865384049 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 10 monclient(hunting): get_auth_request con 0x7f022009f990 auth_method 0 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021ffff640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02200a2400 msgr2=0x7f022011ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f02257b4640 1 -- 172.21.15.204:0/3865384049 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0220132930 msgr2=0x7f022011cfd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:20.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021ffff640 1 --2- 172.21.15.204:0/3865384049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02200a2400 0x7f022011ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f02257b4640 1 --2- 172.21.15.204:0/3865384049 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0220132930 0x7f022011cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 10 monclient(hunting): handle_auth_done global_id 15708 payload 824 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 1 -- 172.21.15.204:0/3865384049 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0220132930 msgr2=0x7f022011cfd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 1 --2- 172.21.15.204:0/3865384049 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0220132930 0x7f022011cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 1 -- 172.21.15.204:0/3865384049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02200a2400 msgr2=0x7f022011ca90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 1 --2- 172.21.15.204:0/3865384049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02200a2400 0x7f022011ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 10 monclient: _finish_hunting 0 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 1 monclient: found mon.noname-g 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:20.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 1 -- 172.21.15.204:0/3865384049 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c009450 con 0x7f022009f990 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0224fb3640 1 --2- 172.21.15.204:0/3865384049 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f022009f990 0x7f022011c550 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f021400b600 tx=0x7f02140091a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021cff9640 1 -- 172.21.15.204:0/3865384049 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0214015070 con 0x7f022009f990 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021cff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021cff9640 10 monclient: dump: 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:20.079 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021cff9640 10 monclient: _finish_auth 0 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021cff9640 1 -- 172.21.15.204:0/3865384049 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0214006150 con 0x7f022009f990 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 5 monclient: authenticate success, global_id 15708 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:20.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient: _renew_subs 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021cff9640 1 -- 172.21.15.204:0/3865384049 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0214006740 con 0x7f022009f990 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 1 -- 172.21.15.204:0/3865384049 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022011e720 con 0x7f022009f990 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021cff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021cff9640 10 monclient: dump: 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:20.081 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f021effd640 4 set_mon_vals no callback set 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient: _renew_subs 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.061+0000 7f0227217640 1 -- 172.21.15.204:0/3865384049 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022009c8b0 con 0x7f022009f990 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.062+0000 7f021cff9640 1 -- 172.21.15.204:0/3865384049 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f02140062c0 con 0x7f022009f990 2024-09-15T01:38:20.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.062+0000 7f021cff9640 1 --2- 172.21.15.204:0/3865384049 >> v2:172.21.15.204:6812/28210 conn(0x7f02040396e0 0x7f020403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.062+0000 7f0227217640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.062+0000 7f0227217640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.062+0000 7f021cff9640 1 -- 172.21.15.204:0/3865384049 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f021404b5b0 con 0x7f022009f990 2024-09-15T01:38:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.062+0000 7f0227217640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.062+0000 7f0227217640 1 -- 172.21.15.204:0/3865384049 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022009cb30 con 0x7f022009f990 2024-09-15T01:38:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.062+0000 7f021ffff640 1 --2- 172.21.15.204:0/3865384049 >> v2:172.21.15.204:6812/28210 conn(0x7f02040396e0 0x7f020403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.062+0000 7f021ffff640 10 monclient: get_auth_request con 0x7f02040396e0 auth_method 0 2024-09-15T01:38:20.083 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.063+0000 7f021ffff640 1 --2- 172.21.15.204:0/3865384049 >> v2:172.21.15.204:6812/28210 conn(0x7f02040396e0 0x7f020403bb90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f020c034000 tx=0x7f020c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.067+0000 7f021cff9640 1 -- 172.21.15.204:0/3865384049 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0214010440 con 0x7f022009f990 2024-09-15T01:38:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.067+0000 7f021cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:20.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.067+0000 7f021cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:20.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.230+0000 7f0227217640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}] 2024-09-15T01:38:20.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.230+0000 7f0227217640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}] 2024-09-15T01:38:20.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.230+0000 7f0227217640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:20.232 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.230+0000 7f0227217640 1 -- 172.21.15.204:0/3865384049 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"} v 0) v1 -- 0x7f0220070b80 con 0x7f022009f990 2024-09-15T01:38:20.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.509+0000 7f021cff9640 1 -- 172.21.15.204:0/3865384049 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}]=0 v257) v1 ==== 106+0+3 (secure 0 0 0) 0x7f021400d370 con 0x7f022009f990 2024-09-15T01:38:20.510 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.509+0000 7f021cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}] 2024-09-15T01:38:20.511 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.509+0000 7f021cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:20.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.612+0000 7f0227217640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}] 2024-09-15T01:38:20.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.612+0000 7f0227217640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}] 2024-09-15T01:38:20.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.612+0000 7f0227217640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:20.613 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.612+0000 7f0227217640 1 -- 172.21.15.204:0/3865384049 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"} v 0) v1 -- 0x7f022009e480 con 0x7f022009f990 2024-09-15T01:38:20.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.612+0000 7f021cff9640 1 -- 172.21.15.204:0/3865384049 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}]=0 v257) v1 ==== 106+0+3 (secure 0 0 0) 0x7f021400eb30 con 0x7f022009f990 2024-09-15T01:38:20.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.612+0000 7f021cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}] 2024-09-15T01:38:20.614 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.612+0000 7f021cff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:38:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 1 -- 172.21.15.204:0/3865384049 >> v2:172.21.15.204:6812/28210 conn(0x7f02040396e0 msgr2=0x7f020403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 1 --2- 172.21.15.204:0/3865384049 >> v2:172.21.15.204:6812/28210 conn(0x7f02040396e0 0x7f020403bb90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f020c034000 tx=0x7f020c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 10 monclient: shutdown 2024-09-15T01:38:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 1 -- 172.21.15.204:0/3865384049 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f022009f990 msgr2=0x7f022011c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 1 --2- 172.21.15.204:0/3865384049 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f022009f990 0x7f022011c550 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f021400b600 tx=0x7f02140091a0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 1 -- 172.21.15.204:0/3865384049 shutdown_connections 2024-09-15T01:38:20.616 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 1 --2- 172.21.15.204:0/3865384049 >> v2:172.21.15.204:6812/28210 conn(0x7f02040396e0 0x7f020403bb90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 1 --2- 172.21.15.204:0/3865384049 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0220132930 0x7f022011cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 1 --2- 172.21.15.204:0/3865384049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02200a2400 0x7f022011ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 1 --2- 172.21.15.204:0/3865384049 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f022009f990 0x7f022011c550 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 1 -- 172.21.15.204:0/3865384049 >> 172.21.15.204:0/3865384049 conn(0x7f022006e860 msgr2=0x7f0220132d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:20.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 1 -- 172.21.15.204:0/3865384049 shutdown_connections 2024-09-15T01:38:20.617 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.614+0000 7f0227217640 1 -- 172.21.15.204:0/3865384049 wait complete. 2024-09-15T01:38:20.624 INFO:tasks.workunit.client.0.smithi204.stderr:+ nb=14 2024-09-15T01:38:20.625 INFO:tasks.workunit.client.0.smithi204.stderr:++ ceph osd create fa8c99c4-ab34-4381-a4b1-0acedd0271cf 2024-09-15T01:38:20.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.695+0000 7fcfe5c78640 10 monclient: get_monmap_and_config 2024-09-15T01:38:20.705 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.695+0000 7fcfe5c78640 10 monclient: build_initial_monmap 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.695+0000 7fcfe5c78640 10 monclient: monmap: 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:20.695676+0000 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:20.695676+0000 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:20.706 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.695+0000 7fcfe5c78640 1 Processor -- start 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 1 -- start start 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 10 monclient: init 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 10 monclient: _add_conns ranks=[8,4,6,1,5,7,2,3,10,9,0] 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcfe0067130 0x7fcfe0097d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 10 monclient(hunting): picked mon.noname-e con 0x7fcfe0067130 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:20.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcfe0067500 0x7fcfe0067930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 10 monclient(hunting): picked mon.noname-c con 0x7fcfe0067500 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:20.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcfe0132e60 0x7fcfe01332b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 10 monclient(hunting): picked mon.noname-d con 0x7fcfe0132e60 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:20.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:20.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe002a980 con 0x7fcfe0067500 2024-09-15T01:38:20.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:20.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe0099220 con 0x7fcfe0132e60 2024-09-15T01:38:20.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:20.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe002a6d0 con 0x7fcfe0067130 2024-09-15T01:38:20.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:20.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfe5c78640 10 monclient(hunting): authenticate will time out at 1116.338605s 2024-09-15T01:38:20.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfdf7fe640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcfe0067130 msgr2=0x7fcfe0097d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfdeffd640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcfe0067500 msgr2=0x7fcfe0067930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfdeffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcfe0067500 0x7fcfe0067930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfdf7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcfe0067130 0x7fcfe0097d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfdffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcfe0132e60 0x7fcfe01332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfdffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcfe0132e60 0x7fcfe01332b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:42820/0 (socket says 172.21.15.204:42820) 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfdffff640 1 -- 172.21.15.204:0/3854152221 learned_addr learned my addr 172.21.15.204:0/3854152221 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfdffff640 10 monclient(hunting): get_auth_request con 0x7fcfe0132e60 auth_method 0 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfdffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfdffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.696+0000 7fcfdffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:20.709 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 10 monclient(hunting): handle_auth_done global_id 24639 payload 274 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 1 -- 172.21.15.204:0/3854152221 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcfe0067130 msgr2=0x7fcfe0097d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 1 --2- 172.21.15.204:0/3854152221 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcfe0067130 0x7fcfe0097d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 1 -- 172.21.15.204:0/3854152221 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcfe0067500 msgr2=0x7fcfe0067930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 1 --2- 172.21.15.204:0/3854152221 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcfe0067500 0x7fcfe0067930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 1 monclient: found mon.noname-d 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 1 -- 172.21.15.204:0/3854152221 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe00648a0 con 0x7fcfe0132e60 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.697+0000 7fcfdffff640 1 --2- 172.21.15.204:0/3854152221 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcfe0132e60 0x7fcfe01332b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009f80 tx=0x7fcfd8009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=4060f702ba6ef6db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfde7fc640 1 -- 172.21.15.204:0/3854152221 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfd8007880 con 0x7fcfe0132e60 2024-09-15T01:38:20.710 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfde7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfde7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfde7fc640 10 monclient: dump: 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:20.711 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfde7fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfde7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfde7fc640 1 -- 172.21.15.204:0/3854152221 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd802d050 con 0x7fcfe0132e60 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 5 monclient: authenticate success, global_id 24639 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfde7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 4 set_mon_vals no callback set 2024-09-15T01:38:20.712 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfde7fc640 1 -- 172.21.15.204:0/3854152221 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfd8030360 con 0x7fcfe0132e60 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfde7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfde7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfde7fc640 10 monclient: dump: 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:20.713 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 10 monclient: shutdown 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3854152221 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcfe0132e60 msgr2=0x7fcfe01332b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 1 --2- 172.21.15.204:0/3854152221 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcfe0132e60 0x7fcfe01332b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009f80 tx=0x7fcfd8009840 comp rx=0 tx=0).stop 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3854152221 shutdown_connections 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 1 --2- 172.21.15.204:0/3854152221 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcfe0132e60 0x7fcfe01332b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 1 --2- 172.21.15.204:0/3854152221 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcfe0067500 0x7fcfe0067930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.714 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 1 --2- 172.21.15.204:0/3854152221 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcfe0067130 0x7fcfe0097d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3854152221 >> 172.21.15.204:0/3854152221 conn(0x7fcfe00a2ce0 msgr2=0x7fcfe00a0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3854152221 shutdown_connections 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.698+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3854152221 wait complete. 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.699+0000 7fcfe5c78640 10 monclient: build_initial_monmap 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.699+0000 7fcfe5c78640 10 monclient: monmap: 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:20.715 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.699+0000 7fcfe5c78640 1 Processor -- start 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.699+0000 7fcfe5c78640 1 -- start start 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.699+0000 7fcfe5c78640 10 monclient: init 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 10 monclient: _add_conns ranks=[0,6,7,5,3,1,4,8,2] 2024-09-15T01:38:20.716 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfe0067130 0x7fcfe006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 10 monclient(hunting): picked mon.noname-a con 0x7fcfe0067130 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcfe0067500 0x7fcfe011cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 10 monclient(hunting): picked mon.noname-d con 0x7fcfe0067500 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcfe0132e60 0x7fcfe011d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 10 monclient(hunting): picked mon.noname-i con 0x7fcfe0132e60 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe011fc70 con 0x7fcfe0067130 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe0098630 con 0x7fcfe0067500 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe002a4a0 con 0x7fcfe0132e60 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:20.717 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfe5c78640 10 monclient(hunting): authenticate will time out at 1116.342221s 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdf7fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfe0067130 msgr2=0x7fcfe006eef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdf7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfe0067130 0x7fcfe006eef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcfe0132e60 0x7fcfe011d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcfe0132e60 0x7fcfe011d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:39524/0 (socket says 172.21.15.204:39524) 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 1 -- 172.21.15.204:0/3013624107 learned_addr learned my addr 172.21.15.204:0/3013624107 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 10 monclient(hunting): get_auth_request con 0x7fcfe0132e60 auth_method 0 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:20.718 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdeffd640 1 --2- 172.21.15.204:0/3013624107 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcfe0067500 0x7fcfe011cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdeffd640 10 monclient(hunting): get_auth_request con 0x7fcfe0067500 auth_method 0 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 10 monclient(hunting): handle_auth_done global_id 24802 payload 824 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 1 -- 172.21.15.204:0/3013624107 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcfe0067500 msgr2=0x7fcfe011cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 1 --2- 172.21.15.204:0/3013624107 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcfe0067500 0x7fcfe011cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 1 -- 172.21.15.204:0/3013624107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfe0067130 msgr2=0x7fcfe006eef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 1 --2- 172.21.15.204:0/3013624107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfe0067130 0x7fcfe006eef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.719 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 1 monclient: found mon.noname-i 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 1 -- 172.21.15.204:0/3013624107 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd8009ae0 con 0x7fcfe0132e60 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfdffff640 1 --2- 172.21.15.204:0/3013624107 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcfe0132e60 0x7fcfe011d240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009880 tx=0x7fcfd8036000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfd77fe640 1 -- 172.21.15.204:0/3013624107 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfd8007b10 con 0x7fcfe0132e60 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfd77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfd77fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.700+0000 7fcfd77fe640 10 monclient: dump: 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.720 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:20.721 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfd77fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfd77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfe5c78640 5 monclient: authenticate success, global_id 24802 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfd77fe640 1 -- 172.21.15.204:0/3013624107 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd8007cb0 con 0x7fcfe0132e60 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfe5c78640 10 monclient: _renew_subs 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfe5c78640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3013624107 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe012f9b0 con 0x7fcfe0132e60 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfd77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfd77fe640 1 -- 172.21.15.204:0/3013624107 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfd8007e20 con 0x7fcfe0132e60 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfd77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfd77fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfd77fe640 10 monclient: dump: 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:20.722 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.723 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:20.723 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:20.723 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:20.723 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfddffb640 4 set_mon_vals no callback set 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfe5c78640 10 monclient: _renew_subs 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfe5c78640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:20.724 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3013624107 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe011de80 con 0x7fcfe0132e60 2024-09-15T01:38:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfd77fe640 1 -- 172.21.15.204:0/3013624107 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fcfd8003040 con 0x7fcfe0132e60 2024-09-15T01:38:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfe5c78640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfe5c78640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfe5c78640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.701+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3013624107 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe0097550 con 0x7fcfe0132e60 2024-09-15T01:38:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.702+0000 7fcfd77fe640 1 --2- 172.21.15.204:0/3013624107 >> v2:172.21.15.204:6812/28210 conn(0x7fcfc00397b0 0x7fcfc003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.702+0000 7fcfd77fe640 1 -- 172.21.15.204:0/3013624107 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fcfd802ae40 con 0x7fcfe0132e60 2024-09-15T01:38:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.702+0000 7fcfdf7fe640 1 --2- 172.21.15.204:0/3013624107 >> v2:172.21.15.204:6812/28210 conn(0x7fcfc00397b0 0x7fcfc003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.702+0000 7fcfdf7fe640 10 monclient: get_auth_request con 0x7fcfc00397b0 auth_method 0 2024-09-15T01:38:20.725 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.702+0000 7fcfdf7fe640 1 --2- 172.21.15.204:0/3013624107 >> v2:172.21.15.204:6812/28210 conn(0x7fcfc00397b0 0x7fcfc003bc60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8006fd0 tx=0x7fcfc8006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:20.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.706+0000 7fcfd77fe640 1 -- 172.21.15.204:0/3013624107 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcfd8043dd0 con 0x7fcfe0132e60 2024-09-15T01:38:20.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.706+0000 7fcfd77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:20.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.706+0000 7fcfd77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:20.873 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.872+0000 7fcfe5c78640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}] 2024-09-15T01:38:20.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.872+0000 7fcfe5c78640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}] 2024-09-15T01:38:20.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.872+0000 7fcfe5c78640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:20.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.872+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3013624107 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"} v 0) v1 -- 0x7fcfe0070b80 con 0x7fcfe0132e60 2024-09-15T01:38:20.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.872+0000 7fcfd77fe640 1 -- 172.21.15.204:0/3013624107 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}]=0 v257) v1 ==== 106+0+3 (secure 0 0 0) 0x7fcfd804a020 con 0x7fcfe0132e60 2024-09-15T01:38:20.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.872+0000 7fcfd77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}] 2024-09-15T01:38:20.874 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.872+0000 7fcfd77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:20.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.974+0000 7fcfe5c78640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}] 2024-09-15T01:38:20.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.974+0000 7fcfe5c78640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}] 2024-09-15T01:38:20.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.974+0000 7fcfe5c78640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:20.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.974+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3013624107 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"} v 0) v1 -- 0x7fcfe0096f90 con 0x7fcfe0132e60 2024-09-15T01:38:20.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.975+0000 7fcfd77fe640 1 -- 172.21.15.204:0/3013624107 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}]=0 v257) v1 ==== 106+0+3 (secure 0 0 0) 0x7fcfd8040070 con 0x7fcfe0132e60 2024-09-15T01:38:20.976 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.975+0000 7fcfd77fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "fa8c99c4-ab34-4381-a4b1-0acedd0271cf"}] 2024-09-15T01:38:20.977 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.975+0000 7fcfd77fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:38:20.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.976+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3013624107 >> v2:172.21.15.204:6812/28210 conn(0x7fcfc00397b0 msgr2=0x7fcfc003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:20.978 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 1 --2- 172.21.15.204:0/3013624107 >> v2:172.21.15.204:6812/28210 conn(0x7fcfc00397b0 0x7fcfc003bc60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8006fd0 tx=0x7fcfc8006d30 comp rx=0 tx=0).stop 2024-09-15T01:38:20.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 10 monclient: shutdown 2024-09-15T01:38:20.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:20.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3013624107 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcfe0132e60 msgr2=0x7fcfe011d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:20.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 1 --2- 172.21.15.204:0/3013624107 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcfe0132e60 0x7fcfe011d240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009880 tx=0x7fcfd8036000 comp rx=0 tx=0).stop 2024-09-15T01:38:20.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3013624107 shutdown_connections 2024-09-15T01:38:20.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 1 --2- 172.21.15.204:0/3013624107 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fcfe0067500 0x7fcfe011cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 1 --2- 172.21.15.204:0/3013624107 >> v2:172.21.15.204:6812/28210 conn(0x7fcfc00397b0 0x7fcfc003bc60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 1 --2- 172.21.15.204:0/3013624107 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fcfe0132e60 0x7fcfe011d240 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 1 --2- 172.21.15.204:0/3013624107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfe0067130 0x7fcfe006eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:20.979 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3013624107 >> 172.21.15.204:0/3013624107 conn(0x7fcfe00a2ce0 msgr2=0x7fcfe0099770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:20.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3013624107 shutdown_connections 2024-09-15T01:38:20.980 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:20.977+0000 7fcfe5c78640 1 -- 172.21.15.204:0/3013624107 wait complete. 2024-09-15T01:38:20.987 INFO:tasks.workunit.client.0.smithi204.stderr:+ test 14 -eq 14 2024-09-15T01:38:20.987 INFO:tasks.workunit.client.0.smithi204.stderr:+ test 14 -ne 12 2024-09-15T01:38:20.987 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd rm 12 2024-09-15T01:38:21.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.058+0000 7f35c2954640 10 monclient: get_monmap_and_config 2024-09-15T01:38:21.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.058+0000 7f35c2954640 10 monclient: build_initial_monmap 2024-09-15T01:38:21.059 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.058+0000 7f35c2954640 10 monclient: monmap: 2024-09-15T01:38:21.060 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:21.060 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:21.060 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:21.058614+0000 2024-09-15T01:38:21.060 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:21.058614+0000 2024-09-15T01:38:21.060 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:21.060 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:21.060 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:21.060 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:21.060 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:21.060 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:21.060 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:21.061 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:21.061 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:21.061 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:21.061 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:21.061 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:21.061 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:21.061 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:21.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 1 Processor -- start 2024-09-15T01:38:21.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 1 -- start start 2024-09-15T01:38:21.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 10 monclient: init 2024-09-15T01:38:21.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:21.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 10 monclient: _add_conns ranks=[0,7,2,4,5,8,3,6,9,1,10] 2024-09-15T01:38:21.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f35bc09f6c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:21.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc069eb0 0x7f35bc137d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 10 monclient(hunting): picked mon.noname-i con 0x7f35bc069eb0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:21.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35bc130050 0x7f35bc13a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 10 monclient(hunting): picked mon.noname-b con 0x7f35bc130050 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:21.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35bc0648a0 con 0x7f35bc09f6c0 2024-09-15T01:38:21.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:21.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc069840 con 0x7f35bc130050 2024-09-15T01:38:21.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:21.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc09fa90 con 0x7f35bc069eb0 2024-09-15T01:38:21.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:21.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c2954640 10 monclient(hunting): authenticate will time out at 1116.701683s 2024-09-15T01:38:21.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35bbfff640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f35bc09f6c0 legacy=0x7f35bc12f940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:38:21.070 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c0ef1640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35bc130050 msgr2=0x7f35bc13a270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35b3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc069eb0 0x7f35bc137d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35c0ef1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35bc130050 0x7f35bc13a270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35b3fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc069eb0 0x7f35bc137d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:39536/0 (socket says 172.21.15.204:39536) 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35b3fff640 1 -- 172.21.15.204:0/2078403396 learned_addr learned my addr 172.21.15.204:0/2078403396 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35b3fff640 10 monclient(hunting): get_auth_request con 0x7f35bc069eb0 auth_method 0 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.059+0000 7f35b3fff640 10 monclient(hunting): handle_auth_done global_id 24811 payload 274 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35b3fff640 1 -- 172.21.15.204:0/2078403396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35bc130050 msgr2=0x7f35bc13a270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:21.071 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35b3fff640 1 --2- 172.21.15.204:0/2078403396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35bc130050 0x7f35bc13a270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35b3fff640 1 -- 172.21.15.204:0/2078403396 >> v1:172.21.15.178:6789/0 conn(0x7f35bc09f6c0 legacy=0x7f35bc12f940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35b3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35b3fff640 1 monclient: found mon.noname-i 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35b3fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35b3fff640 1 -- 172.21.15.204:0/2078403396 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc0a0460 con 0x7f35bc069eb0 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35b3fff640 1 --2- 172.21.15.204:0/2078403396 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc069eb0 0x7f35bc137d20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009940 tx=0x7f35a40091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=2444471c6c208a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35bb7fe640 1 -- 172.21.15.204:0/2078403396 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f35a402adb0 con 0x7f35bc069eb0 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35bb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35bb7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35bb7fe640 10 monclient: dump: 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:21.072 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:21.073 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.073 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.073 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:21.073 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:21.073 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:21.073 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:21.073 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:21.073 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:21.073 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:21.073 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:21.073 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:21.073 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35bb7fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35c2954640 5 monclient: authenticate success, global_id 24811 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35c2954640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35bb7fe640 1 -- 172.21.15.204:0/2078403396 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a4031820 con 0x7f35bc069eb0 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35bb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35c2954640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35c2954640 4 set_mon_vals no callback set 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35bb7fe640 1 -- 172.21.15.204:0/2078403396 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f35a4031e50 con 0x7f35bc069eb0 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35bb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:21.074 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35bb7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.060+0000 7f35bb7fe640 10 monclient: dump: 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:21.075 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.061+0000 7f35c2954640 10 monclient: shutdown 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.061+0000 7f35c2954640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.061+0000 7f35c2954640 1 -- 172.21.15.204:0/2078403396 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc069eb0 msgr2=0x7f35bc137d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.061+0000 7f35c2954640 1 --2- 172.21.15.204:0/2078403396 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc069eb0 0x7f35bc137d20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009940 tx=0x7f35a40091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.061+0000 7f35c2954640 1 -- 172.21.15.204:0/2078403396 shutdown_connections 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.061+0000 7f35c2954640 1 --2- 172.21.15.204:0/2078403396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35bc130050 0x7f35bc13a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.061+0000 7f35c2954640 1 --2- 172.21.15.204:0/2078403396 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc069eb0 0x7f35bc137d20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.061+0000 7f35c2954640 1 -- 172.21.15.204:0/2078403396 >> 172.21.15.204:0/2078403396 conn(0x7f35bc067c00 msgr2=0x7f35bc0a3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.061+0000 7f35c2954640 1 -- 172.21.15.204:0/2078403396 shutdown_connections 2024-09-15T01:38:21.076 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.061+0000 7f35c2954640 1 -- 172.21.15.204:0/2078403396 wait complete. 2024-09-15T01:38:21.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient: build_initial_monmap 2024-09-15T01:38:21.077 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient: monmap: 2024-09-15T01:38:21.077 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:21.077 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:21.077 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:21.077 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:21.077 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:21.077 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:21.077 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:21.077 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:21.077 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:21.078 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:21.078 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:21.078 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:21.078 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:21.078 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:21.078 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:21.078 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:21.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 1 Processor -- start 2024-09-15T01:38:21.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 1 -- start start 2024-09-15T01:38:21.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient: init 2024-09-15T01:38:21.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:21.078 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient: _add_conns ranks=[4,3,7,2,0,5,8,6,1] 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f35bc069eb0 0x7f35bc141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient(hunting): picked mon.noname-c con 0x7f35bc069eb0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc09f6c0 0x7f35bc12f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient(hunting): picked mon.noname-g con 0x7f35bc09f6c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc130050 0x7f35bc11be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient(hunting): picked mon.noname-i con 0x7f35bc130050 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc068c10 con 0x7f35bc09f6c0 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc068950 con 0x7f35bc069eb0 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc11f660 con 0x7f35bc130050 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:21.079 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c2954640 10 monclient(hunting): authenticate will time out at 1116.704831s 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35bbfff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f35bc069eb0 msgr2=0x7f35bc141c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c0ef1640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc130050 0x7f35bc11be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35bbfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f35bc069eb0 0x7f35bc141c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35b3fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc09f6c0 0x7f35bc12f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35b3fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc09f6c0 0x7f35bc12f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:44446/0 (socket says 172.21.15.204:44446) 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c0ef1640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc130050 0x7f35bc11be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:39552/0 (socket says 172.21.15.204:39552) 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35b3fff640 1 -- 172.21.15.204:0/755380386 learned_addr learned my addr 172.21.15.204:0/755380386 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35b3fff640 10 monclient(hunting): get_auth_request con 0x7f35bc09f6c0 auth_method 0 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c0ef1640 10 monclient(hunting): get_auth_request con 0x7f35bc130050 auth_method 0 2024-09-15T01:38:21.080 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c0ef1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c0ef1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.062+0000 7f35c0ef1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35c0ef1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35c0ef1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35c0ef1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 10 monclient(hunting): handle_auth_done global_id 15717 payload 824 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 1 -- 172.21.15.204:0/755380386 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc130050 msgr2=0x7f35bc11be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 1 --2- 172.21.15.204:0/755380386 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc130050 0x7f35bc11be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 1 -- 172.21.15.204:0/755380386 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f35bc069eb0 msgr2=0x7f35bc141c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 1 --2- 172.21.15.204:0/755380386 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f35bc069eb0 0x7f35bc141c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.081 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 1 monclient: found mon.noname-g 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 1 -- 172.21.15.204:0/755380386 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a4009450 con 0x7f35bc09f6c0 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35c0ef1640 1 --2- 172.21.15.204:0/755380386 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc130050 0x7f35bc11be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b3fff640 1 --2- 172.21.15.204:0/755380386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc09f6c0 0x7f35bc12f230 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f35a40099a0 tx=0x7f35a4032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b8ff9640 1 -- 172.21.15.204:0/755380386 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f35a4033e30 con 0x7f35bc09f6c0 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b8ff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b8ff9640 10 monclient: dump: 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:21.082 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:21.083 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:21.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b8ff9640 10 monclient: _finish_auth 0 2024-09-15T01:38:21.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:21.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b8ff9640 1 -- 172.21.15.204:0/755380386 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a403b3f0 con 0x7f35bc09f6c0 2024-09-15T01:38:21.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35c2954640 5 monclient: authenticate success, global_id 15717 2024-09-15T01:38:21.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35c2954640 10 monclient: _renew_subs 2024-09-15T01:38:21.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35c2954640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:21.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35c2954640 1 -- 172.21.15.204:0/755380386 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc11f3e0 con 0x7f35bc09f6c0 2024-09-15T01:38:21.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:21.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b8ff9640 1 -- 172.21.15.204:0/755380386 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f35a403b9e0 con 0x7f35bc09f6c0 2024-09-15T01:38:21.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:21.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b8ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:21.084 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35b8ff9640 10 monclient: dump: 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:21.085 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35baffd640 4 set_mon_vals no callback set 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35c2954640 10 monclient: _renew_subs 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35c2954640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.063+0000 7f35c2954640 1 -- 172.21.15.204:0/755380386 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc142b30 con 0x7f35bc09f6c0 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.064+0000 7f35b8ff9640 1 -- 172.21.15.204:0/755380386 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f35a4039040 con 0x7f35bc09f6c0 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.064+0000 7f35b8ff9640 1 --2- 172.21.15.204:0/755380386 >> v2:172.21.15.204:6812/28210 conn(0x7f359c0396e0 0x7f359c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.064+0000 7f35b8ff9640 1 -- 172.21.15.204:0/755380386 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f35a406b830 con 0x7f35bc09f6c0 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.064+0000 7f35c2954640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.064+0000 7f35bbfff640 1 --2- 172.21.15.204:0/755380386 >> v2:172.21.15.204:6812/28210 conn(0x7f359c0396e0 0x7f359c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:21.086 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.064+0000 7f35c2954640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:21.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.064+0000 7f35c2954640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:21.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.064+0000 7f35c2954640 1 -- 172.21.15.204:0/755380386 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc070b80 con 0x7f35bc09f6c0 2024-09-15T01:38:21.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.064+0000 7f35bbfff640 10 monclient: get_auth_request con 0x7f359c0396e0 auth_method 0 2024-09-15T01:38:21.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.065+0000 7f35bbfff640 1 --2- 172.21.15.204:0/755380386 >> v2:172.21.15.204:6812/28210 conn(0x7f359c0396e0 0x7f359c03bb90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f35ac006fd0 tx=0x7f35ac006d30 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:21.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.069+0000 7f35b8ff9640 1 -- 172.21.15.204:0/755380386 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f35a4042020 con 0x7f35bc09f6c0 2024-09-15T01:38:21.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.069+0000 7f35b8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:21.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.069+0000 7f35b8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:21.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.232+0000 7f35c2954640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T01:38:21.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.232+0000 7f35c2954640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T01:38:21.234 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.232+0000 7f35c2954640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:21.235 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.232+0000 7f35c2954640 1 -- 172.21.15.204:0/755380386 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) v1 -- 0x7f35bc069180 con 0x7f35bc09f6c0 2024-09-15T01:38:21.235 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:38:21.233+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e257 definitely_dead 0 2024-09-15T01:38:21.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.540+0000 7f35b8ff9640 1 -- 172.21.15.204:0/755380386 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 removed osd.12 v258) v1 ==== 83+0+0 (secure 0 0 0) 0x7f35a4037070 con 0x7f35bc09f6c0 2024-09-15T01:38:21.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.540+0000 7f35b8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T01:38:21.541 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.540+0000 7f35b8ff9640 10 monclient: _finish_command 2 = system:0 removed osd.12 2024-09-15T01:38:21.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.641+0000 7f35c2954640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T01:38:21.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.641+0000 7f35c2954640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T01:38:21.642 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.641+0000 7f35c2954640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:21.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.641+0000 7f35c2954640 1 -- 172.21.15.204:0/755380386 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) v1 -- 0x7f35bc069180 con 0x7f35bc09f6c0 2024-09-15T01:38:21.643 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:38:21.641+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e258 definitely_dead 0 2024-09-15T01:38:21.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.641+0000 7f35b8ff9640 1 -- 172.21.15.204:0/755380386 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 osd.12 does not exist. v258) v1 ==== 92+0+0 (secure 0 0 0) 0x7f35a403bb40 con 0x7f35bc09f6c0 2024-09-15T01:38:21.643 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.642+0000 7f35b8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T01:38:21.644 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.642+0000 7f35b8ff9640 10 monclient: _finish_command 3 = system:0 osd.12 does not exist. 2024-09-15T01:38:21.644 INFO:tasks.workunit.client.0.smithi204.stderr:osd.12 does not exist. 2024-09-15T01:38:21.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 1 -- 172.21.15.204:0/755380386 >> v2:172.21.15.204:6812/28210 conn(0x7f359c0396e0 msgr2=0x7f359c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:21.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 1 --2- 172.21.15.204:0/755380386 >> v2:172.21.15.204:6812/28210 conn(0x7f359c0396e0 0x7f359c03bb90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f35ac006fd0 tx=0x7f35ac006d30 comp rx=0 tx=0).stop 2024-09-15T01:38:21.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 10 monclient: shutdown 2024-09-15T01:38:21.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:21.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 1 -- 172.21.15.204:0/755380386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc09f6c0 msgr2=0x7f35bc12f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:21.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 1 --2- 172.21.15.204:0/755380386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc09f6c0 0x7f35bc12f230 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f35a40099a0 tx=0x7f35a4032000 comp rx=0 tx=0).stop 2024-09-15T01:38:21.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 1 -- 172.21.15.204:0/755380386 shutdown_connections 2024-09-15T01:38:21.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 1 --2- 172.21.15.204:0/755380386 >> v2:172.21.15.204:6812/28210 conn(0x7f359c0396e0 0x7f359c03bb90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 1 --2- 172.21.15.204:0/755380386 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f35bc130050 0x7f35bc11be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 1 --2- 172.21.15.204:0/755380386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc09f6c0 0x7f35bc12f230 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 1 --2- 172.21.15.204:0/755380386 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f35bc069eb0 0x7f35bc141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.646 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 1 -- 172.21.15.204:0/755380386 >> 172.21.15.204:0/755380386 conn(0x7f35bc067c00 msgr2=0x7f35bc06d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:21.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 1 -- 172.21.15.204:0/755380386 shutdown_connections 2024-09-15T01:38:21.647 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.644+0000 7f35c2954640 1 -- 172.21.15.204:0/755380386 wait complete. 2024-09-15T01:38:21.654 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd rm 12 2024-09-15T01:38:21.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.724+0000 7f6620680640 10 monclient: get_monmap_and_config 2024-09-15T01:38:21.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.724+0000 7f6620680640 10 monclient: build_initial_monmap 2024-09-15T01:38:21.726 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.724+0000 7f6620680640 10 monclient: monmap: 2024-09-15T01:38:21.726 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:21.726 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:21.726 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:21.725204+0000 2024-09-15T01:38:21.726 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:21.725204+0000 2024-09-15T01:38:21.726 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:21.726 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:21.726 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:21.726 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:21.727 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:21.727 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:21.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.725+0000 7f6620680640 1 Processor -- start 2024-09-15T01:38:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.725+0000 7f6620680640 1 -- start start 2024-09-15T01:38:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.725+0000 7f6620680640 10 monclient: init 2024-09-15T01:38:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.725+0000 7f6620680640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.725+0000 7f6620680640 10 monclient: _add_conns ranks=[6,4,3,2,0,1,5,10,9,7,8] 2024-09-15T01:38:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.725+0000 7f6620680640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66180a2400 0x7f66180a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.725+0000 7f6620680640 10 monclient(hunting): picked mon.noname-d con 0x7f66180a2400 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.725+0000 7f6620680640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f661809f990 0x7f661809fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f6620680640 10 monclient(hunting): picked mon.noname-c con 0x7f661809f990 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f6620680640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6618132930 0x7f6618132d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f6620680640 10 monclient(hunting): picked mon.noname-g con 0x7f6618132930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f6620680640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:21.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f6620680640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661809e1a0 con 0x7f6618132930 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f6620680640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f6620680640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66180a27d0 con 0x7f661809f990 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f6620680640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f6620680640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f661809dec0 con 0x7f66180a2400 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f6620680640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f6620680640 10 monclient(hunting): authenticate will time out at 1117.368061s 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661dc1b640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f661809f990 msgr2=0x7f661809fd60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6618132930 0x7f6618132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661dc1b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f661809f990 0x7f661809fd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6618132930 0x7f6618132d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:44450/0 (socket says 172.21.15.204:44450) 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 1 -- 172.21.15.204:0/3620025310 learned_addr learned my addr 172.21.15.204:0/3620025310 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 10 monclient(hunting): get_auth_request con 0x7f6618132930 auth_method 0 2024-09-15T01:38:21.736 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661e41c640 1 --2- 172.21.15.204:0/3620025310 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66180a2400 0x7f66180a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 10 monclient(hunting): handle_auth_done global_id 15726 payload 274 2024-09-15T01:38:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 1 -- 172.21.15.204:0/3620025310 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66180a2400 msgr2=0x7f66180a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 1 --2- 172.21.15.204:0/3620025310 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66180a2400 0x7f66180a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 1 -- 172.21.15.204:0/3620025310 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f661809f990 msgr2=0x7f661809fd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 1 --2- 172.21.15.204:0/3620025310 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f661809f990 0x7f661809fd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.737 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 1 monclient: found mon.noname-g 2024-09-15T01:38:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.726+0000 7f661ec1d640 1 -- 172.21.15.204:0/3620025310 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180648a0 con 0x7f6618132930 2024-09-15T01:38:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661ec1d640 1 --2- 172.21.15.204:0/3620025310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6618132930 0x7f6618132d60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6610009f50 tx=0x7f6610009840 comp rx=0 tx=0).ready entity=mon.3 client_cookie=7f11fdf7702adaa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661d41a640 1 -- 172.21.15.204:0/3620025310 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6610007880 con 0x7f6618132930 2024-09-15T01:38:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661d41a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661d41a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661d41a640 10 monclient: dump: 2024-09-15T01:38:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:21.738 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:21.739 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:21.740 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:21.740 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:21.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661d41a640 10 monclient: _finish_auth 0 2024-09-15T01:38:21.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661d41a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:21.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661d41a640 1 -- 172.21.15.204:0/3620025310 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661002d050 con 0x7f6618132930 2024-09-15T01:38:21.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 5 monclient: authenticate success, global_id 15726 2024-09-15T01:38:21.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:21.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661d41a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:21.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:21.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661d41a640 1 -- 172.21.15.204:0/3620025310 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6610030360 con 0x7f6618132930 2024-09-15T01:38:21.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661d41a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:21.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 4 set_mon_vals no callback set 2024-09-15T01:38:21.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661d41a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:21.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f661d41a640 10 monclient: dump: 2024-09-15T01:38:21.741 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:21.741 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:21.741 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.741 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.741 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:21.741 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:21.741 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:21.741 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:21.741 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:21.741 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:21.742 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:21.742 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:21.742 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:21.742 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:21.742 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:21.742 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:21.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 10 monclient: shutdown 2024-09-15T01:38:21.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:21.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 1 -- 172.21.15.204:0/3620025310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6618132930 msgr2=0x7f6618132d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:21.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 1 --2- 172.21.15.204:0/3620025310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6618132930 0x7f6618132d60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6610009f50 tx=0x7f6610009840 comp rx=0 tx=0).stop 2024-09-15T01:38:21.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 1 -- 172.21.15.204:0/3620025310 shutdown_connections 2024-09-15T01:38:21.742 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 1 --2- 172.21.15.204:0/3620025310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6618132930 0x7f6618132d60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 1 --2- 172.21.15.204:0/3620025310 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f661809f990 0x7f661809fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 1 --2- 172.21.15.204:0/3620025310 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66180a2400 0x7f66180a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 1 -- 172.21.15.204:0/3620025310 >> 172.21.15.204:0/3620025310 conn(0x7f661806e860 msgr2=0x7f661806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.727+0000 7f6620680640 1 -- 172.21.15.204:0/3620025310 shutdown_connections 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.728+0000 7f6620680640 1 -- 172.21.15.204:0/3620025310 wait complete. 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.728+0000 7f6620680640 10 monclient: build_initial_monmap 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.728+0000 7f6620680640 10 monclient: monmap: 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:21.743 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.728+0000 7f6620680640 1 Processor -- start 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.728+0000 7f6620680640 1 -- start start 2024-09-15T01:38:21.744 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.728+0000 7f6620680640 10 monclient: init 2024-09-15T01:38:21.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:21.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 10 monclient: _add_conns ranks=[4,7,8,1,0,6,5,2,3] 2024-09-15T01:38:21.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f661809f990 0x7f6618141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 10 monclient(hunting): picked mon.noname-c con 0x7f661809f990 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:21.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f66180a2400 0x7f6618142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 10 monclient(hunting): picked mon.noname-i con 0x7f66180a2400 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:21.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6618132930 0x7f661811d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 10 monclient(hunting): picked mon.noname-e con 0x7f6618132930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:21.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:21.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f661809d3d0 con 0x7f661809f990 2024-09-15T01:38:21.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:21.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f661809cad0 con 0x7f66180a2400 2024-09-15T01:38:21.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:21.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6618120b10 con 0x7f6618132930 2024-09-15T01:38:21.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:21.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f6620680640 10 monclient(hunting): authenticate will time out at 1117.371342s 2024-09-15T01:38:21.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661e41c640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f661809f990 msgr2=0x7f6618141ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:21.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661e41c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f661809f990 0x7f6618141ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:21.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f66180a2400 0x7f6618142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:21.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661ec1d640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6618132930 msgr2=0x7f661811d220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:38:21.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661ec1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6618132930 0x7f661811d220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:21.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f66180a2400 0x7f6618142530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:39564/0 (socket says 172.21.15.204:39564) 2024-09-15T01:38:21.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 1 -- 172.21.15.204:0/2676957953 learned_addr learned my addr 172.21.15.204:0/2676957953 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 10 monclient(hunting): get_auth_request con 0x7f66180a2400 auth_method 0 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 10 monclient(hunting): handle_auth_done global_id 24829 payload 824 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 1 -- 172.21.15.204:0/2676957953 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6618132930 msgr2=0x7f661811d220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 1 --2- 172.21.15.204:0/2676957953 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6618132930 0x7f661811d220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 1 -- 172.21.15.204:0/2676957953 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f661809f990 msgr2=0x7f6618141ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 1 --2- 172.21.15.204:0/2676957953 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f661809f990 0x7f6618141ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:21.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 10 monclient: _finish_hunting 0 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 1 monclient: found mon.noname-i 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 1 -- 172.21.15.204:0/2676957953 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610009ae0 con 0x7f66180a2400 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f661dc1b640 1 --2- 172.21.15.204:0/2676957953 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f66180a2400 0x7f6618142530 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f660800b630 tx=0x7f66080091a0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f660e7fc640 1 -- 172.21.15.204:0/2676957953 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6608015070 con 0x7f66180a2400 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.729+0000 7f660e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f660e7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f660e7fc640 10 monclient: dump: 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.748 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f660e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:21.749 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f660e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f6620680640 5 monclient: authenticate success, global_id 24829 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f660e7fc640 1 -- 172.21.15.204:0/2676957953 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6608006150 con 0x7f66180a2400 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f660e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f660e7fc640 1 -- 172.21.15.204:0/2676957953 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6608006740 con 0x7f66180a2400 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f6620680640 10 monclient: _renew_subs 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f6620680640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f6620680640 1 -- 172.21.15.204:0/2676957953 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6618120830 con 0x7f66180a2400 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f661cc19640 4 set_mon_vals no callback set 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f660e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f660e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f660e7fc640 10 monclient: dump: 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:21.750 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:21.751 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f6620680640 10 monclient: _renew_subs 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f6620680640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.730+0000 7f6620680640 1 -- 172.21.15.204:0/2676957953 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661811f940 con 0x7f66180a2400 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.731+0000 7f6620680640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.731+0000 7f6620680640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.731+0000 7f6620680640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.731+0000 7f660e7fc640 1 -- 172.21.15.204:0/2676957953 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f66080062c0 con 0x7f66180a2400 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.731+0000 7f6620680640 1 -- 172.21.15.204:0/2676957953 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6618121310 con 0x7f66180a2400 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.731+0000 7f660e7fc640 1 --2- 172.21.15.204:0/2676957953 >> v2:172.21.15.204:6812/28210 conn(0x7f65fc0397e0 0x7f65fc03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.731+0000 7f660e7fc640 1 -- 172.21.15.204:0/2676957953 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f660804b510 con 0x7f66180a2400 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.731+0000 7f661e41c640 1 --2- 172.21.15.204:0/2676957953 >> v2:172.21.15.204:6812/28210 conn(0x7f65fc0397e0 0x7f65fc03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.731+0000 7f661e41c640 10 monclient: get_auth_request con 0x7f65fc0397e0 auth_method 0 2024-09-15T01:38:21.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.731+0000 7f661e41c640 1 --2- 172.21.15.204:0/2676957953 >> v2:172.21.15.204:6812/28210 conn(0x7f65fc0397e0 0x7f65fc03bc90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6614006fd0 tx=0x7f6614008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:21.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.735+0000 7f660e7fc640 1 -- 172.21.15.204:0/2676957953 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6618121310 con 0x7f66180a2400 2024-09-15T01:38:21.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.735+0000 7f660e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:21.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.735+0000 7f660e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:21.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.900+0000 7f6620680640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T01:38:21.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.900+0000 7f6620680640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T01:38:21.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.900+0000 7f6620680640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:21.902 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.900+0000 7f6620680640 1 -- 172.21.15.204:0/2676957953 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) v1 -- 0x7f6618121310 con 0x7f66180a2400 2024-09-15T01:38:21.903 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:38:21.901+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e258 definitely_dead 0 2024-09-15T01:38:21.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.901+0000 7f660e7fc640 1 -- 172.21.15.204:0/2676957953 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 osd.12 does not exist. v258) v1 ==== 92+0+0 (secure 0 0 0) 0x7f6618121310 con 0x7f66180a2400 2024-09-15T01:38:21.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.901+0000 7f660e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T01:38:21.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:21.901+0000 7f660e7fc640 10 monclient: _finish_command 2 = system:0 osd.12 does not exist. 2024-09-15T01:38:22.002 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.001+0000 7f6620680640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T01:38:22.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.001+0000 7f6620680640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T01:38:22.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.001+0000 7f6620680640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:22.003 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.001+0000 7f6620680640 1 -- 172.21.15.204:0/2676957953 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) v1 -- 0x7f6618121310 con 0x7f66180a2400 2024-09-15T01:38:22.003 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:38:22.001+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e258 definitely_dead 0 2024-09-15T01:38:22.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.002+0000 7f660e7fc640 1 -- 172.21.15.204:0/2676957953 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 osd.12 does not exist. v258) v1 ==== 92+0+0 (secure 0 0 0) 0x7f6618121310 con 0x7f66180a2400 2024-09-15T01:38:22.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.002+0000 7f660e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-09-15T01:38:22.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.002+0000 7f660e7fc640 10 monclient: _finish_command 3 = system:0 osd.12 does not exist. 2024-09-15T01:38:22.004 INFO:tasks.workunit.client.0.smithi204.stderr:osd.12 does not exist. 2024-09-15T01:38:22.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.003+0000 7f6620680640 1 -- 172.21.15.204:0/2676957953 >> v2:172.21.15.204:6812/28210 conn(0x7f65fc0397e0 msgr2=0x7f65fc03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:22.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.003+0000 7f6620680640 1 --2- 172.21.15.204:0/2676957953 >> v2:172.21.15.204:6812/28210 conn(0x7f65fc0397e0 0x7f65fc03bc90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6614006fd0 tx=0x7f6614008040 comp rx=0 tx=0).stop 2024-09-15T01:38:22.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.004+0000 7f6620680640 10 monclient: shutdown 2024-09-15T01:38:22.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.004+0000 7f6620680640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:22.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.004+0000 7f6620680640 1 -- 172.21.15.204:0/2676957953 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f66180a2400 msgr2=0x7f6618142530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:22.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.004+0000 7f6620680640 1 --2- 172.21.15.204:0/2676957953 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f66180a2400 0x7f6618142530 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f660800b630 tx=0x7f66080091a0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.004+0000 7f6620680640 1 -- 172.21.15.204:0/2676957953 shutdown_connections 2024-09-15T01:38:22.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.004+0000 7f6620680640 1 --2- 172.21.15.204:0/2676957953 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6618132930 0x7f661811d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.004+0000 7f6620680640 1 --2- 172.21.15.204:0/2676957953 >> v2:172.21.15.204:6812/28210 conn(0x7f65fc0397e0 0x7f65fc03bc90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.004+0000 7f6620680640 1 --2- 172.21.15.204:0/2676957953 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f66180a2400 0x7f6618142530 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.004+0000 7f6620680640 1 --2- 172.21.15.204:0/2676957953 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f661809f990 0x7f6618141ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.004+0000 7f6620680640 1 -- 172.21.15.204:0/2676957953 >> 172.21.15.204:0/2676957953 conn(0x7f661806e860 msgr2=0x7f661809fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:22.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.004+0000 7f6620680640 1 -- 172.21.15.204:0/2676957953 shutdown_connections 2024-09-15T01:38:22.007 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.004+0000 7f6620680640 1 -- 172.21.15.204:0/2676957953 wait complete. 2024-09-15T01:38:22.014 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd rm 14 2024-09-15T01:38:22.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.085+0000 7fdcde31d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:22.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.085+0000 7fdcde31d640 10 monclient: build_initial_monmap 2024-09-15T01:38:22.087 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.085+0000 7fdcde31d640 10 monclient: monmap: 2024-09-15T01:38:22.087 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:22.087 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:22.087 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:22.085920+0000 2024-09-15T01:38:22.087 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:22.085920+0000 2024-09-15T01:38:22.087 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:22.087 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:22.087 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:22.088 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:22.088 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:22.088 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:22.088 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:22.088 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:22.088 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:22.088 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:22.088 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:22.088 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:22.088 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:22.088 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 1 Processor -- start 2024-09-15T01:38:22.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 1 -- start start 2024-09-15T01:38:22.098 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 10 monclient: init 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 10 monclient: _add_conns ranks=[1,6,0,2,4,8,5,10,7,9,3] 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fdcd80a2400 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdcd809f990 0x7fdcd809fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 10 monclient(hunting): picked mon.noname-d con 0x7fdcd809f990 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdcd8137c60 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd80648a0 con 0x7fdcd8137c60 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd809e1a0 con 0x7fdcd80a2400 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd80a11f0 con 0x7fdcd809f990 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcde31d640 10 monclient(hunting): authenticate will time out at 1117.728731s 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcd7fff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fdcd80a2400 0x7fdcd80a0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:54266/0 (socket says 172.21.15.204:54266) 2024-09-15T01:38:22.099 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcd7fff640 1 -- 172.21.15.204:0/2787272675 learned_addr learned my addr 172.21.15.204:0/2787272675 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:38:22.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcdc8ba640 1 -- 172.21.15.204:0/2787272675 >> v1:172.21.15.178:6789/0 conn(0x7fdcd8137c60 legacy=0x7fdcd813a040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:38:22.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.086+0000 7fdcd77fe640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdcd809f990 0x7fdcd809fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd77fe640 10 monclient(hunting): get_auth_request con 0x7fdcd809f990 auth_method 0 2024-09-15T01:38:22.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:22.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429928903 0 0) 0x7fdcd809e1a0 con 0x7fdcd80a2400 2024-09-15T01:38:22.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:22.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient(hunting): my global_id is 7489 2024-09-15T01:38:22.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc00034e0 con 0x7fdcd80a2400 2024-09-15T01:38:22.100 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4273327543 0 0) 0x7fdcc00034e0 con 0x7fdcd80a2400 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdcd809f990 msgr2=0x7fdcd809fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdcd809f990 0x7fdcd809fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 >> v1:172.21.15.178:6789/0 conn(0x7fdcd8137c60 legacy=0x7fdcd813a040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd80a27d0 con 0x7fdcd80a2400 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:38:22.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fdcc80018f0 con 0x7fdcd80a2400 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcde31d640 5 monclient: authenticate success, global_id 7489 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcde31d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient: dump: 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:22.102 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 >> v1:172.21.15.204:6789/0 conn(0x7fdcd80a2400 legacy=0x7fdcd80a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:22.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient: _add_conns ranks=[4,2,7,3,8,5,1,0,6] 2024-09-15T01:38:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdcc0004220 0x7fdcc0004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient(hunting): picked mon.c con 0x7fdcc0004220 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcc0004cc0 0x7fdcc0005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient(hunting): picked mon.b con 0x7fdcc0004cc0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdcc0009c60 0x7fdcc000c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient(hunting): picked mon.i con 0x7fdcc0009c60 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd80a11f0 con 0x7fdcc0004cc0 2024-09-15T01:38:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc000c9c0 con 0x7fdcc0004220 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc000cb30 con 0x7fdcc0009c60 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd6ffd640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcde31d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd7fff640 1 -- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdcc0004220 msgr2=0x7fdcc0004780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd77fe640 1 -- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcc0004cc0 msgr2=0x7fdcc0005130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd7fff640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdcc0004220 0x7fdcc0004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcd77fe640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcc0004cc0 0x7fdcc0005130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdcc0009c60 0x7fdcc000c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 10 monclient(hunting): get_auth_request con 0x7fdcc0009c60 auth_method 0 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:38:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 10 monclient(hunting): handle_auth_done global_id 7489 payload 293 2024-09-15T01:38:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 1 -- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdcc0004220 msgr2=0x7fdcc0004780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdcc0004220 0x7fdcc0004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 1 -- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcc0004cc0 msgr2=0x7fdcc0005130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcc0004cc0 0x7fdcc0005130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 10 monclient: _finish_hunting 0 2024-09-15T01:38:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 1 monclient: found mon.i 2024-09-15T01:38:22.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 1 -- 172.21.15.204:0/2787272675 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc00034e0 con 0x7fdcc0009c60 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 10 monclient: _finish_auth 0 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.087+0000 7fdcdc8ba640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcdc8ba640 1 -- 172.21.15.204:0/2787272675 reap_dead start 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcdc8ba640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdcc0009c60 0x7fdcc000c0e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009f90 tx=0x7fdcd00095d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=eee24f82dc5c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdcd00328b0 con 0x7fdcc0009c60 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcd6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcd6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcd6ffd640 10 monclient: dump: 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.107 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcde31d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:22.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd002d070 con 0x7fdcc0009c60 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcd6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcde31d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcde31d640 4 set_mon_vals no callback set 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcd6ffd640 1 -- 172.21.15.204:0/2787272675 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdcd00050d0 con 0x7fdcc0009c60 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcd6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcd6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.088+0000 7fdcd6ffd640 10 monclient: dump: 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:22.109 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.089+0000 7fdcde31d640 10 monclient: shutdown 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.089+0000 7fdcde31d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.089+0000 7fdcde31d640 1 -- 172.21.15.204:0/2787272675 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdcc0009c60 msgr2=0x7fdcc000c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:22.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.089+0000 7fdcde31d640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdcc0009c60 0x7fdcc000c0e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009f90 tx=0x7fdcd00095d0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.089+0000 7fdcde31d640 1 -- 172.21.15.204:0/2787272675 shutdown_connections 2024-09-15T01:38:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.089+0000 7fdcde31d640 1 --2- 172.21.15.204:0/2787272675 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdcc0009c60 0x7fdcc000c0e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.089+0000 7fdcde31d640 1 -- 172.21.15.204:0/2787272675 >> 172.21.15.204:0/2787272675 conn(0x7fdcd806e860 msgr2=0x7fdcd806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.089+0000 7fdcde31d640 1 -- 172.21.15.204:0/2787272675 shutdown_connections 2024-09-15T01:38:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.089+0000 7fdcde31d640 1 -- 172.21.15.204:0/2787272675 wait complete. 2024-09-15T01:38:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.089+0000 7fdcde31d640 10 monclient: build_initial_monmap 2024-09-15T01:38:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.089+0000 7fdcde31d640 10 monclient: monmap: 2024-09-15T01:38:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:22.111 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.089+0000 7fdcde31d640 1 Processor -- start 2024-09-15T01:38:22.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 1 -- start start 2024-09-15T01:38:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 10 monclient: init 2024-09-15T01:38:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 10 monclient: _add_conns ranks=[6,3,2,8,5,1,4,7,0] 2024-09-15T01:38:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdcc0009c60 0x7fdcd806a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 10 monclient(hunting): picked mon.noname-d con 0x7fdcc0009c60 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcd8137600 0x7fdcd806a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 10 monclient(hunting): picked mon.noname-g con 0x7fdcd8137600 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcd806aed0 0x7fdcd814c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 10 monclient(hunting): picked mon.noname-b con 0x7fdcd806aed0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.113 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd806b2e0 con 0x7fdcd806aed0 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd809cad0 con 0x7fdcd8137600 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd806c830 con 0x7fdcc0009c60 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcde31d640 10 monclient(hunting): authenticate will time out at 1117.732612s 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcd8137600 0x7fdcd806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcdc8ba640 1 -- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcd806aed0 msgr2=0x7fdcd814c140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3301/0 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcdc8ba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcd806aed0 0x7fdcd814c140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcd8137600 0x7fdcd806a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:44458/0 (socket says 172.21.15.204:44458) 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 1 -- 172.21.15.204:0/2898136303 learned_addr learned my addr 172.21.15.204:0/2898136303 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 10 monclient(hunting): get_auth_request con 0x7fdcd8137600 auth_method 0 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd7fff640 1 --2- 172.21.15.204:0/2898136303 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdcc0009c60 0x7fdcd806a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd7fff640 10 monclient(hunting): get_auth_request con 0x7fdcc0009c60 auth_method 0 2024-09-15T01:38:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 10 monclient(hunting): handle_auth_done global_id 15735 payload 824 2024-09-15T01:38:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 1 -- 172.21.15.204:0/2898136303 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdcc0009c60 msgr2=0x7fdcd806a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:22.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 1 --2- 172.21.15.204:0/2898136303 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdcc0009c60 0x7fdcd806a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 1 -- 172.21.15.204:0/2898136303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcd806aed0 msgr2=0x7fdcd814c140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 1 --2- 172.21.15.204:0/2898136303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcd806aed0 0x7fdcd814c140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:38:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 1 monclient: found mon.noname-g 2024-09-15T01:38:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.090+0000 7fdcd77fe640 1 -- 172.21.15.204:0/2898136303 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd809d3d0 con 0x7fdcd8137600 2024-09-15T01:38:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcd77fe640 1 --2- 172.21.15.204:0/2898136303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcd8137600 0x7fdcd806a990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00ea70 tx=0x7fdccc0090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcbffff640 1 -- 172.21.15.204:0/2898136303 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdccc00ce90 con 0x7fdcd8137600 2024-09-15T01:38:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcbffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcbffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:22.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcbffff640 10 monclient: dump: 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:22.117 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcbffff640 10 monclient: _finish_auth 0 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcbffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcbffff640 1 -- 172.21.15.204:0/2898136303 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc013470 con 0x7fdcd8137600 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcde31d640 5 monclient: authenticate success, global_id 15735 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcbffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcde31d640 10 monclient: _renew_subs 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcbffff640 1 -- 172.21.15.204:0/2898136303 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdccc013a60 con 0x7fdcd8137600 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcde31d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcde31d640 1 -- 172.21.15.204:0/2898136303 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd806c4c0 con 0x7fdcd8137600 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcbffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:22.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcbffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcbffff640 10 monclient: dump: 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:22.119 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:22.120 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcde31d640 10 monclient: _renew_subs 2024-09-15T01:38:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcd67fc640 4 set_mon_vals no callback set 2024-09-15T01:38:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcde31d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.091+0000 7fdcde31d640 1 -- 172.21.15.204:0/2898136303 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd806e490 con 0x7fdcd8137600 2024-09-15T01:38:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.092+0000 7fdcde31d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.092+0000 7fdcde31d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.092+0000 7fdcde31d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:22.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.092+0000 7fdcde31d640 1 -- 172.21.15.204:0/2898136303 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd8070b80 con 0x7fdcd8137600 2024-09-15T01:38:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.092+0000 7fdcbffff640 1 -- 172.21.15.204:0/2898136303 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdccc013bc0 con 0x7fdcd8137600 2024-09-15T01:38:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.092+0000 7fdcbffff640 1 --2- 172.21.15.204:0/2898136303 >> v2:172.21.15.204:6812/28210 conn(0x7fdcc40397b0 0x7fdcc403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.092+0000 7fdcbffff640 1 -- 172.21.15.204:0/2898136303 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fdccc04bff0 con 0x7fdcd8137600 2024-09-15T01:38:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.092+0000 7fdcd7fff640 1 --2- 172.21.15.204:0/2898136303 >> v2:172.21.15.204:6812/28210 conn(0x7fdcc40397b0 0x7fdcc403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.092+0000 7fdcd7fff640 10 monclient: get_auth_request con 0x7fdcc40397b0 auth_method 0 2024-09-15T01:38:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.093+0000 7fdcd7fff640 1 --2- 172.21.15.204:0/2898136303 >> v2:172.21.15.204:6812/28210 conn(0x7fdcc40397b0 0x7fdcc403bc60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8000c00 tx=0x7fdcd809ceb0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.096+0000 7fdcbffff640 1 -- 172.21.15.204:0/2898136303 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fdccc07de40 con 0x7fdcd8137600 2024-09-15T01:38:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.096+0000 7fdcbffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:22.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.096+0000 7fdcbffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:22.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.260+0000 7fdcde31d640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["14"]}] 2024-09-15T01:38:22.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.260+0000 7fdcde31d640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["14"]}] 2024-09-15T01:38:22.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.260+0000 7fdcde31d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:22.262 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.260+0000 7fdcde31d640 1 -- 172.21.15.204:0/2898136303 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd rm", "ids": ["14"]} v 0) v1 -- 0x7fdcd809dec0 con 0x7fdcd8137600 2024-09-15T01:38:22.263 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:38:22.261+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e258 definitely_dead 0 2024-09-15T01:38:22.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.548+0000 7fdcbffff640 1 -- 172.21.15.204:0/2898136303 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["14"]}]=0 removed osd.14 v259) v1 ==== 83+0+0 (secure 0 0 0) 0x7fdccc0238f0 con 0x7fdcd8137600 2024-09-15T01:38:22.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.548+0000 7fdcbffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["14"]}] 2024-09-15T01:38:22.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.548+0000 7fdcbffff640 10 monclient: _finish_command 2 = system:0 removed osd.14 2024-09-15T01:38:22.614 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T01:38:22.614 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-15T01:38:22.615 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T01:38:22.615 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T01:38:22.618 INFO:tasks.ceph.mon.c:Started 2024-09-15T01:38:22.618 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-15T01:38:22.618 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-09-15T01:38:22.618 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-15T01:38:22.618 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-15T01:38:22.621 INFO:tasks.ceph.mon.e:Started 2024-09-15T01:38:22.621 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T01:38:22.621 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-15T01:38:22.621 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T01:38:22.621 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T01:38:22.623 INFO:tasks.ceph.mon.a:Started 2024-09-15T01:38:22.624 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T01:38:22.624 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T01:38:22.624 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T01:38:22.624 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T01:38:22.626 INFO:tasks.ceph.mon.b:Started 2024-09-15T01:38:22.626 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:38:22.626 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:38:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.649+0000 7fdcde31d640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["14"]}] 2024-09-15T01:38:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.649+0000 7fdcde31d640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["14"]}] 2024-09-15T01:38:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.649+0000 7fdcde31d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:22.650 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.649+0000 7fdcde31d640 1 -- 172.21.15.204:0/2898136303 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd rm", "ids": ["14"]} v 0) v1 -- 0x7fdcd806b5f0 con 0x7fdcd8137600 2024-09-15T01:38:22.651 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:38:22.649+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e259 definitely_dead 0 2024-09-15T01:38:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.649+0000 7fdcbffff640 1 -- 172.21.15.204:0/2898136303 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["14"]}]=0 osd.14 does not exist. v259) v1 ==== 92+0+0 (secure 0 0 0) 0x7fdccc02a080 con 0x7fdcd8137600 2024-09-15T01:38:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.649+0000 7fdcbffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["14"]}] 2024-09-15T01:38:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.649+0000 7fdcbffff640 10 monclient: _finish_command 3 = system:0 osd.14 does not exist. 2024-09-15T01:38:22.651 INFO:tasks.workunit.client.0.smithi204.stderr:osd.14 does not exist. 2024-09-15T01:38:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 1 -- 172.21.15.204:0/2898136303 >> v2:172.21.15.204:6812/28210 conn(0x7fdcc40397b0 msgr2=0x7fdcc403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 1 --2- 172.21.15.204:0/2898136303 >> v2:172.21.15.204:6812/28210 conn(0x7fdcc40397b0 0x7fdcc403bc60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8000c00 tx=0x7fdcd809ceb0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 10 monclient: shutdown 2024-09-15T01:38:22.653 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 1 -- 172.21.15.204:0/2898136303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcd8137600 msgr2=0x7fdcd806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 1 --2- 172.21.15.204:0/2898136303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcd8137600 0x7fdcd806a990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00ea70 tx=0x7fdccc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 1 -- 172.21.15.204:0/2898136303 shutdown_connections 2024-09-15T01:38:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 1 --2- 172.21.15.204:0/2898136303 >> v2:172.21.15.204:6812/28210 conn(0x7fdcc40397b0 0x7fdcc403bc60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 1 --2- 172.21.15.204:0/2898136303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcd806aed0 0x7fdcd814c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 1 --2- 172.21.15.204:0/2898136303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcd8137600 0x7fdcd806a990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 1 --2- 172.21.15.204:0/2898136303 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdcc0009c60 0x7fdcd806a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 1 -- 172.21.15.204:0/2898136303 >> 172.21.15.204:0/2898136303 conn(0x7fdcd806e860 msgr2=0x7fdcd80a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 1 -- 172.21.15.204:0/2898136303 shutdown_connections 2024-09-15T01:38:22.654 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.651+0000 7fdcde31d640 1 -- 172.21.15.204:0/2898136303 wait complete. 2024-09-15T01:38:22.661 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph osd rm 1000 2024-09-15T01:38:22.726 INFO:tasks.ceph.mon.b.smithi178.stderr:2024-09-15T01:38:22.724+0000 7fecb20ea900 -1 mon.b@-1(???).log v0 log_external_backlog rewinding external_log_to from 214 -> 0 (sync_force? mon rebuild?) 2024-09-15T01:38:22.727 INFO:tasks.ceph.mon.a.smithi178.stderr:2024-09-15T01:38:22.725+0000 7f8d3840e900 -1 mon.a@-1(???).log v0 log_external_backlog rewinding external_log_to from 214 -> 0 (sync_force? mon rebuild?) 2024-09-15T01:38:22.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.733+0000 7fa676d5d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:22.734 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.733+0000 7fa676d5d640 10 monclient: build_initial_monmap 2024-09-15T01:38:22.735 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.733+0000 7fa676d5d640 10 monclient: monmap: 2024-09-15T01:38:22.735 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:22.735 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:22.735 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:22.733902+0000 2024-09-15T01:38:22.735 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:22.733902+0000 2024-09-15T01:38:22.735 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:22.735 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:22.735 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:22.735 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:22.735 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:22.735 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:22.736 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:22.736 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:22.736 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:22.736 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:22.736 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:22.736 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:22.736 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:22.736 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 1 Processor -- start 2024-09-15T01:38:22.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 1 -- start start 2024-09-15T01:38:22.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 10 monclient: init 2024-09-15T01:38:22.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:22.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 10 monclient: _add_conns ranks=[5,10,1,0,9,3,6,8,4,2,7] 2024-09-15T01:38:22.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa670067130 0x7fa670097c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 10 monclient(hunting): picked mon.noname-h con 0x7fa670067130 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:22.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa670098150 0x7fa670067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 10 monclient(hunting): picked mon.noname-f con 0x7fa670098150 addr v2:172.21.15.204:3300/0 2024-09-15T01:38:22.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa670067a40 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:22.745 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6700648a0 con 0x7fa670067a40 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa670097510 con 0x7fa670098150 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa670096f90 con 0x7fa670067130 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa676d5d640 10 monclient(hunting): authenticate will time out at 1118.376843s 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa674af9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa670067130 0x7fa670097c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa674af9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa670067130 0x7fa670097c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:52474/0 (socket says 172.21.15.204:52474) 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa66ffff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa670098150 0x7fa670067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa674af9640 1 -- 172.21.15.204:0/2351029835 learned_addr learned my addr 172.21.15.204:0/2351029835 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa674af9640 10 monclient(hunting): get_auth_request con 0x7fa670067130 auth_method 0 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa674af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.746 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa674af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa674af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa66ffff640 10 monclient(hunting): get_auth_request con 0x7fa670098150 auth_method 0 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa66ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa66ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.734+0000 7fa66ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa66ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa66ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa66ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 10 monclient(hunting): handle_auth_done global_id 6629 payload 274 2024-09-15T01:38:22.747 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 1 -- 172.21.15.204:0/2351029835 >> v2:172.21.15.204:3300/0 conn(0x7fa670098150 msgr2=0x7fa670067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 1 --2- 172.21.15.204:0/2351029835 >> v2:172.21.15.204:3300/0 conn(0x7fa670098150 0x7fa670067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 1 -- 172.21.15.204:0/2351029835 >> v1:172.21.15.204:6789/0 conn(0x7fa670067a40 legacy=0x7fa670138e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 10 monclient: _finish_hunting 0 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 1 monclient: found mon.noname-h 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 1 -- 172.21.15.204:0/2351029835 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa670097230 con 0x7fa670067130 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa66ffff640 1 --2- 172.21.15.204:0/2351029835 >> v2:172.21.15.204:3300/0 conn(0x7fa670098150 0x7fa670067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.735+0000 7fa674af9640 1 --2- 172.21.15.204:0/2351029835 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa670067130 0x7fa670097c10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa664009910 tx=0x7fa6640091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=4f8f7bbb6bcc0ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa66f7fe640 1 -- 172.21.15.204:0/2351029835 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa66402ad20 con 0x7fa670067130 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa66f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa66f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:38:22.748 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa66f7fe640 10 monclient: dump: 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:22.749 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:22.750 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:22.750 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:22.750 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa66f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:22.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa66f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:22.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa66f7fe640 1 -- 172.21.15.204:0/2351029835 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa664031820 con 0x7fa670067130 2024-09-15T01:38:22.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa676d5d640 5 monclient: authenticate success, global_id 6629 2024-09-15T01:38:22.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa676d5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:22.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa66f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:22.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa66f7fe640 1 -- 172.21.15.204:0/2351029835 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa66402aea0 con 0x7fa670067130 2024-09-15T01:38:22.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa676d5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:22.750 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa676d5d640 4 set_mon_vals no callback set 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa66f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa66f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.736+0000 7fa66f7fe640 10 monclient: dump: 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:22.751 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.737+0000 7fa676d5d640 10 monclient: shutdown 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.737+0000 7fa676d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.737+0000 7fa676d5d640 1 -- 172.21.15.204:0/2351029835 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa670067130 msgr2=0x7fa670097c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.737+0000 7fa676d5d640 1 --2- 172.21.15.204:0/2351029835 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa670067130 0x7fa670097c10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa664009910 tx=0x7fa6640091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.737+0000 7fa676d5d640 1 -- 172.21.15.204:0/2351029835 shutdown_connections 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.737+0000 7fa676d5d640 1 --2- 172.21.15.204:0/2351029835 >> v2:172.21.15.204:3300/0 conn(0x7fa670098150 0x7fa670067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.752 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.737+0000 7fa676d5d640 1 --2- 172.21.15.204:0/2351029835 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa670067130 0x7fa670097c10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.737+0000 7fa676d5d640 1 -- 172.21.15.204:0/2351029835 >> 172.21.15.204:0/2351029835 conn(0x7fa6700a2c80 msgr2=0x7fa6700a0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.737+0000 7fa676d5d640 1 -- 172.21.15.204:0/2351029835 shutdown_connections 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.737+0000 7fa676d5d640 1 -- 172.21.15.204:0/2351029835 wait complete. 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 10 monclient: build_initial_monmap 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 10 monclient: monmap: 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:22.753 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 1 Processor -- start 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 1 -- start start 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 10 monclient: init 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:22.754 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 10 monclient: _add_conns ranks=[2,8,6,3,7,4,5,0,1] 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa670067130 0x7fa670133d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 10 monclient(hunting): picked mon.noname-b con 0x7fa670067130 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa670067a40 0x7fa67011f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 10 monclient(hunting): picked mon.noname-e con 0x7fa670067a40 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa670098150 0x7fa67011ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 10 monclient(hunting): picked mon.noname-d con 0x7fa670098150 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa670122c50 con 0x7fa670067130 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa6701231b0 con 0x7fa670098150 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.738+0000 7fa676d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa676d5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa670121f70 con 0x7fa670067a40 2024-09-15T01:38:22.755 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa676d5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa676d5d640 10 monclient(hunting): authenticate will time out at 1118.381014s 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa66ffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa670067a40 0x7fa67011f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa670098150 0x7fa67011ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa66ffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa670067a40 0x7fa67011f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.204:36302/0 (socket says 172.21.15.204:36302) 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 10 monclient(hunting): get_auth_request con 0x7fa670098150 auth_method 0 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa66ffff640 1 -- 172.21.15.204:0/927023322 learned_addr learned my addr 172.21.15.204:0/927023322 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa66ffff640 10 monclient(hunting): get_auth_request con 0x7fa670067a40 auth_method 0 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa66ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa66ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.756 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa66ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa674af9640 1 --2- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa670067130 0x7fa670133d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa674af9640 10 monclient(hunting): get_auth_request con 0x7fa670067130 auth_method 0 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa674af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa674af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa674af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa66ffff640 1 -- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa670067a40 msgr2=0x7fa67011f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa66ffff640 1 -- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa670067a40 msgr2=0x7fa67011f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa674af9640 1 -- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa670067130 msgr2=0x7fa670133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa66ffff640 1 --2- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa670067a40 0x7fa67011f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:22.757 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa674af9640 1 -- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa670067130 msgr2=0x7fa670133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa674af9640 1 --2- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa670067130 0x7fa670133d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa66ffff640 1 --2- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa670067a40 0x7fa67011f9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa674af9640 1 --2- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa670067130 0x7fa670133d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 10 monclient(hunting): handle_auth_done global_id 24675 payload 824 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 1 -- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa670067a40 msgr2=0x7fa67011f9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 1 --2- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa670067a40 0x7fa67011f9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 1 -- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa670067130 msgr2=0x7fa670133d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 1 --2- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa670067130 0x7fa670133d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 10 monclient: _finish_hunting 0 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 1 monclient: found mon.noname-d 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:38:22.758 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 1 -- 172.21.15.204:0/927023322 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c009390 con 0x7fa670098150 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.739+0000 7fa6752fa640 1 --2- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa670098150 0x7fa67011ff20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa66800c790 tx=0x7fa668001610 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66cff9640 1 -- 172.21.15.204:0/927023322 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa66800aac0 con 0x7fa670098150 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66cff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66cff9640 10 monclient: dump: 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:22.759 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66cff9640 10 monclient: _finish_auth 0 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa676d5d640 5 monclient: authenticate success, global_id 24675 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66cff9640 1 -- 172.21.15.204:0/927023322 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66800ac60 con 0x7fa670098150 2024-09-15T01:38:22.760 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66cff9640 1 -- 172.21.15.204:0/927023322 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa66800add0 con 0x7fa670098150 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa676d5d640 10 monclient: _renew_subs 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa676d5d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa676d5d640 1 -- 172.21.15.204:0/927023322 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa664009450 con 0x7fa670098150 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66effd640 4 set_mon_vals no callback set 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa66cff9640 10 monclient: dump: 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:22.761 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa676d5d640 10 monclient: _renew_subs 2024-09-15T01:38:22.762 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa676d5d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:22.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.740+0000 7fa676d5d640 1 -- 172.21.15.204:0/927023322 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa670123490 con 0x7fa670098150 2024-09-15T01:38:22.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.741+0000 7fa66cff9640 1 -- 172.21.15.204:0/927023322 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa66800a640 con 0x7fa670098150 2024-09-15T01:38:22.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.741+0000 7fa676d5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:22.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.741+0000 7fa676d5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:22.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.741+0000 7fa676d5d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:22.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.741+0000 7fa676d5d640 1 -- 172.21.15.204:0/927023322 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa670070b80 con 0x7fa670098150 2024-09-15T01:38:22.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.741+0000 7fa66cff9640 1 --2- 172.21.15.204:0/927023322 >> v2:172.21.15.204:6812/28210 conn(0x7fa6540397b0 0x7fa65403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.741+0000 7fa66cff9640 1 -- 172.21.15.204:0/927023322 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fa66804bda0 con 0x7fa670098150 2024-09-15T01:38:22.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.741+0000 7fa674af9640 1 --2- 172.21.15.204:0/927023322 >> v2:172.21.15.204:6812/28210 conn(0x7fa6540397b0 0x7fa65403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.741+0000 7fa674af9640 10 monclient: get_auth_request con 0x7fa6540397b0 auth_method 0 2024-09-15T01:38:22.763 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.742+0000 7fa674af9640 1 --2- 172.21.15.204:0/927023322 >> v2:172.21.15.204:6812/28210 conn(0x7fa6540397b0 0x7fa65403bc60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa664034000 tx=0x7fa664032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:22.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.750+0000 7fa66cff9640 1 -- 172.21.15.204:0/927023322 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa66800a8f0 con 0x7fa670098150 2024-09-15T01:38:22.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.750+0000 7fa66cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:22.764 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.750+0000 7fa66cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:22.764 INFO:tasks.ceph.mon.c.smithi178.stderr:2024-09-15T01:38:22.747+0000 7f985accd900 -1 mon.c@-1(???).log v0 log_external_backlog rewinding external_log_to from 214 -> 0 (sync_force? mon rebuild?) 2024-09-15T01:38:22.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.765+0000 7f68dcd9d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.765+0000 7f68dcd9d640 10 monclient: build_initial_monmap 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.765+0000 7f68dcd9d640 10 monclient: monmap: 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:22.766077+0000 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:22.766077+0000 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:22.767 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:22.768 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:22.768 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:22.768 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:22.768 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:22.768 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:22.768 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:22.768 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:22.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 1 Processor -- start 2024-09-15T01:38:22.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 1 -- start start 2024-09-15T01:38:22.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 10 monclient: init 2024-09-15T01:38:22.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:22.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 10 monclient: _add_conns ranks=[8,9,2,7,6,1,10,4,5,0,3] 2024-09-15T01:38:22.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 10 monclient(hunting): picked mon.noname-e con 0x7f68d80987b0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:22.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 10 monclient(hunting): picked mon.noname-a con 0x7f68d812ef10 addr v2:172.21.15.178:3300/0 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 10 monclient(hunting): picked mon.noname-b con 0x7f68d812f880 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d8096e00 con 0x7f68d812ef10 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d8097d50 con 0x7f68d812f880 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f68d80990b0 con 0x7f68d80987b0 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68dcd9d640 10 monclient(hunting): authenticate will time out at 1107.516725s 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68d659c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68d659c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:58090/0 (socket says 172.21.15.178:58090) 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68cffff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 learned_addr learned my addr 172.21.15.178:0/1568452792 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68d659c640 10 monclient(hunting): get_auth_request con 0x7f68d80987b0 auth_method 0 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68d659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68d659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68cffff640 10 monclient(hunting): get_auth_request con 0x7f68d812ef10 auth_method 0 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request con 0x7f68d812f880 auth_method 0 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68d6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.766+0000 7f68d6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:22.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.767+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 msgr2=0x7f68d809ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:38:22.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.767+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 msgr2=0x7f68d809ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:22.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.767+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:22.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.767+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 msgr2=0x7f68d8063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:38:22.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.767+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 msgr2=0x7f68d8063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:22.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.767+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:22.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.767+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:22.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.767+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:22.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.767+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 msgr2=0x7f68d812f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:22.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.767+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 msgr2=0x7f68d812f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:22.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.767+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:22.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.767+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:22.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.920+0000 7fa676d5d640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["1000"]}] 2024-09-15T01:38:22.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.920+0000 7fa676d5d640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-09-15T01:38:22.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.920+0000 7fa676d5d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:22.922 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.920+0000 7fa676d5d640 1 -- 172.21.15.204:0/927023322 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd rm", "ids": ["1000"]} v 0) v1 -- 0x7fa670067e80 con 0x7fa670098150 2024-09-15T01:38:22.923 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:38:22.921+0000 7fbcdddf1640 -1 mon.f@1(leader).osd e259 definitely_dead 0 2024-09-15T01:38:22.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.922+0000 7fa66cff9640 1 -- 172.21.15.204:0/927023322 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["1000"]}]=0 osd.1000 does not exist. v259) v1 ==== 96+0+0 (secure 0 0 0) 0x7fa6680102e0 con 0x7fa670098150 2024-09-15T01:38:22.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.922+0000 7fa66cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-09-15T01:38:22.923 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:22.922+0000 7fa66cff9640 10 monclient: _finish_command 2 = system:0 osd.1000 does not exist. 2024-09-15T01:38:22.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d659c640 10 monclient(hunting): get_auth_request con 0x7f68d80987b0 auth_method 0 2024-09-15T01:38:22.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:22.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:22.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request con 0x7f68d812f880 auth_method 0 2024-09-15T01:38:22.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d6d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:22.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68cffff640 10 monclient(hunting): get_auth_request con 0x7f68d812ef10 auth_method 0 2024-09-15T01:38:22.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68cffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 msgr2=0x7f68d8063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 msgr2=0x7f68d8063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 msgr2=0x7f68d812f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 msgr2=0x7f68d812f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 msgr2=0x7f68d809ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 msgr2=0x7f68d809ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:22.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:22.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:22.968+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:23.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:23.030+0000 7fa676d5d640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["1000"]}] 2024-09-15T01:38:23.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:23.030+0000 7fa676d5d640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-09-15T01:38:23.031 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:23.030+0000 7fa676d5d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:23.032 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:23.030+0000 7fa676d5d640 1 -- 172.21.15.204:0/927023322 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "osd rm", "ids": ["1000"]} v 0) v1 -- 0x7fa6700a0ed0 con 0x7fa670098150 2024-09-15T01:38:23.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68cffff640 10 monclient(hunting): get_auth_request con 0x7f68d812ef10 auth_method 0 2024-09-15T01:38:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68cffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request con 0x7f68d812f880 auth_method 0 2024-09-15T01:38:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d6d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d659c640 10 monclient(hunting): get_auth_request con 0x7f68d80987b0 auth_method 0 2024-09-15T01:38:23.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 msgr2=0x7f68d812f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 msgr2=0x7f68d812f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 msgr2=0x7f68d8063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 msgr2=0x7f68d8063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 msgr2=0x7f68d809ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 msgr2=0x7f68d809ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:23.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:23.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:23.369+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:24.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:24.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:24.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d659c640 10 monclient(hunting): get_auth_request con 0x7f68d80987b0 auth_method 0 2024-09-15T01:38:24.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:24.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:24.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:24.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68cffff640 10 monclient(hunting): get_auth_request con 0x7f68d812ef10 auth_method 0 2024-09-15T01:38:24.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:24.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:24.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68cffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:24.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request con 0x7f68d812f880 auth_method 0 2024-09-15T01:38:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d6d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 msgr2=0x7f68d812f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 msgr2=0x7f68d812f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 msgr2=0x7f68d8063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:38:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 msgr2=0x7f68d809ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:38:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 msgr2=0x7f68d8063620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:24.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 msgr2=0x7f68d809ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:24.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:24.170+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:25.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:25.739+0000 7fa65b7fe640 10 monclient: tick 2024-09-15T01:38:25.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:25.739+0000 7fa65b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:25.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:25.739+0000 7fa65b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:25.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 10 monclient(hunting): tick 2024-09-15T01:38:25.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 1 monclient(hunting): continuing hunt 2024-09-15T01:38:25.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:38:25.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 msgr2=0x7f68d809ccd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:25.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f68d80987b0 0x7f68d809ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:25.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 msgr2=0x7f68d8063620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:25.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68d812f880 0x7f68d8063620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:25.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 1 -- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 msgr2=0x7f68d812f340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:25.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 1 --2- 172.21.15.178:0/1568452792 >> v2:172.21.15.178:3300/0 conn(0x7f68d812ef10 0x7f68d812f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:25.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 10 monclient: _add_conns ranks=[7,5,3,0,6,4,10,1] 2024-09-15T01:38:25.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:25.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 10 monclient(hunting): picked mon.noname-i con 0x7f68c40037e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:25.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:25.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 10 monclient(hunting): picked mon.noname-h con 0x7f68c40061e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:25.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:25.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 10 monclient(hunting): picked mon.noname-g con 0x7f68c4008d50 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:25.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.766+0000 7f68d4d99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:25.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d4d99640 1 -- 172.21.15.178:0/1568452792 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d8096e00 con 0x7f68c4008d50 2024-09-15T01:38:25.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d4d99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:25.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d4d99640 1 -- 172.21.15.178:0/1568452792 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68d8097d50 con 0x7f68c40061e0 2024-09-15T01:38:25.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d4d99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:25.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d4d99640 1 -- 172.21.15.178:0/1568452792 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f68d80990b0 con 0x7f68c40037e0 2024-09-15T01:38:25.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d4d99640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:25.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:25.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:25.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68cffff640 10 monclient(hunting): get_auth_request con 0x7f68c40061e0 auth_method 0 2024-09-15T01:38:25.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:25.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:25.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:25.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d659c640 10 monclient(hunting): get_auth_request con 0x7f68c40037e0 auth_method 0 2024-09-15T01:38:25.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:25.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:25.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:25.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:25.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request con 0x7f68c4008d50 auth_method 0 2024-09-15T01:38:25.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:25.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:25.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:25.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 msgr2=0x7f68c4008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:25.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 msgr2=0x7f68c4008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:25.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:25.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 msgr2=0x7f68c4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:38:25.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 msgr2=0x7f68c4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:25.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:25.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:25.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:25.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 msgr2=0x7f68c400b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:38:25.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 msgr2=0x7f68c400b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:25.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:25.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.767+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:25.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68cffff640 10 monclient(hunting): get_auth_request con 0x7f68c40061e0 auth_method 0 2024-09-15T01:38:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68cffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request con 0x7f68c4008d50 auth_method 0 2024-09-15T01:38:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d6d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d659c640 10 monclient(hunting): get_auth_request con 0x7f68c40037e0 auth_method 0 2024-09-15T01:38:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 msgr2=0x7f68c4008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 msgr2=0x7f68c4008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 msgr2=0x7f68c400b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:38:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 msgr2=0x7f68c400b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 msgr2=0x7f68c4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:38:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 msgr2=0x7f68c4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:25.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:25.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:25.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:25.968+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:26.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:26.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:26.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:26.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68cffff640 10 monclient(hunting): get_auth_request con 0x7f68c40061e0 auth_method 0 2024-09-15T01:38:26.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:26.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:26.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68cffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:26.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request con 0x7f68c4008d50 auth_method 0 2024-09-15T01:38:26.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:26.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:26.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d6d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:26.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d659c640 10 monclient(hunting): get_auth_request con 0x7f68c40037e0 auth_method 0 2024-09-15T01:38:26.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:26.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:26.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:26.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 msgr2=0x7f68c4008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:38:26.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 msgr2=0x7f68c4008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:26.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 msgr2=0x7f68c400b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:26.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 msgr2=0x7f68c400b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:26.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:26.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:26.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:26.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:26.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 msgr2=0x7f68c4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:38:26.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 msgr2=0x7f68c4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:26.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:26.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:26.369+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:26.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:26.739+0000 7fa65b7fe640 10 monclient: tick 2024-09-15T01:38:26.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:26.739+0000 7fa65b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:27.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:27.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68cffff640 10 monclient(hunting): get_auth_request con 0x7f68c40061e0 auth_method 0 2024-09-15T01:38:27.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:27.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:27.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68cffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:27.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 msgr2=0x7f68c4008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:38:27.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68cffff640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 msgr2=0x7f68c4008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:27.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:27.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68cffff640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:27.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:27.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request con 0x7f68c4008d50 auth_method 0 2024-09-15T01:38:27.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:27.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:27.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d6d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:27.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 msgr2=0x7f68c400b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:27.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 msgr2=0x7f68c400b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:27.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:27.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:27.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:27.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d659c640 10 monclient(hunting): get_auth_request con 0x7f68c40037e0 auth_method 0 2024-09-15T01:38:27.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:27.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:27.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.170+0000 7f68d659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:27.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.171+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 msgr2=0x7f68c4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:38:27.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.171+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 msgr2=0x7f68c4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:27.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.171+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:27.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:27.171+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:38:27.740 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:27.739+0000 7fa65b7fe640 10 monclient: tick 2024-09-15T01:38:27.741 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:27.739+0000 7fa65b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:28.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:27.999+0000 7fa9c23f1640 -1 mon.h@5(leader).osd e259 definitely_dead 0 2024-09-15T01:38:28.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:27.999+0000 7fa66cff9640 1 -- 172.21.15.204:0/927023322 <== mon.6 v2:172.21.15.178:3303/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["1000"]}]=0 osd.1000 does not exist. v259) v1 ==== 96+0+0 (secure 0 0 0) 0x7fa668029080 con 0x7fa670098150 2024-09-15T01:38:28.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:27.999+0000 7fa66cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-09-15T01:38:28.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:27.999+0000 7fa66cff9640 10 monclient: _finish_command 3 = system:0 osd.1000 does not exist. 2024-09-15T01:38:28.001 INFO:tasks.workunit.client.0.smithi204.stderr:osd.1000 does not exist. 2024-09-15T01:38:28.004 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 1 -- 172.21.15.204:0/927023322 >> v2:172.21.15.204:6812/28210 conn(0x7fa6540397b0 msgr2=0x7fa65403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:28.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 1 --2- 172.21.15.204:0/927023322 >> v2:172.21.15.204:6812/28210 conn(0x7fa6540397b0 0x7fa65403bc60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa664034000 tx=0x7fa664032000 comp rx=0 tx=0).stop 2024-09-15T01:38:28.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 10 monclient: shutdown 2024-09-15T01:38:28.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:28.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 1 -- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa670098150 msgr2=0x7fa67011ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:28.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 1 --2- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa670098150 0x7fa67011ff20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa66800c790 tx=0x7fa668001610 comp rx=0 tx=0).stop 2024-09-15T01:38:28.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 1 -- 172.21.15.204:0/927023322 shutdown_connections 2024-09-15T01:38:28.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 1 --2- 172.21.15.204:0/927023322 >> v2:172.21.15.204:6812/28210 conn(0x7fa6540397b0 0x7fa65403bc60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 1 --2- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa670098150 0x7fa67011ff20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 1 --2- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa670067a40 0x7fa67011f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 1 --2- 172.21.15.204:0/927023322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa670067130 0x7fa670133d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.005 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 1 -- 172.21.15.204:0/927023322 >> 172.21.15.204:0/927023322 conn(0x7fa6700a2c80 msgr2=0x7fa67009ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:28.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 1 -- 172.21.15.204:0/927023322 shutdown_connections 2024-09-15T01:38:28.006 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.003+0000 7fa676d5d640 1 -- 172.21.15.204:0/927023322 wait complete. 2024-09-15T01:38:28.016 INFO:tasks.workunit.client.0.smithi204.stderr:++ ceph osd create e5de064c-679b-416c-b9b7-f61aa58c84ce 2024-09-15T01:38:28.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.095+0000 7f7494700640 10 monclient: get_monmap_and_config 2024-09-15T01:38:28.096 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.095+0000 7f7494700640 10 monclient: build_initial_monmap 2024-09-15T01:38:28.097 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.095+0000 7f7494700640 10 monclient: monmap: 2024-09-15T01:38:28.097 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:28.097 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:28.097 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:28.095607+0000 2024-09-15T01:38:28.097 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:28.095607+0000 2024-09-15T01:38:28.097 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:28.097 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:28.097 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:28.097 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:28.097 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:28.097 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:28.097 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:28.098 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:28.098 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:28.098 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:28.098 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:28.098 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:28.098 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:28.098 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:28.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.095+0000 7f7494700640 1 Processor -- start 2024-09-15T01:38:28.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.095+0000 7f7494700640 1 -- start start 2024-09-15T01:38:28.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.095+0000 7f7494700640 10 monclient: init 2024-09-15T01:38:28.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:28.106 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 10 monclient: _add_conns ranks=[7,3,1,8,0,4,2,5,9,6,10] 2024-09-15T01:38:28.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f748c0a2400 0x7f748c0a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:28.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 10 monclient(hunting): picked mon.noname-i con 0x7f748c0a2400 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:28.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c09f990 0x7f748c09fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:28.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 10 monclient(hunting): picked mon.noname-g con 0x7f748c09f990 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:28.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f748c132930 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:28.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c0648a0 con 0x7f748c132930 2024-09-15T01:38:28.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:28.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c09e1a0 con 0x7f748c09f990 2024-09-15T01:38:28.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:28.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f748c0a27d0 con 0x7f748c0a2400 2024-09-15T01:38:28.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:28.107 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7494700640 10 monclient(hunting): authenticate will time out at 1123.738474s 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f749249c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f748c0a2400 0x7f748c0a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c09f990 0x7f748c09fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7492c9d640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f748c132930 0x7f748c132d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:40374/0 (socket says 172.21.15.204:40374) 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7492c9d640 1 -- 172.21.15.204:0/563220147 learned_addr learned my addr 172.21.15.204:0/563220147 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f749249c640 10 monclient(hunting): get_auth_request con 0x7f748c0a2400 auth_method 0 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f749249c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f749249c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f749249c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 10 monclient(hunting): get_auth_request con 0x7f748c09f990 auth_method 0 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:28.108 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f749249c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:28.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f749249c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:28.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f749249c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:28.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:28.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:28.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:28.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 10 monclient(hunting): handle_auth_done global_id 15744 payload 274 2024-09-15T01:38:28.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 1 -- 172.21.15.204:0/563220147 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f748c0a2400 msgr2=0x7f748c0a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:28.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f749149a640 1 -- 172.21.15.204:0/563220147 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548497125 0 0) 0x7f748c0648a0 con 0x7f748c132930 2024-09-15T01:38:28.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 1 --2- 172.21.15.204:0/563220147 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f748c0a2400 0x7f748c0a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 1 -- 172.21.15.204:0/563220147 >> v1:172.21.15.204:6789/0 conn(0x7f748c132930 legacy=0x7f748c132d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:28.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 10 monclient: _finish_hunting 0 2024-09-15T01:38:28.109 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 1 monclient: found mon.noname-g 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f7491c9b640 1 -- 172.21.15.204:0/563220147 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c09dec0 con 0x7f748c09f990 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f749149a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.096+0000 7f749249c640 1 --2- 172.21.15.204:0/563220147 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f748c0a2400 0x7f748c0a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f7491c9b640 1 --2- 172.21.15.204:0/563220147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c09f990 0x7f748c09fd60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7480009880 tx=0x7f74800090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=9fd6283d4a8189c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f749149a640 1 -- 172.21.15.204:0/563220147 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f748002ac50 con 0x7f748c09f990 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f749149a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f749149a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f749149a640 10 monclient: dump: 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:28.110 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:28.111 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f749149a640 10 monclient: _finish_auth 0 2024-09-15T01:38:28.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f749149a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:28.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f7494700640 5 monclient: authenticate success, global_id 15744 2024-09-15T01:38:28.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f7494700640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:28.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f749149a640 1 -- 172.21.15.204:0/563220147 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748002add0 con 0x7f748c09f990 2024-09-15T01:38:28.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f749149a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:28.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f7494700640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:28.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f7494700640 4 set_mon_vals no callback set 2024-09-15T01:38:28.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f749149a640 1 -- 172.21.15.204:0/563220147 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7480031ca0 con 0x7f748c09f990 2024-09-15T01:38:28.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f749149a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:28.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f749149a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:28.112 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.097+0000 7f749149a640 10 monclient: dump: 2024-09-15T01:38:28.112 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:28.113 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:28.114 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:28.114 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:28.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.098+0000 7f7494700640 10 monclient: shutdown 2024-09-15T01:38:28.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.098+0000 7f7494700640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:28.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.098+0000 7f7494700640 1 -- 172.21.15.204:0/563220147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c09f990 msgr2=0x7f748c09fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:28.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.098+0000 7f7494700640 1 --2- 172.21.15.204:0/563220147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c09f990 0x7f748c09fd60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7480009880 tx=0x7f74800090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.098+0000 7f7494700640 1 -- 172.21.15.204:0/563220147 shutdown_connections 2024-09-15T01:38:28.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.098+0000 7f7494700640 1 --2- 172.21.15.204:0/563220147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c09f990 0x7f748c09fd60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.098+0000 7f7494700640 1 --2- 172.21.15.204:0/563220147 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f748c0a2400 0x7f748c0a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.098+0000 7f7494700640 1 -- 172.21.15.204:0/563220147 >> 172.21.15.204:0/563220147 conn(0x7f748c06e860 msgr2=0x7f748c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:28.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.098+0000 7f7494700640 1 -- 172.21.15.204:0/563220147 shutdown_connections 2024-09-15T01:38:28.114 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.098+0000 7f7494700640 1 -- 172.21.15.204:0/563220147 wait complete. 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.098+0000 7f7494700640 10 monclient: build_initial_monmap 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.098+0000 7f7494700640 10 monclient: monmap: 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:28.115 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 1 Processor -- start 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 1 -- start start 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 10 monclient: init 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 10 monclient: _add_conns ranks=[2,0,3,8,7,6,4,1,5] 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f748c09f990 0x7f748c11d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 10 monclient(hunting): picked mon.noname-b con 0x7f748c09f990 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:28.116 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f748c0a2400 0x7f748c11d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 10 monclient(hunting): picked mon.noname-a con 0x7f748c0a2400 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c132930 0x7f748c11dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 10 monclient(hunting): picked mon.noname-g con 0x7f748c132930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c12fc30 con 0x7f748c0a2400 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c120840 con 0x7f748c09f990 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c09d3d0 con 0x7f748c132930 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7494700640 10 monclient(hunting): authenticate will time out at 1123.741701s 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7492c9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c132930 0x7f748c11dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:28.117 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7492c9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c132930 0x7f748c11dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:49458/0 (socket says 172.21.15.204:49458) 2024-09-15T01:38:28.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7492c9d640 1 -- 172.21.15.204:0/1900659740 learned_addr learned my addr 172.21.15.204:0/1900659740 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:28.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7492c9d640 10 monclient(hunting): get_auth_request con 0x7f748c132930 auth_method 0 2024-09-15T01:38:28.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7492c9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:28.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7492c9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:28.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7492c9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:28.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7492c9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:28.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7492c9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:28.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7492c9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:28.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f749249c640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f748c09f990 0x7f748c11d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:28.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.099+0000 7f7491c9b640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f748c0a2400 0x7f748c11d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:28.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7491c9b640 10 monclient(hunting): get_auth_request con 0x7f748c0a2400 auth_method 0 2024-09-15T01:38:28.118 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7491c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7491c9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7491c9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f749249c640 10 monclient(hunting): get_auth_request con 0x7f748c09f990 auth_method 0 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f749249c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f749249c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f749249c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7492c9d640 10 monclient(hunting): handle_auth_done global_id 15753 payload 824 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f749249c640 1 -- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f748c09f990 msgr2=0x7f748c11d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7491c9b640 1 -- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f748c0a2400 msgr2=0x7f748c11d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f749249c640 1 -- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f748c09f990 msgr2=0x7f748c11d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7491c9b640 1 -- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f748c0a2400 msgr2=0x7f748c11d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7492c9d640 1 -- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f748c09f990 msgr2=0x7f748c11d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:28.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f749249c640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f748c09f990 0x7f748c11d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:28.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7491c9b640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f748c0a2400 0x7f748c11d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:28.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f749249c640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f748c09f990 0x7f748c11d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:28.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7491c9b640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f748c0a2400 0x7f748c11d790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:28.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7492c9d640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f748c09f990 0x7f748c11d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7492c9d640 1 -- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f748c0a2400 msgr2=0x7f748c11d790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:28.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7492c9d640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f748c0a2400 0x7f748c11d790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7492c9d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:28.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7492c9d640 1 monclient: found mon.noname-g 2024-09-15T01:38:28.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7492c9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:28.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7492c9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:28.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7492c9d640 1 -- 172.21.15.204:0/1900659740 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484009390 con 0x7f748c132930 2024-09-15T01:38:28.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7492c9d640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c132930 0x7f748c11dcd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f747c00cc10 tx=0x7f747c001970 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f748a7fc640 1 -- 172.21.15.204:0/1900659740 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f747c00ce30 con 0x7f748c132930 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f748a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f748a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f748a7fc640 10 monclient: dump: 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:28.121 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f748a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f748a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7494700640 5 monclient: authenticate success, global_id 15753 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f748a7fc640 1 -- 172.21.15.204:0/1900659740 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c00ac70 con 0x7f748c132930 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f748a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f748a7fc640 1 -- 172.21.15.204:0/1900659740 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f747c00ae20 con 0x7f748c132930 2024-09-15T01:38:28.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f748a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f748a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f748a7fc640 10 monclient: dump: 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:28.123 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7490c99640 4 set_mon_vals no callback set 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7494700640 10 monclient: _renew_subs 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7494700640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7494700640 1 -- 172.21.15.204:0/1900659740 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7480009390 con 0x7f748c132930 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7494700640 10 monclient: _renew_subs 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7494700640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.100+0000 7f7494700640 1 -- 172.21.15.204:0/1900659740 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c09cad0 con 0x7f748c132930 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.101+0000 7f748a7fc640 1 -- 172.21.15.204:0/1900659740 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f747c00a7f0 con 0x7f748c132930 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.101+0000 7f7494700640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.101+0000 7f7494700640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:28.124 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.101+0000 7f7494700640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:28.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.101+0000 7f7494700640 1 -- 172.21.15.204:0/1900659740 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c070b80 con 0x7f748c132930 2024-09-15T01:38:28.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.101+0000 7f748a7fc640 1 --2- 172.21.15.204:0/1900659740 >> v2:172.21.15.204:6812/28210 conn(0x7f74780397b0 0x7f747803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:28.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.101+0000 7f748a7fc640 1 -- 172.21.15.204:0/1900659740 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f747c04bd80 con 0x7f748c132930 2024-09-15T01:38:28.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.101+0000 7f749249c640 1 --2- 172.21.15.204:0/1900659740 >> v2:172.21.15.204:6812/28210 conn(0x7f74780397b0 0x7f747803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:28.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.101+0000 7f749249c640 10 monclient: get_auth_request con 0x7f74780397b0 auth_method 0 2024-09-15T01:38:28.125 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.102+0000 7f749249c640 1 --2- 172.21.15.204:0/1900659740 >> v2:172.21.15.204:6812/28210 conn(0x7f74780397b0 0x7f747803bc60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7484009360 tx=0x7f7484008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:28.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.106+0000 7f748a7fc640 1 -- 172.21.15.204:0/1900659740 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f747c04f030 con 0x7f748c132930 2024-09-15T01:38:28.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.106+0000 7f748a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:28.128 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.106+0000 7f748a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:28.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.274+0000 7f7494700640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:28.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.274+0000 7f7494700640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:28.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.274+0000 7f7494700640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:28.276 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:28.274+0000 7f7494700640 1 -- 172.21.15.204:0/1900659740 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"} v 0) v1 -- 0x7f748c09e480 con 0x7f748c132930 2024-09-15T01:38:28.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 10 monclient(hunting): tick 2024-09-15T01:38:28.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 1 monclient(hunting): continuing hunt 2024-09-15T01:38:28.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:38:28.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 msgr2=0x7f68c4005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:28.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f68c40037e0 0x7f68c4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 msgr2=0x7f68c4008660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:28.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f68c40061e0 0x7f68c4008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 msgr2=0x7f68c400b1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:28.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68c4008d50 0x7f68c400b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 10 monclient: _add_conns ranks=[4,0,6,1,10] 2024-09-15T01:38:28.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 reap_dead start 2024-09-15T01:38:28.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c400be30 0x7f68c400e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:28.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 10 monclient(hunting): picked mon.noname-c con 0x7f68c400be30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:28.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 reap_dead start 2024-09-15T01:38:28.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f68c40091c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:28.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f68c40052f0 0x7f68c40085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:28.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 10 monclient(hunting): picked mon.noname-d con 0x7f68c40052f0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:28.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 1 -- 172.21.15.178:0/1568452792 --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d804bf60 con 0x7f68c40091c0 2024-09-15T01:38:28.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 1 -- 172.21.15.178:0/1568452792 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68d8096e00 con 0x7f68c400be30 2024-09-15T01:38:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 1 -- 172.21.15.178:0/1568452792 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f68d8097d50 con 0x7f68c40052f0 2024-09-15T01:38:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d4d99640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f68c40052f0 0x7f68c40085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request con 0x7f68c40052f0 auth_method 0 2024-09-15T01:38:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c400be30 0x7f68c400e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d659c640 10 monclient(hunting): get_auth_request con 0x7f68c400be30 auth_method 0 2024-09-15T01:38:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c400be30 msgr2=0x7f68c400e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:38:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d659c640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c400be30 msgr2=0x7f68c400e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c400be30 0x7f68c400e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.767+0000 7f68d659c640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c400be30 0x7f68c400e210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.768+0000 7f68d6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:28.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.768+0000 7f68d6d9d640 10 monclient(hunting): handle_auth_done global_id 24684 payload 274 2024-09-15T01:38:28.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.768+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c400be30 msgr2=0x7f68c400e210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:28.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.768+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c400be30 0x7f68c400e210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.768+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 >> v1:172.21.15.178:6789/0 conn(0x7f68c40091c0 legacy=0x7f68c4003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:28.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.768+0000 7f68d6d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:28.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.768+0000 7f68d6d9d640 1 monclient: found mon.noname-d 2024-09-15T01:38:28.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.768+0000 7f68d6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:28.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.768+0000 7f68d6d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:38:28.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.768+0000 7f68d6d9d640 1 -- 172.21.15.178:0/1568452792 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c4005800 con 0x7f68c40052f0 2024-09-15T01:38:28.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f68c40052f0 0x7f68c40085e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68d809fde0 tx=0x7f68d000b040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=941cfeef26d8618e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:28.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d5d9b640 1 -- 172.21.15.178:0/1568452792 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68d002ccf0 con 0x7f68c40052f0 2024-09-15T01:38:28.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:28.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d5d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:38:28.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d5d9b640 10 monclient: dump: 2024-09-15T01:38:28.778 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:28.778 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:28.778 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.778 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.778 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:28.778 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:28.778 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:28.778 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:28.779 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:28.779 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:28.779 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:28.779 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:28.779 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:28.779 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:28.779 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:28.779 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:28.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d5d9b640 10 monclient: _finish_auth 0 2024-09-15T01:38:28.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:28.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d5d9b640 1 -- 172.21.15.178:0/1568452792 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d0030070 con 0x7f68c40052f0 2024-09-15T01:38:28.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68dcd9d640 5 monclient: authenticate success, global_id 24684 2024-09-15T01:38:28.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68dcd9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:28.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d5d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:28.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68dcd9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:28.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d5d9b640 1 -- 172.21.15.178:0/1568452792 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68d0035830 con 0x7f68c40052f0 2024-09-15T01:38:28.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68dcd9d640 4 set_mon_vals no callback set 2024-09-15T01:38:28.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:28.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d5d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:28.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.769+0000 7f68d5d9b640 10 monclient: dump: 2024-09-15T01:38:28.780 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:28.780 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:28.781 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.781 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.781 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:28.781 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:28.781 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:28.781 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:28.781 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:28.781 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:28.781 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:28.781 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:28.782 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:28.782 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:28.782 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:28.782 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:28.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.770+0000 7f68dcd9d640 10 monclient: shutdown 2024-09-15T01:38:28.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.770+0000 7f68dcd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:28.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.770+0000 7f68dcd9d640 1 -- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f68c40052f0 msgr2=0x7f68c40085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:28.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.770+0000 7f68dcd9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f68c40052f0 0x7f68c40085e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68d809fde0 tx=0x7f68d000b040 comp rx=0 tx=0).stop 2024-09-15T01:38:28.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.770+0000 7f68dcd9d640 1 -- 172.21.15.178:0/1568452792 shutdown_connections 2024-09-15T01:38:28.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.770+0000 7f68dcd9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f68c40052f0 0x7f68c40085e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.770+0000 7f68dcd9d640 1 --2- 172.21.15.178:0/1568452792 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c400be30 0x7f68c400e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.770+0000 7f68dcd9d640 1 -- 172.21.15.178:0/1568452792 >> 172.21.15.178:0/1568452792 conn(0x7f68d80a22e0 msgr2=0x7f68d80a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.770+0000 7f68dcd9d640 1 -- 172.21.15.178:0/1568452792 shutdown_connections 2024-09-15T01:38:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.770+0000 7f68dcd9d640 1 -- 172.21.15.178:0/1568452792 wait complete. 2024-09-15T01:38:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.770+0000 7f68dcd9d640 10 monclient: build_initial_monmap 2024-09-15T01:38:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.770+0000 7f68dcd9d640 10 monclient: monmap: 2024-09-15T01:38:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:28.784 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 1 Processor -- start 2024-09-15T01:38:28.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 1 -- start start 2024-09-15T01:38:28.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 10 monclient: init 2024-09-15T01:38:28.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:28.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 10 monclient: _add_conns ranks=[4,1,6,3,5,8,0,7,2] 2024-09-15T01:38:28.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c40052f0 0x7f68d8142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:28.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 10 monclient(hunting): picked mon.noname-c con 0x7f68c40052f0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:28.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68c40091c0 0x7f68d8137f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:28.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 10 monclient(hunting): picked mon.noname-f con 0x7f68c40091c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:38:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f68c400be30 0x7f68d812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 10 monclient(hunting): picked mon.noname-d con 0x7f68c400be30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d8131e70 con 0x7f68c40091c0 2024-09-15T01:38:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68d8096f40 con 0x7f68c40052f0 2024-09-15T01:38:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f68d806f1f0 con 0x7f68c400be30 2024-09-15T01:38:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.771+0000 7f68dcd9d640 10 monclient(hunting): authenticate will time out at 1113.521911s 2024-09-15T01:38:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c40052f0 0x7f68d8142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f68c400be30 0x7f68d812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c40052f0 0x7f68d8142570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:55350/0 (socket says 172.21.15.178:55350) 2024-09-15T01:38:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d659c640 1 -- 172.21.15.178:0/4107662928 learned_addr learned my addr 172.21.15.178:0/4107662928 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d659c640 10 monclient(hunting): get_auth_request con 0x7f68c40052f0 auth_method 0 2024-09-15T01:38:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request con 0x7f68c400be30 auth_method 0 2024-09-15T01:38:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d659c640 1 -- 172.21.15.178:0/4107662928 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c40052f0 msgr2=0x7f68d8142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:38:28.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d659c640 1 -- 172.21.15.178:0/4107662928 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c40052f0 msgr2=0x7f68d8142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:28.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:28.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:28.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68cffff640 1 --2- 172.21.15.178:0/4107662928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68c40091c0 0x7f68d8137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:28.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d659c640 1 --2- 172.21.15.178:0/4107662928 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c40052f0 0x7f68d8142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:28.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:28.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68cffff640 10 monclient(hunting): get_auth_request con 0x7f68c40091c0 auth_method 0 2024-09-15T01:38:28.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:28.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68cffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:28.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:28.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d659c640 1 --2- 172.21.15.178:0/4107662928 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c40052f0 0x7f68d8142570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:28.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:28.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:28.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:28.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 10 monclient(hunting): handle_auth_done global_id 24693 payload 824 2024-09-15T01:38:28.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 1 -- 172.21.15.178:0/4107662928 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c40052f0 msgr2=0x7f68d8142570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:28.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/4107662928 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c40052f0 0x7f68d8142570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 1 -- 172.21.15.178:0/4107662928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68c40091c0 msgr2=0x7f68d8137f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:28.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/4107662928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68c40091c0 0x7f68d8137f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:28.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:28.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 1 monclient: found mon.noname-d 2024-09-15T01:38:28.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:28.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:38:28.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 1 -- 172.21.15.178:0/4107662928 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d00099d0 con 0x7f68c400be30 2024-09-15T01:38:28.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d6d9d640 1 --2- 172.21.15.178:0/4107662928 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f68c400be30 0x7f68d812ff90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009590 tx=0x7f68d0038000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:28.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68ce7fc640 1 -- 172.21.15.178:0/4107662928 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68d0040a80 con 0x7f68c400be30 2024-09-15T01:38:28.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68ce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:28.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68ce7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:38:28.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68ce7fc640 10 monclient: dump: 2024-09-15T01:38:28.794 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:28.794 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:28.794 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.795 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.795 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:28.795 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:28.795 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:28.795 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:28.795 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:28.795 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:28.795 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:28.795 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:28.795 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:28.796 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:28.796 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:28.796 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:28.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68ce7fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:28.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:28.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68ce7fc640 1 -- 172.21.15.178:0/4107662928 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d002ce90 con 0x7f68c400be30 2024-09-15T01:38:28.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68dcd9d640 5 monclient: authenticate success, global_id 24693 2024-09-15T01:38:28.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68dcd9d640 10 monclient: _renew_subs 2024-09-15T01:38:28.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68dcd9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:28.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68dcd9d640 1 -- 172.21.15.178:0/4107662928 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d806eaa0 con 0x7f68c400be30 2024-09-15T01:38:28.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68ce7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:28.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68ce7fc640 1 -- 172.21.15.178:0/4107662928 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68d0040c40 con 0x7f68c400be30 2024-09-15T01:38:28.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68ce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:28.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68ce7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:28.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68ce7fc640 10 monclient: dump: 2024-09-15T01:38:28.797 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:28.797 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:28.797 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.797 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:28.797 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:28.797 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:28.797 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:28.798 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:28.798 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:28.798 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:28.798 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:28.798 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:28.798 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:28.798 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:28.798 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:28.798 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:28.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.772+0000 7f68d559a640 4 set_mon_vals no callback set 2024-09-15T01:38:28.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.773+0000 7f68dcd9d640 10 monclient: _renew_subs 2024-09-15T01:38:28.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.773+0000 7f68dcd9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:28.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.773+0000 7f68dcd9d640 1 -- 172.21.15.178:0/4107662928 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d8131200 con 0x7f68c400be30 2024-09-15T01:38:28.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.774+0000 7f68ce7fc640 1 -- 172.21.15.178:0/4107662928 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f68d0003040 con 0x7f68c400be30 2024-09-15T01:38:28.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.774+0000 7f68dcd9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:28.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.774+0000 7f68dcd9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:28.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.774+0000 7f68dcd9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:28.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.774+0000 7f68dcd9d640 1 -- 172.21.15.178:0/4107662928 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d804b720 con 0x7f68c400be30 2024-09-15T01:38:28.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.774+0000 7f68ce7fc640 1 --2- 172.21.15.178:0/4107662928 >> v2:172.21.15.204:6812/28210 conn(0x7f68c403b7c0 0x7f68c403dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:28.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.774+0000 7f68ce7fc640 1 -- 172.21.15.178:0/4107662928 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f68d006e230 con 0x7f68c400be30 2024-09-15T01:38:28.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.774+0000 7f68d659c640 1 --2- 172.21.15.178:0/4107662928 >> v2:172.21.15.204:6812/28210 conn(0x7f68c403b7c0 0x7f68c403dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:28.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.774+0000 7f68d659c640 10 monclient: get_auth_request con 0x7f68c403b7c0 auth_method 0 2024-09-15T01:38:28.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.775+0000 7f68d659c640 1 --2- 172.21.15.178:0/4107662928 >> v2:172.21.15.204:6812/28210 conn(0x7f68c403b7c0 0x7f68c403dc70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f68c8006d90 tx=0x7f68c8008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:28.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.779+0000 7f68ce7fc640 1 -- 172.21.15.178:0/4107662928 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f68d003db20 con 0x7f68c400be30 2024-09-15T01:38:28.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.779+0000 7f68ce7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:28.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:28.779+0000 7f68ce7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:29.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:29.000+0000 7f68dcd9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:38:29.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:29.000+0000 7f68dcd9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:38:29.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:29.000+0000 7f68dcd9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:29.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:29.000+0000 7f68dcd9d640 1 -- 172.21.15.178:0/4107662928 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f68d8070770 con 0x7f68c400be30 2024-09-15T01:38:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:31.100+0000 7f74897fa640 10 monclient: tick 2024-09-15T01:38:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:31.100+0000 7f74897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:31.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:31.100+0000 7f74897fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:31.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:31.772+0000 7f68cd7fa640 10 monclient: tick 2024-09-15T01:38:31.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:31.772+0000 7f68cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:31.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:31.772+0000 7f68cd7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:32.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:32.100+0000 7f74897fa640 10 monclient: tick 2024-09-15T01:38:32.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:32.100+0000 7f74897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:32.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:32.772+0000 7f68cd7fa640 10 monclient: tick 2024-09-15T01:38:32.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:32.772+0000 7f68cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:33.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:33.100+0000 7f74897fa640 10 monclient: tick 2024-09-15T01:38:33.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:33.100+0000 7f74897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:33.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.772+0000 7f68cd7fa640 10 monclient: tick 2024-09-15T01:38:33.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.772+0000 7f68cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:33.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.942+0000 7f68ce7fc640 1 -- 172.21.15.178:0/4107662928 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3177 (secure 0 0 0) 0x7f68d003d340 con 0x7f68c400be30 2024-09-15T01:38:33.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.942+0000 7f68ce7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:38:33.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.942+0000 7f68ce7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:33.945 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":74,"quorum":[0,1,2,3,5,6,7,8],"quorum_names":["a","f","b","g","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:38:33.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 1 -- 172.21.15.178:0/4107662928 >> v2:172.21.15.204:6812/28210 conn(0x7f68c403b7c0 msgr2=0x7f68c403dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:33.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 1 --2- 172.21.15.178:0/4107662928 >> v2:172.21.15.204:6812/28210 conn(0x7f68c403b7c0 0x7f68c403dc70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f68c8006d90 tx=0x7f68c8008040 comp rx=0 tx=0).stop 2024-09-15T01:38:33.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 10 monclient: shutdown 2024-09-15T01:38:33.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:33.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 1 -- 172.21.15.178:0/4107662928 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f68c400be30 msgr2=0x7f68d812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:33.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 1 --2- 172.21.15.178:0/4107662928 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f68c400be30 0x7f68d812ff90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009590 tx=0x7f68d0038000 comp rx=0 tx=0).stop 2024-09-15T01:38:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 1 -- 172.21.15.178:0/4107662928 shutdown_connections 2024-09-15T01:38:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 1 --2- 172.21.15.178:0/4107662928 >> v2:172.21.15.204:6812/28210 conn(0x7f68c403b7c0 0x7f68c403dc70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 1 --2- 172.21.15.178:0/4107662928 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f68c400be30 0x7f68d812ff90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 1 --2- 172.21.15.178:0/4107662928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68c40091c0 0x7f68d8137f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 1 --2- 172.21.15.178:0/4107662928 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f68c40052f0 0x7f68d8142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 1 -- 172.21.15.178:0/4107662928 >> 172.21.15.178:0/4107662928 conn(0x7f68d80a22e0 msgr2=0x7f68d809b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 1 -- 172.21.15.178:0/4107662928 shutdown_connections 2024-09-15T01:38:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:33.946+0000 7f68dcd9d640 1 -- 172.21.15.178:0/4107662928 wait complete. 2024-09-15T01:38:34.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.012+0000 7f748a7fc640 1 -- 172.21.15.204:0/1900659740 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}]=0 v260) v1 ==== 106+0+3 (secure 0 0 0) 0x7f747c01a960 con 0x7f748c132930 2024-09-15T01:38:34.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.012+0000 7f748a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:34.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.012+0000 7f748a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:34.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.100+0000 7f74897fa640 10 monclient: tick 2024-09-15T01:38:34.101 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.100+0000 7f74897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:34.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.118+0000 7f7494700640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:34.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.118+0000 7f7494700640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:34.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.118+0000 7f7494700640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:34.119 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.118+0000 7f7494700640 1 -- 172.21.15.204:0/1900659740 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"} v 0) v1 -- 0x7f748c09a0c0 con 0x7f748c132930 2024-09-15T01:38:34.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.118+0000 7f748a7fc640 1 -- 172.21.15.204:0/1900659740 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}]=0 v260) v1 ==== 106+0+3 (secure 0 0 0) 0x7f747c010300 con 0x7f748c132930 2024-09-15T01:38:34.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.118+0000 7f748a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "e5de064c-679b-416c-b9b7-f61aa58c84ce"}] 2024-09-15T01:38:34.120 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.118+0000 7f748a7fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:38:34.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 1 -- 172.21.15.204:0/1900659740 >> v2:172.21.15.204:6812/28210 conn(0x7f74780397b0 msgr2=0x7f747803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:34.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 1 --2- 172.21.15.204:0/1900659740 >> v2:172.21.15.204:6812/28210 conn(0x7f74780397b0 0x7f747803bc60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7484009360 tx=0x7f7484008040 comp rx=0 tx=0).stop 2024-09-15T01:38:34.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 10 monclient: shutdown 2024-09-15T01:38:34.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:34.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 1 -- 172.21.15.204:0/1900659740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c132930 msgr2=0x7f748c11dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:34.122 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c132930 0x7f748c11dcd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f747c00cc10 tx=0x7f747c001970 comp rx=0 tx=0).stop 2024-09-15T01:38:34.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 1 -- 172.21.15.204:0/1900659740 shutdown_connections 2024-09-15T01:38:34.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f748c132930 0x7f748c11dcd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:34.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 1 --2- 172.21.15.204:0/1900659740 >> v2:172.21.15.204:6812/28210 conn(0x7f74780397b0 0x7f747803bc60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:34.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f748c0a2400 0x7f748c11d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:34.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 1 --2- 172.21.15.204:0/1900659740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f748c09f990 0x7f748c11d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:34.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 1 -- 172.21.15.204:0/1900659740 >> 172.21.15.204:0/1900659740 conn(0x7f748c06e860 msgr2=0x7f748c0a45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:34.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 1 -- 172.21.15.204:0/1900659740 shutdown_connections 2024-09-15T01:38:34.123 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:34.120+0000 7f7494700640 1 -- 172.21.15.204:0/1900659740 wait complete. 2024-09-15T01:38:34.130 INFO:tasks.workunit.client.0.smithi204.stderr:+ na2=12 2024-09-15T01:38:34.131 INFO:tasks.workunit.client.0.smithi204.stdout:OK 2024-09-15T01:38:34.131 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo OK 2024-09-15T01:38:34.132 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T01:38:34.132 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T01:38:34.156 INFO:tasks.workunit:Running workunits matching mon/caps.sh on client.0... 2024-09-15T01:38:34.158 INFO:tasks.workunit:Running workunit mon/caps.sh... 2024-09-15T01:38:34.158 DEBUG:teuthology.orchestra.run.smithi204:workunit test mon/caps.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d75f23820806da622f561ee5ee7551015285993f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/caps.sh 2024-09-15T01:38:34.216 INFO:tasks.workunit.client.0.smithi204.stderr:+ tmp=/tmp/cephtest-mon-caps-madness 2024-09-15T01:38:34.217 INFO:tasks.workunit.client.0.smithi204.stderr:+ exit_on_error=1 2024-09-15T01:38:34.217 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ ! -z '' ]] 2024-09-15T01:38:34.217 INFO:tasks.workunit.client.0.smithi204.stderr:++ uname 2024-09-15T01:38:34.218 INFO:tasks.workunit.client.0.smithi204.stderr:+ '[' Linux = FreeBSD ']' 2024-09-15T01:38:34.218 INFO:tasks.workunit.client.0.smithi204.stderr:+ ETIMEDOUT=110 2024-09-15T01:38:34.218 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph auth get-or-create client.bazar > /tmp/cephtest-mon-caps-madness.bazar.keyring' 0 2024-09-15T01:38:34.218 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph auth get-or-create client.bazar > /tmp/cephtest-mon-caps-madness.bazar.keyring' 2024-09-15T01:38:34.218 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=0 2024-09-15T01:38:34.218 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph auth get-or-create client.bazar '>' /tmp/cephtest-mon-caps-madness.bazar.keyring 2024-09-15T01:38:34.218 INFO:tasks.workunit.client.0.smithi204.stdout:ceph auth get-or-create client.bazar > /tmp/cephtest-mon-caps-madness.bazar.keyring 2024-09-15T01:38:34.218 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph auth get-or-create client.bazar '>' /tmp/cephtest-mon-caps-madness.bazar.keyring 2024-09-15T01:38:34.685 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=0 2024-09-15T01:38:34.685 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 0 -ne 0 ]] 2024-09-15T01:38:34.685 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:38:34.685 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status' 13 2024-09-15T01:38:34.685 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status' 2024-09-15T01:38:34.685 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=13 2024-09-15T01:38:34.686 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status 2024-09-15T01:38:34.686 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status 2024-09-15T01:38:34.686 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status 2024-09-15T01:38:36.964 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:38:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.064+0000 7f7625100640 10 monclient: get_monmap_and_config 2024-09-15T01:38:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.064+0000 7f7625100640 10 monclient: build_initial_monmap 2024-09-15T01:38:37.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.064+0000 7f7625100640 10 monclient: monmap: 2024-09-15T01:38:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:37.065155+0000 2024-09-15T01:38:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:37.065155+0000 2024-09-15T01:38:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:37.083 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 1 Processor -- start 2024-09-15T01:38:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 1 -- start start 2024-09-15T01:38:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 10 monclient: init 2024-09-15T01:38:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 10 monclient: _add_conns ranks=[7,1,3,9,0,8,2,5,6,4,10] 2024-09-15T01:38:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f76200a2930 0x7f7620132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 10 monclient(hunting): picked mon.noname-i con 0x7f76200a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f762009faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f762009fe70 0x7f76200a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:37.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 10 monclient(hunting): picked mon.noname-g con 0x7f762009fe70 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762004bf60 con 0x7f762009faa0 2024-09-15T01:38:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762009dd80 con 0x7f762009fe70 2024-09-15T01:38:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f76200a2d00 con 0x7f76200a2930 2024-09-15T01:38:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7625100640 10 monclient(hunting): authenticate will time out at 1121.815564s 2024-09-15T01:38:37.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761f59e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f762009fe70 0x7f76200a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7617fff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f762009faa0 0x7f76200a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:46718/0 (socket says 172.21.15.178:46718) 2024-09-15T01:38:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f7617fff640 1 -- 172.21.15.178:0/3574205934 learned_addr learned my addr 172.21.15.178:0/3574205934 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:38:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761f59e640 10 monclient(hunting): get_auth_request con 0x7f762009fe70 auth_method 0 2024-09-15T01:38:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3574205934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f76200a2930 0x7f7620132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761ed9d640 10 monclient(hunting): get_auth_request con 0x7f76200a2930 auth_method 0 2024-09-15T01:38:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:37.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.065+0000 7f761ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761f59e640 10 monclient(hunting): handle_auth_done global_id 15780 payload 274 2024-09-15T01:38:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761f59e640 1 -- 172.21.15.178:0/3574205934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f76200a2930 msgr2=0x7f7620132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761f59e640 1 --2- 172.21.15.178:0/3574205934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f76200a2930 0x7f7620132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761f59e640 1 -- 172.21.15.178:0/3574205934 >> v1:172.21.15.204:6789/0 conn(0x7f762009faa0 legacy=0x7f76200a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761e59c640 1 -- 172.21.15.178:0/3574205934 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999818208 0 0) 0x7f762004bf60 con 0x7f762009faa0 2024-09-15T01:38:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761f59e640 10 monclient: _finish_hunting 0 2024-09-15T01:38:37.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761f59e640 1 monclient: found mon.noname-g 2024-09-15T01:38:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761f59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761f59e640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761f59e640 1 -- 172.21.15.178:0/3574205934 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76200a0310 con 0x7f762009fe70 2024-09-15T01:38:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761e59c640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:38:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761f59e640 1 --2- 172.21.15.178:0/3574205934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f762009fe70 0x7f76200a0fd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7618009e90 tx=0x7f7618009780 comp rx=0 tx=0).ready entity=mon.3 client_cookie=19897fadbe154e25 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.066+0000 7f761e59c640 1 -- 172.21.15.178:0/3574205934 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7618031880 con 0x7f762009fe70 2024-09-15T01:38:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f761e59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f761e59c640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f761e59c640 10 monclient: dump: 2024-09-15T01:38:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:37.089 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:37.090 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:37.091 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:37.091 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f761e59c640 10 monclient: _finish_auth 0 2024-09-15T01:38:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f761e59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f761e59c640 1 -- 172.21.15.178:0/3574205934 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761802d050 con 0x7f762009fe70 2024-09-15T01:38:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 5 monclient: authenticate success, global_id 15780 2024-09-15T01:38:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:37.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f761e59c640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f761e59c640 1 -- 172.21.15.178:0/3574205934 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76180303a0 con 0x7f762009fe70 2024-09-15T01:38:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 4 set_mon_vals no callback set 2024-09-15T01:38:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f761e59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f761e59c640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:37.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f761e59c640 10 monclient: dump: 2024-09-15T01:38:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:37.094 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:37.095 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:37.096 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 10 monclient: shutdown 2024-09-15T01:38:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 1 -- 172.21.15.178:0/3574205934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f762009fe70 msgr2=0x7f76200a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 1 --2- 172.21.15.178:0/3574205934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f762009fe70 0x7f76200a0fd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7618009e90 tx=0x7f7618009780 comp rx=0 tx=0).stop 2024-09-15T01:38:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 1 -- 172.21.15.178:0/3574205934 shutdown_connections 2024-09-15T01:38:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 1 --2- 172.21.15.178:0/3574205934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f762009fe70 0x7f76200a0fd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 1 --2- 172.21.15.178:0/3574205934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f76200a2930 0x7f7620132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 1 -- 172.21.15.178:0/3574205934 >> 172.21.15.178:0/3574205934 conn(0x7f76200a5ed0 msgr2=0x7f76200a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 1 -- 172.21.15.178:0/3574205934 shutdown_connections 2024-09-15T01:38:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.067+0000 7f7625100640 1 -- 172.21.15.178:0/3574205934 wait complete. 2024-09-15T01:38:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient: build_initial_monmap 2024-09-15T01:38:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient: monmap: 2024-09-15T01:38:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:37.098 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:37.099 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:37.099 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:37.099 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:37.099 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:37.099 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:37.099 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:37.099 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:37.099 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:37.099 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:37.099 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 1 Processor -- start 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 1 -- start start 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient: init 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient: _add_conns ranks=[4,8,0,7,6,1,2,3,5] 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient(hunting): picked mon.noname-c con 0x7f762009faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient(hunting): picked mon.noname-e con 0x7f762009fe70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient(hunting): picked mon.noname-a con 0x7f76200a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762011eae0 con 0x7f76200a2930 2024-09-15T01:38:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76200a2390 con 0x7f762009faa0 2024-09-15T01:38:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f762009d630 con 0x7f762009fe70 2024-09-15T01:38:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.068+0000 7f7625100640 10 monclient(hunting): authenticate will time out at 1121.818887s 2024-09-15T01:38:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761ed9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761f59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f7617fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761ed9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:55364/0 (socket says 172.21.15.178:55364) 2024-09-15T01:38:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761ed9d640 1 -- 172.21.15.178:0/3867747599 learned_addr learned my addr 172.21.15.178:0/3867747599 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761f59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53048/0 (socket says 172.21.15.178:53048) 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761ed9d640 10 monclient(hunting): get_auth_request con 0x7f762009faa0 auth_method 0 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f7617fff640 10 monclient(hunting): get_auth_request con 0x7f762009fe70 auth_method 0 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f7617fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f7617fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f7617fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761f59e640 10 monclient(hunting): get_auth_request con 0x7f76200a2930 auth_method 0 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761ed9d640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 msgr2=0x7f762009ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:38:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f7617fff640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 msgr2=0x7f76201316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:38:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761ed9d640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 msgr2=0x7f762009ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f7617fff640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 msgr2=0x7f76201316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f7617fff640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761f59e640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 msgr2=0x7f762011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:38:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761f59e640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 msgr2=0x7f762011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761f59e640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f7617fff640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:37.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.069+0000 7f761f59e640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:37.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f7617fff640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761f59e640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761ed9d640 10 monclient(hunting): get_auth_request con 0x7f762009faa0 auth_method 0 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f7617fff640 10 monclient(hunting): get_auth_request con 0x7f762009fe70 auth_method 0 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f7617fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f7617fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f7617fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761f59e640 10 monclient(hunting): get_auth_request con 0x7f76200a2930 auth_method 0 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761ed9d640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 msgr2=0x7f762009ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761ed9d640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 msgr2=0x7f762009ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f7617fff640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 msgr2=0x7f76201316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f7617fff640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 msgr2=0x7f76201316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f7617fff640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761f59e640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 msgr2=0x7f762011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761f59e640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 msgr2=0x7f762011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761f59e640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f7617fff640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:37.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.270+0000 7f761f59e640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:37.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761f59e640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f7617fff640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761ed9d640 10 monclient(hunting): get_auth_request con 0x7f762009faa0 auth_method 0 2024-09-15T01:38:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761f59e640 10 monclient(hunting): get_auth_request con 0x7f76200a2930 auth_method 0 2024-09-15T01:38:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f7617fff640 10 monclient(hunting): get_auth_request con 0x7f762009fe70 auth_method 0 2024-09-15T01:38:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f7617fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f7617fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f7617fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761ed9d640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 msgr2=0x7f762009ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:38:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761ed9d640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 msgr2=0x7f762009ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761f59e640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 msgr2=0x7f762011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:38:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f7617fff640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 msgr2=0x7f76201316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:38:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761f59e640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 msgr2=0x7f762011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f7617fff640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 msgr2=0x7f76201316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761f59e640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f7617fff640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f761f59e640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:37.671+0000 7f7617fff640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=13 2024-09-15T01:38:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T01:38:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:38:37.759 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph auth del client.bazar 2024-09-15T01:38:37.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.839+0000 7fd69ea6f640 10 monclient: get_monmap_and_config 2024-09-15T01:38:37.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.839+0000 7fd69ea6f640 10 monclient: build_initial_monmap 2024-09-15T01:38:37.840 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.839+0000 7fd69ea6f640 10 monclient: monmap: 2024-09-15T01:38:37.840 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:38:37.839616+0000 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:38:37.839616+0000 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:37.841 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:37.842 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:37.842 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:37.842 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:37.842 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:37.850 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.839+0000 7fd69ea6f640 1 Processor -- start 2024-09-15T01:38:37.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 1 -- start start 2024-09-15T01:38:37.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 10 monclient: init 2024-09-15T01:38:37.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:37.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 10 monclient: _add_conns ranks=[3,0,2,1,6,9,4,7,5,10,8] 2024-09-15T01:38:37.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd698099a10 0x7fd69809ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:37.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 10 monclient(hunting): picked mon.noname-g con 0x7fd698099a10 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:37.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd69809c160 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:37.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd69809ddb0 0x7fd6981374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:37.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 10 monclient(hunting): picked mon.noname-b con 0x7fd69809ddb0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:37.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980648a0 con 0x7fd69809c160 2024-09-15T01:38:37.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:37.851 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69809cb60 con 0x7fd69809ddb0 2024-09-15T01:38:37.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:37.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69809bdb0 con 0x7fd698099a10 2024-09-15T01:38:37.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:37.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69ea6f640 10 monclient(hunting): authenticate will time out at 1133.482560s 2024-09-15T01:38:37.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd698099a10 0x7fd69809ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd698099a10 0x7fd69809ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:49120/0 (socket says 172.21.15.204:49120) 2024-09-15T01:38:37.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 1 -- 172.21.15.204:0/1412342367 learned_addr learned my addr 172.21.15.204:0/1412342367 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:37.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 10 monclient(hunting): get_auth_request con 0x7fd698099a10 auth_method 0 2024-09-15T01:38:37.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.852 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:37.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:37.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:37.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69d00c640 1 --2- 172.21.15.204:0/1412342367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd69809ddb0 0x7fd6981374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:37.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69d00c640 10 monclient(hunting): get_auth_request con 0x7fd69809ddb0 auth_method 0 2024-09-15T01:38:37.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69d00c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69d00c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69d00c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:37.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69d00c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:37.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69d00c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:37.853 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69d00c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 10 monclient(hunting): handle_auth_done global_id 15789 payload 274 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 1 -- 172.21.15.204:0/1412342367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd69809ddb0 msgr2=0x7fd6981374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 1 --2- 172.21.15.204:0/1412342367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd69809ddb0 0x7fd6981374c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 1 -- 172.21.15.204:0/1412342367 >> v1:172.21.15.178:6789/0 conn(0x7fd69809c160 legacy=0x7fd69809d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 10 monclient: _finish_hunting 0 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 1 monclient: found mon.noname-g 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.840+0000 7fd69c80b640 1 -- 172.21.15.204:0/1412342367 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69809c530 con 0x7fd698099a10 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd69c80b640 1 --2- 172.21.15.204:0/1412342367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd698099a10 0x7fd69809ae20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd688009940 tx=0x7fd6880091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=b8012079c3f49b33 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd6977fe640 1 -- 172.21.15.204:0/1412342367 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd68802ad00 con 0x7fd698099a10 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd6977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd6977fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:37.854 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd6977fe640 10 monclient: dump: 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:37.855 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd6977fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd6977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd6977fe640 1 -- 172.21.15.204:0/1412342367 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68802ae80 con 0x7fd698099a10 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd69ea6f640 5 monclient: authenticate success, global_id 15789 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd6977fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd69ea6f640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd6977fe640 1 -- 172.21.15.204:0/1412342367 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd688031ca0 con 0x7fd698099a10 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd6977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd69ea6f640 4 set_mon_vals no callback set 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd6977fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:37.856 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.841+0000 7fd6977fe640 10 monclient: dump: 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:37.857 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.842+0000 7fd69ea6f640 10 monclient: shutdown 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.842+0000 7fd69ea6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.842+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1412342367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd698099a10 msgr2=0x7fd69809ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.842+0000 7fd69ea6f640 1 --2- 172.21.15.204:0/1412342367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd698099a10 0x7fd69809ae20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd688009940 tx=0x7fd6880091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.842+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1412342367 shutdown_connections 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.842+0000 7fd69ea6f640 1 --2- 172.21.15.204:0/1412342367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd69809ddb0 0x7fd6981374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.842+0000 7fd69ea6f640 1 --2- 172.21.15.204:0/1412342367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd698099a10 0x7fd69809ae20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.842+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1412342367 >> 172.21.15.204:0/1412342367 conn(0x7fd698068b30 msgr2=0x7fd6980a2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.842+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1412342367 shutdown_connections 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.842+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1412342367 wait complete. 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.843+0000 7fd69ea6f640 10 monclient: build_initial_monmap 2024-09-15T01:38:37.858 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.843+0000 7fd69ea6f640 10 monclient: monmap: 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:37.859 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.843+0000 7fd69ea6f640 1 Processor -- start 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.843+0000 7fd69ea6f640 1 -- start start 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.843+0000 7fd69ea6f640 10 monclient: init 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 10 monclient: _add_conns ranks=[7,3,0,2,4,1,6,5,8] 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd698099a10 0x7fd6981315d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 10 monclient(hunting): picked mon.noname-i con 0x7fd698099a10 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd69809c160 0x7fd69811b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 10 monclient(hunting): picked mon.noname-g con 0x7fd69809c160 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd69809ddb0 0x7fd69811b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:37.860 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 10 monclient(hunting): picked mon.noname-a con 0x7fd69809ddb0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69809ccf0 con 0x7fd69809ddb0 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69811cca0 con 0x7fd69809c160 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd69811d140 con 0x7fd698099a10 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 10 monclient(hunting): authenticate will time out at 1133.486111s 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd698099a10 0x7fd6981315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd698099a10 0x7fd6981315d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:38848/0 (socket says 172.21.15.204:38848) 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 1 -- 172.21.15.204:0/1049484393 learned_addr learned my addr 172.21.15.204:0/1049484393 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 10 monclient(hunting): get_auth_request con 0x7fd698099a10 auth_method 0 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.861 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd697fff640 1 --2- 172.21.15.204:0/1049484393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd69809c160 0x7fd69811b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd697fff640 10 monclient(hunting): get_auth_request con 0x7fd69809c160 auth_method 0 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd697fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd697fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd697fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69d00c640 1 --2- 172.21.15.204:0/1049484393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd69809ddb0 0x7fd69811b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69d00c640 10 monclient(hunting): get_auth_request con 0x7fd69809ddb0 auth_method 0 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69d00c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69d00c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:37.862 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69d00c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:37.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd697fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:37.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd697fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:37.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd697fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:37.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69d00c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:37.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69d00c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:37.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69d00c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:37.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 10 monclient(hunting): handle_auth_done global_id 24856 payload 824 2024-09-15T01:38:37.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 1 -- 172.21.15.204:0/1049484393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd69809c160 msgr2=0x7fd69811b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:37.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 1 --2- 172.21.15.204:0/1049484393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd69809c160 0x7fd69811b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:37.863 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 1 -- 172.21.15.204:0/1049484393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd69809ddb0 msgr2=0x7fd69811b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:37.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 1 --2- 172.21.15.204:0/1049484393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd69809ddb0 0x7fd69811b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:37.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 10 monclient: _finish_hunting 0 2024-09-15T01:38:37.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 1 monclient: found mon.noname-i 2024-09-15T01:38:37.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:37.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:38:37.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 1 -- 172.21.15.204:0/1049484393 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd688009450 con 0x7fd698099a10 2024-09-15T01:38:37.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69d00c640 1 --2- 172.21.15.204:0/1049484393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd69809ddb0 0x7fd69811b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:37.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd697fff640 1 --2- 172.21.15.204:0/1049484393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd69809c160 0x7fd69811b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:37.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69c80b640 1 --2- 172.21.15.204:0/1049484393 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd698099a10 0x7fd6981315d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd68802a700 tx=0x7fd688032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:37.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd694ff9640 1 -- 172.21.15.204:0/1049484393 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd68803ca90 con 0x7fd698099a10 2024-09-15T01:38:37.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd694ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:37.864 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd694ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd694ff9640 10 monclient: dump: 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:37.865 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd694ff9640 10 monclient: _finish_auth 0 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd694ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd694ff9640 1 -- 172.21.15.204:0/1049484393 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68803cc10 con 0x7fd698099a10 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd694ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd694ff9640 1 -- 172.21.15.204:0/1049484393 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd68803cd80 con 0x7fd698099a10 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.844+0000 7fd69ea6f640 5 monclient: authenticate success, global_id 24856 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.845+0000 7fd694ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.845+0000 7fd694ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.845+0000 7fd696ffd640 4 set_mon_vals no callback set 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.845+0000 7fd694ff9640 10 monclient: dump: 2024-09-15T01:38:37.866 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:37.867 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.845+0000 7fd69ea6f640 10 monclient: _renew_subs 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.845+0000 7fd69ea6f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.845+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1049484393 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69813e640 con 0x7fd698099a10 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.845+0000 7fd69ea6f640 10 monclient: _renew_subs 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.845+0000 7fd69ea6f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.845+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1049484393 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd698132c40 con 0x7fd698099a10 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.845+0000 7fd694ff9640 1 -- 172.21.15.204:0/1049484393 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd68803c610 con 0x7fd698099a10 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.845+0000 7fd69ea6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.846+0000 7fd69ea6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.846+0000 7fd69ea6f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.846+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1049484393 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69811e880 con 0x7fd698099a10 2024-09-15T01:38:37.868 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.846+0000 7fd694ff9640 1 --2- 172.21.15.204:0/1049484393 >> v2:172.21.15.204:6812/28210 conn(0x7fd6780397b0 0x7fd67803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:37.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.846+0000 7fd694ff9640 1 -- 172.21.15.204:0/1049484393 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd68806bb20 con 0x7fd698099a10 2024-09-15T01:38:37.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.846+0000 7fd697fff640 1 --2- 172.21.15.204:0/1049484393 >> v2:172.21.15.204:6812/28210 conn(0x7fd6780397b0 0x7fd67803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:37.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.846+0000 7fd697fff640 10 monclient: get_auth_request con 0x7fd6780397b0 auth_method 0 2024-09-15T01:38:37.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.846+0000 7fd697fff640 1 --2- 172.21.15.204:0/1049484393 >> v2:172.21.15.204:6812/28210 conn(0x7fd6780397b0 0x7fd67803bc60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd680007fd0 tx=0x7fd680009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:37.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.850+0000 7fd694ff9640 1 -- 172.21.15.204:0/1049484393 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd68802f2a0 con 0x7fd698099a10 2024-09-15T01:38:37.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.850+0000 7fd694ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:37.869 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:37.850+0000 7fd694ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:38.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.066+0000 7fd69ea6f640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bazar"}] 2024-09-15T01:38:38.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.066+0000 7fd69ea6f640 10 monclient: _send_command 2 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-09-15T01:38:38.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.066+0000 7fd69ea6f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:38.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.066+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1049484393 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "auth del", "entity": "client.bazar"} v 0) v1 -- 0x7fd698070b80 con 0x7fd698099a10 2024-09-15T01:38:38.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.070+0000 7fd694ff9640 1 -- 172.21.15.204:0/1049484393 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bazar"}]=0 v7) v1 ==== 82+0+0 (secure 0 0 0) 0x7fd688030e60 con 0x7fd698099a10 2024-09-15T01:38:38.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.070+0000 7fd694ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-09-15T01:38:38.072 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.070+0000 7fd694ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:38.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.220+0000 7fd69ea6f640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bazar"}] 2024-09-15T01:38:38.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.220+0000 7fd69ea6f640 10 monclient: _send_command 3 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-09-15T01:38:38.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.220+0000 7fd69ea6f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:38:38.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.220+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1049484393 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "auth del", "entity": "client.bazar"} v 0) v1 -- 0x7fd69806f750 con 0x7fd698099a10 2024-09-15T01:38:38.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.221+0000 7fd694ff9640 1 -- 172.21.15.204:0/1049484393 <== mon.7 v2:172.21.15.204:3303/0 8 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bazar"}]=0 v7) v1 ==== 82+0+0 (secure 0 0 0) 0x7fd688037020 con 0x7fd698099a10 2024-09-15T01:38:38.222 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.221+0000 7fd694ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-09-15T01:38:38.223 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.221+0000 7fd694ff9640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:38:38.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1049484393 >> v2:172.21.15.204:6812/28210 conn(0x7fd6780397b0 msgr2=0x7fd67803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 1 --2- 172.21.15.204:0/1049484393 >> v2:172.21.15.204:6812/28210 conn(0x7fd6780397b0 0x7fd67803bc60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd680007fd0 tx=0x7fd680009040 comp rx=0 tx=0).stop 2024-09-15T01:38:38.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 10 monclient: shutdown 2024-09-15T01:38:38.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:38.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1049484393 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd698099a10 msgr2=0x7fd6981315d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 1 --2- 172.21.15.204:0/1049484393 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd698099a10 0x7fd6981315d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd68802a700 tx=0x7fd688032000 comp rx=0 tx=0).stop 2024-09-15T01:38:38.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1049484393 shutdown_connections 2024-09-15T01:38:38.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 1 --2- 172.21.15.204:0/1049484393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd69809c160 0x7fd69811b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 1 --2- 172.21.15.204:0/1049484393 >> v2:172.21.15.204:6812/28210 conn(0x7fd6780397b0 0x7fd67803bc60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 1 --2- 172.21.15.204:0/1049484393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd69809ddb0 0x7fd69811b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 1 --2- 172.21.15.204:0/1049484393 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd698099a10 0x7fd6981315d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.225 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1049484393 >> 172.21.15.204:0/1049484393 conn(0x7fd698068b30 msgr2=0x7fd69806d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:38.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1049484393 shutdown_connections 2024-09-15T01:38:38.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:38:38.223+0000 7fd69ea6f640 1 -- 172.21.15.204:0/1049484393 wait complete. 2024-09-15T01:38:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:+ c=''\''allow command "auth ls", allow command quorum_status'\''' 2024-09-15T01:38:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph auth get-or-create client.foo mon '\''allow command "auth ls", allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.foo.keyring' 0 2024-09-15T01:38:38.233 INFO:tasks.workunit.client.0.smithi204.stdout:ceph auth get-or-create client.foo mon 'allow command "auth ls", allow command quorum_status' > /tmp/cephtest-mon-caps-madness.foo.keyring 2024-09-15T01:38:38.233 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph auth get-or-create client.foo mon '\''allow command "auth ls", allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.foo.keyring' 2024-09-15T01:38:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=0 2024-09-15T01:38:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph auth get-or-create client.foo mon ''\''allow' command '"auth' 'ls",' allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.foo.keyring 2024-09-15T01:38:38.234 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph auth get-or-create client.foo mon ''\''allow' command '"auth' 'ls",' allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.foo.keyring 2024-09-15T01:38:38.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f7617fff640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761f59e640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f7617fff640 10 monclient(hunting): get_auth_request con 0x7f762009fe70 auth_method 0 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f7617fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f7617fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f7617fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 10 monclient(hunting): get_auth_request con 0x7f762009faa0 auth_method 0 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761f59e640 10 monclient(hunting): get_auth_request con 0x7f76200a2930 auth_method 0 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:38.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f7617fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f7617fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f7617fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 10 monclient(hunting): handle_auth_done global_id 34105 payload 824 2024-09-15T01:38:38.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 msgr2=0x7f76201316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 msgr2=0x7f762011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 1 monclient: found mon.noname-c 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761ed9d640 1 -- 172.21.15.178:0/3867747599 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7618009a20 con 0x7f762009faa0 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f7617fff640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.472+0000 7f761f59e640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f761ed9d640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7610015a90 tx=0x7f7610008040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7616ffd640 1 -- 172.21.15.178:0/3867747599 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7610017440 con 0x7f762009faa0 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7616ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:38.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7616ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:38:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7616ffd640 10 monclient: dump: 2024-09-15T01:38:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7616ffd640 10 monclient: _finish_auth 0 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7616ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7616ffd640 1 -- 172.21.15.178:0/3867747599 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76100175a0 con 0x7f762009faa0 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7625100640 5 monclient: authenticate success, global_id 34105 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7625100640 10 monclient: _renew_subs 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7625100640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7625100640 1 -- 172.21.15.178:0/3867747599 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7620142a50 con 0x7f762009faa0 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7616ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7616ffd640 1 -- 172.21.15.178:0/3867747599 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7610017bd0 con 0x7f762009faa0 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7616ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7616ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7616ffd640 10 monclient: dump: 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7625100640 10 monclient: _renew_subs 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7625100640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f7625100640 1 -- 172.21.15.178:0/3867747599 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762011e800 con 0x7f762009faa0 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.473+0000 7f761dd9b640 4 set_mon_vals no callback set 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.474+0000 7f7616ffd640 1 -- 172.21.15.178:0/3867747599 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7610017d30 con 0x7f762009faa0 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.474+0000 7f7625100640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.474+0000 7f7625100640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.474+0000 7f7625100640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:38:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.474+0000 7f7625100640 1 -- 172.21.15.178:0/3867747599 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f762011ece0 con 0x7f762009faa0 2024-09-15T01:38:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.475+0000 7f7616ffd640 1 --2- 172.21.15.178:0/3867747599 >> v2:172.21.15.204:6812/28210 conn(0x7f76000397b0 0x7f760003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.475+0000 7f7616ffd640 1 -- 172.21.15.178:0/3867747599 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f761004ca30 con 0x7f762009faa0 2024-09-15T01:38:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.475+0000 7f7617fff640 1 --2- 172.21.15.178:0/3867747599 >> v2:172.21.15.204:6812/28210 conn(0x7f76000397b0 0x7f760003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.475+0000 7f7617fff640 10 monclient: get_auth_request con 0x7f76000397b0 auth_method 0 2024-09-15T01:38:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.475+0000 7f7617fff640 1 --2- 172.21.15.178:0/3867747599 >> v2:172.21.15.204:6812/28210 conn(0x7f76000397b0 0x7f760003bc60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7608000c00 tx=0x7f7608007f90 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.480+0000 7f7616ffd640 1 -- 172.21.15.178:0/3867747599 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7610020ad0 con 0x7f762009faa0 2024-09-15T01:38:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.480+0000 7f7616ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.480+0000 7f7616ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:38.699 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=0 2024-09-15T01:38:38.699 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 0 -ne 0 ]] 2024-09-15T01:38:38.699 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:38:38.699 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 0 2024-09-15T01:38:38.699 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-09-15T01:38:38.699 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 2024-09-15T01:38:38.700 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=0 2024-09-15T01:38:38.700 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-09-15T01:38:38.700 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-09-15T01:38:38.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.700+0000 7f7625100640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:38:38.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.700+0000 7f7625100640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:38:38.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.700+0000 7f7625100640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:38:38.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.700+0000 7f7625100640 1 -- 172.21.15.178:0/3867747599 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7620142a50 con 0x7f762009faa0 2024-09-15T01:38:38.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.700+0000 7f7616ffd640 1 -- 172.21.15.178:0/3867747599 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7f7620142a50 con 0x7f762009faa0 2024-09-15T01:38:38.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.700+0000 7f7616ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:38:38.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.700+0000 7f7616ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:38.703 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":76,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:38:38.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 1 -- 172.21.15.178:0/3867747599 >> v2:172.21.15.204:6812/28210 conn(0x7f76000397b0 msgr2=0x7f760003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 1 --2- 172.21.15.178:0/3867747599 >> v2:172.21.15.204:6812/28210 conn(0x7f76000397b0 0x7f760003bc60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7608000c00 tx=0x7f7608007f90 comp rx=0 tx=0).stop 2024-09-15T01:38:38.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 10 monclient: shutdown 2024-09-15T01:38:38.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:38.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 1 -- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 msgr2=0x7f762009ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7610015a90 tx=0x7f7610008040 comp rx=0 tx=0).stop 2024-09-15T01:38:38.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 1 -- 172.21.15.178:0/3867747599 shutdown_connections 2024-09-15T01:38:38.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 1 --2- 172.21.15.178:0/3867747599 >> v2:172.21.15.204:6812/28210 conn(0x7f76000397b0 0x7f760003bc60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76200a2930 0x7f762011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f762009fe70 0x7f76201316e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 1 --2- 172.21.15.178:0/3867747599 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f762009faa0 0x7f762009ae30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 1 -- 172.21.15.178:0/3867747599 >> 172.21.15.178:0/3867747599 conn(0x7f76200a5ed0 msgr2=0x7f762009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:38.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 1 -- 172.21.15.178:0/3867747599 shutdown_connections 2024-09-15T01:38:38.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.702+0000 7f7625100640 1 -- 172.21.15.178:0/3867747599 wait complete. 2024-09-15T01:38:38.715 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:38:38.715 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:38:38.715 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:38:38.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.809+0000 7f9e48d9d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:38.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.809+0000 7f9e48d9d640 10 monclient: build_initial_monmap 2024-09-15T01:38:38.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.809+0000 7f9e48d9d640 10 monclient: monmap: 2024-09-15T01:38:38.811 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:38.810106+0000 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:38.810106+0000 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:38.813 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.809+0000 7f9e48d9d640 1 Processor -- start 2024-09-15T01:38:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.809+0000 7f9e48d9d640 1 -- start start 2024-09-15T01:38:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.809+0000 7f9e48d9d640 10 monclient: init 2024-09-15T01:38:38.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:38.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 10 monclient: _add_conns ranks=[9,5,3,8,0,4,10,2,1,6,7] 2024-09-15T01:38:38.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9e440a2930 0x7f9e44132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 10 monclient(hunting): picked mon.noname-a con 0x7f9e440a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:38:38.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9e4409faa0 0x7f9e4409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 10 monclient(hunting): picked mon.noname-h con 0x7f9e4409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:38.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e440a08c0 0x7f9e440a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 10 monclient(hunting): picked mon.noname-g con 0x7f9e440a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4409e0f0 con 0x7f9e440a2930 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4409dd80 con 0x7f9e440a08c0 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440a2d00 con 0x7f9e4409faa0 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e48d9d640 10 monclient(hunting): authenticate will time out at 1123.560575s 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9e440a2930 0x7f9e44132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9e440a2930 0x7f9e44132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49680/0 (socket says 172.21.15.178:49680) 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 1 -- 172.21.15.178:0/828041577 learned_addr learned my addr 172.21.15.178:0/828041577 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 10 monclient(hunting): get_auth_request con 0x7f9e440a2930 auth_method 0 2024-09-15T01:38:38.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e42d9d640 1 --2- 172.21.15.178:0/828041577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e440a08c0 0x7f9e440a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e41d9b640 1 --2- 172.21.15.178:0/828041577 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9e4409faa0 0x7f9e4409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e42d9d640 10 monclient(hunting): get_auth_request con 0x7f9e440a08c0 auth_method 0 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e42d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e42d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e42d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e41d9b640 10 monclient(hunting): get_auth_request con 0x7f9e4409faa0 auth_method 0 2024-09-15T01:38:38.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e41d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e41d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e41d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e42d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e42d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e42d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 10 monclient(hunting): handle_auth_done global_id 34155 payload 274 2024-09-15T01:38:38.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 1 -- 172.21.15.178:0/828041577 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9e4409faa0 msgr2=0x7f9e4409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:38.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 1 --2- 172.21.15.178:0/828041577 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9e4409faa0 0x7f9e4409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 1 -- 172.21.15.178:0/828041577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e440a08c0 msgr2=0x7f9e440a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:38.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 1 --2- 172.21.15.178:0/828041577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e440a08c0 0x7f9e440a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 10 monclient: _finish_hunting 0 2024-09-15T01:38:38.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 1 monclient: found mon.noname-a 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e4259c640 1 -- 172.21.15.178:0/828041577 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4404bf60 con 0x7f9e440a2930 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.810+0000 7f9e41d9b640 1 --2- 172.21.15.178:0/828041577 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9e4409faa0 0x7f9e4409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4259c640 1 --2- 172.21.15.178:0/828041577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e440a2930 0x7f9e44132de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009910 tx=0x7f9e3c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee65668f4ccedaa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4159a640 1 -- 172.21.15.178:0/828041577 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e3c02ac80 con 0x7f9e440a2930 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4159a640 10 monclient: dump: 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4159a640 10 monclient: _finish_auth 0 2024-09-15T01:38:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e48d9d640 5 monclient: authenticate success, global_id 34155 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4159a640 1 -- 172.21.15.178:0/828041577 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c02ae00 con 0x7f9e440a2930 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e48d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e48d9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e48d9d640 4 set_mon_vals no callback set 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4159a640 1 -- 172.21.15.178:0/828041577 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e3c031ca0 con 0x7f9e440a2930 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.811+0000 7f9e4159a640 10 monclient: dump: 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.828 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 10 monclient: shutdown 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 1 -- 172.21.15.178:0/828041577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e440a2930 msgr2=0x7f9e44132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 1 --2- 172.21.15.178:0/828041577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e440a2930 0x7f9e44132de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009910 tx=0x7f9e3c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 1 -- 172.21.15.178:0/828041577 shutdown_connections 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 1 --2- 172.21.15.178:0/828041577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e440a08c0 0x7f9e440a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 1 --2- 172.21.15.178:0/828041577 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9e4409faa0 0x7f9e4409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 1 --2- 172.21.15.178:0/828041577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e440a2930 0x7f9e44132de0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 1 -- 172.21.15.178:0/828041577 >> 172.21.15.178:0/828041577 conn(0x7f9e440a5ed0 msgr2=0x7f9e440a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 1 -- 172.21.15.178:0/828041577 shutdown_connections 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 1 -- 172.21.15.178:0/828041577 wait complete. 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 10 monclient: build_initial_monmap 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.812+0000 7f9e48d9d640 10 monclient: monmap: 2024-09-15T01:38:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:38.831 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 1 Processor -- start 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 1 -- start start 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 10 monclient: init 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 10 monclient: _add_conns ranks=[8,4,0,3,1,5,7,6,2] 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e4409faa0 0x7f9e4409af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 10 monclient(hunting): picked mon.noname-e con 0x7f9e4409faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9e440a08c0 0x7f9e441316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 10 monclient(hunting): picked mon.noname-c con 0x7f9e440a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e440a2930 0x7f9e4411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 10 monclient(hunting): picked mon.noname-a con 0x7f9e440a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:38:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4409d630 con 0x7f9e440a2930 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4411eae0 con 0x7f9e440a08c0 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44142a30 con 0x7f9e4409faa0 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e48d9d640 10 monclient(hunting): authenticate will time out at 1123.563764s 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e4259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e4409faa0 0x7f9e4409af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e42d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e440a2930 0x7f9e4411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e41d9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9e440a08c0 0x7f9e441316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e4259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e4409faa0 0x7f9e4409af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43876/0 (socket says 172.21.15.178:43876) 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e4259c640 1 -- 172.21.15.178:0/51357500 learned_addr learned my addr 172.21.15.178:0/51357500 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e4259c640 10 monclient(hunting): get_auth_request con 0x7f9e4409faa0 auth_method 0 2024-09-15T01:38:38.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e4259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e4259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e4259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e41d9b640 10 monclient(hunting): get_auth_request con 0x7f9e440a08c0 auth_method 0 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e41d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e41d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e41d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e42d9d640 10 monclient(hunting): get_auth_request con 0x7f9e440a2930 auth_method 0 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e42d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e42d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e42d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e42d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.813+0000 7f9e42d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e42d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e4259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e4259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e4259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e41d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e41d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e41d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e42d9d640 10 monclient(hunting): handle_auth_done global_id 34164 payload 824 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e42d9d640 1 -- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e4409faa0 msgr2=0x7f9e4409af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e42d9d640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e4409faa0 0x7f9e4409af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e42d9d640 1 -- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9e440a08c0 msgr2=0x7f9e441316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e42d9d640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9e440a08c0 0x7f9e441316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e42d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e42d9d640 1 monclient: found mon.noname-a 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e42d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e42d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e42d9d640 1 -- 172.21.15.178:0/51357500 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c009450 con 0x7f9e440a2930 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e4259c640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9e4409faa0 0x7f9e4409af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e41d9b640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9e440a08c0 0x7f9e441316e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e42d9d640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e440a2930 0x7f9e4411c530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400e900 tx=0x7f9e340090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e34009520 con 0x7f9e440a2930 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e327fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e327fc640 10 monclient: dump: 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e327fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e48d9d640 5 monclient: authenticate success, global_id 34164 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3400cdc0 con 0x7f9e440a2930 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e327fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e340138f0 con 0x7f9e440a2930 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e48d9d640 10 monclient: _renew_subs 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e48d9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e48d9d640 1 -- 172.21.15.178:0/51357500 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e440a2390 con 0x7f9e440a2930 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e40d99640 4 set_mon_vals no callback set 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e327fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e327fc640 10 monclient: dump: 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e48d9d640 10 monclient: _renew_subs 2024-09-15T01:38:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e48d9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.814+0000 7f9e48d9d640 1 -- 172.21.15.178:0/51357500 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4411d8a0 con 0x7f9e440a2930 2024-09-15T01:38:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e48d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9e34013470 con 0x7f9e440a2930 2024-09-15T01:38:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e48d9d640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44136f90 0x7f9e4411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e48d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e48d9d640 1 -- 172.21.15.178:0/51357500 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4404c4a0 con 0x7f9e44136f90 2024-09-15T01:38:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e4259c640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44136f90 0x7f9e4411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e4259c640 10 monclient: get_auth_request con 0x7f9e44136f90 auth_method 0 2024-09-15T01:38:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e4259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e4259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e4259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e4259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e4259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e4259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e327fc640 1 --2- 172.21.15.178:0/51357500 >> v2:172.21.15.204:6812/28210 conn(0x7f9e240396e0 0x7f9e2403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f9e3404c130 con 0x7f9e440a2930 2024-09-15T01:38:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e4259c640 10 monclient(hunting): handle_auth_done global_id 34173 payload 824 2024-09-15T01:38:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e4259c640 1 -- 172.21.15.178:0/51357500 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e4405ed70 con 0x7f9e44136f90 2024-09-15T01:38:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e41d9b640 1 --2- 172.21.15.178:0/51357500 >> v2:172.21.15.204:6812/28210 conn(0x7f9e240396e0 0x7f9e2403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e41d9b640 10 monclient: get_auth_request con 0x7f9e240396e0 auth_method 0 2024-09-15T01:38:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e4259c640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44136f90 0x7f9e4411ca70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c035040 tx=0x7f9e3c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e3c02ac80 con 0x7f9e44136f90 2024-09-15T01:38:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e327fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.815+0000 7f9e327fc640 10 monclient: dump: 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:38.843 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:38.843 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.816+0000 7f9e41d9b640 1 --2- 172.21.15.178:0/51357500 >> v2:172.21.15.204:6812/28210 conn(0x7f9e240396e0 0x7f9e2403bb90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9e440a6ec0 tx=0x7f9e380098a0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:38.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.816+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f9e3c031960 con 0x7f9e44136f90 2024-09-15T01:38:38.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.816+0000 7f9e327fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:38.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.816+0000 7f9e327fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:38.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.816+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44136f90 msgr2=0x7f9e4411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.816+0000 7f9e327fc640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44136f90 0x7f9e4411ca70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c035040 tx=0x7f9e3c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:38.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.816+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44136f90 msgr2=0x7f9e4411ca70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:38.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.816+0000 7f9e327fc640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44136f90 0x7f9e4411ca70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.821+0000 7f9e48d9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:38.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.821+0000 7f9e48d9d640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44131c20 0x7f9e4411cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.821+0000 7f9e48d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.821+0000 7f9e48d9d640 1 -- 172.21.15.178:0/51357500 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44131ff0 con 0x7f9e44131c20 2024-09-15T01:38:38.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.821+0000 7f9e4259c640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44131c20 0x7f9e4411cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e4259c640 10 monclient: get_auth_request con 0x7f9e44131c20 auth_method 0 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e4259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e4259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e4259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e4259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e4259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e4259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e4259c640 10 monclient(hunting): handle_auth_done global_id 34182 payload 824 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e4259c640 1 -- 172.21.15.178:0/51357500 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9e4404bc80 con 0x7f9e44131c20 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e4259c640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44131c20 0x7f9e4411cfb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009910 tx=0x7f9e3c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e3c045b60 con 0x7f9e44131c20 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e327fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e327fc640 10 monclient: dump: 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:38.848 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:38.848 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f9e3c045cc0 con 0x7f9e44131c20 2024-09-15T01:38:38.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e327fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:38.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e327fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:38.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44131c20 msgr2=0x7f9e4411cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e327fc640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44131c20 0x7f9e4411cfb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009910 tx=0x7f9e3c03d000 comp rx=0 tx=0).stop 2024-09-15T01:38:38.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e327fc640 1 -- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44131c20 msgr2=0x7f9e4411cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:38.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.822+0000 7f9e327fc640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e44131c20 0x7f9e4411cfb0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009910 tx=0x7f9e3c03d000 comp rx=0 tx=0).stop 2024-09-15T01:38:38.848 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:38.849 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:38.850 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:38.851 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:38.852 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:38.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:38.854 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:38.855 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:38.856 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:38.857 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:38.858 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:38.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.859 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:38.859 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:38.859 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:38.859 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:38.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.859 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:38.859 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:38.859 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.860 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.860 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:38.860 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:38.860 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.860 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:38.860 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:38.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.860 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:38.860 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:38.860 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.861 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:38.862 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.863 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:38.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:38.867 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:38.868 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:38.869 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:38.870 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:38.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:38.870 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:38.870 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:38.870 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:38:38.870 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:38.870 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:38.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:38.870 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:38.870 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:38.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.823+0000 7f9e48d9d640 1 -- 172.21.15.178:0/51357500 >> v2:172.21.15.204:6812/28210 conn(0x7f9e240396e0 msgr2=0x7f9e2403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.823+0000 7f9e48d9d640 1 --2- 172.21.15.178:0/51357500 >> v2:172.21.15.204:6812/28210 conn(0x7f9e240396e0 0x7f9e2403bb90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9e440a6ec0 tx=0x7f9e380098a0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.823+0000 7f9e48d9d640 10 monclient: shutdown 2024-09-15T01:38:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.823+0000 7f9e48d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.823+0000 7f9e48d9d640 1 -- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e440a2930 msgr2=0x7f9e4411c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.823+0000 7f9e48d9d640 1 --2- 172.21.15.178:0/51357500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e440a2930 0x7f9e4411c530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400e900 tx=0x7f9e340090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.823+0000 7f9e42d9d640 1 -- 172.21.15.178:0/51357500 reap_dead start 2024-09-15T01:38:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.823+0000 7f9e42d9d640 1 -- 172.21.15.178:0/51357500 reap_dead start 2024-09-15T01:38:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.823+0000 7f9e48d9d640 1 -- 172.21.15.178:0/51357500 shutdown_connections 2024-09-15T01:38:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.823+0000 7f9e48d9d640 1 -- 172.21.15.178:0/51357500 >> 172.21.15.178:0/51357500 conn(0x7f9e440a5ed0 msgr2=0x7f9e44135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.823+0000 7f9e48d9d640 1 -- 172.21.15.178:0/51357500 shutdown_connections 2024-09-15T01:38:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.823+0000 7f9e48d9d640 1 -- 172.21.15.178:0/51357500 wait complete. 2024-09-15T01:38:38.873 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:38.873 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:38:38.873 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:38:38.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.966+0000 7fdb3a8af640 10 monclient: get_monmap_and_config 2024-09-15T01:38:38.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.966+0000 7fdb3a8af640 10 monclient: build_initial_monmap 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.966+0000 7fdb3a8af640 10 monclient: monmap: 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:38.967252+0000 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:38.967252+0000 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:38.970 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:38.970 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:38.970 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:38.970 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:38.970 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 1 Processor -- start 2024-09-15T01:38:38.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 1 -- start start 2024-09-15T01:38:38.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 10 monclient: init 2024-09-15T01:38:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 10 monclient: _add_conns ranks=[4,6,7,9,10,8,1,3,2,5,0] 2024-09-15T01:38:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdb340a2930 0x7fdb34132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 10 monclient(hunting): picked mon.noname-c con 0x7fdb340a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb3409faa0 0x7fdb3409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 10 monclient(hunting): picked mon.noname-d con 0x7fdb3409faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdb340a08c0 0x7fdb340a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 10 monclient(hunting): picked mon.noname-i con 0x7fdb340a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3409e0f0 con 0x7fdb340a2930 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3409dd80 con 0x7fdb3409faa0 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdb340a2d00 con 0x7fdb340a08c0 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb3a8af640 10 monclient(hunting): authenticate will time out at 1123.717630s 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb337fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb3409faa0 0x7fdb3409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb33fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdb340a2930 0x7fdb34132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb33fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdb340a2930 0x7fdb34132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:43210/0 (socket says 172.21.15.178:43210) 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb337fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb3409faa0 0x7fdb3409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:34526/0 (socket says 172.21.15.178:34526) 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb33fff640 1 -- 172.21.15.178:0/3107812978 learned_addr learned my addr 172.21.15.178:0/3107812978 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb337fe640 10 monclient(hunting): get_auth_request con 0x7fdb3409faa0 auth_method 0 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb33fff640 10 monclient(hunting): get_auth_request con 0x7fdb340a2930 auth_method 0 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb33fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb33fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb38e4c640 1 --2- 172.21.15.178:0/3107812978 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdb340a08c0 0x7fdb340a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb38e4c640 10 monclient(hunting): get_auth_request con 0x7fdb340a08c0 auth_method 0 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb38e4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb38e4c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb38e4c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb38e4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb38e4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb38e4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.967+0000 7fdb33fff640 10 monclient(hunting): handle_auth_done global_id 34123 payload 274 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb33fff640 1 -- 172.21.15.178:0/3107812978 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdb340a08c0 msgr2=0x7fdb340a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb33fff640 1 --2- 172.21.15.178:0/3107812978 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdb340a08c0 0x7fdb340a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb33fff640 1 -- 172.21.15.178:0/3107812978 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb3409faa0 msgr2=0x7fdb3409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb33fff640 1 --2- 172.21.15.178:0/3107812978 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb3409faa0 0x7fdb3409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb33fff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb33fff640 1 monclient: found mon.noname-c 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb33fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:38.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb33fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb33fff640 1 -- 172.21.15.178:0/3107812978 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3404bf60 con 0x7fdb340a2930 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb33fff640 1 --2- 172.21.15.178:0/3107812978 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdb340a2930 0x7fdb34132de0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009940 tx=0x7fdb240091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=58d62b2245fc8132 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb32ffd640 1 -- 172.21.15.178:0/3107812978 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb2402ac60 con 0x7fdb340a2930 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb32ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb32ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb32ffd640 10 monclient: dump: 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.987 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb32ffd640 10 monclient: _finish_auth 0 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb32ffd640 1 -- 172.21.15.178:0/3107812978 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2402ade0 con 0x7fdb340a2930 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb3a8af640 5 monclient: authenticate success, global_id 34123 2024-09-15T01:38:38.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb3a8af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb32ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb3a8af640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb32ffd640 1 -- 172.21.15.178:0/3107812978 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb24031ca0 con 0x7fdb340a2930 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb3a8af640 4 set_mon_vals no callback set 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb32ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb32ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.968+0000 7fdb32ffd640 10 monclient: dump: 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 10 monclient: shutdown 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3107812978 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdb340a2930 msgr2=0x7fdb34132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 1 --2- 172.21.15.178:0/3107812978 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdb340a2930 0x7fdb34132de0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009940 tx=0x7fdb240091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3107812978 shutdown_connections 2024-09-15T01:38:38.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 1 --2- 172.21.15.178:0/3107812978 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdb340a08c0 0x7fdb340a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 1 --2- 172.21.15.178:0/3107812978 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb3409faa0 0x7fdb3409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 1 --2- 172.21.15.178:0/3107812978 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fdb340a2930 0x7fdb34132de0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3107812978 >> 172.21.15.178:0/3107812978 conn(0x7fdb340a5ed0 msgr2=0x7fdb340a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3107812978 shutdown_connections 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3107812978 wait complete. 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 10 monclient: build_initial_monmap 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.969+0000 7fdb3a8af640 10 monclient: monmap: 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:38.991 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:38.992 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:38.992 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:38.992 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:38.992 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:38.992 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:38.992 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:38.992 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:38.992 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:38.992 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 1 Processor -- start 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 1 -- start start 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 10 monclient: init 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 10 monclient: _add_conns ranks=[1,6,8,7,4,2,0,5,3] 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3409faa0 0x7fdb3409b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 10 monclient(hunting): picked mon.noname-f con 0x7fdb3409faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb340a08c0 0x7fdb341317a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 10 monclient(hunting): picked mon.noname-d con 0x7fdb340a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdb340a2930 0x7fdb3411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 10 monclient(hunting): picked mon.noname-e con 0x7fdb340a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb340a2390 con 0x7fdb3409faa0 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3409d630 con 0x7fdb340a08c0 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdb34142b40 con 0x7fdb340a2930 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb3a8af640 10 monclient(hunting): authenticate will time out at 1123.720903s 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.970+0000 7fdb337fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb340a08c0 0x7fdb341317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb38e4c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdb340a2930 0x7fdb3411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb340a08c0 0x7fdb341317a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:34532/0 (socket says 172.21.15.178:34532) 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 1 -- 172.21.15.178:0/3631906118 learned_addr learned my addr 172.21.15.178:0/3631906118 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 10 monclient(hunting): get_auth_request con 0x7fdb340a08c0 auth_method 0 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb33fff640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3409faa0 0x7fdb3409b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb38e4c640 10 monclient(hunting): get_auth_request con 0x7fdb340a2930 auth_method 0 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb38e4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb38e4c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb38e4c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb33fff640 10 monclient(hunting): get_auth_request con 0x7fdb3409faa0 auth_method 0 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb33fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb33fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb38e4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb38e4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb38e4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 10 monclient(hunting): handle_auth_done global_id 24729 payload 824 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 1 -- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdb340a2930 msgr2=0x7fdb3411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdb340a2930 0x7fdb3411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 1 -- 172.21.15.178:0/3631906118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3409faa0 msgr2=0x7fdb3409b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3409faa0 0x7fdb3409b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 10 monclient: _finish_hunting 0 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 1 monclient: found mon.noname-d 2024-09-15T01:38:38.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 1 -- 172.21.15.178:0/3631906118 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb24009450 con 0x7fdb340a08c0 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb33fff640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb3409faa0 0x7fdb3409b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb38e4c640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdb340a2930 0x7fdb3411c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb337fe640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb340a08c0 0x7fdb341317a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800e8d0 tx=0x7fdb280090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb28009480 con 0x7fdb340a08c0 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb1bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb1bfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb1bfff640 10 monclient: dump: 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb1bfff640 10 monclient: _finish_auth 0 2024-09-15T01:38:38.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb1bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2800cd60 con 0x7fdb340a08c0 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb3a8af640 5 monclient: authenticate success, global_id 24729 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb1bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb280138f0 con 0x7fdb340a08c0 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb1bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb1bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb1bfff640 10 monclient: dump: 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:38.999 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb3a8af640 10 monclient: _renew_subs 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb3a8af640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3631906118 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3409bc20 con 0x7fdb340a08c0 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb327fc640 4 set_mon_vals no callback set 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb3a8af640 10 monclient: _renew_subs 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb3a8af640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.971+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3631906118 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3411eba0 con 0x7fdb340a08c0 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb3a8af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb3a8af640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb341370f0 0x7fdb3411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb3a8af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3631906118 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3404c4a0 con 0x7fdb341370f0 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdb28013470 con 0x7fdb340a08c0 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb33fff640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb341370f0 0x7fdb3411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb33fff640 10 monclient: get_auth_request con 0x7fdb341370f0 auth_method 0 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb33fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb33fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb1bfff640 1 --2- 172.21.15.178:0/3631906118 >> v2:172.21.15.204:6812/28210 conn(0x7fdb200397b0 0x7fdb2003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fdb2804bde0 con 0x7fdb340a08c0 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb33fff640 10 monclient(hunting): handle_auth_done global_id 34112 payload 824 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.972+0000 7fdb33fff640 1 -- 172.21.15.178:0/3631906118 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb3405ed70 con 0x7fdb341370f0 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb38e4c640 1 --2- 172.21.15.178:0/3631906118 >> v2:172.21.15.204:6812/28210 conn(0x7fdb200397b0 0x7fdb2003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb38e4c640 10 monclient: get_auth_request con 0x7fdb200397b0 auth_method 0 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb33fff640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb341370f0 0x7fdb3411ca70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009910 tx=0x7fdb24032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb24031ea0 con 0x7fdb341370f0 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb1bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb1bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb1bfff640 10 monclient: dump: 2024-09-15T01:38:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.003 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb38e4c640 1 --2- 172.21.15.178:0/3631906118 >> v2:172.21.15.204:6812/28210 conn(0x7fdb200397b0 0x7fdb2003bc60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdb3409b810 tx=0x7fdb2c00b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fdb2402ac60 con 0x7fdb341370f0 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb1bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb1bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb341370f0 msgr2=0x7fdb3411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb1bfff640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb341370f0 0x7fdb3411ca70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009910 tx=0x7fdb24032000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb341370f0 msgr2=0x7fdb3411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.973+0000 7fdb1bfff640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb341370f0 0x7fdb3411ca70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb3a8af640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb3a8af640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb34131ce0 0x7fdb3411cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb3a8af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3631906118 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3404bc80 con 0x7fdb34131ce0 2024-09-15T01:38:39.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb33fff640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb34131ce0 0x7fdb3411cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb33fff640 10 monclient: get_auth_request con 0x7fdb34131ce0 auth_method 0 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb33fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb33fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb33fff640 10 monclient(hunting): handle_auth_done global_id 34121 payload 824 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb33fff640 1 -- 172.21.15.178:0/3631906118 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdb3411eed0 con 0x7fdb34131ce0 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb33fff640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb34131ce0 0x7fdb3411cfb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009940 tx=0x7fdb2403d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb2403c8c0 con 0x7fdb34131ce0 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb1bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb1bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb1bfff640 10 monclient: dump: 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.006 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.007 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.007 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7fdb2403c440 con 0x7fdb34131ce0 2024-09-15T01:38:39.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb1bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:39.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.979+0000 7fdb1bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:39.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb34131ce0 msgr2=0x7fdb3411cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb1bfff640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb34131ce0 0x7fdb3411cfb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009940 tx=0x7fdb2403d000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb1bfff640 1 -- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb34131ce0 msgr2=0x7fdb3411cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb1bfff640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb34131ce0 0x7fdb3411cfb0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009940 tx=0x7fdb2403d000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.007 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:39.012 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.013 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.014 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:39.015 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.016 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.017 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:39.019 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.020 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.021 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:39.022 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.023 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.024 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.025 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.026 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.027 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.028 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.029 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:39.030 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.030 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.030 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:39.030 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:39.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3631906118 >> v2:172.21.15.204:6812/28210 conn(0x7fdb200397b0 msgr2=0x7fdb2003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb3a8af640 1 --2- 172.21.15.178:0/3631906118 >> v2:172.21.15.204:6812/28210 conn(0x7fdb200397b0 0x7fdb2003bc60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdb3409b810 tx=0x7fdb2c00b040 comp rx=0 tx=0).stop 2024-09-15T01:38:39.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb3a8af640 10 monclient: shutdown 2024-09-15T01:38:39.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb3a8af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb340a08c0 msgr2=0x7fdb341317a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb3a8af640 1 --2- 172.21.15.178:0/3631906118 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdb340a08c0 0x7fdb341317a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800e8d0 tx=0x7fdb280090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb38e4c640 1 -- 172.21.15.178:0/3631906118 reap_dead start 2024-09-15T01:38:39.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb38e4c640 1 -- 172.21.15.178:0/3631906118 reap_dead start 2024-09-15T01:38:39.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3631906118 shutdown_connections 2024-09-15T01:38:39.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3631906118 >> 172.21.15.178:0/3631906118 conn(0x7fdb340a5ed0 msgr2=0x7fdb3409e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3631906118 shutdown_connections 2024-09-15T01:38:39.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:38.980+0000 7fdb3a8af640 1 -- 172.21.15.178:0/3631906118 wait complete. 2024-09-15T01:38:39.032 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:39.033 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:38:39.033 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:38:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.125+0000 7fa9ab39d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.125+0000 7fa9ab39d640 10 monclient: build_initial_monmap 2024-09-15T01:38:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.125+0000 7fa9ab39d640 10 monclient: monmap: 2024-09-15T01:38:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:39.126757+0000 2024-09-15T01:38:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:39.126757+0000 2024-09-15T01:38:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:39.129 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:39.129 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:39.129 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:39.129 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:39.129 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:39.129 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:39.129 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:39.129 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:39.129 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:39.129 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:39.129 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.126+0000 7fa9ab39d640 1 Processor -- start 2024-09-15T01:38:39.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.126+0000 7fa9ab39d640 1 -- start start 2024-09-15T01:38:39.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.126+0000 7fa9ab39d640 10 monclient: init 2024-09-15T01:38:39.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9ab39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:39.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9ab39d640 10 monclient: _add_conns ranks=[0,10,1,7,5,9,8,4,6,3,2] 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9ab39d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa9a40a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9ab39d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa9a4132de0 0x7fa9a4137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9ab39d640 10 monclient(hunting): picked mon.noname-f con 0x7fa9a4132de0 addr v2:172.21.15.204:3300/0 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9ab39d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa9a41380e0 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9ab39d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a404bf60 con 0x7fa9a40a2930 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9ab39d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a409dd80 con 0x7fa9a41380e0 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9ab39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9ab39d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a40a2d00 con 0x7fa9a4132de0 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9ab39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9ab39d640 10 monclient(hunting): authenticate will time out at 1123.877223s 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a9139640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa9a40a2930 0x7fa9a409faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:50814/0 (socket says 172.21.15.178:50814) 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a9139640 1 -- 172.21.15.178:0/1123912325 learned_addr learned my addr 172.21.15.178:0/1123912325 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:38:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 1 --2- 172.21.15.178:0/1123912325 >> v2:172.21.15.204:3300/0 conn(0x7fa9a4132de0 0x7fa9a4137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 10 monclient(hunting): get_auth_request con 0x7fa9a4132de0 auth_method 0 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a3fff640 1 -- 172.21.15.178:0/1123912325 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607480272 0 0) 0x7fa9a404bf60 con 0x7fa9a40a2930 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a3fff640 10 monclient(hunting): my global_id is 34191 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a3fff640 1 -- 172.21.15.178:0/1123912325 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa98c0034e0 con 0x7fa9a40a2930 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a3fff640 1 -- 172.21.15.178:0/1123912325 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128095991 0 0) 0x7fa9a409dd80 con 0x7fa9a41380e0 2024-09-15T01:38:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a3fff640 10 monclient(hunting): my global_id is 7624 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a3fff640 1 -- 172.21.15.178:0/1123912325 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9a404bf60 con 0x7fa9a41380e0 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 10 monclient(hunting): handle_auth_done global_id 7615 payload 274 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 1 -- 172.21.15.178:0/1123912325 >> v1:172.21.15.204:6789/0 conn(0x7fa9a41380e0 legacy=0x7fa9a413a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 1 -- 172.21.15.178:0/1123912325 >> v1:172.21.15.178:6789/0 conn(0x7fa9a40a2930 legacy=0x7fa9a409faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 10 monclient: _finish_hunting 0 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 1 monclient: found mon.noname-f 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.127+0000 7fa9a8938640 1 -- 172.21.15.178:0/1123912325 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a41331b0 con 0x7fa9a4132de0 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a8938640 1 --2- 172.21.15.178:0/1123912325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a4132de0 0x7fa9a4137ba0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa998009850 tx=0x7fa9980090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ba2db00f32687f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a3fff640 1 -- 172.21.15.178:0/1123912325 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa99802ad00 con 0x7fa9a4132de0 2024-09-15T01:38:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a3fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a3fff640 10 monclient: dump: 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.146 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a3fff640 10 monclient: _finish_auth 0 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a3fff640 1 -- 172.21.15.178:0/1123912325 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99802ae80 con 0x7fa9a4132de0 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9ab39d640 5 monclient: authenticate success, global_id 7615 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9ab39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a3fff640 1 -- 172.21.15.178:0/1123912325 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa998031ca0 con 0x7fa9a4132de0 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9ab39d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9ab39d640 4 set_mon_vals no callback set 2024-09-15T01:38:39.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a3fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.128+0000 7fa9a3fff640 10 monclient: dump: 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.129+0000 7fa9ab39d640 10 monclient: shutdown 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.129+0000 7fa9ab39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.129+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1123912325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a4132de0 msgr2=0x7fa9a4137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.129+0000 7fa9ab39d640 1 --2- 172.21.15.178:0/1123912325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a4132de0 0x7fa9a4137ba0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa998009850 tx=0x7fa9980090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.129+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1123912325 shutdown_connections 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.129+0000 7fa9ab39d640 1 --2- 172.21.15.178:0/1123912325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a4132de0 0x7fa9a4137ba0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.129+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1123912325 >> 172.21.15.178:0/1123912325 conn(0x7fa9a40a5ed0 msgr2=0x7fa9a40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.129+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1123912325 shutdown_connections 2024-09-15T01:38:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.129+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1123912325 wait complete. 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.129+0000 7fa9ab39d640 10 monclient: build_initial_monmap 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.129+0000 7fa9ab39d640 10 monclient: monmap: 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:39.150 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 1 Processor -- start 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 1 -- start start 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 10 monclient: init 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 10 monclient: _add_conns ranks=[1,2,7,8,3,0,5,6,4] 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a40a2930 0x7fa9a409a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 10 monclient(hunting): picked mon.noname-f con 0x7fa9a40a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9a4132de0 0x7fa9a409ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 10 monclient(hunting): picked mon.noname-b con 0x7fa9a4132de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa9a41380e0 0x7fa9a409b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 10 monclient(hunting): picked mon.noname-i con 0x7fa9a41380e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a411d890 con 0x7fa9a40a2930 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a409c1a0 con 0x7fa9a4132de0 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a411d5b0 con 0x7fa9a41380e0 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9ab39d640 10 monclient(hunting): authenticate will time out at 1123.880718s 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a9139640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a40a2930 0x7fa9a409a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a8938640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9a4132de0 0x7fa9a409ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a9139640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a40a2930 0x7fa9a409a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:52910/0 (socket says 172.21.15.178:52910) 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a9139640 1 -- 172.21.15.178:0/1921011480 learned_addr learned my addr 172.21.15.178:0/1921011480 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:39.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a9139640 10 monclient(hunting): get_auth_request con 0x7fa9a40a2930 auth_method 0 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a8938640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9a4132de0 0x7fa9a409ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42928/0 (socket says 172.21.15.178:42928) 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a9139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a9139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a9139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a8938640 10 monclient(hunting): get_auth_request con 0x7fa9a4132de0 auth_method 0 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a8938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a8938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a8938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a993a640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa9a41380e0 0x7fa9a409b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a993a640 10 monclient(hunting): get_auth_request con 0x7fa9a41380e0 auth_method 0 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a993a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a993a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a993a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a8938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a8938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a8938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a9139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a9139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.130+0000 7fa9a9139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a993a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a993a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a993a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a8938640 10 monclient(hunting): handle_auth_done global_id 34130 payload 824 2024-09-15T01:38:39.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a8938640 1 -- 172.21.15.178:0/1921011480 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa9a41380e0 msgr2=0x7fa9a409b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a8938640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa9a41380e0 0x7fa9a409b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a8938640 1 -- 172.21.15.178:0/1921011480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a40a2930 msgr2=0x7fa9a409a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a8938640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a40a2930 0x7fa9a409a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a8938640 10 monclient: _finish_hunting 0 2024-09-15T01:38:39.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a8938640 1 monclient: found mon.noname-b 2024-09-15T01:38:39.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a8938640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:39.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a8938640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:38:39.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a8938640 1 -- 172.21.15.178:0/1921011480 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa998009390 con 0x7fa9a4132de0 2024-09-15T01:38:39.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a9139640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a40a2930 0x7fa9a409a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:39.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a8938640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9a4132de0 0x7fa9a409ab90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa998031000 tx=0x7fa998032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa99803c880 con 0x7fa9a4132de0 2024-09-15T01:38:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:38:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a17fa640 10 monclient: dump: 2024-09-15T01:38:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a17fa640 10 monclient: _finish_auth 0 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9ab39d640 5 monclient: authenticate success, global_id 34130 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99803ca20 con 0x7fa9a4132de0 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9ab39d640 10 monclient: _renew_subs 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9ab39d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:39.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1921011480 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a411db70 con 0x7fa9a4132de0 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa99803cb90 con 0x7fa9a4132de0 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a17fa640 10 monclient: dump: 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.158 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9a37fe640 4 set_mon_vals no callback set 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9ab39d640 10 monclient: _renew_subs 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9ab39d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.131+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1921011480 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a411e410 con 0x7fa9a4132de0 2024-09-15T01:38:39.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9ab39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa99803cd10 con 0x7fa9a4132de0 2024-09-15T01:38:39.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9ab39d640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a411c650 0x7fa9a409b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9ab39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1921011480 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a411dd00 con 0x7fa9a411c650 2024-09-15T01:38:39.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a9139640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a411c650 0x7fa9a409b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a9139640 10 monclient: get_auth_request con 0x7fa9a411c650 auth_method 0 2024-09-15T01:38:39.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a9139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a9139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a9139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a9139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a17fa640 1 --2- 172.21.15.178:0/1921011480 >> v2:172.21.15.204:6812/28210 conn(0x7fa9900397b0 0x7fa99003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a9139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a9139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fa99806ba20 con 0x7fa9a4132de0 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a9139640 10 monclient(hunting): handle_auth_done global_id 34132 payload 824 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a9139640 1 -- 172.21.15.178:0/1921011480 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9a405ed70 con 0x7fa9a411c650 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a993a640 1 --2- 172.21.15.178:0/1921011480 >> v2:172.21.15.204:6812/28210 conn(0x7fa9900397b0 0x7fa99003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.132+0000 7fa9a993a640 10 monclient: get_auth_request con 0x7fa9900397b0 auth_method 0 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a9139640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a411c650 0x7fa9a409b610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa994005bc0 tx=0x7fa994009040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa994007550 con 0x7fa9a411c650 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a17fa640 10 monclient: dump: 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.161 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.162 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a993a640 1 --2- 172.21.15.178:0/1921011480 >> v2:172.21.15.204:6812/28210 conn(0x7fa9900397b0 0x7fa99003bc60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa9a409bdc0 tx=0x7fa99c007c60 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fa9940070d0 con 0x7fa9a411c650 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a411c650 msgr2=0x7fa9a409b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a17fa640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a411c650 0x7fa9a409b610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa994005bc0 tx=0x7fa994009040 comp rx=0 tx=0).stop 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a411c650 msgr2=0x7fa9a409b610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.133+0000 7fa9a17fa640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a411c650 0x7fa9a409b610 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa994005bc0 tx=0x7fa994009040 comp rx=0 tx=0).stop 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9ab39d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9ab39d640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a4096a30 0x7fa9a4096e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9ab39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1921011480 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a40a6460 con 0x7fa9a4096a30 2024-09-15T01:38:39.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a9139640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a4096a30 0x7fa9a4096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a9139640 10 monclient: get_auth_request con 0x7fa9a4096a30 auth_method 0 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a9139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a9139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a9139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a9139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a9139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a9139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a9139640 10 monclient(hunting): handle_auth_done global_id 34141 payload 824 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a9139640 1 -- 172.21.15.178:0/1921011480 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa9a404bc80 con 0x7fa9a4096a30 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a9139640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a4096a30 0x7fa9a4096e30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa994005bc0 tx=0x7fa994013000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa994013ea0 con 0x7fa9a4096a30 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a17fa640 10 monclient: dump: 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.165 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.166 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.166 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.166 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.166 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7fa99400f9e0 con 0x7fa9a4096a30 2024-09-15T01:38:39.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:39.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a17fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:39.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a4096a30 msgr2=0x7fa9a4096e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a17fa640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a4096a30 0x7fa9a4096e30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa994005bc0 tx=0x7fa994013000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a17fa640 1 -- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a4096a30 msgr2=0x7fa9a4096e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.139+0000 7fa9a17fa640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa9a4096a30 0x7fa9a4096e30 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa994005bc0 tx=0x7fa994013000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.167 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:39.167 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:39.167 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:39.167 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:39.167 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:39.167 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:39.167 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:39.167 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:39.167 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:39.167 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.168 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:39.170 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.171 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:39.172 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:39.173 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.173 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.173 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.173 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.173 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:39.173 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.173 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.173 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.173 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.173 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:39.173 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.173 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.174 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.175 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:39.176 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.177 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.178 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:39.179 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.180 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.181 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.182 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.183 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.184 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.185 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.186 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.187 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:39.188 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.189 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.189 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:39.189 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.189 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.189 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.189 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:39.189 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.189 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.189 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:39.189 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:39.191 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=0 2024-09-15T01:38:39.191 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 0 -ne 0 ]] 2024-09-15T01:38:39.191 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:38:39.191 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls' 0 2024-09-15T01:38:39.191 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls' 2024-09-15T01:38:39.191 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=0 2024-09-15T01:38:39.192 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls 2024-09-15T01:38:39.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.140+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1921011480 >> v2:172.21.15.204:6812/28210 conn(0x7fa9900397b0 msgr2=0x7fa99003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.140+0000 7fa9ab39d640 1 --2- 172.21.15.178:0/1921011480 >> v2:172.21.15.204:6812/28210 conn(0x7fa9900397b0 0x7fa99003bc60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa9a409bdc0 tx=0x7fa99c007c60 comp rx=0 tx=0).stop 2024-09-15T01:38:39.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.140+0000 7fa9ab39d640 10 monclient: shutdown 2024-09-15T01:38:39.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.140+0000 7fa9ab39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.140+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9a4132de0 msgr2=0x7fa9a409ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.140+0000 7fa9ab39d640 1 --2- 172.21.15.178:0/1921011480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9a4132de0 0x7fa9a409ab90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa998031000 tx=0x7fa998032000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.140+0000 7fa9a993a640 1 -- 172.21.15.178:0/1921011480 reap_dead start 2024-09-15T01:38:39.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.140+0000 7fa9a993a640 1 -- 172.21.15.178:0/1921011480 reap_dead start 2024-09-15T01:38:39.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.140+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1921011480 shutdown_connections 2024-09-15T01:38:39.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.140+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1921011480 >> 172.21.15.178:0/1921011480 conn(0x7fa9a40a5ed0 msgr2=0x7fa9a40a48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.140+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1921011480 shutdown_connections 2024-09-15T01:38:39.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.140+0000 7fa9ab39d640 1 -- 172.21.15.178:0/1921011480 wait complete. 2024-09-15T01:38:39.193 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:39.193 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:38:39.194 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:38:39.194 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls 2024-09-15T01:38:39.194 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls 2024-09-15T01:38:39.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.285+0000 7fbabc309640 10 monclient: get_monmap_and_config 2024-09-15T01:38:39.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.285+0000 7fbabc309640 10 monclient: build_initial_monmap 2024-09-15T01:38:39.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.285+0000 7fbabc309640 10 monclient: monmap: 2024-09-15T01:38:39.287 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:39.287 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:39.286242+0000 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:39.286242+0000 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:39.288 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:39.289 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:39.289 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:39.289 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.285+0000 7fbabc309640 1 Processor -- start 2024-09-15T01:38:39.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 1 -- start start 2024-09-15T01:38:39.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 10 monclient: init 2024-09-15T01:38:39.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:39.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 10 monclient: _add_conns ranks=[8,5,9,7,10,6,1,4,2,3,0] 2024-09-15T01:38:39.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbab40a2930 0x7fbab4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 10 monclient(hunting): picked mon.noname-e con 0x7fbab40a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbab409faa0 0x7fbab409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 10 monclient(hunting): picked mon.noname-h con 0x7fbab409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbab40a08c0 0x7fbab40a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 10 monclient(hunting): picked mon.noname-a con 0x7fbab40a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab409e0f0 con 0x7fbab40a08c0 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbab409dd80 con 0x7fbab409faa0 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40a2d00 con 0x7fbab40a2930 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbabc309640 10 monclient(hunting): authenticate will time out at 1124.036685s 2024-09-15T01:38:39.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba8a6640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbab40a08c0 0x7fbab40a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba0a5640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbab40a2930 0x7fbab4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba0a5640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbab40a2930 0x7fbab4132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43882/0 (socket says 172.21.15.178:43882) 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba8a6640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbab40a08c0 0x7fbab40a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49716/0 (socket says 172.21.15.178:49716) 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba0a5640 1 -- 172.21.15.178:0/3834244903 learned_addr learned my addr 172.21.15.178:0/3834244903 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba0a5640 10 monclient(hunting): get_auth_request con 0x7fbab40a2930 auth_method 0 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba0a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba0a5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba0a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba8a6640 10 monclient(hunting): get_auth_request con 0x7fbab40a08c0 auth_method 0 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba8a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba8a6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba8a6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbab98a4640 1 --2- 172.21.15.178:0/3834244903 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbab409faa0 0x7fbab409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba0a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba0a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba0a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbab98a4640 10 monclient(hunting): get_auth_request con 0x7fbab409faa0 auth_method 0 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbab98a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbab98a4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbab98a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba8a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.286+0000 7fbaba8a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba8a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbab98a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbab98a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbab98a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba0a5640 10 monclient(hunting): handle_auth_done global_id 34136 payload 274 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba0a5640 1 -- 172.21.15.178:0/3834244903 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbab409faa0 msgr2=0x7fbab409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba0a5640 1 --2- 172.21.15.178:0/3834244903 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbab409faa0 0x7fbab409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba0a5640 1 -- 172.21.15.178:0/3834244903 >> v2:172.21.15.178:3300/0 conn(0x7fbab40a08c0 msgr2=0x7fbab40a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba0a5640 1 --2- 172.21.15.178:0/3834244903 >> v2:172.21.15.178:3300/0 conn(0x7fbab40a08c0 0x7fbab40a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba0a5640 10 monclient: _finish_hunting 0 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba0a5640 1 monclient: found mon.noname-e 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba0a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba0a5640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba0a5640 1 -- 172.21.15.178:0/3834244903 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab404bf60 con 0x7fbab40a2930 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba8a6640 1 --2- 172.21.15.178:0/3834244903 >> v2:172.21.15.178:3300/0 conn(0x7fbab40a08c0 0x7fbab40a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbaba0a5640 1 --2- 172.21.15.178:0/3834244903 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbab40a2930 0x7fbab4132de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009910 tx=0x7fbab00091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=39ff1e50d042f534 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.287+0000 7fbab90a3640 1 -- 172.21.15.178:0/3834244903 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbab002ac80 con 0x7fbab40a2930 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbab90a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbab90a3640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbab90a3640 10 monclient: dump: 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.303 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbab90a3640 10 monclient: _finish_auth 0 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbab90a3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbab90a3640 1 -- 172.21.15.178:0/3834244903 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab002ae00 con 0x7fbab40a2930 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 5 monclient: authenticate success, global_id 34136 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbab90a3640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:39.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:39.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbab90a3640 1 -- 172.21.15.178:0/3834244903 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbab0031ca0 con 0x7fbab40a2930 2024-09-15T01:38:39.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbab90a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 4 set_mon_vals no callback set 2024-09-15T01:38:39.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbab90a3640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:38:39.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbab90a3640 10 monclient: dump: 2024-09-15T01:38:39.305 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.305 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.305 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.305 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.305 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.305 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.305 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 10 monclient: shutdown 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 1 -- 172.21.15.178:0/3834244903 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbab40a2930 msgr2=0x7fbab4132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 1 --2- 172.21.15.178:0/3834244903 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbab40a2930 0x7fbab4132de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009910 tx=0x7fbab00091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 1 -- 172.21.15.178:0/3834244903 shutdown_connections 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 1 --2- 172.21.15.178:0/3834244903 >> v2:172.21.15.178:3300/0 conn(0x7fbab40a08c0 0x7fbab40a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 1 --2- 172.21.15.178:0/3834244903 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbab409faa0 0x7fbab409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 1 --2- 172.21.15.178:0/3834244903 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbab40a2930 0x7fbab4132de0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 1 -- 172.21.15.178:0/3834244903 >> 172.21.15.178:0/3834244903 conn(0x7fbab40a5ed0 msgr2=0x7fbab40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 1 -- 172.21.15.178:0/3834244903 shutdown_connections 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.288+0000 7fbabc309640 1 -- 172.21.15.178:0/3834244903 wait complete. 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient: build_initial_monmap 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient: monmap: 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:39.307 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 1 Processor -- start 2024-09-15T01:38:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 1 -- start start 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient: init 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient: _add_conns ranks=[6,5,3,7,1,2,0,8,4] 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab409faa0 0x7fbab409b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient(hunting): picked mon.noname-d con 0x7fbab409faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbab40a08c0 0x7fbab4131990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient(hunting): picked mon.noname-h con 0x7fbab40a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbab40a2930 0x7fbab4141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient(hunting): picked mon.noname-g con 0x7fbab40a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40a2390 con 0x7fbab40a2930 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbab409d630 con 0x7fbab40a08c0 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbab411ee20 con 0x7fbab409faa0 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.289+0000 7fbabc309640 10 monclient(hunting): authenticate will time out at 1124.039950s 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab409faa0 0x7fbab409b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab409faa0 0x7fbab409b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:34546/0 (socket says 172.21.15.178:34546) 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 1 -- 172.21.15.178:0/1725782467 learned_addr learned my addr 172.21.15.178:0/1725782467 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 10 monclient(hunting): get_auth_request con 0x7fbab409faa0 auth_method 0 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbab98a4640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbab40a08c0 0x7fbab4131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbab98a4640 10 monclient(hunting): get_auth_request con 0x7fbab40a08c0 auth_method 0 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbab98a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbab98a4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbab98a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba8a6640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbab40a2930 0x7fbab4141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba8a6640 10 monclient(hunting): get_auth_request con 0x7fbab40a2930 auth_method 0 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba8a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba8a6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba8a6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbab98a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbab98a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbab98a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba8a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba8a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba8a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 10 monclient(hunting): handle_auth_done global_id 24738 payload 824 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 1 -- 172.21.15.178:0/1725782467 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbab40a08c0 msgr2=0x7fbab4131990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbab40a08c0 0x7fbab4131990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 1 -- 172.21.15.178:0/1725782467 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbab40a2930 msgr2=0x7fbab4141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbab40a2930 0x7fbab4141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 10 monclient: _finish_hunting 0 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 1 monclient: found mon.noname-d 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:38:39.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 1 -- 172.21.15.178:0/1725782467 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac009a20 con 0x7fbab409faa0 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaba0a5640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab409faa0 0x7fbab409b540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbab0000c00 tx=0x7fbab0032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbab0031820 con 0x7fbab409faa0 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaaa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaaa7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaaa7fc640 10 monclient: dump: 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.313 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaaa7fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaaa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbabc309640 5 monclient: authenticate success, global_id 24738 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab00319e0 con 0x7fbab409faa0 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbabc309640 10 monclient: _renew_subs 2024-09-15T01:38:39.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbabc309640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbabc309640 1 -- 172.21.15.178:0/1725782467 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab0009450 con 0x7fbab409faa0 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaaa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbab003a390 con 0x7fbab409faa0 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaaa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaaa7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbaaa7fc640 10 monclient: dump: 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.315 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbab88a2640 4 set_mon_vals no callback set 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbabc309640 10 monclient: _renew_subs 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbabc309640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:39.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.290+0000 7fbabc309640 1 -- 172.21.15.178:0/1725782467 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab411d6e0 con 0x7fbab409faa0 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fbab0039040 con 0x7fbab409faa0 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbabc309640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbabc309640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab411ca00 0x7fbab41423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbabc309640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbabc309640 1 -- 172.21.15.178:0/1725782467 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbab411ef90 con 0x7fbab411ca00 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbab98a4640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab411ca00 0x7fbab41423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbab98a4640 10 monclient: get_auth_request con 0x7fbab411ca00 auth_method 0 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbab98a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbab98a4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbab98a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbaaa7fc640 1 --2- 172.21.15.178:0/1725782467 >> v2:172.21.15.204:6812/28210 conn(0x7fba980396e0 0x7fba9803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbab98a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbab98a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbab98a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fbab006afc0 con 0x7fbab409faa0 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbab98a4640 10 monclient(hunting): handle_auth_done global_id 24747 payload 824 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbab98a4640 1 -- 172.21.15.178:0/1725782467 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbab404c4a0 con 0x7fbab411ca00 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbaba8a6640 1 --2- 172.21.15.178:0/1725782467 >> v2:172.21.15.204:6812/28210 conn(0x7fba980396e0 0x7fba9803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.291+0000 7fbaba8a6640 10 monclient: get_auth_request con 0x7fba980396e0 auth_method 0 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbab98a4640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab411ca00 0x7fbab41423c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbab409d340 tx=0x7fbaa40090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbaa400fc50 con 0x7fbab411ca00 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbaaa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbaaa7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbaaa7fc640 10 monclient: dump: 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.318 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.319 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbaba8a6640 1 --2- 172.21.15.178:0/1725782467 >> v2:172.21.15.204:6812/28210 conn(0x7fba980396e0 0x7fba9803bb90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00f700 tx=0x7fbaac00b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fbaa400fdb0 con 0x7fbab411ca00 2024-09-15T01:38:39.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbaaa7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbaaa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:39.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab411ca00 msgr2=0x7fbab41423c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbaaa7fc640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab411ca00 0x7fbab41423c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbab409d340 tx=0x7fbaa40090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab411ca00 msgr2=0x7fbab41423c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.292+0000 7fbaaa7fc640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab411ca00 0x7fbab41423c0 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbab409d340 tx=0x7fbaa40090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbabc309640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:39.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbabc309640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab4131ed0 0x7fbab4142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbabc309640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbabc309640 1 -- 172.21.15.178:0/1725782467 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbab411f550 con 0x7fbab4131ed0 2024-09-15T01:38:39.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbab98a4640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab4131ed0 0x7fbab4142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbab98a4640 10 monclient: get_auth_request con 0x7fbab4131ed0 auth_method 0 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbab98a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbab98a4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbab98a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbab98a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbab98a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbab98a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbab98a4640 10 monclient(hunting): handle_auth_done global_id 24756 payload 824 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbab98a4640 1 -- 172.21.15.178:0/1725782467 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbab404bc80 con 0x7fbab4131ed0 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbab98a4640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab4131ed0 0x7fbab4142900 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000c00 tx=0x7fbaa401c000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbaa4024040 con 0x7fbab4131ed0 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbaaa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbaaa7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.298+0000 7fbaaa7fc640 10 monclient: dump: 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.324 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.325 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.325 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.325 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.325 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7fbaa40133a0 con 0x7fbab4131ed0 2024-09-15T01:38:39.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbaaa7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:39.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbaaa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:39.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab4131ed0 msgr2=0x7fbab4142900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbaaa7fc640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab4131ed0 0x7fbab4142900 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000c00 tx=0x7fbaa401c000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbaaa7fc640 1 -- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab4131ed0 msgr2=0x7fbab4142900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbaaa7fc640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab4131ed0 0x7fbab4142900 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000c00 tx=0x7fbaa401c000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.326 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:39.326 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:39.326 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:39.326 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:39.326 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:39.326 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:39.326 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:39.326 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:39.326 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:39.326 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.327 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.328 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:39.329 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.330 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.331 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.331 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.331 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.331 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.331 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.331 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.331 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.331 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.331 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:39.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.331 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:39.331 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.332 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.333 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.334 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.335 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.336 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.337 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:39.338 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.339 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.340 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.341 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.342 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.343 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.344 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.345 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.346 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.347 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.348 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:39.348 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.348 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.348 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.348 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:38:39.348 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.348 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.348 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:39.348 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:39.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbabc309640 1 -- 172.21.15.178:0/1725782467 >> v2:172.21.15.204:6812/28210 conn(0x7fba980396e0 msgr2=0x7fba9803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbabc309640 1 --2- 172.21.15.178:0/1725782467 >> v2:172.21.15.204:6812/28210 conn(0x7fba980396e0 0x7fba9803bb90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00f700 tx=0x7fbaac00b040 comp rx=0 tx=0).stop 2024-09-15T01:38:39.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbabc309640 10 monclient: shutdown 2024-09-15T01:38:39.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbabc309640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbabc309640 1 -- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab409faa0 msgr2=0x7fbab409b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbabc309640 1 --2- 172.21.15.178:0/1725782467 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbab409faa0 0x7fbab409b540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbab0000c00 tx=0x7fbab0032000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbaba8a6640 1 -- 172.21.15.178:0/1725782467 reap_dead start 2024-09-15T01:38:39.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbaba8a6640 1 -- 172.21.15.178:0/1725782467 reap_dead start 2024-09-15T01:38:39.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbabc309640 1 -- 172.21.15.178:0/1725782467 shutdown_connections 2024-09-15T01:38:39.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbabc309640 1 -- 172.21.15.178:0/1725782467 >> 172.21.15.178:0/1725782467 conn(0x7fbab40a5ed0 msgr2=0x7fbab40a5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbabc309640 1 -- 172.21.15.178:0/1725782467 shutdown_connections 2024-09-15T01:38:39.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.299+0000 7fbabc309640 1 -- 172.21.15.178:0/1725782467 wait complete. 2024-09-15T01:38:39.351 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:39.351 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:38:39.351 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:38:39.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.442+0000 7f66a8c20640 10 monclient: get_monmap_and_config 2024-09-15T01:38:39.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.442+0000 7f66a8c20640 10 monclient: build_initial_monmap 2024-09-15T01:38:39.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.442+0000 7f66a8c20640 10 monclient: monmap: 2024-09-15T01:38:39.444 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:39.443128+0000 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:39.443128+0000 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:39.446 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:39.446 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:39.446 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.442+0000 7f66a8c20640 1 Processor -- start 2024-09-15T01:38:39.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.442+0000 7f66a8c20640 1 -- start start 2024-09-15T01:38:39.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.442+0000 7f66a8c20640 10 monclient: init 2024-09-15T01:38:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 10 monclient: _add_conns ranks=[9,1,4,2,10,8,3,6,5,0,7] 2024-09-15T01:38:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f66a4067130 0x7f66a4067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 10 monclient(hunting): picked mon.noname-a con 0x7f66a4067130 addr v2:172.21.15.178:3300/0 2024-09-15T01:38:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f66a409d580 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a409f000 0x7f66a4067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 10 monclient(hunting): picked mon.noname-c con 0x7f66a409f000 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a404bf60 con 0x7f66a409d580 2024-09-15T01:38:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a409e0c0 con 0x7f66a4067130 2024-09-15T01:38:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66a4067f80 con 0x7f66a409f000 2024-09-15T01:38:39.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:39.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a8c20640 10 monclient(hunting): authenticate will time out at 1124.193519s 2024-09-15T01:38:39.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f66a4067130 0x7f66a4067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a2d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a409f000 0x7f66a4067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a2d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a409f000 0x7f66a4067a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:43230/0 (socket says 172.21.15.178:43230) 2024-09-15T01:38:39.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f66a4067130 0x7f66a4067500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49718/0 (socket says 172.21.15.178:49718) 2024-09-15T01:38:39.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a2d9d640 1 -- 172.21.15.178:0/2556783970 learned_addr learned my addr 172.21.15.178:0/2556783970 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:39.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a2d9d640 10 monclient(hunting): get_auth_request con 0x7f66a409f000 auth_method 0 2024-09-15T01:38:39.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 10 monclient(hunting): get_auth_request con 0x7f66a4067130 auth_method 0 2024-09-15T01:38:39.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 10 monclient(hunting): handle_auth_done global_id 34209 payload 274 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 1 -- 172.21.15.178:0/2556783970 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a409f000 msgr2=0x7f66a4067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 1 --2- 172.21.15.178:0/2556783970 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a409f000 0x7f66a4067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 1 -- 172.21.15.178:0/2556783970 >> v1:172.21.15.204:6789/0 conn(0x7f66a409d580 legacy=0x7f66a412d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 10 monclient: _finish_hunting 0 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 1 monclient: found mon.noname-a 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a259c640 1 -- 172.21.15.178:0/2556783970 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a409dde0 con 0x7f66a4067130 2024-09-15T01:38:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.443+0000 7f66a2d9d640 1 --2- 172.21.15.178:0/2556783970 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a409f000 0x7f66a4067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a259c640 1 --2- 172.21.15.178:0/2556783970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a4067130 0x7f66a4067500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f669c009850 tx=0x7f669c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42779410678b9e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a159a640 1 -- 172.21.15.178:0/2556783970 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f669c02ac80 con 0x7f66a4067130 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a159a640 10 monclient: dump: 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a159a640 10 monclient: _finish_auth 0 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a159a640 1 -- 172.21.15.178:0/2556783970 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669c02ae00 con 0x7f66a4067130 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a8c20640 5 monclient: authenticate success, global_id 34209 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a8c20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a8c20640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a159a640 1 -- 172.21.15.178:0/2556783970 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f669c031ca0 con 0x7f66a4067130 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a8c20640 4 set_mon_vals no callback set 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.444+0000 7f66a159a640 10 monclient: dump: 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.463 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.463 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.463 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.463 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.463 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.464 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.464 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.464 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.464 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.445+0000 7f66a8c20640 10 monclient: shutdown 2024-09-15T01:38:39.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.445+0000 7f66a8c20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.445+0000 7f66a8c20640 1 -- 172.21.15.178:0/2556783970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a4067130 msgr2=0x7f66a4067500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.445+0000 7f66a8c20640 1 --2- 172.21.15.178:0/2556783970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a4067130 0x7f66a4067500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f669c009850 tx=0x7f669c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.445+0000 7f66a8c20640 1 -- 172.21.15.178:0/2556783970 shutdown_connections 2024-09-15T01:38:39.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.445+0000 7f66a8c20640 1 --2- 172.21.15.178:0/2556783970 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f66a409f000 0x7f66a4067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.445+0000 7f66a8c20640 1 --2- 172.21.15.178:0/2556783970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a4067130 0x7f66a4067500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.445+0000 7f66a8c20640 1 -- 172.21.15.178:0/2556783970 >> 172.21.15.178:0/2556783970 conn(0x7f66a4068a60 msgr2=0x7f66a40a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.445+0000 7f66a8c20640 1 -- 172.21.15.178:0/2556783970 shutdown_connections 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.445+0000 7f66a8c20640 1 -- 172.21.15.178:0/2556783970 wait complete. 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.445+0000 7f66a8c20640 10 monclient: build_initial_monmap 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.445+0000 7f66a8c20640 10 monclient: monmap: 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 1 Processor -- start 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 1 -- start start 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 10 monclient: init 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 10 monclient: _add_conns ranks=[5,0,8,2,3,1,4,6,7] 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a4067130 0x7f66a413fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 10 monclient(hunting): picked mon.noname-h con 0x7f66a4067130 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a409d580 0x7f66a41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 10 monclient(hunting): picked mon.noname-a con 0x7f66a409d580 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a409f000 0x7f66a411d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 10 monclient(hunting): picked mon.noname-e con 0x7f66a409f000 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a409cb50 con 0x7f66a409d580 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66a409d9e0 con 0x7f66a4067130 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f66a411f0c0 con 0x7f66a409f000 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:39.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a8c20640 10 monclient(hunting): authenticate will time out at 1124.196770s 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a1d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a409d580 0x7f66a41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a2d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a409f000 0x7f66a411d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a1d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a409d580 0x7f66a41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49732/0 (socket says 172.21.15.178:49732) 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a259c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a4067130 0x7f66a413fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a1d9b640 1 -- 172.21.15.178:0/4075660302 learned_addr learned my addr 172.21.15.178:0/4075660302 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a1d9b640 10 monclient(hunting): get_auth_request con 0x7f66a409d580 auth_method 0 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a2d9d640 10 monclient(hunting): get_auth_request con 0x7f66a409f000 auth_method 0 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a259c640 10 monclient(hunting): get_auth_request con 0x7f66a4067130 auth_method 0 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.446+0000 7f66a2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a1d9b640 10 monclient(hunting): handle_auth_done global_id 34218 payload 824 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a1d9b640 1 -- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a409f000 msgr2=0x7f66a411d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a1d9b640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a409f000 0x7f66a411d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a1d9b640 1 -- 172.21.15.178:0/4075660302 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a4067130 msgr2=0x7f66a413fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a1d9b640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f66a4067130 0x7f66a413fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a1d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a1d9b640 1 monclient: found mon.noname-a 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a1d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a1d9b640 1 -- 172.21.15.178:0/4075660302 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6694009ae0 con 0x7f66a409d580 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a1d9b640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a409d580 0x7f66a41339f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f668c00c560 tx=0x7f668c007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f668c011a10 con 0x7f66a409d580 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f669a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f669a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:38:39.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f669a7fc640 10 monclient: dump: 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.471 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f669a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f669a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c011bd0 con 0x7f66a409d580 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a8c20640 5 monclient: authenticate success, global_id 34218 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a8c20640 10 monclient: _renew_subs 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a8c20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a8c20640 1 -- 172.21.15.178:0/4075660302 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669c009390 con 0x7f66a409d580 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f669a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f668c011d80 con 0x7f66a409d580 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f669a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f669a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f669a7fc640 10 monclient: dump: 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a0d99640 4 set_mon_vals no callback set 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a8c20640 10 monclient: _renew_subs 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a8c20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.447+0000 7f66a8c20640 1 -- 172.21.15.178:0/4075660302 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a4120df0 con 0x7f66a409d580 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a8c20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f668c011590 con 0x7f66a409d580 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a8c20640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a41400a0 0x7f66a411da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a8c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a8c20640 1 -- 172.21.15.178:0/4075660302 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f66a4140470 con 0x7f66a41400a0 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a259c640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a41400a0 0x7f66a411da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a259c640 10 monclient: get_auth_request con 0x7f66a41400a0 auth_method 0 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f669a7fc640 1 --2- 172.21.15.178:0/4075660302 >> v2:172.21.15.204:6812/28210 conn(0x7f66840397e0 0x7f668403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f668c04aba0 con 0x7f66a409d580 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a259c640 10 monclient(hunting): handle_auth_done global_id 34154 payload 824 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a259c640 1 -- 172.21.15.178:0/4075660302 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66a404c4a0 con 0x7f66a41400a0 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a2d9d640 1 --2- 172.21.15.178:0/4075660302 >> v2:172.21.15.204:6812/28210 conn(0x7f66840397e0 0x7f668403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a2d9d640 10 monclient: get_auth_request con 0x7f66840397e0 auth_method 0 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f66a259c640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a41400a0 0x7f66a411da90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66a40a3520 tx=0x7f669c032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f669c02ac80 con 0x7f66a41400a0 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f669a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f669a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.448+0000 7f669a7fc640 10 monclient: dump: 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.476 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.449+0000 7f66a2d9d640 1 --2- 172.21.15.178:0/4075660302 >> v2:172.21.15.204:6812/28210 conn(0x7f66840397e0 0x7f668403bc90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f66a411ec10 tx=0x7f669400b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.449+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f669c0319b0 con 0x7f66a41400a0 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.449+0000 7f669a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.449+0000 7f669a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.449+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a41400a0 msgr2=0x7f66a411da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.449+0000 7f669a7fc640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a41400a0 0x7f66a411da90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66a40a3520 tx=0x7f669c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.449+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a41400a0 msgr2=0x7f66a411da90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.449+0000 7f669a7fc640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a41400a0 0x7f66a411da90 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66a40a3520 tx=0x7f669c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.454+0000 7f66a8c20640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:39.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.454+0000 7f66a8c20640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a4133f30 0x7f66a411dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.454+0000 7f66a8c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.454+0000 7f66a8c20640 1 -- 172.21.15.178:0/4075660302 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f66a40641f0 con 0x7f66a4133f30 2024-09-15T01:38:39.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f66a259c640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a4133f30 0x7f66a411dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f66a259c640 10 monclient: get_auth_request con 0x7f66a4133f30 auth_method 0 2024-09-15T01:38:39.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f66a259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f66a259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f66a259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f66a259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f66a259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f66a259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f66a259c640 10 monclient(hunting): handle_auth_done global_id 34163 payload 824 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f66a259c640 1 -- 172.21.15.178:0/4075660302 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f66a404bc80 con 0x7f66a4133f30 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f66a259c640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a4133f30 0x7f66a411dfd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f669c009720 tx=0x7f669c03d000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f669c04b070 con 0x7f66a4133f30 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f669a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f669a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f669a7fc640 10 monclient: dump: 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.481 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f669c031620 con 0x7f66a4133f30 2024-09-15T01:38:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f669a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:39.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f669a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:39.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a4133f30 msgr2=0x7f66a411dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f669a7fc640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a4133f30 0x7f66a411dfd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f669c009720 tx=0x7f669c03d000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f669a7fc640 1 -- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a4133f30 msgr2=0x7f66a411dfd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.455+0000 7f669a7fc640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f66a4133f30 0x7f66a411dfd0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f669c009720 tx=0x7f669c03d000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.483 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:39.483 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:39.483 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:39.483 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.484 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.485 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.486 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:39.487 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.488 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.489 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.490 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.491 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.492 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.493 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.494 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.495 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:39.496 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.497 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.498 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.499 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.500 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:39.501 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.502 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.503 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.504 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.505 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:39.505 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.505 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.505 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.505 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:39.505 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.505 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.505 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.505 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:39.505 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.456+0000 7f66a8c20640 1 -- 172.21.15.178:0/4075660302 >> v2:172.21.15.204:6812/28210 conn(0x7f66840397e0 msgr2=0x7f668403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.456+0000 7f66a8c20640 1 --2- 172.21.15.178:0/4075660302 >> v2:172.21.15.204:6812/28210 conn(0x7f66840397e0 0x7f668403bc90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f66a411ec10 tx=0x7f669400b040 comp rx=0 tx=0).stop 2024-09-15T01:38:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.456+0000 7f66a8c20640 10 monclient: shutdown 2024-09-15T01:38:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.456+0000 7f66a8c20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.456+0000 7f66a8c20640 1 -- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a409d580 msgr2=0x7f66a41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.456+0000 7f66a8c20640 1 --2- 172.21.15.178:0/4075660302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a409d580 0x7f66a41339f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f668c00c560 tx=0x7f668c007eb0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.456+0000 7f66a2d9d640 1 -- 172.21.15.178:0/4075660302 reap_dead start 2024-09-15T01:38:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.456+0000 7f66a2d9d640 1 -- 172.21.15.178:0/4075660302 reap_dead start 2024-09-15T01:38:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.456+0000 7f66a8c20640 1 -- 172.21.15.178:0/4075660302 shutdown_connections 2024-09-15T01:38:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.456+0000 7f66a8c20640 1 -- 172.21.15.178:0/4075660302 >> 172.21.15.178:0/4075660302 conn(0x7f66a4068a60 msgr2=0x7f66a40a0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.456+0000 7f66a8c20640 1 -- 172.21.15.178:0/4075660302 shutdown_connections 2024-09-15T01:38:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.456+0000 7f66a8c20640 1 -- 172.21.15.178:0/4075660302 wait complete. 2024-09-15T01:38:39.508 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:39.508 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:38:39.508 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:38:39.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.600+0000 7f408ed9d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:39.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.600+0000 7f408ed9d640 10 monclient: build_initial_monmap 2024-09-15T01:38:39.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.600+0000 7f408ed9d640 10 monclient: monmap: 2024-09-15T01:38:39.602 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:39.602 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.602 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:39.600956+0000 2024-09-15T01:38:39.602 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:39.600956+0000 2024-09-15T01:38:39.602 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:39.602 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.602 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:39.602 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:39.602 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:39.602 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:39.603 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:39.603 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:39.603 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:39.603 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:39.603 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:39.603 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:39.603 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:39.603 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.600+0000 7f408ed9d640 1 Processor -- start 2024-09-15T01:38:39.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.600+0000 7f408ed9d640 1 -- start start 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.600+0000 7f408ed9d640 10 monclient: init 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 10 monclient: _add_conns ranks=[1,7,10,4,0,6,2,8,5,3,9] 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f40880678d0 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f408806ea30 0x7f408809ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 10 monclient(hunting): picked mon.noname-i con 0x7f408806ea30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f40881359e0 0x7f4088137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 10 monclient(hunting): picked mon.noname-f con 0x7f40881359e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408804bf60 con 0x7f40880678d0 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408806d5b0 con 0x7f40881359e0 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4088067ca0 con 0x7f408806ea30 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408ed9d640 10 monclient(hunting): authenticate will time out at 1124.351328s 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f408806ea30 0x7f408809ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408d33a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f40881359e0 0x7f4088137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408d33a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f40881359e0 0x7f4088137dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:52912/0 (socket says 172.21.15.178:52912) 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408cb39640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f40880678d0 0x7f408809e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:55116/0 (socket says 172.21.15.178:55116) 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408d33a640 1 -- 172.21.15.178:0/2562827847 learned_addr learned my addr 172.21.15.178:0/2562827847 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f408806ea30 0x7f408809ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:46212/0 (socket says 172.21.15.178:46212) 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408d33a640 10 monclient(hunting): get_auth_request con 0x7f40881359e0 auth_method 0 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408d33a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408d33a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408d33a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 10 monclient(hunting): get_auth_request con 0x7f408806ea30 auth_method 0 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408d33a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408d33a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408d33a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 10 monclient(hunting): handle_auth_done global_id 24892 payload 274 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 1 -- 172.21.15.178:0/2562827847 >> v2:172.21.15.204:3300/0 conn(0x7f40881359e0 msgr2=0x7f4088137dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 1 --2- 172.21.15.178:0/2562827847 >> v2:172.21.15.204:3300/0 conn(0x7f40881359e0 0x7f4088137dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 1 -- 172.21.15.178:0/2562827847 >> v1:172.21.15.204:6789/0 conn(0x7f40880678d0 legacy=0x7f408809e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 1 monclient: found mon.noname-i 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407f7fe640 1 -- 172.21.15.178:0/2562827847 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46032220 0 0) 0x7f408804bf60 con 0x7f40880678d0 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407ffff640 1 -- 172.21.15.178:0/2562827847 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408806d890 con 0x7f408806ea30 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f407f7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.601+0000 7f408d33a640 1 --2- 172.21.15.178:0/2562827847 >> v2:172.21.15.204:3300/0 conn(0x7f40881359e0 0x7f4088137dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407ffff640 1 --2- 172.21.15.178:0/2562827847 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f408806ea30 0x7f408809ee10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f407800dc70 tx=0x7f407800d5b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=a33832456ffa8b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407f7fe640 1 -- 172.21.15.178:0/2562827847 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f407800d2e0 con 0x7f408806ea30 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407f7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407f7fe640 10 monclient: dump: 2024-09-15T01:38:39.616 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407f7fe640 1 -- 172.21.15.178:0/2562827847 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4078031050 con 0x7f408806ea30 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f408ed9d640 5 monclient: authenticate success, global_id 24892 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f408ed9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f408ed9d640 4 set_mon_vals no callback set 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407f7fe640 1 -- 172.21.15.178:0/2562827847 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f407800b8d0 con 0x7f408806ea30 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407f7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.602+0000 7f407f7fe640 10 monclient: dump: 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.619 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.603+0000 7f408ed9d640 10 monclient: shutdown 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.603+0000 7f408ed9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.603+0000 7f408ed9d640 1 -- 172.21.15.178:0/2562827847 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f408806ea30 msgr2=0x7f408809ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.603+0000 7f408ed9d640 1 --2- 172.21.15.178:0/2562827847 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f408806ea30 0x7f408809ee10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f407800dc70 tx=0x7f407800d5b0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.603+0000 7f408ed9d640 1 -- 172.21.15.178:0/2562827847 shutdown_connections 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.603+0000 7f408ed9d640 1 --2- 172.21.15.178:0/2562827847 >> v2:172.21.15.204:3300/0 conn(0x7f40881359e0 0x7f4088137dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.603+0000 7f408ed9d640 1 --2- 172.21.15.178:0/2562827847 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f408806ea30 0x7f408809ee10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.603+0000 7f408ed9d640 1 -- 172.21.15.178:0/2562827847 >> 172.21.15.178:0/2562827847 conn(0x7f40880a4390 msgr2=0x7f4088067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.603+0000 7f408ed9d640 1 -- 172.21.15.178:0/2562827847 shutdown_connections 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.603+0000 7f408ed9d640 1 -- 172.21.15.178:0/2562827847 wait complete. 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.603+0000 7f408ed9d640 10 monclient: build_initial_monmap 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.603+0000 7f408ed9d640 10 monclient: monmap: 2024-09-15T01:38:39.620 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:39.621 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 1 Processor -- start 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 1 -- start start 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 10 monclient: init 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 10 monclient: _add_conns ranks=[2,5,4,3,1,7,0,6,8] 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40880678d0 0x7f408809a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 10 monclient(hunting): picked mon.noname-b con 0x7f40880678d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f408806ea30 0x7f4088131e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 10 monclient(hunting): picked mon.noname-h con 0x7f408806ea30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f40881359e0 0x7f4088132350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 10 monclient(hunting): picked mon.noname-c con 0x7f40881359e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:39.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408806ccd0 con 0x7f40880678d0 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f408806c900 con 0x7f40881359e0 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40880970a0 con 0x7f408806ea30 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408ed9d640 10 monclient(hunting): authenticate will time out at 1124.354722s 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408cb39640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40880678d0 0x7f408809a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408d33a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f40881359e0 0x7f4088132350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408cb39640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40880678d0 0x7f408809a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42934/0 (socket says 172.21.15.178:42934) 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408cb39640 1 -- 172.21.15.178:0/1709444995 learned_addr learned my addr 172.21.15.178:0/1709444995 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408cb39640 10 monclient(hunting): get_auth_request con 0x7f40880678d0 auth_method 0 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408cb39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408cb39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408cb39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f407ffff640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f408806ea30 0x7f4088131e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408d33a640 10 monclient(hunting): get_auth_request con 0x7f40881359e0 auth_method 0 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408d33a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408d33a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408d33a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f407ffff640 10 monclient(hunting): get_auth_request con 0x7f408806ea30 auth_method 0 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f407ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f407ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f407ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408cb39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408cb39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408cb39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408d33a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408d33a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f408d33a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f407ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.604+0000 7f407ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408cb39640 10 monclient(hunting): handle_auth_done global_id 34148 payload 824 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408cb39640 1 -- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f408806ea30 msgr2=0x7f4088131e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408cb39640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f408806ea30 0x7f4088131e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408cb39640 1 -- 172.21.15.178:0/1709444995 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f40881359e0 msgr2=0x7f4088132350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408cb39640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f40881359e0 0x7f4088132350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408cb39640 10 monclient: _finish_hunting 0 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408cb39640 1 monclient: found mon.noname-b 2024-09-15T01:38:39.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408cb39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408cb39640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408cb39640 1 -- 172.21.15.178:0/1709444995 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4080009ae0 con 0x7f40880678d0 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408d33a640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f40881359e0 0x7f4088132350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408cb39640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40880678d0 0x7f408809a320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f408400c610 tx=0x7f4084007d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40840047a0 con 0x7f40880678d0 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407cff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407cff9640 10 monclient: dump: 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.626 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407cff9640 10 monclient: _finish_auth 0 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4084004940 con 0x7f40880678d0 2024-09-15T01:38:39.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408ed9d640 5 monclient: authenticate success, global_id 34148 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408ed9d640 10 monclient: _renew_subs 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408ed9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408ed9d640 1 -- 172.21.15.178:0/1709444995 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407800d830 con 0x7f40880678d0 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40840113a0 con 0x7f40880678d0 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407cff9640 10 monclient: dump: 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.628 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408ed9d640 10 monclient: _renew_subs 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408ed9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:39.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f407effd640 4 set_mon_vals no callback set 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.605+0000 7f408ed9d640 1 -- 172.21.15.178:0/1709444995 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4088133830 con 0x7f40880678d0 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4084007300 con 0x7f40880678d0 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408ed9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f407cff9640 1 --2- 172.21.15.178:0/1709444995 >> v2:172.21.15.204:6812/28210 conn(0x7f40740397b0 0x7f407403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408ed9d640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4088134400 0x7f4088132890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408ed9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408ed9d640 1 -- 172.21.15.178:0/1709444995 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408804c4a0 con 0x7f4088134400 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f408404ac80 con 0x7f40880678d0 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f407ffff640 1 --2- 172.21.15.178:0/1709444995 >> v2:172.21.15.204:6812/28210 conn(0x7f40740397b0 0x7f407403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f407ffff640 10 monclient: get_auth_request con 0x7f40740397b0 auth_method 0 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408d33a640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4088134400 0x7f4088132890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408d33a640 10 monclient: get_auth_request con 0x7f4088134400 auth_method 0 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408d33a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408d33a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408d33a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408d33a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408d33a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408d33a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408d33a640 10 monclient(hunting): handle_auth_done global_id 7678 payload 824 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.606+0000 7f408d33a640 1 -- 172.21.15.178:0/1709444995 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f408804bc80 con 0x7f4088134400 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f407ffff640 1 --2- 172.21.15.178:0/1709444995 >> v2:172.21.15.204:6812/28210 conn(0x7f40740397b0 0x7f407403bc60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4078037000 tx=0x7f407803a000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f408d33a640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4088134400 0x7f4088132890 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f408806aa20 tx=0x7f408000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4080011950 con 0x7f4088134400 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f407cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f407cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f407cff9640 10 monclient: dump: 2024-09-15T01:38:39.631 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.632 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.633 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f408000f040 con 0x7f4088134400 2024-09-15T01:38:39.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f407cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f407cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:39.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4088134400 msgr2=0x7f4088132890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f407cff9640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4088134400 0x7f4088132890 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f408806aa20 tx=0x7f408000b040 comp rx=0 tx=0).stop 2024-09-15T01:38:39.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4088134400 msgr2=0x7f4088132890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.607+0000 7f407cff9640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4088134400 0x7f4088132890 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.634 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=0 2024-09-15T01:38:39.634 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 0 -ne 0 ]] 2024-09-15T01:38:39.634 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:38:39.634 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export' 13 2024-09-15T01:38:39.635 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export' 2024-09-15T01:38:39.635 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=13 2024-09-15T01:38:39.635 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export 2024-09-15T01:38:39.636 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export 2024-09-15T01:38:39.636 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export 2024-09-15T01:38:39.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408ed9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:39.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408ed9d640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f408809b0d0 0x7f408809b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408ed9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408ed9d640 1 -- 172.21.15.178:0/1709444995 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408805ed70 con 0x7f408809b0d0 2024-09-15T01:38:39.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408d33a640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f408809b0d0 0x7f408809b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408d33a640 10 monclient: get_auth_request con 0x7f408809b0d0 auth_method 0 2024-09-15T01:38:39.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408d33a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408d33a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408d33a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408d33a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408d33a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408d33a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408d33a640 10 monclient(hunting): handle_auth_done global_id 7687 payload 824 2024-09-15T01:38:39.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408d33a640 1 -- 172.21.15.178:0/1709444995 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f40881353b0 con 0x7f408809b0d0 2024-09-15T01:38:39.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f408d33a640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f408809b0d0 0x7f408809b4d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f408001b000 tx=0x7f408001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4080023070 con 0x7f408809b0d0 2024-09-15T01:38:39.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f407cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f407cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:38:39.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.613+0000 7f407cff9640 10 monclient: dump: 2024-09-15T01:38:39.639 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.639 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.639 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.639 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.639 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.639 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.639 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.639 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.639 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.639 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.639 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.640 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.640 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.640 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.640 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.640 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9311 (secure 0 0 0) 0x7f4080015430 con 0x7f408809b0d0 2024-09-15T01:38:39.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f407cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:39.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f407cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:39.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f408809b0d0 msgr2=0x7f408809b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f407cff9640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f408809b0d0 0x7f408809b4d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f408001b000 tx=0x7f408001c000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f407cff9640 1 -- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f408809b0d0 msgr2=0x7f408809b4d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f407cff9640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f408809b0d0 0x7f408809b4d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.641 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:39.641 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:39.641 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:39.641 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:39.641 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:39.641 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:39.641 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:39.641 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:39.641 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:39.641 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:39.641 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:39.641 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:39.642 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:39.642 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:39.642 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:39.642 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.642 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:38:39.642 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.642 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:39.642 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:39.642 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.644 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.644 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.645 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.645 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.645 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.645 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.645 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.645 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.645 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.645 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:39.645 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:39.646 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.646 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.646 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.646 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.646 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.646 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.647 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.648 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.649 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.650 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:39.651 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:39.653 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.654 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.655 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.656 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.657 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.658 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:39.659 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.661 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.662 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.663 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:38:39.665 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.666 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.666 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:39.666 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.666 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.666 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.666 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:39.666 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.666 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.666 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:39.666 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:39.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f408ed9d640 1 -- 172.21.15.178:0/1709444995 >> v2:172.21.15.204:6812/28210 conn(0x7f40740397b0 msgr2=0x7f407403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f408ed9d640 1 --2- 172.21.15.178:0/1709444995 >> v2:172.21.15.204:6812/28210 conn(0x7f40740397b0 0x7f407403bc60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4078037000 tx=0x7f407803a000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f408ed9d640 10 monclient: shutdown 2024-09-15T01:38:39.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f408ed9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f408ed9d640 1 -- 172.21.15.178:0/1709444995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40880678d0 msgr2=0x7f408809a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f408ed9d640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40880678d0 0x7f408809a320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f408400c610 tx=0x7f4084007d00 comp rx=0 tx=0).stop 2024-09-15T01:38:39.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f408d33a640 1 -- 172.21.15.178:0/1709444995 reap_dead start 2024-09-15T01:38:39.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f408ed9d640 1 -- 172.21.15.178:0/1709444995 shutdown_connections 2024-09-15T01:38:39.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f408ed9d640 1 --2- 172.21.15.178:0/1709444995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40880678d0 0x7f408809a320 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f408ed9d640 1 -- 172.21.15.178:0/1709444995 >> 172.21.15.178:0/1709444995 conn(0x7f40880a4390 msgr2=0x7f4088069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f408ed9d640 1 -- 172.21.15.178:0/1709444995 shutdown_connections 2024-09-15T01:38:39.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.614+0000 7f408ed9d640 1 -- 172.21.15.178:0/1709444995 wait complete. 2024-09-15T01:38:39.669 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:39.669 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:38:39.669 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:38:39.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.762+0000 7fea5a99d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:39.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.762+0000 7fea5a99d640 10 monclient: build_initial_monmap 2024-09-15T01:38:39.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.762+0000 7fea5a99d640 10 monclient: monmap: 2024-09-15T01:38:39.764 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:39.764 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.764 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:39.763194+0000 2024-09-15T01:38:39.764 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:39.763194+0000 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:39.765 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 1 Processor -- start 2024-09-15T01:38:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 1 -- start start 2024-09-15T01:38:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 10 monclient: init 2024-09-15T01:38:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 10 monclient: _add_conns ranks=[9,5,8,6,2,7,1,10,0,3,4] 2024-09-15T01:38:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea540a2930 0x7fea54132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 10 monclient(hunting): picked mon.noname-a con 0x7fea540a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:38:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea5409faa0 0x7fea5409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 10 monclient(hunting): picked mon.noname-h con 0x7fea5409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea540a08c0 0x7fea540a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 10 monclient(hunting): picked mon.noname-e con 0x7fea540a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea5409e0f0 con 0x7fea540a2930 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea5409dd80 con 0x7fea5409faa0 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fea540a2d00 con 0x7fea540a08c0 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea5a99d640 10 monclient(hunting): authenticate will time out at 1124.513649s 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea53fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea540a2930 0x7fea54132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea58f3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea540a08c0 0x7fea540a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea53fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea540a2930 0x7fea54132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49734/0 (socket says 172.21.15.178:49734) 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea53fff640 1 -- 172.21.15.178:0/1917901089 learned_addr learned my addr 172.21.15.178:0/1917901089 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea53fff640 10 monclient(hunting): get_auth_request con 0x7fea540a2930 auth_method 0 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea53fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea53fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea58f3a640 10 monclient(hunting): get_auth_request con 0x7fea540a08c0 auth_method 0 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea58f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea58f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea58f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea537fe640 1 --2- 172.21.15.178:0/1917901089 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea5409faa0 0x7fea5409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea537fe640 1 -- 172.21.15.178:0/1917901089 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea5409faa0 msgr2=0x7fea5409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea537fe640 1 -- 172.21.15.178:0/1917901089 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea5409faa0 msgr2=0x7fea5409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea58f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea58f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea537fe640 1 --2- 172.21.15.178:0/1917901089 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea5409faa0 0x7fea5409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea58f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.763+0000 7fea537fe640 1 --2- 172.21.15.178:0/1917901089 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea5409faa0 0x7fea5409fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea53fff640 10 monclient(hunting): handle_auth_done global_id 34227 payload 274 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea53fff640 1 -- 172.21.15.178:0/1917901089 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea540a08c0 msgr2=0x7fea540a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea53fff640 1 --2- 172.21.15.178:0/1917901089 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea540a08c0 0x7fea540a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea53fff640 1 -- 172.21.15.178:0/1917901089 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea5409faa0 msgr2=0x7fea5409fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea53fff640 1 --2- 172.21.15.178:0/1917901089 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea5409faa0 0x7fea5409fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea53fff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea53fff640 1 monclient: found mon.noname-a 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea53fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea53fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea53fff640 1 -- 172.21.15.178:0/1917901089 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5404bf60 con 0x7fea540a2930 2024-09-15T01:38:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea58f3a640 1 --2- 172.21.15.178:0/1917901089 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea540a08c0 0x7fea540a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea53fff640 1 --2- 172.21.15.178:0/1917901089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea540a2930 0x7fea54132de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009910 tx=0x7fea3c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcfa9ecedfd1c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea52ffd640 1 -- 172.21.15.178:0/1917901089 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea3c02ac80 con 0x7fea540a2930 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea52ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea52ffd640 10 monclient: dump: 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.784 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea52ffd640 10 monclient: _finish_auth 0 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea52ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea52ffd640 1 -- 172.21.15.178:0/1917901089 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c02ae00 con 0x7fea540a2930 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea5a99d640 5 monclient: authenticate success, global_id 34227 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea5a99d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:39.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea52ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea5a99d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea52ffd640 1 -- 172.21.15.178:0/1917901089 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea3c031ca0 con 0x7fea540a2930 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea5a99d640 4 set_mon_vals no callback set 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea52ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.764+0000 7fea52ffd640 10 monclient: dump: 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.765+0000 7fea5a99d640 10 monclient: shutdown 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.765+0000 7fea5a99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.765+0000 7fea5a99d640 1 -- 172.21.15.178:0/1917901089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea540a2930 msgr2=0x7fea54132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.765+0000 7fea5a99d640 1 --2- 172.21.15.178:0/1917901089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea540a2930 0x7fea54132de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009910 tx=0x7fea3c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.765+0000 7fea5a99d640 1 -- 172.21.15.178:0/1917901089 shutdown_connections 2024-09-15T01:38:39.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.765+0000 7fea5a99d640 1 --2- 172.21.15.178:0/1917901089 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea540a08c0 0x7fea540a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.765+0000 7fea5a99d640 1 --2- 172.21.15.178:0/1917901089 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea5409faa0 0x7fea5409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.765+0000 7fea5a99d640 1 --2- 172.21.15.178:0/1917901089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea540a2930 0x7fea54132de0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.765+0000 7fea5a99d640 1 -- 172.21.15.178:0/1917901089 >> 172.21.15.178:0/1917901089 conn(0x7fea540a5ed0 msgr2=0x7fea540a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.765+0000 7fea5a99d640 1 -- 172.21.15.178:0/1917901089 shutdown_connections 2024-09-15T01:38:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.765+0000 7fea5a99d640 1 -- 172.21.15.178:0/1917901089 wait complete. 2024-09-15T01:38:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.765+0000 7fea5a99d640 10 monclient: build_initial_monmap 2024-09-15T01:38:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 10 monclient: monmap: 2024-09-15T01:38:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 1 Processor -- start 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 1 -- start start 2024-09-15T01:38:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 10 monclient: init 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 10 monclient: _add_conns ranks=[1,8,6,7,5,3,2,4,0] 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea5409faa0 0x7fea5409b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 10 monclient(hunting): picked mon.noname-f con 0x7fea5409faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea540a08c0 0x7fea54131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 10 monclient(hunting): picked mon.noname-e con 0x7fea540a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea540a2930 0x7fea5411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 10 monclient(hunting): picked mon.noname-d con 0x7fea540a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5411ec20 con 0x7fea5409faa0 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea5411e940 con 0x7fea540a2930 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fea5411ef00 con 0x7fea540a08c0 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.766+0000 7fea5a99d640 10 monclient(hunting): authenticate will time out at 1124.516989s 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea540a08c0 0x7fea54131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea58f3a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea540a2930 0x7fea5411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea540a08c0 0x7fea54131820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43920/0 (socket says 172.21.15.178:43920) 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 1 -- 172.21.15.178:0/4204709126 learned_addr learned my addr 172.21.15.178:0/4204709126 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 10 monclient(hunting): get_auth_request con 0x7fea540a08c0 auth_method 0 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea58f3a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea540a2930 0x7fea5411c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:34572/0 (socket says 172.21.15.178:34572) 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea58f3a640 10 monclient(hunting): get_auth_request con 0x7fea540a2930 auth_method 0 2024-09-15T01:38:39.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea58f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea58f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea58f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea53fff640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea5409faa0 0x7fea5409b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea53fff640 10 monclient(hunting): get_auth_request con 0x7fea5409faa0 auth_method 0 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea53fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea53fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea58f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea58f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea58f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 10 monclient(hunting): handle_auth_done global_id 34199 payload 824 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 1 -- 172.21.15.178:0/4204709126 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea540a2930 msgr2=0x7fea5411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea540a2930 0x7fea5411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 1 -- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea5409faa0 msgr2=0x7fea5409b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea5409faa0 0x7fea5409b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 10 monclient: _finish_hunting 0 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 1 monclient: found mon.noname-e 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 1 -- 172.21.15.178:0/4204709126 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c009a20 con 0x7fea540a08c0 2024-09-15T01:38:39.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea537fe640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea540a08c0 0x7fea54131820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea4400c880 tx=0x7fea44008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea44007aa0 con 0x7fea540a08c0 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea4bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea4bfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea4bfff640 10 monclient: dump: 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.794 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea4bfff640 10 monclient: _finish_auth 0 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea4bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44007c60 con 0x7fea540a08c0 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea5a99d640 5 monclient: authenticate success, global_id 34199 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea5a99d640 10 monclient: _renew_subs 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea5a99d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:38:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea5a99d640 1 -- 172.21.15.178:0/4204709126 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c009450 con 0x7fea540a08c0 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea4bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea44007e10 con 0x7fea540a08c0 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea4bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea4bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea4bfff640 10 monclient: dump: 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea5a99d640 10 monclient: _renew_subs 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea527fc640 4 set_mon_vals no callback set 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea5a99d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.767+0000 7fea5a99d640 1 -- 172.21.15.178:0/4204709126 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea54142120 con 0x7fea540a08c0 2024-09-15T01:38:39.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea5a99d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea5a99d640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea54131d60 0x7fea5411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea5a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea5a99d640 1 -- 172.21.15.178:0/4204709126 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea54132130 con 0x7fea54131d60 2024-09-15T01:38:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fea44007620 con 0x7fea540a08c0 2024-09-15T01:38:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea53fff640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea54131d60 0x7fea5411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea53fff640 10 monclient: get_auth_request con 0x7fea54131d60 auth_method 0 2024-09-15T01:38:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea53fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea53fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea4bfff640 1 --2- 172.21.15.178:0/4204709126 >> v2:172.21.15.204:6812/28210 conn(0x7fea340397b0 0x7fea3403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fea4404b900 con 0x7fea540a08c0 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea58f3a640 1 --2- 172.21.15.178:0/4204709126 >> v2:172.21.15.204:6812/28210 conn(0x7fea340397b0 0x7fea3403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea58f3a640 10 monclient: get_auth_request con 0x7fea340397b0 auth_method 0 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea53fff640 10 monclient(hunting): handle_auth_done global_id 15870 payload 824 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.768+0000 7fea53fff640 1 -- 172.21.15.178:0/4204709126 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea5411eab0 con 0x7fea54131d60 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea53fff640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea54131d60 0x7fea5411ca70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea540a2f70 tx=0x7fea3c032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea3c02ac80 con 0x7fea54131d60 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea4bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea4bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:39.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea4bfff640 10 monclient: dump: 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.800 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea58f3a640 1 --2- 172.21.15.178:0/4204709126 >> v2:172.21.15.204:6812/28210 conn(0x7fea340397b0 0x7fea3403bc60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fea5409b970 tx=0x7fea4c00b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fea3c0319b0 con 0x7fea54131d60 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea4bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea4bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea54131d60 msgr2=0x7fea5411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea4bfff640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea54131d60 0x7fea5411ca70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea540a2f70 tx=0x7fea3c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea54131d60 msgr2=0x7fea5411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.769+0000 7fea4bfff640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea54131d60 0x7fea5411ca70 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea540a2f70 tx=0x7fea3c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea5a99d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea5a99d640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea5411cfb0 0x7fea54096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea5a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea5a99d640 1 -- 172.21.15.178:0/4204709126 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5411ed90 con 0x7fea5411cfb0 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea53fff640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea5411cfb0 0x7fea54096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea53fff640 10 monclient: get_auth_request con 0x7fea5411cfb0 auth_method 0 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea53fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea53fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea53fff640 10 monclient(hunting): handle_auth_done global_id 15879 payload 824 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.775+0000 7fea53fff640 1 -- 172.21.15.178:0/4204709126 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fea5404bc80 con 0x7fea5411cfb0 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.776+0000 7fea53fff640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea5411cfb0 0x7fea54096a30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009910 tx=0x7fea3c03d000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.776+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea3c03c860 con 0x7fea5411cfb0 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.776+0000 7fea4bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.776+0000 7fea4bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.776+0000 7fea4bfff640 10 monclient: dump: 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.804 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.804 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.804 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.804 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.804 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.776+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9311 (secure 0 0 0) 0x7fea3c031620 con 0x7fea5411cfb0 2024-09-15T01:38:39.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.776+0000 7fea4bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:39.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.776+0000 7fea4bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:39.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.776+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea5411cfb0 msgr2=0x7fea54096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.776+0000 7fea4bfff640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea5411cfb0 0x7fea54096a30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009910 tx=0x7fea3c03d000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.776+0000 7fea4bfff640 1 -- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea5411cfb0 msgr2=0x7fea54096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.776+0000 7fea4bfff640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea5411cfb0 0x7fea54096a30 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009910 tx=0x7fea3c03d000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.805 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:39.805 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:39.805 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:39.805 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:39.805 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:39.805 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:39.805 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:39.805 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:39.805 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:39.805 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.806 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.807 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.808 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.808 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.808 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.808 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.808 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.808 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:39.808 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:39.808 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:39.809 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:39.809 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:39.809 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:39.809 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.809 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.809 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:39.809 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:39.809 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:39.809 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:39.809 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:39.809 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:39.809 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:39.810 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.810 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:39.810 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.810 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.810 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.810 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.810 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.810 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.810 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.810 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.810 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:39.811 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.813 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:39.814 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.815 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:39.816 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.817 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.818 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:39.818 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:39.818 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.818 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.818 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.818 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:39.818 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.819 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.820 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:39.821 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.823 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:39.824 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.824 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.824 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.824 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.824 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:39.824 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.824 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.824 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.824 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.825 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.826 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:39.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.828 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:39.829 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:39.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.777+0000 7fea5a99d640 1 -- 172.21.15.178:0/4204709126 >> v2:172.21.15.204:6812/28210 conn(0x7fea340397b0 msgr2=0x7fea3403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.777+0000 7fea5a99d640 1 --2- 172.21.15.178:0/4204709126 >> v2:172.21.15.204:6812/28210 conn(0x7fea340397b0 0x7fea3403bc60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fea5409b970 tx=0x7fea4c00b040 comp rx=0 tx=0).stop 2024-09-15T01:38:39.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.777+0000 7fea5a99d640 10 monclient: shutdown 2024-09-15T01:38:39.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.777+0000 7fea5a99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.777+0000 7fea5a99d640 1 -- 172.21.15.178:0/4204709126 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea540a08c0 msgr2=0x7fea54131820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.777+0000 7fea5a99d640 1 --2- 172.21.15.178:0/4204709126 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea540a08c0 0x7fea54131820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea4400c880 tx=0x7fea44008040 comp rx=0 tx=0).stop 2024-09-15T01:38:39.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.777+0000 7fea58f3a640 1 -- 172.21.15.178:0/4204709126 reap_dead start 2024-09-15T01:38:39.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.777+0000 7fea58f3a640 1 -- 172.21.15.178:0/4204709126 reap_dead start 2024-09-15T01:38:39.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.777+0000 7fea5a99d640 1 -- 172.21.15.178:0/4204709126 shutdown_connections 2024-09-15T01:38:39.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.777+0000 7fea5a99d640 1 -- 172.21.15.178:0/4204709126 >> 172.21.15.178:0/4204709126 conn(0x7fea540a5ed0 msgr2=0x7fea54135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.777+0000 7fea5a99d640 1 -- 172.21.15.178:0/4204709126 shutdown_connections 2024-09-15T01:38:39.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.777+0000 7fea5a99d640 1 -- 172.21.15.178:0/4204709126 wait complete. 2024-09-15T01:38:39.831 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:39.831 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:38:39.831 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:38:39.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.922+0000 7f95a94dd640 10 monclient: get_monmap_and_config 2024-09-15T01:38:39.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.922+0000 7f95a94dd640 10 monclient: build_initial_monmap 2024-09-15T01:38:39.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.922+0000 7f95a94dd640 10 monclient: monmap: 2024-09-15T01:38:39.925 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:39.925 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.925 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:39.923514+0000 2024-09-15T01:38:39.925 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:39.923514+0000 2024-09-15T01:38:39.925 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:39.925 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.925 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:39.925 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 1 Processor -- start 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 1 -- start start 2024-09-15T01:38:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 10 monclient: init 2024-09-15T01:38:39.935 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=13 2024-09-15T01:38:39.935 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T01:38:39.935 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:38:39.935 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar' 13 2024-09-15T01:38:39.935 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar' 2024-09-15T01:38:39.935 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=13 2024-09-15T01:38:39.935 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar 2024-09-15T01:38:39.936 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar 2024-09-15T01:38:39.936 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar 2024-09-15T01:38:39.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:39.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 10 monclient: _add_conns ranks=[10,3,7,4,2,0,9,6,8,5,1] 2024-09-15T01:38:39.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f95a40a2930 0x7f95a4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 10 monclient(hunting): picked mon.noname-f con 0x7f95a40a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:38:39.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95a409faa0 0x7f95a409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 10 monclient(hunting): picked mon.noname-g con 0x7f95a409faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:39.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95a40a08c0 0x7f95a40a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 10 monclient(hunting): picked mon.noname-i con 0x7f95a40a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:39.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a409e0f0 con 0x7f95a40a2930 2024-09-15T01:38:39.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a409dd80 con 0x7f95a409faa0 2024-09-15T01:38:39.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40a2d00 con 0x7f95a40a08c0 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a94dd640 10 monclient(hunting): authenticate will time out at 1124.673742s 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a37fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95a40a08c0 0x7f95a40a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a2ffd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f95a40a2930 0x7f95a4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f959a7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95a409faa0 0x7f95a409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a37fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95a40a08c0 0x7f95a40a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:46218/0 (socket says 172.21.15.178:46218) 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a2ffd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f95a40a2930 0x7f95a4132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:52936/0 (socket says 172.21.15.178:52936) 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.923+0000 7f95a37fe640 1 -- 172.21.15.178:0/2542039411 learned_addr learned my addr 172.21.15.178:0/2542039411 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a37fe640 10 monclient(hunting): get_auth_request con 0x7f95a40a08c0 auth_method 0 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 10 monclient(hunting): get_auth_request con 0x7f95a409faa0 auth_method 0 2024-09-15T01:38:39.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a2ffd640 10 monclient(hunting): get_auth_request con 0x7f95a40a2930 auth_method 0 2024-09-15T01:38:39.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 10 monclient(hunting): handle_auth_done global_id 15888 payload 274 2024-09-15T01:38:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 1 -- 172.21.15.178:0/2542039411 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95a40a08c0 msgr2=0x7f95a40a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 1 --2- 172.21.15.178:0/2542039411 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95a40a08c0 0x7f95a40a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 1 -- 172.21.15.178:0/2542039411 >> v2:172.21.15.204:3300/0 conn(0x7f95a40a2930 msgr2=0x7f95a4132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 1 --2- 172.21.15.178:0/2542039411 >> v2:172.21.15.204:3300/0 conn(0x7f95a40a2930 0x7f95a4132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:38:39.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 1 monclient: found mon.noname-g 2024-09-15T01:38:39.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:39.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:39.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f959a7fc640 1 -- 172.21.15.178:0/2542039411 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a404bf60 con 0x7f95a409faa0 2024-09-15T01:38:39.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a2ffd640 1 --2- 172.21.15.178:0/2542039411 >> v2:172.21.15.204:3300/0 conn(0x7f95a40a2930 0x7f95a4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:39.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.924+0000 7f95a37fe640 1 --2- 172.21.15.178:0/2542039411 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95a40a08c0 0x7f95a40a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:39.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f959a7fc640 1 --2- 172.21.15.178:0/2542039411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95a409faa0 0x7f95a409fe70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f958c009850 tx=0x7f958c0090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=3b3eb21f35465bad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a27fc640 1 -- 172.21.15.178:0/2542039411 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f958c02abc0 con 0x7f95a409faa0 2024-09-15T01:38:39.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a27fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a27fc640 10 monclient: dump: 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.945 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a27fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a27fc640 1 -- 172.21.15.178:0/2542039411 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c02ad40 con 0x7f95a409faa0 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 5 monclient: authenticate success, global_id 15888 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:39.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a27fc640 1 -- 172.21.15.178:0/2542039411 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f958c031ca0 con 0x7f95a409faa0 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 4 set_mon_vals no callback set 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a27fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a27fc640 10 monclient: dump: 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 10 monclient: shutdown 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 1 -- 172.21.15.178:0/2542039411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95a409faa0 msgr2=0x7f95a409fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 1 --2- 172.21.15.178:0/2542039411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95a409faa0 0x7f95a409fe70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f958c009850 tx=0x7f958c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 1 -- 172.21.15.178:0/2542039411 shutdown_connections 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 1 --2- 172.21.15.178:0/2542039411 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f95a40a08c0 0x7f95a40a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 1 --2- 172.21.15.178:0/2542039411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95a409faa0 0x7f95a409fe70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 1 --2- 172.21.15.178:0/2542039411 >> v2:172.21.15.204:3300/0 conn(0x7f95a40a2930 0x7f95a4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 1 -- 172.21.15.178:0/2542039411 >> 172.21.15.178:0/2542039411 conn(0x7f95a40a5ed0 msgr2=0x7f95a40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 1 -- 172.21.15.178:0/2542039411 shutdown_connections 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.925+0000 7f95a94dd640 1 -- 172.21.15.178:0/2542039411 wait complete. 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.926+0000 7f95a94dd640 10 monclient: build_initial_monmap 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.926+0000 7f95a94dd640 10 monclient: monmap: 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:39.949 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.926+0000 7f95a94dd640 1 Processor -- start 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.926+0000 7f95a94dd640 1 -- start start 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.926+0000 7f95a94dd640 10 monclient: init 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 10 monclient: _add_conns ranks=[4,8,6,0,1,5,7,3,2] 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f95a409faa0 0x7f95a409b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 10 monclient(hunting): picked mon.noname-c con 0x7f95a409faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95a40a08c0 0x7f95a41317b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 10 monclient(hunting): picked mon.noname-e con 0x7f95a40a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95a40a2930 0x7f95a411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 10 monclient(hunting): picked mon.noname-d con 0x7f95a40a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95a411f200 con 0x7f95a409faa0 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40a2390 con 0x7f95a40a2930 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f95a409d630 con 0x7f95a40a08c0 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:39.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 10 monclient(hunting): authenticate will time out at 1124.677158s 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a2ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f95a409faa0 0x7f95a409b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959a7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95a40a08c0 0x7f95a41317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a2ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f95a409faa0 0x7f95a409b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:43238/0 (socket says 172.21.15.178:43238) 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959a7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95a40a08c0 0x7f95a41317b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43932/0 (socket says 172.21.15.178:43932) 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a2ffd640 1 -- 172.21.15.178:0/3586119130 learned_addr learned my addr 172.21.15.178:0/3586119130 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a2ffd640 10 monclient(hunting): get_auth_request con 0x7f95a409faa0 auth_method 0 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95a40a2930 0x7f95a411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959a7fc640 10 monclient(hunting): get_auth_request con 0x7f95a40a08c0 auth_method 0 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 10 monclient(hunting): get_auth_request con 0x7f95a40a2930 auth_method 0 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 10 monclient(hunting): handle_auth_done global_id 24774 payload 824 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 1 -- 172.21.15.178:0/3586119130 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95a40a08c0 msgr2=0x7f95a41317b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95a40a08c0 0x7f95a41317b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 1 -- 172.21.15.178:0/3586119130 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f95a409faa0 msgr2=0x7f95a409b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f95a409faa0 0x7f95a409b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 1 monclient: found mon.noname-d 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 1 -- 172.21.15.178:0/3586119130 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c009ae0 con 0x7f95a40a2930 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959a7fc640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f95a40a08c0 0x7f95a41317b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a2ffd640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f95a409faa0 0x7f95a409b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a37fe640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95a40a2930 0x7f95a411c530 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f959c009f70 tx=0x7f959c00b040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f959c018070 con 0x7f95a40a2930 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959b7fe640 10 monclient: dump: 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.956 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.956 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.956 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.956 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.956 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:39.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:39.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 5 monclient: authenticate success, global_id 24774 2024-09-15T01:38:39.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c00f040 con 0x7f95a40a2930 2024-09-15T01:38:39.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:39.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f959c0092e0 con 0x7f95a40a2930 2024-09-15T01:38:39.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:39.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f959b7fe640 10 monclient: dump: 2024-09-15T01:38:39.957 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.957 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.957 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.957 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.957 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.957 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a94dd640 10 monclient: _renew_subs 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.928+0000 7f95a94dd640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:39.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.927+0000 7f95a1ffb640 4 set_mon_vals no callback set 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.928+0000 7f95a94dd640 1 -- 172.21.15.178:0/3586119130 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9594009390 con 0x7f95a40a2930 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.928+0000 7f95a94dd640 10 monclient: _renew_subs 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.928+0000 7f95a94dd640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.928+0000 7f95a94dd640 1 -- 172.21.15.178:0/3586119130 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958c009390 con 0x7f95a40a2930 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.928+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f959c003020 con 0x7f95a40a2930 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.928+0000 7f95a94dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.928+0000 7f95a94dd640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a4131cf0 0x7f95a411e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.928+0000 7f95a94dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.928+0000 7f95a94dd640 1 -- 172.21.15.178:0/3586119130 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95a4070e70 con 0x7f95a4131cf0 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.928+0000 7f959b7fe640 1 --2- 172.21.15.178:0/3586119130 >> v2:172.21.15.204:6812/28210 conn(0x7f95840397b0 0x7f958403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f959c04bc80 con 0x7f95a40a2930 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f95a2ffd640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a4131cf0 0x7f95a411e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f95a2ffd640 10 monclient: get_auth_request con 0x7f95a4131cf0 auth_method 0 2024-09-15T01:38:39.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f95a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f95a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f95a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959a7fc640 1 --2- 172.21.15.178:0/3586119130 >> v2:172.21.15.204:6812/28210 conn(0x7f95840397b0 0x7f958403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959a7fc640 10 monclient: get_auth_request con 0x7f95840397b0 auth_method 0 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_done global_id 6719 payload 824 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f95a2ffd640 1 -- 172.21.15.178:0/3586119130 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95a404c4a0 con 0x7f95a4131cf0 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f95a2ffd640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a4131cf0 0x7f95a411e8f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f959400f4a0 tx=0x7f95940090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959a7fc640 1 --2- 172.21.15.178:0/3586119130 >> v2:172.21.15.204:6812/28210 conn(0x7f95840397b0 0x7f958403bc60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f958c034000 tx=0x7f958c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9594013bd0 con 0x7f95a4131cf0 2024-09-15T01:38:39.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959b7fe640 10 monclient: dump: 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.961 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f9594013d30 con 0x7f95a4131cf0 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a4131cf0 msgr2=0x7f95a411e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959b7fe640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a4131cf0 0x7f95a411e8f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f959400f4a0 tx=0x7f95940090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a4131cf0 msgr2=0x7f95a411e8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.929+0000 7f959b7fe640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a4131cf0 0x7f95a411e8f0 secure :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f959400f4a0 tx=0x7f95940090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.935+0000 7f95a94dd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.935+0000 7f95a94dd640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a411ca70 0x7f95a411ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.935+0000 7f95a94dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.935+0000 7f95a94dd640 1 -- 172.21.15.178:0/3586119130 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95a404bc80 con 0x7f95a411ca70 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.935+0000 7f95a2ffd640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a411ca70 0x7f95a411ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.935+0000 7f95a2ffd640 10 monclient: get_auth_request con 0x7f95a411ca70 auth_method 0 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.935+0000 7f95a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.935+0000 7f95a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.935+0000 7f95a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f95a2ffd640 10 monclient(hunting): handle_auth_done global_id 6728 payload 824 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f95a2ffd640 1 -- 172.21.15.178:0/3586119130 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f95a405ed70 con 0x7f95a411ca70 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f95a2ffd640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a411ca70 0x7f95a411ce70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9594000c00 tx=0x7f959401c000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:39.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9594024040 con 0x7f95a411ca70 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f959b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f959b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f959b7fe640 10 monclient: dump: 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:39.965 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:39.965 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:39.965 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:39.965 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:39.965 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:39.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9313 (secure 0 0 0) 0x7f959400fed0 con 0x7f95a411ca70 2024-09-15T01:38:39.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f959b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:39.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f959b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:39.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a411ca70 msgr2=0x7f95a411ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f959b7fe640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a411ca70 0x7f95a411ce70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9594000c00 tx=0x7f959401c000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f959b7fe640 1 -- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a411ca70 msgr2=0x7f95a411ce70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:39.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.936+0000 7f959b7fe640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f95a411ca70 0x7f95a411ce70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:39.966 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:39.966 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:39.966 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:39.966 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:38:39.966 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:39.966 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:39.966 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:39.966 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:39.966 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:39.966 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:39.966 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.967 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.968 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.969 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.969 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.969 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.969 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.969 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.969 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:39.969 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:39.969 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:39.969 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:39.969 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:39.969 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:39.970 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:39.971 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.972 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.973 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.974 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.975 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.976 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.977 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.978 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.978 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:39.978 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.978 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.978 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.978 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.978 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:39.978 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:39.978 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.978 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.979 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:39.980 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.981 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:39.982 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.983 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.984 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:39.986 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.987 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 4 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:39.988 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.937+0000 7f95a94dd640 1 -- 172.21.15.178:0/3586119130 >> v2:172.21.15.204:6812/28210 conn(0x7f95840397b0 msgr2=0x7f958403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.937+0000 7f95a94dd640 1 --2- 172.21.15.178:0/3586119130 >> v2:172.21.15.204:6812/28210 conn(0x7f95840397b0 0x7f958403bc60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f958c034000 tx=0x7f958c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.937+0000 7f95a94dd640 10 monclient: shutdown 2024-09-15T01:38:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.937+0000 7f95a94dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.937+0000 7f95a94dd640 1 -- 172.21.15.178:0/3586119130 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95a40a2930 msgr2=0x7f95a411c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.937+0000 7f95a94dd640 1 --2- 172.21.15.178:0/3586119130 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f95a40a2930 0x7f95a411c530 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f959c009f70 tx=0x7f959c00b040 comp rx=0 tx=0).stop 2024-09-15T01:38:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.937+0000 7f95a37fe640 1 -- 172.21.15.178:0/3586119130 reap_dead start 2024-09-15T01:38:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.937+0000 7f95a37fe640 1 -- 172.21.15.178:0/3586119130 reap_dead start 2024-09-15T01:38:39.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.937+0000 7f95a94dd640 1 -- 172.21.15.178:0/3586119130 shutdown_connections 2024-09-15T01:38:39.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.937+0000 7f95a94dd640 1 -- 172.21.15.178:0/3586119130 >> 172.21.15.178:0/3586119130 conn(0x7f95a40a5ed0 msgr2=0x7f95a40a4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:39.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.937+0000 7f95a94dd640 1 -- 172.21.15.178:0/3586119130 shutdown_connections 2024-09-15T01:38:39.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:39.937+0000 7f95a94dd640 1 -- 172.21.15.178:0/3586119130 wait complete. 2024-09-15T01:38:39.991 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:39.991 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:38:39.991 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:38:40.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.083+0000 7f99bfc6f640 10 monclient: get_monmap_and_config 2024-09-15T01:38:40.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.083+0000 7f99bfc6f640 10 monclient: build_initial_monmap 2024-09-15T01:38:40.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.083+0000 7f99bfc6f640 10 monclient: monmap: 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:40.084164+0000 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:40.084164+0000 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:40.087 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:40.087 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:40.087 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:40.087 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:40.087 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.083+0000 7f99bfc6f640 1 Processor -- start 2024-09-15T01:38:40.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 1 -- start start 2024-09-15T01:38:40.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 10 monclient: init 2024-09-15T01:38:40.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:40.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 10 monclient: _add_conns ranks=[3,10,2,9,1,8,0,6,5,4,7] 2024-09-15T01:38:40.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b80a2c80 0x7f99b80a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 10 monclient(hunting): picked mon.noname-g con 0x7f99b80a2c80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:40.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f99b809fc70 0x7f99b8132ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 10 monclient(hunting): picked mon.noname-f con 0x7f99b809fc70 addr v2:172.21.15.204:3300/0 2024-09-15T01:38:40.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a00a0 0x7f99b8132ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 10 monclient(hunting): picked mon.noname-b con 0x7f99b80a00a0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:40.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b80a3050 con 0x7f99b809fc70 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b80a1e50 con 0x7f99b80a00a0 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b809dbc0 con 0x7f99b80a2c80 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bfc6f640 10 monclient(hunting): authenticate will time out at 1124.834644s 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99be20c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a00a0 0x7f99b8132ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99be20c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a00a0 0x7f99b8132ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42938/0 (socket says 172.21.15.178:42938) 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99be20c640 1 -- 172.21.15.178:0/3282868914 learned_addr learned my addr 172.21.15.178:0/3282868914 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99be20c640 10 monclient(hunting): get_auth_request con 0x7f99b80a00a0 auth_method 0 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99be20c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99be20c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99be20c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99be20c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99be20c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bda0b640 1 --2- 172.21.15.178:0/3282868914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b80a2c80 0x7f99b80a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99be20c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bd20a640 1 --2- 172.21.15.178:0/3282868914 >> v2:172.21.15.204:3300/0 conn(0x7f99b809fc70 0x7f99b8132ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bda0b640 10 monclient(hunting): get_auth_request con 0x7f99b80a2c80 auth_method 0 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bda0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bda0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bda0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bd20a640 10 monclient(hunting): get_auth_request con 0x7f99b809fc70 auth_method 0 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bd20a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bd20a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.084+0000 7f99bd20a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bda0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bda0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bda0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bd20a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bd20a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bd20a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99be20c640 10 monclient(hunting): handle_auth_done global_id 34166 payload 274 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99be20c640 1 -- 172.21.15.178:0/3282868914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b80a2c80 msgr2=0x7f99b80a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99be20c640 1 --2- 172.21.15.178:0/3282868914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b80a2c80 0x7f99b80a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99be20c640 1 -- 172.21.15.178:0/3282868914 >> v2:172.21.15.204:3300/0 conn(0x7f99b809fc70 msgr2=0x7f99b8132ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99be20c640 1 --2- 172.21.15.178:0/3282868914 >> v2:172.21.15.204:3300/0 conn(0x7f99b809fc70 0x7f99b8132ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99be20c640 10 monclient: _finish_hunting 0 2024-09-15T01:38:40.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99be20c640 1 monclient: found mon.noname-b 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99be20c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99be20c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99be20c640 1 -- 172.21.15.178:0/3282868914 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b804bf60 con 0x7f99b80a00a0 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99be20c640 1 --2- 172.21.15.178:0/3282868914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a00a0 0x7f99b8132ff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009f20 tx=0x7f99b0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c8f0f3a7626bbf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bca09640 1 -- 172.21.15.178:0/3282868914 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99b0007880 con 0x7f99b80a00a0 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bca09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bca09640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bca09640 10 monclient: dump: 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.101 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bca09640 10 monclient: _finish_auth 0 2024-09-15T01:38:40.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bca09640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:40.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bca09640 1 -- 172.21.15.178:0/3282868914 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b002d050 con 0x7f99b80a00a0 2024-09-15T01:38:40.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bfc6f640 5 monclient: authenticate success, global_id 34166 2024-09-15T01:38:40.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bfc6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:40.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bca09640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:40.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bfc6f640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:40.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bca09640 1 -- 172.21.15.178:0/3282868914 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99b0030360 con 0x7f99b80a00a0 2024-09-15T01:38:40.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bfc6f640 4 set_mon_vals no callback set 2024-09-15T01:38:40.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bca09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:40.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bca09640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:40.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.085+0000 7f99bca09640 10 monclient: dump: 2024-09-15T01:38:40.105 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:40.105 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:40.105 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.105 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.105 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:40.105 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:40.105 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:40.105 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:40.105 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:40.105 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:40.106 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:40.106 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:40.106 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:40.106 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:40.106 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:40.106 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.086+0000 7f99bfc6f640 10 monclient: shutdown 2024-09-15T01:38:40.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.086+0000 7f99bfc6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:40.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.086+0000 7f99bfc6f640 1 -- 172.21.15.178:0/3282868914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a00a0 msgr2=0x7f99b8132ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:40.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.086+0000 7f99bfc6f640 1 --2- 172.21.15.178:0/3282868914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a00a0 0x7f99b8132ff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009f20 tx=0x7f99b0009840 comp rx=0 tx=0).stop 2024-09-15T01:38:40.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.086+0000 7f99bfc6f640 1 -- 172.21.15.178:0/3282868914 shutdown_connections 2024-09-15T01:38:40.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.086+0000 7f99bfc6f640 1 --2- 172.21.15.178:0/3282868914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a00a0 0x7f99b8132ff0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.086+0000 7f99bfc6f640 1 --2- 172.21.15.178:0/3282868914 >> v2:172.21.15.204:3300/0 conn(0x7f99b809fc70 0x7f99b8132ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.086+0000 7f99bfc6f640 1 --2- 172.21.15.178:0/3282868914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b80a2c80 0x7f99b80a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.086+0000 7f99bfc6f640 1 -- 172.21.15.178:0/3282868914 >> 172.21.15.178:0/3282868914 conn(0x7f99b806e860 msgr2=0x7f99b806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.086+0000 7f99bfc6f640 1 -- 172.21.15.178:0/3282868914 shutdown_connections 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.086+0000 7f99bfc6f640 1 -- 172.21.15.178:0/3282868914 wait complete. 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient: build_initial_monmap 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient: monmap: 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 1 Processor -- start 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 1 -- start start 2024-09-15T01:38:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient: init 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient: _add_conns ranks=[5,1,2,4,7,6,0,3,8] 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f99b809fc70 0x7f99b811efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient(hunting): picked mon.noname-h con 0x7f99b809fc70 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99b80a00a0 0x7f99b811f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient(hunting): picked mon.noname-f con 0x7f99b80a00a0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a2c80 0x7f99b811fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient(hunting): picked mon.noname-b con 0x7f99b80a2c80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b8120af0 con 0x7f99b80a00a0 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b809d8b0 con 0x7f99b80a2c80 2024-09-15T01:38:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99b809d5f0 con 0x7f99b809fc70 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bfc6f640 10 monclient(hunting): authenticate will time out at 1124.837838s 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99be20c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a2c80 0x7f99b811fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99be20c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a2c80 0x7f99b811fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42944/0 (socket says 172.21.15.178:42944) 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99be20c640 1 -- 172.21.15.178:0/1935880737 learned_addr learned my addr 172.21.15.178:0/1935880737 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99be20c640 10 monclient(hunting): get_auth_request con 0x7f99b80a2c80 auth_method 0 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99be20c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99be20c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99be20c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bd20a640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99b80a00a0 0x7f99b811f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.087+0000 7f99bda0b640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f99b809fc70 0x7f99b811efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bd20a640 10 monclient(hunting): get_auth_request con 0x7f99b80a00a0 auth_method 0 2024-09-15T01:38:40.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bd20a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bd20a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bd20a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bda0b640 10 monclient(hunting): get_auth_request con 0x7f99b809fc70 auth_method 0 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bda0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bda0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bda0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bd20a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bd20a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bd20a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bda0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bda0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bda0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 10 monclient(hunting): handle_auth_done global_id 34175 payload 824 2024-09-15T01:38:40.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 1 -- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f99b809fc70 msgr2=0x7f99b811efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:40.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f99b809fc70 0x7f99b811efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 1 -- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99b80a00a0 msgr2=0x7f99b811f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:40.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99b80a00a0 0x7f99b811f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 10 monclient: _finish_hunting 0 2024-09-15T01:38:40.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 1 monclient: found mon.noname-b 2024-09-15T01:38:40.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:40.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:38:40.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 1 -- 172.21.15.178:0/1935880737 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b0009ae0 con 0x7f99b80a2c80 2024-09-15T01:38:40.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bd20a640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99b80a00a0 0x7f99b811f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99be20c640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a2c80 0x7f99b811fa40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009df0 tx=0x7f99b0036000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99b003baa0 con 0x7f99b80a2c80 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99adffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99adffb640 10 monclient: dump: 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:40.113 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99adffb640 10 monclient: _finish_auth 0 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b003bc40 con 0x7f99b80a2c80 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bfc6f640 5 monclient: authenticate success, global_id 34175 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bfc6f640 10 monclient: _renew_subs 2024-09-15T01:38:40.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bfc6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bfc6f640 1 -- 172.21.15.178:0/1935880737 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b8135720 con 0x7f99b80a2c80 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99adffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99b003bdb0 con 0x7f99b80a2c80 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99adffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99adffb640 10 monclient: dump: 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:40.115 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:40.116 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:40.116 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:40.116 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:40.116 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:40.116 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:40.116 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:40.116 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:40.116 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99affff640 4 set_mon_vals no callback set 2024-09-15T01:38:40.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bfc6f640 10 monclient: _renew_subs 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bfc6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.088+0000 7f99bfc6f640 1 -- 172.21.15.178:0/1935880737 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b8122590 con 0x7f99b80a2c80 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f99b0038070 con 0x7f99b80a2c80 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bfc6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bfc6f640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b811ff80 0x7f99b8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bfc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bfc6f640 1 -- 172.21.15.178:0/1935880737 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f99b8064390 con 0x7f99b811ff80 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99adffb640 1 --2- 172.21.15.178:0/1935880737 >> v2:172.21.15.204:6812/28210 conn(0x7f999c0397e0 0x7f999c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bda0b640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b811ff80 0x7f99b8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f99b006bca0 con 0x7f99b80a2c80 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bda0b640 10 monclient: get_auth_request con 0x7f99b811ff80 auth_method 0 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bda0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bda0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bda0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bda0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bda0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bd20a640 1 --2- 172.21.15.178:0/1935880737 >> v2:172.21.15.204:6812/28210 conn(0x7f999c0397e0 0x7f999c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bda0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.089+0000 7f99bd20a640 10 monclient: get_auth_request con 0x7f999c0397e0 auth_method 0 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99bda0b640 10 monclient(hunting): handle_auth_done global_id 24919 payload 824 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99bda0b640 1 -- 172.21.15.178:0/1935880737 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99b804bc80 con 0x7f99b811ff80 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99bda0b640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b811ff80 0x7f99b8099a50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f99a8011c20 tx=0x7f99a80090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99bd20a640 1 --2- 172.21.15.178:0/1935880737 >> v2:172.21.15.204:6812/28210 conn(0x7f999c0397e0 0x7f999c03bc90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f99b80a70e0 tx=0x7f99b40090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99a8011e60 con 0x7f99b811ff80 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99adffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:40.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99adffb640 10 monclient: dump: 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:40.119 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:40.120 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:40.120 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f99a8009e00 con 0x7f99b811ff80 2024-09-15T01:38:40.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99adffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:40.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99adffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:40.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b811ff80 msgr2=0x7f99b8099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:40.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99adffb640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b811ff80 0x7f99b8099a50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f99a8011c20 tx=0x7f99a80090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b811ff80 msgr2=0x7f99b8099a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:40.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.090+0000 7f99adffb640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b811ff80 0x7f99b8099a50 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f99a8011c20 tx=0x7f99a80090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bfc6f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:40.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bfc6f640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b8099f90 0x7f99b809a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bfc6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bfc6f640 1 -- 172.21.15.178:0/1935880737 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f99b8122be0 con 0x7f99b8099f90 2024-09-15T01:38:40.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bda0b640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b8099f90 0x7f99b809a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bda0b640 10 monclient: get_auth_request con 0x7f99b8099f90 auth_method 0 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bda0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bda0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bda0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bda0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bda0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bda0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bda0b640 10 monclient(hunting): handle_auth_done global_id 24928 payload 824 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.096+0000 7f99bda0b640 1 -- 172.21.15.178:0/1935880737 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f99b804c4a0 con 0x7f99b8099f90 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99bda0b640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b8099f90 0x7f99b809a390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f99a8011c20 tx=0x7f99a801c000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99a801c3e0 con 0x7f99b8099f90 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99adffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:40.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99adffb640 10 monclient: dump: 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:40.124 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:40.125 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:40.125 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:40.125 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f99a801c540 con 0x7f99b8099f90 2024-09-15T01:38:40.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99adffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:40.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99adffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:40.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b8099f90 msgr2=0x7f99b809a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:40.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99adffb640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b8099f90 0x7f99b809a390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f99a8011c20 tx=0x7f99a801c000 comp rx=0 tx=0).stop 2024-09-15T01:38:40.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99adffb640 1 -- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b8099f90 msgr2=0x7f99b809a390 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:40.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99adffb640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f99b8099f90 0x7f99b809a390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.125 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:40.126 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:40.127 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:40.128 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:40.130 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:40.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:40.133 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:40.134 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:40.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:40.137 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.139 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:40.140 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:40.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:40.142 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.144 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:40.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:40.144 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.144 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:40.144 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:40.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:40.144 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:40.144 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:40.144 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:40.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:40.147 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:40.148 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:40.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:40.148 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:40.148 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:40.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99bfc6f640 1 -- 172.21.15.178:0/1935880737 >> v2:172.21.15.204:6812/28210 conn(0x7f999c0397e0 msgr2=0x7f999c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:40.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99bfc6f640 1 --2- 172.21.15.178:0/1935880737 >> v2:172.21.15.204:6812/28210 conn(0x7f999c0397e0 0x7f999c03bc90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f99b80a70e0 tx=0x7f99b40090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99bfc6f640 10 monclient: shutdown 2024-09-15T01:38:40.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99bfc6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:40.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99bfc6f640 1 -- 172.21.15.178:0/1935880737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a2c80 msgr2=0x7f99b811fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:40.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99bfc6f640 1 --2- 172.21.15.178:0/1935880737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99b80a2c80 0x7f99b811fa40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009df0 tx=0x7f99b0036000 comp rx=0 tx=0).stop 2024-09-15T01:38:40.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99be20c640 1 -- 172.21.15.178:0/1935880737 reap_dead start 2024-09-15T01:38:40.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99be20c640 1 -- 172.21.15.178:0/1935880737 reap_dead start 2024-09-15T01:38:40.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99bfc6f640 1 -- 172.21.15.178:0/1935880737 shutdown_connections 2024-09-15T01:38:40.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.097+0000 7f99bfc6f640 1 -- 172.21.15.178:0/1935880737 >> 172.21.15.178:0/1935880737 conn(0x7f99b806e860 msgr2=0x7f99b8135970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:40.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.098+0000 7f99bfc6f640 1 -- 172.21.15.178:0/1935880737 shutdown_connections 2024-09-15T01:38:40.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.098+0000 7f99bfc6f640 1 -- 172.21.15.178:0/1935880737 wait complete. 2024-09-15T01:38:40.150 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:40.150 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T01:38:40.151 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T01:38:40.234 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=13 2024-09-15T01:38:40.234 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T01:38:40.234 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:38:40.234 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump' 13 2024-09-15T01:38:40.234 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump' 2024-09-15T01:38:40.234 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=13 2024-09-15T01:38:40.235 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump 2024-09-15T01:38:40.235 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump 2024-09-15T01:38:40.235 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump 2024-09-15T01:38:40.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.240+0000 7f88e9a8b640 10 monclient: get_monmap_and_config 2024-09-15T01:38:40.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.240+0000 7f88e9a8b640 10 monclient: build_initial_monmap 2024-09-15T01:38:40.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.241+0000 7f88e9a8b640 10 monclient: monmap: 2024-09-15T01:38:40.243 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:40.243 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:40.243 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:40.241887+0000 2024-09-15T01:38:40.243 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:40.241887+0000 2024-09-15T01:38:40.243 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:40.243 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:40.243 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:40.244 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:40.244 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:40.244 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:40.244 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:40.244 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:40.244 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:40.244 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:40.244 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:40.244 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:40.245 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:40.245 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.241+0000 7f88e9a8b640 1 Processor -- start 2024-09-15T01:38:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.241+0000 7f88e9a8b640 1 -- start start 2024-09-15T01:38:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.241+0000 7f88e9a8b640 10 monclient: init 2024-09-15T01:38:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 10 monclient: _add_conns ranks=[6,8,4,10,2,1,0,5,7,9,3] 2024-09-15T01:38:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f88e40a2930 0x7f88e4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 10 monclient(hunting): picked mon.noname-d con 0x7f88e40a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f88e409faa0 0x7f88e409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 10 monclient(hunting): picked mon.noname-e con 0x7f88e409faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f88e40a08c0 0x7f88e40a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 10 monclient(hunting): picked mon.noname-c con 0x7f88e40a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88e409e0f0 con 0x7f88e40a08c0 2024-09-15T01:38:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f88e409dd80 con 0x7f88e40a2930 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f88e40a2d00 con 0x7f88e409faa0 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e9a8b640 10 monclient(hunting): authenticate will time out at 1124.992269s 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f88e409faa0 0x7f88e409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e37fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f88e40a2930 0x7f88e4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e3fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f88e40a08c0 0x7f88e40a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f88e409faa0 0x7f88e409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43940/0 (socket says 172.21.15.178:43940) 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 1 -- 172.21.15.178:0/325873982 learned_addr learned my addr 172.21.15.178:0/325873982 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e3fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f88e40a08c0 0x7f88e40a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:43254/0 (socket says 172.21.15.178:43254) 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e37fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f88e40a2930 0x7f88e4132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:34586/0 (socket says 172.21.15.178:34586) 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 10 monclient(hunting): get_auth_request con 0x7f88e409faa0 auth_method 0 2024-09-15T01:38:40.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e3fff640 10 monclient(hunting): get_auth_request con 0x7f88e40a08c0 auth_method 0 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e37fe640 10 monclient(hunting): get_auth_request con 0x7f88e40a2930 auth_method 0 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 10 monclient(hunting): handle_auth_done global_id 34217 payload 274 2024-09-15T01:38:40.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 1 -- 172.21.15.178:0/325873982 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f88e40a2930 msgr2=0x7f88e4132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:40.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 1 --2- 172.21.15.178:0/325873982 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f88e40a2930 0x7f88e4132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 1 -- 172.21.15.178:0/325873982 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f88e40a08c0 msgr2=0x7f88e40a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:40.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 1 --2- 172.21.15.178:0/325873982 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f88e40a08c0 0x7f88e40a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 1 monclient: found mon.noname-e 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e2ffd640 1 -- 172.21.15.178:0/325873982 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e404bf60 con 0x7f88e409faa0 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.242+0000 7f88e3fff640 1 --2- 172.21.15.178:0/325873982 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f88e40a08c0 0x7f88e40a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e2ffd640 1 --2- 172.21.15.178:0/325873982 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f88e409faa0 0x7f88e409fe70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009940 tx=0x7f88d80091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a97b845b327211b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e27fc640 1 -- 172.21.15.178:0/325873982 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f88d802ac60 con 0x7f88e409faa0 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e27fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e27fc640 10 monclient: dump: 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:40.259 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:40.260 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e27fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e9a8b640 5 monclient: authenticate success, global_id 34217 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e27fc640 1 -- 172.21.15.178:0/325873982 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d802ade0 con 0x7f88e409faa0 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e9a8b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e9a8b640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e9a8b640 4 set_mon_vals no callback set 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e27fc640 1 -- 172.21.15.178:0/325873982 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f88d8031ca0 con 0x7f88e409faa0 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e27fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:38:40.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.243+0000 7f88e27fc640 10 monclient: dump: 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:40.262 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 10 monclient: shutdown 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 1 -- 172.21.15.178:0/325873982 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f88e409faa0 msgr2=0x7f88e409fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 1 --2- 172.21.15.178:0/325873982 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f88e409faa0 0x7f88e409fe70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009940 tx=0x7f88d80091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 1 -- 172.21.15.178:0/325873982 shutdown_connections 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 1 --2- 172.21.15.178:0/325873982 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f88e40a08c0 0x7f88e40a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 1 --2- 172.21.15.178:0/325873982 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f88e409faa0 0x7f88e409fe70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 1 --2- 172.21.15.178:0/325873982 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f88e40a2930 0x7f88e4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 1 -- 172.21.15.178:0/325873982 >> 172.21.15.178:0/325873982 conn(0x7f88e40a5ed0 msgr2=0x7f88e40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 1 -- 172.21.15.178:0/325873982 shutdown_connections 2024-09-15T01:38:40.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 1 -- 172.21.15.178:0/325873982 wait complete. 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 10 monclient: build_initial_monmap 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.244+0000 7f88e9a8b640 10 monclient: monmap: 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:40.264 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:40.265 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:40.265 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:40.265 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:40.265 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:40.265 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 1 Processor -- start 2024-09-15T01:38:40.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 1 -- start start 2024-09-15T01:38:40.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 10 monclient: init 2024-09-15T01:38:40.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:40.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 10 monclient: _add_conns ranks=[0,1,7,2,6,3,8,5,4] 2024-09-15T01:38:40.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e409faa0 0x7f88e409b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 10 monclient(hunting): picked mon.noname-a con 0x7f88e409faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88e40a08c0 0x7f88e4131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 10 monclient(hunting): picked mon.noname-f con 0x7f88e40a08c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f88e40a2930 0x7f88e4141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 10 monclient(hunting): picked mon.noname-i con 0x7f88e40a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e411d760 con 0x7f88e409faa0 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e411ed20 con 0x7f88e40a08c0 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f88e411ea40 con 0x7f88e40a2930 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e9a8b640 10 monclient(hunting): authenticate will time out at 1124.995700s 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e409faa0 0x7f88e409b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e409faa0 0x7f88e409b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49736/0 (socket says 172.21.15.178:49736) 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e37fe640 1 -- 172.21.15.178:0/1305745737 learned_addr learned my addr 172.21.15.178:0/1305745737 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e37fe640 10 monclient(hunting): get_auth_request con 0x7f88e409faa0 auth_method 0 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e2ffd640 1 --2- 172.21.15.178:0/1305745737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88e40a08c0 0x7f88e4131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e2ffd640 10 monclient(hunting): get_auth_request con 0x7f88e40a08c0 auth_method 0 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e3fff640 1 --2- 172.21.15.178:0/1305745737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f88e40a2930 0x7f88e4141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e3fff640 10 monclient(hunting): get_auth_request con 0x7f88e40a2930 auth_method 0 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.245+0000 7f88e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e37fe640 10 monclient(hunting): handle_auth_done global_id 34245 payload 824 2024-09-15T01:38:40.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e37fe640 1 -- 172.21.15.178:0/1305745737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f88e40a2930 msgr2=0x7f88e4141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:40.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e37fe640 1 --2- 172.21.15.178:0/1305745737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f88e40a2930 0x7f88e4141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e37fe640 1 -- 172.21.15.178:0/1305745737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88e40a08c0 msgr2=0x7f88e4131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:40.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e37fe640 1 --2- 172.21.15.178:0/1305745737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88e40a08c0 0x7f88e4131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:38:40.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e37fe640 1 monclient: found mon.noname-a 2024-09-15T01:38:40.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:40.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:38:40.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e37fe640 1 -- 172.21.15.178:0/1305745737 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc009a20 con 0x7f88e409faa0 2024-09-15T01:38:40.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e37fe640 1 --2- 172.21.15.178:0/1305745737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e409faa0 0x7f88e409b2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f88d400ea20 tx=0x7f88d40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:40.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88cb7fe640 1 -- 172.21.15.178:0/1305745737 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f88d400ce90 con 0x7f88e409faa0 2024-09-15T01:38:40.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88cb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:40.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88cb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88cb7fe640 10 monclient: dump: 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:40.270 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88cb7fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88cb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88cb7fe640 1 -- 172.21.15.178:0/1305745737 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d4009600 con 0x7f88e409faa0 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e9a8b640 5 monclient: authenticate success, global_id 34245 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e9a8b640 10 monclient: _renew_subs 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e9a8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e9a8b640 1 -- 172.21.15.178:0/1305745737 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d8009450 con 0x7f88e409faa0 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88cb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88cb7fe640 1 -- 172.21.15.178:0/1305745737 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f88d40138f0 con 0x7f88e409faa0 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88cb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:40.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88cb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88cb7fe640 10 monclient: dump: 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:40.272 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e9a8b640 10 monclient: _renew_subs 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e9a8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e9a8b640 1 -- 172.21.15.178:0/1305745737 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e411db50 con 0x7f88e409faa0 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.246+0000 7f88e1ffb640 4 set_mon_vals no callback set 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.247+0000 7f88cb7fe640 1 -- 172.21.15.178:0/1305745737 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f88d4013470 con 0x7f88e409faa0 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.247+0000 7f88e9a8b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.247+0000 7f88cb7fe640 1 --2- 172.21.15.178:0/1305745737 >> v2:172.21.15.204:6812/28210 conn(0x7f88d00396e0 0x7f88d003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.247+0000 7f88e9a8b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:40.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.247+0000 7f88e9a8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:40.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.247+0000 7f88e9a8b640 1 -- 172.21.15.178:0/1305745737 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e404b720 con 0x7f88e409faa0 2024-09-15T01:38:40.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.247+0000 7f88cb7fe640 1 -- 172.21.15.178:0/1305745737 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f88d404c220 con 0x7f88e409faa0 2024-09-15T01:38:40.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.247+0000 7f88e2ffd640 1 --2- 172.21.15.178:0/1305745737 >> v2:172.21.15.204:6812/28210 conn(0x7f88d00396e0 0x7f88d003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:40.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.247+0000 7f88e2ffd640 10 monclient: get_auth_request con 0x7f88d00396e0 auth_method 0 2024-09-15T01:38:40.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.248+0000 7f88e2ffd640 1 --2- 172.21.15.178:0/1305745737 >> v2:172.21.15.204:6812/28210 conn(0x7f88d00396e0 0x7f88d003bb90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f88d8034000 tx=0x7f88d8032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:40.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.252+0000 7f88cb7fe640 1 -- 172.21.15.178:0/1305745737 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f88d401c450 con 0x7f88e409faa0 2024-09-15T01:38:40.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.252+0000 7f88cb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:40.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.252+0000 7f88cb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:40.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.466+0000 7f88e9a8b640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T01:38:40.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.466+0000 7f88e9a8b640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T01:38:40.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.466+0000 7f88e9a8b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:40.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.466+0000 7f88e9a8b640 1 -- 172.21.15.178:0/1305745737 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f88e409cef0 con 0x7f88e409faa0 2024-09-15T01:38:40.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.468+0000 7f88cb7fe640 1 -- 172.21.15.178:0/1305745737 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f88d4011350 con 0x7f88e409faa0 2024-09-15T01:38:40.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.468+0000 7f88cb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T01:38:40.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.468+0000 7f88cb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:40.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 1 -- 172.21.15.178:0/1305745737 >> v2:172.21.15.204:6812/28210 conn(0x7f88d00396e0 msgr2=0x7f88d003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:40.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 1 --2- 172.21.15.178:0/1305745737 >> v2:172.21.15.204:6812/28210 conn(0x7f88d00396e0 0x7f88d003bb90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f88d8034000 tx=0x7f88d8032000 comp rx=0 tx=0).stop 2024-09-15T01:38:40.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 10 monclient: shutdown 2024-09-15T01:38:40.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:40.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 1 -- 172.21.15.178:0/1305745737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e409faa0 msgr2=0x7f88e409b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:40.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 1 --2- 172.21.15.178:0/1305745737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e409faa0 0x7f88e409b2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f88d400ea20 tx=0x7f88d40090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 1 -- 172.21.15.178:0/1305745737 shutdown_connections 2024-09-15T01:38:40.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 1 --2- 172.21.15.178:0/1305745737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88e40a08c0 0x7f88e4131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 1 --2- 172.21.15.178:0/1305745737 >> v2:172.21.15.204:6812/28210 conn(0x7f88d00396e0 0x7f88d003bb90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 1 --2- 172.21.15.178:0/1305745737 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f88e40a2930 0x7f88e4141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 1 --2- 172.21.15.178:0/1305745737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e409faa0 0x7f88e409b2f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:40.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 1 -- 172.21.15.178:0/1305745737 >> 172.21.15.178:0/1305745737 conn(0x7f88e40a5ed0 msgr2=0x7f88e40a4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:40.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 1 -- 172.21.15.178:0/1305745737 shutdown_connections 2024-09-15T01:38:40.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:40.471+0000 7f88e9a8b640 1 -- 172.21.15.178:0/1305745737 wait complete. 2024-09-15T01:38:40.488 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T01:38:40.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=195799729,config=3140514009,health=2864126176,kv=706962213,logm=1052943048}) 2024-09-15T01:38:40.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=8,logm=65} crc {auth=3143608520,config=3140514009,health=2864126176,kv=706962213,logm=4244221095}) 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=195799729,config=3140514009,health=2864126176,kv=706962213,logm=1052943048}) 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=11,config=2,health=11,kv=7,logm=69} crc {auth=993957813,config=3140514009,health=3814374855,kv=2892508859,logm=3579412685}) 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=195799729,config=3140514009,health=2864126176,kv=706962213,logm=1052943048}) 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=11,config=2,health=11,kv=8,logm=68} crc {auth=3962154605,config=3140514009,health=347675044,kv=706962213,logm=3917141147}) 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=195799729,config=3140514009,health=2864126176,kv=706962213,logm=1052943048}) 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=12,config=2,health=10,kv=6,logm=70} crc {auth=1382405569,config=3140514009,health=4013947633,kv=1611954805,logm=3183802691}) 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=195799729,config=3140514009,health=2864126176,kv=706962213,logm=1052943048}) 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=12,config=2,health=12,kv=7,logm=67} crc {auth=2693735881,config=3140514009,health=2864126176,kv=963045094,logm=2278732930}) 2024-09-15T01:38:40.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=195799729,config=3140514009,health=2864126176,kv=706962213,logm=1052943048}) 2024-09-15T01:38:40.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=13,config=1,health=12,kv=8,logm=66} crc {auth=3143608520,config=1228700967,health=2864126176,kv=706962213,logm=1576783199}) 2024-09-15T01:38:40.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=195799729,config=3140514009,health=2864126176,kv=706962213,logm=1052943048}) 2024-09-15T01:38:40.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=12,config=2,health=12,kv=8,logm=66} crc {auth=3719008877,config=3140514009,health=2864126176,kv=706962213,logm=570685936}) 2024-09-15T01:38:40.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=195799729,config=3140514009,health=2864126176,kv=706962213,logm=1052943048}) 2024-09-15T01:38:40.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=11,config=2,health=10,kv=8,logm=69} crc {auth=1281723857,config=3140514009,health=2308645196,kv=706962213,logm=3094615426}) 2024-09-15T01:38:40.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1234192269}) 2024-09-15T01:38:40.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2764515963}) 2024-09-15T01:38:40.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1234192269}) 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1371474121}) 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1234192269}) 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4230066059}) 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1234192269}) 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=566602986}) 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1234192269}) 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3901570686}) 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1234192269}) 2024-09-15T01:38:40.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3920419424}) 2024-09-15T01:38:40.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1234192269}) 2024-09-15T01:38:40.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=956871769}) 2024-09-15T01:38:40.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1234192269}) 2024-09-15T01:38:40.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1767042268}) 2024-09-15T01:38:40.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=474636486}) 2024-09-15T01:38:40.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1095266331}) 2024-09-15T01:38:40.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=474636486}) 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1431559002}) 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=474636486}) 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=538813653}) 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=474636486}) 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=721476990}) 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=474636486}) 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=21999209}) 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=474636486}) 2024-09-15T01:38:40.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1790043779}) 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=474636486}) 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4234365329}) 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=474636486}) 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1927422438}) 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330427298}) 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167002527}) 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330427298}) 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2771794901}) 2024-09-15T01:38:40.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330427298}) 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3173313287}) 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330427298}) 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1530185921}) 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330427298}) 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=449002007}) 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330427298}) 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3306567665}) 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330427298}) 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=213081064}) 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330427298}) 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2576454076}) 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4173467143}) 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=995746546}) 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4173467143}) 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=189908473}) 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4173467143}) 2024-09-15T01:38:40.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2738031062}) 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4173467143}) 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2146873833}) 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4173467143}) 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1283621370}) 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4173467143}) 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=424291674}) 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4173467143}) 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=284290274}) 2024-09-15T01:38:40.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4173467143}) 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2934687276}) 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1942827794}) 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=215521032}) 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1942827794}) 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4120424284}) 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1942827794}) 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2347821954}) 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1942827794}) 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1928277736}) 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1942827794}) 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=96681061}) 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1942827794}) 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2671404041}) 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1942827794}) 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3667380397}) 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1942827794}) 2024-09-15T01:38:40.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=9982039}) 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2448751311}) 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3107991924}) 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2448751311}) 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4233426612}) 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2448751311}) 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4091986432}) 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2448751311}) 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3190001293}) 2024-09-15T01:38:40.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2448751311}) 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2705977154}) 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2448751311}) 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3898915173}) 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2448751311}) 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3876015437}) 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2448751311}) 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3998765030}) 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071754216}) 2024-09-15T01:38:40.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1609152117}) 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071754216}) 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2730464598}) 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071754216}) 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=711652728}) 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071754216}) 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2835096806}) 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071754216}) 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=904504557}) 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071754216}) 2024-09-15T01:38:40.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1296237044}) 2024-09-15T01:38:40.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071754216}) 2024-09-15T01:38:40.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1277924770}) 2024-09-15T01:38:40.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071754216}) 2024-09-15T01:38:40.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1830484049}) 2024-09-15T01:38:40.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2418684975}) 2024-09-15T01:38:40.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=809683711}) 2024-09-15T01:38:40.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2418684975}) 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2946255738}) 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2418684975}) 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=553699896}) 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2418684975}) 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2941711271}) 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2418684975}) 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2617477627}) 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2418684975}) 2024-09-15T01:38:40.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=691037344}) 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2418684975}) 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3096944315}) 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2418684975}) 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=478762471}) 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211113655}) 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=981539938}) 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211113655}) 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3031840339}) 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211113655}) 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=962667175}) 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211113655}) 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=359523632}) 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211113655}) 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3374696026}) 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211113655}) 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4168693609}) 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211113655}) 2024-09-15T01:38:40.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3377075348}) 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211113655}) 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1381418379}) 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1399116883}) 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3214339878}) 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1399116883}) 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2524665765}) 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1399116883}) 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2929157504}) 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1399116883}) 2024-09-15T01:38:40.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2273242905}) 2024-09-15T01:38:40.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1399116883}) 2024-09-15T01:38:40.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=230810026}) 2024-09-15T01:38:40.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1399116883}) 2024-09-15T01:38:40.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2379860653}) 2024-09-15T01:38:40.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1399116883}) 2024-09-15T01:38:40.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1161750865}) 2024-09-15T01:38:40.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1399116883}) 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4276082197}) 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1925192963}) 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1763708488}) 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1925192963}) 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1369206250}) 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1925192963}) 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4181244803}) 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1925192963}) 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1170000341}) 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1925192963}) 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3728549263}) 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1925192963}) 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=297517801}) 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1925192963}) 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2755306932}) 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1925192963}) 2024-09-15T01:38:40.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=501186266}) 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11940997}) 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3544018030}) 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11940997}) 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3189155092}) 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11940997}) 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3492689813}) 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11940997}) 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=617897624}) 2024-09-15T01:38:40.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11940997}) 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=549376200}) 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11940997}) 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2023666056}) 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11940997}) 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=809646592}) 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11940997}) 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1724138921}) 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233273266}) 2024-09-15T01:38:40.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1599073894}) 2024-09-15T01:38:40.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233273266}) 2024-09-15T01:38:40.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2659590111}) 2024-09-15T01:38:40.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233273266}) 2024-09-15T01:38:40.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1733752354}) 2024-09-15T01:38:40.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233273266}) 2024-09-15T01:38:40.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2479528368}) 2024-09-15T01:38:40.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233273266}) 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=722417386}) 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233273266}) 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3927366729}) 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233273266}) 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1088500621}) 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233273266}) 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1856332400}) 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=234999690}) 2024-09-15T01:38:40.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=408862737}) 2024-09-15T01:38:40.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=234999690}) 2024-09-15T01:38:40.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1126082482}) 2024-09-15T01:38:40.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=234999690}) 2024-09-15T01:38:40.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4206519122}) 2024-09-15T01:38:40.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=234999690}) 2024-09-15T01:38:40.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=285702396}) 2024-09-15T01:38:40.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=234999690}) 2024-09-15T01:38:40.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4023040506}) 2024-09-15T01:38:40.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=234999690}) 2024-09-15T01:38:40.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1191404720}) 2024-09-15T01:38:40.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=234999690}) 2024-09-15T01:38:40.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1096018405}) 2024-09-15T01:38:40.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=234999690}) 2024-09-15T01:38:40.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=466549058}) 2024-09-15T01:38:40.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2226196980}) 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=514996015}) 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2226196980}) 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=472644412}) 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2226196980}) 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=459111462}) 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2226196980}) 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=771498486}) 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2226196980}) 2024-09-15T01:38:40.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=64032495}) 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2226196980}) 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1296267683}) 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2226196980}) 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4111822138}) 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2226196980}) 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2260771333}) 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1277382506}) 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2609371595}) 2024-09-15T01:38:40.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1277382506}) 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=334534912}) 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1277382506}) 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3879622094}) 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1277382506}) 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1581708620}) 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1277382506}) 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=408719541}) 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1277382506}) 2024-09-15T01:38:40.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1420289290}) 2024-09-15T01:38:40.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1277382506}) 2024-09-15T01:38:40.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3535184351}) 2024-09-15T01:38:40.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1277382506}) 2024-09-15T01:38:40.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2841699426}) 2024-09-15T01:38:40.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4018587320}) 2024-09-15T01:38:40.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4128813212}) 2024-09-15T01:38:40.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4018587320}) 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=492243662}) 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4018587320}) 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3258795532}) 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4018587320}) 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3691081267}) 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4018587320}) 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=440053560}) 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4018587320}) 2024-09-15T01:38:40.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3283865897}) 2024-09-15T01:38:40.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4018587320}) 2024-09-15T01:38:40.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2889881221}) 2024-09-15T01:38:40.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4018587320}) 2024-09-15T01:38:40.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3184469118}) 2024-09-15T01:38:40.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=212584924}) 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4191846776}) 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=212584924}) 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=450472206}) 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=212584924}) 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3396614165}) 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=212584924}) 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=879238839}) 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=212584924}) 2024-09-15T01:38:40.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1506967897}) 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=212584924}) 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=822808982}) 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=212584924}) 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2679467515}) 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=212584924}) 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2393747819}) 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3406932321}) 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1922103032}) 2024-09-15T01:38:40.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3406932321}) 2024-09-15T01:38:40.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1722936692}) 2024-09-15T01:38:40.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3406932321}) 2024-09-15T01:38:40.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3344924807}) 2024-09-15T01:38:40.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3406932321}) 2024-09-15T01:38:40.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=399091198}) 2024-09-15T01:38:40.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3406932321}) 2024-09-15T01:38:40.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1927042422}) 2024-09-15T01:38:40.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3406932321}) 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2921532195}) 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3406932321}) 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=363532930}) 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3406932321}) 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1722570226}) 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=993308156}) 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2674078185}) 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=993308156}) 2024-09-15T01:38:40.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2889505538}) 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=993308156}) 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1803489758}) 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=993308156}) 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=835139388}) 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=993308156}) 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3284921327}) 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=993308156}) 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1880769566}) 2024-09-15T01:38:40.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=993308156}) 2024-09-15T01:38:40.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1418221579}) 2024-09-15T01:38:40.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=993308156}) 2024-09-15T01:38:40.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=5934644}) 2024-09-15T01:38:40.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2586033949}) 2024-09-15T01:38:40.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=317819705}) 2024-09-15T01:38:40.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2586033949}) 2024-09-15T01:38:40.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=886836544}) 2024-09-15T01:38:40.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2586033949}) 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3099172500}) 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2586033949}) 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3728941615}) 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2586033949}) 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2864074641}) 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2586033949}) 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=110646433}) 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2586033949}) 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4131494053}) 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2586033949}) 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1042568190}) 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3965556712}) 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=322186531}) 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3965556712}) 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1624366556}) 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3965556712}) 2024-09-15T01:38:40.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1453474127}) 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3965556712}) 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=849176860}) 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3965556712}) 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3402375362}) 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3965556712}) 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=952069575}) 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3965556712}) 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4263910991}) 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3965556712}) 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=573354583}) 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3201054414}) 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=725300961}) 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3201054414}) 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1413983587}) 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3201054414}) 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4149947967}) 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3201054414}) 2024-09-15T01:38:40.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=30785604}) 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3201054414}) 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1530007521}) 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3201054414}) 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3575114541}) 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3201054414}) 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1803690696}) 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3201054414}) 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=832739607}) 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1656582443}) 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3367080742}) 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1656582443}) 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1325710401}) 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1656582443}) 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2040371832}) 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1656582443}) 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2541570933}) 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1656582443}) 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1569525471}) 2024-09-15T01:38:40.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1656582443}) 2024-09-15T01:38:40.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=537585329}) 2024-09-15T01:38:40.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1656582443}) 2024-09-15T01:38:40.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2441116863}) 2024-09-15T01:38:40.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1656582443}) 2024-09-15T01:38:40.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1562781019}) 2024-09-15T01:38:40.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2981204113}) 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2976278720}) 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2981204113}) 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1654532097}) 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2981204113}) 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=726876968}) 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2981204113}) 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1478299815}) 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2981204113}) 2024-09-15T01:38:40.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4211456371}) 2024-09-15T01:38:40.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2981204113}) 2024-09-15T01:38:40.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2070588361}) 2024-09-15T01:38:40.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2981204113}) 2024-09-15T01:38:40.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2191994498}) 2024-09-15T01:38:40.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2981204113}) 2024-09-15T01:38:40.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2367678017}) 2024-09-15T01:38:40.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3268679208}) 2024-09-15T01:38:40.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2102641686}) 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3268679208}) 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1154418715}) 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3268679208}) 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1573054258}) 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3268679208}) 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3901240778}) 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3268679208}) 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1425874858}) 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3268679208}) 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2966286423}) 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3268679208}) 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3380399822}) 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3268679208}) 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=304604451}) 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=289659374}) 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=725945660}) 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=289659374}) 2024-09-15T01:38:40.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=868838586}) 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=289659374}) 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4200111150}) 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=289659374}) 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4292583269}) 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=289659374}) 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1108365158}) 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=289659374}) 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3581804667}) 2024-09-15T01:38:40.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=289659374}) 2024-09-15T01:38:40.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3738647741}) 2024-09-15T01:38:40.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=289659374}) 2024-09-15T01:38:40.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220468592}) 2024-09-15T01:38:40.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262556141}) 2024-09-15T01:38:40.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3395660546}) 2024-09-15T01:38:40.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262556141}) 2024-09-15T01:38:40.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2853127990}) 2024-09-15T01:38:40.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262556141}) 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=16421278}) 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262556141}) 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1398591566}) 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262556141}) 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1928860720}) 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262556141}) 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1437244286}) 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262556141}) 2024-09-15T01:38:40.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=791622794}) 2024-09-15T01:38:40.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262556141}) 2024-09-15T01:38:40.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1712164498}) 2024-09-15T01:38:40.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2955629348}) 2024-09-15T01:38:40.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=803958863}) 2024-09-15T01:38:40.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2955629348}) 2024-09-15T01:38:40.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2394164075}) 2024-09-15T01:38:40.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2955629348}) 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3755963566}) 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2955629348}) 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=445888704}) 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2955629348}) 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3914233076}) 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2955629348}) 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3494299231}) 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2955629348}) 2024-09-15T01:38:40.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2157999759}) 2024-09-15T01:38:40.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2955629348}) 2024-09-15T01:38:40.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2509674215}) 2024-09-15T01:38:40.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3630451581}) 2024-09-15T01:38:40.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1966300112}) 2024-09-15T01:38:40.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3630451581}) 2024-09-15T01:38:40.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1059024048}) 2024-09-15T01:38:40.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3630451581}) 2024-09-15T01:38:40.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3799334645}) 2024-09-15T01:38:40.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3630451581}) 2024-09-15T01:38:40.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3342836425}) 2024-09-15T01:38:40.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3630451581}) 2024-09-15T01:38:40.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=756269577}) 2024-09-15T01:38:40.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3630451581}) 2024-09-15T01:38:40.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1345354383}) 2024-09-15T01:38:40.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3630451581}) 2024-09-15T01:38:40.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1339726400}) 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3630451581}) 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=198474223}) 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3634427606}) 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2192993540}) 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3634427606}) 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2533009110}) 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3634427606}) 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1183423079}) 2024-09-15T01:38:40.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3634427606}) 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=176675445}) 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3634427606}) 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=630826993}) 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3634427606}) 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=275036223}) 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3634427606}) 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3798615432}) 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3634427606}) 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4288326405}) 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=634441666}) 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=349796426}) 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=634441666}) 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3878928945}) 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=634441666}) 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3670639423}) 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=634441666}) 2024-09-15T01:38:40.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2751112489}) 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=634441666}) 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1373572379}) 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=634441666}) 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=887689674}) 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=634441666}) 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3046497154}) 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=634441666}) 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2489814564}) 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355011063}) 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3817715820}) 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355011063}) 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=589631127}) 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355011063}) 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=486420349}) 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355011063}) 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1042806679}) 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355011063}) 2024-09-15T01:38:40.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=964769761}) 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355011063}) 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=28906656}) 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355011063}) 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3469415300}) 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355011063}) 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=200007944}) 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898433714}) 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4277494307}) 2024-09-15T01:38:40.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898433714}) 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=17466533}) 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898433714}) 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1421701398}) 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898433714}) 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4230034649}) 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898433714}) 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2199526849}) 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898433714}) 2024-09-15T01:38:40.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=542146742}) 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898433714}) 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1589508508}) 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898433714}) 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2369683432}) 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=592150051}) 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3676319001}) 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=592150051}) 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3787195440}) 2024-09-15T01:38:40.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=592150051}) 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3872174571}) 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=592150051}) 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=349778056}) 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=592150051}) 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1288585196}) 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=592150051}) 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1071665108}) 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=592150051}) 2024-09-15T01:38:40.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2044260300}) 2024-09-15T01:38:40.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=592150051}) 2024-09-15T01:38:40.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1528574927}) 2024-09-15T01:38:40.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2067333893}) 2024-09-15T01:38:40.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3631758320}) 2024-09-15T01:38:40.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2067333893}) 2024-09-15T01:38:40.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3429882548}) 2024-09-15T01:38:40.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2067333893}) 2024-09-15T01:38:40.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=538887002}) 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2067333893}) 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2129383192}) 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2067333893}) 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2886411294}) 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2067333893}) 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4091149422}) 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2067333893}) 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4117600608}) 2024-09-15T01:38:40.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2067333893}) 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4190999869}) 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3766707347}) 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1727994874}) 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3766707347}) 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3484389964}) 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3766707347}) 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4115940913}) 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3766707347}) 2024-09-15T01:38:40.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3193048382}) 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3766707347}) 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3256649187}) 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3766707347}) 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1652921659}) 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3766707347}) 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3941537042}) 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3766707347}) 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1584725788}) 2024-09-15T01:38:40.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=282647763}) 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2587273905}) 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=282647763}) 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1573885329}) 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=282647763}) 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3292326158}) 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=282647763}) 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=757674369}) 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=282647763}) 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1114736583}) 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=282647763}) 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1569806155}) 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=282647763}) 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4142235147}) 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=282647763}) 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3295632674}) 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3593872180}) 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730217367}) 2024-09-15T01:38:40.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3593872180}) 2024-09-15T01:38:40.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=349437205}) 2024-09-15T01:38:40.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3593872180}) 2024-09-15T01:38:40.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1194715882}) 2024-09-15T01:38:40.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3593872180}) 2024-09-15T01:38:40.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=662275391}) 2024-09-15T01:38:40.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3593872180}) 2024-09-15T01:38:40.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=748256635}) 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3593872180}) 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1935713610}) 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3593872180}) 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3406041451}) 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3593872180}) 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=599832418}) 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3610878797}) 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1535790975}) 2024-09-15T01:38:40.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3610878797}) 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2441678231}) 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3610878797}) 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3585902481}) 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3610878797}) 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4074373063}) 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3610878797}) 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3111454156}) 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3610878797}) 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4262141562}) 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3610878797}) 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2046381043}) 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3610878797}) 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3588481134}) 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1636597929}) 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1388641790}) 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1636597929}) 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1533763238}) 2024-09-15T01:38:40.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1636597929}) 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1238456661}) 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1636597929}) 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2770687029}) 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1636597929}) 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3930576704}) 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1636597929}) 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1290013426}) 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1636597929}) 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1518941802}) 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1636597929}) 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2748924856}) 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2735708829}) 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2569375113}) 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2735708829}) 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2282567967}) 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2735708829}) 2024-09-15T01:38:40.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1344934902}) 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2735708829}) 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3484999597}) 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2735708829}) 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2833195228}) 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2735708829}) 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=518487604}) 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2735708829}) 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3030949124}) 2024-09-15T01:38:40.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2735708829}) 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1551799535}) 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3009224588}) 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2601945460}) 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3009224588}) 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2020416631}) 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3009224588}) 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1589271741}) 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3009224588}) 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2517837705}) 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3009224588}) 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1866339708}) 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3009224588}) 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1433538156}) 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3009224588}) 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2452493101}) 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3009224588}) 2024-09-15T01:38:40.654 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1525921519}) 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2215099369}) 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=232556561}) 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2215099369}) 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2966285506}) 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2215099369}) 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2247237327}) 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2215099369}) 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3033400180}) 2024-09-15T01:38:40.655 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2215099369}) 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2602530755}) 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2215099369}) 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3270331443}) 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2215099369}) 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1681567909}) 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2215099369}) 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2037587128}) 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3629420273}) 2024-09-15T01:38:40.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1739717375}) 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3629420273}) 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2541359994}) 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3629420273}) 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3404357267}) 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3629420273}) 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2858287309}) 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3629420273}) 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1105566968}) 2024-09-15T01:38:40.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3629420273}) 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3896031166}) 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3629420273}) 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2648024098}) 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3629420273}) 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3395372523}) 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=4,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=44} crc {logm=313416867,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=840066673,mon_config_key=1763697598,monmap=1281023599,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3823785925,osdmap=2141467485}) 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=14,mon_config_key=4,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=44} crc {logm=313416867,mdsmap=1531151339,mgr=2983835071,mgr_metadata=3150520333,mgrstat=840066673,mon_config_key=3476930649,monmap=2850442478,osd_metadata=4292802864,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2459315477}) 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=4,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=44} crc {logm=313416867,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=840066673,mon_config_key=1763697598,monmap=1281023599,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3823785925,osdmap=2141467485}) 2024-09-15T01:38:40.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=13,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=46} crc {logm=544063487,mdsmap=3140514009,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3223219087,mon_config_key=1111662797,monmap=1281023599,osd_metadata=860431027,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2758264895}) 2024-09-15T01:38:40.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=4,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=44} crc {logm=313416867,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=840066673,mon_config_key=1763697598,monmap=1281023599,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3823785925,osdmap=2141467485}) 2024-09-15T01:38:40.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=14,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=44} crc {logm=2932459072,mdsmap=1531151339,mgr=1785319101,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1092919263,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=3180129280}) 2024-09-15T01:38:40.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=4,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=44} crc {logm=313416867,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=840066673,mon_config_key=1763697598,monmap=1281023599,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3823785925,osdmap=2141467485}) 2024-09-15T01:38:40.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=14,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=45} crc {logm=367574281,mdsmap=3585761405,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1438813526,mon_config_key=1111662797,monmap=1281023599,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=1828353509}) 2024-09-15T01:38:40.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=4,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=44} crc {logm=313416867,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=840066673,mon_config_key=1763697598,monmap=1281023599,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3823785925,osdmap=2141467485}) 2024-09-15T01:38:40.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=44} crc {logm=417988610,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1228274307,mon_config_key=1841931312,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=4120078884}) 2024-09-15T01:38:40.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=4,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=44} crc {logm=313416867,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=840066673,mon_config_key=1763697598,monmap=1281023599,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3823785925,osdmap=2141467485}) 2024-09-15T01:38:40.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=15,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=42} crc {logm=313416867,mdsmap=1531151339,mgr=4079914519,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3145474325,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2059313747}) 2024-09-15T01:38:40.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=4,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=44} crc {logm=313416867,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=840066673,mon_config_key=1763697598,monmap=1281023599,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3823785925,osdmap=2141467485}) 2024-09-15T01:38:40.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=13,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=46} crc {logm=1148438531,mdsmap=1531151339,mgr=4213765795,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2490643071,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=701092230}) 2024-09-15T01:38:40.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=4,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=44} crc {logm=313416867,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=840066673,mon_config_key=1763697598,monmap=1281023599,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3823785925,osdmap=2141467485}) 2024-09-15T01:38:40.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=12,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=46} crc {logm=3965960859,mdsmap=1531151339,mgr=24063485,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1179015590,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=1397838192}) 2024-09-15T01:38:40.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2744381841}) 2024-09-15T01:38:40.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3158683819}) 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2744381841}) 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4159805844}) 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2744381841}) 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=841394979}) 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2744381841}) 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4162134311}) 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2744381841}) 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=4042712589}) 2024-09-15T01:38:40.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2744381841}) 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=505629127}) 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2744381841}) 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3854433329}) 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2744381841}) 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2965336563}) 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3730716510}) 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1135723369}) 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3730716510}) 2024-09-15T01:38:40.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1477785183}) 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3730716510}) 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1784972035}) 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3730716510}) 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2727855207}) 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3730716510}) 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1648540995}) 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3730716510}) 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1568457921}) 2024-09-15T01:38:40.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3730716510}) 2024-09-15T01:38:40.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1951834107}) 2024-09-15T01:38:40.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3730716510}) 2024-09-15T01:38:40.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3922733169}) 2024-09-15T01:38:40.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=58322075}) 2024-09-15T01:38:40.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2741510365}) 2024-09-15T01:38:40.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=58322075}) 2024-09-15T01:38:40.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3850430729}) 2024-09-15T01:38:40.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=58322075}) 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1808167925}) 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=58322075}) 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1702975167}) 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=58322075}) 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3619687435}) 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=58322075}) 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2827789466}) 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=58322075}) 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1193540030}) 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=58322075}) 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=976482529}) 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=42} crc {osdmap=2357724441}) 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=47} crc {osdmap=1315182050}) 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=42} crc {osdmap=2357724441}) 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=45} crc {osdmap=180570925}) 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=42} crc {osdmap=2357724441}) 2024-09-15T01:38:40.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=42} crc {osdmap=277441255}) 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=42} crc {osdmap=2357724441}) 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=45} crc {osdmap=372624145}) 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=42} crc {osdmap=2357724441}) 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=46} crc {osdmap=2044342532}) 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=42} crc {osdmap=2357724441}) 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=46} crc {osdmap=2737156934}) 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=42} crc {osdmap=2357724441}) 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=45} crc {osdmap=306491290}) 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:38:40.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=42} crc {osdmap=2357724441}) 2024-09-15T01:38:40.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:38:40.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=46} crc {osdmap=580894624}) 2024-09-15T01:38:40.668 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=13 2024-09-15T01:38:40.668 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T01:38:40.669 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:38:40.669 INFO:tasks.workunit.client.0.smithi204.stderr:+ export CEPH_ARGS=--rados-mon-op-timeout=300 2024-09-15T01:38:40.669 INFO:tasks.workunit.client.0.smithi204.stderr:+ CEPH_ARGS=--rados-mon-op-timeout=300 2024-09-15T01:38:40.669 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump' 110 2024-09-15T01:38:40.669 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump' 2024-09-15T01:38:40.669 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=110 2024-09-15T01:38:40.669 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump 2024-09-15T01:38:40.669 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump 2024-09-15T01:38:40.670 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump 2024-09-15T01:38:41.489 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:38:41.490 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:38:41.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.592+0000 7fed8d234640 10 monclient: get_monmap_and_config 2024-09-15T01:38:41.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.592+0000 7fed8d234640 10 monclient: build_initial_monmap 2024-09-15T01:38:41.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.592+0000 7fed8d234640 10 monclient: monmap: 2024-09-15T01:38:41.610 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:41.610 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:41.610 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:41.593281+0000 2024-09-15T01:38:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:41.593281+0000 2024-09-15T01:38:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:41.611 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:41.612 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:41.612 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:41.612 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:41.612 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:41.612 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:41.612 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:41.613 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:41.613 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:41.613 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.592+0000 7fed8d234640 1 Processor -- start 2024-09-15T01:38:41.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 1 -- start start 2024-09-15T01:38:41.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 10 monclient: init 2024-09-15T01:38:41.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:41.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 10 monclient: _add_conns ranks=[10,4,3,5,2,9,1,0,6,8,7] 2024-09-15T01:38:41.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fed880a2930 0x7fed88132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 10 monclient(hunting): picked mon.noname-f con 0x7fed880a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:38:41.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fed8809faa0 0x7fed8809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 10 monclient(hunting): picked mon.noname-c con 0x7fed8809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:41.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed880a08c0 0x7fed880a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 10 monclient(hunting): picked mon.noname-g con 0x7fed880a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:41.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8809e0f0 con 0x7fed880a2930 2024-09-15T01:38:41.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8809dd80 con 0x7fed880a08c0 2024-09-15T01:38:41.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed880a2d00 con 0x7fed8809faa0 2024-09-15T01:38:41.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:41.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8d234640 10 monclient(hunting): authenticate will time out at 1126.343495s 2024-09-15T01:38:41.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fed8809faa0 0x7fed8809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fed8809faa0 0x7fed8809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:43264/0 (socket says 172.21.15.178:43264) 2024-09-15T01:38:41.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 1 -- 172.21.15.178:0/1452283430 learned_addr learned my addr 172.21.15.178:0/1452283430 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:41.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 10 monclient(hunting): get_auth_request con 0x7fed8809faa0 auth_method 0 2024-09-15T01:38:41.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:41.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:41.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:41.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed86d9d640 1 --2- 172.21.15.178:0/1452283430 >> v2:172.21.15.204:3300/0 conn(0x7fed880a2930 0x7fed88132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8759e640 1 --2- 172.21.15.178:0/1452283430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed880a08c0 0x7fed880a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed86d9d640 10 monclient(hunting): get_auth_request con 0x7fed880a2930 auth_method 0 2024-09-15T01:38:41.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed86d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed86d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed86d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8759e640 10 monclient(hunting): get_auth_request con 0x7fed880a08c0 auth_method 0 2024-09-15T01:38:41.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 10 monclient(hunting): handle_auth_done global_id 34240 payload 274 2024-09-15T01:38:41.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 1 -- 172.21.15.178:0/1452283430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed880a08c0 msgr2=0x7fed880a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:41.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 1 --2- 172.21.15.178:0/1452283430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed880a08c0 0x7fed880a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 1 -- 172.21.15.178:0/1452283430 >> v2:172.21.15.204:3300/0 conn(0x7fed880a2930 msgr2=0x7fed88132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:41.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 1 --2- 172.21.15.178:0/1452283430 >> v2:172.21.15.204:3300/0 conn(0x7fed880a2930 0x7fed88132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 10 monclient: _finish_hunting 0 2024-09-15T01:38:41.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 1 monclient: found mon.noname-c 2024-09-15T01:38:41.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:41.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:38:41.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed8659c640 1 -- 172.21.15.178:0/1452283430 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8804bf60 con 0x7fed8809faa0 2024-09-15T01:38:41.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.593+0000 7fed86d9d640 1 --2- 172.21.15.178:0/1452283430 >> v2:172.21.15.204:3300/0 conn(0x7fed880a2930 0x7fed88132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:38:41.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed8659c640 1 --2- 172.21.15.178:0/1452283430 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fed8809faa0 0x7fed8809fe70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed70009910 tx=0x7fed700091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1296ed911b3233a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:41.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed85d9b640 1 -- 172.21.15.178:0/1452283430 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed7002ac80 con 0x7fed8809faa0 2024-09-15T01:38:41.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed85d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:41.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed85d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:38:41.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed85d9b640 10 monclient: dump: 2024-09-15T01:38:41.623 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:41.623 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:41.623 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.623 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.623 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:41.624 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:41.624 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:41.624 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:41.624 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:41.624 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:41.624 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:41.625 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:41.625 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:41.625 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:41.625 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:41.625 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed85d9b640 10 monclient: _finish_auth 0 2024-09-15T01:38:41.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed85d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:41.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed8d234640 5 monclient: authenticate success, global_id 34240 2024-09-15T01:38:41.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed85d9b640 1 -- 172.21.15.178:0/1452283430 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7002ae00 con 0x7fed8809faa0 2024-09-15T01:38:41.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed8d234640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:41.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed85d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:41.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed8d234640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:41.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed8d234640 4 set_mon_vals no callback set 2024-09-15T01:38:41.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed85d9b640 1 -- 172.21.15.178:0/1452283430 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed70031ca0 con 0x7fed8809faa0 2024-09-15T01:38:41.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed85d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:41.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed85d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:38:41.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.594+0000 7fed85d9b640 10 monclient: dump: 2024-09-15T01:38:41.628 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:41.628 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:41.628 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.628 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.628 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:41.628 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:41.629 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:41.629 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:41.629 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:41.629 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:41.629 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:41.629 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:41.630 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:41.630 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:41.630 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:41.630 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 10 monclient: shutdown 2024-09-15T01:38:41.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:41.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 1 -- 172.21.15.178:0/1452283430 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fed8809faa0 msgr2=0x7fed8809fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:41.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 1 --2- 172.21.15.178:0/1452283430 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fed8809faa0 0x7fed8809fe70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed70009910 tx=0x7fed700091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 1 -- 172.21.15.178:0/1452283430 shutdown_connections 2024-09-15T01:38:41.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 1 --2- 172.21.15.178:0/1452283430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed880a08c0 0x7fed880a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 1 --2- 172.21.15.178:0/1452283430 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fed8809faa0 0x7fed8809fe70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 1 --2- 172.21.15.178:0/1452283430 >> v2:172.21.15.204:3300/0 conn(0x7fed880a2930 0x7fed88132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 1 -- 172.21.15.178:0/1452283430 >> 172.21.15.178:0/1452283430 conn(0x7fed880a5ed0 msgr2=0x7fed880a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:41.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 1 -- 172.21.15.178:0/1452283430 shutdown_connections 2024-09-15T01:38:41.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 1 -- 172.21.15.178:0/1452283430 wait complete. 2024-09-15T01:38:41.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 10 monclient: build_initial_monmap 2024-09-15T01:38:41.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.595+0000 7fed8d234640 10 monclient: monmap: 2024-09-15T01:38:41.632 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:41.632 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:41.633 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:41.633 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:41.633 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:41.633 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:41.633 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:41.633 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:41.633 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:41.633 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:41.634 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:41.634 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:41.634 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:41.634 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:41.634 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:41.634 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 1 Processor -- start 2024-09-15T01:38:41.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 1 -- start start 2024-09-15T01:38:41.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 10 monclient: init 2024-09-15T01:38:41.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:41.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 10 monclient: _add_conns ranks=[8,1,2,4,5,3,7,0,6] 2024-09-15T01:38:41.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fed8809faa0 0x7fed8811ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 10 monclient(hunting): picked mon.noname-e con 0x7fed8809faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:41.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed880a08c0 0x7fed8809b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 10 monclient(hunting): picked mon.noname-f con 0x7fed880a08c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:38:41.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fed880a2930 0x7fed88131950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 10 monclient(hunting): picked mon.noname-b con 0x7fed880a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:41.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8809d630 con 0x7fed880a08c0 2024-09-15T01:38:41.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8811f030 con 0x7fed880a2930 2024-09-15T01:38:41.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fed8811ed50 con 0x7fed8809faa0 2024-09-15T01:38:41.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:41.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8d234640 10 monclient(hunting): authenticate will time out at 1126.346878s 2024-09-15T01:38:41.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed86d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fed8809faa0 0x7fed8811ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed86d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fed8809faa0 0x7fed8811ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43944/0 (socket says 172.21.15.178:43944) 2024-09-15T01:38:41.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed86d9d640 1 -- 172.21.15.178:0/474140338 learned_addr learned my addr 172.21.15.178:0/474140338 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:41.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed86d9d640 10 monclient(hunting): get_auth_request con 0x7fed8809faa0 auth_method 0 2024-09-15T01:38:41.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed86d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed86d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed86d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8759e640 1 --2- 172.21.15.178:0/474140338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fed880a2930 0x7fed88131950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8759e640 10 monclient(hunting): get_auth_request con 0x7fed880a2930 auth_method 0 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.596+0000 7fed8759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed86d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed86d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed86d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8659c640 1 --2- 172.21.15.178:0/474140338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed880a08c0 0x7fed8809b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8659c640 10 monclient(hunting): get_auth_request con 0x7fed880a08c0 auth_method 0 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 10 monclient(hunting): handle_auth_done global_id 34184 payload 824 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 1 -- 172.21.15.178:0/474140338 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fed8809faa0 msgr2=0x7fed8811ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 1 --2- 172.21.15.178:0/474140338 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fed8809faa0 0x7fed8811ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 1 -- 172.21.15.178:0/474140338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed880a08c0 msgr2=0x7fed8809b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 1 --2- 172.21.15.178:0/474140338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed880a08c0 0x7fed8809b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 10 monclient: _finish_hunting 0 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 1 monclient: found mon.noname-b 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 1 -- 172.21.15.178:0/474140338 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed70009450 con 0x7fed880a2930 2024-09-15T01:38:41.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed86d9d640 1 --2- 172.21.15.178:0/474140338 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fed8809faa0 0x7fed8811ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8659c640 1 --2- 172.21.15.178:0/474140338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed880a08c0 0x7fed8809b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8759e640 1 --2- 172.21.15.178:0/474140338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fed880a2930 0x7fed88131950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fed7800d660 tx=0x7fed78008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed7effd640 1 -- 172.21.15.178:0/474140338 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed78012a70 con 0x7fed880a2930 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed7effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed7effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed7effd640 10 monclient: dump: 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:41.640 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:41.641 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:41.641 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:41.641 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:41.641 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:41.641 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:41.641 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:41.641 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:41.641 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:41.641 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed7effd640 10 monclient: _finish_auth 0 2024-09-15T01:38:41.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed7effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:41.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed7effd640 1 -- 172.21.15.178:0/474140338 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed78012bf0 con 0x7fed880a2930 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8d234640 5 monclient: authenticate success, global_id 34184 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8d234640 10 monclient: _renew_subs 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8d234640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8d234640 1 -- 172.21.15.178:0/474140338 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8811d6a0 con 0x7fed880a2930 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed7effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed7effd640 1 -- 172.21.15.178:0/474140338 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed78012d60 con 0x7fed880a2930 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed7effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed7effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed7effd640 10 monclient: dump: 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.642 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8559a640 4 set_mon_vals no callback set 2024-09-15T01:38:41.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8d234640 10 monclient: _renew_subs 2024-09-15T01:38:41.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8d234640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:41.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.597+0000 7fed8d234640 1 -- 172.21.15.178:0/474140338 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8811ea70 con 0x7fed880a2930 2024-09-15T01:38:41.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.598+0000 7fed7effd640 1 -- 172.21.15.178:0/474140338 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fed780125f0 con 0x7fed880a2930 2024-09-15T01:38:41.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.598+0000 7fed7effd640 1 --2- 172.21.15.178:0/474140338 >> v2:172.21.15.204:6812/28210 conn(0x7fed680396e0 0x7fed6803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.598+0000 7fed8d234640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:41.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.598+0000 7fed8d234640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:41.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.598+0000 7fed8d234640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:41.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.598+0000 7fed7effd640 1 -- 172.21.15.178:0/474140338 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fed7804bf00 con 0x7fed880a2930 2024-09-15T01:38:41.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.598+0000 7fed8d234640 1 -- 172.21.15.178:0/474140338 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed880a1900 con 0x7fed880a2930 2024-09-15T01:38:41.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.598+0000 7fed86d9d640 1 --2- 172.21.15.178:0/474140338 >> v2:172.21.15.204:6812/28210 conn(0x7fed680396e0 0x7fed6803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.598+0000 7fed86d9d640 10 monclient: get_auth_request con 0x7fed680396e0 auth_method 0 2024-09-15T01:38:41.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.599+0000 7fed86d9d640 1 --2- 172.21.15.178:0/474140338 >> v2:172.21.15.204:6812/28210 conn(0x7fed680396e0 0x7fed6803bb90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fed80007c10 tx=0x7fed80009a90 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:41.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.602+0000 7fed7effd640 1 -- 172.21.15.178:0/474140338 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fed78014430 con 0x7fed880a2930 2024-09-15T01:38:41.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.602+0000 7fed7effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:41.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.602+0000 7fed7effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:41.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.814+0000 7fed8d234640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:38:41.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.814+0000 7fed8d234640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:38:41.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.814+0000 7fed8d234640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:41.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.814+0000 7fed8d234640 1 -- 172.21.15.178:0/474140338 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fed8804b720 con 0x7fed880a2930 2024-09-15T01:38:41.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.814+0000 7fed7effd640 1 -- 172.21.15.178:0/474140338 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7fed7801d3c0 con 0x7fed880a2930 2024-09-15T01:38:41.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.814+0000 7fed7effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:38:41.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.814+0000 7fed7effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:41.817 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":76,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":6,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:38:41.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 1 -- 172.21.15.178:0/474140338 >> v2:172.21.15.204:6812/28210 conn(0x7fed680396e0 msgr2=0x7fed6803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:41.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 1 --2- 172.21.15.178:0/474140338 >> v2:172.21.15.204:6812/28210 conn(0x7fed680396e0 0x7fed6803bb90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fed80007c10 tx=0x7fed80009a90 comp rx=0 tx=0).stop 2024-09-15T01:38:41.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 10 monclient: shutdown 2024-09-15T01:38:41.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:41.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 1 -- 172.21.15.178:0/474140338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fed880a2930 msgr2=0x7fed88131950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:41.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 1 --2- 172.21.15.178:0/474140338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fed880a2930 0x7fed88131950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fed7800d660 tx=0x7fed78008040 comp rx=0 tx=0).stop 2024-09-15T01:38:41.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 1 -- 172.21.15.178:0/474140338 shutdown_connections 2024-09-15T01:38:41.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 1 --2- 172.21.15.178:0/474140338 >> v2:172.21.15.204:6812/28210 conn(0x7fed680396e0 0x7fed6803bb90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 1 --2- 172.21.15.178:0/474140338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed880a08c0 0x7fed8809b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 1 --2- 172.21.15.178:0/474140338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fed880a2930 0x7fed88131950 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 1 --2- 172.21.15.178:0/474140338 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fed8809faa0 0x7fed8811ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 1 -- 172.21.15.178:0/474140338 >> 172.21.15.178:0/474140338 conn(0x7fed880a5ed0 msgr2=0x7fed880a5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:41.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 1 -- 172.21.15.178:0/474140338 shutdown_connections 2024-09-15T01:38:41.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.816+0000 7fed8d234640 1 -- 172.21.15.178:0/474140338 wait complete. 2024-09-15T01:38:41.829 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:38:41.829 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T01:38:41.829 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:38:41.829 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:38:41.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.920+0000 7f4b5955d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:41.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.920+0000 7f4b5955d640 10 monclient: build_initial_monmap 2024-09-15T01:38:41.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.920+0000 7f4b5955d640 10 monclient: monmap: 2024-09-15T01:38:41.922 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:41.922 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:41.922 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:41.921094+0000 2024-09-15T01:38:41.922 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:41.921094+0000 2024-09-15T01:38:41.922 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:41.922 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:41.923 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:41.923 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:41.923 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:41.923 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:41.923 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:41.923 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:41.923 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:41.923 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:41.923 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:41.923 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:41.923 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:41.923 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.920+0000 7f4b5955d640 1 Processor -- start 2024-09-15T01:38:41.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.920+0000 7f4b5955d640 1 -- start start 2024-09-15T01:38:41.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.920+0000 7f4b5955d640 10 monclient: init 2024-09-15T01:38:41.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:41.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 10 monclient: _add_conns ranks=[9,8,2,3,7,0,10,6,5,1,4] 2024-09-15T01:38:41.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4b540a2930 0x7f4b54132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 10 monclient(hunting): picked mon.noname-a con 0x7f4b540a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:38:41.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b5409faa0 0x7f4b5409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 10 monclient(hunting): picked mon.noname-e con 0x7f4b5409faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b540a08c0 0x7f4b540a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 10 monclient(hunting): picked mon.noname-b con 0x7f4b540a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5409e0f0 con 0x7f4b540a2930 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5409dd80 con 0x7f4b540a08c0 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540a2d00 con 0x7f4b5409faa0 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b5955d640 10 monclient(hunting): authenticate will time out at 1126.671389s 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b527fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b5409faa0 0x7f4b5409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b540a08c0 0x7f4b540a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b52ffd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4b540a2930 0x7f4b54132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b540a08c0 0x7f4b540a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42962/0 (socket says 172.21.15.178:42962) 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b527fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b5409faa0 0x7f4b5409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43948/0 (socket says 172.21.15.178:43948) 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 1 -- 172.21.15.178:0/534202735 learned_addr learned my addr 172.21.15.178:0/534202735 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 10 monclient(hunting): get_auth_request con 0x7f4b540a08c0 auth_method 0 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request con 0x7f4b540a2930 auth_method 0 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b52ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b527fc640 10 monclient(hunting): get_auth_request con 0x7f4b5409faa0 auth_method 0 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 10 monclient(hunting): handle_auth_done global_id 34193 payload 274 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 1 -- 172.21.15.178:0/534202735 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b5409faa0 msgr2=0x7f4b5409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 1 --2- 172.21.15.178:0/534202735 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b5409faa0 0x7f4b5409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 1 -- 172.21.15.178:0/534202735 >> v2:172.21.15.178:3300/0 conn(0x7f4b540a2930 msgr2=0x7f4b54132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 1 --2- 172.21.15.178:0/534202735 >> v2:172.21.15.178:3300/0 conn(0x7f4b540a2930 0x7f4b54132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 10 monclient: _finish_hunting 0 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 1 monclient: found mon.noname-b 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b537fe640 1 -- 172.21.15.178:0/534202735 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5404bf60 con 0x7f4b540a08c0 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b527fc640 1 --2- 172.21.15.178:0/534202735 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b5409faa0 0x7f4b5409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.921+0000 7f4b52ffd640 1 --2- 172.21.15.178:0/534202735 >> v2:172.21.15.178:3300/0 conn(0x7f4b540a2930 0x7f4b54132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b537fe640 1 --2- 172.21.15.178:0/534202735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b540a08c0 0x7f4b540a0d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009910 tx=0x7f4b440091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69102909a3af7a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b51ffb640 1 -- 172.21.15.178:0/534202735 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b4402ac80 con 0x7f4b540a08c0 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b51ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b51ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b51ffb640 10 monclient: dump: 2024-09-15T01:38:41.940 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:41.941 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b51ffb640 10 monclient: _finish_auth 0 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b51ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b51ffb640 1 -- 172.21.15.178:0/534202735 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4402ae00 con 0x7f4b540a08c0 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b5955d640 5 monclient: authenticate success, global_id 34193 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b5955d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b51ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b5955d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b51ffb640 1 -- 172.21.15.178:0/534202735 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b44031ca0 con 0x7f4b540a08c0 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b5955d640 4 set_mon_vals no callback set 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b51ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b51ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:41.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.922+0000 7f4b51ffb640 10 monclient: dump: 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:41.943 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:41.944 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:41.944 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.923+0000 7f4b5955d640 10 monclient: shutdown 2024-09-15T01:38:41.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.923+0000 7f4b5955d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:41.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.923+0000 7f4b5955d640 1 -- 172.21.15.178:0/534202735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b540a08c0 msgr2=0x7f4b540a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:41.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.923+0000 7f4b5955d640 1 --2- 172.21.15.178:0/534202735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b540a08c0 0x7f4b540a0d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009910 tx=0x7f4b440091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.923+0000 7f4b5955d640 1 -- 172.21.15.178:0/534202735 shutdown_connections 2024-09-15T01:38:41.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.923+0000 7f4b5955d640 1 --2- 172.21.15.178:0/534202735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b540a08c0 0x7f4b540a0d60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.923+0000 7f4b5955d640 1 --2- 172.21.15.178:0/534202735 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4b5409faa0 0x7f4b5409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.923+0000 7f4b5955d640 1 --2- 172.21.15.178:0/534202735 >> v2:172.21.15.178:3300/0 conn(0x7f4b540a2930 0x7f4b54132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.923+0000 7f4b5955d640 1 -- 172.21.15.178:0/534202735 >> 172.21.15.178:0/534202735 conn(0x7f4b540a5ed0 msgr2=0x7f4b540a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.923+0000 7f4b5955d640 1 -- 172.21.15.178:0/534202735 shutdown_connections 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.923+0000 7f4b5955d640 1 -- 172.21.15.178:0/534202735 wait complete. 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.923+0000 7f4b5955d640 10 monclient: build_initial_monmap 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 10 monclient: monmap: 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:41.945 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 1 Processor -- start 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 1 -- start start 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 10 monclient: init 2024-09-15T01:38:41.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:41.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 10 monclient: _add_conns ranks=[0,7,6,5,2,3,4,1,8] 2024-09-15T01:38:41.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5409faa0 0x7f4b5409b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 10 monclient(hunting): picked mon.noname-a con 0x7f4b5409faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:38:41.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b540a08c0 0x7f4b54131800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 10 monclient(hunting): picked mon.noname-i con 0x7f4b540a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:41.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4b540a2930 0x7f4b5411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 10 monclient(hunting): picked mon.noname-d con 0x7f4b540a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:41.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540a2390 con 0x7f4b5409faa0 2024-09-15T01:38:41.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5409d630 con 0x7f4b540a2930 2024-09-15T01:38:41.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5411ec00 con 0x7f4b540a08c0 2024-09-15T01:38:41.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:41.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b5955d640 10 monclient(hunting): authenticate will time out at 1126.674864s 2024-09-15T01:38:41.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b52ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5409faa0 0x7f4b5409b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b52ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5409faa0 0x7f4b5409b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49760/0 (socket says 172.21.15.178:49760) 2024-09-15T01:38:41.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b52ffd640 1 -- 172.21.15.178:0/2927155769 learned_addr learned my addr 172.21.15.178:0/2927155769 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:41.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request con 0x7f4b5409faa0 auth_method 0 2024-09-15T01:38:41.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b52ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.924+0000 7f4b537fe640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4b540a2930 0x7f4b5411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b537fe640 10 monclient(hunting): get_auth_request con 0x7f4b540a2930 auth_method 0 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:41.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:41.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b527fc640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b540a08c0 0x7f4b54131800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b527fc640 10 monclient(hunting): get_auth_request con 0x7f4b540a08c0 auth_method 0 2024-09-15T01:38:41.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 10 monclient(hunting): handle_auth_done global_id 34272 payload 824 2024-09-15T01:38:41.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 1 -- 172.21.15.178:0/2927155769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b540a08c0 msgr2=0x7f4b54131800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:41.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4b540a08c0 0x7f4b54131800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 1 -- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4b540a2930 msgr2=0x7f4b5411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:41.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4b540a2930 0x7f4b5411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:38:41.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 1 monclient: found mon.noname-a 2024-09-15T01:38:41.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:41.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:38:41.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 1 -- 172.21.15.178:0/2927155769 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c009390 con 0x7f4b5409faa0 2024-09-15T01:38:41.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b537fe640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4b540a2930 0x7f4b5411c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:41.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b52ffd640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5409faa0 0x7f4b5409b170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00c170 tx=0x7f4b4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:41.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b4c014900 con 0x7f4b5409faa0 2024-09-15T01:38:41.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b4affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b4affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b4affd640 10 monclient: dump: 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b4affd640 10 monclient: _finish_auth 0 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b4affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c00f040 con 0x7f4b5409faa0 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b5955d640 5 monclient: authenticate success, global_id 34272 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b4affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b4c0133e0 con 0x7f4b5409faa0 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b5955d640 10 monclient: _renew_subs 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b5955d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:41.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b5955d640 1 -- 172.21.15.178:0/2927155769 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4c009a20 con 0x7f4b5409faa0 2024-09-15T01:38:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b4affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b4affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b4affd640 10 monclient: dump: 2024-09-15T01:38:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b517fa640 4 set_mon_vals no callback set 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b5955d640 10 monclient: _renew_subs 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b5955d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.925+0000 7f4b5955d640 1 -- 172.21.15.178:0/2927155769 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b44009450 con 0x7f4b5409faa0 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4b4c003020 con 0x7f4b5409faa0 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b5955d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b5955d640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b54131d40 0x7f4b5411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b5955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b5955d640 1 -- 172.21.15.178:0/2927155769 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54132110 con 0x7f4b54131d40 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b527fc640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b54131d40 0x7f4b5411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b527fc640 10 monclient: get_auth_request con 0x7f4b54131d40 auth_method 0 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b4affd640 1 --2- 172.21.15.178:0/2927155769 >> v2:172.21.15.204:6812/28210 conn(0x7f4b340397e0 0x7f4b3403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f4b4c04c680 con 0x7f4b5409faa0 2024-09-15T01:38:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b527fc640 10 monclient(hunting): handle_auth_done global_id 34281 payload 824 2024-09-15T01:38:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b527fc640 1 -- 172.21.15.178:0/2927155769 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b5404c4a0 con 0x7f4b54131d40 2024-09-15T01:38:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b537fe640 1 --2- 172.21.15.178:0/2927155769 >> v2:172.21.15.204:6812/28210 conn(0x7f4b340397e0 0x7f4b3403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b537fe640 10 monclient: get_auth_request con 0x7f4b340397e0 auth_method 0 2024-09-15T01:38:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.926+0000 7f4b527fc640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b54131d40 0x7f4b5411ca70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c011810 tx=0x7f4b3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.927+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b3c00fd30 con 0x7f4b54131d40 2024-09-15T01:38:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.927+0000 7f4b4affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.927+0000 7f4b4affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.927+0000 7f4b4affd640 10 monclient: dump: 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:41.959 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.927+0000 7f4b537fe640 1 --2- 172.21.15.178:0/2927155769 >> v2:172.21.15.204:6812/28210 conn(0x7f4b340397e0 0x7f4b3403bc90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4b44034000 tx=0x7f4b44032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.927+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f4b3c00fe90 con 0x7f4b54131d40 2024-09-15T01:38:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.927+0000 7f4b4affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.927+0000 7f4b4affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.927+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b54131d40 msgr2=0x7f4b5411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.927+0000 7f4b4affd640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b54131d40 0x7f4b5411ca70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c011810 tx=0x7f4b3c008040 comp rx=0 tx=0).stop 2024-09-15T01:38:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.927+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b54131d40 msgr2=0x7f4b5411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.927+0000 7f4b4affd640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b54131d40 0x7f4b5411ca70 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c011810 tx=0x7f4b3c008040 comp rx=0 tx=0).stop 2024-09-15T01:38:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.932+0000 7f4b5955d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.932+0000 7f4b5955d640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5411cfb0 0x7f4b54096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.932+0000 7f4b5955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.932+0000 7f4b5955d640 1 -- 172.21.15.178:0/2927155769 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5404bc80 con 0x7f4b5411cfb0 2024-09-15T01:38:41.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b527fc640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5411cfb0 0x7f4b54096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b527fc640 10 monclient: get_auth_request con 0x7f4b5411cfb0 auth_method 0 2024-09-15T01:38:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b527fc640 10 monclient(hunting): handle_auth_done global_id 34290 payload 824 2024-09-15T01:38:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b527fc640 1 -- 172.21.15.178:0/2927155769 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4b5405ed70 con 0x7f4b5411cfb0 2024-09-15T01:38:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b527fc640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5411cfb0 0x7f4b54096a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c011810 tx=0x7f4b3c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b3c022070 con 0x7f4b5411cfb0 2024-09-15T01:38:41.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b4affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b4affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b4affd640 10 monclient: dump: 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:41.962 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:41.963 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f4b3c00f300 con 0x7f4b5411cfb0 2024-09-15T01:38:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b4affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b4affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5411cfb0 msgr2=0x7f4b54096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b4affd640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5411cfb0 0x7f4b54096a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c011810 tx=0x7f4b3c01c000 comp rx=0 tx=0).stop 2024-09-15T01:38:41.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b4affd640 1 -- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5411cfb0 msgr2=0x7f4b54096a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:41.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.933+0000 7f4b4affd640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5411cfb0 0x7f4b54096a30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:41.964 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:41.964 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:41.964 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:41.964 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:41.964 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:41.964 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:41.964 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:41.965 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:41.966 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:41.966 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:41.966 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:41.966 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:41.966 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:41.966 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:41.966 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:41.966 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:41.966 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:41.967 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:41.967 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:41.967 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:41.967 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:41.967 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:41.967 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:41.967 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:41.967 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:41.968 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:41.968 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:41.968 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:41.968 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:41.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.968 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:41.969 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:41.969 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:41.969 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:41.969 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:41.969 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:41.969 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:41.969 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:41.970 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:41.970 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:41.970 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:41.970 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:41.970 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:41.970 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:41.970 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:41.970 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:41.970 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:41.970 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:41.970 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:41.970 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:41.971 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:41.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.973 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:41.974 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:41.975 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:41.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.975 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.975 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:41.975 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:41.975 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:41.975 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:41.975 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.975 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:41.975 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:41.975 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.976 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.976 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:41.976 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:41.976 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.976 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:41.976 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:41.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.976 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:41.976 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:41.977 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:41.977 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:41.977 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:41.977 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.977 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.977 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:41.977 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:41.977 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:41.977 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.978 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:41.979 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:41.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:41.981 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:41.982 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:41.983 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.983 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:41.983 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:41.983 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.983 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.983 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:41.983 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:41.983 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.983 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:41.983 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:41.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.984 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:41.984 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:41.984 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:41.984 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:41.984 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:41.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.984 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.984 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:41.984 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:41.984 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:41.984 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:41.985 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:41.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:41.988 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:38:41.989 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:41.989 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:41.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:41.989 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:41.989 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:41.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.934+0000 7f4b5955d640 1 -- 172.21.15.178:0/2927155769 >> v2:172.21.15.204:6812/28210 conn(0x7f4b340397e0 msgr2=0x7f4b3403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:41.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.934+0000 7f4b5955d640 1 --2- 172.21.15.178:0/2927155769 >> v2:172.21.15.204:6812/28210 conn(0x7f4b340397e0 0x7f4b3403bc90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4b44034000 tx=0x7f4b44032000 comp rx=0 tx=0).stop 2024-09-15T01:38:41.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.934+0000 7f4b5955d640 10 monclient: shutdown 2024-09-15T01:38:41.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.934+0000 7f4b5955d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:41.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.934+0000 7f4b5955d640 1 -- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5409faa0 msgr2=0x7f4b5409b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:41.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.934+0000 7f4b5955d640 1 --2- 172.21.15.178:0/2927155769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b5409faa0 0x7f4b5409b170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00c170 tx=0x7f4b4c00b040 comp rx=0 tx=0).stop 2024-09-15T01:38:41.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.934+0000 7f4b537fe640 1 -- 172.21.15.178:0/2927155769 reap_dead start 2024-09-15T01:38:41.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.934+0000 7f4b537fe640 1 -- 172.21.15.178:0/2927155769 reap_dead start 2024-09-15T01:38:41.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.934+0000 7f4b5955d640 1 -- 172.21.15.178:0/2927155769 shutdown_connections 2024-09-15T01:38:41.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.934+0000 7f4b5955d640 1 -- 172.21.15.178:0/2927155769 >> 172.21.15.178:0/2927155769 conn(0x7f4b540a5ed0 msgr2=0x7f4b5409e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:41.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.934+0000 7f4b5955d640 1 -- 172.21.15.178:0/2927155769 shutdown_connections 2024-09-15T01:38:41.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:41.934+0000 7f4b5955d640 1 -- 172.21.15.178:0/2927155769 wait complete. 2024-09-15T01:38:41.991 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:41.992 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:38:41.992 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:38:42.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.083+0000 7f3b14d9d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:42.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.083+0000 7f3b14d9d640 10 monclient: build_initial_monmap 2024-09-15T01:38:42.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.083+0000 7f3b14d9d640 10 monclient: monmap: 2024-09-15T01:38:42.085 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:42.085 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:42.083997+0000 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:42.083997+0000 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:42.086 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:42.087 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:42.087 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.083+0000 7f3b14d9d640 1 Processor -- start 2024-09-15T01:38:42.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.083+0000 7f3b14d9d640 1 -- start start 2024-09-15T01:38:42.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.083+0000 7f3b14d9d640 10 monclient: init 2024-09-15T01:38:42.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 10 monclient: _add_conns ranks=[7,0,10,8,3,2,6,9,4,1,5] 2024-09-15T01:38:42.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3b100a2930 0x7f3b10132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 10 monclient(hunting): picked mon.noname-i con 0x7f3b100a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:42.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3b1009faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:42.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3b1009fe70 0x7f3b100a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 10 monclient(hunting): picked mon.noname-f con 0x7f3b1009fe70 addr v2:172.21.15.204:3300/0 2024-09-15T01:38:42.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1004bf60 con 0x7f3b1009faa0 2024-09-15T01:38:42.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1009dd80 con 0x7f3b1009fe70 2024-09-15T01:38:42.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100a2d00 con 0x7f3b100a2930 2024-09-15T01:38:42.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b14d9d640 10 monclient(hunting): authenticate will time out at 1126.834416s 2024-09-15T01:38:42.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0dd9b640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f3b1009faa0 0x7f3b100a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:50816/0 (socket says 172.21.15.178:50816) 2024-09-15T01:38:42.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0dd9b640 1 -- 172.21.15.178:0/2004636837 learned_addr learned my addr 172.21.15.178:0/2004636837 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:38:42.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0ed9d640 1 --2- 172.21.15.178:0/2004636837 >> v2:172.21.15.204:3300/0 conn(0x7f3b1009fe70 0x7f3b100a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0e59c640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3b100a2930 0x7f3b10132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0e59c640 10 monclient(hunting): get_auth_request con 0x7f3b100a2930 auth_method 0 2024-09-15T01:38:42.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465613281 0 0) 0x7f3b1004bf60 con 0x7f3b1009faa0 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0d59a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0d59a640 10 monclient(hunting): my global_id is 34299 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3afc0034e0 con 0x7f3b1009faa0 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0ed9d640 10 monclient(hunting): get_auth_request con 0x7f3b1009fe70 auth_method 0 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.084+0000 7f3b0ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3914179557 0 0) 0x7f3afc0034e0 con 0x7f3b1009faa0 2024-09-15T01:38:42.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3b100a2930 msgr2=0x7f3b10132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:42.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3b100a2930 0x7f3b10132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 >> v2:172.21.15.204:3300/0 conn(0x7f3b1009fe70 msgr2=0x7f3b100a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:42.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 --2- 172.21.15.178:0/2004636837 >> v2:172.21.15.204:3300/0 conn(0x7f3b1009fe70 0x7f3b100a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:38:42.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:38:42.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b100a0310 con 0x7f3b1009faa0 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f3af80018f0 con 0x7f3b1009faa0 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b14d9d640 5 monclient: authenticate success, global_id 34299 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b14d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient: dump: 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 >> v1:172.21.15.178:6789/0 conn(0x7f3b1009faa0 legacy=0x7f3b100a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient: _add_conns ranks=[7,5,3,8,1,2,6,4,0] 2024-09-15T01:38:42.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3afc004220 0x7f3afc004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient(hunting): picked mon.i con 0x7f3afc004220 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:42.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3afc004cc0 0x7f3afc005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient(hunting): picked mon.h con 0x7f3afc004cc0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:42.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3afc009a60 0x7f3afc00bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient(hunting): picked mon.g con 0x7f3afc009a60 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:42.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1009dd80 con 0x7f3afc009a60 2024-09-15T01:38:42.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100a2d00 con 0x7f3afc004cc0 2024-09-15T01:38:42.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc00c7d0 con 0x7f3afc004220 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0d59a640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b14d9d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0e59c640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3b100a2930 0x7f3b10132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3afc009a60 0x7f3afc00bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0dd9b640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3afc004cc0 0x7f3afc005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 10 monclient(hunting): get_auth_request con 0x7f3afc009a60 auth_method 0 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0dd9b640 10 monclient(hunting): get_auth_request con 0x7f3afc004cc0 auth_method 0 2024-09-15T01:38:42.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0dd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0e59c640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3afc004220 0x7f3afc004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0e59c640 10 monclient(hunting): get_auth_request con 0x7f3afc004220 auth_method 0 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0e59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:38:42.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 10 monclient(hunting): handle_auth_done global_id 34299 payload 293 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 1 -- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3afc004220 msgr2=0x7f3afc004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3afc004220 0x7f3afc004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 1 -- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3afc004cc0 msgr2=0x7f3afc005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3afc004cc0 0x7f3afc005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 1 monclient: found mon.g 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 1 -- 172.21.15.178:0/2004636837 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc0034e0 con 0x7f3afc009a60 2024-09-15T01:38:42.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0ed9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0dd9b640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3afc004cc0 0x7f3afc005130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.085+0000 7f3b0e59c640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3afc004220 0x7f3afc004780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b0ed9d640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3afc009a60 0x7f3afc00bec0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009b70 tx=0x7f3b000090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=e38ade70d5fb998e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b00031880 con 0x7f3afc009a60 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b0d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b0d59a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b0d59a640 10 monclient: dump: 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0002d050 con 0x7f3afc009a60 2024-09-15T01:38:42.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b14d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b0d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b14d9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b14d9d640 4 set_mon_vals no callback set 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b0d59a640 1 -- 172.21.15.178:0/2004636837 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b00031a00 con 0x7f3afc009a60 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b0d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b0d59a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.086+0000 7f3b0d59a640 10 monclient: dump: 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.087+0000 7f3b14d9d640 10 monclient: shutdown 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.087+0000 7f3b14d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.087+0000 7f3b14d9d640 1 -- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3afc009a60 msgr2=0x7f3afc00bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.087+0000 7f3b14d9d640 1 --2- 172.21.15.178:0/2004636837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3afc009a60 0x7f3afc00bec0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009b70 tx=0x7f3b000090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.087+0000 7f3b0ed9d640 1 -- 172.21.15.178:0/2004636837 reap_dead start 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.087+0000 7f3b14d9d640 1 -- 172.21.15.178:0/2004636837 shutdown_connections 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.087+0000 7f3b14d9d640 1 -- 172.21.15.178:0/2004636837 >> 172.21.15.178:0/2004636837 conn(0x7f3b100a5ed0 msgr2=0x7f3b100a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.087+0000 7f3b14d9d640 1 -- 172.21.15.178:0/2004636837 shutdown_connections 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.087+0000 7f3b14d9d640 1 -- 172.21.15.178:0/2004636837 wait complete. 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.087+0000 7f3b14d9d640 10 monclient: build_initial_monmap 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.087+0000 7f3b14d9d640 10 monclient: monmap: 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 1 Processor -- start 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 1 -- start start 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 10 monclient: init 2024-09-15T01:38:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 10 monclient: _add_conns ranks=[5,6,0,3,2,7,8,4,1] 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3afc009a60 0x7f3b10141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 10 monclient(hunting): picked mon.noname-h con 0x7f3afc009a60 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b10120ac0 0x7f3b101318d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 10 monclient(hunting): picked mon.noname-d con 0x7f3b10120ac0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10131e10 0x7f3b1011e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 10 monclient(hunting): picked mon.noname-a con 0x7f3b10131e10 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1011ff80 con 0x7f3b10131e10 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1009d630 con 0x7f3afc009a60 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1009f870 con 0x7f3b10120ac0 2024-09-15T01:38:42.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b14d9d640 10 monclient(hunting): authenticate will time out at 1126.838623s 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10131e10 0x7f3b1011e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0dd9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b10120ac0 0x7f3b101318d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0ed9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10131e10 0x7f3b1011e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49798/0 (socket says 172.21.15.178:49798) 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0ed9d640 1 -- 172.21.15.178:0/3803918262 learned_addr learned my addr 172.21.15.178:0/3803918262 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0ed9d640 10 monclient(hunting): get_auth_request con 0x7f3b10131e10 auth_method 0 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0dd9b640 10 monclient(hunting): get_auth_request con 0x7f3b10120ac0 auth_method 0 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0e59c640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3afc009a60 0x7f3b10141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0e59c640 10 monclient(hunting): get_auth_request con 0x7f3afc009a60 auth_method 0 2024-09-15T01:38:42.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.088+0000 7f3b0ed9d640 10 monclient(hunting): handle_auth_done global_id 34308 payload 824 2024-09-15T01:38:42.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b0ed9d640 1 -- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b10120ac0 msgr2=0x7f3b101318d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b0ed9d640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b10120ac0 0x7f3b101318d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b0ed9d640 1 -- 172.21.15.178:0/3803918262 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3afc009a60 msgr2=0x7f3b10141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b0ed9d640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3afc009a60 0x7f3b10141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b0ed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b0ed9d640 1 monclient: found mon.noname-a 2024-09-15T01:38:42.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b0ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b0ed9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b0ed9d640 1 -- 172.21.15.178:0/3803918262 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b101428b0 con 0x7f3b10131e10 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b0ed9d640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10131e10 0x7f3b1011e3b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3b00031e30 tx=0x7f3b00035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b00042890 con 0x7f3b10131e10 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b067fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b067fc640 10 monclient: dump: 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.123 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.124 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.124 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.124 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.124 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.124 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.124 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.124 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.124 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.124 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.124 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b067fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b00042a30 con 0x7f3b10131e10 2024-09-15T01:38:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b14d9d640 5 monclient: authenticate success, global_id 34308 2024-09-15T01:38:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b14d9d640 10 monclient: _renew_subs 2024-09-15T01:38:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b14d9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b14d9d640 1 -- 172.21.15.178:0/3803918262 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b10121a20 con 0x7f3b10131e10 2024-09-15T01:38:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b067fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b00042ba0 con 0x7f3b10131e10 2024-09-15T01:38:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b067fc640 10 monclient: dump: 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b14d9d640 10 monclient: _renew_subs 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b14d9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b14d9d640 1 -- 172.21.15.178:0/3803918262 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b10142120 con 0x7f3b10131e10 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.089+0000 7f3b0cd99640 4 set_mon_vals no callback set 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b14d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b14d9d640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b100298c0 0x7f3b10029c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3b00042d00 con 0x7f3b10131e10 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b14d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b14d9d640 1 -- 172.21.15.178:0/3803918262 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b10121bb0 con 0x7f3b100298c0 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0e59c640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b100298c0 0x7f3b10029c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0e59c640 10 monclient: get_auth_request con 0x7f3b100298c0 auth_method 0 2024-09-15T01:38:42.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b067fc640 1 --2- 172.21.15.178:0/3803918262 >> v2:172.21.15.204:6812/28210 conn(0x7f3af00396e0 0x7f3af003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f3b00000e40 con 0x7f3b10131e10 2024-09-15T01:38:42.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_done global_id 34202 payload 824 2024-09-15T01:38:42.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0e59c640 1 -- 172.21.15.178:0/3803918262 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b1004c4a0 con 0x7f3b100298c0 2024-09-15T01:38:42.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0dd9b640 1 --2- 172.21.15.178:0/3803918262 >> v2:172.21.15.204:6812/28210 conn(0x7f3af00396e0 0x7f3af003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0dd9b640 10 monclient: get_auth_request con 0x7f3af00396e0 auth_method 0 2024-09-15T01:38:42.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b0e59c640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b100298c0 0x7f3b10029c90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b100a2f70 tx=0x7f3b0800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b0800e040 con 0x7f3b100298c0 2024-09-15T01:38:42.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b067fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:42.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.090+0000 7f3b067fc640 10 monclient: dump: 2024-09-15T01:38:42.129 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.129 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.129 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.130 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.130 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.130 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.130 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.130 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.130 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.130 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.130 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.130 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.130 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.130 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.130 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.091+0000 7f3b0dd9b640 1 --2- 172.21.15.178:0/3803918262 >> v2:172.21.15.204:6812/28210 conn(0x7f3af00396e0 0x7f3af003bb90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3af8000c00 tx=0x7f3af8001670 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.091+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f3b0800f040 con 0x7f3b100298c0 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.091+0000 7f3b067fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.091+0000 7f3b067fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.091+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b100298c0 msgr2=0x7f3b10029c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.091+0000 7f3b067fc640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b100298c0 0x7f3b10029c90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b100a2f70 tx=0x7f3b0800b040 comp rx=0 tx=0).stop 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.091+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b100298c0 msgr2=0x7f3b10029c90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.091+0000 7f3b067fc640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b100298c0 0x7f3b10029c90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.096+0000 7f3b14d9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.096+0000 7f3b14d9d640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b1002a1d0 0x7f3b1002a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.096+0000 7f3b14d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.096+0000 7f3b14d9d640 1 -- 172.21.15.178:0/3803918262 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b10132de0 con 0x7f3b1002a1d0 2024-09-15T01:38:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b0e59c640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b1002a1d0 0x7f3b1002a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b0e59c640 10 monclient: get_auth_request con 0x7f3b1002a1d0 auth_method 0 2024-09-15T01:38:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b0e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b0e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b0e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b0e59c640 10 monclient(hunting): handle_auth_done global_id 34211 payload 824 2024-09-15T01:38:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b0e59c640 1 -- 172.21.15.178:0/3803918262 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3b10122070 con 0x7f3b1002a1d0 2024-09-15T01:38:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b0e59c640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b1002a1d0 0x7f3b1002a5d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3b08012400 tx=0x7f3b0801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b08016e70 con 0x7f3b1002a1d0 2024-09-15T01:38:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b067fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b067fc640 10 monclient: dump: 2024-09-15T01:38:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.134 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.134 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.134 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.134 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.134 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.134 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.134 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.134 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f3b080169f0 con 0x7f3b1002a1d0 2024-09-15T01:38:42.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b067fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:42.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b067fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:42.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b1002a1d0 msgr2=0x7f3b1002a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b067fc640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b1002a1d0 0x7f3b1002a5d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3b08012400 tx=0x7f3b0801f000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b067fc640 1 -- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b1002a1d0 msgr2=0x7f3b1002a5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:42.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.097+0000 7f3b067fc640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b1002a1d0 0x7f3b1002a5d0 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3b08012400 tx=0x7f3b0801f000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.135 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:42.135 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:42.135 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:42.136 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.137 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:38:42.137 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:42.137 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:42.137 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:42.137 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.137 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.137 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.137 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.137 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.137 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.137 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:42.139 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.140 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.143 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:42.143 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:42.143 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.143 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:42.143 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.144 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:42.144 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.144 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.144 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:42.144 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:42.144 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.144 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.144 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.145 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:42.145 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:42.145 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.145 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:42.145 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.146 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.146 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:42.146 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.146 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.146 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.146 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:42.146 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:42.146 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.146 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.146 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:42.147 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.148 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.148 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.148 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:42.148 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:42.148 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.148 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.148 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.148 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:42.148 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:42.149 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.149 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:42.149 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.149 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:42.149 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.149 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.150 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.150 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:42.150 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:42.150 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.150 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.150 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.150 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:42.150 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:42.151 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.151 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:42.151 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.151 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:42.151 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.151 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.155 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.155 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:42.155 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.155 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.155 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.155 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:42.155 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.155 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.155 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.156 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.157 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.158 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.158 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.158 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.158 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:42.158 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:42.158 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.158 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.158 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.158 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:42.158 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.158 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.159 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:42.159 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.159 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.159 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.159 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:42.159 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.159 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.159 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:42.159 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:42.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.098+0000 7f3b14d9d640 1 -- 172.21.15.178:0/3803918262 >> v2:172.21.15.204:6812/28210 conn(0x7f3af00396e0 msgr2=0x7f3af003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.098+0000 7f3b14d9d640 1 --2- 172.21.15.178:0/3803918262 >> v2:172.21.15.204:6812/28210 conn(0x7f3af00396e0 0x7f3af003bb90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3af8000c00 tx=0x7f3af8001670 comp rx=0 tx=0).stop 2024-09-15T01:38:42.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.098+0000 7f3b14d9d640 10 monclient: shutdown 2024-09-15T01:38:42.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.098+0000 7f3b14d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:42.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.098+0000 7f3b14d9d640 1 -- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10131e10 msgr2=0x7f3b1011e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.098+0000 7f3b14d9d640 1 --2- 172.21.15.178:0/3803918262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10131e10 0x7f3b1011e3b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3b00031e30 tx=0x7f3b00035000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.098+0000 7f3b0ed9d640 1 -- 172.21.15.178:0/3803918262 reap_dead start 2024-09-15T01:38:42.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.098+0000 7f3b0ed9d640 1 -- 172.21.15.178:0/3803918262 reap_dead start 2024-09-15T01:38:42.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.098+0000 7f3b14d9d640 1 -- 172.21.15.178:0/3803918262 shutdown_connections 2024-09-15T01:38:42.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.098+0000 7f3b14d9d640 1 -- 172.21.15.178:0/3803918262 >> 172.21.15.178:0/3803918262 conn(0x7f3b100a5ed0 msgr2=0x7f3b100a53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:42.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.098+0000 7f3b14d9d640 1 -- 172.21.15.178:0/3803918262 shutdown_connections 2024-09-15T01:38:42.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.098+0000 7f3b14d9d640 1 -- 172.21.15.178:0/3803918262 wait complete. 2024-09-15T01:38:42.162 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:42.162 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:38:42.162 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:38:42.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.254+0000 7fe72ef1d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:42.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.254+0000 7fe72ef1d640 10 monclient: build_initial_monmap 2024-09-15T01:38:42.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.254+0000 7fe72ef1d640 10 monclient: monmap: 2024-09-15T01:38:42.256 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:42.256 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.256 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:42.254988+0000 2024-09-15T01:38:42.256 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:42.254988+0000 2024-09-15T01:38:42.256 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:42.257 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.257 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:42.257 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:42.257 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:42.257 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:42.257 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:42.257 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:42.257 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:42.257 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:42.257 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:42.258 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:42.258 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:42.258 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.254+0000 7fe72ef1d640 1 Processor -- start 2024-09-15T01:38:42.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.254+0000 7fe72ef1d640 1 -- start start 2024-09-15T01:38:42.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.254+0000 7fe72ef1d640 10 monclient: init 2024-09-15T01:38:42.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 10 monclient: _add_conns ranks=[3,0,6,1,7,10,8,2,9,5,4] 2024-09-15T01:38:42.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe728067130 0x7fe728067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 10 monclient(hunting): picked mon.noname-g con 0x7fe728067130 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:42.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe72809d580 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:42.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe72809f000 0x7fe728067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 10 monclient(hunting): picked mon.noname-d con 0x7fe72809f000 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:42.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72804bf60 con 0x7fe72809d580 2024-09-15T01:38:42.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72809e0c0 con 0x7fe728067130 2024-09-15T01:38:42.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe728067f80 con 0x7fe72809f000 2024-09-15T01:38:42.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ef1d640 10 monclient(hunting): authenticate will time out at 1127.005398s 2024-09-15T01:38:42.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe72809f000 0x7fe728067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe727fff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe72809d580 0x7fe72812d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:50830/0 (socket says 172.21.15.178:50830) 2024-09-15T01:38:42.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe727fff640 1 -- 172.21.15.178:0/1330739208 learned_addr learned my addr 172.21.15.178:0/1330739208 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:38:42.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 10 monclient(hunting): get_auth_request con 0x7fe72809f000 auth_method 0 2024-09-15T01:38:42.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ccb9640 1 --2- 172.21.15.178:0/1330739208 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe728067130 0x7fe728067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ccb9640 10 monclient(hunting): get_auth_request con 0x7fe728067130 auth_method 0 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ccb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ccb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ccb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe7277fe640 1 -- 172.21.15.178:0/1330739208 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606531695 0 0) 0x7fe72804bf60 con 0x7fe72809d580 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe7277fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe7277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe7277fe640 10 monclient(hunting): my global_id is 34317 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe7277fe640 1 -- 172.21.15.178:0/1330739208 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7100034e0 con 0x7fe72809d580 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 10 monclient(hunting): handle_auth_done global_id 24819 payload 274 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 1 -- 172.21.15.178:0/1330739208 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe728067130 msgr2=0x7fe728067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 1 --2- 172.21.15.178:0/1330739208 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe728067130 0x7fe728067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 1 -- 172.21.15.178:0/1330739208 >> v1:172.21.15.178:6789/0 conn(0x7fe72809d580 legacy=0x7fe72812d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 1 monclient: found mon.noname-d 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72d4ba640 1 -- 172.21.15.178:0/1330739208 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72809dde0 con 0x7fe72809f000 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.255+0000 7fe72ccb9640 1 --2- 172.21.15.178:0/1330739208 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe728067130 0x7fe728067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe72d4ba640 1 --2- 172.21.15.178:0/1330739208 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe72809f000 0x7fe728067a40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe720009e90 tx=0x7fe720009780 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7f8a1a7009750117 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe7277fe640 1 -- 172.21.15.178:0/1330739208 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe720031880 con 0x7fe72809f000 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe7277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe7277fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe7277fe640 10 monclient: dump: 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.275 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.276 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe7277fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe7277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe7277fe640 1 -- 172.21.15.178:0/1330739208 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72002d050 con 0x7fe72809f000 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe72ef1d640 5 monclient: authenticate success, global_id 24819 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe72ef1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe7277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe72ef1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe72ef1d640 4 set_mon_vals no callback set 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe7277fe640 1 -- 172.21.15.178:0/1330739208 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe7200303a0 con 0x7fe72809f000 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe7277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe7277fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:42.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.256+0000 7fe7277fe640 10 monclient: dump: 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.278 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.257+0000 7fe72ef1d640 10 monclient: shutdown 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.257+0000 7fe72ef1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.257+0000 7fe72ef1d640 1 -- 172.21.15.178:0/1330739208 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe72809f000 msgr2=0x7fe728067a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.257+0000 7fe72ef1d640 1 --2- 172.21.15.178:0/1330739208 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe72809f000 0x7fe728067a40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe720009e90 tx=0x7fe720009780 comp rx=0 tx=0).stop 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.257+0000 7fe72ef1d640 1 -- 172.21.15.178:0/1330739208 shutdown_connections 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.257+0000 7fe72ef1d640 1 --2- 172.21.15.178:0/1330739208 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe72809f000 0x7fe728067a40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.257+0000 7fe72ef1d640 1 --2- 172.21.15.178:0/1330739208 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe728067130 0x7fe728067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.257+0000 7fe72ef1d640 1 -- 172.21.15.178:0/1330739208 >> 172.21.15.178:0/1330739208 conn(0x7fe728068a60 msgr2=0x7fe7280a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.257+0000 7fe72ef1d640 1 -- 172.21.15.178:0/1330739208 shutdown_connections 2024-09-15T01:38:42.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.257+0000 7fe72ef1d640 1 -- 172.21.15.178:0/1330739208 wait complete. 2024-09-15T01:38:42.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.257+0000 7fe72ef1d640 10 monclient: build_initial_monmap 2024-09-15T01:38:42.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.257+0000 7fe72ef1d640 10 monclient: monmap: 2024-09-15T01:38:42.280 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:42.280 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:42.280 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:42.280 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:42.280 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:42.280 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:42.280 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:42.280 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:42.280 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:42.280 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:42.281 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:42.281 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:42.281 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:42.281 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:42.281 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:42.281 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 1 Processor -- start 2024-09-15T01:38:42.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 1 -- start start 2024-09-15T01:38:42.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 10 monclient: init 2024-09-15T01:38:42.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 10 monclient: _add_conns ranks=[0,6,7,4,5,3,8,2,1] 2024-09-15T01:38:42.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe728067130 0x7fe728141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 10 monclient(hunting): picked mon.noname-a con 0x7fe728067130 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe72809d580 0x7fe728142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 10 monclient(hunting): picked mon.noname-d con 0x7fe72809d580 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe72809f000 0x7fe72806b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 10 monclient(hunting): picked mon.noname-i con 0x7fe72809f000 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72811f830 con 0x7fe728067130 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe72811fe60 con 0x7fe72809d580 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe72811e700 con 0x7fe72809f000 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ef1d640 10 monclient(hunting): authenticate will time out at 1127.008741s 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ccb9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe728067130 0x7fe728141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe727fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe72809d580 0x7fe728142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ccb9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe728067130 0x7fe728141b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49806/0 (socket says 172.21.15.178:49806) 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ccb9640 1 -- 172.21.15.178:0/3761505244 learned_addr learned my addr 172.21.15.178:0/3761505244 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ccb9640 10 monclient(hunting): get_auth_request con 0x7fe728067130 auth_method 0 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ccb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ccb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ccb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe727fff640 10 monclient(hunting): get_auth_request con 0x7fe72809d580 auth_method 0 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe727fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe727fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe727fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ccb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ccb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72ccb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe727fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe727fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe727fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72d4ba640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe72809f000 0x7fe72806b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72d4ba640 10 monclient(hunting): get_auth_request con 0x7fe72809f000 auth_method 0 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72d4ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72d4ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.258+0000 7fe72d4ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe72d4ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe72d4ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe72d4ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe727fff640 10 monclient(hunting): handle_auth_done global_id 24828 payload 824 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe727fff640 1 -- 172.21.15.178:0/3761505244 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe72809f000 msgr2=0x7fe72806b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe727fff640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe72809f000 0x7fe72806b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe727fff640 1 -- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe728067130 msgr2=0x7fe728141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe727fff640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe728067130 0x7fe728141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe727fff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe727fff640 1 monclient: found mon.noname-d 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe727fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe727fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe727fff640 1 -- 172.21.15.178:0/3761505244 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe720009a20 con 0x7fe72809d580 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe72ccb9640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe728067130 0x7fe728141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe727fff640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe72809d580 0x7fe728142060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00c560 tx=0x7fe71c007c80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe71c0070d0 con 0x7fe72809d580 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe724ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe724ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe724ff9640 10 monclient: dump: 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.286 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe724ff9640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe724ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c007270 con 0x7fe72809d580 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe72ef1d640 5 monclient: authenticate success, global_id 24828 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe72ef1d640 10 monclient: _renew_subs 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe72ef1d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe72ef1d640 1 -- 172.21.15.178:0/3761505244 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72809cb50 con 0x7fe72809d580 2024-09-15T01:38:42.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe724ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:42.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe71c011360 con 0x7fe72809d580 2024-09-15T01:38:42.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe724ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe724ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:42.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe724ff9640 10 monclient: dump: 2024-09-15T01:38:42.288 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.288 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.288 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.288 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.288 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.288 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.288 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.289 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.289 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.289 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.289 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.289 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.289 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.289 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.289 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.289 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe72ef1d640 10 monclient: _renew_subs 2024-09-15T01:38:42.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe72ef1d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe72ef1d640 1 -- 172.21.15.178:0/3761505244 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72806cf40 con 0x7fe72809d580 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.259+0000 7fe726ffd640 4 set_mon_vals no callback set 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe71c0073e0 con 0x7fe72809d580 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ef1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ef1d640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe7281425a0 0x7fe72806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ef1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ef1d640 1 -- 172.21.15.178:0/3761505244 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7280641f0 con 0x7fe7281425a0 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ccb9640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe7281425a0 0x7fe72806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe724ff9640 1 --2- 172.21.15.178:0/3761505244 >> v2:172.21.15.204:6812/28210 conn(0x7fe7140397b0 0x7fe71403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ccb9640 10 monclient: get_auth_request con 0x7fe7281425a0 auth_method 0 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ccb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ccb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ccb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fe71c04ac10 con 0x7fe72809d580 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ccb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ccb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ccb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72d4ba640 1 --2- 172.21.15.178:0/3761505244 >> v2:172.21.15.204:6812/28210 conn(0x7fe7140397b0 0x7fe71403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72d4ba640 10 monclient: get_auth_request con 0x7fe7140397b0 auth_method 0 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ccb9640 10 monclient(hunting): handle_auth_done global_id 34249 payload 824 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ccb9640 1 -- 172.21.15.178:0/3761505244 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe72804c4a0 con 0x7fe7281425a0 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.260+0000 7fe72ccb9640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe7281425a0 0x7fe72806b9e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe718007ef0 tx=0x7fe718008040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.261+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe71800ee90 con 0x7fe7281425a0 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.261+0000 7fe724ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.261+0000 7fe724ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.261+0000 7fe724ff9640 10 monclient: dump: 2024-09-15T01:38:42.291 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.292 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.293 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.261+0000 7fe72d4ba640 1 --2- 172.21.15.178:0/3761505244 >> v2:172.21.15.204:6812/28210 conn(0x7fe7140397b0 0x7fe71403bc60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe720034000 tx=0x7fe720037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.261+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fe718014640 con 0x7fe7281425a0 2024-09-15T01:38:42.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.261+0000 7fe724ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:42.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.261+0000 7fe724ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:42.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.261+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe7281425a0 msgr2=0x7fe72806b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.261+0000 7fe724ff9640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe7281425a0 0x7fe72806b9e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe718007ef0 tx=0x7fe718008040 comp rx=0 tx=0).stop 2024-09-15T01:38:42.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.261+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe7281425a0 msgr2=0x7fe72806b9e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:42.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.261+0000 7fe724ff9640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe7281425a0 0x7fe72806b9e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.266+0000 7fe72ef1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:42.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.266+0000 7fe72ef1d640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe72806bf20 0x7fe72806c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.266+0000 7fe72ef1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.266+0000 7fe72ef1d640 1 -- 172.21.15.178:0/3761505244 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe72804bc80 con 0x7fe72806bf20 2024-09-15T01:38:42.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe72ccb9640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe72806bf20 0x7fe72806c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe72ccb9640 10 monclient: get_auth_request con 0x7fe72806bf20 auth_method 0 2024-09-15T01:38:42.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe72ccb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe72ccb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe72ccb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe72ccb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe72ccb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe72ccb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe72ccb9640 10 monclient(hunting): handle_auth_done global_id 34258 payload 824 2024-09-15T01:38:42.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe72ccb9640 1 -- 172.21.15.178:0/3761505244 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe72805ed70 con 0x7fe72806bf20 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe72ccb9640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe72806bf20 0x7fe72806c320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe718011010 tx=0x7fe71801d000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe718023070 con 0x7fe72806bf20 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe724ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe724ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe724ff9640 10 monclient: dump: 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.295 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7fe71801ece0 con 0x7fe72806bf20 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe724ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe724ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe72806bf20 msgr2=0x7fe72806c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe724ff9640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe72806bf20 0x7fe72806c320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe718011010 tx=0x7fe71801d000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe724ff9640 1 -- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe72806bf20 msgr2=0x7fe72806c320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:42.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.267+0000 7fe724ff9640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe72806bf20 0x7fe72806c320 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe718011010 tx=0x7fe71801d000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.297 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:42.297 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:42.297 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:42.297 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:42.297 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:42.297 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:42.297 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:42.298 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:42.299 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.299 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.299 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.299 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.299 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.299 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.299 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.299 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.299 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.299 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.299 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:42.299 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:42.300 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:42.301 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:42.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.304 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.305 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.306 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:42.307 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:42.307 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.307 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.307 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.308 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:42.309 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.311 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:42.312 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.313 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.313 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.313 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:42.313 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:42.313 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.313 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.313 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.313 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.313 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:42.313 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.314 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.315 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:42.316 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.317 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.317 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.317 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:42.317 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:42.317 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.317 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.317 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.317 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.318 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.319 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:42.320 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.320 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.320 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.320 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:42.320 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.320 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.320 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:42.320 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:42.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.268+0000 7fe72ef1d640 1 -- 172.21.15.178:0/3761505244 >> v2:172.21.15.204:6812/28210 conn(0x7fe7140397b0 msgr2=0x7fe71403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.268+0000 7fe72ef1d640 1 --2- 172.21.15.178:0/3761505244 >> v2:172.21.15.204:6812/28210 conn(0x7fe7140397b0 0x7fe71403bc60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe720034000 tx=0x7fe720037000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.268+0000 7fe72ef1d640 10 monclient: shutdown 2024-09-15T01:38:42.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.268+0000 7fe72ef1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:42.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.268+0000 7fe72ef1d640 1 -- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe72809d580 msgr2=0x7fe728142060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.268+0000 7fe72ef1d640 1 --2- 172.21.15.178:0/3761505244 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe72809d580 0x7fe728142060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00c560 tx=0x7fe71c007c80 comp rx=0 tx=0).stop 2024-09-15T01:38:42.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.268+0000 7fe72d4ba640 1 -- 172.21.15.178:0/3761505244 reap_dead start 2024-09-15T01:38:42.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.268+0000 7fe72d4ba640 1 -- 172.21.15.178:0/3761505244 reap_dead start 2024-09-15T01:38:42.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.268+0000 7fe72ef1d640 1 -- 172.21.15.178:0/3761505244 shutdown_connections 2024-09-15T01:38:42.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.268+0000 7fe72ef1d640 1 -- 172.21.15.178:0/3761505244 >> 172.21.15.178:0/3761505244 conn(0x7fe728068a60 msgr2=0x7fe7280a1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:42.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.268+0000 7fe72ef1d640 1 -- 172.21.15.178:0/3761505244 shutdown_connections 2024-09-15T01:38:42.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.268+0000 7fe72ef1d640 1 -- 172.21.15.178:0/3761505244 wait complete. 2024-09-15T01:38:42.323 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:42.323 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:38:42.323 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:38:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.415+0000 7f782ee89640 10 monclient: get_monmap_and_config 2024-09-15T01:38:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.415+0000 7f782ee89640 10 monclient: build_initial_monmap 2024-09-15T01:38:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.415+0000 7f782ee89640 10 monclient: monmap: 2024-09-15T01:38:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:42.415937+0000 2024-09-15T01:38:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:42.415937+0000 2024-09-15T01:38:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:42.418 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:42.418 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:42.418 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:42.418 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:42.418 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:42.418 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:42.418 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:42.418 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:42.418 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:42.418 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:42.418 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.415+0000 7f782ee89640 1 Processor -- start 2024-09-15T01:38:42.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.415+0000 7f782ee89640 1 -- start start 2024-09-15T01:38:42.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.415+0000 7f782ee89640 10 monclient: init 2024-09-15T01:38:42.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 10 monclient: _add_conns ranks=[4,9,1,5,10,0,7,3,6,8,2] 2024-09-15T01:38:42.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78280a2930 0x7f7828132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 10 monclient(hunting): picked mon.noname-c con 0x7f78280a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:42.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f782809faa0 0x7f782809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 10 monclient(hunting): picked mon.noname-a con 0x7f782809faa0 addr v2:172.21.15.178:3300/0 2024-09-15T01:38:42.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f78280a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:42.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782804bf60 con 0x7f78280a08c0 2024-09-15T01:38:42.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f782809e0f0 con 0x7f782809faa0 2024-09-15T01:38:42.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f782809dd80 con 0x7f78280a2930 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782ee89640 10 monclient(hunting): authenticate will time out at 1127.166332s 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78280a2930 0x7f7828132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f7827fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f782809faa0 0x7f782809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78280a2930 0x7f7828132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:43314/0 (socket says 172.21.15.178:43314) 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f7827fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f782809faa0 0x7f782809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49808/0 (socket says 172.21.15.178:49808) 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 1 -- 172.21.15.178:0/1916778752 learned_addr learned my addr 172.21.15.178:0/1916778752 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 10 monclient(hunting): get_auth_request con 0x7f78280a2930 auth_method 0 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f7827fff640 10 monclient(hunting): get_auth_request con 0x7f782809faa0 auth_method 0 2024-09-15T01:38:42.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f7827fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f7827fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f7827fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f7827fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f7827fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f7827fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 10 monclient(hunting): handle_auth_done global_id 34267 payload 274 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 1 -- 172.21.15.178:0/1916778752 >> v2:172.21.15.178:3300/0 conn(0x7f782809faa0 msgr2=0x7f782809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 1 --2- 172.21.15.178:0/1916778752 >> v2:172.21.15.178:3300/0 conn(0x7f782809faa0 0x7f782809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 1 -- 172.21.15.178:0/1916778752 >> v1:172.21.15.204:6789/0 conn(0x7f78280a08c0 legacy=0x7f78280a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 1 monclient: found mon.noname-c 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f782cc25640 1 -- 172.21.15.178:0/1916778752 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78280a2d00 con 0x7f78280a2930 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.416+0000 7f7827fff640 1 --2- 172.21.15.178:0/1916778752 >> v2:172.21.15.178:3300/0 conn(0x7f782809faa0 0x7f782809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f782cc25640 1 --2- 172.21.15.178:0/1916778752 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78280a2930 0x7f7828132de0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f781c009910 tx=0x7f781c0091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8971bd49eebfedeb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f78277fe640 1 -- 172.21.15.178:0/1916778752 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f781c02ad20 con 0x7f78280a2930 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f78277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f78277fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f78277fe640 10 monclient: dump: 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.431 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.432 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.432 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.432 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.432 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.432 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.432 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.432 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.432 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.432 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.432 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f78277fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f78277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f782ee89640 5 monclient: authenticate success, global_id 34267 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f782ee89640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f78277fe640 1 -- 172.21.15.178:0/1916778752 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c031820 con 0x7f78280a2930 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f78277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f782ee89640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f782ee89640 4 set_mon_vals no callback set 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f78277fe640 1 -- 172.21.15.178:0/1916778752 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f781c02aea0 con 0x7f78280a2930 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f78277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f78277fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.417+0000 7f78277fe640 10 monclient: dump: 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.434 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.435 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.435 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.435 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.435 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.418+0000 7f782ee89640 10 monclient: shutdown 2024-09-15T01:38:42.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.418+0000 7f782ee89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:42.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.418+0000 7f782ee89640 1 -- 172.21.15.178:0/1916778752 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78280a2930 msgr2=0x7f7828132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.418+0000 7f782ee89640 1 --2- 172.21.15.178:0/1916778752 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78280a2930 0x7f7828132de0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f781c009910 tx=0x7f781c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.418+0000 7f782ee89640 1 -- 172.21.15.178:0/1916778752 shutdown_connections 2024-09-15T01:38:42.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.418+0000 7f782ee89640 1 --2- 172.21.15.178:0/1916778752 >> v2:172.21.15.178:3300/0 conn(0x7f782809faa0 0x7f782809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.418+0000 7f782ee89640 1 --2- 172.21.15.178:0/1916778752 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78280a2930 0x7f7828132de0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.418+0000 7f782ee89640 1 -- 172.21.15.178:0/1916778752 >> 172.21.15.178:0/1916778752 conn(0x7f78280a5ed0 msgr2=0x7f78280a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.418+0000 7f782ee89640 1 -- 172.21.15.178:0/1916778752 shutdown_connections 2024-09-15T01:38:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.418+0000 7f782ee89640 1 -- 172.21.15.178:0/1916778752 wait complete. 2024-09-15T01:38:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.418+0000 7f782ee89640 10 monclient: build_initial_monmap 2024-09-15T01:38:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.418+0000 7f782ee89640 10 monclient: monmap: 2024-09-15T01:38:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:42.436 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:42.437 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 1 Processor -- start 2024-09-15T01:38:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 1 -- start start 2024-09-15T01:38:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 10 monclient: init 2024-09-15T01:38:42.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 10 monclient: _add_conns ranks=[4,0,2,3,7,8,6,1,5] 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f782809faa0 0x7f782809b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 10 monclient(hunting): picked mon.noname-c con 0x7f782809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78280a08c0 0x7f7828131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 10 monclient(hunting): picked mon.noname-a con 0x7f78280a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f78280a2930 0x7f7828141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 10 monclient(hunting): picked mon.noname-b con 0x7f78280a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782809d6a0 con 0x7f78280a08c0 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782811ea40 con 0x7f78280a2930 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f782811ed20 con 0x7f782809faa0 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782ee89640 10 monclient(hunting): authenticate will time out at 1127.169784s 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782cc25640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f782809faa0 0x7f782809b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f7827fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78280a08c0 0x7f7828131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782cc25640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f782809faa0 0x7f782809b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:43316/0 (socket says 172.21.15.178:43316) 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782cc25640 1 -- 172.21.15.178:0/2608629411 learned_addr learned my addr 172.21.15.178:0/2608629411 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f7827fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78280a08c0 0x7f7828131920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49814/0 (socket says 172.21.15.178:49814) 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782cc25640 10 monclient(hunting): get_auth_request con 0x7f782809faa0 auth_method 0 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782cc25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782cc25640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782cc25640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f7827fff640 10 monclient(hunting): get_auth_request con 0x7f78280a08c0 auth_method 0 2024-09-15T01:38:42.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f7827fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f7827fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f7827fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782d426640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f78280a2930 0x7f7828141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782d426640 10 monclient(hunting): get_auth_request con 0x7f78280a2930 auth_method 0 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782d426640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782d426640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782d426640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782cc25640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782cc25640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f782cc25640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f7827fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f7827fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.419+0000 7f7827fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f782d426640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f782d426640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f782d426640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7827fff640 10 monclient(hunting): handle_auth_done global_id 34344 payload 824 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7827fff640 1 -- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f782809faa0 msgr2=0x7f782809b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7827fff640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f782809faa0 0x7f782809b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7827fff640 1 -- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f78280a2930 msgr2=0x7f7828141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7827fff640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f78280a2930 0x7f7828141c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7827fff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7827fff640 1 monclient: found mon.noname-a 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7827fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7827fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:38:42.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7827fff640 1 -- 172.21.15.178:0/2608629411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7814009390 con 0x7f78280a08c0 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f782cc25640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f782809faa0 0x7f782809b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7827fff640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78280a08c0 0x7f7828131920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78280a6ec0 tx=0x7f78140090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78140095d0 con 0x7f78280a08c0 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7824ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7824ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7824ff9640 10 monclient: dump: 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.442 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7824ff9640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7824ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f782ee89640 5 monclient: authenticate success, global_id 34344 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781400cdc0 con 0x7f78280a08c0 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f782ee89640 10 monclient: _renew_subs 2024-09-15T01:38:42.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f782ee89640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f782ee89640 1 -- 172.21.15.178:0/2608629411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781c009450 con 0x7f78280a08c0 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7824ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78140138f0 con 0x7f78280a08c0 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7824ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7824ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7824ff9640 10 monclient: dump: 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.444 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.445 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.445 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.445 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.445 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.445 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.445 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.445 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.445 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.445 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f7826ffd640 4 set_mon_vals no callback set 2024-09-15T01:38:42.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f782ee89640 10 monclient: _renew_subs 2024-09-15T01:38:42.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f782ee89640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:38:42.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.420+0000 7f782ee89640 1 -- 172.21.15.178:0/2608629411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782811d590 con 0x7f78280a08c0 2024-09-15T01:38:42.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7814013470 con 0x7f78280a08c0 2024-09-15T01:38:42.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782ee89640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:42.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782ee89640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f782811dae0 0x7f7828142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782ee89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782ee89640 1 -- 172.21.15.178:0/2608629411 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f782811f1a0 con 0x7f782811dae0 2024-09-15T01:38:42.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782cc25640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f782811dae0 0x7f7828142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f7824ff9640 1 --2- 172.21.15.178:0/2608629411 >> v2:172.21.15.204:6812/28210 conn(0x7f780c0397b0 0x7f780c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782cc25640 10 monclient: get_auth_request con 0x7f782811dae0 auth_method 0 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782cc25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782cc25640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782cc25640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f781404be20 con 0x7f78280a08c0 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782cc25640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782cc25640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782cc25640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782cc25640 10 monclient(hunting): handle_auth_done global_id 24837 payload 824 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782d426640 1 --2- 172.21.15.178:0/2608629411 >> v2:172.21.15.204:6812/28210 conn(0x7f780c0397b0 0x7f780c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782cc25640 1 -- 172.21.15.178:0/2608629411 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f782804c4a0 con 0x7f782811dae0 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782d426640 10 monclient: get_auth_request con 0x7f780c0397b0 auth_method 0 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.421+0000 7f782cc25640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f782811dae0 0x7f7828142190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f781c035040 tx=0x7f781c032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.422+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f781c03d900 con 0x7f782811dae0 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.422+0000 7f7824ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.422+0000 7f7824ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.422+0000 7f7824ff9640 10 monclient: dump: 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.448 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.449 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.422+0000 7f782d426640 1 --2- 172.21.15.178:0/2608629411 >> v2:172.21.15.204:6812/28210 conn(0x7f780c0397b0 0x7f780c03bc60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7820004290 tx=0x7f7820001610 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.422+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f781c03d480 con 0x7f782811dae0 2024-09-15T01:38:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.422+0000 7f7824ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.422+0000 7f7824ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.422+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f782811dae0 msgr2=0x7f7828142190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.422+0000 7f7824ff9640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f782811dae0 0x7f7828142190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f781c035040 tx=0x7f781c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.422+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f782811dae0 msgr2=0x7f7828142190 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:42.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.422+0000 7f7824ff9640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f782811dae0 0x7f7828142190 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f781c035040 tx=0x7f781c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.427+0000 7f782ee89640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782ee89640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7828131e60 0x7f78281426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782ee89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782ee89640 1 -- 172.21.15.178:0/2608629411 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f782804bc80 con 0x7f7828131e60 2024-09-15T01:38:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782cc25640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7828131e60 0x7f78281426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782cc25640 10 monclient: get_auth_request con 0x7f7828131e60 auth_method 0 2024-09-15T01:38:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782cc25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782cc25640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782cc25640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782cc25640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782cc25640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782cc25640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782cc25640 10 monclient(hunting): handle_auth_done global_id 24846 payload 824 2024-09-15T01:38:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782cc25640 1 -- 172.21.15.178:0/2608629411 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f782805ed70 con 0x7f7828131e60 2024-09-15T01:38:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f782cc25640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7828131e60 0x7f78281426d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f781c033000 tx=0x7f781c03d000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f781c046e00 con 0x7f7828131e60 2024-09-15T01:38:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f7824ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f7824ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f7824ff9640 10 monclient: dump: 2024-09-15T01:38:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.457 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.458 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f781c04d070 con 0x7f7828131e60 2024-09-15T01:38:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f7824ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:42.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f7824ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7828131e60 msgr2=0x7f78281426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f7824ff9640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7828131e60 0x7f78281426d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f781c033000 tx=0x7f781c03d000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f7824ff9640 1 -- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7828131e60 msgr2=0x7f78281426d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:42.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.428+0000 7f7824ff9640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7828131e60 0x7f78281426d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.459 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:42.459 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:42.459 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:42.460 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:42.460 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:42.460 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:42.460 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:42.460 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:42.460 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:42.460 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:42.460 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:42.460 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:42.460 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 7, 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.461 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.462 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.465 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.465 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.465 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.465 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:42.465 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.465 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:42.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.465 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:42.465 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:42.465 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.465 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:42.466 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:42.467 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.468 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.469 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:42.470 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.471 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.472 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:42.473 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.474 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:42.475 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:42.476 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.477 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.478 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:42.480 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:42.481 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.483 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:42.483 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:42.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.429+0000 7f782ee89640 1 -- 172.21.15.178:0/2608629411 >> v2:172.21.15.204:6812/28210 conn(0x7f780c0397b0 msgr2=0x7f780c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.429+0000 7f782ee89640 1 --2- 172.21.15.178:0/2608629411 >> v2:172.21.15.204:6812/28210 conn(0x7f780c0397b0 0x7f780c03bc60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7820004290 tx=0x7f7820001610 comp rx=0 tx=0).stop 2024-09-15T01:38:42.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.429+0000 7f782ee89640 10 monclient: shutdown 2024-09-15T01:38:42.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.429+0000 7f782ee89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:42.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.429+0000 7f782ee89640 1 -- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78280a08c0 msgr2=0x7f7828131920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.429+0000 7f782ee89640 1 --2- 172.21.15.178:0/2608629411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78280a08c0 0x7f7828131920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78280a6ec0 tx=0x7f78140090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.429+0000 7f782d426640 1 -- 172.21.15.178:0/2608629411 reap_dead start 2024-09-15T01:38:42.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.429+0000 7f782d426640 1 -- 172.21.15.178:0/2608629411 reap_dead start 2024-09-15T01:38:42.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.429+0000 7f782ee89640 1 -- 172.21.15.178:0/2608629411 shutdown_connections 2024-09-15T01:38:42.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.429+0000 7f782ee89640 1 -- 172.21.15.178:0/2608629411 >> 172.21.15.178:0/2608629411 conn(0x7f78280a5ed0 msgr2=0x7f782809e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:42.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.429+0000 7f782ee89640 1 -- 172.21.15.178:0/2608629411 shutdown_connections 2024-09-15T01:38:42.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.429+0000 7f782ee89640 1 -- 172.21.15.178:0/2608629411 wait complete. 2024-09-15T01:38:42.485 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:42.485 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:38:42.485 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:38:42.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.578+0000 7f6f8c2c0640 10 monclient: get_monmap_and_config 2024-09-15T01:38:42.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.578+0000 7f6f8c2c0640 10 monclient: build_initial_monmap 2024-09-15T01:38:42.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.578+0000 7f6f8c2c0640 10 monclient: monmap: 2024-09-15T01:38:42.581 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:42.581 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.581 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:42.579466+0000 2024-09-15T01:38:42.581 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:42.579466+0000 2024-09-15T01:38:42.581 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:42.582 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.582 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:42.582 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:42.582 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:42.582 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:42.582 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:42.583 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:42.583 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:42.583 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:42.583 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:42.583 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:42.584 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:42.584 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 1 Processor -- start 2024-09-15T01:38:42.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 1 -- start start 2024-09-15T01:38:42.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 10 monclient: init 2024-09-15T01:38:42.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 10 monclient: _add_conns ranks=[7,0,2,4,1,8,10,6,5,3,9] 2024-09-15T01:38:42.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f840a2930 0x7f6f84132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 10 monclient(hunting): picked mon.noname-i con 0x7f6f840a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:42.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6f8409faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:42.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f8409fe70 0x7f6f840a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 10 monclient(hunting): picked mon.noname-b con 0x7f6f8409fe70 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:42.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8404bf60 con 0x7f6f8409faa0 2024-09-15T01:38:42.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8409dd80 con 0x7f6f8409fe70 2024-09-15T01:38:42.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6f840a2d00 con 0x7f6f840a2930 2024-09-15T01:38:42.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.579+0000 7f6f8c2c0640 10 monclient(hunting): authenticate will time out at 1127.329913s 2024-09-15T01:38:42.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f8409fe70 0x7f6f840a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f8409fe70 0x7f6f840a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42998/0 (socket says 172.21.15.178:42998) 2024-09-15T01:38:42.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8985b640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f6f8409faa0 0x7f6f840a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:50834/0 (socket says 172.21.15.178:50834) 2024-09-15T01:38:42.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 1 -- 172.21.15.178:0/958848531 learned_addr learned my addr 172.21.15.178:0/958848531 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:42.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 10 monclient(hunting): get_auth_request con 0x7f6f8409fe70 auth_method 0 2024-09-15T01:38:42.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a05c640 1 --2- 172.21.15.178:0/958848531 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f840a2930 0x7f6f84132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a05c640 10 monclient(hunting): get_auth_request con 0x7f6f840a2930 auth_method 0 2024-09-15T01:38:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a05c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a05c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a05c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a05c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a05c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a05c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 10 monclient(hunting): handle_auth_done global_id 34229 payload 274 2024-09-15T01:38:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 1 -- 172.21.15.178:0/958848531 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f840a2930 msgr2=0x7f6f84132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8905a640 1 -- 172.21.15.178:0/958848531 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368372175 0 0) 0x7f6f8404bf60 con 0x7f6f8409faa0 2024-09-15T01:38:42.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 1 --2- 172.21.15.178:0/958848531 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f840a2930 0x7f6f84132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 1 -- 172.21.15.178:0/958848531 >> v1:172.21.15.178:6789/0 conn(0x7f6f8409faa0 legacy=0x7f6f840a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 1 monclient: found mon.noname-b 2024-09-15T01:38:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:38:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8a85d640 1 -- 172.21.15.178:0/958848531 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f840a0310 con 0x7f6f8409fe70 2024-09-15T01:38:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.580+0000 7f6f8905a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:38:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8a85d640 1 --2- 172.21.15.178:0/958848531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f8409fe70 0x7f6f840a0fd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009f50 tx=0x7f6f7c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f269d1ca38f50ed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8905a640 1 -- 172.21.15.178:0/958848531 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f7c007880 con 0x7f6f8409fe70 2024-09-15T01:38:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8905a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8905a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:38:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8905a640 10 monclient: dump: 2024-09-15T01:38:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.615 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.616 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.616 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.616 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.617 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.618 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8905a640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8905a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 5 monclient: authenticate success, global_id 34229 2024-09-15T01:38:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8905a640 1 -- 172.21.15.178:0/958848531 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c02d050 con 0x7f6f8409fe70 2024-09-15T01:38:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8905a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:42.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 4 set_mon_vals no callback set 2024-09-15T01:38:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8905a640 1 -- 172.21.15.178:0/958848531 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f7c030360 con 0x7f6f8409fe70 2024-09-15T01:38:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8905a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8905a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8905a640 10 monclient: dump: 2024-09-15T01:38:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.620 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.621 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.622 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.623 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 10 monclient: shutdown 2024-09-15T01:38:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:42.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/958848531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f8409fe70 msgr2=0x7f6f840a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 1 --2- 172.21.15.178:0/958848531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f8409fe70 0x7f6f840a0fd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009f50 tx=0x7f6f7c009840 comp rx=0 tx=0).stop 2024-09-15T01:38:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/958848531 shutdown_connections 2024-09-15T01:38:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 1 --2- 172.21.15.178:0/958848531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f8409fe70 0x7f6f840a0fd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 1 --2- 172.21.15.178:0/958848531 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6f840a2930 0x7f6f84132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/958848531 >> 172.21.15.178:0/958848531 conn(0x7f6f840a5ed0 msgr2=0x7f6f840a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:42.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/958848531 shutdown_connections 2024-09-15T01:38:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.581+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/958848531 wait complete. 2024-09-15T01:38:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.582+0000 7f6f8c2c0640 10 monclient: build_initial_monmap 2024-09-15T01:38:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.582+0000 7f6f8c2c0640 10 monclient: monmap: 2024-09-15T01:38:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:42.625 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:42.626 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:42.627 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:42.627 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:42.627 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:42.627 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:42.627 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.582+0000 7f6f8c2c0640 1 Processor -- start 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.582+0000 7f6f8c2c0640 1 -- start start 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.582+0000 7f6f8c2c0640 10 monclient: init 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient: _add_conns ranks=[5,4,3,7,1,2,8,0,6] 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f8409faa0 0x7f6f8409b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient(hunting): picked mon.noname-h con 0x7f6f8409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f8409fe70 0x7f6f84131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient(hunting): picked mon.noname-c con 0x7f6f8409fe70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840a2930 0x7f6f84141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient(hunting): picked mon.noname-g con 0x7f6f840a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f840a2420 con 0x7f6f840a2930 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8409d630 con 0x7f6f8409fe70 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8411ed20 con 0x7f6f8409faa0 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient(hunting): authenticate will time out at 1127.333137s 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f8409fe70 0x7f6f84131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f8409fe70 0x7f6f84131920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:43318/0 (socket says 172.21.15.178:43318) 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 1 -- 172.21.15.178:0/1864895148 learned_addr learned my addr 172.21.15.178:0/1864895148 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 10 monclient(hunting): get_auth_request con 0x7f6f8409fe70 auth_method 0 2024-09-15T01:38:42.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8a05c640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f8409faa0 0x7f6f8409b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 10 monclient(hunting): handle_auth_done global_id 34285 payload 824 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8a85d640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840a2930 0x7f6f84141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 1 -- 172.21.15.178:0/1864895148 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f8409faa0 msgr2=0x7f6f8409b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f8409faa0 0x7f6f8409b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 1 -- 172.21.15.178:0/1864895148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840a2930 msgr2=0x7f6f84141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840a2930 0x7f6f84141d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 1 monclient: found mon.noname-c 2024-09-15T01:38:42.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:38:42.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 1 -- 172.21.15.178:0/1864895148 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c009ae0 con 0x7f6f8409fe70 2024-09-15T01:38:42.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8a05c640 10 monclient: get_auth_request con 0x7f6f8409faa0 auth_method 0 2024-09-15T01:38:42.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8a05c640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6f8409faa0 0x7f6f8409b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:38:42.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8985b640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f8409fe70 0x7f6f84131920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400c540 tx=0x7f6f74007ea0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f74001040 con 0x7f6f8409fe70 2024-09-15T01:38:42.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f7a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f7a7fc640 10 monclient: dump: 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.632 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f7a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f740047a0 con 0x7f6f8409fe70 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 5 monclient: authenticate success, global_id 34285 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f7a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f74004dd0 con 0x7f6f8409fe70 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient: _renew_subs 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/1864895148 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8409bb00 con 0x7f6f8409fe70 2024-09-15T01:38:42.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f7a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:38:42.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f88859640 4 set_mon_vals no callback set 2024-09-15T01:38:42.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f7a7fc640 10 monclient: dump: 2024-09-15T01:38:42.634 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.634 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.634 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.634 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.634 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.634 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.634 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient: _renew_subs 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.583+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/1864895148 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8411dae0 con 0x7f6f8409fe70 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6f74004950 con 0x7f6f8409fe70 2024-09-15T01:38:42.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8c2c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:42.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8c2c0640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f84131e60 0x7f6f84142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8c2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/1864895148 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8411f0c0 con 0x7f6f84131e60 2024-09-15T01:38:42.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8a05c640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f84131e60 0x7f6f84142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8a05c640 10 monclient: get_auth_request con 0x7f6f84131e60 auth_method 0 2024-09-15T01:38:42.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8a05c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8a05c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8a05c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f7a7fc640 1 --2- 172.21.15.178:0/1864895148 >> v2:172.21.15.204:6812/28210 conn(0x7f6f680396e0 0x7f6f6803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8a05c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8a05c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8a05c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f6f7404b560 con 0x7f6f8409fe70 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.584+0000 7f6f8a85d640 1 --2- 172.21.15.178:0/1864895148 >> v2:172.21.15.204:6812/28210 conn(0x7f6f680396e0 0x7f6f6803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f8a85d640 10 monclient: get_auth_request con 0x7f6f680396e0 auth_method 0 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f8a05c640 10 monclient(hunting): handle_auth_done global_id 34262 payload 824 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f8a05c640 1 -- 172.21.15.178:0/1864895148 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f8404c4a0 con 0x7f6f84131e60 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f8a05c640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f84131e60 0x7f6f84142290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f80010920 tx=0x7f6f80008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f800143f0 con 0x7f6f84131e60 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f7a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f7a7fc640 10 monclient: dump: 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.637 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f8a85d640 1 --2- 172.21.15.178:0/1864895148 >> v2:172.21.15.204:6812/28210 conn(0x7f6f680396e0 0x7f6f6803bb90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c033000 tx=0x7f6f7c036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f6f8000eae0 con 0x7f6f84131e60 2024-09-15T01:38:42.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f7a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:42.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f7a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:42.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f84131e60 msgr2=0x7f6f84142290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f7a7fc640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f84131e60 0x7f6f84142290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f80010920 tx=0x7f6f80008040 comp rx=0 tx=0).stop 2024-09-15T01:38:42.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f84131e60 msgr2=0x7f6f84142290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:42.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.585+0000 7f6f7a7fc640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f84131e60 0x7f6f84142290 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8c2c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:42.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8c2c0640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f841427d0 0x7f6f84096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8c2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/1864895148 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6f84132230 con 0x7f6f841427d0 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8a05c640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f841427d0 0x7f6f84096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8a05c640 10 monclient: get_auth_request con 0x7f6f841427d0 auth_method 0 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8a05c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8a05c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8a05c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8a05c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8a05c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8a05c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8a05c640 10 monclient(hunting): handle_auth_done global_id 34271 payload 824 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8a05c640 1 -- 172.21.15.178:0/1864895148 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6f8404bc80 con 0x7f6f841427d0 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f8a05c640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f841427d0 0x7f6f84096a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f8001a000 tx=0x7f6f8001b000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f80021070 con 0x7f6f841427d0 2024-09-15T01:38:42.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f7a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f7a7fc640 10 monclient: dump: 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.641 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.642 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.642 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.642 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.642 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.642 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f6f80014550 con 0x7f6f841427d0 2024-09-15T01:38:42.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f7a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:42.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f7a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:42.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f841427d0 msgr2=0x7f6f84096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f7a7fc640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f841427d0 0x7f6f84096a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f8001a000 tx=0x7f6f8001b000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f7a7fc640 1 -- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f841427d0 msgr2=0x7f6f84096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:42.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.591+0000 7f6f7a7fc640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6f841427d0 0x7f6f84096a30 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f8001a000 tx=0x7f6f8001b000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.643 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:42.643 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 7, 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.644 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.645 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:42.646 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:42.647 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:42.648 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.649 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:42.651 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.653 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.653 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.653 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:42.653 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.653 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.653 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.653 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.653 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:42.653 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:42.653 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.657 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.657 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.658 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.661 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.662 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.663 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:42.664 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.665 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.666 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.667 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:42.668 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.669 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.669 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.669 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:42.669 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.669 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.669 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:42.669 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:42.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.592+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/1864895148 >> v2:172.21.15.204:6812/28210 conn(0x7f6f680396e0 msgr2=0x7f6f6803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.592+0000 7f6f8c2c0640 1 --2- 172.21.15.178:0/1864895148 >> v2:172.21.15.204:6812/28210 conn(0x7f6f680396e0 0x7f6f6803bb90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c033000 tx=0x7f6f7c036000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.592+0000 7f6f8c2c0640 10 monclient: shutdown 2024-09-15T01:38:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.592+0000 7f6f8c2c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.592+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f8409fe70 msgr2=0x7f6f84131920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.592+0000 7f6f8c2c0640 1 --2- 172.21.15.178:0/1864895148 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6f8409fe70 0x7f6f84131920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400c540 tx=0x7f6f74007ea0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.592+0000 7f6f8a85d640 1 -- 172.21.15.178:0/1864895148 reap_dead start 2024-09-15T01:38:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.592+0000 7f6f8a85d640 1 -- 172.21.15.178:0/1864895148 reap_dead start 2024-09-15T01:38:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.592+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/1864895148 shutdown_connections 2024-09-15T01:38:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.592+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/1864895148 >> 172.21.15.178:0/1864895148 conn(0x7f6f840a5ed0 msgr2=0x7f6f8409e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.592+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/1864895148 shutdown_connections 2024-09-15T01:38:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.592+0000 7f6f8c2c0640 1 -- 172.21.15.178:0/1864895148 wait complete. 2024-09-15T01:38:42.672 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:42.672 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:38:42.672 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:38:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.764+0000 7f703e699640 10 monclient: get_monmap_and_config 2024-09-15T01:38:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.764+0000 7f703e699640 10 monclient: build_initial_monmap 2024-09-15T01:38:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.764+0000 7f703e699640 10 monclient: monmap: 2024-09-15T01:38:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.766 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:42.765354+0000 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:42.765354+0000 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:42.767 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:42.768 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 1 Processor -- start 2024-09-15T01:38:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 1 -- start start 2024-09-15T01:38:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 10 monclient: init 2024-09-15T01:38:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 10 monclient: _add_conns ranks=[6,0,10,8,2,1,3,5,7,4,9] 2024-09-15T01:38:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f70380a2930 0x7f7038132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 10 monclient(hunting): picked mon.noname-d con 0x7f70380a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f703809faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f703809fe70 0x7f70380a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 10 monclient(hunting): picked mon.noname-f con 0x7f703809fe70 addr v2:172.21.15.204:3300/0 2024-09-15T01:38:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703804bf60 con 0x7f703809faa0 2024-09-15T01:38:42.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f703809dd80 con 0x7f703809fe70 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f70380a2d00 con 0x7f70380a2930 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f703e699640 10 monclient(hunting): authenticate will time out at 1127.515737s 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f7037fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f70380a2930 0x7f7038132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f7037fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f70380a2930 0x7f7038132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:34650/0 (socket says 172.21.15.178:34650) 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f70377fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f703809faa0 0x7f70380a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:50840/0 (socket says 172.21.15.178:50840) 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f7037fff640 1 -- 172.21.15.178:0/3500120199 learned_addr learned my addr 172.21.15.178:0/3500120199 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f7037fff640 10 monclient(hunting): get_auth_request con 0x7f70380a2930 auth_method 0 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f7037fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f7037fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f7037fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f7037fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.765+0000 7f7037fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7037fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7037fff640 10 monclient(hunting): handle_auth_done global_id 24855 payload 274 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f703cc36640 1 --2- 172.21.15.178:0/3500120199 >> v2:172.21.15.204:3300/0 conn(0x7f703809fe70 0x7f70380a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7037fff640 1 -- 172.21.15.178:0/3500120199 >> v2:172.21.15.204:3300/0 conn(0x7f703809fe70 msgr2=0x7f70380a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7037fff640 1 --2- 172.21.15.178:0/3500120199 >> v2:172.21.15.204:3300/0 conn(0x7f703809fe70 0x7f70380a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7036ffd640 1 -- 172.21.15.178:0/3500120199 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192948833 0 0) 0x7f703804bf60 con 0x7f703809faa0 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7037fff640 1 -- 172.21.15.178:0/3500120199 >> v1:172.21.15.178:6789/0 conn(0x7f703809faa0 legacy=0x7f70380a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7037fff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7037fff640 1 monclient: found mon.noname-d 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7037fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7037fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7037fff640 1 -- 172.21.15.178:0/3500120199 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70380a0310 con 0x7f70380a2930 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7036ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.766+0000 7f7037fff640 1 --2- 172.21.15.178:0/3500120199 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f70380a2930 0x7f7038132de0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7028009910 tx=0x7f70280091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=36b7120d7f28dbd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f7036ffd640 1 -- 172.21.15.178:0/3500120199 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f702802ad20 con 0x7f70380a2930 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f7036ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f7036ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f7036ffd640 10 monclient: dump: 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f7036ffd640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f7036ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f7036ffd640 1 -- 172.21.15.178:0/3500120199 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7028031820 con 0x7f70380a2930 2024-09-15T01:38:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 5 monclient: authenticate success, global_id 24855 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f7036ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 4 set_mon_vals no callback set 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f7036ffd640 1 -- 172.21.15.178:0/3500120199 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f702802aea0 con 0x7f70380a2930 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f7036ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f7036ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f7036ffd640 10 monclient: dump: 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.782 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 10 monclient: shutdown 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 1 -- 172.21.15.178:0/3500120199 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f70380a2930 msgr2=0x7f7038132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 1 --2- 172.21.15.178:0/3500120199 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f70380a2930 0x7f7038132de0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7028009910 tx=0x7f70280091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 1 -- 172.21.15.178:0/3500120199 shutdown_connections 2024-09-15T01:38:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 1 --2- 172.21.15.178:0/3500120199 >> v2:172.21.15.204:3300/0 conn(0x7f703809fe70 0x7f70380a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 1 --2- 172.21.15.178:0/3500120199 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f70380a2930 0x7f7038132de0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 1 -- 172.21.15.178:0/3500120199 >> 172.21.15.178:0/3500120199 conn(0x7f70380a5ed0 msgr2=0x7f70380a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 1 -- 172.21.15.178:0/3500120199 shutdown_connections 2024-09-15T01:38:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.767+0000 7f703e699640 1 -- 172.21.15.178:0/3500120199 wait complete. 2024-09-15T01:38:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 10 monclient: build_initial_monmap 2024-09-15T01:38:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 10 monclient: monmap: 2024-09-15T01:38:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 1 Processor -- start 2024-09-15T01:38:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 1 -- start start 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 10 monclient: init 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 10 monclient: _add_conns ranks=[4,2,7,8,3,0,6,1,5] 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f703809faa0 0x7f703809b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 10 monclient(hunting): picked mon.noname-c con 0x7f703809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f703809fe70 0x7f7038131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 10 monclient(hunting): picked mon.noname-b con 0x7f703809fe70 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f70380a2930 0x7f703811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.768+0000 7f703e699640 10 monclient(hunting): picked mon.noname-i con 0x7f70380a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7038142b80 con 0x7f703809fe70 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f703809f870 con 0x7f703809faa0 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f703811efb0 con 0x7f70380a2930 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 10 monclient(hunting): authenticate will time out at 1127.519023s 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f7037fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f703809faa0 0x7f703809b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f703809fe70 0x7f7038131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f7037fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f703809faa0 0x7f703809b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:43330/0 (socket says 172.21.15.178:43330) 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f7037fff640 1 -- 172.21.15.178:0/853628106 learned_addr learned my addr 172.21.15.178:0/853628106 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f7037fff640 10 monclient(hunting): get_auth_request con 0x7f703809faa0 auth_method 0 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f7037fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f7037fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f7037fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 10 monclient(hunting): get_auth_request con 0x7f703809fe70 auth_method 0 2024-09-15T01:38:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f7037fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f7037fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703cc36640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f70380a2930 0x7f703811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f7037fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703cc36640 10 monclient(hunting): get_auth_request con 0x7f70380a2930 auth_method 0 2024-09-15T01:38:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703cc36640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703cc36640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703cc36640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703cc36640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703cc36640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703cc36640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 10 monclient(hunting): handle_auth_done global_id 34238 payload 824 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 1 -- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f70380a2930 msgr2=0x7f703811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f70380a2930 0x7f703811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 1 -- 172.21.15.178:0/853628106 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f703809faa0 msgr2=0x7f703809b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f703809faa0 0x7f703809b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 1 monclient: found mon.noname-b 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:38:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 1 -- 172.21.15.178:0/853628106 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7028009450 con 0x7f703809fe70 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f7037fff640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f703809faa0 0x7f703809b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70377fe640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f703809fe70 0x7f7038131820 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f70380a6ec0 tx=0x7f702c007e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f702c0047a0 con 0x7f703809fe70 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f701ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f701ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f701ffff640 10 monclient: dump: 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f701ffff640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f701ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c004940 con 0x7f703809fe70 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 5 monclient: authenticate success, global_id 34238 2024-09-15T01:38:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 10 monclient: _renew_subs 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 1 -- 172.21.15.178:0/853628106 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703809bd90 con 0x7f703809fe70 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f701ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f702c013400 con 0x7f703809fe70 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f701ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f701ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f701ffff640 10 monclient: dump: 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.792 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 10 monclient: _renew_subs 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f70367fc640 4 set_mon_vals no callback set 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.769+0000 7f703e699640 1 -- 172.21.15.178:0/853628106 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703811ec40 con 0x7f703809fe70 2024-09-15T01:38:42.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f703e699640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f702c004b30 con 0x7f703809fe70 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f703e699640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7038131d60 0x7f703811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f703e699640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f703e699640 1 -- 172.21.15.178:0/853628106 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703811eb00 con 0x7f7038131d60 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f7037fff640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7038131d60 0x7f703811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f701ffff640 1 --2- 172.21.15.178:0/853628106 >> v2:172.21.15.204:6812/28210 conn(0x7f70240397b0 0x7f702403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f7037fff640 10 monclient: get_auth_request con 0x7f7038131d60 auth_method 0 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f7037fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f7037fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f7037fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f702c04ac80 con 0x7f703809fe70 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f7037fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f7037fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f7037fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f703cc36640 1 --2- 172.21.15.178:0/853628106 >> v2:172.21.15.204:6812/28210 conn(0x7f70240397b0 0x7f702403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.770+0000 7f703cc36640 10 monclient: get_auth_request con 0x7f70240397b0 auth_method 0 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f7037fff640 10 monclient(hunting): handle_auth_done global_id 7795 payload 824 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f7037fff640 1 -- 172.21.15.178:0/853628106 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f703804c4a0 con 0x7f7038131d60 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f7037fff640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7038131d60 0x7f703811ca70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7028035040 tx=0x7f7028032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f703cc36640 1 --2- 172.21.15.178:0/853628106 >> v2:172.21.15.204:6812/28210 conn(0x7f70240397b0 0x7f702403bc60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f703809d570 tx=0x7f70300077b0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f702803d900 con 0x7f7038131d60 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f701ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f701ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f701ffff640 10 monclient: dump: 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.795 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.796 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f702803d480 con 0x7f7038131d60 2024-09-15T01:38:42.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f701ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:42.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f701ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:42.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7038131d60 msgr2=0x7f703811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f701ffff640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7038131d60 0x7f703811ca70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7028035040 tx=0x7f7028032000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7038131d60 msgr2=0x7f703811ca70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:42.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.771+0000 7f701ffff640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7038131d60 0x7f703811ca70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f703e699640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:42.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f703e699640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f703811cfb0 0x7f7038096b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f703e699640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f703e699640 1 -- 172.21.15.178:0/853628106 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703804bc80 con 0x7f703811cfb0 2024-09-15T01:38:42.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f7037fff640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f703811cfb0 0x7f7038096b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f7037fff640 10 monclient: get_auth_request con 0x7f703811cfb0 auth_method 0 2024-09-15T01:38:42.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f7037fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f7037fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f7037fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f7037fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f7037fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f7037fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f7037fff640 10 monclient(hunting): handle_auth_done global_id 7804 payload 824 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.777+0000 7f7037fff640 1 -- 172.21.15.178:0/853628106 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f703805ed70 con 0x7f703811cfb0 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.778+0000 7f7037fff640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f703811cfb0 0x7f7038096b30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7028033000 tx=0x7f702803d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.778+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7028049890 con 0x7f703811cfb0 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.778+0000 7f701ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.778+0000 7f701ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.778+0000 7f701ffff640 10 monclient: dump: 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.800 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.778+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9311 (secure 0 0 0) 0x7f702804d070 con 0x7f703811cfb0 2024-09-15T01:38:42.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.778+0000 7f701ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:42.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.778+0000 7f701ffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:42.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.778+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f703811cfb0 msgr2=0x7f7038096b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.778+0000 7f701ffff640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f703811cfb0 0x7f7038096b30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7028033000 tx=0x7f702803d000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.778+0000 7f701ffff640 1 -- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f703811cfb0 msgr2=0x7f7038096b30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:42.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.778+0000 7f701ffff640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f703811cfb0 0x7f7038096b30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.802 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:42.802 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 7, 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:42.803 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:42.804 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:42.805 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:42.806 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.807 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:42.808 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:42.809 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.810 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.810 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.810 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.810 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:42.810 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.810 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.810 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.813 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:42.814 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.815 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.816 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.817 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.818 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.819 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.820 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:42.821 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.822 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.823 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.824 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:42.825 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.825 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.825 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:42.825 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:42.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.779+0000 7f703e699640 1 -- 172.21.15.178:0/853628106 >> v2:172.21.15.204:6812/28210 conn(0x7f70240397b0 msgr2=0x7f702403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.779+0000 7f703e699640 1 --2- 172.21.15.178:0/853628106 >> v2:172.21.15.204:6812/28210 conn(0x7f70240397b0 0x7f702403bc60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f703809d570 tx=0x7f70300077b0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.779+0000 7f703e699640 10 monclient: shutdown 2024-09-15T01:38:42.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.779+0000 7f703e699640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:42.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.779+0000 7f703e699640 1 -- 172.21.15.178:0/853628106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f703809fe70 msgr2=0x7f7038131820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.779+0000 7f703e699640 1 --2- 172.21.15.178:0/853628106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f703809fe70 0x7f7038131820 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f70380a6ec0 tx=0x7f702c007e40 comp rx=0 tx=0).stop 2024-09-15T01:38:42.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.779+0000 7f703cc36640 1 -- 172.21.15.178:0/853628106 reap_dead start 2024-09-15T01:38:42.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.779+0000 7f703cc36640 1 -- 172.21.15.178:0/853628106 reap_dead start 2024-09-15T01:38:42.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.779+0000 7f703e699640 1 -- 172.21.15.178:0/853628106 shutdown_connections 2024-09-15T01:38:42.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.779+0000 7f703e699640 1 -- 172.21.15.178:0/853628106 >> 172.21.15.178:0/853628106 conn(0x7f70380a5ed0 msgr2=0x7f70380a55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:42.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.779+0000 7f703e699640 1 -- 172.21.15.178:0/853628106 shutdown_connections 2024-09-15T01:38:42.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.779+0000 7f703e699640 1 -- 172.21.15.178:0/853628106 wait complete. 2024-09-15T01:38:42.827 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:42.827 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:38:42.828 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:38:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.919+0000 7f423309f640 10 monclient: get_monmap_and_config 2024-09-15T01:38:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.919+0000 7f423309f640 10 monclient: build_initial_monmap 2024-09-15T01:38:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.919+0000 7f423309f640 10 monclient: monmap: 2024-09-15T01:38:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:42.920001+0000 2024-09-15T01:38:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:42.920001+0000 2024-09-15T01:38:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.921 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:42.922 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:42.922 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.919+0000 7f423309f640 1 Processor -- start 2024-09-15T01:38:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.919+0000 7f423309f640 1 -- start start 2024-09-15T01:38:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.919+0000 7f423309f640 10 monclient: init 2024-09-15T01:38:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 10 monclient: _add_conns ranks=[4,5,1,10,3,2,8,6,0,9,7] 2024-09-15T01:38:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f422c067130 0x7f422c067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 10 monclient(hunting): picked mon.noname-c con 0x7f422c067130 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f422c09d580 0x7f422c067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 10 monclient(hunting): picked mon.noname-h con 0x7f422c09d580 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f422c09f000 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c04bf60 con 0x7f422c09f000 2024-09-15T01:38:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f422c09e0c0 con 0x7f422c067130 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f422c067f80 con 0x7f422c09d580 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f423309f640 10 monclient(hunting): authenticate will time out at 1127.670455s 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f422c067130 0x7f422c067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f422c067130 0x7f422c067500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:43346/0 (socket says 172.21.15.178:43346) 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 1 -- 172.21.15.178:0/2754161866 learned_addr learned my addr 172.21.15.178:0/2754161866 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 10 monclient(hunting): get_auth_request con 0x7f422c067130 auth_method 0 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f422bfff640 1 --2- 172.21.15.178:0/2754161866 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f422c09d580 0x7f422c067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f422bfff640 10 monclient(hunting): get_auth_request con 0x7f422c09d580 auth_method 0 2024-09-15T01:38:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f422bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f422bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f422bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f422bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f422bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f422bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 10 monclient(hunting): handle_auth_done global_id 34303 payload 274 2024-09-15T01:38:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 1 -- 172.21.15.178:0/2754161866 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f422c09d580 msgr2=0x7f422c067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:42.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 1 --2- 172.21.15.178:0/2754161866 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f422c09d580 0x7f422c067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 1 -- 172.21.15.178:0/2754161866 >> v1:172.21.15.204:6789/0 conn(0x7f422c09f000 legacy=0x7f422c12d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 1 monclient: found mon.noname-c 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.920+0000 7f4230e3b640 1 -- 172.21.15.178:0/2754161866 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c09dde0 con 0x7f422c067130 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f4230e3b640 1 --2- 172.21.15.178:0/2754161866 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f422c067130 0x7f422c067500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f421c009940 tx=0x7f421c0091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c4d2c58b4006958f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f422b7fe640 1 -- 172.21.15.178:0/2754161866 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f421c02ad00 con 0x7f422c067130 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f422b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f422b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f422b7fe640 10 monclient: dump: 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.934 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f422b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f422b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f422b7fe640 1 -- 172.21.15.178:0/2754161866 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c02ae80 con 0x7f422c067130 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f423309f640 5 monclient: authenticate success, global_id 34303 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f423309f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f422b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f423309f640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f422b7fe640 1 -- 172.21.15.178:0/2754161866 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f421c031ca0 con 0x7f422c067130 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f423309f640 4 set_mon_vals no callback set 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f422b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f422b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.921+0000 7f422b7fe640 10 monclient: dump: 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.936 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.922+0000 7f423309f640 10 monclient: shutdown 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.922+0000 7f423309f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.922+0000 7f423309f640 1 -- 172.21.15.178:0/2754161866 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f422c067130 msgr2=0x7f422c067500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.922+0000 7f423309f640 1 --2- 172.21.15.178:0/2754161866 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f422c067130 0x7f422c067500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f421c009940 tx=0x7f421c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.922+0000 7f423309f640 1 -- 172.21.15.178:0/2754161866 shutdown_connections 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.922+0000 7f423309f640 1 --2- 172.21.15.178:0/2754161866 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f422c09d580 0x7f422c067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.922+0000 7f423309f640 1 --2- 172.21.15.178:0/2754161866 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f422c067130 0x7f422c067500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.922+0000 7f423309f640 1 -- 172.21.15.178:0/2754161866 >> 172.21.15.178:0/2754161866 conn(0x7f422c068a60 msgr2=0x7f422c0a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.922+0000 7f423309f640 1 -- 172.21.15.178:0/2754161866 shutdown_connections 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.922+0000 7f423309f640 1 -- 172.21.15.178:0/2754161866 wait complete. 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.922+0000 7f423309f640 10 monclient: build_initial_monmap 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.922+0000 7f423309f640 10 monclient: monmap: 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:42.938 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:42.939 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 1 Processor -- start 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 1 -- start start 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 10 monclient: init 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 10 monclient: _add_conns ranks=[7,2,6,8,4,0,5,3,1] 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f422c067130 0x7f422c133920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 10 monclient(hunting): picked mon.noname-i con 0x7f422c067130 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f422c09d580 0x7f422c141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 10 monclient(hunting): picked mon.noname-b con 0x7f422c09d580 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f422c09f000 0x7f422c1420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 10 monclient(hunting): picked mon.noname-d con 0x7f422c09f000 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c09d9e0 con 0x7f422c09d580 2024-09-15T01:38:42.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f422c122d40 con 0x7f422c09f000 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f422c06cfb0 con 0x7f422c067130 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423309f640 10 monclient(hunting): authenticate will time out at 1127.673767s 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f422bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f422c09d580 0x7f422c141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423163c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f422c09f000 0x7f422c1420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f4230e3b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f422c067130 0x7f422c133920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f422bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f422c09d580 0x7f422c141b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:43018/0 (socket says 172.21.15.178:43018) 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f422bfff640 1 -- 172.21.15.178:0/3441497157 learned_addr learned my addr 172.21.15.178:0/3441497157 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f422bfff640 10 monclient(hunting): get_auth_request con 0x7f422c09d580 auth_method 0 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f422bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f422bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f422bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423163c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f422c09f000 0x7f422c1420a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:34660/0 (socket says 172.21.15.178:34660) 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423163c640 10 monclient(hunting): get_auth_request con 0x7f422c09f000 auth_method 0 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423163c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423163c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f423163c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f4230e3b640 10 monclient(hunting): get_auth_request con 0x7f422c067130 auth_method 0 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f4230e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f4230e3b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f4230e3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f422bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.923+0000 7f422bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f423163c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f423163c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f423163c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4230e3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4230e3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4230e3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422bfff640 10 monclient(hunting): handle_auth_done global_id 34247 payload 824 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422bfff640 1 -- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f422c067130 msgr2=0x7f422c133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422bfff640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f422c067130 0x7f422c133920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422bfff640 1 -- 172.21.15.178:0/3441497157 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f422c09f000 msgr2=0x7f422c1420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422bfff640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f422c09f000 0x7f422c1420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422bfff640 1 monclient: found mon.noname-b 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422bfff640 1 -- 172.21.15.178:0/3441497157 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c009450 con 0x7f422c09d580 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f423163c640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f422c09f000 0x7f422c1420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422bfff640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f422c09d580 0x7f422c141b60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f422000ea20 tx=0x7f42200090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f422000ce90 con 0x7f422c09d580 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4228ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4228ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4228ff9640 10 monclient: dump: 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4228ff9640 10 monclient: _finish_auth 0 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4228ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4220009600 con 0x7f422c09d580 2024-09-15T01:38:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f423309f640 5 monclient: authenticate success, global_id 34247 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f423309f640 10 monclient: _renew_subs 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f423309f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f423309f640 1 -- 172.21.15.178:0/3441497157 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c122a60 con 0x7f422c09d580 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4228ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42200138f0 con 0x7f422c09d580 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4228ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4228ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f4228ff9640 10 monclient: dump: 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.947 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f423309f640 10 monclient: _renew_subs 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f423309f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f423309f640 1 -- 172.21.15.178:0/3441497157 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c133e60 con 0x7f422c09d580 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.924+0000 7f422affd640 4 set_mon_vals no callback set 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4220013470 con 0x7f422c09d580 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f423309f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f423309f640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06b4c0 0x7f422c06b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f423309f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f423309f640 1 -- 172.21.15.178:0/3441497157 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0641f0 con 0x7f422c06b4c0 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4230e3b640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06b4c0 0x7f422c06b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4228ff9640 1 --2- 172.21.15.178:0/3441497157 >> v2:172.21.15.204:6812/28210 conn(0x7f42180397b0 0x7f421803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4230e3b640 10 monclient: get_auth_request con 0x7f422c06b4c0 auth_method 0 2024-09-15T01:38:42.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4230e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4230e3b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4230e3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f422004c260 con 0x7f422c09d580 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4230e3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4230e3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4230e3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f423163c640 1 --2- 172.21.15.178:0/3441497157 >> v2:172.21.15.204:6812/28210 conn(0x7f42180397b0 0x7f421803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f423163c640 10 monclient: get_auth_request con 0x7f42180397b0 auth_method 0 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4230e3b640 10 monclient(hunting): handle_auth_done global_id 15951 payload 824 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.925+0000 7f4230e3b640 1 -- 172.21.15.178:0/3441497157 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f422c05ed70 con 0x7f422c06b4c0 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f4230e3b640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06b4c0 0x7f422c06b890 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f422c0a3520 tx=0x7f421c032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f423163c640 1 --2- 172.21.15.178:0/3441497157 >> v2:172.21.15.204:6812/28210 conn(0x7f42180397b0 0x7f421803bc60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f422c06d3e0 tx=0x7f4224009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f421c032e20 con 0x7f422c06b4c0 2024-09-15T01:38:42.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f4228ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f4228ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f4228ff9640 10 monclient: dump: 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.950 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.951 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.951 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.951 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.951 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f421c031a40 con 0x7f422c06b4c0 2024-09-15T01:38:42.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f4228ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:42.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f4228ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:42.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06b4c0 msgr2=0x7f422c06b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f4228ff9640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06b4c0 0x7f422c06b890 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f422c0a3520 tx=0x7f421c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06b4c0 msgr2=0x7f422c06b890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:42.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.926+0000 7f4228ff9640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06b4c0 0x7f422c06b890 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f422c0a3520 tx=0x7f421c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f423309f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:42.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f423309f640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06bdd0 0x7f422c1191f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:42.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f423309f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:42.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f423309f640 1 -- 172.21.15.178:0/3441497157 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c121680 con 0x7f422c06bdd0 2024-09-15T01:38:42.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f4230e3b640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06bdd0 0x7f422c1191f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:42.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f4230e3b640 10 monclient: get_auth_request con 0x7f422c06bdd0 auth_method 0 2024-09-15T01:38:42.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f4230e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:42.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f4230e3b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:42.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f4230e3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:42.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f4230e3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:42.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f4230e3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:42.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f4230e3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:42.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f4230e3b640 10 monclient(hunting): handle_auth_done global_id 15960 payload 824 2024-09-15T01:38:42.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.932+0000 7f4230e3b640 1 -- 172.21.15.178:0/3441497157 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f422c04bc80 con 0x7f422c06bdd0 2024-09-15T01:38:42.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f4230e3b640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06bdd0 0x7f422c1191f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f421c009210 tx=0x7f421c03d000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:42.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f421c0488c0 con 0x7f422c06bdd0 2024-09-15T01:38:42.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f4228ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f4228ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f4228ff9640 10 monclient: dump: 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:42.955 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:42.956 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:42.956 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:42.956 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:42.956 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:42.956 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:42.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9311 (secure 0 0 0) 0x7f421c048440 con 0x7f422c06bdd0 2024-09-15T01:38:42.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f4228ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:42.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f4228ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:42.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06bdd0 msgr2=0x7f422c1191f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f4228ff9640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06bdd0 0x7f422c1191f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f421c009210 tx=0x7f421c03d000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f4228ff9640 1 -- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06bdd0 msgr2=0x7f422c1191f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:42.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f4228ff9640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f422c06bdd0 0x7f422c1191f0 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f421c009210 tx=0x7f421c03d000 comp rx=0 tx=0).stop 2024-09-15T01:38:42.957 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:42.957 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 7, 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.958 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.959 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.972 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.973 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.973 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:42.973 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.973 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.973 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.973 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:42.973 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.973 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.973 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.974 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.975 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.976 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.976 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.976 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:42.976 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:42.976 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:42.976 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:42.976 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.976 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:42.976 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:42.977 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.977 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.977 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.977 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:42.977 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:42.977 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:42.977 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.977 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.977 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.977 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:42.977 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:42.979 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:42.980 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:42.980 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:42.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:42.980 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:42.980 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:42.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f423309f640 1 -- 172.21.15.178:0/3441497157 >> v2:172.21.15.204:6812/28210 conn(0x7f42180397b0 msgr2=0x7f421803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f423309f640 1 --2- 172.21.15.178:0/3441497157 >> v2:172.21.15.204:6812/28210 conn(0x7f42180397b0 0x7f421803bc60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f422c06d3e0 tx=0x7f4224009040 comp rx=0 tx=0).stop 2024-09-15T01:38:42.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f423309f640 10 monclient: shutdown 2024-09-15T01:38:42.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.933+0000 7f423309f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:42.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.934+0000 7f423309f640 1 -- 172.21.15.178:0/3441497157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f422c09d580 msgr2=0x7f422c141b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:42.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.934+0000 7f423309f640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f422c09d580 0x7f422c141b60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f422000ea20 tx=0x7f42200090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.934+0000 7f423163c640 1 -- 172.21.15.178:0/3441497157 reap_dead start 2024-09-15T01:38:42.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.934+0000 7f423309f640 1 -- 172.21.15.178:0/3441497157 shutdown_connections 2024-09-15T01:38:42.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.934+0000 7f423309f640 1 --2- 172.21.15.178:0/3441497157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f422c09d580 0x7f422c141b60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:42.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.934+0000 7f423309f640 1 -- 172.21.15.178:0/3441497157 >> 172.21.15.178:0/3441497157 conn(0x7f422c068a60 msgr2=0x7f422c0a09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:42.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.934+0000 7f423309f640 1 -- 172.21.15.178:0/3441497157 shutdown_connections 2024-09-15T01:38:42.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:42.934+0000 7f423309f640 1 -- 172.21.15.178:0/3441497157 wait complete. 2024-09-15T01:38:42.983 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:42.983 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:38:42.984 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:38:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.075+0000 7f6cea51d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.075+0000 7f6cea51d640 10 monclient: build_initial_monmap 2024-09-15T01:38:43.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.075+0000 7f6cea51d640 10 monclient: monmap: 2024-09-15T01:38:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:43.076032+0000 2024-09-15T01:38:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:43.076032+0000 2024-09-15T01:38:43.078 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:43.079 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:43.080 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:43.081 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.075+0000 7f6cea51d640 1 Processor -- start 2024-09-15T01:38:43.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 1 -- start start 2024-09-15T01:38:43.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 10 monclient: init 2024-09-15T01:38:43.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:43.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 10 monclient: _add_conns ranks=[8,2,1,6,4,9,0,3,7,5,10] 2024-09-15T01:38:43.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ce40678d0 0x7f6ce409e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 10 monclient(hunting): picked mon.noname-e con 0x7f6ce40678d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ce406ea30 0x7f6ce409ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 10 monclient(hunting): picked mon.noname-b con 0x7f6ce406ea30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6ce409f180 addr v1:172.21.15.204:6789/0 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce404bf60 con 0x7f6ce409f180 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce409f590 con 0x7f6ce406ea30 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce406d5b0 con 0x7f6ce40678d0 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cea51d640 10 monclient(hunting): authenticate will time out at 1127.826528s 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ce40678d0 0x7f6ce409e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cdbfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ce406ea30 0x7f6ce409ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ce40678d0 0x7f6ce409e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43974/0 (socket says 172.21.15.178:43974) 2024-09-15T01:38:43.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 1 -- 172.21.15.178:0/1129781603 learned_addr learned my addr 172.21.15.178:0/1129781603 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:43.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cdbfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ce406ea30 0x7f6ce409ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:43022/0 (socket says 172.21.15.178:43022) 2024-09-15T01:38:43.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 10 monclient(hunting): get_auth_request con 0x7f6ce40678d0 auth_method 0 2024-09-15T01:38:43.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cdbfff640 10 monclient(hunting): get_auth_request con 0x7f6ce406ea30 auth_method 0 2024-09-15T01:38:43.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cdbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 10 monclient(hunting): handle_auth_done global_id 34280 payload 274 2024-09-15T01:38:43.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 1 -- 172.21.15.178:0/1129781603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ce406ea30 msgr2=0x7f6ce409ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:43.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 1 --2- 172.21.15.178:0/1129781603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ce406ea30 0x7f6ce409ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 1 -- 172.21.15.178:0/1129781603 >> v1:172.21.15.204:6789/0 conn(0x7f6ce409f180 legacy=0x7f6ce41306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:43.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 1 monclient: found mon.noname-e 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.076+0000 7f6ce3fff640 1 -- 172.21.15.178:0/1129781603 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4067ca0 con 0x7f6ce40678d0 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce3fff640 1 --2- 172.21.15.178:0/1129781603 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ce40678d0 0x7f6ce409e700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009970 tx=0x7f6cd40091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e8e9664142ce92d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce37fe640 1 -- 172.21.15.178:0/1129781603 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6cd402ad40 con 0x7f6ce40678d0 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce37fe640 10 monclient: dump: 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce37fe640 10 monclient: _finish_auth 0 2024-09-15T01:38:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce37fe640 1 -- 172.21.15.178:0/1129781603 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd4031820 con 0x7f6ce40678d0 2024-09-15T01:38:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6cea51d640 5 monclient: authenticate success, global_id 34280 2024-09-15T01:38:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6cea51d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6cea51d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce37fe640 1 -- 172.21.15.178:0/1129781603 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6cd4031e50 con 0x7f6ce40678d0 2024-09-15T01:38:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6cea51d640 4 set_mon_vals no callback set 2024-09-15T01:38:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:38:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.077+0000 7f6ce37fe640 10 monclient: dump: 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.107 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.107 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.107 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.078+0000 7f6cea51d640 10 monclient: shutdown 2024-09-15T01:38:43.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.078+0000 7f6cea51d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:43.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.078+0000 7f6cea51d640 1 -- 172.21.15.178:0/1129781603 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ce40678d0 msgr2=0x7f6ce409e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.078+0000 7f6cea51d640 1 --2- 172.21.15.178:0/1129781603 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ce40678d0 0x7f6ce409e700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009970 tx=0x7f6cd40091d0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.078+0000 7f6cea51d640 1 -- 172.21.15.178:0/1129781603 shutdown_connections 2024-09-15T01:38:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.078+0000 7f6cea51d640 1 --2- 172.21.15.178:0/1129781603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ce406ea30 0x7f6ce409ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.078+0000 7f6cea51d640 1 --2- 172.21.15.178:0/1129781603 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ce40678d0 0x7f6ce409e700 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.078+0000 7f6cea51d640 1 -- 172.21.15.178:0/1129781603 >> 172.21.15.178:0/1129781603 conn(0x7f6ce40a4390 msgr2=0x7f6ce4067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.078+0000 7f6cea51d640 1 -- 172.21.15.178:0/1129781603 shutdown_connections 2024-09-15T01:38:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.078+0000 7f6cea51d640 1 -- 172.21.15.178:0/1129781603 wait complete. 2024-09-15T01:38:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.078+0000 7f6cea51d640 10 monclient: build_initial_monmap 2024-09-15T01:38:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.078+0000 7f6cea51d640 10 monclient: monmap: 2024-09-15T01:38:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 1 Processor -- start 2024-09-15T01:38:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 1 -- start start 2024-09-15T01:38:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 10 monclient: init 2024-09-15T01:38:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 10 monclient: _add_conns ranks=[1,7,5,2,3,0,6,8,4] 2024-09-15T01:38:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce40678d0 0x7f6ce413f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 10 monclient(hunting): picked mon.noname-f con 0x7f6ce40678d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:38:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ce406ea30 0x7f6ce4133740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 10 monclient(hunting): picked mon.noname-i con 0x7f6ce406ea30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce409f180 0x7f6ce411d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 10 monclient(hunting): picked mon.noname-h con 0x7f6ce409f180 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:38:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce406c900 con 0x7f6ce40678d0 2024-09-15T01:38:43.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce4120bd0 con 0x7f6ce409f180 2024-09-15T01:38:43.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce411eda0 con 0x7f6ce406ea30 2024-09-15T01:38:43.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:43.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.079+0000 7f6cea51d640 10 monclient(hunting): authenticate will time out at 1127.829777s 2024-09-15T01:38:43.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cdbfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ce406ea30 0x7f6ce4133740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce40678d0 0x7f6ce413f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cdbfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ce406ea30 0x7f6ce4133740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:46328/0 (socket says 172.21.15.178:46328) 2024-09-15T01:38:43.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cdbfff640 1 -- 172.21.15.178:0/1368063555 learned_addr learned my addr 172.21.15.178:0/1368063555 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:43.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cdbfff640 10 monclient(hunting): get_auth_request con 0x7f6ce406ea30 auth_method 0 2024-09-15T01:38:43.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cdbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce8aba640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce409f180 0x7f6ce411d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 10 monclient(hunting): get_auth_request con 0x7f6ce40678d0 auth_method 0 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce8aba640 10 monclient(hunting): get_auth_request con 0x7f6ce409f180 auth_method 0 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce8aba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce8aba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce8aba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce8aba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce8aba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce8aba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 10 monclient(hunting): handle_auth_done global_id 7822 payload 824 2024-09-15T01:38:43.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 1 -- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ce406ea30 msgr2=0x7f6ce4133740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ce406ea30 0x7f6ce4133740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 1 -- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce409f180 msgr2=0x7f6ce411d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce409f180 0x7f6ce411d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:38:43.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 1 monclient: found mon.noname-f 2024-09-15T01:38:43.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:43.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:38:43.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 1 -- 172.21.15.178:0/1368063555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd4009450 con 0x7f6ce40678d0 2024-09-15T01:38:43.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cdbfff640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ce406ea30 0x7f6ce4133740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:43.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce3fff640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce40678d0 0x7f6ce413f9c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4031000 tx=0x7f6cd4032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6cd4041070 con 0x7f6ce40678d0 2024-09-15T01:38:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce0ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:38:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce0ff9640 10 monclient: dump: 2024-09-15T01:38:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.117 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce0ff9640 10 monclient: _finish_auth 0 2024-09-15T01:38:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd4032ba0 con 0x7f6ce40678d0 2024-09-15T01:38:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cea51d640 5 monclient: authenticate success, global_id 7822 2024-09-15T01:38:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cea51d640 10 monclient: _renew_subs 2024-09-15T01:38:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cea51d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cea51d640 1 -- 172.21.15.178:0/1368063555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce4120860 con 0x7f6ce40678d0 2024-09-15T01:38:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6cd4032d50 con 0x7f6ce40678d0 2024-09-15T01:38:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce0ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:38:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce0ff9640 10 monclient: dump: 2024-09-15T01:38:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6ce2ffd640 4 set_mon_vals no callback set 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cea51d640 10 monclient: _renew_subs 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cea51d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.080+0000 7f6cea51d640 1 -- 172.21.15.178:0/1368063555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce4140760 con 0x7f6ce40678d0 2024-09-15T01:38:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6cea51d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:43.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6cea51d640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce411f970 0x7f6ce40298c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6cea51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6cea51d640 1 -- 172.21.15.178:0/1368063555 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce41209f0 con 0x7f6ce411f970 2024-09-15T01:38:43.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6cd4039070 con 0x7f6ce40678d0 2024-09-15T01:38:43.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6cdbfff640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce411f970 0x7f6ce40298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6cdbfff640 10 monclient: get_auth_request con 0x7f6ce411f970 auth_method 0 2024-09-15T01:38:43.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6cdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6cdbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6cdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.081+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 1 --2- 172.21.15.178:0/1368063555 >> v2:172.21.15.204:6812/28210 conn(0x7f6cc40397b0 0x7f6cc403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_done global_id 6809 payload 824 2024-09-15T01:38:43.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f6cd406bab0 con 0x7f6ce40678d0 2024-09-15T01:38:43.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6cdbfff640 1 -- 172.21.15.178:0/1368063555 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ce404c4a0 con 0x7f6ce411f970 2024-09-15T01:38:43.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce8aba640 1 --2- 172.21.15.178:0/1368063555 >> v2:172.21.15.204:6812/28210 conn(0x7f6cc40397b0 0x7f6cc403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce8aba640 10 monclient: get_auth_request con 0x7f6cc40397b0 auth_method 0 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6cdbfff640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce411f970 0x7f6ce40298c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009f60 tx=0x7f6ccc0090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ccc014ad0 con 0x7f6ce411f970 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 10 monclient: dump: 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce8aba640 1 --2- 172.21.15.178:0/1368063555 >> v2:172.21.15.204:6812/28210 conn(0x7f6cc40397b0 0x7f6cc403bc60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6ce411e910 tx=0x7f6cdc008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f6ccc014c30 con 0x7f6ce411f970 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce411f970 msgr2=0x7f6ce40298c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce411f970 0x7f6ce40298c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009f60 tx=0x7f6ccc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce411f970 msgr2=0x7f6ce40298c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:43.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.082+0000 7f6ce0ff9640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce411f970 0x7f6ce40298c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cea51d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:43.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cea51d640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce4133c80 0x7f6ce4029e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cea51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cea51d640 1 -- 172.21.15.178:0/1368063555 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce4120d40 con 0x7f6ce4133c80 2024-09-15T01:38:43.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cdbfff640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce4133c80 0x7f6ce4029e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cdbfff640 10 monclient: get_auth_request con 0x7f6ce4133c80 auth_method 0 2024-09-15T01:38:43.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cdbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cdbfff640 10 monclient(hunting): handle_auth_done global_id 6818 payload 824 2024-09-15T01:38:43.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.088+0000 7f6cdbfff640 1 -- 172.21.15.178:0/1368063555 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6ce406d890 con 0x7f6ce4133c80 2024-09-15T01:38:43.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6cdbfff640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce4133c80 0x7f6ce4029e00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc01b000 tx=0x7f6ccc01a000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ccc024040 con 0x7f6ce4133c80 2024-09-15T01:38:43.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce0ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:38:43.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce0ff9640 10 monclient: dump: 2024-09-15T01:38:43.125 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.125 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.126 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.126 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.126 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.126 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.126 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.126 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.126 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.126 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.126 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.126 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.127 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.127 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.127 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.127 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9313 (secure 0 0 0) 0x7f6ccc0133a0 con 0x7f6ce4133c80 2024-09-15T01:38:43.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:43.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:43.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce4133c80 msgr2=0x7f6ce4029e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce0ff9640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce4133c80 0x7f6ce4029e00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc01b000 tx=0x7f6ccc01a000 comp rx=0 tx=0).stop 2024-09-15T01:38:43.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce0ff9640 1 -- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce4133c80 msgr2=0x7f6ce4029e00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:43.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce0ff9640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ce4133c80 0x7f6ce4029e00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.128 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:43.128 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:43.128 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:43.128 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:38:43.128 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:43.128 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:43.129 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:43.129 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:43.129 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:43.129 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:43.129 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:43.129 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:43.129 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:43.129 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:43.129 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:43.129 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:43.129 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 7, 2024-09-15T01:38:43.129 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:43.130 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:43.130 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:43.130 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:43.130 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:43.130 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:43.130 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:43.130 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:43.130 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:43.130 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:43.130 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:43.130 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:43.131 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:43.131 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:43.131 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:43.131 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:43.131 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:43.131 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:43.131 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:43.131 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:43.131 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:43.131 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:43.132 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:43.132 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:43.132 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.132 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:43.132 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:43.132 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:43.132 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:43.132 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:43.132 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:43.132 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:43.132 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:43.133 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:43.134 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:43.134 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:43.134 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:43.134 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:43.134 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:43.134 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:43.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.134 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:43.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.134 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:43.134 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:43.134 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.135 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.135 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.135 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:43.135 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.135 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.135 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:43.135 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.135 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.135 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.136 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:43.136 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:43.136 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.136 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.136 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.136 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:43.136 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.137 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.138 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:43.138 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:43.138 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.138 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.138 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.138 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:43.138 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:43.138 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.139 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.139 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.139 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.139 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:43.139 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.139 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.139 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.139 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:43.139 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.140 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.140 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.140 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:43.140 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:43.140 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.140 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.140 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.140 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:43.141 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:43.141 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.141 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.141 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.141 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:43.141 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.141 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.141 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:43.142 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.143 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:43.144 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:43.144 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.144 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.144 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.144 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:43.144 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:43.145 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.145 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:43.145 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.145 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:43.145 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.145 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.146 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.146 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:43.146 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:43.146 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.146 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.146 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.146 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.146 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:43.146 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:43.146 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.147 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:43.147 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.147 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:43.147 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.148 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.148 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.148 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:43.148 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:43.148 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.148 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.148 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.148 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.149 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.150 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.150 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:43.150 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:43.150 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.150 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.150 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.150 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:43.150 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:43.151 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.151 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:43.151 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.152 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:43.152 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.152 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.152 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:43.152 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:43.152 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.152 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.152 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:38:43.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.154 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:43.154 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:43.154 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:38:43.154 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.154 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:43.154 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:43.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.154 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:43.154 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:43.154 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 4 2024-09-15T01:38:43.154 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.155 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.155 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:43.155 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:43.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6cea51d640 1 -- 172.21.15.178:0/1368063555 >> v2:172.21.15.204:6812/28210 conn(0x7f6cc40397b0 msgr2=0x7f6cc403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6cea51d640 1 --2- 172.21.15.178:0/1368063555 >> v2:172.21.15.204:6812/28210 conn(0x7f6cc40397b0 0x7f6cc403bc60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6ce411e910 tx=0x7f6cdc008040 comp rx=0 tx=0).stop 2024-09-15T01:38:43.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6cea51d640 10 monclient: shutdown 2024-09-15T01:38:43.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6cea51d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:43.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6cea51d640 1 -- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce40678d0 msgr2=0x7f6ce413f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6cea51d640 1 --2- 172.21.15.178:0/1368063555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce40678d0 0x7f6ce413f9c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4031000 tx=0x7f6cd4032000 comp rx=0 tx=0).stop 2024-09-15T01:38:43.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce8aba640 1 -- 172.21.15.178:0/1368063555 reap_dead start 2024-09-15T01:38:43.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6ce8aba640 1 -- 172.21.15.178:0/1368063555 reap_dead start 2024-09-15T01:38:43.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6cea51d640 1 -- 172.21.15.178:0/1368063555 shutdown_connections 2024-09-15T01:38:43.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6cea51d640 1 -- 172.21.15.178:0/1368063555 >> 172.21.15.178:0/1368063555 conn(0x7f6ce40a4390 msgr2=0x7f6ce40a2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:43.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6cea51d640 1 -- 172.21.15.178:0/1368063555 shutdown_connections 2024-09-15T01:38:43.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.089+0000 7f6cea51d640 1 -- 172.21.15.178:0/1368063555 wait complete. 2024-09-15T01:38:43.158 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:43.158 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:38:43.158 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:38:43.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.250+0000 7f791bb5d640 10 monclient: get_monmap_and_config 2024-09-15T01:38:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.250+0000 7f791bb5d640 10 monclient: build_initial_monmap 2024-09-15T01:38:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.251+0000 7f791bb5d640 10 monclient: monmap: 2024-09-15T01:38:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:43.251887+0000 2024-09-15T01:38:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:43.251887+0000 2024-09-15T01:38:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:43.255 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:43.255 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:43.255 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:43.255 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:43.255 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.251+0000 7f791bb5d640 1 Processor -- start 2024-09-15T01:38:43.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.251+0000 7f791bb5d640 1 -- start start 2024-09-15T01:38:43.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.251+0000 7f791bb5d640 10 monclient: init 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 10 monclient: _add_conns ranks=[9,0,2,3,7,4,1,5,6,8,10] 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f79140a2930 0x7f7914132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 10 monclient(hunting): picked mon.noname-a con 0x7f79140a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f791409faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f791409fe70 0x7f79140a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 10 monclient(hunting): picked mon.noname-b con 0x7f791409fe70 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791404bf60 con 0x7f791409faa0 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f791409dd80 con 0x7f79140a2930 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79140a2d00 con 0x7f791409fe70 2024-09-15T01:38:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791bb5d640 10 monclient(hunting): authenticate will time out at 1128.002322s 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f79198f9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f79140a2930 0x7f7914132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f791409fe70 0x7f79140a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f79198f9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f79140a2930 0x7f7914132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:49830/0 (socket says 172.21.15.178:49830) 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f79190f8640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f791409faa0 0x7f79140a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:50844/0 (socket says 172.21.15.178:50844) 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f79198f9640 1 -- 172.21.15.178:0/4023405923 learned_addr learned my addr 172.21.15.178:0/4023405923 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 10 monclient(hunting): get_auth_request con 0x7f791409fe70 auth_method 0 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f79198f9640 10 monclient(hunting): get_auth_request con 0x7f79140a2930 auth_method 0 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f79198f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f79198f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f79198f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f79198f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f79198f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f79198f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 10 monclient(hunting): handle_auth_done global_id 34265 payload 274 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 1 -- 172.21.15.178:0/4023405923 >> v2:172.21.15.178:3300/0 conn(0x7f79140a2930 msgr2=0x7f7914132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 1 --2- 172.21.15.178:0/4023405923 >> v2:172.21.15.178:3300/0 conn(0x7f79140a2930 0x7f7914132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 1 -- 172.21.15.178:0/4023405923 >> v1:172.21.15.178:6789/0 conn(0x7f791409faa0 legacy=0x7f79140a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 10 monclient: _finish_hunting 0 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 1 monclient: found mon.noname-b 2024-09-15T01:38:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f791a0fa640 1 -- 172.21.15.178:0/4023405923 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7914133440 con 0x7f791409fe70 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.252+0000 7f79198f9640 1 --2- 172.21.15.178:0/4023405923 >> v2:172.21.15.178:3300/0 conn(0x7f79140a2930 0x7f7914132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f791a0fa640 1 --2- 172.21.15.178:0/4023405923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f791409fe70 0x7f79140a0fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f790c009850 tx=0x7f790c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2104cbbeffa6477 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f79188f7640 1 -- 172.21.15.178:0/4023405923 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f790c02ac80 con 0x7f791409fe70 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f79188f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f79188f7640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f79188f7640 10 monclient: dump: 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.280 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f79188f7640 10 monclient: _finish_auth 0 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f79188f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:43.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f79188f7640 1 -- 172.21.15.178:0/4023405923 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c02ae00 con 0x7f791409fe70 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f791bb5d640 5 monclient: authenticate success, global_id 34265 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f791bb5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f79188f7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f791bb5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f79188f7640 1 -- 172.21.15.178:0/4023405923 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f790c031ca0 con 0x7f791409fe70 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f79188f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f791bb5d640 4 set_mon_vals no callback set 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f79188f7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.253+0000 7f79188f7640 10 monclient: dump: 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.282 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.254+0000 7f791bb5d640 10 monclient: shutdown 2024-09-15T01:38:43.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.254+0000 7f791bb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.254+0000 7f791bb5d640 1 -- 172.21.15.178:0/4023405923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f791409fe70 msgr2=0x7f79140a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.254+0000 7f791bb5d640 1 --2- 172.21.15.178:0/4023405923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f791409fe70 0x7f79140a0fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f790c009850 tx=0x7f790c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.254+0000 7f791bb5d640 1 -- 172.21.15.178:0/4023405923 shutdown_connections 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.254+0000 7f791bb5d640 1 --2- 172.21.15.178:0/4023405923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f791409fe70 0x7f79140a0fd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.254+0000 7f791bb5d640 1 --2- 172.21.15.178:0/4023405923 >> v2:172.21.15.178:3300/0 conn(0x7f79140a2930 0x7f7914132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.254+0000 7f791bb5d640 1 -- 172.21.15.178:0/4023405923 >> 172.21.15.178:0/4023405923 conn(0x7f79140a5ed0 msgr2=0x7f79140a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.254+0000 7f791bb5d640 1 -- 172.21.15.178:0/4023405923 shutdown_connections 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.254+0000 7f791bb5d640 1 -- 172.21.15.178:0/4023405923 wait complete. 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.254+0000 7f791bb5d640 10 monclient: build_initial_monmap 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.254+0000 7f791bb5d640 10 monclient: monmap: 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:43.284 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 1 Processor -- start 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 1 -- start start 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 10 monclient: init 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 10 monclient: _add_conns ranks=[3,8,6,0,4,7,5,1,2] 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791409faa0 0x7f791409af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 10 monclient(hunting): picked mon.noname-g con 0x7f791409faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f791409fe70 0x7f7914131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 10 monclient(hunting): picked mon.noname-e con 0x7f791409fe70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79140a2930 0x7f791411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 10 monclient(hunting): picked mon.noname-d con 0x7f79140a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791409d630 con 0x7f791409faa0 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f791411e960 con 0x7f79140a2930 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f791411ec40 con 0x7f791409fe70 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791bb5d640 10 monclient(hunting): authenticate will time out at 1128.005598s 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79190f8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f791409fe70 0x7f7914131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791a0fa640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79140a2930 0x7f791411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791a0fa640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79140a2930 0x7f791411c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:34662/0 (socket says 172.21.15.178:34662) 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79190f8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f791409fe70 0x7f7914131820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43982/0 (socket says 172.21.15.178:43982) 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791a0fa640 1 -- 172.21.15.178:0/507210122 learned_addr learned my addr 172.21.15.178:0/507210122 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791a0fa640 10 monclient(hunting): get_auth_request con 0x7f79140a2930 auth_method 0 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791a0fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791a0fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791a0fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79190f8640 10 monclient(hunting): get_auth_request con 0x7f791409fe70 auth_method 0 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79190f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79190f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79190f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791a0fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791a0fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791a0fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79190f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79190f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79190f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79198f9640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791409faa0 0x7f791409af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79198f9640 10 monclient(hunting): get_auth_request con 0x7f791409faa0 auth_method 0 2024-09-15T01:38:43.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79198f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79198f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f79198f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.255+0000 7f791a0fa640 10 monclient(hunting): handle_auth_done global_id 24873 payload 824 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791a0fa640 1 -- 172.21.15.178:0/507210122 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f791409fe70 msgr2=0x7f7914131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791a0fa640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f791409fe70 0x7f7914131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791a0fa640 1 -- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791409faa0 msgr2=0x7f791409af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791a0fa640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791409faa0 0x7f791409af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791a0fa640 10 monclient: _finish_hunting 0 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791a0fa640 1 monclient: found mon.noname-d 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791a0fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791a0fa640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791a0fa640 1 -- 172.21.15.178:0/507210122 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7904009ae0 con 0x7f79140a2930 2024-09-15T01:38:43.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f79198f9640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f791409faa0 0x7f791409af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791a0fa640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79140a2930 0x7f791411c530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f790c009850 tx=0x7f790c032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f790c03b870 con 0x7f79140a2930 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7911ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7911ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7911ffb640 10 monclient: dump: 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.290 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7911ffb640 10 monclient: _finish_auth 0 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7911ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c03b9f0 con 0x7f79140a2930 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791bb5d640 5 monclient: authenticate success, global_id 24873 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7911ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:43.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f790c0313a0 con 0x7f79140a2930 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791bb5d640 10 monclient: _renew_subs 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791bb5d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791bb5d640 1 -- 172.21.15.178:0/507210122 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790c009390 con 0x7f79140a2930 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7911ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7911ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7911ffb640 10 monclient: dump: 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.292 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f7913fff640 4 set_mon_vals no callback set 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791bb5d640 10 monclient: _renew_subs 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791bb5d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:38:43.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.256+0000 7f791bb5d640 1 -- 172.21.15.178:0/507210122 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7914142980 con 0x7f79140a2930 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f790c039040 con 0x7f79140a2930 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f791bb5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f791bb5d640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914136f70 0x7f791411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f791bb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f791bb5d640 1 -- 172.21.15.178:0/507210122 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7914142b10 con 0x7f7914136f70 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f7911ffb640 1 --2- 172.21.15.178:0/507210122 >> v2:172.21.15.204:6812/28210 conn(0x7f79000397b0 0x7f790003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f790c06b440 con 0x7f79140a2930 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79198f9640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914136f70 0x7f791411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79198f9640 10 monclient: get_auth_request con 0x7f7914136f70 auth_method 0 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79198f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79198f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79198f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79198f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79198f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79190f8640 1 --2- 172.21.15.178:0/507210122 >> v2:172.21.15.204:6812/28210 conn(0x7f79000397b0 0x7f790003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79198f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79190f8640 10 monclient: get_auth_request con 0x7f79000397b0 auth_method 0 2024-09-15T01:38:43.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79190f8640 1 -- 172.21.15.178:0/507210122 >> v2:172.21.15.204:6812/28210 conn(0x7f79000397b0 msgr2=0x7f790003bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:38:43.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79190f8640 1 -- 172.21.15.178:0/507210122 >> v2:172.21.15.204:6812/28210 conn(0x7f79000397b0 msgr2=0x7f790003bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:38:43.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79190f8640 1 --2- 172.21.15.178:0/507210122 >> v2:172.21.15.204:6812/28210 conn(0x7f79000397b0 0x7f790003bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:43.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79190f8640 1 --2- 172.21.15.178:0/507210122 >> v2:172.21.15.204:6812/28210 conn(0x7f79000397b0 0x7f790003bc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:43.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79198f9640 10 monclient(hunting): handle_auth_done global_id 25009 payload 824 2024-09-15T01:38:43.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.257+0000 7f79198f9640 1 -- 172.21.15.178:0/507210122 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f791411ead0 con 0x7f7914136f70 2024-09-15T01:38:43.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.258+0000 7f79198f9640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914136f70 0x7f791411ca70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7904009350 tx=0x7f790400b040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.258+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f790400e050 con 0x7f7914136f70 2024-09-15T01:38:43.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.258+0000 7f7911ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.258+0000 7f7911ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.258+0000 7f7911ffb640 10 monclient: dump: 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.296 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.258+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f790400f040 con 0x7f7914136f70 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.258+0000 7f7911ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.258+0000 7f7911ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.258+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914136f70 msgr2=0x7f791411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.258+0000 7f7911ffb640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914136f70 0x7f791411ca70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7904009350 tx=0x7f790400b040 comp rx=0 tx=0).stop 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.258+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914136f70 msgr2=0x7f791411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.258+0000 7f7911ffb640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914136f70 0x7f791411ca70 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7904009350 tx=0x7f790400b040 comp rx=0 tx=0).stop 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f791bb5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:38:43.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f791bb5d640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914131d60 0x7f791411cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f791bb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f791bb5d640 1 -- 172.21.15.178:0/507210122 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f791404bc80 con 0x7f7914131d60 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f79198f9640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914131d60 0x7f791411cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f79198f9640 10 monclient: get_auth_request con 0x7f7914131d60 auth_method 0 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f79198f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f79198f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f79198f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f79198f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f79198f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f79198f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f79198f9640 10 monclient(hunting): handle_auth_done global_id 25018 payload 824 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f79198f9640 1 -- 172.21.15.178:0/507210122 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f791405ed70 con 0x7f7914131d60 2024-09-15T01:38:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.264+0000 7f79198f9640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914131d60 0x7f791411cfb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7904009350 tx=0x7f790401a000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7904022470 con 0x7f7914131d60 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f7911ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f7911ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f7911ffb640 10 monclient: dump: 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.300 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.300 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.300 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.300 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.300 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.300 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f7904016b50 con 0x7f7914131d60 2024-09-15T01:38:43.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f7911ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:38:43.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f7911ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:43.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914131d60 msgr2=0x7f791411cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f7911ffb640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914131d60 0x7f791411cfb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7904009350 tx=0x7f790401a000 comp rx=0 tx=0).stop 2024-09-15T01:38:43.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f7911ffb640 1 -- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914131d60 msgr2=0x7f791411cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:43.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f7911ffb640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7914131d60 0x7f791411cfb0 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7904009350 tx=0x7f790401a000 comp rx=0 tx=0).stop 2024-09-15T01:38:43.301 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:43.301 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:43.301 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:43.301 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:38:43.301 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 76, 2024-09-15T01:38:43.301 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:38:43.301 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 7, 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:38:43.302 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:43.303 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:38:43.304 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:38:43.305 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.306 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.307 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:38:43.309 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.311 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:38:43.312 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:38:43.313 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.316 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.317 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:38:43.318 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:38:43.320 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:38:43.321 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:38:43.322 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:38:43.323 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:38:43.323 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:38:43.323 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:38:43.323 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:38:43.323 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:38:43.323 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:38:43.323 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:38:43.323 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:43.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f791bb5d640 1 -- 172.21.15.178:0/507210122 >> v2:172.21.15.204:6812/28210 conn(0x7f79000397b0 msgr2=0x7f790003bc60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:43.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f791bb5d640 1 --2- 172.21.15.178:0/507210122 >> v2:172.21.15.204:6812/28210 conn(0x7f79000397b0 0x7f790003bc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f791bb5d640 10 monclient: shutdown 2024-09-15T01:38:43.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f791bb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:43.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f791bb5d640 1 -- 172.21.15.178:0/507210122 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79140a2930 msgr2=0x7f791411c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f791bb5d640 1 --2- 172.21.15.178:0/507210122 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f79140a2930 0x7f791411c530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f790c009850 tx=0x7f790c032000 comp rx=0 tx=0).stop 2024-09-15T01:38:43.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f791a0fa640 1 -- 172.21.15.178:0/507210122 reap_dead start 2024-09-15T01:38:43.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f791a0fa640 1 -- 172.21.15.178:0/507210122 reap_dead start 2024-09-15T01:38:43.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f791bb5d640 1 -- 172.21.15.178:0/507210122 shutdown_connections 2024-09-15T01:38:43.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f791bb5d640 1 -- 172.21.15.178:0/507210122 >> 172.21.15.178:0/507210122 conn(0x7f79140a5ed0 msgr2=0x7f791409e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:43.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f791bb5d640 1 -- 172.21.15.178:0/507210122 shutdown_connections 2024-09-15T01:38:43.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.265+0000 7f791bb5d640 1 -- 172.21.15.178:0/507210122 wait complete. 2024-09-15T01:38:43.326 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:38:43.326 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c', 'i'] 2024-09-15T01:38:43.326 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['d'] 2024-09-15T01:38:43.326 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-15T01:38:43.326 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T01:38:43.327 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T01:38:43.327 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:38:43.339 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T01:38:43.339 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-15T01:38:43.339 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i store 2024-09-15T01:38:43.339 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i sync_force --yes-i-really-mean-it 2024-09-15T01:38:43.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.471+0000 7f9bbb909640 10 monclient: get_monmap_and_config 2024-09-15T01:38:43.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.471+0000 7f9bbb909640 10 monclient: build_initial_monmap 2024-09-15T01:38:43.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.471+0000 7f9bbb909640 10 monclient: monmap: 2024-09-15T01:38:43.475 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:43.475 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.475 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:43.472345+0000 2024-09-15T01:38:43.475 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:43.472345+0000 2024-09-15T01:38:43.475 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:43.475 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.476 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:43.476 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:43.476 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:43.476 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:43.476 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:43.477 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:43.477 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:43.477 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:43.477 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:43.477 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:43.477 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:43.478 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 1 Processor -- start 2024-09-15T01:38:43.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 1 -- start start 2024-09-15T01:38:43.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 10 monclient: init 2024-09-15T01:38:43.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:43.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 10 monclient: _add_conns ranks=[3,0,4,7,5,10,2,1,9,8,6] 2024-09-15T01:38:43.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bb40a2930 0x7f9bb4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 10 monclient(hunting): picked mon.noname-g con 0x7f9bb40a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:43.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9bb409faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:43.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9bb409fe70 0x7f9bb40a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 10 monclient(hunting): picked mon.noname-c con 0x7f9bb409fe70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:43.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb404bf60 con 0x7f9bb409faa0 2024-09-15T01:38:43.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb409dd80 con 0x7f9bb40a2930 2024-09-15T01:38:43.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb40a2d00 con 0x7f9bb409fe70 2024-09-15T01:38:43.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:43.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bbb909640 10 monclient(hunting): authenticate will time out at 1128.222752s 2024-09-15T01:38:43.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bb9ea6640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9bb409fe70 msgr2=0x7f9bb40a0fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:43.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bb9ea6640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9bb409fe70 0x7f9bb40a0fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:43.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bb8ea4640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f9bb409faa0 0x7f9bb40a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:50850/0 (socket says 172.21.15.178:50850) 2024-09-15T01:38:43.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bb8ea4640 1 -- 172.21.15.178:0/3593648505 learned_addr learned my addr 172.21.15.178:0/3593648505 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:38:43.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.472+0000 7f9bb96a5640 1 --2- 172.21.15.178:0/3593648505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bb40a2930 0x7f9bb4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 10 monclient(hunting): get_auth_request con 0x7f9bb40a2930 auth_method 0 2024-09-15T01:38:43.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9ba7fff640 1 -- 172.21.15.178:0/3593648505 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471202541 0 0) 0x7f9bb404bf60 con 0x7f9bb409faa0 2024-09-15T01:38:43.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9ba7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9ba7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9ba7fff640 10 monclient(hunting): my global_id is 34389 2024-09-15T01:38:43.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9ba7fff640 1 -- 172.21.15.178:0/3593648505 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ba00034e0 con 0x7f9bb409faa0 2024-09-15T01:38:43.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 10 monclient(hunting): handle_auth_done global_id 15978 payload 274 2024-09-15T01:38:43.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 1 -- 172.21.15.178:0/3593648505 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9bb409fe70 msgr2=0x7f9bb40a0fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:38:43.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 1 --2- 172.21.15.178:0/3593648505 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9bb409fe70 0x7f9bb40a0fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 1 -- 172.21.15.178:0/3593648505 >> v1:172.21.15.178:6789/0 conn(0x7f9bb409faa0 legacy=0x7f9bb40a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 10 monclient: _finish_hunting 0 2024-09-15T01:38:43.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 1 monclient: found mon.noname-g 2024-09-15T01:38:43.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:43.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:38:43.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.473+0000 7f9bb96a5640 1 -- 172.21.15.178:0/3593648505 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb40a0310 con 0x7f9bb40a2930 2024-09-15T01:38:43.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bb96a5640 1 --2- 172.21.15.178:0/3593648505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bb40a2930 0x7f9bb4132de0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009880 tx=0x7f9ba80090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=10a09278f95e94af server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9ba7fff640 1 -- 172.21.15.178:0/3593648505 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ba802ac80 con 0x7f9bb40a2930 2024-09-15T01:38:43.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9ba7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9ba7fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:38:43.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9ba7fff640 10 monclient: dump: 2024-09-15T01:38:43.509 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.509 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.509 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.509 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.509 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.510 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.510 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.510 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.510 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.510 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.510 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.511 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.511 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.511 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.511 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.511 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9ba7fff640 10 monclient: _finish_auth 0 2024-09-15T01:38:43.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9ba7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:43.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 5 monclient: authenticate success, global_id 15978 2024-09-15T01:38:43.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9ba7fff640 1 -- 172.21.15.178:0/3593648505 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba802ae00 con 0x7f9bb40a2930 2024-09-15T01:38:43.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9ba7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:43.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9ba7fff640 1 -- 172.21.15.178:0/3593648505 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ba8031ca0 con 0x7f9bb40a2930 2024-09-15T01:38:43.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 10 monclient: get_monmap_and_config success 2024-09-15T01:38:43.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 4 set_mon_vals no callback set 2024-09-15T01:38:43.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9ba7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9ba7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:38:43.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9ba7fff640 10 monclient: dump: 2024-09-15T01:38:43.513 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.514 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.514 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.514 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.514 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.514 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.514 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.514 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.514 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.514 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.514 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.514 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 10 monclient: shutdown 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 1 -- 172.21.15.178:0/3593648505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bb40a2930 msgr2=0x7f9bb4132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 1 --2- 172.21.15.178:0/3593648505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bb40a2930 0x7f9bb4132de0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009880 tx=0x7f9ba80090f0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 1 -- 172.21.15.178:0/3593648505 shutdown_connections 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 1 --2- 172.21.15.178:0/3593648505 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9bb409fe70 0x7f9bb40a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 1 --2- 172.21.15.178:0/3593648505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bb40a2930 0x7f9bb4132de0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 1 -- 172.21.15.178:0/3593648505 >> 172.21.15.178:0/3593648505 conn(0x7f9bb40a5ed0 msgr2=0x7f9bb40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:43.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 1 -- 172.21.15.178:0/3593648505 shutdown_connections 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.474+0000 7f9bbb909640 1 -- 172.21.15.178:0/3593648505 wait complete. 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.475+0000 7f9bbb909640 10 monclient: build_initial_monmap 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.475+0000 7f9bbb909640 10 monclient: monmap: 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:43.516 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:43.517 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:43.517 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:43.517 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:43.517 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:43.517 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:43.517 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.475+0000 7f9bbb909640 1 Processor -- start 2024-09-15T01:38:43.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.475+0000 7f9bbb909640 1 -- start start 2024-09-15T01:38:43.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.475+0000 7f9bbb909640 10 monclient: init 2024-09-15T01:38:43.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:43.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 10 monclient: _add_conns ranks=[8,7,4,2,3,1,5,6,0] 2024-09-15T01:38:43.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9bb409faa0 0x7f9bb409af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 10 monclient(hunting): picked mon.noname-e con 0x7f9bb409faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:38:43.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb409fe70 0x7f9bb41317a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 10 monclient(hunting): picked mon.noname-i con 0x7f9bb409fe70 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:38:43.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9bb40a2930 0x7f9bb411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 10 monclient(hunting): picked mon.noname-c con 0x7f9bb40a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:38:43.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb40a2390 con 0x7f9bb40a2930 2024-09-15T01:38:43.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb409d630 con 0x7f9bb409fe70 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb409bad0 con 0x7f9bb409faa0 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bbb909640 10 monclient(hunting): authenticate will time out at 1128.226389s 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb9ea6640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9bb40a2930 msgr2=0x7f9bb411c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb9ea6640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9bb40a2930 0x7f9bb411c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9bb409faa0 0x7f9bb409af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9bb409faa0 0x7f9bb409af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43990/0 (socket says 172.21.15.178:43990) 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 1 -- 172.21.15.178:0/878236227 learned_addr learned my addr 172.21.15.178:0/878236227 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 10 monclient(hunting): get_auth_request con 0x7f9bb409faa0 auth_method 0 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb8ea4640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb409fe70 0x7f9bb41317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb8ea4640 10 monclient(hunting): get_auth_request con 0x7f9bb409fe70 auth_method 0 2024-09-15T01:38:43.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb8ea4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb8ea4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb8ea4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb8ea4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb8ea4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb8ea4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 10 monclient(hunting): handle_auth_done global_id 34298 payload 824 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 1 -- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb409fe70 msgr2=0x7f9bb41317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb409fe70 0x7f9bb41317a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 1 -- 172.21.15.178:0/878236227 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9bb40a2930 msgr2=0x7f9bb411c530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9bb40a2930 0x7f9bb411c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 10 monclient: _finish_hunting 0 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 1 monclient: found mon.noname-e 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 1 -- 172.21.15.178:0/878236227 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba8009390 con 0x7f9bb409faa0 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.476+0000 7f9bb96a5640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9bb409faa0 0x7f9bb409af60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8035040 tx=0x7f9ba8032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ba803cba0 con 0x7f9bb409faa0 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:38:43.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 10 monclient: dump: 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.522 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 10 monclient: _finish_auth 0 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba803cd40 con 0x7f9bb409faa0 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9bbb909640 5 monclient: authenticate success, global_id 34298 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9bbb909640 10 monclient: _renew_subs 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9bbb909640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9bbb909640 1 -- 172.21.15.178:0/878236227 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb411e8c0 con 0x7f9bb409faa0 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ba8031820 con 0x7f9bb409faa0 2024-09-15T01:38:43.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:38:43.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 10 monclient: dump: 2024-09-15T01:38:43.524 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.524 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.524 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.524 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.524 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.524 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.524 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.524 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba77fe640 4 set_mon_vals no callback set 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9bbb909640 10 monclient: _renew_subs 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9bbb909640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9bbb909640 1 -- 172.21.15.178:0/878236227 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb411d960 con 0x7f9bb409faa0 2024-09-15T01:38:43.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9ba80313a0 con 0x7f9bb409faa0 2024-09-15T01:38:43.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9bbb909640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:43.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9bbb909640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4137080 0x7f9bb411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9bbb909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.477+0000 7f9bbb909640 1 -- 172.21.15.178:0/878236227 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb411ed40 con 0x7f9bb4137080 2024-09-15T01:38:43.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9ba57fa640 1 --2- 172.21.15.178:0/878236227 >> v2:172.21.15.204:6812/28210 conn(0x7f9b980397b0 0x7f9b9803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f9ba806c160 con 0x7f9bb409faa0 2024-09-15T01:38:43.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb8ea4640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4137080 0x7f9bb411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb8ea4640 10 monclient: get_auth_request con 0x7f9bb4137080 auth_method 0 2024-09-15T01:38:43.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb8ea4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb8ea4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb8ea4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb8ea4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb8ea4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb9ea6640 1 --2- 172.21.15.178:0/878236227 >> v2:172.21.15.204:6812/28210 conn(0x7f9b980397b0 0x7f9b9803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb8ea4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb9ea6640 10 monclient: get_auth_request con 0x7f9b980397b0 auth_method 0 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb8ea4640 10 monclient(hunting): handle_auth_done global_id 25036 payload 824 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb8ea4640 1 -- 172.21.15.178:0/878236227 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9bb404c4a0 con 0x7f9bb4137080 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb8ea4640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4137080 0x7f9bb411ca70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40a6ec0 tx=0x7f9bb0009040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9bb9ea6640 1 --2- 172.21.15.178:0/878236227 >> v2:172.21.15.204:6812/28210 conn(0x7f9b980397b0 0x7f9b9803bc60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9bac003b90 tx=0x7f9bac009290 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9bb00070d0 con 0x7f9bb4137080 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9ba57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9ba57fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.478+0000 7f9ba57fa640 10 monclient: dump: 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.527 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.528 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.479+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f9bb0005a80 con 0x7f9bb4137080 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.479+0000 7f9ba57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.479+0000 7f9ba57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.479+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4137080 msgr2=0x7f9bb411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.479+0000 7f9ba57fa640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4137080 0x7f9bb411ca70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40a6ec0 tx=0x7f9bb0009040 comp rx=0 tx=0).stop 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.479+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4137080 msgr2=0x7f9bb411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.479+0000 7f9ba57fa640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4137080 0x7f9bb411ca70 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40a6ec0 tx=0x7f9bb0009040 comp rx=0 tx=0).stop 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.485+0000 7f9bbb909640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.485+0000 7f9bbb909640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4131ce0 0x7f9bb411cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.485+0000 7f9bbb909640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.485+0000 7f9bbb909640 1 -- 172.21.15.178:0/878236227 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb409f8a0 con 0x7f9bb4131ce0 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.485+0000 7f9bb8ea4640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4131ce0 0x7f9bb411cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:43.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.485+0000 7f9bb8ea4640 10 monclient: get_auth_request con 0x7f9bb4131ce0 auth_method 0 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.485+0000 7f9bb8ea4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.485+0000 7f9bb8ea4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.485+0000 7f9bb8ea4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.485+0000 7f9bb8ea4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.485+0000 7f9bb8ea4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.486+0000 7f9bb8ea4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.486+0000 7f9bb8ea4640 10 monclient(hunting): handle_auth_done global_id 25045 payload 824 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.486+0000 7f9bb8ea4640 1 -- 172.21.15.178:0/878236227 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7f9bb404bc80 con 0x7f9bb4131ce0 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.486+0000 7f9bb8ea4640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4131ce0 0x7f9bb411cfb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40a6ec0 tx=0x7f9bb0013000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.486+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9bb000dd50 con 0x7f9bb4131ce0 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.486+0000 7f9ba57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.486+0000 7f9ba57fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.486+0000 7f9ba57fa640 10 monclient: dump: 2024-09-15T01:38:43.530 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:38:43.531 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:38:43.532 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7f9bb000deb0 con 0x7f9bb4131ce0 2024-09-15T01:38:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9ba57fa640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:38:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9ba57fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:38:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4131ce0 msgr2=0x7f9bb411cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9ba57fa640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4131ce0 0x7f9bb411cfb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40a6ec0 tx=0x7f9bb0013000 comp rx=0 tx=0).stop 2024-09-15T01:38:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9ba57fa640 1 -- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4131ce0 msgr2=0x7f9bb411cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:38:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9ba57fa640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9bb4131ce0 0x7f9bb411cfb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:38:43.532 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:38:43.533 INFO:teuthology.orchestra.run.smithi178.stdout: "ret": 0, 2024-09-15T01:38:43.533 INFO:teuthology.orchestra.run.smithi178.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-15T01:38:43.533 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:38:43.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9bbb909640 1 -- 172.21.15.178:0/878236227 >> v2:172.21.15.204:6812/28210 conn(0x7f9b980397b0 msgr2=0x7f9b9803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9bbb909640 1 --2- 172.21.15.178:0/878236227 >> v2:172.21.15.204:6812/28210 conn(0x7f9b980397b0 0x7f9b9803bc60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9bac003b90 tx=0x7f9bac009290 comp rx=0 tx=0).stop 2024-09-15T01:38:43.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9bbb909640 10 monclient: shutdown 2024-09-15T01:38:43.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9bbb909640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:38:43.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9bbb909640 1 -- 172.21.15.178:0/878236227 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9bb409faa0 msgr2=0x7f9bb409af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:38:43.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9bbb909640 1 --2- 172.21.15.178:0/878236227 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9bb409faa0 0x7f9bb409af60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8035040 tx=0x7f9ba8032000 comp rx=0 tx=0).stop 2024-09-15T01:38:43.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9bb9ea6640 1 -- 172.21.15.178:0/878236227 reap_dead start 2024-09-15T01:38:43.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9bb9ea6640 1 -- 172.21.15.178:0/878236227 reap_dead start 2024-09-15T01:38:43.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9bbb909640 1 -- 172.21.15.178:0/878236227 shutdown_connections 2024-09-15T01:38:43.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9bbb909640 1 -- 172.21.15.178:0/878236227 >> 172.21.15.178:0/878236227 conn(0x7f9bb40a5ed0 msgr2=0x7f9bb409e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:38:43.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9bbb909640 1 -- 172.21.15.178:0/878236227 shutdown_connections 2024-09-15T01:38:43.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:43.487+0000 7f9bbb909640 1 -- 172.21.15.178:0/878236227 wait complete. 2024-09-15T01:38:43.535 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-15T01:38:43.535 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-15T01:38:43.536 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:38:43.550 INFO:tasks.ceph.mon.i:Stopped 2024-09-15T01:38:43.551 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-15T01:38:43.551 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-15T01:38:43.551 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T01:38:58.552 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-15T01:38:58.553 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-15T01:38:58.553 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-15T01:38:58.553 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:38:58.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.652+0000 7f9d71034640 10 monclient: get_monmap_and_config 2024-09-15T01:38:58.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.652+0000 7f9d71034640 10 monclient: build_initial_monmap 2024-09-15T01:38:58.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.652+0000 7f9d71034640 10 monclient: monmap: 2024-09-15T01:38:58.656 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:38:58.656 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:38:58.656 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:38:58.654361+0000 2024-09-15T01:38:58.656 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:38:58.654361+0000 2024-09-15T01:38:58.656 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:38:58.656 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:38:58.656 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:38:58.656 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:38:58.656 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:38:58.656 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:38:58.656 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:38:58.657 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:38:58.657 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:38:58.657 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:38:58.657 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:38:58.657 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:38:58.657 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:38:58.657 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:38:58.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 1 Processor -- start 2024-09-15T01:38:58.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 1 -- start start 2024-09-15T01:38:58.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 10 monclient: init 2024-09-15T01:38:58.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 10 monclient: _reopen_session rank -1 2024-09-15T01:38:58.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 10 monclient: _add_conns ranks=[0,3,9,7,5,1,8,10,4,6,2] 2024-09-15T01:38:58.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9d6c0a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:38:58.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:58.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 10 monclient(hunting): picked mon.noname-g con 0x7f9d6c132de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:38:58.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:38:58.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 10 monclient(hunting): picked mon.noname-a con 0x7f9d6c1380e0 addr v2:172.21.15.178:3300/0 2024-09-15T01:38:58.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6c04bf60 con 0x7f9d6c0a2930 2024-09-15T01:38:58.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:58.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6c09dd80 con 0x7f9d6c1380e0 2024-09-15T01:38:58.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:38:58.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0a2d00 con 0x7f9d6c132de0 2024-09-15T01:38:58.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 10 monclient(hunting): _renew_subs 2024-09-15T01:38:58.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d71034640 10 monclient(hunting): authenticate will time out at 1143.404611s 2024-09-15T01:38:58.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6b59e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:58.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6b59e640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40398/0 (socket says 172.21.15.178:40398) 2024-09-15T01:38:58.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6ad9d640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f9d6c0a2930 0x7f9d6c09faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:58354/0 (socket says 172.21.15.178:58354) 2024-09-15T01:38:58.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 learned_addr learned my addr 172.21.15.178:0/914717980 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:38:58.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d6c1380e0 auth_method 0 2024-09-15T01:38:58.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:58.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 msgr2=0x7f9d6c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 msgr2=0x7f9d6c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request con 0x7f9d6c132de0 auth_method 0 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.653+0000 7f9d6a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.654+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 msgr2=0x7f9d6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.654+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 msgr2=0x7f9d6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:58.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.654+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:58.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.654+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:38:58.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.854+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:58.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.854+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d6c1380e0 auth_method 0 2024-09-15T01:38:58.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.854+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:58.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.854+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:58.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.854+0000 7f9d6b59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:58.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.854+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 msgr2=0x7f9d6c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:58.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.854+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 msgr2=0x7f9d6c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:58.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.854+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:58.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.854+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:58.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.855+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:58.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.855+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request con 0x7f9d6c132de0 auth_method 0 2024-09-15T01:38:58.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.855+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:58.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.855+0000 7f9d6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:58.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.855+0000 7f9d6a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:58.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.855+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 msgr2=0x7f9d6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:58.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.855+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 msgr2=0x7f9d6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:58.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.855+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:58.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:58.855+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:38:59.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:59.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d6c1380e0 auth_method 0 2024-09-15T01:38:59.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:59.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:59.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6b59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:59.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 msgr2=0x7f9d6c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:59.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 msgr2=0x7f9d6c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:59.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:59.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:38:59.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:38:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request con 0x7f9d6c132de0 auth_method 0 2024-09-15T01:38:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:38:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:38:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:38:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 msgr2=0x7f9d6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:38:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 msgr2=0x7f9d6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:38:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:38:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:38:59.256+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:39:00.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d6c1380e0 auth_method 0 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6b59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 msgr2=0x7f9d6c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 msgr2=0x7f9d6c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request con 0x7f9d6c132de0 auth_method 0 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:00.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:00.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 msgr2=0x7f9d6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:39:00.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 msgr2=0x7f9d6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:00.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:00.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:00.057+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:39:01.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 10 monclient(hunting): tick 2024-09-15T01:39:01.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 1 monclient(hunting): continuing hunt 2024-09-15T01:39:01.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:39:01.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 msgr2=0x7f9d6c137ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:01.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d6c132de0 0x7f9d6c137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:01.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 msgr2=0x7f9d6c13a4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:01.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.178:3300/0 conn(0x7f9d6c1380e0 0x7f9d6c13a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:01.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 >> v1:172.21.15.178:6789/0 conn(0x7f9d6c0a2930 legacy=0x7f9d6c09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 10 monclient: _add_conns ranks=[5,4,10,7,6,1,2,8] 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 10 monclient(hunting): picked mon.noname-h con 0x7f9d580037b0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d58006270 0x7f9d580086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 10 monclient(hunting): picked mon.noname-c con 0x7f9d58006270 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 10 monclient(hunting): picked mon.noname-f con 0x7f9d58008da0 addr v2:172.21.15.204:3300/0 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6c09dd80 con 0x7f9d58008da0 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0a2d00 con 0x7f9d58006270 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:01.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5800b9a0 con 0x7f9d580037b0 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d68d99640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d58006270 msgr2=0x7f9d580086d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.654+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d58006270 0x7f9d580086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d58008da0 auth_method 0 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request con 0x7f9d580037b0 auth_method 0 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:01.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:01.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:39:01.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:01.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:01.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 msgr2=0x7f9d58005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:39:01.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 msgr2=0x7f9d58005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:01.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:01.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:01.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.655+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:01.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.855+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d58006270 msgr2=0x7f9d580086d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:39:01.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.855+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d58006270 0x7f9d580086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:39:01.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:01.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d58008da0 auth_method 0 2024-09-15T01:39:01.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:01.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:01.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6b59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:01.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:01.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request con 0x7f9d580037b0 auth_method 0 2024-09-15T01:39:01.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:01.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:01.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:01.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:39:01.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:01.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:01.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:39:01.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 msgr2=0x7f9d58005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:39:01.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 msgr2=0x7f9d58005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:01.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:01.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:01.856+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:39:02.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.256+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d58006270 msgr2=0x7f9d580086d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:39:02.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.256+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d58006270 0x7f9d580086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:39:02.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:02.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:02.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d58008da0 auth_method 0 2024-09-15T01:39:02.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:02.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:02.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6b59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:02.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request con 0x7f9d580037b0 auth_method 0 2024-09-15T01:39:02.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:02.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:02.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:02.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:39:02.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:02.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:02.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 msgr2=0x7f9d58005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:39:02.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 msgr2=0x7f9d58005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:02.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:02.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:39:02.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:02.257+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:39:03.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.057+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d58006270 msgr2=0x7f9d580086d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:39:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.057+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d58006270 0x7f9d580086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:39:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request con 0x7f9d580037b0 auth_method 0 2024-09-15T01:39:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d58008da0 auth_method 0 2024-09-15T01:39:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:03.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6b59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:03.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 msgr2=0x7f9d58005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:39:03.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 msgr2=0x7f9d58005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:03.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:39:03.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:03.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:03.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:03.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:39:03.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:03.058+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:39:04.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 10 monclient(hunting): tick 2024-09-15T01:39:04.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 monclient(hunting): continuing hunt 2024-09-15T01:39:04.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:39:04.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 msgr2=0x7f9d58005bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:04.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9d580037b0 0x7f9d58005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:04.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d58006270 msgr2=0x7f9d580086d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:04.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d58006270 0x7f9d580086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:04.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:04.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> v2:172.21.15.204:3300/0 conn(0x7f9d58008da0 0x7f9d5800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:04.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 10 monclient: _add_conns ranks=[6,7,2,1,8] 2024-09-15T01:39:04.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 reap_dead start 2024-09-15T01:39:04.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:04.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 10 monclient(hunting): picked mon.noname-d con 0x7f9d5800bf50 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:39:04.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 reap_dead start 2024-09-15T01:39:04.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d58003be0 0x7f9d58006660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:04.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 10 monclient(hunting): picked mon.noname-i con 0x7f9d58003be0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:39:04.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:04.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 10 monclient(hunting): picked mon.noname-b con 0x7f9d58006ba0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c09dd80 con 0x7f9d58006ba0 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0a2d00 con 0x7f9d5800bf50 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5800b9a0 con 0x7f9d58003be0 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d68d99640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request con 0x7f9d5800bf50 auth_method 0 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:04.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d58003be0 msgr2=0x7f9d58006660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d58003be0 0x7f9d58006660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d58006ba0 auth_method 0 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 msgr2=0x7f9d5800e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 msgr2=0x7f9d5800e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:04.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:04.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.655+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d58003be0 msgr2=0x7f9d58006660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:39:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request con 0x7f9d5800bf50 auth_method 0 2024-09-15T01:39:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d58003be0 0x7f9d58006660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:39:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d58006ba0 auth_method 0 2024-09-15T01:39:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6b59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 msgr2=0x7f9d5800e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:39:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 msgr2=0x7f9d5800e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:39:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:39:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:04.856+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:39:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d58003be0 msgr2=0x7f9d58006660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:39:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d58003be0 0x7f9d58006660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:39:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request con 0x7f9d5800bf50 auth_method 0 2024-09-15T01:39:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d58006ba0 auth_method 0 2024-09-15T01:39:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6b59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 msgr2=0x7f9d5800e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:39:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:39:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 msgr2=0x7f9d5800e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:39:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:05.257+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:39:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d58003be0 msgr2=0x7f9d58006660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:39:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d58003be0 0x7f9d58006660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:39:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request con 0x7f9d5800bf50 auth_method 0 2024-09-15T01:39:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d58006ba0 auth_method 0 2024-09-15T01:39:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6b59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 msgr2=0x7f9d5800e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:39:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6ad9d640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 msgr2=0x7f9d5800e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:39:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:39:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:06.058+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:39:07.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 10 monclient(hunting): tick 2024-09-15T01:39:07.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 1 monclient(hunting): continuing hunt 2024-09-15T01:39:07.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:39:07.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d58003be0 msgr2=0x7f9d58006660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:07.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d58003be0 0x7f9d58006660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:07.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 msgr2=0x7f9d5800e330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:07.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d5800bf50 0x7f9d5800e330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:07.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 msgr2=0x7f9d5800b250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:07.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d58006ba0 0x7f9d5800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:07.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 10 monclient: _add_conns ranks=[1,8] 2024-09-15T01:39:07.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9d58007420 addr v1:172.21.15.204:6789/0 2024-09-15T01:39:07.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 0x7f9d580083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:07.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 10 monclient(hunting): picked mon.noname-e con 0x7f9d58007fd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:39:07.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6c1331b0 con 0x7f9d58007420 2024-09-15T01:39:07.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:07.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 1 -- 172.21.15.178:0/914717980 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c09dd80 con 0x7f9d58007fd0 2024-09-15T01:39:07.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d68d99640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:07.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 0x7f9d580083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:07.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request con 0x7f9d58007fd0 auth_method 0 2024-09-15T01:39:07.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:07.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:07.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.655+0000 7f9d6a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:07.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.656+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 msgr2=0x7f9d580083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:39:07.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.656+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 msgr2=0x7f9d580083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:07.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.656+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 0x7f9d580083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:07.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.656+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 0x7f9d580083d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.857+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 0x7f9d580083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.857+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request con 0x7f9d58007fd0 auth_method 0 2024-09-15T01:39:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.857+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.857+0000 7f9d6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.857+0000 7f9d6a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.857+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 msgr2=0x7f9d580083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:39:07.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.857+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 msgr2=0x7f9d580083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:07.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.857+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 0x7f9d580083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:07.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:07.857+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 0x7f9d580083d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:39:08.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.258+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 0x7f9d580083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.258+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request con 0x7f9d58007fd0 auth_method 0 2024-09-15T01:39:08.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.258+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:08.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.258+0000 7f9d6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:08.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.258+0000 7f9d6a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:08.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.258+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 msgr2=0x7f9d580083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:39:08.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.258+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 msgr2=0x7f9d580083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:39:08.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.258+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 0x7f9d580083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:39:08.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.258+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 0x7f9d580083d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:39:08.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.565+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993802522 0 0) 0x7f9d6c1331b0 con 0x7f9d58007420 2024-09-15T01:39:08.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.565+0000 7f9d69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:08.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.565+0000 7f9d69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:08.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.565+0000 7f9d69d9b640 10 monclient(hunting): my global_id is 7831 2024-09-15T01:39:08.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.565+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d540034e0 con 0x7f9d58007420 2024-09-15T01:39:08.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 386425686 0 0) 0x7f9d540034e0 con 0x7f9d58007420 2024-09-15T01:39:08.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 msgr2=0x7f9d580083d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:08.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d58007fd0 0x7f9d580083d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:39:08.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 monclient: found mon.noname-f-legacy 2024-09-15T01:39:08.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:08.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.204:6789/0 2024-09-15T01:39:08.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d58014950 con 0x7f9d58007420 2024-09-15T01:39:08.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient: _finish_auth 0 2024-09-15T01:39:08.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:08.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f9d5c014900 con 0x7f9d58007420 2024-09-15T01:39:08.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d71034640 5 monclient: authenticate success, global_id 7831 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d71034640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient: dump: 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:08.571 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:08.572 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:08.572 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:08.572 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:08.572 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:08.572 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:08.572 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:08.572 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] but i'm connected to v1:172.21.15.204:6789/0, reconnecting 2024-09-15T01:39:08.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:08.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 >> v1:172.21.15.204:6789/0 conn(0x7f9d58007420 legacy=0x7f9d58007820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:08.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 reap_dead start 2024-09-15T01:39:08.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient: _add_conns ranks=[6,1,7,3,5,8,2,0,4] 2024-09-15T01:39:08.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d54004220 0x7f9d54004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient(hunting): picked mon.d con 0x7f9d54004220 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:39:08.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d54004de0 0x7f9d54005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient(hunting): picked mon.f con 0x7f9d54004de0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:39:08.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d54009cb0 0x7f9d5400c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient(hunting): picked mon.i con 0x7f9d54009cb0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c09dd80 con 0x7f9d54004de0 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5400ca10 con 0x7f9d54004220 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5400cb80 con 0x7f9d54009cb0 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.566+0000 7f9d69d9b640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6b59e640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d54009cb0 msgr2=0x7f9d5400c130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d54009cb0 0x7f9d5400c130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d71034640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d54004de0 0x7f9d54005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d54004220 0x7f9d54004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request con 0x7f9d54004de0 auth_method 0 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request con 0x7f9d54004220 auth_method 0 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:08.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:08.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:39:08.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 10 monclient(hunting): handle_auth_done global_id 7831 payload 293 2024-09-15T01:39:08.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d54009cb0 msgr2=0x7f9d5400c130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:08.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d54009cb0 0x7f9d5400c130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d54004220 msgr2=0x7f9d54004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:39:08.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d54004220 0x7f9d54004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 10 monclient: _finish_hunting 0 2024-09-15T01:39:08.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 1 monclient: found mon.f 2024-09-15T01:39:08.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:08.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:39:08.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 1 -- 172.21.15.178:0/914717980 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d540034e0 con 0x7f9d54004de0 2024-09-15T01:39:08.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 10 monclient: _finish_auth 0 2024-09-15T01:39:08.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6a59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:08.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.567+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d54004220 0x7f9d54004780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:39:08.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.568+0000 7f9d6a59c640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d54004de0 0x7f9d54005250 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0a6ec0 tx=0x7f9d60008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e56d95a7655ed7f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:08.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9d600347f0 con 0x7f9d54004de0 2024-09-15T01:39:08.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d69d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:08.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d69d9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:39:08.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d69d9b640 10 monclient: dump: 2024-09-15T01:39:08.584 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:08.584 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:08.584 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.584 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.584 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:08.584 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:08.584 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:08.584 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:08.584 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:08.584 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:08.584 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6002f030 con 0x7f9d54004de0 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d71034640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d69d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d71034640 10 monclient: get_monmap_and_config success 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d69d9b640 1 -- 172.21.15.178:0/914717980 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9d60034df0 con 0x7f9d54004de0 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d71034640 4 set_mon_vals no callback set 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d69d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:08.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d69d9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.569+0000 7f9d69d9b640 10 monclient: dump: 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:08.586 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.570+0000 7f9d71034640 10 monclient: shutdown 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.570+0000 7f9d71034640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.570+0000 7f9d71034640 1 -- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d54004de0 msgr2=0x7f9d54005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.570+0000 7f9d71034640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d54004de0 0x7f9d54005250 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0a6ec0 tx=0x7f9d60008040 comp rx=0 tx=0).stop 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.570+0000 7f9d71034640 1 -- 172.21.15.178:0/914717980 shutdown_connections 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.570+0000 7f9d71034640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f9d54009cb0 0x7f9d5400c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.570+0000 7f9d71034640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d54004de0 0x7f9d54005250 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.570+0000 7f9d71034640 1 --2- 172.21.15.178:0/914717980 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9d54004220 0x7f9d54004780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.570+0000 7f9d71034640 1 -- 172.21.15.178:0/914717980 >> 172.21.15.178:0/914717980 conn(0x7f9d6c0a5ed0 msgr2=0x7f9d6c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:08.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.570+0000 7f9d71034640 1 -- 172.21.15.178:0/914717980 shutdown_connections 2024-09-15T01:39:08.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.570+0000 7f9d71034640 1 -- 172.21.15.178:0/914717980 wait complete. 2024-09-15T01:39:08.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.571+0000 7f9d71034640 10 monclient: build_initial_monmap 2024-09-15T01:39:08.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.571+0000 7f9d71034640 10 monclient: monmap: 2024-09-15T01:39:08.588 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:08.588 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:39:08.588 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:39:08.588 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:39:08.588 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:08.588 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:39:08.588 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:39:08.588 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:39:08.588 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 1 Processor -- start 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 1 -- start start 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 10 monclient: init 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 10 monclient: _add_conns ranks=[3,4,8,2,7,6,0,5,1] 2024-09-15T01:39:08.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d54004220 0x7f9d6c141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 10 monclient(hunting): picked mon.noname-g con 0x7f9d54004220 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d54004de0 0x7f9d6c09c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 10 monclient(hunting): picked mon.noname-c con 0x7f9d54004de0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d54009cb0 0x7f9d6c135b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 10 monclient(hunting): picked mon.noname-e con 0x7f9d54009cb0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c09f870 con 0x7f9d54004220 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c096c30 con 0x7f9d54004de0 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c141c60 con 0x7f9d54009cb0 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:08.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d71034640 10 monclient(hunting): authenticate will time out at 1153.322814s 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d6a59c640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d54004de0 msgr2=0x7f9d6c09c040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d6a59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d54004de0 0x7f9d6c09c040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d6b59e640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d54009cb0 0x7f9d6c135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d6ad9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d54004220 0x7f9d6c141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.572+0000 7f9d6b59e640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d54009cb0 0x7f9d6c135b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43722/0 (socket says 172.21.15.178:43722) 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 1 -- 172.21.15.178:0/1185418860 learned_addr learned my addr 172.21.15.178:0/1185418860 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request con 0x7f9d54009cb0 auth_method 0 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request con 0x7f9d54004220 auth_method 0 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:08.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 10 monclient(hunting): handle_auth_done global_id 34307 payload 824 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 1 -- 172.21.15.178:0/1185418860 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d54004de0 msgr2=0x7f9d6c09c040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/1185418860 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d54004de0 0x7f9d6c09c040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 1 -- 172.21.15.178:0/1185418860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d54004220 msgr2=0x7f9d6c141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/1185418860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d54004220 0x7f9d6c141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 10 monclient: _finish_hunting 0 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 1 monclient: found mon.noname-e 2024-09-15T01:39:08.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 1 -- 172.21.15.178:0/1185418860 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0972f0 con 0x7f9d54009cb0 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/1185418860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d54004220 0x7f9d6c141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d6b59e640 1 --2- 172.21.15.178:0/1185418860 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d54009cb0 0x7f9d6c135b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009650 tx=0x7f9d64009040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d52ffd640 1 -- 172.21.15.178:0/1185418860 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9d64009720 con 0x7f9d54009cb0 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d52ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d52ffd640 10 monclient: dump: 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.593 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d52ffd640 10 monclient: _finish_auth 0 2024-09-15T01:39:08.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d52ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d52ffd640 1 -- 172.21.15.178:0/1185418860 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6400f040 con 0x7f9d54009cb0 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d71034640 5 monclient: authenticate success, global_id 34307 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d71034640 10 monclient: _renew_subs 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d71034640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d71034640 1 -- 172.21.15.178:0/1185418860 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c098810 con 0x7f9d54009cb0 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d52ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d52ffd640 1 -- 172.21.15.178:0/1185418860 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9d64009d80 con 0x7f9d54009cb0 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d52ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d52ffd640 10 monclient: dump: 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.595 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d71034640 10 monclient: _renew_subs 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d68d99640 4 set_mon_vals no callback set 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d71034640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.573+0000 7f9d71034640 1 -- 172.21.15.178:0/1185418860 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c1375d0 con 0x7f9d54009cb0 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.574+0000 7f9d52ffd640 1 -- 172.21.15.178:0/1185418860 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9d64009900 con 0x7f9d54009cb0 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.575+0000 7f9d71034640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.575+0000 7f9d71034640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.575+0000 7f9d71034640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.575+0000 7f9d71034640 1 -- 172.21.15.178:0/1185418860 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c04b720 con 0x7f9d54009cb0 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.575+0000 7f9d52ffd640 1 --2- 172.21.15.178:0/1185418860 >> v2:172.21.15.204:6812/28210 conn(0x7f9d5803ada0 0x7f9d5803d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.575+0000 7f9d52ffd640 1 -- 172.21.15.178:0/1185418860 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f9d6404ca50 con 0x7f9d54009cb0 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.575+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/1185418860 >> v2:172.21.15.204:6812/28210 conn(0x7f9d5803ada0 0x7f9d5803d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.575+0000 7f9d6ad9d640 10 monclient: get_auth_request con 0x7f9d5803ada0 auth_method 0 2024-09-15T01:39:08.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.576+0000 7f9d6ad9d640 1 --2- 172.21.15.178:0/1185418860 >> v2:172.21.15.204:6812/28210 conn(0x7f9d5803ada0 0x7f9d5803d250 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c007ac0 tx=0x7f9d5c01b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:08.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.580+0000 7f9d52ffd640 1 -- 172.21.15.178:0/1185418860 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9d64016b20 con 0x7f9d54009cb0 2024-09-15T01:39:08.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.580+0000 7f9d52ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:08.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.580+0000 7f9d52ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:39:08.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.798+0000 7f9d71034640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:39:08.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.798+0000 7f9d71034640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:39:08.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.798+0000 7f9d71034640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:39:08.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.798+0000 7f9d71034640 1 -- 172.21.15.178:0/1185418860 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9d6c098450 con 0x7f9d54009cb0 2024-09-15T01:39:08.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.799+0000 7f9d52ffd640 1 -- 172.21.15.178:0/1185418860 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3171 (secure 0 0 0) 0x7f9d64016340 con 0x7f9d54009cb0 2024-09-15T01:39:08.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.799+0000 7f9d52ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:39:08.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.799+0000 7f9d52ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:39:08.801 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":94,"quorum":[0,1,2,3,5,6,8],"quorum_names":["a","f","b","g","h","d","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:39:08.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.800+0000 7f9d71034640 1 -- 172.21.15.178:0/1185418860 >> v2:172.21.15.204:6812/28210 conn(0x7f9d5803ada0 msgr2=0x7f9d5803d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:08.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.800+0000 7f9d71034640 1 --2- 172.21.15.178:0/1185418860 >> v2:172.21.15.204:6812/28210 conn(0x7f9d5803ada0 0x7f9d5803d250 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c007ac0 tx=0x7f9d5c01b040 comp rx=0 tx=0).stop 2024-09-15T01:39:08.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.800+0000 7f9d71034640 10 monclient: shutdown 2024-09-15T01:39:08.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.800+0000 7f9d71034640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:08.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.801+0000 7f9d71034640 1 -- 172.21.15.178:0/1185418860 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d54009cb0 msgr2=0x7f9d6c135b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:08.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.801+0000 7f9d71034640 1 --2- 172.21.15.178:0/1185418860 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d54009cb0 0x7f9d6c135b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009650 tx=0x7f9d64009040 comp rx=0 tx=0).stop 2024-09-15T01:39:08.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.801+0000 7f9d71034640 1 -- 172.21.15.178:0/1185418860 shutdown_connections 2024-09-15T01:39:08.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.801+0000 7f9d71034640 1 --2- 172.21.15.178:0/1185418860 >> v2:172.21.15.204:6812/28210 conn(0x7f9d5803ada0 0x7f9d5803d250 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.801+0000 7f9d71034640 1 --2- 172.21.15.178:0/1185418860 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9d54009cb0 0x7f9d6c135b20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.801+0000 7f9d71034640 1 --2- 172.21.15.178:0/1185418860 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d54004de0 0x7f9d6c09c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.801+0000 7f9d71034640 1 --2- 172.21.15.178:0/1185418860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d54004220 0x7f9d6c141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.801+0000 7f9d71034640 1 -- 172.21.15.178:0/1185418860 >> 172.21.15.178:0/1185418860 conn(0x7f9d6c0a5ed0 msgr2=0x7f9d6c0a5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:08.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.801+0000 7f9d71034640 1 -- 172.21.15.178:0/1185418860 shutdown_connections 2024-09-15T01:39:08.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.801+0000 7f9d71034640 1 -- 172.21.15.178:0/1185418860 wait complete. 2024-09-15T01:39:08.813 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-15T01:39:08.813 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-09-15T01:39:08.814 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:39:08.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.905+0000 7fe2fc39d640 10 monclient: get_monmap_and_config 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.905+0000 7fe2fc39d640 10 monclient: build_initial_monmap 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.905+0000 7fe2fc39d640 10 monclient: monmap: 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:39:08.906785+0000 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:39:08.906785+0000 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:08.908 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:39:08.909 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:39:08.909 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.906+0000 7fe2fc39d640 1 Processor -- start 2024-09-15T01:39:08.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.906+0000 7fe2fc39d640 1 -- start start 2024-09-15T01:39:08.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.906+0000 7fe2fc39d640 10 monclient: init 2024-09-15T01:39:08.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 10 monclient: _add_conns ranks=[10,5,2,0,8,6,7,3,1,4,9] 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe2f40a2930 0x7fe2f4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 10 monclient(hunting): picked mon.noname-f con 0x7fe2f40a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2f409faa0 0x7fe2f409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 10 monclient(hunting): picked mon.noname-h con 0x7fe2f409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f40a08c0 0x7fe2f40a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 10 monclient(hunting): picked mon.noname-b con 0x7fe2f40a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f409e0f0 con 0x7fe2f40a2930 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f409dd80 con 0x7fe2f40a08c0 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f40a2d00 con 0x7fe2f409faa0 2024-09-15T01:39:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fc39d640 10 monclient(hunting): authenticate will time out at 1153.657345s 2024-09-15T01:39:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f40a08c0 0x7fe2f40a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f40a08c0 0x7fe2f40a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55344/0 (socket says 172.21.15.178:55344) 2024-09-15T01:39:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 1 -- 172.21.15.178:0/4175183193 learned_addr learned my addr 172.21.15.178:0/4175183193 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 10 monclient(hunting): get_auth_request con 0x7fe2f40a08c0 auth_method 0 2024-09-15T01:39:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:08.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa139640 1 --2- 172.21.15.178:0/4175183193 >> v2:172.21.15.204:3300/0 conn(0x7fe2f40a2930 0x7fe2f4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa139640 10 monclient(hunting): get_auth_request con 0x7fe2f40a2930 auth_method 0 2024-09-15T01:39:08.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:08.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:08.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:08.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2f9938640 1 --2- 172.21.15.178:0/4175183193 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2f409faa0 0x7fe2f409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 10 monclient(hunting): handle_auth_done global_id 34274 payload 274 2024-09-15T01:39:08.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 1 -- 172.21.15.178:0/4175183193 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2f409faa0 msgr2=0x7fe2f409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:39:08.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 1 --2- 172.21.15.178:0/4175183193 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2f409faa0 0x7fe2f409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 1 -- 172.21.15.178:0/4175183193 >> v2:172.21.15.204:3300/0 conn(0x7fe2f40a2930 msgr2=0x7fe2f4132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 1 --2- 172.21.15.178:0/4175183193 >> v2:172.21.15.204:3300/0 conn(0x7fe2f40a2930 0x7fe2f4132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 10 monclient: _finish_hunting 0 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 1 monclient: found mon.noname-b 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2fa93a640 1 -- 172.21.15.178:0/4175183193 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f404bf60 con 0x7fe2f40a08c0 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2f9938640 10 monclient: get_auth_request con 0x7fe2f409faa0 auth_method 0 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.907+0000 7fe2f9938640 1 --2- 172.21.15.178:0/4175183193 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2f409faa0 0x7fe2f409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2fa93a640 1 --2- 172.21.15.178:0/4175183193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f40a08c0 0x7fe2f40a0d60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009f20 tx=0x7fe2ec009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc54e444215c7bcc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2f9137640 1 -- 172.21.15.178:0/4175183193 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2ec007880 con 0x7fe2f40a08c0 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2f9137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2f9137640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:39:08.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2f9137640 10 monclient: dump: 2024-09-15T01:39:08.924 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:08.924 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:08.924 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.924 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.924 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:08.924 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:08.924 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:08.924 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:08.924 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:08.924 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:08.924 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:08.924 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2f9137640 10 monclient: _finish_auth 0 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2f9137640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2f9137640 1 -- 172.21.15.178:0/4175183193 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec02d050 con 0x7fe2f40a08c0 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2fc39d640 5 monclient: authenticate success, global_id 34274 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2fc39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2f9137640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2fc39d640 10 monclient: get_monmap_and_config success 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2f9137640 1 -- 172.21.15.178:0/4175183193 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2ec030360 con 0x7fe2f40a08c0 2024-09-15T01:39:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2fc39d640 4 set_mon_vals no callback set 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2f9137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2f9137640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.908+0000 7fe2f9137640 10 monclient: dump: 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 10 monclient: shutdown 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 1 -- 172.21.15.178:0/4175183193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f40a08c0 msgr2=0x7fe2f40a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 1 --2- 172.21.15.178:0/4175183193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f40a08c0 0x7fe2f40a0d60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009f20 tx=0x7fe2ec009840 comp rx=0 tx=0).stop 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 1 -- 172.21.15.178:0/4175183193 shutdown_connections 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 1 --2- 172.21.15.178:0/4175183193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe2f40a08c0 0x7fe2f40a0d60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 1 --2- 172.21.15.178:0/4175183193 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2f409faa0 0x7fe2f409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 1 --2- 172.21.15.178:0/4175183193 >> v2:172.21.15.204:3300/0 conn(0x7fe2f40a2930 0x7fe2f4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 1 -- 172.21.15.178:0/4175183193 >> 172.21.15.178:0/4175183193 conn(0x7fe2f40a5ed0 msgr2=0x7fe2f40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 1 -- 172.21.15.178:0/4175183193 shutdown_connections 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 1 -- 172.21.15.178:0/4175183193 wait complete. 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 10 monclient: build_initial_monmap 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.909+0000 7fe2fc39d640 10 monclient: monmap: 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:39:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:39:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:39:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:08.929 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 1 Processor -- start 2024-09-15T01:39:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 1 -- start start 2024-09-15T01:39:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 10 monclient: init 2024-09-15T01:39:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 10 monclient: _add_conns ranks=[4,0,8,7,3,5,1,6,2] 2024-09-15T01:39:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe2f409faa0 0x7fe2f409ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 10 monclient(hunting): picked mon.noname-c con 0x7fe2f409faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:39:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f40a08c0 0x7fe2f41315e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 10 monclient(hunting): picked mon.noname-a con 0x7fe2f40a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:39:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe2f40a2930 0x7fe2f411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fa139640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe2f409faa0 msgr2=0x7fe2f409ad10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:39:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 10 monclient(hunting): picked mon.noname-e con 0x7fe2f40a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:39:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f411e700 con 0x7fe2f40a08c0 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f411f010 con 0x7fe2f409faa0 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f40a2390 con 0x7fe2f40a2930 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fa139640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe2f409faa0 0x7fe2f409ad10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fc39d640 10 monclient(hunting): authenticate will time out at 1153.660664s 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2f9938640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f40a08c0 0x7fe2f41315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fa93a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe2f40a2930 0x7fe2f411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fa93a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe2f40a2930 0x7fe2f411c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43730/0 (socket says 172.21.15.178:43730) 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fa93a640 1 -- 172.21.15.178:0/1509227315 learned_addr learned my addr 172.21.15.178:0/1509227315 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2f9938640 10 monclient(hunting): get_auth_request con 0x7fe2f40a08c0 auth_method 0 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2f9938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2f9938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2f9938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fa93a640 10 monclient(hunting): get_auth_request con 0x7fe2f40a2930 auth_method 0 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fa93a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fa93a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fa93a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2f9938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2f9938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2f9938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fa93a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fa93a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.910+0000 7fe2fa93a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2f9938640 10 monclient(hunting): handle_auth_done global_id 34398 payload 824 2024-09-15T01:39:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2f9938640 1 -- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe2f40a2930 msgr2=0x7fe2f411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2f9938640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe2f40a2930 0x7fe2f411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2f9938640 1 -- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe2f409faa0 msgr2=0x7fe2f409ad10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:39:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2f9938640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe2f409faa0 0x7fe2f409ad10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2f9938640 10 monclient: _finish_hunting 0 2024-09-15T01:39:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2f9938640 1 monclient: found mon.noname-a 2024-09-15T01:39:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2f9938640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2f9938640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2f9938640 1 -- 172.21.15.178:0/1509227315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec009ae0 con 0x7fe2f40a08c0 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2fa93a640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe2f40a2930 0x7fe2f411c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2f9938640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f40a08c0 0x7fe2f41315e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400c850 tx=0x7fe2e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2e4007aa0 con 0x7fe2f40a08c0 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2ea7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2ea7fc640 10 monclient: dump: 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2ea7fc640 10 monclient: _finish_auth 0 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e4007c40 con 0x7fe2f40a08c0 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2fc39d640 5 monclient: authenticate success, global_id 34398 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2fc39d640 10 monclient: _renew_subs 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2fc39d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2fc39d640 1 -- 172.21.15.178:0/1509227315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f409d630 con 0x7fe2f40a08c0 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2ea7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2e4007db0 con 0x7fe2f40a08c0 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2ea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2ea7fc640 10 monclient: dump: 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2f8936640 4 set_mon_vals no callback set 2024-09-15T01:39:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2fc39d640 10 monclient: _renew_subs 2024-09-15T01:39:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2fc39d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:39:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.911+0000 7fe2fc39d640 1 -- 172.21.15.178:0/1509227315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f41429b0 con 0x7fe2f40a08c0 2024-09-15T01:39:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fc39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fc39d640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f4131b20 0x7fe2f411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fc39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fc39d640 1 -- 172.21.15.178:0/1509227315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f4131ef0 con 0x7fe2f4131b20 2024-09-15T01:39:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa139640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f4131b20 0x7fe2f411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa139640 10 monclient: get_auth_request con 0x7fe2f4131b20 auth_method 0 2024-09-15T01:39:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe2e4007620 con 0x7fe2f40a08c0 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa139640 10 monclient(hunting): handle_auth_done global_id 34407 payload 824 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa139640 1 -- 172.21.15.178:0/1509227315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2f404c4a0 con 0x7fe2f4131b20 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2ea7fc640 1 --2- 172.21.15.178:0/1509227315 >> v2:172.21.15.204:6812/28210 conn(0x7fe2d80397b0 0x7fe2d803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fe2e404bbd0 con 0x7fe2f40a08c0 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa139640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f4131b20 0x7fe2f411ca70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0006f90 tx=0x7fe2f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa93a640 1 --2- 172.21.15.178:0/1509227315 >> v2:172.21.15.204:6812/28210 conn(0x7fe2d80397b0 0x7fe2d803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2fa93a640 10 monclient: get_auth_request con 0x7fe2d80397b0 auth_method 0 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2f0012a70 con 0x7fe2f4131b20 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2ea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.912+0000 7fe2ea7fc640 10 monclient: dump: 2024-09-15T01:39:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:08.941 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.913+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fe2f0012bd0 con 0x7fe2f4131b20 2024-09-15T01:39:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.913+0000 7fe2ea7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.913+0000 7fe2ea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:39:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.913+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f4131b20 msgr2=0x7fe2f411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.913+0000 7fe2ea7fc640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f4131b20 0x7fe2f411ca70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0006f90 tx=0x7fe2f0008040 comp rx=0 tx=0).stop 2024-09-15T01:39:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.913+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f4131b20 msgr2=0x7fe2f411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.913+0000 7fe2ea7fc640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f4131b20 0x7fe2f411ca70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:08.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.913+0000 7fe2fa93a640 1 --2- 172.21.15.178:0/1509227315 >> v2:172.21.15.204:6812/28210 conn(0x7fe2d80397b0 0x7fe2d803bc60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009f20 tx=0x7fe2ec036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.918+0000 7fe2fc39d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:39:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.918+0000 7fe2fc39d640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f411cfb0 0x7fe2f4096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.918+0000 7fe2fc39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.918+0000 7fe2fc39d640 1 -- 172.21.15.178:0/1509227315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f4070e90 con 0x7fe2f411cfb0 2024-09-15T01:39:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2fa139640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f411cfb0 0x7fe2f4096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2fa139640 10 monclient: get_auth_request con 0x7fe2f411cfb0 auth_method 0 2024-09-15T01:39:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2fa139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2fa139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:08.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2fa139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2fa139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2fa139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2fa139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2fa139640 10 monclient(hunting): handle_auth_done global_id 34416 payload 824 2024-09-15T01:39:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2fa139640 1 -- 172.21.15.178:0/1509227315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe2f404bc80 con 0x7fe2f411cfb0 2024-09-15T01:39:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2fa139640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f411cfb0 0x7fe2f4096a30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0006f90 tx=0x7fe2f001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2f0012d60 con 0x7fe2f411cfb0 2024-09-15T01:39:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2ea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:39:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2ea7fc640 10 monclient: dump: 2024-09-15T01:39:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:08.944 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:08.945 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:08.946 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9119 (secure 0 0 0) 0x7fe2f001ddd0 con 0x7fe2f411cfb0 2024-09-15T01:39:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2ea7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:39:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2ea7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:39:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f411cfb0 msgr2=0x7fe2f4096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2ea7fc640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f411cfb0 0x7fe2f4096a30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0006f90 tx=0x7fe2f001b000 comp rx=0 tx=0).stop 2024-09-15T01:39:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2ea7fc640 1 -- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f411cfb0 msgr2=0x7fe2f4096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:08.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.919+0000 7fe2ea7fc640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f411cfb0 0x7fe2f4096a30 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0006f90 tx=0x7fe2f001b000 comp rx=0 tx=0).stop 2024-09-15T01:39:08.946 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:39:08.946 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:39:08.946 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 94, 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:39:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:39:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:39:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:39:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:39:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:39:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:39:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:39:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:39:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:39:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:39:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:39:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:39:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:39:08.968 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:39:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.920+0000 7fe2fc39d640 1 -- 172.21.15.178:0/1509227315 >> v2:172.21.15.204:6812/28210 conn(0x7fe2d80397b0 msgr2=0x7fe2d803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.920+0000 7fe2fc39d640 1 --2- 172.21.15.178:0/1509227315 >> v2:172.21.15.204:6812/28210 conn(0x7fe2d80397b0 0x7fe2d803bc60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009f20 tx=0x7fe2ec036000 comp rx=0 tx=0).stop 2024-09-15T01:39:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.920+0000 7fe2fc39d640 10 monclient: shutdown 2024-09-15T01:39:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.920+0000 7fe2fc39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.920+0000 7fe2fc39d640 1 -- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f40a08c0 msgr2=0x7fe2f41315e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.920+0000 7fe2fc39d640 1 --2- 172.21.15.178:0/1509227315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2f40a08c0 0x7fe2f41315e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400c850 tx=0x7fe2e4008040 comp rx=0 tx=0).stop 2024-09-15T01:39:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.920+0000 7fe2fa93a640 1 -- 172.21.15.178:0/1509227315 reap_dead start 2024-09-15T01:39:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.920+0000 7fe2fa93a640 1 -- 172.21.15.178:0/1509227315 reap_dead start 2024-09-15T01:39:08.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.920+0000 7fe2fc39d640 1 -- 172.21.15.178:0/1509227315 shutdown_connections 2024-09-15T01:39:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.920+0000 7fe2fc39d640 1 -- 172.21.15.178:0/1509227315 >> 172.21.15.178:0/1509227315 conn(0x7fe2f40a5ed0 msgr2=0x7fe2f409e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.920+0000 7fe2fc39d640 1 -- 172.21.15.178:0/1509227315 shutdown_connections 2024-09-15T01:39:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:08.920+0000 7fe2fc39d640 1 -- 172.21.15.178:0/1509227315 wait complete. 2024-09-15T01:39:08.971 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:39:08.971 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-09-15T01:39:08.971 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:39:09.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.063+0000 7fa5ebb5d640 10 monclient: get_monmap_and_config 2024-09-15T01:39:09.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.063+0000 7fa5ebb5d640 10 monclient: build_initial_monmap 2024-09-15T01:39:09.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.063+0000 7fa5ebb5d640 10 monclient: monmap: 2024-09-15T01:39:09.065 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:39:09.064565+0000 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:39:09.064565+0000 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:09.066 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:09.067 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:09.067 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:39:09.067 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:39:09.067 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 1 Processor -- start 2024-09-15T01:39:09.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 1 -- start start 2024-09-15T01:39:09.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 10 monclient: init 2024-09-15T01:39:09.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 10 monclient: _add_conns ranks=[2,1,10,3,7,6,0,4,9,8,5] 2024-09-15T01:39:09.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 0x7fa5e4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 10 monclient(hunting): picked mon.noname-b con 0x7fa5e40a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa5e409faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa5e409fe70 0x7fa5e40a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 10 monclient(hunting): picked mon.noname-f con 0x7fa5e409fe70 addr v2:172.21.15.204:3300/0 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e404bf60 con 0x7fa5e409faa0 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e409dd80 con 0x7fa5e409fe70 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40a2d00 con 0x7fa5e40a2930 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.064+0000 7fa5ebb5d640 10 monclient(hunting): authenticate will time out at 1153.814924s 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 0x7fa5e4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 0x7fa5e4132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55346/0 (socket says 172.21.15.178:55346) 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 1 -- 172.21.15.178:0/4184725691 learned_addr learned my addr 172.21.15.178:0/4184725691 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 10 monclient(hunting): get_auth_request con 0x7fa5e40a2930 auth_method 0 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5ea0fa640 1 --2- 172.21.15.178:0/4184725691 >> v2:172.21.15.204:3300/0 conn(0x7fa5e409fe70 0x7fa5e40a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5ea0fa640 10 monclient(hunting): get_auth_request con 0x7fa5e409fe70 auth_method 0 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5ea0fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5ea0fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5ea0fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5ea0fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5ea0fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5ea0fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 10 monclient(hunting): handle_auth_done global_id 34283 payload 274 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 1 -- 172.21.15.178:0/4184725691 >> v2:172.21.15.204:3300/0 conn(0x7fa5e409fe70 msgr2=0x7fa5e40a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 1 --2- 172.21.15.178:0/4184725691 >> v2:172.21.15.204:3300/0 conn(0x7fa5e409fe70 0x7fa5e40a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 1 -- 172.21.15.178:0/4184725691 >> v1:172.21.15.204:6789/0 conn(0x7fa5e409faa0 legacy=0x7fa5e40a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 1 monclient: found mon.noname-b 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 1 -- 172.21.15.178:0/4184725691 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e40a0310 con 0x7fa5e40a2930 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.065+0000 7fa5e98f9640 1 --2- 172.21.15.178:0/4184725691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 0x7fa5e4132de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009910 tx=0x7fa5e00091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7d35fff6d3cdbd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5e88f7640 1 -- 172.21.15.178:0/4184725691 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5e002ad20 con 0x7fa5e40a2930 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5e88f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5e88f7640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5e88f7640 10 monclient: dump: 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.080 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5e88f7640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5e88f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5e88f7640 1 -- 172.21.15.178:0/4184725691 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e0031820 con 0x7fa5e40a2930 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 5 monclient: authenticate success, global_id 34283 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5e88f7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:39:09.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5e88f7640 1 -- 172.21.15.178:0/4184725691 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5e002aea0 con 0x7fa5e40a2930 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5e88f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 4 set_mon_vals no callback set 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5e88f7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5e88f7640 10 monclient: dump: 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.082 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 10 monclient: shutdown 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4184725691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 msgr2=0x7fa5e4132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 1 --2- 172.21.15.178:0/4184725691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 0x7fa5e4132de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009910 tx=0x7fa5e00091d0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4184725691 shutdown_connections 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 1 --2- 172.21.15.178:0/4184725691 >> v2:172.21.15.204:3300/0 conn(0x7fa5e409fe70 0x7fa5e40a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 1 --2- 172.21.15.178:0/4184725691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 0x7fa5e4132de0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4184725691 >> 172.21.15.178:0/4184725691 conn(0x7fa5e40a5ed0 msgr2=0x7fa5e40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:09.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4184725691 shutdown_connections 2024-09-15T01:39:09.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.066+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4184725691 wait complete. 2024-09-15T01:39:09.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.067+0000 7fa5ebb5d640 10 monclient: build_initial_monmap 2024-09-15T01:39:09.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.067+0000 7fa5ebb5d640 10 monclient: monmap: 2024-09-15T01:39:09.084 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:09.084 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:39:09.084 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:39:09.084 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:39:09.084 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:09.084 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:39:09.084 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:39:09.084 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.067+0000 7fa5ebb5d640 1 Processor -- start 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.067+0000 7fa5ebb5d640 1 -- start start 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.067+0000 7fa5ebb5d640 10 monclient: init 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.067+0000 7fa5ebb5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.067+0000 7fa5ebb5d640 10 monclient: _add_conns ranks=[4,6,2,1,0,3,8,7,5] 2024-09-15T01:39:09.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.067+0000 7fa5ebb5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa5e409faa0 0x7fa5e409b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.067+0000 7fa5ebb5d640 10 monclient(hunting): picked mon.noname-c con 0x7fa5e409faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:39:09.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.067+0000 7fa5ebb5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa5e409fe70 0x7fa5e4131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 10 monclient(hunting): picked mon.noname-d con 0x7fa5e409fe70 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:39:09.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 0x7fa5e4141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 10 monclient(hunting): picked mon.noname-b con 0x7fa5e40a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:39:09.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e411ed20 con 0x7fa5e40a2930 2024-09-15T01:39:09.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e409c0f0 con 0x7fa5e409faa0 2024-09-15T01:39:09.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e409f870 con 0x7fa5e409fe70 2024-09-15T01:39:09.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 10 monclient(hunting): authenticate will time out at 1153.818029s 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5e98f9640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa5e409faa0 msgr2=0x7fa5e409b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5e98f9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa5e409faa0 0x7fa5e409b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 0x7fa5e4141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5e90f8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa5e409fe70 0x7fa5e4131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 0x7fa5e4141eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55358/0 (socket says 172.21.15.178:55358) 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 1 -- 172.21.15.178:0/4200913752 learned_addr learned my addr 172.21.15.178:0/4200913752 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 10 monclient(hunting): get_auth_request con 0x7fa5e40a2930 auth_method 0 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5e90f8640 10 monclient(hunting): get_auth_request con 0x7fa5e409fe70 auth_method 0 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5e90f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5e90f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5e90f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5e90f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5e90f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5e90f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 10 monclient(hunting): handle_auth_done global_id 34292 payload 824 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 1 -- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa5e409fe70 msgr2=0x7fa5e4131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa5e409fe70 0x7fa5e4131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 1 -- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa5e409faa0 msgr2=0x7fa5e409b540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa5e409faa0 0x7fa5e409b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 1 monclient: found mon.noname-b 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 1 -- 172.21.15.178:0/4200913752 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e0009450 con 0x7fa5e40a2930 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5e90f8640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa5e409fe70 0x7fa5e4131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ea0fa640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 0x7fa5e4141eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009e90 tx=0x7fa5dc00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5dc00cd40 con 0x7fa5e40a2930 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5d9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5d9ffb640 10 monclient: dump: 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.089 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5d9ffb640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5d9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc00f040 con 0x7fa5e40a2930 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 5 monclient: authenticate success, global_id 34292 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5d9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5dc00cee0 con 0x7fa5e40a2930 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 10 monclient: _renew_subs 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4200913752 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e411ea40 con 0x7fa5e40a2930 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5d9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5d9ffb640 10 monclient: dump: 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.091 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5dbfff640 4 set_mon_vals no callback set 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 10 monclient: _renew_subs 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.068+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4200913752 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e411d5e0 con 0x7fa5e40a2930 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5ebb5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5ebb5d640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e411db50 0x7fa5e41423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5ebb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4200913752 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e404c4a0 con 0x7fa5e411db50 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa5dc014480 con 0x7fa5e40a2930 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5e98f9640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e411db50 0x7fa5e41423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5e98f9640 10 monclient: get_auth_request con 0x7fa5e411db50 auth_method 0 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5e98f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5e98f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5e98f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5e98f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5e98f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5e98f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5d9ffb640 1 --2- 172.21.15.178:0/4200913752 >> v2:172.21.15.204:6812/28210 conn(0x7fa5c80397b0 0x7fa5c803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.069+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fa5dc04c760 con 0x7fa5e40a2930 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5e98f9640 10 monclient(hunting): handle_auth_done global_id 34301 payload 824 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5e98f9640 1 -- 172.21.15.178:0/4200913752 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5e405ed70 con 0x7fa5e411db50 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5e90f8640 1 --2- 172.21.15.178:0/4200913752 >> v2:172.21.15.204:6812/28210 conn(0x7fa5c80397b0 0x7fa5c803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5e90f8640 10 monclient: get_auth_request con 0x7fa5c80397b0 auth_method 0 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5e98f9640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e411db50 0x7fa5e41423f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4067590 tx=0x7fa5e0032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5e0031e60 con 0x7fa5e411db50 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5d9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:39:09.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5d9ffb640 10 monclient: dump: 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.095 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.096 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.096 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5e90f8640 1 --2- 172.21.15.178:0/4200913752 >> v2:172.21.15.204:6812/28210 conn(0x7fa5c80397b0 0x7fa5c803bc60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4007fd0 tx=0x7fa5d4009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fa5e0032650 con 0x7fa5e411db50 2024-09-15T01:39:09.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5d9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:09.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5d9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:39:09.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e411db50 msgr2=0x7fa5e41423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5d9ffb640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e411db50 0x7fa5e41423f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4067590 tx=0x7fa5e0032000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e411db50 msgr2=0x7fa5e41423f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:09.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.070+0000 7fa5d9ffb640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e411db50 0x7fa5e41423f0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4067590 tx=0x7fa5e0032000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5ebb5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:39:09.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5ebb5d640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e4131e60 0x7fa5e4142930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5ebb5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4200913752 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e409daa0 con 0x7fa5e4131e60 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5e98f9640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e4131e60 0x7fa5e4142930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5e98f9640 10 monclient: get_auth_request con 0x7fa5e4131e60 auth_method 0 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5e98f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5e98f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5e98f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5e98f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5e98f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5e98f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5e98f9640 10 monclient(hunting): handle_auth_done global_id 34310 payload 824 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5e98f9640 1 -- 172.21.15.178:0/4200913752 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa5e404bc80 con 0x7fa5e4131e60 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5e98f9640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e4131e60 0x7fa5e4142930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009910 tx=0x7fa5e003c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5e0032e80 con 0x7fa5e4131e60 2024-09-15T01:39:09.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5d9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.076+0000 7fa5d9ffb640 10 monclient: dump: 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.100 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.101 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9119 (secure 0 0 0) 0x7fa5e0048430 con 0x7fa5e4131e60 2024-09-15T01:39:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5d9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:39:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5d9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:39:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e4131e60 msgr2=0x7fa5e4142930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5d9ffb640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e4131e60 0x7fa5e4142930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009910 tx=0x7fa5e003c000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5d9ffb640 1 -- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e4131e60 msgr2=0x7fa5e4142930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:09.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5d9ffb640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e4131e60 0x7fa5e4142930 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009910 tx=0x7fa5e003c000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.102 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:39:09.102 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:39:09.102 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:39:09.102 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:39:09.102 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 94, 2024-09-15T01:39:09.102 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:39:09.102 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:39:09.102 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:39:09.102 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:39:09.102 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:39:09.102 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:39:09.103 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:39:09.103 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:39:09.103 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.103 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:39:09.103 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:09.103 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:39:09.103 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:39:09.103 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.104 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:39:09.105 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:39:09.106 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.107 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:39:09.108 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:39:09.108 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.108 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.108 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.108 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:39:09.108 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.108 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.109 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:39:09.110 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:39:09.110 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.110 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:39:09.110 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:39:09.112 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:39:09.114 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.116 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.117 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:39:09.120 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:39:09.120 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.120 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.120 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.120 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:39:09.120 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:39:09.120 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:39:09.121 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:39:09.122 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.123 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.124 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.125 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.125 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:39:09.125 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.125 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.125 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:39:09.125 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:39:09.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4200913752 >> v2:172.21.15.204:6812/28210 conn(0x7fa5c80397b0 msgr2=0x7fa5c803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5ebb5d640 1 --2- 172.21.15.178:0/4200913752 >> v2:172.21.15.204:6812/28210 conn(0x7fa5c80397b0 0x7fa5c803bc60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4007fd0 tx=0x7fa5d4009040 comp rx=0 tx=0).stop 2024-09-15T01:39:09.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5ebb5d640 10 monclient: shutdown 2024-09-15T01:39:09.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5ebb5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:09.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 msgr2=0x7fa5e4141eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5ebb5d640 1 --2- 172.21.15.178:0/4200913752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5e40a2930 0x7fa5e4141eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009e90 tx=0x7fa5dc00b040 comp rx=0 tx=0).stop 2024-09-15T01:39:09.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5ea0fa640 1 -- 172.21.15.178:0/4200913752 reap_dead start 2024-09-15T01:39:09.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5ea0fa640 1 -- 172.21.15.178:0/4200913752 reap_dead start 2024-09-15T01:39:09.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4200913752 shutdown_connections 2024-09-15T01:39:09.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4200913752 >> 172.21.15.178:0/4200913752 conn(0x7fa5e40a5ed0 msgr2=0x7fa5e4135980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:09.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4200913752 shutdown_connections 2024-09-15T01:39:09.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.077+0000 7fa5ebb5d640 1 -- 172.21.15.178:0/4200913752 wait complete. 2024-09-15T01:39:09.127 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:39:09.128 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-15T01:39:09.128 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:39:09.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.219+0000 7fb945c89640 10 monclient: get_monmap_and_config 2024-09-15T01:39:09.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.219+0000 7fb945c89640 10 monclient: build_initial_monmap 2024-09-15T01:39:09.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.219+0000 7fb945c89640 10 monclient: monmap: 2024-09-15T01:39:09.221 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:09.221 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.221 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:39:09.220214+0000 2024-09-15T01:39:09.221 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:39:09.220214+0000 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:39:09.222 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 1 Processor -- start 2024-09-15T01:39:09.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 1 -- start start 2024-09-15T01:39:09.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 10 monclient: init 2024-09-15T01:39:09.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 10 monclient: _add_conns ranks=[3,1,0,10,8,5,7,6,4,2,9] 2024-09-15T01:39:09.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9400a2930 0x7fb940132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 10 monclient(hunting): picked mon.noname-g con 0x7fb9400a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:39:09.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb94009faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:39:09.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb94009fe70 addr v1:172.21.15.178:6789/0 2024-09-15T01:39:09.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94004bf60 con 0x7fb94009fe70 2024-09-15T01:39:09.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94009dd80 con 0x7fb94009faa0 2024-09-15T01:39:09.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400a2d00 con 0x7fb9400a2930 2024-09-15T01:39:09.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb945c89640 10 monclient(hunting): authenticate will time out at 1153.970684s 2024-09-15T01:39:09.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb93ffff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fb94009fe70 0x7fb9400a0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:56672/0 (socket says 172.21.15.178:56672) 2024-09-15T01:39:09.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb93ffff640 1 -- 172.21.15.178:0/3900507263 learned_addr learned my addr 172.21.15.178:0/3900507263 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:39:09.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.220+0000 7fb93f7fe640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9400a2930 0x7fb940132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93f7fe640 10 monclient(hunting): get_auth_request con 0x7fb9400a2930 auth_method 0 2024-09-15T01:39:09.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376436404 0 0) 0x7fb94004bf60 con 0x7fb94009fe70 2024-09-15T01:39:09.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): my global_id is 34425 2024-09-15T01:39:09.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9280034e0 con 0x7fb94009fe70 2024-09-15T01:39:09.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507692318 0 0) 0x7fb94009dd80 con 0x7fb94009faa0 2024-09-15T01:39:09.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): my global_id is 7867 2024-09-15T01:39:09.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb94004bf60 con 0x7fb94009faa0 2024-09-15T01:39:09.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3231968969 0 0) 0x7fb9280034e0 con 0x7fb94009fe70 2024-09-15T01:39:09.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9400a2930 msgr2=0x7fb940132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:39:09.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9400a2930 0x7fb940132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 >> v1:172.21.15.204:6789/0 conn(0x7fb94009faa0 legacy=0x7fb9400a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:39:09.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:39:09.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9400a16e0 con 0x7fb94009fe70 2024-09-15T01:39:09.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb945c89640 5 monclient: authenticate success, global_id 34425 2024-09-15T01:39:09.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fb938001c70 con 0x7fb94009fe70 2024-09-15T01:39:09.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb945c89640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:39:09.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:39:09.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient: dump: 2024-09-15T01:39:09.252 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.252 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.252 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.253 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.253 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.253 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.253 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.253 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.253 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.253 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.254 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.254 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.254 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.254 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.254 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.254 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:39:09.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 >> v1:172.21.15.178:6789/0 conn(0x7fb94009fe70 legacy=0x7fb9400a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient: _add_conns ranks=[4,2,7,8,3,0,6,5,1] 2024-09-15T01:39:09.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb928004780 0x7fb928004b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): picked mon.c con 0x7fb928004780 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:39:09.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb928005090 0x7fb9280099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): picked mon.b con 0x7fb928005090 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:39:09.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb928009f00 0x7fb92800c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): picked mon.i con 0x7fb928009f00 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:39:09.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400a2d00 con 0x7fb928005090 2024-09-15T01:39:09.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb92800c9b0 con 0x7fb928004780 2024-09-15T01:39:09.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb92800cb20 con 0x7fb928009f00 2024-09-15T01:39:09.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb945c89640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93f7fe640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9400a2930 0x7fb940132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93ffff640 1 -- 172.21.15.178:0/3900507263 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb928009f00 msgr2=0x7fb92800c2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93ffff640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb928009f00 0x7fb92800c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93f7fe640 1 -- 172.21.15.178:0/3900507263 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb928004780 msgr2=0x7fb928004b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93f7fe640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb928004780 0x7fb928004b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb928005090 0x7fb9280099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 10 monclient(hunting): get_auth_request con 0x7fb928005090 auth_method 0 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:39:09.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 10 monclient(hunting): handle_auth_done global_id 34425 payload 293 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 1 -- 172.21.15.178:0/3900507263 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb928009f00 msgr2=0x7fb92800c2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb928009f00 0x7fb92800c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 1 -- 172.21.15.178:0/3900507263 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb928004780 msgr2=0x7fb928004b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb928004780 0x7fb928004b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 1 monclient: found mon.b 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93ffff640 1 -- 172.21.15.178:0/3900507263 reap_dead start 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 1 -- 172.21.15.178:0/3900507263 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9280034e0 con 0x7fb928005090 2024-09-15T01:39:09.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.221+0000 7fb93effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb93effd640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb928005090 0x7fb9280099c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb9400a6ec0 tx=0x7fb934007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df1c91b11cd36728 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb934031390 con 0x7fb928005090 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb93e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb93e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb93e7fc640 10 monclient: dump: 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.259 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb934031530 con 0x7fb928005090 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb945c89640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb93e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:09.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb93e7fc640 1 -- 172.21.15.178:0/3900507263 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb934031b20 con 0x7fb928005090 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb945c89640 10 monclient: get_monmap_and_config success 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb945c89640 4 set_mon_vals no callback set 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb93e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb93e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.222+0000 7fb93e7fc640 10 monclient: dump: 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.261 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.223+0000 7fb945c89640 10 monclient: shutdown 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.223+0000 7fb945c89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.223+0000 7fb945c89640 1 -- 172.21.15.178:0/3900507263 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb928005090 msgr2=0x7fb9280099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.223+0000 7fb945c89640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb928005090 0x7fb9280099c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb9400a6ec0 tx=0x7fb934007c80 comp rx=0 tx=0).stop 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.223+0000 7fb945c89640 1 -- 172.21.15.178:0/3900507263 shutdown_connections 2024-09-15T01:39:09.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.223+0000 7fb945c89640 1 --2- 172.21.15.178:0/3900507263 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb928005090 0x7fb9280099c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.223+0000 7fb945c89640 1 -- 172.21.15.178:0/3900507263 >> 172.21.15.178:0/3900507263 conn(0x7fb9400a5ed0 msgr2=0x7fb9400a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.223+0000 7fb945c89640 1 -- 172.21.15.178:0/3900507263 shutdown_connections 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.223+0000 7fb945c89640 1 -- 172.21.15.178:0/3900507263 wait complete. 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.223+0000 7fb945c89640 10 monclient: build_initial_monmap 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.223+0000 7fb945c89640 10 monclient: monmap: 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:39:09.263 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 1 Processor -- start 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 1 -- start start 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 10 monclient: init 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 10 monclient: _add_conns ranks=[0,6,4,5,1,8,2,3,7] 2024-09-15T01:39:09.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928005090 0x7fb94009b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 10 monclient(hunting): picked mon.noname-a con 0x7fb928005090 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb940131920 0x7fb940131cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 10 monclient(hunting): picked mon.noname-d con 0x7fb940131920 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb940141c50 0x7fb940142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 10 monclient(hunting): picked mon.noname-c con 0x7fb940141c50 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94009be30 con 0x7fb928005090 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb94009d6a0 con 0x7fb940141c50 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb94009f870 con 0x7fb940131920 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb945c89640 10 monclient(hunting): authenticate will time out at 1153.974776s 2024-09-15T01:39:09.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93ffff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb940141c50 msgr2=0x7fb940142060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93ffff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb940141c50 0x7fb940142060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928005090 0x7fb94009b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928005090 0x7fb94009b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43402/0 (socket says 172.21.15.178:43402) 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93effd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb940131920 0x7fb940131cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93f7fe640 1 -- 172.21.15.178:0/4019066464 learned_addr learned my addr 172.21.15.178:0/4019066464 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93f7fe640 10 monclient(hunting): get_auth_request con 0x7fb928005090 auth_method 0 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93effd640 10 monclient(hunting): get_auth_request con 0x7fb940131920 auth_method 0 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.224+0000 7fb93effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93f7fe640 10 monclient(hunting): handle_auth_done global_id 34434 payload 824 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93f7fe640 1 -- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb940131920 msgr2=0x7fb940131cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93f7fe640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb940131920 0x7fb940131cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93f7fe640 1 -- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb940141c50 msgr2=0x7fb940142060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93f7fe640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb940141c50 0x7fb940142060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93f7fe640 1 monclient: found mon.noname-a 2024-09-15T01:39:09.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93f7fe640 1 -- 172.21.15.178:0/4019066464 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94011ed20 con 0x7fb928005090 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93f7fe640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928005090 0x7fb94009b270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb93000ec50 tx=0x7fb9300090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb93000ee90 con 0x7fb928005090 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb9277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb9277fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb9277fe640 10 monclient: dump: 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.268 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb9277fe640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb9277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9300096f0 con 0x7fb928005090 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb945c89640 5 monclient: authenticate success, global_id 34434 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb9277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb930013d70 con 0x7fb928005090 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb945c89640 10 monclient: _renew_subs 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb945c89640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb945c89640 1 -- 172.21.15.178:0/4019066464 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94011ea40 con 0x7fb928005090 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb9277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb9277fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb93dffb640 4 set_mon_vals no callback set 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb9277fe640 10 monclient: dump: 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.270 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb945c89640 10 monclient: _renew_subs 2024-09-15T01:39:09.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb945c89640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:39:09.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.225+0000 7fb945c89640 1 -- 172.21.15.178:0/4019066464 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94011d530 con 0x7fb928005090 2024-09-15T01:39:09.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb93000cbc0 con 0x7fb928005090 2024-09-15T01:39:09.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb945c89640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:09.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb9277fe640 1 --2- 172.21.15.178:0/4019066464 >> v2:172.21.15.204:6812/28210 conn(0x7fb92c0397b0 0x7fb92c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb945c89640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb9401425a0 0x7fb940096b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb945c89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb945c89640 1 -- 172.21.15.178:0/4019066464 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb94011ebd0 con 0x7fb9401425a0 2024-09-15T01:39:09.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fb93004bf80 con 0x7fb928005090 2024-09-15T01:39:09.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93ffff640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb9401425a0 0x7fb940096b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93ffff640 10 monclient: get_auth_request con 0x7fb9401425a0 auth_method 0 2024-09-15T01:39:09.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93ffff640 10 monclient(hunting): handle_auth_done global_id 24900 payload 824 2024-09-15T01:39:09.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93effd640 1 --2- 172.21.15.178:0/4019066464 >> v2:172.21.15.204:6812/28210 conn(0x7fb92c0397b0 0x7fb92c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93ffff640 1 -- 172.21.15.178:0/4019066464 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb94004c4a0 con 0x7fb9401425a0 2024-09-15T01:39:09.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93effd640 10 monclient: get_auth_request con 0x7fb92c0397b0 auth_method 0 2024-09-15T01:39:09.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.226+0000 7fb93ffff640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb9401425a0 0x7fb940096b30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb9400a5ea0 tx=0x7fb938002160 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.227+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb93800cd80 con 0x7fb9401425a0 2024-09-15T01:39:09.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.227+0000 7fb9277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.227+0000 7fb9277fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.227+0000 7fb9277fe640 10 monclient: dump: 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.274 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.227+0000 7fb93effd640 1 --2- 172.21.15.178:0/4019066464 >> v2:172.21.15.204:6812/28210 conn(0x7fb92c0397b0 0x7fb92c03bc60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb934001050 tx=0x7fb934003000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.227+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fb938004b20 con 0x7fb9401425a0 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.227+0000 7fb9277fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.227+0000 7fb9277fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.227+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb9401425a0 msgr2=0x7fb940096b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.227+0000 7fb9277fe640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb9401425a0 0x7fb940096b30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb9400a5ea0 tx=0x7fb938002160 comp rx=0 tx=0).stop 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.227+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb9401425a0 msgr2=0x7fb940096b30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.227+0000 7fb9277fe640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb9401425a0 0x7fb940096b30 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb9400a5ea0 tx=0x7fb938002160 comp rx=0 tx=0).stop 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.232+0000 7fb945c89640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb945c89640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb940097070 0x7fb940097440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb945c89640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb945c89640 1 -- 172.21.15.178:0/4019066464 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb94004bc80 con 0x7fb940097070 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb93ffff640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb940097070 0x7fb940097440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb93ffff640 10 monclient: get_auth_request con 0x7fb940097070 auth_method 0 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb93ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb93ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb93ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb93ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb93ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb93ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb93ffff640 10 monclient(hunting): handle_auth_done global_id 24909 payload 824 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb93ffff640 1 -- 172.21.15.178:0/4019066464 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb94005ed70 con 0x7fb940097070 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb93ffff640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb940097070 0x7fb940097440 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb938000f80 tx=0x7fb938011000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb938014af0 con 0x7fb940097070 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb9277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb9277fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb9277fe640 10 monclient: dump: 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.277 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.278 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.278 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.278 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.278 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9119 (secure 0 0 0) 0x7fb938014c50 con 0x7fb940097070 2024-09-15T01:39:09.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb9277fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:39:09.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb9277fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:39:09.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb940097070 msgr2=0x7fb940097440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb9277fe640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb940097070 0x7fb940097440 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb938000f80 tx=0x7fb938011000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb9277fe640 1 -- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb940097070 msgr2=0x7fb940097440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:09.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.233+0000 7fb9277fe640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb940097070 0x7fb940097440 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb938000f80 tx=0x7fb938011000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.279 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:39:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:39:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:39:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:39:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 94, 2024-09-15T01:39:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:39:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:39:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:39:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:39:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:39:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:39:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:39:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:39:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:39:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:39:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:39:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:39:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:39:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:39:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:39:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:39:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:39:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:39:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:39:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:39:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:39:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:39:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:39:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.301 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:39:09.301 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.301 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.301 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:39:09.301 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:39:09.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.234+0000 7fb945c89640 1 -- 172.21.15.178:0/4019066464 >> v2:172.21.15.204:6812/28210 conn(0x7fb92c0397b0 msgr2=0x7fb92c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.234+0000 7fb945c89640 1 --2- 172.21.15.178:0/4019066464 >> v2:172.21.15.204:6812/28210 conn(0x7fb92c0397b0 0x7fb92c03bc60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb934001050 tx=0x7fb934003000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.234+0000 7fb945c89640 10 monclient: shutdown 2024-09-15T01:39:09.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.234+0000 7fb945c89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:09.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.234+0000 7fb945c89640 1 -- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928005090 msgr2=0x7fb94009b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.234+0000 7fb945c89640 1 --2- 172.21.15.178:0/4019066464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb928005090 0x7fb94009b270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb93000ec50 tx=0x7fb9300090f0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.234+0000 7fb93ffff640 1 -- 172.21.15.178:0/4019066464 reap_dead start 2024-09-15T01:39:09.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.234+0000 7fb93ffff640 1 -- 172.21.15.178:0/4019066464 reap_dead start 2024-09-15T01:39:09.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.234+0000 7fb945c89640 1 -- 172.21.15.178:0/4019066464 shutdown_connections 2024-09-15T01:39:09.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.234+0000 7fb945c89640 1 -- 172.21.15.178:0/4019066464 >> 172.21.15.178:0/4019066464 conn(0x7fb9400a5ed0 msgr2=0x7fb94009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:09.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.234+0000 7fb945c89640 1 -- 172.21.15.178:0/4019066464 shutdown_connections 2024-09-15T01:39:09.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.234+0000 7fb945c89640 1 -- 172.21.15.178:0/4019066464 wait complete. 2024-09-15T01:39:09.303 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:39:09.304 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-09-15T01:39:09.304 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:39:09.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.397+0000 7fb07a714640 10 monclient: get_monmap_and_config 2024-09-15T01:39:09.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.397+0000 7fb07a714640 10 monclient: build_initial_monmap 2024-09-15T01:39:09.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.397+0000 7fb07a714640 10 monclient: monmap: 2024-09-15T01:39:09.399 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:09.399 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.399 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:39:09.398041+0000 2024-09-15T01:39:09.399 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:39:09.398041+0000 2024-09-15T01:39:09.399 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:09.399 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.399 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:39:09.399 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:39:09.399 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:09.400 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:09.400 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:09.400 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:09.400 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:09.400 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:09.400 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:09.400 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:39:09.400 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:39:09.400 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.397+0000 7fb07a714640 1 Processor -- start 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.397+0000 7fb07a714640 1 -- start start 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.397+0000 7fb07a714640 10 monclient: init 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 10 monclient: _add_conns ranks=[2,9,0,5,4,7,1,8,10,6,3] 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb07409bf80 0x7fb07409ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 10 monclient(hunting): picked mon.noname-b con 0x7fb07409bf80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb074067130 0x7fb074067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 10 monclient(hunting): picked mon.noname-a con 0x7fb074067130 addr v2:172.21.15.178:3300/0 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb074067a40 addr v1:172.21.15.178:6789/0 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07404bf60 con 0x7fb074067a40 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07409e220 con 0x7fb074067130 2024-09-15T01:39:09.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07409b270 con 0x7fb07409bf80 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb07a714640 10 monclient(hunting): authenticate will time out at 1154.148500s 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb073fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb07409bf80 0x7fb07409ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb074067130 0x7fb074067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb073fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb07409bf80 0x7fb07409ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55394/0 (socket says 172.21.15.178:55394) 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb073fff640 1 -- 172.21.15.178:0/4102014308 learned_addr learned my addr 172.21.15.178:0/4102014308 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 10 monclient(hunting): get_auth_request con 0x7fb074067130 auth_method 0 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb078cb1640 1 --1- 172.21.15.178:0/4102014308 >> v1:172.21.15.178:6789/0 conn(0x7fb074067a40 0x7fb0741393c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:56678/0 (socket says 172.21.15.178:56678) 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb073fff640 10 monclient(hunting): get_auth_request con 0x7fb07409bf80 auth_method 0 2024-09-15T01:39:09.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb073fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb073fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb073fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb073fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb073fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb073fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 10 monclient(hunting): handle_auth_done global_id 34443 payload 274 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 1 -- 172.21.15.178:0/4102014308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb07409bf80 msgr2=0x7fb07409ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 1 --2- 172.21.15.178:0/4102014308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb07409bf80 0x7fb07409ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 1 -- 172.21.15.178:0/4102014308 >> v1:172.21.15.178:6789/0 conn(0x7fb074067a40 legacy=0x7fb0741393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 1 monclient: found mon.noname-a 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb0737fe640 1 -- 172.21.15.178:0/4102014308 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07409cf80 con 0x7fb074067130 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb072ffd640 1 -- 172.21.15.178:0/4102014308 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125136635 0 0) 0x7fb07404bf60 con 0x7fb074067a40 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb072ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.398+0000 7fb073fff640 1 --2- 172.21.15.178:0/4102014308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb07409bf80 0x7fb07409ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb0737fe640 1 --2- 172.21.15.178:0/4102014308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb074067130 0x7fb074067500 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb068009850 tx=0x7fb0680090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f537319c205f54d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb072ffd640 1 -- 172.21.15.178:0/4102014308 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb06802ac80 con 0x7fb074067130 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb072ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb072ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb072ffd640 10 monclient: dump: 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.413 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.414 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb072ffd640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb072ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb072ffd640 1 -- 172.21.15.178:0/4102014308 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06802ae00 con 0x7fb074067130 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb07a714640 5 monclient: authenticate success, global_id 34443 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb07a714640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb072ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb07a714640 10 monclient: get_monmap_and_config success 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb072ffd640 1 -- 172.21.15.178:0/4102014308 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb068031ca0 con 0x7fb074067130 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb072ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb07a714640 4 set_mon_vals no callback set 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb072ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.399+0000 7fb072ffd640 10 monclient: dump: 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.415 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.416 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.400+0000 7fb07a714640 10 monclient: shutdown 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.400+0000 7fb07a714640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.400+0000 7fb07a714640 1 -- 172.21.15.178:0/4102014308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb074067130 msgr2=0x7fb074067500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.400+0000 7fb07a714640 1 --2- 172.21.15.178:0/4102014308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb074067130 0x7fb074067500 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb068009850 tx=0x7fb0680090f0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.400+0000 7fb07a714640 1 -- 172.21.15.178:0/4102014308 shutdown_connections 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.400+0000 7fb07a714640 1 --2- 172.21.15.178:0/4102014308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb074067130 0x7fb074067500 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.400+0000 7fb07a714640 1 --2- 172.21.15.178:0/4102014308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb07409bf80 0x7fb07409ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.400+0000 7fb07a714640 1 -- 172.21.15.178:0/4102014308 >> 172.21.15.178:0/4102014308 conn(0x7fb074068a60 msgr2=0x7fb0740a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.400+0000 7fb07a714640 1 -- 172.21.15.178:0/4102014308 shutdown_connections 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.400+0000 7fb07a714640 1 -- 172.21.15.178:0/4102014308 wait complete. 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.400+0000 7fb07a714640 10 monclient: build_initial_monmap 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.400+0000 7fb07a714640 10 monclient: monmap: 2024-09-15T01:39:09.417 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:09.418 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 1 Processor -- start 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 1 -- start start 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 10 monclient: init 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 10 monclient: _add_conns ranks=[0,3,2,7,5,6,8,1,4] 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb074067130 0x7fb074141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 10 monclient(hunting): picked mon.noname-a con 0x7fb074067130 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb074067a40 0x7fb074134600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 10 monclient(hunting): picked mon.noname-g con 0x7fb074067a40 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb07409bf80 0x7fb07411e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 10 monclient(hunting): picked mon.noname-b con 0x7fb07409bf80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb074121ce0 con 0x7fb074067130 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07409f340 con 0x7fb07409bf80 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb074121a00 con 0x7fb074067a40 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb07a714640 10 monclient(hunting): authenticate will time out at 1154.151866s 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb073fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb074067130 0x7fb074141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb078cb1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb07409bf80 0x7fb07411e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb073fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb074067130 0x7fb074141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43414/0 (socket says 172.21.15.178:43414) 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb073fff640 1 -- 172.21.15.178:0/1718274014 learned_addr learned my addr 172.21.15.178:0/1718274014 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb073fff640 10 monclient(hunting): get_auth_request con 0x7fb074067130 auth_method 0 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb073fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb073fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb073fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb078cb1640 10 monclient(hunting): get_auth_request con 0x7fb07409bf80 auth_method 0 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb078cb1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb078cb1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.401+0000 7fb078cb1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb078cb1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb078cb1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb078cb1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb0737fe640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb074067a40 0x7fb074134600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb0737fe640 10 monclient(hunting): get_auth_request con 0x7fb074067a40 auth_method 0 2024-09-15T01:39:09.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb0737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb0737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb0737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 10 monclient(hunting): handle_auth_done global_id 34461 payload 824 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 1 -- 172.21.15.178:0/1718274014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb074067a40 msgr2=0x7fb074134600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb074067a40 0x7fb074134600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 1 -- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb07409bf80 msgr2=0x7fb07411e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb07409bf80 0x7fb07411e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 1 monclient: found mon.noname-a 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 1 -- 172.21.15.178:0/1718274014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb064009450 con 0x7fb074067130 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb078cb1640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb07409bf80 0x7fb07411e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:39:09.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb073fff640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb074067130 0x7fb074141930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb06400ea60 tx=0x7fb0640091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb06400ce90 con 0x7fb074067130 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb05bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb05bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb05bfff640 10 monclient: dump: 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.423 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb05bfff640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb05bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb07a714640 5 monclient: authenticate success, global_id 34461 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb064013470 con 0x7fb074067130 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb07a714640 10 monclient: _renew_subs 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb07a714640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb07a714640 1 -- 172.21.15.178:0/1718274014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb068009390 con 0x7fb074067130 2024-09-15T01:39:09.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb05bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb064013a60 con 0x7fb074067130 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb05bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb05bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb05bfff640 10 monclient: dump: 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.425 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb0727fc640 4 set_mon_vals no callback set 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb07a714640 10 monclient: _renew_subs 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb07a714640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.402+0000 7fb07a714640 1 -- 172.21.15.178:0/1718274014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb074120830 con 0x7fb074067130 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb07a714640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb07a714640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074141e70 0x7fb07411e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb07a714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb07a714640 1 -- 172.21.15.178:0/1718274014 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb0741218c0 con 0x7fb074141e70 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb064013bc0 con 0x7fb074067130 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb0737fe640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074141e70 0x7fb07411e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb0737fe640 10 monclient: get_auth_request con 0x7fb074141e70 auth_method 0 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb0737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb0737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb0737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb0737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb0737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb0737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb05bfff640 1 --2- 172.21.15.178:0/1718274014 >> v2:172.21.15.204:6812/28210 conn(0x7fb0600397b0 0x7fb06003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb0737fe640 10 monclient(hunting): handle_auth_done global_id 34325 payload 824 2024-09-15T01:39:09.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb0737fe640 1 -- 172.21.15.178:0/1718274014 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb07404c4a0 con 0x7fb074141e70 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fb06404bfe0 con 0x7fb074067130 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.403+0000 7fb0737fe640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074141e70 0x7fb07411e650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb07409f2d0 tx=0x7fb068032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb078cb1640 1 --2- 172.21.15.178:0/1718274014 >> v2:172.21.15.204:6812/28210 conn(0x7fb0600397b0 0x7fb06003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb078cb1640 10 monclient: get_auth_request con 0x7fb0600397b0 auth_method 0 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb06802ac80 con 0x7fb074141e70 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb05bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb05bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb05bfff640 10 monclient: dump: 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.428 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fb06803c620 con 0x7fb074141e70 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb05bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb05bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:39:09.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074141e70 msgr2=0x7fb07411e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb05bfff640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074141e70 0x7fb07411e650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb07409f2d0 tx=0x7fb068032000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074141e70 msgr2=0x7fb07411e650 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:09.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb05bfff640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074141e70 0x7fb07411e650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.404+0000 7fb078cb1640 1 --2- 172.21.15.178:0/1718274014 >> v2:172.21.15.204:6812/28210 conn(0x7fb0600397b0 0x7fb06003bc60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb06c004fd0 tx=0x7fb06c007cb0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.409+0000 7fb07a714640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:39:09.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb07a714640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074134b40 0x7fb07411eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb07a714640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb07a714640 1 -- 172.21.15.178:0/1718274014 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb0740641f0 con 0x7fb074134b40 2024-09-15T01:39:09.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb0737fe640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074134b40 0x7fb07411eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb0737fe640 10 monclient: get_auth_request con 0x7fb074134b40 auth_method 0 2024-09-15T01:39:09.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb0737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb0737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb0737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb0737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb0737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb0737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb0737fe640 10 monclient(hunting): handle_auth_done global_id 34334 payload 824 2024-09-15T01:39:09.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb0737fe640 1 -- 172.21.15.178:0/1718274014 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb07404bc80 con 0x7fb074134b40 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb0737fe640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074134b40 0x7fb07411eb90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb068000c00 tx=0x7fb06803c000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb068049070 con 0x7fb074134b40 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb05bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb05bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb05bfff640 10 monclient: dump: 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.433 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9119 (secure 0 0 0) 0x7fb06803b370 con 0x7fb074134b40 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb05bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb05bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074134b40 msgr2=0x7fb07411eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb05bfff640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074134b40 0x7fb07411eb90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb068000c00 tx=0x7fb06803c000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb05bfff640 1 -- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074134b40 msgr2=0x7fb07411eb90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:09.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.410+0000 7fb05bfff640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb074134b40 0x7fb07411eb90 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb068000c00 tx=0x7fb06803c000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.435 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:39:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:39:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:39:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:39:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 94, 2024-09-15T01:39:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:39:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:39:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:39:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:39:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:39:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:39:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:39:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:39:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:39:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:39:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:39:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:39:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:39:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:39:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:39:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:39:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.457 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:39:09.457 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:39:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.411+0000 7fb07a714640 1 -- 172.21.15.178:0/1718274014 >> v2:172.21.15.204:6812/28210 conn(0x7fb0600397b0 msgr2=0x7fb06003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.411+0000 7fb07a714640 1 --2- 172.21.15.178:0/1718274014 >> v2:172.21.15.204:6812/28210 conn(0x7fb0600397b0 0x7fb06003bc60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb06c004fd0 tx=0x7fb06c007cb0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.411+0000 7fb07a714640 10 monclient: shutdown 2024-09-15T01:39:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.411+0000 7fb07a714640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.411+0000 7fb07a714640 1 -- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb074067130 msgr2=0x7fb074141930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.411+0000 7fb07a714640 1 --2- 172.21.15.178:0/1718274014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb074067130 0x7fb074141930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb06400ea60 tx=0x7fb0640091d0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.411+0000 7fb078cb1640 1 -- 172.21.15.178:0/1718274014 reap_dead start 2024-09-15T01:39:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.411+0000 7fb078cb1640 1 -- 172.21.15.178:0/1718274014 reap_dead start 2024-09-15T01:39:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.411+0000 7fb07a714640 1 -- 172.21.15.178:0/1718274014 shutdown_connections 2024-09-15T01:39:09.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.411+0000 7fb07a714640 1 -- 172.21.15.178:0/1718274014 >> 172.21.15.178:0/1718274014 conn(0x7fb074068a60 msgr2=0x7fb074135490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:09.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.411+0000 7fb07a714640 1 -- 172.21.15.178:0/1718274014 shutdown_connections 2024-09-15T01:39:09.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.411+0000 7fb07a714640 1 -- 172.21.15.178:0/1718274014 wait complete. 2024-09-15T01:39:09.459 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:39:09.459 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-09-15T01:39:09.459 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:39:09.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.552+0000 7f507159d640 10 monclient: get_monmap_and_config 2024-09-15T01:39:09.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.552+0000 7f507159d640 10 monclient: build_initial_monmap 2024-09-15T01:39:09.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.552+0000 7f507159d640 10 monclient: monmap: 2024-09-15T01:39:09.554 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:09.554 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.554 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:39:09.553128+0000 2024-09-15T01:39:09.554 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:39:09.553128+0000 2024-09-15T01:39:09.554 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:09.554 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.554 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:39:09.555 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:39:09.555 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:09.555 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:09.555 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:09.555 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:09.555 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:09.555 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:09.555 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:09.555 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:39:09.555 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:39:09.555 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.552+0000 7f507159d640 1 Processor -- start 2024-09-15T01:39:09.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.552+0000 7f507159d640 1 -- start start 2024-09-15T01:39:09.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.552+0000 7f507159d640 10 monclient: init 2024-09-15T01:39:09.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 10 monclient: _add_conns ranks=[3,8,5,1,7,6,10,4,9,0,2] 2024-09-15T01:39:09.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f506c0678d0 0x7f506c09e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 10 monclient(hunting): picked mon.noname-g con 0x7f506c0678d0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:39:09.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c06ea30 0x7f506c09ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 10 monclient(hunting): picked mon.noname-e con 0x7f506c06ea30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f506c09f180 0x7f506c1306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 10 monclient(hunting): picked mon.noname-h con 0x7f506c09f180 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c09f570 con 0x7f506c0678d0 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f506c06d5b0 con 0x7f506c09f180 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f506c067ca0 con 0x7f506c06ea30 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f507159d640 10 monclient(hunting): authenticate will time out at 1154.303378s 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c06ea30 0x7f506c09ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c06ea30 0x7f506c09ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43766/0 (socket says 172.21.15.178:43766) 2024-09-15T01:39:09.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 1 -- 172.21.15.178:0/1864834093 learned_addr learned my addr 172.21.15.178:0/1864834093 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:09.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 10 monclient(hunting): get_auth_request con 0x7f506c06ea30 auth_method 0 2024-09-15T01:39:09.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 10 monclient(hunting): handle_auth_done global_id 34343 payload 274 2024-09-15T01:39:09.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 1 -- 172.21.15.178:0/1864834093 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f506c09f180 msgr2=0x7f506c1306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:39:09.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506affd640 1 --2- 172.21.15.178:0/1864834093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f506c0678d0 0x7f506c09e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506b7fe640 1 --2- 172.21.15.178:0/1864834093 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f506c09f180 0x7f506c1306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 1 --2- 172.21.15.178:0/1864834093 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f506c09f180 0x7f506c1306b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 1 -- 172.21.15.178:0/1864834093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f506c0678d0 msgr2=0x7f506c09e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:39:09.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 1 --2- 172.21.15.178:0/1864834093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f506c0678d0 0x7f506c09e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 1 monclient: found mon.noname-e 2024-09-15T01:39:09.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:39:09.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506a7fc640 1 -- 172.21.15.178:0/1864834093 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c04bf60 con 0x7f506c06ea30 2024-09-15T01:39:09.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506affd640 10 monclient: get_auth_request con 0x7f506c0678d0 auth_method 0 2024-09-15T01:39:09.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.553+0000 7f506affd640 1 --2- 172.21.15.178:0/1864834093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f506c0678d0 0x7f506c09e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:39:09.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f506a7fc640 1 --2- 172.21.15.178:0/1864834093 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c06ea30 0x7f506c09ec40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5054009940 tx=0x7f50540091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6f7c14df388de00f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f5069ffb640 1 -- 172.21.15.178:0/1864834093 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f505402ad00 con 0x7f506c06ea30 2024-09-15T01:39:09.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f5069ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f5069ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:39:09.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f5069ffb640 10 monclient: dump: 2024-09-15T01:39:09.572 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.572 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.572 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.572 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.572 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.572 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.572 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.572 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.572 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.572 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f5069ffb640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f5069ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f507159d640 5 monclient: authenticate success, global_id 34343 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f5069ffb640 1 -- 172.21.15.178:0/1864834093 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505402ae80 con 0x7f506c06ea30 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f507159d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f5069ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:09.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f507159d640 10 monclient: get_monmap_and_config success 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f507159d640 4 set_mon_vals no callback set 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f5069ffb640 1 -- 172.21.15.178:0/1864834093 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5054031ca0 con 0x7f506c06ea30 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f5069ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f5069ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.554+0000 7f5069ffb640 10 monclient: dump: 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.574 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.555+0000 7f507159d640 10 monclient: shutdown 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.555+0000 7f507159d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.555+0000 7f507159d640 1 -- 172.21.15.178:0/1864834093 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c06ea30 msgr2=0x7f506c09ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.555+0000 7f507159d640 1 --2- 172.21.15.178:0/1864834093 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c06ea30 0x7f506c09ec40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5054009940 tx=0x7f50540091d0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.555+0000 7f507159d640 1 -- 172.21.15.178:0/1864834093 shutdown_connections 2024-09-15T01:39:09.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.555+0000 7f507159d640 1 --2- 172.21.15.178:0/1864834093 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f506c09f180 0x7f506c1306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.555+0000 7f507159d640 1 --2- 172.21.15.178:0/1864834093 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c06ea30 0x7f506c09ec40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.555+0000 7f507159d640 1 --2- 172.21.15.178:0/1864834093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f506c0678d0 0x7f506c09e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.555+0000 7f507159d640 1 -- 172.21.15.178:0/1864834093 >> 172.21.15.178:0/1864834093 conn(0x7f506c0a4390 msgr2=0x7f506c067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.555+0000 7f507159d640 1 -- 172.21.15.178:0/1864834093 shutdown_connections 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.555+0000 7f507159d640 1 -- 172.21.15.178:0/1864834093 wait complete. 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.556+0000 7f507159d640 10 monclient: build_initial_monmap 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.556+0000 7f507159d640 10 monclient: monmap: 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:09.576 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:39:09.577 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:39:09.577 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:39:09.577 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:09.577 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:09.577 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:09.578 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:09.578 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:09.578 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:09.578 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:09.578 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.556+0000 7f507159d640 1 Processor -- start 2024-09-15T01:39:09.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.556+0000 7f507159d640 1 -- start start 2024-09-15T01:39:09.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.556+0000 7f507159d640 10 monclient: init 2024-09-15T01:39:09.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient: _add_conns ranks=[8,2,6,3,1,4,5,0,7] 2024-09-15T01:39:09.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c0678d0 0x7f506c1335d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient(hunting): picked mon.noname-e con 0x7f506c0678d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:39:09.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f506c06ea30 0x7f506c11d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient(hunting): picked mon.noname-b con 0x7f506c06ea30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:39:09.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f506c09f180 0x7f506c11d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient(hunting): picked mon.noname-d con 0x7f506c09f180 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:39:09.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c11f140 con 0x7f506c06ea30 2024-09-15T01:39:09.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f506c1206f0 con 0x7f506c09f180 2024-09-15T01:39:09.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f506c140610 con 0x7f506c0678d0 2024-09-15T01:39:09.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient(hunting): authenticate will time out at 1154.307127s 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c0678d0 0x7f506c1335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506a7fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f506c06ea30 0x7f506c11d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c0678d0 0x7f506c1335d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:43774/0 (socket says 172.21.15.178:43774) 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 1 -- 172.21.15.178:0/2941814269 learned_addr learned my addr 172.21.15.178:0/2941814269 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 10 monclient(hunting): get_auth_request con 0x7f506c0678d0 auth_method 0 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506a7fc640 10 monclient(hunting): get_auth_request con 0x7f506c06ea30 auth_method 0 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506b7fe640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f506c09f180 0x7f506c11d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506b7fe640 10 monclient(hunting): get_auth_request con 0x7f506c09f180 auth_method 0 2024-09-15T01:39:09.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 10 monclient(hunting): handle_auth_done global_id 34352 payload 824 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 1 -- 172.21.15.178:0/2941814269 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f506c09f180 msgr2=0x7f506c11d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f506c09f180 0x7f506c11d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 1 -- 172.21.15.178:0/2941814269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f506c06ea30 msgr2=0x7f506c11d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f506c06ea30 0x7f506c11d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 1 monclient: found mon.noname-e 2024-09-15T01:39:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:39:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 1 -- 172.21.15.178:0/2941814269 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5054009450 con 0x7f506c0678d0 2024-09-15T01:39:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506a7fc640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f506c06ea30 0x7f506c11d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:39:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506b7fe640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f506c09f180 0x7f506c11d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:39:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f506affd640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c0678d0 0x7f506c1335d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f506400a400 tx=0x7f5064007910 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f506400dad0 con 0x7f506c0678d0 2024-09-15T01:39:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f5062ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f5062ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:39:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f5062ffd640 10 monclient: dump: 2024-09-15T01:39:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.586 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f5062ffd640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f5062ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506400dc70 con 0x7f506c0678d0 2024-09-15T01:39:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 5 monclient: authenticate success, global_id 34352 2024-09-15T01:39:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f5062ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient: _renew_subs 2024-09-15T01:39:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f506400de20 con 0x7f506c0678d0 2024-09-15T01:39:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:39:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 1 -- 172.21.15.178:0/2941814269 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506c06ccd0 con 0x7f506c0678d0 2024-09-15T01:39:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f5062ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f5062ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:39:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f5062ffd640 10 monclient: dump: 2024-09-15T01:39:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.589 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f50697fa640 4 set_mon_vals no callback set 2024-09-15T01:39:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient: _renew_subs 2024-09-15T01:39:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:39:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.557+0000 7f507159d640 1 -- 172.21.15.178:0/2941814269 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506c11f790 con 0x7f506c0678d0 2024-09-15T01:39:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.558+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f506400d650 con 0x7f506c0678d0 2024-09-15T01:39:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.558+0000 7f5062ffd640 1 --2- 172.21.15.178:0/2941814269 >> v2:172.21.15.204:6812/28210 conn(0x7f504c0397e0 0x7f504c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.558+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f506404c270 con 0x7f506c0678d0 2024-09-15T01:39:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.558+0000 7f507159d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.558+0000 7f507159d640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c133b10 0x7f506c11db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.558+0000 7f507159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.558+0000 7f507159d640 1 -- 172.21.15.178:0/2941814269 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c133ee0 con 0x7f506c133b10 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506b7fe640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c133b10 0x7f506c11db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506a7fc640 1 --2- 172.21.15.178:0/2941814269 >> v2:172.21.15.204:6812/28210 conn(0x7f504c0397e0 0x7f504c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506a7fc640 10 monclient: get_auth_request con 0x7f504c0397e0 auth_method 0 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506b7fe640 10 monclient: get_auth_request con 0x7f506c133b10 auth_method 0 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506b7fe640 10 monclient(hunting): handle_auth_done global_id 7876 payload 824 2024-09-15T01:39:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506b7fe640 1 -- 172.21.15.178:0/2941814269 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f506c04bc80 con 0x7f506c133b10 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506a7fc640 1 --2- 172.21.15.178:0/2941814269 >> v2:172.21.15.204:6812/28210 conn(0x7f504c0397e0 0x7f504c03bc90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5054034000 tx=0x7f5054032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f506b7fe640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c133b10 0x7f506c11db00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f506c11cfd0 tx=0x7f505c008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f505c00dc30 con 0x7f506c133b10 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f5062ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f5062ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.559+0000 7f5062ffd640 10 monclient: dump: 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.591 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.592 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.592 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.592 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.592 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.592 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.592 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.592 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.592 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.592 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.560+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f505c00c040 con 0x7f506c133b10 2024-09-15T01:39:09.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.560+0000 7f5062ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:09.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.560+0000 7f5062ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:39:09.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.560+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c133b10 msgr2=0x7f506c11db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.560+0000 7f5062ffd640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c133b10 0x7f506c11db00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f506c11cfd0 tx=0x7f505c008040 comp rx=0 tx=0).stop 2024-09-15T01:39:09.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.560+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c133b10 msgr2=0x7f506c11db00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:09.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.560+0000 7f5062ffd640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c133b10 0x7f506c11db00 secure :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f506c11cfd0 tx=0x7f505c008040 comp rx=0 tx=0).stop 2024-09-15T01:39:09.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.565+0000 7f507159d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:39:09.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.565+0000 7f507159d640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c11e040 0x7f506c098a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.565+0000 7f507159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.565+0000 7f507159d640 1 -- 172.21.15.178:0/2941814269 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c05ed70 con 0x7f506c11e040 2024-09-15T01:39:09.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f506b7fe640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c11e040 0x7f506c098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f506b7fe640 10 monclient: get_auth_request con 0x7f506c11e040 auth_method 0 2024-09-15T01:39:09.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f506b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f506b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f506b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f506b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f506b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f506b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f506b7fe640 10 monclient(hunting): handle_auth_done global_id 7885 payload 824 2024-09-15T01:39:09.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f506b7fe640 1 -- 172.21.15.178:0/2941814269 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f506c04b720 con 0x7f506c11e040 2024-09-15T01:39:09.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f506b7fe640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c11e040 0x7f506c098a40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f505c01a000 tx=0x7f505c01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f505c01be90 con 0x7f506c11e040 2024-09-15T01:39:09.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f5062ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f5062ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:39:09.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.566+0000 7f5062ffd640 10 monclient: dump: 2024-09-15T01:39:09.597 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.597 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.597 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.597 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.597 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.597 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.597 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.598 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.598 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.598 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.598 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.598 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.598 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.598 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.598 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.598 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9291 (secure 0 0 0) 0x7f505c00d5e0 con 0x7f506c11e040 2024-09-15T01:39:09.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f5062ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:39:09.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f5062ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:39:09.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c11e040 msgr2=0x7f506c098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f5062ffd640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c11e040 0x7f506c098a40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f505c01a000 tx=0x7f505c01d000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f5062ffd640 1 -- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c11e040 msgr2=0x7f506c098a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:09.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f5062ffd640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f506c11e040 0x7f506c098a40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.599 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:39:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:39:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:39:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:39:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 94, 2024-09-15T01:39:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:39:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:39:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:39:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:39:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:39:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:39:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:39:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:39:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:39:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:39:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:39:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:39:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:39:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:39:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:39:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:39:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:39:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:39:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:39:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:39:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:39:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:39:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:39:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:39:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:39:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:39:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:39:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:39:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:39:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:39:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:39:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:39:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:39:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:39:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:39:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:39:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:39:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:39:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:39:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:39:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:39:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:39:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:39:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:39:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:39:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:39:09.623 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:39:09.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f507159d640 1 -- 172.21.15.178:0/2941814269 >> v2:172.21.15.204:6812/28210 conn(0x7f504c0397e0 msgr2=0x7f504c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f507159d640 1 --2- 172.21.15.178:0/2941814269 >> v2:172.21.15.204:6812/28210 conn(0x7f504c0397e0 0x7f504c03bc90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5054034000 tx=0x7f5054032000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f507159d640 10 monclient: shutdown 2024-09-15T01:39:09.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f507159d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:09.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f507159d640 1 -- 172.21.15.178:0/2941814269 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c0678d0 msgr2=0x7f506c1335d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f507159d640 1 --2- 172.21.15.178:0/2941814269 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f506c0678d0 0x7f506c1335d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f506400a400 tx=0x7f5064007910 comp rx=0 tx=0).stop 2024-09-15T01:39:09.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f506b7fe640 1 -- 172.21.15.178:0/2941814269 reap_dead start 2024-09-15T01:39:09.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f506b7fe640 1 -- 172.21.15.178:0/2941814269 reap_dead start 2024-09-15T01:39:09.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f507159d640 1 -- 172.21.15.178:0/2941814269 shutdown_connections 2024-09-15T01:39:09.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f507159d640 1 -- 172.21.15.178:0/2941814269 >> 172.21.15.178:0/2941814269 conn(0x7f506c0a4390 msgr2=0x7f506c0a23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:09.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f507159d640 1 -- 172.21.15.178:0/2941814269 shutdown_connections 2024-09-15T01:39:09.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.567+0000 7f507159d640 1 -- 172.21.15.178:0/2941814269 wait complete. 2024-09-15T01:39:09.626 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:39:09.626 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-09-15T01:39:09.627 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:39:09.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.718+0000 7f850315d640 10 monclient: get_monmap_and_config 2024-09-15T01:39:09.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.718+0000 7f850315d640 10 monclient: build_initial_monmap 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.718+0000 7f850315d640 10 monclient: monmap: 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:39:09.719785+0000 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:39:09.719785+0000 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:09.721 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:09.722 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:09.722 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:09.722 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:09.722 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:39:09.722 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:39:09.722 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.719+0000 7f850315d640 1 Processor -- start 2024-09-15T01:39:09.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.719+0000 7f850315d640 1 -- start start 2024-09-15T01:39:09.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.719+0000 7f850315d640 10 monclient: init 2024-09-15T01:39:09.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 10 monclient: _add_conns ranks=[2,4,9,8,10,1,3,6,0,5,7] 2024-09-15T01:39:09.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84fc0a2930 0x7f84fc132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 10 monclient(hunting): picked mon.noname-b con 0x7f84fc0a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:39:09.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84fc09faa0 0x7f84fc09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 10 monclient(hunting): picked mon.noname-c con 0x7f84fc09faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:39:09.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f84fc0a08c0 0x7f84fc0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 10 monclient(hunting): picked mon.noname-a con 0x7f84fc0a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc09e0f0 con 0x7f84fc0a08c0 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc09dd80 con 0x7f84fc0a2930 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc0a2d00 con 0x7f84fc09faa0 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f850315d640 10 monclient(hunting): authenticate will time out at 1154.470265s 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f84fbfff640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84fc09faa0 msgr2=0x7f84fc09fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f84fbfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84fc09faa0 0x7f84fc09fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f84fc0a08c0 0x7f84fc0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f84fc0a08c0 0x7f84fc0a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43418/0 (socket says 172.21.15.178:43418) 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 1 -- 172.21.15.178:0/1993408159 learned_addr learned my addr 172.21.15.178:0/1993408159 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 10 monclient(hunting): get_auth_request con 0x7f84fc0a08c0 auth_method 0 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f8500ef9640 1 --2- 172.21.15.178:0/1993408159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84fc0a2930 0x7f84fc132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f8500ef9640 10 monclient(hunting): get_auth_request con 0x7f84fc0a2930 auth_method 0 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f8500ef9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f8500ef9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f8500ef9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f8500ef9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f8500ef9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f8500ef9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 10 monclient(hunting): handle_auth_done global_id 34470 payload 274 2024-09-15T01:39:09.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 1 -- 172.21.15.178:0/1993408159 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84fc09faa0 msgr2=0x7f84fc09fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:39:09.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 1 --2- 172.21.15.178:0/1993408159 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84fc09faa0 0x7f84fc09fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 1 -- 172.21.15.178:0/1993408159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84fc0a2930 msgr2=0x7f84fc132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:39:09.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 1 --2- 172.21.15.178:0/1993408159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84fc0a2930 0x7f84fc132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 1 monclient: found mon.noname-a 2024-09-15T01:39:09.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f85016fa640 1 -- 172.21.15.178:0/1993408159 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc04bf60 con 0x7f84fc0a08c0 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.720+0000 7f8500ef9640 1 --2- 172.21.15.178:0/1993408159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84fc0a2930 0x7f84fc132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f85016fa640 1 --2- 172.21.15.178:0/1993408159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84fc0a08c0 0x7f84fc0a0d30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009f20 tx=0x7f84f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a160106e4e2fbc84 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f84fb7fe640 1 -- 172.21.15.178:0/1993408159 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84f4007880 con 0x7f84fc0a08c0 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f84fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f84fb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f84fb7fe640 10 monclient: dump: 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.735 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.736 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.736 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.736 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.736 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.736 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.736 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.736 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.736 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.736 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f84fb7fe640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f84fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f84fb7fe640 1 -- 172.21.15.178:0/1993408159 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f402d050 con 0x7f84fc0a08c0 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f850315d640 5 monclient: authenticate success, global_id 34470 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f850315d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f84fb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f850315d640 10 monclient: get_monmap_and_config success 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f84fb7fe640 1 -- 172.21.15.178:0/1993408159 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84f4030360 con 0x7f84fc0a08c0 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f850315d640 4 set_mon_vals no callback set 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f84fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f84fb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.721+0000 7f84fb7fe640 10 monclient: dump: 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.737 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 10 monclient: shutdown 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:09.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 1 -- 172.21.15.178:0/1993408159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84fc0a08c0 msgr2=0x7f84fc0a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 1 --2- 172.21.15.178:0/1993408159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84fc0a08c0 0x7f84fc0a0d30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009f20 tx=0x7f84f4009840 comp rx=0 tx=0).stop 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 1 -- 172.21.15.178:0/1993408159 shutdown_connections 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 1 --2- 172.21.15.178:0/1993408159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84fc0a08c0 0x7f84fc0a0d30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 1 --2- 172.21.15.178:0/1993408159 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84fc09faa0 0x7f84fc09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 1 --2- 172.21.15.178:0/1993408159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84fc0a2930 0x7f84fc132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 1 -- 172.21.15.178:0/1993408159 >> 172.21.15.178:0/1993408159 conn(0x7f84fc0a5ed0 msgr2=0x7f84fc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 1 -- 172.21.15.178:0/1993408159 shutdown_connections 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 1 -- 172.21.15.178:0/1993408159 wait complete. 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 10 monclient: build_initial_monmap 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 10 monclient: monmap: 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:39:09.739 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:09.740 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.722+0000 7f850315d640 1 Processor -- start 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 1 -- start start 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 10 monclient: init 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 10 monclient: _add_conns ranks=[4,5,0,8,6,3,7,2,1] 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84fc09faa0 0x7f84fc09ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 10 monclient(hunting): picked mon.noname-c con 0x7f84fc09faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f84fc0a08c0 0x7f84fc131750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 10 monclient(hunting): picked mon.noname-h con 0x7f84fc0a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84fc0a2930 0x7f84fc11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 10 monclient(hunting): picked mon.noname-a con 0x7f84fc0a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc11d740 con 0x7f84fc0a2930 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc09d630 con 0x7f84fc09faa0 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc142a20 con 0x7f84fc0a08c0 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f850315d640 10 monclient(hunting): authenticate will time out at 1154.473587s 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f8500ef9640 1 -- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84fc09faa0 msgr2=0x7f84fc09ae90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3302/0 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f8500ef9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84fc09faa0 0x7f84fc09ae90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84fc0a2930 0x7f84fc11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84fc0a2930 0x7f84fc11c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43420/0 (socket says 172.21.15.178:43420) 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 1 -- 172.21.15.178:0/3583996694 learned_addr learned my addr 172.21.15.178:0/3583996694 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 10 monclient(hunting): get_auth_request con 0x7f84fc0a2930 auth_method 0 2024-09-15T01:39:09.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 10 monclient(hunting): handle_auth_done global_id 34479 payload 824 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 1 -- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f84fc0a08c0 msgr2=0x7f84fc131750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f84fc0a08c0 0x7f84fc131750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 1 -- 172.21.15.178:0/3583996694 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84fc09faa0 msgr2=0x7f84fc09ae90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84fc09faa0 0x7f84fc09ae90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 1 monclient: found mon.noname-a 2024-09-15T01:39:09.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.723+0000 7f85016fa640 1 -- 172.21.15.178:0/3583996694 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec009390 con 0x7f84fc0a2930 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f85016fa640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84fc0a2930 0x7f84fc11c530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009df0 tx=0x7f84f4036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84f4007170 con 0x7f84fc0a2930 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84f8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84f8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84f8ff9640 10 monclient: dump: 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84f8ff9640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84f8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f403baa0 con 0x7f84fc0a2930 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f850315d640 5 monclient: authenticate success, global_id 34479 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f850315d640 10 monclient: _renew_subs 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f850315d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f850315d640 1 -- 172.21.15.178:0/3583996694 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f4009ae0 con 0x7f84fc0a2930 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84f8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84f403bc50 con 0x7f84fc0a2930 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84f8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84f8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84f8ff9640 10 monclient: dump: 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f850315d640 10 monclient: _renew_subs 2024-09-15T01:39:09.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f850315d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:39:09.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f84faffd640 4 set_mon_vals no callback set 2024-09-15T01:39:09.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.724+0000 7f850315d640 1 -- 172.21.15.178:0/3583996694 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc11d910 con 0x7f84fc0a2930 2024-09-15T01:39:09.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f84f4038070 con 0x7f84fc0a2930 2024-09-15T01:39:09.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f850315d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:09.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f850315d640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc131c90 0x7f84fc11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f850315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f850315d640 1 -- 172.21.15.178:0/3583996694 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc132060 con 0x7f84fc131c90 2024-09-15T01:39:09.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f84f8ff9640 1 --2- 172.21.15.178:0/3583996694 >> v2:172.21.15.204:6812/28210 conn(0x7f84e80397b0 0x7f84e803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f84f406bf20 con 0x7f84fc0a2930 2024-09-15T01:39:09.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f8500ef9640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc131c90 0x7f84fc11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f8500ef9640 10 monclient: get_auth_request con 0x7f84fc131c90 auth_method 0 2024-09-15T01:39:09.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f8500ef9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f8500ef9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f8500ef9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f84fbfff640 1 --2- 172.21.15.178:0/3583996694 >> v2:172.21.15.204:6812/28210 conn(0x7f84e80397b0 0x7f84e803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f84fbfff640 10 monclient: get_auth_request con 0x7f84e80397b0 auth_method 0 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f8500ef9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f8500ef9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f8500ef9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f8500ef9640 10 monclient(hunting): handle_auth_done global_id 16014 payload 824 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.725+0000 7f8500ef9640 1 -- 172.21.15.178:0/3583996694 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f84fc04c4a0 con 0x7f84fc131c90 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f8500ef9640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc131c90 0x7f84fc11ca70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0095e0 tx=0x7f84ec0090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f84fbfff640 1 --2- 172.21.15.178:0/3583996694 >> v2:172.21.15.204:6812/28210 conn(0x7f84e80397b0 0x7f84e803bc60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0a6ec0 tx=0x7f84f0006c80 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84ec013470 con 0x7f84fc131c90 2024-09-15T01:39:09.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f84f8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f84f8ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f84f8ff9640 10 monclient: dump: 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.750 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.751 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.751 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.751 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.751 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.751 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f84ec00ca90 con 0x7f84fc131c90 2024-09-15T01:39:09.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f84f8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:09.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f84f8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:39:09.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc131c90 msgr2=0x7f84fc11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f84f8ff9640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc131c90 0x7f84fc11ca70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0095e0 tx=0x7f84ec0090f0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc131c90 msgr2=0x7f84fc11ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:09.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.726+0000 7f84f8ff9640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc131c90 0x7f84fc11ca70 secure :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0095e0 tx=0x7f84ec0090f0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f850315d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:39:09.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f850315d640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc11cfb0 0x7f84fc096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f850315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f850315d640 1 -- 172.21.15.178:0/3583996694 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc070cb0 con 0x7f84fc11cfb0 2024-09-15T01:39:09.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f8500ef9640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc11cfb0 0x7f84fc096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f8500ef9640 10 monclient: get_auth_request con 0x7f84fc11cfb0 auth_method 0 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f8500ef9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f8500ef9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f8500ef9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f8500ef9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f8500ef9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f8500ef9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f8500ef9640 10 monclient(hunting): handle_auth_done global_id 16023 payload 824 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f8500ef9640 1 -- 172.21.15.178:0/3583996694 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f84fc04bc80 con 0x7f84fc11cfb0 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.732+0000 7f8500ef9640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc11cfb0 0x7f84fc096a30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f84ec000c00 tx=0x7f84ec016000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84ec005c50 con 0x7f84fc11cfb0 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f84f8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f84f8ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f84f8ff9640 10 monclient: dump: 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.755 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.756 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.756 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.756 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9289 (secure 0 0 0) 0x7f84ec00c610 con 0x7f84fc11cfb0 2024-09-15T01:39:09.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f84f8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:39:09.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f84f8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:39:09.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc11cfb0 msgr2=0x7f84fc096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f84f8ff9640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc11cfb0 0x7f84fc096a30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f84ec000c00 tx=0x7f84ec016000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f84f8ff9640 1 -- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc11cfb0 msgr2=0x7f84fc096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:09.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f84f8ff9640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84fc11cfb0 0x7f84fc096a30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.757 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:39:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:39:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:39:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:39:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 94, 2024-09-15T01:39:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:39:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:39:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:39:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:39:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:39:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:39:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:39:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:39:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:39:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:39:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:39:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:39:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:39:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:39:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:39:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:39:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:39:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:39:09.780 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:39:09.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f850315d640 1 -- 172.21.15.178:0/3583996694 >> v2:172.21.15.204:6812/28210 conn(0x7f84e80397b0 msgr2=0x7f84e803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f850315d640 1 --2- 172.21.15.178:0/3583996694 >> v2:172.21.15.204:6812/28210 conn(0x7f84e80397b0 0x7f84e803bc60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0a6ec0 tx=0x7f84f0006c80 comp rx=0 tx=0).stop 2024-09-15T01:39:09.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f850315d640 10 monclient: shutdown 2024-09-15T01:39:09.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f850315d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:09.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f850315d640 1 -- 172.21.15.178:0/3583996694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84fc0a2930 msgr2=0x7f84fc11c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f850315d640 1 --2- 172.21.15.178:0/3583996694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84fc0a2930 0x7f84fc11c530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009df0 tx=0x7f84f4036000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f85016fa640 1 -- 172.21.15.178:0/3583996694 reap_dead start 2024-09-15T01:39:09.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f85016fa640 1 -- 172.21.15.178:0/3583996694 reap_dead start 2024-09-15T01:39:09.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f850315d640 1 -- 172.21.15.178:0/3583996694 shutdown_connections 2024-09-15T01:39:09.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f850315d640 1 -- 172.21.15.178:0/3583996694 >> 172.21.15.178:0/3583996694 conn(0x7f84fc0a5ed0 msgr2=0x7f84fc09e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:09.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f850315d640 1 -- 172.21.15.178:0/3583996694 shutdown_connections 2024-09-15T01:39:09.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.733+0000 7f850315d640 1 -- 172.21.15.178:0/3583996694 wait complete. 2024-09-15T01:39:09.783 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:39:09.783 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-15T01:39:09.783 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:39:09.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.874+0000 7f504e434640 10 monclient: get_monmap_and_config 2024-09-15T01:39:09.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.874+0000 7f504e434640 10 monclient: build_initial_monmap 2024-09-15T01:39:09.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.874+0000 7f504e434640 10 monclient: monmap: 2024-09-15T01:39:09.876 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:39:09.875435+0000 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:39:09.875435+0000 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:09.877 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:09.878 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:09.878 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:39:09.878 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:39:09.878 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 1 Processor -- start 2024-09-15T01:39:09.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 1 -- start start 2024-09-15T01:39:09.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 10 monclient: init 2024-09-15T01:39:09.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 10 monclient: _add_conns ranks=[3,2,5,0,1,6,4,7,9,8,10] 2024-09-15T01:39:09.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50480a2930 0x7f5048132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 10 monclient(hunting): picked mon.noname-g con 0x7f50480a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:39:09.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f504809faa0 0x7f504809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 10 monclient(hunting): picked mon.noname-b con 0x7f504809faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f50480a08c0 0x7f50480a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 10 monclient(hunting): picked mon.noname-h con 0x7f50480a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504809e0f0 con 0x7f504809faa0 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504809dd80 con 0x7f50480a2930 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50480a2d00 con 0x7f50480a08c0 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f504e434640 10 monclient(hunting): authenticate will time out at 1154.625868s 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.875+0000 7f50477fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f504809faa0 0x7f504809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f504809faa0 0x7f504809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55444/0 (socket says 172.21.15.178:55444) 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 1 -- 172.21.15.178:0/798726416 learned_addr learned my addr 172.21.15.178:0/798726416 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:09.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 10 monclient(hunting): get_auth_request con 0x7f504809faa0 auth_method 0 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f5047fff640 1 --2- 172.21.15.178:0/798726416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50480a2930 0x7f5048132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f504c9d1640 1 --2- 172.21.15.178:0/798726416 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f50480a08c0 0x7f50480a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 10 monclient(hunting): handle_auth_done global_id 34355 payload 274 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 1 -- 172.21.15.178:0/798726416 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f50480a08c0 msgr2=0x7f50480a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 1 --2- 172.21.15.178:0/798726416 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f50480a08c0 0x7f50480a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 1 -- 172.21.15.178:0/798726416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50480a2930 msgr2=0x7f5048132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:39:09.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 1 --2- 172.21.15.178:0/798726416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50480a2930 0x7f5048132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 1 monclient: found mon.noname-b 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 1 -- 172.21.15.178:0/798726416 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504804bf60 con 0x7f504809faa0 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f5047fff640 10 monclient: get_auth_request con 0x7f50480a2930 auth_method 0 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f5047fff640 1 --2- 172.21.15.178:0/798726416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50480a2930 0x7f5048132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f504c9d1640 10 monclient: get_auth_request con 0x7f50480a08c0 auth_method 0 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f504c9d1640 1 --2- 172.21.15.178:0/798726416 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f50480a08c0 0x7f50480a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.876+0000 7f50477fe640 1 --2- 172.21.15.178:0/798726416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f504809faa0 0x7f504809fe70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f503c009940 tx=0x7f503c0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c04934910b2a525 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f5046ffd640 1 -- 172.21.15.178:0/798726416 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f503c02ac60 con 0x7f504809faa0 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f5046ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f5046ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f5046ffd640 10 monclient: dump: 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.893 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f5046ffd640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f5046ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f5046ffd640 1 -- 172.21.15.178:0/798726416 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c02ade0 con 0x7f504809faa0 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 5 monclient: authenticate success, global_id 34355 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f5046ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f5046ffd640 1 -- 172.21.15.178:0/798726416 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f503c031ca0 con 0x7f504809faa0 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 10 monclient: get_monmap_and_config success 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 4 set_mon_vals no callback set 2024-09-15T01:39:09.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f5046ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f5046ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f5046ffd640 10 monclient: dump: 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 10 monclient: shutdown 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 1 -- 172.21.15.178:0/798726416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f504809faa0 msgr2=0x7f504809fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 1 --2- 172.21.15.178:0/798726416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f504809faa0 0x7f504809fe70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f503c009940 tx=0x7f503c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 1 -- 172.21.15.178:0/798726416 shutdown_connections 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 1 --2- 172.21.15.178:0/798726416 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f50480a08c0 0x7f50480a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 1 --2- 172.21.15.178:0/798726416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f504809faa0 0x7f504809fe70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 1 --2- 172.21.15.178:0/798726416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50480a2930 0x7f5048132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 1 -- 172.21.15.178:0/798726416 >> 172.21.15.178:0/798726416 conn(0x7f50480a5ed0 msgr2=0x7f50480a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 1 -- 172.21.15.178:0/798726416 shutdown_connections 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.877+0000 7f504e434640 1 -- 172.21.15.178:0/798726416 wait complete. 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.878+0000 7f504e434640 10 monclient: build_initial_monmap 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.878+0000 7f504e434640 10 monclient: monmap: 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:39:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.878+0000 7f504e434640 1 Processor -- start 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.878+0000 7f504e434640 1 -- start start 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.878+0000 7f504e434640 10 monclient: init 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 10 monclient: _reopen_session rank -1 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 10 monclient: _add_conns ranks=[0,6,2,5,1,3,4,7,8] 2024-09-15T01:39:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f504809faa0 0x7f504809aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 10 monclient(hunting): picked mon.noname-a con 0x7f504809faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50480a08c0 0x7f5048131750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 10 monclient(hunting): picked mon.noname-d con 0x7f50480a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50480a2930 0x7f5048131c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 10 monclient(hunting): picked mon.noname-b con 0x7f50480a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504811f180 con 0x7f504809faa0 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504811eb50 con 0x7f50480a2930 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f504809d630 con 0x7f50480a08c0 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 10 monclient(hunting): _renew_subs 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504e434640 10 monclient(hunting): authenticate will time out at 1154.629338s 2024-09-15T01:39:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50480a2930 0x7f5048131c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f5047fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f504809faa0 0x7f504809aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f50477fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50480a08c0 0x7f5048131750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50480a2930 0x7f5048131c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55460/0 (socket says 172.21.15.178:55460) 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f5047fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f504809faa0 0x7f504809aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43422/0 (socket says 172.21.15.178:43422) 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 1 -- 172.21.15.178:0/2853768764 learned_addr learned my addr 172.21.15.178:0/2853768764 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 10 monclient(hunting): get_auth_request con 0x7f50480a2930 auth_method 0 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f5047fff640 10 monclient(hunting): get_auth_request con 0x7f504809faa0 auth_method 0 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f5047fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f5047fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f5047fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f50477fe640 10 monclient(hunting): get_auth_request con 0x7f50480a08c0 auth_method 0 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f50477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f50477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f50477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f50477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f50477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f50477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f5047fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f5047fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f5047fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 10 monclient(hunting): handle_auth_done global_id 34364 payload 824 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 1 -- 172.21.15.178:0/2853768764 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50480a08c0 msgr2=0x7f5048131750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50480a08c0 0x7f5048131750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 1 -- 172.21.15.178:0/2853768764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f504809faa0 msgr2=0x7f504809aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f504809faa0 0x7f504809aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 10 monclient: _finish_hunting 0 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 1 monclient: found mon.noname-b 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 1 -- 172.21.15.178:0/2853768764 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c009450 con 0x7f50480a2930 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f5047fff640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f504809faa0 0x7f504809aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f50477fe640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50480a08c0 0x7f5048131750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:39:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.879+0000 7f504c9d1640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50480a2930 0x7f5048131c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f504000a380 tx=0x7f50400078f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f504000dad0 con 0x7f50480a2930 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f502ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f502ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f502ffff640 10 monclient: dump: 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f502ffff640 10 monclient: _finish_auth 0 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f502ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504000dc70 con 0x7f50480a2930 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f504e434640 5 monclient: authenticate success, global_id 34364 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f504e434640 10 monclient: _renew_subs 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f504e434640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:39:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f504e434640 1 -- 172.21.15.178:0/2853768764 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50481429d0 con 0x7f50480a2930 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f502ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f504000dde0 con 0x7f50480a2930 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f502ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f502ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f502ffff640 10 monclient: dump: 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f504e434640 10 monclient: _renew_subs 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f504e434640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f504e434640 1 -- 172.21.15.178:0/2853768764 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504809b9f0 con 0x7f50480a2930 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f50467fc640 4 set_mon_vals no callback set 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.880+0000 7f504e434640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:09.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f504e434640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811c530 0x7f504811c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f504e434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f504e434640 1 -- 172.21.15.178:0/2853768764 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f504811ea10 con 0x7f504811c530 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5040003020 con 0x7f50480a2930 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f502ffff640 1 --2- 172.21.15.178:0/2853768764 >> v2:172.21.15.204:6812/28210 conn(0x7f50340397b0 0x7f503403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f504004c520 con 0x7f50480a2930 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f5047fff640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811c530 0x7f504811c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f5047fff640 10 monclient: get_auth_request con 0x7f504811c530 auth_method 0 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f5047fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f5047fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f5047fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f50477fe640 1 --2- 172.21.15.178:0/2853768764 >> v2:172.21.15.204:6812/28210 conn(0x7f50340397b0 0x7f503403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f50477fe640 10 monclient: get_auth_request con 0x7f50340397b0 auth_method 0 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f5047fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f5047fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f5047fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f5047fff640 10 monclient(hunting): handle_auth_done global_id 6827 payload 824 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f5047fff640 1 -- 172.21.15.178:0/2853768764 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f504804c4a0 con 0x7f504811c530 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f50477fe640 1 --2- 172.21.15.178:0/2853768764 >> v2:172.21.15.204:6812/28210 conn(0x7f50340397b0 0x7f503403bc60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f503c034000 tx=0x7f503c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.881+0000 7f5047fff640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811c530 0x7f504811c900 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f503800fd90 tx=0x7f5038008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.882+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50380138c0 con 0x7f504811c530 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.882+0000 7f502ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.882+0000 7f502ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.882+0000 7f502ffff640 10 monclient: dump: 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.908 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.882+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f503800c040 con 0x7f504811c530 2024-09-15T01:39:09.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.882+0000 7f502ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:39:09.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.882+0000 7f502ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:39:09.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.882+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811c530 msgr2=0x7f504811c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.882+0000 7f502ffff640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811c530 0x7f504811c900 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f503800fd90 tx=0x7f5038008040 comp rx=0 tx=0).stop 2024-09-15T01:39:09.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.882+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811c530 msgr2=0x7f504811c900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:09.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.882+0000 7f502ffff640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811c530 0x7f504811c900 secure :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f503800fd90 tx=0x7f5038008040 comp rx=0 tx=0).stop 2024-09-15T01:39:09.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.887+0000 7f504e434640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:39:09.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.887+0000 7f504e434640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811ce40 0x7f5048096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:39:09.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.887+0000 7f504e434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:39:09.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.887+0000 7f504e434640 1 -- 172.21.15.178:0/2853768764 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f504811ecc0 con 0x7f504811ce40 2024-09-15T01:39:09.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f5047fff640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811ce40 0x7f5048096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:39:09.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f5047fff640 10 monclient: get_auth_request con 0x7f504811ce40 auth_method 0 2024-09-15T01:39:09.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f5047fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:39:09.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f5047fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:39:09.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f5047fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:39:09.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f5047fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:39:09.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f5047fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:39:09.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f5047fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:39:09.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f5047fff640 10 monclient(hunting): handle_auth_done global_id 6836 payload 824 2024-09-15T01:39:09.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f5047fff640 1 -- 172.21.15.178:0/2853768764 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f504804bc80 con 0x7f504811ce40 2024-09-15T01:39:09.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f5047fff640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811ce40 0x7f5048096a30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5038000c00 tx=0x7f503801d000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:39:09.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50380123c0 con 0x7f504811ce40 2024-09-15T01:39:09.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f502ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:39:09.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f502ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:39:09.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.888+0000 7f502ffff640 10 monclient: dump: 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:39:09.914 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:39:09.915 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:39:09.915 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:39:09.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9289 (secure 0 0 0) 0x7f503800daa0 con 0x7f504811ce40 2024-09-15T01:39:09.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f502ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:39:09.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f502ffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:39:09.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811ce40 msgr2=0x7f5048096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f502ffff640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811ce40 0x7f5048096a30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5038000c00 tx=0x7f503801d000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f502ffff640 1 -- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811ce40 msgr2=0x7f5048096a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:39:09.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f502ffff640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f504811ce40 0x7f5048096a30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.915 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:39:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:39:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:39:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:39:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 94, 2024-09-15T01:39:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:39:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:39:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:39:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:39:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:39:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:39:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:39:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:39:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:39:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:39:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:39:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:39:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:39:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:39:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:39:09.938 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:39:09.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f504e434640 1 -- 172.21.15.178:0/2853768764 >> v2:172.21.15.204:6812/28210 conn(0x7f50340397b0 msgr2=0x7f503403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f504e434640 1 --2- 172.21.15.178:0/2853768764 >> v2:172.21.15.204:6812/28210 conn(0x7f50340397b0 0x7f503403bc60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f503c034000 tx=0x7f503c032000 comp rx=0 tx=0).stop 2024-09-15T01:39:09.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f504e434640 10 monclient: shutdown 2024-09-15T01:39:09.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f504e434640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:39:09.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f504e434640 1 -- 172.21.15.178:0/2853768764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50480a2930 msgr2=0x7f5048131c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:39:09.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f504e434640 1 --2- 172.21.15.178:0/2853768764 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50480a2930 0x7f5048131c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f504000a380 tx=0x7f50400078f0 comp rx=0 tx=0).stop 2024-09-15T01:39:09.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f504c9d1640 1 -- 172.21.15.178:0/2853768764 reap_dead start 2024-09-15T01:39:09.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f504c9d1640 1 -- 172.21.15.178:0/2853768764 reap_dead start 2024-09-15T01:39:09.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f504e434640 1 -- 172.21.15.178:0/2853768764 shutdown_connections 2024-09-15T01:39:09.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f504e434640 1 -- 172.21.15.178:0/2853768764 >> 172.21.15.178:0/2853768764 conn(0x7f50480a5ed0 msgr2=0x7f504809e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:39:09.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f504e434640 1 -- 172.21.15.178:0/2853768764 shutdown_connections 2024-09-15T01:39:09.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:39:09.889+0000 7f504e434640 1 -- 172.21.15.178:0/2853768764 wait complete. 2024-09-15T01:39:09.941 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:39:09.941 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T01:40:39.943 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T01:40:39.943 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-15T01:40:39.944 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T01:40:39.944 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T01:40:39.948 INFO:tasks.ceph.mon.c:Started 2024-09-15T01:40:39.949 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-15T01:40:39.949 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-15T01:40:39.949 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-15T01:40:39.949 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-15T01:40:39.953 INFO:tasks.ceph.mon.i:Started 2024-09-15T01:40:39.953 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-15T01:40:39.953 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-15T01:40:39.953 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T01:40:40.052 INFO:tasks.ceph.mon.i.smithi204.stderr:2024-09-15T01:40:40.050+0000 7f01bd81f900 -1 mon.i@-1(???).log v0 log_external_backlog rewinding external_log_to from 314 -> 0 (sync_force? mon rebuild?) 2024-09-15T01:40:54.954 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-15T01:40:54.955 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-15T01:40:54.955 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:40:54.955 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:40:55.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.065+0000 7f2f9979d640 10 monclient: get_monmap_and_config 2024-09-15T01:40:55.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.065+0000 7f2f9979d640 10 monclient: build_initial_monmap 2024-09-15T01:40:55.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.065+0000 7f2f9979d640 10 monclient: monmap: 2024-09-15T01:40:55.067 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:55.067 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.067 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:55.066063+0000 2024-09-15T01:40:55.067 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:55.066063+0000 2024-09-15T01:40:55.067 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:55.067 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.068 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:55.068 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:55.068 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:55.068 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:55.068 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:55.068 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:55.068 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:55.068 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:55.068 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:55.068 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:55.068 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:55.068 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.065+0000 7f2f9979d640 1 Processor -- start 2024-09-15T01:40:55.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 1 -- start start 2024-09-15T01:40:55.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 10 monclient: init 2024-09-15T01:40:55.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:55.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 10 monclient: _add_conns ranks=[10,7,3,5,0,6,8,2,9,4,1] 2024-09-15T01:40:55.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2f9409e690 0x7f2f9409ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 10 monclient(hunting): picked mon.noname-f con 0x7f2f9409e690 addr v2:172.21.15.204:3300/0 2024-09-15T01:40:55.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f9406e9f0 0x7f2f9409efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 10 monclient(hunting): picked mon.noname-i con 0x7f2f9406e9f0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:40:55.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9412f450 0x7f2f9412f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 10 monclient(hunting): picked mon.noname-g con 0x7f2f9412f450 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:55.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f940a0640 con 0x7f2f9409e690 2024-09-15T01:40:55.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f94069260 con 0x7f2f9412f450 2024-09-15T01:40:55.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9406a2a0 con 0x7f2f9406e9f0 2024-09-15T01:40:55.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:55.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f9979d640 10 monclient(hunting): authenticate will time out at 1259.816559s 2024-09-15T01:40:55.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f9406e9f0 0x7f2f9409efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f937fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9412f450 0x7f2f9412f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f92ffd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2f9409e690 0x7f2f9409ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.066+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f9406e9f0 0x7f2f9409efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:47674/0 (socket says 172.21.15.178:47674) 2024-09-15T01:40:55.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 10 monclient(hunting): get_auth_request con 0x7f2f9409e690 auth_method 0 2024-09-15T01:40:55.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f937fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9412f450 0x7f2f9412f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:56670/0 (socket says 172.21.15.178:56670) 2024-09-15T01:40:55.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f8bfff640 1 -- 172.21.15.178:0/2912743223 learned_addr learned my addr 172.21.15.178:0/2912743223 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:55.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f8bfff640 10 monclient(hunting): get_auth_request con 0x7f2f9406e9f0 auth_method 0 2024-09-15T01:40:55.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f937fe640 10 monclient(hunting): get_auth_request con 0x7f2f9412f450 auth_method 0 2024-09-15T01:40:55.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f937fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 10 monclient(hunting): handle_auth_done global_id 7894 payload 274 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 1 -- 172.21.15.178:0/2912743223 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f9406e9f0 msgr2=0x7f2f9409efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 1 --2- 172.21.15.178:0/2912743223 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f9406e9f0 0x7f2f9409efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 1 -- 172.21.15.178:0/2912743223 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9412f450 msgr2=0x7f2f9412f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 1 --2- 172.21.15.178:0/2912743223 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9412f450 0x7f2f9412f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 1 monclient: found mon.noname-f 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:40:55.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f92ffd640 1 -- 172.21.15.178:0/2912743223 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9404bf60 con 0x7f2f9409e690 2024-09-15T01:40:55.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.067+0000 7f2f937fe640 1 --2- 172.21.15.178:0/2912743223 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9412f450 0x7f2f9412f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:55.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f92ffd640 1 --2- 172.21.15.178:0/2912743223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f9409e690 0x7f2f9409ea60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009850 tx=0x7f2f840090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=111dfdaf4b7c768 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f927fc640 1 -- 172.21.15.178:0/2912743223 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f8402abc0 con 0x7f2f9409e690 2024-09-15T01:40:55.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f927fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:40:55.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f927fc640 10 monclient: dump: 2024-09-15T01:40:55.082 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.082 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.082 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.082 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.082 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.082 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f927fc640 10 monclient: _finish_auth 0 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f927fc640 1 -- 172.21.15.178:0/2912743223 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8402ad40 con 0x7f2f9409e690 2024-09-15T01:40:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 5 monclient: authenticate success, global_id 7894 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f927fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f927fc640 1 -- 172.21.15.178:0/2912743223 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f84031ca0 con 0x7f2f9409e690 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 4 set_mon_vals no callback set 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f927fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f927fc640 10 monclient: dump: 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 10 monclient: shutdown 2024-09-15T01:40:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 1 -- 172.21.15.178:0/2912743223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f9409e690 msgr2=0x7f2f9409ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 1 --2- 172.21.15.178:0/2912743223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f9409e690 0x7f2f9409ea60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009850 tx=0x7f2f840090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 1 -- 172.21.15.178:0/2912743223 shutdown_connections 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 1 --2- 172.21.15.178:0/2912743223 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9412f450 0x7f2f9412f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 1 --2- 172.21.15.178:0/2912743223 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f9406e9f0 0x7f2f9409efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 1 --2- 172.21.15.178:0/2912743223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f9409e690 0x7f2f9409ea60 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 1 -- 172.21.15.178:0/2912743223 >> 172.21.15.178:0/2912743223 conn(0x7f2f94063810 msgr2=0x7f2f940a3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 1 -- 172.21.15.178:0/2912743223 shutdown_connections 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.068+0000 7f2f9979d640 1 -- 172.21.15.178:0/2912743223 wait complete. 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.069+0000 7f2f9979d640 10 monclient: build_initial_monmap 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.069+0000 7f2f9979d640 10 monclient: monmap: 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.069+0000 7f2f9979d640 1 Processor -- start 2024-09-15T01:40:55.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.069+0000 7f2f9979d640 1 -- start start 2024-09-15T01:40:55.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.069+0000 7f2f9979d640 10 monclient: init 2024-09-15T01:40:55.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:55.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 10 monclient: _add_conns ranks=[3,8,2,4,0,1,5,7,6] 2024-09-15T01:40:55.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9406e9f0 0x7f2f94141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 10 monclient(hunting): picked mon.noname-g con 0x7f2f9406e9f0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:55.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f9409e690 0x7f2f9412e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 10 monclient(hunting): picked mon.noname-e con 0x7f2f9409e690 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:55.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f9412f450 0x7f2f9411c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 10 monclient(hunting): picked mon.noname-b con 0x7f2f9412f450 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:40:55.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f940684e0 con 0x7f2f9412f450 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9411fe10 con 0x7f2f9406e9f0 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2f94142100 con 0x7f2f9409e690 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f9979d640 10 monclient(hunting): authenticate will time out at 1259.820374s 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f9409e690 0x7f2f9412e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f9412f450 0x7f2f9411c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f9409e690 0x7f2f9412e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:46074/0 (socket says 172.21.15.178:46074) 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f9412f450 0x7f2f9411c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55768/0 (socket says 172.21.15.178:55768) 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f8bfff640 1 -- 172.21.15.178:0/1949571569 learned_addr learned my addr 172.21.15.178:0/1949571569 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f8bfff640 10 monclient(hunting): get_auth_request con 0x7f2f9409e690 auth_method 0 2024-09-15T01:40:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 10 monclient(hunting): get_auth_request con 0x7f2f9412f450 auth_method 0 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f92ffd640 1 --2- 172.21.15.178:0/1949571569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9406e9f0 0x7f2f94141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f92ffd640 10 monclient(hunting): get_auth_request con 0x7f2f9406e9f0 auth_method 0 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f92ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f92ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f92ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f92ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f92ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 10 monclient(hunting): handle_auth_done global_id 34373 payload 824 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 1 -- 172.21.15.178:0/1949571569 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f9409e690 msgr2=0x7f2f9412e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 1 --2- 172.21.15.178:0/1949571569 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f9409e690 0x7f2f9412e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 1 -- 172.21.15.178:0/1949571569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9406e9f0 msgr2=0x7f2f94141bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 1 --2- 172.21.15.178:0/1949571569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9406e9f0 0x7f2f94141bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 10 monclient: _finish_hunting 0 2024-09-15T01:40:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 1 monclient: found mon.noname-b 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f937fe640 1 -- 172.21.15.178:0/1949571569 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c009a20 con 0x7f2f9412f450 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f92ffd640 1 --2- 172.21.15.178:0/1949571569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9406e9f0 0x7f2f94141bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.070+0000 7f2f8bfff640 1 --2- 172.21.15.178:0/1949571569 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f9409e690 0x7f2f9412e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f937fe640 1 --2- 172.21.15.178:0/1949571569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f9412f450 0x7f2f9411c250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2f941422d0 tx=0x7f2f8c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 1 -- 172.21.15.178:0/1949571569 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f8c00cd40 con 0x7f2f9412f450 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 10 monclient: dump: 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.092 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 10 monclient: _finish_auth 0 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 1 -- 172.21.15.178:0/1949571569 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c00f040 con 0x7f2f9412f450 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f9979d640 5 monclient: authenticate success, global_id 34373 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f9979d640 10 monclient: _renew_subs 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f9979d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f9979d640 1 -- 172.21.15.178:0/1949571569 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f84009390 con 0x7f2f9412f450 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 1 -- 172.21.15.178:0/1949571569 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f8c014d80 con 0x7f2f9412f450 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 10 monclient: dump: 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.095 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f91ffb640 4 set_mon_vals no callback set 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f9979d640 10 monclient: _renew_subs 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f9979d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f9979d640 1 -- 172.21.15.178:0/1949571569 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f94142a00 con 0x7f2f9412f450 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.071+0000 7f2f8affd640 1 -- 172.21.15.178:0/1949571569 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2f8c003020 con 0x7f2f9412f450 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.072+0000 7f2f8affd640 1 --2- 172.21.15.178:0/1949571569 >> v2:172.21.15.204:6812/28210 conn(0x7f2f740397b0 0x7f2f7403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.072+0000 7f2f8affd640 1 -- 172.21.15.178:0/1949571569 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f2f8c04c570 con 0x7f2f9412f450 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.072+0000 7f2f9979d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.072+0000 7f2f9979d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.072+0000 7f2f9979d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.072+0000 7f2f9979d640 1 -- 172.21.15.178:0/1949571569 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9404b720 con 0x7f2f9412f450 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.072+0000 7f2f92ffd640 1 --2- 172.21.15.178:0/1949571569 >> v2:172.21.15.204:6812/28210 conn(0x7f2f740397b0 0x7f2f7403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.072+0000 7f2f92ffd640 10 monclient: get_auth_request con 0x7f2f740397b0 auth_method 0 2024-09-15T01:40:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.072+0000 7f2f92ffd640 1 --2- 172.21.15.178:0/1949571569 >> v2:172.21.15.204:6812/28210 conn(0x7f2f740397b0 0x7f2f7403bc60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009360 tx=0x7f2f84032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.076+0000 7f2f8affd640 1 -- 172.21.15.178:0/1949571569 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2f8c012920 con 0x7f2f9412f450 2024-09-15T01:40:55.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.076+0000 7f2f8affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:55.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.076+0000 7f2f8affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:55.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.294+0000 7f2f9979d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:40:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.294+0000 7f2f9979d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:40:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.294+0000 7f2f9979d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:40:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.294+0000 7f2f9979d640 1 -- 172.21.15.178:0/1949571569 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2f94070bb0 con 0x7f2f9412f450 2024-09-15T01:40:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.295+0000 7f2f8affd640 1 -- 172.21.15.178:0/1949571569 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7f2f8c018020 con 0x7f2f9412f450 2024-09-15T01:40:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.295+0000 7f2f8affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:40:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.295+0000 7f2f8affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:55.297 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":110,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:40:55.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 1 -- 172.21.15.178:0/1949571569 >> v2:172.21.15.204:6812/28210 conn(0x7f2f740397b0 msgr2=0x7f2f7403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 1 --2- 172.21.15.178:0/1949571569 >> v2:172.21.15.204:6812/28210 conn(0x7f2f740397b0 0x7f2f7403bc60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009360 tx=0x7f2f84032000 comp rx=0 tx=0).stop 2024-09-15T01:40:55.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 10 monclient: shutdown 2024-09-15T01:40:55.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:55.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 1 -- 172.21.15.178:0/1949571569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f9412f450 msgr2=0x7f2f9411c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 1 --2- 172.21.15.178:0/1949571569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f9412f450 0x7f2f9411c250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2f941422d0 tx=0x7f2f8c00b040 comp rx=0 tx=0).stop 2024-09-15T01:40:55.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 1 -- 172.21.15.178:0/1949571569 shutdown_connections 2024-09-15T01:40:55.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 1 --2- 172.21.15.178:0/1949571569 >> v2:172.21.15.204:6812/28210 conn(0x7f2f740397b0 0x7f2f7403bc60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 1 --2- 172.21.15.178:0/1949571569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f9412f450 0x7f2f9411c250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 1 --2- 172.21.15.178:0/1949571569 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f9409e690 0x7f2f9412e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 1 --2- 172.21.15.178:0/1949571569 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f9406e9f0 0x7f2f94141bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 1 -- 172.21.15.178:0/1949571569 >> 172.21.15.178:0/1949571569 conn(0x7f2f94063810 msgr2=0x7f2f940a2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 1 -- 172.21.15.178:0/1949571569 shutdown_connections 2024-09-15T01:40:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.297+0000 7f2f9979d640 1 -- 172.21.15.178:0/1949571569 wait complete. 2024-09-15T01:40:55.309 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:40:55.309 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:40:55.309 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:40:55.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.400+0000 7f264975d640 10 monclient: get_monmap_and_config 2024-09-15T01:40:55.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.400+0000 7f264975d640 10 monclient: build_initial_monmap 2024-09-15T01:40:55.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.400+0000 7f264975d640 10 monclient: monmap: 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:55.401640+0000 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:55.401640+0000 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:55.403 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:55.404 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:55.404 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:55.404 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:55.404 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:55.404 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 1 Processor -- start 2024-09-15T01:40:55.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 1 -- start start 2024-09-15T01:40:55.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 10 monclient: init 2024-09-15T01:40:55.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:55.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 10 monclient: _add_conns ranks=[2,3,9,8,1,7,10,5,4,6,0] 2024-09-15T01:40:55.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26440a2930 0x7f2644132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 10 monclient(hunting): picked mon.noname-b con 0x7f26440a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:40:55.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f264409faa0 0x7f264409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 10 monclient(hunting): picked mon.noname-g con 0x7f264409faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:55.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f26440a08c0 0x7f26440a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 10 monclient(hunting): picked mon.noname-a con 0x7f26440a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264409e0f0 con 0x7f26440a08c0 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264409dd80 con 0x7f26440a2930 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26440a2d00 con 0x7f264409faa0 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f264975d640 10 monclient(hunting): authenticate will time out at 1260.151852s 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f2642ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26440a2930 0x7f2644132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.401+0000 7f26437fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f26440a08c0 0x7f26440a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26440a2930 0x7f2644132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55782/0 (socket says 172.21.15.178:55782) 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 1 -- 172.21.15.178:0/2032160755 learned_addr learned my addr 172.21.15.178:0/2032160755 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:55.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26437fe640 10 monclient(hunting): get_auth_request con 0x7f26440a08c0 auth_method 0 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 10 monclient(hunting): get_auth_request con 0x7f26440a2930 auth_method 0 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26427fc640 1 --2- 172.21.15.178:0/2032160755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f264409faa0 0x7f264409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26427fc640 10 monclient(hunting): get_auth_request con 0x7f264409faa0 auth_method 0 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 10 monclient(hunting): handle_auth_done global_id 34382 payload 274 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 1 -- 172.21.15.178:0/2032160755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f264409faa0 msgr2=0x7f264409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 1 --2- 172.21.15.178:0/2032160755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f264409faa0 0x7f264409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 1 -- 172.21.15.178:0/2032160755 >> v2:172.21.15.178:3300/0 conn(0x7f26440a08c0 msgr2=0x7f26440a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 1 --2- 172.21.15.178:0/2032160755 >> v2:172.21.15.178:3300/0 conn(0x7f26440a08c0 0x7f26440a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 1 monclient: found mon.noname-b 2024-09-15T01:40:55.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 1 -- 172.21.15.178:0/2032160755 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264404bf60 con 0x7f26440a2930 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26437fe640 1 --2- 172.21.15.178:0/2032160755 >> v2:172.21.15.178:3300/0 conn(0x7f26440a08c0 0x7f26440a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f26427fc640 1 --2- 172.21.15.178:0/2032160755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f264409faa0 0x7f264409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.402+0000 7f2642ffd640 1 --2- 172.21.15.178:0/2032160755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26440a2930 0x7f2644132de0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2634009910 tx=0x7f26340091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e89583583d71c025 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f2641ffb640 1 -- 172.21.15.178:0/2032160755 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f263402ac80 con 0x7f26440a2930 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f2641ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f2641ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f2641ffb640 10 monclient: dump: 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.418 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f2641ffb640 10 monclient: _finish_auth 0 2024-09-15T01:40:55.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f2641ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f2641ffb640 1 -- 172.21.15.178:0/2032160755 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263402ae00 con 0x7f26440a2930 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 5 monclient: authenticate success, global_id 34382 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f2641ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f2641ffb640 1 -- 172.21.15.178:0/2032160755 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2634031ca0 con 0x7f26440a2930 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 4 set_mon_vals no callback set 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f2641ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f2641ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f2641ffb640 10 monclient: dump: 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.421 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 10 monclient: shutdown 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 1 -- 172.21.15.178:0/2032160755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26440a2930 msgr2=0x7f2644132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 1 --2- 172.21.15.178:0/2032160755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26440a2930 0x7f2644132de0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2634009910 tx=0x7f26340091d0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 1 -- 172.21.15.178:0/2032160755 shutdown_connections 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 1 --2- 172.21.15.178:0/2032160755 >> v2:172.21.15.178:3300/0 conn(0x7f26440a08c0 0x7f26440a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 1 --2- 172.21.15.178:0/2032160755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f264409faa0 0x7f264409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 1 --2- 172.21.15.178:0/2032160755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26440a2930 0x7f2644132de0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 1 -- 172.21.15.178:0/2032160755 >> 172.21.15.178:0/2032160755 conn(0x7f26440a5ed0 msgr2=0x7f26440a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 1 -- 172.21.15.178:0/2032160755 shutdown_connections 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.403+0000 7f264975d640 1 -- 172.21.15.178:0/2032160755 wait complete. 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.404+0000 7f264975d640 10 monclient: build_initial_monmap 2024-09-15T01:40:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.404+0000 7f264975d640 10 monclient: monmap: 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:55.423 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.404+0000 7f264975d640 1 Processor -- start 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.404+0000 7f264975d640 1 -- start start 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.404+0000 7f264975d640 10 monclient: init 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient: _add_conns ranks=[4,8,5,2,3,7,6,0,1] 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f264409faa0 0x7f264409af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient(hunting): picked mon.noname-c con 0x7f264409faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f26440a08c0 0x7f26441317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient(hunting): picked mon.noname-e con 0x7f26440a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:55.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f26440a2930 0x7f2644131d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient(hunting): picked mon.noname-h con 0x7f26440a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f264411ef30 con 0x7f264409faa0 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f264411ebc0 con 0x7f26440a2930 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2644142a70 con 0x7f26440a08c0 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient(hunting): authenticate will time out at 1260.155165s 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f2642ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f264409faa0 0x7f264409af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f26440a08c0 0x7f26441317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f2642ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f264409faa0 0x7f264409af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:32862/0 (socket says 172.21.15.178:32862) 2024-09-15T01:40:55.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f2642ffd640 1 -- 172.21.15.178:0/1761959531 learned_addr learned my addr 172.21.15.178:0/1761959531 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 10 monclient(hunting): get_auth_request con 0x7f26440a08c0 auth_method 0 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f2642ffd640 10 monclient(hunting): get_auth_request con 0x7f264409faa0 auth_method 0 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f2642ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f2642ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f2642ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f2642ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f2642ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f2642ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26437fe640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f26440a2930 0x7f2644131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26437fe640 10 monclient(hunting): get_auth_request con 0x7f26440a2930 auth_method 0 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 10 monclient(hunting): handle_auth_done global_id 34370 payload 824 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 1 -- 172.21.15.178:0/1761959531 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f26440a2930 msgr2=0x7f2644131d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f26440a2930 0x7f2644131d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 1 -- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f264409faa0 msgr2=0x7f264409af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f264409faa0 0x7f264409af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 10 monclient: _finish_hunting 0 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 1 monclient: found mon.noname-e 2024-09-15T01:40:55.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 1 -- 172.21.15.178:0/1761959531 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c009a20 con 0x7f26440a08c0 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f2642ffd640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f264409faa0 0x7f264409af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26427fc640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f26440a08c0 0x7f26441317c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f263800d860 tx=0x7f2638008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2638009b20 con 0x7f26440a08c0 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f262affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f262affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f262affd640 10 monclient: dump: 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.428 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f262affd640 10 monclient: _finish_auth 0 2024-09-15T01:40:55.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f262affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2638009cc0 con 0x7f26440a08c0 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 5 monclient: authenticate success, global_id 34370 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient: _renew_subs 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 1 -- 172.21.15.178:0/1761959531 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2634009450 con 0x7f26440a08c0 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f262affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2638009e30 con 0x7f26440a08c0 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f262affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f262affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f262affd640 10 monclient: dump: 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient: _renew_subs 2024-09-15T01:40:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f264975d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.406+0000 7f264975d640 1 -- 172.21.15.178:0/1761959531 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264411e8e0 con 0x7f26440a08c0 2024-09-15T01:40:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.405+0000 7f26417fa640 4 set_mon_vals no callback set 2024-09-15T01:40:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.406+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2638009640 con 0x7f26440a08c0 2024-09-15T01:40:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.406+0000 7f264975d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.406+0000 7f264975d640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2644137050 0x7f264411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.406+0000 7f264975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.406+0000 7f264975d640 1 -- 172.21.15.178:0/1761959531 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264404c4a0 con 0x7f2644137050 2024-09-15T01:40:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.406+0000 7f262affd640 1 --2- 172.21.15.178:0/1761959531 >> v2:172.21.15.204:6812/28210 conn(0x7f26300397b0 0x7f263003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.406+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f263804bd30 con 0x7f26440a08c0 2024-09-15T01:40:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f2642ffd640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2644137050 0x7f264411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f2642ffd640 10 monclient: get_auth_request con 0x7f2644137050 auth_method 0 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f2642ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f2642ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f2642ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f2642ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f2642ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f2642ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f26437fe640 1 --2- 172.21.15.178:0/1761959531 >> v2:172.21.15.204:6812/28210 conn(0x7f26300397b0 0x7f263003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f26437fe640 10 monclient: get_auth_request con 0x7f26300397b0 auth_method 0 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f2642ffd640 10 monclient(hunting): handle_auth_done global_id 34506 payload 824 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f2642ffd640 1 -- 172.21.15.178:0/1761959531 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f264405ed70 con 0x7f2644137050 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f2642ffd640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2644137050 0x7f264411c530 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2634035040 tx=0x7f2634032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f263402ac80 con 0x7f2644137050 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f262affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f262affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f262affd640 10 monclient: dump: 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.435 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f26437fe640 1 --2- 172.21.15.178:0/1761959531 >> v2:172.21.15.204:6812/28210 conn(0x7f26300397b0 0x7f263003bc60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f263c0099f0 tx=0x7f263c00b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f2634031960 con 0x7f2644137050 2024-09-15T01:40:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f262affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f262affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2644137050 msgr2=0x7f264411c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f262affd640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2644137050 0x7f264411c530 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2634035040 tx=0x7f2634032000 comp rx=0 tx=0).stop 2024-09-15T01:40:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2644137050 msgr2=0x7f264411c530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.407+0000 7f262affd640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2644137050 0x7f264411c530 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2634035040 tx=0x7f2634032000 comp rx=0 tx=0).stop 2024-09-15T01:40:55.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f264975d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:55.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f264975d640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f264411ca70 0x7f264411ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f264975d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f264975d640 1 -- 172.21.15.178:0/1761959531 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264411ed30 con 0x7f264411ca70 2024-09-15T01:40:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f2642ffd640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f264411ca70 0x7f264411ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f2642ffd640 10 monclient: get_auth_request con 0x7f264411ca70 auth_method 0 2024-09-15T01:40:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f2642ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f2642ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f2642ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f2642ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f2642ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f2642ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f2642ffd640 10 monclient(hunting): handle_auth_done global_id 34515 payload 824 2024-09-15T01:40:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.413+0000 7f2642ffd640 1 -- 172.21.15.178:0/1761959531 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f264404bc80 con 0x7f264411ca70 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f2642ffd640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f264411ca70 0x7f264411ce70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2634009910 tx=0x7f263403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2634046890 con 0x7f264411ca70 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f262affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f262affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f262affd640 10 monclient: dump: 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f2634046410 con 0x7f264411ca70 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f262affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f262affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f264411ca70 msgr2=0x7f264411ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f262affd640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f264411ca70 0x7f264411ce70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2634009910 tx=0x7f263403d000 comp rx=0 tx=0).stop 2024-09-15T01:40:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f262affd640 1 -- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f264411ca70 msgr2=0x7f264411ce70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f262affd640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f264411ca70 0x7f264411ce70 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2634009910 tx=0x7f263403d000 comp rx=0 tx=0).stop 2024-09-15T01:40:55.441 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:55.441 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:55.441 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:55.441 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:55.441 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:55.441 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:55.441 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:55.441 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:55.442 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:55.443 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:55.443 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:55.443 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:55.443 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:55.443 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:55.443 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:55.443 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:55.443 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:55.443 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:55.443 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:55.443 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:55.443 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:55.444 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:55.445 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:55.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.447 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:55.447 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:55.447 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.447 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.447 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.447 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:55.447 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.447 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.447 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.450 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:55.451 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.453 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.453 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.453 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:55.453 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.453 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.453 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.453 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.453 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:55.453 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:55.453 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.453 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.453 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.454 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.455 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.456 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.457 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:55.458 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:55.464 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:55.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f264975d640 1 -- 172.21.15.178:0/1761959531 >> v2:172.21.15.204:6812/28210 conn(0x7f26300397b0 msgr2=0x7f263003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f264975d640 1 --2- 172.21.15.178:0/1761959531 >> v2:172.21.15.204:6812/28210 conn(0x7f26300397b0 0x7f263003bc60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f263c0099f0 tx=0x7f263c00b040 comp rx=0 tx=0).stop 2024-09-15T01:40:55.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f264975d640 10 monclient: shutdown 2024-09-15T01:40:55.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f264975d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:55.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f264975d640 1 -- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f26440a08c0 msgr2=0x7f26441317c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f264975d640 1 --2- 172.21.15.178:0/1761959531 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f26440a08c0 0x7f26441317c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f263800d860 tx=0x7f2638008040 comp rx=0 tx=0).stop 2024-09-15T01:40:55.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f26437fe640 1 -- 172.21.15.178:0/1761959531 reap_dead start 2024-09-15T01:40:55.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f26437fe640 1 -- 172.21.15.178:0/1761959531 reap_dead start 2024-09-15T01:40:55.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f264975d640 1 -- 172.21.15.178:0/1761959531 shutdown_connections 2024-09-15T01:40:55.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f264975d640 1 -- 172.21.15.178:0/1761959531 >> 172.21.15.178:0/1761959531 conn(0x7f26440a5ed0 msgr2=0x7f264409e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:55.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f264975d640 1 -- 172.21.15.178:0/1761959531 shutdown_connections 2024-09-15T01:40:55.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.414+0000 7f264975d640 1 -- 172.21.15.178:0/1761959531 wait complete. 2024-09-15T01:40:55.466 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:55.467 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:40:55.467 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:40:55.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.558+0000 7f3b8d82b640 10 monclient: get_monmap_and_config 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.558+0000 7f3b8d82b640 10 monclient: build_initial_monmap 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.558+0000 7f3b8d82b640 10 monclient: monmap: 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:55.559386+0000 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:55.559386+0000 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:55.561 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:55.562 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:55.562 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:55.562 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:55.562 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:55.562 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:55.562 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:55.562 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 1 Processor -- start 2024-09-15T01:40:55.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 1 -- start start 2024-09-15T01:40:55.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 10 monclient: init 2024-09-15T01:40:55.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:55.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 10 monclient: _add_conns ranks=[5,4,6,3,10,8,0,1,9,7,2] 2024-09-15T01:40:55.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b880a2930 0x7f3b88132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 10 monclient(hunting): picked mon.noname-h con 0x7f3b880a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:55.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3b8809faa0 0x7f3b8809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 10 monclient(hunting): picked mon.noname-c con 0x7f3b8809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:55.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b880a08c0 0x7f3b880a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 10 monclient(hunting): picked mon.noname-d con 0x7f3b880a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:40:55.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8809e0f0 con 0x7f3b8809faa0 2024-09-15T01:40:55.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8809dd80 con 0x7f3b880a2930 2024-09-15T01:40:55.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3b880a2d00 con 0x7f3b880a08c0 2024-09-15T01:40:55.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:55.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b8d82b640 10 monclient(hunting): authenticate will time out at 1260.309636s 2024-09-15T01:40:55.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b867fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3b8809faa0 0x7f3b8809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b877fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b880a08c0 0x7f3b880a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b867fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3b8809faa0 0x7f3b8809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:32876/0 (socket says 172.21.15.178:32876) 2024-09-15T01:40:55.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b867fc640 1 -- 172.21.15.178:0/2959180053 learned_addr learned my addr 172.21.15.178:0/2959180053 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:55.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b867fc640 10 monclient(hunting): get_auth_request con 0x7f3b8809faa0 auth_method 0 2024-09-15T01:40:55.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b867fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b867fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b877fe640 10 monclient(hunting): get_auth_request con 0x7f3b880a08c0 auth_method 0 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b877fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b867fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b867fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b867fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.559+0000 7f3b877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b867fc640 10 monclient(hunting): handle_auth_done global_id 44113 payload 274 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b867fc640 1 -- 172.21.15.178:0/2959180053 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b880a08c0 msgr2=0x7f3b880a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:55.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b86ffd640 1 --2- 172.21.15.178:0/2959180053 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b880a2930 0x7f3b88132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b867fc640 1 --2- 172.21.15.178:0/2959180053 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b880a08c0 0x7f3b880a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b867fc640 1 -- 172.21.15.178:0/2959180053 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b880a2930 msgr2=0x7f3b88132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b867fc640 1 --2- 172.21.15.178:0/2959180053 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b880a2930 0x7f3b88132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b867fc640 10 monclient: _finish_hunting 0 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b867fc640 1 monclient: found mon.noname-c 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b867fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b867fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b867fc640 1 -- 172.21.15.178:0/2959180053 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8804bf60 con 0x7f3b8809faa0 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b86ffd640 10 monclient: get_auth_request con 0x7f3b880a2930 auth_method 0 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b877fe640 1 --2- 172.21.15.178:0/2959180053 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b880a08c0 0x7f3b880a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b86ffd640 1 --2- 172.21.15.178:0/2959180053 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b880a2930 0x7f3b88132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b867fc640 1 --2- 172.21.15.178:0/2959180053 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3b8809faa0 0x7f3b8809fe70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009940 tx=0x7f3b700091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=5f8b9f4b90e15893 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b85ffb640 1 -- 172.21.15.178:0/2959180053 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b7002ac60 con 0x7f3b8809faa0 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b85ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b85ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b85ffb640 10 monclient: dump: 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.575 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b85ffb640 10 monclient: _finish_auth 0 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b85ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b85ffb640 1 -- 172.21.15.178:0/2959180053 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7002ade0 con 0x7f3b8809faa0 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b8d82b640 5 monclient: authenticate success, global_id 44113 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b8d82b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b85ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b85ffb640 1 -- 172.21.15.178:0/2959180053 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b70031ca0 con 0x7f3b8809faa0 2024-09-15T01:40:55.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b85ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b85ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.560+0000 7f3b85ffb640 10 monclient: dump: 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.577 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 4 set_mon_vals no callback set 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 10 monclient: shutdown 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 1 -- 172.21.15.178:0/2959180053 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3b8809faa0 msgr2=0x7f3b8809fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 1 --2- 172.21.15.178:0/2959180053 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3b8809faa0 0x7f3b8809fe70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009940 tx=0x7f3b700091d0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 1 -- 172.21.15.178:0/2959180053 shutdown_connections 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 1 --2- 172.21.15.178:0/2959180053 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b880a08c0 0x7f3b880a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 1 --2- 172.21.15.178:0/2959180053 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3b8809faa0 0x7f3b8809fe70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 1 --2- 172.21.15.178:0/2959180053 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3b880a2930 0x7f3b88132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 1 -- 172.21.15.178:0/2959180053 >> 172.21.15.178:0/2959180053 conn(0x7f3b880a5ed0 msgr2=0x7f3b880a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 1 -- 172.21.15.178:0/2959180053 shutdown_connections 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.561+0000 7f3b8d82b640 1 -- 172.21.15.178:0/2959180053 wait complete. 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.562+0000 7f3b8d82b640 10 monclient: build_initial_monmap 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.562+0000 7f3b8d82b640 10 monclient: monmap: 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:55.580 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.562+0000 7f3b8d82b640 1 Processor -- start 2024-09-15T01:40:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.562+0000 7f3b8d82b640 1 -- start start 2024-09-15T01:40:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.562+0000 7f3b8d82b640 10 monclient: init 2024-09-15T01:40:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 10 monclient: _add_conns ranks=[6,4,7,2,1,5,3,0,8] 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b8809faa0 0x7f3b8809b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 10 monclient(hunting): picked mon.noname-d con 0x7f3b8809faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3b880a08c0 0x7f3b88131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 10 monclient(hunting): picked mon.noname-c con 0x7f3b880a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3b880a2930 0x7f3b88141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 10 monclient(hunting): picked mon.noname-i con 0x7f3b880a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8811ea40 con 0x7f3b880a08c0 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8811f000 con 0x7f3b8809faa0 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8809d5e0 con 0x7f3b880a2930 2024-09-15T01:40:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 10 monclient(hunting): authenticate will time out at 1260.313290s 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b86ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b8809faa0 0x7f3b8809b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3b880a08c0 0x7f3b88131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b86ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b8809faa0 0x7f3b8809b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:51510/0 (socket says 172.21.15.178:51510) 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b86ffd640 1 -- 172.21.15.178:0/3252007775 learned_addr learned my addr 172.21.15.178:0/3252007775 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b86ffd640 10 monclient(hunting): get_auth_request con 0x7f3b8809faa0 auth_method 0 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b86ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b86ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 10 monclient(hunting): get_auth_request con 0x7f3b880a08c0 auth_method 0 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b86ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b86ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b86ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 10 monclient(hunting): handle_auth_done global_id 44122 payload 824 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 1 -- 172.21.15.178:0/3252007775 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3b880a2930 msgr2=0x7f3b88141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f3b880a2930 0x7f3b88141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 1 -- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b8809faa0 msgr2=0x7f3b8809b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3b8809faa0 0x7f3b8809b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 10 monclient: _finish_hunting 0 2024-09-15T01:40:55.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 1 monclient: found mon.noname-c 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 1 -- 172.21.15.178:0/3252007775 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b80009a20 con 0x7f3b880a08c0 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b867fc640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3b880a08c0 0x7f3b88131920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3b70031000 tx=0x7f3b70032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b70031820 con 0x7f3b880a08c0 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b7effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b7effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b7effd640 10 monclient: dump: 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.584 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b7effd640 10 monclient: _finish_auth 0 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b7effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:55.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b700319a0 con 0x7f3b880a08c0 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 5 monclient: authenticate success, global_id 44122 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b7effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b7003a390 con 0x7f3b880a08c0 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 10 monclient: _renew_subs 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b8d82b640 1 -- 172.21.15.178:0/3252007775 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b70009450 con 0x7f3b880a08c0 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b7effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b7effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b857fa640 4 set_mon_vals no callback set 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.563+0000 7f3b7effd640 10 monclient: dump: 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.586 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.564+0000 7f3b8d82b640 10 monclient: _renew_subs 2024-09-15T01:40:55.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.564+0000 7f3b8d82b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:40:55.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.564+0000 7f3b8d82b640 1 -- 172.21.15.178:0/3252007775 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8811ca40 con 0x7f3b880a08c0 2024-09-15T01:40:55.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.564+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3b70039040 con 0x7f3b880a08c0 2024-09-15T01:40:55.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.564+0000 7f3b8d82b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:55.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.564+0000 7f3b8d82b640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b8811dae0 0x7f3b88142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.564+0000 7f3b8d82b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.564+0000 7f3b8d82b640 1 -- 172.21.15.178:0/3252007775 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8811ebb0 con 0x7f3b8811dae0 2024-09-15T01:40:55.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b86ffd640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b8811dae0 0x7f3b88142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 1 --2- 172.21.15.178:0/3252007775 >> v2:172.21.15.204:6812/28210 conn(0x7f3b680396e0 0x7f3b6803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b86ffd640 10 monclient: get_auth_request con 0x7f3b8811dae0 auth_method 0 2024-09-15T01:40:55.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b86ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b86ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f3b7006af80 con 0x7f3b880a08c0 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b86ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b86ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b86ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b86ffd640 10 monclient(hunting): handle_auth_done global_id 34391 payload 824 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b86ffd640 1 -- 172.21.15.178:0/3252007775 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b8811f170 con 0x7f3b8811dae0 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b86ffd640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b8811dae0 0x7f3b88142280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800fba0 tx=0x7f3b78008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b877fe640 1 --2- 172.21.15.178:0/3252007775 >> v2:172.21.15.204:6812/28210 conn(0x7f3b680396e0 0x7f3b6803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b877fe640 10 monclient: get_auth_request con 0x7f3b680396e0 auth_method 0 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b78013440 con 0x7f3b8811dae0 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:40:55.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 10 monclient: dump: 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.590 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.591 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.591 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f3b7800c040 con 0x7f3b8811dae0 2024-09-15T01:40:55.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:55.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:55.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b8811dae0 msgr2=0x7f3b88142280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b8811dae0 0x7f3b88142280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800fba0 tx=0x7f3b78008040 comp rx=0 tx=0).stop 2024-09-15T01:40:55.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b8811dae0 msgr2=0x7f3b88142280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:55.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b7effd640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b8811dae0 0x7f3b88142280 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.565+0000 7f3b877fe640 1 --2- 172.21.15.178:0/3252007775 >> v2:172.21.15.204:6812/28210 conn(0x7f3b680396e0 0x7f3b6803bb90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3b8809b830 tx=0x7f3b8000b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b8d82b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:55.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b8d82b640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b88131e60 0x7f3b881427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b8d82b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b8d82b640 1 -- 172.21.15.178:0/3252007775 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8804bc80 con 0x7f3b88131e60 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b86ffd640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b88131e60 0x7f3b881427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b86ffd640 10 monclient: get_auth_request con 0x7f3b88131e60 auth_method 0 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b86ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b86ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b86ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b86ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b86ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b86ffd640 10 monclient(hunting): handle_auth_done global_id 34400 payload 824 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b86ffd640 1 -- 172.21.15.178:0/3252007775 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3b8805ed70 con 0x7f3b88131e60 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.571+0000 7f3b86ffd640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b88131e60 0x7f3b881427c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800a8a0 tx=0x7f3b7801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b78023040 con 0x7f3b88131e60 2024-09-15T01:40:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b7effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b7effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b7effd640 10 monclient: dump: 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.596 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f3b78005ec0 con 0x7f3b88131e60 2024-09-15T01:40:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b7effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b7effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b88131e60 msgr2=0x7f3b881427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b7effd640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b88131e60 0x7f3b881427c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800a8a0 tx=0x7f3b7801d000 comp rx=0 tx=0).stop 2024-09-15T01:40:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b7effd640 1 -- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b88131e60 msgr2=0x7f3b881427c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b7effd640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b88131e60 0x7f3b881427c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.597 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:55.597 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:55.597 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:55.597 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:55.597 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:55.597 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:55.597 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:55.597 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:55.597 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:55.597 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:55.597 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:55.597 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:55.598 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:55.599 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:55.600 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:55.601 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.602 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:55.603 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:55.604 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.605 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:55.607 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.609 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.610 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.611 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.612 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.613 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.614 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:55.615 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.616 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.616 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.616 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:55.616 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:55.616 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.616 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.616 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.616 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:55.616 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.617 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:55.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.619 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:55.619 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:55.619 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:55.619 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.619 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.619 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:55.619 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:55.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b8d82b640 1 -- 172.21.15.178:0/3252007775 >> v2:172.21.15.204:6812/28210 conn(0x7f3b680396e0 msgr2=0x7f3b6803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b8d82b640 1 --2- 172.21.15.178:0/3252007775 >> v2:172.21.15.204:6812/28210 conn(0x7f3b680396e0 0x7f3b6803bb90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3b8809b830 tx=0x7f3b8000b040 comp rx=0 tx=0).stop 2024-09-15T01:40:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b8d82b640 10 monclient: shutdown 2024-09-15T01:40:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b8d82b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b8d82b640 1 -- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3b880a08c0 msgr2=0x7f3b88131920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b8d82b640 1 --2- 172.21.15.178:0/3252007775 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f3b880a08c0 0x7f3b88131920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3b70031000 tx=0x7f3b70032000 comp rx=0 tx=0).stop 2024-09-15T01:40:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b877fe640 1 -- 172.21.15.178:0/3252007775 reap_dead start 2024-09-15T01:40:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b877fe640 1 -- 172.21.15.178:0/3252007775 reap_dead start 2024-09-15T01:40:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b8d82b640 1 -- 172.21.15.178:0/3252007775 shutdown_connections 2024-09-15T01:40:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b8d82b640 1 -- 172.21.15.178:0/3252007775 >> 172.21.15.178:0/3252007775 conn(0x7f3b880a5ed0 msgr2=0x7f3b8809e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b8d82b640 1 -- 172.21.15.178:0/3252007775 shutdown_connections 2024-09-15T01:40:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.572+0000 7f3b8d82b640 1 -- 172.21.15.178:0/3252007775 wait complete. 2024-09-15T01:40:55.622 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:55.622 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:40:55.622 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:40:55.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.714+0000 7f84d455d640 10 monclient: get_monmap_and_config 2024-09-15T01:40:55.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.714+0000 7f84d455d640 10 monclient: build_initial_monmap 2024-09-15T01:40:55.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.714+0000 7f84d455d640 10 monclient: monmap: 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:55.715235+0000 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:55.715235+0000 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:55.717 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:55.718 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:55.718 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:55.718 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:55.718 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:55.718 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.714+0000 7f84d455d640 1 Processor -- start 2024-09-15T01:40:55.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.714+0000 7f84d455d640 1 -- start start 2024-09-15T01:40:55.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.714+0000 7f84d455d640 10 monclient: init 2024-09-15T01:40:55.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:55.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 10 monclient: _add_conns ranks=[5,0,7,1,2,9,3,6,8,4,10] 2024-09-15T01:40:55.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f84cc0a2930 0x7f84cc132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 10 monclient(hunting): picked mon.noname-h con 0x7f84cc0a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:55.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f84cc09faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:40:55.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84cc09fe70 0x7f84cc0a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 10 monclient(hunting): picked mon.noname-i con 0x7f84cc09fe70 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:40:55.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84cc04bf60 con 0x7f84cc09faa0 2024-09-15T01:40:55.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc09dd80 con 0x7f84cc0a2930 2024-09-15T01:40:55.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc0a2d00 con 0x7f84cc09fe70 2024-09-15T01:40:55.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:55.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d455d640 10 monclient(hunting): authenticate will time out at 1260.465588s 2024-09-15T01:40:55.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d1af8640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f84cc09faa0 0x7f84cc0a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:52102/0 (socket says 172.21.15.178:52102) 2024-09-15T01:40:55.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d1af8640 1 -- 172.21.15.178:0/3982217872 learned_addr learned my addr 172.21.15.178:0/3982217872 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:40:55.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d2afa640 1 --2- 172.21.15.178:0/3982217872 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84cc09fe70 0x7f84cc0a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d22f9640 1 --2- 172.21.15.178:0/3982217872 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f84cc0a2930 0x7f84cc132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d2afa640 10 monclient(hunting): get_auth_request con 0x7f84cc09fe70 auth_method 0 2024-09-15T01:40:55.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d2afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d2afa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d2afa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d22f9640 10 monclient(hunting): get_auth_request con 0x7f84cc0a2930 auth_method 0 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d22f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d22f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d22f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d2afa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d2afa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d2afa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.715+0000 7f84d12f7640 1 -- 172.21.15.178:0/3982217872 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904620485 0 0) 0x7f84cc04bf60 con 0x7f84cc09faa0 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d22f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d22f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d22f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d12f7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d12f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d12f7640 10 monclient(hunting): my global_id is 34524 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d12f7640 1 -- 172.21.15.178:0/3982217872 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84b80034e0 con 0x7f84cc09faa0 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d2afa640 10 monclient(hunting): handle_auth_done global_id 44116 payload 274 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d2afa640 1 -- 172.21.15.178:0/3982217872 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f84cc0a2930 msgr2=0x7f84cc132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d2afa640 1 --2- 172.21.15.178:0/3982217872 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f84cc0a2930 0x7f84cc132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d2afa640 1 -- 172.21.15.178:0/3982217872 >> v1:172.21.15.178:6789/0 conn(0x7f84cc09faa0 legacy=0x7f84cc0a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d2afa640 10 monclient: _finish_hunting 0 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d2afa640 1 monclient: found mon.noname-i 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d2afa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d2afa640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d2afa640 1 -- 172.21.15.178:0/3982217872 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc0a0310 con 0x7f84cc09fe70 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d22f9640 1 --2- 172.21.15.178:0/3982217872 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f84cc0a2930 0x7f84cc132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:55.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.716+0000 7f84d2afa640 1 --2- 172.21.15.178:0/3982217872 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84cc09fe70 0x7f84cc0a0fd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009e90 tx=0x7f84c4009780 comp rx=0 tx=0).ready entity=mon.7 client_cookie=9fabce421238160d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d12f7640 1 -- 172.21.15.178:0/3982217872 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84c4031880 con 0x7f84cc09fe70 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d12f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d12f7640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d12f7640 10 monclient: dump: 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.731 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d12f7640 10 monclient: _finish_auth 0 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d12f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d12f7640 1 -- 172.21.15.178:0/3982217872 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c402d050 con 0x7f84cc09fe70 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 5 monclient: authenticate success, global_id 44116 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:55.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d12f7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d12f7640 1 -- 172.21.15.178:0/3982217872 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84c40303a0 con 0x7f84cc09fe70 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 4 set_mon_vals no callback set 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d12f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d12f7640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d12f7640 10 monclient: dump: 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.733 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 10 monclient: shutdown 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 1 -- 172.21.15.178:0/3982217872 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84cc09fe70 msgr2=0x7f84cc0a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 1 --2- 172.21.15.178:0/3982217872 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84cc09fe70 0x7f84cc0a0fd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009e90 tx=0x7f84c4009780 comp rx=0 tx=0).stop 2024-09-15T01:40:55.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 1 -- 172.21.15.178:0/3982217872 shutdown_connections 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 1 --2- 172.21.15.178:0/3982217872 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84cc09fe70 0x7f84cc0a0fd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 1 --2- 172.21.15.178:0/3982217872 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f84cc0a2930 0x7f84cc132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 1 -- 172.21.15.178:0/3982217872 >> 172.21.15.178:0/3982217872 conn(0x7f84cc0a5ed0 msgr2=0x7f84cc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 1 -- 172.21.15.178:0/3982217872 shutdown_connections 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.717+0000 7f84d455d640 1 -- 172.21.15.178:0/3982217872 wait complete. 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient: build_initial_monmap 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient: monmap: 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:55.735 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 1 Processor -- start 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 1 -- start start 2024-09-15T01:40:55.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient: init 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient: _add_conns ranks=[8,7,3,1,6,4,2,5,0] 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f84cc09faa0 0x7f84cc09b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient(hunting): picked mon.noname-e con 0x7f84cc09faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84cc09fe70 0x7f84cc131750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient(hunting): picked mon.noname-i con 0x7f84cc09fe70 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84cc0a2930 0x7f84cc11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient(hunting): picked mon.noname-g con 0x7f84cc0a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc09bf90 con 0x7f84cc0a2930 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc142a30 con 0x7f84cc09fe70 2024-09-15T01:40:55.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc0a2390 con 0x7f84cc09faa0 2024-09-15T01:40:55.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:55.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.718+0000 7f84d455d640 10 monclient(hunting): authenticate will time out at 1260.468992s 2024-09-15T01:40:55.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f84cc09faa0 0x7f84cc09b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f84cc09faa0 0x7f84cc09b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:46084/0 (socket says 172.21.15.178:46084) 2024-09-15T01:40:55.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 1 -- 172.21.15.178:0/2482739614 learned_addr learned my addr 172.21.15.178:0/2482739614 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:55.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 10 monclient(hunting): get_auth_request con 0x7f84cc09faa0 auth_method 0 2024-09-15T01:40:55.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d1af8640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84cc09fe70 0x7f84cc131750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d2afa640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84cc0a2930 0x7f84cc11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d1af8640 10 monclient(hunting): get_auth_request con 0x7f84cc09fe70 auth_method 0 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d1af8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d1af8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d1af8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d2afa640 10 monclient(hunting): get_auth_request con 0x7f84cc0a2930 auth_method 0 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d2afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d2afa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d2afa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d1af8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d1af8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d1af8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d2afa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d2afa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d2afa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 10 monclient(hunting): handle_auth_done global_id 34379 payload 824 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 1 -- 172.21.15.178:0/2482739614 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84cc09fe70 msgr2=0x7f84cc131750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f84cc09fe70 0x7f84cc131750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 1 -- 172.21.15.178:0/2482739614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84cc0a2930 msgr2=0x7f84cc11c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84cc0a2930 0x7f84cc11c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 10 monclient: _finish_hunting 0 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 1 monclient: found mon.noname-e 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:55.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:40:55.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 1 -- 172.21.15.178:0/2482739614 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c8009390 con 0x7f84cc09faa0 2024-09-15T01:40:55.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d22f9640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f84cc09faa0 0x7f84cc09b010 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f84c800ecb0 tx=0x7f84c80090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84c8013470 con 0x7f84cc09faa0 2024-09-15T01:40:55.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84c27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84c27fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:40:55.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84c27fc640 10 monclient: dump: 2024-09-15T01:40:55.741 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.741 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.742 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.742 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.742 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.742 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.742 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.742 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.742 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.742 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.742 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.742 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.742 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.743 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.743 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.743 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84c27fc640 10 monclient: _finish_auth 0 2024-09-15T01:40:55.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:55.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c80135f0 con 0x7f84cc09faa0 2024-09-15T01:40:55.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d455d640 5 monclient: authenticate success, global_id 34379 2024-09-15T01:40:55.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d455d640 10 monclient: _renew_subs 2024-09-15T01:40:55.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d455d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:55.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d455d640 1 -- 172.21.15.178:0/2482739614 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c4009a20 con 0x7f84cc09faa0 2024-09-15T01:40:55.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84c27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:55.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84c8013be0 con 0x7f84cc09faa0 2024-09-15T01:40:55.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84c27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84c27fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:55.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d0af6640 4 set_mon_vals no callback set 2024-09-15T01:40:55.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84c27fc640 10 monclient: dump: 2024-09-15T01:40:55.744 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.744 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.744 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.744 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.744 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.744 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.745 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.745 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.745 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.745 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.745 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.745 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.745 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.745 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.745 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.745 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d455d640 10 monclient: _renew_subs 2024-09-15T01:40:55.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d455d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:55.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.719+0000 7f84d455d640 1 -- 172.21.15.178:0/2482739614 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84cc11d910 con 0x7f84cc09faa0 2024-09-15T01:40:55.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d455d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:55.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f84c8013d40 con 0x7f84cc09faa0 2024-09-15T01:40:55.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d455d640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc131c90 0x7f84cc11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d455d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d455d640 1 -- 172.21.15.178:0/2482739614 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc04c4a0 con 0x7f84cc131c90 2024-09-15T01:40:55.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d1af8640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc131c90 0x7f84cc11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d1af8640 10 monclient: get_auth_request con 0x7f84cc131c90 auth_method 0 2024-09-15T01:40:55.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d1af8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d1af8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d1af8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84c27fc640 1 --2- 172.21.15.178:0/2482739614 >> v2:172.21.15.204:6812/28210 conn(0x7f84b00397b0 0x7f84b003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f84c804c340 con 0x7f84cc09faa0 2024-09-15T01:40:55.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d1af8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d1af8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d1af8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d2afa640 1 --2- 172.21.15.178:0/2482739614 >> v2:172.21.15.204:6812/28210 conn(0x7f84b00397b0 0x7f84b003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d2afa640 10 monclient: get_auth_request con 0x7f84b00397b0 auth_method 0 2024-09-15T01:40:55.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d1af8640 10 monclient(hunting): handle_auth_done global_id 44131 payload 824 2024-09-15T01:40:55.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.720+0000 7f84d1af8640 1 -- 172.21.15.178:0/2482739614 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f84cc05ed70 con 0x7f84cc131c90 2024-09-15T01:40:55.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84d1af8640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc131c90 0x7f84cc11ca70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0a6ec0 tx=0x7f84bc009040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84bc0070d0 con 0x7f84cc131c90 2024-09-15T01:40:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84c27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84c27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:40:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84c27fc640 10 monclient: dump: 2024-09-15T01:40:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.749 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84d2afa640 1 --2- 172.21.15.178:0/2482739614 >> v2:172.21.15.204:6812/28210 conn(0x7f84b00397b0 0x7f84b003bc60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f84c4034000 tx=0x7f84c4037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f84bc005a80 con 0x7f84cc131c90 2024-09-15T01:40:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84c27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:55.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84c27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:55.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc131c90 msgr2=0x7f84cc11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84c27fc640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc131c90 0x7f84cc11ca70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0a6ec0 tx=0x7f84bc009040 comp rx=0 tx=0).stop 2024-09-15T01:40:55.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc131c90 msgr2=0x7f84cc11ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:55.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.721+0000 7f84c27fc640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc131c90 0x7f84cc11ca70 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0a6ec0 tx=0x7f84bc009040 comp rx=0 tx=0).stop 2024-09-15T01:40:55.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d455d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:55.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d455d640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc11cfb0 0x7f84cc096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d455d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d455d640 1 -- 172.21.15.178:0/2482739614 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc04bc80 con 0x7f84cc11cfb0 2024-09-15T01:40:55.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d1af8640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc11cfb0 0x7f84cc096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d1af8640 10 monclient: get_auth_request con 0x7f84cc11cfb0 auth_method 0 2024-09-15T01:40:55.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d1af8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d1af8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d1af8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d1af8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d1af8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d1af8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d1af8640 10 monclient(hunting): handle_auth_done global_id 44140 payload 824 2024-09-15T01:40:55.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d1af8640 1 -- 172.21.15.178:0/2482739614 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f84cc04b720 con 0x7f84cc11cfb0 2024-09-15T01:40:55.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84d1af8640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc11cfb0 0x7f84cc096a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f84bc001460 tx=0x7f84bc013000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84bc00dd70 con 0x7f84cc11cfb0 2024-09-15T01:40:55.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84c27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84c27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:40:55.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.727+0000 7f84c27fc640 10 monclient: dump: 2024-09-15T01:40:55.754 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.754 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.754 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.754 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.754 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.754 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.754 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.754 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.754 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.755 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.755 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.755 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.755 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.755 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.755 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.755 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f84bc00ded0 con 0x7f84cc11cfb0 2024-09-15T01:40:55.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84c27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:55.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84c27fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:55.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc11cfb0 msgr2=0x7f84cc096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84c27fc640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc11cfb0 0x7f84cc096a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f84bc001460 tx=0x7f84bc013000 comp rx=0 tx=0).stop 2024-09-15T01:40:55.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84c27fc640 1 -- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc11cfb0 msgr2=0x7f84cc096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:55.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84c27fc640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f84cc11cfb0 0x7f84cc096a30 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f84bc001460 tx=0x7f84bc013000 comp rx=0 tx=0).stop 2024-09-15T01:40:55.756 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:55.756 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:55.756 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:55.756 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:55.757 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:55.757 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:55.757 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:55.757 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:55.757 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:55.757 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:55.757 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:55.757 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:55.757 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:55.757 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:55.757 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:55.758 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:55.758 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:40:55.758 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:55.758 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:55.758 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:55.758 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:55.758 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:55.758 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:55.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:55.758 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:55.758 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:55.759 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:55.759 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:55.759 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:55.759 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:55.759 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:55.759 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:55.759 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:55.759 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:55.759 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:55.760 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:55.760 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:55.760 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:55.760 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:55.760 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:55.760 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:55.760 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.760 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:55.761 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:55.761 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:55.761 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:55.761 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:55.761 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:55.761 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:55.761 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:55.761 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:55.761 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:55.761 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:55.761 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:55.762 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:55.762 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:55.762 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:55.762 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:55.762 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:55.762 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:55.762 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:55.762 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:55.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:55.762 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:55.762 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:55.763 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:55.763 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:55.763 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:55.763 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:55.763 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:55.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.763 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:55.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.763 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:55.763 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:55.763 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.764 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:55.764 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.764 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:55.764 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.764 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.765 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:55.765 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:55.765 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.765 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.765 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.765 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:55.765 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:55.766 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.766 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:55.766 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.766 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:55.766 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.766 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.767 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:55.767 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:55.767 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.767 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.767 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.767 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:55.767 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:55.768 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.768 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:55.768 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.768 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:55.768 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.768 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.769 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.769 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:55.769 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:55.769 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.769 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.769 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.769 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.769 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:55.769 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:55.770 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.770 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:55.770 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.770 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:55.770 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.771 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.771 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.771 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:55.771 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:55.771 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.771 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.771 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.771 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.771 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:55.772 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:55.772 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.772 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:55.772 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.772 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:55.773 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.773 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.773 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:55.773 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:55.773 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.773 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.773 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.774 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:55.774 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:55.774 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:55.774 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:55.775 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.775 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.775 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:55.775 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:55.775 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.775 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.775 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.776 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.776 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:55.776 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:55.776 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.776 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.776 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:55.776 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.777 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:55.777 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.777 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.777 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:55.777 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:55.777 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.777 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.778 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.778 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:55.778 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:55.778 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.778 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:55.778 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.779 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.779 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:55.779 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.779 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.779 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.779 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:55.779 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:55.779 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.779 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.780 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.780 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:55.780 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:55.780 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:55.780 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:55.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.780 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:55.780 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:55.781 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.781 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:55.781 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:55.781 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:55.781 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.781 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.781 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:55.781 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:55.781 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:55.782 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:55.782 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:55.782 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.782 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.782 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:55.782 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:55.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.782 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:55.782 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:55.782 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:55.783 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.783 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:55.783 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:55.783 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:55.783 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:55.783 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:55.783 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:55.783 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:55.783 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:55.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:55.783 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:55.784 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:55.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84d455d640 1 -- 172.21.15.178:0/2482739614 >> v2:172.21.15.204:6812/28210 conn(0x7f84b00397b0 msgr2=0x7f84b003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84d455d640 1 --2- 172.21.15.178:0/2482739614 >> v2:172.21.15.204:6812/28210 conn(0x7f84b00397b0 0x7f84b003bc60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f84c4034000 tx=0x7f84c4037000 comp rx=0 tx=0).stop 2024-09-15T01:40:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84d455d640 10 monclient: shutdown 2024-09-15T01:40:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84d455d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84d455d640 1 -- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f84cc09faa0 msgr2=0x7f84cc09b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84d455d640 1 --2- 172.21.15.178:0/2482739614 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f84cc09faa0 0x7f84cc09b010 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f84c800ecb0 tx=0x7f84c80090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84d2afa640 1 -- 172.21.15.178:0/2482739614 reap_dead start 2024-09-15T01:40:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84d2afa640 1 -- 172.21.15.178:0/2482739614 reap_dead start 2024-09-15T01:40:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84d455d640 1 -- 172.21.15.178:0/2482739614 shutdown_connections 2024-09-15T01:40:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84d455d640 1 -- 172.21.15.178:0/2482739614 >> 172.21.15.178:0/2482739614 conn(0x7f84cc0a5ed0 msgr2=0x7f84cc09e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:55.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84d455d640 1 -- 172.21.15.178:0/2482739614 shutdown_connections 2024-09-15T01:40:55.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.728+0000 7f84d455d640 1 -- 172.21.15.178:0/2482739614 wait complete. 2024-09-15T01:40:55.787 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:55.787 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:40:55.787 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:40:55.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.879+0000 7f7b47f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:40:55.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.879+0000 7f7b47f9d640 10 monclient: build_initial_monmap 2024-09-15T01:40:55.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.879+0000 7f7b47f9d640 10 monclient: monmap: 2024-09-15T01:40:55.881 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:55.881 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.881 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:55.879983+0000 2024-09-15T01:40:55.881 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:55.879983+0000 2024-09-15T01:40:55.881 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:55.881 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.881 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:55.881 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:55.881 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:55.882 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:55.882 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:55.882 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:55.882 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:55.882 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:55.882 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:55.882 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:55.882 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:55.882 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.879+0000 7f7b47f9d640 1 Processor -- start 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.879+0000 7f7b47f9d640 1 -- start start 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.879+0000 7f7b47f9d640 10 monclient: init 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 10 monclient: _add_conns ranks=[8,0,9,4,1,2,3,10,7,5,6] 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7b4009c020 0x7f7b4009ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f7b4009c020 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7b40067130 addr v1:172.21.15.178:6789/0 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7b40067c10 0x7f7b4013a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 10 monclient(hunting): picked mon.noname-a con 0x7f7b40067c10 addr v2:172.21.15.178:3300/0 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4004bf60 con 0x7f7b40067130 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b4009e220 con 0x7f7b40067c10 2024-09-15T01:40:55.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4009b270 con 0x7f7b4009c020 2024-09-15T01:40:55.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:55.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b47f9d640 10 monclient(hunting): authenticate will time out at 1260.630494s 2024-09-15T01:40:55.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b4653a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7b40067c10 0x7f7b4013a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7b4009c020 0x7f7b4009ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45538640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f7b40067130 0x7f7b40067500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:52118/0 (socket says 172.21.15.178:52118) 2024-09-15T01:40:55.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45538640 1 -- 172.21.15.178:0/4021500290 learned_addr learned my addr 172.21.15.178:0/4021500290 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:40:55.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b4653a640 10 monclient(hunting): get_auth_request con 0x7f7b40067c10 auth_method 0 2024-09-15T01:40:55.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b4653a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b4653a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b4653a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 10 monclient(hunting): get_auth_request con 0x7f7b4009c020 auth_method 0 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b4653a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b4653a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b4653a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 10 monclient(hunting): handle_auth_done global_id 34388 payload 274 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 1 -- 172.21.15.178:0/4021500290 >> v2:172.21.15.178:3300/0 conn(0x7f7b40067c10 msgr2=0x7f7b4013a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 1 --2- 172.21.15.178:0/4021500290 >> v2:172.21.15.178:3300/0 conn(0x7f7b40067c10 0x7f7b4013a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 1 -- 172.21.15.178:0/4021500290 >> v1:172.21.15.178:6789/0 conn(0x7f7b40067130 legacy=0x7f7b40067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 10 monclient: _finish_hunting 0 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 1 monclient: found mon.noname-e 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b45d39640 1 -- 172.21.15.178:0/4021500290 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4009af90 con 0x7f7b4009c020 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.880+0000 7f7b4653a640 1 --2- 172.21.15.178:0/4021500290 >> v2:172.21.15.178:3300/0 conn(0x7f7b40067c10 0x7f7b4013a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b45d39640 1 --2- 172.21.15.178:0/4021500290 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7b4009c020 0x7f7b4009ea80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009880 tx=0x7f7b3c0090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=edafdd1c32ff0409 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b44d37640 1 -- 172.21.15.178:0/4021500290 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b3c02ac50 con 0x7f7b4009c020 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b44d37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b44d37640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b44d37640 10 monclient: dump: 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.896 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.897 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b44d37640 10 monclient: _finish_auth 0 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b44d37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b47f9d640 5 monclient: authenticate success, global_id 34388 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b47f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b44d37640 1 -- 172.21.15.178:0/4021500290 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c02add0 con 0x7f7b4009c020 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b44d37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b47f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b47f9d640 4 set_mon_vals no callback set 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b44d37640 1 -- 172.21.15.178:0/4021500290 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b3c031ca0 con 0x7f7b4009c020 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b44d37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b44d37640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:55.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.881+0000 7f7b44d37640 10 monclient: dump: 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.899 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.882+0000 7f7b47f9d640 10 monclient: shutdown 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.882+0000 7f7b47f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.882+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4021500290 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7b4009c020 msgr2=0x7f7b4009ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.882+0000 7f7b47f9d640 1 --2- 172.21.15.178:0/4021500290 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7b4009c020 0x7f7b4009ea80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009880 tx=0x7f7b3c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.882+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4021500290 shutdown_connections 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.882+0000 7f7b47f9d640 1 --2- 172.21.15.178:0/4021500290 >> v2:172.21.15.178:3300/0 conn(0x7f7b40067c10 0x7f7b4013a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.882+0000 7f7b47f9d640 1 --2- 172.21.15.178:0/4021500290 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7b4009c020 0x7f7b4009ea80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.882+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4021500290 >> 172.21.15.178:0/4021500290 conn(0x7f7b40068a60 msgr2=0x7f7b400a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.882+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4021500290 shutdown_connections 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.882+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4021500290 wait complete. 2024-09-15T01:40:55.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.882+0000 7f7b47f9d640 10 monclient: build_initial_monmap 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.882+0000 7f7b47f9d640 10 monclient: monmap: 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:55.901 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 1 Processor -- start 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 1 -- start start 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 10 monclient: init 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 10 monclient: _add_conns ranks=[6,5,1,7,0,3,8,4,2] 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40067130 0x7f7b40141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 10 monclient(hunting): picked mon.noname-d con 0x7f7b40067130 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7b40067c10 0x7f7b401325a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 10 monclient(hunting): picked mon.noname-h con 0x7f7b40067c10 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:55.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b4009c020 0x7f7b4011c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 10 monclient(hunting): picked mon.noname-f con 0x7f7b4009c020 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:40:55.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4011dc90 con 0x7f7b4009c020 2024-09-15T01:40:55.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4011f9a0 con 0x7f7b40067c10 2024-09-15T01:40:55.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4011f6c0 con 0x7f7b40067130 2024-09-15T01:40:55.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:55.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b47f9d640 10 monclient(hunting): authenticate will time out at 1260.633713s 2024-09-15T01:40:55.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b45d39640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40067130 0x7f7b40141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b45d39640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40067130 0x7f7b40141b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:51526/0 (socket says 172.21.15.178:51526) 2024-09-15T01:40:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b45d39640 1 -- 172.21.15.178:0/4292368263 learned_addr learned my addr 172.21.15.178:0/4292368263 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b45d39640 10 monclient(hunting): get_auth_request con 0x7f7b40067130 auth_method 0 2024-09-15T01:40:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b45d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b45d39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b45d39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b45d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b45d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.883+0000 7f7b45d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b45d39640 10 monclient(hunting): handle_auth_done global_id 24954 payload 824 2024-09-15T01:40:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b45d39640 1 -- 172.21.15.178:0/4292368263 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7b40067c10 msgr2=0x7f7b401325a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b45d39640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7b40067c10 0x7f7b401325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b45d39640 1 -- 172.21.15.178:0/4292368263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b4009c020 msgr2=0x7f7b4011c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b45d39640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b4009c020 0x7f7b4011c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b45d39640 10 monclient: _finish_hunting 0 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b45d39640 1 monclient: found mon.noname-d 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b45d39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b45d39640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b45d39640 1 -- 172.21.15.178:0/4292368263 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b38009a20 con 0x7f7b40067130 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b45d39640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40067130 0x7f7b40141b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009880 tx=0x7f7b3c032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b3c03b870 con 0x7f7b40067130 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b35ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b35ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b35ffb640 10 monclient: dump: 2024-09-15T01:40:55.905 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.907 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b35ffb640 10 monclient: _finish_auth 0 2024-09-15T01:40:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b35ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b47f9d640 5 monclient: authenticate success, global_id 24954 2024-09-15T01:40:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c03b9f0 con 0x7f7b40067130 2024-09-15T01:40:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b35ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b3c0313a0 con 0x7f7b40067130 2024-09-15T01:40:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b47f9d640 10 monclient: _renew_subs 2024-09-15T01:40:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b47f9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:40:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4292368263 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3c009390 con 0x7f7b40067130 2024-09-15T01:40:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b35ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b37fff640 4 set_mon_vals no callback set 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b35ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b35ffb640 10 monclient: dump: 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b47f9d640 10 monclient: _renew_subs 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b47f9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.884+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4292368263 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b40142080 con 0x7f7b40067130 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7b3c039040 con 0x7f7b40067130 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b47f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b47f9d640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b4011e7d0 0x7f7b4011c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b47f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:55.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4292368263 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7b400641f0 con 0x7f7b4011e7d0 2024-09-15T01:40:55.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b4011e7d0 0x7f7b4011c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 10 monclient: get_auth_request con 0x7f7b4011e7d0 auth_method 0 2024-09-15T01:40:55.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:55.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:55.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:55.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b35ffb640 1 --2- 172.21.15.178:0/4292368263 >> v2:172.21.15.204:6812/28210 conn(0x7f7b240397e0 0x7f7b2403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f7b3c06b7c0 con 0x7f7b40067130 2024-09-15T01:40:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 10 monclient(hunting): handle_auth_done global_id 24963 payload 824 2024-09-15T01:40:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 1 -- 172.21.15.178:0/4292368263 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b4004c4a0 con 0x7f7b4011e7d0 2024-09-15T01:40:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b4653a640 1 --2- 172.21.15.178:0/4292368263 >> v2:172.21.15.204:6812/28210 conn(0x7f7b240397e0 0x7f7b2403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b4653a640 10 monclient: get_auth_request con 0x7f7b240397e0 auth_method 0 2024-09-15T01:40:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 1 -- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b4011e7d0 msgr2=0x7f7b4011c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T01:40:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 1 -- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b4011e7d0 msgr2=0x7f7b4011c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:40:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.885+0000 7f7b45538640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b4011e7d0 0x7f7b4011c5b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f7b30005840 tx=0x7f7b30007df0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:40:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.886+0000 7f7b45538640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b4011e7d0 0x7f7b4011c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:40:55.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:55.886+0000 7f7b4653a640 1 --2- 172.21.15.178:0/4292368263 >> v2:172.21.15.204:6812/28210 conn(0x7f7b240397e0 0x7f7b2403bc90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7b4011da30 tx=0x7f7b3800b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b45538640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b4011e7d0 0x7f7b4011c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b45538640 10 monclient: get_auth_request con 0x7f7b4011e7d0 auth_method 0 2024-09-15T01:40:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b45538640 10 monclient: get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b45538640 10 monclient: _init_auth method 2 2024-09-15T01:40:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b45538640 10 monclient: _init_auth already have auth, reseting 2024-09-15T01:40:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b45538640 10 monclient: handle_auth_reply_more payload 9 2024-09-15T01:40:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b45538640 10 monclient: handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b45538640 10 monclient: handle_auth_reply_more responding with 132 bytes 2024-09-15T01:40:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b45538640 10 monclient: handle_auth_done global_id 24963 payload 293 2024-09-15T01:40:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b45538640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b4011e7d0 0x7f7b4011c5b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000cfd0 tx=0x7f7b30007df0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b30005e80 con 0x7f7b4011e7d0 2024-09-15T01:40:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b35ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b35ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:40:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b35ffb640 10 monclient: dump: 2024-09-15T01:40:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f7b300070d0 con 0x7f7b4011e7d0 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b35ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b35ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b4011e7d0 msgr2=0x7f7b4011c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b35ffb640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b4011e7d0 0x7f7b4011c5b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000cfd0 tx=0x7f7b30007df0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b4011e7d0 msgr2=0x7f7b4011c5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.087+0000 7f7b35ffb640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b4011e7d0 0x7f7b4011c5b0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000cfd0 tx=0x7f7b30007df0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.093+0000 7f7b47f9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.093+0000 7f7b47f9d640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40132ae0 0x7f7b4011caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.093+0000 7f7b47f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.093+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4292368263 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4009b650 con 0x7f7b40132ae0 2024-09-15T01:40:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.093+0000 7f7b45538640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40132ae0 0x7f7b4011caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.093+0000 7f7b45538640 10 monclient: get_auth_request con 0x7f7b40132ae0 auth_method 0 2024-09-15T01:40:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.093+0000 7f7b45538640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.093+0000 7f7b45538640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.093+0000 7f7b45538640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.093+0000 7f7b45538640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.093+0000 7f7b45538640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.093+0000 7f7b45538640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b45538640 10 monclient(hunting): handle_auth_done global_id 24972 payload 824 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b45538640 1 -- 172.21.15.178:0/4292368263 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7b4011fb10 con 0x7f7b40132ae0 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b45538640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40132ae0 0x7f7b4011caf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b30007fd0 tx=0x7f7b30019000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b30013420 con 0x7f7b40132ae0 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b35ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b35ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b35ffb640 10 monclient: dump: 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f7b30025040 con 0x7f7b40132ae0 2024-09-15T01:40:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b35ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b35ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40132ae0 msgr2=0x7f7b4011caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b35ffb640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40132ae0 0x7f7b4011caf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b30007fd0 tx=0x7f7b30019000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b35ffb640 1 -- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40132ae0 msgr2=0x7f7b4011caf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.094+0000 7f7b35ffb640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40132ae0 0x7f7b4011caf0 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b30007fd0 tx=0x7f7b30019000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.100 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:56.100 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:56.100 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:56.100 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.101 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:40:56.102 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:56.102 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:56.102 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:56.102 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.102 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.102 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.102 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.102 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.102 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.102 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.102 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.102 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.103 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:56.104 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.105 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.106 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.107 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:40:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:56.127 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:56.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.095+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4292368263 >> v2:172.21.15.204:6812/28210 conn(0x7f7b240397e0 msgr2=0x7f7b2403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.095+0000 7f7b47f9d640 1 --2- 172.21.15.178:0/4292368263 >> v2:172.21.15.204:6812/28210 conn(0x7f7b240397e0 0x7f7b2403bc90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7b4011da30 tx=0x7f7b3800b040 comp rx=0 tx=0).stop 2024-09-15T01:40:56.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.095+0000 7f7b47f9d640 10 monclient: shutdown 2024-09-15T01:40:56.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.095+0000 7f7b47f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:56.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.095+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40067130 msgr2=0x7f7b40141b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.095+0000 7f7b47f9d640 1 --2- 172.21.15.178:0/4292368263 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7b40067130 0x7f7b40141b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009880 tx=0x7f7b3c032000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.095+0000 7f7b4653a640 1 -- 172.21.15.178:0/4292368263 reap_dead start 2024-09-15T01:40:56.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.095+0000 7f7b4653a640 1 -- 172.21.15.178:0/4292368263 reap_dead start 2024-09-15T01:40:56.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.095+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4292368263 shutdown_connections 2024-09-15T01:40:56.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.095+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4292368263 >> 172.21.15.178:0/4292368263 conn(0x7f7b40068a60 msgr2=0x7f7b4006b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:56.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.095+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4292368263 shutdown_connections 2024-09-15T01:40:56.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.095+0000 7f7b47f9d640 1 -- 172.21.15.178:0/4292368263 wait complete. 2024-09-15T01:40:56.131 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:56.131 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:40:56.131 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:40:56.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.223+0000 7f562324f640 10 monclient: get_monmap_and_config 2024-09-15T01:40:56.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.223+0000 7f562324f640 10 monclient: build_initial_monmap 2024-09-15T01:40:56.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.223+0000 7f562324f640 10 monclient: monmap: 2024-09-15T01:40:56.225 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:56.225 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.225 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:56.224022+0000 2024-09-15T01:40:56.225 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:56.224022+0000 2024-09-15T01:40:56.225 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:56.225 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.226 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:56.226 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:56.226 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:56.226 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:56.226 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:56.226 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:56.226 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:56.226 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:56.226 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:56.226 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:56.226 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:56.226 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.223+0000 7f562324f640 1 Processor -- start 2024-09-15T01:40:56.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.223+0000 7f562324f640 1 -- start start 2024-09-15T01:40:56.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.223+0000 7f562324f640 10 monclient: init 2024-09-15T01:40:56.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:56.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 10 monclient: _add_conns ranks=[1,7,2,6,9,4,10,3,0,5,8] 2024-09-15T01:40:56.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f561c0a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:40:56.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f561c132de0 0x7f561c137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 10 monclient(hunting): picked mon.noname-i con 0x7f561c132de0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:40:56.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f561c1380e0 0x7f561c13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 10 monclient(hunting): picked mon.noname-b con 0x7f561c1380e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561c04bf60 con 0x7f561c0a2930 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561c09dd80 con 0x7f561c1380e0 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0a2d00 con 0x7f561c132de0 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f562324f640 10 monclient(hunting): authenticate will time out at 1260.974521s 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f561c1380e0 0x7f561c13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f561c1380e0 0x7f561c13a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55798/0 (socket says 172.21.15.178:55798) 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 1 -- 172.21.15.178:0/3276423356 learned_addr learned my addr 172.21.15.178:0/3276423356 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 10 monclient(hunting): get_auth_request con 0x7f561c1380e0 auth_method 0 2024-09-15T01:40:56.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f561bfff640 1 --2- 172.21.15.178:0/3276423356 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f561c132de0 0x7f561c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 10 monclient(hunting): handle_auth_done global_id 34409 payload 274 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 1 -- 172.21.15.178:0/3276423356 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f561c132de0 msgr2=0x7f561c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 1 --2- 172.21.15.178:0/3276423356 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f561c132de0 0x7f561c137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 1 -- 172.21.15.178:0/3276423356 >> v1:172.21.15.204:6789/0 conn(0x7f561c0a2930 legacy=0x7f561c09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.224+0000 7f56217ec640 10 monclient: _finish_hunting 0 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f56217ec640 1 monclient: found mon.noname-b 2024-09-15T01:40:56.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f56217ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f56217ec640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f56217ec640 1 -- 172.21.15.178:0/3276423356 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c1331b0 con 0x7f561c1380e0 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561bfff640 10 monclient: get_auth_request con 0x7f561c132de0 auth_method 0 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561bfff640 1 --2- 172.21.15.178:0/3276423356 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f561c132de0 0x7f561c137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f56217ec640 1 --2- 172.21.15.178:0/3276423356 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f561c1380e0 0x7f561c13a4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5614009940 tx=0x7f56140091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c0ab8c0d7a5f245 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561b7fe640 1 -- 172.21.15.178:0/3276423356 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f561402ad00 con 0x7f561c1380e0 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561b7fe640 10 monclient: dump: 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.238 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:40:56.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561b7fe640 1 -- 172.21.15.178:0/3276423356 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561402ae80 con 0x7f561c1380e0 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f562324f640 5 monclient: authenticate success, global_id 34409 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f562324f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f562324f640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561b7fe640 1 -- 172.21.15.178:0/3276423356 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5614031ca0 con 0x7f561c1380e0 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f562324f640 4 set_mon_vals no callback set 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.225+0000 7f561b7fe640 10 monclient: dump: 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.240 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.226+0000 7f562324f640 10 monclient: shutdown 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.226+0000 7f562324f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.226+0000 7f562324f640 1 -- 172.21.15.178:0/3276423356 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f561c1380e0 msgr2=0x7f561c13a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.226+0000 7f562324f640 1 --2- 172.21.15.178:0/3276423356 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f561c1380e0 0x7f561c13a4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5614009940 tx=0x7f56140091d0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.226+0000 7f562324f640 1 -- 172.21.15.178:0/3276423356 shutdown_connections 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.226+0000 7f562324f640 1 --2- 172.21.15.178:0/3276423356 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f561c1380e0 0x7f561c13a4c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.226+0000 7f562324f640 1 --2- 172.21.15.178:0/3276423356 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f561c132de0 0x7f561c137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.226+0000 7f562324f640 1 -- 172.21.15.178:0/3276423356 >> 172.21.15.178:0/3276423356 conn(0x7f561c0a5ed0 msgr2=0x7f561c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.226+0000 7f562324f640 1 -- 172.21.15.178:0/3276423356 shutdown_connections 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.226+0000 7f562324f640 1 -- 172.21.15.178:0/3276423356 wait complete. 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.226+0000 7f562324f640 10 monclient: build_initial_monmap 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.226+0000 7f562324f640 10 monclient: monmap: 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 1 Processor -- start 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 1 -- start start 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 10 monclient: init 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 10 monclient: _add_conns ranks=[4,1,7,8,6,0,3,5,2] 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f561c0a2930 0x7f561c09a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 10 monclient(hunting): picked mon.noname-c con 0x7f561c0a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f561c132de0 0x7f561c09ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 10 monclient(hunting): picked mon.noname-f con 0x7f561c132de0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f561c1380e0 0x7f561c09b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 10 monclient(hunting): picked mon.noname-i con 0x7f561c1380e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0a2390 con 0x7f561c132de0 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f561c09d630 con 0x7f561c0a2930 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f561c11de50 con 0x7f561c1380e0 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.227+0000 7f562324f640 10 monclient(hunting): authenticate will time out at 1260.977933s 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f561c0a2930 0x7f561c09a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f561c0a2930 0x7f561c09a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:32922/0 (socket says 172.21.15.178:32922) 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 1 -- 172.21.15.178:0/528395887 learned_addr learned my addr 172.21.15.178:0/528395887 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 10 monclient(hunting): get_auth_request con 0x7f561c0a2930 auth_method 0 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f561bfff640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f561c132de0 0x7f561c09ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f56217ec640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f561c1380e0 0x7f561c09b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f561bfff640 10 monclient(hunting): get_auth_request con 0x7f561c132de0 auth_method 0 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f561bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f561bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f561bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f56217ec640 10 monclient(hunting): get_auth_request con 0x7f561c1380e0 auth_method 0 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f56217ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f56217ec640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f56217ec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 10 monclient(hunting): handle_auth_done global_id 44149 payload 824 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 1 -- 172.21.15.178:0/528395887 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f561c1380e0 msgr2=0x7f561c09b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f561c1380e0 0x7f561c09b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 1 -- 172.21.15.178:0/528395887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f561c132de0 msgr2=0x7f561c09ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f561c132de0 0x7f561c09ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 10 monclient: _finish_hunting 0 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 1 monclient: found mon.noname-c 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 1 -- 172.21.15.178:0/528395887 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5614009450 con 0x7f561c0a2930 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5620feb640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f561c0a2930 0x7f561c09a630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f560c00c870 tx=0x7f560c001630 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f560c00ce60 con 0x7f561c0a2930 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5618ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5618ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5618ff9640 10 monclient: dump: 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5618ff9640 10 monclient: _finish_auth 0 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5618ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f562324f640 5 monclient: authenticate success, global_id 44149 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560c00acd0 con 0x7f561c0a2930 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5618ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f562324f640 10 monclient: _renew_subs 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f560c00ae40 con 0x7f561c0a2930 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f562324f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f562324f640 1 -- 172.21.15.178:0/528395887 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561c11db00 con 0x7f561c0a2930 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5618ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5618ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f561affd640 4 set_mon_vals no callback set 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f5618ff9640 10 monclient: dump: 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.250 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f562324f640 10 monclient: _renew_subs 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f562324f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.228+0000 7f562324f640 1 -- 172.21.15.178:0/528395887 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561c131a60 con 0x7f561c0a2930 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f560c003030 con 0x7f561c0a2930 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f562324f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f562324f640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c11c650 0x7f561c09b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f562324f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f562324f640 1 -- 172.21.15.178:0/528395887 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f561c11dc90 con 0x7f561c11c650 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f561bfff640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c11c650 0x7f561c09b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f561bfff640 10 monclient: get_auth_request con 0x7f561c11c650 auth_method 0 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f561bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f561bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f561bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f5618ff9640 1 --2- 172.21.15.178:0/528395887 >> v2:172.21.15.204:6812/28210 conn(0x7f55fc0397e0 0x7f55fc03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f561bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f561bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f561bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f560c04bb40 con 0x7f561c0a2930 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f56217ec640 1 --2- 172.21.15.178:0/528395887 >> v2:172.21.15.204:6812/28210 conn(0x7f55fc0397e0 0x7f55fc03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f561bfff640 10 monclient(hunting): handle_auth_done global_id 34397 payload 824 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.229+0000 7f561bfff640 1 -- 172.21.15.178:0/528395887 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f561c05ed70 con 0x7f561c11c650 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f56217ec640 10 monclient: get_auth_request con 0x7f55fc0397e0 auth_method 0 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f561bfff640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c11c650 0x7f561c09b5f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f560400e780 tx=0x7f5604008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f560400cd30 con 0x7f561c11c650 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f5618ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f5618ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f5618ff9640 10 monclient: dump: 2024-09-15T01:40:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.254 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f56217ec640 1 --2- 172.21.15.178:0/528395887 >> v2:172.21.15.204:6812/28210 conn(0x7f55fc0397e0 0x7f55fc03bc90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5614034000 tx=0x7f5614032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f560400ce90 con 0x7f561c11c650 2024-09-15T01:40:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f5618ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f5618ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c11c650 msgr2=0x7f561c09b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f5618ff9640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c11c650 0x7f561c09b5f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f560400e780 tx=0x7f5604008040 comp rx=0 tx=0).stop 2024-09-15T01:40:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c11c650 msgr2=0x7f561c09b5f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.230+0000 7f5618ff9640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c11c650 0x7f561c09b5f0 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f560400e780 tx=0x7f5604008040 comp rx=0 tx=0).stop 2024-09-15T01:40:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f562324f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f562324f640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c096a30 0x7f561c096e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f562324f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f562324f640 1 -- 172.21.15.178:0/528395887 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f561c04bc80 con 0x7f561c096a30 2024-09-15T01:40:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f561bfff640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c096a30 0x7f561c096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f561bfff640 10 monclient: get_auth_request con 0x7f561c096a30 auth_method 0 2024-09-15T01:40:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f561bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f561bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f561bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f561bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f561bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f561bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f561bfff640 10 monclient(hunting): handle_auth_done global_id 34406 payload 824 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f561bfff640 1 -- 172.21.15.178:0/528395887 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f561c04b720 con 0x7f561c096a30 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f561bfff640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c096a30 0x7f561c096e30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f560400e780 tx=0x7f560401d000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5604013bb0 con 0x7f561c096a30 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f5618ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f5618ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f5618ff9640 10 monclient: dump: 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f5604013d10 con 0x7f561c096a30 2024-09-15T01:40:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f5618ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f5618ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.236+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c096a30 msgr2=0x7f561c096e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f5618ff9640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c096a30 0x7f561c096e30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f560400e780 tx=0x7f560401d000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f5618ff9640 1 -- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c096a30 msgr2=0x7f561c096e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f5618ff9640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f561c096a30 0x7f561c096e30 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f560400e780 tx=0x7f560401d000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.259 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:56.259 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:56.259 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:56.259 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:56.259 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:56.260 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:56.260 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:56.260 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:56.260 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:56.260 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:56.260 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:56.260 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:56.260 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:56.260 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:56.260 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:56.260 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.260 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.261 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:56.262 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:56.266 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:56.268 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.269 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.271 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.272 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.273 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:56.274 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.275 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.276 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.277 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:56.278 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.279 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.280 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:56.281 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.281 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.281 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:56.281 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.281 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.281 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.281 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:56.281 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.281 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.281 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.281 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:56.281 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f562324f640 1 -- 172.21.15.178:0/528395887 >> v2:172.21.15.204:6812/28210 conn(0x7f55fc0397e0 msgr2=0x7f55fc03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f562324f640 1 --2- 172.21.15.178:0/528395887 >> v2:172.21.15.204:6812/28210 conn(0x7f55fc0397e0 0x7f55fc03bc90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5614034000 tx=0x7f5614032000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f562324f640 10 monclient: shutdown 2024-09-15T01:40:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f562324f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f562324f640 1 -- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f561c0a2930 msgr2=0x7f561c09a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f562324f640 1 --2- 172.21.15.178:0/528395887 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f561c0a2930 0x7f561c09a630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f560c00c870 tx=0x7f560c001630 comp rx=0 tx=0).stop 2024-09-15T01:40:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f56217ec640 1 -- 172.21.15.178:0/528395887 reap_dead start 2024-09-15T01:40:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f56217ec640 1 -- 172.21.15.178:0/528395887 reap_dead start 2024-09-15T01:40:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f562324f640 1 -- 172.21.15.178:0/528395887 shutdown_connections 2024-09-15T01:40:56.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f562324f640 1 -- 172.21.15.178:0/528395887 >> 172.21.15.178:0/528395887 conn(0x7f561c0a5ed0 msgr2=0x7f561c09f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:56.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f562324f640 1 -- 172.21.15.178:0/528395887 shutdown_connections 2024-09-15T01:40:56.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.237+0000 7f562324f640 1 -- 172.21.15.178:0/528395887 wait complete. 2024-09-15T01:40:56.284 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:56.284 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:40:56.284 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:40:56.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.376+0000 7fdacbe80640 10 monclient: get_monmap_and_config 2024-09-15T01:40:56.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.376+0000 7fdacbe80640 10 monclient: build_initial_monmap 2024-09-15T01:40:56.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.377+0000 7fdacbe80640 10 monclient: monmap: 2024-09-15T01:40:56.379 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:56.379 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.379 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:56.377848+0000 2024-09-15T01:40:56.379 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:56.377848+0000 2024-09-15T01:40:56.379 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:56.379 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.379 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:56.379 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:56.379 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:56.379 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:56.380 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:56.380 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:56.380 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:56.380 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:56.380 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:56.380 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:56.380 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:56.380 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.377+0000 7fdacbe80640 1 Processor -- start 2024-09-15T01:40:56.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.377+0000 7fdacbe80640 1 -- start start 2024-09-15T01:40:56.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.377+0000 7fdacbe80640 10 monclient: init 2024-09-15T01:40:56.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:56.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 10 monclient: _add_conns ranks=[1,5,10,3,2,7,0,6,4,8,9] 2024-09-15T01:40:56.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fdac40a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:40:56.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdac4132de0 0x7fdac4137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 10 monclient(hunting): picked mon.noname-h con 0x7fdac4132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdac41380e0 0x7fdac413a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 10 monclient(hunting): picked mon.noname-f con 0x7fdac41380e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac404bf60 con 0x7fdac40a2930 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac409dd80 con 0x7fdac41380e0 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdac40a2d00 con 0x7fdac4132de0 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdacbe80640 10 monclient(hunting): authenticate will time out at 1261.128283s 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdac41380e0 0x7fdac413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdac941b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdac4132de0 0x7fdac4137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdac9c1c640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fdac40a2930 0x7fdac409faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:41912/0 (socket says 172.21.15.178:41912) 2024-09-15T01:40:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdac9c1c640 1 -- 172.21.15.178:0/1211646369 learned_addr learned my addr 172.21.15.178:0/1211646369 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:40:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdac941b640 10 monclient(hunting): get_auth_request con 0x7fdac4132de0 auth_method 0 2024-09-15T01:40:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdac941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdac941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdac941b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 10 monclient(hunting): get_auth_request con 0x7fdac41380e0 auth_method 0 2024-09-15T01:40:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdac941b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdac941b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdac941b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 10 monclient(hunting): handle_auth_done global_id 7912 payload 274 2024-09-15T01:40:56.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 1 -- 172.21.15.178:0/1211646369 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdac4132de0 msgr2=0x7fdac4137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:56.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 1 --2- 172.21.15.178:0/1211646369 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdac4132de0 0x7fdac4137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 1 -- 172.21.15.178:0/1211646369 >> v1:172.21.15.204:6789/0 conn(0x7fdac40a2930 legacy=0x7fdac409faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 10 monclient: _finish_hunting 0 2024-09-15T01:40:56.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 1 monclient: found mon.noname-f 2024-09-15T01:40:56.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:56.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdaca41d640 1 -- 172.21.15.178:0/1211646369 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac41331b0 con 0x7fdac41380e0 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.378+0000 7fdac941b640 1 --2- 172.21.15.178:0/1211646369 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdac4132de0 0x7fdac4137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdaca41d640 1 --2- 172.21.15.178:0/1211646369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac41380e0 0x7fdac413a4c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009e60 tx=0x7fdabc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=877cb4c37a8ea188 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdac8c1a640 1 -- 172.21.15.178:0/1211646369 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdabc0079d0 con 0x7fdac41380e0 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdac8c1a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdac8c1a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdac8c1a640 10 monclient: dump: 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.394 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdac8c1a640 10 monclient: _finish_auth 0 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdac8c1a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdac8c1a640 1 -- 172.21.15.178:0/1211646369 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc02d050 con 0x7fdac41380e0 2024-09-15T01:40:56.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdacbe80640 5 monclient: authenticate success, global_id 7912 2024-09-15T01:40:56.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdacbe80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:56.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdac8c1a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:56.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdac8c1a640 1 -- 172.21.15.178:0/1211646369 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdabc007bd0 con 0x7fdac41380e0 2024-09-15T01:40:56.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdac8c1a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdac8c1a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:40:56.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdac8c1a640 10 monclient: dump: 2024-09-15T01:40:56.396 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.396 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.396 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.396 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.396 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.396 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.397 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.397 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.397 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.397 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.397 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.397 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.397 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.397 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.397 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.397 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdacbe80640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:56.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.379+0000 7fdacbe80640 4 set_mon_vals no callback set 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.380+0000 7fdacbe80640 10 monclient: shutdown 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.380+0000 7fdacbe80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.380+0000 7fdacbe80640 1 -- 172.21.15.178:0/1211646369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac41380e0 msgr2=0x7fdac413a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.380+0000 7fdacbe80640 1 --2- 172.21.15.178:0/1211646369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac41380e0 0x7fdac413a4c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009e60 tx=0x7fdabc009780 comp rx=0 tx=0).stop 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.380+0000 7fdacbe80640 1 -- 172.21.15.178:0/1211646369 shutdown_connections 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.380+0000 7fdacbe80640 1 --2- 172.21.15.178:0/1211646369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac41380e0 0x7fdac413a4c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.380+0000 7fdacbe80640 1 --2- 172.21.15.178:0/1211646369 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdac4132de0 0x7fdac4137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.380+0000 7fdacbe80640 1 -- 172.21.15.178:0/1211646369 >> 172.21.15.178:0/1211646369 conn(0x7fdac40a5ed0 msgr2=0x7fdac40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.380+0000 7fdacbe80640 1 -- 172.21.15.178:0/1211646369 shutdown_connections 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.380+0000 7fdacbe80640 1 -- 172.21.15.178:0/1211646369 wait complete. 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient: build_initial_monmap 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient: monmap: 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:56.398 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:56.399 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 1 Processor -- start 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 1 -- start start 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient: init 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient: _add_conns ranks=[8,3,6,5,2,7,4,1,0] 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdac40a2930 0x7fdac409a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient(hunting): picked mon.noname-e con 0x7fdac40a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac4132de0 0x7fdac409aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient(hunting): picked mon.noname-g con 0x7fdac4132de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdac41380e0 0x7fdac409afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient(hunting): picked mon.noname-d con 0x7fdac41380e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac411e1e0 con 0x7fdac4132de0 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdac409c5d0 con 0x7fdac41380e0 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdac411dae0 con 0x7fdac40a2930 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdacbe80640 10 monclient(hunting): authenticate will time out at 1261.131926s 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.381+0000 7fdac9c1c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdac40a2930 0x7fdac409a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdaca41d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdac41380e0 0x7fdac409afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdac40a2930 0x7fdac409a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:46124/0 (socket says 172.21.15.178:46124) 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdaca41d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdac41380e0 0x7fdac409afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:51566/0 (socket says 172.21.15.178:51566) 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 1 -- 172.21.15.178:0/1732249880 learned_addr learned my addr 172.21.15.178:0/1732249880 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 10 monclient(hunting): get_auth_request con 0x7fdac40a2930 auth_method 0 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdaca41d640 10 monclient(hunting): get_auth_request con 0x7fdac41380e0 auth_method 0 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdaca41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdaca41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdaca41d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdaca41d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdaca41d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdaca41d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac941b640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac4132de0 0x7fdac409aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac941b640 10 monclient(hunting): get_auth_request con 0x7fdac4132de0 auth_method 0 2024-09-15T01:40:56.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac941b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac941b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac941b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac941b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 10 monclient(hunting): handle_auth_done global_id 34415 payload 824 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 1 -- 172.21.15.178:0/1732249880 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdac41380e0 msgr2=0x7fdac409afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdac41380e0 0x7fdac409afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 1 -- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac4132de0 msgr2=0x7fdac409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdac4132de0 0x7fdac409aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 10 monclient: _finish_hunting 0 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 1 monclient: found mon.noname-e 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 1 -- 172.21.15.178:0/1732249880 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab4009390 con 0x7fdac40a2930 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdaca41d640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdac41380e0 0x7fdac409afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdac9c1c640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdac40a2930 0x7fdac409a530 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c590 tx=0x7fdac0007f40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdac00070d0 con 0x7fdac40a2930 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdab9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdab9ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdab9ffb640 10 monclient: dump: 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdab9ffb640 10 monclient: _finish_auth 0 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdab9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac0007270 con 0x7fdac40a2930 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdacbe80640 5 monclient: authenticate success, global_id 34415 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdacbe80640 10 monclient: _renew_subs 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdacbe80640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdacbe80640 1 -- 172.21.15.178:0/1732249880 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc009a20 con 0x7fdac40a2930 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdab9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdac0011360 con 0x7fdac40a2930 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdab9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdab9ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdab9ffb640 10 monclient: dump: 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdacbe80640 10 monclient: _renew_subs 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdabbfff640 4 set_mon_vals no callback set 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdacbe80640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.382+0000 7fdacbe80640 1 -- 172.21.15.178:0/1732249880 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac409d630 con 0x7fdac40a2930 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.383+0000 7fdacbe80640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.383+0000 7fdacbe80640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac409b4f0 0x7fdac4096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.383+0000 7fdacbe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.383+0000 7fdacbe80640 1 -- 172.21.15.178:0/1732249880 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac404c4a0 con 0x7fdac409b4f0 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.383+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdac0007420 con 0x7fdac40a2930 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.383+0000 7fdac941b640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac409b4f0 0x7fdac4096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdab9ffb640 1 --2- 172.21.15.178:0/1732249880 >> v2:172.21.15.204:6812/28210 conn(0x7fdaa80397b0 0x7fdaa803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdac941b640 10 monclient: get_auth_request con 0x7fdac409b4f0 auth_method 0 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdac941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdac941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdac941b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fdac004ac80 con 0x7fdac40a2930 2024-09-15T01:40:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdac941b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdac941b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdac941b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdaca41d640 1 --2- 172.21.15.178:0/1732249880 >> v2:172.21.15.204:6812/28210 conn(0x7fdaa80397b0 0x7fdaa803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdaca41d640 10 monclient: get_auth_request con 0x7fdaa80397b0 auth_method 0 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdac941b640 10 monclient(hunting): handle_auth_done global_id 7930 payload 824 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdac941b640 1 -- 172.21.15.178:0/1732249880 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdac405ed70 con 0x7fdac409b4f0 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdac941b640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac409b4f0 0x7fdac4096a30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdab4011bf0 tx=0x7fdab40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdab400fdb0 con 0x7fdac409b4f0 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdab9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdab9ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdab9ffb640 10 monclient: dump: 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdaca41d640 1 --2- 172.21.15.178:0/1732249880 >> v2:172.21.15.204:6812/28210 conn(0x7fdaa80397b0 0x7fdaa803bc60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdabc034000 tx=0x7fdabc037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fdab401d480 con 0x7fdac409b4f0 2024-09-15T01:40:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdab9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdab9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac409b4f0 msgr2=0x7fdac4096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.384+0000 7fdab9ffb640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac409b4f0 0x7fdac4096a30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdab4011bf0 tx=0x7fdab40090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.385+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac409b4f0 msgr2=0x7fdac4096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.385+0000 7fdab9ffb640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac409b4f0 0x7fdac4096a30 secure :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdab4011bf0 tx=0x7fdab40090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.390+0000 7fdacbe80640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:56.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.390+0000 7fdacbe80640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac4096f70 0x7fdac4097370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.390+0000 7fdacbe80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.390+0000 7fdacbe80640 1 -- 172.21.15.178:0/1732249880 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac411d5c0 con 0x7fdac4096f70 2024-09-15T01:40:56.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.390+0000 7fdac941b640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac4096f70 0x7fdac4097370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.390+0000 7fdac941b640 10 monclient: get_auth_request con 0x7fdac4096f70 auth_method 0 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.390+0000 7fdac941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.390+0000 7fdac941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.390+0000 7fdac941b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.390+0000 7fdac941b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.390+0000 7fdac941b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.390+0000 7fdac941b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdac941b640 10 monclient(hunting): handle_auth_done global_id 7939 payload 824 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdac941b640 1 -- 172.21.15.178:0/1732249880 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdac404bc80 con 0x7fdac4096f70 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdac941b640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac4096f70 0x7fdac4097370 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdab4011bf0 tx=0x7fdab401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdab4025040 con 0x7fdac4096f70 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdab9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdab9ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdab9ffb640 10 monclient: dump: 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.415 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.416 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.416 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.416 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9314 (secure 0 0 0) 0x7fdab401c3e0 con 0x7fdac4096f70 2024-09-15T01:40:56.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdab9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:56.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdab9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:56.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac4096f70 msgr2=0x7fdac4097370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdab9ffb640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac4096f70 0x7fdac4097370 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdab4011bf0 tx=0x7fdab401d000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdab9ffb640 1 -- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac4096f70 msgr2=0x7fdac4097370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:56.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.391+0000 7fdab9ffb640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdac4096f70 0x7fdac4097370 secure :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdab4011bf0 tx=0x7fdab401d000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.416 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:40:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:56.417 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.418 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.419 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:56.420 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.421 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.422 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.422 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.422 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.422 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:56.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.422 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:56.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.422 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:56.422 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:56.422 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:56.423 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:56.426 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:56.428 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:56.429 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.431 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.432 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:56.433 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.435 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.436 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.437 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.438 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.439 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:40:56.439 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.439 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.439 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:56.439 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.392+0000 7fdacbe80640 1 -- 172.21.15.178:0/1732249880 >> v2:172.21.15.204:6812/28210 conn(0x7fdaa80397b0 msgr2=0x7fdaa803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.392+0000 7fdacbe80640 1 --2- 172.21.15.178:0/1732249880 >> v2:172.21.15.204:6812/28210 conn(0x7fdaa80397b0 0x7fdaa803bc60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdabc034000 tx=0x7fdabc037000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.392+0000 7fdacbe80640 10 monclient: shutdown 2024-09-15T01:40:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.392+0000 7fdacbe80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.392+0000 7fdacbe80640 1 -- 172.21.15.178:0/1732249880 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdac40a2930 msgr2=0x7fdac409a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.392+0000 7fdacbe80640 1 --2- 172.21.15.178:0/1732249880 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdac40a2930 0x7fdac409a530 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c590 tx=0x7fdac0007f40 comp rx=0 tx=0).stop 2024-09-15T01:40:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.392+0000 7fdaca41d640 1 -- 172.21.15.178:0/1732249880 reap_dead start 2024-09-15T01:40:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.392+0000 7fdaca41d640 1 -- 172.21.15.178:0/1732249880 reap_dead start 2024-09-15T01:40:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.392+0000 7fdacbe80640 1 -- 172.21.15.178:0/1732249880 shutdown_connections 2024-09-15T01:40:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.392+0000 7fdacbe80640 1 -- 172.21.15.178:0/1732249880 >> 172.21.15.178:0/1732249880 conn(0x7fdac40a5ed0 msgr2=0x7fdac40a4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.392+0000 7fdacbe80640 1 -- 172.21.15.178:0/1732249880 shutdown_connections 2024-09-15T01:40:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.392+0000 7fdacbe80640 1 -- 172.21.15.178:0/1732249880 wait complete. 2024-09-15T01:40:56.442 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:56.442 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:40:56.442 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:40:56.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.547+0000 7f2e56c6f640 10 monclient: get_monmap_and_config 2024-09-15T01:40:56.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.547+0000 7f2e56c6f640 10 monclient: build_initial_monmap 2024-09-15T01:40:56.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.547+0000 7f2e56c6f640 10 monclient: monmap: 2024-09-15T01:40:56.549 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:56.549 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.549 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:56.548433+0000 2024-09-15T01:40:56.549 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:56.548433+0000 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:56.550 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 1 Processor -- start 2024-09-15T01:40:56.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 1 -- start start 2024-09-15T01:40:56.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 10 monclient: init 2024-09-15T01:40:56.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:56.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 10 monclient: _add_conns ranks=[9,4,2,1,10,6,7,8,3,5,0] 2024-09-15T01:40:56.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2e500a2930 0x7f2e50132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 10 monclient(hunting): picked mon.noname-a con 0x7f2e500a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2e5009faa0 0x7f2e5009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 10 monclient(hunting): picked mon.noname-c con 0x7f2e5009faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e500a08c0 0x7f2e500a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 10 monclient(hunting): picked mon.noname-b con 0x7f2e500a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5009e0f0 con 0x7f2e500a2930 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5009dd80 con 0x7f2e500a08c0 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e500a2d00 con 0x7f2e5009faa0 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e56c6f640 10 monclient(hunting): authenticate will time out at 1261.298779s 2024-09-15T01:40:56.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e54a0b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2e500a2930 0x7f2e50132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e5520c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e500a08c0 0x7f2e500a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e54a0b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2e500a2930 0x7f2e50132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51968/0 (socket says 172.21.15.178:51968) 2024-09-15T01:40:56.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e54a0b640 1 -- 172.21.15.178:0/2873104582 learned_addr learned my addr 172.21.15.178:0/2873104582 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:56.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e4ffff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2e5009faa0 0x7f2e5009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e54a0b640 10 monclient(hunting): get_auth_request con 0x7f2e500a2930 auth_method 0 2024-09-15T01:40:56.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e54a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e54a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e54a0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e5520c640 10 monclient(hunting): get_auth_request con 0x7f2e500a08c0 auth_method 0 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e5520c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e5520c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e5520c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e4ffff640 10 monclient(hunting): get_auth_request con 0x7f2e5009faa0 auth_method 0 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.548+0000 7f2e4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e5520c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e5520c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e5520c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e54a0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e54a0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e54a0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 10 monclient(hunting): handle_auth_done global_id 44158 payload 274 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 1 -- 172.21.15.178:0/2873104582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e500a08c0 msgr2=0x7f2e500a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 1 --2- 172.21.15.178:0/2873104582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e500a08c0 0x7f2e500a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 1 -- 172.21.15.178:0/2873104582 >> v2:172.21.15.178:3300/0 conn(0x7f2e500a2930 msgr2=0x7f2e50132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 1 --2- 172.21.15.178:0/2873104582 >> v2:172.21.15.178:3300/0 conn(0x7f2e500a2930 0x7f2e50132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 1 monclient: found mon.noname-c 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 1 -- 172.21.15.178:0/2873104582 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5004bf60 con 0x7f2e5009faa0 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e5520c640 1 --2- 172.21.15.178:0/2873104582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e500a08c0 0x7f2e500a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:56.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.549+0000 7f2e4ffff640 1 --2- 172.21.15.178:0/2873104582 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2e5009faa0 0x7f2e5009fe70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009850 tx=0x7f2e3c0090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=610cb2678cf83436 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e4f7fe640 1 -- 172.21.15.178:0/2873104582 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e3c02abc0 con 0x7f2e5009faa0 2024-09-15T01:40:56.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e4f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e4f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:40:56.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e4f7fe640 10 monclient: dump: 2024-09-15T01:40:56.564 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.564 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.564 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.564 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.564 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.564 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.564 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e4f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e4f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e4f7fe640 1 -- 172.21.15.178:0/2873104582 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c02ad40 con 0x7f2e5009faa0 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 5 monclient: authenticate success, global_id 44158 2024-09-15T01:40:56.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e4f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e4f7fe640 1 -- 172.21.15.178:0/2873104582 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e3c031ca0 con 0x7f2e5009faa0 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e4f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e4f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e4f7fe640 10 monclient: dump: 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.566 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 4 set_mon_vals no callback set 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 10 monclient: shutdown 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 1 -- 172.21.15.178:0/2873104582 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2e5009faa0 msgr2=0x7f2e5009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 1 --2- 172.21.15.178:0/2873104582 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2e5009faa0 0x7f2e5009fe70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009850 tx=0x7f2e3c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 1 -- 172.21.15.178:0/2873104582 shutdown_connections 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 1 --2- 172.21.15.178:0/2873104582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e500a08c0 0x7f2e500a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 1 --2- 172.21.15.178:0/2873104582 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2e5009faa0 0x7f2e5009fe70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 1 --2- 172.21.15.178:0/2873104582 >> v2:172.21.15.178:3300/0 conn(0x7f2e500a2930 0x7f2e50132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 1 -- 172.21.15.178:0/2873104582 >> 172.21.15.178:0/2873104582 conn(0x7f2e500a5ed0 msgr2=0x7f2e500a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 1 -- 172.21.15.178:0/2873104582 shutdown_connections 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.550+0000 7f2e56c6f640 1 -- 172.21.15.178:0/2873104582 wait complete. 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.551+0000 7f2e56c6f640 10 monclient: build_initial_monmap 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.551+0000 7f2e56c6f640 10 monclient: monmap: 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:56.568 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.551+0000 7f2e56c6f640 1 Processor -- start 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.551+0000 7f2e56c6f640 1 -- start start 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.551+0000 7f2e56c6f640 10 monclient: init 2024-09-15T01:40:56.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient: _add_conns ranks=[5,2,4,1,8,7,3,0,6] 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2e5009faa0 0x7f2e5009b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient(hunting): picked mon.noname-h con 0x7f2e5009faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e500a08c0 0x7f2e50131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient(hunting): picked mon.noname-b con 0x7f2e500a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2e500a2930 0x7f2e5011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient(hunting): picked mon.noname-c con 0x7f2e500a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5011ef20 con 0x7f2e500a08c0 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5009bff0 con 0x7f2e500a2930 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5011ec40 con 0x7f2e5009faa0 2024-09-15T01:40:56.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient(hunting): authenticate will time out at 1261.302163s 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e500a08c0 0x7f2e50131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2e500a2930 0x7f2e5011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e500a08c0 0x7f2e50131840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55806/0 (socket says 172.21.15.178:55806) 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4ffff640 1 -- 172.21.15.178:0/247648452 learned_addr learned my addr 172.21.15.178:0/247648452 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 10 monclient(hunting): get_auth_request con 0x7f2e500a2930 auth_method 0 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4ffff640 10 monclient(hunting): get_auth_request con 0x7f2e500a08c0 auth_method 0 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e54a0b640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2e5009faa0 0x7f2e5009b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e54a0b640 10 monclient(hunting): get_auth_request con 0x7f2e5009faa0 auth_method 0 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e54a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e54a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e54a0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 10 monclient(hunting): handle_auth_done global_id 44167 payload 824 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 1 -- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2e5009faa0 msgr2=0x7f2e5009b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f2e5009faa0 0x7f2e5009b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 1 -- 172.21.15.178:0/247648452 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e500a08c0 msgr2=0x7f2e50131840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e500a08c0 0x7f2e50131840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 10 monclient: _finish_hunting 0 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 1 monclient: found mon.noname-c 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 1 -- 172.21.15.178:0/247648452 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e48009a20 con 0x7f2e500a2930 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4ffff640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e500a08c0 0x7f2e50131840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e5520c640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2e500a2930 0x7f2e5011c530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009f40 tx=0x7f2e4800b040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e4800ce10 con 0x7f2e500a2930 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4cff9640 10 monclient: dump: 2024-09-15T01:40:56.573 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.574 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4cff9640 10 monclient: _finish_auth 0 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4800f040 con 0x7f2e500a2930 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 5 monclient: authenticate success, global_id 44167 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient: _renew_subs 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 1 -- 172.21.15.178:0/247648452 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c009390 con 0x7f2e500a2930 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e48014d80 con 0x7f2e500a2930 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4cff9640 10 monclient: dump: 2024-09-15T01:40:56.575 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.576 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient: _renew_subs 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e4effd640 4 set_mon_vals no callback set 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.552+0000 7f2e56c6f640 1 -- 172.21.15.178:0/247648452 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5011e960 con 0x7f2e500a2930 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.553+0000 7f2e56c6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.553+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2e48003020 con 0x7f2e500a2930 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.553+0000 7f2e56c6f640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e50131d80 0x7f2e5011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.553+0000 7f2e56c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.553+0000 7f2e56c6f640 1 -- 172.21.15.178:0/247648452 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e50132150 con 0x7f2e50131d80 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.553+0000 7f2e4cff9640 1 --2- 172.21.15.178:0/247648452 >> v2:172.21.15.204:6812/28210 conn(0x7f2e340397b0 0x7f2e3403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.553+0000 7f2e54a0b640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e50131d80 0x7f2e5011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.553+0000 7f2e54a0b640 10 monclient: get_auth_request con 0x7f2e50131d80 auth_method 0 2024-09-15T01:40:56.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.553+0000 7f2e54a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.553+0000 7f2e54a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.553+0000 7f2e54a0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.553+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f2e4804c730 con 0x7f2e500a2930 2024-09-15T01:40:56.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e54a0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e54a0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e54a0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4ffff640 1 --2- 172.21.15.178:0/247648452 >> v2:172.21.15.204:6812/28210 conn(0x7f2e340397b0 0x7f2e3403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4ffff640 10 monclient: get_auth_request con 0x7f2e340397b0 auth_method 0 2024-09-15T01:40:56.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e54a0b640 10 monclient(hunting): handle_auth_done global_id 16077 payload 824 2024-09-15T01:40:56.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e54a0b640 1 -- 172.21.15.178:0/247648452 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e5004c4a0 con 0x7f2e50131d80 2024-09-15T01:40:56.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e54a0b640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e50131d80 0x7f2e5011ca70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2e44011840 tx=0x7f2e44008040 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4ffff640 1 --2- 172.21.15.178:0/247648452 >> v2:172.21.15.204:6812/28210 conn(0x7f2e340397b0 0x7f2e3403bc60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009720 tx=0x7f2e3c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e4400fdc0 con 0x7f2e50131d80 2024-09-15T01:40:56.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4cff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:40:56.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4cff9640 10 monclient: dump: 2024-09-15T01:40:56.580 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.580 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.581 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.582 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f2e4400f940 con 0x7f2e50131d80 2024-09-15T01:40:56.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:56.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:56.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e50131d80 msgr2=0x7f2e5011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4cff9640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e50131d80 0x7f2e5011ca70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2e44011840 tx=0x7f2e44008040 comp rx=0 tx=0).stop 2024-09-15T01:40:56.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e50131d80 msgr2=0x7f2e5011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:56.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.554+0000 7f2e4cff9640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e50131d80 0x7f2e5011ca70 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2e44011840 tx=0x7f2e44008040 comp rx=0 tx=0).stop 2024-09-15T01:40:56.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.560+0000 7f2e56c6f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:56.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.560+0000 7f2e56c6f640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e5011cfb0 0x7f2e50096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.560+0000 7f2e56c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.560+0000 7f2e56c6f640 1 -- 172.21.15.178:0/247648452 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5004bc80 con 0x7f2e5011cfb0 2024-09-15T01:40:56.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.560+0000 7f2e54a0b640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e5011cfb0 0x7f2e50096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.560+0000 7f2e54a0b640 10 monclient: get_auth_request con 0x7f2e5011cfb0 auth_method 0 2024-09-15T01:40:56.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.560+0000 7f2e54a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.560+0000 7f2e54a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.560+0000 7f2e54a0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.560+0000 7f2e54a0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.560+0000 7f2e54a0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.560+0000 7f2e54a0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e54a0b640 10 monclient(hunting): handle_auth_done global_id 16086 payload 824 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e54a0b640 1 -- 172.21.15.178:0/247648452 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2e5005ed70 con 0x7f2e5011cfb0 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e54a0b640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e5011cfb0 0x7f2e50096a30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2e44000c00 tx=0x7f2e4401d000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e44006660 con 0x7f2e5011cfb0 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e4cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e4cff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e4cff9640 10 monclient: dump: 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.585 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f2e4400f6d0 con 0x7f2e5011cfb0 2024-09-15T01:40:56.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e4cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:56.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e4cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:56.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e5011cfb0 msgr2=0x7f2e50096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e4cff9640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e5011cfb0 0x7f2e50096a30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2e44000c00 tx=0x7f2e4401d000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e4cff9640 1 -- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e5011cfb0 msgr2=0x7f2e50096a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:56.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.561+0000 7f2e4cff9640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e5011cfb0 0x7f2e50096a30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.587 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:56.587 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:56.587 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:56.587 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:56.587 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:56.588 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:56.588 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:56.588 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:56.588 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:56.588 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:56.588 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:56.588 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:56.588 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:56.588 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.589 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.590 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:56.591 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.594 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.595 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.596 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.597 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.598 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:56.599 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:56.603 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:56.605 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:56.606 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.606 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.606 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.606 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:56.606 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.608 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.609 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.610 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:56.610 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.610 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.610 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:56.610 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:56.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.562+0000 7f2e56c6f640 1 -- 172.21.15.178:0/247648452 >> v2:172.21.15.204:6812/28210 conn(0x7f2e340397b0 msgr2=0x7f2e3403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.562+0000 7f2e56c6f640 1 --2- 172.21.15.178:0/247648452 >> v2:172.21.15.204:6812/28210 conn(0x7f2e340397b0 0x7f2e3403bc60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009720 tx=0x7f2e3c032000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.562+0000 7f2e56c6f640 10 monclient: shutdown 2024-09-15T01:40:56.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.562+0000 7f2e56c6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:56.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.562+0000 7f2e56c6f640 1 -- 172.21.15.178:0/247648452 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2e500a2930 msgr2=0x7f2e5011c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.562+0000 7f2e56c6f640 1 --2- 172.21.15.178:0/247648452 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2e500a2930 0x7f2e5011c530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009f40 tx=0x7f2e4800b040 comp rx=0 tx=0).stop 2024-09-15T01:40:56.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.562+0000 7f2e5520c640 1 -- 172.21.15.178:0/247648452 reap_dead start 2024-09-15T01:40:56.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.562+0000 7f2e5520c640 1 -- 172.21.15.178:0/247648452 reap_dead start 2024-09-15T01:40:56.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.562+0000 7f2e56c6f640 1 -- 172.21.15.178:0/247648452 shutdown_connections 2024-09-15T01:40:56.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.562+0000 7f2e56c6f640 1 -- 172.21.15.178:0/247648452 >> 172.21.15.178:0/247648452 conn(0x7f2e500a5ed0 msgr2=0x7f2e50135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:56.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.562+0000 7f2e56c6f640 1 -- 172.21.15.178:0/247648452 shutdown_connections 2024-09-15T01:40:56.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.562+0000 7f2e56c6f640 1 -- 172.21.15.178:0/247648452 wait complete. 2024-09-15T01:40:56.612 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:56.613 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:40:56.613 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:40:56.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.705+0000 7ff46ea20640 10 monclient: get_monmap_and_config 2024-09-15T01:40:56.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.705+0000 7ff46ea20640 10 monclient: build_initial_monmap 2024-09-15T01:40:56.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.705+0000 7ff46ea20640 10 monclient: monmap: 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:56.706612+0000 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:56.706612+0000 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:56.708 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:56.709 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:56.709 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:56.709 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:56.709 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 1 Processor -- start 2024-09-15T01:40:56.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 1 -- start start 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 10 monclient: init 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 10 monclient: _add_conns ranks=[6,4,2,10,7,9,8,1,5,0,3] 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4680a2930 0x7ff468132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 10 monclient(hunting): picked mon.noname-d con 0x7ff4680a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff46809faa0 0x7ff46809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 10 monclient(hunting): picked mon.noname-c con 0x7ff46809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 0x7ff4680a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 10 monclient(hunting): picked mon.noname-b con 0x7ff4680a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46809e0f0 con 0x7ff4680a08c0 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff46809dd80 con 0x7ff46809faa0 2024-09-15T01:40:56.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680a2d00 con 0x7ff4680a2930 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.706+0000 7ff46ea20640 10 monclient(hunting): authenticate will time out at 1261.456994s 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff467fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4680a2930 0x7ff468132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff4677fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff46809faa0 0x7ff46809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 0x7ff4680a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff467fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4680a2930 0x7ff468132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:51578/0 (socket says 172.21.15.178:51578) 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff4677fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff46809faa0 0x7ff46809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:32942/0 (socket says 172.21.15.178:32942) 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 10 monclient(hunting): get_auth_request con 0x7ff4680a08c0 auth_method 0 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff467fff640 1 -- 172.21.15.178:0/36434299 learned_addr learned my addr 172.21.15.178:0/36434299 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff467fff640 10 monclient(hunting): get_auth_request con 0x7ff4680a2930 auth_method 0 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff467fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff467fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff467fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff4677fe640 10 monclient(hunting): get_auth_request con 0x7ff46809faa0 auth_method 0 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff4677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff4677fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff4677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff467fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff467fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff467fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff4677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff4677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff4677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 10 monclient(hunting): handle_auth_done global_id 34436 payload 274 2024-09-15T01:40:56.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 1 -- 172.21.15.178:0/36434299 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4680a2930 msgr2=0x7ff468132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:56.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 1 --2- 172.21.15.178:0/36434299 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4680a2930 0x7ff468132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 1 -- 172.21.15.178:0/36434299 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff46809faa0 msgr2=0x7ff46809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 1 --2- 172.21.15.178:0/36434299 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff46809faa0 0x7ff46809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 10 monclient: _finish_hunting 0 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 1 monclient: found mon.noname-b 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff46cfbd640 1 -- 172.21.15.178:0/36434299 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46804bf60 con 0x7ff4680a08c0 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff467fff640 1 --2- 172.21.15.178:0/36434299 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4680a2930 0x7ff468132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.707+0000 7ff4677fe640 1 --2- 172.21.15.178:0/36434299 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff46809faa0 0x7ff46809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46cfbd640 1 --2- 172.21.15.178:0/36434299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 0x7ff4680a0d30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff460009e90 tx=0x7ff460009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6e082ec7fe84ac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff466ffd640 1 -- 172.21.15.178:0/36434299 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4600079d0 con 0x7ff4680a08c0 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff466ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff466ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:40:56.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff466ffd640 10 monclient: dump: 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.724 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff466ffd640 10 monclient: _finish_auth 0 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff466ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff466ffd640 1 -- 172.21.15.178:0/36434299 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46002d050 con 0x7ff4680a08c0 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 5 monclient: authenticate success, global_id 34436 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff466ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff466ffd640 1 -- 172.21.15.178:0/36434299 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff460007bd0 con 0x7ff4680a08c0 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 4 set_mon_vals no callback set 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff466ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff466ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff466ffd640 10 monclient: dump: 2024-09-15T01:40:56.725 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.726 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 10 monclient: shutdown 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 1 -- 172.21.15.178:0/36434299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 msgr2=0x7ff4680a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 1 --2- 172.21.15.178:0/36434299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 0x7ff4680a0d30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff460009e90 tx=0x7ff460009780 comp rx=0 tx=0).stop 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 1 -- 172.21.15.178:0/36434299 shutdown_connections 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 1 --2- 172.21.15.178:0/36434299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 0x7ff4680a0d30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 1 --2- 172.21.15.178:0/36434299 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff46809faa0 0x7ff46809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 1 --2- 172.21.15.178:0/36434299 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4680a2930 0x7ff468132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 1 -- 172.21.15.178:0/36434299 >> 172.21.15.178:0/36434299 conn(0x7ff4680a5ed0 msgr2=0x7ff4680a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 1 -- 172.21.15.178:0/36434299 shutdown_connections 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.708+0000 7ff46ea20640 1 -- 172.21.15.178:0/36434299 wait complete. 2024-09-15T01:40:56.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.709+0000 7ff46ea20640 10 monclient: build_initial_monmap 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.709+0000 7ff46ea20640 10 monclient: monmap: 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:56.728 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.709+0000 7ff46ea20640 1 Processor -- start 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.709+0000 7ff46ea20640 1 -- start start 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.709+0000 7ff46ea20640 10 monclient: init 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 10 monclient: _add_conns ranks=[5,2,4,0,3,7,8,6,1] 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff46809faa0 0x7ff46809b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 10 monclient(hunting): picked mon.noname-h con 0x7ff46809faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 0x7ff4681318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 10 monclient(hunting): picked mon.noname-b con 0x7ff4680a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:40:56.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff4680a2930 0x7ff46811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 10 monclient(hunting): picked mon.noname-c con 0x7ff4680a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46809d630 con 0x7ff4680a08c0 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff46811ecb0 con 0x7ff4680a2930 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff46811e9d0 con 0x7ff46809faa0 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46ea20640 10 monclient(hunting): authenticate will time out at 1261.460431s 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 0x7ff4681318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 0x7ff4681318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55826/0 (socket says 172.21.15.178:55826) 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 1 -- 172.21.15.178:0/3497554081 learned_addr learned my addr 172.21.15.178:0/3497554081 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:56.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 10 monclient(hunting): get_auth_request con 0x7ff4680a08c0 auth_method 0 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46cfbd640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff4680a2930 0x7ff46811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46cfbd640 10 monclient(hunting): get_auth_request con 0x7ff4680a2930 auth_method 0 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46cfbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46cfbd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46cfbd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46cfbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46cfbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff46cfbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 10 monclient(hunting): handle_auth_done global_id 34445 payload 824 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff467fff640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff46809faa0 0x7ff46809b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 1 -- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff46809faa0 msgr2=0x7ff46809b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff46809faa0 0x7ff46809b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 1 -- 172.21.15.178:0/3497554081 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff4680a2930 msgr2=0x7ff46811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff4680a2930 0x7ff46811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 10 monclient: _finish_hunting 0 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 1 monclient: found mon.noname-b 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.710+0000 7ff4677fe640 1 -- 172.21.15.178:0/3497554081 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c009390 con 0x7ff4680a08c0 2024-09-15T01:40:56.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff4677fe640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 0x7ff4681318b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00ea20 tx=0x7ff45c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff45c00ce90 con 0x7ff4680a08c0 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff44ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff44ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff44ffff640 10 monclient: dump: 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.733 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff44ffff640 10 monclient: _finish_auth 0 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff44ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff46ea20640 5 monclient: authenticate success, global_id 34445 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c009650 con 0x7ff4680a08c0 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff46ea20640 10 monclient: _renew_subs 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff46ea20640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff46ea20640 1 -- 172.21.15.178:0/3497554081 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff458009450 con 0x7ff4680a08c0 2024-09-15T01:40:56.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff44ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff45c0138f0 con 0x7ff4680a08c0 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff44ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff44ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff44ffff640 10 monclient: dump: 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.735 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff46ea20640 10 monclient: _renew_subs 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff46ea20640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff4667fc640 4 set_mon_vals no callback set 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff46ea20640 1 -- 172.21.15.178:0/3497554081 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff460009a20 con 0x7ff4680a08c0 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.711+0000 7ff46ea20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff46ea20640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff46811da70 0x7ff46811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff46ea20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff46ea20640 1 -- 172.21.15.178:0/3497554081 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff46811eb40 con 0x7ff46811da70 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff45c013470 con 0x7ff4680a08c0 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff467fff640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff46811da70 0x7ff46811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff44ffff640 1 --2- 172.21.15.178:0/3497554081 >> v2:172.21.15.204:6812/28210 conn(0x7ff4540397e0 0x7ff45403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff467fff640 10 monclient: get_auth_request con 0x7ff46811da70 auth_method 0 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff467fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff467fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff467fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7ff45c04bec0 con 0x7ff4680a08c0 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff467fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff467fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff467fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff46cfbd640 1 --2- 172.21.15.178:0/3497554081 >> v2:172.21.15.204:6812/28210 conn(0x7ff4540397e0 0x7ff45403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff46cfbd640 10 monclient: get_auth_request con 0x7ff4540397e0 auth_method 0 2024-09-15T01:40:56.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff467fff640 10 monclient(hunting): handle_auth_done global_id 6881 payload 824 2024-09-15T01:40:56.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff467fff640 1 -- 172.21.15.178:0/3497554081 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff46804c4a0 con 0x7ff46811da70 2024-09-15T01:40:56.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff467fff640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff46811da70 0x7ff46811ca70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff458010900 tx=0x7ff458008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.712+0000 7ff46cfbd640 1 --2- 172.21.15.178:0/3497554081 >> v2:172.21.15.204:6812/28210 conn(0x7ff4540397e0 0x7ff45403bc90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff460034000 tx=0x7ff460037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.713+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff45801c400 con 0x7ff46811da70 2024-09-15T01:40:56.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.713+0000 7ff44ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.713+0000 7ff44ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:40:56.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.713+0000 7ff44ffff640 10 monclient: dump: 2024-09-15T01:40:56.738 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.738 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.738 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.713+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7ff45800eae0 con 0x7ff46811da70 2024-09-15T01:40:56.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.713+0000 7ff44ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:56.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.713+0000 7ff44ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:56.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.713+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff46811da70 msgr2=0x7ff46811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.713+0000 7ff44ffff640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff46811da70 0x7ff46811ca70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff458010900 tx=0x7ff458008040 comp rx=0 tx=0).stop 2024-09-15T01:40:56.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.713+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff46811da70 msgr2=0x7ff46811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:56.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.713+0000 7ff44ffff640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff46811da70 0x7ff46811ca70 secure :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff458010900 tx=0x7ff458008040 comp rx=0 tx=0).stop 2024-09-15T01:40:56.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.718+0000 7ff46ea20640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:56.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.718+0000 7ff46ea20640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff468131df0 0x7ff46811cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.718+0000 7ff46ea20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.718+0000 7ff46ea20640 1 -- 172.21.15.178:0/3497554081 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff46804bc80 con 0x7ff468131df0 2024-09-15T01:40:56.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff467fff640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff468131df0 0x7ff46811cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff467fff640 10 monclient: get_auth_request con 0x7ff468131df0 auth_method 0 2024-09-15T01:40:56.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff467fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff467fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff467fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff467fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff467fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff467fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff467fff640 10 monclient(hunting): handle_auth_done global_id 6890 payload 824 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff467fff640 1 -- 172.21.15.178:0/3497554081 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff46805ed70 con 0x7ff468131df0 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff467fff640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff468131df0 0x7ff46811cfb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff45801b000 tx=0x7ff45801c000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff458014b50 con 0x7ff468131df0 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff44ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff44ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.719+0000 7ff44ffff640 10 monclient: dump: 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.743 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.744 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7ff45800e660 con 0x7ff468131df0 2024-09-15T01:40:56.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff44ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:56.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff44ffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:56.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff468131df0 msgr2=0x7ff46811cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff44ffff640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff468131df0 0x7ff46811cfb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff45801b000 tx=0x7ff45801c000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff44ffff640 1 -- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff468131df0 msgr2=0x7ff46811cfb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:56.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff44ffff640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff468131df0 0x7ff46811cfb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.745 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:56.745 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:56.745 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.746 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.747 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.748 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:56.749 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.750 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.751 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.752 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.753 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:56.755 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:56.756 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.757 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.758 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:56.759 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.761 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.762 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:56.762 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.762 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.762 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.762 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:56.762 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.762 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.762 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.763 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:56.764 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.766 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:56.768 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:56.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff46ea20640 1 -- 172.21.15.178:0/3497554081 >> v2:172.21.15.204:6812/28210 conn(0x7ff4540397e0 msgr2=0x7ff45403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff46ea20640 1 --2- 172.21.15.178:0/3497554081 >> v2:172.21.15.204:6812/28210 conn(0x7ff4540397e0 0x7ff45403bc90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff460034000 tx=0x7ff460037000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff46ea20640 10 monclient: shutdown 2024-09-15T01:40:56.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff46ea20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:56.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff46ea20640 1 -- 172.21.15.178:0/3497554081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 msgr2=0x7ff4681318b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff46ea20640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 0x7ff4681318b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00ea20 tx=0x7ff45c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff46cfbd640 1 -- 172.21.15.178:0/3497554081 reap_dead start 2024-09-15T01:40:56.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff46ea20640 1 -- 172.21.15.178:0/3497554081 shutdown_connections 2024-09-15T01:40:56.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff46ea20640 1 --2- 172.21.15.178:0/3497554081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4680a08c0 0x7ff4681318b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.720+0000 7ff46ea20640 1 -- 172.21.15.178:0/3497554081 >> 172.21.15.178:0/3497554081 conn(0x7ff4680a5ed0 msgr2=0x7ff46809e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:56.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.721+0000 7ff46ea20640 1 -- 172.21.15.178:0/3497554081 shutdown_connections 2024-09-15T01:40:56.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.721+0000 7ff46ea20640 1 -- 172.21.15.178:0/3497554081 wait complete. 2024-09-15T01:40:56.771 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:56.771 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:40:56.771 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:40:56.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.863+0000 7f08f6c6f640 10 monclient: get_monmap_and_config 2024-09-15T01:40:56.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.864+0000 7f08f6c6f640 10 monclient: build_initial_monmap 2024-09-15T01:40:56.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.864+0000 7f08f6c6f640 10 monclient: monmap: 2024-09-15T01:40:56.866 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:56.866 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.867 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:56.864904+0000 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:56.864904+0000 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:56.868 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:56.869 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:56.869 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.864+0000 7f08f6c6f640 1 Processor -- start 2024-09-15T01:40:56.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.864+0000 7f08f6c6f640 1 -- start start 2024-09-15T01:40:56.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.864+0000 7f08f6c6f640 10 monclient: init 2024-09-15T01:40:56.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:56.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 10 monclient: _add_conns ranks=[9,5,2,10,7,6,8,4,0,3,1] 2024-09-15T01:40:56.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f08f00a2930 0x7f08f0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 10 monclient(hunting): picked mon.noname-a con 0x7f08f00a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:40:56.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f08f009faa0 0x7f08f009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 10 monclient(hunting): picked mon.noname-h con 0x7f08f009faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:56.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f00a08c0 0x7f08f00a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 10 monclient(hunting): picked mon.noname-b con 0x7f08f00a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:40:56.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f009e0f0 con 0x7f08f00a2930 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f009dd80 con 0x7f08f00a08c0 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00a2d00 con 0x7f08f009faa0 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f6c6f640 10 monclient(hunting): authenticate will time out at 1261.615257s 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f08f00a2930 0x7f08f0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f520c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f00a08c0 0x7f08f00a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f08f00a2930 0x7f08f0132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51972/0 (socket says 172.21.15.178:51972) 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 1 -- 172.21.15.178:0/2698878546 learned_addr learned my addr 172.21.15.178:0/2698878546 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 10 monclient(hunting): get_auth_request con 0x7f08f00a2930 auth_method 0 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f520c640 10 monclient(hunting): get_auth_request con 0x7f08f00a08c0 auth_method 0 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f520c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f520c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f520c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f520c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f520c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f520c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08e7fff640 1 --2- 172.21.15.178:0/2698878546 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f08f009faa0 0x7f08f009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 10 monclient(hunting): handle_auth_done global_id 34560 payload 274 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 1 -- 172.21.15.178:0/2698878546 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f08f009faa0 msgr2=0x7f08f009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 1 --2- 172.21.15.178:0/2698878546 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f08f009faa0 0x7f08f009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 1 -- 172.21.15.178:0/2698878546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f00a08c0 msgr2=0x7f08f00a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 1 --2- 172.21.15.178:0/2698878546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f00a08c0 0x7f08f00a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 10 monclient: _finish_hunting 0 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 1 monclient: found mon.noname-a 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f4a0b640 1 -- 172.21.15.178:0/2698878546 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f004bf60 con 0x7f08f00a2930 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.865+0000 7f08f520c640 1 --2- 172.21.15.178:0/2698878546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f00a08c0 0x7f08f00a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08f4a0b640 1 --2- 172.21.15.178:0/2698878546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f00a2930 0x7f08f0132de0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009f20 tx=0x7f08ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0b779b2585e44a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08e77fe640 1 -- 172.21.15.178:0/2698878546 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08ec007880 con 0x7f08f00a2930 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08e77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08e77fe640 10 monclient: dump: 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.882 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08e77fe640 10 monclient: _finish_auth 0 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08f6c6f640 5 monclient: authenticate success, global_id 34560 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08e77fe640 1 -- 172.21.15.178:0/2698878546 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec02d050 con 0x7f08f00a2930 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08f6c6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08e77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08f6c6f640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08e77fe640 1 -- 172.21.15.178:0/2698878546 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08ec030360 con 0x7f08f00a2930 2024-09-15T01:40:56.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08f6c6f640 4 set_mon_vals no callback set 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08e77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.866+0000 7f08e77fe640 10 monclient: dump: 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.884 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 10 monclient: shutdown 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 1 -- 172.21.15.178:0/2698878546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f00a2930 msgr2=0x7f08f0132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 1 --2- 172.21.15.178:0/2698878546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f00a2930 0x7f08f0132de0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009f20 tx=0x7f08ec009840 comp rx=0 tx=0).stop 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 1 -- 172.21.15.178:0/2698878546 shutdown_connections 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 1 --2- 172.21.15.178:0/2698878546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08f00a08c0 0x7f08f00a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 1 --2- 172.21.15.178:0/2698878546 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f08f009faa0 0x7f08f009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 1 --2- 172.21.15.178:0/2698878546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08f00a2930 0x7f08f0132de0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 1 -- 172.21.15.178:0/2698878546 >> 172.21.15.178:0/2698878546 conn(0x7f08f00a5ed0 msgr2=0x7f08f00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 1 -- 172.21.15.178:0/2698878546 shutdown_connections 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 1 -- 172.21.15.178:0/2698878546 wait complete. 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 10 monclient: build_initial_monmap 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 10 monclient: monmap: 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:56.886 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:56.887 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:56.887 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:56.887 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:56.887 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:56.887 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:56.887 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:56.887 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:56.887 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:56.887 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.867+0000 7f08f6c6f640 1 Processor -- start 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 1 -- start start 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 10 monclient: init 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 10 monclient: _add_conns ranks=[6,4,7,0,5,1,2,8,3] 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f08f009faa0 0x7f08f009b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 10 monclient(hunting): picked mon.noname-d con 0x7f08f009faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f08f00a08c0 0x7f08f01317d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 10 monclient(hunting): picked mon.noname-c con 0x7f08f00a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f00a2930 0x7f08f011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 10 monclient(hunting): picked mon.noname-i con 0x7f08f00a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08f009d630 con 0x7f08f00a08c0 2024-09-15T01:40:56.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f08f011ebd0 con 0x7f08f009faa0 2024-09-15T01:40:56.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f08f009bb40 con 0x7f08f00a2930 2024-09-15T01:40:56.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:56.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f6c6f640 10 monclient(hunting): authenticate will time out at 1261.618628s 2024-09-15T01:40:56.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08e7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f08f00a08c0 0x7f08f01317d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f4a0b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f08f009faa0 0x7f08f009b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f4a0b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f08f009faa0 0x7f08f009b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:51584/0 (socket says 172.21.15.178:51584) 2024-09-15T01:40:56.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08e7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f08f00a08c0 0x7f08f01317d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:32966/0 (socket says 172.21.15.178:32966) 2024-09-15T01:40:56.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f4a0b640 1 -- 172.21.15.178:0/104747463 learned_addr learned my addr 172.21.15.178:0/104747463 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:56.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f4a0b640 10 monclient(hunting): get_auth_request con 0x7f08f009faa0 auth_method 0 2024-09-15T01:40:56.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f4a0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f4a0b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f4a0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08e7fff640 10 monclient(hunting): get_auth_request con 0x7f08f00a08c0 auth_method 0 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f4a0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f4a0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f520c640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f00a2930 0x7f08f011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f4a0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f520c640 10 monclient(hunting): get_auth_request con 0x7f08f00a2930 auth_method 0 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f520c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f520c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f520c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f520c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f520c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.868+0000 7f08f520c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f4a0b640 10 monclient(hunting): handle_auth_done global_id 24999 payload 824 2024-09-15T01:40:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f4a0b640 1 -- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f00a2930 msgr2=0x7f08f011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f4a0b640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f00a2930 0x7f08f011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f4a0b640 1 -- 172.21.15.178:0/104747463 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f08f00a08c0 msgr2=0x7f08f01317d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f4a0b640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f08f00a08c0 0x7f08f01317d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f4a0b640 10 monclient: _finish_hunting 0 2024-09-15T01:40:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f4a0b640 1 monclient: found mon.noname-d 2024-09-15T01:40:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f4a0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f4a0b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f4a0b640 1 -- 172.21.15.178:0/104747463 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e8009390 con 0x7f08f009faa0 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e7fff640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f08f00a08c0 0x7f08f01317d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f4a0b640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f08f009faa0 0x7f08f009b050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f08ec000f80 tx=0x7f08ec036000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08ec03baa0 con 0x7f08f009faa0 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e4ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e4ff9640 10 monclient: dump: 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.893 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e4ff9640 10 monclient: _finish_auth 0 2024-09-15T01:40:56.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f6c6f640 5 monclient: authenticate success, global_id 24999 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec03bc40 con 0x7f08f009faa0 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08ec03bdb0 con 0x7f08f009faa0 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f6c6f640 10 monclient: _renew_subs 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f6c6f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f6c6f640 1 -- 172.21.15.178:0/104747463 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec009ae0 con 0x7f08f009faa0 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e4ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e6ffd640 4 set_mon_vals no callback set 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08e4ff9640 10 monclient: dump: 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.895 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.896 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.897 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f6c6f640 10 monclient: _renew_subs 2024-09-15T01:40:56.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f6c6f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:40:56.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.869+0000 7f08f6c6f640 1 -- 172.21.15.178:0/104747463 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f01429a0 con 0x7f08f009faa0 2024-09-15T01:40:56.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08f6c6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:56.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08f6c6f640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f0131d10 0x7f08f011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f08ec038070 con 0x7f08f009faa0 2024-09-15T01:40:56.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08f6c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08f6c6f640 1 -- 172.21.15.178:0/104747463 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f08f011ea90 con 0x7f08f0131d10 2024-09-15T01:40:56.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e4ff9640 1 --2- 172.21.15.178:0/104747463 >> v2:172.21.15.204:6812/28210 conn(0x7f08dc0396e0 0x7f08dc03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f08ec06c110 con 0x7f08f009faa0 2024-09-15T01:40:56.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e7fff640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f0131d10 0x7f08f011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e7fff640 10 monclient: get_auth_request con 0x7f08f0131d10 auth_method 0 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08f520c640 1 --2- 172.21.15.178:0/104747463 >> v2:172.21.15.204:6812/28210 conn(0x7f08dc0396e0 0x7f08dc03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08f520c640 10 monclient: get_auth_request con 0x7f08dc0396e0 auth_method 0 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e7fff640 10 monclient(hunting): handle_auth_done global_id 44152 payload 824 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.870+0000 7f08e7fff640 1 -- 172.21.15.178:0/104747463 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08f004c4a0 con 0x7f08f0131d10 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08e7fff640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f0131d10 0x7f08f011ca70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f08f00a6ec0 tx=0x7f08e0008040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08f520c640 1 --2- 172.21.15.178:0/104747463 >> v2:172.21.15.204:6812/28210 conn(0x7f08dc0396e0 0x7f08dc03bb90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009e30 tx=0x7f08e80090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08e000de80 con 0x7f08f0131d10 2024-09-15T01:40:56.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08e4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08e4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:40:56.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08e4ff9640 10 monclient: dump: 2024-09-15T01:40:56.899 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.899 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.899 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.899 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.899 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.899 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.899 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.899 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f08e000c040 con 0x7f08f0131d10 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08e4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08e4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f0131d10 msgr2=0x7f08f011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08e4ff9640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f0131d10 0x7f08f011ca70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f08f00a6ec0 tx=0x7f08e0008040 comp rx=0 tx=0).stop 2024-09-15T01:40:56.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f0131d10 msgr2=0x7f08f011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:56.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.871+0000 7f08e4ff9640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f0131d10 0x7f08f011ca70 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f08f00a6ec0 tx=0x7f08e0008040 comp rx=0 tx=0).stop 2024-09-15T01:40:56.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08f6c6f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:56.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08f6c6f640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f011cfb0 0x7f08f0096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:56.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08f6c6f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:56.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08f6c6f640 1 -- 172.21.15.178:0/104747463 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f08f01320e0 con 0x7f08f011cfb0 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08e7fff640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f011cfb0 0x7f08f0096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08e7fff640 10 monclient: get_auth_request con 0x7f08f011cfb0 auth_method 0 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08e7fff640 10 monclient(hunting): handle_auth_done global_id 44161 payload 824 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08e7fff640 1 -- 172.21.15.178:0/104747463 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f08f004bc80 con 0x7f08f011cfb0 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.877+0000 7f08e7fff640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f011cfb0 0x7f08f0096a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08e0019000 tx=0x7f08e001c000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08e0013e80 con 0x7f08f011cfb0 2024-09-15T01:40:56.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08e4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08e4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08e4ff9640 10 monclient: dump: 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:56.905 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:56.906 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:56.906 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:56.906 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:56.906 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:56.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f08e001ba60 con 0x7f08f011cfb0 2024-09-15T01:40:56.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08e4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:56.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08e4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:56.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f011cfb0 msgr2=0x7f08f0096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08e4ff9640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f011cfb0 0x7f08f0096a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08e0019000 tx=0x7f08e001c000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08e4ff9640 1 -- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f011cfb0 msgr2=0x7f08f0096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:56.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08e4ff9640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f08f011cfb0 0x7f08f0096a30 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08e0019000 tx=0x7f08e001c000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.907 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:56.907 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:56.907 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:56.907 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:56.907 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:56.907 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:56.907 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:56.908 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:56.909 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:56.911 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:56.911 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:56.911 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:56.911 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:56.911 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:56.911 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:56.912 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:56.913 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:56.913 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:56.913 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:56.913 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.913 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:56.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.913 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:56.913 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.913 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:56.914 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:56.914 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.914 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.914 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.914 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.914 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:56.914 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.915 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:56.917 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.918 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.919 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.920 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.921 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.922 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:56.923 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:56.923 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.923 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.923 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.923 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:56.923 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.923 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.924 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.926 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:56.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:56.929 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:56.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:56.931 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:56.931 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:56.931 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:56.931 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:56.931 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:56.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:56.931 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:56.931 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:56.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08f6c6f640 1 -- 172.21.15.178:0/104747463 >> v2:172.21.15.204:6812/28210 conn(0x7f08dc0396e0 msgr2=0x7f08dc03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08f6c6f640 1 --2- 172.21.15.178:0/104747463 >> v2:172.21.15.204:6812/28210 conn(0x7f08dc0396e0 0x7f08dc03bb90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009e30 tx=0x7f08e80090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:56.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08f6c6f640 10 monclient: shutdown 2024-09-15T01:40:56.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08f6c6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:56.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08f6c6f640 1 -- 172.21.15.178:0/104747463 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f08f009faa0 msgr2=0x7f08f009b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:56.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08f6c6f640 1 --2- 172.21.15.178:0/104747463 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f08f009faa0 0x7f08f009b050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f08ec000f80 tx=0x7f08ec036000 comp rx=0 tx=0).stop 2024-09-15T01:40:56.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08f520c640 1 -- 172.21.15.178:0/104747463 reap_dead start 2024-09-15T01:40:56.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08f520c640 1 -- 172.21.15.178:0/104747463 reap_dead start 2024-09-15T01:40:56.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08f6c6f640 1 -- 172.21.15.178:0/104747463 shutdown_connections 2024-09-15T01:40:56.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08f6c6f640 1 -- 172.21.15.178:0/104747463 >> 172.21.15.178:0/104747463 conn(0x7f08f00a5ed0 msgr2=0x7f08f009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:56.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08f6c6f640 1 -- 172.21.15.178:0/104747463 shutdown_connections 2024-09-15T01:40:56.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:56.878+0000 7f08f6c6f640 1 -- 172.21.15.178:0/104747463 wait complete. 2024-09-15T01:40:56.934 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:56.934 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T01:40:56.934 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T01:40:57.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.025+0000 7f9cdb48b640 10 monclient: get_monmap_and_config 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.025+0000 7f9cdb48b640 10 monclient: build_initial_monmap 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.025+0000 7f9cdb48b640 10 monclient: monmap: 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:57.026568+0000 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:57.026568+0000 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:57.029 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 1 Processor -- start 2024-09-15T01:40:57.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 1 -- start start 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 10 monclient: init 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 10 monclient: _add_conns ranks=[1,10,8,7,9,0,2,4,3,5,6] 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9cd40a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9cd4132de0 0x7f9cd4137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 10 monclient(hunting): picked mon.noname-f con 0x7f9cd4132de0 addr v2:172.21.15.204:3300/0 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 0x7f9cd413a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 10 monclient(hunting): picked mon.noname-e con 0x7f9cd41380e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd404bf60 con 0x7f9cd40a2930 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd409dd80 con 0x7f9cd4132de0 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:57.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd40a2d00 con 0x7f9cd41380e0 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.026+0000 7f9cdb48b640 10 monclient(hunting): authenticate will time out at 1261.776975s 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 0x7f9cd413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 0x7f9cd413a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:46126/0 (socket says 172.21.15.178:46126) 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 1 -- 172.21.15.178:0/859120473 learned_addr learned my addr 172.21.15.178:0/859120473 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 10 monclient(hunting): get_auth_request con 0x7f9cd41380e0 auth_method 0 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:57.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd8a26640 1 --2- 172.21.15.178:0/859120473 >> v2:172.21.15.204:3300/0 conn(0x7f9cd4132de0 0x7f9cd4137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd8a26640 10 monclient(hunting): get_auth_request con 0x7f9cd4132de0 auth_method 0 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd8a26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd8a26640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd8a26640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 10 monclient(hunting): handle_auth_done global_id 34424 payload 274 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 1 -- 172.21.15.178:0/859120473 >> v2:172.21.15.204:3300/0 conn(0x7f9cd4132de0 msgr2=0x7f9cd4137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 1 --2- 172.21.15.178:0/859120473 >> v2:172.21.15.204:3300/0 conn(0x7f9cd4132de0 0x7f9cd4137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 1 -- 172.21.15.178:0/859120473 >> v1:172.21.15.204:6789/0 conn(0x7f9cd40a2930 legacy=0x7f9cd409faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 10 monclient: _finish_hunting 0 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 1 monclient: found mon.noname-e 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:40:57.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.027+0000 7f9cd9a28640 1 -- 172.21.15.178:0/859120473 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd41331b0 con 0x7f9cd41380e0 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cd9a28640 1 --2- 172.21.15.178:0/859120473 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 0x7f9cd413a4c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009f20 tx=0x7f9ccc009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=7986dd33a42f51ef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9ccbfff640 1 -- 172.21.15.178:0/859120473 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ccc007880 con 0x7f9cd41380e0 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9ccbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9ccbfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9ccbfff640 10 monclient: dump: 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:57.041 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9ccbfff640 10 monclient: _finish_auth 0 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9ccbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 5 monclient: authenticate success, global_id 34424 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9ccbfff640 1 -- 172.21.15.178:0/859120473 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ccc02d050 con 0x7f9cd41380e0 2024-09-15T01:40:57.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9ccbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 4 set_mon_vals no callback set 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9ccbfff640 1 -- 172.21.15.178:0/859120473 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ccc030360 con 0x7f9cd41380e0 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9ccbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9ccbfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9ccbfff640 10 monclient: dump: 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:57.043 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 10 monclient: shutdown 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 1 -- 172.21.15.178:0/859120473 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 msgr2=0x7f9cd413a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 1 --2- 172.21.15.178:0/859120473 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 0x7f9cd413a4c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009f20 tx=0x7f9ccc009840 comp rx=0 tx=0).stop 2024-09-15T01:40:57.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 1 -- 172.21.15.178:0/859120473 shutdown_connections 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 1 --2- 172.21.15.178:0/859120473 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 0x7f9cd413a4c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 1 --2- 172.21.15.178:0/859120473 >> v2:172.21.15.204:3300/0 conn(0x7f9cd4132de0 0x7f9cd4137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 1 -- 172.21.15.178:0/859120473 >> 172.21.15.178:0/859120473 conn(0x7f9cd40a5ed0 msgr2=0x7f9cd40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 1 -- 172.21.15.178:0/859120473 shutdown_connections 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.028+0000 7f9cdb48b640 1 -- 172.21.15.178:0/859120473 wait complete. 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.029+0000 7f9cdb48b640 10 monclient: build_initial_monmap 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.029+0000 7f9cdb48b640 10 monclient: monmap: 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:57.045 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.029+0000 7f9cdb48b640 1 Processor -- start 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.029+0000 7f9cdb48b640 1 -- start start 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.029+0000 7f9cdb48b640 10 monclient: init 2024-09-15T01:40:57.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 10 monclient: _add_conns ranks=[4,5,8,0,3,7,2,1,6] 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd40a2930 0x7f9cd409a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 10 monclient(hunting): picked mon.noname-c con 0x7f9cd40a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9cd4132de0 0x7f9cd409ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 10 monclient(hunting): picked mon.noname-h con 0x7f9cd4132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 0x7f9cd409b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 10 monclient(hunting): picked mon.noname-e con 0x7f9cd41380e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd40a2390 con 0x7f9cd40a2930 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd409d630 con 0x7f9cd4132de0 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:57.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd409c1a0 con 0x7f9cd41380e0 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cdb48b640 10 monclient(hunting): authenticate will time out at 1261.780295s 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 0x7f9cd409b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 0x7f9cd409b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:46138/0 (socket says 172.21.15.178:46138) 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9227640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd40a2930 0x7f9cd409a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 1 -- 172.21.15.178:0/583863033 learned_addr learned my addr 172.21.15.178:0/583863033 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 10 monclient(hunting): get_auth_request con 0x7f9cd41380e0 auth_method 0 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9227640 10 monclient(hunting): get_auth_request con 0x7f9cd40a2930 auth_method 0 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9227640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:57.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9227640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9227640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd8a26640 1 --2- 172.21.15.178:0/583863033 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9cd4132de0 0x7f9cd409ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9227640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9227640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9227640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd8a26640 10 monclient(hunting): get_auth_request con 0x7f9cd4132de0 auth_method 0 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd8a26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd8a26640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd8a26640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:57.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd8a26640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd8a26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd8a26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 10 monclient(hunting): handle_auth_done global_id 34433 payload 824 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 1 -- 172.21.15.178:0/583863033 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9cd4132de0 msgr2=0x7f9cd409ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 1 --2- 172.21.15.178:0/583863033 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9cd4132de0 0x7f9cd409ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 1 -- 172.21.15.178:0/583863033 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd40a2930 msgr2=0x7f9cd409a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 1 --2- 172.21.15.178:0/583863033 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd40a2930 0x7f9cd409a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 10 monclient: _finish_hunting 0 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 1 monclient: found mon.noname-e 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 1 -- 172.21.15.178:0/583863033 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc009ae0 con 0x7f9cd41380e0 2024-09-15T01:40:57.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9227640 1 --2- 172.21.15.178:0/583863033 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd40a2930 0x7f9cd409a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.030+0000 7f9cd9a28640 1 --2- 172.21.15.178:0/583863033 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 0x7f9cd409b070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009880 tx=0x7f9ccc036000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cc97fa640 1 -- 172.21.15.178:0/583863033 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ccc03cab0 con 0x7f9cd41380e0 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cc97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cc97fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cc97fa640 10 monclient: dump: 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:57.051 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cc97fa640 10 monclient: _finish_auth 0 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cc97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cc97fa640 1 -- 172.21.15.178:0/583863033 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ccc03cc70 con 0x7f9cd41380e0 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cdb48b640 5 monclient: authenticate success, global_id 34433 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cc97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:57.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cc97fa640 1 -- 172.21.15.178:0/583863033 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ccc03cde0 con 0x7f9cd41380e0 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cc97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cc97fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cc97fa640 10 monclient: dump: 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:57.053 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9ccb7fe640 4 set_mon_vals no callback set 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cdb48b640 10 monclient: _renew_subs 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cdb48b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cdb48b640 1 -- 172.21.15.178:0/583863033 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd411d540 con 0x7f9cd41380e0 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cdb48b640 10 monclient: _renew_subs 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cdb48b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.031+0000 7f9cdb48b640 1 -- 172.21.15.178:0/583863033 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd4132640 con 0x7f9cd41380e0 2024-09-15T01:40:57.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.032+0000 7f9cc97fa640 1 -- 172.21.15.178:0/583863033 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9ccc003040 con 0x7f9cd41380e0 2024-09-15T01:40:57.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.032+0000 7f9cdb48b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:57.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.032+0000 7f9cdb48b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:57.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.032+0000 7f9cdb48b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:57.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.032+0000 7f9cdb48b640 1 -- 172.21.15.178:0/583863033 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd404b720 con 0x7f9cd41380e0 2024-09-15T01:40:57.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.032+0000 7f9cc97fa640 1 --2- 172.21.15.178:0/583863033 >> v2:172.21.15.204:6812/28210 conn(0x7f9cb80397b0 0x7f9cb803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:57.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.032+0000 7f9cc97fa640 1 -- 172.21.15.178:0/583863033 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f9ccc06c350 con 0x7f9cd41380e0 2024-09-15T01:40:57.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.032+0000 7f9cd9227640 1 --2- 172.21.15.178:0/583863033 >> v2:172.21.15.204:6812/28210 conn(0x7f9cb80397b0 0x7f9cb803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:57.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.032+0000 7f9cd9227640 10 monclient: get_auth_request con 0x7f9cb80397b0 auth_method 0 2024-09-15T01:40:57.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.032+0000 7f9cd9227640 1 --2- 172.21.15.178:0/583863033 >> v2:172.21.15.204:6812/28210 conn(0x7f9cb80397b0 0x7f9cb803bc60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0007fd0 tx=0x7f9cd0009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:57.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.036+0000 7f9cc97fa640 1 -- 172.21.15.178:0/583863033 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9ccc03aad0 con 0x7f9cd41380e0 2024-09-15T01:40:57.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.036+0000 7f9cc97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:57.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.036+0000 7f9cc97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:57.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.254+0000 7f9cdb48b640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T01:40:57.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.254+0000 7f9cdb48b640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T01:40:57.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.254+0000 7f9cdb48b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:57.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.254+0000 7f9cdb48b640 1 -- 172.21.15.178:0/583863033 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f9cd409cef0 con 0x7f9cd41380e0 2024-09-15T01:40:57.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.256+0000 7f9cc97fa640 1 -- 172.21.15.178:0/583863033 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f9ccc03a310 con 0x7f9cd41380e0 2024-09-15T01:40:57.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.256+0000 7f9cc97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T01:40:57.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.256+0000 7f9cc97fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:57.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 1 -- 172.21.15.178:0/583863033 >> v2:172.21.15.204:6812/28210 conn(0x7f9cb80397b0 msgr2=0x7f9cb803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:57.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 1 --2- 172.21.15.178:0/583863033 >> v2:172.21.15.204:6812/28210 conn(0x7f9cb80397b0 0x7f9cb803bc60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0007fd0 tx=0x7f9cd0009040 comp rx=0 tx=0).stop 2024-09-15T01:40:57.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 10 monclient: shutdown 2024-09-15T01:40:57.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:57.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 1 -- 172.21.15.178:0/583863033 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 msgr2=0x7f9cd409b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:57.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 1 --2- 172.21.15.178:0/583863033 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 0x7f9cd409b070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009880 tx=0x7f9ccc036000 comp rx=0 tx=0).stop 2024-09-15T01:40:57.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 1 -- 172.21.15.178:0/583863033 shutdown_connections 2024-09-15T01:40:57.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 1 --2- 172.21.15.178:0/583863033 >> v2:172.21.15.204:6812/28210 conn(0x7f9cb80397b0 0x7f9cb803bc60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:57.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 1 --2- 172.21.15.178:0/583863033 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9cd41380e0 0x7f9cd409b070 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:57.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 1 --2- 172.21.15.178:0/583863033 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9cd4132de0 0x7f9cd409ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:57.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 1 --2- 172.21.15.178:0/583863033 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9cd40a2930 0x7f9cd409a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:57.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 1 -- 172.21.15.178:0/583863033 >> 172.21.15.178:0/583863033 conn(0x7f9cd40a5ed0 msgr2=0x7f9cd41358c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:57.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 1 -- 172.21.15.178:0/583863033 shutdown_connections 2024-09-15T01:40:57.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:57.259+0000 7f9cdb48b640 1 -- 172.21.15.178:0/583863033 wait complete. 2024-09-15T01:40:57.279 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T01:40:57.379 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.256+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.379 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.256+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=13,config=1,health=12,kv=8,logm=66} crc {auth=3865496874,config=1228700967,health=1509209890,kv=706962213,logm=857658823}) 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.256+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=11,kv=7,logm=65} crc {auth=2973463805,config=3140514009,health=2564983629,kv=2951374574,logm=547070969}) 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=13,config=1,health=12,kv=8,logm=66} crc {auth=3865496874,config=1228700967,health=1509209890,kv=706962213,logm=857658823}) 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=15,config=2,health=11,kv=8,logm=64} crc {auth=2973463805,config=3140514009,health=1295987503,kv=706962213,logm=1221026299}) 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=13,config=1,health=12,kv=8,logm=66} crc {auth=3865496874,config=1228700967,health=1509209890,kv=706962213,logm=857658823}) 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=15,config=2,health=12,kv=6,logm=65} crc {auth=2973463805,config=3140514009,health=1170029810,kv=837677417,logm=2789947619}) 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=13,config=1,health=12,kv=8,logm=66} crc {auth=3865496874,config=1228700967,health=1509209890,kv=706962213,logm=857658823}) 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=15,config=1,health=12,kv=7,logm=65} crc {auth=2491284285,config=1228700967,health=3279024376,kv=963045094,logm=3901748238}) 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=13,config=1,health=12,kv=8,logm=66} crc {auth=3865496874,config=1228700967,health=1509209890,kv=706962213,logm=857658823}) 2024-09-15T01:40:57.380 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=14,config=2,health=12,kv=8,logm=64} crc {auth=832752066,config=3140514009,health=1170029810,kv=706962213,logm=3361921152}) 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=13,config=1,health=12,kv=8,logm=66} crc {auth=3865496874,config=1228700967,health=1509209890,kv=706962213,logm=857658823}) 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=15,config=2,health=11,kv=6,logm=66} crc {auth=2973463805,config=3140514009,health=2564983629,kv=682160210,logm=2159454240}) 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=13,config=1,health=12,kv=8,logm=66} crc {auth=3865496874,config=1228700967,health=1509209890,kv=706962213,logm=857658823}) 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=15,config=2,health=12,kv=8,logm=63} crc {auth=2973463805,config=3140514009,health=1170029810,kv=706962213,logm=1179959414}) 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=13,config=1,health=12,kv=8,logm=66} crc {auth=3865496874,config=1228700967,health=1509209890,kv=706962213,logm=857658823}) 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.257+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=15,config=2,health=10,kv=8,logm=65} crc {auth=2973463805,config=3140514009,health=1763747969,kv=706962213,logm=1507077083}) 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=220624134}) 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=266055950}) 2024-09-15T01:40:57.381 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=220624134}) 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=396458052}) 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=220624134}) 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3950620503}) 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=220624134}) 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1352338506}) 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=220624134}) 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=793679155}) 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=220624134}) 2024-09-15T01:40:57.382 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4209993020}) 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=220624134}) 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=329109989}) 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=220624134}) 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.259+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2790504446}) 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3902006714}) 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3768660848}) 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3902006714}) 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3184272484}) 2024-09-15T01:40:57.383 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.384 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3902006714}) 2024-09-15T01:40:57.384 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1882611991}) 2024-09-15T01:40:57.384 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.384 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3902006714}) 2024-09-15T01:40:57.384 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3278430673}) 2024-09-15T01:40:57.384 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.384 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3902006714}) 2024-09-15T01:40:57.384 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3583241213}) 2024-09-15T01:40:57.384 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.384 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3902006714}) 2024-09-15T01:40:57.384 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3563642133}) 2024-09-15T01:40:57.384 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3902006714}) 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2538471963}) 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3902006714}) 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.261+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=971752034}) 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2587667524}) 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=418464713}) 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2587667524}) 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2337560350}) 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.385 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2587667524}) 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2499582238}) 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2587667524}) 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=159905553}) 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2587667524}) 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1394511264}) 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2587667524}) 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3519109025}) 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2587667524}) 2024-09-15T01:40:57.386 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2255533953}) 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2587667524}) 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.263+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4159491796}) 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2323005082}) 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1758800489}) 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2323005082}) 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1026658504}) 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2323005082}) 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3238879166}) 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.387 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2323005082}) 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1204269009}) 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2323005082}) 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3123884947}) 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2323005082}) 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1043473462}) 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2323005082}) 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2105703845}) 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2323005082}) 2024-09-15T01:40:57.388 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.264+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=566733247}) 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.265+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.265+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2298521637}) 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.265+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3916390845}) 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.265+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.265+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2298521637}) 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.265+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3022548128}) 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.265+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2298521637}) 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=652714529}) 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2298521637}) 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=891834262}) 2024-09-15T01:40:57.389 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2298521637}) 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1914964371}) 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2298521637}) 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2766550343}) 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2298521637}) 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=805267425}) 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2298521637}) 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.266+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3461548736}) 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.390 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=312330489}) 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3857109350}) 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=312330489}) 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3722489606}) 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=312330489}) 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1863378798}) 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=312330489}) 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2325917471}) 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=312330489}) 2024-09-15T01:40:57.391 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3603823010}) 2024-09-15T01:40:57.392 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.392 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=312330489}) 2024-09-15T01:40:57.392 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=972688880}) 2024-09-15T01:40:57.392 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.392 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=312330489}) 2024-09-15T01:40:57.392 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3102168282}) 2024-09-15T01:40:57.392 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.392 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=312330489}) 2024-09-15T01:40:57.392 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.267+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1279886744}) 2024-09-15T01:40:57.392 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.392 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=559277542}) 2024-09-15T01:40:57.392 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2336902806}) 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=559277542}) 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1439406385}) 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=559277542}) 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=859348145}) 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=559277542}) 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3238087928}) 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=559277542}) 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.268+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1794936553}) 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.269+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.393 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.269+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=559277542}) 2024-09-15T01:40:57.394 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.269+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4089792233}) 2024-09-15T01:40:57.394 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.269+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.394 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.269+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=559277542}) 2024-09-15T01:40:57.394 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.269+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3391867217}) 2024-09-15T01:40:57.394 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.269+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.394 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.269+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=559277542}) 2024-09-15T01:40:57.394 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.269+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3290371588}) 2024-09-15T01:40:57.394 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.394 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3278163094}) 2024-09-15T01:40:57.394 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1493705580}) 2024-09-15T01:40:57.394 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.394 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3278163094}) 2024-09-15T01:40:57.395 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3071660786}) 2024-09-15T01:40:57.395 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.395 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3278163094}) 2024-09-15T01:40:57.395 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3572856383}) 2024-09-15T01:40:57.395 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.395 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3278163094}) 2024-09-15T01:40:57.395 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1035227033}) 2024-09-15T01:40:57.395 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.395 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3278163094}) 2024-09-15T01:40:57.395 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3311249707}) 2024-09-15T01:40:57.395 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.395 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3278163094}) 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=764609239}) 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3278163094}) 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1428702251}) 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3278163094}) 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.270+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1052854530}) 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1136042175}) 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3699271937}) 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1136042175}) 2024-09-15T01:40:57.396 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=158812551}) 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1136042175}) 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4130990984}) 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1136042175}) 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2305810851}) 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1136042175}) 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4145146262}) 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1136042175}) 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3292244871}) 2024-09-15T01:40:57.397 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.398 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1136042175}) 2024-09-15T01:40:57.398 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=268244804}) 2024-09-15T01:40:57.398 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.398 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1136042175}) 2024-09-15T01:40:57.398 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.271+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2365441807}) 2024-09-15T01:40:57.398 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.398 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4112621279}) 2024-09-15T01:40:57.400 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3617147812}) 2024-09-15T01:40:57.400 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.400 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4112621279}) 2024-09-15T01:40:57.400 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2812681494}) 2024-09-15T01:40:57.400 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.400 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4112621279}) 2024-09-15T01:40:57.400 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=521983592}) 2024-09-15T01:40:57.400 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.400 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4112621279}) 2024-09-15T01:40:57.400 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=669938405}) 2024-09-15T01:40:57.400 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.400 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4112621279}) 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=888630926}) 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4112621279}) 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=10212071}) 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4112621279}) 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=280372759}) 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4112621279}) 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.273+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1030763148}) 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2511859131}) 2024-09-15T01:40:57.401 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2369869333}) 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2511859131}) 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1963568956}) 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2511859131}) 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1200797831}) 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2511859131}) 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3722184685}) 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2511859131}) 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2241633416}) 2024-09-15T01:40:57.402 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2511859131}) 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3839089680}) 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2511859131}) 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2099343927}) 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2511859131}) 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.274+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3047027630}) 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.275+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.275+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=565806650}) 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.275+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4161658613}) 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.275+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.403 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.275+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=565806650}) 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.275+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2008246833}) 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.275+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.275+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=565806650}) 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.275+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3707836088}) 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.275+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.275+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=565806650}) 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.275+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2999117593}) 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.276+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.276+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=565806650}) 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.276+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=716229166}) 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.276+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.276+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=565806650}) 2024-09-15T01:40:57.404 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.276+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1269420087}) 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.276+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.276+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=565806650}) 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.276+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1094354909}) 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.276+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.276+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=565806650}) 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.276+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4050752547}) 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2615350090}) 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3807897407}) 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2615350090}) 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1696086058}) 2024-09-15T01:40:57.405 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2615350090}) 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1174462622}) 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2615350090}) 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1690862075}) 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2615350090}) 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4266370000}) 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2615350090}) 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=851373876}) 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.406 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2615350090}) 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2239885175}) 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2615350090}) 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.277+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3717762030}) 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3815432467}) 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3582023638}) 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3815432467}) 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2349470915}) 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3815432467}) 2024-09-15T01:40:57.407 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3192254921}) 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3815432467}) 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4104382759}) 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3815432467}) 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1249967820}) 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3815432467}) 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2256586241}) 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3815432467}) 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=716555402}) 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.408 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3815432467}) 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.279+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2058095584}) 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190431682}) 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2351247364}) 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190431682}) 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3751034766}) 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190431682}) 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2157365214}) 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190431682}) 2024-09-15T01:40:57.409 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4042973025}) 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190431682}) 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3039308474}) 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190431682}) 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2238644830}) 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190431682}) 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3963903269}) 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190431682}) 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.280+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1952913558}) 2024-09-15T01:40:57.410 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.281+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.281+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2498240692}) 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.281+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1764252772}) 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.281+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.281+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2498240692}) 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.281+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1114124469}) 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2498240692}) 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1547896995}) 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2498240692}) 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=265784964}) 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.411 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2498240692}) 2024-09-15T01:40:57.412 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3246312896}) 2024-09-15T01:40:57.412 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.412 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2498240692}) 2024-09-15T01:40:57.412 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2678751363}) 2024-09-15T01:40:57.412 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.412 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2498240692}) 2024-09-15T01:40:57.412 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=655174675}) 2024-09-15T01:40:57.412 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.412 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2498240692}) 2024-09-15T01:40:57.412 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.282+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4002326527}) 2024-09-15T01:40:57.412 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.413 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3963945803}) 2024-09-15T01:40:57.413 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1080019261}) 2024-09-15T01:40:57.413 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.413 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3963945803}) 2024-09-15T01:40:57.413 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=451825958}) 2024-09-15T01:40:57.413 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.413 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3963945803}) 2024-09-15T01:40:57.413 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3676532248}) 2024-09-15T01:40:57.413 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.413 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3963945803}) 2024-09-15T01:40:57.413 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2027706280}) 2024-09-15T01:40:57.413 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.414 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3963945803}) 2024-09-15T01:40:57.414 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4285186517}) 2024-09-15T01:40:57.414 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.414 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3963945803}) 2024-09-15T01:40:57.414 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3787485120}) 2024-09-15T01:40:57.414 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.414 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3963945803}) 2024-09-15T01:40:57.414 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2141595576}) 2024-09-15T01:40:57.414 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.414 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3963945803}) 2024-09-15T01:40:57.414 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.283+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3041915948}) 2024-09-15T01:40:57.414 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=457272938}) 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3264687804}) 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=457272938}) 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1175033577}) 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=457272938}) 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3917814015}) 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=457272938}) 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1535184416}) 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=457272938}) 2024-09-15T01:40:57.415 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=113565835}) 2024-09-15T01:40:57.416 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.416 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=457272938}) 2024-09-15T01:40:57.416 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1941114165}) 2024-09-15T01:40:57.416 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.416 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=457272938}) 2024-09-15T01:40:57.416 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3882202701}) 2024-09-15T01:40:57.416 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.416 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=457272938}) 2024-09-15T01:40:57.416 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.284+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2847247412}) 2024-09-15T01:40:57.416 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.285+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.416 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.285+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3888696209}) 2024-09-15T01:40:57.416 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.285+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2512611067}) 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.285+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.285+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3888696209}) 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.285+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1732801015}) 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.285+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3888696209}) 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2536659149}) 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3888696209}) 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1811436359}) 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3888696209}) 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3162414981}) 2024-09-15T01:40:57.417 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3888696209}) 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3799118305}) 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3888696209}) 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3001611084}) 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3888696209}) 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.286+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2093696216}) 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4077403094}) 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1155435977}) 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.418 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4077403094}) 2024-09-15T01:40:57.419 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3065780564}) 2024-09-15T01:40:57.419 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.419 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4077403094}) 2024-09-15T01:40:57.419 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=53619581}) 2024-09-15T01:40:57.419 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.419 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4077403094}) 2024-09-15T01:40:57.419 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=465914269}) 2024-09-15T01:40:57.419 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.419 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4077403094}) 2024-09-15T01:40:57.419 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1393052792}) 2024-09-15T01:40:57.419 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.419 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4077403094}) 2024-09-15T01:40:57.420 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3858491135}) 2024-09-15T01:40:57.420 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.420 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4077403094}) 2024-09-15T01:40:57.420 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2218895199}) 2024-09-15T01:40:57.420 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.420 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4077403094}) 2024-09-15T01:40:57.420 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.287+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4168330435}) 2024-09-15T01:40:57.420 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.420 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4106961840}) 2024-09-15T01:40:57.420 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3606719554}) 2024-09-15T01:40:57.420 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.420 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4106961840}) 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3038217624}) 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4106961840}) 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2032203113}) 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4106961840}) 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=190782544}) 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4106961840}) 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1229346752}) 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4106961840}) 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2056727551}) 2024-09-15T01:40:57.421 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.422 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4106961840}) 2024-09-15T01:40:57.422 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=898363211}) 2024-09-15T01:40:57.422 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.422 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4106961840}) 2024-09-15T01:40:57.422 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.288+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1093850954}) 2024-09-15T01:40:57.422 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.289+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.422 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.289+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4279182800}) 2024-09-15T01:40:57.422 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.289+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2152368558}) 2024-09-15T01:40:57.422 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.422 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4279182800}) 2024-09-15T01:40:57.422 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=945626374}) 2024-09-15T01:40:57.422 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4279182800}) 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=316933757}) 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4279182800}) 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=194571509}) 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4279182800}) 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2580640837}) 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4279182800}) 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3364165787}) 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.423 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4279182800}) 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=594122812}) 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4279182800}) 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.290+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1519797898}) 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=702948209}) 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2311819621}) 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=702948209}) 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3816450972}) 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=702948209}) 2024-09-15T01:40:57.424 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1896508058}) 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=702948209}) 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=289834555}) 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=702948209}) 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=763558248}) 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=702948209}) 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1350790356}) 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=702948209}) 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3721169359}) 2024-09-15T01:40:57.425 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.426 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=702948209}) 2024-09-15T01:40:57.426 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.291+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1628475219}) 2024-09-15T01:40:57.426 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.292+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.426 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.292+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=28771343}) 2024-09-15T01:40:57.426 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3242364681}) 2024-09-15T01:40:57.426 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.426 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=28771343}) 2024-09-15T01:40:57.426 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=178643777}) 2024-09-15T01:40:57.426 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.426 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=28771343}) 2024-09-15T01:40:57.426 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3200552292}) 2024-09-15T01:40:57.426 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=28771343}) 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3438559745}) 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=28771343}) 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=424618177}) 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=28771343}) 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=276884278}) 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=28771343}) 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3960744437}) 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.427 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=28771343}) 2024-09-15T01:40:57.428 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.293+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3246418446}) 2024-09-15T01:40:57.428 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.428 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2299636633}) 2024-09-15T01:40:57.428 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3173685270}) 2024-09-15T01:40:57.428 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.428 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2299636633}) 2024-09-15T01:40:57.428 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2025071144}) 2024-09-15T01:40:57.428 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.428 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2299636633}) 2024-09-15T01:40:57.428 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3256689372}) 2024-09-15T01:40:57.428 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.428 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2299636633}) 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=109834713}) 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2299636633}) 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1463471100}) 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2299636633}) 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2693544085}) 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2299636633}) 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1683775897}) 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2299636633}) 2024-09-15T01:40:57.429 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.294+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=526777387}) 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=48046731}) 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1429105639}) 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=48046731}) 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=912540211}) 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=48046731}) 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3655334078}) 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=48046731}) 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1398992657}) 2024-09-15T01:40:57.430 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=48046731}) 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3707937213}) 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=48046731}) 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4104716676}) 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=48046731}) 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=262937716}) 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.295+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=48046731}) 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.296+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1510750423}) 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.431 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190699579}) 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2810694685}) 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190699579}) 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2911721628}) 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190699579}) 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3435237454}) 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190699579}) 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3422480276}) 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190699579}) 2024-09-15T01:40:57.432 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2814156308}) 2024-09-15T01:40:57.433 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.433 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190699579}) 2024-09-15T01:40:57.433 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2786682952}) 2024-09-15T01:40:57.433 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.433 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190699579}) 2024-09-15T01:40:57.433 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3465850504}) 2024-09-15T01:40:57.433 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.433 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4190699579}) 2024-09-15T01:40:57.433 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.297+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=351158509}) 2024-09-15T01:40:57.433 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.433 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1987315569}) 2024-09-15T01:40:57.433 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=388849827}) 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1987315569}) 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1556397825}) 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1987315569}) 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1932433924}) 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1987315569}) 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=543898026}) 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1987315569}) 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3431846246}) 2024-09-15T01:40:57.434 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1987315569}) 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2824068086}) 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1987315569}) 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1156299851}) 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1987315569}) 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.298+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2679378066}) 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=318630605}) 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142775207}) 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.435 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=318630605}) 2024-09-15T01:40:57.436 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2318355652}) 2024-09-15T01:40:57.436 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.436 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=318630605}) 2024-09-15T01:40:57.436 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3180466786}) 2024-09-15T01:40:57.436 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.436 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=318630605}) 2024-09-15T01:40:57.436 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=504571619}) 2024-09-15T01:40:57.436 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.436 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=318630605}) 2024-09-15T01:40:57.436 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=488381088}) 2024-09-15T01:40:57.436 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.436 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=318630605}) 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2159399419}) 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=318630605}) 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3580258024}) 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=318630605}) 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.300+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1330640146}) 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4199483409}) 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1061508859}) 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4199483409}) 2024-09-15T01:40:57.437 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=897306190}) 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4199483409}) 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3166619561}) 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4199483409}) 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2674320660}) 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4199483409}) 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=738742336}) 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4199483409}) 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2716420234}) 2024-09-15T01:40:57.438 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.439 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4199483409}) 2024-09-15T01:40:57.439 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1575084107}) 2024-09-15T01:40:57.439 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.439 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4199483409}) 2024-09-15T01:40:57.439 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.301+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1939402612}) 2024-09-15T01:40:57.439 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.439 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3404767644}) 2024-09-15T01:40:57.439 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3949907546}) 2024-09-15T01:40:57.439 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.439 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3404767644}) 2024-09-15T01:40:57.439 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=332928920}) 2024-09-15T01:40:57.439 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.440 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3404767644}) 2024-09-15T01:40:57.440 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3427865965}) 2024-09-15T01:40:57.440 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.440 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3404767644}) 2024-09-15T01:40:57.440 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2633810050}) 2024-09-15T01:40:57.440 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.440 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3404767644}) 2024-09-15T01:40:57.440 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1290104252}) 2024-09-15T01:40:57.440 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.440 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3404767644}) 2024-09-15T01:40:57.440 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1980309403}) 2024-09-15T01:40:57.440 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.441 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3404767644}) 2024-09-15T01:40:57.441 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4111363280}) 2024-09-15T01:40:57.441 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.441 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3404767644}) 2024-09-15T01:40:57.441 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.302+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3553295603}) 2024-09-15T01:40:57.441 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.303+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.441 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.303+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3876900917}) 2024-09-15T01:40:57.441 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.303+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1922061770}) 2024-09-15T01:40:57.441 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.441 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3876900917}) 2024-09-15T01:40:57.441 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1009657883}) 2024-09-15T01:40:57.441 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3876900917}) 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3244159260}) 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3876900917}) 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1851341942}) 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3876900917}) 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3945005071}) 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3876900917}) 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=829285634}) 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3876900917}) 2024-09-15T01:40:57.442 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=757547639}) 2024-09-15T01:40:57.443 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.443 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3876900917}) 2024-09-15T01:40:57.443 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.304+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2058126839}) 2024-09-15T01:40:57.443 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.443 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4205021656}) 2024-09-15T01:40:57.443 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2047889035}) 2024-09-15T01:40:57.443 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.443 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4205021656}) 2024-09-15T01:40:57.443 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3067944786}) 2024-09-15T01:40:57.443 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.443 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4205021656}) 2024-09-15T01:40:57.443 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2612841361}) 2024-09-15T01:40:57.444 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.444 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4205021656}) 2024-09-15T01:40:57.444 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=521452251}) 2024-09-15T01:40:57.444 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.444 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4205021656}) 2024-09-15T01:40:57.444 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3724213366}) 2024-09-15T01:40:57.444 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.444 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4205021656}) 2024-09-15T01:40:57.444 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=332061841}) 2024-09-15T01:40:57.444 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.444 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4205021656}) 2024-09-15T01:40:57.444 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2420174587}) 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4205021656}) 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.305+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=646618123}) 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1045757171}) 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=208972035}) 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1045757171}) 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=879314314}) 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1045757171}) 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2199639292}) 2024-09-15T01:40:57.445 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1045757171}) 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1501998842}) 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1045757171}) 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1588046085}) 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1045757171}) 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1485113321}) 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1045757171}) 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3509752369}) 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.446 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1045757171}) 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.306+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1358979064}) 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.307+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.307+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583706289}) 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.307+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698565060}) 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583706289}) 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2184498285}) 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583706289}) 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1683839630}) 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583706289}) 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=417471601}) 2024-09-15T01:40:57.447 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.448 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583706289}) 2024-09-15T01:40:57.448 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=732583987}) 2024-09-15T01:40:57.448 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.448 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583706289}) 2024-09-15T01:40:57.448 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=285534449}) 2024-09-15T01:40:57.448 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.448 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583706289}) 2024-09-15T01:40:57.448 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=816195466}) 2024-09-15T01:40:57.448 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.448 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583706289}) 2024-09-15T01:40:57.448 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.308+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3629854246}) 2024-09-15T01:40:57.448 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=579657100}) 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3116899379}) 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=579657100}) 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3197652470}) 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=579657100}) 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2982248883}) 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=579657100}) 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2849539309}) 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.449 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=579657100}) 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=649995806}) 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=579657100}) 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1023022679}) 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=579657100}) 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2552541962}) 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=579657100}) 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.309+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4248010568}) 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1992715115}) 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3172742186}) 2024-09-15T01:40:57.450 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1992715115}) 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3468915136}) 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1992715115}) 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2532346214}) 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1992715115}) 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3465412938}) 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1992715115}) 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=9494228}) 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1992715115}) 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1097949981}) 2024-09-15T01:40:57.451 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1992715115}) 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1863958299}) 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1992715115}) 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.310+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=115307254}) 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.311+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.311+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1811319604}) 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.311+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=508632083}) 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.311+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.311+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1811319604}) 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.311+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4210869173}) 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.452 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1811319604}) 2024-09-15T01:40:57.453 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2814031198}) 2024-09-15T01:40:57.453 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.453 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1811319604}) 2024-09-15T01:40:57.453 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2332502853}) 2024-09-15T01:40:57.453 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.453 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1811319604}) 2024-09-15T01:40:57.453 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=826777850}) 2024-09-15T01:40:57.453 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.453 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1811319604}) 2024-09-15T01:40:57.453 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=846160442}) 2024-09-15T01:40:57.453 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.453 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1811319604}) 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2628020252}) 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1811319604}) 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.312+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4186251258}) 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3475191012}) 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2487751018}) 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3475191012}) 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=229452248}) 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3475191012}) 2024-09-15T01:40:57.454 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1274538649}) 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3475191012}) 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2728162087}) 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3475191012}) 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1874277250}) 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3475191012}) 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1232769804}) 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3475191012}) 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3682878245}) 2024-09-15T01:40:57.455 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3475191012}) 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.313+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3325203876}) 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=891413826}) 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3390433340}) 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=891413826}) 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=81155243}) 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=891413826}) 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=548308192}) 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.456 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=891413826}) 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1830719219}) 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=891413826}) 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=470045220}) 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=891413826}) 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4054037437}) 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=891413826}) 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3759557564}) 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=891413826}) 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.314+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2171716684}) 2024-09-15T01:40:57.457 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.315+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.315+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2131652117}) 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.315+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1561961405}) 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.315+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.315+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2131652117}) 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.315+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3299754165}) 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.315+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.315+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2131652117}) 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.315+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2931372412}) 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.315+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.315+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2131652117}) 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4285041731}) 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2131652117}) 2024-09-15T01:40:57.458 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1526219266}) 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2131652117}) 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1798555161}) 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2131652117}) 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1357919057}) 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2131652117}) 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.316+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2412370451}) 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1945078004}) 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=700531304}) 2024-09-15T01:40:57.459 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1945078004}) 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4027987759}) 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1945078004}) 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3308097942}) 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1945078004}) 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=388310275}) 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1945078004}) 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=521739308}) 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.460 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1945078004}) 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4151189068}) 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1945078004}) 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2120285748}) 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1945078004}) 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.317+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2421760611}) 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2834841377}) 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=33640094}) 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2834841377}) 2024-09-15T01:40:57.461 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3430330648}) 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2834841377}) 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3009435012}) 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2834841377}) 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=670307094}) 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2834841377}) 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3938687763}) 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2834841377}) 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=336901913}) 2024-09-15T01:40:57.462 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2834841377}) 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2099495576}) 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2834841377}) 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.318+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2147133912}) 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=485022813}) 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=120104392}) 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=485022813}) 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2938666322}) 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=485022813}) 2024-09-15T01:40:57.463 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4008773149}) 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=485022813}) 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4156944136}) 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.319+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.320+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=485022813}) 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.320+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2944314948}) 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.320+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.320+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=485022813}) 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.320+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3034658320}) 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.320+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.320+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=485022813}) 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.320+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=6014814}) 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.320+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.464 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.320+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=485022813}) 2024-09-15T01:40:57.465 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.320+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2218616581}) 2024-09-15T01:40:57.465 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.465 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2800693637}) 2024-09-15T01:40:57.465 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=447354104}) 2024-09-15T01:40:57.465 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.465 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2800693637}) 2024-09-15T01:40:57.465 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4215772533}) 2024-09-15T01:40:57.465 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.465 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2800693637}) 2024-09-15T01:40:57.465 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=925870759}) 2024-09-15T01:40:57.465 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.465 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2800693637}) 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1896097003}) 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2800693637}) 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2128934627}) 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2800693637}) 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1071234585}) 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2800693637}) 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=148440693}) 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2800693637}) 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.321+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4261294996}) 2024-09-15T01:40:57.466 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=317892172}) 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2850331139}) 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=317892172}) 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2702011091}) 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=317892172}) 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3451275657}) 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=317892172}) 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2249471701}) 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.467 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=317892172}) 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=143271159}) 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=317892172}) 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=922510433}) 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.322+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=317892172}) 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.323+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1077135335}) 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.323+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.323+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=317892172}) 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.323+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3153543327}) 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956820375}) 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3211952792}) 2024-09-15T01:40:57.468 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956820375}) 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=443318821}) 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956820375}) 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1413124146}) 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956820375}) 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2873061494}) 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956820375}) 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2901146441}) 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.469 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956820375}) 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=11922495}) 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956820375}) 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2243283336}) 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956820375}) 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.324+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1713816552}) 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659992853}) 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3962878425}) 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659992853}) 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3844941981}) 2024-09-15T01:40:57.470 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659992853}) 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=941772432}) 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659992853}) 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2033030385}) 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659992853}) 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.325+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3465183270}) 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.326+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.326+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659992853}) 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.326+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1548336749}) 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.326+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.326+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659992853}) 2024-09-15T01:40:57.471 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.326+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=819709288}) 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.326+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.326+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659992853}) 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.326+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3378964902}) 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3172162650}) 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3143172384}) 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3172162650}) 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1854939234}) 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3172162650}) 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2442407097}) 2024-09-15T01:40:57.472 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3172162650}) 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=829688525}) 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3172162650}) 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2251407382}) 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3172162650}) 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2542415036}) 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3172162650}) 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=655800843}) 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3172162650}) 2024-09-15T01:40:57.473 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.327+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667905167}) 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1695569443}) 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3782807171}) 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1695569443}) 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1341864131}) 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1695569443}) 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3830269134}) 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1695569443}) 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2013739658}) 2024-09-15T01:40:57.474 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1695569443}) 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1303403519}) 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1695569443}) 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3408547843}) 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1695569443}) 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1065343649}) 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1695569443}) 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.328+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=430344958}) 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.329+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.329+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4039443590}) 2024-09-15T01:40:57.475 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=201331278}) 2024-09-15T01:40:57.476 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.476 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4039443590}) 2024-09-15T01:40:57.476 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4176880177}) 2024-09-15T01:40:57.476 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.476 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4039443590}) 2024-09-15T01:40:57.476 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2102730691}) 2024-09-15T01:40:57.476 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.476 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4039443590}) 2024-09-15T01:40:57.476 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4261611265}) 2024-09-15T01:40:57.476 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.476 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4039443590}) 2024-09-15T01:40:57.476 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4138569605}) 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4039443590}) 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=991123803}) 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4039443590}) 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1273490370}) 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4039443590}) 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.330+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4214801894}) 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3459299387}) 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=754157994}) 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.477 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3459299387}) 2024-09-15T01:40:57.478 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2028282212}) 2024-09-15T01:40:57.478 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.478 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3459299387}) 2024-09-15T01:40:57.478 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3341280295}) 2024-09-15T01:40:57.478 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.478 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3459299387}) 2024-09-15T01:40:57.478 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1699568410}) 2024-09-15T01:40:57.478 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.478 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3459299387}) 2024-09-15T01:40:57.478 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3815691010}) 2024-09-15T01:40:57.478 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.478 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3459299387}) 2024-09-15T01:40:57.479 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=336062048}) 2024-09-15T01:40:57.479 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.479 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3459299387}) 2024-09-15T01:40:57.479 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4143248546}) 2024-09-15T01:40:57.479 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.479 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3459299387}) 2024-09-15T01:40:57.479 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.331+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1915991887}) 2024-09-15T01:40:57.479 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.479 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2494856063}) 2024-09-15T01:40:57.479 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2667756211}) 2024-09-15T01:40:57.479 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.480 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2494856063}) 2024-09-15T01:40:57.480 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3737041707}) 2024-09-15T01:40:57.480 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.480 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2494856063}) 2024-09-15T01:40:57.480 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1870468060}) 2024-09-15T01:40:57.480 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.480 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2494856063}) 2024-09-15T01:40:57.480 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=954268519}) 2024-09-15T01:40:57.480 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.480 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2494856063}) 2024-09-15T01:40:57.480 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1095536711}) 2024-09-15T01:40:57.480 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.332+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2494856063}) 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.333+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3341077371}) 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.333+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.333+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2494856063}) 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.333+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1374270341}) 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.333+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.333+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2494856063}) 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.333+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3707551154}) 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3872629644}) 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1304133760}) 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3872629644}) 2024-09-15T01:40:57.481 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2128958097}) 2024-09-15T01:40:57.482 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.482 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3872629644}) 2024-09-15T01:40:57.482 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2834311394}) 2024-09-15T01:40:57.482 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.482 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3872629644}) 2024-09-15T01:40:57.482 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2677769125}) 2024-09-15T01:40:57.482 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.482 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3872629644}) 2024-09-15T01:40:57.482 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2451417}) 2024-09-15T01:40:57.482 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.482 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3872629644}) 2024-09-15T01:40:57.482 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4162495801}) 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3872629644}) 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1249932009}) 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3872629644}) 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.334+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500497995}) 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1212756799}) 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3085553428}) 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1212756799}) 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1125633632}) 2024-09-15T01:40:57.483 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.484 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1212756799}) 2024-09-15T01:40:57.484 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=573866806}) 2024-09-15T01:40:57.484 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.484 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1212756799}) 2024-09-15T01:40:57.484 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3712338138}) 2024-09-15T01:40:57.484 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.484 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1212756799}) 2024-09-15T01:40:57.484 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3240310529}) 2024-09-15T01:40:57.484 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.484 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1212756799}) 2024-09-15T01:40:57.484 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=127439141}) 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1212756799}) 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4177009566}) 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1212756799}) 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.335+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1836214994}) 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2099642297}) 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4278833312}) 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2099642297}) 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1811218080}) 2024-09-15T01:40:57.485 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.486 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2099642297}) 2024-09-15T01:40:57.486 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3363037125}) 2024-09-15T01:40:57.486 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.486 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2099642297}) 2024-09-15T01:40:57.486 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=67367539}) 2024-09-15T01:40:57.486 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.486 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2099642297}) 2024-09-15T01:40:57.486 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2079543805}) 2024-09-15T01:40:57.486 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.486 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.336+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2099642297}) 2024-09-15T01:40:57.486 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.337+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4226972503}) 2024-09-15T01:40:57.486 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.337+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.337+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2099642297}) 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.337+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2541273622}) 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.337+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.337+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2099642297}) 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.337+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3113851451}) 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=937448484}) 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4128796956}) 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=937448484}) 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1369795261}) 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=937448484}) 2024-09-15T01:40:57.487 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1637078788}) 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=937448484}) 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1375113353}) 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=937448484}) 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3738124483}) 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=937448484}) 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=70696111}) 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=937448484}) 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2807922885}) 2024-09-15T01:40:57.488 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=937448484}) 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.338+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1341107125}) 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3509708947}) 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2665648251}) 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3509708947}) 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3901221954}) 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3509708947}) 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1689728309}) 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3509708947}) 2024-09-15T01:40:57.489 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=492439392}) 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3509708947}) 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1098560507}) 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3509708947}) 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=948923430}) 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3509708947}) 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2943826282}) 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3509708947}) 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.339+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1709118747}) 2024-09-15T01:40:57.490 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1273032376}) 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1364599592}) 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1273032376}) 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3426084528}) 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1273032376}) 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3182606415}) 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1273032376}) 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1852626739}) 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1273032376}) 2024-09-15T01:40:57.491 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2022718359}) 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1273032376}) 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2529052387}) 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1273032376}) 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1963441393}) 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1273032376}) 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.340+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=230456414}) 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.342+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.342+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {logm=1291367673,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2726691891,mon_config_key=1111662797,monmap=2850442478,osd_metadata=3185943182,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2301271107}) 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=53} crc {logm=3048322803,mdsmap=1531151340,mgr=2983835071,mgr_command_descs=3545737493,mgr_metadata=3150520333,mgrstat=2407862716,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2050129728}) 2024-09-15T01:40:57.492 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.493 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {logm=1291367673,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2726691891,mon_config_key=1111662797,monmap=2850442478,osd_metadata=3185943182,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2301271107}) 2024-09-15T01:40:57.493 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=9,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {logm=2702626401,mdsmap=1531151339,mgr=3506364876,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1266067822,mon_config_key=829636513,monmap=2850442478,osd_metadata=1202423041,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2022473590}) 2024-09-15T01:40:57.493 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.493 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {logm=1291367673,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2726691891,mon_config_key=1111662797,monmap=2850442478,osd_metadata=3185943182,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2301271107}) 2024-09-15T01:40:57.493 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=52} crc {logm=434417900,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1266067822,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=3226480218}) 2024-09-15T01:40:57.493 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.493 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {logm=1291367673,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2726691891,mon_config_key=1111662797,monmap=2850442478,osd_metadata=3185943182,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2301271107}) 2024-09-15T01:40:57.493 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=51} crc {logm=1291367673,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1266067822,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2238695407}) 2024-09-15T01:40:57.493 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.493 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {logm=1291367673,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2726691891,mon_config_key=1111662797,monmap=2850442478,osd_metadata=3185943182,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2301271107}) 2024-09-15T01:40:57.493 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=4,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {logm=1291367673,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3601892101,mon_config_key=1841931312,monmap=2850442478,osd_metadata=860431027,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2958379372}) 2024-09-15T01:40:57.493 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {logm=1291367673,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2726691891,mon_config_key=1111662797,monmap=2850442478,osd_metadata=3185943182,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2301271107}) 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=54} crc {logm=3203709191,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2300896919,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=956710772,osdmap=1772347529}) 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {logm=1291367673,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2726691891,mon_config_key=1111662797,monmap=2850442478,osd_metadata=3185943182,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2301271107}) 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=53} crc {logm=1291367673,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3093502506,mon_config_key=3476930649,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=1617330510}) 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {logm=1291367673,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2726691891,mon_config_key=1111662797,monmap=2850442478,osd_metadata=3185943182,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2301271107}) 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.343+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=10,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=52} crc {logm=1291367673,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1266067822,mon_config_key=1111662797,monmap=2850442478,osd_metadata=2302127439,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=1607093682}) 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.351+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.351+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=76} crc {osdmap=1755748616}) 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.351+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=71} crc {osdmap=1276472917}) 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.351+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.494 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.351+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=76} crc {osdmap=1755748616}) 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.351+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=75} crc {osdmap=657989930}) 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.351+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.351+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=76} crc {osdmap=1755748616}) 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.351+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=74} crc {osdmap=2573950966}) 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.351+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=76} crc {osdmap=1755748616}) 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=74} crc {osdmap=1130869851}) 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=76} crc {osdmap=1755748616}) 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=72} crc {osdmap=1923859250}) 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=76} crc {osdmap=1755748616}) 2024-09-15T01:40:57.495 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=75} crc {osdmap=1117797653}) 2024-09-15T01:40:57.496 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.496 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=76} crc {osdmap=1755748616}) 2024-09-15T01:40:57.496 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=72} crc {osdmap=3323532175}) 2024-09-15T01:40:57.496 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:40:57.496 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=76} crc {osdmap=1755748616}) 2024-09-15T01:40:57.496 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:40:57.352+0000 7fbcdddf1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=75} crc {osdmap=4217641228}) 2024-09-15T01:40:58.280 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:40:58.280 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:40:58.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.374+0000 7f7695f5d640 10 monclient: get_monmap_and_config 2024-09-15T01:40:58.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.374+0000 7f7695f5d640 10 monclient: build_initial_monmap 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.374+0000 7f7695f5d640 10 monclient: monmap: 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:58.375684+0000 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:58.375684+0000 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:58.377 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:58.378 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:58.378 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:58.378 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:58.378 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:58.378 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:58.378 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.375+0000 7f7695f5d640 1 Processor -- start 2024-09-15T01:40:58.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.375+0000 7f7695f5d640 1 -- start start 2024-09-15T01:40:58.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.375+0000 7f7695f5d640 10 monclient: init 2024-09-15T01:40:58.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:58.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 10 monclient: _add_conns ranks=[3,5,0,6,8,1,10,4,9,7,2] 2024-09-15T01:40:58.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76900a2930 0x7f7690132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 10 monclient(hunting): picked mon.noname-g con 0x7f76900a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:58.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769009faa0 0x7f769009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 10 monclient(hunting): picked mon.noname-h con 0x7f769009faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:58.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f76900a08c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769004bf60 con 0x7f76900a08c0 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769009e0f0 con 0x7f76900a2930 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f769009dd80 con 0x7f769009faa0 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f7695f5d640 10 monclient(hunting): authenticate will time out at 1263.126091s 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768ffff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f76900a08c0 0x7f76900a0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:47280/0 (socket says 172.21.15.178:47280) 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768ffff640 1 -- 172.21.15.178:0/2360605590 learned_addr learned my addr 172.21.15.178:0/2360605590 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768f7fe640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76900a2930 0x7f7690132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768effd640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769009faa0 0x7f769009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176636297 0 0) 0x7f769004bf60 con 0x7f76900a08c0 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient(hunting): my global_id is 34569 2024-09-15T01:40:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76780034e0 con 0x7f76900a08c0 2024-09-15T01:40:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768f7fe640 10 monclient(hunting): get_auth_request con 0x7f76900a2930 auth_method 0 2024-09-15T01:40:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768effd640 10 monclient(hunting): get_auth_request con 0x7f769009faa0 auth_method 0 2024-09-15T01:40:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3038527412 0 0) 0x7f76780034e0 con 0x7f76900a08c0 2024-09-15T01:40:58.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769009faa0 msgr2=0x7f769009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:58.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769009faa0 0x7f769009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76900a2930 msgr2=0x7f7690132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:58.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76900a2930 0x7f7690132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76900a2d00 con 0x7f76900a08c0 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f7688001c70 con 0x7f76900a08c0 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient: dump: 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.394 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.395 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.395 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.395 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.395 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.395 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.395 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.395 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.395 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.395 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.395 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.395 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.395 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.396 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:40:58.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:58.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 >> v1:172.21.15.178:6789/0 conn(0x7f76900a08c0 legacy=0x7f76900a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient: _add_conns ranks=[6,1,5,3,7,8,0,4,2] 2024-09-15T01:40:58.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7678004220 0x7f7678004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient(hunting): picked mon.d con 0x7f7678004220 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:40:58.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7678004d10 0x7f7678005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient(hunting): picked mon.f con 0x7f7678004d10 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:40:58.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769009faa0 0x7f767800bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient(hunting): picked mon.h con 0x7f769009faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:58.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769009e0f0 con 0x7f7678004d10 2024-09-15T01:40:58.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f769009dd80 con 0x7f769009faa0 2024-09-15T01:40:58.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f767800c290 con 0x7f7678004220 2024-09-15T01:40:58.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:58.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.376+0000 7f768e7fc640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:40:58.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7678004220 0x7f7678004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 10 monclient(hunting): get_auth_request con 0x7f7678004220 auth_method 0 2024-09-15T01:40:58.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:40:58.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768effd640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7678004d10 0x7f7678005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768effd640 10 monclient(hunting): get_auth_request con 0x7f7678004d10 auth_method 0 2024-09-15T01:40:58.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:40:58.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768ffff640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769009faa0 0x7f767800bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:40:58.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768ffff640 10 monclient(hunting): get_auth_request con 0x7f769009faa0 auth_method 0 2024-09-15T01:40:58.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:40:58.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768effd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:40:58.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:40:58.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 10 monclient(hunting): handle_auth_done global_id 34569 payload 293 2024-09-15T01:40:58.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 1 -- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769009faa0 msgr2=0x7f767800bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:58.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769009faa0 0x7f767800bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 1 -- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7678004d10 msgr2=0x7f7678005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:58.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7678004d10 0x7f7678005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:40:58.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 1 monclient: found mon.d 2024-09-15T01:40:58.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:58.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:40:58.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 1 -- 172.21.15.178:0/2360605590 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76780034e0 con 0x7f7678004220 2024-09-15T01:40:58.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:40:58.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f768f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:58.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f7695f5d640 5 monclient: authenticate success, global_id 34569 2024-09-15T01:40:58.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.377+0000 7f7695f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:58.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f768f7fe640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7678004220 0x7f7678004780 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7680009b70 tx=0x7f76800090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=328ad80b28b0cb52 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7680031880 con 0x7f7678004220 2024-09-15T01:40:58.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f768e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f768e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:40:58.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f768e7fc640 10 monclient: dump: 2024-09-15T01:40:58.401 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.401 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.401 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.401 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.401 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.401 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.401 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.401 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.401 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768002d050 con 0x7f7678004220 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f7695f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f768e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f768e7fc640 1 -- 172.21.15.178:0/2360605590 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7680031e80 con 0x7f7678004220 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f7695f5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:58.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f7695f5d640 4 set_mon_vals no callback set 2024-09-15T01:40:58.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f768e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f768e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:40:58.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f768e7fc640 10 monclient: dump: 2024-09-15T01:40:58.403 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.403 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.403 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.403 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.403 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.403 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.403 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f7695f5d640 10 monclient: shutdown 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f7695f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f7695f5d640 1 -- 172.21.15.178:0/2360605590 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7678004220 msgr2=0x7f7678004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f7695f5d640 1 --2- 172.21.15.178:0/2360605590 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7678004220 0x7f7678004780 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7680009b70 tx=0x7f76800090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f768ffff640 1 -- 172.21.15.178:0/2360605590 reap_dead start 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f7695f5d640 1 -- 172.21.15.178:0/2360605590 shutdown_connections 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f7695f5d640 1 -- 172.21.15.178:0/2360605590 >> 172.21.15.178:0/2360605590 conn(0x7f76900a5ed0 msgr2=0x7f76900a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f7695f5d640 1 -- 172.21.15.178:0/2360605590 shutdown_connections 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.378+0000 7f7695f5d640 1 -- 172.21.15.178:0/2360605590 wait complete. 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.379+0000 7f7695f5d640 10 monclient: build_initial_monmap 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.379+0000 7f7695f5d640 10 monclient: monmap: 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:58.405 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.379+0000 7f7695f5d640 1 Processor -- start 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.379+0000 7f7695f5d640 1 -- start start 2024-09-15T01:40:58.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.379+0000 7f7695f5d640 10 monclient: init 2024-09-15T01:40:58.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:58.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 10 monclient: _add_conns ranks=[1,3,5,0,4,2,8,6,7] 2024-09-15T01:40:58.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7678004220 0x7f76901317a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 10 monclient(hunting): picked mon.noname-f con 0x7f7678004220 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:40:58.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7690131ce0 0x7f769011e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 10 monclient(hunting): picked mon.noname-g con 0x7f7690131ce0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:58.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769011e8b0 0x7f769011ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 10 monclient(hunting): picked mon.noname-h con 0x7f769011e8b0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:58.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769009d630 con 0x7f7678004220 2024-09-15T01:40:58.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76900a2390 con 0x7f7690131ce0 2024-09-15T01:40:58.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76901427d0 con 0x7f769011e8b0 2024-09-15T01:40:58.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:58.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f7695f5d640 10 monclient(hunting): authenticate will time out at 1263.130256s 2024-09-15T01:40:58.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768effd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7690131ce0 0x7f769011e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7678004220 0x7f76901317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768effd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7690131ce0 0x7f769011e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:44194/0 (socket says 172.21.15.178:44194) 2024-09-15T01:40:58.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768effd640 1 -- 172.21.15.178:0/1876287765 learned_addr learned my addr 172.21.15.178:0/1876287765 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:58.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768effd640 10 monclient(hunting): get_auth_request con 0x7f7690131ce0 auth_method 0 2024-09-15T01:40:58.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 10 monclient(hunting): get_auth_request con 0x7f7678004220 auth_method 0 2024-09-15T01:40:58.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768ffff640 1 --2- 172.21.15.178:0/1876287765 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769011e8b0 0x7f769011ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768ffff640 10 monclient(hunting): get_auth_request con 0x7f769011e8b0 auth_method 0 2024-09-15T01:40:58.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 10 monclient(hunting): handle_auth_done global_id 7957 payload 824 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 1 -- 172.21.15.178:0/1876287765 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769011e8b0 msgr2=0x7f769011ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 1 --2- 172.21.15.178:0/1876287765 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769011e8b0 0x7f769011ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 1 -- 172.21.15.178:0/1876287765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7690131ce0 msgr2=0x7f769011e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 1 --2- 172.21.15.178:0/1876287765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7690131ce0 0x7f769011e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 1 monclient: found mon.noname-f 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 1 -- 172.21.15.178:0/1876287765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76901218f0 con 0x7f7678004220 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768effd640 1 --2- 172.21.15.178:0/1876287765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7690131ce0 0x7f769011e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.380+0000 7f768f7fe640 1 --2- 172.21.15.178:0/1876287765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7678004220 0x7f76901317a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7680031000 tx=0x7f7680035000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 1 -- 172.21.15.178:0/1876287765 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f768003ca40 con 0x7f7678004220 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 10 monclient: dump: 2024-09-15T01:40:58.411 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.412 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 10 monclient: _finish_auth 0 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 1 -- 172.21.15.178:0/1876287765 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7680040070 con 0x7f7678004220 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f7695f5d640 5 monclient: authenticate success, global_id 7957 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 1 -- 172.21.15.178:0/1876287765 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f768003cbe0 con 0x7f7678004220 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f7695f5d640 10 monclient: _renew_subs 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f7695f5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f7695f5d640 1 -- 172.21.15.178:0/1876287765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7690121bd0 con 0x7f7678004220 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:40:58.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 10 monclient: dump: 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.414 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f768dffb640 4 set_mon_vals no callback set 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f7695f5d640 10 monclient: _renew_subs 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f7695f5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f7695f5d640 1 -- 172.21.15.178:0/1876287765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7690120990 con 0x7f7678004220 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.381+0000 7f76777fe640 1 -- 172.21.15.178:0/1876287765 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7680042070 con 0x7f7678004220 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.382+0000 7f7695f5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.382+0000 7f7695f5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.382+0000 7f7695f5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.382+0000 7f7695f5d640 1 -- 172.21.15.178:0/1876287765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769004b720 con 0x7f7678004220 2024-09-15T01:40:58.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.382+0000 7f76777fe640 1 --2- 172.21.15.178:0/1876287765 >> v2:172.21.15.204:6812/28210 conn(0x7f767c0397b0 0x7f767c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.382+0000 7f76777fe640 1 -- 172.21.15.178:0/1876287765 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f768006ca00 con 0x7f7678004220 2024-09-15T01:40:58.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.382+0000 7f768effd640 1 --2- 172.21.15.178:0/1876287765 >> v2:172.21.15.204:6812/28210 conn(0x7f767c0397b0 0x7f767c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.382+0000 7f768effd640 10 monclient: get_auth_request con 0x7f767c0397b0 auth_method 0 2024-09-15T01:40:58.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.382+0000 7f768effd640 1 --2- 172.21.15.178:0/1876287765 >> v2:172.21.15.204:6812/28210 conn(0x7f767c0397b0 0x7f767c03bc60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7684009fd0 tx=0x7f76840090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.386+0000 7f76777fe640 1 -- 172.21.15.178:0/1876287765 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f768004e3a0 con 0x7f7678004220 2024-09-15T01:40:58.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.386+0000 7f76777fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:58.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.386+0000 7f76777fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:58.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.598+0000 7f7695f5d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:40:58.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.599+0000 7f7695f5d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:40:58.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.599+0000 7f7695f5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:40:58.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.599+0000 7f7695f5d640 1 -- 172.21.15.178:0/1876287765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f769009cef0 con 0x7f7678004220 2024-09-15T01:40:58.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.600+0000 7f76777fe640 1 -- 172.21.15.178:0/1876287765 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7f768004a070 con 0x7f7678004220 2024-09-15T01:40:58.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.600+0000 7f76777fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:40:58.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.600+0000 7f76777fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:58.602 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":110,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:40:58.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.601+0000 7f7695f5d640 1 -- 172.21.15.178:0/1876287765 >> v2:172.21.15.204:6812/28210 conn(0x7f767c0397b0 msgr2=0x7f767c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.601+0000 7f7695f5d640 1 --2- 172.21.15.178:0/1876287765 >> v2:172.21.15.204:6812/28210 conn(0x7f767c0397b0 0x7f767c03bc60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7684009fd0 tx=0x7f76840090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.601+0000 7f7695f5d640 10 monclient: shutdown 2024-09-15T01:40:58.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.601+0000 7f7695f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:58.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.601+0000 7f7695f5d640 1 -- 172.21.15.178:0/1876287765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7678004220 msgr2=0x7f76901317a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.601+0000 7f7695f5d640 1 --2- 172.21.15.178:0/1876287765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7678004220 0x7f76901317a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7680031000 tx=0x7f7680035000 comp rx=0 tx=0).stop 2024-09-15T01:40:58.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.602+0000 7f7695f5d640 1 -- 172.21.15.178:0/1876287765 shutdown_connections 2024-09-15T01:40:58.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.602+0000 7f7695f5d640 1 --2- 172.21.15.178:0/1876287765 >> v2:172.21.15.204:6812/28210 conn(0x7f767c0397b0 0x7f767c03bc60 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.602+0000 7f7695f5d640 1 --2- 172.21.15.178:0/1876287765 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f769011e8b0 0x7f769011ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.602+0000 7f7695f5d640 1 --2- 172.21.15.178:0/1876287765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7690131ce0 0x7f769011e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.602+0000 7f7695f5d640 1 --2- 172.21.15.178:0/1876287765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7678004220 0x7f76901317a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.602+0000 7f7695f5d640 1 -- 172.21.15.178:0/1876287765 >> 172.21.15.178:0/1876287765 conn(0x7f76900a5ed0 msgr2=0x7f769009e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:58.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.602+0000 7f7695f5d640 1 -- 172.21.15.178:0/1876287765 shutdown_connections 2024-09-15T01:40:58.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.602+0000 7f7695f5d640 1 -- 172.21.15.178:0/1876287765 wait complete. 2024-09-15T01:40:58.614 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:40:58.614 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T01:40:58.614 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:40:58.614 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:40:58.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.705+0000 7fd0862c0640 10 monclient: get_monmap_and_config 2024-09-15T01:40:58.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.705+0000 7fd0862c0640 10 monclient: build_initial_monmap 2024-09-15T01:40:58.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.705+0000 7fd0862c0640 10 monclient: monmap: 2024-09-15T01:40:58.708 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:58.708 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.708 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:58.706429+0000 2024-09-15T01:40:58.708 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:58.706429+0000 2024-09-15T01:40:58.708 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:58.708 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.708 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:58.708 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:58.708 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:58.708 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:58.708 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:58.709 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:58.709 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:58.709 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:58.709 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:58.709 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:58.709 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:58.709 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 1 Processor -- start 2024-09-15T01:40:58.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 1 -- start start 2024-09-15T01:40:58.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 10 monclient: init 2024-09-15T01:40:58.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:58.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 10 monclient: _add_conns ranks=[5,6,4,10,0,7,1,3,9,8,2] 2024-09-15T01:40:58.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd0800a2930 0x7fd080132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 10 monclient(hunting): picked mon.noname-h con 0x7fd0800a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:58.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd08009faa0 0x7fd08009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 10 monclient(hunting): picked mon.noname-d con 0x7fd08009faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:40:58.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd0800a08c0 0x7fd0800a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 10 monclient(hunting): picked mon.noname-c con 0x7fd0800a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:58.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd08009e0f0 con 0x7fd0800a08c0 2024-09-15T01:40:58.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd08009dd80 con 0x7fd0800a2930 2024-09-15T01:40:58.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd0800a2d00 con 0x7fd08009faa0 2024-09-15T01:40:58.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:58.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd0862c0640 10 monclient(hunting): authenticate will time out at 1263.456785s 2024-09-15T01:40:58.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd08485d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd0800a08c0 0x7fd0800a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd07f7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd08009faa0 0x7fd08009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd08485d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd0800a08c0 0x7fd0800a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:57124/0 (socket says 172.21.15.178:57124) 2024-09-15T01:40:58.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd08485d640 1 -- 172.21.15.178:0/4196820295 learned_addr learned my addr 172.21.15.178:0/4196820295 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:58.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd07f7fe640 10 monclient(hunting): get_auth_request con 0x7fd08009faa0 auth_method 0 2024-09-15T01:40:58.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd07f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd07f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd07f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd08485d640 10 monclient(hunting): get_auth_request con 0x7fd0800a08c0 auth_method 0 2024-09-15T01:40:58.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd08485d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd08485d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.706+0000 7fd08485d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07ffff640 1 --2- 172.21.15.178:0/4196820295 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd0800a2930 0x7fd080132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd08485d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd08485d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd08485d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07ffff640 10 monclient(hunting): get_auth_request con 0x7fd0800a2930 auth_method 0 2024-09-15T01:40:58.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_done global_id 25008 payload 274 2024-09-15T01:40:58.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 1 -- 172.21.15.178:0/4196820295 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd0800a2930 msgr2=0x7fd080132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:58.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 1 --2- 172.21.15.178:0/4196820295 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd0800a2930 0x7fd080132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 1 -- 172.21.15.178:0/4196820295 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd0800a08c0 msgr2=0x7fd0800a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:58.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 1 --2- 172.21.15.178:0/4196820295 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd0800a08c0 0x7fd0800a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:40:58.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 1 monclient: found mon.noname-d 2024-09-15T01:40:58.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:58.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:40:58.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 1 -- 172.21.15.178:0/4196820295 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08004bf60 con 0x7fd08009faa0 2024-09-15T01:40:58.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd08485d640 1 --2- 172.21.15.178:0/4196820295 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd0800a08c0 0x7fd0800a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:58.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.707+0000 7fd07f7fe640 1 --2- 172.21.15.178:0/4196820295 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd08009faa0 0x7fd08009fe70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd074009880 tx=0x7fd0740090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=6d9da24b599f4502 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd07effd640 1 -- 172.21.15.178:0/4196820295 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd07402abb0 con 0x7fd08009faa0 2024-09-15T01:40:58.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd07effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd07effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:40:58.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd07effd640 10 monclient: dump: 2024-09-15T01:40:58.727 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.727 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.727 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.727 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.728 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.728 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.728 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.728 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.728 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.728 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.728 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.728 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.728 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.728 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.728 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.729 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd07effd640 10 monclient: _finish_auth 0 2024-09-15T01:40:58.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd07effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:58.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd07effd640 1 -- 172.21.15.178:0/4196820295 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07402ad30 con 0x7fd08009faa0 2024-09-15T01:40:58.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 5 monclient: authenticate success, global_id 25008 2024-09-15T01:40:58.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:58.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd07effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:58.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:58.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd07effd640 1 -- 172.21.15.178:0/4196820295 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd074031ca0 con 0x7fd08009faa0 2024-09-15T01:40:58.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 4 set_mon_vals no callback set 2024-09-15T01:40:58.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd07effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd07effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:40:58.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd07effd640 10 monclient: dump: 2024-09-15T01:40:58.730 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.730 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.730 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.730 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.730 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.730 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.730 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.731 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.731 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.731 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.731 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.731 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.731 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.731 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.731 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.732 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 10 monclient: shutdown 2024-09-15T01:40:58.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:58.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 1 -- 172.21.15.178:0/4196820295 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd08009faa0 msgr2=0x7fd08009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 1 --2- 172.21.15.178:0/4196820295 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd08009faa0 0x7fd08009fe70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd074009880 tx=0x7fd0740090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 1 -- 172.21.15.178:0/4196820295 shutdown_connections 2024-09-15T01:40:58.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 1 --2- 172.21.15.178:0/4196820295 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd0800a08c0 0x7fd0800a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 1 --2- 172.21.15.178:0/4196820295 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd08009faa0 0x7fd08009fe70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 1 --2- 172.21.15.178:0/4196820295 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd0800a2930 0x7fd080132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 1 -- 172.21.15.178:0/4196820295 >> 172.21.15.178:0/4196820295 conn(0x7fd0800a5ed0 msgr2=0x7fd0800a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:58.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 1 -- 172.21.15.178:0/4196820295 shutdown_connections 2024-09-15T01:40:58.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.708+0000 7fd0862c0640 1 -- 172.21.15.178:0/4196820295 wait complete. 2024-09-15T01:40:58.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.709+0000 7fd0862c0640 10 monclient: build_initial_monmap 2024-09-15T01:40:58.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.709+0000 7fd0862c0640 10 monclient: monmap: 2024-09-15T01:40:58.733 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:58.733 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:58.733 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:58.733 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:58.733 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:58.733 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:58.733 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:58.734 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:58.734 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:58.734 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:58.734 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:58.734 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:58.734 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:58.734 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:58.734 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:58.734 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.709+0000 7fd0862c0640 1 Processor -- start 2024-09-15T01:40:58.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.709+0000 7fd0862c0640 1 -- start start 2024-09-15T01:40:58.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.709+0000 7fd0862c0640 10 monclient: init 2024-09-15T01:40:58.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:58.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 10 monclient: _add_conns ranks=[0,3,7,6,5,4,8,1,2] 2024-09-15T01:40:58.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08009faa0 0x7fd08009b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 10 monclient(hunting): picked mon.noname-a con 0x7fd08009faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:40:58.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0800a08c0 0x7fd080131bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 10 monclient(hunting): picked mon.noname-g con 0x7fd0800a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:58.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd0800a2930 0x7fd080132110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 10 monclient(hunting): picked mon.noname-i con 0x7fd0800a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:40:58.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08011efd0 con 0x7fd08009faa0 2024-09-15T01:40:58.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd080142ed0 con 0x7fd0800a08c0 2024-09-15T01:40:58.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd0800a2390 con 0x7fd0800a2930 2024-09-15T01:40:58.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:58.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd0862c0640 10 monclient(hunting): authenticate will time out at 1263.460237s 2024-09-15T01:40:58.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08009faa0 0x7fd08009b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08009faa0 0x7fd08009b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51232/0 (socket says 172.21.15.178:51232) 2024-09-15T01:40:58.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 1 -- 172.21.15.178:0/1898747827 learned_addr learned my addr 172.21.15.178:0/1898747827 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:58.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 10 monclient(hunting): get_auth_request con 0x7fd08009faa0 auth_method 0 2024-09-15T01:40:58.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07f7fe640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0800a08c0 0x7fd080131bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07f7fe640 10 monclient(hunting): get_auth_request con 0x7fd0800a08c0 auth_method 0 2024-09-15T01:40:58.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd08485d640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd0800a2930 0x7fd080132110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd08485d640 10 monclient(hunting): get_auth_request con 0x7fd0800a2930 auth_method 0 2024-09-15T01:40:58.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd08485d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd08485d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd08485d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd08485d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd08485d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd08485d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 10 monclient(hunting): handle_auth_done global_id 34578 payload 824 2024-09-15T01:40:58.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 1 -- 172.21.15.178:0/1898747827 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd0800a2930 msgr2=0x7fd080132110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd0800a2930 0x7fd080132110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 1 -- 172.21.15.178:0/1898747827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0800a08c0 msgr2=0x7fd080131bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0800a08c0 0x7fd080131bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:40:58.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 1 monclient: found mon.noname-a 2024-09-15T01:40:58.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:58.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:40:58.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 1 -- 172.21.15.178:0/1898747827 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd078009ae0 con 0x7fd08009faa0 2024-09-15T01:40:58.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07f7fe640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0800a08c0 0x7fd080131bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:58.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd07ffff640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08009faa0 0x7fd08009b560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd07000ea50 tx=0x7fd0700090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd07000ce90 con 0x7fd08009faa0 2024-09-15T01:40:58.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd067fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd067fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:40:58.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd067fff640 10 monclient: dump: 2024-09-15T01:40:58.740 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.740 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.740 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.740 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.740 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.740 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.740 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.741 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.741 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.741 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.741 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.741 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.741 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.741 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.741 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.741 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd067fff640 10 monclient: _finish_auth 0 2024-09-15T01:40:58.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.710+0000 7fd067fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:58.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd070009600 con 0x7fd08009faa0 2024-09-15T01:40:58.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd0862c0640 5 monclient: authenticate success, global_id 34578 2024-09-15T01:40:58.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd0862c0640 10 monclient: _renew_subs 2024-09-15T01:40:58.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd0862c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:40:58.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd0862c0640 1 -- 172.21.15.178:0/1898747827 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd074009390 con 0x7fd08009faa0 2024-09-15T01:40:58.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd067fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:58.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd0700138f0 con 0x7fd08009faa0 2024-09-15T01:40:58.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd067fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd067fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:40:58.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd067fff640 10 monclient: dump: 2024-09-15T01:40:58.742 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.742 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.743 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.743 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.743 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.743 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.743 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.743 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.743 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.743 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.743 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.743 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.744 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.744 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.744 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.744 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd0862c0640 10 monclient: _renew_subs 2024-09-15T01:40:58.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd0862c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:40:58.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd0862c0640 1 -- 172.21.15.178:0/1898747827 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08011ecf0 con 0x7fd08009faa0 2024-09-15T01:40:58.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd07e7fc640 4 set_mon_vals no callback set 2024-09-15T01:40:58.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd0862c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:58.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd0862c0640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080142490 0x7fd08011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd0862c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd0862c0640 1 -- 172.21.15.178:0/1898747827 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080142860 con 0x7fd080142490 2024-09-15T01:40:58.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd070013470 con 0x7fd08009faa0 2024-09-15T01:40:58.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd07f7fe640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080142490 0x7fd08011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd07f7fe640 10 monclient: get_auth_request con 0x7fd080142490 auth_method 0 2024-09-15T01:40:58.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd07f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd07f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.711+0000 7fd07f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 1 --2- 172.21.15.178:0/1898747827 >> v2:172.21.15.204:6812/28210 conn(0x7fd06c0396e0 0x7fd06c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd07004bf30 con 0x7fd08009faa0 2024-09-15T01:40:58.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_done global_id 34587 payload 824 2024-09-15T01:40:58.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd07f7fe640 1 -- 172.21.15.178:0/1898747827 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd08004c4a0 con 0x7fd080142490 2024-09-15T01:40:58.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd08485d640 1 --2- 172.21.15.178:0/1898747827 >> v2:172.21.15.204:6812/28210 conn(0x7fd06c0396e0 0x7fd06c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd08485d640 10 monclient: get_auth_request con 0x7fd06c0396e0 auth_method 0 2024-09-15T01:40:58.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd07f7fe640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080142490 0x7fd08011c530 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd074009850 tx=0x7fd074032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd07402aea0 con 0x7fd080142490 2024-09-15T01:40:58.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:40:58.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 10 monclient: dump: 2024-09-15T01:40:58.746 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.747 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.747 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.747 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.747 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.747 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.747 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.747 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.747 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.747 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.747 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.747 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.747 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.748 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.748 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.748 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd08485d640 1 --2- 172.21.15.178:0/1898747827 >> v2:172.21.15.204:6812/28210 conn(0x7fd06c0396e0 0x7fd06c03bb90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd0780072f0 tx=0x7fd07800b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fd07402abb0 con 0x7fd080142490 2024-09-15T01:40:58.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:58.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:58.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080142490 msgr2=0x7fd08011c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080142490 0x7fd08011c530 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd074009850 tx=0x7fd074032000 comp rx=0 tx=0).stop 2024-09-15T01:40:58.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080142490 msgr2=0x7fd08011c530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:58.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.712+0000 7fd067fff640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080142490 0x7fd08011c530 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd074009850 tx=0x7fd074032000 comp rx=0 tx=0).stop 2024-09-15T01:40:58.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd0862c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:58.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd0862c0640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08011ca70 0x7fd08011ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd0862c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd0862c0640 1 -- 172.21.15.178:0/1898747827 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08011f440 con 0x7fd08011ca70 2024-09-15T01:40:58.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd07f7fe640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08011ca70 0x7fd08011ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd07f7fe640 10 monclient: get_auth_request con 0x7fd08011ca70 auth_method 0 2024-09-15T01:40:58.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd07f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd07f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd07f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd07f7fe640 10 monclient(hunting): handle_auth_done global_id 34596 payload 824 2024-09-15T01:40:58.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd07f7fe640 1 -- 172.21.15.178:0/1898747827 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd08004bc80 con 0x7fd08011ca70 2024-09-15T01:40:58.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.718+0000 7fd07f7fe640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08011ca70 0x7fd08011ce70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd074009880 tx=0x7fd07403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd07404c070 con 0x7fd08011ca70 2024-09-15T01:40:58.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd067fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd067fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:40:58.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd067fff640 10 monclient: dump: 2024-09-15T01:40:58.750 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.750 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.750 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.750 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.750 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.751 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.751 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.751 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.751 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.751 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.751 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.751 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.751 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.751 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.751 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.751 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fd07403c440 con 0x7fd08011ca70 2024-09-15T01:40:58.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd067fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:58.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd067fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:58.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08011ca70 msgr2=0x7fd08011ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd067fff640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08011ca70 0x7fd08011ce70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd074009880 tx=0x7fd07403d000 comp rx=0 tx=0).stop 2024-09-15T01:40:58.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd067fff640 1 -- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08011ca70 msgr2=0x7fd08011ce70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:58.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd067fff640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08011ca70 0x7fd08011ce70 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd074009880 tx=0x7fd07403d000 comp rx=0 tx=0).stop 2024-09-15T01:40:58.752 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:58.753 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:58.753 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:58.753 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:58.753 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:58.753 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:58.753 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:58.753 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:58.753 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:58.753 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:58.753 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:58.753 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:58.754 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:58.754 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:58.754 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:58.754 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:58.754 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:40:58.754 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:58.754 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:58.754 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:58.754 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:58.754 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:58.754 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:58.755 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:58.755 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:58.755 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:58.755 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:58.755 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:58.755 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:58.755 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:58.755 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:58.755 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:58.755 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:58.755 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:58.756 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:58.756 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:58.756 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:58.756 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:58.756 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:58.756 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:58.756 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:58.756 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.756 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.756 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:58.756 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:58.757 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:58.757 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:58.757 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:58.757 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:58.757 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:58.757 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:58.757 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:58.757 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:58.757 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:58.757 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:58.757 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:58.758 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:58.758 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:58.758 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:58.758 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:58.758 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:58.758 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:58.758 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:58.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:58.758 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:58.758 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:58.758 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:58.759 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:58.759 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:58.759 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:58.759 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:58.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.759 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:58.759 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.759 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:58.759 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:58.759 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.759 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.759 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.760 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.760 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:58.760 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.760 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.760 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.760 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:58.760 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.760 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.760 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.762 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.762 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.762 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:58.762 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.762 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.762 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.762 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:58.762 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.762 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.762 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:58.763 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:58.763 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.763 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.763 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.763 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:58.763 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:58.763 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.764 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:58.764 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.764 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:58.764 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.764 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.764 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:58.765 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:58.765 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.765 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.765 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.765 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:58.765 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:58.765 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.766 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:58.766 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.766 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:58.766 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.766 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.766 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:58.767 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:58.767 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.767 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.767 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.767 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:58.767 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:58.767 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.768 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:58.768 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.768 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:58.768 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.768 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.768 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:58.769 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:58.769 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.769 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.769 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.769 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.769 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:58.769 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:58.769 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.769 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.769 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.770 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:58.770 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.770 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:58.770 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.770 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.770 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.771 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.772 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:58.772 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.772 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:58.772 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.772 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.772 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:58.773 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:58.773 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.773 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.773 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.773 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:58.773 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:58.773 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:58.774 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:58.774 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.774 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.774 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:58.775 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:58.775 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.775 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.775 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.775 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:58.775 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:58.775 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.776 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:58.776 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.776 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.776 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:58.776 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.776 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.776 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:58.777 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:58.777 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.777 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.777 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.777 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.777 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.777 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:58.777 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:58.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.778 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:58.778 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:58.778 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:58.778 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.778 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:58.778 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:58.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.778 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:58.778 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:58.778 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:40:58.778 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.778 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.779 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:58.779 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd0862c0640 1 -- 172.21.15.178:0/1898747827 >> v2:172.21.15.204:6812/28210 conn(0x7fd06c0396e0 msgr2=0x7fd06c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd0862c0640 1 --2- 172.21.15.178:0/1898747827 >> v2:172.21.15.204:6812/28210 conn(0x7fd06c0396e0 0x7fd06c03bb90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd0780072f0 tx=0x7fd07800b040 comp rx=0 tx=0).stop 2024-09-15T01:40:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd0862c0640 10 monclient: shutdown 2024-09-15T01:40:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd0862c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd0862c0640 1 -- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08009faa0 msgr2=0x7fd08009b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd0862c0640 1 --2- 172.21.15.178:0/1898747827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08009faa0 0x7fd08009b560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd07000ea50 tx=0x7fd0700090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd08485d640 1 -- 172.21.15.178:0/1898747827 reap_dead start 2024-09-15T01:40:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd08485d640 1 -- 172.21.15.178:0/1898747827 reap_dead start 2024-09-15T01:40:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd0862c0640 1 -- 172.21.15.178:0/1898747827 shutdown_connections 2024-09-15T01:40:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd0862c0640 1 -- 172.21.15.178:0/1898747827 >> 172.21.15.178:0/1898747827 conn(0x7fd0800a5ed0 msgr2=0x7fd0800a5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd0862c0640 1 -- 172.21.15.178:0/1898747827 shutdown_connections 2024-09-15T01:40:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.719+0000 7fd0862c0640 1 -- 172.21.15.178:0/1898747827 wait complete. 2024-09-15T01:40:58.782 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:58.782 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:40:58.782 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:40:58.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.873+0000 7f5bfef9d640 10 monclient: get_monmap_and_config 2024-09-15T01:40:58.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.873+0000 7f5bfef9d640 10 monclient: build_initial_monmap 2024-09-15T01:40:58.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.873+0000 7f5bfef9d640 10 monclient: monmap: 2024-09-15T01:40:58.875 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:58.875 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.875 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:58.874098+0000 2024-09-15T01:40:58.875 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:58.874098+0000 2024-09-15T01:40:58.875 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:58.876 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.873+0000 7f5bfef9d640 1 Processor -- start 2024-09-15T01:40:58.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 1 -- start start 2024-09-15T01:40:58.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 10 monclient: init 2024-09-15T01:40:58.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:58.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 10 monclient: _add_conns ranks=[10,1,4,8,3,2,5,7,9,0,6] 2024-09-15T01:40:58.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5bf80a2930 0x7f5bf8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 10 monclient(hunting): picked mon.noname-f con 0x7f5bf80a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:40:58.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5bf809faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:40:58.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5bf809fe70 0x7f5bf80a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 10 monclient(hunting): picked mon.noname-c con 0x7f5bf809fe70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:58.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf804bf60 con 0x7f5bf809faa0 2024-09-15T01:40:58.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf809dd80 con 0x7f5bf80a2930 2024-09-15T01:40:58.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80a2d00 con 0x7f5bf809fe70 2024-09-15T01:40:58.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:58.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfef9d640 10 monclient(hunting): authenticate will time out at 1263.624553s 2024-09-15T01:40:58.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5bf809fe70 0x7f5bf80a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5bf809fe70 0x7f5bf80a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:57136/0 (socket says 172.21.15.178:57136) 2024-09-15T01:40:58.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 1 -- 172.21.15.178:0/226488096 learned_addr learned my addr 172.21.15.178:0/226488096 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:58.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 10 monclient(hunting): get_auth_request con 0x7f5bf809fe70 auth_method 0 2024-09-15T01:40:58.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfcd39640 1 --2- 172.21.15.178:0/226488096 >> v2:172.21.15.204:3300/0 conn(0x7f5bf80a2930 0x7f5bf8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfcd39640 10 monclient(hunting): get_auth_request con 0x7f5bf80a2930 auth_method 0 2024-09-15T01:40:58.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfcd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfcd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfcd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 10 monclient(hunting): handle_auth_done global_id 44221 payload 274 2024-09-15T01:40:58.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 1 -- 172.21.15.178:0/226488096 >> v2:172.21.15.204:3300/0 conn(0x7f5bf80a2930 msgr2=0x7f5bf8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:58.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 1 --2- 172.21.15.178:0/226488096 >> v2:172.21.15.204:3300/0 conn(0x7f5bf80a2930 0x7f5bf8132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 1 -- 172.21.15.178:0/226488096 >> v1:172.21.15.204:6789/0 conn(0x7f5bf809faa0 legacy=0x7f5bf80a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 10 monclient: _finish_hunting 0 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 1 monclient: found mon.noname-c 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.874+0000 7f5bfd53a640 1 -- 172.21.15.178:0/226488096 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf8133440 con 0x7f5bf809fe70 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bfd53a640 1 --2- 172.21.15.178:0/226488096 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5bf809fe70 0x7f5bf80a0fd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009910 tx=0x7f5be80091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=9997795568e23c4e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bf77fe640 1 -- 172.21.15.178:0/226488096 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5be802ad20 con 0x7f5bf809fe70 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bf77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bf77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bf77fe640 10 monclient: dump: 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.889 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.890 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bf77fe640 10 monclient: _finish_auth 0 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bf77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bfef9d640 5 monclient: authenticate success, global_id 44221 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bfef9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bf77fe640 1 -- 172.21.15.178:0/226488096 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be8031820 con 0x7f5bf809fe70 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bf77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bfef9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bfef9d640 4 set_mon_vals no callback set 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bf77fe640 1 -- 172.21.15.178:0/226488096 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5be802aea0 con 0x7f5bf809fe70 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bf77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bf77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.875+0000 7f5bf77fe640 10 monclient: dump: 2024-09-15T01:40:58.891 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.892 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.876+0000 7f5bfef9d640 10 monclient: shutdown 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.876+0000 7f5bfef9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.876+0000 7f5bfef9d640 1 -- 172.21.15.178:0/226488096 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5bf809fe70 msgr2=0x7f5bf80a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.876+0000 7f5bfef9d640 1 --2- 172.21.15.178:0/226488096 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5bf809fe70 0x7f5bf80a0fd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009910 tx=0x7f5be80091d0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.876+0000 7f5bfef9d640 1 -- 172.21.15.178:0/226488096 shutdown_connections 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.876+0000 7f5bfef9d640 1 --2- 172.21.15.178:0/226488096 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f5bf809fe70 0x7f5bf80a0fd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.876+0000 7f5bfef9d640 1 --2- 172.21.15.178:0/226488096 >> v2:172.21.15.204:3300/0 conn(0x7f5bf80a2930 0x7f5bf8132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.876+0000 7f5bfef9d640 1 -- 172.21.15.178:0/226488096 >> 172.21.15.178:0/226488096 conn(0x7f5bf80a5ed0 msgr2=0x7f5bf80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.876+0000 7f5bfef9d640 1 -- 172.21.15.178:0/226488096 shutdown_connections 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.876+0000 7f5bfef9d640 1 -- 172.21.15.178:0/226488096 wait complete. 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.876+0000 7f5bfef9d640 10 monclient: build_initial_monmap 2024-09-15T01:40:58.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.876+0000 7f5bfef9d640 10 monclient: monmap: 2024-09-15T01:40:58.894 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:58.894 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:58.894 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:58.894 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:58.894 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:58.894 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:58.894 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:58.894 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:58.894 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:58.894 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:58.894 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:58.895 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:58.895 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:58.895 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:58.895 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:58.895 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 1 Processor -- start 2024-09-15T01:40:58.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 1 -- start start 2024-09-15T01:40:58.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 10 monclient: init 2024-09-15T01:40:58.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:58.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 10 monclient: _add_conns ranks=[5,8,1,3,4,2,7,0,6] 2024-09-15T01:40:58.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5bf809faa0 0x7f5bf809b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 10 monclient(hunting): picked mon.noname-h con 0x7f5bf809faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5bf809fe70 0x7f5bf811ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 10 monclient(hunting): picked mon.noname-e con 0x7f5bf809fe70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf80a2930 0x7f5bf811cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 10 monclient(hunting): picked mon.noname-f con 0x7f5bf80a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80a2390 con 0x7f5bf80a2930 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf809d630 con 0x7f5bf809faa0 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf811d480 con 0x7f5bf809fe70 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfef9d640 10 monclient(hunting): authenticate will time out at 1263.627817s 2024-09-15T01:40:58.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bf7fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5bf809fe70 0x7f5bf811ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bf7fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5bf809fe70 0x7f5bf811ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:39054/0 (socket says 172.21.15.178:39054) 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bf7fff640 1 -- 172.21.15.178:0/467695493 learned_addr learned my addr 172.21.15.178:0/467695493 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bf7fff640 10 monclient(hunting): get_auth_request con 0x7f5bf809fe70 auth_method 0 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bf7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bf7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfd53a640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf80a2930 0x7f5bf811cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfcd39640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5bf809faa0 0x7f5bf809b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.877+0000 7f5bfd53a640 10 monclient(hunting): get_auth_request con 0x7f5bf80a2930 auth_method 0 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfd53a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfd53a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfd53a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfcd39640 10 monclient(hunting): get_auth_request con 0x7f5bf809faa0 auth_method 0 2024-09-15T01:40:58.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfcd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfcd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfcd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfd53a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfd53a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfd53a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfcd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfcd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfcd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 10 monclient(hunting): handle_auth_done global_id 34442 payload 824 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 1 -- 172.21.15.178:0/467695493 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5bf809faa0 msgr2=0x7f5bf809b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5bf809faa0 0x7f5bf809b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 1 -- 172.21.15.178:0/467695493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf80a2930 msgr2=0x7f5bf811cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf80a2930 0x7f5bf811cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 1 monclient: found mon.noname-e 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 1 -- 172.21.15.178:0/467695493 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be8009450 con 0x7f5bf809fe70 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf7fff640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5bf809fe70 0x7f5bf811ca00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5be000c8a0 tx=0x7f5be0001630 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5be000ce90 con 0x7f5bf809fe70 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf4ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf4ff9640 10 monclient: dump: 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.900 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf4ff9640 10 monclient: _finish_auth 0 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfef9d640 5 monclient: authenticate success, global_id 34442 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be000acd0 con 0x7f5bf809fe70 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfef9d640 10 monclient: _renew_subs 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5be000ae40 con 0x7f5bf809fe70 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfef9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:58.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfef9d640 1 -- 172.21.15.178:0/467695493 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf811d8a0 con 0x7f5bf809fe70 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf6ffd640 4 set_mon_vals no callback set 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bf4ff9640 10 monclient: dump: 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.902 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfef9d640 10 monclient: _renew_subs 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfef9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.878+0000 7f5bfef9d640 1 -- 172.21.15.178:0/467695493 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf811eec0 con 0x7f5bf809fe70 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5be0003030 con 0x7f5bf809fe70 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfef9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfef9d640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf811df60 0x7f5bf8142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfef9d640 1 -- 172.21.15.178:0/467695493 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf804c4a0 con 0x7f5bf811df60 2024-09-15T01:40:58.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfcd39640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf811df60 0x7f5bf8142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfcd39640 10 monclient: get_auth_request con 0x7f5bf811df60 auth_method 0 2024-09-15T01:40:58.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfcd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfcd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfcd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bf4ff9640 1 --2- 172.21.15.178:0/467695493 >> v2:172.21.15.204:6812/28210 conn(0x7f5bd80396e0 0x7f5bd803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfcd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfcd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfcd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f5be004baf0 con 0x7f5bf809fe70 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfd53a640 1 --2- 172.21.15.178:0/467695493 >> v2:172.21.15.204:6812/28210 conn(0x7f5bd80396e0 0x7f5bd803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfd53a640 10 monclient: get_auth_request con 0x7f5bd80396e0 auth_method 0 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfcd39640 10 monclient(hunting): handle_auth_done global_id 34463 payload 824 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.879+0000 7f5bfcd39640 1 -- 172.21.15.178:0/467695493 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bf805ed70 con 0x7f5bf811df60 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bfcd39640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf811df60 0x7f5bf8142180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00108b0 tx=0x7f5bf0008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5bf0014430 con 0x7f5bf811df60 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bf4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bf4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bf4ff9640 10 monclient: dump: 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.905 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.906 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bfd53a640 1 --2- 172.21.15.178:0/467695493 >> v2:172.21.15.204:6812/28210 conn(0x7f5bd80396e0 0x7f5bd803bb90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5be8034000 tx=0x7f5be8032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f5bf000eb40 con 0x7f5bf811df60 2024-09-15T01:40:58.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bf4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:58.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bf4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:58.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf811df60 msgr2=0x7f5bf8142180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bf4ff9640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf811df60 0x7f5bf8142180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00108b0 tx=0x7f5bf0008040 comp rx=0 tx=0).stop 2024-09-15T01:40:58.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf811df60 msgr2=0x7f5bf8142180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:58.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.880+0000 7f5bf4ff9640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf811df60 0x7f5bf8142180 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00108b0 tx=0x7f5bf0008040 comp rx=0 tx=0).stop 2024-09-15T01:40:58.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.885+0000 7f5bfef9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:58.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfef9d640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf81426c0 0x7f5bf8142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:58.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:58.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfef9d640 1 -- 172.21.15.178:0/467695493 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf804bc80 con 0x7f5bf81426c0 2024-09-15T01:40:58.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfcd39640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf81426c0 0x7f5bf8142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:58.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfcd39640 10 monclient: get_auth_request con 0x7f5bf81426c0 auth_method 0 2024-09-15T01:40:58.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfcd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:58.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfcd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:58.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfcd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:58.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfcd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:58.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfcd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:58.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfcd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:58.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfcd39640 10 monclient(hunting): handle_auth_done global_id 34472 payload 824 2024-09-15T01:40:58.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfcd39640 1 -- 172.21.15.178:0/467695493 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5bf8143000 con 0x7f5bf81426c0 2024-09-15T01:40:58.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bfcd39640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf81426c0 0x7f5bf8142ac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00108b0 tx=0x7f5bf001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:58.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5bf001cc30 con 0x7f5bf81426c0 2024-09-15T01:40:58.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bf4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:58.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bf4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:40:58.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bf4ff9640 10 monclient: dump: 2024-09-15T01:40:58.910 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:58.911 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:58.912 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:58.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f5bf001c7b0 con 0x7f5bf81426c0 2024-09-15T01:40:58.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bf4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:58.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bf4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:58.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf81426c0 msgr2=0x7f5bf8142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bf4ff9640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf81426c0 0x7f5bf8142ac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00108b0 tx=0x7f5bf001c000 comp rx=0 tx=0).stop 2024-09-15T01:40:58.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bf4ff9640 1 -- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf81426c0 msgr2=0x7f5bf8142ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:58.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.886+0000 7f5bf4ff9640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf81426c0 0x7f5bf8142ac0 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00108b0 tx=0x7f5bf001c000 comp rx=0 tx=0).stop 2024-09-15T01:40:58.912 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:58.935 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.887+0000 7f5bfef9d640 1 -- 172.21.15.178:0/467695493 >> v2:172.21.15.204:6812/28210 conn(0x7f5bd80396e0 msgr2=0x7f5bd803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.887+0000 7f5bfef9d640 1 --2- 172.21.15.178:0/467695493 >> v2:172.21.15.204:6812/28210 conn(0x7f5bd80396e0 0x7f5bd803bb90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5be8034000 tx=0x7f5be8032000 comp rx=0 tx=0).stop 2024-09-15T01:40:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.887+0000 7f5bfef9d640 10 monclient: shutdown 2024-09-15T01:40:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.887+0000 7f5bfef9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.887+0000 7f5bfef9d640 1 -- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5bf809fe70 msgr2=0x7f5bf811ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.887+0000 7f5bfef9d640 1 --2- 172.21.15.178:0/467695493 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5bf809fe70 0x7f5bf811ca00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5be000c8a0 tx=0x7f5be0001630 comp rx=0 tx=0).stop 2024-09-15T01:40:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.887+0000 7f5bfd53a640 1 -- 172.21.15.178:0/467695493 reap_dead start 2024-09-15T01:40:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.887+0000 7f5bfd53a640 1 -- 172.21.15.178:0/467695493 reap_dead start 2024-09-15T01:40:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.887+0000 7f5bfef9d640 1 -- 172.21.15.178:0/467695493 shutdown_connections 2024-09-15T01:40:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.887+0000 7f5bfef9d640 1 -- 172.21.15.178:0/467695493 >> 172.21.15.178:0/467695493 conn(0x7f5bf80a5ed0 msgr2=0x7f5bf809e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.887+0000 7f5bfef9d640 1 -- 172.21.15.178:0/467695493 shutdown_connections 2024-09-15T01:40:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:58.887+0000 7f5bfef9d640 1 -- 172.21.15.178:0/467695493 wait complete. 2024-09-15T01:40:58.938 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:58.938 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:40:58.938 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:40:59.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.030+0000 7f99b559d640 10 monclient: get_monmap_and_config 2024-09-15T01:40:59.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.030+0000 7f99b559d640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.030+0000 7f99b559d640 10 monclient: monmap: 2024-09-15T01:40:59.043 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.043 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.043 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:59.031279+0000 2024-09-15T01:40:59.043 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:59.031279+0000 2024-09-15T01:40:59.043 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.043 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.043 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:59.044 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:59.044 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.044 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.044 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.044 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.044 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.044 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.044 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.044 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:59.044 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:59.044 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.030+0000 7f99b559d640 1 Processor -- start 2024-09-15T01:40:59.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 1 -- start start 2024-09-15T01:40:59.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 10 monclient: init 2024-09-15T01:40:59.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:59.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 10 monclient: _add_conns ranks=[1,4,9,5,0,2,8,7,6,10,3] 2024-09-15T01:40:59.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f99b00a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:40:59.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132de0 0x7f99b0137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 10 monclient(hunting): picked mon.noname-c con 0x7f99b0132de0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:59.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f99b01380e0 0x7f99b013a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 10 monclient(hunting): picked mon.noname-a con 0x7f99b01380e0 addr v2:172.21.15.178:3300/0 2024-09-15T01:40:59.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b004bf60 con 0x7f99b00a2930 2024-09-15T01:40:59.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b009dd80 con 0x7f99b01380e0 2024-09-15T01:40:59.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00a2d00 con 0x7f99b0132de0 2024-09-15T01:40:59.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:59.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99b559d640 10 monclient(hunting): authenticate will time out at 1263.781495s 2024-09-15T01:40:59.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132de0 0x7f99b0137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99af7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f99b01380e0 0x7f99b013a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132de0 0x7f99b0137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:57146/0 (socket says 172.21.15.178:57146) 2024-09-15T01:40:59.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99af7fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f99b01380e0 0x7f99b013a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51256/0 (socket says 172.21.15.178:51256) 2024-09-15T01:40:59.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 1 -- 172.21.15.178:0/2458811298 learned_addr learned my addr 172.21.15.178:0/2458811298 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:59.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 10 monclient(hunting): get_auth_request con 0x7f99b0132de0 auth_method 0 2024-09-15T01:40:59.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99af7fe640 10 monclient(hunting): get_auth_request con 0x7f99b01380e0 auth_method 0 2024-09-15T01:40:59.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_done global_id 44230 payload 274 2024-09-15T01:40:59.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 1 -- 172.21.15.178:0/2458811298 >> v2:172.21.15.178:3300/0 conn(0x7f99b01380e0 msgr2=0x7f99b013a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:59.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 1 --2- 172.21.15.178:0/2458811298 >> v2:172.21.15.178:3300/0 conn(0x7f99b01380e0 0x7f99b013a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 1 -- 172.21.15.178:0/2458811298 >> v1:172.21.15.204:6789/0 conn(0x7f99b00a2930 legacy=0x7f99b009faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:40:59.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 1 monclient: found mon.noname-c 2024-09-15T01:40:59.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:59.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:40:59.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99ae7fc640 1 -- 172.21.15.178:0/2458811298 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b01331b0 con 0x7f99b0132de0 2024-09-15T01:40:59.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.031+0000 7f99af7fe640 1 --2- 172.21.15.178:0/2458811298 >> v2:172.21.15.178:3300/0 conn(0x7f99b01380e0 0x7f99b013a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:59.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99ae7fc640 1 --2- 172.21.15.178:0/2458811298 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132de0 0x7f99b0137ba0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009910 tx=0x7f99a40091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=26a0e3a6ff00ee32 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99adffb640 1 -- 172.21.15.178:0/2458811298 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99a402ad20 con 0x7f99b0132de0 2024-09-15T01:40:59.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99adffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:40:59.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99adffb640 10 monclient: dump: 2024-09-15T01:40:59.049 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.049 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.049 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.049 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.049 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.049 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.049 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.050 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.050 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.050 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.050 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.050 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.050 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.050 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.050 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.050 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99adffb640 10 monclient: _finish_auth 0 2024-09-15T01:40:59.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:59.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99adffb640 1 -- 172.21.15.178:0/2458811298 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a4031820 con 0x7f99b0132de0 2024-09-15T01:40:59.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99b559d640 5 monclient: authenticate success, global_id 44230 2024-09-15T01:40:59.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99b559d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:59.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99adffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:59.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99b559d640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:59.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99adffb640 1 -- 172.21.15.178:0/2458811298 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99a402aea0 con 0x7f99b0132de0 2024-09-15T01:40:59.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99b559d640 4 set_mon_vals no callback set 2024-09-15T01:40:59.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99adffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:40:59.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.032+0000 7f99adffb640 10 monclient: dump: 2024-09-15T01:40:59.051 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.051 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.051 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.052 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.052 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.052 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.052 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.052 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.052 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.052 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.052 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.052 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.052 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.052 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.053 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.053 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.033+0000 7f99b559d640 10 monclient: shutdown 2024-09-15T01:40:59.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.033+0000 7f99b559d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:59.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.033+0000 7f99b559d640 1 -- 172.21.15.178:0/2458811298 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132de0 msgr2=0x7f99b0137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.033+0000 7f99b559d640 1 --2- 172.21.15.178:0/2458811298 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132de0 0x7f99b0137ba0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009910 tx=0x7f99a40091d0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.033+0000 7f99b559d640 1 -- 172.21.15.178:0/2458811298 shutdown_connections 2024-09-15T01:40:59.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.033+0000 7f99b559d640 1 --2- 172.21.15.178:0/2458811298 >> v2:172.21.15.178:3300/0 conn(0x7f99b01380e0 0x7f99b013a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.033+0000 7f99b559d640 1 --2- 172.21.15.178:0/2458811298 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132de0 0x7f99b0137ba0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.033+0000 7f99b559d640 1 -- 172.21.15.178:0/2458811298 >> 172.21.15.178:0/2458811298 conn(0x7f99b00a5ed0 msgr2=0x7f99b00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:59.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.033+0000 7f99b559d640 1 -- 172.21.15.178:0/2458811298 shutdown_connections 2024-09-15T01:40:59.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.033+0000 7f99b559d640 1 -- 172.21.15.178:0/2458811298 wait complete. 2024-09-15T01:40:59.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient: monmap: 2024-09-15T01:40:59.054 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.054 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:59.054 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:59.054 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:59.054 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.054 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:59.054 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:59.054 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:59.054 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.054 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.055 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.055 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.055 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.055 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.055 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.055 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 1 Processor -- start 2024-09-15T01:40:59.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 1 -- start start 2024-09-15T01:40:59.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient: init 2024-09-15T01:40:59.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:59.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient: _add_conns ranks=[4,3,6,1,2,0,5,7,8] 2024-09-15T01:40:59.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b00a2930 0x7f99b009a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient(hunting): picked mon.noname-c con 0x7f99b00a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:59.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b0132de0 0x7f99b009ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient(hunting): picked mon.noname-g con 0x7f99b0132de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:59.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f99b01380e0 0x7f99b009b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient(hunting): picked mon.noname-d con 0x7f99b01380e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:40:59.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b009ca40 con 0x7f99b0132de0 2024-09-15T01:40:59.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99b011c750 con 0x7f99b00a2930 2024-09-15T01:40:59.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f99b009d630 con 0x7f99b01380e0 2024-09-15T01:40:59.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:59.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99b559d640 10 monclient(hunting): authenticate will time out at 1263.784825s 2024-09-15T01:40:59.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99aeffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b00a2930 0x7f99b009a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99af7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f99b01380e0 0x7f99b009b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99aeffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b00a2930 0x7f99b009a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:57152/0 (socket says 172.21.15.178:57152) 2024-09-15T01:40:59.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99aeffd640 1 -- 172.21.15.178:0/1568563540 learned_addr learned my addr 172.21.15.178:0/1568563540 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:59.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99aeffd640 10 monclient(hunting): get_auth_request con 0x7f99b00a2930 auth_method 0 2024-09-15T01:40:59.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99af7fe640 10 monclient(hunting): get_auth_request con 0x7f99b01380e0 auth_method 0 2024-09-15T01:40:59.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.034+0000 7f99af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99ae7fc640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b0132de0 0x7f99b009ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99ae7fc640 10 monclient(hunting): get_auth_request con 0x7f99b0132de0 auth_method 0 2024-09-15T01:40:59.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 10 monclient(hunting): handle_auth_done global_id 44239 payload 824 2024-09-15T01:40:59.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 1 -- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f99b01380e0 msgr2=0x7f99b009b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f99b01380e0 0x7f99b009b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 1 -- 172.21.15.178:0/1568563540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b0132de0 msgr2=0x7f99b009ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99b0132de0 0x7f99b009ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 10 monclient: _finish_hunting 0 2024-09-15T01:40:59.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 1 monclient: found mon.noname-c 2024-09-15T01:40:59.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:59.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:40:59.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 1 -- 172.21.15.178:0/1568563540 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a8009a20 con 0x7f99b00a2930 2024-09-15T01:40:59.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99af7fe640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f99b01380e0 0x7f99b009b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:59.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99aeffd640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b00a2930 0x7f99b009a6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99a000c4c0 tx=0x7f99a0007ea0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99a0011a10 con 0x7f99b00a2930 2024-09-15T01:40:59.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f9996ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f9996ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:40:59.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f9996ffd640 10 monclient: dump: 2024-09-15T01:40:59.060 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.060 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.061 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.061 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.061 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.061 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.061 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.061 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.061 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.061 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.061 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.061 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.061 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.061 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.062 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.062 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f9996ffd640 10 monclient: _finish_auth 0 2024-09-15T01:40:59.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f9996ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:59.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99b559d640 5 monclient: authenticate success, global_id 44239 2024-09-15T01:40:59.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a0011bd0 con 0x7f99b00a2930 2024-09-15T01:40:59.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99b559d640 10 monclient: _renew_subs 2024-09-15T01:40:59.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99b559d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:40:59.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99b559d640 1 -- 172.21.15.178:0/1568563540 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a4009450 con 0x7f99b00a2930 2024-09-15T01:40:59.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f9996ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:59.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99a0011d40 con 0x7f99b00a2930 2024-09-15T01:40:59.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f9996ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f9996ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:40:59.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f9996ffd640 10 monclient: dump: 2024-09-15T01:40:59.063 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.063 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.063 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.063 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.063 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.063 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.063 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.063 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.063 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.064 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.064 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.064 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.064 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.064 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.064 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.064 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99b559d640 10 monclient: _renew_subs 2024-09-15T01:40:59.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99ad7fa640 4 set_mon_vals no callback set 2024-09-15T01:40:59.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99b559d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:40:59.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.035+0000 7f99b559d640 1 -- 172.21.15.178:0/1568563540 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b011d640 con 0x7f99b00a2930 2024-09-15T01:40:59.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99b559d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:59.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99b559d640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132740 0x7f99b009b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f99a0011590 con 0x7f99b00a2930 2024-09-15T01:40:59.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99b559d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99b559d640 1 -- 172.21.15.178:0/1568563540 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99b004c4a0 con 0x7f99b0132740 2024-09-15T01:40:59.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99ae7fc640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132740 0x7f99b009b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99ae7fc640 10 monclient: get_auth_request con 0x7f99b0132740 auth_method 0 2024-09-15T01:40:59.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f9996ffd640 1 --2- 172.21.15.178:0/1568563540 >> v2:172.21.15.204:6812/28210 conn(0x7f999c0397b0 0x7f999c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f99a004abf0 con 0x7f99b00a2930 2024-09-15T01:40:59.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_done global_id 44248 payload 824 2024-09-15T01:40:59.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.036+0000 7f99ae7fc640 1 -- 172.21.15.178:0/1568563540 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99b005ed70 con 0x7f99b0132740 2024-09-15T01:40:59.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f99af7fe640 1 --2- 172.21.15.178:0/1568563540 >> v2:172.21.15.204:6812/28210 conn(0x7f999c0397b0 0x7f999c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f99af7fe640 10 monclient: get_auth_request con 0x7f999c0397b0 auth_method 0 2024-09-15T01:40:59.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f99ae7fc640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132740 0x7f99b009b6b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99b00a2f70 tx=0x7f99a4032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99a403c860 con 0x7f99b0132740 2024-09-15T01:40:59.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f9996ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f9996ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:40:59.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f9996ffd640 10 monclient: dump: 2024-09-15T01:40:59.067 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.067 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.067 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.067 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.067 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.067 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.067 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.067 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.067 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.068 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.068 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.068 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.068 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.068 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.068 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.068 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f99af7fe640 1 --2- 172.21.15.178:0/1568563540 >> v2:172.21.15.204:6812/28210 conn(0x7f999c0397b0 0x7f999c03bc60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f99b009be20 tx=0x7f99a800b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f99a403c3e0 con 0x7f99b0132740 2024-09-15T01:40:59.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f9996ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:59.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f9996ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:59.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132740 msgr2=0x7f99b009b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f9996ffd640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132740 0x7f99b009b6b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99b00a2f70 tx=0x7f99a4032000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132740 msgr2=0x7f99b009b6b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:59.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.037+0000 7f9996ffd640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0132740 0x7f99b009b6b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99b559d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:59.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99b559d640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0096a30 0x7f99b0096e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99b559d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99b559d640 1 -- 172.21.15.178:0/1568563540 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99b004bc80 con 0x7f99b0096a30 2024-09-15T01:40:59.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99ae7fc640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0096a30 0x7f99b0096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99ae7fc640 10 monclient: get_auth_request con 0x7f99b0096a30 auth_method 0 2024-09-15T01:40:59.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99ae7fc640 10 monclient(hunting): handle_auth_done global_id 44257 payload 824 2024-09-15T01:40:59.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99ae7fc640 1 -- 172.21.15.178:0/1568563540 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f99b004b720 con 0x7f99b0096a30 2024-09-15T01:40:59.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f99ae7fc640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0096a30 0x7f99b0096e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009910 tx=0x7f99a403d000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99a403dc60 con 0x7f99b0096a30 2024-09-15T01:40:59.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f9996ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f9996ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:40:59.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f9996ffd640 10 monclient: dump: 2024-09-15T01:40:59.073 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.073 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.073 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.073 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.073 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.073 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.073 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.073 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.074 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.074 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.074 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.074 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.074 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.074 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.074 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.074 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f99a403ddc0 con 0x7f99b0096a30 2024-09-15T01:40:59.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f9996ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:59.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f9996ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:59.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0096a30 msgr2=0x7f99b0096e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f9996ffd640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0096a30 0x7f99b0096e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009910 tx=0x7f99a403d000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f9996ffd640 1 -- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0096a30 msgr2=0x7f99b0096e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:59.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.043+0000 7f9996ffd640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b0096a30 0x7f99b0096e30 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009910 tx=0x7f99a403d000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.075 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:59.075 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:59.076 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:40:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.077 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.078 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:59.079 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.080 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.081 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.083 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.086 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.089 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:59.090 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.091 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:59.092 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.093 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:59.094 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.095 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:59.100 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.101 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:40:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.102 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:59.103 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.044+0000 7f99b559d640 1 -- 172.21.15.178:0/1568563540 >> v2:172.21.15.204:6812/28210 conn(0x7f999c0397b0 msgr2=0x7f999c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.044+0000 7f99b559d640 1 --2- 172.21.15.178:0/1568563540 >> v2:172.21.15.204:6812/28210 conn(0x7f999c0397b0 0x7f999c03bc60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f99b009be20 tx=0x7f99a800b040 comp rx=0 tx=0).stop 2024-09-15T01:40:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.044+0000 7f99b559d640 10 monclient: shutdown 2024-09-15T01:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.044+0000 7f99b559d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.044+0000 7f99b559d640 1 -- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b00a2930 msgr2=0x7f99b009a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.044+0000 7f99b559d640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b00a2930 0x7f99b009a6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99a000c4c0 tx=0x7f99a0007ea0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.044+0000 7f99af7fe640 1 -- 172.21.15.178:0/1568563540 reap_dead start 2024-09-15T01:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.044+0000 7f99b559d640 1 -- 172.21.15.178:0/1568563540 shutdown_connections 2024-09-15T01:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.044+0000 7f99b559d640 1 --2- 172.21.15.178:0/1568563540 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f99b00a2930 0x7f99b009a6f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.044+0000 7f99b559d640 1 -- 172.21.15.178:0/1568563540 >> 172.21.15.178:0/1568563540 conn(0x7f99b00a5ed0 msgr2=0x7f99b00a4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.044+0000 7f99b559d640 1 -- 172.21.15.178:0/1568563540 shutdown_connections 2024-09-15T01:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.044+0000 7f99b559d640 1 -- 172.21.15.178:0/1568563540 wait complete. 2024-09-15T01:40:59.106 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:59.106 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:40:59.106 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:40:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.197+0000 7f4f92a2b640 10 monclient: get_monmap_and_config 2024-09-15T01:40:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.197+0000 7f4f92a2b640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.197+0000 7f4f92a2b640 10 monclient: monmap: 2024-09-15T01:40:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:59.198718+0000 2024-09-15T01:40:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:59.198718+0000 2024-09-15T01:40:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:59.201 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.198+0000 7f4f92a2b640 1 Processor -- start 2024-09-15T01:40:59.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.198+0000 7f4f92a2b640 1 -- start start 2024-09-15T01:40:59.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.198+0000 7f4f92a2b640 10 monclient: init 2024-09-15T01:40:59.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:59.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 10 monclient: _add_conns ranks=[5,7,8,10,4,3,6,2,9,0,1] 2024-09-15T01:40:59.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4f8c0a2930 0x7f4f8c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 10 monclient(hunting): picked mon.noname-h con 0x7f4f8c0a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:59.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4f8c09faa0 0x7f4f8c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 10 monclient(hunting): picked mon.noname-i con 0x7f4f8c09faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:40:59.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4f8c0a08c0 0x7f4f8c0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 10 monclient(hunting): picked mon.noname-e con 0x7f4f8c0a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:59.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c09e0f0 con 0x7f4f8c0a2930 2024-09-15T01:40:59.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c09dd80 con 0x7f4f8c09faa0 2024-09-15T01:40:59.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c0a2d00 con 0x7f4f8c0a08c0 2024-09-15T01:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f92a2b640 10 monclient(hunting): authenticate will time out at 1263.949251s 2024-09-15T01:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4f8c0a08c0 0x7f4f8c0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4f8c0a08c0 0x7f4f8c0a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:39068/0 (socket says 172.21.15.178:39068) 2024-09-15T01:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 1 -- 172.21.15.178:0/3785387489 learned_addr learned my addr 172.21.15.178:0/3785387489 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 10 monclient(hunting): get_auth_request con 0x7f4f8c0a08c0 auth_method 0 2024-09-15T01:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f8b7fe640 1 --2- 172.21.15.178:0/3785387489 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4f8c09faa0 0x7f4f8c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f8bfff640 1 --2- 172.21.15.178:0/3785387489 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4f8c0a2930 0x7f4f8c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 10 monclient(hunting): handle_auth_done global_id 34451 payload 274 2024-09-15T01:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 1 -- 172.21.15.178:0/3785387489 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4f8c09faa0 msgr2=0x7f4f8c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 1 --2- 172.21.15.178:0/3785387489 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4f8c09faa0 0x7f4f8c09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 1 -- 172.21.15.178:0/3785387489 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4f8c0a2930 msgr2=0x7f4f8c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 1 --2- 172.21.15.178:0/3785387489 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4f8c0a2930 0x7f4f8c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 10 monclient: _finish_hunting 0 2024-09-15T01:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 1 monclient: found mon.noname-e 2024-09-15T01:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f90fc8640 1 -- 172.21.15.178:0/3785387489 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c04bf60 con 0x7f4f8c0a08c0 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f8b7fe640 10 monclient: get_auth_request con 0x7f4f8c09faa0 auth_method 0 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f8bfff640 10 monclient: get_auth_request con 0x7f4f8c0a2930 auth_method 0 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f8b7fe640 1 --2- 172.21.15.178:0/3785387489 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4f8c09faa0 0x7f4f8c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.199+0000 7f4f8bfff640 1 --2- 172.21.15.178:0/3785387489 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4f8c0a2930 0x7f4f8c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f90fc8640 1 --2- 172.21.15.178:0/3785387489 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4f8c0a08c0 0x7f4f8c0a0d30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009f50 tx=0x7f4f84009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=7314ebc878b57fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f8affd640 1 -- 172.21.15.178:0/3785387489 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f84007880 con 0x7f4f8c0a08c0 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f8affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f8affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f8affd640 10 monclient: dump: 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f8affd640 10 monclient: _finish_auth 0 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f8affd640 1 -- 172.21.15.178:0/3785387489 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8402d050 con 0x7f4f8c0a08c0 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f92a2b640 5 monclient: authenticate success, global_id 34451 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f92a2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f8affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f92a2b640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f8affd640 1 -- 172.21.15.178:0/3785387489 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f84030360 con 0x7f4f8c0a08c0 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f92a2b640 4 set_mon_vals no callback set 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f8affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f8affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.200+0000 7f4f8affd640 10 monclient: dump: 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.221 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.221 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.221 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.221 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.221 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.221 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.221 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.221 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.221 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.221 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.221 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.221 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 10 monclient: shutdown 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 1 -- 172.21.15.178:0/3785387489 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4f8c0a08c0 msgr2=0x7f4f8c0a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 1 --2- 172.21.15.178:0/3785387489 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4f8c0a08c0 0x7f4f8c0a0d30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009f50 tx=0x7f4f84009840 comp rx=0 tx=0).stop 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 1 -- 172.21.15.178:0/3785387489 shutdown_connections 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 1 --2- 172.21.15.178:0/3785387489 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4f8c0a08c0 0x7f4f8c0a0d30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 1 --2- 172.21.15.178:0/3785387489 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f4f8c09faa0 0x7f4f8c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 1 --2- 172.21.15.178:0/3785387489 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4f8c0a2930 0x7f4f8c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 1 -- 172.21.15.178:0/3785387489 >> 172.21.15.178:0/3785387489 conn(0x7f4f8c0a5ed0 msgr2=0x7f4f8c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 1 -- 172.21.15.178:0/3785387489 shutdown_connections 2024-09-15T01:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 1 -- 172.21.15.178:0/3785387489 wait complete. 2024-09-15T01:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 10 monclient: monmap: 2024-09-15T01:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.201+0000 7f4f92a2b640 1 Processor -- start 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 1 -- start start 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 10 monclient: init 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 10 monclient: _add_conns ranks=[5,4,6,0,8,1,3,7,2] 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4f8c09faa0 0x7f4f8c09b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 10 monclient(hunting): picked mon.noname-h con 0x7f4f8c09faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4f8c0a08c0 0x7f4f8c131750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 10 monclient(hunting): picked mon.noname-c con 0x7f4f8c0a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c0a2930 0x7f4f8c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 10 monclient(hunting): picked mon.noname-d con 0x7f4f8c0a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c142b10 con 0x7f4f8c0a08c0 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c11eec0 con 0x7f4f8c09faa0 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c09c020 con 0x7f4f8c0a2930 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f92a2b640 10 monclient(hunting): authenticate will time out at 1263.952750s 2024-09-15T01:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f8b7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4f8c0a08c0 0x7f4f8c131750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f90fc8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c0a2930 0x7f4f8c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f8b7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4f8c0a08c0 0x7f4f8c131750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:57178/0 (socket says 172.21.15.178:57178) 2024-09-15T01:40:59.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f8b7fe640 1 -- 172.21.15.178:0/976984496 learned_addr learned my addr 172.21.15.178:0/976984496 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:59.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f90fc8640 10 monclient(hunting): get_auth_request con 0x7f4f8c0a2930 auth_method 0 2024-09-15T01:40:59.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f90fc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f90fc8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f90fc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f8b7fe640 10 monclient(hunting): get_auth_request con 0x7f4f8c0a08c0 auth_method 0 2024-09-15T01:40:59.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f90fc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.202+0000 7f4f90fc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f90fc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8bfff640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4f8c09faa0 0x7f4f8c09b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8bfff640 10 monclient(hunting): get_auth_request con 0x7f4f8c09faa0 auth_method 0 2024-09-15T01:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f90fc8640 10 monclient(hunting): handle_auth_done global_id 25026 payload 824 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f90fc8640 1 -- 172.21.15.178:0/976984496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4f8c09faa0 msgr2=0x7f4f8c09b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f90fc8640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4f8c09faa0 0x7f4f8c09b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f90fc8640 1 -- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4f8c0a08c0 msgr2=0x7f4f8c131750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f90fc8640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4f8c0a08c0 0x7f4f8c131750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f90fc8640 10 monclient: _finish_hunting 0 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f90fc8640 1 monclient: found mon.noname-d 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f90fc8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f90fc8640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f90fc8640 1 -- 172.21.15.178:0/976984496 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f84009ae0 con 0x7f4f8c0a2930 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8b7fe640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4f8c0a08c0 0x7f4f8c131750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:59.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f90fc8640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c0a2930 0x7f4f8c11c530 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4f84030000 tx=0x7f4f84036000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f84008b20 con 0x7f4f8c0a2930 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f73fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f73fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f73fff640 10 monclient: dump: 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f73fff640 10 monclient: _finish_auth 0 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f73fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f92a2b640 5 monclient: authenticate success, global_id 25026 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f84008cc0 con 0x7f4f8c0a2930 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f73fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f84008e30 con 0x7f4f8c0a2930 2024-09-15T01:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f73fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f73fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f8a7fc640 4 set_mon_vals no callback set 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f73fff640 10 monclient: dump: 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f92a2b640 10 monclient: _renew_subs 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f92a2b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f92a2b640 1 -- 172.21.15.178:0/976984496 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8c09d630 con 0x7f4f8c0a2930 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f92a2b640 10 monclient: _renew_subs 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f92a2b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.203+0000 7f4f92a2b640 1 -- 172.21.15.178:0/976984496 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8c09baa0 con 0x7f4f8c0a2930 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4f84038050 con 0x7f4f8c0a2930 2024-09-15T01:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f92a2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f92a2b640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c131c90 0x7f4f8c11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f92a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f92a2b640 1 -- 172.21.15.178:0/976984496 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c04c4a0 con 0x7f4f8c131c90 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f73fff640 1 --2- 172.21.15.178:0/976984496 >> v2:172.21.15.204:6812/28210 conn(0x7f4f780397b0 0x7f4f7803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f8bfff640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c131c90 0x7f4f8c11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f8bfff640 10 monclient: get_auth_request con 0x7f4f8c131c90 auth_method 0 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f4f8406bfa0 con 0x7f4f8c0a2930 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_done global_id 25035 payload 824 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f8bfff640 1 -- 172.21.15.178:0/976984496 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f8c05ed70 con 0x7f4f8c131c90 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.204+0000 7f4f8b7fe640 1 --2- 172.21.15.178:0/976984496 >> v2:172.21.15.204:6812/28210 conn(0x7f4f780397b0 0x7f4f7803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f8b7fe640 10 monclient: get_auth_request con 0x7f4f780397b0 auth_method 0 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f8bfff640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c131c90 0x7f4f8c11ca70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00fba0 tx=0x7f4f7c008040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f7c013440 con 0x7f4f8c131c90 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f73fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f73fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f73fff640 10 monclient: dump: 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.234 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f4f7c00c040 con 0x7f4f8c131c90 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f73fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f73fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c131c90 msgr2=0x7f4f8c11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f73fff640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c131c90 0x7f4f8c11ca70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00fba0 tx=0x7f4f7c008040 comp rx=0 tx=0).stop 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c131c90 msgr2=0x7f4f8c11ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f73fff640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c131c90 0x7f4f8c11ca70 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00fba0 tx=0x7f4f7c008040 comp rx=0 tx=0).stop 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.205+0000 7f4f8b7fe640 1 --2- 172.21.15.178:0/976984496 >> v2:172.21.15.204:6812/28210 conn(0x7f4f780397b0 0x7f4f7803bc60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4f80006fd0 tx=0x7f4f80008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f92a2b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f92a2b640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c11cfb0 0x7f4f8c096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f92a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f92a2b640 1 -- 172.21.15.178:0/976984496 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c04bc80 con 0x7f4f8c11cfb0 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f8bfff640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c11cfb0 0x7f4f8c096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f8bfff640 10 monclient: get_auth_request con 0x7f4f8c11cfb0 auth_method 0 2024-09-15T01:40:59.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_done global_id 25044 payload 824 2024-09-15T01:40:59.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f8bfff640 1 -- 172.21.15.178:0/976984496 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4f8c11d3b0 con 0x7f4f8c11cfb0 2024-09-15T01:40:59.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f8bfff640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c11cfb0 0x7f4f8c096a30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c01a000 tx=0x7f4f7c01d000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f7c023040 con 0x7f4f8c11cfb0 2024-09-15T01:40:59.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f73fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f73fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f73fff640 10 monclient: dump: 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.238 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.239 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.239 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.239 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f4f7c00d610 con 0x7f4f8c11cfb0 2024-09-15T01:40:59.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f73fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:59.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f73fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:59.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c11cfb0 msgr2=0x7f4f8c096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f73fff640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c11cfb0 0x7f4f8c096a30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c01a000 tx=0x7f4f7c01d000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f73fff640 1 -- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c11cfb0 msgr2=0x7f4f8c096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:59.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.211+0000 7f4f73fff640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c11cfb0 0x7f4f8c096a30 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c01a000 tx=0x7f4f7c01d000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.240 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:59.240 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:59.240 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:59.240 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:59.240 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:59.240 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:59.240 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:59.240 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:59.240 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:59.240 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.241 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.242 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.242 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.242 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.242 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.242 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.242 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.242 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.243 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:59.243 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:59.244 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:59.245 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.246 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:59.247 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:59.248 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.251 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.252 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:59.253 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.254 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.255 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.256 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.258 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.259 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:59.261 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.262 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:59.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.263 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.263 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.263 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:40:59.263 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.263 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.263 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:59.263 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.212+0000 7f4f92a2b640 1 -- 172.21.15.178:0/976984496 >> v2:172.21.15.204:6812/28210 conn(0x7f4f780397b0 msgr2=0x7f4f7803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.212+0000 7f4f92a2b640 1 --2- 172.21.15.178:0/976984496 >> v2:172.21.15.204:6812/28210 conn(0x7f4f780397b0 0x7f4f7803bc60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4f80006fd0 tx=0x7f4f80008040 comp rx=0 tx=0).stop 2024-09-15T01:40:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.212+0000 7f4f92a2b640 10 monclient: shutdown 2024-09-15T01:40:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.212+0000 7f4f92a2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.212+0000 7f4f92a2b640 1 -- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c0a2930 msgr2=0x7f4f8c11c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.212+0000 7f4f92a2b640 1 --2- 172.21.15.178:0/976984496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4f8c0a2930 0x7f4f8c11c530 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4f84030000 tx=0x7f4f84036000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.212+0000 7f4f90fc8640 1 -- 172.21.15.178:0/976984496 reap_dead start 2024-09-15T01:40:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.212+0000 7f4f90fc8640 1 -- 172.21.15.178:0/976984496 reap_dead start 2024-09-15T01:40:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.212+0000 7f4f92a2b640 1 -- 172.21.15.178:0/976984496 shutdown_connections 2024-09-15T01:40:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.212+0000 7f4f92a2b640 1 -- 172.21.15.178:0/976984496 >> 172.21.15.178:0/976984496 conn(0x7f4f8c0a5ed0 msgr2=0x7f4f8c135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.212+0000 7f4f92a2b640 1 -- 172.21.15.178:0/976984496 shutdown_connections 2024-09-15T01:40:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.212+0000 7f4f92a2b640 1 -- 172.21.15.178:0/976984496 wait complete. 2024-09-15T01:40:59.266 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:59.266 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:40:59.266 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:40:59.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.357+0000 7f454e6d4640 10 monclient: get_monmap_and_config 2024-09-15T01:40:59.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.357+0000 7f454e6d4640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.357+0000 7f454e6d4640 10 monclient: monmap: 2024-09-15T01:40:59.359 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.359 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.359 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:59.357971+0000 2024-09-15T01:40:59.359 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:59.357971+0000 2024-09-15T01:40:59.359 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.359 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.360 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:59.360 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:59.360 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.360 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.360 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.360 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.360 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.360 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.360 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.360 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:59.360 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:59.360 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.357+0000 7f454e6d4640 1 Processor -- start 2024-09-15T01:40:59.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 1 -- start start 2024-09-15T01:40:59.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 10 monclient: init 2024-09-15T01:40:59.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 10 monclient: _add_conns ranks=[8,2,9,6,0,10,1,3,7,5,4] 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45480a2930 0x7f4548132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 10 monclient(hunting): picked mon.noname-e con 0x7f45480a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f454809faa0 0x7f454809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 10 monclient(hunting): picked mon.noname-b con 0x7f454809faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f45480a08c0 0x7f45480a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 10 monclient(hunting): picked mon.noname-a con 0x7f45480a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454809e0f0 con 0x7f45480a08c0 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454809dd80 con 0x7f454809faa0 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f45480a2d00 con 0x7f45480a2930 2024-09-15T01:40:59.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:59.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454e6d4640 10 monclient(hunting): authenticate will time out at 1264.108496s 2024-09-15T01:40:59.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454cc71640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f45480a08c0 0x7f45480a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45480a2930 0x7f4548132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f45477fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f454809faa0 0x7f454809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454cc71640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f45480a08c0 0x7f45480a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51264/0 (socket says 172.21.15.178:51264) 2024-09-15T01:40:59.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45480a2930 0x7f4548132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:39076/0 (socket says 172.21.15.178:39076) 2024-09-15T01:40:59.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454cc71640 1 -- 172.21.15.178:0/1531979521 learned_addr learned my addr 172.21.15.178:0/1531979521 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:59.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454cc71640 10 monclient(hunting): get_auth_request con 0x7f45480a08c0 auth_method 0 2024-09-15T01:40:59.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454cc71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454cc71640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454cc71640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 10 monclient(hunting): get_auth_request con 0x7f45480a2930 auth_method 0 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f45477fe640 10 monclient(hunting): get_auth_request con 0x7f454809faa0 auth_method 0 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f45477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f45477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f45477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454cc71640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454cc71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454cc71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f45477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f45477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f45477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 10 monclient(hunting): handle_auth_done global_id 34460 payload 274 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 1 -- 172.21.15.178:0/1531979521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f454809faa0 msgr2=0x7f454809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 1 --2- 172.21.15.178:0/1531979521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f454809faa0 0x7f454809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 1 -- 172.21.15.178:0/1531979521 >> v2:172.21.15.178:3300/0 conn(0x7f45480a08c0 msgr2=0x7f45480a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 1 --2- 172.21.15.178:0/1531979521 >> v2:172.21.15.178:3300/0 conn(0x7f45480a08c0 0x7f45480a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 10 monclient: _finish_hunting 0 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 1 monclient: found mon.noname-e 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f4547fff640 1 -- 172.21.15.178:0/1531979521 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454804bf60 con 0x7f45480a2930 2024-09-15T01:40:59.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f454cc71640 1 --2- 172.21.15.178:0/1531979521 >> v2:172.21.15.178:3300/0 conn(0x7f45480a08c0 0x7f45480a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.358+0000 7f45477fe640 1 --2- 172.21.15.178:0/1531979521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f454809faa0 0x7f454809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4547fff640 1 --2- 172.21.15.178:0/1531979521 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45480a2930 0x7f4548132de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4538009850 tx=0x7f45380090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e002e2123a8888d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4546ffd640 1 -- 172.21.15.178:0/1531979521 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f453802abc0 con 0x7f45480a2930 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4546ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4546ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4546ffd640 10 monclient: dump: 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.375 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4546ffd640 10 monclient: _finish_auth 0 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4546ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4546ffd640 1 -- 172.21.15.178:0/1531979521 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453802ad40 con 0x7f45480a2930 2024-09-15T01:40:59.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4546ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:59.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4546ffd640 1 -- 172.21.15.178:0/1531979521 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4538031ca0 con 0x7f45480a2930 2024-09-15T01:40:59.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f454e6d4640 5 monclient: authenticate success, global_id 34460 2024-09-15T01:40:59.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f454e6d4640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:59.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f454e6d4640 4 set_mon_vals no callback set 2024-09-15T01:40:59.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4546ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4546ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:59.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.359+0000 7f4546ffd640 10 monclient: dump: 2024-09-15T01:40:59.377 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.377 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.377 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.377 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.377 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 10 monclient: shutdown 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:59.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 1 -- 172.21.15.178:0/1531979521 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45480a2930 msgr2=0x7f4548132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 1 --2- 172.21.15.178:0/1531979521 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45480a2930 0x7f4548132de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4538009850 tx=0x7f45380090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 1 -- 172.21.15.178:0/1531979521 shutdown_connections 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 1 --2- 172.21.15.178:0/1531979521 >> v2:172.21.15.178:3300/0 conn(0x7f45480a08c0 0x7f45480a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 1 --2- 172.21.15.178:0/1531979521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f454809faa0 0x7f454809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 1 --2- 172.21.15.178:0/1531979521 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f45480a2930 0x7f4548132de0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 1 -- 172.21.15.178:0/1531979521 >> 172.21.15.178:0/1531979521 conn(0x7f45480a5ed0 msgr2=0x7f45480a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 1 -- 172.21.15.178:0/1531979521 shutdown_connections 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 1 -- 172.21.15.178:0/1531979521 wait complete. 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.360+0000 7f454e6d4640 10 monclient: monmap: 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:59.379 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.361+0000 7f454e6d4640 1 Processor -- start 2024-09-15T01:40:59.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.361+0000 7f454e6d4640 1 -- start start 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.361+0000 7f454e6d4640 10 monclient: init 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.361+0000 7f454e6d4640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.361+0000 7f454e6d4640 10 monclient: _add_conns ranks=[5,2,3,7,6,1,8,0,4] 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.361+0000 7f454e6d4640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f454809faa0 0x7f454809b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.361+0000 7f454e6d4640 10 monclient(hunting): picked mon.noname-h con 0x7f454809faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45480a08c0 0x7f4548131990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 10 monclient(hunting): picked mon.noname-b con 0x7f45480a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45480a2930 0x7f4548141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 10 monclient(hunting): picked mon.noname-g con 0x7f45480a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45480a2390 con 0x7f45480a08c0 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454809d630 con 0x7f45480a2930 2024-09-15T01:40:59.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f454811ee20 con 0x7f454809faa0 2024-09-15T01:40:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 10 monclient(hunting): authenticate will time out at 1264.112062s 2024-09-15T01:40:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45480a08c0 0x7f4548131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45480a08c0 0x7f4548131990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:51416/0 (socket says 172.21.15.178:51416) 2024-09-15T01:40:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 1 -- 172.21.15.178:0/2110811429 learned_addr learned my addr 172.21.15.178:0/2110811429 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 10 monclient(hunting): get_auth_request con 0x7f45480a08c0 auth_method 0 2024-09-15T01:40:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f4547fff640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f454809faa0 0x7f454809b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f4547fff640 10 monclient(hunting): get_auth_request con 0x7f454809faa0 auth_method 0 2024-09-15T01:40:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f4547fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f4547fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f4547fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454cc71640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45480a2930 0x7f4548141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454cc71640 10 monclient(hunting): get_auth_request con 0x7f45480a2930 auth_method 0 2024-09-15T01:40:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454cc71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454cc71640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454cc71640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f4547fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f4547fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f4547fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454cc71640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454cc71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454cc71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 10 monclient(hunting): handle_auth_done global_id 34490 payload 824 2024-09-15T01:40:59.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 1 -- 172.21.15.178:0/2110811429 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f454809faa0 msgr2=0x7f454809b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f454809faa0 0x7f454809b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 1 -- 172.21.15.178:0/2110811429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45480a2930 msgr2=0x7f4548141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45480a2930 0x7f4548141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 10 monclient: _finish_hunting 0 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 1 monclient: found mon.noname-b 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 1 -- 172.21.15.178:0/2110811429 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c009390 con 0x7f45480a08c0 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45477fe640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45480a08c0 0x7f4548131990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f453c00ea20 tx=0x7f453c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f453c00ce90 con 0x7f45480a08c0 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f452ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f452ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f452ffff640 10 monclient: dump: 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.385 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f452ffff640 10 monclient: _finish_auth 0 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f452ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c009650 con 0x7f45480a08c0 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 5 monclient: authenticate success, global_id 34490 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f452ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 10 monclient: _renew_subs 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f453c0138f0 con 0x7f45480a08c0 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 1 -- 172.21.15.178:0/2110811429 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4540009ae0 con 0x7f45480a08c0 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f452ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f452ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f452ffff640 10 monclient: dump: 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.387 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.388 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f45467fc640 4 set_mon_vals no callback set 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 10 monclient: _renew_subs 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.362+0000 7f454e6d4640 1 -- 172.21.15.178:0/2110811429 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4538009390 con 0x7f45480a08c0 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f454e6d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f454e6d4640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f454811ca00 0x7f45481423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f454e6d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f454e6d4640 1 -- 172.21.15.178:0/2110811429 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f454811f550 con 0x7f454811ca00 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f453c013470 con 0x7f45480a08c0 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f4547fff640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f454811ca00 0x7f45481423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f4547fff640 10 monclient: get_auth_request con 0x7f454811ca00 auth_method 0 2024-09-15T01:40:59.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f4547fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f4547fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f4547fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f4547fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f4547fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f4547fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f452ffff640 1 --2- 172.21.15.178:0/2110811429 >> v2:172.21.15.204:6812/28210 conn(0x7f45340396e0 0x7f453403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f453c04bec0 con 0x7f45480a08c0 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.363+0000 7f4547fff640 10 monclient(hunting): handle_auth_done global_id 34469 payload 824 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f4547fff640 1 -- 172.21.15.178:0/2110811429 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f454804c4a0 con 0x7f454811ca00 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f454cc71640 1 --2- 172.21.15.178:0/2110811429 >> v2:172.21.15.204:6812/28210 conn(0x7f45340396e0 0x7f453403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f454cc71640 10 monclient: get_auth_request con 0x7f45340396e0 auth_method 0 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f4547fff640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f454811ca00 0x7f45481423c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f45480a3290 tx=0x7f4538032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f453803da80 con 0x7f454811ca00 2024-09-15T01:40:59.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f452ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f452ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f452ffff640 10 monclient: dump: 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.391 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.392 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.392 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.392 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.392 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f454cc71640 1 --2- 172.21.15.178:0/2110811429 >> v2:172.21.15.204:6812/28210 conn(0x7f45340396e0 0x7f453403bb90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f45400072f0 tx=0x7f454000b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f453803dbe0 con 0x7f454811ca00 2024-09-15T01:40:59.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f452ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:59.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f452ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:59.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f454811ca00 msgr2=0x7f45481423c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f452ffff640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f454811ca00 0x7f45481423c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f45480a3290 tx=0x7f4538032000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f454811ca00 msgr2=0x7f45481423c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:59.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.364+0000 7f452ffff640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f454811ca00 0x7f45481423c0 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f45480a3290 tx=0x7f4538032000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f454e6d4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:59.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f454e6d4640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4548131ed0 0x7f4548142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f454e6d4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f454e6d4640 1 -- 172.21.15.178:0/2110811429 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f454804bc80 con 0x7f4548131ed0 2024-09-15T01:40:59.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f4547fff640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4548131ed0 0x7f4548142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f4547fff640 10 monclient: get_auth_request con 0x7f4548131ed0 auth_method 0 2024-09-15T01:40:59.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f4547fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f4547fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f4547fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f4547fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f4547fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f4547fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f4547fff640 10 monclient(hunting): handle_auth_done global_id 34478 payload 824 2024-09-15T01:40:59.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f4547fff640 1 -- 172.21.15.178:0/2110811429 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f454805ed70 con 0x7f4548131ed0 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f4547fff640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4548131ed0 0x7f4548142900 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4538009720 tx=0x7f453803e000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f453804b070 con 0x7f4548131ed0 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f452ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f452ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f452ffff640 10 monclient: dump: 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.396 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.370+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f453804e3b0 con 0x7f4548131ed0 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f452ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f452ffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4548131ed0 msgr2=0x7f4548142900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f452ffff640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4548131ed0 0x7f4548142900 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4538009720 tx=0x7f453803e000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f452ffff640 1 -- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4548131ed0 msgr2=0x7f4548142900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:59.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f452ffff640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f4548131ed0 0x7f4548142900 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.398 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:59.398 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:59.398 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:59.398 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:59.398 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:59.398 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:59.398 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:59.398 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:59.398 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.399 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.401 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.401 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.401 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.401 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.401 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.401 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.401 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.401 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:59.401 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:59.401 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:59.401 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:59.402 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.403 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:59.405 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.407 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:59.408 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.411 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.412 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:59.412 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:59.412 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.412 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.412 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.412 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:59.412 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:59.412 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:59.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:59.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.417 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.417 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:59.417 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:59.417 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.417 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.417 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:59.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:59.419 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.420 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.421 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:59.421 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f454e6d4640 1 -- 172.21.15.178:0/2110811429 >> v2:172.21.15.204:6812/28210 conn(0x7f45340396e0 msgr2=0x7f453403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f454e6d4640 1 --2- 172.21.15.178:0/2110811429 >> v2:172.21.15.204:6812/28210 conn(0x7f45340396e0 0x7f453403bb90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f45400072f0 tx=0x7f454000b040 comp rx=0 tx=0).stop 2024-09-15T01:40:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f454e6d4640 10 monclient: shutdown 2024-09-15T01:40:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f454e6d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f454e6d4640 1 -- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45480a08c0 msgr2=0x7f4548131990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f454e6d4640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45480a08c0 0x7f4548131990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f453c00ea20 tx=0x7f453c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f454cc71640 1 -- 172.21.15.178:0/2110811429 reap_dead start 2024-09-15T01:40:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f454e6d4640 1 -- 172.21.15.178:0/2110811429 shutdown_connections 2024-09-15T01:40:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f454e6d4640 1 --2- 172.21.15.178:0/2110811429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45480a08c0 0x7f4548131990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f454e6d4640 1 -- 172.21.15.178:0/2110811429 >> 172.21.15.178:0/2110811429 conn(0x7f45480a5ed0 msgr2=0x7f45480a5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f454e6d4640 1 -- 172.21.15.178:0/2110811429 shutdown_connections 2024-09-15T01:40:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.371+0000 7f454e6d4640 1 -- 172.21.15.178:0/2110811429 wait complete. 2024-09-15T01:40:59.424 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:59.424 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:40:59.424 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:40:59.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.515+0000 7fdb84ec9640 10 monclient: get_monmap_and_config 2024-09-15T01:40:59.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.515+0000 7fdb84ec9640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.515+0000 7fdb84ec9640 10 monclient: monmap: 2024-09-15T01:40:59.527 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.527 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.527 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:59.516016+0000 2024-09-15T01:40:59.527 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:59.516016+0000 2024-09-15T01:40:59.527 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.527 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.527 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:59.527 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.515+0000 7fdb84ec9640 1 Processor -- start 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.515+0000 7fdb84ec9640 1 -- start start 2024-09-15T01:40:59.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.515+0000 7fdb84ec9640 10 monclient: init 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 10 monclient: _add_conns ranks=[5,9,1,2,0,10,6,4,8,3,7] 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdb800a2930 0x7fdb80132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 10 monclient(hunting): picked mon.noname-h con 0x7fdb800a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdb8009faa0 0x7fdb8009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 10 monclient(hunting): picked mon.noname-a con 0x7fdb8009faa0 addr v2:172.21.15.178:3300/0 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fdb800a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8004bf60 con 0x7fdb800a08c0 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8009e0f0 con 0x7fdb8009faa0 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8009dd80 con 0x7fdb800a2930 2024-09-15T01:40:59.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb84ec9640 10 monclient(hunting): authenticate will time out at 1264.266501s 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdb8009faa0 0x7fdb8009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdb8009faa0 0x7fdb8009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51278/0 (socket says 172.21.15.178:51278) 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 1 -- 172.21.15.178:0/1595954261 learned_addr learned my addr 172.21.15.178:0/1595954261 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 10 monclient(hunting): get_auth_request con 0x7fdb8009faa0 auth_method 0 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_done global_id 34623 payload 274 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7e59c640 1 --2- 172.21.15.178:0/1595954261 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdb800a2930 0x7fdb80132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 1 -- 172.21.15.178:0/1595954261 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdb800a2930 msgr2=0x7fdb80132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 1 --2- 172.21.15.178:0/1595954261 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdb800a2930 0x7fdb80132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 1 -- 172.21.15.178:0/1595954261 >> v1:172.21.15.204:6789/0 conn(0x7fdb800a08c0 legacy=0x7fdb800a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 10 monclient: _finish_hunting 0 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 1 monclient: found mon.noname-a 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7dd9b640 1 -- 172.21.15.178:0/1595954261 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb800a2d00 con 0x7fdb8009faa0 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7e59c640 10 monclient: get_auth_request con 0x7fdb800a2930 auth_method 0 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.516+0000 7fdb7e59c640 1 --2- 172.21.15.178:0/1595954261 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdb800a2930 0x7fdb80132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7dd9b640 1 --2- 172.21.15.178:0/1595954261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb8009faa0 0x7fdb8009fe70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009910 tx=0x7fdb740091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345f58016d98edf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7d59a640 1 -- 172.21.15.178:0/1595954261 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb7402ad20 con 0x7fdb8009faa0 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7d59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7d59a640 10 monclient: dump: 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.532 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7d59a640 10 monclient: _finish_auth 0 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7d59a640 1 -- 172.21.15.178:0/1595954261 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb74031820 con 0x7fdb8009faa0 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb84ec9640 5 monclient: authenticate success, global_id 34623 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb84ec9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb84ec9640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7d59a640 1 -- 172.21.15.178:0/1595954261 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb7402aea0 con 0x7fdb8009faa0 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb84ec9640 4 set_mon_vals no callback set 2024-09-15T01:40:59.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.517+0000 7fdb7d59a640 10 monclient: dump: 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.534 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.518+0000 7fdb84ec9640 10 monclient: shutdown 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.518+0000 7fdb84ec9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.518+0000 7fdb84ec9640 1 -- 172.21.15.178:0/1595954261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb8009faa0 msgr2=0x7fdb8009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.518+0000 7fdb84ec9640 1 --2- 172.21.15.178:0/1595954261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb8009faa0 0x7fdb8009fe70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009910 tx=0x7fdb740091d0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.518+0000 7fdb84ec9640 1 -- 172.21.15.178:0/1595954261 shutdown_connections 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.518+0000 7fdb84ec9640 1 --2- 172.21.15.178:0/1595954261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb8009faa0 0x7fdb8009fe70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.518+0000 7fdb84ec9640 1 --2- 172.21.15.178:0/1595954261 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdb800a2930 0x7fdb80132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.518+0000 7fdb84ec9640 1 -- 172.21.15.178:0/1595954261 >> 172.21.15.178:0/1595954261 conn(0x7fdb800a5ed0 msgr2=0x7fdb800a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:59.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.518+0000 7fdb84ec9640 1 -- 172.21.15.178:0/1595954261 shutdown_connections 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.518+0000 7fdb84ec9640 1 -- 172.21.15.178:0/1595954261 wait complete. 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.518+0000 7fdb84ec9640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.518+0000 7fdb84ec9640 10 monclient: monmap: 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.536 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 1 Processor -- start 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 1 -- start start 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 10 monclient: init 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 10 monclient: _add_conns ranks=[8,2,3,1,4,6,7,0,5] 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdb8009faa0 0x7fdb8009afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 10 monclient(hunting): picked mon.noname-e con 0x7fdb8009faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:59.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb800a08c0 0x7fdb80131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 10 monclient(hunting): picked mon.noname-b con 0x7fdb800a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb800a2930 0x7fdb8011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 10 monclient(hunting): picked mon.noname-g con 0x7fdb800a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8009d630 con 0x7fdb800a08c0 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb80142a20 con 0x7fdb800a2930 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8009bfc0 con 0x7fdb8009faa0 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb84ec9640 10 monclient(hunting): authenticate will time out at 1264.269764s 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7e59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdb8009faa0 0x7fdb8009afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7dd9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb800a08c0 0x7fdb80131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7e59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdb8009faa0 0x7fdb8009afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:39114/0 (socket says 172.21.15.178:39114) 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7e59c640 1 -- 172.21.15.178:0/3846128732 learned_addr learned my addr 172.21.15.178:0/3846128732 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7e59c640 10 monclient(hunting): get_auth_request con 0x7fdb8009faa0 auth_method 0 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7dd9b640 10 monclient(hunting): get_auth_request con 0x7fdb800a08c0 auth_method 0 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7ed9d640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb800a2930 0x7fdb8011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7ed9d640 10 monclient(hunting): get_auth_request con 0x7fdb800a2930 auth_method 0 2024-09-15T01:40:59.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.519+0000 7fdb7ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7e59c640 10 monclient(hunting): handle_auth_done global_id 34487 payload 824 2024-09-15T01:40:59.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7e59c640 1 -- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb800a2930 msgr2=0x7fdb8011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7e59c640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb800a2930 0x7fdb8011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7e59c640 1 -- 172.21.15.178:0/3846128732 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb800a08c0 msgr2=0x7fdb80131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7e59c640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb800a08c0 0x7fdb80131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:40:59.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7e59c640 1 monclient: found mon.noname-e 2024-09-15T01:40:59.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:59.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7e59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:40:59.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7e59c640 1 -- 172.21.15.178:0/3846128732 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb74009450 con 0x7fdb8009faa0 2024-09-15T01:40:59.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7dd9b640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdb800a08c0 0x7fdb80131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:59.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7e59c640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdb8009faa0 0x7fdb8009afd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000c8b0 tx=0x7fdb70008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb7000ce40 con 0x7fdb8009faa0 2024-09-15T01:40:59.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb6e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:40:59.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb6e7fc640 10 monclient: dump: 2024-09-15T01:40:59.542 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.542 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.542 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.542 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.542 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.542 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.542 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.542 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb6e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb84ec9640 5 monclient: authenticate success, global_id 34487 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb84ec9640 10 monclient: _renew_subs 2024-09-15T01:40:59.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb70007d10 con 0x7fdb8009faa0 2024-09-15T01:40:59.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb84ec9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:59.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb84ec9640 1 -- 172.21.15.178:0/3846128732 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8011e940 con 0x7fdb8009faa0 2024-09-15T01:40:59.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb6e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:59.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb70007e80 con 0x7fdb8009faa0 2024-09-15T01:40:59.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb6e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:59.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb6e7fc640 10 monclient: dump: 2024-09-15T01:40:59.544 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.544 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.544 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb7cd99640 4 set_mon_vals no callback set 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb84ec9640 10 monclient: _renew_subs 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb84ec9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.520+0000 7fdb84ec9640 1 -- 172.21.15.178:0/3846128732 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8011d9e0 con 0x7fdb8009faa0 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdb70007890 con 0x7fdb8009faa0 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb84ec9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb84ec9640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb8011eda0 0x7fdb8011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb84ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb84ec9640 1 -- 172.21.15.178:0/3846128732 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8011ead0 con 0x7fdb8011eda0 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb6e7fc640 1 --2- 172.21.15.178:0/3846128732 >> v2:172.21.15.204:6812/28210 conn(0x7fdb600397b0 0x7fdb6003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fdb7004bc60 con 0x7fdb8009faa0 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb7dd9b640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb8011eda0 0x7fdb8011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb7dd9b640 10 monclient: get_auth_request con 0x7fdb8011eda0 auth_method 0 2024-09-15T01:40:59.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb7dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb7dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb7dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb7ed9d640 1 --2- 172.21.15.178:0/3846128732 >> v2:172.21.15.204:6812/28210 conn(0x7fdb600397b0 0x7fdb6003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb7ed9d640 10 monclient: get_auth_request con 0x7fdb600397b0 auth_method 0 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_done global_id 8002 payload 824 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.521+0000 7fdb7dd9b640 1 -- 172.21.15.178:0/3846128732 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb8004c4a0 con 0x7fdb8011eda0 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb7dd9b640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb8011eda0 0x7fdb8011ca70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdb800a5e70 tx=0x7fdb74032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb7ed9d640 1 --2- 172.21.15.178:0/3846128732 >> v2:172.21.15.204:6812/28210 conn(0x7fdb600397b0 0x7fdb6003bc60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdb8009b6d0 tx=0x7fdb78009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb7403c860 con 0x7fdb8011eda0 2024-09-15T01:40:59.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb6e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:40:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb6e7fc640 10 monclient: dump: 2024-09-15T01:40:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fdb7403c3e0 con 0x7fdb8011eda0 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb6e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb6e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb8011eda0 msgr2=0x7fdb8011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb6e7fc640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb8011eda0 0x7fdb8011ca70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdb800a5e70 tx=0x7fdb74032000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb8011eda0 msgr2=0x7fdb8011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.522+0000 7fdb6e7fc640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb8011eda0 0x7fdb8011ca70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb84ec9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb84ec9640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb80131d60 0x7fdb8011cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb84ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb84ec9640 1 -- 172.21.15.178:0/3846128732 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8004bc80 con 0x7fdb80131d60 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb7dd9b640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb80131d60 0x7fdb8011cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb7dd9b640 10 monclient: get_auth_request con 0x7fdb80131d60 auth_method 0 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb7dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb7dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb7dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb7dd9b640 10 monclient(hunting): handle_auth_done global_id 8011 payload 824 2024-09-15T01:40:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.528+0000 7fdb7dd9b640 1 -- 172.21.15.178:0/3846128732 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdb8005ed70 con 0x7fdb80131d60 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb7dd9b640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb80131d60 0x7fdb8011cfb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009910 tx=0x7fdb7403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb74048850 con 0x7fdb80131d60 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb6e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb6e7fc640 10 monclient: dump: 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.554 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9314 (secure 0 0 0) 0x7fdb740483d0 con 0x7fdb80131d60 2024-09-15T01:40:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb6e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb6e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb80131d60 msgr2=0x7fdb8011cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb6e7fc640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb80131d60 0x7fdb8011cfb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009910 tx=0x7fdb7403d000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb6e7fc640 1 -- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb80131d60 msgr2=0x7fdb8011cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:59.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb6e7fc640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb80131d60 0x7fdb8011cfb0 secure :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009910 tx=0x7fdb7403d000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.555 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:59.555 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:59.555 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:59.555 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:40:59.555 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:59.555 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:59.555 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:59.556 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:59.557 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:59.558 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:59.559 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.560 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:59.561 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:59.562 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.563 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.564 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:59.565 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.565 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.565 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.565 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:59.565 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.565 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.565 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.565 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:59.565 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:59.565 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.568 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:59.569 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.571 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.572 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.575 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.577 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:59.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.578 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.578 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.578 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:40:59.578 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.578 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.578 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:59.578 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:59.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb84ec9640 1 -- 172.21.15.178:0/3846128732 >> v2:172.21.15.204:6812/28210 conn(0x7fdb600397b0 msgr2=0x7fdb6003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb84ec9640 1 --2- 172.21.15.178:0/3846128732 >> v2:172.21.15.204:6812/28210 conn(0x7fdb600397b0 0x7fdb6003bc60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdb8009b6d0 tx=0x7fdb78009040 comp rx=0 tx=0).stop 2024-09-15T01:40:59.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb84ec9640 10 monclient: shutdown 2024-09-15T01:40:59.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb84ec9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:59.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb84ec9640 1 -- 172.21.15.178:0/3846128732 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdb8009faa0 msgr2=0x7fdb8009afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb84ec9640 1 --2- 172.21.15.178:0/3846128732 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdb8009faa0 0x7fdb8009afd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000c8b0 tx=0x7fdb70008040 comp rx=0 tx=0).stop 2024-09-15T01:40:59.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb7ed9d640 1 -- 172.21.15.178:0/3846128732 reap_dead start 2024-09-15T01:40:59.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb7ed9d640 1 -- 172.21.15.178:0/3846128732 reap_dead start 2024-09-15T01:40:59.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb84ec9640 1 -- 172.21.15.178:0/3846128732 shutdown_connections 2024-09-15T01:40:59.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb84ec9640 1 -- 172.21.15.178:0/3846128732 >> 172.21.15.178:0/3846128732 conn(0x7fdb800a5ed0 msgr2=0x7fdb8009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:59.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb84ec9640 1 -- 172.21.15.178:0/3846128732 shutdown_connections 2024-09-15T01:40:59.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.529+0000 7fdb84ec9640 1 -- 172.21.15.178:0/3846128732 wait complete. 2024-09-15T01:40:59.581 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:59.581 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:40:59.581 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:40:59.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.673+0000 7f92d5234640 10 monclient: get_monmap_and_config 2024-09-15T01:40:59.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.673+0000 7f92d5234640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.673+0000 7f92d5234640 10 monclient: monmap: 2024-09-15T01:40:59.675 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.675 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.675 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:59.674176+0000 2024-09-15T01:40:59.675 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:59.674176+0000 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:59.676 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.673+0000 7f92d5234640 1 Processor -- start 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.673+0000 7f92d5234640 1 -- start start 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.673+0000 7f92d5234640 10 monclient: init 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92d5234640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92d5234640 10 monclient: _add_conns ranks=[0,8,1,2,6,5,4,7,3,10,9] 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92d5234640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f92d00a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92d5234640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f92d0132de0 0x7f92d0137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92d5234640 10 monclient(hunting): picked mon.noname-e con 0x7f92d0132de0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92d5234640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f92d01380e0 addr v1:172.21.15.204:6789/0 2024-09-15T01:40:59.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92d5234640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d004bf60 con 0x7f92d00a2930 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92d5234640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d009dd80 con 0x7f92d01380e0 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92d5234640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92d5234640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f92d00a2d00 con 0x7f92d0132de0 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92d5234640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92d5234640 10 monclient(hunting): authenticate will time out at 1264.424468s 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f92d0132de0 0x7f92d0137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ced9d640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f92d00a2930 0x7f92d009faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:47292/0 (socket says 172.21.15.178:47292) 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ced9d640 1 -- 172.21.15.178:0/2171190338 learned_addr learned my addr 172.21.15.178:0/2171190338 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 10 monclient(hunting): get_auth_request con 0x7f92d0132de0 auth_method 0 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92cdd9b640 1 -- 172.21.15.178:0/2171190338 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75156306 0 0) 0x7f92d004bf60 con 0x7f92d00a2930 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92cdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92cdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92cdd9b640 10 monclient(hunting): my global_id is 34632 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92cdd9b640 1 -- 172.21.15.178:0/2171190338 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92bc0034e0 con 0x7f92d00a2930 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 10 monclient(hunting): handle_auth_done global_id 34496 payload 274 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 1 -- 172.21.15.178:0/2171190338 >> v1:172.21.15.204:6789/0 conn(0x7f92d01380e0 legacy=0x7f92d013a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 1 -- 172.21.15.178:0/2171190338 >> v1:172.21.15.178:6789/0 conn(0x7f92d00a2930 legacy=0x7f92d009faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 10 monclient: _finish_hunting 0 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 1 monclient: found mon.noname-e 2024-09-15T01:40:59.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.674+0000 7f92ce59c640 1 -- 172.21.15.178:0/2171190338 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d01331b0 con 0x7f92d0132de0 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92ce59c640 1 --2- 172.21.15.178:0/2171190338 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f92d0132de0 0x7f92d0137ba0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f92d009df10 tx=0x7f92c00090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=d1eb120b783281a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92cdd9b640 1 -- 172.21.15.178:0/2171190338 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f92c002acd0 con 0x7f92d0132de0 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92cdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92cdd9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92cdd9b640 10 monclient: dump: 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.689 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92cdd9b640 10 monclient: _finish_auth 0 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92cdd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92d5234640 5 monclient: authenticate success, global_id 34496 2024-09-15T01:40:59.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92cdd9b640 1 -- 172.21.15.178:0/2171190338 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c002ae50 con 0x7f92d0132de0 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92d5234640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92cdd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92d5234640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92cdd9b640 1 -- 172.21.15.178:0/2171190338 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f92c0031ca0 con 0x7f92d0132de0 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92cdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92d5234640 4 set_mon_vals no callback set 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92cdd9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.675+0000 7f92cdd9b640 10 monclient: dump: 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.691 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.676+0000 7f92d5234640 10 monclient: shutdown 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.676+0000 7f92d5234640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:59.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.676+0000 7f92d5234640 1 -- 172.21.15.178:0/2171190338 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f92d0132de0 msgr2=0x7f92d0137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.676+0000 7f92d5234640 1 --2- 172.21.15.178:0/2171190338 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f92d0132de0 0x7f92d0137ba0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f92d009df10 tx=0x7f92c00090f0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.676+0000 7f92d5234640 1 -- 172.21.15.178:0/2171190338 shutdown_connections 2024-09-15T01:40:59.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.676+0000 7f92d5234640 1 --2- 172.21.15.178:0/2171190338 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f92d0132de0 0x7f92d0137ba0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.676+0000 7f92d5234640 1 -- 172.21.15.178:0/2171190338 >> 172.21.15.178:0/2171190338 conn(0x7f92d00a5ed0 msgr2=0x7f92d00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:59.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.676+0000 7f92d5234640 1 -- 172.21.15.178:0/2171190338 shutdown_connections 2024-09-15T01:40:59.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.676+0000 7f92d5234640 1 -- 172.21.15.178:0/2171190338 wait complete. 2024-09-15T01:40:59.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.676+0000 7f92d5234640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.676+0000 7f92d5234640 10 monclient: monmap: 2024-09-15T01:40:59.693 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.693 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:59.693 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:59.693 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 1 Processor -- start 2024-09-15T01:40:59.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 1 -- start start 2024-09-15T01:40:59.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 10 monclient: init 2024-09-15T01:40:59.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:59.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 10 monclient: _add_conns ranks=[6,3,8,1,0,7,4,5,2] 2024-09-15T01:40:59.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f92d00a2930 0x7f92d009c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 10 monclient(hunting): picked mon.noname-d con 0x7f92d00a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:40:59.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d0132de0 0x7f92d0134e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 10 monclient(hunting): picked mon.noname-g con 0x7f92d0132de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:59.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f92d01380e0 0x7f92d01353b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 10 monclient(hunting): picked mon.noname-e con 0x7f92d01380e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:40:59.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d00a2390 con 0x7f92d0132de0 2024-09-15T01:40:59.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f92d009d630 con 0x7f92d00a2930 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f92d0097950 con 0x7f92d01380e0 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92d5234640 10 monclient(hunting): authenticate will time out at 1264.427850s 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92ced9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f92d00a2930 0x7f92d009c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92ced9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f92d00a2930 0x7f92d009c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:35290/0 (socket says 172.21.15.178:35290) 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92ced9d640 1 -- 172.21.15.178:0/3343961181 learned_addr learned my addr 172.21.15.178:0/3343961181 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92ced9d640 10 monclient(hunting): get_auth_request con 0x7f92d00a2930 auth_method 0 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92ced9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92cf59e640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f92d01380e0 0x7f92d01353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92cf59e640 10 monclient(hunting): get_auth_request con 0x7f92d01380e0 auth_method 0 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92cf59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92cf59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.677+0000 7f92cf59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ce59c640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d0132de0 0x7f92d0134e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92cf59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92cf59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92cf59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ce59c640 10 monclient(hunting): get_auth_request con 0x7f92d0132de0 auth_method 0 2024-09-15T01:40:59.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ce59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ce59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ce59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 10 monclient(hunting): handle_auth_done global_id 25053 payload 824 2024-09-15T01:40:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 1 -- 172.21.15.178:0/3343961181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f92d01380e0 msgr2=0x7f92d01353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f92d01380e0 0x7f92d01353b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 1 -- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d0132de0 msgr2=0x7f92d0134e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d0132de0 0x7f92d0134e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 10 monclient: _finish_hunting 0 2024-09-15T01:40:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 1 monclient: found mon.noname-d 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 1 -- 172.21.15.178:0/3343961181 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c0009390 con 0x7f92d00a2930 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92cf59e640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f92d01380e0 0x7f92d01353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92ced9d640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f92d00a2930 0x7f92d009c600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f92b800c4e0 tx=0x7f92b8007d60 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f92b80070d0 con 0x7f92d00a2930 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92c6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92c6ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92c6ffd640 10 monclient: dump: 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.700 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.700 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.700 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.700 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.700 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.700 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.700 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.700 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.700 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.700 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.700 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.700 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92c6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92d5234640 5 monclient: authenticate success, global_id 25053 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b8007250 con 0x7f92d00a2930 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92d5234640 10 monclient: _renew_subs 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92d5234640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92d5234640 1 -- 172.21.15.178:0/3343961181 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d0136f10 con 0x7f92d00a2930 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92c6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f92b8011360 con 0x7f92d00a2930 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92c6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92c6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92c6ffd640 10 monclient: dump: 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.701 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.702 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92cd59a640 4 set_mon_vals no callback set 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92d5234640 10 monclient: _renew_subs 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92d5234640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.678+0000 7f92d5234640 1 -- 172.21.15.178:0/3343961181 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d0137350 con 0x7f92d00a2930 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92d5234640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92d5234640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d01358f0 0x7f92d00a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92d5234640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92d5234640 1 -- 172.21.15.178:0/3343961181 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d004c4a0 con 0x7f92d01358f0 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f92b80073c0 con 0x7f92d00a2930 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92c6ffd640 1 --2- 172.21.15.178:0/3343961181 >> v2:172.21.15.204:6812/28210 conn(0x7f92b00397b0 0x7f92b003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f92b804abf0 con 0x7f92d00a2930 2024-09-15T01:40:59.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92ce59c640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d01358f0 0x7f92d00a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92ce59c640 10 monclient: get_auth_request con 0x7f92d01358f0 auth_method 0 2024-09-15T01:40:59.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92ce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92ce59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92ce59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92ce59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92cf59e640 1 --2- 172.21.15.178:0/3343961181 >> v2:172.21.15.204:6812/28210 conn(0x7f92b00397b0 0x7f92b003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.679+0000 7f92cf59e640 10 monclient: get_auth_request con 0x7f92b00397b0 auth_method 0 2024-09-15T01:40:59.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92ce59c640 10 monclient(hunting): handle_auth_done global_id 16158 payload 824 2024-09-15T01:40:59.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92ce59c640 1 -- 172.21.15.178:0/3343961181 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92d005ed70 con 0x7f92d01358f0 2024-09-15T01:40:59.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92ce59c640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d01358f0 0x7f92d00a4590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f92d00a2f70 tx=0x7f92c0032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92cf59e640 1 --2- 172.21.15.178:0/3343961181 >> v2:172.21.15.204:6812/28210 conn(0x7f92b00397b0 0x7f92b003bc60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f92d0134dc0 tx=0x7f92c8009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f92c003cad0 con 0x7f92d01358f0 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92c6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92c6ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92c6ffd640 10 monclient: dump: 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.705 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f92c003cc30 con 0x7f92d01358f0 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92c6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92c6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d01358f0 msgr2=0x7f92d00a4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92c6ffd640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d01358f0 0x7f92d00a4590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f92d00a2f70 tx=0x7f92c0032000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d01358f0 msgr2=0x7f92d00a4590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:59.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.680+0000 7f92c6ffd640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d01358f0 0x7f92d00a4590 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.686+0000 7f92d5234640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:59.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.686+0000 7f92d5234640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d00a4ad0 0x7f92d00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.686+0000 7f92d5234640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.686+0000 7f92d5234640 1 -- 172.21.15.178:0/3343961181 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d004bc80 con 0x7f92d00a4ad0 2024-09-15T01:40:59.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.686+0000 7f92ce59c640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d00a4ad0 0x7f92d00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.686+0000 7f92ce59c640 10 monclient: get_auth_request con 0x7f92d00a4ad0 auth_method 0 2024-09-15T01:40:59.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.686+0000 7f92ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.686+0000 7f92ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.686+0000 7f92ce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.686+0000 7f92ce59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.686+0000 7f92ce59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.686+0000 7f92ce59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92ce59c640 10 monclient(hunting): handle_auth_done global_id 16167 payload 824 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92ce59c640 1 -- 172.21.15.178:0/3343961181 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f92d004b720 con 0x7f92d00a4ad0 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92ce59c640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d00a4ad0 0x7f92d00a4ed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f92c0000c00 tx=0x7f92c003d000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f92c003ce80 con 0x7f92d00a4ad0 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92c6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92c6ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92c6ffd640 10 monclient: dump: 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.710 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.711 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.711 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.711 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.711 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.711 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.711 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.711 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.711 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.711 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f92c003b3d0 con 0x7f92d00a4ad0 2024-09-15T01:40:59.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92c6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:59.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92c6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:59.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d00a4ad0 msgr2=0x7f92d00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92c6ffd640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d00a4ad0 0x7f92d00a4ed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f92c0000c00 tx=0x7f92c003d000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92c6ffd640 1 -- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d00a4ad0 msgr2=0x7f92d00a4ed0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:59.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.687+0000 7f92c6ffd640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92d00a4ad0 0x7f92d00a4ed0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.712 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:59.712 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:59.712 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:59.713 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:59.713 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:59.713 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:59.713 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:59.713 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:59.713 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:59.713 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:59.713 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:59.713 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:59.713 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:59.713 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:59.713 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:59.714 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.714 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:40:59.714 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:59.714 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:59.714 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:59.714 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.714 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.714 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.714 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.714 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.714 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.714 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.715 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.715 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.715 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.715 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:59.715 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:59.715 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.715 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.715 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.715 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.715 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.715 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.715 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:59.716 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:59.717 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:59.717 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:59.717 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:59.717 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:59.717 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:59.717 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:59.717 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:59.717 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:59.717 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.717 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.717 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.718 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.718 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.718 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.718 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.718 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.718 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.718 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.718 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:59.718 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.719 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.720 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:59.720 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.720 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.720 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.720 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.720 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:59.720 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:59.720 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.720 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.720 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.720 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.720 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:59.721 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:59.722 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.723 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:59.724 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:59.724 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.724 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.724 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.724 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.724 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.724 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:59.724 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:59.724 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.724 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.724 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.724 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.725 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.726 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.727 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:59.728 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:59.729 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.730 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.731 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.732 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.734 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.735 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.736 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:40:59.736 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.736 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.736 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.736 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:59.736 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.736 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.736 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.736 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:59.736 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:59.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.688+0000 7f92d5234640 1 -- 172.21.15.178:0/3343961181 >> v2:172.21.15.204:6812/28210 conn(0x7f92b00397b0 msgr2=0x7f92b003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.688+0000 7f92d5234640 1 --2- 172.21.15.178:0/3343961181 >> v2:172.21.15.204:6812/28210 conn(0x7f92b00397b0 0x7f92b003bc60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f92d0134dc0 tx=0x7f92c8009040 comp rx=0 tx=0).stop 2024-09-15T01:40:59.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.688+0000 7f92d5234640 10 monclient: shutdown 2024-09-15T01:40:59.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.688+0000 7f92d5234640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:59.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.688+0000 7f92d5234640 1 -- 172.21.15.178:0/3343961181 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f92d00a2930 msgr2=0x7f92d009c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.688+0000 7f92d5234640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f92d00a2930 0x7f92d009c600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f92b800c4e0 tx=0x7f92b8007d60 comp rx=0 tx=0).stop 2024-09-15T01:40:59.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.688+0000 7f92cf59e640 1 -- 172.21.15.178:0/3343961181 reap_dead start 2024-09-15T01:40:59.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.688+0000 7f92d5234640 1 -- 172.21.15.178:0/3343961181 shutdown_connections 2024-09-15T01:40:59.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.688+0000 7f92d5234640 1 --2- 172.21.15.178:0/3343961181 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f92d00a2930 0x7f92d009c600 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.688+0000 7f92d5234640 1 -- 172.21.15.178:0/3343961181 >> 172.21.15.178:0/3343961181 conn(0x7f92d00a5ed0 msgr2=0x7f92d009fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:59.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.688+0000 7f92d5234640 1 -- 172.21.15.178:0/3343961181 shutdown_connections 2024-09-15T01:40:59.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.688+0000 7f92d5234640 1 -- 172.21.15.178:0/3343961181 wait complete. 2024-09-15T01:40:59.739 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:59.739 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:40:59.739 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:40:59.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.831+0000 7ff475d1d640 10 monclient: get_monmap_and_config 2024-09-15T01:40:59.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.831+0000 7ff475d1d640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.831+0000 7ff475d1d640 10 monclient: monmap: 2024-09-15T01:40:59.833 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.833 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.833 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:59.832143+0000 2024-09-15T01:40:59.833 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:59.832143+0000 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:59.834 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 1 Processor -- start 2024-09-15T01:40:59.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 1 -- start start 2024-09-15T01:40:59.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 10 monclient: init 2024-09-15T01:40:59.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:59.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 10 monclient: _add_conns ranks=[6,1,10,7,3,0,8,9,4,5,2] 2024-09-15T01:40:59.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4700a2930 0x7ff470132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 10 monclient(hunting): picked mon.noname-d con 0x7ff4700a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:40:59.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff47009faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff47009fe70 0x7ff4700a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 10 monclient(hunting): picked mon.noname-f con 0x7ff47009fe70 addr v2:172.21.15.204:3300/0 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47004bf60 con 0x7ff47009faa0 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47009dd80 con 0x7ff47009fe70 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700a2d00 con 0x7ff4700a2930 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff475d1d640 10 monclient(hunting): authenticate will time out at 1264.582558s 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4700a2930 0x7ff470132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4700a2930 0x7ff470132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:35296/0 (socket says 172.21.15.178:35296) 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 1 -- 172.21.15.178:0/1124133511 learned_addr learned my addr 172.21.15.178:0/1124133511 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:59.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 10 monclient(hunting): get_auth_request con 0x7ff4700a2930 auth_method 0 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46ffff640 1 --2- 172.21.15.178:0/1124133511 >> v2:172.21.15.204:3300/0 conn(0x7ff47009fe70 0x7ff4700a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46ffff640 10 monclient(hunting): get_auth_request con 0x7ff47009fe70 auth_method 0 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 10 monclient(hunting): handle_auth_done global_id 25062 payload 274 2024-09-15T01:40:59.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 1 -- 172.21.15.178:0/1124133511 >> v2:172.21.15.204:3300/0 conn(0x7ff47009fe70 msgr2=0x7ff4700a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 1 --2- 172.21.15.178:0/1124133511 >> v2:172.21.15.204:3300/0 conn(0x7ff47009fe70 0x7ff4700a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 1 -- 172.21.15.178:0/1124133511 >> v1:172.21.15.204:6789/0 conn(0x7ff47009faa0 legacy=0x7ff4700a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 1 monclient: found mon.noname-d 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.832+0000 7ff46f7fe640 1 -- 172.21.15.178:0/1124133511 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4700a0310 con 0x7ff4700a2930 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46f7fe640 1 --2- 172.21.15.178:0/1124133511 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4700a2930 0x7ff470132de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff460009f20 tx=0x7ff460009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c337d66780de4927 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46e7fc640 1 -- 172.21.15.178:0/1124133511 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff460007880 con 0x7ff4700a2930 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46e7fc640 10 monclient: dump: 2024-09-15T01:40:59.847 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.848 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46e7fc640 1 -- 172.21.15.178:0/1124133511 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46002d050 con 0x7ff4700a2930 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff475d1d640 5 monclient: authenticate success, global_id 25062 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff475d1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff475d1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46e7fc640 1 -- 172.21.15.178:0/1124133511 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff460030360 con 0x7ff4700a2930 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff475d1d640 4 set_mon_vals no callback set 2024-09-15T01:40:59.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:40:59.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.833+0000 7ff46e7fc640 10 monclient: dump: 2024-09-15T01:40:59.850 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.850 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.850 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.850 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.850 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.850 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.850 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.850 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.850 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.850 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.850 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.834+0000 7ff475d1d640 10 monclient: shutdown 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.834+0000 7ff475d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.834+0000 7ff475d1d640 1 -- 172.21.15.178:0/1124133511 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4700a2930 msgr2=0x7ff470132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.834+0000 7ff475d1d640 1 --2- 172.21.15.178:0/1124133511 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4700a2930 0x7ff470132de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff460009f20 tx=0x7ff460009840 comp rx=0 tx=0).stop 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.834+0000 7ff475d1d640 1 -- 172.21.15.178:0/1124133511 shutdown_connections 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.834+0000 7ff475d1d640 1 --2- 172.21.15.178:0/1124133511 >> v2:172.21.15.204:3300/0 conn(0x7ff47009fe70 0x7ff4700a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.834+0000 7ff475d1d640 1 --2- 172.21.15.178:0/1124133511 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff4700a2930 0x7ff470132de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.834+0000 7ff475d1d640 1 -- 172.21.15.178:0/1124133511 >> 172.21.15.178:0/1124133511 conn(0x7ff4700a5ed0 msgr2=0x7ff4700a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:59.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.834+0000 7ff475d1d640 1 -- 172.21.15.178:0/1124133511 shutdown_connections 2024-09-15T01:40:59.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.834+0000 7ff475d1d640 1 -- 172.21.15.178:0/1124133511 wait complete. 2024-09-15T01:40:59.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.834+0000 7ff475d1d640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.834+0000 7ff475d1d640 10 monclient: monmap: 2024-09-15T01:40:59.852 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.852 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:40:59.852 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:40:59.852 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:40:59.852 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.852 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:40:59.852 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:40:59.852 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:40:59.852 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 1 Processor -- start 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 1 -- start start 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 10 monclient: init 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 10 monclient: _add_conns ranks=[3,7,1,5,2,4,8,0,6] 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff47009faa0 0x7ff47009b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 10 monclient(hunting): picked mon.noname-g con 0x7ff47009faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:40:59.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff47009fe70 0x7ff470131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 10 monclient(hunting): picked mon.noname-i con 0x7ff47009fe70 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:40:59.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4700a2930 0x7ff470141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 10 monclient(hunting): picked mon.noname-f con 0x7ff4700a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:40:59.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47009c120 con 0x7ff4700a2930 2024-09-15T01:40:59.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47009d6a0 con 0x7ff47009faa0 2024-09-15T01:40:59.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff47009f870 con 0x7ff47009fe70 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.835+0000 7ff475d1d640 10 monclient(hunting): authenticate will time out at 1264.585839s 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4700a2930 0x7ff470141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46effd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff47009fe70 0x7ff470131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff47009faa0 0x7ff47009b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4700a2930 0x7ff470141e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:44872/0 (socket says 172.21.15.178:44872) 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46ffff640 1 -- 172.21.15.178:0/2701917301 learned_addr learned my addr 172.21.15.178:0/2701917301 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46ffff640 10 monclient(hunting): get_auth_request con 0x7ff4700a2930 auth_method 0 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 10 monclient(hunting): get_auth_request con 0x7ff47009faa0 auth_method 0 2024-09-15T01:40:59.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46effd640 10 monclient(hunting): get_auth_request con 0x7ff47009fe70 auth_method 0 2024-09-15T01:40:59.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 10 monclient(hunting): handle_auth_done global_id 16176 payload 824 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 1 -- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff47009fe70 msgr2=0x7ff470131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff47009fe70 0x7ff470131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 1 -- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4700a2930 msgr2=0x7ff470141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4700a2930 0x7ff470141e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 1 monclient: found mon.noname-g 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:40:59.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 1 -- 172.21.15.178:0/2701917301 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff460009ae0 con 0x7ff47009faa0 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46ffff640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4700a2930 0x7ff470141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46f7fe640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff47009faa0 0x7ff47009b5a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff460030000 tx=0x7ff460036000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff460007880 con 0x7ff47009faa0 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff4577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff4577fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff4577fe640 10 monclient: dump: 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.860 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff4577fe640 10 monclient: _finish_auth 0 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff4577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff475d1d640 5 monclient: authenticate success, global_id 16176 2024-09-15T01:40:59.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff475d1d640 10 monclient: _renew_subs 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4600079e0 con 0x7ff47009faa0 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff475d1d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff475d1d640 1 -- 172.21.15.178:0/2701917301 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47011d720 con 0x7ff47009faa0 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff4577fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff46003a370 con 0x7ff47009faa0 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff4577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff4577fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff4577fe640 10 monclient: dump: 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff46dffb640 4 set_mon_vals no callback set 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff475d1d640 10 monclient: _renew_subs 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff475d1d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:40:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.836+0000 7ff475d1d640 1 -- 172.21.15.178:0/2701917301 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47011db50 con 0x7ff47009faa0 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff475d1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff475d1d640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011ebe0 0x7ff470142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff475d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff475d1d640 1 -- 172.21.15.178:0/2701917301 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff47004c4a0 con 0x7ff47011ebe0 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff460038070 con 0x7ff47009faa0 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff46effd640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011ebe0 0x7ff470142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff46effd640 10 monclient: get_auth_request con 0x7ff47011ebe0 auth_method 0 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff46effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff46effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff46effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff46effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff46effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff46effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff4577fe640 1 --2- 172.21.15.178:0/2701917301 >> v2:172.21.15.204:6812/28210 conn(0x7ff45c0397b0 0x7ff45c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.837+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7ff46006bd70 con 0x7ff47009faa0 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff46effd640 10 monclient(hunting): handle_auth_done global_id 6962 payload 824 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff46effd640 1 -- 172.21.15.178:0/2701917301 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff47005ed70 con 0x7ff47011ebe0 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff46ffff640 1 --2- 172.21.15.178:0/2701917301 >> v2:172.21.15.204:6812/28210 conn(0x7ff45c0397b0 0x7ff45c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff46ffff640 10 monclient: get_auth_request con 0x7ff45c0397b0 auth_method 0 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff46effd640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011ebe0 0x7ff470142380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff464000c00 tx=0x7ff464007e10 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff464002b20 con 0x7ff47011ebe0 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff4577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff4577fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff4577fe640 10 monclient: dump: 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.867 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff46ffff640 1 --2- 172.21.15.178:0/2701917301 >> v2:172.21.15.204:6812/28210 conn(0x7ff45c0397b0 0x7ff45c03bc60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff468010370 tx=0x7ff468008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7ff464002c80 con 0x7ff47011ebe0 2024-09-15T01:40:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff4577fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:40:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff4577fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:40:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011ebe0 msgr2=0x7ff470142380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff4577fe640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011ebe0 0x7ff470142380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff464000c00 tx=0x7ff464007e10 comp rx=0 tx=0).stop 2024-09-15T01:40:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011ebe0 msgr2=0x7ff470142380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.838+0000 7ff4577fe640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011ebe0 0x7ff470142380 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.844+0000 7ff475d1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:40:59.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.844+0000 7ff475d1d640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011c9f0 0x7ff4701428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:40:59.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.844+0000 7ff475d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:40:59.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.844+0000 7ff475d1d640 1 -- 172.21.15.178:0/2701917301 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff47004bc80 con 0x7ff47011c9f0 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.844+0000 7ff46effd640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011c9f0 0x7ff4701428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.844+0000 7ff46effd640 10 monclient: get_auth_request con 0x7ff47011c9f0 auth_method 0 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.844+0000 7ff46effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.844+0000 7ff46effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.844+0000 7ff46effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.844+0000 7ff46effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.844+0000 7ff46effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.844+0000 7ff46effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff46effd640 10 monclient(hunting): handle_auth_done global_id 6971 payload 824 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff46effd640 1 -- 172.21.15.178:0/2701917301 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff47011f120 con 0x7ff47011c9f0 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff46effd640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011c9f0 0x7ff4701428c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff464003f10 tx=0x7ff464018000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff464006790 con 0x7ff47011c9f0 2024-09-15T01:40:59.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff4577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff4577fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff4577fe640 10 monclient: dump: 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:40:59.871 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:40:59.872 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:40:59.872 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:40:59.872 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:40:59.872 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7ff464002030 con 0x7ff47011c9f0 2024-09-15T01:40:59.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff4577fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:40:59.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff4577fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:40:59.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011c9f0 msgr2=0x7ff4701428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff4577fe640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011c9f0 0x7ff4701428c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff464003f10 tx=0x7ff464018000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff4577fe640 1 -- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011c9f0 msgr2=0x7ff4701428c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:40:59.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.845+0000 7ff4577fe640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff47011c9f0 0x7ff4701428c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:40:59.873 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:40:59.873 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:59.873 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:59.873 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:40:59.873 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:40:59.873 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:40:59.873 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:40:59.873 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:40:59.873 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:40:59.873 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:40:59.873 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:40:59.873 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.874 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.875 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:40:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:40:59.877 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.878 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.879 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.882 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.883 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.884 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:40:59.885 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.886 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.887 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:40:59.888 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.890 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.891 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.892 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.892 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:40:59.892 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:40:59.892 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:40:59.892 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:40:59.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.892 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:40:59.892 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:40:59.892 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:40:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.894 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:40:59.895 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:40:59.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.846+0000 7ff475d1d640 1 -- 172.21.15.178:0/2701917301 >> v2:172.21.15.204:6812/28210 conn(0x7ff45c0397b0 msgr2=0x7ff45c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.846+0000 7ff475d1d640 1 --2- 172.21.15.178:0/2701917301 >> v2:172.21.15.204:6812/28210 conn(0x7ff45c0397b0 0x7ff45c03bc60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff468010370 tx=0x7ff468008040 comp rx=0 tx=0).stop 2024-09-15T01:40:59.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.846+0000 7ff475d1d640 10 monclient: shutdown 2024-09-15T01:40:59.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.846+0000 7ff475d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:40:59.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.846+0000 7ff475d1d640 1 -- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff47009faa0 msgr2=0x7ff47009b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:40:59.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.846+0000 7ff475d1d640 1 --2- 172.21.15.178:0/2701917301 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff47009faa0 0x7ff47009b5a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff460030000 tx=0x7ff460036000 comp rx=0 tx=0).stop 2024-09-15T01:40:59.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.846+0000 7ff46ffff640 1 -- 172.21.15.178:0/2701917301 reap_dead start 2024-09-15T01:40:59.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.846+0000 7ff46ffff640 1 -- 172.21.15.178:0/2701917301 reap_dead start 2024-09-15T01:40:59.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.846+0000 7ff475d1d640 1 -- 172.21.15.178:0/2701917301 shutdown_connections 2024-09-15T01:40:59.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.846+0000 7ff475d1d640 1 -- 172.21.15.178:0/2701917301 >> 172.21.15.178:0/2701917301 conn(0x7ff4700a5ed0 msgr2=0x7ff470135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:40:59.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.846+0000 7ff475d1d640 1 -- 172.21.15.178:0/2701917301 shutdown_connections 2024-09-15T01:40:59.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.846+0000 7ff475d1d640 1 -- 172.21.15.178:0/2701917301 wait complete. 2024-09-15T01:40:59.898 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:40:59.898 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:40:59.898 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:40:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.991+0000 7ff12d19d640 10 monclient: get_monmap_and_config 2024-09-15T01:40:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.991+0000 7ff12d19d640 10 monclient: build_initial_monmap 2024-09-15T01:40:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.991+0000 7ff12d19d640 10 monclient: monmap: 2024-09-15T01:40:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:40:59.994 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:40:59.994 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:40:59.992195+0000 2024-09-15T01:40:59.994 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:40:59.992195+0000 2024-09-15T01:40:59.994 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:40:59.994 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:40:59.994 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:40:59.994 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:40:59.994 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:40:59.994 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:40:59.994 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:40:59.994 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:40:59.994 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:40:59.995 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:40:59.995 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:40:59.995 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:40:59.995 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:40:59.995 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:40:59.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.991+0000 7ff12d19d640 1 Processor -- start 2024-09-15T01:40:59.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.991+0000 7ff12d19d640 1 -- start start 2024-09-15T01:40:59.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.991+0000 7ff12d19d640 10 monclient: init 2024-09-15T01:41:00.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:00.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 10 monclient: _add_conns ranks=[6,4,3,9,5,2,10,7,1,0,8] 2024-09-15T01:41:00.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff1280a2930 0x7ff128132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:00.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 10 monclient(hunting): picked mon.noname-d con 0x7ff1280a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:41:00.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff12809faa0 0x7ff12809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:00.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 10 monclient(hunting): picked mon.noname-c con 0x7ff12809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1280a08c0 0x7ff1280a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 10 monclient(hunting): picked mon.noname-g con 0x7ff1280a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12809e0f0 con 0x7ff1280a08c0 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff12809dd80 con 0x7ff12809faa0 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280a2d00 con 0x7ff1280a2930 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12d19d640 10 monclient(hunting): authenticate will time out at 1264.742489s 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff12809faa0 0x7ff12809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff1280a2930 0x7ff128132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:00.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff12809faa0 0x7ff12809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:57190/0 (socket says 172.21.15.178:57190) 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12659c640 1 -- 172.21.15.178:0/61328275 learned_addr learned my addr 172.21.15.178:0/61328275 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12659c640 10 monclient(hunting): get_auth_request con 0x7ff12809faa0 auth_method 0 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 10 monclient(hunting): get_auth_request con 0x7ff1280a2930 auth_method 0 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:00.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 10 monclient(hunting): handle_auth_done global_id 25071 payload 274 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12759e640 1 --2- 172.21.15.178:0/61328275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1280a08c0 0x7ff1280a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 1 -- 172.21.15.178:0/61328275 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff12809faa0 msgr2=0x7ff12809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 1 --2- 172.21.15.178:0/61328275 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff12809faa0 0x7ff12809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 1 -- 172.21.15.178:0/61328275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1280a08c0 msgr2=0x7ff1280a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 1 --2- 172.21.15.178:0/61328275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1280a08c0 0x7ff1280a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 1 monclient: found mon.noname-d 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff126d9d640 1 -- 172.21.15.178:0/61328275 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12804bf60 con 0x7ff1280a2930 2024-09-15T01:41:00.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.992+0000 7ff12659c640 1 --2- 172.21.15.178:0/61328275 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff12809faa0 0x7ff12809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff126d9d640 1 --2- 172.21.15.178:0/61328275 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff1280a2930 0x7ff128132de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff110009880 tx=0x7ff1100090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=8934c37b294acad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff125d9b640 1 -- 172.21.15.178:0/61328275 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff11002abb0 con 0x7ff1280a2930 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff125d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff125d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff125d9b640 10 monclient: dump: 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:00.008 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff125d9b640 10 monclient: _finish_auth 0 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff125d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff125d9b640 1 -- 172.21.15.178:0/61328275 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11002ad30 con 0x7ff1280a2930 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff12d19d640 5 monclient: authenticate success, global_id 25071 2024-09-15T01:41:00.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff12d19d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff125d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff12d19d640 10 monclient: get_monmap_and_config success 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff12d19d640 4 set_mon_vals no callback set 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff125d9b640 1 -- 172.21.15.178:0/61328275 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff110031ca0 con 0x7ff1280a2930 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff125d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff125d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.993+0000 7ff125d9b640 10 monclient: dump: 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:00.010 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 10 monclient: shutdown 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 1 -- 172.21.15.178:0/61328275 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff1280a2930 msgr2=0x7ff128132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:00.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 1 --2- 172.21.15.178:0/61328275 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff1280a2930 0x7ff128132de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff110009880 tx=0x7ff1100090f0 comp rx=0 tx=0).stop 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 1 -- 172.21.15.178:0/61328275 shutdown_connections 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 1 --2- 172.21.15.178:0/61328275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1280a08c0 0x7ff1280a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 1 --2- 172.21.15.178:0/61328275 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff12809faa0 0x7ff12809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 1 --2- 172.21.15.178:0/61328275 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff1280a2930 0x7ff128132de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 1 -- 172.21.15.178:0/61328275 >> 172.21.15.178:0/61328275 conn(0x7ff1280a5ed0 msgr2=0x7ff1280a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 1 -- 172.21.15.178:0/61328275 shutdown_connections 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 1 -- 172.21.15.178:0/61328275 wait complete. 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 10 monclient: build_initial_monmap 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.994+0000 7ff12d19d640 10 monclient: monmap: 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:41:00.012 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:00.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 1 Processor -- start 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 1 -- start start 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 10 monclient: init 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 10 monclient: _add_conns ranks=[3,4,5,0,1,8,6,2,7] 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff12809faa0 0x7ff12809b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 10 monclient(hunting): picked mon.noname-g con 0x7ff12809faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff1280a08c0 0x7ff128131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 10 monclient(hunting): picked mon.noname-c con 0x7ff1280a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff1280a2930 0x7ff128141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 10 monclient(hunting): picked mon.noname-h con 0x7ff1280a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12809d630 con 0x7ff12809faa0 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:00.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff12811ed20 con 0x7ff1280a08c0 2024-09-15T01:41:00.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:00.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff12811ea40 con 0x7ff1280a2930 2024-09-15T01:41:00.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:00.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12d19d640 10 monclient(hunting): authenticate will time out at 1264.745740s 2024-09-15T01:41:00.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff1280a08c0 0x7ff128131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:00.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff1280a08c0 0x7ff128131920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:57200/0 (socket says 172.21.15.178:57200) 2024-09-15T01:41:00.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12659c640 1 -- 172.21.15.178:0/885978934 learned_addr learned my addr 172.21.15.178:0/885978934 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:00.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12659c640 10 monclient(hunting): get_auth_request con 0x7ff1280a08c0 auth_method 0 2024-09-15T01:41:00.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:00.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:00.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:00.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff126d9d640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff12809faa0 0x7ff12809b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12759e640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff1280a2930 0x7ff128141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff126d9d640 10 monclient(hunting): get_auth_request con 0x7ff12809faa0 auth_method 0 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff126d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff126d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff126d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12759e640 10 monclient(hunting): get_auth_request con 0x7ff1280a2930 auth_method 0 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12759e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.995+0000 7ff12759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:00.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff126d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff126d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff126d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12659c640 10 monclient(hunting): handle_auth_done global_id 44284 payload 824 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12659c640 1 -- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff1280a2930 msgr2=0x7ff128141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12659c640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff1280a2930 0x7ff128141d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12659c640 1 -- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff12809faa0 msgr2=0x7ff12809b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12659c640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff12809faa0 0x7ff12809b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12659c640 10 monclient: _finish_hunting 0 2024-09-15T01:41:00.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12659c640 1 monclient: found mon.noname-c 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12659c640 1 -- 172.21.15.178:0/885978934 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118009450 con 0x7ff1280a08c0 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12659c640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff1280a08c0 0x7ff128131920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff11800ea50 tx=0x7ff1180091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff11800ce90 con 0x7ff1280a08c0 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff11effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff11effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff11effd640 10 monclient: dump: 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:00.018 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff11effd640 10 monclient: _finish_auth 0 2024-09-15T01:41:00.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff11effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff118013470 con 0x7ff1280a08c0 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff11effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff118013a60 con 0x7ff1280a08c0 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12d19d640 5 monclient: authenticate success, global_id 44284 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12d19d640 10 monclient: _renew_subs 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12d19d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12d19d640 1 -- 172.21.15.178:0/885978934 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff110009390 con 0x7ff1280a08c0 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff11effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12559a640 4 set_mon_vals no callback set 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff11effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff11effd640 10 monclient: dump: 2024-09-15T01:41:00.020 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:00.021 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12d19d640 10 monclient: _renew_subs 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12d19d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.996+0000 7ff12d19d640 1 -- 172.21.15.178:0/885978934 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12811d780 con 0x7ff1280a08c0 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff12d19d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff12d19d640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff12811db50 0x7ff128142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff12d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff12d19d640 1 -- 172.21.15.178:0/885978934 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff12804c4a0 con 0x7ff12811db50 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff118013bc0 con 0x7ff1280a08c0 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff126d9d640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff12811db50 0x7ff128142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff126d9d640 10 monclient: get_auth_request con 0x7ff12811db50 auth_method 0 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff126d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff126d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:00.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff126d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff11effd640 1 --2- 172.21.15.178:0/885978934 >> v2:172.21.15.204:6812/28210 conn(0x7ff1080396e0 0x7ff10803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff126d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff126d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7ff11804bec0 con 0x7ff1280a08c0 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff126d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff12759e640 1 --2- 172.21.15.178:0/885978934 >> v2:172.21.15.204:6812/28210 conn(0x7ff1080396e0 0x7ff10803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff12759e640 10 monclient: get_auth_request con 0x7ff1080396e0 auth_method 0 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff126d9d640 10 monclient(hunting): handle_auth_done global_id 44188 payload 824 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff126d9d640 1 -- 172.21.15.178:0/885978934 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff12805ed70 con 0x7ff12811db50 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.997+0000 7ff126d9d640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff12811db50 0x7ff128142290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff110009850 tx=0x7ff110032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.998+0000 7ff12759e640 1 --2- 172.21.15.178:0/885978934 >> v2:172.21.15.204:6812/28210 conn(0x7ff1080396e0 0x7ff10803bb90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff12809b7a0 tx=0x7ff1200077d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.998+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff11002aea0 con 0x7ff12811db50 2024-09-15T01:41:00.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.998+0000 7ff11effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:00.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.998+0000 7ff11effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:41:00.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.998+0000 7ff11effd640 10 monclient: dump: 2024-09-15T01:41:00.024 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:00.024 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:00.024 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:00.024 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:00.024 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:00.024 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:00.024 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:00.024 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:00.024 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:00.024 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.998+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7ff11002abb0 con 0x7ff12811db50 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.998+0000 7ff11effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.998+0000 7ff11effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.998+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff12811db50 msgr2=0x7ff128142290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.998+0000 7ff11effd640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff12811db50 0x7ff128142290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff110009850 tx=0x7ff110032000 comp rx=0 tx=0).stop 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.998+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff12811db50 msgr2=0x7ff128142290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:00.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:40:59.998+0000 7ff11effd640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff12811db50 0x7ff128142290 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff110009850 tx=0x7ff110032000 comp rx=0 tx=0).stop 2024-09-15T01:41:00.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff12d19d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:41:00.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff12d19d640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff128131e60 0x7ff1281427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:00.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff12d19d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:00.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff12d19d640 1 -- 172.21.15.178:0/885978934 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff12804bc80 con 0x7ff128131e60 2024-09-15T01:41:00.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff126d9d640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff128131e60 0x7ff1281427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:00.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff126d9d640 10 monclient: get_auth_request con 0x7ff128131e60 auth_method 0 2024-09-15T01:41:00.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff126d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:00.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff126d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:00.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff126d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:00.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff126d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:00.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff126d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:00.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff126d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff126d9d640 10 monclient(hunting): handle_auth_done global_id 44197 payload 824 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff126d9d640 1 -- 172.21.15.178:0/885978934 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff1280a2400 con 0x7ff128131e60 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.004+0000 7ff126d9d640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff128131e60 0x7ff1281427d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff110009720 tx=0x7ff11003e000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff11004b070 con 0x7ff128131e60 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff11effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff11effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff11effd640 10 monclient: dump: 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:00.029 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7ff1100315b0 con 0x7ff128131e60 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff11effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:41:00.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff11effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:41:00.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff128131e60 msgr2=0x7ff1281427d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:00.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff11effd640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff128131e60 0x7ff1281427d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff110009720 tx=0x7ff11003e000 comp rx=0 tx=0).stop 2024-09-15T01:41:00.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff11effd640 1 -- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff128131e60 msgr2=0x7ff1281427d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:00.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff11effd640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff128131e60 0x7ff1281427d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:00.031 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:41:00.031 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:41:00.031 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:41:00.031 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:41:00.031 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 110, 2024-09-15T01:41:00.031 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:41:00.032 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:41:00.032 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:41:00.032 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:41:00.032 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:41:00.032 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:41:00.032 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:41:00.032 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:41:00.032 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:41:00.032 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:41:00.032 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:00.032 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 5, 2024-09-15T01:41:00.032 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:00.033 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:41:00.033 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:41:00.033 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:00.033 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:00.033 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:00.033 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:00.033 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:00.033 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:00.033 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:00.033 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:00.033 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:00.033 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:00.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:41:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:00.036 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:41:00.037 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:00.038 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.039 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.040 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:41:00.041 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:41:00.041 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:00.041 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:00.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.041 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:00.041 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:41:00.041 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.041 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:00.041 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:41:00.041 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:41:00.042 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.044 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:41:00.045 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:41:00.046 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:00.047 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:00.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.048 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:41:00.048 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:41:00.048 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:00.048 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:00.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.048 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:00.048 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:41:00.048 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.049 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.050 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:00.051 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.052 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:00.053 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:00.054 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:00.054 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:00.054 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:41:00.054 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:00.054 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:00.054 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:00.054 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:00.054 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:00.054 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:00.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:00.054 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:41:00.054 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:41:00.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff12d19d640 1 -- 172.21.15.178:0/885978934 >> v2:172.21.15.204:6812/28210 conn(0x7ff1080396e0 msgr2=0x7ff10803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:00.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff12d19d640 1 --2- 172.21.15.178:0/885978934 >> v2:172.21.15.204:6812/28210 conn(0x7ff1080396e0 0x7ff10803bb90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff12809b7a0 tx=0x7ff1200077d0 comp rx=0 tx=0).stop 2024-09-15T01:41:00.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff12d19d640 10 monclient: shutdown 2024-09-15T01:41:00.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff12d19d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:00.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff12d19d640 1 -- 172.21.15.178:0/885978934 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff1280a08c0 msgr2=0x7ff128131920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:00.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff12d19d640 1 --2- 172.21.15.178:0/885978934 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff1280a08c0 0x7ff128131920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff11800ea50 tx=0x7ff1180091d0 comp rx=0 tx=0).stop 2024-09-15T01:41:00.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff12759e640 1 -- 172.21.15.178:0/885978934 reap_dead start 2024-09-15T01:41:00.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff12759e640 1 -- 172.21.15.178:0/885978934 reap_dead start 2024-09-15T01:41:00.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff12d19d640 1 -- 172.21.15.178:0/885978934 shutdown_connections 2024-09-15T01:41:00.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff12d19d640 1 -- 172.21.15.178:0/885978934 >> 172.21.15.178:0/885978934 conn(0x7ff1280a5ed0 msgr2=0x7ff128135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:00.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff12d19d640 1 -- 172.21.15.178:0/885978934 shutdown_connections 2024-09-15T01:41:00.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:00.005+0000 7ff12d19d640 1 -- 172.21.15.178:0/885978934 wait complete. 2024-09-15T01:41:00.057 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:41:00.057 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['f'] 2024-09-15T01:41:00.057 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['e'] 2024-09-15T01:41:00.058 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-15T01:41:00.058 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T01:41:00.058 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-15T01:41:00.058 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:41:00.077 INFO:tasks.ceph.mon.f:Stopped 2024-09-15T01:41:00.077 INFO:tasks.mon_thrash:Sending STOP to mon e 2024-09-15T01:41:00.077 INFO:tasks.ceph.mon.e:Sent signal 19 2024-09-15T01:41:00.077 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T01:41:15.078 INFO:tasks.mon_thrash:Sending CONT to mon e 2024-09-15T01:41:15.078 INFO:tasks.ceph.mon.e:Sent signal 18 2024-09-15T01:41:15.079 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-09-15T01:41:15.079 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:41:15.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.182+0000 7f782be80640 10 monclient: get_monmap_and_config 2024-09-15T01:41:15.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.182+0000 7f782be80640 10 monclient: build_initial_monmap 2024-09-15T01:41:15.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.182+0000 7f782be80640 10 monclient: monmap: 2024-09-15T01:41:15.184 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:41:15.183519+0000 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:41:15.183519+0000 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:15.185 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:15.186 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:41:15.186 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:41:15.186 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:15.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 1 Processor -- start 2024-09-15T01:41:15.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 1 -- start start 2024-09-15T01:41:15.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 10 monclient: init 2024-09-15T01:41:15.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:15.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 10 monclient: _add_conns ranks=[0,2,3,4,1,6,7,10,9,8,5] 2024-09-15T01:41:15.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f78240a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 10 monclient(hunting): picked mon.noname-b con 0x7f7824132de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 10 monclient(hunting): picked mon.noname-g con 0x7f78241380e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782404bf60 con 0x7f78240a2930 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782409dd80 con 0x7f7824132de0 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78240a2d00 con 0x7f78241380e0 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.183+0000 7f782be80640 10 monclient(hunting): authenticate will time out at 1279.933963s 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782941b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:15.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f7829c1c640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f78240a2930 0x7f782409faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:41396/0 (socket says 172.21.15.178:41396) 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f7829c1c640 1 -- 172.21.15.178:0/1746729782 learned_addr learned my addr 172.21.15.178:0/1746729782 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782941b640 10 monclient(hunting): get_auth_request con 0x7f7824132de0 auth_method 0 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782941b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 msgr2=0x7f7824137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 msgr2=0x7f7824137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782a41d640 10 monclient(hunting): get_auth_request con 0x7f78241380e0 auth_method 0 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782a41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782a41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:15.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782a41d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:15.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 msgr2=0x7f782413a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:41:15.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 msgr2=0x7f782413a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:15.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:15.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.184+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:41:15.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:15.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782941b640 10 monclient(hunting): get_auth_request con 0x7f7824132de0 auth_method 0 2024-09-15T01:41:15.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:15.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782941b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:41:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 msgr2=0x7f7824137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:41:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 msgr2=0x7f7824137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:41:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782a41d640 10 monclient(hunting): get_auth_request con 0x7f78241380e0 auth_method 0 2024-09-15T01:41:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782a41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782a41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782a41d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:41:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 msgr2=0x7f782413a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:41:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 msgr2=0x7f782413a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:15.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:15.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.385+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:41:15.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:15.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782941b640 10 monclient(hunting): get_auth_request con 0x7f7824132de0 auth_method 0 2024-09-15T01:41:15.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:15.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:15.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782941b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:41:15.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 msgr2=0x7f7824137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:41:15.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 msgr2=0x7f7824137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:15.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:15.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:41:15.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:15.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782a41d640 10 monclient(hunting): get_auth_request con 0x7f78241380e0 auth_method 0 2024-09-15T01:41:15.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782a41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:15.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782a41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:15.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782a41d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:41:15.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 msgr2=0x7f782413a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:41:15.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 msgr2=0x7f782413a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:15.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:15.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:15.786+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:41:16.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:16.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782941b640 10 monclient(hunting): get_auth_request con 0x7f7824132de0 auth_method 0 2024-09-15T01:41:16.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:16.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:16.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782941b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:41:16.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 msgr2=0x7f7824137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:41:16.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 msgr2=0x7f7824137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:16.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:16.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:41:16.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:16.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782a41d640 10 monclient(hunting): get_auth_request con 0x7f78241380e0 auth_method 0 2024-09-15T01:41:16.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782a41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:16.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782a41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:16.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782a41d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:41:16.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 msgr2=0x7f782413a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:41:16.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 msgr2=0x7f782413a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:16.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:16.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:16.587+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:41:18.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 10 monclient(hunting): tick 2024-09-15T01:41:18.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 1 monclient(hunting): continuing hunt 2024-09-15T01:41:18.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:41:18.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 msgr2=0x7f782413a4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:41:18.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78241380e0 0x7f782413a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:18.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 msgr2=0x7f7824137ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:41:18.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7824132de0 0x7f7824137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:18.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 1 -- 172.21.15.178:0/1746729782 >> v1:172.21.15.178:6789/0 conn(0x7f78240a2930 legacy=0x7f782409faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:18.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 10 monclient: _add_conns ranks=[1,5,6,7,8,9,10,4] 2024-09-15T01:41:18.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7808003800 addr v1:172.21.15.204:6789/0 2024-09-15T01:41:18.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:18.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 10 monclient(hunting): picked mon.noname-h con 0x7f78080073e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:41:18.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:18.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 10 monclient(hunting): picked mon.noname-d con 0x7f7808009ef0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:41:18.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 1 -- 172.21.15.178:0/1746729782 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78241331b0 con 0x7f7808003800 2024-09-15T01:41:18.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:18.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 1 -- 172.21.15.178:0/1746729782 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f782409dd80 con 0x7f78080073e0 2024-09-15T01:41:18.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:18.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 1 -- 172.21.15.178:0/1746729782 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f78240a2d00 con 0x7f7808009ef0 2024-09-15T01:41:18.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f781b7fe640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:18.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f7829c1c640 1 -- 172.21.15.178:0/1746729782 >> v1:172.21.15.204:6789/0 conn(0x7f7808003800 legacy=0x7f7808005c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:41:18.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:18.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782a41d640 10 monclient(hunting): get_auth_request con 0x7f7808009ef0 auth_method 0 2024-09-15T01:41:18.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782a41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:18.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782a41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:18.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782a41d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:18.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 msgr2=0x7f780800c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:41:18.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 msgr2=0x7f780800c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:18.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:18.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:18.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:41:18.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782941b640 10 monclient(hunting): get_auth_request con 0x7f78080073e0 auth_method 0 2024-09-15T01:41:18.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:18.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:18.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782941b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:18.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 msgr2=0x7f7808009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:41:18.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 msgr2=0x7f7808009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:18.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:18.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.183+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:41:18.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f7829c1c640 1 -- 172.21.15.178:0/1746729782 >> v1:172.21.15.204:6789/0 conn(0x7f7808003800 legacy=0x7f7808005c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:41:18.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:18.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782a41d640 10 monclient(hunting): get_auth_request con 0x7f7808009ef0 auth_method 0 2024-09-15T01:41:18.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782a41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:18.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782a41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:18.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782a41d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:41:18.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:18.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 msgr2=0x7f780800c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:41:18.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 msgr2=0x7f780800c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:18.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:18.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782941b640 10 monclient(hunting): get_auth_request con 0x7f78080073e0 auth_method 0 2024-09-15T01:41:18.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:18.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:18.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782941b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:41:18.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:41:18.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 msgr2=0x7f7808009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:41:18.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 msgr2=0x7f7808009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:18.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:18.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.384+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:41:18.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f7829c1c640 1 -- 172.21.15.178:0/1746729782 >> v1:172.21.15.204:6789/0 conn(0x7f7808003800 legacy=0x7f7808005c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:41:18.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:18.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782a41d640 10 monclient(hunting): get_auth_request con 0x7f7808009ef0 auth_method 0 2024-09-15T01:41:18.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782a41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:18.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782a41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:18.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782a41d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:41:18.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 msgr2=0x7f780800c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:41:18.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 msgr2=0x7f780800c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:18.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:18.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:41:18.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:18.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782941b640 10 monclient(hunting): get_auth_request con 0x7f78080073e0 auth_method 0 2024-09-15T01:41:18.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:18.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:18.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782941b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:41:18.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 msgr2=0x7f7808009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:41:18.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 msgr2=0x7f7808009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:18.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:18.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:18.785+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:41:19.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f7829c1c640 1 -- 172.21.15.178:0/1746729782 >> v1:172.21.15.204:6789/0 conn(0x7f7808003800 legacy=0x7f7808005c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:41:19.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:19.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782a41d640 10 monclient(hunting): get_auth_request con 0x7f7808009ef0 auth_method 0 2024-09-15T01:41:19.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782a41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:19.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782a41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:19.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782a41d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:41:19.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 msgr2=0x7f780800c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:41:19.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 msgr2=0x7f780800c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:19.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:19.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:41:19.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:19.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782941b640 10 monclient(hunting): get_auth_request con 0x7f78080073e0 auth_method 0 2024-09-15T01:41:19.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:19.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:19.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782941b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:41:19.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 msgr2=0x7f7808009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:41:19.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782941b640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 msgr2=0x7f7808009840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:41:19.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:41:19.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:19.586+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:41:21.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 10 monclient(hunting): tick 2024-09-15T01:41:21.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 1 monclient(hunting): continuing hunt 2024-09-15T01:41:21.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:41:21.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 msgr2=0x7f780800c3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:41:21.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808009ef0 0x7f780800c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 msgr2=0x7f7808009840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:41:21.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f78080073e0 0x7f7808009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 1 -- 172.21.15.178:0/1746729782 >> v1:172.21.15.204:6789/0 conn(0x7f7808003800 legacy=0x7f7808005c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:41:21.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 reap_dead start 2024-09-15T01:41:21.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 10 monclient: _add_conns ranks=[10,4,9,8,7] 2024-09-15T01:41:21.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 1 --2- 172.21.15.178:0/1746729782 >> v2:172.21.15.204:3300/0 conn(0x7f780800cb10 0x7f780800c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 10 monclient(hunting): picked mon.noname-f con 0x7f780800cb10 addr v2:172.21.15.204:3300/0 2024-09-15T01:41:21.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f780800cfb0 0x7f78080097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 10 monclient(hunting): picked mon.noname-c con 0x7f780800cfb0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:41:21.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 1 --2- 172.21.15.178:0/1746729782 >> v2:172.21.15.178:3300/0 conn(0x7f780800d510 0x7f78080100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f780800d510 addr v2:172.21.15.178:3300/0 2024-09-15T01:41:21.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 1 -- 172.21.15.178:0/1746729782 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f782409dd80 con 0x7f780800d510 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 1 -- 172.21.15.178:0/1746729782 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78240a2d00 con 0x7f780800cb10 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 1 -- 172.21.15.178:0/1746729782 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78080105e0 con 0x7f780800cfb0 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f781b7fe640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f7829c1c640 1 -- 172.21.15.178:0/1746729782 >> v2:172.21.15.204:3300/0 conn(0x7f780800cb10 msgr2=0x7f780800c380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f7829c1c640 1 --2- 172.21.15.178:0/1746729782 >> v2:172.21.15.204:3300/0 conn(0x7f780800cb10 0x7f780800c380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> v2:172.21.15.178:3300/0 conn(0x7f780800d510 0x7f78080100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f780800cfb0 0x7f78080097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 10 monclient(hunting): get_auth_request con 0x7f780800d510 auth_method 0 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782941b640 10 monclient(hunting): get_auth_request con 0x7f780800cfb0 auth_method 0 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782941b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782941b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782941b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782941b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 10 monclient(hunting): handle_auth_done global_id 34641 payload 274 2024-09-15T01:41:21.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f780800cfb0 msgr2=0x7f78080097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:41:21.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f780800cfb0 0x7f78080097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 >> v2:172.21.15.204:3300/0 conn(0x7f780800cb10 msgr2=0x7f780800c380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:41:21.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> v2:172.21.15.204:3300/0 conn(0x7f780800cb10 0x7f780800c380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 10 monclient: _finish_hunting 0 2024-09-15T01:41:21.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 1 monclient: found mon.noname-a 2024-09-15T01:41:21.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:21.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:41:21.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 1 -- 172.21.15.178:0/1746729782 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780800dc20 con 0x7f780800d510 2024-09-15T01:41:21.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782941b640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f780800cfb0 0x7f78080097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:41:21.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.184+0000 7f782a41d640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f780800d510 0x7f78080100a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f78240a1e30 tx=0x7f781c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e671c4e8d4a9a338 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f7828c1a640 1 -- 172.21.15.178:0/1746729782 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f781c02cd60 con 0x7f780800d510 2024-09-15T01:41:21.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f7828c1a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f7828c1a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:41:21.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f7828c1a640 10 monclient: dump: 2024-09-15T01:41:21.195 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.195 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.195 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.195 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.195 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.195 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.195 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f7828c1a640 10 monclient: _finish_auth 0 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f7828c1a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f7828c1a640 1 -- 172.21.15.178:0/1746729782 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c030070 con 0x7f780800d510 2024-09-15T01:41:21.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 5 monclient: authenticate success, global_id 34641 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f7828c1a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 10 monclient: get_monmap_and_config success 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f7828c1a640 1 -- 172.21.15.178:0/1746729782 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f781c035830 con 0x7f780800d510 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f7828c1a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 4 set_mon_vals no callback set 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f7828c1a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f7828c1a640 10 monclient: dump: 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.197 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.198 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.198 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.198 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.198 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.198 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.198 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.198 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.198 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.198 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.198 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.198 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 10 monclient: shutdown 2024-09-15T01:41:21.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:21.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 1 -- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f780800d510 msgr2=0x7f78080100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f780800d510 0x7f78080100a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f78240a1e30 tx=0x7f781c00b040 comp rx=0 tx=0).stop 2024-09-15T01:41:21.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 1 -- 172.21.15.178:0/1746729782 shutdown_connections 2024-09-15T01:41:21.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f780800d510 0x7f78080100a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 1 --2- 172.21.15.178:0/1746729782 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f780800cfb0 0x7f78080097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 1 --2- 172.21.15.178:0/1746729782 >> v2:172.21.15.204:3300/0 conn(0x7f780800cb10 0x7f780800c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.185+0000 7f782be80640 1 -- 172.21.15.178:0/1746729782 >> 172.21.15.178:0/1746729782 conn(0x7f78240a5ed0 msgr2=0x7f78240a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:21.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.186+0000 7f782be80640 1 -- 172.21.15.178:0/1746729782 shutdown_connections 2024-09-15T01:41:21.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.186+0000 7f782be80640 1 -- 172.21.15.178:0/1746729782 wait complete. 2024-09-15T01:41:21.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.186+0000 7f782be80640 10 monclient: build_initial_monmap 2024-09-15T01:41:21.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.186+0000 7f782be80640 10 monclient: monmap: 2024-09-15T01:41:21.200 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:21.200 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:41:21.200 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:41:21.200 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:41:21.200 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:21.200 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:41:21.200 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:41:21.200 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:41:21.200 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:21.200 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:21.200 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:21.200 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:21.201 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:21.201 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:21.201 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:21.201 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 1 Processor -- start 2024-09-15T01:41:21.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 1 -- start start 2024-09-15T01:41:21.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 10 monclient: init 2024-09-15T01:41:21.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:21.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 10 monclient: _add_conns ranks=[6,8,3,2,1,4,7,0,5] 2024-09-15T01:41:21.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808003800 0x7f7824132470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 10 monclient(hunting): picked mon.noname-d con 0x7f7808003800 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:41:21.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f780800cb10 0x7f782409a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 10 monclient(hunting): picked mon.noname-e con 0x7f780800cb10 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:41:21.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f780800cfb0 0x7f782409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 10 monclient(hunting): picked mon.noname-g con 0x7f780800cfb0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:41:21.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782411dd10 con 0x7f780800cfb0 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f782409d630 con 0x7f7808003800 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f782409f870 con 0x7f780800cb10 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782be80640 10 monclient(hunting): authenticate will time out at 1285.938620s 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782941b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f780800cb10 0x7f782409a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f7829c1c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808003800 0x7f7824132470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f7829c1c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808003800 0x7f7824132470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44204/0 (socket says 172.21.15.178:44204) 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782941b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f780800cb10 0x7f782409a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:57424/0 (socket says 172.21.15.178:57424) 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f7829c1c640 1 -- 172.21.15.178:0/439288197 learned_addr learned my addr 172.21.15.178:0/439288197 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f7829c1c640 10 monclient(hunting): get_auth_request con 0x7f7808003800 auth_method 0 2024-09-15T01:41:21.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f7829c1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f7829c1c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f7829c1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782941b640 10 monclient(hunting): get_auth_request con 0x7f780800cb10 auth_method 0 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782941b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782941b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782941b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782941b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782941b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782941b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f7829c1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f7829c1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f7829c1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.187+0000 7f782a41d640 1 --2- 172.21.15.178:0/439288197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f780800cfb0 0x7f782409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782a41d640 10 monclient(hunting): get_auth_request con 0x7f780800cfb0 auth_method 0 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782a41d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782a41d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782a41d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782941b640 10 monclient(hunting): handle_auth_done global_id 34514 payload 824 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782941b640 1 -- 172.21.15.178:0/439288197 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808003800 msgr2=0x7f7824132470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782941b640 1 --2- 172.21.15.178:0/439288197 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808003800 0x7f7824132470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782941b640 1 -- 172.21.15.178:0/439288197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f780800cfb0 msgr2=0x7f782409a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782941b640 1 --2- 172.21.15.178:0/439288197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f780800cfb0 0x7f782409a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782941b640 10 monclient: _finish_hunting 0 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782941b640 1 monclient: found mon.noname-e 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782941b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782941b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:41:21.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782941b640 1 -- 172.21.15.178:0/439288197 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781400e040 con 0x7f780800cb10 2024-09-15T01:41:21.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7829c1c640 1 --2- 172.21.15.178:0/439288197 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808003800 0x7f7824132470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:41:21.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782941b640 1 --2- 172.21.15.178:0/439288197 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f780800cb10 0x7f782409a480 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7808003760 tx=0x7f7814008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7819ffb640 1 -- 172.21.15.178:0/439288197 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f781401e050 con 0x7f780800cb10 2024-09-15T01:41:21.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7819ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7819ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:41:21.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7819ffb640 10 monclient: dump: 2024-09-15T01:41:21.210 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.210 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.210 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.210 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.210 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7819ffb640 10 monclient: _finish_auth 0 2024-09-15T01:41:21.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7819ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:21.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7819ffb640 1 -- 172.21.15.178:0/439288197 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781401f040 con 0x7f780800cb10 2024-09-15T01:41:21.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782be80640 5 monclient: authenticate success, global_id 34514 2024-09-15T01:41:21.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782be80640 10 monclient: _renew_subs 2024-09-15T01:41:21.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782be80640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:41:21.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782be80640 1 -- 172.21.15.178:0/439288197 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781c0099f0 con 0x7f780800cb10 2024-09-15T01:41:21.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7819ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:21.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7819ffb640 1 -- 172.21.15.178:0/439288197 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7814016c10 con 0x7f780800cb10 2024-09-15T01:41:21.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7819ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7819ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:41:21.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f7819ffb640 10 monclient: dump: 2024-09-15T01:41:21.212 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.213 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.213 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.213 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.213 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.213 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.213 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.213 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.213 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.213 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.213 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.213 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.213 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.214 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.214 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.214 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f781bfff640 4 set_mon_vals no callback set 2024-09-15T01:41:21.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782be80640 10 monclient: _renew_subs 2024-09-15T01:41:21.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782be80640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:41:21.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.188+0000 7f782be80640 1 -- 172.21.15.178:0/439288197 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782409c940 con 0x7f780800cb10 2024-09-15T01:41:21.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.189+0000 7f7819ffb640 1 -- 172.21.15.178:0/439288197 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7814016d70 con 0x7f780800cb10 2024-09-15T01:41:21.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.189+0000 7f7819ffb640 1 --2- 172.21.15.178:0/439288197 >> v2:172.21.15.204:6812/28210 conn(0x7f780803b4d0 0x7f780803d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.189+0000 7f7819ffb640 1 -- 172.21.15.178:0/439288197 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f781404ce50 con 0x7f780800cb10 2024-09-15T01:41:21.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.190+0000 7f7829c1c640 1 --2- 172.21.15.178:0/439288197 >> v2:172.21.15.204:6812/28210 conn(0x7f780803b4d0 0x7f780803d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.190+0000 7f782be80640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:21.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.190+0000 7f782be80640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:21.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.190+0000 7f782be80640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:41:21.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.190+0000 7f782be80640 1 -- 172.21.15.178:0/439288197 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782404b720 con 0x7f780800cb10 2024-09-15T01:41:21.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.190+0000 7f7829c1c640 10 monclient: get_auth_request con 0x7f780803b4d0 auth_method 0 2024-09-15T01:41:21.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.190+0000 7f7829c1c640 1 --2- 172.21.15.178:0/439288197 >> v2:172.21.15.204:6812/28210 conn(0x7f780803b4d0 0x7f780803d980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7820001260 tx=0x7f7820009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.195+0000 7f7819ffb640 1 -- 172.21.15.178:0/439288197 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7814029020 con 0x7f780800cb10 2024-09-15T01:41:21.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.195+0000 7f7819ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:21.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.195+0000 7f7819ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:41:21.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.412+0000 7f782be80640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:41:21.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.412+0000 7f782be80640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:41:21.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.412+0000 7f782be80640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:41:21.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.412+0000 7f782be80640 1 -- 172.21.15.178:0/439288197 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f782411d3a0 con 0x7f780800cb10 2024-09-15T01:41:21.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.412+0000 7f7819ffb640 1 -- 172.21.15.178:0/439288197 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3178 (secure 0 0 0) 0x7f78140233c0 con 0x7f780800cb10 2024-09-15T01:41:21.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.412+0000 7f7819ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:41:21.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.412+0000 7f7819ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:41:21.416 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":114,"quorum":[0,2,3,4,5,6,7,8],"quorum_names":["a","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":1,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:41:21.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 1 -- 172.21.15.178:0/439288197 >> v2:172.21.15.204:6812/28210 conn(0x7f780803b4d0 msgr2=0x7f780803d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 1 --2- 172.21.15.178:0/439288197 >> v2:172.21.15.204:6812/28210 conn(0x7f780803b4d0 0x7f780803d980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7820001260 tx=0x7f7820009040 comp rx=0 tx=0).stop 2024-09-15T01:41:21.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 10 monclient: shutdown 2024-09-15T01:41:21.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:21.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 1 -- 172.21.15.178:0/439288197 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f780800cb10 msgr2=0x7f782409a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 1 --2- 172.21.15.178:0/439288197 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f780800cb10 0x7f782409a480 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7808003760 tx=0x7f7814008040 comp rx=0 tx=0).stop 2024-09-15T01:41:21.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 1 -- 172.21.15.178:0/439288197 shutdown_connections 2024-09-15T01:41:21.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 1 --2- 172.21.15.178:0/439288197 >> v2:172.21.15.204:6812/28210 conn(0x7f780803b4d0 0x7f780803d980 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 1 --2- 172.21.15.178:0/439288197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f780800cfb0 0x7f782409a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 1 --2- 172.21.15.178:0/439288197 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f780800cb10 0x7f782409a480 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 1 --2- 172.21.15.178:0/439288197 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f7808003800 0x7f7824132470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 1 -- 172.21.15.178:0/439288197 >> 172.21.15.178:0/439288197 conn(0x7f78240a5ed0 msgr2=0x7f78240a4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:21.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 1 -- 172.21.15.178:0/439288197 shutdown_connections 2024-09-15T01:41:21.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.414+0000 7f782be80640 1 -- 172.21.15.178:0/439288197 wait complete. 2024-09-15T01:41:21.429 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-09-15T01:41:21.429 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-09-15T01:41:21.429 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:41:21.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.534+0000 7f373a4c0640 10 monclient: get_monmap_and_config 2024-09-15T01:41:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.534+0000 7f373a4c0640 10 monclient: build_initial_monmap 2024-09-15T01:41:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.534+0000 7f373a4c0640 10 monclient: monmap: 2024-09-15T01:41:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:41:21.536344+0000 2024-09-15T01:41:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:41:21.536344+0000 2024-09-15T01:41:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:41:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:41:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:41:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:41:21.541 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 1 Processor -- start 2024-09-15T01:41:21.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 1 -- start start 2024-09-15T01:41:21.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 10 monclient: init 2024-09-15T01:41:21.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:21.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 10 monclient: _add_conns ranks=[0,8,9,4,6,5,1,10,7,3,2] 2024-09-15T01:41:21.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f37340a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:41:21.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3734132de0 0x7f3734137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 10 monclient(hunting): picked mon.noname-e con 0x7f3734132de0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:41:21.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f37341380e0 0x7f373413a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 10 monclient(hunting): picked mon.noname-a con 0x7f37341380e0 addr v2:172.21.15.178:3300/0 2024-09-15T01:41:21.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373404bf60 con 0x7f37340a2930 2024-09-15T01:41:21.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373409dd80 con 0x7f37341380e0 2024-09-15T01:41:21.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f37340a2d00 con 0x7f3734132de0 2024-09-15T01:41:21.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:21.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.535+0000 7f373a4c0640 10 monclient(hunting): authenticate will time out at 1286.286893s 2024-09-15T01:41:21.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3734132de0 0x7f3734137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3734132de0 0x7f3734137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:57428/0 (socket says 172.21.15.178:57428) 2024-09-15T01:41:21.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f3738a5d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f37341380e0 0x7f373413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 1 -- 172.21.15.178:0/562142492 learned_addr learned my addr 172.21.15.178:0/562142492 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:21.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 10 monclient(hunting): get_auth_request con 0x7f3734132de0 auth_method 0 2024-09-15T01:41:21.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f3733fff640 1 --1- 172.21.15.178:0/562142492 >> v1:172.21.15.178:6789/0 conn(0x7f37340a2930 0x7f373409faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:41264/0 (socket says 172.21.15.178:41264) 2024-09-15T01:41:21.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f3738a5d640 10 monclient(hunting): get_auth_request con 0x7f37341380e0 auth_method 0 2024-09-15T01:41:21.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f3738a5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f3738a5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f3738a5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f3738a5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f3738a5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f3738a5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 10 monclient(hunting): handle_auth_done global_id 34523 payload 274 2024-09-15T01:41:21.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 1 -- 172.21.15.178:0/562142492 >> v2:172.21.15.178:3300/0 conn(0x7f37341380e0 msgr2=0x7f373413a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:41:21.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 1 --2- 172.21.15.178:0/562142492 >> v2:172.21.15.178:3300/0 conn(0x7f37341380e0 0x7f373413a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 1 -- 172.21.15.178:0/562142492 >> v1:172.21.15.178:6789/0 conn(0x7f37340a2930 legacy=0x7f373409faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 10 monclient: _finish_hunting 0 2024-09-15T01:41:21.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 1 monclient: found mon.noname-e 2024-09-15T01:41:21.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:21.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:41:21.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f3732ffd640 1 -- 172.21.15.178:0/562142492 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112285721 0 0) 0x7f373404bf60 con 0x7f37340a2930 2024-09-15T01:41:21.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f37337fe640 1 -- 172.21.15.178:0/562142492 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37341331b0 con 0x7f3734132de0 2024-09-15T01:41:21.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f3732ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:41:21.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.536+0000 7f3738a5d640 1 --2- 172.21.15.178:0/562142492 >> v2:172.21.15.178:3300/0 conn(0x7f37341380e0 0x7f373413a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:41:21.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f37337fe640 1 --2- 172.21.15.178:0/562142492 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3734132de0 0x7f3734137ba0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3728009910 tx=0x7f37280091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=542475f57ae8165f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f3732ffd640 1 -- 172.21.15.178:0/562142492 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f372802ad20 con 0x7f3734132de0 2024-09-15T01:41:21.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f3732ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f3732ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:41:21.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f3732ffd640 10 monclient: dump: 2024-09-15T01:41:21.571 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.571 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.571 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.571 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.571 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.572 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.572 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.572 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.572 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.572 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.572 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.572 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.572 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.573 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.573 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.573 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f3732ffd640 10 monclient: _finish_auth 0 2024-09-15T01:41:21.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f3732ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:21.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f373a4c0640 5 monclient: authenticate success, global_id 34523 2024-09-15T01:41:21.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f3732ffd640 1 -- 172.21.15.178:0/562142492 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3728031820 con 0x7f3734132de0 2024-09-15T01:41:21.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f373a4c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:41:21.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f3732ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:21.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f373a4c0640 10 monclient: get_monmap_and_config success 2024-09-15T01:41:21.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f373a4c0640 4 set_mon_vals no callback set 2024-09-15T01:41:21.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f3732ffd640 1 -- 172.21.15.178:0/562142492 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f372802aea0 con 0x7f3734132de0 2024-09-15T01:41:21.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f3732ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f3732ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:41:21.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.537+0000 7f3732ffd640 10 monclient: dump: 2024-09-15T01:41:21.575 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.575 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.575 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.575 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.575 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.575 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.575 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.575 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.576 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.576 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.576 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.576 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.576 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.576 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.576 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.577 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.538+0000 7f373a4c0640 10 monclient: shutdown 2024-09-15T01:41:21.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.538+0000 7f373a4c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:21.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.538+0000 7f373a4c0640 1 -- 172.21.15.178:0/562142492 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3734132de0 msgr2=0x7f3734137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.538+0000 7f373a4c0640 1 --2- 172.21.15.178:0/562142492 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3734132de0 0x7f3734137ba0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3728009910 tx=0x7f37280091d0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.538+0000 7f373a4c0640 1 -- 172.21.15.178:0/562142492 shutdown_connections 2024-09-15T01:41:21.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.538+0000 7f373a4c0640 1 --2- 172.21.15.178:0/562142492 >> v2:172.21.15.178:3300/0 conn(0x7f37341380e0 0x7f373413a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.538+0000 7f373a4c0640 1 --2- 172.21.15.178:0/562142492 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f3734132de0 0x7f3734137ba0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.538+0000 7f373a4c0640 1 -- 172.21.15.178:0/562142492 >> 172.21.15.178:0/562142492 conn(0x7f37340a5ed0 msgr2=0x7f37340a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:21.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.538+0000 7f373a4c0640 1 -- 172.21.15.178:0/562142492 shutdown_connections 2024-09-15T01:41:21.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.538+0000 7f373a4c0640 1 -- 172.21.15.178:0/562142492 wait complete. 2024-09-15T01:41:21.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.538+0000 7f373a4c0640 10 monclient: build_initial_monmap 2024-09-15T01:41:21.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.538+0000 7f373a4c0640 10 monclient: monmap: 2024-09-15T01:41:21.578 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:21.579 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:41:21.579 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:41:21.579 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:41:21.579 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:21.579 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:41:21.579 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:41:21.579 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:41:21.579 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:21.580 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:21.580 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:21.580 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:21.580 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:21.580 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:21.580 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:21.580 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 1 Processor -- start 2024-09-15T01:41:21.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 1 -- start start 2024-09-15T01:41:21.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 10 monclient: init 2024-09-15T01:41:21.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:21.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 10 monclient: _add_conns ranks=[3,6,2,0,4,1,5,7,8] 2024-09-15T01:41:21.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37340a2930 0x7f373409a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 10 monclient(hunting): picked mon.noname-g con 0x7f37340a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:41:21.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3734132de0 0x7f373409ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 10 monclient(hunting): picked mon.noname-d con 0x7f3734132de0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:41:21.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37341380e0 0x7f373409b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 10 monclient(hunting): picked mon.noname-b con 0x7f37341380e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:41:21.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373411dbe0 con 0x7f37341380e0 2024-09-15T01:41:21.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373409d630 con 0x7f37340a2930 2024-09-15T01:41:21.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f373409f870 con 0x7f3734132de0 2024-09-15T01:41:21.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:21.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f373a4c0640 10 monclient(hunting): authenticate will time out at 1286.290824s 2024-09-15T01:41:21.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f37337fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3734132de0 0x7f373409ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f37337fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3734132de0 0x7f373409ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44218/0 (socket says 172.21.15.178:44218) 2024-09-15T01:41:21.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f37337fe640 1 -- 172.21.15.178:0/4266472324 learned_addr learned my addr 172.21.15.178:0/4266472324 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:21.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f3738a5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37341380e0 0x7f373409b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f37337fe640 10 monclient(hunting): get_auth_request con 0x7f3734132de0 auth_method 0 2024-09-15T01:41:21.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f37337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f37337fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f37337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f3738a5d640 10 monclient(hunting): get_auth_request con 0x7f37341380e0 auth_method 0 2024-09-15T01:41:21.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f3738a5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f3738a5d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.539+0000 7f3738a5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f3738a5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f3738a5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f3738a5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f3733fff640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37340a2930 0x7f373409a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f3733fff640 10 monclient(hunting): get_auth_request con 0x7f37340a2930 auth_method 0 2024-09-15T01:41:21.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f3733fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f3733fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f3733fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f3733fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f3733fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f3733fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 10 monclient(hunting): handle_auth_done global_id 25089 payload 824 2024-09-15T01:41:21.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 1 -- 172.21.15.178:0/4266472324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37340a2930 msgr2=0x7f373409a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37340a2930 0x7f373409a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 1 -- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37341380e0 msgr2=0x7f373409b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37341380e0 0x7f373409b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 10 monclient: _finish_hunting 0 2024-09-15T01:41:21.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 1 monclient: found mon.noname-d 2024-09-15T01:41:21.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:21.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:41:21.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 1 -- 172.21.15.178:0/4266472324 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372c009a20 con 0x7f3734132de0 2024-09-15T01:41:21.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f3738a5d640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37341380e0 0x7f373409b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:41:21.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37337fe640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3734132de0 0x7f373409ab60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f37280097e0 tx=0x7f3728032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3728033d30 con 0x7f3734132de0 2024-09-15T01:41:21.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f371bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f371bfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:41:21.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f371bfff640 10 monclient: dump: 2024-09-15T01:41:21.590 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.590 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.590 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.590 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.590 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.590 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.590 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.590 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.590 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.591 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f371bfff640 10 monclient: _finish_auth 0 2024-09-15T01:41:21.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f371bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:21.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372803a390 con 0x7f3734132de0 2024-09-15T01:41:21.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f373a4c0640 5 monclient: authenticate success, global_id 25089 2024-09-15T01:41:21.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f373a4c0640 10 monclient: _renew_subs 2024-09-15T01:41:21.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f373a4c0640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:41:21.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f373a4c0640 1 -- 172.21.15.178:0/4266472324 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3728009450 con 0x7f3734132de0 2024-09-15T01:41:21.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f371bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:21.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f372803a980 con 0x7f3734132de0 2024-09-15T01:41:21.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f371bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f371bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:41:21.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f371bfff640 10 monclient: dump: 2024-09-15T01:41:21.593 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.593 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.593 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.593 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.594 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.594 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.594 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.594 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.594 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.594 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.594 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.594 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.594 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.594 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f373a4c0640 10 monclient: _renew_subs 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f37327fc640 4 set_mon_vals no callback set 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f373a4c0640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.540+0000 7f373a4c0640 1 -- 172.21.15.178:0/4266472324 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373411d5b0 con 0x7f3734132de0 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f373a4c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f373a4c0640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f373411c6c0 0x7f373409b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f373a4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3728039040 con 0x7f3734132de0 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f373a4c0640 1 -- 172.21.15.178:0/4266472324 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373404c4a0 con 0x7f373411c6c0 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f3733fff640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f373411c6c0 0x7f373409b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f3733fff640 10 monclient: get_auth_request con 0x7f373411c6c0 auth_method 0 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f3733fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f3733fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f3733fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f3733fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f3733fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f3733fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f371bfff640 1 --2- 172.21.15.178:0/4266472324 >> v2:172.21.15.204:6812/28210 conn(0x7f37200397b0 0x7f372003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f372806b750 con 0x7f3734132de0 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f3733fff640 10 monclient(hunting): handle_auth_done global_id 34668 payload 824 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.541+0000 7f3733fff640 1 -- 172.21.15.178:0/4266472324 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f373405ed70 con 0x7f373411c6c0 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f3733fff640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f373411c6c0 0x7f373409b5e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3724007800 tx=0x7f3724007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f3738a5d640 1 --2- 172.21.15.178:0/4266472324 >> v2:172.21.15.204:6812/28210 conn(0x7f37200397b0 0x7f372003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f3738a5d640 10 monclient: get_auth_request con 0x7f37200397b0 auth_method 0 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3724011a10 con 0x7f373411c6c0 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f371bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f371bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f371bfff640 10 monclient: dump: 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.597 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f3724011590 con 0x7f373411c6c0 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f371bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f371bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f373411c6c0 msgr2=0x7f373409b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f371bfff640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f373411c6c0 0x7f373409b5e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3724007800 tx=0x7f3724007e40 comp rx=0 tx=0).stop 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f373411c6c0 msgr2=0x7f373409b5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:21.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f371bfff640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f373411c6c0 0x7f373409b5e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.542+0000 7f3738a5d640 1 --2- 172.21.15.178:0/4266472324 >> v2:172.21.15.204:6812/28210 conn(0x7f37200397b0 0x7f372003bc60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f373409c030 tx=0x7f372c00b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.548+0000 7f373a4c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.548+0000 7f373a4c0640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734096a30 0x7f3734096e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.548+0000 7f373a4c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.548+0000 7f373a4c0640 1 -- 172.21.15.178:0/4266472324 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373404bc80 con 0x7f3734096a30 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.548+0000 7f3733fff640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734096a30 0x7f3734096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.548+0000 7f3733fff640 10 monclient: get_auth_request con 0x7f3734096a30 auth_method 0 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.548+0000 7f3733fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.548+0000 7f3733fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.548+0000 7f3733fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.548+0000 7f3733fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.548+0000 7f3733fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.548+0000 7f3733fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f3733fff640 10 monclient(hunting): handle_auth_done global_id 34677 payload 824 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f3733fff640 1 -- 172.21.15.178:0/4266472324 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f373411d8e0 con 0x7f3734096a30 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f3733fff640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734096a30 0x7f3734096e30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3724000fb0 tx=0x7f3724015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f37240208f0 con 0x7f3734096a30 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f371bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f371bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f371bfff640 10 monclient: dump: 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.600 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9301 (secure 0 0 0) 0x7f3724020470 con 0x7f3734096a30 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f371bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f371bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734096a30 msgr2=0x7f3734096e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f371bfff640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734096a30 0x7f3734096e30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3724000fb0 tx=0x7f3724015000 comp rx=0 tx=0).stop 2024-09-15T01:41:21.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f371bfff640 1 -- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734096a30 msgr2=0x7f3734096e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:21.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f371bfff640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3734096a30 0x7f3734096e30 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3724000fb0 tx=0x7f3724015000 comp rx=0 tx=0).stop 2024-09-15T01:41:21.602 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:41:21.602 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:41:21.602 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:41:21.602 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:41:21.602 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 114, 2024-09-15T01:41:21.602 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:41:21.603 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:41:21.604 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:41:21.605 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:21.606 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.607 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:41:21.609 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.612 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.614 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:41:21.615 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:41:21.615 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.615 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:41:21.615 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.615 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:41:21.615 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.616 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.616 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.616 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:41:21.616 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:41:21.616 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.616 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.616 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:41:21.617 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:41:21.618 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.620 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:41:21.621 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.623 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:21.624 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:41:21.625 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:41:21.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f373a4c0640 1 -- 172.21.15.178:0/4266472324 >> v2:172.21.15.204:6812/28210 conn(0x7f37200397b0 msgr2=0x7f372003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f373a4c0640 1 --2- 172.21.15.178:0/4266472324 >> v2:172.21.15.204:6812/28210 conn(0x7f37200397b0 0x7f372003bc60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f373409c030 tx=0x7f372c00b040 comp rx=0 tx=0).stop 2024-09-15T01:41:21.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.549+0000 7f373a4c0640 10 monclient: shutdown 2024-09-15T01:41:21.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.550+0000 7f373a4c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:21.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.550+0000 7f373a4c0640 1 -- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3734132de0 msgr2=0x7f373409ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.550+0000 7f373a4c0640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3734132de0 0x7f373409ab60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f37280097e0 tx=0x7f3728032000 comp rx=0 tx=0).stop 2024-09-15T01:41:21.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.550+0000 7f3738a5d640 1 -- 172.21.15.178:0/4266472324 reap_dead start 2024-09-15T01:41:21.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.550+0000 7f373a4c0640 1 -- 172.21.15.178:0/4266472324 shutdown_connections 2024-09-15T01:41:21.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.550+0000 7f373a4c0640 1 --2- 172.21.15.178:0/4266472324 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f3734132de0 0x7f373409ab60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.550+0000 7f373a4c0640 1 -- 172.21.15.178:0/4266472324 >> 172.21.15.178:0/4266472324 conn(0x7f37340a5ed0 msgr2=0x7f37341358d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:21.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.550+0000 7f373a4c0640 1 -- 172.21.15.178:0/4266472324 shutdown_connections 2024-09-15T01:41:21.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.550+0000 7f373a4c0640 1 -- 172.21.15.178:0/4266472324 wait complete. 2024-09-15T01:41:21.628 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T01:41:21.628 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-09-15T01:41:21.628 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:41:21.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.720+0000 7fde50834640 10 monclient: get_monmap_and_config 2024-09-15T01:41:21.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.720+0000 7fde50834640 10 monclient: build_initial_monmap 2024-09-15T01:41:21.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.720+0000 7fde50834640 10 monclient: monmap: 2024-09-15T01:41:21.723 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:21.723 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.723 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:41:21.721911+0000 2024-09-15T01:41:21.723 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:41:21.721911+0000 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:41:21.724 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:41:21.725 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.720+0000 7fde50834640 1 Processor -- start 2024-09-15T01:41:21.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.720+0000 7fde50834640 1 -- start start 2024-09-15T01:41:21.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.720+0000 7fde50834640 10 monclient: init 2024-09-15T01:41:21.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:21.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 10 monclient: _add_conns ranks=[2,10,8,3,1,0,9,7,5,4,6] 2024-09-15T01:41:21.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c0a2930 0x7fde4c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 10 monclient(hunting): picked mon.noname-b con 0x7fde4c0a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:41:21.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fde4c09faa0 0x7fde4c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 10 monclient(hunting): picked mon.noname-f con 0x7fde4c09faa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:41:21.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fde4c0a08c0 0x7fde4c0a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 10 monclient(hunting): picked mon.noname-e con 0x7fde4c0a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:41:21.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4c09e0f0 con 0x7fde4c09faa0 2024-09-15T01:41:21.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c09dd80 con 0x7fde4c0a2930 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0a2d00 con 0x7fde4c0a08c0 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde50834640 10 monclient(hunting): authenticate will time out at 1286.472147s 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4a59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c0a2930 0x7fde4c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fde4c0a08c0 0x7fde4c0a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde41d9b640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7fde4c09faa0 msgr2=0x7fde4c09fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4a59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c0a2930 0x7fde4c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:58232/0 (socket says 172.21.15.178:58232) 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde41d9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fde4c09faa0 0x7fde4c09fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4a59c640 1 -- 172.21.15.178:0/3876179651 learned_addr learned my addr 172.21.15.178:0/3876179651 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 10 monclient(hunting): get_auth_request con 0x7fde4c0a08c0 auth_method 0 2024-09-15T01:41:21.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4a59c640 10 monclient(hunting): get_auth_request con 0x7fde4c0a2930 auth_method 0 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 10 monclient(hunting): handle_auth_done global_id 34532 payload 274 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 1 -- 172.21.15.178:0/3876179651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c0a2930 msgr2=0x7fde4c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 1 --2- 172.21.15.178:0/3876179651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c0a2930 0x7fde4c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 1 -- 172.21.15.178:0/3876179651 >> v2:172.21.15.204:3300/0 conn(0x7fde4c09faa0 msgr2=0x7fde4c09fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 1 --2- 172.21.15.178:0/3876179651 >> v2:172.21.15.204:3300/0 conn(0x7fde4c09faa0 0x7fde4c09fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 1 monclient: found mon.noname-e 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4ad9d640 1 -- 172.21.15.178:0/3876179651 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c04bf60 con 0x7fde4c0a08c0 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.721+0000 7fde4a59c640 1 --2- 172.21.15.178:0/3876179651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c0a2930 0x7fde4c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde4ad9d640 1 --2- 172.21.15.178:0/3876179651 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fde4c0a08c0 0x7fde4c0a0d60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fde44009e60 tx=0x7fde44009780 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e2370c768f4df7ee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde49d9b640 1 -- 172.21.15.178:0/3876179651 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde440079d0 con 0x7fde4c0a08c0 2024-09-15T01:41:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde49d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde49d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:41:21.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde49d9b640 10 monclient: dump: 2024-09-15T01:41:21.738 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.738 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.738 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.738 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.738 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.738 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.738 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.738 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.738 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.738 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde49d9b640 10 monclient: _finish_auth 0 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde49d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde50834640 5 monclient: authenticate success, global_id 34532 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde50834640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde49d9b640 1 -- 172.21.15.178:0/3876179651 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4402d050 con 0x7fde4c0a08c0 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde49d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:21.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde50834640 10 monclient: get_monmap_and_config success 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde49d9b640 1 -- 172.21.15.178:0/3876179651 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde44007bd0 con 0x7fde4c0a08c0 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde50834640 4 set_mon_vals no callback set 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde49d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde49d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde49d9b640 10 monclient: dump: 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.740 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde50834640 10 monclient: shutdown 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde50834640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde50834640 1 -- 172.21.15.178:0/3876179651 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fde4c0a08c0 msgr2=0x7fde4c0a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.722+0000 7fde50834640 1 --2- 172.21.15.178:0/3876179651 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fde4c0a08c0 0x7fde4c0a0d60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fde44009e60 tx=0x7fde44009780 comp rx=0 tx=0).stop 2024-09-15T01:41:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.723+0000 7fde50834640 1 -- 172.21.15.178:0/3876179651 shutdown_connections 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.723+0000 7fde50834640 1 --2- 172.21.15.178:0/3876179651 >> v2:172.21.15.204:3300/0 conn(0x7fde4c09faa0 0x7fde4c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.723+0000 7fde50834640 1 --2- 172.21.15.178:0/3876179651 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fde4c0a08c0 0x7fde4c0a0d60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.723+0000 7fde50834640 1 --2- 172.21.15.178:0/3876179651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c0a2930 0x7fde4c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.723+0000 7fde50834640 1 -- 172.21.15.178:0/3876179651 >> 172.21.15.178:0/3876179651 conn(0x7fde4c0a5ed0 msgr2=0x7fde4c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.723+0000 7fde50834640 1 -- 172.21.15.178:0/3876179651 shutdown_connections 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.723+0000 7fde50834640 1 -- 172.21.15.178:0/3876179651 wait complete. 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.723+0000 7fde50834640 10 monclient: build_initial_monmap 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.723+0000 7fde50834640 10 monclient: monmap: 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:41:21.742 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.723+0000 7fde50834640 1 Processor -- start 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 1 -- start start 2024-09-15T01:41:21.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 10 monclient: init 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 10 monclient: _add_conns ranks=[2,0,8,1,7,3,6,4,5] 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c09faa0 0x7fde4c09b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 10 monclient(hunting): picked mon.noname-b con 0x7fde4c09faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde4c0a08c0 0x7fde4c131a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 10 monclient(hunting): picked mon.noname-a con 0x7fde4c0a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fde4c0a2930 0x7fde4c141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 10 monclient(hunting): picked mon.noname-e con 0x7fde4c0a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c09d630 con 0x7fde4c0a08c0 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c11d790 con 0x7fde4c09faa0 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0a2390 con 0x7fde4c0a2930 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde50834640 10 monclient(hunting): authenticate will time out at 1286.475472s 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c09faa0 0x7fde4c09b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4ad9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fde4c0a2930 0x7fde4c141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde41d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde4c0a08c0 0x7fde4c131a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c09faa0 0x7fde4c09b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:58238/0 (socket says 172.21.15.178:58238) 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 1 -- 172.21.15.178:0/122976003 learned_addr learned my addr 172.21.15.178:0/122976003 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 10 monclient(hunting): get_auth_request con 0x7fde4c09faa0 auth_method 0 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4ad9d640 10 monclient(hunting): get_auth_request con 0x7fde4c0a2930 auth_method 0 2024-09-15T01:41:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde41d9b640 10 monclient(hunting): get_auth_request con 0x7fde4c0a08c0 auth_method 0 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde41d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde41d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde41d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde41d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde41d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde41d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 10 monclient(hunting): handle_auth_done global_id 34526 payload 824 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 1 -- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fde4c0a2930 msgr2=0x7fde4c141dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fde4c0a2930 0x7fde4c141dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 1 -- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde4c0a08c0 msgr2=0x7fde4c131a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde4c0a08c0 0x7fde4c131a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 10 monclient: _finish_hunting 0 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 1 monclient: found mon.noname-b 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4a59c640 1 -- 172.21.15.178:0/122976003 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c009450 con 0x7fde4c09faa0 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.724+0000 7fde4ad9d640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fde4c0a2930 0x7fde4c141dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:41:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde4a59c640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c09faa0 0x7fde4c09b380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00ecb0 tx=0x7fde3c0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde3c009730 con 0x7fde4c09faa0 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde42ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde42ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde42ffd640 10 monclient: dump: 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.748 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde42ffd640 10 monclient: _finish_auth 0 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde50834640 5 monclient: authenticate success, global_id 34526 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c013470 con 0x7fde4c09faa0 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde50834640 10 monclient: _renew_subs 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde50834640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:41:21.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde50834640 1 -- 172.21.15.178:0/122976003 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde44009a20 con 0x7fde4c09faa0 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde42ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde3c013a60 con 0x7fde4c09faa0 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde42ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde42ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde42ffd640 10 monclient: dump: 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.750 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde4959a640 4 set_mon_vals no callback set 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde50834640 10 monclient: _renew_subs 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde50834640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.725+0000 7fde50834640 1 -- 172.21.15.178:0/122976003 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c11eba0 con 0x7fde4c09faa0 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde50834640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:21.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde50834640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c11ed90 0x7fde4c142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde50834640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde50834640 1 -- 172.21.15.178:0/122976003 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c11f2f0 con 0x7fde4c11ed90 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fde3c013be0 con 0x7fde4c09faa0 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde41d9b640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c11ed90 0x7fde4c142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde41d9b640 10 monclient: get_auth_request con 0x7fde4c11ed90 auth_method 0 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde41d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde41d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde41d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde41d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde41d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde41d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 1 --2- 172.21.15.178:0/122976003 >> v2:172.21.15.204:6812/28210 conn(0x7fde2c0397b0 0x7fde2c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fde3c04bfd0 con 0x7fde4c09faa0 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde41d9b640 10 monclient(hunting): handle_auth_done global_id 34535 payload 824 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde41d9b640 1 -- 172.21.15.178:0/122976003 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde4c04c4a0 con 0x7fde4c11ed90 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde41d9b640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c11ed90 0x7fde4c142300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde4c09d9a0 tx=0x7fde340090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde3400eb20 con 0x7fde4c11ed90 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde4ad9d640 1 --2- 172.21.15.178:0/122976003 >> v2:172.21.15.204:6812/28210 conn(0x7fde2c0397b0 0x7fde2c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 10 monclient: dump: 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.753 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.754 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.754 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.754 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.754 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.754 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.754 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.754 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.754 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.754 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.754 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde4ad9d640 10 monclient: get_auth_request con 0x7fde2c0397b0 auth_method 0 2024-09-15T01:41:21.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fde3400ec80 con 0x7fde4c11ed90 2024-09-15T01:41:21.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:21.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:41:21.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c11ed90 msgr2=0x7fde4c142300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c11ed90 0x7fde4c142300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde4c09d9a0 tx=0x7fde340090f0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c11ed90 msgr2=0x7fde4c142300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:21.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.726+0000 7fde42ffd640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c11ed90 0x7fde4c142300 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde4c09d9a0 tx=0x7fde340090f0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.727+0000 7fde4ad9d640 1 --2- 172.21.15.178:0/122976003 >> v2:172.21.15.204:6812/28210 conn(0x7fde2c0397b0 0x7fde2c03bc60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fde44034000 tx=0x7fde44037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde50834640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:41:21.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde50834640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c131fc0 0x7fde4c142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde50834640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde50834640 1 -- 172.21.15.178:0/122976003 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0a00a0 con 0x7fde4c131fc0 2024-09-15T01:41:21.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde41d9b640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c131fc0 0x7fde4c142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde41d9b640 10 monclient: get_auth_request con 0x7fde4c131fc0 auth_method 0 2024-09-15T01:41:21.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde41d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde41d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde41d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde41d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde41d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde41d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde41d9b640 10 monclient(hunting): handle_auth_done global_id 34544 payload 824 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.732+0000 7fde41d9b640 1 -- 172.21.15.178:0/122976003 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fde4c04bc80 con 0x7fde4c131fc0 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde41d9b640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c131fc0 0x7fde4c142840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fde34017000 tx=0x7fde34013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde34025070 con 0x7fde4c131fc0 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde42ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde42ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde42ffd640 10 monclient: dump: 2024-09-15T01:41:21.758 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.759 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.760 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9131 (secure 0 0 0) 0x7fde3401b450 con 0x7fde4c131fc0 2024-09-15T01:41:21.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde42ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:41:21.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde42ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:41:21.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c131fc0 msgr2=0x7fde4c142840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde42ffd640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c131fc0 0x7fde4c142840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fde34017000 tx=0x7fde34013000 comp rx=0 tx=0).stop 2024-09-15T01:41:21.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde42ffd640 1 -- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c131fc0 msgr2=0x7fde4c142840 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:21.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde42ffd640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c131fc0 0x7fde4c142840 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fde34017000 tx=0x7fde34013000 comp rx=0 tx=0).stop 2024-09-15T01:41:21.760 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 114, 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:41:21.761 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:21.762 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:21.763 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.764 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:41:21.764 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:41:21.764 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:41:21.764 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:41:21.764 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:41:21.764 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:41:21.764 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:21.764 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:21.764 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:41:21.764 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:41:21.764 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:21.765 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.767 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.769 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.770 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.771 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:41:21.773 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.774 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:41:21.776 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.779 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.780 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:41:21.781 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.782 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.783 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:41:21.783 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:41:21.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde50834640 1 -- 172.21.15.178:0/122976003 >> v2:172.21.15.204:6812/28210 conn(0x7fde2c0397b0 msgr2=0x7fde2c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde50834640 1 --2- 172.21.15.178:0/122976003 >> v2:172.21.15.204:6812/28210 conn(0x7fde2c0397b0 0x7fde2c03bc60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fde44034000 tx=0x7fde44037000 comp rx=0 tx=0).stop 2024-09-15T01:41:21.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde50834640 10 monclient: shutdown 2024-09-15T01:41:21.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde50834640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:21.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde50834640 1 -- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c09faa0 msgr2=0x7fde4c09b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde50834640 1 --2- 172.21.15.178:0/122976003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde4c09faa0 0x7fde4c09b380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00ecb0 tx=0x7fde3c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde4ad9d640 1 -- 172.21.15.178:0/122976003 reap_dead start 2024-09-15T01:41:21.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde4ad9d640 1 -- 172.21.15.178:0/122976003 reap_dead start 2024-09-15T01:41:21.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde50834640 1 -- 172.21.15.178:0/122976003 shutdown_connections 2024-09-15T01:41:21.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde50834640 1 -- 172.21.15.178:0/122976003 >> 172.21.15.178:0/122976003 conn(0x7fde4c0a5ed0 msgr2=0x7fde4c135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:21.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde50834640 1 -- 172.21.15.178:0/122976003 shutdown_connections 2024-09-15T01:41:21.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.733+0000 7fde50834640 1 -- 172.21.15.178:0/122976003 wait complete. 2024-09-15T01:41:21.785 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T01:41:21.785 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-09-15T01:41:21.785 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:41:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.876+0000 7f667151d640 10 monclient: get_monmap_and_config 2024-09-15T01:41:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.876+0000 7f667151d640 10 monclient: build_initial_monmap 2024-09-15T01:41:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.876+0000 7f667151d640 10 monclient: monmap: 2024-09-15T01:41:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:41:21.877987+0000 2024-09-15T01:41:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:41:21.877987+0000 2024-09-15T01:41:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:41:21.880 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.876+0000 7f667151d640 1 Processor -- start 2024-09-15T01:41:21.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.876+0000 7f667151d640 1 -- start start 2024-09-15T01:41:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.876+0000 7f667151d640 10 monclient: init 2024-09-15T01:41:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 10 monclient: _add_conns ranks=[9,2,0,1,8,6,3,5,10,4,7] 2024-09-15T01:41:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f666c0678d0 0x7f666c09e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 10 monclient(hunting): picked mon.noname-a con 0x7f666c0678d0 addr v2:172.21.15.178:3300/0 2024-09-15T01:41:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f666c06ea30 0x7f666c09ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 10 monclient(hunting): picked mon.noname-b con 0x7f666c06ea30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:41:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f666c09f180 addr v1:172.21.15.178:6789/0 2024-09-15T01:41:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c04bf60 con 0x7f666c09f180 2024-09-15T01:41:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666c09f5a0 con 0x7f666c0678d0 2024-09-15T01:41:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666c06d5b0 con 0x7f666c06ea30 2024-09-15T01:41:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f667151d640 10 monclient(hunting): authenticate will time out at 1286.628380s 2024-09-15T01:41:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666a7fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f666c06ea30 0x7f666c09ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f666c0678d0 0x7f666c09e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666b7fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f666c09f180 0x7f666c1306b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:41280/0 (socket says 172.21.15.178:41280) 2024-09-15T01:41:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f666c0678d0 0x7f666c09e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33480/0 (socket says 172.21.15.178:33480) 2024-09-15T01:41:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666a7fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f666c06ea30 0x7f666c09ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:58268/0 (socket says 172.21.15.178:58268) 2024-09-15T01:41:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666b7fe640 1 -- 172.21.15.178:0/2363509224 learned_addr learned my addr 172.21.15.178:0/2363509224 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:41:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 10 monclient(hunting): get_auth_request con 0x7f666c0678d0 auth_method 0 2024-09-15T01:41:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666a7fc640 10 monclient(hunting): get_auth_request con 0x7f666c06ea30 auth_method 0 2024-09-15T01:41:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 10 monclient(hunting): handle_auth_done global_id 34695 payload 274 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 1 -- 172.21.15.178:0/2363509224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f666c06ea30 msgr2=0x7f666c09ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 1 --2- 172.21.15.178:0/2363509224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f666c06ea30 0x7f666c09ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 1 -- 172.21.15.178:0/2363509224 >> v1:172.21.15.178:6789/0 conn(0x7f666c09f180 legacy=0x7f666c1306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 10 monclient: _finish_hunting 0 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 1 monclient: found mon.noname-a 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f6669ffb640 1 -- 172.21.15.178:0/2363509224 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927800119 0 0) 0x7f666c04bf60 con 0x7f666c09f180 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666affd640 1 -- 172.21.15.178:0/2363509224 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c067ca0 con 0x7f666c0678d0 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f6669ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.877+0000 7f666a7fc640 1 --2- 172.21.15.178:0/2363509224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f666c06ea30 0x7f666c09ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f666affd640 1 --2- 172.21.15.178:0/2363509224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f666c0678d0 0x7f666c09e700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f665c009850 tx=0x7f665c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26f1a594e9ce320 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f6669ffb640 1 -- 172.21.15.178:0/2363509224 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f665c02ad00 con 0x7f666c0678d0 2024-09-15T01:41:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f6669ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f6669ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f6669ffb640 10 monclient: dump: 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.894 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f6669ffb640 10 monclient: _finish_auth 0 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f6669ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f6669ffb640 1 -- 172.21.15.178:0/2363509224 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c02ae80 con 0x7f666c0678d0 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f667151d640 5 monclient: authenticate success, global_id 34695 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f667151d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f6669ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f667151d640 10 monclient: get_monmap_and_config success 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f6669ffb640 1 -- 172.21.15.178:0/2363509224 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f665c031ca0 con 0x7f666c0678d0 2024-09-15T01:41:21.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f667151d640 4 set_mon_vals no callback set 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f6669ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f6669ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.878+0000 7f6669ffb640 10 monclient: dump: 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.879+0000 7f667151d640 10 monclient: shutdown 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.879+0000 7f667151d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.879+0000 7f667151d640 1 -- 172.21.15.178:0/2363509224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f666c0678d0 msgr2=0x7f666c09e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.879+0000 7f667151d640 1 --2- 172.21.15.178:0/2363509224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f666c0678d0 0x7f666c09e700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f665c009850 tx=0x7f665c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.879+0000 7f667151d640 1 -- 172.21.15.178:0/2363509224 shutdown_connections 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.879+0000 7f667151d640 1 --2- 172.21.15.178:0/2363509224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f666c06ea30 0x7f666c09ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.879+0000 7f667151d640 1 --2- 172.21.15.178:0/2363509224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f666c0678d0 0x7f666c09e700 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.879+0000 7f667151d640 1 -- 172.21.15.178:0/2363509224 >> 172.21.15.178:0/2363509224 conn(0x7f666c0a4390 msgr2=0x7f666c067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.879+0000 7f667151d640 1 -- 172.21.15.178:0/2363509224 shutdown_connections 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.879+0000 7f667151d640 1 -- 172.21.15.178:0/2363509224 wait complete. 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.879+0000 7f667151d640 10 monclient: build_initial_monmap 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.879+0000 7f667151d640 10 monclient: monmap: 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:41:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 1 Processor -- start 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 1 -- start start 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 10 monclient: init 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 10 monclient: _add_conns ranks=[0,6,7,8,4,5,1,2,3] 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f666c0678d0 0x7f666c13c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 10 monclient(hunting): picked mon.noname-a con 0x7f666c0678d0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f666c06ea30 0x7f666c13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 10 monclient(hunting): picked mon.noname-d con 0x7f666c06ea30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f666c09f180 0x7f666c13f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 10 monclient(hunting): picked mon.noname-i con 0x7f666c09f180 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c1210a0 con 0x7f666c0678d0 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f666c120ae0 con 0x7f666c06ea30 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f666c11e630 con 0x7f666c09f180 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f667151d640 10 monclient(hunting): authenticate will time out at 1286.631740s 2024-09-15T01:41:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f666c0678d0 0x7f666c13c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f666c0678d0 0x7f666c13c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33496/0 (socket says 172.21.15.178:33496) 2024-09-15T01:41:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666affd640 1 -- 172.21.15.178:0/2529351404 learned_addr learned my addr 172.21.15.178:0/2529351404 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666a7fc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f666c06ea30 0x7f666c13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666affd640 10 monclient(hunting): get_auth_request con 0x7f666c0678d0 auth_method 0 2024-09-15T01:41:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666a7fc640 10 monclient(hunting): get_auth_request con 0x7f666c06ea30 auth_method 0 2024-09-15T01:41:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.880+0000 7f666a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f666affd640 10 monclient(hunting): handle_auth_done global_id 34713 payload 824 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f666affd640 1 -- 172.21.15.178:0/2529351404 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f666c09f180 msgr2=0x7f666c13f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f666affd640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f666c09f180 0x7f666c13f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f666affd640 1 -- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f666c06ea30 msgr2=0x7f666c13b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f666affd640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f666c06ea30 0x7f666c13b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f666affd640 10 monclient: _finish_hunting 0 2024-09-15T01:41:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f666affd640 1 monclient: found mon.noname-a 2024-09-15T01:41:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f666affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f666affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:41:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f666affd640 1 -- 172.21.15.178:0/2529351404 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6660009390 con 0x7f666c0678d0 2024-09-15T01:41:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f666a7fc640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f666c06ea30 0x7f666c13b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:41:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f666affd640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f666c0678d0 0x7f666c13c0b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f665c035040 tx=0x7f665c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f665c03cb60 con 0x7f666c0678d0 2024-09-15T01:41:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f6652ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f6652ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:41:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f6652ffd640 10 monclient: dump: 2024-09-15T01:41:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.904 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f6652ffd640 10 monclient: _finish_auth 0 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f6652ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f667151d640 5 monclient: authenticate success, global_id 34713 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c03ccc0 con 0x7f666c0678d0 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f667151d640 10 monclient: _renew_subs 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f667151d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f667151d640 1 -- 172.21.15.178:0/2529351404 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c009390 con 0x7f666c0678d0 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f6652ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f665c03ce30 con 0x7f666c0678d0 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f6652ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f6652ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f6652ffd640 10 monclient: dump: 2024-09-15T01:41:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f667151d640 10 monclient: _renew_subs 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f66697fa640 4 set_mon_vals no callback set 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f667151d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.881+0000 7f667151d640 1 -- 172.21.15.178:0/2529351404 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666c120dc0 con 0x7f666c0678d0 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f667151d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f667151d640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c11f930 0x7f666c13fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f667151d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f667151d640 1 -- 172.21.15.178:0/2529351404 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f666c05ed70 con 0x7f666c11f930 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f665c03c6e0 con 0x7f666c0678d0 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666a7fc640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c11f930 0x7f666c13fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666a7fc640 10 monclient: get_auth_request con 0x7f666c11f930 auth_method 0 2024-09-15T01:41:21.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f6652ffd640 1 --2- 172.21.15.178:0/2529351404 >> v2:172.21.15.204:6812/28210 conn(0x7f66580397b0 0x7f665803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f665c06bfb0 con 0x7f666c0678d0 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666a7fc640 10 monclient(hunting): handle_auth_done global_id 44302 payload 824 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666a7fc640 1 -- 172.21.15.178:0/2529351404 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f666c04bc80 con 0x7f666c11f930 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666b7fe640 1 --2- 172.21.15.178:0/2529351404 >> v2:172.21.15.204:6812/28210 conn(0x7f66580397b0 0x7f665803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666a7fc640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c11f930 0x7f666c13fb60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6660011f20 tx=0x7f66600090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f666b7fe640 10 monclient: get_auth_request con 0x7f66580397b0 auth_method 0 2024-09-15T01:41:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6660014ad0 con 0x7f666c11f930 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.882+0000 7f6652ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.883+0000 7f6652ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.883+0000 7f6652ffd640 10 monclient: dump: 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.883+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f6660014c30 con 0x7f666c11f930 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.883+0000 7f6652ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.883+0000 7f6652ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.883+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c11f930 msgr2=0x7f666c13fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.883+0000 7f6652ffd640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c11f930 0x7f666c13fb60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6660011f20 tx=0x7f66600090f0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.883+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c11f930 msgr2=0x7f666c13fb60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:21.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.883+0000 7f6652ffd640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c11f930 0x7f666c13fb60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:21.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.883+0000 7f666b7fe640 1 --2- 172.21.15.178:0/2529351404 >> v2:172.21.15.204:6812/28210 conn(0x7f66580397b0 0x7f665803bc60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6664000f80 tx=0x7f6664001560 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.888+0000 7f667151d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:41:21.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.888+0000 7f667151d640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c13ca30 0x7f666c1400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:21.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.888+0000 7f667151d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:21.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.888+0000 7f667151d640 1 -- 172.21.15.178:0/2529351404 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f666c06d890 con 0x7f666c13ca30 2024-09-15T01:41:21.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.888+0000 7f666a7fc640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c13ca30 0x7f666c1400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:21.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.888+0000 7f666a7fc640 10 monclient: get_auth_request con 0x7f666c13ca30 auth_method 0 2024-09-15T01:41:21.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f666a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:21.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f666a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:21.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f666a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:21.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f666a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:21.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f666a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:21.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f666a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:21.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f666a7fc640 10 monclient(hunting): handle_auth_done global_id 44311 payload 824 2024-09-15T01:41:21.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f666a7fc640 1 -- 172.21.15.178:0/2529351404 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f666c04b720 con 0x7f666c13ca30 2024-09-15T01:41:21.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f666a7fc640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c13ca30 0x7f666c1400a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6660011f20 tx=0x7f666001c000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:21.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f666001ca70 con 0x7f666c13ca30 2024-09-15T01:41:21.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f6652ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:21.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f6652ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:41:21.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f6652ffd640 10 monclient: dump: 2024-09-15T01:41:21.914 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:21.914 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:21.914 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.914 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:21.914 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:21.914 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:21.914 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9301 (secure 0 0 0) 0x7f666001cbd0 con 0x7f666c13ca30 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f6652ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f6652ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:41:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c13ca30 msgr2=0x7f666c1400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f6652ffd640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c13ca30 0x7f666c1400a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6660011f20 tx=0x7f666001c000 comp rx=0 tx=0).stop 2024-09-15T01:41:21.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f6652ffd640 1 -- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c13ca30 msgr2=0x7f666c1400a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:21.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.889+0000 7f6652ffd640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f666c13ca30 0x7f666c1400a0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6660011f20 tx=0x7f666001c000 comp rx=0 tx=0).stop 2024-09-15T01:41:21.916 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:41:21.916 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:41:21.916 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:41:21.916 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:41:21.916 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 114, 2024-09-15T01:41:21.916 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:41:21.917 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:41:21.917 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:41:21.917 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:41:21.917 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:41:21.917 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:41:21.917 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:41:21.917 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:41:21.917 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:41:21.917 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.917 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:41:21.917 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:21.917 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:41:21.918 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:41:21.918 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:21.918 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:21.918 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:21.918 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:21.918 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:21.918 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:21.918 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:21.918 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:21.918 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:21.918 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.918 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.919 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:41:21.920 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:41:21.920 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:41:21.920 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:41:21.920 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:41:21.920 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:41:21.920 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:21.920 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:21.920 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:41:21.920 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:41:21.920 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:41:21.920 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:41:21.920 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:41:21.921 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:41:21.921 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:41:21.921 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:21.921 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:41:21.921 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:21.921 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:21.921 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:21.921 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:21.921 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:21.921 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:21.921 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:21.921 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:21.922 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:21.922 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.922 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:41:21.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.922 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:41:21.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.922 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:41:21.922 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:41:21.922 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.922 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.923 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:41:21.923 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.923 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.923 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:41:21.923 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.923 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.923 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.923 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:41:21.923 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:41:21.924 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.925 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.926 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.928 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:41:21.928 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:41:21.928 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.928 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:41:21.928 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.928 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:41:21.929 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.929 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.929 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.929 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:41:21.929 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:41:21.929 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.929 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.930 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.930 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:41:21.930 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:41:21.930 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.930 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.930 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.930 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:41:21.930 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.931 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.931 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.931 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:41:21.931 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.931 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.931 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.931 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:41:21.931 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:41:21.931 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.931 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.931 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.932 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.933 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.934 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:41:21.936 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:41:21.937 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.937 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.937 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.937 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:41:21.937 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:41:21.937 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:21.937 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:21.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.937 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:21.937 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:21.938 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.939 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:21.940 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:21.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:21.941 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:41:21.941 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:41:21.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.890+0000 7f667151d640 1 -- 172.21.15.178:0/2529351404 >> v2:172.21.15.204:6812/28210 conn(0x7f66580397b0 msgr2=0x7f665803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.890+0000 7f667151d640 1 --2- 172.21.15.178:0/2529351404 >> v2:172.21.15.204:6812/28210 conn(0x7f66580397b0 0x7f665803bc60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6664000f80 tx=0x7f6664001560 comp rx=0 tx=0).stop 2024-09-15T01:41:21.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.890+0000 7f667151d640 10 monclient: shutdown 2024-09-15T01:41:21.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.890+0000 7f667151d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:21.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.890+0000 7f667151d640 1 -- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f666c0678d0 msgr2=0x7f666c13c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:21.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.890+0000 7f667151d640 1 --2- 172.21.15.178:0/2529351404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f666c0678d0 0x7f666c13c0b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f665c035040 tx=0x7f665c032000 comp rx=0 tx=0).stop 2024-09-15T01:41:21.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.890+0000 7f666b7fe640 1 -- 172.21.15.178:0/2529351404 reap_dead start 2024-09-15T01:41:21.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.890+0000 7f666b7fe640 1 -- 172.21.15.178:0/2529351404 reap_dead start 2024-09-15T01:41:21.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.890+0000 7f667151d640 1 -- 172.21.15.178:0/2529351404 shutdown_connections 2024-09-15T01:41:21.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.890+0000 7f667151d640 1 -- 172.21.15.178:0/2529351404 >> 172.21.15.178:0/2529351404 conn(0x7f666c0a4390 msgr2=0x7f666c09b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:21.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.890+0000 7f667151d640 1 -- 172.21.15.178:0/2529351404 shutdown_connections 2024-09-15T01:41:21.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:21.890+0000 7f667151d640 1 -- 172.21.15.178:0/2529351404 wait complete. 2024-09-15T01:41:21.943 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T01:41:21.944 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-09-15T01:41:21.944 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:41:22.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.035+0000 7ffa6bf14640 10 monclient: get_monmap_and_config 2024-09-15T01:41:22.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.035+0000 7ffa6bf14640 10 monclient: build_initial_monmap 2024-09-15T01:41:22.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.035+0000 7ffa6bf14640 10 monclient: monmap: 2024-09-15T01:41:22.038 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:41:22.037069+0000 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:41:22.037069+0000 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:22.040 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:22.040 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:41:22.040 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:41:22.040 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.035+0000 7ffa6bf14640 1 Processor -- start 2024-09-15T01:41:22.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 1 -- start start 2024-09-15T01:41:22.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 10 monclient: init 2024-09-15T01:41:22.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:22.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 10 monclient: _add_conns ranks=[7,8,1,0,9,10,5,4,6,2,3] 2024-09-15T01:41:22.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffa640678d0 0x7ffa6409e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 10 monclient(hunting): picked mon.noname-i con 0x7ffa640678d0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:41:22.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa6406ea30 0x7ffa6409ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 10 monclient(hunting): picked mon.noname-e con 0x7ffa6406ea30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ffa6409f180 addr v1:172.21.15.204:6789/0 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6404bf60 con 0x7ffa6409f180 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6409f590 con 0x7ffa640678d0 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6406d5b0 con 0x7ffa6406ea30 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6bf14640 10 monclient(hunting): authenticate will time out at 1286.787498s 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa6406ea30 0x7ffa6409ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa6a4b1640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7ffa6409f180 legacy=0x7ffa641306b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa6406ea30 0x7ffa6409ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:57440/0 (socket says 172.21.15.178:57440) 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 1 -- 172.21.15.178:0/2855379947 learned_addr learned my addr 172.21.15.178:0/2855379947 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 10 monclient(hunting): get_auth_request con 0x7ffa6406ea30 auth_method 0 2024-09-15T01:41:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa69cb0640 1 --2- 172.21.15.178:0/2855379947 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffa640678d0 0x7ffa6409e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa69cb0640 10 monclient(hunting): get_auth_request con 0x7ffa640678d0 auth_method 0 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa69cb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa69cb0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa69cb0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 10 monclient(hunting): handle_auth_done global_id 34550 payload 274 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 1 -- 172.21.15.178:0/2855379947 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffa640678d0 msgr2=0x7ffa6409e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:41:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 1 --2- 172.21.15.178:0/2855379947 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffa640678d0 0x7ffa6409e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 1 -- 172.21.15.178:0/2855379947 >> v1:172.21.15.204:6789/0 conn(0x7ffa6409f180 legacy=0x7ffa641306b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 10 monclient: _finish_hunting 0 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 1 monclient: found mon.noname-e 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.036+0000 7ffa694af640 1 -- 172.21.15.178:0/2855379947 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64067ca0 con 0x7ffa6406ea30 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa694af640 1 --2- 172.21.15.178:0/2855379947 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa6406ea30 0x7ffa6409ec40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009970 tx=0x7ffa540091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=23fd00b55e7f7b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa68cae640 1 -- 172.21.15.178:0/2855379947 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa5402ad40 con 0x7ffa6406ea30 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa68cae640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa68cae640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa68cae640 10 monclient: dump: 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.054 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa68cae640 10 monclient: _finish_auth 0 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa68cae640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa6bf14640 5 monclient: authenticate success, global_id 34550 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa68cae640 1 -- 172.21.15.178:0/2855379947 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa54031820 con 0x7ffa6406ea30 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa6bf14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa68cae640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa6bf14640 10 monclient: get_monmap_and_config success 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa6bf14640 4 set_mon_vals no callback set 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa68cae640 1 -- 172.21.15.178:0/2855379947 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa54031e50 con 0x7ffa6406ea30 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa68cae640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa68cae640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.037+0000 7ffa68cae640 10 monclient: dump: 2024-09-15T01:41:22.055 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.056 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.038+0000 7ffa6bf14640 10 monclient: shutdown 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.038+0000 7ffa6bf14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.038+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2855379947 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa6406ea30 msgr2=0x7ffa6409ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.038+0000 7ffa6bf14640 1 --2- 172.21.15.178:0/2855379947 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa6406ea30 0x7ffa6409ec40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009970 tx=0x7ffa540091d0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.038+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2855379947 shutdown_connections 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.038+0000 7ffa6bf14640 1 --2- 172.21.15.178:0/2855379947 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa6406ea30 0x7ffa6409ec40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.038+0000 7ffa6bf14640 1 --2- 172.21.15.178:0/2855379947 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ffa640678d0 0x7ffa6409e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.038+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2855379947 >> 172.21.15.178:0/2855379947 conn(0x7ffa640a4390 msgr2=0x7ffa64067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.038+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2855379947 shutdown_connections 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.038+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2855379947 wait complete. 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.038+0000 7ffa6bf14640 10 monclient: build_initial_monmap 2024-09-15T01:41:22.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.038+0000 7ffa6bf14640 10 monclient: monmap: 2024-09-15T01:41:22.058 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:22.058 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:41:22.058 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:41:22.058 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:41:22.058 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:22.058 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:41:22.058 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:41:22.058 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:41:22.058 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:22.058 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 1 Processor -- start 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 1 -- start start 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 10 monclient: init 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 10 monclient: _add_conns ranks=[8,5,0,2,1,6,4,7,3] 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa640678d0 0x7ffa6413f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 10 monclient(hunting): picked mon.noname-e con 0x7ffa640678d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:41:22.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa6406ea30 0x7ffa64133730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 10 monclient(hunting): picked mon.noname-h con 0x7ffa6406ea30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:41:22.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa6409f180 0x7ffa6411d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 10 monclient(hunting): picked mon.noname-a con 0x7ffa6409f180 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:41:22.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6406c900 con 0x7ffa6409f180 2024-09-15T01:41:22.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa64120b30 con 0x7ffa6406ea30 2024-09-15T01:41:22.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ffa64140740 con 0x7ffa640678d0 2024-09-15T01:41:22.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:22.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6bf14640 10 monclient(hunting): authenticate will time out at 1286.790725s 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa69cb0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa640678d0 0x7ffa6413f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa69cb0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa640678d0 0x7ffa6413f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:57456/0 (socket says 172.21.15.178:57456) 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6a4b1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa6409f180 0x7ffa6411d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa69cb0640 1 -- 172.21.15.178:0/2858217534 learned_addr learned my addr 172.21.15.178:0/2858217534 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa69cb0640 10 monclient(hunting): get_auth_request con 0x7ffa640678d0 auth_method 0 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa69cb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa69cb0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa69cb0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6a4b1640 10 monclient(hunting): get_auth_request con 0x7ffa6409f180 auth_method 0 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6a4b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6a4b1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6a4b1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa69cb0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa69cb0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa69cb0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6a4b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6a4b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.039+0000 7ffa6a4b1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa694af640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa6406ea30 0x7ffa64133730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa69cb0640 10 monclient(hunting): handle_auth_done global_id 34559 payload 824 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa69cb0640 1 -- 172.21.15.178:0/2858217534 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa6406ea30 msgr2=0x7ffa64133730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa69cb0640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa6406ea30 0x7ffa64133730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa69cb0640 1 -- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa6409f180 msgr2=0x7ffa6411d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa69cb0640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa6409f180 0x7ffa6411d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa69cb0640 10 monclient: _finish_hunting 0 2024-09-15T01:41:22.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa69cb0640 1 monclient: found mon.noname-e 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa69cb0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa69cb0640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa69cb0640 1 -- 172.21.15.178:0/2858217534 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa54009450 con 0x7ffa640678d0 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa694af640 10 monclient: get_auth_request con 0x7ffa6406ea30 auth_method 0 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa694af640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ffa6406ea30 0x7ffa64133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa69cb0640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa640678d0 0x7ffa6413f8b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00e190 tx=0x7ffa5c009920 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa5c0148a0 con 0x7ffa640678d0 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa59ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa59ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa59ffb640 10 monclient: dump: 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.063 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa59ffb640 10 monclient: _finish_auth 0 2024-09-15T01:41:22.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa59ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:22.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c014a20 con 0x7ffa640678d0 2024-09-15T01:41:22.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa6bf14640 5 monclient: authenticate success, global_id 34559 2024-09-15T01:41:22.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa6bf14640 10 monclient: _renew_subs 2024-09-15T01:41:22.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa6bf14640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:41:22.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2858217534 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6411ed20 con 0x7ffa640678d0 2024-09-15T01:41:22.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa59ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:22.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa5c014b90 con 0x7ffa640678d0 2024-09-15T01:41:22.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa59ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa59ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:41:22.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa59ffb640 10 monclient: dump: 2024-09-15T01:41:22.065 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.066 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa6bf14640 10 monclient: _renew_subs 2024-09-15T01:41:22.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa5bfff640 4 set_mon_vals no callback set 2024-09-15T01:41:22.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa6bf14640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:41:22.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.040+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2858217534 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6413fdf0 con 0x7ffa640678d0 2024-09-15T01:41:22.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ffa5c014d10 con 0x7ffa640678d0 2024-09-15T01:41:22.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa6bf14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:22.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa6bf14640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa6411f8f0 0x7ffa640298c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa6bf14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2858217534 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffa641209f0 con 0x7ffa6411f8f0 2024-09-15T01:41:22.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa694af640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa6411f8f0 0x7ffa640298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa694af640 10 monclient: get_auth_request con 0x7ffa6411f8f0 auth_method 0 2024-09-15T01:41:22.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa694af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa694af640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa694af640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa59ffb640 1 --2- 172.21.15.178:0/2858217534 >> v2:172.21.15.204:6812/28210 conn(0x7ffa480397b0 0x7ffa4803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa694af640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7ffa5c04c300 con 0x7ffa640678d0 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa694af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa694af640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa694af640 10 monclient(hunting): handle_auth_done global_id 25107 payload 824 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa694af640 1 -- 172.21.15.178:0/2858217534 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa6404c4a0 con 0x7ffa6411f8f0 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa6a4b1640 1 --2- 172.21.15.178:0/2858217534 >> v2:172.21.15.204:6812/28210 conn(0x7ffa480397b0 0x7ffa4803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa694af640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa6411f8f0 0x7ffa640298c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa640678a0 tx=0x7ffa54032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.041+0000 7ffa6a4b1640 10 monclient: get_auth_request con 0x7ffa480397b0 auth_method 0 2024-09-15T01:41:22.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.042+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa5403b870 con 0x7ffa6411f8f0 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.042+0000 7ffa59ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.042+0000 7ffa59ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.042+0000 7ffa59ffb640 10 monclient: dump: 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.069 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.042+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7ffa5403b3f0 con 0x7ffa6411f8f0 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.042+0000 7ffa59ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.042+0000 7ffa59ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.042+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa6411f8f0 msgr2=0x7ffa640298c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.042+0000 7ffa59ffb640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa6411f8f0 0x7ffa640298c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa640678a0 tx=0x7ffa54032000 comp rx=0 tx=0).stop 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.042+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa6411f8f0 msgr2=0x7ffa640298c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.042+0000 7ffa59ffb640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa6411f8f0 0x7ffa640298c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.042+0000 7ffa6a4b1640 1 --2- 172.21.15.178:0/2858217534 >> v2:172.21.15.204:6812/28210 conn(0x7ffa480397b0 0x7ffa4803bc60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffa640a4930 tx=0x7ffa60009060 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.047+0000 7ffa6bf14640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:41:22.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa6bf14640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa64133c70 0x7ffa64029e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa6bf14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2858217534 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6404bc80 con 0x7ffa64133c70 2024-09-15T01:41:22.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa694af640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa64133c70 0x7ffa64029e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa694af640 10 monclient: get_auth_request con 0x7ffa64133c70 auth_method 0 2024-09-15T01:41:22.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa694af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa694af640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa694af640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa694af640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa694af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa694af640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa694af640 10 monclient(hunting): handle_auth_done global_id 25116 payload 824 2024-09-15T01:41:22.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa694af640 1 -- 172.21.15.178:0/2858217534 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffa6405ed70 con 0x7ffa64133c70 2024-09-15T01:41:22.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa694af640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa64133c70 0x7ffa64029e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009890 tx=0x7ffa5403a000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa5404a890 con 0x7ffa64133c70 2024-09-15T01:41:22.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa59ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa59ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa59ffb640 10 monclient: dump: 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.074 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.075 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.075 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.075 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.075 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9131 (secure 0 0 0) 0x7ffa5404a410 con 0x7ffa64133c70 2024-09-15T01:41:22.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa59ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:41:22.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa59ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:41:22.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa64133c70 msgr2=0x7ffa64029e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa59ffb640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa64133c70 0x7ffa64029e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009890 tx=0x7ffa5403a000 comp rx=0 tx=0).stop 2024-09-15T01:41:22.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa59ffb640 1 -- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa64133c70 msgr2=0x7ffa64029e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:22.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.048+0000 7ffa59ffb640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ffa64133c70 0x7ffa64029e00 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009890 tx=0x7ffa5403a000 comp rx=0 tx=0).stop 2024-09-15T01:41:22.076 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:41:22.076 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:41:22.076 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:41:22.076 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:41:22.076 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 114, 2024-09-15T01:41:22.076 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:41:22.076 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:41:22.076 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:41:22.076 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.077 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.078 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.079 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.079 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.079 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.079 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.079 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.079 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.079 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:41:22.079 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:41:22.080 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:41:22.080 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:41:22.080 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:41:22.080 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:41:22.080 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:22.080 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:22.080 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:41:22.081 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:41:22.081 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:41:22.081 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:41:22.081 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:41:22.081 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:41:22.081 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:41:22.081 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:22.081 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:41:22.081 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.081 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.081 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.081 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.082 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.082 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.082 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.082 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.082 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.082 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.082 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:41:22.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.082 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:41:22.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.082 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:41:22.083 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:41:22.083 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.083 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:41:22.083 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.085 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.086 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.086 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:41:22.086 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:41:22.086 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.086 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.086 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.086 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:41:22.086 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.087 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.088 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:41:22.089 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:41:22.091 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.091 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.091 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:41:22.091 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:41:22.091 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.091 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:41:22.094 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:41:22.095 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.097 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:41:22.098 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.099 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.099 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.099 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.099 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.099 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:41:22.099 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:41:22.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.100 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.100 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.100 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:22.100 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.100 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.100 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:41:22.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.100 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.100 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.100 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:41:22.100 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.101 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.101 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:41:22.101 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:41:22.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.049+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2858217534 >> v2:172.21.15.204:6812/28210 conn(0x7ffa480397b0 msgr2=0x7ffa4803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.049+0000 7ffa6bf14640 1 --2- 172.21.15.178:0/2858217534 >> v2:172.21.15.204:6812/28210 conn(0x7ffa480397b0 0x7ffa4803bc60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffa640a4930 tx=0x7ffa60009060 comp rx=0 tx=0).stop 2024-09-15T01:41:22.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.049+0000 7ffa6bf14640 10 monclient: shutdown 2024-09-15T01:41:22.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.049+0000 7ffa6bf14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:22.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.049+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa640678d0 msgr2=0x7ffa6413f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.049+0000 7ffa6bf14640 1 --2- 172.21.15.178:0/2858217534 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ffa640678d0 0x7ffa6413f8b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00e190 tx=0x7ffa5c009920 comp rx=0 tx=0).stop 2024-09-15T01:41:22.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.049+0000 7ffa6a4b1640 1 -- 172.21.15.178:0/2858217534 reap_dead start 2024-09-15T01:41:22.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.049+0000 7ffa6a4b1640 1 -- 172.21.15.178:0/2858217534 reap_dead start 2024-09-15T01:41:22.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.049+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2858217534 shutdown_connections 2024-09-15T01:41:22.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.049+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2858217534 >> 172.21.15.178:0/2858217534 conn(0x7ffa640a4390 msgr2=0x7ffa6409d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:22.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.049+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2858217534 shutdown_connections 2024-09-15T01:41:22.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.049+0000 7ffa6bf14640 1 -- 172.21.15.178:0/2858217534 wait complete. 2024-09-15T01:41:22.104 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T01:41:22.104 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-09-15T01:41:22.104 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:41:22.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.196+0000 7f31eded4640 10 monclient: get_monmap_and_config 2024-09-15T01:41:22.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.196+0000 7f31eded4640 10 monclient: build_initial_monmap 2024-09-15T01:41:22.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.197+0000 7f31eded4640 10 monclient: monmap: 2024-09-15T01:41:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:41:22.198879+0000 2024-09-15T01:41:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:41:22.198879+0000 2024-09-15T01:41:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:41:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:41:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:41:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:41:22.204 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.197+0000 7f31eded4640 1 Processor -- start 2024-09-15T01:41:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.197+0000 7f31eded4640 1 -- start start 2024-09-15T01:41:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.197+0000 7f31eded4640 10 monclient: init 2024-09-15T01:41:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 10 monclient: _add_conns ranks=[0,8,5,4,2,9,10,1,6,3,7] 2024-09-15T01:41:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f31e80a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:41:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8132de0 0x7f31e8137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 10 monclient(hunting): picked mon.noname-e con 0x7f31e8132de0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:41:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31e81380e0 0x7f31e813a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 10 monclient(hunting): picked mon.noname-h con 0x7f31e81380e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:41:22.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e804bf60 con 0x7f31e80a2930 2024-09-15T01:41:22.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31e809dd80 con 0x7f31e81380e0 2024-09-15T01:41:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f31e80a2d00 con 0x7f31e8132de0 2024-09-15T01:41:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31eded4640 10 monclient(hunting): authenticate will time out at 1286.949415s 2024-09-15T01:41:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31e77fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f31e80a2930 0x7f31e809faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:41286/0 (socket says 172.21.15.178:41286) 2024-09-15T01:41:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31e77fe640 1 -- 172.21.15.178:0/2302130426 learned_addr learned my addr 172.21.15.178:0/2302130426 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:41:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 1 --2- 172.21.15.178:0/2302130426 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8132de0 0x7f31e8137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 10 monclient(hunting): get_auth_request con 0x7f31e8132de0 auth_method 0 2024-09-15T01:41:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31e7fff640 1 --2- 172.21.15.178:0/2302130426 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31e81380e0 0x7f31e813a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31e7fff640 10 monclient(hunting): get_auth_request con 0x7f31e81380e0 auth_method 0 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 10 monclient(hunting): handle_auth_done global_id 34568 payload 274 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31e6ffd640 1 -- 172.21.15.178:0/2302130426 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553778490 0 0) 0x7f31e804bf60 con 0x7f31e80a2930 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 1 -- 172.21.15.178:0/2302130426 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31e81380e0 msgr2=0x7f31e813a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:41:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 1 --2- 172.21.15.178:0/2302130426 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31e81380e0 0x7f31e813a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 1 -- 172.21.15.178:0/2302130426 >> v1:172.21.15.178:6789/0 conn(0x7f31e80a2930 legacy=0x7f31e809faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 10 monclient: _finish_hunting 0 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 1 monclient: found mon.noname-e 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31dffff640 1 -- 172.21.15.178:0/2302130426 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e81331b0 con 0x7f31e8132de0 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31e6ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.198+0000 7f31e7fff640 1 --2- 172.21.15.178:0/2302130426 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31e81380e0 0x7f31e813a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31dffff640 1 --2- 172.21.15.178:0/2302130426 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8132de0 0x7f31e8137ba0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009880 tx=0x7f31d00090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=8a53629aa791e997 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31e6ffd640 1 -- 172.21.15.178:0/2302130426 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31d002ac50 con 0x7f31e8132de0 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31e6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:41:22.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31e6ffd640 10 monclient: dump: 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.219 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.220 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31e6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:41:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31eded4640 5 monclient: authenticate success, global_id 34568 2024-09-15T01:41:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31eded4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:41:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31e6ffd640 1 -- 172.21.15.178:0/2302130426 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d002add0 con 0x7f31e8132de0 2024-09-15T01:41:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31e6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31eded4640 10 monclient: get_monmap_and_config success 2024-09-15T01:41:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31eded4640 4 set_mon_vals no callback set 2024-09-15T01:41:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31e6ffd640 1 -- 172.21.15.178:0/2302130426 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31d0031ca0 con 0x7f31e8132de0 2024-09-15T01:41:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31e6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.199+0000 7f31e6ffd640 10 monclient: dump: 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.222 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.200+0000 7f31eded4640 10 monclient: shutdown 2024-09-15T01:41:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.200+0000 7f31eded4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.200+0000 7f31eded4640 1 -- 172.21.15.178:0/2302130426 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8132de0 msgr2=0x7f31e8137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.200+0000 7f31eded4640 1 --2- 172.21.15.178:0/2302130426 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8132de0 0x7f31e8137ba0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009880 tx=0x7f31d00090f0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.200+0000 7f31eded4640 1 -- 172.21.15.178:0/2302130426 shutdown_connections 2024-09-15T01:41:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.200+0000 7f31eded4640 1 --2- 172.21.15.178:0/2302130426 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31e81380e0 0x7f31e813a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.200+0000 7f31eded4640 1 --2- 172.21.15.178:0/2302130426 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8132de0 0x7f31e8137ba0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.200+0000 7f31eded4640 1 -- 172.21.15.178:0/2302130426 >> 172.21.15.178:0/2302130426 conn(0x7f31e80a5ed0 msgr2=0x7f31e80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.200+0000 7f31eded4640 1 -- 172.21.15.178:0/2302130426 shutdown_connections 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.200+0000 7f31eded4640 1 -- 172.21.15.178:0/2302130426 wait complete. 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.200+0000 7f31eded4640 10 monclient: build_initial_monmap 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.200+0000 7f31eded4640 10 monclient: monmap: 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:41:22.223 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 1 Processor -- start 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 1 -- start start 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 10 monclient: init 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:22.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 10 monclient: _add_conns ranks=[2,5,4,0,1,6,8,7,3] 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31e80a2930 0x7f31e813e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 10 monclient(hunting): picked mon.noname-b con 0x7f31e80a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31e8132de0 0x7f31e809c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 10 monclient(hunting): picked mon.noname-h con 0x7f31e8132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f31e81380e0 0x7f31e8141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 10 monclient(hunting): picked mon.noname-c con 0x7f31e81380e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e80a2390 con 0x7f31e80a2930 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31e81362e0 con 0x7f31e81380e0 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31e809d630 con 0x7f31e8132de0 2024-09-15T01:41:22.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31eded4640 10 monclient(hunting): authenticate will time out at 1286.952797s 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e77fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31e80a2930 0x7f31e813e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f31e81380e0 0x7f31e8141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f31e81380e0 0x7f31e8141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:42236/0 (socket says 172.21.15.178:42236) 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e77fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31e80a2930 0x7f31e813e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:58276/0 (socket says 172.21.15.178:58276) 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e7fff640 1 -- 172.21.15.178:0/1340499144 learned_addr learned my addr 172.21.15.178:0/1340499144 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e7fff640 10 monclient(hunting): get_auth_request con 0x7f31e81380e0 auth_method 0 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e77fe640 10 monclient(hunting): get_auth_request con 0x7f31e80a2930 auth_method 0 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.201+0000 7f31e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31dffff640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31e8132de0 0x7f31e809c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31dffff640 10 monclient(hunting): get_auth_request con 0x7f31e8132de0 auth_method 0 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31dffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 10 monclient(hunting): handle_auth_done global_id 44320 payload 824 2024-09-15T01:41:22.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 1 -- 172.21.15.178:0/1340499144 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31e8132de0 msgr2=0x7f31e809c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f31e8132de0 0x7f31e809c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 1 -- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31e80a2930 msgr2=0x7f31e813e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31e80a2930 0x7f31e813e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:41:22.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 1 monclient: found mon.noname-c 2024-09-15T01:41:22.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:22.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:41:22.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 1 -- 172.21.15.178:0/1340499144 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0009390 con 0x7f31e81380e0 2024-09-15T01:41:22.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e77fe640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31e80a2930 0x7f31e813e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:41:22.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e7fff640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f31e81380e0 0x7f31e8141a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f31e000d180 tx=0x7f31e0009a50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31e0015880 con 0x7f31e81380e0 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31df7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31df7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31df7fe640 10 monclient: dump: 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.229 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.230 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.230 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.230 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.230 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.230 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31df7fe640 10 monclient: _finish_auth 0 2024-09-15T01:41:22.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31df7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:22.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e000f040 con 0x7f31e81380e0 2024-09-15T01:41:22.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31eded4640 5 monclient: authenticate success, global_id 44320 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31df7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31eded4640 10 monclient: _renew_subs 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31e0015ea0 con 0x7f31e81380e0 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31eded4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31eded4640 1 -- 172.21.15.178:0/1340499144 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e8097590 con 0x7f31e81380e0 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31df7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31df7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31e67fc640 4 set_mon_vals no callback set 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31df7fe640 10 monclient: dump: 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.231 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31eded4640 10 monclient: _renew_subs 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31eded4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.202+0000 7f31eded4640 1 -- 172.21.15.178:0/1340499144 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e8137590 con 0x7f31e81380e0 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f31e0015a20 con 0x7f31e81380e0 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31eded4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31eded4640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e813ef20 0x7f31e8141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31eded4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31eded4640 1 -- 172.21.15.178:0/1340499144 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f31e8097a10 con 0x7f31e813ef20 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31df7fe640 1 --2- 172.21.15.178:0/1340499144 >> v2:172.21.15.204:6812/28210 conn(0x7f31c80397b0 0x7f31c803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f31e004c500 con 0x7f31e81380e0 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31e77fe640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e813ef20 0x7f31e8141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31e77fe640 10 monclient: get_auth_request con 0x7f31e813ef20 auth_method 0 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31dffff640 1 --2- 172.21.15.178:0/1340499144 >> v2:172.21.15.204:6812/28210 conn(0x7f31c80397b0 0x7f31c803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31e77fe640 10 monclient(hunting): handle_auth_done global_id 34577 payload 824 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31e77fe640 1 -- 172.21.15.178:0/1340499144 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31e804c4a0 con 0x7f31e813ef20 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.203+0000 7f31dffff640 10 monclient: get_auth_request con 0x7f31c80397b0 auth_method 0 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31e77fe640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e813ef20 0x7f31e8141fd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f31d800fb00 tx=0x7f31d8009040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31d80070d0 con 0x7f31e813ef20 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31df7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31df7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:41:22.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31df7fe640 10 monclient: dump: 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.235 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31dffff640 1 --2- 172.21.15.178:0/1340499144 >> v2:172.21.15.204:6812/28210 conn(0x7f31c80397b0 0x7f31c803bc60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f31d0034000 tx=0x7f31d0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f31d800dac0 con 0x7f31e813ef20 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31df7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31df7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e813ef20 msgr2=0x7f31e8141fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31df7fe640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e813ef20 0x7f31e8141fd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f31d800fb00 tx=0x7f31d8009040 comp rx=0 tx=0).stop 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e813ef20 msgr2=0x7f31e8141fd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.204+0000 7f31df7fe640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e813ef20 0x7f31e8141fd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31eded4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31eded4640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8142510 0x7f31e8130f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31eded4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31eded4640 1 -- 172.21.15.178:0/1340499144 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f31e804bc80 con 0x7f31e8142510 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31e77fe640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8142510 0x7f31e8130f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31e77fe640 10 monclient: get_auth_request con 0x7f31e8142510 auth_method 0 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31e77fe640 10 monclient(hunting): handle_auth_done global_id 34586 payload 824 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31e77fe640 1 -- 172.21.15.178:0/1340499144 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f31e805ed70 con 0x7f31e8142510 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31e77fe640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8142510 0x7f31e8130f60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f31d800fb00 tx=0x7f31d8018000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31d8007720 con 0x7f31e8142510 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31df7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31df7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.210+0000 7f31df7fe640 10 monclient: dump: 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.238 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.239 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.239 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.239 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.239 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.239 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9301 (secure 0 0 0) 0x7f31d8013840 con 0x7f31e8142510 2024-09-15T01:41:22.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31df7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:41:22.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31df7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:41:22.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8142510 msgr2=0x7f31e8130f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31df7fe640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8142510 0x7f31e8130f60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f31d800fb00 tx=0x7f31d8018000 comp rx=0 tx=0).stop 2024-09-15T01:41:22.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31df7fe640 1 -- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8142510 msgr2=0x7f31e8130f60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:22.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31df7fe640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f31e8142510 0x7f31e8130f60 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f31d800fb00 tx=0x7f31d8018000 comp rx=0 tx=0).stop 2024-09-15T01:41:22.240 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:41:22.240 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:41:22.240 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:41:22.240 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:41:22.240 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 114, 2024-09-15T01:41:22.240 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:41:22.240 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:41:22.240 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:41:22.240 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.241 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.242 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.243 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.243 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.243 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.243 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.243 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.243 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:41:22.243 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:41:22.243 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:41:22.243 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:41:22.243 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:41:22.244 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.245 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:41:22.246 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:41:22.246 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:41:22.246 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:41:22.246 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.250 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.251 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:41:22.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.252 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:41:22.252 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:41:22.252 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.252 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.252 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.253 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:41:22.255 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:41:22.256 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:41:22.259 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.259 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.259 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:41:22.259 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:41:22.259 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.259 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.259 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.260 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:41:22.261 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.261 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.261 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:41:22.261 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:41:22.261 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.261 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.261 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.261 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.261 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.262 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.263 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:22.263 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.263 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.263 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:41:22.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.263 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.263 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.263 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:22.263 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.263 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.263 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:41:22.264 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:41:22.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31eded4640 1 -- 172.21.15.178:0/1340499144 >> v2:172.21.15.204:6812/28210 conn(0x7f31c80397b0 msgr2=0x7f31c803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31eded4640 1 --2- 172.21.15.178:0/1340499144 >> v2:172.21.15.204:6812/28210 conn(0x7f31c80397b0 0x7f31c803bc60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f31d0034000 tx=0x7f31d0032000 comp rx=0 tx=0).stop 2024-09-15T01:41:22.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31eded4640 10 monclient: shutdown 2024-09-15T01:41:22.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31eded4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:22.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31eded4640 1 -- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f31e81380e0 msgr2=0x7f31e8141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31eded4640 1 --2- 172.21.15.178:0/1340499144 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f31e81380e0 0x7f31e8141a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f31e000d180 tx=0x7f31e0009a50 comp rx=0 tx=0).stop 2024-09-15T01:41:22.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31e7fff640 1 -- 172.21.15.178:0/1340499144 reap_dead start 2024-09-15T01:41:22.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31e7fff640 1 -- 172.21.15.178:0/1340499144 reap_dead start 2024-09-15T01:41:22.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31eded4640 1 -- 172.21.15.178:0/1340499144 shutdown_connections 2024-09-15T01:41:22.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31eded4640 1 -- 172.21.15.178:0/1340499144 >> 172.21.15.178:0/1340499144 conn(0x7f31e80a5ed0 msgr2=0x7f31e80a4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:22.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31eded4640 1 -- 172.21.15.178:0/1340499144 shutdown_connections 2024-09-15T01:41:22.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.211+0000 7f31eded4640 1 -- 172.21.15.178:0/1340499144 wait complete. 2024-09-15T01:41:22.266 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T01:41:22.267 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-09-15T01:41:22.267 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:41:22.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.360+0000 7fd41a55d640 10 monclient: get_monmap_and_config 2024-09-15T01:41:22.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.360+0000 7fd41a55d640 10 monclient: build_initial_monmap 2024-09-15T01:41:22.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.360+0000 7fd41a55d640 10 monclient: monmap: 2024-09-15T01:41:22.363 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:22.364 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.364 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:41:22.362373+0000 2024-09-15T01:41:22.364 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:41:22.362373+0000 2024-09-15T01:41:22.364 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:22.364 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.364 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:41:22.364 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:41:22.364 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:22.365 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:22.365 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:22.365 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:22.365 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:22.365 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:22.365 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:22.365 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:41:22.365 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:41:22.365 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 1 Processor -- start 2024-09-15T01:41:22.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 1 -- start start 2024-09-15T01:41:22.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 10 monclient: init 2024-09-15T01:41:22.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:22.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 10 monclient: _add_conns ranks=[2,0,9,6,10,1,8,5,4,3,7] 2024-09-15T01:41:22.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4140a2930 0x7fd414132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 10 monclient(hunting): picked mon.noname-b con 0x7fd4140a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:41:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd41409faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:41:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd41409fe70 0x7fd4140a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 10 monclient(hunting): picked mon.noname-a con 0x7fd41409fe70 addr v2:172.21.15.178:3300/0 2024-09-15T01:41:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41404bf60 con 0x7fd41409faa0 2024-09-15T01:41:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41409dd80 con 0x7fd41409fe70 2024-09-15T01:41:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140a2d00 con 0x7fd4140a2930 2024-09-15T01:41:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd41a55d640 10 monclient(hunting): authenticate will time out at 1287.112756s 2024-09-15T01:41:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd418afa640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd41409fe70 0x7fd4140a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd413fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4140a2930 0x7fd414132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd4137fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd41409faa0 0x7fd4140a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:41294/0 (socket says 172.21.15.178:41294) 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd4137fe640 1 -- 172.21.15.178:0/2408507649 learned_addr learned my addr 172.21.15.178:0/2408507649 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd418afa640 10 monclient(hunting): get_auth_request con 0x7fd41409fe70 auth_method 0 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd418afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd418afa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd418afa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd413fff640 10 monclient(hunting): get_auth_request con 0x7fd4140a2930 auth_method 0 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd413fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd413fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.361+0000 7fd413fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd413fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd413fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd413fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 10 monclient(hunting): handle_auth_done global_id 34740 payload 274 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 1 -- 172.21.15.178:0/2408507649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4140a2930 msgr2=0x7fd414132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 1 --2- 172.21.15.178:0/2408507649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4140a2930 0x7fd414132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 1 -- 172.21.15.178:0/2408507649 >> v1:172.21.15.178:6789/0 conn(0x7fd41409faa0 legacy=0x7fd4140a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 10 monclient: _finish_hunting 0 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 1 monclient: found mon.noname-a 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd412ffd640 1 -- 172.21.15.178:0/2408507649 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027240212 0 0) 0x7fd41404bf60 con 0x7fd41409faa0 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 1 -- 172.21.15.178:0/2408507649 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4140a0310 con 0x7fd41409fe70 2024-09-15T01:41:22.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd412ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd413fff640 1 --2- 172.21.15.178:0/2408507649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4140a2930 0x7fd414132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.362+0000 7fd418afa640 1 --2- 172.21.15.178:0/2408507649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd41409fe70 0x7fd4140a0fd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009e60 tx=0x7fd40c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c0dc059059c466 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd412ffd640 1 -- 172.21.15.178:0/2408507649 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd40c0079d0 con 0x7fd41409fe70 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd412ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd412ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd412ffd640 10 monclient: dump: 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.379 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd412ffd640 10 monclient: _finish_auth 0 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd412ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 5 monclient: authenticate success, global_id 34740 2024-09-15T01:41:22.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd412ffd640 1 -- 172.21.15.178:0/2408507649 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c02d050 con 0x7fd41409fe70 2024-09-15T01:41:22.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:41:22.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd412ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:22.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 10 monclient: get_monmap_and_config success 2024-09-15T01:41:22.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 4 set_mon_vals no callback set 2024-09-15T01:41:22.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd412ffd640 1 -- 172.21.15.178:0/2408507649 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd40c007b90 con 0x7fd41409fe70 2024-09-15T01:41:22.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd412ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd412ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:41:22.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd412ffd640 10 monclient: dump: 2024-09-15T01:41:22.381 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.381 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.381 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.381 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.382 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.382 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 10 monclient: shutdown 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 1 -- 172.21.15.178:0/2408507649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd41409fe70 msgr2=0x7fd4140a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 1 --2- 172.21.15.178:0/2408507649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd41409fe70 0x7fd4140a0fd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009e60 tx=0x7fd40c009780 comp rx=0 tx=0).stop 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 1 -- 172.21.15.178:0/2408507649 shutdown_connections 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 1 --2- 172.21.15.178:0/2408507649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd41409fe70 0x7fd4140a0fd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 1 --2- 172.21.15.178:0/2408507649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4140a2930 0x7fd414132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 1 -- 172.21.15.178:0/2408507649 >> 172.21.15.178:0/2408507649 conn(0x7fd4140a5ed0 msgr2=0x7fd4140a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 1 -- 172.21.15.178:0/2408507649 shutdown_connections 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.363+0000 7fd41a55d640 1 -- 172.21.15.178:0/2408507649 wait complete. 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.364+0000 7fd41a55d640 10 monclient: build_initial_monmap 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.364+0000 7fd41a55d640 10 monclient: monmap: 2024-09-15T01:41:22.383 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:22.384 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:22.385 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.364+0000 7fd41a55d640 1 Processor -- start 2024-09-15T01:41:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.364+0000 7fd41a55d640 1 -- start start 2024-09-15T01:41:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.364+0000 7fd41a55d640 10 monclient: init 2024-09-15T01:41:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient: _add_conns ranks=[6,8,5,1,4,0,7,3,2] 2024-09-15T01:41:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd41409faa0 0x7fd41409b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient(hunting): picked mon.noname-d con 0x7fd41409faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:41:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd41409fe70 0x7fd414131770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient(hunting): picked mon.noname-e con 0x7fd41409fe70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:41:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd4140a2930 0x7fd41411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient(hunting): picked mon.noname-h con 0x7fd4140a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd41411eb70 con 0x7fd4140a2930 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd41409f870 con 0x7fd41409faa0 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd414142b70 con 0x7fd41409fe70 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient(hunting): authenticate will time out at 1287.116155s 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd41409faa0 0x7fd41409b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd4137fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd41409fe70 0x7fd414131770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd41409faa0 0x7fd41409b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44236/0 (socket says 172.21.15.178:44236) 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 1 -- 172.21.15.178:0/2238025517 learned_addr learned my addr 172.21.15.178:0/2238025517 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:22.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 10 monclient(hunting): get_auth_request con 0x7fd41409faa0 auth_method 0 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd4137fe640 10 monclient(hunting): get_auth_request con 0x7fd41409fe70 auth_method 0 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd4137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd4137fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd4137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd418afa640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd4140a2930 0x7fd41411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd418afa640 10 monclient(hunting): get_auth_request con 0x7fd4140a2930 auth_method 0 2024-09-15T01:41:22.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd418afa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd418afa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd418afa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd4137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd4137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd4137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd418afa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd418afa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd418afa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 10 monclient(hunting): handle_auth_done global_id 25125 payload 824 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 1 -- 172.21.15.178:0/2238025517 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd41409fe70 msgr2=0x7fd414131770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd41409fe70 0x7fd414131770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 1 -- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd4140a2930 msgr2=0x7fd41411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd4140a2930 0x7fd41411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 10 monclient: _finish_hunting 0 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 1 monclient: found mon.noname-d 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 1 -- 172.21.15.178:0/2238025517 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd404009390 con 0x7fd41409faa0 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd413fff640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd41409faa0 0x7fd41409b280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd40400ec80 tx=0x7fd4040090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd40400eec0 con 0x7fd41409faa0 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd3fbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd3fbfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd3fbfff640 10 monclient: dump: 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.389 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd3fbfff640 10 monclient: _finish_auth 0 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd3fbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd404013470 con 0x7fd41409faa0 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 5 monclient: authenticate success, global_id 25125 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient: _renew_subs 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 1 -- 172.21.15.178:0/2238025517 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c009a20 con 0x7fd41409faa0 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd3fbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd404013a60 con 0x7fd41409faa0 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd3fbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd3fbfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd3fbfff640 10 monclient: dump: 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.391 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.392 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient: _renew_subs 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd41a55d640 1 -- 172.21.15.178:0/2238025517 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41411e890 con 0x7fd41409faa0 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.365+0000 7fd4127fc640 4 set_mon_vals no callback set 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.366+0000 7fd41a55d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.366+0000 7fd41a55d640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414137150 0x7fd41411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.366+0000 7fd41a55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.366+0000 7fd41a55d640 1 -- 172.21.15.178:0/2238025517 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41404c4a0 con 0x7fd414137150 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.366+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd404013bc0 con 0x7fd41409faa0 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd4137fe640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414137150 0x7fd41411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd3fbfff640 1 --2- 172.21.15.178:0/2238025517 >> v2:172.21.15.204:6812/28210 conn(0x7fd4000397e0 0x7fd40003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd4137fe640 10 monclient: get_auth_request con 0x7fd414137150 auth_method 0 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd4137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd4137fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd4137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd40404c2f0 con 0x7fd41409faa0 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd4137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd4137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd4137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd418afa640 1 --2- 172.21.15.178:0/2238025517 >> v2:172.21.15.204:6812/28210 conn(0x7fd4000397e0 0x7fd40003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd418afa640 10 monclient: get_auth_request con 0x7fd4000397e0 auth_method 0 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd4137fe640 10 monclient(hunting): handle_auth_done global_id 16212 payload 824 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd4137fe640 1 -- 172.21.15.178:0/2238025517 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd41405ed70 con 0x7fd414137150 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd4137fe640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414137150 0x7fd41411ca70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd4140a6ec0 tx=0x7fd408007ea0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd418afa640 1 --2- 172.21.15.178:0/2238025517 >> v2:172.21.15.204:6812/28210 conn(0x7fd4000397e0 0x7fd40003bc90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd40c034000 tx=0x7fd40c037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd408005ea0 con 0x7fd414137150 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd3fbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd3fbfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.367+0000 7fd3fbfff640 10 monclient: dump: 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.395 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.396 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.368+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fd408005a20 con 0x7fd414137150 2024-09-15T01:41:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.368+0000 7fd3fbfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.368+0000 7fd3fbfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:41:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.368+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414137150 msgr2=0x7fd41411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.368+0000 7fd3fbfff640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414137150 0x7fd41411ca70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd4140a6ec0 tx=0x7fd408007ea0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.368+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414137150 msgr2=0x7fd41411ca70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:22.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.368+0000 7fd3fbfff640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414137150 0x7fd41411ca70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.373+0000 7fd41a55d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:41:22.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.373+0000 7fd41a55d640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414131cb0 0x7fd41411cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.373+0000 7fd41a55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.373+0000 7fd41a55d640 1 -- 172.21.15.178:0/2238025517 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41404bc80 con 0x7fd414131cb0 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd4137fe640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414131cb0 0x7fd41411cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd4137fe640 10 monclient: get_auth_request con 0x7fd414131cb0 auth_method 0 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd4137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd4137fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd4137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd4137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd4137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd4137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd4137fe640 10 monclient(hunting): handle_auth_done global_id 16221 payload 824 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd4137fe640 1 -- 172.21.15.178:0/2238025517 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd41411edb0 con 0x7fd414131cb0 2024-09-15T01:41:22.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd4137fe640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414131cb0 0x7fd41411cfb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd408000fb0 tx=0x7fd408015000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd408013de0 con 0x7fd414131cb0 2024-09-15T01:41:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd3fbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd3fbfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:41:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.374+0000 7fd3fbfff640 10 monclient: dump: 2024-09-15T01:41:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.400 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.401 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9301 (secure 0 0 0) 0x7fd40802b460 con 0x7fd414131cb0 2024-09-15T01:41:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd3fbfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:41:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd3fbfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:41:22.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414131cb0 msgr2=0x7fd41411cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd3fbfff640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414131cb0 0x7fd41411cfb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd408000fb0 tx=0x7fd408015000 comp rx=0 tx=0).stop 2024-09-15T01:41:22.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd3fbfff640 1 -- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414131cb0 msgr2=0x7fd41411cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:22.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd3fbfff640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd414131cb0 0x7fd41411cfb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.402 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:41:22.402 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:41:22.402 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:41:22.403 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:41:22.403 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 114, 2024-09-15T01:41:22.403 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:41:22.403 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:41:22.403 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:41:22.403 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:41:22.403 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:41:22.403 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:41:22.403 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:41:22.403 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:41:22.403 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:41:22.403 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.404 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:41:22.404 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:22.404 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:41:22.404 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:41:22.404 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.404 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.404 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.404 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.404 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.404 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.404 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.404 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.405 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.405 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.405 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:41:22.405 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:41:22.405 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.405 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.405 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.405 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.405 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.405 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.405 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.405 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:41:22.406 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:41:22.407 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:41:22.407 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:41:22.407 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:41:22.407 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:41:22.407 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:41:22.407 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:22.407 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:41:22.407 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.407 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.407 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.407 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.407 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.408 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.408 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.408 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.408 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.408 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.408 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:41:22.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.408 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:41:22.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.408 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:41:22.408 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:41:22.408 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:41:22.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:41:22.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.409 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.409 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.411 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:41:22.411 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.411 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:41:22.412 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.412 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.412 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:41:22.413 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:41:22.413 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.413 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.413 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.413 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:41:22.413 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:41:22.415 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:41:22.415 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.415 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.415 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.415 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:41:22.415 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:41:22.415 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:41:22.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:41:22.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.416 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.416 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.417 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:41:22.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:41:22.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.418 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:41:22.419 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:41:22.419 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.419 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.419 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.420 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:41:22.420 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:41:22.420 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:41:22.421 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:41:22.422 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.422 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.422 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:41:22.423 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:41:22.423 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.423 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.423 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.424 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:41:22.424 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:41:22.424 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.425 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.425 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:41:22.425 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.426 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:41:22.426 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.426 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.426 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.427 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:41:22.427 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:41:22.427 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.427 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.427 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.429 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:41:22.429 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:41:22.429 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.429 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.430 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.430 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:41:22.430 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.430 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.431 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:41:22.431 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.431 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.431 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.431 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:41:22.431 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:41:22.432 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.432 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.432 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.433 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:41:22.433 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:41:22.433 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.433 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.433 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.434 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:41:22.434 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.434 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.434 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:41:22.435 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.435 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.435 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:41:22.435 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:41:22.436 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.436 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.436 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.436 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.436 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.437 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:41:22.437 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:41:22.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.437 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.437 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.437 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:22.438 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.438 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.438 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:41:22.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.438 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.438 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.439 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:41:22.439 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.439 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.439 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:41:22.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.440 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.440 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.440 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:22.440 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.440 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.441 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:41:22.441 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:41:22.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd41a55d640 1 -- 172.21.15.178:0/2238025517 >> v2:172.21.15.204:6812/28210 conn(0x7fd4000397e0 msgr2=0x7fd40003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd41a55d640 1 --2- 172.21.15.178:0/2238025517 >> v2:172.21.15.204:6812/28210 conn(0x7fd4000397e0 0x7fd40003bc90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd40c034000 tx=0x7fd40c037000 comp rx=0 tx=0).stop 2024-09-15T01:41:22.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd41a55d640 10 monclient: shutdown 2024-09-15T01:41:22.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd41a55d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:22.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd41a55d640 1 -- 172.21.15.178:0/2238025517 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd41409faa0 msgr2=0x7fd41409b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd41a55d640 1 --2- 172.21.15.178:0/2238025517 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd41409faa0 0x7fd41409b280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd40400ec80 tx=0x7fd4040090f0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd418afa640 1 -- 172.21.15.178:0/2238025517 reap_dead start 2024-09-15T01:41:22.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd418afa640 1 -- 172.21.15.178:0/2238025517 reap_dead start 2024-09-15T01:41:22.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd41a55d640 1 -- 172.21.15.178:0/2238025517 shutdown_connections 2024-09-15T01:41:22.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd41a55d640 1 -- 172.21.15.178:0/2238025517 >> 172.21.15.178:0/2238025517 conn(0x7fd4140a5ed0 msgr2=0x7fd414135980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:22.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd41a55d640 1 -- 172.21.15.178:0/2238025517 shutdown_connections 2024-09-15T01:41:22.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.375+0000 7fd41a55d640 1 -- 172.21.15.178:0/2238025517 wait complete. 2024-09-15T01:41:22.446 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T01:41:22.446 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-09-15T01:41:22.447 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:41:22.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient: get_monmap_and_config 2024-09-15T01:41:22.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient: build_initial_monmap 2024-09-15T01:41:22.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient: monmap: 2024-09-15T01:41:22.549 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:22.549 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.549 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:41:22.547804+0000 2024-09-15T01:41:22.549 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:41:22.547804+0000 2024-09-15T01:41:22.549 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:22.549 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.549 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:41:22.549 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:41:22.549 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:22.550 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:22.550 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:22.550 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:22.550 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:22.550 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:22.550 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:22.550 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:41:22.550 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:41:22.550 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 1 Processor -- start 2024-09-15T01:41:22.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 1 -- start start 2024-09-15T01:41:22.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient: init 2024-09-15T01:41:22.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:22.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient: _add_conns ranks=[10,8,7,6,5,3,0,9,1,2,4] 2024-09-15T01:41:22.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc0200a2930 0x7fc020132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient(hunting): picked mon.noname-f con 0x7fc0200a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:41:22.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc02009faa0 0x7fc02009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient(hunting): picked mon.noname-e con 0x7fc02009faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:41:22.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0200a08c0 0x7fc0200a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient(hunting): picked mon.noname-i con 0x7fc0200a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:41:22.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02009e0f0 con 0x7fc0200a2930 2024-09-15T01:41:22.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc02009dd80 con 0x7fc0200a08c0 2024-09-15T01:41:22.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200a2d00 con 0x7fc02009faa0 2024-09-15T01:41:22.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:22.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.546+0000 7fc02571d640 10 monclient(hunting): authenticate will time out at 1287.297999s 2024-09-15T01:41:22.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc02009faa0 0x7fc02009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01effd640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7fc0200a2930 msgr2=0x7fc020132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:41:22.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01effd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc0200a2930 0x7fc020132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:41:22.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc02009faa0 0x7fc02009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:57496/0 (socket says 172.21.15.178:57496) 2024-09-15T01:41:22.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 1 -- 172.21.15.178:0/1440416465 learned_addr learned my addr 172.21.15.178:0/1440416465 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:22.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 10 monclient(hunting): get_auth_request con 0x7fc02009faa0 auth_method 0 2024-09-15T01:41:22.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01f7fe640 1 --2- 172.21.15.178:0/1440416465 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0200a08c0 0x7fc0200a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01f7fe640 10 monclient(hunting): get_auth_request con 0x7fc0200a08c0 auth_method 0 2024-09-15T01:41:22.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 10 monclient(hunting): handle_auth_done global_id 34604 payload 274 2024-09-15T01:41:22.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 1 -- 172.21.15.178:0/1440416465 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0200a08c0 msgr2=0x7fc0200a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:41:22.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 1 --2- 172.21.15.178:0/1440416465 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0200a08c0 0x7fc0200a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 1 -- 172.21.15.178:0/1440416465 >> v2:172.21.15.204:3300/0 conn(0x7fc0200a2930 msgr2=0x7fc020132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 1 --2- 172.21.15.178:0/1440416465 >> v2:172.21.15.204:3300/0 conn(0x7fc0200a2930 0x7fc020132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 1 monclient: found mon.noname-e 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01e7fc640 1 -- 172.21.15.178:0/1440416465 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02004bf60 con 0x7fc02009faa0 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.547+0000 7fc01f7fe640 1 --2- 172.21.15.178:0/1440416465 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0200a08c0 0x7fc0200a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01e7fc640 1 --2- 172.21.15.178:0/1440416465 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc02009faa0 0x7fc02009fe70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc014009f20 tx=0x7fc014009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=3de104c3087c2dc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01dffb640 1 -- 172.21.15.178:0/1440416465 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc014007880 con 0x7fc02009faa0 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01dffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01dffb640 10 monclient: dump: 2024-09-15T01:41:22.565 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.566 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.567 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.567 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01dffb640 10 monclient: _finish_auth 0 2024-09-15T01:41:22.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:22.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01dffb640 1 -- 172.21.15.178:0/1440416465 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01402d050 con 0x7fc02009faa0 2024-09-15T01:41:22.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 5 monclient: authenticate success, global_id 34604 2024-09-15T01:41:22.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:41:22.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:22.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 10 monclient: get_monmap_and_config success 2024-09-15T01:41:22.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01dffb640 1 -- 172.21.15.178:0/1440416465 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc014030360 con 0x7fc02009faa0 2024-09-15T01:41:22.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 4 set_mon_vals no callback set 2024-09-15T01:41:22.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc01dffb640 10 monclient: dump: 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.568 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.569 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.569 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.569 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.569 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 10 monclient: shutdown 2024-09-15T01:41:22.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:22.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 1 -- 172.21.15.178:0/1440416465 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc02009faa0 msgr2=0x7fc02009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 1 --2- 172.21.15.178:0/1440416465 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc02009faa0 0x7fc02009fe70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc014009f20 tx=0x7fc014009840 comp rx=0 tx=0).stop 2024-09-15T01:41:22.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 1 -- 172.21.15.178:0/1440416465 shutdown_connections 2024-09-15T01:41:22.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 1 --2- 172.21.15.178:0/1440416465 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc0200a08c0 0x7fc0200a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 1 --2- 172.21.15.178:0/1440416465 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc02009faa0 0x7fc02009fe70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 1 --2- 172.21.15.178:0/1440416465 >> v2:172.21.15.204:3300/0 conn(0x7fc0200a2930 0x7fc020132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 1 -- 172.21.15.178:0/1440416465 >> 172.21.15.178:0/1440416465 conn(0x7fc0200a5ed0 msgr2=0x7fc0200a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 1 -- 172.21.15.178:0/1440416465 shutdown_connections 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.548+0000 7fc02571d640 1 -- 172.21.15.178:0/1440416465 wait complete. 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.549+0000 7fc02571d640 10 monclient: build_initial_monmap 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.549+0000 7fc02571d640 10 monclient: monmap: 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:41:22.570 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.549+0000 7fc02571d640 1 Processor -- start 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.549+0000 7fc02571d640 1 -- start start 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.549+0000 7fc02571d640 10 monclient: init 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 10 monclient: _add_conns ranks=[1,3,0,7,2,4,8,5,6] 2024-09-15T01:41:22.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc02009faa0 0x7fc02009ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 10 monclient(hunting): picked mon.noname-f con 0x7fc02009faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0200a08c0 0x7fc020131750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 10 monclient(hunting): picked mon.noname-g con 0x7fc0200a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0200a2930 0x7fc02011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 10 monclient(hunting): picked mon.noname-a con 0x7fc0200a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02011e870 con 0x7fc0200a2930 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200a2390 con 0x7fc02009faa0 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc020142a30 con 0x7fc0200a08c0 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc02571d640 10 monclient(hunting): authenticate will time out at 1287.301482s 2024-09-15T01:41:22.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01effd640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc02009faa0 msgr2=0x7fc02009ae70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:41:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc02009faa0 0x7fc02009ae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:41:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0200a2930 0x7fc02011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0200a2930 0x7fc02011c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33534/0 (socket says 172.21.15.178:33534) 2024-09-15T01:41:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 1 -- 172.21.15.178:0/3907770114 learned_addr learned my addr 172.21.15.178:0/3907770114 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 10 monclient(hunting): get_auth_request con 0x7fc0200a2930 auth_method 0 2024-09-15T01:41:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01e7fc640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0200a08c0 0x7fc020131750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01e7fc640 10 monclient(hunting): get_auth_request con 0x7fc0200a08c0 auth_method 0 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 10 monclient(hunting): handle_auth_done global_id 34758 payload 824 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 1 -- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0200a08c0 msgr2=0x7fc020131750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0200a08c0 0x7fc020131750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 1 -- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc02009faa0 msgr2=0x7fc02009ae70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc02009faa0 0x7fc02009ae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:41:22.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 1 monclient: found mon.noname-a 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.550+0000 7fc01f7fe640 1 -- 172.21.15.178:0/3907770114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc014009ae0 con 0x7fc0200a2930 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc01f7fe640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0200a2930 0x7fc02011c530 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d890 tx=0x7fc010008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc010007470 con 0x7fc0200a2930 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc006ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc006ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc006ffd640 10 monclient: dump: 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.575 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc006ffd640 10 monclient: _finish_auth 0 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc006ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:22.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01000bb40 con 0x7fc0200a2930 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc02571d640 5 monclient: authenticate success, global_id 34758 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc02571d640 10 monclient: _renew_subs 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc02571d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc02571d640 1 -- 172.21.15.178:0/3907770114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02009bf30 con 0x7fc0200a2930 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc006ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc01000bcb0 con 0x7fc0200a2930 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc006ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc006ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc006ffd640 10 monclient: dump: 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.577 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc01d7fa640 4 set_mon_vals no callback set 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc02571d640 10 monclient: _renew_subs 2024-09-15T01:41:22.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc02571d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.551+0000 7fc02571d640 1 -- 172.21.15.178:0/3907770114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02009ba20 con 0x7fc0200a2930 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc02571d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc02571d640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc020131c90 0x7fc02011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc02571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc02571d640 1 -- 172.21.15.178:0/3907770114 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc020132060 con 0x7fc020131c90 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc01001e460 con 0x7fc0200a2930 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc006ffd640 1 --2- 172.21.15.178:0/3907770114 >> v2:172.21.15.204:6812/28210 conn(0x7fc00c0397e0 0x7fc00c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc01effd640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc020131c90 0x7fc02011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc01effd640 10 monclient: get_auth_request con 0x7fc020131c90 auth_method 0 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc01effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc01effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc01effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fc01004c320 con 0x7fc0200a2930 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc01effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc01effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc01effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc01e7fc640 1 --2- 172.21.15.178:0/3907770114 >> v2:172.21.15.204:6812/28210 conn(0x7fc00c0397e0 0x7fc00c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc01e7fc640 10 monclient: get_auth_request con 0x7fc00c0397e0 auth_method 0 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc01effd640 10 monclient(hunting): handle_auth_done global_id 7016 payload 824 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.552+0000 7fc01effd640 1 -- 172.21.15.178:0/3907770114 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc02004c4a0 con 0x7fc020131c90 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc01effd640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc020131c90 0x7fc02011ca70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b440 tx=0x7fc018009120 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc01e7fc640 1 --2- 172.21.15.178:0/3907770114 >> v2:172.21.15.204:6812/28210 conn(0x7fc00c0397e0 0x7fc00c03bc90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc014033000 tx=0x7fc014036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0180050f0 con 0x7fc020131c90 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc006ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc006ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:41:22.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc006ffd640 10 monclient: dump: 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.581 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.582 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fc018003b80 con 0x7fc020131c90 2024-09-15T01:41:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc006ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc006ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:41:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc020131c90 msgr2=0x7fc02011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc006ffd640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc020131c90 0x7fc02011ca70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b440 tx=0x7fc018009120 comp rx=0 tx=0).stop 2024-09-15T01:41:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc020131c90 msgr2=0x7fc02011ca70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:22.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.553+0000 7fc006ffd640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc020131c90 0x7fc02011ca70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc02571d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:41:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc02571d640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc02011cfb0 0x7fc020096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc02571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc02571d640 1 -- 172.21.15.178:0/3907770114 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc02011ec10 con 0x7fc02011cfb0 2024-09-15T01:41:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc01effd640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc02011cfb0 0x7fc020096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc01effd640 10 monclient: get_auth_request con 0x7fc02011cfb0 auth_method 0 2024-09-15T01:41:22.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc01effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc01effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc01effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc01effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc01effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc01effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc01effd640 10 monclient(hunting): handle_auth_done global_id 7025 payload 824 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc01effd640 1 -- 172.21.15.178:0/3907770114 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc02004bc80 con 0x7fc02011cfb0 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.559+0000 7fc01effd640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc02011cfb0 0x7fc020096a30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc018000c00 tx=0x7fc018015000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0180057c0 con 0x7fc02011cfb0 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc006ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc006ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc006ffd640 10 monclient: dump: 2024-09-15T01:41:22.586 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.587 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.588 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9303 (secure 0 0 0) 0x7fc01801e620 con 0x7fc02011cfb0 2024-09-15T01:41:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc006ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:41:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc006ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:41:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc02011cfb0 msgr2=0x7fc020096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc006ffd640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc02011cfb0 0x7fc020096a30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc018000c00 tx=0x7fc018015000 comp rx=0 tx=0).stop 2024-09-15T01:41:22.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc006ffd640 1 -- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc02011cfb0 msgr2=0x7fc020096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:22.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc006ffd640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc02011cfb0 0x7fc020096a30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.589 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:41:22.589 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:41:22.589 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:41:22.589 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:41:22.589 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 114, 2024-09-15T01:41:22.589 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:41:22.590 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:41:22.590 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:41:22.590 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:41:22.590 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:41:22.590 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:41:22.590 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:41:22.590 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:41:22.590 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:41:22.590 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.590 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.591 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.592 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:41:22.592 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:41:22.592 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.592 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.592 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.592 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.592 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.592 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.593 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.593 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.593 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.593 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.593 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:41:22.593 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:41:22.593 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:41:22.593 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:41:22.593 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:41:22.593 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:41:22.593 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:22.594 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:22.594 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:41:22.594 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:41:22.594 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:41:22.594 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:41:22.594 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:41:22.594 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:41:22.594 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:41:22.594 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:22.594 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:41:22.594 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.595 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.595 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.595 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.595 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.595 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.595 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.595 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.595 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.595 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.595 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:41:22.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.595 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:41:22.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.596 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:41:22.596 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:41:22.596 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.596 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.596 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.596 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:41:22.596 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:41:22.599 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.599 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.599 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:41:22.599 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:41:22.599 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.599 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.599 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.600 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:41:22.600 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:41:22.600 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.601 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:41:22.601 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.601 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:41:22.601 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.601 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.601 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.602 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:41:22.602 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:41:22.602 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.602 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.602 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.603 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:41:22.603 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:41:22.603 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:41:22.603 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.604 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.605 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:41:22.608 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:41:22.608 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.608 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.608 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.608 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:41:22.608 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:41:22.608 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.608 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.608 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:41:22.609 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.610 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.610 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.610 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:41:22.610 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:41:22.610 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.610 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:41:22.610 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.611 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:41:22.614 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.615 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.615 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:41:22.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.615 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.616 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.616 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:41:22.616 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.616 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.616 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:41:22.617 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:41:22.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc02571d640 1 -- 172.21.15.178:0/3907770114 >> v2:172.21.15.204:6812/28210 conn(0x7fc00c0397e0 msgr2=0x7fc00c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc02571d640 1 --2- 172.21.15.178:0/3907770114 >> v2:172.21.15.204:6812/28210 conn(0x7fc00c0397e0 0x7fc00c03bc90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc014033000 tx=0x7fc014036000 comp rx=0 tx=0).stop 2024-09-15T01:41:22.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc02571d640 10 monclient: shutdown 2024-09-15T01:41:22.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc02571d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:22.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc02571d640 1 -- 172.21.15.178:0/3907770114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0200a2930 msgr2=0x7fc02011c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc02571d640 1 --2- 172.21.15.178:0/3907770114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0200a2930 0x7fc02011c530 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d890 tx=0x7fc010008040 comp rx=0 tx=0).stop 2024-09-15T01:41:22.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc01f7fe640 1 -- 172.21.15.178:0/3907770114 reap_dead start 2024-09-15T01:41:22.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc01f7fe640 1 -- 172.21.15.178:0/3907770114 reap_dead start 2024-09-15T01:41:22.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc02571d640 1 -- 172.21.15.178:0/3907770114 shutdown_connections 2024-09-15T01:41:22.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc02571d640 1 -- 172.21.15.178:0/3907770114 >> 172.21.15.178:0/3907770114 conn(0x7fc0200a5ed0 msgr2=0x7fc020135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:22.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc02571d640 1 -- 172.21.15.178:0/3907770114 shutdown_connections 2024-09-15T01:41:22.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.560+0000 7fc02571d640 1 -- 172.21.15.178:0/3907770114 wait complete. 2024-09-15T01:41:22.619 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T01:41:22.619 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-09-15T01:41:22.620 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:41:22.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.712+0000 7fd72475d640 10 monclient: get_monmap_and_config 2024-09-15T01:41:22.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.712+0000 7fd72475d640 10 monclient: build_initial_monmap 2024-09-15T01:41:22.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.712+0000 7fd72475d640 10 monclient: monmap: 2024-09-15T01:41:22.715 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:22.715 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.715 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:41:22.713896+0000 2024-09-15T01:41:22.715 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:41:22.713896+0000 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:41:22.716 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:41:22.717 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.712+0000 7fd72475d640 1 Processor -- start 2024-09-15T01:41:22.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.712+0000 7fd72475d640 1 -- start start 2024-09-15T01:41:22.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.712+0000 7fd72475d640 10 monclient: init 2024-09-15T01:41:22.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:22.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 10 monclient: _add_conns ranks=[0,6,7,1,5,10,4,2,3,9,8] 2024-09-15T01:41:22.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd71c0a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:41:22.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd71c132de0 0x7fd71c137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 10 monclient(hunting): picked mon.noname-d con 0x7fd71c132de0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c1380e0 0x7fd71c13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 10 monclient(hunting): picked mon.noname-i con 0x7fd71c1380e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71c04bf60 con 0x7fd71c0a2930 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c09dd80 con 0x7fd71c132de0 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0a2d00 con 0x7fd71c1380e0 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd72475d640 10 monclient(hunting): authenticate will time out at 1287.464272s 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd7224f9640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd71c0a2930 0x7fd71c09faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:41310/0 (socket says 172.21.15.178:41310) 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd7224f9640 1 -- 172.21.15.178:0/3751875692 learned_addr learned my addr 172.21.15.178:0/3751875692 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 1 --2- 172.21.15.178:0/3751875692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd71c132de0 0x7fd71c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 10 monclient(hunting): get_auth_request con 0x7fd71c132de0 auth_method 0 2024-09-15T01:41:22.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd722cfa640 1 --2- 172.21.15.178:0/3751875692 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c1380e0 0x7fd71c13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd722cfa640 10 monclient(hunting): get_auth_request con 0x7fd71c1380e0 auth_method 0 2024-09-15T01:41:22.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd722cfa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd722cfa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd722cfa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd7214f7640 1 -- 172.21.15.178:0/3751875692 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854733510 0 0) 0x7fd71c04bf60 con 0x7fd71c0a2930 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd7214f7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd7214f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd7214f7640 10 monclient(hunting): my global_id is 34767 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd7214f7640 1 -- 172.21.15.178:0/3751875692 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7040034e0 con 0x7fd71c0a2930 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 10 monclient(hunting): handle_auth_done global_id 25134 payload 274 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 1 -- 172.21.15.178:0/3751875692 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c1380e0 msgr2=0x7fd71c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 1 --2- 172.21.15.178:0/3751875692 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c1380e0 0x7fd71c13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 1 -- 172.21.15.178:0/3751875692 >> v1:172.21.15.178:6789/0 conn(0x7fd71c0a2930 legacy=0x7fd71c09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 10 monclient: _finish_hunting 0 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 1 monclient: found mon.noname-d 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:41:22.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.713+0000 7fd721cf8640 1 -- 172.21.15.178:0/3751875692 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c1331b0 con 0x7fd71c132de0 2024-09-15T01:41:22.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd721cf8640 1 --2- 172.21.15.178:0/3751875692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd71c132de0 0x7fd71c137ba0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd710009880 tx=0x7fd7100090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=96585c08aca59b5d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd7214f7640 1 -- 172.21.15.178:0/3751875692 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd71002ac50 con 0x7fd71c132de0 2024-09-15T01:41:22.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd7214f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd7214f7640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:41:22.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd7214f7640 10 monclient: dump: 2024-09-15T01:41:22.730 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.730 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.730 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.730 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.730 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.731 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.731 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.731 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.731 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.731 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.731 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.731 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.731 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.731 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.731 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.731 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd7214f7640 10 monclient: _finish_auth 0 2024-09-15T01:41:22.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd7214f7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:22.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd72475d640 5 monclient: authenticate success, global_id 25134 2024-09-15T01:41:22.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd7214f7640 1 -- 172.21.15.178:0/3751875692 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71002add0 con 0x7fd71c132de0 2024-09-15T01:41:22.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd7214f7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:22.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd72475d640 10 monclient: get_monmap_and_config success 2024-09-15T01:41:22.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd7214f7640 1 -- 172.21.15.178:0/3751875692 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd710031ca0 con 0x7fd71c132de0 2024-09-15T01:41:22.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd7214f7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd72475d640 4 set_mon_vals no callback set 2024-09-15T01:41:22.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd7214f7640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:41:22.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.714+0000 7fd7214f7640 10 monclient: dump: 2024-09-15T01:41:22.732 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.732 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.733 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.733 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.733 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.733 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.733 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.733 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.733 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.733 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.733 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.733 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.734 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.734 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.734 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.734 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 10 monclient: shutdown 2024-09-15T01:41:22.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:22.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 1 -- 172.21.15.178:0/3751875692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd71c132de0 msgr2=0x7fd71c137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 1 --2- 172.21.15.178:0/3751875692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd71c132de0 0x7fd71c137ba0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd710009880 tx=0x7fd7100090f0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 1 -- 172.21.15.178:0/3751875692 shutdown_connections 2024-09-15T01:41:22.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 1 --2- 172.21.15.178:0/3751875692 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c1380e0 0x7fd71c13a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 1 --2- 172.21.15.178:0/3751875692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd71c132de0 0x7fd71c137ba0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 1 -- 172.21.15.178:0/3751875692 >> 172.21.15.178:0/3751875692 conn(0x7fd71c0a5ed0 msgr2=0x7fd71c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:22.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 1 -- 172.21.15.178:0/3751875692 shutdown_connections 2024-09-15T01:41:22.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 1 -- 172.21.15.178:0/3751875692 wait complete. 2024-09-15T01:41:22.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 10 monclient: build_initial_monmap 2024-09-15T01:41:22.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 10 monclient: monmap: 2024-09-15T01:41:22.735 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:41:22.735 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:41:22.735 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:41:22.735 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:41:22.735 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:41:22.735 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:41:22.736 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:41:22.736 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:41:22.736 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:41:22.736 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:41:22.736 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:41:22.736 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:41:22.736 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:41:22.736 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:41:22.736 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:41:22.736 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.715+0000 7fd72475d640 1 Processor -- start 2024-09-15T01:41:22.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 1 -- start start 2024-09-15T01:41:22.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 10 monclient: init 2024-09-15T01:41:22.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 10 monclient: _reopen_session rank -1 2024-09-15T01:41:22.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 10 monclient: _add_conns ranks=[2,8,5,3,4,0,1,6,7] 2024-09-15T01:41:22.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd71c0a2930 0x7fd71c09a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 10 monclient(hunting): picked mon.noname-b con 0x7fd71c0a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:41:22.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd71c132de0 0x7fd71c09ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 10 monclient(hunting): picked mon.noname-e con 0x7fd71c132de0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:41:22.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd71c1380e0 0x7fd71c09b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 10 monclient(hunting): picked mon.noname-h con 0x7fd71c1380e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:41:22.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c09d630 con 0x7fd71c0a2930 2024-09-15T01:41:22.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c11d970 con 0x7fd71c1380e0 2024-09-15T01:41:22.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c09c280 con 0x7fd71c132de0 2024-09-15T01:41:22.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 10 monclient(hunting): _renew_subs 2024-09-15T01:41:22.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd72475d640 10 monclient(hunting): authenticate will time out at 1287.467512s 2024-09-15T01:41:22.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd71c0a2930 0x7fd71c09a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd71c0a2930 0x7fd71c09a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:58300/0 (socket says 172.21.15.178:58300) 2024-09-15T01:41:22.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd721cf8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd71c132de0 0x7fd71c09ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 1 -- 172.21.15.178:0/3913911966 learned_addr learned my addr 172.21.15.178:0/3913911966 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:41:22.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 10 monclient(hunting): get_auth_request con 0x7fd71c0a2930 auth_method 0 2024-09-15T01:41:22.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd721cf8640 10 monclient(hunting): get_auth_request con 0x7fd71c132de0 auth_method 0 2024-09-15T01:41:22.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd721cf8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd721cf8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd721cf8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd721cf8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd721cf8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd722cfa640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd71c1380e0 0x7fd71c09b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd721cf8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd722cfa640 10 monclient(hunting): get_auth_request con 0x7fd71c1380e0 auth_method 0 2024-09-15T01:41:22.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd722cfa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd722cfa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd722cfa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd722cfa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd722cfa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd722cfa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 10 monclient(hunting): handle_auth_done global_id 34580 payload 824 2024-09-15T01:41:22.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 1 -- 172.21.15.178:0/3913911966 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd71c132de0 msgr2=0x7fd71c09ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd71c132de0 0x7fd71c09ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 1 -- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd71c1380e0 msgr2=0x7fd71c09b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd71c1380e0 0x7fd71c09b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 10 monclient: _finish_hunting 0 2024-09-15T01:41:22.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 1 monclient: found mon.noname-b 2024-09-15T01:41:22.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:41:22.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:41:22.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.716+0000 7fd7224f9640 1 -- 172.21.15.178:0/3913911966 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710009390 con 0x7fd71c0a2930 2024-09-15T01:41:22.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd7224f9640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd71c0a2930 0x7fd71c09a840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c4b0 tx=0x7fd714007d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7140070d0 con 0x7fd71c0a2930 2024-09-15T01:41:22.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd71a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd71a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:41:22.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd71a7fc640 10 monclient: dump: 2024-09-15T01:41:22.742 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.742 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.742 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.743 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd71a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd71a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd714007290 con 0x7fd71c0a2930 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd72475d640 5 monclient: authenticate success, global_id 34580 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd72475d640 10 monclient: _renew_subs 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd72475d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd72475d640 1 -- 172.21.15.178:0/3913911966 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c11d690 con 0x7fd71c0a2930 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd71a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd714011360 con 0x7fd71c0a2930 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd71a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd71a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd71a7fc640 10 monclient: dump: 2024-09-15T01:41:22.744 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.745 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.745 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.745 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.745 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.745 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.745 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.745 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.745 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd72475d640 10 monclient: _renew_subs 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd720cf6640 4 set_mon_vals no callback set 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd72475d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd72475d640 1 -- 172.21.15.178:0/3913911966 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c1326e0 con 0x7fd71c0a2930 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd72475d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd72475d640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c09b800 0x7fd71c096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd72475d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.717+0000 7fd72475d640 1 -- 172.21.15.178:0/3913911966 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c04c4a0 con 0x7fd71c09b800 2024-09-15T01:41:22.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd714007400 con 0x7fd71c0a2930 2024-09-15T01:41:22.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd71a7fc640 1 --2- 172.21.15.178:0/3913911966 >> v2:172.21.15.204:6812/28210 conn(0x7fd7080397b0 0x7fd70803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd71404ad30 con 0x7fd71c0a2930 2024-09-15T01:41:22.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd721cf8640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c09b800 0x7fd71c096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd721cf8640 10 monclient: get_auth_request con 0x7fd71c09b800 auth_method 0 2024-09-15T01:41:22.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd721cf8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd721cf8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd721cf8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd722cfa640 1 --2- 172.21.15.178:0/3913911966 >> v2:172.21.15.204:6812/28210 conn(0x7fd7080397b0 0x7fd70803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd721cf8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd721cf8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd721cf8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd722cfa640 10 monclient: get_auth_request con 0x7fd7080397b0 auth_method 0 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd721cf8640 10 monclient(hunting): handle_auth_done global_id 44233 payload 824 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd721cf8640 1 -- 172.21.15.178:0/3913911966 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd71c05ed70 con 0x7fd71c09b800 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd722cfa640 1 --2- 172.21.15.178:0/3913911966 >> v2:172.21.15.204:6812/28210 conn(0x7fd7080397b0 0x7fd70803bc60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0a3a90 tx=0x7fd70c009850 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.718+0000 7fd721cf8640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c09b800 0x7fd71c096a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0a2f70 tx=0x7fd710032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.719+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd71002ac50 con 0x7fd71c09b800 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.719+0000 7fd71a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.719+0000 7fd71a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.719+0000 7fd71a7fc640 10 monclient: dump: 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.748 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.749 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.719+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fd710031a40 con 0x7fd71c09b800 2024-09-15T01:41:22.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.719+0000 7fd71a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:41:22.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.719+0000 7fd71a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:41:22.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.719+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c09b800 msgr2=0x7fd71c096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.719+0000 7fd71a7fc640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c09b800 0x7fd71c096a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0a2f70 tx=0x7fd710032000 comp rx=0 tx=0).stop 2024-09-15T01:41:22.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.719+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c09b800 msgr2=0x7fd71c096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:22.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.719+0000 7fd71a7fc640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c09b800 0x7fd71c096a30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.724+0000 7fd72475d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:41:22.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.724+0000 7fd72475d640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c096f70 0x7fd71c097370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:41:22.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd72475d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:41:22.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd72475d640 1 -- 172.21.15.178:0/3913911966 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c04bc80 con 0x7fd71c096f70 2024-09-15T01:41:22.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd721cf8640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c096f70 0x7fd71c097370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:41:22.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd721cf8640 10 monclient: get_auth_request con 0x7fd71c096f70 auth_method 0 2024-09-15T01:41:22.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd721cf8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd721cf8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd721cf8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd721cf8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd721cf8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd721cf8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd721cf8640 10 monclient(hunting): handle_auth_done global_id 44242 payload 824 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd721cf8640 1 -- 172.21.15.178:0/3913911966 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd71c132960 con 0x7fd71c096f70 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd721cf8640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c096f70 0x7fd71c097370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd71c04c5e0 tx=0x7fd71003d000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd71003dc60 con 0x7fd71c096f70 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd71a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd71a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.725+0000 7fd71a7fc640 10 monclient: dump: 2024-09-15T01:41:22.753 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:41:22.754 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:41:22.755 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:41:22.755 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:41:22.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9131 (secure 0 0 0) 0x7fd71003ddc0 con 0x7fd71c096f70 2024-09-15T01:41:22.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd71a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:41:22.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd71a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:41:22.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c096f70 msgr2=0x7fd71c097370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd71a7fc640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c096f70 0x7fd71c097370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd71c04c5e0 tx=0x7fd71003d000 comp rx=0 tx=0).stop 2024-09-15T01:41:22.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd71a7fc640 1 -- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c096f70 msgr2=0x7fd71c097370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:41:22.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd71a7fc640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd71c096f70 0x7fd71c097370 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.755 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 114, 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:41:22.756 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.757 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:41:22.759 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:41:22.760 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:41:22.761 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.763 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:41:22.763 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:41:22.763 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.763 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:41:22.763 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.763 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:41:22.764 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.764 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.764 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:41:22.764 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:41:22.764 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.764 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.764 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.765 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:41:22.768 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:41:22.768 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.768 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.768 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.768 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:41:22.768 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:41:22.768 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.769 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.769 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:41:22.769 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.769 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.769 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.769 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:41:22.769 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.769 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.769 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:41:22.770 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:41:22.770 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.770 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.770 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.770 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:41:22.770 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:41:22.770 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.771 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.771 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:41:22.771 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.771 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.771 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.771 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:41:22.771 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.771 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.771 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:41:22.772 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:41:22.772 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.772 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.772 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.772 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:41:22.772 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:41:22.772 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.773 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:41:22.773 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.773 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:41:22.773 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.773 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.773 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:41:22.774 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:41:22.774 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.774 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.774 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.774 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:41:22.774 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:41:22.775 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.775 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:41:22.775 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.775 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:41:22.775 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.776 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.776 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:41:22.776 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:41:22.776 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.776 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.776 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.777 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:41:22.777 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:41:22.777 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:41:22.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:41:22.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.777 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:41:22.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:41:22.777 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.778 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:41:22.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:41:22.778 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:41:22.778 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.778 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:41:22.778 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:41:22.779 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:41:22.779 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:41:22.779 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:41:22.779 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.779 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.779 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:41:22.779 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:41:22.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.779 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.779 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.779 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:22.780 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.780 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:41:22.780 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:41:22.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:41:22.780 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:41:22.780 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:41:22.780 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:41:22.780 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:41:22.780 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:41:22.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:41:22.780 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:41:22.780 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:41:22.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd72475d640 1 -- 172.21.15.178:0/3913911966 >> v2:172.21.15.204:6812/28210 conn(0x7fd7080397b0 msgr2=0x7fd70803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd72475d640 1 --2- 172.21.15.178:0/3913911966 >> v2:172.21.15.204:6812/28210 conn(0x7fd7080397b0 0x7fd70803bc60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0a3a90 tx=0x7fd70c009850 comp rx=0 tx=0).stop 2024-09-15T01:41:22.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd72475d640 10 monclient: shutdown 2024-09-15T01:41:22.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd72475d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:41:22.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd72475d640 1 -- 172.21.15.178:0/3913911966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd71c0a2930 msgr2=0x7fd71c09a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:41:22.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd72475d640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd71c0a2930 0x7fd71c09a840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c4b0 tx=0x7fd714007d60 comp rx=0 tx=0).stop 2024-09-15T01:41:22.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd722cfa640 1 -- 172.21.15.178:0/3913911966 reap_dead start 2024-09-15T01:41:22.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd72475d640 1 -- 172.21.15.178:0/3913911966 shutdown_connections 2024-09-15T01:41:22.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd72475d640 1 --2- 172.21.15.178:0/3913911966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd71c0a2930 0x7fd71c09a840 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:41:22.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd72475d640 1 -- 172.21.15.178:0/3913911966 >> 172.21.15.178:0/3913911966 conn(0x7fd71c0a5ed0 msgr2=0x7fd71c09fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:41:22.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd72475d640 1 -- 172.21.15.178:0/3913911966 shutdown_connections 2024-09-15T01:41:22.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:41:22.726+0000 7fd72475d640 1 -- 172.21.15.178:0/3913911966 wait complete. 2024-09-15T01:41:22.783 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T01:41:22.783 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T01:42:52.784 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T01:42:52.785 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-15T01:42:52.785 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-15T01:42:52.785 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-15T01:42:52.789 INFO:tasks.ceph.mon.f:Started 2024-09-15T01:42:52.789 INFO:tasks.mon_thrash:Sending STOP to mon e 2024-09-15T01:42:52.790 INFO:tasks.ceph.mon.e:Sent signal 19 2024-09-15T01:42:52.790 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T01:43:07.791 INFO:tasks.mon_thrash:Sending CONT to mon e 2024-09-15T01:43:07.791 INFO:tasks.ceph.mon.e:Sent signal 18 2024-09-15T01:43:07.791 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:43:07.792 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:43:07.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.894+0000 7fbd70d9d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:07.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.894+0000 7fbd70d9d640 10 monclient: build_initial_monmap 2024-09-15T01:43:07.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.894+0000 7fbd70d9d640 10 monclient: monmap: 2024-09-15T01:43:07.897 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:07.897 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:07.897 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:07.895364+0000 2024-09-15T01:43:07.897 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:07.895364+0000 2024-09-15T01:43:07.897 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:07.897 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:07.897 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:07.897 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 1 Processor -- start 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 1 -- start start 2024-09-15T01:43:07.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 10 monclient: init 2024-09-15T01:43:07.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:07.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 10 monclient: _add_conns ranks=[1,5,0,4,8,10,2,6,3,7,9] 2024-09-15T01:43:07.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbd6c0a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:07.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:07.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 10 monclient(hunting): picked mon.noname-h con 0x7fbd6c132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:07.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbd6c1380e0 addr v1:172.21.15.178:6789/0 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6c04bf60 con 0x7fbd6c1380e0 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6c09dd80 con 0x7fbd6c0a2930 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0a2d00 con 0x7fbd6c132de0 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd70d9d640 10 monclient(hunting): authenticate will time out at 1392.645837s 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd6ad9d640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fbd6c1380e0 0x7fbd6c13a4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:41584/0 (socket says 172.21.15.178:41584) 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.895+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 learned_addr learned my addr 172.21.15.178:0/1822019688 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.896+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.896+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request con 0x7fbd6c132de0 auth_method 0 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.896+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.896+0000 7fbd69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.896+0000 7fbd69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:07.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.896+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 msgr2=0x7fbd6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:07.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.896+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 msgr2=0x7fbd6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:07.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.896+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:07.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:07.896+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:08.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.097+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:08.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.097+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request con 0x7fbd6c132de0 auth_method 0 2024-09-15T01:43:08.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.097+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:08.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.097+0000 7fbd69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:08.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.097+0000 7fbd69d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:08.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.097+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 msgr2=0x7fbd6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:08.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.097+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 msgr2=0x7fbd6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:08.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.097+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:08.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.097+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:08.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.498+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:08.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.498+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request con 0x7fbd6c132de0 auth_method 0 2024-09-15T01:43:08.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.498+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:08.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.498+0000 7fbd69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:08.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.498+0000 7fbd69d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:08.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.498+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 msgr2=0x7fbd6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:08.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.498+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 msgr2=0x7fbd6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:08.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.498+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:08.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:08.498+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:09.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:09.299+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:09.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:09.299+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request con 0x7fbd6c132de0 auth_method 0 2024-09-15T01:43:09.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:09.299+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:09.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:09.299+0000 7fbd69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:09.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:09.299+0000 7fbd69d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:09.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:09.300+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 msgr2=0x7fbd6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:09.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:09.300+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 msgr2=0x7fbd6c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:09.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:09.300+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:09.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:09.300+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:10.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.895+0000 7fbd5bfff640 10 monclient(hunting): tick 2024-09-15T01:43:10.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.895+0000 7fbd5bfff640 1 monclient(hunting): continuing hunt 2024-09-15T01:43:10.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.895+0000 7fbd5bfff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:43:10.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.895+0000 7fbd5bfff640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 msgr2=0x7fbd6c137ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:10.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.895+0000 7fbd5bfff640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbd6c132de0 0x7fbd6c137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:10.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.895+0000 7fbd5bfff640 1 -- 172.21.15.178:0/1822019688 >> v1:172.21.15.204:6789/0 conn(0x7fbd6c0a2930 legacy=0x7fbd6c09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:10.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 1 -- 172.21.15.178:0/1822019688 >> v1:172.21.15.178:6789/0 conn(0x7fbd6c1380e0 legacy=0x7fbd6c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:10.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 10 monclient: _add_conns ranks=[3,8,6,9,2,7,4,10] 2024-09-15T01:43:10.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:10.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 10 monclient(hunting): picked mon.noname-g con 0x7fbd4c0037e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:10.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:10.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 10 monclient(hunting): picked mon.noname-e con 0x7fbd4c006310 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:10.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:10.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 10 monclient(hunting): picked mon.noname-d con 0x7fbd4c008e80 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:10.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:10.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 1 -- 172.21.15.178:0/1822019688 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0a2d00 con 0x7fbd4c0037e0 2024-09-15T01:43:10.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:10.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 1 -- 172.21.15.178:0/1822019688 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c00ba20 con 0x7fbd4c008e80 2024-09-15T01:43:10.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 1 -- 172.21.15.178:0/1822019688 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c00bbb0 con 0x7fbd4c006310 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd5bfff640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request con 0x7fbd4c006310 auth_method 0 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6ad9d640 10 monclient(hunting): get_auth_request con 0x7fbd4c008e80 auth_method 0 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:10.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:10.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6a59c640 10 monclient(hunting): get_auth_request con 0x7fbd4c0037e0 auth_method 0 2024-09-15T01:43:10.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:10.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:10.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:10.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 msgr2=0x7fbd4c00b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:10.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 msgr2=0x7fbd4c008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:10.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 msgr2=0x7fbd4c00b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:10.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 msgr2=0x7fbd4c008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:10.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:10.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:10.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:10.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6a59c640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 msgr2=0x7fbd4c005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:10.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:10.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6a59c640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 msgr2=0x7fbd4c005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:10.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:10.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:10.896+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:11.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:11.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:11.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6ad9d640 10 monclient(hunting): get_auth_request con 0x7fbd4c008e80 auth_method 0 2024-09-15T01:43:11.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:11.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:11.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:11.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request con 0x7fbd4c006310 auth_method 0 2024-09-15T01:43:11.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:11.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd69d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 msgr2=0x7fbd4c00b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 msgr2=0x7fbd4c00b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 msgr2=0x7fbd4c008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 msgr2=0x7fbd4c008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6a59c640 10 monclient(hunting): get_auth_request con 0x7fbd4c0037e0 auth_method 0 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:11.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:11.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:11.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6a59c640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 msgr2=0x7fbd4c005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:11.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6a59c640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 msgr2=0x7fbd4c005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:11.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:11.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.097+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:11.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:11.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request con 0x7fbd4c006310 auth_method 0 2024-09-15T01:43:11.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:11.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:11.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd69d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:11.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:11.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6ad9d640 10 monclient(hunting): get_auth_request con 0x7fbd4c008e80 auth_method 0 2024-09-15T01:43:11.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:11.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 msgr2=0x7fbd4c008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 msgr2=0x7fbd4c008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 msgr2=0x7fbd4c00b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 msgr2=0x7fbd4c00b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6a59c640 10 monclient(hunting): get_auth_request con 0x7fbd4c0037e0 auth_method 0 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:11.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:11.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:11.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6a59c640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 msgr2=0x7fbd4c005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:11.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6a59c640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 msgr2=0x7fbd4c005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:11.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:11.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:11.498+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:12.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:12.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request con 0x7fbd4c006310 auth_method 0 2024-09-15T01:43:12.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:12.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:12.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd69d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:12.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:12.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6ad9d640 10 monclient(hunting): get_auth_request con 0x7fbd4c008e80 auth_method 0 2024-09-15T01:43:12.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:12.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:12.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:12.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 msgr2=0x7fbd4c008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:12.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd69d9b640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 msgr2=0x7fbd4c008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:12.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:12.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:12.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 msgr2=0x7fbd4c00b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:12.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 msgr2=0x7fbd4c00b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:12.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:12.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:12.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:12.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6a59c640 10 monclient(hunting): get_auth_request con 0x7fbd4c0037e0 auth_method 0 2024-09-15T01:43:12.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:12.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:12.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:12.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6a59c640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 msgr2=0x7fbd4c005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:12.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6a59c640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 msgr2=0x7fbd4c005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:12.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:12.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:12.299+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 10 monclient(hunting): tick 2024-09-15T01:43:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 monclient(hunting): continuing hunt 2024-09-15T01:43:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:43:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 msgr2=0x7fbd4c008790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006310 0x7fbd4c008790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 msgr2=0x7fbd4c00b2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbd4c008e80 0x7fbd4c00b2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 msgr2=0x7fbd4c005c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd4c0037e0 0x7fbd4c005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 reap_dead start 2024-09-15T01:43:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 10 monclient: _add_conns ranks=[7,2,9,4,10] 2024-09-15T01:43:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 reap_dead start 2024-09-15T01:43:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbd4c003c10 0x7fbd4c006780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 10 monclient(hunting): picked mon.noname-i con 0x7fbd4c003c10 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd4c006cc0 0x7fbd4c00b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 10 monclient(hunting): picked mon.noname-b con 0x7fbd4c006cc0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 --2- 172.21.15.178:0/1822019688 >> v2:172.21.15.178:3300/0 conn(0x7fbd4c007130 0x7fbd4c0075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 10 monclient(hunting): picked mon.noname-a con 0x7fbd4c007130 addr v2:172.21.15.178:3300/0 2024-09-15T01:43:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 -- 172.21.15.178:0/1822019688 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c0a2d00 con 0x7fbd4c007130 2024-09-15T01:43:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 -- 172.21.15.178:0/1822019688 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c00ba20 con 0x7fbd4c006cc0 2024-09-15T01:43:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 1 -- 172.21.15.178:0/1822019688 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c00bbb0 con 0x7fbd4c003c10 2024-09-15T01:43:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd5bfff640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> v2:172.21.15.178:3300/0 conn(0x7fbd4c007130 0x7fbd4c0075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd4c006cc0 0x7fbd4c00b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6ad9d640 10 monclient(hunting): get_auth_request con 0x7fbd4c007130 auth_method 0 2024-09-15T01:43:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:13.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:13.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:13.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request con 0x7fbd4c006cc0 auth_method 0 2024-09-15T01:43:13.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:13.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:13.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:13.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbd4c003c10 0x7fbd4c006780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:13.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6a59c640 10 monclient(hunting): get_auth_request con 0x7fbd4c003c10 auth_method 0 2024-09-15T01:43:13.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:13.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:13.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:13.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd69d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd69d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd69d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.896+0000 7fbd6a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.897+0000 7fbd6ad9d640 10 monclient(hunting): handle_auth_done global_id 34776 payload 274 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.897+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbd4c003c10 msgr2=0x7fbd4c006780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.897+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbd4c003c10 0x7fbd4c006780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.897+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd4c006cc0 msgr2=0x7fbd4c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:13.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.897+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd4c006cc0 0x7fbd4c00b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.897+0000 7fbd6ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:43:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.897+0000 7fbd6ad9d640 1 monclient: found mon.noname-a 2024-09-15T01:43:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.897+0000 7fbd6ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.897+0000 7fbd6ad9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:43:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.897+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/1822019688 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c1331b0 con 0x7fbd4c007130 2024-09-15T01:43:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.897+0000 7fbd69d9b640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd4c006cc0 0x7fbd4c00b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.897+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbd4c003c10 0x7fbd4c006780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd4c007130 0x7fbd4c0075a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0a1e30 tx=0x7fbd64009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de024dfdda96f9a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6959a640 1 -- 172.21.15.178:0/1822019688 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd6402c8d0 con 0x7fbd4c007130 2024-09-15T01:43:13.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:43:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6959a640 10 monclient: dump: 2024-09-15T01:43:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:13.908 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6959a640 10 monclient: _finish_auth 0 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6959a640 1 -- 172.21.15.178:0/1822019688 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6402ca70 con 0x7fbd4c007130 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd70d9d640 5 monclient: authenticate success, global_id 34776 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd70d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd70d9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:13.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd70d9d640 4 set_mon_vals no callback set 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6959a640 1 -- 172.21.15.178:0/1822019688 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd6402cbe0 con 0x7fbd4c007130 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.898+0000 7fbd6959a640 10 monclient: dump: 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:13.910 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.899+0000 7fbd70d9d640 10 monclient: shutdown 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.899+0000 7fbd70d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.899+0000 7fbd70d9d640 1 -- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd4c007130 msgr2=0x7fbd4c0075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.899+0000 7fbd70d9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd4c007130 0x7fbd4c0075a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0a1e30 tx=0x7fbd64009040 comp rx=0 tx=0).stop 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.899+0000 7fbd70d9d640 1 -- 172.21.15.178:0/1822019688 shutdown_connections 2024-09-15T01:43:13.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.899+0000 7fbd70d9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd4c007130 0x7fbd4c0075a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:13.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.899+0000 7fbd70d9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd4c006cc0 0x7fbd4c00b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:13.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.899+0000 7fbd70d9d640 1 --2- 172.21.15.178:0/1822019688 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbd4c003c10 0x7fbd4c006780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:13.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.899+0000 7fbd70d9d640 1 -- 172.21.15.178:0/1822019688 >> 172.21.15.178:0/1822019688 conn(0x7fbd6c0a5ed0 msgr2=0x7fbd6c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:13.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.899+0000 7fbd70d9d640 1 -- 172.21.15.178:0/1822019688 shutdown_connections 2024-09-15T01:43:13.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.899+0000 7fbd70d9d640 1 -- 172.21.15.178:0/1822019688 wait complete. 2024-09-15T01:43:13.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.900+0000 7fbd70d9d640 10 monclient: build_initial_monmap 2024-09-15T01:43:13.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.900+0000 7fbd70d9d640 10 monclient: monmap: 2024-09-15T01:43:13.912 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:13.912 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:13.912 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:13.912 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:13.912 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.900+0000 7fbd70d9d640 1 Processor -- start 2024-09-15T01:43:13.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.900+0000 7fbd70d9d640 1 -- start start 2024-09-15T01:43:13.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.900+0000 7fbd70d9d640 10 monclient: init 2024-09-15T01:43:13.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:13.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 10 monclient: _add_conns ranks=[4,8,2,5,0,6,3,1,7] 2024-09-15T01:43:13.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbd4c003c10 0x7fbd6c09a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:13.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 10 monclient(hunting): picked mon.noname-c con 0x7fbd4c003c10 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:13.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006cc0 0x7fbd6c09ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:13.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 10 monclient(hunting): picked mon.noname-e con 0x7fbd4c006cc0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:13.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd4c007130 0x7fbd6c09b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:13.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 10 monclient(hunting): picked mon.noname-b con 0x7fbd4c007130 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:13.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c11e190 con 0x7fbd4c007130 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c131aa0 con 0x7fbd4c003c10 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c11d4d0 con 0x7fbd4c006cc0 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd70d9d640 10 monclient(hunting): authenticate will time out at 1398.651151s 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6a59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbd4c003c10 0x7fbd6c09a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd69d9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006cc0 0x7fbd6c09ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd4c007130 0x7fbd6c09b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6a59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbd4c003c10 0x7fbd6c09a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:32782/0 (socket says 172.21.15.178:32782) 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6a59c640 1 -- 172.21.15.178:0/3905111847 learned_addr learned my addr 172.21.15.178:0/3905111847 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:13.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 10 monclient(hunting): get_auth_request con 0x7fbd4c007130 auth_method 0 2024-09-15T01:43:13.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:13.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:13.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:13.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request con 0x7fbd4c006cc0 auth_method 0 2024-09-15T01:43:13.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd69d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:13.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd69d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:13.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd69d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:13.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6a59c640 10 monclient(hunting): get_auth_request con 0x7fbd4c003c10 auth_method 0 2024-09-15T01:43:13.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:13.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:13.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:13.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:13.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:13.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:13.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd69d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:13.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd69d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:13.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd69d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:13.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:13.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:13.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:13.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 10 monclient(hunting): handle_auth_done global_id 34598 payload 824 2024-09-15T01:43:13.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/3905111847 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006cc0 msgr2=0x7fbd6c09ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:13.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/3905111847 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006cc0 0x7fbd6c09ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:13.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/3905111847 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbd4c003c10 msgr2=0x7fbd6c09a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:13.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/3905111847 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbd4c003c10 0x7fbd6c09a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:13.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:43:13.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 1 monclient: found mon.noname-b 2024-09-15T01:43:13.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:13.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:13.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 1 -- 172.21.15.178:0/3905111847 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c00f040 con 0x7fbd4c007130 2024-09-15T01:43:13.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/3905111847 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbd4c003c10 0x7fbd6c09a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:13.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.901+0000 7fbd6ad9d640 1 --2- 172.21.15.178:0/3905111847 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd4c007130 0x7fbd6c09b040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c140 tx=0x7fbd64039000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:13.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5a7fc640 1 -- 172.21.15.178:0/3905111847 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd64004d30 con 0x7fbd4c007130 2024-09-15T01:43:13.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:13.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:13.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5a7fc640 10 monclient: dump: 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:13.923 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5a7fc640 1 -- 172.21.15.178:0/3905111847 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd640054d0 con 0x7fbd4c007130 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd70d9d640 5 monclient: authenticate success, global_id 34598 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd70d9d640 10 monclient: _renew_subs 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd70d9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd70d9d640 1 -- 172.21.15.178:0/3905111847 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6000e040 con 0x7fbd4c007130 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5a7fc640 1 -- 172.21.15.178:0/3905111847 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd64038430 con 0x7fbd4c007130 2024-09-15T01:43:13.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5a7fc640 10 monclient: dump: 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:13.925 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd5bfff640 4 set_mon_vals no callback set 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd70d9d640 10 monclient: _renew_subs 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd70d9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.902+0000 7fbd70d9d640 1 -- 172.21.15.178:0/3905111847 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6400f040 con 0x7fbd4c007130 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.903+0000 7fbd5a7fc640 1 -- 172.21.15.178:0/3905111847 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fbd64005050 con 0x7fbd4c007130 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.903+0000 7fbd70d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.903+0000 7fbd70d9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.903+0000 7fbd70d9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:13.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.903+0000 7fbd70d9d640 1 -- 172.21.15.178:0/3905111847 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6c04b720 con 0x7fbd4c007130 2024-09-15T01:43:13.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.903+0000 7fbd5a7fc640 1 --2- 172.21.15.178:0/3905111847 >> v2:172.21.15.204:6812/28210 conn(0x7fbd4c03ba80 0x7fbd4c03df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:13.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.903+0000 7fbd5a7fc640 1 -- 172.21.15.178:0/3905111847 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fbd64033030 con 0x7fbd4c007130 2024-09-15T01:43:13.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.904+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/3905111847 >> v2:172.21.15.204:6812/28210 conn(0x7fbd4c03ba80 0x7fbd4c03df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:13.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.904+0000 7fbd6a59c640 10 monclient: get_auth_request con 0x7fbd4c03ba80 auth_method 0 2024-09-15T01:43:13.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.904+0000 7fbd6a59c640 1 --2- 172.21.15.178:0/3905111847 >> v2:172.21.15.204:6812/28210 conn(0x7fbd4c03ba80 0x7fbd4c03df30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0185f0 tx=0x7fbd5c007f90 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:13.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.908+0000 7fbd5a7fc640 1 -- 172.21.15.178:0/3905111847 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fbd64005300 con 0x7fbd4c007130 2024-09-15T01:43:13.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.908+0000 7fbd5a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:13.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:13.908+0000 7fbd5a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:14.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.126+0000 7fbd70d9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:43:14.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.126+0000 7fbd70d9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:43:14.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.126+0000 7fbd70d9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:14.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.126+0000 7fbd70d9d640 1 -- 172.21.15.178:0/3905111847 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbd6c0a19b0 con 0x7fbd4c007130 2024-09-15T01:43:14.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.127+0000 7fbd5a7fc640 1 -- 172.21.15.178:0/3905111847 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7fbd6402de60 con 0x7fbd4c007130 2024-09-15T01:43:14.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.127+0000 7fbd5a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:43:14.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.127+0000 7fbd5a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:14.129 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":122,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":1,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:43:14.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 1 -- 172.21.15.178:0/3905111847 >> v2:172.21.15.204:6812/28210 conn(0x7fbd4c03ba80 msgr2=0x7fbd4c03df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 1 --2- 172.21.15.178:0/3905111847 >> v2:172.21.15.204:6812/28210 conn(0x7fbd4c03ba80 0x7fbd4c03df30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0185f0 tx=0x7fbd5c007f90 comp rx=0 tx=0).stop 2024-09-15T01:43:14.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 10 monclient: shutdown 2024-09-15T01:43:14.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:14.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 1 -- 172.21.15.178:0/3905111847 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd4c007130 msgr2=0x7fbd6c09b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 1 --2- 172.21.15.178:0/3905111847 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd4c007130 0x7fbd6c09b040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c140 tx=0x7fbd64039000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 1 -- 172.21.15.178:0/3905111847 shutdown_connections 2024-09-15T01:43:14.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 1 --2- 172.21.15.178:0/3905111847 >> v2:172.21.15.204:6812/28210 conn(0x7fbd4c03ba80 0x7fbd4c03df30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 1 --2- 172.21.15.178:0/3905111847 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd4c007130 0x7fbd6c09b040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 1 --2- 172.21.15.178:0/3905111847 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbd4c006cc0 0x7fbd6c09ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 1 --2- 172.21.15.178:0/3905111847 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbd4c003c10 0x7fbd6c09a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 1 -- 172.21.15.178:0/3905111847 >> 172.21.15.178:0/3905111847 conn(0x7fbd6c0a5ed0 msgr2=0x7fbd6c0a48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:14.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 1 -- 172.21.15.178:0/3905111847 shutdown_connections 2024-09-15T01:43:14.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.129+0000 7fbd70d9d640 1 -- 172.21.15.178:0/3905111847 wait complete. 2024-09-15T01:43:14.141 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:43:14.142 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:43:14.142 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:43:14.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.233+0000 7f50e095d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:14.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.233+0000 7f50e095d640 10 monclient: build_initial_monmap 2024-09-15T01:43:14.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.233+0000 7f50e095d640 10 monclient: monmap: 2024-09-15T01:43:14.235 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:14.235 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.235 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:14.234011+0000 2024-09-15T01:43:14.235 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:14.234011+0000 2024-09-15T01:43:14.235 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:14.235 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.235 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:14.236 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:14.236 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:14.236 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:14.236 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:14.236 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:14.236 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:14.236 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:14.236 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:14.236 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:14.236 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:14.236 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.233+0000 7f50e095d640 1 Processor -- start 2024-09-15T01:43:14.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.233+0000 7f50e095d640 1 -- start start 2024-09-15T01:43:14.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.233+0000 7f50e095d640 10 monclient: init 2024-09-15T01:43:14.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 10 monclient: _add_conns ranks=[1,5,2,10,7,6,8,3,9,0,4] 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f50dc0a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f50dc132de0 0x7f50dc137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 10 monclient(hunting): picked mon.noname-h con 0x7f50dc132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc1380e0 0x7f50dc13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 10 monclient(hunting): picked mon.noname-b con 0x7f50dc1380e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50dc04bf60 con 0x7f50dc0a2930 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc09dd80 con 0x7f50dc1380e0 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0a2d00 con 0x7f50dc132de0 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50e095d640 10 monclient(hunting): authenticate will time out at 1398.984405s 2024-09-15T01:43:14.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc1380e0 0x7f50dc13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc1380e0 0x7f50dc13a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:48212/0 (socket says 172.21.15.178:48212) 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 1 -- 172.21.15.178:0/1500125451 learned_addr learned my addr 172.21.15.178:0/1500125451 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 10 monclient(hunting): get_auth_request con 0x7f50dc1380e0 auth_method 0 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50d9d9b640 1 --2- 172.21.15.178:0/1500125451 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f50dc132de0 0x7f50dc137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50d9d9b640 10 monclient(hunting): get_auth_request con 0x7f50dc132de0 auth_method 0 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50d9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50d9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50d9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 10 monclient(hunting): handle_auth_done global_id 34607 payload 274 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 1 -- 172.21.15.178:0/1500125451 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f50dc132de0 msgr2=0x7f50dc137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 1 --2- 172.21.15.178:0/1500125451 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f50dc132de0 0x7f50dc137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 1 -- 172.21.15.178:0/1500125451 >> v1:172.21.15.204:6789/0 conn(0x7f50dc0a2930 legacy=0x7f50dc09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 1 monclient: found mon.noname-b 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.234+0000 7f50dad9d640 1 -- 172.21.15.178:0/1500125451 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc1331b0 con 0x7f50dc1380e0 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50dad9d640 1 --2- 172.21.15.178:0/1500125451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc1380e0 0x7f50dc13a4c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009940 tx=0x7f50d40091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e72bec304c94e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50d959a640 1 -- 172.21.15.178:0/1500125451 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50d402ad00 con 0x7f50dc1380e0 2024-09-15T01:43:14.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50d959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50d959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50d959a640 10 monclient: dump: 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.249 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50d959a640 10 monclient: _finish_auth 0 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50d959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50d959a640 1 -- 172.21.15.178:0/1500125451 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d402ae80 con 0x7f50dc1380e0 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50e095d640 5 monclient: authenticate success, global_id 34607 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50e095d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50d959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50d959a640 1 -- 172.21.15.178:0/1500125451 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50d4031ca0 con 0x7f50dc1380e0 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50e095d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:14.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50e095d640 4 set_mon_vals no callback set 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50d959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50d959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.235+0000 7f50d959a640 10 monclient: dump: 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.251 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.236+0000 7f50e095d640 10 monclient: shutdown 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.236+0000 7f50e095d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.236+0000 7f50e095d640 1 -- 172.21.15.178:0/1500125451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc1380e0 msgr2=0x7f50dc13a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.236+0000 7f50e095d640 1 --2- 172.21.15.178:0/1500125451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc1380e0 0x7f50dc13a4c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009940 tx=0x7f50d40091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.236+0000 7f50e095d640 1 -- 172.21.15.178:0/1500125451 shutdown_connections 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.236+0000 7f50e095d640 1 --2- 172.21.15.178:0/1500125451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc1380e0 0x7f50dc13a4c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.236+0000 7f50e095d640 1 --2- 172.21.15.178:0/1500125451 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f50dc132de0 0x7f50dc137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.236+0000 7f50e095d640 1 -- 172.21.15.178:0/1500125451 >> 172.21.15.178:0/1500125451 conn(0x7f50dc0a5ed0 msgr2=0x7f50dc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:14.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.236+0000 7f50e095d640 1 -- 172.21.15.178:0/1500125451 shutdown_connections 2024-09-15T01:43:14.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.236+0000 7f50e095d640 1 -- 172.21.15.178:0/1500125451 wait complete. 2024-09-15T01:43:14.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.236+0000 7f50e095d640 10 monclient: build_initial_monmap 2024-09-15T01:43:14.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.236+0000 7f50e095d640 10 monclient: monmap: 2024-09-15T01:43:14.253 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:14.253 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:14.253 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:14.253 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:14.253 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:14.253 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:14.253 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:14.253 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 1 Processor -- start 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 1 -- start start 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 10 monclient: init 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 10 monclient: _add_conns ranks=[0,2,3,7,8,1,5,6,4] 2024-09-15T01:43:14.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc0a2930 0x7f50dc09a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 10 monclient(hunting): picked mon.noname-a con 0x7f50dc0a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc132de0 0x7f50dc09ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 10 monclient(hunting): picked mon.noname-b con 0x7f50dc132de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50dc1380e0 0x7f50dc09b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 10 monclient(hunting): picked mon.noname-g con 0x7f50dc1380e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc09d630 con 0x7f50dc0a2930 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc131b90 con 0x7f50dc132de0 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc11d540 con 0x7f50dc1380e0 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:14.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50e095d640 10 monclient(hunting): authenticate will time out at 1398.987732s 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50d9d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc132de0 0x7f50dc09ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50da59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc0a2930 0x7f50dc09a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50d9d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc132de0 0x7f50dc09ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:48222/0 (socket says 172.21.15.178:48222) 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50d9d9b640 1 -- 172.21.15.178:0/3106522520 learned_addr learned my addr 172.21.15.178:0/3106522520 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50d9d9b640 10 monclient(hunting): get_auth_request con 0x7f50dc132de0 auth_method 0 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50d9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50d9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50d9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50da59c640 10 monclient(hunting): get_auth_request con 0x7f50dc0a2930 auth_method 0 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50da59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50da59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50da59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50d9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50d9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50d9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50da59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50da59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50da59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50dad9d640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50dc1380e0 0x7f50dc09b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50dad9d640 10 monclient(hunting): get_auth_request con 0x7f50dc1380e0 auth_method 0 2024-09-15T01:43:14.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50dad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50dad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.237+0000 7f50dad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50dad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50dad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50dad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50d9d9b640 10 monclient(hunting): handle_auth_done global_id 34616 payload 824 2024-09-15T01:43:14.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50d9d9b640 1 -- 172.21.15.178:0/3106522520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50dc1380e0 msgr2=0x7f50dc09b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50d9d9b640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50dc1380e0 0x7f50dc09b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50d9d9b640 1 -- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc0a2930 msgr2=0x7f50dc09a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50d9d9b640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc0a2930 0x7f50dc09a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50d9d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:43:14.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50d9d9b640 1 monclient: found mon.noname-b 2024-09-15T01:43:14.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50d9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50d9d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50d9d9b640 1 -- 172.21.15.178:0/3106522520 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d4009450 con 0x7f50dc132de0 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50da59c640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc0a2930 0x7f50dc09a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50d9d9b640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc132de0 0x7f50dc09ab00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f50d000d860 tx=0x7f50d0008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50d000bb40 con 0x7f50dc132de0 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50ca7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50ca7fc640 10 monclient: dump: 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.259 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50ca7fc640 10 monclient: _finish_auth 0 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:14.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d000bce0 con 0x7f50dc132de0 2024-09-15T01:43:14.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50e095d640 5 monclient: authenticate success, global_id 34616 2024-09-15T01:43:14.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50ca7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:14.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50d000be50 con 0x7f50dc132de0 2024-09-15T01:43:14.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50e095d640 10 monclient: _renew_subs 2024-09-15T01:43:14.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50e095d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:14.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50e095d640 1 -- 172.21.15.178:0/3106522520 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50dc11d820 con 0x7f50dc132de0 2024-09-15T01:43:14.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50d8d99640 4 set_mon_vals no callback set 2024-09-15T01:43:14.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50ca7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:14.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50ca7fc640 10 monclient: dump: 2024-09-15T01:43:14.261 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.261 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.262 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50e095d640 10 monclient: _renew_subs 2024-09-15T01:43:14.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50e095d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:14.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.238+0000 7f50e095d640 1 -- 172.21.15.178:0/3106522520 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50dc11c780 con 0x7f50dc132de0 2024-09-15T01:43:14.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f50d000b6c0 con 0x7f50dc132de0 2024-09-15T01:43:14.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50e095d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:14.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50e095d640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc11e200 0x7f50dc09b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50e095d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50e095d640 1 -- 172.21.15.178:0/3106522520 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0a2450 con 0x7f50dc11e200 2024-09-15T01:43:14.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50ca7fc640 1 --2- 172.21.15.178:0/3106522520 >> v2:172.21.15.204:6812/28210 conn(0x7f50bc0397b0 0x7f50bc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50da59c640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc11e200 0x7f50dc09b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f50d004be20 con 0x7f50dc132de0 2024-09-15T01:43:14.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50da59c640 10 monclient: get_auth_request con 0x7f50dc11e200 auth_method 0 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50da59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50da59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50da59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50da59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50da59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50da59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50dad9d640 1 --2- 172.21.15.178:0/3106522520 >> v2:172.21.15.204:6812/28210 conn(0x7f50bc0397b0 0x7f50bc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50dad9d640 10 monclient: get_auth_request con 0x7f50bc0397b0 auth_method 0 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50da59c640 10 monclient(hunting): handle_auth_done global_id 34794 payload 824 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50da59c640 1 -- 172.21.15.178:0/3106522520 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50dc05ed70 con 0x7f50dc11e200 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.239+0000 7f50da59c640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc11e200 0x7f50dc09b580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f50cc000f80 tx=0x7f50cc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.240+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50cc00deb0 con 0x7f50dc11e200 2024-09-15T01:43:14.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.240+0000 7f50ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.240+0000 7f50ca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.240+0000 7f50ca7fc640 10 monclient: dump: 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.265 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.266 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.266 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.266 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.266 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.240+0000 7f50dad9d640 1 --2- 172.21.15.178:0/3106522520 >> v2:172.21.15.204:6812/28210 conn(0x7f50bc0397b0 0x7f50bc03bc60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f50d4034000 tx=0x7f50d4032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.240+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f50cc0133f0 con 0x7f50dc11e200 2024-09-15T01:43:14.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.240+0000 7f50ca7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:14.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.240+0000 7f50ca7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:14.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.240+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc11e200 msgr2=0x7f50dc09b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.240+0000 7f50ca7fc640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc11e200 0x7f50dc09b580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f50cc000f80 tx=0x7f50cc009040 comp rx=0 tx=0).stop 2024-09-15T01:43:14.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.240+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc11e200 msgr2=0x7f50dc09b580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:14.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.240+0000 7f50ca7fc640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc11e200 0x7f50dc09b580 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f50cc000f80 tx=0x7f50cc009040 comp rx=0 tx=0).stop 2024-09-15T01:43:14.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.245+0000 7f50e095d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:14.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.245+0000 7f50e095d640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc096a30 0x7f50dc096e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.245+0000 7f50e095d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.245+0000 7f50e095d640 1 -- 172.21.15.178:0/3106522520 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc04bc80 con 0x7f50dc096a30 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50da59c640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc096a30 0x7f50dc096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50da59c640 10 monclient: get_auth_request con 0x7f50dc096a30 auth_method 0 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50da59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50da59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50da59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50da59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50da59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50da59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50da59c640 10 monclient(hunting): handle_auth_done global_id 34803 payload 824 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50da59c640 1 -- 172.21.15.178:0/3106522520 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f50dc04b720 con 0x7f50dc096a30 2024-09-15T01:43:14.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50da59c640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc096a30 0x7f50dc096e30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e010 tx=0x7f50cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50cc01de10 con 0x7f50dc096a30 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50ca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50ca7fc640 10 monclient: dump: 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.270 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f50cc00da30 con 0x7f50dc096a30 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50ca7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50ca7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc096a30 msgr2=0x7f50dc096e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50ca7fc640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc096a30 0x7f50dc096e30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e010 tx=0x7f50cc019000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50ca7fc640 1 -- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc096a30 msgr2=0x7f50dc096e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:14.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.246+0000 7f50ca7fc640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50dc096a30 0x7f50dc096e30 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e010 tx=0x7f50cc019000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.272 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:14.272 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:14.272 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:14.272 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:14.272 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:14.272 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:14.272 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:14.272 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:14.272 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:14.272 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.273 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.274 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:14.275 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.276 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.277 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.277 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.277 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.277 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.277 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.277 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.277 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:14.277 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.277 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:14.277 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.277 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:14.277 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.278 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.279 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:14.280 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:14.281 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.281 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.281 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.281 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.281 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.281 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:14.281 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:14.281 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.281 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.281 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.281 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.281 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.282 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.283 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.284 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:14.285 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:14.286 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:14.288 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.289 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.290 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:14.291 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.292 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:14.293 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.294 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.294 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:14.294 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.294 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.294 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.294 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:14.294 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.294 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.294 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.294 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:14.294 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:14.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.247+0000 7f50e095d640 1 -- 172.21.15.178:0/3106522520 >> v2:172.21.15.204:6812/28210 conn(0x7f50bc0397b0 msgr2=0x7f50bc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.247+0000 7f50e095d640 1 --2- 172.21.15.178:0/3106522520 >> v2:172.21.15.204:6812/28210 conn(0x7f50bc0397b0 0x7f50bc03bc60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f50d4034000 tx=0x7f50d4032000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.247+0000 7f50e095d640 10 monclient: shutdown 2024-09-15T01:43:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.247+0000 7f50e095d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.247+0000 7f50e095d640 1 -- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc132de0 msgr2=0x7f50dc09ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.247+0000 7f50e095d640 1 --2- 172.21.15.178:0/3106522520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50dc132de0 0x7f50dc09ab00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f50d000d860 tx=0x7f50d0008040 comp rx=0 tx=0).stop 2024-09-15T01:43:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.247+0000 7f50dad9d640 1 -- 172.21.15.178:0/3106522520 reap_dead start 2024-09-15T01:43:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.247+0000 7f50dad9d640 1 -- 172.21.15.178:0/3106522520 reap_dead start 2024-09-15T01:43:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.247+0000 7f50e095d640 1 -- 172.21.15.178:0/3106522520 shutdown_connections 2024-09-15T01:43:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.247+0000 7f50e095d640 1 -- 172.21.15.178:0/3106522520 >> 172.21.15.178:0/3106522520 conn(0x7f50dc0a5ed0 msgr2=0x7f50dc135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.247+0000 7f50e095d640 1 -- 172.21.15.178:0/3106522520 shutdown_connections 2024-09-15T01:43:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.247+0000 7f50e095d640 1 -- 172.21.15.178:0/3106522520 wait complete. 2024-09-15T01:43:14.297 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:14.297 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:43:14.297 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:43:14.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.389+0000 7f784d59d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:14.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.389+0000 7f784d59d640 10 monclient: build_initial_monmap 2024-09-15T01:43:14.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.389+0000 7f784d59d640 10 monclient: monmap: 2024-09-15T01:43:14.391 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:14.391 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.391 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:14.390261+0000 2024-09-15T01:43:14.392 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:14.390261+0000 2024-09-15T01:43:14.392 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:14.392 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.392 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:14.392 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:14.392 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:14.392 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:14.392 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:14.392 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:14.392 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:14.392 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:14.392 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:14.393 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:14.393 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:14.393 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 1 Processor -- start 2024-09-15T01:43:14.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 1 -- start start 2024-09-15T01:43:14.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 10 monclient: init 2024-09-15T01:43:14.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:14.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 10 monclient: _add_conns ranks=[4,1,3,2,8,9,6,5,7,10,0] 2024-09-15T01:43:14.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78480a2930 0x7f7848132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 10 monclient(hunting): picked mon.noname-c con 0x7f78480a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f784809faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f784809fe70 0x7f78480a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 10 monclient(hunting): picked mon.noname-g con 0x7f784809fe70 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784804bf60 con 0x7f784809faa0 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784809dd80 con 0x7f784809fe70 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78480a2d00 con 0x7f78480a2930 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f784d59d640 10 monclient(hunting): authenticate will time out at 1399.140613s 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78480a2930 0x7f7848132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78480a2930 0x7f7848132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:32798/0 (socket says 172.21.15.178:32798) 2024-09-15T01:43:14.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 1 -- 172.21.15.178:0/595113081 learned_addr learned my addr 172.21.15.178:0/595113081 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 10 monclient(hunting): get_auth_request con 0x7f78480a2930 auth_method 0 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f78477fe640 1 --2- 172.21.15.178:0/595113081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f784809fe70 0x7f78480a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f78477fe640 10 monclient(hunting): get_auth_request con 0x7f784809fe70 auth_method 0 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f78477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f78477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f78477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 10 monclient(hunting): handle_auth_done global_id 44338 payload 274 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 1 -- 172.21.15.178:0/595113081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f784809fe70 msgr2=0x7f78480a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 1 --2- 172.21.15.178:0/595113081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f784809fe70 0x7f78480a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 1 -- 172.21.15.178:0/595113081 >> v1:172.21.15.204:6789/0 conn(0x7f784809faa0 legacy=0x7f78480a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 1 monclient: found mon.noname-c 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.390+0000 7f7846ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7846ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7846ffd640 1 -- 172.21.15.178:0/595113081 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78480a0310 con 0x7f78480a2930 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7846ffd640 1 --2- 172.21.15.178:0/595113081 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78480a2930 0x7f7848132de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7838009f50 tx=0x7f7838009840 comp rx=0 tx=0).ready entity=mon.4 client_cookie=9b047cc19b25c0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7845ffb640 1 -- 172.21.15.178:0/595113081 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7838007880 con 0x7f78480a2930 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7845ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7845ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:43:14.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7845ffb640 10 monclient: dump: 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.406 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7845ffb640 10 monclient: _finish_auth 0 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7845ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f784d59d640 5 monclient: authenticate success, global_id 44338 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7845ffb640 1 -- 172.21.15.178:0/595113081 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783802d050 con 0x7f78480a2930 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7845ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7845ffb640 1 -- 172.21.15.178:0/595113081 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7838030360 con 0x7f78480a2930 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f784d59d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f784d59d640 4 set_mon_vals no callback set 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7845ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7845ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.391+0000 7f7845ffb640 10 monclient: dump: 2024-09-15T01:43:14.407 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.408 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.392+0000 7f784d59d640 10 monclient: shutdown 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.392+0000 7f784d59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.392+0000 7f784d59d640 1 -- 172.21.15.178:0/595113081 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78480a2930 msgr2=0x7f7848132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.392+0000 7f784d59d640 1 --2- 172.21.15.178:0/595113081 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78480a2930 0x7f7848132de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7838009f50 tx=0x7f7838009840 comp rx=0 tx=0).stop 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.392+0000 7f784d59d640 1 -- 172.21.15.178:0/595113081 shutdown_connections 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.392+0000 7f784d59d640 1 --2- 172.21.15.178:0/595113081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f784809fe70 0x7f78480a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.392+0000 7f784d59d640 1 --2- 172.21.15.178:0/595113081 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f78480a2930 0x7f7848132de0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.392+0000 7f784d59d640 1 -- 172.21.15.178:0/595113081 >> 172.21.15.178:0/595113081 conn(0x7f78480a5ed0 msgr2=0x7f78480a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.392+0000 7f784d59d640 1 -- 172.21.15.178:0/595113081 shutdown_connections 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.392+0000 7f784d59d640 1 -- 172.21.15.178:0/595113081 wait complete. 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient: build_initial_monmap 2024-09-15T01:43:14.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient: monmap: 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:14.410 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 1 Processor -- start 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 1 -- start start 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient: init 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient: _add_conns ranks=[2,8,1,0,5,3,6,7,4] 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f784809faa0 0x7f784809b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient(hunting): picked mon.noname-b con 0x7f784809faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f784809fe70 0x7f7848131800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient(hunting): picked mon.noname-e con 0x7f784809fe70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78480a2930 0x7f784811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient(hunting): picked mon.noname-f con 0x7f78480a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784811ec00 con 0x7f78480a2930 2024-09-15T01:43:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784811e920 con 0x7f784809faa0 2024-09-15T01:43:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f784811eee0 con 0x7f784809fe70 2024-09-15T01:43:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f784d59d640 10 monclient(hunting): authenticate will time out at 1399.143900s 2024-09-15T01:43:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f7846ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f784809faa0 0x7f784809b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.393+0000 7f78467fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f784809fe70 0x7f7848131800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f784809faa0 0x7f784809b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:48228/0 (socket says 172.21.15.178:48228) 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 1 -- 172.21.15.178:0/2243407320 learned_addr learned my addr 172.21.15.178:0/2243407320 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 10 monclient(hunting): get_auth_request con 0x7f784809faa0 auth_method 0 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78467fc640 10 monclient(hunting): get_auth_request con 0x7f784809fe70 auth_method 0 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78467fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78467fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78467fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78467fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78477fe640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78480a2930 0x7f784811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78467fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78467fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78477fe640 10 monclient(hunting): get_auth_request con 0x7f78480a2930 auth_method 0 2024-09-15T01:43:14.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78477fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 10 monclient(hunting): handle_auth_done global_id 34625 payload 824 2024-09-15T01:43:14.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 1 -- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f784809fe70 msgr2=0x7f7848131800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f784809fe70 0x7f7848131800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 1 -- 172.21.15.178:0/2243407320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78480a2930 msgr2=0x7f784811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78480a2930 0x7f784811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:43:14.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 1 monclient: found mon.noname-b 2024-09-15T01:43:14.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:14.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:14.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 1 -- 172.21.15.178:0/2243407320 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7838009ae0 con 0x7f784809faa0 2024-09-15T01:43:14.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f7846ffd640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f784809faa0 0x7f784809b140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7838009fb0 tx=0x7f7838036000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f783803cab0 con 0x7f784809faa0 2024-09-15T01:43:14.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f782effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f782effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f782effd640 10 monclient: dump: 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.416 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f782effd640 10 monclient: _finish_auth 0 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f782effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f784d59d640 5 monclient: authenticate success, global_id 34625 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f784d59d640 10 monclient: _renew_subs 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783803cc50 con 0x7f784809faa0 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f784d59d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f784d59d640 1 -- 172.21.15.178:0/2243407320 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784809bc80 con 0x7f784809faa0 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f782effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:14.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f783803cdc0 con 0x7f784809faa0 2024-09-15T01:43:14.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f782effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f782effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:14.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f782effd640 10 monclient: dump: 2024-09-15T01:43:14.418 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.418 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.418 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.418 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.418 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.418 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.418 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.418 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f78457fa640 4 set_mon_vals no callback set 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f784d59d640 10 monclient: _renew_subs 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f784d59d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.394+0000 7f784d59d640 1 -- 172.21.15.178:0/2243407320 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7848142a30 con 0x7f784809faa0 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f784d59d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:14.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f784d59d640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848131d40 0x7f784811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f784d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f784d59d640 1 -- 172.21.15.178:0/2243407320 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7848132110 con 0x7f7848131d40 2024-09-15T01:43:14.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7838003040 con 0x7f784809faa0 2024-09-15T01:43:14.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f78467fc640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848131d40 0x7f784811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f78467fc640 10 monclient: get_auth_request con 0x7f7848131d40 auth_method 0 2024-09-15T01:43:14.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f78467fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f78467fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f78467fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f78467fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f78467fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f78467fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f782effd640 1 --2- 172.21.15.178:0/2243407320 >> v2:172.21.15.204:6812/28210 conn(0x7f78340397b0 0x7f783403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f78467fc640 10 monclient(hunting): handle_auth_done global_id 34634 payload 824 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f78467fc640 1 -- 172.21.15.178:0/2243407320 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f784811ea90 con 0x7f7848131d40 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.395+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f783806c1a0 con 0x7f784809faa0 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f78467fc640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848131d40 0x7f784811ca70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f78480a6ec0 tx=0x7f783c007e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f78477fe640 1 --2- 172.21.15.178:0/2243407320 >> v2:172.21.15.204:6812/28210 conn(0x7f78340397b0 0x7f783403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f78477fe640 10 monclient: get_auth_request con 0x7f78340397b0 auth_method 0 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f783c002b60 con 0x7f7848131d40 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f782effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f782effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f782effd640 10 monclient: dump: 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.421 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.422 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.422 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.422 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.422 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.422 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.422 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.422 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.422 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.422 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.422 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.422 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.423 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f783c002cc0 con 0x7f7848131d40 2024-09-15T01:43:14.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f782effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:14.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f782effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:14.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848131d40 msgr2=0x7f784811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f782effd640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848131d40 0x7f784811ca70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f78480a6ec0 tx=0x7f783c007e30 comp rx=0 tx=0).stop 2024-09-15T01:43:14.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848131d40 msgr2=0x7f784811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:14.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f782effd640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848131d40 0x7f784811ca70 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f78480a6ec0 tx=0x7f783c007e30 comp rx=0 tx=0).stop 2024-09-15T01:43:14.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.396+0000 7f78477fe640 1 --2- 172.21.15.178:0/2243407320 >> v2:172.21.15.204:6812/28210 conn(0x7f78340397b0 0x7f783403bc60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f784809b090 tx=0x7f7840008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f784d59d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:14.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f784d59d640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848070620 0x7f784811cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f784d59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f784d59d640 1 -- 172.21.15.178:0/2243407320 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784811ed70 con 0x7f7848070620 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f78467fc640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848070620 0x7f784811cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f78467fc640 10 monclient: get_auth_request con 0x7f7848070620 auth_method 0 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f78467fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f78467fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f78467fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f78467fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f78467fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f78467fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f78467fc640 10 monclient(hunting): handle_auth_done global_id 34643 payload 824 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f78467fc640 1 -- 172.21.15.178:0/2243407320 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f784811f050 con 0x7f7848070620 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f78467fc640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848070620 0x7f784811cfb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f78480a6ec0 tx=0x7f783c019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f783c0076e0 con 0x7f7848070620 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f782effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f782effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f782effd640 10 monclient: dump: 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.427 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.428 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.428 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.428 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.428 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.428 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f783c007260 con 0x7f7848070620 2024-09-15T01:43:14.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f782effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:14.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f782effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:14.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848070620 msgr2=0x7f784811cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f782effd640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848070620 0x7f784811cfb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f78480a6ec0 tx=0x7f783c019000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f782effd640 1 -- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848070620 msgr2=0x7f784811cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:14.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.402+0000 7f782effd640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7848070620 0x7f784811cfb0 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f78480a6ec0 tx=0x7f783c019000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.429 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:14.429 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:14.429 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:14.429 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:14.429 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:14.429 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:14.429 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:14.429 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:14.429 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:14.429 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:14.429 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.431 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:14.432 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.433 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.434 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.435 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.436 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:14.437 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.437 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.437 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:14.437 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.437 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.437 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.437 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:14.437 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:14.437 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.440 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:14.440 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:14.440 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.440 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.440 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.440 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:14.440 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.440 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.440 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:14.441 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.442 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.443 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.444 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.445 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.448 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:14.449 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.450 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:14.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.451 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.451 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.451 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:43:14.451 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.451 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.451 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:14.451 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:14.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.403+0000 7f784d59d640 1 -- 172.21.15.178:0/2243407320 >> v2:172.21.15.204:6812/28210 conn(0x7f78340397b0 msgr2=0x7f783403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.403+0000 7f784d59d640 1 --2- 172.21.15.178:0/2243407320 >> v2:172.21.15.204:6812/28210 conn(0x7f78340397b0 0x7f783403bc60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f784809b090 tx=0x7f7840008040 comp rx=0 tx=0).stop 2024-09-15T01:43:14.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.403+0000 7f784d59d640 10 monclient: shutdown 2024-09-15T01:43:14.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.403+0000 7f784d59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:14.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.403+0000 7f784d59d640 1 -- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f784809faa0 msgr2=0x7f784809b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.403+0000 7f784d59d640 1 --2- 172.21.15.178:0/2243407320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f784809faa0 0x7f784809b140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7838009fb0 tx=0x7f7838036000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.403+0000 7f78477fe640 1 -- 172.21.15.178:0/2243407320 reap_dead start 2024-09-15T01:43:14.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.403+0000 7f78477fe640 1 -- 172.21.15.178:0/2243407320 reap_dead start 2024-09-15T01:43:14.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.403+0000 7f784d59d640 1 -- 172.21.15.178:0/2243407320 shutdown_connections 2024-09-15T01:43:14.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.403+0000 7f784d59d640 1 -- 172.21.15.178:0/2243407320 >> 172.21.15.178:0/2243407320 conn(0x7f78480a5ed0 msgr2=0x7f7848135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:14.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.403+0000 7f784d59d640 1 -- 172.21.15.178:0/2243407320 shutdown_connections 2024-09-15T01:43:14.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.403+0000 7f784d59d640 1 -- 172.21.15.178:0/2243407320 wait complete. 2024-09-15T01:43:14.454 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:14.454 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:43:14.454 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:43:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.546+0000 7f10c7f14640 10 monclient: get_monmap_and_config 2024-09-15T01:43:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.546+0000 7f10c7f14640 10 monclient: build_initial_monmap 2024-09-15T01:43:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.546+0000 7f10c7f14640 10 monclient: monmap: 2024-09-15T01:43:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:14.547702+0000 2024-09-15T01:43:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:14.547702+0000 2024-09-15T01:43:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:14.550 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:14.550 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:14.550 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:14.550 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:14.550 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:14.550 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:14.550 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:14.550 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:14.550 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:14.550 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:14.550 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.547+0000 7f10c7f14640 1 Processor -- start 2024-09-15T01:43:14.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.547+0000 7f10c7f14640 1 -- start start 2024-09-15T01:43:14.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.547+0000 7f10c7f14640 10 monclient: init 2024-09-15T01:43:14.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:14.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 10 monclient: _add_conns ranks=[5,0,2,7,6,10,1,4,9,8,3] 2024-09-15T01:43:14.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f10c012f5d0 0x7f10c012f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 10 monclient(hunting): picked mon.noname-h con 0x7f10c012f5d0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:14.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f10c006e510 addr v1:172.21.15.178:6789/0 2024-09-15T01:43:14.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10c006e8e0 0x7f10c009dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 10 monclient(hunting): picked mon.noname-b con 0x7f10c006e8e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:14.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c004bf60 con 0x7f10c006e510 2024-09-15T01:43:14.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c009f2c0 con 0x7f10c006e8e0 2024-09-15T01:43:14.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10c006b850 con 0x7f10c012f5d0 2024-09-15T01:43:14.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:14.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c7f14640 10 monclient(hunting): authenticate will time out at 1399.298181s 2024-09-15T01:43:14.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10c006e8e0 0x7f10c009dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c54af640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f10c006e510 0x7f10c009d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43134/0 (socket says 172.21.15.178:43134) 2024-09-15T01:43:14.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c54af640 1 -- 172.21.15.178:0/105915011 learned_addr learned my addr 172.21.15.178:0/105915011 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:43:14.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 10 monclient(hunting): get_auth_request con 0x7f10c006e8e0 auth_method 0 2024-09-15T01:43:14.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c5cb0640 1 --2- 172.21.15.178:0/105915011 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f10c012f5d0 0x7f10c012f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c5cb0640 10 monclient(hunting): get_auth_request con 0x7f10c012f5d0 auth_method 0 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c5cb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c5cb0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c5cb0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c5cb0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c5cb0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c5cb0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 10 monclient(hunting): handle_auth_done global_id 34652 payload 274 2024-09-15T01:43:14.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c4cae640 1 -- 172.21.15.178:0/105915011 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743096763 0 0) 0x7f10c004bf60 con 0x7f10c006e510 2024-09-15T01:43:14.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 1 -- 172.21.15.178:0/105915011 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f10c012f5d0 msgr2=0x7f10c012f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:14.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 1 --2- 172.21.15.178:0/105915011 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f10c012f5d0 0x7f10c012f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 1 -- 172.21.15.178:0/105915011 >> v1:172.21.15.178:6789/0 conn(0x7f10c006e510 legacy=0x7f10c009d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 10 monclient: _finish_hunting 0 2024-09-15T01:43:14.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 1 monclient: found mon.noname-b 2024-09-15T01:43:14.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:14.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:14.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c64b1640 1 -- 172.21.15.178:0/105915011 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c006c740 con 0x7f10c006e8e0 2024-09-15T01:43:14.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.548+0000 7f10c4cae640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:43:14.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c64b1640 1 --2- 172.21.15.178:0/105915011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10c006e8e0 0x7f10c009dc90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009e90 tx=0x7f10b8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41c5d51abf3d7339 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c4cae640 1 -- 172.21.15.178:0/105915011 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10b8031880 con 0x7f10c006e8e0 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c4cae640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c4cae640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c4cae640 10 monclient: dump: 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.564 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.565 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.565 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.565 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.565 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.565 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.565 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c4cae640 10 monclient: _finish_auth 0 2024-09-15T01:43:14.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c4cae640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:14.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c4cae640 1 -- 172.21.15.178:0/105915011 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b802d050 con 0x7f10c006e8e0 2024-09-15T01:43:14.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c7f14640 5 monclient: authenticate success, global_id 34652 2024-09-15T01:43:14.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c7f14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:14.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c4cae640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:14.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c7f14640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:14.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c4cae640 1 -- 172.21.15.178:0/105915011 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10b80303a0 con 0x7f10c006e8e0 2024-09-15T01:43:14.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c7f14640 4 set_mon_vals no callback set 2024-09-15T01:43:14.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c4cae640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c4cae640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:14.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c4cae640 10 monclient: dump: 2024-09-15T01:43:14.566 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.566 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.566 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.566 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.566 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.567 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.567 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.567 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.567 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.567 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.567 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.567 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.567 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.567 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.567 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.567 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c7f14640 10 monclient: shutdown 2024-09-15T01:43:14.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c7f14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:14.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c7f14640 1 -- 172.21.15.178:0/105915011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10c006e8e0 msgr2=0x7f10c009dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c7f14640 1 --2- 172.21.15.178:0/105915011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10c006e8e0 0x7f10c009dc90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009e90 tx=0x7f10b8009780 comp rx=0 tx=0).stop 2024-09-15T01:43:14.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c7f14640 1 -- 172.21.15.178:0/105915011 shutdown_connections 2024-09-15T01:43:14.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c7f14640 1 --2- 172.21.15.178:0/105915011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10c006e8e0 0x7f10c009dc90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c7f14640 1 --2- 172.21.15.178:0/105915011 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f10c012f5d0 0x7f10c012f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.549+0000 7f10c7f14640 1 -- 172.21.15.178:0/105915011 >> 172.21.15.178:0/105915011 conn(0x7f10c0068a60 msgr2=0x7f10c00a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:14.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.550+0000 7f10c7f14640 1 -- 172.21.15.178:0/105915011 shutdown_connections 2024-09-15T01:43:14.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.550+0000 7f10c7f14640 1 -- 172.21.15.178:0/105915011 wait complete. 2024-09-15T01:43:14.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.550+0000 7f10c7f14640 10 monclient: build_initial_monmap 2024-09-15T01:43:14.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.550+0000 7f10c7f14640 10 monclient: monmap: 2024-09-15T01:43:14.569 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:14.569 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:14.569 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:14.569 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:14.569 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:14.569 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:14.569 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:14.569 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.550+0000 7f10c7f14640 1 Processor -- start 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.550+0000 7f10c7f14640 1 -- start start 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.550+0000 7f10c7f14640 10 monclient: init 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 10 monclient: _add_conns ranks=[6,1,7,4,3,8,2,5,0] 2024-09-15T01:43:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f10c006e510 0x7f10c01334d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 10 monclient(hunting): picked mon.noname-d con 0x7f10c006e510 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10c006e8e0 0x7f10c012e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 10 monclient(hunting): picked mon.noname-f con 0x7f10c006e8e0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f10c012f5d0 0x7f10c012e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 10 monclient(hunting): picked mon.noname-i con 0x7f10c012f5d0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c006bb00 con 0x7f10c006e8e0 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f10c009ccc0 con 0x7f10c006e510 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f10c01343e0 con 0x7f10c012f5d0 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c7f14640 10 monclient(hunting): authenticate will time out at 1399.301391s 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f10c006e510 0x7f10c01334d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f10c006e510 0x7f10c01334d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44302/0 (socket says 172.21.15.178:44302) 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 1 -- 172.21.15.178:0/1319298189 learned_addr learned my addr 172.21.15.178:0/1319298189 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 10 monclient(hunting): get_auth_request con 0x7f10c006e510 auth_method 0 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c54af640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10c006e8e0 0x7f10c012e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c54af640 10 monclient(hunting): get_auth_request con 0x7f10c006e8e0 auth_method 0 2024-09-15T01:43:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c54af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c54af640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c54af640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c64b1640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f10c012f5d0 0x7f10c012e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 10 monclient(hunting): handle_auth_done global_id 25143 payload 824 2024-09-15T01:43:14.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 1 -- 172.21.15.178:0/1319298189 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f10c012f5d0 msgr2=0x7f10c012e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f10c012f5d0 0x7f10c012e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 1 -- 172.21.15.178:0/1319298189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10c006e8e0 msgr2=0x7f10c012e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10c006e8e0 0x7f10c012e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 10 monclient: _finish_hunting 0 2024-09-15T01:43:14.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 1 monclient: found mon.noname-d 2024-09-15T01:43:14.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:14.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 1 -- 172.21.15.178:0/1319298189 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b8009a20 con 0x7f10c006e510 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c64b1640 10 monclient: get_auth_request con 0x7f10c012f5d0 auth_method 0 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c64b1640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f10c012f5d0 0x7f10c012e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c54af640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10c006e8e0 0x7f10c012e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.551+0000 7f10c5cb0640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f10c006e510 0x7f10c01334d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00e930 tx=0x7f10bc0090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10bc009480 con 0x7f10c006e510 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b5ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b5ffb640 10 monclient: dump: 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:43:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10bc00cd60 con 0x7f10c006e510 2024-09-15T01:43:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10c7f14640 5 monclient: authenticate success, global_id 25143 2024-09-15T01:43:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10c7f14640 10 monclient: _renew_subs 2024-09-15T01:43:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10c7f14640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:43:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10c7f14640 1 -- 172.21.15.178:0/1319298189 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c0140820 con 0x7f10c006e510 2024-09-15T01:43:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10bc00ced0 con 0x7f10c006e510 2024-09-15T01:43:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b5ffb640 10 monclient: dump: 2024-09-15T01:43:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.578 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10c7f14640 10 monclient: _renew_subs 2024-09-15T01:43:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10b7fff640 4 set_mon_vals no callback set 2024-09-15T01:43:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10c7f14640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:43:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.552+0000 7f10c7f14640 1 -- 172.21.15.178:0/1319298189 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c011b4c0 con 0x7f10c006e510 2024-09-15T01:43:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f10bc00c8e0 con 0x7f10c006e510 2024-09-15T01:43:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c7f14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c7f14640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c01199b0 0x7f10c00298c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c7f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c7f14640 1 -- 172.21.15.178:0/1319298189 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00641f0 con 0x7f10c01199b0 2024-09-15T01:43:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c54af640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c01199b0 0x7f10c00298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c54af640 10 monclient: get_auth_request con 0x7f10c01199b0 auth_method 0 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c54af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c54af640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c54af640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10b5ffb640 1 --2- 172.21.15.178:0/1319298189 >> v2:172.21.15.204:6812/28210 conn(0x7f10a40397b0 0x7f10a403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c54af640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c54af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c54af640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f10bc04be60 con 0x7f10c006e510 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c54af640 10 monclient(hunting): handle_auth_done global_id 44347 payload 824 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c54af640 1 -- 172.21.15.178:0/1319298189 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f10c004c4a0 con 0x7f10c01199b0 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c64b1640 1 --2- 172.21.15.178:0/1319298189 >> v2:172.21.15.204:6812/28210 conn(0x7f10a40397b0 0x7f10a403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c64b1640 10 monclient: get_auth_request con 0x7f10a40397b0 auth_method 0 2024-09-15T01:43:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10c54af640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c01199b0 0x7f10c00298c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f10b00011e0 tx=0x7f10b0009040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10b00070d0 con 0x7f10c01199b0 2024-09-15T01:43:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10b5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10b5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.553+0000 7f10b5ffb640 10 monclient: dump: 2024-09-15T01:43:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.554+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f10b0005a80 con 0x7f10c01199b0 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.554+0000 7f10b5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.554+0000 7f10b5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.554+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c01199b0 msgr2=0x7f10c00298c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.554+0000 7f10b5ffb640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c01199b0 0x7f10c00298c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f10b00011e0 tx=0x7f10b0009040 comp rx=0 tx=0).stop 2024-09-15T01:43:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.554+0000 7f10c64b1640 1 --2- 172.21.15.178:0/1319298189 >> v2:172.21.15.204:6812/28210 conn(0x7f10a40397b0 0x7f10a403bc60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f10b8034000 tx=0x7f10b8037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.554+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c01199b0 msgr2=0x7f10c00298c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:14.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.554+0000 7f10b5ffb640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c01199b0 0x7f10c00298c0 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f10b00011e0 tx=0x7f10b0009040 comp rx=0 tx=0).stop 2024-09-15T01:43:14.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.559+0000 7f10c7f14640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:14.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.559+0000 7f10c7f14640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c0119d80 0x7f10c0029e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.559+0000 7f10c7f14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.559+0000 7f10c7f14640 1 -- 172.21.15.178:0/1319298189 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10c004bc80 con 0x7f10c0119d80 2024-09-15T01:43:14.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.559+0000 7f10c54af640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c0119d80 0x7f10c0029e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.559+0000 7f10c54af640 10 monclient: get_auth_request con 0x7f10c0119d80 auth_method 0 2024-09-15T01:43:14.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.559+0000 7f10c54af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.559+0000 7f10c54af640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.559+0000 7f10c54af640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.559+0000 7f10c54af640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.559+0000 7f10c54af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.559+0000 7f10c54af640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10c54af640 10 monclient(hunting): handle_auth_done global_id 44356 payload 824 2024-09-15T01:43:14.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10c54af640 1 -- 172.21.15.178:0/1319298189 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f10c005ed70 con 0x7f10c0119d80 2024-09-15T01:43:14.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10c54af640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c0119d80 0x7f10c0029e00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f10b0017000 tx=0x7f10b001a000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10b0013d70 con 0x7f10c0119d80 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10b5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10b5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10b5ffb640 10 monclient: dump: 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.585 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f10b0028430 con 0x7f10c0119d80 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10b5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10b5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c0119d80 msgr2=0x7f10c0029e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10b5ffb640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c0119d80 0x7f10c0029e00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f10b0017000 tx=0x7f10b001a000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10b5ffb640 1 -- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c0119d80 msgr2=0x7f10c0029e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:14.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10b5ffb640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f10c0119d80 0x7f10c0029e00 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f10b0017000 tx=0x7f10b001a000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.587 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:43:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:14.609 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:14.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10c7f14640 1 -- 172.21.15.178:0/1319298189 >> v2:172.21.15.204:6812/28210 conn(0x7f10a40397b0 msgr2=0x7f10a403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10c7f14640 1 --2- 172.21.15.178:0/1319298189 >> v2:172.21.15.204:6812/28210 conn(0x7f10a40397b0 0x7f10a403bc60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f10b8034000 tx=0x7f10b8037000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10c7f14640 10 monclient: shutdown 2024-09-15T01:43:14.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10c7f14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:14.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10c7f14640 1 -- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f10c006e510 msgr2=0x7f10c01334d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.560+0000 7f10c7f14640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f10c006e510 0x7f10c01334d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00e930 tx=0x7f10bc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.561+0000 7f10c64b1640 1 -- 172.21.15.178:0/1319298189 reap_dead start 2024-09-15T01:43:14.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.561+0000 7f10c7f14640 1 -- 172.21.15.178:0/1319298189 shutdown_connections 2024-09-15T01:43:14.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.561+0000 7f10c7f14640 1 --2- 172.21.15.178:0/1319298189 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f10c006e510 0x7f10c01334d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.561+0000 7f10c7f14640 1 -- 172.21.15.178:0/1319298189 >> 172.21.15.178:0/1319298189 conn(0x7f10c0068a60 msgr2=0x7f10c00a0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:14.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.561+0000 7f10c7f14640 1 -- 172.21.15.178:0/1319298189 shutdown_connections 2024-09-15T01:43:14.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.561+0000 7f10c7f14640 1 -- 172.21.15.178:0/1319298189 wait complete. 2024-09-15T01:43:14.611 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:14.612 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:43:14.612 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:43:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.706+0000 7f910782b640 10 monclient: get_monmap_and_config 2024-09-15T01:43:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.706+0000 7f910782b640 10 monclient: build_initial_monmap 2024-09-15T01:43:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.706+0000 7f910782b640 10 monclient: monmap: 2024-09-15T01:43:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:14.708 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:14.707136+0000 2024-09-15T01:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:14.707136+0000 2024-09-15T01:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:14.709 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:14.710 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:14.710 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.706+0000 7f910782b640 1 Processor -- start 2024-09-15T01:43:14.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 1 -- start start 2024-09-15T01:43:14.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 10 monclient: init 2024-09-15T01:43:14.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:14.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 10 monclient: _add_conns ranks=[2,10,9,4,5,3,0,6,1,8,7] 2024-09-15T01:43:14.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f910009e700 0x7f9100069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 10 monclient(hunting): picked mon.noname-b con 0x7f910009e700 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:14.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f910009ead0 0x7f910009eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 10 monclient(hunting): picked mon.noname-f con 0x7f910009ead0 addr v2:172.21.15.204:3300/0 2024-09-15T01:43:14.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f910012f4c0 0x7f910012f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 10 monclient(hunting): picked mon.noname-a con 0x7f910012f4c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:43:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9100068340 con 0x7f910012f4c0 2024-09-15T01:43:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9100069210 con 0x7f910009ead0 2024-09-15T01:43:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9100068f30 con 0x7f910009e700 2024-09-15T01:43:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f910782b640 10 monclient(hunting): authenticate will time out at 1399.457520s 2024-09-15T01:43:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f910009e700 0x7f9100069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f9105dc8640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f910012f4c0 0x7f910012f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f910009e700 0x7f9100069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:48262/0 (socket says 172.21.15.178:48262) 2024-09-15T01:43:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 1 -- 172.21.15.178:0/2883472938 learned_addr learned my addr 172.21.15.178:0/2883472938 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 10 monclient(hunting): get_auth_request con 0x7f910009e700 auth_method 0 2024-09-15T01:43:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f9105dc8640 10 monclient(hunting): get_auth_request con 0x7f910012f4c0 auth_method 0 2024-09-15T01:43:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f9105dc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f9105dc8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f9105dc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f9105dc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f9105dc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f9105dc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f9104dc6640 1 --2- 172.21.15.178:0/2883472938 >> v2:172.21.15.204:3300/0 conn(0x7f910009ead0 0x7f910009eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 10 monclient(hunting): handle_auth_done global_id 34661 payload 274 2024-09-15T01:43:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 1 -- 172.21.15.178:0/2883472938 >> v2:172.21.15.204:3300/0 conn(0x7f910009ead0 msgr2=0x7f910009eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 1 --2- 172.21.15.178:0/2883472938 >> v2:172.21.15.204:3300/0 conn(0x7f910009ead0 0x7f910009eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 1 -- 172.21.15.178:0/2883472938 >> v2:172.21.15.178:3300/0 conn(0x7f910012f4c0 msgr2=0x7f910012f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 1 --2- 172.21.15.178:0/2883472938 >> v2:172.21.15.178:3300/0 conn(0x7f910012f4c0 0x7f910012f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 10 monclient: _finish_hunting 0 2024-09-15T01:43:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 1 monclient: found mon.noname-b 2024-09-15T01:43:14.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:14.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:14.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f91055c7640 1 -- 172.21.15.178:0/2883472938 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910004bf60 con 0x7f910009e700 2024-09-15T01:43:14.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f9104dc6640 10 monclient: get_auth_request con 0x7f910009ead0 auth_method 0 2024-09-15T01:43:14.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.707+0000 7f9104dc6640 1 --2- 172.21.15.178:0/2883472938 >> v2:172.21.15.204:3300/0 conn(0x7f910009ead0 0x7f910009eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:43:14.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f91055c7640 1 --2- 172.21.15.178:0/2883472938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f910009e700 0x7f9100069bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f90f4009910 tx=0x7f90f40091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea6ca60f5c9fa62 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f90f3fff640 1 -- 172.21.15.178:0/2883472938 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90f402ac90 con 0x7f910009e700 2024-09-15T01:43:14.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f90f3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f90f3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:14.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f90f3fff640 10 monclient: dump: 2024-09-15T01:43:14.730 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.730 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.731 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.731 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.731 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.731 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.731 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.731 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.731 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.731 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.731 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.731 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.731 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.732 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f90f3fff640 10 monclient: _finish_auth 0 2024-09-15T01:43:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f90f3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f90f3fff640 1 -- 172.21.15.178:0/2883472938 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f402ae10 con 0x7f910009e700 2024-09-15T01:43:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f910782b640 5 monclient: authenticate success, global_id 34661 2024-09-15T01:43:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f910782b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f90f3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f910782b640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f90f3fff640 1 -- 172.21.15.178:0/2883472938 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90f4031ca0 con 0x7f910009e700 2024-09-15T01:43:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f90f3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f910782b640 4 set_mon_vals no callback set 2024-09-15T01:43:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f90f3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.708+0000 7f90f3fff640 10 monclient: dump: 2024-09-15T01:43:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.734 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 10 monclient: shutdown 2024-09-15T01:43:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 1 -- 172.21.15.178:0/2883472938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f910009e700 msgr2=0x7f9100069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 1 --2- 172.21.15.178:0/2883472938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f910009e700 0x7f9100069bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f90f4009910 tx=0x7f90f40091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 1 -- 172.21.15.178:0/2883472938 shutdown_connections 2024-09-15T01:43:14.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 1 --2- 172.21.15.178:0/2883472938 >> v2:172.21.15.178:3300/0 conn(0x7f910012f4c0 0x7f910012f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 1 --2- 172.21.15.178:0/2883472938 >> v2:172.21.15.204:3300/0 conn(0x7f910009ead0 0x7f910009eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 1 --2- 172.21.15.178:0/2883472938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f910009e700 0x7f9100069bf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 1 -- 172.21.15.178:0/2883472938 >> 172.21.15.178:0/2883472938 conn(0x7f91000a4390 msgr2=0x7f9100067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:14.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 1 -- 172.21.15.178:0/2883472938 shutdown_connections 2024-09-15T01:43:14.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 1 -- 172.21.15.178:0/2883472938 wait complete. 2024-09-15T01:43:14.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 10 monclient: build_initial_monmap 2024-09-15T01:43:14.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.709+0000 7f910782b640 10 monclient: monmap: 2024-09-15T01:43:14.735 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:14.735 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:14.735 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:14.736 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:14.736 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:14.736 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:14.736 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:14.736 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:14.736 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:14.736 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:14.736 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:14.736 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:14.736 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:14.736 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:14.737 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:14.737 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 1 Processor -- start 2024-09-15T01:43:14.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 1 -- start start 2024-09-15T01:43:14.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 10 monclient: init 2024-09-15T01:43:14.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:14.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 10 monclient: _add_conns ranks=[3,8,6,2,5,0,7,1,4] 2024-09-15T01:43:14.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f910009e700 0x7f9100141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 10 monclient(hunting): picked mon.noname-g con 0x7f910009e700 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:14.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f910009ead0 0x7f9100141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 10 monclient(hunting): picked mon.noname-e con 0x7f910009ead0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:14.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910012f4c0 0x7f9100142410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 10 monclient(hunting): picked mon.noname-d con 0x7f910012f4c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:14.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910011e620 con 0x7f910009e700 2024-09-15T01:43:14.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f91000687f0 con 0x7f910012f4c0 2024-09-15T01:43:14.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9100068520 con 0x7f910009ead0 2024-09-15T01:43:14.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:14.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.710+0000 7f910782b640 10 monclient(hunting): authenticate will time out at 1399.460936s 2024-09-15T01:43:14.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f910009ead0 0x7f9100141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9105dc8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910012f4c0 0x7f9100142410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9105dc8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910012f4c0 0x7f9100142410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44312/0 (socket says 172.21.15.178:44312) 2024-09-15T01:43:14.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f910009ead0 0x7f9100141ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:55058/0 (socket says 172.21.15.178:55058) 2024-09-15T01:43:14.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9105dc8640 1 -- 172.21.15.178:0/2199171940 learned_addr learned my addr 172.21.15.178:0/2199171940 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:14.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9105dc8640 10 monclient(hunting): get_auth_request con 0x7f910012f4c0 auth_method 0 2024-09-15T01:43:14.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9105dc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9105dc8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9105dc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 10 monclient(hunting): get_auth_request con 0x7f910009ead0 auth_method 0 2024-09-15T01:43:14.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9105dc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9105dc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9105dc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f91055c7640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f910009e700 0x7f9100141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 10 monclient(hunting): handle_auth_done global_id 34640 payload 824 2024-09-15T01:43:14.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 1 -- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910012f4c0 msgr2=0x7f9100142410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910012f4c0 0x7f9100142410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 1 -- 172.21.15.178:0/2199171940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f910009e700 msgr2=0x7f9100141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f910009e700 0x7f9100141990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 10 monclient: _finish_hunting 0 2024-09-15T01:43:14.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 1 monclient: found mon.noname-e 2024-09-15T01:43:14.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:14.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:14.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 1 -- 172.21.15.178:0/2199171940 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4009450 con 0x7f910009ead0 2024-09-15T01:43:14.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f91055c7640 10 monclient: get_auth_request con 0x7f910009e700 auth_method 0 2024-09-15T01:43:14.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f91055c7640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f910009e700 0x7f9100141990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:43:14.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f9104dc6640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f910009ead0 0x7f9100141ed0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c880 tx=0x7f90fc008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90fc00ce10 con 0x7f910009ead0 2024-09-15T01:43:14.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f17fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:14.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f17fa640 10 monclient: dump: 2024-09-15T01:43:14.742 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.742 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.742 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.742 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.742 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.742 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.742 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.742 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.742 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.742 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.743 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.743 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.743 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.743 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.743 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.743 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f17fa640 10 monclient: _finish_auth 0 2024-09-15T01:43:14.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:14.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc007cb0 con 0x7f910009ead0 2024-09-15T01:43:14.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f910782b640 5 monclient: authenticate success, global_id 34640 2024-09-15T01:43:14.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f910782b640 10 monclient: _renew_subs 2024-09-15T01:43:14.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f910782b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:43:14.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f910782b640 1 -- 172.21.15.178:0/2199171940 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910011dc20 con 0x7f910009ead0 2024-09-15T01:43:14.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:14.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90fc007e60 con 0x7f910009ead0 2024-09-15T01:43:14.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:14.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f17fa640 10 monclient: dump: 2024-09-15T01:43:14.744 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.744 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.744 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.744 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.744 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.745 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.745 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.745 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.745 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.745 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.745 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.745 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.745 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.745 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.745 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.745 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f90f37fe640 4 set_mon_vals no callback set 2024-09-15T01:43:14.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f910782b640 10 monclient: _renew_subs 2024-09-15T01:43:14.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f910782b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:43:14.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.711+0000 7f910782b640 1 -- 172.21.15.178:0/2199171940 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910011f750 con 0x7f910009ead0 2024-09-15T01:43:14.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f910782b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:14.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f910782b640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910011d250 0x7f910005e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f910782b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f910782b640 1 -- 172.21.15.178:0/2199171940 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f910004c4a0 con 0x7f910011d250 2024-09-15T01:43:14.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f90fc007830 con 0x7f910009ead0 2024-09-15T01:43:14.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f91055c7640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910011d250 0x7f910005e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f91055c7640 10 monclient: get_auth_request con 0x7f910011d250 auth_method 0 2024-09-15T01:43:14.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f91055c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f91055c7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f91055c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f91055c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f91055c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f91055c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f90f17fa640 1 --2- 172.21.15.178:0/2199171940 >> v2:172.21.15.204:6812/28210 conn(0x7f90e40397b0 0x7f90e403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f91055c7640 10 monclient(hunting): handle_auth_done global_id 25161 payload 824 2024-09-15T01:43:14.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f91055c7640 1 -- 172.21.15.178:0/2199171940 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f910004bc80 con 0x7f910011d250 2024-09-15T01:43:14.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f90fc04bce0 con 0x7f910009ead0 2024-09-15T01:43:14.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.712+0000 7f91055c7640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910011d250 0x7f910005e310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f910006e720 tx=0x7f90f4032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90f402ac90 con 0x7f910011d250 2024-09-15T01:43:14.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f90f17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f90f17fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:14.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f90f17fa640 10 monclient: dump: 2024-09-15T01:43:14.748 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.748 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.748 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.748 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.748 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.748 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.748 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.748 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.748 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.748 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.749 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.749 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.749 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.749 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.749 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.749 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f9105dc8640 1 --2- 172.21.15.178:0/2199171940 >> v2:172.21.15.204:6812/28210 conn(0x7f90e40397b0 0x7f90e403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f9105dc8640 10 monclient: get_auth_request con 0x7f90e40397b0 auth_method 0 2024-09-15T01:43:14.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f90f40313a0 con 0x7f910011d250 2024-09-15T01:43:14.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f90f17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:14.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f90f17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:14.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910011d250 msgr2=0x7f910005e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f90f17fa640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910011d250 0x7f910005e310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f910006e720 tx=0x7f90f4032000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910011d250 msgr2=0x7f910005e310 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:14.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f90f17fa640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910011d250 0x7f910005e310 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.713+0000 7f9105dc8640 1 --2- 172.21.15.178:0/2199171940 >> v2:172.21.15.204:6812/28210 conn(0x7f90e40397b0 0x7f90e403bc60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f910006cf70 tx=0x7f90f800b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f910782b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:14.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f910782b640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910005e850 0x7f91000298c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f910782b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f910782b640 1 -- 172.21.15.178:0/2199171940 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f910011fac0 con 0x7f910005e850 2024-09-15T01:43:14.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f91055c7640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910005e850 0x7f91000298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f91055c7640 10 monclient: get_auth_request con 0x7f910005e850 auth_method 0 2024-09-15T01:43:14.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f91055c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f91055c7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f91055c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f91055c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f91055c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f91055c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f91055c7640 10 monclient(hunting): handle_auth_done global_id 25170 payload 824 2024-09-15T01:43:14.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f91055c7640 1 -- 172.21.15.178:0/2199171940 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f910005ed70 con 0x7f910005e850 2024-09-15T01:43:14.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f91055c7640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910005e850 0x7f91000298c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f90f4000c00 tx=0x7f90f403b000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90f403c880 con 0x7f910005e850 2024-09-15T01:43:14.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f90f17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f90f17fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:14.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f90f17fa640 10 monclient: dump: 2024-09-15T01:43:14.752 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.752 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.752 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.752 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.752 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.752 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.752 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.752 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.752 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.752 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.753 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.753 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.753 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.753 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.753 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.753 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f90f403c400 con 0x7f910005e850 2024-09-15T01:43:14.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f90f17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:14.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f90f17fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:14.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910005e850 msgr2=0x7f91000298c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f90f17fa640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910005e850 0x7f91000298c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f90f4000c00 tx=0x7f90f403b000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f90f17fa640 1 -- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910005e850 msgr2=0x7f91000298c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:14.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.719+0000 7f90f17fa640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f910005e850 0x7f91000298c0 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f90f4000c00 tx=0x7f90f403b000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.754 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:14.754 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:14.754 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:14.754 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:14.754 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:14.754 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:14.754 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:14.755 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:14.755 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:14.755 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:14.755 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:14.755 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:14.755 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:14.755 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:14.755 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:14.755 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.755 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:43:14.755 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:14.756 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:14.756 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:14.756 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.756 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.756 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.756 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.756 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.756 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.756 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.756 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.756 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.757 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.757 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:14.757 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:14.757 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.757 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.757 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.757 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.757 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.757 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.757 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.757 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.758 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.758 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.758 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.758 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:14.758 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:14.758 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:14.758 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:14.758 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:14.758 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:14.758 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:14.758 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:14.759 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:14.759 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:14.759 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:14.759 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:14.759 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:14.759 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:14.759 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:14.759 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:14.759 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:14.759 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.759 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.759 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.760 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.760 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.760 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.760 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.760 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.760 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.760 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.760 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:14.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.760 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:14.760 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.760 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:14.761 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:14.761 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.761 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:14.761 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.761 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.761 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:14.761 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.762 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.762 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.762 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:14.762 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:14.762 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.762 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.762 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.762 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.762 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:14.762 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:14.763 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.763 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:14.763 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.763 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.763 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:14.763 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.764 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.764 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.764 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:14.764 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:14.764 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.764 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.764 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.764 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:14.764 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.765 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.766 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.766 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:14.766 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:14.766 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.766 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.766 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.766 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:14.766 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:14.766 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.767 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:14.767 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.767 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.767 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:14.767 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.767 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.768 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.768 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:14.768 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:14.768 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.768 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.768 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.768 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:14.768 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:14.768 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.769 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.769 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.769 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.769 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:14.769 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.769 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.770 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:14.770 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.770 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.770 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:14.770 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:14.770 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.770 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.770 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.771 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.771 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:14.771 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:14.771 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.771 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.771 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.771 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.771 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:14.771 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.771 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.772 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:14.772 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.772 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.772 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:14.772 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:14.772 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.772 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.772 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.773 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:14.773 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:14.773 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.773 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:14.773 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.773 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:14.774 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.774 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.774 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:14.774 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:14.774 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.774 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.774 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.775 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:14.775 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:14.775 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.775 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:14.775 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.776 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:14.776 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.776 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.776 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:14.776 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:14.776 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.776 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.776 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.777 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:14.777 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:14.777 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.777 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:14.777 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.778 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:14.778 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.778 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.778 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:14.778 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:14.778 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.778 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.778 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.779 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.779 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.779 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:14.779 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:14.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.779 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.779 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.779 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:14.779 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.779 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.779 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:14.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.780 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.780 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.780 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:43:14.780 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.780 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.780 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:14.780 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:14.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.720+0000 7f910782b640 1 -- 172.21.15.178:0/2199171940 >> v2:172.21.15.204:6812/28210 conn(0x7f90e40397b0 msgr2=0x7f90e403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.720+0000 7f910782b640 1 --2- 172.21.15.178:0/2199171940 >> v2:172.21.15.204:6812/28210 conn(0x7f90e40397b0 0x7f90e403bc60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f910006cf70 tx=0x7f90f800b040 comp rx=0 tx=0).stop 2024-09-15T01:43:14.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.720+0000 7f910782b640 10 monclient: shutdown 2024-09-15T01:43:14.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.720+0000 7f910782b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:14.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.720+0000 7f910782b640 1 -- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f910009ead0 msgr2=0x7f9100141ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.720+0000 7f910782b640 1 --2- 172.21.15.178:0/2199171940 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f910009ead0 0x7f9100141ed0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c880 tx=0x7f90fc008040 comp rx=0 tx=0).stop 2024-09-15T01:43:14.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.720+0000 7f9105dc8640 1 -- 172.21.15.178:0/2199171940 reap_dead start 2024-09-15T01:43:14.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.720+0000 7f9105dc8640 1 -- 172.21.15.178:0/2199171940 reap_dead start 2024-09-15T01:43:14.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.720+0000 7f910782b640 1 -- 172.21.15.178:0/2199171940 shutdown_connections 2024-09-15T01:43:14.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.720+0000 7f910782b640 1 -- 172.21.15.178:0/2199171940 >> 172.21.15.178:0/2199171940 conn(0x7f91000a4390 msgr2=0x7f910006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:14.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.720+0000 7f910782b640 1 -- 172.21.15.178:0/2199171940 shutdown_connections 2024-09-15T01:43:14.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.720+0000 7f910782b640 1 -- 172.21.15.178:0/2199171940 wait complete. 2024-09-15T01:43:14.783 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:14.783 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:43:14.783 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:43:14.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.874+0000 7fafea434640 10 monclient: get_monmap_and_config 2024-09-15T01:43:14.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.875+0000 7fafea434640 10 monclient: build_initial_monmap 2024-09-15T01:43:14.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.875+0000 7fafea434640 10 monclient: monmap: 2024-09-15T01:43:14.877 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:14.877 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.877 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:14.875905+0000 2024-09-15T01:43:14.877 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:14.875905+0000 2024-09-15T01:43:14.877 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:14.877 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.878 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:14.878 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:14.878 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:14.878 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:14.878 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:14.878 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:14.878 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:14.878 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:14.878 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:14.878 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:14.878 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:14.878 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.875+0000 7fafea434640 1 Processor -- start 2024-09-15T01:43:14.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 1 -- start start 2024-09-15T01:43:14.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 10 monclient: init 2024-09-15T01:43:14.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:14.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 10 monclient: _add_conns ranks=[5,3,7,0,6,4,2,10,1,8,9] 2024-09-15T01:43:14.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fafe40a2930 0x7fafe4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 10 monclient(hunting): picked mon.noname-h con 0x7fafe40a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:14.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe409faa0 0x7fafe409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 10 monclient(hunting): picked mon.noname-g con 0x7fafe409faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:14.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fafe40a08c0 0x7fafe40a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 10 monclient(hunting): picked mon.noname-i con 0x7fafe40a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:14.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe409e0f0 con 0x7fafe409faa0 2024-09-15T01:43:14.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafe409dd80 con 0x7fafe40a2930 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40a2d00 con 0x7fafe40a08c0 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafea434640 10 monclient(hunting): authenticate will time out at 1399.626503s 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafdbfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe409faa0 0x7fafe409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafe89d1640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fafe40a08c0 0x7fafe40a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafe3fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fafe40a2930 0x7fafe4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafdbfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe409faa0 0x7fafe409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:35142/0 (socket says 172.21.15.178:35142) 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafe89d1640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fafe40a08c0 0x7fafe40a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:42804/0 (socket says 172.21.15.178:42804) 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafdbfff640 1 -- 172.21.15.178:0/4099792568 learned_addr learned my addr 172.21.15.178:0/4099792568 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafdbfff640 10 monclient(hunting): get_auth_request con 0x7fafe409faa0 auth_method 0 2024-09-15T01:43:14.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafdbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafe89d1640 10 monclient(hunting): get_auth_request con 0x7fafe40a08c0 auth_method 0 2024-09-15T01:43:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafe89d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafe89d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafe89d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafe3fff640 10 monclient(hunting): get_auth_request con 0x7fafe40a2930 auth_method 0 2024-09-15T01:43:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafe3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafe3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.876+0000 7fafe3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafe3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafe3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafe3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafe89d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafe89d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafe89d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 10 monclient(hunting): handle_auth_done global_id 16257 payload 274 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 1 -- 172.21.15.178:0/4099792568 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fafe40a08c0 msgr2=0x7fafe40a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 1 --2- 172.21.15.178:0/4099792568 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fafe40a08c0 0x7fafe40a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 1 -- 172.21.15.178:0/4099792568 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fafe40a2930 msgr2=0x7fafe4132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 1 --2- 172.21.15.178:0/4099792568 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fafe40a2930 0x7fafe4132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 10 monclient: _finish_hunting 0 2024-09-15T01:43:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 1 monclient: found mon.noname-g 2024-09-15T01:43:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:43:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 1 -- 172.21.15.178:0/4099792568 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe404bf60 con 0x7fafe409faa0 2024-09-15T01:43:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafe3fff640 1 --2- 172.21.15.178:0/4099792568 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fafe40a2930 0x7fafe4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.877+0000 7fafdbfff640 1 --2- 172.21.15.178:0/4099792568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe409faa0 0x7fafe409fe70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009940 tx=0x7fafcc0091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=72d869caf0b333ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafe37fe640 1 -- 172.21.15.178:0/4099792568 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fafcc02ac60 con 0x7fafe409faa0 2024-09-15T01:43:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafe37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafe37fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:43:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafe37fe640 10 monclient: dump: 2024-09-15T01:43:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafe37fe640 10 monclient: _finish_auth 0 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafe37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafe37fe640 1 -- 172.21.15.178:0/4099792568 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc02ade0 con 0x7fafe409faa0 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 5 monclient: authenticate success, global_id 16257 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafe37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafe37fe640 1 -- 172.21.15.178:0/4099792568 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fafcc031ca0 con 0x7fafe409faa0 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 4 set_mon_vals no callback set 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafe37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafe37fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:43:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafe37fe640 10 monclient: dump: 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 10 monclient: shutdown 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 1 -- 172.21.15.178:0/4099792568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe409faa0 msgr2=0x7fafe409fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 1 --2- 172.21.15.178:0/4099792568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe409faa0 0x7fafe409fe70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009940 tx=0x7fafcc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 1 -- 172.21.15.178:0/4099792568 shutdown_connections 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 1 --2- 172.21.15.178:0/4099792568 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fafe40a08c0 0x7fafe40a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 1 --2- 172.21.15.178:0/4099792568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafe409faa0 0x7fafe409fe70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 1 --2- 172.21.15.178:0/4099792568 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fafe40a2930 0x7fafe4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 1 -- 172.21.15.178:0/4099792568 >> 172.21.15.178:0/4099792568 conn(0x7fafe40a5ed0 msgr2=0x7fafe40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 1 -- 172.21.15.178:0/4099792568 shutdown_connections 2024-09-15T01:43:14.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.878+0000 7fafea434640 1 -- 172.21.15.178:0/4099792568 wait complete. 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.879+0000 7fafea434640 10 monclient: build_initial_monmap 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.879+0000 7fafea434640 10 monclient: monmap: 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:14.898 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.879+0000 7fafea434640 1 Processor -- start 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.879+0000 7fafea434640 1 -- start start 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.879+0000 7fafea434640 10 monclient: init 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 10 monclient: _add_conns ranks=[1,0,6,5,7,3,4,2,8] 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe409faa0 0x7fafe411c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 10 monclient(hunting): picked mon.noname-f con 0x7fafe409faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafe40a08c0 0x7fafe409b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 10 monclient(hunting): picked mon.noname-a con 0x7fafe40a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fafe40a2930 0x7fafe4131950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 10 monclient(hunting): picked mon.noname-d con 0x7fafe40a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe411f030 con 0x7fafe40a08c0 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe409d630 con 0x7fafe409faa0 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fafe411db80 con 0x7fafe40a2930 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafea434640 10 monclient(hunting): authenticate will time out at 1399.630239s 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafdbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafe40a08c0 0x7fafe409b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fafe40a2930 0x7fafe4131950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafdbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafe40a08c0 0x7fafe409b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48928/0 (socket says 172.21.15.178:48928) 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafdbfff640 1 -- 172.21.15.178:0/2692990418 learned_addr learned my addr 172.21.15.178:0/2692990418 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:14.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafdbfff640 10 monclient(hunting): get_auth_request con 0x7fafe40a08c0 auth_method 0 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafdbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 10 monclient(hunting): get_auth_request con 0x7fafe40a2930 auth_method 0 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe3fff640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe409faa0 0x7fafe411c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafdbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafdbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafdbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe3fff640 10 monclient(hunting): get_auth_request con 0x7fafe409faa0 auth_method 0 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 10 monclient(hunting): handle_auth_done global_id 25179 payload 824 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 1 -- 172.21.15.178:0/2692990418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe409faa0 msgr2=0x7fafe411c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe409faa0 0x7fafe411c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 1 -- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafe40a08c0 msgr2=0x7fafe409b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafe40a08c0 0x7fafe409b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 10 monclient: _finish_hunting 0 2024-09-15T01:43:14.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 1 monclient: found mon.noname-d 2024-09-15T01:43:14.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:14.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:14.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 1 -- 172.21.15.178:0/2692990418 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd4009390 con 0x7fafe40a2930 2024-09-15T01:43:14.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafdbfff640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafe40a08c0 0x7fafe409b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:14.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe89d1640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fafe40a2930 0x7fafe4131950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009e20 tx=0x7fafdc00b040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fafdc00cd40 con 0x7fafe40a2930 2024-09-15T01:43:14.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe0ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:14.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.880+0000 7fafe0ff9640 10 monclient: dump: 2024-09-15T01:43:14.903 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.903 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafe0ff9640 10 monclient: _finish_auth 0 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafe0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafea434640 5 monclient: authenticate success, global_id 25179 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc00f040 con 0x7fafe40a2930 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafea434640 10 monclient: _renew_subs 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafea434640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafea434640 1 -- 172.21.15.178:0/2692990418 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc009450 con 0x7fafe40a2930 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafe0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fafdc014d80 con 0x7fafe40a2930 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafe0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafe0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafe2ffd640 4 set_mon_vals no callback set 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafe0ff9640 10 monclient: dump: 2024-09-15T01:43:14.905 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.906 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafea434640 10 monclient: _renew_subs 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafea434640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafea434640 1 -- 172.21.15.178:0/2692990418 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe411d4a0 con 0x7fafe40a2930 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafea434640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafea434640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe4131e90 0x7fafe4142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafea434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafea434640 1 -- 172.21.15.178:0/2692990418 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fafe411eef0 con 0x7fafe4131e90 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fafdc014480 con 0x7fafe40a2930 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafe3fff640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe4131e90 0x7fafe4142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.881+0000 7fafe3fff640 10 monclient: get_auth_request con 0x7fafe4131e90 auth_method 0 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 1 --2- 172.21.15.178:0/2692990418 >> v2:172.21.15.204:6812/28210 conn(0x7fafc40396e0 0x7fafc403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe3fff640 10 monclient(hunting): handle_auth_done global_id 34649 payload 824 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fafdc04c9f0 con 0x7fafe40a2930 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe3fff640 1 -- 172.21.15.178:0/2692990418 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafe404c4a0 con 0x7fafe4131e90 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe3fff640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe4131e90 0x7fafe4142090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fafd4011c20 tx=0x7fafd40090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafdbfff640 1 --2- 172.21.15.178:0/2692990418 >> v2:172.21.15.204:6812/28210 conn(0x7fafc40396e0 0x7fafc403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafdbfff640 10 monclient: get_auth_request con 0x7fafc40396e0 auth_method 0 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fafd4011e60 con 0x7fafe4131e90 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 10 monclient: dump: 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.909 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fafd4014440 con 0x7fafe4131e90 2024-09-15T01:43:14.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:14.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:14.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe4131e90 msgr2=0x7fafe4142090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe4131e90 0x7fafe4142090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fafd4011c20 tx=0x7fafd40090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe4131e90 msgr2=0x7fafe4142090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:14.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.882+0000 7fafe0ff9640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe4131e90 0x7fafe4142090 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fafd4011c20 tx=0x7fafd40090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:14.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.883+0000 7fafdbfff640 1 --2- 172.21.15.178:0/2692990418 >> v2:172.21.15.204:6812/28210 conn(0x7fafc40396e0 0x7fafc403bb90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fafe409bab0 tx=0x7fafcc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafea434640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:14.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafea434640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe41425d0 0x7fafe41429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:14.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafea434640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:14.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafea434640 1 -- 172.21.15.178:0/2692990418 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fafe404bc80 con 0x7fafe41425d0 2024-09-15T01:43:14.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafe3fff640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe41425d0 0x7fafe41429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:14.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafe3fff640 10 monclient: get_auth_request con 0x7fafe41425d0 auth_method 0 2024-09-15T01:43:14.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafe3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:14.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafe3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:14.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafe3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:14.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafe3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:14.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafe3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:14.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafe3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafe3fff640 10 monclient(hunting): handle_auth_done global_id 34658 payload 824 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafe3fff640 1 -- 172.21.15.178:0/2692990418 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fafe405ed70 con 0x7fafe41425d0 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.888+0000 7fafe3fff640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe41425d0 0x7fafe41429d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fafd4000c00 tx=0x7fafd401c000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fafd401dbc0 con 0x7fafe41425d0 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe0ff9640 10 monclient: dump: 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:14.913 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:14.914 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:14.914 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:14.914 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:14.914 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:14.914 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:14.914 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:14.914 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:14.914 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:14.914 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:14.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7fafd401c3e0 con 0x7fafe41425d0 2024-09-15T01:43:14.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:14.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:14.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe41425d0 msgr2=0x7fafe41429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe0ff9640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe41425d0 0x7fafe41429d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fafd4000c00 tx=0x7fafd401c000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe0ff9640 1 -- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe41425d0 msgr2=0x7fafe41429d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:14.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe0ff9640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fafe41425d0 0x7fafe41429d0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fafd4000c00 tx=0x7fafd401c000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.915 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:14.915 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:14.915 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:14.916 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.917 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.918 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.918 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.918 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:14.918 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:14.918 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.918 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.918 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.918 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.918 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.918 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.918 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.918 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.919 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.919 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.919 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:14.919 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:14.919 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:14.919 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:14.919 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:14.919 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:14.919 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:14.919 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:14.919 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:14.920 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:14.921 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:14.921 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:14.921 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:14.921 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:14.921 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.921 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:14.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.921 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:14.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.921 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:14.921 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:14.924 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.924 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.924 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:14.924 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.926 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:14.926 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:14.926 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.926 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.926 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.926 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:14.926 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.926 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:14.927 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.929 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.930 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.931 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.932 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.933 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:14.935 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.937 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.937 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:14.937 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.937 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.937 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.937 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:14.937 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:14.937 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.937 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.937 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:14.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:14.939 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.940 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:14.941 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:14.941 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:14.941 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:14.941 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:14.941 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:14.941 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:14.941 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:14.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:14.941 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:14.941 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:14.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafea434640 1 -- 172.21.15.178:0/2692990418 >> v2:172.21.15.204:6812/28210 conn(0x7fafc40396e0 msgr2=0x7fafc403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafea434640 1 --2- 172.21.15.178:0/2692990418 >> v2:172.21.15.204:6812/28210 conn(0x7fafc40396e0 0x7fafc403bb90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fafe409bab0 tx=0x7fafcc032000 comp rx=0 tx=0).stop 2024-09-15T01:43:14.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafea434640 10 monclient: shutdown 2024-09-15T01:43:14.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafea434640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:14.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafea434640 1 -- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fafe40a2930 msgr2=0x7fafe4131950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:14.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafea434640 1 --2- 172.21.15.178:0/2692990418 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fafe40a2930 0x7fafe4131950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009e20 tx=0x7fafdc00b040 comp rx=0 tx=0).stop 2024-09-15T01:43:14.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe89d1640 1 -- 172.21.15.178:0/2692990418 reap_dead start 2024-09-15T01:43:14.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafe89d1640 1 -- 172.21.15.178:0/2692990418 reap_dead start 2024-09-15T01:43:14.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafea434640 1 -- 172.21.15.178:0/2692990418 shutdown_connections 2024-09-15T01:43:14.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafea434640 1 -- 172.21.15.178:0/2692990418 >> 172.21.15.178:0/2692990418 conn(0x7fafe40a5ed0 msgr2=0x7fafe409e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:14.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafea434640 1 -- 172.21.15.178:0/2692990418 shutdown_connections 2024-09-15T01:43:14.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:14.889+0000 7fafea434640 1 -- 172.21.15.178:0/2692990418 wait complete. 2024-09-15T01:43:14.944 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:14.944 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:43:14.944 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:43:15.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.037+0000 7f868a95d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:15.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.037+0000 7f868a95d640 10 monclient: build_initial_monmap 2024-09-15T01:43:15.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.037+0000 7f868a95d640 10 monclient: monmap: 2024-09-15T01:43:15.040 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:15.040 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.040 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:15.038275+0000 2024-09-15T01:43:15.040 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:15.038275+0000 2024-09-15T01:43:15.040 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:15.040 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.040 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:15.040 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:15.040 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:15.041 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:15.041 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:15.041 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:15.041 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:15.041 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:15.041 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:15.041 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:15.041 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:15.041 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 1 Processor -- start 2024-09-15T01:43:15.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 1 -- start start 2024-09-15T01:43:15.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 10 monclient: init 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 10 monclient: _add_conns ranks=[8,5,10,4,2,3,6,0,1,7,9] 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86840a2930 0x7f8684132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 10 monclient(hunting): picked mon.noname-e con 0x7f86840a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f868409faa0 0x7f868409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 10 monclient(hunting): picked mon.noname-h con 0x7f868409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f86840a08c0 0x7f86840a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 10 monclient(hunting): picked mon.noname-f con 0x7f86840a08c0 addr v2:172.21.15.204:3300/0 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868409e0f0 con 0x7f86840a08c0 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f868409dd80 con 0x7f868409faa0 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f86840a2d00 con 0x7f86840a2930 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f868a95d640 10 monclient(hunting): authenticate will time out at 1399.788750s 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f8683fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86840a2930 0x7f8684132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f8683fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86840a2930 0x7f8684132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:55094/0 (socket says 172.21.15.178:55094) 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f8683fff640 1 -- 172.21.15.178:0/2779216 learned_addr learned my addr 172.21.15.178:0/2779216 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f8683fff640 10 monclient(hunting): get_auth_request con 0x7f86840a2930 auth_method 0 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f8683fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f8683fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f8683fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.038+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8688efa640 1 --2- 172.21.15.178:0/2779216 >> v2:172.21.15.204:3300/0 conn(0x7f86840a08c0 0x7f86840a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8688efa640 10 monclient(hunting): get_auth_request con 0x7f86840a08c0 auth_method 0 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8688efa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8688efa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8688efa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f86837fe640 1 --2- 172.21.15.178:0/2779216 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f868409faa0 0x7f868409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f86837fe640 10 monclient(hunting): get_auth_request con 0x7f868409faa0 auth_method 0 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f86837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f86837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f86837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8683fff640 10 monclient(hunting): handle_auth_done global_id 34667 payload 274 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8683fff640 1 -- 172.21.15.178:0/2779216 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f868409faa0 msgr2=0x7f868409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8683fff640 1 --2- 172.21.15.178:0/2779216 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f868409faa0 0x7f868409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8683fff640 1 -- 172.21.15.178:0/2779216 >> v2:172.21.15.204:3300/0 conn(0x7f86840a08c0 msgr2=0x7f86840a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8683fff640 1 --2- 172.21.15.178:0/2779216 >> v2:172.21.15.204:3300/0 conn(0x7f86840a08c0 0x7f86840a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8683fff640 10 monclient: _finish_hunting 0 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8683fff640 1 monclient: found mon.noname-e 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8683fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8683fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8683fff640 1 -- 172.21.15.178:0/2779216 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868404bf60 con 0x7f86840a2930 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8688efa640 1 --2- 172.21.15.178:0/2779216 >> v2:172.21.15.204:3300/0 conn(0x7f86840a08c0 0x7f86840a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8683fff640 1 --2- 172.21.15.178:0/2779216 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86840a2930 0x7f8684132de0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8674009910 tx=0x7f86740091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=2858e72627fbefab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.039+0000 7f8682ffd640 1 -- 172.21.15.178:0/2779216 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f867402ac80 con 0x7f86840a2930 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f8682ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f8682ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f8682ffd640 10 monclient: dump: 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.054 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.055 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f8682ffd640 10 monclient: _finish_auth 0 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f8682ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 5 monclient: authenticate success, global_id 34667 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f8682ffd640 1 -- 172.21.15.178:0/2779216 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867402ae00 con 0x7f86840a2930 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f8682ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f8682ffd640 1 -- 172.21.15.178:0/2779216 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8674031ca0 con 0x7f86840a2930 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 4 set_mon_vals no callback set 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f8682ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f8682ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f8682ffd640 10 monclient: dump: 2024-09-15T01:43:15.056 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.057 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 10 monclient: shutdown 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 1 -- 172.21.15.178:0/2779216 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86840a2930 msgr2=0x7f8684132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 1 --2- 172.21.15.178:0/2779216 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86840a2930 0x7f8684132de0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8674009910 tx=0x7f86740091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 1 -- 172.21.15.178:0/2779216 shutdown_connections 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 1 --2- 172.21.15.178:0/2779216 >> v2:172.21.15.204:3300/0 conn(0x7f86840a08c0 0x7f86840a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 1 --2- 172.21.15.178:0/2779216 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f868409faa0 0x7f868409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 1 --2- 172.21.15.178:0/2779216 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86840a2930 0x7f8684132de0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 1 -- 172.21.15.178:0/2779216 >> 172.21.15.178:0/2779216 conn(0x7f86840a5ed0 msgr2=0x7f86840a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 1 -- 172.21.15.178:0/2779216 shutdown_connections 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.040+0000 7f868a95d640 1 -- 172.21.15.178:0/2779216 wait complete. 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.041+0000 7f868a95d640 10 monclient: build_initial_monmap 2024-09-15T01:43:15.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.041+0000 7f868a95d640 10 monclient: monmap: 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:15.059 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:15.060 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:15.060 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:15.060 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.041+0000 7f868a95d640 1 Processor -- start 2024-09-15T01:43:15.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.041+0000 7f868a95d640 1 -- start start 2024-09-15T01:43:15.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.041+0000 7f868a95d640 10 monclient: init 2024-09-15T01:43:15.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:15.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient: _add_conns ranks=[7,2,1,5,3,0,6,4,8] 2024-09-15T01:43:15.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f868409faa0 0x7f868409b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient(hunting): picked mon.noname-i con 0x7f868409faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:15.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86840a08c0 0x7f8684131b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient(hunting): picked mon.noname-b con 0x7f86840a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86840a2930 0x7f86841320a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient(hunting): picked mon.noname-f con 0x7f86840a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868409d630 con 0x7f86840a2930 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868411eff0 con 0x7f86840a08c0 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f86840a2390 con 0x7f868409faa0 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient(hunting): authenticate will time out at 1399.792163s 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86840a08c0 0x7f8684131b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86840a08c0 0x7f8684131b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:48266/0 (socket says 172.21.15.178:48266) 2024-09-15T01:43:15.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 1 -- 172.21.15.178:0/4058570048 learned_addr learned my addr 172.21.15.178:0/4058570048 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 10 monclient(hunting): get_auth_request con 0x7f86840a08c0 auth_method 0 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8683fff640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f868409faa0 0x7f868409b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8683fff640 10 monclient(hunting): get_auth_request con 0x7f868409faa0 auth_method 0 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8683fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8683fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8683fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8688efa640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86840a2930 0x7f86841320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8688efa640 10 monclient(hunting): get_auth_request con 0x7f86840a2930 auth_method 0 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8688efa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8688efa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8688efa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8688efa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8688efa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f8688efa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 10 monclient(hunting): handle_auth_done global_id 34670 payload 824 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 1 -- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f868409faa0 msgr2=0x7f868409b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f868409faa0 0x7f868409b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 1 -- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86840a2930 msgr2=0x7f86841320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86840a2930 0x7f86841320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 10 monclient: _finish_hunting 0 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 1 monclient: found mon.noname-b 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 1 -- 172.21.15.178:0/4058570048 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8674009450 con 0x7f86840a08c0 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86837fe640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86840a08c0 0x7f8684131b60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f867800d890 tx=0x7f8678008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f867800de20 con 0x7f86840a08c0 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f866bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f866bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f866bfff640 10 monclient: dump: 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.064 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.065 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.065 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.065 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.065 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.065 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.065 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.065 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.065 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.065 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.065 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.065 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.065 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f866bfff640 10 monclient: _finish_auth 0 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f866bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 5 monclient: authenticate success, global_id 34670 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8678009d30 con 0x7f86840a08c0 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient: _renew_subs 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 1 -- 172.21.15.178:0/4058570048 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86841424f0 con 0x7f86840a08c0 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f866bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8678009ea0 con 0x7f86840a08c0 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f866bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f866bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:15.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f866bfff640 10 monclient: dump: 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.067 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.068 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.068 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient: _renew_subs 2024-09-15T01:43:15.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f868a95d640 1 -- 172.21.15.178:0/4058570048 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868409c140 con 0x7f86840a08c0 2024-09-15T01:43:15.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.042+0000 7f86827fc640 4 set_mon_vals no callback set 2024-09-15T01:43:15.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f868a95d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:15.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f868a95d640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411c530 0x7f868411c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f868a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f868a95d640 1 -- 172.21.15.178:0/4058570048 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8684132670 con 0x7f868411c530 2024-09-15T01:43:15.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f86780098b0 con 0x7f86840a08c0 2024-09-15T01:43:15.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f8683fff640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411c530 0x7f868411c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f8683fff640 10 monclient: get_auth_request con 0x7f868411c530 auth_method 0 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f8683fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f8683fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f8683fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f866bfff640 1 --2- 172.21.15.178:0/4058570048 >> v2:172.21.15.204:6812/28210 conn(0x7f8670039710 0x7f867003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f867804c030 con 0x7f86840a08c0 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.043+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f8688efa640 1 --2- 172.21.15.178:0/4058570048 >> v2:172.21.15.204:6812/28210 conn(0x7f8670039710 0x7f867003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f8683fff640 10 monclient(hunting): handle_auth_done global_id 54163 payload 824 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f8683fff640 1 -- 172.21.15.178:0/4058570048 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f868404c4a0 con 0x7f868411c530 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f8688efa640 10 monclient: get_auth_request con 0x7f8670039710 auth_method 0 2024-09-15T01:43:15.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f8683fff640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411c530 0x7f868411c900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f86840a3290 tx=0x7f8674032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f867402ac80 con 0x7f868411c530 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f866bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f866bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f866bfff640 10 monclient: dump: 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.070 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f8688efa640 1 --2- 172.21.15.178:0/4058570048 >> v2:172.21.15.204:6812/28210 conn(0x7f8670039710 0x7f867003bbc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f867c0076e0 tx=0x7f867c009990 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f8674031a40 con 0x7f868411c530 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f866bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f866bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411c530 msgr2=0x7f868411c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f866bfff640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411c530 0x7f868411c900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f86840a3290 tx=0x7f8674032000 comp rx=0 tx=0).stop 2024-09-15T01:43:15.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411c530 msgr2=0x7f868411c900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:15.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.044+0000 7f866bfff640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411c530 0x7f868411c900 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f86840a3290 tx=0x7f8674032000 comp rx=0 tx=0).stop 2024-09-15T01:43:15.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.050+0000 7f868a95d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:15.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.050+0000 7f868a95d640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411ce40 0x7f8684096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.050+0000 7f868a95d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.050+0000 7f868a95d640 1 -- 172.21.15.178:0/4058570048 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868409bb00 con 0x7f868411ce40 2024-09-15T01:43:15.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.050+0000 7f8683fff640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411ce40 0x7f8684096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.050+0000 7f8683fff640 10 monclient: get_auth_request con 0x7f868411ce40 auth_method 0 2024-09-15T01:43:15.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.050+0000 7f8683fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.050+0000 7f8683fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.050+0000 7f8683fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.050+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.050+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.050+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f8683fff640 10 monclient(hunting): handle_auth_done global_id 54172 payload 824 2024-09-15T01:43:15.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f8683fff640 1 -- 172.21.15.178:0/4058570048 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f868404bc80 con 0x7f868411ce40 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f8683fff640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411ce40 0x7f8684096a30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8674009210 tx=0x7f867403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f867404e070 con 0x7f868411ce40 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f866bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f866bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f866bfff640 10 monclient: dump: 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.075 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f867404a640 con 0x7f868411ce40 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f866bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f866bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411ce40 msgr2=0x7f8684096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f866bfff640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411ce40 0x7f8684096a30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8674009210 tx=0x7f867403d000 comp rx=0 tx=0).stop 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f866bfff640 1 -- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411ce40 msgr2=0x7f8684096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:15.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.051+0000 7f866bfff640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868411ce40 0x7f8684096a30 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8674009210 tx=0x7f867403d000 comp rx=0 tx=0).stop 2024-09-15T01:43:15.077 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:15.077 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:15.077 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:15.077 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:15.077 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:15.077 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:15.077 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:15.077 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:15.077 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:15.077 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:15.077 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:15.078 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:15.079 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:15.080 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:15.081 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:15.081 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:15.081 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:15.081 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:15.081 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:15.081 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:15.081 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:15.081 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:15.081 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:15.081 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:15.081 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:15.081 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:15.082 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.085 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.087 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:15.088 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:15.088 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.088 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:15.088 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.088 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:15.088 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:15.089 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:15.091 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.092 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.095 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:15.097 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.098 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.099 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:15.099 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:15.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.052+0000 7f868a95d640 1 -- 172.21.15.178:0/4058570048 >> v2:172.21.15.204:6812/28210 conn(0x7f8670039710 msgr2=0x7f867003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.052+0000 7f868a95d640 1 --2- 172.21.15.178:0/4058570048 >> v2:172.21.15.204:6812/28210 conn(0x7f8670039710 0x7f867003bbc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f867c0076e0 tx=0x7f867c009990 comp rx=0 tx=0).stop 2024-09-15T01:43:15.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.052+0000 7f868a95d640 10 monclient: shutdown 2024-09-15T01:43:15.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.052+0000 7f868a95d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:15.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.052+0000 7f868a95d640 1 -- 172.21.15.178:0/4058570048 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86840a08c0 msgr2=0x7f8684131b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.052+0000 7f868a95d640 1 --2- 172.21.15.178:0/4058570048 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86840a08c0 0x7f8684131b60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f867800d890 tx=0x7f8678008040 comp rx=0 tx=0).stop 2024-09-15T01:43:15.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.052+0000 7f8688efa640 1 -- 172.21.15.178:0/4058570048 reap_dead start 2024-09-15T01:43:15.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.052+0000 7f8688efa640 1 -- 172.21.15.178:0/4058570048 reap_dead start 2024-09-15T01:43:15.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.052+0000 7f868a95d640 1 -- 172.21.15.178:0/4058570048 shutdown_connections 2024-09-15T01:43:15.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.052+0000 7f868a95d640 1 -- 172.21.15.178:0/4058570048 >> 172.21.15.178:0/4058570048 conn(0x7f86840a5ed0 msgr2=0x7f868409e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:15.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.052+0000 7f868a95d640 1 -- 172.21.15.178:0/4058570048 shutdown_connections 2024-09-15T01:43:15.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.052+0000 7f868a95d640 1 -- 172.21.15.178:0/4058570048 wait complete. 2024-09-15T01:43:15.101 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:15.101 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:43:15.101 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:43:15.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.194+0000 7fa4e6ee8640 10 monclient: get_monmap_and_config 2024-09-15T01:43:15.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.194+0000 7fa4e6ee8640 10 monclient: build_initial_monmap 2024-09-15T01:43:15.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.194+0000 7fa4e6ee8640 10 monclient: monmap: 2024-09-15T01:43:15.196 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:15.196 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.196 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:15.194917+0000 2024-09-15T01:43:15.196 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:15.194917+0000 2024-09-15T01:43:15.196 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:15.196 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.196 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:15.196 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:15.196 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:15.197 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:15.197 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:15.197 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:15.197 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:15.197 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:15.197 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:15.197 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:15.197 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:15.197 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.194+0000 7fa4e6ee8640 1 Processor -- start 2024-09-15T01:43:15.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.194+0000 7fa4e6ee8640 1 -- start start 2024-09-15T01:43:15.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.194+0000 7fa4e6ee8640 10 monclient: init 2024-09-15T01:43:15.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:15.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 10 monclient: _add_conns ranks=[5,0,8,1,6,2,4,3,9,10,7] 2024-09-15T01:43:15.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa4e00a2930 0x7fa4e0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 10 monclient(hunting): picked mon.noname-h con 0x7fa4e00a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:15.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa4e009faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa4e009fe70 0x7fa4e00a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 10 monclient(hunting): picked mon.noname-e con 0x7fa4e009fe70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e004bf60 con 0x7fa4e009faa0 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e009dd80 con 0x7fa4e00a2930 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e00a2d00 con 0x7fa4e009fe70 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e6ee8640 10 monclient(hunting): authenticate will time out at 1399.945469s 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4dffff640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa4e009faa0 0x7fa4e00a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43150/0 (socket says 172.21.15.178:43150) 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa4e009fe70 0x7fa4e00a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4dffff640 1 -- 172.21.15.178:0/2498827530 learned_addr learned my addr 172.21.15.178:0/2498827530 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:43:15.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 10 monclient(hunting): get_auth_request con 0x7fa4e009fe70 auth_method 0 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e4c84640 1 --2- 172.21.15.178:0/2498827530 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa4e00a2930 0x7fa4e0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e4c84640 10 monclient(hunting): get_auth_request con 0x7fa4e00a2930 auth_method 0 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e4c84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e4c84640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e4c84640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 10 monclient(hunting): handle_auth_done global_id 34676 payload 274 2024-09-15T01:43:15.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4df7fe640 1 -- 172.21.15.178:0/2498827530 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510583105 0 0) 0x7fa4e004bf60 con 0x7fa4e009faa0 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 1 -- 172.21.15.178:0/2498827530 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa4e00a2930 msgr2=0x7fa4e0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 1 --2- 172.21.15.178:0/2498827530 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa4e00a2930 0x7fa4e0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 1 -- 172.21.15.178:0/2498827530 >> v1:172.21.15.178:6789/0 conn(0x7fa4e009faa0 legacy=0x7fa4e00a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 10 monclient: _finish_hunting 0 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 1 monclient: found mon.noname-e 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4e5485640 1 -- 172.21.15.178:0/2498827530 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e00a0310 con 0x7fa4e009fe70 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.195+0000 7fa4df7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4e5485640 1 --2- 172.21.15.178:0/2498827530 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa4e009fe70 0x7fa4e00a0fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009e90 tx=0x7fa4d8009780 comp rx=0 tx=0).ready entity=mon.8 client_cookie=ae02326c6c72d88b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4df7fe640 1 -- 172.21.15.178:0/2498827530 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4d8031880 con 0x7fa4e009fe70 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4df7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4df7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:15.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4df7fe640 10 monclient: dump: 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.210 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4df7fe640 10 monclient: _finish_auth 0 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4df7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4df7fe640 1 -- 172.21.15.178:0/2498827530 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d802d050 con 0x7fa4e009fe70 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4e6ee8640 5 monclient: authenticate success, global_id 34676 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4e6ee8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4df7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4e6ee8640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4e6ee8640 4 set_mon_vals no callback set 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4df7fe640 1 -- 172.21.15.178:0/2498827530 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4d80303a0 con 0x7fa4e009fe70 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4df7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4df7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.196+0000 7fa4df7fe640 10 monclient: dump: 2024-09-15T01:43:15.211 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.212 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.197+0000 7fa4e6ee8640 10 monclient: shutdown 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.197+0000 7fa4e6ee8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.197+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/2498827530 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa4e009fe70 msgr2=0x7fa4e00a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.197+0000 7fa4e6ee8640 1 --2- 172.21.15.178:0/2498827530 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa4e009fe70 0x7fa4e00a0fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009e90 tx=0x7fa4d8009780 comp rx=0 tx=0).stop 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.197+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/2498827530 shutdown_connections 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.197+0000 7fa4e6ee8640 1 --2- 172.21.15.178:0/2498827530 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa4e009fe70 0x7fa4e00a0fd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.197+0000 7fa4e6ee8640 1 --2- 172.21.15.178:0/2498827530 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa4e00a2930 0x7fa4e0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.197+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/2498827530 >> 172.21.15.178:0/2498827530 conn(0x7fa4e00a5ed0 msgr2=0x7fa4e00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.197+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/2498827530 shutdown_connections 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.197+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/2498827530 wait complete. 2024-09-15T01:43:15.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.197+0000 7fa4e6ee8640 10 monclient: build_initial_monmap 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.197+0000 7fa4e6ee8640 10 monclient: monmap: 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:15.214 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:15.215 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:15.215 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:15.215 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 1 Processor -- start 2024-09-15T01:43:15.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 1 -- start start 2024-09-15T01:43:15.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 10 monclient: init 2024-09-15T01:43:15.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:15.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 10 monclient: _add_conns ranks=[3,6,2,7,8,4,0,1,5] 2024-09-15T01:43:15.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e009faa0 0x7fa4e009af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 10 monclient(hunting): picked mon.noname-g con 0x7fa4e009faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:15.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa4e009fe70 0x7fa4e0131730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 10 monclient(hunting): picked mon.noname-d con 0x7fa4e009fe70 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:15.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4e00a2930 0x7fa4e011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 10 monclient(hunting): picked mon.noname-b con 0x7fa4e00a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:15.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e00a2390 con 0x7fa4e00a2930 2024-09-15T01:43:15.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e009d630 con 0x7fa4e009faa0 2024-09-15T01:43:15.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e0142a10 con 0x7fa4e009fe70 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e6ee8640 10 monclient(hunting): authenticate will time out at 1399.948794s 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4dffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa4e009fe70 0x7fa4e0131730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e5485640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4e00a2930 0x7fa4e011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e5485640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4e00a2930 0x7fa4e011c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:48274/0 (socket says 172.21.15.178:48274) 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4dffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa4e009fe70 0x7fa4e0131730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44350/0 (socket says 172.21.15.178:44350) 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e5485640 1 -- 172.21.15.178:0/3492306216 learned_addr learned my addr 172.21.15.178:0/3492306216 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e5485640 10 monclient(hunting): get_auth_request con 0x7fa4e00a2930 auth_method 0 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e5485640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e5485640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4e5485640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4dffff640 10 monclient(hunting): get_auth_request con 0x7fa4e009fe70 auth_method 0 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4dffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.198+0000 7fa4dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e4c84640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e009faa0 0x7fa4e009af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e4c84640 10 monclient(hunting): get_auth_request con 0x7fa4e009faa0 auth_method 0 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e4c84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e4c84640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e4c84640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e4c84640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e4c84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e4c84640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 10 monclient(hunting): handle_auth_done global_id 34679 payload 824 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 1 -- 172.21.15.178:0/3492306216 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa4e009fe70 msgr2=0x7fa4e0131730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa4e009fe70 0x7fa4e0131730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 1 -- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e009faa0 msgr2=0x7fa4e009af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e009faa0 0x7fa4e009af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 10 monclient: _finish_hunting 0 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 1 monclient: found mon.noname-b 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 1 -- 172.21.15.178:0/3492306216 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d8009a20 con 0x7fa4e00a2930 2024-09-15T01:43:15.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e5485640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4e00a2930 0x7fa4e011c530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009d30 tx=0x7fa4d8037000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4d8030c80 con 0x7fa4e00a2930 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dcff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dcff9640 10 monclient: dump: 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.220 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dcff9640 10 monclient: _finish_auth 0 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d8030e00 con 0x7fa4e00a2930 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e6ee8640 5 monclient: authenticate success, global_id 34679 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e6ee8640 10 monclient: _renew_subs 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e6ee8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/3492306216 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e011eb30 con 0x7fa4e00a2930 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dcff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4d803d8b0 con 0x7fa4e00a2930 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dcff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4dcff9640 10 monclient: dump: 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.222 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4deffd640 4 set_mon_vals no callback set 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e6ee8640 10 monclient: _renew_subs 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e6ee8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.199+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/3492306216 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e011e850 con 0x7fa4e00a2930 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e6ee8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e6ee8640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0136fd0 0x7fa4e011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e6ee8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/3492306216 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e004c4a0 con 0x7fa4e0136fd0 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa4d8003040 con 0x7fa4e00a2930 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e4c84640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0136fd0 0x7fa4e011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4dcff9640 1 --2- 172.21.15.178:0/3492306216 >> v2:172.21.15.204:6812/28210 conn(0x7fa4c00397e0 0x7fa4c003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e4c84640 10 monclient: get_auth_request con 0x7fa4e0136fd0 auth_method 0 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e4c84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e4c84640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e4c84640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fa4d806d050 con 0x7fa4e00a2930 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e4c84640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e4c84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e4c84640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4dffff640 1 --2- 172.21.15.178:0/3492306216 >> v2:172.21.15.204:6812/28210 conn(0x7fa4c00397e0 0x7fa4c003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4dffff640 10 monclient: get_auth_request con 0x7fa4c00397e0 auth_method 0 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e4c84640 10 monclient(hunting): handle_auth_done global_id 16275 payload 824 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.200+0000 7fa4e4c84640 1 -- 172.21.15.178:0/3492306216 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4e005ed70 con 0x7fa4e0136fd0 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4e4c84640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0136fd0 0x7fa4e011ca70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00108b0 tx=0x7fa4d0008040 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4d0014430 con 0x7fa4e0136fd0 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4dcff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4dcff9640 10 monclient: dump: 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.225 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4dffff640 1 --2- 172.21.15.178:0/3492306216 >> v2:172.21.15.204:6812/28210 conn(0x7fa4c00397e0 0x7fa4c003bc90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8000c00 tx=0x7fa4c8007ea0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fa4d000eb40 con 0x7fa4e0136fd0 2024-09-15T01:43:15.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4dcff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:15.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4dcff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:15.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0136fd0 msgr2=0x7fa4e011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4dcff9640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0136fd0 0x7fa4e011ca70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00108b0 tx=0x7fa4d0008040 comp rx=0 tx=0).stop 2024-09-15T01:43:15.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0136fd0 msgr2=0x7fa4e011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:15.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.201+0000 7fa4dcff9640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0136fd0 0x7fa4e011ca70 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00108b0 tx=0x7fa4d0008040 comp rx=0 tx=0).stop 2024-09-15T01:43:15.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e6ee8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:15.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e6ee8640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0131c70 0x7fa4e011cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e6ee8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/3492306216 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e004bc80 con 0x7fa4e0131c70 2024-09-15T01:43:15.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e4c84640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0131c70 0x7fa4e011cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e4c84640 10 monclient: get_auth_request con 0x7fa4e0131c70 auth_method 0 2024-09-15T01:43:15.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e4c84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e4c84640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e4c84640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e4c84640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e4c84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e4c84640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e4c84640 10 monclient(hunting): handle_auth_done global_id 16284 payload 824 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.207+0000 7fa4e4c84640 1 -- 172.21.15.178:0/3492306216 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa4e011ed70 con 0x7fa4e0131c70 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4e4c84640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0131c70 0x7fa4e011cfb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00108b0 tx=0x7fa4d001c000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4d001cb80 con 0x7fa4e0131c70 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4dcff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4dcff9640 10 monclient: dump: 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.230 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.231 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7fa4d001c700 con 0x7fa4e0131c70 2024-09-15T01:43:15.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4dcff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:15.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4dcff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:15.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0131c70 msgr2=0x7fa4e011cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4dcff9640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0131c70 0x7fa4e011cfb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00108b0 tx=0x7fa4d001c000 comp rx=0 tx=0).stop 2024-09-15T01:43:15.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4dcff9640 1 -- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0131c70 msgr2=0x7fa4e011cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:15.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4dcff9640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4e0131c70 0x7fa4e011cfb0 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00108b0 tx=0x7fa4d001c000 comp rx=0 tx=0).stop 2024-09-15T01:43:15.232 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:15.232 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:15.232 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:15.233 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:15.233 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:15.233 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:15.233 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:15.233 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:15.233 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:15.233 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:15.233 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:15.233 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:15.233 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:15.233 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:15.233 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:15.234 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.235 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:15.236 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.237 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:15.238 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:15.240 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.241 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:15.242 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:15.243 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.245 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:15.247 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.249 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:15.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.251 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:15.252 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:15.253 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:15.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/3492306216 >> v2:172.21.15.204:6812/28210 conn(0x7fa4c00397e0 msgr2=0x7fa4c003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4e6ee8640 1 --2- 172.21.15.178:0/3492306216 >> v2:172.21.15.204:6812/28210 conn(0x7fa4c00397e0 0x7fa4c003bc90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8000c00 tx=0x7fa4c8007ea0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4e6ee8640 10 monclient: shutdown 2024-09-15T01:43:15.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4e6ee8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:15.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/3492306216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4e00a2930 msgr2=0x7fa4e011c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4e6ee8640 1 --2- 172.21.15.178:0/3492306216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4e00a2930 0x7fa4e011c530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009d30 tx=0x7fa4d8037000 comp rx=0 tx=0).stop 2024-09-15T01:43:15.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4e5485640 1 -- 172.21.15.178:0/3492306216 reap_dead start 2024-09-15T01:43:15.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4e5485640 1 -- 172.21.15.178:0/3492306216 reap_dead start 2024-09-15T01:43:15.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/3492306216 shutdown_connections 2024-09-15T01:43:15.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.208+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/3492306216 >> 172.21.15.178:0/3492306216 conn(0x7fa4e00a5ed0 msgr2=0x7fa4e009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:15.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.209+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/3492306216 shutdown_connections 2024-09-15T01:43:15.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.209+0000 7fa4e6ee8640 1 -- 172.21.15.178:0/3492306216 wait complete. 2024-09-15T01:43:15.256 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:15.256 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:43:15.256 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:43:15.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.347+0000 7f97f8034640 10 monclient: get_monmap_and_config 2024-09-15T01:43:15.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.347+0000 7f97f8034640 10 monclient: build_initial_monmap 2024-09-15T01:43:15.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.347+0000 7f97f8034640 10 monclient: monmap: 2024-09-15T01:43:15.349 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:15.349 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:15.348431+0000 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:15.348431+0000 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:15.350 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:15.351 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:15.351 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 1 Processor -- start 2024-09-15T01:43:15.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 1 -- start start 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 10 monclient: init 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 10 monclient: _add_conns ranks=[1,9,8,3,4,7,6,2,5,0,10] 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f97f00a2390 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97f009f8a0 0x7f97f009fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 10 monclient(hunting): picked mon.noname-a con 0x7f97f009f8a0 addr v2:172.21.15.178:3300/0 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97f0137bb0 0x7f97f0139f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 10 monclient(hunting): picked mon.noname-e con 0x7f97f0137bb0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f004bf60 con 0x7f97f00a2390 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f009d700 con 0x7f97f009f8a0 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f97f009ddb0 con 0x7f97f0137bb0 2024-09-15T01:43:15.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:15.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.348+0000 7f97f8034640 10 monclient(hunting): authenticate will time out at 1400.098951s 2024-09-15T01:43:15.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97f0137bb0 0x7f97f0139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f55cf640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97f009f8a0 0x7f97f009fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f55cf640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97f009f8a0 0x7f97f009fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48936/0 (socket says 172.21.15.178:48936) 2024-09-15T01:43:15.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f55cf640 1 -- 172.21.15.178:0/1097417490 learned_addr learned my addr 172.21.15.178:0/1097417490 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:15.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f55cf640 10 monclient(hunting): get_auth_request con 0x7f97f009f8a0 auth_method 0 2024-09-15T01:43:15.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97f0137bb0 0x7f97f0139f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:55106/0 (socket says 172.21.15.178:55106) 2024-09-15T01:43:15.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f55cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f55cf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f55cf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 10 monclient(hunting): get_auth_request con 0x7f97f0137bb0 auth_method 0 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f55cf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f55cf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f55cf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 10 monclient(hunting): handle_auth_done global_id 34685 payload 274 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 1 -- 172.21.15.178:0/1097417490 >> v2:172.21.15.178:3300/0 conn(0x7f97f009f8a0 msgr2=0x7f97f009fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 1 --2- 172.21.15.178:0/1097417490 >> v2:172.21.15.178:3300/0 conn(0x7f97f009f8a0 0x7f97f009fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 1 -- 172.21.15.178:0/1097417490 >> v1:172.21.15.204:6789/0 conn(0x7f97f00a2390 legacy=0x7f97f0132a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 10 monclient: _finish_hunting 0 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 1 monclient: found mon.noname-e 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f65d1640 1 -- 172.21.15.178:0/1097417490 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f00a2760 con 0x7f97f0137bb0 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.349+0000 7f97f55cf640 1 --2- 172.21.15.178:0/1097417490 >> v2:172.21.15.178:3300/0 conn(0x7f97f009f8a0 0x7f97f009fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f65d1640 1 --2- 172.21.15.178:0/1097417490 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97f0137bb0 0x7f97f0139f90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009e60 tx=0x7f97e8009780 comp rx=0 tx=0).ready entity=mon.8 client_cookie=45e458f28d871be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f4dce640 1 -- 172.21.15.178:0/1097417490 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f97e80079d0 con 0x7f97f0137bb0 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f4dce640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f4dce640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f4dce640 10 monclient: dump: 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.363 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.364 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f4dce640 10 monclient: _finish_auth 0 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f4dce640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f4dce640 1 -- 172.21.15.178:0/1097417490 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e802d050 con 0x7f97f0137bb0 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 5 monclient: authenticate success, global_id 34685 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f4dce640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f4dce640 1 -- 172.21.15.178:0/1097417490 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f97e8007bd0 con 0x7f97f0137bb0 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 4 set_mon_vals no callback set 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f4dce640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f4dce640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f4dce640 10 monclient: dump: 2024-09-15T01:43:15.365 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.366 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 10 monclient: shutdown 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 1 -- 172.21.15.178:0/1097417490 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97f0137bb0 msgr2=0x7f97f0139f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 1 --2- 172.21.15.178:0/1097417490 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97f0137bb0 0x7f97f0139f90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009e60 tx=0x7f97e8009780 comp rx=0 tx=0).stop 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 1 -- 172.21.15.178:0/1097417490 shutdown_connections 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 1 --2- 172.21.15.178:0/1097417490 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f97f0137bb0 0x7f97f0139f90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 1 --2- 172.21.15.178:0/1097417490 >> v2:172.21.15.178:3300/0 conn(0x7f97f009f8a0 0x7f97f009fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 1 -- 172.21.15.178:0/1097417490 >> 172.21.15.178:0/1097417490 conn(0x7f97f00631b0 msgr2=0x7f97f00635b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 1 -- 172.21.15.178:0/1097417490 shutdown_connections 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.350+0000 7f97f8034640 1 -- 172.21.15.178:0/1097417490 wait complete. 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.351+0000 7f97f8034640 10 monclient: build_initial_monmap 2024-09-15T01:43:15.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.351+0000 7f97f8034640 10 monclient: monmap: 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:15.368 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:15.369 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:15.369 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.351+0000 7f97f8034640 1 Processor -- start 2024-09-15T01:43:15.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.351+0000 7f97f8034640 1 -- start start 2024-09-15T01:43:15.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.351+0000 7f97f8034640 10 monclient: init 2024-09-15T01:43:15.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:15.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 10 monclient: _add_conns ranks=[2,5,1,0,3,6,8,7,4] 2024-09-15T01:43:15.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f009f8a0 0x7f97f009c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 10 monclient(hunting): picked mon.noname-b con 0x7f97f009f8a0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:15.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f00a2390 0x7f97f0136640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 10 monclient(hunting): picked mon.noname-h con 0x7f97f00a2390 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:15.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f0137bb0 0x7f97f01348d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 10 monclient(hunting): picked mon.noname-f con 0x7f97f0137bb0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f0096c80 con 0x7f97f0137bb0 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f01361f0 con 0x7f97f009f8a0 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97f0135df0 con 0x7f97f00a2390 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f8034640 10 monclient(hunting): authenticate will time out at 1400.102389s 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f009f8a0 0x7f97f009c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f009f8a0 0x7f97f009c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:48288/0 (socket says 172.21.15.178:48288) 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 1 -- 172.21.15.178:0/1819130586 learned_addr learned my addr 172.21.15.178:0/1819130586 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:15.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 10 monclient(hunting): get_auth_request con 0x7f97f009f8a0 auth_method 0 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f65d1640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f0137bb0 0x7f97f01348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f55cf640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f00a2390 0x7f97f0136640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f65d1640 10 monclient(hunting): get_auth_request con 0x7f97f0137bb0 auth_method 0 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f65d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f65d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f65d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f55cf640 10 monclient(hunting): get_auth_request con 0x7f97f00a2390 auth_method 0 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f55cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f55cf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f55cf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 10 monclient(hunting): handle_auth_done global_id 34688 payload 824 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 1 -- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f00a2390 msgr2=0x7f97f0136640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f00a2390 0x7f97f0136640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 1 -- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f0137bb0 msgr2=0x7f97f01348d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97f0137bb0 0x7f97f01348d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 10 monclient: _finish_hunting 0 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 1 monclient: found mon.noname-b 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 1 -- 172.21.15.178:0/1819130586 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e0009450 con 0x7f97f009f8a0 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.352+0000 7f97f5dd0640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f009f8a0 0x7f97f009c640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00c560 tx=0x7f97ec007ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f97ec001120 con 0x7f97f009f8a0 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 10 monclient: dump: 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.373 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec011a10 con 0x7f97f009f8a0 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97f8034640 5 monclient: authenticate success, global_id 34688 2024-09-15T01:43:15.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97f8034640 10 monclient: _renew_subs 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97f8034640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97f8034640 1 -- 172.21.15.178:0/1819130586 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e8009a20 con 0x7f97f009f8a0 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f97ec011bc0 con 0x7f97f009f8a0 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 10 monclient: dump: 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.375 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e7fff640 4 set_mon_vals no callback set 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97f8034640 10 monclient: _renew_subs 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97f8034640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97f8034640 1 -- 172.21.15.178:0/1819130586 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f0097380 con 0x7f97f009f8a0 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.353+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f97ec011d20 con 0x7f97f009f8a0 2024-09-15T01:43:15.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f8034640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:15.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f8034640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f0134e10 0x7f97f01351e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f8034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f8034640 1 -- 172.21.15.178:0/1819130586 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97f006f430 con 0x7f97f0134e10 2024-09-15T01:43:15.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97e5ffb640 1 --2- 172.21.15.178:0/1819130586 >> v2:172.21.15.204:6812/28210 conn(0x7f97d40397b0 0x7f97d403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f97ec04ae10 con 0x7f97f009f8a0 2024-09-15T01:43:15.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f55cf640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f0134e10 0x7f97f01351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f55cf640 10 monclient: get_auth_request con 0x7f97f0134e10 auth_method 0 2024-09-15T01:43:15.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f55cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f55cf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f55cf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f65d1640 1 --2- 172.21.15.178:0/1819130586 >> v2:172.21.15.204:6812/28210 conn(0x7f97d40397b0 0x7f97d403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f65d1640 10 monclient: get_auth_request con 0x7f97d40397b0 auth_method 0 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f55cf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f55cf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f55cf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f55cf640 10 monclient(hunting): handle_auth_done global_id 7097 payload 824 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f55cf640 1 -- 172.21.15.178:0/1819130586 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97f004c4a0 con 0x7f97f0134e10 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f65d1640 1 --2- 172.21.15.178:0/1819130586 >> v2:172.21.15.204:6812/28210 conn(0x7f97d40397b0 0x7f97d403bc60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f97e8034000 tx=0x7f97e8037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.354+0000 7f97f55cf640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f0134e10 0x7f97f01351e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f97e0011840 tx=0x7f97e0008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.355+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f97e000fdc0 con 0x7f97f0134e10 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.355+0000 7f97e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.355+0000 7f97e5ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.355+0000 7f97e5ffb640 10 monclient: dump: 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.378 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.355+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f97e000f940 con 0x7f97f0134e10 2024-09-15T01:43:15.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.355+0000 7f97e5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:15.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.355+0000 7f97e5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:15.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.355+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f0134e10 msgr2=0x7f97f01351e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.355+0000 7f97e5ffb640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f0134e10 0x7f97f01351e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f97e0011840 tx=0x7f97e0008040 comp rx=0 tx=0).stop 2024-09-15T01:43:15.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.355+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f0134e10 msgr2=0x7f97f01351e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:15.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.355+0000 7f97e5ffb640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f0134e10 0x7f97f01351e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.360+0000 7f97f8034640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:15.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.360+0000 7f97f8034640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f012ed10 0x7f97f012f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.360+0000 7f97f8034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.360+0000 7f97f8034640 1 -- 172.21.15.178:0/1819130586 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00a3e70 con 0x7f97f012ed10 2024-09-15T01:43:15.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.361+0000 7f97f55cf640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f012ed10 0x7f97f012f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.361+0000 7f97f55cf640 10 monclient: get_auth_request con 0x7f97f012ed10 auth_method 0 2024-09-15T01:43:15.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.361+0000 7f97f55cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.361+0000 7f97f55cf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.361+0000 7f97f55cf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.361+0000 7f97f55cf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.361+0000 7f97f55cf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.361+0000 7f97f55cf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.361+0000 7f97f55cf640 10 monclient(hunting): handle_auth_done global_id 7106 payload 824 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.361+0000 7f97f55cf640 1 -- 172.21.15.178:0/1819130586 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f97f004bc80 con 0x7f97f012ed10 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.361+0000 7f97f55cf640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f012ed10 0x7f97f012f110 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f97e0011840 tx=0x7f97e001d000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f97e001d7b0 con 0x7f97f012ed10 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97e5ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97e5ffb640 10 monclient: dump: 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.383 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9314 (secure 0 0 0) 0x7f97e0005b80 con 0x7f97f012ed10 2024-09-15T01:43:15.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97e5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:15.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97e5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:15.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f012ed10 msgr2=0x7f97f012f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97e5ffb640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f012ed10 0x7f97f012f110 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f97e0011840 tx=0x7f97e001d000 comp rx=0 tx=0).stop 2024-09-15T01:43:15.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97e5ffb640 1 -- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f012ed10 msgr2=0x7f97f012f110 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:15.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97e5ffb640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f97f012ed10 0x7f97f012f110 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.385 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:15.385 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:15.385 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:15.385 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:43:15.385 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:15.385 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:15.386 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:15.388 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:15.388 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:15.388 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:15.388 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:15.388 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:15.388 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:15.388 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:15.388 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.388 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.388 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:15.388 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:15.388 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:15.389 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:15.398 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.399 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.400 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.400 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:15.400 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.400 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.400 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:15.400 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.400 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.400 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.401 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:15.402 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:15.404 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.404 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.404 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:15.404 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:15.404 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.404 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.404 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.406 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:15.407 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:15.408 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:43:15.408 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.408 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.408 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:15.408 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:15.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97f8034640 1 -- 172.21.15.178:0/1819130586 >> v2:172.21.15.204:6812/28210 conn(0x7f97d40397b0 msgr2=0x7f97d403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97f8034640 1 --2- 172.21.15.178:0/1819130586 >> v2:172.21.15.204:6812/28210 conn(0x7f97d40397b0 0x7f97d403bc60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f97e8034000 tx=0x7f97e8037000 comp rx=0 tx=0).stop 2024-09-15T01:43:15.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97f8034640 10 monclient: shutdown 2024-09-15T01:43:15.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97f8034640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:15.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97f8034640 1 -- 172.21.15.178:0/1819130586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f009f8a0 msgr2=0x7f97f009c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97f8034640 1 --2- 172.21.15.178:0/1819130586 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97f009f8a0 0x7f97f009c640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00c560 tx=0x7f97ec007ef0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97f65d1640 1 -- 172.21.15.178:0/1819130586 reap_dead start 2024-09-15T01:43:15.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97f65d1640 1 -- 172.21.15.178:0/1819130586 reap_dead start 2024-09-15T01:43:15.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97f8034640 1 -- 172.21.15.178:0/1819130586 shutdown_connections 2024-09-15T01:43:15.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97f8034640 1 -- 172.21.15.178:0/1819130586 >> 172.21.15.178:0/1819130586 conn(0x7f97f00631b0 msgr2=0x7f97f009e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:15.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97f8034640 1 -- 172.21.15.178:0/1819130586 shutdown_connections 2024-09-15T01:43:15.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.362+0000 7f97f8034640 1 -- 172.21.15.178:0/1819130586 wait complete. 2024-09-15T01:43:15.410 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:15.410 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:43:15.410 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:43:15.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.502+0000 7fd1b0500640 10 monclient: get_monmap_and_config 2024-09-15T01:43:15.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.502+0000 7fd1b0500640 10 monclient: build_initial_monmap 2024-09-15T01:43:15.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.503+0000 7fd1b0500640 10 monclient: monmap: 2024-09-15T01:43:15.505 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:15.505 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.505 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:15.503869+0000 2024-09-15T01:43:15.505 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:15.503869+0000 2024-09-15T01:43:15.505 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:15.505 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.505 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:15.505 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:15.506 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:15.506 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:15.506 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:15.506 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:15.506 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:15.506 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:15.506 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:15.506 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:15.506 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:15.506 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.503+0000 7fd1b0500640 1 Processor -- start 2024-09-15T01:43:15.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.503+0000 7fd1b0500640 1 -- start start 2024-09-15T01:43:15.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.503+0000 7fd1b0500640 10 monclient: init 2024-09-15T01:43:15.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:15.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 10 monclient: _add_conns ranks=[5,0,3,7,6,9,8,1,10,4,2] 2024-09-15T01:43:15.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd1a809e700 0x7fd1a8069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 10 monclient(hunting): picked mon.noname-h con 0x7fd1a809e700 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:15.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd1a809ead0 addr v1:172.21.15.178:6789/0 2024-09-15T01:43:15.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1a8137db0 0x7fd1a813a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 10 monclient(hunting): picked mon.noname-g con 0x7fd1a8137db0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:15.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a804bf60 con 0x7fd1a809ead0 2024-09-15T01:43:15.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a809f5b0 con 0x7fd1a8137db0 2024-09-15T01:43:15.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8068340 con 0x7fd1a809e700 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 10 monclient(hunting): authenticate will time out at 1400.254370s 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ada9b640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd1a809ead0 0x7fd1a809eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:43162/0 (socket says 172.21.15.178:43162) 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ada9b640 1 -- 172.21.15.178:0/1619026133 learned_addr learned my addr 172.21.15.178:0/1619026133 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614703655 0 0) 0x7fd1a804bf60 con 0x7fd1a809ead0 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 10 monclient(hunting): my global_id is 34857 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ae29c640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd1a809e700 0x7fd1a8069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1aea9d640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1a8137db0 0x7fd1a813a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1940034e0 con 0x7fd1a809ead0 2024-09-15T01:43:15.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1aea9d640 10 monclient(hunting): get_auth_request con 0x7fd1a8137db0 auth_method 0 2024-09-15T01:43:15.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1aea9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1aea9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1aea9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ae29c640 10 monclient(hunting): get_auth_request con 0x7fd1a809e700 auth_method 0 2024-09-15T01:43:15.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ae29c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ae29c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ae29c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1aea9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1aea9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1aea9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2676682948 0 0) 0x7fd1940034e0 con 0x7fd1a809ead0 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd1a809e700 msgr2=0x7fd1a8069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd1a809e700 0x7fd1a8069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1a8137db0 msgr2=0x7fd1a813a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1a8137db0 0x7fd1a813a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 10 monclient: _finish_hunting 0 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a8069210 con 0x7fd1a809ead0 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 10 monclient: _finish_auth 0 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:15.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fd1980018f0 con 0x7fd1a809ead0 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1b0500640 5 monclient: authenticate success, global_id 34857 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 10 monclient: dump: 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.525 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.504+0000 7fd1ad29a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 >> v1:172.21.15.178:6789/0 conn(0x7fd1a809ead0 legacy=0x7fd1a809eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 10 monclient: _add_conns ranks=[2,3,6,1,8,5,7,0,4] 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd194004250 0x7fd194004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 10 monclient(hunting): picked mon.b con 0x7fd194004250 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:15.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1a8137db0 0x7fd194004d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 10 monclient(hunting): picked mon.g con 0x7fd1a8137db0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:15.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd194005340 0x7fd19400b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 10 monclient(hunting): picked mon.d con 0x7fd194005340 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:15.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a809f5b0 con 0x7fd194004250 2024-09-15T01:43:15.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8068340 con 0x7fd1a8137db0 2024-09-15T01:43:15.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd19400c2a0 con 0x7fd194005340 2024-09-15T01:43:15.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:15.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ad29a640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1b0500640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd194004250 0x7fd194004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 10 monclient(hunting): get_auth_request con 0x7fd194004250 auth_method 0 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1aea9d640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd194005340 0x7fd19400b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ada9b640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1a8137db0 0x7fd194004d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1aea9d640 10 monclient(hunting): get_auth_request con 0x7fd194005340 auth_method 0 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1aea9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1aea9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1aea9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:15.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:43:15.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ada9b640 10 monclient(hunting): get_auth_request con 0x7fd1a8137db0 auth_method 0 2024-09-15T01:43:15.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ada9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ada9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ada9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:15.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1aea9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1aea9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1aea9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:43:15.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ada9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ada9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ada9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_done global_id 34857 payload 293 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 1 -- 172.21.15.178:0/1619026133 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd194005340 msgr2=0x7fd19400b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd194005340 0x7fd19400b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 1 -- 172.21.15.178:0/1619026133 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1a8137db0 msgr2=0x7fd194004d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1a8137db0 0x7fd194004d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 10 monclient: _finish_hunting 0 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 1 monclient: found mon.b 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 1 -- 172.21.15.178:0/1619026133 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1940034e0 con 0x7fd194004250 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 10 monclient: _finish_auth 0 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.505+0000 7fd1ae29c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:15.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1ae29c640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd194004250 0x7fd194004780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4006e70 tx=0x7fd1a40090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e891ee2667104b53 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1a4031880 con 0x7fd194004250 2024-09-15T01:43:15.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1ad29a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1ad29a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:15.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1ad29a640 10 monclient: dump: 2024-09-15T01:43:15.531 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.531 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.531 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.531 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.531 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.531 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a402d050 con 0x7fd194004250 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1b0500640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1ad29a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:15.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1ad29a640 1 -- 172.21.15.178:0/1619026133 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1a4031e80 con 0x7fd194004250 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1b0500640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1b0500640 4 set_mon_vals no callback set 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1ad29a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1ad29a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1ad29a640 10 monclient: dump: 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.533 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1b0500640 10 monclient: shutdown 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1b0500640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1b0500640 1 -- 172.21.15.178:0/1619026133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd194004250 msgr2=0x7fd194004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1b0500640 1 --2- 172.21.15.178:0/1619026133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd194004250 0x7fd194004780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4006e70 tx=0x7fd1a40090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1aea9d640 1 -- 172.21.15.178:0/1619026133 reap_dead start 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1b0500640 1 -- 172.21.15.178:0/1619026133 shutdown_connections 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1b0500640 1 -- 172.21.15.178:0/1619026133 >> 172.21.15.178:0/1619026133 conn(0x7fd1a80a4390 msgr2=0x7fd1a8067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.506+0000 7fd1b0500640 1 -- 172.21.15.178:0/1619026133 shutdown_connections 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.507+0000 7fd1b0500640 1 -- 172.21.15.178:0/1619026133 wait complete. 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.507+0000 7fd1b0500640 10 monclient: build_initial_monmap 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.507+0000 7fd1b0500640 10 monclient: monmap: 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:15.535 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.507+0000 7fd1b0500640 1 Processor -- start 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.507+0000 7fd1b0500640 1 -- start start 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.507+0000 7fd1b0500640 10 monclient: init 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 10 monclient: _add_conns ranks=[7,6,0,2,5,3,1,4,8] 2024-09-15T01:43:15.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd194004250 0x7fd1a812fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 10 monclient(hunting): picked mon.noname-i con 0x7fd194004250 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:15.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd1a81302c0 0x7fd1a8130690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 10 monclient(hunting): picked mon.noname-d con 0x7fd1a81302c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:15.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1a8130bd0 0x7fd1a8148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 10 monclient(hunting): picked mon.noname-a con 0x7fd1a8130bd0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:15.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8133a90 con 0x7fd1a8130bd0 2024-09-15T01:43:15.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8137430 con 0x7fd1a81302c0 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80687f0 con 0x7fd194004250 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1b0500640 10 monclient(hunting): authenticate will time out at 1400.258323s 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd1a81302c0 0x7fd1a8130690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1aea9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1a8130bd0 0x7fd1a8148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd1a81302c0 0x7fd1a8130690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44372/0 (socket says 172.21.15.178:44372) 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 1 -- 172.21.15.178:0/960240014 learned_addr learned my addr 172.21.15.178:0/960240014 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 10 monclient(hunting): get_auth_request con 0x7fd1a81302c0 auth_method 0 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1aea9d640 10 monclient(hunting): get_auth_request con 0x7fd1a8130bd0 auth_method 0 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1aea9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1aea9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1aea9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1aea9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1aea9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1aea9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 10 monclient(hunting): handle_auth_done global_id 25197 payload 824 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ae29c640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd194004250 0x7fd1a812fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 1 -- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd194004250 msgr2=0x7fd1a812fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd194004250 0x7fd1a812fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 1 -- 172.21.15.178:0/960240014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1a8130bd0 msgr2=0x7fd1a8148250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1a8130bd0 0x7fd1a8148250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 10 monclient: _finish_hunting 0 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 1 monclient: found mon.noname-d 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 1 -- 172.21.15.178:0/960240014 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a8068520 con 0x7fd1a81302c0 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ae29c640 10 monclient: get_auth_request con 0x7fd194004250 auth_method 0 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ae29c640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd194004250 0x7fd1a812fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1aea9d640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1a8130bd0 0x7fd1a8148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd1ada9b640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd1a81302c0 0x7fd1a8130690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd198004ad0 tx=0x7fd198001630 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd198007e40 con 0x7fd1a81302c0 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd19e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd19e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.508+0000 7fd19e7fc640 10 monclient: dump: 2024-09-15T01:43:15.540 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.541 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.541 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.541 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.541 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.541 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.541 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.541 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.541 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.541 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.544 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.544 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.544 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.544 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.544 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.544 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd19e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd19e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19800fe40 con 0x7fd1a81302c0 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd1b0500640 5 monclient: authenticate success, global_id 25197 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd19e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd1b0500640 10 monclient: _renew_subs 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd198013870 con 0x7fd1a81302c0 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd1b0500640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd1b0500640 1 -- 172.21.15.178:0/960240014 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a8131e20 con 0x7fd1a81302c0 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd19e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd19e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd1aca99640 4 set_mon_vals no callback set 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd19e7fc640 10 monclient: dump: 2024-09-15T01:43:15.545 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.546 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd1b0500640 10 monclient: _renew_subs 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd1b0500640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd1b0500640 1 -- 172.21.15.178:0/960240014 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a8133440 con 0x7fd1a81302c0 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd1b0500640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd1b0500640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809afc0 0x7fd1a809b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd1b0500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd1b0500640 1 -- 172.21.15.178:0/960240014 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a804c4a0 con 0x7fd1a809afc0 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.509+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd1980133f0 con 0x7fd1a81302c0 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd19e7fc640 1 --2- 172.21.15.178:0/960240014 >> v2:172.21.15.204:6812/28210 conn(0x7fd18c0397b0 0x7fd18c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1ae29c640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809afc0 0x7fd1a809b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd19804bd00 con 0x7fd1a81302c0 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1ae29c640 10 monclient: get_auth_request con 0x7fd1a809afc0 auth_method 0 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1ae29c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1ae29c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1ae29c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1aea9d640 1 --2- 172.21.15.178:0/960240014 >> v2:172.21.15.204:6812/28210 conn(0x7fd18c0397b0 0x7fd18c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1aea9d640 10 monclient: get_auth_request con 0x7fd18c0397b0 auth_method 0 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_done global_id 44278 payload 824 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1ae29c640 1 -- 172.21.15.178:0/960240014 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1a804bc80 con 0x7fd1a809afc0 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1aea9d640 1 --2- 172.21.15.178:0/960240014 >> v2:172.21.15.204:6812/28210 conn(0x7fd18c0397b0 0x7fd18c03bc60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd1a809aea0 tx=0x7fd1a000b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd1ae29c640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809afc0 0x7fd1a809b390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd194004700 tx=0x7fd1a4035000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1a403bc70 con 0x7fd1a809afc0 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd19e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd19e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.510+0000 7fd19e7fc640 10 monclient: dump: 2024-09-15T01:43:15.548 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.549 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.511+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fd1a4040070 con 0x7fd1a809afc0 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.511+0000 7fd19e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.511+0000 7fd19e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.511+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809afc0 msgr2=0x7fd1a809b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.511+0000 7fd19e7fc640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809afc0 0x7fd1a809b390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd194004700 tx=0x7fd1a4035000 comp rx=0 tx=0).stop 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.511+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809afc0 msgr2=0x7fd1a809b390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.511+0000 7fd19e7fc640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809afc0 0x7fd1a809b390 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd194004700 tx=0x7fd1a4035000 comp rx=0 tx=0).stop 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.516+0000 7fd1b0500640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.516+0000 7fd1b0500640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809b8d0 0x7fd1a814c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.516+0000 7fd1b0500640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.516+0000 7fd1b0500640 1 -- 172.21.15.178:0/960240014 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8136980 con 0x7fd1a809b8d0 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.516+0000 7fd1ae29c640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809b8d0 0x7fd1a814c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.516+0000 7fd1ae29c640 10 monclient: get_auth_request con 0x7fd1a809b8d0 auth_method 0 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.516+0000 7fd1ae29c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.516+0000 7fd1ae29c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.516+0000 7fd1ae29c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd1ae29c640 10 monclient(hunting): handle_auth_done global_id 44287 payload 824 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd1ae29c640 1 -- 172.21.15.178:0/960240014 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd1a805ed70 con 0x7fd1a809b8d0 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd1ae29c640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809b8d0 0x7fd1a814c7b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009a40 tx=0x7fd1a403e070 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1a403c8a0 con 0x7fd1a809b8d0 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd19e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd19e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:43:15.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd19e7fc640 10 monclient: dump: 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.552 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.553 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.553 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.553 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fd1a403c420 con 0x7fd1a809b8d0 2024-09-15T01:43:15.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd19e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:15.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd19e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:15.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809b8d0 msgr2=0x7fd1a814c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd19e7fc640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809b8d0 0x7fd1a814c7b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009a40 tx=0x7fd1a403e070 comp rx=0 tx=0).stop 2024-09-15T01:43:15.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd19e7fc640 1 -- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809b8d0 msgr2=0x7fd1a814c7b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:15.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.517+0000 7fd19e7fc640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd1a809b8d0 0x7fd1a814c7b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.553 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:15.554 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:15.554 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:15.554 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:15.554 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:15.554 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:15.554 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:15.554 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:15.554 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:15.554 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:15.554 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:15.554 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:15.555 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:15.556 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:15.557 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:15.558 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:15.560 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:15.561 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.561 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.561 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.561 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:15.561 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:15.561 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.561 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.561 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.561 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:15.562 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.562 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.562 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.562 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:15.562 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.562 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.562 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.563 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:15.563 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:15.563 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.563 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.563 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.563 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.563 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:15.563 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:15.563 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.563 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:15.564 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:15.565 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.565 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.565 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.565 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.565 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:15.565 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:15.565 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.565 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.565 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.565 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.566 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:15.566 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.566 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.566 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:15.566 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.566 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.566 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:15.568 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.570 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:15.570 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.570 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.570 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:15.570 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:15.570 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.570 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.572 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.572 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:15.572 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.572 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.572 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.572 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:15.572 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:15.572 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.572 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:15.574 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:15.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:15.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.575 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:15.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:15.575 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.575 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:15.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:15.575 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:15.575 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.576 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:15.577 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:15.578 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:15.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.518+0000 7fd1b0500640 1 -- 172.21.15.178:0/960240014 >> v2:172.21.15.204:6812/28210 conn(0x7fd18c0397b0 msgr2=0x7fd18c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.518+0000 7fd1b0500640 1 --2- 172.21.15.178:0/960240014 >> v2:172.21.15.204:6812/28210 conn(0x7fd18c0397b0 0x7fd18c03bc60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd1a809aea0 tx=0x7fd1a000b040 comp rx=0 tx=0).stop 2024-09-15T01:43:15.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.518+0000 7fd1b0500640 10 monclient: shutdown 2024-09-15T01:43:15.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.518+0000 7fd1b0500640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:15.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.518+0000 7fd1b0500640 1 -- 172.21.15.178:0/960240014 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd1a81302c0 msgr2=0x7fd1a8130690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.518+0000 7fd1b0500640 1 --2- 172.21.15.178:0/960240014 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd1a81302c0 0x7fd1a8130690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd198004ad0 tx=0x7fd198001630 comp rx=0 tx=0).stop 2024-09-15T01:43:15.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.518+0000 7fd1aea9d640 1 -- 172.21.15.178:0/960240014 reap_dead start 2024-09-15T01:43:15.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.518+0000 7fd1aea9d640 1 -- 172.21.15.178:0/960240014 reap_dead start 2024-09-15T01:43:15.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.518+0000 7fd1b0500640 1 -- 172.21.15.178:0/960240014 shutdown_connections 2024-09-15T01:43:15.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.518+0000 7fd1b0500640 1 -- 172.21.15.178:0/960240014 >> 172.21.15.178:0/960240014 conn(0x7fd1a80a4390 msgr2=0x7fd1a80a30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:15.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.518+0000 7fd1b0500640 1 -- 172.21.15.178:0/960240014 shutdown_connections 2024-09-15T01:43:15.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.518+0000 7fd1b0500640 1 -- 172.21.15.178:0/960240014 wait complete. 2024-09-15T01:43:15.580 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:15.580 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T01:43:15.580 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T01:43:15.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.674+0000 7f1b7315d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:15.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.674+0000 7f1b7315d640 10 monclient: build_initial_monmap 2024-09-15T01:43:15.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.674+0000 7f1b7315d640 10 monclient: monmap: 2024-09-15T01:43:15.676 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:15.677 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.677 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:15.675108+0000 2024-09-15T01:43:15.677 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:15.675108+0000 2024-09-15T01:43:15.677 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:15.677 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.677 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:15.677 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:15.677 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:15.678 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:15.678 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:15.678 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:15.678 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:15.678 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:15.678 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:15.678 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:15.678 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:15.679 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.674+0000 7f1b7315d640 1 Processor -- start 2024-09-15T01:43:15.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.674+0000 7f1b7315d640 1 -- start start 2024-09-15T01:43:15.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.674+0000 7f1b7315d640 10 monclient: init 2024-09-15T01:43:15.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:15.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 10 monclient: _add_conns ranks=[1,7,6,5,3,2,4,10,8,9,0] 2024-09-15T01:43:15.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1b6c067130 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:15.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1b6c09d580 0x7f1b6c135bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 10 monclient(hunting): picked mon.noname-i con 0x7f1b6c09d580 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:15.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1b6c09f000 0x7f1b6c138120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 10 monclient(hunting): picked mon.noname-d con 0x7f1b6c09f000 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:15.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6c04bf60 con 0x7f1b6c067130 2024-09-15T01:43:15.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c09e0c0 con 0x7f1b6c09f000 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c09dde0 con 0x7f1b6c09d580 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b7315d640 10 monclient(hunting): authenticate will time out at 1400.425536s 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1b6c09f000 0x7f1b6c138120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b70ef9640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f1b6c067130 0x7f1b6c067500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:38356/0 (socket says 172.21.15.178:38356) 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b70ef9640 1 -- 172.21.15.178:0/2444797160 learned_addr learned my addr 172.21.15.178:0/2444797160 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 10 monclient(hunting): get_auth_request con 0x7f1b6c09f000 auth_method 0 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b6bfff640 1 --2- 172.21.15.178:0/2444797160 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1b6c09d580 0x7f1b6c135bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b6bfff640 10 monclient(hunting): get_auth_request con 0x7f1b6c09d580 auth_method 0 2024-09-15T01:43:15.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b6bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 10 monclient(hunting): handle_auth_done global_id 25206 payload 274 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 1 -- 172.21.15.178:0/2444797160 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1b6c09d580 msgr2=0x7f1b6c135bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 1 --2- 172.21.15.178:0/2444797160 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1b6c09d580 0x7f1b6c135bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 1 -- 172.21.15.178:0/2444797160 >> v1:172.21.15.204:6789/0 conn(0x7f1b6c067130 legacy=0x7f1b6c067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 10 monclient: _finish_hunting 0 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b716fa640 1 monclient: found mon.noname-d 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b716fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b716fa640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b716fa640 1 -- 172.21.15.178:0/2444797160 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c09e3a0 con 0x7f1b6c09f000 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.675+0000 7f1b6b7fe640 1 -- 172.21.15.178:0/2444797160 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861430517 0 0) 0x7f1b6c04bf60 con 0x7f1b6c067130 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b716fa640 1 --2- 172.21.15.178:0/2444797160 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1b6c09f000 0x7f1b6c138120 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009e90 tx=0x7f1b64009780 comp rx=0 tx=0).ready entity=mon.6 client_cookie=a1da7b551646d8c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 1 -- 172.21.15.178:0/2444797160 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b64031880 con 0x7f1b6c09f000 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 10 monclient: dump: 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.694 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b7315d640 5 monclient: authenticate success, global_id 25206 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 1 -- 172.21.15.178:0/2444797160 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6402d050 con 0x7f1b6c09f000 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b7315d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b7315d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b7315d640 4 set_mon_vals no callback set 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 1 -- 172.21.15.178:0/2444797160 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b640303a0 con 0x7f1b6c09f000 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.676+0000 7f1b6b7fe640 10 monclient: dump: 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.696 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.697 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.677+0000 7f1b7315d640 10 monclient: shutdown 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.677+0000 7f1b7315d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.677+0000 7f1b7315d640 1 -- 172.21.15.178:0/2444797160 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1b6c09f000 msgr2=0x7f1b6c138120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.677+0000 7f1b7315d640 1 --2- 172.21.15.178:0/2444797160 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1b6c09f000 0x7f1b6c138120 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009e90 tx=0x7f1b64009780 comp rx=0 tx=0).stop 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.677+0000 7f1b7315d640 1 -- 172.21.15.178:0/2444797160 shutdown_connections 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.677+0000 7f1b7315d640 1 --2- 172.21.15.178:0/2444797160 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1b6c09f000 0x7f1b6c138120 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.677+0000 7f1b7315d640 1 --2- 172.21.15.178:0/2444797160 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1b6c09d580 0x7f1b6c135bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.677+0000 7f1b7315d640 1 -- 172.21.15.178:0/2444797160 >> 172.21.15.178:0/2444797160 conn(0x7f1b6c068a60 msgr2=0x7f1b6c0a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.677+0000 7f1b7315d640 1 -- 172.21.15.178:0/2444797160 shutdown_connections 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.677+0000 7f1b7315d640 1 -- 172.21.15.178:0/2444797160 wait complete. 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.677+0000 7f1b7315d640 10 monclient: build_initial_monmap 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.677+0000 7f1b7315d640 10 monclient: monmap: 2024-09-15T01:43:15.698 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:15.699 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 1 Processor -- start 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 1 -- start start 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 10 monclient: init 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 10 monclient: _add_conns ranks=[2,5,8,7,1,6,0,3,4] 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b6c067130 0x7f1b6c12e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 10 monclient(hunting): picked mon.noname-b con 0x7f1b6c067130 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1b6c09d580 0x7f1b6c11a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 10 monclient(hunting): picked mon.noname-h con 0x7f1b6c09d580 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1b6c09f000 0x7f1b6c11a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 10 monclient(hunting): picked mon.noname-e con 0x7f1b6c09f000 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:15.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c09cb50 con 0x7f1b6c067130 2024-09-15T01:43:15.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c09d9e0 con 0x7f1b6c09d580 2024-09-15T01:43:15.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:15.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c11c7d0 con 0x7f1b6c09f000 2024-09-15T01:43:15.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:15.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b7315d640 10 monclient(hunting): authenticate will time out at 1400.428943s 2024-09-15T01:43:15.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.678+0000 7f1b70ef9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b6c067130 0x7f1b6c12e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b6c067130 0x7f1b6c12e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:48304/0 (socket says 172.21.15.178:48304) 2024-09-15T01:43:15.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 1 -- 172.21.15.178:0/3256629390 learned_addr learned my addr 172.21.15.178:0/3256629390 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:15.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 10 monclient(hunting): get_auth_request con 0x7f1b6c067130 auth_method 0 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b716fa640 1 --2- 172.21.15.178:0/3256629390 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1b6c09f000 0x7f1b6c11a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b716fa640 10 monclient(hunting): get_auth_request con 0x7f1b6c09f000 auth_method 0 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b716fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b716fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b716fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b6bfff640 1 --2- 172.21.15.178:0/3256629390 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1b6c09d580 0x7f1b6c11a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b6bfff640 10 monclient(hunting): get_auth_request con 0x7f1b6c09d580 auth_method 0 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:15.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b6bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b716fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b716fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b716fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 10 monclient(hunting): handle_auth_done global_id 34697 payload 824 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 1 -- 172.21.15.178:0/3256629390 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1b6c09f000 msgr2=0x7f1b6c11a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 1 --2- 172.21.15.178:0/3256629390 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1b6c09f000 0x7f1b6c11a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 1 -- 172.21.15.178:0/3256629390 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1b6c09d580 msgr2=0x7f1b6c11a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 1 --2- 172.21.15.178:0/3256629390 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1b6c09d580 0x7f1b6c11a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 10 monclient: _finish_hunting 0 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 1 monclient: found mon.noname-b 2024-09-15T01:43:15.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 1 -- 172.21.15.178:0/3256629390 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b64009a20 con 0x7f1b6c067130 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b716fa640 1 --2- 172.21.15.178:0/3256629390 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1b6c09f000 0x7f1b6c11a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b70ef9640 1 --2- 172.21.15.178:0/3256629390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b6c067130 0x7f1b6c12e5f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c11c9a0 tx=0x7f1b60007d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b68ff9640 1 -- 172.21.15.178:0/3256629390 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b600070f0 con 0x7f1b6c067130 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b68ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b68ff9640 10 monclient: dump: 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.704 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b68ff9640 10 monclient: _finish_auth 0 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b68ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b68ff9640 1 -- 172.21.15.178:0/3256629390 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b600072b0 con 0x7f1b6c067130 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b7315d640 5 monclient: authenticate success, global_id 34697 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b7315d640 10 monclient: _renew_subs 2024-09-15T01:43:15.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b7315d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b7315d640 1 -- 172.21.15.178:0/3256629390 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6c11c110 con 0x7f1b6c067130 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b68ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b68ff9640 1 -- 172.21.15.178:0/3256629390 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b60012400 con 0x7f1b6c067130 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b68ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b68ff9640 10 monclient: dump: 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:15.706 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:15.707 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:15.707 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:15.707 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:15.707 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:15.707 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:15.707 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:15.707 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:15.707 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:15.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b6affd640 4 set_mon_vals no callback set 2024-09-15T01:43:15.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b7315d640 10 monclient: _renew_subs 2024-09-15T01:43:15.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b7315d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.679+0000 7f1b7315d640 1 -- 172.21.15.178:0/3256629390 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c1406d0 con 0x7f1b6c067130 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.680+0000 7f1b68ff9640 1 -- 172.21.15.178:0/3256629390 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1b60007420 con 0x7f1b6c067130 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.680+0000 7f1b7315d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.680+0000 7f1b7315d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.680+0000 7f1b7315d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.680+0000 7f1b7315d640 1 -- 172.21.15.178:0/3256629390 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6c04b720 con 0x7f1b6c067130 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.680+0000 7f1b68ff9640 1 --2- 172.21.15.178:0/3256629390 >> v2:172.21.15.204:6812/28210 conn(0x7f1b500397b0 0x7f1b5003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.680+0000 7f1b68ff9640 1 -- 172.21.15.178:0/3256629390 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f1b6004b530 con 0x7f1b6c067130 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.680+0000 7f1b6bfff640 1 --2- 172.21.15.178:0/3256629390 >> v2:172.21.15.204:6812/28210 conn(0x7f1b500397b0 0x7f1b5003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.681+0000 7f1b6bfff640 10 monclient: get_auth_request con 0x7f1b500397b0 auth_method 0 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.681+0000 7f1b6bfff640 1 --2- 172.21.15.178:0/3256629390 >> v2:172.21.15.204:6812/28210 conn(0x7f1b500397b0 0x7f1b5003bc60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009d20 tx=0x7f1b580090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.685+0000 7f1b68ff9640 1 -- 172.21.15.178:0/3256629390 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1b60012560 con 0x7f1b6c067130 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.685+0000 7f1b68ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:15.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.685+0000 7f1b68ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:15.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.900+0000 7f1b7315d640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T01:43:15.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.900+0000 7f1b7315d640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T01:43:15.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.900+0000 7f1b7315d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:15.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.900+0000 7f1b7315d640 1 -- 172.21.15.178:0/3256629390 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f1b6c070770 con 0x7f1b6c067130 2024-09-15T01:43:15.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.902+0000 7f1b68ff9640 1 -- 172.21.15.178:0/3256629390 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f1b60019300 con 0x7f1b6c067130 2024-09-15T01:43:15.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.902+0000 7f1b68ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T01:43:15.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.902+0000 7f1b68ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:15.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:15.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=2,health=12,kv=7,logm=63} crc {auth=2974431290,config=3140514009,health=3171711444,kv=963045094,logm=2746095955}) 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=7,logm=64} crc {auth=757697896,config=3140514009,health=521794272,kv=2253581948,logm=1545099776}) 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=2,health=12,kv=7,logm=63} crc {auth=2974431290,config=3140514009,health=3171711444,kv=963045094,logm=2746095955}) 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=14,config=2,health=11,kv=8,logm=65} crc {auth=587453655,config=3140514009,health=1749830715,kv=706962213,logm=636412802}) 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=2,health=12,kv=7,logm=63} crc {auth=2974431290,config=3140514009,health=3171711444,kv=963045094,logm=2746095955}) 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=15,config=2,health=12,kv=3,logm=68} crc {auth=3333366264,config=3140514009,health=3171711444,kv=1278844342,logm=3019007921}) 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=2,health=12,kv=7,logm=63} crc {auth=2974431290,config=3140514009,health=3171711444,kv=963045094,logm=2746095955}) 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=16,config=2,health=11,kv=7,logm=64} crc {auth=1837881752,config=3140514009,health=841448276,kv=1446063039,logm=1238937385}) 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=2,health=12,kv=7,logm=63} crc {auth=2974431290,config=3140514009,health=3171711444,kv=963045094,logm=2746095955}) 2024-09-15T01:43:15.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=16,config=2,health=12,kv=8,logm=62} crc {auth=2974431290,config=3140514009,health=3171711444,kv=706962213,logm=4157459166}) 2024-09-15T01:43:15.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:15.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=2,health=12,kv=7,logm=63} crc {auth=2974431290,config=3140514009,health=3171711444,kv=963045094,logm=2746095955}) 2024-09-15T01:43:15.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=16,config=1,health=11,kv=8,logm=64} crc {auth=2974431290,config=1228700967,health=841448276,kv=706962213,logm=668764185}) 2024-09-15T01:43:15.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:15.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=2,health=12,kv=7,logm=63} crc {auth=2974431290,config=3140514009,health=3171711444,kv=963045094,logm=2746095955}) 2024-09-15T01:43:15.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=13,config=2,health=12,kv=8,logm=65} crc {auth=2196999054,config=3140514009,health=3171711444,kv=706962213,logm=2629215767}) 2024-09-15T01:43:15.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:15.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=2,health=12,kv=7,logm=63} crc {auth=2974431290,config=3140514009,health=3171711444,kv=963045094,logm=2746095955}) 2024-09-15T01:43:15.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.903+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=15,config=2,health=11,kv=7,logm=65} crc {auth=3333366264,config=3140514009,health=4201526202,kv=1446063039,logm=2835979031}) 2024-09-15T01:43:15.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 1 -- 172.21.15.178:0/3256629390 >> v2:172.21.15.204:6812/28210 conn(0x7f1b500397b0 msgr2=0x7f1b5003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 1 --2- 172.21.15.178:0/3256629390 >> v2:172.21.15.204:6812/28210 conn(0x7f1b500397b0 0x7f1b5003bc60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009d20 tx=0x7f1b580090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 10 monclient: shutdown 2024-09-15T01:43:15.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:15.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 1 -- 172.21.15.178:0/3256629390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b6c067130 msgr2=0x7f1b6c12e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:15.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 1 --2- 172.21.15.178:0/3256629390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b6c067130 0x7f1b6c12e5f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c11c9a0 tx=0x7f1b60007d00 comp rx=0 tx=0).stop 2024-09-15T01:43:15.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 1 -- 172.21.15.178:0/3256629390 shutdown_connections 2024-09-15T01:43:15.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 1 --2- 172.21.15.178:0/3256629390 >> v2:172.21.15.204:6812/28210 conn(0x7f1b500397b0 0x7f1b5003bc60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 1 --2- 172.21.15.178:0/3256629390 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1b6c09d580 0x7f1b6c11a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 1 --2- 172.21.15.178:0/3256629390 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1b6c09f000 0x7f1b6c11a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 1 --2- 172.21.15.178:0/3256629390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b6c067130 0x7f1b6c12e5f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:15.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 1 -- 172.21.15.178:0/3256629390 >> 172.21.15.178:0/3256629390 conn(0x7f1b6c068a60 msgr2=0x7f1b6c0a1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:15.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 1 -- 172.21.15.178:0/3256629390 shutdown_connections 2024-09-15T01:43:15.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:15.905+0000 7f1b7315d640 1 -- 172.21.15.178:0/3256629390 wait complete. 2024-09-15T01:43:15.922 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T01:43:16.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2322143957}) 2024-09-15T01:43:16.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3606355180}) 2024-09-15T01:43:16.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2322143957}) 2024-09-15T01:43:16.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2138547680}) 2024-09-15T01:43:16.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2322143957}) 2024-09-15T01:43:16.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3669440884}) 2024-09-15T01:43:16.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2322143957}) 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3458593808}) 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2322143957}) 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2690539590}) 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2322143957}) 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1423246895}) 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2322143957}) 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3256826732}) 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2322143957}) 2024-09-15T01:43:16.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1621051762}) 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144296391}) 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=12972321}) 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144296391}) 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4086298867}) 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144296391}) 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1119287291}) 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144296391}) 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2724904164}) 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144296391}) 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3920764558}) 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144296391}) 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3453616001}) 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144296391}) 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=952277568}) 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144296391}) 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.908+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2033462504}) 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583468607}) 2024-09-15T01:43:16.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3847450678}) 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583468607}) 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2010799268}) 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583468607}) 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3623364724}) 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583468607}) 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4084264583}) 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583468607}) 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2062235153}) 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583468607}) 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3372515248}) 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583468607}) 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1650761671}) 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583468607}) 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.910+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1633912718}) 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169517232}) 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1802459986}) 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169517232}) 2024-09-15T01:43:16.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1297244654}) 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169517232}) 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1362181908}) 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169517232}) 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=536661323}) 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169517232}) 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2079784184}) 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169517232}) 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1264161459}) 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169517232}) 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1999582473}) 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169517232}) 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.911+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=542300551}) 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4270051720}) 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3440729187}) 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4270051720}) 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2206317976}) 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4270051720}) 2024-09-15T01:43:16.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2860658454}) 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4270051720}) 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2060451517}) 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4270051720}) 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3522192637}) 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4270051720}) 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=741698606}) 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4270051720}) 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3546366545}) 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4270051720}) 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.912+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=757346182}) 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.913+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2103140541}) 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2357385115}) 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2103140541}) 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=649246611}) 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2103140541}) 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2829460797}) 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2103140541}) 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2584333615}) 2024-09-15T01:43:16.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2103140541}) 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1072338099}) 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2103140541}) 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=390580179}) 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2103140541}) 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2678548349}) 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2103140541}) 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.914+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3786882238}) 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3784594898}) 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3204323928}) 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3784594898}) 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=598979116}) 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3784594898}) 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2012205664}) 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3784594898}) 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=361758191}) 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3784594898}) 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4054044985}) 2024-09-15T01:43:16.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3784594898}) 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3014401874}) 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3784594898}) 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1669066902}) 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3784594898}) 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.915+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=914620758}) 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.916+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.916+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4203135436}) 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.916+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4195054955}) 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.916+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.916+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4203135436}) 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.916+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4291942836}) 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.916+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4203135436}) 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3390656806}) 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4203135436}) 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1276382628}) 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4203135436}) 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2588414218}) 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4203135436}) 2024-09-15T01:43:16.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=895276203}) 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4203135436}) 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4166287659}) 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4203135436}) 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.917+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2005554881}) 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=785579692}) 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1924434832}) 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=785579692}) 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4231628473}) 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=785579692}) 2024-09-15T01:43:16.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2351977083}) 2024-09-15T01:43:16.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=785579692}) 2024-09-15T01:43:16.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1876114591}) 2024-09-15T01:43:16.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=785579692}) 2024-09-15T01:43:16.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1093631102}) 2024-09-15T01:43:16.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=785579692}) 2024-09-15T01:43:16.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2899396640}) 2024-09-15T01:43:16.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=785579692}) 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1722217487}) 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=785579692}) 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.918+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=760018718}) 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1690304328}) 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3022376417}) 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1690304328}) 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2265585338}) 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1690304328}) 2024-09-15T01:43:16.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=228907433}) 2024-09-15T01:43:16.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1690304328}) 2024-09-15T01:43:16.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1096645650}) 2024-09-15T01:43:16.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1690304328}) 2024-09-15T01:43:16.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=386875134}) 2024-09-15T01:43:16.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1690304328}) 2024-09-15T01:43:16.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1468058684}) 2024-09-15T01:43:16.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1690304328}) 2024-09-15T01:43:16.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1640325566}) 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1690304328}) 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.919+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1304551604}) 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4258346951}) 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3602419042}) 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4258346951}) 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1917125205}) 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4258346951}) 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2570775143}) 2024-09-15T01:43:16.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4258346951}) 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2231665230}) 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4258346951}) 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1104983231}) 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4258346951}) 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1410961848}) 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4258346951}) 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3321541113}) 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4258346951}) 2024-09-15T01:43:16.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.920+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1041459827}) 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1217943487}) 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2468729286}) 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1217943487}) 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1214334953}) 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1217943487}) 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2193203559}) 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1217943487}) 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3819151586}) 2024-09-15T01:43:16.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1217943487}) 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4104553906}) 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1217943487}) 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=532706052}) 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.921+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.922+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1217943487}) 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.922+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3697640450}) 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.922+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.922+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1217943487}) 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.922+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1362559012}) 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584237861}) 2024-09-15T01:43:16.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2489142091}) 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584237861}) 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2498227554}) 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584237861}) 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3416788431}) 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584237861}) 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3087989751}) 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584237861}) 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1325810}) 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584237861}) 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1369972830}) 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584237861}) 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=227660797}) 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584237861}) 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.923+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2620406055}) 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=934191244}) 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034071452}) 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=934191244}) 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1720478229}) 2024-09-15T01:43:16.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=934191244}) 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=778126176}) 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=934191244}) 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=292318765}) 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=934191244}) 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=438177971}) 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=934191244}) 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2730954055}) 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=934191244}) 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=635411438}) 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=934191244}) 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.924+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3560367127}) 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4145446588}) 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4084648252}) 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4145446588}) 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2786038742}) 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4145446588}) 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=842001265}) 2024-09-15T01:43:16.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4145446588}) 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2997774405}) 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4145446588}) 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1471881551}) 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4145446588}) 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=544409717}) 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4145446588}) 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2178969861}) 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4145446588}) 2024-09-15T01:43:16.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.925+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1560305657}) 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1433934357}) 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2768721327}) 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1433934357}) 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3498436305}) 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1433934357}) 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855137566}) 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1433934357}) 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3947311978}) 2024-09-15T01:43:16.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1433934357}) 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.926+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2661891984}) 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.927+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.927+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1433934357}) 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.927+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3256870844}) 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.927+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.927+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1433934357}) 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.927+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3781629761}) 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.927+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.927+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1433934357}) 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.927+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1815192021}) 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4015300253}) 2024-09-15T01:43:16.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1560787914}) 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4015300253}) 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3208384718}) 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4015300253}) 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1110822375}) 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4015300253}) 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=328886612}) 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4015300253}) 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2593447798}) 2024-09-15T01:43:16.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4015300253}) 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1465297545}) 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4015300253}) 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2369281852}) 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4015300253}) 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.928+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1541504148}) 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=822340872}) 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3580846064}) 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=822340872}) 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2634121017}) 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=822340872}) 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3552610955}) 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=822340872}) 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4285277994}) 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=822340872}) 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4067417277}) 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=822340872}) 2024-09-15T01:43:16.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1025368878}) 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=822340872}) 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3071709204}) 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=822340872}) 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.929+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2631309053}) 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030710078}) 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=854234865}) 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030710078}) 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3212235242}) 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030710078}) 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1195025821}) 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030710078}) 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1497495103}) 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030710078}) 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3989549241}) 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030710078}) 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3565796403}) 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030710078}) 2024-09-15T01:43:16.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3472606099}) 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030710078}) 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.930+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2404605916}) 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.931+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.931+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3992739273}) 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.931+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3398713838}) 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.931+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.931+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3992739273}) 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.931+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2855291576}) 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.931+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.931+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3992739273}) 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.931+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2090977057}) 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3992739273}) 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3725527533}) 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3992739273}) 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1659036467}) 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3992739273}) 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2477831199}) 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3992739273}) 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1997256258}) 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3992739273}) 2024-09-15T01:43:16.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.932+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1568708401}) 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107140236}) 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=803766426}) 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107140236}) 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3379362724}) 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107140236}) 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3243587738}) 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107140236}) 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1670025988}) 2024-09-15T01:43:16.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107140236}) 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3308875159}) 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107140236}) 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1259948351}) 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107140236}) 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1255244027}) 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107140236}) 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.933+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1068153242}) 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2636898396}) 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3042267576}) 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2636898396}) 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=10224719}) 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2636898396}) 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3358551895}) 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2636898396}) 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=245358920}) 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2636898396}) 2024-09-15T01:43:16.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3169165048}) 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2636898396}) 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2498867970}) 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2636898396}) 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=382758989}) 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2636898396}) 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.934+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3217119724}) 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.935+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.935+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675961423}) 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.935+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1543283674}) 2024-09-15T01:43:16.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.935+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.935+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675961423}) 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.935+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2236926227}) 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.935+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.935+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675961423}) 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.935+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3261592890}) 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675961423}) 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=774214786}) 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675961423}) 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2295471352}) 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675961423}) 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1249877432}) 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675961423}) 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2784618685}) 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675961423}) 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.936+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=926312717}) 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3284803702}) 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1158973445}) 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3284803702}) 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2126350945}) 2024-09-15T01:43:16.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3284803702}) 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=176024481}) 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3284803702}) 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1644660832}) 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3284803702}) 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=309388546}) 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3284803702}) 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1910782295}) 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3284803702}) 2024-09-15T01:43:16.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1690759041}) 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3284803702}) 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.937+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1956996413}) 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=842613921}) 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2872447483}) 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=842613921}) 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=207252199}) 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=842613921}) 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2310027688}) 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=842613921}) 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4287024025}) 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=842613921}) 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2034980056}) 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=842613921}) 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3623282342}) 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=842613921}) 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=665017488}) 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=842613921}) 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.938+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1667502108}) 2024-09-15T01:43:16.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3862970024}) 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3823206711}) 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3862970024}) 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2348119041}) 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3862970024}) 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4206494001}) 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3862970024}) 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3281444001}) 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3862970024}) 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=853167300}) 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3862970024}) 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2087296724}) 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3862970024}) 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2803312043}) 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3862970024}) 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.939+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1585538289}) 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.940+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.940+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=376242501}) 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4132511959}) 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=376242501}) 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3527791924}) 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=376242501}) 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1801865653}) 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=376242501}) 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=68337463}) 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=376242501}) 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3504673896}) 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=376242501}) 2024-09-15T01:43:16.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1691301122}) 2024-09-15T01:43:16.125 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.125 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=376242501}) 2024-09-15T01:43:16.125 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3498877234}) 2024-09-15T01:43:16.125 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.125 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=376242501}) 2024-09-15T01:43:16.125 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.941+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3662169401}) 2024-09-15T01:43:16.125 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.125 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3073244009}) 2024-09-15T01:43:16.125 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=805678768}) 2024-09-15T01:43:16.125 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.125 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3073244009}) 2024-09-15T01:43:16.125 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3382643550}) 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3073244009}) 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4080806125}) 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3073244009}) 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3889057428}) 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3073244009}) 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3736372257}) 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3073244009}) 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=224331644}) 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.126 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3073244009}) 2024-09-15T01:43:16.127 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=826113520}) 2024-09-15T01:43:16.127 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.127 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3073244009}) 2024-09-15T01:43:16.127 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.942+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3809321444}) 2024-09-15T01:43:16.127 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.127 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=171522512}) 2024-09-15T01:43:16.127 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=45918852}) 2024-09-15T01:43:16.127 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.127 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=171522512}) 2024-09-15T01:43:16.127 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1553742190}) 2024-09-15T01:43:16.127 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.127 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=171522512}) 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2665303625}) 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=171522512}) 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2352635282}) 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=171522512}) 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=333114906}) 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=171522512}) 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1437117067}) 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=171522512}) 2024-09-15T01:43:16.128 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3089979837}) 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=171522512}) 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.943+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3323681676}) 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294623099}) 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3679755116}) 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294623099}) 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=929331036}) 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294623099}) 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=968096903}) 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.129 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294623099}) 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2825938451}) 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294623099}) 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1638272336}) 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.944+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.945+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294623099}) 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.945+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2893475439}) 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.945+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.945+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294623099}) 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.945+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3233510529}) 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.945+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.945+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294623099}) 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.945+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3064715750}) 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.130 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505934682}) 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1770942314}) 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505934682}) 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=566009808}) 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505934682}) 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2939844203}) 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505934682}) 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2133590669}) 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505934682}) 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2433487404}) 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.131 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505934682}) 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2518856053}) 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505934682}) 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2350249038}) 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505934682}) 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.946+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2886517313}) 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2819609708}) 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2041938265}) 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2819609708}) 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=875824640}) 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.132 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2819609708}) 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1386953014}) 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2819609708}) 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3297305052}) 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2819609708}) 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2982634238}) 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2819609708}) 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1290428167}) 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2819609708}) 2024-09-15T01:43:16.133 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=161097167}) 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2819609708}) 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.947+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=525093520}) 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=14893669}) 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3158702693}) 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=14893669}) 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2137125803}) 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=14893669}) 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2779157365}) 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.134 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=14893669}) 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3864049643}) 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=14893669}) 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3672131601}) 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=14893669}) 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=696448301}) 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=14893669}) 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.948+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=55947298}) 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.949+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.949+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=14893669}) 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.949+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3879582881}) 2024-09-15T01:43:16.135 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109175601}) 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1124542893}) 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109175601}) 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=910808747}) 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109175601}) 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2404698869}) 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109175601}) 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1951859142}) 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109175601}) 2024-09-15T01:43:16.136 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2874185524}) 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109175601}) 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3832222540}) 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109175601}) 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2320555107}) 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109175601}) 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.950+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3287252653}) 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3557562740}) 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2401694194}) 2024-09-15T01:43:16.137 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3557562740}) 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=113857867}) 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3557562740}) 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3671571977}) 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3557562740}) 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2725945073}) 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3557562740}) 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=61424267}) 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3557562740}) 2024-09-15T01:43:16.138 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2356566417}) 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3557562740}) 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1868975224}) 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3557562740}) 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.951+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=965399216}) 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4256971416}) 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=542353691}) 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4256971416}) 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1039498963}) 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4256971416}) 2024-09-15T01:43:16.139 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=239706258}) 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4256971416}) 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3316270589}) 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4256971416}) 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2001678777}) 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.952+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.953+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4256971416}) 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.953+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=544495997}) 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.953+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.953+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4256971416}) 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.953+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=583626695}) 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.953+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.953+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4256971416}) 2024-09-15T01:43:16.140 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.953+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4238239489}) 2024-09-15T01:43:16.141 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.141 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3564103039}) 2024-09-15T01:43:16.141 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2409510983}) 2024-09-15T01:43:16.141 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.141 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3564103039}) 2024-09-15T01:43:16.141 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2790638256}) 2024-09-15T01:43:16.141 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.141 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3564103039}) 2024-09-15T01:43:16.141 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3674409280}) 2024-09-15T01:43:16.141 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.141 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3564103039}) 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3361277932}) 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3564103039}) 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3370221682}) 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3564103039}) 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=664581471}) 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3564103039}) 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3180936808}) 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3564103039}) 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.954+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3075289446}) 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.142 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2690092499}) 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3823055997}) 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2690092499}) 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3461373177}) 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2690092499}) 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3811930508}) 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2690092499}) 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2653860024}) 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2690092499}) 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2326010504}) 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.143 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2690092499}) 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2288743343}) 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2690092499}) 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=809594765}) 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2690092499}) 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.955+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2668611471}) 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=913860228}) 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1560293954}) 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=913860228}) 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1287051430}) 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.144 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=913860228}) 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3269441365}) 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=913860228}) 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1152970696}) 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=913860228}) 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3244548488}) 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.956+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.957+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=913860228}) 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.957+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2023564326}) 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.957+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.957+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=913860228}) 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.957+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4226206215}) 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.957+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.145 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.957+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=913860228}) 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.957+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2452193170}) 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2797712514}) 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2981263494}) 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2797712514}) 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1942808881}) 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2797712514}) 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2888236514}) 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2797712514}) 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=556828445}) 2024-09-15T01:43:16.146 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2797712514}) 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=45332969}) 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2797712514}) 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=163543938}) 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2797712514}) 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2808434557}) 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2797712514}) 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.958+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3649300384}) 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2041918739}) 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4118036723}) 2024-09-15T01:43:16.147 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2041918739}) 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=722117753}) 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2041918739}) 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2888746602}) 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2041918739}) 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2418085065}) 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2041918739}) 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1550829289}) 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2041918739}) 2024-09-15T01:43:16.148 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1474141522}) 2024-09-15T01:43:16.149 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.149 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2041918739}) 2024-09-15T01:43:16.152 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=942245371}) 2024-09-15T01:43:16.152 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.152 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2041918739}) 2024-09-15T01:43:16.152 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.959+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2669854479}) 2024-09-15T01:43:16.152 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.152 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635968408}) 2024-09-15T01:43:16.152 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1841093355}) 2024-09-15T01:43:16.152 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635968408}) 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1081323035}) 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635968408}) 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1260562305}) 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635968408}) 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1495964803}) 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635968408}) 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=557335333}) 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.153 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635968408}) 2024-09-15T01:43:16.154 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1657529015}) 2024-09-15T01:43:16.154 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.154 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635968408}) 2024-09-15T01:43:16.154 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3956701130}) 2024-09-15T01:43:16.154 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.154 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.960+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635968408}) 2024-09-15T01:43:16.154 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.961+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=706175478}) 2024-09-15T01:43:16.154 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.154 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1623968656}) 2024-09-15T01:43:16.154 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3333302882}) 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1623968656}) 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3596520455}) 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1623968656}) 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2351552371}) 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1623968656}) 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1164524620}) 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1623968656}) 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3490176952}) 2024-09-15T01:43:16.155 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1623968656}) 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3911540343}) 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1623968656}) 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=868802542}) 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1623968656}) 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.962+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3965664974}) 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2487718137}) 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3729814275}) 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.156 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2487718137}) 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2311509146}) 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2487718137}) 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3105337243}) 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2487718137}) 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3246211282}) 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2487718137}) 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=984486608}) 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2487718137}) 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1157983571}) 2024-09-15T01:43:16.157 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2487718137}) 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=316910684}) 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2487718137}) 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.963+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3212622992}) 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.964+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.964+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3024464887}) 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.964+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2292297002}) 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.964+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.964+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3024464887}) 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.964+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2088685598}) 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.964+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.158 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.964+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3024464887}) 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.964+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3994468814}) 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.964+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.964+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3024464887}) 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4145984381}) 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3024464887}) 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3316878371}) 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3024464887}) 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2791252180}) 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3024464887}) 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4263233785}) 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.159 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3024464887}) 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.965+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1071573628}) 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=806894281}) 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3672849815}) 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=806894281}) 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2993181119}) 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=806894281}) 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3216906114}) 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=806894281}) 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=489050092}) 2024-09-15T01:43:16.160 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=806894281}) 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1623852188}) 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=806894281}) 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=334719582}) 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=806894281}) 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3374271406}) 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=806894281}) 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.966+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3568227252}) 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.161 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1227756990}) 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797247533}) 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1227756990}) 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1003093725}) 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1227756990}) 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=497019991}) 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1227756990}) 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3197853532}) 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1227756990}) 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1250791850}) 2024-09-15T01:43:16.162 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1227756990}) 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=541670104}) 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1227756990}) 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3460620442}) 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1227756990}) 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.967+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=282638804}) 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.968+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.968+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=739077531}) 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2110589892}) 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.163 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=739077531}) 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1744262356}) 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=739077531}) 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1743535699}) 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=739077531}) 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=714993474}) 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=739077531}) 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2528970752}) 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=739077531}) 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=442688240}) 2024-09-15T01:43:16.164 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.165 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=739077531}) 2024-09-15T01:43:16.165 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=464074040}) 2024-09-15T01:43:16.165 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.165 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=739077531}) 2024-09-15T01:43:16.165 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.969+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1481141488}) 2024-09-15T01:43:16.165 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.165 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3702520285}) 2024-09-15T01:43:16.165 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1872190968}) 2024-09-15T01:43:16.165 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.165 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3702520285}) 2024-09-15T01:43:16.165 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=617162292}) 2024-09-15T01:43:16.165 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3702520285}) 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1314834624}) 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3702520285}) 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1154043191}) 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3702520285}) 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2968560905}) 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3702520285}) 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2822457252}) 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3702520285}) 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3167200437}) 2024-09-15T01:43:16.166 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3702520285}) 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.970+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=708783900}) 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=99257391}) 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1855140124}) 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=99257391}) 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2392626669}) 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=99257391}) 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1243927512}) 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.167 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=99257391}) 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3837329396}) 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=99257391}) 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2659012583}) 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.971+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.972+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=99257391}) 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.972+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=786537390}) 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.972+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.972+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=99257391}) 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.972+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2805779424}) 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.972+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.972+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=99257391}) 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.972+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3001437870}) 2024-09-15T01:43:16.168 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1815437507}) 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1013614899}) 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1815437507}) 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2217531718}) 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1815437507}) 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=424822958}) 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1815437507}) 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3926963397}) 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.169 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1815437507}) 2024-09-15T01:43:16.170 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2727960103}) 2024-09-15T01:43:16.170 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.170 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1815437507}) 2024-09-15T01:43:16.170 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2579264612}) 2024-09-15T01:43:16.170 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.170 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1815437507}) 2024-09-15T01:43:16.170 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2686709934}) 2024-09-15T01:43:16.170 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.170 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1815437507}) 2024-09-15T01:43:16.170 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.973+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1267944602}) 2024-09-15T01:43:16.170 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.170 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3686574718}) 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=675770665}) 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3686574718}) 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3908395364}) 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3686574718}) 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2716118388}) 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3686574718}) 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1784263481}) 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3686574718}) 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=613407631}) 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.171 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3686574718}) 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1996156548}) 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3686574718}) 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1311529385}) 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3686574718}) 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.974+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3103635324}) 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3741492969}) 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2192213141}) 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3741492969}) 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=774407167}) 2024-09-15T01:43:16.172 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3741492969}) 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1613371481}) 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3741492969}) 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1654926383}) 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3741492969}) 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3373706049}) 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3741492969}) 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2094752550}) 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3741492969}) 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4048160042}) 2024-09-15T01:43:16.173 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.174 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3741492969}) 2024-09-15T01:43:16.174 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.975+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=676997087}) 2024-09-15T01:43:16.174 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.174 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2934362353}) 2024-09-15T01:43:16.174 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2813773165}) 2024-09-15T01:43:16.174 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.174 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2934362353}) 2024-09-15T01:43:16.174 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3140330787}) 2024-09-15T01:43:16.174 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.174 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2934362353}) 2024-09-15T01:43:16.174 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1637352273}) 2024-09-15T01:43:16.174 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.175 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2934362353}) 2024-09-15T01:43:16.175 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4039716590}) 2024-09-15T01:43:16.175 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.175 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2934362353}) 2024-09-15T01:43:16.175 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1925315512}) 2024-09-15T01:43:16.175 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.175 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2934362353}) 2024-09-15T01:43:16.175 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=698890832}) 2024-09-15T01:43:16.175 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.175 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2934362353}) 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2840384107}) 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2934362353}) 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.977+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=144822708}) 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=204505836}) 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4023115394}) 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=204505836}) 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3378872377}) 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=204505836}) 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3960278388}) 2024-09-15T01:43:16.176 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=204505836}) 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2412661515}) 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=204505836}) 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4186104840}) 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=204505836}) 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1317197349}) 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=204505836}) 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1172481768}) 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.177 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=204505836}) 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.978+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=815999795}) 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3117543176}) 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2872948506}) 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3117543176}) 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629779893}) 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3117543176}) 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2182016875}) 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3117543176}) 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4143108164}) 2024-09-15T01:43:16.178 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3117543176}) 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=68085708}) 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3117543176}) 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=699621441}) 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3117543176}) 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2950901408}) 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3117543176}) 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.980+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2628688442}) 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=936169763}) 2024-09-15T01:43:16.179 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2024060422}) 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=936169763}) 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1613234820}) 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=936169763}) 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=121107296}) 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=936169763}) 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3858803166}) 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.981+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.982+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=936169763}) 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.982+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=378682776}) 2024-09-15T01:43:16.180 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.982+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.181 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.982+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=936169763}) 2024-09-15T01:43:16.181 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.982+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3234237123}) 2024-09-15T01:43:16.181 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.982+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.181 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.982+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=936169763}) 2024-09-15T01:43:16.181 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.982+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3822646467}) 2024-09-15T01:43:16.181 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.982+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.181 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.982+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=936169763}) 2024-09-15T01:43:16.181 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.982+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2850285930}) 2024-09-15T01:43:16.181 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.181 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4276199437}) 2024-09-15T01:43:16.181 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=126396797}) 2024-09-15T01:43:16.181 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4276199437}) 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1080393838}) 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4276199437}) 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2253601691}) 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4276199437}) 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1924577044}) 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4276199437}) 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=981521696}) 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.182 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4276199437}) 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3657055582}) 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4276199437}) 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=181997373}) 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4276199437}) 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.983+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=227759633}) 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116607620}) 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3937881221}) 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116607620}) 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2018853924}) 2024-09-15T01:43:16.183 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116607620}) 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1298544550}) 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116607620}) 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=37971580}) 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116607620}) 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2528443821}) 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116607620}) 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1141851424}) 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.184 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116607620}) 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1888094091}) 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116607620}) 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.984+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1434026627}) 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622086456}) 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3080292189}) 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622086456}) 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=710283968}) 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622086456}) 2024-09-15T01:43:16.185 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=968205530}) 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622086456}) 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1554781075}) 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622086456}) 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=688399357}) 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622086456}) 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.985+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1669565415}) 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.986+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.986+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622086456}) 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.986+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3779605822}) 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.986+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.186 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.986+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622086456}) 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.986+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2507105867}) 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4037435477}) 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1674207331}) 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4037435477}) 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4189647965}) 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4037435477}) 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1029463640}) 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4037435477}) 2024-09-15T01:43:16.187 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3927925224}) 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4037435477}) 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2338902066}) 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4037435477}) 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1681660444}) 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4037435477}) 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3229769285}) 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4037435477}) 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.987+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1942567694}) 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.188 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1630081844}) 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2147550150}) 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1630081844}) 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2654416712}) 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1630081844}) 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=212157290}) 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1630081844}) 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1839562489}) 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1630081844}) 2024-09-15T01:43:16.189 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3855608646}) 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1630081844}) 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=924427363}) 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1630081844}) 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=67558982}) 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1630081844}) 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.988+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=645353092}) 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1293288275}) 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2994358910}) 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.190 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1293288275}) 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3233071861}) 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1293288275}) 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=69633044}) 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1293288275}) 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2819142298}) 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1293288275}) 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2403045256}) 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1293288275}) 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4270870756}) 2024-09-15T01:43:16.191 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1293288275}) 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1916506471}) 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1293288275}) 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.989+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2429004773}) 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.990+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.990+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=80732885}) 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.990+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=402782200}) 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.990+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=80732885}) 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1823193127}) 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.192 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=80732885}) 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3945219425}) 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=80732885}) 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=646917354}) 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=80732885}) 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1779219194}) 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=80732885}) 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3697108569}) 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=80732885}) 2024-09-15T01:43:16.193 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=142617418}) 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=80732885}) 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.991+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=167277055}) 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3917160701}) 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4292074955}) 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3917160701}) 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=603563564}) 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3917160701}) 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2138119318}) 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.194 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3917160701}) 2024-09-15T01:43:16.195 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1518207667}) 2024-09-15T01:43:16.195 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.195 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3917160701}) 2024-09-15T01:43:16.195 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=94341748}) 2024-09-15T01:43:16.195 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.195 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3917160701}) 2024-09-15T01:43:16.195 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1538473426}) 2024-09-15T01:43:16.195 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.195 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3917160701}) 2024-09-15T01:43:16.195 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3032541571}) 2024-09-15T01:43:16.195 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.195 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.992+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3917160701}) 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.993+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3184925699}) 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.993+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389329786}) 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1886669374}) 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389329786}) 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3253061735}) 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389329786}) 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4018326605}) 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389329786}) 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2989875757}) 2024-09-15T01:43:16.196 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389329786}) 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=285913422}) 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389329786}) 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=194802730}) 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389329786}) 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1343275463}) 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389329786}) 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.994+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3340028834}) 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943673285}) 2024-09-15T01:43:16.197 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413423448}) 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943673285}) 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=803071397}) 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943673285}) 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2071524164}) 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943673285}) 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=371159924}) 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943673285}) 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=242552608}) 2024-09-15T01:43:16.198 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943673285}) 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3279393703}) 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943673285}) 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1630530356}) 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943673285}) 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.995+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3399066615}) 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3547782306}) 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2949404641}) 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3547782306}) 2024-09-15T01:43:16.199 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1348368268}) 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3547782306}) 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1393676790}) 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3547782306}) 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2420722623}) 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3547782306}) 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2276488962}) 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3547782306}) 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3682712893}) 2024-09-15T01:43:16.200 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3547782306}) 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3965730192}) 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3547782306}) 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.996+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1680985250}) 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.997+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.997+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1142529249}) 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.997+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2404724969}) 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.997+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1142529249}) 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1944755966}) 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1142529249}) 2024-09-15T01:43:16.201 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1925433227}) 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1142529249}) 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3100189004}) 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1142529249}) 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=997149725}) 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1142529249}) 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2817086874}) 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1142529249}) 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2995872587}) 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.202 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1142529249}) 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.998+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3568354242}) 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596386916}) 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=80222234}) 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596386916}) 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1178507198}) 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596386916}) 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1014039763}) 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596386916}) 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1981458453}) 2024-09-15T01:43:16.203 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596386916}) 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4133116424}) 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596386916}) 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3925127751}) 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596386916}) 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2293572001}) 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596386916}) 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:15.999+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2354817384}) 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.204 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1073771082}) 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=863559484}) 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1073771082}) 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3368905319}) 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1073771082}) 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3211848485}) 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1073771082}) 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1825391948}) 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1073771082}) 2024-09-15T01:43:16.205 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=639941271}) 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1073771082}) 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=212171492}) 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1073771082}) 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2657497715}) 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1073771082}) 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.000+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4119541291}) 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2187575005}) 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1460135333}) 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2187575005}) 2024-09-15T01:43:16.206 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1144961532}) 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2187575005}) 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4233737186}) 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2187575005}) 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2587718119}) 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2187575005}) 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1124412354}) 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2187575005}) 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.001+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4114267840}) 2024-09-15T01:43:16.207 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.002+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.002+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2187575005}) 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.002+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=888988096}) 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.002+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.002+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2187575005}) 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.002+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=541772997}) 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1131932217}) 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3338742932}) 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1131932217}) 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2201939795}) 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.208 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1131932217}) 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3019239754}) 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1131932217}) 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3114906722}) 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1131932217}) 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=956357950}) 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1131932217}) 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=766845306}) 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1131932217}) 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2226208085}) 2024-09-15T01:43:16.209 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1131932217}) 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.003+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=395441097}) 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2725560973}) 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3596067062}) 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2725560973}) 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2733845444}) 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2725560973}) 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2080026142}) 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.210 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2725560973}) 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2804331356}) 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2725560973}) 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2295914553}) 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2725560973}) 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3020765698}) 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2725560973}) 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1967094520}) 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2725560973}) 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.004+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1352990314}) 2024-09-15T01:43:16.211 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2229521239}) 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2681093345}) 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2229521239}) 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4122311326}) 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2229521239}) 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=499196398}) 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2229521239}) 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4073586343}) 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.212 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2229521239}) 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2333464912}) 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2229521239}) 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1978333481}) 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2229521239}) 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4131715103}) 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2229521239}) 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.005+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3175036342}) 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.007+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.007+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=55} crc {logm=1680192630,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3746332116,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=956710772,osdmap=3732762086}) 2024-09-15T01:43:16.213 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=55} crc {logm=1680192630,mdsmap=1531151339,mgr=3532922113,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3855028688,mon_config_key=1111662797,monmap=2850442478,osd_metadata=3185943182,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=929039172}) 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=55} crc {logm=1680192630,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3746332116,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=956710772,osdmap=3732762086}) 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=56} crc {logm=1680192630,mdsmap=1531151339,mgr=743505257,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3332854910,mon_config_key=1111662797,monmap=2850442479,osd_metadata=4061885386,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=420651870}) 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=55} crc {logm=1680192630,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3746332116,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=956710772,osdmap=3732762086}) 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=54} crc {logm=1680192630,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3643501266,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=893346404}) 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=55} crc {logm=1680192630,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3746332116,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=956710772,osdmap=3732762086}) 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=56} crc {logm=1680192630,mdsmap=1531151339,mgr=4079914519,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2734874960,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=288124044}) 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=55} crc {logm=1680192630,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3746332116,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=956710772,osdmap=3732762086}) 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=55} crc {logm=1680192630,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3878110537,mon_config_key=1111662797,monmap=2850442478,osd_metadata=4292802864,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2309035071}) 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.214 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=55} crc {logm=1680192630,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3746332116,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=956710772,osdmap=3732762086}) 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=55} crc {logm=1680192630,mdsmap=1531151339,mgr=3506364876,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1169732317,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=1660971494}) 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=55} crc {logm=1680192630,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3746332116,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=956710772,osdmap=3732762086}) 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=2,osd_metadata=5,osd_snap=2,osdmap=57} crc {logm=3519195171,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3855028688,mon_config_key=1111662797,monmap=1281023599,osd_metadata=2302127439,osd_snap=3178633393,osdmap=1530628615}) 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=55} crc {logm=1680192630,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3746332116,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=956710772,osdmap=3732762086}) 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.008+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=6,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=57} crc {logm=1149032884,mdsmap=1531151339,mgr=4228988328,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=138151189,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=543505836,osd_snap=3178633393,osdmap=2344132633}) 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=68} crc {osdmap=3173456640}) 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=67} crc {osdmap=3046542465}) 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=68} crc {osdmap=3173456640}) 2024-09-15T01:43:16.215 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=69} crc {osdmap=3175816719}) 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=68} crc {osdmap=3173456640}) 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=69} crc {osdmap=3406224216}) 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=68} crc {osdmap=3173456640}) 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=67} crc {osdmap=634582212}) 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=68} crc {osdmap=3173456640}) 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=67} crc {osdmap=3241794277}) 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=68} crc {osdmap=3173456640}) 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=69} crc {osdmap=3994788573}) 2024-09-15T01:43:16.216 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.217 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=68} crc {osdmap=3173456640}) 2024-09-15T01:43:16.217 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=69} crc {osdmap=1769232117}) 2024-09-15T01:43:16.217 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:43:16.217 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=68} crc {osdmap=3173456640}) 2024-09-15T01:43:16.217 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:43:16.017+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=67} crc {osdmap=4228290213}) 2024-09-15T01:43:16.923 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:43:16.924 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:43:17.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.016+0000 7f985fcc9640 10 monclient: get_monmap_and_config 2024-09-15T01:43:17.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.016+0000 7f985fcc9640 10 monclient: build_initial_monmap 2024-09-15T01:43:17.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.016+0000 7f985fcc9640 10 monclient: monmap: 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:17.017106+0000 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:17.017106+0000 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:17.019 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:17.020 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:17.020 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:17.020 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:17.020 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:17.020 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.016+0000 7f985fcc9640 1 Processor -- start 2024-09-15T01:43:17.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.016+0000 7f985fcc9640 1 -- start start 2024-09-15T01:43:17.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.016+0000 7f985fcc9640 10 monclient: init 2024-09-15T01:43:17.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:17.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 10 monclient: _add_conns ranks=[4,5,8,6,10,3,9,0,2,1,7] 2024-09-15T01:43:17.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98580a2930 0x7f9858132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 10 monclient(hunting): picked mon.noname-c con 0x7f98580a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:17.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f985809faa0 0x7f985809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 10 monclient(hunting): picked mon.noname-h con 0x7f985809faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:17.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f98580a08c0 0x7f98580a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 10 monclient(hunting): picked mon.noname-e con 0x7f98580a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:17.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f985809e0f0 con 0x7f98580a2930 2024-09-15T01:43:17.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f985809dd80 con 0x7f985809faa0 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f98580a2d00 con 0x7f98580a08c0 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985fcc9640 10 monclient(hunting): authenticate will time out at 1401.767503s 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98580a2930 0x7f9858132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98580a2930 0x7f9858132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:32824/0 (socket says 172.21.15.178:32824) 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 1 -- 172.21.15.178:0/2764457047 learned_addr learned my addr 172.21.15.178:0/2764457047 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 10 monclient(hunting): get_auth_request con 0x7f98580a2930 auth_method 0 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985e266640 1 --2- 172.21.15.178:0/2764457047 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f98580a08c0 0x7f98580a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985e266640 10 monclient(hunting): get_auth_request con 0x7f98580a08c0 auth_method 0 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985e266640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985e266640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985e266640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985e266640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985e266640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985e266640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985d264640 1 --2- 172.21.15.178:0/2764457047 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f985809faa0 0x7f985809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 10 monclient(hunting): handle_auth_done global_id 44365 payload 274 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 1 -- 172.21.15.178:0/2764457047 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f98580a08c0 msgr2=0x7f98580a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 1 --2- 172.21.15.178:0/2764457047 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f98580a08c0 0x7f98580a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 1 -- 172.21.15.178:0/2764457047 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f985809faa0 msgr2=0x7f985809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 1 --2- 172.21.15.178:0/2764457047 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f985809faa0 0x7f985809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 10 monclient: _finish_hunting 0 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 1 monclient: found mon.noname-c 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985da65640 1 -- 172.21.15.178:0/2764457047 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985804bf60 con 0x7f98580a2930 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.017+0000 7f985e266640 1 --2- 172.21.15.178:0/2764457047 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f98580a08c0 0x7f98580a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985da65640 1 --2- 172.21.15.178:0/2764457047 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98580a2930 0x7f9858132de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9854009940 tx=0x7f98540091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=41cc00a9694ff5f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985ca63640 1 -- 172.21.15.178:0/2764457047 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f985402ac60 con 0x7f98580a2930 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985ca63640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985ca63640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985ca63640 10 monclient: dump: 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.033 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.034 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985ca63640 10 monclient: _finish_auth 0 2024-09-15T01:43:17.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985ca63640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:17.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985fcc9640 5 monclient: authenticate success, global_id 44365 2024-09-15T01:43:17.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985ca63640 1 -- 172.21.15.178:0/2764457047 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985402ade0 con 0x7f98580a2930 2024-09-15T01:43:17.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985fcc9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:17.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985ca63640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:17.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985fcc9640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:17.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985fcc9640 4 set_mon_vals no callback set 2024-09-15T01:43:17.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985ca63640 1 -- 172.21.15.178:0/2764457047 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9854031ca0 con 0x7f98580a2930 2024-09-15T01:43:17.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985ca63640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985ca63640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.018+0000 7f985ca63640 10 monclient: dump: 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.036 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 10 monclient: shutdown 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 1 -- 172.21.15.178:0/2764457047 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98580a2930 msgr2=0x7f9858132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 1 --2- 172.21.15.178:0/2764457047 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98580a2930 0x7f9858132de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9854009940 tx=0x7f98540091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 1 -- 172.21.15.178:0/2764457047 shutdown_connections 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 1 --2- 172.21.15.178:0/2764457047 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f98580a08c0 0x7f98580a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 1 --2- 172.21.15.178:0/2764457047 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f985809faa0 0x7f985809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 1 --2- 172.21.15.178:0/2764457047 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98580a2930 0x7f9858132de0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 1 -- 172.21.15.178:0/2764457047 >> 172.21.15.178:0/2764457047 conn(0x7f98580a5ed0 msgr2=0x7f98580a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:17.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 1 -- 172.21.15.178:0/2764457047 shutdown_connections 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 1 -- 172.21.15.178:0/2764457047 wait complete. 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 10 monclient: build_initial_monmap 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.019+0000 7f985fcc9640 10 monclient: monmap: 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:17.038 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 1 Processor -- start 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 1 -- start start 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 10 monclient: init 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 10 monclient: _add_conns ranks=[0,8,5,6,3,2,7,4,1] 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f985809faa0 0x7f985809b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 10 monclient(hunting): picked mon.noname-a con 0x7f985809faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f98580a08c0 0x7f98581318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 10 monclient(hunting): picked mon.noname-e con 0x7f98580a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f98580a2930 0x7f985811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 10 monclient(hunting): picked mon.noname-h con 0x7f98580a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985809d630 con 0x7f985809faa0 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f985811ef90 con 0x7f98580a2930 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9858142b70 con 0x7f98580a08c0 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985fcc9640 10 monclient(hunting): authenticate will time out at 1401.770743s 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985da65640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f985809faa0 0x7f985809b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985d264640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f98580a08c0 0x7f98581318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985da65640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f985809faa0 0x7f985809b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48950/0 (socket says 172.21.15.178:48950) 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985da65640 1 -- 172.21.15.178:0/1034198921 learned_addr learned my addr 172.21.15.178:0/1034198921 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985da65640 10 monclient(hunting): get_auth_request con 0x7f985809faa0 auth_method 0 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985da65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985da65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985da65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985d264640 10 monclient(hunting): get_auth_request con 0x7f98580a08c0 auth_method 0 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985d264640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985d264640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985d264640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985da65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985da65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985da65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985d264640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985d264640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985d264640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.020+0000 7f985e266640 1 --2- 172.21.15.178:0/1034198921 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f98580a2930 0x7f985811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985e266640 10 monclient(hunting): get_auth_request con 0x7f98580a2930 auth_method 0 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985e266640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985e266640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985e266640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985da65640 10 monclient(hunting): handle_auth_done global_id 34875 payload 824 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985da65640 1 -- 172.21.15.178:0/1034198921 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f98580a08c0 msgr2=0x7f98581318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985da65640 1 --2- 172.21.15.178:0/1034198921 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f98580a08c0 0x7f98581318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985da65640 1 -- 172.21.15.178:0/1034198921 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f98580a2930 msgr2=0x7f985811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985da65640 1 --2- 172.21.15.178:0/1034198921 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f98580a2930 0x7f985811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985da65640 10 monclient: _finish_hunting 0 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985da65640 1 monclient: found mon.noname-a 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985da65640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985da65640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985da65640 1 -- 172.21.15.178:0/1034198921 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9850009a20 con 0x7f985809faa0 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985e266640 1 --2- 172.21.15.178:0/1034198921 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f98580a2930 0x7f985811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985d264640 1 --2- 172.21.15.178:0/1034198921 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f98580a08c0 0x7f98581318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985da65640 1 --2- 172.21.15.178:0/1034198921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f985809faa0 0x7f985809b280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9854035040 tx=0x7f9854032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984dffb640 1 -- 172.21.15.178:0/1034198921 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9854031820 con 0x7f985809faa0 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984dffb640 10 monclient: dump: 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.043 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.044 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984dffb640 10 monclient: _finish_auth 0 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985fcc9640 5 monclient: authenticate success, global_id 34875 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984dffb640 1 -- 172.21.15.178:0/1034198921 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98540319e0 con 0x7f985809faa0 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985fcc9640 10 monclient: _renew_subs 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984dffb640 1 -- 172.21.15.178:0/1034198921 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f985403a390 con 0x7f985809faa0 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985fcc9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985fcc9640 1 -- 172.21.15.178:0/1034198921 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9854009450 con 0x7f985809faa0 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984ffff640 4 set_mon_vals no callback set 2024-09-15T01:43:17.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f984dffb640 10 monclient: dump: 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.046 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985fcc9640 10 monclient: _renew_subs 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985fcc9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.021+0000 7f985fcc9640 1 -- 172.21.15.178:0/1034198921 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985811ecb0 con 0x7f985809faa0 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.022+0000 7f984dffb640 1 -- 172.21.15.178:0/1034198921 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9854039040 con 0x7f985809faa0 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.022+0000 7f985fcc9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.022+0000 7f985fcc9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.022+0000 7f985fcc9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.022+0000 7f985fcc9640 1 -- 172.21.15.178:0/1034198921 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985804b720 con 0x7f985809faa0 2024-09-15T01:43:17.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.022+0000 7f984dffb640 1 --2- 172.21.15.178:0/1034198921 >> v2:172.21.15.204:6812/28210 conn(0x7f983c0397b0 0x7f983c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.022+0000 7f984dffb640 1 -- 172.21.15.178:0/1034198921 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f985406b190 con 0x7f985809faa0 2024-09-15T01:43:17.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.023+0000 7f985d264640 1 --2- 172.21.15.178:0/1034198921 >> v2:172.21.15.204:6812/28210 conn(0x7f983c0397b0 0x7f983c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.023+0000 7f985d264640 10 monclient: get_auth_request con 0x7f983c0397b0 auth_method 0 2024-09-15T01:43:17.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.023+0000 7f985d264640 1 --2- 172.21.15.178:0/1034198921 >> v2:172.21.15.204:6812/28210 conn(0x7f983c0397b0 0x7f983c03bc60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f985809b1d0 tx=0x7f9848008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.027+0000 7f984dffb640 1 -- 172.21.15.178:0/1034198921 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f985402fa80 con 0x7f985809faa0 2024-09-15T01:43:17.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.027+0000 7f984dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:17.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.027+0000 7f984dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:17.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.240+0000 7f985fcc9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:43:17.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.240+0000 7f985fcc9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:43:17.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.240+0000 7f985fcc9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:43:17.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.240+0000 7f985fcc9640 1 -- 172.21.15.178:0/1034198921 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f985809cef0 con 0x7f985809faa0 2024-09-15T01:43:17.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.241+0000 7f984dffb640 1 -- 172.21.15.178:0/1034198921 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7f9854041080 con 0x7f985809faa0 2024-09-15T01:43:17.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.241+0000 7f984dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:43:17.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.241+0000 7f984dffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:17.243 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":122,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":4,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:43:17.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 1 -- 172.21.15.178:0/1034198921 >> v2:172.21.15.204:6812/28210 conn(0x7f983c0397b0 msgr2=0x7f983c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 1 --2- 172.21.15.178:0/1034198921 >> v2:172.21.15.204:6812/28210 conn(0x7f983c0397b0 0x7f983c03bc60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f985809b1d0 tx=0x7f9848008040 comp rx=0 tx=0).stop 2024-09-15T01:43:17.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 10 monclient: shutdown 2024-09-15T01:43:17.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:17.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 1 -- 172.21.15.178:0/1034198921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f985809faa0 msgr2=0x7f985809b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 1 --2- 172.21.15.178:0/1034198921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f985809faa0 0x7f985809b280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9854035040 tx=0x7f9854032000 comp rx=0 tx=0).stop 2024-09-15T01:43:17.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 1 -- 172.21.15.178:0/1034198921 shutdown_connections 2024-09-15T01:43:17.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 1 --2- 172.21.15.178:0/1034198921 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f98580a2930 0x7f985811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 1 --2- 172.21.15.178:0/1034198921 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f98580a08c0 0x7f98581318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 1 --2- 172.21.15.178:0/1034198921 >> v2:172.21.15.204:6812/28210 conn(0x7f983c0397b0 0x7f983c03bc60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 1 --2- 172.21.15.178:0/1034198921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f985809faa0 0x7f985809b280 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 1 -- 172.21.15.178:0/1034198921 >> 172.21.15.178:0/1034198921 conn(0x7f98580a5ed0 msgr2=0x7f9858135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:17.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 1 -- 172.21.15.178:0/1034198921 shutdown_connections 2024-09-15T01:43:17.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.243+0000 7f985fcc9640 1 -- 172.21.15.178:0/1034198921 wait complete. 2024-09-15T01:43:17.255 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:43:17.255 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T01:43:17.255 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:43:17.255 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:43:17.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.346+0000 7efd7319d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:17.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.346+0000 7efd7319d640 10 monclient: build_initial_monmap 2024-09-15T01:43:17.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.346+0000 7efd7319d640 10 monclient: monmap: 2024-09-15T01:43:17.349 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:17.349 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.349 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:17.347687+0000 2024-09-15T01:43:17.349 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:17.347687+0000 2024-09-15T01:43:17.349 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:17.349 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.349 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:17.349 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:17.349 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:17.350 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:17.350 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:17.350 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:17.350 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:17.350 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:17.350 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:17.350 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:17.350 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:17.350 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.347+0000 7efd7319d640 1 Processor -- start 2024-09-15T01:43:17.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.347+0000 7efd7319d640 1 -- start start 2024-09-15T01:43:17.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.347+0000 7efd7319d640 10 monclient: init 2024-09-15T01:43:17.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 10 monclient: _add_conns ranks=[6,3,9,1,5,7,0,2,8,4,10] 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 0x7efd6c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 10 monclient(hunting): picked mon.noname-d con 0x7efd6c0a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd6c09faa0 0x7efd6c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 10 monclient(hunting): picked mon.noname-g con 0x7efd6c09faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efd6c0a08c0 0x7efd6c0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 10 monclient(hunting): picked mon.noname-a con 0x7efd6c0a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd6c09e0f0 con 0x7efd6c0a08c0 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c09dd80 con 0x7efd6c09faa0 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c0a2d00 con 0x7efd6c0a2930 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7319d640 10 monclient(hunting): authenticate will time out at 1402.098106s 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 0x7efd6c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7173a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efd6c0a08c0 0x7efd6c0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7173a640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efd6c0a08c0 0x7efd6c0a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48966/0 (socket says 172.21.15.178:48966) 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 0x7efd6c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44392/0 (socket says 172.21.15.178:44392) 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7173a640 1 -- 172.21.15.178:0/2154758708 learned_addr learned my addr 172.21.15.178:0/2154758708 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7173a640 10 monclient(hunting): get_auth_request con 0x7efd6c0a08c0 auth_method 0 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7173a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7173a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7173a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 10 monclient(hunting): get_auth_request con 0x7efd6c0a2930 auth_method 0 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7173a640 1 -- 172.21.15.178:0/2154758708 >> v2:172.21.15.178:3300/0 conn(0x7efd6c0a08c0 msgr2=0x7efd6c0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7173a640 1 -- 172.21.15.178:0/2154758708 >> v2:172.21.15.178:3300/0 conn(0x7efd6c0a08c0 msgr2=0x7efd6c0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7173a640 1 --2- 172.21.15.178:0/2154758708 >> v2:172.21.15.178:3300/0 conn(0x7efd6c0a08c0 0x7efd6c0a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd7173a640 1 --2- 172.21.15.178:0/2154758708 >> v2:172.21.15.178:3300/0 conn(0x7efd6c0a08c0 0x7efd6c0a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd6bfff640 1 --2- 172.21.15.178:0/2154758708 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd6c09faa0 0x7efd6c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd6bfff640 10 monclient(hunting): get_auth_request con 0x7efd6c09faa0 auth_method 0 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd6bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 10 monclient(hunting): handle_auth_done global_id 25215 payload 274 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 1 -- 172.21.15.178:0/2154758708 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd6c09faa0 msgr2=0x7efd6c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 1 --2- 172.21.15.178:0/2154758708 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd6c09faa0 0x7efd6c09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 1 -- 172.21.15.178:0/2154758708 >> v2:172.21.15.178:3300/0 conn(0x7efd6c0a08c0 msgr2=0x7efd6c0a0d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 1 --2- 172.21.15.178:0/2154758708 >> v2:172.21.15.178:3300/0 conn(0x7efd6c0a08c0 0x7efd6c0a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 10 monclient: _finish_hunting 0 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 1 monclient: found mon.noname-d 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:17.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.348+0000 7efd70f39640 1 -- 172.21.15.178:0/2154758708 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c04bf60 con 0x7efd6c0a2930 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd70f39640 1 --2- 172.21.15.178:0/2154758708 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 0x7efd6c132de0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009e60 tx=0x7efd5c009780 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7685e394462a883f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd6b7fe640 1 -- 172.21.15.178:0/2154758708 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd5c0079d0 con 0x7efd6c0a2930 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd6b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd6b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd6b7fe640 10 monclient: dump: 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.365 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.366 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.366 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.366 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.366 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.366 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.366 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.366 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.366 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd6b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:43:17.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:17.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd7319d640 5 monclient: authenticate success, global_id 25215 2024-09-15T01:43:17.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd6b7fe640 1 -- 172.21.15.178:0/2154758708 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c02d050 con 0x7efd6c0a2930 2024-09-15T01:43:17.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd7319d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:17.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd6b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:17.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd7319d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:17.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd6b7fe640 1 -- 172.21.15.178:0/2154758708 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd5c007bd0 con 0x7efd6c0a2930 2024-09-15T01:43:17.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd7319d640 4 set_mon_vals no callback set 2024-09-15T01:43:17.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd6b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd6b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:17.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd6b7fe640 10 monclient: dump: 2024-09-15T01:43:17.367 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.367 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.368 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd7319d640 10 monclient: shutdown 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd7319d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd7319d640 1 -- 172.21.15.178:0/2154758708 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 msgr2=0x7efd6c132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd7319d640 1 --2- 172.21.15.178:0/2154758708 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 0x7efd6c132de0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009e60 tx=0x7efd5c009780 comp rx=0 tx=0).stop 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd7319d640 1 -- 172.21.15.178:0/2154758708 shutdown_connections 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd7319d640 1 --2- 172.21.15.178:0/2154758708 >> v2:172.21.15.178:3300/0 conn(0x7efd6c0a08c0 0x7efd6c0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.349+0000 7efd7319d640 1 --2- 172.21.15.178:0/2154758708 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd6c09faa0 0x7efd6c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.350+0000 7efd7319d640 1 --2- 172.21.15.178:0/2154758708 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 0x7efd6c132de0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.350+0000 7efd7319d640 1 -- 172.21.15.178:0/2154758708 >> 172.21.15.178:0/2154758708 conn(0x7efd6c0a5ed0 msgr2=0x7efd6c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.350+0000 7efd7319d640 1 -- 172.21.15.178:0/2154758708 shutdown_connections 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.350+0000 7efd7319d640 1 -- 172.21.15.178:0/2154758708 wait complete. 2024-09-15T01:43:17.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.350+0000 7efd7319d640 10 monclient: build_initial_monmap 2024-09-15T01:43:17.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.350+0000 7efd7319d640 10 monclient: monmap: 2024-09-15T01:43:17.370 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:17.370 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:17.370 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:17.370 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:17.370 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:17.370 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:17.370 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:17.370 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:17.370 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:17.370 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:17.370 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.350+0000 7efd7319d640 1 Processor -- start 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 1 -- start start 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 10 monclient: init 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 10 monclient: _add_conns ranks=[3,0,6,7,4,5,2,8,1] 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd6c09faa0 0x7efd6c09af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 10 monclient(hunting): picked mon.noname-g con 0x7efd6c09faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:17.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c0a08c0 0x7efd6c131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 10 monclient(hunting): picked mon.noname-a con 0x7efd6c0a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:17.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 0x7efd6c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 10 monclient(hunting): picked mon.noname-d con 0x7efd6c0a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:17.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c142a60 con 0x7efd6c0a08c0 2024-09-15T01:43:17.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c09d630 con 0x7efd6c09faa0 2024-09-15T01:43:17.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c09bef0 con 0x7efd6c0a2930 2024-09-15T01:43:17.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:17.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7319d640 10 monclient(hunting): authenticate will time out at 1402.101439s 2024-09-15T01:43:17.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 0x7efd6c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd6bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c0a08c0 0x7efd6c131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 0x7efd6c11c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44408/0 (socket says 172.21.15.178:44408) 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 1 -- 172.21.15.178:0/2791047738 learned_addr learned my addr 172.21.15.178:0/2791047738 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 10 monclient(hunting): get_auth_request con 0x7efd6c0a2930 auth_method 0 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd6bfff640 10 monclient(hunting): get_auth_request con 0x7efd6c0a08c0 auth_method 0 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd6bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd70f39640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd6c09faa0 0x7efd6c09af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd70f39640 10 monclient(hunting): get_auth_request con 0x7efd6c09faa0 auth_method 0 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd70f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd70f39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd70f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd70f39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd70f39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd70f39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 10 monclient(hunting): handle_auth_done global_id 25224 payload 824 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 1 -- 172.21.15.178:0/2791047738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd6c09faa0 msgr2=0x7efd6c09af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd6c09faa0 0x7efd6c09af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 1 -- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c0a08c0 msgr2=0x7efd6c131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c0a08c0 0x7efd6c131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 10 monclient: _finish_hunting 0 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 1 monclient: found mon.noname-d 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 1 -- 172.21.15.178:0/2791047738 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c009a20 con 0x7efd6c0a2930 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd6bfff640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c0a08c0 0x7efd6c131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.351+0000 7efd7173a640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 0x7efd6c11c530 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efd6400d890 tx=0x7efd64008040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd6400b3f0 con 0x7efd6c0a2930 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd68ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:17.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd68ff9640 10 monclient: dump: 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.376 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd68ff9640 10 monclient: _finish_auth 0 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd68ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd7319d640 5 monclient: authenticate success, global_id 25224 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd64009b20 con 0x7efd6c0a2930 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd68ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd7319d640 10 monclient: _renew_subs 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd64009cd0 con 0x7efd6c0a2930 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd7319d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd7319d640 1 -- 172.21.15.178:0/2791047738 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c09b9e0 con 0x7efd6c0a2930 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd68ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd6affd640 4 set_mon_vals no callback set 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd68ff9640 10 monclient: dump: 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.378 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.379 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.379 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.379 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.380 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd7319d640 10 monclient: _renew_subs 2024-09-15T01:43:17.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd7319d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:43:17.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.352+0000 7efd7319d640 1 -- 172.21.15.178:0/2791047738 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c11d9e0 con 0x7efd6c0a2930 2024-09-15T01:43:17.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7efd640096a0 con 0x7efd6c0a2930 2024-09-15T01:43:17.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd7319d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:17.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd7319d640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c131d60 0x7efd6c11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd7319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd7319d640 1 -- 172.21.15.178:0/2791047738 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c11eae0 con 0x7efd6c131d60 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd70f39640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c131d60 0x7efd6c11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd70f39640 10 monclient: get_auth_request con 0x7efd6c131d60 auth_method 0 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd70f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd70f39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd70f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd68ff9640 1 --2- 172.21.15.178:0/2791047738 >> v2:172.21.15.204:6812/28210 conn(0x7efd580397e0 0x7efd5803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd70f39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd70f39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd70f39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7efd6404c050 con 0x7efd6c0a2930 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd70f39640 10 monclient(hunting): handle_auth_done global_id 34893 payload 824 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd70f39640 1 -- 172.21.15.178:0/2791047738 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd6c04c4a0 con 0x7efd6c131d60 2024-09-15T01:43:17.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd6bfff640 1 --2- 172.21.15.178:0/2791047738 >> v2:172.21.15.204:6812/28210 conn(0x7efd580397e0 0x7efd5803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd6bfff640 10 monclient: get_auth_request con 0x7efd580397e0 auth_method 0 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd70f39640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c131d60 0x7efd6c11ca70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0a3290 tx=0x7efd5c037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd5c031400 con 0x7efd6c131d60 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd68ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.353+0000 7efd68ff9640 10 monclient: dump: 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.382 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.354+0000 7efd6bfff640 1 --2- 172.21.15.178:0/2791047738 >> v2:172.21.15.204:6812/28210 conn(0x7efd580397e0 0x7efd5803bc90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0a6ec0 tx=0x7efd600090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.354+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7efd5c03eb30 con 0x7efd6c131d60 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.354+0000 7efd68ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.354+0000 7efd68ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:17.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.354+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c131d60 msgr2=0x7efd6c11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.354+0000 7efd68ff9640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c131d60 0x7efd6c11ca70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0a3290 tx=0x7efd5c037000 comp rx=0 tx=0).stop 2024-09-15T01:43:17.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.354+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c131d60 msgr2=0x7efd6c11ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:17.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.354+0000 7efd68ff9640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c131d60 0x7efd6c11ca70 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0a3290 tx=0x7efd5c037000 comp rx=0 tx=0).stop 2024-09-15T01:43:17.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.359+0000 7efd7319d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:17.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.359+0000 7efd7319d640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c11cfb0 0x7efd6c096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.359+0000 7efd7319d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.359+0000 7efd7319d640 1 -- 172.21.15.178:0/2791047738 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c132130 con 0x7efd6c11cfb0 2024-09-15T01:43:17.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.359+0000 7efd70f39640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c11cfb0 0x7efd6c096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.359+0000 7efd70f39640 10 monclient: get_auth_request con 0x7efd6c11cfb0 auth_method 0 2024-09-15T01:43:17.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.359+0000 7efd70f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.359+0000 7efd70f39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.359+0000 7efd70f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.359+0000 7efd70f39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.359+0000 7efd70f39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd70f39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd70f39640 10 monclient(hunting): handle_auth_done global_id 34902 payload 824 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd70f39640 1 -- 172.21.15.178:0/2791047738 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efd6c04bc80 con 0x7efd6c11cfb0 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd70f39640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c11cfb0 0x7efd6c096a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0097c0 tx=0x7efd5c040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd5c031de0 con 0x7efd6c11cfb0 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd68ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd68ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd68ff9640 10 monclient: dump: 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.387 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7efd5c03e710 con 0x7efd6c11cfb0 2024-09-15T01:43:17.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd68ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:17.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd68ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:17.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c11cfb0 msgr2=0x7efd6c096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd68ff9640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c11cfb0 0x7efd6c096a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0097c0 tx=0x7efd5c040040 comp rx=0 tx=0).stop 2024-09-15T01:43:17.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd68ff9640 1 -- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c11cfb0 msgr2=0x7efd6c096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:17.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd68ff9640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd6c11cfb0 0x7efd6c096a30 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0097c0 tx=0x7efd5c040040 comp rx=0 tx=0).stop 2024-09-15T01:43:17.389 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:17.389 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:17.389 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:17.389 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:17.390 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:17.390 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:17.390 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:17.390 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:17.390 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:17.390 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:17.390 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:17.390 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:17.390 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:17.390 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:17.390 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:17.390 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:17.391 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.392 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:17.392 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:17.392 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:17.392 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:17.392 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:17.392 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:17.392 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:17.392 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:17.392 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:17.392 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:17.392 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:17.392 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:17.393 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:17.394 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:17.395 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:17.397 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.398 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:17.399 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:17.400 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:17.402 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:17.405 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:17.407 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:17.408 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:17.410 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:17.411 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:17.412 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:17.412 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.412 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.412 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:17.412 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:17.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.360+0000 7efd7319d640 1 -- 172.21.15.178:0/2791047738 >> v2:172.21.15.204:6812/28210 conn(0x7efd580397e0 msgr2=0x7efd5803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.361+0000 7efd7319d640 1 --2- 172.21.15.178:0/2791047738 >> v2:172.21.15.204:6812/28210 conn(0x7efd580397e0 0x7efd5803bc90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0a6ec0 tx=0x7efd600090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.361+0000 7efd7319d640 10 monclient: shutdown 2024-09-15T01:43:17.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.361+0000 7efd7319d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:17.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.361+0000 7efd7319d640 1 -- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 msgr2=0x7efd6c11c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.361+0000 7efd7319d640 1 --2- 172.21.15.178:0/2791047738 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7efd6c0a2930 0x7efd6c11c530 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efd6400d890 tx=0x7efd64008040 comp rx=0 tx=0).stop 2024-09-15T01:43:17.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.361+0000 7efd7173a640 1 -- 172.21.15.178:0/2791047738 reap_dead start 2024-09-15T01:43:17.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.361+0000 7efd7173a640 1 -- 172.21.15.178:0/2791047738 reap_dead start 2024-09-15T01:43:17.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.361+0000 7efd7319d640 1 -- 172.21.15.178:0/2791047738 shutdown_connections 2024-09-15T01:43:17.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.361+0000 7efd7319d640 1 -- 172.21.15.178:0/2791047738 >> 172.21.15.178:0/2791047738 conn(0x7efd6c0a5ed0 msgr2=0x7efd6c09e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:17.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.361+0000 7efd7319d640 1 -- 172.21.15.178:0/2791047738 shutdown_connections 2024-09-15T01:43:17.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.361+0000 7efd7319d640 1 -- 172.21.15.178:0/2791047738 wait complete. 2024-09-15T01:43:17.414 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:17.414 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:43:17.415 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:43:17.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.506+0000 7fa32fea8640 10 monclient: get_monmap_and_config 2024-09-15T01:43:17.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.506+0000 7fa32fea8640 10 monclient: build_initial_monmap 2024-09-15T01:43:17.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.506+0000 7fa32fea8640 10 monclient: monmap: 2024-09-15T01:43:17.509 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:17.509 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.509 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:17.507571+0000 2024-09-15T01:43:17.509 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:17.507571+0000 2024-09-15T01:43:17.509 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:17.509 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.509 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 1 Processor -- start 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 1 -- start start 2024-09-15T01:43:17.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 10 monclient: init 2024-09-15T01:43:17.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:17.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 10 monclient: _add_conns ranks=[6,7,8,1,9,5,2,3,10,4,0] 2024-09-15T01:43:17.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa3280a2930 0x7fa328132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 10 monclient(hunting): picked mon.noname-d con 0x7fa3280a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:17.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa32809faa0 0x7fa32809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 10 monclient(hunting): picked mon.noname-i con 0x7fa32809faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:17.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 0x7fa3280a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 10 monclient(hunting): picked mon.noname-e con 0x7fa3280a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:17.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa32809e0f0 con 0x7fa3280a2930 2024-09-15T01:43:17.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa32809dd80 con 0x7fa32809faa0 2024-09-15T01:43:17.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa3280a2d00 con 0x7fa3280a08c0 2024-09-15T01:43:17.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:17.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.507+0000 7fa32fea8640 10 monclient(hunting): authenticate will time out at 1402.257900s 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 0x7fa3280a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32dc44640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa3280a2930 0x7fa328132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 0x7fa3280a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:55158/0 (socket says 172.21.15.178:55158) 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32dc44640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa3280a2930 0x7fa328132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44420/0 (socket says 172.21.15.178:44420) 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 1 -- 172.21.15.178:0/942195654 learned_addr learned my addr 172.21.15.178:0/942195654 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 10 monclient(hunting): get_auth_request con 0x7fa3280a08c0 auth_method 0 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32dc44640 10 monclient(hunting): get_auth_request con 0x7fa3280a2930 auth_method 0 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32dc44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32dc44640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32dc44640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32d443640 1 --2- 172.21.15.178:0/942195654 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa32809faa0 0x7fa32809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32dc44640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32dc44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32dc44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32d443640 10 monclient(hunting): get_auth_request con 0x7fa32809faa0 auth_method 0 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32d443640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32d443640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32d443640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32d443640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32d443640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32d443640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 10 monclient(hunting): handle_auth_done global_id 34721 payload 274 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 1 -- 172.21.15.178:0/942195654 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa32809faa0 msgr2=0x7fa32809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 1 --2- 172.21.15.178:0/942195654 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa32809faa0 0x7fa32809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 1 -- 172.21.15.178:0/942195654 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa3280a2930 msgr2=0x7fa328132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 1 --2- 172.21.15.178:0/942195654 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa3280a2930 0x7fa328132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 10 monclient: _finish_hunting 0 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 1 monclient: found mon.noname-e 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32e445640 1 -- 172.21.15.178:0/942195654 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32804bf60 con 0x7fa3280a08c0 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.508+0000 7fa32dc44640 1 --2- 172.21.15.178:0/942195654 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa3280a2930 0x7fa328132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32e445640 1 --2- 172.21.15.178:0/942195654 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 0x7fa3280a0d30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa320009f50 tx=0x7fa320009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=91e15f96ff1247d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32cc42640 1 -- 172.21.15.178:0/942195654 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa320007880 con 0x7fa3280a08c0 2024-09-15T01:43:17.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32cc42640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32cc42640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:17.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32cc42640 10 monclient: dump: 2024-09-15T01:43:17.524 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.524 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.524 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.524 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.524 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.524 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.524 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32cc42640 10 monclient: _finish_auth 0 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32cc42640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32cc42640 1 -- 172.21.15.178:0/942195654 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32002d050 con 0x7fa3280a08c0 2024-09-15T01:43:17.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 5 monclient: authenticate success, global_id 34721 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32cc42640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32cc42640 1 -- 172.21.15.178:0/942195654 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa320030360 con 0x7fa3280a08c0 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 4 set_mon_vals no callback set 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32cc42640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32cc42640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32cc42640 10 monclient: dump: 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.526 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 10 monclient: shutdown 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:17.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 1 -- 172.21.15.178:0/942195654 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 msgr2=0x7fa3280a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 1 --2- 172.21.15.178:0/942195654 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 0x7fa3280a0d30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa320009f50 tx=0x7fa320009840 comp rx=0 tx=0).stop 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 1 -- 172.21.15.178:0/942195654 shutdown_connections 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 1 --2- 172.21.15.178:0/942195654 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 0x7fa3280a0d30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 1 --2- 172.21.15.178:0/942195654 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa32809faa0 0x7fa32809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 1 --2- 172.21.15.178:0/942195654 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa3280a2930 0x7fa328132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 1 -- 172.21.15.178:0/942195654 >> 172.21.15.178:0/942195654 conn(0x7fa3280a5ed0 msgr2=0x7fa3280a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 1 -- 172.21.15.178:0/942195654 shutdown_connections 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.509+0000 7fa32fea8640 1 -- 172.21.15.178:0/942195654 wait complete. 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.510+0000 7fa32fea8640 10 monclient: build_initial_monmap 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.510+0000 7fa32fea8640 10 monclient: monmap: 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:17.528 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:17.529 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.510+0000 7fa32fea8640 1 Processor -- start 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.510+0000 7fa32fea8640 1 -- start start 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.510+0000 7fa32fea8640 10 monclient: init 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 10 monclient: _add_conns ranks=[5,8,2,3,4,0,7,6,1] 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa32809faa0 0x7fa32809b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 10 monclient(hunting): picked mon.noname-h con 0x7fa32809faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 0x7fa328131b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 10 monclient(hunting): picked mon.noname-e con 0x7fa3280a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3280a2930 0x7fa3281320a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 10 monclient(hunting): picked mon.noname-b con 0x7fa3280a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32809c100 con 0x7fa3280a2930 2024-09-15T01:43:17.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3280a2390 con 0x7fa32809faa0 2024-09-15T01:43:17.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa32809d630 con 0x7fa3280a08c0 2024-09-15T01:43:17.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:17.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 10 monclient(hunting): authenticate will time out at 1402.261181s 2024-09-15T01:43:17.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 0x7fa328131b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 0x7fa328131b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:55170/0 (socket says 172.21.15.178:55170) 2024-09-15T01:43:17.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 1 -- 172.21.15.178:0/429967899 learned_addr learned my addr 172.21.15.178:0/429967899 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:17.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 10 monclient(hunting): get_auth_request con 0x7fa3280a08c0 auth_method 0 2024-09-15T01:43:17.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32e445640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3280a2930 0x7fa3281320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32e445640 10 monclient(hunting): get_auth_request con 0x7fa3280a2930 auth_method 0 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32e445640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32e445640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32e445640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32dc44640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa32809faa0 0x7fa32809b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32dc44640 10 monclient(hunting): get_auth_request con 0x7fa32809faa0 auth_method 0 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32dc44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32dc44640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32dc44640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32e445640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32e445640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32e445640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32dc44640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32dc44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32dc44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 10 monclient(hunting): handle_auth_done global_id 34730 payload 824 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 1 -- 172.21.15.178:0/429967899 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa32809faa0 msgr2=0x7fa32809b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa32809faa0 0x7fa32809b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 1 -- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3280a2930 msgr2=0x7fa3281320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3280a2930 0x7fa3281320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 10 monclient: _finish_hunting 0 2024-09-15T01:43:17.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 1 monclient: found mon.noname-e 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 1 -- 172.21.15.178:0/429967899 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa324009390 con 0x7fa3280a08c0 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32d443640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 0x7fa328131b60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa31800e900 tx=0x7fa3180090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa318009480 con 0x7fa3280a08c0 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31dffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31dffb640 10 monclient: dump: 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.534 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31dffb640 10 monclient: _finish_auth 0 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31800cd60 con 0x7fa3280a08c0 2024-09-15T01:43:17.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 5 monclient: authenticate success, global_id 34730 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 10 monclient: _renew_subs 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa31800ced0 con 0x7fa3280a08c0 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa32fea8640 1 -- 172.21.15.178:0/429967899 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa320009ae0 con 0x7fa3280a08c0 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31ffff640 4 set_mon_vals no callback set 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.511+0000 7fa31dffb640 10 monclient: dump: 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.536 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.512+0000 7fa32fea8640 10 monclient: _renew_subs 2024-09-15T01:43:17.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.512+0000 7fa32fea8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:43:17.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.512+0000 7fa32fea8640 1 -- 172.21.15.178:0/429967899 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32811ec80 con 0x7fa3280a08c0 2024-09-15T01:43:17.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.512+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa31800c8e0 con 0x7fa3280a08c0 2024-09-15T01:43:17.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.512+0000 7fa32fea8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:17.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.512+0000 7fa32fea8640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa328142520 0x7fa32811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.512+0000 7fa32fea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.512+0000 7fa32fea8640 1 -- 172.21.15.178:0/429967899 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32804c4a0 con 0x7fa328142520 2024-09-15T01:43:17.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32dc44640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa328142520 0x7fa32811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32dc44640 10 monclient: get_auth_request con 0x7fa328142520 auth_method 0 2024-09-15T01:43:17.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32dc44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32dc44640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32dc44640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 1 --2- 172.21.15.178:0/429967899 >> v2:172.21.15.204:6812/28210 conn(0x7fa30c0397e0 0x7fa30c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32dc44640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32dc44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32dc44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fa31804bc90 con 0x7fa3280a08c0 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32dc44640 10 monclient(hunting): handle_auth_done global_id 34715 payload 824 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32dc44640 1 -- 172.21.15.178:0/429967899 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa32805ed70 con 0x7fa328142520 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32dc44640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa328142520 0x7fa32811c530 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa324011bf0 tx=0x7fa3240090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32e445640 1 --2- 172.21.15.178:0/429967899 >> v2:172.21.15.204:6812/28210 conn(0x7fa30c0397e0 0x7fa30c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32e445640 10 monclient: get_auth_request con 0x7fa30c0397e0 auth_method 0 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa32400fdb0 con 0x7fa328142520 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 10 monclient: dump: 2024-09-15T01:43:17.539 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.540 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.541 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.541 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fa32401d480 con 0x7fa328142520 2024-09-15T01:43:17.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:17.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:17.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa328142520 msgr2=0x7fa32811c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa328142520 0x7fa32811c530 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa324011bf0 tx=0x7fa3240090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa328142520 msgr2=0x7fa32811c530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:17.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa31dffb640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa328142520 0x7fa32811c530 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.513+0000 7fa32e445640 1 --2- 172.21.15.178:0/429967899 >> v2:172.21.15.204:6812/28210 conn(0x7fa30c0397e0 0x7fa30c03bc90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa320033000 tx=0x7fa320036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32fea8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:17.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32fea8640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa32811ca70 0x7fa32811ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32fea8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32fea8640 1 -- 172.21.15.178:0/429967899 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32804bc80 con 0x7fa32811ca70 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32dc44640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa32811ca70 0x7fa32811ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32dc44640 10 monclient: get_auth_request con 0x7fa32811ca70 auth_method 0 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32dc44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32dc44640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32dc44640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32dc44640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32dc44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32dc44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32dc44640 10 monclient(hunting): handle_auth_done global_id 34724 payload 824 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.519+0000 7fa32dc44640 1 -- 172.21.15.178:0/429967899 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa32809fe70 con 0x7fa32811ca70 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32dc44640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa32811ca70 0x7fa32811ce70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa324011bf0 tx=0x7fa32401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa324025040 con 0x7fa32811ca70 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa31dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa31dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa31dffb640 10 monclient: dump: 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.545 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fa32401c3e0 con 0x7fa32811ca70 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa31dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa31dffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa32811ca70 msgr2=0x7fa32811ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa31dffb640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa32811ca70 0x7fa32811ce70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa324011bf0 tx=0x7fa32401d000 comp rx=0 tx=0).stop 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa31dffb640 1 -- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa32811ca70 msgr2=0x7fa32811ce70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:17.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa31dffb640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa32811ca70 0x7fa32811ce70 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa324011bf0 tx=0x7fa32401d000 comp rx=0 tx=0).stop 2024-09-15T01:43:17.547 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:17.547 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:17.547 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:17.547 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:17.547 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:17.547 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:17.547 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:17.547 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:17.547 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:17.547 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:17.547 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:17.548 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:17.549 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:17.550 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:17.551 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.552 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.553 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:17.554 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.555 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.556 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:17.557 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.558 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.559 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.560 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.561 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:17.562 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.563 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.564 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.565 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.568 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:17.568 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:17.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.568 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:17.568 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:17.568 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:17.568 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.568 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.568 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:17.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.568 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:17.568 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:17.569 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:17.569 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.569 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.569 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:17.569 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32fea8640 1 -- 172.21.15.178:0/429967899 >> v2:172.21.15.204:6812/28210 conn(0x7fa30c0397e0 msgr2=0x7fa30c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32fea8640 1 --2- 172.21.15.178:0/429967899 >> v2:172.21.15.204:6812/28210 conn(0x7fa30c0397e0 0x7fa30c03bc90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa320033000 tx=0x7fa320036000 comp rx=0 tx=0).stop 2024-09-15T01:43:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32fea8640 10 monclient: shutdown 2024-09-15T01:43:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32fea8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32fea8640 1 -- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 msgr2=0x7fa328131b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32fea8640 1 --2- 172.21.15.178:0/429967899 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa3280a08c0 0x7fa328131b60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa31800e900 tx=0x7fa3180090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32e445640 1 -- 172.21.15.178:0/429967899 reap_dead start 2024-09-15T01:43:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32e445640 1 -- 172.21.15.178:0/429967899 reap_dead start 2024-09-15T01:43:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32fea8640 1 -- 172.21.15.178:0/429967899 shutdown_connections 2024-09-15T01:43:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32fea8640 1 -- 172.21.15.178:0/429967899 >> 172.21.15.178:0/429967899 conn(0x7fa3280a5ed0 msgr2=0x7fa3280a53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32fea8640 1 -- 172.21.15.178:0/429967899 shutdown_connections 2024-09-15T01:43:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.520+0000 7fa32fea8640 1 -- 172.21.15.178:0/429967899 wait complete. 2024-09-15T01:43:17.572 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:17.572 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:43:17.572 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:43:17.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.664+0000 7f987955d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:17.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.664+0000 7f987955d640 10 monclient: build_initial_monmap 2024-09-15T01:43:17.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.664+0000 7f987955d640 10 monclient: monmap: 2024-09-15T01:43:17.666 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:17.666 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.666 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:17.664991+0000 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:17.664991+0000 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:17.667 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:17.668 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.664+0000 7f987955d640 1 Processor -- start 2024-09-15T01:43:17.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.664+0000 7f987955d640 1 -- start start 2024-09-15T01:43:17.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.664+0000 7f987955d640 10 monclient: init 2024-09-15T01:43:17.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 10 monclient: _add_conns ranks=[6,7,4,0,3,8,2,1,10,5,9] 2024-09-15T01:43:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 0x7f9874132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 10 monclient(hunting): picked mon.noname-d con 0x7f98740a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f987409faa0 0x7f987409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 10 monclient(hunting): picked mon.noname-i con 0x7f987409faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98740a08c0 0x7f98740a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 10 monclient(hunting): picked mon.noname-c con 0x7f98740a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f987409e0f0 con 0x7f98740a08c0 2024-09-15T01:43:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f987409dd80 con 0x7f98740a2930 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f98740a2d00 con 0x7f987409faa0 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f987955d640 10 monclient(hunting): authenticate will time out at 1402.415214s 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f98737fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98740a08c0 0x7f98740a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 0x7f9874132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 0x7f9874132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44424/0 (socket says 172.21.15.178:44424) 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f98737fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98740a08c0 0x7f98740a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:32826/0 (socket says 172.21.15.178:32826) 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 1 -- 172.21.15.178:0/312918713 learned_addr learned my addr 172.21.15.178:0/312918713 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 10 monclient(hunting): get_auth_request con 0x7f98740a2930 auth_method 0 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f98737fe640 10 monclient(hunting): get_auth_request con 0x7f98740a08c0 auth_method 0 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f98737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f98737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f98737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f98737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f98737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f98737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f986a7fc640 1 --2- 172.21.15.178:0/312918713 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f987409faa0 0x7f987409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 10 monclient(hunting): handle_auth_done global_id 25242 payload 274 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 1 -- 172.21.15.178:0/312918713 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f987409faa0 msgr2=0x7f987409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 1 --2- 172.21.15.178:0/312918713 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f987409faa0 0x7f987409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 1 -- 172.21.15.178:0/312918713 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98740a08c0 msgr2=0x7f98740a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 1 --2- 172.21.15.178:0/312918713 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98740a08c0 0x7f98740a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:43:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 1 monclient: found mon.noname-d 2024-09-15T01:43:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f9872ffd640 1 -- 172.21.15.178:0/312918713 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987404bf60 con 0x7f98740a2930 2024-09-15T01:43:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.665+0000 7f98737fe640 1 --2- 172.21.15.178:0/312918713 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98740a08c0 0x7f98740a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f9872ffd640 1 --2- 172.21.15.178:0/312918713 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 0x7f9874132de0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9864009940 tx=0x7f98640091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=2e31a694c298d71b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f98727fc640 1 -- 172.21.15.178:0/312918713 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f986402ac60 con 0x7f98740a2930 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f98727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f98727fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f98727fc640 10 monclient: dump: 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f98727fc640 10 monclient: _finish_auth 0 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f98727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f987955d640 5 monclient: authenticate success, global_id 25242 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f987955d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f98727fc640 1 -- 172.21.15.178:0/312918713 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986402ade0 con 0x7f98740a2930 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f98727fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f987955d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f987955d640 4 set_mon_vals no callback set 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f98727fc640 1 -- 172.21.15.178:0/312918713 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9864031ca0 con 0x7f98740a2930 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f98727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f98727fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.666+0000 7f98727fc640 10 monclient: dump: 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 10 monclient: shutdown 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 1 -- 172.21.15.178:0/312918713 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 msgr2=0x7f9874132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 1 --2- 172.21.15.178:0/312918713 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 0x7f9874132de0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9864009940 tx=0x7f98640091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 1 -- 172.21.15.178:0/312918713 shutdown_connections 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 1 --2- 172.21.15.178:0/312918713 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f98740a08c0 0x7f98740a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 1 --2- 172.21.15.178:0/312918713 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f987409faa0 0x7f987409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 1 --2- 172.21.15.178:0/312918713 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 0x7f9874132de0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 1 -- 172.21.15.178:0/312918713 >> 172.21.15.178:0/312918713 conn(0x7f98740a5ed0 msgr2=0x7f98740a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 1 -- 172.21.15.178:0/312918713 shutdown_connections 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 1 -- 172.21.15.178:0/312918713 wait complete. 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 10 monclient: build_initial_monmap 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.667+0000 7f987955d640 10 monclient: monmap: 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 1 Processor -- start 2024-09-15T01:43:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 1 -- start start 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 10 monclient: init 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 10 monclient: _add_conns ranks=[5,7,6,0,3,1,4,2,8] 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f987409faa0 0x7f987409b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 10 monclient(hunting): picked mon.noname-h con 0x7f987409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f98740a08c0 0x7f9874131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 10 monclient(hunting): picked mon.noname-i con 0x7f98740a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 0x7f987411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 10 monclient(hunting): picked mon.noname-d con 0x7f98740a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f987409d630 con 0x7f987409faa0 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f987409bcc0 con 0x7f98740a2930 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9874142ba0 con 0x7f98740a08c0 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f987955d640 10 monclient(hunting): authenticate will time out at 1402.418582s 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 0x7f987411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 0x7f987411c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44426/0 (socket says 172.21.15.178:44426) 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 1 -- 172.21.15.178:0/813350031 learned_addr learned my addr 172.21.15.178:0/813350031 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 10 monclient(hunting): get_auth_request con 0x7f98740a2930 auth_method 0 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f986a7fc640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f98740a08c0 0x7f9874131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f9872ffd640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f987409faa0 0x7f987409b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f986a7fc640 10 monclient(hunting): get_auth_request con 0x7f98740a08c0 auth_method 0 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f986a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f986a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f986a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f9872ffd640 10 monclient(hunting): get_auth_request con 0x7f987409faa0 auth_method 0 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f9872ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f9872ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f9872ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f986a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f986a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f986a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f9872ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f9872ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f9872ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 10 monclient(hunting): handle_auth_done global_id 25251 payload 824 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 1 -- 172.21.15.178:0/813350031 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f98740a08c0 msgr2=0x7f9874131840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.668+0000 7f98737fe640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f98740a08c0 0x7f9874131840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f98737fe640 1 -- 172.21.15.178:0/813350031 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f987409faa0 msgr2=0x7f987409b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f98737fe640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f987409faa0 0x7f987409b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f98737fe640 10 monclient: _finish_hunting 0 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f98737fe640 1 monclient: found mon.noname-d 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f98737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f98737fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:17.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f98737fe640 1 -- 172.21.15.178:0/813350031 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c009a20 con 0x7f98740a2930 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f98737fe640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 0x7f987411c530 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f986c009fc0 tx=0x7f986c00b040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f986c00ce70 con 0x7f98740a2930 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f986b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f986b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f986b7fe640 10 monclient: dump: 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.692 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f986b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f986b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c00f040 con 0x7f98740a2930 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f987955d640 5 monclient: authenticate success, global_id 25251 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f986b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:17.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f986c014d80 con 0x7f98740a2930 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f987955d640 10 monclient: _renew_subs 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f987955d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f987955d640 1 -- 172.21.15.178:0/813350031 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9864009450 con 0x7f98740a2930 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f986b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f986b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f986b7fe640 10 monclient: dump: 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.694 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f9871ffb640 4 set_mon_vals no callback set 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f987955d640 10 monclient: _renew_subs 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f987955d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.669+0000 7f987955d640 1 -- 172.21.15.178:0/813350031 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987411ec40 con 0x7f98740a2930 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f986c014480 con 0x7f98740a2930 2024-09-15T01:43:17.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986b7fe640 1 --2- 172.21.15.178:0/813350031 >> v2:172.21.15.204:6812/28210 conn(0x7f98540396e0 0x7f985403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f987955d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f987955d640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9874131d80 0x7f987411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f987955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f987955d640 1 -- 172.21.15.178:0/813350031 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9874132150 con 0x7f9874131d80 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f986c04c7d0 con 0x7f98740a2930 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986a7fc640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9874131d80 0x7f987411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f9872ffd640 1 --2- 172.21.15.178:0/813350031 >> v2:172.21.15.204:6812/28210 conn(0x7f98540396e0 0x7f985403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986a7fc640 10 monclient: get_auth_request con 0x7f9874131d80 auth_method 0 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f9872ffd640 10 monclient: get_auth_request con 0x7f98540396e0 auth_method 0 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986a7fc640 10 monclient(hunting): handle_auth_done global_id 44383 payload 824 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986a7fc640 1 -- 172.21.15.178:0/813350031 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f987404c4a0 con 0x7f9874131d80 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986a7fc640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9874131d80 0x7f987411ca70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f985c00f7f0 tx=0x7f985c008040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f9872ffd640 1 --2- 172.21.15.178:0/813350031 >> v2:172.21.15.204:6812/28210 conn(0x7f98540396e0 0x7f985403bb90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9864034000 tx=0x7f9864032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f985c00dda0 con 0x7f9874131d80 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.670+0000 7f986b7fe640 10 monclient: dump: 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.697 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.671+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f985c00c040 con 0x7f9874131d80 2024-09-15T01:43:17.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.671+0000 7f986b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:17.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.671+0000 7f986b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:17.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.671+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9874131d80 msgr2=0x7f987411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.671+0000 7f986b7fe640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9874131d80 0x7f987411ca70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f985c00f7f0 tx=0x7f985c008040 comp rx=0 tx=0).stop 2024-09-15T01:43:17.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.671+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9874131d80 msgr2=0x7f987411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:17.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.671+0000 7f986b7fe640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9874131d80 0x7f987411ca70 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f985c00f7f0 tx=0x7f985c008040 comp rx=0 tx=0).stop 2024-09-15T01:43:17.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.678+0000 7f987955d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:17.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.678+0000 7f987955d640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f987411cfb0 0x7f9874096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.678+0000 7f987955d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.678+0000 7f987955d640 1 -- 172.21.15.178:0/813350031 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f987404bc80 con 0x7f987411cfb0 2024-09-15T01:43:17.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.678+0000 7f986a7fc640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f987411cfb0 0x7f9874096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.678+0000 7f986a7fc640 10 monclient: get_auth_request con 0x7f987411cfb0 auth_method 0 2024-09-15T01:43:17.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.678+0000 7f986a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.678+0000 7f986a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.678+0000 7f986a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986a7fc640 10 monclient(hunting): handle_auth_done global_id 44392 payload 824 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986a7fc640 1 -- 172.21.15.178:0/813350031 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f987405ed70 con 0x7f987411cfb0 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986a7fc640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f987411cfb0 0x7f9874096a30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f985c00f7f0 tx=0x7f985c01d000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f985c023040 con 0x7f987411cfb0 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986b7fe640 10 monclient: dump: 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.702 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f985c00d610 con 0x7f987411cfb0 2024-09-15T01:43:17.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:17.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:17.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f987411cfb0 msgr2=0x7f9874096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986b7fe640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f987411cfb0 0x7f9874096a30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f985c00f7f0 tx=0x7f985c01d000 comp rx=0 tx=0).stop 2024-09-15T01:43:17.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986b7fe640 1 -- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f987411cfb0 msgr2=0x7f9874096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:17.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.679+0000 7f986b7fe640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f987411cfb0 0x7f9874096a30 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f985c00f7f0 tx=0x7f985c01d000 comp rx=0 tx=0).stop 2024-09-15T01:43:17.704 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:17.704 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:17.704 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:17.704 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.705 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:17.706 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.707 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:17.708 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:17.708 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:17.708 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:17.708 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:17.708 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:17.708 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:17.708 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:17.708 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:17.708 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:17.708 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:17.708 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:17.708 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:17.709 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.710 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.711 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.712 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:17.713 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.714 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.715 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.716 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.717 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:17.718 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.719 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.720 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:17.721 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.722 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.723 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.724 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:17.724 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:17.724 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.724 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.724 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.724 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.724 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:17.724 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.724 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.724 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.724 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.725 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:17.725 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.725 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.725 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.725 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.725 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:17.725 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:17.725 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.725 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.725 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.725 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.726 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.726 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.726 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:17.726 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:17.726 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.726 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:17.726 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:17.726 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:17.726 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:17.727 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.728 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.728 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.728 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:17.728 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:17.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.680+0000 7f987955d640 1 -- 172.21.15.178:0/813350031 >> v2:172.21.15.204:6812/28210 conn(0x7f98540396e0 msgr2=0x7f985403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.680+0000 7f987955d640 1 --2- 172.21.15.178:0/813350031 >> v2:172.21.15.204:6812/28210 conn(0x7f98540396e0 0x7f985403bb90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9864034000 tx=0x7f9864032000 comp rx=0 tx=0).stop 2024-09-15T01:43:17.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.680+0000 7f987955d640 10 monclient: shutdown 2024-09-15T01:43:17.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.680+0000 7f987955d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:17.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.680+0000 7f987955d640 1 -- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 msgr2=0x7f987411c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.680+0000 7f987955d640 1 --2- 172.21.15.178:0/813350031 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f98740a2930 0x7f987411c530 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f986c009fc0 tx=0x7f986c00b040 comp rx=0 tx=0).stop 2024-09-15T01:43:17.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.680+0000 7f98737fe640 1 -- 172.21.15.178:0/813350031 reap_dead start 2024-09-15T01:43:17.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.680+0000 7f98737fe640 1 -- 172.21.15.178:0/813350031 reap_dead start 2024-09-15T01:43:17.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.680+0000 7f987955d640 1 -- 172.21.15.178:0/813350031 shutdown_connections 2024-09-15T01:43:17.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.680+0000 7f987955d640 1 -- 172.21.15.178:0/813350031 >> 172.21.15.178:0/813350031 conn(0x7f98740a5ed0 msgr2=0x7f98740a5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:17.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.680+0000 7f987955d640 1 -- 172.21.15.178:0/813350031 shutdown_connections 2024-09-15T01:43:17.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.680+0000 7f987955d640 1 -- 172.21.15.178:0/813350031 wait complete. 2024-09-15T01:43:17.730 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:17.731 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:43:17.731 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:43:17.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.823+0000 7f50c639d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:17.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.823+0000 7f50c639d640 10 monclient: build_initial_monmap 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.823+0000 7f50c639d640 10 monclient: monmap: 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:17.824484+0000 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:17.824484+0000 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:17.826 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:17.827 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:17.827 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:17.827 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:17.827 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:17.827 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:17.827 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.824+0000 7f50c639d640 1 Processor -- start 2024-09-15T01:43:17.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.824+0000 7f50c639d640 1 -- start start 2024-09-15T01:43:17.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.824+0000 7f50c639d640 10 monclient: init 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.824+0000 7f50c639d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.824+0000 7f50c639d640 10 monclient: _add_conns ranks=[8,2,1,4,6,3,7,10,9,0,5] 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.824+0000 7f50c639d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c00a2930 0x7f50c0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.824+0000 7f50c639d640 10 monclient(hunting): picked mon.noname-e con 0x7f50c00a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50c639d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50c009faa0 0x7f50c009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50c639d640 10 monclient(hunting): picked mon.noname-b con 0x7f50c009faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50c639d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f50c00a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50c639d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c004bf60 con 0x7f50c00a08c0 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50c639d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50c639d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c009e0f0 con 0x7f50c009faa0 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50c639d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50c639d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f50c009dd80 con 0x7f50c00a2930 2024-09-15T01:43:17.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50c639d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50c639d640 10 monclient(hunting): authenticate will time out at 1402.575130s 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c00a2930 0x7f50c0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bf7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50c009faa0 0x7f50c009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c00a2930 0x7f50c0132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:55186/0 (socket says 172.21.15.178:55186) 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 1 -- 172.21.15.178:0/4170734336 learned_addr learned my addr 172.21.15.178:0/4170734336 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 10 monclient(hunting): get_auth_request con 0x7f50c00a2930 auth_method 0 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bf7fe640 10 monclient(hunting): get_auth_request con 0x7f50c009faa0 auth_method 0 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 10 monclient(hunting): handle_auth_done global_id 34739 payload 274 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 1 -- 172.21.15.178:0/4170734336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50c009faa0 msgr2=0x7f50c009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 1 --2- 172.21.15.178:0/4170734336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50c009faa0 0x7f50c009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 1 -- 172.21.15.178:0/4170734336 >> v1:172.21.15.204:6789/0 conn(0x7f50c00a08c0 legacy=0x7f50c00a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 10 monclient: _finish_hunting 0 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 1 monclient: found mon.noname-e 2024-09-15T01:43:17.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bffff640 1 -- 172.21.15.178:0/4170734336 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00a2d00 con 0x7f50c00a2930 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.825+0000 7f50bf7fe640 1 --2- 172.21.15.178:0/4170734336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50c009faa0 0x7f50c009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50bffff640 1 --2- 172.21.15.178:0/4170734336 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c00a2930 0x7f50c0132de0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009940 tx=0x7f50b00091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=111b4f20eb6315bd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50beffd640 1 -- 172.21.15.178:0/4170734336 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50b002ad00 con 0x7f50c00a2930 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50beffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50beffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50beffd640 10 monclient: dump: 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.840 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50beffd640 10 monclient: _finish_auth 0 2024-09-15T01:43:17.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50beffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:17.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50c639d640 5 monclient: authenticate success, global_id 34739 2024-09-15T01:43:17.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50beffd640 1 -- 172.21.15.178:0/4170734336 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b002ae80 con 0x7f50c00a2930 2024-09-15T01:43:17.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50c639d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:17.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50beffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:17.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50c639d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:17.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50c639d640 4 set_mon_vals no callback set 2024-09-15T01:43:17.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50beffd640 1 -- 172.21.15.178:0/4170734336 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50b0031ca0 con 0x7f50c00a2930 2024-09-15T01:43:17.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50beffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50beffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:17.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50beffd640 10 monclient: dump: 2024-09-15T01:43:17.842 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.842 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.843 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50c639d640 10 monclient: shutdown 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50c639d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50c639d640 1 -- 172.21.15.178:0/4170734336 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c00a2930 msgr2=0x7f50c0132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50c639d640 1 --2- 172.21.15.178:0/4170734336 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c00a2930 0x7f50c0132de0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009940 tx=0x7f50b00091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50c639d640 1 -- 172.21.15.178:0/4170734336 shutdown_connections 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50c639d640 1 --2- 172.21.15.178:0/4170734336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50c009faa0 0x7f50c009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50c639d640 1 --2- 172.21.15.178:0/4170734336 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c00a2930 0x7f50c0132de0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.826+0000 7f50c639d640 1 -- 172.21.15.178:0/4170734336 >> 172.21.15.178:0/4170734336 conn(0x7f50c00a5ed0 msgr2=0x7f50c00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.827+0000 7f50c639d640 1 -- 172.21.15.178:0/4170734336 shutdown_connections 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.827+0000 7f50c639d640 1 -- 172.21.15.178:0/4170734336 wait complete. 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.827+0000 7f50c639d640 10 monclient: build_initial_monmap 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.827+0000 7f50c639d640 10 monclient: monmap: 2024-09-15T01:43:17.844 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:17.845 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:17.845 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:17.845 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:17.845 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:17.845 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:17.845 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:17.845 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:17.845 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:17.845 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:17.845 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:17.845 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:17.845 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.827+0000 7f50c639d640 1 Processor -- start 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.827+0000 7f50c639d640 1 -- start start 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.827+0000 7f50c639d640 10 monclient: init 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 10 monclient: _add_conns ranks=[8,1,7,3,4,2,6,0,5] 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c009faa0 0x7f50c009b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 10 monclient(hunting): picked mon.noname-e con 0x7f50c009faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50c00a08c0 0x7f50c0131a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 10 monclient(hunting): picked mon.noname-f con 0x7f50c00a08c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:17.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f50c00a2930 0x7f50c011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 10 monclient(hunting): picked mon.noname-i con 0x7f50c00a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c011ef60 con 0x7f50c00a08c0 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0142d30 con 0x7f50c00a2930 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f50c00a2390 con 0x7f50c009faa0 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c639d640 10 monclient(hunting): authenticate will time out at 1402.578394s 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c009faa0 0x7f50c009b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c009faa0 0x7f50c009b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:55200/0 (socket says 172.21.15.178:55200) 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 1 -- 172.21.15.178:0/3122292438 learned_addr learned my addr 172.21.15.178:0/3122292438 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:17.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 10 monclient(hunting): get_auth_request con 0x7f50c009faa0 auth_method 0 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bf7fe640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50c00a08c0 0x7f50c0131a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c493a640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f50c00a2930 0x7f50c011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bf7fe640 10 monclient(hunting): get_auth_request con 0x7f50c00a08c0 auth_method 0 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c493a640 10 monclient(hunting): get_auth_request con 0x7f50c00a2930 auth_method 0 2024-09-15T01:43:17.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c493a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c493a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c493a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 10 monclient(hunting): handle_auth_done global_id 34748 payload 824 2024-09-15T01:43:17.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 1 -- 172.21.15.178:0/3122292438 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f50c00a2930 msgr2=0x7f50c011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f50c00a2930 0x7f50c011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 1 -- 172.21.15.178:0/3122292438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50c00a08c0 msgr2=0x7f50c0131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50c00a08c0 0x7f50c0131a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 10 monclient: _finish_hunting 0 2024-09-15T01:43:17.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 1 monclient: found mon.noname-e 2024-09-15T01:43:17.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:17.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 1 -- 172.21.15.178:0/3122292438 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b4009390 con 0x7f50c009faa0 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50c493a640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f50c00a2930 0x7f50c011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bf7fe640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50c00a08c0 0x7f50c0131a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.828+0000 7f50bffff640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c009faa0 0x7f50c009b730 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f50b0035040 tx=0x7f50b0032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50b003cb60 con 0x7f50c009faa0 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50a7fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50a7fff640 10 monclient: dump: 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.850 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50a7fff640 10 monclient: _finish_auth 0 2024-09-15T01:43:17.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b003cd00 con 0x7f50c009faa0 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50c639d640 5 monclient: authenticate success, global_id 34748 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50a7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50c639d640 10 monclient: _renew_subs 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50b003ce70 con 0x7f50c009faa0 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50c639d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50c639d640 1 -- 172.21.15.178:0/3122292438 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b0009450 con 0x7f50c009faa0 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50a7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50a7fff640 10 monclient: dump: 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.852 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.853 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.853 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.853 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.853 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.853 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.853 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.853 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.853 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.853 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.853 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.853 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.853 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50be7fc640 4 set_mon_vals no callback set 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50c639d640 10 monclient: _renew_subs 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50c639d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.829+0000 7f50c639d640 1 -- 172.21.15.178:0/3122292438 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c009d630 con 0x7f50c009faa0 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50c639d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50c639d640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0142270 0x7f50c011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50c639d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50c639d640 1 -- 172.21.15.178:0/3122292438 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f50c005ed70 con 0x7f50c0142270 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f50b003c6e0 con 0x7f50c009faa0 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50bf7fe640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0142270 0x7f50c011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50bf7fe640 10 monclient: get_auth_request con 0x7f50c0142270 auth_method 0 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50a7fff640 1 --2- 172.21.15.178:0/3122292438 >> v2:172.21.15.204:6812/28210 conn(0x7f50ac0397d0 0x7f50ac03bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f50b006bfa0 con 0x7f50c009faa0 2024-09-15T01:43:17.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50bf7fe640 10 monclient(hunting): handle_auth_done global_id 25260 payload 824 2024-09-15T01:43:17.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50bf7fe640 1 -- 172.21.15.178:0/3122292438 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50c004bc80 con 0x7f50c0142270 2024-09-15T01:43:17.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50c493a640 1 --2- 172.21.15.178:0/3122292438 >> v2:172.21.15.204:6812/28210 conn(0x7f50ac0397d0 0x7f50ac03bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50c493a640 10 monclient: get_auth_request con 0x7f50ac0397d0 auth_method 0 2024-09-15T01:43:17.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50bf7fe640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0142270 0x7f50c011ca70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f50c00a6ec0 tx=0x7f50b4008040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50b400fdc0 con 0x7f50c0142270 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50a7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.830+0000 7f50a7fff640 10 monclient: dump: 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.856 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.857 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.857 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.857 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.857 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.857 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.857 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.831+0000 7f50c493a640 1 --2- 172.21.15.178:0/3122292438 >> v2:172.21.15.204:6812/28210 conn(0x7f50ac0397d0 0x7f50ac03bc80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f50c009bde0 tx=0x7f50b8009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.831+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f50b400f940 con 0x7f50c0142270 2024-09-15T01:43:17.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.831+0000 7f50a7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:17.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.831+0000 7f50a7fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:17.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.831+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0142270 msgr2=0x7f50c011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.831+0000 7f50a7fff640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0142270 0x7f50c011ca70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f50c00a6ec0 tx=0x7f50b4008040 comp rx=0 tx=0).stop 2024-09-15T01:43:17.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.831+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0142270 msgr2=0x7f50c011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:17.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.831+0000 7f50a7fff640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0142270 0x7f50c011ca70 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f50c00a6ec0 tx=0x7f50b4008040 comp rx=0 tx=0).stop 2024-09-15T01:43:17.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.836+0000 7f50c639d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:17.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.836+0000 7f50c639d640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0131fa0 0x7f50c011cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.836+0000 7f50c639d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.836+0000 7f50c639d640 1 -- 172.21.15.178:0/3122292438 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f50c004b720 con 0x7f50c0131fa0 2024-09-15T01:43:17.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.836+0000 7f50bf7fe640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0131fa0 0x7f50c011cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.836+0000 7f50bf7fe640 10 monclient: get_auth_request con 0x7f50c0131fa0 auth_method 0 2024-09-15T01:43:17.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.836+0000 7f50bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.836+0000 7f50bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.836+0000 7f50bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.836+0000 7f50bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.836+0000 7f50bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.836+0000 7f50bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50bf7fe640 10 monclient(hunting): handle_auth_done global_id 25269 payload 824 2024-09-15T01:43:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50bf7fe640 1 -- 172.21.15.178:0/3122292438 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f50c009cef0 con 0x7f50c0131fa0 2024-09-15T01:43:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50bf7fe640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0131fa0 0x7f50c011cfb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f50b401c000 tx=0x7f50b401a000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50b400fc50 con 0x7f50c0131fa0 2024-09-15T01:43:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50a7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50a7fff640 10 monclient: dump: 2024-09-15T01:43:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.861 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f50b400f6d0 con 0x7f50c0131fa0 2024-09-15T01:43:17.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50a7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:17.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50a7fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:17.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0131fa0 msgr2=0x7f50c011cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50a7fff640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0131fa0 0x7f50c011cfb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f50b401c000 tx=0x7f50b401a000 comp rx=0 tx=0).stop 2024-09-15T01:43:17.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50a7fff640 1 -- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0131fa0 msgr2=0x7f50c011cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:17.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.837+0000 7f50a7fff640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f50c0131fa0 0x7f50c011cfb0 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f50b401c000 tx=0x7f50b401a000 comp rx=0 tx=0).stop 2024-09-15T01:43:17.863 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:17.863 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:17.863 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:17.863 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.864 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 5, 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:17.865 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:17.867 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:17.868 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:17.869 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.870 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.870 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:17.870 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.870 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.870 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.870 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:17.870 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:17.870 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.870 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.871 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.871 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:17.871 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:17.871 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.871 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.871 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.871 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:17.871 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.873 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:17.874 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.875 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.875 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.875 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:17.875 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.875 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.875 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:17.875 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.875 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.875 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.876 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.877 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:17.877 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.877 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.877 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:17.877 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.877 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.877 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.877 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:17.877 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:17.880 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.881 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.882 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.882 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:17.882 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.882 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:17.883 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:17.884 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:17.884 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:17.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.884 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:17.884 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:17.884 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.884 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:17.884 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:17.884 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:17.884 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.885 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:17.886 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:17.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.838+0000 7f50c639d640 1 -- 172.21.15.178:0/3122292438 >> v2:172.21.15.204:6812/28210 conn(0x7f50ac0397d0 msgr2=0x7f50ac03bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.838+0000 7f50c639d640 1 --2- 172.21.15.178:0/3122292438 >> v2:172.21.15.204:6812/28210 conn(0x7f50ac0397d0 0x7f50ac03bc80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f50c009bde0 tx=0x7f50b8009040 comp rx=0 tx=0).stop 2024-09-15T01:43:17.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.838+0000 7f50c639d640 10 monclient: shutdown 2024-09-15T01:43:17.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.838+0000 7f50c639d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:17.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.838+0000 7f50c639d640 1 -- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c009faa0 msgr2=0x7f50c009b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.838+0000 7f50c639d640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c009faa0 0x7f50c009b730 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f50b0035040 tx=0x7f50b0032000 comp rx=0 tx=0).stop 2024-09-15T01:43:17.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.838+0000 7f50c493a640 1 -- 172.21.15.178:0/3122292438 reap_dead start 2024-09-15T01:43:17.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.838+0000 7f50c639d640 1 -- 172.21.15.178:0/3122292438 shutdown_connections 2024-09-15T01:43:17.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.838+0000 7f50c639d640 1 --2- 172.21.15.178:0/3122292438 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f50c009faa0 0x7f50c009b730 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.838+0000 7f50c639d640 1 -- 172.21.15.178:0/3122292438 >> 172.21.15.178:0/3122292438 conn(0x7f50c00a5ed0 msgr2=0x7f50c009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:17.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.838+0000 7f50c639d640 1 -- 172.21.15.178:0/3122292438 shutdown_connections 2024-09-15T01:43:17.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.838+0000 7f50c639d640 1 -- 172.21.15.178:0/3122292438 wait complete. 2024-09-15T01:43:17.889 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:17.889 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:43:17.889 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:43:17.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.981+0000 7ff74079d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:17.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.981+0000 7ff74079d640 10 monclient: build_initial_monmap 2024-09-15T01:43:17.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.981+0000 7ff74079d640 10 monclient: monmap: 2024-09-15T01:43:17.994 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:17.994 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.994 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:17.982194+0000 2024-09-15T01:43:17.994 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:17.982194+0000 2024-09-15T01:43:17.994 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:17.994 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.981+0000 7ff74079d640 1 Processor -- start 2024-09-15T01:43:17.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.981+0000 7ff74079d640 1 -- start start 2024-09-15T01:43:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.981+0000 7ff74079d640 10 monclient: init 2024-09-15T01:43:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 10 monclient: _add_conns ranks=[4,1,3,0,8,9,10,6,7,5,2] 2024-09-15T01:43:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff738098c40 0x7ff73809ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 10 monclient(hunting): picked mon.noname-c con 0x7ff738098c40 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff73809dfd0 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff738135f10 0x7ff7381382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 10 monclient(hunting): picked mon.noname-g con 0x7ff738135f10 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73804bf60 con 0x7ff73809dfd0 2024-09-15T01:43:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff738099e80 con 0x7ff738135f10 2024-09-15T01:43:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff738099b30 con 0x7ff738098c40 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff74079d640 10 monclient(hunting): authenticate will time out at 1402.732675s 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff738098c40 0x7ff73809ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff738098c40 0x7ff73809ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:32854/0 (socket says 172.21.15.178:32854) 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 1 -- 172.21.15.178:0/2467536078 learned_addr learned my addr 172.21.15.178:0/2467536078 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 10 monclient(hunting): get_auth_request con 0x7ff738098c40 auth_method 0 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73ed3a640 1 --2- 172.21.15.178:0/2467536078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff738135f10 0x7ff7381382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73ed3a640 10 monclient(hunting): get_auth_request con 0x7ff738135f10 auth_method 0 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73ed3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73ed3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73ed3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73ed3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73ed3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73ed3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 10 monclient(hunting): handle_auth_done global_id 44401 payload 274 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 1 -- 172.21.15.178:0/2467536078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff738135f10 msgr2=0x7ff7381382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 1 --2- 172.21.15.178:0/2467536078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff738135f10 0x7ff7381382f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.982+0000 7ff73e539640 1 -- 172.21.15.178:0/2467536078 >> v1:172.21.15.204:6789/0 conn(0x7ff73809dfd0 legacy=0x7ff73809d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73e539640 10 monclient: _finish_hunting 0 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73e539640 1 monclient: found mon.noname-c 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73e539640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73e539640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73e539640 1 -- 172.21.15.178:0/2467536078 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff738063620 con 0x7ff738098c40 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73e539640 1 --2- 172.21.15.178:0/2467536078 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff738098c40 0x7ff73809ccd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff734009970 tx=0x7ff7340091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8354deb0e47f02c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73d537640 1 -- 172.21.15.178:0/2467536078 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff73402ad00 con 0x7ff738098c40 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73d537640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73d537640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73d537640 10 monclient: dump: 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73d537640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73d537640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73d537640 1 -- 172.21.15.178:0/2467536078 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73402ae80 con 0x7ff738098c40 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff74079d640 5 monclient: authenticate success, global_id 44401 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff74079d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73d537640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff74079d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73d537640 1 -- 172.21.15.178:0/2467536078 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff734031ca0 con 0x7ff738098c40 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73d537640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff74079d640 4 set_mon_vals no callback set 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73d537640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.983+0000 7ff73d537640 10 monclient: dump: 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.002 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.984+0000 7ff74079d640 10 monclient: shutdown 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.984+0000 7ff74079d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.984+0000 7ff74079d640 1 -- 172.21.15.178:0/2467536078 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff738098c40 msgr2=0x7ff73809ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.984+0000 7ff74079d640 1 --2- 172.21.15.178:0/2467536078 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff738098c40 0x7ff73809ccd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff734009970 tx=0x7ff7340091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.984+0000 7ff74079d640 1 -- 172.21.15.178:0/2467536078 shutdown_connections 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.984+0000 7ff74079d640 1 --2- 172.21.15.178:0/2467536078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff738135f10 0x7ff7381382f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.984+0000 7ff74079d640 1 --2- 172.21.15.178:0/2467536078 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff738098c40 0x7ff73809ccd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.984+0000 7ff74079d640 1 -- 172.21.15.178:0/2467536078 >> 172.21.15.178:0/2467536078 conn(0x7ff7380a22e0 msgr2=0x7ff7380a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.984+0000 7ff74079d640 1 -- 172.21.15.178:0/2467536078 shutdown_connections 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.984+0000 7ff74079d640 1 -- 172.21.15.178:0/2467536078 wait complete. 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.984+0000 7ff74079d640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.984+0000 7ff74079d640 10 monclient: monmap: 2024-09-15T01:43:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 1 Processor -- start 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 1 -- start start 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 10 monclient: init 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 10 monclient: _add_conns ranks=[6,4,0,8,1,2,5,7,3] 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff738098c40 0x7ff7381358d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 10 monclient(hunting): picked mon.noname-d con 0x7ff738098c40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff73809dfd0 0x7ff73811d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 10 monclient(hunting): picked mon.noname-c con 0x7ff73809dfd0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff738135f10 0x7ff73811d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 10 monclient(hunting): picked mon.noname-a con 0x7ff738135f10 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73809bd50 con 0x7ff738135f10 2024-09-15T01:43:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff73809e660 con 0x7ff73809dfd0 2024-09-15T01:43:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff738120860 con 0x7ff738098c40 2024-09-15T01:43:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff74079d640 10 monclient(hunting): authenticate will time out at 1402.735831s 2024-09-15T01:43:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73e539640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff738098c40 0x7ff7381358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73dd38640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff73809dfd0 0x7ff73811d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73e539640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff738098c40 0x7ff7381358d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:44456/0 (socket says 172.21.15.178:44456) 2024-09-15T01:43:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73e539640 1 -- 172.21.15.178:0/1575926584 learned_addr learned my addr 172.21.15.178:0/1575926584 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73e539640 10 monclient(hunting): get_auth_request con 0x7ff738098c40 auth_method 0 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73e539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73e539640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73e539640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73dd38640 10 monclient(hunting): get_auth_request con 0x7ff73809dfd0 auth_method 0 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73dd38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73dd38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73dd38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73ed3a640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff738135f10 0x7ff73811d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73e539640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.985+0000 7ff73e539640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73e539640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73ed3a640 10 monclient(hunting): get_auth_request con 0x7ff738135f10 auth_method 0 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73ed3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73ed3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73ed3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73dd38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73dd38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73dd38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73ed3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73ed3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73ed3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73e539640 10 monclient(hunting): handle_auth_done global_id 25278 payload 824 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73e539640 1 -- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff73809dfd0 msgr2=0x7ff73811d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73e539640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff73809dfd0 0x7ff73811d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73e539640 1 -- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff738135f10 msgr2=0x7ff73811d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73e539640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff738135f10 0x7ff73811d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73e539640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73e539640 1 monclient: found mon.noname-d 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73e539640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73e539640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73e539640 1 -- 172.21.15.178:0/1575926584 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff734009450 con 0x7ff738098c40 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73e539640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff738098c40 0x7ff7381358d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff7340097e0 tx=0x7ff734032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff73403ca90 con 0x7ff738098c40 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff72e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff72e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff72e7fc640 10 monclient: dump: 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff72e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff72e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73403cc10 con 0x7ff738098c40 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff74079d640 5 monclient: authenticate success, global_id 25278 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff74079d640 10 monclient: _renew_subs 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff74079d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff74079d640 1 -- 172.21.15.178:0/1575926584 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73811ed30 con 0x7ff738098c40 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff72e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff73403cd80 con 0x7ff738098c40 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff72e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff72e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff72e7fc640 10 monclient: dump: 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.012 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff74079d640 10 monclient: _renew_subs 2024-09-15T01:43:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff74079d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:43:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff74079d640 1 -- 172.21.15.178:0/1575926584 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff738134e10 con 0x7ff738098c40 2024-09-15T01:43:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.986+0000 7ff73cd36640 4 set_mon_vals no callback set 2024-09-15T01:43:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff74079d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff73403c610 con 0x7ff738098c40 2024-09-15T01:43:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff74079d640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811f900 0x7ff73802aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff74079d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff74079d640 1 -- 172.21.15.178:0/1575926584 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff738068530 con 0x7ff73811f900 2024-09-15T01:43:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73dd38640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811f900 0x7ff73802aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73dd38640 10 monclient: get_auth_request con 0x7ff73811f900 auth_method 0 2024-09-15T01:43:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73dd38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73dd38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73dd38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff72e7fc640 1 --2- 172.21.15.178:0/1575926584 >> v2:172.21.15.204:6812/28210 conn(0x7ff71c0397b0 0x7ff71c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73dd38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73dd38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73dd38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7ff73406beb0 con 0x7ff738098c40 2024-09-15T01:43:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73dd38640 10 monclient(hunting): handle_auth_done global_id 34757 payload 824 2024-09-15T01:43:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73dd38640 1 -- 172.21.15.178:0/1575926584 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff73805ed70 con 0x7ff73811f900 2024-09-15T01:43:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73ed3a640 1 --2- 172.21.15.178:0/1575926584 >> v2:172.21.15.204:6812/28210 conn(0x7ff71c0397b0 0x7ff71c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73ed3a640 10 monclient: get_auth_request con 0x7ff71c0397b0 auth_method 0 2024-09-15T01:43:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.987+0000 7ff73dd38640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811f900 0x7ff73802aa40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff7280038f0 tx=0x7ff728007ec0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.988+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff72800e2e0 con 0x7ff73811f900 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.988+0000 7ff72e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.988+0000 7ff72e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.988+0000 7ff72e7fc640 10 monclient: dump: 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.988+0000 7ff73ed3a640 1 --2- 172.21.15.178:0/1575926584 >> v2:172.21.15.204:6812/28210 conn(0x7ff71c0397b0 0x7ff71c03bc60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff730009fd0 tx=0x7ff73000b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.988+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7ff728014040 con 0x7ff73811f900 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.988+0000 7ff72e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.988+0000 7ff72e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.988+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811f900 msgr2=0x7ff73802aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.988+0000 7ff72e7fc640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811f900 0x7ff73802aa40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff7280038f0 tx=0x7ff728007ec0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.988+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811f900 msgr2=0x7ff73802aa40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.988+0000 7ff72e7fc640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811f900 0x7ff73802aa40 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff7280038f0 tx=0x7ff728007ec0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.993+0000 7ff74079d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:18.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.993+0000 7ff74079d640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811dd60 0x7ff73805e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.993+0000 7ff74079d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.993+0000 7ff74079d640 1 -- 172.21.15.178:0/1575926584 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff7381209d0 con 0x7ff73811dd60 2024-09-15T01:43:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff73dd38640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811dd60 0x7ff73805e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff73dd38640 10 monclient: get_auth_request con 0x7ff73811dd60 auth_method 0 2024-09-15T01:43:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff73dd38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff73dd38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff73dd38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff73dd38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff73dd38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff73dd38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff73dd38640 10 monclient(hunting): handle_auth_done global_id 34766 payload 824 2024-09-15T01:43:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff73dd38640 1 -- 172.21.15.178:0/1575926584 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff73804bc80 con 0x7ff73811dd60 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff73dd38640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811dd60 0x7ff73805e310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff7280038f0 tx=0x7ff72801c000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff72800c450 con 0x7ff73811dd60 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff72e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff72e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff72e7fc640 10 monclient: dump: 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7ff728006b70 con 0x7ff73811dd60 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff72e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff72e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811dd60 msgr2=0x7ff73805e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff72e7fc640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811dd60 0x7ff73805e310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff7280038f0 tx=0x7ff72801c000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff72e7fc640 1 -- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811dd60 msgr2=0x7ff73805e310 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.994+0000 7ff72e7fc640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7ff73811dd60 0x7ff73805e310 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff7280038f0 tx=0x7ff72801c000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.022 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:18.022 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:18.022 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:18.022 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:18.022 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:18.022 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:18.022 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 5, 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:18.023 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.024 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.025 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.025 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.025 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.025 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.025 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:18.026 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.027 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.028 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:18.029 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.031 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:18.032 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:18.033 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.035 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.036 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.037 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.038 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.039 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:18.041 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.043 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.044 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.045 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:18.046 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:18.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.995+0000 7ff74079d640 1 -- 172.21.15.178:0/1575926584 >> v2:172.21.15.204:6812/28210 conn(0x7ff71c0397b0 msgr2=0x7ff71c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.995+0000 7ff74079d640 1 --2- 172.21.15.178:0/1575926584 >> v2:172.21.15.204:6812/28210 conn(0x7ff71c0397b0 0x7ff71c03bc60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff730009fd0 tx=0x7ff73000b040 comp rx=0 tx=0).stop 2024-09-15T01:43:18.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.995+0000 7ff74079d640 10 monclient: shutdown 2024-09-15T01:43:18.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.995+0000 7ff74079d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.995+0000 7ff74079d640 1 -- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff738098c40 msgr2=0x7ff7381358d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.995+0000 7ff74079d640 1 --2- 172.21.15.178:0/1575926584 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7ff738098c40 0x7ff7381358d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff7340097e0 tx=0x7ff734032000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.995+0000 7ff73ed3a640 1 -- 172.21.15.178:0/1575926584 reap_dead start 2024-09-15T01:43:18.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.995+0000 7ff73ed3a640 1 -- 172.21.15.178:0/1575926584 reap_dead start 2024-09-15T01:43:18.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.995+0000 7ff74079d640 1 -- 172.21.15.178:0/1575926584 shutdown_connections 2024-09-15T01:43:18.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.995+0000 7ff74079d640 1 -- 172.21.15.178:0/1575926584 >> 172.21.15.178:0/1575926584 conn(0x7ff7380a22e0 msgr2=0x7ff73806dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.995+0000 7ff74079d640 1 -- 172.21.15.178:0/1575926584 shutdown_connections 2024-09-15T01:43:18.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:17.995+0000 7ff74079d640 1 -- 172.21.15.178:0/1575926584 wait complete. 2024-09-15T01:43:18.048 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:18.048 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:43:18.049 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:43:18.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.140+0000 7f9431ec9640 10 monclient: get_monmap_and_config 2024-09-15T01:43:18.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.140+0000 7f9431ec9640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.140+0000 7f9431ec9640 10 monclient: monmap: 2024-09-15T01:43:18.142 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.142 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.142 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:18.140982+0000 2024-09-15T01:43:18.142 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:18.140982+0000 2024-09-15T01:43:18.142 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.142 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.142 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:18.143 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:18.143 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.143 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.143 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.143 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.143 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.143 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.143 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.143 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:18.143 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:18.143 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.140+0000 7f9431ec9640 1 Processor -- start 2024-09-15T01:43:18.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 1 -- start start 2024-09-15T01:43:18.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 10 monclient: init 2024-09-15T01:43:18.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 10 monclient: _add_conns ranks=[10,9,5,2,3,6,0,1,7,4,8] 2024-09-15T01:43:18.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f942c0678d0 0x7f942c09e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 10 monclient(hunting): picked mon.noname-f con 0x7f942c0678d0 addr v2:172.21.15.204:3300/0 2024-09-15T01:43:18.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f942c06ea30 0x7f942c09ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 10 monclient(hunting): picked mon.noname-a con 0x7f942c06ea30 addr v2:172.21.15.178:3300/0 2024-09-15T01:43:18.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f942c09f180 0x7f942c1306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 10 monclient(hunting): picked mon.noname-h con 0x7f942c09f180 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:18.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c09f570 con 0x7f942c06ea30 2024-09-15T01:43:18.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c06d5b0 con 0x7f942c0678d0 2024-09-15T01:43:18.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f942c067ca0 con 0x7f942c09f180 2024-09-15T01:43:18.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9431ec9640 10 monclient(hunting): authenticate will time out at 1402.891541s 2024-09-15T01:43:18.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9423fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f942c06ea30 0x7f942c09ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9423fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f942c06ea30 0x7f942c09ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60826/0 (socket says 172.21.15.178:60826) 2024-09-15T01:43:18.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9423fff640 1 -- 172.21.15.178:0/3456821914 learned_addr learned my addr 172.21.15.178:0/3456821914 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:18.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9423fff640 10 monclient(hunting): get_auth_request con 0x7f942c06ea30 auth_method 0 2024-09-15T01:43:18.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9423fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9423fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9423fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9423fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9423fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9423fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942b7fe640 1 --2- 172.21.15.178:0/3456821914 >> v2:172.21.15.204:3300/0 conn(0x7f942c0678d0 0x7f942c09e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942bfff640 1 --2- 172.21.15.178:0/3456821914 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f942c09f180 0x7f942c1306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942b7fe640 10 monclient(hunting): get_auth_request con 0x7f942c0678d0 auth_method 0 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942bfff640 10 monclient(hunting): get_auth_request con 0x7f942c09f180 auth_method 0 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f942bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.141+0000 7f9423fff640 10 monclient(hunting): handle_auth_done global_id 34920 payload 274 2024-09-15T01:43:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9423fff640 1 -- 172.21.15.178:0/3456821914 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f942c09f180 msgr2=0x7f942c1306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9423fff640 1 --2- 172.21.15.178:0/3456821914 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f942c09f180 0x7f942c1306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9423fff640 1 -- 172.21.15.178:0/3456821914 >> v2:172.21.15.204:3300/0 conn(0x7f942c0678d0 msgr2=0x7f942c09e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9423fff640 1 --2- 172.21.15.178:0/3456821914 >> v2:172.21.15.204:3300/0 conn(0x7f942c0678d0 0x7f942c09e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9423fff640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9423fff640 1 monclient: found mon.noname-a 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9423fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9423fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9423fff640 1 -- 172.21.15.178:0/3456821914 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c04bf60 con 0x7f942c06ea30 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9423fff640 1 --2- 172.21.15.178:0/3456821914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c06ea30 0x7f942c09ec40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f942c068080 tx=0x7f94140091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95c0100a91e989d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f942affd640 1 -- 172.21.15.178:0/3456821914 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f941402ad40 con 0x7f942c06ea30 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f942affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f942affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f942affd640 10 monclient: dump: 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f942affd640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f942affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9431ec9640 5 monclient: authenticate success, global_id 34920 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9431ec9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f942affd640 1 -- 172.21.15.178:0/3456821914 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9414031820 con 0x7f942c06ea30 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f942affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9431ec9640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f9431ec9640 4 set_mon_vals no callback set 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f942affd640 1 -- 172.21.15.178:0/3456821914 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9414031e90 con 0x7f942c06ea30 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f942affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f942affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.142+0000 7f942affd640 10 monclient: dump: 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.162 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.143+0000 7f9431ec9640 10 monclient: shutdown 2024-09-15T01:43:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.143+0000 7f9431ec9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.143+0000 7f9431ec9640 1 -- 172.21.15.178:0/3456821914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c06ea30 msgr2=0x7f942c09ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.143+0000 7f9431ec9640 1 --2- 172.21.15.178:0/3456821914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c06ea30 0x7f942c09ec40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f942c068080 tx=0x7f94140091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.143+0000 7f9431ec9640 1 -- 172.21.15.178:0/3456821914 shutdown_connections 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.143+0000 7f9431ec9640 1 --2- 172.21.15.178:0/3456821914 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f942c09f180 0x7f942c1306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.143+0000 7f9431ec9640 1 --2- 172.21.15.178:0/3456821914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c06ea30 0x7f942c09ec40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.143+0000 7f9431ec9640 1 --2- 172.21.15.178:0/3456821914 >> v2:172.21.15.204:3300/0 conn(0x7f942c0678d0 0x7f942c09e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.143+0000 7f9431ec9640 1 -- 172.21.15.178:0/3456821914 >> 172.21.15.178:0/3456821914 conn(0x7f942c0a4390 msgr2=0x7f942c067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.143+0000 7f9431ec9640 1 -- 172.21.15.178:0/3456821914 shutdown_connections 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.143+0000 7f9431ec9640 1 -- 172.21.15.178:0/3456821914 wait complete. 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient: monmap: 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 1 Processor -- start 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 1 -- start start 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient: init 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient: _add_conns ranks=[1,0,3,5,4,8,6,7,2] 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0678d0 0x7f942c141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient(hunting): picked mon.noname-f con 0x7f942c0678d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c06ea30 0x7f942c1336c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient(hunting): picked mon.noname-a con 0x7f942c06ea30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f942c09f180 0x7f942c133c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient(hunting): picked mon.noname-g con 0x7f942c09f180 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c06c900 con 0x7f942c06ea30 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c122800 con 0x7f942c0678d0 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c122ae0 con 0x7f942c09f180 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.144+0000 7f9431ec9640 10 monclient(hunting): authenticate will time out at 1402.894960s 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c06ea30 0x7f942c1336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c06ea30 0x7f942c1336c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60834/0 (socket says 172.21.15.178:60834) 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 1 -- 172.21.15.178:0/1642270102 learned_addr learned my addr 172.21.15.178:0/1642270102 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 10 monclient(hunting): get_auth_request con 0x7f942c06ea30 auth_method 0 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942b7fe640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0678d0 0x7f942c141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942b7fe640 10 monclient(hunting): get_auth_request con 0x7f942c0678d0 auth_method 0 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942bfff640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f942c09f180 0x7f942c133c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942bfff640 10 monclient(hunting): get_auth_request con 0x7f942c09f180 auth_method 0 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942b7fe640 1 -- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0678d0 msgr2=0x7f942c141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942b7fe640 1 -- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0678d0 msgr2=0x7f942c141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 10 monclient(hunting): handle_auth_done global_id 34929 payload 824 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942b7fe640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0678d0 0x7f942c141a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942b7fe640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0678d0 0x7f942c141a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 1 -- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f942c09f180 msgr2=0x7f942c133c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f942c09f180 0x7f942c133c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 1 -- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0678d0 msgr2=0x7f942c141a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:43:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0678d0 0x7f942c141a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 1 monclient: found mon.noname-a 2024-09-15T01:43:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:43:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 1 -- 172.21.15.178:0/1642270102 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9414009450 con 0x7f942c06ea30 2024-09-15T01:43:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9423fff640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c06ea30 0x7f942c1336c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9414035040 tx=0x7f9414032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9414033cb0 con 0x7f942c06ea30 2024-09-15T01:43:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f94237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f94237fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:43:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f94237fe640 10 monclient: dump: 2024-09-15T01:43:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.171 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.172 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.172 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f94237fe640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f94237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9431ec9640 5 monclient: authenticate success, global_id 34929 2024-09-15T01:43:18.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9414033e90 con 0x7f942c06ea30 2024-09-15T01:43:18.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9431ec9640 10 monclient: _renew_subs 2024-09-15T01:43:18.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9431ec9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:43:18.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9431ec9640 1 -- 172.21.15.178:0/1642270102 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c141fd0 con 0x7f942c06ea30 2024-09-15T01:43:18.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f94237fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f941403a810 con 0x7f942c06ea30 2024-09-15T01:43:18.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f94237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f94237fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f942a7fc640 4 set_mon_vals no callback set 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f94237fe640 10 monclient: dump: 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.173 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9431ec9640 10 monclient: _renew_subs 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9431ec9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.145+0000 7f9431ec9640 1 -- 172.21.15.178:0/1642270102 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c121b50 con 0x7f942c06ea30 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f9431ec9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f9431ec9640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0682c0 0x7f942c068690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f9431ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f9431ec9640 1 -- 172.21.15.178:0/1642270102 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c122970 con 0x7f942c0682c0 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9414039040 con 0x7f942c06ea30 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f942b7fe640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0682c0 0x7f942c068690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f942b7fe640 10 monclient: get_auth_request con 0x7f942c0682c0 auth_method 0 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f942b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f942b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f942b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f94237fe640 1 --2- 172.21.15.178:0/1642270102 >> v2:172.21.15.204:6812/28210 conn(0x7f940c0397e0 0x7f940c03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f942b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f942b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f942b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f941406b6d0 con 0x7f942c06ea30 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f942bfff640 1 --2- 172.21.15.178:0/1642270102 >> v2:172.21.15.204:6812/28210 conn(0x7f940c0397e0 0x7f940c03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f942b7fe640 10 monclient(hunting): handle_auth_done global_id 54244 payload 824 2024-09-15T01:43:18.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f942b7fe640 1 -- 172.21.15.178:0/1642270102 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f942c04c4a0 con 0x7f942c0682c0 2024-09-15T01:43:18.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.146+0000 7f942bfff640 10 monclient: get_auth_request con 0x7f940c0397e0 auth_method 0 2024-09-15T01:43:18.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f942b7fe640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0682c0 0x7f942c068690 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f941c011bf0 tx=0x7f941c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f941c00fd50 con 0x7f942c0682c0 2024-09-15T01:43:18.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f94237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f94237fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:43:18.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f94237fe640 10 monclient: dump: 2024-09-15T01:43:18.176 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.176 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.176 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.176 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.176 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f942bfff640 1 --2- 172.21.15.178:0/1642270102 >> v2:172.21.15.204:6812/28210 conn(0x7f940c0397e0 0x7f940c03bc90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9424009fd0 tx=0x7f942400b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f941c00feb0 con 0x7f942c0682c0 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f94237fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f94237fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0682c0 msgr2=0x7f942c068690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f94237fe640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0682c0 0x7f942c068690 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f941c011bf0 tx=0x7f941c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0682c0 msgr2=0x7f942c068690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.147+0000 7f94237fe640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c0682c0 0x7f942c068690 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f9431ec9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f9431ec9640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c068bd0 0x7f942c098a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f9431ec9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f9431ec9640 1 -- 172.21.15.178:0/1642270102 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c04bc80 con 0x7f942c068bd0 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f942b7fe640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c068bd0 0x7f942c098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f942b7fe640 10 monclient: get_auth_request con 0x7f942c068bd0 auth_method 0 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f942b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f942b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f942b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f942b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f942b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f942b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f942b7fe640 10 monclient(hunting): handle_auth_done global_id 54253 payload 824 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f942b7fe640 1 -- 172.21.15.178:0/1642270102 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f942c0697a0 con 0x7f942c068bd0 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.153+0000 7f942b7fe640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c068bd0 0x7f942c098a40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f941c011bf0 tx=0x7f941c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f941c01de10 con 0x7f942c068bd0 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f94237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f94237fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f94237fe640 10 monclient: dump: 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.179 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.180 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f941c01c3e0 con 0x7f942c068bd0 2024-09-15T01:43:18.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f94237fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:18.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f94237fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:18.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c068bd0 msgr2=0x7f942c098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f94237fe640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c068bd0 0x7f942c098a40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f941c011bf0 tx=0x7f941c01c000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f94237fe640 1 -- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c068bd0 msgr2=0x7f942c098a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f94237fe640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f942c068bd0 0x7f942c098a40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.182 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:18.182 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:18.182 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:18.182 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:18.182 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:18.183 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:18.183 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:18.183 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:18.183 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:18.183 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:18.183 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:18.183 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:18.183 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:18.183 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:18.183 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:18.183 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.183 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 5, 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.184 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.185 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:18.185 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:18.185 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.185 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.185 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.185 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.185 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.185 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.186 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.186 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.186 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.186 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.186 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.186 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:18.186 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:18.186 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:18.186 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:18.186 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:18.186 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:18.186 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.187 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.188 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:18.189 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:18.190 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.191 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.192 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:18.193 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.194 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.195 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:18.196 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:18.197 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.198 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:18.199 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:18.200 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.201 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.202 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.203 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.204 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:18.205 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.205 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.205 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.205 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:18.205 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.205 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.205 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.205 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:18.205 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:18.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f9431ec9640 1 -- 172.21.15.178:0/1642270102 >> v2:172.21.15.204:6812/28210 conn(0x7f940c0397e0 msgr2=0x7f940c03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f9431ec9640 1 --2- 172.21.15.178:0/1642270102 >> v2:172.21.15.204:6812/28210 conn(0x7f940c0397e0 0x7f940c03bc90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9424009fd0 tx=0x7f942400b040 comp rx=0 tx=0).stop 2024-09-15T01:43:18.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f9431ec9640 10 monclient: shutdown 2024-09-15T01:43:18.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f9431ec9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f9431ec9640 1 -- 172.21.15.178:0/1642270102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c06ea30 msgr2=0x7f942c1336c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f9431ec9640 1 --2- 172.21.15.178:0/1642270102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942c06ea30 0x7f942c1336c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9414035040 tx=0x7f9414032000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f942bfff640 1 -- 172.21.15.178:0/1642270102 reap_dead start 2024-09-15T01:43:18.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f942bfff640 1 -- 172.21.15.178:0/1642270102 reap_dead start 2024-09-15T01:43:18.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f9431ec9640 1 -- 172.21.15.178:0/1642270102 shutdown_connections 2024-09-15T01:43:18.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f9431ec9640 1 -- 172.21.15.178:0/1642270102 >> 172.21.15.178:0/1642270102 conn(0x7f942c0a4390 msgr2=0x7f942c09b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f9431ec9640 1 -- 172.21.15.178:0/1642270102 shutdown_connections 2024-09-15T01:43:18.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.154+0000 7f9431ec9640 1 -- 172.21.15.178:0/1642270102 wait complete. 2024-09-15T01:43:18.208 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:18.208 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:43:18.208 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:43:18.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.300+0000 7f1c7ac0f640 10 monclient: get_monmap_and_config 2024-09-15T01:43:18.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.300+0000 7f1c7ac0f640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.300+0000 7f1c7ac0f640 10 monclient: monmap: 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:18.301670+0000 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:18.301670+0000 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.304 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.304 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.304 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:18.304 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:18.304 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.301+0000 7f1c7ac0f640 1 Processor -- start 2024-09-15T01:43:18.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.301+0000 7f1c7ac0f640 1 -- start start 2024-09-15T01:43:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.301+0000 7f1c7ac0f640 10 monclient: init 2024-09-15T01:43:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 10 monclient: _add_conns ranks=[10,2,9,6,5,7,8,1,0,4,3] 2024-09-15T01:43:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1c740a2930 0x7f1c74132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 10 monclient(hunting): picked mon.noname-f con 0x7f1c740a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:43:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7409faa0 0x7f1c7409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 10 monclient(hunting): picked mon.noname-b con 0x7f1c7409faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1c740a08c0 0x7f1c740a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 10 monclient(hunting): picked mon.noname-a con 0x7f1c740a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:43:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7409e0f0 con 0x7f1c740a08c0 2024-09-15T01:43:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7409dd80 con 0x7f1c740a2930 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740a2d00 con 0x7f1c7409faa0 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c7ac0f640 10 monclient(hunting): authenticate will time out at 1403.052212s 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1c740a08c0 0x7f1c740a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c73fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7409faa0 0x7f1c7409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1c740a08c0 0x7f1c740a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60840/0 (socket says 172.21.15.178:60840) 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 1 -- 172.21.15.178:0/3077231265 learned_addr learned my addr 172.21.15.178:0/3077231265 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 10 monclient(hunting): get_auth_request con 0x7f1c740a08c0 auth_method 0 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c73fff640 10 monclient(hunting): get_auth_request con 0x7f1c7409faa0 auth_method 0 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c73fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c73fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c789ab640 1 --2- 172.21.15.178:0/3077231265 >> v2:172.21.15.204:3300/0 conn(0x7f1c740a2930 0x7f1c74132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c789ab640 10 monclient(hunting): get_auth_request con 0x7f1c740a2930 auth_method 0 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c789ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c789ab640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c789ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 10 monclient(hunting): handle_auth_done global_id 34938 payload 274 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 1 -- 172.21.15.178:0/3077231265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7409faa0 msgr2=0x7f1c7409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 1 --2- 172.21.15.178:0/3077231265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7409faa0 0x7f1c7409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 1 -- 172.21.15.178:0/3077231265 >> v2:172.21.15.204:3300/0 conn(0x7f1c740a2930 msgr2=0x7f1c74132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 1 --2- 172.21.15.178:0/3077231265 >> v2:172.21.15.204:3300/0 conn(0x7f1c740a2930 0x7f1c74132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 1 monclient: found mon.noname-a 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c791ac640 1 -- 172.21.15.178:0/3077231265 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7404bf60 con 0x7f1c740a08c0 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c789ab640 1 --2- 172.21.15.178:0/3077231265 >> v2:172.21.15.204:3300/0 conn(0x7f1c740a2930 0x7f1c74132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:43:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.302+0000 7f1c73fff640 1 --2- 172.21.15.178:0/3077231265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7409faa0 0x7f1c7409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c791ac640 1 --2- 172.21.15.178:0/3077231265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c740a08c0 0x7f1c740a0d60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c740a2f70 tx=0x7f1c6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326919fadaadaa90 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c737fe640 1 -- 172.21.15.178:0/3077231265 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c6c007880 con 0x7f1c740a08c0 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c737fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c737fe640 10 monclient: dump: 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c737fe640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c737fe640 1 -- 172.21.15.178:0/3077231265 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c02d050 con 0x7f1c740a08c0 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c7ac0f640 5 monclient: authenticate success, global_id 34938 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c7ac0f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c737fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c737fe640 1 -- 172.21.15.178:0/3077231265 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c6c030360 con 0x7f1c740a08c0 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c7ac0f640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c7ac0f640 4 set_mon_vals no callback set 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c737fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.303+0000 7f1c737fe640 10 monclient: dump: 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.320 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 10 monclient: shutdown 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/3077231265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c740a08c0 msgr2=0x7f1c740a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 1 --2- 172.21.15.178:0/3077231265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c740a08c0 0x7f1c740a0d60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c740a2f70 tx=0x7f1c6c009840 comp rx=0 tx=0).stop 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/3077231265 shutdown_connections 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 1 --2- 172.21.15.178:0/3077231265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c740a08c0 0x7f1c740a0d60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 1 --2- 172.21.15.178:0/3077231265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7409faa0 0x7f1c7409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 1 --2- 172.21.15.178:0/3077231265 >> v2:172.21.15.204:3300/0 conn(0x7f1c740a2930 0x7f1c74132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/3077231265 >> 172.21.15.178:0/3077231265 conn(0x7f1c740a5ed0 msgr2=0x7f1c740a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/3077231265 shutdown_connections 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/3077231265 wait complete. 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 10 monclient: monmap: 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:18.322 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.304+0000 7f1c7ac0f640 1 Processor -- start 2024-09-15T01:43:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 1 -- start start 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 10 monclient: init 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 10 monclient: _add_conns ranks=[1,8,4,6,3,7,5,0,2] 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c7409faa0 0x7f1c7409af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 10 monclient(hunting): picked mon.noname-f con 0x7f1c7409faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1c740a08c0 0x7f1c741316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 10 monclient(hunting): picked mon.noname-e con 0x7f1c740a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1c740a2930 0x7f1c7411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 10 monclient(hunting): picked mon.noname-c con 0x7f1c740a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7411f130 con 0x7f1c7409faa0 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740a2390 con 0x7f1c740a2930 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7409d630 con 0x7f1c740a08c0 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c7ac0f640 10 monclient(hunting): authenticate will time out at 1403.055685s 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c73fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1c740a08c0 0x7f1c741316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c791ac640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1c740a2930 0x7f1c7411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c791ac640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1c740a2930 0x7f1c7411c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:60238/0 (socket says 172.21.15.178:60238) 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c73fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1c740a08c0 0x7f1c741316e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42114/0 (socket says 172.21.15.178:42114) 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c791ac640 1 -- 172.21.15.178:0/2988633254 learned_addr learned my addr 172.21.15.178:0/2988633254 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c791ac640 10 monclient(hunting): get_auth_request con 0x7f1c740a2930 auth_method 0 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c791ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c791ac640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c791ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c73fff640 10 monclient(hunting): get_auth_request con 0x7f1c740a08c0 auth_method 0 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c73fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c73fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c791ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c791ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c789ab640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c7409faa0 0x7f1c7409af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c791ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c789ab640 10 monclient(hunting): get_auth_request con 0x7f1c7409faa0 auth_method 0 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c789ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c789ab640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.305+0000 7f1c789ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c789ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c789ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c789ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c791ac640 10 monclient(hunting): handle_auth_done global_id 44419 payload 824 2024-09-15T01:43:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c791ac640 1 -- 172.21.15.178:0/2988633254 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1c740a08c0 msgr2=0x7f1c741316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c791ac640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1c740a08c0 0x7f1c741316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c791ac640 1 -- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c7409faa0 msgr2=0x7f1c7409af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c791ac640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c7409faa0 0x7f1c7409af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c791ac640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c791ac640 1 monclient: found mon.noname-c 2024-09-15T01:43:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c791ac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c791ac640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:43:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c791ac640 1 -- 172.21.15.178:0/2988633254 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68009390 con 0x7f1c740a2930 2024-09-15T01:43:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c73fff640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f1c740a08c0 0x7f1c741316e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c791ac640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1c740a2930 0x7f1c7411c530 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009f50 tx=0x7f1c6c036000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c6c009520 con 0x7f1c740a2930 2024-09-15T01:43:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c70ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c70ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:43:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c70ff9640 10 monclient: dump: 2024-09-15T01:43:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.329 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c70ff9640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c70ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c7ac0f640 5 monclient: authenticate success, global_id 44419 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c03baa0 con 0x7f1c740a2930 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c70ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c6c03bc50 con 0x7f1c740a2930 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c7ac0f640 10 monclient: _renew_subs 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c7ac0f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/2988633254 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c009ae0 con 0x7f1c740a2930 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c70ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c70ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c72ffd640 4 set_mon_vals no callback set 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c70ff9640 10 monclient: dump: 2024-09-15T01:43:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c7ac0f640 10 monclient: _renew_subs 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c7ac0f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.306+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/2988633254 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7409b9b0 con 0x7f1c740a2930 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1c6c038070 con 0x7f1c740a2930 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c7ac0f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c7ac0f640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c74131c20 0x7f1c7411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c7ac0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/2988633254 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c74131ff0 con 0x7f1c74131c20 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c70ff9640 1 --2- 172.21.15.178:0/2988633254 >> v2:172.21.15.204:6812/28210 conn(0x7f1c600396e0 0x7f1c6003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c789ab640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c74131c20 0x7f1c7411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c789ab640 10 monclient: get_auth_request con 0x7f1c74131c20 auth_method 0 2024-09-15T01:43:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c789ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c789ab640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c789ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f1c6c06c560 con 0x7f1c740a2930 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c789ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c789ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c789ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c73fff640 1 --2- 172.21.15.178:0/2988633254 >> v2:172.21.15.204:6812/28210 conn(0x7f1c600396e0 0x7f1c6003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c73fff640 10 monclient: get_auth_request con 0x7f1c600396e0 auth_method 0 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c789ab640 10 monclient(hunting): handle_auth_done global_id 16338 payload 824 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.307+0000 7f1c789ab640 1 -- 172.21.15.178:0/2988633254 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c7404c4a0 con 0x7f1c74131c20 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c789ab640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c74131c20 0x7f1c7411ca70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1c64010880 tx=0x7f1c64008040 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c73fff640 1 --2- 172.21.15.178:0/2988633254 >> v2:172.21.15.204:6812/28210 conn(0x7f1c600396e0 0x7f1c6003bb90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009fd0 tx=0x7f1c680090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c64014430 con 0x7f1c74131c20 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c70ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c70ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c70ff9640 10 monclient: dump: 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f1c6400eb40 con 0x7f1c74131c20 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c70ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c70ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c74131c20 msgr2=0x7f1c7411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c70ff9640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c74131c20 0x7f1c7411ca70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1c64010880 tx=0x7f1c64008040 comp rx=0 tx=0).stop 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c74131c20 msgr2=0x7f1c7411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.308+0000 7f1c70ff9640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c74131c20 0x7f1c7411ca70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c7ac0f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:18.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c7ac0f640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c7411cfb0 0x7f1c74096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c7ac0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/2988633254 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7404bc80 con 0x7f1c7411cfb0 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c789ab640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c7411cfb0 0x7f1c74096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c789ab640 10 monclient: get_auth_request con 0x7f1c7411cfb0 auth_method 0 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c789ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c789ab640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c789ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c789ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c789ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c789ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c789ab640 10 monclient(hunting): handle_auth_done global_id 16347 payload 824 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.314+0000 7f1c789ab640 1 -- 172.21.15.178:0/2988633254 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1c7405ed70 con 0x7f1c7411cfb0 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c789ab640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c7411cfb0 0x7f1c74096a30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1c6401a000 tx=0x7f1c6401d000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c64023070 con 0x7f1c7411cfb0 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c70ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c70ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c70ff9640 10 monclient: dump: 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.339 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.340 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f1c64014e00 con 0x7f1c7411cfb0 2024-09-15T01:43:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c70ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c70ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c7411cfb0 msgr2=0x7f1c74096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c70ff9640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c7411cfb0 0x7f1c74096a30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1c6401a000 tx=0x7f1c6401d000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c70ff9640 1 -- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c7411cfb0 msgr2=0x7f1c74096a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c70ff9640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c7411cfb0 0x7f1c74096a30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.341 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:18.341 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:18.341 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:18.341 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:18.341 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:18.341 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:18.341 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:18.341 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:18.341 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:18.341 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:18.341 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 5, 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.342 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.343 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:18.344 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.345 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:18.346 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.347 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:18.348 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:18.348 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.348 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.348 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.348 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.348 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:18.348 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:18.348 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.348 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.348 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.348 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:18.349 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.350 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.351 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:18.352 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:18.353 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.354 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.355 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:18.356 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.357 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.358 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.359 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.360 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:18.361 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.362 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.363 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.364 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.364 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:18.364 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:18.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/2988633254 >> v2:172.21.15.204:6812/28210 conn(0x7f1c600396e0 msgr2=0x7f1c6003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c7ac0f640 1 --2- 172.21.15.178:0/2988633254 >> v2:172.21.15.204:6812/28210 conn(0x7f1c600396e0 0x7f1c6003bb90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009fd0 tx=0x7f1c680090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c7ac0f640 10 monclient: shutdown 2024-09-15T01:43:18.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c7ac0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/2988633254 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1c740a2930 msgr2=0x7f1c7411c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c7ac0f640 1 --2- 172.21.15.178:0/2988633254 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1c740a2930 0x7f1c7411c530 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009f50 tx=0x7f1c6c036000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c791ac640 1 -- 172.21.15.178:0/2988633254 reap_dead start 2024-09-15T01:43:18.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c791ac640 1 -- 172.21.15.178:0/2988633254 reap_dead start 2024-09-15T01:43:18.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/2988633254 shutdown_connections 2024-09-15T01:43:18.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.315+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/2988633254 >> 172.21.15.178:0/2988633254 conn(0x7f1c740a5ed0 msgr2=0x7f1c7409e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.316+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/2988633254 shutdown_connections 2024-09-15T01:43:18.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.316+0000 7f1c7ac0f640 1 -- 172.21.15.178:0/2988633254 wait complete. 2024-09-15T01:43:18.366 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:18.366 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:43:18.367 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:43:18.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.458+0000 7ff3eb86f640 10 monclient: get_monmap_and_config 2024-09-15T01:43:18.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.458+0000 7ff3eb86f640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.458+0000 7ff3eb86f640 10 monclient: monmap: 2024-09-15T01:43:18.460 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.460 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.460 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:18.459194+0000 2024-09-15T01:43:18.460 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:18.459194+0000 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:18.461 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.458+0000 7ff3eb86f640 1 Processor -- start 2024-09-15T01:43:18.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 1 -- start start 2024-09-15T01:43:18.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 10 monclient: init 2024-09-15T01:43:18.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 10 monclient: _add_conns ranks=[5,10,7,9,1,2,3,6,4,8,0] 2024-09-15T01:43:18.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e40a2390 0x7ff3e4132a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 10 monclient(hunting): picked mon.noname-h con 0x7ff3e40a2390 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:18.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff3e409f8a0 0x7ff3e409fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 10 monclient(hunting): picked mon.noname-f con 0x7ff3e409f8a0 addr v2:172.21.15.204:3300/0 2024-09-15T01:43:18.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3e40a0830 0x7ff3e40a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 10 monclient(hunting): picked mon.noname-i con 0x7ff3e40a0830 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:18.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e409d700 con 0x7ff3e409f8a0 2024-09-15T01:43:18.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e409ddb0 con 0x7ff3e40a2390 2024-09-15T01:43:18.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e40a2760 con 0x7ff3e40a0830 2024-09-15T01:43:18.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3eb86f640 10 monclient(hunting): authenticate will time out at 1403.209676s 2024-09-15T01:43:18.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3e960b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e40a2390 0x7ff3e4132a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.459+0000 7ff3e8e0a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff3e409f8a0 0x7ff3e409fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3e40a0830 0x7ff3e40a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e8e0a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff3e409f8a0 0x7ff3e409fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:32924/0 (socket says 172.21.15.178:32924) 2024-09-15T01:43:18.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e960b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e40a2390 0x7ff3e4132a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:42848/0 (socket says 172.21.15.178:42848) 2024-09-15T01:43:18.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e8e0a640 1 -- 172.21.15.178:0/3590632361 learned_addr learned my addr 172.21.15.178:0/3590632361 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:18.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 10 monclient(hunting): get_auth_request con 0x7ff3e40a0830 auth_method 0 2024-09-15T01:43:18.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e8e0a640 10 monclient(hunting): get_auth_request con 0x7ff3e409f8a0 auth_method 0 2024-09-15T01:43:18.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e8e0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e8e0a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e8e0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e960b640 10 monclient(hunting): get_auth_request con 0x7ff3e40a2390 auth_method 0 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e960b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e960b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e960b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e8e0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e8e0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e8e0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e960b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e960b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e960b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 10 monclient(hunting): handle_auth_done global_id 44332 payload 274 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 1 -- 172.21.15.178:0/3590632361 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e40a2390 msgr2=0x7ff3e4132a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 1 --2- 172.21.15.178:0/3590632361 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e40a2390 0x7ff3e4132a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 1 -- 172.21.15.178:0/3590632361 >> v2:172.21.15.204:3300/0 conn(0x7ff3e409f8a0 msgr2=0x7ff3e409fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 1 --2- 172.21.15.178:0/3590632361 >> v2:172.21.15.204:3300/0 conn(0x7ff3e409f8a0 0x7ff3e409fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 1 monclient: found mon.noname-i 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e9e0c640 1 -- 172.21.15.178:0/3590632361 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e404bf60 con 0x7ff3e40a0830 2024-09-15T01:43:18.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.460+0000 7ff3e8e0a640 1 --2- 172.21.15.178:0/3590632361 >> v2:172.21.15.204:3300/0 conn(0x7ff3e409f8a0 0x7ff3e409fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:18.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3e9e0c640 1 --2- 172.21.15.178:0/3590632361 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3e40a0830 0x7ff3e40a0ca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009e60 tx=0x7ff3dc009780 comp rx=0 tx=0).ready entity=mon.7 client_cookie=43cf048c092c409f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3d7fff640 1 -- 172.21.15.178:0/3590632361 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3dc0079d0 con 0x7ff3e40a0830 2024-09-15T01:43:18.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3d7fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:43:18.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3d7fff640 10 monclient: dump: 2024-09-15T01:43:18.476 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.476 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.476 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.476 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.476 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.476 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.476 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3d7fff640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 5 monclient: authenticate success, global_id 44332 2024-09-15T01:43:18.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3d7fff640 1 -- 172.21.15.178:0/3590632361 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc02d050 con 0x7ff3e40a0830 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3d7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 4 set_mon_vals no callback set 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3d7fff640 1 -- 172.21.15.178:0/3590632361 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3dc007bd0 con 0x7ff3e40a0830 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3d7fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3d7fff640 10 monclient: dump: 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.478 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 10 monclient: shutdown 2024-09-15T01:43:18.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 1 -- 172.21.15.178:0/3590632361 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3e40a0830 msgr2=0x7ff3e40a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 1 --2- 172.21.15.178:0/3590632361 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3e40a0830 0x7ff3e40a0ca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009e60 tx=0x7ff3dc009780 comp rx=0 tx=0).stop 2024-09-15T01:43:18.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 1 -- 172.21.15.178:0/3590632361 shutdown_connections 2024-09-15T01:43:18.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 1 --2- 172.21.15.178:0/3590632361 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3e40a0830 0x7ff3e40a0ca0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 1 --2- 172.21.15.178:0/3590632361 >> v2:172.21.15.204:3300/0 conn(0x7ff3e409f8a0 0x7ff3e409fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 1 --2- 172.21.15.178:0/3590632361 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e40a2390 0x7ff3e4132a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 1 -- 172.21.15.178:0/3590632361 >> 172.21.15.178:0/3590632361 conn(0x7ff3e40631b0 msgr2=0x7ff3e40635b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 1 -- 172.21.15.178:0/3590632361 shutdown_connections 2024-09-15T01:43:18.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.461+0000 7ff3eb86f640 1 -- 172.21.15.178:0/3590632361 wait complete. 2024-09-15T01:43:18.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.462+0000 7ff3eb86f640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.462+0000 7ff3eb86f640 10 monclient: monmap: 2024-09-15T01:43:18.480 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.481 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.462+0000 7ff3eb86f640 1 Processor -- start 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.462+0000 7ff3eb86f640 1 -- start start 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.462+0000 7ff3eb86f640 10 monclient: init 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 10 monclient: _add_conns ranks=[7,2,4,0,8,5,6,1,3] 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3e409f8a0 0x7ff3e411d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 10 monclient(hunting): picked mon.noname-i con 0x7ff3e409f8a0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3e40a0830 0x7ff3e411d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 10 monclient(hunting): picked mon.noname-b con 0x7ff3e40a0830 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff3e40a2390 0x7ff3e411dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 10 monclient(hunting): picked mon.noname-c con 0x7ff3e40a2390 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:18.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e409d3d0 con 0x7ff3e40a0830 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e41208e0 con 0x7ff3e40a2390 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e411ead0 con 0x7ff3e409f8a0 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3eb86f640 10 monclient(hunting): authenticate will time out at 1403.213322s 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff3e40a2390 0x7ff3e411dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e8e0a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3e40a0830 0x7ff3e411d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff3e40a2390 0x7ff3e411dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:60254/0 (socket says 172.21.15.178:60254) 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 1 -- 172.21.15.178:0/581037783 learned_addr learned my addr 172.21.15.178:0/581037783 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 10 monclient(hunting): get_auth_request con 0x7ff3e40a2390 auth_method 0 2024-09-15T01:43:18.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e8e0a640 10 monclient(hunting): get_auth_request con 0x7ff3e40a0830 auth_method 0 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e8e0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e8e0a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e8e0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e960b640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3e409f8a0 0x7ff3e411d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e960b640 10 monclient(hunting): get_auth_request con 0x7ff3e409f8a0 auth_method 0 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e960b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e960b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e960b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e8e0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e8e0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e8e0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e960b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e960b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e960b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 10 monclient(hunting): handle_auth_done global_id 44428 payload 824 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 1 -- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3e409f8a0 msgr2=0x7ff3e411d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7ff3e409f8a0 0x7ff3e411d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 1 -- 172.21.15.178:0/581037783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3e40a0830 msgr2=0x7ff3e411d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3e40a0830 0x7ff3e411d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 1 monclient: found mon.noname-c 2024-09-15T01:43:18.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:43:18.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 1 -- 172.21.15.178:0/581037783 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e0009450 con 0x7ff3e40a2390 2024-09-15T01:43:18.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e8e0a640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3e40a0830 0x7ff3e411d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:18.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.463+0000 7ff3e9e0c640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff3e40a2390 0x7ff3e411dce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0099f0 tx=0x7ff3dc037000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3dc031c10 con 0x7ff3e40a2390 2024-09-15T01:43:18.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 10 monclient: dump: 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.487 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3eb86f640 5 monclient: authenticate success, global_id 44428 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc031d90 con 0x7ff3e40a2390 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3dc03d8b0 con 0x7ff3e40a2390 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3eb86f640 10 monclient: _renew_subs 2024-09-15T01:43:18.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3eb86f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3eb86f640 1 -- 172.21.15.178:0/581037783 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3dc009a20 con 0x7ff3e40a2390 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d77fe640 4 set_mon_vals no callback set 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 10 monclient: dump: 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.489 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.490 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.490 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.490 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.490 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.490 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.490 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.490 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.490 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3eb86f640 10 monclient: _renew_subs 2024-09-15T01:43:18.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3eb86f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:18.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3eb86f640 1 -- 172.21.15.178:0/581037783 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e411f6a0 con 0x7ff3e40a2390 2024-09-15T01:43:18.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff3dc003040 con 0x7ff3e40a2390 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3eb86f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.464+0000 7ff3eb86f640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e411e220 0x7ff3e402aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3eb86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3eb86f640 1 -- 172.21.15.178:0/581037783 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e41207a0 con 0x7ff3e411e220 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3d57fa640 1 --2- 172.21.15.178:0/581037783 >> v2:172.21.15.204:6812/28210 conn(0x7ff3c80397b0 0x7ff3c803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7ff3dc06d0e0 con 0x7ff3e40a2390 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e960b640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e411e220 0x7ff3e402aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e960b640 10 monclient: get_auth_request con 0x7ff3e411e220 auth_method 0 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e960b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e960b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e960b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e8e0a640 1 --2- 172.21.15.178:0/581037783 >> v2:172.21.15.204:6812/28210 conn(0x7ff3c80397b0 0x7ff3c803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e8e0a640 10 monclient: get_auth_request con 0x7ff3c80397b0 auth_method 0 2024-09-15T01:43:18.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e960b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e960b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e960b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e960b640 10 monclient(hunting): handle_auth_done global_id 7178 payload 824 2024-09-15T01:43:18.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e960b640 1 -- 172.21.15.178:0/581037783 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3e404c4a0 con 0x7ff3e411e220 2024-09-15T01:43:18.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e960b640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e411e220 0x7ff3e402aa40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8011c20 tx=0x7ff3d80090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3e8e0a640 1 --2- 172.21.15.178:0/581037783 >> v2:172.21.15.204:6812/28210 conn(0x7ff3c80397b0 0x7ff3c803bc60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff3e411f920 tx=0x7ff3e00091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3d8011e60 con 0x7ff3e411e220 2024-09-15T01:43:18.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3d57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:43:18.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.465+0000 7ff3d57fa640 10 monclient: dump: 2024-09-15T01:43:18.492 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.493 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.494 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.494 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.466+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7ff3d8009e00 con 0x7ff3e411e220 2024-09-15T01:43:18.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.466+0000 7ff3d57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.466+0000 7ff3d57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:18.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.466+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e411e220 msgr2=0x7ff3e402aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.466+0000 7ff3d57fa640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e411e220 0x7ff3e402aa40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8011c20 tx=0x7ff3d80090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.466+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e411e220 msgr2=0x7ff3e402aa40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.466+0000 7ff3d57fa640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e411e220 0x7ff3e402aa40 secure :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8011c20 tx=0x7ff3d80090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.471+0000 7ff3eb86f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:18.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.471+0000 7ff3eb86f640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e405e310 0x7ff3e405e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.471+0000 7ff3eb86f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.471+0000 7ff3eb86f640 1 -- 172.21.15.178:0/581037783 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e406f430 con 0x7ff3e405e310 2024-09-15T01:43:18.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3e960b640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e405e310 0x7ff3e405e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3e960b640 10 monclient: get_auth_request con 0x7ff3e405e310 auth_method 0 2024-09-15T01:43:18.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3e960b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3e960b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3e960b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3e960b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3e960b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3e960b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3e960b640 10 monclient(hunting): handle_auth_done global_id 7187 payload 824 2024-09-15T01:43:18.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3e960b640 1 -- 172.21.15.178:0/581037783 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff3e404bc80 con 0x7ff3e405e310 2024-09-15T01:43:18.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3e960b640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e405e310 0x7ff3e405e710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8011c20 tx=0x7ff3d801c000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3d801c3e0 con 0x7ff3e405e310 2024-09-15T01:43:18.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3d57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.472+0000 7ff3d57fa640 10 monclient: dump: 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.498 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.499 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.499 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.499 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.499 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9314 (secure 0 0 0) 0x7ff3d801c540 con 0x7ff3e405e310 2024-09-15T01:43:18.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3d57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:18.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3d57fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:18.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e405e310 msgr2=0x7ff3e405e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3d57fa640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e405e310 0x7ff3e405e710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8011c20 tx=0x7ff3d801c000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3d57fa640 1 -- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e405e310 msgr2=0x7ff3e405e710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3d57fa640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff3e405e310 0x7ff3e405e710 secure :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8011c20 tx=0x7ff3d801c000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.500 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:18.500 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:18.500 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:18.500 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:43:18.500 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:18.500 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:18.500 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:18.500 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:18.500 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:18.500 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:18.500 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:18.501 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:18.501 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:18.501 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:18.501 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:18.501 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.501 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 5, 2024-09-15T01:43:18.501 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:18.501 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:18.501 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:18.501 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.501 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.501 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.502 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:18.503 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:18.504 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:18.504 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:18.504 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:18.504 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:18.504 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:18.504 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:18.504 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:18.504 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:18.504 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:18.504 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:18.504 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:18.504 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:18.505 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:18.506 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.507 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.508 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:18.509 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.510 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.511 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.511 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:18.511 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:18.511 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.511 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.511 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.511 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.511 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:18.511 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.511 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.511 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.511 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.512 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:18.512 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.512 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.512 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.512 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.512 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:18.512 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:18.512 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.512 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.512 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.512 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.512 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.513 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.514 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:18.515 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:18.516 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.517 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.518 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:18.519 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.520 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.520 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.520 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.520 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:18.520 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.520 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.520 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.520 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.520 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:18.520 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.520 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.520 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.521 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.522 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.523 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.523 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:43:18.523 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.523 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.523 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.523 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:18.523 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:18.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3eb86f640 1 -- 172.21.15.178:0/581037783 >> v2:172.21.15.204:6812/28210 conn(0x7ff3c80397b0 msgr2=0x7ff3c803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3eb86f640 1 --2- 172.21.15.178:0/581037783 >> v2:172.21.15.204:6812/28210 conn(0x7ff3c80397b0 0x7ff3c803bc60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff3e411f920 tx=0x7ff3e00091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3eb86f640 10 monclient: shutdown 2024-09-15T01:43:18.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3eb86f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3eb86f640 1 -- 172.21.15.178:0/581037783 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff3e40a2390 msgr2=0x7ff3e411dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3eb86f640 1 --2- 172.21.15.178:0/581037783 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff3e40a2390 0x7ff3e411dce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0099f0 tx=0x7ff3dc037000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3e9e0c640 1 -- 172.21.15.178:0/581037783 reap_dead start 2024-09-15T01:43:18.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3e9e0c640 1 -- 172.21.15.178:0/581037783 reap_dead start 2024-09-15T01:43:18.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3eb86f640 1 -- 172.21.15.178:0/581037783 shutdown_connections 2024-09-15T01:43:18.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3eb86f640 1 -- 172.21.15.178:0/581037783 >> 172.21.15.178:0/581037783 conn(0x7ff3e40631b0 msgr2=0x7ff3e40a4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3eb86f640 1 -- 172.21.15.178:0/581037783 shutdown_connections 2024-09-15T01:43:18.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.473+0000 7ff3eb86f640 1 -- 172.21.15.178:0/581037783 wait complete. 2024-09-15T01:43:18.526 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:18.526 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:43:18.526 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:43:18.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.617+0000 7f319444f640 10 monclient: get_monmap_and_config 2024-09-15T01:43:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.617+0000 7f319444f640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.617+0000 7f319444f640 10 monclient: monmap: 2024-09-15T01:43:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:18.618540+0000 2024-09-15T01:43:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:18.618540+0000 2024-09-15T01:43:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 1 Processor -- start 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 1 -- start start 2024-09-15T01:43:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 10 monclient: init 2024-09-15T01:43:18.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 10 monclient: _add_conns ranks=[10,6,7,0,4,9,8,3,2,5,1] 2024-09-15T01:43:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f318c0a2930 0x7f318c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 10 monclient(hunting): picked mon.noname-f con 0x7f318c0a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:43:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f318c09faa0 0x7f318c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 10 monclient(hunting): picked mon.noname-d con 0x7f318c09faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c0a08c0 0x7f318c0a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 10 monclient(hunting): picked mon.noname-i con 0x7f318c0a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318c09e0f0 con 0x7f318c0a2930 2024-09-15T01:43:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f318c09dd80 con 0x7f318c09faa0 2024-09-15T01:43:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.618+0000 7f319444f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f318c0a2d00 con 0x7f318c0a08c0 2024-09-15T01:43:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f319444f640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f319444f640 10 monclient(hunting): authenticate will time out at 1403.369011s 2024-09-15T01:43:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f318c09faa0 0x7f318c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f318c09faa0 0x7f318c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:51038/0 (socket says 172.21.15.178:51038) 2024-09-15T01:43:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 1 -- 172.21.15.178:0/2255448593 learned_addr learned my addr 172.21.15.178:0/2255448593 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 10 monclient(hunting): get_auth_request con 0x7f318c09faa0 auth_method 0 2024-09-15T01:43:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 10 monclient(hunting): handle_auth_done global_id 25287 payload 274 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31929ec640 1 --2- 172.21.15.178:0/2255448593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c0a08c0 0x7f318c0a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31921eb640 1 --2- 172.21.15.178:0/2255448593 >> v2:172.21.15.204:3300/0 conn(0x7f318c0a2930 0x7f318c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 1 -- 172.21.15.178:0/2255448593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c0a08c0 msgr2=0x7f318c0a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 1 --2- 172.21.15.178:0/2255448593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c0a08c0 0x7f318c0a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 1 -- 172.21.15.178:0/2255448593 >> v2:172.21.15.204:3300/0 conn(0x7f318c0a2930 msgr2=0x7f318c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 1 --2- 172.21.15.178:0/2255448593 >> v2:172.21.15.204:3300/0 conn(0x7f318c0a2930 0x7f318c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 1 monclient: found mon.noname-d 2024-09-15T01:43:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.619+0000 7f31919ea640 1 -- 172.21.15.178:0/2255448593 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c04bf60 con 0x7f318c09faa0 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31919ea640 1 --2- 172.21.15.178:0/2255448593 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f318c09faa0 0x7f318c09fe70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f317c009910 tx=0x7f317c0091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=50812b5cde88790f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31911e9640 1 -- 172.21.15.178:0/2255448593 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f317c02ac80 con 0x7f318c09faa0 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31911e9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31911e9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31911e9640 10 monclient: dump: 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.633 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31911e9640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31911e9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31911e9640 1 -- 172.21.15.178:0/2255448593 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c02ae00 con 0x7f318c09faa0 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 5 monclient: authenticate success, global_id 25287 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31911e9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31911e9640 1 -- 172.21.15.178:0/2255448593 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f317c031ca0 con 0x7f318c09faa0 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 4 set_mon_vals no callback set 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31911e9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31911e9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f31911e9640 10 monclient: dump: 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.635 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 10 monclient: shutdown 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 1 -- 172.21.15.178:0/2255448593 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f318c09faa0 msgr2=0x7f318c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 1 --2- 172.21.15.178:0/2255448593 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f318c09faa0 0x7f318c09fe70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f317c009910 tx=0x7f317c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 1 -- 172.21.15.178:0/2255448593 shutdown_connections 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 1 --2- 172.21.15.178:0/2255448593 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c0a08c0 0x7f318c0a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 1 --2- 172.21.15.178:0/2255448593 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f318c09faa0 0x7f318c09fe70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 1 --2- 172.21.15.178:0/2255448593 >> v2:172.21.15.204:3300/0 conn(0x7f318c0a2930 0x7f318c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 1 -- 172.21.15.178:0/2255448593 >> 172.21.15.178:0/2255448593 conn(0x7f318c0a5ed0 msgr2=0x7f318c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.620+0000 7f319444f640 1 -- 172.21.15.178:0/2255448593 shutdown_connections 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.621+0000 7f319444f640 1 -- 172.21.15.178:0/2255448593 wait complete. 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.621+0000 7f319444f640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.621+0000 7f319444f640 10 monclient: monmap: 2024-09-15T01:43:18.637 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.638 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.621+0000 7f319444f640 1 Processor -- start 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.621+0000 7f319444f640 1 -- start start 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.621+0000 7f319444f640 10 monclient: init 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 10 monclient: _add_conns ranks=[0,2,1,4,5,6,3,7,8] 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f318c09faa0 0x7f318c09b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 10 monclient(hunting): picked mon.noname-a con 0x7f318c09faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c0a08c0 0x7f318c131800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 10 monclient(hunting): picked mon.noname-b con 0x7f318c0a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318c0a2930 0x7f318c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 10 monclient(hunting): picked mon.noname-f con 0x7f318c0a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:18.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c11d640 con 0x7f318c09faa0 2024-09-15T01:43:18.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c0a2390 con 0x7f318c0a2930 2024-09-15T01:43:18.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318c09d630 con 0x7f318c0a08c0 2024-09-15T01:43:18.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 10 monclient(hunting): authenticate will time out at 1403.372219s 2024-09-15T01:43:18.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f318c09faa0 0x7f318c09b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31919ea640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c0a08c0 0x7f318c131800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31919ea640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c0a08c0 0x7f318c131800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:58244/0 (socket says 172.21.15.178:58244) 2024-09-15T01:43:18.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f318c09faa0 0x7f318c09b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60852/0 (socket says 172.21.15.178:60852) 2024-09-15T01:43:18.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31919ea640 1 -- 172.21.15.178:0/4254551367 learned_addr learned my addr 172.21.15.178:0/4254551367 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:18.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31919ea640 10 monclient(hunting): get_auth_request con 0x7f318c0a08c0 auth_method 0 2024-09-15T01:43:18.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31919ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31919ea640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31919ea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 10 monclient(hunting): get_auth_request con 0x7f318c09faa0 auth_method 0 2024-09-15T01:43:18.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31919ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31919ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31929ec640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318c0a2930 0x7f318c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31919ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31929ec640 10 monclient(hunting): get_auth_request con 0x7f318c0a2930 auth_method 0 2024-09-15T01:43:18.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31929ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31929ec640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31929ec640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31929ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31929ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31929ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 10 monclient(hunting): handle_auth_done global_id 34947 payload 824 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 1 -- 172.21.15.178:0/4254551367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c0a08c0 msgr2=0x7f318c131800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c0a08c0 0x7f318c131800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 1 -- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318c0a2930 msgr2=0x7f318c11c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f318c0a2930 0x7f318c11c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 1 monclient: found mon.noname-a 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 1 -- 172.21.15.178:0/4254551367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c009450 con 0x7f318c09faa0 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31919ea640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c0a08c0 0x7f318c131800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:18.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31921eb640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f318c09faa0 0x7f318c09b0c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f318800c880 tx=0x7f3188008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3188007aa0 con 0x7f318c09faa0 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31827fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31827fc640 10 monclient: dump: 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.644 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.645 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.645 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.645 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.645 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.645 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.645 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.645 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31827fc640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3188007c40 con 0x7f318c09faa0 2024-09-15T01:43:18.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 5 monclient: authenticate success, global_id 34947 2024-09-15T01:43:18.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 10 monclient: _renew_subs 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f319444f640 1 -- 172.21.15.178:0/4254551367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c1370c0 con 0x7f318c09faa0 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31827fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3188007df0 con 0x7f318c09faa0 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31827fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.623+0000 7f31827fc640 10 monclient: dump: 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.622+0000 7f31909e8640 4 set_mon_vals no callback set 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.623+0000 7f319444f640 10 monclient: _renew_subs 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.623+0000 7f319444f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:43:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.623+0000 7f319444f640 1 -- 172.21.15.178:0/4254551367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c142a70 con 0x7f318c09faa0 2024-09-15T01:43:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.623+0000 7f319444f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.623+0000 7f319444f640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c131d40 0x7f318c11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.623+0000 7f319444f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.623+0000 7f319444f640 1 -- 172.21.15.178:0/4254551367 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f318c132110 con 0x7f318c131d40 2024-09-15T01:43:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.623+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3188007620 con 0x7f318c09faa0 2024-09-15T01:43:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.623+0000 7f31827fc640 1 --2- 172.21.15.178:0/4254551367 >> v2:172.21.15.204:6812/28210 conn(0x7f31700396e0 0x7f317003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.623+0000 7f31919ea640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c131d40 0x7f318c11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31919ea640 10 monclient: get_auth_request con 0x7f318c131d40 auth_method 0 2024-09-15T01:43:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31919ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31919ea640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31919ea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f318804b910 con 0x7f318c09faa0 2024-09-15T01:43:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31919ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31919ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31919ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31929ec640 1 --2- 172.21.15.178:0/4254551367 >> v2:172.21.15.204:6812/28210 conn(0x7f31700396e0 0x7f317003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31929ec640 10 monclient: get_auth_request con 0x7f31700396e0 auth_method 0 2024-09-15T01:43:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31919ea640 10 monclient(hunting): handle_auth_done global_id 44350 payload 824 2024-09-15T01:43:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31919ea640 1 -- 172.21.15.178:0/4254551367 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f318c04c4a0 con 0x7f318c131d40 2024-09-15T01:43:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31919ea640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c131d40 0x7f318c11ca70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f317c035040 tx=0x7f317c032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31929ec640 1 --2- 172.21.15.178:0/4254551367 >> v2:172.21.15.204:6812/28210 conn(0x7f31700396e0 0x7f317003bb90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f318c0a0070 tx=0x7f31840077b0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f317c02ac80 con 0x7f318c131d40 2024-09-15T01:43:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31827fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:43:18.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31827fc640 10 monclient: dump: 2024-09-15T01:43:18.650 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.650 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.650 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.650 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.650 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.650 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.650 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.650 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.650 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.650 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.650 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.651 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.651 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.651 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.651 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.651 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f317c033e30 con 0x7f318c131d40 2024-09-15T01:43:18.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31827fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31827fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:18.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c131d40 msgr2=0x7f318c11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31827fc640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c131d40 0x7f318c11ca70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f317c035040 tx=0x7f317c032000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c131d40 msgr2=0x7f318c11ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.624+0000 7f31827fc640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c131d40 0x7f318c11ca70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.630+0000 7f319444f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:18.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.630+0000 7f319444f640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c11cfb0 0x7f318c096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.630+0000 7f319444f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.630+0000 7f319444f640 1 -- 172.21.15.178:0/4254551367 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f318c04bc80 con 0x7f318c11cfb0 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.630+0000 7f31919ea640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c11cfb0 0x7f318c096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.630+0000 7f31919ea640 10 monclient: get_auth_request con 0x7f318c11cfb0 auth_method 0 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.630+0000 7f31919ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.630+0000 7f31919ea640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.630+0000 7f31919ea640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.630+0000 7f31919ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.630+0000 7f31919ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.630+0000 7f31919ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31919ea640 10 monclient(hunting): handle_auth_done global_id 44359 payload 824 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31919ea640 1 -- 172.21.15.178:0/4254551367 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f318c05ed70 con 0x7f318c11cfb0 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31919ea640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c11cfb0 0x7f318c096a30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f317c009420 tx=0x7f317c03d000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f317c03cd20 con 0x7f318c11cfb0 2024-09-15T01:43:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31827fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:43:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31827fc640 10 monclient: dump: 2024-09-15T01:43:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f317c03ce80 con 0x7f318c11cfb0 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31827fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31827fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c11cfb0 msgr2=0x7f318c096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31827fc640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c11cfb0 0x7f318c096a30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f317c009420 tx=0x7f317c03d000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31827fc640 1 -- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c11cfb0 msgr2=0x7f318c096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.631+0000 7f31827fc640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f318c11cfb0 0x7f318c096a30 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f317c009420 tx=0x7f317c03d000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.657 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:18.657 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:18.657 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:18.657 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:18.657 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 122, 2024-09-15T01:43:18.657 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:18.657 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:43:18.658 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:18.658 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:18.658 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:43:18.658 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:18.658 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:18.658 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:18.658 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:43:18.658 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:18.658 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.658 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 5, 2024-09-15T01:43:18.658 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:18.659 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:18.659 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:18.659 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.659 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.659 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.659 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.659 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.659 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.659 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.659 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.659 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.659 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.661 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:18.661 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:18.661 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:18.661 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:18.661 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:18.661 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:18.661 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:18.661 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:18.661 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:18.661 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:18.661 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:18.662 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:18.663 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:18.663 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:18.663 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.663 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:18.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.663 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:18.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.663 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:18.663 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:18.663 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.663 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:18.664 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:18.665 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.665 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.665 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.665 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:18.665 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:18.665 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.665 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.665 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.665 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:18.666 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.666 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.666 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.666 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:18.666 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.666 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.666 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.666 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:18.666 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:18.666 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.668 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.668 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.668 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:18.668 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.668 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.668 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.668 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:18.668 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:18.668 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.668 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.668 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.669 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.670 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:18.671 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:18.671 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.671 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.671 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.671 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:18.671 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.671 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.671 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:18.673 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:18.673 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.673 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.673 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.673 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.673 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:18.673 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.673 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.673 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.673 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:18.673 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.674 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:18.675 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:18.676 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.677 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.678 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.678 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:18.678 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:18.678 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:18.678 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:18.678 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.678 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:18.678 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:18.678 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.678 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.679 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:18.680 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:18.680 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.680 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.680 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.680 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:18.680 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.680 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:18.680 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:18.680 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:18.680 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:18.680 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:18.680 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:18.681 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:18.681 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:18.681 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:18.681 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:18.681 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:18.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.632+0000 7f319444f640 1 -- 172.21.15.178:0/4254551367 >> v2:172.21.15.204:6812/28210 conn(0x7f31700396e0 msgr2=0x7f317003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.632+0000 7f319444f640 1 --2- 172.21.15.178:0/4254551367 >> v2:172.21.15.204:6812/28210 conn(0x7f31700396e0 0x7f317003bb90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f318c0a0070 tx=0x7f31840077b0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.632+0000 7f319444f640 10 monclient: shutdown 2024-09-15T01:43:18.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.632+0000 7f319444f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.632+0000 7f319444f640 1 -- 172.21.15.178:0/4254551367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f318c09faa0 msgr2=0x7f318c09b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.632+0000 7f319444f640 1 --2- 172.21.15.178:0/4254551367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f318c09faa0 0x7f318c09b0c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f318800c880 tx=0x7f3188008040 comp rx=0 tx=0).stop 2024-09-15T01:43:18.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.632+0000 7f31929ec640 1 -- 172.21.15.178:0/4254551367 reap_dead start 2024-09-15T01:43:18.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.632+0000 7f31929ec640 1 -- 172.21.15.178:0/4254551367 reap_dead start 2024-09-15T01:43:18.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.632+0000 7f319444f640 1 -- 172.21.15.178:0/4254551367 shutdown_connections 2024-09-15T01:43:18.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.632+0000 7f319444f640 1 -- 172.21.15.178:0/4254551367 >> 172.21.15.178:0/4254551367 conn(0x7f318c0a5ed0 msgr2=0x7f318c0a51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.632+0000 7f319444f640 1 -- 172.21.15.178:0/4254551367 shutdown_connections 2024-09-15T01:43:18.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.632+0000 7f319444f640 1 -- 172.21.15.178:0/4254551367 wait complete. 2024-09-15T01:43:18.683 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:43:18.684 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['g', 'i', 'a'] 2024-09-15T01:43:18.684 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b', 'd', 'h'] 2024-09-15T01:43:18.684 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-15T01:43:18.684 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g store 2024-09-15T01:43:18.684 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g sync_force --yes-i-really-mean-it 2024-09-15T01:43:18.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.776+0000 7f2d27d1d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:18.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.776+0000 7f2d27d1d640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.776+0000 7f2d27d1d640 10 monclient: monmap: 2024-09-15T01:43:18.779 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.779 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.779 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:18.777680+0000 2024-09-15T01:43:18.779 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:18.777680+0000 2024-09-15T01:43:18.779 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.779 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.779 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:18.779 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:18.779 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.779 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.780 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.780 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.780 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.780 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.780 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.780 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:18.780 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:18.780 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.777+0000 7f2d27d1d640 1 Processor -- start 2024-09-15T01:43:18.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.777+0000 7f2d27d1d640 1 -- start start 2024-09-15T01:43:18.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.777+0000 7f2d27d1d640 10 monclient: init 2024-09-15T01:43:18.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.777+0000 7f2d27d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.777+0000 7f2d27d1d640 10 monclient: _add_conns ranks=[10,4,2,8,1,9,5,6,0,3,7] 2024-09-15T01:43:18.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2d2006e7c0 0x7f2d201305b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 10 monclient(hunting): picked mon.noname-f con 0x7f2d2006e7c0 addr v2:172.21.15.204:3300/0 2024-09-15T01:43:18.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2d20130af0 0x7f2d20130ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 10 monclient(hunting): picked mon.noname-c con 0x7f2d20130af0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:18.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2009cac0 0x7f2d2009cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 10 monclient(hunting): picked mon.noname-b con 0x7f2d2009cac0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:18.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d20131420 con 0x7f2d2006e7c0 2024-09-15T01:43:18.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2006e2c0 con 0x7f2d2009cac0 2024-09-15T01:43:18.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2006dd00 con 0x7f2d20130af0 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d27d1d640 10 monclient(hunting): authenticate will time out at 1403.528072s 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2d20130af0 0x7f2d20130ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2d20130af0 0x7f2d20130ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:60282/0 (socket says 172.21.15.178:60282) 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d262ba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2009cac0 0x7f2d2009cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 1 -- 172.21.15.178:0/324018299 learned_addr learned my addr 172.21.15.178:0/324018299 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 10 monclient(hunting): get_auth_request con 0x7f2d20130af0 auth_method 0 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d262ba640 10 monclient(hunting): get_auth_request con 0x7f2d2009cac0 auth_method 0 2024-09-15T01:43:18.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d262ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d262ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d262ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d262ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d262ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d262ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d25ab9640 1 --2- 172.21.15.178:0/324018299 >> v2:172.21.15.204:3300/0 conn(0x7f2d2006e7c0 0x7f2d201305b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d25ab9640 10 monclient(hunting): get_auth_request con 0x7f2d2006e7c0 auth_method 0 2024-09-15T01:43:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d25ab9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d25ab9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d25ab9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 10 monclient(hunting): handle_auth_done global_id 44437 payload 274 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 1 -- 172.21.15.178:0/324018299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2009cac0 msgr2=0x7f2d2009cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 1 --2- 172.21.15.178:0/324018299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2009cac0 0x7f2d2009cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 1 -- 172.21.15.178:0/324018299 >> v2:172.21.15.204:3300/0 conn(0x7f2d2006e7c0 msgr2=0x7f2d201305b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 1 --2- 172.21.15.178:0/324018299 >> v2:172.21.15.204:3300/0 conn(0x7f2d2006e7c0 0x7f2d201305b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 1 monclient: found mon.noname-c 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d252b8640 1 -- 172.21.15.178:0/324018299 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2004bf60 con 0x7f2d20130af0 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.778+0000 7f2d262ba640 1 --2- 172.21.15.178:0/324018299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2009cac0 0x7f2d2009cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d252b8640 1 --2- 172.21.15.178:0/324018299 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2d20130af0 0x7f2d20130ee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009910 tx=0x7f2d140091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=4590e802c29da62d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d24ab7640 1 -- 172.21.15.178:0/324018299 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d1402ac80 con 0x7f2d20130af0 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d24ab7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d24ab7640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d24ab7640 10 monclient: dump: 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.796 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d24ab7640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d24ab7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 5 monclient: authenticate success, global_id 44437 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d24ab7640 1 -- 172.21.15.178:0/324018299 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1402ae00 con 0x7f2d20130af0 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d24ab7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d24ab7640 1 -- 172.21.15.178:0/324018299 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d14031ca0 con 0x7f2d20130af0 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d24ab7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 4 set_mon_vals no callback set 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d24ab7640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d24ab7640 10 monclient: dump: 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.798 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.799 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.799 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.799 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.799 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.799 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.799 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.799 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 10 monclient: shutdown 2024-09-15T01:43:18.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 1 -- 172.21.15.178:0/324018299 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2d20130af0 msgr2=0x7f2d20130ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 1 --2- 172.21.15.178:0/324018299 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2d20130af0 0x7f2d20130ee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009910 tx=0x7f2d140091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 1 -- 172.21.15.178:0/324018299 shutdown_connections 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 1 --2- 172.21.15.178:0/324018299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2009cac0 0x7f2d2009cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 1 --2- 172.21.15.178:0/324018299 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2d20130af0 0x7f2d20130ee0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 1 --2- 172.21.15.178:0/324018299 >> v2:172.21.15.204:3300/0 conn(0x7f2d2006e7c0 0x7f2d201305b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.779+0000 7f2d27d1d640 1 -- 172.21.15.178:0/324018299 >> 172.21.15.178:0/324018299 conn(0x7f2d20067d20 msgr2=0x7f2d200a1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.780+0000 7f2d27d1d640 1 -- 172.21.15.178:0/324018299 shutdown_connections 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.780+0000 7f2d27d1d640 1 -- 172.21.15.178:0/324018299 wait complete. 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.780+0000 7f2d27d1d640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.780+0000 7f2d27d1d640 10 monclient: monmap: 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:18.800 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:18.801 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.801 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:18.801 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:18.801 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:18.801 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.801 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.801 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.801 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.801 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.801 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.801 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.801 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.780+0000 7f2d27d1d640 1 Processor -- start 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.780+0000 7f2d27d1d640 1 -- start start 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.780+0000 7f2d27d1d640 10 monclient: init 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 10 monclient: _add_conns ranks=[8,6,2,1,3,4,0,7,5] 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2d2006e7c0 0x7f2d2012f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 10 monclient(hunting): picked mon.noname-e con 0x7f2d2006e7c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2d2009cac0 0x7f2d2006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 10 monclient(hunting): picked mon.noname-d con 0x7f2d2009cac0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d20130af0 0x7f2d2006a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 10 monclient(hunting): picked mon.noname-b con 0x7f2d20130af0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2006d120 con 0x7f2d20130af0 2024-09-15T01:43:18.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2006cde0 con 0x7f2d2009cac0 2024-09-15T01:43:18.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2011e980 con 0x7f2d2006e7c0 2024-09-15T01:43:18.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d27d1d640 10 monclient(hunting): authenticate will time out at 1403.531309s 2024-09-15T01:43:18.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2d2006e7c0 0x7f2d2012f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d252b8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2d2009cac0 0x7f2d2006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2d2006e7c0 0x7f2d2012f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42122/0 (socket says 172.21.15.178:42122) 2024-09-15T01:43:18.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 1 -- 172.21.15.178:0/1362593635 learned_addr learned my addr 172.21.15.178:0/1362593635 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:18.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d262ba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d20130af0 0x7f2d2006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 10 monclient(hunting): get_auth_request con 0x7f2d2006e7c0 auth_method 0 2024-09-15T01:43:18.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d252b8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2d2009cac0 0x7f2d2006a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:51050/0 (socket says 172.21.15.178:51050) 2024-09-15T01:43:18.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d252b8640 10 monclient(hunting): get_auth_request con 0x7f2d2009cac0 auth_method 0 2024-09-15T01:43:18.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d252b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d252b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d252b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d262ba640 10 monclient(hunting): get_auth_request con 0x7f2d20130af0 auth_method 0 2024-09-15T01:43:18.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d262ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d262ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d262ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d262ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d262ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d262ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d252b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d252b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d252b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 10 monclient(hunting): handle_auth_done global_id 34784 payload 824 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 1 -- 172.21.15.178:0/1362593635 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2d2009cac0 msgr2=0x7f2d2006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2d2009cac0 0x7f2d2006a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 1 -- 172.21.15.178:0/1362593635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d20130af0 msgr2=0x7f2d2006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d20130af0 0x7f2d2006a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 1 monclient: found mon.noname-e 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 1 -- 172.21.15.178:0/1362593635 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d10009390 con 0x7f2d2006e7c0 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d262ba640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d20130af0 0x7f2d2006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.781+0000 7f2d25ab9640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2d2006e7c0 0x7f2d2012f540 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800e1c0 tx=0x7f2d18009920 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d1800eee0 con 0x7f2d2006e7c0 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1dffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1dffb640 10 monclient: dump: 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.806 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.807 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1dffb640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d27d1d640 5 monclient: authenticate success, global_id 34784 2024-09-15T01:43:18.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d180148a0 con 0x7f2d2006e7c0 2024-09-15T01:43:18.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d27d1d640 10 monclient: _renew_subs 2024-09-15T01:43:18.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d27d1d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:43:18.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d27d1d640 1 -- 172.21.15.178:0/1362593635 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d14009450 con 0x7f2d2006e7c0 2024-09-15T01:43:18.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d18014e90 con 0x7f2d2006e7c0 2024-09-15T01:43:18.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:18.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1dffb640 10 monclient: dump: 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.809 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.810 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.810 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d27d1d640 10 monclient: _renew_subs 2024-09-15T01:43:18.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d27d1d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:43:18.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d27d1d640 1 -- 172.21.15.178:0/1362593635 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d20142630 con 0x7f2d2006e7c0 2024-09-15T01:43:18.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.782+0000 7f2d1ffff640 4 set_mon_vals no callback set 2024-09-15T01:43:18.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2d18014a10 con 0x7f2d2006e7c0 2024-09-15T01:43:18.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d27d1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d27d1d640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2012fa80 0x7f2d2006ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d27d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d27d1d640 1 -- 172.21.15.178:0/1362593635 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2011e840 con 0x7f2d2012fa80 2024-09-15T01:43:18.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d1dffb640 1 --2- 172.21.15.178:0/1362593635 >> v2:172.21.15.204:6812/28210 conn(0x7f2d0c0397b0 0x7f2d0c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f2d1804c2b0 con 0x7f2d2006e7c0 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d252b8640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2012fa80 0x7f2d2006ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d262ba640 1 --2- 172.21.15.178:0/1362593635 >> v2:172.21.15.204:6812/28210 conn(0x7f2d0c0397b0 0x7f2d0c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d262ba640 10 monclient: get_auth_request con 0x7f2d0c0397b0 auth_method 0 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d252b8640 10 monclient: get_auth_request con 0x7f2d2012fa80 auth_method 0 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d252b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d252b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d252b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d252b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d252b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d252b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d252b8640 10 monclient(hunting): handle_auth_done global_id 16356 payload 824 2024-09-15T01:43:18.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.783+0000 7f2d252b8640 1 -- 172.21.15.178:0/1362593635 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d2004c4a0 con 0x7f2d2012fa80 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d262ba640 1 --2- 172.21.15.178:0/1362593635 >> v2:172.21.15.204:6812/28210 conn(0x7f2d0c0397b0 0x7f2d0c03bc60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2d2006bb60 tx=0x7f2d100090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d252b8640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2012fa80 0x7f2d2006ae70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2d200764d0 tx=0x7f2d14032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d1402ac80 con 0x7f2d2012fa80 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d1dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d1dffb640 10 monclient: dump: 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.812 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f2d140319b0 con 0x7f2d2012fa80 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d1dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d1dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:18.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2012fa80 msgr2=0x7f2d2006ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d1dffb640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2012fa80 0x7f2d2006ae70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2d200764d0 tx=0x7f2d14032000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2012fa80 msgr2=0x7f2d2006ae70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.784+0000 7f2d1dffb640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2012fa80 0x7f2d2006ae70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d27d1d640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:43:18.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d27d1d640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2006b3b0 0x7f2d20097a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d27d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d27d1d640 1 -- 172.21.15.178:0/1362593635 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d20064250 con 0x7f2d2006b3b0 2024-09-15T01:43:18.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d252b8640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2006b3b0 0x7f2d20097a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d252b8640 10 monclient: get_auth_request con 0x7f2d2006b3b0 auth_method 0 2024-09-15T01:43:18.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d252b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d252b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d252b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d252b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d252b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d252b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d252b8640 10 monclient(hunting): handle_auth_done global_id 16365 payload 824 2024-09-15T01:43:18.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.791+0000 7f2d252b8640 1 -- 172.21.15.178:0/1362593635 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7f2d2004bc80 con 0x7f2d2006b3b0 2024-09-15T01:43:18.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d252b8640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2006b3b0 0x7f2d20097a40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009910 tx=0x7f2d1403d000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d14049c40 con 0x7f2d2006b3b0 2024-09-15T01:43:18.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d1dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:43:18.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d1dffb640 10 monclient: dump: 2024-09-15T01:43:18.819 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.820 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.820 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.820 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.820 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.820 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.820 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.820 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.820 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.820 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.820 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.820 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.821 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.821 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.821 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.821 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7f2d14049da0 con 0x7f2d2006b3b0 2024-09-15T01:43:18.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d1dffb640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:43:18.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d1dffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:18.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2006b3b0 msgr2=0x7f2d20097a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d1dffb640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2006b3b0 0x7f2d20097a40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009910 tx=0x7f2d1403d000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d1dffb640 1 -- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2006b3b0 msgr2=0x7f2d20097a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d1dffb640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d2006b3b0 0x7f2d20097a40 secure :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009910 tx=0x7f2d1403d000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.822 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:18.822 INFO:teuthology.orchestra.run.smithi178.stdout: "ret": 0, 2024-09-15T01:43:18.822 INFO:teuthology.orchestra.run.smithi178.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-15T01:43:18.822 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:18.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d27d1d640 1 -- 172.21.15.178:0/1362593635 >> v2:172.21.15.204:6812/28210 conn(0x7f2d0c0397b0 msgr2=0x7f2d0c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d27d1d640 1 --2- 172.21.15.178:0/1362593635 >> v2:172.21.15.204:6812/28210 conn(0x7f2d0c0397b0 0x7f2d0c03bc60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2d2006bb60 tx=0x7f2d100090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d27d1d640 10 monclient: shutdown 2024-09-15T01:43:18.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.792+0000 7f2d27d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.793+0000 7f2d27d1d640 1 -- 172.21.15.178:0/1362593635 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2d2006e7c0 msgr2=0x7f2d2012f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.793+0000 7f2d27d1d640 1 --2- 172.21.15.178:0/1362593635 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2d2006e7c0 0x7f2d2012f540 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800e1c0 tx=0x7f2d18009920 comp rx=0 tx=0).stop 2024-09-15T01:43:18.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.793+0000 7f2d262ba640 1 -- 172.21.15.178:0/1362593635 reap_dead start 2024-09-15T01:43:18.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.793+0000 7f2d262ba640 1 -- 172.21.15.178:0/1362593635 reap_dead start 2024-09-15T01:43:18.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.793+0000 7f2d27d1d640 1 -- 172.21.15.178:0/1362593635 shutdown_connections 2024-09-15T01:43:18.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.793+0000 7f2d27d1d640 1 -- 172.21.15.178:0/1362593635 >> 172.21.15.178:0/1362593635 conn(0x7f2d20067d20 msgr2=0x7f2d20133570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.793+0000 7f2d27d1d640 1 -- 172.21.15.178:0/1362593635 shutdown_connections 2024-09-15T01:43:18.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.793+0000 7f2d27d1d640 1 -- 172.21.15.178:0/1362593635 wait complete. 2024-09-15T01:43:18.824 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T01:43:18.824 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-15T01:43:18.825 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:43:18.841 INFO:tasks.ceph.mon.g:Stopped 2024-09-15T01:43:18.841 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-15T01:43:18.842 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i store 2024-09-15T01:43:18.842 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i sync_force --yes-i-really-mean-it 2024-09-15T01:43:18.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.934+0000 7f230059d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:18.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.934+0000 7f230059d640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.934+0000 7f230059d640 10 monclient: monmap: 2024-09-15T01:43:18.937 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.937 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.937 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:18.935742+0000 2024-09-15T01:43:18.937 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:18.935742+0000 2024-09-15T01:43:18.937 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.937 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.937 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:18.937 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:18.937 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.938 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.938 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.938 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.938 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.938 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.938 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.938 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:18.938 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:18.938 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.935+0000 7f230059d640 1 Processor -- start 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.935+0000 7f230059d640 1 -- start start 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.935+0000 7f230059d640 10 monclient: init 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 10 monclient: _add_conns ranks=[8,6,0,1,10,7,9,3,2,5,4] 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f22f80a2930 0x7f22f8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 10 monclient(hunting): picked mon.noname-e con 0x7f22f80a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f22f809faa0 0x7f22f809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 10 monclient(hunting): picked mon.noname-d con 0x7f22f809faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f22f80a08c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f804bf60 con 0x7f22f80a08c0 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f22f809e0f0 con 0x7f22f809faa0 2024-09-15T01:43:18.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f22f809dd80 con 0x7f22f80a2930 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f230059d640 10 monclient(hunting): authenticate will time out at 1403.686197s 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f22f809faa0 0x7f22f809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fe339640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f22f80a2930 0x7f22f8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22feb3a640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f22f80a08c0 0x7f22f80a0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:37852/0 (socket says 172.21.15.178:37852) 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22feb3a640 1 -- 172.21.15.178:0/606858551 learned_addr learned my addr 172.21.15.178:0/606858551 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 10 monclient(hunting): get_auth_request con 0x7f22f809faa0 auth_method 0 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fe339640 10 monclient(hunting): get_auth_request con 0x7f22f80a2930 auth_method 0 2024-09-15T01:43:18.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fe339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fe339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fe339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fe339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fe339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fe339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 10 monclient(hunting): handle_auth_done global_id 25305 payload 274 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 1 -- 172.21.15.178:0/606858551 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f22f80a2930 msgr2=0x7f22f8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 1 --2- 172.21.15.178:0/606858551 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f22f80a2930 0x7f22f8132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 1 -- 172.21.15.178:0/606858551 >> v1:172.21.15.178:6789/0 conn(0x7f22f80a08c0 legacy=0x7f22f80a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 1 monclient: found mon.noname-d 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fd337640 1 -- 172.21.15.178:0/606858551 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720108453 0 0) 0x7f22f804bf60 con 0x7f22f80a08c0 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fdb38640 1 -- 172.21.15.178:0/606858551 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f80a2d00 con 0x7f22f809faa0 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fe339640 1 --2- 172.21.15.178:0/606858551 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f22f80a2930 0x7f22f8132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.936+0000 7f22fd337640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fdb38640 1 --2- 172.21.15.178:0/606858551 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f22f809faa0 0x7f22f809fe70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009880 tx=0x7f22e80090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c20a31094e0d1f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fd337640 1 -- 172.21.15.178:0/606858551 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22e802ac50 con 0x7f22f809faa0 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fd337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fd337640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fd337640 10 monclient: dump: 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.951 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fd337640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fd337640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fd337640 1 -- 172.21.15.178:0/606858551 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e802add0 con 0x7f22f809faa0 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f230059d640 5 monclient: authenticate success, global_id 25305 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f230059d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fd337640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f230059d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fd337640 1 -- 172.21.15.178:0/606858551 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22e8031ca0 con 0x7f22f809faa0 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fd337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f230059d640 4 set_mon_vals no callback set 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fd337640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.937+0000 7f22fd337640 10 monclient: dump: 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.953 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.938+0000 7f230059d640 10 monclient: shutdown 2024-09-15T01:43:18.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.938+0000 7f230059d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.938+0000 7f230059d640 1 -- 172.21.15.178:0/606858551 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f22f809faa0 msgr2=0x7f22f809fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.938+0000 7f230059d640 1 --2- 172.21.15.178:0/606858551 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f22f809faa0 0x7f22f809fe70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009880 tx=0x7f22e80090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.938+0000 7f230059d640 1 -- 172.21.15.178:0/606858551 shutdown_connections 2024-09-15T01:43:18.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.938+0000 7f230059d640 1 --2- 172.21.15.178:0/606858551 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f22f809faa0 0x7f22f809fe70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.938+0000 7f230059d640 1 --2- 172.21.15.178:0/606858551 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f22f80a2930 0x7f22f8132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.938+0000 7f230059d640 1 -- 172.21.15.178:0/606858551 >> 172.21.15.178:0/606858551 conn(0x7f22f80a5ed0 msgr2=0x7f22f80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.938+0000 7f230059d640 1 -- 172.21.15.178:0/606858551 shutdown_connections 2024-09-15T01:43:18.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.938+0000 7f230059d640 1 -- 172.21.15.178:0/606858551 wait complete. 2024-09-15T01:43:18.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.938+0000 7f230059d640 10 monclient: build_initial_monmap 2024-09-15T01:43:18.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.938+0000 7f230059d640 10 monclient: monmap: 2024-09-15T01:43:18.955 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:18.956 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 1 Processor -- start 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 1 -- start start 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 10 monclient: init 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 10 monclient: _add_conns ranks=[4,5,0,2,7,3,6,1,8] 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f22f809faa0 0x7f22f8141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 10 monclient(hunting): picked mon.noname-c con 0x7f22f809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f22f80a08c0 0x7f22f8131b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 10 monclient(hunting): picked mon.noname-h con 0x7f22f80a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f80a2930 0x7f22f811e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 10 monclient(hunting): picked mon.noname-a con 0x7f22f80a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:18.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f8142b70 con 0x7f22f80a2930 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22f80a2390 con 0x7f22f809faa0 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22f809d630 con 0x7f22f80a08c0 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f230059d640 10 monclient(hunting): authenticate will time out at 1403.689609s 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f22f809faa0 0x7f22f8141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22feb3a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f80a2930 0x7f22f811e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f22f809faa0 0x7f22f8141c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:60284/0 (socket says 172.21.15.178:60284) 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 1 -- 172.21.15.178:0/953386883 learned_addr learned my addr 172.21.15.178:0/953386883 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 10 monclient(hunting): get_auth_request con 0x7f22f809faa0 auth_method 0 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22feb3a640 10 monclient(hunting): get_auth_request con 0x7f22f80a2930 auth_method 0 2024-09-15T01:43:18.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22feb3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22feb3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22feb3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22feb3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22feb3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fdb38640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f22f80a08c0 0x7f22f8131b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22feb3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fdb38640 10 monclient(hunting): get_auth_request con 0x7f22f80a08c0 auth_method 0 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fdb38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fdb38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fdb38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fdb38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fdb38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fdb38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 10 monclient(hunting): handle_auth_done global_id 44446 payload 824 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 1 -- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f22f80a08c0 msgr2=0x7f22f8131b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f22f80a08c0 0x7f22f8131b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 1 -- 172.21.15.178:0/953386883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f80a2930 msgr2=0x7f22f811e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f80a2930 0x7f22f811e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 10 monclient: _finish_hunting 0 2024-09-15T01:43:18.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.939+0000 7f22fe339640 1 monclient: found mon.noname-c 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22fe339640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22fe339640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22fe339640 1 -- 172.21.15.178:0/953386883 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f4009390 con 0x7f22f809faa0 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22fe339640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f22f809faa0 0x7f22f8141c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f22f400ec80 tx=0x7f22f40090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22f400eec0 con 0x7f22f809faa0 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22ee7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22ee7fc640 10 monclient: dump: 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.961 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22ee7fc640 10 monclient: _finish_auth 0 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:18.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f4013470 con 0x7f22f809faa0 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f230059d640 5 monclient: authenticate success, global_id 44446 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22ee7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f230059d640 10 monclient: _renew_subs 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22f4013a60 con 0x7f22f809faa0 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f230059d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f230059d640 1 -- 172.21.15.178:0/953386883 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e8009390 con 0x7f22f809faa0 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22ee7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22fcb36640 4 set_mon_vals no callback set 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f22ee7fc640 10 monclient: dump: 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.963 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.964 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f230059d640 10 monclient: _renew_subs 2024-09-15T01:43:18.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f230059d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:18.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.940+0000 7f230059d640 1 -- 172.21.15.178:0/953386883 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f8120140 con 0x7f22f809faa0 2024-09-15T01:43:18.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f230059d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f22f4013bc0 con 0x7f22f809faa0 2024-09-15T01:43:18.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f230059d640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f8132060 0x7f22f811eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f230059d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f230059d640 1 -- 172.21.15.178:0/953386883 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f22f8121e10 con 0x7f22f8132060 2024-09-15T01:43:18.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22ee7fc640 1 --2- 172.21.15.178:0/953386883 >> v2:172.21.15.204:6812/28210 conn(0x7f22dc0397b0 0x7f22dc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f22f404c2f0 con 0x7f22f809faa0 2024-09-15T01:43:18.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22fdb38640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f8132060 0x7f22f811eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22fdb38640 10 monclient: get_auth_request con 0x7f22f8132060 auth_method 0 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22fdb38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22fdb38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22fdb38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22feb3a640 1 --2- 172.21.15.178:0/953386883 >> v2:172.21.15.204:6812/28210 conn(0x7f22dc0397b0 0x7f22dc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22fdb38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22fdb38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22fdb38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22feb3a640 10 monclient: get_auth_request con 0x7f22dc0397b0 auth_method 0 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22fdb38640 10 monclient(hunting): handle_auth_done global_id 44368 payload 824 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22fdb38640 1 -- 172.21.15.178:0/953386883 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22f804c4a0 con 0x7f22f8132060 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.941+0000 7f22fdb38640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f8132060 0x7f22f811eb90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f22f80a3290 tx=0x7f22e8032000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.942+0000 7f22feb3a640 1 --2- 172.21.15.178:0/953386883 >> v2:172.21.15.204:6812/28210 conn(0x7f22dc0397b0 0x7f22dc03bc60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f22f0001760 tx=0x7f22f0009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.942+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22e802ac50 con 0x7f22f8132060 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.942+0000 7f22ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.942+0000 7f22ee7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.942+0000 7f22ee7fc640 10 monclient: dump: 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.967 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.968 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.968 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.968 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.968 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.968 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.968 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.942+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f22e8031a40 con 0x7f22f8132060 2024-09-15T01:43:18.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.942+0000 7f22ee7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:18.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.942+0000 7f22ee7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:18.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.942+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f8132060 msgr2=0x7f22f811eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.942+0000 7f22ee7fc640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f8132060 0x7f22f811eb90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f22f80a3290 tx=0x7f22e8032000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.942+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f8132060 msgr2=0x7f22f811eb90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.942+0000 7f22ee7fc640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f8132060 0x7f22f811eb90 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f22f80a3290 tx=0x7f22e8032000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.948+0000 7f230059d640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:43:18.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.948+0000 7f230059d640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f811f0d0 0x7f22f8099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:18.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.948+0000 7f230059d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:18.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.948+0000 7f230059d640 1 -- 172.21.15.178:0/953386883 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f22f8132430 con 0x7f22f811f0d0 2024-09-15T01:43:18.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22fdb38640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f811f0d0 0x7f22f8099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:18.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22fdb38640 10 monclient: get_auth_request con 0x7f22f811f0d0 auth_method 0 2024-09-15T01:43:18.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22fdb38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:18.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22fdb38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:18.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22fdb38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:18.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22fdb38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:18.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22fdb38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:18.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22fdb38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:18.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22fdb38640 10 monclient(hunting): handle_auth_done global_id 44377 payload 824 2024-09-15T01:43:18.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22fdb38640 1 -- 172.21.15.178:0/953386883 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7f22f804bc80 con 0x7f22f811f0d0 2024-09-15T01:43:18.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22fdb38640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f811f0d0 0x7f22f8099a50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f22e8033000 tx=0x7f22e803d000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:18.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22e8046980 con 0x7f22f811f0d0 2024-09-15T01:43:18.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:18.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22ee7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:43:18.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.949+0000 7f22ee7fc640 10 monclient: dump: 2024-09-15T01:43:18.972 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:18.972 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:18.972 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.972 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:18.972 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:18.972 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7f22e8046500 con 0x7f22f811f0d0 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f22ee7fc640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f22ee7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:18.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f811f0d0 msgr2=0x7f22f8099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f22ee7fc640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f811f0d0 0x7f22f8099a50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f22e8033000 tx=0x7f22e803d000 comp rx=0 tx=0).stop 2024-09-15T01:43:18.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f22ee7fc640 1 -- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f811f0d0 msgr2=0x7f22f8099a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:18.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f22ee7fc640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f22f811f0d0 0x7f22f8099a50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.974 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: "ret": 0, 2024-09-15T01:43:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-15T01:43:18.974 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:18.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f230059d640 1 -- 172.21.15.178:0/953386883 >> v2:172.21.15.204:6812/28210 conn(0x7f22dc0397b0 msgr2=0x7f22dc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f230059d640 1 --2- 172.21.15.178:0/953386883 >> v2:172.21.15.204:6812/28210 conn(0x7f22dc0397b0 0x7f22dc03bc60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f22f0001760 tx=0x7f22f0009040 comp rx=0 tx=0).stop 2024-09-15T01:43:18.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f230059d640 10 monclient: shutdown 2024-09-15T01:43:18.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f230059d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:18.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f230059d640 1 -- 172.21.15.178:0/953386883 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f22f809faa0 msgr2=0x7f22f8141c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:18.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f22feb3a640 1 -- 172.21.15.178:0/953386883 reap_dead start 2024-09-15T01:43:18.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f230059d640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f22f809faa0 0x7f22f8141c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f22f400ec80 tx=0x7f22f40090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f230059d640 1 -- 172.21.15.178:0/953386883 shutdown_connections 2024-09-15T01:43:18.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f230059d640 1 --2- 172.21.15.178:0/953386883 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f22f809faa0 0x7f22f8141c50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:18.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f230059d640 1 -- 172.21.15.178:0/953386883 >> 172.21.15.178:0/953386883 conn(0x7f22f80a5ed0 msgr2=0x7f22f809e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:18.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f230059d640 1 -- 172.21.15.178:0/953386883 shutdown_connections 2024-09-15T01:43:18.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:18.950+0000 7f230059d640 1 -- 172.21.15.178:0/953386883 wait complete. 2024-09-15T01:43:18.977 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-15T01:43:18.977 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-15T01:43:18.977 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:43:18.991 INFO:tasks.ceph.mon.i:Stopped 2024-09-15T01:43:18.991 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-15T01:43:18.991 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a store 2024-09-15T01:43:18.992 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a sync_force --yes-i-really-mean-it 2024-09-15T01:43:19.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.084+0000 7f99acd9d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:19.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.084+0000 7f99acd9d640 10 monclient: build_initial_monmap 2024-09-15T01:43:19.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.084+0000 7f99acd9d640 10 monclient: monmap: 2024-09-15T01:43:19.086 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:19.086 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:19.087 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:19.084943+0000 2024-09-15T01:43:19.087 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:19.084943+0000 2024-09-15T01:43:19.087 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:19.087 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:19.087 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:19.087 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:19.087 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:19.087 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:19.087 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:19.087 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:19.087 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:19.087 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:19.088 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:19.088 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:19.088 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:19.088 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:19.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.084+0000 7f99acd9d640 1 Processor -- start 2024-09-15T01:43:19.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.084+0000 7f99acd9d640 1 -- start start 2024-09-15T01:43:19.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.084+0000 7f99acd9d640 10 monclient: init 2024-09-15T01:43:19.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:19.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 10 monclient: _add_conns ranks=[8,5,1,2,6,3,10,9,7,0,4] 2024-09-15T01:43:19.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f99a80a2930 0x7f99a8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:19.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 10 monclient(hunting): picked mon.noname-e con 0x7f99a80a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:19.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f99a809faa0 0x7f99a809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:19.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 10 monclient(hunting): picked mon.noname-h con 0x7f99a809faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:19.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f99a80a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:19.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a804bf60 con 0x7f99a80a08c0 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99a809e0f0 con 0x7f99a809faa0 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f99a809dd80 con 0x7f99a80a2930 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99acd9d640 10 monclient(hunting): authenticate will time out at 1403.835431s 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f99a80a2930 0x7f99a8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f99a80a2930 0x7f99a8132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42140/0 (socket says 172.21.15.178:42140) 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 1 -- 172.21.15.178:0/3830574515 learned_addr learned my addr 172.21.15.178:0/3830574515 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 10 monclient(hunting): get_auth_request con 0x7f99a80a2930 auth_method 0 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:19.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:19.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:19.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:19.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 10 monclient(hunting): handle_auth_done global_id 34802 payload 274 2024-09-15T01:43:19.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 1 -- 172.21.15.178:0/3830574515 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f99a809faa0 msgr2=0x7f99a809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:19.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a5d9b640 1 --2- 172.21.15.178:0/3830574515 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f99a809faa0 0x7f99a809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:19.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 1 --2- 172.21.15.178:0/3830574515 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f99a809faa0 0x7f99a809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:19.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 1 -- 172.21.15.178:0/3830574515 >> v1:172.21.15.204:6789/0 conn(0x7f99a80a08c0 legacy=0x7f99a80a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:19.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 10 monclient: _finish_hunting 0 2024-09-15T01:43:19.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 1 monclient: found mon.noname-e 2024-09-15T01:43:19.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a659c640 1 -- 172.21.15.178:0/3830574515 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a80a2d00 con 0x7f99a80a2930 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a5d9b640 10 monclient: get_auth_request con 0x7f99a809faa0 auth_method 0 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.085+0000 7f99a5d9b640 1 --2- 172.21.15.178:0/3830574515 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f99a809faa0 0x7f99a809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a659c640 1 --2- 172.21.15.178:0/3830574515 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f99a80a2930 0x7f99a8132de0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9998009940 tx=0x7f99980091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=3a3f82d3564ecaaf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a559a640 1 -- 172.21.15.178:0/3830574515 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f999802ac70 con 0x7f99a80a2930 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a559a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a559a640 10 monclient: dump: 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:19.101 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:19.102 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:19.102 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:19.102 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:19.102 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:19.102 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:19.102 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:19.102 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:19.102 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:19.102 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:19.102 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:19.102 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:19.102 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a559a640 10 monclient: _finish_auth 0 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a559a640 1 -- 172.21.15.178:0/3830574515 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999802adf0 con 0x7f99a80a2930 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99acd9d640 5 monclient: authenticate success, global_id 34802 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99acd9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a559a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99acd9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a559a640 1 -- 172.21.15.178:0/3830574515 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9998031ca0 con 0x7f99a80a2930 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99acd9d640 4 set_mon_vals no callback set 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a559a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.086+0000 7f99a559a640 10 monclient: dump: 2024-09-15T01:43:19.103 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:19.104 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:19.104 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:19.104 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:19.104 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:19.104 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:19.104 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:19.104 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:19.104 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:19.104 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:19.104 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:19.104 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:19.104 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:19.105 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:19.105 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:19.105 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:19.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.087+0000 7f99acd9d640 10 monclient: shutdown 2024-09-15T01:43:19.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.087+0000 7f99acd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:19.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.087+0000 7f99acd9d640 1 -- 172.21.15.178:0/3830574515 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f99a80a2930 msgr2=0x7f99a8132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:19.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.087+0000 7f99acd9d640 1 --2- 172.21.15.178:0/3830574515 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f99a80a2930 0x7f99a8132de0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9998009940 tx=0x7f99980091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:19.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.087+0000 7f99acd9d640 1 -- 172.21.15.178:0/3830574515 shutdown_connections 2024-09-15T01:43:19.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.087+0000 7f99acd9d640 1 --2- 172.21.15.178:0/3830574515 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f99a809faa0 0x7f99a809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:19.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.087+0000 7f99acd9d640 1 --2- 172.21.15.178:0/3830574515 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f99a80a2930 0x7f99a8132de0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:19.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.087+0000 7f99acd9d640 1 -- 172.21.15.178:0/3830574515 >> 172.21.15.178:0/3830574515 conn(0x7f99a80a5ed0 msgr2=0x7f99a80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:19.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.087+0000 7f99acd9d640 1 -- 172.21.15.178:0/3830574515 shutdown_connections 2024-09-15T01:43:19.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.087+0000 7f99acd9d640 1 -- 172.21.15.178:0/3830574515 wait complete. 2024-09-15T01:43:19.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.087+0000 7f99acd9d640 10 monclient: build_initial_monmap 2024-09-15T01:43:19.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.087+0000 7f99acd9d640 10 monclient: monmap: 2024-09-15T01:43:19.106 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:19.106 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:19.106 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:19.106 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:19.106 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:19.106 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:19.106 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:19.106 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:19.106 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 1 Processor -- start 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 1 -- start start 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 10 monclient: init 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 10 monclient: _add_conns ranks=[1,0,2,5,3,7,6,4,8] 2024-09-15T01:43:19.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99a809faa0 0x7f99a809b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:19.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 10 monclient(hunting): picked mon.noname-f con 0x7f99a809faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:19.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a80a08c0 0x7f99a81318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:19.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 10 monclient(hunting): picked mon.noname-a con 0x7f99a80a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:19.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99a80a2930 0x7f99a811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:19.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 10 monclient(hunting): picked mon.noname-b con 0x7f99a80a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:19.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:19.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a809d630 con 0x7f99a80a08c0 2024-09-15T01:43:19.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:19.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a811ef90 con 0x7f99a809faa0 2024-09-15T01:43:19.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:19.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a811ecb0 con 0x7f99a80a2930 2024-09-15T01:43:19.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99acd9d640 10 monclient(hunting): authenticate will time out at 1403.838610s 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a5d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a80a08c0 0x7f99a81318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a6d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99a80a2930 0x7f99a811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a5d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a80a08c0 0x7f99a81318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60864/0 (socket says 172.21.15.178:60864) 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a5d9b640 1 -- 172.21.15.178:0/3293669062 learned_addr learned my addr 172.21.15.178:0/3293669062 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a5d9b640 10 monclient(hunting): get_auth_request con 0x7f99a80a08c0 auth_method 0 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a659c640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99a809faa0 0x7f99a809b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a6d9d640 10 monclient(hunting): get_auth_request con 0x7f99a80a2930 auth_method 0 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:19.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a659c640 10 monclient(hunting): get_auth_request con 0x7f99a809faa0 auth_method 0 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:19.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.088+0000 7f99a5d9b640 10 monclient(hunting): handle_auth_done global_id 34974 payload 824 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99a5d9b640 1 -- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99a80a2930 msgr2=0x7f99a811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99a5d9b640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99a80a2930 0x7f99a811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99a5d9b640 1 -- 172.21.15.178:0/3293669062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99a809faa0 msgr2=0x7f99a809b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99a5d9b640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99a809faa0 0x7f99a809b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99a5d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99a5d9b640 1 monclient: found mon.noname-a 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99a5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99a5d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99a5d9b640 1 -- 172.21.15.178:0/3293669062 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9998009450 con 0x7f99a80a08c0 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99a6d9d640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99a80a2930 0x7f99a811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99a5d9b640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a80a08c0 0x7f99a81318b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c8b0 tx=0x7f999c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:19.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f999c00ce40 con 0x7f99a80a08c0 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99967fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99967fc640 10 monclient: dump: 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:19.112 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:19.113 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:19.113 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:19.113 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:19.113 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:19.113 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:19.113 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:19.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99967fc640 10 monclient: _finish_auth 0 2024-09-15T01:43:19.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:19.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99acd9d640 5 monclient: authenticate success, global_id 34974 2024-09-15T01:43:19.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c007cb0 con 0x7f99a80a08c0 2024-09-15T01:43:19.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99acd9d640 10 monclient: _renew_subs 2024-09-15T01:43:19.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99acd9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:43:19.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99acd9d640 1 -- 172.21.15.178:0/3293669062 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a811e9d0 con 0x7f99a80a08c0 2024-09-15T01:43:19.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99967fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:19.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f999c007e20 con 0x7f99a80a08c0 2024-09-15T01:43:19.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:19.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99967fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:19.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99a4d99640 4 set_mon_vals no callback set 2024-09-15T01:43:19.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99967fc640 10 monclient: dump: 2024-09-15T01:43:19.114 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:19.114 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:19.114 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:19.114 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:19.115 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:19.115 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:19.115 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:19.115 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:19.115 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:19.115 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:19.115 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:19.115 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:19.115 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:19.115 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:19.115 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99acd9d640 10 monclient: _renew_subs 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99acd9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.089+0000 7f99acd9d640 1 -- 172.21.15.178:0/3293669062 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a8142430 con 0x7f99a80a08c0 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99acd9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99acd9d640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a811da70 0x7f99a811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99acd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99acd9d640 1 -- 172.21.15.178:0/3293669062 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a804c4a0 con 0x7f99a811da70 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f999c007830 con 0x7f99a80a08c0 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a659c640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a811da70 0x7f99a811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a659c640 10 monclient: get_auth_request con 0x7f99a811da70 auth_method 0 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:19.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:19.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:19.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:19.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:19.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:19.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99967fc640 1 --2- 172.21.15.178:0/3293669062 >> v2:172.21.15.204:6812/28210 conn(0x7f99880396e0 0x7f998803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:19.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f999c04bc90 con 0x7f99a80a08c0 2024-09-15T01:43:19.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a659c640 10 monclient(hunting): handle_auth_done global_id 34983 payload 824 2024-09-15T01:43:19.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a659c640 1 -- 172.21.15.178:0/3293669062 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99a805ed70 con 0x7f99a811da70 2024-09-15T01:43:19.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a6d9d640 1 --2- 172.21.15.178:0/3293669062 >> v2:172.21.15.204:6812/28210 conn(0x7f99880396e0 0x7f998803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:19.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a6d9d640 10 monclient: get_auth_request con 0x7f99880396e0 auth_method 0 2024-09-15T01:43:19.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99a659c640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a811da70 0x7f99a811ca70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f99a80a3290 tx=0x7f9998032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:19.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99980318b0 con 0x7f99a811da70 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99967fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.090+0000 7f99967fc640 10 monclient: dump: 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:19.118 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.091+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f9998031430 con 0x7f99a811da70 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.091+0000 7f99967fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.091+0000 7f99967fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.091+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a811da70 msgr2=0x7f99a811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.091+0000 7f99a6d9d640 1 --2- 172.21.15.178:0/3293669062 >> v2:172.21.15.204:6812/28210 conn(0x7f99880396e0 0x7f998803bb90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f99a809b9c0 tx=0x7f99a00025b0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.091+0000 7f99967fc640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a811da70 0x7f99a811ca70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f99a80a3290 tx=0x7f9998032000 comp rx=0 tx=0).stop 2024-09-15T01:43:19.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.091+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a811da70 msgr2=0x7f99a811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:19.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.091+0000 7f99967fc640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a811da70 0x7f99a811ca70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:19.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.097+0000 7f99acd9d640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:43:19.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.097+0000 7f99acd9d640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a8131df0 0x7f99a811cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:19.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.097+0000 7f99acd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:19.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.097+0000 7f99acd9d640 1 -- 172.21.15.178:0/3293669062 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a809bcd0 con 0x7f99a8131df0 2024-09-15T01:43:19.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.097+0000 7f99a659c640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a8131df0 0x7f99a811cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:19.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.097+0000 7f99a659c640 10 monclient: get_auth_request con 0x7f99a8131df0 auth_method 0 2024-09-15T01:43:19.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.097+0000 7f99a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:19.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.097+0000 7f99a659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:19.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.097+0000 7f99a659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:19.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.097+0000 7f99a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:19.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.097+0000 7f99a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:19.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.097+0000 7f99a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:19.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99a659c640 10 monclient(hunting): handle_auth_done global_id 34992 payload 824 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99a659c640 1 -- 172.21.15.178:0/3293669062 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7f99a804bc80 con 0x7f99a8131df0 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99a659c640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a8131df0 0x7f99a811cfb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f99980097e0 tx=0x7f999803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f999803ce70 con 0x7f99a8131df0 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99967fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99967fc640 10 monclient: dump: 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:19.123 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7f999803de60 con 0x7f99a8131df0 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99967fc640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99967fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:19.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a8131df0 msgr2=0x7f99a811cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:19.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99967fc640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a8131df0 0x7f99a811cfb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f99980097e0 tx=0x7f999803d000 comp rx=0 tx=0).stop 2024-09-15T01:43:19.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99967fc640 1 -- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a8131df0 msgr2=0x7f99a811cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:19.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.098+0000 7f99967fc640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a8131df0 0x7f99a811cfb0 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f99980097e0 tx=0x7f999803d000 comp rx=0 tx=0).stop 2024-09-15T01:43:19.125 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "ret": 0, 2024-09-15T01:43:19.125 INFO:teuthology.orchestra.run.smithi178.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-15T01:43:19.125 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:19.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.099+0000 7f99acd9d640 1 -- 172.21.15.178:0/3293669062 >> v2:172.21.15.204:6812/28210 conn(0x7f99880396e0 msgr2=0x7f998803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:19.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.099+0000 7f99acd9d640 1 --2- 172.21.15.178:0/3293669062 >> v2:172.21.15.204:6812/28210 conn(0x7f99880396e0 0x7f998803bb90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f99a809b9c0 tx=0x7f99a00025b0 comp rx=0 tx=0).stop 2024-09-15T01:43:19.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.099+0000 7f99acd9d640 10 monclient: shutdown 2024-09-15T01:43:19.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.099+0000 7f99acd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:19.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.099+0000 7f99acd9d640 1 -- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a80a08c0 msgr2=0x7f99a81318b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:19.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.099+0000 7f99acd9d640 1 --2- 172.21.15.178:0/3293669062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99a80a08c0 0x7f99a81318b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c8b0 tx=0x7f999c008040 comp rx=0 tx=0).stop 2024-09-15T01:43:19.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.099+0000 7f99a6d9d640 1 -- 172.21.15.178:0/3293669062 reap_dead start 2024-09-15T01:43:19.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.099+0000 7f99a6d9d640 1 -- 172.21.15.178:0/3293669062 reap_dead start 2024-09-15T01:43:19.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.099+0000 7f99acd9d640 1 -- 172.21.15.178:0/3293669062 shutdown_connections 2024-09-15T01:43:19.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.099+0000 7f99acd9d640 1 -- 172.21.15.178:0/3293669062 >> 172.21.15.178:0/3293669062 conn(0x7f99a80a5ed0 msgr2=0x7f99a809e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:19.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.099+0000 7f99acd9d640 1 -- 172.21.15.178:0/3293669062 shutdown_connections 2024-09-15T01:43:19.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:19.099+0000 7f99acd9d640 1 -- 172.21.15.178:0/3293669062 wait complete. 2024-09-15T01:43:19.128 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T01:43:19.128 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T01:43:19.128 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:43:19.141 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T01:43:19.141 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-15T01:43:19.141 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-15T01:43:19.142 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-15T01:43:19.142 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-15T01:43:19.142 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-09-15T01:43:19.142 INFO:tasks.ceph.mon.h:Sent signal 19 2024-09-15T01:43:19.142 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T01:43:34.143 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-15T01:43:34.143 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-15T01:43:34.143 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-15T01:43:34.144 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-15T01:43:34.144 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-09-15T01:43:34.144 INFO:tasks.ceph.mon.h:Sent signal 18 2024-09-15T01:43:34.144 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-09-15T01:43:34.144 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:43:34.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.246+0000 7fd7f4d14640 10 monclient: get_monmap_and_config 2024-09-15T01:43:34.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.246+0000 7fd7f4d14640 10 monclient: build_initial_monmap 2024-09-15T01:43:34.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.247+0000 7fd7f4d14640 10 monclient: monmap: 2024-09-15T01:43:34.249 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:34.249 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:34.247889+0000 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:34.247889+0000 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:34.251 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:34.251 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:34.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.247+0000 7fd7f4d14640 1 Processor -- start 2024-09-15T01:43:34.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.247+0000 7fd7f4d14640 1 -- start start 2024-09-15T01:43:34.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.247+0000 7fd7f4d14640 10 monclient: init 2024-09-15T01:43:34.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:34.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 10 monclient: _add_conns ranks=[2,3,9,6,8,4,10,1,0,5,7] 2024-09-15T01:43:34.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:34.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 10 monclient(hunting): picked mon.noname-b con 0x7fd7f00a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:34.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f009faa0 0x7fd7f009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:34.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 10 monclient(hunting): picked mon.noname-g con 0x7fd7f009faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:34.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd7f00a08c0 0x7fd7f00a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:34.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 10 monclient(hunting): picked mon.noname-a con 0x7fd7f00a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:43:34.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:34.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f009e0f0 con 0x7fd7f00a08c0 2024-09-15T01:43:34.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:34.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f009dd80 con 0x7fd7f00a2930 2024-09-15T01:43:34.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00a2d00 con 0x7fd7f009faa0 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7f4d14640 10 monclient(hunting): authenticate will time out at 1418.998399s 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7eed9d640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7fd7f00a08c0 msgr2=0x7fd7f00a0d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7eed9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd7f00a08c0 0x7fd7f00a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7ee59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7edd9b640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f009faa0 msgr2=0x7fd7f009fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7edd9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f009faa0 0x7fd7f009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7ee59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:51624/0 (socket says 172.21.15.178:51624) 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 learned_addr learned my addr 172.21.15.178:0/852397496 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7f00a2930 auth_method 0 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:34.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:34.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7ee59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:34.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 msgr2=0x7fd7f0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:34.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 msgr2=0x7fd7f0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:34.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:34.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.248+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:34.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.178:3300/0 conn(0x7fd7f00a08c0 msgr2=0x7fd7f00a0d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:43:34.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.178:3300/0 conn(0x7fd7f00a08c0 0x7fd7f00a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:34.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f009faa0 msgr2=0x7fd7f009fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:43:34.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f009faa0 0x7fd7f009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:34.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:34.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7f00a2930 auth_method 0 2024-09-15T01:43:34.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:34.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:34.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7ee59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:34.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 msgr2=0x7fd7f0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:34.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 msgr2=0x7fd7f0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:34.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:34.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.449+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:34.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.178:3300/0 conn(0x7fd7f00a08c0 msgr2=0x7fd7f00a0d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:43:34.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.178:3300/0 conn(0x7fd7f00a08c0 0x7fd7f00a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f009faa0 msgr2=0x7fd7f009fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:43:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7f00a2930 auth_method 0 2024-09-15T01:43:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f009faa0 0x7fd7f009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7ee59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 msgr2=0x7fd7f0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 msgr2=0x7fd7f0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:34.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:34.850+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:35.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f009faa0 msgr2=0x7fd7f009fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:43:35.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f009faa0 0x7fd7f009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:35.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.178:3300/0 conn(0x7fd7f00a08c0 msgr2=0x7fd7f00a0d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:43:35.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.178:3300/0 conn(0x7fd7f00a08c0 0x7fd7f00a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:35.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:35.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7f00a2930 auth_method 0 2024-09-15T01:43:35.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:35.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:35.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7ee59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:35.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 msgr2=0x7fd7f0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:35.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 msgr2=0x7fd7f0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:35.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:35.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:35.651+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 10 monclient(hunting): tick 2024-09-15T01:43:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 monclient(hunting): continuing hunt 2024-09-15T01:43:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:43:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f009faa0 msgr2=0x7fd7f009fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7f009faa0 0x7fd7f009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 msgr2=0x7fd7f0132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd7f00a2930 0x7fd7f0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:37.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.178:3300/0 conn(0x7fd7f00a08c0 msgr2=0x7fd7f00a0d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:37.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.178:3300/0 conn(0x7fd7f00a08c0 0x7fd7f00a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:37.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 10 monclient: _add_conns ranks=[4,10,8,7,1,6,0,5] 2024-09-15T01:43:37.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:37.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 10 monclient(hunting): picked mon.noname-c con 0x7fd7d00037d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:37.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:37.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 10 monclient(hunting): picked mon.noname-f con 0x7fd7d0006210 addr v2:172.21.15.204:3300/0 2024-09-15T01:43:37.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:37.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 10 monclient(hunting): picked mon.noname-e con 0x7fd7d0008cd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:37.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:37.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f009e0f0 con 0x7fd7d0006210 2024-09-15T01:43:37.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:37.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f009dd80 con 0x7fd7d00037d0 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00a2d00 con 0x7fd7d0008cd0 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7dffff640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request con 0x7fd7d0008cd0 auth_method 0 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7eed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7eed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7d00037d0 auth_method 0 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7ee59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 msgr2=0x7fd7d000b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 msgr2=0x7fd7d000b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 msgr2=0x7fd7d0005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 msgr2=0x7fd7d0005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7edd9b640 10 monclient(hunting): get_auth_request con 0x7fd7d0006210 auth_method 0 2024-09-15T01:43:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7edd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:37.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7edd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7edd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.248+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.249+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 msgr2=0x7fd7d0008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.249+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 msgr2=0x7fd7d0008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.249+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:37.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.249+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:37.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.449+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:37.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.449+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request con 0x7fd7d0008cd0 auth_method 0 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.449+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.449+0000 7fd7eed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.449+0000 7fd7eed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.449+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 msgr2=0x7fd7d000b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.449+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 msgr2=0x7fd7d000b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.449+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.449+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7d00037d0 auth_method 0 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7ee59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:37.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 msgr2=0x7fd7d0005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:37.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 msgr2=0x7fd7d0005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:37.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:37.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:37.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:37.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7edd9b640 10 monclient(hunting): get_auth_request con 0x7fd7d0006210 auth_method 0 2024-09-15T01:43:37.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7edd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:37.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7edd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:37.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7edd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:37.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 msgr2=0x7fd7d0008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:37.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 msgr2=0x7fd7d0008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:37.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:37.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.450+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:37.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.850+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:37.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.850+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request con 0x7fd7d0008cd0 auth_method 0 2024-09-15T01:43:37.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.850+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.850+0000 7fd7eed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.850+0000 7fd7eed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.850+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 msgr2=0x7fd7d000b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.850+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 msgr2=0x7fd7d000b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.850+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.850+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7d00037d0 auth_method 0 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7ee59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 msgr2=0x7fd7d0005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:37.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 msgr2=0x7fd7d0005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:37.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:37.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:37.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7edd9b640 10 monclient(hunting): get_auth_request con 0x7fd7d0006210 auth_method 0 2024-09-15T01:43:37.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7edd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:37.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7edd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:37.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7edd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:37.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:37.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 msgr2=0x7fd7d0008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:37.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 msgr2=0x7fd7d0008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:37.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:37.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:37.851+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:38.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:38.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request con 0x7fd7d0008cd0 auth_method 0 2024-09-15T01:43:38.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:38.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7eed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:38.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7eed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:38.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 msgr2=0x7fd7d000b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:38.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 msgr2=0x7fd7d000b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:38.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:38.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:38.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:38.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7edd9b640 10 monclient(hunting): get_auth_request con 0x7fd7d0006210 auth_method 0 2024-09-15T01:43:38.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7edd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7edd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7edd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7d00037d0 auth_method 0 2024-09-15T01:43:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7ee59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 msgr2=0x7fd7d0008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 msgr2=0x7fd7d0008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 msgr2=0x7fd7d0005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 msgr2=0x7fd7d0005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:38.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:38.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:38.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:38.652+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:40.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 10 monclient(hunting): tick 2024-09-15T01:43:40.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 1 monclient(hunting): continuing hunt 2024-09-15T01:43:40.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:43:40.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 msgr2=0x7fd7d000b180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:40.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd7d0008cd0 0x7fd7d000b180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:40.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 msgr2=0x7fd7d0005c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:40.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d00037d0 0x7fd7d0005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:40.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 msgr2=0x7fd7d0008670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:40.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> v2:172.21.15.204:3300/0 conn(0x7fd7d0006210 0x7fd7d0008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:40.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 reap_dead start 2024-09-15T01:43:40.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 10 monclient: _add_conns ranks=[1,7,6,5,0] 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd7d0006660 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd7d0005320 0x7fd7d000b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 10 monclient(hunting): picked mon.noname-i con 0x7fd7d0005320 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 10 monclient(hunting): picked mon.noname-d con 0x7fd7d00056f0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f004bf60 con 0x7fd7d0006660 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f009e0f0 con 0x7fd7d00056f0 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f009dd80 con 0x7fd7d0005320 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7dffff640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd7d0005320 msgr2=0x7fd7d000b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:43:40.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd7d0005320 0x7fd7d000b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request con 0x7fd7d00056f0 auth_method 0 2024-09-15T01:43:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7eed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7eed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 msgr2=0x7fd7d0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 msgr2=0x7fd7d0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.249+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:40.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.450+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd7d0005320 msgr2=0x7fd7d000b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:43:40.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.450+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd7d0005320 0x7fd7d000b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:40.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.450+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:40.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.450+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request con 0x7fd7d00056f0 auth_method 0 2024-09-15T01:43:40.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.450+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:40.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.450+0000 7fd7eed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:40.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.450+0000 7fd7eed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:40.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.450+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 msgr2=0x7fd7d0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:40.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.450+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 msgr2=0x7fd7d0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:40.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.450+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:40.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.450+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:40.826 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=110 2024-09-15T01:43:40.826 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 110 -ne 110 ]] 2024-09-15T01:43:40.826 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:43:40.826 INFO:tasks.workunit.client.0.smithi204.stderr:+ export CEPH_ARGS= 2024-09-15T01:43:40.826 INFO:tasks.workunit.client.0.smithi204.stderr:+ CEPH_ARGS= 2024-09-15T01:43:40.826 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph auth del client.foo 2024-09-15T01:43:40.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.851+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd7d0005320 msgr2=0x7fd7d000b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:43:40.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.851+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd7d0005320 0x7fd7d000b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:40.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.851+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:40.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.851+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request con 0x7fd7d00056f0 auth_method 0 2024-09-15T01:43:40.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.851+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:40.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.851+0000 7fd7eed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:40.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.851+0000 7fd7eed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:40.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.851+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 msgr2=0x7fd7d0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:40.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.851+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 msgr2=0x7fd7d0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:40.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.851+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:40.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:40.851+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:40.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.897+0000 7fa8e383f640 10 monclient: get_monmap_and_config 2024-09-15T01:43:40.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.897+0000 7fa8e383f640 10 monclient: build_initial_monmap 2024-09-15T01:43:40.900 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.897+0000 7fa8e383f640 10 monclient: monmap: 2024-09-15T01:43:40.900 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:43:40.900 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:40.900 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:43:40.899070+0000 2024-09-15T01:43:40.900 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:43:40.899070+0000 2024-09-15T01:43:40.900 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:40.900 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:43:40.900 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:40.901 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:40.901 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:40.901 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:40.901 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:40.901 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:40.901 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:40.901 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:40.901 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:40.901 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:40.901 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:40.901 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:43:40.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 1 Processor -- start 2024-09-15T01:43:40.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 1 -- start start 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 10 monclient: init 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 10 monclient: _add_conns ranks=[7,1,2,10,8,5,9,3,4,0,6] 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8dc0a2440 0x7fa8dc132ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 10 monclient(hunting): picked mon.noname-i con 0x7fa8dc0a2440 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa8dc09f950 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 10 monclient(hunting): picked mon.noname-b con 0x7fa8dc137c60 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc04b720 con 0x7fa8dc09f950 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc09d7b0 con 0x7fa8dc137c60 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:40.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc09e140 con 0x7fa8dc0a2440 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e383f640 10 monclient(hunting): authenticate will time out at 1436.541947s 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.898+0000 7fa8e15db640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8dc0a2440 msgr2=0x7fa8dc132ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.899+0000 7fa8e15db640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8dc0a2440 0x7fa8dc132ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.899+0000 7fa8e0dda640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa8dc09f950 0x7fa8dc09fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:42586/0 (socket says 172.21.15.204:42586) 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.899+0000 7fa8e0dda640 1 -- 172.21.15.204:0/2569897232 learned_addr learned my addr 172.21.15.204:0/2569897232 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.899+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.899+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request con 0x7fa8dc137c60 auth_method 0 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.899+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.899+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.899+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.899+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 msgr2=0x7fa8dc13a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:40.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.899+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 msgr2=0x7fa8dc13a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:40.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.899+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:40.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:40.899+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:41.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.100+0000 7fa8e15db640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8dc0a2440 msgr2=0x7fa8dc132ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:43:41.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.100+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8dc0a2440 0x7fa8dc132ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:41.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.100+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:41.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.100+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request con 0x7fa8dc137c60 auth_method 0 2024-09-15T01:43:41.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.100+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:41.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.100+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:41.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.100+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:41.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.100+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 msgr2=0x7fa8dc13a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:41.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.100+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 msgr2=0x7fa8dc13a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:41.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.100+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:41.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.100+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:41.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.501+0000 7fa8e15db640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8dc0a2440 msgr2=0x7fa8dc132ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:43:41.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.501+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8dc0a2440 0x7fa8dc132ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:41.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.501+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:41.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.501+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request con 0x7fa8dc137c60 auth_method 0 2024-09-15T01:43:41.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.501+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:41.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.501+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:41.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.501+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:41.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.501+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 msgr2=0x7fa8dc13a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:41.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.501+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 msgr2=0x7fa8dc13a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:41.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.501+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:41.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:41.501+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:41.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:41.652+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd7d0005320 msgr2=0x7fd7d000b100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:43:41.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:41.652+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd7d0005320 0x7fd7d000b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:41.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:41.652+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:41.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:41.652+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request con 0x7fd7d00056f0 auth_method 0 2024-09-15T01:43:41.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:41.652+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:41.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:41.652+0000 7fd7eed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:41.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:41.652+0000 7fd7eed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:41.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:41.652+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 msgr2=0x7fd7d0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:41.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:41.652+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 msgr2=0x7fd7d0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:41.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:41.652+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:41.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:41.652+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:42.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:42.302+0000 7fa8e15db640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8dc0a2440 msgr2=0x7fa8dc132ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:43:42.304 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:42.302+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8dc0a2440 0x7fa8dc132ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:42.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:42.302+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:42.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:42.302+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request con 0x7fa8dc137c60 auth_method 0 2024-09-15T01:43:42.305 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:42.302+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:42.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:42.302+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:42.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:42.302+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:42.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:42.302+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 msgr2=0x7fa8dc13a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:42.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:42.302+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 msgr2=0x7fa8dc13a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:42.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:42.302+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:42.306 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:42.302+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:43.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 10 monclient(hunting): tick 2024-09-15T01:43:43.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 1 monclient(hunting): continuing hunt 2024-09-15T01:43:43.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:43:43.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd7d0005320 msgr2=0x7fd7d000b100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:43.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd7d0005320 0x7fd7d000b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:43.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 msgr2=0x7fd7d0005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:43.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d00056f0 0x7fd7d0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:43.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 >> v1:172.21.15.204:6789/0 conn(0x7fd7d0006660 legacy=0x7fd7d0003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:43.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 10 monclient: _add_conns ranks=[5,0] 2024-09-15T01:43:43.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 10 monclient(hunting): picked mon.noname-h con 0x7fd7d000de60 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd7d000e8a0 addr v1:172.21.15.178:6789/0 2024-09-15T01:43:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d000eee0 con 0x7fd7d000e8a0 2024-09-15T01:43:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f009e0f0 con 0x7fd7d000de60 2024-09-15T01:43:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7dffff640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> v1:172.21.15.178:6789/0 conn(0x7fd7d000e8a0 legacy=0x7fd7d0013130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:43:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7d000de60 auth_method 0 2024-09-15T01:43:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7ee59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 msgr2=0x7fd7d000e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 msgr2=0x7fd7d000e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.250+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:43.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.451+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> v1:172.21.15.178:6789/0 conn(0x7fd7d000e8a0 legacy=0x7fd7d0013130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:43:43.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.451+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:43.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.451+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7d000de60 auth_method 0 2024-09-15T01:43:43.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.451+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:43.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.451+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:43.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.451+0000 7fd7ee59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:43.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.451+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 msgr2=0x7fd7d000e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:43.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.451+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 msgr2=0x7fd7d000e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:43.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.451+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:43.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.451+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.852+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> v1:172.21.15.178:6789/0 conn(0x7fd7d000e8a0 legacy=0x7fd7d0013130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:43:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.852+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.852+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7d000de60 auth_method 0 2024-09-15T01:43:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.852+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.852+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:43.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.852+0000 7fd7ee59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:43.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.852+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 msgr2=0x7fd7d000e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:43.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.852+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 msgr2=0x7fd7d000e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:43.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.852+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:43.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:43.852+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:43.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 10 monclient(hunting): tick 2024-09-15T01:43:43.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 1 monclient(hunting): continuing hunt 2024-09-15T01:43:43.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:43:43.903 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8dc0a2440 msgr2=0x7fa8dc132ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:43.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8dc0a2440 0x7fa8dc132ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:43.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 msgr2=0x7fa8dc13a040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:43.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8dc137c60 0x7fa8dc13a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:43.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 1 -- 172.21.15.204:0/2569897232 >> v1:172.21.15.204:6789/0 conn(0x7fa8dc09f950 legacy=0x7fa8dc09fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:43.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 10 monclient: _add_conns ranks=[5,9,8,10,0,4,6,3] 2024-09-15T01:43:43.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 0x7fa8c0005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:43.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 10 monclient(hunting): picked mon.noname-h con 0x7fa8c0003800 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:43.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 1 --2- 172.21.15.204:0/2569897232 >> v2:172.21.15.178:3300/0 conn(0x7fa8c0006220 0x7fa8c0008680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:43.904 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 10 monclient(hunting): picked mon.noname-a con 0x7fa8c0006220 addr v2:172.21.15.178:3300/0 2024-09-15T01:43:43.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:43.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 10 monclient(hunting): picked mon.noname-e con 0x7fa8c0008d30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:43.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:43.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 1 -- 172.21.15.204:0/2569897232 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc09d7b0 con 0x7fa8c0006220 2024-09-15T01:43:43.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:43.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 1 -- 172.21.15.204:0/2569897232 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc09e140 con 0x7fa8c0003800 2024-09-15T01:43:43.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:43.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 1 -- 172.21.15.204:0/2569897232 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c000b9a0 con 0x7fa8c0008d30 2024-09-15T01:43:43.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8d2ffd640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:43.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 0x7fa8c0005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:43.905 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e0dda640 1 -- 172.21.15.204:0/2569897232 >> v2:172.21.15.178:3300/0 conn(0x7fa8c0006220 msgr2=0x7fa8c0008680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:43:43.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e15db640 10 monclient(hunting): get_auth_request con 0x7fa8c0003800 auth_method 0 2024-09-15T01:43:43.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e15db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:43.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e15db640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:43.906 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e15db640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:43.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e0dda640 1 --2- 172.21.15.204:0/2569897232 >> v2:172.21.15.178:3300/0 conn(0x7fa8c0006220 0x7fa8c0008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:43.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e15db640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 msgr2=0x7fa8c0005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:43.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e15db640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 msgr2=0x7fa8c0005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:43.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 0x7fa8c0005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:43.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 0x7fa8c0005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:43.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:43.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request con 0x7fa8c0008d30 auth_method 0 2024-09-15T01:43:43.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:43.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:43.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:43.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 msgr2=0x7fa8c000b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:43:43.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 msgr2=0x7fa8c000b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:43.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:43.909 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:43.899+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:44.103 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e0dda640 1 -- 172.21.15.204:0/2569897232 >> v2:172.21.15.178:3300/0 conn(0x7fa8c0006220 msgr2=0x7fa8c0008680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:43:44.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e0dda640 1 --2- 172.21.15.204:0/2569897232 >> v2:172.21.15.178:3300/0 conn(0x7fa8c0006220 0x7fa8c0008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:44.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 0x7fa8c0005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:44.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e15db640 10 monclient(hunting): get_auth_request con 0x7fa8c0003800 auth_method 0 2024-09-15T01:43:44.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e15db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:44.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e15db640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:44.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e15db640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:44.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e15db640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 msgr2=0x7fa8c0005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:44.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e15db640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 msgr2=0x7fa8c0005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:44.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 0x7fa8c0005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:44.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:44.104 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request con 0x7fa8c0008d30 auth_method 0 2024-09-15T01:43:44.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:44.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:44.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:44.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 0x7fa8c0005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:44.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 msgr2=0x7fa8c000b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:44.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 msgr2=0x7fa8c000b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:44.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:44.105 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.100+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:43:44.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e0dda640 1 -- 172.21.15.204:0/2569897232 >> v2:172.21.15.178:3300/0 conn(0x7fa8c0006220 msgr2=0x7fa8c0008680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:43:44.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e0dda640 1 --2- 172.21.15.204:0/2569897232 >> v2:172.21.15.178:3300/0 conn(0x7fa8c0006220 0x7fa8c0008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:44.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 0x7fa8c0005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:44.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e15db640 10 monclient(hunting): get_auth_request con 0x7fa8c0003800 auth_method 0 2024-09-15T01:43:44.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e15db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:44.504 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e15db640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e15db640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e15db640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 msgr2=0x7fa8c0005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e15db640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 msgr2=0x7fa8c0005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 0x7fa8c0005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 0x7fa8c0005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request con 0x7fa8c0008d30 auth_method 0 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 msgr2=0x7fa8c000b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 msgr2=0x7fa8c000b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:44.505 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:44.506 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:44.501+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:43:44.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:44.652+0000 7fd7edd9b640 1 -- 172.21.15.178:0/852397496 >> v1:172.21.15.178:6789/0 conn(0x7fd7d000e8a0 legacy=0x7fd7d0013130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:43:44.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:44.652+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:44.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:44.652+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7d000de60 auth_method 0 2024-09-15T01:43:44.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:44.652+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:44.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:44.652+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:44.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:44.652+0000 7fd7ee59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:44.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:44.652+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 msgr2=0x7fd7d000e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:43:44.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:44.652+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 msgr2=0x7fd7d000e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:43:44.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:44.652+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:44.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:44.652+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:45.307 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e0dda640 1 -- 172.21.15.204:0/2569897232 >> v2:172.21.15.178:3300/0 conn(0x7fa8c0006220 msgr2=0x7fa8c0008680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:43:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e0dda640 1 --2- 172.21.15.204:0/2569897232 >> v2:172.21.15.178:3300/0 conn(0x7fa8c0006220 0x7fa8c0008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:43:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request con 0x7fa8c0008d30 auth_method 0 2024-09-15T01:43:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:45.308 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 0x7fa8c0005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e15db640 10 monclient(hunting): get_auth_request con 0x7fa8c0003800 auth_method 0 2024-09-15T01:43:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e15db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e15db640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e15db640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e15db640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e15db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:45.309 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e15db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 10 monclient(hunting): handle_auth_done global_id 34811 payload 274 2024-09-15T01:43:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 msgr2=0x7fa8c0005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0003800 0x7fa8c0005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 >> v2:172.21.15.178:3300/0 conn(0x7fa8c0006220 msgr2=0x7fa8c0008680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> v2:172.21.15.178:3300/0 conn(0x7fa8c0006220 0x7fa8c0008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 10 monclient: _finish_hunting 0 2024-09-15T01:43:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 1 monclient: found mon.noname-e 2024-09-15T01:43:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.302+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc0a2810 con 0x7fa8c0008d30 2024-09-15T01:43:45.310 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8e1ddc640 1 -- 172.21.15.204:0/2569897232 reap_dead start 2024-09-15T01:43:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8e1ddc640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009590 tx=0x7fa8d400b040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a2412fe3c25bf5e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8d3fff640 1 -- 172.21.15.204:0/2569897232 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d4031860 con 0x7fa8c0008d30 2024-09-15T01:43:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8d3fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8d3fff640 10 monclient: dump: 2024-09-15T01:43:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:43:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:45.311 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8d3fff640 10 monclient: _finish_auth 0 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8d3fff640 1 -- 172.21.15.204:0/2569897232 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d402f030 con 0x7fa8c0008d30 2024-09-15T01:43:45.312 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8d3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8d3fff640 1 -- 172.21.15.204:0/2569897232 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d4031e60 con 0x7fa8c0008d30 2024-09-15T01:43:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8e383f640 5 monclient: authenticate success, global_id 34811 2024-09-15T01:43:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8e383f640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8e383f640 4 set_mon_vals no callback set 2024-09-15T01:43:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8d3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.303+0000 7fa8d3fff640 10 monclient: dump: 2024-09-15T01:43:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:43:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:45.313 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.304+0000 7fa8e383f640 10 monclient: shutdown 2024-09-15T01:43:45.314 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.304+0000 7fa8e383f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.304+0000 7fa8e383f640 1 -- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 msgr2=0x7fa8c000b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.304+0000 7fa8e383f640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009590 tx=0x7fa8d400b040 comp rx=0 tx=0).stop 2024-09-15T01:43:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.304+0000 7fa8e383f640 1 -- 172.21.15.204:0/2569897232 shutdown_connections 2024-09-15T01:43:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.304+0000 7fa8e383f640 1 --2- 172.21.15.204:0/2569897232 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8c0008d30 0x7fa8c000b1e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.304+0000 7fa8e383f640 1 -- 172.21.15.204:0/2569897232 >> 172.21.15.204:0/2569897232 conn(0x7fa8dc06e860 msgr2=0x7fa8dc06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.304+0000 7fa8e383f640 1 -- 172.21.15.204:0/2569897232 shutdown_connections 2024-09-15T01:43:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.304+0000 7fa8e383f640 1 -- 172.21.15.204:0/2569897232 wait complete. 2024-09-15T01:43:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.305+0000 7fa8e383f640 10 monclient: build_initial_monmap 2024-09-15T01:43:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.305+0000 7fa8e383f640 10 monclient: monmap: 2024-09-15T01:43:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:43:45.315 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:45.316 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:45.317 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:43:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.305+0000 7fa8e383f640 1 Processor -- start 2024-09-15T01:43:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.305+0000 7fa8e383f640 1 -- start start 2024-09-15T01:43:45.317 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.305+0000 7fa8e383f640 10 monclient: init 2024-09-15T01:43:45.320 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 10 monclient: _add_conns ranks=[5,1,0,6,3,7,2,8,4] 2024-09-15T01:43:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0008d30 0x7fa8dc09c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 10 monclient(hunting): picked mon.noname-h con 0x7fa8c0008d30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc1368f0 0x7fa8dc134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 10 monclient(hunting): picked mon.noname-f con 0x7fa8dc1368f0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8dc136470 0x7fa8dc1346c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 10 monclient(hunting): picked mon.noname-a con 0x7fa8dc136470 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc09d480 con 0x7fa8dc136470 2024-09-15T01:43:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1377e0 con 0x7fa8dc1368f0 2024-09-15T01:43:45.321 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc135d20 con 0x7fa8c0008d30 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e383f640 10 monclient(hunting): authenticate will time out at 1440.949468s 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e1ddc640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8dc136470 msgr2=0x7fa8dc1346c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e1ddc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8dc136470 0x7fa8dc1346c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0008d30 0x7fa8dc09c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e0dda640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc1368f0 0x7fa8dc134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0008d30 0x7fa8dc09c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:36768/0 (socket says 172.21.15.204:36768) 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 1 -- 172.21.15.204:0/2139859792 learned_addr learned my addr 172.21.15.204:0/2139859792 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 10 monclient(hunting): get_auth_request con 0x7fa8c0008d30 auth_method 0 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:45.322 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e0dda640 10 monclient(hunting): get_auth_request con 0x7fa8dc1368f0 auth_method 0 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e0dda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e0dda640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e0dda640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e0dda640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e0dda640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e0dda640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 10 monclient(hunting): handle_auth_done global_id 7214 payload 824 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 1 -- 172.21.15.204:0/2139859792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc1368f0 msgr2=0x7fa8dc134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:45.323 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2139859792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc1368f0 0x7fa8dc134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 1 -- 172.21.15.204:0/2139859792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8dc136470 msgr2=0x7fa8dc1346c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:43:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2139859792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8dc136470 0x7fa8dc1346c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 10 monclient: _finish_hunting 0 2024-09-15T01:43:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 1 monclient: found mon.noname-h 2024-09-15T01:43:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.306+0000 7fa8e15db640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8e15db640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:43:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8e15db640 1 -- 172.21.15.204:0/2139859792 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d4009c00 con 0x7fa8c0008d30 2024-09-15T01:43:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8e15db640 1 --2- 172.21.15.204:0/2139859792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0008d30 0x7fa8dc09c5d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80094b0 tx=0x7fa8d800b040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d17fa640 1 -- 172.21.15.204:0/2139859792 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d8007be0 con 0x7fa8c0008d30 2024-09-15T01:43:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d17fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:43:45.324 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d17fa640 10 monclient: dump: 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:45.325 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d17fa640 10 monclient: _finish_auth 0 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d17fa640 1 -- 172.21.15.204:0/2139859792 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d8007d40 con 0x7fa8c0008d30 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8e383f640 5 monclient: authenticate success, global_id 7214 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8e383f640 10 monclient: _renew_subs 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8e383f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8e383f640 1 -- 172.21.15.204:0/2139859792 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc0972a0 con 0x7fa8c0008d30 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d17fa640 1 -- 172.21.15.204:0/2139859792 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d8007eb0 con 0x7fa8c0008d30 2024-09-15T01:43:45.326 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:45.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d17fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:43:45.327 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d17fa640 10 monclient: dump: 2024-09-15T01:43:45.327 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:43:45.327 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:45.327 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:45.327 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:45.327 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:45.327 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:43:45.327 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8d37fe640 4 set_mon_vals no callback set 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8e383f640 10 monclient: _renew_subs 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8e383f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:43:45.328 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.307+0000 7fa8e383f640 1 -- 172.21.15.204:0/2139859792 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc134da0 con 0x7fa8c0008d30 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.308+0000 7fa8d17fa640 1 -- 172.21.15.204:0/2139859792 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa8d8017440 con 0x7fa8c0008d30 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.308+0000 7fa8e383f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.308+0000 7fa8e383f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.308+0000 7fa8e383f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.308+0000 7fa8e383f640 1 -- 172.21.15.204:0/2139859792 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc135020 con 0x7fa8c0008d30 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.308+0000 7fa8d17fa640 1 --2- 172.21.15.204:0/2139859792 >> v2:172.21.15.204:6812/28210 conn(0x7fa8c003a960 0x7fa8c003ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.308+0000 7fa8d17fa640 1 -- 172.21.15.204:0/2139859792 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fa8d804ce60 con 0x7fa8c0008d30 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.309+0000 7fa8e0dda640 1 --2- 172.21.15.204:0/2139859792 >> v2:172.21.15.204:6812/28210 conn(0x7fa8c003a960 0x7fa8c003ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.309+0000 7fa8e0dda640 10 monclient: get_auth_request con 0x7fa8c003a960 auth_method 0 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.309+0000 7fa8e0dda640 1 --2- 172.21.15.204:0/2139859792 >> v2:172.21.15.204:6812/28210 conn(0x7fa8c003a960 0x7fa8c003ce10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc001120 tx=0x7fa8cc007d20 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.313+0000 7fa8d17fa640 1 -- 172.21.15.204:0/2139859792 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa8d8015930 con 0x7fa8c0008d30 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.313+0000 7fa8d17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:45.329 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.313+0000 7fa8d17fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:45.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.529+0000 7fa8e383f640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.foo"}] 2024-09-15T01:43:45.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.529+0000 7fa8e383f640 10 monclient: _send_command 2 [{"prefix": "auth del", "entity": "client.foo"}] 2024-09-15T01:43:45.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.529+0000 7fa8e383f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:43:45.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.529+0000 7fa8e383f640 1 -- 172.21.15.204:0/2139859792 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "auth del", "entity": "client.foo"} v 0) v1 -- 0x7fa8dc063440 con 0x7fa8c0008d30 2024-09-15T01:43:45.537 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.535+0000 7fa8d17fa640 1 -- 172.21.15.204:0/2139859792 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.foo"}]=0 v12) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa8d801e360 con 0x7fa8c0008d30 2024-09-15T01:43:45.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.535+0000 7fa8d17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth del", "entity": "client.foo"}] 2024-09-15T01:43:45.538 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.535+0000 7fa8d17fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:45.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.689+0000 7fa8e383f640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.foo"}] 2024-09-15T01:43:45.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.689+0000 7fa8e383f640 10 monclient: _send_command 3 [{"prefix": "auth del", "entity": "client.foo"}] 2024-09-15T01:43:45.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.689+0000 7fa8e383f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:43:45.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.689+0000 7fa8e383f640 1 -- 172.21.15.204:0/2139859792 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "auth del", "entity": "client.foo"} v 0) v1 -- 0x7fa8dc09de60 con 0x7fa8c0008d30 2024-09-15T01:43:45.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.690+0000 7fa8d17fa640 1 -- 172.21.15.204:0/2139859792 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.foo"}]=0 v12) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa8d8015330 con 0x7fa8c0008d30 2024-09-15T01:43:45.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.690+0000 7fa8d17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth del", "entity": "client.foo"}] 2024-09-15T01:43:45.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.690+0000 7fa8d17fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:43:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 1 -- 172.21.15.204:0/2139859792 >> v2:172.21.15.204:6812/28210 conn(0x7fa8c003a960 msgr2=0x7fa8c003ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 1 --2- 172.21.15.204:0/2139859792 >> v2:172.21.15.204:6812/28210 conn(0x7fa8c003a960 0x7fa8c003ce10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc001120 tx=0x7fa8cc007d20 comp rx=0 tx=0).stop 2024-09-15T01:43:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 10 monclient: shutdown 2024-09-15T01:43:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 1 -- 172.21.15.204:0/2139859792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0008d30 msgr2=0x7fa8dc09c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 1 --2- 172.21.15.204:0/2139859792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0008d30 0x7fa8dc09c5d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80094b0 tx=0x7fa8d800b040 comp rx=0 tx=0).stop 2024-09-15T01:43:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 1 -- 172.21.15.204:0/2139859792 shutdown_connections 2024-09-15T01:43:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 1 --2- 172.21.15.204:0/2139859792 >> v2:172.21.15.204:6812/28210 conn(0x7fa8c003a960 0x7fa8c003ce10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 1 --2- 172.21.15.204:0/2139859792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8dc136470 0x7fa8dc1346c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 1 --2- 172.21.15.204:0/2139859792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc1368f0 0x7fa8dc134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 1 --2- 172.21.15.204:0/2139859792 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa8c0008d30 0x7fa8dc09c5d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:45.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 1 -- 172.21.15.204:0/2139859792 >> 172.21.15.204:0/2139859792 conn(0x7fa8dc06e860 msgr2=0x7fa8dc12da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:45.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 1 -- 172.21.15.204:0/2139859792 shutdown_connections 2024-09-15T01:43:45.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:43:45.692+0000 7fa8e383f640 1 -- 172.21.15.204:0/2139859792 wait complete. 2024-09-15T01:43:45.703 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 13 2024-09-15T01:43:45.703 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-09-15T01:43:45.703 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 2024-09-15T01:43:45.703 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=13 2024-09-15T01:43:45.703 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-09-15T01:43:45.703 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-09-15T01:43:46.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 10 monclient(hunting): tick 2024-09-15T01:43:46.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 1 monclient(hunting): continuing hunt 2024-09-15T01:43:46.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:43:46.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 msgr2=0x7fd7d000e230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:46.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d000de60 0x7fd7d000e230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 >> v1:172.21.15.178:6789/0 conn(0x7fd7d000e8a0 legacy=0x7fd7d0013130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:46.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 reap_dead start 2024-09-15T01:43:46.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 10 monclient: _add_conns ranks=[6,5,3,2,4,7,1,8,9,0,10] 2024-09-15T01:43:46.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d0005b40 0x7fd7d00148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 10 monclient(hunting): picked mon.noname-d con 0x7fd7d0005b40 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:46.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d0008650 0x7fd7d000b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 10 monclient(hunting): picked mon.noname-h con 0x7fd7d0008650 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:46.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7d0014e20 0x7fd7d0015290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 10 monclient(hunting): picked mon.noname-g con 0x7fd7d0014e20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:46.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f009e0f0 con 0x7fd7d0014e20 2024-09-15T01:43:46.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f009dd80 con 0x7fd7d0008650 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 1 -- 172.21.15.178:0/852397496 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00a2d00 con 0x7fd7d0005b40 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7dffff640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7eed9d640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7d0014e20 msgr2=0x7fd7d0015290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d0005b40 0x7fd7d00148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7d0014e20 0x7fd7d0015290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7d0005b40 auth_method 0 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7edd9b640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d0008650 0x7fd7d000b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7edd9b640 10 monclient(hunting): get_auth_request con 0x7fd7d0008650 auth_method 0 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7edd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7edd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7edd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7edd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7edd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7edd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 10 monclient(hunting): handle_auth_done global_id 25314 payload 274 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d0008650 msgr2=0x7fd7d000b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d0008650 0x7fd7d000b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7d0014e20 msgr2=0x7fd7d0015290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7d0014e20 0x7fd7d0015290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 10 monclient: _finish_hunting 0 2024-09-15T01:43:46.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 1 monclient: found mon.noname-d 2024-09-15T01:43:46.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:46.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:46.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.249+0000 7fd7ee59c640 1 -- 172.21.15.178:0/852397496 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d0015920 con 0x7fd7d0005b40 2024-09-15T01:43:46.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d0005b40 0x7fd7d00148e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0006f60 tx=0x7fd7e0008040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=68bb0f45fee4efe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ed59a640 1 -- 172.21.15.178:0/852397496 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7e002cde0 con 0x7fd7d0005b40 2024-09-15T01:43:46.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ed59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ed59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:46.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ed59a640 10 monclient: dump: 2024-09-15T01:43:46.265 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.265 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.266 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ed59a640 10 monclient: _finish_auth 0 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ed59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7f4d14640 5 monclient: authenticate success, global_id 25314 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ed59a640 1 -- 172.21.15.178:0/852397496 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e000c040 con 0x7fd7d0005b40 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ed59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7f4d14640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7f4d14640 4 set_mon_vals no callback set 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ed59a640 1 -- 172.21.15.178:0/852397496 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7e00199f0 con 0x7fd7d0005b40 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ed59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ed59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.250+0000 7fd7ed59a640 10 monclient: dump: 2024-09-15T01:43:46.267 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.268 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.269 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.269 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 10 monclient: shutdown 2024-09-15T01:43:46.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:46.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 1 -- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d0005b40 msgr2=0x7fd7d00148e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d0005b40 0x7fd7d00148e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0006f60 tx=0x7fd7e0008040 comp rx=0 tx=0).stop 2024-09-15T01:43:46.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 1 -- 172.21.15.178:0/852397496 shutdown_connections 2024-09-15T01:43:46.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7d0014e20 0x7fd7d0015290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd7d0008650 0x7fd7d000b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 1 --2- 172.21.15.178:0/852397496 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd7d0005b40 0x7fd7d00148e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 1 -- 172.21.15.178:0/852397496 >> 172.21.15.178:0/852397496 conn(0x7fd7f00a5ed0 msgr2=0x7fd7f00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:46.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 1 -- 172.21.15.178:0/852397496 shutdown_connections 2024-09-15T01:43:46.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 1 -- 172.21.15.178:0/852397496 wait complete. 2024-09-15T01:43:46.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 10 monclient: build_initial_monmap 2024-09-15T01:43:46.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.251+0000 7fd7f4d14640 10 monclient: monmap: 2024-09-15T01:43:46.270 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:46.270 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:46.270 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:46.270 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:46.270 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:46.270 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:46.270 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:46.271 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:46.271 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:46.271 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:46.271 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:46.271 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:46.271 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:46.271 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:46.271 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:46.271 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 1 Processor -- start 2024-09-15T01:43:46.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 1 -- start start 2024-09-15T01:43:46.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 10 monclient: init 2024-09-15T01:43:46.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:46.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 10 monclient: _add_conns ranks=[1,0,4,3,8,5,6,2,7] 2024-09-15T01:43:46.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7d0005b40 0x7fd7f0131740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 10 monclient(hunting): picked mon.noname-f con 0x7fd7d0005b40 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:46.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0008650 0x7fd7f011e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 10 monclient(hunting): picked mon.noname-a con 0x7fd7d0008650 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:46.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d0014e20 0x7fd7f011e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 10 monclient(hunting): picked mon.noname-c con 0x7fd7d0014e20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:46.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f0121b70 con 0x7fd7d0008650 2024-09-15T01:43:46.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f0120590 con 0x7fd7d0005b40 2024-09-15T01:43:46.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00a2390 con 0x7fd7d0014e20 2024-09-15T01:43:46.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:46.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7f4d14640 10 monclient(hunting): authenticate will time out at 1431.003844s 2024-09-15T01:43:46.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7edd9b640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0008650 msgr2=0x7fd7f011e270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:43:46.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7edd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0008650 0x7fd7f011e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:46.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.252+0000 7fd7eed9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d0014e20 0x7fd7f011e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d0014e20 0x7fd7f011e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:50112/0 (socket says 172.21.15.178:50112) 2024-09-15T01:43:46.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 1 -- 172.21.15.178:0/1772303093 learned_addr learned my addr 172.21.15.178:0/1772303093 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:46.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request con 0x7fd7d0014e20 auth_method 0 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/1772303093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7d0005b40 0x7fd7f0131740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request con 0x7fd7d0005b40 auth_method 0 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7ee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7ee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7ee59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7ee59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7ee59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7ee59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 10 monclient(hunting): handle_auth_done global_id 44455 payload 824 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 1 -- 172.21.15.178:0/1772303093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7d0005b40 msgr2=0x7fd7f0131740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/1772303093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7d0005b40 0x7fd7f0131740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 1 -- 172.21.15.178:0/1772303093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0008650 msgr2=0x7fd7f011e270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/1772303093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0008650 0x7fd7f011e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 1 monclient: found mon.noname-c 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 1 -- 172.21.15.178:0/1772303093 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e000e040 con 0x7fd7d0014e20 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7eed9d640 1 --2- 172.21.15.178:0/1772303093 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d0014e20 0x7fd7f011e7b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0133890 tx=0x7fd7e8003f90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7de7fc640 1 -- 172.21.15.178:0/1772303093 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7e801b070 con 0x7fd7d0014e20 2024-09-15T01:43:46.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7de7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:43:46.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7de7fc640 10 monclient: dump: 2024-09-15T01:43:46.276 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.276 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.276 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.276 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.276 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7de7fc640 10 monclient: _finish_auth 0 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:46.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7f4d14640 5 monclient: authenticate success, global_id 44455 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7de7fc640 1 -- 172.21.15.178:0/1772303093 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e8016d00 con 0x7fd7d0014e20 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7f4d14640 10 monclient: _renew_subs 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7f4d14640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7f4d14640 1 -- 172.21.15.178:0/1772303093 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f009d630 con 0x7fd7d0014e20 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7de7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7de7fc640 1 -- 172.21.15.178:0/1772303093 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7e800b040 con 0x7fd7d0014e20 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7de7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7de7fc640 10 monclient: dump: 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.278 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7ecd99640 4 set_mon_vals no callback set 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7f4d14640 10 monclient: _renew_subs 2024-09-15T01:43:46.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7f4d14640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:46.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.253+0000 7fd7f4d14640 1 -- 172.21.15.178:0/1772303093 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f0121890 con 0x7fd7d0014e20 2024-09-15T01:43:46.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.254+0000 7fd7de7fc640 1 -- 172.21.15.178:0/1772303093 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd7e8014420 con 0x7fd7d0014e20 2024-09-15T01:43:46.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.254+0000 7fd7f4d14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:46.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.254+0000 7fd7f4d14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:46.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.254+0000 7fd7f4d14640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:46.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.254+0000 7fd7f4d14640 1 -- 172.21.15.178:0/1772303093 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f004b720 con 0x7fd7d0014e20 2024-09-15T01:43:46.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.255+0000 7fd7de7fc640 1 --2- 172.21.15.178:0/1772303093 >> v2:172.21.15.204:6812/28210 conn(0x7fd7d003bf10 0x7fd7d003e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.255+0000 7fd7de7fc640 1 -- 172.21.15.178:0/1772303093 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd7e804c740 con 0x7fd7d0014e20 2024-09-15T01:43:46.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.255+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/1772303093 >> v2:172.21.15.204:6812/28210 conn(0x7fd7d003bf10 0x7fd7d003e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.255+0000 7fd7ee59c640 10 monclient: get_auth_request con 0x7fd7d003bf10 auth_method 0 2024-09-15T01:43:46.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.255+0000 7fd7ee59c640 1 --2- 172.21.15.178:0/1772303093 >> v2:172.21.15.204:6812/28210 conn(0x7fd7d003bf10 0x7fd7d003e3c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0035000 tx=0x7fd7e0036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.259+0000 7fd7de7fc640 1 -- 172.21.15.178:0/1772303093 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd7e80146d0 con 0x7fd7d0014e20 2024-09-15T01:43:46.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.259+0000 7fd7de7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:46.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.259+0000 7fd7de7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:46.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.480+0000 7fd7f4d14640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:43:46.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.480+0000 7fd7f4d14640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:43:46.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.480+0000 7fd7f4d14640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:46.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.480+0000 7fd7f4d14640 1 -- 172.21.15.178:0/1772303093 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd7f009cef0 con 0x7fd7d0014e20 2024-09-15T01:43:46.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.480+0000 7fd7de7fc640 1 -- 172.21.15.178:0/1772303093 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3166 (secure 0 0 0) 0x7fd7e8012ac0 con 0x7fd7d0014e20 2024-09-15T01:43:46.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.480+0000 7fd7de7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:43:46.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.480+0000 7fd7de7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:46.484 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":128,"quorum":[1,2,4,5,6,8],"quorum_names":["f","b","c","h","d","e"],"quorum_leader_name":"h","quorum_age":1,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:43:46.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 1 -- 172.21.15.178:0/1772303093 >> v2:172.21.15.204:6812/28210 conn(0x7fd7d003bf10 msgr2=0x7fd7d003e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 1 --2- 172.21.15.178:0/1772303093 >> v2:172.21.15.204:6812/28210 conn(0x7fd7d003bf10 0x7fd7d003e3c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0035000 tx=0x7fd7e0036000 comp rx=0 tx=0).stop 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 10 monclient: shutdown 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 1 -- 172.21.15.178:0/1772303093 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d0014e20 msgr2=0x7fd7f011e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 1 --2- 172.21.15.178:0/1772303093 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d0014e20 0x7fd7f011e7b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0133890 tx=0x7fd7e8003f90 comp rx=0 tx=0).stop 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 1 -- 172.21.15.178:0/1772303093 shutdown_connections 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 1 --2- 172.21.15.178:0/1772303093 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd7d0014e20 0x7fd7f011e7b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 1 --2- 172.21.15.178:0/1772303093 >> v2:172.21.15.204:6812/28210 conn(0x7fd7d003bf10 0x7fd7d003e3c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 1 --2- 172.21.15.178:0/1772303093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d0008650 0x7fd7f011e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 1 --2- 172.21.15.178:0/1772303093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7d0005b40 0x7fd7f0131740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 1 -- 172.21.15.178:0/1772303093 >> 172.21.15.178:0/1772303093 conn(0x7fd7f00a5ed0 msgr2=0x7fd7f00a3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 1 -- 172.21.15.178:0/1772303093 shutdown_connections 2024-09-15T01:43:46.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.482+0000 7fd7f4d14640 1 -- 172.21.15.178:0/1772303093 wait complete. 2024-09-15T01:43:46.496 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-09-15T01:43:46.496 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon b 2024-09-15T01:43:46.497 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:43:46.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.587+0000 7f39a946f640 10 monclient: get_monmap_and_config 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.587+0000 7f39a946f640 10 monclient: build_initial_monmap 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.587+0000 7f39a946f640 10 monclient: monmap: 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:46.589624+0000 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:46.589624+0000 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:46.591 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:46.592 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:46.592 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:46.592 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:46.592 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:46.592 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:46.592 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:46.592 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 1 Processor -- start 2024-09-15T01:43:46.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 1 -- start start 2024-09-15T01:43:46.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 10 monclient: init 2024-09-15T01:43:46.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:46.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 10 monclient: _add_conns ranks=[3,4,2,8,9,1,6,7,10,5,0] 2024-09-15T01:43:46.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39a40a2930 0x7f39a4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 10 monclient(hunting): picked mon.noname-g con 0x7f39a40a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:46.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 0x7f39a409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 10 monclient(hunting): picked mon.noname-c con 0x7f39a409faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:46.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a40a08c0 0x7f39a40a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 10 monclient(hunting): picked mon.noname-b con 0x7f39a40a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:46.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a409e0f0 con 0x7f39a40a08c0 2024-09-15T01:43:46.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a409dd80 con 0x7f39a40a2930 2024-09-15T01:43:46.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39a40a2d00 con 0x7f39a409faa0 2024-09-15T01:43:46.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:46.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a946f640 10 monclient(hunting): authenticate will time out at 1431.339844s 2024-09-15T01:43:46.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a27fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 0x7f39a409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a37fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a40a08c0 0x7f39a40a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.588+0000 7f39a2ffd640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39a40a2930 msgr2=0x7f39a4132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:43:46.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a27fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 0x7f39a409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:50126/0 (socket says 172.21.15.178:50126) 2024-09-15T01:43:46.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a27fc640 1 -- 172.21.15.178:0/3000909822 learned_addr learned my addr 172.21.15.178:0/3000909822 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:46.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a27fc640 10 monclient(hunting): get_auth_request con 0x7f39a409faa0 auth_method 0 2024-09-15T01:43:46.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a40a08c0 0x7f39a40a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:41672/0 (socket says 172.21.15.178:41672) 2024-09-15T01:43:46.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a2ffd640 1 --2- 172.21.15.178:0/3000909822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39a40a2930 0x7f39a4132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:46.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 10 monclient(hunting): get_auth_request con 0x7f39a40a08c0 auth_method 0 2024-09-15T01:43:46.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 10 monclient(hunting): handle_auth_done global_id 34796 payload 274 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 1 -- 172.21.15.178:0/3000909822 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 msgr2=0x7f39a409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 1 --2- 172.21.15.178:0/3000909822 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 0x7f39a409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 1 -- 172.21.15.178:0/3000909822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39a40a2930 msgr2=0x7f39a4132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 1 --2- 172.21.15.178:0/3000909822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39a40a2930 0x7f39a4132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 1 monclient: found mon.noname-b 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:46.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 1 -- 172.21.15.178:0/3000909822 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a404bf60 con 0x7f39a40a08c0 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a27fc640 1 --2- 172.21.15.178:0/3000909822 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 0x7f39a409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.589+0000 7f39a37fe640 1 --2- 172.21.15.178:0/3000909822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a40a08c0 0x7f39a40a0d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f399c009e90 tx=0x7f399c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14118e8849036332 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a1ffb640 1 -- 172.21.15.178:0/3000909822 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f399c0079d0 con 0x7f39a40a08c0 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a1ffb640 10 monclient: dump: 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.607 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 5 monclient: authenticate success, global_id 34796 2024-09-15T01:43:46.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a1ffb640 1 -- 172.21.15.178:0/3000909822 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c02d050 con 0x7f39a40a08c0 2024-09-15T01:43:46.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:46.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:46.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a1ffb640 1 -- 172.21.15.178:0/3000909822 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f399c007bd0 con 0x7f39a40a08c0 2024-09-15T01:43:46.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 4 set_mon_vals no callback set 2024-09-15T01:43:46.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:46.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a1ffb640 10 monclient: dump: 2024-09-15T01:43:46.609 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.609 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.609 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.609 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.609 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 10 monclient: shutdown 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:46.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 1 -- 172.21.15.178:0/3000909822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a40a08c0 msgr2=0x7f39a40a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 1 --2- 172.21.15.178:0/3000909822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a40a08c0 0x7f39a40a0d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f399c009e90 tx=0x7f399c009780 comp rx=0 tx=0).stop 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 1 -- 172.21.15.178:0/3000909822 shutdown_connections 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 1 --2- 172.21.15.178:0/3000909822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a40a08c0 0x7f39a40a0d30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 1 --2- 172.21.15.178:0/3000909822 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 0x7f39a409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 1 --2- 172.21.15.178:0/3000909822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39a40a2930 0x7f39a4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 1 -- 172.21.15.178:0/3000909822 >> 172.21.15.178:0/3000909822 conn(0x7f39a40a5ed0 msgr2=0x7f39a40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 1 -- 172.21.15.178:0/3000909822 shutdown_connections 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.590+0000 7f39a946f640 1 -- 172.21.15.178:0/3000909822 wait complete. 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.591+0000 7f39a946f640 10 monclient: build_initial_monmap 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.591+0000 7f39a946f640 10 monclient: monmap: 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:46.611 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:46.612 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.591+0000 7f39a946f640 1 Processor -- start 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.591+0000 7f39a946f640 1 -- start start 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.591+0000 7f39a946f640 10 monclient: init 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient: _add_conns ranks=[4,8,5,6,1,2,0,7,3] 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 0x7f39a409b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient(hunting): picked mon.noname-c con 0x7f39a409faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f39a40a08c0 0x7f39a4131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient(hunting): picked mon.noname-e con 0x7f39a40a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f39a40a2930 0x7f39a4141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient(hunting): picked mon.noname-h con 0x7f39a40a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39a409d630 con 0x7f39a409faa0 2024-09-15T01:43:46.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39a411f000 con 0x7f39a40a2930 2024-09-15T01:43:46.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f39a411ed20 con 0x7f39a40a08c0 2024-09-15T01:43:46.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:46.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient(hunting): authenticate will time out at 1431.343087s 2024-09-15T01:43:46.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 0x7f39a409b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 0x7f39a409b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:50132/0 (socket says 172.21.15.178:50132) 2024-09-15T01:43:46.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 1 -- 172.21.15.178:0/3006102836 learned_addr learned my addr 172.21.15.178:0/3006102836 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:46.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 10 monclient(hunting): get_auth_request con 0x7f39a409faa0 auth_method 0 2024-09-15T01:43:46.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a27fc640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f39a40a08c0 0x7f39a4131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a27fc640 10 monclient(hunting): get_auth_request con 0x7f39a40a08c0 auth_method 0 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 10 monclient(hunting): handle_auth_done global_id 44473 payload 824 2024-09-15T01:43:46.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 1 -- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f39a40a08c0 msgr2=0x7f39a4131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a37fe640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f39a40a2930 0x7f39a4141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f39a40a08c0 0x7f39a4131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 1 -- 172.21.15.178:0/3006102836 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f39a40a2930 msgr2=0x7f39a4141cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f39a40a2930 0x7f39a4141cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:43:46.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 1 monclient: found mon.noname-c 2024-09-15T01:43:46.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:46.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:43:46.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 1 -- 172.21.15.178:0/3006102836 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998009450 con 0x7f39a409faa0 2024-09-15T01:43:46.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a2ffd640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 0x7f39a409b290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f399400b600 tx=0x7f39940091a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3994015070 con 0x7f39a409faa0 2024-09-15T01:43:46.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f398affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f398affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:43:46.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f398affd640 10 monclient: dump: 2024-09-15T01:43:46.617 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.617 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.617 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.617 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.617 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.617 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.617 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.617 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.617 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f398affd640 10 monclient: _finish_auth 0 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f398affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994006150 con 0x7f39a409faa0 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 5 monclient: authenticate success, global_id 44473 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient: _renew_subs 2024-09-15T01:43:46.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 1 -- 172.21.15.178:0/3006102836 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c009a20 con 0x7f39a409faa0 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f398affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3994006740 con 0x7f39a409faa0 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f398affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f398affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f398affd640 10 monclient: dump: 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.619 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.620 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.620 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.620 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.620 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.620 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.620 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.620 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.620 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.620 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient: _renew_subs 2024-09-15T01:43:46.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:46.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a946f640 1 -- 172.21.15.178:0/3006102836 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a411d5d0 con 0x7f39a409faa0 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.592+0000 7f39a17fa640 4 set_mon_vals no callback set 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f39940062c0 con 0x7f39a409faa0 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a946f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a946f640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411c9e0 0x7f39a4142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a946f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a946f640 1 -- 172.21.15.178:0/3006102836 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a411ebe0 con 0x7f39a411c9e0 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f398affd640 1 --2- 172.21.15.178:0/3006102836 >> v2:172.21.15.204:6812/28210 conn(0x7f39900397e0 0x7f399003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a27fc640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411c9e0 0x7f39a4142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a27fc640 10 monclient: get_auth_request con 0x7f39a411c9e0 auth_method 0 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f399404b5e0 con 0x7f39a409faa0 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a27fc640 10 monclient(hunting): handle_auth_done global_id 34805 payload 824 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a37fe640 1 --2- 172.21.15.178:0/3006102836 >> v2:172.21.15.204:6812/28210 conn(0x7f39900397e0 0x7f399003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.593+0000 7f39a27fc640 1 -- 172.21.15.178:0/3006102836 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39a404c4a0 con 0x7f39a411c9e0 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f39a37fe640 10 monclient: get_auth_request con 0x7f39900397e0 auth_method 0 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f39a27fc640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411c9e0 0x7f39a4142210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f39a40a6ec0 tx=0x7f39980091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f399800fd50 con 0x7f39a411c9e0 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f398affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f398affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f398affd640 10 monclient: dump: 2024-09-15T01:43:46.622 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.623 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.624 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f39a37fe640 1 --2- 172.21.15.178:0/3006102836 >> v2:172.21.15.204:6812/28210 conn(0x7f39900397e0 0x7f399003bc90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f399c034000 tx=0x7f399c037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f399800feb0 con 0x7f39a411c9e0 2024-09-15T01:43:46.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f398affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:46.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f398affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:46.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411c9e0 msgr2=0x7f39a4142210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f398affd640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411c9e0 0x7f39a4142210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f39a40a6ec0 tx=0x7f39980091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411c9e0 msgr2=0x7f39a4142210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:46.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.594+0000 7f398affd640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411c9e0 0x7f39a4142210 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f39a40a6ec0 tx=0x7f39980091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a946f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:46.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a946f640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411dae0 0x7f39a4142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a946f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a946f640 1 -- 172.21.15.178:0/3006102836 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a404bc80 con 0x7f39a411dae0 2024-09-15T01:43:46.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a27fc640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411dae0 0x7f39a4142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a27fc640 10 monclient: get_auth_request con 0x7f39a411dae0 auth_method 0 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a27fc640 10 monclient(hunting): handle_auth_done global_id 34814 payload 824 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a27fc640 1 -- 172.21.15.178:0/3006102836 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f39a405ed70 con 0x7f39a411dae0 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f39a27fc640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411dae0 0x7f39a4142750 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3998000c00 tx=0x7f399801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f399801c3e0 con 0x7f39a411dae0 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f398affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f398affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:46.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.600+0000 7f398affd640 10 monclient: dump: 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.628 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.629 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.629 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.629 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9279 (secure 0 0 0) 0x7f399801c540 con 0x7f39a411dae0 2024-09-15T01:43:46.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f398affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:46.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f398affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:46.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411dae0 msgr2=0x7f39a4142750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f398affd640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411dae0 0x7f39a4142750 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3998000c00 tx=0x7f399801d000 comp rx=0 tx=0).stop 2024-09-15T01:43:46.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f398affd640 1 -- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411dae0 msgr2=0x7f39a4142750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:46.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f398affd640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a411dae0 0x7f39a4142750 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3998000c00 tx=0x7f399801d000 comp rx=0 tx=0).stop 2024-09-15T01:43:46.630 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:46.630 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:46.630 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:46.630 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:46.630 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 128, 2024-09-15T01:43:46.630 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:46.630 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:46.630 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:46.630 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:46.630 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:46.630 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:46.630 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:46.631 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:46.632 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:46.632 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.632 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:46.632 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:46.632 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:46.632 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:46.632 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:46.632 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:46.632 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:46.632 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:46.632 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:46.632 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:46.633 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:46.634 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:46.635 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:46.635 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:46.635 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.635 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:46.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.635 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:46.635 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.635 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:46.635 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:46.636 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.636 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:46.636 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.636 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.636 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:46.636 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.636 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.637 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.637 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:46.637 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:46.637 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.637 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.637 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.637 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:46.637 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:46.637 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.638 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.638 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.638 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.638 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:46.638 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.638 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.638 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.638 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:46.638 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.638 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.638 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:46.640 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:46.641 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.642 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.642 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.642 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:46.642 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.642 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.642 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.642 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:46.642 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:46.642 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.642 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.643 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:46.644 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:46.645 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.645 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.645 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.645 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:46.645 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.645 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.645 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.645 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:46.645 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.645 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.646 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.647 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:46.648 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.649 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.650 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:46.652 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.653 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.653 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:46.653 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.653 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:46.653 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:46.653 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:46.653 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.653 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.653 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.653 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:46.653 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:46.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f39a946f640 1 -- 172.21.15.178:0/3006102836 >> v2:172.21.15.204:6812/28210 conn(0x7f39900397e0 msgr2=0x7f399003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f39a946f640 1 --2- 172.21.15.178:0/3006102836 >> v2:172.21.15.204:6812/28210 conn(0x7f39900397e0 0x7f399003bc90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f399c034000 tx=0x7f399c037000 comp rx=0 tx=0).stop 2024-09-15T01:43:46.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f39a946f640 10 monclient: shutdown 2024-09-15T01:43:46.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f39a946f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:46.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f39a946f640 1 -- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 msgr2=0x7f39a409b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f39a946f640 1 --2- 172.21.15.178:0/3006102836 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f39a409faa0 0x7f39a409b290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f399400b600 tx=0x7f39940091a0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f39a37fe640 1 -- 172.21.15.178:0/3006102836 reap_dead start 2024-09-15T01:43:46.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f39a37fe640 1 -- 172.21.15.178:0/3006102836 reap_dead start 2024-09-15T01:43:46.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f39a946f640 1 -- 172.21.15.178:0/3006102836 shutdown_connections 2024-09-15T01:43:46.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f39a946f640 1 -- 172.21.15.178:0/3006102836 >> 172.21.15.178:0/3006102836 conn(0x7f39a40a5ed0 msgr2=0x7f39a4135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:46.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f39a946f640 1 -- 172.21.15.178:0/3006102836 shutdown_connections 2024-09-15T01:43:46.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.601+0000 7f39a946f640 1 -- 172.21.15.178:0/3006102836 wait complete. 2024-09-15T01:43:46.656 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T01:43:46.656 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon c 2024-09-15T01:43:46.656 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:43:46.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.747+0000 7f0c506af640 10 monclient: get_monmap_and_config 2024-09-15T01:43:46.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.747+0000 7f0c506af640 10 monclient: build_initial_monmap 2024-09-15T01:43:46.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.747+0000 7f0c506af640 10 monclient: monmap: 2024-09-15T01:43:46.750 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:46.750 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.750 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:46.749020+0000 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:46.749020+0000 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:46.751 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:46.752 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:46.752 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.747+0000 7f0c506af640 1 Processor -- start 2024-09-15T01:43:46.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.747+0000 7f0c506af640 1 -- start start 2024-09-15T01:43:46.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.747+0000 7f0c506af640 10 monclient: init 2024-09-15T01:43:46.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:46.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 10 monclient: _add_conns ranks=[8,2,1,4,10,3,9,6,0,5,7] 2024-09-15T01:43:46.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 0x7f0c48132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 10 monclient(hunting): picked mon.noname-e con 0x7f0c480a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:46.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c4809faa0 0x7f0c4809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 10 monclient(hunting): picked mon.noname-b con 0x7f0c4809faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:46.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0c480a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:46.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4804bf60 con 0x7f0c480a08c0 2024-09-15T01:43:46.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4809e0f0 con 0x7f0c4809faa0 2024-09-15T01:43:46.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4809dd80 con 0x7f0c480a2930 2024-09-15T01:43:46.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:46.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c506af640 10 monclient(hunting): authenticate will time out at 1431.499453s 2024-09-15T01:43:46.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c4809faa0 0x7f0c4809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c4809faa0 0x7f0c4809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:41704/0 (socket says 172.21.15.178:41704) 2024-09-15T01:43:46.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 1 -- 172.21.15.178:0/990925242 learned_addr learned my addr 172.21.15.178:0/990925242 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:46.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 10 monclient(hunting): get_auth_request con 0x7f0c4809faa0 auth_method 0 2024-09-15T01:43:46.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4e44b640 1 --2- 172.21.15.178:0/990925242 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 0x7f0c48132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4e44b640 10 monclient(hunting): get_auth_request con 0x7f0c480a2930 auth_method 0 2024-09-15T01:43:46.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4e44b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4e44b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4e44b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4e44b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4e44b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4e44b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 10 monclient(hunting): handle_auth_done global_id 34823 payload 274 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 1 -- 172.21.15.178:0/990925242 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 msgr2=0x7f0c48132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 1 --2- 172.21.15.178:0/990925242 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 0x7f0c48132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 1 -- 172.21.15.178:0/990925242 >> v1:172.21.15.204:6789/0 conn(0x7f0c480a08c0 legacy=0x7f0c480a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 10 monclient: _finish_hunting 0 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 1 monclient: found mon.noname-b 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:46.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4dc4a640 1 -- 172.21.15.178:0/990925242 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c480a2d00 con 0x7f0c4809faa0 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4d449640 1 -- 172.21.15.178:0/990925242 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592926640 0 0) 0x7f0c4804bf60 con 0x7f0c480a08c0 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4d449640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.748+0000 7f0c4e44b640 1 --2- 172.21.15.178:0/990925242 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 0x7f0c48132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4dc4a640 1 --2- 172.21.15.178:0/990925242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c4809faa0 0x7f0c4809fe70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009940 tx=0x7f0c380091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25c22a4f45c751dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4d449640 1 -- 172.21.15.178:0/990925242 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c3802ad00 con 0x7f0c4809faa0 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4d449640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4d449640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4d449640 10 monclient: dump: 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.767 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4d449640 10 monclient: _finish_auth 0 2024-09-15T01:43:46.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4d449640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c506af640 5 monclient: authenticate success, global_id 34823 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4d449640 1 -- 172.21.15.178:0/990925242 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3802ae80 con 0x7f0c4809faa0 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c506af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4d449640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c506af640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c506af640 4 set_mon_vals no callback set 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4d449640 1 -- 172.21.15.178:0/990925242 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c38031ca0 con 0x7f0c4809faa0 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4d449640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4d449640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.749+0000 7f0c4d449640 10 monclient: dump: 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.769 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.770 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.750+0000 7f0c506af640 10 monclient: shutdown 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.750+0000 7f0c506af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.750+0000 7f0c506af640 1 -- 172.21.15.178:0/990925242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c4809faa0 msgr2=0x7f0c4809fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.750+0000 7f0c506af640 1 --2- 172.21.15.178:0/990925242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c4809faa0 0x7f0c4809fe70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009940 tx=0x7f0c380091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.750+0000 7f0c506af640 1 -- 172.21.15.178:0/990925242 shutdown_connections 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.750+0000 7f0c506af640 1 --2- 172.21.15.178:0/990925242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c4809faa0 0x7f0c4809fe70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.750+0000 7f0c506af640 1 --2- 172.21.15.178:0/990925242 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 0x7f0c48132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.750+0000 7f0c506af640 1 -- 172.21.15.178:0/990925242 >> 172.21.15.178:0/990925242 conn(0x7f0c480a5ed0 msgr2=0x7f0c480a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.750+0000 7f0c506af640 1 -- 172.21.15.178:0/990925242 shutdown_connections 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.750+0000 7f0c506af640 1 -- 172.21.15.178:0/990925242 wait complete. 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.750+0000 7f0c506af640 10 monclient: build_initial_monmap 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.750+0000 7f0c506af640 10 monclient: monmap: 2024-09-15T01:43:46.771 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:46.772 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 1 Processor -- start 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 1 -- start start 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 10 monclient: init 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 10 monclient: _add_conns ranks=[0,7,8,3,6,2,4,1,5] 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c4809faa0 0x7f0c48141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 10 monclient(hunting): picked mon.noname-a con 0x7f0c4809faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c480a08c0 0x7f0c48131b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 10 monclient(hunting): picked mon.noname-i con 0x7f0c480a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 0x7f0c4811e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 10 monclient(hunting): picked mon.noname-e con 0x7f0c480a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:46.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48142b70 con 0x7f0c4809faa0 2024-09-15T01:43:46.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0c480a2390 con 0x7f0c480a08c0 2024-09-15T01:43:46.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4809d630 con 0x7f0c480a2930 2024-09-15T01:43:46.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4e44b640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c4809faa0 msgr2=0x7f0c48141c50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:43:46.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:46.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c506af640 10 monclient(hunting): authenticate will time out at 1431.502621s 2024-09-15T01:43:46.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4e44b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c4809faa0 0x7f0c48141c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:46.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4dc4a640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c480a08c0 msgr2=0x7f0c48131b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:43:46.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4dc4a640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c480a08c0 0x7f0c48131b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:46.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 0x7f0c4811e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 0x7f0c4811e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40536/0 (socket says 172.21.15.178:40536) 2024-09-15T01:43:46.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 1 -- 172.21.15.178:0/718661559 learned_addr learned my addr 172.21.15.178:0/718661559 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:46.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 10 monclient(hunting): get_auth_request con 0x7f0c480a2930 auth_method 0 2024-09-15T01:43:46.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 10 monclient(hunting): handle_auth_done global_id 34838 payload 824 2024-09-15T01:43:46.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 1 -- 172.21.15.178:0/718661559 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c480a08c0 msgr2=0x7f0c48131b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:43:46.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c480a08c0 0x7f0c48131b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 1 -- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c4809faa0 msgr2=0x7f0c48141c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:43:46.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c4809faa0 0x7f0c48141c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 10 monclient: _finish_hunting 0 2024-09-15T01:43:46.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 1 monclient: found mon.noname-e 2024-09-15T01:43:46.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:46.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:46.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.751+0000 7f0c4ec4c640 1 -- 172.21.15.178:0/718661559 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c44009a20 con 0x7f0c480a2930 2024-09-15T01:43:46.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c4ec4c640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 0x7f0c4811e650 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c510 tx=0x7f0c40007c80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c40007cf0 con 0x7f0c480a2930 2024-09-15T01:43:46.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c3e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c3e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c3e7fc640 10 monclient: dump: 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.777 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.778 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.778 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.778 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.778 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.778 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c3e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:43:46.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c3e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:46.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c40011a10 con 0x7f0c480a2930 2024-09-15T01:43:46.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c506af640 5 monclient: authenticate success, global_id 34838 2024-09-15T01:43:46.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c3e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:46.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c40011bc0 con 0x7f0c480a2930 2024-09-15T01:43:46.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c506af640 10 monclient: _renew_subs 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c506af640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c506af640 1 -- 172.21.15.178:0/718661559 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c38009450 con 0x7f0c480a2930 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c4cc48640 4 set_mon_vals no callback set 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c3e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c3e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c3e7fc640 10 monclient: dump: 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.779 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c506af640 10 monclient: _renew_subs 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c506af640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.752+0000 7f0c506af640 1 -- 172.21.15.178:0/718661559 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c48120140 con 0x7f0c480a2930 2024-09-15T01:43:46.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c506af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c506af640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c48132060 0x7f0c4811eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c506af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c506af640 1 -- 172.21.15.178:0/718661559 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48121e10 con 0x7f0c48132060 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0c40011d40 con 0x7f0c480a2930 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4e44b640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c48132060 0x7f0c4811eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4e44b640 10 monclient: get_auth_request con 0x7f0c48132060 auth_method 0 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4e44b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4e44b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4e44b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 1 --2- 172.21.15.178:0/718661559 >> v2:172.21.15.204:6812/28210 conn(0x7f0c2c0397b0 0x7f0c2c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4e44b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4e44b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4e44b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f0c4004ac40 con 0x7f0c480a2930 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4e44b640 10 monclient(hunting): handle_auth_done global_id 44482 payload 824 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4e44b640 1 -- 172.21.15.178:0/718661559 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c4805ed70 con 0x7f0c48132060 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4e44b640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c48132060 0x7f0c4811eb90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009c70 tx=0x7f0c4400b040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4dc4a640 1 --2- 172.21.15.178:0/718661559 >> v2:172.21.15.204:6812/28210 conn(0x7f0c2c0397b0 0x7f0c2c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c4dc4a640 10 monclient: get_auth_request con 0x7f0c2c0397b0 auth_method 0 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c4400ce90 con 0x7f0c48132060 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 10 monclient: dump: 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.782 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f0c4400f040 con 0x7f0c48132060 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c48132060 msgr2=0x7f0c4811eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c48132060 0x7f0c4811eb90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009c70 tx=0x7f0c4400b040 comp rx=0 tx=0).stop 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c48132060 msgr2=0x7f0c4811eb90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.753+0000 7f0c3e7fc640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c48132060 0x7f0c4811eb90 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009c70 tx=0x7f0c4400b040 comp rx=0 tx=0).stop 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.754+0000 7f0c4dc4a640 1 --2- 172.21.15.178:0/718661559 >> v2:172.21.15.204:6812/28210 conn(0x7f0c2c0397b0 0x7f0c2c03bc60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0c38034000 tx=0x7f0c38032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.759+0000 7f0c506af640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.759+0000 7f0c506af640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c4811f0d0 0x7f0c48099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.759+0000 7f0c506af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.759+0000 7f0c506af640 1 -- 172.21.15.178:0/718661559 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48132430 con 0x7f0c4811f0d0 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.759+0000 7f0c4e44b640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c4811f0d0 0x7f0c48099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.759+0000 7f0c4e44b640 10 monclient: get_auth_request con 0x7f0c4811f0d0 auth_method 0 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.759+0000 7f0c4e44b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.759+0000 7f0c4e44b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.759+0000 7f0c4e44b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.759+0000 7f0c4e44b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.759+0000 7f0c4e44b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.759+0000 7f0c4e44b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c4e44b640 10 monclient(hunting): handle_auth_done global_id 44491 payload 824 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c4e44b640 1 -- 172.21.15.178:0/718661559 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0c4804bc80 con 0x7f0c4811f0d0 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c4e44b640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c4811f0d0 0x7f0c48099a50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009e70 tx=0x7f0c44019000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c4401dbe0 con 0x7f0c4811f0d0 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c3e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c3e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:46.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c3e7fc640 10 monclient: dump: 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.786 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.787 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.787 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.787 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9449 (secure 0 0 0) 0x7f0c4401dd40 con 0x7f0c4811f0d0 2024-09-15T01:43:46.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c3e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:46.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c3e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:46.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c4811f0d0 msgr2=0x7f0c48099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c3e7fc640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c4811f0d0 0x7f0c48099a50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009e70 tx=0x7f0c44019000 comp rx=0 tx=0).stop 2024-09-15T01:43:46.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c3e7fc640 1 -- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c4811f0d0 msgr2=0x7f0c48099a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:46.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c3e7fc640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c4811f0d0 0x7f0c48099a50 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009e70 tx=0x7f0c44019000 comp rx=0 tx=0).stop 2024-09-15T01:43:46.788 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:46.788 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:46.788 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:46.788 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:46.788 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 128, 2024-09-15T01:43:46.788 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:46.788 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:46.788 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:46.788 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:46.788 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:46.788 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:46.788 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:46.789 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:46.790 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:46.791 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:46.792 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:46.792 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:46.792 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:46.792 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:46.792 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:46.792 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:46.792 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:46.792 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:46.792 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:46.792 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:46.792 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:46.792 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:46.794 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:46.795 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.795 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.795 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.795 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:46.795 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:46.795 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.795 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.795 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.796 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:46.796 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.796 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.796 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:46.796 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.796 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.796 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:46.798 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:46.799 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.800 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.801 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:46.801 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:46.801 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.801 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:46.801 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.803 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:46.803 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:46.803 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.803 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.803 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.803 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:46.803 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.804 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.804 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:46.804 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.804 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.804 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.804 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:46.804 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:46.804 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.804 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.804 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.805 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:46.806 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.808 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.809 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.810 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.810 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.810 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.810 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:46.810 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:46.810 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.810 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:46.810 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:46.810 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:46.810 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.810 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:46.811 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.812 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.812 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:43:46.812 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.812 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:46.812 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:46.812 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:46.812 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.812 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.812 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:46.812 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:46.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c506af640 1 -- 172.21.15.178:0/718661559 >> v2:172.21.15.204:6812/28210 conn(0x7f0c2c0397b0 msgr2=0x7f0c2c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c506af640 1 --2- 172.21.15.178:0/718661559 >> v2:172.21.15.204:6812/28210 conn(0x7f0c2c0397b0 0x7f0c2c03bc60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0c38034000 tx=0x7f0c38032000 comp rx=0 tx=0).stop 2024-09-15T01:43:46.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c506af640 10 monclient: shutdown 2024-09-15T01:43:46.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c506af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:46.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c506af640 1 -- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 msgr2=0x7f0c4811e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c506af640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 0x7f0c4811e650 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c510 tx=0x7f0c40007c80 comp rx=0 tx=0).stop 2024-09-15T01:43:46.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.760+0000 7f0c4ec4c640 1 -- 172.21.15.178:0/718661559 reap_dead start 2024-09-15T01:43:46.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.761+0000 7f0c506af640 1 -- 172.21.15.178:0/718661559 shutdown_connections 2024-09-15T01:43:46.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.761+0000 7f0c506af640 1 --2- 172.21.15.178:0/718661559 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0c480a2930 0x7f0c4811e650 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.761+0000 7f0c506af640 1 -- 172.21.15.178:0/718661559 >> 172.21.15.178:0/718661559 conn(0x7f0c480a5ed0 msgr2=0x7f0c4809e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:46.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.761+0000 7f0c506af640 1 -- 172.21.15.178:0/718661559 shutdown_connections 2024-09-15T01:43:46.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.761+0000 7f0c506af640 1 -- 172.21.15.178:0/718661559 wait complete. 2024-09-15T01:43:46.815 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T01:43:46.815 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-09-15T01:43:46.815 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:43:46.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.905+0000 7fccd6089640 10 monclient: get_monmap_and_config 2024-09-15T01:43:46.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.905+0000 7fccd6089640 10 monclient: build_initial_monmap 2024-09-15T01:43:46.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.906+0000 7fccd6089640 10 monclient: monmap: 2024-09-15T01:43:46.909 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:46.909 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.909 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:46.907885+0000 2024-09-15T01:43:46.909 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:46.907885+0000 2024-09-15T01:43:46.909 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:46.909 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.909 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:46.910 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:46.910 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:46.910 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:46.910 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:46.910 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:46.910 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:46.910 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:46.910 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:46.910 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:46.910 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:46.910 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.906+0000 7fccd6089640 1 Processor -- start 2024-09-15T01:43:46.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.906+0000 7fccd6089640 1 -- start start 2024-09-15T01:43:46.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.906+0000 7fccd6089640 10 monclient: init 2024-09-15T01:43:46.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:46.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 10 monclient: _add_conns ranks=[7,5,6,4,0,10,9,1,2,8,3] 2024-09-15T01:43:46.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fccd00a2930 0x7fccd0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 10 monclient(hunting): picked mon.noname-i con 0x7fccd00a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:46.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fccd009faa0 0x7fccd009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 10 monclient(hunting): picked mon.noname-h con 0x7fccd009faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:46.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd00a08c0 0x7fccd00a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 10 monclient(hunting): picked mon.noname-d con 0x7fccd00a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccd009e0f0 con 0x7fccd009faa0 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fccd009dd80 con 0x7fccd00a08c0 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00a2d00 con 0x7fccd00a2930 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccd6089640 10 monclient(hunting): authenticate will time out at 1431.658438s 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccf7fe640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fccd00a2930 msgr2=0x7fccd0132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd00a08c0 0x7fccd00a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccf7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fccd00a2930 0x7fccd0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:46.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd00a08c0 0x7fccd00a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:55836/0 (socket says 172.21.15.178:55836) 2024-09-15T01:43:46.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 1 -- 172.21.15.178:0/441583051 learned_addr learned my addr 172.21.15.178:0/441583051 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:46.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 10 monclient(hunting): get_auth_request con 0x7fccd00a08c0 auth_method 0 2024-09-15T01:43:46.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccceffd640 1 --2- 172.21.15.178:0/441583051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fccd009faa0 0x7fccd009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccceffd640 10 monclient(hunting): get_auth_request con 0x7fccd009faa0 auth_method 0 2024-09-15T01:43:46.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccceffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fccceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 10 monclient(hunting): handle_auth_done global_id 25323 payload 274 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 1 -- 172.21.15.178:0/441583051 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fccd00a2930 msgr2=0x7fccd0132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 1 --2- 172.21.15.178:0/441583051 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fccd00a2930 0x7fccd0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 1 -- 172.21.15.178:0/441583051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fccd009faa0 msgr2=0x7fccd009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 1 --2- 172.21.15.178:0/441583051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fccd009faa0 0x7fccd009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 10 monclient: _finish_hunting 0 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 1 monclient: found mon.noname-d 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:46.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:43:46.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.907+0000 7fcccffff640 1 -- 172.21.15.178:0/441583051 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd004bf60 con 0x7fccd00a08c0 2024-09-15T01:43:46.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fcccffff640 1 --2- 172.21.15.178:0/441583051 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd00a08c0 0x7fccd00a0d30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009940 tx=0x7fccc80091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=6b54704f3be29a2c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccce7fc640 1 -- 172.21.15.178:0/441583051 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccc802ac80 con 0x7fccd00a08c0 2024-09-15T01:43:46.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccce7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:43:46.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccce7fc640 10 monclient: dump: 2024-09-15T01:43:46.924 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.924 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.924 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.924 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.924 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.924 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccce7fc640 10 monclient: _finish_auth 0 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:46.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccce7fc640 1 -- 172.21.15.178:0/441583051 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc802ae00 con 0x7fccd00a08c0 2024-09-15T01:43:46.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccd6089640 5 monclient: authenticate success, global_id 25323 2024-09-15T01:43:46.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccd6089640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:46.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccce7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:46.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccd6089640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:46.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccd6089640 4 set_mon_vals no callback set 2024-09-15T01:43:46.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccce7fc640 1 -- 172.21.15.178:0/441583051 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccc8031ca0 con 0x7fccd00a08c0 2024-09-15T01:43:46.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccce7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:46.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.908+0000 7fccce7fc640 10 monclient: dump: 2024-09-15T01:43:46.926 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.926 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.926 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 10 monclient: shutdown 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 1 -- 172.21.15.178:0/441583051 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd00a08c0 msgr2=0x7fccd00a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 1 --2- 172.21.15.178:0/441583051 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd00a08c0 0x7fccd00a0d30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009940 tx=0x7fccc80091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 1 -- 172.21.15.178:0/441583051 shutdown_connections 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 1 --2- 172.21.15.178:0/441583051 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd00a08c0 0x7fccd00a0d30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 1 --2- 172.21.15.178:0/441583051 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fccd009faa0 0x7fccd009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 1 --2- 172.21.15.178:0/441583051 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fccd00a2930 0x7fccd0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 1 -- 172.21.15.178:0/441583051 >> 172.21.15.178:0/441583051 conn(0x7fccd00a5ed0 msgr2=0x7fccd00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 1 -- 172.21.15.178:0/441583051 shutdown_connections 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 1 -- 172.21.15.178:0/441583051 wait complete. 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 10 monclient: build_initial_monmap 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.909+0000 7fccd6089640 10 monclient: monmap: 2024-09-15T01:43:46.928 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:46.929 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 1 Processor -- start 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 1 -- start start 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 10 monclient: init 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 10 monclient: _add_conns ranks=[2,4,7,5,0,6,3,1,8] 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccd009faa0 0x7fccd009aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 10 monclient(hunting): picked mon.noname-b con 0x7fccd009faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccd00a08c0 0x7fccd01317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 10 monclient(hunting): picked mon.noname-c con 0x7fccd00a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fccd00a2930 0x7fccd0131d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 10 monclient(hunting): picked mon.noname-i con 0x7fccd00a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:46.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd011ebc0 con 0x7fccd009faa0 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00a2390 con 0x7fccd00a08c0 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fccd009d630 con 0x7fccd00a2930 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccd6089640 10 monclient(hunting): authenticate will time out at 1431.661817s 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fcccf7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccd009faa0 0x7fccd009aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fcccffff640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fccd00a2930 msgr2=0x7fccd0131d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccceffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccd00a08c0 0x7fccd01317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fcccffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fccd00a2930 0x7fccd0131d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fcccf7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccd009faa0 0x7fccd009aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:41712/0 (socket says 172.21.15.178:41712) 2024-09-15T01:43:46.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fcccf7fe640 1 -- 172.21.15.178:0/2084492734 learned_addr learned my addr 172.21.15.178:0/2084492734 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fcccf7fe640 10 monclient(hunting): get_auth_request con 0x7fccd009faa0 auth_method 0 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fcccf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fcccf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fcccf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccceffd640 10 monclient(hunting): get_auth_request con 0x7fccd00a08c0 auth_method 0 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccceffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.910+0000 7fccceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fcccf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fcccf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fcccf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 10 monclient(hunting): handle_auth_done global_id 44500 payload 824 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 1 -- 172.21.15.178:0/2084492734 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fccd00a2930 msgr2=0x7fccd0131d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fccd00a2930 0x7fccd0131d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 1 -- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccd009faa0 msgr2=0x7fccd009aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccd009faa0 0x7fccd009aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 10 monclient: _finish_hunting 0 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 1 monclient: found mon.noname-c 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 1 -- 172.21.15.178:0/2084492734 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc8009450 con 0x7fccd00a08c0 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fcccf7fe640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccd009faa0 0x7fccd009aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:46.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccceffd640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccd00a08c0 0x7fccd01317c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fccc400e900 tx=0x7fccc40090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccc40093e0 con 0x7fccd00a08c0 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccb77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccb77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccb77fe640 10 monclient: dump: 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.934 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccb77fe640 10 monclient: _finish_auth 0 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccb77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc400cdc0 con 0x7fccd00a08c0 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccd6089640 5 monclient: authenticate success, global_id 44500 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccb77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:46.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccd6089640 10 monclient: _renew_subs 2024-09-15T01:43:46.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccc40138f0 con 0x7fccd00a08c0 2024-09-15T01:43:46.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccd6089640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:46.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccd6089640 1 -- 172.21.15.178:0/2084492734 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd009ba70 con 0x7fccd00a08c0 2024-09-15T01:43:46.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccb77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccb77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:46.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccb77fe640 10 monclient: dump: 2024-09-15T01:43:46.936 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.936 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.937 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fcccdffb640 4 set_mon_vals no callback set 2024-09-15T01:43:46.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccd6089640 10 monclient: _renew_subs 2024-09-15T01:43:46.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccd6089640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:43:46.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.911+0000 7fccd6089640 1 -- 172.21.15.178:0/2084492734 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd0142980 con 0x7fccd00a08c0 2024-09-15T01:43:46.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fccc4013470 con 0x7fccd00a08c0 2024-09-15T01:43:46.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fccd6089640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:46.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fccd6089640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011c530 0x7fccd011c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fccd6089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fccd6089640 1 -- 172.21.15.178:0/2084492734 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fccd011ea80 con 0x7fccd011c530 2024-09-15T01:43:46.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fccb77fe640 1 --2- 172.21.15.178:0/2084492734 >> v2:172.21.15.204:6812/28210 conn(0x7fccbc0397b0 0x7fccbc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fcccf7fe640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011c530 0x7fccd011c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fcccf7fe640 10 monclient: get_auth_request con 0x7fccd011c530 auth_method 0 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fccc404bce0 con 0x7fccd00a08c0 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fcccf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fcccf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fcccf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fcccf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fcccf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.912+0000 7fcccf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fcccf7fe640 10 monclient(hunting): handle_auth_done global_id 25332 payload 824 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fcccf7fe640 1 -- 172.21.15.178:0/2084492734 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccd004c4a0 con 0x7fccd011c530 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fcccffff640 1 --2- 172.21.15.178:0/2084492734 >> v2:172.21.15.204:6812/28210 conn(0x7fccbc0397b0 0x7fccbc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fcccffff640 10 monclient: get_auth_request con 0x7fccbc0397b0 auth_method 0 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fcccf7fe640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011c530 0x7fccd011c900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e4f0 tx=0x7fccc00097f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccc0007a40 con 0x7fccd011c530 2024-09-15T01:43:46.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fccb77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fccb77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:46.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fccb77fe640 10 monclient: dump: 2024-09-15T01:43:46.940 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.940 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.940 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.940 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.940 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.940 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.940 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.940 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.940 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.940 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.941 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.941 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.941 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.941 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.941 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.941 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fcccffff640 1 --2- 172.21.15.178:0/2084492734 >> v2:172.21.15.204:6812/28210 conn(0x7fccbc0397b0 0x7fccbc03bc60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fccc8034000 tx=0x7fccc8032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fccc000f040 con 0x7fccd011c530 2024-09-15T01:43:46.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fccb77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:46.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fccb77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:46.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011c530 msgr2=0x7fccd011c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fccb77fe640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011c530 0x7fccd011c900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e4f0 tx=0x7fccc00097f0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011c530 msgr2=0x7fccd011c900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:46.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.913+0000 7fccb77fe640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011c530 0x7fccd011c900 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e4f0 tx=0x7fccc00097f0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fccd6089640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:46.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fccd6089640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011ce40 0x7fccd0096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:46.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fccd6089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:46.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fccd6089640 1 -- 172.21.15.178:0/2084492734 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fccd004bc80 con 0x7fccd011ce40 2024-09-15T01:43:46.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fcccf7fe640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011ce40 0x7fccd0096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:46.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fcccf7fe640 10 monclient: get_auth_request con 0x7fccd011ce40 auth_method 0 2024-09-15T01:43:46.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fcccf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:46.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fcccf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fcccf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fcccf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fcccf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fcccf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fcccf7fe640 10 monclient(hunting): handle_auth_done global_id 25341 payload 824 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fcccf7fe640 1 -- 172.21.15.178:0/2084492734 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fccd005ed70 con 0x7fccd011ce40 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fcccf7fe640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011ce40 0x7fccd0096a30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000f80 tx=0x7fccc0015000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccc0013e60 con 0x7fccd011ce40 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fccb77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fccb77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.919+0000 7fccb77fe640 10 monclient: dump: 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:46.945 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:46.946 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:46.947 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:46.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9109 (secure 0 0 0) 0x7fccc002b460 con 0x7fccd011ce40 2024-09-15T01:43:46.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccb77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:46.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccb77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:46.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011ce40 msgr2=0x7fccd0096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccb77fe640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011ce40 0x7fccd0096a30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000f80 tx=0x7fccc0015000 comp rx=0 tx=0).stop 2024-09-15T01:43:46.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccb77fe640 1 -- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011ce40 msgr2=0x7fccd0096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:46.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccb77fe640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fccd011ce40 0x7fccd0096a30 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000f80 tx=0x7fccc0015000 comp rx=0 tx=0).stop 2024-09-15T01:43:46.947 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:46.947 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 128, 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:43:46.948 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:46.949 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:46.949 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:46.949 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:46.949 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:46.949 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:46.949 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:46.949 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:46.949 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:46.949 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:46.949 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:46.949 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:46.949 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.950 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:46.951 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:46.952 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.953 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:46.953 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.953 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:46.953 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.953 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:46.953 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:46.953 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.953 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.953 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.955 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.955 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.955 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.955 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:46.955 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:46.955 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.955 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.955 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.955 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.955 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:46.955 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.956 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.956 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.956 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:46.956 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.956 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.956 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.956 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:46.956 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:46.956 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.956 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.958 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.959 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:46.959 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:46.959 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.959 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:46.959 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.959 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.960 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:46.960 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.960 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.960 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.960 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:46.960 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:46.960 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.960 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.960 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.960 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:46.961 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.962 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.962 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.962 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:46.962 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:46.962 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.962 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.962 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.962 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:46.962 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.963 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:46.965 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:46.966 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:46.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.969 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:46.970 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:46.971 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:46.971 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:46.971 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:46.971 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:46.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccd6089640 1 -- 172.21.15.178:0/2084492734 >> v2:172.21.15.204:6812/28210 conn(0x7fccbc0397b0 msgr2=0x7fccbc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccd6089640 1 --2- 172.21.15.178:0/2084492734 >> v2:172.21.15.204:6812/28210 conn(0x7fccbc0397b0 0x7fccbc03bc60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fccc8034000 tx=0x7fccc8032000 comp rx=0 tx=0).stop 2024-09-15T01:43:46.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccd6089640 10 monclient: shutdown 2024-09-15T01:43:46.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccd6089640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:46.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccd6089640 1 -- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccd00a08c0 msgr2=0x7fccd01317c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:46.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccd6089640 1 --2- 172.21.15.178:0/2084492734 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fccd00a08c0 0x7fccd01317c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fccc400e900 tx=0x7fccc40090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:46.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fcccffff640 1 -- 172.21.15.178:0/2084492734 reap_dead start 2024-09-15T01:43:46.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fcccffff640 1 -- 172.21.15.178:0/2084492734 reap_dead start 2024-09-15T01:43:46.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccd6089640 1 -- 172.21.15.178:0/2084492734 shutdown_connections 2024-09-15T01:43:46.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccd6089640 1 -- 172.21.15.178:0/2084492734 >> 172.21.15.178:0/2084492734 conn(0x7fccd00a5ed0 msgr2=0x7fccd0135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:46.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccd6089640 1 -- 172.21.15.178:0/2084492734 shutdown_connections 2024-09-15T01:43:46.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:46.920+0000 7fccd6089640 1 -- 172.21.15.178:0/2084492734 wait complete. 2024-09-15T01:43:46.973 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T01:43:46.973 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-09-15T01:43:46.974 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:43:47.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.064+0000 7fcc2f11d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:47.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.064+0000 7fcc2f11d640 10 monclient: build_initial_monmap 2024-09-15T01:43:47.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.065+0000 7fcc2f11d640 10 monclient: monmap: 2024-09-15T01:43:47.068 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:47.068 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.068 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:47.066855+0000 2024-09-15T01:43:47.068 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:47.066855+0000 2024-09-15T01:43:47.068 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:47.068 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.068 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:47.069 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:47.069 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:47.069 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:47.069 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:47.069 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:47.069 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:47.069 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:47.069 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:47.069 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:47.069 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:47.069 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.065+0000 7fcc2f11d640 1 Processor -- start 2024-09-15T01:43:47.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.065+0000 7fcc2f11d640 1 -- start start 2024-09-15T01:43:47.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.065+0000 7fcc2f11d640 10 monclient: init 2024-09-15T01:43:47.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:47.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 10 monclient: _add_conns ranks=[2,1,4,3,6,10,9,5,8,0,7] 2024-09-15T01:43:47.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 0x7fcc28132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 10 monclient(hunting): picked mon.noname-b con 0x7fcc280a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:47.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fcc2809faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:47.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcc2809fe70 0x7fcc280a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 10 monclient(hunting): picked mon.noname-c con 0x7fcc2809fe70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:47.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2804bf60 con 0x7fcc2809faa0 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2809dd80 con 0x7fcc280a2930 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280a2d00 con 0x7fcc2809fe70 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2f11d640 10 monclient(hunting): authenticate will time out at 1431.817230s 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2ceb9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 0x7fcc28132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcc2809fe70 0x7fcc280a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2ceb9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 0x7fcc28132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:41720/0 (socket says 172.21.15.178:41720) 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2ceb9640 1 -- 172.21.15.178:0/3522083969 learned_addr learned my addr 172.21.15.178:0/3522083969 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 10 monclient(hunting): get_auth_request con 0x7fcc2809fe70 auth_method 0 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2ceb9640 10 monclient(hunting): get_auth_request con 0x7fcc280a2930 auth_method 0 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2ceb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2ceb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2ceb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 10 monclient(hunting): handle_auth_done global_id 44509 payload 274 2024-09-15T01:43:47.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 1 -- 172.21.15.178:0/3522083969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 msgr2=0x7fcc28132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 1 --2- 172.21.15.178:0/3522083969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 0x7fcc28132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 1 -- 172.21.15.178:0/3522083969 >> v1:172.21.15.204:6789/0 conn(0x7fcc2809faa0 legacy=0x7fcc280a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 10 monclient: _finish_hunting 0 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 1 monclient: found mon.noname-c 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2d6ba640 1 -- 172.21.15.178:0/3522083969 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc280a0310 con 0x7fcc2809fe70 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.066+0000 7fcc2ceb9640 1 --2- 172.21.15.178:0/3522083969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 0x7fcc28132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc2d6ba640 1 --2- 172.21.15.178:0/3522083969 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcc2809fe70 0x7fcc280a0fd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009880 tx=0x7fcc100090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=9efdadcc4546513e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc277fe640 1 -- 172.21.15.178:0/3522083969 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc1002abc0 con 0x7fcc2809fe70 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc277fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc277fe640 10 monclient: dump: 2024-09-15T01:43:47.082 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.083 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc277fe640 10 monclient: _finish_auth 0 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc277fe640 1 -- 172.21.15.178:0/3522083969 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1002ad40 con 0x7fcc2809fe70 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc277fe640 1 -- 172.21.15.178:0/3522083969 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc10031ca0 con 0x7fcc2809fe70 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc2f11d640 5 monclient: authenticate success, global_id 44509 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc2f11d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc2f11d640 4 set_mon_vals no callback set 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc277fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.067+0000 7fcc277fe640 10 monclient: dump: 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.084 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 10 monclient: shutdown 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 1 -- 172.21.15.178:0/3522083969 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcc2809fe70 msgr2=0x7fcc280a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 1 --2- 172.21.15.178:0/3522083969 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcc2809fe70 0x7fcc280a0fd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009880 tx=0x7fcc100090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 1 -- 172.21.15.178:0/3522083969 shutdown_connections 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 1 --2- 172.21.15.178:0/3522083969 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fcc2809fe70 0x7fcc280a0fd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 1 --2- 172.21.15.178:0/3522083969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 0x7fcc28132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 1 -- 172.21.15.178:0/3522083969 >> 172.21.15.178:0/3522083969 conn(0x7fcc280a5ed0 msgr2=0x7fcc280a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 1 -- 172.21.15.178:0/3522083969 shutdown_connections 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 1 -- 172.21.15.178:0/3522083969 wait complete. 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 10 monclient: build_initial_monmap 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 10 monclient: monmap: 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:47.086 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.068+0000 7fcc2f11d640 1 Processor -- start 2024-09-15T01:43:47.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 1 -- start start 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 10 monclient: init 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 10 monclient: _add_conns ranks=[5,8,2,4,6,3,0,7,1] 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc2809faa0 0x7fcc28137170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 10 monclient(hunting): picked mon.noname-h con 0x7fcc2809faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc2809fe70 0x7fcc2809b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 10 monclient(hunting): picked mon.noname-e con 0x7fcc2809fe70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 0x7fcc28131780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 10 monclient(hunting): picked mon.noname-b con 0x7fcc280a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2811ee60 con 0x7fcc280a2930 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2811eb80 con 0x7fcc2809faa0 2024-09-15T01:43:47.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2811f140 con 0x7fcc2809fe70 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2f11d640 10 monclient(hunting): authenticate will time out at 1431.820596s 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 0x7fcc28131780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc2809fe70 0x7fcc2809b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 0x7fcc28131780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:41728/0 (socket says 172.21.15.178:41728) 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 1 -- 172.21.15.178:0/4090890139 learned_addr learned my addr 172.21.15.178:0/4090890139 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 10 monclient(hunting): get_auth_request con 0x7fcc280a2930 auth_method 0 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc27fff640 10 monclient(hunting): get_auth_request con 0x7fcc2809fe70 auth_method 0 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc27fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2ceb9640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc2809faa0 0x7fcc28137170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2ceb9640 10 monclient(hunting): get_auth_request con 0x7fcc2809faa0 auth_method 0 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2ceb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2ceb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2ceb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 10 monclient(hunting): handle_auth_done global_id 34850 payload 824 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 1 -- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc2809fe70 msgr2=0x7fcc2809b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc2809fe70 0x7fcc2809b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.069+0000 7fcc2d6ba640 1 -- 172.21.15.178:0/4090890139 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc2809faa0 msgr2=0x7fcc28137170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2d6ba640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fcc2809faa0 0x7fcc28137170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2d6ba640 10 monclient: _finish_hunting 0 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2d6ba640 1 monclient: found mon.noname-b 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2d6ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2d6ba640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2d6ba640 1 -- 172.21.15.178:0/4090890139 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc20009450 con 0x7fcc280a2930 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc27fff640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc2809fe70 0x7fcc2809b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:47.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2d6ba640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 0x7fcc28131780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcc10035040 tx=0x7fcc10032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc1003b870 con 0x7fcc280a2930 2024-09-15T01:43:47.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc24ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc24ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:43:47.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc24ff9640 10 monclient: dump: 2024-09-15T01:43:47.092 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.092 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.092 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.092 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.092 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.092 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.092 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.092 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc24ff9640 10 monclient: _finish_auth 0 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc24ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1003ba10 con 0x7fcc280a2930 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2f11d640 5 monclient: authenticate success, global_id 34850 2024-09-15T01:43:47.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2f11d640 10 monclient: _renew_subs 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2f11d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2f11d640 1 -- 172.21.15.178:0/4090890139 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc10009390 con 0x7fcc280a2930 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc24ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc100313a0 con 0x7fcc280a2930 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc24ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc24ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc24ff9640 10 monclient: dump: 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.094 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2f11d640 10 monclient: _renew_subs 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc26ffd640 4 set_mon_vals no callback set 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2f11d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.070+0000 7fcc2f11d640 1 -- 172.21.15.178:0/4090890139 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc28131cc0 con 0x7fcc280a2930 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fcc10039040 con 0x7fcc280a2930 2024-09-15T01:43:47.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2f11d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2f11d640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28096b10 0x7fcc28096ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2f11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2f11d640 1 -- 172.21.15.178:0/4090890139 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2811ecf0 con 0x7fcc28096b10 2024-09-15T01:43:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2ceb9640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28096b10 0x7fcc28096ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2ceb9640 10 monclient: get_auth_request con 0x7fcc28096b10 auth_method 0 2024-09-15T01:43:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2ceb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2ceb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2ceb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc24ff9640 1 --2- 172.21.15.178:0/4090890139 >> v2:172.21.15.204:6812/28210 conn(0x7fcc080397b0 0x7fcc0803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fcc1006b880 con 0x7fcc280a2930 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_done global_id 34856 payload 824 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2ceb9640 1 -- 172.21.15.178:0/4090890139 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc2804c4a0 con 0x7fcc28096b10 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc27fff640 1 --2- 172.21.15.178:0/4090890139 >> v2:172.21.15.204:6812/28210 conn(0x7fcc080397b0 0x7fcc0803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc27fff640 10 monclient: get_auth_request con 0x7fcc080397b0 auth_method 0 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc2ceb9640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28096b10 0x7fcc28096ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcc20011bf0 tx=0x7fcc200091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc2000fdb0 con 0x7fcc28096b10 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc24ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc24ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.071+0000 7fcc24ff9640 10 monclient: dump: 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.097 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.072+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fcc2001d480 con 0x7fcc28096b10 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.072+0000 7fcc24ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:47.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.072+0000 7fcc24ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:47.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.072+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28096b10 msgr2=0x7fcc28096ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.072+0000 7fcc24ff9640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28096b10 0x7fcc28096ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcc20011bf0 tx=0x7fcc200091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.072+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28096b10 msgr2=0x7fcc28096ee0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:47.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.072+0000 7fcc24ff9640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28096b10 0x7fcc28096ee0 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcc20011bf0 tx=0x7fcc200091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.072+0000 7fcc27fff640 1 --2- 172.21.15.178:0/4090890139 >> v2:172.21.15.204:6812/28210 conn(0x7fcc080397b0 0x7fcc0803bc60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcc280a6ec0 tx=0x7fcc18009760 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.077+0000 7fcc2f11d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:47.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.077+0000 7fcc2f11d640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28097420 0x7fcc2811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.077+0000 7fcc2f11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.077+0000 7fcc2f11d640 1 -- 172.21.15.178:0/4090890139 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2804bc80 con 0x7fcc28097420 2024-09-15T01:43:47.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.077+0000 7fcc2ceb9640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28097420 0x7fcc2811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.077+0000 7fcc2ceb9640 10 monclient: get_auth_request con 0x7fcc28097420 auth_method 0 2024-09-15T01:43:47.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.077+0000 7fcc2ceb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.077+0000 7fcc2ceb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.077+0000 7fcc2ceb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.078+0000 7fcc2ceb9640 1 -- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28097420 msgr2=0x7fcc2811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:43:47.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.078+0000 7fcc2ceb9640 1 -- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28097420 msgr2=0x7fcc2811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:43:47.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.078+0000 7fcc2ceb9640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28097420 0x7fcc2811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:43:47.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.078+0000 7fcc2ceb9640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28097420 0x7fcc2811c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:47.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc2ceb9640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28097420 0x7fcc2811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc2ceb9640 10 monclient: get_auth_request con 0x7fcc28097420 auth_method 0 2024-09-15T01:43:47.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc2ceb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc2ceb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc2ceb9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:43:47.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc2ceb9640 10 monclient(hunting): handle_auth_done global_id 34874 payload 824 2024-09-15T01:43:47.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc2ceb9640 1 -- 172.21.15.178:0/4090890139 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcc2805ed70 con 0x7fcc28097420 2024-09-15T01:43:47.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc2ceb9640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28097420 0x7fcc2811c530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcc20011bf0 tx=0x7fcc2001d000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc20025040 con 0x7fcc28097420 2024-09-15T01:43:47.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc24ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc24ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:47.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.279+0000 7fcc24ff9640 10 monclient: dump: 2024-09-15T01:43:47.284 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.284 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.284 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.284 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9279 (secure 0 0 0) 0x7fcc2001c3e0 con 0x7fcc28097420 2024-09-15T01:43:47.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc24ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:47.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc24ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:47.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28097420 msgr2=0x7fcc2811c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc24ff9640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28097420 0x7fcc2811c530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcc20011bf0 tx=0x7fcc2001d000 comp rx=0 tx=0).stop 2024-09-15T01:43:47.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc24ff9640 1 -- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28097420 msgr2=0x7fcc2811c530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:47.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc24ff9640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fcc28097420 0x7fcc2811c530 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcc20011bf0 tx=0x7fcc2001d000 comp rx=0 tx=0).stop 2024-09-15T01:43:47.286 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:47.286 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:47.286 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:47.286 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:47.287 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 128, 2024-09-15T01:43:47.287 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:47.287 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:47.287 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:47.287 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:47.287 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:47.287 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:47.287 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:47.287 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.287 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:43:47.287 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:47.287 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:47.288 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:47.288 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:47.288 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:47.288 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:47.288 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:47.288 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:47.288 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:47.288 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:47.288 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:47.288 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:47.288 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.289 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:47.289 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:47.289 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:47.289 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:47.289 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:47.289 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:47.289 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:47.289 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:47.289 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:47.289 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:47.289 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:47.289 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.290 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:47.290 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:47.290 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:47.290 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:47.290 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:47.290 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:47.290 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:47.290 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:47.290 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:47.291 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.292 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.293 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.293 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.293 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:47.293 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.293 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.293 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.293 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:47.293 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.293 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.293 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.294 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.295 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.296 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.296 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.296 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:47.296 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:47.296 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.296 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.296 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.296 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.296 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:47.296 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.297 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.298 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:47.299 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.300 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.301 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:47.301 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:47.301 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.301 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.301 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.301 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.301 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:47.301 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:47.301 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.301 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.302 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.302 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.302 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:47.302 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.302 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.302 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.302 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:47.302 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.302 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.302 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.303 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:47.303 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:47.303 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.303 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.303 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.303 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.303 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:47.303 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:47.303 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.303 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.304 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.304 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.304 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:47.304 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.304 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.304 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.304 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:47.304 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.304 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.304 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.305 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:47.305 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:47.305 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.305 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.305 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.305 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:47.305 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:47.305 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.305 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.306 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.308 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:47.309 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.310 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.310 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:47.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.310 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:47.310 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:47.310 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:47.310 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.310 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.310 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:47.310 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:47.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc2f11d640 1 -- 172.21.15.178:0/4090890139 >> v2:172.21.15.204:6812/28210 conn(0x7fcc080397b0 msgr2=0x7fcc0803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc2f11d640 1 --2- 172.21.15.178:0/4090890139 >> v2:172.21.15.204:6812/28210 conn(0x7fcc080397b0 0x7fcc0803bc60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcc280a6ec0 tx=0x7fcc18009760 comp rx=0 tx=0).stop 2024-09-15T01:43:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc2f11d640 10 monclient: shutdown 2024-09-15T01:43:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc2f11d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc2f11d640 1 -- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 msgr2=0x7fcc28131780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc2f11d640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 0x7fcc28131780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcc10035040 tx=0x7fcc10032000 comp rx=0 tx=0).stop 2024-09-15T01:43:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc2d6ba640 1 -- 172.21.15.178:0/4090890139 reap_dead start 2024-09-15T01:43:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc2f11d640 1 -- 172.21.15.178:0/4090890139 shutdown_connections 2024-09-15T01:43:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc2f11d640 1 --2- 172.21.15.178:0/4090890139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc280a2930 0x7fcc28131780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc2f11d640 1 -- 172.21.15.178:0/4090890139 >> 172.21.15.178:0/4090890139 conn(0x7fcc280a5ed0 msgr2=0x7fcc2809e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc2f11d640 1 -- 172.21.15.178:0/4090890139 shutdown_connections 2024-09-15T01:43:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.280+0000 7fcc2f11d640 1 -- 172.21.15.178:0/4090890139 wait complete. 2024-09-15T01:43:47.313 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T01:43:47.313 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon f 2024-09-15T01:43:47.313 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:43:47.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.403+0000 7f8a3579d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:47.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.403+0000 7f8a3579d640 10 monclient: build_initial_monmap 2024-09-15T01:43:47.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.403+0000 7f8a3579d640 10 monclient: monmap: 2024-09-15T01:43:47.407 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:47.407 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.407 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:47.405771+0000 2024-09-15T01:43:47.407 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:47.405771+0000 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:47.408 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.404+0000 7f8a3579d640 1 Processor -- start 2024-09-15T01:43:47.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.404+0000 7f8a3579d640 1 -- start start 2024-09-15T01:43:47.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.404+0000 7f8a3579d640 10 monclient: init 2024-09-15T01:43:47.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.404+0000 7f8a3579d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:47.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.404+0000 7f8a3579d640 10 monclient: _add_conns ranks=[4,8,1,5,9,7,2,0,3,10,6] 2024-09-15T01:43:47.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.404+0000 7f8a3579d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8a300a2930 0x7f8a30132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.404+0000 7f8a3579d640 10 monclient(hunting): picked mon.noname-c con 0x7f8a300a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:47.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.404+0000 7f8a3579d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 0x7f8a3009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.404+0000 7f8a3579d640 10 monclient(hunting): picked mon.noname-e con 0x7f8a3009faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:47.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.404+0000 7f8a3579d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8a300a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:47.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a3579d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3004bf60 con 0x7f8a300a08c0 2024-09-15T01:43:47.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a3579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a3579d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3009e0f0 con 0x7f8a300a2930 2024-09-15T01:43:47.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a3579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a3579d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3009dd80 con 0x7f8a3009faa0 2024-09-15T01:43:47.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a3579d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:47.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a3579d640 10 monclient(hunting): authenticate will time out at 1432.156036s 2024-09-15T01:43:47.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 0x7f8a3009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2effd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8a300a2930 0x7f8a30132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 0x7f8a3009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40588/0 (socket says 172.21.15.178:40588) 2024-09-15T01:43:47.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 1 -- 172.21.15.178:0/4227494905 learned_addr learned my addr 172.21.15.178:0/4227494905 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:47.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2effd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8a300a2930 0x7f8a30132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:50198/0 (socket says 172.21.15.178:50198) 2024-09-15T01:43:47.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 10 monclient(hunting): get_auth_request con 0x7f8a3009faa0 auth_method 0 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2effd640 10 monclient(hunting): get_auth_request con 0x7f8a300a2930 auth_method 0 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 10 monclient(hunting): handle_auth_done global_id 34883 payload 274 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 1 -- 172.21.15.178:0/4227494905 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8a300a2930 msgr2=0x7f8a30132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 1 --2- 172.21.15.178:0/4227494905 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8a300a2930 0x7f8a30132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 1 -- 172.21.15.178:0/4227494905 >> v1:172.21.15.204:6789/0 conn(0x7f8a300a08c0 legacy=0x7f8a300a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 1 monclient: found mon.noname-e 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2e7fc640 1 -- 172.21.15.178:0/4227494905 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a300a2d00 con 0x7f8a3009faa0 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.405+0000 7f8a2effd640 1 --2- 172.21.15.178:0/4227494905 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8a300a2930 0x7f8a30132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2e7fc640 1 --2- 172.21.15.178:0/4227494905 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 0x7f8a3009fe70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009940 tx=0x7f8a200091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9d884c56c8790348 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2dffb640 1 -- 172.21.15.178:0/4227494905 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a2002ad00 con 0x7f8a3009faa0 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2dffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2dffb640 10 monclient: dump: 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.424 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2dffb640 10 monclient: _finish_auth 0 2024-09-15T01:43:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2dffb640 1 -- 172.21.15.178:0/4227494905 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2002ae80 con 0x7f8a3009faa0 2024-09-15T01:43:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 5 monclient: authenticate success, global_id 34883 2024-09-15T01:43:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2dffb640 1 -- 172.21.15.178:0/4227494905 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a20031ca0 con 0x7f8a3009faa0 2024-09-15T01:43:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 4 set_mon_vals no callback set 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a2dffb640 10 monclient: dump: 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 10 monclient: shutdown 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 1 -- 172.21.15.178:0/4227494905 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 msgr2=0x7f8a3009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 1 --2- 172.21.15.178:0/4227494905 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 0x7f8a3009fe70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009940 tx=0x7f8a200091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 1 -- 172.21.15.178:0/4227494905 shutdown_connections 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 1 --2- 172.21.15.178:0/4227494905 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 0x7f8a3009fe70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 1 --2- 172.21.15.178:0/4227494905 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f8a300a2930 0x7f8a30132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 1 -- 172.21.15.178:0/4227494905 >> 172.21.15.178:0/4227494905 conn(0x7f8a300a5ed0 msgr2=0x7f8a300a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 1 -- 172.21.15.178:0/4227494905 shutdown_connections 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.406+0000 7f8a3579d640 1 -- 172.21.15.178:0/4227494905 wait complete. 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.407+0000 7f8a3579d640 10 monclient: build_initial_monmap 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.407+0000 7f8a3579d640 10 monclient: monmap: 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:47.427 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.407+0000 7f8a3579d640 1 Processor -- start 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.407+0000 7f8a3579d640 1 -- start start 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.407+0000 7f8a3579d640 10 monclient: init 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 10 monclient: _add_conns ranks=[8,3,1,5,2,7,4,6,0] 2024-09-15T01:43:47.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 0x7f8a3009b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 10 monclient(hunting): picked mon.noname-e con 0x7f8a3009faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a300a08c0 0x7f8a30131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 10 monclient(hunting): picked mon.noname-g con 0x7f8a300a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a300a2930 0x7f8a30141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 10 monclient(hunting): picked mon.noname-f con 0x7f8a300a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3009d630 con 0x7f8a300a2930 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3011dae0 con 0x7f8a300a08c0 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3011ea40 con 0x7f8a3009faa0 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a3579d640 10 monclient(hunting): authenticate will time out at 1432.159462s 2024-09-15T01:43:47.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 0x7f8a3009b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 0x7f8a3009b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40592/0 (socket says 172.21.15.178:40592) 2024-09-15T01:43:47.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 1 -- 172.21.15.178:0/1606242817 learned_addr learned my addr 172.21.15.178:0/1606242817 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:47.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 10 monclient(hunting): get_auth_request con 0x7f8a3009faa0 auth_method 0 2024-09-15T01:43:47.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2e7fc640 1 -- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a300a08c0 msgr2=0x7f8a30131920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:43:47.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2e7fc640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a300a08c0 0x7f8a30131920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:47.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2f7fe640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a300a2930 0x7f8a30141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2f7fe640 10 monclient(hunting): get_auth_request con 0x7f8a300a2930 auth_method 0 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 10 monclient(hunting): handle_auth_done global_id 34892 payload 824 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 1 -- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a300a08c0 msgr2=0x7f8a30131920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a300a08c0 0x7f8a30131920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 1 -- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a300a2930 msgr2=0x7f8a30141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a300a2930 0x7f8a30141d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 10 monclient: _finish_hunting 0 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 1 monclient: found mon.noname-e 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:47.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 1 -- 172.21.15.178:0/1606242817 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a24009390 con 0x7f8a3009faa0 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2f7fe640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a300a2930 0x7f8a30141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.408+0000 7f8a2effd640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 0x7f8a3009b100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400ec80 tx=0x7f8a240090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a2400eec0 con 0x7f8a3009faa0 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 10 monclient: dump: 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.432 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 10 monclient: _finish_auth 0 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:47.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a3579d640 5 monclient: authenticate success, global_id 34892 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a24013470 con 0x7f8a3009faa0 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a3579d640 10 monclient: _renew_subs 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a3579d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a3579d640 1 -- 172.21.15.178:0/1606242817 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a20009450 con 0x7f8a3009faa0 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a24013a60 con 0x7f8a3009faa0 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 10 monclient: dump: 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.434 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a2d7fa640 4 set_mon_vals no callback set 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a3579d640 10 monclient: _renew_subs 2024-09-15T01:43:47.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a3579d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:43:47.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a3579d640 1 -- 172.21.15.178:0/1606242817 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3011f000 con 0x7f8a3009faa0 2024-09-15T01:43:47.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.409+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8a24013bc0 con 0x7f8a3009faa0 2024-09-15T01:43:47.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a3579d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:47.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a3579d640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a3011c9f0 0x7f8a30142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a3579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a3579d640 1 -- 172.21.15.178:0/1606242817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3011eec0 con 0x7f8a3011c9f0 2024-09-15T01:43:47.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a16ffd640 1 --2- 172.21.15.178:0/1606242817 >> v2:172.21.15.204:6812/28210 conn(0x7f8a1c0396e0 0x7f8a1c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f8a2404c340 con 0x7f8a3009faa0 2024-09-15T01:43:47.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2e7fc640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a3011c9f0 0x7f8a30142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2e7fc640 10 monclient: get_auth_request con 0x7f8a3011c9f0 auth_method 0 2024-09-15T01:43:47.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2f7fe640 1 --2- 172.21.15.178:0/1606242817 >> v2:172.21.15.204:6812/28210 conn(0x7f8a1c0396e0 0x7f8a1c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2f7fe640 10 monclient: get_auth_request con 0x7f8a1c0396e0 auth_method 0 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2e7fc640 10 monclient(hunting): handle_auth_done global_id 54370 payload 824 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2e7fc640 1 -- 172.21.15.178:0/1606242817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a3005ed70 con 0x7f8a3011c9f0 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2e7fc640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a3011c9f0 0x7f8a30142280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a300a3290 tx=0x7f8a20032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a2f7fe640 1 --2- 172.21.15.178:0/1606242817 >> v2:172.21.15.204:6812/28210 conn(0x7f8a1c0396e0 0x7f8a1c03bb90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8a3009b680 tx=0x7f8a28009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a20031ea0 con 0x7f8a3011c9f0 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.410+0000 7f8a16ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.411+0000 7f8a16ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.411+0000 7f8a16ffd640 10 monclient: dump: 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.438 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.439 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.439 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.439 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.439 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.411+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f8a20031c90 con 0x7f8a3011c9f0 2024-09-15T01:43:47.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.411+0000 7f8a16ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:47.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.411+0000 7f8a16ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:47.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.411+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a3011c9f0 msgr2=0x7f8a30142280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.411+0000 7f8a16ffd640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a3011c9f0 0x7f8a30142280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a300a3290 tx=0x7f8a20032000 comp rx=0 tx=0).stop 2024-09-15T01:43:47.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.411+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a3011c9f0 msgr2=0x7f8a30142280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:47.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.411+0000 7f8a16ffd640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a3011c9f0 0x7f8a30142280 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a300a3290 tx=0x7f8a20032000 comp rx=0 tx=0).stop 2024-09-15T01:43:47.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.416+0000 7f8a3579d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:47.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.416+0000 7f8a3579d640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30131e60 0x7f8a301427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.416+0000 7f8a3579d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.416+0000 7f8a3579d640 1 -- 172.21.15.178:0/1606242817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3011cdc0 con 0x7f8a30131e60 2024-09-15T01:43:47.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.417+0000 7f8a2e7fc640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30131e60 0x7f8a301427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.417+0000 7f8a2e7fc640 10 monclient: get_auth_request con 0x7f8a30131e60 auth_method 0 2024-09-15T01:43:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.417+0000 7f8a2e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.417+0000 7f8a2e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.417+0000 7f8a2e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.417+0000 7f8a2e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.417+0000 7f8a2e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.417+0000 7f8a2e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.417+0000 7f8a2e7fc640 10 monclient(hunting): handle_auth_done global_id 54379 payload 824 2024-09-15T01:43:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.417+0000 7f8a2e7fc640 1 -- 172.21.15.178:0/1606242817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8a3004bc80 con 0x7f8a30131e60 2024-09-15T01:43:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.418+0000 7f8a2e7fc640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30131e60 0x7f8a301427c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a200097e0 tx=0x7f8a2003c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.418+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a200508e0 con 0x7f8a30131e60 2024-09-15T01:43:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.418+0000 7f8a16ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.418+0000 7f8a16ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.418+0000 7f8a16ffd640 10 monclient: dump: 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.443 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.444 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.444 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.444 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.444 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.418+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9109 (secure 0 0 0) 0x7f8a20050460 con 0x7f8a30131e60 2024-09-15T01:43:47.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.418+0000 7f8a16ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:47.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.418+0000 7f8a16ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:47.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.418+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30131e60 msgr2=0x7f8a301427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.418+0000 7f8a16ffd640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30131e60 0x7f8a301427c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a200097e0 tx=0x7f8a2003c000 comp rx=0 tx=0).stop 2024-09-15T01:43:47.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.418+0000 7f8a16ffd640 1 -- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30131e60 msgr2=0x7f8a301427c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:47.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.418+0000 7f8a16ffd640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a30131e60 0x7f8a301427c0 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a200097e0 tx=0x7f8a2003c000 comp rx=0 tx=0).stop 2024-09-15T01:43:47.445 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:47.445 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:47.445 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 128, 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:47.446 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:47.447 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:47.448 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:47.449 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:47.449 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:47.449 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:47.449 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:47.449 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:47.449 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:47.449 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:47.449 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:47.449 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:47.449 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:47.449 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:47.449 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:47.450 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.452 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.452 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:47.452 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.452 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.452 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.452 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:47.452 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:47.452 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.452 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.452 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.453 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:47.454 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.456 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.458 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.458 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.458 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:47.458 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:47.458 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.458 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.458 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.458 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.458 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:47.458 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.459 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.459 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.459 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:47.459 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.459 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.459 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.459 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:47.459 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:47.459 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.459 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.459 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.460 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.460 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.460 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:47.460 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:47.460 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.460 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.460 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.460 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.460 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:47.460 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.460 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.460 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.462 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:47.462 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:47.462 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.462 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.462 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.462 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.462 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:47.462 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.462 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.462 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.463 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:47.463 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.463 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.463 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.463 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:47.463 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:47.463 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.463 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.463 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.464 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:47.466 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:47.466 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.466 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:47.466 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.466 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:47.466 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:47.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.468 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:47.469 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:47.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.419+0000 7f8a3579d640 1 -- 172.21.15.178:0/1606242817 >> v2:172.21.15.204:6812/28210 conn(0x7f8a1c0396e0 msgr2=0x7f8a1c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.419+0000 7f8a3579d640 1 --2- 172.21.15.178:0/1606242817 >> v2:172.21.15.204:6812/28210 conn(0x7f8a1c0396e0 0x7f8a1c03bb90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8a3009b680 tx=0x7f8a28009040 comp rx=0 tx=0).stop 2024-09-15T01:43:47.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.419+0000 7f8a3579d640 10 monclient: shutdown 2024-09-15T01:43:47.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.419+0000 7f8a3579d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:47.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.419+0000 7f8a3579d640 1 -- 172.21.15.178:0/1606242817 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 msgr2=0x7f8a3009b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.419+0000 7f8a3579d640 1 --2- 172.21.15.178:0/1606242817 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a3009faa0 0x7f8a3009b100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400ec80 tx=0x7f8a240090f0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.419+0000 7f8a2f7fe640 1 -- 172.21.15.178:0/1606242817 reap_dead start 2024-09-15T01:43:47.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.419+0000 7f8a2f7fe640 1 -- 172.21.15.178:0/1606242817 reap_dead start 2024-09-15T01:43:47.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.419+0000 7f8a3579d640 1 -- 172.21.15.178:0/1606242817 shutdown_connections 2024-09-15T01:43:47.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.419+0000 7f8a3579d640 1 -- 172.21.15.178:0/1606242817 >> 172.21.15.178:0/1606242817 conn(0x7f8a300a5ed0 msgr2=0x7f8a30135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:47.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.419+0000 7f8a3579d640 1 -- 172.21.15.178:0/1606242817 shutdown_connections 2024-09-15T01:43:47.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.419+0000 7f8a3579d640 1 -- 172.21.15.178:0/1606242817 wait complete. 2024-09-15T01:43:47.472 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T01:43:47.472 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-09-15T01:43:47.472 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:43:47.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.561+0000 7f42c3d5d640 10 monclient: get_monmap_and_config 2024-09-15T01:43:47.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.561+0000 7f42c3d5d640 10 monclient: build_initial_monmap 2024-09-15T01:43:47.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.561+0000 7f42c3d5d640 10 monclient: monmap: 2024-09-15T01:43:47.565 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:47.565 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.565 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:43:47.563602+0000 2024-09-15T01:43:47.565 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:43:47.563602+0000 2024-09-15T01:43:47.565 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:47.565 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 1 Processor -- start 2024-09-15T01:43:47.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 1 -- start start 2024-09-15T01:43:47.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 10 monclient: init 2024-09-15T01:43:47.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:47.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 10 monclient: _add_conns ranks=[1,4,3,6,7,2,0,10,9,8,5] 2024-09-15T01:43:47.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f42bc0a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:43:47.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f42bc132de0 0x7f42bc137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 10 monclient(hunting): picked mon.noname-c con 0x7f42bc132de0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:43:47.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42bc1380e0 0x7f42bc13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 10 monclient(hunting): picked mon.noname-g con 0x7f42bc1380e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:43:47.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42bc04bf60 con 0x7f42bc0a2930 2024-09-15T01:43:47.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc09dd80 con 0x7f42bc1380e0 2024-09-15T01:43:47.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0a2d00 con 0x7f42bc132de0 2024-09-15T01:43:47.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:47.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.562+0000 7f42c3d5d640 10 monclient(hunting): authenticate will time out at 1432.313956s 2024-09-15T01:43:47.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f42bc132de0 0x7f42bc137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c22fa640 1 -- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42bc1380e0 msgr2=0x7f42bc13a4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3301/0 2024-09-15T01:43:47.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f42bc132de0 0x7f42bc137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:50212/0 (socket says 172.21.15.178:50212) 2024-09-15T01:43:47.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 1 -- 172.21.15.178:0/944721192 learned_addr learned my addr 172.21.15.178:0/944721192 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:47.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 10 monclient(hunting): get_auth_request con 0x7f42bc132de0 auth_method 0 2024-09-15T01:43:47.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c22fa640 1 --2- 172.21.15.178:0/944721192 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42bc1380e0 0x7f42bc13a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:47.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 10 monclient(hunting): handle_auth_done global_id 44527 payload 274 2024-09-15T01:43:47.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 1 -- 172.21.15.178:0/944721192 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42bc1380e0 msgr2=0x7f42bc13a4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:43:47.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 1 --2- 172.21.15.178:0/944721192 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42bc1380e0 0x7f42bc13a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 1 -- 172.21.15.178:0/944721192 >> v1:172.21.15.204:6789/0 conn(0x7f42bc0a2930 legacy=0x7f42bc09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:43:47.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 10 monclient: _finish_hunting 0 2024-09-15T01:43:47.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 1 monclient: found mon.noname-c 2024-09-15T01:43:47.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:47.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:43:47.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.563+0000 7f42c12f8640 1 -- 172.21.15.178:0/944721192 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc1331b0 con 0x7f42bc132de0 2024-09-15T01:43:47.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c12f8640 1 --2- 172.21.15.178:0/944721192 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f42bc132de0 0x7f42bc137ba0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009940 tx=0x7f42ac0091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=86e6ca12181ff51f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c0af7640 1 -- 172.21.15.178:0/944721192 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42ac02ac90 con 0x7f42bc132de0 2024-09-15T01:43:47.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c0af7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c0af7640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:43:47.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c0af7640 10 monclient: dump: 2024-09-15T01:43:47.581 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.581 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.581 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.581 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.581 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.581 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.581 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.581 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.581 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.582 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.582 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.582 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.582 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.582 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.582 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.582 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c0af7640 10 monclient: _finish_auth 0 2024-09-15T01:43:47.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c0af7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:47.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 5 monclient: authenticate success, global_id 44527 2024-09-15T01:43:47.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:43:47.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c0af7640 1 -- 172.21.15.178:0/944721192 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac02ae10 con 0x7f42bc132de0 2024-09-15T01:43:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c0af7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:43:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c0af7640 1 -- 172.21.15.178:0/944721192 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42ac031ca0 con 0x7f42bc132de0 2024-09-15T01:43:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 4 set_mon_vals no callback set 2024-09-15T01:43:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c0af7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c0af7640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:43:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c0af7640 10 monclient: dump: 2024-09-15T01:43:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.584 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 10 monclient: shutdown 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 1 -- 172.21.15.178:0/944721192 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f42bc132de0 msgr2=0x7f42bc137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 1 --2- 172.21.15.178:0/944721192 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f42bc132de0 0x7f42bc137ba0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009940 tx=0x7f42ac0091d0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 1 -- 172.21.15.178:0/944721192 shutdown_connections 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 1 --2- 172.21.15.178:0/944721192 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42bc1380e0 0x7f42bc13a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 1 --2- 172.21.15.178:0/944721192 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f42bc132de0 0x7f42bc137ba0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 1 -- 172.21.15.178:0/944721192 >> 172.21.15.178:0/944721192 conn(0x7f42bc0a5ed0 msgr2=0x7f42bc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 1 -- 172.21.15.178:0/944721192 shutdown_connections 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.564+0000 7f42c3d5d640 1 -- 172.21.15.178:0/944721192 wait complete. 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.565+0000 7f42c3d5d640 10 monclient: build_initial_monmap 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.565+0000 7f42c3d5d640 10 monclient: monmap: 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:43:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:43:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.565+0000 7f42c3d5d640 1 Processor -- start 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.565+0000 7f42c3d5d640 1 -- start start 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.565+0000 7f42c3d5d640 10 monclient: init 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 10 monclient: _add_conns ranks=[7,5,1,3,4,8,2,0,6] 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f42bc0a2930 0x7f42bc09a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 10 monclient(hunting): picked mon.noname-i con 0x7f42bc0a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc132de0 0x7f42bc09ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 10 monclient(hunting): picked mon.noname-h con 0x7f42bc132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42bc1380e0 0x7f42bc09b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 10 monclient(hunting): picked mon.noname-f con 0x7f42bc1380e0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc09d630 con 0x7f42bc1380e0 2024-09-15T01:43:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc11c830 con 0x7f42bc132de0 2024-09-15T01:43:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc09c330 con 0x7f42bc0a2930 2024-09-15T01:43:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:43:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c3d5d640 10 monclient(hunting): authenticate will time out at 1432.317293s 2024-09-15T01:43:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c1af9640 1 -- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f42bc0a2930 msgr2=0x7f42bc09a930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3303/0 2024-09-15T01:43:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c1af9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f42bc0a2930 0x7f42bc09a930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:43:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42bc1380e0 0x7f42bc09b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42bc1380e0 0x7f42bc09b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:44870/0 (socket says 172.21.15.178:44870) 2024-09-15T01:43:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 1 -- 172.21.15.178:0/3003942825 learned_addr learned my addr 172.21.15.178:0/3003942825 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:43:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 10 monclient(hunting): get_auth_request con 0x7f42bc1380e0 auth_method 0 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c12f8640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc132de0 0x7f42bc09ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c12f8640 10 monclient(hunting): get_auth_request con 0x7f42bc132de0 auth_method 0 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c12f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c12f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c12f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c12f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c12f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c12f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 10 monclient(hunting): handle_auth_done global_id 54388 payload 824 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 1 -- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f42bc0a2930 msgr2=0x7f42bc09a930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f42bc0a2930 0x7f42bc09a930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 1 -- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc132de0 msgr2=0x7f42bc09ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc132de0 0x7f42bc09ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 10 monclient: _finish_hunting 0 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 1 monclient: found mon.noname-f 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.566+0000 7f42c22fa640 1 -- 172.21.15.178:0/3003942825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac009450 con 0x7f42bc1380e0 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42c22fa640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42bc1380e0 0x7f42bc09b3b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f42b400bf80 tx=0x7f42b4009730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42b400cdf0 con 0x7f42bc1380e0 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b1ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b1ffb640 10 monclient: dump: 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.592 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:43:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:43:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b40065d0 con 0x7f42bc1380e0 2024-09-15T01:43:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42c3d5d640 5 monclient: authenticate success, global_id 54388 2024-09-15T01:43:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42c3d5d640 10 monclient: _renew_subs 2024-09-15T01:43:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42c3d5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42c3d5d640 1 -- 172.21.15.178:0/3003942825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc11d6f0 con 0x7f42bc1380e0 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42b40153e0 con 0x7f42bc1380e0 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b1ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b1ffb640 10 monclient: dump: 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42c3d5d640 10 monclient: _renew_subs 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42b3fff640 4 set_mon_vals no callback set 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42c3d5d640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.204:3300/0 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.567+0000 7f42c3d5d640 1 -- 172.21.15.178:0/3003942825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc132760 con 0x7f42bc1380e0 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f42b4012020 con 0x7f42bc1380e0 2024-09-15T01:43:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c3d5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c3d5d640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc09b8f0 0x7f42bc096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c3d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c3d5d640 1 -- 172.21.15.178:0/3003942825 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc04c4a0 con 0x7f42bc09b8f0 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42b1ffb640 1 --2- 172.21.15.178:0/3003942825 >> v2:172.21.15.204:6812/28210 conn(0x7f42a00397b0 0x7f42a003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(261..261 src has 186..261) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f42b404bfe0 con 0x7f42bc1380e0 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c1af9640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc09b8f0 0x7f42bc096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c1af9640 10 monclient: get_auth_request con 0x7f42bc09b8f0 auth_method 0 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c1af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c1af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c1af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c12f8640 1 --2- 172.21.15.178:0/3003942825 >> v2:172.21.15.204:6812/28210 conn(0x7f42a00397b0 0x7f42a003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c12f8640 10 monclient: get_auth_request con 0x7f42a00397b0 auth_method 0 2024-09-15T01:43:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c1af9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c1af9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c1af9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c1af9640 10 monclient(hunting): handle_auth_done global_id 7268 payload 824 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.568+0000 7f42c1af9640 1 -- 172.21.15.178:0/3003942825 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42bc05ed70 con 0x7f42bc09b8f0 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42c1af9640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc09b8f0 0x7f42bc096a30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f42b800c4f0 tx=0x7f42b8002e60 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42c12f8640 1 --2- 172.21.15.178:0/3003942825 >> v2:172.21.15.204:6812/28210 conn(0x7f42a00397b0 0x7f42a003bc60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f42ac034000 tx=0x7f42ac032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42b8006070 con 0x7f42bc09b8f0 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42b1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42b1ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42b1ffb640 10 monclient: dump: 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f42b8010620 con 0x7f42bc09b8f0 2024-09-15T01:43:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42b1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:43:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42b1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:43:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc09b8f0 msgr2=0x7f42bc096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42b1ffb640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc09b8f0 0x7f42bc096a30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f42b800c4f0 tx=0x7f42b8002e60 comp rx=0 tx=0).stop 2024-09-15T01:43:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc09b8f0 msgr2=0x7f42bc096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.569+0000 7f42b1ffb640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc09b8f0 0x7f42bc096a30 secure :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f42b800c4f0 tx=0x7f42b8002e60 comp rx=0 tx=0).stop 2024-09-15T01:43:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.574+0000 7f42c3d5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:43:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.574+0000 7f42c3d5d640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc096f70 0x7f42bc097370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:43:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c3d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:43:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c3d5d640 1 -- 172.21.15.178:0/3003942825 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc09e3b0 con 0x7f42bc096f70 2024-09-15T01:43:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c1af9640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc096f70 0x7f42bc097370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:43:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c1af9640 10 monclient: get_auth_request con 0x7f42bc096f70 auth_method 0 2024-09-15T01:43:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c1af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:43:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c1af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:43:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c1af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:43:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c1af9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:43:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c1af9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:43:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c1af9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:43:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c1af9640 10 monclient(hunting): handle_auth_done global_id 7277 payload 824 2024-09-15T01:43:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c1af9640 1 -- 172.21.15.178:0/3003942825 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f42bc04bc80 con 0x7f42bc096f70 2024-09-15T01:43:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42c1af9640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc096f70 0x7f42bc097370 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f42b800c4f0 tx=0x7f42b8018000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:43:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42b8023070 con 0x7f42bc096f70 2024-09-15T01:43:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42b1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:43:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42b1ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:43:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.575+0000 7f42b1ffb640 10 monclient: dump: 2024-09-15T01:43:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:43:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:43:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:43:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:43:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:43:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:43:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:43:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:43:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:43:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:43:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:43:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:43:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:43:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:43:47.603 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:43:47.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9281 (secure 0 0 0) 0x7f42b8019450 con 0x7f42bc096f70 2024-09-15T01:43:47.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42b1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:43:47.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42b1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:43:47.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc096f70 msgr2=0x7f42bc097370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42b1ffb640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc096f70 0x7f42bc097370 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f42b800c4f0 tx=0x7f42b8018000 comp rx=0 tx=0).stop 2024-09-15T01:43:47.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42b1ffb640 1 -- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc096f70 msgr2=0x7f42bc097370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:43:47.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42b1ffb640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f42bc096f70 0x7f42bc097370 secure :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f42b800c4f0 tx=0x7f42b8018000 comp rx=0 tx=0).stop 2024-09-15T01:43:47.603 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:43:47.604 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:47.604 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:47.604 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:43:47.604 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 128, 2024-09-15T01:43:47.604 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:43:47.604 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:43:47.604 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:43:47.604 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:43:47.604 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:43:47.604 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:43:47.604 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:43:47.604 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.605 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:43:47.605 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:47.605 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:43:47.605 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:43:47.605 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:47.605 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:47.605 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:47.605 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:47.605 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:47.605 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:47.605 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:47.605 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:47.606 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:43:47.607 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:43:47.608 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:43:47.609 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.611 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:43:47.612 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:43:47.613 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.613 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:43:47.613 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.613 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.613 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:43:47.613 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.613 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.614 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.615 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.616 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.617 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.618 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.619 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.620 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.621 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.621 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:43:47.621 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:43:47.621 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.621 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.621 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.621 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.621 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:43:47.621 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:43:47.621 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:43:47.624 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:43:47.624 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.624 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.624 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:43:47.624 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:43:47.624 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:43:47.624 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.624 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.624 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.624 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:43:47.624 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:43:47.625 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:43:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 5 2024-09-15T01:43:47.627 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:43:47.627 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:43:47.627 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:43:47.627 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:43:47.627 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:43:47.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42c3d5d640 1 -- 172.21.15.178:0/3003942825 >> v2:172.21.15.204:6812/28210 conn(0x7f42a00397b0 msgr2=0x7f42a003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42c3d5d640 1 --2- 172.21.15.178:0/3003942825 >> v2:172.21.15.204:6812/28210 conn(0x7f42a00397b0 0x7f42a003bc60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f42ac034000 tx=0x7f42ac032000 comp rx=0 tx=0).stop 2024-09-15T01:43:47.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42c3d5d640 10 monclient: shutdown 2024-09-15T01:43:47.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42c3d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:43:47.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42c3d5d640 1 -- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42bc1380e0 msgr2=0x7f42bc09b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:43:47.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42c3d5d640 1 --2- 172.21.15.178:0/3003942825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42bc1380e0 0x7f42bc09b3b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f42b400bf80 tx=0x7f42b4009730 comp rx=0 tx=0).stop 2024-09-15T01:43:47.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42c22fa640 1 -- 172.21.15.178:0/3003942825 reap_dead start 2024-09-15T01:43:47.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42c22fa640 1 -- 172.21.15.178:0/3003942825 reap_dead start 2024-09-15T01:43:47.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42c3d5d640 1 -- 172.21.15.178:0/3003942825 shutdown_connections 2024-09-15T01:43:47.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42c3d5d640 1 -- 172.21.15.178:0/3003942825 >> 172.21.15.178:0/3003942825 conn(0x7f42bc0a5ed0 msgr2=0x7f42bc135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:43:47.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42c3d5d640 1 -- 172.21.15.178:0/3003942825 shutdown_connections 2024-09-15T01:43:47.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:43:47.576+0000 7f42c3d5d640 1 -- 172.21.15.178:0/3003942825 wait complete. 2024-09-15T01:43:47.629 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T01:43:47.630 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T01:43:48.774 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=13 2024-09-15T01:43:48.774 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T01:43:48.774 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:43:48.774 INFO:tasks.workunit.client.0.smithi204.stderr:+ c=''\''allow command service with prefix=list, allow command quorum_status'\''' 2024-09-15T01:43:48.774 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph auth get-or-create client.bar mon '\''allow command service with prefix=list, allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.bar.keyring' 0 2024-09-15T01:43:48.774 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph auth get-or-create client.bar mon '\''allow command service with prefix=list, allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.bar.keyring' 2024-09-15T01:43:48.774 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=0 2024-09-15T01:43:48.774 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph auth get-or-create client.bar mon ''\''allow' command service with prefix=list, allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.bar.keyring 2024-09-15T01:43:48.775 INFO:tasks.workunit.client.0.smithi204.stdout:ceph auth get-or-create client.bar mon 'allow command service with prefix=list, allow command quorum_status' > /tmp/cephtest-mon-caps-madness.bar.keyring 2024-09-15T01:43:48.775 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph auth get-or-create client.bar mon ''\''allow' command service with prefix=list, allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.bar.keyring 2024-09-15T01:43:49.256 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=0 2024-09-15T01:43:49.256 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 0 -ne 0 ]] 2024-09-15T01:43:49.256 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:43:49.256 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status' 0 2024-09-15T01:43:49.256 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status' 2024-09-15T01:43:49.256 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=0 2024-09-15T01:43:49.257 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-09-15T01:43:49.257 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-09-15T01:43:49.257 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-09-15T01:43:49.709 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=0 2024-09-15T01:43:49.709 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 0 -ne 0 ]] 2024-09-15T01:43:49.710 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:43:49.710 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls' 13 2024-09-15T01:43:49.710 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls' 2024-09-15T01:43:49.710 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=13 2024-09-15T01:43:49.710 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls 2024-09-15T01:43:49.710 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls 2024-09-15T01:43:49.710 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls 2024-09-15T01:43:50.001 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=13 2024-09-15T01:43:50.001 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T01:43:50.002 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:43:50.002 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export' 13 2024-09-15T01:43:50.002 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export' 2024-09-15T01:43:50.002 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=13 2024-09-15T01:43:50.002 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export 2024-09-15T01:43:50.002 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export 2024-09-15T01:43:50.003 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export 2024-09-15T01:43:50.295 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=13 2024-09-15T01:43:50.295 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T01:43:50.296 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:43:50.296 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo' 13 2024-09-15T01:43:50.296 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo' 2024-09-15T01:43:50.296 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=13 2024-09-15T01:43:50.296 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo 2024-09-15T01:43:50.296 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo 2024-09-15T01:43:50.297 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo 2024-09-15T01:43:53.600 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=13 2024-09-15T01:43:53.601 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T01:43:53.601 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:43:53.601 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump' 13 2024-09-15T01:43:53.601 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump' 2024-09-15T01:43:53.601 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=13 2024-09-15T01:43:53.602 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump 2024-09-15T01:43:53.602 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump 2024-09-15T01:43:53.602 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump 2024-09-15T01:43:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=13 2024-09-15T01:43:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T01:43:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:43:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:+ export CEPH_ARGS=--rados-mon-op-timeout=300 2024-09-15T01:43:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:+ CEPH_ARGS=--rados-mon-op-timeout=300 2024-09-15T01:43:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump' 110 2024-09-15T01:43:53.836 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump' 2024-09-15T01:43:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=110 2024-09-15T01:43:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump 2024-09-15T01:43:53.837 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump 2024-09-15T01:43:53.837 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump 2024-09-15T01:45:17.631 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T01:45:17.631 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-15T01:45:17.631 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-15T01:45:17.632 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-15T01:45:17.636 INFO:tasks.ceph.mon.g:Started 2024-09-15T01:45:17.636 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-15T01:45:17.636 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-15T01:45:17.636 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-15T01:45:17.636 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-15T01:45:17.638 INFO:tasks.ceph.mon.i:Started 2024-09-15T01:45:17.638 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T01:45:17.639 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-15T01:45:17.639 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T01:45:17.639 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T01:45:17.641 INFO:tasks.ceph.mon.a:Started 2024-09-15T01:45:17.642 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-15T01:45:17.642 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-15T01:45:17.642 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-15T01:45:17.642 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-15T01:45:17.642 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-09-15T01:45:17.642 INFO:tasks.ceph.mon.h:Sent signal 19 2024-09-15T01:45:17.643 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T01:45:17.745 INFO:tasks.ceph.mon.g.smithi204.stderr:2024-09-15T01:45:17.742+0000 7efd57ea0900 -1 mon.g@-1(???).log v0 log_external_backlog rewinding external_log_to from 427 -> 0 (sync_force? mon rebuild?) 2024-09-15T01:45:17.745 INFO:tasks.ceph.mon.a.smithi178.stderr:2024-09-15T01:45:17.743+0000 7f82fa01f900 -1 mon.a@-1(???).log v0 log_external_backlog rewinding external_log_to from 427 -> 0 (sync_force? mon rebuild?) 2024-09-15T01:45:17.747 INFO:tasks.ceph.mon.i.smithi204.stderr:2024-09-15T01:45:17.744+0000 7f82464c4900 -1 mon.i@-1(???).log v0 log_external_backlog rewinding external_log_to from 427 -> 0 (sync_force? mon rebuild?) 2024-09-15T01:45:32.643 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-15T01:45:32.644 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-15T01:45:32.644 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-15T01:45:32.645 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-15T01:45:32.645 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-09-15T01:45:32.645 INFO:tasks.ceph.mon.h:Sent signal 18 2024-09-15T01:45:32.645 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:45:32.645 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:45:32.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.749+0000 7fd14999d640 10 monclient: get_monmap_and_config 2024-09-15T01:45:32.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.749+0000 7fd14999d640 10 monclient: build_initial_monmap 2024-09-15T01:45:32.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.749+0000 7fd14999d640 10 monclient: monmap: 2024-09-15T01:45:32.751 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:32.751 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:32.751 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:32.750156+0000 2024-09-15T01:45:32.751 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:32.750156+0000 2024-09-15T01:45:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:32.753 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:32.753 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.749+0000 7fd14999d640 1 Processor -- start 2024-09-15T01:45:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.749+0000 7fd14999d640 1 -- start start 2024-09-15T01:45:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.749+0000 7fd14999d640 10 monclient: init 2024-09-15T01:45:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 10 monclient: _add_conns ranks=[1,2,9,3,5,10,0,6,8,7,4] 2024-09-15T01:45:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd144098c40 addr v1:172.21.15.204:6789/0 2024-09-15T01:45:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 10 monclient(hunting): picked mon.noname-b con 0x7fd14409dfd0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 10 monclient(hunting): picked mon.noname-a con 0x7fd144135f10 addr v2:172.21.15.178:3300/0 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14404bf60 con 0x7fd144098c40 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd144099e80 con 0x7fd144135f10 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd144099b30 con 0x7fd14409dfd0 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd14999d640 10 monclient(hunting): authenticate will time out at 1537.500425s 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1427fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1437fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1427fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:45868/0 (socket says 172.21.15.178:45868) 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1437fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:34410/0 (socket says 172.21.15.178:34410) 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 learned_addr learned my addr 172.21.15.178:0/1460551610 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1427fc640 10 monclient(hunting): get_auth_request con 0x7fd14409dfd0 auth_method 0 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1437fe640 10 monclient(hunting): get_auth_request con 0x7fd144135f10 auth_method 0 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 msgr2=0x7fd14409d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 msgr2=0x7fd14409d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 msgr2=0x7fd1441382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 msgr2=0x7fd1441382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:45:32.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.750+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:45:32.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:32.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1427fc640 10 monclient(hunting): get_auth_request con 0x7fd14409dfd0 auth_method 0 2024-09-15T01:45:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1427fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1437fe640 10 monclient(hunting): get_auth_request con 0x7fd144135f10 auth_method 0 2024-09-15T01:45:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1437fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 msgr2=0x7fd14409d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:45:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 msgr2=0x7fd14409d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 msgr2=0x7fd1441382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:45:32.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 msgr2=0x7fd1441382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:32.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:32.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:45:32.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:32.951+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:45:33.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:33.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:33.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1437fe640 10 monclient(hunting): get_auth_request con 0x7fd144135f10 auth_method 0 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1437fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1427fc640 10 monclient(hunting): get_auth_request con 0x7fd14409dfd0 auth_method 0 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1427fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 msgr2=0x7fd1441382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 msgr2=0x7fd14409d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 msgr2=0x7fd1441382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 msgr2=0x7fd14409d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:33.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:45:33.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:33.352+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:45:34.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:34.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1427fc640 10 monclient(hunting): get_auth_request con 0x7fd14409dfd0 auth_method 0 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1427fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1437fe640 10 monclient(hunting): get_auth_request con 0x7fd144135f10 auth_method 0 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1437fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 msgr2=0x7fd14409d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 msgr2=0x7fd14409d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 msgr2=0x7fd1441382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:45:34.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 msgr2=0x7fd1441382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:34.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:34.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:34.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:45:34.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:34.153+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:45:35.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 10 monclient(hunting): tick 2024-09-15T01:45:35.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 1 monclient(hunting): continuing hunt 2024-09-15T01:45:35.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:45:35.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 msgr2=0x7fd14409d3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:45:35.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14409dfd0 0x7fd14409d3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:35.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 1 -- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 msgr2=0x7fd1441382f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:45:35.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 1 --2- 172.21.15.178:0/1460551610 >> v2:172.21.15.178:3300/0 conn(0x7fd144135f10 0x7fd1441382f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:35.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 1 -- 172.21.15.178:0/1460551610 >> v1:172.21.15.204:6789/0 conn(0x7fd144098c40 legacy=0x7fd14409ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:35.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 10 monclient: _add_conns ranks=[7,4,6,10,0,5,3,8] 2024-09-15T01:45:35.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 0x7fd124005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:35.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 10 monclient(hunting): picked mon.noname-i con 0x7fd124003800 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:35.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 0x7fd124008700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:35.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 10 monclient(hunting): picked mon.noname-c con 0x7fd1240062a0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 0x7fd12400b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 10 monclient(hunting): picked mon.noname-d con 0x7fd124008df0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 1 -- 172.21.15.178:0/1460551610 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd144099e80 con 0x7fd1240062a0 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 1 -- 172.21.15.178:0/1460551610 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd144099b30 con 0x7fd124008df0 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 1 -- 172.21.15.178:0/1460551610 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd12400b9f0 con 0x7fd124003800 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd140ff9640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 0x7fd124008700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 0x7fd12400b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1427fc640 10 monclient(hunting): get_auth_request con 0x7fd1240062a0 auth_method 0 2024-09-15T01:45:35.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:35.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:35.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:35.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1437fe640 10 monclient(hunting): get_auth_request con 0x7fd124008df0 auth_method 0 2024-09-15T01:45:35.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:35.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:35.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:35.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 msgr2=0x7fd124008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:45:35.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 msgr2=0x7fd124008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:35.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.750+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 0x7fd124008700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:35.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 0x7fd124008700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:45:35.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd142ffd640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 0x7fd124005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:35.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 msgr2=0x7fd12400b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:45:35.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 msgr2=0x7fd12400b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:35.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 0x7fd12400b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:35.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd142ffd640 10 monclient(hunting): get_auth_request con 0x7fd124003800 auth_method 0 2024-09-15T01:45:35.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd142ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:35.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd142ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:35.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd142ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:35.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 0x7fd12400b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:45:35.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd142ffd640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 msgr2=0x7fd124005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:45:35.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd142ffd640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 msgr2=0x7fd124005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:35.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd142ffd640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 0x7fd124005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:35.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.751+0000 7fd142ffd640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 0x7fd124005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:45:35.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 0x7fd124008700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:35.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 0x7fd12400b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:35.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1437fe640 10 monclient(hunting): get_auth_request con 0x7fd124008df0 auth_method 0 2024-09-15T01:45:35.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:35.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:35.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1437fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:35.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1427fc640 10 monclient(hunting): get_auth_request con 0x7fd1240062a0 auth_method 0 2024-09-15T01:45:35.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:35.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:35.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1427fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:35.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd142ffd640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 0x7fd124005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:35.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd142ffd640 10 monclient(hunting): get_auth_request con 0x7fd124003800 auth_method 0 2024-09-15T01:45:35.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd142ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:35.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd142ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:35.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd142ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:35.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 msgr2=0x7fd124008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:45:35.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1427fc640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 msgr2=0x7fd124008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:35.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 0x7fd124008700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:35.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 msgr2=0x7fd12400b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:45:35.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 msgr2=0x7fd12400b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:35.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 0x7fd124008700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:45:35.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 0x7fd12400b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:35.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd142ffd640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 msgr2=0x7fd124005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:45:35.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd142ffd640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 msgr2=0x7fd124005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:35.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd142ffd640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 0x7fd124005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:35.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 0x7fd12400b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:45:35.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:35.952+0000 7fd142ffd640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 0x7fd124005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:45:36.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 0x7fd12400b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:36.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 0x7fd124008700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 10 monclient(hunting): get_auth_request con 0x7fd124008df0 auth_method 0 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1427fc640 10 monclient(hunting): get_auth_request con 0x7fd1240062a0 auth_method 0 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1427fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 10 monclient(hunting): handle_auth_done global_id 25395 payload 274 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd142ffd640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 0x7fd124005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 msgr2=0x7fd124005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 0x7fd124005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 msgr2=0x7fd124008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 0x7fd124008700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 10 monclient: _finish_hunting 0 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 1 monclient: found mon.noname-d 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd144063620 con 0x7fd124008df0 2024-09-15T01:45:36.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd142ffd640 10 monclient: get_auth_request con 0x7fd124003800 auth_method 0 2024-09-15T01:45:36.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd142ffd640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd124003800 0x7fd124005c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:45:36.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.353+0000 7fd1427fc640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd1240062a0 0x7fd124008700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:36.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd1437fe640 1 -- 172.21.15.178:0/1460551610 reap_dead start 2024-09-15T01:45:36.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd1437fe640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 0x7fd12400b270 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd13400d490 tx=0x7fd13400d040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f45c7cfcf6b564ec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:36.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd141ffb640 1 -- 172.21.15.178:0/1460551610 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1340393b0 con 0x7fd124008df0 2024-09-15T01:45:36.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd141ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:36.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd141ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:45:36.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd141ffb640 10 monclient: dump: 2024-09-15T01:45:36.362 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:36.362 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:36.362 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:36.362 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:36.362 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:36.363 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:36.363 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:36.363 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:36.363 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:36.363 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:36.363 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:36.363 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:36.364 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:36.364 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:36.364 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:36.364 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:36.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd141ffb640 10 monclient: _finish_auth 0 2024-09-15T01:45:36.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd141ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:36.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd141ffb640 1 -- 172.21.15.178:0/1460551610 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd134034070 con 0x7fd124008df0 2024-09-15T01:45:36.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd14999d640 5 monclient: authenticate success, global_id 25395 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd14999d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd141ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd14999d640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd14999d640 4 set_mon_vals no callback set 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd141ffb640 1 -- 172.21.15.178:0/1460551610 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1340399d0 con 0x7fd124008df0 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd141ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd141ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.354+0000 7fd141ffb640 10 monclient: dump: 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:36.365 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:36.366 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:36.366 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:36.366 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:36.366 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:36.366 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:36.366 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:36.366 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:36.366 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:36.366 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:36.366 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:36.366 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:36.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.355+0000 7fd14999d640 10 monclient: shutdown 2024-09-15T01:45:36.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.355+0000 7fd14999d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:36.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.355+0000 7fd14999d640 1 -- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 msgr2=0x7fd12400b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:36.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.355+0000 7fd14999d640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 0x7fd12400b270 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd13400d490 tx=0x7fd13400d040 comp rx=0 tx=0).stop 2024-09-15T01:45:36.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.355+0000 7fd14999d640 1 -- 172.21.15.178:0/1460551610 shutdown_connections 2024-09-15T01:45:36.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.355+0000 7fd14999d640 1 --2- 172.21.15.178:0/1460551610 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd124008df0 0x7fd12400b270 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:36.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.355+0000 7fd14999d640 1 -- 172.21.15.178:0/1460551610 >> 172.21.15.178:0/1460551610 conn(0x7fd1440a22e0 msgr2=0x7fd1440a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:36.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.355+0000 7fd14999d640 1 -- 172.21.15.178:0/1460551610 shutdown_connections 2024-09-15T01:45:36.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.355+0000 7fd14999d640 1 -- 172.21.15.178:0/1460551610 wait complete. 2024-09-15T01:45:36.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.356+0000 7fd14999d640 10 monclient: build_initial_monmap 2024-09-15T01:45:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.356+0000 7fd14999d640 10 monclient: monmap: 2024-09-15T01:45:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:36.368 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:36.369 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:36.370 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:36.370 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:36.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.356+0000 7fd14999d640 1 Processor -- start 2024-09-15T01:45:36.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.356+0000 7fd14999d640 1 -- start start 2024-09-15T01:45:36.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.356+0000 7fd14999d640 10 monclient: init 2024-09-15T01:45:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 10 monclient: _add_conns ranks=[8,1,2,5,0,3,7,6,4] 2024-09-15T01:45:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd124008df0 0x7fd144135100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 10 monclient(hunting): picked mon.noname-e con 0x7fd124008df0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:36.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd14413d740 0x7fd14413db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 10 monclient(hunting): picked mon.noname-f con 0x7fd14413d740 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14413e090 0x7fd14412da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 10 monclient(hunting): picked mon.noname-b con 0x7fd14413e090 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14406c9b0 con 0x7fd14413d740 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14406dc40 con 0x7fd14413e090 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd14409bd50 con 0x7fd124008df0 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd14999d640 10 monclient(hunting): authenticate will time out at 1541.107343s 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd124008df0 0x7fd144135100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:36.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd124008df0 0x7fd144135100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:56328/0 (socket says 172.21.15.178:56328) 2024-09-15T01:45:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14413e090 0x7fd14412da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 1 -- 172.21.15.178:0/2889039537 learned_addr learned my addr 172.21.15.178:0/2889039537 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 10 monclient(hunting): get_auth_request con 0x7fd124008df0 auth_method 0 2024-09-15T01:45:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1437fe640 10 monclient(hunting): get_auth_request con 0x7fd14413e090 auth_method 0 2024-09-15T01:45:36.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1437fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1437fe640 1 -- 172.21.15.178:0/2889039537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14413e090 msgr2=0x7fd14412da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:45:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1437fe640 1 -- 172.21.15.178:0/2889039537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14413e090 msgr2=0x7fd14412da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:45:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1437fe640 1 --2- 172.21.15.178:0/2889039537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14413e090 0x7fd14412da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1427fc640 1 --2- 172.21.15.178:0/2889039537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd14413d740 0x7fd14413db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:36.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1437fe640 1 --2- 172.21.15.178:0/2889039537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14413e090 0x7fd14412da90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1427fc640 10 monclient(hunting): get_auth_request con 0x7fd14413d740 auth_method 0 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1427fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd1427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 10 monclient(hunting): handle_auth_done global_id 34946 payload 824 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 1 -- 172.21.15.178:0/2889039537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14413e090 msgr2=0x7fd14412da90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 1 --2- 172.21.15.178:0/2889039537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14413e090 0x7fd14412da90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 1 -- 172.21.15.178:0/2889039537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd14413d740 msgr2=0x7fd14413db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 1 --2- 172.21.15.178:0/2889039537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd14413d740 0x7fd14413db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:36.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:45:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 1 monclient: found mon.noname-e 2024-09-15T01:45:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:45:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 1 -- 172.21.15.178:0/2889039537 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13400d0f0 con 0x7fd124008df0 2024-09-15T01:45:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.357+0000 7fd142ffd640 1 --2- 172.21.15.178:0/2889039537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd124008df0 0x7fd144135100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd13c001910 tx=0x7fd13c00c040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd13affd640 1 -- 172.21.15.178:0/2889039537 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd13c011c50 con 0x7fd124008df0 2024-09-15T01:45:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd13affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd13affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:45:36.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd13affd640 10 monclient: dump: 2024-09-15T01:45:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:36.381 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd13affd640 10 monclient: _finish_auth 0 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd13affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd14999d640 5 monclient: authenticate success, global_id 34946 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd13affd640 1 -- 172.21.15.178:0/2889039537 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c011dd0 con 0x7fd124008df0 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd14999d640 10 monclient: _renew_subs 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd14999d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd14999d640 1 -- 172.21.15.178:0/2889039537 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14409e660 con 0x7fd124008df0 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd13affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:36.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd13affd640 1 -- 172.21.15.178:0/2889039537 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd13c015890 con 0x7fd124008df0 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd13affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd13affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd13affd640 10 monclient: dump: 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:36.383 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd1417fa640 4 set_mon_vals no callback set 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd14999d640 10 monclient: _renew_subs 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd14999d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.358+0000 7fd14999d640 1 -- 172.21.15.178:0/2889039537 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14406cd50 con 0x7fd124008df0 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.359+0000 7fd13affd640 1 -- 172.21.15.178:0/2889039537 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd13c015410 con 0x7fd124008df0 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.359+0000 7fd14999d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.359+0000 7fd14999d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:36.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.359+0000 7fd14999d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.359+0000 7fd14999d640 1 -- 172.21.15.178:0/2889039537 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14406e440 con 0x7fd124008df0 2024-09-15T01:45:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.359+0000 7fd13affd640 1 --2- 172.21.15.178:0/2889039537 >> v2:172.21.15.204:6812/28210 conn(0x7fd12403aa60 0x7fd12403cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.359+0000 7fd13affd640 1 -- 172.21.15.178:0/2889039537 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd13c04b9e0 con 0x7fd124008df0 2024-09-15T01:45:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.359+0000 7fd1427fc640 1 --2- 172.21.15.178:0/2889039537 >> v2:172.21.15.204:6812/28210 conn(0x7fd12403aa60 0x7fd12403cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.359+0000 7fd1427fc640 10 monclient: get_auth_request con 0x7fd12403aa60 auth_method 0 2024-09-15T01:45:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.360+0000 7fd1427fc640 1 --2- 172.21.15.178:0/2889039537 >> v2:172.21.15.204:6812/28210 conn(0x7fd12403aa60 0x7fd12403cf10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd12c006db0 tx=0x7fd12c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.364+0000 7fd13affd640 1 -- 172.21.15.178:0/2889039537 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd14406e440 con 0x7fd124008df0 2024-09-15T01:45:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.364+0000 7fd13affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:36.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.364+0000 7fd13affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:36.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.579+0000 7fd14999d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:45:36.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.579+0000 7fd14999d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:45:36.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.579+0000 7fd14999d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:36.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.579+0000 7fd14999d640 1 -- 172.21.15.178:0/2889039537 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd14406e440 con 0x7fd124008df0 2024-09-15T01:45:36.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.580+0000 7fd13affd640 1 -- 172.21.15.178:0/2889039537 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3172 (secure 0 0 0) 0x7fd14406e440 con 0x7fd124008df0 2024-09-15T01:45:36.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.580+0000 7fd13affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:45:36.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.580+0000 7fd13affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:36.582 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":130,"quorum":[0,1,3,4,5,6,8],"quorum_names":["a","f","g","c","h","d","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:45:36.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 1 -- 172.21.15.178:0/2889039537 >> v2:172.21.15.204:6812/28210 conn(0x7fd12403aa60 msgr2=0x7fd12403cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:36.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 1 --2- 172.21.15.178:0/2889039537 >> v2:172.21.15.204:6812/28210 conn(0x7fd12403aa60 0x7fd12403cf10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd12c006db0 tx=0x7fd12c008040 comp rx=0 tx=0).stop 2024-09-15T01:45:36.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 10 monclient: shutdown 2024-09-15T01:45:36.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:36.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 1 -- 172.21.15.178:0/2889039537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd124008df0 msgr2=0x7fd144135100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:36.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 1 --2- 172.21.15.178:0/2889039537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd124008df0 0x7fd144135100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd13c001910 tx=0x7fd13c00c040 comp rx=0 tx=0).stop 2024-09-15T01:45:36.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 1 -- 172.21.15.178:0/2889039537 shutdown_connections 2024-09-15T01:45:36.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 1 --2- 172.21.15.178:0/2889039537 >> v2:172.21.15.204:6812/28210 conn(0x7fd12403aa60 0x7fd12403cf10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:36.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 1 --2- 172.21.15.178:0/2889039537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd14413d740 0x7fd14413db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:36.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 1 --2- 172.21.15.178:0/2889039537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd14413e090 0x7fd14412da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:36.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 1 --2- 172.21.15.178:0/2889039537 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd124008df0 0x7fd144135100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:36.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 1 -- 172.21.15.178:0/2889039537 >> 172.21.15.178:0/2889039537 conn(0x7fd1440a22e0 msgr2=0x7fd144130d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:36.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 1 -- 172.21.15.178:0/2889039537 shutdown_connections 2024-09-15T01:45:36.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:36.582+0000 7fd14999d640 1 -- 172.21.15.178:0/2889039537 wait complete. 2024-09-15T01:45:39.596 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:45:39.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.699+0000 7f1f83c20640 10 monclient: get_monmap_and_config 2024-09-15T01:45:39.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.699+0000 7f1f83c20640 10 monclient: build_initial_monmap 2024-09-15T01:45:39.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.699+0000 7f1f83c20640 10 monclient: monmap: 2024-09-15T01:45:39.701 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:39.701 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:39.701 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:39.700064+0000 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:39.700064+0000 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:39.702 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:39.703 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:39.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.699+0000 7f1f83c20640 1 Processor -- start 2024-09-15T01:45:39.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.699+0000 7f1f83c20640 1 -- start start 2024-09-15T01:45:39.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.699+0000 7f1f83c20640 10 monclient: init 2024-09-15T01:45:39.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:39.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 10 monclient: _add_conns ranks=[9,1,6,5,0,3,2,10,8,7,4] 2024-09-15T01:45:39.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1f7c0a2930 0x7f1f7c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:39.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 10 monclient(hunting): picked mon.noname-a con 0x7f1f7c0a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:45:39.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1f7c09faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:45:39.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f7c09fe70 0x7f1f7c0a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:39.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 10 monclient(hunting): picked mon.noname-d con 0x7f1f7c09fe70 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:39.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c04bf60 con 0x7f1f7c09faa0 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c09dd80 con 0x7f1f7c0a2930 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0a2d00 con 0x7f1f7c09fe70 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f83c20640 10 monclient(hunting): authenticate will time out at 1544.450513s 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1f7c0a2930 0x7f1f7c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f821bd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f7c09fe70 0x7f1f7c0a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1f7c0a2930 0x7f1f7c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33516/0 (socket says 172.21.15.178:33516) 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f821bd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f7c09fe70 0x7f1f7c0a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57060/0 (socket says 172.21.15.178:57060) 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 1 -- 172.21.15.178:0/208679307 learned_addr learned my addr 172.21.15.178:0/208679307 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 10 monclient(hunting): get_auth_request con 0x7f1f7c0a2930 auth_method 0 2024-09-15T01:45:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f821bd640 10 monclient(hunting): get_auth_request con 0x7f1f7c09fe70 auth_method 0 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f821bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f821bd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f821bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f821bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f821bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f821bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:39.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 10 monclient(hunting): handle_auth_done global_id 64098 payload 274 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 1 -- 172.21.15.178:0/208679307 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f7c09fe70 msgr2=0x7f1f7c0a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 1 --2- 172.21.15.178:0/208679307 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f7c09fe70 0x7f1f7c0a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 1 -- 172.21.15.178:0/208679307 >> v1:172.21.15.204:6789/0 conn(0x7f1f7c09faa0 legacy=0x7f1f7c0a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 10 monclient: _finish_hunting 0 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 1 monclient: found mon.noname-a 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.700+0000 7f1f819bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f819bc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f819bc640 1 -- 172.21.15.178:0/208679307 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c133440 con 0x7f1f7c0a2930 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f821bd640 1 --2- 172.21.15.178:0/208679307 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f7c09fe70 0x7f1f7c0a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f819bc640 1 --2- 172.21.15.178:0/208679307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c0a2930 0x7f1f7c132de0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009910 tx=0x7f1f780091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac00faf438e23fb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f809ba640 1 -- 172.21.15.178:0/208679307 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f7802ad20 con 0x7f1f7c0a2930 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f809ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f809ba640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:45:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f809ba640 10 monclient: dump: 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:39.716 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f809ba640 10 monclient: _finish_auth 0 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f809ba640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f809ba640 1 -- 172.21.15.178:0/208679307 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f78031820 con 0x7f1f7c0a2930 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f83c20640 5 monclient: authenticate success, global_id 64098 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f83c20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f809ba640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f83c20640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f809ba640 1 -- 172.21.15.178:0/208679307 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f7802aea0 con 0x7f1f7c0a2930 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f809ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f83c20640 4 set_mon_vals no callback set 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f809ba640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.701+0000 7f1f809ba640 10 monclient: dump: 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.702+0000 7f1f83c20640 10 monclient: shutdown 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.702+0000 7f1f83c20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.702+0000 7f1f83c20640 1 -- 172.21.15.178:0/208679307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c0a2930 msgr2=0x7f1f7c132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.702+0000 7f1f83c20640 1 --2- 172.21.15.178:0/208679307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c0a2930 0x7f1f7c132de0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009910 tx=0x7f1f780091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.702+0000 7f1f83c20640 1 -- 172.21.15.178:0/208679307 shutdown_connections 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.702+0000 7f1f83c20640 1 --2- 172.21.15.178:0/208679307 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1f7c09fe70 0x7f1f7c0a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.702+0000 7f1f83c20640 1 --2- 172.21.15.178:0/208679307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c0a2930 0x7f1f7c132de0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.702+0000 7f1f83c20640 1 -- 172.21.15.178:0/208679307 >> 172.21.15.178:0/208679307 conn(0x7f1f7c0a5ed0 msgr2=0x7f1f7c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.702+0000 7f1f83c20640 1 -- 172.21.15.178:0/208679307 shutdown_connections 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.702+0000 7f1f83c20640 1 -- 172.21.15.178:0/208679307 wait complete. 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 10 monclient: build_initial_monmap 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 10 monclient: monmap: 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:39.720 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 1 Processor -- start 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 1 -- start start 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 10 monclient: init 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 10 monclient: _add_conns ranks=[0,4,3,7,8,5,2,1,6] 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c09faa0 0x7f1f7c09b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:39.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 10 monclient(hunting): picked mon.noname-a con 0x7f1f7c09faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f7c09fe70 0x7f1f7c131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 10 monclient(hunting): picked mon.noname-c con 0x7f1f7c09fe70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f7c0a2930 0x7f1f7c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 10 monclient(hunting): picked mon.noname-g con 0x7f1f7c0a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c09d630 con 0x7f1f7c09faa0 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c142b90 con 0x7f1f7c0a2930 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.703+0000 7f1f83c20640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c11ec20 con 0x7f1f7c09fe70 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f83c20640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f83c20640 10 monclient(hunting): authenticate will time out at 1544.454013s 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c09faa0 0x7f1f7c09b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c09faa0 0x7f1f7c09b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33520/0 (socket says 172.21.15.178:33520) 2024-09-15T01:45:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 1 -- 172.21.15.178:0/382900772 learned_addr learned my addr 172.21.15.178:0/382900772 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 10 monclient(hunting): get_auth_request con 0x7f1f7c09faa0 auth_method 0 2024-09-15T01:45:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f811bb640 1 --2- 172.21.15.178:0/382900772 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f7c09fe70 0x7f1f7c131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f811bb640 10 monclient(hunting): get_auth_request con 0x7f1f7c09fe70 auth_method 0 2024-09-15T01:45:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f811bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f811bb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f811bb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f821bd640 1 --2- 172.21.15.178:0/382900772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f7c0a2930 0x7f1f7c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f811bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f811bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f811bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f821bd640 10 monclient(hunting): get_auth_request con 0x7f1f7c0a2930 auth_method 0 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f821bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f821bd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f821bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f821bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f821bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f821bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:39.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 10 monclient(hunting): handle_auth_done global_id 64107 payload 824 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 1 -- 172.21.15.178:0/382900772 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f7c09fe70 msgr2=0x7f1f7c131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 1 --2- 172.21.15.178:0/382900772 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f7c09fe70 0x7f1f7c131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 1 -- 172.21.15.178:0/382900772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f7c0a2930 msgr2=0x7f1f7c11c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 1 --2- 172.21.15.178:0/382900772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f7c0a2930 0x7f1f7c11c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 10 monclient: _finish_hunting 0 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 1 monclient: found mon.noname-a 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 1 -- 172.21.15.178:0/382900772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74009a20 con 0x7f1f7c09faa0 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f811bb640 1 --2- 172.21.15.178:0/382900772 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f7c09fe70 0x7f1f7c131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f819bc640 1 --2- 172.21.15.178:0/382900772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c09faa0 0x7f1f7c09b030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f78000c00 tx=0x7f1f78032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f71ffb640 1 -- 172.21.15.178:0/382900772 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f78033d30 con 0x7f1f7c09faa0 2024-09-15T01:45:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f71ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f71ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f71ffb640 10 monclient: dump: 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:39.726 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f71ffb640 10 monclient: _finish_auth 0 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f71ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f71ffb640 1 -- 172.21.15.178:0/382900772 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7803b3f0 con 0x7f1f7c09faa0 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f71ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f71ffb640 1 -- 172.21.15.178:0/382900772 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f7803b9e0 con 0x7f1f7c09faa0 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f83c20640 5 monclient: authenticate success, global_id 64107 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f83c20640 10 monclient: _renew_subs 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f83c20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:45:39.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f83c20640 1 -- 172.21.15.178:0/382900772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f78009450 con 0x7f1f7c09faa0 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f71ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.704+0000 7f1f73fff640 4 set_mon_vals no callback set 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.705+0000 7f1f71ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.705+0000 7f1f71ffb640 10 monclient: dump: 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.705+0000 7f1f83c20640 10 monclient: _renew_subs 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.705+0000 7f1f83c20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.705+0000 7f1f83c20640 1 -- 172.21.15.178:0/382900772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c09bc00 con 0x7f1f7c09faa0 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.706+0000 7f1f71ffb640 1 -- 172.21.15.178:0/382900772 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1f78039040 con 0x7f1f7c09faa0 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.706+0000 7f1f71ffb640 1 --2- 172.21.15.178:0/382900772 >> v2:172.21.15.204:6812/28210 conn(0x7f1f600397b0 0x7f1f6003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.706+0000 7f1f83c20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.706+0000 7f1f83c20640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:39.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.706+0000 7f1f83c20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:45:39.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.706+0000 7f1f71ffb640 1 -- 172.21.15.178:0/382900772 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f1f7806b560 con 0x7f1f7c09faa0 2024-09-15T01:45:39.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.706+0000 7f1f83c20640 1 -- 172.21.15.178:0/382900772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c04b720 con 0x7f1f7c09faa0 2024-09-15T01:45:39.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.706+0000 7f1f811bb640 1 --2- 172.21.15.178:0/382900772 >> v2:172.21.15.204:6812/28210 conn(0x7f1f600397b0 0x7f1f6003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:39.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.706+0000 7f1f811bb640 10 monclient: get_auth_request con 0x7f1f600397b0 auth_method 0 2024-09-15T01:45:39.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.706+0000 7f1f811bb640 1 --2- 172.21.15.178:0/382900772 >> v2:172.21.15.204:6812/28210 conn(0x7f1f600397b0 0x7f1f6003bc60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c007fd0 tx=0x7f1f6c009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:39.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.710+0000 7f1f71ffb640 1 -- 172.21.15.178:0/382900772 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1f78037080 con 0x7f1f7c09faa0 2024-09-15T01:45:39.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.710+0000 7f1f71ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:39.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.710+0000 7f1f71ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:39.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.926+0000 7f1f83c20640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:45:39.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.926+0000 7f1f83c20640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:45:39.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.926+0000 7f1f83c20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:45:39.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.926+0000 7f1f83c20640 1 -- 172.21.15.178:0/382900772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1f7c09cef0 con 0x7f1f7c09faa0 2024-09-15T01:45:39.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.927+0000 7f1f71ffb640 1 -- 172.21.15.178:0/382900772 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7f1f78042080 con 0x7f1f7c09faa0 2024-09-15T01:45:39.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.927+0000 7f1f71ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:45:39.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.927+0000 7f1f71ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:39.930 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":138,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":2,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:45:39.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 1 -- 172.21.15.178:0/382900772 >> v2:172.21.15.204:6812/28210 conn(0x7f1f600397b0 msgr2=0x7f1f6003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:39.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 1 --2- 172.21.15.178:0/382900772 >> v2:172.21.15.204:6812/28210 conn(0x7f1f600397b0 0x7f1f6003bc60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c007fd0 tx=0x7f1f6c009040 comp rx=0 tx=0).stop 2024-09-15T01:45:39.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 10 monclient: shutdown 2024-09-15T01:45:39.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:39.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 1 -- 172.21.15.178:0/382900772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c09faa0 msgr2=0x7f1f7c09b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:39.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 1 --2- 172.21.15.178:0/382900772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c09faa0 0x7f1f7c09b030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f78000c00 tx=0x7f1f78032000 comp rx=0 tx=0).stop 2024-09-15T01:45:39.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 1 -- 172.21.15.178:0/382900772 shutdown_connections 2024-09-15T01:45:39.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 1 --2- 172.21.15.178:0/382900772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f7c0a2930 0x7f1f7c11c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:39.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 1 --2- 172.21.15.178:0/382900772 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f1f7c09fe70 0x7f1f7c131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:39.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 1 --2- 172.21.15.178:0/382900772 >> v2:172.21.15.204:6812/28210 conn(0x7f1f600397b0 0x7f1f6003bc60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:39.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 1 --2- 172.21.15.178:0/382900772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f7c09faa0 0x7f1f7c09b030 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:39.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 1 -- 172.21.15.178:0/382900772 >> 172.21.15.178:0/382900772 conn(0x7f1f7c0a5ed0 msgr2=0x7f1f7c0a50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:39.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 1 -- 172.21.15.178:0/382900772 shutdown_connections 2024-09-15T01:45:39.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:39.929+0000 7f1f83c20640 1 -- 172.21.15.178:0/382900772 wait complete. 2024-09-15T01:45:39.942 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:45:39.942 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:45:39.942 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:45:40.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.032+0000 7fd208a0f640 10 monclient: get_monmap_and_config 2024-09-15T01:45:40.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.032+0000 7fd208a0f640 10 monclient: build_initial_monmap 2024-09-15T01:45:40.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.032+0000 7fd208a0f640 10 monclient: monmap: 2024-09-15T01:45:40.036 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:40.036 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.036 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:40.033606+0000 2024-09-15T01:45:40.036 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:40.033606+0000 2024-09-15T01:45:40.036 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:40.036 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.036 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:40.036 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:40.036 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:40.036 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:40.036 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:40.037 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:40.037 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:40.037 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:40.037 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:40.037 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:40.037 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:40.037 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 1 Processor -- start 2024-09-15T01:45:40.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 1 -- start start 2024-09-15T01:45:40.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 10 monclient: init 2024-09-15T01:45:40.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:40.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 10 monclient: _add_conns ranks=[8,5,10,1,9,0,3,6,2,4,7] 2024-09-15T01:45:40.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2040a2930 0x7fd204132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 10 monclient(hunting): picked mon.noname-e con 0x7fd2040a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:40.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd20409faa0 0x7fd20409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 10 monclient(hunting): picked mon.noname-h con 0x7fd20409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:40.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd2040a08c0 0x7fd2040a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 10 monclient(hunting): picked mon.noname-f con 0x7fd2040a08c0 addr v2:172.21.15.204:3300/0 2024-09-15T01:45:40.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd20409e0f0 con 0x7fd2040a08c0 2024-09-15T01:45:40.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd20409dd80 con 0x7fd20409faa0 2024-09-15T01:45:40.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040a2d00 con 0x7fd2040a2930 2024-09-15T01:45:40.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:40.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.033+0000 7fd208a0f640 10 monclient(hunting): authenticate will time out at 1544.783957s 2024-09-15T01:45:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2040a2930 0x7fd204132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2040a2930 0x7fd204132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60126/0 (socket says 172.21.15.178:60126) 2024-09-15T01:45:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 1 -- 172.21.15.178:0/1014940491 learned_addr learned my addr 172.21.15.178:0/1014940491 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 10 monclient(hunting): get_auth_request con 0x7fd2040a2930 auth_method 0 2024-09-15T01:45:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd202d9d640 1 --2- 172.21.15.178:0/1014940491 >> v2:172.21.15.204:3300/0 conn(0x7fd2040a08c0 0x7fd2040a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd201d9b640 1 --2- 172.21.15.178:0/1014940491 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd20409faa0 0x7fd20409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd202d9d640 10 monclient(hunting): get_auth_request con 0x7fd2040a08c0 auth_method 0 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd202d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd202d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd202d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd201d9b640 10 monclient(hunting): get_auth_request con 0x7fd20409faa0 auth_method 0 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd201d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd201d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd201d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd202d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd202d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd202d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd201d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd201d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd201d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 10 monclient(hunting): handle_auth_done global_id 34955 payload 274 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 1 -- 172.21.15.178:0/1014940491 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd20409faa0 msgr2=0x7fd20409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 1 --2- 172.21.15.178:0/1014940491 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd20409faa0 0x7fd20409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 1 -- 172.21.15.178:0/1014940491 >> v2:172.21.15.204:3300/0 conn(0x7fd2040a08c0 msgr2=0x7fd2040a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 1 --2- 172.21.15.178:0/1014940491 >> v2:172.21.15.204:3300/0 conn(0x7fd2040a08c0 0x7fd2040a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 10 monclient: _finish_hunting 0 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 1 monclient: found mon.noname-e 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.034+0000 7fd20259c640 1 -- 172.21.15.178:0/1014940491 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20404bf60 con 0x7fd2040a2930 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20259c640 1 --2- 172.21.15.178:0/1014940491 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2040a2930 0x7fd204132de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009910 tx=0x7fd1f40091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c55b1fb42d93a0f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20159a640 1 -- 172.21.15.178:0/1014940491 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1f402ac80 con 0x7fd2040a2930 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20159a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20159a640 10 monclient: dump: 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.053 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20159a640 10 monclient: _finish_auth 0 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 5 monclient: authenticate success, global_id 34955 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20159a640 1 -- 172.21.15.178:0/1014940491 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f402ae00 con 0x7fd2040a2930 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:40.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20159a640 1 -- 172.21.15.178:0/1014940491 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1f4031ca0 con 0x7fd2040a2930 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 4 set_mon_vals no callback set 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20159a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd20159a640 10 monclient: dump: 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.055 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 10 monclient: shutdown 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 1 -- 172.21.15.178:0/1014940491 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2040a2930 msgr2=0x7fd204132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 1 --2- 172.21.15.178:0/1014940491 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2040a2930 0x7fd204132de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009910 tx=0x7fd1f40091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 1 -- 172.21.15.178:0/1014940491 shutdown_connections 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 1 --2- 172.21.15.178:0/1014940491 >> v2:172.21.15.204:3300/0 conn(0x7fd2040a08c0 0x7fd2040a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 1 --2- 172.21.15.178:0/1014940491 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd20409faa0 0x7fd20409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 1 --2- 172.21.15.178:0/1014940491 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd2040a2930 0x7fd204132de0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.035+0000 7fd208a0f640 1 -- 172.21.15.178:0/1014940491 >> 172.21.15.178:0/1014940491 conn(0x7fd2040a5ed0 msgr2=0x7fd2040a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:40.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.036+0000 7fd208a0f640 1 -- 172.21.15.178:0/1014940491 shutdown_connections 2024-09-15T01:45:40.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.036+0000 7fd208a0f640 1 -- 172.21.15.178:0/1014940491 wait complete. 2024-09-15T01:45:40.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.036+0000 7fd208a0f640 10 monclient: build_initial_monmap 2024-09-15T01:45:40.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.036+0000 7fd208a0f640 10 monclient: monmap: 2024-09-15T01:45:40.057 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:40.057 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:40.057 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:40.057 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:40.057 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:40.057 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.036+0000 7fd208a0f640 1 Processor -- start 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.036+0000 7fd208a0f640 1 -- start start 2024-09-15T01:45:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.036+0000 7fd208a0f640 10 monclient: init 2024-09-15T01:45:40.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:40.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 10 monclient: _add_conns ranks=[3,0,1,6,5,4,2,7,8] 2024-09-15T01:45:40.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd20409faa0 0x7fd20409b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 10 monclient(hunting): picked mon.noname-g con 0x7fd20409faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:45:40.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2040a08c0 0x7fd2041317a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 10 monclient(hunting): picked mon.noname-a con 0x7fd2040a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:45:40.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2040a2930 0x7fd20411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 10 monclient(hunting): picked mon.noname-f con 0x7fd2040a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:45:40.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204142bc0 con 0x7fd2040a08c0 2024-09-15T01:45:40.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040a2390 con 0x7fd2040a2930 2024-09-15T01:45:40.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20409d630 con 0x7fd20409faa0 2024-09-15T01:45:40.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:40.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd208a0f640 10 monclient(hunting): authenticate will time out at 1544.787272s 2024-09-15T01:45:40.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2040a08c0 0x7fd2041317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2040a08c0 0x7fd2041317a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33530/0 (socket says 172.21.15.178:33530) 2024-09-15T01:45:40.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 1 -- 172.21.15.178:0/2956570423 learned_addr learned my addr 172.21.15.178:0/2956570423 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:40.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 10 monclient(hunting): get_auth_request con 0x7fd2040a08c0 auth_method 0 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd202d9d640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2040a2930 0x7fd20411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd20259c640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd20409faa0 0x7fd20409b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd202d9d640 10 monclient(hunting): get_auth_request con 0x7fd2040a2930 auth_method 0 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd202d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd202d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd202d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd20259c640 10 monclient(hunting): get_auth_request con 0x7fd20409faa0 auth_method 0 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd20259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd20259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd20259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd202d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd202d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd202d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd20259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd20259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd20259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 10 monclient(hunting): handle_auth_done global_id 64116 payload 824 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 1 -- 172.21.15.178:0/2956570423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd20409faa0 msgr2=0x7fd20409b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd20409faa0 0x7fd20409b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 1 -- 172.21.15.178:0/2956570423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2040a2930 msgr2=0x7fd20411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2040a2930 0x7fd20411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 1 monclient: found mon.noname-a 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 1 -- 172.21.15.178:0/2956570423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f4009450 con 0x7fd2040a08c0 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd201d9b640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2040a08c0 0x7fd2041317a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800e8d0 tx=0x7fd1f80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1f8009480 con 0x7fd2040a08c0 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd1f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd1f27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.037+0000 7fd1f27fc640 10 monclient: dump: 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.063 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd1f27fc640 10 monclient: _finish_auth 0 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd1f27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f800cd60 con 0x7fd2040a08c0 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd1f27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1f80138f0 con 0x7fd2040a08c0 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd208a0f640 5 monclient: authenticate success, global_id 64116 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd1f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd1f27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd200d99640 4 set_mon_vals no callback set 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd1f27fc640 10 monclient: dump: 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.065 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd208a0f640 10 monclient: _renew_subs 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd208a0f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd208a0f640 1 -- 172.21.15.178:0/2956570423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20411eba0 con 0x7fd2040a08c0 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd208a0f640 10 monclient: _renew_subs 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd208a0f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd208a0f640 1 -- 172.21.15.178:0/2956570423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20411d960 con 0x7fd2040a08c0 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd208a0f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd208a0f640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd204131ce0 0x7fd20411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd208a0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd208a0f640 1 -- 172.21.15.178:0/2956570423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20411ea60 con 0x7fd204131ce0 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd1f8013470 con 0x7fd2040a08c0 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd20259c640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd204131ce0 0x7fd20411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd20259c640 10 monclient: get_auth_request con 0x7fd204131ce0 auth_method 0 2024-09-15T01:45:40.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd20259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd20259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.038+0000 7fd20259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd20259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd20259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd20259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 1 --2- 172.21.15.178:0/2956570423 >> v2:172.21.15.204:6812/28210 conn(0x7fd1e40396e0 0x7fd1e403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd1f804bd50 con 0x7fd2040a08c0 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd20259c640 10 monclient(hunting): handle_auth_done global_id 64125 payload 824 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd20259c640 1 -- 172.21.15.178:0/2956570423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd20409bc50 con 0x7fd204131ce0 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd202d9d640 1 --2- 172.21.15.178:0/2956570423 >> v2:172.21.15.204:6812/28210 conn(0x7fd1e40396e0 0x7fd1e403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd202d9d640 10 monclient: get_auth_request con 0x7fd1e40396e0 auth_method 0 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd20259c640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd204131ce0 0x7fd20411ca70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd2040a3290 tx=0x7fd1f4032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1f402ac80 con 0x7fd204131ce0 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 10 monclient: dump: 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fd1f403d690 con 0x7fd204131ce0 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd204131ce0 msgr2=0x7fd20411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd204131ce0 0x7fd20411ca70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd2040a3290 tx=0x7fd1f4032000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd204131ce0 msgr2=0x7fd20411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd1f27fc640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd204131ce0 0x7fd20411ca70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.039+0000 7fd202d9d640 1 --2- 172.21.15.178:0/2956570423 >> v2:172.21.15.204:6812/28210 conn(0x7fd1e40396e0 0x7fd1e403bb90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009fd0 tx=0x7fd1fc00b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd208a0f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd208a0f640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2041370c0 0x7fd20411cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd208a0f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd208a0f640 1 -- 172.21.15.178:0/2956570423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204070d50 con 0x7fd2041370c0 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd20259c640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2041370c0 0x7fd20411cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd20259c640 10 monclient: get_auth_request con 0x7fd2041370c0 auth_method 0 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd20259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd20259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd20259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd20259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd20259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd20259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd20259c640 10 monclient(hunting): handle_auth_done global_id 64134 payload 824 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd20259c640 1 -- 172.21.15.178:0/2956570423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd20404bc80 con 0x7fd2041370c0 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd20259c640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2041370c0 0x7fd20411cfb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009910 tx=0x7fd1f403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.045+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1f403de40 con 0x7fd2041370c0 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd1f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd1f27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd1f27fc640 10 monclient: dump: 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fd1f40453e0 con 0x7fd2041370c0 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd1f27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd1f27fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2041370c0 msgr2=0x7fd20411cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd1f27fc640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2041370c0 0x7fd20411cfb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009910 tx=0x7fd1f403d000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd1f27fc640 1 -- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2041370c0 msgr2=0x7fd20411cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:40.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd1f27fc640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2041370c0 0x7fd20411cfb0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009910 tx=0x7fd1f403d000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.074 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:40.074 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:40.074 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:40.074 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:40.074 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:40.074 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:40.074 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:40.075 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:40.076 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:40.077 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:40.078 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.079 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.080 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:40.081 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.084 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.087 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.088 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.089 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:40.089 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.089 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.089 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:40.089 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.089 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.089 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.090 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.091 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.092 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:40.093 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.094 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.095 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:40.096 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd208a0f640 1 -- 172.21.15.178:0/2956570423 >> v2:172.21.15.204:6812/28210 conn(0x7fd1e40396e0 msgr2=0x7fd1e403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd208a0f640 1 --2- 172.21.15.178:0/2956570423 >> v2:172.21.15.204:6812/28210 conn(0x7fd1e40396e0 0x7fd1e403bb90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009fd0 tx=0x7fd1fc00b040 comp rx=0 tx=0).stop 2024-09-15T01:45:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd208a0f640 10 monclient: shutdown 2024-09-15T01:45:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd208a0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd208a0f640 1 -- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2040a08c0 msgr2=0x7fd2041317a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd208a0f640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2040a08c0 0x7fd2041317a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800e8d0 tx=0x7fd1f80090f0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd202d9d640 1 -- 172.21.15.178:0/2956570423 reap_dead start 2024-09-15T01:45:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd208a0f640 1 -- 172.21.15.178:0/2956570423 shutdown_connections 2024-09-15T01:45:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd208a0f640 1 --2- 172.21.15.178:0/2956570423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2040a08c0 0x7fd2041317a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd208a0f640 1 -- 172.21.15.178:0/2956570423 >> 172.21.15.178:0/2956570423 conn(0x7fd2040a5ed0 msgr2=0x7fd20409e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd208a0f640 1 -- 172.21.15.178:0/2956570423 shutdown_connections 2024-09-15T01:45:40.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.046+0000 7fd208a0f640 1 -- 172.21.15.178:0/2956570423 wait complete. 2024-09-15T01:45:40.099 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:40.099 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:45:40.099 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:45:40.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.191+0000 7f896d55d640 10 monclient: get_monmap_and_config 2024-09-15T01:45:40.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.191+0000 7f896d55d640 10 monclient: build_initial_monmap 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.191+0000 7f896d55d640 10 monclient: monmap: 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:40.192512+0000 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:40.192512+0000 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:40.195 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:40.195 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:40.195 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:40.195 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:40.195 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 1 Processor -- start 2024-09-15T01:45:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 1 -- start start 2024-09-15T01:45:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 10 monclient: init 2024-09-15T01:45:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 10 monclient: _add_conns ranks=[1,10,6,3,5,9,0,8,2,4,7] 2024-09-15T01:45:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f89680a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:45:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8968132de0 0x7f8968137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 10 monclient(hunting): picked mon.noname-f con 0x7f8968132de0 addr v2:172.21.15.204:3300/0 2024-09-15T01:45:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 0x7f896813a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 10 monclient(hunting): picked mon.noname-d con 0x7f89681380e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f896804bf60 con 0x7f89680a2930 2024-09-15T01:45:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896809dd80 con 0x7f8968132de0 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f89680a2d00 con 0x7f89681380e0 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.192+0000 7f896d55d640 10 monclient(hunting): authenticate will time out at 1544.942982s 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 0x7f896813a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 0x7f896813a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57068/0 (socket says 172.21.15.178:57068) 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 1 -- 172.21.15.178:0/845403917 learned_addr learned my addr 172.21.15.178:0/845403917 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 10 monclient(hunting): get_auth_request con 0x7f89681380e0 auth_method 0 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f895ffff640 1 --2- 172.21.15.178:0/845403917 >> v2:172.21.15.204:3300/0 conn(0x7f8968132de0 0x7f8968137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f895ffff640 10 monclient(hunting): get_auth_request con 0x7f8968132de0 auth_method 0 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f895ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f895ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f895ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 10 monclient(hunting): handle_auth_done global_id 25413 payload 274 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 1 -- 172.21.15.178:0/845403917 >> v2:172.21.15.204:3300/0 conn(0x7f8968132de0 msgr2=0x7f8968137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 1 --2- 172.21.15.178:0/845403917 >> v2:172.21.15.204:3300/0 conn(0x7f8968132de0 0x7f8968137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 1 -- 172.21.15.178:0/845403917 >> v1:172.21.15.204:6789/0 conn(0x7f89680a2930 legacy=0x7f896809faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 10 monclient: _finish_hunting 0 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 1 monclient: found mon.noname-d 2024-09-15T01:45:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f89677fe640 1 -- 172.21.15.178:0/845403917 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89681331b0 con 0x7f89681380e0 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.193+0000 7f895ffff640 1 --2- 172.21.15.178:0/845403917 >> v2:172.21.15.204:3300/0 conn(0x7f8968132de0 0x7f8968137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89677fe640 1 --2- 172.21.15.178:0/845403917 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 0x7f896813a4c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8960009f20 tx=0x7f8960009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=39df919ba223a9c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89667fc640 1 -- 172.21.15.178:0/845403917 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8960007880 con 0x7f89681380e0 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89667fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89667fc640 10 monclient: dump: 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89667fc640 10 monclient: _finish_auth 0 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89667fc640 1 -- 172.21.15.178:0/845403917 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896002d050 con 0x7f89681380e0 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 5 monclient: authenticate success, global_id 25413 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89667fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89667fc640 1 -- 172.21.15.178:0/845403917 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8960030360 con 0x7f89681380e0 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 4 set_mon_vals no callback set 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89667fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f89667fc640 10 monclient: dump: 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.210 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 10 monclient: shutdown 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 1 -- 172.21.15.178:0/845403917 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 msgr2=0x7f896813a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 1 --2- 172.21.15.178:0/845403917 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 0x7f896813a4c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8960009f20 tx=0x7f8960009840 comp rx=0 tx=0).stop 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 1 -- 172.21.15.178:0/845403917 shutdown_connections 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 1 --2- 172.21.15.178:0/845403917 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 0x7f896813a4c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 1 --2- 172.21.15.178:0/845403917 >> v2:172.21.15.204:3300/0 conn(0x7f8968132de0 0x7f8968137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 1 -- 172.21.15.178:0/845403917 >> 172.21.15.178:0/845403917 conn(0x7f89680a5ed0 msgr2=0x7f89680a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 1 -- 172.21.15.178:0/845403917 shutdown_connections 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.194+0000 7f896d55d640 1 -- 172.21.15.178:0/845403917 wait complete. 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.195+0000 7f896d55d640 10 monclient: build_initial_monmap 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.195+0000 7f896d55d640 10 monclient: monmap: 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:40.212 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.195+0000 7f896d55d640 1 Processor -- start 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.195+0000 7f896d55d640 1 -- start start 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.195+0000 7f896d55d640 10 monclient: init 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient: _add_conns ranks=[1,2,6,8,0,5,3,7,4] 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89680a2930 0x7f89681338b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient(hunting): picked mon.noname-f con 0x7f89680a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8968132de0 0x7f8968133df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient(hunting): picked mon.noname-b con 0x7f8968132de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 0x7f8968134330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient(hunting): picked mon.noname-d con 0x7f89681380e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89680a2390 con 0x7f89680a2930 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896809d630 con 0x7f8968132de0 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8968136a80 con 0x7f89681380e0 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient(hunting): authenticate will time out at 1544.946095s 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 0x7f8968134330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 0x7f8968134330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57074/0 (socket says 172.21.15.178:57074) 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 1 -- 172.21.15.178:0/1242356842 learned_addr learned my addr 172.21.15.178:0/1242356842 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 10 monclient(hunting): get_auth_request con 0x7f89681380e0 auth_method 0 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f8966ffd640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89680a2930 0x7f89681338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895ffff640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8968132de0 0x7f8968133df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f8966ffd640 10 monclient(hunting): get_auth_request con 0x7f89680a2930 auth_method 0 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f8966ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f8966ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f8966ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895ffff640 10 monclient(hunting): get_auth_request con 0x7f8968132de0 auth_method 0 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f8966ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f8966ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f8966ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 10 monclient(hunting): handle_auth_done global_id 25422 payload 824 2024-09-15T01:45:40.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 1 -- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8968132de0 msgr2=0x7f8968133df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8968132de0 0x7f8968133df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 1 -- 172.21.15.178:0/1242356842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89680a2930 msgr2=0x7f89681338b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89680a2930 0x7f89681338b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 10 monclient: _finish_hunting 0 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 1 monclient: found mon.noname-d 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 1 -- 172.21.15.178:0/1242356842 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8960009ae0 con 0x7f89681380e0 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f89677fe640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 0x7f8968134330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8960009880 tx=0x7f8960036000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8960009520 con 0x7f89681380e0 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:45:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895effd640 10 monclient: dump: 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895effd640 10 monclient: _finish_auth 0 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 5 monclient: authenticate success, global_id 25422 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient: _renew_subs 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896003cab0 con 0x7f89681380e0 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 1 -- 172.21.15.178:0/1242356842 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896809c730 con 0x7f89681380e0 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f896003cc20 con 0x7f89681380e0 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:45:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f895effd640 10 monclient: dump: 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.220 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.221 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.221 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f8965ffb640 4 set_mon_vals no callback set 2024-09-15T01:45:40.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient: _renew_subs 2024-09-15T01:45:40.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:45:40.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.196+0000 7f896d55d640 1 -- 172.21.15.178:0/1242356842 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896812fb30 con 0x7f89681380e0 2024-09-15T01:45:40.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8960003040 con 0x7f89681380e0 2024-09-15T01:45:40.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f896d55d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:40.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f896d55d640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681357b0 0x7f8968134870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f896d55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f896d55d640 1 -- 172.21.15.178:0/1242356842 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89681368b0 con 0x7f89681357b0 2024-09-15T01:45:40.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f8966ffd640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681357b0 0x7f8968134870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f8966ffd640 10 monclient: get_auth_request con 0x7f89681357b0 auth_method 0 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f895effd640 1 --2- 172.21.15.178:0/1242356842 >> v2:172.21.15.204:6812/28210 conn(0x7f89480397e0 0x7f894803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f8966ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f8966ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f8966ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f896006c150 con 0x7f89681380e0 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f8966ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f8966ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.197+0000 7f8966ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f8966ffd640 10 monclient(hunting): handle_auth_done global_id 34922 payload 824 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f8966ffd640 1 -- 172.21.15.178:0/1242356842 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f896804c4a0 con 0x7f89681357b0 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895ffff640 1 --2- 172.21.15.178:0/1242356842 >> v2:172.21.15.204:6812/28210 conn(0x7f89480397e0 0x7f894803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895ffff640 10 monclient: get_auth_request con 0x7f89480397e0 auth_method 0 2024-09-15T01:45:40.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f8966ffd640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681357b0 0x7f8968134870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f895800f780 tx=0x7f8958007e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f895800fe50 con 0x7f89681357b0 2024-09-15T01:45:40.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:40.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895effd640 10 monclient: dump: 2024-09-15T01:45:40.223 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.223 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.223 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.223 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.223 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.223 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.223 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.223 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f89580028a0 con 0x7f89681357b0 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895ffff640 1 --2- 172.21.15.178:0/1242356842 >> v2:172.21.15.204:6812/28210 conn(0x7f89480397e0 0x7f894803bc90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8950006e70 tx=0x7f8950008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681357b0 msgr2=0x7f8968134870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895effd640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681357b0 0x7f8968134870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f895800f780 tx=0x7f8958007e30 comp rx=0 tx=0).stop 2024-09-15T01:45:40.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681357b0 msgr2=0x7f8968134870 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:40.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.198+0000 7f895effd640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681357b0 0x7f8968134870 secure :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f895800f780 tx=0x7f8958007e30 comp rx=0 tx=0).stop 2024-09-15T01:45:40.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f896d55d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f896d55d640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681311b0 0x7f89681315b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f896d55d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f896d55d640 1 -- 172.21.15.178:0/1242356842 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896804bc80 con 0x7f89681311b0 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f8966ffd640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681311b0 0x7f89681315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f8966ffd640 10 monclient: get_auth_request con 0x7f89681311b0 auth_method 0 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f8966ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f8966ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f8966ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f8966ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f8966ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f8966ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f8966ffd640 10 monclient(hunting): handle_auth_done global_id 34931 payload 824 2024-09-15T01:45:40.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f8966ffd640 1 -- 172.21.15.178:0/1242356842 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f896805ed70 con 0x7f89681311b0 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.204+0000 7f8966ffd640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681311b0 0x7f89681315b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8958000fb0 tx=0x7f8958019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89580076e0 con 0x7f89681311b0 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f895effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f895effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f895effd640 10 monclient: dump: 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.228 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f8958007260 con 0x7f89681311b0 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f895effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f895effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681311b0 msgr2=0x7f89681315b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f895effd640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681311b0 0x7f89681315b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8958000fb0 tx=0x7f8958019000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f895effd640 1 -- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681311b0 msgr2=0x7f89681315b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:40.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f895effd640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89681311b0 0x7f89681315b0 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8958000fb0 tx=0x7f8958019000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.230 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:40.230 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:40.230 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:40.230 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:40.230 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:40.230 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:40.230 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:40.230 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.231 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.232 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:40.235 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.236 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.237 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.238 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.240 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.241 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:40.243 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.245 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.246 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:40.248 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:40.249 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.250 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.251 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:40.252 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f896d55d640 1 -- 172.21.15.178:0/1242356842 >> v2:172.21.15.204:6812/28210 conn(0x7f89480397e0 msgr2=0x7f894803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f896d55d640 1 --2- 172.21.15.178:0/1242356842 >> v2:172.21.15.204:6812/28210 conn(0x7f89480397e0 0x7f894803bc90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8950006e70 tx=0x7f8950008040 comp rx=0 tx=0).stop 2024-09-15T01:45:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f896d55d640 10 monclient: shutdown 2024-09-15T01:45:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f896d55d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f896d55d640 1 -- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 msgr2=0x7f8968134330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f896d55d640 1 --2- 172.21.15.178:0/1242356842 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f89681380e0 0x7f8968134330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8960009880 tx=0x7f8960036000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f89677fe640 1 -- 172.21.15.178:0/1242356842 reap_dead start 2024-09-15T01:45:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f89677fe640 1 -- 172.21.15.178:0/1242356842 reap_dead start 2024-09-15T01:45:40.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f896d55d640 1 -- 172.21.15.178:0/1242356842 shutdown_connections 2024-09-15T01:45:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f896d55d640 1 -- 172.21.15.178:0/1242356842 >> 172.21.15.178:0/1242356842 conn(0x7f89680a5ed0 msgr2=0x7f89680a4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f896d55d640 1 -- 172.21.15.178:0/1242356842 shutdown_connections 2024-09-15T01:45:40.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.205+0000 7f896d55d640 1 -- 172.21.15.178:0/1242356842 wait complete. 2024-09-15T01:45:40.255 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:40.255 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:45:40.255 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:45:40.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.347+0000 7fbda9b14640 10 monclient: get_monmap_and_config 2024-09-15T01:45:40.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.347+0000 7fbda9b14640 10 monclient: build_initial_monmap 2024-09-15T01:45:40.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.347+0000 7fbda9b14640 10 monclient: monmap: 2024-09-15T01:45:40.350 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:40.350 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.350 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:40.348468+0000 2024-09-15T01:45:40.350 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:40.348468+0000 2024-09-15T01:45:40.350 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:40.351 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.351 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:40.351 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:40.351 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:40.351 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:40.351 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:40.351 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:40.351 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:40.351 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:40.351 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:40.351 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:40.352 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:40.352 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 1 Processor -- start 2024-09-15T01:45:40.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 1 -- start start 2024-09-15T01:45:40.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 10 monclient: init 2024-09-15T01:45:40.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:40.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 10 monclient: _add_conns ranks=[9,1,10,3,7,5,8,0,4,2,6] 2024-09-15T01:45:40.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbda40a2930 0x7fbda4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 10 monclient(hunting): picked mon.noname-a con 0x7fbda40a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:45:40.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbda409faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:45:40.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbda409fe70 0x7fbda40a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 10 monclient(hunting): picked mon.noname-f con 0x7fbda409fe70 addr v2:172.21.15.204:3300/0 2024-09-15T01:45:40.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda404bf60 con 0x7fbda409faa0 2024-09-15T01:45:40.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda409dd80 con 0x7fbda40a2930 2024-09-15T01:45:40.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda40a2d00 con 0x7fbda409fe70 2024-09-15T01:45:40.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:40.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.348+0000 7fbda9b14640 10 monclient(hunting): authenticate will time out at 1545.098899s 2024-09-15T01:45:40.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbda40a2930 0x7fbda4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbda40a2930 0x7fbda4132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33552/0 (socket says 172.21.15.178:33552) 2024-09-15T01:45:40.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 1 -- 172.21.15.178:0/1022550444 learned_addr learned my addr 172.21.15.178:0/1022550444 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:40.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 10 monclient(hunting): get_auth_request con 0x7fbda40a2930 auth_method 0 2024-09-15T01:45:40.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda3fff640 1 --2- 172.21.15.178:0/1022550444 >> v2:172.21.15.204:3300/0 conn(0x7fbda409fe70 0x7fbda40a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda3fff640 10 monclient(hunting): get_auth_request con 0x7fbda409fe70 auth_method 0 2024-09-15T01:45:40.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 10 monclient(hunting): handle_auth_done global_id 64143 payload 274 2024-09-15T01:45:40.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 1 -- 172.21.15.178:0/1022550444 >> v2:172.21.15.204:3300/0 conn(0x7fbda409fe70 msgr2=0x7fbda40a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:40.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 1 --2- 172.21.15.178:0/1022550444 >> v2:172.21.15.204:3300/0 conn(0x7fbda409fe70 0x7fbda40a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 1 -- 172.21.15.178:0/1022550444 >> v1:172.21.15.204:6789/0 conn(0x7fbda409faa0 legacy=0x7fbda40a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:40.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 10 monclient: _finish_hunting 0 2024-09-15T01:45:40.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 1 monclient: found mon.noname-a 2024-09-15T01:45:40.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:40.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:45:40.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 1 -- 172.21.15.178:0/1022550444 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4133440 con 0x7fbda40a2930 2024-09-15T01:45:40.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.349+0000 7fbda37fe640 1 --2- 172.21.15.178:0/1022550444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbda40a2930 0x7fbda4132de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbda40a2f70 tx=0x7fbd940091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4043b6d6ba23e605 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda27fc640 1 -- 172.21.15.178:0/1022550444 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd9402ac80 con 0x7fbda40a2930 2024-09-15T01:45:40.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:45:40.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda27fc640 10 monclient: dump: 2024-09-15T01:45:40.369 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.369 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.369 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.369 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.369 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.369 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.369 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.369 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.369 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda27fc640 10 monclient: _finish_auth 0 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda27fc640 1 -- 172.21.15.178:0/1022550444 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9402ae00 con 0x7fbda40a2930 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 5 monclient: authenticate success, global_id 64143 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:40.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:40.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda27fc640 1 -- 172.21.15.178:0/1022550444 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd94031ca0 con 0x7fbda40a2930 2024-09-15T01:45:40.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:40.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 4 set_mon_vals no callback set 2024-09-15T01:45:40.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:40.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda27fc640 10 monclient: dump: 2024-09-15T01:45:40.371 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.371 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.371 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.371 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.371 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.372 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.372 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.372 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.372 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.372 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.372 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.372 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.372 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.372 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.372 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.372 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 10 monclient: shutdown 2024-09-15T01:45:40.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:40.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 1 -- 172.21.15.178:0/1022550444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbda40a2930 msgr2=0x7fbda4132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 1 --2- 172.21.15.178:0/1022550444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbda40a2930 0x7fbda4132de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbda40a2f70 tx=0x7fbd940091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 1 -- 172.21.15.178:0/1022550444 shutdown_connections 2024-09-15T01:45:40.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 1 --2- 172.21.15.178:0/1022550444 >> v2:172.21.15.204:3300/0 conn(0x7fbda409fe70 0x7fbda40a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 1 --2- 172.21.15.178:0/1022550444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbda40a2930 0x7fbda4132de0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 1 -- 172.21.15.178:0/1022550444 >> 172.21.15.178:0/1022550444 conn(0x7fbda40a5ed0 msgr2=0x7fbda40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:40.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 1 -- 172.21.15.178:0/1022550444 shutdown_connections 2024-09-15T01:45:40.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.350+0000 7fbda9b14640 1 -- 172.21.15.178:0/1022550444 wait complete. 2024-09-15T01:45:40.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.351+0000 7fbda9b14640 10 monclient: build_initial_monmap 2024-09-15T01:45:40.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.351+0000 7fbda9b14640 10 monclient: monmap: 2024-09-15T01:45:40.374 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:40.374 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:40.374 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:40.374 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:40.374 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:40.374 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:40.374 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:40.374 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:40.374 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:40.374 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:40.375 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:40.375 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:40.375 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:40.375 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:40.375 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:40.375 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.351+0000 7fbda9b14640 1 Processor -- start 2024-09-15T01:45:40.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.351+0000 7fbda9b14640 1 -- start start 2024-09-15T01:45:40.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.351+0000 7fbda9b14640 10 monclient: init 2024-09-15T01:45:40.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:40.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 10 monclient: _add_conns ranks=[5,2,7,6,0,8,3,1,4] 2024-09-15T01:45:40.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbda409faa0 0x7fbda409b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 10 monclient(hunting): picked mon.noname-h con 0x7fbda409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:40.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda409fe70 0x7fbda41316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 10 monclient(hunting): picked mon.noname-b con 0x7fbda409fe70 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:40.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbda40a2930 0x7fbda411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 10 monclient(hunting): picked mon.noname-i con 0x7fbda40a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:40.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda41324f0 con 0x7fbda409fe70 2024-09-15T01:45:40.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbda409d630 con 0x7fbda409faa0 2024-09-15T01:45:40.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbda409f870 con 0x7fbda40a2930 2024-09-15T01:45:40.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:40.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda9b14640 10 monclient(hunting): authenticate will time out at 1545.102321s 2024-09-15T01:45:40.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda409fe70 0x7fbda41316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda409fe70 0x7fbda41316e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:56464/0 (socket says 172.21.15.178:56464) 2024-09-15T01:45:40.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 1 -- 172.21.15.178:0/2941111809 learned_addr learned my addr 172.21.15.178:0/2941111809 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:40.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 10 monclient(hunting): get_auth_request con 0x7fbda409fe70 auth_method 0 2024-09-15T01:45:40.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda3fff640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbda40a2930 0x7fbda411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda37fe640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbda409faa0 0x7fbda409b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda3fff640 10 monclient(hunting): get_auth_request con 0x7fbda40a2930 auth_method 0 2024-09-15T01:45:40.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda37fe640 10 monclient(hunting): get_auth_request con 0x7fbda409faa0 auth_method 0 2024-09-15T01:45:40.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 10 monclient(hunting): handle_auth_done global_id 34940 payload 824 2024-09-15T01:45:40.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 1 -- 172.21.15.178:0/2941111809 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbda40a2930 msgr2=0x7fbda411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbda40a2930 0x7fbda411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 1 -- 172.21.15.178:0/2941111809 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbda409faa0 msgr2=0x7fbda409b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbda409faa0 0x7fbda409b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:45:40.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 1 monclient: found mon.noname-b 2024-09-15T01:45:40.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:40.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:45:40.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 1 -- 172.21.15.178:0/2941111809 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd94009450 con 0x7fbda409fe70 2024-09-15T01:45:40.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.352+0000 7fbda2ffd640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda409fe70 0x7fbda41316e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800b4f0 tx=0x7fbd980090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd9800f990 con 0x7fbda409fe70 2024-09-15T01:45:40.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:45:40.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 10 monclient: dump: 2024-09-15T01:45:40.381 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.381 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.381 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.381 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.381 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.381 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.381 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.381 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.381 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.381 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.381 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.382 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.382 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.382 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.382 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.382 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:45:40.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:40.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbda9b14640 5 monclient: authenticate success, global_id 34940 2024-09-15T01:45:40.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9800fb30 con 0x7fbda409fe70 2024-09-15T01:45:40.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbda9b14640 10 monclient: _renew_subs 2024-09-15T01:45:40.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbda9b14640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:45:40.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbda9b14640 1 -- 172.21.15.178:0/2941111809 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda4142e80 con 0x7fbda409fe70 2024-09-15T01:45:40.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:40.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd9800fca0 con 0x7fbda409fe70 2024-09-15T01:45:40.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:40.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 10 monclient: dump: 2024-09-15T01:45:40.383 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.383 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.383 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.383 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.383 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.383 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.384 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.384 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.384 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.384 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.384 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.384 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.384 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.384 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.384 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.384 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbda1ffb640 4 set_mon_vals no callback set 2024-09-15T01:45:40.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbda9b14640 10 monclient: _renew_subs 2024-09-15T01:45:40.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbda9b14640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:45:40.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbda9b14640 1 -- 172.21.15.178:0/2941111809 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda411e800 con 0x7fbda409fe70 2024-09-15T01:45:40.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbda9b14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:40.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fbd9800f510 con 0x7fbda409fe70 2024-09-15T01:45:40.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbda9b14640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda4131c20 0x7fbda411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbda9b14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.353+0000 7fbda9b14640 1 -- 172.21.15.178:0/2941111809 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4131ff0 con 0x7fbda4131c20 2024-09-15T01:45:40.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda37fe640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda4131c20 0x7fbda411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda37fe640 10 monclient: get_auth_request con 0x7fbda4131c20 auth_method 0 2024-09-15T01:45:40.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 1 --2- 172.21.15.178:0/2941111809 >> v2:172.21.15.204:6812/28210 conn(0x7fbd900396e0 0x7fbd9003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fbd9804b660 con 0x7fbda409fe70 2024-09-15T01:45:40.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda37fe640 10 monclient(hunting): handle_auth_done global_id 44572 payload 824 2024-09-15T01:45:40.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda37fe640 1 -- 172.21.15.178:0/2941111809 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbda404c4a0 con 0x7fbda4131c20 2024-09-15T01:45:40.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda3fff640 1 --2- 172.21.15.178:0/2941111809 >> v2:172.21.15.204:6812/28210 conn(0x7fbd900396e0 0x7fbd9003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda3fff640 10 monclient: get_auth_request con 0x7fbd900396e0 auth_method 0 2024-09-15T01:45:40.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda37fe640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda4131c20 0x7fbda411ca70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbda40a6ec0 tx=0x7fbd94032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd9402ac80 con 0x7fbda4131c20 2024-09-15T01:45:40.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:40.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 10 monclient: dump: 2024-09-15T01:45:40.387 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.387 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.387 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.387 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.387 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.387 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.388 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.388 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.388 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.388 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.388 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.388 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.388 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.388 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.388 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.388 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbda3fff640 1 --2- 172.21.15.178:0/2941111809 >> v2:172.21.15.204:6812/28210 conn(0x7fbd900396e0 0x7fbd9003bb90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbda40a1980 tx=0x7fbd9c009850 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fbd940319b0 con 0x7fbda4131c20 2024-09-15T01:45:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda4131c20 msgr2=0x7fbda411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda4131c20 0x7fbda411ca70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbda40a6ec0 tx=0x7fbd94032000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda4131c20 msgr2=0x7fbda411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.354+0000 7fbd8b7fe640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda4131c20 0x7fbda411ca70 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbda40a6ec0 tx=0x7fbd94032000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.360+0000 7fbda9b14640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:40.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.360+0000 7fbda9b14640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda411cfb0 0x7fbda4096f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.360+0000 7fbda9b14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.360+0000 7fbda9b14640 1 -- 172.21.15.178:0/2941111809 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbda404bc80 con 0x7fbda411cfb0 2024-09-15T01:45:40.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbda37fe640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda411cfb0 0x7fbda4096f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbda37fe640 10 monclient: get_auth_request con 0x7fbda411cfb0 auth_method 0 2024-09-15T01:45:40.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbda37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbda37fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbda37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbda37fe640 10 monclient(hunting): handle_auth_done global_id 44581 payload 824 2024-09-15T01:45:40.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbda37fe640 1 -- 172.21.15.178:0/2941111809 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbda405ed70 con 0x7fbda411cfb0 2024-09-15T01:45:40.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbda37fe640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda411cfb0 0x7fbda4096f40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009210 tx=0x7fbd9403d000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd94045860 con 0x7fbda411cfb0 2024-09-15T01:45:40.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbd8b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbd8b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:40.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbd8b7fe640 10 monclient: dump: 2024-09-15T01:45:40.391 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.391 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.391 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.391 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.392 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.392 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.392 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.392 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.392 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.392 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.392 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.392 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.392 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.392 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.392 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.393 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9482 (secure 0 0 0) 0x7fbd940453e0 con 0x7fbda411cfb0 2024-09-15T01:45:40.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbd8b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:40.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbd8b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:40.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda411cfb0 msgr2=0x7fbda4096f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbd8b7fe640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda411cfb0 0x7fbda4096f40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009210 tx=0x7fbd9403d000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbd8b7fe640 1 -- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda411cfb0 msgr2=0x7fbda4096f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:40.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.361+0000 7fbd8b7fe640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbda411cfb0 0x7fbda4096f40 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009210 tx=0x7fbd9403d000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.395 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:40.396 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:40.396 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:40.396 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:40.396 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:40.396 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:40.396 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:40.396 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:40.396 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:40.397 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:40.397 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:40.397 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:40.397 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:40.397 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:40.397 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:40.397 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.397 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:45:40.397 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:40.397 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:40.397 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:40.398 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.398 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.398 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.398 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.398 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.398 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.398 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.398 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.398 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.398 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.399 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:40.399 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:40.399 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.399 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.399 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.399 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.399 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.399 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.399 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.399 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.399 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.399 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.400 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:40.400 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:40.400 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:40.400 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:40.400 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:40.400 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:40.400 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:40.400 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:40.400 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:40.400 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:40.401 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:40.401 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:40.401 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:40.401 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:40.401 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:40.401 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:40.401 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:40.401 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.401 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.401 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.402 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.402 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.402 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.402 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.402 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.402 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.402 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.402 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:40.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.402 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:40.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.402 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:40.403 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:40.403 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.403 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:40.403 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.403 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:40.404 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.404 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.404 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:40.404 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:40.404 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.404 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.404 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.405 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:40.405 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:40.405 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.405 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:40.405 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.405 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:40.406 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.406 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.406 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:40.406 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:40.406 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.406 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.406 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.409 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:40.409 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:40.409 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:40.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:40.410 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.410 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.410 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:40.410 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:40.410 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.410 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.410 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.411 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:40.411 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:40.411 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.411 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:40.411 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.412 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:40.412 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.412 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.412 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:40.412 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:40.412 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.412 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.412 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.413 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:40.413 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:40.413 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.413 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:40.413 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:40.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.414 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.414 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:40.414 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:40.414 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.414 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.414 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.415 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:40.415 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:40.415 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.415 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:40.415 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:40.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.416 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.416 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:40.416 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:40.416 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.417 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.417 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.417 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:40.417 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:40.417 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:40.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:40.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.418 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:40.419 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:40.419 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.419 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.419 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.419 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:40.419 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:40.419 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.420 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:40.420 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.420 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:40.420 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.420 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.420 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:40.421 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:40.421 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.421 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.421 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.421 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.421 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.421 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:40.421 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:40.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.421 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.422 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.422 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.422 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.422 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.422 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:45:40.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.422 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.422 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.422 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:45:40.422 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.422 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.423 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:40.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.423 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.423 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.423 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.423 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.423 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.423 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:45:40.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.423 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.423 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.424 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.424 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.424 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.424 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:40.424 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:40.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.362+0000 7fbda9b14640 1 -- 172.21.15.178:0/2941111809 >> v2:172.21.15.204:6812/28210 conn(0x7fbd900396e0 msgr2=0x7fbd9003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.362+0000 7fbda9b14640 1 --2- 172.21.15.178:0/2941111809 >> v2:172.21.15.204:6812/28210 conn(0x7fbd900396e0 0x7fbd9003bb90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbda40a1980 tx=0x7fbd9c009850 comp rx=0 tx=0).stop 2024-09-15T01:45:40.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.362+0000 7fbda9b14640 10 monclient: shutdown 2024-09-15T01:45:40.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.362+0000 7fbda9b14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:40.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.362+0000 7fbda9b14640 1 -- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda409fe70 msgr2=0x7fbda41316e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.362+0000 7fbda9b14640 1 --2- 172.21.15.178:0/2941111809 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda409fe70 0x7fbda41316e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800b4f0 tx=0x7fbd980090f0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.362+0000 7fbda3fff640 1 -- 172.21.15.178:0/2941111809 reap_dead start 2024-09-15T01:45:40.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.362+0000 7fbda3fff640 1 -- 172.21.15.178:0/2941111809 reap_dead start 2024-09-15T01:45:40.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.362+0000 7fbda9b14640 1 -- 172.21.15.178:0/2941111809 shutdown_connections 2024-09-15T01:45:40.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.362+0000 7fbda9b14640 1 -- 172.21.15.178:0/2941111809 >> 172.21.15.178:0/2941111809 conn(0x7fbda40a5ed0 msgr2=0x7fbda40a40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:40.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.362+0000 7fbda9b14640 1 -- 172.21.15.178:0/2941111809 shutdown_connections 2024-09-15T01:45:40.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.362+0000 7fbda9b14640 1 -- 172.21.15.178:0/2941111809 wait complete. 2024-09-15T01:45:40.427 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:40.427 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:45:40.427 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:45:40.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.518+0000 7fb6c689f640 10 monclient: get_monmap_and_config 2024-09-15T01:45:40.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.518+0000 7fb6c689f640 10 monclient: build_initial_monmap 2024-09-15T01:45:40.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.518+0000 7fb6c689f640 10 monclient: monmap: 2024-09-15T01:45:40.520 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:40.521 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.521 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:40.519342+0000 2024-09-15T01:45:40.521 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:40.519342+0000 2024-09-15T01:45:40.521 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:40.521 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.521 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:40.521 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:40.521 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:40.521 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:40.521 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:40.522 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:40.522 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:40.522 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:40.522 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:40.522 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:40.522 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:40.522 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 1 Processor -- start 2024-09-15T01:45:40.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 1 -- start start 2024-09-15T01:45:40.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 10 monclient: init 2024-09-15T01:45:40.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:40.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 10 monclient: _add_conns ranks=[5,6,10,7,1,8,0,4,2,3,9] 2024-09-15T01:45:40.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb6c00a2930 0x7fb6c0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 10 monclient(hunting): picked mon.noname-h con 0x7fb6c00a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:40.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c009faa0 0x7fb6c009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 10 monclient(hunting): picked mon.noname-d con 0x7fb6c009faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:40.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb6c00a08c0 0x7fb6c00a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 10 monclient(hunting): picked mon.noname-f con 0x7fb6c00a08c0 addr v2:172.21.15.204:3300/0 2024-09-15T01:45:40.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c009e0f0 con 0x7fb6c00a08c0 2024-09-15T01:45:40.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c009dd80 con 0x7fb6c00a2930 2024-09-15T01:45:40.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00a2d00 con 0x7fb6c009faa0 2024-09-15T01:45:40.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:40.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6c689f640 10 monclient(hunting): authenticate will time out at 1545.269809s 2024-09-15T01:45:40.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6bf7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c009faa0 0x7fb6c009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6bf7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c009faa0 0x7fb6c009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57078/0 (socket says 172.21.15.178:57078) 2024-09-15T01:45:40.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6bf7fe640 1 -- 172.21.15.178:0/1323234653 learned_addr learned my addr 172.21.15.178:0/1323234653 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:40.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6bf7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c009faa0 auth_method 0 2024-09-15T01:45:40.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.519+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6c4e3c640 1 --2- 172.21.15.178:0/1323234653 >> v2:172.21.15.204:3300/0 conn(0x7fb6c00a08c0 0x7fb6c00a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bffff640 1 --2- 172.21.15.178:0/1323234653 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb6c00a2930 0x7fb6c0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6c4e3c640 10 monclient(hunting): get_auth_request con 0x7fb6c00a08c0 auth_method 0 2024-09-15T01:45:40.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6c4e3c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6c4e3c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6c4e3c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bffff640 10 monclient(hunting): get_auth_request con 0x7fb6c00a2930 auth_method 0 2024-09-15T01:45:40.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_done global_id 25431 payload 274 2024-09-15T01:45:40.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 1 -- 172.21.15.178:0/1323234653 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb6c00a2930 msgr2=0x7fb6c0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:40.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 1 --2- 172.21.15.178:0/1323234653 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb6c00a2930 0x7fb6c0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 1 -- 172.21.15.178:0/1323234653 >> v2:172.21.15.204:3300/0 conn(0x7fb6c00a08c0 msgr2=0x7fb6c00a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:40.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 1 --2- 172.21.15.178:0/1323234653 >> v2:172.21.15.204:3300/0 conn(0x7fb6c00a08c0 0x7fb6c00a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:45:40.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 1 monclient: found mon.noname-d 2024-09-15T01:45:40.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:40.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:45:40.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 1 -- 172.21.15.178:0/1323234653 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c004bf60 con 0x7fb6c009faa0 2024-09-15T01:45:40.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6c4e3c640 1 --2- 172.21.15.178:0/1323234653 >> v2:172.21.15.204:3300/0 conn(0x7fb6c00a08c0 0x7fb6c00a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:45:40.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.520+0000 7fb6bf7fe640 1 --2- 172.21.15.178:0/1323234653 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c009faa0 0x7fb6c009fe70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009910 tx=0x7fb6b40091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=6fc91613b66e170b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6beffd640 1 -- 172.21.15.178:0/1323234653 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6b402ac80 con 0x7fb6c009faa0 2024-09-15T01:45:40.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6beffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6beffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:45:40.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6beffd640 10 monclient: dump: 2024-09-15T01:45:40.540 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.540 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.540 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.540 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.540 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.540 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.540 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.540 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.540 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.540 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.541 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.541 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.541 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.541 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.541 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.541 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6beffd640 10 monclient: _finish_auth 0 2024-09-15T01:45:40.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6beffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:40.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6beffd640 1 -- 172.21.15.178:0/1323234653 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b402ae00 con 0x7fb6c009faa0 2024-09-15T01:45:40.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 5 monclient: authenticate success, global_id 25431 2024-09-15T01:45:40.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:40.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6beffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:40.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6beffd640 1 -- 172.21.15.178:0/1323234653 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6b4031ca0 con 0x7fb6c009faa0 2024-09-15T01:45:40.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:40.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 4 set_mon_vals no callback set 2024-09-15T01:45:40.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6beffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6beffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:45:40.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6beffd640 10 monclient: dump: 2024-09-15T01:45:40.542 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.543 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.543 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.543 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.543 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.543 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.543 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.543 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.543 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.543 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.543 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.543 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.544 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.544 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.544 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.544 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 10 monclient: shutdown 2024-09-15T01:45:40.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:40.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 1 -- 172.21.15.178:0/1323234653 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c009faa0 msgr2=0x7fb6c009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 1 --2- 172.21.15.178:0/1323234653 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c009faa0 0x7fb6c009fe70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009910 tx=0x7fb6b40091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 1 -- 172.21.15.178:0/1323234653 shutdown_connections 2024-09-15T01:45:40.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 1 --2- 172.21.15.178:0/1323234653 >> v2:172.21.15.204:3300/0 conn(0x7fb6c00a08c0 0x7fb6c00a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 1 --2- 172.21.15.178:0/1323234653 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c009faa0 0x7fb6c009fe70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 1 --2- 172.21.15.178:0/1323234653 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb6c00a2930 0x7fb6c0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 1 -- 172.21.15.178:0/1323234653 >> 172.21.15.178:0/1323234653 conn(0x7fb6c00a5ed0 msgr2=0x7fb6c00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:40.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 1 -- 172.21.15.178:0/1323234653 shutdown_connections 2024-09-15T01:45:40.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.521+0000 7fb6c689f640 1 -- 172.21.15.178:0/1323234653 wait complete. 2024-09-15T01:45:40.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.522+0000 7fb6c689f640 10 monclient: build_initial_monmap 2024-09-15T01:45:40.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.522+0000 7fb6c689f640 10 monclient: monmap: 2024-09-15T01:45:40.545 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:40.545 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:40.545 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:40.545 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:40.545 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:40.546 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:40.546 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:40.546 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:40.546 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:40.546 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:40.546 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:40.546 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:40.546 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:40.546 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:40.546 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:40.546 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.522+0000 7fb6c689f640 1 Processor -- start 2024-09-15T01:45:40.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.522+0000 7fb6c689f640 1 -- start start 2024-09-15T01:45:40.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.522+0000 7fb6c689f640 10 monclient: init 2024-09-15T01:45:40.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:40.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 10 monclient: _add_conns ranks=[2,6,8,7,4,5,3,1,0] 2024-09-15T01:45:40.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6c009faa0 0x7fb6c009b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 10 monclient(hunting): picked mon.noname-b con 0x7fb6c009faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:40.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c00a08c0 0x7fb6c01318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 10 monclient(hunting): picked mon.noname-d con 0x7fb6c00a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:40.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb6c00a2930 0x7fb6c0141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 10 monclient(hunting): picked mon.noname-e con 0x7fb6c00a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:40.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c009d630 con 0x7fb6c009faa0 2024-09-15T01:45:40.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c011ecb0 con 0x7fb6c00a08c0 2024-09-15T01:45:40.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c011e9d0 con 0x7fb6c00a2930 2024-09-15T01:45:40.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:40.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 10 monclient(hunting): authenticate will time out at 1545.273178s 2024-09-15T01:45:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6c009faa0 0x7fb6c009b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bf7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c00a08c0 0x7fb6c01318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c4e3c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb6c00a2930 0x7fb6c0141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6c009faa0 0x7fb6c009b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:56480/0 (socket says 172.21.15.178:56480) 2024-09-15T01:45:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 1 -- 172.21.15.178:0/1149659240 learned_addr learned my addr 172.21.15.178:0/1149659240 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bf7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c00a08c0 0x7fb6c01318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57084/0 (socket says 172.21.15.178:57084) 2024-09-15T01:45:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 10 monclient(hunting): get_auth_request con 0x7fb6c009faa0 auth_method 0 2024-09-15T01:45:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bf7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c00a08c0 auth_method 0 2024-09-15T01:45:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c4e3c640 10 monclient(hunting): get_auth_request con 0x7fb6c00a2930 auth_method 0 2024-09-15T01:45:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c4e3c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c4e3c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c4e3c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c4e3c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c4e3c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c4e3c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 10 monclient(hunting): handle_auth_done global_id 34949 payload 824 2024-09-15T01:45:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 1 -- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb6c00a2930 msgr2=0x7fb6c0141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb6c00a2930 0x7fb6c0141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 1 -- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c00a08c0 msgr2=0x7fb6c01318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c00a08c0 0x7fb6c01318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 10 monclient: _finish_hunting 0 2024-09-15T01:45:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 1 monclient: found mon.noname-b 2024-09-15T01:45:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:45:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 1 -- 172.21.15.178:0/1149659240 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b4009450 con 0x7fb6c009faa0 2024-09-15T01:45:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bf7fe640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c00a08c0 0x7fb6c01318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6bffff640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6c009faa0 0x7fb6c009b1d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000d890 tx=0x7fb6b0008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6b000de20 con 0x7fb6c009faa0 2024-09-15T01:45:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6a7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:45:40.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6a7fff640 10 monclient: dump: 2024-09-15T01:45:40.553 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.553 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.553 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.553 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.553 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.553 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.553 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.553 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.553 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.553 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.553 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.553 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.554 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.554 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.554 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.554 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6a7fff640 10 monclient: _finish_auth 0 2024-09-15T01:45:40.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:40.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 5 monclient: authenticate success, global_id 34949 2024-09-15T01:45:40.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 10 monclient: _renew_subs 2024-09-15T01:45:40.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b000bd50 con 0x7fb6c009faa0 2024-09-15T01:45:40.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:45:40.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6c689f640 1 -- 172.21.15.178:0/1149659240 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c011d600 con 0x7fb6c009faa0 2024-09-15T01:45:40.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6a7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:40.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6b00148b0 con 0x7fb6c009faa0 2024-09-15T01:45:40.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6a7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:40.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6a7fff640 10 monclient: dump: 2024-09-15T01:45:40.555 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.555 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.555 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.555 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.555 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.555 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.555 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.556 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.556 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.556 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.556 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.556 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.556 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.556 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.556 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.556 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.523+0000 7fb6be7fc640 4 set_mon_vals no callback set 2024-09-15T01:45:40.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6c689f640 10 monclient: _renew_subs 2024-09-15T01:45:40.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6c689f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:45:40.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6c689f640 1 -- 172.21.15.178:0/1149659240 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c011cba0 con 0x7fb6c009faa0 2024-09-15T01:45:40.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6c689f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:40.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6c689f640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c011dae0 0x7fb6c0142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6c689f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6c689f640 1 -- 172.21.15.178:0/1149659240 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c004c4a0 con 0x7fb6c011dae0 2024-09-15T01:45:40.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb6b0014430 con 0x7fb6c009faa0 2024-09-15T01:45:40.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6bf7fe640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c011dae0 0x7fb6c0142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6bf7fe640 10 monclient: get_auth_request con 0x7fb6c011dae0 auth_method 0 2024-09-15T01:45:40.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.524+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_done global_id 25449 payload 824 2024-09-15T01:45:40.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6bf7fe640 1 -- 172.21.15.178:0/1149659240 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6c005ed70 con 0x7fb6c011dae0 2024-09-15T01:45:40.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 1 --2- 172.21.15.178:0/1149659240 >> v2:172.21.15.204:6812/28210 conn(0x7fb6ac0397b0 0x7fb6ac03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fb6b004c2b0 con 0x7fb6c009faa0 2024-09-15T01:45:40.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6bf7fe640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c011dae0 0x7fb6c0142260 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00a3290 tx=0x7fb6b4032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6c4e3c640 1 --2- 172.21.15.178:0/1149659240 >> v2:172.21.15.204:6812/28210 conn(0x7fb6ac0397b0 0x7fb6ac03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6c4e3c640 10 monclient: get_auth_request con 0x7fb6ac0397b0 auth_method 0 2024-09-15T01:45:40.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6b402ac80 con 0x7fb6c011dae0 2024-09-15T01:45:40.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:45:40.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 10 monclient: dump: 2024-09-15T01:45:40.559 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.559 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.559 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.559 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.559 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.559 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.559 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.559 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.560 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.560 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.560 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.560 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.560 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.560 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.560 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.560 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fb6b4031dc0 con 0x7fb6c011dae0 2024-09-15T01:45:40.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:40.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:40.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c011dae0 msgr2=0x7fb6c0142260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c011dae0 0x7fb6c0142260 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00a3290 tx=0x7fb6b4032000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c011dae0 msgr2=0x7fb6c0142260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:40.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6a7fff640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c011dae0 0x7fb6c0142260 secure :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00a3290 tx=0x7fb6b4032000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.525+0000 7fb6c4e3c640 1 --2- 172.21.15.178:0/1149659240 >> v2:172.21.15.204:6812/28210 conn(0x7fb6ac0397b0 0x7fb6ac03bc60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb6c009b880 tx=0x7fb6b8009990 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6c689f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:40.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6c689f640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c0131df0 0x7fb6c01427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6c689f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6c689f640 1 -- 172.21.15.178:0/1149659240 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c004bc80 con 0x7fb6c0131df0 2024-09-15T01:45:40.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6bf7fe640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c0131df0 0x7fb6c01427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6bf7fe640 10 monclient: get_auth_request con 0x7fb6c0131df0 auth_method 0 2024-09-15T01:45:40.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_done global_id 25458 payload 824 2024-09-15T01:45:40.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6bf7fe640 1 -- 172.21.15.178:0/1149659240 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb6c00a2400 con 0x7fb6c0131df0 2024-09-15T01:45:40.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6bf7fe640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c0131df0 0x7fb6c01427a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009210 tx=0x7fb6b403d000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6b403b3d0 con 0x7fb6c0131df0 2024-09-15T01:45:40.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6a7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:45:40.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.531+0000 7fb6a7fff640 10 monclient: dump: 2024-09-15T01:45:40.563 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.563 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.563 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.563 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.563 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.563 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.563 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.563 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.564 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.564 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.564 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.564 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.564 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.564 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.564 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.564 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fb6b4031930 con 0x7fb6c0131df0 2024-09-15T01:45:40.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6a7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:40.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6a7fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:40.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c0131df0 msgr2=0x7fb6c01427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6a7fff640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c0131df0 0x7fb6c01427a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009210 tx=0x7fb6b403d000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6a7fff640 1 -- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c0131df0 msgr2=0x7fb6c01427a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:40.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6a7fff640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb6c0131df0 0x7fb6c01427a0 secure :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009210 tx=0x7fb6b403d000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.565 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:40.565 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:40.566 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:40.566 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:40.566 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:40.566 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:40.566 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:40.566 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:40.566 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:40.567 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:40.567 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:40.567 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:40.567 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:40.567 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:40.567 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:40.567 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.567 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:45:40.567 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:40.567 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:40.567 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:40.567 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.568 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.568 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.568 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.568 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.568 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.568 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.568 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.568 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.568 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.568 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:40.568 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:40.569 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.569 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.569 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.569 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.569 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.569 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.569 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.569 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.569 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.569 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.570 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:40.570 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:40.570 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:40.570 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:40.570 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:40.570 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:40.570 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:40.570 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:40.570 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:40.570 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:40.570 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:40.571 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:40.571 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:40.571 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:40.571 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:40.571 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:40.571 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:40.571 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.571 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.571 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.571 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.571 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.572 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.572 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.572 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.572 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.572 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.572 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:40.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.572 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:40.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.572 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:40.572 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:40.572 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.573 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:40.573 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.573 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:40.573 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.573 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.574 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.574 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:40.574 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:40.574 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.574 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.574 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.574 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:40.574 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:40.575 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.575 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:40.575 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.575 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:40.575 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.576 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.576 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:40.576 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:40.576 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.576 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.576 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.576 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:40.577 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:40.577 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.577 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:40.577 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.577 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:40.577 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.578 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.578 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.578 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:40.578 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:40.578 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.578 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.578 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.579 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:40.579 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:40.579 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.579 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:40.579 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.580 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.580 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:40.580 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.580 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.580 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.580 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:40.580 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:40.580 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.580 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.580 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.581 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:40.581 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:40.581 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.581 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:40.581 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.582 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.582 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:40.582 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.582 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.582 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.582 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:40.582 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:40.583 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.583 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.583 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.583 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:40.583 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:40.583 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.583 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.583 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.584 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:40.584 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.584 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.584 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:40.584 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.584 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.584 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.584 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:40.585 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:40.585 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.585 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.585 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.585 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:40.585 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:40.585 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.585 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.586 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.586 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:40.586 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.586 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.586 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:40.586 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.586 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.586 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.587 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:40.587 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:40.587 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.587 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.587 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.587 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.587 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:40.587 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:40.587 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.587 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.588 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.588 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.588 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:40.588 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.588 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.588 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.588 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:40.588 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.588 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.588 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.589 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:40.589 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:40.589 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.589 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.589 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.589 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:40.589 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:40.589 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.590 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:40.590 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.590 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:40.590 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.591 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.591 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.591 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:40.591 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:40.591 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.591 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.591 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.592 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.592 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.592 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:40.592 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:40.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.592 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.592 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.592 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.592 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.592 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.593 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:40.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.593 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.593 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.593 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.593 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.593 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.593 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:40.593 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:40.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6c689f640 1 -- 172.21.15.178:0/1149659240 >> v2:172.21.15.204:6812/28210 conn(0x7fb6ac0397b0 msgr2=0x7fb6ac03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6c689f640 1 --2- 172.21.15.178:0/1149659240 >> v2:172.21.15.204:6812/28210 conn(0x7fb6ac0397b0 0x7fb6ac03bc60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb6c009b880 tx=0x7fb6b8009990 comp rx=0 tx=0).stop 2024-09-15T01:45:40.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6c689f640 10 monclient: shutdown 2024-09-15T01:45:40.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6c689f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:40.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6c689f640 1 -- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6c009faa0 msgr2=0x7fb6c009b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6c689f640 1 --2- 172.21.15.178:0/1149659240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6c009faa0 0x7fb6c009b1d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000d890 tx=0x7fb6b0008040 comp rx=0 tx=0).stop 2024-09-15T01:45:40.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6c4e3c640 1 -- 172.21.15.178:0/1149659240 reap_dead start 2024-09-15T01:45:40.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6c4e3c640 1 -- 172.21.15.178:0/1149659240 reap_dead start 2024-09-15T01:45:40.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6c689f640 1 -- 172.21.15.178:0/1149659240 shutdown_connections 2024-09-15T01:45:40.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6c689f640 1 -- 172.21.15.178:0/1149659240 >> 172.21.15.178:0/1149659240 conn(0x7fb6c00a5ed0 msgr2=0x7fb6c00a53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:40.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6c689f640 1 -- 172.21.15.178:0/1149659240 shutdown_connections 2024-09-15T01:45:40.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.532+0000 7fb6c689f640 1 -- 172.21.15.178:0/1149659240 wait complete. 2024-09-15T01:45:40.596 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:40.596 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:45:40.597 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:45:40.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.689+0000 7f977b35d640 10 monclient: get_monmap_and_config 2024-09-15T01:45:40.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.689+0000 7f977b35d640 10 monclient: build_initial_monmap 2024-09-15T01:45:40.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.689+0000 7f977b35d640 10 monclient: monmap: 2024-09-15T01:45:40.692 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:40.692 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.692 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:40.690497+0000 2024-09-15T01:45:40.692 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:40.690497+0000 2024-09-15T01:45:40.692 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:40.692 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.692 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:40.693 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:40.693 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:40.693 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:40.693 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:40.693 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:40.693 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:40.693 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:40.693 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:40.693 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:40.693 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:40.693 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 1 Processor -- start 2024-09-15T01:45:40.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 1 -- start start 2024-09-15T01:45:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 10 monclient: init 2024-09-15T01:45:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 10 monclient: _add_conns ranks=[9,1,10,2,3,5,0,6,4,8,7] 2024-09-15T01:45:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97740a2930 0x7f9774132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 10 monclient(hunting): picked mon.noname-a con 0x7f97740a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:45:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f977409faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:45:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f977409fe70 0x7f97740a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 10 monclient(hunting): picked mon.noname-f con 0x7f977409fe70 addr v2:172.21.15.204:3300/0 2024-09-15T01:45:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977404bf60 con 0x7f977409faa0 2024-09-15T01:45:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977409dd80 con 0x7f97740a2930 2024-09-15T01:45:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97740a2d00 con 0x7f977409fe70 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.690+0000 7f977b35d640 10 monclient(hunting): authenticate will time out at 1545.440920s 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97740a2930 0x7f9774132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97740a2930 0x7f9774132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33568/0 (socket says 172.21.15.178:33568) 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 1 -- 172.21.15.178:0/521552957 learned_addr learned my addr 172.21.15.178:0/521552957 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 10 monclient(hunting): get_auth_request con 0x7f97740a2930 auth_method 0 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97798fa640 1 --2- 172.21.15.178:0/521552957 >> v2:172.21.15.204:3300/0 conn(0x7f977409fe70 0x7f97740a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97798fa640 10 monclient(hunting): get_auth_request con 0x7f977409fe70 auth_method 0 2024-09-15T01:45:40.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97798fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97798fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97798fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 10 monclient(hunting): handle_auth_done global_id 64152 payload 274 2024-09-15T01:45:40.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 1 -- 172.21.15.178:0/521552957 >> v2:172.21.15.204:3300/0 conn(0x7f977409fe70 msgr2=0x7f97740a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:40.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 1 --2- 172.21.15.178:0/521552957 >> v2:172.21.15.204:3300/0 conn(0x7f977409fe70 0x7f97740a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 1 -- 172.21.15.178:0/521552957 >> v1:172.21.15.204:6789/0 conn(0x7f977409faa0 legacy=0x7f97740a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 10 monclient: _finish_hunting 0 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 1 monclient: found mon.noname-a 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.691+0000 7f97790f9640 1 -- 172.21.15.178:0/521552957 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9774133440 con 0x7f97740a2930 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f97790f9640 1 --2- 172.21.15.178:0/521552957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97740a2930 0x7f9774132de0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97740a2f70 tx=0x7f97700091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ba720bd2437d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f976bfff640 1 -- 172.21.15.178:0/521552957 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f977002ac80 con 0x7f97740a2930 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f976bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f976bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f976bfff640 10 monclient: dump: 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.708 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f976bfff640 10 monclient: _finish_auth 0 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f976bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 5 monclient: authenticate success, global_id 64152 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f976bfff640 1 -- 172.21.15.178:0/521552957 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977002ae00 con 0x7f97740a2930 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f976bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f976bfff640 1 -- 172.21.15.178:0/521552957 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9770031ca0 con 0x7f97740a2930 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 4 set_mon_vals no callback set 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f976bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f976bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f976bfff640 10 monclient: dump: 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.710 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.710 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.710 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.710 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.710 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.710 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.710 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.710 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.710 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.710 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 10 monclient: shutdown 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 1 -- 172.21.15.178:0/521552957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97740a2930 msgr2=0x7f9774132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 1 --2- 172.21.15.178:0/521552957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97740a2930 0x7f9774132de0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97740a2f70 tx=0x7f97700091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 1 -- 172.21.15.178:0/521552957 shutdown_connections 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 1 --2- 172.21.15.178:0/521552957 >> v2:172.21.15.204:3300/0 conn(0x7f977409fe70 0x7f97740a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 1 --2- 172.21.15.178:0/521552957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97740a2930 0x7f9774132de0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 1 -- 172.21.15.178:0/521552957 >> 172.21.15.178:0/521552957 conn(0x7f97740a5ed0 msgr2=0x7f97740a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 1 -- 172.21.15.178:0/521552957 shutdown_connections 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.692+0000 7f977b35d640 1 -- 172.21.15.178:0/521552957 wait complete. 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.693+0000 7f977b35d640 10 monclient: build_initial_monmap 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.693+0000 7f977b35d640 10 monclient: monmap: 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.693+0000 7f977b35d640 1 Processor -- start 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.693+0000 7f977b35d640 1 -- start start 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.693+0000 7f977b35d640 10 monclient: init 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 10 monclient: _add_conns ranks=[2,5,7,0,8,1,6,4,3] 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f977409faa0 0x7f977409b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 10 monclient(hunting): picked mon.noname-b con 0x7f977409faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f977409fe70 0x7f97741316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 10 monclient(hunting): picked mon.noname-h con 0x7f977409fe70 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97740a2930 0x7f977411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 10 monclient(hunting): picked mon.noname-i con 0x7f97740a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97741324f0 con 0x7f977409faa0 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f977409d630 con 0x7f977409fe70 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f977409f870 con 0x7f97740a2930 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f977b35d640 10 monclient(hunting): authenticate will time out at 1545.444291s 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f977409faa0 0x7f977409b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f977409faa0 0x7f977409b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:56488/0 (socket says 172.21.15.178:56488) 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 1 -- 172.21.15.178:0/2776456110 learned_addr learned my addr 172.21.15.178:0/2776456110 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 10 monclient(hunting): get_auth_request con 0x7f977409faa0 auth_method 0 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97788f8640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f977409fe70 0x7f97741316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97788f8640 10 monclient(hunting): get_auth_request con 0x7f977409fe70 auth_method 0 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97788f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97788f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97788f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97798fa640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97740a2930 0x7f977411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97798fa640 10 monclient(hunting): get_auth_request con 0x7f97740a2930 auth_method 0 2024-09-15T01:45:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97798fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97798fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97798fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97788f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97788f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97788f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97798fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97798fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97798fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 10 monclient(hunting): handle_auth_done global_id 34958 payload 824 2024-09-15T01:45:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 1 -- 172.21.15.178:0/2776456110 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97740a2930 msgr2=0x7f977411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f97740a2930 0x7f977411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 1 -- 172.21.15.178:0/2776456110 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f977409fe70 msgr2=0x7f97741316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f977409fe70 0x7f97741316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 10 monclient: _finish_hunting 0 2024-09-15T01:45:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 1 monclient: found mon.noname-b 2024-09-15T01:45:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:45:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 1 -- 172.21.15.178:0/2776456110 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9770009450 con 0x7f977409faa0 2024-09-15T01:45:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.694+0000 7f97790f9640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f977409faa0 0x7f977409b560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9770009210 tx=0x7f9770032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f977002ac80 con 0x7f977409faa0 2024-09-15T01:45:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:45:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 10 monclient: dump: 2024-09-15T01:45:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.719 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.719 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.719 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.719 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.719 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.719 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.719 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.719 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.719 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.720 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.720 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.720 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.720 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.720 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.720 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 10 monclient: _finish_auth 0 2024-09-15T01:45:40.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:40.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f977b35d640 5 monclient: authenticate success, global_id 34958 2024-09-15T01:45:40.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9770033e30 con 0x7f977409faa0 2024-09-15T01:45:40.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f977b35d640 10 monclient: _renew_subs 2024-09-15T01:45:40.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f977b35d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:45:40.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f977b35d640 1 -- 172.21.15.178:0/2776456110 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9774142e80 con 0x7f977409faa0 2024-09-15T01:45:40.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:40.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f977003b870 con 0x7f977409faa0 2024-09-15T01:45:40.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 10 monclient: dump: 2024-09-15T01:45:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.723 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.723 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.723 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.723 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.723 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.723 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f976b7fe640 4 set_mon_vals no callback set 2024-09-15T01:45:40.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f977b35d640 10 monclient: _renew_subs 2024-09-15T01:45:40.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f977b35d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:45:40.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f977b35d640 1 -- 172.21.15.178:0/2776456110 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977411e800 con 0x7f977409faa0 2024-09-15T01:45:40.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f977b35d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:40.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9770039040 con 0x7f977409faa0 2024-09-15T01:45:40.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f977b35d640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9774131c20 0x7f977411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f977b35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.695+0000 7f977b35d640 1 -- 172.21.15.178:0/2776456110 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f977411e990 con 0x7f9774131c20 2024-09-15T01:45:40.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97788f8640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9774131c20 0x7f977411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97788f8640 10 monclient: get_auth_request con 0x7f9774131c20 auth_method 0 2024-09-15T01:45:40.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97788f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97788f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97788f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 1 --2- 172.21.15.178:0/2776456110 >> v2:172.21.15.204:6812/28210 conn(0x7f97580396e0 0x7f975803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97788f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97788f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f977006b680 con 0x7f977409faa0 2024-09-15T01:45:40.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97788f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97788f8640 10 monclient(hunting): handle_auth_done global_id 34973 payload 824 2024-09-15T01:45:40.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97798fa640 1 --2- 172.21.15.178:0/2776456110 >> v2:172.21.15.204:6812/28210 conn(0x7f97580396e0 0x7f975803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97788f8640 1 -- 172.21.15.178:0/2776456110 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f977404c4a0 con 0x7f9774131c20 2024-09-15T01:45:40.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97798fa640 10 monclient: get_auth_request con 0x7f97580396e0 auth_method 0 2024-09-15T01:45:40.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97788f8640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9774131c20 0x7f977411ca70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f976400e7b0 tx=0x7f9764006f50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f976400ee80 con 0x7f9774131c20 2024-09-15T01:45:40.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:40.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 10 monclient: dump: 2024-09-15T01:45:40.727 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.727 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.727 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.727 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.728 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.728 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.728 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.728 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.728 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.728 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.728 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.728 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.728 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.729 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.729 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.729 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97798fa640 1 --2- 172.21.15.178:0/2776456110 >> v2:172.21.15.204:6812/28210 conn(0x7f97580396e0 0x7f975803bb90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f97740a1980 tx=0x7f976c009850 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f976400c8b0 con 0x7f9774131c20 2024-09-15T01:45:40.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:40.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:40.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9774131c20 msgr2=0x7f977411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9774131c20 0x7f977411ca70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f976400e7b0 tx=0x7f9764006f50 comp rx=0 tx=0).stop 2024-09-15T01:45:40.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9774131c20 msgr2=0x7f977411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:40.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.696+0000 7f97697fa640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f9774131c20 0x7f977411ca70 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f976400e7b0 tx=0x7f9764006f50 comp rx=0 tx=0).stop 2024-09-15T01:45:40.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f977b35d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:40.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f977b35d640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f977411cfb0 0x7f9774096f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f977b35d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f977b35d640 1 -- 172.21.15.178:0/2776456110 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9774131ff0 con 0x7f977411cfb0 2024-09-15T01:45:40.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f97788f8640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f977411cfb0 0x7f9774096f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f97788f8640 10 monclient: get_auth_request con 0x7f977411cfb0 auth_method 0 2024-09-15T01:45:40.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f97788f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f97788f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f97788f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f97788f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f97788f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f97788f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f97788f8640 10 monclient(hunting): handle_auth_done global_id 34982 payload 824 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f97788f8640 1 -- 172.21.15.178:0/2776456110 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f977404bc80 con 0x7f977411cfb0 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.702+0000 7f97788f8640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f977411cfb0 0x7f9774096f40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f976400e7b0 tx=0x7f9764015000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f976401bae0 con 0x7f977411cfb0 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f97697fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f97697fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f97697fa640 10 monclient: dump: 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f976401bc40 con 0x7f977411cfb0 2024-09-15T01:45:40.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f97697fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:40.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f97697fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:40.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f977411cfb0 msgr2=0x7f9774096f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f97697fa640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f977411cfb0 0x7f9774096f40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f976400e7b0 tx=0x7f9764015000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f97697fa640 1 -- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f977411cfb0 msgr2=0x7f9774096f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:40.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f97697fa640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f977411cfb0 0x7f9774096f40 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f976400e7b0 tx=0x7f9764015000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.735 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:40.735 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:40.735 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:40.735 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.736 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.737 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.738 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:40.739 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.740 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.741 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:40.742 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.743 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.744 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:40.745 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.746 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.747 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.748 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:40.749 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.750 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.751 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.752 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.753 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.754 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.755 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.756 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.757 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.758 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.758 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.758 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.758 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.758 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:40.758 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:40.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f977b35d640 1 -- 172.21.15.178:0/2776456110 >> v2:172.21.15.204:6812/28210 conn(0x7f97580396e0 msgr2=0x7f975803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f977b35d640 1 --2- 172.21.15.178:0/2776456110 >> v2:172.21.15.204:6812/28210 conn(0x7f97580396e0 0x7f975803bb90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f97740a1980 tx=0x7f976c009850 comp rx=0 tx=0).stop 2024-09-15T01:45:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f977b35d640 10 monclient: shutdown 2024-09-15T01:45:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f977b35d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f977b35d640 1 -- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f977409faa0 msgr2=0x7f977409b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f977b35d640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f977409faa0 0x7f977409b560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9770009210 tx=0x7f9770032000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f97798fa640 1 -- 172.21.15.178:0/2776456110 reap_dead start 2024-09-15T01:45:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f977b35d640 1 -- 172.21.15.178:0/2776456110 shutdown_connections 2024-09-15T01:45:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f977b35d640 1 --2- 172.21.15.178:0/2776456110 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f977409faa0 0x7f977409b560 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f977b35d640 1 -- 172.21.15.178:0/2776456110 >> 172.21.15.178:0/2776456110 conn(0x7f97740a5ed0 msgr2=0x7f97740a40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f977b35d640 1 -- 172.21.15.178:0/2776456110 shutdown_connections 2024-09-15T01:45:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.703+0000 7f977b35d640 1 -- 172.21.15.178:0/2776456110 wait complete. 2024-09-15T01:45:40.761 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:40.761 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:45:40.761 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:45:40.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.855+0000 7f8b88420640 10 monclient: get_monmap_and_config 2024-09-15T01:45:40.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.855+0000 7f8b88420640 10 monclient: build_initial_monmap 2024-09-15T01:45:40.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.855+0000 7f8b88420640 10 monclient: monmap: 2024-09-15T01:45:40.870 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:40.870 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.870 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:40.856756+0000 2024-09-15T01:45:40.870 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:40.856756+0000 2024-09-15T01:45:40.870 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:40.871 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.871 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:40.871 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:40.871 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:40.871 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:40.871 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:40.871 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:40.871 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.856+0000 7f8b88420640 1 Processor -- start 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.856+0000 7f8b88420640 1 -- start start 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.856+0000 7f8b88420640 10 monclient: init 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 10 monclient: _add_conns ranks=[2,6,7,10,5,8,4,0,1,3,9] 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b800a2930 0x7f8b80132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 10 monclient(hunting): picked mon.noname-b con 0x7f8b800a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 0x7f8b8009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 10 monclient(hunting): picked mon.noname-d con 0x7f8b8009faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:40.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b800a08c0 0x7f8b800a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 10 monclient(hunting): picked mon.noname-i con 0x7f8b800a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:40.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8009e0f0 con 0x7f8b800a2930 2024-09-15T01:45:40.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8009dd80 con 0x7f8b8009faa0 2024-09-15T01:45:40.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800a2d00 con 0x7f8b800a08c0 2024-09-15T01:45:40.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:40.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b88420640 10 monclient(hunting): authenticate will time out at 1545.607247s 2024-09-15T01:45:40.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b800a2930 0x7f8b80132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b859bb640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 0x7f8b8009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b800a2930 0x7f8b80132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:56500/0 (socket says 172.21.15.178:56500) 2024-09-15T01:45:40.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b859bb640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 0x7f8b8009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57096/0 (socket says 172.21.15.178:57096) 2024-09-15T01:45:40.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 1 -- 172.21.15.178:0/2811486557 learned_addr learned my addr 172.21.15.178:0/2811486557 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:40.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 10 monclient(hunting): get_auth_request con 0x7f8b800a2930 auth_method 0 2024-09-15T01:45:40.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b859bb640 10 monclient(hunting): get_auth_request con 0x7f8b8009faa0 auth_method 0 2024-09-15T01:45:40.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b859bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b859bb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b859bb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b859bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b859bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b859bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 10 monclient(hunting): handle_auth_done global_id 34967 payload 274 2024-09-15T01:45:40.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 1 -- 172.21.15.178:0/2811486557 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b800a08c0 msgr2=0x7f8b800a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:40.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b869bd640 1 --2- 172.21.15.178:0/2811486557 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b800a08c0 0x7f8b800a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 1 --2- 172.21.15.178:0/2811486557 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b800a08c0 0x7f8b800a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 1 -- 172.21.15.178:0/2811486557 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 msgr2=0x7f8b8009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:40.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 1 --2- 172.21.15.178:0/2811486557 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 0x7f8b8009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 10 monclient: _finish_hunting 0 2024-09-15T01:45:40.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 1 monclient: found mon.noname-b 2024-09-15T01:45:40.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:40.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:45:40.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b861bc640 1 -- 172.21.15.178:0/2811486557 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8004bf60 con 0x7f8b800a2930 2024-09-15T01:45:40.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b869bd640 10 monclient: get_auth_request con 0x7f8b800a08c0 auth_method 0 2024-09-15T01:45:40.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b859bb640 1 --2- 172.21.15.178:0/2811486557 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 0x7f8b8009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:40.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.857+0000 7f8b869bd640 1 --2- 172.21.15.178:0/2811486557 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b800a08c0 0x7f8b800a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:45:40.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b861bc640 1 --2- 172.21.15.178:0/2811486557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b800a2930 0x7f8b80132de0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009940 tx=0x7f8b7c0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd54ff15391d92b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b851ba640 1 -- 172.21.15.178:0/2811486557 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b7c02ac60 con 0x7f8b800a2930 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b851ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b851ba640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b851ba640 10 monclient: dump: 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.878 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.878 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.878 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.878 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.878 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.878 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.878 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b851ba640 10 monclient: _finish_auth 0 2024-09-15T01:45:40.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b851ba640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:40.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b851ba640 1 -- 172.21.15.178:0/2811486557 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7c02ade0 con 0x7f8b800a2930 2024-09-15T01:45:40.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b88420640 5 monclient: authenticate success, global_id 34967 2024-09-15T01:45:40.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b88420640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:40.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b851ba640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:40.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b851ba640 1 -- 172.21.15.178:0/2811486557 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b7c031ca0 con 0x7f8b800a2930 2024-09-15T01:45:40.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b88420640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:40.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b88420640 4 set_mon_vals no callback set 2024-09-15T01:45:40.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b851ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b851ba640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:40.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.858+0000 7f8b851ba640 10 monclient: dump: 2024-09-15T01:45:40.880 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.880 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.880 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.880 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.880 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.880 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.880 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 10 monclient: shutdown 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 1 -- 172.21.15.178:0/2811486557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b800a2930 msgr2=0x7f8b80132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 1 --2- 172.21.15.178:0/2811486557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b800a2930 0x7f8b80132de0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009940 tx=0x7f8b7c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 1 -- 172.21.15.178:0/2811486557 shutdown_connections 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 1 --2- 172.21.15.178:0/2811486557 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b800a08c0 0x7f8b800a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 1 --2- 172.21.15.178:0/2811486557 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 0x7f8b8009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 1 --2- 172.21.15.178:0/2811486557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b800a2930 0x7f8b80132de0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 1 -- 172.21.15.178:0/2811486557 >> 172.21.15.178:0/2811486557 conn(0x7f8b800a5ed0 msgr2=0x7f8b800a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 1 -- 172.21.15.178:0/2811486557 shutdown_connections 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 1 -- 172.21.15.178:0/2811486557 wait complete. 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 10 monclient: build_initial_monmap 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.859+0000 7f8b88420640 10 monclient: monmap: 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:40.883 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:40.883 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:40.883 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:40.883 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:40.883 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:40.883 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:40.883 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:40.883 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:40.883 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:40.883 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 1 Processor -- start 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 1 -- start start 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 10 monclient: init 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 10 monclient: _add_conns ranks=[6,5,7,0,8,4,1,2,3] 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 0x7f8b8009afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 10 monclient(hunting): picked mon.noname-d con 0x7f8b8009faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b800a08c0 0x7f8b801316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 10 monclient(hunting): picked mon.noname-h con 0x7f8b800a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b800a2930 0x7f8b8011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 10 monclient(hunting): picked mon.noname-i con 0x7f8b800a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800a2390 con 0x7f8b800a08c0 2024-09-15T01:45:40.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8011edc0 con 0x7f8b8009faa0 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8011eae0 con 0x7f8b800a2930 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b88420640 10 monclient(hunting): authenticate will time out at 1545.610592s 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b861bc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 0x7f8b8009afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b861bc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 0x7f8b8009afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57098/0 (socket says 172.21.15.178:57098) 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b861bc640 1 -- 172.21.15.178:0/3438409911 learned_addr learned my addr 172.21.15.178:0/3438409911 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b861bc640 10 monclient(hunting): get_auth_request con 0x7f8b8009faa0 auth_method 0 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b861bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b861bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b861bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b861bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b861bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b861bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b869bd640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b800a2930 0x7f8b8011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b859bb640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b800a08c0 0x7f8b801316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b869bd640 10 monclient(hunting): get_auth_request con 0x7f8b800a2930 auth_method 0 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b869bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b869bd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b869bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b861bc640 10 monclient(hunting): handle_auth_done global_id 25476 payload 824 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.860+0000 7f8b861bc640 1 -- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b800a2930 msgr2=0x7f8b8011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b861bc640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b800a2930 0x7f8b8011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b861bc640 1 -- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b800a08c0 msgr2=0x7f8b801316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b861bc640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b800a08c0 0x7f8b801316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b861bc640 10 monclient: _finish_hunting 0 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b861bc640 1 monclient: found mon.noname-d 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b861bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b861bc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b861bc640 1 -- 172.21.15.178:0/3438409911 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b70009390 con 0x7f8b8009faa0 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b859bb640 10 monclient: get_auth_request con 0x7f8b800a08c0 auth_method 0 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b859bb640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b800a08c0 0x7f8b801316e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b861bc640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 0x7f8b8009afc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c000c00 tx=0x7f8b7c032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b7c03b870 con 0x7f8b8009faa0 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b767fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b767fc640 10 monclient: dump: 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.887 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b767fc640 10 monclient: _finish_auth 0 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7c03ba30 con 0x7f8b8009faa0 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b88420640 5 monclient: authenticate success, global_id 25476 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b767fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b7c03bba0 con 0x7f8b8009faa0 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b88420640 10 monclient: _renew_subs 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b88420640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b88420640 1 -- 172.21.15.178:0/3438409911 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c009450 con 0x7f8b8009faa0 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b767fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b767fc640 10 monclient: dump: 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b849b9640 4 set_mon_vals no callback set 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b88420640 10 monclient: _renew_subs 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b88420640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.861+0000 7f8b88420640 1 -- 172.21.15.178:0/3438409911 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8011e800 con 0x7f8b8009faa0 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b88420640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b88420640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b80131c20 0x7f8b8011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b88420640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b88420640 1 -- 172.21.15.178:0/3438409911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8011ec80 con 0x7f8b80131c20 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8b7c039040 con 0x7f8b8009faa0 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b859bb640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b80131c20 0x7f8b8011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b767fc640 1 --2- 172.21.15.178:0/3438409911 >> v2:172.21.15.204:6812/28210 conn(0x7f8b640397b0 0x7f8b6403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b859bb640 10 monclient: get_auth_request con 0x7f8b80131c20 auth_method 0 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b859bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b859bb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b859bb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f8b7c06b2a0 con 0x7f8b8009faa0 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b859bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b859bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b859bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b869bd640 1 --2- 172.21.15.178:0/3438409911 >> v2:172.21.15.204:6812/28210 conn(0x7f8b640397b0 0x7f8b6403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b869bd640 10 monclient: get_auth_request con 0x7f8b640397b0 auth_method 0 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b859bb640 10 monclient(hunting): handle_auth_done global_id 54541 payload 824 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.862+0000 7f8b859bb640 1 -- 172.21.15.178:0/3438409911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b8004c4a0 con 0x7f8b80131c20 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b859bb640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b80131c20 0x7f8b8011ca70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b70010910 tx=0x7f8b70008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b869bd640 1 --2- 172.21.15.178:0/3438409911 >> v2:172.21.15.204:6812/28210 conn(0x7f8b640397b0 0x7f8b6403bc60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8b8009b670 tx=0x7f8b78007780 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b7001c400 con 0x7f8b80131c20 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b767fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b767fc640 10 monclient: dump: 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.893 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.894 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.894 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.894 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f8b7000eae0 con 0x7f8b80131c20 2024-09-15T01:45:40.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b767fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:40.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b767fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:40.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b80131c20 msgr2=0x7f8b8011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b767fc640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b80131c20 0x7f8b8011ca70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b70010910 tx=0x7f8b70008040 comp rx=0 tx=0).stop 2024-09-15T01:45:40.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b80131c20 msgr2=0x7f8b8011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:40.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.863+0000 7f8b767fc640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b80131c20 0x7f8b8011ca70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b88420640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:40.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b88420640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8011cfb0 0x7f8b80096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:40.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b88420640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:40.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b88420640 1 -- 172.21.15.178:0/3438409911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8011e990 con 0x7f8b8011cfb0 2024-09-15T01:45:40.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b859bb640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8011cfb0 0x7f8b80096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:40.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b859bb640 10 monclient: get_auth_request con 0x7f8b8011cfb0 auth_method 0 2024-09-15T01:45:40.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b859bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:40.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b859bb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:40.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b859bb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:40.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b859bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:40.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b859bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b859bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b859bb640 10 monclient(hunting): handle_auth_done global_id 54550 payload 824 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.869+0000 7f8b859bb640 1 -- 172.21.15.178:0/3438409911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8b8004bc80 con 0x7f8b8011cfb0 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b859bb640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8011cfb0 0x7f8b80096a30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8b70010910 tx=0x7f8b7001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b700278c0 con 0x7f8b8011cfb0 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b767fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b767fc640 10 monclient: dump: 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:40.897 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f8b70027440 con 0x7f8b8011cfb0 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b767fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b767fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8011cfb0 msgr2=0x7f8b80096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b767fc640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8011cfb0 0x7f8b80096a30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8b70010910 tx=0x7f8b7001c000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b767fc640 1 -- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8011cfb0 msgr2=0x7f8b80096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:40.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b767fc640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8011cfb0 0x7f8b80096a30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:40.899 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:40.899 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:40.899 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:40.899 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:40.899 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:40.899 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:40.899 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:40.900 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:40.900 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:40.900 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:40.900 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:40.900 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:40.900 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:40.900 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:40.900 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:40.900 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.900 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:45:40.901 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:40.901 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:40.901 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:40.901 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.901 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.901 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.901 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.901 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.901 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.901 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.901 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.902 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.902 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.902 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:40.902 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:40.902 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.902 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.902 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.902 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.903 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.903 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.903 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.903 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.903 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.903 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.903 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.903 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:40.904 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:40.904 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:40.904 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:40.904 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:40.904 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:40.904 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:40.904 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:40.905 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:40.905 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:40.905 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:40.905 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:40.905 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:40.905 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:40.905 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:40.905 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:40.905 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:40.906 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:40.906 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:40.906 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:40.906 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:40.906 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:40.906 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:40.906 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:40.906 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:40.906 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:40.906 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.906 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:40.906 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.907 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.908 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:40.908 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.908 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.908 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.908 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.908 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:40.908 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:40.908 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.908 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.908 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.909 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:40.909 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:40.909 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.909 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.909 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.909 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:40.909 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.910 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.910 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.910 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:40.910 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.910 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.910 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.910 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:40.910 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:40.910 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.910 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.910 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.911 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:40.911 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:40.911 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.911 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.911 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.911 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:40.912 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.912 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.912 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.912 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.912 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:40.912 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.912 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.912 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.913 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.914 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.915 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.916 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.918 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.919 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.920 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:40.921 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.922 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.923 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.924 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:40.924 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:40.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:40.924 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:40.924 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:40.924 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:40.924 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:40.924 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:40.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:40.924 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:40.924 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:40.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b88420640 1 -- 172.21.15.178:0/3438409911 >> v2:172.21.15.204:6812/28210 conn(0x7f8b640397b0 msgr2=0x7f8b6403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b88420640 1 --2- 172.21.15.178:0/3438409911 >> v2:172.21.15.204:6812/28210 conn(0x7f8b640397b0 0x7f8b6403bc60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8b8009b670 tx=0x7f8b78007780 comp rx=0 tx=0).stop 2024-09-15T01:45:40.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b88420640 10 monclient: shutdown 2024-09-15T01:45:40.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b88420640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:40.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b88420640 1 -- 172.21.15.178:0/3438409911 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 msgr2=0x7f8b8009afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:40.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b88420640 1 --2- 172.21.15.178:0/3438409911 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b8009faa0 0x7f8b8009afc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c000c00 tx=0x7f8b7c032000 comp rx=0 tx=0).stop 2024-09-15T01:45:40.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b869bd640 1 -- 172.21.15.178:0/3438409911 reap_dead start 2024-09-15T01:45:40.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b869bd640 1 -- 172.21.15.178:0/3438409911 reap_dead start 2024-09-15T01:45:40.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b88420640 1 -- 172.21.15.178:0/3438409911 shutdown_connections 2024-09-15T01:45:40.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b88420640 1 -- 172.21.15.178:0/3438409911 >> 172.21.15.178:0/3438409911 conn(0x7f8b800a5ed0 msgr2=0x7f8b80135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:40.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b88420640 1 -- 172.21.15.178:0/3438409911 shutdown_connections 2024-09-15T01:45:40.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:40.870+0000 7f8b88420640 1 -- 172.21.15.178:0/3438409911 wait complete. 2024-09-15T01:45:40.927 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:40.927 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:45:40.927 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:45:41.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.018+0000 7f9373ac0640 10 monclient: get_monmap_and_config 2024-09-15T01:45:41.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.018+0000 7f9373ac0640 10 monclient: build_initial_monmap 2024-09-15T01:45:41.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.019+0000 7f9373ac0640 10 monclient: monmap: 2024-09-15T01:45:41.021 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:41.021 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.021 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:41.019850+0000 2024-09-15T01:45:41.021 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:41.019850+0000 2024-09-15T01:45:41.021 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:41.021 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.021 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:41.021 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:41.022 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:41.022 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:41.022 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:41.022 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:41.022 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:41.022 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:41.022 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:41.022 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:41.022 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:41.022 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.019+0000 7f9373ac0640 1 Processor -- start 2024-09-15T01:45:41.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.019+0000 7f9373ac0640 1 -- start start 2024-09-15T01:45:41.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.019+0000 7f9373ac0640 10 monclient: init 2024-09-15T01:45:41.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:41.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 10 monclient: _add_conns ranks=[0,4,9,8,1,2,5,7,6,3,10] 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f936c0a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f936c132de0 0x7f936c137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 10 monclient(hunting): picked mon.noname-c con 0x7f936c132de0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f936c1380e0 0x7f936c13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 10 monclient(hunting): picked mon.noname-a con 0x7f936c1380e0 addr v2:172.21.15.178:3300/0 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936c04bf60 con 0x7f936c0a2930 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936c09dd80 con 0x7f936c1380e0 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0a2d00 con 0x7f936c132de0 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f9373ac0640 10 monclient(hunting): authenticate will time out at 1545.770395s 2024-09-15T01:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937205d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f936c1380e0 0x7f936c13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f936c132de0 0x7f936c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937185c640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f936c0a2930 0x7f936c09faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:38436/0 (socket says 172.21.15.178:38436) 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937185c640 1 -- 172.21.15.178:0/82282468 learned_addr learned my addr 172.21.15.178:0/82282468 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937205d640 10 monclient(hunting): get_auth_request con 0x7f936c1380e0 auth_method 0 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937205d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937205d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937205d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 10 monclient(hunting): get_auth_request con 0x7f936c132de0 auth_method 0 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937205d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937205d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937205d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 10 monclient(hunting): handle_auth_done global_id 44590 payload 274 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 1 -- 172.21.15.178:0/82282468 >> v2:172.21.15.178:3300/0 conn(0x7f936c1380e0 msgr2=0x7f936c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 1 --2- 172.21.15.178:0/82282468 >> v2:172.21.15.178:3300/0 conn(0x7f936c1380e0 0x7f936c13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 1 -- 172.21.15.178:0/82282468 >> v1:172.21.15.178:6789/0 conn(0x7f936c0a2930 legacy=0x7f936c09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 10 monclient: _finish_hunting 0 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 1 monclient: found mon.noname-c 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937085a640 1 -- 172.21.15.178:0/82282468 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395664185 0 0) 0x7f936c04bf60 con 0x7f936c0a2930 2024-09-15T01:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937105b640 1 -- 172.21.15.178:0/82282468 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c1331b0 con 0x7f936c132de0 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937085a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.020+0000 7f937205d640 1 --2- 172.21.15.178:0/82282468 >> v2:172.21.15.178:3300/0 conn(0x7f936c1380e0 0x7f936c13a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937105b640 1 --2- 172.21.15.178:0/82282468 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f936c132de0 0x7f936c137ba0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f935c009850 tx=0x7f935c0090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1aa6a356f9c11923 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937085a640 1 -- 172.21.15.178:0/82282468 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f935c02ac80 con 0x7f936c132de0 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937085a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937085a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937085a640 10 monclient: dump: 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937085a640 10 monclient: _finish_auth 0 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937085a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f9373ac0640 5 monclient: authenticate success, global_id 44590 2024-09-15T01:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f9373ac0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937085a640 1 -- 172.21.15.178:0/82282468 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c02ae00 con 0x7f936c132de0 2024-09-15T01:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937085a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f9373ac0640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f9373ac0640 4 set_mon_vals no callback set 2024-09-15T01:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937085a640 1 -- 172.21.15.178:0/82282468 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f935c031820 con 0x7f936c132de0 2024-09-15T01:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937085a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937085a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.021+0000 7f937085a640 10 monclient: dump: 2024-09-15T01:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.039 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.039 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.022+0000 7f9373ac0640 10 monclient: shutdown 2024-09-15T01:45:41.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.022+0000 7f9373ac0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:41.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.022+0000 7f9373ac0640 1 -- 172.21.15.178:0/82282468 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f936c132de0 msgr2=0x7f936c137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.022+0000 7f9373ac0640 1 --2- 172.21.15.178:0/82282468 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f936c132de0 0x7f936c137ba0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f935c009850 tx=0x7f935c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.022+0000 7f9373ac0640 1 -- 172.21.15.178:0/82282468 shutdown_connections 2024-09-15T01:45:41.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.022+0000 7f9373ac0640 1 --2- 172.21.15.178:0/82282468 >> v2:172.21.15.178:3300/0 conn(0x7f936c1380e0 0x7f936c13a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.022+0000 7f9373ac0640 1 --2- 172.21.15.178:0/82282468 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f936c132de0 0x7f936c137ba0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.022+0000 7f9373ac0640 1 -- 172.21.15.178:0/82282468 >> 172.21.15.178:0/82282468 conn(0x7f936c0a5ed0 msgr2=0x7f936c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:41.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.022+0000 7f9373ac0640 1 -- 172.21.15.178:0/82282468 shutdown_connections 2024-09-15T01:45:41.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.022+0000 7f9373ac0640 1 -- 172.21.15.178:0/82282468 wait complete. 2024-09-15T01:45:41.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.022+0000 7f9373ac0640 10 monclient: build_initial_monmap 2024-09-15T01:45:41.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.022+0000 7f9373ac0640 10 monclient: monmap: 2024-09-15T01:45:41.040 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:41.040 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:41.040 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:41.040 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:41.040 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:41.040 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:41.040 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:41.040 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:41.041 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:41.041 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:41.041 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:41.041 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:41.041 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:41.041 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:41.041 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:41.041 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 1 Processor -- start 2024-09-15T01:45:41.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 1 -- start start 2024-09-15T01:45:41.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 10 monclient: init 2024-09-15T01:45:41.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:41.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 10 monclient: _add_conns ranks=[2,3,5,0,1,8,6,4,7] 2024-09-15T01:45:41.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f936c0a2930 0x7f936c132460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 10 monclient(hunting): picked mon.noname-b con 0x7f936c0a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:41.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c132de0 0x7f936c09a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 10 monclient(hunting): picked mon.noname-g con 0x7f936c132de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:45:41.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f936c1380e0 0x7f936c09aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 10 monclient(hunting): picked mon.noname-h con 0x7f936c1380e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:41.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c09d630 con 0x7f936c0a2930 2024-09-15T01:45:41.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c09f870 con 0x7f936c132de0 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f936c11d410 con 0x7f936c1380e0 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f9373ac0640 10 monclient(hunting): authenticate will time out at 1545.773724s 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f937185c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f936c0a2930 0x7f936c132460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f937185c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f936c0a2930 0x7f936c132460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:56510/0 (socket says 172.21.15.178:56510) 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f937185c640 1 -- 172.21.15.178:0/792228365 learned_addr learned my addr 172.21.15.178:0/792228365 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f937185c640 10 monclient(hunting): get_auth_request con 0x7f936c0a2930 auth_method 0 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f937185c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f937185c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f937185c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f937185c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f937185c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f937185c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.023+0000 7f937205d640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f936c1380e0 0x7f936c09aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937105b640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c132de0 0x7f936c09a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937205d640 10 monclient(hunting): get_auth_request con 0x7f936c1380e0 auth_method 0 2024-09-15T01:45:41.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937205d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937205d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937205d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937105b640 10 monclient(hunting): get_auth_request con 0x7f936c132de0 auth_method 0 2024-09-15T01:45:41.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937105b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937105b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937105b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937185c640 10 monclient(hunting): handle_auth_done global_id 34976 payload 824 2024-09-15T01:45:41.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937185c640 1 -- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f936c1380e0 msgr2=0x7f936c09aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937185c640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f936c1380e0 0x7f936c09aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937185c640 1 -- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c132de0 msgr2=0x7f936c09a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937185c640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c132de0 0x7f936c09a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937185c640 10 monclient: _finish_hunting 0 2024-09-15T01:45:41.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937185c640 1 monclient: found mon.noname-b 2024-09-15T01:45:41.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937185c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:41.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937185c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:45:41.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937185c640 1 -- 172.21.15.178:0/792228365 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9364009a20 con 0x7f936c0a2930 2024-09-15T01:45:41.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937205d640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f936c1380e0 0x7f936c09aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:45:41.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937105b640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c132de0 0x7f936c09a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:45:41.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f937185c640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f936c0a2930 0x7f936c132460 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f936800c510 tx=0x7f9368007f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93680070d0 con 0x7f936c0a2930 2024-09-15T01:45:41.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9361ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9361ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:45:41.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9361ffb640 10 monclient: dump: 2024-09-15T01:45:41.046 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.046 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.046 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.046 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.046 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.046 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.046 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.047 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.047 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.047 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.047 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.047 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.047 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.047 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.047 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.047 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9361ffb640 10 monclient: _finish_auth 0 2024-09-15T01:45:41.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9361ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:41.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9373ac0640 5 monclient: authenticate success, global_id 34976 2024-09-15T01:45:41.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9368007250 con 0x7f936c0a2930 2024-09-15T01:45:41.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9373ac0640 10 monclient: _renew_subs 2024-09-15T01:45:41.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9373ac0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:45:41.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9373ac0640 1 -- 172.21.15.178:0/792228365 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c009390 con 0x7f936c0a2930 2024-09-15T01:45:41.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9361ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:41.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9368011360 con 0x7f936c0a2930 2024-09-15T01:45:41.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9361ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9361ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:41.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9361ffb640 10 monclient: dump: 2024-09-15T01:45:41.049 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.049 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.049 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9373ac0640 10 monclient: _renew_subs 2024-09-15T01:45:41.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9373ac0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:45:41.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9373ac0640 1 -- 172.21.15.178:0/792228365 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c09c9a0 con 0x7f936c0a2930 2024-09-15T01:45:41.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.024+0000 7f9363fff640 4 set_mon_vals no callback set 2024-09-15T01:45:41.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9368007400 con 0x7f936c0a2930 2024-09-15T01:45:41.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f9373ac0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:41.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f9373ac0640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c09afc0 0x7f936c096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f9373ac0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f9373ac0640 1 -- 172.21.15.178:0/792228365 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c04c4a0 con 0x7f936c09afc0 2024-09-15T01:45:41.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f9361ffb640 1 --2- 172.21.15.178:0/792228365 >> v2:172.21.15.204:6812/28210 conn(0x7f93500396e0 0x7f935003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f936804acc0 con 0x7f936c0a2930 2024-09-15T01:45:41.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f937205d640 1 --2- 172.21.15.178:0/792228365 >> v2:172.21.15.204:6812/28210 conn(0x7f93500396e0 0x7f935003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f937105b640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c09afc0 0x7f936c096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f937205d640 10 monclient: get_auth_request con 0x7f93500396e0 auth_method 0 2024-09-15T01:45:41.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f937105b640 10 monclient: get_auth_request con 0x7f936c09afc0 auth_method 0 2024-09-15T01:45:41.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f937105b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f937105b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.025+0000 7f937105b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f937105b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f937105b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f937105b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f937105b640 10 monclient(hunting): handle_auth_done global_id 64128 payload 824 2024-09-15T01:45:41.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f937105b640 1 -- 172.21.15.178:0/792228365 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f936c05ed70 con 0x7f936c09afc0 2024-09-15T01:45:41.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f937205d640 1 --2- 172.21.15.178:0/792228365 >> v2:172.21.15.204:6812/28210 conn(0x7f93500396e0 0x7f935003bb90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f93640099f0 tx=0x7f936400b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f937105b640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c09afc0 0x7f936c096a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f936c0a2f70 tx=0x7f935c032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f935c02ac80 con 0x7f936c09afc0 2024-09-15T01:45:41.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f9361ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f9361ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:45:41.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f9361ffb640 10 monclient: dump: 2024-09-15T01:45:41.053 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.053 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.053 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.053 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.053 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.053 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.053 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.053 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f935c032e30 con 0x7f936c09afc0 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f9361ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f9361ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c09afc0 msgr2=0x7f936c096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f9361ffb640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c09afc0 0x7f936c096a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f936c0a2f70 tx=0x7f935c032000 comp rx=0 tx=0).stop 2024-09-15T01:45:41.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c09afc0 msgr2=0x7f936c096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:41.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.026+0000 7f9361ffb640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c09afc0 0x7f936c096a30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.032+0000 7f9373ac0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.032+0000 7f9373ac0640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c096f70 0x7f936c097370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.032+0000 7f9373ac0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.032+0000 7f9373ac0640 1 -- 172.21.15.178:0/792228365 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c04bc80 con 0x7f936c096f70 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.032+0000 7f937105b640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c096f70 0x7f936c097370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.032+0000 7f937105b640 10 monclient: get_auth_request con 0x7f936c096f70 auth_method 0 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.032+0000 7f937105b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.032+0000 7f937105b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.032+0000 7f937105b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.032+0000 7f937105b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.032+0000 7f937105b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.032+0000 7f937105b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f937105b640 10 monclient(hunting): handle_auth_done global_id 64137 payload 824 2024-09-15T01:45:41.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f937105b640 1 -- 172.21.15.178:0/792228365 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f936c11e130 con 0x7f936c096f70 2024-09-15T01:45:41.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f937105b640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c096f70 0x7f936c097370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f935c009850 tx=0x7f935c03c000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f935c043810 con 0x7f936c096f70 2024-09-15T01:45:41.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f9361ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f9361ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:45:41.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f9361ffb640 10 monclient: dump: 2024-09-15T01:45:41.058 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.058 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.058 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.058 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.058 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.058 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f935c043390 con 0x7f936c096f70 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f9361ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f9361ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:41.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c096f70 msgr2=0x7f936c097370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f9361ffb640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c096f70 0x7f936c097370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f935c009850 tx=0x7f935c03c000 comp rx=0 tx=0).stop 2024-09-15T01:45:41.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.033+0000 7f9361ffb640 1 -- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c096f70 msgr2=0x7f936c097370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:41.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f9361ffb640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f936c096f70 0x7f936c097370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.060 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:41.060 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:41.060 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:41.060 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:41.060 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:41.060 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:41.060 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:41.061 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:41.062 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:41.062 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:41.062 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:41.062 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:41.062 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:41.062 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:41.062 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:41.062 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:41.062 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:41.062 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.063 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:41.063 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:41.063 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:41.063 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:41.063 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:41.063 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:41.063 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:41.063 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:41.063 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:41.063 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:41.063 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:41.063 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:41.064 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:41.065 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.068 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.068 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:41.068 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:41.068 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.068 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.068 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.069 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.069 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:41.069 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.069 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.069 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:41.069 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.069 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.069 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.070 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.071 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:41.071 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.071 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.071 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:41.071 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.071 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.071 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.073 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.073 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:41.073 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.073 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.073 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:41.073 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.073 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.073 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.074 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.075 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:41.075 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.075 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.075 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:41.075 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.075 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.075 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.075 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.076 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:41.077 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.077 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.077 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:41.077 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.077 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.077 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.077 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:41.077 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:41.077 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.079 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.079 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:41.079 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.079 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.079 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.079 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:41.079 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:41.079 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.079 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.080 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.080 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:41.080 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:41.080 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.080 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.080 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.080 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:41.080 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.081 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.081 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:41.081 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.081 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.081 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.081 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:41.081 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:41.081 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.082 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.082 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.082 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:41.082 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:41.082 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.082 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.082 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.082 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:41.082 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.083 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.084 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.084 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.084 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:41.084 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:41.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.084 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:41.084 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:41.084 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:41.084 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.084 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.085 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:41.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.085 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:41.085 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:41.085 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:41.085 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.085 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.085 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:41.085 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:41.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f9373ac0640 1 -- 172.21.15.178:0/792228365 >> v2:172.21.15.204:6812/28210 conn(0x7f93500396e0 msgr2=0x7f935003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f9373ac0640 1 --2- 172.21.15.178:0/792228365 >> v2:172.21.15.204:6812/28210 conn(0x7f93500396e0 0x7f935003bb90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f93640099f0 tx=0x7f936400b040 comp rx=0 tx=0).stop 2024-09-15T01:45:41.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f9373ac0640 10 monclient: shutdown 2024-09-15T01:45:41.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f9373ac0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:41.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f9373ac0640 1 -- 172.21.15.178:0/792228365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f936c0a2930 msgr2=0x7f936c132460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f9373ac0640 1 --2- 172.21.15.178:0/792228365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f936c0a2930 0x7f936c132460 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f936800c510 tx=0x7f9368007f30 comp rx=0 tx=0).stop 2024-09-15T01:45:41.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f937205d640 1 -- 172.21.15.178:0/792228365 reap_dead start 2024-09-15T01:45:41.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f937205d640 1 -- 172.21.15.178:0/792228365 reap_dead start 2024-09-15T01:45:41.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f9373ac0640 1 -- 172.21.15.178:0/792228365 shutdown_connections 2024-09-15T01:45:41.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f9373ac0640 1 -- 172.21.15.178:0/792228365 >> 172.21.15.178:0/792228365 conn(0x7f936c0a5ed0 msgr2=0x7f936c1358d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:41.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f9373ac0640 1 -- 172.21.15.178:0/792228365 shutdown_connections 2024-09-15T01:45:41.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.034+0000 7f9373ac0640 1 -- 172.21.15.178:0/792228365 wait complete. 2024-09-15T01:45:41.088 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:41.088 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:45:41.088 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:45:41.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.179+0000 7f0c2fc8b640 10 monclient: get_monmap_and_config 2024-09-15T01:45:41.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.179+0000 7f0c2fc8b640 10 monclient: build_initial_monmap 2024-09-15T01:45:41.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.179+0000 7f0c2fc8b640 10 monclient: monmap: 2024-09-15T01:45:41.182 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:41.182 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.182 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:41.180455+0000 2024-09-15T01:45:41.182 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:41.180455+0000 2024-09-15T01:45:41.182 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:41.182 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.182 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:41.182 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:41.182 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:41.182 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:41.183 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:41.183 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:41.183 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:41.183 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:41.183 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:41.183 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:41.183 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:41.183 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 1 Processor -- start 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 1 -- start start 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 10 monclient: init 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 10 monclient: _add_conns ranks=[0,7,10,4,1,9,8,5,3,2,6] 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0c280a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c28132de0 0x7f0c28137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 10 monclient(hunting): picked mon.noname-i con 0x7f0c28132de0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0c281380e0 0x7f0c2813a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 10 monclient(hunting): picked mon.noname-f con 0x7f0c281380e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2804bf60 con 0x7f0c280a2930 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2809dd80 con 0x7f0c281380e0 2024-09-15T01:45:41.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280a2d00 con 0x7f0c28132de0 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.180+0000 7f0c2fc8b640 10 monclient(hunting): authenticate will time out at 1545.930937s 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2da27640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f0c280a2930 0x7f0c2809faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:38442/0 (socket says 172.21.15.178:38442) 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2da27640 1 -- 172.21.15.178:0/2522050069 learned_addr learned my addr 172.21.15.178:0/2522050069 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785464742 0 0) 0x7f0c2804bf60 con 0x7f0c280a2930 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient(hunting): my global_id is 64179 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 1 --2- 172.21.15.178:0/2522050069 >> v2:172.21.15.204:3300/0 conn(0x7f0c281380e0 0x7f0c2813a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c28132de0 0x7f0c28137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c140034e0 con 0x7f0c280a2930 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): get_auth_request con 0x7f0c281380e0 auth_method 0 2024-09-15T01:45:41.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): get_auth_request con 0x7f0c28132de0 auth_method 0 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1795034652 0 0) 0x7f0c140034e0 con 0x7f0c280a2930 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c28132de0 msgr2=0x7f0c28137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c28132de0 0x7f0c28137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 >> v2:172.21.15.204:3300/0 conn(0x7f0c281380e0 msgr2=0x7f0c2813a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 --2- 172.21.15.178:0/2522050069 >> v2:172.21.15.204:3300/0 conn(0x7f0c281380e0 0x7f0c2813a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient: _finish_hunting 0 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c281331b0 con 0x7f0c280a2930 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient: _finish_auth 0 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2fc8b640 5 monclient: authenticate success, global_id 64179 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f0c240018f0 con 0x7f0c280a2930 2024-09-15T01:45:41.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2fc8b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient: dump: 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.202 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 >> v1:172.21.15.178:6789/0 conn(0x7f0c280a2930 legacy=0x7f0c2809faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient: _add_conns ranks=[4,3,6,5,2,0,1,7,8] 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c140041d0 0x7f0c14004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient(hunting): picked mon.c con 0x7f0c140041d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c14004cc0 0x7f0c14005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient(hunting): picked mon.g con 0x7f0c14004cc0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c14009a10 0x7f0c1400be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient(hunting): picked mon.d con 0x7f0c14009a10 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2809dd80 con 0x7f0c14004cc0 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280a2d00 con 0x7f0c140041d0 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1400c7a0 con 0x7f0c14009a10 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2ca25640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2fc8b640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2da27640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c140041d0 0x7f0c14004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2da27640 10 monclient(hunting): get_auth_request con 0x7f0c140041d0 auth_method 0 2024-09-15T01:45:41.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2da27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2da27640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2da27640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c14009a10 0x7f0c1400be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c14004cc0 0x7f0c14005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): get_auth_request con 0x7f0c14009a10 auth_method 0 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): get_auth_request con 0x7f0c14004cc0 auth_method 0 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2da27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2da27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2da27640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2e228640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.181+0000 7f0c2d226640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 10 monclient(hunting): handle_auth_done global_id 64179 payload 293 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 1 -- 172.21.15.178:0/2522050069 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c14009a10 msgr2=0x7f0c1400be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c14009a10 0x7f0c1400be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 1 -- 172.21.15.178:0/2522050069 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c14004cc0 msgr2=0x7f0c14005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c14004cc0 0x7f0c14005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 10 monclient: _finish_hunting 0 2024-09-15T01:45:41.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 1 monclient: found mon.c 2024-09-15T01:45:41.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:41.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:41.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 1 -- 172.21.15.178:0/2522050069 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c140034e0 con 0x7f0c140041d0 2024-09-15T01:45:41.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 10 monclient: _finish_auth 0 2024-09-15T01:45:41.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:41.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2e228640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c14009a10 0x7f0c1400be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:41.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2e228640 1 -- 172.21.15.178:0/2522050069 reap_dead start 2024-09-15T01:45:41.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2da27640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c140041d0 0x7f0c14004780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0c240018c0 tx=0x7f0c240048f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d9a4eec4cfdf43af server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c24033030 con 0x7f0c140041d0 2024-09-15T01:45:41.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2ca25640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2ca25640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2ca25640 10 monclient: dump: 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.209 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.209 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.209 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.209 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2fc8b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:41.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c24005de0 con 0x7f0c140041d0 2024-09-15T01:45:41.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2ca25640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:41.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2fc8b640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:41.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2ca25640 1 -- 172.21.15.178:0/2522050069 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c240085d0 con 0x7f0c140041d0 2024-09-15T01:45:41.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2fc8b640 4 set_mon_vals no callback set 2024-09-15T01:45:41.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2ca25640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2ca25640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.182+0000 7f0c2ca25640 10 monclient: dump: 2024-09-15T01:45:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.210 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.211 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.183+0000 7f0c2fc8b640 10 monclient: shutdown 2024-09-15T01:45:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.183+0000 7f0c2fc8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.183+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/2522050069 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c140041d0 msgr2=0x7f0c14004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.183+0000 7f0c2fc8b640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c140041d0 0x7f0c14004780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0c240018c0 tx=0x7f0c240048f0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.183+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/2522050069 shutdown_connections 2024-09-15T01:45:41.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.183+0000 7f0c2fc8b640 1 --2- 172.21.15.178:0/2522050069 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0c140041d0 0x7f0c14004780 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.183+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/2522050069 >> 172.21.15.178:0/2522050069 conn(0x7f0c280a5ed0 msgr2=0x7f0c280a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.183+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/2522050069 shutdown_connections 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.183+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/2522050069 wait complete. 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient: build_initial_monmap 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient: monmap: 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:41.212 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:41.213 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:41.213 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:41.213 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:41.213 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:41.213 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:41.213 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:41.213 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:41.213 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 1 Processor -- start 2024-09-15T01:45:41.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 1 -- start start 2024-09-15T01:45:41.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient: init 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient: _add_conns ranks=[1,7,2,6,8,4,3,0,5] 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c140041d0 0x7f0c2813e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient(hunting): picked mon.noname-f con 0x7f0c140041d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c2809b680 0x7f0c28132200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient(hunting): picked mon.noname-i con 0x7f0c2809b680 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c28132740 0x7f0c28141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient(hunting): picked mon.noname-b con 0x7f0c28132740 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2809c010 con 0x7f0c140041d0 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2809d630 con 0x7f0c28132740 2024-09-15T01:45:41.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2809f870 con 0x7f0c2809b680 2024-09-15T01:45:41.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:41.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2fc8b640 10 monclient(hunting): authenticate will time out at 1545.934861s 2024-09-15T01:45:41.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2e228640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c28132740 0x7f0c28141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.184+0000 7f0c2e228640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c28132740 0x7f0c28141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:56518/0 (socket says 172.21.15.178:56518) 2024-09-15T01:45:41.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 1 -- 172.21.15.178:0/1432569687 learned_addr learned my addr 172.21.15.178:0/1432569687 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:41.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 10 monclient(hunting): get_auth_request con 0x7f0c28132740 auth_method 0 2024-09-15T01:45:41.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2da27640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c140041d0 0x7f0c2813e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2d226640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c2809b680 0x7f0c28132200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2da27640 10 monclient(hunting): get_auth_request con 0x7f0c140041d0 auth_method 0 2024-09-15T01:45:41.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2da27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2da27640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2da27640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2d226640 10 monclient(hunting): get_auth_request con 0x7f0c2809b680 auth_method 0 2024-09-15T01:45:41.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2d226640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2d226640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2d226640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2da27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2da27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2da27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2d226640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2d226640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2d226640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 10 monclient(hunting): handle_auth_done global_id 34985 payload 824 2024-09-15T01:45:41.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 1 -- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c2809b680 msgr2=0x7f0c28132200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f0c2809b680 0x7f0c28132200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 1 -- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c140041d0 msgr2=0x7f0c2813e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c140041d0 0x7f0c2813e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 10 monclient: _finish_hunting 0 2024-09-15T01:45:41.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 1 monclient: found mon.noname-b 2024-09-15T01:45:41.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:41.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:45:41.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 1 -- 172.21.15.178:0/1432569687 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2813f150 con 0x7f0c28132740 2024-09-15T01:45:41.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2e228640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c28132740 0x7f0c28141a90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0c28064500 tx=0x7f0c2000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c2000e040 con 0x7f0c28132740 2024-09-15T01:45:41.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:45:41.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1dffb640 10 monclient: dump: 2024-09-15T01:45:41.219 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.219 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.219 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.219 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.219 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.219 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1dffb640 10 monclient: _finish_auth 0 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2fc8b640 5 monclient: authenticate success, global_id 34985 2024-09-15T01:45:41.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2000f040 con 0x7f0c28132740 2024-09-15T01:45:41.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2fc8b640 10 monclient: _renew_subs 2024-09-15T01:45:41.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2fc8b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:45:41.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/1432569687 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2811d300 con 0x7f0c28132740 2024-09-15T01:45:41.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:41.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c20015b90 con 0x7f0c28132740 2024-09-15T01:45:41.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:41.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1dffb640 10 monclient: dump: 2024-09-15T01:45:41.221 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.221 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.221 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.221 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c1ffff640 4 set_mon_vals no callback set 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2fc8b640 10 monclient: _renew_subs 2024-09-15T01:45:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2fc8b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:45:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.185+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/1432569687 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2811c530 con 0x7f0c28132740 2024-09-15T01:45:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.186+0000 7f0c2fc8b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.186+0000 7f0c2fc8b640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28141fd0 0x7f0c281423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.186+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0c20015cf0 con 0x7f0c28132740 2024-09-15T01:45:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.186+0000 7f0c2fc8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.186+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/1432569687 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2811d780 con 0x7f0c28141fd0 2024-09-15T01:45:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.186+0000 7f0c1dffb640 1 --2- 172.21.15.178:0/1432569687 >> v2:172.21.15.204:6812/28210 conn(0x7f0c0c0397b0 0x7f0c0c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.186+0000 7f0c2da27640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28141fd0 0x7f0c281423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.186+0000 7f0c2da27640 10 monclient: get_auth_request con 0x7f0c28141fd0 auth_method 0 2024-09-15T01:45:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.186+0000 7f0c2da27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.186+0000 7f0c2da27640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.186+0000 7f0c2da27640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f0c2004c130 con 0x7f0c28132740 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c2da27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c2da27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c2da27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c2d226640 1 --2- 172.21.15.178:0/1432569687 >> v2:172.21.15.204:6812/28210 conn(0x7f0c0c0397b0 0x7f0c0c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c2d226640 10 monclient: get_auth_request con 0x7f0c0c0397b0 auth_method 0 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c2da27640 10 monclient(hunting): handle_auth_done global_id 7385 payload 824 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c2da27640 1 -- 172.21.15.178:0/1432569687 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c2804c4a0 con 0x7f0c28141fd0 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c2da27640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28141fd0 0x7f0c281423a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0c14004700 tx=0x7f0c24034000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c2d226640 1 --2- 172.21.15.178:0/1432569687 >> v2:172.21.15.204:6812/28210 conn(0x7f0c0c0397b0 0x7f0c0c03bc60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0c2809a410 tx=0x7f0c180090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c24033020 con 0x7f0c28141fd0 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c1dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:45:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c1dffb640 10 monclient: dump: 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f0c24008360 con 0x7f0c28141fd0 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c1dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c1dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28141fd0 msgr2=0x7f0c281423a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c1dffb640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28141fd0 0x7f0c281423a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0c14004700 tx=0x7f0c24034000 comp rx=0 tx=0).stop 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28141fd0 msgr2=0x7f0c281423a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.187+0000 7f0c1dffb640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28141fd0 0x7f0c281423a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.193+0000 7f0c2fc8b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.193+0000 7f0c2fc8b640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28096a30 0x7f0c28096e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.193+0000 7f0c2fc8b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.193+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/1432569687 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280a0220 con 0x7f0c28096a30 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.193+0000 7f0c2da27640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28096a30 0x7f0c28096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.193+0000 7f0c2da27640 10 monclient: get_auth_request con 0x7f0c28096a30 auth_method 0 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.193+0000 7f0c2da27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.193+0000 7f0c2da27640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.193+0000 7f0c2da27640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c2da27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c2da27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c2da27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c2da27640 10 monclient(hunting): handle_auth_done global_id 7394 payload 824 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c2da27640 1 -- 172.21.15.178:0/1432569687 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0c2804bc80 con 0x7f0c28096a30 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c2da27640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28096a30 0x7f0c28096e30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c24008d40 tx=0x7f0c24041000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c24041070 con 0x7f0c28096a30 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c1dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c1dffb640 10 monclient: dump: 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.229 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.229 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.229 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9314 (secure 0 0 0) 0x7f0c240467c0 con 0x7f0c28096a30 2024-09-15T01:45:41.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c1dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:41.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c1dffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:41.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28096a30 msgr2=0x7f0c28096e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c1dffb640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28096a30 0x7f0c28096e30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c24008d40 tx=0x7f0c24041000 comp rx=0 tx=0).stop 2024-09-15T01:45:41.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c1dffb640 1 -- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28096a30 msgr2=0x7f0c28096e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:41.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.194+0000 7f0c1dffb640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f0c28096a30 0x7f0c28096e30 secure :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c24008d40 tx=0x7f0c24041000 comp rx=0 tx=0).stop 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:41.230 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:41.231 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:41.232 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:41.233 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:41.234 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:41.235 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.236 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.237 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:41.238 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.241 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.242 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.242 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:41.242 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:41.242 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.242 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.242 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.242 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.243 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.244 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:41.244 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:41.244 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.244 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.244 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:41.244 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.245 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:41.245 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.245 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.245 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.245 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:41.245 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:41.245 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.245 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.247 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:41.247 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.247 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.247 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.247 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:41.247 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:41.247 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.247 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.248 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.248 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:41.248 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:41.248 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.248 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:41.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.250 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:41.250 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:41.250 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.250 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:41.250 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:41.251 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.251 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.251 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.251 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:41.251 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:41.251 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.251 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.251 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.251 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.251 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.252 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:41.252 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:41.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.252 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:41.252 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:41.252 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:41.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.252 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.253 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:45:41.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.253 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:41.253 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:41.253 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:45:41.253 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.253 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.253 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:41.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.253 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:41.254 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:41.254 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 6 2024-09-15T01:45:41.254 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.254 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.254 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:41.254 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:41.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.195+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/1432569687 >> v2:172.21.15.204:6812/28210 conn(0x7f0c0c0397b0 msgr2=0x7f0c0c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.195+0000 7f0c2fc8b640 1 --2- 172.21.15.178:0/1432569687 >> v2:172.21.15.204:6812/28210 conn(0x7f0c0c0397b0 0x7f0c0c03bc60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0c2809a410 tx=0x7f0c180090f0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.195+0000 7f0c2fc8b640 10 monclient: shutdown 2024-09-15T01:45:41.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.195+0000 7f0c2fc8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:41.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.195+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/1432569687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c28132740 msgr2=0x7f0c28141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.195+0000 7f0c2fc8b640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c28132740 0x7f0c28141a90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0c28064500 tx=0x7f0c2000b040 comp rx=0 tx=0).stop 2024-09-15T01:45:41.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.195+0000 7f0c2e228640 1 -- 172.21.15.178:0/1432569687 reap_dead start 2024-09-15T01:45:41.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.195+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/1432569687 shutdown_connections 2024-09-15T01:45:41.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.195+0000 7f0c2fc8b640 1 --2- 172.21.15.178:0/1432569687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c28132740 0x7f0c28141a90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.195+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/1432569687 >> 172.21.15.178:0/1432569687 conn(0x7f0c280a5ed0 msgr2=0x7f0c280a5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:41.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.195+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/1432569687 shutdown_connections 2024-09-15T01:45:41.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.195+0000 7f0c2fc8b640 1 -- 172.21.15.178:0/1432569687 wait complete. 2024-09-15T01:45:41.257 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:41.257 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:45:41.257 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:45:41.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.349+0000 7fea8359d640 10 monclient: get_monmap_and_config 2024-09-15T01:45:41.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.349+0000 7fea8359d640 10 monclient: build_initial_monmap 2024-09-15T01:45:41.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.349+0000 7fea8359d640 10 monclient: monmap: 2024-09-15T01:45:41.351 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:41.351 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.351 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:41.350169+0000 2024-09-15T01:45:41.351 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:41.350169+0000 2024-09-15T01:45:41.351 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:41.352 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.349+0000 7fea8359d640 1 Processor -- start 2024-09-15T01:45:41.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 1 -- start start 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 10 monclient: init 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 10 monclient: _add_conns ranks=[7,3,10,4,0,6,5,2,1,9,8] 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c0a2930 0x7fea7c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 10 monclient(hunting): picked mon.noname-i con 0x7fea7c0a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea7c09faa0 0x7fea7c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 10 monclient(hunting): picked mon.noname-g con 0x7fea7c09faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fea7c0a08c0 0x7fea7c0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 10 monclient(hunting): picked mon.noname-f con 0x7fea7c0a08c0 addr v2:172.21.15.204:3300/0 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea7c09e0f0 con 0x7fea7c0a08c0 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c09dd80 con 0x7fea7c09faa0 2024-09-15T01:45:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c0a2d00 con 0x7fea7c0a2930 2024-09-15T01:45:41.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:41.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea8359d640 10 monclient(hunting): authenticate will time out at 1546.100551s 2024-09-15T01:45:41.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81339640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c0a2930 0x7fea7c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea80b38640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea7c09faa0 0x7fea7c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81b3a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fea7c0a08c0 0x7fea7c0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81b3a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fea7c0a08c0 0x7fea7c0a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:39238/0 (socket says 172.21.15.178:39238) 2024-09-15T01:45:41.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81339640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c0a2930 0x7fea7c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:44226/0 (socket says 172.21.15.178:44226) 2024-09-15T01:45:41.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea80b38640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea7c09faa0 0x7fea7c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:53716/0 (socket says 172.21.15.178:53716) 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81b3a640 1 -- 172.21.15.178:0/1214015105 learned_addr learned my addr 172.21.15.178:0/1214015105 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81b3a640 10 monclient(hunting): get_auth_request con 0x7fea7c0a08c0 auth_method 0 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81339640 10 monclient(hunting): get_auth_request con 0x7fea7c0a2930 auth_method 0 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea80b38640 10 monclient(hunting): get_auth_request con 0x7fea7c09faa0 auth_method 0 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea80b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea80b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea80b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.350+0000 7fea81b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea80b38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea80b38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea80b38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81b3a640 10 monclient(hunting): handle_auth_done global_id 54577 payload 274 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81b3a640 1 -- 172.21.15.178:0/1214015105 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c0a2930 msgr2=0x7fea7c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81b3a640 1 --2- 172.21.15.178:0/1214015105 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c0a2930 0x7fea7c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81b3a640 1 -- 172.21.15.178:0/1214015105 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea7c09faa0 msgr2=0x7fea7c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81b3a640 1 --2- 172.21.15.178:0/1214015105 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea7c09faa0 0x7fea7c09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81b3a640 10 monclient: _finish_hunting 0 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81b3a640 1 monclient: found mon.noname-f 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81b3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81b3a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81b3a640 1 -- 172.21.15.178:0/1214015105 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c04bf60 con 0x7fea7c0a08c0 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea80b38640 1 --2- 172.21.15.178:0/1214015105 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea7c09faa0 0x7fea7c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.351+0000 7fea81b3a640 1 --2- 172.21.15.178:0/1214015105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c0a08c0 0x7fea7c0a0d30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fea74009f20 tx=0x7fea74009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb24c33c888bc5da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea6ffff640 1 -- 172.21.15.178:0/1214015105 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea74007880 con 0x7fea7c0a08c0 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea6ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea6ffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea6ffff640 10 monclient: dump: 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.367 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea6ffff640 10 monclient: _finish_auth 0 2024-09-15T01:45:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea6ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea6ffff640 1 -- 172.21.15.178:0/1214015105 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7402d050 con 0x7fea7c0a08c0 2024-09-15T01:45:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 5 monclient: authenticate success, global_id 54577 2024-09-15T01:45:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea6ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea6ffff640 1 -- 172.21.15.178:0/1214015105 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea74030360 con 0x7fea7c0a08c0 2024-09-15T01:45:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 4 set_mon_vals no callback set 2024-09-15T01:45:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea6ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea6ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:45:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea6ffff640 10 monclient: dump: 2024-09-15T01:45:41.369 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.369 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.369 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.369 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.369 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 10 monclient: shutdown 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:41.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 1 -- 172.21.15.178:0/1214015105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c0a08c0 msgr2=0x7fea7c0a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 1 --2- 172.21.15.178:0/1214015105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c0a08c0 0x7fea7c0a0d30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fea74009f20 tx=0x7fea74009840 comp rx=0 tx=0).stop 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 1 -- 172.21.15.178:0/1214015105 shutdown_connections 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 1 --2- 172.21.15.178:0/1214015105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c0a08c0 0x7fea7c0a0d30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 1 --2- 172.21.15.178:0/1214015105 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea7c09faa0 0x7fea7c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 1 --2- 172.21.15.178:0/1214015105 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c0a2930 0x7fea7c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 1 -- 172.21.15.178:0/1214015105 >> 172.21.15.178:0/1214015105 conn(0x7fea7c0a5ed0 msgr2=0x7fea7c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 1 -- 172.21.15.178:0/1214015105 shutdown_connections 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.352+0000 7fea8359d640 1 -- 172.21.15.178:0/1214015105 wait complete. 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.353+0000 7fea8359d640 10 monclient: build_initial_monmap 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.353+0000 7fea8359d640 10 monclient: monmap: 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.353+0000 7fea8359d640 1 Processor -- start 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.353+0000 7fea8359d640 1 -- start start 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.353+0000 7fea8359d640 10 monclient: init 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 10 monclient: _add_conns ranks=[4,3,1,6,0,7,5,2,8] 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea7c09faa0 0x7fea7c09ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 10 monclient(hunting): picked mon.noname-c con 0x7fea7c09faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea7c0a08c0 0x7fea7c131750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 10 monclient(hunting): picked mon.noname-g con 0x7fea7c0a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c0a2930 0x7fea7c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 10 monclient(hunting): picked mon.noname-f con 0x7fea7c0a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c11eec0 con 0x7fea7c0a2930 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c0a2390 con 0x7fea7c0a08c0 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c09d630 con 0x7fea7c09faa0 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea8359d640 10 monclient(hunting): authenticate will time out at 1546.104337s 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea7c09faa0 0x7fea7c09ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea7c09faa0 0x7fea7c09ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:35312/0 (socket says 172.21.15.178:35312) 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 1 -- 172.21.15.178:0/540099769 learned_addr learned my addr 172.21.15.178:0/540099769 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 10 monclient(hunting): get_auth_request con 0x7fea7c09faa0 auth_method 0 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81b3a640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c0a2930 0x7fea7c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea80b38640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea7c0a08c0 0x7fea7c131750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea80b38640 10 monclient(hunting): get_auth_request con 0x7fea7c0a08c0 auth_method 0 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea80b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea80b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea80b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81b3a640 10 monclient(hunting): get_auth_request con 0x7fea7c0a2930 auth_method 0 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 10 monclient(hunting): handle_auth_done global_id 44599 payload 824 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 1 -- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea7c0a08c0 msgr2=0x7fea7c131750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea7c0a08c0 0x7fea7c131750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 1 -- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c0a2930 msgr2=0x7fea7c11c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c0a2930 0x7fea7c11c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 10 monclient: _finish_hunting 0 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 1 monclient: found mon.noname-c 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 1 -- 172.21.15.178:0/540099769 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea78009390 con 0x7fea7c09faa0 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.354+0000 7fea81339640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea7c09faa0 0x7fea7c09ae70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fea7000ea50 tx=0x7fea700090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea7000ce90 con 0x7fea7c09faa0 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 10 monclient: dump: 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.377 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 10 monclient: _finish_auth 0 2024-09-15T01:45:41.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:41.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea8359d640 5 monclient: authenticate success, global_id 44599 2024-09-15T01:45:41.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea70009600 con 0x7fea7c09faa0 2024-09-15T01:45:41.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:41.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea700138f0 con 0x7fea7c09faa0 2024-09-15T01:45:41.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea8359d640 10 monclient: _renew_subs 2024-09-15T01:45:41.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea8359d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:41.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea8359d640 1 -- 172.21.15.178:0/540099769 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea74009ae0 con 0x7fea7c09faa0 2024-09-15T01:45:41.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6f7fe640 4 set_mon_vals no callback set 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 10 monclient: dump: 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.379 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea8359d640 10 monclient: _renew_subs 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea8359d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea8359d640 1 -- 172.21.15.178:0/540099769 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7c09b900 con 0x7fea7c09faa0 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fea70013470 con 0x7fea7c09faa0 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea8359d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea8359d640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c131c90 0x7fea7c11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea8359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.355+0000 7fea8359d640 1 -- 172.21.15.178:0/540099769 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c11ea10 con 0x7fea7c131c90 2024-09-15T01:45:41.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea6d7fa640 1 --2- 172.21.15.178:0/540099769 >> v2:172.21.15.204:6812/28210 conn(0x7fea600396e0 0x7fea6003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea80b38640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c131c90 0x7fea7c11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea80b38640 10 monclient: get_auth_request con 0x7fea7c131c90 auth_method 0 2024-09-15T01:45:41.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fea7004c3a0 con 0x7fea7c09faa0 2024-09-15T01:45:41.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea80b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea80b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea80b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea80b38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea80b38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea80b38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea81b3a640 1 --2- 172.21.15.178:0/540099769 >> v2:172.21.15.204:6812/28210 conn(0x7fea600396e0 0x7fea6003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea81b3a640 10 monclient: get_auth_request con 0x7fea600396e0 auth_method 0 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea80b38640 10 monclient(hunting): handle_auth_done global_id 64159 payload 824 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea80b38640 1 -- 172.21.15.178:0/540099769 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea7c04c4a0 con 0x7fea7c131c90 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea80b38640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c131c90 0x7fea7c11ca70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea78011760 tx=0x7fea78008040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea81b3a640 1 --2- 172.21.15.178:0/540099769 >> v2:172.21.15.204:6812/28210 conn(0x7fea600396e0 0x7fea6003bb90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fea74033000 tx=0x7fea74036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea7800fd30 con 0x7fea7c131c90 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea6d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea6d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.356+0000 7fea6d7fa640 10 monclient: dump: 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.357+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fea7800fe90 con 0x7fea7c131c90 2024-09-15T01:45:41.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.357+0000 7fea6d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:41.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.357+0000 7fea6d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:41.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.357+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c131c90 msgr2=0x7fea7c11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.357+0000 7fea6d7fa640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c131c90 0x7fea7c11ca70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea78011760 tx=0x7fea78008040 comp rx=0 tx=0).stop 2024-09-15T01:45:41.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.357+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c131c90 msgr2=0x7fea7c11ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:41.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.357+0000 7fea6d7fa640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c131c90 0x7fea7c11ca70 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea78011760 tx=0x7fea78008040 comp rx=0 tx=0).stop 2024-09-15T01:45:41.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea8359d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:41.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea8359d640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c11cfb0 0x7fea7c096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea8359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea8359d640 1 -- 172.21.15.178:0/540099769 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c132060 con 0x7fea7c11cfb0 2024-09-15T01:45:41.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea80b38640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c11cfb0 0x7fea7c096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea80b38640 10 monclient: get_auth_request con 0x7fea7c11cfb0 auth_method 0 2024-09-15T01:45:41.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea80b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea80b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea80b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea80b38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea80b38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea80b38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea80b38640 10 monclient(hunting): handle_auth_done global_id 64168 payload 824 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea80b38640 1 -- 172.21.15.178:0/540099769 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fea7c04bc80 con 0x7fea7c11cfb0 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.363+0000 7fea80b38640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c11cfb0 0x7fea7c096a30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea78011760 tx=0x7fea7801c000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea78022070 con 0x7fea7c11cfb0 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea6d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea6d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea6d7fa640 10 monclient: dump: 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fea78005970 con 0x7fea7c11cfb0 2024-09-15T01:45:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea6d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:41.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea6d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:41.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c11cfb0 msgr2=0x7fea7c096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea6d7fa640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c11cfb0 0x7fea7c096a30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea78011760 tx=0x7fea7801c000 comp rx=0 tx=0).stop 2024-09-15T01:45:41.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea6d7fa640 1 -- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c11cfb0 msgr2=0x7fea7c096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:41.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea6d7fa640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea7c11cfb0 0x7fea7c096a30 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea78011760 tx=0x7fea7801c000 comp rx=0 tx=0).stop 2024-09-15T01:45:41.389 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:41.389 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:41.389 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:41.389 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.390 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.391 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.392 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:41.393 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:41.393 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:41.393 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:41.393 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:41.393 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:41.393 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:41.393 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:41.393 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:41.393 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:41.393 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:41.393 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:41.393 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:41.394 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.397 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:41.399 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:41.402 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.403 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:41.404 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.404 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:41.404 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:41.405 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:41.407 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.408 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:41.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:41.410 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:41.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:41.412 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:41.412 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:41.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea8359d640 1 -- 172.21.15.178:0/540099769 >> v2:172.21.15.204:6812/28210 conn(0x7fea600396e0 msgr2=0x7fea6003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea8359d640 1 --2- 172.21.15.178:0/540099769 >> v2:172.21.15.204:6812/28210 conn(0x7fea600396e0 0x7fea6003bb90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fea74033000 tx=0x7fea74036000 comp rx=0 tx=0).stop 2024-09-15T01:45:41.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea8359d640 10 monclient: shutdown 2024-09-15T01:45:41.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea8359d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:41.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea8359d640 1 -- 172.21.15.178:0/540099769 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea7c09faa0 msgr2=0x7fea7c09ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.364+0000 7fea8359d640 1 --2- 172.21.15.178:0/540099769 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fea7c09faa0 0x7fea7c09ae70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fea7000ea50 tx=0x7fea700090f0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.365+0000 7fea81b3a640 1 -- 172.21.15.178:0/540099769 reap_dead start 2024-09-15T01:45:41.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.365+0000 7fea81b3a640 1 -- 172.21.15.178:0/540099769 reap_dead start 2024-09-15T01:45:41.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.365+0000 7fea8359d640 1 -- 172.21.15.178:0/540099769 shutdown_connections 2024-09-15T01:45:41.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.365+0000 7fea8359d640 1 -- 172.21.15.178:0/540099769 >> 172.21.15.178:0/540099769 conn(0x7fea7c0a5ed0 msgr2=0x7fea7c09e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:41.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.365+0000 7fea8359d640 1 -- 172.21.15.178:0/540099769 shutdown_connections 2024-09-15T01:45:41.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.365+0000 7fea8359d640 1 -- 172.21.15.178:0/540099769 wait complete. 2024-09-15T01:45:41.416 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:41.416 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T01:45:41.416 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T01:45:41.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.508+0000 7f58f3d1d640 10 monclient: get_monmap_and_config 2024-09-15T01:45:41.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.508+0000 7f58f3d1d640 10 monclient: build_initial_monmap 2024-09-15T01:45:41.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.508+0000 7f58f3d1d640 10 monclient: monmap: 2024-09-15T01:45:41.510 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:41.510 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.510 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:41.508935+0000 2024-09-15T01:45:41.510 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:41.508935+0000 2024-09-15T01:45:41.510 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:41.510 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.511 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:41.511 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:41.511 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:41.511 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:41.511 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:41.511 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:41.511 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:41.511 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:41.511 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:41.511 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:41.511 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:41.511 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.508+0000 7f58f3d1d640 1 Processor -- start 2024-09-15T01:45:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.508+0000 7f58f3d1d640 1 -- start start 2024-09-15T01:45:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.508+0000 7f58f3d1d640 10 monclient: init 2024-09-15T01:45:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 10 monclient: _add_conns ranks=[5,8,0,4,2,1,6,9,3,7,10] 2024-09-15T01:45:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58ec0678d0 0x7f58ec09e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 10 monclient(hunting): picked mon.noname-h con 0x7f58ec0678d0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 0x7f58ec09ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 10 monclient(hunting): picked mon.noname-e con 0x7f58ec06ea30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f58ec09f180 addr v1:172.21.15.178:6789/0 2024-09-15T01:45:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ec04bf60 con 0x7f58ec09f180 2024-09-15T01:45:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec09f590 con 0x7f58ec0678d0 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec06d5b0 con 0x7f58ec06ea30 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f3d1d640 10 monclient(hunting): authenticate will time out at 1546.259413s 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f22ba640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f58ec09f180 0x7f58ec1306b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:38454/0 (socket says 172.21.15.178:38454) 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f22ba640 1 -- 172.21.15.178:0/3010794852 learned_addr learned my addr 172.21.15.178:0/3010794852 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 0x7f58ec09ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 10 monclient(hunting): get_auth_request con 0x7f58ec06ea30 auth_method 0 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f1ab9640 1 --2- 172.21.15.178:0/3010794852 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58ec0678d0 0x7f58ec09e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f1ab9640 10 monclient(hunting): get_auth_request con 0x7f58ec0678d0 auth_method 0 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f1ab9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f1ab9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f1ab9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 10 monclient(hunting): handle_auth_done global_id 34991 payload 274 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f0ab7640 1 -- 172.21.15.178:0/3010794852 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803552975 0 0) 0x7f58ec04bf60 con 0x7f58ec09f180 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 1 -- 172.21.15.178:0/3010794852 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58ec0678d0 msgr2=0x7f58ec09e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 1 --2- 172.21.15.178:0/3010794852 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58ec0678d0 0x7f58ec09e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 1 -- 172.21.15.178:0/3010794852 >> v1:172.21.15.178:6789/0 conn(0x7f58ec09f180 legacy=0x7f58ec1306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 10 monclient: _finish_hunting 0 2024-09-15T01:45:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 1 monclient: found mon.noname-e 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f12b8640 1 -- 172.21.15.178:0/3010794852 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec067ca0 con 0x7f58ec06ea30 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.509+0000 7f58f0ab7640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f12b8640 1 --2- 172.21.15.178:0/3010794852 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 0x7f58ec09ec40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0098b0 tx=0x7f58dc0090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=948020947db7cc4b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f0ab7640 1 -- 172.21.15.178:0/3010794852 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f58dc02acc0 con 0x7f58ec06ea30 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f0ab7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f0ab7640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f0ab7640 10 monclient: dump: 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.522 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f0ab7640 10 monclient: _finish_auth 0 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f0ab7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f0ab7640 1 -- 172.21.15.178:0/3010794852 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc02ae40 con 0x7f58ec06ea30 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f3d1d640 5 monclient: authenticate success, global_id 34991 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f3d1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f0ab7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f3d1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f0ab7640 1 -- 172.21.15.178:0/3010794852 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f58dc031ca0 con 0x7f58ec06ea30 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f3d1d640 4 set_mon_vals no callback set 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f0ab7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f0ab7640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.510+0000 7f58f0ab7640 10 monclient: dump: 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.524 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.511+0000 7f58f3d1d640 10 monclient: shutdown 2024-09-15T01:45:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.511+0000 7f58f3d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.511+0000 7f58f3d1d640 1 -- 172.21.15.178:0/3010794852 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 msgr2=0x7f58ec09ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.511+0000 7f58f3d1d640 1 --2- 172.21.15.178:0/3010794852 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 0x7f58ec09ec40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0098b0 tx=0x7f58dc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.511+0000 7f58f3d1d640 1 -- 172.21.15.178:0/3010794852 shutdown_connections 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.511+0000 7f58f3d1d640 1 --2- 172.21.15.178:0/3010794852 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 0x7f58ec09ec40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.511+0000 7f58f3d1d640 1 --2- 172.21.15.178:0/3010794852 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f58ec0678d0 0x7f58ec09e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.511+0000 7f58f3d1d640 1 -- 172.21.15.178:0/3010794852 >> 172.21.15.178:0/3010794852 conn(0x7f58ec0a4390 msgr2=0x7f58ec067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.511+0000 7f58f3d1d640 1 -- 172.21.15.178:0/3010794852 shutdown_connections 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.511+0000 7f58f3d1d640 1 -- 172.21.15.178:0/3010794852 wait complete. 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.511+0000 7f58f3d1d640 10 monclient: build_initial_monmap 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.511+0000 7f58f3d1d640 10 monclient: monmap: 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:41.526 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:41.527 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 1 Processor -- start 2024-09-15T01:45:41.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 1 -- start start 2024-09-15T01:45:41.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 10 monclient: init 2024-09-15T01:45:41.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:41.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 10 monclient: _add_conns ranks=[1,8,4,5,6,7,2,3,0] 2024-09-15T01:45:41.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58ec0678d0 0x7f58ec133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 10 monclient(hunting): picked mon.noname-f con 0x7f58ec0678d0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:45:41.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 0x7f58ec11d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 10 monclient(hunting): picked mon.noname-e con 0x7f58ec06ea30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:41.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58ec09f180 0x7f58ec11d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 10 monclient(hunting): picked mon.noname-c con 0x7f58ec09f180 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:41.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec120b90 con 0x7f58ec0678d0 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec06c900 con 0x7f58ec09f180 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec06ccd0 con 0x7f58ec06ea30 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f3d1d640 10 monclient(hunting): authenticate will time out at 1546.262700s 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f12b8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 0x7f58ec11d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f12b8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 0x7f58ec11d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60154/0 (socket says 172.21.15.178:60154) 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f12b8640 1 -- 172.21.15.178:0/1401349386 learned_addr learned my addr 172.21.15.178:0/1401349386 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f12b8640 10 monclient(hunting): get_auth_request con 0x7f58ec06ea30 auth_method 0 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f12b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f12b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f12b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f22ba640 1 --2- 172.21.15.178:0/1401349386 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58ec09f180 0x7f58ec11d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f22ba640 10 monclient(hunting): get_auth_request con 0x7f58ec09f180 auth_method 0 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f22ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f22ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f22ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f1ab9640 1 --2- 172.21.15.178:0/1401349386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58ec0678d0 0x7f58ec133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f12b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f12b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f12b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f1ab9640 10 monclient(hunting): get_auth_request con 0x7f58ec0678d0 auth_method 0 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f1ab9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f1ab9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:41.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f1ab9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f22ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f22ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f22ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f1ab9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f1ab9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.512+0000 7f58f1ab9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f12b8640 10 monclient(hunting): handle_auth_done global_id 35000 payload 824 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f12b8640 1 -- 172.21.15.178:0/1401349386 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58ec09f180 msgr2=0x7f58ec11d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f12b8640 1 --2- 172.21.15.178:0/1401349386 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58ec09f180 0x7f58ec11d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f12b8640 1 -- 172.21.15.178:0/1401349386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58ec0678d0 msgr2=0x7f58ec133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f12b8640 1 --2- 172.21.15.178:0/1401349386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58ec0678d0 0x7f58ec133690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f12b8640 10 monclient: _finish_hunting 0 2024-09-15T01:45:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f12b8640 1 monclient: found mon.noname-e 2024-09-15T01:45:41.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f12b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:41.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f12b8640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:45:41.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f12b8640 1 -- 172.21.15.178:0/1401349386 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc009390 con 0x7f58ec06ea30 2024-09-15T01:45:41.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f12b8640 1 --2- 172.21.15.178:0/1401349386 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 0x7f58ec11d1d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f58dc035040 tx=0x7f58dc032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e1ffb640 1 -- 172.21.15.178:0/1401349386 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f58dc03b870 con 0x7f58ec06ea30 2024-09-15T01:45:41.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e1ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:45:41.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e1ffb640 10 monclient: dump: 2024-09-15T01:45:41.532 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.532 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.532 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.532 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e1ffb640 10 monclient: _finish_auth 0 2024-09-15T01:45:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e1ffb640 1 -- 172.21.15.178:0/1401349386 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc03b9f0 con 0x7f58ec06ea30 2024-09-15T01:45:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f3d1d640 5 monclient: authenticate success, global_id 35000 2024-09-15T01:45:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f3d1d640 10 monclient: _renew_subs 2024-09-15T01:45:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f3d1d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f3d1d640 1 -- 172.21.15.178:0/1401349386 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ec11ecf0 con 0x7f58ec06ea30 2024-09-15T01:45:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e1ffb640 1 -- 172.21.15.178:0/1401349386 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f58dc0313a0 con 0x7f58ec06ea30 2024-09-15T01:45:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e1ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e1ffb640 10 monclient: dump: 2024-09-15T01:45:41.534 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:41.535 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f3d1d640 10 monclient: _renew_subs 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58e3fff640 4 set_mon_vals no callback set 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f3d1d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.513+0000 7f58f3d1d640 1 -- 172.21.15.178:0/1401349386 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec1405d0 con 0x7f58ec06ea30 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.514+0000 7f58e1ffb640 1 -- 172.21.15.178:0/1401349386 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f58dc039040 con 0x7f58ec06ea30 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.514+0000 7f58f3d1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.514+0000 7f58f3d1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.514+0000 7f58f3d1d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.514+0000 7f58f3d1d640 1 -- 172.21.15.178:0/1401349386 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ec05ed70 con 0x7f58ec06ea30 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.514+0000 7f58e1ffb640 1 --2- 172.21.15.178:0/1401349386 >> v2:172.21.15.204:6812/28210 conn(0x7f58d00397e0 0x7f58d003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.514+0000 7f58e1ffb640 1 -- 172.21.15.178:0/1401349386 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f58dc06b590 con 0x7f58ec06ea30 2024-09-15T01:45:41.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.515+0000 7f58f1ab9640 1 --2- 172.21.15.178:0/1401349386 >> v2:172.21.15.204:6812/28210 conn(0x7f58d00397e0 0x7f58d003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:41.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.515+0000 7f58f1ab9640 10 monclient: get_auth_request con 0x7f58d00397e0 auth_method 0 2024-09-15T01:45:41.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.515+0000 7f58f1ab9640 1 --2- 172.21.15.178:0/1401349386 >> v2:172.21.15.204:6812/28210 conn(0x7f58d00397e0 0x7f58d003bc90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f58e8007ba0 tx=0x7f58e8008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:41.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.519+0000 7f58e1ffb640 1 -- 172.21.15.178:0/1401349386 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f58dc044340 con 0x7f58ec06ea30 2024-09-15T01:45:41.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.519+0000 7f58e1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:41.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.519+0000 7f58e1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:41.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.734+0000 7f58f3d1d640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T01:45:41.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.734+0000 7f58f3d1d640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T01:45:41.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.734+0000 7f58f3d1d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:41.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.734+0000 7f58f3d1d640 1 -- 172.21.15.178:0/1401349386 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f58ec04b720 con 0x7f58ec06ea30 2024-09-15T01:45:41.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.736+0000 7f58e1ffb640 1 -- 172.21.15.178:0/1401349386 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f58dc000ed0 con 0x7f58ec06ea30 2024-09-15T01:45:41.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.736+0000 7f58e1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T01:45:41.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.736+0000 7f58e1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:41.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=8,logm=59} crc {auth=2615088177,config=3140514009,health=928455196,kv=706962213,logm=978875013}) 2024-09-15T01:45:41.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=11,kv=7,logm=58} crc {auth=1175146154,config=3140514009,health=3364167054,kv=1446063039,logm=2769615608}) 2024-09-15T01:45:41.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=8,logm=59} crc {auth=2615088177,config=3140514009,health=928455196,kv=706962213,logm=978875013}) 2024-09-15T01:45:41.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=2414523263,config=3140514009,health=2972958799,kv=3497682465,logm=2721471053}) 2024-09-15T01:45:41.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=8,logm=59} crc {auth=2615088177,config=3140514009,health=928455196,kv=706962213,logm=978875013}) 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=21,config=2,health=12,kv=8,logm=57} crc {auth=4279136096,config=3140514009,health=2972958800,kv=706962213,logm=156014880}) 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=8,logm=59} crc {auth=2615088177,config=3140514009,health=928455196,kv=706962213,logm=978875013}) 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=21,config=2,health=11,kv=8,logm=58} crc {auth=2248504430,config=3140514009,health=3364167054,kv=706962213,logm=3769813989}) 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=8,logm=59} crc {auth=2615088177,config=3140514009,health=928455196,kv=706962213,logm=978875013}) 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=20,config=2,health=11,kv=8,logm=59} crc {auth=1366294697,config=3140514009,health=3364167054,kv=706962213,logm=117249665}) 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=8,logm=59} crc {auth=2615088177,config=3140514009,health=928455196,kv=706962213,logm=978875013}) 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=1175146154,config=3140514009,health=1171936690,kv=706962213,logm=2173064094}) 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=8,logm=59} crc {auth=2615088177,config=3140514009,health=928455196,kv=706962213,logm=978875013}) 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=22,config=1,health=12,kv=8,logm=57} crc {auth=3249055023,config=1228700967,health=2972958799,kv=706962213,logm=4028072593}) 2024-09-15T01:45:41.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=8,logm=59} crc {auth=2615088177,config=3140514009,health=928455196,kv=706962213,logm=978875013}) 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.737+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=7,logm=59} crc {auth=3806878971,config=3140514009,health=2972958799,kv=1446063039,logm=2259778521}) 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4138448609}) 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=763945543}) 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4138448609}) 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3688519796}) 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4138448609}) 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2604568633}) 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4138448609}) 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1276321901}) 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4138448609}) 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2177898044}) 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4138448609}) 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2999075956}) 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4138448609}) 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=608382034}) 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4138448609}) 2024-09-15T01:45:41.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.739+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1215435559}) 2024-09-15T01:45:41.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 1 -- 172.21.15.178:0/1401349386 >> v2:172.21.15.204:6812/28210 conn(0x7f58d00397e0 msgr2=0x7f58d003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 1 --2- 172.21.15.178:0/1401349386 >> v2:172.21.15.204:6812/28210 conn(0x7f58d00397e0 0x7f58d003bc90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f58e8007ba0 tx=0x7f58e8008040 comp rx=0 tx=0).stop 2024-09-15T01:45:41.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 10 monclient: shutdown 2024-09-15T01:45:41.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:41.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 1 -- 172.21.15.178:0/1401349386 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 msgr2=0x7f58ec11d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:41.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 1 --2- 172.21.15.178:0/1401349386 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 0x7f58ec11d1d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f58dc035040 tx=0x7f58dc032000 comp rx=0 tx=0).stop 2024-09-15T01:45:41.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 1 -- 172.21.15.178:0/1401349386 shutdown_connections 2024-09-15T01:45:41.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 1 --2- 172.21.15.178:0/1401349386 >> v2:172.21.15.204:6812/28210 conn(0x7f58d00397e0 0x7f58d003bc90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 1 --2- 172.21.15.178:0/1401349386 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f58ec09f180 0x7f58ec11d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 1 --2- 172.21.15.178:0/1401349386 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f58ec06ea30 0x7f58ec11d1d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 1 --2- 172.21.15.178:0/1401349386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58ec0678d0 0x7f58ec133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:41.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 1 -- 172.21.15.178:0/1401349386 >> 172.21.15.178:0/1401349386 conn(0x7f58ec0a4390 msgr2=0x7f58ec09e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:41.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.739+0000 7f58f3d1d640 1 -- 172.21.15.178:0/1401349386 shutdown_connections 2024-09-15T01:45:41.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:41.740+0000 7f58f3d1d640 1 -- 172.21.15.178:0/1401349386 wait complete. 2024-09-15T01:45:41.758 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T01:45:41.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2922503797}) 2024-09-15T01:45:41.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4227101349}) 2024-09-15T01:45:41.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2922503797}) 2024-09-15T01:45:41.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2289478561}) 2024-09-15T01:45:41.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2922503797}) 2024-09-15T01:45:41.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1955800990}) 2024-09-15T01:45:41.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2922503797}) 2024-09-15T01:45:41.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2705612219}) 2024-09-15T01:45:41.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2922503797}) 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1875974835}) 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2922503797}) 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1968324863}) 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2922503797}) 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2503610981}) 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2922503797}) 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.741+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3350943639}) 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.743+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.743+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3599574463}) 2024-09-15T01:45:41.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.743+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=666643418}) 2024-09-15T01:45:41.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3599574463}) 2024-09-15T01:45:41.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4267253796}) 2024-09-15T01:45:41.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3599574463}) 2024-09-15T01:45:41.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1408202550}) 2024-09-15T01:45:41.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3599574463}) 2024-09-15T01:45:41.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1470906399}) 2024-09-15T01:45:41.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3599574463}) 2024-09-15T01:45:41.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1588673476}) 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3599574463}) 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3038109974}) 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3599574463}) 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2629378821}) 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3599574463}) 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.744+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3090888123}) 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.745+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.745+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2671615360}) 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.745+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2809595778}) 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.745+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.745+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2671615360}) 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.745+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=148226013}) 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.745+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.745+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2671615360}) 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.745+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1914430448}) 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.745+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.745+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2671615360}) 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3236640989}) 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2671615360}) 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1949854259}) 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2671615360}) 2024-09-15T01:45:41.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1968687678}) 2024-09-15T01:45:41.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2671615360}) 2024-09-15T01:45:41.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=198516384}) 2024-09-15T01:45:41.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2671615360}) 2024-09-15T01:45:41.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.746+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=921055448}) 2024-09-15T01:45:41.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=542942334}) 2024-09-15T01:45:41.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2388643007}) 2024-09-15T01:45:41.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=542942334}) 2024-09-15T01:45:41.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3143680494}) 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=542942334}) 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3060944591}) 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=542942334}) 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2691352151}) 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=542942334}) 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=196615567}) 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=542942334}) 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4112056986}) 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=542942334}) 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4208289649}) 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=542942334}) 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.747+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1877216496}) 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026789982}) 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2674366503}) 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026789982}) 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1983200721}) 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026789982}) 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=542039540}) 2024-09-15T01:45:41.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026789982}) 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=187749948}) 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.748+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026789982}) 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.749+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2410080295}) 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.749+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.749+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026789982}) 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.749+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2793405992}) 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.749+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.749+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026789982}) 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.749+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2773731335}) 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.749+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.749+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026789982}) 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.749+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2399465196}) 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4058434863}) 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3491191156}) 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4058434863}) 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1143476661}) 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4058434863}) 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1957098731}) 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4058434863}) 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1305145752}) 2024-09-15T01:45:41.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4058434863}) 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2401024298}) 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4058434863}) 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3546989988}) 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4058434863}) 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2831513672}) 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4058434863}) 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.750+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2733537484}) 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1955159059}) 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3440521762}) 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1955159059}) 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2931448538}) 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1955159059}) 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=217049115}) 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1955159059}) 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2433943208}) 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1955159059}) 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=72741321}) 2024-09-15T01:45:41.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1955159059}) 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=435863242}) 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1955159059}) 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3366095314}) 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.751+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1955159059}) 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3900071037}) 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.752+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811056667}) 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=421751920}) 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811056667}) 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2033966840}) 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811056667}) 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2772240265}) 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811056667}) 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2898659165}) 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811056667}) 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2829229003}) 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811056667}) 2024-09-15T01:45:41.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1936303853}) 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811056667}) 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2133550313}) 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811056667}) 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.753+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3217259480}) 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3619150468}) 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1381708424}) 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3619150468}) 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1753091788}) 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3619150468}) 2024-09-15T01:45:41.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1348868578}) 2024-09-15T01:45:41.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3619150468}) 2024-09-15T01:45:41.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=817811256}) 2024-09-15T01:45:41.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3619150468}) 2024-09-15T01:45:41.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1951879214}) 2024-09-15T01:45:41.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3619150468}) 2024-09-15T01:45:41.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=773234737}) 2024-09-15T01:45:41.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3619150468}) 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4042214664}) 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3619150468}) 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.754+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1796863140}) 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953640955}) 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3161089283}) 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953640955}) 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2688369081}) 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953640955}) 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=775057542}) 2024-09-15T01:45:41.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953640955}) 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2684439246}) 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953640955}) 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1114056512}) 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953640955}) 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2714176526}) 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953640955}) 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=705888377}) 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953640955}) 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.755+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3444372585}) 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1817180134}) 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1344745132}) 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1817180134}) 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1380999003}) 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1817180134}) 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3474437700}) 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1817180134}) 2024-09-15T01:45:41.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3968839718}) 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1817180134}) 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=333331216}) 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1817180134}) 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.756+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=546583305}) 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1817180134}) 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=949428600}) 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1817180134}) 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.757+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2326663024}) 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3214933223}) 2024-09-15T01:45:41.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2144808383}) 2024-09-15T01:45:41.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3214933223}) 2024-09-15T01:45:41.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3475953553}) 2024-09-15T01:45:41.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3214933223}) 2024-09-15T01:45:41.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1702865475}) 2024-09-15T01:45:41.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3214933223}) 2024-09-15T01:45:41.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=681094602}) 2024-09-15T01:45:41.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3214933223}) 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1481922117}) 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3214933223}) 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4251498261}) 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3214933223}) 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3139350736}) 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3214933223}) 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.758+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=886031548}) 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3948007220}) 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1154264402}) 2024-09-15T01:45:41.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3948007220}) 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3609794741}) 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.759+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3948007220}) 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1850273360}) 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3948007220}) 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2581779524}) 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3948007220}) 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=64432454}) 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3948007220}) 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2825508195}) 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3948007220}) 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3814968504}) 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3948007220}) 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.760+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1129711214}) 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2193960192}) 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3133852746}) 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2193960192}) 2024-09-15T01:45:41.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2729327250}) 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2193960192}) 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3627788849}) 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2193960192}) 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3001888052}) 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2193960192}) 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4289685815}) 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2193960192}) 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=741536078}) 2024-09-15T01:45:41.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2193960192}) 2024-09-15T01:45:41.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3851868587}) 2024-09-15T01:45:41.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2193960192}) 2024-09-15T01:45:41.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.761+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1841913502}) 2024-09-15T01:45:41.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3092014093}) 2024-09-15T01:45:41.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=442667405}) 2024-09-15T01:45:41.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3092014093}) 2024-09-15T01:45:41.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1106183549}) 2024-09-15T01:45:41.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.762+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3092014093}) 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3501787975}) 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3092014093}) 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1890228207}) 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3092014093}) 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1351103563}) 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3092014093}) 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=903791726}) 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3092014093}) 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4244585717}) 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3092014093}) 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.763+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4009208207}) 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2082141597}) 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3357891238}) 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2082141597}) 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2156137687}) 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2082141597}) 2024-09-15T01:45:41.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=280444974}) 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2082141597}) 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2133643697}) 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2082141597}) 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4217835416}) 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2082141597}) 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1179570714}) 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2082141597}) 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1076851428}) 2024-09-15T01:45:41.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2082141597}) 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.764+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2677789691}) 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=567236285}) 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1368886351}) 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=567236285}) 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1305323729}) 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=567236285}) 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=811368298}) 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.765+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=567236285}) 2024-09-15T01:45:41.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2418506750}) 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=567236285}) 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2745340630}) 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=567236285}) 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=86078279}) 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=567236285}) 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2497270143}) 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=567236285}) 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.766+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1099739065}) 2024-09-15T01:45:41.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=264289590}) 2024-09-15T01:45:41.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=710043597}) 2024-09-15T01:45:41.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=264289590}) 2024-09-15T01:45:41.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1750234454}) 2024-09-15T01:45:41.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=264289590}) 2024-09-15T01:45:41.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=898462030}) 2024-09-15T01:45:41.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=264289590}) 2024-09-15T01:45:41.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=738994678}) 2024-09-15T01:45:41.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=264289590}) 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3283525905}) 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=264289590}) 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3434688468}) 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=264289590}) 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=795109323}) 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=264289590}) 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.767+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1671716311}) 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3020745744}) 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2740545572}) 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3020745744}) 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=975296856}) 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3020745744}) 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.768+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1161516722}) 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3020745744}) 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1566933282}) 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3020745744}) 2024-09-15T01:45:41.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=684542172}) 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3020745744}) 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=564978718}) 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3020745744}) 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4179120780}) 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3020745744}) 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.769+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3368574070}) 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3753076174}) 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1720021931}) 2024-09-15T01:45:41.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3753076174}) 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3727735987}) 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3753076174}) 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1146455196}) 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3753076174}) 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3904507617}) 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3753076174}) 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3561859701}) 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3753076174}) 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3871279788}) 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3753076174}) 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1924666259}) 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3753076174}) 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.770+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=389456550}) 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=925030340}) 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2098728095}) 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=925030340}) 2024-09-15T01:45:41.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.771+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3901131178}) 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=925030340}) 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3804896225}) 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=925030340}) 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=252478532}) 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=925030340}) 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3873497865}) 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=925030340}) 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3284806274}) 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=925030340}) 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2265612017}) 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=925030340}) 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.772+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=259955793}) 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4177806628}) 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=663100844}) 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4177806628}) 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3912568545}) 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4177806628}) 2024-09-15T01:45:41.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2509735343}) 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4177806628}) 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2098479653}) 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4177806628}) 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=956343786}) 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4177806628}) 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1076290365}) 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4177806628}) 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1871763596}) 2024-09-15T01:45:41.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4177806628}) 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.773+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2418693756}) 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2179450039}) 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4198551328}) 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2179450039}) 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2333913274}) 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2179450039}) 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2904663418}) 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2179450039}) 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2342106481}) 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2179450039}) 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3868475567}) 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2179450039}) 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=433154389}) 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2179450039}) 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=652095027}) 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2179450039}) 2024-09-15T01:45:41.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.774+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=338311551}) 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=588528157}) 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4170809046}) 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=588528157}) 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1578265964}) 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.775+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=588528157}) 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2888931942}) 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=588528157}) 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2595577594}) 2024-09-15T01:45:41.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=588528157}) 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2078824012}) 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=588528157}) 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=477543566}) 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=588528157}) 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2885404546}) 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=588528157}) 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.776+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1643846969}) 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4099705768}) 2024-09-15T01:45:41.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1259543161}) 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4099705768}) 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2101371922}) 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4099705768}) 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2993560849}) 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4099705768}) 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4275446609}) 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4099705768}) 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=830994071}) 2024-09-15T01:45:41.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4099705768}) 2024-09-15T01:45:41.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3010256639}) 2024-09-15T01:45:41.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4099705768}) 2024-09-15T01:45:41.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1566069255}) 2024-09-15T01:45:41.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4099705768}) 2024-09-15T01:45:41.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.777+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1251518370}) 2024-09-15T01:45:41.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3922571978}) 2024-09-15T01:45:41.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3295697075}) 2024-09-15T01:45:41.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3922571978}) 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1369242161}) 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3922571978}) 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1548648985}) 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3922571978}) 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=398662439}) 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3922571978}) 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3327904357}) 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.778+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3922571978}) 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=419690484}) 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3922571978}) 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2626785671}) 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3922571978}) 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.779+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3783184646}) 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=473228190}) 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3020265047}) 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=473228190}) 2024-09-15T01:45:41.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=412095680}) 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=473228190}) 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=676140001}) 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=473228190}) 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1639926344}) 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=473228190}) 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=5395839}) 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=473228190}) 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4211891806}) 2024-09-15T01:45:41.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=473228190}) 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2885915101}) 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=473228190}) 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.780+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2931827185}) 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1532641287}) 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=688146261}) 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1532641287}) 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1540218931}) 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1532641287}) 2024-09-15T01:45:41.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1079405805}) 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1532641287}) 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2058075301}) 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1532641287}) 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=491107427}) 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1532641287}) 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1114785466}) 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1532641287}) 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=937652392}) 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1532641287}) 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.783+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2620274320}) 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3929510702}) 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1215025218}) 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3929510702}) 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2083734587}) 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3929510702}) 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1680617619}) 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3929510702}) 2024-09-15T01:45:41.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2208909732}) 2024-09-15T01:45:41.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3929510702}) 2024-09-15T01:45:41.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1819022210}) 2024-09-15T01:45:41.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3929510702}) 2024-09-15T01:45:41.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1732684043}) 2024-09-15T01:45:41.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3929510702}) 2024-09-15T01:45:41.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=788163484}) 2024-09-15T01:45:41.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3929510702}) 2024-09-15T01:45:41.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.785+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3058198334}) 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=972117370}) 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1392599705}) 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=972117370}) 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1336258684}) 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=972117370}) 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2930921581}) 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=972117370}) 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3614751071}) 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=972117370}) 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3258202909}) 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=972117370}) 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2761412278}) 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=972117370}) 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2657614438}) 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=972117370}) 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.795+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2342736383}) 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=368179823}) 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3743559404}) 2024-09-15T01:45:41.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=368179823}) 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178340667}) 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=368179823}) 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1662940760}) 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=368179823}) 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3784496510}) 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=368179823}) 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4170849520}) 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:41.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=368179823}) 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1370585491}) 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=368179823}) 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3493751906}) 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=368179823}) 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.796+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=922817792}) 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.797+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.797+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1509468938}) 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.797+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3291795391}) 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.797+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.797+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1509468938}) 2024-09-15T01:45:42.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.797+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3372635394}) 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.797+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.797+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1509468938}) 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.797+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3681491986}) 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1509468938}) 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1297100137}) 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1509468938}) 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1499161677}) 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1509468938}) 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2437883658}) 2024-09-15T01:45:42.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1509468938}) 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1043186442}) 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1509468938}) 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.798+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2741188636}) 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3196716854}) 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311456237}) 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3196716854}) 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3529652320}) 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3196716854}) 2024-09-15T01:45:42.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1572756867}) 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3196716854}) 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3232911574}) 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3196716854}) 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=545875731}) 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3196716854}) 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2128046508}) 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3196716854}) 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2560936757}) 2024-09-15T01:45:42.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3196716854}) 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.799+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=367205300}) 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1620697585}) 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1845053707}) 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1620697585}) 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1132487711}) 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1620697585}) 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=219043730}) 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1620697585}) 2024-09-15T01:45:42.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2947589780}) 2024-09-15T01:45:42.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1620697585}) 2024-09-15T01:45:42.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3267789859}) 2024-09-15T01:45:42.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1620697585}) 2024-09-15T01:45:42.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3706892975}) 2024-09-15T01:45:42.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.800+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1620697585}) 2024-09-15T01:45:42.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.801+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=181553684}) 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.801+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.801+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1620697585}) 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.801+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=325728129}) 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2106551673}) 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1230454896}) 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2106551673}) 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1733631917}) 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2106551673}) 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=435559313}) 2024-09-15T01:45:42.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2106551673}) 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2998966174}) 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2106551673}) 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3760517857}) 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2106551673}) 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=427362379}) 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2106551673}) 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1618357957}) 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2106551673}) 2024-09-15T01:45:42.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.802+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3904530320}) 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3653526161}) 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=334473226}) 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3653526161}) 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2850050561}) 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3653526161}) 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4013869793}) 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3653526161}) 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2648826577}) 2024-09-15T01:45:42.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3653526161}) 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1323325464}) 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3653526161}) 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1512696208}) 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3653526161}) 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=109826527}) 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3653526161}) 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.803+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1472103813}) 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1429688380}) 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3455172783}) 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1429688380}) 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1347867317}) 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1429688380}) 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3615725413}) 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1429688380}) 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1616528067}) 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1429688380}) 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=614901654}) 2024-09-15T01:45:42.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1429688380}) 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=776792712}) 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1429688380}) 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.804+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2708245185}) 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.805+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.805+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1429688380}) 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.805+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2397258424}) 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.805+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=794833219}) 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1884044407}) 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=794833219}) 2024-09-15T01:45:42.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=847584047}) 2024-09-15T01:45:42.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=794833219}) 2024-09-15T01:45:42.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1755462284}) 2024-09-15T01:45:42.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=794833219}) 2024-09-15T01:45:42.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1971569629}) 2024-09-15T01:45:42.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=794833219}) 2024-09-15T01:45:42.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2936254755}) 2024-09-15T01:45:42.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=794833219}) 2024-09-15T01:45:42.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=92501991}) 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=794833219}) 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1897362230}) 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=794833219}) 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.806+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1895524177}) 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1082633948}) 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1850014855}) 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1082633948}) 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=503532451}) 2024-09-15T01:45:42.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1082633948}) 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=69019439}) 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1082633948}) 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1375464767}) 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1082633948}) 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2721108111}) 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1082633948}) 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1373110188}) 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1082633948}) 2024-09-15T01:45:42.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2500870952}) 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1082633948}) 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.807+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2645673586}) 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2342499020}) 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3025529499}) 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2342499020}) 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4155375388}) 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2342499020}) 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3133667680}) 2024-09-15T01:45:42.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2342499020}) 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=300546079}) 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2342499020}) 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=118508586}) 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2342499020}) 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=27459414}) 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.808+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.809+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2342499020}) 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.809+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2051971705}) 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.809+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.809+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2342499020}) 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.809+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=551904447}) 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=458703964}) 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3669692011}) 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=458703964}) 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=879715963}) 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=458703964}) 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2570226466}) 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=458703964}) 2024-09-15T01:45:42.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3752053552}) 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=458703964}) 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3459164760}) 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=458703964}) 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=190226275}) 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=458703964}) 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3315336103}) 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=458703964}) 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.810+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1564456758}) 2024-09-15T01:45:42.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1472493428}) 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1243851227}) 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1472493428}) 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2955225038}) 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1472493428}) 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3942242660}) 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1472493428}) 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2959484291}) 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1472493428}) 2024-09-15T01:45:42.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3865112421}) 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1472493428}) 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3593581867}) 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1472493428}) 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3472076387}) 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1472493428}) 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.811+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=867171134}) 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4087522034}) 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1679805864}) 2024-09-15T01:45:42.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4087522034}) 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3426373469}) 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4087522034}) 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2660196615}) 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4087522034}) 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4191096545}) 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4087522034}) 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1849740707}) 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.812+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4087522034}) 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.813+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3150422789}) 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.813+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.813+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4087522034}) 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.813+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2138696979}) 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.813+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.813+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4087522034}) 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.813+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3271436424}) 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1699298316}) 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2620141832}) 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1699298316}) 2024-09-15T01:45:42.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=757859855}) 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1699298316}) 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3522525300}) 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1699298316}) 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=743006071}) 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1699298316}) 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3896481882}) 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1699298316}) 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1979905254}) 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1699298316}) 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1611568226}) 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1699298316}) 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.814+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3743132086}) 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=629277695}) 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495548322}) 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=629277695}) 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3113999134}) 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=629277695}) 2024-09-15T01:45:42.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3407909571}) 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=629277695}) 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=246356164}) 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=629277695}) 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2544700525}) 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=629277695}) 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4185102675}) 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=629277695}) 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2137873295}) 2024-09-15T01:45:42.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=629277695}) 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.815+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=218330852}) 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3212805704}) 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=797478584}) 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3212805704}) 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3060496975}) 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3212805704}) 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2643521764}) 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3212805704}) 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=504991513}) 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3212805704}) 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.816+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1534239963}) 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.817+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.817+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3212805704}) 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.817+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2315998100}) 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.817+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.817+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3212805704}) 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.817+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1350145461}) 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.817+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.817+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3212805704}) 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.817+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=362138169}) 2024-09-15T01:45:42.027 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1865081235}) 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4062687551}) 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1865081235}) 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1317587282}) 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1865081235}) 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=688558510}) 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1865081235}) 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=516327546}) 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.028 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1865081235}) 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3665264652}) 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1865081235}) 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=753649431}) 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1865081235}) 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2493708564}) 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1865081235}) 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.818+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=123489940}) 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898503714}) 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1851089610}) 2024-09-15T01:45:42.029 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898503714}) 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=915057044}) 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898503714}) 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3119992873}) 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898503714}) 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3619133059}) 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898503714}) 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2101304356}) 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.030 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898503714}) 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3953433608}) 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898503714}) 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=392649752}) 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.819+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.820+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898503714}) 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.820+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=393565692}) 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2544873728}) 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2838598405}) 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2544873728}) 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=773496477}) 2024-09-15T01:45:42.031 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2544873728}) 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1863917485}) 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2544873728}) 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2698462073}) 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2544873728}) 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1505333210}) 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2544873728}) 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=42733683}) 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.032 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2544873728}) 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2720633581}) 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2544873728}) 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.821+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1716933351}) 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2255502018}) 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3344538594}) 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2255502018}) 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4036418982}) 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2255502018}) 2024-09-15T01:45:42.033 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2757054215}) 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2255502018}) 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1313233857}) 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2255502018}) 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=131906160}) 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2255502018}) 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2993246291}) 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2255502018}) 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1167029564}) 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.034 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2255502018}) 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.822+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2162466545}) 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=950838461}) 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2039226994}) 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=950838461}) 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1010328671}) 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=950838461}) 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3206561842}) 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=950838461}) 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4285472449}) 2024-09-15T01:45:42.035 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.036 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=950838461}) 2024-09-15T01:45:42.036 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2106678748}) 2024-09-15T01:45:42.036 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.036 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=950838461}) 2024-09-15T01:45:42.036 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1920691667}) 2024-09-15T01:45:42.036 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.036 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=950838461}) 2024-09-15T01:45:42.036 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1226505122}) 2024-09-15T01:45:42.036 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.036 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=950838461}) 2024-09-15T01:45:42.037 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.823+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4082864831}) 2024-09-15T01:45:42.037 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.037 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2107581906}) 2024-09-15T01:45:42.037 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1980156916}) 2024-09-15T01:45:42.037 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.037 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2107581906}) 2024-09-15T01:45:42.037 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=377456620}) 2024-09-15T01:45:42.037 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.037 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2107581906}) 2024-09-15T01:45:42.037 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3849799581}) 2024-09-15T01:45:42.037 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.037 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2107581906}) 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1453764872}) 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.824+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2107581906}) 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.825+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2203872937}) 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.825+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.825+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2107581906}) 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.825+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3718013589}) 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.825+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.825+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2107581906}) 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.825+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1713354032}) 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.825+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.825+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2107581906}) 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.825+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2292808501}) 2024-09-15T01:45:42.038 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3876909738}) 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=987410593}) 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3876909738}) 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2258562122}) 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3876909738}) 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3836862759}) 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3876909738}) 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=777546860}) 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.039 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3876909738}) 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=484357683}) 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3876909738}) 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4151171796}) 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3876909738}) 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3829696298}) 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3876909738}) 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.826+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=152034813}) 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1642598}) 2024-09-15T01:45:42.040 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3083207397}) 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1642598}) 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2187744855}) 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1642598}) 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=62038913}) 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1642598}) 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3614904617}) 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1642598}) 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=742235889}) 2024-09-15T01:45:42.041 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1642598}) 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2173228565}) 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1642598}) 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2073082243}) 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1642598}) 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.827+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1203224994}) 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3443321363}) 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=882654145}) 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.042 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3443321363}) 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1380518602}) 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3443321363}) 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=528590189}) 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3443321363}) 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3941557057}) 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.828+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3443321363}) 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.829+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2326492842}) 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.829+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.829+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3443321363}) 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.829+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3375006111}) 2024-09-15T01:45:42.043 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.829+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.829+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3443321363}) 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.829+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1731926067}) 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.829+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.829+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3443321363}) 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.829+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3230049392}) 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3421407159}) 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3734004295}) 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3421407159}) 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=964365424}) 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.044 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3421407159}) 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4080727874}) 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3421407159}) 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1091304561}) 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3421407159}) 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=691965922}) 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3421407159}) 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1332770440}) 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3421407159}) 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=43719572}) 2024-09-15T01:45:42.045 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3421407159}) 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.830+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1078351994}) 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=888077341}) 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3412495246}) 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=888077341}) 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2150497157}) 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=888077341}) 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3595977493}) 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=888077341}) 2024-09-15T01:45:42.046 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2486566472}) 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=888077341}) 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1700184751}) 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=888077341}) 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2068518634}) 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=888077341}) 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3040740793}) 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=888077341}) 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.831+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=461290724}) 2024-09-15T01:45:42.047 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988986653}) 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=479453265}) 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988986653}) 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2360630231}) 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988986653}) 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=308842566}) 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988986653}) 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3248619272}) 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.048 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988986653}) 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1993032691}) 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988986653}) 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1931606853}) 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.832+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988986653}) 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.833+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1909716441}) 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.833+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.833+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988986653}) 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.833+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3266057731}) 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4165368100}) 2024-09-15T01:45:42.049 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2256458378}) 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4165368100}) 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1116329647}) 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4165368100}) 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3255342764}) 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4165368100}) 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3249783833}) 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4165368100}) 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=696662192}) 2024-09-15T01:45:42.050 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4165368100}) 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3456049591}) 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4165368100}) 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=943494263}) 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4165368100}) 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.834+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=276480225}) 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3491566656}) 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=227518579}) 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.051 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3491566656}) 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=198582359}) 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3491566656}) 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2820845467}) 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3491566656}) 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4189259238}) 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3491566656}) 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3465467960}) 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3491566656}) 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=511822686}) 2024-09-15T01:45:42.052 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3491566656}) 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4117839946}) 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3491566656}) 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.835+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4164153142}) 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240855656}) 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3318870778}) 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240855656}) 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2103890364}) 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.053 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240855656}) 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=380902676}) 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240855656}) 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3058814847}) 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240855656}) 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2604928062}) 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.836+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240855656}) 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.837+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2889581518}) 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.837+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.837+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240855656}) 2024-09-15T01:45:42.054 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.837+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3879717175}) 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.837+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.837+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240855656}) 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.837+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=320713800}) 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1216373628}) 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046351341}) 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1216373628}) 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3223914848}) 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1216373628}) 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=708018630}) 2024-09-15T01:45:42.055 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.056 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1216373628}) 2024-09-15T01:45:42.056 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1097887327}) 2024-09-15T01:45:42.056 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.056 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1216373628}) 2024-09-15T01:45:42.056 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2466632318}) 2024-09-15T01:45:42.056 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.056 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1216373628}) 2024-09-15T01:45:42.056 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=288243502}) 2024-09-15T01:45:42.056 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.056 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1216373628}) 2024-09-15T01:45:42.056 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2634108972}) 2024-09-15T01:45:42.056 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1216373628}) 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.838+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3652774903}) 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=491148971}) 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3476449812}) 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=491148971}) 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2407537003}) 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=491148971}) 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1663230066}) 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.057 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=491148971}) 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1234386938}) 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=491148971}) 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=918223072}) 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=491148971}) 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2735646992}) 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=491148971}) 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1098719715}) 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=491148971}) 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.839+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=437795854}) 2024-09-15T01:45:42.058 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.840+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.840+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218995604}) 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.840+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=999441421}) 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218995604}) 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1320829778}) 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218995604}) 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3982333983}) 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218995604}) 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3739716830}) 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218995604}) 2024-09-15T01:45:42.059 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1924691102}) 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218995604}) 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2387400940}) 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218995604}) 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3049622471}) 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218995604}) 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.841+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1960421934}) 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3833150824}) 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3359978308}) 2024-09-15T01:45:42.060 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3833150824}) 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1304079493}) 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3833150824}) 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1296902222}) 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3833150824}) 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2042647375}) 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3833150824}) 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2569839684}) 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.061 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3833150824}) 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3594606188}) 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3833150824}) 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1225691356}) 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3833150824}) 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.842+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1074137110}) 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2307762906}) 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1814427832}) 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2307762906}) 2024-09-15T01:45:42.062 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=125987164}) 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2307762906}) 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1703769969}) 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2307762906}) 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2642249615}) 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2307762906}) 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3140822963}) 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2307762906}) 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2674632290}) 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.063 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2307762906}) 2024-09-15T01:45:42.064 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3125608366}) 2024-09-15T01:45:42.064 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.064 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2307762906}) 2024-09-15T01:45:42.064 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.844+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4139786334}) 2024-09-15T01:45:42.064 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.064 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1510098170}) 2024-09-15T01:45:42.064 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1900495264}) 2024-09-15T01:45:42.064 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.064 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1510098170}) 2024-09-15T01:45:42.064 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3323314343}) 2024-09-15T01:45:42.064 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.064 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1510098170}) 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1579285530}) 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1510098170}) 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=910656214}) 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1510098170}) 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=84393925}) 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1510098170}) 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2611017102}) 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1510098170}) 2024-09-15T01:45:42.065 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3463584058}) 2024-09-15T01:45:42.066 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.066 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1510098170}) 2024-09-15T01:45:42.066 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.845+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4112398219}) 2024-09-15T01:45:42.066 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.066 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3685699191}) 2024-09-15T01:45:42.066 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2904050580}) 2024-09-15T01:45:42.066 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.066 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3685699191}) 2024-09-15T01:45:42.066 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2965327396}) 2024-09-15T01:45:42.066 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.066 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3685699191}) 2024-09-15T01:45:42.066 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3398946694}) 2024-09-15T01:45:42.067 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.067 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3685699191}) 2024-09-15T01:45:42.067 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4207223520}) 2024-09-15T01:45:42.067 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.067 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3685699191}) 2024-09-15T01:45:42.067 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2285405654}) 2024-09-15T01:45:42.067 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.067 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.846+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3685699191}) 2024-09-15T01:45:42.067 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.847+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=513656273}) 2024-09-15T01:45:42.067 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.847+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.067 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.847+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3685699191}) 2024-09-15T01:45:42.067 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.847+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3059149694}) 2024-09-15T01:45:42.068 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.847+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.068 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.847+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3685699191}) 2024-09-15T01:45:42.068 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.847+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=895234348}) 2024-09-15T01:45:42.068 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.068 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1922027423}) 2024-09-15T01:45:42.068 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=701682994}) 2024-09-15T01:45:42.068 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.068 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1922027423}) 2024-09-15T01:45:42.068 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1926735104}) 2024-09-15T01:45:42.068 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.068 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1922027423}) 2024-09-15T01:45:42.068 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3858687630}) 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1922027423}) 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1595698886}) 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1922027423}) 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2763360602}) 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1922027423}) 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3130051624}) 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1922027423}) 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3530457495}) 2024-09-15T01:45:42.069 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1922027423}) 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.848+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1981995541}) 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294646292}) 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=456149056}) 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294646292}) 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3158390653}) 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294646292}) 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2423911592}) 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.070 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294646292}) 2024-09-15T01:45:42.071 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1954154086}) 2024-09-15T01:45:42.071 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.071 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294646292}) 2024-09-15T01:45:42.071 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1019111269}) 2024-09-15T01:45:42.071 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.071 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294646292}) 2024-09-15T01:45:42.071 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=123630246}) 2024-09-15T01:45:42.071 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.071 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294646292}) 2024-09-15T01:45:42.071 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=860926321}) 2024-09-15T01:45:42.071 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.072 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294646292}) 2024-09-15T01:45:42.072 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.849+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=124240303}) 2024-09-15T01:45:42.072 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.072 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1044639218}) 2024-09-15T01:45:42.072 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2073459835}) 2024-09-15T01:45:42.072 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.072 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1044639218}) 2024-09-15T01:45:42.072 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3095060976}) 2024-09-15T01:45:42.072 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.072 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1044639218}) 2024-09-15T01:45:42.072 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1745393548}) 2024-09-15T01:45:42.073 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.073 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1044639218}) 2024-09-15T01:45:42.073 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3825282087}) 2024-09-15T01:45:42.073 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.073 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1044639218}) 2024-09-15T01:45:42.073 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1481788624}) 2024-09-15T01:45:42.073 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.073 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.850+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1044639218}) 2024-09-15T01:45:42.073 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.851+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=46220602}) 2024-09-15T01:45:42.073 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.851+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.073 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.851+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1044639218}) 2024-09-15T01:45:42.074 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.851+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2478003917}) 2024-09-15T01:45:42.074 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.851+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.074 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.851+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1044639218}) 2024-09-15T01:45:42.074 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.851+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3081988432}) 2024-09-15T01:45:42.074 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.074 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1607875970}) 2024-09-15T01:45:42.074 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1670450409}) 2024-09-15T01:45:42.074 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.074 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1607875970}) 2024-09-15T01:45:42.074 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=571546138}) 2024-09-15T01:45:42.074 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.074 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1607875970}) 2024-09-15T01:45:42.075 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3128979733}) 2024-09-15T01:45:42.075 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.075 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1607875970}) 2024-09-15T01:45:42.075 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3029548285}) 2024-09-15T01:45:42.075 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.075 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1607875970}) 2024-09-15T01:45:42.075 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2545232454}) 2024-09-15T01:45:42.075 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.075 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1607875970}) 2024-09-15T01:45:42.075 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2259632054}) 2024-09-15T01:45:42.075 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.075 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1607875970}) 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2080475190}) 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1607875970}) 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.852+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1948261114}) 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144068208}) 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3864648656}) 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144068208}) 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2319981362}) 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144068208}) 2024-09-15T01:45:42.076 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4020711217}) 2024-09-15T01:45:42.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144068208}) 2024-09-15T01:45:42.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3199786827}) 2024-09-15T01:45:42.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144068208}) 2024-09-15T01:45:42.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2735018513}) 2024-09-15T01:45:42.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144068208}) 2024-09-15T01:45:42.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1512992548}) 2024-09-15T01:45:42.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.077 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144068208}) 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1542844117}) 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4144068208}) 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.853+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3752913086}) 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.854+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.854+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1791946559}) 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.854+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2987266838}) 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.854+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.854+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1791946559}) 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.854+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2812601413}) 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.854+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.854+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1791946559}) 2024-09-15T01:45:42.078 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.854+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1217990913}) 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1791946559}) 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=762515960}) 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1791946559}) 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2155079560}) 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1791946559}) 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1049708575}) 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1791946559}) 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1857805813}) 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.079 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1791946559}) 2024-09-15T01:45:42.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.855+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=74046504}) 2024-09-15T01:45:42.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609741665}) 2024-09-15T01:45:42.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3130626858}) 2024-09-15T01:45:42.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609741665}) 2024-09-15T01:45:42.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1423710085}) 2024-09-15T01:45:42.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609741665}) 2024-09-15T01:45:42.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3524151826}) 2024-09-15T01:45:42.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.080 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609741665}) 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2093713832}) 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609741665}) 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3901092050}) 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609741665}) 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=238231655}) 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609741665}) 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2793126443}) 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609741665}) 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.856+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4228151022}) 2024-09-15T01:45:42.081 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1533113441}) 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4291708488}) 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1533113441}) 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2986375470}) 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1533113441}) 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1334678780}) 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1533113441}) 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2722102902}) 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.082 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1533113441}) 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1037145482}) 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1533113441}) 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3500466324}) 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1533113441}) 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4011443649}) 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1533113441}) 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.857+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1070473789}) 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2038998003}) 2024-09-15T01:45:42.083 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=277156287}) 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2038998003}) 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3716244310}) 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2038998003}) 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3140377691}) 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2038998003}) 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1242957129}) 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2038998003}) 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3416006256}) 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.084 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2038998003}) 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1780094077}) 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2038998003}) 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3491108776}) 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2038998003}) 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.859+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1111032372}) 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=27689202}) 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=431546967}) 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=27689202}) 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1353963961}) 2024-09-15T01:45:42.085 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=27689202}) 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3387018320}) 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=27689202}) 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3735864149}) 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=27689202}) 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2136803363}) 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=27689202}) 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=63882708}) 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.086 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=27689202}) 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4159986579}) 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=27689202}) 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.860+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=481588019}) 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2388864029}) 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3165965457}) 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2388864029}) 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1937574754}) 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2388864029}) 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4181049649}) 2024-09-15T01:45:42.087 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2388864029}) 2024-09-15T01:45:42.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3162927556}) 2024-09-15T01:45:42.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2388864029}) 2024-09-15T01:45:42.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3290139543}) 2024-09-15T01:45:42.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2388864029}) 2024-09-15T01:45:42.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3719009179}) 2024-09-15T01:45:42.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2388864029}) 2024-09-15T01:45:42.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.861+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4021945338}) 2024-09-15T01:45:42.088 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.862+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.862+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2388864029}) 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.862+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1804212114}) 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2276233164}) 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1346951512}) 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2276233164}) 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2538951984}) 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2276233164}) 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1777418586}) 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2276233164}) 2024-09-15T01:45:42.089 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2838037902}) 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2276233164}) 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2957362298}) 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2276233164}) 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3099812981}) 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2276233164}) 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1367435032}) 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2276233164}) 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.863+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2411550815}) 2024-09-15T01:45:42.090 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=299452975}) 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4071184078}) 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=299452975}) 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1573784548}) 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=299452975}) 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2320796328}) 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=299452975}) 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1529629483}) 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=299452975}) 2024-09-15T01:45:42.091 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1260073815}) 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=299452975}) 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=438161012}) 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=299452975}) 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3290412463}) 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=299452975}) 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.864+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3846910956}) 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2453081196}) 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3121606357}) 2024-09-15T01:45:42.092 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2453081196}) 2024-09-15T01:45:42.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3828292106}) 2024-09-15T01:45:42.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2453081196}) 2024-09-15T01:45:42.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1013671862}) 2024-09-15T01:45:42.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2453081196}) 2024-09-15T01:45:42.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=329590279}) 2024-09-15T01:45:42.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2453081196}) 2024-09-15T01:45:42.093 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2329241196}) 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2453081196}) 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4120006921}) 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2453081196}) 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1760227124}) 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2453081196}) 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.865+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4270313494}) 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2891623878}) 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2436797994}) 2024-09-15T01:45:42.094 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2891623878}) 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=61182666}) 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2891623878}) 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855946734}) 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2891623878}) 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2026441643}) 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2891623878}) 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.866+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1474460808}) 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.867+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.095 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.867+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2891623878}) 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.867+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1971419988}) 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.867+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.867+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2891623878}) 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.867+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4114700878}) 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.867+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.867+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2891623878}) 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.867+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4279396699}) 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2520421736}) 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2401732966}) 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2520421736}) 2024-09-15T01:45:42.096 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=471905840}) 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2520421736}) 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3025744260}) 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2520421736}) 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=397230876}) 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2520421736}) 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2760656661}) 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2520421736}) 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3132248277}) 2024-09-15T01:45:42.097 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2520421736}) 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=336929879}) 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2520421736}) 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.868+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2026410270}) 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2182455032}) 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3758170430}) 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2182455032}) 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=744737564}) 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.098 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2182455032}) 2024-09-15T01:45:42.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4015761188}) 2024-09-15T01:45:42.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2182455032}) 2024-09-15T01:45:42.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3561268404}) 2024-09-15T01:45:42.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2182455032}) 2024-09-15T01:45:42.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3925382983}) 2024-09-15T01:45:42.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2182455032}) 2024-09-15T01:45:42.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3454575916}) 2024-09-15T01:45:42.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.099 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2182455032}) 2024-09-15T01:45:42.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3262436900}) 2024-09-15T01:45:42.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2182455032}) 2024-09-15T01:45:42.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.869+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=853872679}) 2024-09-15T01:45:42.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801400243}) 2024-09-15T01:45:42.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1558013275}) 2024-09-15T01:45:42.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801400243}) 2024-09-15T01:45:42.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3683151307}) 2024-09-15T01:45:42.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.100 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801400243}) 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1895232849}) 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801400243}) 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2578087544}) 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801400243}) 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1681478812}) 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801400243}) 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2489494473}) 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801400243}) 2024-09-15T01:45:42.101 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4125872643}) 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801400243}) 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.870+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1362434141}) 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.871+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.871+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2751164905}) 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.871+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2836425323}) 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.871+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.871+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2751164905}) 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.871+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1442912096}) 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.871+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.871+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2751164905}) 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1801245231}) 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.102 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2751164905}) 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=276578427}) 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2751164905}) 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3985033528}) 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2751164905}) 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1732557579}) 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2751164905}) 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4227480133}) 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2751164905}) 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.872+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3517972554}) 2024-09-15T01:45:42.103 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3911939029}) 2024-09-15T01:45:42.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3228824484}) 2024-09-15T01:45:42.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3911939029}) 2024-09-15T01:45:42.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2560748365}) 2024-09-15T01:45:42.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3911939029}) 2024-09-15T01:45:42.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3318415780}) 2024-09-15T01:45:42.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3911939029}) 2024-09-15T01:45:42.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=838866569}) 2024-09-15T01:45:42.104 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3911939029}) 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=785637038}) 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3911939029}) 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3530009556}) 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3911939029}) 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=795080104}) 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3911939029}) 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.873+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1782007470}) 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.105 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800526037}) 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3853729623}) 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800526037}) 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3231243090}) 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800526037}) 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1702649804}) 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800526037}) 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=357454817}) 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800526037}) 2024-09-15T01:45:42.106 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2645757099}) 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800526037}) 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1742530148}) 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800526037}) 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3578271402}) 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800526037}) 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.874+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=995034487}) 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1230696768}) 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1291774408}) 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.107 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1230696768}) 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1249800157}) 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1230696768}) 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2220093433}) 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1230696768}) 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3273815345}) 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.875+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1230696768}) 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.876+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1682028585}) 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.876+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.876+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1230696768}) 2024-09-15T01:45:42.108 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.876+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=446189719}) 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.876+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.876+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1230696768}) 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.876+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1958532055}) 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.876+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.876+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1230696768}) 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.876+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3662478634}) 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2091665013}) 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1303204219}) 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2091665013}) 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1157428355}) 2024-09-15T01:45:42.109 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2091665013}) 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2160724816}) 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2091665013}) 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4177728811}) 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2091665013}) 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3346951248}) 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2091665013}) 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3329834191}) 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2091665013}) 2024-09-15T01:45:42.110 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1991820443}) 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2091665013}) 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.877+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2595322505}) 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3489028944}) 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=157805824}) 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3489028944}) 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1311371135}) 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3489028944}) 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1012436293}) 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.111 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3489028944}) 2024-09-15T01:45:42.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1564799285}) 2024-09-15T01:45:42.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3489028944}) 2024-09-15T01:45:42.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4170847003}) 2024-09-15T01:45:42.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3489028944}) 2024-09-15T01:45:42.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3082370551}) 2024-09-15T01:45:42.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3489028944}) 2024-09-15T01:45:42.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2666263563}) 2024-09-15T01:45:42.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.112 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3489028944}) 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.878+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1087259786}) 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.879+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.879+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4021293198}) 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.879+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=621279091}) 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.879+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.879+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4021293198}) 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.879+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=478257924}) 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.879+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4021293198}) 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2444150429}) 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4021293198}) 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2545429274}) 2024-09-15T01:45:42.113 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4021293198}) 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1090475930}) 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4021293198}) 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3862432667}) 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4021293198}) 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=491428355}) 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4021293198}) 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.880+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1072103945}) 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.114 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2457710955}) 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1390226890}) 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2457710955}) 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1290894699}) 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2457710955}) 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2698302431}) 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2457710955}) 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1561140189}) 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2457710955}) 2024-09-15T01:45:42.115 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2064452479}) 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2457710955}) 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2675124661}) 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2457710955}) 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=186625991}) 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2457710955}) 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.881+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=9432907}) 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1483830661}) 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=287862117}) 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.116 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1483830661}) 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3739958679}) 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1483830661}) 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3642878929}) 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1483830661}) 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2571246447}) 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1483830661}) 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2446652854}) 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1483830661}) 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2256469272}) 2024-09-15T01:45:42.117 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1483830661}) 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2654092307}) 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1483830661}) 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.882+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1204313486}) 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=750965999}) 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=122795619}) 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=750965999}) 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2730954358}) 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.118 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=750965999}) 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=9231051}) 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=750965999}) 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3689797300}) 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=750965999}) 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2018785689}) 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=750965999}) 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3140544968}) 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=750965999}) 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4196856082}) 2024-09-15T01:45:42.119 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=750965999}) 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.883+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2445132076}) 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.884+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.884+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=51,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=15} crc {logm=1106679912,mdsmap=1531151339,mgr=3530002992,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1224078266,mon_config_key=1111662797,monmap=2850442478,osd_metadata=4061885386,osd_pg_creating=2978587036,osd_snap=956710772,osdmap=2747599974}) 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=52,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=12} crc {logm=2366077550,mdsmap=1531151339,mgr=1498814253,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3494899813,mon_config_key=3826684089,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=38904167}) 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=51,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=15} crc {logm=1106679912,mdsmap=1531151339,mgr=3530002992,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1224078266,mon_config_key=1111662797,monmap=2850442478,osd_metadata=4061885386,osd_pg_creating=2978587036,osd_snap=956710772,osdmap=2747599974}) 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=52,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=4,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=13} crc {logm=437577553,mdsmap=1531151339,mgr=2983835072,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=792831904,mon_config_key=1841931312,monmap=1281023599,osd_metadata=4279226934,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=2480911302}) 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=51,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=15} crc {logm=1106679912,mdsmap=1531151339,mgr=3530002992,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1224078266,mon_config_key=1111662797,monmap=2850442478,osd_metadata=4061885386,osd_pg_creating=2978587036,osd_snap=956710772,osdmap=2747599974}) 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=51,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=10} crc {logm=4174531808,mdsmap=1531151339,mgr=4058211233,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1861181113,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=3631557895}) 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.120 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=51,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=15} crc {logm=1106679912,mdsmap=1531151339,mgr=3530002992,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1224078266,mon_config_key=1111662797,monmap=2850442478,osd_metadata=4061885386,osd_pg_creating=2978587036,osd_snap=956710772,osdmap=2747599974}) 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=50,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=12} crc {logm=4052881461,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1884097735,mon_config_key=3826684089,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=2216047939}) 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=51,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=15} crc {logm=1106679912,mdsmap=1531151339,mgr=3530002992,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1224078266,mon_config_key=1111662797,monmap=2850442478,osd_metadata=4061885386,osd_pg_creating=2978587036,osd_snap=956710772,osdmap=2747599974}) 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=53,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=10} crc {logm=468400923,mdsmap=1531151339,mgr=4058211233,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=3263785238,mon_config_key=1763697598,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=3631557895}) 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=51,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=15} crc {logm=1106679912,mdsmap=1531151339,mgr=3530002992,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1224078266,mon_config_key=1111662797,monmap=2850442478,osd_metadata=4061885386,osd_pg_creating=2978587036,osd_snap=956710772,osdmap=2747599974}) 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=49,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_snap=2,osdmap=14} crc {logm=1914818723,mdsmap=3140514009,mgr=2983835072,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1861181113,mon_config_key=1111662798,monmap=2850442478,osd_metadata=4279226934,osd_snap=3178633393,osdmap=2683975787}) 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=51,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=15} crc {logm=1106679912,mdsmap=1531151339,mgr=3530002992,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1224078266,mon_config_key=1111662797,monmap=2850442478,osd_metadata=4061885386,osd_pg_creating=2978587036,osd_snap=956710772,osdmap=2747599974}) 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=50,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=12} crc {logm=891166429,mdsmap=3585761405,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=2231318257,mon_config_key=1111662797,monmap=2850442478,osd_metadata=4279226934,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=2462068657}) 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=51,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=1,osdmap=15} crc {logm=1106679912,mdsmap=1531151339,mgr=3530002992,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1224078266,mon_config_key=1111662797,monmap=2850442478,osd_metadata=4061885386,osd_pg_creating=2978587036,osd_snap=956710772,osdmap=2747599974}) 2024-09-15T01:45:42.121 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.885+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=52,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=9} crc {logm=237720188,mdsmap=2394902515,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1861181113,mon_config_key=3826684089,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=2156860865}) 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=692315318}) 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2428373745}) 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=692315318}) 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2619930058}) 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=692315318}) 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1782523308}) 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=692315318}) 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2375183089}) 2024-09-15T01:45:42.122 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=692315318}) 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3941809667}) 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=692315318}) 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.887+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1472667768}) 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.888+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.888+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=692315318}) 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.888+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2106959726}) 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.888+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.888+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=692315318}) 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.888+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=158919913}) 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.893+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.893+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=10} crc {osdmap=3696499683}) 2024-09-15T01:45:42.123 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.894+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=9} crc {osdmap=3307084030}) 2024-09-15T01:45:42.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.894+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.894+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=10} crc {osdmap=3696499683}) 2024-09-15T01:45:42.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.894+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=8} crc {osdmap=335340155}) 2024-09-15T01:45:42.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.894+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:45:42.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.894+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=10} crc {osdmap=3696499683}) 2024-09-15T01:45:42.124 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:45:41.894+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=9} crc {osdmap=3307084030}) 2024-09-15T01:45:42.759 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:45:42.759 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:45:42.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.877+0000 7fd234ae8640 10 monclient: get_monmap_and_config 2024-09-15T01:45:42.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.877+0000 7fd234ae8640 10 monclient: build_initial_monmap 2024-09-15T01:45:42.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.877+0000 7fd234ae8640 10 monclient: monmap: 2024-09-15T01:45:42.879 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:42.879 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:42.879 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:42.877961+0000 2024-09-15T01:45:42.879 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:42.877961+0000 2024-09-15T01:45:42.879 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:42.879 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:42.879 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:42.880 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:42.880 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:42.880 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:42.880 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:42.880 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:42.880 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:42.880 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:42.880 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:42.880 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:42.880 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:42.880 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.877+0000 7fd234ae8640 1 Processor -- start 2024-09-15T01:45:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.877+0000 7fd234ae8640 1 -- start start 2024-09-15T01:45:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.877+0000 7fd234ae8640 10 monclient: init 2024-09-15T01:45:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 10 monclient: _add_conns ranks=[10,8,5,7,9,2,4,3,6,1,0] 2024-09-15T01:45:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd230067130 0x7fd230067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 10 monclient(hunting): picked mon.noname-f con 0x7fd230067130 addr v2:172.21.15.204:3300/0 2024-09-15T01:45:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009d580 0x7fd230067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 10 monclient(hunting): picked mon.noname-e con 0x7fd23009d580 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd23009f000 0x7fd23012d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 10 monclient(hunting): picked mon.noname-h con 0x7fd23009f000 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23009e0c0 con 0x7fd230067130 2024-09-15T01:45:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd230067f80 con 0x7fd23009f000 2024-09-15T01:45:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd23009dde0 con 0x7fd23009d580 2024-09-15T01:45:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd234ae8640 10 monclient(hunting): authenticate will time out at 1547.628324s 2024-09-15T01:45:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009d580 0x7fd230067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009d580 0x7fd230067a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60164/0 (socket says 172.21.15.178:60164) 2024-09-15T01:45:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 1 -- 172.21.15.178:0/4024960578 learned_addr learned my addr 172.21.15.178:0/4024960578 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 10 monclient(hunting): get_auth_request con 0x7fd23009d580 auth_method 0 2024-09-15T01:45:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22ed9d640 1 --2- 172.21.15.178:0/4024960578 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd23009f000 0x7fd23012d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22e59c640 1 --2- 172.21.15.178:0/4024960578 >> v2:172.21.15.204:3300/0 conn(0x7fd230067130 0x7fd230067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22ed9d640 10 monclient(hunting): get_auth_request con 0x7fd23009f000 auth_method 0 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22e59c640 10 monclient(hunting): get_auth_request con 0x7fd230067130 auth_method 0 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:42.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 10 monclient(hunting): handle_auth_done global_id 35009 payload 274 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 1 -- 172.21.15.178:0/4024960578 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd23009f000 msgr2=0x7fd23012d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 1 --2- 172.21.15.178:0/4024960578 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd23009f000 0x7fd23012d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 1 -- 172.21.15.178:0/4024960578 >> v2:172.21.15.204:3300/0 conn(0x7fd230067130 msgr2=0x7fd230067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 1 --2- 172.21.15.178:0/4024960578 >> v2:172.21.15.204:3300/0 conn(0x7fd230067130 0x7fd230067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 10 monclient: _finish_hunting 0 2024-09-15T01:45:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 1 monclient: found mon.noname-e 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.878+0000 7fd22dd9b640 1 -- 172.21.15.178:0/4024960578 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23004bf60 con 0x7fd23009d580 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22dd9b640 1 --2- 172.21.15.178:0/4024960578 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009d580 0x7fd230067a40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd224009910 tx=0x7fd2240091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c5d8db5728f254d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22d59a640 1 -- 172.21.15.178:0/4024960578 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd22402ac80 con 0x7fd23009d580 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22d59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22d59a640 10 monclient: dump: 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22d59a640 10 monclient: _finish_auth 0 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd234ae8640 5 monclient: authenticate success, global_id 35009 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22d59a640 1 -- 172.21.15.178:0/4024960578 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22402ae00 con 0x7fd23009d580 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd234ae8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd234ae8640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd234ae8640 4 set_mon_vals no callback set 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22d59a640 1 -- 172.21.15.178:0/4024960578 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd224031ca0 con 0x7fd23009d580 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22d59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.879+0000 7fd22d59a640 10 monclient: dump: 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 10 monclient: shutdown 2024-09-15T01:45:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 1 -- 172.21.15.178:0/4024960578 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009d580 msgr2=0x7fd230067a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 1 --2- 172.21.15.178:0/4024960578 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009d580 0x7fd230067a40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd224009910 tx=0x7fd2240091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 1 -- 172.21.15.178:0/4024960578 shutdown_connections 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 1 --2- 172.21.15.178:0/4024960578 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd23009f000 0x7fd23012d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 1 --2- 172.21.15.178:0/4024960578 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009d580 0x7fd230067a40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 1 --2- 172.21.15.178:0/4024960578 >> v2:172.21.15.204:3300/0 conn(0x7fd230067130 0x7fd230067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 1 -- 172.21.15.178:0/4024960578 >> 172.21.15.178:0/4024960578 conn(0x7fd230068a60 msgr2=0x7fd2300a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 1 -- 172.21.15.178:0/4024960578 shutdown_connections 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 1 -- 172.21.15.178:0/4024960578 wait complete. 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 10 monclient: build_initial_monmap 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 10 monclient: monmap: 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 1 Processor -- start 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 1 -- start start 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.880+0000 7fd234ae8640 10 monclient: init 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 10 monclient: _add_conns ranks=[2,4,8,3,0,5,7,6,1] 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd230067130 0x7fd23013c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 10 monclient(hunting): picked mon.noname-b con 0x7fd230067130 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd23009d580 0x7fd23013b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 10 monclient(hunting): picked mon.noname-c con 0x7fd23009d580 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009f000 0x7fd23013f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 10 monclient(hunting): picked mon.noname-e con 0x7fd23009f000 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23009cb50 con 0x7fd230067130 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd23009d9e0 con 0x7fd23009d580 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd23013cd30 con 0x7fd23009f000 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd234ae8640 10 monclient(hunting): authenticate will time out at 1547.631421s 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22e59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd230067130 0x7fd23013c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd23009d580 0x7fd23013b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22ed9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009f000 0x7fd23013f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd23009d580 0x7fd23013b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:35334/0 (socket says 172.21.15.178:35334) 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 1 -- 172.21.15.178:0/204400076 learned_addr learned my addr 172.21.15.178:0/204400076 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 10 monclient(hunting): get_auth_request con 0x7fd23009d580 auth_method 0 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:42.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:42.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22ed9d640 10 monclient(hunting): get_auth_request con 0x7fd23009f000 auth_method 0 2024-09-15T01:45:42.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:42.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:42.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:42.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22e59c640 10 monclient(hunting): get_auth_request con 0x7fd230067130 auth_method 0 2024-09-15T01:45:42.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:42.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:42.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:42.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:42.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:42.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:42.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 10 monclient(hunting): handle_auth_done global_id 44617 payload 824 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 1 -- 172.21.15.178:0/204400076 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009f000 msgr2=0x7fd23013f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 1 --2- 172.21.15.178:0/204400076 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009f000 0x7fd23013f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 1 -- 172.21.15.178:0/204400076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd230067130 msgr2=0x7fd23013c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 1 --2- 172.21.15.178:0/204400076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd230067130 0x7fd23013c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 10 monclient: _finish_hunting 0 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 1 monclient: found mon.noname-c 2024-09-15T01:45:42.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22dd9b640 1 -- 172.21.15.178:0/204400076 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd224009450 con 0x7fd23009d580 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.881+0000 7fd22ed9d640 1 --2- 172.21.15.178:0/204400076 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009f000 0x7fd23013f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd22dd9b640 1 --2- 172.21.15.178:0/204400076 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd23009d580 0x7fd23013b810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd224000c00 tx=0x7fd224032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 1 -- 172.21.15.178:0/204400076 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd22402ac80 con 0x7fd23009d580 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 10 monclient: dump: 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:42.905 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 1 -- 172.21.15.178:0/204400076 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd224031820 con 0x7fd23009d580 2024-09-15T01:45:42.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd234ae8640 5 monclient: authenticate success, global_id 44617 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd234ae8640 10 monclient: _renew_subs 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 1 -- 172.21.15.178:0/204400076 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd224031e10 con 0x7fd23009d580 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd234ae8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd234ae8640 1 -- 172.21.15.178:0/204400076 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2301208b0 con 0x7fd23009d580 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd22cd99640 4 set_mon_vals no callback set 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 10 monclient: dump: 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:42.907 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd234ae8640 10 monclient: _renew_subs 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd234ae8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:42.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd234ae8640 1 -- 172.21.15.178:0/204400076 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23013bd50 con 0x7fd23009d580 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.882+0000 7fd21e7fc640 1 -- 172.21.15.178:0/204400076 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd224039040 con 0x7fd23009d580 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.883+0000 7fd234ae8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.883+0000 7fd234ae8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.883+0000 7fd234ae8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.883+0000 7fd21e7fc640 1 --2- 172.21.15.178:0/204400076 >> v2:172.21.15.204:6812/28210 conn(0x7fd2100397b0 0x7fd21003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.883+0000 7fd234ae8640 1 -- 172.21.15.178:0/204400076 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23004b720 con 0x7fd23009d580 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.883+0000 7fd21e7fc640 1 -- 172.21.15.178:0/204400076 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd22406aff0 con 0x7fd23009d580 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.883+0000 7fd22e59c640 1 --2- 172.21.15.178:0/204400076 >> v2:172.21.15.204:6812/28210 conn(0x7fd2100397b0 0x7fd21003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.883+0000 7fd22e59c640 10 monclient: get_auth_request con 0x7fd2100397b0 auth_method 0 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.883+0000 7fd22e59c640 1 --2- 172.21.15.178:0/204400076 >> v2:172.21.15.204:6812/28210 conn(0x7fd2100397b0 0x7fd21003bc60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd220009d20 tx=0x7fd2200090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.887+0000 7fd21e7fc640 1 -- 172.21.15.178:0/204400076 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd224041020 con 0x7fd23009d580 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.887+0000 7fd21e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:42.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:42.887+0000 7fd21e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.100+0000 7fd234ae8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:45:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.100+0000 7fd234ae8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:45:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.100+0000 7fd234ae8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.100+0000 7fd234ae8640 1 -- 172.21.15.178:0/204400076 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd230070770 con 0x7fd23009d580 2024-09-15T01:45:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.101+0000 7fd21e7fc640 1 -- 172.21.15.178:0/204400076 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7fd224037070 con 0x7fd23009d580 2024-09-15T01:45:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.101+0000 7fd21e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:45:43.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.101+0000 7fd21e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:43.104 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":138,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":5,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:45:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 1 -- 172.21.15.178:0/204400076 >> v2:172.21.15.204:6812/28210 conn(0x7fd2100397b0 msgr2=0x7fd21003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 1 --2- 172.21.15.178:0/204400076 >> v2:172.21.15.204:6812/28210 conn(0x7fd2100397b0 0x7fd21003bc60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd220009d20 tx=0x7fd2200090f0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 10 monclient: shutdown 2024-09-15T01:45:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 1 -- 172.21.15.178:0/204400076 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd23009d580 msgr2=0x7fd23013b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 1 --2- 172.21.15.178:0/204400076 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd23009d580 0x7fd23013b810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd224000c00 tx=0x7fd224032000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 1 -- 172.21.15.178:0/204400076 shutdown_connections 2024-09-15T01:45:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 1 --2- 172.21.15.178:0/204400076 >> v2:172.21.15.204:6812/28210 conn(0x7fd2100397b0 0x7fd21003bc60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 1 --2- 172.21.15.178:0/204400076 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd23009d580 0x7fd23013b810 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 1 --2- 172.21.15.178:0/204400076 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd23009f000 0x7fd23013f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 1 --2- 172.21.15.178:0/204400076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd230067130 0x7fd23013c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 1 -- 172.21.15.178:0/204400076 >> 172.21.15.178:0/204400076 conn(0x7fd230068a60 msgr2=0x7fd2300a09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 1 -- 172.21.15.178:0/204400076 shutdown_connections 2024-09-15T01:45:43.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.103+0000 7fd234ae8640 1 -- 172.21.15.178:0/204400076 wait complete. 2024-09-15T01:45:43.115 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:45:43.115 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T01:45:43.115 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:45:43.115 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:45:43.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.207+0000 7f913446f640 10 monclient: get_monmap_and_config 2024-09-15T01:45:43.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.207+0000 7f913446f640 10 monclient: build_initial_monmap 2024-09-15T01:45:43.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.207+0000 7f913446f640 10 monclient: monmap: 2024-09-15T01:45:43.210 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:43.210 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.210 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:43.208661+0000 2024-09-15T01:45:43.210 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:43.208661+0000 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:43.211 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 1 Processor -- start 2024-09-15T01:45:43.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 1 -- start start 2024-09-15T01:45:43.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 10 monclient: init 2024-09-15T01:45:43.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:43.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 10 monclient: _add_conns ranks=[9,6,5,2,7,1,4,8,0,10,3] 2024-09-15T01:45:43.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f912c0a2930 0x7f912c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 10 monclient(hunting): picked mon.noname-a con 0x7f912c0a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f912c09faa0 0x7f912c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 10 monclient(hunting): picked mon.noname-d con 0x7f912c09faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f912c0a08c0 0x7f912c0a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 10 monclient(hunting): picked mon.noname-h con 0x7f912c0a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912c09e0f0 con 0x7f912c0a2930 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f912c09dd80 con 0x7f912c0a08c0 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0a2d00 con 0x7f912c09faa0 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.208+0000 7f913446f640 10 monclient(hunting): authenticate will time out at 1547.958999s 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f912c09faa0 0x7f912c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f912c09faa0 0x7f912c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57114/0 (socket says 172.21.15.178:57114) 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 1 -- 172.21.15.178:0/1087392131 learned_addr learned my addr 172.21.15.178:0/1087392131 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 10 monclient(hunting): get_auth_request con 0x7f912c09faa0 auth_method 0 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f913220b640 1 --2- 172.21.15.178:0/1087392131 >> v2:172.21.15.178:3300/0 conn(0x7f912c0a2930 0x7f912c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f913220b640 10 monclient(hunting): get_auth_request con 0x7f912c0a2930 auth_method 0 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f913220b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f913220b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f913220b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9132a0c640 1 --2- 172.21.15.178:0/1087392131 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f912c0a08c0 0x7f912c0a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9132a0c640 10 monclient(hunting): get_auth_request con 0x7f912c0a08c0 auth_method 0 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9132a0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9132a0c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9132a0c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f913220b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f913220b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f913220b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9132a0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9132a0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9132a0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 10 monclient(hunting): handle_auth_done global_id 25485 payload 274 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 1 -- 172.21.15.178:0/1087392131 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f912c0a08c0 msgr2=0x7f912c0a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 1 --2- 172.21.15.178:0/1087392131 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f912c0a08c0 0x7f912c0a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 1 -- 172.21.15.178:0/1087392131 >> v2:172.21.15.178:3300/0 conn(0x7f912c0a2930 msgr2=0x7f912c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 1 --2- 172.21.15.178:0/1087392131 >> v2:172.21.15.178:3300/0 conn(0x7f912c0a2930 0x7f912c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 10 monclient: _finish_hunting 0 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 1 monclient: found mon.noname-d 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f9131a0a640 1 -- 172.21.15.178:0/1087392131 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c04bf60 con 0x7f912c09faa0 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.209+0000 7f913220b640 1 --2- 172.21.15.178:0/1087392131 >> v2:172.21.15.178:3300/0 conn(0x7f912c0a2930 0x7f912c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131a0a640 1 --2- 172.21.15.178:0/1087392131 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f912c09faa0 0x7f912c09fe70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f911c009910 tx=0x7f911c0091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c6c46be8f802c8ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131209640 1 -- 172.21.15.178:0/1087392131 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f911c02ac80 con 0x7f912c09faa0 2024-09-15T01:45:43.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131209640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131209640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131209640 10 monclient: dump: 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.228 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.229 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.229 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.229 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.229 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.229 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131209640 10 monclient: _finish_auth 0 2024-09-15T01:45:43.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131209640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:43.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 5 monclient: authenticate success, global_id 25485 2024-09-15T01:45:43.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:43.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131209640 1 -- 172.21.15.178:0/1087392131 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c02ae00 con 0x7f912c09faa0 2024-09-15T01:45:43.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131209640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:43.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 4 set_mon_vals no callback set 2024-09-15T01:45:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131209640 1 -- 172.21.15.178:0/1087392131 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f911c031ca0 con 0x7f912c09faa0 2024-09-15T01:45:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131209640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131209640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:45:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f9131209640 10 monclient: dump: 2024-09-15T01:45:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 10 monclient: shutdown 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 1 -- 172.21.15.178:0/1087392131 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f912c09faa0 msgr2=0x7f912c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 1 --2- 172.21.15.178:0/1087392131 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f912c09faa0 0x7f912c09fe70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f911c009910 tx=0x7f911c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 1 -- 172.21.15.178:0/1087392131 shutdown_connections 2024-09-15T01:45:43.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 1 --2- 172.21.15.178:0/1087392131 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f912c0a08c0 0x7f912c0a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 1 --2- 172.21.15.178:0/1087392131 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f912c09faa0 0x7f912c09fe70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 1 --2- 172.21.15.178:0/1087392131 >> v2:172.21.15.178:3300/0 conn(0x7f912c0a2930 0x7f912c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 1 -- 172.21.15.178:0/1087392131 >> 172.21.15.178:0/1087392131 conn(0x7f912c0a5ed0 msgr2=0x7f912c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:43.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 1 -- 172.21.15.178:0/1087392131 shutdown_connections 2024-09-15T01:45:43.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.210+0000 7f913446f640 1 -- 172.21.15.178:0/1087392131 wait complete. 2024-09-15T01:45:43.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.211+0000 7f913446f640 10 monclient: build_initial_monmap 2024-09-15T01:45:43.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.211+0000 7f913446f640 10 monclient: monmap: 2024-09-15T01:45:43.233 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:43.233 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:43.233 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:43.233 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:43.233 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:43.233 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:43.233 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:43.233 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:43.234 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:43.234 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:43.234 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:43.234 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:43.234 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:43.234 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:43.234 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.211+0000 7f913446f640 1 Processor -- start 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.211+0000 7f913446f640 1 -- start start 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.211+0000 7f913446f640 10 monclient: init 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 10 monclient: _add_conns ranks=[7,4,3,6,2,1,0,5,8] 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f912c09faa0 0x7f912c09b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 10 monclient(hunting): picked mon.noname-i con 0x7f912c09faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f912c0a08c0 0x7f912c131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 10 monclient(hunting): picked mon.noname-c con 0x7f912c0a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c0a2930 0x7f912c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 10 monclient(hunting): picked mon.noname-g con 0x7f912c0a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c09d630 con 0x7f912c0a2930 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f912c11ec40 con 0x7f912c0a08c0 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f912c1429e0 con 0x7f912c09faa0 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 10 monclient(hunting): authenticate will time out at 1547.962203s 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f912c0a08c0 0x7f912c131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f912c0a08c0 0x7f912c131840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:35344/0 (socket says 172.21.15.178:35344) 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 1 -- 172.21.15.178:0/3642456286 learned_addr learned my addr 172.21.15.178:0/3642456286 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 10 monclient(hunting): get_auth_request con 0x7f912c0a08c0 auth_method 0 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913220b640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f912c09faa0 0x7f912c09b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9132a0c640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c0a2930 0x7f912c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913220b640 10 monclient(hunting): get_auth_request con 0x7f912c09faa0 auth_method 0 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913220b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913220b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913220b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9132a0c640 10 monclient(hunting): get_auth_request con 0x7f912c0a2930 auth_method 0 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9132a0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9132a0c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9132a0c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9132a0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9132a0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9132a0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913220b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913220b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913220b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 10 monclient(hunting): handle_auth_done global_id 44626 payload 824 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 1 -- 172.21.15.178:0/3642456286 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f912c09faa0 msgr2=0x7f912c09b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f912c09faa0 0x7f912c09b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 1 -- 172.21.15.178:0/3642456286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c0a2930 msgr2=0x7f912c11c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c0a2930 0x7f912c11c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 10 monclient: _finish_hunting 0 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 1 monclient: found mon.noname-c 2024-09-15T01:45:43.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 1 -- 172.21.15.178:0/3642456286 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128009390 con 0x7f912c0a08c0 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9131a0a640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f912c0a08c0 0x7f912c131840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f911c0097e0 tx=0x7f911c032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f911c02ac80 con 0x7f912c0a08c0 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f91227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f91227fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f91227fc640 10 monclient: dump: 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.239 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f91227fc640 10 monclient: _finish_auth 0 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f91227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 5 monclient: authenticate success, global_id 44626 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c031820 con 0x7f912c0a08c0 2024-09-15T01:45:43.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 10 monclient: _renew_subs 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f913446f640 1 -- 172.21.15.178:0/3642456286 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911c009450 con 0x7f912c0a08c0 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f91227fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f911c031e10 con 0x7f912c0a08c0 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f91227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f91227fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f91227fc640 10 monclient: dump: 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.241 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.212+0000 7f9130a08640 4 set_mon_vals no callback set 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f913446f640 10 monclient: _renew_subs 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f913446f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:43.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f913446f640 1 -- 172.21.15.178:0/3642456286 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c11da70 con 0x7f912c0a08c0 2024-09-15T01:45:43.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f913446f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:43.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f913446f640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c131d80 0x7f912c11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f913446f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f913446f640 1 -- 172.21.15.178:0/3642456286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c11eb00 con 0x7f912c131d80 2024-09-15T01:45:43.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f911c039040 con 0x7f912c0a08c0 2024-09-15T01:45:43.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f913220b640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c131d80 0x7f912c11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f913220b640 10 monclient: get_auth_request con 0x7f912c131d80 auth_method 0 2024-09-15T01:45:43.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f913220b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f913220b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.213+0000 7f913220b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f913220b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f913220b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f913220b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 1 --2- 172.21.15.178:0/3642456286 >> v2:172.21.15.204:6812/28210 conn(0x7f91100397b0 0x7f911003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f913220b640 10 monclient(hunting): handle_auth_done global_id 64206 payload 824 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f913220b640 1 -- 172.21.15.178:0/3642456286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f912c04c4a0 con 0x7f912c131d80 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f911c06af60 con 0x7f912c0a08c0 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f913220b640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c131d80 0x7f912c11ca70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9128011bf0 tx=0x7f91280090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f9132a0c640 1 --2- 172.21.15.178:0/3642456286 >> v2:172.21.15.204:6812/28210 conn(0x7f91100397b0 0x7f911003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f9132a0c640 10 monclient: get_auth_request con 0x7f91100397b0 auth_method 0 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f912800fdb0 con 0x7f912c131d80 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 10 monclient: dump: 2024-09-15T01:45:43.244 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.245 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f912801d480 con 0x7f912c131d80 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c131d80 msgr2=0x7f912c11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f9132a0c640 1 --2- 172.21.15.178:0/3642456286 >> v2:172.21.15.204:6812/28210 conn(0x7f91100397b0 0x7f911003bc60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9124009fd0 tx=0x7f912400b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c131d80 0x7f912c11ca70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9128011bf0 tx=0x7f91280090f0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c131d80 msgr2=0x7f912c11ca70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.214+0000 7f91227fc640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c131d80 0x7f912c11ca70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913446f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913446f640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c11cfb0 0x7f912c096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913446f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913446f640 1 -- 172.21.15.178:0/3642456286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c04bc80 con 0x7f912c11cfb0 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913220b640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c11cfb0 0x7f912c096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913220b640 10 monclient: get_auth_request con 0x7f912c11cfb0 auth_method 0 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913220b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913220b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913220b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913220b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913220b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913220b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913220b640 10 monclient(hunting): handle_auth_done global_id 64215 payload 824 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913220b640 1 -- 172.21.15.178:0/3642456286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f912c05ed70 con 0x7f912c11cfb0 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f913220b640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c11cfb0 0x7f912c096a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9128011bf0 tx=0x7f912801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9128025040 con 0x7f912c11cfb0 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f91227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f91227fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.220+0000 7f91227fc640 10 monclient: dump: 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.248 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.249 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.249 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.249 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f912801c3e0 con 0x7f912c11cfb0 2024-09-15T01:45:43.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f91227fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:43.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f91227fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:43.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c11cfb0 msgr2=0x7f912c096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f91227fc640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c11cfb0 0x7f912c096a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9128011bf0 tx=0x7f912801d000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f91227fc640 1 -- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c11cfb0 msgr2=0x7f912c096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:43.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f91227fc640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f912c11cfb0 0x7f912c096a30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.250 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:43.250 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:43.250 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:43.250 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:43.250 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:43.250 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:43.250 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:43.250 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:43.250 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:43.250 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:43.250 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:43.250 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:43.251 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:43.251 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:43.251 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:43.251 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.251 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 5, 2024-09-15T01:45:43.251 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:43.251 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:43.251 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:43.251 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.251 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.252 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.252 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.252 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.252 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.252 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.252 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.252 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.252 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:43.252 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:43.253 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:43.254 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.255 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:43.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.256 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:43.256 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:43.256 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.256 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:43.256 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.257 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:43.257 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.257 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.257 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:43.257 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:43.259 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.259 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.260 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.261 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.262 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:43.265 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:43.268 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.268 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.268 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.268 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:43.268 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:43.268 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.268 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.268 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.268 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:43.269 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:43.269 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.269 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.269 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.269 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.269 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:43.269 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.269 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.269 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.269 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:43.270 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.272 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.273 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:43.274 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.275 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.275 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.275 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:43.275 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.275 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.275 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:43.275 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f913446f640 1 -- 172.21.15.178:0/3642456286 >> v2:172.21.15.204:6812/28210 conn(0x7f91100397b0 msgr2=0x7f911003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f913446f640 1 --2- 172.21.15.178:0/3642456286 >> v2:172.21.15.204:6812/28210 conn(0x7f91100397b0 0x7f911003bc60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9124009fd0 tx=0x7f912400b040 comp rx=0 tx=0).stop 2024-09-15T01:45:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f913446f640 10 monclient: shutdown 2024-09-15T01:45:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f913446f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f913446f640 1 -- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f912c0a08c0 msgr2=0x7f912c131840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f913446f640 1 --2- 172.21.15.178:0/3642456286 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f912c0a08c0 0x7f912c131840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f911c0097e0 tx=0x7f911c032000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f9132a0c640 1 -- 172.21.15.178:0/3642456286 reap_dead start 2024-09-15T01:45:43.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f9132a0c640 1 -- 172.21.15.178:0/3642456286 reap_dead start 2024-09-15T01:45:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f913446f640 1 -- 172.21.15.178:0/3642456286 shutdown_connections 2024-09-15T01:45:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f913446f640 1 -- 172.21.15.178:0/3642456286 >> 172.21.15.178:0/3642456286 conn(0x7f912c0a5ed0 msgr2=0x7f912c135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f913446f640 1 -- 172.21.15.178:0/3642456286 shutdown_connections 2024-09-15T01:45:43.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.221+0000 7f913446f640 1 -- 172.21.15.178:0/3642456286 wait complete. 2024-09-15T01:45:43.278 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:43.278 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:45:43.278 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:45:43.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.370+0000 7f74e4eaf640 10 monclient: get_monmap_and_config 2024-09-15T01:45:43.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.370+0000 7f74e4eaf640 10 monclient: build_initial_monmap 2024-09-15T01:45:43.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.370+0000 7f74e4eaf640 10 monclient: monmap: 2024-09-15T01:45:43.372 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:43.372 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:43.371306+0000 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:43.371306+0000 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:43.373 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:43.374 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:43.374 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.370+0000 7f74e4eaf640 1 Processor -- start 2024-09-15T01:45:43.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 1 -- start start 2024-09-15T01:45:43.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 10 monclient: init 2024-09-15T01:45:43.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:43.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 10 monclient: _add_conns ranks=[2,5,3,8,7,1,9,0,10,4,6] 2024-09-15T01:45:43.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e00a2930 0x7f74e0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 10 monclient(hunting): picked mon.noname-b con 0x7f74e00a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f74e009faa0 0x7f74e009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 10 monclient(hunting): picked mon.noname-h con 0x7f74e009faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74e00a08c0 0x7f74e00a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 10 monclient(hunting): picked mon.noname-g con 0x7f74e00a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e009e0f0 con 0x7f74e00a2930 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e009dd80 con 0x7f74e00a08c0 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00a2d00 con 0x7f74e009faa0 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:43.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74e4eaf640 10 monclient(hunting): authenticate will time out at 1548.121685s 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74de59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e00a2930 0x7f74e0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74de59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e00a2930 0x7f74e0132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:56538/0 (socket says 172.21.15.178:56538) 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74de59c640 1 -- 172.21.15.178:0/4148926634 learned_addr learned my addr 172.21.15.178:0/4148926634 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74de59c640 10 monclient(hunting): get_auth_request con 0x7f74e00a2930 auth_method 0 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74de59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74de59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74de59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74ddd9b640 1 --2- 172.21.15.178:0/4148926634 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f74e009faa0 0x7f74e009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74de59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74de59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74de59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74ddd9b640 10 monclient(hunting): get_auth_request con 0x7f74e009faa0 auth_method 0 2024-09-15T01:45:43.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74ded9d640 1 --2- 172.21.15.178:0/4148926634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74e00a08c0 0x7f74e00a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74ddd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74ddd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74ddd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74ded9d640 10 monclient(hunting): get_auth_request con 0x7f74e00a08c0 auth_method 0 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74ded9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74ded9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.371+0000 7f74ded9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74ddd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74ddd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74ddd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74ded9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74ded9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74ded9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74de59c640 10 monclient(hunting): handle_auth_done global_id 35003 payload 274 2024-09-15T01:45:43.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74de59c640 1 -- 172.21.15.178:0/4148926634 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f74e009faa0 msgr2=0x7f74e009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:43.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74de59c640 1 --2- 172.21.15.178:0/4148926634 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f74e009faa0 0x7f74e009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74de59c640 1 -- 172.21.15.178:0/4148926634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74e00a08c0 msgr2=0x7f74e00a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:43.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74de59c640 1 --2- 172.21.15.178:0/4148926634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74e00a08c0 0x7f74e00a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74de59c640 10 monclient: _finish_hunting 0 2024-09-15T01:45:43.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74de59c640 1 monclient: found mon.noname-b 2024-09-15T01:45:43.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74de59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:43.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74de59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:45:43.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74de59c640 1 -- 172.21.15.178:0/4148926634 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e004bf60 con 0x7f74e00a2930 2024-09-15T01:45:43.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74de59c640 1 --2- 172.21.15.178:0/4148926634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e00a2930 0x7f74e0132de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009940 tx=0x7f74d00091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a733c09cd0887b6f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74dd59a640 1 -- 172.21.15.178:0/4148926634 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f74d002ac60 con 0x7f74e00a2930 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74dd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74dd59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74dd59a640 10 monclient: dump: 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.389 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74dd59a640 10 monclient: _finish_auth 0 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74dd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74dd59a640 1 -- 172.21.15.178:0/4148926634 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d002ade0 con 0x7f74e00a2930 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74e4eaf640 5 monclient: authenticate success, global_id 35003 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74e4eaf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74dd59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74e4eaf640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:43.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74dd59a640 1 -- 172.21.15.178:0/4148926634 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f74d0031ca0 con 0x7f74e00a2930 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74dd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.372+0000 7f74e4eaf640 4 set_mon_vals no callback set 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74dd59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74dd59a640 10 monclient: dump: 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.391 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74e4eaf640 10 monclient: shutdown 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74e4eaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74e4eaf640 1 -- 172.21.15.178:0/4148926634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e00a2930 msgr2=0x7f74e0132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74e4eaf640 1 --2- 172.21.15.178:0/4148926634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e00a2930 0x7f74e0132de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009940 tx=0x7f74d00091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74e4eaf640 1 -- 172.21.15.178:0/4148926634 shutdown_connections 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74e4eaf640 1 --2- 172.21.15.178:0/4148926634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74e00a08c0 0x7f74e00a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74e4eaf640 1 --2- 172.21.15.178:0/4148926634 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f74e009faa0 0x7f74e009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74e4eaf640 1 --2- 172.21.15.178:0/4148926634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e00a2930 0x7f74e0132de0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74e4eaf640 1 -- 172.21.15.178:0/4148926634 >> 172.21.15.178:0/4148926634 conn(0x7f74e00a5ed0 msgr2=0x7f74e00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74e4eaf640 1 -- 172.21.15.178:0/4148926634 shutdown_connections 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.373+0000 7f74e4eaf640 1 -- 172.21.15.178:0/4148926634 wait complete. 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient: build_initial_monmap 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient: monmap: 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 1 Processor -- start 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 1 -- start start 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient: init 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient: _add_conns ranks=[1,0,4,6,3,2,7,8,5] 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74e009faa0 0x7f74e009b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient(hunting): picked mon.noname-f con 0x7f74e009faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74e00a08c0 0x7f74e0131bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient(hunting): picked mon.noname-a con 0x7f74e00a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f74e00a2930 0x7f74e01320f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient(hunting): picked mon.noname-c con 0x7f74e00a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e011ecd0 con 0x7f74e00a08c0 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e011efb0 con 0x7f74e009faa0 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74e011f290 con 0x7f74e00a2930 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:43.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.374+0000 7f74e4eaf640 10 monclient(hunting): authenticate will time out at 1548.124935s 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74e00a08c0 0x7f74e0131bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ded9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f74e00a2930 0x7f74e01320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74e00a08c0 0x7f74e0131bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33620/0 (socket says 172.21.15.178:33620) 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ded9d640 10 monclient(hunting): get_auth_request con 0x7f74e00a2930 auth_method 0 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ded9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 1 -- 172.21.15.178:0/3726885202 learned_addr learned my addr 172.21.15.178:0/3726885202 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ded9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ded9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 10 monclient(hunting): get_auth_request con 0x7f74e00a08c0 auth_method 0 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74de59c640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74e009faa0 0x7f74e009b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74de59c640 10 monclient(hunting): get_auth_request con 0x7f74e009faa0 auth_method 0 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74de59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74de59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74de59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ded9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ded9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ded9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74de59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74de59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74de59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 10 monclient(hunting): handle_auth_done global_id 64224 payload 824 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 1 -- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f74e00a2930 msgr2=0x7f74e01320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f74e00a2930 0x7f74e01320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 1 -- 172.21.15.178:0/3726885202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74e009faa0 msgr2=0x7f74e009b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74e009faa0 0x7f74e009b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 10 monclient: _finish_hunting 0 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 1 monclient: found mon.noname-a 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 1 -- 172.21.15.178:0/3726885202 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d0009450 con 0x7f74e00a08c0 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ddd9b640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74e00a08c0 0x7f74e0131bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f74d400e8d0 tx=0x7f74d40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f74d400eec0 con 0x7f74e00a08c0 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ce7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:45:43.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ce7fc640 10 monclient: dump: 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.399 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ce7fc640 10 monclient: _finish_auth 0 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74e4eaf640 5 monclient: authenticate success, global_id 64224 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d400cd60 con 0x7f74e00a08c0 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74e4eaf640 10 monclient: _renew_subs 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74e4eaf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74e4eaf640 1 -- 172.21.15.178:0/3726885202 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e011dd70 con 0x7f74e00a08c0 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ce7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f74d40138f0 con 0x7f74e00a08c0 2024-09-15T01:45:43.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ce7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74ce7fc640 10 monclient: dump: 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.401 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74dcd99640 4 set_mon_vals no callback set 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74e4eaf640 10 monclient: _renew_subs 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74e4eaf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.375+0000 7f74e4eaf640 1 -- 172.21.15.178:0/3726885202 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e0142730 con 0x7f74e00a08c0 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f74d4013470 con 0x7f74e00a08c0 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74e4eaf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74e4eaf640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011c530 0x7f74e011c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74e4eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74e4eaf640 1 -- 172.21.15.178:0/3726885202 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e01429b0 con 0x7f74e011c530 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74de59c640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011c530 0x7f74e011c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74de59c640 10 monclient: get_auth_request con 0x7f74e011c530 auth_method 0 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74de59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74de59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74de59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74ce7fc640 1 --2- 172.21.15.178:0/3726885202 >> v2:172.21.15.204:6812/28210 conn(0x7f74c00396e0 0x7f74c003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74de59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74de59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74de59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f74d404bec0 con 0x7f74e00a08c0 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74ded9d640 1 --2- 172.21.15.178:0/3726885202 >> v2:172.21.15.204:6812/28210 conn(0x7f74c00396e0 0x7f74c003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74ded9d640 10 monclient: get_auth_request con 0x7f74c00396e0 auth_method 0 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74de59c640 10 monclient(hunting): handle_auth_done global_id 35012 payload 824 2024-09-15T01:45:43.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.376+0000 7f74de59c640 1 -- 172.21.15.178:0/3726885202 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74e011ee40 con 0x7f74e011c530 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74de59c640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011c530 0x7f74e011c900 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009910 tx=0x7f74d0032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f74d002ac60 con 0x7f74e011c530 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74ce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74ce7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74ce7fc640 10 monclient: dump: 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.404 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74ded9d640 1 --2- 172.21.15.178:0/3726885202 >> v2:172.21.15.204:6812/28210 conn(0x7f74c00396e0 0x7f74c003bb90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009fd0 tx=0x7f74d800b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f74d003d690 con 0x7f74e011c530 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74ce7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74ce7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011c530 msgr2=0x7f74e011c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74ce7fc640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011c530 0x7f74e011c900 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009910 tx=0x7f74d0032000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011c530 msgr2=0x7f74e011c900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:43.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.377+0000 7f74ce7fc640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011c530 0x7f74e011c900 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009910 tx=0x7f74d0032000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74e4eaf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:43.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74e4eaf640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011ce40 0x7f74e0096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74e4eaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74e4eaf640 1 -- 172.21.15.178:0/3726885202 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e009ba30 con 0x7f74e011ce40 2024-09-15T01:45:43.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74de59c640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011ce40 0x7f74e0096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74de59c640 10 monclient: get_auth_request con 0x7f74e011ce40 auth_method 0 2024-09-15T01:45:43.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74de59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74de59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74de59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74de59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74de59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74de59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74de59c640 10 monclient(hunting): handle_auth_done global_id 35021 payload 824 2024-09-15T01:45:43.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74de59c640 1 -- 172.21.15.178:0/3726885202 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f74e011f450 con 0x7f74e011ce40 2024-09-15T01:45:43.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74de59c640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011ce40 0x7f74e0096a30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009940 tx=0x7f74d003d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f74d003cd20 con 0x7f74e011ce40 2024-09-15T01:45:43.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74ce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74ce7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:43.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74ce7fc640 10 monclient: dump: 2024-09-15T01:45:43.409 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.409 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.409 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.409 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.409 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.409 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f74d003ce80 con 0x7f74e011ce40 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74ce7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:43.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74ce7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:43.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011ce40 msgr2=0x7f74e0096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74ce7fc640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011ce40 0x7f74e0096a30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009940 tx=0x7f74d003d000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74ce7fc640 1 -- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011ce40 msgr2=0x7f74e0096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:43.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.383+0000 7f74ce7fc640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74e011ce40 0x7f74e0096a30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.411 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:43.411 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:43.411 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:43.411 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:43.411 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:45:43.412 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.413 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.414 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:43.415 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:43.416 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:43.416 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:43.416 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.416 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.416 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.416 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.416 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.416 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.416 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.416 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.416 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.419 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:43.419 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:43.419 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.419 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:43.419 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.419 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:43.420 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.421 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.422 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:43.423 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:43.426 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.427 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.427 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.427 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:43.427 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:43.427 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.427 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.427 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.427 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:43.427 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:43.428 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.428 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.428 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.428 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:43.428 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.428 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.428 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:43.429 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.431 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:43.431 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:43.431 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.431 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.431 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.431 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:43.431 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:43.431 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.431 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:43.432 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:43.433 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.434 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:43.435 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.435 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.435 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:43.435 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:43.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.384+0000 7f74e4eaf640 1 -- 172.21.15.178:0/3726885202 >> v2:172.21.15.204:6812/28210 conn(0x7f74c00396e0 msgr2=0x7f74c003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.384+0000 7f74e4eaf640 1 --2- 172.21.15.178:0/3726885202 >> v2:172.21.15.204:6812/28210 conn(0x7f74c00396e0 0x7f74c003bb90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009fd0 tx=0x7f74d800b040 comp rx=0 tx=0).stop 2024-09-15T01:45:43.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.384+0000 7f74e4eaf640 10 monclient: shutdown 2024-09-15T01:45:43.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.384+0000 7f74e4eaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:43.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.384+0000 7f74e4eaf640 1 -- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74e00a08c0 msgr2=0x7f74e0131bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.384+0000 7f74e4eaf640 1 --2- 172.21.15.178:0/3726885202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74e00a08c0 0x7f74e0131bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f74d400e8d0 tx=0x7f74d40090f0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.384+0000 7f74ded9d640 1 -- 172.21.15.178:0/3726885202 reap_dead start 2024-09-15T01:45:43.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.384+0000 7f74ded9d640 1 -- 172.21.15.178:0/3726885202 reap_dead start 2024-09-15T01:45:43.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.384+0000 7f74e4eaf640 1 -- 172.21.15.178:0/3726885202 shutdown_connections 2024-09-15T01:45:43.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.384+0000 7f74e4eaf640 1 -- 172.21.15.178:0/3726885202 >> 172.21.15.178:0/3726885202 conn(0x7f74e00a5ed0 msgr2=0x7f74e009e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:43.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.384+0000 7f74e4eaf640 1 -- 172.21.15.178:0/3726885202 shutdown_connections 2024-09-15T01:45:43.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.384+0000 7f74e4eaf640 1 -- 172.21.15.178:0/3726885202 wait complete. 2024-09-15T01:45:43.437 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:43.437 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:45:43.438 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:45:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.529+0000 7f495cd09640 10 monclient: get_monmap_and_config 2024-09-15T01:45:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.529+0000 7f495cd09640 10 monclient: build_initial_monmap 2024-09-15T01:45:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.529+0000 7f495cd09640 10 monclient: monmap: 2024-09-15T01:45:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.532 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:43.530460+0000 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:43.530460+0000 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:43.533 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:43.534 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:43.534 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 1 Processor -- start 2024-09-15T01:45:43.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 1 -- start start 2024-09-15T01:45:43.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 10 monclient: init 2024-09-15T01:45:43.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:43.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 10 monclient: _add_conns ranks=[8,10,4,1,3,0,6,5,9,7,2] 2024-09-15T01:45:43.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49580a2930 0x7f4958132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 10 monclient(hunting): picked mon.noname-e con 0x7f49580a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:43.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f495809faa0 0x7f495809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 10 monclient(hunting): picked mon.noname-f con 0x7f495809faa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:45:43.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49580a08c0 0x7f49580a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 10 monclient(hunting): picked mon.noname-c con 0x7f49580a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:43.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495809e0f0 con 0x7f495809faa0 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f495809dd80 con 0x7f49580a08c0 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f49580a2d00 con 0x7f49580a2930 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495cd09640 10 monclient(hunting): authenticate will time out at 1548.280798s 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495659c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49580a2930 0x7f4958132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f4956d9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49580a08c0 0x7f49580a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495659c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49580a2930 0x7f4958132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60174/0 (socket says 172.21.15.178:60174) 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f4956d9d640 10 monclient(hunting): get_auth_request con 0x7f49580a08c0 auth_method 0 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495659c640 1 -- 172.21.15.178:0/339678865 learned_addr learned my addr 172.21.15.178:0/339678865 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:43.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f4956d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f4956d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f4956d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f495659c640 10 monclient(hunting): get_auth_request con 0x7f49580a2930 auth_method 0 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.530+0000 7f4955d9b640 1 --2- 172.21.15.178:0/339678865 >> v2:172.21.15.204:3300/0 conn(0x7f495809faa0 0x7f495809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f495659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f495659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f495659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4955d9b640 10 monclient(hunting): get_auth_request con 0x7f495809faa0 auth_method 0 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4955d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4955d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4955d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f495659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f495659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f495659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4955d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4955d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4955d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 10 monclient(hunting): handle_auth_done global_id 44644 payload 274 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 1 -- 172.21.15.178:0/339678865 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49580a2930 msgr2=0x7f4958132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 1 --2- 172.21.15.178:0/339678865 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49580a2930 0x7f4958132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 1 -- 172.21.15.178:0/339678865 >> v2:172.21.15.204:3300/0 conn(0x7f495809faa0 msgr2=0x7f495809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 1 --2- 172.21.15.178:0/339678865 >> v2:172.21.15.204:3300/0 conn(0x7f495809faa0 0x7f495809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:45:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 1 monclient: found mon.noname-c 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 1 -- 172.21.15.178:0/339678865 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495804bf60 con 0x7f49580a08c0 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f495659c640 1 --2- 172.21.15.178:0/339678865 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49580a2930 0x7f4958132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4955d9b640 1 --2- 172.21.15.178:0/339678865 >> v2:172.21.15.204:3300/0 conn(0x7f495809faa0 0x7f495809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.531+0000 7f4956d9d640 1 --2- 172.21.15.178:0/339678865 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49580a08c0 0x7f49580a0d60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4950009e60 tx=0x7f4950009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=55f4d474a17e65f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495559a640 1 -- 172.21.15.178:0/339678865 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f49500079d0 con 0x7f49580a08c0 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495559a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495559a640 10 monclient: dump: 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495559a640 10 monclient: _finish_auth 0 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495559a640 1 -- 172.21.15.178:0/339678865 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495002d050 con 0x7f49580a08c0 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 5 monclient: authenticate success, global_id 44644 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495559a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495559a640 1 -- 172.21.15.178:0/339678865 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4950007bd0 con 0x7f49580a08c0 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 4 set_mon_vals no callback set 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495559a640 10 monclient: dump: 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.549 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 10 monclient: shutdown 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 1 -- 172.21.15.178:0/339678865 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49580a08c0 msgr2=0x7f49580a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 1 --2- 172.21.15.178:0/339678865 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49580a08c0 0x7f49580a0d60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4950009e60 tx=0x7f4950009780 comp rx=0 tx=0).stop 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 1 -- 172.21.15.178:0/339678865 shutdown_connections 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 1 --2- 172.21.15.178:0/339678865 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49580a08c0 0x7f49580a0d60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 1 --2- 172.21.15.178:0/339678865 >> v2:172.21.15.204:3300/0 conn(0x7f495809faa0 0x7f495809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 1 --2- 172.21.15.178:0/339678865 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49580a2930 0x7f4958132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 1 -- 172.21.15.178:0/339678865 >> 172.21.15.178:0/339678865 conn(0x7f49580a5ed0 msgr2=0x7f49580a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 1 -- 172.21.15.178:0/339678865 shutdown_connections 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.532+0000 7f495cd09640 1 -- 172.21.15.178:0/339678865 wait complete. 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 10 monclient: build_initial_monmap 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 10 monclient: monmap: 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:43.551 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:43.552 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:43.553 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 1 Processor -- start 2024-09-15T01:45:43.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 1 -- start start 2024-09-15T01:45:43.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 10 monclient: init 2024-09-15T01:45:43.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:43.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 10 monclient: _add_conns ranks=[4,7,1,5,6,3,0,2,8] 2024-09-15T01:45:43.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495809faa0 0x7f495809aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 10 monclient(hunting): picked mon.noname-c con 0x7f495809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:43.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f49580a08c0 0x7f49581317e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 10 monclient(hunting): picked mon.noname-i con 0x7f49580a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49580a2930 0x7f495811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.533+0000 7f495cd09640 10 monclient(hunting): picked mon.noname-f con 0x7f49580a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4958142a80 con 0x7f49580a2930 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f49580a2390 con 0x7f495809faa0 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f495809d630 con 0x7f49580a08c0 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 10 monclient(hunting): authenticate will time out at 1548.284014s 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495809faa0 0x7f495809aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495809faa0 0x7f495809aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:35388/0 (socket says 172.21.15.178:35388) 2024-09-15T01:45:43.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 1 -- 172.21.15.178:0/1401649017 learned_addr learned my addr 172.21.15.178:0/1401649017 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 10 monclient(hunting): get_auth_request con 0x7f495809faa0 auth_method 0 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f4956d9d640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49580a2930 0x7f495811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f4955d9b640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f49580a08c0 0x7f49581317e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 10 monclient(hunting): handle_auth_done global_id 44653 payload 824 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 1 -- 172.21.15.178:0/1401649017 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f49580a08c0 msgr2=0x7f49581317e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f49580a08c0 0x7f49581317e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 1 -- 172.21.15.178:0/1401649017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49580a2930 msgr2=0x7f495811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49580a2930 0x7f495811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 10 monclient: _finish_hunting 0 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 1 monclient: found mon.noname-c 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 1 -- 172.21.15.178:0/1401649017 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c009390 con 0x7f495809faa0 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f4956d9d640 10 monclient: get_auth_request con 0x7f49580a2930 auth_method 0 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f4956d9d640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49580a2930 0x7f495811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495659c640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495809faa0 0x7f495809aee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f494800ec80 tx=0x7f49480091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4948009730 con 0x7f495809faa0 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f49467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f49467fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f49467fc640 10 monclient: dump: 2024-09-15T01:45:43.556 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.557 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f49467fc640 10 monclient: _finish_auth 0 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f49467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 5 monclient: authenticate success, global_id 44653 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 10 monclient: _renew_subs 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494800eec0 con 0x7f495809faa0 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 1 -- 172.21.15.178:0/1401649017 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4948009450 con 0x7f495809faa0 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f49467fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4948013d70 con 0x7f495809faa0 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f49467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f49467fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f4954d99640 4 set_mon_vals no callback set 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f49467fc640 10 monclient: dump: 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.559 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 10 monclient: _renew_subs 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.534+0000 7f495cd09640 1 -- 172.21.15.178:0/1401649017 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4950009a20 con 0x7f495809faa0 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f494800cb60 con 0x7f495809faa0 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f495cd09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f495cd09640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4958131d20 0x7f495811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f495cd09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f495cd09640 1 -- 172.21.15.178:0/1401649017 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f49581320f0 con 0x7f4958131d20 2024-09-15T01:45:43.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f49467fc640 1 --2- 172.21.15.178:0/1401649017 >> v2:172.21.15.204:6812/28210 conn(0x7f49380397e0 0x7f493803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f494804c350 con 0x7f495809faa0 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f4955d9b640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4958131d20 0x7f495811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f4955d9b640 10 monclient: get_auth_request con 0x7f4958131d20 auth_method 0 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f4955d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f4955d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f4955d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f4956d9d640 1 --2- 172.21.15.178:0/1401649017 >> v2:172.21.15.204:6812/28210 conn(0x7f49380397e0 0x7f493803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f4955d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f4955d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f4955d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f4956d9d640 10 monclient: get_auth_request con 0x7f49380397e0 auth_method 0 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f4955d9b640 10 monclient(hunting): handle_auth_done global_id 44662 payload 824 2024-09-15T01:45:43.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.535+0000 7f4955d9b640 1 -- 172.21.15.178:0/1401649017 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f495804c4a0 con 0x7f4958131d20 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f4955d9b640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4958131d20 0x7f495811ca70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f494c010840 tx=0x7f494c008040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f4956d9d640 1 --2- 172.21.15.178:0/1401649017 >> v2:172.21.15.204:6812/28210 conn(0x7f49380397e0 0x7f493803bc90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4950034000 tx=0x7f4950037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f494c00ed50 con 0x7f4958131d20 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f49467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f49467fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f49467fc640 10 monclient: dump: 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.562 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f494c00eeb0 con 0x7f4958131d20 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f49467fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f49467fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4958131d20 msgr2=0x7f495811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f49467fc640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4958131d20 0x7f495811ca70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f494c010840 tx=0x7f494c008040 comp rx=0 tx=0).stop 2024-09-15T01:45:43.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4958131d20 msgr2=0x7f495811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:43.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.536+0000 7f49467fc640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f4958131d20 0x7f495811ca70 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f494c010840 tx=0x7f494c008040 comp rx=0 tx=0).stop 2024-09-15T01:45:43.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f495cd09640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:43.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f495cd09640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495811cfb0 0x7f4958096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f495cd09640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f495cd09640 1 -- 172.21.15.178:0/1401649017 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f495804bc80 con 0x7f495811cfb0 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f4955d9b640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495811cfb0 0x7f4958096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f4955d9b640 10 monclient: get_auth_request con 0x7f495811cfb0 auth_method 0 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f4955d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f4955d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f4955d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f4955d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f4955d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f4955d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f4955d9b640 10 monclient(hunting): handle_auth_done global_id 44671 payload 824 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f4955d9b640 1 -- 172.21.15.178:0/1401649017 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f495805ed70 con 0x7f495811cfb0 2024-09-15T01:45:43.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f4955d9b640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495811cfb0 0x7f4958096a30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f494c000c00 tx=0x7f494c01c000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f494c014b50 con 0x7f495811cfb0 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f49467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f49467fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f49467fc640 10 monclient: dump: 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.567 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9482 (secure 0 0 0) 0x7f494c00e660 con 0x7f495811cfb0 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f49467fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f49467fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495811cfb0 msgr2=0x7f4958096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f49467fc640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495811cfb0 0x7f4958096a30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f494c000c00 tx=0x7f494c01c000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f49467fc640 1 -- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495811cfb0 msgr2=0x7f4958096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:43.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.542+0000 7f49467fc640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495811cfb0 0x7f4958096a30 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f494c000c00 tx=0x7f494c01c000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.569 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:43.569 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:43.569 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:43.569 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:43.569 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:43.569 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:43.569 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:43.569 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:43.570 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:43.570 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:43.570 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:43.570 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:43.570 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:43.570 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:43.570 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:43.570 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.570 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:45:43.570 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:43.570 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:43.571 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:43.572 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:43.573 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.574 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:43.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.575 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:43.575 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:43.575 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.575 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:43.575 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.576 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:43.578 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.580 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:43.581 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.582 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.582 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.582 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:43.582 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:43.582 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.582 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.582 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.582 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:43.582 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.584 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:43.584 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:43.584 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.584 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.584 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.584 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:43.584 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.584 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.585 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.586 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.587 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:43.588 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.590 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.591 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.592 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.593 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.593 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:43.593 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.593 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.593 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:43.593 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:43.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.543+0000 7f495cd09640 1 -- 172.21.15.178:0/1401649017 >> v2:172.21.15.204:6812/28210 conn(0x7f49380397e0 msgr2=0x7f493803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.543+0000 7f495cd09640 1 --2- 172.21.15.178:0/1401649017 >> v2:172.21.15.204:6812/28210 conn(0x7f49380397e0 0x7f493803bc90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4950034000 tx=0x7f4950037000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.543+0000 7f495cd09640 10 monclient: shutdown 2024-09-15T01:45:43.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.543+0000 7f495cd09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:43.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.543+0000 7f495cd09640 1 -- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495809faa0 msgr2=0x7f495809aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.543+0000 7f495cd09640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495809faa0 0x7f495809aee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f494800ec80 tx=0x7f49480091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.543+0000 7f4956d9d640 1 -- 172.21.15.178:0/1401649017 reap_dead start 2024-09-15T01:45:43.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.543+0000 7f495cd09640 1 -- 172.21.15.178:0/1401649017 shutdown_connections 2024-09-15T01:45:43.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.543+0000 7f495cd09640 1 --2- 172.21.15.178:0/1401649017 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f495809faa0 0x7f495809aee0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.543+0000 7f495cd09640 1 -- 172.21.15.178:0/1401649017 >> 172.21.15.178:0/1401649017 conn(0x7f49580a5ed0 msgr2=0x7f4958135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:43.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.543+0000 7f495cd09640 1 -- 172.21.15.178:0/1401649017 shutdown_connections 2024-09-15T01:45:43.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.543+0000 7f495cd09640 1 -- 172.21.15.178:0/1401649017 wait complete. 2024-09-15T01:45:43.595 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:43.596 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:45:43.596 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:45:43.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.689+0000 7f0fc149f640 10 monclient: get_monmap_and_config 2024-09-15T01:45:43.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.689+0000 7f0fc149f640 10 monclient: build_initial_monmap 2024-09-15T01:45:43.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.689+0000 7f0fc149f640 10 monclient: monmap: 2024-09-15T01:45:43.691 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:43.691 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.691 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:43.690276+0000 2024-09-15T01:45:43.691 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:43.690276+0000 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:43.692 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:43.693 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.689+0000 7f0fc149f640 1 Processor -- start 2024-09-15T01:45:43.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 1 -- start start 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 10 monclient: init 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 10 monclient: _add_conns ranks=[2,6,4,9,10,8,7,1,0,3,5] 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fbc0a2930 0x7f0fbc132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 10 monclient(hunting): picked mon.noname-b con 0x7f0fbc0a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc09faa0 0x7f0fbc09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 10 monclient(hunting): picked mon.noname-d con 0x7f0fbc09faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc0a08c0 0x7f0fbc0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 10 monclient(hunting): picked mon.noname-c con 0x7f0fbc0a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc09e0f0 con 0x7f0fbc0a2930 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc09dd80 con 0x7f0fbc0a08c0 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0a2d00 con 0x7f0fbc09faa0 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fc149f640 10 monclient(hunting): authenticate will time out at 1548.440541s 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbb7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc0a08c0 0x7f0fbc0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fbc0a2930 0x7f0fbc132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fba7fc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc09faa0 0x7f0fbc09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fbc0a2930 0x7f0fbc132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:56572/0 (socket says 172.21.15.178:56572) 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 1 -- 172.21.15.178:0/1035356167 learned_addr learned my addr 172.21.15.178:0/1035356167 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbb7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc0a08c0 0x7f0fbc0a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:35414/0 (socket says 172.21.15.178:35414) 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 10 monclient(hunting): get_auth_request con 0x7f0fbc0a2930 auth_method 0 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbb7fe640 10 monclient(hunting): get_auth_request con 0x7f0fbc0a08c0 auth_method 0 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fba7fc640 10 monclient(hunting): get_auth_request con 0x7f0fbc09faa0 auth_method 0 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fba7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 10 monclient(hunting): handle_auth_done global_id 35030 payload 274 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 1 -- 172.21.15.178:0/1035356167 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc09faa0 msgr2=0x7f0fbc09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 1 --2- 172.21.15.178:0/1035356167 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc09faa0 0x7f0fbc09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 1 -- 172.21.15.178:0/1035356167 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc0a08c0 msgr2=0x7f0fbc0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 1 --2- 172.21.15.178:0/1035356167 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc0a08c0 0x7f0fbc0a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 10 monclient: _finish_hunting 0 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 1 monclient: found mon.noname-b 2024-09-15T01:45:43.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.690+0000 7f0fbaffd640 1 -- 172.21.15.178:0/1035356167 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc04bf60 con 0x7f0fbc0a2930 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fba7fc640 1 --2- 172.21.15.178:0/1035356167 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc09faa0 0x7f0fbc09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fbb7fe640 1 --2- 172.21.15.178:0/1035356167 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc0a08c0 0x7f0fbc0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fbaffd640 1 --2- 172.21.15.178:0/1035356167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fbc0a2930 0x7f0fbc132de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009940 tx=0x7f0fac0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a8020def9e6e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fb9ffb640 1 -- 172.21.15.178:0/1035356167 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0fac02ac60 con 0x7f0fbc0a2930 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fb9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fb9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fb9ffb640 10 monclient: dump: 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.707 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.708 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.708 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.708 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.708 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.708 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.708 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.708 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.708 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.708 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.708 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fb9ffb640 10 monclient: _finish_auth 0 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fb9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fc149f640 5 monclient: authenticate success, global_id 35030 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fc149f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fb9ffb640 1 -- 172.21.15.178:0/1035356167 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac02ade0 con 0x7f0fbc0a2930 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fb9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fc149f640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fc149f640 4 set_mon_vals no callback set 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fb9ffb640 1 -- 172.21.15.178:0/1035356167 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0fac031ca0 con 0x7f0fbc0a2930 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fb9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fb9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.691+0000 7f0fb9ffb640 10 monclient: dump: 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.710 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.692+0000 7f0fc149f640 10 monclient: shutdown 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.692+0000 7f0fc149f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.692+0000 7f0fc149f640 1 -- 172.21.15.178:0/1035356167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fbc0a2930 msgr2=0x7f0fbc132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.692+0000 7f0fc149f640 1 --2- 172.21.15.178:0/1035356167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fbc0a2930 0x7f0fbc132de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009940 tx=0x7f0fac0091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.692+0000 7f0fc149f640 1 -- 172.21.15.178:0/1035356167 shutdown_connections 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.692+0000 7f0fc149f640 1 --2- 172.21.15.178:0/1035356167 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc0a08c0 0x7f0fbc0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.692+0000 7f0fc149f640 1 --2- 172.21.15.178:0/1035356167 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc09faa0 0x7f0fbc09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.692+0000 7f0fc149f640 1 --2- 172.21.15.178:0/1035356167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fbc0a2930 0x7f0fbc132de0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.692+0000 7f0fc149f640 1 -- 172.21.15.178:0/1035356167 >> 172.21.15.178:0/1035356167 conn(0x7f0fbc0a5ed0 msgr2=0x7f0fbc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:43.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.692+0000 7f0fc149f640 1 -- 172.21.15.178:0/1035356167 shutdown_connections 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.692+0000 7f0fc149f640 1 -- 172.21.15.178:0/1035356167 wait complete. 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient: build_initial_monmap 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient: monmap: 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:43.712 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 1 Processor -- start 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 1 -- start start 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient: init 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient: _add_conns ranks=[4,8,0,5,6,1,7,3,2] 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc09faa0 0x7f0fbc09b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient(hunting): picked mon.noname-c con 0x7f0fbc09faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0fbc0a08c0 0x7f0fbc131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient(hunting): picked mon.noname-e con 0x7f0fbc0a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fbc0a2930 0x7f0fbc141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient(hunting): picked mon.noname-a con 0x7f0fbc0a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc09bc40 con 0x7f0fbc0a2930 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc09d630 con 0x7f0fbc09faa0 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc11ed20 con 0x7f0fbc0a08c0 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.693+0000 7f0fc149f640 10 monclient(hunting): authenticate will time out at 1548.443910s 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc09faa0 0x7f0fbc09b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fba7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0fbc0a08c0 0x7f0fbc131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc09faa0 0x7f0fbc09b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:35424/0 (socket says 172.21.15.178:35424) 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fba7fc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0fbc0a08c0 0x7f0fbc131920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60184/0 (socket says 172.21.15.178:60184) 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 1 -- 172.21.15.178:0/498089113 learned_addr learned my addr 172.21.15.178:0/498089113 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 10 monclient(hunting): get_auth_request con 0x7f0fbc09faa0 auth_method 0 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbb7fe640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fbc0a2930 0x7f0fbc141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fba7fc640 10 monclient(hunting): get_auth_request con 0x7f0fbc0a08c0 auth_method 0 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fba7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbb7fe640 10 monclient(hunting): get_auth_request con 0x7f0fbc0a2930 auth_method 0 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 10 monclient(hunting): handle_auth_done global_id 44689 payload 824 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 1 -- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0fbc0a08c0 msgr2=0x7f0fbc131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0fbc0a08c0 0x7f0fbc131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 1 -- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fbc0a2930 msgr2=0x7f0fbc141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fbc0a2930 0x7f0fbc141c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 10 monclient: _finish_hunting 0 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 1 monclient: found mon.noname-c 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 1 -- 172.21.15.178:0/498089113 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa4009390 con 0x7f0fbc09faa0 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fba7fc640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0fbc0a08c0 0x7f0fbc131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbb7fe640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fbc0a2930 0x7f0fbc141c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fbaffd640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc09faa0 0x7f0fbc09b110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0fac000c00 tx=0x7f0fac032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0fac031820 con 0x7f0fbc09faa0 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb2ffd640 10 monclient: dump: 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.718 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac0319a0 con 0x7f0fbc09faa0 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fc149f640 5 monclient: authenticate success, global_id 44689 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fc149f640 10 monclient: _renew_subs 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fc149f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:43.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fc149f640 1 -- 172.21.15.178:0/498089113 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb4009a20 con 0x7f0fbc09faa0 2024-09-15T01:45:43.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:43.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0fac03a390 con 0x7f0fbc09faa0 2024-09-15T01:45:43.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:43.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb2ffd640 10 monclient: dump: 2024-09-15T01:45:43.721 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.721 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.721 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.721 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.721 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.721 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fb97fa640 4 set_mon_vals no callback set 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fc149f640 10 monclient: _renew_subs 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fc149f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:45:43.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.694+0000 7f0fc149f640 1 -- 172.21.15.178:0/498089113 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fac009450 con 0x7f0fbc09faa0 2024-09-15T01:45:43.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0fac039040 con 0x7f0fbc09faa0 2024-09-15T01:45:43.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fb2ffd640 1 --2- 172.21.15.178:0/498089113 >> v2:172.21.15.204:6812/28210 conn(0x7f0f9c0396e0 0x7f0f9c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fc149f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:43.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fc149f640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc11dae0 0x7f0fbc142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fc149f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fc149f640 1 -- 172.21.15.178:0/498089113 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc04c4a0 con 0x7f0fbc11dae0 2024-09-15T01:45:43.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f0fac06b060 con 0x7f0fbc09faa0 2024-09-15T01:45:43.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fbb7fe640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc11dae0 0x7f0fbc142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fbb7fe640 10 monclient: get_auth_request con 0x7f0fbc11dae0 auth_method 0 2024-09-15T01:45:43.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fbb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fbb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.695+0000 7f0fba7fc640 1 --2- 172.21.15.178:0/498089113 >> v2:172.21.15.204:6812/28210 conn(0x7f0f9c0396e0 0x7f0f9c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fba7fc640 10 monclient: get_auth_request con 0x7f0f9c0396e0 auth_method 0 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_done global_id 25503 payload 824 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fbb7fe640 1 -- 172.21.15.178:0/498089113 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fbc05ed70 con 0x7f0fbc11dae0 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fbb7fe640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc11dae0 0x7f0fbc142190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc09b7c0 tx=0x7f0fb400b040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0fb4011de0 con 0x7f0fbc11dae0 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fb2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fb2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fb2ffd640 10 monclient: dump: 2024-09-15T01:45:43.724 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.725 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.726 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.726 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fba7fc640 1 --2- 172.21.15.178:0/498089113 >> v2:172.21.15.204:6812/28210 conn(0x7f0f9c0396e0 0x7f0f9c03bb90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009fd0 tx=0x7f0fa40090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f0fb400f040 con 0x7f0fbc11dae0 2024-09-15T01:45:43.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fb2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:43.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fb2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:43.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc11dae0 msgr2=0x7f0fbc142190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fb2ffd640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc11dae0 0x7f0fbc142190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc09b7c0 tx=0x7f0fb400b040 comp rx=0 tx=0).stop 2024-09-15T01:45:43.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc11dae0 msgr2=0x7f0fbc142190 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:43.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.696+0000 7f0fb2ffd640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc11dae0 0x7f0fbc142190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fc149f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:43.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fc149f640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc131e60 0x7f0fbc1426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fc149f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fc149f640 1 -- 172.21.15.178:0/498089113 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc04bc80 con 0x7f0fbc131e60 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fbb7fe640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc131e60 0x7f0fbc1426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fbb7fe640 10 monclient: get_auth_request con 0x7f0fbc131e60 auth_method 0 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fbb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fbb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fbb7fe640 10 monclient(hunting): handle_auth_done global_id 25512 payload 824 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fbb7fe640 1 -- 172.21.15.178:0/498089113 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0fbc11eee0 con 0x7f0fbc131e60 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fbb7fe640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc131e60 0x7f0fbc1426d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0fb401c000 tx=0x7f0fb401d000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0fb401da90 con 0x7f0fbc131e60 2024-09-15T01:45:43.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fb2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fb2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.702+0000 7f0fb2ffd640 10 monclient: dump: 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.730 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.731 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.731 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.731 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.731 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f0fb401dbf0 con 0x7f0fbc131e60 2024-09-15T01:45:43.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fb2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:43.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fb2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:43.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc131e60 msgr2=0x7f0fbc1426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fb2ffd640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc131e60 0x7f0fbc1426d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0fb401c000 tx=0x7f0fb401d000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fb2ffd640 1 -- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc131e60 msgr2=0x7f0fbc1426d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:43.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fb2ffd640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0fbc131e60 0x7f0fbc1426d0 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0fb401c000 tx=0x7f0fb401d000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.732 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:43.754 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:43.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fc149f640 1 -- 172.21.15.178:0/498089113 >> v2:172.21.15.204:6812/28210 conn(0x7f0f9c0396e0 msgr2=0x7f0f9c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fc149f640 1 --2- 172.21.15.178:0/498089113 >> v2:172.21.15.204:6812/28210 conn(0x7f0f9c0396e0 0x7f0f9c03bb90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009fd0 tx=0x7f0fa40090f0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fc149f640 10 monclient: shutdown 2024-09-15T01:45:43.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fc149f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:43.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fc149f640 1 -- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc09faa0 msgr2=0x7f0fbc09b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fc149f640 1 --2- 172.21.15.178:0/498089113 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f0fbc09faa0 0x7f0fbc09b110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0fac000c00 tx=0x7f0fac032000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fbb7fe640 1 -- 172.21.15.178:0/498089113 reap_dead start 2024-09-15T01:45:43.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fbb7fe640 1 -- 172.21.15.178:0/498089113 reap_dead start 2024-09-15T01:45:43.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fc149f640 1 -- 172.21.15.178:0/498089113 shutdown_connections 2024-09-15T01:45:43.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fc149f640 1 -- 172.21.15.178:0/498089113 >> 172.21.15.178:0/498089113 conn(0x7f0fbc0a5ed0 msgr2=0x7f0fbc0a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:43.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fc149f640 1 -- 172.21.15.178:0/498089113 shutdown_connections 2024-09-15T01:45:43.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.703+0000 7f0fc149f640 1 -- 172.21.15.178:0/498089113 wait complete. 2024-09-15T01:45:43.757 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:43.757 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:45:43.757 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:45:43.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.851+0000 7fd292f1d640 10 monclient: get_monmap_and_config 2024-09-15T01:45:43.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.851+0000 7fd292f1d640 10 monclient: build_initial_monmap 2024-09-15T01:45:43.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.851+0000 7fd292f1d640 10 monclient: monmap: 2024-09-15T01:45:43.853 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:43.851995+0000 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:43.851995+0000 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:43.854 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:43.855 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:43.855 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:43.855 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:43.855 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.851+0000 7fd292f1d640 1 Processor -- start 2024-09-15T01:45:43.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.851+0000 7fd292f1d640 1 -- start start 2024-09-15T01:45:43.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.851+0000 7fd292f1d640 10 monclient: init 2024-09-15T01:45:43.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:43.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 10 monclient: _add_conns ranks=[8,5,4,7,10,9,6,0,3,2,1] 2024-09-15T01:45:43.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 0x7fd28c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 10 monclient(hunting): picked mon.noname-e con 0x7fd28c0a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:43.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd28c09faa0 0x7fd28c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 10 monclient(hunting): picked mon.noname-h con 0x7fd28c09faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd28c0a08c0 0x7fd28c0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 10 monclient(hunting): picked mon.noname-c con 0x7fd28c0a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c09e0f0 con 0x7fd28c0a08c0 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c09dd80 con 0x7fd28c09faa0 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c0a2d00 con 0x7fd28c0a2930 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd292f1d640 10 monclient(hunting): authenticate will time out at 1548.602359s 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd290cb9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 0x7fd28c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd28c0a08c0 0x7fd28c0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd28c0a08c0 0x7fd28c0a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:35432/0 (socket says 172.21.15.178:35432) 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd290cb9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 0x7fd28c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60194/0 (socket says 172.21.15.178:60194) 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 1 -- 172.21.15.178:0/65425924 learned_addr learned my addr 172.21.15.178:0/65425924 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 10 monclient(hunting): get_auth_request con 0x7fd28c0a08c0 auth_method 0 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd290cb9640 10 monclient(hunting): get_auth_request con 0x7fd28c0a2930 auth_method 0 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd290cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd290cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd290cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd290cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd290cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd290cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd28bfff640 1 --2- 172.21.15.178:0/65425924 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd28c09faa0 0x7fd28c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd28bfff640 10 monclient(hunting): get_auth_request con 0x7fd28c09faa0 auth_method 0 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd28bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd28bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd28bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 10 monclient(hunting): handle_auth_done global_id 44698 payload 274 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 1 -- 172.21.15.178:0/65425924 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 msgr2=0x7fd28c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 1 --2- 172.21.15.178:0/65425924 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 0x7fd28c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 1 -- 172.21.15.178:0/65425924 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd28c09faa0 msgr2=0x7fd28c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 1 --2- 172.21.15.178:0/65425924 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd28c09faa0 0x7fd28c09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 10 monclient: _finish_hunting 0 2024-09-15T01:45:43.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 1 monclient: found mon.noname-c 2024-09-15T01:45:43.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:43.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:45:43.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd2914ba640 1 -- 172.21.15.178:0/65425924 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c04bf60 con 0x7fd28c0a08c0 2024-09-15T01:45:43.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd28bfff640 1 --2- 172.21.15.178:0/65425924 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd28c09faa0 0x7fd28c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:45:43.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.852+0000 7fd290cb9640 1 --2- 172.21.15.178:0/65425924 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 0x7fd28c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:43.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd2914ba640 1 --2- 172.21.15.178:0/65425924 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd28c0a08c0 0x7fd28c0a0d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009940 tx=0x7fd27c0091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=be0a2819a177b5a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd28b7fe640 1 -- 172.21.15.178:0/65425924 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd27c02ac60 con 0x7fd28c0a08c0 2024-09-15T01:45:43.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd28b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd28b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:45:43.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd28b7fe640 10 monclient: dump: 2024-09-15T01:45:43.868 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.869 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd28b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd28b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd292f1d640 5 monclient: authenticate success, global_id 44698 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd28b7fe640 1 -- 172.21.15.178:0/65425924 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c02ade0 con 0x7fd28c0a08c0 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd292f1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd28b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd292f1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd292f1d640 4 set_mon_vals no callback set 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd28b7fe640 1 -- 172.21.15.178:0/65425924 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd27c031ca0 con 0x7fd28c0a08c0 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd28b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd28b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.853+0000 7fd28b7fe640 10 monclient: dump: 2024-09-15T01:45:43.870 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.871 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 10 monclient: shutdown 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 1 -- 172.21.15.178:0/65425924 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd28c0a08c0 msgr2=0x7fd28c0a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 1 --2- 172.21.15.178:0/65425924 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd28c0a08c0 0x7fd28c0a0d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009940 tx=0x7fd27c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 1 -- 172.21.15.178:0/65425924 shutdown_connections 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 1 --2- 172.21.15.178:0/65425924 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd28c0a08c0 0x7fd28c0a0d30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 1 --2- 172.21.15.178:0/65425924 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd28c09faa0 0x7fd28c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 1 --2- 172.21.15.178:0/65425924 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 0x7fd28c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 1 -- 172.21.15.178:0/65425924 >> 172.21.15.178:0/65425924 conn(0x7fd28c0a5ed0 msgr2=0x7fd28c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 1 -- 172.21.15.178:0/65425924 shutdown_connections 2024-09-15T01:45:43.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 1 -- 172.21.15.178:0/65425924 wait complete. 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 10 monclient: build_initial_monmap 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.854+0000 7fd292f1d640 10 monclient: monmap: 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:43.873 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 1 Processor -- start 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 1 -- start start 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 10 monclient: init 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 10 monclient: _add_conns ranks=[2,6,8,4,3,7,0,5,1] 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd28c09faa0 0x7fd28c09b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 10 monclient(hunting): picked mon.noname-b con 0x7fd28c09faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd28c0a08c0 0x7fd28c131770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 10 monclient(hunting): picked mon.noname-d con 0x7fd28c0a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:43.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 0x7fd28c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 10 monclient(hunting): picked mon.noname-e con 0x7fd28c0a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c0a2390 con 0x7fd28c09faa0 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c09d630 con 0x7fd28c0a08c0 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c11eb70 con 0x7fd28c0a2930 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd292f1d640 10 monclient(hunting): authenticate will time out at 1548.605748s 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd2914ba640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 0x7fd28c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd290cb9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd28c09faa0 0x7fd28c09b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd28bfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd28c0a08c0 0x7fd28c131770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd2914ba640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 0x7fd28c11c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60208/0 (socket says 172.21.15.178:60208) 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd28bfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd28c0a08c0 0x7fd28c131770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57148/0 (socket says 172.21.15.178:57148) 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd2914ba640 1 -- 172.21.15.178:0/1874145739 learned_addr learned my addr 172.21.15.178:0/1874145739 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd290cb9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd28c09faa0 0x7fd28c09b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:56584/0 (socket says 172.21.15.178:56584) 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd290cb9640 10 monclient(hunting): get_auth_request con 0x7fd28c09faa0 auth_method 0 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd290cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd290cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd290cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd2914ba640 10 monclient(hunting): get_auth_request con 0x7fd28c0a2930 auth_method 0 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd2914ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd2914ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd2914ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd28bfff640 10 monclient(hunting): get_auth_request con 0x7fd28c0a08c0 auth_method 0 2024-09-15T01:45:43.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd28bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd28bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd28bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd290cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd290cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd290cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd2914ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd2914ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd2914ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd28bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.855+0000 7fd28bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd28bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd290cb9640 10 monclient(hunting): handle_auth_done global_id 35039 payload 824 2024-09-15T01:45:43.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd290cb9640 1 -- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 msgr2=0x7fd28c11c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd290cb9640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 0x7fd28c11c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd290cb9640 1 -- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd28c0a08c0 msgr2=0x7fd28c131770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd290cb9640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd28c0a08c0 0x7fd28c131770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:43.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd290cb9640 10 monclient: _finish_hunting 0 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd290cb9640 1 monclient: found mon.noname-b 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd290cb9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd290cb9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd290cb9640 1 -- 172.21.15.178:0/1874145739 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd284009a20 con 0x7fd28c09faa0 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd2914ba640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c0a2930 0x7fd28c11c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd28bfff640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fd28c0a08c0 0x7fd28c131770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd290cb9640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd28c09faa0 0x7fd28c09b150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd284009fd0 tx=0x7fd28400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd284009d00 con 0x7fd28c09faa0 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd288ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd288ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd288ff9640 10 monclient: dump: 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.881 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.882 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.882 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.882 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.882 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.882 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.882 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.882 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.882 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.882 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.883 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.883 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.883 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.883 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.883 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd288ff9640 10 monclient: _finish_auth 0 2024-09-15T01:45:43.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd288ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:43.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28400f040 con 0x7fd28c09faa0 2024-09-15T01:45:43.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd292f1d640 5 monclient: authenticate success, global_id 35039 2024-09-15T01:45:43.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd292f1d640 10 monclient: _renew_subs 2024-09-15T01:45:43.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd292f1d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:45:43.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd292f1d640 1 -- 172.21.15.178:0/1874145739 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c009450 con 0x7fd28c09faa0 2024-09-15T01:45:43.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd288ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:43.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd28400ce70 con 0x7fd28c09faa0 2024-09-15T01:45:43.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd288ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd288ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:45:43.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd288ff9640 10 monclient: dump: 2024-09-15T01:45:43.884 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.884 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.885 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd292f1d640 10 monclient: _renew_subs 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd292f1d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd292f1d640 1 -- 172.21.15.178:0/1874145739 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28c11e890 con 0x7fd28c09faa0 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.856+0000 7fd28affd640 4 set_mon_vals no callback set 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd284014480 con 0x7fd28c09faa0 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd292f1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd292f1d640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c131cb0 0x7fd28c11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd292f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd292f1d640 1 -- 172.21.15.178:0/1874145739 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c132080 con 0x7fd28c131cb0 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd28bfff640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c131cb0 0x7fd28c11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd28bfff640 10 monclient: get_auth_request con 0x7fd28c131cb0 auth_method 0 2024-09-15T01:45:43.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd28bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd28bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd28bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd28bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd28bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd28bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd288ff9640 1 --2- 172.21.15.178:0/1874145739 >> v2:172.21.15.204:6812/28210 conn(0x7fd2780397e0 0x7fd27803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd28404ca80 con 0x7fd28c09faa0 2024-09-15T01:45:43.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd28bfff640 10 monclient(hunting): handle_auth_done global_id 35063 payload 824 2024-09-15T01:45:43.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd28bfff640 1 -- 172.21.15.178:0/1874145739 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd28c04c4a0 con 0x7fd28c131cb0 2024-09-15T01:45:43.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd28bfff640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c131cb0 0x7fd28c11ca70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd28000bfd0 tx=0x7fd280008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd2914ba640 1 --2- 172.21.15.178:0/1874145739 >> v2:172.21.15.204:6812/28210 conn(0x7fd2780397e0 0x7fd27803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd2914ba640 10 monclient: get_auth_request con 0x7fd2780397e0 auth_method 0 2024-09-15T01:45:43.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd280010ec0 con 0x7fd28c131cb0 2024-09-15T01:45:43.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd288ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd288ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:43.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.857+0000 7fd288ff9640 10 monclient: dump: 2024-09-15T01:45:43.888 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.888 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.888 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.888 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.889 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.889 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.889 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.889 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.889 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.889 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.889 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.889 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.889 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.889 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.889 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.889 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.858+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fd280016480 con 0x7fd28c131cb0 2024-09-15T01:45:43.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.858+0000 7fd288ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:43.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.858+0000 7fd288ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:43.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.858+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c131cb0 msgr2=0x7fd28c11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.858+0000 7fd288ff9640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c131cb0 0x7fd28c11ca70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd28000bfd0 tx=0x7fd280008040 comp rx=0 tx=0).stop 2024-09-15T01:45:43.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.858+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c131cb0 msgr2=0x7fd28c11ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:43.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.858+0000 7fd288ff9640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c131cb0 0x7fd28c11ca70 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd28000bfd0 tx=0x7fd280008040 comp rx=0 tx=0).stop 2024-09-15T01:45:43.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.858+0000 7fd2914ba640 1 --2- 172.21.15.178:0/1874145739 >> v2:172.21.15.204:6812/28210 conn(0x7fd2780397e0 0x7fd27803bc90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009420 tx=0x7fd27c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.863+0000 7fd292f1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:43.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.863+0000 7fd292f1d640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c11cfb0 0x7fd28c096ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:43.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.863+0000 7fd292f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:43.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.863+0000 7fd292f1d640 1 -- 172.21.15.178:0/1874145739 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c04bc80 con 0x7fd28c11cfb0 2024-09-15T01:45:43.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.863+0000 7fd28bfff640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c11cfb0 0x7fd28c096ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:43.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd28bfff640 10 monclient: get_auth_request con 0x7fd28c11cfb0 auth_method 0 2024-09-15T01:45:43.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd28bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:43.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd28bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd28bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd28bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd28bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd28bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd28bfff640 10 monclient(hunting): handle_auth_done global_id 35072 payload 824 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd28bfff640 1 -- 172.21.15.178:0/1874145739 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd28c05ed70 con 0x7fd28c11cfb0 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd28bfff640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c11cfb0 0x7fd28c096ac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd28000bfd0 tx=0x7fd28001e000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd280020c60 con 0x7fd28c11cfb0 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd288ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd288ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd288ff9640 10 monclient: dump: 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:43.893 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:43.894 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.894 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:43.894 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:43.894 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:43.894 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:43.894 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:43.894 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:43.894 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:43.895 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:43.895 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:43.895 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:43.895 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:43.895 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:43.895 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:43.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7fd280020dc0 con 0x7fd28c11cfb0 2024-09-15T01:45:43.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd288ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:43.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd288ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:43.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c11cfb0 msgr2=0x7fd28c096ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd288ff9640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c11cfb0 0x7fd28c096ac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd28000bfd0 tx=0x7fd28001e000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd288ff9640 1 -- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c11cfb0 msgr2=0x7fd28c096ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:43.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.864+0000 7fd288ff9640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fd28c11cfb0 0x7fd28c096ac0 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd28000bfd0 tx=0x7fd28001e000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.896 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:43.896 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:43.896 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:43.896 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:43.896 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:43.897 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:43.897 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:43.897 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:43.897 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:43.897 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:43.897 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:43.897 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:43.897 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.898 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.899 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:43.900 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:43.901 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.902 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.903 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.904 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:43.905 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.906 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.907 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.908 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.909 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:43.910 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.912 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.914 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:43.915 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.917 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:43.918 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:43.919 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:43.919 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:43.919 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:43.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:43.919 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:43.919 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:43.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.865+0000 7fd292f1d640 1 -- 172.21.15.178:0/1874145739 >> v2:172.21.15.204:6812/28210 conn(0x7fd2780397e0 msgr2=0x7fd27803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.865+0000 7fd292f1d640 1 --2- 172.21.15.178:0/1874145739 >> v2:172.21.15.204:6812/28210 conn(0x7fd2780397e0 0x7fd27803bc90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009420 tx=0x7fd27c032000 comp rx=0 tx=0).stop 2024-09-15T01:45:43.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.865+0000 7fd292f1d640 10 monclient: shutdown 2024-09-15T01:45:43.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.865+0000 7fd292f1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:43.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.865+0000 7fd292f1d640 1 -- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd28c09faa0 msgr2=0x7fd28c09b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:43.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.865+0000 7fd292f1d640 1 --2- 172.21.15.178:0/1874145739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd28c09faa0 0x7fd28c09b150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd284009fd0 tx=0x7fd28400b040 comp rx=0 tx=0).stop 2024-09-15T01:45:43.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.865+0000 7fd2914ba640 1 -- 172.21.15.178:0/1874145739 reap_dead start 2024-09-15T01:45:43.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.865+0000 7fd2914ba640 1 -- 172.21.15.178:0/1874145739 reap_dead start 2024-09-15T01:45:43.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.865+0000 7fd292f1d640 1 -- 172.21.15.178:0/1874145739 shutdown_connections 2024-09-15T01:45:43.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.865+0000 7fd292f1d640 1 -- 172.21.15.178:0/1874145739 >> 172.21.15.178:0/1874145739 conn(0x7fd28c0a5ed0 msgr2=0x7fd28c0a5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:43.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.865+0000 7fd292f1d640 1 -- 172.21.15.178:0/1874145739 shutdown_connections 2024-09-15T01:45:43.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:43.865+0000 7fd292f1d640 1 -- 172.21.15.178:0/1874145739 wait complete. 2024-09-15T01:45:43.921 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:43.921 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:45:43.922 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:45:44.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.013+0000 7f7d3b0c9640 10 monclient: get_monmap_and_config 2024-09-15T01:45:44.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.013+0000 7f7d3b0c9640 10 monclient: build_initial_monmap 2024-09-15T01:45:44.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.013+0000 7f7d3b0c9640 10 monclient: monmap: 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:44.014148+0000 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:44.014148+0000 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:44.016 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:44.017 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:44.017 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:44.017 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:44.017 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:44.017 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.013+0000 7f7d3b0c9640 1 Processor -- start 2024-09-15T01:45:44.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 1 -- start start 2024-09-15T01:45:44.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 10 monclient: init 2024-09-15T01:45:44.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:44.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 10 monclient: _add_conns ranks=[5,8,4,0,7,10,6,1,9,2,3] 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7d340a2330 0x7f7d34132910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 10 monclient(hunting): picked mon.noname-h con 0x7f7d340a2330 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d3409f8e0 0x7f7d3409fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 10 monclient(hunting): picked mon.noname-e con 0x7f7d3409f8e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7d340a08c0 0x7f7d340a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 10 monclient(hunting): picked mon.noname-c con 0x7f7d340a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3409ddf0 con 0x7f7d340a08c0 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340a2700 con 0x7f7d340a2330 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340a1a60 con 0x7f7d3409f8e0 2024-09-15T01:45:44.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:44.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d3b0c9640 10 monclient(hunting): authenticate will time out at 1548.764528s 2024-09-15T01:45:44.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d3409f8e0 0x7f7d3409fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7d340a08c0 0x7f7d340a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7d340a08c0 0x7f7d340a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:35446/0 (socket says 172.21.15.178:35446) 2024-09-15T01:45:44.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d3409f8e0 0x7f7d3409fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60234/0 (socket says 172.21.15.178:60234) 2024-09-15T01:45:44.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 1 -- 172.21.15.178:0/1318040943 learned_addr learned my addr 172.21.15.178:0/1318040943 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:44.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 10 monclient(hunting): get_auth_request con 0x7f7d340a08c0 auth_method 0 2024-09-15T01:45:44.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d33fff640 10 monclient(hunting): get_auth_request con 0x7f7d3409f8e0 auth_method 0 2024-09-15T01:45:44.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d33fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d33fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d38e65640 1 --2- 172.21.15.178:0/1318040943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7d340a2330 0x7f7d34132910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d38e65640 10 monclient(hunting): get_auth_request con 0x7f7d340a2330 auth_method 0 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d38e65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d38e65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d38e65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 10 monclient(hunting): handle_auth_done global_id 44707 payload 274 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 1 -- 172.21.15.178:0/1318040943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d3409f8e0 msgr2=0x7f7d3409fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 1 --2- 172.21.15.178:0/1318040943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d3409f8e0 0x7f7d3409fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 1 -- 172.21.15.178:0/1318040943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7d340a2330 msgr2=0x7f7d34132910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 1 --2- 172.21.15.178:0/1318040943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7d340a2330 0x7f7d34132910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 10 monclient: _finish_hunting 0 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 1 monclient: found mon.noname-c 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d39666640 1 -- 172.21.15.178:0/1318040943 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3404bf60 con 0x7f7d340a08c0 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.014+0000 7f7d33fff640 1 --2- 172.21.15.178:0/1318040943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d3409f8e0 0x7f7d3409fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d39666640 1 --2- 172.21.15.178:0/1318040943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7d340a08c0 0x7f7d340a0d30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009f50 tx=0x7f7d2c009840 comp rx=0 tx=0).ready entity=mon.4 client_cookie=fe17cc3920664be5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d337fe640 1 -- 172.21.15.178:0/1318040943 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7d2c007880 con 0x7f7d340a08c0 2024-09-15T01:45:44.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d337fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d337fe640 10 monclient: dump: 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.031 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d337fe640 10 monclient: _finish_auth 0 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d337fe640 1 -- 172.21.15.178:0/1318040943 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c02d050 con 0x7f7d340a08c0 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d3b0c9640 5 monclient: authenticate success, global_id 44707 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d3b0c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d337fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d3b0c9640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d3b0c9640 4 set_mon_vals no callback set 2024-09-15T01:45:44.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d337fe640 1 -- 172.21.15.178:0/1318040943 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7d2c030360 con 0x7f7d340a08c0 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d337fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.015+0000 7f7d337fe640 10 monclient: dump: 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.033 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 10 monclient: shutdown 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/1318040943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7d340a08c0 msgr2=0x7f7d340a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 1 --2- 172.21.15.178:0/1318040943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7d340a08c0 0x7f7d340a0d30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009f50 tx=0x7f7d2c009840 comp rx=0 tx=0).stop 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/1318040943 shutdown_connections 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 1 --2- 172.21.15.178:0/1318040943 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7d340a08c0 0x7f7d340a0d30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 1 --2- 172.21.15.178:0/1318040943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d3409f8e0 0x7f7d3409fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 1 --2- 172.21.15.178:0/1318040943 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7d340a2330 0x7f7d34132910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/1318040943 >> 172.21.15.178:0/1318040943 conn(0x7f7d34063230 msgr2=0x7f7d34063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/1318040943 shutdown_connections 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/1318040943 wait complete. 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 10 monclient: build_initial_monmap 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.016+0000 7f7d3b0c9640 10 monclient: monmap: 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:44.035 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 1 Processor -- start 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 1 -- start start 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 10 monclient: init 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 10 monclient: _add_conns ranks=[7,5,8,4,0,3,6,2,1] 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7d3409f8e0 0x7f7d3412fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 10 monclient(hunting): picked mon.noname-i con 0x7f7d3409f8e0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:44.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7d340a08c0 0x7f7d3411c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 10 monclient(hunting): picked mon.noname-h con 0x7f7d340a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d340a2330 0x7f7d3411ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 10 monclient(hunting): picked mon.noname-e con 0x7f7d340a2330 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34120480 con 0x7f7d340a08c0 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7d341201a0 con 0x7f7d3409f8e0 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34142840 con 0x7f7d340a2330 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.017+0000 7f7d3b0c9640 10 monclient(hunting): authenticate will time out at 1548.767922s 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d340a2330 0x7f7d3411ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d340a2330 0x7f7d3411ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60246/0 (socket says 172.21.15.178:60246) 2024-09-15T01:45:44.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 1 -- 172.21.15.178:0/4011775390 learned_addr learned my addr 172.21.15.178:0/4011775390 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 10 monclient(hunting): get_auth_request con 0x7f7d340a2330 auth_method 0 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d38e65640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7d3409f8e0 0x7f7d3412fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d33fff640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7d340a08c0 0x7f7d3411c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d33fff640 10 monclient(hunting): get_auth_request con 0x7f7d340a08c0 auth_method 0 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d33fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d33fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d38e65640 10 monclient(hunting): get_auth_request con 0x7f7d3409f8e0 auth_method 0 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d38e65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d38e65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d38e65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d38e65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d38e65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d38e65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 10 monclient(hunting): handle_auth_done global_id 35090 payload 824 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 1 -- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7d3409f8e0 msgr2=0x7f7d3412fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f7d3409f8e0 0x7f7d3412fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 1 -- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7d340a08c0 msgr2=0x7f7d3411c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f7d340a08c0 0x7f7d3411c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 10 monclient: _finish_hunting 0 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 1 monclient: found mon.noname-e 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 1 -- 172.21.15.178:0/4011775390 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20009390 con 0x7f7d340a2330 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d39666640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d340a2330 0x7f7d3411ce60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009df0 tx=0x7f7d2c036000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7d2c03baa0 con 0x7f7d340a2330 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d30ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d30ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d30ff9640 10 monclient: dump: 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d30ff9640 10 monclient: _finish_auth 0 2024-09-15T01:45:44.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d30ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d3b0c9640 5 monclient: authenticate success, global_id 35090 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c03bc20 con 0x7f7d340a2330 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d30ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d3b0c9640 10 monclient: _renew_subs 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7d2c03bd90 con 0x7f7d340a2330 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d3b0c9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/4011775390 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c009ae0 con 0x7f7d340a2330 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d30ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d30ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d32ffd640 4 set_mon_vals no callback set 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d30ff9640 10 monclient: dump: 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.042 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d3b0c9640 10 monclient: _renew_subs 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d3b0c9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.018+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/4011775390 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d34133e80 con 0x7f7d340a2330 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7d2c038070 con 0x7f7d340a2330 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d3b0c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d3b0c9640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d341302b0 0x7f7d340298c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d3b0c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/4011775390 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34120060 con 0x7f7d341302b0 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d30ff9640 1 --2- 172.21.15.178:0/4011775390 >> v2:172.21.15.204:6812/28210 conn(0x7f7d180397b0 0x7f7d1803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d38e65640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d341302b0 0x7f7d340298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f7d2c06bf90 con 0x7f7d340a2330 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d38e65640 10 monclient: get_auth_request con 0x7f7d341302b0 auth_method 0 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d38e65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d38e65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d38e65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d38e65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d38e65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d38e65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d33fff640 1 --2- 172.21.15.178:0/4011775390 >> v2:172.21.15.204:6812/28210 conn(0x7f7d180397b0 0x7f7d1803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.019+0000 7f7d33fff640 10 monclient: get_auth_request con 0x7f7d180397b0 auth_method 0 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d38e65640 10 monclient(hunting): handle_auth_done global_id 54631 payload 824 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d38e65640 1 -- 172.21.15.178:0/4011775390 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d3404c4a0 con 0x7f7d341302b0 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d38e65640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d341302b0 0x7f7d340298c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d280108b0 tx=0x7f7d28008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d33fff640 1 --2- 172.21.15.178:0/4011775390 >> v2:172.21.15.204:6812/28210 conn(0x7f7d180397b0 0x7f7d1803bc60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7d34068fa0 tx=0x7f7d200090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7d28014430 con 0x7f7d341302b0 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d30ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d30ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:45:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d30ff9640 10 monclient: dump: 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.046 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.047 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f7d2800eb40 con 0x7f7d341302b0 2024-09-15T01:45:44.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d30ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:44.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d30ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:44.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d341302b0 msgr2=0x7f7d340298c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d30ff9640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d341302b0 0x7f7d340298c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d280108b0 tx=0x7f7d28008040 comp rx=0 tx=0).stop 2024-09-15T01:45:44.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d341302b0 msgr2=0x7f7d340298c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:44.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.020+0000 7f7d30ff9640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d341302b0 0x7f7d340298c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.026+0000 7f7d3b0c9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:44.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.026+0000 7f7d3b0c9640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d34029e00 0x7f7d3402a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.026+0000 7f7d3b0c9640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.026+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/4011775390 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3406f430 con 0x7f7d34029e00 2024-09-15T01:45:44.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.026+0000 7f7d38e65640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d34029e00 0x7f7d3402a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.026+0000 7f7d38e65640 10 monclient: get_auth_request con 0x7f7d34029e00 auth_method 0 2024-09-15T01:45:44.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.026+0000 7f7d38e65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.026+0000 7f7d38e65640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.026+0000 7f7d38e65640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.026+0000 7f7d38e65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.026+0000 7f7d38e65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.026+0000 7f7d38e65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d38e65640 10 monclient(hunting): handle_auth_done global_id 54640 payload 824 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d38e65640 1 -- 172.21.15.178:0/4011775390 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7d34120310 con 0x7f7d34029e00 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d38e65640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d34029e00 0x7f7d3402a200 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d280108b0 tx=0x7f7d2801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7d28027400 con 0x7f7d34029e00 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d30ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d30ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d30ff9640 10 monclient: dump: 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.050 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f7d28027560 con 0x7f7d34029e00 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d30ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d30ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:44.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d34029e00 msgr2=0x7f7d3402a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d30ff9640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d34029e00 0x7f7d3402a200 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d280108b0 tx=0x7f7d2801c000 comp rx=0 tx=0).stop 2024-09-15T01:45:44.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d30ff9640 1 -- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d34029e00 msgr2=0x7f7d3402a200 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:44.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.027+0000 7f7d30ff9640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d34029e00 0x7f7d3402a200 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.052 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:44.052 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:44.052 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:44.052 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:44.052 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:44.052 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:44.052 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:44.052 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:44.053 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:44.054 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:44.055 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:44.056 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:44.057 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.058 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:44.059 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:44.061 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.062 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:44.063 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.066 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.067 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:44.068 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.070 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.071 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:44.072 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:44.073 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:44.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.028+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/4011775390 >> v2:172.21.15.204:6812/28210 conn(0x7f7d180397b0 msgr2=0x7f7d1803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.028+0000 7f7d3b0c9640 1 --2- 172.21.15.178:0/4011775390 >> v2:172.21.15.204:6812/28210 conn(0x7f7d180397b0 0x7f7d1803bc60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7d34068fa0 tx=0x7f7d200090f0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.028+0000 7f7d3b0c9640 10 monclient: shutdown 2024-09-15T01:45:44.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.028+0000 7f7d3b0c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:44.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.028+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/4011775390 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d340a2330 msgr2=0x7f7d3411ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.028+0000 7f7d3b0c9640 1 --2- 172.21.15.178:0/4011775390 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f7d340a2330 0x7f7d3411ce60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009df0 tx=0x7f7d2c036000 comp rx=0 tx=0).stop 2024-09-15T01:45:44.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.028+0000 7f7d39666640 1 -- 172.21.15.178:0/4011775390 reap_dead start 2024-09-15T01:45:44.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.028+0000 7f7d39666640 1 -- 172.21.15.178:0/4011775390 reap_dead start 2024-09-15T01:45:44.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.028+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/4011775390 shutdown_connections 2024-09-15T01:45:44.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.028+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/4011775390 >> 172.21.15.178:0/4011775390 conn(0x7f7d34063230 msgr2=0x7f7d340a41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:44.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.028+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/4011775390 shutdown_connections 2024-09-15T01:45:44.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.028+0000 7f7d3b0c9640 1 -- 172.21.15.178:0/4011775390 wait complete. 2024-09-15T01:45:44.076 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:44.076 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:45:44.076 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:45:44.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.167+0000 7f63ab660640 10 monclient: get_monmap_and_config 2024-09-15T01:45:44.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.167+0000 7f63ab660640 10 monclient: build_initial_monmap 2024-09-15T01:45:44.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.167+0000 7f63ab660640 10 monclient: monmap: 2024-09-15T01:45:44.170 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:44.170 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.170 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:44.168518+0000 2024-09-15T01:45:44.170 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:44.168518+0000 2024-09-15T01:45:44.170 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:44.170 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.170 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 1 Processor -- start 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 1 -- start start 2024-09-15T01:45:44.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 10 monclient: init 2024-09-15T01:45:44.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:44.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 10 monclient: _add_conns ranks=[7,6,9,5,4,1,0,2,8,3,10] 2024-09-15T01:45:44.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f63a40a2930 0x7f63a4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 10 monclient(hunting): picked mon.noname-i con 0x7f63a40a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:44.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a409faa0 0x7f63a409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 10 monclient(hunting): picked mon.noname-d con 0x7f63a409faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:44.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f63a40a08c0 0x7f63a40a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 10 monclient(hunting): picked mon.noname-a con 0x7f63a40a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:45:44.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a409e0f0 con 0x7f63a40a08c0 2024-09-15T01:45:44.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f63a409dd80 con 0x7f63a409faa0 2024-09-15T01:45:44.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f63a40a2d00 con 0x7f63a40a2930 2024-09-15T01:45:44.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:44.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.168+0000 7f63ab660640 10 monclient(hunting): authenticate will time out at 1548.918924s 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a8bfb640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a409faa0 0x7f63a409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f63a40a08c0 0x7f63a40a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a8bfb640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a409faa0 0x7f63a409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57162/0 (socket says 172.21.15.178:57162) 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f63a40a08c0 0x7f63a40a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33634/0 (socket says 172.21.15.178:33634) 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a8bfb640 1 -- 172.21.15.178:0/656343470 learned_addr learned my addr 172.21.15.178:0/656343470 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a8bfb640 10 monclient(hunting): get_auth_request con 0x7f63a409faa0 auth_method 0 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a8bfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a8bfb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a8bfb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 10 monclient(hunting): get_auth_request con 0x7f63a40a08c0 auth_method 0 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a8bfb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a8bfb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a8bfb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a93fc640 1 --2- 172.21.15.178:0/656343470 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f63a40a2930 0x7f63a4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a93fc640 10 monclient(hunting): get_auth_request con 0x7f63a40a2930 auth_method 0 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a93fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a93fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a93fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 10 monclient(hunting): handle_auth_done global_id 64242 payload 274 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 1 -- 172.21.15.178:0/656343470 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f63a40a2930 msgr2=0x7f63a4132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 1 --2- 172.21.15.178:0/656343470 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f63a40a2930 0x7f63a4132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 1 -- 172.21.15.178:0/656343470 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a409faa0 msgr2=0x7f63a409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:44.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 1 --2- 172.21.15.178:0/656343470 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a409faa0 0x7f63a409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 10 monclient: _finish_hunting 0 2024-09-15T01:45:44.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 1 monclient: found mon.noname-a 2024-09-15T01:45:44.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:44.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:45:44.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a9bfd640 1 -- 172.21.15.178:0/656343470 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a404bf60 con 0x7f63a40a08c0 2024-09-15T01:45:44.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.169+0000 7f63a8bfb640 1 --2- 172.21.15.178:0/656343470 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a409faa0 0x7f63a409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:44.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63a9bfd640 1 --2- 172.21.15.178:0/656343470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63a40a08c0 0x7f63a40a0d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f639c009e60 tx=0x7f639c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb187f60de6ee77 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f639bfff640 1 -- 172.21.15.178:0/656343470 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f639c0079d0 con 0x7f63a40a08c0 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f639bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f639bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f639bfff640 10 monclient: dump: 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.185 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.186 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.186 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.186 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.186 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.186 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.186 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f639bfff640 10 monclient: _finish_auth 0 2024-09-15T01:45:44.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f639bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:44.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f639bfff640 1 -- 172.21.15.178:0/656343470 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c02d050 con 0x7f63a40a08c0 2024-09-15T01:45:44.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 5 monclient: authenticate success, global_id 64242 2024-09-15T01:45:44.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f639bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 4 set_mon_vals no callback set 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f639bfff640 1 -- 172.21.15.178:0/656343470 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f639c007bd0 con 0x7f63a40a08c0 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f639bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f639bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f639bfff640 10 monclient: dump: 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.187 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.188 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.188 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.188 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.188 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.188 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.188 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.188 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.188 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.188 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.188 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 10 monclient: shutdown 2024-09-15T01:45:44.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:44.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 1 -- 172.21.15.178:0/656343470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63a40a08c0 msgr2=0x7f63a40a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 1 --2- 172.21.15.178:0/656343470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63a40a08c0 0x7f63a40a0d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f639c009e60 tx=0x7f639c009780 comp rx=0 tx=0).stop 2024-09-15T01:45:44.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 1 -- 172.21.15.178:0/656343470 shutdown_connections 2024-09-15T01:45:44.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 1 --2- 172.21.15.178:0/656343470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63a40a08c0 0x7f63a40a0d30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 1 --2- 172.21.15.178:0/656343470 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a409faa0 0x7f63a409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 1 --2- 172.21.15.178:0/656343470 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f63a40a2930 0x7f63a4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 1 -- 172.21.15.178:0/656343470 >> 172.21.15.178:0/656343470 conn(0x7f63a40a5ed0 msgr2=0x7f63a40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:44.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 1 -- 172.21.15.178:0/656343470 shutdown_connections 2024-09-15T01:45:44.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.170+0000 7f63ab660640 1 -- 172.21.15.178:0/656343470 wait complete. 2024-09-15T01:45:44.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.171+0000 7f63ab660640 10 monclient: build_initial_monmap 2024-09-15T01:45:44.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.171+0000 7f63ab660640 10 monclient: monmap: 2024-09-15T01:45:44.190 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:44.190 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:44.190 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:44.190 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:44.190 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:44.190 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:44.190 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:44.190 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:44.191 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:44.191 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:44.191 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:44.191 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:44.191 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:44.191 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:44.191 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:44.191 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.171+0000 7f63ab660640 1 Processor -- start 2024-09-15T01:45:44.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.171+0000 7f63ab660640 1 -- start start 2024-09-15T01:45:44.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.171+0000 7f63ab660640 10 monclient: init 2024-09-15T01:45:44.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:44.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 10 monclient: _add_conns ranks=[2,8,6,4,7,3,0,1,5] 2024-09-15T01:45:44.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63a409faa0 0x7f63a409b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 10 monclient(hunting): picked mon.noname-b con 0x7f63a409faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:44.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f63a40a08c0 0x7f63a4131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 10 monclient(hunting): picked mon.noname-e con 0x7f63a40a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:44.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a40a2930 0x7f63a4141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 10 monclient(hunting): picked mon.noname-d con 0x7f63a40a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:44.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a409d6a0 con 0x7f63a409faa0 2024-09-15T01:45:44.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f63a411ea40 con 0x7f63a40a2930 2024-09-15T01:45:44.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f63a411ed20 con 0x7f63a40a08c0 2024-09-15T01:45:44.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:44.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63ab660640 10 monclient(hunting): authenticate will time out at 1548.922377s 2024-09-15T01:45:44.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a93fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63a409faa0 0x7f63a409b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a9bfd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a40a2930 0x7f63a4141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f63a40a08c0 0x7f63a4131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a93fc640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63a409faa0 0x7f63a409b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:56600/0 (socket says 172.21.15.178:56600) 2024-09-15T01:45:44.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a9bfd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a40a2930 0x7f63a4141c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57178/0 (socket says 172.21.15.178:57178) 2024-09-15T01:45:44.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a93fc640 1 -- 172.21.15.178:0/3454754614 learned_addr learned my addr 172.21.15.178:0/3454754614 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:44.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a93fc640 10 monclient(hunting): get_auth_request con 0x7f63a409faa0 auth_method 0 2024-09-15T01:45:44.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a93fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a93fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a93fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 10 monclient(hunting): get_auth_request con 0x7f63a40a08c0 auth_method 0 2024-09-15T01:45:44.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a9bfd640 10 monclient(hunting): get_auth_request con 0x7f63a40a2930 auth_method 0 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a9bfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a9bfd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a9bfd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a93fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a93fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a93fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a9bfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a9bfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a9bfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 10 monclient(hunting): handle_auth_done global_id 35099 payload 824 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 1 -- 172.21.15.178:0/3454754614 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a40a2930 msgr2=0x7f63a4141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a40a2930 0x7f63a4141c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 1 -- 172.21.15.178:0/3454754614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63a409faa0 msgr2=0x7f63a409b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63a409faa0 0x7f63a409b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 10 monclient: _finish_hunting 0 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 1 monclient: found mon.noname-e 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a8bfb640 1 -- 172.21.15.178:0/3454754614 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394009450 con 0x7f63a40a08c0 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a93fc640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63a409faa0 0x7f63a409b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.172+0000 7f63a9bfd640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f63a40a2930 0x7f63a4141c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63a8bfb640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f63a40a08c0 0x7f63a4131920 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f639400e950 tx=0x7f63940091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f63940095e0 con 0x7f63a40a08c0 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63997fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63997fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63997fa640 10 monclient: dump: 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.197 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63997fa640 10 monclient: _finish_auth 0 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63997fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639400cdc0 con 0x7f63a40a08c0 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63ab660640 5 monclient: authenticate success, global_id 35099 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63ab660640 10 monclient: _renew_subs 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63ab660640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63ab660640 1 -- 172.21.15.178:0/3454754614 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c009a20 con 0x7f63a40a08c0 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63997fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f63940138f0 con 0x7f63a40a08c0 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63997fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63997fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63997fa640 10 monclient: dump: 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.199 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63ab660640 10 monclient: _renew_subs 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f639b7fe640 4 set_mon_vals no callback set 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63ab660640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.173+0000 7f63ab660640 1 -- 172.21.15.178:0/3454754614 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a411ca40 con 0x7f63a40a08c0 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6394013470 con 0x7f63a40a08c0 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63ab660640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63ab660640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a411dae0 0x7f63a4142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63ab660640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63ab660640 1 -- 172.21.15.178:0/3454754614 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a411f1a0 con 0x7f63a411dae0 2024-09-15T01:45:44.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63997fa640 1 --2- 172.21.15.178:0/3454754614 >> v2:172.21.15.204:6812/28210 conn(0x7f63880397b0 0x7f638803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f639404bdf0 con 0x7f63a40a08c0 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63a93fc640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a411dae0 0x7f63a4142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63a93fc640 10 monclient: get_auth_request con 0x7f63a411dae0 auth_method 0 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63a93fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63a93fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63a93fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63a9bfd640 1 --2- 172.21.15.178:0/3454754614 >> v2:172.21.15.204:6812/28210 conn(0x7f63880397b0 0x7f638803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63a9bfd640 10 monclient: get_auth_request con 0x7f63880397b0 auth_method 0 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63a93fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63a93fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63a93fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63a93fc640 10 monclient(hunting): handle_auth_done global_id 64182 payload 824 2024-09-15T01:45:44.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.174+0000 7f63a93fc640 1 -- 172.21.15.178:0/3454754614 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f63a404c4a0 con 0x7f63a411dae0 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63a93fc640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a411dae0 0x7f63a4142190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63a0010ce0 tx=0x7f63a0008040 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63a9bfd640 1 --2- 172.21.15.178:0/3454754614 >> v2:172.21.15.204:6812/28210 conn(0x7f63880397b0 0x7f638803bc60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f639c034000 tx=0x7f639c037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f63a00148b0 con 0x7f63a411dae0 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63997fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63997fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63997fa640 10 monclient: dump: 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.202 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f63a0014430 con 0x7f63a411dae0 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63997fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63997fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a411dae0 msgr2=0x7f63a4142190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63997fa640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a411dae0 0x7f63a4142190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63a0010ce0 tx=0x7f63a0008040 comp rx=0 tx=0).stop 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a411dae0 msgr2=0x7f63a4142190 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:44.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.175+0000 7f63997fa640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a411dae0 0x7f63a4142190 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63a0010ce0 tx=0x7f63a0008040 comp rx=0 tx=0).stop 2024-09-15T01:45:44.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63ab660640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:44.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63ab660640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a4131e60 0x7f63a41426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63ab660640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63ab660640 1 -- 172.21.15.178:0/3454754614 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a411f310 con 0x7f63a4131e60 2024-09-15T01:45:44.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63a93fc640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a4131e60 0x7f63a41426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63a93fc640 10 monclient: get_auth_request con 0x7f63a4131e60 auth_method 0 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63a93fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63a93fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63a93fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63a93fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63a93fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63a93fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63a93fc640 10 monclient(hunting): handle_auth_done global_id 64191 payload 824 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.181+0000 7f63a93fc640 1 -- 172.21.15.178:0/3454754614 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f63a404bc80 con 0x7f63a4131e60 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63a93fc640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a4131e60 0x7f63a41426d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63a0010ce0 tx=0x7f63a001c000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f63a001cc80 con 0x7f63a4131e60 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63997fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63997fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:45:44.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63997fa640 10 monclient: dump: 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.207 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.208 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.208 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.208 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f63a00133d0 con 0x7f63a4131e60 2024-09-15T01:45:44.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63997fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:44.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63997fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:44.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a4131e60 msgr2=0x7f63a41426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63997fa640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a4131e60 0x7f63a41426d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63a0010ce0 tx=0x7f63a001c000 comp rx=0 tx=0).stop 2024-09-15T01:45:44.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63997fa640 1 -- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a4131e60 msgr2=0x7f63a41426d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:44.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63997fa640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a4131e60 0x7f63a41426d0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63a0010ce0 tx=0x7f63a001c000 comp rx=0 tx=0).stop 2024-09-15T01:45:44.208 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:44.209 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 6, 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:44.210 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:44.211 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:44.212 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:44.213 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.214 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.215 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.216 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.217 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:44.218 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.219 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.220 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:44.221 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.222 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:44.223 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.224 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.225 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.226 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.227 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:44.228 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.229 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:44.230 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.230 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:44.230 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:44.230 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:44.230 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.230 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.230 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.230 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:44.230 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:44.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63ab660640 1 -- 172.21.15.178:0/3454754614 >> v2:172.21.15.204:6812/28210 conn(0x7f63880397b0 msgr2=0x7f638803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63ab660640 1 --2- 172.21.15.178:0/3454754614 >> v2:172.21.15.204:6812/28210 conn(0x7f63880397b0 0x7f638803bc60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f639c034000 tx=0x7f639c037000 comp rx=0 tx=0).stop 2024-09-15T01:45:44.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63ab660640 10 monclient: shutdown 2024-09-15T01:45:44.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63ab660640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:44.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63ab660640 1 -- 172.21.15.178:0/3454754614 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f63a40a08c0 msgr2=0x7f63a4131920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63ab660640 1 --2- 172.21.15.178:0/3454754614 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f63a40a08c0 0x7f63a4131920 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f639400e950 tx=0x7f63940091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63a9bfd640 1 -- 172.21.15.178:0/3454754614 reap_dead start 2024-09-15T01:45:44.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63a9bfd640 1 -- 172.21.15.178:0/3454754614 reap_dead start 2024-09-15T01:45:44.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63ab660640 1 -- 172.21.15.178:0/3454754614 shutdown_connections 2024-09-15T01:45:44.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.182+0000 7f63ab660640 1 -- 172.21.15.178:0/3454754614 >> 172.21.15.178:0/3454754614 conn(0x7f63a40a5ed0 msgr2=0x7f63a4135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:44.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.183+0000 7f63ab660640 1 -- 172.21.15.178:0/3454754614 shutdown_connections 2024-09-15T01:45:44.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.183+0000 7f63ab660640 1 -- 172.21.15.178:0/3454754614 wait complete. 2024-09-15T01:45:44.233 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:44.233 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:45:44.233 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:45:44.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.324+0000 7f6936b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:45:44.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.324+0000 7f6936b9d640 10 monclient: build_initial_monmap 2024-09-15T01:45:44.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.325+0000 7f6936b9d640 10 monclient: monmap: 2024-09-15T01:45:44.327 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:44.327 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.327 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:44.325891+0000 2024-09-15T01:45:44.327 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:44.325891+0000 2024-09-15T01:45:44.327 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:44.327 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.327 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:44.327 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:44.328 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:44.328 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:44.328 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:44.328 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:44.328 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:44.328 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:44.328 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:44.328 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:44.328 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:44.328 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.325+0000 7f6936b9d640 1 Processor -- start 2024-09-15T01:45:44.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.325+0000 7f6936b9d640 1 -- start start 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.325+0000 7f6936b9d640 10 monclient: init 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 10 monclient: _add_conns ranks=[3,8,5,7,2,6,9,1,10,4,0] 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69300a2930 0x7f6930132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 10 monclient(hunting): picked mon.noname-g con 0x7f69300a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 0x7f693009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 10 monclient(hunting): picked mon.noname-e con 0x7f693009faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f69300a08c0 0x7f69300a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 10 monclient(hunting): picked mon.noname-h con 0x7f69300a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693009e0f0 con 0x7f69300a2930 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f693009dd80 con 0x7f69300a08c0 2024-09-15T01:45:44.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f69300a2d00 con 0x7f693009faa0 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6936b9d640 10 monclient(hunting): authenticate will time out at 1549.076248s 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 0x7f693009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 0x7f693009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60268/0 (socket says 172.21.15.178:60268) 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 1 -- 172.21.15.178:0/439946540 learned_addr learned my addr 172.21.15.178:0/439946540 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 10 monclient(hunting): get_auth_request con 0x7f693009faa0 auth_method 0 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f693513a640 1 --2- 172.21.15.178:0/439946540 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f69300a08c0 0x7f69300a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6934939640 1 --2- 172.21.15.178:0/439946540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69300a2930 0x7f6930132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6934939640 10 monclient(hunting): get_auth_request con 0x7f69300a2930 auth_method 0 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6934939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6934939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6934939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f693513a640 10 monclient(hunting): get_auth_request con 0x7f69300a08c0 auth_method 0 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f693513a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f693513a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f693513a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 10 monclient(hunting): handle_auth_done global_id 35108 payload 274 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 1 -- 172.21.15.178:0/439946540 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f69300a08c0 msgr2=0x7f69300a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:44.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 1 --2- 172.21.15.178:0/439946540 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f69300a08c0 0x7f69300a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 1 -- 172.21.15.178:0/439946540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69300a2930 msgr2=0x7f6930132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 1 --2- 172.21.15.178:0/439946540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69300a2930 0x7f6930132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 10 monclient: _finish_hunting 0 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 1 monclient: found mon.noname-e 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6927fff640 1 -- 172.21.15.178:0/439946540 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693004bf60 con 0x7f693009faa0 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f693513a640 1 --2- 172.21.15.178:0/439946540 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f69300a08c0 0x7f69300a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.326+0000 7f6934939640 1 --2- 172.21.15.178:0/439946540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69300a2930 0x7f6930132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f6927fff640 1 --2- 172.21.15.178:0/439946540 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 0x7f693009fe70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6920009940 tx=0x7f69200091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=4ee3f17972765124 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f69277fe640 1 -- 172.21.15.178:0/439946540 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f692002ac60 con 0x7f693009faa0 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f69277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f69277fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:45:44.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f69277fe640 10 monclient: dump: 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f69277fe640 10 monclient: _finish_auth 0 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f69277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f69277fe640 1 -- 172.21.15.178:0/439946540 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692002ade0 con 0x7f693009faa0 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f6936b9d640 5 monclient: authenticate success, global_id 35108 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f6936b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f69277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f69277fe640 1 -- 172.21.15.178:0/439946540 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6920031ca0 con 0x7f693009faa0 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f6936b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f6936b9d640 4 set_mon_vals no callback set 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f69277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f69277fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.327+0000 7f69277fe640 10 monclient: dump: 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 10 monclient: shutdown 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 1 -- 172.21.15.178:0/439946540 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 msgr2=0x7f693009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 1 --2- 172.21.15.178:0/439946540 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 0x7f693009fe70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6920009940 tx=0x7f69200091d0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 1 -- 172.21.15.178:0/439946540 shutdown_connections 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 1 --2- 172.21.15.178:0/439946540 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f69300a08c0 0x7f69300a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 1 --2- 172.21.15.178:0/439946540 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 0x7f693009fe70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 1 --2- 172.21.15.178:0/439946540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69300a2930 0x7f6930132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 1 -- 172.21.15.178:0/439946540 >> 172.21.15.178:0/439946540 conn(0x7f69300a5ed0 msgr2=0x7f69300a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 1 -- 172.21.15.178:0/439946540 shutdown_connections 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 1 -- 172.21.15.178:0/439946540 wait complete. 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 10 monclient: build_initial_monmap 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 10 monclient: monmap: 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.328+0000 7f6936b9d640 1 Processor -- start 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 1 -- start start 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 10 monclient: init 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 10 monclient: _add_conns ranks=[8,2,5,0,4,7,6,1,3] 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 0x7f693009b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 10 monclient(hunting): picked mon.noname-e con 0x7f693009faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69300a08c0 0x7f69301318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 10 monclient(hunting): picked mon.noname-b con 0x7f69300a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f69300a2930 0x7f693011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 10 monclient(hunting): picked mon.noname-h con 0x7f69300a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693011ecb0 con 0x7f69300a08c0 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f693009bd80 con 0x7f69300a2930 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f693009d630 con 0x7f693009faa0 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6936b9d640 10 monclient(hunting): authenticate will time out at 1549.079552s 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 0x7f693009b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6927fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69300a08c0 0x7f69301318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 0x7f693009b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60280/0 (socket says 172.21.15.178:60280) 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 1 -- 172.21.15.178:0/2753977894 learned_addr learned my addr 172.21.15.178:0/2753977894 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 10 monclient(hunting): get_auth_request con 0x7f693009faa0 auth_method 0 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6927fff640 10 monclient(hunting): get_auth_request con 0x7f69300a08c0 auth_method 0 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6927fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6927fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6927fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6927fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6927fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6927fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f693513a640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f69300a2930 0x7f693011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 10 monclient(hunting): handle_auth_done global_id 35117 payload 824 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 1 -- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f69300a2930 msgr2=0x7f693011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f69300a2930 0x7f693011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 1 -- 172.21.15.178:0/2753977894 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69300a08c0 msgr2=0x7f69301318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69300a08c0 0x7f69301318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 10 monclient: _finish_hunting 0 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 1 monclient: found mon.noname-e 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f6934939640 1 -- 172.21.15.178:0/2753977894 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6920009450 con 0x7f693009faa0 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f693513a640 10 monclient: get_auth_request con 0x7f69300a2930 auth_method 0 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.329+0000 7f693513a640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f69300a2930 0x7f693011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:45:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6934939640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 0x7f693009b1e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f692c00d150 tx=0x7f692c009a10 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f692c014820 con 0x7f693009faa0 2024-09-15T01:45:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6924ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6924ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:45:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6924ff9640 10 monclient: dump: 2024-09-15T01:45:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6924ff9640 10 monclient: _finish_auth 0 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6924ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692c00f040 con 0x7f693009faa0 2024-09-15T01:45:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6936b9d640 5 monclient: authenticate success, global_id 35117 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6936b9d640 10 monclient: _renew_subs 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6936b9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6936b9d640 1 -- 172.21.15.178:0/2753977894 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69301422c0 con 0x7f693009faa0 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6924ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f692c014e00 con 0x7f693009faa0 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6924ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6924ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6924ff9640 10 monclient: dump: 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6936b9d640 10 monclient: _renew_subs 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6936b9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6936b9d640 1 -- 172.21.15.178:0/2753977894 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6930142bb0 con 0x7f693009faa0 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.330+0000 7f6926ffd640 4 set_mon_vals no callback set 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f692c014980 con 0x7f693009faa0 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f6936b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f6924ff9640 1 --2- 172.21.15.178:0/2753977894 >> v2:172.21.15.204:6812/28210 conn(0x7f691c0397b0 0x7f691c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f6936b9d640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f693011da70 0x7f693011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f6936b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f6936b9d640 1 -- 172.21.15.178:0/2753977894 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f693011eb70 con 0x7f693011da70 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f692c04be80 con 0x7f693009faa0 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f693513a640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f693011da70 0x7f693011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f693513a640 10 monclient: get_auth_request con 0x7f693011da70 auth_method 0 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f693513a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f693513a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f693513a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f6927fff640 1 --2- 172.21.15.178:0/2753977894 >> v2:172.21.15.204:6812/28210 conn(0x7f691c0397b0 0x7f691c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f6927fff640 10 monclient: get_auth_request con 0x7f691c0397b0 auth_method 0 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f693513a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f693513a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f693513a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f693513a640 10 monclient(hunting): handle_auth_done global_id 7475 payload 824 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f693513a640 1 -- 172.21.15.178:0/2753977894 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f693004c4a0 con 0x7f693011da70 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.331+0000 7f6927fff640 1 --2- 172.21.15.178:0/2753977894 >> v2:172.21.15.204:6812/28210 conn(0x7f691c0397b0 0x7f691c03bc60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6920034000 tx=0x7f6920032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.332+0000 7f693513a640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f693011da70 0x7f693011ca70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f69300a3290 tx=0x7f6928008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.332+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f692800c900 con 0x7f693011da70 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.332+0000 7f6924ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.332+0000 7f6924ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:45:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.332+0000 7f6924ff9640 10 monclient: dump: 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.357 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.358 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.358 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.332+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f692800c480 con 0x7f693011da70 2024-09-15T01:45:44.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.332+0000 7f6924ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:44.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.332+0000 7f6924ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:44.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.332+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f693011da70 msgr2=0x7f693011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.332+0000 7f6924ff9640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f693011da70 0x7f693011ca70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f69300a3290 tx=0x7f6928008040 comp rx=0 tx=0).stop 2024-09-15T01:45:44.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.332+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f693011da70 msgr2=0x7f693011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:44.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.332+0000 7f6924ff9640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f693011da70 0x7f693011ca70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f6936b9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:44.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f6936b9d640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6930131df0 0x7f693011cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f6936b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f6936b9d640 1 -- 172.21.15.178:0/2753977894 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6930132670 con 0x7f6930131df0 2024-09-15T01:45:44.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f693513a640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6930131df0 0x7f693011cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f693513a640 10 monclient: get_auth_request con 0x7f6930131df0 auth_method 0 2024-09-15T01:45:44.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f693513a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f693513a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f693513a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f693513a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f693513a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f693513a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f693513a640 10 monclient(hunting): handle_auth_done global_id 7484 payload 824 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f693513a640 1 -- 172.21.15.178:0/2753977894 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f693011ee20 con 0x7f6930131df0 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.338+0000 7f693513a640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6930131df0 0x7f693011cfb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f69300a3290 tx=0x7f692801c000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6928021070 con 0x7f6930131df0 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6924ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6924ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6924ff9640 10 monclient: dump: 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.361 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.362 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9314 (secure 0 0 0) 0x7f692800c780 con 0x7f6930131df0 2024-09-15T01:45:44.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6924ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:44.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6924ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:44.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6930131df0 msgr2=0x7f693011cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6924ff9640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6930131df0 0x7f693011cfb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f69300a3290 tx=0x7f692801c000 comp rx=0 tx=0).stop 2024-09-15T01:45:44.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6924ff9640 1 -- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6930131df0 msgr2=0x7f693011cfb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:44.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6924ff9640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6930131df0 0x7f693011cfb0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.363 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:44.363 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:44.363 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:44.364 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 7, 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:44.365 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:44.366 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:44.367 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:44.368 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:44.369 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.370 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.371 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:44.372 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.373 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:44.374 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.375 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.376 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:44.377 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.378 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.379 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.379 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:44.379 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:44.379 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.379 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.379 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.379 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.379 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.379 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.380 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.381 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:44.382 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.383 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.384 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.384 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:44.384 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.384 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.384 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.384 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.384 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:44.384 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:44.384 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.384 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.384 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.384 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:44.385 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 6 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:44.386 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:44.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6936b9d640 1 -- 172.21.15.178:0/2753977894 >> v2:172.21.15.204:6812/28210 conn(0x7f691c0397b0 msgr2=0x7f691c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6936b9d640 1 --2- 172.21.15.178:0/2753977894 >> v2:172.21.15.204:6812/28210 conn(0x7f691c0397b0 0x7f691c03bc60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6920034000 tx=0x7f6920032000 comp rx=0 tx=0).stop 2024-09-15T01:45:44.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6936b9d640 10 monclient: shutdown 2024-09-15T01:45:44.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6936b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:44.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6936b9d640 1 -- 172.21.15.178:0/2753977894 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 msgr2=0x7f693009b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6936b9d640 1 --2- 172.21.15.178:0/2753977894 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f693009faa0 0x7f693009b1e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f692c00d150 tx=0x7f692c009a10 comp rx=0 tx=0).stop 2024-09-15T01:45:44.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f693513a640 1 -- 172.21.15.178:0/2753977894 reap_dead start 2024-09-15T01:45:44.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f693513a640 1 -- 172.21.15.178:0/2753977894 reap_dead start 2024-09-15T01:45:44.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6936b9d640 1 -- 172.21.15.178:0/2753977894 shutdown_connections 2024-09-15T01:45:44.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6936b9d640 1 -- 172.21.15.178:0/2753977894 >> 172.21.15.178:0/2753977894 conn(0x7f69300a5ed0 msgr2=0x7f693009e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:44.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6936b9d640 1 -- 172.21.15.178:0/2753977894 shutdown_connections 2024-09-15T01:45:44.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.339+0000 7f6936b9d640 1 -- 172.21.15.178:0/2753977894 wait complete. 2024-09-15T01:45:44.389 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:44.389 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:45:44.389 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:45:44.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.481+0000 7f5ab90dd640 10 monclient: get_monmap_and_config 2024-09-15T01:45:44.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.481+0000 7f5ab90dd640 10 monclient: build_initial_monmap 2024-09-15T01:45:44.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.481+0000 7f5ab90dd640 10 monclient: monmap: 2024-09-15T01:45:44.483 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:44.481919+0000 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:44.481919+0000 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:44.484 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:44.485 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:44.485 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:44.485 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:44.485 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.481+0000 7f5ab90dd640 1 Processor -- start 2024-09-15T01:45:44.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.481+0000 7f5ab90dd640 1 -- start start 2024-09-15T01:45:44.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.481+0000 7f5ab90dd640 10 monclient: init 2024-09-15T01:45:44.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:44.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 10 monclient: _add_conns ranks=[3,7,6,0,1,8,10,4,9,2,5] 2024-09-15T01:45:44.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ab40a2930 0x7f5ab4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 10 monclient(hunting): picked mon.noname-g con 0x7f5ab40a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:45:44.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab409faa0 0x7f5ab409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 10 monclient(hunting): picked mon.noname-i con 0x7f5ab409faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ab40a08c0 0x7f5ab40a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 10 monclient(hunting): picked mon.noname-d con 0x7f5ab40a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab409e0f0 con 0x7f5ab40a2930 2024-09-15T01:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab409dd80 con 0x7f5ab40a08c0 2024-09-15T01:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40a2d00 con 0x7f5ab409faa0 2024-09-15T01:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab90dd640 10 monclient(hunting): authenticate will time out at 1549.232189s 2024-09-15T01:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ab40a08c0 0x7f5ab40a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ab40a08c0 0x7f5ab40a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57188/0 (socket says 172.21.15.178:57188) 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 1 -- 172.21.15.178:0/2834821919 learned_addr learned my addr 172.21.15.178:0/2834821919 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 10 monclient(hunting): get_auth_request con 0x7f5ab40a08c0 auth_method 0 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab259c640 1 --2- 172.21.15.178:0/2834821919 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab409faa0 0x7f5ab409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab2d9d640 1 --2- 172.21.15.178:0/2834821919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ab40a2930 0x7f5ab4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab2d9d640 10 monclient(hunting): get_auth_request con 0x7f5ab40a2930 auth_method 0 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab259c640 10 monclient(hunting): get_auth_request con 0x7f5ab409faa0 auth_method 0 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 10 monclient(hunting): handle_auth_done global_id 25548 payload 274 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 1 -- 172.21.15.178:0/2834821919 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab409faa0 msgr2=0x7f5ab409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 1 --2- 172.21.15.178:0/2834821919 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab409faa0 0x7f5ab409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 1 -- 172.21.15.178:0/2834821919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ab40a2930 msgr2=0x7f5ab4132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 1 --2- 172.21.15.178:0/2834821919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ab40a2930 0x7f5ab4132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 10 monclient: _finish_hunting 0 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 1 monclient: found mon.noname-d 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:44.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.482+0000 7f5ab359e640 1 -- 172.21.15.178:0/2834821919 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab404bf60 con 0x7f5ab40a08c0 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab359e640 1 --2- 172.21.15.178:0/2834821919 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ab40a08c0 0x7f5ab40a0d30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009f50 tx=0x7f5aac009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=433e1774b5f4b0bf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab1d9b640 1 -- 172.21.15.178:0/2834821919 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5aac007880 con 0x7f5ab40a08c0 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab1d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab1d9b640 10 monclient: dump: 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.500 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab1d9b640 10 monclient: _finish_auth 0 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab1d9b640 1 -- 172.21.15.178:0/2834821919 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac02d050 con 0x7f5ab40a08c0 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 5 monclient: authenticate success, global_id 25548 2024-09-15T01:45:44.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab1d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab1d9b640 1 -- 172.21.15.178:0/2834821919 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5aac030360 con 0x7f5ab40a08c0 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 4 set_mon_vals no callback set 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab1d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab1d9b640 10 monclient: dump: 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.502 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 10 monclient: shutdown 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2834821919 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ab40a08c0 msgr2=0x7f5ab40a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 1 --2- 172.21.15.178:0/2834821919 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ab40a08c0 0x7f5ab40a0d30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009f50 tx=0x7f5aac009840 comp rx=0 tx=0).stop 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2834821919 shutdown_connections 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 1 --2- 172.21.15.178:0/2834821919 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f5ab40a08c0 0x7f5ab40a0d30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 1 --2- 172.21.15.178:0/2834821919 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab409faa0 0x7f5ab409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 1 --2- 172.21.15.178:0/2834821919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ab40a2930 0x7f5ab4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.483+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2834821919 >> 172.21.15.178:0/2834821919 conn(0x7f5ab40a5ed0 msgr2=0x7f5ab40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.484+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2834821919 shutdown_connections 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.484+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2834821919 wait complete. 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.484+0000 7f5ab90dd640 10 monclient: build_initial_monmap 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.484+0000 7f5ab90dd640 10 monclient: monmap: 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:44.504 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.484+0000 7f5ab90dd640 1 Processor -- start 2024-09-15T01:45:44.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.484+0000 7f5ab90dd640 1 -- start start 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.484+0000 7f5ab90dd640 10 monclient: init 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 10 monclient: _add_conns ranks=[8,5,3,6,1,7,0,4,2] 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ab409faa0 0x7f5ab409b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 10 monclient(hunting): picked mon.noname-e con 0x7f5ab409faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5ab40a08c0 0x7f5ab4131b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 10 monclient(hunting): picked mon.noname-h con 0x7f5ab40a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ab40a2930 0x7f5ab41320a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 10 monclient(hunting): picked mon.noname-g con 0x7f5ab40a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab409c100 con 0x7f5ab40a2930 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40a2390 con 0x7f5ab40a08c0 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab409d630 con 0x7f5ab409faa0 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab90dd640 10 monclient(hunting): authenticate will time out at 1549.235422s 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ab409faa0 0x7f5ab409b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ab409faa0 0x7f5ab409b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:60290/0 (socket says 172.21.15.178:60290) 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 1 -- 172.21.15.178:0/2538970210 learned_addr learned my addr 172.21.15.178:0/2538970210 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 10 monclient(hunting): get_auth_request con 0x7f5ab409faa0 auth_method 0 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab259c640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5ab40a08c0 0x7f5ab4131b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab359e640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ab40a2930 0x7f5ab41320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab259c640 10 monclient(hunting): get_auth_request con 0x7f5ab40a08c0 auth_method 0 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab359e640 10 monclient(hunting): get_auth_request con 0x7f5ab40a2930 auth_method 0 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab359e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 10 monclient(hunting): handle_auth_done global_id 35126 payload 824 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 1 -- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5ab40a08c0 msgr2=0x7f5ab4131b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f5ab40a08c0 0x7f5ab4131b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 1 -- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ab40a2930 msgr2=0x7f5ab41320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ab40a2930 0x7f5ab41320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 1 monclient: found mon.noname-e 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:44.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.485+0000 7f5ab2d9d640 1 -- 172.21.15.178:0/2538970210 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac009ae0 con 0x7f5ab409faa0 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5ab2d9d640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ab409faa0 0x7f5ab409b670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400d8c0 tx=0x7f5aa4008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5aa400de50 con 0x7f5ab409faa0 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5a9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5a9affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5a9affd640 10 monclient: dump: 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.510 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5a9affd640 10 monclient: _finish_auth 0 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5a9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa400bd50 con 0x7f5ab409faa0 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5ab90dd640 5 monclient: authenticate success, global_id 35126 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5ab90dd640 10 monclient: _renew_subs 2024-09-15T01:45:44.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5ab90dd640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2538970210 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab4142f80 con 0x7f5ab409faa0 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5a9affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5aa40148b0 con 0x7f5ab409faa0 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5a9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5a9affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5a9affd640 10 monclient: dump: 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.512 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5ab90dd640 10 monclient: _renew_subs 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5ab159a640 4 set_mon_vals no callback set 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5ab90dd640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.486+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2538970210 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab411ef60 con 0x7f5ab409faa0 2024-09-15T01:45:44.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab90dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5aa4014430 con 0x7f5ab409faa0 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab90dd640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab4142520 0x7f5ab411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab90dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2538970210 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab411ee20 con 0x7f5ab4142520 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab259c640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab4142520 0x7f5ab411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab259c640 10 monclient: get_auth_request con 0x7f5ab4142520 auth_method 0 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5a9affd640 1 --2- 172.21.15.178:0/2538970210 >> v2:172.21.15.204:6812/28210 conn(0x7f5aa00397e0 0x7f5aa003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f5aa404c0d0 con 0x7f5ab409faa0 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab359e640 1 --2- 172.21.15.178:0/2538970210 >> v2:172.21.15.204:6812/28210 conn(0x7f5aa00397e0 0x7f5aa003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab359e640 10 monclient: get_auth_request con 0x7f5aa00397e0 auth_method 0 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab259c640 10 monclient(hunting): handle_auth_done global_id 64213 payload 824 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab259c640 1 -- 172.21.15.178:0/2538970210 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ab404c4a0 con 0x7f5ab4142520 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab259c640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab4142520 0x7f5ab411c530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5ab409b5c0 tx=0x7f5aa8008040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5ab359e640 1 --2- 172.21.15.178:0/2538970210 >> v2:172.21.15.204:6812/28210 conn(0x7f5aa00397e0 0x7f5aa003bc90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5aac033000 tx=0x7f5aac036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5aa8016480 con 0x7f5ab4142520 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5a9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5a9affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.487+0000 7f5a9affd640 10 monclient: dump: 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.515 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.488+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f5aa800eb10 con 0x7f5ab4142520 2024-09-15T01:45:44.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.488+0000 7f5a9affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:44.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.488+0000 7f5a9affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:44.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.488+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab4142520 msgr2=0x7f5ab411c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.488+0000 7f5a9affd640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab4142520 0x7f5ab411c530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5ab409b5c0 tx=0x7f5aa8008040 comp rx=0 tx=0).stop 2024-09-15T01:45:44.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.488+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab4142520 msgr2=0x7f5ab411c530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:44.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.488+0000 7f5a9affd640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab4142520 0x7f5ab411c530 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5ab409b5c0 tx=0x7f5aa8008040 comp rx=0 tx=0).stop 2024-09-15T01:45:44.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.493+0000 7f5ab90dd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:45:44.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.493+0000 7f5ab90dd640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab411ca70 0x7f5ab411ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.493+0000 7f5ab90dd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.493+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2538970210 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab411f0f0 con 0x7f5ab411ca70 2024-09-15T01:45:44.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5ab259c640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab411ca70 0x7f5ab411ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5ab259c640 10 monclient: get_auth_request con 0x7f5ab411ca70 auth_method 0 2024-09-15T01:45:44.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5ab259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5ab259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5ab259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5ab259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5ab259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5ab259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5ab259c640 10 monclient(hunting): handle_auth_done global_id 64222 payload 824 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5ab259c640 1 -- 172.21.15.178:0/2538970210 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5ab404bc80 con 0x7f5ab411ca70 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5ab259c640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab411ca70 0x7f5ab411ce70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ab409f870 tx=0x7f5aa801e000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5aa8020c10 con 0x7f5ab411ca70 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5a9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5a9affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.494+0000 7f5a9affd640 10 monclient: dump: 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.520 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f5aa8020d70 con 0x7f5ab411ca70 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5a9affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5a9affd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:44.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab411ca70 msgr2=0x7f5ab411ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5a9affd640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab411ca70 0x7f5ab411ce70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ab409f870 tx=0x7f5aa801e000 comp rx=0 tx=0).stop 2024-09-15T01:45:44.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5a9affd640 1 -- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab411ca70 msgr2=0x7f5ab411ce70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:44.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5a9affd640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f5ab411ca70 0x7f5ab411ce70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.522 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:44.522 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:44.522 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:44.522 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:45:44.522 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 138, 2024-09-15T01:45:44.522 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:45:44.522 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 7, 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:45:44.523 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:44.524 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:45:44.525 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:45:44.526 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:45:44.527 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:45:44.528 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.529 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:45:44.530 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:45:44.531 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.532 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:45:44.533 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:45:44.534 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.535 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.536 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:45:44.536 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:45:44.536 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.536 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.536 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.536 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.536 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.536 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:45:44.536 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:45:44.536 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.536 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.536 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:45:44.537 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.538 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.539 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.540 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:45:44.541 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:45:44.542 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:45:44.543 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:45:44.544 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:45:44.544 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:45:44.544 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:45:44.544 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:45:44.544 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:45:44.544 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:45:44.544 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:45:44.544 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:45:44.544 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:44.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2538970210 >> v2:172.21.15.204:6812/28210 conn(0x7f5aa00397e0 msgr2=0x7f5aa003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5ab90dd640 1 --2- 172.21.15.178:0/2538970210 >> v2:172.21.15.204:6812/28210 conn(0x7f5aa00397e0 0x7f5aa003bc90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5aac033000 tx=0x7f5aac036000 comp rx=0 tx=0).stop 2024-09-15T01:45:44.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5ab90dd640 10 monclient: shutdown 2024-09-15T01:45:44.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5ab90dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:44.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2538970210 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ab409faa0 msgr2=0x7f5ab409b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5ab90dd640 1 --2- 172.21.15.178:0/2538970210 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f5ab409faa0 0x7f5ab409b670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400d8c0 tx=0x7f5aa4008040 comp rx=0 tx=0).stop 2024-09-15T01:45:44.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5ab359e640 1 -- 172.21.15.178:0/2538970210 reap_dead start 2024-09-15T01:45:44.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5ab359e640 1 -- 172.21.15.178:0/2538970210 reap_dead start 2024-09-15T01:45:44.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2538970210 shutdown_connections 2024-09-15T01:45:44.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2538970210 >> 172.21.15.178:0/2538970210 conn(0x7f5ab40a5ed0 msgr2=0x7f5ab40a53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:44.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2538970210 shutdown_connections 2024-09-15T01:45:44.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.495+0000 7f5ab90dd640 1 -- 172.21.15.178:0/2538970210 wait complete. 2024-09-15T01:45:44.547 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:45:44.547 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['e', 'f'] 2024-09-15T01:45:44.547 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c'] 2024-09-15T01:45:44.547 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-09-15T01:45:44.547 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-15T01:45:44.548 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-15T01:45:44.548 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:45:44.561 INFO:tasks.ceph.mon.e:Stopped 2024-09-15T01:45:44.561 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-15T01:45:44.561 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f store 2024-09-15T01:45:44.562 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f sync_force --yes-i-really-mean-it 2024-09-15T01:45:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.695+0000 7fb1fdd5d640 10 monclient: get_monmap_and_config 2024-09-15T01:45:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.695+0000 7fb1fdd5d640 10 monclient: build_initial_monmap 2024-09-15T01:45:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.696+0000 7fb1fdd5d640 10 monclient: monmap: 2024-09-15T01:45:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:44.696864+0000 2024-09-15T01:45:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:44.696864+0000 2024-09-15T01:45:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:44.699 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:44.699 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:44.699 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:44.699 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:44.699 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:44.699 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:44.699 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:44.699 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:44.699 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:44.699 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:44.699 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.696+0000 7fb1fdd5d640 1 Processor -- start 2024-09-15T01:45:44.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.696+0000 7fb1fdd5d640 1 -- start start 2024-09-15T01:45:44.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.696+0000 7fb1fdd5d640 10 monclient: init 2024-09-15T01:45:44.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:44.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 10 monclient: _add_conns ranks=[6,0,9,10,5,4,7,2,3,1,8] 2024-09-15T01:45:44.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb1f80a2930 0x7fb1f8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 10 monclient(hunting): picked mon.noname-d con 0x7fb1f80a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:45:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb1f809faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:45:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb1f809fe70 0x7fb1f80a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 10 monclient(hunting): picked mon.noname-a con 0x7fb1f809fe70 addr v2:172.21.15.178:3300/0 2024-09-15T01:45:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f804bf60 con 0x7fb1f809faa0 2024-09-15T01:45:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f809dd80 con 0x7fb1f809fe70 2024-09-15T01:45:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f80a2d00 con 0x7fb1f80a2930 2024-09-15T01:45:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1fdd5d640 10 monclient(hunting): authenticate will time out at 1549.447501s 2024-09-15T01:45:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f77fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb1f80a2930 0x7fb1f8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb1f809fe70 0x7fb1f80a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f77fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb1f80a2930 0x7fb1f8132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:57194/0 (socket says 172.21.15.178:57194) 2024-09-15T01:45:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f77fe640 1 -- 172.21.15.178:0/2274507032 learned_addr learned my addr 172.21.15.178:0/2274507032 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f77fe640 10 monclient(hunting): get_auth_request con 0x7fb1f80a2930 auth_method 0 2024-09-15T01:45:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 10 monclient(hunting): get_auth_request con 0x7fb1f809fe70 auth_method 0 2024-09-15T01:45:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 10 monclient(hunting): handle_auth_done global_id 64251 payload 274 2024-09-15T01:45:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 1 -- 172.21.15.178:0/2274507032 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb1f80a2930 msgr2=0x7fb1f8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:45:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 1 --2- 172.21.15.178:0/2274507032 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb1f80a2930 0x7fb1f8132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 1 -- 172.21.15.178:0/2274507032 >> v1:172.21.15.178:6789/0 conn(0x7fb1f809faa0 legacy=0x7fb1f80a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:45:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 1 monclient: found mon.noname-a 2024-09-15T01:45:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.697+0000 7fb1f7fff640 1 -- 172.21.15.178:0/2274507032 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f80a0310 con 0x7fb1f809fe70 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f7fff640 1 --2- 172.21.15.178:0/2274507032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1f809fe70 0x7fb1f80a0fd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009e60 tx=0x7fb1f0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e15b3f46d84110 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f6ffd640 1 -- 172.21.15.178:0/2274507032 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1f00079d0 con 0x7fb1f809fe70 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f6ffd640 10 monclient: dump: 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f6ffd640 1 -- 172.21.15.178:0/2274507032 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f002d050 con 0x7fb1f809fe70 2024-09-15T01:45:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1fdd5d640 5 monclient: authenticate success, global_id 64251 2024-09-15T01:45:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1fdd5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:45:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1fdd5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:45:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f6ffd640 1 -- 172.21.15.178:0/2274507032 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1f0007bd0 con 0x7fb1f809fe70 2024-09-15T01:45:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1fdd5d640 4 set_mon_vals no callback set 2024-09-15T01:45:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.698+0000 7fb1f6ffd640 10 monclient: dump: 2024-09-15T01:45:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.717 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.699+0000 7fb1fdd5d640 10 monclient: shutdown 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.699+0000 7fb1fdd5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.699+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2274507032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1f809fe70 msgr2=0x7fb1f80a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.699+0000 7fb1fdd5d640 1 --2- 172.21.15.178:0/2274507032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1f809fe70 0x7fb1f80a0fd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009e60 tx=0x7fb1f0009780 comp rx=0 tx=0).stop 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.699+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2274507032 shutdown_connections 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.699+0000 7fb1fdd5d640 1 --2- 172.21.15.178:0/2274507032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1f809fe70 0x7fb1f80a0fd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.699+0000 7fb1fdd5d640 1 --2- 172.21.15.178:0/2274507032 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb1f80a2930 0x7fb1f8132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.699+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2274507032 >> 172.21.15.178:0/2274507032 conn(0x7fb1f80a5ed0 msgr2=0x7fb1f80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.699+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2274507032 shutdown_connections 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.699+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2274507032 wait complete. 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.699+0000 7fb1fdd5d640 10 monclient: build_initial_monmap 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.699+0000 7fb1fdd5d640 10 monclient: monmap: 2024-09-15T01:45:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:44.719 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 1 Processor -- start 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 1 -- start start 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 10 monclient: init 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 10 monclient: _add_conns ranks=[7,0,8,6,1,3,5,4,2] 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb1f809faa0 0x7fb1f809b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 10 monclient(hunting): picked mon.noname-i con 0x7fb1f809faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1f809fe70 0x7fb1f8131990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 10 monclient(hunting): picked mon.noname-a con 0x7fb1f809fe70 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb1f80a2930 0x7fb1f8131ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 10 monclient(hunting): picked mon.noname-e con 0x7fb1f80a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f809c180 con 0x7fb1f809fe70 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f811d5a0 con 0x7fb1f809faa0 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f811ed90 con 0x7fb1f80a2930 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1fdd5d640 10 monclient(hunting): authenticate will time out at 1549.450833s 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1f7fff640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb1f80a2930 msgr2=0x7fb1f8131ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1f7fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb1f80a2930 0x7fb1f8131ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1effff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1f809fe70 0x7fb1f8131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1effff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1f809fe70 0x7fb1f8131990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33658/0 (socket says 172.21.15.178:33658) 2024-09-15T01:45:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1effff640 1 -- 172.21.15.178:0/2958267291 learned_addr learned my addr 172.21.15.178:0/2958267291 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1effff640 10 monclient(hunting): get_auth_request con 0x7fb1f809fe70 auth_method 0 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.700+0000 7fb1effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1f77fe640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb1f809faa0 0x7fb1f809b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1effff640 10 monclient(hunting): handle_auth_done global_id 64269 payload 824 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1effff640 1 -- 172.21.15.178:0/2958267291 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb1f80a2930 msgr2=0x7fb1f8131ed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1effff640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb1f80a2930 0x7fb1f8131ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1effff640 1 -- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb1f809faa0 msgr2=0x7fb1f809b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1effff640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb1f809faa0 0x7fb1f809b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1effff640 10 monclient: _finish_hunting 0 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1effff640 1 monclient: found mon.noname-a 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1effff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1effff640 1 -- 172.21.15.178:0/2958267291 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f0009a20 con 0x7fb1f809fe70 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1f77fe640 10 monclient: get_auth_request con 0x7fb1f809faa0 auth_method 0 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1f77fe640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb1f809faa0 0x7fb1f809b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1effff640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1f809fe70 0x7fb1f8131990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c490 tx=0x7fb1e0007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1e0011a10 con 0x7fb1f809fe70 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1ef7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1ef7fe640 10 monclient: dump: 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.725 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1ef7fe640 10 monclient: _finish_auth 0 2024-09-15T01:45:44.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:45:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1fdd5d640 5 monclient: authenticate success, global_id 64269 2024-09-15T01:45:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e0011bb0 con 0x7fb1f809fe70 2024-09-15T01:45:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1fdd5d640 10 monclient: _renew_subs 2024-09-15T01:45:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1fdd5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:45:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2958267291 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f811eab0 con 0x7fb1f809fe70 2024-09-15T01:45:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1ef7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:45:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1e0011d20 con 0x7fb1f809fe70 2024-09-15T01:45:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1ef7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:45:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1f67fc640 4 set_mon_vals no callback set 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1ef7fe640 10 monclient: dump: 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1fdd5d640 10 monclient: _renew_subs 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1fdd5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.701+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2958267291 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f811c9f0 con 0x7fb1f809fe70 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1fdd5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb1e0011590 con 0x7fb1f809fe70 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1fdd5d640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f8141eb0 0x7fb1f8142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1fdd5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2958267291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f804c4a0 con 0x7fb1f8141eb0 2024-09-15T01:45:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1ef7fe640 1 --2- 172.21.15.178:0/2958267291 >> v2:172.21.15.204:6812/28210 conn(0x7fb1d80397b0 0x7fb1d803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fb1e004b0e0 con 0x7fb1f809fe70 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1f77fe640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f8141eb0 0x7fb1f8142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1f77fe640 10 monclient: get_auth_request con 0x7fb1f8141eb0 auth_method 0 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1f7fff640 1 --2- 172.21.15.178:0/2958267291 >> v2:172.21.15.204:6812/28210 conn(0x7fb1d80397b0 0x7fb1d803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1f7fff640 10 monclient: get_auth_request con 0x7fb1d80397b0 auth_method 0 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1f77fe640 10 monclient(hunting): handle_auth_done global_id 54649 payload 824 2024-09-15T01:45:44.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.702+0000 7fb1f77fe640 1 -- 172.21.15.178:0/2958267291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1f805ed70 con 0x7fb1f8141eb0 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1f77fe640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f8141eb0 0x7fb1f8142280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8010920 tx=0x7fb1e8008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1f7fff640 1 --2- 172.21.15.178:0/2958267291 >> v2:172.21.15.204:6812/28210 conn(0x7fb1d80397b0 0x7fb1d803bc60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0034000 tx=0x7fb1f0037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1e80143f0 con 0x7fb1f8141eb0 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1ef7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1ef7fe640 10 monclient: dump: 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.730 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fb1e800eae0 con 0x7fb1f8141eb0 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1ef7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1ef7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:45:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f8141eb0 msgr2=0x7fb1f8142280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1ef7fe640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f8141eb0 0x7fb1f8142280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8010920 tx=0x7fb1e8008040 comp rx=0 tx=0).stop 2024-09-15T01:45:44.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f8141eb0 msgr2=0x7fb1f8142280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:44.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.703+0000 7fb1ef7fe640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f8141eb0 0x7fb1f8142280 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8010920 tx=0x7fb1e8008040 comp rx=0 tx=0).stop 2024-09-15T01:45:44.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1fdd5d640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:45:44.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1fdd5d640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f81427c0 0x7fb1f8096d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1fdd5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2958267291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f811ef00 con 0x7fb1f81427c0 2024-09-15T01:45:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1f77fe640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f81427c0 0x7fb1f8096d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1f77fe640 10 monclient: get_auth_request con 0x7fb1f81427c0 auth_method 0 2024-09-15T01:45:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:45:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:45:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:45:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1f77fe640 10 monclient(hunting): handle_auth_done global_id 54658 payload 824 2024-09-15T01:45:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1f77fe640 1 -- 172.21.15.178:0/2958267291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7fb1f804bc80 con 0x7fb1f81427c0 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.710+0000 7fb1f77fe640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f81427c0 0x7fb1f8096d70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8010920 tx=0x7fb1e801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1e8021070 con 0x7fb1f81427c0 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1ef7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1ef7fe640 10 monclient: dump: 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:45:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:45:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:45:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:45:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:45:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:45:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:45:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:45:44.736 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7fb1e8006750 con 0x7fb1f81427c0 2024-09-15T01:45:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1ef7fe640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:45:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1ef7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:45:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f81427c0 msgr2=0x7fb1f8096d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1ef7fe640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f81427c0 0x7fb1f8096d70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8010920 tx=0x7fb1e801b000 comp rx=0 tx=0).stop 2024-09-15T01:45:44.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1ef7fe640 1 -- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f81427c0 msgr2=0x7fb1f8096d70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:45:44.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1ef7fe640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f81427c0 0x7fb1f8096d70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:45:44.737 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:45:44.737 INFO:teuthology.orchestra.run.smithi178.stdout: "ret": 0, 2024-09-15T01:45:44.737 INFO:teuthology.orchestra.run.smithi178.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-15T01:45:44.737 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:45:44.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2958267291 >> v2:172.21.15.204:6812/28210 conn(0x7fb1d80397b0 msgr2=0x7fb1d803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1fdd5d640 1 --2- 172.21.15.178:0/2958267291 >> v2:172.21.15.204:6812/28210 conn(0x7fb1d80397b0 0x7fb1d803bc60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0034000 tx=0x7fb1f0037000 comp rx=0 tx=0).stop 2024-09-15T01:45:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1fdd5d640 10 monclient: shutdown 2024-09-15T01:45:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1fdd5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:45:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2958267291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1f809fe70 msgr2=0x7fb1f8131990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:45:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1fdd5d640 1 --2- 172.21.15.178:0/2958267291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1f809fe70 0x7fb1f8131990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c490 tx=0x7fb1e0007e70 comp rx=0 tx=0).stop 2024-09-15T01:45:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1f7fff640 1 -- 172.21.15.178:0/2958267291 reap_dead start 2024-09-15T01:45:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1f7fff640 1 -- 172.21.15.178:0/2958267291 reap_dead start 2024-09-15T01:45:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2958267291 shutdown_connections 2024-09-15T01:45:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2958267291 >> 172.21.15.178:0/2958267291 conn(0x7fb1f80a5ed0 msgr2=0x7fb1f80a3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:45:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2958267291 shutdown_connections 2024-09-15T01:45:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:44.711+0000 7fb1fdd5d640 1 -- 172.21.15.178:0/2958267291 wait complete. 2024-09-15T01:45:44.740 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T01:45:44.740 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-15T01:45:44.740 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:45:44.754 INFO:tasks.ceph.mon.f:Stopped 2024-09-15T01:45:44.754 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-15T01:45:44.754 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-15T01:45:44.755 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T01:45:59.755 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-15T01:45:59.756 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-15T01:45:59.756 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-15T01:45:59.757 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:45:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.860+0000 7fa2bdd14640 10 monclient: get_monmap_and_config 2024-09-15T01:45:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.860+0000 7fa2bdd14640 10 monclient: build_initial_monmap 2024-09-15T01:45:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.860+0000 7fa2bdd14640 10 monclient: monmap: 2024-09-15T01:45:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:45:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:45:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:45:59.861151+0000 2024-09-15T01:45:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:45:59.861151+0000 2024-09-15T01:45:59.862 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:45:59.863 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:45:59.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 1 Processor -- start 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 1 -- start start 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 10 monclient: init 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 10 monclient: _reopen_session rank -1 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 10 monclient: _add_conns ranks=[10,4,1,9,6,3,5,8,7,2,0] 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa2b80a2930 0x7fa2b8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 10 monclient(hunting): picked mon.noname-f con 0x7fa2b80a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 10 monclient(hunting): picked mon.noname-c con 0x7fa2b809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa2b80a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b804bf60 con 0x7fa2b80a08c0 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b809e0f0 con 0x7fa2b80a2930 2024-09-15T01:45:59.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b809dd80 con 0x7fa2b809faa0 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 10 monclient(hunting): _renew_subs 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.861+0000 7fa2bdd14640 10 monclient(hunting): authenticate will time out at 1564.611942s 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2b77fe640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7fa2b80a2930 msgr2=0x7fa2b8132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2b7fff640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7fa2b80a08c0 legacy=0x7fa2b80a0d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2affff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2b77fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa2b80a2930 0x7fa2b8132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2affff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:51342/0 (socket says 172.21.15.178:51342) 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 learned_addr learned my addr 172.21.15.178:0/1893727787 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2affff640 10 monclient(hunting): get_auth_request con 0x7fa2b809faa0 auth_method 0 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:45:59.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:45:59.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 msgr2=0x7fa2b809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:45:59.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 msgr2=0x7fa2b809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:45:59.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:45:59.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:45:59.862+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:46:00.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.063+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:00.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.063+0000 7fa2affff640 10 monclient(hunting): get_auth_request con 0x7fa2b809faa0 auth_method 0 2024-09-15T01:46:00.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.063+0000 7fa2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:00.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.063+0000 7fa2affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:00.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.063+0000 7fa2affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:46:00.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.063+0000 7fa2b77fe640 1 -- 172.21.15.178:0/1893727787 >> v2:172.21.15.204:3300/0 conn(0x7fa2b80a2930 msgr2=0x7fa2b8132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:46:00.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.063+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 >> v1:172.21.15.204:6789/0 conn(0x7fa2b80a08c0 legacy=0x7fa2b80a0d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:46:00.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.063+0000 7fa2b77fe640 1 --2- 172.21.15.178:0/1893727787 >> v2:172.21.15.204:3300/0 conn(0x7fa2b80a2930 0x7fa2b8132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:46:00.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.063+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 msgr2=0x7fa2b809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:46:00.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.063+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 msgr2=0x7fa2b809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:46:00.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.063+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:46:00.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.063+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:46:00.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.464+0000 7fa2b77fe640 1 -- 172.21.15.178:0/1893727787 >> v2:172.21.15.204:3300/0 conn(0x7fa2b80a2930 msgr2=0x7fa2b8132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:46:00.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.464+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 >> v1:172.21.15.204:6789/0 conn(0x7fa2b80a08c0 legacy=0x7fa2b80a0d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:46:00.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.464+0000 7fa2b77fe640 1 --2- 172.21.15.178:0/1893727787 >> v2:172.21.15.204:3300/0 conn(0x7fa2b80a2930 0x7fa2b8132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:46:00.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.464+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:00.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.464+0000 7fa2affff640 10 monclient(hunting): get_auth_request con 0x7fa2b809faa0 auth_method 0 2024-09-15T01:46:00.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.464+0000 7fa2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:00.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.464+0000 7fa2affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:00.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.464+0000 7fa2affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:46:00.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.464+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 msgr2=0x7fa2b809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:46:00.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.464+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 msgr2=0x7fa2b809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:46:00.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.464+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:46:00.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:00.464+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:46:01.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:01.265+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 >> v1:172.21.15.204:6789/0 conn(0x7fa2b80a08c0 legacy=0x7fa2b80a0d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:46:01.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:01.265+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:01.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:01.265+0000 7fa2b77fe640 1 -- 172.21.15.178:0/1893727787 >> v2:172.21.15.204:3300/0 conn(0x7fa2b80a2930 msgr2=0x7fa2b8132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:46:01.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:01.265+0000 7fa2b77fe640 1 --2- 172.21.15.178:0/1893727787 >> v2:172.21.15.204:3300/0 conn(0x7fa2b80a2930 0x7fa2b8132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:46:01.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:01.265+0000 7fa2affff640 10 monclient(hunting): get_auth_request con 0x7fa2b809faa0 auth_method 0 2024-09-15T01:46:01.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:01.265+0000 7fa2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:01.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:01.265+0000 7fa2affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:01.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:01.265+0000 7fa2affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:46:01.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:01.265+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 msgr2=0x7fa2b809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:46:01.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:01.265+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 msgr2=0x7fa2b809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:46:01.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:01.265+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:46:01.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:01.265+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:46:02.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 10 monclient(hunting): tick 2024-09-15T01:46:02.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 1 monclient(hunting): continuing hunt 2024-09-15T01:46:02.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:46:02.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 msgr2=0x7fa2b809fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:46:02.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa2b809faa0 0x7fa2b809fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 1 -- 172.21.15.178:0/1893727787 >> v2:172.21.15.204:3300/0 conn(0x7fa2b80a2930 msgr2=0x7fa2b8132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 1 --2- 172.21.15.178:0/1893727787 >> v2:172.21.15.204:3300/0 conn(0x7fa2b80a2930 0x7fa2b8132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 1 -- 172.21.15.178:0/1893727787 >> v1:172.21.15.204:6789/0 conn(0x7fa2b80a08c0 legacy=0x7fa2b80a0d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 10 monclient: _add_conns ranks=[0,5,2,3,6,7,8,9] 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa2980037b0 addr v1:172.21.15.178:6789/0 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 10 monclient(hunting): picked mon.noname-h con 0x7fa2980073d0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 10 monclient(hunting): picked mon.noname-b con 0x7fa298009ee0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 1 -- 172.21.15.178:0/1893727787 --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b80a2d00 con 0x7fa2980037b0 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 1 -- 172.21.15.178:0/1893727787 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b809e0f0 con 0x7fa298009ee0 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:02.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 1 -- 172.21.15.178:0/1893727787 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b809dd80 con 0x7fa2980073d0 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b5ffb640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b7fff640 10 monclient(hunting): get_auth_request con 0x7fa298009ee0 auth_method 0 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 msgr2=0x7fa29800c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 msgr2=0x7fa29800c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2affff640 10 monclient(hunting): get_auth_request con 0x7fa2980073d0 auth_method 0 2024-09-15T01:46:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:02.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:02.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:02.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 msgr2=0x7fa298009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:46:02.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 msgr2=0x7fa298009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:46:02.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:46:02.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:02.862+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:46:03.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:03.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2b7fff640 10 monclient(hunting): get_auth_request con 0x7fa298009ee0 auth_method 0 2024-09-15T01:46:03.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:03.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:03.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2b7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:46:03.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 msgr2=0x7fa29800c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:46:03.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 msgr2=0x7fa29800c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:46:03.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:46:03.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:46:03.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:03.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2affff640 10 monclient(hunting): get_auth_request con 0x7fa2980073d0 auth_method 0 2024-09-15T01:46:03.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:03.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:03.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:46:03.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 msgr2=0x7fa298009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:46:03.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 msgr2=0x7fa298009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:46:03.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:46:03.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.063+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:46:03.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:03.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2b7fff640 10 monclient(hunting): get_auth_request con 0x7fa298009ee0 auth_method 0 2024-09-15T01:46:03.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:03.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:03.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2b7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:46:03.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 msgr2=0x7fa29800c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:46:03.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 msgr2=0x7fa29800c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:46:03.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:46:03.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:46:03.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:03.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2affff640 10 monclient(hunting): get_auth_request con 0x7fa2980073d0 auth_method 0 2024-09-15T01:46:03.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:03.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:03.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:46:03.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 msgr2=0x7fa298009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:46:03.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 msgr2=0x7fa298009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:46:03.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:46:03.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:03.464+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:46:04.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2b7fff640 10 monclient(hunting): get_auth_request con 0x7fa298009ee0 auth_method 0 2024-09-15T01:46:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2b7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:46:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 msgr2=0x7fa29800c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:46:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 msgr2=0x7fa29800c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:46:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:46:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:46:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2affff640 10 monclient(hunting): get_auth_request con 0x7fa2980073d0 auth_method 0 2024-09-15T01:46:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:46:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 msgr2=0x7fa298009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:46:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2affff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 msgr2=0x7fa298009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:46:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:46:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.265+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:46:04.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744275971 0 0) 0x7fa2b80a2d00 con 0x7fa2980037b0 2024-09-15T01:46:04.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:04.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:04.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient(hunting): my global_id is 64278 2024-09-15T01:46:04.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a40034e0 con 0x7fa2980037b0 2024-09-15T01:46:04.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3603745086 0 0) 0x7fa2a40034e0 con 0x7fa2980037b0 2024-09-15T01:46:04.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 msgr2=0x7fa298009830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:46:04.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa2980073d0 0x7fa298009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:04.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 msgr2=0x7fa29800c390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa298009ee0 0x7fa29800c390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b804bf60 con 0x7fa2980037b0 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 reap_dead start 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fa2a80043f0 con 0x7fa2980037b0 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:46:04.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient: dump: 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:04.772 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 >> v1:172.21.15.178:6789/0 conn(0x7fa2980037b0 legacy=0x7fa298005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient: _add_conns ranks=[0,3,1,4,2,8,7,6,5] 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2a4004220 0x7fa2a4004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient(hunting): picked mon.a con 0x7fa2a4004220 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2a4004de0 0x7fa2a4005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient(hunting): picked mon.g con 0x7fa2a4004de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4009c00 0x7fa2a400c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient(hunting): picked mon.f con 0x7fa2a4009c00 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:46:04.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b809e0f0 con 0x7fa2a4004220 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b809dd80 con 0x7fa2a4009c00 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a400c970 con 0x7fa2a4004de0 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b6ffd640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b7fff640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4009c00 msgr2=0x7fa2a400c060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b77fe640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2a4004220 0x7fa2a4004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b7fff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4009c00 0x7fa2a400c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b77fe640 10 monclient(hunting): get_auth_request con 0x7fa2a4004220 auth_method 0 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:04.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2b77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2affff640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2a4004de0 0x7fa2a4005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2affff640 10 monclient(hunting): get_auth_request con 0x7fa2a4004de0 auth_method 0 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2affff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.764+0000 7fa2affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2affff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 10 monclient(hunting): handle_auth_done global_id 64278 payload 293 2024-09-15T01:46:04.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2a4004de0 msgr2=0x7fa2a4005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2a4004de0 0x7fa2a4005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4009c00 msgr2=0x7fa2a400c060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4009c00 0x7fa2a400c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 1 monclient: found mon.a 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 1 -- 172.21.15.178:0/1893727787 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a40034e0 con 0x7fa2a4004220 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 10 monclient: _finish_auth 0 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2bdd14640 5 monclient: authenticate success, global_id 64278 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.765+0000 7fa2bdd14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:46:04.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.766+0000 7fa2b77fe640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2a4004220 0x7fa2a4004780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8007fc0 tx=0x7fa2a8004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be20de61c771183 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:04.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.766+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2a8004620 con 0x7fa2a4004220 2024-09-15T01:46:04.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2b6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:04.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2b6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:46:04.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2b6ffd640 10 monclient: dump: 2024-09-15T01:46:04.777 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:04.777 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:04.777 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:04.777 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:04.777 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:04.777 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:04.778 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:04.778 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:04.778 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:04.778 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:04.778 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:04.778 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:04.778 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:04.778 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:04.778 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:04.778 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2bdd14640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a802d070 con 0x7fa2a4004220 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2b6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2b6ffd640 1 -- 172.21.15.178:0/1893727787 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2a8004c20 con 0x7fa2a4004220 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2bdd14640 10 monclient: get_monmap_and_config success 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2bdd14640 4 set_mon_vals no callback set 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2b6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2b6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2b6ffd640 10 monclient: dump: 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:04.779 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2bdd14640 10 monclient: shutdown 2024-09-15T01:46:04.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2bdd14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:04.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2bdd14640 1 -- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2a4004220 msgr2=0x7fa2a4004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:04.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2bdd14640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2a4004220 0x7fa2a4004780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8007fc0 tx=0x7fa2a8004380 comp rx=0 tx=0).stop 2024-09-15T01:46:04.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2bdd14640 1 -- 172.21.15.178:0/1893727787 shutdown_connections 2024-09-15T01:46:04.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2bdd14640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2a4004de0 0x7fa2a4005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:04.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2bdd14640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2a4004220 0x7fa2a4004780 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:04.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2bdd14640 1 --2- 172.21.15.178:0/1893727787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4009c00 0x7fa2a400c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:04.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.767+0000 7fa2bdd14640 1 -- 172.21.15.178:0/1893727787 >> 172.21.15.178:0/1893727787 conn(0x7fa2b80a5ed0 msgr2=0x7fa2b80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:04.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.768+0000 7fa2bdd14640 1 -- 172.21.15.178:0/1893727787 shutdown_connections 2024-09-15T01:46:04.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.768+0000 7fa2bdd14640 1 -- 172.21.15.178:0/1893727787 wait complete. 2024-09-15T01:46:04.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.768+0000 7fa2bdd14640 10 monclient: build_initial_monmap 2024-09-15T01:46:04.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.768+0000 7fa2bdd14640 10 monclient: monmap: 2024-09-15T01:46:04.787 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:04.787 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:46:04.787 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:46:04.787 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:46:04.787 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:04.787 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:46:04.787 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.769+0000 7fa2bdd14640 1 Processor -- start 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.769+0000 7fa2bdd14640 1 -- start start 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.769+0000 7fa2bdd14640 10 monclient: init 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:04.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 10 monclient: _add_conns ranks=[3,1,6,5,0,2,8,4,7] 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2980037b0 0x7fa2b81419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 10 monclient(hunting): picked mon.noname-g con 0x7fa2980037b0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4004220 0x7fa2b81319e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 10 monclient(hunting): picked mon.noname-f con 0x7fa2a4004220 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2a4004de0 0x7fa2b811e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 10 monclient(hunting): picked mon.noname-d con 0x7fa2a4004de0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b81207c0 con 0x7fa2a4004220 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80a2390 con 0x7fa2980037b0 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b809d630 con 0x7fa2a4004de0 2024-09-15T01:46:04.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2bdd14640 10 monclient(hunting): authenticate will time out at 1569.520521s 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2affff640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4004220 msgr2=0x7fa2b81319e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2affff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4004220 0x7fa2b81319e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b7fff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2a4004de0 0x7fa2b811e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b77fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2980037b0 0x7fa2b81419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b77fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2980037b0 0x7fa2b81419c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:56096/0 (socket says 172.21.15.178:56096) 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b77fe640 1 -- 172.21.15.178:0/2865094760 learned_addr learned my addr 172.21.15.178:0/2865094760 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b77fe640 10 monclient(hunting): get_auth_request con 0x7fa2980037b0 auth_method 0 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b7fff640 10 monclient(hunting): get_auth_request con 0x7fa2a4004de0 auth_method 0 2024-09-15T01:46:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.770+0000 7fa2b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2b77fe640 10 monclient(hunting): handle_auth_done global_id 64227 payload 824 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2b77fe640 1 -- 172.21.15.178:0/2865094760 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2a4004de0 msgr2=0x7fa2b811e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2b77fe640 1 --2- 172.21.15.178:0/2865094760 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2a4004de0 0x7fa2b811e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2b77fe640 1 -- 172.21.15.178:0/2865094760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4004220 msgr2=0x7fa2b81319e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2b77fe640 1 --2- 172.21.15.178:0/2865094760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4004220 0x7fa2b81319e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2b77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2b77fe640 1 monclient: found mon.noname-g 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2b77fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2b77fe640 1 -- 172.21.15.178:0/2865094760 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b8121b30 con 0x7fa2980037b0 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2b77fe640 1 --2- 172.21.15.178:0/2865094760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2980037b0 0x7fa2b81419c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8000c00 tx=0x7fa2a8033000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2af7fe640 1 -- 172.21.15.178:0/2865094760 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2a8034db0 con 0x7fa2980037b0 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2af7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2af7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2af7fe640 10 monclient: dump: 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2af7fe640 10 monclient: _finish_auth 0 2024-09-15T01:46:04.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2bdd14640 5 monclient: authenticate success, global_id 64227 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2af7fe640 1 -- 172.21.15.178:0/2865094760 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a8040450 con 0x7fa2980037b0 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2af7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2af7fe640 1 -- 172.21.15.178:0/2865094760 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2a8040a40 con 0x7fa2980037b0 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2af7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2af7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2b67fc640 4 set_mon_vals no callback set 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2af7fe640 10 monclient: dump: 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:04.794 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2bdd14640 10 monclient: _renew_subs 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2bdd14640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:46:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2bdd14640 1 -- 172.21.15.178:0/2865094760 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b81427f0 con 0x7fa2980037b0 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2bdd14640 10 monclient: _renew_subs 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2bdd14640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.771+0000 7fa2bdd14640 1 -- 172.21.15.178:0/2865094760 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b8120bd0 con 0x7fa2980037b0 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.772+0000 7fa2af7fe640 1 -- 172.21.15.178:0/2865094760 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa2a8040bc0 con 0x7fa2980037b0 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.772+0000 7fa2af7fe640 1 --2- 172.21.15.178:0/2865094760 >> v2:172.21.15.204:6812/28210 conn(0x7fa29803a720 0x7fa29803cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.772+0000 7fa2af7fe640 1 -- 172.21.15.178:0/2865094760 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fa2a806bd60 con 0x7fa2980037b0 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.772+0000 7fa2bdd14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.772+0000 7fa2bdd14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.772+0000 7fa2bdd14640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.772+0000 7fa2bdd14640 1 -- 172.21.15.178:0/2865094760 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b8121fa0 con 0x7fa2980037b0 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.773+0000 7fa2affff640 1 --2- 172.21.15.178:0/2865094760 >> v2:172.21.15.204:6812/28210 conn(0x7fa29803a720 0x7fa29803cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.773+0000 7fa2affff640 10 monclient: get_auth_request con 0x7fa29803a720 auth_method 0 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.773+0000 7fa2affff640 1 --2- 172.21.15.178:0/2865094760 >> v2:172.21.15.204:6812/28210 conn(0x7fa29803a720 0x7fa29803cbd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0006df0 tx=0x7fa2a001b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.778+0000 7fa2af7fe640 1 -- 172.21.15.178:0/2865094760 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa2a8034930 con 0x7fa2980037b0 2024-09-15T01:46:04.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.778+0000 7fa2af7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:04.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.778+0000 7fa2af7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:46:04.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.995+0000 7fa2bdd14640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:46:04.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.995+0000 7fa2bdd14640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:46:04.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.995+0000 7fa2bdd14640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:46:04.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.995+0000 7fa2bdd14640 1 -- 172.21.15.178:0/2865094760 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa2b804b720 con 0x7fa2980037b0 2024-09-15T01:46:04.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.995+0000 7fa2af7fe640 1 -- 172.21.15.178:0/2865094760 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3172 (secure 0 0 0) 0x7fa2a80307f0 con 0x7fa2980037b0 2024-09-15T01:46:04.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.995+0000 7fa2af7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:46:04.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.995+0000 7fa2af7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:46:04.998 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":144,"quorum":[0,2,3,4,5,6,7],"quorum_names":["a","b","g","c","h","d","i"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:46:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 1 -- 172.21.15.178:0/2865094760 >> v2:172.21.15.204:6812/28210 conn(0x7fa29803a720 msgr2=0x7fa29803cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 1 --2- 172.21.15.178:0/2865094760 >> v2:172.21.15.204:6812/28210 conn(0x7fa29803a720 0x7fa29803cbd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0006df0 tx=0x7fa2a001b040 comp rx=0 tx=0).stop 2024-09-15T01:46:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 10 monclient: shutdown 2024-09-15T01:46:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 1 -- 172.21.15.178:0/2865094760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2980037b0 msgr2=0x7fa2b81419c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 1 --2- 172.21.15.178:0/2865094760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2980037b0 0x7fa2b81419c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8000c00 tx=0x7fa2a8033000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 1 -- 172.21.15.178:0/2865094760 shutdown_connections 2024-09-15T01:46:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 1 --2- 172.21.15.178:0/2865094760 >> v2:172.21.15.204:6812/28210 conn(0x7fa29803a720 0x7fa29803cbd0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 1 --2- 172.21.15.178:0/2865094760 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa2a4004de0 0x7fa2b811e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 1 --2- 172.21.15.178:0/2865094760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a4004220 0x7fa2b81319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 1 --2- 172.21.15.178:0/2865094760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2980037b0 0x7fa2b81419c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 1 -- 172.21.15.178:0/2865094760 >> 172.21.15.178:0/2865094760 conn(0x7fa2b80a5ed0 msgr2=0x7fa2b80a4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 1 -- 172.21.15.178:0/2865094760 shutdown_connections 2024-09-15T01:46:05.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:04.997+0000 7fa2bdd14640 1 -- 172.21.15.178:0/2865094760 wait complete. 2024-09-15T01:46:05.010 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-15T01:46:05.010 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-09-15T01:46:05.011 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:46:05.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.102+0000 7fe07115d640 10 monclient: get_monmap_and_config 2024-09-15T01:46:05.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.102+0000 7fe07115d640 10 monclient: build_initial_monmap 2024-09-15T01:46:05.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.102+0000 7fe07115d640 10 monclient: monmap: 2024-09-15T01:46:05.104 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:05.104 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:46:05.103116+0000 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:46:05.103116+0000 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:05.105 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:05.106 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:46:05.106 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:46:05.106 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.102+0000 7fe07115d640 1 Processor -- start 2024-09-15T01:46:05.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.102+0000 7fe07115d640 1 -- start start 2024-09-15T01:46:05.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.102+0000 7fe07115d640 10 monclient: init 2024-09-15T01:46:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 10 monclient: _add_conns ranks=[0,3,2,4,5,10,9,6,7,1,8] 2024-09-15T01:46:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe06c0a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:46:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c132de0 0x7fe06c137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 10 monclient(hunting): picked mon.noname-g con 0x7fe06c132de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:46:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06c1380e0 0x7fe06c13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 10 monclient(hunting): picked mon.noname-b con 0x7fe06c1380e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:46:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe06c04bf60 con 0x7fe06c0a2930 2024-09-15T01:46:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c09dd80 con 0x7fe06c1380e0 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c0a2d00 con 0x7fe06c132de0 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe07115d640 10 monclient(hunting): authenticate will time out at 1569.853337s 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06ad9d640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe06c0a2930 0x7fe06c09faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:47078/0 (socket says 172.21.15.178:47078) 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06c1380e0 0x7fe06c13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06ad9d640 1 -- 172.21.15.178:0/199664521 learned_addr learned my addr 172.21.15.178:0/199664521 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 10 monclient(hunting): get_auth_request con 0x7fe06c1380e0 auth_method 0 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe069d9b640 1 -- 172.21.15.178:0/199664521 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193680324 0 0) 0x7fe06c04bf60 con 0x7fe06c0a2930 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe069d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe069d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe069d9b640 10 monclient(hunting): my global_id is 64287 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06a59c640 1 --2- 172.21.15.178:0/199664521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c132de0 0x7fe06c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe069d9b640 1 -- 172.21.15.178:0/199664521 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0580034e0 con 0x7fe06c0a2930 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 10 monclient(hunting): handle_auth_done global_id 35066 payload 274 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 1 -- 172.21.15.178:0/199664521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c132de0 msgr2=0x7fe06c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 1 --2- 172.21.15.178:0/199664521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c132de0 0x7fe06c137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 1 -- 172.21.15.178:0/199664521 >> v1:172.21.15.178:6789/0 conn(0x7fe06c0a2930 legacy=0x7fe06c09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 10 monclient: _finish_hunting 0 2024-09-15T01:46:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 1 monclient: found mon.noname-b 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06b59e640 1 -- 172.21.15.178:0/199664521 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c1331b0 con 0x7fe06c1380e0 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06a59c640 10 monclient: get_auth_request con 0x7fe06c132de0 auth_method 0 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.103+0000 7fe06a59c640 1 --2- 172.21.15.178:0/199664521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c132de0 0x7fe06c137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe06b59e640 1 --2- 172.21.15.178:0/199664521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06c1380e0 0x7fe06c13a4c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009880 tx=0x7fe05c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21354fdce1479099 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe069d9b640 1 -- 172.21.15.178:0/199664521 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe05c02ac50 con 0x7fe06c1380e0 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe069d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe069d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe069d9b640 10 monclient: dump: 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe069d9b640 10 monclient: _finish_auth 0 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe069d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe069d9b640 1 -- 172.21.15.178:0/199664521 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c02add0 con 0x7fe06c1380e0 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe07115d640 5 monclient: authenticate success, global_id 35066 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe07115d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe069d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe07115d640 10 monclient: get_monmap_and_config success 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe069d9b640 1 -- 172.21.15.178:0/199664521 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe05c031ca0 con 0x7fe06c1380e0 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe07115d640 4 set_mon_vals no callback set 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe069d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe069d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.104+0000 7fe069d9b640 10 monclient: dump: 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.105+0000 7fe07115d640 10 monclient: shutdown 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.105+0000 7fe07115d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.105+0000 7fe07115d640 1 -- 172.21.15.178:0/199664521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06c1380e0 msgr2=0x7fe06c13a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.105+0000 7fe07115d640 1 --2- 172.21.15.178:0/199664521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06c1380e0 0x7fe06c13a4c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009880 tx=0x7fe05c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.105+0000 7fe07115d640 1 -- 172.21.15.178:0/199664521 shutdown_connections 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.105+0000 7fe07115d640 1 --2- 172.21.15.178:0/199664521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06c1380e0 0x7fe06c13a4c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.105+0000 7fe07115d640 1 --2- 172.21.15.178:0/199664521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c132de0 0x7fe06c137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.105+0000 7fe07115d640 1 -- 172.21.15.178:0/199664521 >> 172.21.15.178:0/199664521 conn(0x7fe06c0a5ed0 msgr2=0x7fe06c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.105+0000 7fe07115d640 1 -- 172.21.15.178:0/199664521 shutdown_connections 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.105+0000 7fe07115d640 1 -- 172.21.15.178:0/199664521 wait complete. 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.105+0000 7fe07115d640 10 monclient: build_initial_monmap 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.105+0000 7fe07115d640 10 monclient: monmap: 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:46:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:46:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:46:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:46:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:46:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 1 Processor -- start 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 1 -- start start 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 10 monclient: init 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 10 monclient: _add_conns ranks=[2,3,0,7,1,5,8,4,6] 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06c0a2930 0x7fe06c09b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 10 monclient(hunting): picked mon.noname-b con 0x7fe06c0a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c132de0 0x7fe06c132230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 10 monclient(hunting): picked mon.noname-g con 0x7fe06c132de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c1380e0 0x7fe06c141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 10 monclient(hunting): picked mon.noname-a con 0x7fe06c1380e0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c09c810 con 0x7fe06c1380e0 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c0a2390 con 0x7fe06c0a2930 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c09d630 con 0x7fe06c132de0 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe07115d640 10 monclient(hunting): authenticate will time out at 1569.856660s 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06ad9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06c0a2930 0x7fe06c09b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06b59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c1380e0 0x7fe06c141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06ad9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06c0a2930 0x7fe06c09b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:57726/0 (socket says 172.21.15.178:57726) 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06b59e640 10 monclient(hunting): get_auth_request con 0x7fe06c1380e0 auth_method 0 2024-09-15T01:46:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06ad9d640 1 -- 172.21.15.178:0/1977132529 learned_addr learned my addr 172.21.15.178:0/1977132529 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06b59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06ad9d640 10 monclient(hunting): get_auth_request con 0x7fe06c0a2930 auth_method 0 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06a59c640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c132de0 0x7fe06c132230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06a59c640 10 monclient(hunting): get_auth_request con 0x7fe06c132de0 auth_method 0 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.106+0000 7fe06a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06ad9d640 10 monclient(hunting): handle_auth_done global_id 35075 payload 824 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06ad9d640 1 -- 172.21.15.178:0/1977132529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c132de0 msgr2=0x7fe06c132230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06ad9d640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c132de0 0x7fe06c132230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06ad9d640 1 -- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c1380e0 msgr2=0x7fe06c141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06ad9d640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c1380e0 0x7fe06c141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:46:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06ad9d640 1 monclient: found mon.noname-b 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06ad9d640 1 -- 172.21.15.178:0/1977132529 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c009390 con 0x7fe06c0a2930 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06b59e640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c1380e0 0x7fe06c141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06ad9d640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06c0a2930 0x7fe06c09b4f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe06400cc70 tx=0x7fe064001a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe06400ce90 con 0x7fe06c0a2930 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe062ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe062ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe062ffd640 10 monclient: dump: 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe062ffd640 10 monclient: _finish_auth 0 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe062ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06400acd0 con 0x7fe06c0a2930 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe07115d640 5 monclient: authenticate success, global_id 35075 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe07115d640 10 monclient: _renew_subs 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe07115d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe07115d640 1 -- 172.21.15.178:0/1977132529 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06c09bfe0 con 0x7fe06c0a2930 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe062ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe06400ae40 con 0x7fe06c0a2930 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe062ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe062ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe062ffd640 10 monclient: dump: 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe06959a640 4 set_mon_vals no callback set 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe07115d640 10 monclient: _renew_subs 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe07115d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.107+0000 7fe07115d640 1 -- 172.21.15.178:0/1977132529 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06c13f020 con 0x7fe06c0a2930 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe07115d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe07115d640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c13e750 0x7fe06c142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe07115d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe07115d640 1 -- 172.21.15.178:0/1977132529 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c11d4d0 con 0x7fe06c13e750 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe06400a850 con 0x7fe06c0a2930 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06a59c640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c13e750 0x7fe06c142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06a59c640 10 monclient: get_auth_request con 0x7fe06c13e750 auth_method 0 2024-09-15T01:46:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe062ffd640 1 --2- 172.21.15.178:0/1977132529 >> v2:172.21.15.204:6812/28210 conn(0x7fe04c0397b0 0x7fe04c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06a59c640 10 monclient(hunting): handle_auth_done global_id 64305 payload 824 2024-09-15T01:46:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fe06404bc30 con 0x7fe06c0a2930 2024-09-15T01:46:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06a59c640 1 -- 172.21.15.178:0/1977132529 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe06c04c4a0 con 0x7fe06c13e750 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06a59c640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c13e750 0x7fe06c142010 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe06c09bea0 tx=0x7fe054008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06b59e640 1 --2- 172.21.15.178:0/1977132529 >> v2:172.21.15.204:6812/28210 conn(0x7fe04c0397b0 0x7fe04c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe06b59e640 10 monclient: get_auth_request con 0x7fe04c0397b0 auth_method 0 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe054013440 con 0x7fe06c13e750 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe062ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe062ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe062ffd640 10 monclient: dump: 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fe05400c040 con 0x7fe06c13e750 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe062ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.108+0000 7fe062ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:46:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.109+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c13e750 msgr2=0x7fe06c142010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.109+0000 7fe062ffd640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c13e750 0x7fe06c142010 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe06c09bea0 tx=0x7fe054008040 comp rx=0 tx=0).stop 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.109+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c13e750 msgr2=0x7fe06c142010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.109+0000 7fe062ffd640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c13e750 0x7fe06c142010 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe06c09bea0 tx=0x7fe054008040 comp rx=0 tx=0).stop 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.109+0000 7fe06b59e640 1 --2- 172.21.15.178:0/1977132529 >> v2:172.21.15.204:6812/28210 conn(0x7fe04c0397b0 0x7fe04c03bc60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe05c034000 tx=0x7fe05c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe07115d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe07115d640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c142550 0x7fe06c096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe07115d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe07115d640 1 -- 172.21.15.178:0/1977132529 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c04bc80 con 0x7fe06c142550 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe06a59c640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c142550 0x7fe06c096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe06a59c640 10 monclient: get_auth_request con 0x7fe06c142550 auth_method 0 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe06a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe06a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe06a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe06a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe06a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe06a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.114+0000 7fe06a59c640 10 monclient(hunting): handle_auth_done global_id 64314 payload 824 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe06a59c640 1 -- 172.21.15.178:0/1977132529 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe06c05ed70 con 0x7fe06c142550 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe06a59c640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c142550 0x7fe06c096a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe06c09bea0 tx=0x7fe05401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0540067b0 con 0x7fe06c142550 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe062ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe062ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe062ffd640 10 monclient: dump: 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9290 (secure 0 0 0) 0x7fe054006910 con 0x7fe06c142550 2024-09-15T01:46:05.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe062ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:46:05.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe062ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:46:05.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c142550 msgr2=0x7fe06c096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe062ffd640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c142550 0x7fe06c096a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe06c09bea0 tx=0x7fe05401d000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe062ffd640 1 -- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c142550 msgr2=0x7fe06c096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:05.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe062ffd640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe06c142550 0x7fe06c096a30 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe06c09bea0 tx=0x7fe05401d000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.142 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:46:05.142 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 144, 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: 7 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.143 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.144 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:46:05.146 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.147 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:46:05.148 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.149 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:46:05.149 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.149 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.149 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:46:05.149 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:46:05.149 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.149 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.151 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.152 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:46:05.154 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.157 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:46:05.158 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.160 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.161 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.162 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.163 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.164 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.165 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:46:05.165 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.165 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.165 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.165 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.165 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.165 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.165 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.165 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:46:05.165 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:46:05.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe07115d640 1 -- 172.21.15.178:0/1977132529 >> v2:172.21.15.204:6812/28210 conn(0x7fe04c0397b0 msgr2=0x7fe04c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe07115d640 1 --2- 172.21.15.178:0/1977132529 >> v2:172.21.15.204:6812/28210 conn(0x7fe04c0397b0 0x7fe04c03bc60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe05c034000 tx=0x7fe05c032000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe07115d640 10 monclient: shutdown 2024-09-15T01:46:05.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe07115d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe07115d640 1 -- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06c0a2930 msgr2=0x7fe06c09b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe07115d640 1 --2- 172.21.15.178:0/1977132529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06c0a2930 0x7fe06c09b4f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe06400cc70 tx=0x7fe064001a60 comp rx=0 tx=0).stop 2024-09-15T01:46:05.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe06b59e640 1 -- 172.21.15.178:0/1977132529 reap_dead start 2024-09-15T01:46:05.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe06b59e640 1 -- 172.21.15.178:0/1977132529 reap_dead start 2024-09-15T01:46:05.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe07115d640 1 -- 172.21.15.178:0/1977132529 shutdown_connections 2024-09-15T01:46:05.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.115+0000 7fe07115d640 1 -- 172.21.15.178:0/1977132529 >> 172.21.15.178:0/1977132529 conn(0x7fe06c0a5ed0 msgr2=0x7fe06c135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.116+0000 7fe07115d640 1 -- 172.21.15.178:0/1977132529 shutdown_connections 2024-09-15T01:46:05.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.116+0000 7fe07115d640 1 -- 172.21.15.178:0/1977132529 wait complete. 2024-09-15T01:46:05.168 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:46:05.168 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-09-15T01:46:05.168 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:46:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.259+0000 7fd13571d640 10 monclient: get_monmap_and_config 2024-09-15T01:46:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.259+0000 7fd13571d640 10 monclient: build_initial_monmap 2024-09-15T01:46:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.260+0000 7fd13571d640 10 monclient: monmap: 2024-09-15T01:46:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:46:05.260897+0000 2024-09-15T01:46:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:46:05.260897+0000 2024-09-15T01:46:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:46:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:46:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:46:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:46:05.263 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.260+0000 7fd13571d640 1 Processor -- start 2024-09-15T01:46:05.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.260+0000 7fd13571d640 1 -- start start 2024-09-15T01:46:05.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.260+0000 7fd13571d640 10 monclient: init 2024-09-15T01:46:05.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd13571d640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:05.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd13571d640 10 monclient: _add_conns ranks=[0,4,1,3,6,7,2,10,9,5,8] 2024-09-15T01:46:05.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd13571d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd1300a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:46:05.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd13571d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd130132de0 0x7fd130137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd13571d640 10 monclient(hunting): picked mon.noname-c con 0x7fd130132de0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:46:05.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd13571d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd1301380e0 addr v1:172.21.15.204:6789/0 2024-09-15T01:46:05.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd13571d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13004bf60 con 0x7fd1300a2930 2024-09-15T01:46:05.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd13571d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13009dd80 con 0x7fd1301380e0 2024-09-15T01:46:05.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd13571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd13571d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1300a2d00 con 0x7fd130132de0 2024-09-15T01:46:05.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd13571d640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:05.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd13571d640 10 monclient(hunting): authenticate will time out at 1570.011239s 2024-09-15T01:46:05.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12effd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd1300a2930 0x7fd13009faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:47090/0 (socket says 172.21.15.178:47090) 2024-09-15T01:46:05.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12effd640 1 -- 172.21.15.178:0/4205484930 learned_addr learned my addr 172.21.15.178:0/4205484930 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:46:05.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 1 --2- 172.21.15.178:0/4205484930 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd130132de0 0x7fd130137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12f7fe640 1 -- 172.21.15.178:0/4205484930 >> v1:172.21.15.204:6789/0 conn(0x7fd1301380e0 legacy=0x7fd13013a4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:46:05.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 10 monclient(hunting): get_auth_request con 0x7fd130132de0 auth_method 0 2024-09-15T01:46:05.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12dffb640 1 -- 172.21.15.178:0/4205484930 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625961106 0 0) 0x7fd13004bf60 con 0x7fd1300a2930 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12dffb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12dffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12dffb640 10 monclient(hunting): my global_id is 64323 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12dffb640 1 -- 172.21.15.178:0/4205484930 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1180034e0 con 0x7fd1300a2930 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 10 monclient(hunting): handle_auth_done global_id 44716 payload 274 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 1 -- 172.21.15.178:0/4205484930 >> v1:172.21.15.204:6789/0 conn(0x7fd1301380e0 legacy=0x7fd13013a4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 1 -- 172.21.15.178:0/4205484930 >> v1:172.21.15.178:6789/0 conn(0x7fd1300a2930 legacy=0x7fd13009faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:46:05.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 1 monclient: found mon.noname-c 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.261+0000 7fd12e7fc640 1 -- 172.21.15.178:0/4205484930 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1301331b0 con 0x7fd130132de0 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12e7fc640 1 --2- 172.21.15.178:0/4205484930 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd130132de0 0x7fd130137ba0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd13009df10 tx=0x7fd1240090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c995ab635bfad487 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12dffb640 1 -- 172.21.15.178:0/4205484930 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd12402ac50 con 0x7fd130132de0 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12dffb640 10 monclient: dump: 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.276 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12dffb640 10 monclient: _finish_auth 0 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:05.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd13571d640 5 monclient: authenticate success, global_id 44716 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd13571d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12dffb640 1 -- 172.21.15.178:0/4205484930 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12402add0 con 0x7fd130132de0 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd13571d640 10 monclient: get_monmap_and_config success 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd13571d640 4 set_mon_vals no callback set 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12dffb640 1 -- 172.21.15.178:0/4205484930 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd124031ca0 con 0x7fd130132de0 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd12dffb640 10 monclient: dump: 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.278 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd13571d640 10 monclient: shutdown 2024-09-15T01:46:05.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd13571d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.262+0000 7fd13571d640 1 -- 172.21.15.178:0/4205484930 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd130132de0 msgr2=0x7fd130137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.263+0000 7fd13571d640 1 --2- 172.21.15.178:0/4205484930 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd130132de0 0x7fd130137ba0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd13009df10 tx=0x7fd1240090f0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.263+0000 7fd13571d640 1 -- 172.21.15.178:0/4205484930 shutdown_connections 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.263+0000 7fd13571d640 1 --2- 172.21.15.178:0/4205484930 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd130132de0 0x7fd130137ba0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.263+0000 7fd13571d640 1 -- 172.21.15.178:0/4205484930 >> 172.21.15.178:0/4205484930 conn(0x7fd1300a5ed0 msgr2=0x7fd1300a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.263+0000 7fd13571d640 1 -- 172.21.15.178:0/4205484930 shutdown_connections 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.263+0000 7fd13571d640 1 -- 172.21.15.178:0/4205484930 wait complete. 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.263+0000 7fd13571d640 10 monclient: build_initial_monmap 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.263+0000 7fd13571d640 10 monclient: monmap: 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:46:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 1 Processor -- start 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 1 -- start start 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 10 monclient: init 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 10 monclient: _add_conns ranks=[2,7,0,8,5,4,1,3,6] 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1300a2930 0x7fd130132500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 10 monclient(hunting): picked mon.noname-b con 0x7fd1300a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd130132de0 0x7fd13009a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 10 monclient(hunting): picked mon.noname-i con 0x7fd130132de0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1301380e0 0x7fd13009a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 10 monclient(hunting): picked mon.noname-a con 0x7fd1301380e0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1300a2390 con 0x7fd1301380e0 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13009c5c0 con 0x7fd1300a2930 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd13011d9c0 con 0x7fd130132de0 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd13571d640 10 monclient(hunting): authenticate will time out at 1570.014625s 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12effd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1300a2930 0x7fd130132500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1301380e0 0x7fd13009a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12effd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1300a2930 0x7fd130132500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:57736/0 (socket says 172.21.15.178:57736) 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12effd640 1 -- 172.21.15.178:0/588776130 learned_addr learned my addr 172.21.15.178:0/588776130 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12effd640 10 monclient(hunting): get_auth_request con 0x7fd1300a2930 auth_method 0 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 10 monclient(hunting): get_auth_request con 0x7fd1301380e0 auth_method 0 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 10 monclient(hunting): handle_auth_done global_id 64332 payload 824 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 1 -- 172.21.15.178:0/588776130 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd130132de0 msgr2=0x7fd13009a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd130132de0 0x7fd13009a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 1 -- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1300a2930 msgr2=0x7fd130132500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1300a2930 0x7fd130132500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 1 monclient: found mon.noname-a 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.264+0000 7fd12f7fe640 1 -- 172.21.15.178:0/588776130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd124009390 con 0x7fd1301380e0 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd12f7fe640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1301380e0 0x7fd13009a9b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd12800b3f0 tx=0x7fd128006f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd128006150 con 0x7fd1301380e0 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd116ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd116ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd116ffd640 10 monclient: dump: 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.286 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd116ffd640 10 monclient: _finish_auth 0 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd116ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1280062d0 con 0x7fd1301380e0 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd13571d640 5 monclient: authenticate success, global_id 64332 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd13571d640 10 monclient: _renew_subs 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd13571d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd13571d640 1 -- 172.21.15.178:0/588776130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13011c750 con 0x7fd1301380e0 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd116ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd128010360 con 0x7fd1301380e0 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd116ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd116ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd116ffd640 10 monclient: dump: 2024-09-15T01:46:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd12d7fa640 4 set_mon_vals no callback set 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd13571d640 10 monclient: _renew_subs 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd13571d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.265+0000 7fd13571d640 1 -- 172.21.15.178:0/588776130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13011e1a0 con 0x7fd1301380e0 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd128006480 con 0x7fd1301380e0 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd116ffd640 1 --2- 172.21.15.178:0/588776130 >> v2:172.21.15.204:6812/28210 conn(0x7fd11c0397b0 0x7fd11c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd13571d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd13571d640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd13009aef0 0x7fd130096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd13571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd13571d640 1 -- 172.21.15.178:0/588776130 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13009e3b0 con 0x7fd13009aef0 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd12804abd0 con 0x7fd1301380e0 2024-09-15T01:46:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12e7fc640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd13009aef0 0x7fd130096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12e7fc640 10 monclient: get_auth_request con 0x7fd13009aef0 auth_method 0 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12e7fc640 10 monclient(hunting): handle_auth_done global_id 35093 payload 824 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12e7fc640 1 -- 172.21.15.178:0/588776130 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd13004c4a0 con 0x7fd13009aef0 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12effd640 1 --2- 172.21.15.178:0/588776130 >> v2:172.21.15.204:6812/28210 conn(0x7fd11c0397b0 0x7fd11c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12effd640 10 monclient: get_auth_request con 0x7fd11c0397b0 auth_method 0 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd12e7fc640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd13009aef0 0x7fd130096a30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd1300a2f70 tx=0x7fd124032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd12402ac50 con 0x7fd13009aef0 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd116ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd116ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.266+0000 7fd116ffd640 10 monclient: dump: 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.291 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.267+0000 7fd12effd640 1 --2- 172.21.15.178:0/588776130 >> v2:172.21.15.204:6812/28210 conn(0x7fd11c0397b0 0x7fd11c03bc60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd120001560 tx=0x7fd120009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.267+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fd1240313a0 con 0x7fd13009aef0 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.267+0000 7fd116ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.267+0000 7fd116ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.267+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd13009aef0 msgr2=0x7fd130096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.267+0000 7fd116ffd640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd13009aef0 0x7fd130096a30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd1300a2f70 tx=0x7fd124032000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.267+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd13009aef0 msgr2=0x7fd130096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:05.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.267+0000 7fd116ffd640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd13009aef0 0x7fd130096a30 secure :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd1300a2f70 tx=0x7fd124032000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.272+0000 7fd13571d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:46:05.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.272+0000 7fd13571d640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd130096f70 0x7fd130097370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.272+0000 7fd13571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.272+0000 7fd13571d640 1 -- 172.21.15.178:0/588776130 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13004bc80 con 0x7fd130096f70 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd12e7fc640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd130096f70 0x7fd130097370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd12e7fc640 10 monclient: get_auth_request con 0x7fd130096f70 auth_method 0 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd12e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd12e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd12e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd12e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd12e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd12e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd12e7fc640 10 monclient(hunting): handle_auth_done global_id 35102 payload 824 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd12e7fc640 1 -- 172.21.15.178:0/588776130 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd13005ed70 con 0x7fd130096f70 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd12e7fc640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd130096f70 0x7fd130097370 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd1240097d0 tx=0x7fd12403a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd124031e40 con 0x7fd130096f70 2024-09-15T01:46:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd116ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd116ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd116ffd640 10 monclient: dump: 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.297 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9290 (secure 0 0 0) 0x7fd1240319c0 con 0x7fd130096f70 2024-09-15T01:46:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd116ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:46:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd116ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:46:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd130096f70 msgr2=0x7fd130097370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd116ffd640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd130096f70 0x7fd130097370 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd1240097d0 tx=0x7fd12403a000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd116ffd640 1 -- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd130096f70 msgr2=0x7fd130097370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.273+0000 7fd116ffd640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd130096f70 0x7fd130097370 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.298 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:46:05.298 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:46:05.298 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:46:05.298 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:46:05.298 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 144, 2024-09-15T01:46:05.298 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:46:05.298 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:46:05.298 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:46:05.298 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:46:05.298 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:46:05.298 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:46:05.298 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: 7 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.299 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.300 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:46:05.301 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:46:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:46:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.307 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.309 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.312 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.312 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:46:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:46:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.323 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.323 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.323 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:46:05.323 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:46:05.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.274+0000 7fd13571d640 1 -- 172.21.15.178:0/588776130 >> v2:172.21.15.204:6812/28210 conn(0x7fd11c0397b0 msgr2=0x7fd11c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.274+0000 7fd13571d640 1 --2- 172.21.15.178:0/588776130 >> v2:172.21.15.204:6812/28210 conn(0x7fd11c0397b0 0x7fd11c03bc60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd120001560 tx=0x7fd120009040 comp rx=0 tx=0).stop 2024-09-15T01:46:05.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.274+0000 7fd13571d640 10 monclient: shutdown 2024-09-15T01:46:05.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.274+0000 7fd13571d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.274+0000 7fd13571d640 1 -- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1301380e0 msgr2=0x7fd13009a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.274+0000 7fd13571d640 1 --2- 172.21.15.178:0/588776130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1301380e0 0x7fd13009a9b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd12800b3f0 tx=0x7fd128006f70 comp rx=0 tx=0).stop 2024-09-15T01:46:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.274+0000 7fd12f7fe640 1 -- 172.21.15.178:0/588776130 reap_dead start 2024-09-15T01:46:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.274+0000 7fd12f7fe640 1 -- 172.21.15.178:0/588776130 reap_dead start 2024-09-15T01:46:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.274+0000 7fd13571d640 1 -- 172.21.15.178:0/588776130 shutdown_connections 2024-09-15T01:46:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.274+0000 7fd13571d640 1 -- 172.21.15.178:0/588776130 >> 172.21.15.178:0/588776130 conn(0x7fd1300a5ed0 msgr2=0x7fd13009f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.274+0000 7fd13571d640 1 -- 172.21.15.178:0/588776130 shutdown_connections 2024-09-15T01:46:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.274+0000 7fd13571d640 1 -- 172.21.15.178:0/588776130 wait complete. 2024-09-15T01:46:05.325 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:46:05.325 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-09-15T01:46:05.326 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:46:05.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.417+0000 7f1883c94640 10 monclient: get_monmap_and_config 2024-09-15T01:46:05.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.417+0000 7f1883c94640 10 monclient: build_initial_monmap 2024-09-15T01:46:05.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.417+0000 7f1883c94640 10 monclient: monmap: 2024-09-15T01:46:05.419 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:05.419 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.419 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:46:05.418177+0000 2024-09-15T01:46:05.419 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:46:05.418177+0000 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:46:05.420 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.417+0000 7f1883c94640 1 Processor -- start 2024-09-15T01:46:05.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 1 -- start start 2024-09-15T01:46:05.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 10 monclient: init 2024-09-15T01:46:05.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:05.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 10 monclient: _add_conns ranks=[0,8,6,4,10,5,2,3,7,1,9] 2024-09-15T01:46:05.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f187c0a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:46:05.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f187c132de0 0x7f187c137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 10 monclient(hunting): picked mon.noname-e con 0x7f187c132de0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:46:05.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c1380e0 0x7f187c13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 10 monclient(hunting): picked mon.noname-d con 0x7f187c1380e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:46:05.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187c04bf60 con 0x7f187c0a2930 2024-09-15T01:46:05.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f187c09dd80 con 0x7f187c1380e0 2024-09-15T01:46:05.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0a2d00 con 0x7f187c132de0 2024-09-15T01:46:05.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:05.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1883c94640 10 monclient(hunting): authenticate will time out at 1570.168565s 2024-09-15T01:46:05.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f188122f640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f187c132de0 msgr2=0x7f187c137ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:46:05.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f188122f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f187c132de0 0x7f187c137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:46:05.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1881a30640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f187c0a2930 0x7f187c09faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:47102/0 (socket says 172.21.15.178:47102) 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1881a30640 1 -- 172.21.15.178:0/3212751288 learned_addr learned my addr 172.21.15.178:0/3212751288 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 1 --2- 172.21.15.178:0/3212751288 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c1380e0 0x7f187c13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 10 monclient(hunting): get_auth_request con 0x7f187c1380e0 auth_method 0 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 10 monclient(hunting): handle_auth_done global_id 25575 payload 274 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 1 -- 172.21.15.178:0/3212751288 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f187c132de0 msgr2=0x7f187c137ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 1 --2- 172.21.15.178:0/3212751288 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f187c132de0 0x7f187c137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1880a2e640 1 -- 172.21.15.178:0/3212751288 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715640700 0 0) 0x7f187c04bf60 con 0x7f187c0a2930 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 1 -- 172.21.15.178:0/3212751288 >> v1:172.21.15.178:6789/0 conn(0x7f187c0a2930 legacy=0x7f187c09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 10 monclient: _finish_hunting 0 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 1 monclient: found mon.noname-d 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1882231640 1 -- 172.21.15.178:0/3212751288 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c1331b0 con 0x7f187c1380e0 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.418+0000 7f1880a2e640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1882231640 1 --2- 172.21.15.178:0/3212751288 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c1380e0 0x7f187c13a4c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1874009e90 tx=0x7f1874009780 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f4d73b06e96f724e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1880a2e640 1 -- 172.21.15.178:0/3212751288 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f18740079d0 con 0x7f187c1380e0 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1880a2e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1880a2e640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1880a2e640 10 monclient: dump: 2024-09-15T01:46:05.433 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.434 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.434 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.434 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.434 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.434 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.434 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.434 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.434 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.434 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.434 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.434 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1880a2e640 10 monclient: _finish_auth 0 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1880a2e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1883c94640 5 monclient: authenticate success, global_id 25575 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1883c94640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1880a2e640 1 -- 172.21.15.178:0/3212751288 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187402d050 con 0x7f187c1380e0 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1880a2e640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1883c94640 10 monclient: get_monmap_and_config success 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1883c94640 4 set_mon_vals no callback set 2024-09-15T01:46:05.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1880a2e640 1 -- 172.21.15.178:0/3212751288 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1874031400 con 0x7f187c1380e0 2024-09-15T01:46:05.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1880a2e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1880a2e640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:46:05.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.419+0000 7f1880a2e640 10 monclient: dump: 2024-09-15T01:46:05.436 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.436 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.436 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.436 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.436 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.436 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.436 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.436 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.436 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.420+0000 7f1883c94640 10 monclient: shutdown 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.420+0000 7f1883c94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.420+0000 7f1883c94640 1 -- 172.21.15.178:0/3212751288 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c1380e0 msgr2=0x7f187c13a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.420+0000 7f1883c94640 1 --2- 172.21.15.178:0/3212751288 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c1380e0 0x7f187c13a4c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1874009e90 tx=0x7f1874009780 comp rx=0 tx=0).stop 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.420+0000 7f1883c94640 1 -- 172.21.15.178:0/3212751288 shutdown_connections 2024-09-15T01:46:05.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.420+0000 7f1883c94640 1 --2- 172.21.15.178:0/3212751288 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c1380e0 0x7f187c13a4c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.420+0000 7f1883c94640 1 --2- 172.21.15.178:0/3212751288 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f187c132de0 0x7f187c137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.420+0000 7f1883c94640 1 -- 172.21.15.178:0/3212751288 >> 172.21.15.178:0/3212751288 conn(0x7f187c0a5ed0 msgr2=0x7f187c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.420+0000 7f1883c94640 1 -- 172.21.15.178:0/3212751288 shutdown_connections 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.420+0000 7f1883c94640 1 -- 172.21.15.178:0/3212751288 wait complete. 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.420+0000 7f1883c94640 10 monclient: build_initial_monmap 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.420+0000 7f1883c94640 10 monclient: monmap: 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:46:05.438 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 1 Processor -- start 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 1 -- start start 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 10 monclient: init 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:05.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 10 monclient: _add_conns ranks=[6,2,4,0,8,1,5,3,7] 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c0a2930 0x7f187c13e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 10 monclient(hunting): picked mon.noname-d con 0x7f187c0a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f187c132de0 0x7f187c132200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 10 monclient(hunting): picked mon.noname-b con 0x7f187c132de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c1380e0 0x7f187c141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 10 monclient(hunting): picked mon.noname-c con 0x7f187c1380e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187c11d300 con 0x7f187c132de0 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f187c11d5e0 con 0x7f187c1380e0 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f187c11d8c0 con 0x7f187c0a2930 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:05.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1883c94640 10 monclient(hunting): authenticate will time out at 1570.171709s 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1881a30640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c0a2930 0x7f187c13e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1881a30640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c0a2930 0x7f187c13e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:49014/0 (socket says 172.21.15.178:49014) 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1881a30640 1 -- 172.21.15.178:0/1696704166 learned_addr learned my addr 172.21.15.178:0/1696704166 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1881a30640 10 monclient(hunting): get_auth_request con 0x7f187c0a2930 auth_method 0 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1881a30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1881a30640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1881a30640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f188122f640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f187c132de0 0x7f187c132200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1882231640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c1380e0 0x7f187c141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f188122f640 10 monclient(hunting): get_auth_request con 0x7f187c132de0 auth_method 0 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f188122f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f188122f640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f188122f640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1882231640 10 monclient(hunting): get_auth_request con 0x7f187c1380e0 auth_method 0 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1882231640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1882231640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1882231640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1881a30640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1881a30640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1881a30640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f188122f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f188122f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f188122f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1882231640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1882231640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.421+0000 7f1882231640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1881a30640 10 monclient(hunting): handle_auth_done global_id 25584 payload 824 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1881a30640 1 -- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c1380e0 msgr2=0x7f187c141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1881a30640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c1380e0 0x7f187c141a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1881a30640 1 -- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f187c132de0 msgr2=0x7f187c132200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1881a30640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f187c132de0 0x7f187c132200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1881a30640 10 monclient: _finish_hunting 0 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1881a30640 1 monclient: found mon.noname-d 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1881a30640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1881a30640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1881a30640 1 -- 172.21.15.178:0/1696704166 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1874009a20 con 0x7f187c0a2930 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f188122f640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f187c132de0 0x7f187c132200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1881a30640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c0a2930 0x7f187c13e580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f187800c510 tx=0x7f1878007f30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f18780070d0 con 0x7f187c0a2930 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1871ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1871ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1871ffb640 10 monclient: dump: 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.444 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1871ffb640 10 monclient: _finish_auth 0 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1871ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1878007250 con 0x7f187c0a2930 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1883c94640 5 monclient: authenticate success, global_id 25584 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1883c94640 10 monclient: _renew_subs 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1883c94640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1883c94640 1 -- 172.21.15.178:0/1696704166 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187c13f090 con 0x7f187c0a2930 2024-09-15T01:46:05.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1871ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1878011360 con 0x7f187c0a2930 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1871ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1871ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1871ffb640 10 monclient: dump: 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.446 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1883c94640 10 monclient: _renew_subs 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1883c94640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1873fff640 4 set_mon_vals no callback set 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.422+0000 7f1883c94640 1 -- 172.21.15.178:0/1696704166 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187c11c530 con 0x7f187c0a2930 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1878007400 con 0x7f187c0a2930 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1883c94640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1871ffb640 1 --2- 172.21.15.178:0/1696704166 >> v2:172.21.15.204:6812/28210 conn(0x7f18600396e0 0x7f186003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1883c94640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c132740 0x7f187c141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1883c94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1883c94640 1 -- 172.21.15.178:0/1696704166 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f187c11d750 con 0x7f187c132740 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f187804aca0 con 0x7f187c0a2930 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1882231640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c132740 0x7f187c141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1882231640 10 monclient: get_auth_request con 0x7f187c132740 auth_method 0 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1882231640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1882231640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1882231640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1882231640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1882231640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1882231640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f188122f640 1 --2- 172.21.15.178:0/1696704166 >> v2:172.21.15.204:6812/28210 conn(0x7f18600396e0 0x7f186003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f188122f640 10 monclient: get_auth_request con 0x7f18600396e0 auth_method 0 2024-09-15T01:46:05.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1882231640 10 monclient(hunting): handle_auth_done global_id 44734 payload 824 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.423+0000 7f1882231640 1 -- 172.21.15.178:0/1696704166 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f187c04c4a0 con 0x7f187c132740 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f1882231640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c132740 0x7f187c141fd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f187c0a2f70 tx=0x7f1874037000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1874031400 con 0x7f187c132740 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f1871ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f1871ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f1871ffb640 10 monclient: dump: 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.449 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f188122f640 1 --2- 172.21.15.178:0/1696704166 >> v2:172.21.15.204:6812/28210 conn(0x7f18600396e0 0x7f186003bb90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f186c003850 tx=0x7f186c009060 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f187403eb30 con 0x7f187c132740 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f1871ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f1871ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c132740 msgr2=0x7f187c141fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f1871ffb640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c132740 0x7f187c141fd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f187c0a2f70 tx=0x7f1874037000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c132740 msgr2=0x7f187c141fd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:05.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.424+0000 7f1871ffb640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c132740 0x7f187c141fd0 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f187c0a2f70 tx=0x7f1874037000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1883c94640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1883c94640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c09b600 0x7f187c142510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1883c94640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1883c94640 1 -- 172.21.15.178:0/1696704166 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f187c04bc80 con 0x7f187c09b600 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1882231640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c09b600 0x7f187c142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1882231640 10 monclient: get_auth_request con 0x7f187c09b600 auth_method 0 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1882231640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1882231640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1882231640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1882231640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1882231640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1882231640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1882231640 10 monclient(hunting): handle_auth_done global_id 44743 payload 824 2024-09-15T01:46:05.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1882231640 1 -- 172.21.15.178:0/1696704166 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f187c05ed70 con 0x7f187c09b600 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1882231640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c09b600 0x7f187c142510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f18740097c0 tx=0x7f1874040040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1874046870 con 0x7f187c09b600 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1871ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1871ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1871ffb640 10 monclient: dump: 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.454 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9460 (secure 0 0 0) 0x7f18740463f0 con 0x7f187c09b600 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1871ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.430+0000 7f1871ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c09b600 msgr2=0x7f187c142510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1871ffb640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c09b600 0x7f187c142510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f18740097c0 tx=0x7f1874040040 comp rx=0 tx=0).stop 2024-09-15T01:46:05.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1871ffb640 1 -- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c09b600 msgr2=0x7f187c142510 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:05.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1871ffb640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f187c09b600 0x7f187c142510 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f18740097c0 tx=0x7f1874040040 comp rx=0 tx=0).stop 2024-09-15T01:46:05.456 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:46:05.456 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:46:05.456 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:46:05.456 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:46:05.456 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 144, 2024-09-15T01:46:05.456 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:46:05.456 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: 7 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:46:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:46:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:46:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:46:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:46:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:46:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:46:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:46:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:46:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:46:05.479 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:46:05.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1883c94640 1 -- 172.21.15.178:0/1696704166 >> v2:172.21.15.204:6812/28210 conn(0x7f18600396e0 msgr2=0x7f186003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1883c94640 1 --2- 172.21.15.178:0/1696704166 >> v2:172.21.15.204:6812/28210 conn(0x7f18600396e0 0x7f186003bb90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f186c003850 tx=0x7f186c009060 comp rx=0 tx=0).stop 2024-09-15T01:46:05.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1883c94640 10 monclient: shutdown 2024-09-15T01:46:05.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1883c94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1883c94640 1 -- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c0a2930 msgr2=0x7f187c13e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1882231640 1 -- 172.21.15.178:0/1696704166 reap_dead start 2024-09-15T01:46:05.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1883c94640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c0a2930 0x7f187c13e580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f187800c510 tx=0x7f1878007f30 comp rx=0 tx=0).stop 2024-09-15T01:46:05.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1883c94640 1 -- 172.21.15.178:0/1696704166 shutdown_connections 2024-09-15T01:46:05.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1883c94640 1 --2- 172.21.15.178:0/1696704166 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f187c0a2930 0x7f187c13e580 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1883c94640 1 -- 172.21.15.178:0/1696704166 >> 172.21.15.178:0/1696704166 conn(0x7f187c0a5ed0 msgr2=0x7f187c0a4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1883c94640 1 -- 172.21.15.178:0/1696704166 shutdown_connections 2024-09-15T01:46:05.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.431+0000 7f1883c94640 1 -- 172.21.15.178:0/1696704166 wait complete. 2024-09-15T01:46:05.482 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:46:05.482 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-15T01:46:05.482 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:46:05.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.575+0000 7f420179d640 10 monclient: get_monmap_and_config 2024-09-15T01:46:05.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.575+0000 7f420179d640 10 monclient: build_initial_monmap 2024-09-15T01:46:05.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.575+0000 7f420179d640 10 monclient: monmap: 2024-09-15T01:46:05.578 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:05.578 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.578 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:46:05.576189+0000 2024-09-15T01:46:05.578 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:46:05.576189+0000 2024-09-15T01:46:05.578 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:05.578 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.578 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:46:05.578 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:46:05.578 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:05.578 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:05.580 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:05.580 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:05.580 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:05.581 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:05.581 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:05.581 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:46:05.581 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:46:05.581 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.575+0000 7f420179d640 1 Processor -- start 2024-09-15T01:46:05.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.575+0000 7f420179d640 1 -- start start 2024-09-15T01:46:05.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.575+0000 7f420179d640 10 monclient: init 2024-09-15T01:46:05.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:05.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 10 monclient: _add_conns ranks=[0,5,6,1,7,8,2,3,9,4,10] 2024-09-15T01:46:05.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f41fc0a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f41fc132de0 0x7f41fc137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 10 monclient(hunting): picked mon.noname-h con 0x7f41fc132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc1380e0 0x7f41fc13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 10 monclient(hunting): picked mon.noname-d con 0x7f41fc1380e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41fc04bf60 con 0x7f41fc0a2930 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc09dd80 con 0x7f41fc132de0 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0a2d00 con 0x7f41fc1380e0 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f420179d640 10 monclient(hunting): authenticate will time out at 1570.326445s 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41faffd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f41fc0a2930 0x7f41fc09faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:47114/0 (socket says 172.21.15.178:47114) 2024-09-15T01:46:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41faffd640 1 -- 172.21.15.178:0/2121834682 learned_addr learned my addr 172.21.15.178:0/2121834682 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc1380e0 0x7f41fc13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 10 monclient(hunting): get_auth_request con 0x7f41fc1380e0 auth_method 0 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41f27fc640 1 --2- 172.21.15.178:0/2121834682 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f41fc132de0 0x7f41fc137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 10 monclient(hunting): handle_auth_done global_id 25593 payload 274 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 1 -- 172.21.15.178:0/2121834682 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f41fc132de0 msgr2=0x7f41fc137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 1 --2- 172.21.15.178:0/2121834682 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f41fc132de0 0x7f41fc137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 1 -- 172.21.15.178:0/2121834682 >> v1:172.21.15.178:6789/0 conn(0x7f41fc0a2930 legacy=0x7f41fc09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:46:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 1 monclient: found mon.noname-d 2024-09-15T01:46:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:46:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fa7fc640 1 -- 172.21.15.178:0/2121834682 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41663906 0 0) 0x7f41fc04bf60 con 0x7f41fc0a2930 2024-09-15T01:46:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fb7fe640 1 -- 172.21.15.178:0/2121834682 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc1331b0 con 0x7f41fc1380e0 2024-09-15T01:46:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.576+0000 7f41fa7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:46:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fb7fe640 1 --2- 172.21.15.178:0/2121834682 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc1380e0 0x7f41fc13a4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009e90 tx=0x7f41f4009780 comp rx=0 tx=0).ready entity=mon.6 client_cookie=74d3747279b800ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fa7fc640 1 -- 172.21.15.178:0/2121834682 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f41f4031880 con 0x7f41fc1380e0 2024-09-15T01:46:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fa7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:46:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fa7fc640 10 monclient: dump: 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.598 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fa7fc640 10 monclient: _finish_auth 0 2024-09-15T01:46:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f420179d640 5 monclient: authenticate success, global_id 25593 2024-09-15T01:46:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f420179d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:46:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fa7fc640 1 -- 172.21.15.178:0/2121834682 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f402d050 con 0x7f41fc1380e0 2024-09-15T01:46:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f420179d640 10 monclient: get_monmap_and_config success 2024-09-15T01:46:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f420179d640 4 set_mon_vals no callback set 2024-09-15T01:46:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fa7fc640 1 -- 172.21.15.178:0/2121834682 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f41f40303a0 con 0x7f41fc1380e0 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fa7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.577+0000 7f41fa7fc640 10 monclient: dump: 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.600 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.578+0000 7f420179d640 10 monclient: shutdown 2024-09-15T01:46:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.578+0000 7f420179d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.578+0000 7f420179d640 1 -- 172.21.15.178:0/2121834682 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc1380e0 msgr2=0x7f41fc13a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.578+0000 7f420179d640 1 --2- 172.21.15.178:0/2121834682 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc1380e0 0x7f41fc13a4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009e90 tx=0x7f41f4009780 comp rx=0 tx=0).stop 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.578+0000 7f420179d640 1 -- 172.21.15.178:0/2121834682 shutdown_connections 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.578+0000 7f420179d640 1 --2- 172.21.15.178:0/2121834682 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc1380e0 0x7f41fc13a4c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.578+0000 7f420179d640 1 --2- 172.21.15.178:0/2121834682 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f41fc132de0 0x7f41fc137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.578+0000 7f420179d640 1 -- 172.21.15.178:0/2121834682 >> 172.21.15.178:0/2121834682 conn(0x7f41fc0a5ed0 msgr2=0x7f41fc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.578+0000 7f420179d640 1 -- 172.21.15.178:0/2121834682 shutdown_connections 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.578+0000 7f420179d640 1 -- 172.21.15.178:0/2121834682 wait complete. 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.578+0000 7f420179d640 10 monclient: build_initial_monmap 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.578+0000 7f420179d640 10 monclient: monmap: 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:46:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 1 Processor -- start 2024-09-15T01:46:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 1 -- start start 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 10 monclient: init 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 10 monclient: _add_conns ranks=[2,1,8,4,3,6,7,5,0] 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41fc0a2930 0x7f41fc09c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 10 monclient(hunting): picked mon.noname-b con 0x7f41fc0a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41fc132de0 0x7f41fc134fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 10 monclient(hunting): picked mon.noname-f con 0x7f41fc132de0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f41fc1380e0 0x7f41fc135510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 10 monclient(hunting): picked mon.noname-e con 0x7f41fc1380e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0a2390 con 0x7f41fc132de0 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc09d630 con 0x7f41fc0a2930 2024-09-15T01:46:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc097950 con 0x7f41fc1380e0 2024-09-15T01:46:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f420179d640 10 monclient(hunting): authenticate will time out at 1570.329770s 2024-09-15T01:46:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41fb7fe640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f41fc1380e0 msgr2=0x7f41fc135510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:46:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41fb7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f41fc1380e0 0x7f41fc135510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:46:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41f27fc640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41fc132de0 msgr2=0x7f41fc134fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:46:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41f27fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41fc132de0 0x7f41fc134fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:46:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41faffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41fc0a2930 0x7f41fc09c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41faffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41fc0a2930 0x7f41fc09c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:57760/0 (socket says 172.21.15.178:57760) 2024-09-15T01:46:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41faffd640 1 -- 172.21.15.178:0/678019137 learned_addr learned my addr 172.21.15.178:0/678019137 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:46:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41faffd640 10 monclient(hunting): get_auth_request con 0x7f41fc0a2930 auth_method 0 2024-09-15T01:46:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41faffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41faffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.579+0000 7f41faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41faffd640 10 monclient(hunting): handle_auth_done global_id 35120 payload 824 2024-09-15T01:46:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41faffd640 1 -- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f41fc1380e0 msgr2=0x7f41fc135510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:46:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41faffd640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f41fc1380e0 0x7f41fc135510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41faffd640 1 -- 172.21.15.178:0/678019137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41fc132de0 msgr2=0x7f41fc134fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:46:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41faffd640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41fc132de0 0x7f41fc134fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41faffd640 10 monclient: _finish_hunting 0 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41faffd640 1 monclient: found mon.noname-b 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41faffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41faffd640 1 -- 172.21.15.178:0/678019137 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f4009a20 con 0x7f41fc0a2930 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41faffd640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41fc0a2930 0x7f41fc09c550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00c540 tx=0x7f41ec007f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f41ec0070d0 con 0x7f41fc0a2930 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f37fe640 10 monclient: dump: 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.607 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f37fe640 10 monclient: _finish_auth 0 2024-09-15T01:46:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f420179d640 5 monclient: authenticate success, global_id 35120 2024-09-15T01:46:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec007270 con 0x7f41fc0a2930 2024-09-15T01:46:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f420179d640 10 monclient: _renew_subs 2024-09-15T01:46:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f41ec011360 con 0x7f41fc0a2930 2024-09-15T01:46:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f420179d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:46:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f420179d640 1 -- 172.21.15.178:0/678019137 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc097c30 con 0x7f41fc0a2930 2024-09-15T01:46:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f9ffb640 4 set_mon_vals no callback set 2024-09-15T01:46:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f41f37fe640 10 monclient: dump: 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.609 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f420179d640 10 monclient: _renew_subs 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f420179d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.580+0000 7f420179d640 1 -- 172.21.15.178:0/678019137 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc136f40 con 0x7f41fc0a2930 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f420179d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f41ec0073e0 con 0x7f41fc0a2930 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f420179d640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc09b960 0x7f41fc135a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f420179d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f420179d640 1 -- 172.21.15.178:0/678019137 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc04c4a0 con 0x7f41fc09b960 2024-09-15T01:46:05.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f27fc640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc09b960 0x7f41fc135a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f27fc640 10 monclient: get_auth_request con 0x7f41fc09b960 auth_method 0 2024-09-15T01:46:05.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f37fe640 1 --2- 172.21.15.178:0/678019137 >> v2:172.21.15.204:6812/28210 conn(0x7f41dc0397b0 0x7f41dc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f41ec04ac40 con 0x7f41fc0a2930 2024-09-15T01:46:05.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f27fc640 10 monclient(hunting): handle_auth_done global_id 25602 payload 824 2024-09-15T01:46:05.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f27fc640 1 -- 172.21.15.178:0/678019137 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41fc05ed70 con 0x7f41fc09b960 2024-09-15T01:46:05.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41fb7fe640 1 --2- 172.21.15.178:0/678019137 >> v2:172.21.15.204:6812/28210 conn(0x7f41dc0397b0 0x7f41dc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41fb7fe640 10 monclient: get_auth_request con 0x7f41dc0397b0 auth_method 0 2024-09-15T01:46:05.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.581+0000 7f41f27fc640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc09b960 0x7f41fc135a50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c700 tx=0x7f41e4008040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.582+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f41e4011a80 con 0x7f41fc09b960 2024-09-15T01:46:05.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.582+0000 7f41f37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.582+0000 7f41f37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:46:05.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.582+0000 7f41f37fe640 10 monclient: dump: 2024-09-15T01:46:05.612 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.612 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.612 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.612 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.613 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.582+0000 7f41fb7fe640 1 --2- 172.21.15.178:0/678019137 >> v2:172.21.15.204:6812/28210 conn(0x7f41dc0397b0 0x7f41dc03bc60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f41f4034000 tx=0x7f41f4037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.582+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f41e4011be0 con 0x7f41fc09b960 2024-09-15T01:46:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.582+0000 7f41f37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.582+0000 7f41f37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:46:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.582+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc09b960 msgr2=0x7f41fc135a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.582+0000 7f41f37fe640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc09b960 0x7f41fc135a50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f41e400c700 tx=0x7f41e4008040 comp rx=0 tx=0).stop 2024-09-15T01:46:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.582+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc09b960 msgr2=0x7f41fc135a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.582+0000 7f41f37fe640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc09b960 0x7f41fc135a50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.587+0000 7f420179d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:46:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.587+0000 7f420179d640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc131010 0x7f41fc131410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.587+0000 7f420179d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.587+0000 7f420179d640 1 -- 172.21.15.178:0/678019137 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc04bc80 con 0x7f41fc131010 2024-09-15T01:46:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f27fc640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc131010 0x7f41fc131410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f27fc640 10 monclient: get_auth_request con 0x7f41fc131010 auth_method 0 2024-09-15T01:46:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f27fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f27fc640 10 monclient(hunting): handle_auth_done global_id 25611 payload 824 2024-09-15T01:46:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f27fc640 1 -- 172.21.15.178:0/678019137 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f41fc04b720 con 0x7f41fc131010 2024-09-15T01:46:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f27fc640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc131010 0x7f41fc131410 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f41e4017000 tx=0x7f41e401a000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f41e4020070 con 0x7f41fc131010 2024-09-15T01:46:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:46:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f37fe640 10 monclient: dump: 2024-09-15T01:46:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9120 (secure 0 0 0) 0x7f41e40083d0 con 0x7f41fc131010 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f37fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:46:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc131010 msgr2=0x7f41fc131410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f37fe640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc131010 0x7f41fc131410 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f41e4017000 tx=0x7f41e401a000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f37fe640 1 -- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc131010 msgr2=0x7f41fc131410 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:05.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.588+0000 7f41f37fe640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f41fc131010 0x7f41fc131410 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.618 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:46:05.618 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:46:05.618 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 144, 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: 7 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:46:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:46:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:46:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:46:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:46:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:46:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:46:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:46:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:46:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:46:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:46:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:46:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:46:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:46:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:46:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:46:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:46:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:46:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:46:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:46:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:46:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:46:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:46:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:46:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:46:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:46:05.642 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:46:05.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.589+0000 7f420179d640 1 -- 172.21.15.178:0/678019137 >> v2:172.21.15.204:6812/28210 conn(0x7f41dc0397b0 msgr2=0x7f41dc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.589+0000 7f420179d640 1 --2- 172.21.15.178:0/678019137 >> v2:172.21.15.204:6812/28210 conn(0x7f41dc0397b0 0x7f41dc03bc60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f41f4034000 tx=0x7f41f4037000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.589+0000 7f420179d640 10 monclient: shutdown 2024-09-15T01:46:05.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.589+0000 7f420179d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.589+0000 7f420179d640 1 -- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41fc0a2930 msgr2=0x7f41fc09c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.589+0000 7f420179d640 1 --2- 172.21.15.178:0/678019137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41fc0a2930 0x7f41fc09c550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00c540 tx=0x7f41ec007f30 comp rx=0 tx=0).stop 2024-09-15T01:46:05.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.589+0000 7f41fb7fe640 1 -- 172.21.15.178:0/678019137 reap_dead start 2024-09-15T01:46:05.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.589+0000 7f41fb7fe640 1 -- 172.21.15.178:0/678019137 reap_dead start 2024-09-15T01:46:05.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.589+0000 7f420179d640 1 -- 172.21.15.178:0/678019137 shutdown_connections 2024-09-15T01:46:05.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.589+0000 7f420179d640 1 -- 172.21.15.178:0/678019137 >> 172.21.15.178:0/678019137 conn(0x7f41fc0a5ed0 msgr2=0x7f41fc12fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.589+0000 7f420179d640 1 -- 172.21.15.178:0/678019137 shutdown_connections 2024-09-15T01:46:05.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.589+0000 7f420179d640 1 -- 172.21.15.178:0/678019137 wait complete. 2024-09-15T01:46:05.645 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:46:05.646 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-09-15T01:46:05.646 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:46:05.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.737+0000 7f15ed91d640 10 monclient: get_monmap_and_config 2024-09-15T01:46:05.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.737+0000 7f15ed91d640 10 monclient: build_initial_monmap 2024-09-15T01:46:05.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.737+0000 7f15ed91d640 10 monclient: monmap: 2024-09-15T01:46:05.740 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:05.740 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.740 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:46:05.738429+0000 2024-09-15T01:46:05.740 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:46:05.738429+0000 2024-09-15T01:46:05.740 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:05.740 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.741 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:46:05.741 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:46:05.741 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:05.741 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:05.741 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:05.741 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:05.741 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:05.742 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:05.742 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:05.742 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:46:05.742 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:46:05.742 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 1 Processor -- start 2024-09-15T01:46:05.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 1 -- start start 2024-09-15T01:46:05.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 10 monclient: init 2024-09-15T01:46:05.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:05.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 10 monclient: _add_conns ranks=[3,1,7,5,0,6,2,4,10,9,8] 2024-09-15T01:46:05.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e80a2930 0x7f15e8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 10 monclient(hunting): picked mon.noname-g con 0x7f15e80a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:46:05.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f15e809faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:46:05.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f15e809fe70 0x7f15e80a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 10 monclient(hunting): picked mon.noname-i con 0x7f15e809fe70 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:46:05.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e804bf60 con 0x7f15e809faa0 2024-09-15T01:46:05.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e809dd80 con 0x7f15e80a2930 2024-09-15T01:46:05.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80a2d00 con 0x7f15e809fe70 2024-09-15T01:46:05.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:05.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15ed91d640 10 monclient(hunting): authenticate will time out at 1570.488902s 2024-09-15T01:46:05.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.738+0000 7f15dffff640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7f15e809faa0 legacy=0x7f15e80a08c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e6ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e80a2930 0x7f15e8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f15e809fe70 0x7f15e80a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f15e809fe70 0x7f15e80a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:44076/0 (socket says 172.21.15.178:44076) 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e6ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e80a2930 0x7f15e8132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:56138/0 (socket says 172.21.15.178:56138) 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 1 -- 172.21.15.178:0/2318874806 learned_addr learned my addr 172.21.15.178:0/2318874806 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 10 monclient(hunting): get_auth_request con 0x7f15e809fe70 auth_method 0 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e6ffd640 10 monclient(hunting): get_auth_request con 0x7f15e80a2930 auth_method 0 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 10 monclient(hunting): handle_auth_done global_id 64231 payload 274 2024-09-15T01:46:05.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 1 -- 172.21.15.178:0/2318874806 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e80a2930 msgr2=0x7f15e8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:46:05.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 1 --2- 172.21.15.178:0/2318874806 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e80a2930 0x7f15e8132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 1 -- 172.21.15.178:0/2318874806 >> v1:172.21.15.204:6789/0 conn(0x7f15e809faa0 legacy=0x7f15e80a08c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 1 monclient: found mon.noname-i 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.739+0000 7f15e77fe640 1 -- 172.21.15.178:0/2318874806 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e80a0310 con 0x7f15e809fe70 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e77fe640 1 --2- 172.21.15.178:0/2318874806 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f15e809fe70 0x7f15e80a0fd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009f50 tx=0x7f15e0009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=c5bfe44d5b4ec096 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e67fc640 1 -- 172.21.15.178:0/2318874806 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f15e0007880 con 0x7f15e809fe70 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e67fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e67fc640 10 monclient: dump: 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.759 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.760 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e67fc640 10 monclient: _finish_auth 0 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15ed91d640 5 monclient: authenticate success, global_id 64231 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15ed91d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e67fc640 1 -- 172.21.15.178:0/2318874806 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e002d050 con 0x7f15e809fe70 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15ed91d640 10 monclient: get_monmap_and_config success 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e67fc640 1 -- 172.21.15.178:0/2318874806 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f15e0030360 con 0x7f15e809fe70 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15ed91d640 4 set_mon_vals no callback set 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e67fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:46:05.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.740+0000 7f15e67fc640 10 monclient: dump: 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.762 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.741+0000 7f15ed91d640 10 monclient: shutdown 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.741+0000 7f15ed91d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.741+0000 7f15ed91d640 1 -- 172.21.15.178:0/2318874806 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f15e809fe70 msgr2=0x7f15e80a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.741+0000 7f15ed91d640 1 --2- 172.21.15.178:0/2318874806 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f15e809fe70 0x7f15e80a0fd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009f50 tx=0x7f15e0009840 comp rx=0 tx=0).stop 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.741+0000 7f15ed91d640 1 -- 172.21.15.178:0/2318874806 shutdown_connections 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.741+0000 7f15ed91d640 1 --2- 172.21.15.178:0/2318874806 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f15e809fe70 0x7f15e80a0fd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.741+0000 7f15ed91d640 1 --2- 172.21.15.178:0/2318874806 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e80a2930 0x7f15e8132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.741+0000 7f15ed91d640 1 -- 172.21.15.178:0/2318874806 >> 172.21.15.178:0/2318874806 conn(0x7f15e80a5ed0 msgr2=0x7f15e80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.741+0000 7f15ed91d640 1 -- 172.21.15.178:0/2318874806 shutdown_connections 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.741+0000 7f15ed91d640 1 -- 172.21.15.178:0/2318874806 wait complete. 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.741+0000 7f15ed91d640 10 monclient: build_initial_monmap 2024-09-15T01:46:05.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.741+0000 7f15ed91d640 10 monclient: monmap: 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:05.764 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 1 Processor -- start 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 1 -- start start 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 10 monclient: init 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 10 monclient: _add_conns ranks=[8,0,5,1,2,6,3,7,4] 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f15e809faa0 0x7f15e809b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 10 monclient(hunting): picked mon.noname-e con 0x7f15e809faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15e809fe70 0x7f15e81319c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 10 monclient(hunting): picked mon.noname-a con 0x7f15e809fe70 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f15e80a2930 0x7f15e811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15e6ffd640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f15e809faa0 msgr2=0x7f15e809b280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 10 monclient(hunting): picked mon.noname-h con 0x7f15e80a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e811d9b0 con 0x7f15e809fe70 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80a2390 con 0x7f15e80a2930 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f15e809d630 con 0x7f15e809faa0 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15e6ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f15e809faa0 0x7f15e809b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15ed91d640 10 monclient(hunting): authenticate will time out at 1570.492552s 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15e809fe70 0x7f15e81319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15e809fe70 0x7f15e81319c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53994/0 (socket says 172.21.15.178:53994) 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 1 -- 172.21.15.178:0/3448679701 learned_addr learned my addr 172.21.15.178:0/3448679701 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:46:05.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 10 monclient(hunting): get_auth_request con 0x7f15e809fe70 auth_method 0 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15e77fe640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f15e80a2930 0x7f15e811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15e77fe640 10 monclient(hunting): get_auth_request con 0x7f15e80a2930 auth_method 0 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 10 monclient(hunting): handle_auth_done global_id 64359 payload 824 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 1 -- 172.21.15.178:0/3448679701 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f15e809faa0 msgr2=0x7f15e809b280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f15e809faa0 0x7f15e809b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 1 -- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f15e80a2930 msgr2=0x7f15e811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f15e80a2930 0x7f15e811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 10 monclient: _finish_hunting 0 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 1 monclient: found mon.noname-a 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.742+0000 7f15dffff640 1 -- 172.21.15.178:0/3448679701 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e0009ae0 con 0x7f15e809fe70 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15dffff640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15e809fe70 0x7f15e81319c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f15d000b5e0 tx=0x7f15d0006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f15d0006150 con 0x7f15e809fe70 2024-09-15T01:46:05.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15deffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15deffd640 10 monclient: dump: 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.769 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15deffd640 10 monclient: _finish_auth 0 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15ed91d640 5 monclient: authenticate success, global_id 64359 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d00062f0 con 0x7f15e809fe70 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15ed91d640 10 monclient: _renew_subs 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15ed91d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15ed91d640 1 -- 172.21.15.178:0/3448679701 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e811db80 con 0x7f15e809fe70 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15deffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:05.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f15d00103e0 con 0x7f15e809fe70 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15deffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15e5ffb640 4 set_mon_vals no callback set 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15deffd640 10 monclient: dump: 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.771 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15ed91d640 10 monclient: _renew_subs 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15ed91d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.743+0000 7f15ed91d640 1 -- 172.21.15.178:0/3448679701 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e809bdd0 con 0x7f15e809fe70 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f15d00064a0 con 0x7f15e809fe70 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15ed91d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15ed91d640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e8131f00 0x7f15e811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15ed91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15ed91d640 1 -- 172.21.15.178:0/3448679701 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e811ec80 con 0x7f15e8131f00 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15deffd640 1 --2- 172.21.15.178:0/3448679701 >> v2:172.21.15.204:6812/28210 conn(0x7f15c80397b0 0x7f15c803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e6ffd640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e8131f00 0x7f15e811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e6ffd640 10 monclient: get_auth_request con 0x7f15e8131f00 auth_method 0 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f15d004aff0 con 0x7f15e809fe70 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e77fe640 1 --2- 172.21.15.178:0/3448679701 >> v2:172.21.15.204:6812/28210 conn(0x7f15c80397b0 0x7f15c803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e77fe640 10 monclient: get_auth_request con 0x7f15c80397b0 auth_method 0 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e6ffd640 10 monclient(hunting): handle_auth_done global_id 64254 payload 824 2024-09-15T01:46:05.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e6ffd640 1 -- 172.21.15.178:0/3448679701 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15e804c4a0 con 0x7f15e8131f00 2024-09-15T01:46:05.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e6ffd640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e8131f00 0x7f15e811ca70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009480 tx=0x7f15d80090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.744+0000 7f15e77fe640 1 --2- 172.21.15.178:0/3448679701 >> v2:172.21.15.204:6812/28210 conn(0x7f15c80397b0 0x7f15c803bc60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f15e0033000 tx=0x7f15e0036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.745+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f15d800cd30 con 0x7f15e8131f00 2024-09-15T01:46:05.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.745+0000 7f15deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.745+0000 7f15deffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:46:05.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.745+0000 7f15deffd640 10 monclient: dump: 2024-09-15T01:46:05.774 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.774 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.774 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.745+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f15d800ce90 con 0x7f15e8131f00 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.745+0000 7f15deffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.745+0000 7f15deffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.745+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e8131f00 msgr2=0x7f15e811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.745+0000 7f15deffd640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e8131f00 0x7f15e811ca70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009480 tx=0x7f15d80090f0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.745+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e8131f00 msgr2=0x7f15e811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.745+0000 7f15deffd640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e8131f00 0x7f15e811ca70 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009480 tx=0x7f15d80090f0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.750+0000 7f15ed91d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.750+0000 7f15ed91d640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e811cfb0 0x7f15e8096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.750+0000 7f15ed91d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.750+0000 7f15ed91d640 1 -- 172.21.15.178:0/3448679701 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e809f8f0 con 0x7f15e811cfb0 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.751+0000 7f15e6ffd640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e811cfb0 0x7f15e8096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.751+0000 7f15e6ffd640 10 monclient: get_auth_request con 0x7f15e811cfb0 auth_method 0 2024-09-15T01:46:05.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.751+0000 7f15e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.751+0000 7f15e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.751+0000 7f15e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.751+0000 7f15e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.751+0000 7f15e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.751+0000 7f15e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.751+0000 7f15e6ffd640 10 monclient(hunting): handle_auth_done global_id 64263 payload 824 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.751+0000 7f15e6ffd640 1 -- 172.21.15.178:0/3448679701 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f15e804bc80 con 0x7f15e811cfb0 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.752+0000 7f15e6ffd640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e811cfb0 0x7f15e8096a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f15d8000c00 tx=0x7f15d801a000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.752+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f15d8023070 con 0x7f15e811cfb0 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.752+0000 7f15deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.752+0000 7f15deffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.752+0000 7f15deffd640 10 monclient: dump: 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.777 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.778 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.752+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9120 (secure 0 0 0) 0x7f15d8013470 con 0x7f15e811cfb0 2024-09-15T01:46:05.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.752+0000 7f15deffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:46:05.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.752+0000 7f15deffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:46:05.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.752+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e811cfb0 msgr2=0x7f15e8096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.752+0000 7f15deffd640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e811cfb0 0x7f15e8096a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f15d8000c00 tx=0x7f15d801a000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.752+0000 7f15deffd640 1 -- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e811cfb0 msgr2=0x7f15e8096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:05.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.752+0000 7f15deffd640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e811cfb0 0x7f15e8096a30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.779 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:46:05.779 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:46:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:46:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:46:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 144, 2024-09-15T01:46:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:46:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:46:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:46:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:46:05.781 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:46:05.781 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:46:05.781 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:46:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: 7 2024-09-15T01:46:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:46:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:46:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:46:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:46:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:46:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:46:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:46:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:46:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:46:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:46:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:46:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:46:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.805 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.805 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.805 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.805 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.805 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:46:05.805 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:46:05.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.753+0000 7f15ed91d640 1 -- 172.21.15.178:0/3448679701 >> v2:172.21.15.204:6812/28210 conn(0x7f15c80397b0 msgr2=0x7f15c803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.753+0000 7f15ed91d640 1 --2- 172.21.15.178:0/3448679701 >> v2:172.21.15.204:6812/28210 conn(0x7f15c80397b0 0x7f15c803bc60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f15e0033000 tx=0x7f15e0036000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.753+0000 7f15ed91d640 10 monclient: shutdown 2024-09-15T01:46:05.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.753+0000 7f15ed91d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.753+0000 7f15ed91d640 1 -- 172.21.15.178:0/3448679701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15e809fe70 msgr2=0x7f15e81319c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.753+0000 7f15ed91d640 1 --2- 172.21.15.178:0/3448679701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15e809fe70 0x7f15e81319c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f15d000b5e0 tx=0x7f15d0006f90 comp rx=0 tx=0).stop 2024-09-15T01:46:05.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.753+0000 7f15e77fe640 1 -- 172.21.15.178:0/3448679701 reap_dead start 2024-09-15T01:46:05.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.753+0000 7f15e77fe640 1 -- 172.21.15.178:0/3448679701 reap_dead start 2024-09-15T01:46:05.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.753+0000 7f15ed91d640 1 -- 172.21.15.178:0/3448679701 shutdown_connections 2024-09-15T01:46:05.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.753+0000 7f15ed91d640 1 -- 172.21.15.178:0/3448679701 >> 172.21.15.178:0/3448679701 conn(0x7f15e80a5ed0 msgr2=0x7f15e8135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.753+0000 7f15ed91d640 1 -- 172.21.15.178:0/3448679701 shutdown_connections 2024-09-15T01:46:05.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.753+0000 7f15ed91d640 1 -- 172.21.15.178:0/3448679701 wait complete. 2024-09-15T01:46:05.808 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:46:05.808 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-15T01:46:05.808 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:46:05.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.900+0000 7fd183034640 10 monclient: get_monmap_and_config 2024-09-15T01:46:05.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.900+0000 7fd183034640 10 monclient: build_initial_monmap 2024-09-15T01:46:05.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.900+0000 7fd183034640 10 monclient: monmap: 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:46:05.901115+0000 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:46:05.901115+0000 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:05.904 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:05.904 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:05.904 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:46:05.904 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:46:05.904 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.900+0000 7fd183034640 1 Processor -- start 2024-09-15T01:46:05.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 1 -- start start 2024-09-15T01:46:05.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 10 monclient: init 2024-09-15T01:46:05.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:05.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 10 monclient: _add_conns ranks=[5,2,1,9,7,3,10,0,8,4,6] 2024-09-15T01:46:05.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c0a2930 0x7fd17c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 10 monclient(hunting): picked mon.noname-h con 0x7fd17c0a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:46:05.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c09faa0 0x7fd17c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 10 monclient(hunting): picked mon.noname-b con 0x7fd17c09faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:46:05.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd17c0a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:46:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17c04bf60 con 0x7fd17c0a08c0 2024-09-15T01:46:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c09e0f0 con 0x7fd17c09faa0 2024-09-15T01:46:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c09dd80 con 0x7fd17c0a2930 2024-09-15T01:46:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd183034640 10 monclient(hunting): authenticate will time out at 1570.651518s 2024-09-15T01:46:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd1815d1640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7fd17c0a08c0 legacy=0x7fd17c0a0d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:46:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c09faa0 0x7fd17c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c09faa0 0x7fd17c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:57764/0 (socket says 172.21.15.178:57764) 2024-09-15T01:46:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 1 -- 172.21.15.178:0/3235735090 learned_addr learned my addr 172.21.15.178:0/3235735090 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:46:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 10 monclient(hunting): get_auth_request con 0x7fd17c09faa0 auth_method 0 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd180dd0640 1 --2- 172.21.15.178:0/3235735090 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c0a2930 0x7fd17c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd180dd0640 10 monclient(hunting): get_auth_request con 0x7fd17c0a2930 auth_method 0 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd180dd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd180dd0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd180dd0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd180dd0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd180dd0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd180dd0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 10 monclient(hunting): handle_auth_done global_id 35129 payload 274 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 1 -- 172.21.15.178:0/3235735090 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c0a2930 msgr2=0x7fd17c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 1 --2- 172.21.15.178:0/3235735090 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c0a2930 0x7fd17c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 1 -- 172.21.15.178:0/3235735090 >> v1:172.21.15.204:6789/0 conn(0x7fd17c0a08c0 legacy=0x7fd17c0a0d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 1 monclient: found mon.noname-b 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.901+0000 7fd17bfff640 1 -- 172.21.15.178:0/3235735090 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c0a2d00 con 0x7fd17c09faa0 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17bfff640 1 --2- 172.21.15.178:0/3235735090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c09faa0 0x7fd17c09fe70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd168009940 tx=0x7fd1680091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d67b2b3906c23d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17b7fe640 1 -- 172.21.15.178:0/3235735090 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd16802ac70 con 0x7fd17c09faa0 2024-09-15T01:46:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17b7fe640 10 monclient: dump: 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17b7fe640 1 -- 172.21.15.178:0/3235735090 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16802adf0 con 0x7fd17c09faa0 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd183034640 5 monclient: authenticate success, global_id 35129 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd183034640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd183034640 10 monclient: get_monmap_and_config success 2024-09-15T01:46:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17b7fe640 1 -- 172.21.15.178:0/3235735090 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd168031ca0 con 0x7fd17c09faa0 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd183034640 4 set_mon_vals no callback set 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.902+0000 7fd17b7fe640 10 monclient: dump: 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.903+0000 7fd183034640 10 monclient: shutdown 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.903+0000 7fd183034640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.903+0000 7fd183034640 1 -- 172.21.15.178:0/3235735090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c09faa0 msgr2=0x7fd17c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.903+0000 7fd183034640 1 --2- 172.21.15.178:0/3235735090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c09faa0 0x7fd17c09fe70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd168009940 tx=0x7fd1680091d0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.903+0000 7fd183034640 1 -- 172.21.15.178:0/3235735090 shutdown_connections 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.903+0000 7fd183034640 1 --2- 172.21.15.178:0/3235735090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c09faa0 0x7fd17c09fe70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.903+0000 7fd183034640 1 --2- 172.21.15.178:0/3235735090 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c0a2930 0x7fd17c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.903+0000 7fd183034640 1 -- 172.21.15.178:0/3235735090 >> 172.21.15.178:0/3235735090 conn(0x7fd17c0a5ed0 msgr2=0x7fd17c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.903+0000 7fd183034640 1 -- 172.21.15.178:0/3235735090 shutdown_connections 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.903+0000 7fd183034640 1 -- 172.21.15.178:0/3235735090 wait complete. 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.904+0000 7fd183034640 10 monclient: build_initial_monmap 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.904+0000 7fd183034640 10 monclient: monmap: 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:46:05.922 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.904+0000 7fd183034640 1 Processor -- start 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.904+0000 7fd183034640 1 -- start start 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.904+0000 7fd183034640 10 monclient: init 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.904+0000 7fd183034640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.904+0000 7fd183034640 10 monclient: _add_conns ranks=[5,2,4,1,6,8,3,7,0] 2024-09-15T01:46:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.904+0000 7fd183034640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c09faa0 0x7fd17c09b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.904+0000 7fd183034640 10 monclient(hunting): picked mon.noname-h con 0x7fd17c09faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.904+0000 7fd183034640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c0a08c0 0x7fd17c131a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.904+0000 7fd183034640 10 monclient(hunting): picked mon.noname-b con 0x7fd17c0a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.904+0000 7fd183034640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd17c0a2930 0x7fd17c142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 10 monclient(hunting): picked mon.noname-c con 0x7fd17c0a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c09d630 con 0x7fd17c0a08c0 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c11d580 con 0x7fd17c0a2930 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0a2400 con 0x7fd17c09faa0 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 10 monclient(hunting): authenticate will time out at 1570.655031s 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c0a08c0 0x7fd17c131a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c0a08c0 0x7fd17c131a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:57772/0 (socket says 172.21.15.178:57772) 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 1 -- 172.21.15.178:0/2565821096 learned_addr learned my addr 172.21.15.178:0/2565821096 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 10 monclient(hunting): get_auth_request con 0x7fd17c0a08c0 auth_method 0 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd1815d1640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd17c0a2930 0x7fd17c142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd1815d1640 10 monclient(hunting): get_auth_request con 0x7fd17c0a2930 auth_method 0 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd1815d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd1815d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd1815d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd1815d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd1815d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd1815d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd180dd0640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c09faa0 0x7fd17c09b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd180dd0640 10 monclient(hunting): get_auth_request con 0x7fd17c09faa0 auth_method 0 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd180dd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd180dd0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd180dd0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 10 monclient(hunting): handle_auth_done global_id 35138 payload 824 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 1 -- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c09faa0 msgr2=0x7fd17c09b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c09faa0 0x7fd17c09b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 1 -- 172.21.15.178:0/2565821096 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd17c0a2930 msgr2=0x7fd17c142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd17c0a2930 0x7fd17c142090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 1 monclient: found mon.noname-b 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 1 -- 172.21.15.178:0/2565821096 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009450 con 0x7fd17c0a08c0 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17bfff640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c0a08c0 0x7fd17c131a40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd168035040 tx=0x7fd168032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd16803b870 con 0x7fd17c0a08c0 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd178ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd178ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd178ff9640 10 monclient: dump: 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd178ff9640 10 monclient: _finish_auth 0 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd178ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16803ba10 con 0x7fd17c0a08c0 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 5 monclient: authenticate success, global_id 35138 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 10 monclient: _renew_subs 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 1 -- 172.21.15.178:0/2565821096 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c11dc00 con 0x7fd17c0a08c0 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd178ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1680313a0 con 0x7fd17c0a08c0 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd178ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd178ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd178ff9640 10 monclient: dump: 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.930 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd17affd640 4 set_mon_vals no callback set 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 10 monclient: _renew_subs 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.905+0000 7fd183034640 1 -- 172.21.15.178:0/2565821096 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c11ee40 con 0x7fd17c0a08c0 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.906+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd168039040 con 0x7fd17c0a08c0 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.906+0000 7fd178ff9640 1 --2- 172.21.15.178:0/2565821096 >> v2:172.21.15.204:6812/28210 conn(0x7fd1600396e0 0x7fd16003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.906+0000 7fd183034640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.906+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd16806b4e0 con 0x7fd17c0a08c0 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.906+0000 7fd183034640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c131f80 0x7fd17c11ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.906+0000 7fd183034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.906+0000 7fd183034640 1 -- 172.21.15.178:0/2565821096 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c11ed00 con 0x7fd17c131f80 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd180dd0640 1 --2- 172.21.15.178:0/2565821096 >> v2:172.21.15.204:6812/28210 conn(0x7fd1600396e0 0x7fd16003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd1815d1640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c131f80 0x7fd17c11ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd1815d1640 10 monclient: get_auth_request con 0x7fd17c131f80 auth_method 0 2024-09-15T01:46:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd1815d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd1815d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd1815d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd180dd0640 10 monclient: get_auth_request con 0x7fd1600396e0 auth_method 0 2024-09-15T01:46:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd1815d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd1815d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd1815d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd1815d1640 10 monclient(hunting): handle_auth_done global_id 7529 payload 824 2024-09-15T01:46:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd1815d1640 1 -- 172.21.15.178:0/2565821096 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd17c04c4a0 con 0x7fd17c131f80 2024-09-15T01:46:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd1815d1640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c131f80 0x7fd17c11ca00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd17c09c2e0 tx=0x7fd17400d040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd180dd0640 1 --2- 172.21.15.178:0/2565821096 >> v2:172.21.15.204:6812/28210 conn(0x7fd1600396e0 0x7fd16003bb90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd170009cb0 tx=0x7fd170008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd174010040 con 0x7fd17c131f80 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd178ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd178ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd178ff9640 10 monclient: dump: 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fd174007510 con 0x7fd17c131f80 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd178ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd178ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c131f80 msgr2=0x7fd17c11ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd178ff9640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c131f80 0x7fd17c11ca00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd17c09c2e0 tx=0x7fd17400d040 comp rx=0 tx=0).stop 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c131f80 msgr2=0x7fd17c11ca00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.907+0000 7fd178ff9640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c131f80 0x7fd17c11ca00 secure :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd17c09c2e0 tx=0x7fd17400d040 comp rx=0 tx=0).stop 2024-09-15T01:46:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.913+0000 7fd183034640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:46:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.913+0000 7fd183034640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c1425d0 0x7fd17c1429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.913+0000 7fd183034640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.913+0000 7fd183034640 1 -- 172.21.15.178:0/2565821096 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c132350 con 0x7fd17c1425d0 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.913+0000 7fd1815d1640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c1425d0 0x7fd17c1429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.913+0000 7fd1815d1640 10 monclient: get_auth_request con 0x7fd17c1425d0 auth_method 0 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.913+0000 7fd1815d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.913+0000 7fd1815d1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.913+0000 7fd1815d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.913+0000 7fd1815d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.913+0000 7fd1815d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd1815d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd1815d1640 10 monclient(hunting): handle_auth_done global_id 7538 payload 824 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd1815d1640 1 -- 172.21.15.178:0/2565821096 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd17c04bc80 con 0x7fd17c1425d0 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd1815d1640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c1425d0 0x7fd17c1429d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd17c09c2e0 tx=0x7fd174015000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd174026070 con 0x7fd17c1425d0 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd178ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd178ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd178ff9640 10 monclient: dump: 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:05.939 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9292 (secure 0 0 0) 0x7fd174006af0 con 0x7fd17c1425d0 2024-09-15T01:46:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd178ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:46:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd178ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:46:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c1425d0 msgr2=0x7fd17c1429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd178ff9640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c1425d0 0x7fd17c1429d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd17c09c2e0 tx=0x7fd174015000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd178ff9640 1 -- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c1425d0 msgr2=0x7fd17c1429d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.914+0000 7fd178ff9640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fd17c1425d0 0x7fd17c1429d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:05.940 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:46:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:46:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:46:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:46:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 144, 2024-09-15T01:46:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:46:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:46:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:46:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:46:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:46:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: 7 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:46:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:46:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:46:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:46:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:46:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:46:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:46:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:46:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:46:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:46:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:46:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 6 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:05.963 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:46:05.963 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:46:05.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.915+0000 7fd183034640 1 -- 172.21.15.178:0/2565821096 >> v2:172.21.15.204:6812/28210 conn(0x7fd1600396e0 msgr2=0x7fd16003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.915+0000 7fd183034640 1 --2- 172.21.15.178:0/2565821096 >> v2:172.21.15.204:6812/28210 conn(0x7fd1600396e0 0x7fd16003bb90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd170009cb0 tx=0x7fd170008040 comp rx=0 tx=0).stop 2024-09-15T01:46:05.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.915+0000 7fd183034640 10 monclient: shutdown 2024-09-15T01:46:05.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.915+0000 7fd183034640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:05.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.915+0000 7fd183034640 1 -- 172.21.15.178:0/2565821096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c0a08c0 msgr2=0x7fd17c131a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:05.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.915+0000 7fd183034640 1 --2- 172.21.15.178:0/2565821096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd17c0a08c0 0x7fd17c131a40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd168035040 tx=0x7fd168032000 comp rx=0 tx=0).stop 2024-09-15T01:46:05.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.915+0000 7fd1815d1640 1 -- 172.21.15.178:0/2565821096 reap_dead start 2024-09-15T01:46:05.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.915+0000 7fd1815d1640 1 -- 172.21.15.178:0/2565821096 reap_dead start 2024-09-15T01:46:05.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.915+0000 7fd183034640 1 -- 172.21.15.178:0/2565821096 shutdown_connections 2024-09-15T01:46:05.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.915+0000 7fd183034640 1 -- 172.21.15.178:0/2565821096 >> 172.21.15.178:0/2565821096 conn(0x7fd17c0a5ed0 msgr2=0x7fd17c0a4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:05.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.915+0000 7fd183034640 1 -- 172.21.15.178:0/2565821096 shutdown_connections 2024-09-15T01:46:05.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:05.915+0000 7fd183034640 1 -- 172.21.15.178:0/2565821096 wait complete. 2024-09-15T01:46:05.965 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:46:05.966 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-09-15T01:46:05.966 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:46:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.058+0000 7fb67411d640 10 monclient: get_monmap_and_config 2024-09-15T01:46:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.058+0000 7fb67411d640 10 monclient: build_initial_monmap 2024-09-15T01:46:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.058+0000 7fb67411d640 10 monclient: monmap: 2024-09-15T01:46:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:46:06.059035+0000 2024-09-15T01:46:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:46:06.059035+0000 2024-09-15T01:46:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:46:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:46:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:46:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:46:06.063 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:06.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.058+0000 7fb67411d640 1 Processor -- start 2024-09-15T01:46:06.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.058+0000 7fb67411d640 1 -- start start 2024-09-15T01:46:06.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.058+0000 7fb67411d640 10 monclient: init 2024-09-15T01:46:06.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:06.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 10 monclient: _add_conns ranks=[3,2,10,6,0,7,8,5,9,4,1] 2024-09-15T01:46:06.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb66c0678d0 0x7fb66c09e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:06.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 10 monclient(hunting): picked mon.noname-g con 0x7fb66c0678d0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb66c06ea30 0x7fb66c09ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 10 monclient(hunting): picked mon.noname-b con 0x7fb66c06ea30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb66c09f180 0x7fb66c1306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 10 monclient(hunting): picked mon.noname-f con 0x7fb66c09f180 addr v2:172.21.15.204:3300/0 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66c09f590 con 0x7fb66c09f180 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c06d5b0 con 0x7fb66c06ea30 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c067ca0 con 0x7fb66c0678d0 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb67411d640 10 monclient(hunting): authenticate will time out at 1570.809386s 2024-09-15T01:46:06.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb66c06ea30 0x7fb66c09ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6726ba640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7fb66c09f180 msgr2=0x7fb66c1306b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb66c06ea30 0x7fb66c09ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:57776/0 (socket says 172.21.15.178:57776) 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 1 -- 172.21.15.178:0/1901339160 learned_addr learned my addr 172.21.15.178:0/1901339160 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 10 monclient(hunting): get_auth_request con 0x7fb66c06ea30 auth_method 0 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6726ba640 1 --2- 172.21.15.178:0/1901339160 >> v2:172.21.15.204:3300/0 conn(0x7fb66c09f180 0x7fb66c1306b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb671eb9640 1 --2- 172.21.15.178:0/1901339160 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb66c0678d0 0x7fb66c09e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:06.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 10 monclient(hunting): handle_auth_done global_id 35147 payload 274 2024-09-15T01:46:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 1 -- 172.21.15.178:0/1901339160 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb66c0678d0 msgr2=0x7fb66c09e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:46:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 1 --2- 172.21.15.178:0/1901339160 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb66c0678d0 0x7fb66c09e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 1 -- 172.21.15.178:0/1901339160 >> v2:172.21.15.204:3300/0 conn(0x7fb66c09f180 msgr2=0x7fb66c1306b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:46:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 1 --2- 172.21.15.178:0/1901339160 >> v2:172.21.15.204:3300/0 conn(0x7fb66c09f180 0x7fb66c1306b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 10 monclient: _finish_hunting 0 2024-09-15T01:46:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 1 monclient: found mon.noname-b 2024-09-15T01:46:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:46:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.059+0000 7fb6716b8640 1 -- 172.21.15.178:0/1901339160 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c04bf60 con 0x7fb66c06ea30 2024-09-15T01:46:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb6716b8640 1 --2- 172.21.15.178:0/1901339160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb66c06ea30 0x7fb66c09ec40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb660009940 tx=0x7fb6600091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c1f8f2a8763e959 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:06.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb670eb7640 1 -- 172.21.15.178:0/1901339160 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb66002ad20 con 0x7fb66c06ea30 2024-09-15T01:46:06.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb670eb7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:06.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb670eb7640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:46:06.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb670eb7640 10 monclient: dump: 2024-09-15T01:46:06.080 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:06.080 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:06.080 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:06.080 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:06.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb670eb7640 10 monclient: _finish_auth 0 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb670eb7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb670eb7640 1 -- 172.21.15.178:0/1901339160 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb660031820 con 0x7fb66c06ea30 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb67411d640 5 monclient: authenticate success, global_id 35147 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb67411d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb670eb7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb670eb7640 1 -- 172.21.15.178:0/1901339160 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb66002aea0 con 0x7fb66c06ea30 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb67411d640 10 monclient: get_monmap_and_config success 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb67411d640 4 set_mon_vals no callback set 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb670eb7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb670eb7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.060+0000 7fb670eb7640 10 monclient: dump: 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:06.082 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:06.083 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 10 monclient: shutdown 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 1 -- 172.21.15.178:0/1901339160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb66c06ea30 msgr2=0x7fb66c09ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 1 --2- 172.21.15.178:0/1901339160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb66c06ea30 0x7fb66c09ec40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb660009940 tx=0x7fb6600091d0 comp rx=0 tx=0).stop 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 1 -- 172.21.15.178:0/1901339160 shutdown_connections 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 1 --2- 172.21.15.178:0/1901339160 >> v2:172.21.15.204:3300/0 conn(0x7fb66c09f180 0x7fb66c1306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 1 --2- 172.21.15.178:0/1901339160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb66c06ea30 0x7fb66c09ec40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 1 --2- 172.21.15.178:0/1901339160 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb66c0678d0 0x7fb66c09e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 1 -- 172.21.15.178:0/1901339160 >> 172.21.15.178:0/1901339160 conn(0x7fb66c0a4390 msgr2=0x7fb66c067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 1 -- 172.21.15.178:0/1901339160 shutdown_connections 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 1 -- 172.21.15.178:0/1901339160 wait complete. 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 10 monclient: build_initial_monmap 2024-09-15T01:46:06.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.061+0000 7fb67411d640 10 monclient: monmap: 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:46:06.085 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:46:06.086 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:46:06.086 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:06.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 1 Processor -- start 2024-09-15T01:46:06.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 1 -- start start 2024-09-15T01:46:06.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 10 monclient: init 2024-09-15T01:46:06.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 10 monclient: _reopen_session rank -1 2024-09-15T01:46:06.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 10 monclient: _add_conns ranks=[8,7,5,4,2,1,0,6,3] 2024-09-15T01:46:06.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb66c0678d0 0x7fb66c13f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:06.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 10 monclient(hunting): picked mon.noname-e con 0x7fb66c0678d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:46:06.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c06ea30 0x7fb66c133800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:06.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 10 monclient(hunting): picked mon.noname-i con 0x7fb66c06ea30 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:46:06.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb66c09f180 0x7fb66c11d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:06.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 10 monclient(hunting): picked mon.noname-h con 0x7fb66c09f180 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:46:06.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:06.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c120920 con 0x7fb66c09f180 2024-09-15T01:46:06.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:06.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c11f370 con 0x7fb66c06ea30 2024-09-15T01:46:06.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:06.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb671eb9640 1 -- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb66c0678d0 msgr2=0x7fb66c13f940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3304/0 2024-09-15T01:46:06.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c06ccd0 con 0x7fb66c0678d0 2024-09-15T01:46:06.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 10 monclient(hunting): _renew_subs 2024-09-15T01:46:06.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb67411d640 10 monclient(hunting): authenticate will time out at 1570.812701s 2024-09-15T01:46:06.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb671eb9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb66c0678d0 0x7fb66c13f940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:46:06.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb6716b8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c06ea30 0x7fb66c133800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:06.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb6726ba640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb66c09f180 0x7fb66c11d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:06.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb6716b8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c06ea30 0x7fb66c133800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:44080/0 (socket says 172.21.15.178:44080) 2024-09-15T01:46:06.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb6716b8640 1 -- 172.21.15.178:0/2942276352 learned_addr learned my addr 172.21.15.178:0/2942276352 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:46:06.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb6716b8640 10 monclient(hunting): get_auth_request con 0x7fb66c06ea30 auth_method 0 2024-09-15T01:46:06.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb6716b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:06.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb6716b8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:06.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb6716b8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:06.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb6726ba640 10 monclient(hunting): get_auth_request con 0x7fb66c09f180 auth_method 0 2024-09-15T01:46:06.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb6726ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:06.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb6726ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:06.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.062+0000 7fb6726ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:06.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6716b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:06.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6716b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:06.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6716b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:06.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:06.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:06.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:06.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 10 monclient(hunting): handle_auth_done global_id 7547 payload 824 2024-09-15T01:46:06.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 1 -- 172.21.15.178:0/2942276352 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb66c0678d0 msgr2=0x7fb66c13f940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:46:06.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb66c0678d0 0x7fb66c13f940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:06.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 1 -- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c06ea30 msgr2=0x7fb66c133800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:06.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c06ea30 0x7fb66c133800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:46:06.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 10 monclient: _finish_hunting 0 2024-09-15T01:46:06.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 1 monclient: found mon.noname-h 2024-09-15T01:46:06.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:46:06.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:46:06.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 1 -- 172.21.15.178:0/2942276352 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb660009450 con 0x7fb66c09f180 2024-09-15T01:46:06.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6716b8640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c06ea30 0x7fb66c133800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:46:06.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb6726ba640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb66c09f180 0x7fb66c11d390 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00bdf0 tx=0x7fb65c0096a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb65c0067e0 con 0x7fb66c09f180 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb669ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb669ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb669ffb640 10 monclient: dump: 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:06.091 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb669ffb640 10 monclient: _finish_auth 0 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb669ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65c00f070 con 0x7fb66c09f180 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb67411d640 5 monclient: authenticate success, global_id 7547 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb67411d640 10 monclient: _renew_subs 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb67411d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:46:06.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb67411d640 1 -- 172.21.15.178:0/2942276352 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66c06c900 con 0x7fb66c09f180 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb669ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb65c0069e0 con 0x7fb66c09f180 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb669ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb669ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb669ffb640 10 monclient: dump: 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:06.093 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb67411d640 10 monclient: _renew_subs 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb66bfff640 4 set_mon_vals no callback set 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb67411d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.063+0000 7fb67411d640 1 -- 172.21.15.178:0/2942276352 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66c140730 con 0x7fb66c09f180 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb67411d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb67411d640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c133d40 0x7fb66c11d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:06.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb67411d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb65c006360 con 0x7fb66c09f180 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb67411d640 1 -- 172.21.15.178:0/2942276352 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c134110 con 0x7fb66c133d40 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb671eb9640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c133d40 0x7fb66c11d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb669ffb640 1 --2- 172.21.15.178:0/2942276352 >> v2:172.21.15.204:6812/28210 conn(0x7fb6580397b0 0x7fb65803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb671eb9640 10 monclient: get_auth_request con 0x7fb66c133d40 auth_method 0 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb671eb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb671eb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb671eb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fb65c04c830 con 0x7fb66c09f180 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb671eb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb671eb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.064+0000 7fb671eb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:06.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb6716b8640 1 --2- 172.21.15.178:0/2942276352 >> v2:172.21.15.204:6812/28210 conn(0x7fb6580397b0 0x7fb65803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb6716b8640 10 monclient: get_auth_request con 0x7fb6580397b0 auth_method 0 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb671eb9640 10 monclient(hunting): handle_auth_done global_id 64249 payload 824 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb671eb9640 1 -- 172.21.15.178:0/2942276352 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb66c04c4a0 con 0x7fb66c133d40 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb671eb9640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c133d40 0x7fb66c11d8d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb66400c700 tx=0x7fb664008040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb664011a80 con 0x7fb66c133d40 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb669ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb669ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb669ffb640 10 monclient: dump: 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:06.096 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:06.097 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:06.097 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:06.097 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:06.097 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:06.097 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:06.097 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:06.097 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:06.097 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:06.097 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:06.098 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:06.098 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:06.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb6716b8640 1 --2- 172.21.15.178:0/2942276352 >> v2:172.21.15.204:6812/28210 conn(0x7fb6580397b0 0x7fb65803bc60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb660009940 tx=0x7fb660032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:06.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fb664011be0 con 0x7fb66c133d40 2024-09-15T01:46:06.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb669ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:46:06.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb669ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:46:06.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c133d40 msgr2=0x7fb66c11d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:06.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb669ffb640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c133d40 0x7fb66c11d8d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb66400c700 tx=0x7fb664008040 comp rx=0 tx=0).stop 2024-09-15T01:46:06.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c133d40 msgr2=0x7fb66c11d8d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:06.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.065+0000 7fb669ffb640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c133d40 0x7fb66c11d8d0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb66400c700 tx=0x7fb664008040 comp rx=0 tx=0).stop 2024-09-15T01:46:06.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb67411d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:46:06.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb67411d640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c11de10 0x7fb66c098a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb67411d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb67411d640 1 -- 172.21.15.178:0/2942276352 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c04bc80 con 0x7fb66c11de10 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb671eb9640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c11de10 0x7fb66c098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb671eb9640 10 monclient: get_auth_request con 0x7fb66c11de10 auth_method 0 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb671eb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb671eb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb671eb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb671eb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb671eb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb671eb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb671eb9640 10 monclient(hunting): handle_auth_done global_id 64258 payload 824 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb671eb9640 1 -- 172.21.15.178:0/2942276352 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb66c05ed70 con 0x7fb66c11de10 2024-09-15T01:46:06.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.071+0000 7fb671eb9640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c11de10 0x7fb66c098a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb66400a4a0 tx=0x7fb664019000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:46:06.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb664020070 con 0x7fb66c11de10 2024-09-15T01:46:06.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb669ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:46:06.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb669ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:46:06.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb669ffb640 10 monclient: dump: 2024-09-15T01:46:06.100 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:46:06.100 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:46:06.100 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:06.100 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:46:06.100 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:46:06.100 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:46:06.100 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9120 (secure 0 0 0) 0x7fb6640193f0 con 0x7fb66c11de10 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb669ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb669ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:46:06.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c11de10 msgr2=0x7fb66c098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:06.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb669ffb640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c11de10 0x7fb66c098a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb66400a4a0 tx=0x7fb664019000 comp rx=0 tx=0).stop 2024-09-15T01:46:06.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb669ffb640 1 -- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c11de10 msgr2=0x7fb66c098a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:46:06.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb669ffb640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb66c11de10 0x7fb66c098a40 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb66400a4a0 tx=0x7fb664019000 comp rx=0 tx=0).stop 2024-09-15T01:46:06.102 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:46:06.102 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:46:06.102 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 144, 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: 7 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:46:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:46:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:46:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:46:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:46:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:46:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:46:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:46:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:46:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:46:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:46:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:46:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:46:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:46:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:46:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:46:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:46:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:46:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:46:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:46:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:46:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:46:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:46:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:46:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:46:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:46:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:46:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:46:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.113 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:46:06.113 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:46:06.113 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:06.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:06.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.114 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:06.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:46:06.114 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.115 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:06.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:46:06.115 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.115 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:06.115 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:06.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:46:06.116 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:46:06.116 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:06.116 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:06.116 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:06.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.117 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:46:06.117 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:46:06.117 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:06.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:06.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.118 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:06.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:46:06.118 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.119 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:06.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:46:06.119 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.119 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:06.120 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:06.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:46:06.120 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:46:06.120 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:06.120 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:06.121 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:06.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.121 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:46:06.121 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:46:06.121 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:06.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:06.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.122 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:06.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:46:06.122 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:46:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:46:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:46:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:06.125 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:06.125 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:06.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.125 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:46:06.126 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:46:06.126 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:06.126 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:06.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.126 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:06.126 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:46:06.127 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.127 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:06.127 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:46:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:46:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:46:06.129 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:06.129 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:06.129 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:06.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.129 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.130 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:46:06.130 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:46:06.130 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:06.130 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:06.130 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.130 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:06.131 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:46:06.131 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.131 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:06.132 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:46:06.132 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.132 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:06.132 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:06.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.133 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:46:06.133 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:46:06.133 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:06.133 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:06.133 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:06.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.134 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:46:06.134 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:46:06.134 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:06.134 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:06.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.135 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:06.135 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:46:06.135 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.136 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:06.136 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:46:06.136 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.136 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:06.136 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:06.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.137 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:46:06.137 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:46:06.137 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:06.137 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:06.137 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:06.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.138 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:46:06.138 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:46:06.138 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:46:06.138 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:46:06.139 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.139 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:46:06.139 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:46:06.139 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.140 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:46:06.140 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:46:06.140 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:46:06.140 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:06.140 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:06.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.141 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:46:06.141 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:46:06.141 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:46:06.141 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:46:06.142 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:46:06.142 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:06.142 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:06.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.142 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:46:06.142 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:46:06.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.143 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:06.143 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:06.143 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:06.143 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:06.144 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:46:06.144 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:46:06.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:46:06.144 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:46:06.144 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:46:06.144 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:46:06.145 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:46:06.145 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:46:06.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:46:06.145 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:46:06.145 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:46:06.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb67411d640 1 -- 172.21.15.178:0/2942276352 >> v2:172.21.15.204:6812/28210 conn(0x7fb6580397b0 msgr2=0x7fb65803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:06.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb67411d640 1 --2- 172.21.15.178:0/2942276352 >> v2:172.21.15.204:6812/28210 conn(0x7fb6580397b0 0x7fb65803bc60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb660009940 tx=0x7fb660032000 comp rx=0 tx=0).stop 2024-09-15T01:46:06.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb67411d640 10 monclient: shutdown 2024-09-15T01:46:06.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb67411d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:46:06.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb67411d640 1 -- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb66c09f180 msgr2=0x7fb66c11d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:46:06.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb67411d640 1 --2- 172.21.15.178:0/2942276352 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb66c09f180 0x7fb66c11d390 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00bdf0 tx=0x7fb65c0096a0 comp rx=0 tx=0).stop 2024-09-15T01:46:06.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb6726ba640 1 -- 172.21.15.178:0/2942276352 reap_dead start 2024-09-15T01:46:06.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb6726ba640 1 -- 172.21.15.178:0/2942276352 reap_dead start 2024-09-15T01:46:06.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb67411d640 1 -- 172.21.15.178:0/2942276352 shutdown_connections 2024-09-15T01:46:06.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb67411d640 1 -- 172.21.15.178:0/2942276352 >> 172.21.15.178:0/2942276352 conn(0x7fb66c0a4390 msgr2=0x7fb66c09b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:46:06.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb67411d640 1 -- 172.21.15.178:0/2942276352 shutdown_connections 2024-09-15T01:46:06.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:46:06.072+0000 7fb67411d640 1 -- 172.21.15.178:0/2942276352 wait complete. 2024-09-15T01:46:06.150 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:46:06.151 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T01:47:36.152 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-15T01:47:36.152 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-09-15T01:47:36.152 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-15T01:47:36.153 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-15T01:47:36.157 INFO:tasks.ceph.mon.e:Started 2024-09-15T01:47:36.157 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T01:47:36.157 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-15T01:47:36.157 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-15T01:47:36.158 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-15T01:47:36.160 INFO:tasks.ceph.mon.f:Started 2024-09-15T01:47:36.160 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-15T01:47:36.161 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-15T01:47:36.161 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T01:47:36.268 INFO:tasks.ceph.mon.f.smithi204.stderr:2024-09-15T01:47:36.265+0000 7f183164f900 -1 mon.f@-1(???).log v0 log_external_backlog rewinding external_log_to from 486 -> 0 (sync_force? mon rebuild?) 2024-09-15T01:47:51.162 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-15T01:47:51.163 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-15T01:47:51.163 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:47:51.163 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:47:51.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.267+0000 7fad5bd9d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:51.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.267+0000 7fad5bd9d640 10 monclient: build_initial_monmap 2024-09-15T01:47:51.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.267+0000 7fad5bd9d640 10 monclient: monmap: 2024-09-15T01:47:51.271 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:51.271 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.271 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:51.269738+0000 2024-09-15T01:47:51.271 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:51.269738+0000 2024-09-15T01:47:51.271 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:51.271 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:51.272 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.268+0000 7fad5bd9d640 1 Processor -- start 2024-09-15T01:47:51.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.268+0000 7fad5bd9d640 1 -- start start 2024-09-15T01:47:51.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.268+0000 7fad5bd9d640 10 monclient: init 2024-09-15T01:47:51.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:51.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 10 monclient: _add_conns ranks=[5,8,4,3,6,2,10,1,7,0,9] 2024-09-15T01:47:51.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fad540a2930 0x7fad54132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 10 monclient(hunting): picked mon.noname-h con 0x7fad540a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:51.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad5409faa0 0x7fad5409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 10 monclient(hunting): picked mon.noname-e con 0x7fad5409faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:51.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fad540a08c0 0x7fad540a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 10 monclient(hunting): picked mon.noname-c con 0x7fad540a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:47:51.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad5409e0f0 con 0x7fad540a08c0 2024-09-15T01:47:51.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad5409dd80 con 0x7fad540a2930 2024-09-15T01:47:51.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fad540a2d00 con 0x7fad5409faa0 2024-09-15T01:47:51.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:51.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5bd9d640 10 monclient(hunting): authenticate will time out at 1676.020152s 2024-09-15T01:47:51.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59338640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad5409faa0 0x7fad5409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fad540a08c0 0x7fad540a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59338640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad5409faa0 0x7fad5409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42022/0 (socket says 172.21.15.178:42022) 2024-09-15T01:47:51.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fad540a08c0 0x7fad540a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:46584/0 (socket says 172.21.15.178:46584) 2024-09-15T01:47:51.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59338640 1 -- 172.21.15.178:0/3089997695 learned_addr learned my addr 172.21.15.178:0/3089997695 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:51.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59338640 10 monclient(hunting): get_auth_request con 0x7fad5409faa0 auth_method 0 2024-09-15T01:47:51.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59338640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59338640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 10 monclient(hunting): get_auth_request con 0x7fad540a08c0 auth_method 0 2024-09-15T01:47:51.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59338640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59338640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59b39640 1 --2- 172.21.15.178:0/3089997695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fad540a2930 0x7fad54132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59b39640 10 monclient(hunting): get_auth_request con 0x7fad540a2930 auth_method 0 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59b39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 10 monclient(hunting): handle_auth_done global_id 44761 payload 274 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 1 -- 172.21.15.178:0/3089997695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad5409faa0 msgr2=0x7fad5409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 1 --2- 172.21.15.178:0/3089997695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad5409faa0 0x7fad5409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 1 -- 172.21.15.178:0/3089997695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fad540a2930 msgr2=0x7fad54132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 1 --2- 172.21.15.178:0/3089997695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fad540a2930 0x7fad54132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 10 monclient: _finish_hunting 0 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 1 monclient: found mon.noname-c 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad5a33a640 1 -- 172.21.15.178:0/3089997695 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5404bf60 con 0x7fad540a08c0 2024-09-15T01:47:51.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59338640 1 --2- 172.21.15.178:0/3089997695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad5409faa0 0x7fad5409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.269+0000 7fad59b39640 1 --2- 172.21.15.178:0/3089997695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fad540a2930 0x7fad54132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad5a33a640 1 --2- 172.21.15.178:0/3089997695 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fad540a08c0 0x7fad540a0d30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fad44009e90 tx=0x7fad44009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=5e8941b7904f8ec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad58b37640 1 -- 172.21.15.178:0/3089997695 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad440079d0 con 0x7fad540a08c0 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad58b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad58b37640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad58b37640 10 monclient: dump: 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.288 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.289 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.289 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.289 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.289 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.289 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.289 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.289 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.289 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad58b37640 10 monclient: _finish_auth 0 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad58b37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad5bd9d640 5 monclient: authenticate success, global_id 44761 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad5bd9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad58b37640 1 -- 172.21.15.178:0/3089997695 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4402d050 con 0x7fad540a08c0 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad58b37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad58b37640 1 -- 172.21.15.178:0/3089997695 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad44007bd0 con 0x7fad540a08c0 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad5bd9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad5bd9d640 4 set_mon_vals no callback set 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad58b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad58b37640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:47:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.270+0000 7fad58b37640 10 monclient: dump: 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 10 monclient: shutdown 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 1 -- 172.21.15.178:0/3089997695 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fad540a08c0 msgr2=0x7fad540a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 1 --2- 172.21.15.178:0/3089997695 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fad540a08c0 0x7fad540a0d30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fad44009e90 tx=0x7fad44009780 comp rx=0 tx=0).stop 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 1 -- 172.21.15.178:0/3089997695 shutdown_connections 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 1 --2- 172.21.15.178:0/3089997695 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fad540a08c0 0x7fad540a0d30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 1 --2- 172.21.15.178:0/3089997695 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad5409faa0 0x7fad5409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 1 --2- 172.21.15.178:0/3089997695 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fad540a2930 0x7fad54132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 1 -- 172.21.15.178:0/3089997695 >> 172.21.15.178:0/3089997695 conn(0x7fad540a5ed0 msgr2=0x7fad540a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 1 -- 172.21.15.178:0/3089997695 shutdown_connections 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 1 -- 172.21.15.178:0/3089997695 wait complete. 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 10 monclient: build_initial_monmap 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 10 monclient: monmap: 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.271+0000 7fad5bd9d640 1 Processor -- start 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 1 -- start start 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 10 monclient: init 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 10 monclient: _add_conns ranks=[2,8,7,5,1,3,0,4,6] 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad5409faa0 0x7fad5409b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 10 monclient(hunting): picked mon.noname-b con 0x7fad5409faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad540a08c0 0x7fad54131750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 10 monclient(hunting): picked mon.noname-e con 0x7fad540a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fad540a2930 0x7fad5411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 10 monclient(hunting): picked mon.noname-i con 0x7fad540a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5411e870 con 0x7fad5409faa0 2024-09-15T01:47:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fad540a2390 con 0x7fad540a2930 2024-09-15T01:47:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fad54142aa0 con 0x7fad540a08c0 2024-09-15T01:47:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5bd9d640 10 monclient(hunting): authenticate will time out at 1676.023444s 2024-09-15T01:47:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad540a08c0 0x7fad54131750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59b39640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad5409faa0 0x7fad5409b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad540a08c0 0x7fad54131750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42030/0 (socket says 172.21.15.178:42030) 2024-09-15T01:47:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 1 -- 172.21.15.178:0/4010516850 learned_addr learned my addr 172.21.15.178:0/4010516850 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59b39640 10 monclient(hunting): get_auth_request con 0x7fad5409faa0 auth_method 0 2024-09-15T01:47:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59b39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 10 monclient(hunting): get_auth_request con 0x7fad540a08c0 auth_method 0 2024-09-15T01:47:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5a33a640 1 --2- 172.21.15.178:0/4010516850 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fad540a2930 0x7fad5411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59b39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59b39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59b39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5a33a640 10 monclient(hunting): get_auth_request con 0x7fad540a2930 auth_method 0 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5a33a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5a33a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5a33a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5a33a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5a33a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad5a33a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 10 monclient(hunting): handle_auth_done global_id 94121 payload 824 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 1 -- 172.21.15.178:0/4010516850 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fad540a2930 msgr2=0x7fad5411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 1 --2- 172.21.15.178:0/4010516850 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fad540a2930 0x7fad5411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 1 -- 172.21.15.178:0/4010516850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad5409faa0 msgr2=0x7fad5409b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 1 --2- 172.21.15.178:0/4010516850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad5409faa0 0x7fad5409b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 10 monclient: _finish_hunting 0 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 1 monclient: found mon.noname-e 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59338640 1 -- 172.21.15.178:0/4010516850 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad48009450 con 0x7fad540a08c0 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.272+0000 7fad59b39640 1 --2- 172.21.15.178:0/4010516850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad5409faa0 0x7fad5409b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad59338640 1 --2- 172.21.15.178:0/4010516850 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad540a08c0 0x7fad54131750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fad4800ea50 tx=0x7fad480091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad51ffb640 1 -- 172.21.15.178:0/4010516850 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad4800ce90 con 0x7fad540a08c0 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad51ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad51ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad51ffb640 10 monclient: dump: 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad51ffb640 10 monclient: _finish_auth 0 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad51ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad51ffb640 1 -- 172.21.15.178:0/4010516850 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad48013470 con 0x7fad540a08c0 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad51ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad51ffb640 1 -- 172.21.15.178:0/4010516850 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad48013a60 con 0x7fad540a08c0 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad5bd9d640 5 monclient: authenticate success, global_id 94121 2024-09-15T01:47:51.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad51ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad51ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad51ffb640 10 monclient: dump: 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.301 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad53fff640 4 set_mon_vals no callback set 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad5bd9d640 10 monclient: _renew_subs 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad5bd9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad5bd9d640 1 -- 172.21.15.178:0/4010516850 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad44009a20 con 0x7fad540a08c0 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad5bd9d640 10 monclient: _renew_subs 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad5bd9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.273+0000 7fad5bd9d640 1 -- 172.21.15.178:0/4010516850 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5409d630 con 0x7fad540a08c0 2024-09-15T01:47:51.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.274+0000 7fad51ffb640 1 -- 172.21.15.178:0/4010516850 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fad48013bc0 con 0x7fad540a08c0 2024-09-15T01:47:51.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.274+0000 7fad51ffb640 1 --2- 172.21.15.178:0/4010516850 >> v2:172.21.15.204:6812/28210 conn(0x7fad400397b0 0x7fad4003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.274+0000 7fad5bd9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:51.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.274+0000 7fad5bd9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:51.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.274+0000 7fad5bd9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:51.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.274+0000 7fad51ffb640 1 -- 172.21.15.178:0/4010516850 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fad4804beb0 con 0x7fad540a08c0 2024-09-15T01:47:51.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.274+0000 7fad5bd9d640 1 -- 172.21.15.178:0/4010516850 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5404b720 con 0x7fad540a08c0 2024-09-15T01:47:51.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.274+0000 7fad59b39640 1 --2- 172.21.15.178:0/4010516850 >> v2:172.21.15.204:6812/28210 conn(0x7fad400397b0 0x7fad4003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.274+0000 7fad59b39640 10 monclient: get_auth_request con 0x7fad400397b0 auth_method 0 2024-09-15T01:47:51.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.275+0000 7fad59b39640 1 --2- 172.21.15.178:0/4010516850 >> v2:172.21.15.204:6812/28210 conn(0x7fad400397b0 0x7fad4003bc60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fad4c006e70 tx=0x7fad4c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.279+0000 7fad51ffb640 1 -- 172.21.15.178:0/4010516850 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fad480123d0 con 0x7fad540a08c0 2024-09-15T01:47:51.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.279+0000 7fad51ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:51.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.279+0000 7fad51ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:51.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.496+0000 7fad5bd9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:47:51.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.496+0000 7fad5bd9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:47:51.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.496+0000 7fad5bd9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:51.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.496+0000 7fad5bd9d640 1 -- 172.21.15.178:0/4010516850 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fad5409cef0 con 0x7fad540a08c0 2024-09-15T01:47:51.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.496+0000 7fad51ffb640 1 -- 172.21.15.178:0/4010516850 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7fad4802ac80 con 0x7fad540a08c0 2024-09-15T01:47:51.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.496+0000 7fad51ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:47:51.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.496+0000 7fad51ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:51.500 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":150,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:47:51.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 1 -- 172.21.15.178:0/4010516850 >> v2:172.21.15.204:6812/28210 conn(0x7fad400397b0 msgr2=0x7fad4003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 1 --2- 172.21.15.178:0/4010516850 >> v2:172.21.15.204:6812/28210 conn(0x7fad400397b0 0x7fad4003bc60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fad4c006e70 tx=0x7fad4c008040 comp rx=0 tx=0).stop 2024-09-15T01:47:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 10 monclient: shutdown 2024-09-15T01:47:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 1 -- 172.21.15.178:0/4010516850 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad540a08c0 msgr2=0x7fad54131750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 1 --2- 172.21.15.178:0/4010516850 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad540a08c0 0x7fad54131750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fad4800ea50 tx=0x7fad480091d0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 1 -- 172.21.15.178:0/4010516850 shutdown_connections 2024-09-15T01:47:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 1 --2- 172.21.15.178:0/4010516850 >> v2:172.21.15.204:6812/28210 conn(0x7fad400397b0 0x7fad4003bc60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 1 --2- 172.21.15.178:0/4010516850 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fad540a2930 0x7fad5411c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 1 --2- 172.21.15.178:0/4010516850 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fad540a08c0 0x7fad54131750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 1 --2- 172.21.15.178:0/4010516850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad5409faa0 0x7fad5409b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 1 -- 172.21.15.178:0/4010516850 >> 172.21.15.178:0/4010516850 conn(0x7fad540a5ed0 msgr2=0x7fad54135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 1 -- 172.21.15.178:0/4010516850 shutdown_connections 2024-09-15T01:47:51.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.498+0000 7fad5bd9d640 1 -- 172.21.15.178:0/4010516850 wait complete. 2024-09-15T01:47:51.511 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:47:51.511 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:47:51.512 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:47:51.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.602+0000 7f6acfb9d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:51.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.602+0000 7f6acfb9d640 10 monclient: build_initial_monmap 2024-09-15T01:47:51.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.602+0000 7f6acfb9d640 10 monclient: monmap: 2024-09-15T01:47:51.623 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:51.604653+0000 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:51.604653+0000 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:51.624 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.603+0000 7f6acfb9d640 1 Processor -- start 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.603+0000 7f6acfb9d640 1 -- start start 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.603+0000 7f6acfb9d640 10 monclient: init 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.603+0000 7f6acfb9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.603+0000 7f6acfb9d640 10 monclient: _add_conns ranks=[1,0,5,2,8,7,6,3,4,9,10] 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6ac80a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6ac8132de0 addr v1:172.21.15.178:6789/0 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ac81392c0 0x7f6ac813b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 10 monclient(hunting): picked mon.noname-h con 0x7f6ac81392c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:51.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac804bf60 con 0x7f6ac8132de0 2024-09-15T01:47:51.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac809dd80 con 0x7f6ac80a2930 2024-09-15T01:47:51.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac80a2d00 con 0x7f6ac81392c0 2024-09-15T01:47:51.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:51.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 10 monclient(hunting): authenticate will time out at 1676.355080s 2024-09-15T01:47:51.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd138640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f6ac8132de0 0x7f6ac8137ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:36804/0 (socket says 172.21.15.178:36804) 2024-09-15T01:47:51.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd138640 1 -- 172.21.15.178:0/1587501461 learned_addr learned my addr 172.21.15.178:0/1587501461 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:47:51.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197931202 0 0) 0x7f6ac804bf60 con 0x7f6ac8132de0 2024-09-15T01:47:51.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient(hunting): my global_id is 64368 2024-09-15T01:47:51.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ab40034e0 con 0x7f6ac8132de0 2024-09-15T01:47:51.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ac81392c0 0x7f6ac813b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 10 monclient(hunting): get_auth_request con 0x7f6ac81392c0 auth_method 0 2024-09-15T01:47:51.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1488414455 0 0) 0x7f6ab40034e0 con 0x7f6ac8132de0 2024-09-15T01:47:51.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ac81392c0 msgr2=0x7f6ac813b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:51.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f6ac81392c0 0x7f6ac813b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 >> v1:172.21.15.204:6789/0 conn(0x7f6ac80a2930 legacy=0x7f6ac809faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient: _finish_hunting 0 2024-09-15T01:47:51.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:47:51.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:51.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:47:51.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac81331b0 con 0x7f6ac8132de0 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient: _finish_auth 0 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 5 monclient: authenticate success, global_id 64368 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f6ab80018f0 con 0x7f6ac8132de0 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient: dump: 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:51.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 >> v1:172.21.15.178:6789/0 conn(0x7f6ac8132de0 legacy=0x7f6ac8137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient: _add_conns ranks=[7,0,8,2,1,6,4,3,5] 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ab4004250 0x7f6ab4004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient(hunting): picked mon.i con 0x7f6ab4004250 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab4004d30 0x7f6ab40051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient(hunting): picked mon.a con 0x7f6ab4004d30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ab4009c80 0x7f6ab400c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient(hunting): picked mon.e con 0x7f6ab4009c80 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac80a2d00 con 0x7f6ab4004d30 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab400c9e0 con 0x7f6ab4004250 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab400cb50 con 0x7f6ab4009c80 2024-09-15T01:47:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:51.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acc937640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:47:51.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acfb9d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:51.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd138640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab4004d30 0x7f6ab40051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd138640 10 monclient(hunting): get_auth_request con 0x7f6ab4004d30 auth_method 0 2024-09-15T01:47:51.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd138640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:47:51.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ab4009c80 0x7f6ab400c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd939640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ab4004250 0x7f6ab4004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 10 monclient(hunting): get_auth_request con 0x7f6ab4009c80 auth_method 0 2024-09-15T01:47:51.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6ace13a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:47:51.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd939640 10 monclient(hunting): get_auth_request con 0x7f6ab4004250 auth_method 0 2024-09-15T01:47:51.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd939640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd138640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.604+0000 7f6acd138640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6ace13a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6ace13a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6ace13a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd939640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 10 monclient(hunting): handle_auth_done global_id 64368 payload 293 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 1 -- 172.21.15.178:0/1587501461 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ab4009c80 msgr2=0x7f6ab400c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f6ab4009c80 0x7f6ab400c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 1 -- 172.21.15.178:0/1587501461 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ab4004250 msgr2=0x7f6ab4004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f6ab4004250 0x7f6ab4004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 10 monclient: _finish_hunting 0 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 1 monclient: found mon.a 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 1 -- 172.21.15.178:0/1587501461 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab40034e0 con 0x7f6ab4004d30 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 10 monclient: _finish_auth 0 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6ace13a640 1 -- 172.21.15.178:0/1587501461 reap_dead start 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acd138640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab4004d30 0x7f6ab40051a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8007f50 tx=0x7f6ab8001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78b9495fb65856a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ab8005d10 con 0x7f6ab4004d30 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acc937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.605+0000 7f6acc937640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:51.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acc937640 10 monclient: dump: 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.636 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acfb9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab802d070 con 0x7f6ab4004d30 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acc937640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acfb9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acfb9d640 4 set_mon_vals no callback set 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acc937640 1 -- 172.21.15.178:0/1587501461 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ab8007570 con 0x7f6ab4004d30 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acc937640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acc937640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acc937640 10 monclient: dump: 2024-09-15T01:47:51.637 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acfb9d640 10 monclient: shutdown 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acfb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acfb9d640 1 -- 172.21.15.178:0/1587501461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab4004d30 msgr2=0x7f6ab40051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acfb9d640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab4004d30 0x7f6ab40051a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8007f50 tx=0x7f6ab8001630 comp rx=0 tx=0).stop 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acfb9d640 1 -- 172.21.15.178:0/1587501461 shutdown_connections 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acfb9d640 1 --2- 172.21.15.178:0/1587501461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab4004d30 0x7f6ab40051a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acfb9d640 1 -- 172.21.15.178:0/1587501461 >> 172.21.15.178:0/1587501461 conn(0x7f6ac80a5ed0 msgr2=0x7f6ac80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acfb9d640 1 -- 172.21.15.178:0/1587501461 shutdown_connections 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.606+0000 7f6acfb9d640 1 -- 172.21.15.178:0/1587501461 wait complete. 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient: build_initial_monmap 2024-09-15T01:47:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient: monmap: 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 1 Processor -- start 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 1 -- start start 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient: init 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient: _add_conns ranks=[6,1,3,5,2,8,7,4,0] 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ab4004d30 0x7f6ac8141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient(hunting): picked mon.noname-d con 0x7f6ab4004d30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac8141fd0 0x7f6ac809c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient(hunting): picked mon.noname-f con 0x7f6ac8141fd0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ac809c660 0x7f6ac8134bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient(hunting): picked mon.noname-g con 0x7f6ac809c660 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac809d630 con 0x7f6ac8141fd0 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac80a2390 con 0x7f6ac809c660 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac8136760 con 0x7f6ab4004d30 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.607+0000 7f6acfb9d640 10 monclient(hunting): authenticate will time out at 1676.358902s 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ab4004d30 0x7f6ac8141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ab4004d30 0x7f6ac8141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:41958/0 (socket says 172.21.15.178:41958) 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 1 -- 172.21.15.178:0/3529107422 learned_addr learned my addr 172.21.15.178:0/3529107422 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 10 monclient(hunting): get_auth_request con 0x7f6ab4004d30 auth_method 0 2024-09-15T01:47:51.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd138640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac8141fd0 0x7f6ac809c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6ace13a640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ac809c660 0x7f6ac8134bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd138640 10 monclient(hunting): get_auth_request con 0x7f6ac8141fd0 auth_method 0 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd138640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd138640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd138640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6ace13a640 10 monclient(hunting): get_auth_request con 0x7f6ac809c660 auth_method 0 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6ace13a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6ace13a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6ace13a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 10 monclient(hunting): handle_auth_done global_id 25620 payload 824 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 1 -- 172.21.15.178:0/3529107422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ac809c660 msgr2=0x7f6ac8134bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ac809c660 0x7f6ac8134bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 1 -- 172.21.15.178:0/3529107422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac8141fd0 msgr2=0x7f6ac809c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac8141fd0 0x7f6ac809c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 10 monclient: _finish_hunting 0 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 1 monclient: found mon.noname-d 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 1 -- 172.21.15.178:0/3529107422 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac8138220 con 0x7f6ab4004d30 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acd939640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ab4004d30 0x7f6ac8141a90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40028c0 tx=0x7f6ac4007ca0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ac4002e50 con 0x7f6ab4004d30 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abdffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abdffb640 10 monclient: dump: 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abdffb640 10 monclient: _finish_auth 0 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac40033b0 con 0x7f6ab4004d30 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acfb9d640 5 monclient: authenticate success, global_id 25620 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acfb9d640 10 monclient: _renew_subs 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acfb9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acfb9d640 1 -- 172.21.15.178:0/3529107422 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac8138500 con 0x7f6ab4004d30 2024-09-15T01:47:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ac400fdc0 con 0x7f6ab4004d30 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abdffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abdffb640 10 monclient: dump: 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.647 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6abffff640 4 set_mon_vals no callback set 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acfb9d640 10 monclient: _renew_subs 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acfb9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.608+0000 7f6acfb9d640 1 -- 172.21.15.178:0/3529107422 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac81350f0 con 0x7f6ab4004d30 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6ac400f940 con 0x7f6ab4004d30 2024-09-15T01:47:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6acfb9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6abdffb640 1 --2- 172.21.15.178:0/3529107422 >> v2:172.21.15.204:6812/28210 conn(0x7f6aac0397e0 0x7f6aac03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6acfb9d640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81352e0 0x7f6ac812fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6acfb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6acfb9d640 1 -- 172.21.15.178:0/3529107422 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac809cb40 con 0x7f6ac81352e0 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f6ac404bc00 con 0x7f6ab4004d30 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6ace13a640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81352e0 0x7f6ac812fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6ace13a640 10 monclient: get_auth_request con 0x7f6ac81352e0 auth_method 0 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6ace13a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6ace13a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6ace13a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6acd138640 1 --2- 172.21.15.178:0/3529107422 >> v2:172.21.15.204:6812/28210 conn(0x7f6aac0397e0 0x7f6aac03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6acd138640 10 monclient: get_auth_request con 0x7f6aac0397e0 auth_method 0 2024-09-15T01:47:51.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6ace13a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6ace13a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.609+0000 7f6ace13a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6ace13a640 10 monclient(hunting): handle_auth_done global_id 64377 payload 824 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6ace13a640 1 -- 172.21.15.178:0/3529107422 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ac804c4a0 con 0x7f6ac81352e0 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6ace13a640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81352e0 0x7f6ac812fcd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80a3290 tx=0x7f6ac000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ac000e040 con 0x7f6ac81352e0 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6abdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6acd138640 1 --2- 172.21.15.178:0/3529107422 >> v2:172.21.15.204:6812/28210 conn(0x7f6aac0397e0 0x7f6aac03bc90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8032000 tx=0x7f6ab8033000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6abdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6abdffb640 10 monclient: dump: 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.650 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f6ac000f040 con 0x7f6ac81352e0 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6abdffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6abdffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81352e0 msgr2=0x7f6ac812fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6abdffb640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81352e0 0x7f6ac812fcd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80a3290 tx=0x7f6ac000b040 comp rx=0 tx=0).stop 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81352e0 msgr2=0x7f6ac812fcd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.610+0000 7f6abdffb640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81352e0 0x7f6ac812fcd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6acfb9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6acfb9d640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81356b0 0x7f6ac8130210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6acfb9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6acfb9d640 1 -- 172.21.15.178:0/3529107422 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac81423a0 con 0x7f6ac81356b0 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6ace13a640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81356b0 0x7f6ac8130210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6ace13a640 10 monclient: get_auth_request con 0x7f6ac81356b0 auth_method 0 2024-09-15T01:47:51.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6ace13a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6ace13a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6ace13a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6ace13a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6ace13a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6ace13a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6ace13a640 10 monclient(hunting): handle_auth_done global_id 64386 payload 824 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6ace13a640 1 -- 172.21.15.178:0/3529107422 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6ac804bc80 con 0x7f6ac81356b0 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6ace13a640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81356b0 0x7f6ac8130210 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ac81364f0 tx=0x7f6ac001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ac0025070 con 0x7f6ac81356b0 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6abdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6abdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6abdffb640 10 monclient: dump: 2024-09-15T01:47:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.655 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f6ac0016b40 con 0x7f6ac81356b0 2024-09-15T01:47:51.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6abdffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:51.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6abdffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:51.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81356b0 msgr2=0x7f6ac8130210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6abdffb640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81356b0 0x7f6ac8130210 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ac81364f0 tx=0x7f6ac001e000 comp rx=0 tx=0).stop 2024-09-15T01:47:51.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6abdffb640 1 -- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81356b0 msgr2=0x7f6ac8130210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:51.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.616+0000 7f6abdffb640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ac81356b0 0x7f6ac8130210 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ac81364f0 tx=0x7f6ac001e000 comp rx=0 tx=0).stop 2024-09-15T01:47:51.655 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:51.656 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:51.657 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:51.658 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:51.659 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:51.660 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.661 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.662 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:51.662 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.662 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.662 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.662 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:51.662 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.662 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.662 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.663 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:51.664 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.666 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.667 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:51.667 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:51.667 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.667 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.667 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.667 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:51.668 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.668 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.668 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.668 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:51.668 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.668 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.668 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.669 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.670 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.671 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.672 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:51.672 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.672 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.672 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.672 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.672 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:51.672 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:51.672 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.672 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.673 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.673 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.673 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:51.673 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:51.673 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.673 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.673 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.673 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.673 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.674 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.675 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:51.676 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.677 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:51.678 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:51.679 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:51.680 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:51.680 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.680 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.680 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.680 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:51.680 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:51.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.617+0000 7f6acfb9d640 1 -- 172.21.15.178:0/3529107422 >> v2:172.21.15.204:6812/28210 conn(0x7f6aac0397e0 msgr2=0x7f6aac03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.617+0000 7f6acfb9d640 1 --2- 172.21.15.178:0/3529107422 >> v2:172.21.15.204:6812/28210 conn(0x7f6aac0397e0 0x7f6aac03bc90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8032000 tx=0x7f6ab8033000 comp rx=0 tx=0).stop 2024-09-15T01:47:51.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.617+0000 7f6acfb9d640 10 monclient: shutdown 2024-09-15T01:47:51.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.617+0000 7f6acfb9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:51.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.617+0000 7f6acfb9d640 1 -- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ab4004d30 msgr2=0x7f6ac8141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.617+0000 7f6ace13a640 1 -- 172.21.15.178:0/3529107422 reap_dead start 2024-09-15T01:47:51.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.617+0000 7f6acfb9d640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ab4004d30 0x7f6ac8141a90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40028c0 tx=0x7f6ac4007ca0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.617+0000 7f6acfb9d640 1 -- 172.21.15.178:0/3529107422 shutdown_connections 2024-09-15T01:47:51.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.617+0000 7f6acfb9d640 1 --2- 172.21.15.178:0/3529107422 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6ab4004d30 0x7f6ac8141a90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.617+0000 7f6acfb9d640 1 -- 172.21.15.178:0/3529107422 >> 172.21.15.178:0/3529107422 conn(0x7f6ac80a5ed0 msgr2=0x7f6ac812e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:51.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.617+0000 7f6acfb9d640 1 -- 172.21.15.178:0/3529107422 shutdown_connections 2024-09-15T01:47:51.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.617+0000 7f6acfb9d640 1 -- 172.21.15.178:0/3529107422 wait complete. 2024-09-15T01:47:51.683 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:51.683 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:47:51.683 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:47:51.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.773+0000 7f44fad9d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:51.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.773+0000 7f44fad9d640 10 monclient: build_initial_monmap 2024-09-15T01:47:51.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.774+0000 7f44fad9d640 10 monclient: monmap: 2024-09-15T01:47:51.777 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:51.777 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.777 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:51.775850+0000 2024-09-15T01:47:51.777 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:51.775850+0000 2024-09-15T01:47:51.777 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:51.777 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.777 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:51.777 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:51.778 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:51.778 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:51.778 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:51.778 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:51.778 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:51.778 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:51.778 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:51.778 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:51.778 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:51.778 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.774+0000 7f44fad9d640 1 Processor -- start 2024-09-15T01:47:51.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.774+0000 7f44fad9d640 1 -- start start 2024-09-15T01:47:51.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.774+0000 7f44fad9d640 10 monclient: init 2024-09-15T01:47:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 10 monclient: _add_conns ranks=[10,8,6,0,4,9,1,7,3,5,2] 2024-09-15T01:47:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f44f4063620 0x7f44f4097ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 10 monclient(hunting): picked mon.noname-f con 0x7f44f4063620 addr v2:172.21.15.204:3300/0 2024-09-15T01:47:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409ccd0 0x7f44f409d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 10 monclient(hunting): picked mon.noname-e con 0x7f44f409ccd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f44f409d5e0 0x7f44f413d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 10 monclient(hunting): picked mon.noname-d con 0x7f44f409d5e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f4098f20 con 0x7f44f4063620 2024-09-15T01:47:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4096fe0 con 0x7f44f409d5e0 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40639f0 con 0x7f44f409ccd0 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44fad9d640 10 monclient(hunting): authenticate will time out at 1676.526307s 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409ccd0 0x7f44f409d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f933a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f44f409d5e0 0x7f44f413d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409ccd0 0x7f44f409d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42052/0 (socket says 172.21.15.178:42052) 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 1 -- 172.21.15.178:0/3130394562 learned_addr learned my addr 172.21.15.178:0/3130394562 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 10 monclient(hunting): get_auth_request con 0x7f44f409ccd0 auth_method 0 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f933a640 10 monclient(hunting): get_auth_request con 0x7f44f409d5e0 auth_method 0 2024-09-15T01:47:51.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f933a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f933a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f933a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f933a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f933a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f933a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f8b39640 1 --2- 172.21.15.178:0/3130394562 >> v2:172.21.15.204:3300/0 conn(0x7f44f4063620 0x7f44f4097ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f8b39640 10 monclient(hunting): get_auth_request con 0x7f44f4063620 auth_method 0 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f8b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f8b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f8b39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 10 monclient(hunting): handle_auth_done global_id 94130 payload 274 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 1 -- 172.21.15.178:0/3130394562 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f44f409d5e0 msgr2=0x7f44f413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 1 --2- 172.21.15.178:0/3130394562 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f44f409d5e0 0x7f44f413d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 1 -- 172.21.15.178:0/3130394562 >> v2:172.21.15.204:3300/0 conn(0x7f44f4063620 msgr2=0x7f44f4097ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 1 --2- 172.21.15.178:0/3130394562 >> v2:172.21.15.204:3300/0 conn(0x7f44f4063620 0x7f44f4097ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 1 monclient: found mon.noname-e 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f3fff640 1 -- 172.21.15.178:0/3130394562 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f404bf60 con 0x7f44f409ccd0 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.775+0000 7f44f933a640 1 --2- 172.21.15.178:0/3130394562 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f44f409d5e0 0x7f44f413d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f3fff640 1 --2- 172.21.15.178:0/3130394562 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409ccd0 0x7f44f409d0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009940 tx=0x7f44e00091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=69c7c85e4f49b9eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f37fe640 1 -- 172.21.15.178:0/3130394562 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44e002ac70 con 0x7f44f409ccd0 2024-09-15T01:47:51.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:51.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f37fe640 10 monclient: dump: 2024-09-15T01:47:51.793 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.793 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.793 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.793 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.793 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.793 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.793 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f37fe640 10 monclient: _finish_auth 0 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f37fe640 1 -- 172.21.15.178:0/3130394562 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e002adf0 con 0x7f44f409ccd0 2024-09-15T01:47:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44fad9d640 5 monclient: authenticate success, global_id 94130 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44fad9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44fad9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f37fe640 1 -- 172.21.15.178:0/3130394562 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44e0031ca0 con 0x7f44f409ccd0 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44fad9d640 4 set_mon_vals no callback set 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.776+0000 7f44f37fe640 10 monclient: dump: 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.795 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 10 monclient: shutdown 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 1 -- 172.21.15.178:0/3130394562 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409ccd0 msgr2=0x7f44f409d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 1 --2- 172.21.15.178:0/3130394562 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409ccd0 0x7f44f409d0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009940 tx=0x7f44e00091d0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 1 -- 172.21.15.178:0/3130394562 shutdown_connections 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 1 --2- 172.21.15.178:0/3130394562 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409ccd0 0x7f44f409d0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 1 --2- 172.21.15.178:0/3130394562 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f44f409d5e0 0x7f44f413d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 1 --2- 172.21.15.178:0/3130394562 >> v2:172.21.15.204:3300/0 conn(0x7f44f4063620 0x7f44f4097ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 1 -- 172.21.15.178:0/3130394562 >> 172.21.15.178:0/3130394562 conn(0x7f44f40a22e0 msgr2=0x7f44f40a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 1 -- 172.21.15.178:0/3130394562 shutdown_connections 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 1 -- 172.21.15.178:0/3130394562 wait complete. 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 10 monclient: build_initial_monmap 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 10 monclient: monmap: 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:51.797 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.777+0000 7f44fad9d640 1 Processor -- start 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 1 -- start start 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 10 monclient: init 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 10 monclient: _add_conns ranks=[3,7,8,4,2,0,5,6,1] 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44f4063620 0x7f44f4136670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 10 monclient(hunting): picked mon.noname-g con 0x7f44f4063620 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f44f409ccd0 0x7f44f4136bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 10 monclient(hunting): picked mon.noname-i con 0x7f44f409ccd0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409d5e0 0x7f44f41370f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 10 monclient(hunting): picked mon.noname-e con 0x7f44f409d5e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4119ae0 con 0x7f44f4063620 2024-09-15T01:47:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f44f409e610 con 0x7f44f409ccd0 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f44f409e840 con 0x7f44f409d5e0 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44fad9d640 10 monclient(hunting): authenticate will time out at 1676.529702s 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f933a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409d5e0 0x7f44f41370f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f933a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409d5e0 0x7f44f41370f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42066/0 (socket says 172.21.15.178:42066) 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f933a640 1 -- 172.21.15.178:0/228447759 learned_addr learned my addr 172.21.15.178:0/228447759 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f933a640 10 monclient(hunting): get_auth_request con 0x7f44f409d5e0 auth_method 0 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f933a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f933a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f933a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f8b39640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44f4063620 0x7f44f4136670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f3fff640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f44f409ccd0 0x7f44f4136bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f3fff640 10 monclient(hunting): get_auth_request con 0x7f44f409ccd0 auth_method 0 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f8b39640 10 monclient(hunting): get_auth_request con 0x7f44f4063620 auth_method 0 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f8b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f8b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f8b39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f933a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f933a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f933a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f8b39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.778+0000 7f44f8b39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f8b39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f933a640 10 monclient(hunting): handle_auth_done global_id 94139 payload 824 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f933a640 1 -- 172.21.15.178:0/228447759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f44f409ccd0 msgr2=0x7f44f4136bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f933a640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f44f409ccd0 0x7f44f4136bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f933a640 1 -- 172.21.15.178:0/228447759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44f4063620 msgr2=0x7f44f4136670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f933a640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44f4063620 0x7f44f4136670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f933a640 10 monclient: _finish_hunting 0 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f933a640 1 monclient: found mon.noname-e 2024-09-15T01:47:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f933a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:51.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f933a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:51.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f933a640 1 -- 172.21.15.178:0/228447759 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec009a20 con 0x7f44f409d5e0 2024-09-15T01:47:51.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f933a640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409d5e0 0x7f44f41370f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009fc0 tx=0x7f44ec00b040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44ec00ce70 con 0x7f44f409d5e0 2024-09-15T01:47:51.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f0ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:51.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f0ff9640 10 monclient: dump: 2024-09-15T01:47:51.803 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.803 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.803 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.803 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f0ff9640 10 monclient: _finish_auth 0 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44fad9d640 5 monclient: authenticate success, global_id 94139 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec00f040 con 0x7f44f409d5e0 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44fad9d640 10 monclient: _renew_subs 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44ec009d60 con 0x7f44f409d5e0 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44fad9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44fad9d640 1 -- 172.21.15.178:0/228447759 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e0009450 con 0x7f44f409d5e0 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f2ffd640 4 set_mon_vals no callback set 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44f0ff9640 10 monclient: dump: 2024-09-15T01:47:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44fad9d640 10 monclient: _renew_subs 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44fad9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.779+0000 7f44fad9d640 1 -- 172.21.15.178:0/228447759 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f411b650 con 0x7f44f409d5e0 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44fad9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44fad9d640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f411c2b0 0x7f44f4118940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44fad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44fad9d640 1 -- 172.21.15.178:0/228447759 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4068530 con 0x7f44f411c2b0 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f44ec003020 con 0x7f44f409d5e0 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f8b39640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f411c2b0 0x7f44f4118940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f8b39640 10 monclient: get_auth_request con 0x7f44f411c2b0 auth_method 0 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f8b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f8b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f8b39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f8b39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f8b39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f8b39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f0ff9640 1 --2- 172.21.15.178:0/228447759 >> v2:172.21.15.204:6812/28210 conn(0x7f44d80396e0 0x7f44d803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f44ec04c800 con 0x7f44f409d5e0 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f8b39640 10 monclient(hunting): handle_auth_done global_id 35165 payload 824 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f8b39640 1 -- 172.21.15.178:0/228447759 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44f404c4a0 con 0x7f44f411c2b0 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f3fff640 1 --2- 172.21.15.178:0/228447759 >> v2:172.21.15.204:6812/28210 conn(0x7f44d80396e0 0x7f44d803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f8b39640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f411c2b0 0x7f44f4118940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f44e8010bb0 tx=0x7f44e8008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.780+0000 7f44f3fff640 10 monclient: get_auth_request con 0x7f44d80396e0 auth_method 0 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.781+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44e8014430 con 0x7f44f411c2b0 2024-09-15T01:47:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.781+0000 7f44f0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.781+0000 7f44f0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.781+0000 7f44f0ff9640 10 monclient: dump: 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.809 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.781+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f44e800ebc0 con 0x7f44f411c2b0 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.781+0000 7f44f0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.781+0000 7f44f0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.781+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f411c2b0 msgr2=0x7f44f4118940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.781+0000 7f44f0ff9640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f411c2b0 0x7f44f4118940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f44e8010bb0 tx=0x7f44e8008040 comp rx=0 tx=0).stop 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.781+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f411c2b0 msgr2=0x7f44f4118940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.781+0000 7f44f0ff9640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f411c2b0 0x7f44f4118940 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.781+0000 7f44f3fff640 1 --2- 172.21.15.178:0/228447759 >> v2:172.21.15.204:6812/28210 conn(0x7f44d80396e0 0x7f44d803bb90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f44e0034000 tx=0x7f44e0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.786+0000 7f44fad9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:51.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.786+0000 7f44fad9d640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f4118e80 0x7f44f4119280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.786+0000 7f44fad9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.786+0000 7f44fad9d640 1 -- 172.21.15.178:0/228447759 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f404bc80 con 0x7f44f4118e80 2024-09-15T01:47:51.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f8b39640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f4118e80 0x7f44f4119280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f8b39640 10 monclient: get_auth_request con 0x7f44f4118e80 auth_method 0 2024-09-15T01:47:51.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f8b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f8b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f8b39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f8b39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f8b39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f8b39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f8b39640 10 monclient(hunting): handle_auth_done global_id 35174 payload 824 2024-09-15T01:47:51.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f8b39640 1 -- 172.21.15.178:0/228447759 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f44f405ed70 con 0x7f44f4118e80 2024-09-15T01:47:51.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f8b39640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f4118e80 0x7f44f4119280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f44e8010bb0 tx=0x7f44e801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44e8022070 con 0x7f44f4118e80 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f0ff9640 10 monclient: dump: 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.818 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.818 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.818 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.818 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.818 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f44e801de30 con 0x7f44f4118e80 2024-09-15T01:47:51.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:51.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:51.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f4118e80 msgr2=0x7f44f4119280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f0ff9640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f4118e80 0x7f44f4119280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f44e8010bb0 tx=0x7f44e801d000 comp rx=0 tx=0).stop 2024-09-15T01:47:51.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f0ff9640 1 -- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f4118e80 msgr2=0x7f44f4119280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:51.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.787+0000 7f44f0ff9640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44f4118e80 0x7f44f4119280 secure :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f44e8010bb0 tx=0x7f44e801d000 comp rx=0 tx=0).stop 2024-09-15T01:47:51.819 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:47:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:51.843 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:51.843 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:51.843 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.843 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.843 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:51.843 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:51.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.788+0000 7f44fad9d640 1 -- 172.21.15.178:0/228447759 >> v2:172.21.15.204:6812/28210 conn(0x7f44d80396e0 msgr2=0x7f44d803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.788+0000 7f44fad9d640 1 --2- 172.21.15.178:0/228447759 >> v2:172.21.15.204:6812/28210 conn(0x7f44d80396e0 0x7f44d803bb90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f44e0034000 tx=0x7f44e0032000 comp rx=0 tx=0).stop 2024-09-15T01:47:51.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.788+0000 7f44fad9d640 10 monclient: shutdown 2024-09-15T01:47:51.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.788+0000 7f44fad9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:51.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.788+0000 7f44fad9d640 1 -- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409d5e0 msgr2=0x7f44f41370f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.788+0000 7f44f933a640 1 -- 172.21.15.178:0/228447759 reap_dead start 2024-09-15T01:47:51.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.788+0000 7f44fad9d640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409d5e0 0x7f44f41370f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009fc0 tx=0x7f44ec00b040 comp rx=0 tx=0).stop 2024-09-15T01:47:51.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.788+0000 7f44fad9d640 1 -- 172.21.15.178:0/228447759 shutdown_connections 2024-09-15T01:47:51.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.788+0000 7f44fad9d640 1 --2- 172.21.15.178:0/228447759 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f44f409d5e0 0x7f44f41370f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.788+0000 7f44fad9d640 1 -- 172.21.15.178:0/228447759 >> 172.21.15.178:0/228447759 conn(0x7f44f40a22e0 msgr2=0x7f44f4099c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:51.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.788+0000 7f44fad9d640 1 -- 172.21.15.178:0/228447759 shutdown_connections 2024-09-15T01:47:51.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.788+0000 7f44fad9d640 1 -- 172.21.15.178:0/228447759 wait complete. 2024-09-15T01:47:51.846 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:51.846 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:47:51.846 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:47:51.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.937+0000 7f757ee3f640 10 monclient: get_monmap_and_config 2024-09-15T01:47:51.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.937+0000 7f757ee3f640 10 monclient: build_initial_monmap 2024-09-15T01:47:51.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.937+0000 7f757ee3f640 10 monclient: monmap: 2024-09-15T01:47:51.940 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:51.939007+0000 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:51.939007+0000 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:51.942 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:51.942 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:51.942 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:51.942 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.937+0000 7f757ee3f640 1 Processor -- start 2024-09-15T01:47:51.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.937+0000 7f757ee3f640 1 -- start start 2024-09-15T01:47:51.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.937+0000 7f757ee3f640 10 monclient: init 2024-09-15T01:47:51.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:51.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 10 monclient: _add_conns ranks=[2,3,4,6,10,9,8,5,7,1,0] 2024-09-15T01:47:51.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75780a2930 0x7f7578132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 10 monclient(hunting): picked mon.noname-b con 0x7f75780a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:51.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f757809faa0 0x7f757809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 10 monclient(hunting): picked mon.noname-g con 0x7f757809faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:51.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75780a08c0 0x7f75780a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 10 monclient(hunting): picked mon.noname-c con 0x7f75780a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:47:51.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757809e0f0 con 0x7f75780a2930 2024-09-15T01:47:51.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757809dd80 con 0x7f757809faa0 2024-09-15T01:47:51.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75780a2d00 con 0x7f75780a08c0 2024-09-15T01:47:51.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:51.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757ee3f640 10 monclient(hunting): authenticate will time out at 1676.689433s 2024-09-15T01:47:51.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75780a2930 0x7f7578132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75780a2930 0x7f7578132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42004/0 (socket says 172.21.15.178:42004) 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757d3dc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75780a08c0 0x7f75780a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 1 -- 172.21.15.178:0/2716107859 learned_addr learned my addr 172.21.15.178:0/2716107859 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 10 monclient(hunting): get_auth_request con 0x7f75780a2930 auth_method 0 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757d3dc640 10 monclient(hunting): get_auth_request con 0x7f75780a08c0 auth_method 0 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757d3dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757d3dc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757d3dc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757d3dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757d3dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757d3dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f756ffff640 1 --2- 172.21.15.178:0/2716107859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f757809faa0 0x7f757809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f756ffff640 10 monclient(hunting): get_auth_request con 0x7f757809faa0 auth_method 0 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f756ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f756ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f756ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 10 monclient(hunting): handle_auth_done global_id 35183 payload 274 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 1 -- 172.21.15.178:0/2716107859 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75780a08c0 msgr2=0x7f75780a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 1 --2- 172.21.15.178:0/2716107859 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75780a08c0 0x7f75780a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 1 -- 172.21.15.178:0/2716107859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f757809faa0 msgr2=0x7f757809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 1 --2- 172.21.15.178:0/2716107859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f757809faa0 0x7f757809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 10 monclient: _finish_hunting 0 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 1 monclient: found mon.noname-b 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.938+0000 7f757cbdb640 1 -- 172.21.15.178:0/2716107859 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757804bf60 con 0x7f75780a2930 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f757cbdb640 1 --2- 172.21.15.178:0/2716107859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75780a2930 0x7f7578132de0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7574009940 tx=0x7f75740091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6e70f4dbba2716a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f756f7fe640 1 -- 172.21.15.178:0/2716107859 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f757402ac60 con 0x7f75780a2930 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f756f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f756f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f756f7fe640 10 monclient: dump: 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f756f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f756f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f756f7fe640 1 -- 172.21.15.178:0/2716107859 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757402ade0 con 0x7f75780a2930 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f757ee3f640 5 monclient: authenticate success, global_id 35183 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f757ee3f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f756f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f757ee3f640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f756f7fe640 1 -- 172.21.15.178:0/2716107859 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7574031ca0 con 0x7f75780a2930 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f757ee3f640 4 set_mon_vals no callback set 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f756f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f756f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.939+0000 7f756f7fe640 10 monclient: dump: 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.957 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.958 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 10 monclient: shutdown 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 1 -- 172.21.15.178:0/2716107859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75780a2930 msgr2=0x7f7578132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 1 --2- 172.21.15.178:0/2716107859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75780a2930 0x7f7578132de0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7574009940 tx=0x7f75740091d0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 1 -- 172.21.15.178:0/2716107859 shutdown_connections 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 1 --2- 172.21.15.178:0/2716107859 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f75780a08c0 0x7f75780a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 1 --2- 172.21.15.178:0/2716107859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f757809faa0 0x7f757809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 1 --2- 172.21.15.178:0/2716107859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75780a2930 0x7f7578132de0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 1 -- 172.21.15.178:0/2716107859 >> 172.21.15.178:0/2716107859 conn(0x7f75780a5ed0 msgr2=0x7f75780a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 1 -- 172.21.15.178:0/2716107859 shutdown_connections 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 1 -- 172.21.15.178:0/2716107859 wait complete. 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 10 monclient: build_initial_monmap 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.940+0000 7f757ee3f640 10 monclient: monmap: 2024-09-15T01:47:51.959 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 1 Processor -- start 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 1 -- start start 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 10 monclient: init 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 10 monclient: _add_conns ranks=[8,6,1,3,4,0,7,2,5] 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f757809faa0 0x7f757809b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 10 monclient(hunting): picked mon.noname-e con 0x7f757809faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f75780a08c0 0x7f7578131890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 10 monclient(hunting): picked mon.noname-d con 0x7f75780a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75780a2930 0x7f757811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 10 monclient(hunting): picked mon.noname-f con 0x7f75780a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75780a2390 con 0x7f75780a2930 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f757809d630 con 0x7f75780a08c0 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7578142d80 con 0x7f757809faa0 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757ee3f640 10 monclient(hunting): authenticate will time out at 1676.692891s 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.941+0000 7f757cbdb640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f757809faa0 0x7f757809b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f757809faa0 0x7f757809b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42074/0 (socket says 172.21.15.178:42074) 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 1 -- 172.21.15.178:0/3493783294 learned_addr learned my addr 172.21.15.178:0/3493783294 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 10 monclient(hunting): get_auth_request con 0x7f757809faa0 auth_method 0 2024-09-15T01:47:51.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756ffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f75780a08c0 0x7f7578131890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756ffff640 10 monclient(hunting): get_auth_request con 0x7f75780a08c0 auth_method 0 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757d3dc640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75780a2930 0x7f757811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757d3dc640 10 monclient(hunting): get_auth_request con 0x7f75780a2930 auth_method 0 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757d3dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757d3dc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757d3dc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757d3dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757d3dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757d3dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 10 monclient(hunting): handle_auth_done global_id 94148 payload 824 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 1 -- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f75780a08c0 msgr2=0x7f7578131890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f75780a08c0 0x7f7578131890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 1 -- 172.21.15.178:0/3493783294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75780a2930 msgr2=0x7f757811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75780a2930 0x7f757811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 10 monclient: _finish_hunting 0 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 1 monclient: found mon.noname-e 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 1 -- 172.21.15.178:0/3493783294 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7574009450 con 0x7f757809faa0 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757cbdb640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f757809faa0 0x7f757809b380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7574000c00 tx=0x7f7574032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7574031820 con 0x7f757809faa0 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756cff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756cff9640 10 monclient: dump: 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756cff9640 10 monclient: _finish_auth 0 2024-09-15T01:47:51.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757ee3f640 5 monclient: authenticate success, global_id 94148 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757ee3f640 10 monclient: _renew_subs 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75740319e0 con 0x7f757809faa0 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757ee3f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757ee3f640 1 -- 172.21.15.178:0/3493783294 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757809bd80 con 0x7f757809faa0 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f757403a390 con 0x7f757809faa0 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756cff9640 10 monclient: dump: 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.967 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757ee3f640 10 monclient: _renew_subs 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757ee3f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f757ee3f640 1 -- 172.21.15.178:0/3493783294 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75781423c0 con 0x7f757809faa0 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.942+0000 7f756effd640 4 set_mon_vals no callback set 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f757ee3f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f757ee3f640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f757811da50 0x7f757811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f757ee3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f757ee3f640 1 -- 172.21.15.178:0/3493783294 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f757811f470 con 0x7f757811da50 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f7574039040 con 0x7f757809faa0 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756ffff640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f757811da50 0x7f757811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756ffff640 10 monclient: get_auth_request con 0x7f757811da50 auth_method 0 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756cff9640 1 --2- 172.21.15.178:0/3493783294 >> v2:172.21.15.204:6812/28210 conn(0x7f75640396e0 0x7f756403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f757406b090 con 0x7f757809faa0 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756ffff640 10 monclient(hunting): handle_auth_done global_id 44779 payload 824 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.943+0000 7f756ffff640 1 -- 172.21.15.178:0/3493783294 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f757804c4a0 con 0x7f757811da50 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f757d3dc640 1 --2- 172.21.15.178:0/3493783294 >> v2:172.21.15.204:6812/28210 conn(0x7f75640396e0 0x7f756403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f756ffff640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f757811da50 0x7f757811ca70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f75780a6ec0 tx=0x7f7568008040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f757d3dc640 10 monclient: get_auth_request con 0x7f75640396e0 auth_method 0 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7568016480 con 0x7f757811da50 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f756cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f756cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f756cff9640 10 monclient: dump: 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.971 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.972 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.972 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.972 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f756800eb40 con 0x7f757811da50 2024-09-15T01:47:51.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f756cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:51.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f756cff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:51.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f757811da50 msgr2=0x7f757811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f756cff9640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f757811da50 0x7f757811ca70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f75780a6ec0 tx=0x7f7568008040 comp rx=0 tx=0).stop 2024-09-15T01:47:51.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f757811da50 msgr2=0x7f757811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:51.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f756cff9640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f757811da50 0x7f757811ca70 secure :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f75780a6ec0 tx=0x7f7568008040 comp rx=0 tx=0).stop 2024-09-15T01:47:51.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.944+0000 7f757d3dc640 1 --2- 172.21.15.178:0/3493783294 >> v2:172.21.15.204:6812/28210 conn(0x7f75640396e0 0x7f756403bb90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7570009fd0 tx=0x7f757000b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f757ee3f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:51.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f757ee3f640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7578131dd0 0x7f757811cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:51.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f757ee3f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:51.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f757ee3f640 1 -- 172.21.15.178:0/3493783294 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f757804bc80 con 0x7f7578131dd0 2024-09-15T01:47:51.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756ffff640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7578131dd0 0x7f757811cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:51.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756ffff640 10 monclient: get_auth_request con 0x7f7578131dd0 auth_method 0 2024-09-15T01:47:51.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:51.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:51.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:51.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:51.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:51.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756ffff640 10 monclient(hunting): handle_auth_done global_id 44788 payload 824 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756ffff640 1 -- 172.21.15.178:0/3493783294 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f757805ed70 con 0x7f7578131dd0 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756ffff640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7578131dd0 0x7f757811cfb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f756801b000 tx=0x7f756801e000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7568020c90 con 0x7f7578131dd0 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756cff9640 10 monclient: dump: 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:51.976 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9482 (secure 0 0 0) 0x7f7568020df0 con 0x7f7578131dd0 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:51.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756cff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:51.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7578131dd0 msgr2=0x7f757811cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:51.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756cff9640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7578131dd0 0x7f757811cfb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f756801b000 tx=0x7f756801e000 comp rx=0 tx=0).stop 2024-09-15T01:47:51.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756cff9640 1 -- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7578131dd0 msgr2=0x7f757811cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:51.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.950+0000 7f756cff9640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f7578131dd0 0x7f757811cfb0 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f756801b000 tx=0x7f756801e000 comp rx=0 tx=0).stop 2024-09-15T01:47:51.978 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:51.978 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:51.978 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:51.978 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:51.978 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:51.978 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:51.979 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:51.980 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:51.981 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:51.982 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:51.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.984 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:51.985 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.987 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.988 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.989 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:51.990 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.992 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.993 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.994 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:51.995 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.996 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.997 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:51.998 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:47:51.999 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.000 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:47:52.001 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.001 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.001 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.001 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.001 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.001 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.001 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:52.001 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.951+0000 7f757ee3f640 1 -- 172.21.15.178:0/3493783294 >> v2:172.21.15.204:6812/28210 conn(0x7f75640396e0 msgr2=0x7f756403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.951+0000 7f757ee3f640 1 --2- 172.21.15.178:0/3493783294 >> v2:172.21.15.204:6812/28210 conn(0x7f75640396e0 0x7f756403bb90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7570009fd0 tx=0x7f757000b040 comp rx=0 tx=0).stop 2024-09-15T01:47:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.951+0000 7f757ee3f640 10 monclient: shutdown 2024-09-15T01:47:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.951+0000 7f757ee3f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.951+0000 7f757ee3f640 1 -- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f757809faa0 msgr2=0x7f757809b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.951+0000 7f757ee3f640 1 --2- 172.21.15.178:0/3493783294 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f757809faa0 0x7f757809b380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7574000c00 tx=0x7f7574032000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.951+0000 7f757d3dc640 1 -- 172.21.15.178:0/3493783294 reap_dead start 2024-09-15T01:47:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.951+0000 7f757d3dc640 1 -- 172.21.15.178:0/3493783294 reap_dead start 2024-09-15T01:47:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.951+0000 7f757ee3f640 1 -- 172.21.15.178:0/3493783294 shutdown_connections 2024-09-15T01:47:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.951+0000 7f757ee3f640 1 -- 172.21.15.178:0/3493783294 >> 172.21.15.178:0/3493783294 conn(0x7f75780a5ed0 msgr2=0x7f757809e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.951+0000 7f757ee3f640 1 -- 172.21.15.178:0/3493783294 shutdown_connections 2024-09-15T01:47:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:51.951+0000 7f757ee3f640 1 -- 172.21.15.178:0/3493783294 wait complete. 2024-09-15T01:47:52.004 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:52.004 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:47:52.004 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:47:52.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.094+0000 7fdd3f15d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:52.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.094+0000 7fdd3f15d640 10 monclient: build_initial_monmap 2024-09-15T01:47:52.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.094+0000 7fdd3f15d640 10 monclient: monmap: 2024-09-15T01:47:52.098 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:52.098 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.098 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:52.096819+0000 2024-09-15T01:47:52.098 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:52.096819+0000 2024-09-15T01:47:52.098 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:52.098 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.098 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:52.098 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:52.099 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:52.099 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:52.099 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:52.099 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:52.099 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:52.099 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:52.099 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:52.099 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:52.099 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:52.099 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.095+0000 7fdd3f15d640 1 Processor -- start 2024-09-15T01:47:52.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.095+0000 7fdd3f15d640 1 -- start start 2024-09-15T01:47:52.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.095+0000 7fdd3f15d640 10 monclient: init 2024-09-15T01:47:52.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 10 monclient: _add_conns ranks=[10,5,3,9,4,1,0,7,6,2,8] 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdd380a2930 0x7fdd38132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 10 monclient(hunting): picked mon.noname-f con 0x7fdd380a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdd3809faa0 0x7fdd3809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 10 monclient(hunting): picked mon.noname-h con 0x7fdd3809faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd380a08c0 0x7fdd380a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 10 monclient(hunting): picked mon.noname-g con 0x7fdd380a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd3809e0f0 con 0x7fdd380a2930 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3809dd80 con 0x7fdd380a08c0 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd380a2d00 con 0x7fdd3809faa0 2024-09-15T01:47:52.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3f15d640 10 monclient(hunting): authenticate will time out at 1676.847193s 2024-09-15T01:47:52.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd37fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdd3809faa0 0x7fdd3809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdd380a2930 0x7fdd38132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3d6fa640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd380a08c0 0x7fdd380a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdd380a2930 0x7fdd38132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:59954/0 (socket says 172.21.15.178:59954) 2024-09-15T01:47:52.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 1 -- 172.21.15.178:0/2437440226 learned_addr learned my addr 172.21.15.178:0/2437440226 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:52.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd37fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdd3809faa0 0x7fdd3809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:44248/0 (socket says 172.21.15.178:44248) 2024-09-15T01:47:52.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3d6fa640 10 monclient(hunting): get_auth_request con 0x7fdd380a08c0 auth_method 0 2024-09-15T01:47:52.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3d6fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3d6fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3d6fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 10 monclient(hunting): get_auth_request con 0x7fdd380a2930 auth_method 0 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd37fff640 10 monclient(hunting): get_auth_request con 0x7fdd3809faa0 auth_method 0 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd37fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd37fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3d6fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3d6fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3d6fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 10 monclient(hunting): handle_auth_done global_id 94141 payload 274 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 1 -- 172.21.15.178:0/2437440226 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdd3809faa0 msgr2=0x7fdd3809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 1 --2- 172.21.15.178:0/2437440226 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdd3809faa0 0x7fdd3809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 1 -- 172.21.15.178:0/2437440226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd380a08c0 msgr2=0x7fdd380a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 1 --2- 172.21.15.178:0/2437440226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd380a08c0 0x7fdd380a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 1 monclient: found mon.noname-f 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.204:3300/0 2024-09-15T01:47:52.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3cef9640 1 -- 172.21.15.178:0/2437440226 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3804bf60 con 0x7fdd380a2930 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.096+0000 7fdd3d6fa640 1 --2- 172.21.15.178:0/2437440226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd380a08c0 0x7fdd380a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd3cef9640 1 --2- 172.21.15.178:0/2437440226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd380a2930 0x7fdd38132de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009f20 tx=0x7fdd30009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d373e11d6bb4946a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd377fe640 1 -- 172.21.15.178:0/2437440226 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd30007880 con 0x7fdd380a2930 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd377fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd377fe640 10 monclient: dump: 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.114 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd377fe640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd377fe640 1 -- 172.21.15.178:0/2437440226 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3002d050 con 0x7fdd380a2930 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd3f15d640 5 monclient: authenticate success, global_id 94141 2024-09-15T01:47:52.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd3f15d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd377fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd3f15d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd3f15d640 4 set_mon_vals no callback set 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd377fe640 1 -- 172.21.15.178:0/2437440226 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd30030360 con 0x7fdd380a2930 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd377fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.097+0000 7fdd377fe640 10 monclient: dump: 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.116 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 10 monclient: shutdown 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 1 -- 172.21.15.178:0/2437440226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd380a2930 msgr2=0x7fdd38132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 1 --2- 172.21.15.178:0/2437440226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd380a2930 0x7fdd38132de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009f20 tx=0x7fdd30009840 comp rx=0 tx=0).stop 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 1 -- 172.21.15.178:0/2437440226 shutdown_connections 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 1 --2- 172.21.15.178:0/2437440226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd380a08c0 0x7fdd380a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 1 --2- 172.21.15.178:0/2437440226 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdd3809faa0 0x7fdd3809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 1 --2- 172.21.15.178:0/2437440226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd380a2930 0x7fdd38132de0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 1 -- 172.21.15.178:0/2437440226 >> 172.21.15.178:0/2437440226 conn(0x7fdd380a5ed0 msgr2=0x7fdd380a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 1 -- 172.21.15.178:0/2437440226 shutdown_connections 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 1 -- 172.21.15.178:0/2437440226 wait complete. 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 10 monclient: build_initial_monmap 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.098+0000 7fdd3f15d640 10 monclient: monmap: 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:52.118 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 1 Processor -- start 2024-09-15T01:47:52.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 1 -- start start 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 10 monclient: init 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 10 monclient: _add_conns ranks=[0,8,5,2,4,3,1,7,6] 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3809faa0 0x7fdd3809b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 10 monclient(hunting): picked mon.noname-a con 0x7fdd3809faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd380a08c0 0x7fdd3811c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 10 monclient(hunting): picked mon.noname-e con 0x7fdd380a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdd380a2930 0x7fdd381318e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 10 monclient(hunting): picked mon.noname-h con 0x7fdd380a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3811d750 con 0x7fdd3809faa0 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3811cf30 con 0x7fdd380a2930 2024-09-15T01:47:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3811dd10 con 0x7fdd380a08c0 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.099+0000 7fdd3f15d640 10 monclient(hunting): authenticate will time out at 1676.850923s 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd37fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd380a08c0 0x7fdd3811c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3809faa0 0x7fdd3809b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3809faa0 0x7fdd3809b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35828/0 (socket says 172.21.15.178:35828) 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd37fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd380a08c0 0x7fdd3811c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42080/0 (socket says 172.21.15.178:42080) 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 1 -- 172.21.15.178:0/3740511469 learned_addr learned my addr 172.21.15.178:0/3740511469 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 10 monclient(hunting): get_auth_request con 0x7fdd3809faa0 auth_method 0 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd37fff640 10 monclient(hunting): get_auth_request con 0x7fdd380a08c0 auth_method 0 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd37fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd37fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3d6fa640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdd380a2930 0x7fdd381318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3d6fa640 10 monclient(hunting): get_auth_request con 0x7fdd380a2930 auth_method 0 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3d6fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3d6fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3d6fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3d6fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3d6fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3d6fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 10 monclient(hunting): handle_auth_done global_id 64395 payload 824 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 1 -- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd380a08c0 msgr2=0x7fdd3811c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd380a08c0 0x7fdd3811c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 1 -- 172.21.15.178:0/3740511469 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdd380a2930 msgr2=0x7fdd381318e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdd380a2930 0x7fdd381318e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 1 monclient: found mon.noname-a 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 1 -- 172.21.15.178:0/3740511469 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd28009390 con 0x7fdd3809faa0 2024-09-15T01:47:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd37fff640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdd380a08c0 0x7fdd3811c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3cef9640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3809faa0 0x7fdd3809b040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdd30000f80 tx=0x7fdd30036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd3003baa0 con 0x7fdd3809faa0 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd34ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd34ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd34ff9640 10 monclient: dump: 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd34ff9640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3f15d640 5 monclient: authenticate success, global_id 64395 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3003bc60 con 0x7fdd3809faa0 2024-09-15T01:47:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3f15d640 10 monclient: _renew_subs 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3f15d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3f15d640 1 -- 172.21.15.178:0/3740511469 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd30009ae0 con 0x7fdd3809faa0 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd34ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd3003bdd0 con 0x7fdd3809faa0 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd34ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd34ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd34ff9640 10 monclient: dump: 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.126 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3f15d640 10 monclient: _renew_subs 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3f15d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd3f15d640 1 -- 172.21.15.178:0/3740511469 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3809bb30 con 0x7fdd3809faa0 2024-09-15T01:47:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.100+0000 7fdd36ffd640 4 set_mon_vals no callback set 2024-09-15T01:47:52.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdd30038070 con 0x7fdd3809faa0 2024-09-15T01:47:52.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd3f15d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:52.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd3f15d640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38131e20 0x7fdd38141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd3f15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd3f15d640 1 -- 172.21.15.178:0/3740511469 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdd381321f0 con 0x7fdd38131e20 2024-09-15T01:47:52.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd34ff9640 1 --2- 172.21.15.178:0/3740511469 >> v2:172.21.15.204:6812/28210 conn(0x7fdd240397b0 0x7fdd2403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fdd3006c550 con 0x7fdd3809faa0 2024-09-15T01:47:52.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd37fff640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38131e20 0x7fdd38141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd37fff640 10 monclient: get_auth_request con 0x7fdd38131e20 auth_method 0 2024-09-15T01:47:52.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd37fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd37fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.101+0000 7fdd3d6fa640 1 --2- 172.21.15.178:0/3740511469 >> v2:172.21.15.204:6812/28210 conn(0x7fdd240397b0 0x7fdd2403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd3d6fa640 10 monclient: get_auth_request con 0x7fdd240397b0 auth_method 0 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd37fff640 10 monclient(hunting): handle_auth_done global_id 25647 payload 824 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd37fff640 1 -- 172.21.15.178:0/3740511469 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd3804c4a0 con 0x7fdd38131e20 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd37fff640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38131e20 0x7fdd38141d40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c011d60 tx=0x7fdd2c0090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd3d6fa640 1 --2- 172.21.15.178:0/3740511469 >> v2:172.21.15.204:6812/28210 conn(0x7fdd240397b0 0x7fdd2403bc60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdd3809b6f0 tx=0x7fdd280090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd2c0148c0 con 0x7fdd38131e20 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd34ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd34ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd34ff9640 10 monclient: dump: 2024-09-15T01:47:52.129 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.130 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.131 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fdd2c014440 con 0x7fdd38131e20 2024-09-15T01:47:52.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd34ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:52.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd34ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:52.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38131e20 msgr2=0x7fdd38141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd34ff9640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38131e20 0x7fdd38141d40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c011d60 tx=0x7fdd2c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38131e20 msgr2=0x7fdd38141d40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:52.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.102+0000 7fdd34ff9640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38131e20 0x7fdd38141d40 secure :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c011d60 tx=0x7fdd2c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd3f15d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:52.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd3f15d640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38142280 0x7fdd38142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd3f15d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd3f15d640 1 -- 172.21.15.178:0/3740511469 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3804bc80 con 0x7fdd38142280 2024-09-15T01:47:52.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd37fff640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38142280 0x7fdd38142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd37fff640 10 monclient: get_auth_request con 0x7fdd38142280 auth_method 0 2024-09-15T01:47:52.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd37fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd37fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd37fff640 10 monclient(hunting): handle_auth_done global_id 25656 payload 824 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd37fff640 1 -- 172.21.15.178:0/3740511469 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdd3805ed70 con 0x7fdd38142280 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd37fff640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38142280 0x7fdd38142680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c011d60 tx=0x7fdd2c01c000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd2c024040 con 0x7fdd38142280 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd34ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd34ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.108+0000 7fdd34ff9640 10 monclient: dump: 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fdd2c0133a0 con 0x7fdd38142280 2024-09-15T01:47:52.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd34ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:52.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd34ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:52.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38142280 msgr2=0x7fdd38142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd34ff9640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38142280 0x7fdd38142680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c011d60 tx=0x7fdd2c01c000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd34ff9640 1 -- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38142280 msgr2=0x7fdd38142680 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:52.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd34ff9640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdd38142280 0x7fdd38142680 secure :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c011d60 tx=0x7fdd2c01c000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.136 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:52.136 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:52.136 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:52.136 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.137 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.138 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.139 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:52.140 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.141 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.142 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:52.143 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.144 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.145 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.146 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.147 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:52.148 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.150 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:52.150 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:52.150 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.150 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.150 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.150 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:52.150 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.151 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:52.153 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.156 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:52.157 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.157 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.157 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.157 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:52.157 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:52.157 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.157 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.157 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.157 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.157 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.160 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:52.160 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:52.160 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.160 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.160 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.160 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.160 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.160 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.160 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:52.160 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.161 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.161 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.161 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.161 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.161 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.161 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:52.161 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:52.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd3f15d640 1 -- 172.21.15.178:0/3740511469 >> v2:172.21.15.204:6812/28210 conn(0x7fdd240397b0 msgr2=0x7fdd2403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd3f15d640 1 --2- 172.21.15.178:0/3740511469 >> v2:172.21.15.204:6812/28210 conn(0x7fdd240397b0 0x7fdd2403bc60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdd3809b6f0 tx=0x7fdd280090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd3f15d640 10 monclient: shutdown 2024-09-15T01:47:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd3f15d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd3f15d640 1 -- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3809faa0 msgr2=0x7fdd3809b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd3f15d640 1 --2- 172.21.15.178:0/3740511469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd3809faa0 0x7fdd3809b040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdd30000f80 tx=0x7fdd30036000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd3d6fa640 1 -- 172.21.15.178:0/3740511469 reap_dead start 2024-09-15T01:47:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd3d6fa640 1 -- 172.21.15.178:0/3740511469 reap_dead start 2024-09-15T01:47:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd3f15d640 1 -- 172.21.15.178:0/3740511469 shutdown_connections 2024-09-15T01:47:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd3f15d640 1 -- 172.21.15.178:0/3740511469 >> 172.21.15.178:0/3740511469 conn(0x7fdd380a5ed0 msgr2=0x7fdd3809e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd3f15d640 1 -- 172.21.15.178:0/3740511469 shutdown_connections 2024-09-15T01:47:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.109+0000 7fdd3f15d640 1 -- 172.21.15.178:0/3740511469 wait complete. 2024-09-15T01:47:52.163 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:52.164 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:47:52.164 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:47:52.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.254+0000 7f8cb7d5d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.254+0000 7f8cb7d5d640 10 monclient: build_initial_monmap 2024-09-15T01:47:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.254+0000 7f8cb7d5d640 10 monclient: monmap: 2024-09-15T01:47:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:52.256641+0000 2024-09-15T01:47:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:52.256641+0000 2024-09-15T01:47:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:52.258 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:52.259 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:52.259 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:52.259 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:52.259 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:52.259 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:52.259 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:52.259 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:52.259 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 1 Processor -- start 2024-09-15T01:47:52.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 1 -- start start 2024-09-15T01:47:52.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 10 monclient: init 2024-09-15T01:47:52.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 10 monclient: _add_conns ranks=[0,5,10,8,9,1,6,2,3,7,4] 2024-09-15T01:47:52.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8cb00a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:47:52.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cb0132de0 0x7f8cb0137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 10 monclient(hunting): picked mon.noname-h con 0x7f8cb0132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:52.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8cb01380e0 0x7f8cb013a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 10 monclient(hunting): picked mon.noname-f con 0x7f8cb01380e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:47:52.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb004bf60 con 0x7f8cb00a2930 2024-09-15T01:47:52.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.255+0000 7f8cb7d5d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb009dd80 con 0x7f8cb01380e0 2024-09-15T01:47:52.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb7d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb7d5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00a2d00 con 0x7f8cb0132de0 2024-09-15T01:47:52.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb7d5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb7d5d640 10 monclient(hunting): authenticate will time out at 1677.007017s 2024-09-15T01:47:52.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb5af9640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f8cb00a2930 0x7f8cb009faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:36816/0 (socket says 172.21.15.178:36816) 2024-09-15T01:47:52.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb5af9640 1 -- 172.21.15.178:0/1946536684 learned_addr learned my addr 172.21.15.178:0/1946536684 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:47:52.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324933294 0 0) 0x7f8cb004bf60 con 0x7f8cb00a2930 2024-09-15T01:47:52.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient(hunting): my global_id is 64404 2024-09-15T01:47:52.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c9c0034e0 con 0x7f8cb00a2930 2024-09-15T01:47:52.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cb0132de0 0x7f8cb0137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb62fa640 1 --2- 172.21.15.178:0/1946536684 >> v2:172.21.15.204:3300/0 conn(0x7f8cb01380e0 0x7f8cb013a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 10 monclient(hunting): get_auth_request con 0x7f8cb0132de0 auth_method 0 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb62fa640 10 monclient(hunting): get_auth_request con 0x7f8cb01380e0 auth_method 0 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb62fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb62fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb62fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1321854986 0 0) 0x7f8c9c0034e0 con 0x7f8cb00a2930 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cb0132de0 msgr2=0x7f8cb0137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8cb0132de0 0x7f8cb0137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 >> v2:172.21.15.204:3300/0 conn(0x7f8cb01380e0 msgr2=0x7f8cb013a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 --2- 172.21.15.178:0/1946536684 >> v2:172.21.15.204:3300/0 conn(0x7f8cb01380e0 0x7f8cb013a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb01331b0 con 0x7f8cb00a2930 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f8cac0018f0 con 0x7f8cb00a2930 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb7d5d640 5 monclient: authenticate success, global_id 64404 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb7d5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:47:52.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient: dump: 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.274 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 >> v1:172.21.15.178:6789/0 conn(0x7f8cb00a2930 legacy=0x7f8cb009faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient: _add_conns ranks=[1,2,8,4,7,5,3,0,6] 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c9c004220 0x7f8c9c004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient(hunting): picked mon.f con 0x7f8c9c004220 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c9c004de0 0x7f8c9c005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient(hunting): picked mon.b con 0x7f8c9c004de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8c9c009c00 0x7f8c9c00c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient(hunting): picked mon.e con 0x7f8c9c009c00 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:52.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb009dd80 con 0x7f8c9c004220 2024-09-15T01:47:52.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00a2d00 con 0x7f8c9c004de0 2024-09-15T01:47:52.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c00c970 con 0x7f8c9c009c00 2024-09-15T01:47:52.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb4af7640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:47:52.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb7d5d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:52.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c9c004de0 0x7f8c9c005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 10 monclient(hunting): get_auth_request con 0x7f8c9c004de0 auth_method 0 2024-09-15T01:47:52.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:47:52.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb62fa640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8c9c009c00 0x7f8c9c00c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb52f8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:47:52.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb62fa640 10 monclient(hunting): get_auth_request con 0x7f8c9c009c00 auth_method 0 2024-09-15T01:47:52.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb62fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb62fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb62fa640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:47:52.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.256+0000 7f8cb5af9640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c9c004220 0x7f8c9c004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb5af9640 10 monclient(hunting): get_auth_request con 0x7f8c9c004220 auth_method 0 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb5af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb5af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb5af9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb52f8640 10 monclient(hunting): handle_auth_done global_id 64404 payload 293 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb52f8640 1 -- 172.21.15.178:0/1946536684 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8c9c009c00 msgr2=0x7f8c9c00c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb52f8640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8c9c009c00 0x7f8c9c00c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb52f8640 1 -- 172.21.15.178:0/1946536684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c9c004220 msgr2=0x7f8c9c004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb52f8640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c9c004220 0x7f8c9c004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb52f8640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb52f8640 1 monclient: found mon.b 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb52f8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb52f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb52f8640 1 -- 172.21.15.178:0/1946536684 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c0034e0 con 0x7f8c9c004de0 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb52f8640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb52f8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.257+0000 7f8cb62fa640 1 -- 172.21.15.178:0/1946536684 reap_dead start 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.258+0000 7f8cb52f8640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c9c004de0 0x7f8c9c005250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00a6ec0 tx=0x7f8ca0008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbeb37822f69a7c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ca002abc0 con 0x7f8c9c004de0 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb4af7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb4af7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb4af7640 10 monclient: dump: 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.279 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca002e070 con 0x7f8c9c004de0 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb7d5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb4af7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb7d5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb4af7640 1 -- 172.21.15.178:0/1946536684 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ca002ad20 con 0x7f8c9c004de0 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb4af7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb7d5d640 4 set_mon_vals no callback set 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb4af7640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb4af7640 10 monclient: dump: 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.281 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb7d5d640 10 monclient: shutdown 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb7d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1946536684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c9c004de0 msgr2=0x7f8c9c005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.260+0000 7f8cb7d5d640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c9c004de0 0x7f8c9c005250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00a6ec0 tx=0x7f8ca0008040 comp rx=0 tx=0).stop 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.261+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1946536684 shutdown_connections 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.261+0000 7f8cb7d5d640 1 --2- 172.21.15.178:0/1946536684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c9c004de0 0x7f8c9c005250 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.261+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1946536684 >> 172.21.15.178:0/1946536684 conn(0x7f8cb00a5ed0 msgr2=0x7f8cb00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.261+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1946536684 shutdown_connections 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.261+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1946536684 wait complete. 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.261+0000 7f8cb7d5d640 10 monclient: build_initial_monmap 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.261+0000 7f8cb7d5d640 10 monclient: monmap: 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:52.283 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 1 Processor -- start 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 1 -- start start 2024-09-15T01:47:52.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 10 monclient: init 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 10 monclient: _add_conns ranks=[3,2,7,5,0,4,6,8,1] 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c9c004de0 0x7f8cb0132540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 10 monclient(hunting): picked mon.noname-g con 0x7f8c9c004de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cb011c570 0x7f8cb009a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 10 monclient(hunting): picked mon.noname-b con 0x7f8cb011c570 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8cb009ab20 0x7f8cb009af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 10 monclient(hunting): picked mon.noname-i con 0x7f8cb009ab20 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb011e120 con 0x7f8cb011c570 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb009d630 con 0x7f8c9c004de0 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb009f870 con 0x7f8cb009ab20 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb7d5d640 10 monclient(hunting): authenticate will time out at 1677.013545s 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cb011c570 0x7f8cb009a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cb011c570 0x7f8cb009a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42034/0 (socket says 172.21.15.178:42034) 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 1 -- 172.21.15.178:0/1427252111 learned_addr learned my addr 172.21.15.178:0/1427252111 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 10 monclient(hunting): get_auth_request con 0x7f8cb011c570 auth_method 0 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb5af9640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c9c004de0 0x7f8cb0132540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb62fa640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8cb009ab20 0x7f8cb009af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb62fa640 10 monclient(hunting): get_auth_request con 0x7f8cb009ab20 auth_method 0 2024-09-15T01:47:52.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb62fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb62fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb62fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb5af9640 10 monclient(hunting): get_auth_request con 0x7f8c9c004de0 auth_method 0 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb5af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb5af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb5af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb62fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb62fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb62fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb5af9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb5af9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb5af9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 10 monclient(hunting): handle_auth_done global_id 35192 payload 824 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 1 -- 172.21.15.178:0/1427252111 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8cb009ab20 msgr2=0x7f8cb009af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8cb009ab20 0x7f8cb009af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 1 -- 172.21.15.178:0/1427252111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c9c004de0 msgr2=0x7f8cb0132540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c9c004de0 0x7f8cb0132540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 1 monclient: found mon.noname-b 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.262+0000 7f8cb52f8640 1 -- 172.21.15.178:0/1427252111 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb009c5f0 con 0x7f8cb011c570 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8cb52f8640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cb011c570 0x7f8cb009a5e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000b9a0 tx=0x7f8ca0036000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ca003f8f0 con 0x7f8cb011c570 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca5ffb640 10 monclient: dump: 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.289 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca003fa50 con 0x7f8cb011c570 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8cb7d5d640 5 monclient: authenticate success, global_id 35192 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ca003fbc0 con 0x7f8cb011c570 2024-09-15T01:47:52.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca5ffb640 10 monclient: dump: 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.291 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8ca7fff640 4 set_mon_vals no callback set 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8cb7d5d640 10 monclient: _renew_subs 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8cb7d5d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1427252111 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb011da20 con 0x7f8cb011c570 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8cb7d5d640 10 monclient: _renew_subs 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8cb7d5d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.263+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1427252111 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb01319c0 con 0x7f8cb011c570 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8ca003fd20 con 0x7f8cb011c570 2024-09-15T01:47:52.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb7d5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb7d5d640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0096a30 0x7f8cb0096e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb7d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1427252111 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00a0270 con 0x7f8cb0096a30 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb5af9640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0096a30 0x7f8cb0096e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8ca5ffb640 1 --2- 172.21.15.178:0/1427252111 >> v2:172.21.15.204:6812/28210 conn(0x7f8c94039710 0x7f8c9403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb5af9640 10 monclient: get_auth_request con 0x7f8cb0096a30 auth_method 0 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb5af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb5af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb5af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f8ca006c740 con 0x7f8cb011c570 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb5af9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb5af9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb5af9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb62fa640 1 --2- 172.21.15.178:0/1427252111 >> v2:172.21.15.204:6812/28210 conn(0x7f8c94039710 0x7f8c9403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb62fa640 10 monclient: get_auth_request con 0x7f8c94039710 auth_method 0 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb5af9640 10 monclient(hunting): handle_auth_done global_id 94166 payload 824 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb5af9640 1 -- 172.21.15.178:0/1427252111 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cb004c4a0 con 0x7f8cb0096a30 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.264+0000 7f8cb5af9640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0096a30 0x7f8cb0096e00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8cac001200 tx=0x7f8cac007040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.265+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8cac005030 con 0x7f8cb0096a30 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.265+0000 7f8ca5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.265+0000 7f8ca5ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.265+0000 7f8cb62fa640 1 --2- 172.21.15.178:0/1427252111 >> v2:172.21.15.204:6812/28210 conn(0x7f8c94039710 0x7f8c9403bbc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00a1e30 tx=0x7f8ca8009800 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.265+0000 7f8ca5ffb640 10 monclient: dump: 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.294 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.265+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f8cac00faf0 con 0x7f8cb0096a30 2024-09-15T01:47:52.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.265+0000 7f8ca5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:52.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.265+0000 7f8ca5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:52.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.265+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0096a30 msgr2=0x7f8cb0096e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.265+0000 7f8ca5ffb640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0096a30 0x7f8cb0096e00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8cac001200 tx=0x7f8cac007040 comp rx=0 tx=0).stop 2024-09-15T01:47:52.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.265+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0096a30 msgr2=0x7f8cb0096e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:52.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.265+0000 7f8ca5ffb640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0096a30 0x7f8cb0096e00 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8cac001200 tx=0x7f8cac007040 comp rx=0 tx=0).stop 2024-09-15T01:47:52.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.270+0000 7f8cb7d5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:52.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb7d5d640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0097340 0x7f8cb014c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb7d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1427252111 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb004bc80 con 0x7f8cb0097340 2024-09-15T01:47:52.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb5af9640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0097340 0x7f8cb014c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb5af9640 10 monclient: get_auth_request con 0x7f8cb0097340 auth_method 0 2024-09-15T01:47:52.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb5af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb5af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb5af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb5af9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb5af9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb5af9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb5af9640 10 monclient(hunting): handle_auth_done global_id 94175 payload 824 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb5af9640 1 -- 172.21.15.178:0/1427252111 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8cb005ed70 con 0x7f8cb0097340 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8cb5af9640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0097340 0x7f8cb014c5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8cac001200 tx=0x7f8cac016000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8cac026070 con 0x7f8cb0097340 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8ca5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8ca5ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8ca5ffb640 10 monclient: dump: 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.299 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f8cac00f670 con 0x7f8cb0097340 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8ca5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:52.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8ca5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:52.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0097340 msgr2=0x7f8cb014c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8ca5ffb640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0097340 0x7f8cb014c5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8cac001200 tx=0x7f8cac016000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8ca5ffb640 1 -- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0097340 msgr2=0x7f8cb014c5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:52.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.271+0000 7f8ca5ffb640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8cb0097340 0x7f8cb014c5a0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8cac001200 tx=0x7f8cac016000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.301 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:52.301 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:52.301 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:52.301 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:52.301 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:47:52.302 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:52.303 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:52.305 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:52.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:52.308 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.310 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.311 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.312 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:52.313 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.314 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.315 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.316 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.318 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.319 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.320 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:52.321 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.322 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.323 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:52.323 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.323 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.323 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.323 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.323 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.323 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.323 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.323 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:52.323 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:52.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.272+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1427252111 >> v2:172.21.15.204:6812/28210 conn(0x7f8c94039710 msgr2=0x7f8c9403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.272+0000 7f8cb7d5d640 1 --2- 172.21.15.178:0/1427252111 >> v2:172.21.15.204:6812/28210 conn(0x7f8c94039710 0x7f8c9403bbc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00a1e30 tx=0x7f8ca8009800 comp rx=0 tx=0).stop 2024-09-15T01:47:52.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.272+0000 7f8cb7d5d640 10 monclient: shutdown 2024-09-15T01:47:52.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.272+0000 7f8cb7d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.272+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cb011c570 msgr2=0x7f8cb009a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.272+0000 7f8cb7d5d640 1 --2- 172.21.15.178:0/1427252111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cb011c570 0x7f8cb009a5e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000b9a0 tx=0x7f8ca0036000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.272+0000 7f8cb62fa640 1 -- 172.21.15.178:0/1427252111 reap_dead start 2024-09-15T01:47:52.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.272+0000 7f8cb62fa640 1 -- 172.21.15.178:0/1427252111 reap_dead start 2024-09-15T01:47:52.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.272+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1427252111 shutdown_connections 2024-09-15T01:47:52.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.272+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1427252111 >> 172.21.15.178:0/1427252111 conn(0x7f8cb00a5ed0 msgr2=0x7f8cb00a1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.272+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1427252111 shutdown_connections 2024-09-15T01:47:52.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.272+0000 7f8cb7d5d640 1 -- 172.21.15.178:0/1427252111 wait complete. 2024-09-15T01:47:52.326 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:52.326 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:47:52.326 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:47:52.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.417+0000 7fad02089640 10 monclient: get_monmap_and_config 2024-09-15T01:47:52.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.417+0000 7fad02089640 10 monclient: build_initial_monmap 2024-09-15T01:47:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.417+0000 7fad02089640 10 monclient: monmap: 2024-09-15T01:47:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:52.419467+0000 2024-09-15T01:47:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:52.419467+0000 2024-09-15T01:47:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 1 Processor -- start 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 1 -- start start 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 10 monclient: init 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 10 monclient: _add_conns ranks=[8,7,9,1,6,10,2,5,3,0,4] 2024-09-15T01:47:52.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7facfc0a2930 0x7facfc132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 10 monclient(hunting): picked mon.noname-e con 0x7facfc0a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:52.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7facfc09faa0 0x7facfc09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 10 monclient(hunting): picked mon.noname-i con 0x7facfc09faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:52.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7facfc0a08c0 0x7facfc0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 10 monclient(hunting): picked mon.noname-a con 0x7facfc0a08c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:47:52.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facfc09e0f0 con 0x7facfc0a08c0 2024-09-15T01:47:52.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7facfc09dd80 con 0x7facfc09faa0 2024-09-15T01:47:52.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0a2d00 con 0x7facfc0a2930 2024-09-15T01:47:52.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7fad02089640 10 monclient(hunting): authenticate will time out at 1677.169843s 2024-09-15T01:47:52.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7facfb7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7facfc0a2930 0x7facfc132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.418+0000 7facfbfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7facfc0a08c0 0x7facfc0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7facfc0a2930 0x7facfc132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42118/0 (socket says 172.21.15.178:42118) 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfbfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7facfc0a08c0 0x7facfc0a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35834/0 (socket says 172.21.15.178:35834) 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 1 -- 172.21.15.178:0/1065128987 learned_addr learned my addr 172.21.15.178:0/1065128987 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 10 monclient(hunting): get_auth_request con 0x7facfc0a2930 auth_method 0 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfbfff640 10 monclient(hunting): get_auth_request con 0x7facfc0a08c0 auth_method 0 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfaffd640 1 --2- 172.21.15.178:0/1065128987 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7facfc09faa0 0x7facfc09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfaffd640 10 monclient(hunting): get_auth_request con 0x7facfc09faa0 auth_method 0 2024-09-15T01:47:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 10 monclient(hunting): handle_auth_done global_id 94184 payload 274 2024-09-15T01:47:52.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 1 -- 172.21.15.178:0/1065128987 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7facfc09faa0 msgr2=0x7facfc09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 1 --2- 172.21.15.178:0/1065128987 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7facfc09faa0 0x7facfc09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 1 -- 172.21.15.178:0/1065128987 >> v2:172.21.15.178:3300/0 conn(0x7facfc0a08c0 msgr2=0x7facfc0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 1 --2- 172.21.15.178:0/1065128987 >> v2:172.21.15.178:3300/0 conn(0x7facfc0a08c0 0x7facfc0a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 1 monclient: found mon.noname-e 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfb7fe640 1 -- 172.21.15.178:0/1065128987 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc04bf60 con 0x7facfc0a2930 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.419+0000 7facfbfff640 1 --2- 172.21.15.178:0/1065128987 >> v2:172.21.15.178:3300/0 conn(0x7facfc0a08c0 0x7facfc0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfb7fe640 1 --2- 172.21.15.178:0/1065128987 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7facfc0a2930 0x7facfc132de0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7facec009910 tx=0x7facec0091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=afd0ce99ac170667 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfa7fc640 1 -- 172.21.15.178:0/1065128987 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7facec02ac80 con 0x7facfc0a2930 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfa7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfa7fc640 10 monclient: dump: 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfa7fc640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfa7fc640 1 -- 172.21.15.178:0/1065128987 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec02ae00 con 0x7facfc0a2930 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 5 monclient: authenticate success, global_id 94184 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 4 set_mon_vals no callback set 2024-09-15T01:47:52.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfa7fc640 1 -- 172.21.15.178:0/1065128987 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7facec031ca0 con 0x7facfc0a2930 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfa7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7facfa7fc640 10 monclient: dump: 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 10 monclient: shutdown 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 1 -- 172.21.15.178:0/1065128987 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7facfc0a2930 msgr2=0x7facfc132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 1 --2- 172.21.15.178:0/1065128987 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7facfc0a2930 0x7facfc132de0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7facec009910 tx=0x7facec0091d0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 1 -- 172.21.15.178:0/1065128987 shutdown_connections 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 1 --2- 172.21.15.178:0/1065128987 >> v2:172.21.15.178:3300/0 conn(0x7facfc0a08c0 0x7facfc0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 1 --2- 172.21.15.178:0/1065128987 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7facfc09faa0 0x7facfc09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 1 --2- 172.21.15.178:0/1065128987 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7facfc0a2930 0x7facfc132de0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 1 -- 172.21.15.178:0/1065128987 >> 172.21.15.178:0/1065128987 conn(0x7facfc0a5ed0 msgr2=0x7facfc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 1 -- 172.21.15.178:0/1065128987 shutdown_connections 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.420+0000 7fad02089640 1 -- 172.21.15.178:0/1065128987 wait complete. 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.421+0000 7fad02089640 10 monclient: build_initial_monmap 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.421+0000 7fad02089640 10 monclient: monmap: 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.421+0000 7fad02089640 1 Processor -- start 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.421+0000 7fad02089640 1 -- start start 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.421+0000 7fad02089640 10 monclient: init 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 10 monclient: _add_conns ranks=[5,2,7,0,8,3,1,4,6] 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7facfc09faa0 0x7facfc09b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 10 monclient(hunting): picked mon.noname-h con 0x7facfc09faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facfc0a08c0 0x7facfc131990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 10 monclient(hunting): picked mon.noname-b con 0x7facfc0a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7facfc0a2930 0x7facfc141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 10 monclient(hunting): picked mon.noname-i con 0x7facfc0a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0a2390 con 0x7facfc0a08c0 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facfc09d630 con 0x7facfc09faa0 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7facfc11ee20 con 0x7facfc0a2930 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 10 monclient(hunting): authenticate will time out at 1677.173280s 2024-09-15T01:47:52.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facfc0a08c0 0x7facfc131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facfc0a08c0 0x7facfc131990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42042/0 (socket says 172.21.15.178:42042) 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 1 -- 172.21.15.178:0/240877027 learned_addr learned my addr 172.21.15.178:0/240877027 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 10 monclient(hunting): get_auth_request con 0x7facfc0a08c0 auth_method 0 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfbfff640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7facfc0a2930 0x7facfc141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 10 monclient(hunting): handle_auth_done global_id 35201 payload 824 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 1 -- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7facfc0a2930 msgr2=0x7facfc141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfb7fe640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7facfc09faa0 0x7facfc09b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7facfc0a2930 0x7facfc141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 1 -- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7facfc09faa0 msgr2=0x7facfc09b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7facfc09faa0 0x7facfc09b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 1 monclient: found mon.noname-b 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 1 -- 172.21.15.178:0/240877027 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf4009a20 con 0x7facfc0a08c0 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7facfaffd640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facfc0a08c0 0x7facfc131990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7facf000e8d0 tx=0x7facf00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7facf0009480 con 0x7facfc0a08c0 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7face37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7face37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7face37fe640 10 monclient: dump: 2024-09-15T01:47:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7face37fe640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7face37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf000cd60 con 0x7facfc0a08c0 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 5 monclient: authenticate success, global_id 35201 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 10 monclient: _renew_subs 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.422+0000 7fad02089640 1 -- 172.21.15.178:0/240877027 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec009450 con 0x7facfc0a08c0 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7face37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7facf00138f0 con 0x7facfc0a08c0 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7face37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7face37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7face37fe640 10 monclient: dump: 2024-09-15T01:47:52.449 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7fad02089640 10 monclient: _renew_subs 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7fad02089640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7fad02089640 1 -- 172.21.15.178:0/240877027 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facfc11eab0 con 0x7facfc0a08c0 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7facf9ffb640 4 set_mon_vals no callback set 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7fad02089640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7fad02089640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc11ca00 0x7facfc1423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7facf0013470 con 0x7facfc0a08c0 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7fad02089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.423+0000 7fad02089640 1 -- 172.21.15.178:0/240877027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc11f550 con 0x7facfc11ca00 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7face37fe640 1 --2- 172.21.15.178:0/240877027 >> v2:172.21.15.204:6812/28210 conn(0x7face80396e0 0x7face803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7facf004bcd0 con 0x7facfc0a08c0 2024-09-15T01:47:52.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfb7fe640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc11ca00 0x7facfc1423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfb7fe640 10 monclient: get_auth_request con 0x7facfc11ca00 auth_method 0 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfbfff640 1 --2- 172.21.15.178:0/240877027 >> v2:172.21.15.204:6812/28210 conn(0x7face80396e0 0x7face803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfbfff640 10 monclient: get_auth_request con 0x7face80396e0 auth_method 0 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfb7fe640 10 monclient(hunting): handle_auth_done global_id 94159 payload 824 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfb7fe640 1 -- 172.21.15.178:0/240877027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7facfc04c4a0 con 0x7facfc11ca00 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfb7fe640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc11ca00 0x7facfc1423c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7facfc0a3290 tx=0x7facec032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7facfbfff640 1 --2- 172.21.15.178:0/240877027 >> v2:172.21.15.204:6812/28210 conn(0x7face80396e0 0x7face803bb90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7facf4007830 tx=0x7facf400b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7facec02ac80 con 0x7facfc11ca00 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7face37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7face37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.424+0000 7face37fe640 10 monclient: dump: 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.453 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.425+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7facec03c400 con 0x7facfc11ca00 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.425+0000 7face37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.425+0000 7face37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.425+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc11ca00 msgr2=0x7facfc1423c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.425+0000 7face37fe640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc11ca00 0x7facfc1423c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7facfc0a3290 tx=0x7facec032000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.425+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc11ca00 msgr2=0x7facfc1423c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:52.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.425+0000 7face37fe640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc11ca00 0x7facfc1423c0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7facfc0a3290 tx=0x7facec032000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.430+0000 7fad02089640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:52.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.430+0000 7fad02089640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc131ed0 0x7facfc142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.430+0000 7fad02089640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.430+0000 7fad02089640 1 -- 172.21.15.178:0/240877027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc04bc80 con 0x7facfc131ed0 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.430+0000 7facfb7fe640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc131ed0 0x7facfc142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.430+0000 7facfb7fe640 10 monclient: get_auth_request con 0x7facfc131ed0 auth_method 0 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.430+0000 7facfb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.430+0000 7facfb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.430+0000 7facfb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.431+0000 7facfb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.431+0000 7facfb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.431+0000 7facfb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.431+0000 7facfb7fe640 10 monclient(hunting): handle_auth_done global_id 94168 payload 824 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.431+0000 7facfb7fe640 1 -- 172.21.15.178:0/240877027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7facfc05ed70 con 0x7facfc131ed0 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.431+0000 7facfb7fe640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc131ed0 0x7facfc142900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7facec000c00 tx=0x7facec03b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.431+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7facec04c070 con 0x7facfc131ed0 2024-09-15T01:47:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.431+0000 7face37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.431+0000 7face37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:47:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.431+0000 7face37fe640 10 monclient: dump: 2024-09-15T01:47:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7facec03cbe0 con 0x7facfc131ed0 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7face37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7face37fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc131ed0 msgr2=0x7facfc142900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7face37fe640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc131ed0 0x7facfc142900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7facec000c00 tx=0x7facec03b000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7face37fe640 1 -- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc131ed0 msgr2=0x7facfc142900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:52.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7face37fe640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc131ed0 0x7facfc142900 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7facec000c00 tx=0x7facec03b000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.460 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:52.460 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:52.460 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:52.460 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:52.460 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:52.460 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:52.460 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:52.460 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:52.460 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:52.460 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.461 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.462 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:52.463 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.464 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.465 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.465 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.465 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.465 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.465 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.465 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:52.465 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.465 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:52.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.466 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:52.466 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:52.466 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.466 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:52.466 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:52.468 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:52.469 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.471 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.472 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:52.472 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.472 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.472 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.472 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:52.472 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.472 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.472 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.472 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:52.472 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:52.473 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.473 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.473 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.473 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.473 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:52.473 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:52.473 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.473 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.473 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.473 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.473 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.474 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.475 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.475 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:52.475 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:52.475 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.475 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.475 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.475 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.475 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:52.475 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.475 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.476 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.476 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:52.476 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.476 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.476 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.476 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:52.476 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:52.476 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.476 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.476 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:52.477 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.478 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.479 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.480 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.481 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:52.482 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.483 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.483 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.483 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.483 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.483 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.483 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:52.483 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7fad02089640 1 -- 172.21.15.178:0/240877027 >> v2:172.21.15.204:6812/28210 conn(0x7face80396e0 msgr2=0x7face803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7fad02089640 1 --2- 172.21.15.178:0/240877027 >> v2:172.21.15.204:6812/28210 conn(0x7face80396e0 0x7face803bb90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7facf4007830 tx=0x7facf400b040 comp rx=0 tx=0).stop 2024-09-15T01:47:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7fad02089640 10 monclient: shutdown 2024-09-15T01:47:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7fad02089640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7fad02089640 1 -- 172.21.15.178:0/240877027 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facfc0a08c0 msgr2=0x7facfc131990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7fad02089640 1 --2- 172.21.15.178:0/240877027 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facfc0a08c0 0x7facfc131990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7facf000e8d0 tx=0x7facf00090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7facfbfff640 1 -- 172.21.15.178:0/240877027 reap_dead start 2024-09-15T01:47:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7facfbfff640 1 -- 172.21.15.178:0/240877027 reap_dead start 2024-09-15T01:47:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7fad02089640 1 -- 172.21.15.178:0/240877027 shutdown_connections 2024-09-15T01:47:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7fad02089640 1 -- 172.21.15.178:0/240877027 >> 172.21.15.178:0/240877027 conn(0x7facfc0a5ed0 msgr2=0x7facfc0a5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7fad02089640 1 -- 172.21.15.178:0/240877027 shutdown_connections 2024-09-15T01:47:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.432+0000 7fad02089640 1 -- 172.21.15.178:0/240877027 wait complete. 2024-09-15T01:47:52.486 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:52.486 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:47:52.486 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:47:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.576+0000 7f107262b640 10 monclient: get_monmap_and_config 2024-09-15T01:47:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.576+0000 7f107262b640 10 monclient: build_initial_monmap 2024-09-15T01:47:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.576+0000 7f107262b640 10 monclient: monmap: 2024-09-15T01:47:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:52.578195+0000 2024-09-15T01:47:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:52.578195+0000 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:52.580 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:52.581 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 1 Processor -- start 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 1 -- start start 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 10 monclient: init 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 10 monclient: _add_conns ranks=[9,3,7,8,1,4,0,2,10,5,6] 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f106c0a2930 0x7f106c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 10 monclient(hunting): picked mon.noname-a con 0x7f106c0a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c09faa0 0x7f106c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 10 monclient(hunting): picked mon.noname-g con 0x7f106c09faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f106c0a08c0 0x7f106c0a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 10 monclient(hunting): picked mon.noname-i con 0x7f106c0a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106c09e0f0 con 0x7f106c0a2930 2024-09-15T01:47:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c09dd80 con 0x7f106c09faa0 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0a2d00 con 0x7f106c0a08c0 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f107262b640 10 monclient(hunting): authenticate will time out at 1677.328693s 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106bfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f106c0a2930 0x7f106c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106bfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f106c0a2930 0x7f106c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35844/0 (socket says 172.21.15.178:35844) 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106bfff640 1 -- 172.21.15.178:0/1005677272 learned_addr learned my addr 172.21.15.178:0/1005677272 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106bfff640 10 monclient(hunting): get_auth_request con 0x7f106c0a2930 auth_method 0 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f1070bc8640 1 --2- 172.21.15.178:0/1005677272 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f106c0a08c0 0x7f106c0a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106b7fe640 1 --2- 172.21.15.178:0/1005677272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c09faa0 0x7f106c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f1070bc8640 10 monclient(hunting): get_auth_request con 0x7f106c0a08c0 auth_method 0 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f1070bc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f1070bc8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f1070bc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106b7fe640 10 monclient(hunting): get_auth_request con 0x7f106c09faa0 auth_method 0 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.577+0000 7f106b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f1070bc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f1070bc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f1070bc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106bfff640 10 monclient(hunting): handle_auth_done global_id 64422 payload 274 2024-09-15T01:47:52.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106bfff640 1 -- 172.21.15.178:0/1005677272 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f106c0a08c0 msgr2=0x7f106c0a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106bfff640 1 --2- 172.21.15.178:0/1005677272 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f106c0a08c0 0x7f106c0a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106bfff640 1 -- 172.21.15.178:0/1005677272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c09faa0 msgr2=0x7f106c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106bfff640 1 --2- 172.21.15.178:0/1005677272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c09faa0 0x7f106c09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106bfff640 1 monclient: found mon.noname-a 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106bfff640 1 -- 172.21.15.178:0/1005677272 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c04bf60 con 0x7f106c0a2930 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106bfff640 1 --2- 172.21.15.178:0/1005677272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f106c0a2930 0x7f106c132de0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f105c009910 tx=0x7f105c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e44dc9b1ccccf28 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106affd640 1 -- 172.21.15.178:0/1005677272 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f105c02ac80 con 0x7f106c0a2930 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106affd640 10 monclient: dump: 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106affd640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106affd640 1 -- 172.21.15.178:0/1005677272 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c02ae00 con 0x7f106c0a2930 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f107262b640 5 monclient: authenticate success, global_id 64422 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f107262b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f107262b640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106affd640 1 -- 172.21.15.178:0/1005677272 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f105c031ca0 con 0x7f106c0a2930 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f107262b640 4 set_mon_vals no callback set 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.578+0000 7f106affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f106affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f106affd640 10 monclient: dump: 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f107262b640 10 monclient: shutdown 2024-09-15T01:47:52.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f107262b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f107262b640 1 -- 172.21.15.178:0/1005677272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f106c0a2930 msgr2=0x7f106c132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f107262b640 1 --2- 172.21.15.178:0/1005677272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f106c0a2930 0x7f106c132de0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f105c009910 tx=0x7f105c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f107262b640 1 -- 172.21.15.178:0/1005677272 shutdown_connections 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f107262b640 1 --2- 172.21.15.178:0/1005677272 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f106c0a08c0 0x7f106c0a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f107262b640 1 --2- 172.21.15.178:0/1005677272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c09faa0 0x7f106c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f107262b640 1 --2- 172.21.15.178:0/1005677272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f106c0a2930 0x7f106c132de0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f107262b640 1 -- 172.21.15.178:0/1005677272 >> 172.21.15.178:0/1005677272 conn(0x7f106c0a5ed0 msgr2=0x7f106c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f107262b640 1 -- 172.21.15.178:0/1005677272 shutdown_connections 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.579+0000 7f107262b640 1 -- 172.21.15.178:0/1005677272 wait complete. 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.580+0000 7f107262b640 10 monclient: build_initial_monmap 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.580+0000 7f107262b640 10 monclient: monmap: 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:52.599 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.580+0000 7f107262b640 1 Processor -- start 2024-09-15T01:47:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.580+0000 7f107262b640 1 -- start start 2024-09-15T01:47:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.580+0000 7f107262b640 10 monclient: init 2024-09-15T01:47:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 10 monclient: _add_conns ranks=[1,2,4,3,8,7,6,5,0] 2024-09-15T01:47:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f106c09faa0 0x7f106c09afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 10 monclient(hunting): picked mon.noname-f con 0x7f106c09faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:47:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f106c0a08c0 0x7f106c131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 10 monclient(hunting): picked mon.noname-b con 0x7f106c0a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f106c0a2930 0x7f106c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 10 monclient(hunting): picked mon.noname-c con 0x7f106c0a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:47:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c11ec40 con 0x7f106c09faa0 2024-09-15T01:47:52.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c137000 con 0x7f106c0a08c0 2024-09-15T01:47:52.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f106c11e960 con 0x7f106c0a2930 2024-09-15T01:47:52.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 10 monclient(hunting): authenticate will time out at 1677.332222s 2024-09-15T01:47:52.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f106c0a2930 0x7f106c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106b7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f106c0a08c0 0x7f106c131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f106c0a2930 0x7f106c11c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:46614/0 (socket says 172.21.15.178:46614) 2024-09-15T01:47:52.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 1 -- 172.21.15.178:0/3388146903 learned_addr learned my addr 172.21.15.178:0/3388146903 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:52.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 10 monclient(hunting): get_auth_request con 0x7f106c0a2930 auth_method 0 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106b7fe640 10 monclient(hunting): get_auth_request con 0x7f106c0a08c0 auth_method 0 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106bfff640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f106c09faa0 0x7f106c09afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106bfff640 10 monclient(hunting): get_auth_request con 0x7f106c09faa0 auth_method 0 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 10 monclient(hunting): handle_auth_done global_id 44797 payload 824 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 1 -- 172.21.15.178:0/3388146903 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f106c0a08c0 msgr2=0x7f106c131840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f106c0a08c0 0x7f106c131840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 1 -- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f106c09faa0 msgr2=0x7f106c09afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f106c09faa0 0x7f106c09afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 1 monclient: found mon.noname-c 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 1 -- 172.21.15.178:0/3388146903 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c009450 con 0x7f106c0a2930 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106b7fe640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f106c0a08c0 0x7f106c131840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106bfff640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f106c09faa0 0x7f106c09afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1070bc8640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f106c0a2930 0x7f106c11c530 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1064009e90 tx=0x7f106400b040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f106400cd40 con 0x7f106c0a2930 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1053fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1053fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1053fff640 10 monclient: dump: 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1053fff640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1053fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 5 monclient: authenticate success, global_id 44797 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106400f040 con 0x7f106c0a2930 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1053fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f106400cee0 con 0x7f106c0a2930 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 10 monclient: _renew_subs 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f107262b640 1 -- 172.21.15.178:0/3388146903 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c11ef20 con 0x7f106c0a2930 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f1053fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.582+0000 7f1053fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.581+0000 7f106a7fc640 4 set_mon_vals no callback set 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.582+0000 7f1053fff640 10 monclient: dump: 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.582+0000 7f107262b640 10 monclient: _renew_subs 2024-09-15T01:47:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.582+0000 7f107262b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:47:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.582+0000 7f107262b640 1 -- 172.21.15.178:0/3388146903 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c09c080 con 0x7f106c0a2930 2024-09-15T01:47:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.582+0000 7f107262b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.582+0000 7f107262b640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c131d80 0x7f106c11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.582+0000 7f107262b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.582+0000 7f107262b640 1 -- 172.21.15.178:0/3388146903 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c04c4a0 con 0x7f106c131d80 2024-09-15T01:47:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.582+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1064014480 con 0x7f106c0a2930 2024-09-15T01:47:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106bfff640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c131d80 0x7f106c11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f1053fff640 1 --2- 172.21.15.178:0/3388146903 >> v2:172.21.15.204:6812/28210 conn(0x7f10580397b0 0x7f105803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106bfff640 10 monclient: get_auth_request con 0x7f106c131d80 auth_method 0 2024-09-15T01:47:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f106404c7b0 con 0x7f106c0a2930 2024-09-15T01:47:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106b7fe640 1 --2- 172.21.15.178:0/3388146903 >> v2:172.21.15.204:6812/28210 conn(0x7f10580397b0 0x7f105803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106b7fe640 10 monclient: get_auth_request con 0x7f10580397b0 auth_method 0 2024-09-15T01:47:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106bfff640 10 monclient(hunting): handle_auth_done global_id 64326 payload 824 2024-09-15T01:47:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106bfff640 1 -- 172.21.15.178:0/3388146903 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f106c05ed70 con 0x7f106c131d80 2024-09-15T01:47:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106bfff640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c131d80 0x7f106c11ca70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f106c0a2f70 tx=0x7f105c032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f106b7fe640 1 --2- 172.21.15.178:0/3388146903 >> v2:172.21.15.204:6812/28210 conn(0x7f10580397b0 0x7f105803bc60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f106c132670 tx=0x7f10600090f0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f105c02ac80 con 0x7f106c131d80 2024-09-15T01:47:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f1053fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f1053fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.583+0000 7f1053fff640 10 monclient: dump: 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.611 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.611 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.611 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.611 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.611 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.584+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f105c03c620 con 0x7f106c131d80 2024-09-15T01:47:52.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.584+0000 7f1053fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:52.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.584+0000 7f1053fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:52.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.584+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c131d80 msgr2=0x7f106c11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.584+0000 7f1053fff640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c131d80 0x7f106c11ca70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f106c0a2f70 tx=0x7f105c032000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.584+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c131d80 msgr2=0x7f106c11ca70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:52.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.584+0000 7f1053fff640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c131d80 0x7f106c11ca70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.589+0000 7f107262b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:52.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.589+0000 7f107262b640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c11cfb0 0x7f106c096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.589+0000 7f107262b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.589+0000 7f107262b640 1 -- 172.21.15.178:0/3388146903 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c11edb0 con 0x7f106c11cfb0 2024-09-15T01:47:52.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f106bfff640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c11cfb0 0x7f106c096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f106bfff640 10 monclient: get_auth_request con 0x7f106c11cfb0 auth_method 0 2024-09-15T01:47:52.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f106bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f106bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f106bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f106bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f106bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f106bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f106bfff640 10 monclient(hunting): handle_auth_done global_id 64335 payload 824 2024-09-15T01:47:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f106bfff640 1 -- 172.21.15.178:0/3388146903 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f106c04bc80 con 0x7f106c11cfb0 2024-09-15T01:47:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f106bfff640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c11cfb0 0x7f106c096a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f105c009910 tx=0x7f105c03c000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f105c046830 con 0x7f106c11cfb0 2024-09-15T01:47:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f1053fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f1053fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:47:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.590+0000 7f1053fff640 10 monclient: dump: 2024-09-15T01:47:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f105c0463b0 con 0x7f106c11cfb0 2024-09-15T01:47:52.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f1053fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:52.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f1053fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:52.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c11cfb0 msgr2=0x7f106c096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f1053fff640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c11cfb0 0x7f106c096a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f105c009910 tx=0x7f105c03c000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f1053fff640 1 -- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c11cfb0 msgr2=0x7f106c096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:52.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f1053fff640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f106c11cfb0 0x7f106c096a30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.617 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:52.617 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:52.617 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:52.617 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:47:52.618 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:52.619 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:52.619 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:52.619 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.619 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.619 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.619 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.619 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.619 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.619 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.619 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.619 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.619 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:52.621 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.622 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:52.625 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:52.626 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.627 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.628 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.629 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.631 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.631 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:52.631 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.631 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.631 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.631 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.631 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:52.631 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:52.631 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.631 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.631 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.631 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.632 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:52.632 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:52.632 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.632 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.632 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.632 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:52.632 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.632 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.632 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.633 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.634 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.635 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:52.636 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.637 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.637 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.637 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:52.637 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:52.637 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.637 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.637 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.637 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:52.637 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.638 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.639 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.640 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.640 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.640 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.640 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:52.640 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:52.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f107262b640 1 -- 172.21.15.178:0/3388146903 >> v2:172.21.15.204:6812/28210 conn(0x7f10580397b0 msgr2=0x7f105803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f107262b640 1 --2- 172.21.15.178:0/3388146903 >> v2:172.21.15.204:6812/28210 conn(0x7f10580397b0 0x7f105803bc60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f106c132670 tx=0x7f10600090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f107262b640 10 monclient: shutdown 2024-09-15T01:47:52.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f107262b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f107262b640 1 -- 172.21.15.178:0/3388146903 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f106c0a2930 msgr2=0x7f106c11c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f107262b640 1 --2- 172.21.15.178:0/3388146903 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f106c0a2930 0x7f106c11c530 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1064009e90 tx=0x7f106400b040 comp rx=0 tx=0).stop 2024-09-15T01:47:52.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f1070bc8640 1 -- 172.21.15.178:0/3388146903 reap_dead start 2024-09-15T01:47:52.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f1070bc8640 1 -- 172.21.15.178:0/3388146903 reap_dead start 2024-09-15T01:47:52.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f107262b640 1 -- 172.21.15.178:0/3388146903 shutdown_connections 2024-09-15T01:47:52.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f107262b640 1 -- 172.21.15.178:0/3388146903 >> 172.21.15.178:0/3388146903 conn(0x7f106c0a5ed0 msgr2=0x7f106c135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f107262b640 1 -- 172.21.15.178:0/3388146903 shutdown_connections 2024-09-15T01:47:52.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.591+0000 7f107262b640 1 -- 172.21.15.178:0/3388146903 wait complete. 2024-09-15T01:47:52.643 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:52.643 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:47:52.643 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:47:52.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.769+0000 7fb932b5d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.769+0000 7fb932b5d640 10 monclient: build_initial_monmap 2024-09-15T01:47:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.769+0000 7fb932b5d640 10 monclient: monmap: 2024-09-15T01:47:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:52.771762+0000 2024-09-15T01:47:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:52.771762+0000 2024-09-15T01:47:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:52.774 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:52.774 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:52.774 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:52.774 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:52.774 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:52.774 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:52.774 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:52.774 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.770+0000 7fb932b5d640 1 Processor -- start 2024-09-15T01:47:52.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.770+0000 7fb932b5d640 1 -- start start 2024-09-15T01:47:52.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.770+0000 7fb932b5d640 10 monclient: init 2024-09-15T01:47:52.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 10 monclient: _add_conns ranks=[8,6,2,1,5,9,3,7,10,4,0] 2024-09-15T01:47:52.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb92c0a2930 0x7fb92c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 10 monclient(hunting): picked mon.noname-e con 0x7fb92c0a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:52.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb92c09faa0 0x7fb92c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 10 monclient(hunting): picked mon.noname-d con 0x7fb92c09faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:52.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb92c0a08c0 0x7fb92c0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 10 monclient(hunting): picked mon.noname-b con 0x7fb92c0a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c09e0f0 con 0x7fb92c0a08c0 2024-09-15T01:47:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c09dd80 con 0x7fb92c09faa0 2024-09-15T01:47:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c0a2d00 con 0x7fb92c0a2930 2024-09-15T01:47:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb932b5d640 10 monclient(hunting): authenticate will time out at 1677.522203s 2024-09-15T01:47:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb92c09faa0 0x7fb92c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9308f9640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb92c0a2930 0x7fb92c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9310fa640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb92c0a08c0 0x7fb92c0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb92c09faa0 0x7fb92c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42010/0 (socket says 172.21.15.178:42010) 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 1 -- 172.21.15.178:0/2140234120 learned_addr learned my addr 172.21.15.178:0/2140234120 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 10 monclient(hunting): get_auth_request con 0x7fb92c09faa0 auth_method 0 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9310fa640 10 monclient(hunting): get_auth_request con 0x7fb92c0a08c0 auth_method 0 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9310fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9310fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9310fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9308f9640 10 monclient(hunting): get_auth_request con 0x7fb92c0a2930 auth_method 0 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9308f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9308f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9308f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9310fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9310fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9310fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9308f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9308f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9308f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 10 monclient(hunting): handle_auth_done global_id 25665 payload 274 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 1 -- 172.21.15.178:0/2140234120 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb92c0a2930 msgr2=0x7fb92c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 1 --2- 172.21.15.178:0/2140234120 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb92c0a2930 0x7fb92c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 1 -- 172.21.15.178:0/2140234120 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb92c0a08c0 msgr2=0x7fb92c0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 1 --2- 172.21.15.178:0/2140234120 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb92c0a08c0 0x7fb92c0a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 1 monclient: found mon.noname-d 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb92bfff640 1 -- 172.21.15.178:0/2140234120 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c04bf60 con 0x7fb92c09faa0 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9310fa640 1 --2- 172.21.15.178:0/2140234120 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb92c0a08c0 0x7fb92c0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.771+0000 7fb9308f9640 1 --2- 172.21.15.178:0/2140234120 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb92c0a2930 0x7fb92c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92bfff640 1 --2- 172.21.15.178:0/2140234120 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb92c09faa0 0x7fb92c09fe70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb918009940 tx=0x7fb9180091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=99cb15d9e0e54850 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92b7fe640 1 -- 172.21.15.178:0/2140234120 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb91802ac60 con 0x7fb92c09faa0 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92b7fe640 10 monclient: dump: 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.788 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb932b5d640 5 monclient: authenticate success, global_id 25665 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92b7fe640 1 -- 172.21.15.178:0/2140234120 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91802ade0 con 0x7fb92c09faa0 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb932b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb932b5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92b7fe640 1 -- 172.21.15.178:0/2140234120 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb918031ca0 con 0x7fb92c09faa0 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb932b5d640 4 set_mon_vals no callback set 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.772+0000 7fb92b7fe640 10 monclient: dump: 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.790 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 10 monclient: shutdown 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 1 -- 172.21.15.178:0/2140234120 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb92c09faa0 msgr2=0x7fb92c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 1 --2- 172.21.15.178:0/2140234120 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb92c09faa0 0x7fb92c09fe70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb918009940 tx=0x7fb9180091d0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 1 -- 172.21.15.178:0/2140234120 shutdown_connections 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 1 --2- 172.21.15.178:0/2140234120 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb92c09faa0 0x7fb92c09fe70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 1 --2- 172.21.15.178:0/2140234120 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb92c0a08c0 0x7fb92c0a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 1 --2- 172.21.15.178:0/2140234120 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb92c0a2930 0x7fb92c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 1 -- 172.21.15.178:0/2140234120 >> 172.21.15.178:0/2140234120 conn(0x7fb92c0a5ed0 msgr2=0x7fb92c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 1 -- 172.21.15.178:0/2140234120 shutdown_connections 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 1 -- 172.21.15.178:0/2140234120 wait complete. 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 10 monclient: build_initial_monmap 2024-09-15T01:47:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.773+0000 7fb932b5d640 10 monclient: monmap: 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:52.792 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 1 Processor -- start 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 1 -- start start 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 10 monclient: init 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 10 monclient: _add_conns ranks=[2,3,8,7,0,6,1,4,5] 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb92c09faa0 0x7fb92c09b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 10 monclient(hunting): picked mon.noname-b con 0x7fb92c09faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb92c0a08c0 0x7fb92c1316a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 10 monclient(hunting): picked mon.noname-g con 0x7fb92c0a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb92c0a2930 0x7fb92c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 10 monclient(hunting): picked mon.noname-e con 0x7fb92c0a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c09c2a0 con 0x7fb92c09faa0 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c09bd20 con 0x7fb92c0a08c0 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c0a2390 con 0x7fb92c0a2930 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.774+0000 7fb932b5d640 10 monclient(hunting): authenticate will time out at 1677.525959s 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb92c09faa0 0x7fb92c09b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb92c09faa0 0x7fb92c09b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42070/0 (socket says 172.21.15.178:42070) 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 1 -- 172.21.15.178:0/508800937 learned_addr learned my addr 172.21.15.178:0/508800937 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 10 monclient(hunting): get_auth_request con 0x7fb92c09faa0 auth_method 0 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9310fa640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb92c0a2930 0x7fb92c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9310fa640 10 monclient(hunting): get_auth_request con 0x7fb92c0a2930 auth_method 0 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9310fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9310fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9310fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb92bfff640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb92c0a08c0 0x7fb92c1316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9310fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9310fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9310fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb92bfff640 10 monclient(hunting): get_auth_request con 0x7fb92c0a08c0 auth_method 0 2024-09-15T01:47:52.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb92bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb92bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb92bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb92bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb92bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb92bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 10 monclient(hunting): handle_auth_done global_id 35228 payload 824 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 1 -- 172.21.15.178:0/508800937 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb92c0a2930 msgr2=0x7fb92c11c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb92c0a2930 0x7fb92c11c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 1 -- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb92c0a08c0 msgr2=0x7fb92c1316a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb92c0a08c0 0x7fb92c1316a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 1 monclient: found mon.noname-b 2024-09-15T01:47:52.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 1 -- 172.21.15.178:0/508800937 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb924009a20 con 0x7fb92c09faa0 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9310fa640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb92c0a2930 0x7fb92c11c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb9308f9640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb92c09faa0 0x7fb92c09b130 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb92c09bef0 tx=0x7fb9200090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb92000ee90 con 0x7fb92c09faa0 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb928ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb928ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb928ff9640 10 monclient: dump: 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.797 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb928ff9640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb928ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb932b5d640 5 monclient: authenticate success, global_id 35228 2024-09-15T01:47:52.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb920013470 con 0x7fb92c09faa0 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb932b5d640 10 monclient: _renew_subs 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb932b5d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb932b5d640 1 -- 172.21.15.178:0/508800937 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb920009390 con 0x7fb92c09faa0 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb928ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb920013a60 con 0x7fb92c09faa0 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb928ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb928ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb928ff9640 10 monclient: dump: 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.799 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb92affd640 4 set_mon_vals no callback set 2024-09-15T01:47:52.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb932b5d640 10 monclient: _renew_subs 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb932b5d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.775+0000 7fb932b5d640 1 -- 172.21.15.178:0/508800937 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb918009450 con 0x7fb92c09faa0 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb932b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb932b5d640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c131be0 0x7fb92c11ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb932b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb932b5d640 1 -- 172.21.15.178:0/508800937 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c131fb0 con 0x7fb92c131be0 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb920013bc0 con 0x7fb92c09faa0 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb92bfff640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c131be0 0x7fb92c11ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb928ff9640 1 --2- 172.21.15.178:0/508800937 >> v2:172.21.15.204:6812/28210 conn(0x7fb9100396e0 0x7fb91003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb92bfff640 10 monclient: get_auth_request con 0x7fb92c131be0 auth_method 0 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb92bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb92bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb92bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fb92004bf80 con 0x7fb92c09faa0 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb92bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb92bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.776+0000 7fb92bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.777+0000 7fb9310fa640 1 --2- 172.21.15.178:0/508800937 >> v2:172.21.15.204:6812/28210 conn(0x7fb9100396e0 0x7fb91003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.777+0000 7fb9310fa640 10 monclient: get_auth_request con 0x7fb9100396e0 auth_method 0 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.777+0000 7fb9310fa640 1 --2- 172.21.15.178:0/508800937 >> v2:172.21.15.204:6812/28210 conn(0x7fb9100396e0 0x7fb91003bb90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb92c09b890 tx=0x7fb92400b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.777+0000 7fb92bfff640 10 monclient(hunting): handle_auth_done global_id 7601 payload 824 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.777+0000 7fb92bfff640 1 -- 172.21.15.178:0/508800937 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb92c04c4a0 con 0x7fb92c131be0 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.777+0000 7fb92bfff640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c131be0 0x7fb92c11ca70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb918009910 tx=0x7fb918032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.778+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb91802ac60 con 0x7fb92c131be0 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.778+0000 7fb928ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.778+0000 7fb928ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.778+0000 7fb928ff9640 10 monclient: dump: 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.803 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.804 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.804 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.778+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fb91803c3e0 con 0x7fb92c131be0 2024-09-15T01:47:52.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.778+0000 7fb928ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:52.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.778+0000 7fb928ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:52.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.778+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c131be0 msgr2=0x7fb92c11ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.778+0000 7fb928ff9640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c131be0 0x7fb92c11ca70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb918009910 tx=0x7fb918032000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.778+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c131be0 msgr2=0x7fb92c11ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:52.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.778+0000 7fb928ff9640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c131be0 0x7fb92c11ca70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb932b5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:52.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb932b5d640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c11cfb0 0x7fb92c096b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb932b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb932b5d640 1 -- 172.21.15.178:0/508800937 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c04bc80 con 0x7fb92c11cfb0 2024-09-15T01:47:52.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb92bfff640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c11cfb0 0x7fb92c096b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb92bfff640 10 monclient: get_auth_request con 0x7fb92c11cfb0 auth_method 0 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb92bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb92bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb92bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb92bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb92bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb92bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb92bfff640 10 monclient(hunting): handle_auth_done global_id 7610 payload 824 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.785+0000 7fb92bfff640 1 -- 172.21.15.178:0/508800937 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb92c05ed70 con 0x7fb92c11cfb0 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb92bfff640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c11cfb0 0x7fb92c096b10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb918009940 tx=0x7fb91803d000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb91803b7f0 con 0x7fb92c11cfb0 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb928ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb928ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:47:52.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb928ff9640 10 monclient: dump: 2024-09-15T01:47:52.808 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.808 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.808 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.808 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.808 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.808 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.808 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.808 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.808 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.809 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.809 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.809 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.809 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.809 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.809 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.809 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9314 (secure 0 0 0) 0x7fb91803b370 con 0x7fb92c11cfb0 2024-09-15T01:47:52.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb928ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:52.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb928ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:52.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c11cfb0 msgr2=0x7fb92c096b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb928ff9640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c11cfb0 0x7fb92c096b10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb918009940 tx=0x7fb91803d000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb928ff9640 1 -- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c11cfb0 msgr2=0x7fb92c096b10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:52.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb928ff9640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb92c11cfb0 0x7fb92c096b10 secure :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb918009940 tx=0x7fb91803d000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.810 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:52.810 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:52.811 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.812 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.813 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.813 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.813 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:52.813 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:52.813 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.813 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.813 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.813 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.813 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.813 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.813 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.813 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.814 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.814 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.814 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:52.814 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:52.814 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:52.814 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:52.814 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:52.814 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:52.814 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:52.814 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:52.814 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.815 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.816 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.816 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.816 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.816 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.816 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:52.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.816 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:52.816 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.816 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:52.816 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:52.816 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.816 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.817 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.819 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.820 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:52.821 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:52.822 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.822 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.822 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.822 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:52.822 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.822 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.822 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:52.822 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.822 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.823 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:52.824 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:52.825 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.825 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.825 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.825 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:52.825 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:52.825 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.825 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.825 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.825 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.826 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.827 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.828 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:52.828 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.828 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.828 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.828 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:52.828 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:52.828 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.828 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.828 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:52.829 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.830 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.831 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.832 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.833 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 6 2024-09-15T01:47:52.834 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.834 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.836 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:52.836 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:52.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb932b5d640 1 -- 172.21.15.178:0/508800937 >> v2:172.21.15.204:6812/28210 conn(0x7fb9100396e0 msgr2=0x7fb91003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb932b5d640 1 --2- 172.21.15.178:0/508800937 >> v2:172.21.15.204:6812/28210 conn(0x7fb9100396e0 0x7fb91003bb90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb92c09b890 tx=0x7fb92400b040 comp rx=0 tx=0).stop 2024-09-15T01:47:52.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb932b5d640 10 monclient: shutdown 2024-09-15T01:47:52.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb932b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb932b5d640 1 -- 172.21.15.178:0/508800937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb92c09faa0 msgr2=0x7fb92c09b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb932b5d640 1 --2- 172.21.15.178:0/508800937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb92c09faa0 0x7fb92c09b130 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb92c09bef0 tx=0x7fb9200090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb9310fa640 1 -- 172.21.15.178:0/508800937 reap_dead start 2024-09-15T01:47:52.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb9310fa640 1 -- 172.21.15.178:0/508800937 reap_dead start 2024-09-15T01:47:52.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb932b5d640 1 -- 172.21.15.178:0/508800937 shutdown_connections 2024-09-15T01:47:52.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb932b5d640 1 -- 172.21.15.178:0/508800937 >> 172.21.15.178:0/508800937 conn(0x7fb92c0a5ed0 msgr2=0x7fb92c09e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb932b5d640 1 -- 172.21.15.178:0/508800937 shutdown_connections 2024-09-15T01:47:52.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.786+0000 7fb932b5d640 1 -- 172.21.15.178:0/508800937 wait complete. 2024-09-15T01:47:52.839 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:52.839 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:47:52.839 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:47:52.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.930+0000 7f828c900640 10 monclient: get_monmap_and_config 2024-09-15T01:47:52.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.930+0000 7f828c900640 10 monclient: build_initial_monmap 2024-09-15T01:47:52.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.930+0000 7f828c900640 10 monclient: monmap: 2024-09-15T01:47:52.933 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:52.933 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.933 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:52.932097+0000 2024-09-15T01:47:52.933 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:52.932097+0000 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:52.934 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:52.935 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.930+0000 7f828c900640 1 Processor -- start 2024-09-15T01:47:52.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.930+0000 7f828c900640 1 -- start start 2024-09-15T01:47:52.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.930+0000 7f828c900640 10 monclient: init 2024-09-15T01:47:52.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 10 monclient: _add_conns ranks=[3,1,5,8,10,2,7,0,6,4,9] 2024-09-15T01:47:52.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8288067130 0x7f8288067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 10 monclient(hunting): picked mon.noname-g con 0x7f8288067130 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f828809d580 addr v1:172.21.15.204:6789/0 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f828809f000 0x7f8288067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 10 monclient(hunting): picked mon.noname-h con 0x7f828809f000 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828804bf60 con 0x7f828809d580 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828809e0c0 con 0x7f8288067130 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8288067f80 con 0x7f828809f000 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828c900640 10 monclient(hunting): authenticate will time out at 1677.682530s 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f8286d9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f828809f000 0x7f8288067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828659c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8288067130 0x7f8288067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f8285d9b640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f828809d580 0x7f828812d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:36154/0 (socket says 172.21.15.178:36154) 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f8285d9b640 1 -- 172.21.15.178:0/2790938957 learned_addr learned my addr 172.21.15.178:0/2790938957 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f8286d9d640 10 monclient(hunting): get_auth_request con 0x7f828809f000 auth_method 0 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f8286d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f8286d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f8286d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828659c640 10 monclient(hunting): get_auth_request con 0x7f8288067130 auth_method 0 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f8286d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f8286d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f8286d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.931+0000 7f828659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828659c640 10 monclient(hunting): handle_auth_done global_id 64353 payload 274 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828659c640 1 -- 172.21.15.178:0/2790938957 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f828809f000 msgr2=0x7f8288067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828559a640 1 -- 172.21.15.178:0/2790938957 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890643788 0 0) 0x7f828804bf60 con 0x7f828809d580 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828659c640 1 --2- 172.21.15.178:0/2790938957 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f828809f000 0x7f8288067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828659c640 1 -- 172.21.15.178:0/2790938957 >> v1:172.21.15.204:6789/0 conn(0x7f828809d580 legacy=0x7f828812d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828659c640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828659c640 1 monclient: found mon.noname-g 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828659c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:47:52.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828659c640 1 -- 172.21.15.178:0/2790938957 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828809dde0 con 0x7f8288067130 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828559a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f8286d9d640 1 --2- 172.21.15.178:0/2790938957 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f828809f000 0x7f8288067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828659c640 1 --2- 172.21.15.178:0/2790938957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8288067130 0x7f8288067500 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8280009e90 tx=0x7f8280009780 comp rx=0 tx=0).ready entity=mon.3 client_cookie=623bcfdb80a914f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.932+0000 7f828559a640 1 -- 172.21.15.178:0/2790938957 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8280031880 con 0x7f8288067130 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828559a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828559a640 10 monclient: dump: 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.947 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.948 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.948 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.948 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.948 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.948 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.948 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.948 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.948 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.948 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828559a640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828559a640 1 -- 172.21.15.178:0/2790938957 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828002d050 con 0x7f8288067130 2024-09-15T01:47:52.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 5 monclient: authenticate success, global_id 64353 2024-09-15T01:47:52.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:52.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828559a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:52.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:52.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828559a640 1 -- 172.21.15.178:0/2790938957 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f82800303a0 con 0x7f8288067130 2024-09-15T01:47:52.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 4 set_mon_vals no callback set 2024-09-15T01:47:52.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828559a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:47:52.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828559a640 10 monclient: dump: 2024-09-15T01:47:52.949 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.949 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.949 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 10 monclient: shutdown 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 1 -- 172.21.15.178:0/2790938957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8288067130 msgr2=0x7f8288067500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 1 --2- 172.21.15.178:0/2790938957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8288067130 0x7f8288067500 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8280009e90 tx=0x7f8280009780 comp rx=0 tx=0).stop 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 1 -- 172.21.15.178:0/2790938957 shutdown_connections 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 1 --2- 172.21.15.178:0/2790938957 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f828809f000 0x7f8288067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 1 --2- 172.21.15.178:0/2790938957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8288067130 0x7f8288067500 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 1 -- 172.21.15.178:0/2790938957 >> 172.21.15.178:0/2790938957 conn(0x7f8288068a60 msgr2=0x7f82880a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 1 -- 172.21.15.178:0/2790938957 shutdown_connections 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.933+0000 7f828c900640 1 -- 172.21.15.178:0/2790938957 wait complete. 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.934+0000 7f828c900640 10 monclient: build_initial_monmap 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.934+0000 7f828c900640 10 monclient: monmap: 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:52.951 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:52.952 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.934+0000 7f828c900640 1 Processor -- start 2024-09-15T01:47:52.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.934+0000 7f828c900640 1 -- start start 2024-09-15T01:47:52.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.934+0000 7f828c900640 10 monclient: init 2024-09-15T01:47:52.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.934+0000 7f828c900640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:52.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.934+0000 7f828c900640 10 monclient: _add_conns ranks=[3,5,2,0,4,1,8,6,7] 2024-09-15T01:47:52.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.934+0000 7f828c900640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8288067130 0x7f828813fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.934+0000 7f828c900640 10 monclient(hunting): picked mon.noname-g con 0x7f8288067130 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:52.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f828809d580 0x7f8288140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 10 monclient(hunting): picked mon.noname-h con 0x7f828809d580 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:52.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f828809f000 0x7f82881191f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 10 monclient(hunting): picked mon.noname-b con 0x7f828809f000 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828809d9e0 con 0x7f828809f000 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828811afb0 con 0x7f8288067130 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f828809cb50 con 0x7f828809d580 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 10 monclient(hunting): authenticate will time out at 1677.686064s 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828659c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8288067130 0x7f828813fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f828809f000 0x7f82881191f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828659c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8288067130 0x7f828813fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:41432/0 (socket says 172.21.15.178:41432) 2024-09-15T01:47:52.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828659c640 1 -- 172.21.15.178:0/381221912 learned_addr learned my addr 172.21.15.178:0/381221912 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828659c640 10 monclient(hunting): get_auth_request con 0x7f8288067130 auth_method 0 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 10 monclient(hunting): get_auth_request con 0x7f828809f000 auth_method 0 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8285d9b640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f828809d580 0x7f8288140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8285d9b640 10 monclient(hunting): get_auth_request con 0x7f828809d580 auth_method 0 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8285d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8285d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8285d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8285d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8285d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8285d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 10 monclient(hunting): handle_auth_done global_id 35237 payload 824 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 1 -- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f828809d580 msgr2=0x7f8288140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f828809d580 0x7f8288140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 1 -- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8288067130 msgr2=0x7f828813fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8288067130 0x7f828813fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:47:52.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 1 monclient: found mon.noname-b 2024-09-15T01:47:52.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:52.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 1 -- 172.21.15.178:0/381221912 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8278009390 con 0x7f828809f000 2024-09-15T01:47:52.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828659c640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8288067130 0x7f828813fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:52.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8285d9b640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f828809d580 0x7f8288140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:52.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8286d9d640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f828809f000 0x7f82881191f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8278009fd0 tx=0x7f82780090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f827800ee90 con 0x7f828809f000 2024-09-15T01:47:52.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f827e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f827e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f827e7fc640 10 monclient: dump: 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.958 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f827e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f827e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827800cdc0 con 0x7f828809f000 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 5 monclient: authenticate success, global_id 35237 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 10 monclient: _renew_subs 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 1 -- 172.21.15.178:0/381221912 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8280009a20 con 0x7f828809f000 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f827e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f82780138f0 con 0x7f828809f000 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f827e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f827e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f827e7fc640 10 monclient: dump: 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.960 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.961 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.961 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.961 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.961 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f8284d99640 4 set_mon_vals no callback set 2024-09-15T01:47:52.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 10 monclient: _renew_subs 2024-09-15T01:47:52.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:47:52.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.935+0000 7f828c900640 1 -- 172.21.15.178:0/381221912 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828811b640 con 0x7f828809f000 2024-09-15T01:47:52.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f828c900640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:52.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f828c900640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8288119730 0x7f8288119b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f828c900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f828c900640 1 -- 172.21.15.178:0/381221912 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f82880641f0 con 0x7f8288119730 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8278013470 con 0x7f828809f000 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f828659c640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8288119730 0x7f8288119b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f828659c640 10 monclient: get_auth_request con 0x7f8288119730 auth_method 0 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f828659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f828659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f828659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f827e7fc640 1 --2- 172.21.15.178:0/381221912 >> v2:172.21.15.204:6812/28210 conn(0x7f82680397b0 0x7f826803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f828659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f828659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f828659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.936+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f827804be20 con 0x7f828809f000 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f8285d9b640 1 --2- 172.21.15.178:0/381221912 >> v2:172.21.15.204:6812/28210 conn(0x7f82680397b0 0x7f826803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f828659c640 10 monclient(hunting): handle_auth_done global_id 64312 payload 824 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f828659c640 1 -- 172.21.15.178:0/381221912 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f828804c4a0 con 0x7f8288119730 2024-09-15T01:47:52.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f8285d9b640 10 monclient: get_auth_request con 0x7f82680397b0 auth_method 0 2024-09-15T01:47:52.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f828659c640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8288119730 0x7f8288119b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82800097c0 tx=0x7f8280037000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f828003ec80 con 0x7f8288119730 2024-09-15T01:47:52.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f827e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f827e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:47:52.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f827e7fc640 10 monclient: dump: 2024-09-15T01:47:52.963 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.963 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.963 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.963 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.963 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.963 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.963 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f8285d9b640 1 --2- 172.21.15.178:0/381221912 >> v2:172.21.15.204:6812/28210 conn(0x7f82680397b0 0x7f826803bc60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f828809bdd0 tx=0x7f8270009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f828003ede0 con 0x7f8288119730 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f827e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:52.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f827e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:52.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8288119730 msgr2=0x7f8288119b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f827e7fc640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8288119730 0x7f8288119b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82800097c0 tx=0x7f8280037000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8288119730 msgr2=0x7f8288119b00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:52.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.937+0000 7f827e7fc640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8288119730 0x7f8288119b00 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82800097c0 tx=0x7f8280037000 comp rx=0 tx=0).stop 2024-09-15T01:47:52.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828c900640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:52.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828c900640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f828811a040 0x7f8288097060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:52.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828c900640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:52.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828c900640 1 -- 172.21.15.178:0/381221912 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8288064650 con 0x7f828811a040 2024-09-15T01:47:52.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828659c640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f828811a040 0x7f8288097060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:52.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828659c640 10 monclient: get_auth_request con 0x7f828811a040 auth_method 0 2024-09-15T01:47:52.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:52.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:52.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:52.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:52.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:52.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:52.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828659c640 10 monclient(hunting): handle_auth_done global_id 64321 payload 824 2024-09-15T01:47:52.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.943+0000 7f828659c640 1 -- 172.21.15.178:0/381221912 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f828804bc80 con 0x7f828811a040 2024-09-15T01:47:52.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f828659c640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f828811a040 0x7f8288097060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82881315f0 tx=0x7f8280040040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:52.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8280058070 con 0x7f828811a040 2024-09-15T01:47:52.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f827e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f827e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f827e7fc640 10 monclient: dump: 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:52.969 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:52.970 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:52.970 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:52.970 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:52.970 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:52.970 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:52.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f82800536c0 con 0x7f828811a040 2024-09-15T01:47:52.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f827e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:52.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f827e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:52.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f828811a040 msgr2=0x7f8288097060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f827e7fc640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f828811a040 0x7f8288097060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82881315f0 tx=0x7f8280040040 comp rx=0 tx=0).stop 2024-09-15T01:47:52.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f827e7fc640 1 -- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f828811a040 msgr2=0x7f8288097060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:52.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f827e7fc640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f828811a040 0x7f8288097060 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82881315f0 tx=0x7f8280040040 comp rx=0 tx=0).stop 2024-09-15T01:47:52.971 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:52.971 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:52.971 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:52.971 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:52.971 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:52.971 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:52.971 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:52.971 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:52.971 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:52.972 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:52.972 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:52.972 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:52.972 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:52.972 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:52.972 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:52.972 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.972 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:47:52.972 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:52.972 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:52.972 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:52.972 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.973 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:52.974 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:52.975 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:52.976 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:52.976 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:52.976 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:52.976 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:52.976 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:52.976 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:52.976 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:52.976 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:52.976 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:52.976 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.976 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:52.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.977 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.978 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:52.978 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.978 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.978 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.978 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:52.978 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:52.978 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.978 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.978 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.978 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:52.979 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.980 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.980 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.980 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:52.980 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:52.980 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.980 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.980 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.980 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.980 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:52.980 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.981 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.982 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.984 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.984 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.984 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.984 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:52.984 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:52.984 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.984 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.984 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.984 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.984 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:52.984 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.985 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:52.986 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.987 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.988 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.989 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.990 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.991 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.992 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:52.993 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:52.994 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:52.994 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:52.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:52.994 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:52.994 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:52.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f828c900640 1 -- 172.21.15.178:0/381221912 >> v2:172.21.15.204:6812/28210 conn(0x7f82680397b0 msgr2=0x7f826803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f828c900640 1 --2- 172.21.15.178:0/381221912 >> v2:172.21.15.204:6812/28210 conn(0x7f82680397b0 0x7f826803bc60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f828809bdd0 tx=0x7f8270009040 comp rx=0 tx=0).stop 2024-09-15T01:47:52.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f828c900640 10 monclient: shutdown 2024-09-15T01:47:52.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f828c900640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:52.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f828c900640 1 -- 172.21.15.178:0/381221912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f828809f000 msgr2=0x7f82881191f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:52.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f828c900640 1 --2- 172.21.15.178:0/381221912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f828809f000 0x7f82881191f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8278009fd0 tx=0x7f82780090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:52.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f8286d9d640 1 -- 172.21.15.178:0/381221912 reap_dead start 2024-09-15T01:47:52.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f8286d9d640 1 -- 172.21.15.178:0/381221912 reap_dead start 2024-09-15T01:47:52.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f828c900640 1 -- 172.21.15.178:0/381221912 shutdown_connections 2024-09-15T01:47:52.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f828c900640 1 -- 172.21.15.178:0/381221912 >> 172.21.15.178:0/381221912 conn(0x7f8288068a60 msgr2=0x7f8288131ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:52.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f828c900640 1 -- 172.21.15.178:0/381221912 shutdown_connections 2024-09-15T01:47:52.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:52.944+0000 7f828c900640 1 -- 172.21.15.178:0/381221912 wait complete. 2024-09-15T01:47:52.996 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:52.996 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T01:47:52.996 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T01:47:53.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.086+0000 7f4242f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:53.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.086+0000 7f4242f9d640 10 monclient: build_initial_monmap 2024-09-15T01:47:53.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.086+0000 7f4242f9d640 10 monclient: monmap: 2024-09-15T01:47:53.090 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:53.090 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:53.090 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:53.088674+0000 2024-09-15T01:47:53.090 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:53.088674+0000 2024-09-15T01:47:53.090 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:53.090 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:53.090 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:53.090 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:53.090 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:53.090 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:53.090 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:53.091 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:53.091 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:53.091 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:53.091 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:53.091 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:53.091 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:53.091 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:53.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.087+0000 7f4242f9d640 1 Processor -- start 2024-09-15T01:47:53.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.087+0000 7f4242f9d640 1 -- start start 2024-09-15T01:47:53.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.087+0000 7f4242f9d640 10 monclient: init 2024-09-15T01:47:53.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.087+0000 7f4242f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:53.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.087+0000 7f4242f9d640 10 monclient: _add_conns ranks=[4,9,5,1,0,10,7,3,6,8,2] 2024-09-15T01:47:53.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.087+0000 7f4242f9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f423c0a2930 0x7f423c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:53.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.087+0000 7f4242f9d640 10 monclient(hunting): picked mon.noname-c con 0x7f423c0a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:47:53.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.087+0000 7f4242f9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f423c09faa0 0x7f423c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:53.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4242f9d640 10 monclient(hunting): picked mon.noname-a con 0x7f423c09faa0 addr v2:172.21.15.178:3300/0 2024-09-15T01:47:53.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4242f9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f423c0a08c0 0x7f423c0a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4242f9d640 10 monclient(hunting): picked mon.noname-h con 0x7f423c0a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4242f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4242f9d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c09e0f0 con 0x7f423c09faa0 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4242f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4242f9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f423c09dd80 con 0x7f423c0a2930 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4242f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4242f9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0a2d00 con 0x7f423c0a08c0 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4242f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4242f9d640 10 monclient(hunting): authenticate will time out at 1677.839045s 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4240d39640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f423c0a2930 0x7f423c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f423c09faa0 0x7f423c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f423c09faa0 0x7f423c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35856/0 (socket says 172.21.15.178:35856) 2024-09-15T01:47:53.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4240d39640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f423c0a2930 0x7f423c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:46616/0 (socket says 172.21.15.178:46616) 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 1 -- 172.21.15.178:0/1966086869 learned_addr learned my addr 172.21.15.178:0/1966086869 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 10 monclient(hunting): get_auth_request con 0x7f423c09faa0 auth_method 0 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4240d39640 10 monclient(hunting): get_auth_request con 0x7f423c0a2930 auth_method 0 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4240d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4240d39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4240d39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:53.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f424153a640 1 --2- 172.21.15.178:0/1966086869 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f423c0a08c0 0x7f423c0a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4240d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4240d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4240d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f424153a640 10 monclient(hunting): get_auth_request con 0x7f423c0a08c0 auth_method 0 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f424153a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f424153a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f424153a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f424153a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f424153a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f424153a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 10 monclient(hunting): handle_auth_done global_id 64431 payload 274 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 1 -- 172.21.15.178:0/1966086869 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f423c0a08c0 msgr2=0x7f423c0a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 1 --2- 172.21.15.178:0/1966086869 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f423c0a08c0 0x7f423c0a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:53.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 1 -- 172.21.15.178:0/1966086869 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f423c0a2930 msgr2=0x7f423c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 1 --2- 172.21.15.178:0/1966086869 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f423c0a2930 0x7f423c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 1 monclient: found mon.noname-a 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f423bfff640 1 -- 172.21.15.178:0/1966086869 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c04bf60 con 0x7f423c09faa0 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.088+0000 7f4240d39640 1 --2- 172.21.15.178:0/1966086869 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f423c0a2930 0x7f423c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423bfff640 1 --2- 172.21.15.178:0/1966086869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f423c09faa0 0x7f423c09fe70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4230009910 tx=0x7f42300091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18b7e0513d1af57 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423b7fe640 1 -- 172.21.15.178:0/1966086869 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f423002ac90 con 0x7f423c09faa0 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423b7fe640 10 monclient: dump: 2024-09-15T01:47:53.104 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:53.105 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 5 monclient: authenticate success, global_id 64431 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423b7fe640 1 -- 172.21.15.178:0/1966086869 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423002ae10 con 0x7f423c09faa0 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423b7fe640 1 -- 172.21.15.178:0/1966086869 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4230031ca0 con 0x7f423c09faa0 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 4 set_mon_vals no callback set 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:53.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f423b7fe640 10 monclient: dump: 2024-09-15T01:47:53.107 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:53.107 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:53.107 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:53.107 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:53.107 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:53.107 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:53.107 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:53.107 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:53.107 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:53.107 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:53.107 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:53.107 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 10 monclient: shutdown 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 1 -- 172.21.15.178:0/1966086869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f423c09faa0 msgr2=0x7f423c09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 1 --2- 172.21.15.178:0/1966086869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f423c09faa0 0x7f423c09fe70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4230009910 tx=0x7f42300091d0 comp rx=0 tx=0).stop 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 1 -- 172.21.15.178:0/1966086869 shutdown_connections 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 1 --2- 172.21.15.178:0/1966086869 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f423c0a08c0 0x7f423c0a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 1 --2- 172.21.15.178:0/1966086869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f423c09faa0 0x7f423c09fe70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 1 --2- 172.21.15.178:0/1966086869 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f423c0a2930 0x7f423c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 1 -- 172.21.15.178:0/1966086869 >> 172.21.15.178:0/1966086869 conn(0x7f423c0a5ed0 msgr2=0x7f423c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:53.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 1 -- 172.21.15.178:0/1966086869 shutdown_connections 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.089+0000 7f4242f9d640 1 -- 172.21.15.178:0/1966086869 wait complete. 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.090+0000 7f4242f9d640 10 monclient: build_initial_monmap 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.090+0000 7f4242f9d640 10 monclient: monmap: 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:53.109 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.090+0000 7f4242f9d640 1 Processor -- start 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.090+0000 7f4242f9d640 1 -- start start 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.090+0000 7f4242f9d640 10 monclient: init 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 10 monclient: _add_conns ranks=[8,5,0,1,6,2,7,3,4] 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f423c09faa0 0x7f423c09b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 10 monclient(hunting): picked mon.noname-e con 0x7f423c09faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:53.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f423c0a08c0 0x7f423c1317a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 10 monclient(hunting): picked mon.noname-h con 0x7f423c0a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f423c0a2930 0x7f423c11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 10 monclient(hunting): picked mon.noname-a con 0x7f423c0a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0a2390 con 0x7f423c0a2930 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f423c09d630 con 0x7f423c0a08c0 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f423c142d00 con 0x7f423c09faa0 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4242f9d640 10 monclient(hunting): authenticate will time out at 1677.842380s 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f423c09faa0 0x7f423c09b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:53.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f423c09faa0 0x7f423c09b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42152/0 (socket says 172.21.15.178:42152) 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 1 -- 172.21.15.178:0/3822712574 learned_addr learned my addr 172.21.15.178:0/3822712574 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f424153a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f423c0a2930 0x7f423c11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 10 monclient(hunting): get_auth_request con 0x7f423c09faa0 auth_method 0 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f424153a640 10 monclient(hunting): get_auth_request con 0x7f423c0a2930 auth_method 0 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f424153a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f424153a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f424153a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:53.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f424153a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f424153a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f424153a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f423bfff640 1 --2- 172.21.15.178:0/3822712574 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f423c0a08c0 0x7f423c1317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f423bfff640 10 monclient(hunting): get_auth_request con 0x7f423c0a08c0 auth_method 0 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f423bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f423bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f423bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 10 monclient(hunting): handle_auth_done global_id 94211 payload 824 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 1 -- 172.21.15.178:0/3822712574 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f423c0a08c0 msgr2=0x7f423c1317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 1 --2- 172.21.15.178:0/3822712574 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f423c0a08c0 0x7f423c1317a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 1 -- 172.21.15.178:0/3822712574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f423c0a2930 msgr2=0x7f423c11c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 1 --2- 172.21.15.178:0/3822712574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f423c0a2930 0x7f423c11c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:53.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 10 monclient: _finish_hunting 0 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 1 monclient: found mon.noname-e 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 1 -- 172.21.15.178:0/3822712574 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4234009a20 con 0x7f423c09faa0 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f423bfff640 1 --2- 172.21.15.178:0/3822712574 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f423c0a08c0 0x7f423c1317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f424153a640 1 --2- 172.21.15.178:0/3822712574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f423c0a2930 0x7f423c11c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.091+0000 7f4240d39640 1 --2- 172.21.15.178:0/3822712574 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f423c09faa0 0x7f423c09b510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f423400c1a0 tx=0x7f423400b040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4238ff9640 1 -- 172.21.15.178:0/3822712574 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4234014900 con 0x7f423c09faa0 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4238ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4238ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4238ff9640 10 monclient: dump: 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:53.114 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:53.115 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4238ff9640 10 monclient: _finish_auth 0 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4238ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4238ff9640 1 -- 172.21.15.178:0/3822712574 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423400f040 con 0x7f423c09faa0 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4242f9d640 5 monclient: authenticate success, global_id 94211 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4242f9d640 10 monclient: _renew_subs 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4242f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4242f9d640 1 -- 172.21.15.178:0/3822712574 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4230009450 con 0x7f423c09faa0 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4238ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4238ff9640 1 -- 172.21.15.178:0/3822712574 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42340133e0 con 0x7f423c09faa0 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4238ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4238ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4238ff9640 10 monclient: dump: 2024-09-15T01:47:53.116 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:53.117 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f423affd640 4 set_mon_vals no callback set 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4242f9d640 10 monclient: _renew_subs 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4242f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.092+0000 7f4242f9d640 1 -- 172.21.15.178:0/3822712574 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c09bfe0 con 0x7f423c09faa0 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.093+0000 7f4238ff9640 1 -- 172.21.15.178:0/3822712574 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4234003020 con 0x7f423c09faa0 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.093+0000 7f4238ff9640 1 --2- 172.21.15.178:0/3822712574 >> v2:172.21.15.204:6812/28210 conn(0x7f42280397e0 0x7f422803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.093+0000 7f4242f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.093+0000 7f4242f9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.093+0000 7f4242f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.093+0000 7f4242f9d640 1 -- 172.21.15.178:0/3822712574 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c11ed40 con 0x7f423c09faa0 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.093+0000 7f4238ff9640 1 -- 172.21.15.178:0/3822712574 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f423404c670 con 0x7f423c09faa0 2024-09-15T01:47:53.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.093+0000 7f423bfff640 1 --2- 172.21.15.178:0/3822712574 >> v2:172.21.15.204:6812/28210 conn(0x7f42280397e0 0x7f422803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:53.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.093+0000 7f423bfff640 10 monclient: get_auth_request con 0x7f42280397e0 auth_method 0 2024-09-15T01:47:53.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.094+0000 7f423bfff640 1 --2- 172.21.15.178:0/3822712574 >> v2:172.21.15.204:6812/28210 conn(0x7f42280397e0 0x7f422803bc90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4230034000 tx=0x7f4230032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:53.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.097+0000 7f4238ff9640 1 -- 172.21.15.178:0/3822712574 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4234018070 con 0x7f423c09faa0 2024-09-15T01:47:53.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.097+0000 7f4238ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:53.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.097+0000 7f4238ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:53.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.319+0000 7f4242f9d640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T01:47:53.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.319+0000 7f4242f9d640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T01:47:53.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.319+0000 7f4242f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:53.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.319+0000 7f4242f9d640 1 -- 172.21.15.178:0/3822712574 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f423c04b720 con 0x7f423c09faa0 2024-09-15T01:47:53.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.321+0000 7f4238ff9640 1 -- 172.21.15.178:0/3822712574 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f4234014e00 con 0x7f423c09faa0 2024-09-15T01:47:53.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.321+0000 7f4238ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T01:47:53.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.321+0000 7f4238ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:53.327 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.327 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=22,config=2,health=12,kv=7,logm=57} crc {auth=1845068893,config=3140514009,health=3752244928,kv=2892508859,logm=1723892988}) 2024-09-15T01:47:53.327 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=12,kv=6,logm=58} crc {auth=3551352381,config=3140514009,health=3752244928,kv=484741669,logm=3756042514}) 2024-09-15T01:47:53.328 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.328 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=22,config=2,health=12,kv=7,logm=57} crc {auth=1845068893,config=3140514009,health=3752244928,kv=2892508859,logm=1723892988}) 2024-09-15T01:47:53.328 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=22,config=2,health=12,kv=7,logm=57} crc {auth=1209486221,config=3140514009,health=3752244928,kv=3834276449,logm=404768891}) 2024-09-15T01:47:53.328 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.328 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=22,config=2,health=12,kv=7,logm=57} crc {auth=1845068893,config=3140514009,health=3752244928,kv=2892508859,logm=1723892988}) 2024-09-15T01:47:53.328 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=686987223,config=3140514009,health=3649250773,kv=706962213,logm=3380348440}) 2024-09-15T01:47:53.328 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.328 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=22,config=2,health=12,kv=7,logm=57} crc {auth=1845068893,config=3140514009,health=3752244928,kv=2892508859,logm=1723892988}) 2024-09-15T01:47:53.328 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=21,config=2,health=9,kv=8,logm=60} crc {auth=1661072888,config=3140514009,health=2236374839,kv=706962213,logm=4058280070}) 2024-09-15T01:47:53.328 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.328 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=22,config=2,health=12,kv=7,logm=57} crc {auth=1845068893,config=3140514009,health=3752244928,kv=2892508859,logm=1723892988}) 2024-09-15T01:47:53.328 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=20,config=2,health=12,kv=8,logm=58} crc {auth=93244689,config=3140514009,health=3752244928,kv=706962213,logm=1582035920}) 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=22,config=2,health=12,kv=7,logm=57} crc {auth=1845068893,config=3140514009,health=3752244928,kv=2892508859,logm=1723892988}) 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=23,config=2,health=11,kv=7,logm=57} crc {auth=136928164,config=3140514009,health=2213968184,kv=898013727,logm=2903667468}) 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=22,config=2,health=12,kv=7,logm=57} crc {auth=1845068893,config=3140514009,health=3752244928,kv=2892508859,logm=1723892988}) 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=22,config=1,health=10,kv=8,logm=59} crc {auth=2228348669,config=1228700967,health=1058946799,kv=706962213,logm=2041922202}) 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=22,config=2,health=12,kv=7,logm=57} crc {auth=1845068893,config=3140514009,health=3752244928,kv=2892508859,logm=1723892988}) 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.323+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=23,config=2,health=12,kv=7,logm=56} crc {auth=136928164,config=3140514009,health=3752244928,kv=2253581948,logm=731884396}) 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.324+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.324+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=327317555}) 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.324+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1994796487}) 2024-09-15T01:47:53.329 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.324+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.324+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=327317555}) 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.324+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=979961267}) 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.324+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=327317555}) 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2029431946}) 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=327317555}) 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=132789052}) 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=327317555}) 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3559008540}) 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=327317555}) 2024-09-15T01:47:53.330 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=937640606}) 2024-09-15T01:47:53.331 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.331 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=327317555}) 2024-09-15T01:47:53.331 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2248959094}) 2024-09-15T01:47:53.331 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.331 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=327317555}) 2024-09-15T01:47:53.331 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.325+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1683224522}) 2024-09-15T01:47:53.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.323+0000 7f4242f9d640 1 -- 172.21.15.178:0/3822712574 >> v2:172.21.15.204:6812/28210 conn(0x7f42280397e0 msgr2=0x7f422803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:53.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.323+0000 7f4242f9d640 1 --2- 172.21.15.178:0/3822712574 >> v2:172.21.15.204:6812/28210 conn(0x7f42280397e0 0x7f422803bc90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4230034000 tx=0x7f4230032000 comp rx=0 tx=0).stop 2024-09-15T01:47:53.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.323+0000 7f4242f9d640 10 monclient: shutdown 2024-09-15T01:47:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.323+0000 7f4242f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.324+0000 7f4242f9d640 1 -- 172.21.15.178:0/3822712574 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f423c09faa0 msgr2=0x7f423c09b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.324+0000 7f4242f9d640 1 --2- 172.21.15.178:0/3822712574 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f423c09faa0 0x7f423c09b510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f423400c1a0 tx=0x7f423400b040 comp rx=0 tx=0).stop 2024-09-15T01:47:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.324+0000 7f4242f9d640 1 -- 172.21.15.178:0/3822712574 shutdown_connections 2024-09-15T01:47:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.324+0000 7f4242f9d640 1 --2- 172.21.15.178:0/3822712574 >> v2:172.21.15.204:6812/28210 conn(0x7f42280397e0 0x7f422803bc90 secure :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4230034000 tx=0x7f4230032000 comp rx=0 tx=0).stop 2024-09-15T01:47:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.324+0000 7f4242f9d640 1 --2- 172.21.15.178:0/3822712574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f423c0a2930 0x7f423c11c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.324+0000 7f4242f9d640 1 --2- 172.21.15.178:0/3822712574 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f423c0a08c0 0x7f423c1317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.324+0000 7f4242f9d640 1 --2- 172.21.15.178:0/3822712574 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f423c09faa0 0x7f423c09b510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.324+0000 7f4242f9d640 1 -- 172.21.15.178:0/3822712574 >> 172.21.15.178:0/3822712574 conn(0x7f423c0a5ed0 msgr2=0x7f423c135af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.324+0000 7f4242f9d640 1 -- 172.21.15.178:0/3822712574 shutdown_connections 2024-09-15T01:47:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:53.324+0000 7f4242f9d640 1 -- 172.21.15.178:0/3822712574 wait complete. 2024-09-15T01:47:53.345 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T01:47:53.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.326+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.326+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3987289395}) 2024-09-15T01:47:53.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.326+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888817679}) 2024-09-15T01:47:53.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.326+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.326+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3987289395}) 2024-09-15T01:47:53.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.326+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3299975557}) 2024-09-15T01:47:53.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.326+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.326+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3987289395}) 2024-09-15T01:47:53.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.326+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=371610072}) 2024-09-15T01:47:53.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.569 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3987289395}) 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=480369490}) 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3987289395}) 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3769690467}) 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3987289395}) 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1290915426}) 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3987289395}) 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3862171188}) 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3987289395}) 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.327+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4103861745}) 2024-09-15T01:47:53.570 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1674320067}) 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=332444574}) 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1674320067}) 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=958887862}) 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1674320067}) 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=395834100}) 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1674320067}) 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=421497767}) 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.571 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1674320067}) 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1182519676}) 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1674320067}) 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3968011584}) 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1674320067}) 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2868434681}) 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1674320067}) 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.328+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3318835663}) 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4043533764}) 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=81218010}) 2024-09-15T01:47:53.572 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4043533764}) 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=685910109}) 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4043533764}) 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1294031928}) 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4043533764}) 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1170901660}) 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4043533764}) 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=911594498}) 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.573 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4043533764}) 2024-09-15T01:47:53.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=488489213}) 2024-09-15T01:47:53.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4043533764}) 2024-09-15T01:47:53.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=9685917}) 2024-09-15T01:47:53.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4043533764}) 2024-09-15T01:47:53.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.330+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2395654151}) 2024-09-15T01:47:53.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4135719428}) 2024-09-15T01:47:53.574 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=34210721}) 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4135719428}) 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1594022183}) 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4135719428}) 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1311384373}) 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4135719428}) 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2378560451}) 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4135719428}) 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=313551882}) 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.575 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4135719428}) 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1217565572}) 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4135719428}) 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2457343962}) 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4135719428}) 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.331+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=858091775}) 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.332+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.332+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=973487578}) 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.332+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=365505934}) 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.332+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.332+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=973487578}) 2024-09-15T01:47:53.576 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.332+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3110690783}) 2024-09-15T01:47:53.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.332+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.332+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=973487578}) 2024-09-15T01:47:53.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=753780384}) 2024-09-15T01:47:53.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=973487578}) 2024-09-15T01:47:53.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=193131553}) 2024-09-15T01:47:53.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=973487578}) 2024-09-15T01:47:53.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4014946307}) 2024-09-15T01:47:53.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.577 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=973487578}) 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=846987436}) 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=973487578}) 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3838714800}) 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=973487578}) 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.333+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2787160109}) 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=130631962}) 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4066218569}) 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=130631962}) 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=649577216}) 2024-09-15T01:47:53.578 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=130631962}) 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1185322972}) 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=130631962}) 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3219523004}) 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=130631962}) 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1859145146}) 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=130631962}) 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1697116451}) 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.579 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=130631962}) 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4190316894}) 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=130631962}) 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.334+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1087404149}) 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584182201}) 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4278489023}) 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584182201}) 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4129960680}) 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584182201}) 2024-09-15T01:47:53.580 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2070575034}) 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584182201}) 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2178540606}) 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584182201}) 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=462126503}) 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584182201}) 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1096676677}) 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584182201}) 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3792449344}) 2024-09-15T01:47:53.581 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584182201}) 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.335+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3525447751}) 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2721172742}) 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4064663910}) 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2721172742}) 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2448994044}) 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2721172742}) 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=626025703}) 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2721172742}) 2024-09-15T01:47:53.582 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3397745626}) 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2721172742}) 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1856602239}) 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2721172742}) 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2302473853}) 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2721172742}) 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.336+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3773436573}) 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.337+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.337+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2721172742}) 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.337+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197466752}) 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.583 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2066187582}) 2024-09-15T01:47:53.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888815303}) 2024-09-15T01:47:53.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2066187582}) 2024-09-15T01:47:53.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1579977684}) 2024-09-15T01:47:53.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2066187582}) 2024-09-15T01:47:53.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1944876982}) 2024-09-15T01:47:53.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2066187582}) 2024-09-15T01:47:53.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3453009994}) 2024-09-15T01:47:53.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.584 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2066187582}) 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3354222377}) 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2066187582}) 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1520155899}) 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2066187582}) 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2384722736}) 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2066187582}) 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.338+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=876949187}) 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1807051252}) 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=318160814}) 2024-09-15T01:47:53.585 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1807051252}) 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1442634123}) 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1807051252}) 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3286404867}) 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1807051252}) 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1895940121}) 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1807051252}) 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3782928922}) 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1807051252}) 2024-09-15T01:47:53.586 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=194364017}) 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1807051252}) 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3531481668}) 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1807051252}) 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.339+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=338305471}) 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3980812096}) 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1394458826}) 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3980812096}) 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=808070059}) 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.587 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3980812096}) 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1378452886}) 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3980812096}) 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2274562229}) 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3980812096}) 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3371688024}) 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3980812096}) 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3450008649}) 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3980812096}) 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1186831634}) 2024-09-15T01:47:53.588 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3980812096}) 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.340+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=472530542}) 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1885154074}) 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944342571}) 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1885154074}) 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1170789917}) 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1885154074}) 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3643653622}) 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.589 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1885154074}) 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3719892263}) 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.341+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1885154074}) 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.342+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2464988009}) 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.342+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.342+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1885154074}) 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.342+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3876466519}) 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.342+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.342+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1885154074}) 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.342+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2634273799}) 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.342+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.342+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1885154074}) 2024-09-15T01:47:53.590 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.342+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3273734571}) 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1872271950}) 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2345956224}) 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1872271950}) 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1288382815}) 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1872271950}) 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=869658822}) 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1872271950}) 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1964064155}) 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.591 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1872271950}) 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1531357418}) 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1872271950}) 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4037785106}) 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1872271950}) 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=367703028}) 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1872271950}) 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.343+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4168252949}) 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1743966070}) 2024-09-15T01:47:53.592 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1865137742}) 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1743966070}) 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=592524336}) 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1743966070}) 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2652975217}) 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1743966070}) 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3481376180}) 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1743966070}) 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3199221162}) 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.593 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1743966070}) 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3502544905}) 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1743966070}) 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=622203471}) 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1743966070}) 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.344+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1261242301}) 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=463337346}) 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3906910646}) 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=463337346}) 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=921891649}) 2024-09-15T01:47:53.594 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=463337346}) 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3799284470}) 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=463337346}) 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1555354850}) 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=463337346}) 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2906358003}) 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=463337346}) 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1076503269}) 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.595 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=463337346}) 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2622267566}) 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=463337346}) 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.345+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2622326211}) 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.346+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.346+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3276191613}) 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.346+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1286221505}) 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.346+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.346+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3276191613}) 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.346+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2986898090}) 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.346+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.346+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3276191613}) 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.346+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1267997768}) 2024-09-15T01:47:53.596 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3276191613}) 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2025686082}) 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3276191613}) 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1455570817}) 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3276191613}) 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1694596205}) 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3276191613}) 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3105082162}) 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.597 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3276191613}) 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.347+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=678407843}) 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2674242625}) 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=650420194}) 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2674242625}) 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2306677763}) 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2674242625}) 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2183763489}) 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2674242625}) 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1353794493}) 2024-09-15T01:47:53.598 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2674242625}) 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3113682950}) 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2674242625}) 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1872830921}) 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2674242625}) 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4217165494}) 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2674242625}) 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.348+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3304405085}) 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.599 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2850726248}) 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3087093301}) 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2850726248}) 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=30900410}) 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2850726248}) 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3354495415}) 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2850726248}) 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2230390925}) 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2850726248}) 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3737847605}) 2024-09-15T01:47:53.600 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2850726248}) 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=491439230}) 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2850726248}) 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1418905320}) 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2850726248}) 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.349+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2244970991}) 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3039542483}) 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4200037819}) 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.601 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3039542483}) 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3731550543}) 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3039542483}) 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2112209396}) 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3039542483}) 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1550917369}) 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.350+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.351+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3039542483}) 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.351+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1752870494}) 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.351+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.351+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3039542483}) 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.351+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2644363157}) 2024-09-15T01:47:53.602 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.351+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.351+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3039542483}) 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.351+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=152862407}) 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.351+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.351+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3039542483}) 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.351+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3785459655}) 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1476514213}) 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2536255923}) 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1476514213}) 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=612991235}) 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1476514213}) 2024-09-15T01:47:53.603 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=20461974}) 2024-09-15T01:47:53.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1476514213}) 2024-09-15T01:47:53.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2487195797}) 2024-09-15T01:47:53.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1476514213}) 2024-09-15T01:47:53.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4072406287}) 2024-09-15T01:47:53.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1476514213}) 2024-09-15T01:47:53.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=588716193}) 2024-09-15T01:47:53.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1476514213}) 2024-09-15T01:47:53.604 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=604148899}) 2024-09-15T01:47:53.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1476514213}) 2024-09-15T01:47:53.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.352+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1377938635}) 2024-09-15T01:47:53.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1832256771}) 2024-09-15T01:47:53.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3392165224}) 2024-09-15T01:47:53.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1832256771}) 2024-09-15T01:47:53.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=377043464}) 2024-09-15T01:47:53.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.605 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1832256771}) 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2671980016}) 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1832256771}) 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1085865315}) 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1832256771}) 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3403683810}) 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1832256771}) 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3331661662}) 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1832256771}) 2024-09-15T01:47:53.606 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1031065517}) 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1832256771}) 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.353+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1783982115}) 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=664232577}) 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3348545291}) 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=664232577}) 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3201819789}) 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=664232577}) 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=515736923}) 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.607 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=664232577}) 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1005286069}) 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=664232577}) 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1630049754}) 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=664232577}) 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3886615555}) 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=664232577}) 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2786309066}) 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=664232577}) 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.354+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=273208756}) 2024-09-15T01:47:53.608 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.355+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.355+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=607362025}) 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.355+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1137292544}) 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.355+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.355+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=607362025}) 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.355+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2616135930}) 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.355+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.355+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=607362025}) 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.355+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1352377002}) 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.355+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.355+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=607362025}) 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.355+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1178795060}) 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.356+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.609 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.356+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=607362025}) 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.356+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2328856038}) 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.356+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.356+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=607362025}) 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.356+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=644101352}) 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.356+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.356+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=607362025}) 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.356+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3194212724}) 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.356+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.356+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=607362025}) 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.356+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2794590265}) 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4090135145}) 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1260949129}) 2024-09-15T01:47:53.610 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4090135145}) 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3088952476}) 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4090135145}) 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3320766183}) 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4090135145}) 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1784196772}) 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4090135145}) 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3579847786}) 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4090135145}) 2024-09-15T01:47:53.611 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3360088766}) 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4090135145}) 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2482255720}) 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4090135145}) 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.357+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883288747}) 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3288804386}) 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4231961880}) 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3288804386}) 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=729684645}) 2024-09-15T01:47:53.612 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3288804386}) 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1948540308}) 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3288804386}) 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=384758923}) 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3288804386}) 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3412725262}) 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3288804386}) 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1162363108}) 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3288804386}) 2024-09-15T01:47:53.613 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3478604235}) 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3288804386}) 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.358+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2948027519}) 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1525173820}) 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3722170531}) 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1525173820}) 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3162953132}) 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1525173820}) 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1172394364}) 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.614 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1525173820}) 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=155880183}) 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1525173820}) 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=339272858}) 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1525173820}) 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1525928468}) 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1525173820}) 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3305105205}) 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1525173820}) 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.359+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=881769761}) 2024-09-15T01:47:53.615 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1546791144}) 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2826681535}) 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1546791144}) 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=668173472}) 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1546791144}) 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2889979567}) 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1546791144}) 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4102557860}) 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.616 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1546791144}) 2024-09-15T01:47:53.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2602572276}) 2024-09-15T01:47:53.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1546791144}) 2024-09-15T01:47:53.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3257153085}) 2024-09-15T01:47:53.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1546791144}) 2024-09-15T01:47:53.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2260109557}) 2024-09-15T01:47:53.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1546791144}) 2024-09-15T01:47:53.617 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.361+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4194953946}) 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596384188}) 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4196808594}) 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596384188}) 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2840868594}) 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596384188}) 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2419890397}) 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596384188}) 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3364653756}) 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.618 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596384188}) 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2400027400}) 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596384188}) 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2430738916}) 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596384188}) 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=892897843}) 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596384188}) 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.362+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2354178005}) 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.363+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.363+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3969093485}) 2024-09-15T01:47:53.619 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.363+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969898530}) 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.363+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.363+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3969093485}) 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.363+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2851213258}) 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.363+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.363+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3969093485}) 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.363+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2332736423}) 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.363+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.363+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3969093485}) 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.363+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1710980389}) 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.364+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.364+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3969093485}) 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.364+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=562051826}) 2024-09-15T01:47:53.620 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.364+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.364+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3969093485}) 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.364+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2903896667}) 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.364+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.364+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3969093485}) 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.364+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3678562268}) 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.364+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.364+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3969093485}) 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.364+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1529548464}) 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3137199655}) 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1749332098}) 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3137199655}) 2024-09-15T01:47:53.621 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2909131288}) 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3137199655}) 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3885611022}) 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3137199655}) 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3752001170}) 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3137199655}) 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=632726058}) 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3137199655}) 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=497206021}) 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.622 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3137199655}) 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3872978527}) 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3137199655}) 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.365+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=707797040}) 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2998551690}) 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=756676274}) 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2998551690}) 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3235013206}) 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2998551690}) 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3313967148}) 2024-09-15T01:47:53.623 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2998551690}) 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=116333547}) 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2998551690}) 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=755244994}) 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2998551690}) 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1489243446}) 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2998551690}) 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=937319402}) 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.624 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2998551690}) 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.366+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4157406068}) 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265814382}) 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3654256217}) 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265814382}) 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1439636803}) 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265814382}) 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=785800245}) 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265814382}) 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=425113598}) 2024-09-15T01:47:53.625 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265814382}) 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=88899408}) 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265814382}) 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2504755883}) 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265814382}) 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2953821972}) 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265814382}) 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.367+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3088249156}) 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3788700073}) 2024-09-15T01:47:53.626 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2964534811}) 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3788700073}) 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=322497059}) 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3788700073}) 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=895230697}) 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3788700073}) 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3827324432}) 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3788700073}) 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2636583064}) 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.627 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3788700073}) 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1423362407}) 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3788700073}) 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1804785213}) 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3788700073}) 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.368+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1950774772}) 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.369+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.369+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1543933690}) 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.369+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4117045239}) 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.369+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.369+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1543933690}) 2024-09-15T01:47:53.628 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3387477106}) 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1543933690}) 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=939741516}) 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1543933690}) 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1779456852}) 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1543933690}) 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3453464332}) 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1543933690}) 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4120401272}) 2024-09-15T01:47:53.629 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1543933690}) 2024-09-15T01:47:53.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=13018157}) 2024-09-15T01:47:53.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1543933690}) 2024-09-15T01:47:53.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.370+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3186615084}) 2024-09-15T01:47:53.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1496459908}) 2024-09-15T01:47:53.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4116508972}) 2024-09-15T01:47:53.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1496459908}) 2024-09-15T01:47:53.630 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1031421197}) 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1496459908}) 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3982201354}) 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1496459908}) 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1835393058}) 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1496459908}) 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=781304124}) 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1496459908}) 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=204409178}) 2024-09-15T01:47:53.631 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1496459908}) 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2514869619}) 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1496459908}) 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.371+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2160322591}) 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238843653}) 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=300603029}) 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238843653}) 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2412050197}) 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.632 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238843653}) 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=926233426}) 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238843653}) 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1424066993}) 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238843653}) 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=510624444}) 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238843653}) 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=682457863}) 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238843653}) 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1085607825}) 2024-09-15T01:47:53.633 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238843653}) 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.372+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3371770072}) 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1497411604}) 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=817517721}) 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1497411604}) 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1213133449}) 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1497411604}) 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2683603045}) 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1497411604}) 2024-09-15T01:47:53.634 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=943582007}) 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1497411604}) 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=82286424}) 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1497411604}) 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=831200067}) 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1497411604}) 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=578899078}) 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1497411604}) 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.373+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=552608148}) 2024-09-15T01:47:53.635 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566225680}) 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1316000059}) 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566225680}) 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3518921579}) 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566225680}) 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4066013442}) 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566225680}) 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1895480140}) 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.636 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566225680}) 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3085532831}) 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566225680}) 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1015268530}) 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566225680}) 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=693095287}) 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566225680}) 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.374+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=503940929}) 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.375+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.375+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988449807}) 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.375+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3733541152}) 2024-09-15T01:47:53.637 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.375+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.375+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988449807}) 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.375+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=397020290}) 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988449807}) 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1411355278}) 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988449807}) 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2159192564}) 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988449807}) 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3918510350}) 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988449807}) 2024-09-15T01:47:53.638 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3446401827}) 2024-09-15T01:47:53.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988449807}) 2024-09-15T01:47:53.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2354628228}) 2024-09-15T01:47:53.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988449807}) 2024-09-15T01:47:53.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.376+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3069576383}) 2024-09-15T01:47:53.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3292784794}) 2024-09-15T01:47:53.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=837227065}) 2024-09-15T01:47:53.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3292784794}) 2024-09-15T01:47:53.639 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1495628192}) 2024-09-15T01:47:53.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3292784794}) 2024-09-15T01:47:53.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=182747919}) 2024-09-15T01:47:53.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3292784794}) 2024-09-15T01:47:53.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1452290797}) 2024-09-15T01:47:53.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3292784794}) 2024-09-15T01:47:53.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3962717610}) 2024-09-15T01:47:53.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3292784794}) 2024-09-15T01:47:53.640 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1774007804}) 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3292784794}) 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=829548307}) 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3292784794}) 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.377+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4112332240}) 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4091745334}) 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2179066986}) 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4091745334}) 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1022626116}) 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.641 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4091745334}) 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=289487949}) 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4091745334}) 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1775253341}) 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4091745334}) 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2991528518}) 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4091745334}) 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1424085416}) 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4091745334}) 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1890427565}) 2024-09-15T01:47:53.642 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4091745334}) 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.378+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1706292505}) 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1487175191}) 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2264318007}) 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1487175191}) 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649347748}) 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1487175191}) 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2096525952}) 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.643 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1487175191}) 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3710954252}) 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1487175191}) 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3876149494}) 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1487175191}) 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1782051078}) 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1487175191}) 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.379+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2230173068}) 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.380+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.380+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1487175191}) 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.380+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=793835397}) 2024-09-15T01:47:53.644 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3445616344}) 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3907310036}) 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3445616344}) 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=120790027}) 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3445616344}) 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4275678712}) 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3445616344}) 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2972310882}) 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.645 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3445616344}) 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3682540191}) 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3445616344}) 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=952644427}) 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3445616344}) 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2193200740}) 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3445616344}) 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.381+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3852133776}) 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4017592369}) 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1523907115}) 2024-09-15T01:47:53.646 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4017592369}) 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4166019657}) 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4017592369}) 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=738534159}) 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4017592369}) 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2178136231}) 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4017592369}) 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1975399245}) 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4017592369}) 2024-09-15T01:47:53.647 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3290839381}) 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4017592369}) 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2125119061}) 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4017592369}) 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.382+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=751578047}) 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1987393439}) 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1104882901}) 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1987393439}) 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1642823748}) 2024-09-15T01:47:53.648 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1987393439}) 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2668955848}) 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1987393439}) 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1760858457}) 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1987393439}) 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4118604377}) 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.383+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.384+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1987393439}) 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.384+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2609266336}) 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.384+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.384+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1987393439}) 2024-09-15T01:47:53.649 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.384+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1380840076}) 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.384+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.384+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1987393439}) 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.384+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=725816098}) 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1998099037}) 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=932434039}) 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1998099037}) 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3111184051}) 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1998099037}) 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=578508759}) 2024-09-15T01:47:53.650 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1998099037}) 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1212681128}) 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1998099037}) 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3293051246}) 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1998099037}) 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1858950644}) 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1998099037}) 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=185396912}) 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1998099037}) 2024-09-15T01:47:53.651 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.385+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1763560728}) 2024-09-15T01:47:53.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=134821724}) 2024-09-15T01:47:53.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3440829942}) 2024-09-15T01:47:53.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=134821724}) 2024-09-15T01:47:53.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2576015352}) 2024-09-15T01:47:53.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=134821724}) 2024-09-15T01:47:53.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1432612299}) 2024-09-15T01:47:53.652 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=134821724}) 2024-09-15T01:47:53.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3194557411}) 2024-09-15T01:47:53.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=134821724}) 2024-09-15T01:47:53.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2648824977}) 2024-09-15T01:47:53.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=134821724}) 2024-09-15T01:47:53.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3551416728}) 2024-09-15T01:47:53.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=134821724}) 2024-09-15T01:47:53.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3713914320}) 2024-09-15T01:47:53.653 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=134821724}) 2024-09-15T01:47:53.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.386+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2042990955}) 2024-09-15T01:47:53.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2945718956}) 2024-09-15T01:47:53.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3308623932}) 2024-09-15T01:47:53.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2945718956}) 2024-09-15T01:47:53.656 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3238913477}) 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2945718956}) 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2235862014}) 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2945718956}) 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1174027086}) 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2945718956}) 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1140518030}) 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.387+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.388+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2945718956}) 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.388+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3589875757}) 2024-09-15T01:47:53.657 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.388+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.388+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2945718956}) 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.388+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3292173716}) 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.388+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.388+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2945718956}) 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.388+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1972400735}) 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330703036}) 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2589903778}) 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330703036}) 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=838270120}) 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330703036}) 2024-09-15T01:47:53.658 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4047832610}) 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330703036}) 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2695900730}) 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330703036}) 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3445640357}) 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330703036}) 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=874204866}) 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330703036}) 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4244541243}) 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.659 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3330703036}) 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.389+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2066605342}) 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1445065603}) 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1000461265}) 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1445065603}) 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=414859424}) 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1445065603}) 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1233206721}) 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1445065603}) 2024-09-15T01:47:53.660 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=277029409}) 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1445065603}) 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1958018527}) 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1445065603}) 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=329273906}) 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1445065603}) 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3896415131}) 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1445065603}) 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.390+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1546072291}) 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.661 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3430020760}) 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1551968980}) 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3430020760}) 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2443815239}) 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3430020760}) 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=153552795}) 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3430020760}) 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2188904250}) 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3430020760}) 2024-09-15T01:47:53.662 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=987205916}) 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3430020760}) 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2487329732}) 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3430020760}) 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=800030401}) 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3430020760}) 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.391+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4204898566}) 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2092640172}) 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3085954910}) 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.663 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2092640172}) 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4013552417}) 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2092640172}) 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1138381052}) 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2092640172}) 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4203807506}) 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2092640172}) 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2034481741}) 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2092640172}) 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1071526737}) 2024-09-15T01:47:53.664 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2092640172}) 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1828117000}) 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.392+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.393+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2092640172}) 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.393+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2013337599}) 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.393+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.393+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2952567659}) 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.393+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3880790803}) 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2952567659}) 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=631026663}) 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2952567659}) 2024-09-15T01:47:53.665 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3359353569}) 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2952567659}) 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1843165402}) 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2952567659}) 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3738326375}) 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2952567659}) 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=120506326}) 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2952567659}) 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=343365678}) 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.666 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2952567659}) 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.394+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3654173516}) 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2926933720}) 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1999040670}) 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2926933720}) 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4259062005}) 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2926933720}) 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=680694746}) 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2926933720}) 2024-09-15T01:47:53.667 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2299759087}) 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2926933720}) 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1874740486}) 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2926933720}) 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=834646459}) 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2926933720}) 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3782714489}) 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2926933720}) 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.395+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3668676552}) 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.668 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1110172842}) 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4237755070}) 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1110172842}) 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4191175876}) 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1110172842}) 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2946314117}) 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1110172842}) 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2777983333}) 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1110172842}) 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2739134489}) 2024-09-15T01:47:53.669 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1110172842}) 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=443286980}) 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1110172842}) 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3775302589}) 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1110172842}) 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.396+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2717843429}) 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=562664197}) 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2151783268}) 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=562664197}) 2024-09-15T01:47:53.670 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4171140532}) 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=562664197}) 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=507343401}) 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=562664197}) 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3880540060}) 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=562664197}) 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3144015129}) 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=562664197}) 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4126663536}) 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.671 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=562664197}) 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4265911500}) 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=562664197}) 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.397+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2307078001}) 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=446225376}) 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1802744252}) 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=446225376}) 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=272490432}) 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=446225376}) 2024-09-15T01:47:53.672 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3652211373}) 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=446225376}) 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1267194613}) 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=446225376}) 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3459916261}) 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=446225376}) 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=669045877}) 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=446225376}) 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2529950109}) 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.673 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=446225376}) 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.398+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3162241567}) 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2564524603}) 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3360613537}) 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2564524603}) 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3484792813}) 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2564524603}) 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1243614784}) 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2564524603}) 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2856561466}) 2024-09-15T01:47:53.674 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2564524603}) 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.399+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1769071642}) 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.400+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.400+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2564524603}) 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.400+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1988994082}) 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.400+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.400+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2564524603}) 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.400+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2718352665}) 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.400+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.400+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2564524603}) 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.400+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=855026465}) 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=219721566}) 2024-09-15T01:47:53.675 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1316526653}) 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=219721566}) 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3939565659}) 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=219721566}) 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2710566360}) 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=219721566}) 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1895022903}) 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=219721566}) 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2405615235}) 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.676 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=219721566}) 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1582408486}) 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=219721566}) 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2338494337}) 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=219721566}) 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.401+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2224471613}) 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1929322555}) 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=300480218}) 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1929322555}) 2024-09-15T01:47:53.677 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2153135619}) 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1929322555}) 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=985994643}) 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1929322555}) 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2169337245}) 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1929322555}) 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=210383974}) 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1929322555}) 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1608253100}) 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.678 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1929322555}) 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=153458756}) 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1929322555}) 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.402+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4173341673}) 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3244063084}) 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2068101278}) 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3244063084}) 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3600380092}) 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3244063084}) 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2105082788}) 2024-09-15T01:47:53.679 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3244063084}) 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2785904991}) 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3244063084}) 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1824661760}) 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3244063084}) 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1584241630}) 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3244063084}) 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1710216224}) 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.680 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3244063084}) 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.403+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1494831116}) 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2303818200}) 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549561385}) 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2303818200}) 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3791431909}) 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2303818200}) 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1809888328}) 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2303818200}) 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1592481645}) 2024-09-15T01:47:53.681 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2303818200}) 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.404+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2334500970}) 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.405+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.405+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2303818200}) 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.405+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3859426077}) 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.405+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.405+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2303818200}) 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.405+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3068559013}) 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.405+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.405+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2303818200}) 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.405+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3381486472}) 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898350522}) 2024-09-15T01:47:53.682 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2159440100}) 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898350522}) 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4125214113}) 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898350522}) 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3053321890}) 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898350522}) 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3719917767}) 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898350522}) 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2529261781}) 2024-09-15T01:47:53.683 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898350522}) 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=756583953}) 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898350522}) 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3116327540}) 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2898350522}) 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.406+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4194814570}) 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=415529094}) 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2306825665}) 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=415529094}) 2024-09-15T01:47:53.684 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=406586378}) 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=415529094}) 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1256688326}) 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=415529094}) 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1837262411}) 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=415529094}) 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3547359967}) 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=415529094}) 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=551310254}) 2024-09-15T01:47:53.685 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=415529094}) 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1559016955}) 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=415529094}) 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.407+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1977561706}) 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4252712803}) 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3946098780}) 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4252712803}) 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3940815150}) 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4252712803}) 2024-09-15T01:47:53.686 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3913025404}) 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4252712803}) 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2345405718}) 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4252712803}) 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4122721277}) 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.408+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.409+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4252712803}) 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.409+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4042197399}) 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.409+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.409+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4252712803}) 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.409+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1074511531}) 2024-09-15T01:47:53.687 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.409+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.409+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4252712803}) 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.409+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3097858934}) 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40425411}) 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3140277605}) 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40425411}) 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3212355049}) 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40425411}) 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=559264836}) 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40425411}) 2024-09-15T01:47:53.688 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1864449695}) 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40425411}) 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3194859930}) 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40425411}) 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=171635206}) 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40425411}) 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2210670268}) 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40425411}) 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.410+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2253487479}) 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.689 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4161979026}) 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1843861763}) 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4161979026}) 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1657458818}) 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4161979026}) 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1902563162}) 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4161979026}) 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=477399578}) 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4161979026}) 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3488874160}) 2024-09-15T01:47:53.690 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4161979026}) 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1796185783}) 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4161979026}) 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2087071161}) 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4161979026}) 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.411+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=140391589}) 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2783898481}) 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3851613685}) 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2783898481}) 2024-09-15T01:47:53.691 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2468433386}) 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2783898481}) 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1978096258}) 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2783898481}) 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3162815256}) 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2783898481}) 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=697278618}) 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2783898481}) 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3228433486}) 2024-09-15T01:47:53.692 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2783898481}) 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1316403299}) 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2783898481}) 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.412+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3249184274}) 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.413+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.413+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622433605}) 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.413+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1936501794}) 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.413+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.413+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622433605}) 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.413+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=600816747}) 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.413+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.413+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622433605}) 2024-09-15T01:47:53.693 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.413+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3481478536}) 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.413+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.413+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622433605}) 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.413+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3480828930}) 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.414+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.414+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622433605}) 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.414+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2027740948}) 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.414+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.414+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622433605}) 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.414+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4247231668}) 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.414+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.414+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622433605}) 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.414+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2494767438}) 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.414+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.694 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.414+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2622433605}) 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.414+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=787880149}) 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=279033559}) 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738069360}) 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=279033559}) 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3169255686}) 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=279033559}) 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=749222078}) 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=279033559}) 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=694407334}) 2024-09-15T01:47:53.695 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=279033559}) 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1206129847}) 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=279033559}) 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1423626465}) 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=279033559}) 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3857711950}) 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=279033559}) 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.415+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=236710900}) 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2125374221}) 2024-09-15T01:47:53.696 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1822234037}) 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2125374221}) 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=816507521}) 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2125374221}) 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3887721585}) 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2125374221}) 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=254664717}) 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2125374221}) 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1496517422}) 2024-09-15T01:47:53.697 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2125374221}) 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=399366948}) 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2125374221}) 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3161261228}) 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2125374221}) 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.416+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4293580200}) 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898644156}) 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4162904190}) 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898644156}) 2024-09-15T01:47:53.698 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2574840515}) 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898644156}) 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4136160949}) 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898644156}) 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4142852734}) 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898644156}) 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1570093028}) 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898644156}) 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1017276675}) 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.699 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898644156}) 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2601397476}) 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3898644156}) 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.417+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=512977414}) 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.418+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.418+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3921062117}) 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.418+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1795585611}) 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.418+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.418+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3921062117}) 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.418+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3877385998}) 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.418+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3921062117}) 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=298908294}) 2024-09-15T01:47:53.700 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3921062117}) 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2236110849}) 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3921062117}) 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1464380775}) 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3921062117}) 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4267949223}) 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3921062117}) 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3698717656}) 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3921062117}) 2024-09-15T01:47:53.701 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.419+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=280121501}) 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2790661481}) 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2057198548}) 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2790661481}) 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3525491918}) 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2790661481}) 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2682972294}) 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2790661481}) 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3758041549}) 2024-09-15T01:47:53.702 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2790661481}) 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3356989240}) 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2790661481}) 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4005095223}) 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2790661481}) 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2809631131}) 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2790661481}) 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.420+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3836041370}) 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1302306196}) 2024-09-15T01:47:53.703 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3785725414}) 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1302306196}) 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3030380826}) 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1302306196}) 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=723215331}) 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1302306196}) 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1935113241}) 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1302306196}) 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2252949168}) 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.704 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1302306196}) 2024-09-15T01:47:53.705 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3181029107}) 2024-09-15T01:47:53.705 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.705 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1302306196}) 2024-09-15T01:47:53.705 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4081548574}) 2024-09-15T01:47:53.705 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.705 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1302306196}) 2024-09-15T01:47:53.705 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.421+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2504881644}) 2024-09-15T01:47:53.705 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.705 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2667277703}) 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=209324176}) 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2667277703}) 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2272575904}) 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2667277703}) 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1339294572}) 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2667277703}) 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2593785761}) 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.422+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2667277703}) 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.423+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=64734110}) 2024-09-15T01:47:53.706 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.423+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.707 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.423+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2667277703}) 2024-09-15T01:47:53.707 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.423+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2281784882}) 2024-09-15T01:47:53.707 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.423+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.707 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.423+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2667277703}) 2024-09-15T01:47:53.707 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.423+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3721768908}) 2024-09-15T01:47:53.707 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.423+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.707 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.423+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2667277703}) 2024-09-15T01:47:53.707 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.423+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2041753156}) 2024-09-15T01:47:53.707 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.707 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3191897738}) 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1254831246}) 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3191897738}) 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4260004071}) 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3191897738}) 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1973007645}) 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3191897738}) 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2647182381}) 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3191897738}) 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=792774754}) 2024-09-15T01:47:53.708 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3191897738}) 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2017619227}) 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3191897738}) 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1719316447}) 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3191897738}) 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.424+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3499478347}) 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2304010151}) 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1953870564}) 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2304010151}) 2024-09-15T01:47:53.709 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=346394242}) 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2304010151}) 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2801152622}) 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2304010151}) 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=779688970}) 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2304010151}) 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=507837646}) 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2304010151}) 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1074453843}) 2024-09-15T01:47:53.710 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2304010151}) 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3016369003}) 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2304010151}) 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.425+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1629943029}) 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.426+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.426+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3044148764}) 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189733682}) 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3044148764}) 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3311745520}) 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3044148764}) 2024-09-15T01:47:53.711 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=108084931}) 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3044148764}) 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1850538303}) 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3044148764}) 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3821971058}) 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3044148764}) 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1707352177}) 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3044148764}) 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3083472281}) 2024-09-15T01:47:53.712 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3044148764}) 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.427+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2768684254}) 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1708398551}) 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3734902473}) 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1708398551}) 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1718758903}) 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1708398551}) 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1282246343}) 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.713 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1708398551}) 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3370905155}) 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1708398551}) 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=872330362}) 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1708398551}) 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1085995561}) 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1708398551}) 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2836890574}) 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1708398551}) 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.428+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1384889174}) 2024-09-15T01:47:53.714 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3183671115}) 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=78885409}) 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3183671115}) 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=41848814}) 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3183671115}) 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=105604638}) 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3183671115}) 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1826163730}) 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3183671115}) 2024-09-15T01:47:53.715 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2266134385}) 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3183671115}) 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=319463506}) 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3183671115}) 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1057288615}) 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3183671115}) 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.429+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2801168420}) 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.430+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.430+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2409791462}) 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.430+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1074695838}) 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.430+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.716 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.430+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2409791462}) 2024-09-15T01:47:53.717 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.430+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3869662464}) 2024-09-15T01:47:53.717 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.430+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.717 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.430+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2409791462}) 2024-09-15T01:47:53.717 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.430+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1344796492}) 2024-09-15T01:47:53.717 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.430+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.717 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.430+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2409791462}) 2024-09-15T01:47:53.717 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.430+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3622146973}) 2024-09-15T01:47:53.717 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.431+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.717 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.431+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2409791462}) 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.431+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=996980427}) 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.431+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.431+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2409791462}) 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.431+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=655714368}) 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.431+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.431+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2409791462}) 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.431+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2989769874}) 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.431+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.431+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2409791462}) 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.431+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1771820611}) 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3286217210}) 2024-09-15T01:47:53.718 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1540921303}) 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3286217210}) 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=249706637}) 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3286217210}) 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3710606309}) 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3286217210}) 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4276485908}) 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3286217210}) 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3319799092}) 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.719 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3286217210}) 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1866222677}) 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3286217210}) 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1435311603}) 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3286217210}) 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.432+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4190691238}) 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.433+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.433+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2894238269}) 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.433+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3546531298}) 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.433+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.433+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2894238269}) 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.433+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4162959220}) 2024-09-15T01:47:53.720 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.433+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.433+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2894238269}) 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.433+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3228142803}) 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.433+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.433+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2894238269}) 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.433+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=812705001}) 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.434+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.434+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2894238269}) 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.434+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3649692107}) 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.434+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.434+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2894238269}) 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.434+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1785428952}) 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.434+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.721 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.434+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2894238269}) 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.434+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1858193316}) 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.434+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.434+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2894238269}) 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.434+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2361193056}) 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1198011318}) 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3458782811}) 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1198011318}) 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2552648437}) 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1198011318}) 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2392815944}) 2024-09-15T01:47:53.722 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1198011318}) 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1127409093}) 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1198011318}) 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1529499059}) 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1198011318}) 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=293028249}) 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1198011318}) 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3994555181}) 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1198011318}) 2024-09-15T01:47:53.723 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.435+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1386266548}) 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2451671413}) 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1877212471}) 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2451671413}) 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=490732680}) 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2451671413}) 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=443016985}) 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2451671413}) 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3328751718}) 2024-09-15T01:47:53.724 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2451671413}) 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3679094107}) 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.436+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.437+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2451671413}) 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.437+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=999145767}) 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.437+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.437+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2451671413}) 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.437+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4091196791}) 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.437+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.437+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2451671413}) 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.437+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2526344617}) 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=544746893}) 2024-09-15T01:47:53.725 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=148474129}) 2024-09-15T01:47:53.726 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.726 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=544746893}) 2024-09-15T01:47:53.726 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=378801948}) 2024-09-15T01:47:53.726 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.726 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=544746893}) 2024-09-15T01:47:53.726 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3826996253}) 2024-09-15T01:47:53.726 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.726 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=544746893}) 2024-09-15T01:47:53.726 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2982150628}) 2024-09-15T01:47:53.726 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.726 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=544746893}) 2024-09-15T01:47:53.726 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3335606879}) 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=544746893}) 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2466203792}) 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=544746893}) 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2503197502}) 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=544746893}) 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.438+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1323487889}) 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3048028343}) 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235816386}) 2024-09-15T01:47:53.727 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3048028343}) 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=711337341}) 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3048028343}) 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=88924569}) 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3048028343}) 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3558570533}) 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3048028343}) 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=794556198}) 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.439+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.728 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.440+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3048028343}) 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.440+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=944616292}) 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.440+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.440+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3048028343}) 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.440+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1027603308}) 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.440+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.440+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3048028343}) 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.440+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1143487197}) 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3272292143}) 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=108024918}) 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3272292143}) 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=576255747}) 2024-09-15T01:47:53.729 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3272292143}) 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1439905557}) 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3272292143}) 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1806918114}) 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3272292143}) 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=650423304}) 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3272292143}) 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2337538337}) 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.730 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3272292143}) 2024-09-15T01:47:53.731 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2545780010}) 2024-09-15T01:47:53.731 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.731 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3272292143}) 2024-09-15T01:47:53.731 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.441+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1087284505}) 2024-09-15T01:47:53.731 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.731 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017229623}) 2024-09-15T01:47:53.731 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2861121889}) 2024-09-15T01:47:53.731 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.731 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017229623}) 2024-09-15T01:47:53.731 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3035696388}) 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017229623}) 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=68475115}) 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017229623}) 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2571420490}) 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.442+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017229623}) 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.443+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2162633135}) 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.443+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.443+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017229623}) 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.443+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2274728263}) 2024-09-15T01:47:53.732 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.443+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.443+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017229623}) 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.443+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3789536269}) 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.443+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.443+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017229623}) 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.443+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3608545397}) 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2801086012}) 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1167678938}) 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2801086012}) 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3452095555}) 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2801086012}) 2024-09-15T01:47:53.733 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=347023103}) 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2801086012}) 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2064537956}) 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2801086012}) 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1672583777}) 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2801086012}) 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=239414836}) 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2801086012}) 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1185569672}) 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.734 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2801086012}) 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.444+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=449707514}) 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865111105}) 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2611493829}) 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865111105}) 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2249465907}) 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865111105}) 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2289223070}) 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865111105}) 2024-09-15T01:47:53.735 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2338297572}) 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.445+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865111105}) 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.446+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2156335060}) 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.446+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.446+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865111105}) 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.446+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2134349969}) 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.446+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.446+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865111105}) 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.446+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=979173705}) 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.446+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.446+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865111105}) 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.446+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1013554326}) 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.736 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2946653534}) 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1632237260}) 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2946653534}) 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3066914763}) 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2946653534}) 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1090480442}) 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2946653534}) 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3468187849}) 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2946653534}) 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3519537323}) 2024-09-15T01:47:53.737 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2946653534}) 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2857474990}) 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2946653534}) 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4018866989}) 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2946653534}) 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.447+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2922881245}) 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3680149094}) 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=732655074}) 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.738 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3680149094}) 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3059602877}) 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3680149094}) 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2642455972}) 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3680149094}) 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2982816632}) 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.448+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3680149094}) 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.449+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1934223288}) 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.449+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.449+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3680149094}) 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.449+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2199084985}) 2024-09-15T01:47:53.739 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.449+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.449+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3680149094}) 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.449+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1272028803}) 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.449+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.449+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3680149094}) 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.449+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2739629445}) 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423148876}) 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1849853283}) 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423148876}) 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3301677216}) 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423148876}) 2024-09-15T01:47:53.740 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2512668746}) 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423148876}) 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3133586522}) 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423148876}) 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1099115358}) 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423148876}) 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1734838226}) 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423148876}) 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2104918888}) 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.741 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423148876}) 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.450+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=346229261}) 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3813882454}) 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1145622014}) 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3813882454}) 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2143682126}) 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3813882454}) 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=932456531}) 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3813882454}) 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3078691324}) 2024-09-15T01:47:53.742 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3813882454}) 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2391149805}) 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3813882454}) 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2832616117}) 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3813882454}) 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1359631437}) 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3813882454}) 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.452+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2447148312}) 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.743 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677655230}) 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=401835584}) 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677655230}) 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2959852551}) 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677655230}) 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1526428329}) 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677655230}) 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2977864518}) 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677655230}) 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2536947709}) 2024-09-15T01:47:53.744 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677655230}) 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2780548915}) 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677655230}) 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3225760912}) 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677655230}) 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1586495318}) 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011075341}) 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821424632}) 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011075341}) 2024-09-15T01:47:53.745 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=675842936}) 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011075341}) 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1157611027}) 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011075341}) 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4196588039}) 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.454+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011075341}) 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1353674412}) 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011075341}) 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1677028843}) 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.746 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011075341}) 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4083724181}) 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011075341}) 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3166115531}) 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1897186033}) 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2184753861}) 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1897186033}) 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1622947946}) 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1897186033}) 2024-09-15T01:47:53.747 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=61053346}) 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1897186033}) 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=399546637}) 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1897186033}) 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2842037635}) 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1897186033}) 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1731461342}) 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1897186033}) 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3720456962}) 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1897186033}) 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.456+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=230110356}) 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1470487695}) 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1811203044}) 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1470487695}) 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3810409255}) 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1470487695}) 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=629203943}) 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1470487695}) 2024-09-15T01:47:53.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=796927677}) 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1470487695}) 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2501131351}) 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1470487695}) 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=877567177}) 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1470487695}) 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=906798758}) 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1470487695}) 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3421403440}) 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3844049272}) 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3848778355}) 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3844049272}) 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1243614056}) 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3844049272}) 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3414228180}) 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3844049272}) 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2473343002}) 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.458+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3844049272}) 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3093038844}) 2024-09-15T01:47:53.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3844049272}) 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2012529673}) 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3844049272}) 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2276821304}) 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3844049272}) 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2746158731}) 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3781817535}) 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3291193615}) 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3781817535}) 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3702201922}) 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3781817535}) 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2075850198}) 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3781817535}) 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2577506628}) 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3781817535}) 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=374650840}) 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3781817535}) 2024-09-15T01:47:53.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3214698235}) 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3781817535}) 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2130255240}) 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3781817535}) 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.460+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3832510336}) 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=372778886}) 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=294998245}) 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=372778886}) 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3135859630}) 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=372778886}) 2024-09-15T01:47:53.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2743854846}) 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=372778886}) 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2768820359}) 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=372778886}) 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1213302663}) 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=372778886}) 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3721341196}) 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=372778886}) 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=326636227}) 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=372778886}) 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.461+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3777675446}) 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2994583923}) 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1496059028}) 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2994583923}) 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3872278682}) 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2994583923}) 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2442697890}) 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2994583923}) 2024-09-15T01:47:53.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3032388731}) 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2994583923}) 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=963751452}) 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.463+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2994583923}) 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.463+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2075507034}) 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.463+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.463+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2994583923}) 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.463+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2930080152}) 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.463+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.463+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2994583923}) 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.463+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3421467313}) 2024-09-15T01:47:53.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2391769637}) 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969381763}) 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2391769637}) 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1554729251}) 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2391769637}) 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2277231436}) 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2391769637}) 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2097609694}) 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2391769637}) 2024-09-15T01:47:53.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2068115789}) 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2391769637}) 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1681864157}) 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2391769637}) 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=755530607}) 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2391769637}) 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2886913064}) 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3849533051}) 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3321520079}) 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3849533051}) 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=574839169}) 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3849533051}) 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2937813721}) 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3849533051}) 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2910367860}) 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.466+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.466+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3849533051}) 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.466+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2431961457}) 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.466+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.466+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3849533051}) 2024-09-15T01:47:53.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.466+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2479766921}) 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.466+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.466+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3849533051}) 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.466+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2836032822}) 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.466+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.466+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3849533051}) 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.466+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=885064679}) 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4059155659}) 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1403321167}) 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4059155659}) 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=465430788}) 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4059155659}) 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2657538188}) 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4059155659}) 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2799823790}) 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4059155659}) 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2729031363}) 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4059155659}) 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=675761643}) 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4059155659}) 2024-09-15T01:47:53.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3174171748}) 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4059155659}) 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1311732677}) 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2539900344}) 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1146504597}) 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2539900344}) 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3142676254}) 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2539900344}) 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1859937972}) 2024-09-15T01:47:53.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2539900344}) 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2913712847}) 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2539900344}) 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3579381221}) 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2539900344}) 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1595502346}) 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2539900344}) 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1791189416}) 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2539900344}) 2024-09-15T01:47:53.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.468+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=249736253}) 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2504179143}) 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=257496266}) 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2504179143}) 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=322914234}) 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2504179143}) 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=497124314}) 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2504179143}) 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1226543226}) 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2504179143}) 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3704441050}) 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2504179143}) 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1238290571}) 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2504179143}) 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1168194328}) 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2504179143}) 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3858315595}) 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3514867946}) 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1613337436}) 2024-09-15T01:47:53.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3514867946}) 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1619870100}) 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3514867946}) 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=131148576}) 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3514867946}) 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3873689283}) 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3514867946}) 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3500828460}) 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3514867946}) 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3478482784}) 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3514867946}) 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3812749405}) 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3514867946}) 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2991536644}) 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=267653210}) 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2487061167}) 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=267653210}) 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2054788225}) 2024-09-15T01:47:53.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=267653210}) 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=651570307}) 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=267653210}) 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1492081562}) 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=267653210}) 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=731819892}) 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=267653210}) 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3999089114}) 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=267653210}) 2024-09-15T01:47:53.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3047874815}) 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=267653210}) 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=504460837}) 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=938679066}) 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2132585220}) 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=938679066}) 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2776948316}) 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=938679066}) 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3911047616}) 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=938679066}) 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3730525723}) 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=938679066}) 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1957647160}) 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=938679066}) 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3800971230}) 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=938679066}) 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=603335404}) 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=938679066}) 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3461026116}) 2024-09-15T01:47:53.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2577043679}) 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82774475}) 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2577043679}) 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=533952368}) 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2577043679}) 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2290498012}) 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2577043679}) 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3199878980}) 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2577043679}) 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3283732295}) 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2577043679}) 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1133102889}) 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2577043679}) 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2181394256}) 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2577043679}) 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3624557516}) 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3883178874}) 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1528900031}) 2024-09-15T01:47:53.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3883178874}) 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3019710058}) 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3883178874}) 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4055092711}) 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3883178874}) 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3568851663}) 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3883178874}) 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=406573115}) 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3883178874}) 2024-09-15T01:47:53.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4191223668}) 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3883178874}) 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1412492610}) 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3883178874}) 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3089400924}) 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1152784078}) 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3172537237}) 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1152784078}) 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1012925611}) 2024-09-15T01:47:53.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1152784078}) 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2153244658}) 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1152784078}) 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=154655962}) 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1152784078}) 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3289414309}) 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1152784078}) 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2411045619}) 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1152784078}) 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=787995149}) 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1152784078}) 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1776986817}) 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=117651895}) 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3380003165}) 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=117651895}) 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1442870142}) 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=117651895}) 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1153630904}) 2024-09-15T01:47:53.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=117651895}) 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1959595470}) 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=117651895}) 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2688263077}) 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=117651895}) 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=124495164}) 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=117651895}) 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2029161672}) 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=117651895}) 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3138817410}) 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=301939199}) 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1251891765}) 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=301939199}) 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3652194674}) 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=301939199}) 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=71624318}) 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=301939199}) 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=493900992}) 2024-09-15T01:47:53.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=301939199}) 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=283922879}) 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=301939199}) 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2195092691}) 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=301939199}) 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2226300319}) 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=301939199}) 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1356907102}) 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3892039795}) 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2898924462}) 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3892039795}) 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2787366532}) 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3892039795}) 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2974145138}) 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3892039795}) 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2195094199}) 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3892039795}) 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3570660227}) 2024-09-15T01:47:53.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3892039795}) 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2047823263}) 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3892039795}) 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1993468998}) 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3892039795}) 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=935368065}) 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3507289349}) 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=543212947}) 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3507289349}) 2024-09-15T01:47:53.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=622119492}) 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3507289349}) 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=708368380}) 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3507289349}) 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2996900050}) 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3507289349}) 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=614700245}) 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3507289349}) 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3258103611}) 2024-09-15T01:47:53.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3507289349}) 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=786378880}) 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3507289349}) 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=450169075}) 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=57} crc {logm=86463116,mdsmap=1531151339,mgr=1949893131,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1685706360,mon_config_key=1111662797,monmap=2850442478,osd_metadata=2302127439,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=1022277026}) 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=6,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=57} crc {logm=86463116,mdsmap=3140514009,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1213525743,mon_config_key=1111662797,monmap=2850442478,osd_metadata=860431027,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=3658825924}) 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=57} crc {logm=86463116,mdsmap=1531151339,mgr=1949893131,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1685706360,mon_config_key=1111662797,monmap=2850442478,osd_metadata=2302127439,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=1022277026}) 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=5,mdsmap=2,mgr=7,mgr_command_descs=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=59} crc {logm=1445915578,mdsmap=3585761405,mgr=743505257,mgr_command_descs=3545737492,mgrstat=1213525743,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=4178261971}) 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=57} crc {logm=86463116,mdsmap=1531151339,mgr=1949893131,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1685706360,mon_config_key=1111662797,monmap=2850442478,osd_metadata=2302127439,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=1022277026}) 2024-09-15T01:47:53.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=58} crc {logm=86463116,mdsmap=1531151339,mgr=2983835071,mgr_metadata=3150520333,mgrstat=3363716959,mon_config_key=1111662797,monmap=90612552,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=2053951498}) 2024-09-15T01:47:53.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=57} crc {logm=86463116,mdsmap=1531151339,mgr=1949893131,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1685706360,mon_config_key=1111662797,monmap=2850442478,osd_metadata=2302127439,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=1022277026}) 2024-09-15T01:47:53.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=5,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=58} crc {logm=2772233368,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520334,mgrstat=3363716959,mon_config_key=1692164895,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=4042244009}) 2024-09-15T01:47:53.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=57} crc {logm=86463116,mdsmap=1531151339,mgr=1949893131,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1685706360,mon_config_key=1111662797,monmap=2850442478,osd_metadata=2302127439,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=1022277026}) 2024-09-15T01:47:53.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=5,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=59} crc {logm=1445915578,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=140607907,mon_config_key=1841931312,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=2465651172}) 2024-09-15T01:47:53.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=57} crc {logm=86463116,mdsmap=1531151339,mgr=1949893131,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1685706360,mon_config_key=1111662797,monmap=2850442478,osd_metadata=2302127439,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=1022277026}) 2024-09-15T01:47:53.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=5,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=57} crc {logm=1960008939,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1213525743,mon_config_key=1111662797,monmap=2850442478,osd_metadata=4292802864,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=2743130683}) 2024-09-15T01:47:53.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=57} crc {logm=86463116,mdsmap=1531151339,mgr=1949893131,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1685706360,mon_config_key=1111662797,monmap=2850442478,osd_metadata=2302127439,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=1022277026}) 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=6,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=58} crc {logm=86463116,mdsmap=1531151339,mgr=743505257,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1213525743,mon_config_key=1905138545,monmap=3140514009,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=956710772,osdmap=998392161}) 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=57} crc {logm=86463116,mdsmap=1531151339,mgr=1949893131,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1685706360,mon_config_key=1111662797,monmap=2850442478,osd_metadata=2302127439,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=1022277026}) 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=55} crc {logm=86463116,mdsmap=1531151339,mgr=2983835072,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1213525743,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=3803836852}) 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=76} crc {osdmap=3686022971}) 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=74} crc {osdmap=1851302591}) 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=76} crc {osdmap=3686022971}) 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=77} crc {osdmap=4161594551}) 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=76} crc {osdmap=3686022971}) 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=72} crc {osdmap=2715963325}) 2024-09-15T01:47:53.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=76} crc {osdmap=3686022971}) 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=75} crc {osdmap=1448953600}) 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=76} crc {osdmap=3686022971}) 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=75} crc {osdmap=2465074402}) 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=76} crc {osdmap=3686022971}) 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=70} crc {osdmap=3986748953}) 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=76} crc {osdmap=3686022971}) 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=75} crc {osdmap=2419783028}) 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:47:53.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=76} crc {osdmap=3686022971}) 2024-09-15T01:47:53.788 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:47:53.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=75} crc {osdmap=810676630}) 2024-09-15T01:47:54.346 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:47:54.346 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:47:54.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.449+0000 7f28c582b640 10 monclient: get_monmap_and_config 2024-09-15T01:47:54.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.449+0000 7f28c582b640 10 monclient: build_initial_monmap 2024-09-15T01:47:54.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.449+0000 7f28c582b640 10 monclient: monmap: 2024-09-15T01:47:54.453 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:54.453 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.453 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:54.451678+0000 2024-09-15T01:47:54.453 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:54.451678+0000 2024-09-15T01:47:54.453 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 1 Processor -- start 2024-09-15T01:47:54.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 1 -- start start 2024-09-15T01:47:54.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 10 monclient: init 2024-09-15T01:47:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 10 monclient: _add_conns ranks=[3,10,6,4,0,2,8,9,1,7,5] 2024-09-15T01:47:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c00a2930 0x7f28c0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 10 monclient(hunting): picked mon.noname-g con 0x7f28c00a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f28c009faa0 0x7f28c009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 10 monclient(hunting): picked mon.noname-f con 0x7f28c009faa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:47:54.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a08c0 0x7f28c00a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 10 monclient(hunting): picked mon.noname-d con 0x7f28c00a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:54.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c009e0f0 con 0x7f28c009faa0 2024-09-15T01:47:54.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c009dd80 con 0x7f28c00a2930 2024-09-15T01:47:54.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00a2d00 con 0x7f28c00a08c0 2024-09-15T01:47:54.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:54.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.450+0000 7f28c582b640 10 monclient(hunting): authenticate will time out at 1679.201896s 2024-09-15T01:47:54.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a08c0 0x7f28c00a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a08c0 0x7f28c00a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42012/0 (socket says 172.21.15.178:42012) 2024-09-15T01:47:54.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 1 -- 172.21.15.178:0/2608241835 learned_addr learned my addr 172.21.15.178:0/2608241835 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:54.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 10 monclient(hunting): get_auth_request con 0x7f28c00a08c0 auth_method 0 2024-09-15T01:47:54.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28be7fc640 1 --2- 172.21.15.178:0/2608241835 >> v2:172.21.15.204:3300/0 conn(0x7f28c009faa0 0x7f28c009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28beffd640 1 --2- 172.21.15.178:0/2608241835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c00a2930 0x7f28c0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28be7fc640 10 monclient(hunting): get_auth_request con 0x7f28c009faa0 auth_method 0 2024-09-15T01:47:54.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28beffd640 10 monclient(hunting): get_auth_request con 0x7f28c00a2930 auth_method 0 2024-09-15T01:47:54.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28beffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 10 monclient(hunting): handle_auth_done global_id 25674 payload 274 2024-09-15T01:47:54.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 1 -- 172.21.15.178:0/2608241835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c00a2930 msgr2=0x7f28c0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:54.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 1 --2- 172.21.15.178:0/2608241835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c00a2930 0x7f28c0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 1 -- 172.21.15.178:0/2608241835 >> v2:172.21.15.204:3300/0 conn(0x7f28c009faa0 msgr2=0x7f28c009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:54.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 1 --2- 172.21.15.178:0/2608241835 >> v2:172.21.15.204:3300/0 conn(0x7f28c009faa0 0x7f28c009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:47:54.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 1 monclient: found mon.noname-d 2024-09-15T01:47:54.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:54.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:47:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 1 -- 172.21.15.178:0/2608241835 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c004bf60 con 0x7f28c00a08c0 2024-09-15T01:47:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28be7fc640 1 --2- 172.21.15.178:0/2608241835 >> v2:172.21.15.204:3300/0 conn(0x7f28c009faa0 0x7f28c009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:47:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.451+0000 7f28bf7fe640 1 --2- 172.21.15.178:0/2608241835 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a08c0 0x7f28c00a0d60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009f20 tx=0x7f28b8009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ce527160d3dccd2a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28bdffb640 1 -- 172.21.15.178:0/2608241835 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28b8007880 con 0x7f28c00a08c0 2024-09-15T01:47:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28bdffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:47:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28bdffb640 10 monclient: dump: 2024-09-15T01:47:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28bdffb640 10 monclient: _finish_auth 0 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 5 monclient: authenticate success, global_id 25674 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28bdffb640 1 -- 172.21.15.178:0/2608241835 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b802d050 con 0x7f28c00a08c0 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28bdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 4 set_mon_vals no callback set 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28bdffb640 1 -- 172.21.15.178:0/2608241835 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28b8030360 con 0x7f28c00a08c0 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28bdffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:47:54.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28bdffb640 10 monclient: dump: 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.472 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 10 monclient: shutdown 2024-09-15T01:47:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 1 -- 172.21.15.178:0/2608241835 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a08c0 msgr2=0x7f28c00a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 1 --2- 172.21.15.178:0/2608241835 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a08c0 0x7f28c00a0d60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009f20 tx=0x7f28b8009840 comp rx=0 tx=0).stop 2024-09-15T01:47:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 1 -- 172.21.15.178:0/2608241835 shutdown_connections 2024-09-15T01:47:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 1 --2- 172.21.15.178:0/2608241835 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a08c0 0x7f28c00a0d60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 1 --2- 172.21.15.178:0/2608241835 >> v2:172.21.15.204:3300/0 conn(0x7f28c009faa0 0x7f28c009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 1 --2- 172.21.15.178:0/2608241835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c00a2930 0x7f28c0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 1 -- 172.21.15.178:0/2608241835 >> 172.21.15.178:0/2608241835 conn(0x7f28c00a5ed0 msgr2=0x7f28c00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:54.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 1 -- 172.21.15.178:0/2608241835 shutdown_connections 2024-09-15T01:47:54.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.452+0000 7f28c582b640 1 -- 172.21.15.178:0/2608241835 wait complete. 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.453+0000 7f28c582b640 10 monclient: build_initial_monmap 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.453+0000 7f28c582b640 10 monclient: monmap: 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:54.477 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:54.478 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:54.478 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:54.478 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:54.478 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.453+0000 7f28c582b640 1 Processor -- start 2024-09-15T01:47:54.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.453+0000 7f28c582b640 1 -- start start 2024-09-15T01:47:54.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.453+0000 7f28c582b640 10 monclient: init 2024-09-15T01:47:54.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:54.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 10 monclient: _add_conns ranks=[3,7,6,0,5,1,2,8,4] 2024-09-15T01:47:54.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c009faa0 0x7f28c009b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 10 monclient(hunting): picked mon.noname-g con 0x7f28c009faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:54.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f28c00a08c0 0x7f28c01319e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 10 monclient(hunting): picked mon.noname-i con 0x7f28c00a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a2930 0x7f28c0131f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 10 monclient(hunting): picked mon.noname-d con 0x7f28c00a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c011f0e0 con 0x7f28c009faa0 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f28c011eb20 con 0x7f28c00a2930 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f28c011dbc0 con 0x7f28c00a08c0 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28c582b640 10 monclient(hunting): authenticate will time out at 1679.205308s 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a2930 0x7f28c0131f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a2930 0x7f28c0131f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42026/0 (socket says 172.21.15.178:42026) 2024-09-15T01:47:54.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 1 -- 172.21.15.178:0/1211412197 learned_addr learned my addr 172.21.15.178:0/1211412197 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:54.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 10 monclient(hunting): get_auth_request con 0x7f28c00a2930 auth_method 0 2024-09-15T01:47:54.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28beffd640 1 --2- 172.21.15.178:0/1211412197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c009faa0 0x7f28c009b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28beffd640 10 monclient(hunting): get_auth_request con 0x7f28c009faa0 auth_method 0 2024-09-15T01:47:54.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28beffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28be7fc640 1 --2- 172.21.15.178:0/1211412197 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f28c00a08c0 0x7f28c01319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28be7fc640 10 monclient(hunting): get_auth_request con 0x7f28c00a08c0 auth_method 0 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28be7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28be7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28be7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 10 monclient(hunting): handle_auth_done global_id 25683 payload 824 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 1 -- 172.21.15.178:0/1211412197 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f28c00a08c0 msgr2=0x7f28c01319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 1 --2- 172.21.15.178:0/1211412197 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f28c00a08c0 0x7f28c01319e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 1 -- 172.21.15.178:0/1211412197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c009faa0 msgr2=0x7f28c009b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 1 --2- 172.21.15.178:0/1211412197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c009faa0 0x7f28c009b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 1 monclient: found mon.noname-d 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 1 -- 172.21.15.178:0/1211412197 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8009ae0 con 0x7f28c00a2930 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28bf7fe640 1 --2- 172.21.15.178:0/1211412197 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a2930 0x7f28c0131f20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009df0 tx=0x7f28b8036000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28a6ffd640 1 -- 172.21.15.178:0/1211412197 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28b803baa0 con 0x7f28c00a2930 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28a6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28a6ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.454+0000 7f28a6ffd640 10 monclient: dump: 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.484 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.484 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.484 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28a6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:47:54.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:54.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28c582b640 5 monclient: authenticate success, global_id 25683 2024-09-15T01:47:54.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28a6ffd640 1 -- 172.21.15.178:0/1211412197 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b803bc40 con 0x7f28c00a2930 2024-09-15T01:47:54.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28c582b640 10 monclient: _renew_subs 2024-09-15T01:47:54.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28c582b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:47:54.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28c582b640 1 -- 172.21.15.178:0/1211412197 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c011ee00 con 0x7f28c00a2930 2024-09-15T01:47:54.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28a6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:54.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28a6ffd640 1 -- 172.21.15.178:0/1211412197 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28b803bdb0 con 0x7f28c00a2930 2024-09-15T01:47:54.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28a6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28a6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:47:54.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28a6ffd640 10 monclient: dump: 2024-09-15T01:47:54.485 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.485 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.485 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.485 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.485 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.485 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28c582b640 10 monclient: _renew_subs 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28c582b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28bd7fa640 4 set_mon_vals no callback set 2024-09-15T01:47:54.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.455+0000 7f28c582b640 1 -- 172.21.15.178:0/1211412197 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c011ca40 con 0x7f28c00a2930 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.456+0000 7f28a6ffd640 1 -- 172.21.15.178:0/1211412197 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f28b8038070 con 0x7f28c00a2930 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.456+0000 7f28c582b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.456+0000 7f28c582b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.456+0000 7f28c582b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.456+0000 7f28c582b640 1 -- 172.21.15.178:0/1211412197 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c009d6a0 con 0x7f28c00a2930 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.456+0000 7f28a6ffd640 1 --2- 172.21.15.178:0/1211412197 >> v2:172.21.15.204:6812/28210 conn(0x7f28ac0397b0 0x7f28ac03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.456+0000 7f28a6ffd640 1 -- 172.21.15.178:0/1211412197 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f28b806be90 con 0x7f28c00a2930 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.456+0000 7f28beffd640 1 --2- 172.21.15.178:0/1211412197 >> v2:172.21.15.204:6812/28210 conn(0x7f28ac0397b0 0x7f28ac03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.456+0000 7f28beffd640 10 monclient: get_auth_request con 0x7f28ac0397b0 auth_method 0 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.456+0000 7f28beffd640 1 --2- 172.21.15.178:0/1211412197 >> v2:172.21.15.204:6812/28210 conn(0x7f28ac0397b0 0x7f28ac03bc60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f28b0007ba0 tx=0x7f28b0008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.460+0000 7f28a6ffd640 1 -- 172.21.15.178:0/1211412197 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f28c009d6a0 con 0x7f28c00a2930 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.460+0000 7f28a6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:54.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.460+0000 7f28a6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:54.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.673+0000 7f28c582b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:47:54.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.673+0000 7f28c582b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:47:54.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.673+0000 7f28c582b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:47:54.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.673+0000 7f28c582b640 1 -- 172.21.15.178:0/1211412197 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f28c009d6a0 con 0x7f28c00a2930 2024-09-15T01:47:54.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.674+0000 7f28a6ffd640 1 -- 172.21.15.178:0/1211412197 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7f28c009d6a0 con 0x7f28c00a2930 2024-09-15T01:47:54.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.674+0000 7f28a6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:47:54.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.674+0000 7f28a6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:54.680 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":150,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:47:54.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 1 -- 172.21.15.178:0/1211412197 >> v2:172.21.15.204:6812/28210 conn(0x7f28ac0397b0 msgr2=0x7f28ac03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 1 --2- 172.21.15.178:0/1211412197 >> v2:172.21.15.204:6812/28210 conn(0x7f28ac0397b0 0x7f28ac03bc60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f28b0007ba0 tx=0x7f28b0008040 comp rx=0 tx=0).stop 2024-09-15T01:47:54.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 10 monclient: shutdown 2024-09-15T01:47:54.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:54.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 1 -- 172.21.15.178:0/1211412197 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a2930 msgr2=0x7f28c0131f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 1 --2- 172.21.15.178:0/1211412197 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a2930 0x7f28c0131f20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009df0 tx=0x7f28b8036000 comp rx=0 tx=0).stop 2024-09-15T01:47:54.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 1 -- 172.21.15.178:0/1211412197 shutdown_connections 2024-09-15T01:47:54.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 1 --2- 172.21.15.178:0/1211412197 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f28c00a2930 0x7f28c0131f20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 1 --2- 172.21.15.178:0/1211412197 >> v2:172.21.15.204:6812/28210 conn(0x7f28ac0397b0 0x7f28ac03bc60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 1 --2- 172.21.15.178:0/1211412197 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f28c00a08c0 0x7f28c01319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 1 --2- 172.21.15.178:0/1211412197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c009faa0 0x7f28c009b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 1 -- 172.21.15.178:0/1211412197 >> 172.21.15.178:0/1211412197 conn(0x7f28c00a5ed0 msgr2=0x7f28c0135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:54.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 1 -- 172.21.15.178:0/1211412197 shutdown_connections 2024-09-15T01:47:54.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.676+0000 7f28c582b640 1 -- 172.21.15.178:0/1211412197 wait complete. 2024-09-15T01:47:54.689 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:47:54.690 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T01:47:54.690 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:47:54.690 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:47:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.779+0000 7f8b9802b640 10 monclient: get_monmap_and_config 2024-09-15T01:47:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.779+0000 7f8b9802b640 10 monclient: build_initial_monmap 2024-09-15T01:47:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.779+0000 7f8b9802b640 10 monclient: monmap: 2024-09-15T01:47:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:54.781768+0000 2024-09-15T01:47:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:54.781768+0000 2024-09-15T01:47:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:54.784 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:54.784 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:54.784 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:54.784 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:54.784 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:54.784 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:54.784 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:54.784 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:54.784 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.780+0000 7f8b9802b640 1 Processor -- start 2024-09-15T01:47:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.780+0000 7f8b9802b640 1 -- start start 2024-09-15T01:47:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.780+0000 7f8b9802b640 10 monclient: init 2024-09-15T01:47:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 10 monclient: _add_conns ranks=[6,9,5,2,7,0,8,4,3,1,10] 2024-09-15T01:47:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b900a2930 0x7f8b90132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 10 monclient(hunting): picked mon.noname-d con 0x7f8b900a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b9009faa0 0x7f8b9009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 10 monclient(hunting): picked mon.noname-a con 0x7f8b9009faa0 addr v2:172.21.15.178:3300/0 2024-09-15T01:47:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b900a08c0 0x7f8b900a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 10 monclient(hunting): picked mon.noname-h con 0x7f8b900a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b9009e0f0 con 0x7f8b9009faa0 2024-09-15T01:47:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9009dd80 con 0x7f8b900a08c0 2024-09-15T01:47:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b900a2d00 con 0x7f8b900a2930 2024-09-15T01:47:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b9802b640 10 monclient(hunting): authenticate will time out at 1679.532197s 2024-09-15T01:47:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b955c6640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b9009faa0 0x7f8b9009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b900a2930 0x7f8b90132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b955c6640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b9009faa0 0x7f8b9009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35872/0 (socket says 172.21.15.178:35872) 2024-09-15T01:47:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b955c6640 1 -- 172.21.15.178:0/621985990 learned_addr learned my addr 172.21.15.178:0/621985990 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b955c6640 10 monclient(hunting): get_auth_request con 0x7f8b9009faa0 auth_method 0 2024-09-15T01:47:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b955c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b955c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b955c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 10 monclient(hunting): get_auth_request con 0x7f8b900a2930 auth_method 0 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b955c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b955c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b965c8640 1 --2- 172.21.15.178:0/621985990 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b900a08c0 0x7f8b900a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b955c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b965c8640 10 monclient(hunting): get_auth_request con 0x7f8b900a08c0 auth_method 0 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b965c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b965c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b965c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 10 monclient(hunting): handle_auth_done global_id 25692 payload 274 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 1 -- 172.21.15.178:0/621985990 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b900a08c0 msgr2=0x7f8b900a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 1 --2- 172.21.15.178:0/621985990 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b900a08c0 0x7f8b900a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 1 -- 172.21.15.178:0/621985990 >> v2:172.21.15.178:3300/0 conn(0x7f8b9009faa0 msgr2=0x7f8b9009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 1 --2- 172.21.15.178:0/621985990 >> v2:172.21.15.178:3300/0 conn(0x7f8b9009faa0 0x7f8b9009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 10 monclient: _finish_hunting 0 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 1 monclient: found mon.noname-d 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:54.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:47:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b95dc7640 1 -- 172.21.15.178:0/621985990 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9004bf60 con 0x7f8b900a2930 2024-09-15T01:47:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b955c6640 1 --2- 172.21.15.178:0/621985990 >> v2:172.21.15.178:3300/0 conn(0x7f8b9009faa0 0x7f8b9009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.781+0000 7f8b965c8640 1 --2- 172.21.15.178:0/621985990 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b900a08c0 0x7f8b900a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:47:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b95dc7640 1 --2- 172.21.15.178:0/621985990 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b900a2930 0x7f8b90132de0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009850 tx=0x7f8b8c0090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=fa3f1b07a116bb24 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b94dc5640 1 -- 172.21.15.178:0/621985990 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b8c02abc0 con 0x7f8b900a2930 2024-09-15T01:47:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b94dc5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b94dc5640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:47:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b94dc5640 10 monclient: dump: 2024-09-15T01:47:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b94dc5640 10 monclient: _finish_auth 0 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b94dc5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b94dc5640 1 -- 172.21.15.178:0/621985990 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c02ad40 con 0x7f8b900a2930 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b9802b640 5 monclient: authenticate success, global_id 25692 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b9802b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b94dc5640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b9802b640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b94dc5640 1 -- 172.21.15.178:0/621985990 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b8c031ca0 con 0x7f8b900a2930 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b9802b640 4 set_mon_vals no callback set 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b94dc5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b94dc5640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.782+0000 7f8b94dc5640 10 monclient: dump: 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.801 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.802 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 10 monclient: shutdown 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 1 -- 172.21.15.178:0/621985990 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b900a2930 msgr2=0x7f8b90132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 1 --2- 172.21.15.178:0/621985990 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b900a2930 0x7f8b90132de0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009850 tx=0x7f8b8c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 1 -- 172.21.15.178:0/621985990 shutdown_connections 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 1 --2- 172.21.15.178:0/621985990 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8b900a08c0 0x7f8b900a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 1 --2- 172.21.15.178:0/621985990 >> v2:172.21.15.178:3300/0 conn(0x7f8b9009faa0 0x7f8b9009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 1 --2- 172.21.15.178:0/621985990 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8b900a2930 0x7f8b90132de0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 1 -- 172.21.15.178:0/621985990 >> 172.21.15.178:0/621985990 conn(0x7f8b900a5ed0 msgr2=0x7f8b900a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 1 -- 172.21.15.178:0/621985990 shutdown_connections 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 1 -- 172.21.15.178:0/621985990 wait complete. 2024-09-15T01:47:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 10 monclient: build_initial_monmap 2024-09-15T01:47:54.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.783+0000 7f8b9802b640 10 monclient: monmap: 2024-09-15T01:47:54.804 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:54.804 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:54.804 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:54.804 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:54.804 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:54.804 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:54.804 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:54.804 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:54.804 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:54.804 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:54.804 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:54.805 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:54.805 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:54.805 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:54.805 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:54.805 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 1 Processor -- start 2024-09-15T01:47:54.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 1 -- start start 2024-09-15T01:47:54.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 10 monclient: init 2024-09-15T01:47:54.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:54.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 10 monclient: _add_conns ranks=[8,3,7,4,0,6,5,2,1] 2024-09-15T01:47:54.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b9009faa0 0x7f8b9009b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 10 monclient(hunting): picked mon.noname-e con 0x7f8b9009faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b900a08c0 0x7f8b901318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 10 monclient(hunting): picked mon.noname-g con 0x7f8b900a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b900a2930 0x7f8b9011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 10 monclient(hunting): picked mon.noname-i con 0x7f8b900a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9009d630 con 0x7f8b900a08c0 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9009bc90 con 0x7f8b900a2930 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9011e9d0 con 0x7f8b9009faa0 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:54.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b9802b640 10 monclient(hunting): authenticate will time out at 1679.535583s 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b95dc7640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b9009faa0 0x7f8b9009b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b95dc7640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b9009faa0 0x7f8b9009b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42158/0 (socket says 172.21.15.178:42158) 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b95dc7640 1 -- 172.21.15.178:0/1806280254 learned_addr learned my addr 172.21.15.178:0/1806280254 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b95dc7640 10 monclient(hunting): get_auth_request con 0x7f8b9009faa0 auth_method 0 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b95dc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b95dc7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b95dc7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b95dc7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b95dc7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b95dc7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b965c8640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b900a2930 0x7f8b9011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b955c6640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b900a08c0 0x7f8b901318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b965c8640 10 monclient(hunting): get_auth_request con 0x7f8b900a2930 auth_method 0 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b965c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b965c8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b965c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b955c6640 10 monclient(hunting): get_auth_request con 0x7f8b900a08c0 auth_method 0 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b955c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b955c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b955c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b965c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b965c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b965c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b955c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b955c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b955c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.784+0000 7f8b95dc7640 10 monclient(hunting): handle_auth_done global_id 94220 payload 824 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b95dc7640 1 -- 172.21.15.178:0/1806280254 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b900a2930 msgr2=0x7f8b9011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b95dc7640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f8b900a2930 0x7f8b9011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b95dc7640 1 -- 172.21.15.178:0/1806280254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b900a08c0 msgr2=0x7f8b901318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b95dc7640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b900a08c0 0x7f8b901318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b95dc7640 10 monclient: _finish_hunting 0 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b95dc7640 1 monclient: found mon.noname-e 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b95dc7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b95dc7640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b95dc7640 1 -- 172.21.15.178:0/1806280254 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b80009450 con 0x7f8b9009faa0 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b95dc7640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b9009faa0 0x7f8b9009b280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c000c00 tx=0x7f8b8c032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b8c03c880 con 0x7f8b9009faa0 2024-09-15T01:47:54.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b85ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b85ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b85ffb640 10 monclient: dump: 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.811 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.811 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.811 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.811 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.811 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b85ffb640 10 monclient: _finish_auth 0 2024-09-15T01:47:54.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b85ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:54.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c03ca20 con 0x7f8b9009faa0 2024-09-15T01:47:54.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b9802b640 5 monclient: authenticate success, global_id 94220 2024-09-15T01:47:54.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b85ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:54.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b8c03cb90 con 0x7f8b9009faa0 2024-09-15T01:47:54.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b85ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b85ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b85ffb640 10 monclient: dump: 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.812 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b87fff640 4 set_mon_vals no callback set 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b9802b640 10 monclient: _renew_subs 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b9802b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b9802b640 1 -- 172.21.15.178:0/1806280254 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c009390 con 0x7f8b9009faa0 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b9802b640 10 monclient: _renew_subs 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b9802b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.785+0000 7f8b9802b640 1 -- 172.21.15.178:0/1806280254 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9011ef90 con 0x7f8b9009faa0 2024-09-15T01:47:54.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8b8c03ccf0 con 0x7f8b9009faa0 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b9802b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b9802b640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b90131df0 0x7f8b9011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b9802b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b9802b640 1 -- 172.21.15.178:0/1806280254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9011ee50 con 0x7f8b90131df0 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b955c6640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b90131df0 0x7f8b9011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b955c6640 10 monclient: get_auth_request con 0x7f8b90131df0 auth_method 0 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b955c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b955c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b955c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b85ffb640 1 --2- 172.21.15.178:0/1806280254 >> v2:172.21.15.204:6812/28210 conn(0x7f8b740397b0 0x7f8b7403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b955c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b955c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b955c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f8b8c06b8c0 con 0x7f8b9009faa0 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b965c8640 1 --2- 172.21.15.178:0/1806280254 >> v2:172.21.15.204:6812/28210 conn(0x7f8b740397b0 0x7f8b7403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b955c6640 10 monclient(hunting): handle_auth_done global_id 64458 payload 824 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b955c6640 1 -- 172.21.15.178:0/1806280254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b9004c4a0 con 0x7f8b90131df0 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b965c8640 10 monclient: get_auth_request con 0x7f8b740397b0 auth_method 0 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.786+0000 7f8b955c6640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b90131df0 0x7f8b9011ca70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8b80006cb0 tx=0x7f8b800091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.787+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b8000fd20 con 0x7f8b90131df0 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.787+0000 7f8b85ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.787+0000 7f8b85ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.787+0000 7f8b85ffb640 10 monclient: dump: 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.815 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.787+0000 7f8b965c8640 1 --2- 172.21.15.178:0/1806280254 >> v2:172.21.15.204:6812/28210 conn(0x7f8b740397b0 0x7f8b7403bc60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8b880076e0 tx=0x7f8b88009990 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.787+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f8b8000fe80 con 0x7f8b90131df0 2024-09-15T01:47:54.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.787+0000 7f8b85ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:54.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.787+0000 7f8b85ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:54.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.787+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b90131df0 msgr2=0x7f8b9011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.787+0000 7f8b85ffb640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b90131df0 0x7f8b9011ca70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8b80006cb0 tx=0x7f8b800091d0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.787+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b90131df0 msgr2=0x7f8b9011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:54.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.787+0000 7f8b85ffb640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b90131df0 0x7f8b9011ca70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.792+0000 7f8b9802b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:54.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b9802b640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b9011cfb0 0x7f8b90096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b9802b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b9802b640 1 -- 172.21.15.178:0/1806280254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9011f180 con 0x7f8b9011cfb0 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b955c6640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b9011cfb0 0x7f8b90096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b955c6640 10 monclient: get_auth_request con 0x7f8b9011cfb0 auth_method 0 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b955c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b955c6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b955c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b955c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b955c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b955c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b955c6640 10 monclient(hunting): handle_auth_done global_id 64467 payload 824 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b955c6640 1 -- 172.21.15.178:0/1806280254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8b9004bc80 con 0x7f8b9011cfb0 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b955c6640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b9011cfb0 0x7f8b90096a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8b80006cb0 tx=0x7f8b8001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b80014d80 con 0x7f8b9011cfb0 2024-09-15T01:47:54.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b85ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b85ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b85ffb640 10 monclient: dump: 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.822 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.822 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.822 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.822 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.822 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f8b8001c3e0 con 0x7f8b9011cfb0 2024-09-15T01:47:54.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b85ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:54.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b85ffb640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:54.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b9011cfb0 msgr2=0x7f8b90096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b85ffb640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b9011cfb0 0x7f8b90096a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8b80006cb0 tx=0x7f8b8001d000 comp rx=0 tx=0).stop 2024-09-15T01:47:54.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b85ffb640 1 -- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b9011cfb0 msgr2=0x7f8b90096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:54.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.793+0000 7f8b85ffb640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b9011cfb0 0x7f8b90096a30 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8b80006cb0 tx=0x7f8b8001d000 comp rx=0 tx=0).stop 2024-09-15T01:47:54.823 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:54.823 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:54.823 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:54.823 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:54.823 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:54.823 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:54.823 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:54.823 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:54.823 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:54.823 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:54.824 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:54.825 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:54.826 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:54.827 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.828 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.829 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.830 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.830 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:54.830 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:54.830 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.830 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.830 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.830 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.830 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:54.830 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:54.830 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.832 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:54.833 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:54.836 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:54.837 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.838 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.838 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.838 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:54.838 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.838 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.838 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:54.838 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.838 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.838 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.839 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:54.839 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:54.839 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.839 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.839 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.839 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.839 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:54.840 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:54.840 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.840 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.841 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.842 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.843 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.844 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.845 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:54.846 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:54.847 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.847 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:54.847 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:54.847 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.847 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:54.847 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:54.847 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:54.847 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.847 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.847 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:54.847 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:54.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.794+0000 7f8b9802b640 1 -- 172.21.15.178:0/1806280254 >> v2:172.21.15.204:6812/28210 conn(0x7f8b740397b0 msgr2=0x7f8b7403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.794+0000 7f8b9802b640 1 --2- 172.21.15.178:0/1806280254 >> v2:172.21.15.204:6812/28210 conn(0x7f8b740397b0 0x7f8b7403bc60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8b880076e0 tx=0x7f8b88009990 comp rx=0 tx=0).stop 2024-09-15T01:47:54.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.794+0000 7f8b9802b640 10 monclient: shutdown 2024-09-15T01:47:54.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.794+0000 7f8b9802b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:54.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.794+0000 7f8b9802b640 1 -- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b9009faa0 msgr2=0x7f8b9009b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.794+0000 7f8b9802b640 1 --2- 172.21.15.178:0/1806280254 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8b9009faa0 0x7f8b9009b280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c000c00 tx=0x7f8b8c032000 comp rx=0 tx=0).stop 2024-09-15T01:47:54.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.794+0000 7f8b965c8640 1 -- 172.21.15.178:0/1806280254 reap_dead start 2024-09-15T01:47:54.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.794+0000 7f8b965c8640 1 -- 172.21.15.178:0/1806280254 reap_dead start 2024-09-15T01:47:54.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.794+0000 7f8b9802b640 1 -- 172.21.15.178:0/1806280254 shutdown_connections 2024-09-15T01:47:54.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.794+0000 7f8b9802b640 1 -- 172.21.15.178:0/1806280254 >> 172.21.15.178:0/1806280254 conn(0x7f8b900a5ed0 msgr2=0x7f8b90135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:54.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.794+0000 7f8b9802b640 1 -- 172.21.15.178:0/1806280254 shutdown_connections 2024-09-15T01:47:54.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.794+0000 7f8b9802b640 1 -- 172.21.15.178:0/1806280254 wait complete. 2024-09-15T01:47:54.850 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:54.850 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:47:54.850 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:47:54.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.941+0000 7f4405f54640 10 monclient: get_monmap_and_config 2024-09-15T01:47:54.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.941+0000 7f4405f54640 10 monclient: build_initial_monmap 2024-09-15T01:47:54.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.941+0000 7f4405f54640 10 monclient: monmap: 2024-09-15T01:47:54.944 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:54.944 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.944 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:54.943150+0000 2024-09-15T01:47:54.944 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:54.943150+0000 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:54.945 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.941+0000 7f4405f54640 1 Processor -- start 2024-09-15T01:47:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 1 -- start start 2024-09-15T01:47:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 10 monclient: init 2024-09-15T01:47:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 10 monclient: _add_conns ranks=[3,5,7,10,9,0,6,4,2,8,1] 2024-09-15T01:47:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44000a2930 0x7f4400132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 10 monclient(hunting): picked mon.noname-g con 0x7f44000a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f440009faa0 0x7f440009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 10 monclient(hunting): picked mon.noname-h con 0x7f440009faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f44000a08c0 0x7f44000a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 10 monclient(hunting): picked mon.noname-i con 0x7f44000a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440009e0f0 con 0x7f44000a2930 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f440009dd80 con 0x7f440009faa0 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f44000a2d00 con 0x7f44000a08c0 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f4405f54640 10 monclient(hunting): authenticate will time out at 1679.693569s 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f43ff7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44000a2930 0x7f4400132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f43fffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f44000a08c0 0x7f44000a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f43feffd640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f440009faa0 0x7f440009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f43fffff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f44000a08c0 0x7f44000a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:41122/0 (socket says 172.21.15.178:41122) 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f43ff7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44000a2930 0x7f4400132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:41466/0 (socket says 172.21.15.178:41466) 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f43fffff640 1 -- 172.21.15.178:0/1707438876 learned_addr learned my addr 172.21.15.178:0/1707438876 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f43fffff640 10 monclient(hunting): get_auth_request con 0x7f44000a08c0 auth_method 0 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f43fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f43fffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.942+0000 7f43fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43ff7fe640 10 monclient(hunting): get_auth_request con 0x7f44000a2930 auth_method 0 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43feffd640 10 monclient(hunting): get_auth_request con 0x7f440009faa0 auth_method 0 2024-09-15T01:47:54.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43feffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 10 monclient(hunting): handle_auth_done global_id 64348 payload 274 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 1 -- 172.21.15.178:0/1707438876 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f440009faa0 msgr2=0x7f440009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 1 --2- 172.21.15.178:0/1707438876 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f440009faa0 0x7f440009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 1 -- 172.21.15.178:0/1707438876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44000a2930 msgr2=0x7f4400132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 1 --2- 172.21.15.178:0/1707438876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44000a2930 0x7f4400132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 10 monclient: _finish_hunting 0 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 1 monclient: found mon.noname-i 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43fffff640 1 -- 172.21.15.178:0/1707438876 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440004bf60 con 0x7f44000a08c0 2024-09-15T01:47:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43ff7fe640 1 --2- 172.21.15.178:0/1707438876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44000a2930 0x7f4400132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.943+0000 7f43feffd640 1 --2- 172.21.15.178:0/1707438876 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f440009faa0 0x7f440009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fffff640 1 --2- 172.21.15.178:0/1707438876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f44000a08c0 0x7f44000a0d30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009f50 tx=0x7f43f8009840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=4aa281c8bb64709c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fe7fc640 1 -- 172.21.15.178:0/1707438876 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43f8007880 con 0x7f44000a08c0 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fe7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fe7fc640 10 monclient: dump: 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.960 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fe7fc640 10 monclient: _finish_auth 0 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fe7fc640 1 -- 172.21.15.178:0/1707438876 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f802d050 con 0x7f44000a08c0 2024-09-15T01:47:54.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fe7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fe7fc640 1 -- 172.21.15.178:0/1707438876 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43f8030360 con 0x7f44000a08c0 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fe7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f43fe7fc640 10 monclient: dump: 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.962 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 5 monclient: authenticate success, global_id 64348 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 4 set_mon_vals no callback set 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 10 monclient: shutdown 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:54.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 1 -- 172.21.15.178:0/1707438876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f44000a08c0 msgr2=0x7f44000a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 1 --2- 172.21.15.178:0/1707438876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f44000a08c0 0x7f44000a0d30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009f50 tx=0x7f43f8009840 comp rx=0 tx=0).stop 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 1 -- 172.21.15.178:0/1707438876 shutdown_connections 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 1 --2- 172.21.15.178:0/1707438876 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f44000a08c0 0x7f44000a0d30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 1 --2- 172.21.15.178:0/1707438876 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f440009faa0 0x7f440009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 1 --2- 172.21.15.178:0/1707438876 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44000a2930 0x7f4400132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 1 -- 172.21.15.178:0/1707438876 >> 172.21.15.178:0/1707438876 conn(0x7f44000a5ed0 msgr2=0x7f44000a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 1 -- 172.21.15.178:0/1707438876 shutdown_connections 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.944+0000 7f4405f54640 1 -- 172.21.15.178:0/1707438876 wait complete. 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.945+0000 7f4405f54640 10 monclient: build_initial_monmap 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.945+0000 7f4405f54640 10 monclient: monmap: 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:54.964 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.945+0000 7f4405f54640 1 Processor -- start 2024-09-15T01:47:54.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.945+0000 7f4405f54640 1 -- start start 2024-09-15T01:47:54.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.945+0000 7f4405f54640 10 monclient: init 2024-09-15T01:47:54.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:54.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 10 monclient: _add_conns ranks=[0,2,5,4,3,1,7,6,8] 2024-09-15T01:47:54.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f440009faa0 0x7f440009b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 10 monclient(hunting): picked mon.noname-a con 0x7f440009faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:47:54.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44000a08c0 0x7f4400131b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 10 monclient(hunting): picked mon.noname-b con 0x7f44000a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:54.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f44000a2930 0x7f44001320a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 10 monclient(hunting): picked mon.noname-h con 0x7f44000a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:54.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440009c100 con 0x7f440009faa0 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44000a2390 con 0x7f44000a08c0 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f440009d630 con 0x7f44000a2930 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f4405f54640 10 monclient(hunting): authenticate will time out at 1679.697299s 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43feffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44000a08c0 0x7f4400131b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f440009faa0 0x7f440009b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43feffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44000a08c0 0x7f4400131b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42088/0 (socket says 172.21.15.178:42088) 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43feffd640 1 -- 172.21.15.178:0/1039150436 learned_addr learned my addr 172.21.15.178:0/1039150436 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 10 monclient(hunting): get_auth_request con 0x7f440009faa0 auth_method 0 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43feffd640 10 monclient(hunting): get_auth_request con 0x7f44000a08c0 auth_method 0 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43feffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43fffff640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f44000a2930 0x7f44001320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43fffff640 10 monclient(hunting): get_auth_request con 0x7f44000a2930 auth_method 0 2024-09-15T01:47:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43fffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 10 monclient(hunting): handle_auth_done global_id 64476 payload 824 2024-09-15T01:47:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 1 -- 172.21.15.178:0/1039150436 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f44000a2930 msgr2=0x7f44001320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f44000a2930 0x7f44001320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 1 -- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44000a08c0 msgr2=0x7f4400131b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44000a08c0 0x7f4400131b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:47:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 1 monclient: found mon.noname-a 2024-09-15T01:47:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:47:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 1 -- 172.21.15.178:0/1039150436 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f0009390 con 0x7f440009faa0 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43feffd640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44000a08c0 0x7f4400131b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43ff7fe640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f440009faa0 0x7f440009b670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f43f400ea80 tx=0x7f43f40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.946+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43f4009670 con 0x7f440009faa0 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43e77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43e77fe640 10 monclient: dump: 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.970 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43e77fe640 10 monclient: _finish_auth 0 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f4405f54640 5 monclient: authenticate success, global_id 64476 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f400ce90 con 0x7f440009faa0 2024-09-15T01:47:54.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f4405f54640 10 monclient: _renew_subs 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f4405f54640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f4405f54640 1 -- 172.21.15.178:0/1039150436 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f4009390 con 0x7f440009faa0 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43e77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43f40138f0 con 0x7f440009faa0 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43e77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43e77fe640 10 monclient: dump: 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.972 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f4405f54640 10 monclient: _renew_subs 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f4405f54640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f4405f54640 1 -- 172.21.15.178:0/1039150436 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f8009ae0 con 0x7f440009faa0 2024-09-15T01:47:54.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43fdffb640 4 set_mon_vals no callback set 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f4405f54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f43f4013470 con 0x7f440009faa0 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f4405f54640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4400142520 0x7f440011ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f4405f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.947+0000 7f4405f54640 1 -- 172.21.15.178:0/1039150436 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440004c4a0 con 0x7f4400142520 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43feffd640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4400142520 0x7f440011ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43feffd640 10 monclient: get_auth_request con 0x7f4400142520 auth_method 0 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43feffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 1 --2- 172.21.15.178:0/1039150436 >> v2:172.21.15.204:6812/28210 conn(0x7f43ec0397e0 0x7f43ec03bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f43f404be70 con 0x7f440009faa0 2024-09-15T01:47:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43feffd640 10 monclient(hunting): handle_auth_done global_id 35255 payload 824 2024-09-15T01:47:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43feffd640 1 -- 172.21.15.178:0/1039150436 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f440005ed70 con 0x7f4400142520 2024-09-15T01:47:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43fffff640 1 --2- 172.21.15.178:0/1039150436 >> v2:172.21.15.204:6812/28210 conn(0x7f43ec0397e0 0x7f43ec03bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43feffd640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4400142520 0x7f440011ec80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f440009b5c0 tx=0x7f43f00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43fffff640 10 monclient: get_auth_request con 0x7f43ec0397e0 auth_method 0 2024-09-15T01:47:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43f0014ba0 con 0x7f4400142520 2024-09-15T01:47:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 10 monclient: dump: 2024-09-15T01:47:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.976 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.977 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.977 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f43f0014d00 con 0x7f4400142520 2024-09-15T01:47:54.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:54.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43fffff640 1 --2- 172.21.15.178:0/1039150436 >> v2:172.21.15.204:6812/28210 conn(0x7f43ec0397e0 0x7f43ec03bc90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f43f8033000 tx=0x7f43f8036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:54.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4400142520 msgr2=0x7f440011ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4400142520 0x7f440011ec80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f440009b5c0 tx=0x7f43f00090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4400142520 msgr2=0x7f440011ec80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:54.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.948+0000 7f43e77fe640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4400142520 0x7f440011ec80 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f440009b5c0 tx=0x7f43f00090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:54.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f4405f54640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:54.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f4405f54640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440011c530 0x7f440011c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:54.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f4405f54640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:54.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f4405f54640 1 -- 172.21.15.178:0/1039150436 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440004bc80 con 0x7f440011c530 2024-09-15T01:47:54.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f43feffd640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440011c530 0x7f440011c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:54.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f43feffd640 10 monclient: get_auth_request con 0x7f440011c530 auth_method 0 2024-09-15T01:47:54.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f43feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:54.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f43feffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:54.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f43feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:54.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f43feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:54.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f43feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:54.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f43feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:54.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f43feffd640 10 monclient(hunting): handle_auth_done global_id 35264 payload 824 2024-09-15T01:47:54.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.954+0000 7f43feffd640 1 -- 172.21.15.178:0/1039150436 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f440009fe70 con 0x7f440011c530 2024-09-15T01:47:54.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43feffd640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440011c530 0x7f440011c930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f43f001b000 tx=0x7f43f001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:54.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43f0024040 con 0x7f440011c530 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43e77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43e77fe640 10 monclient: dump: 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:54.981 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f43f00133a0 con 0x7f440011c530 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43e77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43e77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440011c530 msgr2=0x7f440011c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43e77fe640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440011c530 0x7f440011c930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f43f001b000 tx=0x7f43f001c000 comp rx=0 tx=0).stop 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43e77fe640 1 -- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440011c530 msgr2=0x7f440011c930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:54.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43e77fe640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440011c530 0x7f440011c930 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f43f001b000 tx=0x7f43f001c000 comp rx=0 tx=0).stop 2024-09-15T01:47:54.983 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:54.983 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:54.983 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:54.983 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:54.983 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:54.983 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:54.983 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:54.983 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:54.983 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:54.984 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:54.984 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:54.984 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:54.984 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:54.984 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:54.984 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:54.984 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:54.984 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:47:54.984 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:54.984 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:54.984 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:54.984 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:54.985 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:54.986 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:54.987 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:54.988 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:54.988 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:54.988 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:54.988 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:54.988 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:54.988 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:54.988 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:54.988 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:54.988 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:54.988 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:54.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.989 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:54.989 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.989 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:54.989 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:54.989 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.989 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.989 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.989 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.989 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.990 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.991 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.991 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.991 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:54.991 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:54.991 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.991 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.991 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.991 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.991 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:54.991 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.992 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.993 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.993 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:54.993 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:54.993 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.993 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.993 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.993 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.993 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:54.993 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.995 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.995 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:54.995 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:54.995 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.995 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.995 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.995 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.995 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.996 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.998 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:54.999 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:55.000 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.001 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.002 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.003 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.006 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:55.007 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:55.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f4405f54640 1 -- 172.21.15.178:0/1039150436 >> v2:172.21.15.204:6812/28210 conn(0x7f43ec0397e0 msgr2=0x7f43ec03bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f4405f54640 1 --2- 172.21.15.178:0/1039150436 >> v2:172.21.15.204:6812/28210 conn(0x7f43ec0397e0 0x7f43ec03bc90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f43f8033000 tx=0x7f43f8036000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f4405f54640 10 monclient: shutdown 2024-09-15T01:47:55.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f4405f54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f4405f54640 1 -- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f440009faa0 msgr2=0x7f440009b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f4405f54640 1 --2- 172.21.15.178:0/1039150436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f440009faa0 0x7f440009b670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f43f400ea80 tx=0x7f43f40090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43fffff640 1 -- 172.21.15.178:0/1039150436 reap_dead start 2024-09-15T01:47:55.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f43fffff640 1 -- 172.21.15.178:0/1039150436 reap_dead start 2024-09-15T01:47:55.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f4405f54640 1 -- 172.21.15.178:0/1039150436 shutdown_connections 2024-09-15T01:47:55.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f4405f54640 1 -- 172.21.15.178:0/1039150436 >> 172.21.15.178:0/1039150436 conn(0x7f44000a5ed0 msgr2=0x7f44000a53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f4405f54640 1 -- 172.21.15.178:0/1039150436 shutdown_connections 2024-09-15T01:47:55.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:54.955+0000 7f4405f54640 1 -- 172.21.15.178:0/1039150436 wait complete. 2024-09-15T01:47:55.010 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:55.010 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:47:55.010 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:47:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.100+0000 7fb368ad4640 10 monclient: get_monmap_and_config 2024-09-15T01:47:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.100+0000 7fb368ad4640 10 monclient: build_initial_monmap 2024-09-15T01:47:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.101+0000 7fb368ad4640 10 monclient: monmap: 2024-09-15T01:47:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:55.102884+0000 2024-09-15T01:47:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:55.102884+0000 2024-09-15T01:47:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:55.105 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.101+0000 7fb368ad4640 1 Processor -- start 2024-09-15T01:47:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.101+0000 7fb368ad4640 1 -- start start 2024-09-15T01:47:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.101+0000 7fb368ad4640 10 monclient: init 2024-09-15T01:47:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 10 monclient: _add_conns ranks=[6,10,2,4,3,9,5,0,8,7,1] 2024-09-15T01:47:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb3640a2930 0x7fb364132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 10 monclient(hunting): picked mon.noname-d con 0x7fb3640a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb36409faa0 0x7fb36409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 10 monclient(hunting): picked mon.noname-f con 0x7fb36409faa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3640a08c0 0x7fb3640a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 10 monclient(hunting): picked mon.noname-b con 0x7fb3640a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36409e0f0 con 0x7fb36409faa0 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36409dd80 con 0x7fb3640a08c0 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640a2d00 con 0x7fb3640a2930 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb368ad4640 10 monclient(hunting): authenticate will time out at 1679.853273s 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb36259c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb3640a2930 0x7fb364132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3640a08c0 0x7fb3640a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb36259c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb3640a2930 0x7fb364132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42052/0 (socket says 172.21.15.178:42052) 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb36259c640 1 -- 172.21.15.178:0/4186500407 learned_addr learned my addr 172.21.15.178:0/4186500407 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb36259c640 10 monclient(hunting): get_auth_request con 0x7fb3640a2930 auth_method 0 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb36259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb36259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb36259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 10 monclient(hunting): get_auth_request con 0x7fb3640a08c0 auth_method 0 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb36259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb36259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb36259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb361d9b640 1 --2- 172.21.15.178:0/4186500407 >> v2:172.21.15.204:3300/0 conn(0x7fb36409faa0 0x7fb36409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb361d9b640 10 monclient(hunting): get_auth_request con 0x7fb36409faa0 auth_method 0 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb361d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb361d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb361d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb361d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb361d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb361d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 10 monclient(hunting): handle_auth_done global_id 35273 payload 274 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 1 -- 172.21.15.178:0/4186500407 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb3640a2930 msgr2=0x7fb364132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 1 --2- 172.21.15.178:0/4186500407 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb3640a2930 0x7fb364132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 1 -- 172.21.15.178:0/4186500407 >> v2:172.21.15.204:3300/0 conn(0x7fb36409faa0 msgr2=0x7fb36409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 1 --2- 172.21.15.178:0/4186500407 >> v2:172.21.15.204:3300/0 conn(0x7fb36409faa0 0x7fb36409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 1 monclient: found mon.noname-b 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb362d9d640 1 -- 172.21.15.178:0/4186500407 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36404bf60 con 0x7fb3640a08c0 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.102+0000 7fb36259c640 1 --2- 172.21.15.178:0/4186500407 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb3640a2930 0x7fb364132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb362d9d640 1 --2- 172.21.15.178:0/4186500407 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3640a08c0 0x7fb3640a0d60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb354009850 tx=0x7fb3540090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e86ffbac7e2ed9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb36159a640 1 -- 172.21.15.178:0/4186500407 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb35402abc0 con 0x7fb3640a08c0 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb36159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb36159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb36159a640 10 monclient: dump: 2024-09-15T01:47:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.120 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb36159a640 10 monclient: _finish_auth 0 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb36159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb36159a640 1 -- 172.21.15.178:0/4186500407 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35402ad40 con 0x7fb3640a08c0 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb368ad4640 5 monclient: authenticate success, global_id 35273 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb368ad4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb36159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb368ad4640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb36159a640 1 -- 172.21.15.178:0/4186500407 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb354031ca0 con 0x7fb3640a08c0 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb368ad4640 4 set_mon_vals no callback set 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb36159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb36159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.103+0000 7fb36159a640 10 monclient: dump: 2024-09-15T01:47:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 10 monclient: shutdown 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 1 -- 172.21.15.178:0/4186500407 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3640a08c0 msgr2=0x7fb3640a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 1 --2- 172.21.15.178:0/4186500407 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3640a08c0 0x7fb3640a0d60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb354009850 tx=0x7fb3540090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 1 -- 172.21.15.178:0/4186500407 shutdown_connections 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 1 --2- 172.21.15.178:0/4186500407 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3640a08c0 0x7fb3640a0d60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 1 --2- 172.21.15.178:0/4186500407 >> v2:172.21.15.204:3300/0 conn(0x7fb36409faa0 0x7fb36409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 1 --2- 172.21.15.178:0/4186500407 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb3640a2930 0x7fb364132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 1 -- 172.21.15.178:0/4186500407 >> 172.21.15.178:0/4186500407 conn(0x7fb3640a5ed0 msgr2=0x7fb3640a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 1 -- 172.21.15.178:0/4186500407 shutdown_connections 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 1 -- 172.21.15.178:0/4186500407 wait complete. 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 10 monclient: build_initial_monmap 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.104+0000 7fb368ad4640 10 monclient: monmap: 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:55.124 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 1 Processor -- start 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 1 -- start start 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 10 monclient: init 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 10 monclient: _add_conns ranks=[8,7,2,4,6,0,1,3,5] 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb36409faa0 0x7fb36409aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 10 monclient(hunting): picked mon.noname-e con 0x7fb36409faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:55.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb3640a08c0 0x7fb364131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 10 monclient(hunting): picked mon.noname-i con 0x7fb3640a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3640a2930 0x7fb36411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 10 monclient(hunting): picked mon.noname-b con 0x7fb3640a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36409d630 con 0x7fb3640a2930 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb36411ec40 con 0x7fb3640a08c0 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb364142b10 con 0x7fb36409faa0 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb368ad4640 10 monclient(hunting): authenticate will time out at 1679.856551s 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3640a2930 0x7fb36411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb36259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb36409faa0 0x7fb36409aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3640a2930 0x7fb36411c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42118/0 (socket says 172.21.15.178:42118) 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 1 -- 172.21.15.178:0/1790161429 learned_addr learned my addr 172.21.15.178:0/1790161429 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb36259c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb36409faa0 0x7fb36409aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42164/0 (socket says 172.21.15.178:42164) 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 10 monclient(hunting): get_auth_request con 0x7fb3640a2930 auth_method 0 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb36259c640 10 monclient(hunting): get_auth_request con 0x7fb36409faa0 auth_method 0 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb36259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb36259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb36259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb361d9b640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb3640a08c0 0x7fb364131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb361d9b640 10 monclient(hunting): get_auth_request con 0x7fb3640a08c0 auth_method 0 2024-09-15T01:47:55.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb361d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb361d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb361d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb36259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb36259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb36259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb361d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb361d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb361d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 10 monclient(hunting): handle_auth_done global_id 35282 payload 824 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 1 -- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb36409faa0 msgr2=0x7fb36409aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb36409faa0 0x7fb36409aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 1 -- 172.21.15.178:0/1790161429 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb3640a08c0 msgr2=0x7fb364131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb3640a08c0 0x7fb364131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 10 monclient: _finish_hunting 0 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 1 monclient: found mon.noname-b 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb362d9d640 1 -- 172.21.15.178:0/1790161429 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c009450 con 0x7fb3640a2930 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.105+0000 7fb361d9b640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb3640a08c0 0x7fb364131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb362d9d640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3640a2930 0x7fb36411c530 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb354009360 tx=0x7fb354032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb35403b870 con 0x7fb3640a2930 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb3527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb3527fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:47:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb3527fc640 10 monclient: dump: 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.130 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb3527fc640 10 monclient: _finish_auth 0 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb3527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb368ad4640 5 monclient: authenticate success, global_id 35282 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35403ba10 con 0x7fb3640a2930 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb3527fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb368ad4640 10 monclient: _renew_subs 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3540313a0 con 0x7fb3640a2930 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb368ad4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb368ad4640 1 -- 172.21.15.178:0/1790161429 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb354009390 con 0x7fb3640a2930 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb3527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb3527fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:47:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb360d99640 4 set_mon_vals no callback set 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb3527fc640 10 monclient: dump: 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.132 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb368ad4640 10 monclient: _renew_subs 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb368ad4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.106+0000 7fb368ad4640 1 -- 172.21.15.178:0/1790161429 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36411e960 con 0x7fb3640a2930 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb368ad4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb354039040 con 0x7fb3640a2930 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb368ad4640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364136ff0 0x7fb36411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb368ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb368ad4640 1 -- 172.21.15.178:0/1790161429 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb36411eaf0 con 0x7fb364136ff0 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb36259c640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364136ff0 0x7fb36411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb36259c640 10 monclient: get_auth_request con 0x7fb364136ff0 auth_method 0 2024-09-15T01:47:55.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb36259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb36259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb36259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb36259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb36259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 1 --2- 172.21.15.178:0/1790161429 >> v2:172.21.15.204:6812/28210 conn(0x7fb3440396e0 0x7fb34403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb36259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fb35406b3f0 con 0x7fb3640a2930 2024-09-15T01:47:55.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb36259c640 10 monclient(hunting): handle_auth_done global_id 44815 payload 824 2024-09-15T01:47:55.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb36259c640 1 -- 172.21.15.178:0/1790161429 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb36404c4a0 con 0x7fb364136ff0 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb361d9b640 1 --2- 172.21.15.178:0/1790161429 >> v2:172.21.15.204:6812/28210 conn(0x7fb3440396e0 0x7fb34403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb361d9b640 10 monclient: get_auth_request con 0x7fb3440396e0 auth_method 0 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb36259c640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364136ff0 0x7fb36411ca70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb35c011bf0 tx=0x7fb35c0091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb35c00fdb0 con 0x7fb364136ff0 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 10 monclient: dump: 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.135 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.136 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.136 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.136 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.136 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.136 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.136 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.136 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.136 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.136 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.137 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb361d9b640 1 --2- 172.21.15.178:0/1790161429 >> v2:172.21.15.204:6812/28210 conn(0x7fb3440396e0 0x7fb34403bb90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb3640a6ec0 tx=0x7fb35800b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fb35c01d480 con 0x7fb364136ff0 2024-09-15T01:47:55.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:55.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:55.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364136ff0 msgr2=0x7fb36411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364136ff0 0x7fb36411ca70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb35c011bf0 tx=0x7fb35c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364136ff0 msgr2=0x7fb36411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:55.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.107+0000 7fb3527fc640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364136ff0 0x7fb36411ca70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.113+0000 7fb368ad4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:55.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.113+0000 7fb368ad4640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364131d60 0x7fb36411cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.113+0000 7fb368ad4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.113+0000 7fb368ad4640 1 -- 172.21.15.178:0/1790161429 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb36404bc80 con 0x7fb364131d60 2024-09-15T01:47:55.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.113+0000 7fb36259c640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364131d60 0x7fb36411cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.113+0000 7fb36259c640 10 monclient: get_auth_request con 0x7fb364131d60 auth_method 0 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.113+0000 7fb36259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.113+0000 7fb36259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.113+0000 7fb36259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.113+0000 7fb36259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.113+0000 7fb36259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.113+0000 7fb36259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb36259c640 10 monclient(hunting): handle_auth_done global_id 44824 payload 824 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb36259c640 1 -- 172.21.15.178:0/1790161429 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb36405ed70 con 0x7fb364131d60 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb36259c640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364131d60 0x7fb36411cfb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb35c011bf0 tx=0x7fb35c01d000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb35c025040 con 0x7fb364131d60 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb3527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb3527fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb3527fc640 10 monclient: dump: 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.141 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.142 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.142 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.142 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.142 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9482 (secure 0 0 0) 0x7fb35c01c3e0 con 0x7fb364131d60 2024-09-15T01:47:55.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb3527fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:55.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb3527fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:55.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364131d60 msgr2=0x7fb36411cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb3527fc640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364131d60 0x7fb36411cfb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb35c011bf0 tx=0x7fb35c01d000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb3527fc640 1 -- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364131d60 msgr2=0x7fb36411cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:55.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.114+0000 7fb3527fc640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb364131d60 0x7fb36411cfb0 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb35c011bf0 tx=0x7fb35c01d000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.143 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:55.143 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:55.143 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:55.143 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:55.143 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:55.143 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:55.143 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:55.143 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:55.143 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:55.143 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:55.143 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:55.143 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 3, 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.144 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.145 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:55.146 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.147 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:55.149 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.151 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.152 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:55.154 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:55.157 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.158 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.159 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:55.160 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.161 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.161 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.161 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:55.161 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.161 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.161 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.161 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.161 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:55.162 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.163 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.164 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.165 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.166 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.166 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.166 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.166 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.166 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.166 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:55.166 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:55.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.115+0000 7fb368ad4640 1 -- 172.21.15.178:0/1790161429 >> v2:172.21.15.204:6812/28210 conn(0x7fb3440396e0 msgr2=0x7fb34403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.115+0000 7fb368ad4640 1 --2- 172.21.15.178:0/1790161429 >> v2:172.21.15.204:6812/28210 conn(0x7fb3440396e0 0x7fb34403bb90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb3640a6ec0 tx=0x7fb35800b040 comp rx=0 tx=0).stop 2024-09-15T01:47:55.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.115+0000 7fb368ad4640 10 monclient: shutdown 2024-09-15T01:47:55.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.115+0000 7fb368ad4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.115+0000 7fb368ad4640 1 -- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3640a2930 msgr2=0x7fb36411c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.115+0000 7fb368ad4640 1 --2- 172.21.15.178:0/1790161429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3640a2930 0x7fb36411c530 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb354009360 tx=0x7fb354032000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.115+0000 7fb362d9d640 1 -- 172.21.15.178:0/1790161429 reap_dead start 2024-09-15T01:47:55.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.115+0000 7fb362d9d640 1 -- 172.21.15.178:0/1790161429 reap_dead start 2024-09-15T01:47:55.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.115+0000 7fb368ad4640 1 -- 172.21.15.178:0/1790161429 shutdown_connections 2024-09-15T01:47:55.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.115+0000 7fb368ad4640 1 -- 172.21.15.178:0/1790161429 >> 172.21.15.178:0/1790161429 conn(0x7fb3640a5ed0 msgr2=0x7fb364135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.115+0000 7fb368ad4640 1 -- 172.21.15.178:0/1790161429 shutdown_connections 2024-09-15T01:47:55.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.115+0000 7fb368ad4640 1 -- 172.21.15.178:0/1790161429 wait complete. 2024-09-15T01:47:55.169 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:55.169 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:47:55.169 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:47:55.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.258+0000 7f647e80f640 10 monclient: get_monmap_and_config 2024-09-15T01:47:55.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.258+0000 7f647e80f640 10 monclient: build_initial_monmap 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.258+0000 7f647e80f640 10 monclient: monmap: 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:55.260557+0000 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:55.260557+0000 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:55.262 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:55.263 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:55.263 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:55.263 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:55.263 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:55.263 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:55.263 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.259+0000 7f647e80f640 1 Processor -- start 2024-09-15T01:47:55.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.259+0000 7f647e80f640 1 -- start start 2024-09-15T01:47:55.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.259+0000 7f647e80f640 10 monclient: init 2024-09-15T01:47:55.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.259+0000 7f647e80f640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:55.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.259+0000 7f647e80f640 10 monclient: _add_conns ranks=[7,10,6,4,3,8,1,0,5,9,2] 2024-09-15T01:47:55.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.259+0000 7f647e80f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f64780a2930 0x7f6478132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.259+0000 7f647e80f640 10 monclient(hunting): picked mon.noname-i con 0x7f64780a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:55.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.259+0000 7f647e80f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f647809faa0 0x7f647809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.259+0000 7f647e80f640 10 monclient(hunting): picked mon.noname-f con 0x7f647809faa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.259+0000 7f647e80f640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f64780a08c0 0x7f64780a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647e80f640 10 monclient(hunting): picked mon.noname-d con 0x7f64780a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647e80f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647e80f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647809e0f0 con 0x7f647809faa0 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647e80f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647e80f640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f647809dd80 con 0x7f64780a08c0 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647e80f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647e80f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f64780a2d00 con 0x7f64780a2930 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647e80f640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647e80f640 10 monclient(hunting): authenticate will time out at 1680.011052s 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f64780a08c0 0x7f64780a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f64780a08c0 0x7f64780a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42058/0 (socket says 172.21.15.178:42058) 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 1 -- 172.21.15.178:0/4272988652 learned_addr learned my addr 172.21.15.178:0/4272988652 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 10 monclient(hunting): get_auth_request con 0x7f64780a08c0 auth_method 0 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f6477fff640 1 --2- 172.21.15.178:0/4272988652 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f64780a2930 0x7f6478132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f64777fe640 1 --2- 172.21.15.178:0/4272988652 >> v2:172.21.15.204:3300/0 conn(0x7f647809faa0 0x7f647809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f64780a2930 auth_method 0 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f6477fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f64777fe640 10 monclient(hunting): get_auth_request con 0x7f647809faa0 auth_method 0 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f64777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f64777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f64777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 10 monclient(hunting): handle_auth_done global_id 25710 payload 274 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 1 -- 172.21.15.178:0/4272988652 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f64780a2930 msgr2=0x7f6478132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 1 --2- 172.21.15.178:0/4272988652 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f64780a2930 0x7f6478132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 1 -- 172.21.15.178:0/4272988652 >> v2:172.21.15.204:3300/0 conn(0x7f647809faa0 msgr2=0x7f647809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 1 --2- 172.21.15.178:0/4272988652 >> v2:172.21.15.204:3300/0 conn(0x7f647809faa0 0x7f647809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 10 monclient: _finish_hunting 0 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 1 monclient: found mon.noname-d 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:47:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.260+0000 7f647cdac640 1 -- 172.21.15.178:0/4272988652 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647804bf60 con 0x7f64780a08c0 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647cdac640 1 --2- 172.21.15.178:0/4272988652 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f64780a08c0 0x7f64780a0d60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6470009f20 tx=0x7f6470009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e3fe36317d3abacf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f6476ffd640 1 -- 172.21.15.178:0/4272988652 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6470007880 con 0x7f64780a08c0 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f6476ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f6476ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f6476ffd640 10 monclient: dump: 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f6476ffd640 10 monclient: _finish_auth 0 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f6476ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 5 monclient: authenticate success, global_id 25710 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f6476ffd640 1 -- 172.21.15.178:0/4272988652 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647002d050 con 0x7f64780a08c0 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f6476ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 4 set_mon_vals no callback set 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f6476ffd640 1 -- 172.21.15.178:0/4272988652 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6470030360 con 0x7f64780a08c0 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f6476ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f6476ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f6476ffd640 10 monclient: dump: 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 10 monclient: shutdown 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 1 -- 172.21.15.178:0/4272988652 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f64780a08c0 msgr2=0x7f64780a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 1 --2- 172.21.15.178:0/4272988652 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f64780a08c0 0x7f64780a0d60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6470009f20 tx=0x7f6470009840 comp rx=0 tx=0).stop 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 1 -- 172.21.15.178:0/4272988652 shutdown_connections 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 1 --2- 172.21.15.178:0/4272988652 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f64780a08c0 0x7f64780a0d60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 1 --2- 172.21.15.178:0/4272988652 >> v2:172.21.15.204:3300/0 conn(0x7f647809faa0 0x7f647809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 1 --2- 172.21.15.178:0/4272988652 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f64780a2930 0x7f6478132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.261+0000 7f647e80f640 1 -- 172.21.15.178:0/4272988652 >> 172.21.15.178:0/4272988652 conn(0x7f64780a5ed0 msgr2=0x7f64780a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.262+0000 7f647e80f640 1 -- 172.21.15.178:0/4272988652 shutdown_connections 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.262+0000 7f647e80f640 1 -- 172.21.15.178:0/4272988652 wait complete. 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.262+0000 7f647e80f640 10 monclient: build_initial_monmap 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.262+0000 7f647e80f640 10 monclient: monmap: 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.262+0000 7f647e80f640 1 Processor -- start 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.262+0000 7f647e80f640 1 -- start start 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.262+0000 7f647e80f640 10 monclient: init 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 10 monclient: _add_conns ranks=[7,8,3,5,2,6,4,1,0] 2024-09-15T01:47:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f647809faa0 0x7f647809b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 10 monclient(hunting): picked mon.noname-i con 0x7f647809faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f64780a08c0 0x7f64781318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 10 monclient(hunting): picked mon.noname-e con 0x7f64780a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64780a2930 0x7f6478141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 10 monclient(hunting): picked mon.noname-g con 0x7f64780a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647811e9d0 con 0x7f64780a2930 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f647811ecb0 con 0x7f647809faa0 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f647811ef90 con 0x7f64780a08c0 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647e80f640 10 monclient(hunting): authenticate will time out at 1680.014424s 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f64780a08c0 0x7f64781318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f64780a08c0 0x7f64781318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42172/0 (socket says 172.21.15.178:42172) 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 1 -- 172.21.15.178:0/2330745899 learned_addr learned my addr 172.21.15.178:0/2330745899 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 10 monclient(hunting): get_auth_request con 0x7f64780a08c0 auth_method 0 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f6477fff640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f647809faa0 0x7f647809b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647cdac640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64780a2930 0x7f6478141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f6477fff640 10 monclient(hunting): get_auth_request con 0x7f647809faa0 auth_method 0 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f6477fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647cdac640 10 monclient(hunting): get_auth_request con 0x7f64780a2930 auth_method 0 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647cdac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647cdac640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647cdac640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f6477fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f6477fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f6477fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647cdac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647cdac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f647cdac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 10 monclient(hunting): handle_auth_done global_id 94238 payload 824 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 1 -- 172.21.15.178:0/2330745899 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f647809faa0 msgr2=0x7f647809b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f647809faa0 0x7f647809b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 1 -- 172.21.15.178:0/2330745899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64780a2930 msgr2=0x7f6478141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64780a2930 0x7f6478141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 10 monclient: _finish_hunting 0 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 1 monclient: found mon.noname-e 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.263+0000 7f64777fe640 1 -- 172.21.15.178:0/2330745899 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6470009ae0 con 0x7f64780a08c0 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f64777fe640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f64780a08c0 0x7f64781318b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f646c00d890 tx=0x7f646c008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f646c00de20 con 0x7f64780a08c0 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:55.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 10 monclient: dump: 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 10 monclient: _finish_auth 0 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646c009d30 con 0x7f64780a08c0 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f646c009ea0 con 0x7f64780a08c0 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f647e80f640 5 monclient: authenticate success, global_id 94238 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f647e80f640 10 monclient: _renew_subs 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f647e80f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f647e80f640 1 -- 172.21.15.178:0/2330745899 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647811f270 con 0x7f64780a08c0 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f64767fc640 4 set_mon_vals no callback set 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 10 monclient: dump: 2024-09-15T01:47:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.290 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.290 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.290 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.290 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.290 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.290 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f647e80f640 10 monclient: _renew_subs 2024-09-15T01:47:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f647e80f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f647e80f640 1 -- 172.21.15.178:0/2330745899 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647811dd00 con 0x7f64780a08c0 2024-09-15T01:47:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f646c0098b0 con 0x7f64780a08c0 2024-09-15T01:47:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.264+0000 7f647e80f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f647e80f640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f647811ca40 0x7f6478142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f647e80f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f647e80f640 1 -- 172.21.15.178:0/2330745899 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f647811ee20 con 0x7f647811ca40 2024-09-15T01:47:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f6477fff640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f647811ca40 0x7f6478142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 1 --2- 172.21.15.178:0/2330745899 >> v2:172.21.15.204:6812/28210 conn(0x7f64640396e0 0x7f646403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f6477fff640 10 monclient: get_auth_request con 0x7f647811ca40 auth_method 0 2024-09-15T01:47:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f6477fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f646c04c0f0 con 0x7f64780a08c0 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f6477fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f6477fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f6477fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f647cdac640 1 --2- 172.21.15.178:0/2330745899 >> v2:172.21.15.204:6812/28210 conn(0x7f64640396e0 0x7f646403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f647cdac640 10 monclient: get_auth_request con 0x7f64640396e0 auth_method 0 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f6477fff640 10 monclient(hunting): handle_auth_done global_id 25719 payload 824 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f6477fff640 1 -- 172.21.15.178:0/2330745899 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f647804c4a0 con 0x7f647811ca40 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f6477fff640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f647811ca40 0x7f6478142200 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f64680108b0 tx=0x7f6468008040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6468014430 con 0x7f647811ca40 2024-09-15T01:47:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:47:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 10 monclient: dump: 2024-09-15T01:47:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.296 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.296 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f647cdac640 1 --2- 172.21.15.178:0/2330745899 >> v2:172.21.15.204:6812/28210 conn(0x7f64640396e0 0x7f646403bb90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6470033000 tx=0x7f6470036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f646800eb40 con 0x7f647811ca40 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f647811ca40 msgr2=0x7f6478142200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f647811ca40 0x7f6478142200 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f64680108b0 tx=0x7f6468008040 comp rx=0 tx=0).stop 2024-09-15T01:47:55.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f647811ca40 msgr2=0x7f6478142200 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:55.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.265+0000 7f645ffff640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f647811ca40 0x7f6478142200 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f647e80f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:55.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f647e80f640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6478131df0 0x7f6478142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f647e80f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f647e80f640 1 -- 172.21.15.178:0/2330745899 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f647809bcf0 con 0x7f6478131df0 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f6477fff640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6478131df0 0x7f6478142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f6477fff640 10 monclient: get_auth_request con 0x7f6478131df0 auth_method 0 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f6477fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f6477fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f6477fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f6477fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f6477fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f6477fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f6477fff640 10 monclient(hunting): handle_auth_done global_id 25728 payload 824 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.271+0000 7f6477fff640 1 -- 172.21.15.178:0/2330745899 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f647804bc80 con 0x7f6478131df0 2024-09-15T01:47:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f6477fff640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6478131df0 0x7f6478142740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f64680108b0 tx=0x7f646801c000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f64680133d0 con 0x7f6478131df0 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f645ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f645ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f645ffff640 10 monclient: dump: 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f6468013530 con 0x7f6478131df0 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f645ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f645ffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6478131df0 msgr2=0x7f6478142740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f645ffff640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6478131df0 0x7f6478142740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f64680108b0 tx=0x7f646801c000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f645ffff640 1 -- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6478131df0 msgr2=0x7f6478142740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f645ffff640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f6478131df0 0x7f6478142740 secure :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f64680108b0 tx=0x7f646801c000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.303 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:55.303 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:55.303 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:55.303 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:55.303 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:55.303 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:55.303 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:55.303 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:55.303 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:55.303 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:55.303 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.304 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.305 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:55.306 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.307 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:55.309 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:55.310 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.312 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:55.313 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.315 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:55.316 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.316 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.316 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:55.316 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.316 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.316 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.316 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:55.316 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.318 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.318 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.318 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:55.318 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.318 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.318 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.318 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.318 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.318 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.318 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.318 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.319 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:55.320 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.322 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.323 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:55.324 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.325 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.325 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.325 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.325 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.325 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.325 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.325 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:55.325 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:55.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f647e80f640 1 -- 172.21.15.178:0/2330745899 >> v2:172.21.15.204:6812/28210 conn(0x7f64640396e0 msgr2=0x7f646403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f647e80f640 1 --2- 172.21.15.178:0/2330745899 >> v2:172.21.15.204:6812/28210 conn(0x7f64640396e0 0x7f646403bb90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6470033000 tx=0x7f6470036000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f647e80f640 10 monclient: shutdown 2024-09-15T01:47:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f647e80f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f647e80f640 1 -- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f64780a08c0 msgr2=0x7f64781318b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f647e80f640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f64780a08c0 0x7f64781318b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f646c00d890 tx=0x7f646c008040 comp rx=0 tx=0).stop 2024-09-15T01:47:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f647cdac640 1 -- 172.21.15.178:0/2330745899 reap_dead start 2024-09-15T01:47:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f647e80f640 1 -- 172.21.15.178:0/2330745899 shutdown_connections 2024-09-15T01:47:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f647e80f640 1 --2- 172.21.15.178:0/2330745899 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f64780a08c0 0x7f64781318b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.272+0000 7f647e80f640 1 -- 172.21.15.178:0/2330745899 >> 172.21.15.178:0/2330745899 conn(0x7f64780a5ed0 msgr2=0x7f647809e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.273+0000 7f647e80f640 1 -- 172.21.15.178:0/2330745899 shutdown_connections 2024-09-15T01:47:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.273+0000 7f647e80f640 1 -- 172.21.15.178:0/2330745899 wait complete. 2024-09-15T01:47:55.328 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:55.328 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:47:55.328 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:47:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.417+0000 7fa551b5d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.417+0000 7fa551b5d640 10 monclient: build_initial_monmap 2024-09-15T01:47:55.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.417+0000 7fa551b5d640 10 monclient: monmap: 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:55.419490+0000 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:55.419490+0000 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:55.421 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:55.422 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:55.422 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 1 Processor -- start 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 1 -- start start 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 10 monclient: init 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 10 monclient: _add_conns ranks=[0,2,9,4,5,1,3,10,6,7,8] 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa54c0a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa54c132de0 0x7fa54c137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 10 monclient(hunting): picked mon.noname-b con 0x7fa54c132de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa54c1380e0 0x7fa54c13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 10 monclient(hunting): picked mon.noname-a con 0x7fa54c1380e0 addr v2:172.21.15.178:3300/0 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54c04bf60 con 0x7fa54c0a2930 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54c09dd80 con 0x7fa54c1380e0 2024-09-15T01:47:55.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c0a2d00 con 0x7fa54c132de0 2024-09-15T01:47:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.418+0000 7fa551b5d640 10 monclient(hunting): authenticate will time out at 1680.169905s 2024-09-15T01:47:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54affd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa54c132de0 0x7fa54c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa54c1380e0 0x7fa54c13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54b7fe640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa54c0a2930 0x7fa54c09faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:36832/0 (socket says 172.21.15.178:36832) 2024-09-15T01:47:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54b7fe640 1 -- 172.21.15.178:0/841758981 learned_addr learned my addr 172.21.15.178:0/841758981 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:47:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54affd640 10 monclient(hunting): get_auth_request con 0x7fa54c132de0 auth_method 0 2024-09-15T01:47:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 10 monclient(hunting): get_auth_request con 0x7fa54c1380e0 auth_method 0 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 10 monclient(hunting): handle_auth_done global_id 64485 payload 274 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 1 -- 172.21.15.178:0/841758981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa54c132de0 msgr2=0x7fa54c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 1 --2- 172.21.15.178:0/841758981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa54c132de0 0x7fa54c137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 1 -- 172.21.15.178:0/841758981 >> v1:172.21.15.178:6789/0 conn(0x7fa54c0a2930 legacy=0x7fa54c09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 1 monclient: found mon.noname-a 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54bfff640 1 -- 172.21.15.178:0/841758981 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c1331b0 con 0x7fa54c1380e0 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54a7fc640 1 -- 172.21.15.178:0/841758981 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331246926 0 0) 0x7fa54c04bf60 con 0x7fa54c0a2930 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54a7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.419+0000 7fa54affd640 1 --2- 172.21.15.178:0/841758981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa54c132de0 0x7fa54c137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54bfff640 1 --2- 172.21.15.178:0/841758981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa54c1380e0 0x7fa54c13a4c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa544009e60 tx=0x7fa544009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e709ba70c33b3524 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54a7fc640 1 -- 172.21.15.178:0/841758981 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5440079d0 con 0x7fa54c1380e0 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54a7fc640 10 monclient: dump: 2024-09-15T01:47:55.434 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.435 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.436 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.436 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.436 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.436 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.436 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.436 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:47:55.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:55.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54a7fc640 1 -- 172.21.15.178:0/841758981 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54402d050 con 0x7fa54c1380e0 2024-09-15T01:47:55.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 5 monclient: authenticate success, global_id 64485 2024-09-15T01:47:55.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:55.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:55.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:55.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54a7fc640 1 -- 172.21.15.178:0/841758981 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa544007b90 con 0x7fa54c1380e0 2024-09-15T01:47:55.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 4 set_mon_vals no callback set 2024-09-15T01:47:55.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:55.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa54a7fc640 10 monclient: dump: 2024-09-15T01:47:55.437 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.437 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.438 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 10 monclient: shutdown 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 1 -- 172.21.15.178:0/841758981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa54c1380e0 msgr2=0x7fa54c13a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 1 --2- 172.21.15.178:0/841758981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa54c1380e0 0x7fa54c13a4c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa544009e60 tx=0x7fa544009780 comp rx=0 tx=0).stop 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 1 -- 172.21.15.178:0/841758981 shutdown_connections 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 1 --2- 172.21.15.178:0/841758981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa54c1380e0 0x7fa54c13a4c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 1 --2- 172.21.15.178:0/841758981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa54c132de0 0x7fa54c137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 1 -- 172.21.15.178:0/841758981 >> 172.21.15.178:0/841758981 conn(0x7fa54c0a5ed0 msgr2=0x7fa54c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 1 -- 172.21.15.178:0/841758981 shutdown_connections 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.420+0000 7fa551b5d640 1 -- 172.21.15.178:0/841758981 wait complete. 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.421+0000 7fa551b5d640 10 monclient: build_initial_monmap 2024-09-15T01:47:55.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.421+0000 7fa551b5d640 10 monclient: monmap: 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:55.440 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.421+0000 7fa551b5d640 1 Processor -- start 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.421+0000 7fa551b5d640 1 -- start start 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.421+0000 7fa551b5d640 10 monclient: init 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 10 monclient: _add_conns ranks=[0,5,6,2,4,1,8,3,7] 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa54c0a2930 0x7fa54c1324c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 10 monclient(hunting): picked mon.noname-a con 0x7fa54c0a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa54c132de0 0x7fa54c09a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 10 monclient(hunting): picked mon.noname-h con 0x7fa54c132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:55.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa54c1380e0 0x7fa54c09ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 10 monclient(hunting): picked mon.noname-d con 0x7fa54c1380e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c09d630 con 0x7fa54c0a2930 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c09f870 con 0x7fa54c132de0 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c11d9a0 con 0x7fa54c1380e0 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa551b5d640 10 monclient(hunting): authenticate will time out at 1680.173381s 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa54c0a2930 0x7fa54c1324c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa54c0a2930 0x7fa54c1324c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35908/0 (socket says 172.21.15.178:35908) 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 1 -- 172.21.15.178:0/2729540293 learned_addr learned my addr 172.21.15.178:0/2729540293 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 10 monclient(hunting): get_auth_request con 0x7fa54c0a2930 auth_method 0 2024-09-15T01:47:55.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54bfff640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa54c1380e0 0x7fa54c09ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54bfff640 10 monclient(hunting): get_auth_request con 0x7fa54c1380e0 auth_method 0 2024-09-15T01:47:55.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 10 monclient(hunting): handle_auth_done global_id 64503 payload 824 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54affd640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa54c132de0 0x7fa54c09a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 1 -- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa54c1380e0 msgr2=0x7fa54c09ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa54c1380e0 0x7fa54c09ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 1 -- 172.21.15.178:0/2729540293 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa54c132de0 msgr2=0x7fa54c09a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa54c132de0 0x7fa54c09a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:47:55.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 1 monclient: found mon.noname-a 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 1 -- 172.21.15.178:0/2729540293 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa540009390 con 0x7fa54c0a2930 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54affd640 10 monclient: get_auth_request con 0x7fa54c132de0 auth_method 0 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54affd640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa54c132de0 0x7fa54c09a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa54b7fe640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa54c0a2930 0x7fa54c1324c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00c510 tx=0x7fa53c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa53c0070d0 con 0x7fa54c0a2930 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa5337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa5337fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.422+0000 7fa5337fe640 10 monclient: dump: 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.445 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.446 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa5337fe640 10 monclient: _finish_auth 0 2024-09-15T01:47:55.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa5337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:55.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53c007250 con 0x7fa54c0a2930 2024-09-15T01:47:55.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa551b5d640 5 monclient: authenticate success, global_id 64503 2024-09-15T01:47:55.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa551b5d640 10 monclient: _renew_subs 2024-09-15T01:47:55.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa551b5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:47:55.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa551b5d640 1 -- 172.21.15.178:0/2729540293 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa544009a20 con 0x7fa54c0a2930 2024-09-15T01:47:55.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa5337fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:55.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa53c011360 con 0x7fa54c0a2930 2024-09-15T01:47:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa5337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa5337fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa549ffb640 4 set_mon_vals no callback set 2024-09-15T01:47:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa5337fe640 10 monclient: dump: 2024-09-15T01:47:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.449 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.449 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.449 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.449 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.449 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.449 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.449 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.449 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa551b5d640 10 monclient: _renew_subs 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa551b5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa551b5d640 1 -- 172.21.15.178:0/2729540293 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54c11d3e0 con 0x7fa54c0a2930 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa551b5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.423+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa53c0073c0 con 0x7fa54c0a2930 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa551b5d640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c11c530 0x7fa54c09b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa551b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa551b5d640 1 -- 172.21.15.178:0/2729540293 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c11d5d0 con 0x7fa54c11c530 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54affd640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c11c530 0x7fa54c09b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54affd640 10 monclient: get_auth_request con 0x7fa54c11c530 auth_method 0 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 1 --2- 172.21.15.178:0/2729540293 >> v2:172.21.15.204:6812/28210 conn(0x7fa5380397e0 0x7fa53803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fa53c04ac40 con 0x7fa54c0a2930 2024-09-15T01:47:55.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54affd640 10 monclient(hunting): handle_auth_done global_id 94247 payload 824 2024-09-15T01:47:55.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54affd640 1 -- 172.21.15.178:0/2729540293 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa54c04c4a0 con 0x7fa54c11c530 2024-09-15T01:47:55.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54bfff640 1 --2- 172.21.15.178:0/2729540293 >> v2:172.21.15.204:6812/28210 conn(0x7fa5380397e0 0x7fa53803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54bfff640 10 monclient: get_auth_request con 0x7fa5380397e0 auth_method 0 2024-09-15T01:47:55.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54affd640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c11c530 0x7fa54c09b040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa540010b20 tx=0x7fa540008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa54001c400 con 0x7fa54c11c530 2024-09-15T01:47:55.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:55.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 10 monclient: dump: 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.452 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.453 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.453 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.453 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fa54000eae0 con 0x7fa54c11c530 2024-09-15T01:47:55.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:55.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:55.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c11c530 msgr2=0x7fa54c09b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c11c530 0x7fa54c09b040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa540010b20 tx=0x7fa540008040 comp rx=0 tx=0).stop 2024-09-15T01:47:55.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa54bfff640 1 --2- 172.21.15.178:0/2729540293 >> v2:172.21.15.204:6812/28210 conn(0x7fa5380397e0 0x7fa53803bc90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa544034000 tx=0x7fa544037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c11c530 msgr2=0x7fa54c09b040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:55.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.424+0000 7fa5337fe640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c11c530 0x7fa54c09b040 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa540010b20 tx=0x7fa540008040 comp rx=0 tx=0).stop 2024-09-15T01:47:55.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa551b5d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:55.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa551b5d640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c096a30 0x7fa54c096e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa551b5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa551b5d640 1 -- 172.21.15.178:0/2729540293 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c04bc80 con 0x7fa54c096a30 2024-09-15T01:47:55.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa54affd640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c096a30 0x7fa54c096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa54affd640 10 monclient: get_auth_request con 0x7fa54c096a30 auth_method 0 2024-09-15T01:47:55.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa54affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa54affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa54affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa54affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa54affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa54affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa54affd640 10 monclient(hunting): handle_auth_done global_id 94256 payload 824 2024-09-15T01:47:55.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.430+0000 7fa54affd640 1 -- 172.21.15.178:0/2729540293 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa54c05ed70 con 0x7fa54c096a30 2024-09-15T01:47:55.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa54affd640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c096a30 0x7fa54c096e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa540009930 tx=0x7fa54001c000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5400278c0 con 0x7fa54c096a30 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa5337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa5337fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa5337fe640 10 monclient: dump: 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.457 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fa540027440 con 0x7fa54c096a30 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa5337fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa5337fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c096a30 msgr2=0x7fa54c096e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa5337fe640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c096a30 0x7fa54c096e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa540009930 tx=0x7fa54001c000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa5337fe640 1 -- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c096a30 msgr2=0x7fa54c096e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:55.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa5337fe640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa54c096a30 0x7fa54c096e30 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa540009930 tx=0x7fa54001c000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.459 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:55.459 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:47:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.460 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.461 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:55.462 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.463 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.464 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.465 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:55.467 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:55.468 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.470 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:55.471 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.472 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.473 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.474 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:55.477 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:55.478 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.479 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.480 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.481 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.481 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.481 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:55.481 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.481 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.481 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.481 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.481 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.481 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.481 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:55.481 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:55.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa551b5d640 1 -- 172.21.15.178:0/2729540293 >> v2:172.21.15.204:6812/28210 conn(0x7fa5380397e0 msgr2=0x7fa53803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa551b5d640 1 --2- 172.21.15.178:0/2729540293 >> v2:172.21.15.204:6812/28210 conn(0x7fa5380397e0 0x7fa53803bc90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa544034000 tx=0x7fa544037000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa551b5d640 10 monclient: shutdown 2024-09-15T01:47:55.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa551b5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa551b5d640 1 -- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa54c0a2930 msgr2=0x7fa54c1324c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa551b5d640 1 --2- 172.21.15.178:0/2729540293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa54c0a2930 0x7fa54c1324c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00c510 tx=0x7fa53c007f30 comp rx=0 tx=0).stop 2024-09-15T01:47:55.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa54bfff640 1 -- 172.21.15.178:0/2729540293 reap_dead start 2024-09-15T01:47:55.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa54bfff640 1 -- 172.21.15.178:0/2729540293 reap_dead start 2024-09-15T01:47:55.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa551b5d640 1 -- 172.21.15.178:0/2729540293 shutdown_connections 2024-09-15T01:47:55.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa551b5d640 1 -- 172.21.15.178:0/2729540293 >> 172.21.15.178:0/2729540293 conn(0x7fa54c0a5ed0 msgr2=0x7fa54c0a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa551b5d640 1 -- 172.21.15.178:0/2729540293 shutdown_connections 2024-09-15T01:47:55.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.431+0000 7fa551b5d640 1 -- 172.21.15.178:0/2729540293 wait complete. 2024-09-15T01:47:55.484 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:55.484 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:47:55.485 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:47:55.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.575+0000 7f16ff59d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.575+0000 7f16ff59d640 10 monclient: build_initial_monmap 2024-09-15T01:47:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.575+0000 7f16ff59d640 10 monclient: monmap: 2024-09-15T01:47:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:55.577247+0000 2024-09-15T01:47:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:55.577247+0000 2024-09-15T01:47:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.579 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:55.580 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.575+0000 7f16ff59d640 1 Processor -- start 2024-09-15T01:47:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 1 -- start start 2024-09-15T01:47:55.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 10 monclient: init 2024-09-15T01:47:55.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 10 monclient: _add_conns ranks=[1,0,7,10,9,3,8,2,6,5,4] 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f16f80a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f16f8132de0 addr v1:172.21.15.178:6789/0 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f16f81392c0 0x7f16f813b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 10 monclient(hunting): picked mon.noname-i con 0x7f16f81392c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f804bf60 con 0x7f16f8132de0 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f809dd80 con 0x7f16f80a2930 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f16f80a2d00 con 0x7f16f81392c0 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16ff59d640 10 monclient(hunting): authenticate will time out at 1680.327601s 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16fcb38640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f16f8132de0 0x7f16f8137ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:36848/0 (socket says 172.21.15.178:36848) 2024-09-15T01:47:55.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16fcb38640 1 -- 172.21.15.178:0/1029829803 learned_addr learned my addr 172.21.15.178:0/1029829803 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:47:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16fdb3a640 1 --2- 172.21.15.178:0/1029829803 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f16f81392c0 0x7f16f813b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16fdb3a640 10 monclient(hunting): get_auth_request con 0x7f16f81392c0 auth_method 0 2024-09-15T01:47:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16fdb3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16fdb3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16fdb3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16fdb3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16fdb3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16fdb3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16f7fff640 1 -- 172.21.15.178:0/1029829803 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059880723 0 0) 0x7f16f804bf60 con 0x7f16f8132de0 2024-09-15T01:47:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16f7fff640 10 monclient(hunting): my global_id is 64512 2024-09-15T01:47:55.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.576+0000 7f16f7fff640 1 -- 172.21.15.178:0/1029829803 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16e00034e0 con 0x7f16f8132de0 2024-09-15T01:47:55.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16f7fff640 1 -- 172.21.15.178:0/1029829803 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357086719 0 0) 0x7f16f809dd80 con 0x7f16f80a2930 2024-09-15T01:47:55.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16f7fff640 10 monclient(hunting): my global_id is 94222 2024-09-15T01:47:55.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16f7fff640 1 -- 172.21.15.178:0/1029829803 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16f804bf60 con 0x7f16f80a2930 2024-09-15T01:47:55.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16fdb3a640 10 monclient(hunting): handle_auth_done global_id 64384 payload 274 2024-09-15T01:47:55.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16fdb3a640 1 -- 172.21.15.178:0/1029829803 >> v1:172.21.15.204:6789/0 conn(0x7f16f80a2930 legacy=0x7f16f809faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16fdb3a640 1 -- 172.21.15.178:0/1029829803 >> v1:172.21.15.178:6789/0 conn(0x7f16f8132de0 legacy=0x7f16f8137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16fdb3a640 10 monclient: _finish_hunting 0 2024-09-15T01:47:55.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16fdb3a640 1 monclient: found mon.noname-i 2024-09-15T01:47:55.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16fdb3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:55.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16fdb3a640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:47:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16fdb3a640 1 -- 172.21.15.178:0/1029829803 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f81331b0 con 0x7f16f81392c0 2024-09-15T01:47:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16fdb3a640 1 --2- 172.21.15.178:0/1029829803 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f16f81392c0 0x7f16f813b6a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f16f809df10 tx=0x7f16e8009780 comp rx=0 tx=0).ready entity=mon.7 client_cookie=3d2037b8a7d4887a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.577+0000 7f16f7fff640 1 -- 172.21.15.178:0/1029829803 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f16e802aea0 con 0x7f16f81392c0 2024-09-15T01:47:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16f7fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:47:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16f7fff640 10 monclient: dump: 2024-09-15T01:47:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.599 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.600 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16f7fff640 10 monclient: _finish_auth 0 2024-09-15T01:47:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16f7fff640 1 -- 172.21.15.178:0/1029829803 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e802d050 con 0x7f16f81392c0 2024-09-15T01:47:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16ff59d640 5 monclient: authenticate success, global_id 64384 2024-09-15T01:47:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16f7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16f7fff640 1 -- 172.21.15.178:0/1029829803 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f16e8007e50 con 0x7f16f81392c0 2024-09-15T01:47:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16f7fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:47:55.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16f7fff640 10 monclient: dump: 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.602 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.603 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.603 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16ff59d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:55.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16ff59d640 4 set_mon_vals no callback set 2024-09-15T01:47:55.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16ff59d640 10 monclient: shutdown 2024-09-15T01:47:55.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16ff59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16ff59d640 1 -- 172.21.15.178:0/1029829803 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f16f81392c0 msgr2=0x7f16f813b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16ff59d640 1 --2- 172.21.15.178:0/1029829803 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f16f81392c0 0x7f16f813b6a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f16f809df10 tx=0x7f16e8009780 comp rx=0 tx=0).stop 2024-09-15T01:47:55.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16ff59d640 1 -- 172.21.15.178:0/1029829803 shutdown_connections 2024-09-15T01:47:55.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16ff59d640 1 --2- 172.21.15.178:0/1029829803 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f16f81392c0 0x7f16f813b6a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16ff59d640 1 -- 172.21.15.178:0/1029829803 >> 172.21.15.178:0/1029829803 conn(0x7f16f80a5ed0 msgr2=0x7f16f80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16ff59d640 1 -- 172.21.15.178:0/1029829803 shutdown_connections 2024-09-15T01:47:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.578+0000 7f16ff59d640 1 -- 172.21.15.178:0/1029829803 wait complete. 2024-09-15T01:47:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.579+0000 7f16ff59d640 10 monclient: build_initial_monmap 2024-09-15T01:47:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.579+0000 7f16ff59d640 10 monclient: monmap: 2024-09-15T01:47:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:55.604 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.579+0000 7f16ff59d640 1 Processor -- start 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.579+0000 7f16ff59d640 1 -- start start 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.579+0000 7f16ff59d640 10 monclient: init 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.579+0000 7f16ff59d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:55.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.579+0000 7f16ff59d640 10 monclient: _add_conns ranks=[8,7,2,6,3,5,1,0,4] 2024-09-15T01:47:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.579+0000 7f16ff59d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f16f80a2930 0x7f16f8138380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.579+0000 7f16ff59d640 10 monclient(hunting): picked mon.noname-e con 0x7f16f80a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f16f8132de0 0x7f16f8141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 10 monclient(hunting): picked mon.noname-i con 0x7f16f8132de0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16f81392c0 0x7f16f8141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 10 monclient(hunting): picked mon.noname-b con 0x7f16f81392c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:47:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f81363d0 con 0x7f16f81392c0 2024-09-15T01:47:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f16f80a2390 con 0x7f16f8132de0 2024-09-15T01:47:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f16f809d630 con 0x7f16f80a2930 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 10 monclient(hunting): authenticate will time out at 1680.331054s 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f16f80a2930 0x7f16f8138380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fdb3a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16f81392c0 0x7f16f8141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f16f80a2930 0x7f16f8138380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42212/0 (socket says 172.21.15.178:42212) 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 1 -- 172.21.15.178:0/1648800943 learned_addr learned my addr 172.21.15.178:0/1648800943 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fdb3a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16f81392c0 0x7f16f8141fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42134/0 (socket says 172.21.15.178:42134) 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 10 monclient(hunting): get_auth_request con 0x7f16f80a2930 auth_method 0 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fdb3a640 10 monclient(hunting): get_auth_request con 0x7f16f81392c0 auth_method 0 2024-09-15T01:47:55.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fdb3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fdb3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fdb3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fdb3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fcb38640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f16f8132de0 0x7f16f8141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fdb3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fdb3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fcb38640 10 monclient(hunting): get_auth_request con 0x7f16f8132de0 auth_method 0 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fcb38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fcb38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fcb38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fcb38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fcb38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fcb38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 10 monclient(hunting): handle_auth_done global_id 94265 payload 824 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 1 -- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f16f8132de0 msgr2=0x7f16f8141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f16f8132de0 0x7f16f8141a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 1 -- 172.21.15.178:0/1648800943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16f81392c0 msgr2=0x7f16f8141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16f81392c0 0x7f16f8141fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 10 monclient: _finish_hunting 0 2024-09-15T01:47:55.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 1 monclient: found mon.noname-e 2024-09-15T01:47:55.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:55.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:55.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 1 -- 172.21.15.178:0/1648800943 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e8009a20 con 0x7f16f80a2930 2024-09-15T01:47:55.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fdb3a640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16f81392c0 0x7f16f8141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:55.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16fd339640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f16f80a2930 0x7f16f8138380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f16f000c490 tx=0x7f16f0007d30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f16f0007da0 con 0x7f16f80a2930 2024-09-15T01:47:55.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f57fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:55.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f57fa640 10 monclient: dump: 2024-09-15T01:47:55.611 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.611 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.611 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.611 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.612 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.612 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.612 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.612 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.612 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.612 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.612 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.612 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.612 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.612 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.613 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.613 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f57fa640 10 monclient: _finish_auth 0 2024-09-15T01:47:55.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:55.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f0011a10 con 0x7f16f80a2930 2024-09-15T01:47:55.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 5 monclient: authenticate success, global_id 94265 2024-09-15T01:47:55.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 10 monclient: _renew_subs 2024-09-15T01:47:55.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:55.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 1 -- 172.21.15.178:0/1648800943 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f8137bd0 con 0x7f16f80a2930 2024-09-15T01:47:55.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:55.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f16f0012420 con 0x7f16f80a2930 2024-09-15T01:47:55.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f57fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:55.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f57fa640 10 monclient: dump: 2024-09-15T01:47:55.614 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.614 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.614 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.614 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.614 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16f77fe640 4 set_mon_vals no callback set 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 10 monclient: _renew_subs 2024-09-15T01:47:55.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:55.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.580+0000 7f16ff59d640 1 -- 172.21.15.178:0/1648800943 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f8135800 con 0x7f16f80a2930 2024-09-15T01:47:55.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f16f0011b80 con 0x7f16f80a2930 2024-09-15T01:47:55.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16ff59d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:55.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16ff59d640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f8142510 0x7f16f812a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16ff59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16ff59d640 1 -- 172.21.15.178:0/1648800943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f806f660 con 0x7f16f8142510 2024-09-15T01:47:55.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16f57fa640 1 --2- 172.21.15.178:0/1648800943 >> v2:172.21.15.204:6812/28210 conn(0x7f16e40397b0 0x7f16e403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f16f004b780 con 0x7f16f80a2930 2024-09-15T01:47:55.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16fcb38640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f8142510 0x7f16f812a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16fcb38640 10 monclient: get_auth_request con 0x7f16f8142510 auth_method 0 2024-09-15T01:47:55.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16fcb38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16fcb38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16fcb38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16fdb3a640 1 --2- 172.21.15.178:0/1648800943 >> v2:172.21.15.204:6812/28210 conn(0x7f16e40397b0 0x7f16e403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16fdb3a640 10 monclient: get_auth_request con 0x7f16e40397b0 auth_method 0 2024-09-15T01:47:55.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16fcb38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16fcb38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.581+0000 7f16fcb38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16fcb38640 10 monclient(hunting): handle_auth_done global_id 94231 payload 824 2024-09-15T01:47:55.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16fcb38640 1 -- 172.21.15.178:0/1648800943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16f804c4a0 con 0x7f16f8142510 2024-09-15T01:47:55.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16fdb3a640 1 --2- 172.21.15.178:0/1648800943 >> v2:172.21.15.204:6812/28210 conn(0x7f16e40397b0 0x7f16e403bc60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f16e8034000 tx=0x7f16e8037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16fcb38640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f8142510 0x7f16f812a610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00f780 tx=0x7f16ec009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f16ec005cf0 con 0x7f16f8142510 2024-09-15T01:47:55.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16f57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16f57fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:47:55.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16f57fa640 10 monclient: dump: 2024-09-15T01:47:55.618 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.618 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.618 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.618 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.618 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.619 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.619 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.619 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.619 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.619 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.619 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.620 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.620 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.620 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.620 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.620 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f16ec005e50 con 0x7f16f8142510 2024-09-15T01:47:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16f57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16f57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f8142510 msgr2=0x7f16f812a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16f57fa640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f8142510 0x7f16f812a610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00f780 tx=0x7f16ec009040 comp rx=0 tx=0).stop 2024-09-15T01:47:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f8142510 msgr2=0x7f16f812a610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.582+0000 7f16f57fa640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f8142510 0x7f16f812a610 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00f780 tx=0x7f16ec009040 comp rx=0 tx=0).stop 2024-09-15T01:47:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16ff59d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:55.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16ff59d640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f812ab50 0x7f16f812af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16ff59d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16ff59d640 1 -- 172.21.15.178:0/1648800943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f804bc80 con 0x7f16f812ab50 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16fcb38640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f812ab50 0x7f16f812af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16fcb38640 10 monclient: get_auth_request con 0x7f16f812ab50 auth_method 0 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16fcb38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16fcb38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16fcb38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16fcb38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16fcb38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16fcb38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16fcb38640 10 monclient(hunting): handle_auth_done global_id 94240 payload 824 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.588+0000 7f16fcb38640 1 -- 172.21.15.178:0/1648800943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f16f805ed70 con 0x7f16f812ab50 2024-09-15T01:47:55.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16fcb38640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f812ab50 0x7f16f812af50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f16ec001500 tx=0x7f16ec015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f16ec029410 con 0x7f16f812ab50 2024-09-15T01:47:55.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16f57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16f57fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:47:55.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16f57fa640 10 monclient: dump: 2024-09-15T01:47:55.623 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.623 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.623 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.623 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.623 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.624 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.624 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.624 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.624 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.624 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.624 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.624 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.624 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.624 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.624 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.624 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f16ec0073b0 con 0x7f16f812ab50 2024-09-15T01:47:55.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16f57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:55.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16f57fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:55.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f812ab50 msgr2=0x7f16f812af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16f57fa640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f812ab50 0x7f16f812af50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f16ec001500 tx=0x7f16ec015000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16f57fa640 1 -- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f812ab50 msgr2=0x7f16f812af50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:55.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16f57fa640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16f812ab50 0x7f16f812af50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.626 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:55.626 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:55.626 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:55.626 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:55.626 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:55.626 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:55.626 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:55.626 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:55.626 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.627 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.628 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.629 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.629 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.629 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.629 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.629 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.629 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.629 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.629 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.629 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:55.629 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:55.630 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:55.630 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:55.630 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:55.630 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:55.630 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:55.630 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:55.630 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:55.630 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:55.630 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:55.630 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:55.631 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:55.631 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:55.631 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:55.631 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:55.631 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:55.631 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.631 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.631 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.631 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.632 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.632 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.632 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.632 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.632 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.632 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.632 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:55.632 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.632 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.633 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:55.634 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.634 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.634 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.634 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.634 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:55.634 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:55.634 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.634 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.634 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.634 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.634 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.634 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:55.635 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:55.635 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.635 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.635 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.635 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.635 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:55.635 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.635 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.635 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:55.636 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.636 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.636 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.636 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.636 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:55.636 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:55.636 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.636 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.636 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.636 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:55.637 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:55.638 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.639 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.639 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.639 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:55.639 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.639 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.639 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:55.639 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.639 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.639 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.640 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:55.640 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:55.640 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.640 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.640 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.640 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.640 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:55.640 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:55.640 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.641 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.641 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.641 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.641 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:55.641 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.641 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.641 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.641 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.641 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:55.641 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.641 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.641 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.642 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:55.642 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:55.642 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.642 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.642 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.642 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.643 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:55.643 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:55.643 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.643 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.643 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.643 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.643 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:55.643 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.643 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.643 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.643 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:55.644 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.644 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.644 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.644 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.644 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.644 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.644 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.644 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.645 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.645 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.645 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:55.645 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:55.645 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.645 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.645 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.645 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:55.645 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.645 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.646 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.647 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.648 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.648 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.648 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.648 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.648 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.648 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.648 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.650 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.651 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.652 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:55.652 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:55.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16ff59d640 1 -- 172.21.15.178:0/1648800943 >> v2:172.21.15.204:6812/28210 conn(0x7f16e40397b0 msgr2=0x7f16e403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16ff59d640 1 --2- 172.21.15.178:0/1648800943 >> v2:172.21.15.204:6812/28210 conn(0x7f16e40397b0 0x7f16e403bc60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f16e8034000 tx=0x7f16e8037000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16ff59d640 10 monclient: shutdown 2024-09-15T01:47:55.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16ff59d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16ff59d640 1 -- 172.21.15.178:0/1648800943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f16f80a2930 msgr2=0x7f16f8138380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16ff59d640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f16f80a2930 0x7f16f8138380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f16f000c490 tx=0x7f16f0007d30 comp rx=0 tx=0).stop 2024-09-15T01:47:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.589+0000 7f16fdb3a640 1 -- 172.21.15.178:0/1648800943 reap_dead start 2024-09-15T01:47:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.590+0000 7f16ff59d640 1 -- 172.21.15.178:0/1648800943 shutdown_connections 2024-09-15T01:47:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.590+0000 7f16ff59d640 1 --2- 172.21.15.178:0/1648800943 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f16f80a2930 0x7f16f8138380 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.590+0000 7f16ff59d640 1 -- 172.21.15.178:0/1648800943 >> 172.21.15.178:0/1648800943 conn(0x7f16f80a5ed0 msgr2=0x7f16f812ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.590+0000 7f16ff59d640 1 -- 172.21.15.178:0/1648800943 shutdown_connections 2024-09-15T01:47:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.590+0000 7f16ff59d640 1 -- 172.21.15.178:0/1648800943 wait complete. 2024-09-15T01:47:55.655 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:55.655 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:47:55.655 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:47:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.745+0000 7f867a71d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.745+0000 7f867a71d640 10 monclient: build_initial_monmap 2024-09-15T01:47:55.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.745+0000 7f867a71d640 10 monclient: monmap: 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:55.746929+0000 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:55.746929+0000 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:55.749 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:55.750 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:55.750 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:55.750 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:55.750 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:55.750 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.745+0000 7f867a71d640 1 Processor -- start 2024-09-15T01:47:55.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.745+0000 7f867a71d640 1 -- start start 2024-09-15T01:47:55.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.745+0000 7f867a71d640 10 monclient: init 2024-09-15T01:47:55.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:55.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 10 monclient: _add_conns ranks=[8,7,3,5,0,4,9,1,10,2,6] 2024-09-15T01:47:55.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86740a2930 0x7f8674132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 10 monclient(hunting): picked mon.noname-e con 0x7f86740a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:55.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f867409faa0 0x7f867409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 10 monclient(hunting): picked mon.noname-i con 0x7f867409faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:55.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86740a08c0 0x7f86740a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 10 monclient(hunting): picked mon.noname-g con 0x7f86740a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:55.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867409e0f0 con 0x7f86740a08c0 2024-09-15T01:47:55.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f867409dd80 con 0x7f867409faa0 2024-09-15T01:47:55.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f86740a2d00 con 0x7f86740a2930 2024-09-15T01:47:55.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:55.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f867a71d640 10 monclient(hunting): authenticate will time out at 1680.497494s 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86740a2930 0x7f8674132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86740a2930 0x7f8674132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42226/0 (socket says 172.21.15.178:42226) 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 1 -- 172.21.15.178:0/1111793832 learned_addr learned my addr 172.21.15.178:0/1111793832 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 10 monclient(hunting): get_auth_request con 0x7f86740a2930 auth_method 0 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f866bfff640 1 --2- 172.21.15.178:0/1111793832 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f867409faa0 0x7f867409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f866bfff640 10 monclient(hunting): get_auth_request con 0x7f867409faa0 auth_method 0 2024-09-15T01:47:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f866bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f866bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f866bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8678cba640 1 --2- 172.21.15.178:0/1111793832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86740a08c0 0x7f86740a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8678cba640 10 monclient(hunting): get_auth_request con 0x7f86740a08c0 auth_method 0 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8678cba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8678cba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8678cba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f866bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f866bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f866bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 10 monclient(hunting): handle_auth_done global_id 94274 payload 274 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 1 -- 172.21.15.178:0/1111793832 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f867409faa0 msgr2=0x7f867409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 1 --2- 172.21.15.178:0/1111793832 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f867409faa0 0x7f867409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 1 -- 172.21.15.178:0/1111793832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86740a08c0 msgr2=0x7f86740a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 1 --2- 172.21.15.178:0/1111793832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86740a08c0 0x7f86740a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 10 monclient: _finish_hunting 0 2024-09-15T01:47:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 1 monclient: found mon.noname-e 2024-09-15T01:47:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8673fff640 1 -- 172.21.15.178:0/1111793832 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867404bf60 con 0x7f86740a2930 2024-09-15T01:47:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.746+0000 7f8678cba640 1 --2- 172.21.15.178:0/1111793832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86740a08c0 0x7f86740a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:47:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f8673fff640 1 --2- 172.21.15.178:0/1111793832 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86740a2930 0x7f8674132de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8664009940 tx=0x7f86640091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e4bfee17d6549611 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f86737fe640 1 -- 172.21.15.178:0/1111793832 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f866402ac60 con 0x7f86740a2930 2024-09-15T01:47:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f86737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f86737fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f86737fe640 10 monclient: dump: 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.766 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f86737fe640 10 monclient: _finish_auth 0 2024-09-15T01:47:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f86737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f86737fe640 1 -- 172.21.15.178:0/1111793832 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866402ade0 con 0x7f86740a2930 2024-09-15T01:47:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f867a71d640 5 monclient: authenticate success, global_id 94274 2024-09-15T01:47:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f867a71d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f86737fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f867a71d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f86737fe640 1 -- 172.21.15.178:0/1111793832 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8664031ca0 con 0x7f86740a2930 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f867a71d640 4 set_mon_vals no callback set 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f86737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f86737fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.747+0000 7f86737fe640 10 monclient: dump: 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.768 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 10 monclient: shutdown 2024-09-15T01:47:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 1 -- 172.21.15.178:0/1111793832 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86740a2930 msgr2=0x7f8674132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 1 --2- 172.21.15.178:0/1111793832 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86740a2930 0x7f8674132de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8664009940 tx=0x7f86640091d0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 1 -- 172.21.15.178:0/1111793832 shutdown_connections 2024-09-15T01:47:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 1 --2- 172.21.15.178:0/1111793832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86740a08c0 0x7f86740a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 1 --2- 172.21.15.178:0/1111793832 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f867409faa0 0x7f867409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 1 --2- 172.21.15.178:0/1111793832 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f86740a2930 0x7f8674132de0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 1 -- 172.21.15.178:0/1111793832 >> 172.21.15.178:0/1111793832 conn(0x7f86740a5ed0 msgr2=0x7f86740a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 1 -- 172.21.15.178:0/1111793832 shutdown_connections 2024-09-15T01:47:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 1 -- 172.21.15.178:0/1111793832 wait complete. 2024-09-15T01:47:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 10 monclient: build_initial_monmap 2024-09-15T01:47:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.748+0000 7f867a71d640 10 monclient: monmap: 2024-09-15T01:47:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 1 Processor -- start 2024-09-15T01:47:55.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 1 -- start start 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 10 monclient: init 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 10 monclient: _add_conns ranks=[4,6,5,7,8,1,2,0,3] 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f867409faa0 0x7f867409b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 10 monclient(hunting): picked mon.noname-c con 0x7f867409faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f86740a08c0 0x7f86741318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 10 monclient(hunting): picked mon.noname-d con 0x7f86740a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f86740a2930 0x7f867411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 10 monclient(hunting): picked mon.noname-h con 0x7f86740a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f867409d630 con 0x7f867409faa0 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f867411ef90 con 0x7f86740a2930 2024-09-15T01:47:55.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8674142b90 con 0x7f86740a08c0 2024-09-15T01:47:55.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:55.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f867a71d640 10 monclient(hunting): authenticate will time out at 1680.500794s 2024-09-15T01:47:55.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f8673fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f867409faa0 0x7f867409b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f866bfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f86740a08c0 0x7f86741318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f8673fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f867409faa0 0x7f867409b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:46638/0 (socket says 172.21.15.178:46638) 2024-09-15T01:47:55.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f8673fff640 1 -- 172.21.15.178:0/186310149 learned_addr learned my addr 172.21.15.178:0/186310149 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:55.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f866bfff640 10 monclient(hunting): get_auth_request con 0x7f86740a08c0 auth_method 0 2024-09-15T01:47:55.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f866bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f866bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f866bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f8673fff640 10 monclient(hunting): get_auth_request con 0x7f867409faa0 auth_method 0 2024-09-15T01:47:55.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f8673fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f8673fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f8673fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f8678cba640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f86740a2930 0x7f867411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f8678cba640 10 monclient(hunting): get_auth_request con 0x7f86740a2930 auth_method 0 2024-09-15T01:47:55.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f8678cba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f8678cba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.749+0000 7f8678cba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8673fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8673fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8673fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8678cba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8678cba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8678cba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 10 monclient(hunting): handle_auth_done global_id 25746 payload 824 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 1 -- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f86740a2930 msgr2=0x7f867411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f86740a2930 0x7f867411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 1 -- 172.21.15.178:0/186310149 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f867409faa0 msgr2=0x7f867409b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f867409faa0 0x7f867409b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 1 monclient: found mon.noname-d 2024-09-15T01:47:55.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:55.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:47:55.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 1 -- 172.21.15.178:0/186310149 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8664009450 con 0x7f86740a08c0 2024-09-15T01:47:55.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8673fff640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f867409faa0 0x7f867409b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:55.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8678cba640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f86740a2930 0x7f867411c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:55.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f866bfff640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f86740a08c0 0x7f86741318b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f865c00e900 tx=0x7f865c0090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f865c00ee90 con 0x7f86740a08c0 2024-09-15T01:47:55.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8670ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8670ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:47:55.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8670ff9640 10 monclient: dump: 2024-09-15T01:47:55.775 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.775 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.776 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.776 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.776 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.776 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.776 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.776 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.776 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.776 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.777 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.777 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.777 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.777 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.777 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.777 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8670ff9640 10 monclient: _finish_auth 0 2024-09-15T01:47:55.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8670ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:55.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c00cdc0 con 0x7f86740a08c0 2024-09-15T01:47:55.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f867a71d640 5 monclient: authenticate success, global_id 25746 2024-09-15T01:47:55.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8670ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:55.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f865c0138f0 con 0x7f86740a08c0 2024-09-15T01:47:55.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f867a71d640 10 monclient: _renew_subs 2024-09-15T01:47:55.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f867a71d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:47:55.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f867a71d640 1 -- 172.21.15.178:0/186310149 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867411ed10 con 0x7f86740a08c0 2024-09-15T01:47:55.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8670ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8670ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:47:55.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8672ffd640 4 set_mon_vals no callback set 2024-09-15T01:47:55.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f8670ff9640 10 monclient: dump: 2024-09-15T01:47:55.778 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.778 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.778 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.778 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f867a71d640 10 monclient: _renew_subs 2024-09-15T01:47:55.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f867a71d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:47:55.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.750+0000 7f867a71d640 1 -- 172.21.15.178:0/186310149 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8674142330 con 0x7f86740a08c0 2024-09-15T01:47:55.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f867a71d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:55.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f867a71d640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f867411da70 0x7f867411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f867a71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f867a71d640 1 -- 172.21.15.178:0/186310149 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867411f120 con 0x7f867411da70 2024-09-15T01:47:55.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f865c013470 con 0x7f86740a08c0 2024-09-15T01:47:55.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8673fff640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f867411da70 0x7f867411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8673fff640 10 monclient: get_auth_request con 0x7f867411da70 auth_method 0 2024-09-15T01:47:55.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8673fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8673fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8673fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8670ff9640 1 --2- 172.21.15.178:0/186310149 >> v2:172.21.15.204:6812/28210 conn(0x7f8654039710 0x7f865403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8673fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8673fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8673fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f865c04bb20 con 0x7f86740a08c0 2024-09-15T01:47:55.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8678cba640 1 --2- 172.21.15.178:0/186310149 >> v2:172.21.15.204:6812/28210 conn(0x7f8654039710 0x7f865403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8678cba640 10 monclient: get_auth_request con 0x7f8654039710 auth_method 0 2024-09-15T01:47:55.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8673fff640 10 monclient(hunting): handle_auth_done global_id 64425 payload 824 2024-09-15T01:47:55.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.751+0000 7f8673fff640 1 -- 172.21.15.178:0/186310149 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f867404c4a0 con 0x7f867411da70 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8673fff640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f867411da70 0x7f867411ca70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8664035040 tx=0x7f8664032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8678cba640 1 --2- 172.21.15.178:0/186310149 >> v2:172.21.15.204:6812/28210 conn(0x7f8654039710 0x7f865403bbc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f867409b0d0 tx=0x7f866c0099b0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f866402ac60 con 0x7f867411da70 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8670ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8670ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8670ff9640 10 monclient: dump: 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.782 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f8664033e30 con 0x7f867411da70 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8670ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8670ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f867411da70 msgr2=0x7f867411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8670ff9640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f867411da70 0x7f867411ca70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8664035040 tx=0x7f8664032000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f867411da70 msgr2=0x7f867411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:55.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.752+0000 7f8670ff9640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f867411da70 0x7f867411ca70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f867a71d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f867a71d640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8674131df0 0x7f867411cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f867a71d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f867a71d640 1 -- 172.21.15.178:0/186310149 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867404bc80 con 0x7f8674131df0 2024-09-15T01:47:55.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f8673fff640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8674131df0 0x7f867411cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f8673fff640 10 monclient: get_auth_request con 0x7f8674131df0 auth_method 0 2024-09-15T01:47:55.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f8673fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f8673fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f8673fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f8673fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f8673fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f8673fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f8673fff640 10 monclient(hunting): handle_auth_done global_id 64434 payload 824 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f8673fff640 1 -- 172.21.15.178:0/186310149 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f867405ed70 con 0x7f8674131df0 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.758+0000 7f8673fff640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8674131df0 0x7f867411cfb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8664043000 tx=0x7f866403d000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f866403cba0 con 0x7f8674131df0 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8670ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8670ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8670ff9640 10 monclient: dump: 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.788 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.789 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f866403cd00 con 0x7f8674131df0 2024-09-15T01:47:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8670ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8670ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8674131df0 msgr2=0x7f867411cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8670ff9640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8674131df0 0x7f867411cfb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8664043000 tx=0x7f866403d000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8670ff9640 1 -- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8674131df0 msgr2=0x7f867411cfb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:55.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8670ff9640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8674131df0 0x7f867411cfb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.790 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:55.790 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:55.790 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:55.791 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.792 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:47:55.792 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:55.792 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:55.792 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:55.792 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.792 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.792 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.792 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.792 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.792 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.792 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.792 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.793 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:55.794 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.795 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.796 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.796 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:55.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.796 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:55.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.796 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:55.796 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:55.796 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.796 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.796 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.796 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.797 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:55.800 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:55.801 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.803 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.804 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:55.804 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.804 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.804 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:55.804 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.804 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.804 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.805 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.806 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.807 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.807 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.807 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.807 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.807 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:55.807 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:55.807 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.807 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.807 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.810 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.811 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:55.812 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.814 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:55.814 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:55.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f867a71d640 1 -- 172.21.15.178:0/186310149 >> v2:172.21.15.204:6812/28210 conn(0x7f8654039710 msgr2=0x7f865403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f867a71d640 1 --2- 172.21.15.178:0/186310149 >> v2:172.21.15.204:6812/28210 conn(0x7f8654039710 0x7f865403bbc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f867409b0d0 tx=0x7f866c0099b0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f867a71d640 10 monclient: shutdown 2024-09-15T01:47:55.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f867a71d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f867a71d640 1 -- 172.21.15.178:0/186310149 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f86740a08c0 msgr2=0x7f86741318b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f867a71d640 1 --2- 172.21.15.178:0/186310149 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f86740a08c0 0x7f86741318b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f865c00e900 tx=0x7f865c0090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8678cba640 1 -- 172.21.15.178:0/186310149 reap_dead start 2024-09-15T01:47:55.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f8678cba640 1 -- 172.21.15.178:0/186310149 reap_dead start 2024-09-15T01:47:55.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f867a71d640 1 -- 172.21.15.178:0/186310149 shutdown_connections 2024-09-15T01:47:55.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f867a71d640 1 -- 172.21.15.178:0/186310149 >> 172.21.15.178:0/186310149 conn(0x7f86740a5ed0 msgr2=0x7f86740a5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f867a71d640 1 -- 172.21.15.178:0/186310149 shutdown_connections 2024-09-15T01:47:55.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.759+0000 7f867a71d640 1 -- 172.21.15.178:0/186310149 wait complete. 2024-09-15T01:47:55.816 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:55.817 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:47:55.817 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:47:55.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.907+0000 7f352359d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.907+0000 7f352359d640 10 monclient: build_initial_monmap 2024-09-15T01:47:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.907+0000 7f352359d640 10 monclient: monmap: 2024-09-15T01:47:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:55.909622+0000 2024-09-15T01:47:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:55.909622+0000 2024-09-15T01:47:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:55.911 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:55.912 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:55.912 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:55.912 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:55.912 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:55.912 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:55.912 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:55.912 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:55.912 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.908+0000 7f352359d640 1 Processor -- start 2024-09-15T01:47:55.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.908+0000 7f352359d640 1 -- start start 2024-09-15T01:47:55.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.908+0000 7f352359d640 10 monclient: init 2024-09-15T01:47:55.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.908+0000 7f352359d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:55.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f352359d640 10 monclient: _add_conns ranks=[4,1,0,7,2,9,3,10,8,5,6] 2024-09-15T01:47:55.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f352359d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f351c0a2930 0x7f351c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f352359d640 10 monclient(hunting): picked mon.noname-c con 0x7f351c0a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:47:55.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f352359d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f351c09faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:47:55.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f352359d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f351c09fe70 addr v1:172.21.15.178:6789/0 2024-09-15T01:47:55.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f352359d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c04bf60 con 0x7f351c09fe70 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f352359d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c09dd80 con 0x7f351c09faa0 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f352359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f352359d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0a2d00 con 0x7f351c0a2930 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f352359d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f352359d640 10 monclient(hunting): authenticate will time out at 1680.660082s 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f351c0a2930 0x7f351c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521b3a640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f351c09fe70 0x7f351c0a0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:36850/0 (socket says 172.21.15.178:36850) 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521b3a640 1 -- 172.21.15.178:0/4201402305 learned_addr learned my addr 172.21.15.178:0/4201402305 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 10 monclient(hunting): get_auth_request con 0x7f351c0a2930 auth_method 0 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 10 monclient(hunting): handle_auth_done global_id 44842 payload 274 2024-09-15T01:47:55.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3513fff640 1 -- 172.21.15.178:0/4201402305 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454923007 0 0) 0x7f351c04bf60 con 0x7f351c09fe70 2024-09-15T01:47:55.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 1 -- 172.21.15.178:0/4201402305 >> v1:172.21.15.204:6789/0 conn(0x7f351c09faa0 legacy=0x7f351c0a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 1 -- 172.21.15.178:0/4201402305 >> v1:172.21.15.178:6789/0 conn(0x7f351c09fe70 legacy=0x7f351c0a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 10 monclient: _finish_hunting 0 2024-09-15T01:47:55.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 1 monclient: found mon.noname-c 2024-09-15T01:47:55.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:55.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:47:55.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3521339640 1 -- 172.21.15.178:0/4201402305 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c0a16e0 con 0x7f351c0a2930 2024-09-15T01:47:55.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.909+0000 7f3513fff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:47:55.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3521339640 1 --2- 172.21.15.178:0/4201402305 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f351c0a2930 0x7f351c132de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3518009880 tx=0x7f35180090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f9806725d3ea150c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3513fff640 1 -- 172.21.15.178:0/4201402305 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f351802ace0 con 0x7f351c0a2930 2024-09-15T01:47:55.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3513fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3513fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:47:55.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3513fff640 10 monclient: dump: 2024-09-15T01:47:55.925 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.925 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.925 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.925 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.926 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.926 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.926 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.926 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.926 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.926 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.926 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.926 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.926 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.926 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.926 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.927 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3513fff640 10 monclient: _finish_auth 0 2024-09-15T01:47:55.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3513fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:55.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 5 monclient: authenticate success, global_id 44842 2024-09-15T01:47:55.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3513fff640 1 -- 172.21.15.178:0/4201402305 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351802ae60 con 0x7f351c0a2930 2024-09-15T01:47:55.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:55.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3513fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:55.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:55.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 4 set_mon_vals no callback set 2024-09-15T01:47:55.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3513fff640 1 -- 172.21.15.178:0/4201402305 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3518031ca0 con 0x7f351c0a2930 2024-09-15T01:47:55.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3513fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3513fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:47:55.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f3513fff640 10 monclient: dump: 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.929 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.930 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.930 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.930 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 10 monclient: shutdown 2024-09-15T01:47:55.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 1 -- 172.21.15.178:0/4201402305 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f351c0a2930 msgr2=0x7f351c132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 1 --2- 172.21.15.178:0/4201402305 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f351c0a2930 0x7f351c132de0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3518009880 tx=0x7f35180090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 1 -- 172.21.15.178:0/4201402305 shutdown_connections 2024-09-15T01:47:55.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 1 --2- 172.21.15.178:0/4201402305 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f351c0a2930 0x7f351c132de0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 1 -- 172.21.15.178:0/4201402305 >> 172.21.15.178:0/4201402305 conn(0x7f351c0a5ed0 msgr2=0x7f351c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 1 -- 172.21.15.178:0/4201402305 shutdown_connections 2024-09-15T01:47:55.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.910+0000 7f352359d640 1 -- 172.21.15.178:0/4201402305 wait complete. 2024-09-15T01:47:55.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.911+0000 7f352359d640 10 monclient: build_initial_monmap 2024-09-15T01:47:55.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.911+0000 7f352359d640 10 monclient: monmap: 2024-09-15T01:47:55.931 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:55.931 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:55.931 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:55.931 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:55.931 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:55.931 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:55.931 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:55.931 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:55.931 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:55.931 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.911+0000 7f352359d640 1 Processor -- start 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.911+0000 7f352359d640 1 -- start start 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.911+0000 7f352359d640 10 monclient: init 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 10 monclient: _add_conns ranks=[7,8,4,1,6,5,0,2,3] 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f351c09faa0 0x7f351c09b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 10 monclient(hunting): picked mon.noname-i con 0x7f351c09faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:55.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f351c09fe70 0x7f351c131900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 10 monclient(hunting): picked mon.noname-e con 0x7f351c09fe70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f351c0a2930 0x7f351c141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 10 monclient(hunting): picked mon.noname-c con 0x7f351c0a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f351c11dac0 con 0x7f351c0a2930 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f351c11ea20 con 0x7f351c09faa0 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f351c11ed00 con 0x7f351c09fe70 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f352359d640 10 monclient(hunting): authenticate will time out at 1680.663462s 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f351c0a2930 0x7f351c141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f351c0a2930 0x7f351c141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:46658/0 (socket says 172.21.15.178:46658) 2024-09-15T01:47:55.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 1 -- 172.21.15.178:0/2209377822 learned_addr learned my addr 172.21.15.178:0/2209377822 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:55.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 10 monclient(hunting): get_auth_request con 0x7f351c0a2930 auth_method 0 2024-09-15T01:47:55.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3520b38640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f351c09fe70 0x7f351c131900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3520b38640 10 monclient(hunting): get_auth_request con 0x7f351c09fe70 auth_method 0 2024-09-15T01:47:55.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3520b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3520b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3520b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521339640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f351c09faa0 0x7f351c09b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3520b38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3520b38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3520b38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521339640 10 monclient(hunting): get_auth_request con 0x7f351c09faa0 auth_method 0 2024-09-15T01:47:55.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 10 monclient(hunting): handle_auth_done global_id 44851 payload 824 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 1 -- 172.21.15.178:0/2209377822 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f351c09fe70 msgr2=0x7f351c131900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f351c09fe70 0x7f351c131900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 1 -- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f351c09faa0 msgr2=0x7f351c09b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f351c09faa0 0x7f351c09b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 10 monclient: _finish_hunting 0 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 1 monclient: found mon.noname-c 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:55.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.912+0000 7f3521b3a640 1 -- 172.21.15.178:0/2209377822 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518009390 con 0x7f351c0a2930 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f3521b3a640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f351c0a2930 0x7f351c141f20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f351400cc90 tx=0x7f3514001a20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f351400acd0 con 0x7f351c0a2930 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35117fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35117fa640 10 monclient: dump: 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.937 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35117fa640 10 monclient: _finish_auth 0 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351400ae50 con 0x7f351c0a2930 2024-09-15T01:47:55.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f352359d640 5 monclient: authenticate success, global_id 44851 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f352359d640 10 monclient: _renew_subs 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f352359d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f352359d640 1 -- 172.21.15.178:0/2209377822 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351c11efe0 con 0x7f351c0a2930 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35117fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3514007570 con 0x7f351c0a2930 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35117fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35117fa640 10 monclient: dump: 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.939 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f35137fe640 4 set_mon_vals no callback set 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f352359d640 10 monclient: _renew_subs 2024-09-15T01:47:55.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f352359d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.913+0000 7f352359d640 1 -- 172.21.15.178:0/2209377822 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c11d560 con 0x7f351c0a2930 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f352359d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f35140070f0 con 0x7f351c0a2930 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f352359d640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c131e40 0x7f351c142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f352359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f352359d640 1 -- 172.21.15.178:0/2209377822 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f351c132210 con 0x7f351c131e40 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3521339640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c131e40 0x7f351c142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f35117fa640 1 --2- 172.21.15.178:0/2209377822 >> v2:172.21.15.204:6812/28210 conn(0x7f35000397e0 0x7f350003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3521339640 10 monclient: get_auth_request con 0x7f351c131e40 auth_method 0 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3521339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3521339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3521339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f351404be40 con 0x7f351c0a2930 2024-09-15T01:47:55.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3521339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3521339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3521339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3520b38640 1 --2- 172.21.15.178:0/2209377822 >> v2:172.21.15.204:6812/28210 conn(0x7f35000397e0 0x7f350003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3520b38640 10 monclient: get_auth_request con 0x7f35000397e0 auth_method 0 2024-09-15T01:47:55.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3521339640 10 monclient(hunting): handle_auth_done global_id 7691 payload 824 2024-09-15T01:47:55.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3521339640 1 -- 172.21.15.178:0/2209377822 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f351c09f910 con 0x7f351c131e40 2024-09-15T01:47:55.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3520b38640 1 --2- 172.21.15.178:0/2209377822 >> v2:172.21.15.204:6812/28210 conn(0x7f35000397e0 0x7f350003bc90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f351c09b6d0 tx=0x7f350c009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f3521339640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c131e40 0x7f351c142460 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f351c0a5e70 tx=0x7f3518032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3518032e20 con 0x7f351c131e40 2024-09-15T01:47:55.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f35117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f35117fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:47:55.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.914+0000 7f35117fa640 10 monclient: dump: 2024-09-15T01:47:55.943 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.943 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.943 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.943 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.943 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.943 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.943 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.943 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.943 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.943 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.943 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.943 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.944 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.944 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.944 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.944 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.915+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f3518031a40 con 0x7f351c131e40 2024-09-15T01:47:55.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.915+0000 7f35117fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:55.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.915+0000 7f35117fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:55.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.915+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c131e40 msgr2=0x7f351c142460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.915+0000 7f35117fa640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c131e40 0x7f351c142460 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f351c0a5e70 tx=0x7f3518032000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.915+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c131e40 msgr2=0x7f351c142460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:55.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.915+0000 7f35117fa640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c131e40 0x7f351c142460 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:55.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.920+0000 7f352359d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:55.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.920+0000 7f352359d640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c1429a0 0x7f351c096f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:55.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.920+0000 7f352359d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:55.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.920+0000 7f352359d640 1 -- 172.21.15.178:0/2209377822 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f351c09c2b0 con 0x7f351c1429a0 2024-09-15T01:47:55.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f3521339640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c1429a0 0x7f351c096f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:55.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f3521339640 10 monclient: get_auth_request con 0x7f351c1429a0 auth_method 0 2024-09-15T01:47:55.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f3521339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:55.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f3521339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:55.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f3521339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:55.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f3521339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:55.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f3521339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:55.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f3521339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:55.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f3521339640 10 monclient(hunting): handle_auth_done global_id 7700 payload 824 2024-09-15T01:47:55.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f3521339640 1 -- 172.21.15.178:0/2209377822 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f351c04bc80 con 0x7f351c1429a0 2024-09-15T01:47:55.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f3521339640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c1429a0 0x7f351c096f40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3518009720 tx=0x7f351803d000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:55.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f351803c8c0 con 0x7f351c1429a0 2024-09-15T01:47:55.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f35117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:55.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f35117fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:47:55.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.921+0000 7f35117fa640 10 monclient: dump: 2024-09-15T01:47:55.950 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:55.950 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:55.950 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.950 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:55.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9314 (secure 0 0 0) 0x7f351803c440 con 0x7f351c1429a0 2024-09-15T01:47:55.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f35117fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:55.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f35117fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:55.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c1429a0 msgr2=0x7f351c096f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f35117fa640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c1429a0 0x7f351c096f40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3518009720 tx=0x7f351803d000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f35117fa640 1 -- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c1429a0 msgr2=0x7f351c096f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:55.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f35117fa640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f351c1429a0 0x7f351c096f40 secure :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3518009720 tx=0x7f351803d000 comp rx=0 tx=0).stop 2024-09-15T01:47:55.952 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:55.952 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.954 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.955 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.956 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.956 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.956 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:55.956 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:55.956 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:55.956 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:55.956 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:55.956 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:55.956 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:55.956 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:55.956 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:55.956 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:55.957 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:55.958 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:55.958 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:55.958 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.958 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:55.958 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.958 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:55.958 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.958 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:55.958 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:55.958 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.958 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.958 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:55.959 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.961 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.961 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:55.961 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.961 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.961 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:55.961 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:55.961 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.961 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.961 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.962 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:55.963 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:55.964 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.964 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:55.964 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.964 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:55.964 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.964 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.965 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.965 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.967 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.968 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.969 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.970 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.971 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.972 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:55.973 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:55.974 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.975 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 6 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:55.976 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:55.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f352359d640 1 -- 172.21.15.178:0/2209377822 >> v2:172.21.15.204:6812/28210 conn(0x7f35000397e0 msgr2=0x7f350003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f352359d640 1 --2- 172.21.15.178:0/2209377822 >> v2:172.21.15.204:6812/28210 conn(0x7f35000397e0 0x7f350003bc90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f351c09b6d0 tx=0x7f350c009040 comp rx=0 tx=0).stop 2024-09-15T01:47:55.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f352359d640 10 monclient: shutdown 2024-09-15T01:47:55.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f352359d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:55.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f352359d640 1 -- 172.21.15.178:0/2209377822 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f351c0a2930 msgr2=0x7f351c141f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:55.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f352359d640 1 --2- 172.21.15.178:0/2209377822 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f351c0a2930 0x7f351c141f20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f351400cc90 tx=0x7f3514001a20 comp rx=0 tx=0).stop 2024-09-15T01:47:55.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f3521b3a640 1 -- 172.21.15.178:0/2209377822 reap_dead start 2024-09-15T01:47:55.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f3521b3a640 1 -- 172.21.15.178:0/2209377822 reap_dead start 2024-09-15T01:47:55.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f352359d640 1 -- 172.21.15.178:0/2209377822 shutdown_connections 2024-09-15T01:47:55.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f352359d640 1 -- 172.21.15.178:0/2209377822 >> 172.21.15.178:0/2209377822 conn(0x7f351c0a5ed0 msgr2=0x7f351c0a3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:55.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f352359d640 1 -- 172.21.15.178:0/2209377822 shutdown_connections 2024-09-15T01:47:55.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:55.922+0000 7f352359d640 1 -- 172.21.15.178:0/2209377822 wait complete. 2024-09-15T01:47:55.979 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:55.979 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:47:55.979 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:47:56.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.069+0000 7fc72eeaf640 10 monclient: get_monmap_and_config 2024-09-15T01:47:56.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.069+0000 7fc72eeaf640 10 monclient: build_initial_monmap 2024-09-15T01:47:56.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.069+0000 7fc72eeaf640 10 monclient: monmap: 2024-09-15T01:47:56.072 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:56.072 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.072 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:56.071232+0000 2024-09-15T01:47:56.073 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:56.071232+0000 2024-09-15T01:47:56.073 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:56.073 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.073 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:56.073 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:56.073 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:56.073 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:56.073 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:56.073 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:56.073 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:56.073 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:56.073 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:56.074 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:56.074 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:56.074 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 1 Processor -- start 2024-09-15T01:47:56.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 1 -- start start 2024-09-15T01:47:56.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 10 monclient: init 2024-09-15T01:47:56.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:56.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 10 monclient: _add_conns ranks=[3,4,0,10,2,8,9,5,1,6,7] 2024-09-15T01:47:56.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7280a2930 0x7fc728132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 10 monclient(hunting): picked mon.noname-g con 0x7fc7280a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:56.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc72809faa0 0x7fc72809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 10 monclient(hunting): picked mon.noname-c con 0x7fc72809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:47:56.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc7280a08c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:47:56.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc72804bf60 con 0x7fc7280a08c0 2024-09-15T01:47:56.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72809e0f0 con 0x7fc7280a2930 2024-09-15T01:47:56.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc72809dd80 con 0x7fc72809faa0 2024-09-15T01:47:56.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72eeaf640 10 monclient(hunting): authenticate will time out at 1680.821692s 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc727fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc72809faa0 0x7fc72809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72d44c640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc7280a08c0 0x7fc7280a0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:36866/0 (socket says 172.21.15.178:36866) 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc727fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc72809faa0 0x7fc72809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:46670/0 (socket says 172.21.15.178:46670) 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc72d44c640 1 -- 172.21.15.178:0/3418256758 learned_addr learned my addr 172.21.15.178:0/3418256758 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc727fff640 10 monclient(hunting): get_auth_request con 0x7fc72809faa0 auth_method 0 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc727fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc727fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc727fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc727fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc727fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.070+0000 7fc727fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc72cc4b640 1 --2- 172.21.15.178:0/3418256758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7280a2930 0x7fc728132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc72cc4b640 10 monclient(hunting): get_auth_request con 0x7fc7280a2930 auth_method 0 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc72cc4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc72cc4b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc72cc4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc727fff640 10 monclient(hunting): handle_auth_done global_id 44860 payload 274 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc7277fe640 1 -- 172.21.15.178:0/3418256758 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191616940 0 0) 0x7fc72804bf60 con 0x7fc7280a08c0 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc727fff640 1 -- 172.21.15.178:0/3418256758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7280a2930 msgr2=0x7fc728132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc727fff640 1 --2- 172.21.15.178:0/3418256758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7280a2930 0x7fc728132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc727fff640 1 -- 172.21.15.178:0/3418256758 >> v1:172.21.15.178:6789/0 conn(0x7fc7280a08c0 legacy=0x7fc7280a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc727fff640 10 monclient: _finish_hunting 0 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc727fff640 1 monclient: found mon.noname-c 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc727fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:56.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc727fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc727fff640 1 -- 172.21.15.178:0/3418256758 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7280a2d00 con 0x7fc72809faa0 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc7277fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc727fff640 1 --2- 172.21.15.178:0/3418256758 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc72809faa0 0x7fc72809fe70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc714009880 tx=0x7fc7140090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=65d3ef08e298b33 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc7277fe640 1 -- 172.21.15.178:0/3418256758 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc71402ac50 con 0x7fc72809faa0 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc7277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc7277fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.071+0000 7fc7277fe640 10 monclient: dump: 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.087 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc7277fe640 10 monclient: _finish_auth 0 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc7277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 5 monclient: authenticate success, global_id 44860 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc7277fe640 1 -- 172.21.15.178:0/3418256758 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71402add0 con 0x7fc72809faa0 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc7277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc7277fe640 1 -- 172.21.15.178:0/3418256758 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc714031ca0 con 0x7fc72809faa0 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc7277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 4 set_mon_vals no callback set 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc7277fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc7277fe640 10 monclient: dump: 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 10 monclient: shutdown 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 1 -- 172.21.15.178:0/3418256758 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc72809faa0 msgr2=0x7fc72809fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 1 --2- 172.21.15.178:0/3418256758 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc72809faa0 0x7fc72809fe70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc714009880 tx=0x7fc7140090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 1 -- 172.21.15.178:0/3418256758 shutdown_connections 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 1 --2- 172.21.15.178:0/3418256758 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc72809faa0 0x7fc72809fe70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 1 --2- 172.21.15.178:0/3418256758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7280a2930 0x7fc728132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 1 -- 172.21.15.178:0/3418256758 >> 172.21.15.178:0/3418256758 conn(0x7fc7280a5ed0 msgr2=0x7fc7280a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 1 -- 172.21.15.178:0/3418256758 shutdown_connections 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 1 -- 172.21.15.178:0/3418256758 wait complete. 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.072+0000 7fc72eeaf640 10 monclient: build_initial_monmap 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.073+0000 7fc72eeaf640 10 monclient: monmap: 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:56.091 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:56.092 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.073+0000 7fc72eeaf640 1 Processor -- start 2024-09-15T01:47:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.073+0000 7fc72eeaf640 1 -- start start 2024-09-15T01:47:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.073+0000 7fc72eeaf640 10 monclient: init 2024-09-15T01:47:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.073+0000 7fc72eeaf640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.073+0000 7fc72eeaf640 10 monclient: _add_conns ranks=[1,8,3,7,2,5,4,0,6] 2024-09-15T01:47:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.073+0000 7fc72eeaf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc72809faa0 0x7fc72809acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.073+0000 7fc72eeaf640 10 monclient(hunting): picked mon.noname-f con 0x7fc72809faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:47:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.073+0000 7fc72eeaf640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7280a08c0 0x7fc728131670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.073+0000 7fc72eeaf640 10 monclient(hunting): picked mon.noname-e con 0x7fc7280a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.073+0000 7fc72eeaf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7280a2930 0x7fc72811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.073+0000 7fc72eeaf640 10 monclient(hunting): picked mon.noname-g con 0x7fc7280a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72811ea70 con 0x7fc72809faa0 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72809bd80 con 0x7fc7280a2930 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280a2390 con 0x7fc7280a08c0 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 10 monclient(hunting): authenticate will time out at 1680.825022s 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7280a08c0 0x7fc728131670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7280a08c0 0x7fc728131670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42236/0 (socket says 172.21.15.178:42236) 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 1 -- 172.21.15.178:0/1374088665 learned_addr learned my addr 172.21.15.178:0/1374088665 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:56.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 10 monclient(hunting): get_auth_request con 0x7fc7280a08c0 auth_method 0 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72d44c640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7280a2930 0x7fc72811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72cc4b640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc72809faa0 0x7fc72809acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72cc4b640 10 monclient(hunting): get_auth_request con 0x7fc72809faa0 auth_method 0 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72cc4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72cc4b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72cc4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72d44c640 10 monclient(hunting): get_auth_request con 0x7fc7280a2930 auth_method 0 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72d44c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72d44c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72d44c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72cc4b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72cc4b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72cc4b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72d44c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72d44c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72d44c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 10 monclient(hunting): handle_auth_done global_id 94292 payload 824 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 1 -- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7280a2930 msgr2=0x7fc72811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7280a2930 0x7fc72811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 1 -- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc72809faa0 msgr2=0x7fc72809acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc72809faa0 0x7fc72809acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 10 monclient: _finish_hunting 0 2024-09-15T01:47:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 1 monclient: found mon.noname-e 2024-09-15T01:47:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 1 -- 172.21.15.178:0/1374088665 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc714009390 con 0x7fc7280a08c0 2024-09-15T01:47:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc727fff640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7280a08c0 0x7fc728131670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc714009720 tx=0x7fc714032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc71402ac50 con 0x7fc7280a08c0 2024-09-15T01:47:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc724ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc724ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc724ff9640 10 monclient: dump: 2024-09-15T01:47:56.097 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.098 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc724ff9640 10 monclient: _finish_auth 0 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc724ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc714033e30 con 0x7fc7280a08c0 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 5 monclient: authenticate success, global_id 94292 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc724ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 10 monclient: _renew_subs 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc71403b870 con 0x7fc7280a08c0 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 1 -- 172.21.15.178:0/1374088665 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72811e790 con 0x7fc7280a08c0 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc724ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc724ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:56.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc724ff9640 10 monclient: dump: 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.100 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc726ffd640 4 set_mon_vals no callback set 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 10 monclient: _renew_subs 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.074+0000 7fc72eeaf640 1 -- 172.21.15.178:0/1374088665 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72811f030 con 0x7fc7280a08c0 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.075+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc714039040 con 0x7fc7280a08c0 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.075+0000 7fc724ff9640 1 --2- 172.21.15.178:0/1374088665 >> v2:172.21.15.204:6812/28210 conn(0x7fc70c0396e0 0x7fc70c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.075+0000 7fc72eeaf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.075+0000 7fc72eeaf640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc728131bb0 0x7fc72811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.075+0000 7fc72eeaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.075+0000 7fc72eeaf640 1 -- 172.21.15.178:0/1374088665 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc728131f80 con 0x7fc728131bb0 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.075+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fc71406b3c0 con 0x7fc7280a08c0 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.075+0000 7fc72cc4b640 1 --2- 172.21.15.178:0/1374088665 >> v2:172.21.15.204:6812/28210 conn(0x7fc70c0396e0 0x7fc70c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.075+0000 7fc72cc4b640 10 monclient: get_auth_request con 0x7fc70c0396e0 auth_method 0 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc72d44c640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc728131bb0 0x7fc72811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc72d44c640 10 monclient: get_auth_request con 0x7fc728131bb0 auth_method 0 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc72d44c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc72d44c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc72d44c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc72d44c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc72d44c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc72d44c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc72d44c640 10 monclient(hunting): handle_auth_done global_id 64420 payload 824 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc72d44c640 1 -- 172.21.15.178:0/1374088665 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc72804c4a0 con 0x7fc728131bb0 2024-09-15T01:47:56.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc72cc4b640 1 --2- 172.21.15.178:0/1374088665 >> v2:172.21.15.204:6812/28210 conn(0x7fc70c0396e0 0x7fc70c03bb90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc71c007ba0 tx=0x7fc71c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc72d44c640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc728131bb0 0x7fc72811ca70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc7200021e0 tx=0x7fc720009040 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7200070d0 con 0x7fc728131bb0 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc724ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc724ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc724ff9640 10 monclient: dump: 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.103 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fc72000db90 con 0x7fc728131bb0 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc724ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc724ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc728131bb0 msgr2=0x7fc72811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc724ff9640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc728131bb0 0x7fc72811ca70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc7200021e0 tx=0x7fc720009040 comp rx=0 tx=0).stop 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc728131bb0 msgr2=0x7fc72811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:56.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.076+0000 7fc724ff9640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc728131bb0 0x7fc72811ca70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.082+0000 7fc72eeaf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:47:56.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.082+0000 7fc72eeaf640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc72811cfb0 0x7fc728096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.082+0000 7fc72eeaf640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.082+0000 7fc72eeaf640 1 -- 172.21.15.178:0/1374088665 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc72804bc80 con 0x7fc72811cfb0 2024-09-15T01:47:56.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.082+0000 7fc72d44c640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc72811cfb0 0x7fc728096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.082+0000 7fc72d44c640 10 monclient: get_auth_request con 0x7fc72811cfb0 auth_method 0 2024-09-15T01:47:56.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.082+0000 7fc72d44c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.082+0000 7fc72d44c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.082+0000 7fc72d44c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.082+0000 7fc72d44c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.082+0000 7fc72d44c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.082+0000 7fc72d44c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc72d44c640 10 monclient(hunting): handle_auth_done global_id 64429 payload 824 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc72d44c640 1 -- 172.21.15.178:0/1374088665 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc72805ed70 con 0x7fc72811cfb0 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc72d44c640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc72811cfb0 0x7fc728096a30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc720010010 tx=0x7fc720015000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc720022410 con 0x7fc72811cfb0 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc724ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc724ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc724ff9640 10 monclient: dump: 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.108 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fc720013b20 con 0x7fc72811cfb0 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc724ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc724ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc72811cfb0 msgr2=0x7fc728096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc724ff9640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc72811cfb0 0x7fc728096a30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc720010010 tx=0x7fc720015000 comp rx=0 tx=0).stop 2024-09-15T01:47:56.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc724ff9640 1 -- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc72811cfb0 msgr2=0x7fc728096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:56.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.083+0000 7fc724ff9640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc72811cfb0 0x7fc728096a30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.110 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:47:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 150, 2024-09-15T01:47:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:47:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:47:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:47:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:47:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:47:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:47:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:47:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:47:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:47:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:47:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:47:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:47:56.121 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:56.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:56.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:47:56.124 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:56.125 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.126 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:47:56.127 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:56.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:47:56.130 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:47:56.131 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:56.131 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:56.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.131 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:47:56.131 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:47:56.131 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:47:56.131 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:47:56.131 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:47:56.131 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:56.131 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:56.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.131 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:47:56.132 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:47:56.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:47:56.133 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:47:56.133 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:56.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.084+0000 7fc72eeaf640 1 -- 172.21.15.178:0/1374088665 >> v2:172.21.15.204:6812/28210 conn(0x7fc70c0396e0 msgr2=0x7fc70c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.084+0000 7fc72eeaf640 1 --2- 172.21.15.178:0/1374088665 >> v2:172.21.15.204:6812/28210 conn(0x7fc70c0396e0 0x7fc70c03bb90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc71c007ba0 tx=0x7fc71c008040 comp rx=0 tx=0).stop 2024-09-15T01:47:56.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.084+0000 7fc72eeaf640 10 monclient: shutdown 2024-09-15T01:47:56.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.084+0000 7fc72eeaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:56.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.084+0000 7fc72eeaf640 1 -- 172.21.15.178:0/1374088665 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7280a08c0 msgr2=0x7fc728131670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.084+0000 7fc72eeaf640 1 --2- 172.21.15.178:0/1374088665 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fc7280a08c0 0x7fc728131670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc714009720 tx=0x7fc714032000 comp rx=0 tx=0).stop 2024-09-15T01:47:56.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.084+0000 7fc72d44c640 1 -- 172.21.15.178:0/1374088665 reap_dead start 2024-09-15T01:47:56.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.084+0000 7fc72d44c640 1 -- 172.21.15.178:0/1374088665 reap_dead start 2024-09-15T01:47:56.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.084+0000 7fc72eeaf640 1 -- 172.21.15.178:0/1374088665 shutdown_connections 2024-09-15T01:47:56.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.084+0000 7fc72eeaf640 1 -- 172.21.15.178:0/1374088665 >> 172.21.15.178:0/1374088665 conn(0x7fc7280a5ed0 msgr2=0x7fc728135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:56.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.084+0000 7fc72eeaf640 1 -- 172.21.15.178:0/1374088665 shutdown_connections 2024-09-15T01:47:56.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.084+0000 7fc72eeaf640 1 -- 172.21.15.178:0/1374088665 wait complete. 2024-09-15T01:47:56.135 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:47:56.135 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a', 'f'] 2024-09-15T01:47:56.136 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T01:47:56.136 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-15T01:47:56.136 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a store 2024-09-15T01:47:56.136 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a sync_force --yes-i-really-mean-it 2024-09-15T01:47:56.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.227+0000 7fabc839d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:56.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.227+0000 7fabc839d640 10 monclient: build_initial_monmap 2024-09-15T01:47:56.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.227+0000 7fabc839d640 10 monclient: monmap: 2024-09-15T01:47:56.231 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:56.231 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.231 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:56.229654+0000 2024-09-15T01:47:56.231 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:56.229654+0000 2024-09-15T01:47:56.231 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.228+0000 7fabc839d640 1 Processor -- start 2024-09-15T01:47:56.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.228+0000 7fabc839d640 1 -- start start 2024-09-15T01:47:56.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.228+0000 7fabc839d640 10 monclient: init 2024-09-15T01:47:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.228+0000 7fabc839d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.228+0000 7fabc839d640 10 monclient: _add_conns ranks=[5,9,8,1,3,4,7,10,2,0,6] 2024-09-15T01:47:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.228+0000 7fabc839d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fabc00a2930 0x7fabc0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 10 monclient(hunting): picked mon.noname-h con 0x7fabc00a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:56.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fabc009faa0 0x7fabc009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 10 monclient(hunting): picked mon.noname-a con 0x7fabc009faa0 addr v2:172.21.15.178:3300/0 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fabc00a08c0 0x7fabc00a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 10 monclient(hunting): picked mon.noname-e con 0x7fabc00a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc009e0f0 con 0x7fabc009faa0 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabc009dd80 con 0x7fabc00a2930 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00a2d00 con 0x7fabc00a08c0 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc839d640 10 monclient(hunting): authenticate will time out at 1680.980049s 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc5938640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fabc009faa0 0x7fabc009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fabc00a08c0 0x7fabc00a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc5938640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fabc009faa0 0x7fabc009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35918/0 (socket says 172.21.15.178:35918) 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc5938640 1 -- 172.21.15.178:0/3220361416 learned_addr learned my addr 172.21.15.178:0/3220361416 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 10 monclient(hunting): get_auth_request con 0x7fabc00a08c0 auth_method 0 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc5938640 10 monclient(hunting): get_auth_request con 0x7fabc009faa0 auth_method 0 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc5938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc5938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc5938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc6139640 1 --2- 172.21.15.178:0/3220361416 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fabc00a2930 0x7fabc0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc6139640 10 monclient(hunting): get_auth_request con 0x7fabc00a2930 auth_method 0 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc6139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc6139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc6139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc5938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc5938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc5938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc6139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc6139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc6139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 10 monclient(hunting): handle_auth_done global_id 94301 payload 274 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 1 -- 172.21.15.178:0/3220361416 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fabc00a2930 msgr2=0x7fabc0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:56.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 1 --2- 172.21.15.178:0/3220361416 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fabc00a2930 0x7fabc0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 1 -- 172.21.15.178:0/3220361416 >> v2:172.21.15.178:3300/0 conn(0x7fabc009faa0 msgr2=0x7fabc009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 1 --2- 172.21.15.178:0/3220361416 >> v2:172.21.15.178:3300/0 conn(0x7fabc009faa0 0x7fabc009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 10 monclient: _finish_hunting 0 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 1 monclient: found mon.noname-e 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc693a640 1 -- 172.21.15.178:0/3220361416 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc004bf60 con 0x7fabc00a08c0 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.229+0000 7fabc5938640 1 --2- 172.21.15.178:0/3220361416 >> v2:172.21.15.178:3300/0 conn(0x7fabc009faa0 0x7fabc009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc693a640 1 --2- 172.21.15.178:0/3220361416 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fabc00a08c0 0x7fabc00a0d60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009850 tx=0x7fabb00090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=dd17ca4715668272 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc5137640 1 -- 172.21.15.178:0/3220361416 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fabb002abc0 con 0x7fabc00a08c0 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc5137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc5137640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:56.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc5137640 10 monclient: dump: 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.247 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.248 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc5137640 10 monclient: _finish_auth 0 2024-09-15T01:47:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc5137640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc5137640 1 -- 172.21.15.178:0/3220361416 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb002ad40 con 0x7fabc00a08c0 2024-09-15T01:47:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc839d640 5 monclient: authenticate success, global_id 94301 2024-09-15T01:47:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc839d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:47:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc5137640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc839d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc5137640 1 -- 172.21.15.178:0/3220361416 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fabb0031ca0 con 0x7fabc00a08c0 2024-09-15T01:47:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc839d640 4 set_mon_vals no callback set 2024-09-15T01:47:56.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc5137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc5137640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc5137640 10 monclient: dump: 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.250 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc839d640 10 monclient: shutdown 2024-09-15T01:47:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc839d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc839d640 1 -- 172.21.15.178:0/3220361416 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fabc00a08c0 msgr2=0x7fabc00a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc839d640 1 --2- 172.21.15.178:0/3220361416 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fabc00a08c0 0x7fabc00a0d60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009850 tx=0x7fabb00090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc839d640 1 -- 172.21.15.178:0/3220361416 shutdown_connections 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc839d640 1 --2- 172.21.15.178:0/3220361416 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fabc00a08c0 0x7fabc00a0d60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc839d640 1 --2- 172.21.15.178:0/3220361416 >> v2:172.21.15.178:3300/0 conn(0x7fabc009faa0 0x7fabc009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.230+0000 7fabc839d640 1 --2- 172.21.15.178:0/3220361416 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fabc00a2930 0x7fabc0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.231+0000 7fabc839d640 1 -- 172.21.15.178:0/3220361416 >> 172.21.15.178:0/3220361416 conn(0x7fabc00a5ed0 msgr2=0x7fabc00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.231+0000 7fabc839d640 1 -- 172.21.15.178:0/3220361416 shutdown_connections 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.231+0000 7fabc839d640 1 -- 172.21.15.178:0/3220361416 wait complete. 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.231+0000 7fabc839d640 10 monclient: build_initial_monmap 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.231+0000 7fabc839d640 10 monclient: monmap: 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:56.251 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.231+0000 7fabc839d640 1 Processor -- start 2024-09-15T01:47:56.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.231+0000 7fabc839d640 1 -- start start 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.231+0000 7fabc839d640 10 monclient: init 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 10 monclient: _add_conns ranks=[6,0,8,5,7,2,1,3,4] 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fabc009faa0 0x7fabc009b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 10 monclient(hunting): picked mon.noname-d con 0x7fabc009faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc00a08c0 0x7fabc0131750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 10 monclient(hunting): picked mon.noname-a con 0x7fabc00a08c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fabc00a2930 0x7fabc011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 10 monclient(hunting): picked mon.noname-e con 0x7fabc00a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc009bfe0 con 0x7fabc00a08c0 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fabc0142b20 con 0x7fabc009faa0 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00a2390 con 0x7fabc00a2930 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc839d640 10 monclient(hunting): authenticate will time out at 1680.983281s 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc6139640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fabc009faa0 0x7fabc009b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc00a08c0 0x7fabc0131750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc693a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fabc00a2930 0x7fabc011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc6139640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fabc009faa0 0x7fabc009b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:42110/0 (socket says 172.21.15.178:42110) 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc693a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fabc00a2930 0x7fabc011c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42266/0 (socket says 172.21.15.178:42266) 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc00a08c0 0x7fabc0131750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35926/0 (socket says 172.21.15.178:35926) 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc6139640 1 -- 172.21.15.178:0/3603165318 learned_addr learned my addr 172.21.15.178:0/3603165318 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc6139640 10 monclient(hunting): get_auth_request con 0x7fabc009faa0 auth_method 0 2024-09-15T01:47:56.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc6139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc6139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc6139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 10 monclient(hunting): get_auth_request con 0x7fabc00a08c0 auth_method 0 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc693a640 10 monclient(hunting): get_auth_request con 0x7fabc00a2930 auth_method 0 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc693a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc693a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc693a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc693a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc693a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc693a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc6139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc6139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc6139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 10 monclient(hunting): handle_auth_done global_id 64548 payload 824 2024-09-15T01:47:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 1 -- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fabc00a2930 msgr2=0x7fabc011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fabc00a2930 0x7fabc011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 1 -- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fabc009faa0 msgr2=0x7fabc009b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fabc009faa0 0x7fabc009b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 10 monclient: _finish_hunting 0 2024-09-15T01:47:56.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 1 monclient: found mon.noname-a 2024-09-15T01:47:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:47:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 1 -- 172.21.15.178:0/3603165318 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb4009450 con 0x7fabc00a08c0 2024-09-15T01:47:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc693a640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fabc00a2930 0x7fabc011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabc5938640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc00a08c0 0x7fabc0131750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fabb400e950 tx=0x7fabb40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fabb40095e0 con 0x7fabc00a08c0 2024-09-15T01:47:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.232+0000 7fabbe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabbe7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:47:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabbe7fc640 10 monclient: dump: 2024-09-15T01:47:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.257 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabbe7fc640 10 monclient: _finish_auth 0 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabbe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb400cdc0 con 0x7fabc00a08c0 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabc839d640 5 monclient: authenticate success, global_id 64548 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabc839d640 10 monclient: _renew_subs 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabc839d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabc839d640 1 -- 172.21.15.178:0/3603165318 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb0009390 con 0x7fabc00a08c0 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabbe7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fabb40138f0 con 0x7fabc00a08c0 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabbe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabbe7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabbe7fc640 10 monclient: dump: 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.259 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.260 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.261 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabc839d640 10 monclient: _renew_subs 2024-09-15T01:47:56.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabc839d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:47:56.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabc4936640 4 set_mon_vals no callback set 2024-09-15T01:47:56.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabc839d640 1 -- 172.21.15.178:0/3603165318 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc011d910 con 0x7fabc00a08c0 2024-09-15T01:47:56.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fabb4013470 con 0x7fabc00a08c0 2024-09-15T01:47:56.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabc839d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:56.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabc839d640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0131c90 0x7fabc011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabc839d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.233+0000 7fabc839d640 1 -- 172.21.15.178:0/3603165318 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc004c4a0 con 0x7fabc0131c90 2024-09-15T01:47:56.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc6139640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0131c90 0x7fabc011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc6139640 10 monclient: get_auth_request con 0x7fabc0131c90 auth_method 0 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc6139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc6139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc6139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 1 --2- 172.21.15.178:0/3603165318 >> v2:172.21.15.204:6812/28210 conn(0x7fabac0396e0 0x7fabac03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc6139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc6139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc6139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fabb404c190 con 0x7fabc00a08c0 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc6139640 10 monclient(hunting): handle_auth_done global_id 64557 payload 824 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc6139640 1 -- 172.21.15.178:0/3603165318 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fabc005ed70 con 0x7fabc0131c90 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc693a640 1 --2- 172.21.15.178:0/3603165318 >> v2:172.21.15.204:6812/28210 conn(0x7fabac0396e0 0x7fabac03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc693a640 10 monclient: get_auth_request con 0x7fabac0396e0 auth_method 0 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc6139640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0131c90 0x7fabc011ca70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabb800f3e0 tx=0x7fabb800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fabb800e040 con 0x7fabc0131c90 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 10 monclient: dump: 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.263 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fabb800f040 con 0x7fabc0131c90 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabc693a640 1 --2- 172.21.15.178:0/3603165318 >> v2:172.21.15.204:6812/28210 conn(0x7fabac0396e0 0x7fabac03bb90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fabb0034000 tx=0x7fabb0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0131c90 msgr2=0x7fabc011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0131c90 0x7fabc011ca70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabb800f3e0 tx=0x7fabb800b040 comp rx=0 tx=0).stop 2024-09-15T01:47:56.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0131c90 msgr2=0x7fabc011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:56.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.234+0000 7fabbe7fc640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0131c90 0x7fabc011ca70 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabb800f3e0 tx=0x7fabb800b040 comp rx=0 tx=0).stop 2024-09-15T01:47:56.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc839d640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc839d640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc011cfb0 0x7fabc0096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc839d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc839d640 1 -- 172.21.15.178:0/3603165318 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc004bc80 con 0x7fabc011cfb0 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc6139640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc011cfb0 0x7fabc0096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc6139640 10 monclient: get_auth_request con 0x7fabc011cfb0 auth_method 0 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc6139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc6139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc6139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc6139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc6139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc6139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc6139640 10 monclient(hunting): handle_auth_done global_id 64566 payload 824 2024-09-15T01:47:56.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc6139640 1 -- 172.21.15.178:0/3603165318 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7fabc004b720 con 0x7fabc011cfb0 2024-09-15T01:47:56.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabc6139640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc011cfb0 0x7fabc0096a30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fabb800f3e0 tx=0x7fabb801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fabb801de80 con 0x7fabc011cfb0 2024-09-15T01:47:56.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabbe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabbe7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:47:56.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.241+0000 7fabbe7fc640 10 monclient: dump: 2024-09-15T01:47:56.268 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.268 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.268 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.268 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.268 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.268 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.268 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7fabb8023040 con 0x7fabc011cfb0 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabbe7fc640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabbe7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:56.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc011cfb0 msgr2=0x7fabc0096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabbe7fc640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc011cfb0 0x7fabc0096a30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fabb800f3e0 tx=0x7fabb801b000 comp rx=0 tx=0).stop 2024-09-15T01:47:56.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabbe7fc640 1 -- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc011cfb0 msgr2=0x7fabc0096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:47:56.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabbe7fc640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc011cfb0 0x7fabc0096a30 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fabb800f3e0 tx=0x7fabb801b000 comp rx=0 tx=0).stop 2024-09-15T01:47:56.270 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:47:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: "ret": 0, 2024-09-15T01:47:56.270 INFO:teuthology.orchestra.run.smithi178.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-15T01:47:56.270 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:47:56.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabc839d640 1 -- 172.21.15.178:0/3603165318 >> v2:172.21.15.204:6812/28210 conn(0x7fabac0396e0 msgr2=0x7fabac03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabc839d640 1 --2- 172.21.15.178:0/3603165318 >> v2:172.21.15.204:6812/28210 conn(0x7fabac0396e0 0x7fabac03bb90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fabb0034000 tx=0x7fabb0032000 comp rx=0 tx=0).stop 2024-09-15T01:47:56.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabc839d640 10 monclient: shutdown 2024-09-15T01:47:56.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabc839d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:56.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabc839d640 1 -- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc00a08c0 msgr2=0x7fabc0131750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabc839d640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc00a08c0 0x7fabc0131750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fabb400e950 tx=0x7fabb40091d0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabc693a640 1 -- 172.21.15.178:0/3603165318 reap_dead start 2024-09-15T01:47:56.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabc839d640 1 -- 172.21.15.178:0/3603165318 shutdown_connections 2024-09-15T01:47:56.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabc839d640 1 --2- 172.21.15.178:0/3603165318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc00a08c0 0x7fabc0131750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabc839d640 1 -- 172.21.15.178:0/3603165318 >> 172.21.15.178:0/3603165318 conn(0x7fabc00a5ed0 msgr2=0x7fabc009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:56.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabc839d640 1 -- 172.21.15.178:0/3603165318 shutdown_connections 2024-09-15T01:47:56.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.242+0000 7fabc839d640 1 -- 172.21.15.178:0/3603165318 wait complete. 2024-09-15T01:47:56.273 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T01:47:56.273 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T01:47:56.273 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:47:56.286 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T01:47:56.286 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-15T01:47:56.286 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T01:47:56.286 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-15T01:47:56.286 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:47:56.301 INFO:tasks.ceph.mon.f:Stopped 2024-09-15T01:47:56.301 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-15T01:47:56.301 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:47:56.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.415+0000 7f65f6ca8640 10 monclient: get_monmap_and_config 2024-09-15T01:47:56.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.415+0000 7f65f6ca8640 10 monclient: build_initial_monmap 2024-09-15T01:47:56.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.415+0000 7f65f6ca8640 10 monclient: monmap: 2024-09-15T01:47:56.418 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:56.418 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.418 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:56.416933+0000 2024-09-15T01:47:56.418 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:56.416933+0000 2024-09-15T01:47:56.418 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:56.418 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.419 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:56.419 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:56.419 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:56.419 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:56.419 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:56.419 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:56.419 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:56.419 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:56.419 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:56.419 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:56.419 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:56.419 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.415+0000 7f65f6ca8640 1 Processor -- start 2024-09-15T01:47:56.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.415+0000 7f65f6ca8640 1 -- start start 2024-09-15T01:47:56.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.415+0000 7f65f6ca8640 10 monclient: init 2024-09-15T01:47:56.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:56.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 10 monclient: _add_conns ranks=[10,8,1,9,6,3,5,4,7,0,2] 2024-09-15T01:47:56.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f65f00a2930 0x7f65f0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 10 monclient(hunting): picked mon.noname-f con 0x7f65f00a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:47:56.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f65f009faa0 0x7f65f009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 10 monclient(hunting): picked mon.noname-e con 0x7f65f009faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:47:56.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f65f00a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:47:56.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f004bf60 con 0x7f65f00a08c0 2024-09-15T01:47:56.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f009e0f0 con 0x7f65f00a2930 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f65f009dd80 con 0x7f65f009faa0 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f6ca8640 10 monclient(hunting): authenticate will time out at 1681.167324s 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f4a44640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7f65f00a2930 msgr2=0x7f65f0132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f5245640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7f65f00a08c0 legacy=0x7f65f00a0d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65f4a44640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f65f00a2930 0x7f65f0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f65f009faa0 0x7f65f009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f65f009faa0 0x7f65f009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42272/0 (socket says 172.21.15.178:42272) 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 1 -- 172.21.15.178:0/1634914522 learned_addr learned my addr 172.21.15.178:0/1634914522 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 10 monclient(hunting): get_auth_request con 0x7f65f009faa0 auth_method 0 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 10 monclient(hunting): handle_auth_done global_id 94319 payload 274 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 1 -- 172.21.15.178:0/1634914522 >> v2:172.21.15.204:3300/0 conn(0x7f65f00a2930 msgr2=0x7f65f0132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 1 --2- 172.21.15.178:0/1634914522 >> v2:172.21.15.204:3300/0 conn(0x7f65f00a2930 0x7f65f0132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 1 -- 172.21.15.178:0/1634914522 >> v1:172.21.15.204:6789/0 conn(0x7f65f00a08c0 legacy=0x7f65f00a0d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 10 monclient: _finish_hunting 0 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 1 monclient: found mon.noname-e 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.416+0000 7f65effff640 1 -- 172.21.15.178:0/1634914522 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f00a2d00 con 0x7f65f009faa0 2024-09-15T01:47:56.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65effff640 1 --2- 172.21.15.178:0/1634914522 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f65f009faa0 0x7f65f009fe70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009f20 tx=0x7f65e0009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=81a50a696e0b0de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65ef7fe640 1 -- 172.21.15.178:0/1634914522 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f65e0007880 con 0x7f65f009faa0 2024-09-15T01:47:56.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65ef7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:47:56.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65ef7fe640 10 monclient: dump: 2024-09-15T01:47:56.432 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.432 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.432 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.432 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.432 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.432 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.432 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.432 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65ef7fe640 10 monclient: _finish_auth 0 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65ef7fe640 1 -- 172.21.15.178:0/1634914522 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e002d050 con 0x7f65f009faa0 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65ef7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65ef7fe640 1 -- 172.21.15.178:0/1634914522 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f65e0030360 con 0x7f65f009faa0 2024-09-15T01:47:56.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65f6ca8640 5 monclient: authenticate success, global_id 94319 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65f6ca8640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65f6ca8640 4 set_mon_vals no callback set 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65ef7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.417+0000 7f65ef7fe640 10 monclient: dump: 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.434 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 10 monclient: shutdown 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1634914522 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f65f009faa0 msgr2=0x7f65f009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 1 --2- 172.21.15.178:0/1634914522 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f65f009faa0 0x7f65f009fe70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009f20 tx=0x7f65e0009840 comp rx=0 tx=0).stop 2024-09-15T01:47:56.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1634914522 shutdown_connections 2024-09-15T01:47:56.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 1 --2- 172.21.15.178:0/1634914522 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f65f009faa0 0x7f65f009fe70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 1 --2- 172.21.15.178:0/1634914522 >> v2:172.21.15.204:3300/0 conn(0x7f65f00a2930 0x7f65f0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1634914522 >> 172.21.15.178:0/1634914522 conn(0x7f65f00a5ed0 msgr2=0x7f65f00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:56.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1634914522 shutdown_connections 2024-09-15T01:47:56.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1634914522 wait complete. 2024-09-15T01:47:56.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 10 monclient: build_initial_monmap 2024-09-15T01:47:56.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 10 monclient: monmap: 2024-09-15T01:47:56.436 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:56.436 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:56.436 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:56.436 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:56.436 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.418+0000 7f65f6ca8640 1 Processor -- start 2024-09-15T01:47:56.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 1 -- start start 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 10 monclient: init 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 10 monclient: _add_conns ranks=[3,7,1,2,6,5,0,8,4] 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65f009faa0 0x7f65f009b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 10 monclient(hunting): picked mon.noname-g con 0x7f65f009faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f65f00a08c0 0x7f65f0131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 10 monclient(hunting): picked mon.noname-i con 0x7f65f00a08c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65f00a2930 0x7f65f011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 10 monclient(hunting): picked mon.noname-f con 0x7f65f00a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f009c230 con 0x7f65f00a2930 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f011ef20 con 0x7f65f009faa0 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f65f009bd20 con 0x7f65f00a08c0 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f6ca8640 10 monclient(hunting): authenticate will time out at 1681.170642s 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f5245640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65f00a2930 msgr2=0x7f65f011c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f5245640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65f00a2930 0x7f65f011c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f4a44640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65f009faa0 0x7f65f009b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65effff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f65f00a08c0 0x7f65f0131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f4a44640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65f009faa0 0x7f65f009b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:41516/0 (socket says 172.21.15.178:41516) 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f4a44640 1 -- 172.21.15.178:0/1815676371 learned_addr learned my addr 172.21.15.178:0/1815676371 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f4a44640 10 monclient(hunting): get_auth_request con 0x7f65f009faa0 auth_method 0 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f4a44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f4a44640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65f4a44640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65effff640 1 --2- 172.21.15.178:0/1815676371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f65f00a08c0 0x7f65f0131840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:41228/0 (socket says 172.21.15.178:41228) 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65effff640 10 monclient(hunting): get_auth_request con 0x7f65f00a08c0 auth_method 0 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65effff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.419+0000 7f65effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:56.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 10 monclient(hunting): handle_auth_done global_id 64461 payload 824 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 1 -- 172.21.15.178:0/1815676371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f65f00a08c0 msgr2=0x7f65f0131840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 1 --2- 172.21.15.178:0/1815676371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f65f00a08c0 0x7f65f0131840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 1 -- 172.21.15.178:0/1815676371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65f00a2930 msgr2=0x7f65f011c530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 1 --2- 172.21.15.178:0/1815676371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65f00a2930 0x7f65f011c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 10 monclient: _finish_hunting 0 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 1 monclient: found mon.noname-g 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 1 -- 172.21.15.178:0/1815676371 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e0009ae0 con 0x7f65f009faa0 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65effff640 1 --2- 172.21.15.178:0/1815676371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f65f00a08c0 0x7f65f0131840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f4a44640 1 --2- 172.21.15.178:0/1815676371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65f009faa0 0x7f65f009b270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f65e800b600 tx=0x7f65e80091a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65ecff9640 1 -- 172.21.15.178:0/1815676371 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f65e8015070 con 0x7f65f009faa0 2024-09-15T01:47:56.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65ecff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65ecff9640 10 monclient: dump: 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.442 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65ecff9640 10 monclient: _finish_auth 0 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65ecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f6ca8640 5 monclient: authenticate success, global_id 64461 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65ecff9640 1 -- 172.21.15.178:0/1815676371 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e8006150 con 0x7f65f009faa0 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65ecff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65ecff9640 1 -- 172.21.15.178:0/1815676371 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f65e80062c0 con 0x7f65f009faa0 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65ecff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:47:56.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65ecff9640 10 monclient: dump: 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65eeffd640 4 set_mon_vals no callback set 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f6ca8640 10 monclient: _renew_subs 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f6ca8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1815676371 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f011e9c0 con 0x7f65f009faa0 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f6ca8640 10 monclient: _renew_subs 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f6ca8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.420+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1815676371 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f0142b70 con 0x7f65f009faa0 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.421+0000 7f65ecff9640 1 -- 172.21.15.178:0/1815676371 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f65e8003b80 con 0x7f65f009faa0 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.421+0000 7f65f6ca8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.421+0000 7f65f6ca8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:56.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.421+0000 7f65f6ca8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:47:56.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.421+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1815676371 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f004b720 con 0x7f65f009faa0 2024-09-15T01:47:56.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.421+0000 7f65ecff9640 1 --2- 172.21.15.178:0/1815676371 >> v2:172.21.15.204:6812/28210 conn(0x7f65d00397b0 0x7f65d003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:56.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.422+0000 7f65ecff9640 1 -- 172.21.15.178:0/1815676371 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f65e804b640 con 0x7f65f009faa0 2024-09-15T01:47:56.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.422+0000 7f65effff640 1 --2- 172.21.15.178:0/1815676371 >> v2:172.21.15.204:6812/28210 conn(0x7f65d00397b0 0x7f65d003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:56.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.422+0000 7f65effff640 10 monclient: get_auth_request con 0x7f65d00397b0 auth_method 0 2024-09-15T01:47:56.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.422+0000 7f65effff640 1 --2- 172.21.15.178:0/1815676371 >> v2:172.21.15.204:6812/28210 conn(0x7f65d00397b0 0x7f65d003bc60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f65e0033000 tx=0x7f65e0036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:56.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.427+0000 7f65ecff9640 1 -- 172.21.15.178:0/1815676371 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f65e8003e30 con 0x7f65f009faa0 2024-09-15T01:47:56.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.427+0000 7f65ecff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:56.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.427+0000 7f65ecff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:56.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.642+0000 7f65f6ca8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:47:56.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.642+0000 7f65f6ca8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:47:56.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.642+0000 7f65f6ca8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:47:56.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.642+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1815676371 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f65f009cef0 con 0x7f65f009faa0 2024-09-15T01:47:56.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.642+0000 7f65ecff9640 1 -- 172.21.15.178:0/1815676371 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7f65e800eb30 con 0x7f65f009faa0 2024-09-15T01:47:56.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.642+0000 7f65ecff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:47:56.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.642+0000 7f65ecff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:56.646 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":150,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":5,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:47:56.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.644+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1815676371 >> v2:172.21.15.204:6812/28210 conn(0x7f65d00397b0 msgr2=0x7f65d003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.644+0000 7f65f6ca8640 1 --2- 172.21.15.178:0/1815676371 >> v2:172.21.15.204:6812/28210 conn(0x7f65d00397b0 0x7f65d003bc60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f65e0033000 tx=0x7f65e0036000 comp rx=0 tx=0).stop 2024-09-15T01:47:56.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.644+0000 7f65f6ca8640 10 monclient: shutdown 2024-09-15T01:47:56.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.644+0000 7f65f6ca8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:56.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.644+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1815676371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65f009faa0 msgr2=0x7f65f009b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:56.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.644+0000 7f65f6ca8640 1 --2- 172.21.15.178:0/1815676371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65f009faa0 0x7f65f009b270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f65e800b600 tx=0x7f65e80091a0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.644+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1815676371 shutdown_connections 2024-09-15T01:47:56.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.644+0000 7f65f6ca8640 1 --2- 172.21.15.178:0/1815676371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65f00a2930 0x7f65f011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.644+0000 7f65f6ca8640 1 --2- 172.21.15.178:0/1815676371 >> v2:172.21.15.204:6812/28210 conn(0x7f65d00397b0 0x7f65d003bc60 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.644+0000 7f65f6ca8640 1 --2- 172.21.15.178:0/1815676371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f65f00a08c0 0x7f65f0131840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.644+0000 7f65f6ca8640 1 --2- 172.21.15.178:0/1815676371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65f009faa0 0x7f65f009b270 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:56.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.645+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1815676371 >> 172.21.15.178:0/1815676371 conn(0x7f65f00a5ed0 msgr2=0x7f65f009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:56.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.645+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1815676371 shutdown_connections 2024-09-15T01:47:56.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:56.645+0000 7f65f6ca8640 1 -- 172.21.15.178:0/1815676371 wait complete. 2024-09-15T01:47:59.659 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:47:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.761+0000 7fe27ef9d640 10 monclient: get_monmap_and_config 2024-09-15T01:47:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.761+0000 7fe27ef9d640 10 monclient: build_initial_monmap 2024-09-15T01:47:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.761+0000 7fe27ef9d640 10 monclient: monmap: 2024-09-15T01:47:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:47:59.761987+0000 2024-09-15T01:47:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:47:59.761987+0000 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:47:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:47:59.765 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.761+0000 7fe27ef9d640 1 Processor -- start 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.761+0000 7fe27ef9d640 1 -- start start 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.761+0000 7fe27ef9d640 10 monclient: init 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 10 monclient: _add_conns ranks=[0,10,6,5,8,1,7,3,4,2,9] 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe2780a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe278132de0 0x7fe278137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 10 monclient(hunting): picked mon.noname-f con 0x7fe278132de0 addr v2:172.21.15.204:3300/0 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2781380e0 0x7fe27813a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 10 monclient(hunting): picked mon.noname-d con 0x7fe2781380e0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27804bf60 con 0x7fe2780a2930 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27809dd80 con 0x7fe278132de0 2024-09-15T01:47:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780a2d00 con 0x7fe2781380e0 2024-09-15T01:47:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27ef9d640 10 monclient(hunting): authenticate will time out at 1684.512474s 2024-09-15T01:47:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27cd39640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7fe2780a2930 legacy=0x7fe27809faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:47:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2781380e0 0x7fe27813a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe277fff640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7fe278132de0 msgr2=0x7fe278137ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:47:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe277fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe278132de0 0x7fe278137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:47:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2781380e0 0x7fe27813a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:59862/0 (socket says 172.21.15.178:59862) 2024-09-15T01:47:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 1 -- 172.21.15.178:0/382155918 learned_addr learned my addr 172.21.15.178:0/382155918 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 10 monclient(hunting): get_auth_request con 0x7fe2781380e0 auth_method 0 2024-09-15T01:47:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 10 monclient(hunting): handle_auth_done global_id 25764 payload 274 2024-09-15T01:47:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 1 -- 172.21.15.178:0/382155918 >> v2:172.21.15.204:3300/0 conn(0x7fe278132de0 msgr2=0x7fe278137ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:47:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 1 --2- 172.21.15.178:0/382155918 >> v2:172.21.15.204:3300/0 conn(0x7fe278132de0 0x7fe278137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 1 -- 172.21.15.178:0/382155918 >> v1:172.21.15.178:6789/0 conn(0x7fe2780a2930 legacy=0x7fe27809faa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:47:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 10 monclient: _finish_hunting 0 2024-09-15T01:47:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 1 monclient: found mon.noname-d 2024-09-15T01:47:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:47:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.762+0000 7fe27d53a640 1 -- 172.21.15.178:0/382155918 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2781331b0 con 0x7fe2781380e0 2024-09-15T01:47:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe27d53a640 1 --2- 172.21.15.178:0/382155918 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2781380e0 0x7fe27813a4c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe270009910 tx=0x7fe2700091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=d80ce0ca8ac40823 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe2777fe640 1 -- 172.21.15.178:0/382155918 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe27002acc0 con 0x7fe2781380e0 2024-09-15T01:47:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe2777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe2777fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:47:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe2777fe640 10 monclient: dump: 2024-09-15T01:47:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:59.779 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe2777fe640 10 monclient: _finish_auth 0 2024-09-15T01:47:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe2777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe2777fe640 1 -- 172.21.15.178:0/382155918 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27002ae40 con 0x7fe2781380e0 2024-09-15T01:47:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe27ef9d640 5 monclient: authenticate success, global_id 25764 2024-09-15T01:47:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe2777fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe2777fe640 1 -- 172.21.15.178:0/382155918 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe270031ca0 con 0x7fe2781380e0 2024-09-15T01:47:59.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe27ef9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:47:59.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe27ef9d640 4 set_mon_vals no callback set 2024-09-15T01:47:59.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe2777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:59.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe2777fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:47:59.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.763+0000 7fe2777fe640 10 monclient: dump: 2024-09-15T01:47:59.780 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:59.780 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:59.780 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:59.780 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:59.780 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:59.780 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:59.780 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:59.781 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:59.781 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:59.781 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:59.781 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:59.781 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:59.781 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:59.781 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:59.781 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:59.781 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:59.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.764+0000 7fe27ef9d640 10 monclient: shutdown 2024-09-15T01:47:59.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.764+0000 7fe27ef9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:59.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.764+0000 7fe27ef9d640 1 -- 172.21.15.178:0/382155918 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2781380e0 msgr2=0x7fe27813a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:59.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.764+0000 7fe27ef9d640 1 --2- 172.21.15.178:0/382155918 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2781380e0 0x7fe27813a4c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe270009910 tx=0x7fe2700091d0 comp rx=0 tx=0).stop 2024-09-15T01:47:59.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.764+0000 7fe27ef9d640 1 -- 172.21.15.178:0/382155918 shutdown_connections 2024-09-15T01:47:59.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.764+0000 7fe27ef9d640 1 --2- 172.21.15.178:0/382155918 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe2781380e0 0x7fe27813a4c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:59.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.764+0000 7fe27ef9d640 1 --2- 172.21.15.178:0/382155918 >> v2:172.21.15.204:3300/0 conn(0x7fe278132de0 0x7fe278137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:59.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.764+0000 7fe27ef9d640 1 -- 172.21.15.178:0/382155918 >> 172.21.15.178:0/382155918 conn(0x7fe2780a5ed0 msgr2=0x7fe2780a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:59.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.764+0000 7fe27ef9d640 1 -- 172.21.15.178:0/382155918 shutdown_connections 2024-09-15T01:47:59.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.764+0000 7fe27ef9d640 1 -- 172.21.15.178:0/382155918 wait complete. 2024-09-15T01:47:59.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.764+0000 7fe27ef9d640 10 monclient: build_initial_monmap 2024-09-15T01:47:59.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.764+0000 7fe27ef9d640 10 monclient: monmap: 2024-09-15T01:47:59.782 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:47:59.782 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:47:59.782 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:59.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 1 Processor -- start 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 1 -- start start 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 10 monclient: init 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 10 monclient: _add_conns ranks=[5,7,1,8,2,4,6,0,3] 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2780a2930 0x7fe27812f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 10 monclient(hunting): picked mon.noname-h con 0x7fe2780a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe278132de0 0x7fe2781338b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 10 monclient(hunting): picked mon.noname-i con 0x7fe278132de0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2781380e0 0x7fe278133df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 10 monclient(hunting): picked mon.noname-f con 0x7fe2781380e0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.765+0000 7fe27ef9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2781374f0 con 0x7fe2781380e0 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27ef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:59.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27ef9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe278136750 con 0x7fe2780a2930 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27ef9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27ef9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe27809d630 con 0x7fe278132de0 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27ef9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27ef9d640 10 monclient(hunting): authenticate will time out at 1684.516017s 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27d53a640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2781380e0 msgr2=0x7fe278133df0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27d53a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2781380e0 0x7fe278133df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27cd39640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2780a2930 0x7fe27812f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe278132de0 0x7fe2781338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27cd39640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2780a2930 0x7fe27812f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:44832/0 (socket says 172.21.15.178:44832) 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe278132de0 0x7fe2781338b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:41906/0 (socket says 172.21.15.178:41906) 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27cd39640 1 -- 172.21.15.178:0/81428410 learned_addr learned my addr 172.21.15.178:0/81428410 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27cd39640 10 monclient(hunting): get_auth_request con 0x7fe2780a2930 auth_method 0 2024-09-15T01:47:59.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27cd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27cd39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27cd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 10 monclient(hunting): get_auth_request con 0x7fe278132de0 auth_method 0 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27cd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27cd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27cd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:47:59.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 10 monclient(hunting): handle_auth_done global_id 64447 payload 824 2024-09-15T01:47:59.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 1 -- 172.21.15.178:0/81428410 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2780a2930 msgr2=0x7fe27812f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:59.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 1 --2- 172.21.15.178:0/81428410 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2780a2930 0x7fe27812f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:59.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 1 -- 172.21.15.178:0/81428410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2781380e0 msgr2=0x7fe278133df0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:47:59.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 1 --2- 172.21.15.178:0/81428410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2781380e0 0x7fe278133df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:59.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 10 monclient: _finish_hunting 0 2024-09-15T01:47:59.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 1 monclient: found mon.noname-i 2024-09-15T01:47:59.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:47:59.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:47:59.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe277fff640 1 -- 172.21.15.178:0/81428410 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe270009450 con 0x7fe278132de0 2024-09-15T01:47:59.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.766+0000 7fe27cd39640 1 --2- 172.21.15.178:0/81428410 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2780a2930 0x7fe27812f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:47:59.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe277fff640 1 --2- 172.21.15.178:0/81428410 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe278132de0 0x7fe2781338b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe26800b5b0 tx=0x7fe2680090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:59.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe274ff9640 1 -- 172.21.15.178:0/81428410 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe268006150 con 0x7fe278132de0 2024-09-15T01:47:59.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe274ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:59.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe274ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:47:59.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe274ff9640 10 monclient: dump: 2024-09-15T01:47:59.788 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:59.788 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:59.788 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:59.788 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:59.788 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe274ff9640 10 monclient: _finish_auth 0 2024-09-15T01:47:59.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe274ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:47:59.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe274ff9640 1 -- 172.21.15.178:0/81428410 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2680062f0 con 0x7fe278132de0 2024-09-15T01:47:59.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe27ef9d640 5 monclient: authenticate success, global_id 64447 2024-09-15T01:47:59.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe274ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:47:59.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe274ff9640 1 -- 172.21.15.178:0/81428410 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2680103e0 con 0x7fe278132de0 2024-09-15T01:47:59.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe27ef9d640 10 monclient: _renew_subs 2024-09-15T01:47:59.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe27ef9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:47:59.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe27ef9d640 1 -- 172.21.15.178:0/81428410 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2780a2390 con 0x7fe278132de0 2024-09-15T01:47:59.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe274ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:47:59.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe274ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:47:59.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe276ffd640 4 set_mon_vals no callback set 2024-09-15T01:47:59.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe274ff9640 10 monclient: dump: 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:47:59.791 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe27ef9d640 10 monclient: _renew_subs 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe27ef9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.767+0000 7fe27ef9d640 1 -- 172.21.15.178:0/81428410 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe278134c20 con 0x7fe278132de0 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.768+0000 7fe274ff9640 1 -- 172.21.15.178:0/81428410 <== mon.7 v2:172.21.15.204:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe268003b80 con 0x7fe278132de0 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.768+0000 7fe27ef9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.768+0000 7fe27ef9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.768+0000 7fe27ef9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.768+0000 7fe27ef9d640 1 -- 172.21.15.178:0/81428410 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27804b720 con 0x7fe278132de0 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.768+0000 7fe274ff9640 1 --2- 172.21.15.178:0/81428410 >> v2:172.21.15.204:6812/28210 conn(0x7fe2580397b0 0x7fe25803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:47:59.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.768+0000 7fe274ff9640 1 -- 172.21.15.178:0/81428410 <== mon.7 v2:172.21.15.204:3303/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fe26804aff0 con 0x7fe278132de0 2024-09-15T01:47:59.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.768+0000 7fe27cd39640 1 --2- 172.21.15.178:0/81428410 >> v2:172.21.15.204:6812/28210 conn(0x7fe2580397b0 0x7fe25803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:47:59.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.768+0000 7fe27cd39640 10 monclient: get_auth_request con 0x7fe2580397b0 auth_method 0 2024-09-15T01:47:59.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.769+0000 7fe27cd39640 1 --2- 172.21.15.178:0/81428410 >> v2:172.21.15.204:6812/28210 conn(0x7fe2580397b0 0x7fe25803bc60 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe260009fd0 tx=0x7fe2600097b0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:47:59.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.773+0000 7fe274ff9640 1 -- 172.21.15.178:0/81428410 <== mon.7 v2:172.21.15.204:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe26800daf0 con 0x7fe278132de0 2024-09-15T01:47:59.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.773+0000 7fe274ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:47:59.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.773+0000 7fe274ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:47:59.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.987+0000 7fe27ef9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:47:59.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.987+0000 7fe27ef9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:47:59.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.987+0000 7fe27ef9d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.204:3303/0 2024-09-15T01:47:59.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.987+0000 7fe27ef9d640 1 -- 172.21.15.178:0/81428410 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe27809cef0 con 0x7fe278132de0 2024-09-15T01:47:59.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.988+0000 7fe274ff9640 1 -- 172.21.15.178:0/81428410 <== mon.7 v2:172.21.15.204:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7fe268016070 con 0x7fe278132de0 2024-09-15T01:47:59.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.988+0000 7fe274ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:47:59.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.988+0000 7fe274ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:47:59.991 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":150,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":8,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:47:59.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 1 -- 172.21.15.178:0/81428410 >> v2:172.21.15.204:6812/28210 conn(0x7fe2580397b0 msgr2=0x7fe25803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:59.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 1 --2- 172.21.15.178:0/81428410 >> v2:172.21.15.204:6812/28210 conn(0x7fe2580397b0 0x7fe25803bc60 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe260009fd0 tx=0x7fe2600097b0 comp rx=0 tx=0).stop 2024-09-15T01:47:59.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 10 monclient: shutdown 2024-09-15T01:47:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:47:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 1 -- 172.21.15.178:0/81428410 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe278132de0 msgr2=0x7fe2781338b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:47:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 1 --2- 172.21.15.178:0/81428410 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe278132de0 0x7fe2781338b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe26800b5b0 tx=0x7fe2680090f0 comp rx=0 tx=0).stop 2024-09-15T01:47:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 1 -- 172.21.15.178:0/81428410 shutdown_connections 2024-09-15T01:47:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 1 --2- 172.21.15.178:0/81428410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2781380e0 0x7fe278133df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 1 --2- 172.21.15.178:0/81428410 >> v2:172.21.15.204:6812/28210 conn(0x7fe2580397b0 0x7fe25803bc60 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 1 --2- 172.21.15.178:0/81428410 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe278132de0 0x7fe2781338b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 1 --2- 172.21.15.178:0/81428410 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe2780a2930 0x7fe27812f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:47:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 1 -- 172.21.15.178:0/81428410 >> 172.21.15.178:0/81428410 conn(0x7fe2780a5ed0 msgr2=0x7fe27812fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:47:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 1 -- 172.21.15.178:0/81428410 shutdown_connections 2024-09-15T01:47:59.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:47:59.990+0000 7fe27ef9d640 1 -- 172.21.15.178:0/81428410 wait complete. 2024-09-15T01:48:03.006 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:48:03.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.108+0000 7fecf08af640 10 monclient: get_monmap_and_config 2024-09-15T01:48:03.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.108+0000 7fecf08af640 10 monclient: build_initial_monmap 2024-09-15T01:48:03.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.108+0000 7fecf08af640 10 monclient: monmap: 2024-09-15T01:48:03.111 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:03.111 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:03.111 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:48:03.109123+0000 2024-09-15T01:48:03.111 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:48:03.109123+0000 2024-09-15T01:48:03.111 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:03.111 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:03.111 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:03.111 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:03.111 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:03.112 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:03.112 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:03.112 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:03.112 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:03.112 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:03.112 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:03.112 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:03.112 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:03.112 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:03.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.108+0000 7fecf08af640 1 Processor -- start 2024-09-15T01:48:03.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.108+0000 7fecf08af640 1 -- start start 2024-09-15T01:48:03.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.108+0000 7fecf08af640 10 monclient: init 2024-09-15T01:48:03.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:03.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 10 monclient: _add_conns ranks=[6,2,1,4,10,7,3,0,9,5,8] 2024-09-15T01:48:03.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fecec0a2930 0x7fecec132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:03.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 10 monclient(hunting): picked mon.noname-d con 0x7fecec0a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:48:03.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fecec09faa0 0x7fecec09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:03.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 10 monclient(hunting): picked mon.noname-b con 0x7fecec09faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:48:03.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fecec0a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:48:03.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec04bf60 con 0x7fecec0a08c0 2024-09-15T01:48:03.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec09e0f0 con 0x7fecec09faa0 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fecec09dd80 con 0x7fecec0a2930 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecf08af640 10 monclient(hunting): authenticate will time out at 1687.859610s 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecead9d640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7fecec0a08c0 legacy=0x7fecec0a0d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fece9d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fecec09faa0 0x7fecec09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecea59c640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fecec0a2930 0x7fecec132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fece9d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fecec09faa0 0x7fecec09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:49342/0 (socket says 172.21.15.178:49342) 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fece9d9b640 1 -- 172.21.15.178:0/2994124987 learned_addr learned my addr 172.21.15.178:0/2994124987 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fece9d9b640 10 monclient(hunting): get_auth_request con 0x7fecec09faa0 auth_method 0 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fece9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:03.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fece9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:03.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fece9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:03.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecea59c640 10 monclient(hunting): get_auth_request con 0x7fecec0a2930 auth_method 0 2024-09-15T01:48:03.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:03.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:03.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:03.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fece9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:03.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fece9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:03.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fece9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:03.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.109+0000 7fecea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fecea59c640 10 monclient(hunting): handle_auth_done global_id 25773 payload 274 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fecea59c640 1 -- 172.21.15.178:0/2994124987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fecec09faa0 msgr2=0x7fecec09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fecea59c640 1 --2- 172.21.15.178:0/2994124987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fecec09faa0 0x7fecec09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fecea59c640 1 -- 172.21.15.178:0/2994124987 >> v1:172.21.15.204:6789/0 conn(0x7fecec0a08c0 legacy=0x7fecec0a0d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fecea59c640 10 monclient: _finish_hunting 0 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fecea59c640 1 monclient: found mon.noname-d 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fecea59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fecea59c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fecea59c640 1 -- 172.21.15.178:0/2994124987 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec0a2d00 con 0x7fecec0a2930 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fece9d9b640 1 --2- 172.21.15.178:0/2994124987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fecec09faa0 0x7fecec09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:48:03.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fecea59c640 1 --2- 172.21.15.178:0/2994124987 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fecec0a2930 0x7fecec132de0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fece0009880 tx=0x7fece00090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=9c7ee53a2bfd9e7b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fece959a640 1 -- 172.21.15.178:0/2994124987 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fece002abc0 con 0x7fecec0a2930 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fece959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fece959a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fece959a640 10 monclient: dump: 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:03.125 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:03.126 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:03.126 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:03.126 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:03.126 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:03.126 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:03.126 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:03.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fece959a640 10 monclient: _finish_auth 0 2024-09-15T01:48:03.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fece959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:03.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fece959a640 1 -- 172.21.15.178:0/2994124987 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece002ad40 con 0x7fecec0a2930 2024-09-15T01:48:03.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.110+0000 7fecf08af640 5 monclient: authenticate success, global_id 25773 2024-09-15T01:48:03.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:48:03.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fece959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fece959a640 1 -- 172.21.15.178:0/2994124987 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fece0031ca0 con 0x7fecec0a2930 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 4 set_mon_vals no callback set 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fece959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fece959a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fece959a640 10 monclient: dump: 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:03.127 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 10 monclient: shutdown 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 1 -- 172.21.15.178:0/2994124987 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fecec0a2930 msgr2=0x7fecec132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 1 --2- 172.21.15.178:0/2994124987 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fecec0a2930 0x7fecec132de0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fece0009880 tx=0x7fece00090f0 comp rx=0 tx=0).stop 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 1 -- 172.21.15.178:0/2994124987 shutdown_connections 2024-09-15T01:48:03.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 1 --2- 172.21.15.178:0/2994124987 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fecec09faa0 0x7fecec09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 1 --2- 172.21.15.178:0/2994124987 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fecec0a2930 0x7fecec132de0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 1 -- 172.21.15.178:0/2994124987 >> 172.21.15.178:0/2994124987 conn(0x7fecec0a5ed0 msgr2=0x7fecec0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 1 -- 172.21.15.178:0/2994124987 shutdown_connections 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.111+0000 7fecf08af640 1 -- 172.21.15.178:0/2994124987 wait complete. 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.112+0000 7fecf08af640 10 monclient: build_initial_monmap 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.112+0000 7fecf08af640 10 monclient: monmap: 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:48:03.129 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.112+0000 7fecf08af640 1 Processor -- start 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.112+0000 7fecf08af640 1 -- start start 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.112+0000 7fecf08af640 10 monclient: init 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.112+0000 7fecf08af640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.112+0000 7fecf08af640 10 monclient: _add_conns ranks=[8,4,2,5,6,0,1,7,3] 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.112+0000 7fecf08af640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fecec09faa0 0x7fecec09b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 10 monclient(hunting): picked mon.noname-e con 0x7fecec09faa0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fecec0a08c0 0x7fecec1319d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 10 monclient(hunting): picked mon.noname-c con 0x7fecec0a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fecec0a2930 0x7fecec142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 10 monclient(hunting): picked mon.noname-b con 0x7fecec0a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec09d630 con 0x7fecec0a2930 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fecec11d7f0 con 0x7fecec0a08c0 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fecec11dc00 con 0x7fecec09faa0 2024-09-15T01:48:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 10 monclient(hunting): authenticate will time out at 1687.863062s 2024-09-15T01:48:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fecec09faa0 0x7fecec09b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fece9d9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fecec0a08c0 0x7fecec1319d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fecec09faa0 0x7fecec09b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:59226/0 (socket says 172.21.15.178:59226) 2024-09-15T01:48:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 1 -- 172.21.15.178:0/917127583 learned_addr learned my addr 172.21.15.178:0/917127583 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 10 monclient(hunting): get_auth_request con 0x7fecec09faa0 auth_method 0 2024-09-15T01:48:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fece9d9b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fecec0a08c0 0x7fecec1319d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:53474/0 (socket says 172.21.15.178:53474) 2024-09-15T01:48:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fece9d9b640 10 monclient(hunting): get_auth_request con 0x7fecec0a08c0 auth_method 0 2024-09-15T01:48:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fece9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fece9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fece9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecead9d640 1 --2- 172.21.15.178:0/917127583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fecec0a2930 0x7fecec142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecead9d640 10 monclient(hunting): get_auth_request con 0x7fecec0a2930 auth_method 0 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecead9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fece9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fece9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fece9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecead9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecead9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecead9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 10 monclient(hunting): handle_auth_done global_id 94328 payload 824 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 1 -- 172.21.15.178:0/917127583 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fecec0a08c0 msgr2=0x7fecec1319d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 1 --2- 172.21.15.178:0/917127583 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fecec0a08c0 0x7fecec1319d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 1 -- 172.21.15.178:0/917127583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fecec0a2930 msgr2=0x7fecec142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 1 --2- 172.21.15.178:0/917127583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fecec0a2930 0x7fecec142060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 10 monclient: _finish_hunting 0 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 1 monclient: found mon.noname-e 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:48:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 1 -- 172.21.15.178:0/917127583 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc009450 con 0x7fecec09faa0 2024-09-15T01:48:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecea59c640 1 --2- 172.21.15.178:0/917127583 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fecec09faa0 0x7fecec09b5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fece0009720 tx=0x7fece0032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecda7fc640 1 -- 172.21.15.178:0/917127583 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fece003b820 con 0x7fecec09faa0 2024-09-15T01:48:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecda7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecda7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:48:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecda7fc640 10 monclient: dump: 2024-09-15T01:48:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecda7fc640 10 monclient: _finish_auth 0 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecda7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecda7fc640 1 -- 172.21.15.178:0/917127583 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece003b980 con 0x7fecec09faa0 2024-09-15T01:48:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 5 monclient: authenticate success, global_id 94328 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 10 monclient: _renew_subs 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecf08af640 1 -- 172.21.15.178:0/917127583 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece0009390 con 0x7fecec09faa0 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecda7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecda7fc640 1 -- 172.21.15.178:0/917127583 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fece00313a0 con 0x7fecec09faa0 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecda7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecda7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fecda7fc640 10 monclient: dump: 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.113+0000 7fece8d99640 4 set_mon_vals no callback set 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.114+0000 7fecf08af640 10 monclient: _renew_subs 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.114+0000 7fecf08af640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:48:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.114+0000 7fecf08af640 1 -- 172.21.15.178:0/917127583 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec11edd0 con 0x7fecec09faa0 2024-09-15T01:48:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.114+0000 7fecda7fc640 1 -- 172.21.15.178:0/917127583 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fece003b3a0 con 0x7fecec09faa0 2024-09-15T01:48:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.114+0000 7fecda7fc640 1 --2- 172.21.15.178:0/917127583 >> v2:172.21.15.204:6812/28210 conn(0x7feccc0396e0 0x7feccc03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.114+0000 7fecda7fc640 1 -- 172.21.15.178:0/917127583 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fece006ba80 con 0x7fecec09faa0 2024-09-15T01:48:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.115+0000 7fecf08af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.115+0000 7fecf08af640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.115+0000 7fecf08af640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:48:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.115+0000 7fecf08af640 1 -- 172.21.15.178:0/917127583 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec0a2400 con 0x7fecec09faa0 2024-09-15T01:48:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.115+0000 7fece9d9b640 1 --2- 172.21.15.178:0/917127583 >> v2:172.21.15.204:6812/28210 conn(0x7feccc0396e0 0x7feccc03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.115+0000 7fece9d9b640 10 monclient: get_auth_request con 0x7feccc0396e0 auth_method 0 2024-09-15T01:48:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.115+0000 7fece9d9b640 1 --2- 172.21.15.178:0/917127583 >> v2:172.21.15.204:6812/28210 conn(0x7feccc0396e0 0x7feccc03bb90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fecec09b4f0 tx=0x7fecdc0091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.119+0000 7fecda7fc640 1 -- 172.21.15.178:0/917127583 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fecec0a2400 con 0x7fecec09faa0 2024-09-15T01:48:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.119+0000 7fecda7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.119+0000 7fecda7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:03.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.332+0000 7fecf08af640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:48:03.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.332+0000 7fecf08af640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:48:03.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.332+0000 7fecf08af640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:48:03.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.332+0000 7fecf08af640 1 -- 172.21.15.178:0/917127583 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fecec0a2400 con 0x7fecec09faa0 2024-09-15T01:48:03.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.333+0000 7fecda7fc640 1 -- 172.21.15.178:0/917127583 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3185 (secure 0 0 0) 0x7fecec0a2400 con 0x7fecec09faa0 2024-09-15T01:48:03.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.333+0000 7fecda7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:48:03.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.333+0000 7fecda7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:03.335 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":150,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":12,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:48:03.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.334+0000 7fecf08af640 1 -- 172.21.15.178:0/917127583 >> v2:172.21.15.204:6812/28210 conn(0x7feccc0396e0 msgr2=0x7feccc03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:03.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.334+0000 7fecf08af640 1 --2- 172.21.15.178:0/917127583 >> v2:172.21.15.204:6812/28210 conn(0x7feccc0396e0 0x7feccc03bb90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fecec09b4f0 tx=0x7fecdc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:48:03.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.334+0000 7fecf08af640 10 monclient: shutdown 2024-09-15T01:48:03.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.334+0000 7fecf08af640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:03.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.334+0000 7fecf08af640 1 -- 172.21.15.178:0/917127583 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fecec09faa0 msgr2=0x7fecec09b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:03.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.334+0000 7fecf08af640 1 --2- 172.21.15.178:0/917127583 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fecec09faa0 0x7fecec09b5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fece0009720 tx=0x7fece0032000 comp rx=0 tx=0).stop 2024-09-15T01:48:03.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.334+0000 7fecf08af640 1 -- 172.21.15.178:0/917127583 shutdown_connections 2024-09-15T01:48:03.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.334+0000 7fecf08af640 1 --2- 172.21.15.178:0/917127583 >> v2:172.21.15.204:6812/28210 conn(0x7feccc0396e0 0x7feccc03bb90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:03.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.334+0000 7fecf08af640 1 --2- 172.21.15.178:0/917127583 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fecec0a08c0 0x7fecec1319d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:03.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.334+0000 7fecf08af640 1 --2- 172.21.15.178:0/917127583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fecec0a2930 0x7fecec142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:03.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.334+0000 7fecf08af640 1 --2- 172.21.15.178:0/917127583 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fecec09faa0 0x7fecec09b5a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:03.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.334+0000 7fecf08af640 1 -- 172.21.15.178:0/917127583 >> 172.21.15.178:0/917127583 conn(0x7fecec0a5ed0 msgr2=0x7fecec0a4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:03.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.335+0000 7fecf08af640 1 -- 172.21.15.178:0/917127583 shutdown_connections 2024-09-15T01:48:03.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:03.335+0000 7fecf08af640 1 -- 172.21.15.178:0/917127583 wait complete. 2024-09-15T01:48:06.348 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:48:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.451+0000 7fd875257640 10 monclient: get_monmap_and_config 2024-09-15T01:48:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.451+0000 7fd875257640 10 monclient: build_initial_monmap 2024-09-15T01:48:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.451+0000 7fd875257640 10 monclient: monmap: 2024-09-15T01:48:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:48:06.452015+0000 2024-09-15T01:48:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:48:06.452015+0000 2024-09-15T01:48:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:06.455 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.451+0000 7fd875257640 1 Processor -- start 2024-09-15T01:48:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.451+0000 7fd875257640 1 -- start start 2024-09-15T01:48:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.451+0000 7fd875257640 10 monclient: init 2024-09-15T01:48:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 10 monclient: _add_conns ranks=[9,7,2,1,6,0,4,5,10,8,3] 2024-09-15T01:48:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd8700a2930 0x7fd870132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 10 monclient(hunting): picked mon.noname-a con 0x7fd8700a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:48:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 10 monclient(hunting): picked mon.noname-i con 0x7fd87009faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:48:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 10 monclient(hunting): picked mon.noname-b con 0x7fd8700a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:48:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87009e0f0 con 0x7fd8700a2930 2024-09-15T01:48:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87009dd80 con 0x7fd8700a08c0 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd8700a2d00 con 0x7fd87009faa0 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd875257640 10 monclient(hunting): authenticate will time out at 1691.202294s 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86ed9d640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7fd8700a2930 msgr2=0x7fd870132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86ed9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd8700a2930 0x7fd870132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86f59e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86f59e640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:49360/0 (socket says 172.21.15.178:49360) 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 learned_addr learned my addr 172.21.15.178:0/72045997 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86f59e640 10 monclient(hunting): get_auth_request con 0x7fd8700a08c0 auth_method 0 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 msgr2=0x7fd8700a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:48:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 msgr2=0x7fd8700a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86659c640 10 monclient(hunting): get_auth_request con 0x7fd87009faa0 auth_method 0 2024-09-15T01:48:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86659c640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 msgr2=0x7fd87009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:48:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86659c640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 msgr2=0x7fd87009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.452+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:06.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86ed9d640 1 -- 172.21.15.178:0/72045997 >> v2:172.21.15.178:3300/0 conn(0x7fd8700a2930 msgr2=0x7fd870132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:06.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86ed9d640 1 --2- 172.21.15.178:0/72045997 >> v2:172.21.15.178:3300/0 conn(0x7fd8700a2930 0x7fd870132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:48:06.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:06.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86f59e640 10 monclient(hunting): get_auth_request con 0x7fd8700a08c0 auth_method 0 2024-09-15T01:48:06.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:06.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:06.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:48:06.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 msgr2=0x7fd8700a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:48:06.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 msgr2=0x7fd8700a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:06.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:06.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:48:06.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:06.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86659c640 10 monclient(hunting): get_auth_request con 0x7fd87009faa0 auth_method 0 2024-09-15T01:48:06.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:06.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:06.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:48:06.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86659c640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 msgr2=0x7fd87009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:48:06.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86659c640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 msgr2=0x7fd87009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:06.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:06.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:06.653+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:48:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86ed9d640 1 -- 172.21.15.178:0/72045997 >> v2:172.21.15.178:3300/0 conn(0x7fd8700a2930 msgr2=0x7fd870132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86ed9d640 1 --2- 172.21.15.178:0/72045997 >> v2:172.21.15.178:3300/0 conn(0x7fd8700a2930 0x7fd870132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:48:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86f59e640 10 monclient(hunting): get_auth_request con 0x7fd8700a08c0 auth_method 0 2024-09-15T01:48:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:48:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 msgr2=0x7fd8700a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:48:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 msgr2=0x7fd8700a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:48:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86659c640 10 monclient(hunting): get_auth_request con 0x7fd87009faa0 auth_method 0 2024-09-15T01:48:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:48:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86659c640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 msgr2=0x7fd87009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:48:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86659c640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 msgr2=0x7fd87009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.054+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:48:07.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86ed9d640 1 -- 172.21.15.178:0/72045997 >> v2:172.21.15.178:3300/0 conn(0x7fd8700a2930 msgr2=0x7fd870132de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:07.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86ed9d640 1 --2- 172.21.15.178:0/72045997 >> v2:172.21.15.178:3300/0 conn(0x7fd8700a2930 0x7fd870132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:48:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86f59e640 10 monclient(hunting): get_auth_request con 0x7fd8700a08c0 auth_method 0 2024-09-15T01:48:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:48:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 msgr2=0x7fd8700a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:48:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 msgr2=0x7fd8700a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:48:07.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:07.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86659c640 10 monclient(hunting): get_auth_request con 0x7fd87009faa0 auth_method 0 2024-09-15T01:48:07.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:07.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:07.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:48:07.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86659c640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 msgr2=0x7fd87009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T01:48:07.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86659c640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 msgr2=0x7fd87009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:07.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:07.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:07.855+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:48:09.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 10 monclient(hunting): tick 2024-09-15T01:48:09.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 1 monclient(hunting): continuing hunt 2024-09-15T01:48:09.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:48:09.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 msgr2=0x7fd87009fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:09.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fd87009faa0 0x7fd87009fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:09.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 msgr2=0x7fd8700a0d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:09.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8700a08c0 0x7fd8700a0d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:09.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 1 -- 172.21.15.178:0/72045997 >> v2:172.21.15.178:3300/0 conn(0x7fd8700a2930 msgr2=0x7fd870132de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:09.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 1 --2- 172.21.15.178:0/72045997 >> v2:172.21.15.178:3300/0 conn(0x7fd8700a2930 0x7fd870132de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:09.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 10 monclient: _add_conns ranks=[1,10,4,6,5,3,8,0] 2024-09-15T01:48:09.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd8500037b0 addr v1:172.21.15.204:6789/0 2024-09-15T01:48:09.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 1 --2- 172.21.15.178:0/72045997 >> v2:172.21.15.204:3300/0 conn(0x7fd8500073e0 0x7fd850009890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:09.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 10 monclient(hunting): picked mon.noname-f con 0x7fd8500073e0 addr v2:172.21.15.204:3300/0 2024-09-15T01:48:09.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 10 monclient(hunting): picked mon.noname-c con 0x7fd850009f20 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 1 -- 172.21.15.178:0/72045997 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87004bf60 con 0x7fd8500037b0 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 1 -- 172.21.15.178:0/72045997 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87009e0f0 con 0x7fd8500073e0 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 1 -- 172.21.15.178:0/72045997 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd87009dd80 con 0x7fd850009f20 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86d59a640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86659c640 1 -- 172.21.15.178:0/72045997 >> v2:172.21.15.204:3300/0 conn(0x7fd8500073e0 msgr2=0x7fd850009890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86ed9d640 1 -- 172.21.15.178:0/72045997 >> v1:172.21.15.204:6789/0 conn(0x7fd8500037b0 legacy=0x7fd850005bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> v2:172.21.15.204:3300/0 conn(0x7fd8500073e0 0x7fd850009890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86f59e640 10 monclient(hunting): get_auth_request con 0x7fd850009f20 auth_method 0 2024-09-15T01:48:09.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.452+0000 7fd86f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.453+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 msgr2=0x7fd85000c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:48:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.453+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 msgr2=0x7fd85000c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.453+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:09.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.453+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:09.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.653+0000 7fd86659c640 1 -- 172.21.15.178:0/72045997 >> v2:172.21.15.204:3300/0 conn(0x7fd8500073e0 msgr2=0x7fd850009890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:09.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.653+0000 7fd86ed9d640 1 -- 172.21.15.178:0/72045997 >> v1:172.21.15.204:6789/0 conn(0x7fd8500037b0 legacy=0x7fd850005bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:09.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.653+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> v2:172.21.15.204:3300/0 conn(0x7fd8500073e0 0x7fd850009890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:48:09.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.654+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:09.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.654+0000 7fd86f59e640 10 monclient(hunting): get_auth_request con 0x7fd850009f20 auth_method 0 2024-09-15T01:48:09.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.654+0000 7fd86f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:09.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.654+0000 7fd86f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:09.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.654+0000 7fd86f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:48:09.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.654+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 msgr2=0x7fd85000c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:48:09.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.654+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 msgr2=0x7fd85000c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:09.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.654+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:09.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:09.654+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:48:10.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.054+0000 7fd86ed9d640 1 -- 172.21.15.178:0/72045997 >> v1:172.21.15.204:6789/0 conn(0x7fd8500037b0 legacy=0x7fd850005bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:10.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.054+0000 7fd86659c640 1 -- 172.21.15.178:0/72045997 >> v2:172.21.15.204:3300/0 conn(0x7fd8500073e0 msgr2=0x7fd850009890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:10.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.054+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> v2:172.21.15.204:3300/0 conn(0x7fd8500073e0 0x7fd850009890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:48:10.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.055+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:10.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.055+0000 7fd86f59e640 10 monclient(hunting): get_auth_request con 0x7fd850009f20 auth_method 0 2024-09-15T01:48:10.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.055+0000 7fd86f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:10.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.055+0000 7fd86f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:10.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.055+0000 7fd86f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:48:10.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.055+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 msgr2=0x7fd85000c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:48:10.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.055+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 msgr2=0x7fd85000c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:10.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.055+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:10.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.055+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:48:10.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.855+0000 7fd86ed9d640 1 -- 172.21.15.178:0/72045997 >> v1:172.21.15.204:6789/0 conn(0x7fd8500037b0 legacy=0x7fd850005bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:10.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.855+0000 7fd86659c640 1 -- 172.21.15.178:0/72045997 >> v2:172.21.15.204:3300/0 conn(0x7fd8500073e0 msgr2=0x7fd850009890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:10.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.855+0000 7fd86659c640 1 --2- 172.21.15.178:0/72045997 >> v2:172.21.15.204:3300/0 conn(0x7fd8500073e0 0x7fd850009890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:48:10.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:10.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 10 monclient(hunting): get_auth_request con 0x7fd850009f20 auth_method 0 2024-09-15T01:48:10.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:10.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:10.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:48:10.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:10.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:10.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:10.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 10 monclient(hunting): handle_auth_done global_id 44878 payload 274 2024-09-15T01:48:10.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> v2:172.21.15.204:3300/0 conn(0x7fd8500073e0 msgr2=0x7fd850009890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:10.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> v2:172.21.15.204:3300/0 conn(0x7fd8500073e0 0x7fd850009890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:10.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 >> v1:172.21.15.204:6789/0 conn(0x7fd8500037b0 legacy=0x7fd850005bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:10.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 10 monclient: _finish_hunting 0 2024-09-15T01:48:10.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 1 monclient: found mon.noname-c 2024-09-15T01:48:10.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:10.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:48:10.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.856+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85000db60 con 0x7fd850009f20 2024-09-15T01:48:10.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.857+0000 7fd86f59e640 1 -- 172.21.15.178:0/72045997 reap_dead start 2024-09-15T01:48:10.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.857+0000 7fd86f59e640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd868009450 tx=0x7fd868009040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=75036698907d71da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:10.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.857+0000 7fd86e59c640 1 -- 172.21.15.178:0/72045997 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd86802ce70 con 0x7fd850009f20 2024-09-15T01:48:10.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.857+0000 7fd86e59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:10.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.857+0000 7fd86e59c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:48:10.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd86e59c640 10 monclient: dump: 2024-09-15T01:48:10.864 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:10.864 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:10.864 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:10.864 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:10.864 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd86e59c640 10 monclient: _finish_auth 0 2024-09-15T01:48:10.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd86e59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd875257640 5 monclient: authenticate success, global_id 44878 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd875257640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd86e59c640 1 -- 172.21.15.178:0/72045997 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd868030070 con 0x7fd850009f20 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd86e59c640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd875257640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd875257640 4 set_mon_vals no callback set 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd86e59c640 1 -- 172.21.15.178:0/72045997 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd868036d10 con 0x7fd850009f20 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd86e59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd86e59c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd86e59c640 10 monclient: dump: 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:10.866 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:10.867 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:10.867 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:10.867 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:10.867 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:10.867 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:10.867 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:10.867 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:10.867 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:10.867 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:10.868 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:10.868 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:10.868 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:10.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd875257640 10 monclient: shutdown 2024-09-15T01:48:10.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd875257640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:10.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd875257640 1 -- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 msgr2=0x7fd85000c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:10.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd875257640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd868009450 tx=0x7fd868009040 comp rx=0 tx=0).stop 2024-09-15T01:48:10.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd875257640 1 -- 172.21.15.178:0/72045997 shutdown_connections 2024-09-15T01:48:10.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd875257640 1 --2- 172.21.15.178:0/72045997 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fd850009f20 0x7fd85000c3d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:10.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.858+0000 7fd875257640 1 -- 172.21.15.178:0/72045997 >> 172.21.15.178:0/72045997 conn(0x7fd8700a5ed0 msgr2=0x7fd8700a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:10.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.859+0000 7fd875257640 1 -- 172.21.15.178:0/72045997 shutdown_connections 2024-09-15T01:48:10.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.859+0000 7fd875257640 1 -- 172.21.15.178:0/72045997 wait complete. 2024-09-15T01:48:10.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.859+0000 7fd875257640 10 monclient: build_initial_monmap 2024-09-15T01:48:10.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.859+0000 7fd875257640 10 monclient: monmap: 2024-09-15T01:48:10.869 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:10.869 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:10.869 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:48:10.869 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:48:10.869 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:10.870 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:48:10.870 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:10.870 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:10.870 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:10.870 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:10.870 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:10.870 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:10.870 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:10.870 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:10.870 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:10.870 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:10.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 1 Processor -- start 2024-09-15T01:48:10.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 1 -- start start 2024-09-15T01:48:10.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 10 monclient: init 2024-09-15T01:48:10.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:10.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 10 monclient: _add_conns ranks=[3,0,1,6,4,2,5,7,8] 2024-09-15T01:48:10.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd850009f20 0x7fd87009b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:10.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 10 monclient(hunting): picked mon.noname-g con 0x7fd850009f20 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:10.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd870136f00 0x7fd870131750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:10.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 10 monclient(hunting): picked mon.noname-a con 0x7fd870136f00 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:48:10.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd870131c90 0x7fd87011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:10.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 10 monclient(hunting): picked mon.noname-f con 0x7fd870131c90 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:48:10.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:10.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8701429b0 con 0x7fd870136f00 2024-09-15T01:48:10.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:10.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87009d630 con 0x7fd870131c90 2024-09-15T01:48:10.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:10.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87011ebe0 con 0x7fd850009f20 2024-09-15T01:48:10.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:10.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd875257640 10 monclient(hunting): authenticate will time out at 1695.610934s 2024-09-15T01:48:10.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd86659c640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd870136f00 msgr2=0x7fd870131750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:10.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.860+0000 7fd86659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd870136f00 0x7fd870131750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:10.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86f59e640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd870131c90 msgr2=0x7fd87011c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:10.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86f59e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd870131c90 0x7fd87011c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:10.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd850009f20 0x7fd87009b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:10.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd850009f20 0x7fd87009b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:57264/0 (socket says 172.21.15.178:57264) 2024-09-15T01:48:10.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 1 -- 172.21.15.178:0/2619917555 learned_addr learned my addr 172.21.15.178:0/2619917555 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:10.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 10 monclient(hunting): get_auth_request con 0x7fd850009f20 auth_method 0 2024-09-15T01:48:10.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:10.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:10.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:10.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:10.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:10.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:10.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 10 monclient(hunting): handle_auth_done global_id 64470 payload 824 2024-09-15T01:48:10.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 1 -- 172.21.15.178:0/2619917555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd870131c90 msgr2=0x7fd87011c530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:10.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 1 --2- 172.21.15.178:0/2619917555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd870131c90 0x7fd87011c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:10.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 1 -- 172.21.15.178:0/2619917555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd870136f00 msgr2=0x7fd870131750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:10.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 1 --2- 172.21.15.178:0/2619917555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd870136f00 0x7fd870131750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:10.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:48:10.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 1 monclient: found mon.noname-g 2024-09-15T01:48:10.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:10.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:48:10.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 1 -- 172.21.15.178:0/2619917555 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868009930 con 0x7fd850009f20 2024-09-15T01:48:10.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.861+0000 7fd86ed9d640 1 --2- 172.21.15.178:0/2619917555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd850009f20 0x7fd87009b3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd86000bd10 tx=0x7fd860009560 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:10.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd8677fe640 1 -- 172.21.15.178:0/2619917555 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd8600153e0 con 0x7fd850009f20 2024-09-15T01:48:10.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd8677fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:10.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd8677fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:48:10.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd8677fe640 10 monclient: dump: 2024-09-15T01:48:10.882 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:10.882 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:10.883 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:10.883 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:10.883 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:10.883 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:10.883 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:10.883 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:10.883 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:10.883 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:10.883 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:10.883 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:10.883 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:10.884 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:10.884 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:10.884 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:10.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd8677fe640 10 monclient: _finish_auth 0 2024-09-15T01:48:10.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd8677fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:10.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd8677fe640 1 -- 172.21.15.178:0/2619917555 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd860015560 con 0x7fd850009f20 2024-09-15T01:48:10.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd875257640 5 monclient: authenticate success, global_id 64470 2024-09-15T01:48:10.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd8677fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:10.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd875257640 10 monclient: _renew_subs 2024-09-15T01:48:10.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd8677fe640 1 -- 172.21.15.178:0/2619917555 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd860015b50 con 0x7fd850009f20 2024-09-15T01:48:10.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd875257640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:10.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd875257640 1 -- 172.21.15.178:0/2619917555 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8700a2390 con 0x7fd850009f20 2024-09-15T01:48:10.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd8677fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:10.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd8677fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:48:10.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd8677fe640 10 monclient: dump: 2024-09-15T01:48:10.885 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:10.885 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:10.885 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:10.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd86dd9b640 4 set_mon_vals no callback set 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd875257640 10 monclient: _renew_subs 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd875257640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.862+0000 7fd875257640 1 -- 172.21.15.178:0/2619917555 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87009bdd0 con 0x7fd850009f20 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.863+0000 7fd8677fe640 1 -- 172.21.15.178:0/2619917555 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fd860012020 con 0x7fd850009f20 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.863+0000 7fd875257640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.863+0000 7fd875257640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.863+0000 7fd875257640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.863+0000 7fd8677fe640 1 --2- 172.21.15.178:0/2619917555 >> v2:172.21.15.204:6812/28210 conn(0x7fd85003a940 0x7fd85003cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.863+0000 7fd875257640 1 -- 172.21.15.178:0/2619917555 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87004b720 con 0x7fd850009f20 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.863+0000 7fd8677fe640 1 -- 172.21.15.178:0/2619917555 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fd86004b7e0 con 0x7fd850009f20 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.863+0000 7fd86659c640 1 --2- 172.21.15.178:0/2619917555 >> v2:172.21.15.204:6812/28210 conn(0x7fd85003a940 0x7fd85003cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.863+0000 7fd86659c640 10 monclient: get_auth_request con 0x7fd85003a940 auth_method 0 2024-09-15T01:48:10.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.864+0000 7fd86659c640 1 --2- 172.21.15.178:0/2619917555 >> v2:172.21.15.204:6812/28210 conn(0x7fd85003a940 0x7fd85003cdf0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd858006fd0 tx=0x7fd858008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:10.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.868+0000 7fd8677fe640 1 -- 172.21.15.178:0/2619917555 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd860015cd0 con 0x7fd850009f20 2024-09-15T01:48:10.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.868+0000 7fd8677fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:10.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:10.868+0000 7fd8677fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:11.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.087+0000 7fd875257640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:48:11.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.087+0000 7fd875257640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:48:11.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.087+0000 7fd875257640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:11.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.087+0000 7fd875257640 1 -- 172.21.15.178:0/2619917555 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd87009cef0 con 0x7fd850009f20 2024-09-15T01:48:11.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.088+0000 7fd8677fe640 1 -- 172.21.15.178:0/2619917555 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3166 (secure 0 0 0) 0x7fd860024070 con 0x7fd850009f20 2024-09-15T01:48:11.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.088+0000 7fd8677fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:48:11.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.088+0000 7fd8677fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:11.091 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":154,"quorum":[2,3,4,5,7,8],"quorum_names":["b","g","c","h","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:48:11.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 1 -- 172.21.15.178:0/2619917555 >> v2:172.21.15.204:6812/28210 conn(0x7fd85003a940 msgr2=0x7fd85003cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:11.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 1 --2- 172.21.15.178:0/2619917555 >> v2:172.21.15.204:6812/28210 conn(0x7fd85003a940 0x7fd85003cdf0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd858006fd0 tx=0x7fd858008040 comp rx=0 tx=0).stop 2024-09-15T01:48:11.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 10 monclient: shutdown 2024-09-15T01:48:11.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:11.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 1 -- 172.21.15.178:0/2619917555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd850009f20 msgr2=0x7fd87009b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:11.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 1 --2- 172.21.15.178:0/2619917555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd850009f20 0x7fd87009b3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd86000bd10 tx=0x7fd860009560 comp rx=0 tx=0).stop 2024-09-15T01:48:11.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 1 -- 172.21.15.178:0/2619917555 shutdown_connections 2024-09-15T01:48:11.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 1 --2- 172.21.15.178:0/2619917555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd870131c90 0x7fd87011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:11.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 1 --2- 172.21.15.178:0/2619917555 >> v2:172.21.15.204:6812/28210 conn(0x7fd85003a940 0x7fd85003cdf0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:11.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 1 --2- 172.21.15.178:0/2619917555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd870136f00 0x7fd870131750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:11.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 1 --2- 172.21.15.178:0/2619917555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd850009f20 0x7fd87009b3c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:11.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 1 -- 172.21.15.178:0/2619917555 >> 172.21.15.178:0/2619917555 conn(0x7fd8700a5ed0 msgr2=0x7fd87009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:11.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 1 -- 172.21.15.178:0/2619917555 shutdown_connections 2024-09-15T01:48:11.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:11.090+0000 7fd875257640 1 -- 172.21.15.178:0/2619917555 wait complete. 2024-09-15T01:48:14.104 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:48:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.207+0000 7ff4c439d640 10 monclient: get_monmap_and_config 2024-09-15T01:48:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.207+0000 7ff4c439d640 10 monclient: build_initial_monmap 2024-09-15T01:48:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.207+0000 7ff4c439d640 10 monclient: monmap: 2024-09-15T01:48:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:48:14.208084+0000 2024-09-15T01:48:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:48:14.208084+0000 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:14.210 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:14.211 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.207+0000 7ff4c439d640 1 Processor -- start 2024-09-15T01:48:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.207+0000 7ff4c439d640 1 -- start start 2024-09-15T01:48:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.207+0000 7ff4c439d640 10 monclient: init 2024-09-15T01:48:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 10 monclient: _add_conns ranks=[0,9,5,2,7,3,10,8,1,6,4] 2024-09-15T01:48:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff4bc0a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:48:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff4bc132de0 0x7ff4bc137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 10 monclient(hunting): picked mon.noname-a con 0x7ff4bc132de0 addr v2:172.21.15.178:3300/0 2024-09-15T01:48:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 10 monclient(hunting): picked mon.noname-h con 0x7ff4bc1380e0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:48:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4bc04bf60 con 0x7ff4bc0a2930 2024-09-15T01:48:14.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4bc09dd80 con 0x7ff4bc132de0 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc0a2d00 con 0x7ff4bc1380e0 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c439d640 10 monclient(hunting): authenticate will time out at 1698.958642s 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c2139640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7ff4bc0a2930 legacy=0x7ff4bc09faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c1938640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7ff4bc132de0 msgr2=0x7ff4bc137ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c1938640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff4bc132de0 0x7ff4bc137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c293a640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c293a640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:39924/0 (socket says 172.21.15.178:39924) 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 learned_addr learned my addr 172.21.15.178:0/849406701 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c293a640 10 monclient(hunting): get_auth_request con 0x7ff4bc1380e0 auth_method 0 2024-09-15T01:48:14.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c293a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c293a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c293a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.208+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 msgr2=0x7ff4bc13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T01:48:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.209+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 msgr2=0x7ff4bc13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.209+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:14.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.209+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.409+0000 7ff4c1938640 1 -- 172.21.15.178:0/849406701 >> v2:172.21.15.178:3300/0 conn(0x7ff4bc132de0 msgr2=0x7ff4bc137ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.409+0000 7ff4c2139640 1 -- 172.21.15.178:0/849406701 >> v1:172.21.15.178:6789/0 conn(0x7ff4bc0a2930 legacy=0x7ff4bc09faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:48:14.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.409+0000 7ff4c1938640 1 --2- 172.21.15.178:0/849406701 >> v2:172.21.15.178:3300/0 conn(0x7ff4bc132de0 0x7ff4bc137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:48:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.410+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.410+0000 7ff4c293a640 10 monclient(hunting): get_auth_request con 0x7ff4bc1380e0 auth_method 0 2024-09-15T01:48:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.410+0000 7ff4c293a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.410+0000 7ff4c293a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.410+0000 7ff4c293a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:48:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.410+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 msgr2=0x7ff4bc13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:48:14.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.410+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 msgr2=0x7ff4bc13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.410+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:14.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.410+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:48:14.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.810+0000 7ff4c1938640 1 -- 172.21.15.178:0/849406701 >> v2:172.21.15.178:3300/0 conn(0x7ff4bc132de0 msgr2=0x7ff4bc137ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:14.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.810+0000 7ff4c2139640 1 -- 172.21.15.178:0/849406701 >> v1:172.21.15.178:6789/0 conn(0x7ff4bc0a2930 legacy=0x7ff4bc09faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:48:14.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.810+0000 7ff4c1938640 1 --2- 172.21.15.178:0/849406701 >> v2:172.21.15.178:3300/0 conn(0x7ff4bc132de0 0x7ff4bc137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:48:14.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.811+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:14.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.811+0000 7ff4c293a640 10 monclient(hunting): get_auth_request con 0x7ff4bc1380e0 auth_method 0 2024-09-15T01:48:14.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.811+0000 7ff4c293a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:14.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.811+0000 7ff4c293a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:14.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.811+0000 7ff4c293a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:48:14.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.811+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 msgr2=0x7ff4bc13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:48:14.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.811+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 msgr2=0x7ff4bc13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:14.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.811+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:14.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:14.811+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:48:15.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:15.611+0000 7ff4c2139640 1 -- 172.21.15.178:0/849406701 >> v1:172.21.15.178:6789/0 conn(0x7ff4bc0a2930 legacy=0x7ff4bc09faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:48:15.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:15.611+0000 7ff4c1938640 1 -- 172.21.15.178:0/849406701 >> v2:172.21.15.178:3300/0 conn(0x7ff4bc132de0 msgr2=0x7ff4bc137ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:15.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:15.611+0000 7ff4c1938640 1 --2- 172.21.15.178:0/849406701 >> v2:172.21.15.178:3300/0 conn(0x7ff4bc132de0 0x7ff4bc137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:48:15.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:15.612+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:15.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:15.612+0000 7ff4c293a640 10 monclient(hunting): get_auth_request con 0x7ff4bc1380e0 auth_method 0 2024-09-15T01:48:15.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:15.612+0000 7ff4c293a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:15.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:15.612+0000 7ff4c293a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:15.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:15.612+0000 7ff4c293a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:48:15.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:15.612+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 msgr2=0x7ff4bc13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:48:15.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:15.612+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 msgr2=0x7ff4bc13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:48:15.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:15.612+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:15.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:15.612+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:48:17.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 10 monclient(hunting): tick 2024-09-15T01:48:17.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 1 monclient(hunting): continuing hunt 2024-09-15T01:48:17.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:48:17.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 1 -- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 msgr2=0x7ff4bc13a4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:17.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7ff4bc1380e0 0x7ff4bc13a4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 1 -- 172.21.15.178:0/849406701 >> v2:172.21.15.178:3300/0 conn(0x7ff4bc132de0 msgr2=0x7ff4bc137ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:17.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 1 --2- 172.21.15.178:0/849406701 >> v2:172.21.15.178:3300/0 conn(0x7ff4bc132de0 0x7ff4bc137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 1 -- 172.21.15.178:0/849406701 >> v1:172.21.15.178:6789/0 conn(0x7ff4bc0a2930 legacy=0x7ff4bc09faa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:17.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 10 monclient: _add_conns ranks=[1,3,4,6,8,7,2,10] 2024-09-15T01:48:17.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff4a80037b0 addr v1:172.21.15.204:6789/0 2024-09-15T01:48:17.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a80073b0 0x7ff4a8009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 10 monclient(hunting): picked mon.noname-g con 0x7ff4a80073b0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:17.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff4a8009ee0 0x7ff4a800c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 10 monclient(hunting): picked mon.noname-c con 0x7ff4a8009ee0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 1 -- 172.21.15.178:0/849406701 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4bc1331b0 con 0x7ff4a80037b0 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 1 -- 172.21.15.178:0/849406701 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc09dd80 con 0x7ff4a80073b0 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 1 -- 172.21.15.178:0/849406701 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc0a2d00 con 0x7ff4a8009ee0 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4bbfff640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.208+0000 7ff4c2139640 1 -- 172.21.15.178:0/849406701 >> v1:172.21.15.204:6789/0 conn(0x7ff4a80037b0 legacy=0x7ff4a8005bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff4a8009ee0 0x7ff4a800c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c1938640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a80073b0 0x7ff4a8009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 10 monclient(hunting): get_auth_request con 0x7ff4a8009ee0 auth_method 0 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:17.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c1938640 10 monclient(hunting): get_auth_request con 0x7ff4a80073b0 auth_method 0 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c1938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c1938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c1938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c1938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c1938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c1938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 10 monclient(hunting): handle_auth_done global_id 44887 payload 274 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a80073b0 msgr2=0x7ff4a8009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:17.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4a80073b0 0x7ff4a8009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 >> v1:172.21.15.204:6789/0 conn(0x7ff4a80037b0 legacy=0x7ff4a8005bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:17.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 10 monclient: _finish_hunting 0 2024-09-15T01:48:17.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 1 monclient: found mon.noname-c 2024-09-15T01:48:17.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:17.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:48:17.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.209+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc04bf60 con 0x7ff4a8009ee0 2024-09-15T01:48:17.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c293a640 1 -- 172.21.15.178:0/849406701 reap_dead start 2024-09-15T01:48:17.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c293a640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff4a8009ee0 0x7ff4a800c390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009c90 tx=0x7ff4b4009040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1e5f51e50e767da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c1137640 1 -- 172.21.15.178:0/849406701 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4b402bbe0 con 0x7ff4a8009ee0 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c1137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c1137640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c1137640 10 monclient: dump: 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:17.221 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c1137640 10 monclient: _finish_auth 0 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c1137640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c1137640 1 -- 172.21.15.178:0/849406701 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b402f070 con 0x7ff4a8009ee0 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c439d640 5 monclient: authenticate success, global_id 44887 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c439d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c1137640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:17.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c439d640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c1137640 1 -- 172.21.15.178:0/849406701 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4b402bd40 con 0x7ff4a8009ee0 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c439d640 4 set_mon_vals no callback set 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c1137640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c1137640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.210+0000 7ff4c1137640 10 monclient: dump: 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:17.223 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.211+0000 7ff4c439d640 10 monclient: shutdown 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.211+0000 7ff4c439d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.211+0000 7ff4c439d640 1 -- 172.21.15.178:0/849406701 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff4a8009ee0 msgr2=0x7ff4a800c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.211+0000 7ff4c439d640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff4a8009ee0 0x7ff4a800c390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009c90 tx=0x7ff4b4009040 comp rx=0 tx=0).stop 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.211+0000 7ff4c439d640 1 -- 172.21.15.178:0/849406701 shutdown_connections 2024-09-15T01:48:17.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.211+0000 7ff4c439d640 1 --2- 172.21.15.178:0/849406701 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7ff4a8009ee0 0x7ff4a800c390 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.211+0000 7ff4c439d640 1 -- 172.21.15.178:0/849406701 >> 172.21.15.178:0/849406701 conn(0x7ff4bc0a5ed0 msgr2=0x7ff4bc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:17.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.211+0000 7ff4c439d640 1 -- 172.21.15.178:0/849406701 shutdown_connections 2024-09-15T01:48:17.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.211+0000 7ff4c439d640 1 -- 172.21.15.178:0/849406701 wait complete. 2024-09-15T01:48:17.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.212+0000 7ff4c439d640 10 monclient: build_initial_monmap 2024-09-15T01:48:17.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.212+0000 7ff4c439d640 10 monclient: monmap: 2024-09-15T01:48:17.225 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:17.225 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:17.225 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:48:17.225 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:48:17.225 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:17.225 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:48:17.225 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:17.226 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:17.226 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:17.226 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:17.226 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:17.226 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:17.226 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:17.226 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:17.226 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:17.226 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.212+0000 7ff4c439d640 1 Processor -- start 2024-09-15T01:48:17.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.212+0000 7ff4c439d640 1 -- start start 2024-09-15T01:48:17.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.212+0000 7ff4c439d640 10 monclient: init 2024-09-15T01:48:17.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.212+0000 7ff4c439d640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:17.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 10 monclient: _add_conns ranks=[1,3,2,8,7,6,4,0,5] 2024-09-15T01:48:17.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a8009ee0 0x7ff4bc1324c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 10 monclient(hunting): picked mon.noname-f con 0x7ff4a8009ee0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:48:17.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4bc11e130 0x7ff4bc09a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 10 monclient(hunting): picked mon.noname-g con 0x7ff4bc11e130 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:17.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4bc09aaa0 0x7ff4bc09aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 10 monclient(hunting): picked mon.noname-b con 0x7ff4bc09aaa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:48:17.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc09b430 con 0x7ff4a8009ee0 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc0a2390 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc09d630 con 0x7ff4bc11e130 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c439d640 10 monclient(hunting): authenticate will time out at 1701.963077s 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c2139640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a8009ee0 msgr2=0x7ff4bc1324c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4bc09aaa0 0x7ff4bc09aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c2139640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a8009ee0 0x7ff4bc1324c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4bc09aaa0 0x7ff4bc09aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:35110/0 (socket says 172.21.15.178:35110) 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 1 -- 172.21.15.178:0/1014717403 learned_addr learned my addr 172.21.15.178:0/1014717403 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:17.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 10 monclient(hunting): get_auth_request con 0x7ff4bc09aaa0 auth_method 0 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c1938640 1 --2- 172.21.15.178:0/1014717403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4bc11e130 0x7ff4bc09a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c1938640 10 monclient(hunting): get_auth_request con 0x7ff4bc11e130 auth_method 0 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c1938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c1938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c1938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c1938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c1938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:17.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c1938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 10 monclient(hunting): handle_auth_done global_id 35327 payload 824 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 1 -- 172.21.15.178:0/1014717403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4bc11e130 msgr2=0x7ff4bc09a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 1 --2- 172.21.15.178:0/1014717403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4bc11e130 0x7ff4bc09a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 1 -- 172.21.15.178:0/1014717403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a8009ee0 msgr2=0x7ff4bc1324c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 1 --2- 172.21.15.178:0/1014717403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a8009ee0 0x7ff4bc1324c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 10 monclient: _finish_hunting 0 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 1 monclient: found mon.noname-b 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 1 -- 172.21.15.178:0/1014717403 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40098f0 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.213+0000 7ff4c293a640 1 --2- 172.21.15.178:0/1014717403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4bc09aaa0 0x7ff4bc09aef0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009650 tx=0x7ff4b4033000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4ba7fc640 1 -- 172.21.15.178:0/1014717403 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4b4035d30 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4ba7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4ba7fc640 10 monclient: dump: 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:17.237 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4ba7fc640 10 monclient: _finish_auth 0 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4c439d640 5 monclient: authenticate success, global_id 35327 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4ba7fc640 1 -- 172.21.15.178:0/1014717403 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b400f040 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4c439d640 10 monclient: _renew_subs 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4c439d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4c439d640 1 -- 172.21.15.178:0/1014717403 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4bc09c5c0 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4ba7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:17.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4ba7fc640 1 -- 172.21.15.178:0/1014717403 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4b40408c0 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4ba7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4ba7fc640 10 monclient: dump: 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:17.239 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:17.240 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:17.240 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:17.240 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:17.240 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:17.240 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4c439d640 10 monclient: _renew_subs 2024-09-15T01:48:17.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4c439d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:48:17.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4c0936640 4 set_mon_vals no callback set 2024-09-15T01:48:17.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.214+0000 7ff4c439d640 1 -- 172.21.15.178:0/1014717403 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4bc09c800 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.215+0000 7ff4ba7fc640 1 -- 172.21.15.178:0/1014717403 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7ff4b4040440 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.215+0000 7ff4ba7fc640 1 --2- 172.21.15.178:0/1014717403 >> v2:172.21.15.204:6812/28210 conn(0x7ff4a803a850 0x7ff4a803cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.215+0000 7ff4c439d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:17.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.215+0000 7ff4c439d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:17.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.215+0000 7ff4c439d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:48:17.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.215+0000 7ff4ba7fc640 1 -- 172.21.15.178:0/1014717403 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7ff4b406ce10 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.215+0000 7ff4c439d640 1 -- 172.21.15.178:0/1014717403 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4bc04b720 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.215+0000 7ff4c2139640 1 --2- 172.21.15.178:0/1014717403 >> v2:172.21.15.204:6812/28210 conn(0x7ff4a803a850 0x7ff4a803cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.215+0000 7ff4c2139640 10 monclient: get_auth_request con 0x7ff4a803a850 auth_method 0 2024-09-15T01:48:17.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.216+0000 7ff4c2139640 1 --2- 172.21.15.178:0/1014717403 >> v2:172.21.15.204:6812/28210 conn(0x7ff4a803a850 0x7ff4a803cd00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac003940 tx=0x7ff4ac006e80 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:17.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.220+0000 7ff4ba7fc640 1 -- 172.21.15.178:0/1014717403 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff4b4098e40 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.220+0000 7ff4ba7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:17.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.220+0000 7ff4ba7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:17.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.439+0000 7ff4c439d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:48:17.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.439+0000 7ff4c439d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:48:17.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.439+0000 7ff4c439d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:48:17.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.439+0000 7ff4c439d640 1 -- 172.21.15.178:0/1014717403 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff4bc04b720 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.440+0000 7ff4ba7fc640 1 -- 172.21.15.178:0/1014717403 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3172 (secure 0 0 0) 0x7ff4b40358b0 con 0x7ff4bc09aaa0 2024-09-15T01:48:17.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.440+0000 7ff4ba7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:48:17.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.440+0000 7ff4ba7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:17.443 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":156,"quorum":[2,3,4,5,6,7,8],"quorum_names":["b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":1,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:48:17.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.441+0000 7ff4c439d640 1 -- 172.21.15.178:0/1014717403 >> v2:172.21.15.204:6812/28210 conn(0x7ff4a803a850 msgr2=0x7ff4a803cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:17.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.441+0000 7ff4c439d640 1 --2- 172.21.15.178:0/1014717403 >> v2:172.21.15.204:6812/28210 conn(0x7ff4a803a850 0x7ff4a803cd00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac003940 tx=0x7ff4ac006e80 comp rx=0 tx=0).stop 2024-09-15T01:48:17.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.442+0000 7ff4c439d640 10 monclient: shutdown 2024-09-15T01:48:17.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.442+0000 7ff4c439d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:17.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.442+0000 7ff4c439d640 1 -- 172.21.15.178:0/1014717403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4bc09aaa0 msgr2=0x7ff4bc09aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:17.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.442+0000 7ff4c439d640 1 --2- 172.21.15.178:0/1014717403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4bc09aaa0 0x7ff4bc09aef0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009650 tx=0x7ff4b4033000 comp rx=0 tx=0).stop 2024-09-15T01:48:17.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.442+0000 7ff4c439d640 1 -- 172.21.15.178:0/1014717403 shutdown_connections 2024-09-15T01:48:17.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.442+0000 7ff4c439d640 1 --2- 172.21.15.178:0/1014717403 >> v2:172.21.15.204:6812/28210 conn(0x7ff4a803a850 0x7ff4a803cd00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.442+0000 7ff4c439d640 1 --2- 172.21.15.178:0/1014717403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4bc09aaa0 0x7ff4bc09aef0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.442+0000 7ff4c439d640 1 --2- 172.21.15.178:0/1014717403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4bc11e130 0x7ff4bc09a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.442+0000 7ff4c439d640 1 --2- 172.21.15.178:0/1014717403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4a8009ee0 0x7ff4bc1324c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.442+0000 7ff4c439d640 1 -- 172.21.15.178:0/1014717403 >> 172.21.15.178:0/1014717403 conn(0x7ff4bc0a5ed0 msgr2=0x7ff4bc0a4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:17.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.442+0000 7ff4c439d640 1 -- 172.21.15.178:0/1014717403 shutdown_connections 2024-09-15T01:48:17.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.442+0000 7ff4c439d640 1 -- 172.21.15.178:0/1014717403 wait complete. 2024-09-15T01:48:17.454 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-15T01:48:17.455 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-09-15T01:48:17.455 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:48:17.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.546+0000 7f1c7e046640 10 monclient: get_monmap_and_config 2024-09-15T01:48:17.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.546+0000 7f1c7e046640 10 monclient: build_initial_monmap 2024-09-15T01:48:17.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.546+0000 7f1c7e046640 10 monclient: monmap: 2024-09-15T01:48:17.548 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:17.548 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:17.548 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:48:17.547075+0000 2024-09-15T01:48:17.548 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:48:17.547075+0000 2024-09-15T01:48:17.548 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:17.549 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.546+0000 7f1c7e046640 1 Processor -- start 2024-09-15T01:48:17.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 1 -- start start 2024-09-15T01:48:17.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 10 monclient: init 2024-09-15T01:48:17.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:17.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 10 monclient: _add_conns ranks=[2,6,5,8,9,0,10,3,7,4,1] 2024-09-15T01:48:17.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c780a2930 0x7f1c78132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 10 monclient(hunting): picked mon.noname-b con 0x7f1c780a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:48:17.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c7809faa0 0x7f1c7809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 10 monclient(hunting): picked mon.noname-d con 0x7f1c7809faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:48:17.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1c780a08c0 0x7f1c780a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 10 monclient(hunting): picked mon.noname-h con 0x7f1c780a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7809e0f0 con 0x7f1c780a2930 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7809dd80 con 0x7f1c780a08c0 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780a2d00 con 0x7f1c7809faa0 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c7e046640 10 monclient(hunting): authenticate will time out at 1702.297481s 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c780a2930 0x7f1c78132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c76ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c7809faa0 0x7f1c7809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c780a2930 0x7f1c78132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:35126/0 (socket says 172.21.15.178:35126) 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c76ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c7809faa0 0x7f1c7809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:38086/0 (socket says 172.21.15.178:38086) 2024-09-15T01:48:17.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 1 -- 172.21.15.178:0/2915256343 learned_addr learned my addr 172.21.15.178:0/2915256343 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 10 monclient(hunting): get_auth_request con 0x7f1c780a2930 auth_method 0 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c76ffd640 10 monclient(hunting): get_auth_request con 0x7f1c7809faa0 auth_method 0 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c76ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c76ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c76ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:17.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c76ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c76ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c77fff640 1 --2- 172.21.15.178:0/2915256343 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1c780a08c0 0x7f1c780a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 10 monclient(hunting): handle_auth_done global_id 35336 payload 274 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 1 -- 172.21.15.178:0/2915256343 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c7809faa0 msgr2=0x7f1c7809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 1 --2- 172.21.15.178:0/2915256343 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c7809faa0 0x7f1c7809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 1 -- 172.21.15.178:0/2915256343 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1c780a08c0 msgr2=0x7f1c780a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 1 --2- 172.21.15.178:0/2915256343 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1c780a08c0 0x7f1c780a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 10 monclient: _finish_hunting 0 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 1 monclient: found mon.noname-b 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c777fe640 1 -- 172.21.15.178:0/2915256343 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7804bf60 con 0x7f1c780a2930 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c77fff640 10 monclient: get_auth_request con 0x7f1c780a08c0 auth_method 0 2024-09-15T01:48:17.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c76ffd640 1 --2- 172.21.15.178:0/2915256343 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c7809faa0 0x7f1c7809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.547+0000 7f1c77fff640 1 --2- 172.21.15.178:0/2915256343 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1c780a08c0 0x7f1c780a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c777fe640 1 --2- 172.21.15.178:0/2915256343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c780a2930 0x7f1c78132de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009940 tx=0x7f1c680091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d55fb429a6e5a86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c767fc640 1 -- 172.21.15.178:0/2915256343 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c6802ac60 con 0x7f1c780a2930 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c767fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c767fc640 10 monclient: dump: 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:17.562 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c767fc640 10 monclient: _finish_auth 0 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c767fc640 1 -- 172.21.15.178:0/2915256343 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6802ade0 con 0x7f1c780a2930 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c7e046640 5 monclient: authenticate success, global_id 35336 2024-09-15T01:48:17.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c7e046640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:48:17.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c767fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:17.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c7e046640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:17.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c767fc640 1 -- 172.21.15.178:0/2915256343 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c68031ca0 con 0x7f1c780a2930 2024-09-15T01:48:17.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:17.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c7e046640 4 set_mon_vals no callback set 2024-09-15T01:48:17.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c767fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:48:17.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.548+0000 7f1c767fc640 10 monclient: dump: 2024-09-15T01:48:17.564 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:17.564 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:17.564 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:17.565 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 10 monclient: shutdown 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 1 -- 172.21.15.178:0/2915256343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c780a2930 msgr2=0x7f1c78132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 1 --2- 172.21.15.178:0/2915256343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c780a2930 0x7f1c78132de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009940 tx=0x7f1c680091d0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 1 -- 172.21.15.178:0/2915256343 shutdown_connections 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 1 --2- 172.21.15.178:0/2915256343 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1c780a08c0 0x7f1c780a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 1 --2- 172.21.15.178:0/2915256343 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c7809faa0 0x7f1c7809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 1 --2- 172.21.15.178:0/2915256343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c780a2930 0x7f1c78132de0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 1 -- 172.21.15.178:0/2915256343 >> 172.21.15.178:0/2915256343 conn(0x7f1c780a5ed0 msgr2=0x7f1c780a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 1 -- 172.21.15.178:0/2915256343 shutdown_connections 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 1 -- 172.21.15.178:0/2915256343 wait complete. 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 10 monclient: build_initial_monmap 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.549+0000 7f1c7e046640 10 monclient: monmap: 2024-09-15T01:48:17.566 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:17.567 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 1 Processor -- start 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 1 -- start start 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 10 monclient: init 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 10 monclient: _add_conns ranks=[7,6,0,2,1,8,5,3,4] 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c7809faa0 0x7f1c7809abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 10 monclient(hunting): picked mon.noname-i con 0x7f1c7809faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c780a08c0 0x7f1c78131750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 10 monclient(hunting): picked mon.noname-d con 0x7f1c780a08c0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c780a2930 0x7f1c7811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 10 monclient(hunting): picked mon.noname-a con 0x7f1c780a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7809d730 con 0x7f1c780a2930 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1c78142a60 con 0x7f1c780a08c0 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780a2390 con 0x7f1c7809faa0 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c7e046640 10 monclient(hunting): authenticate will time out at 1702.300884s 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c77fff640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c780a2930 msgr2=0x7f1c7811c530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c77fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c780a2930 0x7f1c7811c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c76ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c780a08c0 0x7f1c78131750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c76ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c780a08c0 0x7f1c78131750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:38094/0 (socket says 172.21.15.178:38094) 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.550+0000 7f1c76ffd640 1 -- 172.21.15.178:0/1669831910 learned_addr learned my addr 172.21.15.178:0/1669831910 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 10 monclient(hunting): get_auth_request con 0x7f1c780a08c0 auth_method 0 2024-09-15T01:48:17.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c777fe640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c7809faa0 0x7f1c7809abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c777fe640 10 monclient(hunting): get_auth_request con 0x7f1c7809faa0 auth_method 0 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 10 monclient(hunting): handle_auth_done global_id 25791 payload 824 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 1 -- 172.21.15.178:0/1669831910 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c7809faa0 msgr2=0x7f1c7809abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1c7809faa0 0x7f1c7809abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 1 -- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c780a2930 msgr2=0x7f1c7811c530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c780a2930 0x7f1c7811c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 1 monclient: found mon.noname-d 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 1 -- 172.21.15.178:0/1669831910 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c009a20 con 0x7f1c780a08c0 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c76ffd640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c780a08c0 0x7f1c78131750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00c0c0 tx=0x7f1c6c00b040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c6c00cea0 con 0x7f1c780a08c0 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c5f7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c5f7fe640 10 monclient: dump: 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:17.571 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:17.572 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c5f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c5f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c00f040 con 0x7f1c780a08c0 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c7e046640 5 monclient: authenticate success, global_id 25791 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c5f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c7e046640 10 monclient: _renew_subs 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c6c014d80 con 0x7f1c780a08c0 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c7e046640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c7e046640 1 -- 172.21.15.178:0/1669831910 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c68009450 con 0x7f1c780a08c0 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c5f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c5f7fe640 10 monclient: dump: 2024-09-15T01:48:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:17.574 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c75ffb640 4 set_mon_vals no callback set 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c7e046640 10 monclient: _renew_subs 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c7e046640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.551+0000 7f1c7e046640 1 -- 172.21.15.178:0/1669831910 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7809b720 con 0x7f1c780a08c0 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c7e046640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c7e046640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c78131c90 0x7f1c7811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c7e046640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c7e046640 1 -- 172.21.15.178:0/1669831910 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7811ecf0 con 0x7f1c78131c90 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1c6c014480 con 0x7f1c780a08c0 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c777fe640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c78131c90 0x7f1c7811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c777fe640 10 monclient: get_auth_request con 0x7f1c78131c90 auth_method 0 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:17.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c5f7fe640 1 --2- 172.21.15.178:0/1669831910 >> v2:172.21.15.204:6812/28210 conn(0x7f1c640396e0 0x7f1c6403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.552+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f1c6c04cb00 con 0x7f1c780a08c0 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c777fe640 10 monclient(hunting): handle_auth_done global_id 35345 payload 824 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c777fe640 1 -- 172.21.15.178:0/1669831910 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c7804c4a0 con 0x7f1c78131c90 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c77fff640 1 --2- 172.21.15.178:0/1669831910 >> v2:172.21.15.204:6812/28210 conn(0x7f1c640396e0 0x7f1c6403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c777fe640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c78131c90 0x7f1c7811ca70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009910 tx=0x7f1c68032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c77fff640 10 monclient: get_auth_request con 0x7f1c640396e0 auth_method 0 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c6802ac60 con 0x7f1c78131c90 2024-09-15T01:48:17.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c5f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c5f7fe640 10 monclient: dump: 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:17.577 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c77fff640 1 -- 172.21.15.178:0/1669831910 >> v2:172.21.15.204:6812/28210 conn(0x7f1c640396e0 msgr2=0x7f1c6403bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c77fff640 1 -- 172.21.15.178:0/1669831910 >> v2:172.21.15.204:6812/28210 conn(0x7f1c640396e0 msgr2=0x7f1c6403bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c77fff640 1 --2- 172.21.15.178:0/1669831910 >> v2:172.21.15.204:6812/28210 conn(0x7f1c640396e0 0x7f1c6403bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c77fff640 1 --2- 172.21.15.178:0/1669831910 >> v2:172.21.15.204:6812/28210 conn(0x7f1c640396e0 0x7f1c6403bb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f1c680313a0 con 0x7f1c78131c90 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c5f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c5f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c78131c90 msgr2=0x7f1c7811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c5f7fe640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c78131c90 0x7f1c7811ca70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009910 tx=0x7f1c68032000 comp rx=0 tx=0).stop 2024-09-15T01:48:17.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c78131c90 msgr2=0x7f1c7811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:17.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.553+0000 7f1c5f7fe640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c78131c90 0x7f1c7811ca70 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009910 tx=0x7f1c68032000 comp rx=0 tx=0).stop 2024-09-15T01:48:17.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c7e046640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:48:17.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c7e046640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7811cfb0 0x7f1c78096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c7e046640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c7e046640 1 -- 172.21.15.178:0/1669831910 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7804bc80 con 0x7f1c7811cfb0 2024-09-15T01:48:17.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c777fe640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7811cfb0 0x7f1c78096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:17.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c777fe640 10 monclient: get_auth_request con 0x7f1c7811cfb0 auth_method 0 2024-09-15T01:48:17.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:17.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c777fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:17.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:17.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:17.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:17.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:17.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c777fe640 10 monclient(hunting): handle_auth_done global_id 35354 payload 824 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c777fe640 1 -- 172.21.15.178:0/1669831910 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1c7805ed70 con 0x7f1c7811cfb0 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c777fe640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7811cfb0 0x7f1c78096a30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009210 tx=0x7f1c6803c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c68043810 con 0x7f1c7811cfb0 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c5f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.559+0000 7f1c5f7fe640 10 monclient: dump: 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:17.582 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9290 (secure 0 0 0) 0x7f1c68043390 con 0x7f1c7811cfb0 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c5f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c5f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7811cfb0 msgr2=0x7f1c78096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:17.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c5f7fe640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7811cfb0 0x7f1c78096a30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009210 tx=0x7f1c6803c000 comp rx=0 tx=0).stop 2024-09-15T01:48:17.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c5f7fe640 1 -- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7811cfb0 msgr2=0x7f1c78096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:17.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c5f7fe640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c7811cfb0 0x7f1c78096a30 secure :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009210 tx=0x7f1c6803c000 comp rx=0 tx=0).stop 2024-09-15T01:48:17.584 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:48:17.584 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:48:17.584 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:48:17.584 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:48:17.584 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 156, 2024-09-15T01:48:17.584 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:48:17.584 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:48:17.584 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:17.585 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:17.586 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:48:17.587 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:17.588 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:48:17.589 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.590 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:17.591 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:17.592 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.594 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.595 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:17.596 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.597 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:48:17.598 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:17.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:17.600 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:48:17.601 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:17.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:17.603 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:17.604 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:17.605 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:17.606 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:17.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:17.607 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:48:17.607 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:48:17.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c7e046640 1 -- 172.21.15.178:0/1669831910 >> v2:172.21.15.204:6812/28210 conn(0x7f1c640396e0 msgr2=0x7f1c6403bb90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:17.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c7e046640 1 --2- 172.21.15.178:0/1669831910 >> v2:172.21.15.204:6812/28210 conn(0x7f1c640396e0 0x7f1c6403bb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c7e046640 10 monclient: shutdown 2024-09-15T01:48:17.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c7e046640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:17.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c7e046640 1 -- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c780a08c0 msgr2=0x7f1c78131750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:17.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c77fff640 1 -- 172.21.15.178:0/1669831910 reap_dead start 2024-09-15T01:48:17.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c7e046640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c780a08c0 0x7f1c78131750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00c0c0 tx=0x7f1c6c00b040 comp rx=0 tx=0).stop 2024-09-15T01:48:17.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c7e046640 1 -- 172.21.15.178:0/1669831910 shutdown_connections 2024-09-15T01:48:17.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c7e046640 1 --2- 172.21.15.178:0/1669831910 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f1c780a08c0 0x7f1c78131750 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:17.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c7e046640 1 -- 172.21.15.178:0/1669831910 >> 172.21.15.178:0/1669831910 conn(0x7f1c780a5ed0 msgr2=0x7f1c78135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:17.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c7e046640 1 -- 172.21.15.178:0/1669831910 shutdown_connections 2024-09-15T01:48:17.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.560+0000 7f1c7e046640 1 -- 172.21.15.178:0/1669831910 wait complete. 2024-09-15T01:48:17.612 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:48:17.612 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-09-15T01:48:17.612 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:48:17.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.722+0000 7f669fa2b640 10 monclient: get_monmap_and_config 2024-09-15T01:48:17.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.722+0000 7f669fa2b640 10 monclient: build_initial_monmap 2024-09-15T01:48:17.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.722+0000 7f669fa2b640 10 monclient: monmap: 2024-09-15T01:48:17.724 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:17.724 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:48:17.723160+0000 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:48:17.723160+0000 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:17.725 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:17.726 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:17.726 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:17.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.722+0000 7f669fa2b640 1 Processor -- start 2024-09-15T01:48:17.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 1 -- start start 2024-09-15T01:48:17.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 10 monclient: init 2024-09-15T01:48:17.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:17.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 10 monclient: _add_conns ranks=[1,10,0,3,5,2,7,6,4,8,9] 2024-09-15T01:48:17.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f66980a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:48:17.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6698132de0 0x7f6698137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:17.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 10 monclient(hunting): picked mon.noname-f con 0x7f6698132de0 addr v2:172.21.15.204:3300/0 2024-09-15T01:48:17.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f66981380e0 addr v1:172.21.15.178:6789/0 2024-09-15T01:48:17.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669804bf60 con 0x7f66981380e0 2024-09-15T01:48:17.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669809dd80 con 0x7f66980a2930 2024-09-15T01:48:17.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:17.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66980a2d00 con 0x7f6698132de0 2024-09-15T01:48:17.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:17.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669dfc8640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f66981380e0 legacy=0x7f669813a4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:48:17.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669fa2b640 10 monclient(hunting): authenticate will time out at 1702.473630s 2024-09-15T01:48:17.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669d7c7640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7f66980a2930 legacy=0x7f669809faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:17.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669cfc6640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7f6698132de0 msgr2=0x7f6698137ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:17.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.723+0000 7f669cfc6640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6698132de0 0x7f6698137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:17.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.924+0000 7f669dfc8640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f66981380e0 legacy=0x7f669813a4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:48:17.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.924+0000 7f669d7c7640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7f66980a2930 legacy=0x7f669809faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:17.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.924+0000 7f669cfc6640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7f6698132de0 msgr2=0x7f6698137ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:17.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:17.924+0000 7f669cfc6640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6698132de0 0x7f6698137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T01:48:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:18.325+0000 7f669dfc8640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f66981380e0 legacy=0x7f669813a4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:48:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:18.325+0000 7f669cfc6640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7f6698132de0 msgr2=0x7f6698137ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:18.325+0000 7f669d7c7640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7f66980a2930 legacy=0x7f669809faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:18.325+0000 7f669cfc6640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6698132de0 0x7f6698137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T01:48:19.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:19.126+0000 7f669dfc8640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f66981380e0 legacy=0x7f669813a4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:48:19.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:19.126+0000 7f669d7c7640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7f66980a2930 legacy=0x7f669809faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:19.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:19.126+0000 7f669cfc6640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7f6698132de0 msgr2=0x7f6698137ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:19.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:19.126+0000 7f669cfc6640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6698132de0 0x7f6698137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T01:48:20.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 10 monclient(hunting): tick 2024-09-15T01:48:20.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 1 monclient(hunting): continuing hunt 2024-09-15T01:48:20.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 10 monclient(hunting): _reopen_session rank -1 2024-09-15T01:48:20.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7f6698132de0 msgr2=0x7f6698137ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:20.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6698132de0 0x7f6698137ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7f66980a2930 legacy=0x7f669809faa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:20.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f66981380e0 legacy=0x7f669813a4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:20.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 10 monclient: _add_conns ranks=[3,4,7,9,2,5,6,8] 2024-09-15T01:48:20.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c003790 0x7f667c005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 10 monclient(hunting): picked mon.noname-g con 0x7f667c003790 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f667c006270 0x7f667c0086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 10 monclient(hunting): picked mon.noname-c con 0x7f667c006270 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f667c008de0 0x7f667c00b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 10 monclient(hunting): picked mon.noname-i con 0x7f667c008de0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66980a2d00 con 0x7f667c003790 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f667c00b960 con 0x7f667c006270 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f667c00bb20 con 0x7f667c008de0 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f668effd640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:20.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f667c006270 0x7f667c0086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f667c006270 0x7f667c0086f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:34174/0 (socket says 172.21.15.178:34174) 2024-09-15T01:48:20.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 1 -- 172.21.15.178:0/4179752191 learned_addr learned my addr 172.21.15.178:0/4179752191 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:20.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 10 monclient(hunting): get_auth_request con 0x7f667c006270 auth_method 0 2024-09-15T01:48:20.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:20.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:20.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:20.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669d7c7640 1 --2- 172.21.15.178:0/4179752191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c003790 0x7f667c005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669dfc8640 1 --2- 172.21.15.178:0/4179752191 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f667c008de0 0x7f667c00b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669d7c7640 10 monclient(hunting): get_auth_request con 0x7f667c003790 auth_method 0 2024-09-15T01:48:20.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669d7c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:20.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669d7c7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:20.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669d7c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:20.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669dfc8640 10 monclient(hunting): get_auth_request con 0x7f667c008de0 auth_method 0 2024-09-15T01:48:20.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669dfc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:20.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669dfc8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:20.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669dfc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:20.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:20.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:20.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:20.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669d7c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:20.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669d7c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:20.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669d7c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:20.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669dfc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:20.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669dfc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:20.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669dfc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:20.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 10 monclient(hunting): handle_auth_done global_id 44896 payload 274 2024-09-15T01:48:20.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 1 -- 172.21.15.178:0/4179752191 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f667c008de0 msgr2=0x7f667c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:20.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 1 --2- 172.21.15.178:0/4179752191 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f667c008de0 0x7f667c00b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 1 -- 172.21.15.178:0/4179752191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c003790 msgr2=0x7f667c005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:20.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 1 --2- 172.21.15.178:0/4179752191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c003790 0x7f667c005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 10 monclient: _finish_hunting 0 2024-09-15T01:48:20.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 1 monclient: found mon.noname-c 2024-09-15T01:48:20.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:20.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669dfc8640 1 -- 172.21.15.178:0/4179752191 reap_dead start 2024-09-15T01:48:20.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:48:20.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.724+0000 7f669cfc6640 1 -- 172.21.15.178:0/4179752191 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66981331b0 con 0x7f667c006270 2024-09-15T01:48:20.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f669cfc6640 1 --2- 172.21.15.178:0/4179752191 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f667c006270 0x7f667c0086f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f66880099a0 tx=0x7f6688009270 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f5a2850e2b14afca server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:20.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f668ffff640 1 -- 172.21.15.178:0/4179752191 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f668802acb0 con 0x7f667c006270 2024-09-15T01:48:20.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f668ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:20.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f668ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:48:20.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f668ffff640 10 monclient: dump: 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:20.736 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f668ffff640 10 monclient: _finish_auth 0 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f668ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f668ffff640 1 -- 172.21.15.178:0/4179752191 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668802ae50 con 0x7f667c006270 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f669fa2b640 5 monclient: authenticate success, global_id 44896 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f669fa2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f668ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f668ffff640 1 -- 172.21.15.178:0/4179752191 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f66880065d0 con 0x7f667c006270 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f668ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f668ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f668ffff640 10 monclient: dump: 2024-09-15T01:48:20.737 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:20.738 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.738 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.738 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.738 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:20.739 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.739 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:20.739 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f669fa2b640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.725+0000 7f669fa2b640 4 set_mon_vals no callback set 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.726+0000 7f669fa2b640 10 monclient: shutdown 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.726+0000 7f669fa2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:20.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.726+0000 7f669fa2b640 1 -- 172.21.15.178:0/4179752191 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f667c006270 msgr2=0x7f667c0086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.726+0000 7f669fa2b640 1 --2- 172.21.15.178:0/4179752191 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f667c006270 0x7f667c0086f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f66880099a0 tx=0x7f6688009270 comp rx=0 tx=0).stop 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.726+0000 7f669fa2b640 1 -- 172.21.15.178:0/4179752191 shutdown_connections 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.726+0000 7f669fa2b640 1 --2- 172.21.15.178:0/4179752191 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f667c006270 0x7f667c0086f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.726+0000 7f669fa2b640 1 -- 172.21.15.178:0/4179752191 >> 172.21.15.178:0/4179752191 conn(0x7f66980a5ed0 msgr2=0x7f66980a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.726+0000 7f669fa2b640 1 -- 172.21.15.178:0/4179752191 shutdown_connections 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.726+0000 7f669fa2b640 1 -- 172.21.15.178:0/4179752191 wait complete. 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.727+0000 7f669fa2b640 10 monclient: build_initial_monmap 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.727+0000 7f669fa2b640 10 monclient: monmap: 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:48:20.741 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:20.742 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:48:20.742 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:20.742 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:20.742 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:20.742 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:20.742 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:20.742 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:20.742 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:20.742 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:20.742 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:20.742 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.727+0000 7f669fa2b640 1 Processor -- start 2024-09-15T01:48:20.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.727+0000 7f669fa2b640 1 -- start start 2024-09-15T01:48:20.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.727+0000 7f669fa2b640 10 monclient: init 2024-09-15T01:48:20.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:20.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 10 monclient: _add_conns ranks=[0,1,6,7,4,5,8,3,2] 2024-09-15T01:48:20.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f667c006270 0x7f669809b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 10 monclient(hunting): picked mon.noname-a con 0x7f667c006270 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:48:20.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66981374c0 0x7f6698134f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 10 monclient(hunting): picked mon.noname-f con 0x7f66981374c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:48:20.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66981354b0 0x7f66981358c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 10 monclient(hunting): picked mon.noname-d con 0x7f66981354b0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:48:20.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6698097950 con 0x7f667c006270 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6698135e00 con 0x7f66981374c0 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f669809f870 con 0x7f66981354b0 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669fa2b640 10 monclient(hunting): authenticate will time out at 1705.478772s 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669d7c7640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f667c006270 msgr2=0x7f669809b770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669cfc6640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66981374c0 msgr2=0x7f6698134f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669d7c7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f667c006270 0x7f669809b770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669cfc6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66981374c0 0x7f6698134f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669dfc8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66981354b0 0x7f66981358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669dfc8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66981354b0 0x7f66981358c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:39384/0 (socket says 172.21.15.178:39384) 2024-09-15T01:48:20.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669dfc8640 1 -- 172.21.15.178:0/294550607 learned_addr learned my addr 172.21.15.178:0/294550607 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669dfc8640 10 monclient(hunting): get_auth_request con 0x7f66981354b0 auth_method 0 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669dfc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669dfc8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.728+0000 7f669dfc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 10 monclient(hunting): handle_auth_done global_id 25800 payload 824 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 1 -- 172.21.15.178:0/294550607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66981374c0 msgr2=0x7f6698134f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66981374c0 0x7f6698134f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 1 -- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f667c006270 msgr2=0x7f669809b770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f667c006270 0x7f669809b770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 10 monclient: _finish_hunting 0 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 1 monclient: found mon.noname-d 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 1 -- 172.21.15.178:0/294550607 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6688009550 con 0x7f66981354b0 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669dfc8640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66981354b0 0x7f66981358c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f66980644e0 tx=0x7f6690007760 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f669000b040 con 0x7f66981354b0 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668d7fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668d7fa640 10 monclient: dump: 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.753 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668d7fa640 10 monclient: _finish_auth 0 2024-09-15T01:48:20.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6690007e90 con 0x7f66981354b0 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669fa2b640 5 monclient: authenticate success, global_id 25800 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669fa2b640 10 monclient: _renew_subs 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669fa2b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669fa2b640 1 -- 172.21.15.178:0/294550607 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6698098330 con 0x7f66981354b0 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6690011380 con 0x7f66981354b0 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668d7fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668d7fa640 10 monclient: dump: 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.755 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f668effd640 4 set_mon_vals no callback set 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669fa2b640 10 monclient: _renew_subs 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669fa2b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.729+0000 7f669fa2b640 1 -- 172.21.15.178:0/294550607 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66981368f0 con 0x7f66981354b0 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.730+0000 7f669fa2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.730+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6690012630 con 0x7f66981354b0 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.730+0000 7f669fa2b640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131010 0x7f66981313e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.730+0000 7f669fa2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.730+0000 7f669fa2b640 1 -- 172.21.15.178:0/294550607 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6698097810 con 0x7f6698131010 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669d7c7640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131010 0x7f66981313e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669d7c7640 10 monclient: get_auth_request con 0x7f6698131010 auth_method 0 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669d7c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669d7c7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:20.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669d7c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f668d7fa640 1 --2- 172.21.15.178:0/294550607 >> v2:172.21.15.204:6812/28210 conn(0x7f667c03ace0 0x7f667c03d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669d7c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669d7c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669d7c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f669004b5e0 con 0x7f66981354b0 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669d7c7640 10 monclient(hunting): handle_auth_done global_id 44905 payload 824 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669d7c7640 1 -- 172.21.15.178:0/294550607 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f669804c4a0 con 0x7f6698131010 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669cfc6640 1 --2- 172.21.15.178:0/294550607 >> v2:172.21.15.204:6812/28210 conn(0x7f667c03ace0 0x7f667c03d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669cfc6640 10 monclient: get_auth_request con 0x7f667c03ace0 auth_method 0 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f669d7c7640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131010 0x7f66981313e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f66940038f0 tx=0x7f6694009760 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6694006ac0 con 0x7f6698131010 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f668d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:20.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f668d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.731+0000 7f668d7fa640 10 monclient: dump: 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:20.759 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:20.760 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:20.760 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:20.760 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.732+0000 7f669cfc6640 1 --2- 172.21.15.178:0/294550607 >> v2:172.21.15.204:6812/28210 conn(0x7f667c03ace0 0x7f667c03d190 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6688009970 tx=0x7f668802f000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:20.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.732+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f669400f040 con 0x7f6698131010 2024-09-15T01:48:20.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.732+0000 7f668d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:20.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.732+0000 7f668d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:20.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.732+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131010 msgr2=0x7f66981313e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:20.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.732+0000 7f668d7fa640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131010 0x7f66981313e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f66940038f0 tx=0x7f6694009760 comp rx=0 tx=0).stop 2024-09-15T01:48:20.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.732+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131010 msgr2=0x7f66981313e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:20.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.732+0000 7f668d7fa640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131010 0x7f66981313e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669fa2b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669fa2b640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131920 0x7f669814c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669fa2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669fa2b640 1 -- 172.21.15.178:0/294550607 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66980a0220 con 0x7f6698131920 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669d7c7640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131920 0x7f669814c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669d7c7640 10 monclient: get_auth_request con 0x7f6698131920 auth_method 0 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669d7c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669d7c7640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669d7c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669d7c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669d7c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669d7c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669d7c7640 10 monclient(hunting): handle_auth_done global_id 44914 payload 824 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669d7c7640 1 -- 172.21.15.178:0/294550607 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f669804bc80 con 0x7f6698131920 2024-09-15T01:48:20.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f669d7c7640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131920 0x7f669814c5a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6694009a70 tx=0x7f6694015000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6694013ce0 con 0x7f6698131920 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f668d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f668d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.738+0000 7f668d7fa640 10 monclient: dump: 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:20.762 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9460 (secure 0 0 0) 0x7f6694013e40 con 0x7f6698131920 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f668d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f668d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131920 msgr2=0x7f669814c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f668d7fa640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131920 0x7f669814c5a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6694009a70 tx=0x7f6694015000 comp rx=0 tx=0).stop 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f668d7fa640 1 -- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131920 msgr2=0x7f669814c5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:20.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f668d7fa640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f6698131920 0x7f669814c5a0 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6694009a70 tx=0x7f6694015000 comp rx=0 tx=0).stop 2024-09-15T01:48:20.764 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:48:20.764 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:48:20.764 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:48:20.764 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:48:20.764 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 156, 2024-09-15T01:48:20.764 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:48:20.764 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:48:20.764 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:48:20.764 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:20.765 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:20.766 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:20.767 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:20.768 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:48:20.769 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.770 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:48:20.770 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.770 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.770 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:48:20.771 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.772 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.773 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:48:20.774 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.777 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:48:20.778 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.780 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:48:20.781 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.783 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:48:20.784 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.785 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.785 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.785 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.785 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:48:20.785 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:48:20.785 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.785 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.785 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.785 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.785 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.785 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.785 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.786 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:20.787 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.788 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.788 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.788 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:48:20.788 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:48:20.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f669fa2b640 1 -- 172.21.15.178:0/294550607 >> v2:172.21.15.204:6812/28210 conn(0x7f667c03ace0 msgr2=0x7f667c03d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:20.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f669fa2b640 1 --2- 172.21.15.178:0/294550607 >> v2:172.21.15.204:6812/28210 conn(0x7f667c03ace0 0x7f667c03d190 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6688009970 tx=0x7f668802f000 comp rx=0 tx=0).stop 2024-09-15T01:48:20.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f669fa2b640 10 monclient: shutdown 2024-09-15T01:48:20.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f669fa2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:20.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f669fa2b640 1 -- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66981354b0 msgr2=0x7f66981358c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:20.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f669fa2b640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66981354b0 0x7f66981358c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f66980644e0 tx=0x7f6690007760 comp rx=0 tx=0).stop 2024-09-15T01:48:20.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f669dfc8640 1 -- 172.21.15.178:0/294550607 reap_dead start 2024-09-15T01:48:20.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f669fa2b640 1 -- 172.21.15.178:0/294550607 shutdown_connections 2024-09-15T01:48:20.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f669fa2b640 1 --2- 172.21.15.178:0/294550607 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f66981354b0 0x7f66981358c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f669fa2b640 1 -- 172.21.15.178:0/294550607 >> 172.21.15.178:0/294550607 conn(0x7f66980a5ed0 msgr2=0x7f669806e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:20.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f669fa2b640 1 -- 172.21.15.178:0/294550607 shutdown_connections 2024-09-15T01:48:20.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.739+0000 7f669fa2b640 1 -- 172.21.15.178:0/294550607 wait complete. 2024-09-15T01:48:20.791 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:48:20.791 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-15T01:48:20.791 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:48:20.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.887+0000 7fc2e5ed4640 10 monclient: get_monmap_and_config 2024-09-15T01:48:20.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.887+0000 7fc2e5ed4640 10 monclient: build_initial_monmap 2024-09-15T01:48:20.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.887+0000 7fc2e5ed4640 10 monclient: monmap: 2024-09-15T01:48:20.890 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:20.890 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.890 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:48:20.888689+0000 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:48:20.888689+0000 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:20.891 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:20.892 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:20.892 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.888+0000 7fc2e5ed4640 1 Processor -- start 2024-09-15T01:48:20.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.888+0000 7fc2e5ed4640 1 -- start start 2024-09-15T01:48:20.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.888+0000 7fc2e5ed4640 10 monclient: init 2024-09-15T01:48:20.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:20.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 10 monclient: _add_conns ranks=[7,3,0,4,10,6,2,8,5,1,9] 2024-09-15T01:48:20.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc2e00a2930 0x7fc2e0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 10 monclient(hunting): picked mon.noname-i con 0x7fc2e00a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:48:20.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e009faa0 0x7fc2e009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 10 monclient(hunting): picked mon.noname-g con 0x7fc2e009faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:20.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc2e00a08c0 addr v1:172.21.15.178:6789/0 2024-09-15T01:48:20.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e004bf60 con 0x7fc2e00a08c0 2024-09-15T01:48:20.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e009e0f0 con 0x7fc2e009faa0 2024-09-15T01:48:20.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e009dd80 con 0x7fc2e00a2930 2024-09-15T01:48:20.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:20.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2e5ed4640 10 monclient(hunting): authenticate will time out at 1705.639113s 2024-09-15T01:48:20.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2dffff640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7fc2e00a08c0 legacy=0x7fc2e00a0d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:48:20.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2df7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc2e00a2930 0x7fc2e0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e009faa0 0x7fc2e009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2df7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc2e00a2930 0x7fc2e0132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:56744/0 (socket says 172.21.15.178:56744) 2024-09-15T01:48:20.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e009faa0 0x7fc2e009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:49568/0 (socket says 172.21.15.178:49568) 2024-09-15T01:48:20.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2df7fe640 1 -- 172.21.15.178:0/2324268371 learned_addr learned my addr 172.21.15.178:0/2324268371 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:20.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2df7fe640 10 monclient(hunting): get_auth_request con 0x7fc2e00a2930 auth_method 0 2024-09-15T01:48:20.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:20.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:20.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:20.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 10 monclient(hunting): get_auth_request con 0x7fc2e009faa0 auth_method 0 2024-09-15T01:48:20.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:20.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:20.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:20.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:20.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:20.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:20.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:20.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:20.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:20.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 10 monclient(hunting): handle_auth_done global_id 64506 payload 274 2024-09-15T01:48:20.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 1 -- 172.21.15.178:0/2324268371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc2e00a2930 msgr2=0x7fc2e0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:20.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 1 --2- 172.21.15.178:0/2324268371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc2e00a2930 0x7fc2e0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 1 -- 172.21.15.178:0/2324268371 >> v1:172.21.15.178:6789/0 conn(0x7fc2e00a08c0 legacy=0x7fc2e00a0d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:20.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 10 monclient: _finish_hunting 0 2024-09-15T01:48:20.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 1 monclient: found mon.noname-g 2024-09-15T01:48:20.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:20.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:48:20.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2deffd640 1 -- 172.21.15.178:0/2324268371 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e00a2d00 con 0x7fc2e009faa0 2024-09-15T01:48:20.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.889+0000 7fc2df7fe640 1 --2- 172.21.15.178:0/2324268371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc2e00a2930 0x7fc2e0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:48:20.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2deffd640 1 --2- 172.21.15.178:0/2324268371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e009faa0 0x7fc2e009fe70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009880 tx=0x7fc2d40090f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=5861857b738171c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:20.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2de7fc640 1 -- 172.21.15.178:0/2324268371 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2d402abc0 con 0x7fc2e009faa0 2024-09-15T01:48:20.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:20.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2de7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:48:20.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2de7fc640 10 monclient: dump: 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:20.906 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2de7fc640 10 monclient: _finish_auth 0 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2de7fc640 1 -- 172.21.15.178:0/2324268371 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d402ad40 con 0x7fc2e009faa0 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2de7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2de7fc640 1 -- 172.21.15.178:0/2324268371 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2d4031ca0 con 0x7fc2e009faa0 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2e5ed4640 5 monclient: authenticate success, global_id 64506 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2e5ed4640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2e5ed4640 4 set_mon_vals no callback set 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2de7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:48:20.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.890+0000 7fc2de7fc640 10 monclient: dump: 2024-09-15T01:48:20.908 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:20.908 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.908 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.908 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.908 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:20.908 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.908 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:20.908 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:20.908 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:20.908 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.891+0000 7fc2e5ed4640 10 monclient: shutdown 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.891+0000 7fc2e5ed4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.891+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2324268371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e009faa0 msgr2=0x7fc2e009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.891+0000 7fc2e5ed4640 1 --2- 172.21.15.178:0/2324268371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e009faa0 0x7fc2e009fe70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009880 tx=0x7fc2d40090f0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.891+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2324268371 shutdown_connections 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.891+0000 7fc2e5ed4640 1 --2- 172.21.15.178:0/2324268371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e009faa0 0x7fc2e009fe70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.891+0000 7fc2e5ed4640 1 --2- 172.21.15.178:0/2324268371 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc2e00a2930 0x7fc2e0132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.891+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2324268371 >> 172.21.15.178:0/2324268371 conn(0x7fc2e00a5ed0 msgr2=0x7fc2e00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:20.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.891+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2324268371 shutdown_connections 2024-09-15T01:48:20.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.891+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2324268371 wait complete. 2024-09-15T01:48:20.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.891+0000 7fc2e5ed4640 10 monclient: build_initial_monmap 2024-09-15T01:48:20.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.891+0000 7fc2e5ed4640 10 monclient: monmap: 2024-09-15T01:48:20.910 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:20.910 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:20.910 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:20.911 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 1 Processor -- start 2024-09-15T01:48:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 1 -- start start 2024-09-15T01:48:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 10 monclient: init 2024-09-15T01:48:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 10 monclient: _add_conns ranks=[1,3,5,4,7,8,6,0,2] 2024-09-15T01:48:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2e009faa0 0x7fc2e009b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 10 monclient(hunting): picked mon.noname-f con 0x7fc2e009faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:48:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e00a08c0 0x7fc2e01318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 10 monclient(hunting): picked mon.noname-g con 0x7fc2e00a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc2e00a2930 0x7fc2e011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 10 monclient(hunting): picked mon.noname-h con 0x7fc2e00a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:48:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e011ef90 con 0x7fc2e009faa0 2024-09-15T01:48:20.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e009d630 con 0x7fc2e00a08c0 2024-09-15T01:48:20.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e011ecb0 con 0x7fc2e00a2930 2024-09-15T01:48:20.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:20.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2e5ed4640 10 monclient(hunting): authenticate will time out at 1705.642695s 2024-09-15T01:48:20.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2df7fe640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2e009faa0 msgr2=0x7fc2e009b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:20.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2df7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2e009faa0 0x7fc2e009b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:20.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2dffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc2e00a2930 0x7fc2e011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2deffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e00a08c0 0x7fc2e01318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2dffff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc2e00a2930 0x7fc2e011c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:41636/0 (socket says 172.21.15.178:41636) 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2deffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e00a08c0 0x7fc2e01318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:49570/0 (socket says 172.21.15.178:49570) 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2dffff640 1 -- 172.21.15.178:0/2494760111 learned_addr learned my addr 172.21.15.178:0/2494760111 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2dffff640 10 monclient(hunting): get_auth_request con 0x7fc2e00a2930 auth_method 0 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2dffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2deffd640 10 monclient(hunting): get_auth_request con 0x7fc2e00a08c0 auth_method 0 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2deffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:20.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.892+0000 7fc2deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2deffd640 10 monclient(hunting): handle_auth_done global_id 64515 payload 824 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2deffd640 1 -- 172.21.15.178:0/2494760111 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc2e00a2930 msgr2=0x7fc2e011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2deffd640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc2e00a2930 0x7fc2e011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2deffd640 1 -- 172.21.15.178:0/2494760111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2e009faa0 msgr2=0x7fc2e009b120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2deffd640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2e009faa0 0x7fc2e009b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2deffd640 10 monclient: _finish_hunting 0 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2deffd640 1 monclient: found mon.noname-g 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2deffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:48:20.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2deffd640 1 -- 172.21.15.178:0/2494760111 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d0009450 con 0x7fc2e00a08c0 2024-09-15T01:48:20.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2dffff640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fc2e00a2930 0x7fc2e011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:48:20.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2deffd640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e00a08c0 0x7fc2e01318b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4031000 tx=0x7fc2d4032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:20.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2d402abc0 con 0x7fc2e00a08c0 2024-09-15T01:48:20.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:20.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2c77fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:48:20.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2c77fe640 10 monclient: dump: 2024-09-15T01:48:20.916 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:20.916 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.916 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.916 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.916 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:20.916 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2c77fe640 10 monclient: _finish_auth 0 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:20.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d4033e60 con 0x7fc2e00a08c0 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2e5ed4640 5 monclient: authenticate success, global_id 64515 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2c77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2e5ed4640 10 monclient: _renew_subs 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2d403b870 con 0x7fc2e00a08c0 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2e5ed4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2494760111 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d4009390 con 0x7fc2e00a08c0 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2c77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2c77fe640 10 monclient: dump: 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.918 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2ddffb640 4 set_mon_vals no callback set 2024-09-15T01:48:20.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2e5ed4640 10 monclient: _renew_subs 2024-09-15T01:48:20.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2e5ed4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:20.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.893+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2494760111 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e0142360 con 0x7fc2e00a08c0 2024-09-15T01:48:20.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2e5ed4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:20.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2e5ed4640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e011da70 0x7fc2e011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2e5ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2494760111 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e011eb70 con 0x7fc2e011da70 2024-09-15T01:48:20.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc2d4039040 con 0x7fc2e00a08c0 2024-09-15T01:48:20.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2df7fe640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e011da70 0x7fc2e011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2df7fe640 10 monclient: get_auth_request con 0x7fc2e011da70 auth_method 0 2024-09-15T01:48:20.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:20.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:20.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2c77fe640 1 --2- 172.21.15.178:0/2494760111 >> v2:172.21.15.204:6812/28210 conn(0x7fc2cc039700 0x7fc2cc03bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.894+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fc2d406b9f0 con 0x7fc2e00a08c0 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2df7fe640 10 monclient(hunting): handle_auth_done global_id 25809 payload 824 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2dffff640 1 --2- 172.21.15.178:0/2494760111 >> v2:172.21.15.204:6812/28210 conn(0x7fc2cc039700 0x7fc2cc03bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2df7fe640 1 -- 172.21.15.178:0/2494760111 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc2e004c4a0 con 0x7fc2e011da70 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2dffff640 10 monclient: get_auth_request con 0x7fc2cc039700 auth_method 0 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2df7fe640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e011da70 0x7fc2e011ca70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0006c70 tx=0x7fc2d00091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2d0013470 con 0x7fc2e011da70 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:20.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2c77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2c77fe640 10 monclient: dump: 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:20.922 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:20.923 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:20.923 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:20.923 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2dffff640 1 --2- 172.21.15.178:0/2494760111 >> v2:172.21.15.204:6812/28210 conn(0x7fc2cc039700 0x7fc2cc03bbb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc2e009b7d0 tx=0x7fc2d800d040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:20.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fc2d000ca90 con 0x7fc2e011da70 2024-09-15T01:48:20.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2c77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:20.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2c77fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:20.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e011da70 msgr2=0x7fc2e011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:20.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2c77fe640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e011da70 0x7fc2e011ca70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0006c70 tx=0x7fc2d00091d0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e011da70 msgr2=0x7fc2e011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:20.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.895+0000 7fc2c77fe640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e011da70 0x7fc2e011ca70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:20.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2e5ed4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:48:20.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2e5ed4640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e0131df0 0x7fc2e011cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:20.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2e5ed4640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:20.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2494760111 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e009f850 con 0x7fc2e0131df0 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2df7fe640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e0131df0 0x7fc2e011cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2df7fe640 10 monclient: get_auth_request con 0x7fc2e0131df0 auth_method 0 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2df7fe640 10 monclient(hunting): handle_auth_done global_id 25818 payload 824 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2df7fe640 1 -- 172.21.15.178:0/2494760111 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc2e004bc80 con 0x7fc2e0131df0 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2df7fe640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e0131df0 0x7fc2e011cfb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0019000 tx=0x7fc2d0017000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2d0023070 con 0x7fc2e0131df0 2024-09-15T01:48:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2c77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2c77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.901+0000 7fc2c77fe640 10 monclient: dump: 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:20.927 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:20.928 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:20.928 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:20.928 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:20.928 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:20.928 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:20.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9120 (secure 0 0 0) 0x7fc2d00135d0 con 0x7fc2e0131df0 2024-09-15T01:48:20.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2c77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:48:20.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2c77fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:20.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e0131df0 msgr2=0x7fc2e011cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:20.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2c77fe640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e0131df0 0x7fc2e011cfb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0019000 tx=0x7fc2d0017000 comp rx=0 tx=0).stop 2024-09-15T01:48:20.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2c77fe640 1 -- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e0131df0 msgr2=0x7fc2e011cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:20.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2c77fe640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc2e0131df0 0x7fc2e011cfb0 secure :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0019000 tx=0x7fc2d0017000 comp rx=0 tx=0).stop 2024-09-15T01:48:20.929 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:48:20.929 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:48:20.929 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:48:20.929 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:48:20.929 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 156, 2024-09-15T01:48:20.929 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:48:20.929 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:48:20.929 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:48:20.929 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:48:20.929 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:48:20.929 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:20.930 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:20.931 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:20.932 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:20.932 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.932 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:48:20.932 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:48:20.932 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:48:20.932 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:48:20.932 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:48:20.932 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:48:20.932 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:20.932 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:20.932 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:20.933 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:48:20.935 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:48:20.938 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:48:20.939 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.941 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:48:20.942 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.943 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.944 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:48:20.945 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:48:20.946 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.948 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:20.949 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:48:20.949 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.949 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.949 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:20.949 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:48:20.949 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:20.949 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.949 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.949 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.949 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:48:20.949 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:20.950 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.951 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:20.951 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:48:20.951 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:20.951 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:20.951 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:20.951 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:20.951 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:20.951 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:20.951 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:20.951 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:48:20.951 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:48:20.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2494760111 >> v2:172.21.15.204:6812/28210 conn(0x7fc2cc039700 msgr2=0x7fc2cc03bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:20.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2e5ed4640 1 --2- 172.21.15.178:0/2494760111 >> v2:172.21.15.204:6812/28210 conn(0x7fc2cc039700 0x7fc2cc03bbb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc2e009b7d0 tx=0x7fc2d800d040 comp rx=0 tx=0).stop 2024-09-15T01:48:20.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2e5ed4640 10 monclient: shutdown 2024-09-15T01:48:20.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2e5ed4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:20.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2494760111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e00a08c0 msgr2=0x7fc2e01318b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:20.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2e5ed4640 1 --2- 172.21.15.178:0/2494760111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2e00a08c0 0x7fc2e01318b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4031000 tx=0x7fc2d4032000 comp rx=0 tx=0).stop 2024-09-15T01:48:20.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2dffff640 1 -- 172.21.15.178:0/2494760111 reap_dead start 2024-09-15T01:48:20.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2dffff640 1 -- 172.21.15.178:0/2494760111 reap_dead start 2024-09-15T01:48:20.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2494760111 shutdown_connections 2024-09-15T01:48:20.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2494760111 >> 172.21.15.178:0/2494760111 conn(0x7fc2e00a5ed0 msgr2=0x7fc2e009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:20.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2494760111 shutdown_connections 2024-09-15T01:48:20.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:20.902+0000 7fc2e5ed4640 1 -- 172.21.15.178:0/2494760111 wait complete. 2024-09-15T01:48:20.954 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:48:20.954 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-09-15T01:48:20.954 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:48:21.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.046+0000 7fbbed75d640 10 monclient: get_monmap_and_config 2024-09-15T01:48:21.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.046+0000 7fbbed75d640 10 monclient: build_initial_monmap 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.046+0000 7fbbed75d640 10 monclient: monmap: 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:48:21.047270+0000 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:48:21.047270+0000 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:21.049 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:21.050 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:21.050 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:21.050 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:21.050 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:21.050 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:21.050 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.046+0000 7fbbed75d640 1 Processor -- start 2024-09-15T01:48:21.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 1 -- start start 2024-09-15T01:48:21.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 10 monclient: init 2024-09-15T01:48:21.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:21.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 10 monclient: _add_conns ranks=[2,4,8,1,7,0,6,3,9,10,5] 2024-09-15T01:48:21.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbe80a2930 0x7fbbe8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 10 monclient(hunting): picked mon.noname-b con 0x7fbbe80a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:48:21.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe809faa0 0x7fbbe809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 10 monclient(hunting): picked mon.noname-c con 0x7fbbe809faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:48:21.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe80a08c0 0x7fbbe80a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 10 monclient(hunting): picked mon.noname-e con 0x7fbbe80a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:48:21.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe809e0f0 con 0x7fbbe80a2930 2024-09-15T01:48:21.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe809dd80 con 0x7fbbe809faa0 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80a2d00 con 0x7fbbe80a08c0 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbed75d640 10 monclient(hunting): authenticate will time out at 1705.797508s 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe67fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe809faa0 0x7fbbe809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe77fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe80a08c0 0x7fbbe80a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbe80a2930 0x7fbbe8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe67fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe809faa0 0x7fbbe809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:34202/0 (socket says 172.21.15.178:34202) 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 10 monclient(hunting): get_auth_request con 0x7fbbe80a2930 auth_method 0 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe67fc640 1 -- 172.21.15.178:0/364207119 learned_addr learned my addr 172.21.15.178:0/364207119 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe77fe640 10 monclient(hunting): get_auth_request con 0x7fbbe80a08c0 auth_method 0 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe67fc640 10 monclient(hunting): get_auth_request con 0x7fbbe809faa0 auth_method 0 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 10 monclient(hunting): handle_auth_done global_id 35363 payload 274 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 1 -- 172.21.15.178:0/364207119 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe80a08c0 msgr2=0x7fbbe80a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 1 --2- 172.21.15.178:0/364207119 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe80a08c0 0x7fbbe80a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 1 -- 172.21.15.178:0/364207119 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe809faa0 msgr2=0x7fbbe809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 1 --2- 172.21.15.178:0/364207119 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe809faa0 0x7fbbe809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 1 monclient: found mon.noname-b 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe6ffd640 1 -- 172.21.15.178:0/364207119 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe804bf60 con 0x7fbbe80a2930 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe77fe640 1 --2- 172.21.15.178:0/364207119 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe80a08c0 0x7fbbe80a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.047+0000 7fbbe67fc640 1 --2- 172.21.15.178:0/364207119 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe809faa0 0x7fbbe809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe6ffd640 1 --2- 172.21.15.178:0/364207119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbe80a2930 0x7fbbe8132de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009e90 tx=0x7fbbe0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ea0e9fed241d3d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe5ffb640 1 -- 172.21.15.178:0/364207119 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbe00079d0 con 0x7fbbe80a2930 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe5ffb640 10 monclient: dump: 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.064 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:48:21.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe5ffb640 1 -- 172.21.15.178:0/364207119 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe002d050 con 0x7fbbe80a2930 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbed75d640 5 monclient: authenticate success, global_id 35363 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbed75d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbed75d640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe5ffb640 1 -- 172.21.15.178:0/364207119 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbe0007bd0 con 0x7fbbe80a2930 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbed75d640 4 set_mon_vals no callback set 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.048+0000 7fbbe5ffb640 10 monclient: dump: 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.066 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.067 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.049+0000 7fbbed75d640 10 monclient: shutdown 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.049+0000 7fbbed75d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.049+0000 7fbbed75d640 1 -- 172.21.15.178:0/364207119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbe80a2930 msgr2=0x7fbbe8132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.049+0000 7fbbed75d640 1 --2- 172.21.15.178:0/364207119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbe80a2930 0x7fbbe8132de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009e90 tx=0x7fbbe0009780 comp rx=0 tx=0).stop 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.049+0000 7fbbed75d640 1 -- 172.21.15.178:0/364207119 shutdown_connections 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.049+0000 7fbbed75d640 1 --2- 172.21.15.178:0/364207119 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe809faa0 0x7fbbe809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.049+0000 7fbbed75d640 1 --2- 172.21.15.178:0/364207119 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe80a08c0 0x7fbbe80a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.049+0000 7fbbed75d640 1 --2- 172.21.15.178:0/364207119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbe80a2930 0x7fbbe8132de0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.049+0000 7fbbed75d640 1 -- 172.21.15.178:0/364207119 >> 172.21.15.178:0/364207119 conn(0x7fbbe80a5ed0 msgr2=0x7fbbe80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.049+0000 7fbbed75d640 1 -- 172.21.15.178:0/364207119 shutdown_connections 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.049+0000 7fbbed75d640 1 -- 172.21.15.178:0/364207119 wait complete. 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient: build_initial_monmap 2024-09-15T01:48:21.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient: monmap: 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:21.069 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 1 Processor -- start 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 1 -- start start 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient: init 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient: _add_conns ranks=[0,4,3,7,5,1,6,2,8] 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe809faa0 0x7fbbe809af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient(hunting): picked mon.noname-a con 0x7fbbe809faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe80a08c0 0x7fbbe8131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient(hunting): picked mon.noname-c con 0x7fbbe80a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbe80a2930 0x7fbbe811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient(hunting): picked mon.noname-g con 0x7fbbe80a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:21.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe809bbc0 con 0x7fbbe809faa0 2024-09-15T01:48:21.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe809d630 con 0x7fbbe80a2930 2024-09-15T01:48:21.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe811ec40 con 0x7fbbe80a08c0 2024-09-15T01:48:21.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:21.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbed75d640 10 monclient(hunting): authenticate will time out at 1705.800960s 2024-09-15T01:48:21.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbe6ffd640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe809faa0 msgr2=0x7fbbe809af00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:21.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.050+0000 7fbbe6ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe809faa0 0x7fbbe809af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe80a08c0 0x7fbbe8131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe80a08c0 0x7fbbe8131820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:34218/0 (socket says 172.21.15.178:34218) 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 1 -- 172.21.15.178:0/4265995958 learned_addr learned my addr 172.21.15.178:0/4265995958 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 10 monclient(hunting): get_auth_request con 0x7fbbe80a08c0 auth_method 0 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 10 monclient(hunting): handle_auth_done global_id 44932 payload 824 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe77fe640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbe80a2930 0x7fbbe811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 1 -- 172.21.15.178:0/4265995958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbe80a2930 msgr2=0x7fbbe811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbbe80a2930 0x7fbbe811c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 1 -- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe809faa0 msgr2=0x7fbbe809af00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe809faa0 0x7fbbe809af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 10 monclient: _finish_hunting 0 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 1 monclient: found mon.noname-c 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 1 -- 172.21.15.178:0/4265995958 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd0009450 con 0x7fbbe80a08c0 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe67fc640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe80a08c0 0x7fbbe8131820 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000ea80 tx=0x7fbbd00091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbd000ce90 con 0x7fbbe80a08c0 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbdeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbdeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:48:21.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbdeffd640 10 monclient: dump: 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.074 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbdeffd640 10 monclient: _finish_auth 0 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbdeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbed75d640 5 monclient: authenticate success, global_id 44932 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd0013470 con 0x7fbbe80a08c0 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbed75d640 10 monclient: _renew_subs 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbed75d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbed75d640 1 -- 172.21.15.178:0/4265995958 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd8009390 con 0x7fbbe80a08c0 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbdeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbd0013a60 con 0x7fbbe80a08c0 2024-09-15T01:48:21.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbdeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbdeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbdeffd640 10 monclient: dump: 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.076 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.077 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.077 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.077 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.077 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbe57fa640 4 set_mon_vals no callback set 2024-09-15T01:48:21.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbed75d640 10 monclient: _renew_subs 2024-09-15T01:48:21.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbed75d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:48:21.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.051+0000 7fbbed75d640 1 -- 172.21.15.178:0/4265995958 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe0009a20 con 0x7fbbe80a08c0 2024-09-15T01:48:21.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbed75d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:21.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fbbd0013bc0 con 0x7fbbe80a08c0 2024-09-15T01:48:21.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbed75d640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8136ff0 0x7fbbe811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbed75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbed75d640 1 -- 172.21.15.178:0/4265995958 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe804c4a0 con 0x7fbbe8136ff0 2024-09-15T01:48:21.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbe6ffd640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8136ff0 0x7fbbe811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbe6ffd640 10 monclient: get_auth_request con 0x7fbbe8136ff0 auth_method 0 2024-09-15T01:48:21.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbe6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbe6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbe6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbe6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbe6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbe6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbdeffd640 1 --2- 172.21.15.178:0/4265995958 >> v2:172.21.15.204:6812/28210 conn(0x7fbbc80397b0 0x7fbbc803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fbbd004bfa0 con 0x7fbbe80a08c0 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.052+0000 7fbbe6ffd640 10 monclient(hunting): handle_auth_done global_id 94346 payload 824 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbe6ffd640 1 -- 172.21.15.178:0/4265995958 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbe805ed70 con 0x7fbbe8136ff0 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbe6ffd640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8136ff0 0x7fbbe811ca70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009e60 tx=0x7fbbe0037000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbe77fe640 1 --2- 172.21.15.178:0/4265995958 >> v2:172.21.15.204:6812/28210 conn(0x7fbbc80397b0 0x7fbbc803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbe77fe640 10 monclient: get_auth_request con 0x7fbbc80397b0 auth_method 0 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbe0007bd0 con 0x7fbbe8136ff0 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbdeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbdeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbdeffd640 10 monclient: dump: 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.079 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fbbe0031a40 con 0x7fbbe8136ff0 2024-09-15T01:48:21.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbdeffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:21.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbdeffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:21.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8136ff0 msgr2=0x7fbbe811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbdeffd640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8136ff0 0x7fbbe811ca70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009e60 tx=0x7fbbe0037000 comp rx=0 tx=0).stop 2024-09-15T01:48:21.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8136ff0 msgr2=0x7fbbe811ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:21.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbdeffd640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8136ff0 0x7fbbe811ca70 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009e60 tx=0x7fbbe0037000 comp rx=0 tx=0).stop 2024-09-15T01:48:21.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.053+0000 7fbbe77fe640 1 --2- 172.21.15.178:0/4265995958 >> v2:172.21.15.204:6812/28210 conn(0x7fbbc80397b0 0x7fbbc803bc60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8006e70 tx=0x7fbbd8008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbed75d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:48:21.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbed75d640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8131d60 0x7fbbe811cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbed75d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbed75d640 1 -- 172.21.15.178:0/4265995958 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe811f0b0 con 0x7fbbe8131d60 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbe6ffd640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8131d60 0x7fbbe811cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbe6ffd640 10 monclient: get_auth_request con 0x7fbbe8131d60 auth_method 0 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbe6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbe6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbe6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbe6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbe6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbe6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbe6ffd640 10 monclient(hunting): handle_auth_done global_id 94355 payload 824 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbe6ffd640 1 -- 172.21.15.178:0/4265995958 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbbe811edb0 con 0x7fbbe8131d60 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbe6ffd640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8131d60 0x7fbbe811cfb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009e90 tx=0x7fbbe003e000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbe004d040 con 0x7fbbe8131d60 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbdeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbdeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.059+0000 7fbbdeffd640 10 monclient: dump: 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.086 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9120 (secure 0 0 0) 0x7fbbe003de20 con 0x7fbbe8131d60 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbdeffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbdeffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8131d60 msgr2=0x7fbbe811cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbdeffd640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8131d60 0x7fbbe811cfb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009e90 tx=0x7fbbe003e000 comp rx=0 tx=0).stop 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbdeffd640 1 -- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8131d60 msgr2=0x7fbbe811cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:21.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbdeffd640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fbbe8131d60 0x7fbbe811cfb0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009e90 tx=0x7fbbe003e000 comp rx=0 tx=0).stop 2024-09-15T01:48:21.088 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:48:21.088 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:48:21.088 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:48:21.088 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:48:21.088 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 156, 2024-09-15T01:48:21.088 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:48:21.088 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:48:21.088 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:48:21.088 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:21.089 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:21.090 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:21.091 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:21.092 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.093 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:48:21.094 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:48:21.097 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:48:21.098 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.100 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.101 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:48:21.103 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.104 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.105 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.106 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.108 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:21.109 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:21.110 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.110 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.110 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:48:21.110 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:48:21.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbed75d640 1 -- 172.21.15.178:0/4265995958 >> v2:172.21.15.204:6812/28210 conn(0x7fbbc80397b0 msgr2=0x7fbbc803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbed75d640 1 --2- 172.21.15.178:0/4265995958 >> v2:172.21.15.204:6812/28210 conn(0x7fbbc80397b0 0x7fbbc803bc60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8006e70 tx=0x7fbbd8008040 comp rx=0 tx=0).stop 2024-09-15T01:48:21.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbed75d640 10 monclient: shutdown 2024-09-15T01:48:21.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbed75d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:21.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbed75d640 1 -- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe80a08c0 msgr2=0x7fbbe8131820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbed75d640 1 --2- 172.21.15.178:0/4265995958 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbbe80a08c0 0x7fbbe8131820 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000ea80 tx=0x7fbbd00091d0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbe77fe640 1 -- 172.21.15.178:0/4265995958 reap_dead start 2024-09-15T01:48:21.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbe77fe640 1 -- 172.21.15.178:0/4265995958 reap_dead start 2024-09-15T01:48:21.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbed75d640 1 -- 172.21.15.178:0/4265995958 shutdown_connections 2024-09-15T01:48:21.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbed75d640 1 -- 172.21.15.178:0/4265995958 >> 172.21.15.178:0/4265995958 conn(0x7fbbe80a5ed0 msgr2=0x7fbbe80a56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:21.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbed75d640 1 -- 172.21.15.178:0/4265995958 shutdown_connections 2024-09-15T01:48:21.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.060+0000 7fbbed75d640 1 -- 172.21.15.178:0/4265995958 wait complete. 2024-09-15T01:48:21.113 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:48:21.113 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-09-15T01:48:21.113 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:48:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.206+0000 7efee6c17640 10 monclient: get_monmap_and_config 2024-09-15T01:48:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.206+0000 7efee6c17640 10 monclient: build_initial_monmap 2024-09-15T01:48:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.206+0000 7efee6c17640 10 monclient: monmap: 2024-09-15T01:48:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:48:21.206947+0000 2024-09-15T01:48:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:48:21.206947+0000 2024-09-15T01:48:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:21.208 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:21.209 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:21.209 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.206+0000 7efee6c17640 1 Processor -- start 2024-09-15T01:48:21.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.206+0000 7efee6c17640 1 -- start start 2024-09-15T01:48:21.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.206+0000 7efee6c17640 10 monclient: init 2024-09-15T01:48:21.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:21.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 10 monclient: _add_conns ranks=[9,3,7,8,6,5,2,0,4,1,10] 2024-09-15T01:48:21.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efee00678d0 0x7efee009e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 10 monclient(hunting): picked mon.noname-a con 0x7efee00678d0 addr v2:172.21.15.178:3300/0 2024-09-15T01:48:21.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee006ea30 0x7efee009ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 10 monclient(hunting): picked mon.noname-g con 0x7efee006ea30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:21.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efee009f180 0x7efee01306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 10 monclient(hunting): picked mon.noname-i con 0x7efee009f180 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:48:21.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee009f5c0 con 0x7efee00678d0 2024-09-15T01:48:21.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee006d5b0 con 0x7efee006ea30 2024-09-15T01:48:21.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efee0067ca0 con 0x7efee009f180 2024-09-15T01:48:21.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:21.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee6c17640 10 monclient(hunting): authenticate will time out at 1705.957315s 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee49b3640 1 -- >> v2:172.21.15.178:3300/0 conn(0x7efee00678d0 msgr2=0x7efee009e700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee49b3640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efee00678d0 0x7efee009e700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee006ea30 0x7efee009ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee006ea30 0x7efee009ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:49592/0 (socket says 172.21.15.178:49592) 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 1 -- 172.21.15.178:0/2279450682 learned_addr learned my addr 172.21.15.178:0/2279450682 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 10 monclient(hunting): get_auth_request con 0x7efee006ea30 auth_method 0 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee51b4640 1 --2- 172.21.15.178:0/2279450682 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efee009f180 0x7efee01306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee51b4640 10 monclient(hunting): get_auth_request con 0x7efee009f180 auth_method 0 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee51b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee51b4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee51b4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee51b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee51b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efee51b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 10 monclient(hunting): handle_auth_done global_id 64524 payload 274 2024-09-15T01:48:21.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 1 -- 172.21.15.178:0/2279450682 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efee009f180 msgr2=0x7efee01306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:21.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 1 --2- 172.21.15.178:0/2279450682 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efee009f180 0x7efee01306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 1 -- 172.21.15.178:0/2279450682 >> v2:172.21.15.178:3300/0 conn(0x7efee00678d0 msgr2=0x7efee009e700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:21.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 1 --2- 172.21.15.178:0/2279450682 >> v2:172.21.15.178:3300/0 conn(0x7efee00678d0 0x7efee009e700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 1 monclient: found mon.noname-g 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.207+0000 7efed7fff640 1 -- 172.21.15.178:0/2279450682 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee004bf60 con 0x7efee006ea30 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed7fff640 1 --2- 172.21.15.178:0/2279450682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee006ea30 0x7efee009ec40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efed0009940 tx=0x7efed00091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=b3cccfc647896880 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed77fe640 1 -- 172.21.15.178:0/2279450682 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efed002ad20 con 0x7efee006ea30 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed77fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed77fe640 10 monclient: dump: 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.223 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.224 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.224 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.224 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.224 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.224 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.224 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.224 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.224 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.224 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.224 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.224 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.224 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed77fe640 10 monclient: _finish_auth 0 2024-09-15T01:48:21.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:21.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed77fe640 1 -- 172.21.15.178:0/2279450682 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed0031820 con 0x7efee006ea30 2024-09-15T01:48:21.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efee6c17640 5 monclient: authenticate success, global_id 64524 2024-09-15T01:48:21.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efee6c17640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:48:21.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:21.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efee6c17640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:21.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed77fe640 1 -- 172.21.15.178:0/2279450682 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efed002aea0 con 0x7efee006ea30 2024-09-15T01:48:21.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efee6c17640 4 set_mon_vals no callback set 2024-09-15T01:48:21.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:48:21.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.208+0000 7efed77fe640 10 monclient: dump: 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.226 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 10 monclient: shutdown 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 1 -- 172.21.15.178:0/2279450682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee006ea30 msgr2=0x7efee009ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 1 --2- 172.21.15.178:0/2279450682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee006ea30 0x7efee009ec40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efed0009940 tx=0x7efed00091d0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 1 -- 172.21.15.178:0/2279450682 shutdown_connections 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 1 --2- 172.21.15.178:0/2279450682 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efee009f180 0x7efee01306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 1 --2- 172.21.15.178:0/2279450682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee006ea30 0x7efee009ec40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 1 --2- 172.21.15.178:0/2279450682 >> v2:172.21.15.178:3300/0 conn(0x7efee00678d0 0x7efee009e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 1 -- 172.21.15.178:0/2279450682 >> 172.21.15.178:0/2279450682 conn(0x7efee00a4390 msgr2=0x7efee0067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:21.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 1 -- 172.21.15.178:0/2279450682 shutdown_connections 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 1 -- 172.21.15.178:0/2279450682 wait complete. 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 10 monclient: build_initial_monmap 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.209+0000 7efee6c17640 10 monclient: monmap: 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:21.228 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 1 Processor -- start 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 1 -- start start 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 10 monclient: init 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 10 monclient: _add_conns ranks=[4,0,1,6,8,5,2,3,7] 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efee00678d0 0x7efee013f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 10 monclient(hunting): picked mon.noname-c con 0x7efee00678d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee006ea30 0x7efee0133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 10 monclient(hunting): picked mon.noname-a con 0x7efee006ea30 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee009f180 0x7efee011d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 10 monclient(hunting): picked mon.noname-f con 0x7efee009f180 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee006c900 con 0x7efee006ea30 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01406a0 con 0x7efee009f180 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efee0120830 con 0x7efee00678d0 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efee6c17640 10 monclient(hunting): authenticate will time out at 1705.960905s 2024-09-15T01:48:21.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efed7fff640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee006ea30 msgr2=0x7efee0133710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:21.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.210+0000 7efed7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee006ea30 0x7efee0133710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:21.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee51b4640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee009f180 msgr2=0x7efee011d250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:21.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efee00678d0 0x7efee013f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee51b4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee009f180 0x7efee011d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:21.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efee00678d0 0x7efee013f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:34234/0 (socket says 172.21.15.178:34234) 2024-09-15T01:48:21.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 1 -- 172.21.15.178:0/1002158236 learned_addr learned my addr 172.21.15.178:0/1002158236 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:21.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 10 monclient(hunting): get_auth_request con 0x7efee00678d0 auth_method 0 2024-09-15T01:48:21.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 10 monclient(hunting): handle_auth_done global_id 44941 payload 824 2024-09-15T01:48:21.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 1 -- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee009f180 msgr2=0x7efee011d250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:21.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee009f180 0x7efee011d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 1 -- 172.21.15.178:0/1002158236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee006ea30 msgr2=0x7efee0133710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:21.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee006ea30 0x7efee0133710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 10 monclient: _finish_hunting 0 2024-09-15T01:48:21.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 1 monclient: found mon.noname-c 2024-09-15T01:48:21.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:21.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:48:21.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 1 -- 172.21.15.178:0/1002158236 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed0009450 con 0x7efee00678d0 2024-09-15T01:48:21.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee49b3640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efee00678d0 0x7efee013f840 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efedc00b5c0 tx=0x7efedc009120 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efedc006150 con 0x7efee00678d0 2024-09-15T01:48:21.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:48:21.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed4ff9640 10 monclient: dump: 2024-09-15T01:48:21.233 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.233 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.233 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.233 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.234 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.234 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.234 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.234 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.234 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.234 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.234 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.234 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.234 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.234 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.234 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.234 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed4ff9640 10 monclient: _finish_auth 0 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc0062d0 con 0x7efee00678d0 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee6c17640 5 monclient: authenticate success, global_id 44941 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee6c17640 10 monclient: _renew_subs 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee6c17640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee6c17640 1 -- 172.21.15.178:0/1002158236 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee0120b10 con 0x7efee00678d0 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efedc0103c0 con 0x7efee00678d0 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed4ff9640 10 monclient: dump: 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.235 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.236 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.237 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee6c17640 10 monclient: _renew_subs 2024-09-15T01:48:21.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee6c17640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:48:21.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efee6c17640 1 -- 172.21.15.178:0/1002158236 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee013fd80 con 0x7efee00678d0 2024-09-15T01:48:21.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.211+0000 7efed6ffd640 4 set_mon_vals no callback set 2024-09-15T01:48:21.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7efedc003b80 con 0x7efee00678d0 2024-09-15T01:48:21.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efee6c17640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:21.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efee6c17640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee011f8d0 0x7efee011d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efee6c17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efee6c17640 1 -- 172.21.15.178:0/1002158236 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee004c4a0 con 0x7efee011f8d0 2024-09-15T01:48:21.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efed4ff9640 1 --2- 172.21.15.178:0/1002158236 >> v2:172.21.15.204:6812/28210 conn(0x7efecc0397b0 0x7efecc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7efedc04afb0 con 0x7efee00678d0 2024-09-15T01:48:21.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efed7fff640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee011f8d0 0x7efee011d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efed7fff640 10 monclient: get_auth_request con 0x7efee011f8d0 auth_method 0 2024-09-15T01:48:21.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efed7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efed7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efed7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efed7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efed7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efee51b4640 1 --2- 172.21.15.178:0/1002158236 >> v2:172.21.15.204:6812/28210 conn(0x7efecc0397b0 0x7efecc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efed7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.212+0000 7efee51b4640 10 monclient: get_auth_request con 0x7efecc0397b0 auth_method 0 2024-09-15T01:48:21.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed7fff640 10 monclient(hunting): handle_auth_done global_id 64533 payload 824 2024-09-15T01:48:21.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed7fff640 1 -- 172.21.15.178:0/1002158236 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efee005ed70 con 0x7efee011f8d0 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed7fff640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee011f8d0 0x7efee011d790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efee0068080 tx=0x7efed0032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efee51b4640 1 --2- 172.21.15.178:0/1002158236 >> v2:172.21.15.204:6812/28210 conn(0x7efecc0397b0 0x7efecc03bc60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7efee011e910 tx=0x7efed800b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efed00328c0 con 0x7efee011f8d0 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed4ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed4ff9640 10 monclient: dump: 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.239 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7efed0032440 con 0x7efee011f8d0 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee011f8d0 msgr2=0x7efee011d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed4ff9640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee011f8d0 0x7efee011d790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efee0068080 tx=0x7efed0032000 comp rx=0 tx=0).stop 2024-09-15T01:48:21.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee011f8d0 msgr2=0x7efee011d790 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:21.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.213+0000 7efed4ff9640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee011f8d0 0x7efee011d790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.219+0000 7efee6c17640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:48:21.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.219+0000 7efee6c17640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee0133c50 0x7efee011dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.219+0000 7efee6c17640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.219+0000 7efee6c17640 1 -- 172.21.15.178:0/1002158236 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee00a3640 con 0x7efee0133c50 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.219+0000 7efed7fff640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee0133c50 0x7efee011dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.219+0000 7efed7fff640 10 monclient: get_auth_request con 0x7efee0133c50 auth_method 0 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.219+0000 7efed7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.219+0000 7efed7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.219+0000 7efed7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.219+0000 7efed7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.219+0000 7efed7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.219+0000 7efed7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed7fff640 10 monclient(hunting): handle_auth_done global_id 64542 payload 824 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed7fff640 1 -- 172.21.15.178:0/1002158236 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efee004bc80 con 0x7efee0133c50 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed7fff640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee0133c50 0x7efee011dcd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efed00097e0 tx=0x7efed003c000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efed0047ab0 con 0x7efee0133c50 2024-09-15T01:48:21.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed4ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:48:21.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed4ff9640 10 monclient: dump: 2024-09-15T01:48:21.244 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.244 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.244 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.244 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.244 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.244 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.244 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.244 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.244 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.244 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9120 (secure 0 0 0) 0x7efed0047c10 con 0x7efee0133c50 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee0133c50 msgr2=0x7efee011dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed4ff9640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee0133c50 0x7efee011dcd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efed00097e0 tx=0x7efed003c000 comp rx=0 tx=0).stop 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.220+0000 7efed4ff9640 1 -- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee0133c50 msgr2=0x7efee011dcd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:21.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efed4ff9640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efee0133c50 0x7efee011dcd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.246 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:48:21.246 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:48:21.246 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:48:21.246 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:48:21.246 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 156, 2024-09-15T01:48:21.246 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:48:21.246 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:48:21.246 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:48:21.246 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:48:21.246 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:48:21.247 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:48:21.247 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:48:21.247 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:48:21.247 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.247 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 4, 2024-09-15T01:48:21.247 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:21.247 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:48:21.247 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:48:21.247 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:21.247 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:21.247 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:21.247 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:21.248 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:21.249 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:21.249 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:21.249 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:21.249 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:21.249 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.249 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:48:21.249 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:48:21.249 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:48:21.249 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:48:21.249 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:48:21.249 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:48:21.250 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:21.250 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:21.250 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:48:21.250 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:48:21.250 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:48:21.250 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:48:21.250 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:48:21.250 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:48:21.250 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:48:21.250 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:21.250 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:48:21.250 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.251 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:48:21.252 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:48:21.252 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.252 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:48:21.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.252 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.252 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:48:21.252 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.253 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.254 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.255 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.256 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.258 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:48:21.258 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.258 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.258 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.258 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:48:21.258 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:48:21.258 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.258 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.258 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.259 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:48:21.260 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.261 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:48:21.261 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.261 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.261 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:48:21.261 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.261 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.261 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.262 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:48:21.263 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.264 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.264 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.264 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:48:21.264 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:48:21.264 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.264 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.264 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.265 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:48:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:48:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:48:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:48:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:48:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:48:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.267 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.268 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:21.269 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:21.269 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:21.269 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.269 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.269 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:48:21.269 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:48:21.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efee6c17640 1 -- 172.21.15.178:0/1002158236 >> v2:172.21.15.204:6812/28210 conn(0x7efecc0397b0 msgr2=0x7efecc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efee6c17640 1 --2- 172.21.15.178:0/1002158236 >> v2:172.21.15.204:6812/28210 conn(0x7efecc0397b0 0x7efecc03bc60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7efee011e910 tx=0x7efed800b040 comp rx=0 tx=0).stop 2024-09-15T01:48:21.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efee6c17640 10 monclient: shutdown 2024-09-15T01:48:21.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efee6c17640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:21.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efee6c17640 1 -- 172.21.15.178:0/1002158236 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efee00678d0 msgr2=0x7efee013f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efee6c17640 1 --2- 172.21.15.178:0/1002158236 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efee00678d0 0x7efee013f840 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efedc00b5c0 tx=0x7efedc009120 comp rx=0 tx=0).stop 2024-09-15T01:48:21.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efee51b4640 1 -- 172.21.15.178:0/1002158236 reap_dead start 2024-09-15T01:48:21.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efee51b4640 1 -- 172.21.15.178:0/1002158236 reap_dead start 2024-09-15T01:48:21.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efee6c17640 1 -- 172.21.15.178:0/1002158236 shutdown_connections 2024-09-15T01:48:21.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efee6c17640 1 -- 172.21.15.178:0/1002158236 >> 172.21.15.178:0/1002158236 conn(0x7efee00a4390 msgr2=0x7efee009d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:21.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efee6c17640 1 -- 172.21.15.178:0/1002158236 shutdown_connections 2024-09-15T01:48:21.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.221+0000 7efee6c17640 1 -- 172.21.15.178:0/1002158236 wait complete. 2024-09-15T01:48:21.272 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:48:21.272 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-15T01:48:21.272 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:48:21.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.362+0000 7f1f00a34640 10 monclient: get_monmap_and_config 2024-09-15T01:48:21.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.362+0000 7f1f00a34640 10 monclient: build_initial_monmap 2024-09-15T01:48:21.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.362+0000 7f1f00a34640 10 monclient: monmap: 2024-09-15T01:48:21.365 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:21.365 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.365 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:48:21.363711+0000 2024-09-15T01:48:21.365 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:48:21.363711+0000 2024-09-15T01:48:21.366 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:21.366 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.366 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:21.366 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:21.366 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:21.366 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:21.366 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:21.366 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:21.366 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:21.366 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:21.366 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:21.366 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:21.367 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:21.367 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.363+0000 7f1f00a34640 1 Processor -- start 2024-09-15T01:48:21.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.363+0000 7f1f00a34640 1 -- start start 2024-09-15T01:48:21.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.363+0000 7f1f00a34640 10 monclient: init 2024-09-15T01:48:21.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:21.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 10 monclient: _add_conns ranks=[1,7,2,6,8,4,10,0,3,5,9] 2024-09-15T01:48:21.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1efc0a2930 addr v1:172.21.15.204:6789/0 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1efc132de0 0x7f1efc137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 10 monclient(hunting): picked mon.noname-i con 0x7f1efc132de0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1efc1380e0 0x7f1efc13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 10 monclient(hunting): picked mon.noname-b con 0x7f1efc1380e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc04bf60 con 0x7f1efc0a2930 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc09dd80 con 0x7f1efc1380e0 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0a2d00 con 0x7f1efc132de0 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1f00a34640 10 monclient(hunting): authenticate will time out at 1706.114199s 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efa59c640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7f1efc0a2930 legacy=0x7f1efc09faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:21.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1efc1380e0 0x7f1efc13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1efc1380e0 0x7f1efc13a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:44436/0 (socket says 172.21.15.178:44436) 2024-09-15T01:48:21.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 1 -- 172.21.15.178:0/1138618067 learned_addr learned my addr 172.21.15.178:0/1138618067 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:21.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 10 monclient(hunting): get_auth_request con 0x7f1efc1380e0 auth_method 0 2024-09-15T01:48:21.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1ef9d9b640 1 --2- 172.21.15.178:0/1138618067 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1efc132de0 0x7f1efc137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1ef9d9b640 10 monclient(hunting): get_auth_request con 0x7f1efc132de0 auth_method 0 2024-09-15T01:48:21.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1ef9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1ef9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1ef9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 10 monclient(hunting): handle_auth_done global_id 35372 payload 274 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 1 -- 172.21.15.178:0/1138618067 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1efc132de0 msgr2=0x7f1efc137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 1 --2- 172.21.15.178:0/1138618067 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1efc132de0 0x7f1efc137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 1 -- 172.21.15.178:0/1138618067 >> v1:172.21.15.204:6789/0 conn(0x7f1efc0a2930 legacy=0x7f1efc09faa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 1 monclient: found mon.noname-b 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.364+0000 7f1efad9d640 1 -- 172.21.15.178:0/1138618067 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc1331b0 con 0x7f1efc1380e0 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1efad9d640 1 --2- 172.21.15.178:0/1138618067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1efc1380e0 0x7f1efc13a4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009f50 tx=0x7f1ef4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d372454898913ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1ef959a640 1 -- 172.21.15.178:0/1138618067 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1ef4007880 con 0x7f1efc1380e0 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1ef959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1ef959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1ef959a640 10 monclient: dump: 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.380 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1ef959a640 10 monclient: _finish_auth 0 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1ef959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1f00a34640 5 monclient: authenticate success, global_id 35372 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1ef959a640 1 -- 172.21.15.178:0/1138618067 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef402d050 con 0x7f1efc1380e0 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1ef959a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1f00a34640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1f00a34640 4 set_mon_vals no callback set 2024-09-15T01:48:21.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1ef959a640 1 -- 172.21.15.178:0/1138618067 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1ef4030360 con 0x7f1efc1380e0 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1ef959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1ef959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1ef959a640 10 monclient: dump: 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.382 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1f00a34640 10 monclient: shutdown 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1f00a34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1f00a34640 1 -- 172.21.15.178:0/1138618067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1efc1380e0 msgr2=0x7f1efc13a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.365+0000 7f1f00a34640 1 --2- 172.21.15.178:0/1138618067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1efc1380e0 0x7f1efc13a4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009f50 tx=0x7f1ef4009840 comp rx=0 tx=0).stop 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.366+0000 7f1f00a34640 1 -- 172.21.15.178:0/1138618067 shutdown_connections 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.366+0000 7f1f00a34640 1 --2- 172.21.15.178:0/1138618067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1efc1380e0 0x7f1efc13a4c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.366+0000 7f1f00a34640 1 --2- 172.21.15.178:0/1138618067 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f1efc132de0 0x7f1efc137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.366+0000 7f1f00a34640 1 -- 172.21.15.178:0/1138618067 >> 172.21.15.178:0/1138618067 conn(0x7f1efc0a5ed0 msgr2=0x7f1efc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.366+0000 7f1f00a34640 1 -- 172.21.15.178:0/1138618067 shutdown_connections 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.366+0000 7f1f00a34640 1 -- 172.21.15.178:0/1138618067 wait complete. 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.366+0000 7f1f00a34640 10 monclient: build_initial_monmap 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.366+0000 7f1f00a34640 10 monclient: monmap: 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:21.384 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.366+0000 7f1f00a34640 1 Processor -- start 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.366+0000 7f1f00a34640 1 -- start start 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.366+0000 7f1f00a34640 10 monclient: init 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 10 monclient: _add_conns ranks=[0,5,1,7,2,6,8,4,3] 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1efc0a2930 0x7f1efc09a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 10 monclient(hunting): picked mon.noname-a con 0x7f1efc0a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc132de0 0x7f1efc09adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 10 monclient(hunting): picked mon.noname-h con 0x7f1efc132de0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1efc1380e0 0x7f1efc09b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 10 monclient(hunting): picked mon.noname-f con 0x7f1efc1380e0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc09c3a0 con 0x7f1efc0a2930 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0a2390 con 0x7f1efc1380e0 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc09d630 con 0x7f1efc132de0 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1efa59c640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1efc0a2930 msgr2=0x7f1efc09a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1f00a34640 10 monclient(hunting): authenticate will time out at 1706.117396s 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1efa59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1efc0a2930 0x7f1efc09a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1efad9d640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1efc1380e0 msgr2=0x7f1efc09b330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1efad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1efc1380e0 0x7f1efc09b330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc132de0 0x7f1efc09adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc132de0 0x7f1efc09adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:41646/0 (socket says 172.21.15.178:41646) 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 1 -- 172.21.15.178:0/1801715166 learned_addr learned my addr 172.21.15.178:0/1801715166 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 10 monclient(hunting): get_auth_request con 0x7f1efc132de0 auth_method 0 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 10 monclient(hunting): handle_auth_done global_id 7736 payload 824 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 1 -- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1efc1380e0 msgr2=0x7f1efc09b330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1efc1380e0 0x7f1efc09b330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 1 -- 172.21.15.178:0/1801715166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1efc0a2930 msgr2=0x7f1efc09a8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1efc0a2930 0x7f1efc09a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 10 monclient: _finish_hunting 0 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 1 monclient: found mon.noname-h 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.367+0000 7f1ef9d9b640 1 -- 172.21.15.178:0/1801715166 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef4009ae0 con 0x7f1efc132de0 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1ef9d9b640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc132de0 0x7f1efc09adf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000d8c0 tx=0x7f1ef0008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1ef000de50 con 0x7f1efc132de0 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1eea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1eea7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:48:21.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1eea7fc640 10 monclient: dump: 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1eea7fc640 10 monclient: _finish_auth 0 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1eea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1f00a34640 5 monclient: authenticate success, global_id 7736 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef000bd50 con 0x7f1efc132de0 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1f00a34640 10 monclient: _renew_subs 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1f00a34640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1f00a34640 1 -- 172.21.15.178:0/1801715166 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc11d6d0 con 0x7f1efc132de0 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1eea7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1ef00128a0 con 0x7f1efc132de0 2024-09-15T01:48:21.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1eea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1eea7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1eea7fc640 10 monclient: dump: 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.391 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1f00a34640 10 monclient: _renew_subs 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1ef8d99640 4 set_mon_vals no callback set 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1f00a34640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.368+0000 7f1f00a34640 1 -- 172.21.15.178:0/1801715166 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc11c910 con 0x7f1efc132de0 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1f00a34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f1ef0012420 con 0x7f1efc132de0 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1f00a34640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc070620 0x7f1efc09b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1f00a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1f00a34640 1 -- 172.21.15.178:0/1801715166 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc04c4a0 con 0x7f1efc070620 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1eea7fc640 1 --2- 172.21.15.178:0/1801715166 >> v2:172.21.15.204:6812/28210 conn(0x7f1edc0397b0 0x7f1edc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1efa59c640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc070620 0x7f1efc09b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1efa59c640 10 monclient: get_auth_request con 0x7f1efc070620 auth_method 0 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1efa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f1ef004bf10 con 0x7f1efc132de0 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1efa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1efa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1efa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1efa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1efa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1efad9d640 1 --2- 172.21.15.178:0/1801715166 >> v2:172.21.15.204:6812/28210 conn(0x7f1edc0397b0 0x7f1edc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1efad9d640 10 monclient: get_auth_request con 0x7f1edc0397b0 auth_method 0 2024-09-15T01:48:21.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1efa59c640 10 monclient(hunting): handle_auth_done global_id 7745 payload 824 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.369+0000 7f1efa59c640 1 -- 172.21.15.178:0/1801715166 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1efc05ed70 con 0x7f1efc070620 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1efa59c640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc070620 0x7f1efc09b870 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0038f0 tx=0x7f1eec0090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1efad9d640 1 --2- 172.21.15.178:0/1801715166 >> v2:172.21.15.204:6812/28210 conn(0x7f1edc0397b0 0x7f1edc03bc60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4033000 tx=0x7f1ef4036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1eec0050f0 con 0x7f1efc070620 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1eea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1eea7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1eea7fc640 10 monclient: dump: 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.394 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f1eec003b30 con 0x7f1efc070620 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1eea7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1eea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:21.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc070620 msgr2=0x7f1efc09b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1eea7fc640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc070620 0x7f1efc09b870 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0038f0 tx=0x7f1eec0090f0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc070620 msgr2=0x7f1efc09b870 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:21.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.370+0000 7f1eea7fc640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc070620 0x7f1efc09b870 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1f00a34640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:48:21.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1f00a34640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc096a30 0x7f1efc096e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1f00a34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1f00a34640 1 -- 172.21.15.178:0/1801715166 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc04bc80 con 0x7f1efc096a30 2024-09-15T01:48:21.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1efa59c640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc096a30 0x7f1efc096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1efa59c640 10 monclient: get_auth_request con 0x7f1efc096a30 auth_method 0 2024-09-15T01:48:21.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1efa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1efa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1efa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1efa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1efa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1efa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1efa59c640 10 monclient(hunting): handle_auth_done global_id 7754 payload 824 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.376+0000 7f1efa59c640 1 -- 172.21.15.178:0/1801715166 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1efc09bdb0 con 0x7f1efc096a30 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.377+0000 7f1efa59c640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc096a30 0x7f1efc096e30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009380 tx=0x7f1eec015000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.377+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1eec0208a0 con 0x7f1efc096a30 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.377+0000 7f1eea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.377+0000 7f1eea7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.377+0000 7f1eea7fc640 10 monclient: dump: 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.399 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.377+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9292 (secure 0 0 0) 0x7f1eec020420 con 0x7f1efc096a30 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.377+0000 7f1eea7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:48:21.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.377+0000 7f1eea7fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:21.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.377+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc096a30 msgr2=0x7f1efc096e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.377+0000 7f1eea7fc640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc096a30 0x7f1efc096e30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009380 tx=0x7f1eec015000 comp rx=0 tx=0).stop 2024-09-15T01:48:21.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.377+0000 7f1eea7fc640 1 -- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc096a30 msgr2=0x7f1efc096e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:21.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.377+0000 7f1eea7fc640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc096a30 0x7f1efc096e30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.401 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:48:21.401 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:48:21.401 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:48:21.401 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 156, 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 5, 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:21.402 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:48:21.403 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:48:21.404 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:21.404 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:21.404 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:21.404 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:21.404 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:21.404 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:21.404 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:21.404 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:21.404 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:21.404 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.404 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:48:21.405 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:21.406 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.409 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:48:21.409 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:48:21.409 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.409 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:48:21.409 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.409 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.410 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.411 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:48:21.413 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.416 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:48:21.417 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.419 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.420 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:48:21.423 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.424 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.424 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:48:21.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.424 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:21.424 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:21.424 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 7 2024-09-15T01:48:21.424 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.424 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.424 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:48:21.424 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:48:21.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.378+0000 7f1f00a34640 1 -- 172.21.15.178:0/1801715166 >> v2:172.21.15.204:6812/28210 conn(0x7f1edc0397b0 msgr2=0x7f1edc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.378+0000 7f1f00a34640 1 --2- 172.21.15.178:0/1801715166 >> v2:172.21.15.204:6812/28210 conn(0x7f1edc0397b0 0x7f1edc03bc60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4033000 tx=0x7f1ef4036000 comp rx=0 tx=0).stop 2024-09-15T01:48:21.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.378+0000 7f1f00a34640 10 monclient: shutdown 2024-09-15T01:48:21.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.378+0000 7f1f00a34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:21.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.378+0000 7f1f00a34640 1 -- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc132de0 msgr2=0x7f1efc09adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.378+0000 7f1f00a34640 1 --2- 172.21.15.178:0/1801715166 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f1efc132de0 0x7f1efc09adf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000d8c0 tx=0x7f1ef0008040 comp rx=0 tx=0).stop 2024-09-15T01:48:21.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.378+0000 7f1efad9d640 1 -- 172.21.15.178:0/1801715166 reap_dead start 2024-09-15T01:48:21.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.378+0000 7f1efad9d640 1 -- 172.21.15.178:0/1801715166 reap_dead start 2024-09-15T01:48:21.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.378+0000 7f1f00a34640 1 -- 172.21.15.178:0/1801715166 shutdown_connections 2024-09-15T01:48:21.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.378+0000 7f1f00a34640 1 -- 172.21.15.178:0/1801715166 >> 172.21.15.178:0/1801715166 conn(0x7f1efc0a5ed0 msgr2=0x7f1efc0a46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:21.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.378+0000 7f1f00a34640 1 -- 172.21.15.178:0/1801715166 shutdown_connections 2024-09-15T01:48:21.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.378+0000 7f1f00a34640 1 -- 172.21.15.178:0/1801715166 wait complete. 2024-09-15T01:48:21.427 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:48:21.427 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-09-15T01:48:21.427 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:48:21.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.518+0000 7f43e235d640 10 monclient: get_monmap_and_config 2024-09-15T01:48:21.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.518+0000 7f43e235d640 10 monclient: build_initial_monmap 2024-09-15T01:48:21.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.518+0000 7f43e235d640 10 monclient: monmap: 2024-09-15T01:48:21.521 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:21.521 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.521 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:48:21.519650+0000 2024-09-15T01:48:21.521 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:48:21.519650+0000 2024-09-15T01:48:21.521 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:21.521 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.521 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:21.522 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:21.522 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:21.522 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:21.522 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:21.522 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:21.522 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:21.522 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:21.522 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:21.522 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:21.522 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:21.522 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.519+0000 7f43e235d640 1 Processor -- start 2024-09-15T01:48:21.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.519+0000 7f43e235d640 1 -- start start 2024-09-15T01:48:21.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.519+0000 7f43e235d640 10 monclient: init 2024-09-15T01:48:21.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:21.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 10 monclient: _add_conns ranks=[0,2,3,8,6,7,5,9,4,1,10] 2024-09-15T01:48:21.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f43dc0a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:48:21.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43dc132de0 0x7f43dc137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 10 monclient(hunting): picked mon.noname-b con 0x7f43dc132de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:48:21.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43dc1380e0 0x7f43dc13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 10 monclient(hunting): picked mon.noname-g con 0x7f43dc1380e0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:21.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43dc04bf60 con 0x7f43dc0a2930 2024-09-15T01:48:21.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc09dd80 con 0x7f43dc132de0 2024-09-15T01:48:21.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0a2d00 con 0x7f43dc1380e0 2024-09-15T01:48:21.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:21.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e235d640 10 monclient(hunting): authenticate will time out at 1706.270108s 2024-09-15T01:48:21.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43dbfff640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f43dc0a2930 legacy=0x7f43dc09faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:48:21.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43dc132de0 0x7f43dc137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43dc132de0 0x7f43dc137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:44438/0 (socket says 172.21.15.178:44438) 2024-09-15T01:48:21.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 1 -- 172.21.15.178:0/3656544362 learned_addr learned my addr 172.21.15.178:0/3656544362 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:21.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 10 monclient(hunting): get_auth_request con 0x7f43dc132de0 auth_method 0 2024-09-15T01:48:21.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e08fa640 1 --2- 172.21.15.178:0/3656544362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43dc1380e0 0x7f43dc13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 10 monclient(hunting): handle_auth_done global_id 35381 payload 274 2024-09-15T01:48:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 1 -- 172.21.15.178:0/3656544362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43dc1380e0 msgr2=0x7f43dc13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 1 --2- 172.21.15.178:0/3656544362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43dc1380e0 0x7f43dc13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 1 -- 172.21.15.178:0/3656544362 >> v1:172.21.15.178:6789/0 conn(0x7f43dc0a2930 legacy=0x7f43dc09faa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:48:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 1 monclient: found mon.noname-b 2024-09-15T01:48:21.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:48:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43db7fe640 1 -- 172.21.15.178:0/3656544362 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc1331b0 con 0x7f43dc132de0 2024-09-15T01:48:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e08fa640 10 monclient: get_auth_request con 0x7f43dc1380e0 auth_method 0 2024-09-15T01:48:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.520+0000 7f43e08fa640 1 --2- 172.21.15.178:0/3656544362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43dc1380e0 0x7f43dc13a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:48:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43db7fe640 1 --2- 172.21.15.178:0/3656544362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43dc132de0 0x7f43dc137ba0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009940 tx=0x7f43d00091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=688c658190884f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43daffd640 1 -- 172.21.15.178:0/3656544362 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43d002ac70 con 0x7f43dc132de0 2024-09-15T01:48:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43daffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:48:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43daffd640 10 monclient: dump: 2024-09-15T01:48:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.539 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.541 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.541 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43daffd640 10 monclient: _finish_auth 0 2024-09-15T01:48:21.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:21.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43e235d640 5 monclient: authenticate success, global_id 35381 2024-09-15T01:48:21.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43daffd640 1 -- 172.21.15.178:0/3656544362 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d002adf0 con 0x7f43dc132de0 2024-09-15T01:48:21.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43e235d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:48:21.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43daffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:21.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43e235d640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:21.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43e235d640 4 set_mon_vals no callback set 2024-09-15T01:48:21.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43daffd640 1 -- 172.21.15.178:0/3656544362 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43d0031ca0 con 0x7f43dc132de0 2024-09-15T01:48:21.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43daffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:48:21.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43daffd640 10 monclient: dump: 2024-09-15T01:48:21.542 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.542 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.542 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.542 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.542 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.542 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.542 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.542 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.542 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.543 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.543 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.543 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.543 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.543 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.543 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.543 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43e235d640 10 monclient: shutdown 2024-09-15T01:48:21.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43e235d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:21.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43e235d640 1 -- 172.21.15.178:0/3656544362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43dc132de0 msgr2=0x7f43dc137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43e235d640 1 --2- 172.21.15.178:0/3656544362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43dc132de0 0x7f43dc137ba0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009940 tx=0x7f43d00091d0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43e235d640 1 -- 172.21.15.178:0/3656544362 shutdown_connections 2024-09-15T01:48:21.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43e235d640 1 --2- 172.21.15.178:0/3656544362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43dc1380e0 0x7f43dc13a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43e235d640 1 --2- 172.21.15.178:0/3656544362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43dc132de0 0x7f43dc137ba0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.521+0000 7f43e235d640 1 -- 172.21.15.178:0/3656544362 >> 172.21.15.178:0/3656544362 conn(0x7f43dc0a5ed0 msgr2=0x7f43dc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:21.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.522+0000 7f43e235d640 1 -- 172.21.15.178:0/3656544362 shutdown_connections 2024-09-15T01:48:21.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.522+0000 7f43e235d640 1 -- 172.21.15.178:0/3656544362 wait complete. 2024-09-15T01:48:21.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.522+0000 7f43e235d640 10 monclient: build_initial_monmap 2024-09-15T01:48:21.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.522+0000 7f43e235d640 10 monclient: monmap: 2024-09-15T01:48:21.544 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:48:21.544 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:21.544 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:48:21.545 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:48:21.545 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:21.545 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:48:21.545 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:21.545 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:21.545 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:21.545 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:21.545 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:21.545 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:21.545 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:21.545 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:21.545 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:21.546 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.522+0000 7f43e235d640 1 Processor -- start 2024-09-15T01:48:21.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 1 -- start start 2024-09-15T01:48:21.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 10 monclient: init 2024-09-15T01:48:21.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:21.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 10 monclient: _add_conns ranks=[0,6,2,8,3,5,7,4,1] 2024-09-15T01:48:21.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc0a2930 0x7f43dc132470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 10 monclient(hunting): picked mon.noname-a con 0x7f43dc0a2930 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:48:21.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f43dc132de0 0x7f43dc09a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 10 monclient(hunting): picked mon.noname-d con 0x7f43dc132de0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:48:21.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43dc1380e0 0x7f43dc09ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 10 monclient(hunting): picked mon.noname-b con 0x7f43dc1380e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:48:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc11d430 con 0x7f43dc0a2930 2024-09-15T01:48:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc09c830 con 0x7f43dc1380e0 2024-09-15T01:48:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc11e0f0 con 0x7f43dc132de0 2024-09-15T01:48:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e235d640 10 monclient(hunting): authenticate will time out at 1706.273440s 2024-09-15T01:48:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43dbfff640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc0a2930 msgr2=0x7f43dc132470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43dbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc0a2930 0x7f43dc132470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f43dc132de0 0x7f43dc09a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e08fa640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43dc1380e0 0x7f43dc09ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f43dc132de0 0x7f43dc09a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:39408/0 (socket says 172.21.15.178:39408) 2024-09-15T01:48:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 1 -- 172.21.15.178:0/2474434522 learned_addr learned my addr 172.21.15.178:0/2474434522 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:48:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 10 monclient(hunting): get_auth_request con 0x7f43dc132de0 auth_method 0 2024-09-15T01:48:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e08fa640 10 monclient(hunting): get_auth_request con 0x7f43dc1380e0 auth_method 0 2024-09-15T01:48:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e08fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e08fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e08fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e08fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e08fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e08fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 10 monclient(hunting): handle_auth_done global_id 25827 payload 824 2024-09-15T01:48:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 1 -- 172.21.15.178:0/2474434522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43dc1380e0 msgr2=0x7f43dc09ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43dc1380e0 0x7f43dc09ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 1 -- 172.21.15.178:0/2474434522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc0a2930 msgr2=0x7f43dc132470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43dc0a2930 0x7f43dc132470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:48:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 1 monclient: found mon.noname-d 2024-09-15T01:48:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:48:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 1 -- 172.21.15.178:0/2474434522 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0009450 con 0x7f43dc132de0 2024-09-15T01:48:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43e08fa640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43dc1380e0 0x7f43dc09ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:48:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.523+0000 7f43db7fe640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f43dc132de0 0x7f43dc09a630 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f43d0035040 tx=0x7f43d0032000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43d0031820 con 0x7f43dc132de0 2024-09-15T01:48:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43c3fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:48:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43c3fff640 10 monclient: dump: 2024-09-15T01:48:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.551 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.551 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.551 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.551 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.551 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.551 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.551 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.551 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.551 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.551 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.551 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.552 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.552 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.552 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43c3fff640 10 monclient: _finish_auth 0 2024-09-15T01:48:21.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:21.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d00319c0 con 0x7f43dc132de0 2024-09-15T01:48:21.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43e235d640 5 monclient: authenticate success, global_id 25827 2024-09-15T01:48:21.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43e235d640 10 monclient: _renew_subs 2024-09-15T01:48:21.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43e235d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:48:21.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43e235d640 1 -- 172.21.15.178:0/2474434522 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43dc09d630 con 0x7f43dc132de0 2024-09-15T01:48:21.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43c3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:21.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43d003a390 con 0x7f43dc132de0 2024-09-15T01:48:21.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43c3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:48:21.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43c3fff640 10 monclient: dump: 2024-09-15T01:48:21.553 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.553 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.553 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.553 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.553 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.553 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.553 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.553 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.554 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.554 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.554 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.554 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.554 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.554 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.554 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.554 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43da7fc640 4 set_mon_vals no callback set 2024-09-15T01:48:21.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43e235d640 10 monclient: _renew_subs 2024-09-15T01:48:21.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43e235d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:48:21.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43e235d640 1 -- 172.21.15.178:0/2474434522 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc09c070 con 0x7f43dc132de0 2024-09-15T01:48:21.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43e235d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:21.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43e235d640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc11c530 0x7f43dc09b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43e235d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.524+0000 7f43e235d640 1 -- 172.21.15.178:0/2474434522 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc04c4a0 con 0x7f43dc11c530 2024-09-15T01:48:21.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f43d0039040 con 0x7f43dc132de0 2024-09-15T01:48:21.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43c3fff640 1 --2- 172.21.15.178:0/2474434522 >> v2:172.21.15.204:6812/28210 conn(0x7f43c80397b0 0x7f43c803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(265..265 src has 186..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f43d006b240 con 0x7f43dc132de0 2024-09-15T01:48:21.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43dbfff640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc11c530 0x7f43dc09b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43dbfff640 10 monclient: get_auth_request con 0x7f43dc11c530 auth_method 0 2024-09-15T01:48:21.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43e08fa640 1 --2- 172.21.15.178:0/2474434522 >> v2:172.21.15.204:6812/28210 conn(0x7f43c80397b0 0x7f43c803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43e08fa640 10 monclient: get_auth_request con 0x7f43c80397b0 auth_method 0 2024-09-15T01:48:21.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43dbfff640 10 monclient(hunting): handle_auth_done global_id 64501 payload 824 2024-09-15T01:48:21.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43dbfff640 1 -- 172.21.15.178:0/2474434522 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43dc05ed70 con 0x7f43dc11c530 2024-09-15T01:48:21.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43dbfff640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc11c530 0x7f43dc09b0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00b460 tx=0x7f43cc0090f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43e08fa640 1 --2- 172.21.15.178:0/2474434522 >> v2:172.21.15.204:6812/28210 conn(0x7f43c80397b0 0x7f43c803bc60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f43dc09bd20 tx=0x7f43d4007860 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43cc003df0 con 0x7f43dc11c530 2024-09-15T01:48:21.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43c3fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:48:21.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.525+0000 7f43c3fff640 10 monclient: dump: 2024-09-15T01:48:21.557 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.557 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.557 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.557 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.557 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.557 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.557 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.557 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.558 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.558 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.558 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.558 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.558 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.558 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.558 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.558 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.526+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f43cc003970 con 0x7f43dc11c530 2024-09-15T01:48:21.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.526+0000 7f43c3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:21.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.526+0000 7f43c3fff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:21.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.526+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc11c530 msgr2=0x7f43dc09b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.526+0000 7f43c3fff640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc11c530 0x7f43dc09b0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00b460 tx=0x7f43cc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.526+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc11c530 msgr2=0x7f43dc09b0b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:21.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.526+0000 7f43c3fff640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc11c530 0x7f43dc09b0b0 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00b460 tx=0x7f43cc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:48:21.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.531+0000 7f43e235d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:48:21.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.531+0000 7f43e235d640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc096a30 0x7f43dc096e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:21.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.531+0000 7f43e235d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:21.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.531+0000 7f43e235d640 1 -- 172.21.15.178:0/2474434522 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc04bc80 con 0x7f43dc096a30 2024-09-15T01:48:21.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43dbfff640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc096a30 0x7f43dc096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:21.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43dbfff640 10 monclient: get_auth_request con 0x7f43dc096a30 auth_method 0 2024-09-15T01:48:21.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:21.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:21.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:21.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:21.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:21.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:21.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43dbfff640 10 monclient(hunting): handle_auth_done global_id 64510 payload 824 2024-09-15T01:48:21.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43dbfff640 1 -- 172.21.15.178:0/2474434522 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f43dc04b720 con 0x7f43dc096a30 2024-09-15T01:48:21.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43dbfff640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc096a30 0x7f43dc096e30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00b460 tx=0x7f43cc00f000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:21.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43cc011be0 con 0x7f43dc096a30 2024-09-15T01:48:21.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:21.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43c3fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:48:21.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.532+0000 7f43c3fff640 10 monclient: dump: 2024-09-15T01:48:21.561 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:48:21.561 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:21.561 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.561 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:21.561 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:21.561 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:48:21.561 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:21.561 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:21.561 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:21.561 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:21.561 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:21.561 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:21.562 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:21.562 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:21.562 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:21.562 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:48:21.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9120 (secure 0 0 0) 0x7f43cc00bd70 con 0x7f43dc096a30 2024-09-15T01:48:21.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43c3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:48:21.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43c3fff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:21.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc096a30 msgr2=0x7f43dc096e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43c3fff640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc096a30 0x7f43dc096e30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00b460 tx=0x7f43cc00f000 comp rx=0 tx=0).stop 2024-09-15T01:48:21.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43c3fff640 1 -- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc096a30 msgr2=0x7f43dc096e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:48:21.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43c3fff640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f43dc096a30 0x7f43dc096e30 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00b460 tx=0x7f43cc00f000 comp rx=0 tx=0).stop 2024-09-15T01:48:21.563 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:48:21.563 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:48:21.563 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:48:21.563 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:48:21.563 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 156, 2024-09-15T01:48:21.563 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:48:21.563 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:48:21.564 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:48:21.564 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:48:21.564 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:48:21.564 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:48:21.564 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:48:21.564 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:48:21.564 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.564 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 5, 2024-09-15T01:48:21.564 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:21.564 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:48:21.564 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:48:21.565 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:21.565 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:21.565 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:21.565 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:21.565 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:21.565 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:21.565 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:21.565 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:21.565 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:21.565 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.565 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:48:21.566 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:48:21.566 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:21.566 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:21.566 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:21.566 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:21.566 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:21.566 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:21.566 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:21.566 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:21.566 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:21.566 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.567 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:48:21.567 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:48:21.567 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:48:21.567 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:48:21.567 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:48:21.567 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:48:21.567 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:21.567 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:48:21.567 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:48:21.567 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:48:21.568 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:48:21.568 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:48:21.568 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:48:21.568 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:48:21.568 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:48:21.568 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:48:21.568 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:48:21.568 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:48:21.568 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:48:21.568 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:48:21.568 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:48:21.568 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:48:21.569 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:48:21.569 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:48:21.569 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:48:21.569 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:48:21.569 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.569 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:48:21.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.569 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:48:21.569 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.569 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:48:21.569 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:48:21.570 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.570 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:48:21.570 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.570 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.570 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:48:21.570 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.571 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.571 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.571 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:48:21.571 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:48:21.571 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.571 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.571 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.571 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:48:21.571 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:48:21.572 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.572 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.572 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.572 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:48:21.572 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.572 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.572 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:48:21.572 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.573 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.573 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.573 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:48:21.573 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:48:21.573 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.573 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.573 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.573 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:48:21.574 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:48:21.574 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.574 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.574 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.574 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:48:21.574 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.574 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.574 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:48:21.575 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.575 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.575 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.575 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:48:21.575 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:48:21.575 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.575 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.575 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.575 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:48:21.576 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:48:21.576 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.576 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:48:21.576 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.576 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.576 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:48:21.577 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.577 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.577 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.577 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:48:21.577 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:48:21.577 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.577 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.577 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.577 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:48:21.578 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:48:21.578 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.578 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.578 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.578 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:48:21.578 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.578 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.578 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:48:21.579 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.579 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.579 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.579 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:48:21.579 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:48:21.579 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.579 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.579 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.580 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:48:21.580 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:48:21.580 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.580 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.580 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.580 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:48:21.580 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.581 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:48:21.581 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.581 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.581 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.581 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:48:21.581 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:48:21.581 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.581 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.582 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.582 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:48:21.582 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:48:21.582 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.582 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.582 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.582 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:48:21.582 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.583 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.583 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:48:21.583 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.583 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.583 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.583 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:48:21.583 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:48:21.583 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.583 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.584 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.584 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:48:21.584 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:48:21.584 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.584 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.584 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.584 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:48:21.584 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.585 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.585 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:48:21.585 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.585 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.585 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.585 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:48:21.585 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:48:21.585 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.585 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.586 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.586 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:48:21.586 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:48:21.586 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:48:21.586 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:48:21.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.586 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:48:21.586 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:48:21.586 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.587 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.587 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:48:21.587 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:48:21.587 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:48:21.587 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.587 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.587 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:48:21.587 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:48:21.587 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:48:21.587 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:48:21.588 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:48:21.588 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.588 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.588 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:48:21.588 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:48:21.588 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.588 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:21.588 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:21.588 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:21.588 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.589 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:48:21.589 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:48:21.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:48:21.589 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:48:21.589 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:48:21.589 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:48:21.589 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:48:21.589 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:48:21.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:48:21.589 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:48:21.589 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:48:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43e235d640 1 -- 172.21.15.178:0/2474434522 >> v2:172.21.15.204:6812/28210 conn(0x7f43c80397b0 msgr2=0x7f43c803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43e235d640 1 --2- 172.21.15.178:0/2474434522 >> v2:172.21.15.204:6812/28210 conn(0x7f43c80397b0 0x7f43c803bc60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f43dc09bd20 tx=0x7f43d4007860 comp rx=0 tx=0).stop 2024-09-15T01:48:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43e235d640 10 monclient: shutdown 2024-09-15T01:48:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43e235d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43e235d640 1 -- 172.21.15.178:0/2474434522 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f43dc132de0 msgr2=0x7f43dc09a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43e235d640 1 --2- 172.21.15.178:0/2474434522 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f43dc132de0 0x7f43dc09a630 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f43d0035040 tx=0x7f43d0032000 comp rx=0 tx=0).stop 2024-09-15T01:48:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43e08fa640 1 -- 172.21.15.178:0/2474434522 reap_dead start 2024-09-15T01:48:21.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43e08fa640 1 -- 172.21.15.178:0/2474434522 reap_dead start 2024-09-15T01:48:21.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43e235d640 1 -- 172.21.15.178:0/2474434522 shutdown_connections 2024-09-15T01:48:21.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43e235d640 1 -- 172.21.15.178:0/2474434522 >> 172.21.15.178:0/2474434522 conn(0x7f43dc0a5ed0 msgr2=0x7f43dc0a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:21.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43e235d640 1 -- 172.21.15.178:0/2474434522 shutdown_connections 2024-09-15T01:48:21.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:48:21.533+0000 7f43e235d640 1 -- 172.21.15.178:0/2474434522 wait complete. 2024-09-15T01:48:21.592 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-15T01:48:21.592 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T01:48:54.072 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=110 2024-09-15T01:48:54.073 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 110 -ne 110 ]] 2024-09-15T01:48:54.073 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:48:54.073 INFO:tasks.workunit.client.0.smithi204.stderr:+ export CEPH_ARGS= 2024-09-15T01:48:54.073 INFO:tasks.workunit.client.0.smithi204.stderr:+ CEPH_ARGS= 2024-09-15T01:48:54.073 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph auth del client.bar 2024-09-15T01:48:54.146 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.145+0000 7f6175b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:48:54.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.145+0000 7f6175b9d640 10 monclient: build_initial_monmap 2024-09-15T01:48:54.147 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.145+0000 7f6175b9d640 10 monclient: monmap: 2024-09-15T01:48:54.147 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:48:54.147 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:54.147 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:48:54.145546+0000 2024-09-15T01:48:54.147 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:48:54.145546+0000 2024-09-15T01:48:54.147 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:54.147 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:54.147 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:54.147 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:54.148 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:54.148 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:54.148 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:54.148 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:54.148 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:54.148 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:54.148 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:54.148 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:54.148 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:54.148 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:54.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.145+0000 7f6175b9d640 1 Processor -- start 2024-09-15T01:48:54.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.145+0000 7f6175b9d640 1 -- start start 2024-09-15T01:48:54.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.145+0000 7f6175b9d640 10 monclient: init 2024-09-15T01:48:54.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:54.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 10 monclient: _add_conns ranks=[7,8,4,10,0,5,2,3,9,1,6] 2024-09-15T01:48:54.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f61700a2440 0x7f6170132ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 10 monclient(hunting): picked mon.noname-i con 0x7f61700a2440 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:48:54.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f617009f950 0x7f617009fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 10 monclient(hunting): picked mon.noname-e con 0x7f617009f950 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:48:54.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f61700a08e0 0x7f61700a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 10 monclient(hunting): picked mon.noname-c con 0x7f61700a08e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:48:54.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:54.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f617009d7b0 con 0x7f61700a08e0 2024-09-15T01:48:54.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:54.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f617009e140 con 0x7f61700a2440 2024-09-15T01:48:54.160 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:54.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f61700a2810 con 0x7f617009f950 2024-09-15T01:48:54.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:54.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f6175b9d640 10 monclient(hunting): authenticate will time out at 1749.788511s 2024-09-15T01:48:54.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f61700a2440 0x7f6170132ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:54.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f61700a2440 0x7f6170132ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49784/0 (socket says 172.21.15.204:49784) 2024-09-15T01:48:54.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 1 -- 172.21.15.204:0/2749252791 learned_addr learned my addr 172.21.15.204:0/2749252791 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:48:54.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 10 monclient(hunting): get_auth_request con 0x7f61700a2440 auth_method 0 2024-09-15T01:48:54.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:54.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:54.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:54.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:54.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616ffff640 1 --2- 172.21.15.204:0/2749252791 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f61700a08e0 0x7f61700a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616ffff640 10 monclient(hunting): get_auth_request con 0x7f61700a08e0 auth_method 0 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616effd640 1 --2- 172.21.15.204:0/2749252791 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f617009f950 0x7f617009fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616effd640 10 monclient(hunting): get_auth_request con 0x7f617009f950 auth_method 0 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:54.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 10 monclient(hunting): handle_auth_done global_id 64519 payload 274 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 1 -- 172.21.15.204:0/2749252791 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f617009f950 msgr2=0x7f617009fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 1 --2- 172.21.15.204:0/2749252791 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f617009f950 0x7f617009fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 1 -- 172.21.15.204:0/2749252791 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f61700a08e0 msgr2=0x7f61700a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 1 --2- 172.21.15.204:0/2749252791 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f61700a08e0 0x7f61700a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 1 monclient: found mon.noname-i 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.204:3303/0 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.146+0000 7f616f7fe640 1 -- 172.21.15.204:0/2749252791 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617004b720 con 0x7f61700a2440 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616effd640 1 --2- 172.21.15.204:0/2749252791 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f617009f950 0x7f617009fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:48:54.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616f7fe640 1 --2- 172.21.15.204:0/2749252791 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f61700a2440 0x7f6170132ae0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6160009940 tx=0x7f61600091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=83296c569c6be696 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:54.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616e7fc640 1 -- 172.21.15.204:0/2749252791 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f616002ac60 con 0x7f61700a2440 2024-09-15T01:48:54.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:54.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616e7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T01:48:54.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616e7fc640 10 monclient: dump: 2024-09-15T01:48:54.164 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:54.164 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:54.164 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.164 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.164 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:54.164 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:54.164 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:54.164 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616e7fc640 1 -- 172.21.15.204:0/2749252791 <== mon.7 v2:172.21.15.204:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616002ade0 con 0x7f61700a2440 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:54.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616e7fc640 1 -- 172.21.15.204:0/2749252791 <== mon.7 v2:172.21.15.204:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6160031ca0 con 0x7f61700a2440 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f6175b9d640 5 monclient: authenticate success, global_id 64519 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f6175b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f6175b9d640 4 set_mon_vals no callback set 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.147+0000 7f616e7fc640 10 monclient: dump: 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:54.166 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 10 monclient: shutdown 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 1 -- 172.21.15.204:0/2749252791 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f61700a2440 msgr2=0x7f6170132ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 1 --2- 172.21.15.204:0/2749252791 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f61700a2440 0x7f6170132ae0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6160009940 tx=0x7f61600091d0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 1 -- 172.21.15.204:0/2749252791 shutdown_connections 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 1 --2- 172.21.15.204:0/2749252791 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f61700a08e0 0x7f61700a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 1 --2- 172.21.15.204:0/2749252791 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f617009f950 0x7f617009fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 1 --2- 172.21.15.204:0/2749252791 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f61700a2440 0x7f6170132ae0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 1 -- 172.21.15.204:0/2749252791 >> 172.21.15.204:0/2749252791 conn(0x7f617006e860 msgr2=0x7f617006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 1 -- 172.21.15.204:0/2749252791 shutdown_connections 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 1 -- 172.21.15.204:0/2749252791 wait complete. 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 10 monclient: build_initial_monmap 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.148+0000 7f6175b9d640 10 monclient: monmap: 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:54.168 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 1 Processor -- start 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 1 -- start start 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 10 monclient: init 2024-09-15T01:48:54.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 10 monclient: _add_conns ranks=[1,2,6,8,3,5,0,4,7] 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f617009f950 0x7f6170098550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 10 monclient(hunting): picked mon.noname-f con 0x7f617009f950 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61700a08e0 0x7f6170098a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 10 monclient(hunting): picked mon.noname-b con 0x7f61700a08e0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f61700a2440 0x7f6170098fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 10 monclient(hunting): picked mon.noname-d con 0x7f61700a2440 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617009a420 con 0x7f617009f950 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617009d480 con 0x7f61700a08e0 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f617009ca40 con 0x7f61700a2440 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:54.170 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f6175b9d640 10 monclient(hunting): authenticate will time out at 1749.791951s 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f616f7fe640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f617009f950 msgr2=0x7f6170098550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.149+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f617009f950 0x7f6170098550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616effd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61700a08e0 0x7f6170098a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f61700a2440 0x7f6170098fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616effd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61700a08e0 0x7f6170098a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.204:58180/0 (socket says 172.21.15.204:58180) 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616effd640 1 -- 172.21.15.204:0/4067992824 learned_addr learned my addr 172.21.15.204:0/4067992824 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f61700a2440 0x7f6170098fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:40564/0 (socket says 172.21.15.204:40564) 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616effd640 10 monclient(hunting): get_auth_request con 0x7f61700a08e0 auth_method 0 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 10 monclient(hunting): get_auth_request con 0x7f61700a2440 auth_method 0 2024-09-15T01:48:54.171 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 10 monclient(hunting): handle_auth_done global_id 25836 payload 824 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 1 -- 172.21.15.204:0/4067992824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61700a08e0 msgr2=0x7f6170098a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 1 --2- 172.21.15.204:0/4067992824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61700a08e0 0x7f6170098a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 1 -- 172.21.15.204:0/4067992824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f617009f950 msgr2=0x7f6170098550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:54.172 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 1 --2- 172.21.15.204:0/4067992824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f617009f950 0x7f6170098550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:48:54.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 1 monclient: found mon.noname-d 2024-09-15T01:48:54.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:54.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:48:54.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 1 -- 172.21.15.204:0/4067992824 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6160009450 con 0x7f61700a2440 2024-09-15T01:48:54.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616effd640 1 --2- 172.21.15.204:0/4067992824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61700a08e0 0x7f6170098a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:48:54.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.150+0000 7f616ffff640 1 --2- 172.21.15.204:0/4067992824 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f61700a2440 0x7f6170098fd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6168009e50 tx=0x7f616800b040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:54.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f61577fe640 1 -- 172.21.15.204:0/4067992824 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f616800cd40 con 0x7f61700a2440 2024-09-15T01:48:54.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f61577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:54.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f61577fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:48:54.173 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f61577fe640 10 monclient: dump: 2024-09-15T01:48:54.174 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:54.174 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:54.174 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.174 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.174 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:54.174 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:54.174 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:54.174 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:54.174 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:54.174 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:54.174 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f61577fe640 10 monclient: _finish_auth 0 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f61577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f61577fe640 1 -- 172.21.15.204:0/4067992824 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616800f040 con 0x7f61700a2440 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f6175b9d640 5 monclient: authenticate success, global_id 25836 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f6175b9d640 10 monclient: _renew_subs 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f6175b9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f6175b9d640 1 -- 172.21.15.204:0/4067992824 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6170099ea0 con 0x7f61700a2440 2024-09-15T01:48:54.175 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f61577fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:54.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f61577fe640 1 -- 172.21.15.204:0/4067992824 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f616800cee0 con 0x7f61700a2440 2024-09-15T01:48:54.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f61577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:54.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f61577fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:48:54.176 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f61577fe640 10 monclient: dump: 2024-09-15T01:48:54.176 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:54.176 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:54.176 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.176 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.176 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:54.176 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:54.176 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:54.176 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f6175b9d640 10 monclient: _renew_subs 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f6175b9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f6175b9d640 1 -- 172.21.15.204:0/4067992824 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617009b650 con 0x7f61700a2440 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.151+0000 7f616dffb640 4 set_mon_vals no callback set 2024-09-15T01:48:54.177 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.152+0000 7f61577fe640 1 -- 172.21.15.204:0/4067992824 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6168014480 con 0x7f61700a2440 2024-09-15T01:48:54.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.152+0000 7f6175b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:54.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.152+0000 7f6175b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:54.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.152+0000 7f6175b9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:48:54.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.152+0000 7f6175b9d640 1 -- 172.21.15.204:0/4067992824 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6170063440 con 0x7f61700a2440 2024-09-15T01:48:54.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.152+0000 7f61577fe640 1 --2- 172.21.15.204:0/4067992824 >> v2:172.21.15.204:6812/28210 conn(0x7f615c0397b0 0x7f615c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.152+0000 7f61577fe640 1 -- 172.21.15.204:0/4067992824 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f616804c9c0 con 0x7f61700a2440 2024-09-15T01:48:54.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.152+0000 7f616f7fe640 1 --2- 172.21.15.204:0/4067992824 >> v2:172.21.15.204:6812/28210 conn(0x7f615c0397b0 0x7f615c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:54.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.152+0000 7f616f7fe640 10 monclient: get_auth_request con 0x7f615c0397b0 auth_method 0 2024-09-15T01:48:54.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.152+0000 7f616f7fe640 1 --2- 172.21.15.204:0/4067992824 >> v2:172.21.15.204:6812/28210 conn(0x7f615c0397b0 0x7f615c03bc60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6160034000 tx=0x7f6160032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:54.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.156+0000 7f61577fe640 1 -- 172.21.15.204:0/4067992824 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6168012d90 con 0x7f61700a2440 2024-09-15T01:48:54.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.156+0000 7f61577fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:54.178 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.156+0000 7f61577fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:54.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.371+0000 7f6175b9d640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bar"}] 2024-09-15T01:48:54.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.371+0000 7f6175b9d640 10 monclient: _send_command 2 [{"prefix": "auth del", "entity": "client.bar"}] 2024-09-15T01:48:54.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.371+0000 7f6175b9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:48:54.373 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.371+0000 7f6175b9d640 1 -- 172.21.15.204:0/4067992824 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "auth del", "entity": "client.bar"} v 0) v1 -- 0x7f617009de60 con 0x7f61700a2440 2024-09-15T01:48:54.376 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.375+0000 7f61577fe640 1 -- 172.21.15.204:0/4067992824 <== mon.6 v2:172.21.15.178:3303/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bar"}]=0 v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6168018090 con 0x7f61700a2440 2024-09-15T01:48:54.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.375+0000 7f61577fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth del", "entity": "client.bar"}] 2024-09-15T01:48:54.377 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.375+0000 7f61577fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:54.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.526+0000 7f6175b9d640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bar"}] 2024-09-15T01:48:54.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.526+0000 7f6175b9d640 10 monclient: _send_command 3 [{"prefix": "auth del", "entity": "client.bar"}] 2024-09-15T01:48:54.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.526+0000 7f6175b9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:48:54.528 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.526+0000 7f6175b9d640 1 -- 172.21.15.204:0/4067992824 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "auth del", "entity": "client.bar"} v 0) v1 -- 0x7f617004bc80 con 0x7f61700a2440 2024-09-15T01:48:54.529 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.527+0000 7f61577fe640 1 -- 172.21.15.204:0/4067992824 <== mon.6 v2:172.21.15.178:3303/0 8 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bar"}]=0 v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f61680127b0 con 0x7f61700a2440 2024-09-15T01:48:54.529 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.527+0000 7f61577fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth del", "entity": "client.bar"}] 2024-09-15T01:48:54.529 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.527+0000 7f61577fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:48:54.531 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 1 -- 172.21.15.204:0/4067992824 >> v2:172.21.15.204:6812/28210 conn(0x7f615c0397b0 msgr2=0x7f615c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 1 --2- 172.21.15.204:0/4067992824 >> v2:172.21.15.204:6812/28210 conn(0x7f615c0397b0 0x7f615c03bc60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6160034000 tx=0x7f6160032000 comp rx=0 tx=0).stop 2024-09-15T01:48:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 10 monclient: shutdown 2024-09-15T01:48:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 1 -- 172.21.15.204:0/4067992824 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f61700a2440 msgr2=0x7f6170098fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 1 --2- 172.21.15.204:0/4067992824 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f61700a2440 0x7f6170098fd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6168009e50 tx=0x7f616800b040 comp rx=0 tx=0).stop 2024-09-15T01:48:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 1 -- 172.21.15.204:0/4067992824 shutdown_connections 2024-09-15T01:48:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 1 --2- 172.21.15.204:0/4067992824 >> v2:172.21.15.204:6812/28210 conn(0x7f615c0397b0 0x7f615c03bc60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 1 --2- 172.21.15.204:0/4067992824 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f61700a2440 0x7f6170098fd0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 1 --2- 172.21.15.204:0/4067992824 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61700a08e0 0x7f6170098a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 1 --2- 172.21.15.204:0/4067992824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f617009f950 0x7f6170098550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.532 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 1 -- 172.21.15.204:0/4067992824 >> 172.21.15.204:0/4067992824 conn(0x7f617006e860 msgr2=0x7f617012ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 1 -- 172.21.15.204:0/4067992824 shutdown_connections 2024-09-15T01:48:54.533 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.529+0000 7f6175b9d640 1 -- 172.21.15.204:0/4067992824 wait complete. 2024-09-15T01:48:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status' 13 2024-09-15T01:48:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status' 2024-09-15T01:48:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=13 2024-09-15T01:48:54.540 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-09-15T01:48:54.540 INFO:tasks.workunit.client.0.smithi204.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-09-15T01:48:54.541 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-09-15T01:48:54.611 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=13 2024-09-15T01:48:54.611 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 13 -ne 13 ]] 2024-09-15T01:48:54.611 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:48:54.611 INFO:tasks.workunit.client.0.smithi204.stderr:+ rm /tmp/cephtest-mon-caps-madness.bazar.keyring /tmp/cephtest-mon-caps-madness.foo.keyring /tmp/cephtest-mon-caps-madness.bar.keyring 2024-09-15T01:48:54.612 INFO:tasks.workunit.client.0.smithi204.stderr:+ cat 2024-09-15T01:48:54.612 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph auth import -i - 2024-09-15T01:48:54.679 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.678+0000 7f4d9f457640 10 monclient: get_monmap_and_config 2024-09-15T01:48:54.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.678+0000 7f4d9f457640 10 monclient: build_initial_monmap 2024-09-15T01:48:54.680 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.678+0000 7f4d9f457640 10 monclient: monmap: 2024-09-15T01:48:54.680 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:48:54.680 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:54.680 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:48:54.678577+0000 2024-09-15T01:48:54.680 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:48:54.678577+0000 2024-09-15T01:48:54.680 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:54.680 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:54.680 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:54.680 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:54.680 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:54.681 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:54.681 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:54.681 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:54.681 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:54.681 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:54.681 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:54.681 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:54.681 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:54.681 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:54.681 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.678+0000 7f4d9f457640 1 Processor -- start 2024-09-15T01:48:54.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.678+0000 7f4d9f457640 1 -- start start 2024-09-15T01:48:54.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.678+0000 7f4d9f457640 10 monclient: init 2024-09-15T01:48:54.688 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:54.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 10 monclient: _add_conns ranks=[6,1,10,4,3,5,9,8,2,0,7] 2024-09-15T01:48:54.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d980a2440 0x7f4d98132ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 10 monclient(hunting): picked mon.noname-d con 0x7f4d980a2440 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:48:54.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4d9809f950 addr v1:172.21.15.204:6789/0 2024-09-15T01:48:54.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4d98137c60 0x7f4d9813a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 10 monclient(hunting): picked mon.noname-f con 0x7f4d98137c60 addr v2:172.21.15.204:3300/0 2024-09-15T01:48:54.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9804b720 con 0x7f4d9809f950 2024-09-15T01:48:54.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:54.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d9809d7b0 con 0x7f4d98137c60 2024-09-15T01:48:54.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:54.689 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9809e140 con 0x7f4d980a2440 2024-09-15T01:48:54.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:54.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9f457640 10 monclient(hunting): authenticate will time out at 1750.321427s 2024-09-15T01:48:54.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9c9f2640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7f4d9809f950 legacy=0x7f4d9809fd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:54.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9d9f4640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7f4d98137c60 msgr2=0x7f4d9813a040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:54.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9d9f4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4d98137c60 0x7f4d9813a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:54.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9d1f3640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d980a2440 0x7f4d98132ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:54.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9d1f3640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d980a2440 0x7f4d98132ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.204:40588/0 (socket says 172.21.15.204:40588) 2024-09-15T01:48:54.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9d1f3640 1 -- 172.21.15.204:0/3099427601 learned_addr learned my addr 172.21.15.204:0/3099427601 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:48:54.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9d1f3640 10 monclient(hunting): get_auth_request con 0x7f4d980a2440 auth_method 0 2024-09-15T01:48:54.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9d1f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:54.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9d1f3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:54.690 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9d1f3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9d1f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9d1f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.679+0000 7f4d9d1f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.680+0000 7f4d9d1f3640 10 monclient(hunting): handle_auth_done global_id 25854 payload 274 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.680+0000 7f4d9d1f3640 1 -- 172.21.15.204:0/3099427601 >> v2:172.21.15.204:3300/0 conn(0x7f4d98137c60 msgr2=0x7f4d9813a040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.680+0000 7f4d9d1f3640 1 --2- 172.21.15.204:0/3099427601 >> v2:172.21.15.204:3300/0 conn(0x7f4d98137c60 0x7f4d9813a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.680+0000 7f4d9d1f3640 1 -- 172.21.15.204:0/3099427601 >> v1:172.21.15.204:6789/0 conn(0x7f4d9809f950 legacy=0x7f4d9809fd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.680+0000 7f4d9d1f3640 10 monclient: _finish_hunting 0 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.680+0000 7f4d9d1f3640 1 monclient: found mon.noname-d 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.680+0000 7f4d9d1f3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.680+0000 7f4d9d1f3640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.680+0000 7f4d9d1f3640 1 -- 172.21.15.204:0/3099427601 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d980a2810 con 0x7f4d980a2440 2024-09-15T01:48:54.691 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.680+0000 7f4d9d1f3640 1 --2- 172.21.15.204:0/3099427601 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d980a2440 0x7f4d98132ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009910 tx=0x7f4d900091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=8cfcc2df18cacd9f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.680+0000 7f4d97fff640 1 -- 172.21.15.204:0/3099427601 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d9002acc0 con 0x7f4d980a2440 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d97fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d97fff640 10 monclient: dump: 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:54.692 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:54.693 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:54.693 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:54.693 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:54.693 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:54.693 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:54.693 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:54.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d97fff640 10 monclient: _finish_auth 0 2024-09-15T01:48:54.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d97fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:54.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d97fff640 1 -- 172.21.15.204:0/3099427601 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9002ae40 con 0x7f4d980a2440 2024-09-15T01:48:54.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 5 monclient: authenticate success, global_id 25854 2024-09-15T01:48:54.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:48:54.693 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d97fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:54.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:54.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d97fff640 1 -- 172.21.15.204:0/3099427601 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d90031ca0 con 0x7f4d980a2440 2024-09-15T01:48:54.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 4 set_mon_vals no callback set 2024-09-15T01:48:54.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:54.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d97fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:48:54.694 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d97fff640 10 monclient: dump: 2024-09-15T01:48:54.694 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:54.694 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:54.694 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.694 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.694 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 10 monclient: shutdown 2024-09-15T01:48:54.695 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:54.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 1 -- 172.21.15.204:0/3099427601 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d980a2440 msgr2=0x7f4d98132ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:54.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 1 --2- 172.21.15.204:0/3099427601 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d980a2440 0x7f4d98132ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009910 tx=0x7f4d900091d0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 1 -- 172.21.15.204:0/3099427601 shutdown_connections 2024-09-15T01:48:54.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 1 --2- 172.21.15.204:0/3099427601 >> v2:172.21.15.204:3300/0 conn(0x7f4d98137c60 0x7f4d9813a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 1 --2- 172.21.15.204:0/3099427601 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f4d980a2440 0x7f4d98132ae0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 1 -- 172.21.15.204:0/3099427601 >> 172.21.15.204:0/3099427601 conn(0x7f4d9806e860 msgr2=0x7f4d9806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:54.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 1 -- 172.21.15.204:0/3099427601 shutdown_connections 2024-09-15T01:48:54.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.681+0000 7f4d9f457640 1 -- 172.21.15.204:0/3099427601 wait complete. 2024-09-15T01:48:54.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.682+0000 7f4d9f457640 10 monclient: build_initial_monmap 2024-09-15T01:48:54.696 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.682+0000 7f4d9f457640 10 monclient: monmap: 2024-09-15T01:48:54.696 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:48:54.696 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:54.697 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:54.698 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:54.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.682+0000 7f4d9f457640 1 Processor -- start 2024-09-15T01:48:54.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.682+0000 7f4d9f457640 1 -- start start 2024-09-15T01:48:54.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.682+0000 7f4d9f457640 10 monclient: init 2024-09-15T01:48:54.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:54.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient: _add_conns ranks=[0,3,5,1,7,2,6,8,4] 2024-09-15T01:48:54.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d9809f950 0x7f4d9812f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient(hunting): picked mon.noname-a con 0x7f4d9809f950 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:48:54.698 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d980a2440 0x7f4d980988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient(hunting): picked mon.noname-g con 0x7f4d980a2440 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:54.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d98137c60 0x7f4d98098df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient(hunting): picked mon.noname-h con 0x7f4d98137c60 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:48:54.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:54.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9809d480 con 0x7f4d9809f950 2024-09-15T01:48:54.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:54.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9809c3c0 con 0x7f4d980a2440 2024-09-15T01:48:54.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:54.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9809c050 con 0x7f4d98137c60 2024-09-15T01:48:54.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:54.699 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient(hunting): authenticate will time out at 1750.325127s 2024-09-15T01:48:54.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d1f3640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d9809f950 msgr2=0x7f4d9812f780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:54.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d98137c60 0x7f4d98098df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:54.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9c9f2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d980a2440 0x7f4d980988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:54.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d1f3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d9809f950 0x7f4d9812f780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:54.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d98137c60 0x7f4d98098df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:55372/0 (socket says 172.21.15.204:55372) 2024-09-15T01:48:54.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 1 -- 172.21.15.204:0/110300355 learned_addr learned my addr 172.21.15.204:0/110300355 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:48:54.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9c9f2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d980a2440 0x7f4d980988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:38976/0 (socket says 172.21.15.204:38976) 2024-09-15T01:48:54.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 10 monclient(hunting): get_auth_request con 0x7f4d98137c60 auth_method 0 2024-09-15T01:48:54.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:54.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:54.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:54.700 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9c9f2640 10 monclient(hunting): get_auth_request con 0x7f4d980a2440 auth_method 0 2024-09-15T01:48:54.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9c9f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:54.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9c9f2640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:54.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9c9f2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:54.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:54.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:54.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:54.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9c9f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:54.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9c9f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:54.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9c9f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:54.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 10 monclient(hunting): handle_auth_done global_id 7763 payload 824 2024-09-15T01:48:54.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 1 -- 172.21.15.204:0/110300355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d980a2440 msgr2=0x7f4d980988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:54.701 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 1 --2- 172.21.15.204:0/110300355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d980a2440 0x7f4d980988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 1 -- 172.21.15.204:0/110300355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d9809f950 msgr2=0x7f4d9812f780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 1 --2- 172.21.15.204:0/110300355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d9809f950 0x7f4d9812f780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 10 monclient: _finish_hunting 0 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 1 monclient: found mon.noname-h 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 1 -- 172.21.15.204:0/110300355 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d90009450 con 0x7f4d98137c60 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9c9f2640 1 --2- 172.21.15.204:0/110300355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d980a2440 0x7f4d980988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9d9f4640 1 --2- 172.21.15.204:0/110300355 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d98137c60 0x7f4d98098df0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800bee0 tx=0x7f4d880096e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d957fa640 1 -- 172.21.15.204:0/110300355 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d8800cd00 con 0x7f4d98137c60 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d957fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d957fa640 10 monclient: dump: 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:54.702 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:54.703 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d957fa640 10 monclient: _finish_auth 0 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d957fa640 1 -- 172.21.15.204:0/110300355 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8800ce80 con 0x7f4d98137c60 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 5 monclient: authenticate success, global_id 7763 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient: _renew_subs 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 1 -- 172.21.15.204:0/110300355 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9809ba60 con 0x7f4d98137c60 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d957fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d957fa640 1 -- 172.21.15.204:0/110300355 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d88006a50 con 0x7f4d98137c60 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d957fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:48:54.704 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d957fa640 10 monclient: dump: 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:54.705 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:54.706 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:54.706 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:54.706 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:54.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d977fe640 4 set_mon_vals no callback set 2024-09-15T01:48:54.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient: _renew_subs 2024-09-15T01:48:54.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:48:54.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.683+0000 7f4d9f457640 1 -- 172.21.15.204:0/110300355 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9809a240 con 0x7f4d98137c60 2024-09-15T01:48:54.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.684+0000 7f4d9f457640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:54.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.684+0000 7f4d957fa640 1 -- 172.21.15.204:0/110300355 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f4d88012020 con 0x7f4d98137c60 2024-09-15T01:48:54.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.684+0000 7f4d9f457640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:54.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.684+0000 7f4d9f457640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:48:54.706 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.684+0000 7f4d9f457640 1 -- 172.21.15.204:0/110300355 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d98063440 con 0x7f4d98137c60 2024-09-15T01:48:54.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.685+0000 7f4d957fa640 1 --2- 172.21.15.204:0/110300355 >> v2:172.21.15.204:6812/28210 conn(0x7f4d840397b0 0x7f4d8403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:54.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.685+0000 7f4d957fa640 1 -- 172.21.15.204:0/110300355 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f4d8804bea0 con 0x7f4d98137c60 2024-09-15T01:48:54.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.685+0000 7f4d9d1f3640 1 --2- 172.21.15.204:0/110300355 >> v2:172.21.15.204:6812/28210 conn(0x7f4d840397b0 0x7f4d8403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:54.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.685+0000 7f4d9d1f3640 10 monclient: get_auth_request con 0x7f4d840397b0 auth_method 0 2024-09-15T01:48:54.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.685+0000 7f4d9d1f3640 1 --2- 172.21.15.204:0/110300355 >> v2:172.21.15.204:6812/28210 conn(0x7f4d840397b0 0x7f4d8403bc60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4d90034000 tx=0x7f4d90032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:54.707 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.689+0000 7f4d957fa640 1 -- 172.21.15.204:0/110300355 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4d88013ac0 con 0x7f4d98137c60 2024-09-15T01:48:54.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.689+0000 7f4d957fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:54.708 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.689+0000 7f4d957fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:54.907 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.905+0000 7f4d9f457640 10 start_mon_command cmd=[{"prefix": "auth import"}] 2024-09-15T01:48:54.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.905+0000 7f4d9f457640 10 monclient: _send_command 2 [{"prefix": "auth import"}] 2024-09-15T01:48:54.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.905+0000 7f4d9f457640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:48:54.908 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.905+0000 7f4d9f457640 1 -- 172.21.15.204:0/110300355 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "auth import"} v 0) v1 -- 0x7f4d9809de60 con 0x7f4d98137c60 2024-09-15T01:48:54.908 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:48:54.906+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 auth entities have invalid capabilities (AUTH_BAD_CAPS) 2024-09-15T01:48:54.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.909+0000 7f4d957fa640 1 -- 172.21.15.204:0/110300355 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "auth import"}]=0 v21) v1 ==== 59+0+0 (secure 0 0 0) 0x7f4d880132e0 con 0x7f4d98137c60 2024-09-15T01:48:54.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.909+0000 7f4d957fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth import"}] 2024-09-15T01:48:54.910 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:54.909+0000 7f4d957fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:55.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.060+0000 7f4d9f457640 10 start_mon_command cmd=[{"prefix": "auth import"}] 2024-09-15T01:48:55.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.060+0000 7f4d9f457640 10 monclient: _send_command 3 [{"prefix": "auth import"}] 2024-09-15T01:48:55.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.060+0000 7f4d9f457640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:48:55.062 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.060+0000 7f4d9f457640 1 -- 172.21.15.204:0/110300355 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "auth import"} v 0) v1 -- 0x7f4d9804bc80 con 0x7f4d98137c60 2024-09-15T01:48:55.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.064+0000 7f4d957fa640 1 -- 172.21.15.204:0/110300355 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "auth import"}]=0 v22) v1 ==== 59+0+0 (secure 0 0 0) 0x7f4d8804b020 con 0x7f4d98137c60 2024-09-15T01:48:55.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.064+0000 7f4d957fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth import"}] 2024-09-15T01:48:55.065 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.064+0000 7f4d957fa640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:48:55.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 1 -- 172.21.15.204:0/110300355 >> v2:172.21.15.204:6812/28210 conn(0x7f4d840397b0 msgr2=0x7f4d8403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:55.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 1 --2- 172.21.15.204:0/110300355 >> v2:172.21.15.204:6812/28210 conn(0x7f4d840397b0 0x7f4d8403bc60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4d90034000 tx=0x7f4d90032000 comp rx=0 tx=0).stop 2024-09-15T01:48:55.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 10 monclient: shutdown 2024-09-15T01:48:55.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:55.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 1 -- 172.21.15.204:0/110300355 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d98137c60 msgr2=0x7f4d98098df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:55.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 1 --2- 172.21.15.204:0/110300355 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d98137c60 0x7f4d98098df0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800bee0 tx=0x7f4d880096e0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 1 -- 172.21.15.204:0/110300355 shutdown_connections 2024-09-15T01:48:55.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 1 --2- 172.21.15.204:0/110300355 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f4d98137c60 0x7f4d98098df0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 1 --2- 172.21.15.204:0/110300355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d980a2440 0x7f4d980988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.068 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 1 --2- 172.21.15.204:0/110300355 >> v2:172.21.15.204:6812/28210 conn(0x7f4d840397b0 0x7f4d8403bc60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 1 --2- 172.21.15.204:0/110300355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d9809f950 0x7f4d9812f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 1 -- 172.21.15.204:0/110300355 >> 172.21.15.204:0/110300355 conn(0x7f4d9806e860 msgr2=0x7f4d980a3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:55.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 1 -- 172.21.15.204:0/110300355 shutdown_connections 2024-09-15T01:48:55.069 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.066+0000 7f4d9f457640 1 -- 172.21.15.204:0/110300355 wait complete. 2024-09-15T01:48:55.077 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph health 2024-09-15T01:48:55.077 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep AUTH_BAD_CAP 2024-09-15T01:48:55.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.141+0000 7f568048b640 10 monclient: get_monmap_and_config 2024-09-15T01:48:55.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.141+0000 7f568048b640 10 monclient: build_initial_monmap 2024-09-15T01:48:55.143 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.141+0000 7f568048b640 10 monclient: monmap: 2024-09-15T01:48:55.143 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:48:55.143 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:55.143 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:48:55.141833+0000 2024-09-15T01:48:55.143 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:48:55.141833+0000 2024-09-15T01:48:55.143 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:55.144 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.141+0000 7f568048b640 1 Processor -- start 2024-09-15T01:48:55.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.141+0000 7f568048b640 1 -- start start 2024-09-15T01:48:55.145 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.141+0000 7f568048b640 10 monclient: init 2024-09-15T01:48:55.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:55.151 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 10 monclient: _add_conns ranks=[9,5,4,1,3,10,0,7,6,2,8] 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f56780a2440 0x7f5678132ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 10 monclient(hunting): picked mon.noname-a con 0x7f56780a2440 addr v2:172.21.15.178:3300/0 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f567809f950 0x7f567809fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 10 monclient(hunting): picked mon.noname-h con 0x7f567809f950 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56780a08e0 0x7f56780a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 10 monclient(hunting): picked mon.noname-c con 0x7f56780a08e0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f567809d7b0 con 0x7f56780a2440 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f567809e140 con 0x7f56780a08e0 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56780a2810 con 0x7f567809f950 2024-09-15T01:48:55.152 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f568048b640 10 monclient(hunting): authenticate will time out at 1750.784690s 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567da26640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f567809f950 0x7f567809fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567da26640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f567809f950 0x7f567809fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:55376/0 (socket says 172.21.15.204:55376) 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567da26640 1 -- 172.21.15.204:0/1581817 learned_addr learned my addr 172.21.15.204:0/1581817 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567da26640 10 monclient(hunting): get_auth_request con 0x7f567809f950 auth_method 0 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567da26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567da26640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567da26640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567e227640 1 -- 172.21.15.204:0/1581817 >> v2:172.21.15.178:3300/0 conn(0x7f56780a2440 msgr2=0x7f5678132ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567e227640 1 --2- 172.21.15.204:0/1581817 >> v2:172.21.15.178:3300/0 conn(0x7f56780a2440 0x7f5678132ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567da26640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567da26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:55.153 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567da26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.142+0000 7f567ea28640 1 --2- 172.21.15.204:0/1581817 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56780a08e0 0x7f56780a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567ea28640 10 monclient(hunting): get_auth_request con 0x7f56780a08e0 auth_method 0 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567ea28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567ea28640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567ea28640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567da26640 10 monclient(hunting): handle_auth_done global_id 7772 payload 274 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567da26640 1 -- 172.21.15.204:0/1581817 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56780a08e0 msgr2=0x7f56780a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567da26640 1 --2- 172.21.15.204:0/1581817 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56780a08e0 0x7f56780a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567da26640 1 -- 172.21.15.204:0/1581817 >> v2:172.21.15.178:3300/0 conn(0x7f56780a2440 msgr2=0x7f5678132ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567da26640 1 --2- 172.21.15.204:0/1581817 >> v2:172.21.15.178:3300/0 conn(0x7f56780a2440 0x7f5678132ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567da26640 10 monclient: _finish_hunting 0 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567da26640 1 monclient: found mon.noname-h 2024-09-15T01:48:55.154 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567da26640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567da26640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567da26640 1 -- 172.21.15.204:0/1581817 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567804b720 con 0x7f567809f950 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567da26640 1 --2- 172.21.15.204:0/1581817 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f567809f950 0x7f567809fd20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f566c009910 tx=0x7f566c0091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d2d881033c9d7e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567d225640 1 -- 172.21.15.204:0/1581817 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f566c02acb0 con 0x7f567809f950 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567d225640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567d225640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567d225640 10 monclient: dump: 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:55.155 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567d225640 10 monclient: _finish_auth 0 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567d225640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f568048b640 5 monclient: authenticate success, global_id 7772 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f568048b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:48:55.156 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567d225640 1 -- 172.21.15.204:0/1581817 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c02ae30 con 0x7f567809f950 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567d225640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f568048b640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f568048b640 4 set_mon_vals no callback set 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567d225640 1 -- 172.21.15.204:0/1581817 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f566c031ca0 con 0x7f567809f950 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.143+0000 7f567d225640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f567d225640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f567d225640 10 monclient: dump: 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:55.157 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f568048b640 10 monclient: shutdown 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f568048b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f568048b640 1 -- 172.21.15.204:0/1581817 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f567809f950 msgr2=0x7f567809fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:55.158 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f568048b640 1 --2- 172.21.15.204:0/1581817 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f567809f950 0x7f567809fd20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f566c009910 tx=0x7f566c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f568048b640 1 -- 172.21.15.204:0/1581817 shutdown_connections 2024-09-15T01:48:55.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f568048b640 1 --2- 172.21.15.204:0/1581817 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f56780a08e0 0x7f56780a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f568048b640 1 --2- 172.21.15.204:0/1581817 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f567809f950 0x7f567809fd20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f568048b640 1 --2- 172.21.15.204:0/1581817 >> v2:172.21.15.178:3300/0 conn(0x7f56780a2440 0x7f5678132ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f568048b640 1 -- 172.21.15.204:0/1581817 >> 172.21.15.204:0/1581817 conn(0x7f567806e860 msgr2=0x7f567806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:55.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f568048b640 1 -- 172.21.15.204:0/1581817 shutdown_connections 2024-09-15T01:48:55.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.144+0000 7f568048b640 1 -- 172.21.15.204:0/1581817 wait complete. 2024-09-15T01:48:55.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient: build_initial_monmap 2024-09-15T01:48:55.159 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient: monmap: 2024-09-15T01:48:55.159 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:48:55.159 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:55.159 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:55.160 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 1 Processor -- start 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 1 -- start start 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient: init 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient: _add_conns ranks=[3,1,0,5,8,7,2,4,6] 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f567809f950 0x7f567811c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient(hunting): picked mon.noname-g con 0x7f567809f950 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56780a08e0 0x7f567811ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient(hunting): picked mon.noname-f con 0x7f56780a08e0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56780a2440 0x7f567811d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient(hunting): picked mon.noname-a con 0x7f56780a2440 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:55.161 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567811dc80 con 0x7f56780a2440 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567809d480 con 0x7f56780a08e0 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567809ca20 con 0x7f567809f950 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f568048b640 10 monclient(hunting): authenticate will time out at 1750.787821s 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f567da26640 1 -- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56780a08e0 msgr2=0x7f567811ccf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f567da26640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56780a08e0 0x7f567811ccf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f567e227640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f567809f950 0x7f567811c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f567e227640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f567809f950 0x7f567811c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:38980/0 (socket says 172.21.15.204:38980) 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f567e227640 1 -- 172.21.15.204:0/3791305232 learned_addr learned my addr 172.21.15.204:0/3791305232 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f567ea28640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56780a2440 msgr2=0x7f567811d230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f567e227640 10 monclient(hunting): get_auth_request con 0x7f567809f950 auth_method 0 2024-09-15T01:48:55.162 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f567e227640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f567e227640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f567e227640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.145+0000 7f567ea28640 1 --2- 172.21.15.204:0/3791305232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56780a2440 0x7f567811d230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 10 monclient(hunting): handle_auth_done global_id 64569 payload 824 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 1 -- 172.21.15.204:0/3791305232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56780a08e0 msgr2=0x7f567811ccf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 1 --2- 172.21.15.204:0/3791305232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56780a08e0 0x7f567811ccf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 1 -- 172.21.15.204:0/3791305232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56780a2440 msgr2=0x7f567811d230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 1 --2- 172.21.15.204:0/3791305232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56780a2440 0x7f567811d230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 10 monclient: _finish_hunting 0 2024-09-15T01:48:55.163 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 1 monclient: found mon.noname-g 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 1 -- 172.21.15.204:0/3791305232 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c009450 con 0x7f567809f950 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567e227640 1 --2- 172.21.15.204:0/3791305232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f567809f950 0x7f567811c7b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f567000bbb0 tx=0x7f56700097f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f56767fc640 1 -- 172.21.15.204:0/3791305232 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5670006150 con 0x7f567809f950 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f56767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f56767fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f56767fc640 10 monclient: dump: 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.164 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f56767fc640 10 monclient: _finish_auth 0 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f56767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:55.165 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f56767fc640 1 -- 172.21.15.204:0/3791305232 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56700062f0 con 0x7f567809f950 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f568048b640 5 monclient: authenticate success, global_id 64569 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f568048b640 10 monclient: _renew_subs 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f568048b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f568048b640 1 -- 172.21.15.204:0/3791305232 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567811e7a0 con 0x7f567809f950 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f56767fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f56767fc640 1 -- 172.21.15.204:0/3791305232 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5670006920 con 0x7f567809f950 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f56767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f56767fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f56767fc640 10 monclient: dump: 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.166 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f567ca24640 4 set_mon_vals no callback set 2024-09-15T01:48:55.167 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f568048b640 10 monclient: _renew_subs 2024-09-15T01:48:55.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f568048b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:55.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.146+0000 7f568048b640 1 -- 172.21.15.204:0/3791305232 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5678120990 con 0x7f567809f950 2024-09-15T01:48:55.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.147+0000 7f56767fc640 1 -- 172.21.15.204:0/3791305232 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f5670012020 con 0x7f567809f950 2024-09-15T01:48:55.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.147+0000 7f568048b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:55.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.147+0000 7f568048b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:55.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.147+0000 7f568048b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:55.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.147+0000 7f56767fc640 1 --2- 172.21.15.204:0/3791305232 >> v2:172.21.15.204:6812/28210 conn(0x7f56640397e0 0x7f566403bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.147+0000 7f568048b640 1 -- 172.21.15.204:0/3791305232 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5678064a60 con 0x7f567809f950 2024-09-15T01:48:55.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.147+0000 7f56767fc640 1 -- 172.21.15.204:0/3791305232 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f567004bbf0 con 0x7f567809f950 2024-09-15T01:48:55.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.147+0000 7f567da26640 1 --2- 172.21.15.204:0/3791305232 >> v2:172.21.15.204:6812/28210 conn(0x7f56640397e0 0x7f566403bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:55.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.147+0000 7f567da26640 10 monclient: get_auth_request con 0x7f56640397e0 auth_method 0 2024-09-15T01:48:55.168 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.147+0000 7f567da26640 1 --2- 172.21.15.204:0/3791305232 >> v2:172.21.15.204:6812/28210 conn(0x7f56640397e0 0x7f566403bc90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f566c034000 tx=0x7f566c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:55.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.151+0000 7f56767fc640 1 -- 172.21.15.204:0/3791305232 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f567001d900 con 0x7f567809f950 2024-09-15T01:48:55.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.151+0000 7f56767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:55.169 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.151+0000 7f56767fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:55.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.336+0000 7f568048b640 10 start_mon_command cmd=[{"prefix": "health"}] 2024-09-15T01:48:55.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.336+0000 7f568048b640 10 monclient: _send_command 2 [{"prefix": "health"}] 2024-09-15T01:48:55.338 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.336+0000 7f568048b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:55.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.336+0000 7f568048b640 1 -- 172.21.15.204:0/3791305232 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "health"} v 0) v1 -- 0x7f5678063440 con 0x7f567809f950 2024-09-15T01:48:55.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.337+0000 7f56767fc640 1 -- 172.21.15.204:0/3791305232 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "health"}]=0 v0) v1 ==== 54+0+136 (secure 0 0 0) 0x7f5670014340 con 0x7f567809f950 2024-09-15T01:48:55.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.337+0000 7f56767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health"}] 2024-09-15T01:48:55.339 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.337+0000 7f56767fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:55.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.458+0000 7f568048b640 10 start_mon_command cmd=[{"prefix": "health"}] 2024-09-15T01:48:55.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.458+0000 7f568048b640 10 monclient: _send_command 3 [{"prefix": "health"}] 2024-09-15T01:48:55.460 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.458+0000 7f568048b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:55.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.458+0000 7f568048b640 1 -- 172.21.15.204:0/3791305232 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "health"} v 0) v1 -- 0x7f567809de60 con 0x7f567809f950 2024-09-15T01:48:55.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.458+0000 7f56767fc640 1 -- 172.21.15.204:0/3791305232 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "health"}]=0 v0) v1 ==== 54+0+136 (secure 0 0 0) 0x7f567001d300 con 0x7f567809f950 2024-09-15T01:48:55.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.459+0000 7f56767fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "health"}] 2024-09-15T01:48:55.461 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.459+0000 7f56767fc640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:48:55.462 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 1 -- 172.21.15.204:0/3791305232 >> v2:172.21.15.204:6812/28210 conn(0x7f56640397e0 msgr2=0x7f566403bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:55.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 1 --2- 172.21.15.204:0/3791305232 >> v2:172.21.15.204:6812/28210 conn(0x7f56640397e0 0x7f566403bc90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f566c034000 tx=0x7f566c032000 comp rx=0 tx=0).stop 2024-09-15T01:48:55.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 10 monclient: shutdown 2024-09-15T01:48:55.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:55.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 1 -- 172.21.15.204:0/3791305232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f567809f950 msgr2=0x7f567811c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:55.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 1 --2- 172.21.15.204:0/3791305232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f567809f950 0x7f567811c7b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f567000bbb0 tx=0x7f56700097f0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 1 -- 172.21.15.204:0/3791305232 shutdown_connections 2024-09-15T01:48:55.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 1 --2- 172.21.15.204:0/3791305232 >> v2:172.21.15.204:6812/28210 conn(0x7f56640397e0 0x7f566403bc90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 1 --2- 172.21.15.204:0/3791305232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56780a2440 0x7f567811d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 1 --2- 172.21.15.204:0/3791305232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56780a08e0 0x7f567811ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 1 --2- 172.21.15.204:0/3791305232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f567809f950 0x7f567811c7b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.463 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 1 -- 172.21.15.204:0/3791305232 >> 172.21.15.204:0/3791305232 conn(0x7f567806e860 msgr2=0x7f56780969e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:55.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 1 -- 172.21.15.204:0/3791305232 shutdown_connections 2024-09-15T01:48:55.464 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.461+0000 7f568048b640 1 -- 172.21.15.204:0/3791305232 wait complete. 2024-09-15T01:48:55.472 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph health detail 2024-09-15T01:48:55.472 INFO:tasks.workunit.client.0.smithi204.stderr:+ grep client.bad 2024-09-15T01:48:55.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.538+0000 7f348db1d640 10 monclient: get_monmap_and_config 2024-09-15T01:48:55.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.538+0000 7f348db1d640 10 monclient: build_initial_monmap 2024-09-15T01:48:55.540 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.538+0000 7f348db1d640 10 monclient: monmap: 2024-09-15T01:48:55.540 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:48:55.541 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:55.541 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:48:55.539371+0000 2024-09-15T01:48:55.541 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:48:55.539371+0000 2024-09-15T01:48:55.541 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:55.541 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:55.541 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:55.541 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:55.541 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:55.541 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:55.541 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:55.541 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:55.541 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:55.542 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:55.542 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:55.542 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:55.542 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:55.542 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:55.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.539+0000 7f348db1d640 1 Processor -- start 2024-09-15T01:48:55.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.539+0000 7f348db1d640 1 -- start start 2024-09-15T01:48:55.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.539+0000 7f348db1d640 10 monclient: init 2024-09-15T01:48:55.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:55.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 10 monclient: _add_conns ranks=[0,10,5,4,8,3,7,2,1,9,6] 2024-09-15T01:48:55.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f34880a2440 addr v1:172.21.15.178:6789/0 2024-09-15T01:48:55.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f348809f950 0x7f348809fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 10 monclient(hunting): picked mon.noname-f con 0x7f348809f950 addr v2:172.21.15.204:3300/0 2024-09-15T01:48:55.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3488137c60 0x7f348813a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.550 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 10 monclient(hunting): picked mon.noname-h con 0x7f3488137c60 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:48:55.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348804b720 con 0x7f34880a2440 2024-09-15T01:48:55.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:55.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348809d7b0 con 0x7f348809f950 2024-09-15T01:48:55.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:55.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f348809e140 con 0x7f3488137c60 2024-09-15T01:48:55.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:55.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f348db1d640 10 monclient(hunting): authenticate will time out at 1751.182157s 2024-09-15T01:48:55.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3486ffd640 1 -- >> v2:172.21.15.204:3300/0 conn(0x7f348809f950 msgr2=0x7f348809fd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.204:3300/0 2024-09-15T01:48:55.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3486ffd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f348809f950 0x7f348809fd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:55.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f34877fe640 1 -- >> v1:172.21.15.178:6789/0 conn(0x7f34880a2440 legacy=0x7f3488132ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.178:6789/0 2024-09-15T01:48:55.551 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3488137c60 0x7f348813a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:55.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3488137c60 0x7f348813a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:55390/0 (socket says 172.21.15.204:55390) 2024-09-15T01:48:55.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 1 -- 172.21.15.204:0/2000199449 learned_addr learned my addr 172.21.15.204:0/2000199449 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:48:55.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 10 monclient(hunting): get_auth_request con 0x7f3488137c60 auth_method 0 2024-09-15T01:48:55.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:55.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:55.552 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:55.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:55.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:55.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:55.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 10 monclient(hunting): handle_auth_done global_id 7781 payload 274 2024-09-15T01:48:55.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 1 -- 172.21.15.204:0/2000199449 >> v2:172.21.15.204:3300/0 conn(0x7f348809f950 msgr2=0x7f348809fd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:55.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 1 --2- 172.21.15.204:0/2000199449 >> v2:172.21.15.204:3300/0 conn(0x7f348809f950 0x7f348809fd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 1 -- 172.21.15.204:0/2000199449 >> v1:172.21.15.178:6789/0 conn(0x7f34880a2440 legacy=0x7f3488132ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:55.553 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 10 monclient: _finish_hunting 0 2024-09-15T01:48:55.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 1 monclient: found mon.noname-h 2024-09-15T01:48:55.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:55.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:48:55.554 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.540+0000 7f3487fff640 1 -- 172.21.15.204:0/2000199449 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34881331f0 con 0x7f3488137c60 2024-09-15T01:48:55.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f3487fff640 1 --2- 172.21.15.204:0/2000199449 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3488137c60 0x7f348813a040 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3480009910 tx=0x7f34800091d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=77c8e193eb81b3e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:55.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f34867fc640 1 -- 172.21.15.204:0/2000199449 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f348002acc0 con 0x7f3488137c60 2024-09-15T01:48:55.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f34867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:55.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f34867fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:48:55.555 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f34867fc640 10 monclient: dump: 2024-09-15T01:48:55.555 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:55.555 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:55.555 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:55.556 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:55.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f34867fc640 10 monclient: _finish_auth 0 2024-09-15T01:48:55.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f34867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:55.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f34867fc640 1 -- 172.21.15.204:0/2000199449 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348002ae40 con 0x7f3488137c60 2024-09-15T01:48:55.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f348db1d640 5 monclient: authenticate success, global_id 7781 2024-09-15T01:48:55.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f348db1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:48:55.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f34867fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:55.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f348db1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:55.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f34867fc640 1 -- 172.21.15.204:0/2000199449 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3480031ca0 con 0x7f3488137c60 2024-09-15T01:48:55.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f34867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:55.557 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f348db1d640 4 set_mon_vals no callback set 2024-09-15T01:48:55.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f34867fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:48:55.558 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f34867fc640 10 monclient: dump: 2024-09-15T01:48:55.558 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:55.558 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:55.558 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.558 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.558 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:55.558 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:55.558 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:55.558 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:55.558 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:55.558 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f348db1d640 10 monclient: shutdown 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f348db1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f348db1d640 1 -- 172.21.15.204:0/2000199449 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3488137c60 msgr2=0x7f348813a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f348db1d640 1 --2- 172.21.15.204:0/2000199449 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3488137c60 0x7f348813a040 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3480009910 tx=0x7f34800091d0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.541+0000 7f348db1d640 1 -- 172.21.15.204:0/2000199449 shutdown_connections 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.542+0000 7f348db1d640 1 --2- 172.21.15.204:0/2000199449 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f3488137c60 0x7f348813a040 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.542+0000 7f348db1d640 1 --2- 172.21.15.204:0/2000199449 >> v2:172.21.15.204:3300/0 conn(0x7f348809f950 0x7f348809fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.559 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.542+0000 7f348db1d640 1 -- 172.21.15.204:0/2000199449 >> 172.21.15.204:0/2000199449 conn(0x7f348806e860 msgr2=0x7f348806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:55.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.542+0000 7f348db1d640 1 -- 172.21.15.204:0/2000199449 shutdown_connections 2024-09-15T01:48:55.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.542+0000 7f348db1d640 1 -- 172.21.15.204:0/2000199449 wait complete. 2024-09-15T01:48:55.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.542+0000 7f348db1d640 10 monclient: build_initial_monmap 2024-09-15T01:48:55.560 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.542+0000 7f348db1d640 10 monclient: monmap: 2024-09-15T01:48:55.560 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:48:55.560 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:55.560 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:48:55.560 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:55.561 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.542+0000 7f348db1d640 1 Processor -- start 2024-09-15T01:48:55.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.542+0000 7f348db1d640 1 -- start start 2024-09-15T01:48:55.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.542+0000 7f348db1d640 10 monclient: init 2024-09-15T01:48:55.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:55.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 10 monclient: _add_conns ranks=[0,6,3,8,5,7,2,1,4] 2024-09-15T01:48:55.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f348809f950 0x7f34881339b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 10 monclient(hunting): picked mon.noname-a con 0x7f348809f950 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:48:55.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34880a2440 0x7f3488133ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 10 monclient(hunting): picked mon.noname-d con 0x7f34880a2440 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:48:55.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3488137c60 0x7f3488134430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.562 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 10 monclient(hunting): picked mon.noname-g con 0x7f3488137c60 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:55.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:55.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3488136f40 con 0x7f348809f950 2024-09-15T01:48:55.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:55.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348809d480 con 0x7f3488137c60 2024-09-15T01:48:55.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:55.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3488136c60 con 0x7f34880a2440 2024-09-15T01:48:55.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:55.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f348db1d640 10 monclient(hunting): authenticate will time out at 1751.185418s 2024-09-15T01:48:55.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f34877fe640 1 -- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f348809f950 msgr2=0x7f34881339b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:3300/0 2024-09-15T01:48:55.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f34877fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f348809f950 0x7f34881339b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:48:55.563 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3488137c60 0x7f3488134430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:55.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3488137c60 0x7f3488134430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:38990/0 (socket says 172.21.15.204:38990) 2024-09-15T01:48:55.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 1 -- 172.21.15.204:0/1399168474 learned_addr learned my addr 172.21.15.204:0/1399168474 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:48:55.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 10 monclient(hunting): get_auth_request con 0x7f3488137c60 auth_method 0 2024-09-15T01:48:55.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:55.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:55.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:55.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:55.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:55.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:55.564 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3486ffd640 1 --2- 172.21.15.204:0/1399168474 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34880a2440 0x7f3488133ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:55.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 10 monclient(hunting): handle_auth_done global_id 64578 payload 824 2024-09-15T01:48:55.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 1 -- 172.21.15.204:0/1399168474 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34880a2440 msgr2=0x7f3488133ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:55.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 1 --2- 172.21.15.204:0/1399168474 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34880a2440 0x7f3488133ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 1 -- 172.21.15.204:0/1399168474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f348809f950 msgr2=0x7f34881339b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:48:55.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 1 --2- 172.21.15.204:0/1399168474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f348809f950 0x7f34881339b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 10 monclient: _finish_hunting 0 2024-09-15T01:48:55.565 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 1 monclient: found mon.noname-g 2024-09-15T01:48:55.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:55.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:48:55.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 1 -- 172.21.15.204:0/1399168474 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480009450 con 0x7f3488137c60 2024-09-15T01:48:55.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3486ffd640 10 monclient: get_auth_request con 0x7f34880a2440 auth_method 0 2024-09-15T01:48:55.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3486ffd640 1 --2- 172.21.15.204:0/1399168474 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34880a2440 0x7f3488133ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:48:55.566 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.543+0000 7f3487fff640 1 --2- 172.21.15.204:0/1399168474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3488137c60 0x7f3488134430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3480009210 tx=0x7f3480032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:55.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 1 -- 172.21.15.204:0/1399168474 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f348003b870 con 0x7f3488137c60 2024-09-15T01:48:55.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:55.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:48:55.567 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 10 monclient: dump: 2024-09-15T01:48:55.567 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:55.567 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:55.567 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.567 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.567 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:55.567 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:55.567 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:55.567 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:55.568 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:55.568 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:55.568 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:55.568 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:55.568 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:55.568 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:55.568 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:55.568 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:55.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:48:55.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:55.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f348db1d640 5 monclient: authenticate success, global_id 64578 2024-09-15T01:48:55.568 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 1 -- 172.21.15.204:0/1399168474 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348003b9f0 con 0x7f3488137c60 2024-09-15T01:48:55.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f348db1d640 10 monclient: _renew_subs 2024-09-15T01:48:55.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f348db1d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:55.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f348db1d640 1 -- 172.21.15.204:0/1399168474 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3488136980 con 0x7f3488137c60 2024-09-15T01:48:55.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:55.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 1 -- 172.21.15.204:0/1399168474 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f34800313a0 con 0x7f3488137c60 2024-09-15T01:48:55.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:55.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:48:55.569 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 10 monclient: dump: 2024-09-15T01:48:55.569 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:55.569 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:55.569 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.570 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:55.570 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:55.570 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:55.570 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:55.570 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:55.570 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:55.570 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:55.570 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:55.570 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:55.570 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:55.570 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:55.570 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:55.571 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:55.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f348db1d640 10 monclient: _renew_subs 2024-09-15T01:48:55.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f348db1d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:55.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f348db1d640 1 -- 172.21.15.204:0/1399168474 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3488136560 con 0x7f3488137c60 2024-09-15T01:48:55.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f3485ffb640 4 set_mon_vals no callback set 2024-09-15T01:48:55.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.544+0000 7f346f7fe640 1 -- 172.21.15.204:0/1399168474 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f3480039040 con 0x7f3488137c60 2024-09-15T01:48:55.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.545+0000 7f348db1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:55.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.545+0000 7f348db1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:55.571 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.545+0000 7f348db1d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:55.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.545+0000 7f348db1d640 1 -- 172.21.15.204:0/1399168474 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3488063440 con 0x7f3488137c60 2024-09-15T01:48:55.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.545+0000 7f346f7fe640 1 --2- 172.21.15.204:0/1399168474 >> v2:172.21.15.204:6812/28210 conn(0x7f34740396e0 0x7f347403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:55.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.545+0000 7f346f7fe640 1 -- 172.21.15.204:0/1399168474 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f348006b630 con 0x7f3488137c60 2024-09-15T01:48:55.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.545+0000 7f34877fe640 1 --2- 172.21.15.204:0/1399168474 >> v2:172.21.15.204:6812/28210 conn(0x7f34740396e0 0x7f347403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:55.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.545+0000 7f34877fe640 10 monclient: get_auth_request con 0x7f34740396e0 auth_method 0 2024-09-15T01:48:55.572 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.545+0000 7f34877fe640 1 --2- 172.21.15.204:0/1399168474 >> v2:172.21.15.204:6812/28210 conn(0x7f34740396e0 0x7f347403bb90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3478003c70 tx=0x7f3478009310 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:55.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.549+0000 7f346f7fe640 1 -- 172.21.15.204:0/1399168474 <== mon.3 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f3480044340 con 0x7f3488137c60 2024-09-15T01:48:55.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.549+0000 7f346f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:55.573 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.549+0000 7f346f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.765+0000 7f348db1d640 10 start_mon_command cmd=[{"prefix": "health", "detail": "detail"}] 2024-09-15T01:48:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.765+0000 7f348db1d640 10 monclient: _send_command 2 [{"prefix": "health", "detail": "detail"}] 2024-09-15T01:48:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.765+0000 7f348db1d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:55.767 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.765+0000 7f348db1d640 1 -- 172.21.15.204:0/1399168474 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f348804bc80 con 0x7f3488137c60 2024-09-15T01:48:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.767+0000 7f346f7fe640 1 -- 172.21.15.204:0/1399168474 <== mon.3 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1326 (secure 0 0 0) 0x7f34800453a0 con 0x7f3488137c60 2024-09-15T01:48:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.767+0000 7f346f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "detail": "detail"}] 2024-09-15T01:48:55.768 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.767+0000 7f346f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:55.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.918+0000 7f348db1d640 10 start_mon_command cmd=[{"prefix": "health", "detail": "detail"}] 2024-09-15T01:48:55.920 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.918+0000 7f348db1d640 10 monclient: _send_command 3 [{"prefix": "health", "detail": "detail"}] 2024-09-15T01:48:55.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.918+0000 7f348db1d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:48:55.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.918+0000 7f348db1d640 1 -- 172.21.15.204:0/1399168474 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f348804bc80 con 0x7f3488137c60 2024-09-15T01:48:55.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.919+0000 7f346f7fe640 1 -- 172.21.15.204:0/1399168474 <== mon.3 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1326 (secure 0 0 0) 0x7f34800459f0 con 0x7f3488137c60 2024-09-15T01:48:55.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.919+0000 7f346f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "health", "detail": "detail"}] 2024-09-15T01:48:55.921 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.919+0000 7f346f7fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:48:55.924 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.920+0000 7f348db1d640 1 -- 172.21.15.204:0/1399168474 >> v2:172.21.15.204:6812/28210 conn(0x7f34740396e0 msgr2=0x7f347403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:55.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.920+0000 7f348db1d640 1 --2- 172.21.15.204:0/1399168474 >> v2:172.21.15.204:6812/28210 conn(0x7f34740396e0 0x7f347403bb90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3478003c70 tx=0x7f3478009310 comp rx=0 tx=0).stop 2024-09-15T01:48:55.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.920+0000 7f348db1d640 10 monclient: shutdown 2024-09-15T01:48:55.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.920+0000 7f348db1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:55.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.920+0000 7f348db1d640 1 -- 172.21.15.204:0/1399168474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3488137c60 msgr2=0x7f3488134430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:55.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.920+0000 7f348db1d640 1 --2- 172.21.15.204:0/1399168474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3488137c60 0x7f3488134430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3480009210 tx=0x7f3480032000 comp rx=0 tx=0).stop 2024-09-15T01:48:55.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.920+0000 7f348db1d640 1 -- 172.21.15.204:0/1399168474 shutdown_connections 2024-09-15T01:48:55.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.920+0000 7f348db1d640 1 --2- 172.21.15.204:0/1399168474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3488137c60 0x7f3488134430 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.920+0000 7f348db1d640 1 --2- 172.21.15.204:0/1399168474 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f34880a2440 0x7f3488133ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.920+0000 7f348db1d640 1 --2- 172.21.15.204:0/1399168474 >> v2:172.21.15.204:6812/28210 conn(0x7f34740396e0 0x7f347403bb90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.920+0000 7f348db1d640 1 --2- 172.21.15.204:0/1399168474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f348809f950 0x7f34881339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:55.925 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.920+0000 7f348db1d640 1 -- 172.21.15.204:0/1399168474 >> 172.21.15.204:0/1399168474 conn(0x7f348806e860 msgr2=0x7f348812dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:55.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.921+0000 7f348db1d640 1 -- 172.21.15.204:0/1399168474 shutdown_connections 2024-09-15T01:48:55.926 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.921+0000 7f348db1d640 1 -- 172.21.15.204:0/1399168474 wait complete. 2024-09-15T01:48:55.931 INFO:tasks.workunit.client.0.smithi204.stdout: client.bad mds capability parse failed, stopped at 'also does not parse' of 'also does not parse' 2024-09-15T01:48:55.931 INFO:tasks.workunit.client.0.smithi204.stdout: client.bad mon capability parse failed, stopped at 'this is wrong' of 'this is wrong' 2024-09-15T01:48:55.931 INFO:tasks.workunit.client.0.smithi204.stdout: client.bad osd capability parse failed, stopped at 'does not parse' of 'does not parse' 2024-09-15T01:48:55.931 INFO:tasks.workunit.client.0.smithi204.stdout: client.bad mds capability parse failed, stopped at 'also does not parse' of 'also does not parse' 2024-09-15T01:48:55.931 INFO:tasks.workunit.client.0.smithi204.stdout: client.bad mon capability parse failed, stopped at 'this is wrong' of 'this is wrong' 2024-09-15T01:48:55.931 INFO:tasks.workunit.client.0.smithi204.stdout: client.bad osd capability parse failed, stopped at 'does not parse' of 'does not parse' 2024-09-15T01:48:55.931 INFO:tasks.workunit.client.0.smithi204.stderr:+ ceph auth rm client.bad 2024-09-15T01:48:56.000 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.999+0000 7efcb222b640 10 monclient: get_monmap_and_config 2024-09-15T01:48:56.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.999+0000 7efcb222b640 10 monclient: build_initial_monmap 2024-09-15T01:48:56.001 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:55.999+0000 7efcb222b640 10 monclient: monmap: 2024-09-15T01:48:56.001 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:48:56.001 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:56.001 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:48:55.999594+0000 2024-09-15T01:48:56.001 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:48:55.999594+0000 2024-09-15T01:48:56.001 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:56.001 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:56.002 INFO:tasks.workunit.client.0.smithi204.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:48:56.002 INFO:tasks.workunit.client.0.smithi204.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:48:56.002 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:56.002 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:56.002 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:56.002 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:56.002 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:56.002 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:56.002 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:56.002 INFO:tasks.workunit.client.0.smithi204.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:48:56.002 INFO:tasks.workunit.client.0.smithi204.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:48:56.002 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:56.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 1 Processor -- start 2024-09-15T01:48:56.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 1 -- start start 2024-09-15T01:48:56.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 10 monclient: init 2024-09-15T01:48:56.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:56.010 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 10 monclient: _add_conns ranks=[7,3,1,5,9,2,8,10,4,6,0] 2024-09-15T01:48:56.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efcac0a2440 0x7efcac132ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:56.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 10 monclient(hunting): picked mon.noname-i con 0x7efcac0a2440 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:48:56.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcac09f950 0x7efcac09fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:56.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 10 monclient(hunting): picked mon.noname-g con 0x7efcac09f950 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:48:56.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7efcac0a08e0 addr v1:172.21.15.204:6789/0 2024-09-15T01:48:56.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac04b720 con 0x7efcac0a08e0 2024-09-15T01:48:56.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:56.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac09d7b0 con 0x7efcac09f950 2024-09-15T01:48:56.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:56.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efcac09e140 con 0x7efcac0a2440 2024-09-15T01:48:56.011 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:56.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcb222b640 10 monclient(hunting): authenticate will time out at 1751.642595s 2024-09-15T01:48:56.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcabfff640 1 -- >> v1:172.21.15.204:6789/0 conn(0x7efcac0a08e0 legacy=0x7efcac0a0cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.204:6789/0 2024-09-15T01:48:56.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcab7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efcac0a2440 0x7efcac132ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:56.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcac09f950 0x7efcac09fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:56.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcac09f950 0x7efcac09fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:39006/0 (socket says 172.21.15.204:39006) 2024-09-15T01:48:56.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcab7fe640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efcac0a2440 0x7efcac132ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.204:49788/0 (socket says 172.21.15.204:49788) 2024-09-15T01:48:56.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 1 -- 172.21.15.204:0/700086293 learned_addr learned my addr 172.21.15.204:0/700086293 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:48:56.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 10 monclient(hunting): get_auth_request con 0x7efcac09f950 auth_method 0 2024-09-15T01:48:56.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:56.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:56.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:56.012 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcab7fe640 10 monclient(hunting): get_auth_request con 0x7efcac0a2440 auth_method 0 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efcab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 10 monclient(hunting): handle_auth_done global_id 64587 payload 274 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 1 -- 172.21.15.204:0/700086293 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efcac0a2440 msgr2=0x7efcac132ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 1 --2- 172.21.15.204:0/700086293 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efcac0a2440 0x7efcac132ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:56.013 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 1 -- 172.21.15.204:0/700086293 >> v1:172.21.15.204:6789/0 conn(0x7efcac0a08e0 legacy=0x7efcac0a0cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 1 monclient: found mon.noname-g 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.000+0000 7efca3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efca3fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efca3fff640 1 -- 172.21.15.204:0/700086293 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac0a2810 con 0x7efcac09f950 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcab7fe640 1 --2- 172.21.15.204:0/700086293 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efcac0a2440 0x7efcac132ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efca3fff640 1 --2- 172.21.15.204:0/700086293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcac09f950 0x7efcac09fd20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc94009940 tx=0x7efc940091d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=7325d484a4b0fa28 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcaaffd640 1 -- 172.21.15.204:0/700086293 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc9402ac70 con 0x7efcac09f950 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcaaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcaaffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcaaffd640 10 monclient: dump: 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:56.014 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:56.015 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:56.015 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:56.015 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:56.015 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:56.015 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:56.015 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:56.015 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:56.015 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:56.015 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:56.015 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:56.015 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:56.015 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcaaffd640 10 monclient: _finish_auth 0 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcaaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcb222b640 5 monclient: authenticate success, global_id 64587 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcb222b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcaaffd640 1 -- 172.21.15.204:0/700086293 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9402adf0 con 0x7efcac09f950 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcaaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcb222b640 10 monclient: get_monmap_and_config success 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcb222b640 4 set_mon_vals no callback set 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcaaffd640 1 -- 172.21.15.204:0/700086293 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc94031ca0 con 0x7efcac09f950 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcaaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcaaffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:48:56.016 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.001+0000 7efcaaffd640 10 monclient: dump: 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:56.017 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.002+0000 7efcb222b640 10 monclient: shutdown 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.002+0000 7efcb222b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.002+0000 7efcb222b640 1 -- 172.21.15.204:0/700086293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcac09f950 msgr2=0x7efcac09fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.002+0000 7efcb222b640 1 --2- 172.21.15.204:0/700086293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcac09f950 0x7efcac09fd20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc94009940 tx=0x7efc940091d0 comp rx=0 tx=0).stop 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.002+0000 7efcb222b640 1 -- 172.21.15.204:0/700086293 shutdown_connections 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.002+0000 7efcb222b640 1 --2- 172.21.15.204:0/700086293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcac09f950 0x7efcac09fd20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.002+0000 7efcb222b640 1 --2- 172.21.15.204:0/700086293 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7efcac0a2440 0x7efcac132ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.002+0000 7efcb222b640 1 -- 172.21.15.204:0/700086293 >> 172.21.15.204:0/700086293 conn(0x7efcac06e860 msgr2=0x7efcac06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.002+0000 7efcb222b640 1 -- 172.21.15.204:0/700086293 shutdown_connections 2024-09-15T01:48:56.018 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.002+0000 7efcb222b640 1 -- 172.21.15.204:0/700086293 wait complete. 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.002+0000 7efcb222b640 10 monclient: build_initial_monmap 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.002+0000 7efcb222b640 10 monclient: monmap: 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 0 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 0.000000 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:created 0.000000 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 0 (unknown) 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 1 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:48:56.019 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 1 Processor -- start 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 1 -- start start 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 10 monclient: init 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 10 monclient: _reopen_session rank -1 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 10 monclient: _add_conns ranks=[4,8,5,1,6,3,7,0,2] 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efcac09f950 0x7efcac131180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 10 monclient(hunting): picked mon.noname-c con 0x7efcac09f950 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efcac0a08e0 0x7efcac09b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:56.020 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 10 monclient(hunting): picked mon.noname-e con 0x7efcac0a08e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efcac0a2440 0x7efcac09b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 10 monclient(hunting): picked mon.noname-h con 0x7efcac0a2440 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcac09d480 con 0x7efcac09f950 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcac11e580 con 0x7efcac0a2440 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7efcac1370c0 con 0x7efcac0a08e0 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 10 monclient(hunting): _renew_subs 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcb222b640 10 monclient(hunting): authenticate will time out at 1751.645801s 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcabfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efcac0a2440 0x7efcac09b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcabfff640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efcac0a2440 0x7efcac09b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.204:55406/0 (socket says 172.21.15.204:55406) 2024-09-15T01:48:56.021 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcabfff640 1 -- 172.21.15.204:0/2499074501 learned_addr learned my addr 172.21.15.204:0/2499074501 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcabfff640 10 monclient(hunting): get_auth_request con 0x7efcac0a2440 auth_method 0 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcabfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.003+0000 7efcabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcab7fe640 1 --2- 172.21.15.204:0/2499074501 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efcac09f950 0x7efcac131180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca3fff640 1 --2- 172.21.15.204:0/2499074501 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efcac0a08e0 0x7efcac09b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 10 monclient(hunting): handle_auth_done global_id 7790 payload 824 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 1 -- 172.21.15.204:0/2499074501 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efcac0a08e0 msgr2=0x7efcac09b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 1 --2- 172.21.15.204:0/2499074501 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efcac0a08e0 0x7efcac09b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:56.022 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 1 -- 172.21.15.204:0/2499074501 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efcac09f950 msgr2=0x7efcac131180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:56.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 1 --2- 172.21.15.204:0/2499074501 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efcac09f950 0x7efcac131180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:56.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 10 monclient: _finish_hunting 0 2024-09-15T01:48:56.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 1 monclient: found mon.noname-h 2024-09-15T01:48:56.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:48:56.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:48:56.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 1 -- 172.21.15.204:0/2499074501 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9c009390 con 0x7efcac0a2440 2024-09-15T01:48:56.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcab7fe640 10 monclient: get_auth_request con 0x7efcac09f950 auth_method 0 2024-09-15T01:48:56.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcab7fe640 1 --2- 172.21.15.204:0/2499074501 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efcac09f950 0x7efcac131180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:48:56.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca3fff640 10 monclient: get_auth_request con 0x7efcac0a08e0 auth_method 0 2024-09-15T01:48:56.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca3fff640 1 --2- 172.21.15.204:0/2499074501 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efcac0a08e0 0x7efcac09b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:48:56.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcabfff640 1 --2- 172.21.15.204:0/2499074501 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efcac0a2440 0x7efcac09b6f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7efca400bee0 tx=0x7efca40096e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:56.023 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca37fe640 1 -- 172.21.15.204:0/2499074501 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efca400cd00 con 0x7efcac0a2440 2024-09-15T01:48:56.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:56.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca37fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:48:56.024 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca37fe640 10 monclient: dump: 2024-09-15T01:48:56.024 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:56.024 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:56.024 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:56.024 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:56.024 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:56.024 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:56.024 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:56.024 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:56.024 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:56.025 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:56.025 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:56.025 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:56.025 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:56.025 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:56.025 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:56.025 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:56.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca37fe640 10 monclient: _finish_auth 0 2024-09-15T01:48:56.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:48:56.025 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca37fe640 1 -- 172.21.15.204:0/2499074501 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca400ce80 con 0x7efcac0a2440 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcb222b640 5 monclient: authenticate success, global_id 7790 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcb222b640 10 monclient: _renew_subs 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcb222b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcb222b640 1 -- 172.21.15.204:0/2499074501 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc94009450 con 0x7efcac0a2440 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca37fe640 1 -- 172.21.15.204:0/2499074501 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efca4006a50 con 0x7efcac0a2440 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca37fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efca37fe640 10 monclient: dump: 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:epoch 1 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:56.026 INFO:tasks.workunit.client.0.smithi204.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr:min_mon_release 17 (quincy) 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr:election_strategy: 3 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr: 2024-09-15T01:48:56.027 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcaa7fc640 4 set_mon_vals no callback set 2024-09-15T01:48:56.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcb222b640 10 monclient: _renew_subs 2024-09-15T01:48:56.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcb222b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:48:56.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.004+0000 7efcb222b640 1 -- 172.21.15.204:0/2499074501 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac11d340 con 0x7efcac0a2440 2024-09-15T01:48:56.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.005+0000 7efca37fe640 1 -- 172.21.15.204:0/2499074501 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7efca4012020 con 0x7efcac0a2440 2024-09-15T01:48:56.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.005+0000 7efcb222b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:56.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.005+0000 7efcb222b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:56.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.005+0000 7efcb222b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:48:56.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.005+0000 7efcb222b640 1 -- 172.21.15.204:0/2499074501 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcac11e740 con 0x7efcac0a2440 2024-09-15T01:48:56.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.005+0000 7efca37fe640 1 --2- 172.21.15.204:0/2499074501 >> v2:172.21.15.204:6812/28210 conn(0x7efc8c0397b0 0x7efc8c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:48:56.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.005+0000 7efca37fe640 1 -- 172.21.15.204:0/2499074501 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7efca404bf40 con 0x7efcac0a2440 2024-09-15T01:48:56.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.005+0000 7efcab7fe640 1 --2- 172.21.15.204:0/2499074501 >> v2:172.21.15.204:6812/28210 conn(0x7efc8c0397b0 0x7efc8c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:48:56.028 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.005+0000 7efcab7fe640 10 monclient: get_auth_request con 0x7efc8c0397b0 auth_method 0 2024-09-15T01:48:56.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.006+0000 7efcab7fe640 1 --2- 172.21.15.204:0/2499074501 >> v2:172.21.15.204:6812/28210 conn(0x7efc8c0397b0 0x7efc8c03bc60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009360 tx=0x7efc9c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:48:56.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.010+0000 7efca37fe640 1 -- 172.21.15.204:0/2499074501 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7efca40138c0 con 0x7efcac0a2440 2024-09-15T01:48:56.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.010+0000 7efca37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:48:56.029 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.010+0000 7efca37fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:48:56.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.224+0000 7efcb222b640 10 start_mon_command cmd=[{"prefix": "auth rm", "entity": "client.bad"}] 2024-09-15T01:48:56.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.224+0000 7efcb222b640 10 monclient: _send_command 2 [{"prefix": "auth rm", "entity": "client.bad"}] 2024-09-15T01:48:56.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.224+0000 7efcb222b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:48:56.226 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.224+0000 7efcb222b640 1 -- 172.21.15.204:0/2499074501 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "auth rm", "entity": "client.bad"} v 0) v1 -- 0x7efcac063440 con 0x7efcac0a2440 2024-09-15T01:48:56.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.227+0000 7efca37fe640 1 -- 172.21.15.204:0/2499074501 <== mon.5 v2:172.21.15.204:3302/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.bad"}]=0 v23) v1 ==== 79+0+0 (secure 0 0 0) 0x7efca40132e0 con 0x7efcac0a2440 2024-09-15T01:48:56.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.227+0000 7efca37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth rm", "entity": "client.bad"}] 2024-09-15T01:48:56.229 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.227+0000 7efca37fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:48:56.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.378+0000 7efcb222b640 10 start_mon_command cmd=[{"prefix": "auth rm", "entity": "client.bad"}] 2024-09-15T01:48:56.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.378+0000 7efcb222b640 10 monclient: _send_command 3 [{"prefix": "auth rm", "entity": "client.bad"}] 2024-09-15T01:48:56.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.378+0000 7efcb222b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:48:56.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.378+0000 7efcb222b640 1 -- 172.21.15.204:0/2499074501 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "auth rm", "entity": "client.bad"} v 0) v1 -- 0x7efcac09de60 con 0x7efcac0a2440 2024-09-15T01:48:56.380 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.378+0000 7efca37fe640 1 -- 172.21.15.204:0/2499074501 <== mon.5 v2:172.21.15.204:3302/0 8 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.bad"}]=0 v23) v1 ==== 79+0+0 (secure 0 0 0) 0x7efca40138c0 con 0x7efcac0a2440 2024-09-15T01:48:56.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.378+0000 7efca37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth rm", "entity": "client.bad"}] 2024-09-15T01:48:56.381 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.378+0000 7efca37fe640 10 monclient: _finish_command 3 = system:0 2024-09-15T01:48:56.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.380+0000 7efcb222b640 1 -- 172.21.15.204:0/2499074501 >> v2:172.21.15.204:6812/28210 conn(0x7efc8c0397b0 msgr2=0x7efc8c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:56.382 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.380+0000 7efcb222b640 1 --2- 172.21.15.204:0/2499074501 >> v2:172.21.15.204:6812/28210 conn(0x7efc8c0397b0 0x7efc8c03bc60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009360 tx=0x7efc9c008040 comp rx=0 tx=0).stop 2024-09-15T01:48:56.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.380+0000 7efcb222b640 10 monclient: shutdown 2024-09-15T01:48:56.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.380+0000 7efcb222b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:48:56.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.380+0000 7efcb222b640 1 -- 172.21.15.204:0/2499074501 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efcac0a2440 msgr2=0x7efcac09b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:48:56.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.380+0000 7efcb222b640 1 --2- 172.21.15.204:0/2499074501 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efcac0a2440 0x7efcac09b6f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7efca400bee0 tx=0x7efca40096e0 comp rx=0 tx=0).stop 2024-09-15T01:48:56.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.380+0000 7efcb222b640 1 -- 172.21.15.204:0/2499074501 shutdown_connections 2024-09-15T01:48:56.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.380+0000 7efcb222b640 1 --2- 172.21.15.204:0/2499074501 >> v2:172.21.15.204:6812/28210 conn(0x7efc8c0397b0 0x7efc8c03bc60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:56.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.380+0000 7efcb222b640 1 --2- 172.21.15.204:0/2499074501 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7efcac0a2440 0x7efcac09b6f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:56.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.380+0000 7efcb222b640 1 --2- 172.21.15.204:0/2499074501 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7efcac0a08e0 0x7efcac09b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:56.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.380+0000 7efcb222b640 1 --2- 172.21.15.204:0/2499074501 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7efcac09f950 0x7efcac131180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:48:56.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.380+0000 7efcb222b640 1 -- 172.21.15.204:0/2499074501 >> 172.21.15.204:0/2499074501 conn(0x7efcac06e860 msgr2=0x7efcac0a5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:48:56.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.381+0000 7efcb222b640 1 -- 172.21.15.204:0/2499074501 shutdown_connections 2024-09-15T01:48:56.383 INFO:tasks.workunit.client.0.smithi204.stderr:2024-09-15T01:48:56.381+0000 7efcb222b640 1 -- 172.21.15.204:0/2499074501 wait complete. 2024-09-15T01:48:56.391 INFO:tasks.workunit.client.0.smithi204.stderr:+ expect 'ceph auth health | grep AUTH_BAD_CAP' 1 2024-09-15T01:48:56.391 INFO:tasks.workunit.client.0.smithi204.stderr:+ cmd='ceph auth health | grep AUTH_BAD_CAP' 2024-09-15T01:48:56.391 INFO:tasks.workunit.client.0.smithi204.stdout:ceph auth health | grep AUTH_BAD_CAP 2024-09-15T01:48:56.392 INFO:tasks.workunit.client.0.smithi204.stderr:+ expected_ret=1 2024-09-15T01:48:56.392 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo ceph auth health '|' grep AUTH_BAD_CAP 2024-09-15T01:48:56.392 INFO:tasks.workunit.client.0.smithi204.stderr:+ eval ceph auth health '|' grep AUTH_BAD_CAP 2024-09-15T01:48:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:+ ret=1 2024-09-15T01:48:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:+ [[ 1 -ne 1 ]] 2024-09-15T01:48:56.692 INFO:tasks.workunit.client.0.smithi204.stderr:+ return 0 2024-09-15T01:48:56.692 INFO:tasks.workunit.client.0.smithi204.stdout:OK 2024-09-15T01:48:56.693 INFO:tasks.workunit.client.0.smithi204.stderr:+ echo OK 2024-09-15T01:48:56.694 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T01:48:56.694 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T01:48:56.718 INFO:tasks.workunit:Stopping ['mon/pool_ops.sh', 'mon/crush_ops.sh', 'mon/osd.sh', 'mon/caps.sh'] on client.0... 2024-09-15T01:48:56.719 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-15T01:48:57.025 DEBUG:teuthology.parallel:result is None 2024-09-15T01:48:57.026 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T01:48:57.050 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T01:48:57.050 DEBUG:teuthology.orchestra.run.smithi204:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-15T01:48:57.117 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T01:48:57.118 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-09-15T01:48:57.127 INFO:tasks.mon_thrash:joining mon_thrasher 2024-09-15T01:49:51.593 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T01:49:51.593 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-15T01:49:51.594 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T01:49:51.594 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T01:49:51.598 INFO:tasks.ceph.mon.a:Started 2024-09-15T01:49:51.598 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T01:49:51.598 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-15T01:49:51.598 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-15T01:49:51.598 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-15T01:49:51.601 INFO:tasks.ceph.mon.f:Started 2024-09-15T01:49:51.601 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:49:51.601 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:49:51.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.703+0000 7f46cb646640 10 monclient: get_monmap_and_config 2024-09-15T01:49:51.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.703+0000 7f46cb646640 10 monclient: build_initial_monmap 2024-09-15T01:49:51.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.703+0000 7f46cb646640 10 monclient: monmap: 2024-09-15T01:49:51.713 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:51.713 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:51.713 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:49:51.704767+0000 2024-09-15T01:49:51.713 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:49:51.704767+0000 2024-09-15T01:49:51.713 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:49:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:49:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:49:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:49:51.715 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:51.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.704+0000 7f46cb646640 1 Processor -- start 2024-09-15T01:49:51.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.704+0000 7f46cb646640 1 -- start start 2024-09-15T01:49:51.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.704+0000 7f46cb646640 10 monclient: init 2024-09-15T01:49:51.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:51.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 10 monclient: _add_conns ranks=[7,1,4,5,3,0,2,10,6,8,9] 2024-09-15T01:49:51.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f46c40a2c80 0x7f46c40a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:51.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 10 monclient(hunting): picked mon.noname-i con 0x7f46c40a2c80 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:49:51.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f46c409fc70 addr v1:172.21.15.204:6789/0 2024-09-15T01:49:51.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46c40a00a0 0x7f46c41331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:51.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 10 monclient(hunting): picked mon.noname-c con 0x7f46c40a00a0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:49:51.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c404bf60 con 0x7f46c409fc70 2024-09-15T01:49:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46c40a3050 con 0x7f46c40a00a0 2024-09-15T01:49:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f46c40a1e50 con 0x7f46c40a2c80 2024-09-15T01:49:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46cb646640 10 monclient(hunting): authenticate will time out at 1796.455376s 2024-09-15T01:49:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c93e2640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f46c40a2c80 0x7f46c40a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c93e2640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f46c40a2c80 0x7f46c40a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3303/0 says I am v2:172.21.15.178:47064/0 (socket says 172.21.15.178:47064) 2024-09-15T01:49:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c93e2640 1 -- 172.21.15.178:0/3402732880 learned_addr learned my addr 172.21.15.178:0/3402732880 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c93e2640 10 monclient(hunting): get_auth_request con 0x7f46c40a2c80 auth_method 0 2024-09-15T01:49:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c93e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c93e2640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c93e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 1 --2- 172.21.15.178:0/3402732880 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46c40a00a0 0x7f46c41331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 10 monclient(hunting): get_auth_request con 0x7f46c40a00a0 auth_method 0 2024-09-15T01:49:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c93e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c93e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c93e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:51.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 10 monclient(hunting): handle_auth_done global_id 44977 payload 274 2024-09-15T01:49:51.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 1 -- 172.21.15.178:0/3402732880 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f46c40a2c80 msgr2=0x7f46c40a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:51.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 1 --2- 172.21.15.178:0/3402732880 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f46c40a2c80 0x7f46c40a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:51.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 1 -- 172.21.15.178:0/3402732880 >> v1:172.21.15.204:6789/0 conn(0x7f46c409fc70 legacy=0x7f46c4132ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:51.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 10 monclient: _finish_hunting 0 2024-09-15T01:49:51.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 1 monclient: found mon.noname-c 2024-09-15T01:49:51.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:51.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:49:51.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c9be3640 1 -- 172.21.15.178:0/3402732880 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c409dbc0 con 0x7f46c40a00a0 2024-09-15T01:49:51.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.705+0000 7f46c93e2640 1 --2- 172.21.15.178:0/3402732880 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f46c40a2c80 0x7f46c40a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:51.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.706+0000 7f46c9be3640 1 --2- 172.21.15.178:0/3402732880 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46c40a00a0 0x7f46c41331c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009e90 tx=0x7f46bc009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2d1086fffbdcb786 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:51.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.706+0000 7f46bbfff640 1 -- 172.21.15.178:0/3402732880 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46bc0079d0 con 0x7f46c40a00a0 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.706+0000 7f46bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46bbfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46bbfff640 10 monclient: dump: 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:51.719 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46bbfff640 10 monclient: _finish_auth 0 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46bbfff640 1 -- 172.21.15.178:0/3402732880 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc02d050 con 0x7f46c40a00a0 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 5 monclient: authenticate success, global_id 44977 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46bbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 10 monclient: get_monmap_and_config success 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46bbfff640 1 -- 172.21.15.178:0/3402732880 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46bc007bd0 con 0x7f46c40a00a0 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 4 set_mon_vals no callback set 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46bbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46bbfff640 10 monclient: dump: 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 10 monclient: shutdown 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 1 -- 172.21.15.178:0/3402732880 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46c40a00a0 msgr2=0x7f46c41331c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 1 --2- 172.21.15.178:0/3402732880 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46c40a00a0 0x7f46c41331c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009e90 tx=0x7f46bc009780 comp rx=0 tx=0).stop 2024-09-15T01:49:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 1 -- 172.21.15.178:0/3402732880 shutdown_connections 2024-09-15T01:49:51.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 1 --2- 172.21.15.178:0/3402732880 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46c40a00a0 0x7f46c41331c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:51.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 1 --2- 172.21.15.178:0/3402732880 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f46c40a2c80 0x7f46c40a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:51.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 1 -- 172.21.15.178:0/3402732880 >> 172.21.15.178:0/3402732880 conn(0x7f46c406e860 msgr2=0x7f46c406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:51.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 1 -- 172.21.15.178:0/3402732880 shutdown_connections 2024-09-15T01:49:51.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.707+0000 7f46cb646640 1 -- 172.21.15.178:0/3402732880 wait complete. 2024-09-15T01:49:51.723 INFO:tasks.ceph.mon.a.smithi178.stderr:2024-09-15T01:49:51.708+0000 7fd5afec9900 -1 mon.a@-1(???).log v0 log_external_backlog rewinding external_log_to from 541 -> 0 (sync_force? mon rebuild?) 2024-09-15T01:49:51.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.708+0000 7f46cb646640 10 monclient: build_initial_monmap 2024-09-15T01:49:51.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.708+0000 7f46cb646640 10 monclient: monmap: 2024-09-15T01:49:51.728 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:51.728 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:49:51.728 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:49:51.728 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:49:51.728 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:51.728 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:49:51.728 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:49:51.728 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:49:51.728 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:51.728 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:51.728 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.708+0000 7f46cb646640 1 Processor -- start 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.708+0000 7f46cb646640 1 -- start start 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.708+0000 7f46cb646640 10 monclient: init 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 10 monclient: _add_conns ranks=[0,2,3,4,7,6,8,5,1] 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46c409fc70 0x7f46c41421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 10 monclient(hunting): picked mon.noname-a con 0x7f46c409fc70 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:49:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46c40a00a0 0x7f46c4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:51.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 10 monclient(hunting): picked mon.noname-b con 0x7f46c40a00a0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:49:51.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46c40a2c80 0x7f46c411c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:51.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 10 monclient(hunting): picked mon.noname-g con 0x7f46c40a2c80 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:49:51.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:51.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c41204c0 con 0x7f46c409fc70 2024-09-15T01:49:51.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:51.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c409d5f0 con 0x7f46c40a00a0 2024-09-15T01:49:51.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:51.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c41201e0 con 0x7f46c40a2c80 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46cb646640 10 monclient(hunting): authenticate will time out at 1796.459540s 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c8be1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46c40a00a0 0x7f46c4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c8be1640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46c40a00a0 0x7f46c4132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:48846/0 (socket says 172.21.15.178:48846) 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c8be1640 1 -- 172.21.15.178:0/1341796525 learned_addr learned my addr 172.21.15.178:0/1341796525 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c8be1640 10 monclient(hunting): get_auth_request con 0x7f46c40a00a0 auth_method 0 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c8be1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c8be1640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c8be1640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c9be3640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46c40a2c80 0x7f46c411c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c9be3640 10 monclient(hunting): get_auth_request con 0x7f46c40a2c80 auth_method 0 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c9be3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c9be3640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c9be3640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c8be1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c8be1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c8be1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c9be3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c9be3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c93e2640 1 --2- 172.21.15.178:0/1341796525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46c409fc70 0x7f46c41421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c9be3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c93e2640 10 monclient(hunting): get_auth_request con 0x7f46c409fc70 auth_method 0 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c93e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c93e2640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c93e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c93e2640 1 -- 172.21.15.178:0/1341796525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46c409fc70 msgr2=0x7f46c41421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:49:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c93e2640 1 -- 172.21.15.178:0/1341796525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46c409fc70 msgr2=0x7f46c41421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:49:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c93e2640 1 --2- 172.21.15.178:0/1341796525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46c409fc70 0x7f46c41421b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:49:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c8be1640 10 monclient(hunting): handle_auth_done global_id 35417 payload 824 2024-09-15T01:49:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c8be1640 1 -- 172.21.15.178:0/1341796525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46c40a2c80 msgr2=0x7f46c411c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46c8be1640 1 --2- 172.21.15.178:0/1341796525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46c40a2c80 0x7f46c411c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.709+0000 7f46c93e2640 1 --2- 172.21.15.178:0/1341796525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46c409fc70 0x7f46c41421b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:49:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46c8be1640 1 -- 172.21.15.178:0/1341796525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46c409fc70 msgr2=0x7f46c41421b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:49:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46c8be1640 1 --2- 172.21.15.178:0/1341796525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46c409fc70 0x7f46c41421b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46c8be1640 10 monclient: _finish_hunting 0 2024-09-15T01:49:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46c8be1640 1 monclient: found mon.noname-b 2024-09-15T01:49:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46c8be1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46c8be1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46c8be1640 1 -- 172.21.15.178:0/1341796525 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b4009450 con 0x7f46c40a00a0 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46c8be1640 1 --2- 172.21.15.178:0/1341796525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46c40a00a0 0x7f46c4132de0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f46c000b5e0 tx=0x7f46c00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46b97fa640 1 -- 172.21.15.178:0/1341796525 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46c000fa10 con 0x7f46c40a00a0 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46b97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46b97fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46b97fa640 10 monclient: dump: 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:51.734 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46b97fa640 10 monclient: _finish_auth 0 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46b97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46b97fa640 1 -- 172.21.15.178:0/1341796525 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c000fbb0 con 0x7f46c40a00a0 2024-09-15T01:49:51.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46b97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46b97fa640 1 -- 172.21.15.178:0/1341796525 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46c000fd60 con 0x7f46c40a00a0 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46b97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46b97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46bb7fe640 4 set_mon_vals no callback set 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46b97fa640 10 monclient: dump: 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46cb646640 5 monclient: authenticate success, global_id 35417 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46cb646640 10 monclient: _renew_subs 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46cb646640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46cb646640 1 -- 172.21.15.178:0/1341796525 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46bc009a20 con 0x7f46c40a00a0 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46cb646640 10 monclient: _renew_subs 2024-09-15T01:49:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46cb646640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.710+0000 7f46cb646640 1 -- 172.21.15.178:0/1341796525 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c411ff00 con 0x7f46c40a00a0 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.711+0000 7f46cb646640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.711+0000 7f46cb646640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.711+0000 7f46cb646640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.711+0000 7f46cb646640 1 -- 172.21.15.178:0/1341796525 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c404b720 con 0x7f46c40a00a0 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.711+0000 7f46b97fa640 1 -- 172.21.15.178:0/1341796525 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f46c000f590 con 0x7f46c40a00a0 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.712+0000 7f46b97fa640 1 --2- 172.21.15.178:0/1341796525 >> v2:172.21.15.204:6812/28210 conn(0x7f46a80397b0 0x7f46a803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.712+0000 7f46b97fa640 1 -- 172.21.15.178:0/1341796525 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f46c004b6c0 con 0x7f46c40a00a0 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.712+0000 7f46c93e2640 1 --2- 172.21.15.178:0/1341796525 >> v2:172.21.15.204:6812/28210 conn(0x7f46a80397b0 0x7f46a803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.712+0000 7f46c93e2640 10 monclient: get_auth_request con 0x7f46a80397b0 auth_method 0 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.712+0000 7f46c93e2640 1 --2- 172.21.15.178:0/1341796525 >> v2:172.21.15.204:6812/28210 conn(0x7f46a80397b0 0x7f46a803bc60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f46b4006bc0 tx=0x7f46b4008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.717+0000 7f46b97fa640 1 -- 172.21.15.178:0/1341796525 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f46c000f840 con 0x7f46c40a00a0 2024-09-15T01:49:51.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.717+0000 7f46b97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:51.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.717+0000 7f46b97fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:49:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.958+0000 7f46cb646640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:49:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.958+0000 7f46cb646640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:49:51.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.958+0000 7f46cb646640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:49:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.958+0000 7f46cb646640 1 -- 172.21.15.178:0/1341796525 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f46c4070950 con 0x7f46c40a00a0 2024-09-15T01:49:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.959+0000 7f46b97fa640 1 -- 172.21.15.178:0/1341796525 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3173 (secure 0 0 0) 0x7f46c000d310 con 0x7f46c40a00a0 2024-09-15T01:49:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.959+0000 7f46b97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:49:51.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.959+0000 7f46b97fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:49:51.962 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":156,"quorum":[2,3,4,5,6,7,8],"quorum_names":["b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":95,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:49:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 1 -- 172.21.15.178:0/1341796525 >> v2:172.21.15.204:6812/28210 conn(0x7f46a80397b0 msgr2=0x7f46a803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 1 --2- 172.21.15.178:0/1341796525 >> v2:172.21.15.204:6812/28210 conn(0x7f46a80397b0 0x7f46a803bc60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f46b4006bc0 tx=0x7f46b4008040 comp rx=0 tx=0).stop 2024-09-15T01:49:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 10 monclient: shutdown 2024-09-15T01:49:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 1 -- 172.21.15.178:0/1341796525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46c40a00a0 msgr2=0x7f46c4132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:51.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 1 --2- 172.21.15.178:0/1341796525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46c40a00a0 0x7f46c4132de0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f46c000b5e0 tx=0x7f46c00090f0 comp rx=0 tx=0).stop 2024-09-15T01:49:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 1 -- 172.21.15.178:0/1341796525 shutdown_connections 2024-09-15T01:49:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 1 --2- 172.21.15.178:0/1341796525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46c40a2c80 0x7f46c411c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 1 --2- 172.21.15.178:0/1341796525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46c40a00a0 0x7f46c4132de0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 1 --2- 172.21.15.178:0/1341796525 >> v2:172.21.15.204:6812/28210 conn(0x7f46a80397b0 0x7f46a803bc60 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 1 --2- 172.21.15.178:0/1341796525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46c409fc70 0x7f46c41421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 1 -- 172.21.15.178:0/1341796525 >> 172.21.15.178:0/1341796525 conn(0x7f46c406e860 msgr2=0x7f46c40a68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 1 -- 172.21.15.178:0/1341796525 shutdown_connections 2024-09-15T01:49:51.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:51.962+0000 7f46cb646640 1 -- 172.21.15.178:0/1341796525 wait complete. 2024-09-15T01:49:54.981 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:49:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.083+0000 7f466a99d640 10 monclient: get_monmap_and_config 2024-09-15T01:49:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.083+0000 7f466a99d640 10 monclient: build_initial_monmap 2024-09-15T01:49:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.083+0000 7f466a99d640 10 monclient: monmap: 2024-09-15T01:49:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:49:55.083916+0000 2024-09-15T01:49:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:49:55.083916+0000 2024-09-15T01:49:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:49:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:49:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:49:55.086 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:49:55.086 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:55.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.083+0000 7f466a99d640 1 Processor -- start 2024-09-15T01:49:55.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 1 -- start start 2024-09-15T01:49:55.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 10 monclient: init 2024-09-15T01:49:55.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:55.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 10 monclient: _add_conns ranks=[5,3,8,7,6,10,9,2,4,1,0] 2024-09-15T01:49:55.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f46640a2930 0x7f4664132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:55.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 10 monclient(hunting): picked mon.noname-h con 0x7f46640a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:49:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f466409faa0 0x7f466409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 10 monclient(hunting): picked mon.noname-g con 0x7f466409faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:49:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a08c0 0x7f46640a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 10 monclient(hunting): picked mon.noname-e con 0x7f46640a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:49:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466409e0f0 con 0x7f466409faa0 2024-09-15T01:49:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f466409dd80 con 0x7f46640a2930 2024-09-15T01:49:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f46640a2d00 con 0x7f46640a08c0 2024-09-15T01:49:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f466a99d640 10 monclient(hunting): authenticate will time out at 1799.834557s 2024-09-15T01:49:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4668f3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a08c0 0x7f46640a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4668f3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a08c0 0x7f46640a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:56460/0 (socket says 172.21.15.178:56460) 2024-09-15T01:49:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4668f3a640 1 -- 172.21.15.178:0/1971621377 learned_addr learned my addr 172.21.15.178:0/1971621377 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4668f3a640 10 monclient(hunting): get_auth_request con 0x7f46640a08c0 auth_method 0 2024-09-15T01:49:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4668f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4668f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4668f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4668f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4668f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4668f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4663fff640 1 --2- 172.21.15.178:0/1971621377 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f46640a2930 0x7f4664132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f465bfff640 1 --2- 172.21.15.178:0/1971621377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f466409faa0 0x7f466409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4663fff640 10 monclient(hunting): get_auth_request con 0x7f46640a2930 auth_method 0 2024-09-15T01:49:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4663fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4663fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4663fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f465bfff640 10 monclient(hunting): get_auth_request con 0x7f466409faa0 auth_method 0 2024-09-15T01:49:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f465bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f465bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f465bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4663fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:55.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.084+0000 7f4663fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:55.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f4663fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:55.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f465bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:55.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f465bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:55.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f465bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:55.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f4668f3a640 10 monclient(hunting): handle_auth_done global_id 94382 payload 274 2024-09-15T01:49:55.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f4668f3a640 1 -- 172.21.15.178:0/1971621377 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f46640a2930 msgr2=0x7f4664132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:55.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f4668f3a640 1 --2- 172.21.15.178:0/1971621377 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f46640a2930 0x7f4664132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:55.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f4668f3a640 1 -- 172.21.15.178:0/1971621377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f466409faa0 msgr2=0x7f466409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f4668f3a640 1 --2- 172.21.15.178:0/1971621377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f466409faa0 0x7f466409fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f4668f3a640 10 monclient: _finish_hunting 0 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f4668f3a640 1 monclient: found mon.noname-e 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f4668f3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f4668f3a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f4668f3a640 1 -- 172.21.15.178:0/1971621377 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466404bf60 con 0x7f46640a08c0 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f4668f3a640 1 --2- 172.21.15.178:0/1971621377 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a08c0 0x7f46640a0d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f465c009f50 tx=0x7f465c009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=7f4239412c72dc37 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f46637fe640 1 -- 172.21.15.178:0/1971621377 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f465c007880 con 0x7f46640a08c0 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f46637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f46637fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.085+0000 7f46637fe640 10 monclient: dump: 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:55.103 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f46637fe640 10 monclient: _finish_auth 0 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f46637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f46637fe640 1 -- 172.21.15.178:0/1971621377 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c02d050 con 0x7f46640a08c0 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 5 monclient: authenticate success, global_id 94382 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f46637fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 10 monclient: get_monmap_and_config success 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f46637fe640 1 -- 172.21.15.178:0/1971621377 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f465c030360 con 0x7f46640a08c0 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 4 set_mon_vals no callback set 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f46637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:55.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f46637fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f46637fe640 10 monclient: dump: 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:55.106 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 10 monclient: shutdown 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 1 -- 172.21.15.178:0/1971621377 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a08c0 msgr2=0x7f46640a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 1 --2- 172.21.15.178:0/1971621377 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a08c0 0x7f46640a0d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f465c009f50 tx=0x7f465c009840 comp rx=0 tx=0).stop 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 1 -- 172.21.15.178:0/1971621377 shutdown_connections 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 1 --2- 172.21.15.178:0/1971621377 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a08c0 0x7f46640a0d30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 1 --2- 172.21.15.178:0/1971621377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f466409faa0 0x7f466409fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 1 --2- 172.21.15.178:0/1971621377 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f46640a2930 0x7f4664132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 1 -- 172.21.15.178:0/1971621377 >> 172.21.15.178:0/1971621377 conn(0x7f46640a5ed0 msgr2=0x7f46640a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 1 -- 172.21.15.178:0/1971621377 shutdown_connections 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.086+0000 7f466a99d640 1 -- 172.21.15.178:0/1971621377 wait complete. 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient: build_initial_monmap 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient: monmap: 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:55.108 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:55.109 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:55.109 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:55.109 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:55.109 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:55.109 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:55.109 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:55.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 1 Processor -- start 2024-09-15T01:49:55.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 1 -- start start 2024-09-15T01:49:55.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient: init 2024-09-15T01:49:55.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:55.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient: _add_conns ranks=[1,4,8,5,2,7,0,3,6] 2024-09-15T01:49:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f466409faa0 0x7f466409b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient(hunting): picked mon.noname-f con 0x7f466409faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:49:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46640a08c0 0x7f4664131a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient(hunting): picked mon.noname-c con 0x7f46640a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:49:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a2930 0x7f4664141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient(hunting): picked mon.noname-e con 0x7f46640a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:49:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466411eb20 con 0x7f466409faa0 2024-09-15T01:49:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f466411ee00 con 0x7f46640a08c0 2024-09-15T01:49:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f466411f0e0 con 0x7f46640a2930 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.087+0000 7f466a99d640 10 monclient(hunting): authenticate will time out at 1799.837976s 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f465bfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46640a08c0 0x7f4664131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a2930 0x7f4664141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f465bfff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46640a08c0 0x7f4664131a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:58594/0 (socket says 172.21.15.178:58594) 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f465bfff640 1 -- 172.21.15.178:0/2702923536 learned_addr learned my addr 172.21.15.178:0/2702923536 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f465bfff640 10 monclient(hunting): get_auth_request con 0x7f46640a08c0 auth_method 0 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a2930 0x7f4664141dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:56472/0 (socket says 172.21.15.178:56472) 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f465bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f465bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f465bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 10 monclient(hunting): get_auth_request con 0x7f46640a2930 auth_method 0 2024-09-15T01:49:55.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:55.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:55.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:55.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4663fff640 1 --2- 172.21.15.178:0/2702923536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f466409faa0 0x7f466409b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:55.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4663fff640 10 monclient(hunting): get_auth_request con 0x7f466409faa0 auth_method 0 2024-09-15T01:49:55.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4663fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:55.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4663fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:55.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4663fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:55.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:55.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:55.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:55.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f465bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:55.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f465bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:55.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f465bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:55.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4663fff640 1 -- 172.21.15.178:0/2702923536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f466409faa0 msgr2=0x7f466409b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T01:49:55.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4663fff640 1 -- 172.21.15.178:0/2702923536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f466409faa0 msgr2=0x7f466409b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T01:49:55.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4663fff640 1 --2- 172.21.15.178:0/2702923536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f466409faa0 0x7f466409b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:49:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4663fff640 1 --2- 172.21.15.178:0/2702923536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f466409faa0 0x7f466409b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:49:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 10 monclient(hunting): handle_auth_done global_id 94391 payload 824 2024-09-15T01:49:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 1 -- 172.21.15.178:0/2702923536 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46640a08c0 msgr2=0x7f4664131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 1 --2- 172.21.15.178:0/2702923536 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46640a08c0 0x7f4664131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 1 -- 172.21.15.178:0/2702923536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f466409faa0 msgr2=0x7f466409b290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T01:49:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 1 --2- 172.21.15.178:0/2702923536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f466409faa0 0x7f466409b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 10 monclient: _finish_hunting 0 2024-09-15T01:49:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 1 monclient: found mon.noname-e 2024-09-15T01:49:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:49:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 1 -- 172.21.15.178:0/2702923536 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c009ae0 con 0x7f46640a2930 2024-09-15T01:49:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4668f3a640 1 --2- 172.21.15.178:0/2702923536 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a2930 0x7f4664141dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f465c030000 tx=0x7f465c036000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4660ff9640 1 -- 172.21.15.178:0/2702923536 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f465c03baa0 con 0x7f46640a2930 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4660ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4660ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4660ff9640 10 monclient: dump: 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:55.115 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:55.116 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4660ff9640 10 monclient: _finish_auth 0 2024-09-15T01:49:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4660ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4660ff9640 1 -- 172.21.15.178:0/2702923536 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c03bc60 con 0x7f46640a2930 2024-09-15T01:49:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f466a99d640 5 monclient: authenticate success, global_id 94391 2024-09-15T01:49:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f466a99d640 10 monclient: _renew_subs 2024-09-15T01:49:55.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f466a99d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f466a99d640 1 -- 172.21.15.178:0/2702923536 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466409bde0 con 0x7f46640a2930 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4660ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4660ff9640 1 -- 172.21.15.178:0/2702923536 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f465c03bdd0 con 0x7f46640a2930 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4660ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4660ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4660ff9640 10 monclient: dump: 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f466a99d640 10 monclient: _renew_subs 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f466a99d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f466a99d640 1 -- 172.21.15.178:0/2702923536 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466411dbc0 con 0x7f46640a2930 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.088+0000 7f4662ffd640 4 set_mon_vals no callback set 2024-09-15T01:49:55.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.089+0000 7f4660ff9640 1 -- 172.21.15.178:0/2702923536 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f465c038070 con 0x7f46640a2930 2024-09-15T01:49:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.089+0000 7f466a99d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.089+0000 7f466a99d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.089+0000 7f466a99d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:49:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.089+0000 7f466a99d640 1 -- 172.21.15.178:0/2702923536 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466411ec60 con 0x7f46640a2930 2024-09-15T01:49:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.089+0000 7f4660ff9640 1 --2- 172.21.15.178:0/2702923536 >> v2:172.21.15.204:6812/28210 conn(0x7f46440396e0 0x7f464403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.090+0000 7f4660ff9640 1 -- 172.21.15.178:0/2702923536 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f465c06c4b0 con 0x7f46640a2930 2024-09-15T01:49:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.090+0000 7f4663fff640 1 --2- 172.21.15.178:0/2702923536 >> v2:172.21.15.204:6812/28210 conn(0x7f46440396e0 0x7f464403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.090+0000 7f4663fff640 10 monclient: get_auth_request con 0x7f46440396e0 auth_method 0 2024-09-15T01:49:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.090+0000 7f4663fff640 1 --2- 172.21.15.178:0/2702923536 >> v2:172.21.15.204:6812/28210 conn(0x7f46440396e0 0x7f464403bb90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4654009490 tx=0x7f4654008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.094+0000 7f4660ff9640 1 -- 172.21.15.178:0/2702923536 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f466411ec60 con 0x7f46640a2930 2024-09-15T01:49:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.094+0000 7f4660ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:55.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.094+0000 7f4660ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:49:55.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.310+0000 7f466a99d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:49:55.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.310+0000 7f466a99d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:49:55.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.310+0000 7f466a99d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:49:55.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.310+0000 7f466a99d640 1 -- 172.21.15.178:0/2702923536 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f466411ec60 con 0x7f46640a2930 2024-09-15T01:49:55.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.310+0000 7f4660ff9640 1 -- 172.21.15.178:0/2702923536 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3173 (secure 0 0 0) 0x7f465c03fc80 con 0x7f46640a2930 2024-09-15T01:49:55.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.310+0000 7f4660ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:49:55.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.310+0000 7f4660ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:49:55.313 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":156,"quorum":[2,3,4,5,6,7,8],"quorum_names":["b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":98,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:49:55.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 1 -- 172.21.15.178:0/2702923536 >> v2:172.21.15.204:6812/28210 conn(0x7f46440396e0 msgr2=0x7f464403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:55.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 1 --2- 172.21.15.178:0/2702923536 >> v2:172.21.15.204:6812/28210 conn(0x7f46440396e0 0x7f464403bb90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4654009490 tx=0x7f4654008040 comp rx=0 tx=0).stop 2024-09-15T01:49:55.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 10 monclient: shutdown 2024-09-15T01:49:55.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:55.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 1 -- 172.21.15.178:0/2702923536 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a2930 msgr2=0x7f4664141dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 1 --2- 172.21.15.178:0/2702923536 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a2930 0x7f4664141dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f465c030000 tx=0x7f465c036000 comp rx=0 tx=0).stop 2024-09-15T01:49:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 1 -- 172.21.15.178:0/2702923536 shutdown_connections 2024-09-15T01:49:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 1 --2- 172.21.15.178:0/2702923536 >> v2:172.21.15.204:6812/28210 conn(0x7f46440396e0 0x7f464403bb90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 1 --2- 172.21.15.178:0/2702923536 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f46640a2930 0x7f4664141dd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 1 --2- 172.21.15.178:0/2702923536 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f46640a08c0 0x7f4664131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 1 --2- 172.21.15.178:0/2702923536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f466409faa0 0x7f466409b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 1 -- 172.21.15.178:0/2702923536 >> 172.21.15.178:0/2702923536 conn(0x7f46640a5ed0 msgr2=0x7f4664135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 1 -- 172.21.15.178:0/2702923536 shutdown_connections 2024-09-15T01:49:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:55.312+0000 7f466a99d640 1 -- 172.21.15.178:0/2702923536 wait complete. 2024-09-15T01:49:58.328 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:49:58.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.423+0000 7fa88a678640 10 monclient: get_monmap_and_config 2024-09-15T01:49:58.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.423+0000 7fa88a678640 10 monclient: build_initial_monmap 2024-09-15T01:49:58.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.423+0000 7fa88a678640 10 monclient: monmap: 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:49:58.424488+0000 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:49:58.424488+0000 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:58.426 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:58.427 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:58.427 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:58.427 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:49:58.427 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:49:58.427 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 1 Processor -- start 2024-09-15T01:49:58.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 1 -- start start 2024-09-15T01:49:58.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 10 monclient: init 2024-09-15T01:49:58.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 10 monclient: _add_conns ranks=[4,6,3,8,7,10,5,1,2,9,0] 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa8840a2930 0x7fa884132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 10 monclient(hunting): picked mon.noname-c con 0x7fa8840a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa88409faa0 0x7fa88409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 10 monclient(hunting): picked mon.noname-d con 0x7fa88409faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8840a08c0 0x7fa8840a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 10 monclient(hunting): picked mon.noname-g con 0x7fa8840a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88409e0f0 con 0x7fa8840a08c0 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa88409dd80 con 0x7fa8840a2930 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840a2d00 con 0x7fa88409faa0 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.424+0000 7fa88a678640 10 monclient(hunting): authenticate will time out at 1803.174922s 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa88409faa0 0x7fa88409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa883fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa8840a2930 0x7fa884132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa88409faa0 0x7fa88409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:35378/0 (socket says 172.21.15.178:35378) 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa883fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa8840a2930 0x7fa884132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44702/0 (socket says 172.21.15.178:44702) 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 1 -- 172.21.15.178:0/3582960778 learned_addr learned my addr 172.21.15.178:0/3582960778 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 10 monclient(hunting): get_auth_request con 0x7fa88409faa0 auth_method 0 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa883fff640 10 monclient(hunting): get_auth_request con 0x7fa8840a2930 auth_method 0 2024-09-15T01:49:58.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa883fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa883fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa883fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa883fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa883fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa883fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa888c15640 1 --2- 172.21.15.178:0/3582960778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8840a08c0 0x7fa8840a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa888c15640 10 monclient(hunting): get_auth_request con 0x7fa8840a08c0 auth_method 0 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa888c15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa888c15640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa888c15640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 10 monclient(hunting): handle_auth_done global_id 25863 payload 274 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 1 -- 172.21.15.178:0/3582960778 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa8840a2930 msgr2=0x7fa884132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 1 --2- 172.21.15.178:0/3582960778 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa8840a2930 0x7fa884132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 1 -- 172.21.15.178:0/3582960778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8840a08c0 msgr2=0x7fa8840a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 1 --2- 172.21.15.178:0/3582960778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8840a08c0 0x7fa8840a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 10 monclient: _finish_hunting 0 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 1 monclient: found mon.noname-d 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa8837fe640 1 -- 172.21.15.178:0/3582960778 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88404bf60 con 0x7fa88409faa0 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa883fff640 1 --2- 172.21.15.178:0/3582960778 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa8840a2930 0x7fa884132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.425+0000 7fa888c15640 1 --2- 172.21.15.178:0/3582960778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8840a08c0 0x7fa8840a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:49:58.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa8837fe640 1 --2- 172.21.15.178:0/3582960778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa88409faa0 0x7fa88409fe70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa878009940 tx=0x7fa8780091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=83bc6e7901022ca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa882ffd640 1 -- 172.21.15.178:0/3582960778 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa87802ac60 con 0x7fa88409faa0 2024-09-15T01:49:58.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa882ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa882ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:49:58.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa882ffd640 10 monclient: dump: 2024-09-15T01:49:58.440 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.440 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.440 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.440 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.440 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.440 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.440 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.440 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa882ffd640 10 monclient: _finish_auth 0 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa882ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 5 monclient: authenticate success, global_id 25863 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa882ffd640 1 -- 172.21.15.178:0/3582960778 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87802ade0 con 0x7fa88409faa0 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:58.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa882ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 10 monclient: get_monmap_and_config success 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 4 set_mon_vals no callback set 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa882ffd640 1 -- 172.21.15.178:0/3582960778 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa878031ca0 con 0x7fa88409faa0 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa882ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa882ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa882ffd640 10 monclient: dump: 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.442 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 10 monclient: shutdown 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 1 -- 172.21.15.178:0/3582960778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa88409faa0 msgr2=0x7fa88409fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 1 --2- 172.21.15.178:0/3582960778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa88409faa0 0x7fa88409fe70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa878009940 tx=0x7fa8780091d0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 1 -- 172.21.15.178:0/3582960778 shutdown_connections 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 1 --2- 172.21.15.178:0/3582960778 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa88409faa0 0x7fa88409fe70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 1 --2- 172.21.15.178:0/3582960778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8840a08c0 0x7fa8840a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 1 --2- 172.21.15.178:0/3582960778 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fa8840a2930 0x7fa884132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 1 -- 172.21.15.178:0/3582960778 >> 172.21.15.178:0/3582960778 conn(0x7fa8840a5ed0 msgr2=0x7fa8840a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 1 -- 172.21.15.178:0/3582960778 shutdown_connections 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.426+0000 7fa88a678640 1 -- 172.21.15.178:0/3582960778 wait complete. 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.427+0000 7fa88a678640 10 monclient: build_initial_monmap 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.427+0000 7fa88a678640 10 monclient: monmap: 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:49:58.444 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.427+0000 7fa88a678640 1 Processor -- start 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.427+0000 7fa88a678640 1 -- start start 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.427+0000 7fa88a678640 10 monclient: init 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 10 monclient: _add_conns ranks=[5,8,7,1,2,4,3,6,0] 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa88409faa0 0x7fa88409b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 10 monclient(hunting): picked mon.noname-h con 0x7fa88409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8840a08c0 0x7fa88411ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 10 monclient(hunting): picked mon.noname-e con 0x7fa8840a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8840a2930 0x7fa884131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 10 monclient(hunting): picked mon.noname-i con 0x7fa8840a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa88411db10 con 0x7fa88409faa0 2024-09-15T01:49:58.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa88411ea40 con 0x7fa8840a2930 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa88411ed20 con 0x7fa8840a08c0 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa88a678640 10 monclient(hunting): authenticate will time out at 1803.178301s 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8840a08c0 0x7fa88411ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8840a08c0 0x7fa88411ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:39972/0 (socket says 172.21.15.178:39972) 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 1 -- 172.21.15.178:0/12011000 learned_addr learned my addr 172.21.15.178:0/12011000 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 10 monclient(hunting): get_auth_request con 0x7fa8840a08c0 auth_method 0 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa883fff640 1 --2- 172.21.15.178:0/12011000 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa88409faa0 0x7fa88409b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa888c15640 1 --2- 172.21.15.178:0/12011000 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8840a2930 0x7fa884131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa888c15640 10 monclient(hunting): get_auth_request con 0x7fa8840a2930 auth_method 0 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa888c15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa888c15640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa888c15640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 10 monclient(hunting): handle_auth_done global_id 94400 payload 824 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 1 -- 172.21.15.178:0/12011000 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8840a2930 msgr2=0x7fa884131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 1 --2- 172.21.15.178:0/12011000 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8840a2930 0x7fa884131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 1 -- 172.21.15.178:0/12011000 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa88409faa0 msgr2=0x7fa88409b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 1 --2- 172.21.15.178:0/12011000 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa88409faa0 0x7fa88409b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 10 monclient: _finish_hunting 0 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 1 monclient: found mon.noname-e 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 1 -- 172.21.15.178:0/12011000 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa874009390 con 0x7fa8840a08c0 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa883fff640 10 monclient: get_auth_request con 0x7fa88409faa0 auth_method 0 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa883fff640 1 --2- 172.21.15.178:0/12011000 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa88409faa0 0x7fa88409b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa8837fe640 1 --2- 172.21.15.178:0/12011000 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8840a08c0 0x7fa88411ca40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa878000c00 tx=0x7fa878032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa86bfff640 1 -- 172.21.15.178:0/12011000 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa878031820 con 0x7fa8840a08c0 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa86bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa86bfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.428+0000 7fa86bfff640 10 monclient: dump: 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.449 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.450 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa86bfff640 10 monclient: _finish_auth 0 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa86bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa86bfff640 1 -- 172.21.15.178:0/12011000 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8780319c0 con 0x7fa8840a08c0 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa88a678640 5 monclient: authenticate success, global_id 94400 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa88a678640 10 monclient: _renew_subs 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa88a678640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa88a678640 1 -- 172.21.15.178:0/12011000 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa878009450 con 0x7fa8840a08c0 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa86bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa86bfff640 1 -- 172.21.15.178:0/12011000 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa87803a390 con 0x7fa8840a08c0 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa86bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa86bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:49:58.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa86bfff640 10 monclient: dump: 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.452 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.453 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.453 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa88a678640 10 monclient: _renew_subs 2024-09-15T01:49:58.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa8827fc640 4 set_mon_vals no callback set 2024-09-15T01:49:58.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa88a678640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:49:58.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa88a678640 1 -- 172.21.15.178:0/12011000 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88411f2e0 con 0x7fa8840a08c0 2024-09-15T01:49:58.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.429+0000 7fa86bfff640 1 -- 172.21.15.178:0/12011000 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa878039040 con 0x7fa8840a08c0 2024-09-15T01:49:58.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.430+0000 7fa88a678640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:58.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.430+0000 7fa88a678640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:58.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.430+0000 7fa88a678640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:49:58.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.430+0000 7fa88a678640 1 -- 172.21.15.178:0/12011000 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88404b720 con 0x7fa8840a08c0 2024-09-15T01:49:58.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.430+0000 7fa86bfff640 1 --2- 172.21.15.178:0/12011000 >> v2:172.21.15.204:6812/28210 conn(0x7fa8700397b0 0x7fa87003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.430+0000 7fa86bfff640 1 -- 172.21.15.178:0/12011000 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fa87806b240 con 0x7fa8840a08c0 2024-09-15T01:49:58.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.430+0000 7fa883fff640 1 --2- 172.21.15.178:0/12011000 >> v2:172.21.15.204:6812/28210 conn(0x7fa8700397b0 0x7fa87003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.430+0000 7fa883fff640 10 monclient: get_auth_request con 0x7fa8700397b0 auth_method 0 2024-09-15T01:49:58.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.430+0000 7fa883fff640 1 --2- 172.21.15.178:0/12011000 >> v2:172.21.15.204:6812/28210 conn(0x7fa8700397b0 0x7fa87003bc60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa874009360 tx=0x7fa874008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.434+0000 7fa86bfff640 1 -- 172.21.15.178:0/12011000 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa87802fa80 con 0x7fa8840a08c0 2024-09-15T01:49:58.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.434+0000 7fa86bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:58.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.434+0000 7fa86bfff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:49:58.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.648+0000 7fa88a678640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:49:58.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.648+0000 7fa88a678640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:49:58.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.648+0000 7fa88a678640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:49:58.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.648+0000 7fa88a678640 1 -- 172.21.15.178:0/12011000 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa88409cef0 con 0x7fa8840a08c0 2024-09-15T01:49:58.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.649+0000 7fa86bfff640 1 -- 172.21.15.178:0/12011000 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7fa878041c80 con 0x7fa8840a08c0 2024-09-15T01:49:58.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.649+0000 7fa86bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:49:58.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.649+0000 7fa86bfff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:49:58.652 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":158,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:49:58.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 1 -- 172.21.15.178:0/12011000 >> v2:172.21.15.204:6812/28210 conn(0x7fa8700397b0 msgr2=0x7fa87003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 1 --2- 172.21.15.178:0/12011000 >> v2:172.21.15.204:6812/28210 conn(0x7fa8700397b0 0x7fa87003bc60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa874009360 tx=0x7fa874008040 comp rx=0 tx=0).stop 2024-09-15T01:49:58.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 10 monclient: shutdown 2024-09-15T01:49:58.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:58.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 1 -- 172.21.15.178:0/12011000 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8840a08c0 msgr2=0x7fa88411ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 1 --2- 172.21.15.178:0/12011000 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8840a08c0 0x7fa88411ca40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa878000c00 tx=0x7fa878032000 comp rx=0 tx=0).stop 2024-09-15T01:49:58.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 1 -- 172.21.15.178:0/12011000 shutdown_connections 2024-09-15T01:49:58.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 1 --2- 172.21.15.178:0/12011000 >> v2:172.21.15.204:6812/28210 conn(0x7fa8700397b0 0x7fa87003bc60 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 1 --2- 172.21.15.178:0/12011000 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa8840a2930 0x7fa884131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 1 --2- 172.21.15.178:0/12011000 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa8840a08c0 0x7fa88411ca40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 1 --2- 172.21.15.178:0/12011000 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa88409faa0 0x7fa88409b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 1 -- 172.21.15.178:0/12011000 >> 172.21.15.178:0/12011000 conn(0x7fa8840a5ed0 msgr2=0x7fa884135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:58.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 1 -- 172.21.15.178:0/12011000 shutdown_connections 2024-09-15T01:49:58.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.651+0000 7fa88a678640 1 -- 172.21.15.178:0/12011000 wait complete. 2024-09-15T01:49:58.664 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:49:58.665 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T01:49:58.665 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T01:49:58.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.758+0000 7fb189c78640 10 monclient: get_monmap_and_config 2024-09-15T01:49:58.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.758+0000 7fb189c78640 10 monclient: build_initial_monmap 2024-09-15T01:49:58.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.759+0000 7fb189c78640 10 monclient: monmap: 2024-09-15T01:49:58.761 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:58.761 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.761 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:49:58.759884+0000 2024-09-15T01:49:58.761 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:49:58.759884+0000 2024-09-15T01:49:58.761 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:49:58.762 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.759+0000 7fb189c78640 1 Processor -- start 2024-09-15T01:49:58.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.759+0000 7fb189c78640 1 -- start start 2024-09-15T01:49:58.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.759+0000 7fb189c78640 10 monclient: init 2024-09-15T01:49:58.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:58.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 10 monclient: _add_conns ranks=[4,0,10,6,9,1,8,5,7,2,3] 2024-09-15T01:49:58.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb1840a2930 0x7fb184132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 10 monclient(hunting): picked mon.noname-c con 0x7fb1840a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:49:58.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb18409faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:49:58.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb18409fe70 0x7fb1840a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 10 monclient(hunting): picked mon.noname-f con 0x7fb18409fe70 addr v2:172.21.15.204:3300/0 2024-09-15T01:49:58.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18404bf60 con 0x7fb18409faa0 2024-09-15T01:49:58.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18409dd80 con 0x7fb18409fe70 2024-09-15T01:49:58.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1840a2d00 con 0x7fb1840a2930 2024-09-15T01:49:58.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:58.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb189c78640 10 monclient(hunting): authenticate will time out at 1803.510395s 2024-09-15T01:49:58.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb1840a2930 0x7fb184132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb182ffd640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fb18409faa0 0x7fb1840a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:37000/0 (socket says 172.21.15.178:37000) 2024-09-15T01:49:58.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb1840a2930 0x7fb184132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44704/0 (socket says 172.21.15.178:44704) 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb182ffd640 1 -- 172.21.15.178:0/1985861354 learned_addr learned my addr 172.21.15.178:0/1985861354 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 10 monclient(hunting): get_auth_request con 0x7fb1840a2930 auth_method 0 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 10 monclient(hunting): handle_auth_done global_id 45004 payload 274 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 1 -- 172.21.15.178:0/1985861354 >> v2:172.21.15.204:3300/0 conn(0x7fb18409fe70 msgr2=0x7fb1840a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb183fff640 1 --2- 172.21.15.178:0/1985861354 >> v2:172.21.15.204:3300/0 conn(0x7fb18409fe70 0x7fb1840a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 1 --2- 172.21.15.178:0/1985861354 >> v2:172.21.15.204:3300/0 conn(0x7fb18409fe70 0x7fb1840a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 1 -- 172.21.15.178:0/1985861354 >> v1:172.21.15.178:6789/0 conn(0x7fb18409faa0 legacy=0x7fb1840a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 10 monclient: _finish_hunting 0 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 1 monclient: found mon.noname-c 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.760+0000 7fb1837fe640 1 -- 172.21.15.178:0/1985861354 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1840a0310 con 0x7fb1840a2930 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1837fe640 1 --2- 172.21.15.178:0/1985861354 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb1840a2930 0x7fb184132de0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb174009850 tx=0x7fb1740090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8e25bf9567f53f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1827fc640 1 -- 172.21.15.178:0/1985861354 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb17402ac80 con 0x7fb1840a2930 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1827fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1827fc640 10 monclient: dump: 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.776 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.777 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.777 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.777 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.777 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.777 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.777 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.777 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.777 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.777 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.777 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.777 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.777 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1827fc640 10 monclient: _finish_auth 0 2024-09-15T01:49:58.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:58.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb189c78640 5 monclient: authenticate success, global_id 45004 2024-09-15T01:49:58.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb189c78640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:58.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1827fc640 1 -- 172.21.15.178:0/1985861354 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17402ade0 con 0x7fb1840a2930 2024-09-15T01:49:58.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1827fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:58.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1827fc640 1 -- 172.21.15.178:0/1985861354 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb174031820 con 0x7fb1840a2930 2024-09-15T01:49:58.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1827fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:49:58.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb1827fc640 10 monclient: dump: 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.779 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb189c78640 10 monclient: get_monmap_and_config success 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.761+0000 7fb189c78640 4 set_mon_vals no callback set 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.762+0000 7fb189c78640 10 monclient: shutdown 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.762+0000 7fb189c78640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.762+0000 7fb189c78640 1 -- 172.21.15.178:0/1985861354 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb1840a2930 msgr2=0x7fb184132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.762+0000 7fb189c78640 1 --2- 172.21.15.178:0/1985861354 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb1840a2930 0x7fb184132de0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb174009850 tx=0x7fb1740090f0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.762+0000 7fb189c78640 1 -- 172.21.15.178:0/1985861354 shutdown_connections 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.762+0000 7fb189c78640 1 --2- 172.21.15.178:0/1985861354 >> v2:172.21.15.204:3300/0 conn(0x7fb18409fe70 0x7fb1840a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.762+0000 7fb189c78640 1 --2- 172.21.15.178:0/1985861354 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb1840a2930 0x7fb184132de0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.762+0000 7fb189c78640 1 -- 172.21.15.178:0/1985861354 >> 172.21.15.178:0/1985861354 conn(0x7fb1840a5ed0 msgr2=0x7fb1840a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.762+0000 7fb189c78640 1 -- 172.21.15.178:0/1985861354 shutdown_connections 2024-09-15T01:49:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.762+0000 7fb189c78640 1 -- 172.21.15.178:0/1985861354 wait complete. 2024-09-15T01:49:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.763+0000 7fb189c78640 10 monclient: build_initial_monmap 2024-09-15T01:49:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.763+0000 7fb189c78640 10 monclient: monmap: 2024-09-15T01:49:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:49:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:49:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:49:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:49:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:49:58.781 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.764+0000 7fb189c78640 1 Processor -- start 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.764+0000 7fb189c78640 1 -- start start 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.764+0000 7fb189c78640 10 monclient: init 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.764+0000 7fb189c78640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.764+0000 7fb189c78640 10 monclient: _add_conns ranks=[5,7,3,1,4,6,8,2,0] 2024-09-15T01:49:58.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.764+0000 7fb189c78640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb18409faa0 0x7fb18409b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 10 monclient(hunting): picked mon.noname-h con 0x7fb18409faa0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:49:58.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb18409fe70 0x7fb184131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 10 monclient(hunting): picked mon.noname-i con 0x7fb18409fe70 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:49:58.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1840a2930 0x7fb184142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 10 monclient(hunting): picked mon.noname-g con 0x7fb1840a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:49:58.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18409f870 con 0x7fb1840a2930 2024-09-15T01:49:58.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb18411ed20 con 0x7fb18409faa0 2024-09-15T01:49:58.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb18411ea40 con 0x7fb18409fe70 2024-09-15T01:49:58.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb189c78640 10 monclient(hunting): authenticate will time out at 1803.515069s 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1840a2930 0x7fb184142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb182ffd640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb18409fe70 0x7fb184131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1840a2930 0x7fb184142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.178:38932/0 (socket says 172.21.15.178:38932) 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 1 -- 172.21.15.178:0/4058218185 learned_addr learned my addr 172.21.15.178:0/4058218185 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 10 monclient(hunting): get_auth_request con 0x7fb1840a2930 auth_method 0 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb182ffd640 10 monclient(hunting): get_auth_request con 0x7fb18409fe70 auth_method 0 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb182ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb182ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb182ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb1837fe640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb18409faa0 0x7fb18409b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb1837fe640 10 monclient(hunting): get_auth_request con 0x7fb18409faa0 auth_method 0 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb1837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb1837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb1837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb182ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb182ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb182ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb1837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb1837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb1837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 10 monclient(hunting): handle_auth_done global_id 64623 payload 824 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 1 -- 172.21.15.178:0/4058218185 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb18409fe70 msgr2=0x7fb184131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb18409fe70 0x7fb184131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 1 -- 172.21.15.178:0/4058218185 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb18409faa0 msgr2=0x7fb18409b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb18409faa0 0x7fb18409b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 10 monclient: _finish_hunting 0 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 1 monclient: found mon.noname-g 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.204:3301/0 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 1 -- 172.21.15.178:0/4058218185 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb174009390 con 0x7fb1840a2930 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb1837fe640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb18409faa0 0x7fb18409b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:58.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb182ffd640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fb18409fe70 0x7fb184131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb183fff640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1840a2930 0x7fb184142050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00ceb0 tx=0x7fb17c007870 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.765+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb17c0079b0 con 0x7fb1840a2930 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb16b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb16b7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb16b7fe640 10 monclient: dump: 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.787 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb16b7fe640 10 monclient: _finish_auth 0 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb16b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb189c78640 5 monclient: authenticate success, global_id 64623 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 <== mon.3 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c0168a0 con 0x7fb1840a2930 2024-09-15T01:49:58.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb189c78640 10 monclient: _renew_subs 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb189c78640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb189c78640 1 -- 172.21.15.178:0/4058218185 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18411d5e0 con 0x7fb1840a2930 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb16b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 <== mon.3 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb17c00e400 con 0x7fb1840a2930 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb16b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb16b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb16b7fe640 10 monclient: dump: 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.789 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb181ffb640 4 set_mon_vals no callback set 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb189c78640 10 monclient: _renew_subs 2024-09-15T01:49:58.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb189c78640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.204:3301/0 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb189c78640 1 -- 172.21.15.178:0/4058218185 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18411cbe0 con 0x7fb1840a2930 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 <== mon.3 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb17c003020 con 0x7fb1840a2930 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb189c78640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb189c78640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb18411db50 0x7fb184142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb189c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.766+0000 7fb189c78640 1 -- 172.21.15.178:0/4058218185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18404c4a0 con 0x7fb18411db50 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb1837fe640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb18411db50 0x7fb184142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 1 --2- 172.21.15.178:0/4058218185 >> v2:172.21.15.204:6812/28210 conn(0x7fb1700396e0 0x7fb17003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb1837fe640 10 monclient: get_auth_request con 0x7fb18411db50 auth_method 0 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb1837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb1837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb1837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 <== mon.3 v2:172.21.15.204:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fb17c012070 con 0x7fb1840a2930 2024-09-15T01:49:58.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb1837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb1837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb1837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb182ffd640 1 --2- 172.21.15.178:0/4058218185 >> v2:172.21.15.204:6812/28210 conn(0x7fb1700396e0 0x7fb17003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb182ffd640 10 monclient: get_auth_request con 0x7fb1700396e0 auth_method 0 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb1837fe640 10 monclient(hunting): handle_auth_done global_id 104103 payload 824 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb1837fe640 1 -- 172.21.15.178:0/4058218185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb18405ed70 con 0x7fb18411db50 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb1837fe640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb18411db50 0x7fb184142590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb184067590 tx=0x7fb174032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb17402ac80 con 0x7fb18411db50 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 10 monclient: dump: 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.792 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.793 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.793 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.793 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.793 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.793 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.793 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.793 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.793 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.793 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.793 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.793 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.793 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.794 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.794 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb182ffd640 1 --2- 172.21.15.178:0/4058218185 >> v2:172.21.15.204:6812/28210 conn(0x7fb1700396e0 0x7fb17003bb90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb1840a6ec0 tx=0x7fb178000bc0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fb174031e10 con 0x7fb18411db50 2024-09-15T01:49:58.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:58.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:49:58.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb18411db50 msgr2=0x7fb184142590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb18411db50 0x7fb184142590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb184067590 tx=0x7fb174032000 comp rx=0 tx=0).stop 2024-09-15T01:49:58.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb18411db50 msgr2=0x7fb184142590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:58.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.767+0000 7fb16b7fe640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb18411db50 0x7fb184142590 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb189c78640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:49:58.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb189c78640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb184131e60 0x7fb184142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb189c78640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb189c78640 1 -- 172.21.15.178:0/4058218185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18404bc80 con 0x7fb184131e60 2024-09-15T01:49:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb1837fe640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb184131e60 0x7fb184142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb1837fe640 10 monclient: get_auth_request con 0x7fb184131e60 auth_method 0 2024-09-15T01:49:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb1837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb1837fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb1837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb1837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb1837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb1837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb1837fe640 10 monclient(hunting): handle_auth_done global_id 104112 payload 824 2024-09-15T01:49:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.773+0000 7fb1837fe640 1 -- 172.21.15.178:0/4058218185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb18409d6a0 con 0x7fb184131e60 2024-09-15T01:49:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb1837fe640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb184131e60 0x7fb184142ad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb174009850 tx=0x7fb17403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb17404c070 con 0x7fb184131e60 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb16b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb16b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb16b7fe640 10 monclient: dump: 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.798 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 <== mon.0 v2:172.21.15.178:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fb17403c7f0 con 0x7fb184131e60 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb16b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb16b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb184131e60 msgr2=0x7fb184142ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb16b7fe640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb184131e60 0x7fb184142ad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb174009850 tx=0x7fb17403d000 comp rx=0 tx=0).stop 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb16b7fe640 1 -- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb184131e60 msgr2=0x7fb184142ad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:58.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb16b7fe640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb184131e60 0x7fb184142ad0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb174009850 tx=0x7fb17403d000 comp rx=0 tx=0).stop 2024-09-15T01:49:58.800 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:49:58.800 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:49:58.800 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:49:58.800 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:49:58.800 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 158, 2024-09-15T01:49:58.800 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:49:58.800 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:49:58.800 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:49:58.800 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:49:58.800 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 0, 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:58.801 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:58.802 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:49:58.803 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:58.804 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.805 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.806 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.807 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.808 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.809 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.810 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.811 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.812 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:49:58.813 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:49:58.813 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.813 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.813 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.813 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:49:58.813 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.813 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.813 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:49:58.813 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.814 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.815 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:49:58.816 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:49:58.816 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.816 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.816 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.816 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.816 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:49:58.816 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:49:58.816 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.816 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.816 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.816 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.817 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.818 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.819 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.820 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.821 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:58.822 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:58.822 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:58.822 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.822 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:58.822 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:49:58.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.822 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:58.822 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:58.822 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:58.822 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.822 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.823 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:49:58.823 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:49:58.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb189c78640 1 -- 172.21.15.178:0/4058218185 >> v2:172.21.15.204:6812/28210 conn(0x7fb1700396e0 msgr2=0x7fb17003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb189c78640 1 --2- 172.21.15.178:0/4058218185 >> v2:172.21.15.204:6812/28210 conn(0x7fb1700396e0 0x7fb17003bb90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb1840a6ec0 tx=0x7fb178000bc0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb189c78640 10 monclient: shutdown 2024-09-15T01:49:58.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb189c78640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:58.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb189c78640 1 -- 172.21.15.178:0/4058218185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1840a2930 msgr2=0x7fb184142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb189c78640 1 --2- 172.21.15.178:0/4058218185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1840a2930 0x7fb184142050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00ceb0 tx=0x7fb17c007870 comp rx=0 tx=0).stop 2024-09-15T01:49:58.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb183fff640 1 -- 172.21.15.178:0/4058218185 reap_dead start 2024-09-15T01:49:58.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb183fff640 1 -- 172.21.15.178:0/4058218185 reap_dead start 2024-09-15T01:49:58.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb189c78640 1 -- 172.21.15.178:0/4058218185 shutdown_connections 2024-09-15T01:49:58.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb189c78640 1 -- 172.21.15.178:0/4058218185 >> 172.21.15.178:0/4058218185 conn(0x7fb1840a5ed0 msgr2=0x7fb1840a4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:58.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb189c78640 1 -- 172.21.15.178:0/4058218185 shutdown_connections 2024-09-15T01:49:58.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.774+0000 7fb189c78640 1 -- 172.21.15.178:0/4058218185 wait complete. 2024-09-15T01:49:58.825 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:49:58.825 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T01:49:58.825 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T01:49:58.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.918+0000 7f96daf1d640 10 monclient: get_monmap_and_config 2024-09-15T01:49:58.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.918+0000 7f96daf1d640 10 monclient: build_initial_monmap 2024-09-15T01:49:58.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.918+0000 7f96daf1d640 10 monclient: monmap: 2024-09-15T01:49:58.920 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:58.920 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:49:58.919140+0000 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:49:58.919140+0000 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:58.921 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:58.922 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:49:58.922 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:49:58.922 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.918+0000 7f96daf1d640 1 Processor -- start 2024-09-15T01:49:58.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 1 -- start start 2024-09-15T01:49:58.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 10 monclient: init 2024-09-15T01:49:58.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:58.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 10 monclient: _add_conns ranks=[3,9,8,4,2,0,10,1,6,7,5] 2024-09-15T01:49:58.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40678d0 0x7f96d409e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 10 monclient(hunting): picked mon.noname-g con 0x7f96d40678d0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f96d406ea30 0x7f96d409ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 10 monclient(hunting): picked mon.noname-a con 0x7f96d406ea30 addr v2:172.21.15.178:3300/0 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 0x7f96d41306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 10 monclient(hunting): picked mon.noname-e con 0x7f96d409f180 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d409f5c0 con 0x7f96d406ea30 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d406d5b0 con 0x7f96d40678d0 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f96d4067ca0 con 0x7f96d409f180 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96daf1d640 10 monclient(hunting): authenticate will time out at 1803.669545s 2024-09-15T01:49:58.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d3fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f96d406ea30 0x7f96d409ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 0x7f96d41306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d3fff640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f96d406ea30 0x7f96d409ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35052/0 (socket says 172.21.15.178:35052) 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d3fff640 1 -- 172.21.15.178:0/3517531998 learned_addr learned my addr 172.21.15.178:0/3517531998 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 0x7f96d41306b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:39974/0 (socket says 172.21.15.178:39974) 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d3fff640 10 monclient(hunting): get_auth_request con 0x7f96d406ea30 auth_method 0 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 10 monclient(hunting): get_auth_request con 0x7f96d409f180 auth_method 0 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d8cb9640 1 --2- 172.21.15.178:0/3517531998 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40678d0 0x7f96d409e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d8cb9640 10 monclient(hunting): get_auth_request con 0x7f96d40678d0 auth_method 0 2024-09-15T01:49:58.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d8cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d8cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d8cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 10 monclient(hunting): handle_auth_done global_id 94409 payload 274 2024-09-15T01:49:58.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 1 -- 172.21.15.178:0/3517531998 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40678d0 msgr2=0x7f96d409e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 1 --2- 172.21.15.178:0/3517531998 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40678d0 0x7f96d409e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 1 -- 172.21.15.178:0/3517531998 >> v2:172.21.15.178:3300/0 conn(0x7f96d406ea30 msgr2=0x7f96d409ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 1 --2- 172.21.15.178:0/3517531998 >> v2:172.21.15.178:3300/0 conn(0x7f96d406ea30 0x7f96d409ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 10 monclient: _finish_hunting 0 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 1 monclient: found mon.noname-e 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d94ba640 1 -- 172.21.15.178:0/3517531998 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d404bf60 con 0x7f96d409f180 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.919+0000 7f96d3fff640 1 --2- 172.21.15.178:0/3517531998 >> v2:172.21.15.178:3300/0 conn(0x7f96d406ea30 0x7f96d409ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d94ba640 1 --2- 172.21.15.178:0/3517531998 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 0x7f96d41306b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009e90 tx=0x7f96cc009780 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c24e89545afd0614 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d37fe640 1 -- 172.21.15.178:0/3517531998 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96cc031880 con 0x7f96d409f180 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:49:58.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d37fe640 10 monclient: dump: 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.936 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d37fe640 10 monclient: _finish_auth 0 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d37fe640 1 -- 172.21.15.178:0/3517531998 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc02d050 con 0x7f96d409f180 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96daf1d640 5 monclient: authenticate success, global_id 94409 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96daf1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96daf1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d37fe640 1 -- 172.21.15.178:0/3517531998 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96cc0303a0 con 0x7f96d409f180 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96daf1d640 4 set_mon_vals no callback set 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.920+0000 7f96d37fe640 10 monclient: dump: 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.938 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 10 monclient: shutdown 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 1 -- 172.21.15.178:0/3517531998 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 msgr2=0x7f96d41306b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 1 --2- 172.21.15.178:0/3517531998 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 0x7f96d41306b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009e90 tx=0x7f96cc009780 comp rx=0 tx=0).stop 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 1 -- 172.21.15.178:0/3517531998 shutdown_connections 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 1 --2- 172.21.15.178:0/3517531998 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 0x7f96d41306b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 1 --2- 172.21.15.178:0/3517531998 >> v2:172.21.15.178:3300/0 conn(0x7f96d406ea30 0x7f96d409ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 1 --2- 172.21.15.178:0/3517531998 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40678d0 0x7f96d409e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 1 -- 172.21.15.178:0/3517531998 >> 172.21.15.178:0/3517531998 conn(0x7f96d40a4390 msgr2=0x7f96d4067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 1 -- 172.21.15.178:0/3517531998 shutdown_connections 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 1 -- 172.21.15.178:0/3517531998 wait complete. 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 10 monclient: build_initial_monmap 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.921+0000 7f96daf1d640 10 monclient: monmap: 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:49:58.940 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:58.941 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:58.941 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:58.941 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:58.941 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:58.941 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:58.941 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:58.941 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 1 Processor -- start 2024-09-15T01:49:58.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 1 -- start start 2024-09-15T01:49:58.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 10 monclient: init 2024-09-15T01:49:58.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:58.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 10 monclient: _add_conns ranks=[2,4,8,6,5,7,3,1,0] 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d40678d0 0x7f96d41335d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 10 monclient(hunting): picked mon.noname-b con 0x7f96d40678d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f96d406ea30 0x7f96d411d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 10 monclient(hunting): picked mon.noname-c con 0x7f96d406ea30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 0x7f96d411d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 10 monclient(hunting): picked mon.noname-e con 0x7f96d409f180 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d411f1b0 con 0x7f96d40678d0 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96d406c900 con 0x7f96d406ea30 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f96d41405d0 con 0x7f96d409f180 2024-09-15T01:49:58.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96daf1d640 10 monclient(hunting): authenticate will time out at 1803.672896s 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.922+0000 7f96d94ba640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 0x7f96d411d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f96d406ea30 0x7f96d411d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d8cb9640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d40678d0 0x7f96d41335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d94ba640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 0x7f96d411d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:39990/0 (socket says 172.21.15.178:39990) 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d94ba640 1 -- 172.21.15.178:0/791284227 learned_addr learned my addr 172.21.15.178:0/791284227 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d94ba640 10 monclient(hunting): get_auth_request con 0x7f96d409f180 auth_method 0 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d94ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d94ba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d94ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 10 monclient(hunting): get_auth_request con 0x7f96d406ea30 auth_method 0 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d8cb9640 10 monclient(hunting): get_auth_request con 0x7f96d40678d0 auth_method 0 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d8cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d8cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d8cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d94ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d94ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d94ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d8cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d8cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d8cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 10 monclient(hunting): handle_auth_done global_id 45013 payload 824 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 1 -- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 msgr2=0x7f96d411d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 0x7f96d411d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 1 -- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d40678d0 msgr2=0x7f96d41335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d40678d0 0x7f96d41335d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 1 monclient: found mon.noname-c 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 1 -- 172.21.15.178:0/791284227 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c0009450 con 0x7f96d406ea30 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d8cb9640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d40678d0 0x7f96d41335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:58.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d94ba640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f96d409f180 0x7f96d411d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d3fff640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f96d406ea30 0x7f96d411d0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f96c0010db0 tx=0x7f96c00091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96c0015070 con 0x7f96d406ea30 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d0ff9640 10 monclient: dump: 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.946 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d0ff9640 10 monclient: _finish_auth 0 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c0009680 con 0x7f96d406ea30 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96daf1d640 5 monclient: authenticate success, global_id 45013 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96daf1d640 10 monclient: _renew_subs 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96daf1d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96daf1d640 1 -- 172.21.15.178:0/791284227 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc009a20 con 0x7f96d406ea30 2024-09-15T01:49:58.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96c000bdc0 con 0x7f96d406ea30 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d0ff9640 10 monclient: dump: 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.948 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96d2ffd640 4 set_mon_vals no callback set 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96daf1d640 10 monclient: _renew_subs 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96daf1d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.923+0000 7f96daf1d640 1 -- 172.21.15.178:0/791284227 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d411f800 con 0x7f96d406ea30 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96daf1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96daf1d640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d4133b10 0x7f96d411db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96daf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96daf1d640 1 -- 172.21.15.178:0/791284227 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d4133ee0 con 0x7f96d4133b10 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f96c00183b0 con 0x7f96d406ea30 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d8cb9640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d4133b10 0x7f96d411db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d8cb9640 10 monclient: get_auth_request con 0x7f96d4133b10 auth_method 0 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d8cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d8cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d8cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d8cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d8cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d8cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d0ff9640 1 --2- 172.21.15.178:0/791284227 >> v2:172.21.15.204:6812/28210 conn(0x7f96b80397b0 0x7f96b803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d8cb9640 10 monclient(hunting): handle_auth_done global_id 35435 payload 824 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f96c004f410 con 0x7f96d406ea30 2024-09-15T01:49:58.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.924+0000 7f96d8cb9640 1 -- 172.21.15.178:0/791284227 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f96d404c4a0 con 0x7f96d4133b10 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d94ba640 1 --2- 172.21.15.178:0/791284227 >> v2:172.21.15.204:6812/28210 conn(0x7f96b80397b0 0x7f96b803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d8cb9640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d4133b10 0x7f96d411db20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f96c8010cb0 tx=0x7f96c8008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d94ba640 10 monclient: get_auth_request con 0x7f96b80397b0 auth_method 0 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96c80148b0 con 0x7f96d4133b10 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d0ff9640 10 monclient: dump: 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.951 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.952 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.952 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.952 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.952 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.952 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.952 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.952 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.952 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.952 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.953 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f96c8014430 con 0x7f96d4133b10 2024-09-15T01:49:58.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:58.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:49:58.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d4133b10 msgr2=0x7f96d411db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d0ff9640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d4133b10 0x7f96d411db20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f96c8010cb0 tx=0x7f96c8008040 comp rx=0 tx=0).stop 2024-09-15T01:49:58.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d4133b10 msgr2=0x7f96d411db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:58.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d0ff9640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d4133b10 0x7f96d411db20 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f96c8010cb0 tx=0x7f96c8008040 comp rx=0 tx=0).stop 2024-09-15T01:49:58.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.925+0000 7f96d94ba640 1 --2- 172.21.15.178:0/791284227 >> v2:172.21.15.204:6812/28210 conn(0x7f96b80397b0 0x7f96b803bc60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f96d411cff0 tx=0x7f96cc037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96daf1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:49:58.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96daf1d640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d406a250 0x7f96d411e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:58.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96daf1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:58.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96daf1d640 1 -- 172.21.15.178:0/791284227 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d404bc80 con 0x7f96d406a250 2024-09-15T01:49:58.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d8cb9640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d406a250 0x7f96d411e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:58.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d8cb9640 10 monclient: get_auth_request con 0x7f96d406a250 auth_method 0 2024-09-15T01:49:58.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d8cb9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:58.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d8cb9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:58.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d8cb9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:58.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d8cb9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:58.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d8cb9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:58.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d8cb9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:58.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d8cb9640 10 monclient(hunting): handle_auth_done global_id 35444 payload 824 2024-09-15T01:49:58.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d8cb9640 1 -- 172.21.15.178:0/791284227 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f96d405ed70 con 0x7f96d406a250 2024-09-15T01:49:58.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d8cb9640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d406a250 0x7f96d411e060 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f96c8000c00 tx=0x7f96c801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:58.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96c802b070 con 0x7f96d406a250 2024-09-15T01:49:58.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:58.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d0ff9640 10 monclient: dump: 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:58.957 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:58.958 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:58.958 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:58.958 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:58.958 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:58.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 <== mon.2 v2:172.21.15.178:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9312 (secure 0 0 0) 0x7f96c8023040 con 0x7f96d406a250 2024-09-15T01:49:58.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:49:58.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:49:58.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d406a250 msgr2=0x7f96d411e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d0ff9640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d406a250 0x7f96d411e060 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f96c8000c00 tx=0x7f96c801d000 comp rx=0 tx=0).stop 2024-09-15T01:49:58.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d0ff9640 1 -- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d406a250 msgr2=0x7f96d411e060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:58.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.931+0000 7f96d0ff9640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96d406a250 0x7f96d411e060 secure :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f96c8000c00 tx=0x7f96c801d000 comp rx=0 tx=0).stop 2024-09-15T01:49:58.959 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:49:58.959 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:49:58.959 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:49:58.959 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:49:58.959 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 158, 2024-09-15T01:49:58.959 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:49:58.959 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:49:58.959 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:49:58.959 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:49:58.959 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:49:58.959 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:58.960 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:58.961 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:58.961 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:58.961 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:58.961 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:58.961 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:58.961 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:58.961 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:49:58.961 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:49:58.961 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:58.961 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:49:58.962 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:58.963 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.964 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.965 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:49:58.966 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:49:58.968 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:49:58.968 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.968 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.968 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.968 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:49:58.968 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:49:58.968 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.969 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.969 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:49:58.969 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.969 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.969 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.969 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:49:58.969 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.969 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.969 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.970 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.970 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:49:58.970 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:49:58.970 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.970 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.970 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.970 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.970 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.970 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:49:58.970 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:49:58.970 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.970 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.971 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:49:58.972 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:49:58.972 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.972 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.972 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.972 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.972 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:49:58.972 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:49:58.972 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.972 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.972 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.972 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:49:58.973 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.976 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:49:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:49:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:49:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.977 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:49:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:49:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:49:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.978 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:49:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:49:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:49:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.979 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.980 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.981 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:58.982 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:49:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:58.983 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:49:58.983 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:49:58.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.932+0000 7f96daf1d640 1 -- 172.21.15.178:0/791284227 >> v2:172.21.15.204:6812/28210 conn(0x7f96b80397b0 msgr2=0x7f96b803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.932+0000 7f96daf1d640 1 --2- 172.21.15.178:0/791284227 >> v2:172.21.15.204:6812/28210 conn(0x7f96b80397b0 0x7f96b803bc60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f96d411cff0 tx=0x7f96cc037000 comp rx=0 tx=0).stop 2024-09-15T01:49:58.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.932+0000 7f96daf1d640 10 monclient: shutdown 2024-09-15T01:49:58.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.932+0000 7f96daf1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:58.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.932+0000 7f96daf1d640 1 -- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f96d406ea30 msgr2=0x7f96d411d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:58.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.932+0000 7f96daf1d640 1 --2- 172.21.15.178:0/791284227 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f96d406ea30 0x7f96d411d0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f96c0010db0 tx=0x7f96c00091d0 comp rx=0 tx=0).stop 2024-09-15T01:49:58.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.932+0000 7f96d94ba640 1 -- 172.21.15.178:0/791284227 reap_dead start 2024-09-15T01:49:58.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.932+0000 7f96d94ba640 1 -- 172.21.15.178:0/791284227 reap_dead start 2024-09-15T01:49:58.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.932+0000 7f96daf1d640 1 -- 172.21.15.178:0/791284227 shutdown_connections 2024-09-15T01:49:58.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.932+0000 7f96daf1d640 1 -- 172.21.15.178:0/791284227 >> 172.21.15.178:0/791284227 conn(0x7f96d40a4390 msgr2=0x7f96d409c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:58.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.932+0000 7f96daf1d640 1 -- 172.21.15.178:0/791284227 shutdown_connections 2024-09-15T01:49:58.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:58.932+0000 7f96daf1d640 1 -- 172.21.15.178:0/791284227 wait complete. 2024-09-15T01:49:58.986 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:49:58.986 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T01:49:58.986 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T01:49:59.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.078+0000 7fbfc826f640 10 monclient: get_monmap_and_config 2024-09-15T01:49:59.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.078+0000 7fbfc826f640 10 monclient: build_initial_monmap 2024-09-15T01:49:59.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.078+0000 7fbfc826f640 10 monclient: monmap: 2024-09-15T01:49:59.080 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:59.080 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.080 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:49:59.079063+0000 2024-09-15T01:49:59.080 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:49:59.079063+0000 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:49:59.081 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:49:59.082 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.078+0000 7fbfc826f640 1 Processor -- start 2024-09-15T01:49:59.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.078+0000 7fbfc826f640 1 -- start start 2024-09-15T01:49:59.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.078+0000 7fbfc826f640 10 monclient: init 2024-09-15T01:49:59.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc826f640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc826f640 10 monclient: _add_conns ranks=[7,1,0,9,6,4,5,8,2,3,10] 2024-09-15T01:49:59.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc826f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfc00a2930 0x7fbfc0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc826f640 10 monclient(hunting): picked mon.noname-i con 0x7fbfc00a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:49:59.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc826f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbfc009faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:49:59.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc826f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbfc009fe70 addr v1:172.21.15.178:6789/0 2024-09-15T01:49:59.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc826f640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc004bf60 con 0x7fbfc009fe70 2024-09-15T01:49:59.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc826f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc009dd80 con 0x7fbfc009faa0 2024-09-15T01:49:59.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc826f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc826f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00a2d00 con 0x7fbfc00a2930 2024-09-15T01:49:59.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc826f640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc826f640 10 monclient(hunting): authenticate will time out at 1803.829510s 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc680c640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fbfc009fe70 0x7fbfc00a0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:37008/0 (socket says 172.21.15.178:37008) 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc680c640 1 -- 172.21.15.178:0/1241075244 learned_addr learned my addr 172.21.15.178:0/1241075244 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc600b640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfc00a2930 0x7fbfc0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc600b640 10 monclient(hunting): get_auth_request con 0x7fbfc00a2930 auth_method 0 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc600b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc600b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc600b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862881940 0 0) 0x7fbfc004bf60 con 0x7fbfc009fe70 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc5009640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc5009640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc5009640 10 monclient(hunting): my global_id is 104130 2024-09-15T01:49:59.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.079+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfac0034e0 con 0x7fbfc009fe70 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495932962 0 0) 0x7fbfc009dd80 con 0x7fbfc009faa0 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient(hunting): my global_id is 104104 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfc004bf60 con 0x7fbfc009faa0 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2109220181 0 0) 0x7fbfac0034e0 con 0x7fbfc009fe70 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfc00a2930 msgr2=0x7fbfc0132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfc00a2930 0x7fbfc0132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 >> v1:172.21.15.204:6789/0 conn(0x7fbfc009faa0 legacy=0x7fbfc00a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc00a16e0 con 0x7fbfc009fe70 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7fbfb8001c70 con 0x7fbfc009fe70 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc826f640 5 monclient: authenticate success, global_id 104130 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc826f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient: dump: 2024-09-15T01:49:59.100 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.101 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 >> v1:172.21.15.178:6789/0 conn(0x7fbfc009fe70 legacy=0x7fbfc00a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient: _add_conns ranks=[3,6,5,2,0,1,8,7,4] 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfac004780 0x7fbfac004b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient(hunting): picked mon.g con 0x7fbfac004780 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfac005090 0x7fbfac0099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient(hunting): picked mon.d con 0x7fbfac005090 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbfac009f00 0x7fbfac00c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient(hunting): picked mon.h con 0x7fbfac009f00 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:49:59.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00a2d00 con 0x7fbfac004780 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac00c9b0 con 0x7fbfac009f00 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac00cb20 con 0x7fbfac005090 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc5009640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc826f640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfac005090 0x7fbfac0099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc600b640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfac004780 0x7fbfac004b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc600b640 10 monclient(hunting): get_auth_request con 0x7fbfac004780 auth_method 0 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc680c640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbfac009f00 0x7fbfac00c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc600b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc600b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc600b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 10 monclient(hunting): get_auth_request con 0x7fbfac005090 auth_method 0 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc680c640 10 monclient(hunting): get_auth_request con 0x7fbfac009f00 auth_method 0 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc680c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc680c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc680c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:49:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc680c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc680c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc680c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 10 monclient(hunting): handle_auth_done global_id 104130 payload 293 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 1 -- 172.21.15.178:0/1241075244 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbfac009f00 msgr2=0x7fbfac00c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fbfac009f00 0x7fbfac00c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 1 -- 172.21.15.178:0/1241075244 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfac004780 msgr2=0x7fbfac004b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfac004780 0x7fbfac004b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 1 monclient: found mon.d 2024-09-15T01:49:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc680c640 1 -- 172.21.15.178:0/1241075244 reap_dead start 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 1 -- 172.21.15.178:0/1241075244 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac0034e0 con 0x7fbfac005090 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc580a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.080+0000 7fbfc600b640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfac004780 0x7fbfac004b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc580a640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfac005090 0x7fbfac0099c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00a6ec0 tx=0x7fbfb0007ca0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=78698a50d04b322c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfb002ae70 con 0x7fbfac005090 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc5009640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc5009640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc5009640 10 monclient: dump: 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.107 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc826f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb0031390 con 0x7fbfac005090 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc5009640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc826f640 10 monclient: get_monmap_and_config success 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc826f640 4 set_mon_vals no callback set 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc5009640 1 -- 172.21.15.178:0/1241075244 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfb00319c0 con 0x7fbfac005090 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc5009640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc5009640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.081+0000 7fbfc5009640 10 monclient: dump: 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.082+0000 7fbfc826f640 10 monclient: shutdown 2024-09-15T01:49:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.082+0000 7fbfc826f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.082+0000 7fbfc826f640 1 -- 172.21.15.178:0/1241075244 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfac005090 msgr2=0x7fbfac0099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.082+0000 7fbfc826f640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfac005090 0x7fbfac0099c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00a6ec0 tx=0x7fbfb0007ca0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.082+0000 7fbfc826f640 1 -- 172.21.15.178:0/1241075244 shutdown_connections 2024-09-15T01:49:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.082+0000 7fbfc826f640 1 --2- 172.21.15.178:0/1241075244 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fbfac005090 0x7fbfac0099c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.082+0000 7fbfc826f640 1 -- 172.21.15.178:0/1241075244 >> 172.21.15.178:0/1241075244 conn(0x7fbfc00a5ed0 msgr2=0x7fbfc00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.082+0000 7fbfc826f640 1 -- 172.21.15.178:0/1241075244 shutdown_connections 2024-09-15T01:49:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.082+0000 7fbfc826f640 1 -- 172.21.15.178:0/1241075244 wait complete. 2024-09-15T01:49:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.082+0000 7fbfc826f640 10 monclient: build_initial_monmap 2024-09-15T01:49:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.082+0000 7fbfc826f640 10 monclient: monmap: 2024-09-15T01:49:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:59.111 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 1 Processor -- start 2024-09-15T01:49:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 1 -- start start 2024-09-15T01:49:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 10 monclient: init 2024-09-15T01:49:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 10 monclient: _add_conns ranks=[7,2,3,5,6,8,4,1,0] 2024-09-15T01:49:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfac005090 0x7fbfc0141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 10 monclient(hunting): picked mon.noname-i con 0x7fbfac005090 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:49:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfc0142290 0x7fbfc0131b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 10 monclient(hunting): picked mon.noname-b con 0x7fbfc0142290 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:49:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfc0132070 0x7fbfc011e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 10 monclient(hunting): picked mon.noname-g con 0x7fbfc0132070 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:49:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01209f0 con 0x7fbfc0142290 2024-09-15T01:49:59.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc009d630 con 0x7fbfc0132070 2024-09-15T01:49:59.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc009f870 con 0x7fbfac005090 2024-09-15T01:49:59.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc826f640 10 monclient(hunting): authenticate will time out at 1803.833587s 2024-09-15T01:49:59.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc580a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfc0142290 0x7fbfc0131b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc600b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfac005090 0x7fbfc0141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc580a640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfc0142290 0x7fbfc0131b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39516/0 (socket says 172.21.15.178:39516) 2024-09-15T01:49:59.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc580a640 1 -- 172.21.15.178:0/3221730366 learned_addr learned my addr 172.21.15.178:0/3221730366 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:59.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc600b640 10 monclient(hunting): get_auth_request con 0x7fbfac005090 auth_method 0 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc600b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc600b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc600b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc580a640 10 monclient(hunting): get_auth_request con 0x7fbfc0142290 auth_method 0 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc580a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc580a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc580a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc680c640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfc0132070 0x7fbfc011e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc680c640 10 monclient(hunting): get_auth_request con 0x7fbfc0132070 auth_method 0 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc680c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc680c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc680c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc580a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc580a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc580a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc680c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc680c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.083+0000 7fbfc680c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc580a640 10 monclient(hunting): handle_auth_done global_id 35453 payload 824 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc580a640 1 -- 172.21.15.178:0/3221730366 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfac005090 msgr2=0x7fbfc0141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc580a640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfac005090 0x7fbfc0141d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc580a640 1 -- 172.21.15.178:0/3221730366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfc0132070 msgr2=0x7fbfc011e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc580a640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfc0132070 0x7fbfc011e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc580a640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc580a640 1 monclient: found mon.noname-b 2024-09-15T01:49:59.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc580a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc580a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:49:59.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc580a640 1 -- 172.21.15.178:0/3221730366 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc01324c0 con 0x7fbfc0142290 2024-09-15T01:49:59.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc600b640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fbfac005090 0x7fbfc0141d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:59.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc680c640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbfc0132070 0x7fbfc011e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:59.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc580a640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfc0142290 0x7fbfc0131b30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbfb002a6c0 tx=0x7fbfb0003000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfb0031b50 con 0x7fbfc0142290 2024-09-15T01:49:59.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfb67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfb67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:49:59.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfb67fc640 10 monclient: dump: 2024-09-15T01:49:59.116 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.117 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfb67fc640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfb67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb0031cf0 con 0x7fbfc0142290 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc826f640 5 monclient: authenticate success, global_id 35453 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc826f640 10 monclient: _renew_subs 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc826f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc826f640 1 -- 172.21.15.178:0/3221730366 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc01201c0 con 0x7fbfc0142290 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfb67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfb0031e60 con 0x7fbfc0142290 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfb67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfb67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfb67fc640 10 monclient: dump: 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.119 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc4808640 4 set_mon_vals no callback set 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc826f640 10 monclient: _renew_subs 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc826f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.084+0000 7fbfc826f640 1 -- 172.21.15.178:0/3221730366 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc0121f60 con 0x7fbfc0142290 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc826f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fbfb00316d0 con 0x7fbfc0142290 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc826f640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc00298c0 0x7fbfc0029c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc826f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc826f640 1 -- 172.21.15.178:0/3221730366 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc004c4a0 con 0x7fbfc00298c0 2024-09-15T01:49:59.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc600b640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc00298c0 0x7fbfc0029c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc600b640 10 monclient: get_auth_request con 0x7fbfc00298c0 auth_method 0 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc600b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc600b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc600b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfb67fc640 1 --2- 172.21.15.178:0/3221730366 >> v2:172.21.15.204:6812/28210 conn(0x7fbfa40397b0 0x7fbfa403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fbfb006ba50 con 0x7fbfc0142290 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc680c640 1 --2- 172.21.15.178:0/3221730366 >> v2:172.21.15.204:6812/28210 conn(0x7fbfa40397b0 0x7fbfa403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc600b640 10 monclient(hunting): handle_auth_done global_id 45022 payload 824 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc600b640 1 -- 172.21.15.178:0/3221730366 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfc005ed70 con 0x7fbfc00298c0 2024-09-15T01:49:59.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc680c640 10 monclient: get_auth_request con 0x7fbfa40397b0 auth_method 0 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.085+0000 7fbfc600b640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc00298c0 0x7fbfc0029c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00608f0 tx=0x7fbfbc0090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.086+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfbc01d8b0 con 0x7fbfc00298c0 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.086+0000 7fbfb67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.086+0000 7fbfb67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.086+0000 7fbfb67fc640 10 monclient: dump: 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.122 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.086+0000 7fbfc680c640 1 --2- 172.21.15.178:0/3221730366 >> v2:172.21.15.204:6812/28210 conn(0x7fbfa40397b0 0x7fbfa403bc60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00a0240 tx=0x7fbfb8001c00 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.086+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fbfbc01d430 con 0x7fbfc00298c0 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.086+0000 7fbfb67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.086+0000 7fbfb67fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.086+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc00298c0 msgr2=0x7fbfc0029c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.086+0000 7fbfb67fc640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc00298c0 0x7fbfc0029c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00608f0 tx=0x7fbfbc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.086+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc00298c0 msgr2=0x7fbfc0029c90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.086+0000 7fbfb67fc640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc00298c0 0x7fbfc0029c90 secure :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00608f0 tx=0x7fbfbc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.091+0000 7fbfc826f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.091+0000 7fbfc826f640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc002a1d0 0x7fbfc002a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.091+0000 7fbfc826f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfc826f640 1 -- 172.21.15.178:0/3221730366 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc004bc80 con 0x7fbfc002a1d0 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfc600b640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc002a1d0 0x7fbfc002a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfc600b640 10 monclient: get_auth_request con 0x7fbfc002a1d0 auth_method 0 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfc600b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfc600b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfc600b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfc600b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfc600b640 10 monclient(hunting): handle_auth_done global_id 45031 payload 824 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfc600b640 1 -- 172.21.15.178:0/3221730366 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbfc0122180 con 0x7fbfc002a1d0 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfc600b640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc002a1d0 0x7fbfc002a5d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc01e000 tx=0x7fbfbc01c000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfbc0067a0 con 0x7fbfc002a1d0 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfb67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfb67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfb67fc640 10 monclient: dump: 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 <== mon.4 v2:172.21.15.178:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9482 (secure 0 0 0) 0x7fbfbc006320 con 0x7fbfc002a1d0 2024-09-15T01:49:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfb67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:49:59.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfb67fc640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:49:59.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc002a1d0 msgr2=0x7fbfc002a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfb67fc640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc002a1d0 0x7fbfc002a5d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc01e000 tx=0x7fbfbc01c000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfb67fc640 1 -- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc002a1d0 msgr2=0x7fbfc002a5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:59.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.092+0000 7fbfb67fc640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fbfc002a1d0 0x7fbfc002a5d0 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc01e000 tx=0x7fbfbc01c000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.127 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:49:59.127 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:49:59.127 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:49:59.127 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:49:59.127 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 158, 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:49:59.128 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.129 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:49:59.130 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:49:59.131 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.132 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:49:59.135 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:49:59.136 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:49:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.139 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.140 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:49:59.141 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.143 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.144 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.145 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.145 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:49:59.145 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:49:59.145 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.145 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.145 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:49:59.145 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.146 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.147 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:49:59.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 3 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:49:59.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr": [ 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.151 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:49:59.151 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:49:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.093+0000 7fbfc826f640 1 -- 172.21.15.178:0/3221730366 >> v2:172.21.15.204:6812/28210 conn(0x7fbfa40397b0 msgr2=0x7fbfa403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.093+0000 7fbfc826f640 1 --2- 172.21.15.178:0/3221730366 >> v2:172.21.15.204:6812/28210 conn(0x7fbfa40397b0 0x7fbfa403bc60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00a0240 tx=0x7fbfb8001c00 comp rx=0 tx=0).stop 2024-09-15T01:49:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.093+0000 7fbfc826f640 10 monclient: shutdown 2024-09-15T01:49:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.093+0000 7fbfc826f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.093+0000 7fbfc826f640 1 -- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfc0142290 msgr2=0x7fbfc0131b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.093+0000 7fbfc826f640 1 --2- 172.21.15.178:0/3221730366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfc0142290 0x7fbfc0131b30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbfb002a6c0 tx=0x7fbfb0003000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.093+0000 7fbfc680c640 1 -- 172.21.15.178:0/3221730366 reap_dead start 2024-09-15T01:49:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.093+0000 7fbfc680c640 1 -- 172.21.15.178:0/3221730366 reap_dead start 2024-09-15T01:49:59.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.093+0000 7fbfc826f640 1 -- 172.21.15.178:0/3221730366 shutdown_connections 2024-09-15T01:49:59.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.093+0000 7fbfc826f640 1 -- 172.21.15.178:0/3221730366 >> 172.21.15.178:0/3221730366 conn(0x7fbfc00a5ed0 msgr2=0x7fbfc00a4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:59.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.093+0000 7fbfc826f640 1 -- 172.21.15.178:0/3221730366 shutdown_connections 2024-09-15T01:49:59.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.093+0000 7fbfc826f640 1 -- 172.21.15.178:0/3221730366 wait complete. 2024-09-15T01:49:59.153 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:49:59.153 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T01:49:59.153 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T01:49:59.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.245+0000 7fea5506f640 10 monclient: get_monmap_and_config 2024-09-15T01:49:59.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.245+0000 7fea5506f640 10 monclient: build_initial_monmap 2024-09-15T01:49:59.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.246+0000 7fea5506f640 10 monclient: monmap: 2024-09-15T01:49:59.248 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:59.248 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.248 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:49:59.246872+0000 2024-09-15T01:49:59.248 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:49:59.246872+0000 2024-09-15T01:49:59.248 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:59.248 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.248 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:49:59.249 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:49:59.249 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:59.249 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:59.249 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:59.249 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:59.249 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:59.249 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:59.249 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:59.249 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:49:59.249 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:49:59.249 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.246+0000 7fea5506f640 1 Processor -- start 2024-09-15T01:49:59.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.246+0000 7fea5506f640 1 -- start start 2024-09-15T01:49:59.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.246+0000 7fea5506f640 10 monclient: init 2024-09-15T01:49:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 10 monclient: _add_conns ranks=[10,7,8,5,6,0,3,4,2,9,1] 2024-09-15T01:49:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fea500a2930 0x7fea50132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 10 monclient(hunting): picked mon.noname-f con 0x7fea500a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:49:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea5009faa0 0x7fea5009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 10 monclient(hunting): picked mon.noname-i con 0x7fea5009faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:49:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea500a08c0 0x7fea500a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 10 monclient(hunting): picked mon.noname-e con 0x7fea500a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:49:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea5009e0f0 con 0x7fea500a2930 2024-09-15T01:49:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea5009dd80 con 0x7fea5009faa0 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fea500a2d00 con 0x7fea500a08c0 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea5506f640 10 monclient(hunting): authenticate will time out at 1803.997091s 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea500a08c0 0x7fea500a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea500a08c0 0x7fea500a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40006/0 (socket says 172.21.15.178:40006) 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 1 -- 172.21.15.178:0/3825961070 learned_addr learned my addr 172.21.15.178:0/3825961070 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 10 monclient(hunting): get_auth_request con 0x7fea500a08c0 auth_method 0 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4659c640 1 --2- 172.21.15.178:0/3825961070 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea5009faa0 0x7fea5009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4ed9d640 1 --2- 172.21.15.178:0/3825961070 >> v2:172.21.15.204:3300/0 conn(0x7fea500a2930 0x7fea50132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 10 monclient(hunting): handle_auth_done global_id 94427 payload 274 2024-09-15T01:49:59.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 1 -- 172.21.15.178:0/3825961070 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea5009faa0 msgr2=0x7fea5009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:59.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 1 --2- 172.21.15.178:0/3825961070 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea5009faa0 0x7fea5009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 1 -- 172.21.15.178:0/3825961070 >> v2:172.21.15.204:3300/0 conn(0x7fea500a2930 msgr2=0x7fea50132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:59.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 1 --2- 172.21.15.178:0/3825961070 >> v2:172.21.15.204:3300/0 conn(0x7fea500a2930 0x7fea50132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 1 monclient: found mon.noname-e 2024-09-15T01:49:59.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4f59e640 1 -- 172.21.15.178:0/3825961070 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5004bf60 con 0x7fea500a08c0 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4ed9d640 10 monclient: get_auth_request con 0x7fea500a2930 auth_method 0 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4ed9d640 1 --2- 172.21.15.178:0/3825961070 >> v2:172.21.15.204:3300/0 conn(0x7fea500a2930 0x7fea50132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4659c640 10 monclient: get_auth_request con 0x7fea5009faa0 auth_method 0 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.247+0000 7fea4659c640 1 --2- 172.21.15.178:0/3825961070 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea5009faa0 0x7fea5009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4f59e640 1 --2- 172.21.15.178:0/3825961070 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea500a08c0 0x7fea500a0d60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea48009f20 tx=0x7fea48009840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a50fe32fd7dbb0bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4e59c640 1 -- 172.21.15.178:0/3825961070 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea48007880 con 0x7fea500a08c0 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4e59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4e59c640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4e59c640 10 monclient: dump: 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.263 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.264 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4e59c640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4e59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4e59c640 1 -- 172.21.15.178:0/3825961070 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4802d050 con 0x7fea500a08c0 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 5 monclient: authenticate success, global_id 94427 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4e59c640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 10 monclient: get_monmap_and_config success 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4e59c640 1 -- 172.21.15.178:0/3825961070 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea48030360 con 0x7fea500a08c0 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4e59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 4 set_mon_vals no callback set 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4e59c640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea4e59c640 10 monclient: dump: 2024-09-15T01:49:59.265 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.266 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 10 monclient: shutdown 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 1 -- 172.21.15.178:0/3825961070 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea500a08c0 msgr2=0x7fea500a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 1 --2- 172.21.15.178:0/3825961070 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea500a08c0 0x7fea500a0d60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea48009f20 tx=0x7fea48009840 comp rx=0 tx=0).stop 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 1 -- 172.21.15.178:0/3825961070 shutdown_connections 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 1 --2- 172.21.15.178:0/3825961070 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fea500a08c0 0x7fea500a0d60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 1 --2- 172.21.15.178:0/3825961070 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea5009faa0 0x7fea5009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 1 --2- 172.21.15.178:0/3825961070 >> v2:172.21.15.204:3300/0 conn(0x7fea500a2930 0x7fea50132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 1 -- 172.21.15.178:0/3825961070 >> 172.21.15.178:0/3825961070 conn(0x7fea500a5ed0 msgr2=0x7fea500a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 1 -- 172.21.15.178:0/3825961070 shutdown_connections 2024-09-15T01:49:59.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.248+0000 7fea5506f640 1 -- 172.21.15.178:0/3825961070 wait complete. 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.249+0000 7fea5506f640 10 monclient: build_initial_monmap 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.249+0000 7fea5506f640 10 monclient: monmap: 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:59.268 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.249+0000 7fea5506f640 1 Processor -- start 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.249+0000 7fea5506f640 1 -- start start 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.249+0000 7fea5506f640 10 monclient: init 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 10 monclient: _add_conns ranks=[3,5,7,0,1,2,6,8,4] 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea5009faa0 0x7fea5009b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 10 monclient(hunting): picked mon.noname-g con 0x7fea5009faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:49:59.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea500a08c0 0x7fea50131990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 10 monclient(hunting): picked mon.noname-h con 0x7fea500a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea500a2930 0x7fea50141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 10 monclient(hunting): picked mon.noname-i con 0x7fea500a2930 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5009d630 con 0x7fea5009faa0 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea5011f100 con 0x7fea500a08c0 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea5011d470 con 0x7fea500a2930 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea5506f640 10 monclient(hunting): authenticate will time out at 1804.000387s 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea500a08c0 0x7fea50131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4ed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea5009faa0 0x7fea5009b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4f59e640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea500a2930 0x7fea50141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea500a08c0 0x7fea50131990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:44010/0 (socket says 172.21.15.178:44010) 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 1 -- 172.21.15.178:0/4205119805 learned_addr learned my addr 172.21.15.178:0/4205119805 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 10 monclient(hunting): get_auth_request con 0x7fea500a08c0 auth_method 0 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4ed9d640 10 monclient(hunting): get_auth_request con 0x7fea5009faa0 auth_method 0 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4f59e640 10 monclient(hunting): get_auth_request con 0x7fea500a2930 auth_method 0 2024-09-15T01:49:59.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4f59e640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 10 monclient(hunting): handle_auth_done global_id 7817 payload 824 2024-09-15T01:49:59.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 1 -- 172.21.15.178:0/4205119805 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea500a2930 msgr2=0x7fea50141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.250+0000 7fea4659c640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fea500a2930 0x7fea50141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea4659c640 1 -- 172.21.15.178:0/4205119805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea5009faa0 msgr2=0x7fea5009b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea4659c640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea5009faa0 0x7fea5009b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea4659c640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea4659c640 1 monclient: found mon.noname-h 2024-09-15T01:49:59.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea4659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea4659c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:49:59.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea4659c640 1 -- 172.21.15.178:0/4205119805 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea48009ae0 con 0x7fea500a08c0 2024-09-15T01:49:59.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea4ed9d640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea5009faa0 0x7fea5009b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:59.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea4659c640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea500a08c0 0x7fea50131990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fea3800c8b0 tx=0x7fea38008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea3800ce40 con 0x7fea500a08c0 2024-09-15T01:49:59.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea477fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea477fe640 10 monclient: dump: 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.274 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea477fe640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea38007cb0 con 0x7fea500a08c0 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea5506f640 5 monclient: authenticate success, global_id 7817 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea5506f640 10 monclient: _renew_subs 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea5506f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea5506f640 1 -- 172.21.15.178:0/4205119805 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5011eb10 con 0x7fea500a08c0 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea477fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea38007e20 con 0x7fea500a08c0 2024-09-15T01:49:59.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea477fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea477fe640 10 monclient: dump: 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.276 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea4dd9b640 4 set_mon_vals no callback set 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea5506f640 10 monclient: _renew_subs 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea5506f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.251+0000 7fea5506f640 1 -- 172.21.15.178:0/4205119805 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5011db50 con 0x7fea500a08c0 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea5506f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea5506f640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011eca0 0x7fea50142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea5506f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea5506f640 1 -- 172.21.15.178:0/4205119805 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea5004c4a0 con 0x7fea5011eca0 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4ed9d640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011eca0 0x7fea50142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4ed9d640 10 monclient: get_auth_request con 0x7fea5011eca0 auth_method 0 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fea38007830 con 0x7fea500a08c0 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4ed9d640 10 monclient(hunting): handle_auth_done global_id 25872 payload 824 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea477fe640 1 --2- 172.21.15.178:0/4205119805 >> v2:172.21.15.204:6812/28210 conn(0x7fea300396e0 0x7fea3003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4ed9d640 1 -- 172.21.15.178:0/4205119805 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea5005ed70 con 0x7fea5011eca0 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fea3804bc10 con 0x7fea500a08c0 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4ed9d640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011eca0 0x7fea50142310 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fea4000aeb0 tx=0x7fea40008040 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4f59e640 1 --2- 172.21.15.178:0/4205119805 >> v2:172.21.15.204:6812/28210 conn(0x7fea300396e0 0x7fea3003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea4f59e640 10 monclient: get_auth_request con 0x7fea300396e0 auth_method 0 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea400138c0 con 0x7fea5011eca0 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea477fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.252+0000 7fea477fe640 10 monclient: dump: 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.280 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.280 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.280 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.280 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.280 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.280 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.280 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.253+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fea4000c040 con 0x7fea5011eca0 2024-09-15T01:49:59.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.253+0000 7fea477fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:59.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.253+0000 7fea477fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:49:59.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.253+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011eca0 msgr2=0x7fea50142310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.253+0000 7fea477fe640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011eca0 0x7fea50142310 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fea4000aeb0 tx=0x7fea40008040 comp rx=0 tx=0).stop 2024-09-15T01:49:59.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.253+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011eca0 msgr2=0x7fea50142310 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:59.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.253+0000 7fea477fe640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011eca0 0x7fea50142310 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.253+0000 7fea4f59e640 1 --2- 172.21.15.178:0/4205119805 >> v2:172.21.15.204:6812/28210 conn(0x7fea300396e0 0x7fea3003bb90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fea48033000 tx=0x7fea48036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.258+0000 7fea5506f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:49:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.258+0000 7fea5506f640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011ca00 0x7fea50142850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.258+0000 7fea5506f640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.258+0000 7fea5506f640 1 -- 172.21.15.178:0/4205119805 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea5004bc80 con 0x7fea5011ca00 2024-09-15T01:49:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.258+0000 7fea4ed9d640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011ca00 0x7fea50142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea4ed9d640 10 monclient: get_auth_request con 0x7fea5011ca00 auth_method 0 2024-09-15T01:49:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea4ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea4ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea4ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea4ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea4ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea4ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea4ed9d640 10 monclient(hunting): handle_auth_done global_id 25881 payload 824 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea4ed9d640 1 -- 172.21.15.178:0/4205119805 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fea500a64b0 con 0x7fea5011ca00 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea4ed9d640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011ca00 0x7fea50142850 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fea4000aeb0 tx=0x7fea4001d000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea40012840 con 0x7fea5011ca00 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea477fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea477fe640 10 monclient: dump: 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.286 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 <== mon.6 v2:172.21.15.178:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fea400123c0 con 0x7fea5011ca00 2024-09-15T01:49:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea477fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:49:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea477fe640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:49:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011ca00 msgr2=0x7fea50142850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea477fe640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011ca00 0x7fea50142850 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fea4000aeb0 tx=0x7fea4001d000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea477fe640 1 -- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011ca00 msgr2=0x7fea50142850 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.259+0000 7fea477fe640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fea5011ca00 0x7fea50142850 secure :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fea4000aeb0 tx=0x7fea4001d000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.287 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:49:59.287 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:49:59.287 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:49:59.287 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:49:59.287 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 158, 2024-09-15T01:49:59.287 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:49:59.287 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:49:59.287 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:49:59.287 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:49:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:49:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:49:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:49:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:49:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:49:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:49:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:49:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:49:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:49:59.288 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.289 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.290 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.290 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.290 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.290 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.290 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.290 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.290 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.290 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.290 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:49:59.290 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:49:59.290 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:49:59.291 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:59.292 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:49:59.292 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.292 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.292 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.292 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.292 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.292 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.292 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.292 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.292 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.292 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.292 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:49:59.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.293 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:49:59.293 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.293 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:49:59.293 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:49:59.293 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.293 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.293 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.293 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.293 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:49:59.293 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.294 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.295 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:49:59.296 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:49:59.297 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.297 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:49:59.297 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.297 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.297 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.297 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:49:59.297 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.297 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:49:59.299 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.300 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.300 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.300 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.300 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:49:59.300 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:49:59.300 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.300 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.300 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.300 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.300 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:49:59.300 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.301 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.302 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.303 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.304 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.305 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:49:59.306 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.308 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.309 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.310 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:59.310 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.310 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.310 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:49:59.310 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:49:59.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.260+0000 7fea5506f640 1 -- 172.21.15.178:0/4205119805 >> v2:172.21.15.204:6812/28210 conn(0x7fea300396e0 msgr2=0x7fea3003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.260+0000 7fea5506f640 1 --2- 172.21.15.178:0/4205119805 >> v2:172.21.15.204:6812/28210 conn(0x7fea300396e0 0x7fea3003bb90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fea48033000 tx=0x7fea48036000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.260+0000 7fea5506f640 10 monclient: shutdown 2024-09-15T01:49:59.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.260+0000 7fea5506f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:59.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.260+0000 7fea5506f640 1 -- 172.21.15.178:0/4205119805 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea500a08c0 msgr2=0x7fea50131990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.260+0000 7fea5506f640 1 --2- 172.21.15.178:0/4205119805 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fea500a08c0 0x7fea50131990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fea3800c8b0 tx=0x7fea38008040 comp rx=0 tx=0).stop 2024-09-15T01:49:59.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.260+0000 7fea4f59e640 1 -- 172.21.15.178:0/4205119805 reap_dead start 2024-09-15T01:49:59.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.260+0000 7fea4f59e640 1 -- 172.21.15.178:0/4205119805 reap_dead start 2024-09-15T01:49:59.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.260+0000 7fea5506f640 1 -- 172.21.15.178:0/4205119805 shutdown_connections 2024-09-15T01:49:59.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.260+0000 7fea5506f640 1 -- 172.21.15.178:0/4205119805 >> 172.21.15.178:0/4205119805 conn(0x7fea500a5ed0 msgr2=0x7fea50135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:59.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.260+0000 7fea5506f640 1 -- 172.21.15.178:0/4205119805 shutdown_connections 2024-09-15T01:49:59.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.260+0000 7fea5506f640 1 -- 172.21.15.178:0/4205119805 wait complete. 2024-09-15T01:49:59.312 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:49:59.313 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T01:49:59.313 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T01:49:59.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.404+0000 7f0d8b51d640 10 monclient: get_monmap_and_config 2024-09-15T01:49:59.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.404+0000 7f0d8b51d640 10 monclient: build_initial_monmap 2024-09-15T01:49:59.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.404+0000 7f0d8b51d640 10 monclient: monmap: 2024-09-15T01:49:59.407 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:59.407 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.407 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:49:59.405573+0000 2024-09-15T01:49:59.407 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:49:59.405573+0000 2024-09-15T01:49:59.407 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:59.407 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.407 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:49:59.407 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:49:59.408 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:59.408 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:59.408 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:59.408 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:59.408 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:59.408 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:59.408 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:59.408 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:49:59.408 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:49:59.408 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 1 Processor -- start 2024-09-15T01:49:59.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 1 -- start start 2024-09-15T01:49:59.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 10 monclient: init 2024-09-15T01:49:59.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 10 monclient: _add_conns ranks=[0,3,9,4,1,5,2,8,10,6,7] 2024-09-15T01:49:59.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0d840a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:49:59.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d84132de0 0x7f0d84137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 10 monclient(hunting): picked mon.noname-g con 0x7f0d84132de0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:49:59.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0d841380e0 0x7f0d8413a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 10 monclient(hunting): picked mon.noname-a con 0x7f0d841380e0 addr v2:172.21.15.178:3300/0 2024-09-15T01:49:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8404bf60 con 0x7f0d840a2930 2024-09-15T01:49:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8409dd80 con 0x7f0d841380e0 2024-09-15T01:49:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840a2d00 con 0x7f0d84132de0 2024-09-15T01:49:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.405+0000 7f0d8b51d640 10 monclient(hunting): authenticate will time out at 1804.155937s 2024-09-15T01:49:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0d841380e0 0x7f0d8413a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d892b9640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f0d840a2930 0x7f0d8409faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:37016/0 (socket says 172.21.15.178:37016) 2024-09-15T01:49:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d892b9640 1 -- 172.21.15.178:0/2280208293 learned_addr learned my addr 172.21.15.178:0/2280208293 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:49:59.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 10 monclient(hunting): get_auth_request con 0x7f0d841380e0 auth_method 0 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d88ab8640 1 --2- 172.21.15.178:0/2280208293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d84132de0 0x7f0d84137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 10 monclient(hunting): handle_auth_done global_id 104139 payload 274 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 1 -- 172.21.15.178:0/2280208293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d84132de0 msgr2=0x7f0d84137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 1 --2- 172.21.15.178:0/2280208293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d84132de0 0x7f0d84137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 1 -- 172.21.15.178:0/2280208293 >> v1:172.21.15.178:6789/0 conn(0x7f0d840a2930 legacy=0x7f0d8409faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 1 monclient: found mon.noname-a 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d7bfff640 1 -- 172.21.15.178:0/2280208293 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021608131 0 0) 0x7f0d8404bf60 con 0x7f0d840a2930 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d89aba640 1 -- 172.21.15.178:0/2280208293 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d841331b0 con 0x7f0d841380e0 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d88ab8640 10 monclient: get_auth_request con 0x7f0d84132de0 auth_method 0 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d88ab8640 1 --2- 172.21.15.178:0/2280208293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d84132de0 0x7f0d84137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.406+0000 7f0d7bfff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d89aba640 1 --2- 172.21.15.178:0/2280208293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d841380e0 0x7f0d8413a4c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009e60 tx=0x7f0d7c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c343b952198815d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d7bfff640 1 -- 172.21.15.178:0/2280208293 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d7c0079d0 con 0x7f0d841380e0 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d7bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d7bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d7bfff640 10 monclient: dump: 2024-09-15T01:49:59.424 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d7bfff640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d7bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d7bfff640 1 -- 172.21.15.178:0/2280208293 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c02d050 con 0x7f0d841380e0 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 5 monclient: authenticate success, global_id 104139 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d7bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 10 monclient: get_monmap_and_config success 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d7bfff640 1 -- 172.21.15.178:0/2280208293 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d7c007b90 con 0x7f0d841380e0 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d7bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 4 set_mon_vals no callback set 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d7bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:49:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d7bfff640 10 monclient: dump: 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 10 monclient: shutdown 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2280208293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d841380e0 msgr2=0x7f0d8413a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 1 --2- 172.21.15.178:0/2280208293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d841380e0 0x7f0d8413a4c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009e60 tx=0x7f0d7c009780 comp rx=0 tx=0).stop 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2280208293 shutdown_connections 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 1 --2- 172.21.15.178:0/2280208293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d841380e0 0x7f0d8413a4c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 1 --2- 172.21.15.178:0/2280208293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d84132de0 0x7f0d84137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2280208293 >> 172.21.15.178:0/2280208293 conn(0x7f0d840a5ed0 msgr2=0x7f0d840a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2280208293 shutdown_connections 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.407+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2280208293 wait complete. 2024-09-15T01:49:59.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.408+0000 7f0d8b51d640 10 monclient: build_initial_monmap 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.408+0000 7f0d8b51d640 10 monclient: monmap: 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.408+0000 7f0d8b51d640 1 Processor -- start 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.408+0000 7f0d8b51d640 1 -- start start 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.408+0000 7f0d8b51d640 10 monclient: init 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 10 monclient: _add_conns ranks=[3,2,8,0,6,7,5,4,1] 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d840a2930 0x7f0d841324c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 10 monclient(hunting): picked mon.noname-g con 0x7f0d840a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d84132de0 0x7f0d8409a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 10 monclient(hunting): picked mon.noname-b con 0x7f0d84132de0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d841380e0 0x7f0d8409ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 10 monclient(hunting): picked mon.noname-e con 0x7f0d841380e0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8409d630 con 0x7f0d84132de0 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8409f870 con 0x7f0d840a2930 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8411d9a0 con 0x7f0d841380e0 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d8b51d640 10 monclient(hunting): authenticate will time out at 1804.159272s 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d84132de0 0x7f0d8409a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d84132de0 0x7f0d8409a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39526/0 (socket says 172.21.15.178:39526) 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 1 -- 172.21.15.178:0/2851008505 learned_addr learned my addr 172.21.15.178:0/2851008505 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 10 monclient(hunting): get_auth_request con 0x7f0d84132de0 auth_method 0 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d89aba640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d841380e0 0x7f0d8409ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d89aba640 10 monclient(hunting): get_auth_request con 0x7f0d841380e0 auth_method 0 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d89aba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d89aba640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d89aba640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d892b9640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d840a2930 0x7f0d841324c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d892b9640 10 monclient(hunting): get_auth_request con 0x7f0d840a2930 auth_method 0 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d892b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d892b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d892b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d89aba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d89aba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d89aba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d892b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d892b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d892b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 10 monclient(hunting): handle_auth_done global_id 35462 payload 824 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 1 -- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d841380e0 msgr2=0x7f0d8409ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d841380e0 0x7f0d8409ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 1 -- 172.21.15.178:0/2851008505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d840a2930 msgr2=0x7f0d841324c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0d840a2930 0x7f0d841324c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 1 monclient: found mon.noname-b 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 1 -- 172.21.15.178:0/2851008505 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c009a20 con 0x7f0d84132de0 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d89aba640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d841380e0 0x7f0d8409ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.409+0000 7f0d88ab8640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d84132de0 0x7f0d8409a5c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c850 tx=0x7f0d74008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d74007b00 con 0x7f0d84132de0 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 10 monclient: dump: 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d8b51d640 5 monclient: authenticate success, global_id 35462 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d74007cc0 con 0x7f0d84132de0 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d74007e30 con 0x7f0d84132de0 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d8b51d640 10 monclient: _renew_subs 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d8b51d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2851008505 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8409c770 con 0x7f0d84132de0 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d7b7fe640 4 set_mon_vals no callback set 2024-09-15T01:49:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:49:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 10 monclient: dump: 2024-09-15T01:49:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d8b51d640 10 monclient: _renew_subs 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d8b51d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2851008505 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8409c020 con 0x7f0d84132de0 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d8b51d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d8b51d640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8411c530 0x7f0d8409b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0d74007680 con 0x7f0d84132de0 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d8b51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.410+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2851008505 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8404c4a0 con 0x7f0d8411c530 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d892b9640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8411c530 0x7f0d8409b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d892b9640 10 monclient: get_auth_request con 0x7f0d8411c530 auth_method 0 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d892b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d892b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d892b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d892b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d892b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d892b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 1 --2- 172.21.15.178:0/2851008505 >> v2:172.21.15.204:6812/28210 conn(0x7f0d680397e0 0x7f0d6803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f0d7404bbc0 con 0x7f0d84132de0 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d892b9640 10 monclient(hunting): handle_auth_done global_id 94445 payload 824 2024-09-15T01:49:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d892b9640 1 -- 172.21.15.178:0/2851008505 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d8405ed70 con 0x7f0d8411c530 2024-09-15T01:49:59.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d89aba640 1 --2- 172.21.15.178:0/2851008505 >> v2:172.21.15.204:6812/28210 conn(0x7f0d680397e0 0x7f0d6803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d89aba640 10 monclient: get_auth_request con 0x7f0d680397e0 auth_method 0 2024-09-15T01:49:59.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d892b9640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8411c530 0x7f0d8409b040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d80001390 tx=0x7f0d80007ea0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d80005ea0 con 0x7f0d8411c530 2024-09-15T01:49:59.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:49:59.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 10 monclient: dump: 2024-09-15T01:49:59.440 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.440 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.441 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f0d80005a20 con 0x7f0d8411c530 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8411c530 msgr2=0x7f0d8409b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8411c530 0x7f0d8409b040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d80001390 tx=0x7f0d80007ea0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8411c530 msgr2=0x7f0d8409b040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d797fa640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d8411c530 0x7f0d8409b040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.411+0000 7f0d89aba640 1 --2- 172.21.15.178:0/2851008505 >> v2:172.21.15.204:6812/28210 conn(0x7f0d680397e0 0x7f0d6803bc90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c034000 tx=0x7f0d7c037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d8b51d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d8b51d640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d841317b0 0x7f0d84096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d8b51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2851008505 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8404bc80 con 0x7f0d841317b0 2024-09-15T01:49:59.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d892b9640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d841317b0 0x7f0d84096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d892b9640 10 monclient: get_auth_request con 0x7f0d841317b0 auth_method 0 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d892b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d892b9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d892b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d892b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d892b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d892b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d892b9640 10 monclient(hunting): handle_auth_done global_id 94454 payload 824 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.417+0000 7f0d892b9640 1 -- 172.21.15.178:0/2851008505 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0d8409e340 con 0x7f0d841317b0 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d892b9640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d841317b0 0x7f0d84096a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d80000fb0 tx=0x7f0d80018000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d80013dd0 con 0x7f0d841317b0 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d797fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:49:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d797fa640 10 monclient: dump: 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.444 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.445 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.445 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.445 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 <== mon.8 v2:172.21.15.178:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f0d80005030 con 0x7f0d841317b0 2024-09-15T01:49:59.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d797fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:49:59.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d797fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:49:59.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d841317b0 msgr2=0x7f0d84096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d797fa640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d841317b0 0x7f0d84096a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d80000fb0 tx=0x7f0d80018000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d797fa640 1 -- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d841317b0 msgr2=0x7f0d84096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:59.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d797fa640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f0d841317b0 0x7f0d84096a30 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d80000fb0 tx=0x7f0d80018000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.446 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:49:59.446 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:49:59.446 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:49:59.446 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:49:59.446 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 158, 2024-09-15T01:49:59.446 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:49:59.448 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:49:59.449 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:49:59.450 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.451 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.452 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:49:59.453 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.454 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.454 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.454 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:49:59.454 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.454 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.454 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:49:59.455 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.456 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.457 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:49:59.458 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.460 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.460 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.460 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:49:59.460 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.460 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.460 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.460 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.460 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:49:59.460 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:49:59.460 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.460 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.460 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.461 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:49:59.461 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:49:59.461 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.461 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.461 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.461 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:49:59.461 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.463 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:49:59.464 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.465 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:49:59.467 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.468 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.469 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:49:59.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.469 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.469 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.469 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:59.469 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.469 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.469 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:49:59.469 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:49:59.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2851008505 >> v2:172.21.15.204:6812/28210 conn(0x7f0d680397e0 msgr2=0x7f0d6803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d8b51d640 1 --2- 172.21.15.178:0/2851008505 >> v2:172.21.15.204:6812/28210 conn(0x7f0d680397e0 0x7f0d6803bc90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c034000 tx=0x7f0d7c037000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d8b51d640 10 monclient: shutdown 2024-09-15T01:49:59.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d8b51d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:59.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d84132de0 msgr2=0x7f0d8409a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d8b51d640 1 --2- 172.21.15.178:0/2851008505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d84132de0 0x7f0d8409a5c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c850 tx=0x7f0d74008040 comp rx=0 tx=0).stop 2024-09-15T01:49:59.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d89aba640 1 -- 172.21.15.178:0/2851008505 reap_dead start 2024-09-15T01:49:59.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d89aba640 1 -- 172.21.15.178:0/2851008505 reap_dead start 2024-09-15T01:49:59.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2851008505 shutdown_connections 2024-09-15T01:49:59.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2851008505 >> 172.21.15.178:0/2851008505 conn(0x7f0d840a5ed0 msgr2=0x7f0d840a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:59.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2851008505 shutdown_connections 2024-09-15T01:49:59.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.418+0000 7f0d8b51d640 1 -- 172.21.15.178:0/2851008505 wait complete. 2024-09-15T01:49:59.472 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:49:59.472 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T01:49:59.472 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T01:49:59.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.564+0000 7f0c6571d640 10 monclient: get_monmap_and_config 2024-09-15T01:49:59.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.564+0000 7f0c6571d640 10 monclient: build_initial_monmap 2024-09-15T01:49:59.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.564+0000 7f0c6571d640 10 monclient: monmap: 2024-09-15T01:49:59.567 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:59.567 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.567 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:49:59.565681+0000 2024-09-15T01:49:59.567 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:49:59.565681+0000 2024-09-15T01:49:59.567 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:59.567 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.567 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:49:59.568 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:49:59.568 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:59.568 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:59.568 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:59.568 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:59.568 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:59.568 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:59.568 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:59.568 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:49:59.568 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:49:59.568 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 1 Processor -- start 2024-09-15T01:49:59.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 1 -- start start 2024-09-15T01:49:59.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 10 monclient: init 2024-09-15T01:49:59.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 10 monclient: _add_conns ranks=[10,6,3,8,7,2,5,0,1,9,4] 2024-09-15T01:49:59.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0c600a2930 0x7f0c60132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 10 monclient(hunting): picked mon.noname-f con 0x7f0c600a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:49:59.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c6009faa0 0x7f0c6009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 10 monclient(hunting): picked mon.noname-d con 0x7f0c6009faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:49:59.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c600a08c0 0x7f0c600a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 10 monclient(hunting): picked mon.noname-g con 0x7f0c600a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:49:59.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6009e0f0 con 0x7f0c600a2930 2024-09-15T01:49:59.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6009dd80 con 0x7f0c600a08c0 2024-09-15T01:49:59.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600a2d00 con 0x7f0c6009faa0 2024-09-15T01:49:59.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.565+0000 7f0c6571d640 10 monclient(hunting): authenticate will time out at 1804.315946s 2024-09-15T01:49:59.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c6009faa0 0x7f0c6009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c6009faa0 0x7f0c6009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:35426/0 (socket says 172.21.15.178:35426) 2024-09-15T01:49:59.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 1 -- 172.21.15.178:0/603141600 learned_addr learned my addr 172.21.15.178:0/603141600 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:59.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 10 monclient(hunting): get_auth_request con 0x7f0c6009faa0 auth_method 0 2024-09-15T01:49:59.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5effd640 1 --2- 172.21.15.178:0/603141600 >> v2:172.21.15.204:3300/0 conn(0x7f0c600a2930 0x7f0c60132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5f7fe640 1 --2- 172.21.15.178:0/603141600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c600a08c0 0x7f0c600a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5effd640 10 monclient(hunting): get_auth_request con 0x7f0c600a2930 auth_method 0 2024-09-15T01:49:59.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5f7fe640 10 monclient(hunting): get_auth_request con 0x7f0c600a08c0 auth_method 0 2024-09-15T01:49:59.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 10 monclient(hunting): handle_auth_done global_id 25890 payload 274 2024-09-15T01:49:59.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 1 -- 172.21.15.178:0/603141600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c600a08c0 msgr2=0x7f0c600a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:59.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 1 --2- 172.21.15.178:0/603141600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c600a08c0 0x7f0c600a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 1 -- 172.21.15.178:0/603141600 >> v2:172.21.15.204:3300/0 conn(0x7f0c600a2930 msgr2=0x7f0c60132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:59.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 1 --2- 172.21.15.178:0/603141600 >> v2:172.21.15.204:3300/0 conn(0x7f0c600a2930 0x7f0c60132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 1 monclient: found mon.noname-d 2024-09-15T01:49:59.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:49:59.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5e7fc640 1 -- 172.21.15.178:0/603141600 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6004bf60 con 0x7f0c6009faa0 2024-09-15T01:49:59.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5effd640 1 --2- 172.21.15.178:0/603141600 >> v2:172.21.15.204:3300/0 conn(0x7f0c600a2930 0x7f0c60132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:49:59.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.566+0000 7f0c5f7fe640 1 --2- 172.21.15.178:0/603141600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c600a08c0 0x7f0c600a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:49:59.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5e7fc640 1 --2- 172.21.15.178:0/603141600 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c6009faa0 0x7f0c6009fe70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009910 tx=0x7f0c480091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=31ccc3b02d545741 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5dffb640 1 -- 172.21.15.178:0/603141600 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c4802ac80 con 0x7f0c6009faa0 2024-09-15T01:49:59.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5dffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:49:59.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5dffb640 10 monclient: dump: 2024-09-15T01:49:59.587 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.587 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.587 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.587 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.587 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.587 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.587 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.587 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.587 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.587 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.587 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.588 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.588 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.588 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.588 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.588 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5dffb640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5dffb640 1 -- 172.21.15.178:0/603141600 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4802ae00 con 0x7f0c6009faa0 2024-09-15T01:49:59.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 5 monclient: authenticate success, global_id 25890 2024-09-15T01:49:59.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:59.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:59.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 10 monclient: get_monmap_and_config success 2024-09-15T01:49:59.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 4 set_mon_vals no callback set 2024-09-15T01:49:59.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5dffb640 1 -- 172.21.15.178:0/603141600 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c48031ca0 con 0x7f0c6009faa0 2024-09-15T01:49:59.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:49:59.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c5dffb640 10 monclient: dump: 2024-09-15T01:49:59.589 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.589 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.589 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.589 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.589 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.589 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.590 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.590 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.590 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.590 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.590 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.590 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.590 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.590 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.590 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.590 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 10 monclient: shutdown 2024-09-15T01:49:59.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:59.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 1 -- 172.21.15.178:0/603141600 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c6009faa0 msgr2=0x7f0c6009fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 1 --2- 172.21.15.178:0/603141600 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c6009faa0 0x7f0c6009fe70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009910 tx=0x7f0c480091d0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 1 -- 172.21.15.178:0/603141600 shutdown_connections 2024-09-15T01:49:59.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 1 --2- 172.21.15.178:0/603141600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c600a08c0 0x7f0c600a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 1 --2- 172.21.15.178:0/603141600 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f0c6009faa0 0x7f0c6009fe70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 1 --2- 172.21.15.178:0/603141600 >> v2:172.21.15.204:3300/0 conn(0x7f0c600a2930 0x7f0c60132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 1 -- 172.21.15.178:0/603141600 >> 172.21.15.178:0/603141600 conn(0x7f0c600a5ed0 msgr2=0x7f0c600a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:59.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 1 -- 172.21.15.178:0/603141600 shutdown_connections 2024-09-15T01:49:59.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.567+0000 7f0c6571d640 1 -- 172.21.15.178:0/603141600 wait complete. 2024-09-15T01:49:59.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.568+0000 7f0c6571d640 10 monclient: build_initial_monmap 2024-09-15T01:49:59.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.568+0000 7f0c6571d640 10 monclient: monmap: 2024-09-15T01:49:59.591 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:59.592 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:49:59.592 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:49:59.592 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:49:59.592 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:59.592 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:49:59.592 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:49:59.592 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:49:59.592 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:59.592 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:59.592 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:59.592 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:59.593 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:59.593 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:59.593 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:59.593 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.568+0000 7f0c6571d640 1 Processor -- start 2024-09-15T01:49:59.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.568+0000 7f0c6571d640 1 -- start start 2024-09-15T01:49:59.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.568+0000 7f0c6571d640 10 monclient: init 2024-09-15T01:49:59.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 10 monclient: _add_conns ranks=[0,1,2,4,3,5,7,8,6] 2024-09-15T01:49:59.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c6009faa0 0x7f0c6009b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 10 monclient(hunting): picked mon.noname-a con 0x7f0c6009faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:49:59.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c600a08c0 0x7f0c601317e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 10 monclient(hunting): picked mon.noname-f con 0x7f0c600a08c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:49:59.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c600a2930 0x7f0c6011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 10 monclient(hunting): picked mon.noname-b con 0x7f0c600a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:49:59.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600a2400 con 0x7f0c6009faa0 2024-09-15T01:49:59.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60142b30 con 0x7f0c600a08c0 2024-09-15T01:49:59.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6009bc50 con 0x7f0c600a2930 2024-09-15T01:49:59.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c6571d640 10 monclient(hunting): authenticate will time out at 1804.319338s 2024-09-15T01:49:59.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c6009faa0 0x7f0c6009b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c600a2930 0x7f0c6011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c6009faa0 0x7f0c6009b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35066/0 (socket says 172.21.15.178:35066) 2024-09-15T01:49:59.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c600a2930 0x7f0c6011c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39536/0 (socket says 172.21.15.178:39536) 2024-09-15T01:49:59.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5effd640 1 -- 172.21.15.178:0/3151622028 learned_addr learned my addr 172.21.15.178:0/3151622028 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:59.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5effd640 10 monclient(hunting): get_auth_request con 0x7f0c6009faa0 auth_method 0 2024-09-15T01:49:59.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 10 monclient(hunting): get_auth_request con 0x7f0c600a2930 auth_method 0 2024-09-15T01:49:59.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5e7fc640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c600a08c0 0x7f0c601317e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5e7fc640 10 monclient(hunting): get_auth_request con 0x7f0c600a08c0 auth_method 0 2024-09-15T01:49:59.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 10 monclient(hunting): handle_auth_done global_id 35471 payload 824 2024-09-15T01:49:59.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 1 -- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c600a08c0 msgr2=0x7f0c601317e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c600a08c0 0x7f0c601317e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 1 -- 172.21.15.178:0/3151622028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c6009faa0 msgr2=0x7f0c6009b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c6009faa0 0x7f0c6009b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 1 monclient: found mon.noname-b 2024-09-15T01:49:59.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:49:59.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 1 -- 172.21.15.178:0/3151622028 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c48009450 con 0x7f0c600a2930 2024-09-15T01:49:59.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5e7fc640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c600a08c0 0x7f0c601317e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:49:59.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.569+0000 7f0c5f7fe640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c600a2930 0x7f0c6011c530 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000d890 tx=0x7f0c50008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c5000bb40 con 0x7f0c600a2930 2024-09-15T01:49:59.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:49:59.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 10 monclient: dump: 2024-09-15T01:49:59.598 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.598 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.599 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.599 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.599 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.599 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.599 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.599 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.599 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.599 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.599 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.599 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.599 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.599 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.600 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.600 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5000bce0 con 0x7f0c600a2930 2024-09-15T01:49:59.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c6571d640 5 monclient: authenticate success, global_id 35471 2024-09-15T01:49:59.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c6571d640 10 monclient: _renew_subs 2024-09-15T01:49:59.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c6571d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:49:59.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c6571d640 1 -- 172.21.15.178:0/3151622028 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6011d700 con 0x7f0c600a2930 2024-09-15T01:49:59.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:59.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c5000be50 con 0x7f0c600a2930 2024-09-15T01:49:59.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:49:59.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c5d7fa640 4 set_mon_vals no callback set 2024-09-15T01:49:59.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 10 monclient: dump: 2024-09-15T01:49:59.601 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.601 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.601 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.601 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.601 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.601 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.601 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.602 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.602 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.602 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.602 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.602 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.602 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.602 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.602 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.602 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c6571d640 10 monclient: _renew_subs 2024-09-15T01:49:59.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c6571d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:49:59.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c6571d640 1 -- 172.21.15.178:0/3151622028 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6011e900 con 0x7f0c600a2930 2024-09-15T01:49:59.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c6571d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:59.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c6571d640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c60131d20 0x7f0c6011ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c6571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c6571d640 1 -- 172.21.15.178:0/3151622028 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c601320f0 con 0x7f0c60131d20 2024-09-15T01:49:59.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.570+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f0c5000b6c0 con 0x7f0c600a2930 2024-09-15T01:49:59.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5effd640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c60131d20 0x7f0c6011ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5effd640 10 monclient: get_auth_request con 0x7f0c60131d20 auth_method 0 2024-09-15T01:49:59.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c56ffd640 1 --2- 172.21.15.178:0/3151622028 >> v2:172.21.15.204:6812/28210 conn(0x7f0c400396e0 0x7f0c4003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f0c5004c1d0 con 0x7f0c600a2930 2024-09-15T01:49:59.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5e7fc640 1 --2- 172.21.15.178:0/3151622028 >> v2:172.21.15.204:6812/28210 conn(0x7f0c400396e0 0x7f0c4003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5e7fc640 10 monclient: get_auth_request con 0x7f0c400396e0 auth_method 0 2024-09-15T01:49:59.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5effd640 10 monclient(hunting): handle_auth_done global_id 104131 payload 824 2024-09-15T01:49:59.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5effd640 1 -- 172.21.15.178:0/3151622028 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c6004c4a0 con 0x7f0c60131d20 2024-09-15T01:49:59.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5effd640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c60131d20 0x7f0c6011ca70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c580024e0 tx=0x7f0c58009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c58014a70 con 0x7f0c60131d20 2024-09-15T01:49:59.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c56ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:49:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c56ffd640 10 monclient: dump: 2024-09-15T01:49:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.606 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.606 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.606 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.606 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.606 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.571+0000 7f0c5e7fc640 1 --2- 172.21.15.178:0/3151622028 >> v2:172.21.15.204:6812/28210 conn(0x7f0c400396e0 0x7f0c4003bb90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0c48034000 tx=0x7f0c48032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.572+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7f0c5800f040 con 0x7f0c60131d20 2024-09-15T01:49:59.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.572+0000 7f0c56ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:59.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.572+0000 7f0c56ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:49:59.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.572+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c60131d20 msgr2=0x7f0c6011ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.572+0000 7f0c56ffd640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c60131d20 0x7f0c6011ca70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c580024e0 tx=0x7f0c58009a10 comp rx=0 tx=0).stop 2024-09-15T01:49:59.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.572+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c60131d20 msgr2=0x7f0c6011ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:59.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.572+0000 7f0c56ffd640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c60131d20 0x7f0c6011ca70 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c580024e0 tx=0x7f0c58009a10 comp rx=0 tx=0).stop 2024-09-15T01:49:59.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.577+0000 7f0c6571d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:49:59.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.577+0000 7f0c6571d640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6011cfb0 0x7f0c60096ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.577+0000 7f0c6571d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.577+0000 7f0c6571d640 1 -- 172.21.15.178:0/3151622028 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6011ec70 con 0x7f0c6011cfb0 2024-09-15T01:49:59.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c5effd640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6011cfb0 0x7f0c60096ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c5effd640 10 monclient: get_auth_request con 0x7f0c6011cfb0 auth_method 0 2024-09-15T01:49:59.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c5effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c5effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c5effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c5effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c5effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c5effd640 10 monclient(hunting): handle_auth_done global_id 104140 payload 824 2024-09-15T01:49:59.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c5effd640 1 -- 172.21.15.178:0/3151622028 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0c6004bc80 con 0x7f0c6011cfb0 2024-09-15T01:49:59.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c5effd640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6011cfb0 0x7f0c60096ac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c58000f80 tx=0x7f0c58017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c58015c30 con 0x7f0c6011cfb0 2024-09-15T01:49:59.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c56ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T01:49:59.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.578+0000 7f0c56ffd640 10 monclient: dump: 2024-09-15T01:49:59.608 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.608 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.610 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.610 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 <== mon.1 v2:172.21.15.204:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7f0c58015d90 con 0x7f0c6011cfb0 2024-09-15T01:49:59.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c56ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:49:59.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c56ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:49:59.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6011cfb0 msgr2=0x7f0c60096ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c56ffd640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6011cfb0 0x7f0c60096ac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c58000f80 tx=0x7f0c58017000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c56ffd640 1 -- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6011cfb0 msgr2=0x7f0c60096ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:59.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c56ffd640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6011cfb0 0x7f0c60096ac0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c58000f80 tx=0x7f0c58017000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.611 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:49:59.611 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:49:59.611 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:49:59.611 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:49:59.611 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 158, 2024-09-15T01:49:59.611 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:49:59.611 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:49:59.611 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:49:59.611 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:49:59.611 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:49:59.612 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:49:59.612 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:49:59.612 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:49:59.612 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:49:59.612 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:49:59.612 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.612 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:49:59.612 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:59.612 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:49:59.612 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:49:59.612 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.613 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.613 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.613 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.613 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.613 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.613 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.613 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.613 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.613 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.613 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:49:59.613 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:49:59.614 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.614 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.614 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.614 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.614 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.614 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.614 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.614 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.614 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.614 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.615 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:49:59.615 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:49:59.615 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:49:59.615 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:49:59.615 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:49:59.615 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:49:59.615 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:59.615 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:59.615 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:49:59.615 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:49:59.615 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:49:59.615 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:49:59.616 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:49:59.616 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:49:59.616 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:49:59.616 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:59.616 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:49:59.616 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.616 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.616 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.616 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.616 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.616 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.617 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.617 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.617 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.617 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.617 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:49:59.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.617 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:49:59.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.617 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:49:59.617 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:49:59.617 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.618 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.618 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.618 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:49:59.618 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.618 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.618 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:49:59.618 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.619 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.619 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.619 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:49:59.619 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:49:59.619 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.619 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.619 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.619 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:49:59.620 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:49:59.620 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.620 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:49:59.620 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.620 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.620 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:49:59.621 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.621 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.621 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.621 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:49:59.621 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:49:59.621 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.621 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.621 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.621 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.622 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:49:59.622 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:49:59.622 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.622 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.622 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.622 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:49:59.622 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.622 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.622 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:49:59.623 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.623 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.623 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.623 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:49:59.623 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:49:59.623 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.623 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.623 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.623 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:49:59.624 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:49:59.624 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.624 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.624 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.624 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:49:59.624 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.624 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.624 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.625 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:49:59.625 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.625 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.625 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.625 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.625 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:49:59.625 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:49:59.625 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.625 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.625 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.625 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.626 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:49:59.626 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:49:59.626 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.626 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.626 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.626 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:49:59.626 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.626 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.627 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.627 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:49:59.627 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.627 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.627 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.627 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.627 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:49:59.627 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:49:59.627 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.627 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.627 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.628 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.628 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.628 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:49:59.628 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:49:59.628 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.628 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.628 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.628 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.628 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:49:59.628 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.629 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.629 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.629 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.629 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:49:59.629 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.629 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.629 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.629 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.629 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:49:59.629 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:49:59.629 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.630 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.630 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.630 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.630 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:49:59.630 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:49:59.630 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.630 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.630 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.630 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:49:59.631 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.631 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.631 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.631 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.631 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:49:59.631 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.631 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.631 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.631 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.631 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:49:59.631 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:49:59.631 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.632 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.632 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.632 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.632 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:49:59.632 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:49:59.632 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.632 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.632 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.632 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:49:59.633 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.633 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.633 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.633 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:49:59.633 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.633 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.633 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.633 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:49:59.633 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:49:59.633 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.634 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.634 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.634 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.634 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.634 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:49:59.634 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:49:59.634 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.634 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.634 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.634 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.634 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:49:59.635 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.635 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.635 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.635 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:49:59.635 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.635 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.635 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.635 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:49:59.635 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:49:59.636 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.636 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.636 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.636 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.636 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.636 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.636 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:49:59.636 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:49:59.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.636 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.636 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.637 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:59.637 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.637 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.637 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:49:59.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.637 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.637 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.637 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:59.637 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.637 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.638 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:49:59.638 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:49:59.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c6571d640 1 -- 172.21.15.178:0/3151622028 >> v2:172.21.15.204:6812/28210 conn(0x7f0c400396e0 msgr2=0x7f0c4003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c6571d640 1 --2- 172.21.15.178:0/3151622028 >> v2:172.21.15.204:6812/28210 conn(0x7f0c400396e0 0x7f0c4003bb90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0c48034000 tx=0x7f0c48032000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c6571d640 10 monclient: shutdown 2024-09-15T01:49:59.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c6571d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:59.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c6571d640 1 -- 172.21.15.178:0/3151622028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c600a2930 msgr2=0x7f0c6011c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c6571d640 1 --2- 172.21.15.178:0/3151622028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c600a2930 0x7f0c6011c530 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000d890 tx=0x7f0c50008040 comp rx=0 tx=0).stop 2024-09-15T01:49:59.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c5f7fe640 1 -- 172.21.15.178:0/3151622028 reap_dead start 2024-09-15T01:49:59.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c5f7fe640 1 -- 172.21.15.178:0/3151622028 reap_dead start 2024-09-15T01:49:59.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c6571d640 1 -- 172.21.15.178:0/3151622028 shutdown_connections 2024-09-15T01:49:59.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c6571d640 1 -- 172.21.15.178:0/3151622028 >> 172.21.15.178:0/3151622028 conn(0x7f0c600a5ed0 msgr2=0x7f0c6009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:59.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c6571d640 1 -- 172.21.15.178:0/3151622028 shutdown_connections 2024-09-15T01:49:59.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.579+0000 7f0c6571d640 1 -- 172.21.15.178:0/3151622028 wait complete. 2024-09-15T01:49:59.641 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:49:59.641 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T01:49:59.641 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T01:49:59.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.733+0000 7fe1aab9d640 10 monclient: get_monmap_and_config 2024-09-15T01:49:59.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.733+0000 7fe1aab9d640 10 monclient: build_initial_monmap 2024-09-15T01:49:59.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.733+0000 7fe1aab9d640 10 monclient: monmap: 2024-09-15T01:49:59.735 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:59.735 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.736 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:49:59.734222+0000 2024-09-15T01:49:59.736 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:49:59.734222+0000 2024-09-15T01:49:59.736 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:59.736 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.736 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:49:59.736 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:49:59.736 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:59.736 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:59.736 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:59.736 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:59.736 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:59.737 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:59.737 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:59.737 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:49:59.737 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:49:59.737 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 1 Processor -- start 2024-09-15T01:49:59.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 1 -- start start 2024-09-15T01:49:59.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 10 monclient: init 2024-09-15T01:49:59.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 10 monclient: _add_conns ranks=[3,4,8,0,6,1,7,9,2,5,10] 2024-09-15T01:49:59.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a40a2930 0x7fe1a4132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 10 monclient(hunting): picked mon.noname-g con 0x7fe1a40a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:49:59.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1a409faa0 0x7fe1a409fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 10 monclient(hunting): picked mon.noname-c con 0x7fe1a409faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:49:59.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1a40a08c0 0x7fe1a40a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 10 monclient(hunting): picked mon.noname-e con 0x7fe1a40a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:49:59.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a409e0f0 con 0x7fe1a40a2930 2024-09-15T01:49:59.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a409dd80 con 0x7fe1a409faa0 2024-09-15T01:49:59.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a40a2d00 con 0x7fe1a40a08c0 2024-09-15T01:49:59.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1aab9d640 10 monclient(hunting): authenticate will time out at 1804.484617s 2024-09-15T01:49:59.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a913a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1a40a08c0 0x7fe1a40a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a3fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1a409faa0 0x7fe1a409fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a3fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1a409faa0 0x7fe1a409fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44744/0 (socket says 172.21.15.178:44744) 2024-09-15T01:49:59.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a913a640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1a40a08c0 0x7fe1a40a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40044/0 (socket says 172.21.15.178:40044) 2024-09-15T01:49:59.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a3fff640 1 -- 172.21.15.178:0/605412512 learned_addr learned my addr 172.21.15.178:0/605412512 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:59.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a3fff640 10 monclient(hunting): get_auth_request con 0x7fe1a409faa0 auth_method 0 2024-09-15T01:49:59.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a913a640 10 monclient(hunting): get_auth_request con 0x7fe1a40a08c0 auth_method 0 2024-09-15T01:49:59.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a913a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a913a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a913a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a8939640 1 --2- 172.21.15.178:0/605412512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a40a2930 0x7fe1a4132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a913a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a913a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a913a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a8939640 10 monclient(hunting): get_auth_request con 0x7fe1a40a2930 auth_method 0 2024-09-15T01:49:59.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a8939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a8939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a8939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a8939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.734+0000 7fe1a8939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a8939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_done global_id 45040 payload 274 2024-09-15T01:49:59.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a3fff640 1 -- 172.21.15.178:0/605412512 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1a40a08c0 msgr2=0x7fe1a40a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:59.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a3fff640 1 --2- 172.21.15.178:0/605412512 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1a40a08c0 0x7fe1a40a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a3fff640 1 -- 172.21.15.178:0/605412512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a40a2930 msgr2=0x7fe1a4132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:59.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a3fff640 1 --2- 172.21.15.178:0/605412512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a40a2930 0x7fe1a4132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a3fff640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a3fff640 1 monclient: found mon.noname-c 2024-09-15T01:49:59.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:49:59.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a3fff640 1 -- 172.21.15.178:0/605412512 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a404bf60 con 0x7fe1a409faa0 2024-09-15T01:49:59.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a913a640 1 --2- 172.21.15.178:0/605412512 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1a40a08c0 0x7fe1a40a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:59.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a3fff640 1 --2- 172.21.15.178:0/605412512 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1a409faa0 0x7fe1a409fe70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe190009940 tx=0x7fe1900091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=ac1cc5c8f989c4d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a37fe640 1 -- 172.21.15.178:0/605412512 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe19002ac60 con 0x7fe1a409faa0 2024-09-15T01:49:59.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:49:59.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a37fe640 10 monclient: dump: 2024-09-15T01:49:59.755 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.755 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.755 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.755 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.756 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.756 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.756 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.756 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.756 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.756 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.756 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.756 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.756 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.756 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.756 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.756 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a37fe640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a37fe640 1 -- 172.21.15.178:0/605412512 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19002ade0 con 0x7fe1a409faa0 2024-09-15T01:49:59.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1aab9d640 5 monclient: authenticate success, global_id 45040 2024-09-15T01:49:59.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1aab9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:59.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:59.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1aab9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:49:59.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a37fe640 1 -- 172.21.15.178:0/605412512 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe190031ca0 con 0x7fe1a409faa0 2024-09-15T01:49:59.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1aab9d640 4 set_mon_vals no callback set 2024-09-15T01:49:59.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:49:59.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.735+0000 7fe1a37fe640 10 monclient: dump: 2024-09-15T01:49:59.758 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.758 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.758 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.758 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.758 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.758 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.758 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.758 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.758 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.758 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.758 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.759 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.759 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.759 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.759 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.759 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.736+0000 7fe1aab9d640 10 monclient: shutdown 2024-09-15T01:49:59.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.736+0000 7fe1aab9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:59.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.736+0000 7fe1aab9d640 1 -- 172.21.15.178:0/605412512 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1a409faa0 msgr2=0x7fe1a409fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.736+0000 7fe1aab9d640 1 --2- 172.21.15.178:0/605412512 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1a409faa0 0x7fe1a409fe70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe190009940 tx=0x7fe1900091d0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.736+0000 7fe1aab9d640 1 -- 172.21.15.178:0/605412512 shutdown_connections 2024-09-15T01:49:59.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.736+0000 7fe1aab9d640 1 --2- 172.21.15.178:0/605412512 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe1a40a08c0 0x7fe1a40a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.736+0000 7fe1aab9d640 1 --2- 172.21.15.178:0/605412512 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe1a409faa0 0x7fe1a409fe70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.736+0000 7fe1aab9d640 1 --2- 172.21.15.178:0/605412512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a40a2930 0x7fe1a4132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.736+0000 7fe1aab9d640 1 -- 172.21.15.178:0/605412512 >> 172.21.15.178:0/605412512 conn(0x7fe1a40a5ed0 msgr2=0x7fe1a40a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:59.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.736+0000 7fe1aab9d640 1 -- 172.21.15.178:0/605412512 shutdown_connections 2024-09-15T01:49:59.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.736+0000 7fe1aab9d640 1 -- 172.21.15.178:0/605412512 wait complete. 2024-09-15T01:49:59.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.736+0000 7fe1aab9d640 10 monclient: build_initial_monmap 2024-09-15T01:49:59.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 10 monclient: monmap: 2024-09-15T01:49:59.760 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:59.760 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:49:59.760 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:49:59.760 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:49:59.760 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:59.761 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:49:59.761 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:49:59.761 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:49:59.761 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:59.761 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:59.761 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:59.761 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:59.761 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:59.761 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:59.761 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:59.761 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 1 Processor -- start 2024-09-15T01:49:59.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 1 -- start start 2024-09-15T01:49:59.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 10 monclient: init 2024-09-15T01:49:59.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 10 monclient: _add_conns ranks=[0,1,5,4,7,3,2,8,6] 2024-09-15T01:49:59.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1a409faa0 0x7fe1a409af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 10 monclient(hunting): picked mon.noname-a con 0x7fe1a409faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:49:59.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1a40a08c0 0x7fe1a4131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 10 monclient(hunting): picked mon.noname-f con 0x7fe1a40a08c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:49:59.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe1a40a2930 0x7fe1a411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 10 monclient(hunting): picked mon.noname-h con 0x7fe1a40a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:49:59.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.737+0000 7fe1aab9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a411ec40 con 0x7fe1a409faa0 2024-09-15T01:49:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a409d630 con 0x7fe1a40a08c0 2024-09-15T01:49:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a411da00 con 0x7fe1a40a2930 2024-09-15T01:49:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 10 monclient(hunting): authenticate will time out at 1804.488019s 2024-09-15T01:49:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1a409faa0 0x7fe1a409af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1a409faa0 0x7fe1a409af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35074/0 (socket says 172.21.15.178:35074) 2024-09-15T01:49:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 1 -- 172.21.15.178:0/4131433109 learned_addr learned my addr 172.21.15.178:0/4131433109 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:59.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 10 monclient(hunting): get_auth_request con 0x7fe1a409faa0 auth_method 0 2024-09-15T01:49:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a913a640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe1a40a2930 0x7fe1a411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a3fff640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1a40a08c0 0x7fe1a4131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a913a640 10 monclient(hunting): get_auth_request con 0x7fe1a40a2930 auth_method 0 2024-09-15T01:49:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a913a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a913a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a913a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a3fff640 10 monclient(hunting): get_auth_request con 0x7fe1a40a08c0 auth_method 0 2024-09-15T01:49:59.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a913a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a913a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a913a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 10 monclient(hunting): handle_auth_done global_id 104166 payload 824 2024-09-15T01:49:59.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 1 -- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe1a40a2930 msgr2=0x7fe1a411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fe1a40a2930 0x7fe1a411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 1 -- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1a40a08c0 msgr2=0x7fe1a4131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1a40a08c0 0x7fe1a4131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 1 monclient: found mon.noname-a 2024-09-15T01:49:59.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:49:59.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 1 -- 172.21.15.178:0/4131433109 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19c009a20 con 0x7fe1a409faa0 2024-09-15T01:49:59.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a8939640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1a409faa0 0x7fe1a409af80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe19800ea20 tx=0x7fe1980090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe19800ce90 con 0x7fe1a409faa0 2024-09-15T01:49:59.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a0ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:49:59.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a0ff9640 10 monclient: dump: 2024-09-15T01:49:59.767 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.767 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.767 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.767 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.767 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.767 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.767 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.767 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.768 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.768 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.768 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.768 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.768 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.768 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.768 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.768 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a0ff9640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 5 monclient: authenticate success, global_id 104166 2024-09-15T01:49:59.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe198013470 con 0x7fe1a409faa0 2024-09-15T01:49:59.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 10 monclient: _renew_subs 2024-09-15T01:49:59.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:49:59.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 1 -- 172.21.15.178:0/4131433109 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe190009450 con 0x7fe1a409faa0 2024-09-15T01:49:59.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:59.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe198013a60 con 0x7fe1a409faa0 2024-09-15T01:49:59.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:49:59.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a2ffd640 4 set_mon_vals no callback set 2024-09-15T01:49:59.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1a0ff9640 10 monclient: dump: 2024-09-15T01:49:59.769 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.770 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.770 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.770 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.770 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.770 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.770 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.770 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.770 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.770 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.770 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.770 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.770 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.771 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.771 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.771 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 10 monclient: _renew_subs 2024-09-15T01:49:59.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:49:59.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.738+0000 7fe1aab9d640 1 -- 172.21.15.178:0/4131433109 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a4142a10 con 0x7fe1a409faa0 2024-09-15T01:49:59.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe198013bc0 con 0x7fe1a409faa0 2024-09-15T01:49:59.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1aab9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:59.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1aab9d640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a4131d60 0x7fe1a411ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1aab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1aab9d640 1 -- 172.21.15.178:0/4131433109 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a411eb00 con 0x7fe1a4131d60 2024-09-15T01:49:59.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a0ff9640 1 --2- 172.21.15.178:0/4131433109 >> v2:172.21.15.204:6812/28210 conn(0x7fe1880397b0 0x7fe18803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fe19804bfc0 con 0x7fe1a409faa0 2024-09-15T01:49:59.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a3fff640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a4131d60 0x7fe1a411ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a3fff640 10 monclient: get_auth_request con 0x7fe1a4131d60 auth_method 0 2024-09-15T01:49:59.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a913a640 1 --2- 172.21.15.178:0/4131433109 >> v2:172.21.15.204:6812/28210 conn(0x7fe1880397b0 0x7fe18803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.739+0000 7fe1a913a640 10 monclient: get_auth_request con 0x7fe1880397b0 auth_method 0 2024-09-15T01:49:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_done global_id 64686 payload 824 2024-09-15T01:49:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a3fff640 1 -- 172.21.15.178:0/4131433109 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1a404c4a0 con 0x7fe1a4131d60 2024-09-15T01:49:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a3fff640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a4131d60 0x7fe1a411ca70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe190009910 tx=0x7fe190032000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a913a640 1 --2- 172.21.15.178:0/4131433109 >> v2:172.21.15.204:6812/28210 conn(0x7fe1880397b0 0x7fe18803bc60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009fd0 tx=0x7fe19c00b040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe19002ac60 con 0x7fe1a4131d60 2024-09-15T01:49:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a0ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:49:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a0ff9640 10 monclient: dump: 2024-09-15T01:49:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.773 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.774 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.775 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fe19003c3e0 con 0x7fe1a4131d60 2024-09-15T01:49:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:49:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a4131d60 msgr2=0x7fe1a411ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a0ff9640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a4131d60 0x7fe1a411ca70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe190009910 tx=0x7fe190032000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a4131d60 msgr2=0x7fe1a411ca70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.740+0000 7fe1a0ff9640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a4131d60 0x7fe1a411ca70 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe190009910 tx=0x7fe190032000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1aab9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:49:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1aab9d640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a411cfb0 0x7fe1a4096a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1aab9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1aab9d640 1 -- 172.21.15.178:0/4131433109 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a404bc80 con 0x7fe1a411cfb0 2024-09-15T01:49:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1a3fff640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a411cfb0 0x7fe1a4096a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1a3fff640 10 monclient: get_auth_request con 0x7fe1a411cfb0 auth_method 0 2024-09-15T01:49:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1a3fff640 10 monclient(hunting): handle_auth_done global_id 64695 payload 824 2024-09-15T01:49:59.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.746+0000 7fe1a3fff640 1 -- 172.21.15.178:0/4131433109 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe1a405ed70 con 0x7fe1a411cfb0 2024-09-15T01:49:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1a3fff640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a411cfb0 0x7fe1a4096a30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe190009940 tx=0x7fe19003d000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 <== mon.3 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe19003cdd0 con 0x7fe1a411cfb0 2024-09-15T01:49:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1a0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1a0ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T01:49:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1a0ff9640 10 monclient: dump: 2024-09-15T01:49:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.777 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.778 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 <== mon.3 v2:172.21.15.204:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fe1900483d0 con 0x7fe1a411cfb0 2024-09-15T01:49:59.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1a0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:49:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1a0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:49:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a411cfb0 msgr2=0x7fe1a4096a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1a0ff9640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a411cfb0 0x7fe1a4096a30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe190009940 tx=0x7fe19003d000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1a0ff9640 1 -- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a411cfb0 msgr2=0x7fe1a4096a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:59.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1a0ff9640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1a411cfb0 0x7fe1a4096a30 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe190009940 tx=0x7fe19003d000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.779 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:49:59.780 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:49:59.780 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:49:59.780 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:49:59.780 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 158, 2024-09-15T01:49:59.780 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:49:59.780 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:49:59.780 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:49:59.780 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:49:59.780 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:49:59.780 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:49:59.780 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:49:59.781 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:49:59.781 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:49:59.781 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:49:59.781 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.781 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:49:59.781 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:59.781 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:49:59.781 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:49:59.781 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.781 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.781 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.782 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.782 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.782 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.782 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.782 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.782 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.782 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.782 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:49:59.782 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:49:59.782 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.782 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.783 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.783 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.783 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.783 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.783 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.783 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.783 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.783 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.783 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:49:59.783 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:49:59.784 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:49:59.784 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:49:59.784 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:49:59.784 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:49:59.784 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:59.784 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:59.784 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:49:59.784 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:49:59.784 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:49:59.784 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:49:59.784 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:49:59.785 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:49:59.785 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:49:59.785 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:59.785 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:49:59.785 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.785 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.785 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.785 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.785 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.785 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.785 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.786 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.786 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.786 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.786 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:49:59.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.786 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:49:59.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.786 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:49:59.786 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:49:59.786 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.786 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.787 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.787 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:49:59.787 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.787 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.787 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.787 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:49:59.787 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.787 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.787 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.788 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:49:59.788 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:49:59.788 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.788 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.788 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.788 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.788 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.788 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:49:59.788 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:49:59.788 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.789 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.789 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.789 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:49:59.789 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.789 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.789 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:49:59.789 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.789 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.790 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.790 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:49:59.790 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:49:59.790 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.790 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.790 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.790 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.790 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:49:59.790 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:49:59.791 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.791 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.791 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.791 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:49:59.791 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.791 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.791 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:49:59.791 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.791 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.792 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.792 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:49:59.792 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:49:59.792 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.792 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.792 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.792 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:49:59.792 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:49:59.792 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.793 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.793 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.793 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:49:59.793 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.793 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.793 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:49:59.793 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.793 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.794 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.794 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:49:59.794 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:49:59.794 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.794 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.794 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.794 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:49:59.794 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:49:59.795 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.795 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.795 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.795 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:49:59.795 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.795 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.795 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:49:59.795 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.795 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.796 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.796 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:49:59.796 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:49:59.796 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.796 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.796 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.796 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:49:59.796 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:49:59.797 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.797 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.797 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.797 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:49:59.797 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.797 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.797 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:49:59.797 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.797 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.798 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.798 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:49:59.798 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:49:59.798 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.798 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.798 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.798 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:49:59.798 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:49:59.798 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.799 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:49:59.799 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.799 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.799 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:49:59.799 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.799 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.800 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.800 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:49:59.800 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:49:59.800 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.800 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.800 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.800 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:49:59.800 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:49:59.801 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.801 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:49:59.801 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.801 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.801 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:49:59.801 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.802 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.802 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.802 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:49:59.802 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:49:59.802 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.802 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.802 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.802 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:49:59.803 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:49:59.803 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.803 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.803 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.803 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:49:59.803 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.803 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.803 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:49:59.804 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.804 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.804 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.804 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:49:59.804 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:49:59.804 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.804 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.804 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.804 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.804 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.805 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:49:59.805 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:49:59.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.805 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.805 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.805 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:59.805 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.805 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.805 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:49:59.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.805 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.806 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.806 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:59.806 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.806 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.806 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:49:59.806 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:49:59.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1aab9d640 1 -- 172.21.15.178:0/4131433109 >> v2:172.21.15.204:6812/28210 conn(0x7fe1880397b0 msgr2=0x7fe18803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.747+0000 7fe1aab9d640 1 --2- 172.21.15.178:0/4131433109 >> v2:172.21.15.204:6812/28210 conn(0x7fe1880397b0 0x7fe18803bc60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009fd0 tx=0x7fe19c00b040 comp rx=0 tx=0).stop 2024-09-15T01:49:59.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.748+0000 7fe1aab9d640 10 monclient: shutdown 2024-09-15T01:49:59.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.748+0000 7fe1aab9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:59.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.748+0000 7fe1aab9d640 1 -- 172.21.15.178:0/4131433109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1a409faa0 msgr2=0x7fe1a409af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.748+0000 7fe1aab9d640 1 --2- 172.21.15.178:0/4131433109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1a409faa0 0x7fe1a409af80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe19800ea20 tx=0x7fe1980090f0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.748+0000 7fe1a913a640 1 -- 172.21.15.178:0/4131433109 reap_dead start 2024-09-15T01:49:59.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.748+0000 7fe1a913a640 1 -- 172.21.15.178:0/4131433109 reap_dead start 2024-09-15T01:49:59.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.748+0000 7fe1aab9d640 1 -- 172.21.15.178:0/4131433109 shutdown_connections 2024-09-15T01:49:59.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.748+0000 7fe1aab9d640 1 -- 172.21.15.178:0/4131433109 >> 172.21.15.178:0/4131433109 conn(0x7fe1a40a5ed0 msgr2=0x7fe1a409e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:59.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.748+0000 7fe1aab9d640 1 -- 172.21.15.178:0/4131433109 shutdown_connections 2024-09-15T01:49:59.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.748+0000 7fe1aab9d640 1 -- 172.21.15.178:0/4131433109 wait complete. 2024-09-15T01:49:59.809 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:49:59.809 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T01:49:59.809 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T01:49:59.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.901+0000 7fa73d51d640 10 monclient: get_monmap_and_config 2024-09-15T01:49:59.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.901+0000 7fa73d51d640 10 monclient: build_initial_monmap 2024-09-15T01:49:59.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.901+0000 7fa73d51d640 10 monclient: monmap: 2024-09-15T01:49:59.903 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:59.903 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.904 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:49:59.902217+0000 2024-09-15T01:49:59.904 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:49:59.902217+0000 2024-09-15T01:49:59.904 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:59.904 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.904 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:49:59.904 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:49:59.904 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:59.904 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:59.904 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:59.904 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:59.905 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:59.905 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:59.905 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:59.905 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:49:59.905 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:49:59.905 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 1 Processor -- start 2024-09-15T01:49:59.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 1 -- start start 2024-09-15T01:49:59.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 10 monclient: init 2024-09-15T01:49:59.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 10 monclient: _add_conns ranks=[8,2,1,0,3,5,10,7,6,9,4] 2024-09-15T01:49:59.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa7380a2930 0x7fa738132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 10 monclient(hunting): picked mon.noname-e con 0x7fa7380a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:49:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa73809faa0 0x7fa73809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 10 monclient(hunting): picked mon.noname-b con 0x7fa73809faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:49:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa7380a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:49:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa73804bf60 con 0x7fa7380a08c0 2024-09-15T01:49:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73809e0f0 con 0x7fa73809faa0 2024-09-15T01:49:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa73809dd80 con 0x7fa7380a2930 2024-09-15T01:49:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa73d51d640 10 monclient(hunting): authenticate will time out at 1804.652661s 2024-09-15T01:49:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa736ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa7380a2930 0x7fa738132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa72ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa73809faa0 0x7fa73809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa736ffd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa7380a2930 0x7fa738132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40054/0 (socket says 172.21.15.178:40054) 2024-09-15T01:49:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa736ffd640 1 -- 172.21.15.178:0/1891277116 learned_addr learned my addr 172.21.15.178:0/1891277116 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa736ffd640 10 monclient(hunting): get_auth_request con 0x7fa7380a2930 auth_method 0 2024-09-15T01:49:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa736ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa736ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa736ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa72ffff640 10 monclient(hunting): get_auth_request con 0x7fa73809faa0 auth_method 0 2024-09-15T01:49:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa72ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa72ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa72ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa736ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa736ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa736ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa72ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa72ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.902+0000 7fa72ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa736ffd640 10 monclient(hunting): handle_auth_done global_id 94472 payload 274 2024-09-15T01:49:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa736ffd640 1 -- 172.21.15.178:0/1891277116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa73809faa0 msgr2=0x7fa73809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa736ffd640 1 --2- 172.21.15.178:0/1891277116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa73809faa0 0x7fa73809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa736ffd640 1 -- 172.21.15.178:0/1891277116 >> v1:172.21.15.204:6789/0 conn(0x7fa7380a08c0 legacy=0x7fa7380a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:49:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa736ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa736ffd640 1 monclient: found mon.noname-e 2024-09-15T01:49:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa736ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa736ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:49:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa736ffd640 1 -- 172.21.15.178:0/1891277116 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7380a2d00 con 0x7fa7380a2930 2024-09-15T01:49:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa72ffff640 1 --2- 172.21.15.178:0/1891277116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa73809faa0 0x7fa73809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:49:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa736ffd640 1 --2- 172.21.15.178:0/1891277116 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa7380a2930 0x7fa738132de0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa728009940 tx=0x7fa7280091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=379fda9561106c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa7367fc640 1 -- 172.21.15.178:0/1891277116 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa72802ac70 con 0x7fa7380a2930 2024-09-15T01:49:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.903+0000 7fa7367fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa7367fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:49:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa7367fc640 10 monclient: dump: 2024-09-15T01:49:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.921 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa7367fc640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa7367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 5 monclient: authenticate success, global_id 94472 2024-09-15T01:49:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:49:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa7367fc640 1 -- 172.21.15.178:0/1891277116 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72802adf0 con 0x7fa7380a2930 2024-09-15T01:49:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa7367fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 10 monclient: get_monmap_and_config success 2024-09-15T01:49:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 4 set_mon_vals no callback set 2024-09-15T01:49:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa7367fc640 1 -- 172.21.15.178:0/1891277116 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa728031ca0 con 0x7fa7380a2930 2024-09-15T01:49:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa7367fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa7367fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:49:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa7367fc640 10 monclient: dump: 2024-09-15T01:49:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.923 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 10 monclient: shutdown 2024-09-15T01:49:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 1 -- 172.21.15.178:0/1891277116 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa7380a2930 msgr2=0x7fa738132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 1 --2- 172.21.15.178:0/1891277116 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa7380a2930 0x7fa738132de0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa728009940 tx=0x7fa7280091d0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 1 -- 172.21.15.178:0/1891277116 shutdown_connections 2024-09-15T01:49:59.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 1 --2- 172.21.15.178:0/1891277116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa73809faa0 0x7fa73809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 1 --2- 172.21.15.178:0/1891277116 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fa7380a2930 0x7fa738132de0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 1 -- 172.21.15.178:0/1891277116 >> 172.21.15.178:0/1891277116 conn(0x7fa7380a5ed0 msgr2=0x7fa7380a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:59.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 1 -- 172.21.15.178:0/1891277116 shutdown_connections 2024-09-15T01:49:59.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.904+0000 7fa73d51d640 1 -- 172.21.15.178:0/1891277116 wait complete. 2024-09-15T01:49:59.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient: build_initial_monmap 2024-09-15T01:49:59.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient: monmap: 2024-09-15T01:49:59.924 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:49:59.924 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:49:59.924 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:49:59.924 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:49:59.925 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:49:59.925 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:49:59.925 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:49:59.925 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:49:59.925 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:49:59.925 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:49:59.925 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:49:59.925 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:49:59.925 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:49:59.925 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:49:59.925 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:49:59.925 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 1 Processor -- start 2024-09-15T01:49:59.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 1 -- start start 2024-09-15T01:49:59.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient: init 2024-09-15T01:49:59.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient: _reopen_session rank -1 2024-09-15T01:49:59.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient: _add_conns ranks=[7,2,6,1,4,0,5,3,8] 2024-09-15T01:49:59.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa73809faa0 0x7fa73809b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient(hunting): picked mon.noname-i con 0x7fa73809faa0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:49:59.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7380a08c0 0x7fa7381318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient(hunting): picked mon.noname-b con 0x7fa7380a08c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:49:59.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa7380a2930 0x7fa73811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient(hunting): picked mon.noname-d con 0x7fa7380a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:49:59.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73809d630 con 0x7fa7380a08c0 2024-09-15T01:49:59.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa73811ef90 con 0x7fa7380a2930 2024-09-15T01:49:59.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa73811ecb0 con 0x7fa73809faa0 2024-09-15T01:49:59.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient(hunting): _renew_subs 2024-09-15T01:49:59.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.905+0000 7fa73d51d640 10 monclient(hunting): authenticate will time out at 1804.655998s 2024-09-15T01:49:59.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7380a08c0 0x7fa7381318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7380a08c0 0x7fa7381318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39556/0 (socket says 172.21.15.178:39556) 2024-09-15T01:49:59.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 1 -- 172.21.15.178:0/132373690 learned_addr learned my addr 172.21.15.178:0/132373690 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:49:59.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 10 monclient(hunting): get_auth_request con 0x7fa7380a08c0 auth_method 0 2024-09-15T01:49:59.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa7377fe640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa7380a2930 0x7fa73811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa7377fe640 10 monclient(hunting): get_auth_request con 0x7fa7380a2930 auth_method 0 2024-09-15T01:49:59.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa7377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa7377fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa7377fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa7377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa7377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa7377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa736ffd640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa73809faa0 0x7fa73809b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 10 monclient(hunting): handle_auth_done global_id 35489 payload 824 2024-09-15T01:49:59.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 1 -- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa73809faa0 msgr2=0x7fa73809b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa73809faa0 0x7fa73809b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 1 -- 172.21.15.178:0/132373690 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa7380a2930 msgr2=0x7fa73811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fa7380a2930 0x7fa73811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:49:59.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 1 monclient: found mon.noname-b 2024-09-15T01:49:59.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:49:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:49:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 1 -- 172.21.15.178:0/132373690 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa720009390 con 0x7fa7380a08c0 2024-09-15T01:49:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa736ffd640 10 monclient: get_auth_request con 0x7fa73809faa0 auth_method 0 2024-09-15T01:49:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa736ffd640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fa73809faa0 0x7fa73809b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:49:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72ffff640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7380a08c0 0x7fa7381318b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa72000ea50 tx=0x7fa7200090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa72000ce90 con 0x7fa7380a08c0 2024-09-15T01:49:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:49:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72effd640 10 monclient: dump: 2024-09-15T01:49:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.931 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.931 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.931 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.931 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.931 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.931 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.931 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.931 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.931 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.931 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.931 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.931 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.932 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.932 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72effd640 10 monclient: _finish_auth 0 2024-09-15T01:49:59.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:49:59.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa720009650 con 0x7fa7380a08c0 2024-09-15T01:49:59.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa73d51d640 5 monclient: authenticate success, global_id 35489 2024-09-15T01:49:59.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72effd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:49:59.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa7200138f0 con 0x7fa7380a08c0 2024-09-15T01:49:59.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:49:59.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa72effd640 10 monclient: dump: 2024-09-15T01:49:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.934 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa735ffb640 4 set_mon_vals no callback set 2024-09-15T01:49:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa73d51d640 10 monclient: _renew_subs 2024-09-15T01:49:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa73d51d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:49:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa73d51d640 1 -- 172.21.15.178:0/132373690 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa728009450 con 0x7fa7380a08c0 2024-09-15T01:49:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa73d51d640 10 monclient: _renew_subs 2024-09-15T01:49:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa73d51d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:49:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.906+0000 7fa73d51d640 1 -- 172.21.15.178:0/132373690 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa738142d70 con 0x7fa7380a08c0 2024-09-15T01:49:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fa720013470 con 0x7fa7380a08c0 2024-09-15T01:49:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa73d51d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa73d51d640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa7381423c0 0x7fa73811ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa73d51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa73d51d640 1 -- 172.21.15.178:0/132373690 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa73804c4a0 con 0x7fa7381423c0 2024-09-15T01:49:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa72effd640 1 --2- 172.21.15.178:0/132373690 >> v2:172.21.15.204:6812/28210 conn(0x7fa7180396e0 0x7fa71803bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fa72004beb0 con 0x7fa7380a08c0 2024-09-15T01:49:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa736ffd640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa7381423c0 0x7fa73811ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa736ffd640 10 monclient: get_auth_request con 0x7fa7381423c0 auth_method 0 2024-09-15T01:49:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa736ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa736ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa736ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa736ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa736ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa736ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa7377fe640 1 --2- 172.21.15.178:0/132373690 >> v2:172.21.15.204:6812/28210 conn(0x7fa7180396e0 0x7fa71803bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.907+0000 7fa7377fe640 10 monclient: get_auth_request con 0x7fa7180396e0 auth_method 0 2024-09-15T01:49:59.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa736ffd640 10 monclient(hunting): handle_auth_done global_id 7835 payload 824 2024-09-15T01:49:59.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa736ffd640 1 -- 172.21.15.178:0/132373690 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa73805ed70 con 0x7fa7381423c0 2024-09-15T01:49:59.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa736ffd640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa7381423c0 0x7fa73811ca70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa7380a3290 tx=0x7fa728032000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa7377fe640 1 --2- 172.21.15.178:0/132373690 >> v2:172.21.15.204:6812/28210 conn(0x7fa7180396e0 0x7fa71803bb90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa73809b9c0 tx=0x7fa7300025b0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa72803b820 con 0x7fa7381423c0 2024-09-15T01:49:59.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa72effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa72effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:49:59.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa72effd640 10 monclient: dump: 2024-09-15T01:49:59.937 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.937 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.937 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.937 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.937 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.937 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.937 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.937 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.938 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.938 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.938 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.938 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.938 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.938 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.938 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.938 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fa72803b3a0 con 0x7fa7381423c0 2024-09-15T01:49:59.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa72effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:49:59.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa72effd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:49:59.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa7381423c0 msgr2=0x7fa73811ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa72effd640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa7381423c0 0x7fa73811ca70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa7380a3290 tx=0x7fa728032000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa7381423c0 msgr2=0x7fa73811ca70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:59.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.908+0000 7fa72effd640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa7381423c0 0x7fa73811ca70 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa73d51d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:49:59.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa73d51d640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa73811da70 0x7fa73811cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:49:59.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa73d51d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:49:59.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa73d51d640 1 -- 172.21.15.178:0/132373690 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa73809bcd0 con 0x7fa73811da70 2024-09-15T01:49:59.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa736ffd640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa73811da70 0x7fa73811cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:49:59.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa736ffd640 10 monclient: get_auth_request con 0x7fa73811da70 auth_method 0 2024-09-15T01:49:59.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa736ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:49:59.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa736ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:49:59.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa736ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:49:59.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa736ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:49:59.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa736ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:49:59.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa736ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:49:59.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa736ffd640 10 monclient(hunting): handle_auth_done global_id 7844 payload 824 2024-09-15T01:49:59.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa736ffd640 1 -- 172.21.15.178:0/132373690 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa73804bc80 con 0x7fa73811da70 2024-09-15T01:49:59.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa736ffd640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa73811da70 0x7fa73811cfb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa728009210 tx=0x7fa72803b000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:49:59.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa72803ca90 con 0x7fa73811da70 2024-09-15T01:49:59.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa72effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:49:59.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa72effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:49:59.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.915+0000 7fa72effd640 10 monclient: dump: 2024-09-15T01:49:59.943 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:49:59.943 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:49:59.943 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.943 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:49:59.943 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:49:59.943 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:49:59.943 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:49:59.943 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:49:59.943 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:49:59.943 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:49:59.943 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:49:59.943 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:49:59.944 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:49:59.944 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:49:59.944 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:49:59.944 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:49:59.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 <== mon.5 v2:172.21.15.204:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9314 (secure 0 0 0) 0x7fa72803cbf0 con 0x7fa73811da70 2024-09-15T01:49:59.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa72effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:49:59.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa72effd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:49:59.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa73811da70 msgr2=0x7fa73811cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa72effd640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa73811da70 0x7fa73811cfb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa728009210 tx=0x7fa72803b000 comp rx=0 tx=0).stop 2024-09-15T01:49:59.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa72effd640 1 -- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa73811da70 msgr2=0x7fa73811cfb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:49:59.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa72effd640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fa73811da70 0x7fa73811cfb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.945 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:49:59.945 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:49:59.945 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:49:59.945 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "leader", 2024-09-15T01:49:59.945 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 158, 2024-09-15T01:49:59.945 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:49:59.945 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:49:59.945 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:49:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:49:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:49:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:49:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:49:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:49:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:49:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:49:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 1, 2024-09-15T01:49:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:49:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:49:59.947 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.947 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.947 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.947 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.947 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.947 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.947 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.947 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.947 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.947 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.947 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:49:59.948 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:49:59.948 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.948 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.948 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.948 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.948 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.948 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.948 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.948 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.948 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.948 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.949 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.949 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:49:59.949 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:49:59.949 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:49:59.949 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:49:59.949 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:49:59.949 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:49:59.949 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:59.949 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:49:59.949 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:49:59.949 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:49:59.949 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:49:59.950 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:49:59.950 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:49:59.950 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:49:59.950 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:49:59.950 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:49:59.950 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:49:59.950 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:49:59.950 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:49:59.950 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:49:59.950 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:49:59.950 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:49:59.951 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:49:59.951 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:49:59.951 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:49:59.951 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:49:59.951 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.951 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:49:59.951 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.951 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:49:59.951 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.951 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:49:59.951 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:49:59.951 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.952 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:49:59.952 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.952 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.952 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:49:59.952 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.952 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.953 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.953 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.953 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:49:59.953 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:49:59.953 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.953 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.953 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.953 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.953 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.953 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:49:59.953 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:49:59.953 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.954 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:49:59.954 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.954 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.954 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.954 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:49:59.954 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.954 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.954 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.955 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:49:59.955 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:49:59.955 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.955 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.955 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.955 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.955 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:49:59.955 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:49:59.955 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.956 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.956 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:49:59.956 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.956 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.956 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.956 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.956 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:49:59.956 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.956 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.956 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.957 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:49:59.957 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:49:59.957 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.957 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.957 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.957 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.957 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:49:59.957 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:49:59.957 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.958 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.958 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.958 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.958 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:49:59.958 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.958 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.958 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.958 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.958 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:49:59.958 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.958 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.958 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:49:59.959 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:49:59.959 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.959 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.959 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.959 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:49:59.959 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:49:59.959 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.959 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.960 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.960 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.960 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:49:59.960 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.960 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.960 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.960 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:49:59.960 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.960 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.961 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.961 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:49:59.961 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:49:59.961 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.961 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.961 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.961 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.961 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:49:59.961 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:49:59.962 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.962 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.962 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.962 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:49:59.962 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.962 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.962 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:49:59.962 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.963 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.963 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.963 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:49:59.963 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:49:59.963 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.963 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.963 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.963 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.963 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:49:59.963 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:49:59.964 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.964 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:49:59.964 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.964 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.964 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:49:59.964 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.965 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.965 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.965 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:49:59.965 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:49:59.965 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.965 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.965 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.965 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.965 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:49:59.965 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:49:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:49:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:49:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.967 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.967 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:49:59.967 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:49:59.967 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.967 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.967 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.967 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.967 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:49:59.967 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:49:59.967 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:49:59.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:49:59.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.968 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:49:59.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:49:59.968 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.968 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:49:59.968 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:49:59.968 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:49:59.968 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.969 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.969 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:49:59.969 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:49:59.969 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:49:59.969 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:49:59.969 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:49:59.969 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.969 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.969 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:49:59.969 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:49:59.970 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.970 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.970 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.970 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:49:59.970 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.970 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.970 INFO:teuthology.orchestra.run.smithi178.stdout: "osd": [ 2024-09-15T01:49:59.970 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.970 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.970 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.970 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 2 2024-09-15T01:49:59.971 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.971 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:49:59.971 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:49:59.971 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:49:59.971 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:49:59.971 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:49:59.971 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 5 2024-09-15T01:49:59.971 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:49:59.971 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:49:59.971 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:49:59.971 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:49:59.971 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:49:59.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa73d51d640 1 -- 172.21.15.178:0/132373690 >> v2:172.21.15.204:6812/28210 conn(0x7fa7180396e0 msgr2=0x7fa71803bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa73d51d640 1 --2- 172.21.15.178:0/132373690 >> v2:172.21.15.204:6812/28210 conn(0x7fa7180396e0 0x7fa71803bb90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa73809b9c0 tx=0x7fa7300025b0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa73d51d640 10 monclient: shutdown 2024-09-15T01:49:59.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa73d51d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:49:59.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa73d51d640 1 -- 172.21.15.178:0/132373690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7380a08c0 msgr2=0x7fa7381318b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:49:59.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa73d51d640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7380a08c0 0x7fa7381318b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa72000ea50 tx=0x7fa7200090f0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa7377fe640 1 -- 172.21.15.178:0/132373690 reap_dead start 2024-09-15T01:49:59.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa73d51d640 1 -- 172.21.15.178:0/132373690 shutdown_connections 2024-09-15T01:49:59.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa73d51d640 1 --2- 172.21.15.178:0/132373690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7380a08c0 0x7fa7381318b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:49:59.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa73d51d640 1 -- 172.21.15.178:0/132373690 >> 172.21.15.178:0/132373690 conn(0x7fa7380a5ed0 msgr2=0x7fa73809e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:49:59.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa73d51d640 1 -- 172.21.15.178:0/132373690 shutdown_connections 2024-09-15T01:49:59.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:49:59.916+0000 7fa73d51d640 1 -- 172.21.15.178:0/132373690 wait complete. 2024-09-15T01:49:59.975 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:49:59.975 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T01:49:59.975 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T01:50:00.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.067+0000 7fee9339d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:00.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.067+0000 7fee9339d640 10 monclient: build_initial_monmap 2024-09-15T01:50:00.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.067+0000 7fee9339d640 10 monclient: monmap: 2024-09-15T01:50:00.070 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:00.070 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:00.070 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:00.068443+0000 2024-09-15T01:50:00.070 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:00.068443+0000 2024-09-15T01:50:00.070 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:00.070 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:00.070 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:00.070 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:00.070 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:00.070 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:00.071 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:00.071 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:00.071 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:00.071 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:00.071 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:00.071 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:00.071 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:00.071 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 1 Processor -- start 2024-09-15T01:50:00.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 1 -- start start 2024-09-15T01:50:00.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 10 monclient: init 2024-09-15T01:50:00.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:00.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 10 monclient: _add_conns ranks=[8,6,4,5,10,3,2,0,9,7,1] 2024-09-15T01:50:00.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fee8c0a2930 0x7fee8c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 10 monclient(hunting): picked mon.noname-e con 0x7fee8c0a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:00.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 0x7fee8c09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 10 monclient(hunting): picked mon.noname-d con 0x7fee8c09faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:00.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee8c0a08c0 0x7fee8c0a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 10 monclient(hunting): picked mon.noname-c con 0x7fee8c0a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c09e0f0 con 0x7fee8c0a08c0 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c09dd80 con 0x7fee8c09faa0 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c0a2d00 con 0x7fee8c0a2930 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9339d640 10 monclient(hunting): authenticate will time out at 1804.818861s 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee90938640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 0x7fee8c09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee9193a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee8c0a08c0 0x7fee8c0a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.068+0000 7fee91139640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fee8c0a2930 0x7fee8c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee90938640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 0x7fee8c09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:35454/0 (socket says 172.21.15.178:35454) 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee90938640 1 -- 172.21.15.178:0/1519845162 learned_addr learned my addr 172.21.15.178:0/1519845162 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:00.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee8c0a08c0 0x7fee8c0a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44746/0 (socket says 172.21.15.178:44746) 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee91139640 10 monclient(hunting): get_auth_request con 0x7fee8c0a2930 auth_method 0 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee91139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee91139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee91139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee90938640 10 monclient(hunting): get_auth_request con 0x7fee8c09faa0 auth_method 0 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee90938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee90938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee90938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 10 monclient(hunting): get_auth_request con 0x7fee8c0a08c0 auth_method 0 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee91139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee91139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee91139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee90938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee90938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee90938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 10 monclient(hunting): handle_auth_done global_id 45049 payload 274 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 1 -- 172.21.15.178:0/1519845162 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fee8c0a2930 msgr2=0x7fee8c132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 1 --2- 172.21.15.178:0/1519845162 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fee8c0a2930 0x7fee8c132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 1 -- 172.21.15.178:0/1519845162 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 msgr2=0x7fee8c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 1 --2- 172.21.15.178:0/1519845162 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 0x7fee8c09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 10 monclient: _finish_hunting 0 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 1 monclient: found mon.noname-c 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee9193a640 1 -- 172.21.15.178:0/1519845162 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c04bf60 con 0x7fee8c0a08c0 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee91139640 1 --2- 172.21.15.178:0/1519845162 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fee8c0a2930 0x7fee8c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.069+0000 7fee90938640 1 --2- 172.21.15.178:0/1519845162 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 0x7fee8c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9193a640 1 --2- 172.21.15.178:0/1519845162 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee8c0a08c0 0x7fee8c0a0d30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fee84009e90 tx=0x7fee84009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1f4821fd743eed84 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee83fff640 1 -- 172.21.15.178:0/1519845162 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee840079d0 con 0x7fee8c0a08c0 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee83fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee83fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee83fff640 10 monclient: dump: 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:00.085 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:00.086 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee83fff640 10 monclient: _finish_auth 0 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee83fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee83fff640 1 -- 172.21.15.178:0/1519845162 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8402d050 con 0x7fee8c0a08c0 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 5 monclient: authenticate success, global_id 45049 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee83fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee83fff640 1 -- 172.21.15.178:0/1519845162 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee84007bd0 con 0x7fee8c0a08c0 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee83fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 4 set_mon_vals no callback set 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee83fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee83fff640 10 monclient: dump: 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:00.087 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:00.088 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 10 monclient: shutdown 2024-09-15T01:50:00.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:00.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 1 -- 172.21.15.178:0/1519845162 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee8c0a08c0 msgr2=0x7fee8c0a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:00.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 1 --2- 172.21.15.178:0/1519845162 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee8c0a08c0 0x7fee8c0a0d30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fee84009e90 tx=0x7fee84009780 comp rx=0 tx=0).stop 2024-09-15T01:50:00.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 1 -- 172.21.15.178:0/1519845162 shutdown_connections 2024-09-15T01:50:00.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 1 --2- 172.21.15.178:0/1519845162 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee8c0a08c0 0x7fee8c0a0d30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 1 --2- 172.21.15.178:0/1519845162 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 0x7fee8c09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 1 --2- 172.21.15.178:0/1519845162 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fee8c0a2930 0x7fee8c132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 1 -- 172.21.15.178:0/1519845162 >> 172.21.15.178:0/1519845162 conn(0x7fee8c0a5ed0 msgr2=0x7fee8c0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:00.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 1 -- 172.21.15.178:0/1519845162 shutdown_connections 2024-09-15T01:50:00.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.070+0000 7fee9339d640 1 -- 172.21.15.178:0/1519845162 wait complete. 2024-09-15T01:50:00.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.071+0000 7fee9339d640 10 monclient: build_initial_monmap 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.071+0000 7fee9339d640 10 monclient: monmap: 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:00.090 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.071+0000 7fee9339d640 1 Processor -- start 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.071+0000 7fee9339d640 1 -- start start 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.071+0000 7fee9339d640 10 monclient: init 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 10 monclient: _add_conns ranks=[6,4,1,3,7,2,8,0,5] 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 0x7fee8c09b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 10 monclient(hunting): picked mon.noname-d con 0x7fee8c09faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee8c0a08c0 0x7fee8c1318e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 10 monclient(hunting): picked mon.noname-c con 0x7fee8c0a08c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee8c0a2930 0x7fee8c141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 10 monclient(hunting): picked mon.noname-f con 0x7fee8c0a2930 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:50:00.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c11ece0 con 0x7fee8c0a2930 2024-09-15T01:50:00.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c11ea00 con 0x7fee8c0a08c0 2024-09-15T01:50:00.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c11efc0 con 0x7fee8c09faa0 2024-09-15T01:50:00.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:00.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9339d640 10 monclient(hunting): authenticate will time out at 1804.822366s 2024-09-15T01:50:00.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 0x7fee8c09b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee90938640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee8c0a08c0 0x7fee8c1318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 0x7fee8c09b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:35460/0 (socket says 172.21.15.178:35460) 2024-09-15T01:50:00.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 1 -- 172.21.15.178:0/3883427457 learned_addr learned my addr 172.21.15.178:0/3883427457 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee90938640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee8c0a08c0 0x7fee8c1318e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44762/0 (socket says 172.21.15.178:44762) 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 10 monclient(hunting): get_auth_request con 0x7fee8c09faa0 auth_method 0 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee90938640 10 monclient(hunting): get_auth_request con 0x7fee8c0a08c0 auth_method 0 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee90938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee90938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee90938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee90938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:00.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee90938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee90938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9193a640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee8c0a2930 0x7fee8c141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9193a640 10 monclient(hunting): get_auth_request con 0x7fee8c0a2930 auth_method 0 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9193a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9193a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee9193a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 10 monclient(hunting): handle_auth_done global_id 25917 payload 824 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 1 -- 172.21.15.178:0/3883427457 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee8c0a08c0 msgr2=0x7fee8c1318e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fee8c0a08c0 0x7fee8c1318e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 1 -- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee8c0a2930 msgr2=0x7fee8c141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee8c0a2930 0x7fee8c141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 10 monclient: _finish_hunting 0 2024-09-15T01:50:00.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 1 monclient: found mon.noname-d 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 1 -- 172.21.15.178:0/3883427457 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c009450 con 0x7fee8c09faa0 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.072+0000 7fee91139640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 0x7fee8c09b350 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fee8800ec80 tx=0x7fee880090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee8800eec0 con 0x7fee8c09faa0 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 10 monclient: dump: 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.095 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 10 monclient: _finish_auth 0 2024-09-15T01:50:00.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee88013470 con 0x7fee8c09faa0 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee9339d640 5 monclient: authenticate success, global_id 25917 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee9339d640 10 monclient: _renew_subs 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee9339d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee9339d640 1 -- 172.21.15.178:0/3883427457 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee88009390 con 0x7fee8c09faa0 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee88013a60 con 0x7fee8c09faa0 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 10 monclient: dump: 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:00.097 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee9339d640 10 monclient: _renew_subs 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee9339d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee9339d640 1 -- 172.21.15.178:0/3883427457 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee84009a20 con 0x7fee8c09faa0 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee837fe640 4 set_mon_vals no callback set 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fee88013bc0 con 0x7fee8c09faa0 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee9339d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee9339d640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11db10 0x7fee8c1422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee9339d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.073+0000 7fee9339d640 1 -- 172.21.15.178:0/3883427457 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c11eb70 con 0x7fee8c11db10 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee817fa640 1 --2- 172.21.15.178:0/3883427457 >> v2:172.21.15.204:6812/28210 conn(0x7fee700397b0 0x7fee7003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fee8804c3f0 con 0x7fee8c09faa0 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee90938640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11db10 0x7fee8c1422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee90938640 10 monclient: get_auth_request con 0x7fee8c11db10 auth_method 0 2024-09-15T01:50:00.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee90938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee90938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee90938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee9193a640 1 --2- 172.21.15.178:0/3883427457 >> v2:172.21.15.204:6812/28210 conn(0x7fee700397b0 0x7fee7003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee9193a640 10 monclient: get_auth_request con 0x7fee700397b0 auth_method 0 2024-09-15T01:50:00.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee90938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:00.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee90938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:00.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee90938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:00.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee90938640 10 monclient(hunting): handle_auth_done global_id 64600 payload 824 2024-09-15T01:50:00.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee90938640 1 -- 172.21.15.178:0/3883427457 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee8c11f130 con 0x7fee8c11db10 2024-09-15T01:50:00.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee9193a640 1 --2- 172.21.15.178:0/3883427457 >> v2:172.21.15.204:6812/28210 conn(0x7fee700397b0 0x7fee7003bc60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fee84034000 tx=0x7fee84037000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:00.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee90938640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11db10 0x7fee8c1422f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0a6ec0 tx=0x7fee7c0091d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:00.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee7c00fd50 con 0x7fee8c11db10 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee817fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.074+0000 7fee817fa640 10 monclient: dump: 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:00.101 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:00.102 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:00.102 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:00.102 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:00.102 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:00.102 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.075+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (secure 0 0 0) 0x7fee7c00feb0 con 0x7fee8c11db10 2024-09-15T01:50:00.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.075+0000 7fee817fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:00.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.075+0000 7fee817fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:00.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.075+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11db10 msgr2=0x7fee8c1422f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:00.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.075+0000 7fee817fa640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11db10 0x7fee8c1422f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0a6ec0 tx=0x7fee7c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.075+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11db10 msgr2=0x7fee8c1422f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:50:00.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.075+0000 7fee817fa640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11db10 0x7fee8c1422f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.080+0000 7fee9339d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T01:50:00.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.080+0000 7fee9339d640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11ca40 0x7fee8c142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.080+0000 7fee9339d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.080+0000 7fee9339d640 1 -- 172.21.15.178:0/3883427457 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c11ee50 con 0x7fee8c11ca40 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee90938640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11ca40 0x7fee8c142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee90938640 10 monclient: get_auth_request con 0x7fee8c11ca40 auth_method 0 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee90938640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee90938640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee90938640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee90938640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee90938640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee90938640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee90938640 10 monclient(hunting): handle_auth_done global_id 64609 payload 824 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee90938640 1 -- 172.21.15.178:0/3883427457 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fee8c04bc80 con 0x7fee8c11ca40 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee90938640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11ca40 0x7fee8c142830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fee7c000c00 tx=0x7fee7c01d000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 <== mon.7 v2:172.21.15.204:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee7c014dd0 con 0x7fee8c11ca40 2024-09-15T01:50:00.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee817fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.081+0000 7fee817fa640 10 monclient: dump: 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:00.106 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:00.107 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:00.107 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:00.107 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:00.107 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 <== mon.7 v2:172.21.15.204:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9142 (secure 0 0 0) 0x7fee7c01de90 con 0x7fee8c11ca40 2024-09-15T01:50:00.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee817fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T01:50:00.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee817fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:50:00.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11ca40 msgr2=0x7fee8c142830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:00.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee817fa640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11ca40 0x7fee8c142830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fee7c000c00 tx=0x7fee7c01d000 comp rx=0 tx=0).stop 2024-09-15T01:50:00.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee817fa640 1 -- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11ca40 msgr2=0x7fee8c142830 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T01:50:00.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee817fa640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fee8c11ca40 0x7fee8c142830 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.108 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:50:00.108 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:50:00.108 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:50:00.108 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "peon", 2024-09-15T01:50:00.108 INFO:teuthology.orchestra.run.smithi178.stdout: "election_epoch": 158, 2024-09-15T01:50:00.108 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum": [ 2024-09-15T01:50:00.108 INFO:teuthology.orchestra.run.smithi178.stdout: 0, 2024-09-15T01:50:00.108 INFO:teuthology.orchestra.run.smithi178.stdout: 1, 2024-09-15T01:50:00.108 INFO:teuthology.orchestra.run.smithi178.stdout: 2, 2024-09-15T01:50:00.108 INFO:teuthology.orchestra.run.smithi178.stdout: 3, 2024-09-15T01:50:00.108 INFO:teuthology.orchestra.run.smithi178.stdout: 4, 2024-09-15T01:50:00.108 INFO:teuthology.orchestra.run.smithi178.stdout: 5, 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: 6, 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: 7, 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: 8 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_age": 2, 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: "required_con": "2449958755906961412", 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: "required_mon": [ 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:50:00.109 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_con": "4540138320759226367", 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: "quorum_mon": [ 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:50:00.110 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: "outside_quorum": [], 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: "extra_probe_peers": [], 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: "sync_provider": [], 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: "monmap": { 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 1, 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: "fsid": "6ff069e2-2827-44d3-9460-72f9fc8c094b", 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: "modified": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: "created": "2024-09-15T01:31:38.100732Z", 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release": 17, 2024-09-15T01:50:00.111 INFO:teuthology.orchestra.run.smithi178.stdout: "min_mon_release_name": "quincy", 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "election_strategy": 3, 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "disallowed_leaders: ": "", 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false, 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "tiebreaker_mon": "", 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "removed_ranks: ": "", 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "features": { 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "persistent": [ 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "kraken", 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "luminous", 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "mimic", 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap-prune", 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "nautilus", 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus", 2024-09-15T01:50:00.112 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific", 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: "elector-pinging", 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy" 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: "optional": [] 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: "mons": [ 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 0, 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "a", 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:50:00.113 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3300", 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789", 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6789/0", 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6789/0", 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:50:00.114 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 1, 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "f", 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3300", 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:50:00.115 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789", 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6789/0", 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6789/0", 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 2, 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "b", 2024-09-15T01:50:00.116 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3301", 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790", 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.117 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6790/0", 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6790/0", 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 3, 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "g", 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3301", 2024-09-15T01:50:00.118 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790", 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6790/0", 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6790/0", 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:50:00.119 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 4, 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "c", 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3302", 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:50:00.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791", 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6791/0", 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6791/0", 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 5, 2024-09-15T01:50:00.121 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "h", 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3302", 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791", 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:50:00.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6791/0", 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6791/0", 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 6, 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "d", 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:50:00.123 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3303", 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792", 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6792/0", 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6792/0", 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:50:00.124 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 7, 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "i", 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:3303", 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792", 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.204:6792/0", 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.204:6792/0", 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.126 INFO:teuthology.orchestra.run.smithi178.stdout: "rank": 8, 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "e", 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addrs": { 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:3304", 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793", 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 0 2024-09-15T01:50:00.127 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6793/0", 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: "public_addr": "172.21.15.178:6793/0", 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: "priority": 0, 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: "weight": 0, 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_location": "{}" 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: "feature_map": { 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: "mon": [ 2024-09-15T01:50:00.128 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.129 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:50:00.129 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:50:00.129 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:50:00.129 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:50:00.129 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-09-15T01:50:00.129 INFO:teuthology.orchestra.run.smithi178.stdout: "client": [ 2024-09-15T01:50:00.129 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-09-15T01:50:00.129 INFO:teuthology.orchestra.run.smithi178.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T01:50:00.129 INFO:teuthology.orchestra.run.smithi178.stdout: "release": "luminous", 2024-09-15T01:50:00.129 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 1 2024-09-15T01:50:00.130 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-15T01:50:00.130 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-15T01:50:00.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-15T01:50:00.130 INFO:teuthology.orchestra.run.smithi178.stdout: "stretch_mode": false 2024-09-15T01:50:00.130 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:50:00.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee9339d640 1 -- 172.21.15.178:0/3883427457 >> v2:172.21.15.204:6812/28210 conn(0x7fee700397b0 msgr2=0x7fee7003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:00.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee9339d640 1 --2- 172.21.15.178:0/3883427457 >> v2:172.21.15.204:6812/28210 conn(0x7fee700397b0 0x7fee7003bc60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fee84034000 tx=0x7fee84037000 comp rx=0 tx=0).stop 2024-09-15T01:50:00.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee9339d640 10 monclient: shutdown 2024-09-15T01:50:00.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee9339d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:00.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee9339d640 1 -- 172.21.15.178:0/3883427457 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 msgr2=0x7fee8c09b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:00.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee9339d640 1 --2- 172.21.15.178:0/3883427457 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fee8c09faa0 0x7fee8c09b350 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fee8800ec80 tx=0x7fee880090f0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee9193a640 1 -- 172.21.15.178:0/3883427457 reap_dead start 2024-09-15T01:50:00.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee9193a640 1 -- 172.21.15.178:0/3883427457 reap_dead start 2024-09-15T01:50:00.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee9339d640 1 -- 172.21.15.178:0/3883427457 shutdown_connections 2024-09-15T01:50:00.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee9339d640 1 -- 172.21.15.178:0/3883427457 >> 172.21.15.178:0/3883427457 conn(0x7fee8c0a5ed0 msgr2=0x7fee8c0a55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:00.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee9339d640 1 -- 172.21.15.178:0/3883427457 shutdown_connections 2024-09-15T01:50:00.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.082+0000 7fee9339d640 1 -- 172.21.15.178:0/3883427457 wait complete. 2024-09-15T01:50:00.133 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T01:50:00.133 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T01:50:00.133 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T01:50:00.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.225+0000 7f30ee2c0640 10 monclient: get_monmap_and_config 2024-09-15T01:50:00.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.225+0000 7f30ee2c0640 10 monclient: build_initial_monmap 2024-09-15T01:50:00.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.225+0000 7f30ee2c0640 10 monclient: monmap: 2024-09-15T01:50:00.228 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:00.228 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:00.228 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:00.226338+0000 2024-09-15T01:50:00.228 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:00.226338+0000 2024-09-15T01:50:00.228 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:00.228 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:00.228 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:00.228 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:00.228 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:00.228 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:00.228 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:00.229 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:00.229 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:00.229 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:00.229 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:00.229 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:00.229 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:00.229 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 1 Processor -- start 2024-09-15T01:50:00.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 1 -- start start 2024-09-15T01:50:00.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 10 monclient: init 2024-09-15T01:50:00.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:00.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 10 monclient: _add_conns ranks=[4,10,3,9,0,6,8,7,2,1,5] 2024-09-15T01:50:00.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30e80a2930 0x7f30e8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 10 monclient(hunting): picked mon.noname-c con 0x7f30e80a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:00.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f30e809faa0 0x7f30e809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 10 monclient(hunting): picked mon.noname-f con 0x7f30e809faa0 addr v2:172.21.15.204:3300/0 2024-09-15T01:50:00.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e80a08c0 0x7f30e80a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 10 monclient(hunting): picked mon.noname-g con 0x7f30e80a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:00.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e809e0f0 con 0x7f30e809faa0 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e809dd80 con 0x7f30e80a08c0 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30e80a2d00 con 0x7f30e80a2930 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30ee2c0640 10 monclient(hunting): authenticate will time out at 1804.976721s 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30e80a2930 0x7f30e8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e7fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30e80a2930 0x7f30e8132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44766/0 (socket says 172.21.15.178:44766) 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e7fff640 1 -- 172.21.15.178:0/2086629767 learned_addr learned my addr 172.21.15.178:0/2086629767 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e7fff640 10 monclient(hunting): get_auth_request con 0x7f30e80a2930 auth_method 0 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e77fe640 1 --2- 172.21.15.178:0/2086629767 >> v2:172.21.15.204:3300/0 conn(0x7f30e809faa0 0x7f30e809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e77fe640 10 monclient(hunting): get_auth_request con 0x7f30e809faa0 auth_method 0 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.226+0000 7f30e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30ec85d640 1 --2- 172.21.15.178:0/2086629767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e80a08c0 0x7f30e80a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30ec85d640 10 monclient(hunting): get_auth_request con 0x7f30e80a08c0 auth_method 0 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30ec85d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30ec85d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30ec85d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e7fff640 10 monclient(hunting): handle_auth_done global_id 45067 payload 274 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e7fff640 1 -- 172.21.15.178:0/2086629767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e80a08c0 msgr2=0x7f30e80a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e7fff640 1 --2- 172.21.15.178:0/2086629767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e80a08c0 0x7f30e80a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e7fff640 1 -- 172.21.15.178:0/2086629767 >> v2:172.21.15.204:3300/0 conn(0x7f30e809faa0 msgr2=0x7f30e809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e7fff640 1 --2- 172.21.15.178:0/2086629767 >> v2:172.21.15.204:3300/0 conn(0x7f30e809faa0 0x7f30e809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e7fff640 1 monclient: found mon.noname-c 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e7fff640 1 -- 172.21.15.178:0/2086629767 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e804bf60 con 0x7f30e80a2930 2024-09-15T01:50:00.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30ec85d640 1 --2- 172.21.15.178:0/2086629767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e80a08c0 0x7f30e80a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.227+0000 7f30e7fff640 1 --2- 172.21.15.178:0/2086629767 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30e80a2930 0x7f30e8132de0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009910 tx=0x7f30d80091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8998c514f3050fc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30e6ffd640 1 -- 172.21.15.178:0/2086629767 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30d802ac80 con 0x7f30e80a2930 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30e6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30e6ffd640 10 monclient: dump: 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:00.242 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30e6ffd640 10 monclient: _finish_auth 0 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 5 monclient: authenticate success, global_id 45067 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30e6ffd640 1 -- 172.21.15.178:0/2086629767 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d802ae00 con 0x7f30e80a2930 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:00.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30e6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 4 set_mon_vals no callback set 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30e6ffd640 1 -- 172.21.15.178:0/2086629767 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30d8031ca0 con 0x7f30e80a2930 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30e6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30e6ffd640 10 monclient: dump: 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:00.244 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 10 monclient: shutdown 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2086629767 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30e80a2930 msgr2=0x7f30e8132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 1 --2- 172.21.15.178:0/2086629767 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30e80a2930 0x7f30e8132de0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009910 tx=0x7f30d80091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2086629767 shutdown_connections 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 1 --2- 172.21.15.178:0/2086629767 >> v2:172.21.15.204:3300/0 conn(0x7f30e809faa0 0x7f30e809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 1 --2- 172.21.15.178:0/2086629767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e80a08c0 0x7f30e80a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 1 --2- 172.21.15.178:0/2086629767 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f30e80a2930 0x7f30e8132de0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2086629767 >> 172.21.15.178:0/2086629767 conn(0x7f30e80a5ed0 msgr2=0x7f30e80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2086629767 shutdown_connections 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.228+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2086629767 wait complete. 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.229+0000 7f30ee2c0640 10 monclient: build_initial_monmap 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.229+0000 7f30ee2c0640 10 monclient: monmap: 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:00.246 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.229+0000 7f30ee2c0640 1 Processor -- start 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.229+0000 7f30ee2c0640 1 -- start start 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.229+0000 7f30ee2c0640 10 monclient: init 2024-09-15T01:50:00.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 10 monclient: _add_conns ranks=[0,5,2,6,4,8,1,3,7] 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30e809faa0 0x7f30e809b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 10 monclient(hunting): picked mon.noname-a con 0x7f30e809faa0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30e80a08c0 0x7f30e81318b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 10 monclient(hunting): picked mon.noname-h con 0x7f30e80a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30e80a2930 0x7f30e811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 10 monclient(hunting): picked mon.noname-b con 0x7f30e80a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e8142c00 con 0x7f30e809faa0 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e809d630 con 0x7f30e80a2930 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:00.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30e811ef90 con 0x7f30e80a08c0 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ee2c0640 10 monclient(hunting): authenticate will time out at 1804.980241s 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30e809faa0 0x7f30e809b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ec85d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30e80a2930 0x7f30e811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30e809faa0 0x7f30e809b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35082/0 (socket says 172.21.15.178:35082) 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 1 -- 172.21.15.178:0/2328137643 learned_addr learned my addr 172.21.15.178:0/2328137643 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 10 monclient(hunting): get_auth_request con 0x7f30e809faa0 auth_method 0 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ec85d640 10 monclient(hunting): get_auth_request con 0x7f30e80a2930 auth_method 0 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ec85d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ec85d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ec85d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ec85d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ec85d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ec85d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e77fe640 1 --2- 172.21.15.178:0/2328137643 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30e80a08c0 0x7f30e81318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e77fe640 10 monclient(hunting): get_auth_request con 0x7f30e80a08c0 auth_method 0 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 10 monclient(hunting): handle_auth_done global_id 104175 payload 824 2024-09-15T01:50:00.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 1 -- 172.21.15.178:0/2328137643 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30e80a08c0 msgr2=0x7f30e81318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 1 --2- 172.21.15.178:0/2328137643 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30e80a08c0 0x7f30e81318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 1 -- 172.21.15.178:0/2328137643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30e80a2930 msgr2=0x7f30e811c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 1 --2- 172.21.15.178:0/2328137643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30e80a2930 0x7f30e811c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 10 monclient: _finish_hunting 0 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 1 monclient: found mon.noname-a 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 1 -- 172.21.15.178:0/2328137643 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d8009450 con 0x7f30e809faa0 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30ec85d640 1 --2- 172.21.15.178:0/2328137643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30e80a2930 0x7f30e811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e77fe640 1 --2- 172.21.15.178:0/2328137643 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30e80a08c0 0x7f30e81318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30e7fff640 1 --2- 172.21.15.178:0/2328137643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30e809faa0 0x7f30e809b0f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30d8000c00 tx=0x7f30d8032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30cffff640 1 -- 172.21.15.178:0/2328137643 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30d8031820 con 0x7f30e809faa0 2024-09-15T01:50:00.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.230+0000 7f30cffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30cffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30cffff640 10 monclient: dump: 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:00.252 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30cffff640 10 monclient: _finish_auth 0 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30cffff640 1 -- 172.21.15.178:0/2328137643 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d80319a0 con 0x7f30e809faa0 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30cffff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30cffff640 1 -- 172.21.15.178:0/2328137643 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30d803a390 con 0x7f30e809faa0 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30cffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30cffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30cffff640 10 monclient: dump: 2024-09-15T01:50:00.253 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30ee2c0640 5 monclient: authenticate success, global_id 104175 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30ee2c0640 10 monclient: _renew_subs 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30e67fc640 4 set_mon_vals no callback set 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30ee2c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2328137643 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e811e9d0 con 0x7f30e809faa0 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30ee2c0640 10 monclient: _renew_subs 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30ee2c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.231+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2328137643 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e8142330 con 0x7f30e809faa0 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.232+0000 7f30cffff640 1 -- 172.21.15.178:0/2328137643 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f30d8039040 con 0x7f30e809faa0 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.232+0000 7f30ee2c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.232+0000 7f30ee2c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.232+0000 7f30ee2c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:00.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.232+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2328137643 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e811f0d0 con 0x7f30e809faa0 2024-09-15T01:50:00.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.232+0000 7f30cffff640 1 --2- 172.21.15.178:0/2328137643 >> v2:172.21.15.204:6812/28210 conn(0x7f30d40396e0 0x7f30d403bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:00.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.232+0000 7f30cffff640 1 -- 172.21.15.178:0/2328137643 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f30d806ae40 con 0x7f30e809faa0 2024-09-15T01:50:00.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.232+0000 7f30e77fe640 1 --2- 172.21.15.178:0/2328137643 >> v2:172.21.15.204:6812/28210 conn(0x7f30d40396e0 0x7f30d403bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:00.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.232+0000 7f30e77fe640 10 monclient: get_auth_request con 0x7f30d40396e0 auth_method 0 2024-09-15T01:50:00.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.233+0000 7f30e77fe640 1 --2- 172.21.15.178:0/2328137643 >> v2:172.21.15.204:6812/28210 conn(0x7f30d40396e0 0x7f30d403bb90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f30e80a6ec0 tx=0x7f30dc008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:00.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.236+0000 7f30cffff640 1 -- 172.21.15.178:0/2328137643 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f30e811f0d0 con 0x7f30e809faa0 2024-09-15T01:50:00.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.236+0000 7f30cffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:00.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.236+0000 7f30cffff640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:00.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.450+0000 7f30ee2c0640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T01:50:00.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.450+0000 7f30ee2c0640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T01:50:00.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.450+0000 7f30ee2c0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:00.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.450+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2328137643 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f30e811f0d0 con 0x7f30e809faa0 2024-09-15T01:50:00.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.452+0000 7f30cffff640 1 -- 172.21.15.178:0/2328137643 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f30e811f0d0 con 0x7f30e809faa0 2024-09-15T01:50:00.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.453+0000 7f30cffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T01:50:00.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.453+0000 7f30cffff640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:50:00.457 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.458 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=29,config=2,health=11,kv=8,logm=50} crc {auth=3567547397,config=3140514009,health=1552921670,kv=706962213,logm=2292351123}) 2024-09-15T01:50:00.458 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=29,config=2,health=12,kv=8,logm=49} crc {auth=3484855647,config=3140514009,health=2891791466,kv=706962214,logm=1376018517}) 2024-09-15T01:50:00.458 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.458 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=29,config=2,health=11,kv=8,logm=50} crc {auth=3567547397,config=3140514009,health=1552921670,kv=706962213,logm=2292351123}) 2024-09-15T01:50:00.458 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=29,config=2,health=11,kv=8,logm=50} crc {auth=2574094526,config=3140514009,health=1844703166,kv=706962213,logm=3227035212}) 2024-09-15T01:50:00.458 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.458 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=29,config=2,health=11,kv=8,logm=50} crc {auth=3567547397,config=3140514009,health=1552921670,kv=706962213,logm=2292351123}) 2024-09-15T01:50:00.458 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=29,config=2,health=12,kv=8,logm=49} crc {auth=1767402915,config=3140514009,health=2891791466,kv=706962213,logm=2074229669}) 2024-09-15T01:50:00.458 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.458 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=29,config=2,health=11,kv=8,logm=50} crc {auth=3567547397,config=3140514009,health=1552921670,kv=706962213,logm=2292351123}) 2024-09-15T01:50:00.459 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=29,config=2,health=12,kv=8,logm=49} crc {auth=933237875,config=3140514009,health=2891791466,kv=706962213,logm=1097067291}) 2024-09-15T01:50:00.459 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.459 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=29,config=2,health=11,kv=8,logm=50} crc {auth=3567547397,config=3140514009,health=1552921670,kv=706962213,logm=2292351123}) 2024-09-15T01:50:00.459 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=27,config=2,health=11,kv=7,logm=53} crc {auth=3411615233,config=3140514009,health=1552921670,kv=2892508859,logm=189482582}) 2024-09-15T01:50:00.459 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.459 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=29,config=2,health=11,kv=8,logm=50} crc {auth=3567547397,config=3140514009,health=1552921670,kv=706962213,logm=2292351123}) 2024-09-15T01:50:00.459 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=28,config=2,health=11,kv=6,logm=53} crc {auth=3885361307,config=3140514009,health=1844703166,kv=3871184673,logm=2325206995}) 2024-09-15T01:50:00.459 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.459 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=29,config=2,health=11,kv=8,logm=50} crc {auth=3567547397,config=3140514009,health=1552921670,kv=706962213,logm=2292351123}) 2024-09-15T01:50:00.459 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=30,config=1,health=11,kv=8,logm=50} crc {auth=2616235425,config=1228700967,health=416703007,kv=706962213,logm=3172167879}) 2024-09-15T01:50:00.459 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.459 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=29,config=2,health=11,kv=8,logm=50} crc {auth=3567547397,config=3140514009,health=1552921670,kv=706962213,logm=2292351123}) 2024-09-15T01:50:00.460 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.453+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=30,config=2,health=11,kv=8,logm=49} crc {auth=2616235425,config=3140514009,health=2496068776,kv=706962213,logm=1465235937}) 2024-09-15T01:50:00.460 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.460 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2829222456}) 2024-09-15T01:50:00.460 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3790159567}) 2024-09-15T01:50:00.460 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.460 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2829222456}) 2024-09-15T01:50:00.460 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2435177644}) 2024-09-15T01:50:00.460 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.460 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2829222456}) 2024-09-15T01:50:00.460 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1292632633}) 2024-09-15T01:50:00.460 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.461 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2829222456}) 2024-09-15T01:50:00.461 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1397735816}) 2024-09-15T01:50:00.461 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.461 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2829222456}) 2024-09-15T01:50:00.461 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=681553964}) 2024-09-15T01:50:00.461 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.461 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2829222456}) 2024-09-15T01:50:00.461 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2184362925}) 2024-09-15T01:50:00.461 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.461 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2829222456}) 2024-09-15T01:50:00.461 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2404613439}) 2024-09-15T01:50:00.462 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.462 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2829222456}) 2024-09-15T01:50:00.462 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.455+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1612638873}) 2024-09-15T01:50:00.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2328137643 >> v2:172.21.15.204:6812/28210 conn(0x7f30d40396e0 msgr2=0x7f30d403bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:00.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 1 --2- 172.21.15.178:0/2328137643 >> v2:172.21.15.204:6812/28210 conn(0x7f30d40396e0 0x7f30d403bb90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f30e80a6ec0 tx=0x7f30dc008040 comp rx=0 tx=0).stop 2024-09-15T01:50:00.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 10 monclient: shutdown 2024-09-15T01:50:00.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:00.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2328137643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30e809faa0 msgr2=0x7f30e809b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:00.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 1 --2- 172.21.15.178:0/2328137643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30e809faa0 0x7f30e809b0f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30d8000c00 tx=0x7f30d8032000 comp rx=0 tx=0).stop 2024-09-15T01:50:00.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2328137643 shutdown_connections 2024-09-15T01:50:00.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 1 --2- 172.21.15.178:0/2328137643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30e80a2930 0x7f30e811c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 1 --2- 172.21.15.178:0/2328137643 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f30e80a08c0 0x7f30e81318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 1 --2- 172.21.15.178:0/2328137643 >> v2:172.21.15.204:6812/28210 conn(0x7f30d40396e0 0x7f30d403bb90 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 1 --2- 172.21.15.178:0/2328137643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30e809faa0 0x7f30e809b0f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:00.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2328137643 >> 172.21.15.178:0/2328137643 conn(0x7f30e80a5ed0 msgr2=0x7f30e8135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:00.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2328137643 shutdown_connections 2024-09-15T01:50:00.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:00.455+0000 7f30ee2c0640 1 -- 172.21.15.178:0/2328137643 wait complete. 2024-09-15T01:50:00.477 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T01:50:00.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2415039567}) 2024-09-15T01:50:00.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777297710}) 2024-09-15T01:50:00.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2415039567}) 2024-09-15T01:50:00.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3025784839}) 2024-09-15T01:50:00.748 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2415039567}) 2024-09-15T01:50:00.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3225819129}) 2024-09-15T01:50:00.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2415039567}) 2024-09-15T01:50:00.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1471271188}) 2024-09-15T01:50:00.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2415039567}) 2024-09-15T01:50:00.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2767075942}) 2024-09-15T01:50:00.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2415039567}) 2024-09-15T01:50:00.749 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2846711568}) 2024-09-15T01:50:00.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2415039567}) 2024-09-15T01:50:00.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=709974704}) 2024-09-15T01:50:00.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2415039567}) 2024-09-15T01:50:00.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.457+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2269625672}) 2024-09-15T01:50:00.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1075569927}) 2024-09-15T01:50:00.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3502672000}) 2024-09-15T01:50:00.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.750 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1075569927}) 2024-09-15T01:50:00.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3835501097}) 2024-09-15T01:50:00.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1075569927}) 2024-09-15T01:50:00.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1200379688}) 2024-09-15T01:50:00.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1075569927}) 2024-09-15T01:50:00.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1876116013}) 2024-09-15T01:50:00.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1075569927}) 2024-09-15T01:50:00.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2819752670}) 2024-09-15T01:50:00.751 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1075569927}) 2024-09-15T01:50:00.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1226595551}) 2024-09-15T01:50:00.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1075569927}) 2024-09-15T01:50:00.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1920185344}) 2024-09-15T01:50:00.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1075569927}) 2024-09-15T01:50:00.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.459+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4035706223}) 2024-09-15T01:50:00.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2464209655}) 2024-09-15T01:50:00.752 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=292060214}) 2024-09-15T01:50:00.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2464209655}) 2024-09-15T01:50:00.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=950252247}) 2024-09-15T01:50:00.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2464209655}) 2024-09-15T01:50:00.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2427502876}) 2024-09-15T01:50:00.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2464209655}) 2024-09-15T01:50:00.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2059929063}) 2024-09-15T01:50:00.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2464209655}) 2024-09-15T01:50:00.753 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3425514883}) 2024-09-15T01:50:00.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2464209655}) 2024-09-15T01:50:00.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2430998435}) 2024-09-15T01:50:00.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2464209655}) 2024-09-15T01:50:00.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=103390136}) 2024-09-15T01:50:00.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2464209655}) 2024-09-15T01:50:00.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.462+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4008765388}) 2024-09-15T01:50:00.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.463+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.754 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.463+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1065770581}) 2024-09-15T01:50:00.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3516502291}) 2024-09-15T01:50:00.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1065770581}) 2024-09-15T01:50:00.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2678356694}) 2024-09-15T01:50:00.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1065770581}) 2024-09-15T01:50:00.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=8105486}) 2024-09-15T01:50:00.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1065770581}) 2024-09-15T01:50:00.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3239161218}) 2024-09-15T01:50:00.755 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1065770581}) 2024-09-15T01:50:00.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4009777967}) 2024-09-15T01:50:00.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1065770581}) 2024-09-15T01:50:00.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3429049879}) 2024-09-15T01:50:00.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1065770581}) 2024-09-15T01:50:00.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1674630409}) 2024-09-15T01:50:00.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1065770581}) 2024-09-15T01:50:00.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.464+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3235848604}) 2024-09-15T01:50:00.756 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1514818347}) 2024-09-15T01:50:00.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=157417671}) 2024-09-15T01:50:00.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1514818347}) 2024-09-15T01:50:00.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3019795175}) 2024-09-15T01:50:00.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1514818347}) 2024-09-15T01:50:00.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=585677809}) 2024-09-15T01:50:00.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1514818347}) 2024-09-15T01:50:00.757 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3718601040}) 2024-09-15T01:50:00.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1514818347}) 2024-09-15T01:50:00.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2345386137}) 2024-09-15T01:50:00.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1514818347}) 2024-09-15T01:50:00.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1912243561}) 2024-09-15T01:50:00.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1514818347}) 2024-09-15T01:50:00.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=564864511}) 2024-09-15T01:50:00.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.758 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1514818347}) 2024-09-15T01:50:00.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.465+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=735892140}) 2024-09-15T01:50:00.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3056552791}) 2024-09-15T01:50:00.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2590333557}) 2024-09-15T01:50:00.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3056552791}) 2024-09-15T01:50:00.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=266548187}) 2024-09-15T01:50:00.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3056552791}) 2024-09-15T01:50:00.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3565441599}) 2024-09-15T01:50:00.759 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3056552791}) 2024-09-15T01:50:00.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2490193542}) 2024-09-15T01:50:00.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3056552791}) 2024-09-15T01:50:00.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2453948548}) 2024-09-15T01:50:00.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3056552791}) 2024-09-15T01:50:00.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2934881637}) 2024-09-15T01:50:00.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.760 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3056552791}) 2024-09-15T01:50:00.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4006892110}) 2024-09-15T01:50:00.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3056552791}) 2024-09-15T01:50:00.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.467+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1049171683}) 2024-09-15T01:50:00.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2093532369}) 2024-09-15T01:50:00.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1192595421}) 2024-09-15T01:50:00.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2093532369}) 2024-09-15T01:50:00.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4184116704}) 2024-09-15T01:50:00.761 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2093532369}) 2024-09-15T01:50:00.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1133772630}) 2024-09-15T01:50:00.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2093532369}) 2024-09-15T01:50:00.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4158944158}) 2024-09-15T01:50:00.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2093532369}) 2024-09-15T01:50:00.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3655173196}) 2024-09-15T01:50:00.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2093532369}) 2024-09-15T01:50:00.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=578852319}) 2024-09-15T01:50:00.762 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2093532369}) 2024-09-15T01:50:00.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=245831585}) 2024-09-15T01:50:00.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2093532369}) 2024-09-15T01:50:00.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.469+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2324817237}) 2024-09-15T01:50:00.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=640520291}) 2024-09-15T01:50:00.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3196871778}) 2024-09-15T01:50:00.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=640520291}) 2024-09-15T01:50:00.763 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3558851376}) 2024-09-15T01:50:00.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=640520291}) 2024-09-15T01:50:00.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2119440655}) 2024-09-15T01:50:00.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=640520291}) 2024-09-15T01:50:00.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4208761443}) 2024-09-15T01:50:00.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=640520291}) 2024-09-15T01:50:00.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3964765595}) 2024-09-15T01:50:00.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=640520291}) 2024-09-15T01:50:00.764 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3394702735}) 2024-09-15T01:50:00.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=640520291}) 2024-09-15T01:50:00.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=850529079}) 2024-09-15T01:50:00.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=640520291}) 2024-09-15T01:50:00.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.470+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3416464371}) 2024-09-15T01:50:00.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1307357860}) 2024-09-15T01:50:00.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2756368474}) 2024-09-15T01:50:00.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.765 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1307357860}) 2024-09-15T01:50:00.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1983342946}) 2024-09-15T01:50:00.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1307357860}) 2024-09-15T01:50:00.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3885000708}) 2024-09-15T01:50:00.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1307357860}) 2024-09-15T01:50:00.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2496632638}) 2024-09-15T01:50:00.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1307357860}) 2024-09-15T01:50:00.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2749603214}) 2024-09-15T01:50:00.766 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1307357860}) 2024-09-15T01:50:00.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=630915455}) 2024-09-15T01:50:00.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1307357860}) 2024-09-15T01:50:00.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.471+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=969907123}) 2024-09-15T01:50:00.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1307357860}) 2024-09-15T01:50:00.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.472+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2748356047}) 2024-09-15T01:50:00.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3961566422}) 2024-09-15T01:50:00.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882274643}) 2024-09-15T01:50:00.767 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3961566422}) 2024-09-15T01:50:00.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=696073816}) 2024-09-15T01:50:00.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3961566422}) 2024-09-15T01:50:00.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3680317935}) 2024-09-15T01:50:00.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3961566422}) 2024-09-15T01:50:00.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=137174156}) 2024-09-15T01:50:00.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3961566422}) 2024-09-15T01:50:00.768 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1082400570}) 2024-09-15T01:50:00.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3961566422}) 2024-09-15T01:50:00.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3191887279}) 2024-09-15T01:50:00.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3961566422}) 2024-09-15T01:50:00.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2243004351}) 2024-09-15T01:50:00.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3961566422}) 2024-09-15T01:50:00.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.473+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2334719407}) 2024-09-15T01:50:00.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.769 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194485025}) 2024-09-15T01:50:00.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1495640980}) 2024-09-15T01:50:00.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194485025}) 2024-09-15T01:50:00.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3028489171}) 2024-09-15T01:50:00.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194485025}) 2024-09-15T01:50:00.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=32113516}) 2024-09-15T01:50:00.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194485025}) 2024-09-15T01:50:00.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3806612245}) 2024-09-15T01:50:00.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.770 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194485025}) 2024-09-15T01:50:00.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1141628601}) 2024-09-15T01:50:00.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194485025}) 2024-09-15T01:50:00.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=310647599}) 2024-09-15T01:50:00.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194485025}) 2024-09-15T01:50:00.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2594121091}) 2024-09-15T01:50:00.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194485025}) 2024-09-15T01:50:00.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.474+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=229591262}) 2024-09-15T01:50:00.771 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3153347820}) 2024-09-15T01:50:00.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2238383128}) 2024-09-15T01:50:00.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3153347820}) 2024-09-15T01:50:00.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=332038188}) 2024-09-15T01:50:00.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3153347820}) 2024-09-15T01:50:00.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3160599858}) 2024-09-15T01:50:00.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.772 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3153347820}) 2024-09-15T01:50:00.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=71757183}) 2024-09-15T01:50:00.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3153347820}) 2024-09-15T01:50:00.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3212908670}) 2024-09-15T01:50:00.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3153347820}) 2024-09-15T01:50:00.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4059815151}) 2024-09-15T01:50:00.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3153347820}) 2024-09-15T01:50:00.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=660348219}) 2024-09-15T01:50:00.773 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3153347820}) 2024-09-15T01:50:00.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.475+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=676238991}) 2024-09-15T01:50:00.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283608902}) 2024-09-15T01:50:00.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3229567235}) 2024-09-15T01:50:00.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283608902}) 2024-09-15T01:50:00.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.476+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4206340333}) 2024-09-15T01:50:00.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283608902}) 2024-09-15T01:50:00.774 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=610048125}) 2024-09-15T01:50:00.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283608902}) 2024-09-15T01:50:00.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=202782037}) 2024-09-15T01:50:00.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283608902}) 2024-09-15T01:50:00.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2820831059}) 2024-09-15T01:50:00.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283608902}) 2024-09-15T01:50:00.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2166389376}) 2024-09-15T01:50:00.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.775 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283608902}) 2024-09-15T01:50:00.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1809986840}) 2024-09-15T01:50:00.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283608902}) 2024-09-15T01:50:00.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.477+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=88770992}) 2024-09-15T01:50:00.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2462574225}) 2024-09-15T01:50:00.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3017936375}) 2024-09-15T01:50:00.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2462574225}) 2024-09-15T01:50:00.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1119593312}) 2024-09-15T01:50:00.776 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2462574225}) 2024-09-15T01:50:00.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1387647164}) 2024-09-15T01:50:00.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2462574225}) 2024-09-15T01:50:00.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1085894028}) 2024-09-15T01:50:00.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2462574225}) 2024-09-15T01:50:00.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2198454078}) 2024-09-15T01:50:00.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2462574225}) 2024-09-15T01:50:00.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2545281467}) 2024-09-15T01:50:00.777 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2462574225}) 2024-09-15T01:50:00.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4244572302}) 2024-09-15T01:50:00.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2462574225}) 2024-09-15T01:50:00.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.478+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3381068416}) 2024-09-15T01:50:00.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2645778324}) 2024-09-15T01:50:00.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2133779091}) 2024-09-15T01:50:00.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.778 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2645778324}) 2024-09-15T01:50:00.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.479+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=299246269}) 2024-09-15T01:50:00.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2645778324}) 2024-09-15T01:50:00.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1725493543}) 2024-09-15T01:50:00.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2645778324}) 2024-09-15T01:50:00.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3042314984}) 2024-09-15T01:50:00.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2645778324}) 2024-09-15T01:50:00.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1132956437}) 2024-09-15T01:50:00.779 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2645778324}) 2024-09-15T01:50:00.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3713691291}) 2024-09-15T01:50:00.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2645778324}) 2024-09-15T01:50:00.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2972024927}) 2024-09-15T01:50:00.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2645778324}) 2024-09-15T01:50:00.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.480+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1441664700}) 2024-09-15T01:50:00.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4032671765}) 2024-09-15T01:50:00.780 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1416204698}) 2024-09-15T01:50:00.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4032671765}) 2024-09-15T01:50:00.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1589421419}) 2024-09-15T01:50:00.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4032671765}) 2024-09-15T01:50:00.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1613712758}) 2024-09-15T01:50:00.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4032671765}) 2024-09-15T01:50:00.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=498212931}) 2024-09-15T01:50:00.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4032671765}) 2024-09-15T01:50:00.781 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3842511576}) 2024-09-15T01:50:00.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4032671765}) 2024-09-15T01:50:00.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3491070846}) 2024-09-15T01:50:00.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4032671765}) 2024-09-15T01:50:00.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.481+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3913705503}) 2024-09-15T01:50:00.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4032671765}) 2024-09-15T01:50:00.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.482+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3159869532}) 2024-09-15T01:50:00.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.782 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3223127306}) 2024-09-15T01:50:00.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231253073}) 2024-09-15T01:50:00.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3223127306}) 2024-09-15T01:50:00.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3936774385}) 2024-09-15T01:50:00.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3223127306}) 2024-09-15T01:50:00.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1447186872}) 2024-09-15T01:50:00.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3223127306}) 2024-09-15T01:50:00.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1838194024}) 2024-09-15T01:50:00.783 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3223127306}) 2024-09-15T01:50:00.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3498080553}) 2024-09-15T01:50:00.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3223127306}) 2024-09-15T01:50:00.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4279938341}) 2024-09-15T01:50:00.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3223127306}) 2024-09-15T01:50:00.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1215303718}) 2024-09-15T01:50:00.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3223127306}) 2024-09-15T01:50:00.784 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.483+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=923400491}) 2024-09-15T01:50:00.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4008711814}) 2024-09-15T01:50:00.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1799150797}) 2024-09-15T01:50:00.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4008711814}) 2024-09-15T01:50:00.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1833764442}) 2024-09-15T01:50:00.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4008711814}) 2024-09-15T01:50:00.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=579471156}) 2024-09-15T01:50:00.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4008711814}) 2024-09-15T01:50:00.785 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3797060985}) 2024-09-15T01:50:00.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4008711814}) 2024-09-15T01:50:00.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3594477956}) 2024-09-15T01:50:00.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4008711814}) 2024-09-15T01:50:00.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3244863931}) 2024-09-15T01:50:00.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4008711814}) 2024-09-15T01:50:00.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1637458923}) 2024-09-15T01:50:00.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.786 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4008711814}) 2024-09-15T01:50:00.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.484+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3275768183}) 2024-09-15T01:50:00.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3722181867}) 2024-09-15T01:50:00.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3552116221}) 2024-09-15T01:50:00.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3722181867}) 2024-09-15T01:50:00.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2948286762}) 2024-09-15T01:50:00.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3722181867}) 2024-09-15T01:50:00.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3583072900}) 2024-09-15T01:50:00.787 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.788 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3722181867}) 2024-09-15T01:50:00.788 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1999009886}) 2024-09-15T01:50:00.788 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.788 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3722181867}) 2024-09-15T01:50:00.788 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3625619304}) 2024-09-15T01:50:00.788 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.788 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3722181867}) 2024-09-15T01:50:00.788 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1519418060}) 2024-09-15T01:50:00.788 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.485+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.788 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3722181867}) 2024-09-15T01:50:00.788 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=875106405}) 2024-09-15T01:50:00.789 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.789 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3722181867}) 2024-09-15T01:50:00.789 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.486+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=564574908}) 2024-09-15T01:50:00.789 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.789 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=401531482}) 2024-09-15T01:50:00.789 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2996690178}) 2024-09-15T01:50:00.789 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.789 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=401531482}) 2024-09-15T01:50:00.789 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=611718238}) 2024-09-15T01:50:00.789 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.789 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=401531482}) 2024-09-15T01:50:00.789 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2707901018}) 2024-09-15T01:50:00.790 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.790 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=401531482}) 2024-09-15T01:50:00.790 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3297741149}) 2024-09-15T01:50:00.790 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.790 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=401531482}) 2024-09-15T01:50:00.790 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1936740776}) 2024-09-15T01:50:00.790 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.790 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=401531482}) 2024-09-15T01:50:00.790 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1227600837}) 2024-09-15T01:50:00.790 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.790 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=401531482}) 2024-09-15T01:50:00.791 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3755934275}) 2024-09-15T01:50:00.791 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.791 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=401531482}) 2024-09-15T01:50:00.791 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.487+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2937669374}) 2024-09-15T01:50:00.791 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.791 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2235113592}) 2024-09-15T01:50:00.791 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=862340969}) 2024-09-15T01:50:00.791 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.791 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2235113592}) 2024-09-15T01:50:00.791 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3319345591}) 2024-09-15T01:50:00.791 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.792 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2235113592}) 2024-09-15T01:50:00.792 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4249730922}) 2024-09-15T01:50:00.792 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.792 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2235113592}) 2024-09-15T01:50:00.792 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2280962432}) 2024-09-15T01:50:00.792 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.792 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2235113592}) 2024-09-15T01:50:00.792 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=989393968}) 2024-09-15T01:50:00.792 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.792 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2235113592}) 2024-09-15T01:50:00.792 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3589371864}) 2024-09-15T01:50:00.792 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.793 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2235113592}) 2024-09-15T01:50:00.793 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1068915012}) 2024-09-15T01:50:00.793 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.793 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2235113592}) 2024-09-15T01:50:00.793 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.488+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2512030418}) 2024-09-15T01:50:00.793 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.793 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3768649263}) 2024-09-15T01:50:00.793 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1774824658}) 2024-09-15T01:50:00.793 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.793 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3768649263}) 2024-09-15T01:50:00.793 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2767411751}) 2024-09-15T01:50:00.794 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.794 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3768649263}) 2024-09-15T01:50:00.794 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3491768947}) 2024-09-15T01:50:00.794 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.794 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3768649263}) 2024-09-15T01:50:00.794 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2368014061}) 2024-09-15T01:50:00.794 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.794 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3768649263}) 2024-09-15T01:50:00.794 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2181487892}) 2024-09-15T01:50:00.794 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.794 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3768649263}) 2024-09-15T01:50:00.794 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=757859821}) 2024-09-15T01:50:00.795 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.795 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3768649263}) 2024-09-15T01:50:00.795 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=853056522}) 2024-09-15T01:50:00.795 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.795 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3768649263}) 2024-09-15T01:50:00.795 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.489+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4007096280}) 2024-09-15T01:50:00.795 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.795 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3279816329}) 2024-09-15T01:50:00.795 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=382557985}) 2024-09-15T01:50:00.795 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.795 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3279816329}) 2024-09-15T01:50:00.795 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1860358650}) 2024-09-15T01:50:00.796 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.796 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3279816329}) 2024-09-15T01:50:00.796 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3328897775}) 2024-09-15T01:50:00.796 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.796 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.490+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3279816329}) 2024-09-15T01:50:00.796 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1544045566}) 2024-09-15T01:50:00.796 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.796 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3279816329}) 2024-09-15T01:50:00.796 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3057457070}) 2024-09-15T01:50:00.797 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.797 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3279816329}) 2024-09-15T01:50:00.797 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1090344754}) 2024-09-15T01:50:00.797 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.797 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3279816329}) 2024-09-15T01:50:00.797 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3918344842}) 2024-09-15T01:50:00.797 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.797 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3279816329}) 2024-09-15T01:50:00.797 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.491+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2589207130}) 2024-09-15T01:50:00.797 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.797 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1813119466}) 2024-09-15T01:50:00.798 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2871146670}) 2024-09-15T01:50:00.798 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.798 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1813119466}) 2024-09-15T01:50:00.798 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2951114310}) 2024-09-15T01:50:00.798 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.798 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1813119466}) 2024-09-15T01:50:00.798 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2519325744}) 2024-09-15T01:50:00.798 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.798 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1813119466}) 2024-09-15T01:50:00.798 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2691582179}) 2024-09-15T01:50:00.798 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.799 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1813119466}) 2024-09-15T01:50:00.799 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2733348921}) 2024-09-15T01:50:00.799 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.799 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1813119466}) 2024-09-15T01:50:00.799 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=837540266}) 2024-09-15T01:50:00.799 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.799 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1813119466}) 2024-09-15T01:50:00.799 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1036265276}) 2024-09-15T01:50:00.799 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.799 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1813119466}) 2024-09-15T01:50:00.799 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.492+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3518031387}) 2024-09-15T01:50:00.799 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.800 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178923855}) 2024-09-15T01:50:00.800 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=942937707}) 2024-09-15T01:50:00.800 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.800 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178923855}) 2024-09-15T01:50:00.800 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2622271735}) 2024-09-15T01:50:00.800 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.800 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178923855}) 2024-09-15T01:50:00.800 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2403235368}) 2024-09-15T01:50:00.800 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.800 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178923855}) 2024-09-15T01:50:00.800 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2556965986}) 2024-09-15T01:50:00.800 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.801 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178923855}) 2024-09-15T01:50:00.801 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3470096894}) 2024-09-15T01:50:00.801 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.801 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178923855}) 2024-09-15T01:50:00.801 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3780329114}) 2024-09-15T01:50:00.801 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.801 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178923855}) 2024-09-15T01:50:00.801 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2167354791}) 2024-09-15T01:50:00.801 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.801 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178923855}) 2024-09-15T01:50:00.801 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.493+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3385464181}) 2024-09-15T01:50:00.802 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.802 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=876517773}) 2024-09-15T01:50:00.802 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3497809611}) 2024-09-15T01:50:00.802 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.802 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=876517773}) 2024-09-15T01:50:00.802 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3349378551}) 2024-09-15T01:50:00.802 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.802 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=876517773}) 2024-09-15T01:50:00.802 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2930792010}) 2024-09-15T01:50:00.802 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.802 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=876517773}) 2024-09-15T01:50:00.803 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3120954730}) 2024-09-15T01:50:00.803 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.803 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=876517773}) 2024-09-15T01:50:00.803 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3308438676}) 2024-09-15T01:50:00.803 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.803 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=876517773}) 2024-09-15T01:50:00.803 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3468026901}) 2024-09-15T01:50:00.803 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.803 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=876517773}) 2024-09-15T01:50:00.803 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1686181598}) 2024-09-15T01:50:00.803 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.804 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=876517773}) 2024-09-15T01:50:00.804 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.494+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2561614553}) 2024-09-15T01:50:00.804 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.804 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3345232603}) 2024-09-15T01:50:00.804 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=867122726}) 2024-09-15T01:50:00.804 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.804 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3345232603}) 2024-09-15T01:50:00.804 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1964972346}) 2024-09-15T01:50:00.804 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.804 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3345232603}) 2024-09-15T01:50:00.804 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3240941702}) 2024-09-15T01:50:00.805 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.805 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3345232603}) 2024-09-15T01:50:00.805 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=821861778}) 2024-09-15T01:50:00.805 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.805 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3345232603}) 2024-09-15T01:50:00.805 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.495+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2423925803}) 2024-09-15T01:50:00.805 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.805 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3345232603}) 2024-09-15T01:50:00.805 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1968787814}) 2024-09-15T01:50:00.805 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.805 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3345232603}) 2024-09-15T01:50:00.806 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=78095495}) 2024-09-15T01:50:00.806 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.806 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3345232603}) 2024-09-15T01:50:00.806 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.496+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1719552987}) 2024-09-15T01:50:00.806 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.806 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3563528332}) 2024-09-15T01:50:00.806 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3872013372}) 2024-09-15T01:50:00.806 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.806 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3563528332}) 2024-09-15T01:50:00.806 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1522265029}) 2024-09-15T01:50:00.806 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.807 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3563528332}) 2024-09-15T01:50:00.807 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1404566255}) 2024-09-15T01:50:00.807 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.807 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3563528332}) 2024-09-15T01:50:00.807 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2364059686}) 2024-09-15T01:50:00.807 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.807 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3563528332}) 2024-09-15T01:50:00.807 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=557449305}) 2024-09-15T01:50:00.807 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.807 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3563528332}) 2024-09-15T01:50:00.807 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2116273999}) 2024-09-15T01:50:00.807 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.808 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3563528332}) 2024-09-15T01:50:00.808 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=731033419}) 2024-09-15T01:50:00.808 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.808 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3563528332}) 2024-09-15T01:50:00.808 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.497+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=580605215}) 2024-09-15T01:50:00.808 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.808 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1265425007}) 2024-09-15T01:50:00.808 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2925914156}) 2024-09-15T01:50:00.808 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.808 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1265425007}) 2024-09-15T01:50:00.808 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=533703094}) 2024-09-15T01:50:00.809 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.809 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1265425007}) 2024-09-15T01:50:00.809 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1303858476}) 2024-09-15T01:50:00.809 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.809 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1265425007}) 2024-09-15T01:50:00.809 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2521778617}) 2024-09-15T01:50:00.809 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.809 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1265425007}) 2024-09-15T01:50:00.809 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1321162644}) 2024-09-15T01:50:00.809 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.809 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1265425007}) 2024-09-15T01:50:00.810 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3080086839}) 2024-09-15T01:50:00.810 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.810 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1265425007}) 2024-09-15T01:50:00.810 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3458948938}) 2024-09-15T01:50:00.810 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.810 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1265425007}) 2024-09-15T01:50:00.810 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.498+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2816718197}) 2024-09-15T01:50:00.810 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.810 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3757426247}) 2024-09-15T01:50:00.810 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2825749356}) 2024-09-15T01:50:00.810 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.810 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3757426247}) 2024-09-15T01:50:00.811 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3681766671}) 2024-09-15T01:50:00.811 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.499+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.811 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3757426247}) 2024-09-15T01:50:00.811 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3212856675}) 2024-09-15T01:50:00.811 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.811 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3757426247}) 2024-09-15T01:50:00.811 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=738152462}) 2024-09-15T01:50:00.811 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.811 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3757426247}) 2024-09-15T01:50:00.811 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3766198731}) 2024-09-15T01:50:00.811 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.812 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3757426247}) 2024-09-15T01:50:00.812 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1135411833}) 2024-09-15T01:50:00.812 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.812 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3757426247}) 2024-09-15T01:50:00.812 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2554209635}) 2024-09-15T01:50:00.812 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.812 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3757426247}) 2024-09-15T01:50:00.812 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.500+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=347861178}) 2024-09-15T01:50:00.812 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.812 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1481491946}) 2024-09-15T01:50:00.812 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3639769265}) 2024-09-15T01:50:00.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1481491946}) 2024-09-15T01:50:00.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=595655288}) 2024-09-15T01:50:00.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1481491946}) 2024-09-15T01:50:00.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3860431048}) 2024-09-15T01:50:00.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1481491946}) 2024-09-15T01:50:00.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=371354734}) 2024-09-15T01:50:00.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.813 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1481491946}) 2024-09-15T01:50:00.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3970446165}) 2024-09-15T01:50:00.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1481491946}) 2024-09-15T01:50:00.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3903404644}) 2024-09-15T01:50:00.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1481491946}) 2024-09-15T01:50:00.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=78798246}) 2024-09-15T01:50:00.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1481491946}) 2024-09-15T01:50:00.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.501+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2439012902}) 2024-09-15T01:50:00.814 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2610458265}) 2024-09-15T01:50:00.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1720648080}) 2024-09-15T01:50:00.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2610458265}) 2024-09-15T01:50:00.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2734991555}) 2024-09-15T01:50:00.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2610458265}) 2024-09-15T01:50:00.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=535328117}) 2024-09-15T01:50:00.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2610458265}) 2024-09-15T01:50:00.815 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=673956002}) 2024-09-15T01:50:00.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2610458265}) 2024-09-15T01:50:00.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2759454017}) 2024-09-15T01:50:00.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2610458265}) 2024-09-15T01:50:00.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3240137355}) 2024-09-15T01:50:00.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2610458265}) 2024-09-15T01:50:00.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1285222712}) 2024-09-15T01:50:00.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.502+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2610458265}) 2024-09-15T01:50:00.816 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.503+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3775316184}) 2024-09-15T01:50:00.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764850550}) 2024-09-15T01:50:00.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3833222515}) 2024-09-15T01:50:00.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764850550}) 2024-09-15T01:50:00.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=914339608}) 2024-09-15T01:50:00.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764850550}) 2024-09-15T01:50:00.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4054209203}) 2024-09-15T01:50:00.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.817 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764850550}) 2024-09-15T01:50:00.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3052548165}) 2024-09-15T01:50:00.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764850550}) 2024-09-15T01:50:00.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3666804212}) 2024-09-15T01:50:00.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764850550}) 2024-09-15T01:50:00.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1666349686}) 2024-09-15T01:50:00.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764850550}) 2024-09-15T01:50:00.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3761372358}) 2024-09-15T01:50:00.818 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764850550}) 2024-09-15T01:50:00.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.504+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4245472031}) 2024-09-15T01:50:00.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2644820127}) 2024-09-15T01:50:00.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3419040812}) 2024-09-15T01:50:00.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2644820127}) 2024-09-15T01:50:00.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=820655611}) 2024-09-15T01:50:00.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.819 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2644820127}) 2024-09-15T01:50:00.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3832531934}) 2024-09-15T01:50:00.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2644820127}) 2024-09-15T01:50:00.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2728447416}) 2024-09-15T01:50:00.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2644820127}) 2024-09-15T01:50:00.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=939789675}) 2024-09-15T01:50:00.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2644820127}) 2024-09-15T01:50:00.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2125603907}) 2024-09-15T01:50:00.820 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2644820127}) 2024-09-15T01:50:00.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2588004280}) 2024-09-15T01:50:00.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2644820127}) 2024-09-15T01:50:00.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.505+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4070388073}) 2024-09-15T01:50:00.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.506+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1717233030}) 2024-09-15T01:50:00.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2312512821}) 2024-09-15T01:50:00.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1717233030}) 2024-09-15T01:50:00.821 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4265231662}) 2024-09-15T01:50:00.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1717233030}) 2024-09-15T01:50:00.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3508412034}) 2024-09-15T01:50:00.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1717233030}) 2024-09-15T01:50:00.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1763170463}) 2024-09-15T01:50:00.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1717233030}) 2024-09-15T01:50:00.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2246572994}) 2024-09-15T01:50:00.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1717233030}) 2024-09-15T01:50:00.822 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2974850529}) 2024-09-15T01:50:00.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1717233030}) 2024-09-15T01:50:00.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1647806561}) 2024-09-15T01:50:00.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1717233030}) 2024-09-15T01:50:00.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.507+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1929129188}) 2024-09-15T01:50:00.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1030136439}) 2024-09-15T01:50:00.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2792016506}) 2024-09-15T01:50:00.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1030136439}) 2024-09-15T01:50:00.823 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=723104885}) 2024-09-15T01:50:00.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1030136439}) 2024-09-15T01:50:00.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=86708504}) 2024-09-15T01:50:00.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1030136439}) 2024-09-15T01:50:00.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2775833684}) 2024-09-15T01:50:00.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1030136439}) 2024-09-15T01:50:00.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2131962350}) 2024-09-15T01:50:00.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.824 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1030136439}) 2024-09-15T01:50:00.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2405176611}) 2024-09-15T01:50:00.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1030136439}) 2024-09-15T01:50:00.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=144046705}) 2024-09-15T01:50:00.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1030136439}) 2024-09-15T01:50:00.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.508+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1409199290}) 2024-09-15T01:50:00.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017579935}) 2024-09-15T01:50:00.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=160510111}) 2024-09-15T01:50:00.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.825 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017579935}) 2024-09-15T01:50:00.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.509+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2912285488}) 2024-09-15T01:50:00.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017579935}) 2024-09-15T01:50:00.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1519322171}) 2024-09-15T01:50:00.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017579935}) 2024-09-15T01:50:00.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3777509987}) 2024-09-15T01:50:00.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017579935}) 2024-09-15T01:50:00.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=848925169}) 2024-09-15T01:50:00.826 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017579935}) 2024-09-15T01:50:00.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1667346342}) 2024-09-15T01:50:00.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017579935}) 2024-09-15T01:50:00.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2824421741}) 2024-09-15T01:50:00.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3017579935}) 2024-09-15T01:50:00.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.510+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1120793382}) 2024-09-15T01:50:00.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=563197287}) 2024-09-15T01:50:00.827 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=45179730}) 2024-09-15T01:50:00.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=563197287}) 2024-09-15T01:50:00.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3555866747}) 2024-09-15T01:50:00.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=563197287}) 2024-09-15T01:50:00.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3660910300}) 2024-09-15T01:50:00.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=563197287}) 2024-09-15T01:50:00.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3404470736}) 2024-09-15T01:50:00.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.828 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=563197287}) 2024-09-15T01:50:00.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2577599454}) 2024-09-15T01:50:00.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=563197287}) 2024-09-15T01:50:00.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1429237668}) 2024-09-15T01:50:00.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=563197287}) 2024-09-15T01:50:00.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1428655541}) 2024-09-15T01:50:00.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=563197287}) 2024-09-15T01:50:00.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.511+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=346397361}) 2024-09-15T01:50:00.829 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2528433319}) 2024-09-15T01:50:00.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=370982226}) 2024-09-15T01:50:00.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2528433319}) 2024-09-15T01:50:00.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2205763587}) 2024-09-15T01:50:00.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2528433319}) 2024-09-15T01:50:00.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3359689410}) 2024-09-15T01:50:00.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2528433319}) 2024-09-15T01:50:00.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1914755436}) 2024-09-15T01:50:00.830 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2528433319}) 2024-09-15T01:50:00.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4014724517}) 2024-09-15T01:50:00.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.512+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2528433319}) 2024-09-15T01:50:00.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2077819504}) 2024-09-15T01:50:00.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2528433319}) 2024-09-15T01:50:00.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2695699647}) 2024-09-15T01:50:00.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2528433319}) 2024-09-15T01:50:00.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.513+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2073815043}) 2024-09-15T01:50:00.831 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2963635740}) 2024-09-15T01:50:00.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3715867021}) 2024-09-15T01:50:00.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2963635740}) 2024-09-15T01:50:00.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2455202776}) 2024-09-15T01:50:00.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2963635740}) 2024-09-15T01:50:00.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=669202999}) 2024-09-15T01:50:00.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.832 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2963635740}) 2024-09-15T01:50:00.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3679074680}) 2024-09-15T01:50:00.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2963635740}) 2024-09-15T01:50:00.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=754499906}) 2024-09-15T01:50:00.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2963635740}) 2024-09-15T01:50:00.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=91374126}) 2024-09-15T01:50:00.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2963635740}) 2024-09-15T01:50:00.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2723243180}) 2024-09-15T01:50:00.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.833 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2963635740}) 2024-09-15T01:50:00.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.514+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4286881760}) 2024-09-15T01:50:00.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1515297543}) 2024-09-15T01:50:00.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1555085501}) 2024-09-15T01:50:00.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1515297543}) 2024-09-15T01:50:00.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2620740624}) 2024-09-15T01:50:00.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1515297543}) 2024-09-15T01:50:00.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=238719783}) 2024-09-15T01:50:00.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.834 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1515297543}) 2024-09-15T01:50:00.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2058242559}) 2024-09-15T01:50:00.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1515297543}) 2024-09-15T01:50:00.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2986428459}) 2024-09-15T01:50:00.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1515297543}) 2024-09-15T01:50:00.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=881979471}) 2024-09-15T01:50:00.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1515297543}) 2024-09-15T01:50:00.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=245956157}) 2024-09-15T01:50:00.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.835 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1515297543}) 2024-09-15T01:50:00.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.515+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=695356098}) 2024-09-15T01:50:00.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4111058722}) 2024-09-15T01:50:00.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2397526512}) 2024-09-15T01:50:00.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4111058722}) 2024-09-15T01:50:00.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2950729550}) 2024-09-15T01:50:00.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4111058722}) 2024-09-15T01:50:00.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=180580951}) 2024-09-15T01:50:00.836 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4111058722}) 2024-09-15T01:50:00.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3009361996}) 2024-09-15T01:50:00.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4111058722}) 2024-09-15T01:50:00.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3682992729}) 2024-09-15T01:50:00.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4111058722}) 2024-09-15T01:50:00.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2278497704}) 2024-09-15T01:50:00.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4111058722}) 2024-09-15T01:50:00.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=811531772}) 2024-09-15T01:50:00.837 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4111058722}) 2024-09-15T01:50:00.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.516+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3683664434}) 2024-09-15T01:50:00.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2562067927}) 2024-09-15T01:50:00.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1543628874}) 2024-09-15T01:50:00.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2562067927}) 2024-09-15T01:50:00.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3629473015}) 2024-09-15T01:50:00.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2562067927}) 2024-09-15T01:50:00.838 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3107290471}) 2024-09-15T01:50:00.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.517+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2562067927}) 2024-09-15T01:50:00.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2779481781}) 2024-09-15T01:50:00.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2562067927}) 2024-09-15T01:50:00.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1342746444}) 2024-09-15T01:50:00.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2562067927}) 2024-09-15T01:50:00.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=469557566}) 2024-09-15T01:50:00.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.839 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2562067927}) 2024-09-15T01:50:00.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3268387109}) 2024-09-15T01:50:00.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2562067927}) 2024-09-15T01:50:00.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.518+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3652835208}) 2024-09-15T01:50:00.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4217382200}) 2024-09-15T01:50:00.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=986015759}) 2024-09-15T01:50:00.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4217382200}) 2024-09-15T01:50:00.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1736832490}) 2024-09-15T01:50:00.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.840 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4217382200}) 2024-09-15T01:50:00.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4232085034}) 2024-09-15T01:50:00.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4217382200}) 2024-09-15T01:50:00.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2033710898}) 2024-09-15T01:50:00.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4217382200}) 2024-09-15T01:50:00.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3991086842}) 2024-09-15T01:50:00.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4217382200}) 2024-09-15T01:50:00.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1537526157}) 2024-09-15T01:50:00.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.841 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4217382200}) 2024-09-15T01:50:00.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1939156956}) 2024-09-15T01:50:00.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4217382200}) 2024-09-15T01:50:00.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.519+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2359370422}) 2024-09-15T01:50:00.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1712006994}) 2024-09-15T01:50:00.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=98032577}) 2024-09-15T01:50:00.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1712006994}) 2024-09-15T01:50:00.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837877529}) 2024-09-15T01:50:00.842 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1712006994}) 2024-09-15T01:50:00.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2862071647}) 2024-09-15T01:50:00.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1712006994}) 2024-09-15T01:50:00.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2278967047}) 2024-09-15T01:50:00.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1712006994}) 2024-09-15T01:50:00.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2504220616}) 2024-09-15T01:50:00.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1712006994}) 2024-09-15T01:50:00.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2105147496}) 2024-09-15T01:50:00.843 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1712006994}) 2024-09-15T01:50:00.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3299777320}) 2024-09-15T01:50:00.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1712006994}) 2024-09-15T01:50:00.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.520+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009734238}) 2024-09-15T01:50:00.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965202400}) 2024-09-15T01:50:00.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3959684677}) 2024-09-15T01:50:00.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965202400}) 2024-09-15T01:50:00.844 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4049071276}) 2024-09-15T01:50:00.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965202400}) 2024-09-15T01:50:00.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2465116025}) 2024-09-15T01:50:00.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965202400}) 2024-09-15T01:50:00.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2038113615}) 2024-09-15T01:50:00.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965202400}) 2024-09-15T01:50:00.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=200851651}) 2024-09-15T01:50:00.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.845 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965202400}) 2024-09-15T01:50:00.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2867334559}) 2024-09-15T01:50:00.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965202400}) 2024-09-15T01:50:00.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2425088267}) 2024-09-15T01:50:00.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965202400}) 2024-09-15T01:50:00.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.521+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3586165349}) 2024-09-15T01:50:00.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2256759663}) 2024-09-15T01:50:00.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2122907008}) 2024-09-15T01:50:00.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.846 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2256759663}) 2024-09-15T01:50:00.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2349229615}) 2024-09-15T01:50:00.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.522+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2256759663}) 2024-09-15T01:50:00.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1545557771}) 2024-09-15T01:50:00.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2256759663}) 2024-09-15T01:50:00.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2197871768}) 2024-09-15T01:50:00.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2256759663}) 2024-09-15T01:50:00.847 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2939622340}) 2024-09-15T01:50:00.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2256759663}) 2024-09-15T01:50:00.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2127522380}) 2024-09-15T01:50:00.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2256759663}) 2024-09-15T01:50:00.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2361173256}) 2024-09-15T01:50:00.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2256759663}) 2024-09-15T01:50:00.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.523+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=509432102}) 2024-09-15T01:50:00.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.848 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4019539959}) 2024-09-15T01:50:00.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=645505131}) 2024-09-15T01:50:00.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4019539959}) 2024-09-15T01:50:00.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1065440376}) 2024-09-15T01:50:00.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4019539959}) 2024-09-15T01:50:00.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=848550936}) 2024-09-15T01:50:00.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4019539959}) 2024-09-15T01:50:00.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4251242925}) 2024-09-15T01:50:00.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.849 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4019539959}) 2024-09-15T01:50:00.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4209401830}) 2024-09-15T01:50:00.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4019539959}) 2024-09-15T01:50:00.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2536960046}) 2024-09-15T01:50:00.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4019539959}) 2024-09-15T01:50:00.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2851119723}) 2024-09-15T01:50:00.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4019539959}) 2024-09-15T01:50:00.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.524+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1790770898}) 2024-09-15T01:50:00.850 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2218336605}) 2024-09-15T01:50:00.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=281004834}) 2024-09-15T01:50:00.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2218336605}) 2024-09-15T01:50:00.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2812275261}) 2024-09-15T01:50:00.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2218336605}) 2024-09-15T01:50:00.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1206785956}) 2024-09-15T01:50:00.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2218336605}) 2024-09-15T01:50:00.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2959411848}) 2024-09-15T01:50:00.851 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2218336605}) 2024-09-15T01:50:00.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=450862667}) 2024-09-15T01:50:00.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2218336605}) 2024-09-15T01:50:00.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=529205510}) 2024-09-15T01:50:00.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2218336605}) 2024-09-15T01:50:00.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=217106270}) 2024-09-15T01:50:00.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2218336605}) 2024-09-15T01:50:00.852 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.525+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2867937506}) 2024-09-15T01:50:00.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1239673750}) 2024-09-15T01:50:00.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=788684408}) 2024-09-15T01:50:00.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1239673750}) 2024-09-15T01:50:00.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3240083404}) 2024-09-15T01:50:00.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1239673750}) 2024-09-15T01:50:00.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3317713370}) 2024-09-15T01:50:00.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1239673750}) 2024-09-15T01:50:00.853 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2597902825}) 2024-09-15T01:50:00.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1239673750}) 2024-09-15T01:50:00.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=449456276}) 2024-09-15T01:50:00.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1239673750}) 2024-09-15T01:50:00.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1997522566}) 2024-09-15T01:50:00.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1239673750}) 2024-09-15T01:50:00.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4036340261}) 2024-09-15T01:50:00.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1239673750}) 2024-09-15T01:50:00.854 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.526+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=728054753}) 2024-09-15T01:50:00.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2554754656}) 2024-09-15T01:50:00.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1420374468}) 2024-09-15T01:50:00.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2554754656}) 2024-09-15T01:50:00.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3039556198}) 2024-09-15T01:50:00.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.527+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2554754656}) 2024-09-15T01:50:00.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2271721112}) 2024-09-15T01:50:00.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.855 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2554754656}) 2024-09-15T01:50:00.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2564132084}) 2024-09-15T01:50:00.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2554754656}) 2024-09-15T01:50:00.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3183453474}) 2024-09-15T01:50:00.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2554754656}) 2024-09-15T01:50:00.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3113705222}) 2024-09-15T01:50:00.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2554754656}) 2024-09-15T01:50:00.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3176325898}) 2024-09-15T01:50:00.856 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2554754656}) 2024-09-15T01:50:00.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.528+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4128321114}) 2024-09-15T01:50:00.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1174910322}) 2024-09-15T01:50:00.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3650928196}) 2024-09-15T01:50:00.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1174910322}) 2024-09-15T01:50:00.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3902293658}) 2024-09-15T01:50:00.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1174910322}) 2024-09-15T01:50:00.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=742017529}) 2024-09-15T01:50:00.857 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1174910322}) 2024-09-15T01:50:00.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=138793300}) 2024-09-15T01:50:00.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1174910322}) 2024-09-15T01:50:00.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3241357015}) 2024-09-15T01:50:00.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1174910322}) 2024-09-15T01:50:00.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4122585209}) 2024-09-15T01:50:00.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1174910322}) 2024-09-15T01:50:00.858 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=909619767}) 2024-09-15T01:50:00.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1174910322}) 2024-09-15T01:50:00.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.529+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=154003502}) 2024-09-15T01:50:00.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=407690623}) 2024-09-15T01:50:00.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=216176370}) 2024-09-15T01:50:00.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=407690623}) 2024-09-15T01:50:00.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2286487173}) 2024-09-15T01:50:00.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=407690623}) 2024-09-15T01:50:00.859 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3588115246}) 2024-09-15T01:50:00.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=407690623}) 2024-09-15T01:50:00.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2824799205}) 2024-09-15T01:50:00.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=407690623}) 2024-09-15T01:50:00.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3761443388}) 2024-09-15T01:50:00.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=407690623}) 2024-09-15T01:50:00.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=763806499}) 2024-09-15T01:50:00.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.860 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=407690623}) 2024-09-15T01:50:00.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2144006651}) 2024-09-15T01:50:00.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=407690623}) 2024-09-15T01:50:00.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.530+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1787211950}) 2024-09-15T01:50:00.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=32303215}) 2024-09-15T01:50:00.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3010993200}) 2024-09-15T01:50:00.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=32303215}) 2024-09-15T01:50:00.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3523412055}) 2024-09-15T01:50:00.861 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=32303215}) 2024-09-15T01:50:00.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2699929653}) 2024-09-15T01:50:00.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=32303215}) 2024-09-15T01:50:00.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1093523096}) 2024-09-15T01:50:00.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=32303215}) 2024-09-15T01:50:00.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2387760481}) 2024-09-15T01:50:00.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=32303215}) 2024-09-15T01:50:00.862 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2181825553}) 2024-09-15T01:50:00.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=32303215}) 2024-09-15T01:50:00.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2027912855}) 2024-09-15T01:50:00.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=32303215}) 2024-09-15T01:50:00.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.531+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1111655930}) 2024-09-15T01:50:00.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184569118}) 2024-09-15T01:50:00.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3383471103}) 2024-09-15T01:50:00.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.863 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184569118}) 2024-09-15T01:50:00.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1062515294}) 2024-09-15T01:50:00.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.532+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184569118}) 2024-09-15T01:50:00.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1552642675}) 2024-09-15T01:50:00.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184569118}) 2024-09-15T01:50:00.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2923004911}) 2024-09-15T01:50:00.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184569118}) 2024-09-15T01:50:00.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2532270788}) 2024-09-15T01:50:00.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.864 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184569118}) 2024-09-15T01:50:00.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1543469807}) 2024-09-15T01:50:00.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184569118}) 2024-09-15T01:50:00.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=441293508}) 2024-09-15T01:50:00.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184569118}) 2024-09-15T01:50:00.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.533+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=778086716}) 2024-09-15T01:50:00.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=182641080}) 2024-09-15T01:50:00.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2586300839}) 2024-09-15T01:50:00.865 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=182641080}) 2024-09-15T01:50:00.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1438133827}) 2024-09-15T01:50:00.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=182641080}) 2024-09-15T01:50:00.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=928626703}) 2024-09-15T01:50:00.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=182641080}) 2024-09-15T01:50:00.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4071262302}) 2024-09-15T01:50:00.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=182641080}) 2024-09-15T01:50:00.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3618617410}) 2024-09-15T01:50:00.866 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.867 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=182641080}) 2024-09-15T01:50:00.867 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1018439305}) 2024-09-15T01:50:00.867 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.867 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=182641080}) 2024-09-15T01:50:00.867 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3075107830}) 2024-09-15T01:50:00.867 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.867 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=182641080}) 2024-09-15T01:50:00.867 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.534+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3991276249}) 2024-09-15T01:50:00.867 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.867 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2049252674}) 2024-09-15T01:50:00.867 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4093095102}) 2024-09-15T01:50:00.867 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.868 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2049252674}) 2024-09-15T01:50:00.868 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3571239008}) 2024-09-15T01:50:00.868 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.868 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2049252674}) 2024-09-15T01:50:00.868 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2363559654}) 2024-09-15T01:50:00.868 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.868 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2049252674}) 2024-09-15T01:50:00.868 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3015142797}) 2024-09-15T01:50:00.868 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.868 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2049252674}) 2024-09-15T01:50:00.869 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2474400659}) 2024-09-15T01:50:00.869 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.869 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2049252674}) 2024-09-15T01:50:00.869 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3033858903}) 2024-09-15T01:50:00.869 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.869 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.535+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2049252674}) 2024-09-15T01:50:00.869 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.536+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1461675863}) 2024-09-15T01:50:00.869 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.536+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.869 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.536+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2049252674}) 2024-09-15T01:50:00.869 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.536+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1685027789}) 2024-09-15T01:50:00.869 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.869 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=730054180}) 2024-09-15T01:50:00.870 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4138347591}) 2024-09-15T01:50:00.870 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.870 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=730054180}) 2024-09-15T01:50:00.870 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1868200453}) 2024-09-15T01:50:00.870 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.870 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=730054180}) 2024-09-15T01:50:00.870 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=575678377}) 2024-09-15T01:50:00.870 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.870 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=730054180}) 2024-09-15T01:50:00.870 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2188558858}) 2024-09-15T01:50:00.870 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.871 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=730054180}) 2024-09-15T01:50:00.871 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1059707748}) 2024-09-15T01:50:00.871 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.871 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=730054180}) 2024-09-15T01:50:00.871 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3032427926}) 2024-09-15T01:50:00.871 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.871 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=730054180}) 2024-09-15T01:50:00.871 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3039742833}) 2024-09-15T01:50:00.871 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.871 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=730054180}) 2024-09-15T01:50:00.871 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.537+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2262198813}) 2024-09-15T01:50:00.871 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.872 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3866652308}) 2024-09-15T01:50:00.872 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4253216774}) 2024-09-15T01:50:00.872 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.872 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3866652308}) 2024-09-15T01:50:00.872 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1777399514}) 2024-09-15T01:50:00.872 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.872 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3866652308}) 2024-09-15T01:50:00.872 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=752996691}) 2024-09-15T01:50:00.872 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.872 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3866652308}) 2024-09-15T01:50:00.872 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=474179760}) 2024-09-15T01:50:00.872 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.873 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3866652308}) 2024-09-15T01:50:00.873 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3118928569}) 2024-09-15T01:50:00.873 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.873 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3866652308}) 2024-09-15T01:50:00.873 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=266053891}) 2024-09-15T01:50:00.873 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.873 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3866652308}) 2024-09-15T01:50:00.873 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2761777407}) 2024-09-15T01:50:00.873 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.873 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3866652308}) 2024-09-15T01:50:00.873 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.538+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2726983727}) 2024-09-15T01:50:00.874 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.539+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.874 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1427110258}) 2024-09-15T01:50:00.874 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3156228594}) 2024-09-15T01:50:00.874 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.874 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1427110258}) 2024-09-15T01:50:00.874 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=49095663}) 2024-09-15T01:50:00.874 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.874 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1427110258}) 2024-09-15T01:50:00.874 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3782221798}) 2024-09-15T01:50:00.874 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.874 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1427110258}) 2024-09-15T01:50:00.875 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3849041427}) 2024-09-15T01:50:00.875 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.875 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1427110258}) 2024-09-15T01:50:00.875 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2951870447}) 2024-09-15T01:50:00.875 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.875 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1427110258}) 2024-09-15T01:50:00.875 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1459234492}) 2024-09-15T01:50:00.875 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.875 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1427110258}) 2024-09-15T01:50:00.875 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4078074525}) 2024-09-15T01:50:00.875 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.876 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1427110258}) 2024-09-15T01:50:00.876 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.540+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1613564514}) 2024-09-15T01:50:00.876 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.876 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1255342490}) 2024-09-15T01:50:00.876 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592664844}) 2024-09-15T01:50:00.876 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.876 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1255342490}) 2024-09-15T01:50:00.876 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3299205573}) 2024-09-15T01:50:00.876 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.876 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1255342490}) 2024-09-15T01:50:00.876 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1967727799}) 2024-09-15T01:50:00.877 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.877 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1255342490}) 2024-09-15T01:50:00.877 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1139871395}) 2024-09-15T01:50:00.877 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.877 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1255342490}) 2024-09-15T01:50:00.877 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1117271492}) 2024-09-15T01:50:00.877 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.877 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1255342490}) 2024-09-15T01:50:00.877 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1613734081}) 2024-09-15T01:50:00.877 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.877 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1255342490}) 2024-09-15T01:50:00.878 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=198723914}) 2024-09-15T01:50:00.878 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.878 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1255342490}) 2024-09-15T01:50:00.878 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.541+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1675780668}) 2024-09-15T01:50:00.878 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.878 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2456941176}) 2024-09-15T01:50:00.878 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2261924692}) 2024-09-15T01:50:00.878 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.878 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2456941176}) 2024-09-15T01:50:00.878 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3623609510}) 2024-09-15T01:50:00.878 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.879 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2456941176}) 2024-09-15T01:50:00.879 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=562697455}) 2024-09-15T01:50:00.879 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.879 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2456941176}) 2024-09-15T01:50:00.879 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3384723368}) 2024-09-15T01:50:00.879 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.879 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2456941176}) 2024-09-15T01:50:00.879 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=619797460}) 2024-09-15T01:50:00.879 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.879 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2456941176}) 2024-09-15T01:50:00.879 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=52181831}) 2024-09-15T01:50:00.880 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.880 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2456941176}) 2024-09-15T01:50:00.880 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1149159745}) 2024-09-15T01:50:00.880 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.880 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2456941176}) 2024-09-15T01:50:00.880 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.542+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=801944346}) 2024-09-15T01:50:00.880 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.543+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.880 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.543+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4248295330}) 2024-09-15T01:50:00.880 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.543+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1184532258}) 2024-09-15T01:50:00.880 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.543+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.880 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.543+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4248295330}) 2024-09-15T01:50:00.881 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.543+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2567727840}) 2024-09-15T01:50:00.881 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.543+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.881 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.543+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4248295330}) 2024-09-15T01:50:00.881 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1880783635}) 2024-09-15T01:50:00.881 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.881 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4248295330}) 2024-09-15T01:50:00.881 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1554557}) 2024-09-15T01:50:00.881 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.881 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4248295330}) 2024-09-15T01:50:00.881 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1235663736}) 2024-09-15T01:50:00.881 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.882 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4248295330}) 2024-09-15T01:50:00.882 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1586549934}) 2024-09-15T01:50:00.882 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.882 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4248295330}) 2024-09-15T01:50:00.882 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4161393357}) 2024-09-15T01:50:00.882 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.882 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4248295330}) 2024-09-15T01:50:00.882 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.544+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=280791689}) 2024-09-15T01:50:00.882 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.882 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565117942}) 2024-09-15T01:50:00.882 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2652127335}) 2024-09-15T01:50:00.882 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.883 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565117942}) 2024-09-15T01:50:00.883 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=365900674}) 2024-09-15T01:50:00.883 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.883 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565117942}) 2024-09-15T01:50:00.883 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3986016659}) 2024-09-15T01:50:00.883 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.883 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565117942}) 2024-09-15T01:50:00.883 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=55169581}) 2024-09-15T01:50:00.883 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.883 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565117942}) 2024-09-15T01:50:00.883 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1309707260}) 2024-09-15T01:50:00.883 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.884 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565117942}) 2024-09-15T01:50:00.884 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=303127446}) 2024-09-15T01:50:00.884 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.884 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565117942}) 2024-09-15T01:50:00.884 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=762500949}) 2024-09-15T01:50:00.884 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.884 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565117942}) 2024-09-15T01:50:00.884 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.545+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1030153468}) 2024-09-15T01:50:00.884 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.884 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=361842371}) 2024-09-15T01:50:00.884 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1624472389}) 2024-09-15T01:50:00.885 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.885 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=361842371}) 2024-09-15T01:50:00.885 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3105398103}) 2024-09-15T01:50:00.885 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.885 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=361842371}) 2024-09-15T01:50:00.885 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1042103628}) 2024-09-15T01:50:00.885 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.885 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=361842371}) 2024-09-15T01:50:00.885 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3864897855}) 2024-09-15T01:50:00.885 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.885 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=361842371}) 2024-09-15T01:50:00.885 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3883719697}) 2024-09-15T01:50:00.886 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.886 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=361842371}) 2024-09-15T01:50:00.886 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2175727325}) 2024-09-15T01:50:00.886 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.886 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=361842371}) 2024-09-15T01:50:00.886 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=920266588}) 2024-09-15T01:50:00.886 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.886 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=361842371}) 2024-09-15T01:50:00.886 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.546+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4011109429}) 2024-09-15T01:50:00.886 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.886 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1979569996}) 2024-09-15T01:50:00.887 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=617497398}) 2024-09-15T01:50:00.887 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.887 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1979569996}) 2024-09-15T01:50:00.887 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2190793696}) 2024-09-15T01:50:00.887 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.887 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1979569996}) 2024-09-15T01:50:00.887 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3398719692}) 2024-09-15T01:50:00.887 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.887 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1979569996}) 2024-09-15T01:50:00.887 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2527962754}) 2024-09-15T01:50:00.887 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.887 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1979569996}) 2024-09-15T01:50:00.888 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=744783080}) 2024-09-15T01:50:00.888 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.888 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1979569996}) 2024-09-15T01:50:00.888 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802277272}) 2024-09-15T01:50:00.888 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.888 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.547+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1979569996}) 2024-09-15T01:50:00.888 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.548+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=490121899}) 2024-09-15T01:50:00.888 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.548+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.888 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.548+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1979569996}) 2024-09-15T01:50:00.888 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.548+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1131413547}) 2024-09-15T01:50:00.888 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.548+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.889 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.548+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109497554}) 2024-09-15T01:50:00.889 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=348609948}) 2024-09-15T01:50:00.889 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.889 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109497554}) 2024-09-15T01:50:00.889 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=502084271}) 2024-09-15T01:50:00.889 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.889 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109497554}) 2024-09-15T01:50:00.889 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1278017796}) 2024-09-15T01:50:00.889 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.889 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109497554}) 2024-09-15T01:50:00.889 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2863880207}) 2024-09-15T01:50:00.889 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.890 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109497554}) 2024-09-15T01:50:00.890 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=675107826}) 2024-09-15T01:50:00.890 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.890 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109497554}) 2024-09-15T01:50:00.890 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2484329469}) 2024-09-15T01:50:00.890 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.890 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109497554}) 2024-09-15T01:50:00.890 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2262355652}) 2024-09-15T01:50:00.890 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.890 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=109497554}) 2024-09-15T01:50:00.890 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.549+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2145415349}) 2024-09-15T01:50:00.891 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.891 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3123243417}) 2024-09-15T01:50:00.891 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1789964590}) 2024-09-15T01:50:00.891 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.891 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3123243417}) 2024-09-15T01:50:00.891 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2246565601}) 2024-09-15T01:50:00.891 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.891 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3123243417}) 2024-09-15T01:50:00.891 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3924128950}) 2024-09-15T01:50:00.891 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.891 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3123243417}) 2024-09-15T01:50:00.891 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=843846714}) 2024-09-15T01:50:00.892 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.892 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3123243417}) 2024-09-15T01:50:00.892 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3080447111}) 2024-09-15T01:50:00.892 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.892 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3123243417}) 2024-09-15T01:50:00.892 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2103507243}) 2024-09-15T01:50:00.892 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.892 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3123243417}) 2024-09-15T01:50:00.892 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3500865303}) 2024-09-15T01:50:00.892 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.892 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3123243417}) 2024-09-15T01:50:00.893 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.550+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3709267321}) 2024-09-15T01:50:00.893 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.893 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519212133}) 2024-09-15T01:50:00.893 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874301430}) 2024-09-15T01:50:00.893 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.893 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519212133}) 2024-09-15T01:50:00.893 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2685136768}) 2024-09-15T01:50:00.893 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.893 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519212133}) 2024-09-15T01:50:00.893 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=850102620}) 2024-09-15T01:50:00.893 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.894 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519212133}) 2024-09-15T01:50:00.894 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1839873499}) 2024-09-15T01:50:00.894 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.894 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519212133}) 2024-09-15T01:50:00.894 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2877217121}) 2024-09-15T01:50:00.894 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.894 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519212133}) 2024-09-15T01:50:00.894 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=190087142}) 2024-09-15T01:50:00.894 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.894 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519212133}) 2024-09-15T01:50:00.894 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2090447210}) 2024-09-15T01:50:00.894 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.895 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.551+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519212133}) 2024-09-15T01:50:00.895 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.552+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2641091318}) 2024-09-15T01:50:00.895 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.895 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=493504220}) 2024-09-15T01:50:00.895 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738904514}) 2024-09-15T01:50:00.895 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.895 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=493504220}) 2024-09-15T01:50:00.895 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1775885655}) 2024-09-15T01:50:00.895 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.895 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=493504220}) 2024-09-15T01:50:00.895 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2654489080}) 2024-09-15T01:50:00.896 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.896 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=493504220}) 2024-09-15T01:50:00.896 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=54073527}) 2024-09-15T01:50:00.896 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.896 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=493504220}) 2024-09-15T01:50:00.896 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=182772020}) 2024-09-15T01:50:00.896 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.896 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=493504220}) 2024-09-15T01:50:00.896 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3769249120}) 2024-09-15T01:50:00.896 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.896 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=493504220}) 2024-09-15T01:50:00.897 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4206445142}) 2024-09-15T01:50:00.897 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.897 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=493504220}) 2024-09-15T01:50:00.897 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.553+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2237953292}) 2024-09-15T01:50:00.897 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.897 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3738916953}) 2024-09-15T01:50:00.897 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1573635062}) 2024-09-15T01:50:00.897 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.897 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3738916953}) 2024-09-15T01:50:00.897 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=738967847}) 2024-09-15T01:50:00.897 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.898 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3738916953}) 2024-09-15T01:50:00.898 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1432657467}) 2024-09-15T01:50:00.898 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.898 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3738916953}) 2024-09-15T01:50:00.898 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1531401983}) 2024-09-15T01:50:00.898 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.898 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3738916953}) 2024-09-15T01:50:00.898 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=212966636}) 2024-09-15T01:50:00.898 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.898 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3738916953}) 2024-09-15T01:50:00.898 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1656693567}) 2024-09-15T01:50:00.898 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.899 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3738916953}) 2024-09-15T01:50:00.899 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.554+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=749509594}) 2024-09-15T01:50:00.899 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.899 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3738916953}) 2024-09-15T01:50:00.899 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3099610161}) 2024-09-15T01:50:00.899 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.899 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3442492615}) 2024-09-15T01:50:00.899 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3656615539}) 2024-09-15T01:50:00.899 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.899 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.555+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3442492615}) 2024-09-15T01:50:00.899 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3441383515}) 2024-09-15T01:50:00.900 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.900 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3442492615}) 2024-09-15T01:50:00.900 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3746964773}) 2024-09-15T01:50:00.900 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.900 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3442492615}) 2024-09-15T01:50:00.900 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2091249403}) 2024-09-15T01:50:00.900 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.900 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3442492615}) 2024-09-15T01:50:00.900 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2035295084}) 2024-09-15T01:50:00.900 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.900 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3442492615}) 2024-09-15T01:50:00.900 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2466580255}) 2024-09-15T01:50:00.901 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.901 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3442492615}) 2024-09-15T01:50:00.901 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1490160996}) 2024-09-15T01:50:00.901 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.901 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3442492615}) 2024-09-15T01:50:00.901 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.556+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3260373232}) 2024-09-15T01:50:00.901 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.901 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026176345}) 2024-09-15T01:50:00.901 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1531018229}) 2024-09-15T01:50:00.901 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.901 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026176345}) 2024-09-15T01:50:00.902 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2826878520}) 2024-09-15T01:50:00.902 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.902 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026176345}) 2024-09-15T01:50:00.902 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2748368348}) 2024-09-15T01:50:00.902 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.902 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026176345}) 2024-09-15T01:50:00.902 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2473630186}) 2024-09-15T01:50:00.902 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.902 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026176345}) 2024-09-15T01:50:00.902 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2195682991}) 2024-09-15T01:50:00.902 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.902 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026176345}) 2024-09-15T01:50:00.903 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1499382078}) 2024-09-15T01:50:00.903 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.903 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026176345}) 2024-09-15T01:50:00.903 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2693010081}) 2024-09-15T01:50:00.903 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.903 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1026176345}) 2024-09-15T01:50:00.903 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.557+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=233000759}) 2024-09-15T01:50:00.903 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.903 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=554805810}) 2024-09-15T01:50:00.903 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1184759172}) 2024-09-15T01:50:00.903 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.904 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=554805810}) 2024-09-15T01:50:00.904 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=723169811}) 2024-09-15T01:50:00.904 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.904 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=554805810}) 2024-09-15T01:50:00.904 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2970825247}) 2024-09-15T01:50:00.904 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.904 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=554805810}) 2024-09-15T01:50:00.904 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1034968922}) 2024-09-15T01:50:00.904 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.904 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=554805810}) 2024-09-15T01:50:00.904 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2422639744}) 2024-09-15T01:50:00.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=554805810}) 2024-09-15T01:50:00.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=121569333}) 2024-09-15T01:50:00.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=554805810}) 2024-09-15T01:50:00.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2350923139}) 2024-09-15T01:50:00.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=554805810}) 2024-09-15T01:50:00.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.558+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2319824035}) 2024-09-15T01:50:00.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.559+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.905 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.559+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=738397831}) 2024-09-15T01:50:00.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.559+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=280447300}) 2024-09-15T01:50:00.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=738397831}) 2024-09-15T01:50:00.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1669904786}) 2024-09-15T01:50:00.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=738397831}) 2024-09-15T01:50:00.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=774223901}) 2024-09-15T01:50:00.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=738397831}) 2024-09-15T01:50:00.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2104291256}) 2024-09-15T01:50:00.906 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=738397831}) 2024-09-15T01:50:00.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4286534233}) 2024-09-15T01:50:00.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.907 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=738397831}) 2024-09-15T01:50:00.908 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3006903271}) 2024-09-15T01:50:00.908 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.908 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=738397831}) 2024-09-15T01:50:00.908 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2681036302}) 2024-09-15T01:50:00.908 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.908 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=738397831}) 2024-09-15T01:50:00.908 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.560+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=865471126}) 2024-09-15T01:50:00.908 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.908 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4114592004}) 2024-09-15T01:50:00.908 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2842955351}) 2024-09-15T01:50:00.908 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.909 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4114592004}) 2024-09-15T01:50:00.909 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2561874545}) 2024-09-15T01:50:00.909 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.909 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4114592004}) 2024-09-15T01:50:00.909 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2713319755}) 2024-09-15T01:50:00.909 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.909 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4114592004}) 2024-09-15T01:50:00.909 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=122907687}) 2024-09-15T01:50:00.909 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.909 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4114592004}) 2024-09-15T01:50:00.909 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2872608716}) 2024-09-15T01:50:00.910 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.910 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4114592004}) 2024-09-15T01:50:00.910 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3469582075}) 2024-09-15T01:50:00.910 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.910 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4114592004}) 2024-09-15T01:50:00.910 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1637689134}) 2024-09-15T01:50:00.910 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.910 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4114592004}) 2024-09-15T01:50:00.910 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.561+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3645968450}) 2024-09-15T01:50:00.910 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.910 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=615553517}) 2024-09-15T01:50:00.911 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2340416167}) 2024-09-15T01:50:00.911 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.911 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=615553517}) 2024-09-15T01:50:00.911 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1652275235}) 2024-09-15T01:50:00.911 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.911 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=615553517}) 2024-09-15T01:50:00.911 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1412040555}) 2024-09-15T01:50:00.911 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.911 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=615553517}) 2024-09-15T01:50:00.911 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4036478788}) 2024-09-15T01:50:00.911 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.912 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.562+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=615553517}) 2024-09-15T01:50:00.912 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.563+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2502704684}) 2024-09-15T01:50:00.912 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.563+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.912 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.563+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=615553517}) 2024-09-15T01:50:00.912 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.563+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=485914056}) 2024-09-15T01:50:00.912 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.563+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.912 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.563+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=615553517}) 2024-09-15T01:50:00.912 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.563+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3082493661}) 2024-09-15T01:50:00.912 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.563+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.912 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.563+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=615553517}) 2024-09-15T01:50:00.912 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.563+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1170085261}) 2024-09-15T01:50:00.912 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.913 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3521393734}) 2024-09-15T01:50:00.913 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1251514616}) 2024-09-15T01:50:00.913 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.913 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3521393734}) 2024-09-15T01:50:00.913 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4021974213}) 2024-09-15T01:50:00.913 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.913 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3521393734}) 2024-09-15T01:50:00.913 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2366701348}) 2024-09-15T01:50:00.913 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.913 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3521393734}) 2024-09-15T01:50:00.913 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=593968099}) 2024-09-15T01:50:00.914 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.914 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3521393734}) 2024-09-15T01:50:00.914 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1667707347}) 2024-09-15T01:50:00.914 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.914 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3521393734}) 2024-09-15T01:50:00.914 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=916799982}) 2024-09-15T01:50:00.914 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.914 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3521393734}) 2024-09-15T01:50:00.914 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4013696150}) 2024-09-15T01:50:00.914 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.914 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3521393734}) 2024-09-15T01:50:00.914 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.564+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2132504690}) 2024-09-15T01:50:00.915 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.915 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2051612674}) 2024-09-15T01:50:00.915 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2270673642}) 2024-09-15T01:50:00.915 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.915 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2051612674}) 2024-09-15T01:50:00.915 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2931082194}) 2024-09-15T01:50:00.915 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.915 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2051612674}) 2024-09-15T01:50:00.915 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=353868028}) 2024-09-15T01:50:00.915 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.915 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2051612674}) 2024-09-15T01:50:00.915 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3511185876}) 2024-09-15T01:50:00.916 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.916 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2051612674}) 2024-09-15T01:50:00.916 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=716204508}) 2024-09-15T01:50:00.916 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.916 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2051612674}) 2024-09-15T01:50:00.916 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=528676388}) 2024-09-15T01:50:00.916 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.916 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2051612674}) 2024-09-15T01:50:00.916 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3660382675}) 2024-09-15T01:50:00.916 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.917 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2051612674}) 2024-09-15T01:50:00.917 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.565+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3793038935}) 2024-09-15T01:50:00.917 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.917 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=311043658}) 2024-09-15T01:50:00.917 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=368980448}) 2024-09-15T01:50:00.917 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.917 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=311043658}) 2024-09-15T01:50:00.917 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=555660811}) 2024-09-15T01:50:00.917 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.917 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=311043658}) 2024-09-15T01:50:00.917 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1861642502}) 2024-09-15T01:50:00.917 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.918 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=311043658}) 2024-09-15T01:50:00.918 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2993908021}) 2024-09-15T01:50:00.918 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.918 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=311043658}) 2024-09-15T01:50:00.918 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1734782353}) 2024-09-15T01:50:00.918 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.918 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=311043658}) 2024-09-15T01:50:00.918 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1425966164}) 2024-09-15T01:50:00.918 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.918 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.566+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=311043658}) 2024-09-15T01:50:00.918 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.567+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=514777409}) 2024-09-15T01:50:00.918 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.567+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.919 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.567+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=311043658}) 2024-09-15T01:50:00.919 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.567+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1006215807}) 2024-09-15T01:50:00.919 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.919 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107876223}) 2024-09-15T01:50:00.919 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3964473954}) 2024-09-15T01:50:00.919 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.919 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107876223}) 2024-09-15T01:50:00.919 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3480295698}) 2024-09-15T01:50:00.919 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.919 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107876223}) 2024-09-15T01:50:00.919 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3722435508}) 2024-09-15T01:50:00.920 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.920 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107876223}) 2024-09-15T01:50:00.920 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1059586271}) 2024-09-15T01:50:00.920 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.920 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107876223}) 2024-09-15T01:50:00.920 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3657191632}) 2024-09-15T01:50:00.920 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.920 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107876223}) 2024-09-15T01:50:00.920 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2655527708}) 2024-09-15T01:50:00.920 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.920 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107876223}) 2024-09-15T01:50:00.920 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1185198231}) 2024-09-15T01:50:00.921 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.921 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=107876223}) 2024-09-15T01:50:00.921 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.568+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3655185540}) 2024-09-15T01:50:00.921 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.921 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3523289748}) 2024-09-15T01:50:00.921 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=630886556}) 2024-09-15T01:50:00.921 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.921 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3523289748}) 2024-09-15T01:50:00.921 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4076535404}) 2024-09-15T01:50:00.921 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.921 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3523289748}) 2024-09-15T01:50:00.922 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4129879545}) 2024-09-15T01:50:00.922 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.922 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3523289748}) 2024-09-15T01:50:00.922 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=137169483}) 2024-09-15T01:50:00.922 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.922 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3523289748}) 2024-09-15T01:50:00.922 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1847562707}) 2024-09-15T01:50:00.922 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.922 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3523289748}) 2024-09-15T01:50:00.922 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2144048387}) 2024-09-15T01:50:00.922 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.923 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3523289748}) 2024-09-15T01:50:00.923 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2008032688}) 2024-09-15T01:50:00.923 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.923 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3523289748}) 2024-09-15T01:50:00.923 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.569+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3658380383}) 2024-09-15T01:50:00.923 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.923 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=861544073}) 2024-09-15T01:50:00.923 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=318237234}) 2024-09-15T01:50:00.923 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.923 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=861544073}) 2024-09-15T01:50:00.923 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3124528755}) 2024-09-15T01:50:00.923 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.924 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=861544073}) 2024-09-15T01:50:00.924 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2689752454}) 2024-09-15T01:50:00.924 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.924 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=861544073}) 2024-09-15T01:50:00.924 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=277095678}) 2024-09-15T01:50:00.924 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.924 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=861544073}) 2024-09-15T01:50:00.924 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=294478794}) 2024-09-15T01:50:00.924 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.924 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=861544073}) 2024-09-15T01:50:00.925 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1402352129}) 2024-09-15T01:50:00.925 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.925 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.570+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=861544073}) 2024-09-15T01:50:00.925 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.571+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2482414925}) 2024-09-15T01:50:00.925 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.571+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.925 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.571+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=861544073}) 2024-09-15T01:50:00.925 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.571+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3108164571}) 2024-09-15T01:50:00.925 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.925 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=349419147}) 2024-09-15T01:50:00.925 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1802323273}) 2024-09-15T01:50:00.925 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.925 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=349419147}) 2024-09-15T01:50:00.926 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4244269176}) 2024-09-15T01:50:00.926 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.926 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=349419147}) 2024-09-15T01:50:00.926 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3120628412}) 2024-09-15T01:50:00.926 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.926 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=349419147}) 2024-09-15T01:50:00.926 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3213770849}) 2024-09-15T01:50:00.926 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.926 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=349419147}) 2024-09-15T01:50:00.926 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=785128841}) 2024-09-15T01:50:00.926 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.927 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=349419147}) 2024-09-15T01:50:00.927 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2835784857}) 2024-09-15T01:50:00.927 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.927 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=349419147}) 2024-09-15T01:50:00.927 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4009973673}) 2024-09-15T01:50:00.927 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.927 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=349419147}) 2024-09-15T01:50:00.927 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.572+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=803276298}) 2024-09-15T01:50:00.927 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.927 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2330516666}) 2024-09-15T01:50:00.927 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=658386010}) 2024-09-15T01:50:00.927 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.928 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2330516666}) 2024-09-15T01:50:00.928 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1017789667}) 2024-09-15T01:50:00.928 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.928 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2330516666}) 2024-09-15T01:50:00.928 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4278188878}) 2024-09-15T01:50:00.928 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.928 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2330516666}) 2024-09-15T01:50:00.928 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=901055804}) 2024-09-15T01:50:00.928 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.928 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2330516666}) 2024-09-15T01:50:00.928 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3991524148}) 2024-09-15T01:50:00.929 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.929 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2330516666}) 2024-09-15T01:50:00.929 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=83571433}) 2024-09-15T01:50:00.929 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.929 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2330516666}) 2024-09-15T01:50:00.929 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2113749143}) 2024-09-15T01:50:00.929 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.929 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2330516666}) 2024-09-15T01:50:00.929 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.573+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2031358881}) 2024-09-15T01:50:00.929 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.574+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.929 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.574+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2308770188}) 2024-09-15T01:50:00.929 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2796021547}) 2024-09-15T01:50:00.930 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.930 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2308770188}) 2024-09-15T01:50:00.930 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2614522296}) 2024-09-15T01:50:00.930 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.930 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2308770188}) 2024-09-15T01:50:00.930 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2113585871}) 2024-09-15T01:50:00.930 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.930 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2308770188}) 2024-09-15T01:50:00.930 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4272750359}) 2024-09-15T01:50:00.930 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.930 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2308770188}) 2024-09-15T01:50:00.930 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3454273908}) 2024-09-15T01:50:00.931 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.931 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2308770188}) 2024-09-15T01:50:00.931 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=461292874}) 2024-09-15T01:50:00.931 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.931 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2308770188}) 2024-09-15T01:50:00.931 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3196781053}) 2024-09-15T01:50:00.931 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.931 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2308770188}) 2024-09-15T01:50:00.931 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.575+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3765615029}) 2024-09-15T01:50:00.931 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.931 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1857887044}) 2024-09-15T01:50:00.932 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3088561976}) 2024-09-15T01:50:00.932 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.932 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1857887044}) 2024-09-15T01:50:00.932 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2491366486}) 2024-09-15T01:50:00.932 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.932 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1857887044}) 2024-09-15T01:50:00.932 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3527032897}) 2024-09-15T01:50:00.932 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.932 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1857887044}) 2024-09-15T01:50:00.932 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2651102174}) 2024-09-15T01:50:00.932 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.933 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1857887044}) 2024-09-15T01:50:00.933 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2029865955}) 2024-09-15T01:50:00.933 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.933 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1857887044}) 2024-09-15T01:50:00.933 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1957386257}) 2024-09-15T01:50:00.933 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.933 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1857887044}) 2024-09-15T01:50:00.933 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4294230925}) 2024-09-15T01:50:00.933 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.933 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1857887044}) 2024-09-15T01:50:00.933 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.576+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4046590012}) 2024-09-15T01:50:00.933 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.934 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=771719500}) 2024-09-15T01:50:00.934 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3737760711}) 2024-09-15T01:50:00.934 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.934 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=771719500}) 2024-09-15T01:50:00.934 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3817725381}) 2024-09-15T01:50:00.934 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.934 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=771719500}) 2024-09-15T01:50:00.934 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2474757214}) 2024-09-15T01:50:00.934 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.934 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=771719500}) 2024-09-15T01:50:00.934 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2421258643}) 2024-09-15T01:50:00.935 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.935 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=771719500}) 2024-09-15T01:50:00.935 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3170489960}) 2024-09-15T01:50:00.935 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.935 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=771719500}) 2024-09-15T01:50:00.935 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3498759282}) 2024-09-15T01:50:00.935 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.577+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.935 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.578+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=771719500}) 2024-09-15T01:50:00.935 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.578+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3826697898}) 2024-09-15T01:50:00.935 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.578+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.935 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.578+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=771719500}) 2024-09-15T01:50:00.936 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.578+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1928203523}) 2024-09-15T01:50:00.936 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.936 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3238356628}) 2024-09-15T01:50:00.936 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1427751115}) 2024-09-15T01:50:00.936 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.936 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3238356628}) 2024-09-15T01:50:00.936 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1315910111}) 2024-09-15T01:50:00.936 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.936 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3238356628}) 2024-09-15T01:50:00.936 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1401420280}) 2024-09-15T01:50:00.936 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.937 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3238356628}) 2024-09-15T01:50:00.937 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=981003125}) 2024-09-15T01:50:00.937 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.937 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3238356628}) 2024-09-15T01:50:00.937 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2889940449}) 2024-09-15T01:50:00.937 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.937 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3238356628}) 2024-09-15T01:50:00.937 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2550078176}) 2024-09-15T01:50:00.937 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.937 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3238356628}) 2024-09-15T01:50:00.938 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1287987255}) 2024-09-15T01:50:00.938 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.938 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3238356628}) 2024-09-15T01:50:00.938 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.579+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=603739793}) 2024-09-15T01:50:00.938 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.938 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=774557210}) 2024-09-15T01:50:00.938 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2245373072}) 2024-09-15T01:50:00.938 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.938 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=774557210}) 2024-09-15T01:50:00.938 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=503308868}) 2024-09-15T01:50:00.938 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.939 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=774557210}) 2024-09-15T01:50:00.939 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2305133985}) 2024-09-15T01:50:00.939 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.939 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=774557210}) 2024-09-15T01:50:00.939 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3210487337}) 2024-09-15T01:50:00.939 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.939 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=774557210}) 2024-09-15T01:50:00.939 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1275769765}) 2024-09-15T01:50:00.939 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.939 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=774557210}) 2024-09-15T01:50:00.939 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1738011785}) 2024-09-15T01:50:00.939 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.940 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=774557210}) 2024-09-15T01:50:00.940 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1228559385}) 2024-09-15T01:50:00.940 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.940 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=774557210}) 2024-09-15T01:50:00.940 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.580+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2023284675}) 2024-09-15T01:50:00.940 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.581+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.940 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.581+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1995951294}) 2024-09-15T01:50:00.940 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.581+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636436486}) 2024-09-15T01:50:00.940 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.581+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.940 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.581+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1995951294}) 2024-09-15T01:50:00.940 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2320598422}) 2024-09-15T01:50:00.941 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.941 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1995951294}) 2024-09-15T01:50:00.941 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2926317037}) 2024-09-15T01:50:00.941 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.941 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1995951294}) 2024-09-15T01:50:00.941 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2797806516}) 2024-09-15T01:50:00.941 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.941 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1995951294}) 2024-09-15T01:50:00.941 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=931394551}) 2024-09-15T01:50:00.941 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.941 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1995951294}) 2024-09-15T01:50:00.942 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3561979973}) 2024-09-15T01:50:00.942 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.942 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1995951294}) 2024-09-15T01:50:00.942 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=701178514}) 2024-09-15T01:50:00.942 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.942 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1995951294}) 2024-09-15T01:50:00.942 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.582+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924746244}) 2024-09-15T01:50:00.942 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.942 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=379496128}) 2024-09-15T01:50:00.942 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1085552413}) 2024-09-15T01:50:00.942 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=379496128}) 2024-09-15T01:50:00.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2189308950}) 2024-09-15T01:50:00.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=379496128}) 2024-09-15T01:50:00.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=902944092}) 2024-09-15T01:50:00.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=379496128}) 2024-09-15T01:50:00.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3928533021}) 2024-09-15T01:50:00.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=379496128}) 2024-09-15T01:50:00.943 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3448767034}) 2024-09-15T01:50:00.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=379496128}) 2024-09-15T01:50:00.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2455206693}) 2024-09-15T01:50:00.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=379496128}) 2024-09-15T01:50:00.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=961345535}) 2024-09-15T01:50:00.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=379496128}) 2024-09-15T01:50:00.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.583+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2387681140}) 2024-09-15T01:50:00.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.944 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1945122441}) 2024-09-15T01:50:00.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2516722502}) 2024-09-15T01:50:00.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1945122441}) 2024-09-15T01:50:00.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3580470237}) 2024-09-15T01:50:00.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1945122441}) 2024-09-15T01:50:00.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3584356317}) 2024-09-15T01:50:00.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1945122441}) 2024-09-15T01:50:00.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3748362072}) 2024-09-15T01:50:00.945 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1945122441}) 2024-09-15T01:50:00.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1121141494}) 2024-09-15T01:50:00.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1945122441}) 2024-09-15T01:50:00.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1226045253}) 2024-09-15T01:50:00.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1945122441}) 2024-09-15T01:50:00.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=611914149}) 2024-09-15T01:50:00.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1945122441}) 2024-09-15T01:50:00.946 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.584+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2074449386}) 2024-09-15T01:50:00.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340679011}) 2024-09-15T01:50:00.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4061055309}) 2024-09-15T01:50:00.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340679011}) 2024-09-15T01:50:00.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2303901215}) 2024-09-15T01:50:00.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340679011}) 2024-09-15T01:50:00.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2147517033}) 2024-09-15T01:50:00.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.947 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340679011}) 2024-09-15T01:50:00.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2575979367}) 2024-09-15T01:50:00.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.585+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340679011}) 2024-09-15T01:50:00.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.586+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1570904088}) 2024-09-15T01:50:00.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.586+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.586+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340679011}) 2024-09-15T01:50:00.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.586+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2103586519}) 2024-09-15T01:50:00.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.586+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.586+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340679011}) 2024-09-15T01:50:00.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.586+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=25133601}) 2024-09-15T01:50:00.948 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.586+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.586+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340679011}) 2024-09-15T01:50:00.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.586+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2490520087}) 2024-09-15T01:50:00.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2880127118}) 2024-09-15T01:50:00.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839062935}) 2024-09-15T01:50:00.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2880127118}) 2024-09-15T01:50:00.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=678568442}) 2024-09-15T01:50:00.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2880127118}) 2024-09-15T01:50:00.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1744955019}) 2024-09-15T01:50:00.949 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2880127118}) 2024-09-15T01:50:00.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3324641967}) 2024-09-15T01:50:00.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2880127118}) 2024-09-15T01:50:00.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3623251469}) 2024-09-15T01:50:00.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2880127118}) 2024-09-15T01:50:00.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3029596365}) 2024-09-15T01:50:00.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2880127118}) 2024-09-15T01:50:00.950 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.587+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4069105988}) 2024-09-15T01:50:00.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.588+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.588+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2880127118}) 2024-09-15T01:50:00.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.588+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3555447743}) 2024-09-15T01:50:00.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3381358995}) 2024-09-15T01:50:00.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2878258384}) 2024-09-15T01:50:00.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3381358995}) 2024-09-15T01:50:00.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=997891022}) 2024-09-15T01:50:00.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3381358995}) 2024-09-15T01:50:00.951 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2034224057}) 2024-09-15T01:50:00.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3381358995}) 2024-09-15T01:50:00.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=788126256}) 2024-09-15T01:50:00.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3381358995}) 2024-09-15T01:50:00.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1800206453}) 2024-09-15T01:50:00.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.952 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3381358995}) 2024-09-15T01:50:00.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1865009600}) 2024-09-15T01:50:00.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3381358995}) 2024-09-15T01:50:00.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3994418153}) 2024-09-15T01:50:00.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3381358995}) 2024-09-15T01:50:00.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.589+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3004436645}) 2024-09-15T01:50:00.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992921202}) 2024-09-15T01:50:00.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=307816033}) 2024-09-15T01:50:00.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.953 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992921202}) 2024-09-15T01:50:00.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=36016473}) 2024-09-15T01:50:00.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992921202}) 2024-09-15T01:50:00.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=196030643}) 2024-09-15T01:50:00.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992921202}) 2024-09-15T01:50:00.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1561577619}) 2024-09-15T01:50:00.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992921202}) 2024-09-15T01:50:00.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=626511251}) 2024-09-15T01:50:00.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.954 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992921202}) 2024-09-15T01:50:00.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2668314999}) 2024-09-15T01:50:00.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992921202}) 2024-09-15T01:50:00.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.590+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1202289481}) 2024-09-15T01:50:00.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.591+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.591+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992921202}) 2024-09-15T01:50:00.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.591+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3431193837}) 2024-09-15T01:50:00.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.591+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.591+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419567249}) 2024-09-15T01:50:00.955 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.591+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3117548639}) 2024-09-15T01:50:00.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419567249}) 2024-09-15T01:50:00.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=577536662}) 2024-09-15T01:50:00.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419567249}) 2024-09-15T01:50:00.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2313734880}) 2024-09-15T01:50:00.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419567249}) 2024-09-15T01:50:00.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4016452592}) 2024-09-15T01:50:00.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.956 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419567249}) 2024-09-15T01:50:00.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=387765166}) 2024-09-15T01:50:00.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419567249}) 2024-09-15T01:50:00.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=89509842}) 2024-09-15T01:50:00.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419567249}) 2024-09-15T01:50:00.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2602075964}) 2024-09-15T01:50:00.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419567249}) 2024-09-15T01:50:00.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.592+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=7197630}) 2024-09-15T01:50:00.957 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609181907}) 2024-09-15T01:50:00.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1899460636}) 2024-09-15T01:50:00.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609181907}) 2024-09-15T01:50:00.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=872133990}) 2024-09-15T01:50:00.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609181907}) 2024-09-15T01:50:00.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4028526695}) 2024-09-15T01:50:00.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609181907}) 2024-09-15T01:50:00.958 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2302108504}) 2024-09-15T01:50:00.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609181907}) 2024-09-15T01:50:00.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=202318690}) 2024-09-15T01:50:00.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609181907}) 2024-09-15T01:50:00.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1101574753}) 2024-09-15T01:50:00.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609181907}) 2024-09-15T01:50:00.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1796541672}) 2024-09-15T01:50:00.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.593+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.594+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609181907}) 2024-09-15T01:50:00.959 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.594+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1379976188}) 2024-09-15T01:50:00.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=514310282}) 2024-09-15T01:50:00.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1005328938}) 2024-09-15T01:50:00.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=514310282}) 2024-09-15T01:50:00.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3718930936}) 2024-09-15T01:50:00.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=514310282}) 2024-09-15T01:50:00.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3976452092}) 2024-09-15T01:50:00.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.960 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=514310282}) 2024-09-15T01:50:00.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3035677870}) 2024-09-15T01:50:00.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=514310282}) 2024-09-15T01:50:00.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1905605729}) 2024-09-15T01:50:00.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=514310282}) 2024-09-15T01:50:00.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1418956810}) 2024-09-15T01:50:00.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=514310282}) 2024-09-15T01:50:00.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=349195814}) 2024-09-15T01:50:00.961 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=514310282}) 2024-09-15T01:50:00.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.595+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1173891139}) 2024-09-15T01:50:00.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1683739598}) 2024-09-15T01:50:00.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4136000872}) 2024-09-15T01:50:00.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1683739598}) 2024-09-15T01:50:00.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3753458394}) 2024-09-15T01:50:00.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1683739598}) 2024-09-15T01:50:00.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=526244722}) 2024-09-15T01:50:00.962 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1683739598}) 2024-09-15T01:50:00.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=189265772}) 2024-09-15T01:50:00.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1683739598}) 2024-09-15T01:50:00.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=423313152}) 2024-09-15T01:50:00.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1683739598}) 2024-09-15T01:50:00.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1175951078}) 2024-09-15T01:50:00.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1683739598}) 2024-09-15T01:50:00.963 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3093220004}) 2024-09-15T01:50:00.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1683739598}) 2024-09-15T01:50:00.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.596+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3933997668}) 2024-09-15T01:50:00.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.597+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.597+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1262809022}) 2024-09-15T01:50:00.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.597+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1722213692}) 2024-09-15T01:50:00.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.597+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.597+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1262809022}) 2024-09-15T01:50:00.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1407803436}) 2024-09-15T01:50:00.964 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1262809022}) 2024-09-15T01:50:00.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2318070212}) 2024-09-15T01:50:00.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1262809022}) 2024-09-15T01:50:00.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3458351517}) 2024-09-15T01:50:00.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1262809022}) 2024-09-15T01:50:00.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3513442757}) 2024-09-15T01:50:00.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1262809022}) 2024-09-15T01:50:00.965 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3817921441}) 2024-09-15T01:50:00.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1262809022}) 2024-09-15T01:50:00.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1833385152}) 2024-09-15T01:50:00.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1262809022}) 2024-09-15T01:50:00.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.598+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1194493595}) 2024-09-15T01:50:00.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1127832259}) 2024-09-15T01:50:00.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=120433613}) 2024-09-15T01:50:00.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.966 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1127832259}) 2024-09-15T01:50:00.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2899221847}) 2024-09-15T01:50:00.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1127832259}) 2024-09-15T01:50:00.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2332045912}) 2024-09-15T01:50:00.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1127832259}) 2024-09-15T01:50:00.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=580632246}) 2024-09-15T01:50:00.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1127832259}) 2024-09-15T01:50:00.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1931568530}) 2024-09-15T01:50:00.967 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1127832259}) 2024-09-15T01:50:00.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1442951093}) 2024-09-15T01:50:00.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1127832259}) 2024-09-15T01:50:00.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1979627217}) 2024-09-15T01:50:00.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1127832259}) 2024-09-15T01:50:00.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.599+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=916411455}) 2024-09-15T01:50:00.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3536477662}) 2024-09-15T01:50:00.968 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561081873}) 2024-09-15T01:50:00.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3536477662}) 2024-09-15T01:50:00.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4070893214}) 2024-09-15T01:50:00.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3536477662}) 2024-09-15T01:50:00.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=850856797}) 2024-09-15T01:50:00.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3536477662}) 2024-09-15T01:50:00.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=663234957}) 2024-09-15T01:50:00.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.969 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3536477662}) 2024-09-15T01:50:00.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=583907638}) 2024-09-15T01:50:00.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3536477662}) 2024-09-15T01:50:00.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1355057067}) 2024-09-15T01:50:00.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.600+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3536477662}) 2024-09-15T01:50:00.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.601+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3153228453}) 2024-09-15T01:50:00.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.601+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.601+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3536477662}) 2024-09-15T01:50:00.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.601+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2547946824}) 2024-09-15T01:50:00.970 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2412126105}) 2024-09-15T01:50:00.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4118735638}) 2024-09-15T01:50:00.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2412126105}) 2024-09-15T01:50:00.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3975102076}) 2024-09-15T01:50:00.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2412126105}) 2024-09-15T01:50:00.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1641657350}) 2024-09-15T01:50:00.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2412126105}) 2024-09-15T01:50:00.971 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=705270017}) 2024-09-15T01:50:00.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2412126105}) 2024-09-15T01:50:00.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=965119128}) 2024-09-15T01:50:00.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2412126105}) 2024-09-15T01:50:00.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=479935872}) 2024-09-15T01:50:00.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2412126105}) 2024-09-15T01:50:00.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3068802294}) 2024-09-15T01:50:00.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.972 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2412126105}) 2024-09-15T01:50:00.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.602+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=622872012}) 2024-09-15T01:50:00.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178055893}) 2024-09-15T01:50:00.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4195999334}) 2024-09-15T01:50:00.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178055893}) 2024-09-15T01:50:00.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1891700186}) 2024-09-15T01:50:00.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178055893}) 2024-09-15T01:50:00.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=128278521}) 2024-09-15T01:50:00.973 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178055893}) 2024-09-15T01:50:00.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2983376521}) 2024-09-15T01:50:00.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178055893}) 2024-09-15T01:50:00.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3065986019}) 2024-09-15T01:50:00.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178055893}) 2024-09-15T01:50:00.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2638275418}) 2024-09-15T01:50:00.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.603+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178055893}) 2024-09-15T01:50:00.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.604+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3660149783}) 2024-09-15T01:50:00.974 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.604+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.604+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3178055893}) 2024-09-15T01:50:00.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.604+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1247375195}) 2024-09-15T01:50:00.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=946101348}) 2024-09-15T01:50:00.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1688760391}) 2024-09-15T01:50:00.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=946101348}) 2024-09-15T01:50:00.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2366511826}) 2024-09-15T01:50:00.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=946101348}) 2024-09-15T01:50:00.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2579892685}) 2024-09-15T01:50:00.975 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=946101348}) 2024-09-15T01:50:00.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4236141403}) 2024-09-15T01:50:00.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=946101348}) 2024-09-15T01:50:00.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3235775250}) 2024-09-15T01:50:00.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=946101348}) 2024-09-15T01:50:00.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1267306741}) 2024-09-15T01:50:00.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.976 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=946101348}) 2024-09-15T01:50:00.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3882676155}) 2024-09-15T01:50:00.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=946101348}) 2024-09-15T01:50:00.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.605+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=240939202}) 2024-09-15T01:50:00.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2507572849}) 2024-09-15T01:50:00.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1371811486}) 2024-09-15T01:50:00.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2507572849}) 2024-09-15T01:50:00.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3666298509}) 2024-09-15T01:50:00.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.977 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2507572849}) 2024-09-15T01:50:00.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1707750965}) 2024-09-15T01:50:00.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2507572849}) 2024-09-15T01:50:00.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=648917930}) 2024-09-15T01:50:00.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2507572849}) 2024-09-15T01:50:00.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2415550662}) 2024-09-15T01:50:00.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2507572849}) 2024-09-15T01:50:00.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1324780036}) 2024-09-15T01:50:00.978 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2507572849}) 2024-09-15T01:50:00.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1013043933}) 2024-09-15T01:50:00.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2507572849}) 2024-09-15T01:50:00.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.606+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=557419965}) 2024-09-15T01:50:00.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2485968082}) 2024-09-15T01:50:00.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3466958671}) 2024-09-15T01:50:00.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2485968082}) 2024-09-15T01:50:00.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3270877346}) 2024-09-15T01:50:00.979 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2485968082}) 2024-09-15T01:50:00.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2636605925}) 2024-09-15T01:50:00.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2485968082}) 2024-09-15T01:50:00.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3031226488}) 2024-09-15T01:50:00.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2485968082}) 2024-09-15T01:50:00.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3742500656}) 2024-09-15T01:50:00.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2485968082}) 2024-09-15T01:50:00.980 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3088488726}) 2024-09-15T01:50:00.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2485968082}) 2024-09-15T01:50:00.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1588089825}) 2024-09-15T01:50:00.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2485968082}) 2024-09-15T01:50:00.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.608+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3850725824}) 2024-09-15T01:50:00.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3308514720}) 2024-09-15T01:50:00.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2767283554}) 2024-09-15T01:50:00.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.981 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3308514720}) 2024-09-15T01:50:00.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3940960079}) 2024-09-15T01:50:00.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3308514720}) 2024-09-15T01:50:00.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=473365865}) 2024-09-15T01:50:00.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3308514720}) 2024-09-15T01:50:00.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=507793988}) 2024-09-15T01:50:00.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3308514720}) 2024-09-15T01:50:00.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1787899547}) 2024-09-15T01:50:00.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.982 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3308514720}) 2024-09-15T01:50:00.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3763059180}) 2024-09-15T01:50:00.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3308514720}) 2024-09-15T01:50:00.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3591436916}) 2024-09-15T01:50:00.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3308514720}) 2024-09-15T01:50:00.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.609+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1626668696}) 2024-09-15T01:50:00.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3457412597}) 2024-09-15T01:50:00.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3383265210}) 2024-09-15T01:50:00.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.983 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3457412597}) 2024-09-15T01:50:00.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=254747336}) 2024-09-15T01:50:00.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3457412597}) 2024-09-15T01:50:00.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4038028999}) 2024-09-15T01:50:00.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3457412597}) 2024-09-15T01:50:00.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4237461971}) 2024-09-15T01:50:00.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.610+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3457412597}) 2024-09-15T01:50:00.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.611+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3648728833}) 2024-09-15T01:50:00.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.611+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.984 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.611+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3457412597}) 2024-09-15T01:50:00.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.611+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1236712536}) 2024-09-15T01:50:00.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.611+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.611+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3457412597}) 2024-09-15T01:50:00.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.611+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3041422499}) 2024-09-15T01:50:00.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.611+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.611+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3457412597}) 2024-09-15T01:50:00.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.611+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4150325738}) 2024-09-15T01:50:00.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3290408188}) 2024-09-15T01:50:00.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3994916639}) 2024-09-15T01:50:00.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.985 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3290408188}) 2024-09-15T01:50:00.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=348563282}) 2024-09-15T01:50:00.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3290408188}) 2024-09-15T01:50:00.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=61276445}) 2024-09-15T01:50:00.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3290408188}) 2024-09-15T01:50:00.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3596069984}) 2024-09-15T01:50:00.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3290408188}) 2024-09-15T01:50:00.986 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1427184663}) 2024-09-15T01:50:00.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3290408188}) 2024-09-15T01:50:00.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2954819893}) 2024-09-15T01:50:00.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3290408188}) 2024-09-15T01:50:00.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1339222836}) 2024-09-15T01:50:00.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3290408188}) 2024-09-15T01:50:00.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.612+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=341604918}) 2024-09-15T01:50:00.987 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.613+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.613+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=61161262}) 2024-09-15T01:50:00.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.613+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2704185858}) 2024-09-15T01:50:00.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.613+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.613+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=61161262}) 2024-09-15T01:50:00.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.613+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4248098779}) 2024-09-15T01:50:00.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.613+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.613+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=61161262}) 2024-09-15T01:50:00.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1907560970}) 2024-09-15T01:50:00.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.988 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=61161262}) 2024-09-15T01:50:00.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2590826456}) 2024-09-15T01:50:00.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=61161262}) 2024-09-15T01:50:00.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=143826641}) 2024-09-15T01:50:00.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=61161262}) 2024-09-15T01:50:00.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1377311541}) 2024-09-15T01:50:00.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=61161262}) 2024-09-15T01:50:00.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2265683677}) 2024-09-15T01:50:00.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.989 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=61161262}) 2024-09-15T01:50:00.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.614+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3649139405}) 2024-09-15T01:50:00.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211387413}) 2024-09-15T01:50:00.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3930184289}) 2024-09-15T01:50:00.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211387413}) 2024-09-15T01:50:00.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=102891901}) 2024-09-15T01:50:00.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211387413}) 2024-09-15T01:50:00.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2823733146}) 2024-09-15T01:50:00.990 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211387413}) 2024-09-15T01:50:00.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1476597366}) 2024-09-15T01:50:00.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211387413}) 2024-09-15T01:50:00.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=645687334}) 2024-09-15T01:50:00.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211387413}) 2024-09-15T01:50:00.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1160825295}) 2024-09-15T01:50:00.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211387413}) 2024-09-15T01:50:00.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=674136207}) 2024-09-15T01:50:00.991 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1211387413}) 2024-09-15T01:50:00.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.615+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2170483077}) 2024-09-15T01:50:00.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.616+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.616+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3112455199}) 2024-09-15T01:50:00.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.616+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3808290221}) 2024-09-15T01:50:00.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.616+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.616+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3112455199}) 2024-09-15T01:50:00.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1881591044}) 2024-09-15T01:50:00.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3112455199}) 2024-09-15T01:50:00.992 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=295065372}) 2024-09-15T01:50:00.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3112455199}) 2024-09-15T01:50:00.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4174702582}) 2024-09-15T01:50:00.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3112455199}) 2024-09-15T01:50:00.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=407995413}) 2024-09-15T01:50:00.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3112455199}) 2024-09-15T01:50:00.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=233862850}) 2024-09-15T01:50:00.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3112455199}) 2024-09-15T01:50:00.993 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3336398399}) 2024-09-15T01:50:00.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3112455199}) 2024-09-15T01:50:00.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.617+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1341626315}) 2024-09-15T01:50:00.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3094380356}) 2024-09-15T01:50:00.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4156475427}) 2024-09-15T01:50:00.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3094380356}) 2024-09-15T01:50:00.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4188884281}) 2024-09-15T01:50:00.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.994 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3094380356}) 2024-09-15T01:50:00.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2616619526}) 2024-09-15T01:50:00.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3094380356}) 2024-09-15T01:50:00.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1985104342}) 2024-09-15T01:50:00.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3094380356}) 2024-09-15T01:50:00.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=832356720}) 2024-09-15T01:50:00.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3094380356}) 2024-09-15T01:50:00.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4041814064}) 2024-09-15T01:50:00.995 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3094380356}) 2024-09-15T01:50:00.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1764284543}) 2024-09-15T01:50:00.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3094380356}) 2024-09-15T01:50:00.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.618+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2740172725}) 2024-09-15T01:50:00.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2026777141}) 2024-09-15T01:50:00.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1165717231}) 2024-09-15T01:50:00.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2026777141}) 2024-09-15T01:50:00.996 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1879283842}) 2024-09-15T01:50:00.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2026777141}) 2024-09-15T01:50:00.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3461302534}) 2024-09-15T01:50:00.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2026777141}) 2024-09-15T01:50:00.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=64550224}) 2024-09-15T01:50:00.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2026777141}) 2024-09-15T01:50:00.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=107096864}) 2024-09-15T01:50:00.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2026777141}) 2024-09-15T01:50:00.997 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1322465132}) 2024-09-15T01:50:00.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2026777141}) 2024-09-15T01:50:00.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=609172656}) 2024-09-15T01:50:00.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2026777141}) 2024-09-15T01:50:00.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.619+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2538727340}) 2024-09-15T01:50:00.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.620+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.620+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=691031604}) 2024-09-15T01:50:00.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=179638922}) 2024-09-15T01:50:00.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.998 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=691031604}) 2024-09-15T01:50:00.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=860848920}) 2024-09-15T01:50:00.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=691031604}) 2024-09-15T01:50:00.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2383031347}) 2024-09-15T01:50:00.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=691031604}) 2024-09-15T01:50:00.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1664437030}) 2024-09-15T01:50:00.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:00.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=691031604}) 2024-09-15T01:50:00.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3509512090}) 2024-09-15T01:50:00.999 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=691031604}) 2024-09-15T01:50:01.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1749919440}) 2024-09-15T01:50:01.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=691031604}) 2024-09-15T01:50:01.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3464834014}) 2024-09-15T01:50:01.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=691031604}) 2024-09-15T01:50:01.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.621+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2722256008}) 2024-09-15T01:50:01.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3700364271}) 2024-09-15T01:50:01.000 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3415499710}) 2024-09-15T01:50:01.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3700364271}) 2024-09-15T01:50:01.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=339972139}) 2024-09-15T01:50:01.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3700364271}) 2024-09-15T01:50:01.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=952585955}) 2024-09-15T01:50:01.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3700364271}) 2024-09-15T01:50:01.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1515782078}) 2024-09-15T01:50:01.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3700364271}) 2024-09-15T01:50:01.001 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2973557827}) 2024-09-15T01:50:01.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3700364271}) 2024-09-15T01:50:01.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=216092168}) 2024-09-15T01:50:01.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3700364271}) 2024-09-15T01:50:01.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3467176816}) 2024-09-15T01:50:01.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3700364271}) 2024-09-15T01:50:01.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.622+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1957654788}) 2024-09-15T01:50:01.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.002 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801303253}) 2024-09-15T01:50:01.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3507372741}) 2024-09-15T01:50:01.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801303253}) 2024-09-15T01:50:01.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2477411959}) 2024-09-15T01:50:01.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801303253}) 2024-09-15T01:50:01.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=252297209}) 2024-09-15T01:50:01.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801303253}) 2024-09-15T01:50:01.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1047460701}) 2024-09-15T01:50:01.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.003 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801303253}) 2024-09-15T01:50:01.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3972973481}) 2024-09-15T01:50:01.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.623+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801303253}) 2024-09-15T01:50:01.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.624+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=58248387}) 2024-09-15T01:50:01.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.624+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.624+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801303253}) 2024-09-15T01:50:01.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.624+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2036905780}) 2024-09-15T01:50:01.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.624+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.624+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3801303253}) 2024-09-15T01:50:01.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.624+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1324604651}) 2024-09-15T01:50:01.004 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262659547}) 2024-09-15T01:50:01.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4037324772}) 2024-09-15T01:50:01.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262659547}) 2024-09-15T01:50:01.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1363853139}) 2024-09-15T01:50:01.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262659547}) 2024-09-15T01:50:01.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3280021970}) 2024-09-15T01:50:01.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262659547}) 2024-09-15T01:50:01.005 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2738827928}) 2024-09-15T01:50:01.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262659547}) 2024-09-15T01:50:01.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2452105706}) 2024-09-15T01:50:01.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262659547}) 2024-09-15T01:50:01.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1852457584}) 2024-09-15T01:50:01.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262659547}) 2024-09-15T01:50:01.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=98126005}) 2024-09-15T01:50:01.006 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262659547}) 2024-09-15T01:50:01.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.625+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3955611875}) 2024-09-15T01:50:01.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2561791650}) 2024-09-15T01:50:01.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292863316}) 2024-09-15T01:50:01.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2561791650}) 2024-09-15T01:50:01.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4154169014}) 2024-09-15T01:50:01.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2561791650}) 2024-09-15T01:50:01.007 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3566620183}) 2024-09-15T01:50:01.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2561791650}) 2024-09-15T01:50:01.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1629408563}) 2024-09-15T01:50:01.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2561791650}) 2024-09-15T01:50:01.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=487569093}) 2024-09-15T01:50:01.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2561791650}) 2024-09-15T01:50:01.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1200639724}) 2024-09-15T01:50:01.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.008 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2561791650}) 2024-09-15T01:50:01.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=309027519}) 2024-09-15T01:50:01.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2561791650}) 2024-09-15T01:50:01.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.626+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3800157902}) 2024-09-15T01:50:01.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.627+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.627+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1083973452}) 2024-09-15T01:50:01.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.627+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2253729901}) 2024-09-15T01:50:01.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.627+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.627+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1083973452}) 2024-09-15T01:50:01.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.627+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1475348588}) 2024-09-15T01:50:01.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.627+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.009 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.627+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1083973452}) 2024-09-15T01:50:01.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.627+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=681470319}) 2024-09-15T01:50:01.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1083973452}) 2024-09-15T01:50:01.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=134848851}) 2024-09-15T01:50:01.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1083973452}) 2024-09-15T01:50:01.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=315480775}) 2024-09-15T01:50:01.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1083973452}) 2024-09-15T01:50:01.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2306182940}) 2024-09-15T01:50:01.010 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1083973452}) 2024-09-15T01:50:01.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3318606140}) 2024-09-15T01:50:01.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1083973452}) 2024-09-15T01:50:01.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.628+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1664854805}) 2024-09-15T01:50:01.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1210907921}) 2024-09-15T01:50:01.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=26506105}) 2024-09-15T01:50:01.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1210907921}) 2024-09-15T01:50:01.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3383185033}) 2024-09-15T01:50:01.011 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1210907921}) 2024-09-15T01:50:01.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1315223897}) 2024-09-15T01:50:01.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1210907921}) 2024-09-15T01:50:01.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=891013318}) 2024-09-15T01:50:01.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1210907921}) 2024-09-15T01:50:01.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1244771967}) 2024-09-15T01:50:01.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1210907921}) 2024-09-15T01:50:01.012 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=158230409}) 2024-09-15T01:50:01.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1210907921}) 2024-09-15T01:50:01.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1898704076}) 2024-09-15T01:50:01.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1210907921}) 2024-09-15T01:50:01.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.629+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=721178720}) 2024-09-15T01:50:01.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=55863963}) 2024-09-15T01:50:01.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1211882417}) 2024-09-15T01:50:01.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.013 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=55863963}) 2024-09-15T01:50:01.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3161534853}) 2024-09-15T01:50:01.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=55863963}) 2024-09-15T01:50:01.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4145470395}) 2024-09-15T01:50:01.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=55863963}) 2024-09-15T01:50:01.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=6296778}) 2024-09-15T01:50:01.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=55863963}) 2024-09-15T01:50:01.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=524207557}) 2024-09-15T01:50:01.014 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=55863963}) 2024-09-15T01:50:01.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=929447367}) 2024-09-15T01:50:01.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=55863963}) 2024-09-15T01:50:01.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2630323760}) 2024-09-15T01:50:01.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.630+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=55863963}) 2024-09-15T01:50:01.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.631+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3725286599}) 2024-09-15T01:50:01.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.631+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.631+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=163730755}) 2024-09-15T01:50:01.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.631+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2202452869}) 2024-09-15T01:50:01.015 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=163730755}) 2024-09-15T01:50:01.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1687440394}) 2024-09-15T01:50:01.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=163730755}) 2024-09-15T01:50:01.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2004961181}) 2024-09-15T01:50:01.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=163730755}) 2024-09-15T01:50:01.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2326444348}) 2024-09-15T01:50:01.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=163730755}) 2024-09-15T01:50:01.016 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2192493577}) 2024-09-15T01:50:01.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=163730755}) 2024-09-15T01:50:01.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4239352020}) 2024-09-15T01:50:01.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=163730755}) 2024-09-15T01:50:01.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2254980062}) 2024-09-15T01:50:01.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=163730755}) 2024-09-15T01:50:01.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.632+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3170950159}) 2024-09-15T01:50:01.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=655834775}) 2024-09-15T01:50:01.017 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2885448157}) 2024-09-15T01:50:01.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=655834775}) 2024-09-15T01:50:01.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3309858089}) 2024-09-15T01:50:01.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=655834775}) 2024-09-15T01:50:01.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2556117073}) 2024-09-15T01:50:01.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=655834775}) 2024-09-15T01:50:01.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2536025268}) 2024-09-15T01:50:01.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=655834775}) 2024-09-15T01:50:01.018 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2139760515}) 2024-09-15T01:50:01.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=655834775}) 2024-09-15T01:50:01.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=399822827}) 2024-09-15T01:50:01.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=655834775}) 2024-09-15T01:50:01.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2379842070}) 2024-09-15T01:50:01.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=655834775}) 2024-09-15T01:50:01.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.633+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=840896383}) 2024-09-15T01:50:01.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.019 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=364779043}) 2024-09-15T01:50:01.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=993332822}) 2024-09-15T01:50:01.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=364779043}) 2024-09-15T01:50:01.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=62461716}) 2024-09-15T01:50:01.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=364779043}) 2024-09-15T01:50:01.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2111864643}) 2024-09-15T01:50:01.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=364779043}) 2024-09-15T01:50:01.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3731331775}) 2024-09-15T01:50:01.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.020 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=364779043}) 2024-09-15T01:50:01.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2459814494}) 2024-09-15T01:50:01.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.634+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=364779043}) 2024-09-15T01:50:01.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.635+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4263429921}) 2024-09-15T01:50:01.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.635+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.635+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=364779043}) 2024-09-15T01:50:01.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.635+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3101598037}) 2024-09-15T01:50:01.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.635+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.635+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=364779043}) 2024-09-15T01:50:01.021 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.635+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=551142486}) 2024-09-15T01:50:01.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=59,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=1} crc {logm=3937510698,mdsmap=1531151339,mgr=2268303531,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=22200351,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=669406158}) 2024-09-15T01:50:01.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=58,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=4,monmap=3,osd_metadata=5,osd_snap=2,osdmap=4} crc {logm=4109805245,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1253770707,mon_config_key=1692164896,monmap=2850442478,osd_metadata=4061885386,osd_snap=3178633393,osdmap=2232032458}) 2024-09-15T01:50:01.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=59,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=1} crc {logm=3937510698,mdsmap=1531151339,mgr=2268303531,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=22200351,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=669406158}) 2024-09-15T01:50:01.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=57,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=1} crc {logm=2518081371,mdsmap=1531151339,mgr=1788687062,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=22200351,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=669406158}) 2024-09-15T01:50:01.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=59,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=1} crc {logm=3937510698,mdsmap=1531151339,mgr=2268303531,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=22200351,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=669406158}) 2024-09-15T01:50:01.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=58,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2} crc {logm=34861848,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=22200351,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393}) 2024-09-15T01:50:01.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.022 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=59,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=1} crc {logm=3937510698,mdsmap=1531151339,mgr=2268303531,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=22200351,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=669406158}) 2024-09-15T01:50:01.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=58,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=2} crc {logm=2582763066,mdsmap=3585761405,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1919170993,mon_config_key=1111662797,monmap=2850442478,osd_metadata=3185943182,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=3244750139}) 2024-09-15T01:50:01.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=59,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=1} crc {logm=3937510698,mdsmap=1531151339,mgr=2268303531,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=22200351,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=669406158}) 2024-09-15T01:50:01.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=59,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2} crc {logm=2424462075,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1253770707,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905836,osd_pg_creating=2978587036,osd_snap=3178633393}) 2024-09-15T01:50:01.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=59,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=1} crc {logm=3937510698,mdsmap=1531151339,mgr=2268303531,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=22200351,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=669406158}) 2024-09-15T01:50:01.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=58,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=2} crc {logm=449661632,mdsmap=3585761405,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=22200351,mon_config_key=1111662797,monmap=1281023599,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=3244750139}) 2024-09-15T01:50:01.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=59,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=1} crc {logm=3937510698,mdsmap=1531151339,mgr=2268303531,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=22200351,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=669406158}) 2024-09-15T01:50:01.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=58,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=2} crc {logm=3759990191,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1287373993,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=956710772,osdmap=3325747870}) 2024-09-15T01:50:01.023 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=59,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=1} crc {logm=3937510698,mdsmap=1531151339,mgr=2268303531,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=22200351,mon_config_key=1111662797,monmap=2850442478,osd_metadata=1261905835,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=669406158}) 2024-09-15T01:50:01.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.636+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=58,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=2} crc {logm=3750646161,mdsmap=1531151339,mgr=2983835071,mgr_command_descs=3545737492,mgr_metadata=3150520333,mgrstat=1287373993,mon_config_key=3476930649,monmap=2850442478,osd_metadata=1202423041,osd_pg_creating=2978587036,osd_snap=3178633393,osdmap=3244750139}) 2024-09-15T01:50:01.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=51} crc {osdmap=1581028155}) 2024-09-15T01:50:01.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=51} crc {osdmap=2940291408}) 2024-09-15T01:50:01.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=51} crc {osdmap=1581028155}) 2024-09-15T01:50:01.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=50} crc {osdmap=545130232}) 2024-09-15T01:50:01.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.024 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=51} crc {osdmap=1581028155}) 2024-09-15T01:50:01.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=53} crc {osdmap=4280671046}) 2024-09-15T01:50:01.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=51} crc {osdmap=1581028155}) 2024-09-15T01:50:01.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=53} crc {osdmap=1252126219}) 2024-09-15T01:50:01.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=51} crc {osdmap=1581028155}) 2024-09-15T01:50:01.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=51} crc {osdmap=2496938210}) 2024-09-15T01:50:01.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=51} crc {osdmap=1581028155}) 2024-09-15T01:50:01.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=50} crc {osdmap=286476308}) 2024-09-15T01:50:01.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.025 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=51} crc {osdmap=1581028155}) 2024-09-15T01:50:01.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=53} crc {osdmap=1469971767}) 2024-09-15T01:50:01.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T01:50:01.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=51} crc {osdmap=1581028155}) 2024-09-15T01:50:01.026 INFO:tasks.ceph.mon.h.smithi204.stderr:2024-09-15T01:50:00.643+0000 7fa9c23f1640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=52} crc {osdmap=1444996348}) 2024-09-15T01:50:01.478 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T01:50:01.479 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T01:50:01.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.571+0000 7f233b6a8640 10 monclient: get_monmap_and_config 2024-09-15T01:50:01.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.571+0000 7f233b6a8640 10 monclient: build_initial_monmap 2024-09-15T01:50:01.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.571+0000 7f233b6a8640 10 monclient: monmap: 2024-09-15T01:50:01.573 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:01.573 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:01.573 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:01.572055+0000 2024-09-15T01:50:01.573 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:01.572055+0000 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:01.574 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:01.575 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:01.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.571+0000 7f233b6a8640 1 Processor -- start 2024-09-15T01:50:01.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 1 -- start start 2024-09-15T01:50:01.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 10 monclient: init 2024-09-15T01:50:01.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:01.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 10 monclient: _add_conns ranks=[4,5,10,2,3,9,0,8,7,1,6] 2024-09-15T01:50:01.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2334067130 0x7f2334067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 10 monclient(hunting): picked mon.noname-c con 0x7f2334067130 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f233409d580 0x7f2334067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 10 monclient(hunting): picked mon.noname-h con 0x7f233409d580 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f233409f000 0x7f233412d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 10 monclient(hunting): picked mon.noname-f con 0x7f233409f000 addr v2:172.21.15.204:3300/0 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f233409e0c0 con 0x7f233409f000 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2334067f80 con 0x7f2334067130 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f233409dde0 con 0x7f233409d580 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f233b6a8640 10 monclient(hunting): authenticate will time out at 1806.322594s 2024-09-15T01:50:01.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339444640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2334067130 0x7f2334067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339444640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2334067130 0x7f2334067500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44772/0 (socket says 172.21.15.178:44772) 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339444640 1 -- 172.21.15.178:0/2095232059 learned_addr learned my addr 172.21.15.178:0/2095232059 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339444640 10 monclient(hunting): get_auth_request con 0x7f2334067130 auth_method 0 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339444640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339444640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339444640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339444640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339444640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339444640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2338c43640 1 --2- 172.21.15.178:0/2095232059 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f233409d580 0x7f2334067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339c45640 1 --2- 172.21.15.178:0/2095232059 >> v2:172.21.15.204:3300/0 conn(0x7f233409f000 0x7f233412d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339c45640 10 monclient(hunting): get_auth_request con 0x7f233409f000 auth_method 0 2024-09-15T01:50:01.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339c45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339c45640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339c45640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2338c43640 10 monclient(hunting): get_auth_request con 0x7f233409d580 auth_method 0 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2338c43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2338c43640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2338c43640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339c45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.572+0000 7f2339c45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2339c45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2338c43640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2338c43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2338c43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2339444640 10 monclient(hunting): handle_auth_done global_id 45076 payload 274 2024-09-15T01:50:01.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2339444640 1 -- 172.21.15.178:0/2095232059 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f233409d580 msgr2=0x7f2334067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2339444640 1 --2- 172.21.15.178:0/2095232059 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f233409d580 0x7f2334067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2339444640 1 -- 172.21.15.178:0/2095232059 >> v2:172.21.15.204:3300/0 conn(0x7f233409f000 msgr2=0x7f233412d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2339444640 1 --2- 172.21.15.178:0/2095232059 >> v2:172.21.15.204:3300/0 conn(0x7f233409f000 0x7f233412d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2339444640 10 monclient: _finish_hunting 0 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2339444640 1 monclient: found mon.noname-c 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2339444640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2339444640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2339444640 1 -- 172.21.15.178:0/2095232059 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233404bf60 con 0x7f2334067130 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f2339444640 1 --2- 172.21.15.178:0/2095232059 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2334067130 0x7f2334067500 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2330009910 tx=0x7f23300091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8ed6d2f8381041bf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f232bfff640 1 -- 172.21.15.178:0/2095232059 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f233002ac80 con 0x7f2334067130 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f232bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f232bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:01.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f232bfff640 10 monclient: dump: 2024-09-15T01:50:01.588 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:01.588 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:01.588 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.588 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.588 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:01.588 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:01.588 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:01.588 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:01.588 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:01.588 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:01.588 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:01.589 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:01.589 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:01.589 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:01.589 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:01.589 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:01.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f232bfff640 10 monclient: _finish_auth 0 2024-09-15T01:50:01.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f232bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:01.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f233b6a8640 5 monclient: authenticate success, global_id 45076 2024-09-15T01:50:01.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f232bfff640 1 -- 172.21.15.178:0/2095232059 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233002ae00 con 0x7f2334067130 2024-09-15T01:50:01.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f233b6a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:01.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f232bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:01.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f233b6a8640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f233b6a8640 4 set_mon_vals no callback set 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f232bfff640 1 -- 172.21.15.178:0/2095232059 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2330031ca0 con 0x7f2334067130 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f232bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f232bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.573+0000 7f232bfff640 10 monclient: dump: 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:01.590 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:01.591 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:01.591 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:01.591 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:01.591 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:01.591 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:01.591 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:01.591 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:01.591 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:01.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.574+0000 7f233b6a8640 10 monclient: shutdown 2024-09-15T01:50:01.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.574+0000 7f233b6a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:01.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.574+0000 7f233b6a8640 1 -- 172.21.15.178:0/2095232059 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2334067130 msgr2=0x7f2334067500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:01.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.574+0000 7f233b6a8640 1 --2- 172.21.15.178:0/2095232059 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2334067130 0x7f2334067500 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2330009910 tx=0x7f23300091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.574+0000 7f233b6a8640 1 -- 172.21.15.178:0/2095232059 shutdown_connections 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.574+0000 7f233b6a8640 1 --2- 172.21.15.178:0/2095232059 >> v2:172.21.15.204:3300/0 conn(0x7f233409f000 0x7f233412d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.574+0000 7f233b6a8640 1 --2- 172.21.15.178:0/2095232059 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f233409d580 0x7f2334067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.574+0000 7f233b6a8640 1 --2- 172.21.15.178:0/2095232059 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f2334067130 0x7f2334067500 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.574+0000 7f233b6a8640 1 -- 172.21.15.178:0/2095232059 >> 172.21.15.178:0/2095232059 conn(0x7f2334068a60 msgr2=0x7f23340a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.574+0000 7f233b6a8640 1 -- 172.21.15.178:0/2095232059 shutdown_connections 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.574+0000 7f233b6a8640 1 -- 172.21.15.178:0/2095232059 wait complete. 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.575+0000 7f233b6a8640 10 monclient: build_initial_monmap 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.575+0000 7f233b6a8640 10 monclient: monmap: 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:01.592 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.575+0000 7f233b6a8640 1 Processor -- start 2024-09-15T01:50:01.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.575+0000 7f233b6a8640 1 -- start start 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.575+0000 7f233b6a8640 10 monclient: init 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.575+0000 7f233b6a8640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient: _add_conns ranks=[1,5,8,7,4,6,3,0,2] 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2334067130 0x7f233413c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient(hunting): picked mon.noname-f con 0x7f2334067130 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f233409d580 0x7f233413b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient(hunting): picked mon.noname-h con 0x7f233409d580 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f233409f000 0x7f233413f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient(hunting): picked mon.noname-e con 0x7f233409f000 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23341212b0 con 0x7f2334067130 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2334120cf0 con 0x7f233409d580 2024-09-15T01:50:01.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f233411e7b0 con 0x7f233409f000 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient(hunting): authenticate will time out at 1806.326098s 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f233409f000 0x7f233413f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f233409f000 0x7f233413f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40070/0 (socket says 172.21.15.178:40070) 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339444640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2334067130 0x7f233413c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 1 -- 172.21.15.178:0/935408150 learned_addr learned my addr 172.21.15.178:0/935408150 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 10 monclient(hunting): get_auth_request con 0x7f233409f000 auth_method 0 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2338c43640 1 --2- 172.21.15.178:0/935408150 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f233409d580 0x7f233413b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339444640 10 monclient(hunting): get_auth_request con 0x7f2334067130 auth_method 0 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339444640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339444640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339444640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2338c43640 10 monclient(hunting): get_auth_request con 0x7f233409d580 auth_method 0 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2338c43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2338c43640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2338c43640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339444640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339444640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:01.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339444640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2338c43640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2338c43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2338c43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 10 monclient(hunting): handle_auth_done global_id 94490 payload 824 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 1 -- 172.21.15.178:0/935408150 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f233409d580 msgr2=0x7f233413b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 1 --2- 172.21.15.178:0/935408150 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f233409d580 0x7f233413b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 1 -- 172.21.15.178:0/935408150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2334067130 msgr2=0x7f233413c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 1 --2- 172.21.15.178:0/935408150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2334067130 0x7f233413c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 10 monclient: _finish_hunting 0 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 1 monclient: found mon.noname-e 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 1 -- 172.21.15.178:0/935408150 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2330009450 con 0x7f233409f000 2024-09-15T01:50:01.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f2339c45640 1 --2- 172.21.15.178:0/935408150 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f233409f000 0x7f233413f830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f232c009d70 tx=0x7f232c00b040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f23297fa640 1 -- 172.21.15.178:0/935408150 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f232c009ad0 con 0x7f233409f000 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f23297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f23297fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f23297fa640 10 monclient: dump: 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:01.598 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f23297fa640 10 monclient: _finish_auth 0 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f23297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 5 monclient: authenticate success, global_id 94490 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f23297fa640 1 -- 172.21.15.178:0/935408150 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c00f040 con 0x7f233409f000 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient: _renew_subs 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:01.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 1 -- 172.21.15.178:0/935408150 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2334120a10 con 0x7f233409f000 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f23297fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f23297fa640 1 -- 172.21.15.178:0/935408150 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f232c014d80 con 0x7f233409f000 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f23297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f23297fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f232b7fe640 4 set_mon_vals no callback set 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f23297fa640 10 monclient: dump: 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:01.600 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:01.601 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:01.601 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:01.601 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:01.601 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:01.601 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:01.601 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:01.601 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:01.601 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:01.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient: _renew_subs 2024-09-15T01:50:01.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:01.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.576+0000 7f233b6a8640 1 -- 172.21.15.178:0/935408150 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233413beb0 con 0x7f233409f000 2024-09-15T01:50:01.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.577+0000 7f23297fa640 1 -- 172.21.15.178:0/935408150 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f232c00cd40 con 0x7f233409f000 2024-09-15T01:50:01.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.577+0000 7f233b6a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:01.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.577+0000 7f233b6a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:01.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.577+0000 7f233b6a8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:01.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.577+0000 7f23297fa640 1 --2- 172.21.15.178:0/935408150 >> v2:172.21.15.204:6812/28210 conn(0x7f23180397b0 0x7f231803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.577+0000 7f233b6a8640 1 -- 172.21.15.178:0/935408150 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233404b720 con 0x7f233409f000 2024-09-15T01:50:01.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.577+0000 7f23297fa640 1 -- 172.21.15.178:0/935408150 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f232c04c670 con 0x7f233409f000 2024-09-15T01:50:01.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.578+0000 7f2339444640 1 --2- 172.21.15.178:0/935408150 >> v2:172.21.15.204:6812/28210 conn(0x7f23180397b0 0x7f231803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.578+0000 7f2339444640 10 monclient: get_auth_request con 0x7f23180397b0 auth_method 0 2024-09-15T01:50:01.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.578+0000 7f2339444640 1 --2- 172.21.15.178:0/935408150 >> v2:172.21.15.204:6812/28210 conn(0x7f23180397b0 0x7f231803bc60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2330034000 tx=0x7f2330032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:01.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.582+0000 7f23297fa640 1 -- 172.21.15.178:0/935408150 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f232c011aa0 con 0x7f233409f000 2024-09-15T01:50:01.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.582+0000 7f23297fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:01.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.582+0000 7f23297fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:01.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.795+0000 7f233b6a8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T01:50:01.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.795+0000 7f233b6a8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T01:50:01.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.795+0000 7f233b6a8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:01.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.795+0000 7f233b6a8640 1 -- 172.21.15.178:0/935408150 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2334070770 con 0x7f233409f000 2024-09-15T01:50:01.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.795+0000 7f23297fa640 1 -- 172.21.15.178:0/935408150 <== mon.8 v2:172.21.15.178:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7f232c018020 con 0x7f233409f000 2024-09-15T01:50:01.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.795+0000 7f23297fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T01:50:01.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.795+0000 7f23297fa640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:50:01.798 INFO:teuthology.orchestra.run.smithi178.stdout:{"election_epoch":158,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","modified":"2024-09-15T01:31:38.100732Z","created":"2024-09-15T01:31:38.100732Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3301","nonce":0},{"type":"v1","addr":"172.21.15.178:6790","nonce":0}]},"addr":"172.21.15.178:6790/0","public_addr":"172.21.15.178:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3301","nonce":0},{"type":"v1","addr":"172.21.15.204:6790","nonce":0}]},"addr":"172.21.15.204:6790/0","public_addr":"172.21.15.204:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3302","nonce":0},{"type":"v1","addr":"172.21.15.178:6791","nonce":0}]},"addr":"172.21.15.178:6791/0","public_addr":"172.21.15.178:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3302","nonce":0},{"type":"v1","addr":"172.21.15.204:6791","nonce":0}]},"addr":"172.21.15.204:6791/0","public_addr":"172.21.15.204:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3303","nonce":0},{"type":"v1","addr":"172.21.15.178:6792","nonce":0}]},"addr":"172.21.15.178:6792/0","public_addr":"172.21.15.178:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3303","nonce":0},{"type":"v1","addr":"172.21.15.204:6792","nonce":0}]},"addr":"172.21.15.204:6792/0","public_addr":"172.21.15.204:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3304","nonce":0},{"type":"v1","addr":"172.21.15.178:6793","nonce":0}]},"addr":"172.21.15.178:6793/0","public_addr":"172.21.15.178:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T01:50:01.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 1 -- 172.21.15.178:0/935408150 >> v2:172.21.15.204:6812/28210 conn(0x7f23180397b0 msgr2=0x7f231803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:01.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 1 --2- 172.21.15.178:0/935408150 >> v2:172.21.15.204:6812/28210 conn(0x7f23180397b0 0x7f231803bc60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2330034000 tx=0x7f2330032000 comp rx=0 tx=0).stop 2024-09-15T01:50:01.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 10 monclient: shutdown 2024-09-15T01:50:01.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:01.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 1 -- 172.21.15.178:0/935408150 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f233409f000 msgr2=0x7f233413f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:01.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 1 --2- 172.21.15.178:0/935408150 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f233409f000 0x7f233413f830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f232c009d70 tx=0x7f232c00b040 comp rx=0 tx=0).stop 2024-09-15T01:50:01.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 1 -- 172.21.15.178:0/935408150 shutdown_connections 2024-09-15T01:50:01.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 1 --2- 172.21.15.178:0/935408150 >> v2:172.21.15.204:6812/28210 conn(0x7f23180397b0 0x7f231803bc60 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 1 --2- 172.21.15.178:0/935408150 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f233409f000 0x7f233413f830 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 1 --2- 172.21.15.178:0/935408150 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f233409d580 0x7f233413b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 1 --2- 172.21.15.178:0/935408150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2334067130 0x7f233413c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 1 -- 172.21.15.178:0/935408150 >> 172.21.15.178:0/935408150 conn(0x7f2334068a60 msgr2=0x7f23340a09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:01.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 1 -- 172.21.15.178:0/935408150 shutdown_connections 2024-09-15T01:50:01.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.797+0000 7f233b6a8640 1 -- 172.21.15.178:0/935408150 wait complete. 2024-09-15T01:50:01.810 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T01:50:01.810 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T01:50:01.820 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-15T01:50:01.820 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:50:01.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.911+0000 7fb650ce8640 10 monclient: get_monmap_and_config 2024-09-15T01:50:01.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.911+0000 7fb650ce8640 10 monclient: build_initial_monmap 2024-09-15T01:50:01.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.911+0000 7fb650ce8640 10 monclient: monmap: 2024-09-15T01:50:01.913 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:01.912453+0000 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:01.912453+0000 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:01.914 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:01.915 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:01.915 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:01.915 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:01.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 1 Processor -- start 2024-09-15T01:50:01.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 1 -- start start 2024-09-15T01:50:01.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 10 monclient: init 2024-09-15T01:50:01.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:01.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 10 monclient: _add_conns ranks=[4,5,6,9,7,3,2,1,8,10,0] 2024-09-15T01:50:01.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c067130 0x7fb64c067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 10 monclient(hunting): picked mon.noname-c con 0x7fb64c067130 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:01.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb64c09d580 0x7fb64c067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 10 monclient(hunting): picked mon.noname-h con 0x7fb64c09d580 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:01.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb64c09f000 0x7fb64c12d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 10 monclient(hunting): picked mon.noname-d con 0x7fb64c09f000 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:01.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:01.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c09e710 con 0x7fb64c067130 2024-09-15T01:50:01.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c09e0c0 con 0x7fb64c09d580 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c067f80 con 0x7fb64c09f000 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb650ce8640 10 monclient(hunting): authenticate will time out at 1806.662640s 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64a59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c067130 0x7fb64c067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64ad9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb64c09f000 0x7fb64c12d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64a59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c067130 0x7fb64c067500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44788/0 (socket says 172.21.15.178:44788) 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64a59c640 1 -- 172.21.15.178:0/1661450061 learned_addr learned my addr 172.21.15.178:0/1661450061 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64a59c640 10 monclient(hunting): get_auth_request con 0x7fb64c067130 auth_method 0 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:01.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb641d9b640 1 --2- 172.21.15.178:0/1661450061 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb64c09d580 0x7fb64c067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64ad9d640 10 monclient(hunting): get_auth_request con 0x7fb64c09f000 auth_method 0 2024-09-15T01:50:01.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:01.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:01.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:01.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb641d9b640 10 monclient(hunting): get_auth_request con 0x7fb64c09d580 auth_method 0 2024-09-15T01:50:01.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb641d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:01.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb641d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:01.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb641d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:01.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:01.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:01.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:01.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:01.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:01.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb64ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:01.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb641d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:01.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb641d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:01.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.912+0000 7fb641d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:01.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb64ad9d640 10 monclient(hunting): handle_auth_done global_id 25926 payload 274 2024-09-15T01:50:01.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb64ad9d640 1 -- 172.21.15.178:0/1661450061 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb64c09d580 msgr2=0x7fb64c067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:01.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb64ad9d640 1 --2- 172.21.15.178:0/1661450061 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb64c09d580 0x7fb64c067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb64ad9d640 1 -- 172.21.15.178:0/1661450061 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c067130 msgr2=0x7fb64c067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:01.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb64ad9d640 1 --2- 172.21.15.178:0/1661450061 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c067130 0x7fb64c067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb64ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb64ad9d640 1 monclient: found mon.noname-d 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb64ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb64ad9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb64ad9d640 1 -- 172.21.15.178:0/1661450061 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c04bf60 con 0x7fb64c09f000 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb64a59c640 1 --2- 172.21.15.178:0/1661450061 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c067130 0x7fb64c067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb64ad9d640 1 --2- 172.21.15.178:0/1661450061 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb64c09f000 0x7fb64c12d330 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb644009e90 tx=0x7fb644009780 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e006235d65e287cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.913+0000 7fb649d9b640 1 -- 172.21.15.178:0/1661450061 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6440079d0 con 0x7fb64c09f000 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb649d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb649d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb649d9b640 10 monclient: dump: 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:01.929 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:01.930 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb649d9b640 10 monclient: _finish_auth 0 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb649d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 5 monclient: authenticate success, global_id 25926 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb649d9b640 1 -- 172.21.15.178:0/1661450061 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64402d050 con 0x7fb64c09f000 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb649d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb649d9b640 1 -- 172.21.15.178:0/1661450061 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb644007bd0 con 0x7fb64c09f000 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb649d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 4 set_mon_vals no callback set 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb649d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb649d9b640 10 monclient: dump: 2024-09-15T01:50:01.931 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:01.932 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:01.932 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.932 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.932 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:01.932 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:01.932 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:01.932 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:01.932 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:01.932 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:01.932 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:01.933 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:01.933 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:01.933 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:01.933 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:01.933 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:01.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 10 monclient: shutdown 2024-09-15T01:50:01.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:01.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 1 -- 172.21.15.178:0/1661450061 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb64c09f000 msgr2=0x7fb64c12d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:01.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 1 --2- 172.21.15.178:0/1661450061 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb64c09f000 0x7fb64c12d330 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb644009e90 tx=0x7fb644009780 comp rx=0 tx=0).stop 2024-09-15T01:50:01.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 1 -- 172.21.15.178:0/1661450061 shutdown_connections 2024-09-15T01:50:01.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 1 --2- 172.21.15.178:0/1661450061 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fb64c09f000 0x7fb64c12d330 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 1 --2- 172.21.15.178:0/1661450061 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fb64c09d580 0x7fb64c067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 1 --2- 172.21.15.178:0/1661450061 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c067130 0x7fb64c067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 1 -- 172.21.15.178:0/1661450061 >> 172.21.15.178:0/1661450061 conn(0x7fb64c068a60 msgr2=0x7fb64c0a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 1 -- 172.21.15.178:0/1661450061 shutdown_connections 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.914+0000 7fb650ce8640 1 -- 172.21.15.178:0/1661450061 wait complete. 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient: build_initial_monmap 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient: monmap: 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:01.934 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:01.935 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:01.935 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:01.935 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:01.935 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:01.935 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:01.935 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:01.935 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:01.935 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:01.935 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:01.935 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:01.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 1 Processor -- start 2024-09-15T01:50:01.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 1 -- start start 2024-09-15T01:50:01.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient: init 2024-09-15T01:50:01.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:01.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient: _add_conns ranks=[3,2,4,0,5,7,1,8,6] 2024-09-15T01:50:01.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb64c067130 0x7fb64c0972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient(hunting): picked mon.noname-g con 0x7fb64c067130 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:01.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb64c09d580 0x7fb64c097800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient(hunting): picked mon.noname-b con 0x7fb64c09d580 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:01.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c09f000 0x7fb64c097d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient(hunting): picked mon.noname-c con 0x7fb64c09f000 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:01.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:01.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c09af60 con 0x7fb64c09d580 2024-09-15T01:50:01.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:01.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c09cb50 con 0x7fb64c067130 2024-09-15T01:50:01.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:01.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c09d9e0 con 0x7fb64c09f000 2024-09-15T01:50:01.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:01.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.915+0000 7fb650ce8640 10 monclient(hunting): authenticate will time out at 1806.665968s 2024-09-15T01:50:01.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c09f000 0x7fb64c097d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb641d9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb64c09d580 0x7fb64c097800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c09f000 0x7fb64c097d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44790/0 (socket says 172.21.15.178:44790) 2024-09-15T01:50:01.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 1 -- 172.21.15.178:0/207896582 learned_addr learned my addr 172.21.15.178:0/207896582 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:01.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 10 monclient(hunting): get_auth_request con 0x7fb64c09f000 auth_method 0 2024-09-15T01:50:01.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:01.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb641d9b640 10 monclient(hunting): get_auth_request con 0x7fb64c09d580 auth_method 0 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb641d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb641d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb641d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64a59c640 1 --2- 172.21.15.178:0/207896582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb64c067130 0x7fb64c0972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64a59c640 10 monclient(hunting): get_auth_request con 0x7fb64c067130 auth_method 0 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64a59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:01.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb641d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb641d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb641d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 10 monclient(hunting): handle_auth_done global_id 45094 payload 824 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 1 -- 172.21.15.178:0/207896582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb64c067130 msgr2=0x7fb64c0972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 1 --2- 172.21.15.178:0/207896582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb64c067130 0x7fb64c0972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 1 -- 172.21.15.178:0/207896582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb64c09d580 msgr2=0x7fb64c097800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 1 --2- 172.21.15.178:0/207896582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb64c09d580 0x7fb64c097800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:50:01.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 1 monclient: found mon.noname-c 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 1 -- 172.21.15.178:0/207896582 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c009450 con 0x7fb64c09f000 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb641d9b640 1 --2- 172.21.15.178:0/207896582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb64c09d580 0x7fb64c097800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64ad9d640 1 --2- 172.21.15.178:0/207896582 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c09f000 0x7fb64c097d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb644009e90 tx=0x7fb644037000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb642ffd640 1 -- 172.21.15.178:0/207896582 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb644007e00 con 0x7fb64c09f000 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb642ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb642ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb642ffd640 10 monclient: dump: 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.940 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:01.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb642ffd640 10 monclient: _finish_auth 0 2024-09-15T01:50:01.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb642ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:01.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb650ce8640 5 monclient: authenticate success, global_id 45094 2024-09-15T01:50:01.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb642ffd640 1 -- 172.21.15.178:0/207896582 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64403d430 con 0x7fb64c09f000 2024-09-15T01:50:01.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb642ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:01.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb650ce8640 10 monclient: _renew_subs 2024-09-15T01:50:01.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb642ffd640 1 -- 172.21.15.178:0/207896582 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb64403da20 con 0x7fb64c09f000 2024-09-15T01:50:01.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb650ce8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:01.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb650ce8640 1 -- 172.21.15.178:0/207896582 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb644009a20 con 0x7fb64c09f000 2024-09-15T01:50:01.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb642ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:01.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb642ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb64959a640 4 set_mon_vals no callback set 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb642ffd640 10 monclient: dump: 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:01.943 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:01.944 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:01.944 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:01.944 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:01.944 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:01.944 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:01.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb650ce8640 10 monclient: _renew_subs 2024-09-15T01:50:01.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb650ce8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:01.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.916+0000 7fb650ce8640 1 -- 172.21.15.178:0/207896582 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64c098e00 con 0x7fb64c09f000 2024-09-15T01:50:01.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.917+0000 7fb642ffd640 1 -- 172.21.15.178:0/207896582 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb644003040 con 0x7fb64c09f000 2024-09-15T01:50:01.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.917+0000 7fb650ce8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:01.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.917+0000 7fb650ce8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:01.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.917+0000 7fb650ce8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:01.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.917+0000 7fb650ce8640 1 -- 172.21.15.178:0/207896582 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64c04b720 con 0x7fb64c09f000 2024-09-15T01:50:01.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.917+0000 7fb642ffd640 1 --2- 172.21.15.178:0/207896582 >> v2:172.21.15.204:6812/28210 conn(0x7fb62c0397b0 0x7fb62c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:01.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.917+0000 7fb642ffd640 1 -- 172.21.15.178:0/207896582 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fb64406d3d0 con 0x7fb64c09f000 2024-09-15T01:50:01.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.918+0000 7fb64a59c640 1 --2- 172.21.15.178:0/207896582 >> v2:172.21.15.204:6812/28210 conn(0x7fb62c0397b0 0x7fb62c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:01.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.918+0000 7fb64a59c640 10 monclient: get_auth_request con 0x7fb62c0397b0 auth_method 0 2024-09-15T01:50:01.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.918+0000 7fb64a59c640 1 --2- 172.21.15.178:0/207896582 >> v2:172.21.15.204:6812/28210 conn(0x7fb62c0397b0 0x7fb62c03bc60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb63c006c90 tx=0x7fb63c0091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:01.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.922+0000 7fb642ffd640 1 -- 172.21.15.178:0/207896582 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb644044590 con 0x7fb64c09f000 2024-09-15T01:50:01.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.922+0000 7fb642ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:01.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:01.922+0000 7fb642ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:02.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.077+0000 7fb650ce8640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:50:02.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.077+0000 7fb650ce8640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:50:02.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.077+0000 7fb650ce8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:02.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.077+0000 7fb650ce8640 1 -- 172.21.15.178:0/207896582 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb64c070770 con 0x7fb64c09f000 2024-09-15T01:50:02.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.078+0000 7fb642ffd640 1 -- 172.21.15.178:0/207896582 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v265) v1 ==== 74+0+19230 (secure 0 0 0) 0x7fb6440330c0 con 0x7fb64c09f000 2024-09-15T01:50:02.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.078+0000 7fb642ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:50:02.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.078+0000 7fb642ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:50:02.081 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:50:02.081 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":265,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","created":"2024-09-15T01:31:51.657329+0000","modified":"2024-09-15T01:44:29.155274+0000","last_up_change":"2024-09-15T01:31:56.999458+0000","last_in_change":"2024-09-15T01:44:29.155274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":181,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":15,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:31:58.183767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:32:02.130544+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"fooset","create_time":"2024-09-15T01:35:26.816876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}},{"pool":10,"pool_name":"barset","create_time":"2024-09-15T01:35:27.950688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"690d2bc0-3a5e-4f4a-b00b-43a73741b7b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":29232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6801","nonce":29232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":29232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":29232}]},"public_addr":"172.21.15.178:6800/29232","cluster_addr":"172.21.15.178:6801/29232","heartbeat_back_addr":"172.21.15.178:6803/29232","heartbeat_front_addr":"172.21.15.178:6802/29232","state":["exists","up"]},{"osd":1,"uuid":"bbba9997-deb4-42ca-8140-6553a9b6c45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":29233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6805","nonce":29233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6807","nonce":29233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":29233}]},"public_addr":"172.21.15.178:6804/29233","cluster_addr":"172.21.15.178:6805/29233","heartbeat_back_addr":"172.21.15.178:6807/29233","heartbeat_front_addr":"172.21.15.178:6806/29233","state":["exists","up"]},{"osd":2,"uuid":"24b56e74-a222-40f9-a8bf-38d597553f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":29234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":29234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6811","nonce":29234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":29234}]},"public_addr":"172.21.15.178:6808/29234","cluster_addr":"172.21.15.178:6809/29234","heartbeat_back_addr":"172.21.15.178:6811/29234","heartbeat_front_addr":"172.21.15.178:6810/29234","state":["exists","up"]},{"osd":3,"uuid":"a5bf3f7d-4c59-470a-8f69-3fea018c13f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":28752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":28752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":28752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":28752}]},"public_addr":"172.21.15.204:6801/28752","cluster_addr":"172.21.15.204:6802/28752","heartbeat_back_addr":"172.21.15.204:6806/28752","heartbeat_front_addr":"172.21.15.204:6804/28752","state":["exists","up"]},{"osd":4,"uuid":"cdaf5723-3a5c-4238-a8e6-38f2325ccbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":28753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":28753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":28753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6805","nonce":28753}]},"public_addr":"172.21.15.204:6800/28753","cluster_addr":"172.21.15.204:6803/28753","heartbeat_back_addr":"172.21.15.204:6807/28753","heartbeat_front_addr":"172.21.15.204:6805/28753","state":["exists","up"]},{"osd":5,"uuid":"6b67035f-8718-4762-98bf-0dd0aa86e6e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":28754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":28754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":28754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":28754}]},"public_addr":"172.21.15.204:6808/28754","cluster_addr":"172.21.15.204:6809/28754","heartbeat_back_addr":"172.21.15.204:6811/28754","heartbeat_front_addr":"172.21.15.204:6810/28754","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["autoout","exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["autoout","exists","new"]},{"osd":8,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["autoout","exists","new"]},{"osd":9,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["autoout","exists","new"]},{"osd":10,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"e5de064c-679b-416c-b9b7-f61aa58c84ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.093711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.117496+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.123704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.082038+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.088260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.090342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":65536,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":65536,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":65536,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":65536,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":0,"to":3}]},{"pgid":"9.3","mappings":[{"from":0,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"ec-foo-ssd":{"crush-device-class":"ssd","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:50:02.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 1 -- 172.21.15.178:0/207896582 >> v2:172.21.15.204:6812/28210 conn(0x7fb62c0397b0 msgr2=0x7fb62c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 1 --2- 172.21.15.178:0/207896582 >> v2:172.21.15.204:6812/28210 conn(0x7fb62c0397b0 0x7fb62c03bc60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb63c006c90 tx=0x7fb63c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 10 monclient: shutdown 2024-09-15T01:50:02.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:02.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 1 -- 172.21.15.178:0/207896582 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c09f000 msgr2=0x7fb64c097d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 1 --2- 172.21.15.178:0/207896582 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c09f000 0x7fb64c097d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb644009e90 tx=0x7fb644037000 comp rx=0 tx=0).stop 2024-09-15T01:50:02.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 1 -- 172.21.15.178:0/207896582 shutdown_connections 2024-09-15T01:50:02.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 1 --2- 172.21.15.178:0/207896582 >> v2:172.21.15.204:6812/28210 conn(0x7fb62c0397b0 0x7fb62c03bc60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 1 --2- 172.21.15.178:0/207896582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb64c09d580 0x7fb64c097800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 1 --2- 172.21.15.178:0/207896582 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb64c09f000 0x7fb64c097d40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 1 --2- 172.21.15.178:0/207896582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb64c067130 0x7fb64c0972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 1 -- 172.21.15.178:0/207896582 >> 172.21.15.178:0/207896582 conn(0x7fb64c068a60 msgr2=0x7fb64c12dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:02.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 1 -- 172.21.15.178:0/207896582 shutdown_connections 2024-09-15T01:50:02.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.080+0000 7fb650ce8640 1 -- 172.21.15.178:0/207896582 wait complete. 2024-09-15T01:50:02.093 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T01:50:02.094 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:50:02.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.183+0000 7f33d7f5d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:02.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.183+0000 7f33d7f5d640 10 monclient: build_initial_monmap 2024-09-15T01:50:02.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.183+0000 7f33d7f5d640 10 monclient: monmap: 2024-09-15T01:50:02.186 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:02.186 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.186 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:02.184051+0000 2024-09-15T01:50:02.186 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:02.184051+0000 2024-09-15T01:50:02.186 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:02.186 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.186 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:02.186 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:02.186 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:02.186 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:02.186 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:02.187 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:02.187 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:02.187 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:02.187 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:02.187 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:02.187 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:02.187 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.183+0000 7f33d7f5d640 1 Processor -- start 2024-09-15T01:50:02.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.183+0000 7f33d7f5d640 1 -- start start 2024-09-15T01:50:02.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.183+0000 7f33d7f5d640 10 monclient: init 2024-09-15T01:50:02.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:02.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 10 monclient: _add_conns ranks=[5,1,3,0,8,9,4,7,2,6,10] 2024-09-15T01:50:02.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33d00a2930 0x7f33d0132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 10 monclient(hunting): picked mon.noname-h con 0x7f33d00a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:02.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f33d009faa0 addr v1:172.21.15.204:6789/0 2024-09-15T01:50:02.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33d009fe70 0x7f33d00a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 10 monclient(hunting): picked mon.noname-g con 0x7f33d009fe70 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:02.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d004bf60 con 0x7f33d009faa0 2024-09-15T01:50:02.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d009dd80 con 0x7f33d009fe70 2024-09-15T01:50:02.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33d00a2d00 con 0x7f33d00a2930 2024-09-15T01:50:02.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:02.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d7f5d640 10 monclient(hunting): authenticate will time out at 1806.934519s 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d54f8640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f33d009faa0 0x7f33d00a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.178:54392/0 (socket says 172.21.15.178:54392) 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d54f8640 1 -- 172.21.15.178:0/2386980388 learned_addr learned my addr 172.21.15.178:0/2386980388 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d5cf9640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33d00a2930 0x7f33d0132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d64fa640 1 --2- 172.21.15.178:0/2386980388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33d009fe70 0x7f33d00a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d5cf9640 10 monclient(hunting): get_auth_request con 0x7f33d00a2930 auth_method 0 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d5cf9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d5cf9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d5cf9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d64fa640 10 monclient(hunting): get_auth_request con 0x7f33d009fe70 auth_method 0 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d64fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d64fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d64fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d5cf9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d5cf9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d5cf9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d64fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d64fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.184+0000 7f33d64fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.185+0000 7f33d5cf9640 10 monclient(hunting): handle_auth_done global_id 7889 payload 274 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.185+0000 7f33d5cf9640 1 -- 172.21.15.178:0/2386980388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33d009fe70 msgr2=0x7f33d00a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.185+0000 7f33d5cf9640 1 --2- 172.21.15.178:0/2386980388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33d009fe70 0x7f33d00a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.185+0000 7f33d5cf9640 1 -- 172.21.15.178:0/2386980388 >> v1:172.21.15.204:6789/0 conn(0x7f33d009faa0 legacy=0x7f33d00a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.185+0000 7f33d5cf9640 10 monclient: _finish_hunting 0 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.185+0000 7f33d5cf9640 1 monclient: found mon.noname-h 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.185+0000 7f33d4cf7640 1 -- 172.21.15.178:0/2386980388 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483462703 0 0) 0x7f33d004bf60 con 0x7f33d009faa0 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.185+0000 7f33d5cf9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:02.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.185+0000 7f33d5cf9640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.185+0000 7f33d5cf9640 1 -- 172.21.15.178:0/2386980388 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d00a0310 con 0x7f33d00a2930 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.185+0000 7f33d4cf7640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.185+0000 7f33d5cf9640 1 --2- 172.21.15.178:0/2386980388 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33d00a2930 0x7f33d0132de0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009880 tx=0x7f33cc0090f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=7a150a130e3e67f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d4cf7640 1 -- 172.21.15.178:0/2386980388 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33cc02ac50 con 0x7f33d00a2930 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d4cf7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d4cf7640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d4cf7640 10 monclient: dump: 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:02.199 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d4cf7640 10 monclient: _finish_auth 0 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d4cf7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:02.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d4cf7640 1 -- 172.21.15.178:0/2386980388 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc02add0 con 0x7f33d00a2930 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 5 monclient: authenticate success, global_id 7889 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d4cf7640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d4cf7640 1 -- 172.21.15.178:0/2386980388 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33cc031ca0 con 0x7f33d00a2930 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 4 set_mon_vals no callback set 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d4cf7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d4cf7640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d4cf7640 10 monclient: dump: 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.201 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 10 monclient: shutdown 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 1 -- 172.21.15.178:0/2386980388 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33d00a2930 msgr2=0x7f33d0132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 1 --2- 172.21.15.178:0/2386980388 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33d00a2930 0x7f33d0132de0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009880 tx=0x7f33cc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 1 -- 172.21.15.178:0/2386980388 shutdown_connections 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 1 --2- 172.21.15.178:0/2386980388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33d009fe70 0x7f33d00a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 1 --2- 172.21.15.178:0/2386980388 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33d00a2930 0x7f33d0132de0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 1 -- 172.21.15.178:0/2386980388 >> 172.21.15.178:0/2386980388 conn(0x7f33d00a5ed0 msgr2=0x7f33d00a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.186+0000 7f33d7f5d640 1 -- 172.21.15.178:0/2386980388 shutdown_connections 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.187+0000 7f33d7f5d640 1 -- 172.21.15.178:0/2386980388 wait complete. 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.187+0000 7f33d7f5d640 10 monclient: build_initial_monmap 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.187+0000 7f33d7f5d640 10 monclient: monmap: 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:02.203 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.187+0000 7f33d7f5d640 1 Processor -- start 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.187+0000 7f33d7f5d640 1 -- start start 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.187+0000 7f33d7f5d640 10 monclient: init 2024-09-15T01:50:02.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 10 monclient: _add_conns ranks=[6,0,8,2,5,7,1,4,3] 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f33d009faa0 0x7f33d009afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 10 monclient(hunting): picked mon.noname-d con 0x7f33d009faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33d009fe70 0x7f33d0131730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 10 monclient(hunting): picked mon.noname-a con 0x7f33d009fe70 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33d00a2930 0x7f33d011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 10 monclient(hunting): picked mon.noname-e con 0x7f33d00a2930 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d0142aa0 con 0x7f33d009fe70 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f33d00a2390 con 0x7f33d009faa0 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f33d009d630 con 0x7f33d00a2930 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d7f5d640 10 monclient(hunting): authenticate will time out at 1806.938327s 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d5cf9640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f33d009faa0 0x7f33d009afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33d009fe70 0x7f33d0131730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d5cf9640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f33d009faa0 0x7f33d009afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:35472/0 (socket says 172.21.15.178:35472) 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d5cf9640 1 -- 172.21.15.178:0/3514904132 learned_addr learned my addr 172.21.15.178:0/3514904132 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d5cf9640 10 monclient(hunting): get_auth_request con 0x7f33d009faa0 auth_method 0 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d5cf9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d5cf9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d64fa640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33d00a2930 0x7f33d011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d5cf9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 10 monclient(hunting): get_auth_request con 0x7f33d009fe70 auth_method 0 2024-09-15T01:50:02.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d64fa640 10 monclient(hunting): get_auth_request con 0x7f33d00a2930 auth_method 0 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d64fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d64fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d64fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d5cf9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d5cf9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d5cf9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d64fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d64fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d64fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 10 monclient(hunting): handle_auth_done global_id 104184 payload 824 2024-09-15T01:50:02.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 1 -- 172.21.15.178:0/3514904132 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33d00a2930 msgr2=0x7f33d011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 1 --2- 172.21.15.178:0/3514904132 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33d00a2930 0x7f33d011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 1 -- 172.21.15.178:0/3514904132 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f33d009faa0 msgr2=0x7f33d009afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 1 --2- 172.21.15.178:0/3514904132 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f33d009faa0 0x7f33d009afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 10 monclient: _finish_hunting 0 2024-09-15T01:50:02.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 1 monclient: found mon.noname-a 2024-09-15T01:50:02.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:02.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:50:02.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 1 -- 172.21.15.178:0/3514904132 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc009390 con 0x7f33d009fe70 2024-09-15T01:50:02.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d5cf9640 1 --2- 172.21.15.178:0/3514904132 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f33d009faa0 0x7f33d009afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:02.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.188+0000 7f33d54f8640 1 --2- 172.21.15.178:0/3514904132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33d009fe70 0x7f33d0131730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f33d00a6ec0 tx=0x7f33c0007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:02.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 1 -- 172.21.15.178:0/3514904132 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33c0000fe0 con 0x7f33d009fe70 2024-09-15T01:50:02.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:02.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:50:02.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 10 monclient: dump: 2024-09-15T01:50:02.209 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:02.209 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.209 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.209 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33d7f5d640 5 monclient: authenticate success, global_id 104184 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 1 -- 172.21.15.178:0/3514904132 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c00070d0 con 0x7f33d009fe70 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 1 -- 172.21.15.178:0/3514904132 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33c0007740 con 0x7f33d009fe70 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33d7f5d640 10 monclient: _renew_subs 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33d7f5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33d7f5d640 1 -- 172.21.15.178:0/3514904132 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d009bb80 con 0x7f33d009fe70 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c7fff640 4 set_mon_vals no callback set 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 10 monclient: dump: 2024-09-15T01:50:02.211 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:02.212 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33d7f5d640 10 monclient: _renew_subs 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33d7f5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33d7f5d640 1 -- 172.21.15.178:0/3514904132 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d011eb30 con 0x7f33d009fe70 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.189+0000 7f33c5ffb640 1 -- 172.21.15.178:0/3514904132 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f33c00072c0 con 0x7f33d009fe70 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.190+0000 7f33c5ffb640 1 --2- 172.21.15.178:0/3514904132 >> v2:172.21.15.204:6812/28210 conn(0x7f33b40397b0 0x7f33b403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.190+0000 7f33c5ffb640 1 -- 172.21.15.178:0/3514904132 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f33c004aba0 con 0x7f33d009fe70 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.190+0000 7f33d7f5d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.190+0000 7f33d7f5d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.190+0000 7f33d7f5d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.190+0000 7f33d7f5d640 1 -- 172.21.15.178:0/3514904132 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d004b720 con 0x7f33d009fe70 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.190+0000 7f33d5cf9640 1 --2- 172.21.15.178:0/3514904132 >> v2:172.21.15.204:6812/28210 conn(0x7f33b40397b0 0x7f33b403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.190+0000 7f33d5cf9640 10 monclient: get_auth_request con 0x7f33b40397b0 auth_method 0 2024-09-15T01:50:02.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.190+0000 7f33d5cf9640 1 --2- 172.21.15.178:0/3514904132 >> v2:172.21.15.204:6812/28210 conn(0x7f33b40397b0 0x7f33b403bc60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f33cc000c00 tx=0x7f33cc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:02.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.194+0000 7f33c5ffb640 1 -- 172.21.15.178:0/3514904132 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f33c0017080 con 0x7f33d009fe70 2024-09-15T01:50:02.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.194+0000 7f33c5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:02.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.194+0000 7f33c5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:02.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.346+0000 7f33d7f5d640 1 -- 172.21.15.178:0/3514904132 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33d004c4a0 con 0x7f33b40397b0 2024-09-15T01:50:02.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.350+0000 7f33c5ffb640 1 -- 172.21.15.178:0/3514904132 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+93405 (secure 0 0 0) 0x7f33d004c4a0 con 0x7f33b40397b0 2024-09-15T01:50:02.352 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:50:02.352 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-09-15T01:50:02.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.351+0000 7f33d7f5d640 1 -- 172.21.15.178:0/3514904132 >> v2:172.21.15.204:6812/28210 conn(0x7f33b40397b0 msgr2=0x7f33b403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.351+0000 7f33d7f5d640 1 --2- 172.21.15.178:0/3514904132 >> v2:172.21.15.204:6812/28210 conn(0x7f33b40397b0 0x7f33b403bc60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f33cc000c00 tx=0x7f33cc032000 comp rx=0 tx=0).stop 2024-09-15T01:50:02.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.352+0000 7f33d7f5d640 10 monclient: shutdown 2024-09-15T01:50:02.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.352+0000 7f33d7f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:02.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.352+0000 7f33d7f5d640 1 -- 172.21.15.178:0/3514904132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33d009fe70 msgr2=0x7f33d0131730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.352+0000 7f33d7f5d640 1 --2- 172.21.15.178:0/3514904132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33d009fe70 0x7f33d0131730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f33d00a6ec0 tx=0x7f33c0007ea0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.352+0000 7f33d7f5d640 1 -- 172.21.15.178:0/3514904132 shutdown_connections 2024-09-15T01:50:02.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.352+0000 7f33d7f5d640 1 --2- 172.21.15.178:0/3514904132 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33d00a2930 0x7f33d011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.352+0000 7f33d7f5d640 1 --2- 172.21.15.178:0/3514904132 >> v2:172.21.15.204:6812/28210 conn(0x7f33b40397b0 0x7f33b403bc60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.352+0000 7f33d7f5d640 1 --2- 172.21.15.178:0/3514904132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33d009fe70 0x7f33d0131730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.352+0000 7f33d7f5d640 1 --2- 172.21.15.178:0/3514904132 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f33d009faa0 0x7f33d009afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.352+0000 7f33d7f5d640 1 -- 172.21.15.178:0/3514904132 >> 172.21.15.178:0/3514904132 conn(0x7f33d00a5ed0 msgr2=0x7f33d009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:02.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.352+0000 7f33d7f5d640 1 -- 172.21.15.178:0/3514904132 shutdown_connections 2024-09-15T01:50:02.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.352+0000 7f33d7f5d640 1 -- 172.21.15.178:0/3514904132 wait complete. 2024-09-15T01:50:02.363 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":800,"stamp":"2024-09-15T01:50:00.385960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562077696,"kb_used":4795144,"kb_used_data":557282552,"kb_used_omap":18904,"kb_used_meta":0,"kb_avail":557282552,"statfs":{"total":575567560704,"available":570657333248,"internally_reserved":0,"allocated":570657333248,"data_stored":570657333248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19358691,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002720"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288825+0000","last_change":"2024-09-15T01:36:25.227464+0000","last_active":"2024-09-15T01:44:24.288825+0000","last_peered":"2024-09-15T01:44:24.288825+0000","last_clean":"2024-09-15T01:44:24.288825+0000","last_became_active":"2024-09-15T01:36:25.227161+0000","last_became_peered":"2024-09-15T01:36:25.227161+0000","last_unstale":"2024-09-15T01:44:24.288825+0000","last_undegraded":"2024-09-15T01:44:24.288825+0000","last_fullsized":"2024-09-15T01:44:24.288825+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:44:14.933447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":97,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161809+0000","last_change":"2024-09-15T01:36:25.228477+0000","last_active":"2024-09-15T01:44:24.161809+0000","last_peered":"2024-09-15T01:44:24.161809+0000","last_clean":"2024-09-15T01:44:24.161809+0000","last_became_active":"2024-09-15T01:36:25.228216+0000","last_became_peered":"2024-09-15T01:36:25.228216+0000","last_unstale":"2024-09-15T01:44:24.161809+0000","last_undegraded":"2024-09-15T01:44:24.161809+0000","last_fullsized":"2024-09-15T01:44:24.161809+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:33:26.644405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288874+0000","last_change":"2024-09-15T01:36:25.228316+0000","last_active":"2024-09-15T01:44:24.288874+0000","last_peered":"2024-09-15T01:44:24.288874+0000","last_clean":"2024-09-15T01:44:24.288874+0000","last_became_active":"2024-09-15T01:36:25.228024+0000","last_became_peered":"2024-09-15T01:36:25.228024+0000","last_unstale":"2024-09-15T01:44:24.288874+0000","last_undegraded":"2024-09-15T01:44:24.288874+0000","last_fullsized":"2024-09-15T01:44:24.288874+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:35.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":97,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161781+0000","last_change":"2024-09-15T01:36:25.230047+0000","last_active":"2024-09-15T01:44:24.161781+0000","last_peered":"2024-09-15T01:44:24.161781+0000","last_clean":"2024-09-15T01:44:24.161781+0000","last_became_active":"2024-09-15T01:36:25.229786+0000","last_became_peered":"2024-09-15T01:36:25.229786+0000","last_unstale":"2024-09-15T01:44:24.161781+0000","last_undegraded":"2024-09-15T01:44:24.161781+0000","last_fullsized":"2024-09-15T01:44:24.161781+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:15:18.249275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":92,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160052+0000","last_change":"2024-09-15T01:36:25.233254+0000","last_active":"2024-09-15T01:44:24.160052+0000","last_peered":"2024-09-15T01:44:24.160052+0000","last_clean":"2024-09-15T01:44:24.160052+0000","last_became_active":"2024-09-15T01:36:25.232778+0000","last_became_peered":"2024-09-15T01:36:25.232778+0000","last_unstale":"2024-09-15T01:44:24.160052+0000","last_undegraded":"2024-09-15T01:44:24.160052+0000","last_fullsized":"2024-09-15T01:44:24.160052+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:35:24.397702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.289191+0000","last_change":"2024-09-15T01:36:25.227383+0000","last_active":"2024-09-15T01:44:24.289191+0000","last_peered":"2024-09-15T01:44:24.289191+0000","last_clean":"2024-09-15T01:44:24.289191+0000","last_became_active":"2024-09-15T01:36:25.227142+0000","last_became_peered":"2024-09-15T01:36:25.227142+0000","last_unstale":"2024-09-15T01:44:24.289191+0000","last_undegraded":"2024-09-15T01:44:24.289191+0000","last_fullsized":"2024-09-15T01:44:24.289191+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:22:59.700920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":170,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159692+0000","last_change":"2024-09-15T01:36:25.231017+0000","last_active":"2024-09-15T01:44:24.159692+0000","last_peered":"2024-09-15T01:44:24.159692+0000","last_clean":"2024-09-15T01:44:24.159692+0000","last_became_active":"2024-09-15T01:36:25.230723+0000","last_became_peered":"2024-09-15T01:36:25.230723+0000","last_unstale":"2024-09-15T01:44:24.159692+0000","last_undegraded":"2024-09-15T01:44:24.159692+0000","last_fullsized":"2024-09-15T01:44:24.159692+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:39.782818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160643+0000","last_change":"2024-09-15T01:36:25.228490+0000","last_active":"2024-09-15T01:44:24.160643+0000","last_peered":"2024-09-15T01:44:24.160643+0000","last_clean":"2024-09-15T01:44:24.160643+0000","last_became_active":"2024-09-15T01:36:25.228252+0000","last_became_peered":"2024-09-15T01:36:25.228252+0000","last_unstale":"2024-09-15T01:44:24.160643+0000","last_undegraded":"2024-09-15T01:44:24.160643+0000","last_fullsized":"2024-09-15T01:44:24.160643+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:56:14.136629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":98,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161798+0000","last_change":"2024-09-15T01:36:25.227553+0000","last_active":"2024-09-15T01:44:24.161798+0000","last_peered":"2024-09-15T01:44:24.161798+0000","last_clean":"2024-09-15T01:44:24.161798+0000","last_became_active":"2024-09-15T01:36:25.226915+0000","last_became_peered":"2024-09-15T01:36:25.226915+0000","last_unstale":"2024-09-15T01:44:24.161798+0000","last_undegraded":"2024-09-15T01:44:24.161798+0000","last_fullsized":"2024-09-15T01:44:24.161798+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:18:34.776247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":118,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159771+0000","last_change":"2024-09-15T01:36:25.229923+0000","last_active":"2024-09-15T01:44:24.159771+0000","last_peered":"2024-09-15T01:44:24.159771+0000","last_clean":"2024-09-15T01:44:24.159771+0000","last_became_active":"2024-09-15T01:36:25.229665+0000","last_became_peered":"2024-09-15T01:36:25.229665+0000","last_unstale":"2024-09-15T01:44:24.159771+0000","last_undegraded":"2024-09-15T01:44:24.159771+0000","last_fullsized":"2024-09-15T01:44:24.159771+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:55:30.051647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":176,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159528+0000","last_change":"2024-09-15T01:36:25.227907+0000","last_active":"2024-09-15T01:44:24.159528+0000","last_peered":"2024-09-15T01:44:24.159528+0000","last_clean":"2024-09-15T01:44:24.159528+0000","last_became_active":"2024-09-15T01:36:25.227397+0000","last_became_peered":"2024-09-15T01:36:25.227397+0000","last_unstale":"2024-09-15T01:44:24.159528+0000","last_undegraded":"2024-09-15T01:44:24.159528+0000","last_fullsized":"2024-09-15T01:44:24.159528+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:52.384874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160501+0000","last_change":"2024-09-15T01:36:25.227223+0000","last_active":"2024-09-15T01:44:24.160501+0000","last_peered":"2024-09-15T01:44:24.160501+0000","last_clean":"2024-09-15T01:44:24.160501+0000","last_became_active":"2024-09-15T01:36:25.226969+0000","last_became_peered":"2024-09-15T01:36:25.226969+0000","last_unstale":"2024-09-15T01:44:24.160501+0000","last_undegraded":"2024-09-15T01:44:24.160501+0000","last_fullsized":"2024-09-15T01:44:24.160501+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:11:23.636488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.289013+0000","last_change":"2024-09-15T01:36:26.228988+0000","last_active":"2024-09-15T01:44:24.289013+0000","last_peered":"2024-09-15T01:44:24.289013+0000","last_clean":"2024-09-15T01:44:24.289013+0000","last_became_active":"2024-09-15T01:36:26.228683+0000","last_became_peered":"2024-09-15T01:36:26.228683+0000","last_unstale":"2024-09-15T01:44:24.289013+0000","last_undegraded":"2024-09-15T01:44:24.289013+0000","last_fullsized":"2024-09-15T01:44:24.289013+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:42.069204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":87,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160502+0000","last_change":"2024-09-15T01:36:26.228676+0000","last_active":"2024-09-15T01:44:24.160502+0000","last_peered":"2024-09-15T01:44:24.160502+0000","last_clean":"2024-09-15T01:44:24.160502+0000","last_became_active":"2024-09-15T01:36:26.228429+0000","last_became_peered":"2024-09-15T01:36:26.228429+0000","last_unstale":"2024-09-15T01:44:24.160502+0000","last_undegraded":"2024-09-15T01:44:24.160502+0000","last_fullsized":"2024-09-15T01:44:24.160502+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:41:46.249246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":106,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288775+0000","last_change":"2024-09-15T01:36:26.226092+0000","last_active":"2024-09-15T01:44:24.288775+0000","last_peered":"2024-09-15T01:44:24.288775+0000","last_clean":"2024-09-15T01:44:24.288775+0000","last_became_active":"2024-09-15T01:36:26.225738+0000","last_became_peered":"2024-09-15T01:36:26.225738+0000","last_unstale":"2024-09-15T01:44:24.288775+0000","last_undegraded":"2024-09-15T01:44:24.288775+0000","last_fullsized":"2024-09-15T01:44:24.288775+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:49:14.676091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288958+0000","last_change":"2024-09-15T01:36:25.225943+0000","last_active":"2024-09-15T01:44:24.288958+0000","last_peered":"2024-09-15T01:44:24.288958+0000","last_clean":"2024-09-15T01:44:24.288958+0000","last_became_active":"2024-09-15T01:36:25.225608+0000","last_became_peered":"2024-09-15T01:36:25.225608+0000","last_unstale":"2024-09-15T01:44:24.288958+0000","last_undegraded":"2024-09-15T01:44:24.288958+0000","last_fullsized":"2024-09-15T01:44:24.288958+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:16:43.688463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":100,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159580+0000","last_change":"2024-09-15T01:36:25.260030+0000","last_active":"2024-09-15T01:44:24.159580+0000","last_peered":"2024-09-15T01:44:24.159580+0000","last_clean":"2024-09-15T01:44:24.159580+0000","last_became_active":"2024-09-15T01:36:25.233270+0000","last_became_peered":"2024-09-15T01:36:25.233270+0000","last_unstale":"2024-09-15T01:44:24.159580+0000","last_undegraded":"2024-09-15T01:44:24.159580+0000","last_fullsized":"2024-09-15T01:44:24.159580+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_clean_scrub_stamp":"2024-09-15T01:31:59.015115+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:28:21.577888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":258,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159895+0000","last_change":"2024-09-15T01:36:25.233515+0000","last_active":"2024-09-15T01:44:24.159895+0000","last_peered":"2024-09-15T01:44:24.159895+0000","last_clean":"2024-09-15T01:44:24.159895+0000","last_became_active":"2024-09-15T01:36:25.230032+0000","last_became_peered":"2024-09-15T01:36:25.230032+0000","last_unstale":"2024-09-15T01:44:24.159895+0000","last_undegraded":"2024-09-15T01:44:24.159895+0000","last_fullsized":"2024-09-15T01:44:24.159895+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:44:35.523507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076976900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161688+0000","last_change":"2024-09-15T01:36:25.227415+0000","last_active":"2024-09-15T01:44:24.161688+0000","last_peered":"2024-09-15T01:44:24.161688+0000","last_clean":"2024-09-15T01:44:24.161688+0000","last_became_active":"2024-09-15T01:36:25.226499+0000","last_became_peered":"2024-09-15T01:36:25.226499+0000","last_unstale":"2024-09-15T01:44:24.161688+0000","last_undegraded":"2024-09-15T01:44:24.161688+0000","last_fullsized":"2024-09-15T01:44:24.161688+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:35:59.548191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037511000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160610+0000","last_change":"2024-09-15T01:36:25.231459+0000","last_active":"2024-09-15T01:44:24.160610+0000","last_peered":"2024-09-15T01:44:24.160610+0000","last_clean":"2024-09-15T01:44:24.160610+0000","last_became_active":"2024-09-15T01:36:25.231212+0000","last_became_peered":"2024-09-15T01:36:25.231212+0000","last_unstale":"2024-09-15T01:44:24.160610+0000","last_undegraded":"2024-09-15T01:44:24.160610+0000","last_fullsized":"2024-09-15T01:44:24.160610+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:16:13.072580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046258900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":296,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160823+0000","last_change":"2024-09-15T01:36:25.236741+0000","last_active":"2024-09-15T01:44:24.160823+0000","last_peered":"2024-09-15T01:44:24.160823+0000","last_clean":"2024-09-15T01:44:24.160823+0000","last_became_active":"2024-09-15T01:36:25.226718+0000","last_became_peered":"2024-09-15T01:36:25.226718+0000","last_unstale":"2024-09-15T01:44:24.160823+0000","last_undegraded":"2024-09-15T01:44:24.160823+0000","last_fullsized":"2024-09-15T01:44:24.160823+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:07:25.773736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093022000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.289138+0000","last_change":"2024-09-15T01:36:25.227607+0000","last_active":"2024-09-15T01:44:24.289138+0000","last_peered":"2024-09-15T01:44:24.289138+0000","last_clean":"2024-09-15T01:44:24.289138+0000","last_became_active":"2024-09-15T01:36:25.227361+0000","last_became_peered":"2024-09-15T01:36:25.227361+0000","last_unstale":"2024-09-15T01:44:24.289138+0000","last_undegraded":"2024-09-15T01:44:24.289138+0000","last_fullsized":"2024-09-15T01:44:24.289138+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:15:42.847085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031034999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159956+0000","last_change":"2024-09-15T01:36:25.230157+0000","last_active":"2024-09-15T01:44:24.159956+0000","last_peered":"2024-09-15T01:44:24.159956+0000","last_clean":"2024-09-15T01:44:24.159956+0000","last_became_active":"2024-09-15T01:36:25.229674+0000","last_became_peered":"2024-09-15T01:36:25.229674+0000","last_unstale":"2024-09-15T01:44:24.159956+0000","last_undegraded":"2024-09-15T01:44:24.159956+0000","last_fullsized":"2024-09-15T01:44:24.159956+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:12:26.442496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077134000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288724+0000","last_change":"2024-09-15T01:36:25.225614+0000","last_active":"2024-09-15T01:44:24.288724+0000","last_peered":"2024-09-15T01:44:24.288724+0000","last_clean":"2024-09-15T01:44:24.288724+0000","last_became_active":"2024-09-15T01:36:25.225361+0000","last_became_peered":"2024-09-15T01:36:25.225361+0000","last_unstale":"2024-09-15T01:44:24.288724+0000","last_undegraded":"2024-09-15T01:44:24.288724+0000","last_fullsized":"2024-09-15T01:44:24.288724+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:25:49.332027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041942500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":245,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160207+0000","last_change":"2024-09-15T01:36:26.437860+0000","last_active":"2024-09-15T01:44:24.160207+0000","last_peered":"2024-09-15T01:44:24.160207+0000","last_clean":"2024-09-15T01:44:24.160207+0000","last_became_active":"2024-09-15T01:36:26.437594+0000","last_became_peered":"2024-09-15T01:36:26.437594+0000","last_unstale":"2024-09-15T01:44:24.160207+0000","last_undegraded":"2024-09-15T01:44:24.160207+0000","last_fullsized":"2024-09-15T01:44:24.160207+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:50:29.143741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065783200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640474,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799576,"kb_used_data":92880040,"kb_used_omap":3214,"kb_used_meta":0,"kb_avail":92880040,"statfs":{"total":95927926784,"available":95109160960,"internally_reserved":0,"allocated":95109160960,"data_stored":95109160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3291775,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.39500000000000002,"5min":0.28999999999999998,"15min":0.24299999999999999},"max":{"1min":0.65500000000000003,"5min":0.73499999999999999,"15min":0.95099999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59699999999999998,"5min":0.68100000000000005,"15min":0.76700000000000002},"last":0.48499999999999999}]},{"osd":1,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.28899999999999998,"15min":0.28299999999999997},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.91800000000000004},"last":0.496},{"interface":"front","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.30499999999999999},"max":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.72899999999999998},"last":0.51600000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.23300000000000001},"max":{"1min":0.629,"5min":0.69199999999999995,"15min":0.73399999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.50600000000000001},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.27700000000000002},"max":{"1min":0.629,"5min":0.67200000000000004,"15min":0.81000000000000005},"last":0.52900000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.27200000000000002,"5min":0.224,"15min":0.20699999999999999},"max":{"1min":0.52600000000000002,"5min":0.63400000000000001,"15min":0.96799999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.39700000000000002,"15min":0.38600000000000001},"min":{"1min":0.28199999999999997,"5min":0.23200000000000001,"15min":0.216},"max":{"1min":0.54700000000000004,"5min":0.64700000000000002,"15min":0.86199999999999999},"last":0.42499999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.378,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":1.0289999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40699999999999997,"15min":0.40300000000000002},"min":{"1min":0.26400000000000001,"5min":0.249,"15min":0.22},"max":{"1min":0.54300000000000004,"5min":0.61799999999999999,"15min":0.67200000000000004},"last":0.372}]}]},{"osd":4,"up_from":11,"seq":47244640474,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799000,"kb_used_data":92880616,"kb_used_omap":3258,"kb_used_meta":0,"kb_avail":92880616,"statfs":{"total":95927926784,"available":95109750784,"internally_reserved":0,"allocated":95109750784,"data_stored":95109750784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3336786,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.48999999999999999},"min":{"1min":0.39800000000000002,"5min":0.33500000000000002,"15min":0.219},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.39700000000000002,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.66600000000000004,"5min":0.69399999999999995,"15min":0.70799999999999996},"last":0.54200000000000004}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53600000000000003,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.25600000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.52800000000000002},"min":{"1min":0.47599999999999998,"5min":0.36499999999999999,"15min":0.28999999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.52500000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.71599999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69199999999999995,"5min":0.876,"15min":0.876},"last":0.502}]},{"osd":3,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.36699999999999999},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.22700000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.40699999999999997,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.216},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.3540000000000001},"last":0.35799999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.39600000000000002,"15min":0.376},"min":{"1min":0.27900000000000003,"5min":0.26600000000000001,"15min":0.22700000000000001},"max":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":1.2709999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.39600000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.24199999999999999},"max":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":1.2250000000000001},"last":0.377}]}]},{"osd":3,"up_from":11,"seq":47244640474,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798996,"kb_used_data":92880620,"kb_used_omap":3033,"kb_used_meta":0,"kb_avail":92880620,"statfs":{"total":95927926784,"available":95109754880,"internally_reserved":0,"allocated":95109754880,"data_stored":95109754880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3106015,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.24299999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.69399999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.29899999999999999},"max":{"1min":0.64900000000000002,"5min":0.66400000000000003,"15min":0.81599999999999995},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.27900000000000003},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.79300000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.37,"5min":0.34399999999999997,"15min":0.29899999999999999},"max":{"1min":0.58799999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.61799999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.74299999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.501,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.30099999999999999},"max":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.75800000000000001},"last":0.58799999999999997}]},{"osd":4,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36099999999999999,"15min":0.35999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.46200000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.61699999999999999,"5min":0.65900000000000003,"15min":0.70599999999999996},"last":0.435}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.53200000000000003,"5min":0.57099999999999995,"15min":0.65500000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.39000000000000001,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.49299999999999999,"5min":0.64600000000000002,"15min":0.70399999999999996},"last":0.47199999999999998}]}]},{"osd":2,"up_from":11,"seq":47244640474,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798996,"kb_used_data":92880620,"kb_used_omap":3051,"kb_used_meta":0,"kb_avail":92880620,"statfs":{"total":95927926784,"available":95109754880,"internally_reserved":0,"allocated":95109754880,"data_stored":95109754880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3125031,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.32600000000000001},"min":{"1min":0.23200000000000001,"5min":0.20100000000000001,"15min":0.17499999999999999},"max":{"1min":0.47999999999999998,"5min":0.48499999999999999,"15min":0.53000000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.375,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.23999999999999999,"15min":0.20200000000000001},"max":{"1min":0.501,"5min":0.53000000000000003,"15min":0.59999999999999998},"last":0.39400000000000002}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.33200000000000002,"15min":0.33500000000000002},"min":{"1min":0.20300000000000001,"5min":0.187,"15min":0.182},"max":{"1min":0.46100000000000002,"5min":0.52500000000000002,"15min":0.60599999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.373,"15min":0.373},"min":{"1min":0.24299999999999999,"5min":0.222,"15min":0.20699999999999999},"max":{"1min":0.53300000000000003,"5min":0.57099999999999995,"15min":0.67400000000000004},"last":0.44}]},{"osd":3,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.28899999999999998},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.78900000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61299999999999999,"5min":0.77900000000000003,"15min":1.0129999999999999},"last":0.497}]},{"osd":4,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.55400000000000005,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":0.63500000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.27600000000000002},"max":{"1min":0.59699999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53200000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.27800000000000002},"max":{"1min":0.67300000000000004,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.30099999999999999},"max":{"1min":0.623,"5min":0.75900000000000001,"15min":0.87},"last":0.46500000000000002}]}]},{"osd":1,"up_from":11,"seq":47244640475,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799576,"kb_used_data":92880040,"kb_used_omap":3055,"kb_used_meta":0,"kb_avail":92880040,"statfs":{"total":95927926784,"available":95109160960,"internally_reserved":0,"allocated":95109160960,"data_stored":95109160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3129006,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33700000000000002,"15min":0.33300000000000002},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.188},"max":{"1min":0.47399999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.248},{"interface":"front","average":{"1min":0.376,"5min":0.35499999999999998,"15min":0.35299999999999998},"min":{"1min":0.27800000000000002,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.68200000000000005},"last":0.311}]},{"osd":2,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.37},"min":{"1min":0.29799999999999999,"5min":0.245,"15min":0.23799999999999999},"max":{"1min":0.501,"5min":0.51400000000000001,"15min":0.73399999999999999},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28899999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.70299999999999996},"last":0.26400000000000001}]},{"osd":3,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.46999999999999997,"5min":0.34499999999999997,"15min":0.30099999999999999},"max":{"1min":0.68700000000000006,"5min":0.72099999999999997,"15min":0.76800000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.46800000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.76700000000000002},"last":0.48599999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.49099999999999999,"5min":0.40500000000000003,"15min":0.35599999999999998},"max":{"1min":0.70599999999999996,"5min":0.77700000000000002,"15min":0.80600000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.308},"max":{"1min":0.71699999999999997,"5min":0.76800000000000002,"15min":0.81899999999999995},"last":0.57199999999999995}]},{"osd":5,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.32600000000000001},"max":{"1min":0.69099999999999995,"5min":0.748,"15min":0.89400000000000002},"last":0.496},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.53800000000000003},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.32200000000000001},"max":{"1min":0.64400000000000002,"5min":0.69699999999999995,"15min":0.81000000000000005},"last":0.52100000000000002}]}]},{"osd":0,"up_from":11,"seq":47244640474,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799000,"kb_used_data":92880616,"kb_used_omap":3291,"kb_used_meta":0,"kb_avail":92880616,"statfs":{"total":95927926784,"available":95109750784,"internally_reserved":0,"allocated":95109750784,"data_stored":95109750784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3370078,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32600000000000001,"15min":0.33100000000000002},"min":{"1min":0.22,"5min":0.188,"15min":0.157},"max":{"1min":0.503,"5min":0.52100000000000002,"15min":0.75600000000000001},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.376,"15min":0.379},"min":{"1min":0.254,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.47999999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.38900000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34699999999999998,"15min":0.36899999999999999},"min":{"1min":0.25,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.46300000000000002,"5min":0.54800000000000004,"15min":0.69499999999999995},"last":0.374},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40000000000000002,"15min":0.38100000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.19},"max":{"1min":0.55100000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42999999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.28399999999999997},"max":{"1min":0.58299999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51300000000000001,"15min":0.51400000000000001},"min":{"1min":0.40500000000000003,"5min":0.30599999999999999,"15min":0.26400000000000001},"max":{"1min":0.64000000000000001,"5min":0.745,"15min":0.77400000000000002},"last":0.65900000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.625,"5min":0.71799999999999997,"15min":0.72099999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59699999999999998,"5min":0.69999999999999996,"15min":0.752},"last":0.54400000000000004}]},{"osd":5,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65700000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.46800000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66900000000000004,"5min":0.73899999999999999,"15min":0.75600000000000001},"last":0.68899999999999995}]}]}],"pool_statfs":[]}} 2024-09-15T01:50:02.366 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:50:02.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.456+0000 7f629211d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:02.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.456+0000 7f629211d640 10 monclient: build_initial_monmap 2024-09-15T01:50:02.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.456+0000 7f629211d640 10 monclient: monmap: 2024-09-15T01:50:02.458 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:02.458 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.458 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:02.457195+0000 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:02.457195+0000 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:02.459 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:02.460 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 1 Processor -- start 2024-09-15T01:50:02.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 1 -- start start 2024-09-15T01:50:02.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 10 monclient: init 2024-09-15T01:50:02.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:02.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 10 monclient: _add_conns ranks=[1,8,7,3,10,5,4,6,0,9,2] 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f628c0a2390 addr v1:172.21.15.204:6789/0 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f628c09f8a0 0x7f628c09fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 10 monclient(hunting): picked mon.noname-e con 0x7f628c09f8a0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f628c137bb0 0x7f628c139f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 10 monclient(hunting): picked mon.noname-i con 0x7f628c137bb0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c04bf60 con 0x7f628c0a2390 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f628c09d700 con 0x7f628c137bb0 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f628c09ddb0 con 0x7f628c09f8a0 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f629211d640 10 monclient(hunting): authenticate will time out at 1807.207602s 2024-09-15T01:50:02.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628affd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f628c09f8a0 0x7f628c09fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628affd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f628c09f8a0 0x7f628c09fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40088/0 (socket says 172.21.15.178:40088) 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628affd640 1 -- 172.21.15.178:0/2541336108 learned_addr learned my addr 172.21.15.178:0/2541336108 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628affd640 10 monclient(hunting): get_auth_request con 0x7f628c09f8a0 auth_method 0 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628bfff640 1 --2- 172.21.15.178:0/2541336108 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f628c137bb0 0x7f628c139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628bfff640 10 monclient(hunting): get_auth_request con 0x7f628c137bb0 auth_method 0 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.457+0000 7f628bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628affd640 10 monclient(hunting): handle_auth_done global_id 94508 payload 274 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628affd640 1 -- 172.21.15.178:0/2541336108 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f628c137bb0 msgr2=0x7f628c139f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628affd640 1 --2- 172.21.15.178:0/2541336108 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f628c137bb0 0x7f628c139f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628affd640 1 -- 172.21.15.178:0/2541336108 >> v1:172.21.15.204:6789/0 conn(0x7f628c0a2390 legacy=0x7f628c132a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628affd640 10 monclient: _finish_hunting 0 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628affd640 1 monclient: found mon.noname-e 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:50:02.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628affd640 1 -- 172.21.15.178:0/2541336108 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0a2760 con 0x7f628c09f8a0 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628affd640 1 --2- 172.21.15.178:0/2541336108 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f628c09f8a0 0x7f628c09fc70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6274009940 tx=0x7f62740091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=cc3e969391286321 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628a7fc640 1 -- 172.21.15.178:0/2541336108 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f627402ad00 con 0x7f628c09f8a0 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628a7fc640 10 monclient: dump: 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:02.471 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f628a7fc640 1 -- 172.21.15.178:0/2541336108 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627402ae80 con 0x7f628c09f8a0 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f629211d640 5 monclient: authenticate success, global_id 94508 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.458+0000 7f629211d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:02.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f628a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f629211d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f629211d640 4 set_mon_vals no callback set 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f628a7fc640 1 -- 172.21.15.178:0/2541336108 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6274031ca0 con 0x7f628c09f8a0 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f628a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f628a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f628a7fc640 10 monclient: dump: 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.473 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f629211d640 10 monclient: shutdown 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f629211d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f629211d640 1 -- 172.21.15.178:0/2541336108 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f628c09f8a0 msgr2=0x7f628c09fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f629211d640 1 --2- 172.21.15.178:0/2541336108 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f628c09f8a0 0x7f628c09fc70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6274009940 tx=0x7f62740091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f629211d640 1 -- 172.21.15.178:0/2541336108 shutdown_connections 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f629211d640 1 --2- 172.21.15.178:0/2541336108 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f628c137bb0 0x7f628c139f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f629211d640 1 --2- 172.21.15.178:0/2541336108 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f628c09f8a0 0x7f628c09fc70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f629211d640 1 -- 172.21.15.178:0/2541336108 >> 172.21.15.178:0/2541336108 conn(0x7f628c0631b0 msgr2=0x7f628c0635b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f629211d640 1 -- 172.21.15.178:0/2541336108 shutdown_connections 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.459+0000 7f629211d640 1 -- 172.21.15.178:0/2541336108 wait complete. 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.460+0000 7f629211d640 10 monclient: build_initial_monmap 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.460+0000 7f629211d640 10 monclient: monmap: 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:02.475 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.460+0000 7f629211d640 1 Processor -- start 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.460+0000 7f629211d640 1 -- start start 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.460+0000 7f629211d640 10 monclient: init 2024-09-15T01:50:02.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.460+0000 7f629211d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.460+0000 7f629211d640 10 monclient: _add_conns ranks=[4,1,5,0,2,7,6,3,8] 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.460+0000 7f629211d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f628c09f8a0 0x7f628c0a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.460+0000 7f629211d640 10 monclient(hunting): picked mon.noname-c con 0x7f628c09f8a0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.460+0000 7f629211d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f628c0a2390 0x7f628c0a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.460+0000 7f629211d640 10 monclient(hunting): picked mon.noname-f con 0x7f628c0a2390 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f628c137bb0 0x7f628c0a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 10 monclient(hunting): picked mon.noname-h con 0x7f628c137bb0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c134b00 con 0x7f628c0a2390 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f628c09d3d0 con 0x7f628c09f8a0 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f628c135b00 con 0x7f628c137bb0 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 10 monclient(hunting): authenticate will time out at 1807.211031s 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f628c09f8a0 0x7f628c0a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f628c09f8a0 0x7f628c0a3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44792/0 (socket says 172.21.15.178:44792) 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 1 -- 172.21.15.178:0/4287778404 learned_addr learned my addr 172.21.15.178:0/4287778404 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 10 monclient(hunting): get_auth_request con 0x7f628c09f8a0 auth_method 0 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628affd640 1 --2- 172.21.15.178:0/4287778404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f628c0a2390 0x7f628c0a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628affd640 10 monclient(hunting): get_auth_request con 0x7f628c0a2390 auth_method 0 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628bfff640 1 --2- 172.21.15.178:0/4287778404 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f628c137bb0 0x7f628c0a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628bfff640 10 monclient(hunting): get_auth_request con 0x7f628c137bb0 auth_method 0 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 10 monclient(hunting): handle_auth_done global_id 45103 payload 824 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 1 -- 172.21.15.178:0/4287778404 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f628c137bb0 msgr2=0x7f628c0a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 1 --2- 172.21.15.178:0/4287778404 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f628c137bb0 0x7f628c0a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 1 -- 172.21.15.178:0/4287778404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f628c0a2390 msgr2=0x7f628c0a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 1 --2- 172.21.15.178:0/4287778404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f628c0a2390 0x7f628c0a4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 1 monclient: found mon.noname-c 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 1 -- 172.21.15.178:0/4287778404 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6274009450 con 0x7f628c09f8a0 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f628b7fe640 1 --2- 172.21.15.178:0/4287778404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f628c09f8a0 0x7f628c0a3c40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f627c00c940 tx=0x7f627c0016d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f62837fe640 1 -- 172.21.15.178:0/4287778404 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f627c00acd0 con 0x7f628c09f8a0 2024-09-15T01:50:02.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f62837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f62837fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f62837fe640 10 monclient: dump: 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:02.481 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f62837fe640 10 monclient: _finish_auth 0 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f62837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f62837fe640 1 -- 172.21.15.178:0/4287778404 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c00ae90 con 0x7f628c09f8a0 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 5 monclient: authenticate success, global_id 45103 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 10 monclient: _renew_subs 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 1 -- 172.21.15.178:0/4287778404 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0a55d0 con 0x7f628c09f8a0 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f62837fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f62837fe640 1 -- 172.21.15.178:0/4287778404 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f627c0128a0 con 0x7f628c09f8a0 2024-09-15T01:50:02.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f62837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f62837fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f62837fe640 10 monclient: dump: 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:02.483 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f6289ffb640 4 set_mon_vals no callback set 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 10 monclient: _renew_subs 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.461+0000 7f629211d640 1 -- 172.21.15.178:0/4287778404 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c0a5a10 con 0x7f628c09f8a0 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.462+0000 7f62837fe640 1 -- 172.21.15.178:0/4287778404 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f627c012420 con 0x7f628c09f8a0 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.462+0000 7f629211d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.462+0000 7f629211d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.462+0000 7f629211d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:02.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.462+0000 7f62837fe640 1 --2- 172.21.15.178:0/4287778404 >> v2:172.21.15.204:6812/28210 conn(0x7f626c0397b0 0x7f626c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.462+0000 7f629211d640 1 -- 172.21.15.178:0/4287778404 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c04b720 con 0x7f628c09f8a0 2024-09-15T01:50:02.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.462+0000 7f62837fe640 1 -- 172.21.15.178:0/4287778404 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f627c04b8f0 con 0x7f628c09f8a0 2024-09-15T01:50:02.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.462+0000 7f628affd640 1 --2- 172.21.15.178:0/4287778404 >> v2:172.21.15.204:6812/28210 conn(0x7f626c0397b0 0x7f626c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.462+0000 7f628affd640 10 monclient: get_auth_request con 0x7f626c0397b0 auth_method 0 2024-09-15T01:50:02.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.463+0000 7f628affd640 1 --2- 172.21.15.178:0/4287778404 >> v2:172.21.15.204:6812/28210 conn(0x7f626c0397b0 0x7f626c03bc60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6274034000 tx=0x7f6274032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:02.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.467+0000 7f62837fe640 1 -- 172.21.15.178:0/4287778404 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f627c018c50 con 0x7f628c09f8a0 2024-09-15T01:50:02.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.467+0000 7f62837fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:02.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.467+0000 7f62837fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:02.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.617+0000 7f629211d640 1 -- 172.21.15.178:0/4287778404 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f628c05ed70 con 0x7f626c0397b0 2024-09-15T01:50:02.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.620+0000 7f62837fe640 1 -- 172.21.15.178:0/4287778404 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+93378 (secure 0 0 0) 0x7f628c05ed70 con 0x7f626c0397b0 2024-09-15T01:50:02.622 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:50:02.623 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-09-15T01:50:02.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 1 -- 172.21.15.178:0/4287778404 >> v2:172.21.15.204:6812/28210 conn(0x7f626c0397b0 msgr2=0x7f626c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 1 --2- 172.21.15.178:0/4287778404 >> v2:172.21.15.204:6812/28210 conn(0x7f626c0397b0 0x7f626c03bc60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6274034000 tx=0x7f6274032000 comp rx=0 tx=0).stop 2024-09-15T01:50:02.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 10 monclient: shutdown 2024-09-15T01:50:02.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:02.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 1 -- 172.21.15.178:0/4287778404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f628c09f8a0 msgr2=0x7f628c0a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 1 --2- 172.21.15.178:0/4287778404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f628c09f8a0 0x7f628c0a3c40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f627c00c940 tx=0x7f627c0016d0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 1 -- 172.21.15.178:0/4287778404 shutdown_connections 2024-09-15T01:50:02.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 1 --2- 172.21.15.178:0/4287778404 >> v2:172.21.15.204:6812/28210 conn(0x7f626c0397b0 0x7f626c03bc60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 1 --2- 172.21.15.178:0/4287778404 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f628c137bb0 0x7f628c0a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 1 --2- 172.21.15.178:0/4287778404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f628c0a2390 0x7f628c0a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 1 --2- 172.21.15.178:0/4287778404 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f628c09f8a0 0x7f628c0a3c40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 1 -- 172.21.15.178:0/4287778404 >> 172.21.15.178:0/4287778404 conn(0x7f628c0631b0 msgr2=0x7f628c09e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:02.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 1 -- 172.21.15.178:0/4287778404 shutdown_connections 2024-09-15T01:50:02.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.622+0000 7f629211d640 1 -- 172.21.15.178:0/4287778404 wait complete. 2024-09-15T01:50:02.634 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":801,"stamp":"2024-09-15T01:50:02.386366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562077696,"kb_used":4795144,"kb_used_data":557282552,"kb_used_omap":18904,"kb_used_meta":0,"kb_avail":557282552,"statfs":{"total":575567560704,"available":570657333248,"internally_reserved":0,"allocated":570657333248,"data_stored":570657333248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19358691,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002524"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288825+0000","last_change":"2024-09-15T01:36:25.227464+0000","last_active":"2024-09-15T01:44:24.288825+0000","last_peered":"2024-09-15T01:44:24.288825+0000","last_clean":"2024-09-15T01:44:24.288825+0000","last_became_active":"2024-09-15T01:36:25.227161+0000","last_became_peered":"2024-09-15T01:36:25.227161+0000","last_unstale":"2024-09-15T01:44:24.288825+0000","last_undegraded":"2024-09-15T01:44:24.288825+0000","last_fullsized":"2024-09-15T01:44:24.288825+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:44:14.933447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":97,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161809+0000","last_change":"2024-09-15T01:36:25.228477+0000","last_active":"2024-09-15T01:44:24.161809+0000","last_peered":"2024-09-15T01:44:24.161809+0000","last_clean":"2024-09-15T01:44:24.161809+0000","last_became_active":"2024-09-15T01:36:25.228216+0000","last_became_peered":"2024-09-15T01:36:25.228216+0000","last_unstale":"2024-09-15T01:44:24.161809+0000","last_undegraded":"2024-09-15T01:44:24.161809+0000","last_fullsized":"2024-09-15T01:44:24.161809+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:33:26.644405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288874+0000","last_change":"2024-09-15T01:36:25.228316+0000","last_active":"2024-09-15T01:44:24.288874+0000","last_peered":"2024-09-15T01:44:24.288874+0000","last_clean":"2024-09-15T01:44:24.288874+0000","last_became_active":"2024-09-15T01:36:25.228024+0000","last_became_peered":"2024-09-15T01:36:25.228024+0000","last_unstale":"2024-09-15T01:44:24.288874+0000","last_undegraded":"2024-09-15T01:44:24.288874+0000","last_fullsized":"2024-09-15T01:44:24.288874+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:35.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":97,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161781+0000","last_change":"2024-09-15T01:36:25.230047+0000","last_active":"2024-09-15T01:44:24.161781+0000","last_peered":"2024-09-15T01:44:24.161781+0000","last_clean":"2024-09-15T01:44:24.161781+0000","last_became_active":"2024-09-15T01:36:25.229786+0000","last_became_peered":"2024-09-15T01:36:25.229786+0000","last_unstale":"2024-09-15T01:44:24.161781+0000","last_undegraded":"2024-09-15T01:44:24.161781+0000","last_fullsized":"2024-09-15T01:44:24.161781+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:15:18.249275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":92,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160052+0000","last_change":"2024-09-15T01:36:25.233254+0000","last_active":"2024-09-15T01:44:24.160052+0000","last_peered":"2024-09-15T01:44:24.160052+0000","last_clean":"2024-09-15T01:44:24.160052+0000","last_became_active":"2024-09-15T01:36:25.232778+0000","last_became_peered":"2024-09-15T01:36:25.232778+0000","last_unstale":"2024-09-15T01:44:24.160052+0000","last_undegraded":"2024-09-15T01:44:24.160052+0000","last_fullsized":"2024-09-15T01:44:24.160052+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:35:24.397702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.289191+0000","last_change":"2024-09-15T01:36:25.227383+0000","last_active":"2024-09-15T01:44:24.289191+0000","last_peered":"2024-09-15T01:44:24.289191+0000","last_clean":"2024-09-15T01:44:24.289191+0000","last_became_active":"2024-09-15T01:36:25.227142+0000","last_became_peered":"2024-09-15T01:36:25.227142+0000","last_unstale":"2024-09-15T01:44:24.289191+0000","last_undegraded":"2024-09-15T01:44:24.289191+0000","last_fullsized":"2024-09-15T01:44:24.289191+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:22:59.700920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":170,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159692+0000","last_change":"2024-09-15T01:36:25.231017+0000","last_active":"2024-09-15T01:44:24.159692+0000","last_peered":"2024-09-15T01:44:24.159692+0000","last_clean":"2024-09-15T01:44:24.159692+0000","last_became_active":"2024-09-15T01:36:25.230723+0000","last_became_peered":"2024-09-15T01:36:25.230723+0000","last_unstale":"2024-09-15T01:44:24.159692+0000","last_undegraded":"2024-09-15T01:44:24.159692+0000","last_fullsized":"2024-09-15T01:44:24.159692+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:39.782818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160643+0000","last_change":"2024-09-15T01:36:25.228490+0000","last_active":"2024-09-15T01:44:24.160643+0000","last_peered":"2024-09-15T01:44:24.160643+0000","last_clean":"2024-09-15T01:44:24.160643+0000","last_became_active":"2024-09-15T01:36:25.228252+0000","last_became_peered":"2024-09-15T01:36:25.228252+0000","last_unstale":"2024-09-15T01:44:24.160643+0000","last_undegraded":"2024-09-15T01:44:24.160643+0000","last_fullsized":"2024-09-15T01:44:24.160643+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:56:14.136629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":98,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161798+0000","last_change":"2024-09-15T01:36:25.227553+0000","last_active":"2024-09-15T01:44:24.161798+0000","last_peered":"2024-09-15T01:44:24.161798+0000","last_clean":"2024-09-15T01:44:24.161798+0000","last_became_active":"2024-09-15T01:36:25.226915+0000","last_became_peered":"2024-09-15T01:36:25.226915+0000","last_unstale":"2024-09-15T01:44:24.161798+0000","last_undegraded":"2024-09-15T01:44:24.161798+0000","last_fullsized":"2024-09-15T01:44:24.161798+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:18:34.776247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":118,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159771+0000","last_change":"2024-09-15T01:36:25.229923+0000","last_active":"2024-09-15T01:44:24.159771+0000","last_peered":"2024-09-15T01:44:24.159771+0000","last_clean":"2024-09-15T01:44:24.159771+0000","last_became_active":"2024-09-15T01:36:25.229665+0000","last_became_peered":"2024-09-15T01:36:25.229665+0000","last_unstale":"2024-09-15T01:44:24.159771+0000","last_undegraded":"2024-09-15T01:44:24.159771+0000","last_fullsized":"2024-09-15T01:44:24.159771+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:55:30.051647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":176,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159528+0000","last_change":"2024-09-15T01:36:25.227907+0000","last_active":"2024-09-15T01:44:24.159528+0000","last_peered":"2024-09-15T01:44:24.159528+0000","last_clean":"2024-09-15T01:44:24.159528+0000","last_became_active":"2024-09-15T01:36:25.227397+0000","last_became_peered":"2024-09-15T01:36:25.227397+0000","last_unstale":"2024-09-15T01:44:24.159528+0000","last_undegraded":"2024-09-15T01:44:24.159528+0000","last_fullsized":"2024-09-15T01:44:24.159528+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:52.384874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160501+0000","last_change":"2024-09-15T01:36:25.227223+0000","last_active":"2024-09-15T01:44:24.160501+0000","last_peered":"2024-09-15T01:44:24.160501+0000","last_clean":"2024-09-15T01:44:24.160501+0000","last_became_active":"2024-09-15T01:36:25.226969+0000","last_became_peered":"2024-09-15T01:36:25.226969+0000","last_unstale":"2024-09-15T01:44:24.160501+0000","last_undegraded":"2024-09-15T01:44:24.160501+0000","last_fullsized":"2024-09-15T01:44:24.160501+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:11:23.636488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.289013+0000","last_change":"2024-09-15T01:36:26.228988+0000","last_active":"2024-09-15T01:44:24.289013+0000","last_peered":"2024-09-15T01:44:24.289013+0000","last_clean":"2024-09-15T01:44:24.289013+0000","last_became_active":"2024-09-15T01:36:26.228683+0000","last_became_peered":"2024-09-15T01:36:26.228683+0000","last_unstale":"2024-09-15T01:44:24.289013+0000","last_undegraded":"2024-09-15T01:44:24.289013+0000","last_fullsized":"2024-09-15T01:44:24.289013+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:42.069204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":87,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160502+0000","last_change":"2024-09-15T01:36:26.228676+0000","last_active":"2024-09-15T01:44:24.160502+0000","last_peered":"2024-09-15T01:44:24.160502+0000","last_clean":"2024-09-15T01:44:24.160502+0000","last_became_active":"2024-09-15T01:36:26.228429+0000","last_became_peered":"2024-09-15T01:36:26.228429+0000","last_unstale":"2024-09-15T01:44:24.160502+0000","last_undegraded":"2024-09-15T01:44:24.160502+0000","last_fullsized":"2024-09-15T01:44:24.160502+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:41:46.249246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":106,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288775+0000","last_change":"2024-09-15T01:36:26.226092+0000","last_active":"2024-09-15T01:44:24.288775+0000","last_peered":"2024-09-15T01:44:24.288775+0000","last_clean":"2024-09-15T01:44:24.288775+0000","last_became_active":"2024-09-15T01:36:26.225738+0000","last_became_peered":"2024-09-15T01:36:26.225738+0000","last_unstale":"2024-09-15T01:44:24.288775+0000","last_undegraded":"2024-09-15T01:44:24.288775+0000","last_fullsized":"2024-09-15T01:44:24.288775+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:49:14.676091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288958+0000","last_change":"2024-09-15T01:36:25.225943+0000","last_active":"2024-09-15T01:44:24.288958+0000","last_peered":"2024-09-15T01:44:24.288958+0000","last_clean":"2024-09-15T01:44:24.288958+0000","last_became_active":"2024-09-15T01:36:25.225608+0000","last_became_peered":"2024-09-15T01:36:25.225608+0000","last_unstale":"2024-09-15T01:44:24.288958+0000","last_undegraded":"2024-09-15T01:44:24.288958+0000","last_fullsized":"2024-09-15T01:44:24.288958+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:16:43.688463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":100,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159580+0000","last_change":"2024-09-15T01:36:25.260030+0000","last_active":"2024-09-15T01:44:24.159580+0000","last_peered":"2024-09-15T01:44:24.159580+0000","last_clean":"2024-09-15T01:44:24.159580+0000","last_became_active":"2024-09-15T01:36:25.233270+0000","last_became_peered":"2024-09-15T01:36:25.233270+0000","last_unstale":"2024-09-15T01:44:24.159580+0000","last_undegraded":"2024-09-15T01:44:24.159580+0000","last_fullsized":"2024-09-15T01:44:24.159580+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_clean_scrub_stamp":"2024-09-15T01:31:59.015115+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:28:21.577888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":258,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159895+0000","last_change":"2024-09-15T01:36:25.233515+0000","last_active":"2024-09-15T01:44:24.159895+0000","last_peered":"2024-09-15T01:44:24.159895+0000","last_clean":"2024-09-15T01:44:24.159895+0000","last_became_active":"2024-09-15T01:36:25.230032+0000","last_became_peered":"2024-09-15T01:36:25.230032+0000","last_unstale":"2024-09-15T01:44:24.159895+0000","last_undegraded":"2024-09-15T01:44:24.159895+0000","last_fullsized":"2024-09-15T01:44:24.159895+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:44:35.523507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076976900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161688+0000","last_change":"2024-09-15T01:36:25.227415+0000","last_active":"2024-09-15T01:44:24.161688+0000","last_peered":"2024-09-15T01:44:24.161688+0000","last_clean":"2024-09-15T01:44:24.161688+0000","last_became_active":"2024-09-15T01:36:25.226499+0000","last_became_peered":"2024-09-15T01:36:25.226499+0000","last_unstale":"2024-09-15T01:44:24.161688+0000","last_undegraded":"2024-09-15T01:44:24.161688+0000","last_fullsized":"2024-09-15T01:44:24.161688+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:35:59.548191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037511000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160610+0000","last_change":"2024-09-15T01:36:25.231459+0000","last_active":"2024-09-15T01:44:24.160610+0000","last_peered":"2024-09-15T01:44:24.160610+0000","last_clean":"2024-09-15T01:44:24.160610+0000","last_became_active":"2024-09-15T01:36:25.231212+0000","last_became_peered":"2024-09-15T01:36:25.231212+0000","last_unstale":"2024-09-15T01:44:24.160610+0000","last_undegraded":"2024-09-15T01:44:24.160610+0000","last_fullsized":"2024-09-15T01:44:24.160610+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:16:13.072580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046258900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":296,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160823+0000","last_change":"2024-09-15T01:36:25.236741+0000","last_active":"2024-09-15T01:44:24.160823+0000","last_peered":"2024-09-15T01:44:24.160823+0000","last_clean":"2024-09-15T01:44:24.160823+0000","last_became_active":"2024-09-15T01:36:25.226718+0000","last_became_peered":"2024-09-15T01:36:25.226718+0000","last_unstale":"2024-09-15T01:44:24.160823+0000","last_undegraded":"2024-09-15T01:44:24.160823+0000","last_fullsized":"2024-09-15T01:44:24.160823+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:07:25.773736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093022000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.289138+0000","last_change":"2024-09-15T01:36:25.227607+0000","last_active":"2024-09-15T01:44:24.289138+0000","last_peered":"2024-09-15T01:44:24.289138+0000","last_clean":"2024-09-15T01:44:24.289138+0000","last_became_active":"2024-09-15T01:36:25.227361+0000","last_became_peered":"2024-09-15T01:36:25.227361+0000","last_unstale":"2024-09-15T01:44:24.289138+0000","last_undegraded":"2024-09-15T01:44:24.289138+0000","last_fullsized":"2024-09-15T01:44:24.289138+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:15:42.847085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031034999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159956+0000","last_change":"2024-09-15T01:36:25.230157+0000","last_active":"2024-09-15T01:44:24.159956+0000","last_peered":"2024-09-15T01:44:24.159956+0000","last_clean":"2024-09-15T01:44:24.159956+0000","last_became_active":"2024-09-15T01:36:25.229674+0000","last_became_peered":"2024-09-15T01:36:25.229674+0000","last_unstale":"2024-09-15T01:44:24.159956+0000","last_undegraded":"2024-09-15T01:44:24.159956+0000","last_fullsized":"2024-09-15T01:44:24.159956+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:12:26.442496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077134000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288724+0000","last_change":"2024-09-15T01:36:25.225614+0000","last_active":"2024-09-15T01:44:24.288724+0000","last_peered":"2024-09-15T01:44:24.288724+0000","last_clean":"2024-09-15T01:44:24.288724+0000","last_became_active":"2024-09-15T01:36:25.225361+0000","last_became_peered":"2024-09-15T01:36:25.225361+0000","last_unstale":"2024-09-15T01:44:24.288724+0000","last_undegraded":"2024-09-15T01:44:24.288724+0000","last_fullsized":"2024-09-15T01:44:24.288724+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:25:49.332027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041942500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":245,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160207+0000","last_change":"2024-09-15T01:36:26.437860+0000","last_active":"2024-09-15T01:44:24.160207+0000","last_peered":"2024-09-15T01:44:24.160207+0000","last_clean":"2024-09-15T01:44:24.160207+0000","last_became_active":"2024-09-15T01:36:26.437594+0000","last_became_peered":"2024-09-15T01:36:26.437594+0000","last_unstale":"2024-09-15T01:44:24.160207+0000","last_undegraded":"2024-09-15T01:44:24.160207+0000","last_fullsized":"2024-09-15T01:44:24.160207+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:50:29.143741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065783200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640475,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799576,"kb_used_data":92880040,"kb_used_omap":3214,"kb_used_meta":0,"kb_avail":92880040,"statfs":{"total":95927926784,"available":95109160960,"internally_reserved":0,"allocated":95109160960,"data_stored":95109160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3291775,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.39500000000000002,"5min":0.28999999999999998,"15min":0.24299999999999999},"max":{"1min":0.65500000000000003,"5min":0.73499999999999999,"15min":0.95099999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59699999999999998,"5min":0.68100000000000005,"15min":0.76700000000000002},"last":0.41099999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.28899999999999998,"15min":0.28299999999999997},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.91800000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.30499999999999999},"max":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.72899999999999998},"last":0.45600000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.23300000000000001},"max":{"1min":0.629,"5min":0.69199999999999995,"15min":0.73399999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.50600000000000001},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.27700000000000002},"max":{"1min":0.629,"5min":0.67200000000000004,"15min":0.81000000000000005},"last":0.372}]},{"osd":3,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.27200000000000002,"5min":0.224,"15min":0.20699999999999999},"max":{"1min":0.52600000000000002,"5min":0.63400000000000001,"15min":0.96799999999999997},"last":0.432},{"interface":"front","average":{"1min":0.377,"5min":0.39700000000000002,"15min":0.38600000000000001},"min":{"1min":0.28199999999999997,"5min":0.23200000000000001,"15min":0.216},"max":{"1min":0.54700000000000004,"5min":0.64700000000000002,"15min":0.86199999999999999},"last":0.55500000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.378,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":1.0289999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40699999999999997,"15min":0.40300000000000002},"min":{"1min":0.26400000000000001,"5min":0.249,"15min":0.22},"max":{"1min":0.54300000000000004,"5min":0.61799999999999999,"15min":0.67200000000000004},"last":0.48899999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799000,"kb_used_data":92880616,"kb_used_omap":3258,"kb_used_meta":0,"kb_avail":92880616,"statfs":{"total":95927926784,"available":95109750784,"internally_reserved":0,"allocated":95109750784,"data_stored":95109750784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3336786,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.48999999999999999},"min":{"1min":0.39800000000000002,"5min":0.33500000000000002,"15min":0.219},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.39700000000000002,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.66600000000000004,"5min":0.69399999999999995,"15min":0.70799999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53600000000000003,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.25600000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.52800000000000002},"min":{"1min":0.47599999999999998,"5min":0.36499999999999999,"15min":0.28999999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.60099999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.71599999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69199999999999995,"5min":0.876,"15min":0.876},"last":0.54600000000000004}]},{"osd":3,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.36699999999999999},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.22700000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.40699999999999997,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.216},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.3540000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.39600000000000002,"15min":0.376},"min":{"1min":0.27900000000000003,"5min":0.26600000000000001,"15min":0.22700000000000001},"max":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":1.2709999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.39600000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.24199999999999999},"max":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":1.2250000000000001},"last":0.53300000000000003}]}]},{"osd":3,"up_from":11,"seq":47244640475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798996,"kb_used_data":92880620,"kb_used_omap":3033,"kb_used_meta":0,"kb_avail":92880620,"statfs":{"total":95927926784,"available":95109754880,"internally_reserved":0,"allocated":95109754880,"data_stored":95109754880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3106015,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.24299999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.69399999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.29899999999999999},"max":{"1min":0.64900000000000002,"5min":0.66400000000000003,"15min":0.81599999999999995},"last":0.45300000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.27900000000000003},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.79300000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.37,"5min":0.34399999999999997,"15min":0.29899999999999999},"max":{"1min":0.58799999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.442}]},{"osd":2,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.74299999999999999},"last":0.375},{"interface":"front","average":{"1min":0.501,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.30099999999999999},"max":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.75800000000000001},"last":0.40500000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36099999999999999,"15min":0.35999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.46200000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.313},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.61699999999999999,"5min":0.65900000000000003,"15min":0.70599999999999996},"last":0.33000000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.53200000000000003,"5min":0.57099999999999995,"15min":0.65500000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.39000000000000001,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.49299999999999999,"5min":0.64600000000000002,"15min":0.70399999999999996},"last":0.34699999999999998}]}]},{"osd":2,"up_from":11,"seq":47244640475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798996,"kb_used_data":92880620,"kb_used_omap":3051,"kb_used_meta":0,"kb_avail":92880620,"statfs":{"total":95927926784,"available":95109754880,"internally_reserved":0,"allocated":95109754880,"data_stored":95109754880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3125031,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.32600000000000001},"min":{"1min":0.23200000000000001,"5min":0.20100000000000001,"15min":0.17499999999999999},"max":{"1min":0.47999999999999998,"5min":0.48499999999999999,"15min":0.53000000000000003},"last":0.16300000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.375,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.23999999999999999,"15min":0.20200000000000001},"max":{"1min":0.501,"5min":0.53000000000000003,"15min":0.59999999999999998},"last":0.26300000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.33200000000000002,"15min":0.33500000000000002},"min":{"1min":0.20300000000000001,"5min":0.187,"15min":0.182},"max":{"1min":0.46100000000000002,"5min":0.52500000000000002,"15min":0.60599999999999998},"last":0.191},{"interface":"front","average":{"1min":0.377,"5min":0.373,"15min":0.373},"min":{"1min":0.24299999999999999,"5min":0.222,"15min":0.20699999999999999},"max":{"1min":0.53300000000000003,"5min":0.57099999999999995,"15min":0.67400000000000004},"last":0.252}]},{"osd":3,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.28899999999999998},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.78900000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61299999999999999,"5min":0.77900000000000003,"15min":1.0129999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.55400000000000005,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":0.63500000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":0.371},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.27600000000000002},"max":{"1min":0.59699999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.32200000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.27800000000000002},"max":{"1min":0.67300000000000004,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.314},{"interface":"front","average":{"1min":0.504,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.30099999999999999},"max":{"1min":0.623,"5min":0.75900000000000001,"15min":0.87},"last":0.36299999999999999}]}]},{"osd":1,"up_from":11,"seq":47244640476,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799576,"kb_used_data":92880040,"kb_used_omap":3055,"kb_used_meta":0,"kb_avail":92880040,"statfs":{"total":95927926784,"available":95109160960,"internally_reserved":0,"allocated":95109160960,"data_stored":95109160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3129006,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33700000000000002,"15min":0.33300000000000002},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.188},"max":{"1min":0.47399999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.35499999999999998,"15min":0.35299999999999998},"min":{"1min":0.27800000000000002,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.68200000000000005},"last":0.28499999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.37},"min":{"1min":0.29799999999999999,"5min":0.245,"15min":0.23799999999999999},"max":{"1min":0.501,"5min":0.51400000000000001,"15min":0.73399999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28899999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.70299999999999996},"last":0.313}]},{"osd":3,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.46999999999999997,"5min":0.34499999999999997,"15min":0.30099999999999999},"max":{"1min":0.68700000000000006,"5min":0.72099999999999997,"15min":0.76800000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.46800000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.76700000000000002},"last":0.53500000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.49099999999999999,"5min":0.40500000000000003,"15min":0.35599999999999998},"max":{"1min":0.70599999999999996,"5min":0.77700000000000002,"15min":0.80600000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.308},"max":{"1min":0.71699999999999997,"5min":0.76800000000000002,"15min":0.81899999999999995},"last":0.56799999999999995}]},{"osd":5,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.32600000000000001},"max":{"1min":0.69099999999999995,"5min":0.748,"15min":0.89400000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.53800000000000003},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.32200000000000001},"max":{"1min":0.64400000000000002,"5min":0.69699999999999995,"15min":0.81000000000000005},"last":0.499}]}]},{"osd":0,"up_from":11,"seq":47244640475,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799000,"kb_used_data":92880616,"kb_used_omap":3291,"kb_used_meta":0,"kb_avail":92880616,"statfs":{"total":95927926784,"available":95109750784,"internally_reserved":0,"allocated":95109750784,"data_stored":95109750784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3370078,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32600000000000001,"15min":0.33100000000000002},"min":{"1min":0.22,"5min":0.188,"15min":0.157},"max":{"1min":0.503,"5min":0.52100000000000002,"15min":0.75600000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.376,"15min":0.379},"min":{"1min":0.254,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.47999999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.38700000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34699999999999998,"15min":0.36899999999999999},"min":{"1min":0.25,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.46300000000000002,"5min":0.54800000000000004,"15min":0.69499999999999995},"last":0.376},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40000000000000002,"15min":0.38100000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.19},"max":{"1min":0.55100000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.41399999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.28399999999999997},"max":{"1min":0.58299999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51300000000000001,"15min":0.51400000000000001},"min":{"1min":0.40500000000000003,"5min":0.30599999999999999,"15min":0.26400000000000001},"max":{"1min":0.64000000000000001,"5min":0.745,"15min":0.77400000000000002},"last":0.46300000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.625,"5min":0.71799999999999997,"15min":0.72099999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59699999999999998,"5min":0.69999999999999996,"15min":0.752},"last":0.48099999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65700000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.439},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.46800000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66900000000000004,"5min":0.73899999999999999,"15min":0.75600000000000001},"last":0.49099999999999999}]}]}],"pool_statfs":[]}} 2024-09-15T01:50:02.636 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T01:50:02.637 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:50:02.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.745+0000 7f13e7a20640 10 monclient: get_monmap_and_config 2024-09-15T01:50:02.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.745+0000 7f13e7a20640 10 monclient: build_initial_monmap 2024-09-15T01:50:02.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.745+0000 7f13e7a20640 10 monclient: monmap: 2024-09-15T01:50:02.747 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:02.747 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.747 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:02.746182+0000 2024-09-15T01:50:02.748 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:02.746182+0000 2024-09-15T01:50:02.748 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:02.748 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.748 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:02.748 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:02.748 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:02.748 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:02.748 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:02.748 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:02.749 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:02.749 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:02.749 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:02.749 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:02.749 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:02.749 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.745+0000 7f13e7a20640 1 Processor -- start 2024-09-15T01:50:02.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 1 -- start start 2024-09-15T01:50:02.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 10 monclient: init 2024-09-15T01:50:02.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:02.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 10 monclient: _add_conns ranks=[8,4,5,0,1,2,9,6,10,7,3] 2024-09-15T01:50:02.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f13e009ccd0 0x7f13e0097ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 10 monclient(hunting): picked mon.noname-e con 0x7f13e009ccd0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:02.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f13e009d0a0 0x7f13e009d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 10 monclient(hunting): picked mon.noname-c con 0x7f13e009d0a0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:02.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f13e0136ed0 0x7f13e01392b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 10 monclient(hunting): picked mon.noname-h con 0x7f13e0136ed0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:02.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13e009dfe0 con 0x7f13e009d0a0 2024-09-15T01:50:02.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13e00975a0 con 0x7f13e0136ed0 2024-09-15T01:50:02.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f13e00972c0 con 0x7f13e009ccd0 2024-09-15T01:50:02.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:02.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e7a20640 10 monclient(hunting): authenticate will time out at 1807.496829s 2024-09-15T01:50:02.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e4fbb640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f13e009d0a0 0x7f13e009d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.746+0000 7f13e57bc640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f13e009ccd0 0x7f13e0097ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f13e009d0a0 0x7f13e009d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44808/0 (socket says 172.21.15.178:44808) 2024-09-15T01:50:02.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 1 -- 172.21.15.178:0/2955359083 learned_addr learned my addr 172.21.15.178:0/2955359083 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:02.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 10 monclient(hunting): get_auth_request con 0x7f13e009d0a0 auth_method 0 2024-09-15T01:50:02.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e57bc640 10 monclient(hunting): get_auth_request con 0x7f13e009ccd0 auth_method 0 2024-09-15T01:50:02.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e57bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e57bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e57bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e5fbd640 1 --2- 172.21.15.178:0/2955359083 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f13e0136ed0 0x7f13e01392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e57bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e57bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e57bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e5fbd640 10 monclient(hunting): get_auth_request con 0x7f13e0136ed0 auth_method 0 2024-09-15T01:50:02.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e5fbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e5fbd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e5fbd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e5fbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e5fbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e5fbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 10 monclient(hunting): handle_auth_done global_id 45112 payload 274 2024-09-15T01:50:02.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 1 -- 172.21.15.178:0/2955359083 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f13e009ccd0 msgr2=0x7f13e0097ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:02.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 1 --2- 172.21.15.178:0/2955359083 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f13e009ccd0 0x7f13e0097ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 1 -- 172.21.15.178:0/2955359083 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f13e0136ed0 msgr2=0x7f13e01392b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:02.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 1 --2- 172.21.15.178:0/2955359083 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f13e0136ed0 0x7f13e01392b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 10 monclient: _finish_hunting 0 2024-09-15T01:50:02.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 1 monclient: found mon.noname-c 2024-09-15T01:50:02.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:02.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:02.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e4fbb640 1 -- 172.21.15.178:0/2955359083 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e004bf60 con 0x7f13e009d0a0 2024-09-15T01:50:02.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.747+0000 7f13e57bc640 1 --2- 172.21.15.178:0/2955359083 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f13e009ccd0 0x7f13e0097ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:02.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13e4fbb640 1 --2- 172.21.15.178:0/2955359083 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f13e009d0a0 0x7f13e009d470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009940 tx=0x7f13d00091d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8157928fb3521c32 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:02.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13d7fff640 1 -- 172.21.15.178:0/2955359083 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f13d002ac60 con 0x7f13e009d0a0 2024-09-15T01:50:02.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:02.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13d7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:02.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13d7fff640 10 monclient: dump: 2024-09-15T01:50:02.768 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:02.768 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.768 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13d7fff640 10 monclient: _finish_auth 0 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13d7fff640 1 -- 172.21.15.178:0/2955359083 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d002ade0 con 0x7f13e009d0a0 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13e7a20640 5 monclient: authenticate success, global_id 45112 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13d7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13e7a20640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13e7a20640 4 set_mon_vals no callback set 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13d7fff640 1 -- 172.21.15.178:0/2955359083 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f13d0031ca0 con 0x7f13e009d0a0 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13d7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.748+0000 7f13d7fff640 10 monclient: dump: 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.770 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.749+0000 7f13e7a20640 10 monclient: shutdown 2024-09-15T01:50:02.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.749+0000 7f13e7a20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:02.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.749+0000 7f13e7a20640 1 -- 172.21.15.178:0/2955359083 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f13e009d0a0 msgr2=0x7f13e009d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.749+0000 7f13e7a20640 1 --2- 172.21.15.178:0/2955359083 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f13e009d0a0 0x7f13e009d470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009940 tx=0x7f13d00091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.749+0000 7f13e7a20640 1 -- 172.21.15.178:0/2955359083 shutdown_connections 2024-09-15T01:50:02.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.749+0000 7f13e7a20640 1 --2- 172.21.15.178:0/2955359083 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f13e0136ed0 0x7f13e01392b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.749+0000 7f13e7a20640 1 --2- 172.21.15.178:0/2955359083 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f13e009d0a0 0x7f13e009d470 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.749+0000 7f13e7a20640 1 --2- 172.21.15.178:0/2955359083 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f13e009ccd0 0x7f13e0097ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.749+0000 7f13e7a20640 1 -- 172.21.15.178:0/2955359083 >> 172.21.15.178:0/2955359083 conn(0x7f13e00a22e0 msgr2=0x7f13e00a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:02.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.749+0000 7f13e7a20640 1 -- 172.21.15.178:0/2955359083 shutdown_connections 2024-09-15T01:50:02.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.749+0000 7f13e7a20640 1 -- 172.21.15.178:0/2955359083 wait complete. 2024-09-15T01:50:02.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.750+0000 7f13e7a20640 10 monclient: build_initial_monmap 2024-09-15T01:50:02.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.750+0000 7f13e7a20640 10 monclient: monmap: 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:02.773 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.750+0000 7f13e7a20640 1 Processor -- start 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.750+0000 7f13e7a20640 1 -- start start 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.750+0000 7f13e7a20640 10 monclient: init 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 10 monclient: _add_conns ranks=[0,2,5,7,4,1,3,6,8] 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e009ccd0 0x7f13e0132900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 10 monclient(hunting): picked mon.noname-a con 0x7f13e009ccd0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e009d0a0 0x7f13e006d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 10 monclient(hunting): picked mon.noname-b con 0x7f13e009d0a0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f13e0136ed0 0x7f13e006def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 10 monclient(hunting): picked mon.noname-h con 0x7f13e0136ed0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e011ec60 con 0x7f13e009ccd0 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e011d2f0 con 0x7f13e009d0a0 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13e011ee60 con 0x7f13e0136ed0 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e7a20640 10 monclient(hunting): authenticate will time out at 1807.501440s 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e4fbb640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e009d0a0 0x7f13e006d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e4fbb640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e009d0a0 0x7f13e006d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39578/0 (socket says 172.21.15.178:39578) 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e4fbb640 1 -- 172.21.15.178:0/1815519142 learned_addr learned my addr 172.21.15.178:0/1815519142 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e4fbb640 10 monclient(hunting): get_auth_request con 0x7f13e009d0a0 auth_method 0 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e4fbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e4fbb640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e4fbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 1 --2- 172.21.15.178:0/1815519142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e009ccd0 0x7f13e0132900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 10 monclient(hunting): get_auth_request con 0x7f13e009ccd0 auth_method 0 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e4fbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e4fbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e5fbd640 1 --2- 172.21.15.178:0/1815519142 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f13e0136ed0 0x7f13e006def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e4fbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e5fbd640 10 monclient(hunting): get_auth_request con 0x7f13e0136ed0 auth_method 0 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e5fbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e5fbd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e5fbd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e5fbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e5fbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e5fbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 10 monclient(hunting): handle_auth_done global_id 104193 payload 824 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 1 -- 172.21.15.178:0/1815519142 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f13e0136ed0 msgr2=0x7f13e006def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 1 --2- 172.21.15.178:0/1815519142 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f13e0136ed0 0x7f13e006def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 1 -- 172.21.15.178:0/1815519142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e009d0a0 msgr2=0x7f13e006d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 1 --2- 172.21.15.178:0/1815519142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e009d0a0 0x7f13e006d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 10 monclient: _finish_hunting 0 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 1 monclient: found mon.noname-a 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e57bc640 1 -- 172.21.15.178:0/1815519142 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc009390 con 0x7f13e009ccd0 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.751+0000 7f13e4fbb640 1 --2- 172.21.15.178:0/1815519142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e009d0a0 0x7f13e006d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13e57bc640 1 --2- 172.21.15.178:0/1815519142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e009ccd0 0x7f13e0132900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00ea50 tx=0x7f13dc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d57fa640 1 -- 172.21.15.178:0/1815519142 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f13dc009670 con 0x7f13e009ccd0 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d57fa640 10 monclient: dump: 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.778 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.779 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.779 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:02.779 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.779 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:02.779 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:02.779 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:02.779 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d57fa640 10 monclient: _finish_auth 0 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d57fa640 1 -- 172.21.15.178:0/1815519142 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc00ce90 con 0x7f13e009ccd0 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13e7a20640 5 monclient: authenticate success, global_id 104193 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13e7a20640 10 monclient: _renew_subs 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13e7a20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:02.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13e7a20640 1 -- 172.21.15.178:0/1815519142 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d0009450 con 0x7f13e009ccd0 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d57fa640 1 -- 172.21.15.178:0/1815519142 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f13dc0138f0 con 0x7f13e009ccd0 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d77fe640 4 set_mon_vals no callback set 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13d57fa640 10 monclient: dump: 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:02.781 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13e7a20640 10 monclient: _renew_subs 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13e7a20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.752+0000 7f13e7a20640 1 -- 172.21.15.178:0/1815519142 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e0131ef0 con 0x7f13e009ccd0 2024-09-15T01:50:02.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.753+0000 7f13d57fa640 1 -- 172.21.15.178:0/1815519142 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f13dc013470 con 0x7f13e009ccd0 2024-09-15T01:50:02.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.753+0000 7f13d57fa640 1 --2- 172.21.15.178:0/1815519142 >> v2:172.21.15.204:6812/28210 conn(0x7f13c40397b0 0x7f13c403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:02.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.753+0000 7f13d57fa640 1 -- 172.21.15.178:0/1815519142 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f13dc04c220 con 0x7f13e009ccd0 2024-09-15T01:50:02.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.753+0000 7f13e7a20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:02.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.753+0000 7f13e7a20640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:02.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.753+0000 7f13e7a20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:02.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.753+0000 7f13e7a20640 1 -- 172.21.15.178:0/1815519142 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e009c2e0 con 0x7f13e009ccd0 2024-09-15T01:50:02.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.753+0000 7f13e4fbb640 1 --2- 172.21.15.178:0/1815519142 >> v2:172.21.15.204:6812/28210 conn(0x7f13c40397b0 0x7f13c403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:02.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.753+0000 7f13e4fbb640 10 monclient: get_auth_request con 0x7f13c40397b0 auth_method 0 2024-09-15T01:50:02.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.753+0000 7f13e4fbb640 1 --2- 172.21.15.178:0/1815519142 >> v2:172.21.15.204:6812/28210 conn(0x7f13c40397b0 0x7f13c403bc60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009940 tx=0x7f13d0032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:02.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.758+0000 7f13d57fa640 1 -- 172.21.15.178:0/1815519142 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f13dc013720 con 0x7f13e009ccd0 2024-09-15T01:50:02.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.758+0000 7f13d57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:02.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.758+0000 7f13d57fa640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:02.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.914+0000 7f13e7a20640 1 -- 172.21.15.178:0/1815519142 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f13e004c4a0 con 0x7f13c40397b0 2024-09-15T01:50:02.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.917+0000 7f13d57fa640 1 -- 172.21.15.178:0/1815519142 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+93378 (secure 0 0 0) 0x7f13e004c4a0 con 0x7f13c40397b0 2024-09-15T01:50:02.919 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:50:02.920 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-09-15T01:50:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 1 -- 172.21.15.178:0/1815519142 >> v2:172.21.15.204:6812/28210 conn(0x7f13c40397b0 msgr2=0x7f13c403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 1 --2- 172.21.15.178:0/1815519142 >> v2:172.21.15.204:6812/28210 conn(0x7f13c40397b0 0x7f13c403bc60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009940 tx=0x7f13d0032000 comp rx=0 tx=0).stop 2024-09-15T01:50:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 10 monclient: shutdown 2024-09-15T01:50:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 1 -- 172.21.15.178:0/1815519142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e009ccd0 msgr2=0x7f13e0132900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 1 --2- 172.21.15.178:0/1815519142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e009ccd0 0x7f13e0132900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00ea50 tx=0x7f13dc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 1 -- 172.21.15.178:0/1815519142 shutdown_connections 2024-09-15T01:50:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 1 --2- 172.21.15.178:0/1815519142 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f13e0136ed0 0x7f13e006def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 1 --2- 172.21.15.178:0/1815519142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e009d0a0 0x7f13e006d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 1 --2- 172.21.15.178:0/1815519142 >> v2:172.21.15.204:6812/28210 conn(0x7f13c40397b0 0x7f13c403bc60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 1 --2- 172.21.15.178:0/1815519142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e009ccd0 0x7f13e0132900 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:02.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 1 -- 172.21.15.178:0/1815519142 >> 172.21.15.178:0/1815519142 conn(0x7f13e00a22e0 msgr2=0x7f13e00996c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:02.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 1 -- 172.21.15.178:0/1815519142 shutdown_connections 2024-09-15T01:50:02.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:02.919+0000 7f13e7a20640 1 -- 172.21.15.178:0/1815519142 wait complete. 2024-09-15T01:50:02.931 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":801,"stamp":"2024-09-15T01:50:02.386366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562077696,"kb_used":4795144,"kb_used_data":557282552,"kb_used_omap":18904,"kb_used_meta":0,"kb_avail":557282552,"statfs":{"total":575567560704,"available":570657333248,"internally_reserved":0,"allocated":570657333248,"data_stored":570657333248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19358691,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002524"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288825+0000","last_change":"2024-09-15T01:36:25.227464+0000","last_active":"2024-09-15T01:44:24.288825+0000","last_peered":"2024-09-15T01:44:24.288825+0000","last_clean":"2024-09-15T01:44:24.288825+0000","last_became_active":"2024-09-15T01:36:25.227161+0000","last_became_peered":"2024-09-15T01:36:25.227161+0000","last_unstale":"2024-09-15T01:44:24.288825+0000","last_undegraded":"2024-09-15T01:44:24.288825+0000","last_fullsized":"2024-09-15T01:44:24.288825+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:44:14.933447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":97,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161809+0000","last_change":"2024-09-15T01:36:25.228477+0000","last_active":"2024-09-15T01:44:24.161809+0000","last_peered":"2024-09-15T01:44:24.161809+0000","last_clean":"2024-09-15T01:44:24.161809+0000","last_became_active":"2024-09-15T01:36:25.228216+0000","last_became_peered":"2024-09-15T01:36:25.228216+0000","last_unstale":"2024-09-15T01:44:24.161809+0000","last_undegraded":"2024-09-15T01:44:24.161809+0000","last_fullsized":"2024-09-15T01:44:24.161809+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:33:26.644405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288874+0000","last_change":"2024-09-15T01:36:25.228316+0000","last_active":"2024-09-15T01:44:24.288874+0000","last_peered":"2024-09-15T01:44:24.288874+0000","last_clean":"2024-09-15T01:44:24.288874+0000","last_became_active":"2024-09-15T01:36:25.228024+0000","last_became_peered":"2024-09-15T01:36:25.228024+0000","last_unstale":"2024-09-15T01:44:24.288874+0000","last_undegraded":"2024-09-15T01:44:24.288874+0000","last_fullsized":"2024-09-15T01:44:24.288874+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:35.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":97,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161781+0000","last_change":"2024-09-15T01:36:25.230047+0000","last_active":"2024-09-15T01:44:24.161781+0000","last_peered":"2024-09-15T01:44:24.161781+0000","last_clean":"2024-09-15T01:44:24.161781+0000","last_became_active":"2024-09-15T01:36:25.229786+0000","last_became_peered":"2024-09-15T01:36:25.229786+0000","last_unstale":"2024-09-15T01:44:24.161781+0000","last_undegraded":"2024-09-15T01:44:24.161781+0000","last_fullsized":"2024-09-15T01:44:24.161781+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:15:18.249275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":92,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160052+0000","last_change":"2024-09-15T01:36:25.233254+0000","last_active":"2024-09-15T01:44:24.160052+0000","last_peered":"2024-09-15T01:44:24.160052+0000","last_clean":"2024-09-15T01:44:24.160052+0000","last_became_active":"2024-09-15T01:36:25.232778+0000","last_became_peered":"2024-09-15T01:36:25.232778+0000","last_unstale":"2024-09-15T01:44:24.160052+0000","last_undegraded":"2024-09-15T01:44:24.160052+0000","last_fullsized":"2024-09-15T01:44:24.160052+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:35:24.397702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.289191+0000","last_change":"2024-09-15T01:36:25.227383+0000","last_active":"2024-09-15T01:44:24.289191+0000","last_peered":"2024-09-15T01:44:24.289191+0000","last_clean":"2024-09-15T01:44:24.289191+0000","last_became_active":"2024-09-15T01:36:25.227142+0000","last_became_peered":"2024-09-15T01:36:25.227142+0000","last_unstale":"2024-09-15T01:44:24.289191+0000","last_undegraded":"2024-09-15T01:44:24.289191+0000","last_fullsized":"2024-09-15T01:44:24.289191+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:22:59.700920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":170,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159692+0000","last_change":"2024-09-15T01:36:25.231017+0000","last_active":"2024-09-15T01:44:24.159692+0000","last_peered":"2024-09-15T01:44:24.159692+0000","last_clean":"2024-09-15T01:44:24.159692+0000","last_became_active":"2024-09-15T01:36:25.230723+0000","last_became_peered":"2024-09-15T01:36:25.230723+0000","last_unstale":"2024-09-15T01:44:24.159692+0000","last_undegraded":"2024-09-15T01:44:24.159692+0000","last_fullsized":"2024-09-15T01:44:24.159692+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:39.782818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160643+0000","last_change":"2024-09-15T01:36:25.228490+0000","last_active":"2024-09-15T01:44:24.160643+0000","last_peered":"2024-09-15T01:44:24.160643+0000","last_clean":"2024-09-15T01:44:24.160643+0000","last_became_active":"2024-09-15T01:36:25.228252+0000","last_became_peered":"2024-09-15T01:36:25.228252+0000","last_unstale":"2024-09-15T01:44:24.160643+0000","last_undegraded":"2024-09-15T01:44:24.160643+0000","last_fullsized":"2024-09-15T01:44:24.160643+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:56:14.136629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":98,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161798+0000","last_change":"2024-09-15T01:36:25.227553+0000","last_active":"2024-09-15T01:44:24.161798+0000","last_peered":"2024-09-15T01:44:24.161798+0000","last_clean":"2024-09-15T01:44:24.161798+0000","last_became_active":"2024-09-15T01:36:25.226915+0000","last_became_peered":"2024-09-15T01:36:25.226915+0000","last_unstale":"2024-09-15T01:44:24.161798+0000","last_undegraded":"2024-09-15T01:44:24.161798+0000","last_fullsized":"2024-09-15T01:44:24.161798+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:18:34.776247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":118,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159771+0000","last_change":"2024-09-15T01:36:25.229923+0000","last_active":"2024-09-15T01:44:24.159771+0000","last_peered":"2024-09-15T01:44:24.159771+0000","last_clean":"2024-09-15T01:44:24.159771+0000","last_became_active":"2024-09-15T01:36:25.229665+0000","last_became_peered":"2024-09-15T01:36:25.229665+0000","last_unstale":"2024-09-15T01:44:24.159771+0000","last_undegraded":"2024-09-15T01:44:24.159771+0000","last_fullsized":"2024-09-15T01:44:24.159771+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:55:30.051647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":176,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159528+0000","last_change":"2024-09-15T01:36:25.227907+0000","last_active":"2024-09-15T01:44:24.159528+0000","last_peered":"2024-09-15T01:44:24.159528+0000","last_clean":"2024-09-15T01:44:24.159528+0000","last_became_active":"2024-09-15T01:36:25.227397+0000","last_became_peered":"2024-09-15T01:36:25.227397+0000","last_unstale":"2024-09-15T01:44:24.159528+0000","last_undegraded":"2024-09-15T01:44:24.159528+0000","last_fullsized":"2024-09-15T01:44:24.159528+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:52.384874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160501+0000","last_change":"2024-09-15T01:36:25.227223+0000","last_active":"2024-09-15T01:44:24.160501+0000","last_peered":"2024-09-15T01:44:24.160501+0000","last_clean":"2024-09-15T01:44:24.160501+0000","last_became_active":"2024-09-15T01:36:25.226969+0000","last_became_peered":"2024-09-15T01:36:25.226969+0000","last_unstale":"2024-09-15T01:44:24.160501+0000","last_undegraded":"2024-09-15T01:44:24.160501+0000","last_fullsized":"2024-09-15T01:44:24.160501+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:11:23.636488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.289013+0000","last_change":"2024-09-15T01:36:26.228988+0000","last_active":"2024-09-15T01:44:24.289013+0000","last_peered":"2024-09-15T01:44:24.289013+0000","last_clean":"2024-09-15T01:44:24.289013+0000","last_became_active":"2024-09-15T01:36:26.228683+0000","last_became_peered":"2024-09-15T01:36:26.228683+0000","last_unstale":"2024-09-15T01:44:24.289013+0000","last_undegraded":"2024-09-15T01:44:24.289013+0000","last_fullsized":"2024-09-15T01:44:24.289013+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:42.069204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":87,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160502+0000","last_change":"2024-09-15T01:36:26.228676+0000","last_active":"2024-09-15T01:44:24.160502+0000","last_peered":"2024-09-15T01:44:24.160502+0000","last_clean":"2024-09-15T01:44:24.160502+0000","last_became_active":"2024-09-15T01:36:26.228429+0000","last_became_peered":"2024-09-15T01:36:26.228429+0000","last_unstale":"2024-09-15T01:44:24.160502+0000","last_undegraded":"2024-09-15T01:44:24.160502+0000","last_fullsized":"2024-09-15T01:44:24.160502+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:41:46.249246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":106,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288775+0000","last_change":"2024-09-15T01:36:26.226092+0000","last_active":"2024-09-15T01:44:24.288775+0000","last_peered":"2024-09-15T01:44:24.288775+0000","last_clean":"2024-09-15T01:44:24.288775+0000","last_became_active":"2024-09-15T01:36:26.225738+0000","last_became_peered":"2024-09-15T01:36:26.225738+0000","last_unstale":"2024-09-15T01:44:24.288775+0000","last_undegraded":"2024-09-15T01:44:24.288775+0000","last_fullsized":"2024-09-15T01:44:24.288775+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:49:14.676091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288958+0000","last_change":"2024-09-15T01:36:25.225943+0000","last_active":"2024-09-15T01:44:24.288958+0000","last_peered":"2024-09-15T01:44:24.288958+0000","last_clean":"2024-09-15T01:44:24.288958+0000","last_became_active":"2024-09-15T01:36:25.225608+0000","last_became_peered":"2024-09-15T01:36:25.225608+0000","last_unstale":"2024-09-15T01:44:24.288958+0000","last_undegraded":"2024-09-15T01:44:24.288958+0000","last_fullsized":"2024-09-15T01:44:24.288958+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:16:43.688463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":100,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159580+0000","last_change":"2024-09-15T01:36:25.260030+0000","last_active":"2024-09-15T01:44:24.159580+0000","last_peered":"2024-09-15T01:44:24.159580+0000","last_clean":"2024-09-15T01:44:24.159580+0000","last_became_active":"2024-09-15T01:36:25.233270+0000","last_became_peered":"2024-09-15T01:36:25.233270+0000","last_unstale":"2024-09-15T01:44:24.159580+0000","last_undegraded":"2024-09-15T01:44:24.159580+0000","last_fullsized":"2024-09-15T01:44:24.159580+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_clean_scrub_stamp":"2024-09-15T01:31:59.015115+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:28:21.577888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":258,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159895+0000","last_change":"2024-09-15T01:36:25.233515+0000","last_active":"2024-09-15T01:44:24.159895+0000","last_peered":"2024-09-15T01:44:24.159895+0000","last_clean":"2024-09-15T01:44:24.159895+0000","last_became_active":"2024-09-15T01:36:25.230032+0000","last_became_peered":"2024-09-15T01:36:25.230032+0000","last_unstale":"2024-09-15T01:44:24.159895+0000","last_undegraded":"2024-09-15T01:44:24.159895+0000","last_fullsized":"2024-09-15T01:44:24.159895+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:44:35.523507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076976900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161688+0000","last_change":"2024-09-15T01:36:25.227415+0000","last_active":"2024-09-15T01:44:24.161688+0000","last_peered":"2024-09-15T01:44:24.161688+0000","last_clean":"2024-09-15T01:44:24.161688+0000","last_became_active":"2024-09-15T01:36:25.226499+0000","last_became_peered":"2024-09-15T01:36:25.226499+0000","last_unstale":"2024-09-15T01:44:24.161688+0000","last_undegraded":"2024-09-15T01:44:24.161688+0000","last_fullsized":"2024-09-15T01:44:24.161688+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:35:59.548191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037511000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160610+0000","last_change":"2024-09-15T01:36:25.231459+0000","last_active":"2024-09-15T01:44:24.160610+0000","last_peered":"2024-09-15T01:44:24.160610+0000","last_clean":"2024-09-15T01:44:24.160610+0000","last_became_active":"2024-09-15T01:36:25.231212+0000","last_became_peered":"2024-09-15T01:36:25.231212+0000","last_unstale":"2024-09-15T01:44:24.160610+0000","last_undegraded":"2024-09-15T01:44:24.160610+0000","last_fullsized":"2024-09-15T01:44:24.160610+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:16:13.072580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046258900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":296,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160823+0000","last_change":"2024-09-15T01:36:25.236741+0000","last_active":"2024-09-15T01:44:24.160823+0000","last_peered":"2024-09-15T01:44:24.160823+0000","last_clean":"2024-09-15T01:44:24.160823+0000","last_became_active":"2024-09-15T01:36:25.226718+0000","last_became_peered":"2024-09-15T01:36:25.226718+0000","last_unstale":"2024-09-15T01:44:24.160823+0000","last_undegraded":"2024-09-15T01:44:24.160823+0000","last_fullsized":"2024-09-15T01:44:24.160823+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:07:25.773736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093022000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.289138+0000","last_change":"2024-09-15T01:36:25.227607+0000","last_active":"2024-09-15T01:44:24.289138+0000","last_peered":"2024-09-15T01:44:24.289138+0000","last_clean":"2024-09-15T01:44:24.289138+0000","last_became_active":"2024-09-15T01:36:25.227361+0000","last_became_peered":"2024-09-15T01:36:25.227361+0000","last_unstale":"2024-09-15T01:44:24.289138+0000","last_undegraded":"2024-09-15T01:44:24.289138+0000","last_fullsized":"2024-09-15T01:44:24.289138+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:15:42.847085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031034999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159956+0000","last_change":"2024-09-15T01:36:25.230157+0000","last_active":"2024-09-15T01:44:24.159956+0000","last_peered":"2024-09-15T01:44:24.159956+0000","last_clean":"2024-09-15T01:44:24.159956+0000","last_became_active":"2024-09-15T01:36:25.229674+0000","last_became_peered":"2024-09-15T01:36:25.229674+0000","last_unstale":"2024-09-15T01:44:24.159956+0000","last_undegraded":"2024-09-15T01:44:24.159956+0000","last_fullsized":"2024-09-15T01:44:24.159956+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:12:26.442496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077134000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288724+0000","last_change":"2024-09-15T01:36:25.225614+0000","last_active":"2024-09-15T01:44:24.288724+0000","last_peered":"2024-09-15T01:44:24.288724+0000","last_clean":"2024-09-15T01:44:24.288724+0000","last_became_active":"2024-09-15T01:36:25.225361+0000","last_became_peered":"2024-09-15T01:36:25.225361+0000","last_unstale":"2024-09-15T01:44:24.288724+0000","last_undegraded":"2024-09-15T01:44:24.288724+0000","last_fullsized":"2024-09-15T01:44:24.288724+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:25:49.332027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041942500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":245,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160207+0000","last_change":"2024-09-15T01:36:26.437860+0000","last_active":"2024-09-15T01:44:24.160207+0000","last_peered":"2024-09-15T01:44:24.160207+0000","last_clean":"2024-09-15T01:44:24.160207+0000","last_became_active":"2024-09-15T01:36:26.437594+0000","last_became_peered":"2024-09-15T01:36:26.437594+0000","last_unstale":"2024-09-15T01:44:24.160207+0000","last_undegraded":"2024-09-15T01:44:24.160207+0000","last_fullsized":"2024-09-15T01:44:24.160207+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:50:29.143741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065783200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640475,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799576,"kb_used_data":92880040,"kb_used_omap":3214,"kb_used_meta":0,"kb_avail":92880040,"statfs":{"total":95927926784,"available":95109160960,"internally_reserved":0,"allocated":95109160960,"data_stored":95109160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3291775,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.39500000000000002,"5min":0.28999999999999998,"15min":0.24299999999999999},"max":{"1min":0.65500000000000003,"5min":0.73499999999999999,"15min":0.95099999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59699999999999998,"5min":0.68100000000000005,"15min":0.76700000000000002},"last":0.41099999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.28899999999999998,"15min":0.28299999999999997},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.91800000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.30499999999999999},"max":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.72899999999999998},"last":0.45600000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.23300000000000001},"max":{"1min":0.629,"5min":0.69199999999999995,"15min":0.73399999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.50600000000000001},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.27700000000000002},"max":{"1min":0.629,"5min":0.67200000000000004,"15min":0.81000000000000005},"last":0.372}]},{"osd":3,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.27200000000000002,"5min":0.224,"15min":0.20699999999999999},"max":{"1min":0.52600000000000002,"5min":0.63400000000000001,"15min":0.96799999999999997},"last":0.432},{"interface":"front","average":{"1min":0.377,"5min":0.39700000000000002,"15min":0.38600000000000001},"min":{"1min":0.28199999999999997,"5min":0.23200000000000001,"15min":0.216},"max":{"1min":0.54700000000000004,"5min":0.64700000000000002,"15min":0.86199999999999999},"last":0.55500000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.378,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":1.0289999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40699999999999997,"15min":0.40300000000000002},"min":{"1min":0.26400000000000001,"5min":0.249,"15min":0.22},"max":{"1min":0.54300000000000004,"5min":0.61799999999999999,"15min":0.67200000000000004},"last":0.48899999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799000,"kb_used_data":92880616,"kb_used_omap":3258,"kb_used_meta":0,"kb_avail":92880616,"statfs":{"total":95927926784,"available":95109750784,"internally_reserved":0,"allocated":95109750784,"data_stored":95109750784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3336786,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.48999999999999999},"min":{"1min":0.39800000000000002,"5min":0.33500000000000002,"15min":0.219},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.39700000000000002,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.66600000000000004,"5min":0.69399999999999995,"15min":0.70799999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53600000000000003,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.25600000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.52800000000000002},"min":{"1min":0.47599999999999998,"5min":0.36499999999999999,"15min":0.28999999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.60099999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.71599999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69199999999999995,"5min":0.876,"15min":0.876},"last":0.54600000000000004}]},{"osd":3,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.36699999999999999},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.22700000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.40699999999999997,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.216},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.3540000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.39600000000000002,"15min":0.376},"min":{"1min":0.27900000000000003,"5min":0.26600000000000001,"15min":0.22700000000000001},"max":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":1.2709999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.39600000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.24199999999999999},"max":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":1.2250000000000001},"last":0.53300000000000003}]}]},{"osd":3,"up_from":11,"seq":47244640475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798996,"kb_used_data":92880620,"kb_used_omap":3033,"kb_used_meta":0,"kb_avail":92880620,"statfs":{"total":95927926784,"available":95109754880,"internally_reserved":0,"allocated":95109754880,"data_stored":95109754880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3106015,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.24299999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.69399999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.29899999999999999},"max":{"1min":0.64900000000000002,"5min":0.66400000000000003,"15min":0.81599999999999995},"last":0.45300000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.27900000000000003},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.79300000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.37,"5min":0.34399999999999997,"15min":0.29899999999999999},"max":{"1min":0.58799999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.442}]},{"osd":2,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.74299999999999999},"last":0.375},{"interface":"front","average":{"1min":0.501,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.30099999999999999},"max":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.75800000000000001},"last":0.40500000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36099999999999999,"15min":0.35999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.46200000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.313},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.61699999999999999,"5min":0.65900000000000003,"15min":0.70599999999999996},"last":0.33000000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.53200000000000003,"5min":0.57099999999999995,"15min":0.65500000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.39000000000000001,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.49299999999999999,"5min":0.64600000000000002,"15min":0.70399999999999996},"last":0.34699999999999998}]}]},{"osd":2,"up_from":11,"seq":47244640475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798996,"kb_used_data":92880620,"kb_used_omap":3051,"kb_used_meta":0,"kb_avail":92880620,"statfs":{"total":95927926784,"available":95109754880,"internally_reserved":0,"allocated":95109754880,"data_stored":95109754880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3125031,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.32600000000000001},"min":{"1min":0.23200000000000001,"5min":0.20100000000000001,"15min":0.17499999999999999},"max":{"1min":0.47999999999999998,"5min":0.48499999999999999,"15min":0.53000000000000003},"last":0.16300000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.375,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.23999999999999999,"15min":0.20200000000000001},"max":{"1min":0.501,"5min":0.53000000000000003,"15min":0.59999999999999998},"last":0.26300000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.33200000000000002,"15min":0.33500000000000002},"min":{"1min":0.20300000000000001,"5min":0.187,"15min":0.182},"max":{"1min":0.46100000000000002,"5min":0.52500000000000002,"15min":0.60599999999999998},"last":0.191},{"interface":"front","average":{"1min":0.377,"5min":0.373,"15min":0.373},"min":{"1min":0.24299999999999999,"5min":0.222,"15min":0.20699999999999999},"max":{"1min":0.53300000000000003,"5min":0.57099999999999995,"15min":0.67400000000000004},"last":0.252}]},{"osd":3,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.28899999999999998},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.78900000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61299999999999999,"5min":0.77900000000000003,"15min":1.0129999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.55400000000000005,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":0.63500000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":0.371},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.27600000000000002},"max":{"1min":0.59699999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.32200000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.27800000000000002},"max":{"1min":0.67300000000000004,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.314},{"interface":"front","average":{"1min":0.504,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.30099999999999999},"max":{"1min":0.623,"5min":0.75900000000000001,"15min":0.87},"last":0.36299999999999999}]}]},{"osd":1,"up_from":11,"seq":47244640476,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799576,"kb_used_data":92880040,"kb_used_omap":3055,"kb_used_meta":0,"kb_avail":92880040,"statfs":{"total":95927926784,"available":95109160960,"internally_reserved":0,"allocated":95109160960,"data_stored":95109160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3129006,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33700000000000002,"15min":0.33300000000000002},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.188},"max":{"1min":0.47399999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.35499999999999998,"15min":0.35299999999999998},"min":{"1min":0.27800000000000002,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.68200000000000005},"last":0.28499999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.37},"min":{"1min":0.29799999999999999,"5min":0.245,"15min":0.23799999999999999},"max":{"1min":0.501,"5min":0.51400000000000001,"15min":0.73399999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28899999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.70299999999999996},"last":0.313}]},{"osd":3,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.46999999999999997,"5min":0.34499999999999997,"15min":0.30099999999999999},"max":{"1min":0.68700000000000006,"5min":0.72099999999999997,"15min":0.76800000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.46800000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.76700000000000002},"last":0.53500000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.49099999999999999,"5min":0.40500000000000003,"15min":0.35599999999999998},"max":{"1min":0.70599999999999996,"5min":0.77700000000000002,"15min":0.80600000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.308},"max":{"1min":0.71699999999999997,"5min":0.76800000000000002,"15min":0.81899999999999995},"last":0.56799999999999995}]},{"osd":5,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.32600000000000001},"max":{"1min":0.69099999999999995,"5min":0.748,"15min":0.89400000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.53800000000000003},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.32200000000000001},"max":{"1min":0.64400000000000002,"5min":0.69699999999999995,"15min":0.81000000000000005},"last":0.499}]}]},{"osd":0,"up_from":11,"seq":47244640475,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799000,"kb_used_data":92880616,"kb_used_omap":3291,"kb_used_meta":0,"kb_avail":92880616,"statfs":{"total":95927926784,"available":95109750784,"internally_reserved":0,"allocated":95109750784,"data_stored":95109750784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3370078,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32600000000000001,"15min":0.33100000000000002},"min":{"1min":0.22,"5min":0.188,"15min":0.157},"max":{"1min":0.503,"5min":0.52100000000000002,"15min":0.75600000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.376,"15min":0.379},"min":{"1min":0.254,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.47999999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.38700000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34699999999999998,"15min":0.36899999999999999},"min":{"1min":0.25,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.46300000000000002,"5min":0.54800000000000004,"15min":0.69499999999999995},"last":0.376},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40000000000000002,"15min":0.38100000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.19},"max":{"1min":0.55100000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.41399999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.28399999999999997},"max":{"1min":0.58299999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51300000000000001,"15min":0.51400000000000001},"min":{"1min":0.40500000000000003,"5min":0.30599999999999999,"15min":0.26400000000000001},"max":{"1min":0.64000000000000001,"5min":0.745,"15min":0.77400000000000002},"last":0.46300000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.625,"5min":0.71799999999999997,"15min":0.72099999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59699999999999998,"5min":0.69999999999999996,"15min":0.752},"last":0.48099999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65700000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.439},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.46800000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66900000000000004,"5min":0.73899999999999999,"15min":0.75600000000000001},"last":0.49099999999999999}]}]}],"pool_statfs":[]}} 2024-09-15T01:50:02.933 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T01:50:03.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.023+0000 7f33b0954640 10 monclient: get_monmap_and_config 2024-09-15T01:50:03.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.023+0000 7f33b0954640 10 monclient: build_initial_monmap 2024-09-15T01:50:03.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.023+0000 7f33b0954640 10 monclient: monmap: 2024-09-15T01:50:03.026 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:03.026 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:03.026 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:03.024835+0000 2024-09-15T01:50:03.027 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:03.024835+0000 2024-09-15T01:50:03.027 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:03.027 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:03.027 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:03.027 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:03.027 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:03.027 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:03.027 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:03.027 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:03.028 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:03.028 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:03.028 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:03.028 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:03.028 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:03.028 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:03.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.024+0000 7f33b0954640 1 Processor -- start 2024-09-15T01:50:03.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.024+0000 7f33b0954640 1 -- start start 2024-09-15T01:50:03.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.024+0000 7f33b0954640 10 monclient: init 2024-09-15T01:50:03.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:03.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 10 monclient: _add_conns ranks=[9,3,5,4,2,0,8,7,6,1,10] 2024-09-15T01:50:03.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f33ac0a2930 0x7f33ac132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:03.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 10 monclient(hunting): picked mon.noname-a con 0x7f33ac0a2930 addr v2:172.21.15.178:3300/0 2024-09-15T01:50:03.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ac09faa0 0x7f33ac09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:03.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 10 monclient(hunting): picked mon.noname-g con 0x7f33ac09faa0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:03.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33ac0a08c0 0x7f33ac0a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:03.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 10 monclient(hunting): picked mon.noname-h con 0x7f33ac0a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:03.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:03.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ac09e0f0 con 0x7f33ac0a2930 2024-09-15T01:50:03.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:03.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac09dd80 con 0x7f33ac09faa0 2024-09-15T01:50:03.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:03.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0a2d00 con 0x7f33ac0a08c0 2024-09-15T01:50:03.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:03.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33b0954640 10 monclient(hunting): authenticate will time out at 1807.775238s 2024-09-15T01:50:03.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f33ac0a2930 0x7f33ac132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:03.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f33ac0a2930 0x7f33ac132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35112/0 (socket says 172.21.15.178:35112) 2024-09-15T01:50:03.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 1 -- 172.21.15.178:0/1512372294 learned_addr learned my addr 172.21.15.178:0/1512372294 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:03.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 10 monclient(hunting): get_auth_request con 0x7f33ac0a2930 auth_method 0 2024-09-15T01:50:03.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:03.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:03.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:03.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:03.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:03.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:03.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33a1d9b640 1 --2- 172.21.15.178:0/1512372294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ac09faa0 0x7f33ac09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:03.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33a1d9b640 10 monclient(hunting): get_auth_request con 0x7f33ac09faa0 auth_method 0 2024-09-15T01:50:03.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:03.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:03.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aad9d640 1 --2- 172.21.15.178:0/1512372294 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33ac0a08c0 0x7f33ac0a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:03.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:03.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aad9d640 10 monclient(hunting): get_auth_request con 0x7f33ac0a08c0 auth_method 0 2024-09-15T01:50:03.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:03.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:03.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:03.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 10 monclient(hunting): handle_auth_done global_id 104202 payload 274 2024-09-15T01:50:03.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 1 -- 172.21.15.178:0/1512372294 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33ac0a08c0 msgr2=0x7f33ac0a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:03.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 1 --2- 172.21.15.178:0/1512372294 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33ac0a08c0 0x7f33ac0a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:03.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 1 -- 172.21.15.178:0/1512372294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ac09faa0 msgr2=0x7f33ac09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:03.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 1 --2- 172.21.15.178:0/1512372294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ac09faa0 0x7f33ac09fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:03.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 10 monclient: _finish_hunting 0 2024-09-15T01:50:03.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 1 monclient: found mon.noname-a 2024-09-15T01:50:03.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:03.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:50:03.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.025+0000 7f33aa59c640 1 -- 172.21.15.178:0/1512372294 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac04bf60 con 0x7f33ac0a2930 2024-09-15T01:50:03.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33aa59c640 1 --2- 172.21.15.178:0/1512372294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33ac0a2930 0x7f33ac132de0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f339c009910 tx=0x7f339c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b338ada7d2f6c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:03.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33a9d9b640 1 -- 172.21.15.178:0/1512372294 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f339c02ac80 con 0x7f33ac0a2930 2024-09-15T01:50:03.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33a9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:03.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33a9d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:50:03.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33a9d9b640 10 monclient: dump: 2024-09-15T01:50:03.043 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:03.043 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:03.043 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:03.043 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:03.043 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:03.043 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:03.044 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:03.044 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:03.044 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:03.044 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:03.044 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:03.044 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:03.044 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:03.044 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:03.044 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:03.044 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:03.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33a9d9b640 10 monclient: _finish_auth 0 2024-09-15T01:50:03.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33a9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:03.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33a9d9b640 1 -- 172.21.15.178:0/1512372294 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c02ae00 con 0x7f33ac0a2930 2024-09-15T01:50:03.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 5 monclient: authenticate success, global_id 104202 2024-09-15T01:50:03.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:03.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33a9d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:03.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:03.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33a9d9b640 1 -- 172.21.15.178:0/1512372294 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f339c031ca0 con 0x7f33ac0a2930 2024-09-15T01:50:03.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 4 set_mon_vals no callback set 2024-09-15T01:50:03.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33a9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:03.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33a9d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:50:03.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33a9d9b640 10 monclient: dump: 2024-09-15T01:50:03.046 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:03.046 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:03.046 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:03.046 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:03.046 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:03.046 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:03.046 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:03.046 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:03.047 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:03.047 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:03.047 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:03.047 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:03.047 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:03.047 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:03.047 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:03.047 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:03.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 10 monclient: shutdown 2024-09-15T01:50:03.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:03.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 1 -- 172.21.15.178:0/1512372294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33ac0a2930 msgr2=0x7f33ac132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:03.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 1 --2- 172.21.15.178:0/1512372294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33ac0a2930 0x7f33ac132de0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f339c009910 tx=0x7f339c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:03.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 1 -- 172.21.15.178:0/1512372294 shutdown_connections 2024-09-15T01:50:03.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 1 --2- 172.21.15.178:0/1512372294 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f33ac0a08c0 0x7f33ac0a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:03.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 1 --2- 172.21.15.178:0/1512372294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33ac09faa0 0x7f33ac09fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:03.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 1 --2- 172.21.15.178:0/1512372294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33ac0a2930 0x7f33ac132de0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:03.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.026+0000 7f33b0954640 1 -- 172.21.15.178:0/1512372294 >> 172.21.15.178:0/1512372294 conn(0x7f33ac0a5ed0 msgr2=0x7f33ac0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:03.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.027+0000 7f33b0954640 1 -- 172.21.15.178:0/1512372294 shutdown_connections 2024-09-15T01:50:03.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.027+0000 7f33b0954640 1 -- 172.21.15.178:0/1512372294 wait complete. 2024-09-15T01:50:03.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.027+0000 7f33b0954640 10 monclient: build_initial_monmap 2024-09-15T01:50:03.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.027+0000 7f33b0954640 10 monclient: monmap: 2024-09-15T01:50:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:03.050 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.027+0000 7f33b0954640 1 Processor -- start 2024-09-15T01:50:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.027+0000 7f33b0954640 1 -- start start 2024-09-15T01:50:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.027+0000 7f33b0954640 10 monclient: init 2024-09-15T01:50:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 10 monclient: _add_conns ranks=[2,8,4,5,7,0,1,3,6] 2024-09-15T01:50:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33ac09faa0 0x7f33ac09b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 10 monclient(hunting): picked mon.noname-b con 0x7f33ac09faa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ac0a08c0 0x7f33ac131990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 10 monclient(hunting): picked mon.noname-e con 0x7f33ac0a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ac0a2930 0x7f33ac141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 10 monclient(hunting): picked mon.noname-c con 0x7f33ac0a2930 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac09d630 con 0x7f33ac09faa0 2024-09-15T01:50:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac11eab0 con 0x7f33ac0a2930 2024-09-15T01:50:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac11ed90 con 0x7f33ac0a08c0 2024-09-15T01:50:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33b0954640 10 monclient(hunting): authenticate will time out at 1807.778414s 2024-09-15T01:50:03.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aa59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33ac09faa0 0x7f33ac09b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:03.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33a1d9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ac0a08c0 0x7f33ac131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:03.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ac0a2930 0x7f33ac141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:03.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ac0a2930 0x7f33ac141e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44816/0 (socket says 172.21.15.178:44816) 2024-09-15T01:50:03.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33a1d9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ac0a08c0 0x7f33ac131990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40106/0 (socket says 172.21.15.178:40106) 2024-09-15T01:50:03.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aa59c640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33ac09faa0 0x7f33ac09b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39586/0 (socket says 172.21.15.178:39586) 2024-09-15T01:50:03.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 1 -- 172.21.15.178:0/2046722827 learned_addr learned my addr 172.21.15.178:0/2046722827 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:03.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 10 monclient(hunting): get_auth_request con 0x7f33ac0a2930 auth_method 0 2024-09-15T01:50:03.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:03.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:03.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:03.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33a1d9b640 10 monclient(hunting): get_auth_request con 0x7f33ac0a08c0 auth_method 0 2024-09-15T01:50:03.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:03.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:03.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:03.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aa59c640 10 monclient(hunting): get_auth_request con 0x7f33ac09faa0 auth_method 0 2024-09-15T01:50:03.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:03.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aa59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:03.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:03.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:03.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:03.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:03.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33a1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:03.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33a1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:03.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33a1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:03.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:03.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:03.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:03.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 10 monclient(hunting): handle_auth_done global_id 45121 payload 824 2024-09-15T01:50:03.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 1 -- 172.21.15.178:0/2046722827 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ac0a08c0 msgr2=0x7f33ac131990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:03.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 1 --2- 172.21.15.178:0/2046722827 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ac0a08c0 0x7f33ac131990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:03.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 1 -- 172.21.15.178:0/2046722827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33ac09faa0 msgr2=0x7f33ac09b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 1 --2- 172.21.15.178:0/2046722827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33ac09faa0 0x7f33ac09b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 10 monclient: _finish_hunting 0 2024-09-15T01:50:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 1 monclient: found mon.noname-c 2024-09-15T01:50:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33aad9d640 1 -- 172.21.15.178:0/2046722827 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c009450 con 0x7f33ac0a2930 2024-09-15T01:50:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.028+0000 7f33a1d9b640 1 --2- 172.21.15.178:0/2046722827 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ac0a08c0 0x7f33ac131990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33aad9d640 1 --2- 172.21.15.178:0/2046722827 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ac0a2930 0x7f33ac141e40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009fa0 tx=0x7f33a4009900 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a2ffd640 1 -- 172.21.15.178:0/2046722827 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33a400ee60 con 0x7f33ac0a2930 2024-09-15T01:50:03.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:03.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:03.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a2ffd640 10 monclient: dump: 2024-09-15T01:50:03.056 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:03.056 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:03.056 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:03.056 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:03.056 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:03.056 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:03.056 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:03.056 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:03.056 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:03.057 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:03.057 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:03.057 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:03.057 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:03.057 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:03.057 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:03.057 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:03.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a2ffd640 10 monclient: _finish_auth 0 2024-09-15T01:50:03.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:03.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a2ffd640 1 -- 172.21.15.178:0/2046722827 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a4009990 con 0x7f33ac0a2930 2024-09-15T01:50:03.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33b0954640 5 monclient: authenticate success, global_id 45121 2024-09-15T01:50:03.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33b0954640 10 monclient: _renew_subs 2024-09-15T01:50:03.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33b0954640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:03.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33b0954640 1 -- 172.21.15.178:0/2046722827 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac09bd40 con 0x7f33ac0a2930 2024-09-15T01:50:03.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:03.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a2ffd640 1 -- 172.21.15.178:0/2046722827 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33a4014d20 con 0x7f33ac0a2930 2024-09-15T01:50:03.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:03.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:03.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a2ffd640 10 monclient: dump: 2024-09-15T01:50:03.058 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:03.059 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:03.060 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33a959a640 4 set_mon_vals no callback set 2024-09-15T01:50:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33b0954640 10 monclient: _renew_subs 2024-09-15T01:50:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33b0954640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.029+0000 7f33b0954640 1 -- 172.21.15.178:0/2046722827 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac11ca40 con 0x7f33ac0a2930 2024-09-15T01:50:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.030+0000 7f33a2ffd640 1 -- 172.21.15.178:0/2046722827 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f33a4014420 con 0x7f33ac0a2930 2024-09-15T01:50:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.030+0000 7f33b0954640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:03.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.030+0000 7f33b0954640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.030+0000 7f33b0954640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.030+0000 7f33b0954640 1 -- 172.21.15.178:0/2046722827 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac11cc00 con 0x7f33ac0a2930 2024-09-15T01:50:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.030+0000 7f33a2ffd640 1 --2- 172.21.15.178:0/2046722827 >> v2:172.21.15.204:6812/28210 conn(0x7f338c0396e0 0x7f338c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.030+0000 7f33a2ffd640 1 -- 172.21.15.178:0/2046722827 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f33a404c3d0 con 0x7f33ac0a2930 2024-09-15T01:50:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.030+0000 7f33aa59c640 1 --2- 172.21.15.178:0/2046722827 >> v2:172.21.15.204:6812/28210 conn(0x7f338c0396e0 0x7f338c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.030+0000 7f33aa59c640 10 monclient: get_auth_request con 0x7f338c0396e0 auth_method 0 2024-09-15T01:50:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.030+0000 7f33aa59c640 1 --2- 172.21.15.178:0/2046722827 >> v2:172.21.15.204:6812/28210 conn(0x7f338c0396e0 0x7f338c03bb90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f339c034000 tx=0x7f339c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.034+0000 7f33a2ffd640 1 -- 172.21.15.178:0/2046722827 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f33a40146d0 con 0x7f33ac0a2930 2024-09-15T01:50:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.034+0000 7f33a2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:03.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.034+0000 7f33a2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:03.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.189+0000 7f33b0954640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:50:03.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.190+0000 7f33b0954640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:50:03.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.190+0000 7f33b0954640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:03.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.190+0000 7f33b0954640 1 -- 172.21.15.178:0/2046722827 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33ac04b720 con 0x7f33ac0a2930 2024-09-15T01:50:03.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.190+0000 7f33a2ffd640 1 -- 172.21.15.178:0/2046722827 <== mon.4 v2:172.21.15.178:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v265) v1 ==== 74+0+19230 (secure 0 0 0) 0x7f33a4001250 con 0x7f33ac0a2930 2024-09-15T01:50:03.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.190+0000 7f33a2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T01:50:03.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.190+0000 7f33a2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:50:03.193 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:50:03.193 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":265,"fsid":"6ff069e2-2827-44d3-9460-72f9fc8c094b","created":"2024-09-15T01:31:51.657329+0000","modified":"2024-09-15T01:44:29.155274+0000","last_up_change":"2024-09-15T01:31:56.999458+0000","last_in_change":"2024-09-15T01:44:29.155274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":181,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":15,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T01:31:58.183767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T01:32:02.130544+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"fooset","create_time":"2024-09-15T01:35:26.816876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}},{"pool":10,"pool_name":"barset","create_time":"2024-09-15T01:35:27.950688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"690d2bc0-3a5e-4f4a-b00b-43a73741b7b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":29232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6801","nonce":29232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":29232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":29232}]},"public_addr":"172.21.15.178:6800/29232","cluster_addr":"172.21.15.178:6801/29232","heartbeat_back_addr":"172.21.15.178:6803/29232","heartbeat_front_addr":"172.21.15.178:6802/29232","state":["exists","up"]},{"osd":1,"uuid":"bbba9997-deb4-42ca-8140-6553a9b6c45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":29233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6805","nonce":29233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6807","nonce":29233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":29233}]},"public_addr":"172.21.15.178:6804/29233","cluster_addr":"172.21.15.178:6805/29233","heartbeat_back_addr":"172.21.15.178:6807/29233","heartbeat_front_addr":"172.21.15.178:6806/29233","state":["exists","up"]},{"osd":2,"uuid":"24b56e74-a222-40f9-a8bf-38d597553f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":29234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":29234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6811","nonce":29234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":29234}]},"public_addr":"172.21.15.178:6808/29234","cluster_addr":"172.21.15.178:6809/29234","heartbeat_back_addr":"172.21.15.178:6811/29234","heartbeat_front_addr":"172.21.15.178:6810/29234","state":["exists","up"]},{"osd":3,"uuid":"a5bf3f7d-4c59-470a-8f69-3fea018c13f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":28752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":28752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":28752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":28752}]},"public_addr":"172.21.15.204:6801/28752","cluster_addr":"172.21.15.204:6802/28752","heartbeat_back_addr":"172.21.15.204:6806/28752","heartbeat_front_addr":"172.21.15.204:6804/28752","state":["exists","up"]},{"osd":4,"uuid":"cdaf5723-3a5c-4238-a8e6-38f2325ccbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":28753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":28753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6807","nonce":28753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6805","nonce":28753}]},"public_addr":"172.21.15.204:6800/28753","cluster_addr":"172.21.15.204:6803/28753","heartbeat_back_addr":"172.21.15.204:6807/28753","heartbeat_front_addr":"172.21.15.204:6805/28753","state":["exists","up"]},{"osd":5,"uuid":"6b67035f-8718-4762-98bf-0dd0aa86e6e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":28754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":28754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6811","nonce":28754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":28754}]},"public_addr":"172.21.15.204:6808/28754","cluster_addr":"172.21.15.204:6809/28754","heartbeat_back_addr":"172.21.15.204:6811/28754","heartbeat_front_addr":"172.21.15.204:6810/28754","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["autoout","exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["autoout","exists","new"]},{"osd":8,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["autoout","exists","new"]},{"osd":9,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["autoout","exists","new"]},{"osd":10,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"e5de064c-679b-416c-b9b7-f61aa58c84ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.093711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.117496+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.123704+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.082038+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.088260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T01:31:55.090342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":65536,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":65536,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":65536,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":65536,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":0,"to":3}]},{"pgid":"9.3","mappings":[{"from":0,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"ec-foo-ssd":{"crush-device-class":"ssd","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T01:50:03.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 1 -- 172.21.15.178:0/2046722827 >> v2:172.21.15.204:6812/28210 conn(0x7f338c0396e0 msgr2=0x7f338c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:03.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 1 --2- 172.21.15.178:0/2046722827 >> v2:172.21.15.204:6812/28210 conn(0x7f338c0396e0 0x7f338c03bb90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f339c034000 tx=0x7f339c032000 comp rx=0 tx=0).stop 2024-09-15T01:50:03.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 10 monclient: shutdown 2024-09-15T01:50:03.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:03.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 1 -- 172.21.15.178:0/2046722827 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ac0a2930 msgr2=0x7f33ac141e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:03.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 1 --2- 172.21.15.178:0/2046722827 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ac0a2930 0x7f33ac141e40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009fa0 tx=0x7f33a4009900 comp rx=0 tx=0).stop 2024-09-15T01:50:03.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 1 -- 172.21.15.178:0/2046722827 shutdown_connections 2024-09-15T01:50:03.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 1 --2- 172.21.15.178:0/2046722827 >> v2:172.21.15.204:6812/28210 conn(0x7f338c0396e0 0x7f338c03bb90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:03.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 1 --2- 172.21.15.178:0/2046722827 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f33ac0a2930 0x7f33ac141e40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:03.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 1 --2- 172.21.15.178:0/2046722827 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f33ac0a08c0 0x7f33ac131990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:03.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 1 --2- 172.21.15.178:0/2046722827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33ac09faa0 0x7f33ac09b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:03.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 1 -- 172.21.15.178:0/2046722827 >> 172.21.15.178:0/2046722827 conn(0x7f33ac0a5ed0 msgr2=0x7f33ac09e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:03.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 1 -- 172.21.15.178:0/2046722827 shutdown_connections 2024-09-15T01:50:03.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:03.192+0000 7f33b0954640 1 -- 172.21.15.178:0/2046722827 wait complete. 2024-09-15T01:50:04.206 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T01:50:04.207 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:50:04.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.306+0000 7fdc66d5d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:04.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.306+0000 7fdc66d5d640 10 monclient: build_initial_monmap 2024-09-15T01:50:04.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.306+0000 7fdc66d5d640 10 monclient: monmap: 2024-09-15T01:50:04.316 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:04.316 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:04.307045+0000 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:04.307045+0000 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:04.317 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.306+0000 7fdc66d5d640 1 Processor -- start 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.306+0000 7fdc66d5d640 1 -- start start 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.306+0000 7fdc66d5d640 10 monclient: init 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 10 monclient: _add_conns ranks=[9,0,8,5,10,4,6,3,7,1,2] 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdc6009f8c0 0x7fdc60132dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 10 monclient(hunting): picked mon.noname-a con 0x7fdc6009f8c0 addr v2:172.21.15.178:3300/0 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdc6009fc90 addr v1:172.21.15.178:6789/0 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc600a0090 0x7fdc600a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 10 monclient(hunting): picked mon.noname-e con 0x7fdc600a0090 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:04.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6004bf60 con 0x7fdc6009fc90 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6009e420 con 0x7fdc6009f8c0 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6009d320 con 0x7fdc600a0090 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc66d5d640 10 monclient(hunting): authenticate will time out at 1809.057488s 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc64af9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdc6009f8c0 0x7fdc60132dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc600a0090 0x7fdc600a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc64af9640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdc6009f8c0 0x7fdc60132dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35124/0 (socket says 172.21.15.178:35124) 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc64af9640 1 -- 172.21.15.178:0/3592019783 learned_addr learned my addr 172.21.15.178:0/3592019783 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc600a0090 0x7fdc600a0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40112/0 (socket says 172.21.15.178:40112) 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc64af9640 10 monclient(hunting): get_auth_request con 0x7fdc6009f8c0 auth_method 0 2024-09-15T01:50:04.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc64af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc64af9640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc64af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 10 monclient(hunting): get_auth_request con 0x7fdc600a0090 auth_method 0 2024-09-15T01:50:04.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc5ffff640 1 --1- 172.21.15.178:0/3592019783 >> v1:172.21.15.178:6789/0 conn(0x7fdc6009fc90 0x7fdc600a0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:37018/0 (socket says 172.21.15.178:37018) 2024-09-15T01:50:04.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc64af9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc64af9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc64af9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 10 monclient(hunting): handle_auth_done global_id 94535 payload 274 2024-09-15T01:50:04.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 1 -- 172.21.15.178:0/3592019783 >> v2:172.21.15.178:3300/0 conn(0x7fdc6009f8c0 msgr2=0x7fdc60132dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:04.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 1 --2- 172.21.15.178:0/3592019783 >> v2:172.21.15.178:3300/0 conn(0x7fdc6009f8c0 0x7fdc60132dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 1 -- 172.21.15.178:0/3592019783 >> v1:172.21.15.178:6789/0 conn(0x7fdc6009fc90 legacy=0x7fdc600a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 10 monclient: _finish_hunting 0 2024-09-15T01:50:04.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 1 monclient: found mon.noname-e 2024-09-15T01:50:04.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:04.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:50:04.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc652fa640 1 -- 172.21.15.178:0/3592019783 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc600a6d30 con 0x7fdc600a0090 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.307+0000 7fdc64af9640 1 --2- 172.21.15.178:0/3592019783 >> v2:172.21.15.178:3300/0 conn(0x7fdc6009f8c0 0x7fdc60132dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc652fa640 1 --2- 172.21.15.178:0/3592019783 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc600a0090 0x7fdc600a0f40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009e60 tx=0x7fdc58009780 comp rx=0 tx=0).ready entity=mon.8 client_cookie=86138db4432c671a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc5f7fe640 1 -- 172.21.15.178:0/3592019783 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc580079d0 con 0x7fdc600a0090 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc5f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc5f7fe640 10 monclient: dump: 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.322 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc5f7fe640 10 monclient: _finish_auth 0 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc5f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc5f7fe640 1 -- 172.21.15.178:0/3592019783 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5802d050 con 0x7fdc600a0090 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc66d5d640 5 monclient: authenticate success, global_id 94535 2024-09-15T01:50:04.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc66d5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc5f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc66d5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc5f7fe640 1 -- 172.21.15.178:0/3592019783 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc58007b90 con 0x7fdc600a0090 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc66d5d640 4 set_mon_vals no callback set 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc5f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.308+0000 7fdc5f7fe640 10 monclient: dump: 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.324 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.309+0000 7fdc66d5d640 10 monclient: shutdown 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.309+0000 7fdc66d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:04.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.309+0000 7fdc66d5d640 1 -- 172.21.15.178:0/3592019783 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc600a0090 msgr2=0x7fdc600a0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.309+0000 7fdc66d5d640 1 --2- 172.21.15.178:0/3592019783 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc600a0090 0x7fdc600a0f40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009e60 tx=0x7fdc58009780 comp rx=0 tx=0).stop 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.309+0000 7fdc66d5d640 1 -- 172.21.15.178:0/3592019783 shutdown_connections 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.309+0000 7fdc66d5d640 1 --2- 172.21.15.178:0/3592019783 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdc600a0090 0x7fdc600a0f40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.309+0000 7fdc66d5d640 1 --2- 172.21.15.178:0/3592019783 >> v2:172.21.15.178:3300/0 conn(0x7fdc6009f8c0 0x7fdc60132dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.309+0000 7fdc66d5d640 1 -- 172.21.15.178:0/3592019783 >> 172.21.15.178:0/3592019783 conn(0x7fdc600a61e0 msgr2=0x7fdc600a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.309+0000 7fdc66d5d640 1 -- 172.21.15.178:0/3592019783 shutdown_connections 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.309+0000 7fdc66d5d640 1 -- 172.21.15.178:0/3592019783 wait complete. 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.309+0000 7fdc66d5d640 10 monclient: build_initial_monmap 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.309+0000 7fdc66d5d640 10 monclient: monmap: 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:04.326 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 1 Processor -- start 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 1 -- start start 2024-09-15T01:50:04.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 10 monclient: init 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 10 monclient: _add_conns ranks=[3,5,2,4,1,6,8,0,7] 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc6009f8c0 0x7fdc6012fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 10 monclient(hunting): picked mon.noname-g con 0x7fdc6009f8c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdc6009fc90 0x7fdc6011c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 10 monclient(hunting): picked mon.noname-h con 0x7fdc6009fc90 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc600a0090 0x7fdc6011cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 10 monclient(hunting): picked mon.noname-b con 0x7fdc600a0090 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6011e990 con 0x7fdc600a0090 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6009ca90 con 0x7fdc6009f8c0 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6009d530 con 0x7fdc6009fc90 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc66d5d640 10 monclient(hunting): authenticate will time out at 1809.060770s 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc652fa640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc600a0090 0x7fdc6011cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc652fa640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc600a0090 0x7fdc6011cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39602/0 (socket says 172.21.15.178:39602) 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc652fa640 1 -- 172.21.15.178:0/2698595969 learned_addr learned my addr 172.21.15.178:0/2698595969 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc652fa640 10 monclient(hunting): get_auth_request con 0x7fdc600a0090 auth_method 0 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc652fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc652fa640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc652fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc652fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.310+0000 7fdc652fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc652fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5ffff640 1 --2- 172.21.15.178:0/2698595969 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdc6009fc90 0x7fdc6011c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc64af9640 1 --2- 172.21.15.178:0/2698595969 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc6009f8c0 0x7fdc6012fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc652fa640 10 monclient(hunting): handle_auth_done global_id 35534 payload 824 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc652fa640 1 -- 172.21.15.178:0/2698595969 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdc6009fc90 msgr2=0x7fdc6011c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc652fa640 1 --2- 172.21.15.178:0/2698595969 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdc6009fc90 0x7fdc6011c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc652fa640 1 -- 172.21.15.178:0/2698595969 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc6009f8c0 msgr2=0x7fdc6012fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc652fa640 1 --2- 172.21.15.178:0/2698595969 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc6009f8c0 0x7fdc6012fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc652fa640 10 monclient: _finish_hunting 0 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc652fa640 1 monclient: found mon.noname-b 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc652fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc652fa640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc652fa640 1 -- 172.21.15.178:0/2698595969 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc50009450 con 0x7fdc600a0090 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5ffff640 10 monclient: get_auth_request con 0x7fdc6009fc90 auth_method 0 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc64af9640 10 monclient: get_auth_request con 0x7fdc6009f8c0 auth_method 0 2024-09-15T01:50:04.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5ffff640 1 --2- 172.21.15.178:0/2698595969 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fdc6009fc90 0x7fdc6011c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc64af9640 1 --2- 172.21.15.178:0/2698595969 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc6009f8c0 0x7fdc6012fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc652fa640 1 --2- 172.21.15.178:0/2698595969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc600a0090 0x7fdc6011cc80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009d30 tx=0x7fdc58037000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5cff9640 1 -- 172.21.15.178:0/2698595969 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc58007eb0 con 0x7fdc600a0090 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5cff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5cff9640 10 monclient: dump: 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.331 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5cff9640 10 monclient: _finish_auth 0 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5cff9640 1 -- 172.21.15.178:0/2698595969 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5803c420 con 0x7fdc600a0090 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc66d5d640 5 monclient: authenticate success, global_id 35534 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:04.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc66d5d640 10 monclient: _renew_subs 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5cff9640 1 -- 172.21.15.178:0/2698595969 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc5803ca50 con 0x7fdc600a0090 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc66d5d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc66d5d640 1 -- 172.21.15.178:0/2698595969 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc58009a20 con 0x7fdc600a0090 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5cff9640 10 monclient: dump: 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.333 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc5effd640 4 set_mon_vals no callback set 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc66d5d640 10 monclient: _renew_subs 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc66d5d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.311+0000 7fdc66d5d640 1 -- 172.21.15.178:0/2698595969 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6011e160 con 0x7fdc600a0090 2024-09-15T01:50:04.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.312+0000 7fdc66d5d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.312+0000 7fdc66d5d640 1 -- 172.21.15.178:0/2698595969 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdc6004c4a0 con 0x7fdc600a0090 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.312+0000 7fdc5cff9640 1 -- 172.21.15.178:0/2698595969 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdc58003040 con 0x7fdc600a0090 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.312+0000 7fdc5cff9640 1 --2- 172.21.15.178:0/2698595969 >> v2:172.21.15.204:6812/28210 conn(0x7fdc4c0397b0 0x7fdc4c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.312+0000 7fdc5cff9640 1 -- 172.21.15.178:0/2698595969 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fdc5806ceb0 con 0x7fdc600a0090 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.312+0000 7fdc5cff9640 1 --2- 172.21.15.178:0/2698595969 >> v2:172.21.15.178:6800/29232 conn(0x7fdc4c042340 0x7fdc4c0447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.312+0000 7fdc5cff9640 1 -- 172.21.15.178:0/2698595969 --> v2:172.21.15.178:6800/29232 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc4c044e60 con 0x7fdc4c042340 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.312+0000 7fdc5cff9640 1 -- 172.21.15.178:0/2698595969 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_get_version_reply(handle=1 version=265) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc5806f540 con 0x7fdc600a0090 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.312+0000 7fdc5cff9640 10 monclient: handle_get_version_reply finishing 1 version 265 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.312+0000 7fdc64af9640 1 --2- 172.21.15.178:0/2698595969 >> v2:172.21.15.204:6812/28210 conn(0x7fdc4c0397b0 0x7fdc4c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.312+0000 7fdc64af9640 10 monclient: get_auth_request con 0x7fdc4c0397b0 auth_method 0 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.312+0000 7fdc5ffff640 1 --2- 172.21.15.178:0/2698595969 >> v2:172.21.15.178:6800/29232 conn(0x7fdc4c042340 0x7fdc4c0447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.313+0000 7fdc5ffff640 10 monclient: get_auth_request con 0x7fdc4c042340 auth_method 0 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.313+0000 7fdc5ffff640 1 --2- 172.21.15.178:0/2698595969 >> v2:172.21.15.178:6800/29232 conn(0x7fdc4c042340 0x7fdc4c0447d0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.313+0000 7fdc64af9640 1 --2- 172.21.15.178:0/2698595969 >> v2:172.21.15.204:6812/28210 conn(0x7fdc4c0397b0 0x7fdc4c03bc60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009420 tx=0x7fdc50008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.313+0000 7fdc5cff9640 1 -- 172.21.15.178:0/2698595969 <== osd.0 v2:172.21.15.178:6800/29232 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fdc4c044e60 con 0x7fdc4c042340 2024-09-15T01:50:04.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.324+0000 7fdc66d5d640 1 -- 172.21.15.178:0/2698595969 --> v2:172.21.15.178:6800/29232 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdc6004bc80 con 0x7fdc4c042340 2024-09-15T01:50:04.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.324+0000 7fdc5cff9640 1 -- 172.21.15.178:0/2698595969 <== osd.0 v2:172.21.15.178:6800/29232 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fdc6004bc80 con 0x7fdc4c042340 2024-09-15T01:50:04.337 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:50:04.337 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:50:04.337 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:50:04.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc66d5d640 1 -- 172.21.15.178:0/2698595969 >> v2:172.21.15.178:6800/29232 conn(0x7fdc4c042340 msgr2=0x7fdc4c0447d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc66d5d640 1 --2- 172.21.15.178:0/2698595969 >> v2:172.21.15.178:6800/29232 conn(0x7fdc4c042340 0x7fdc4c0447d0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc66d5d640 1 -- 172.21.15.178:0/2698595969 >> v2:172.21.15.204:6812/28210 conn(0x7fdc4c0397b0 msgr2=0x7fdc4c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc66d5d640 1 --2- 172.21.15.178:0/2698595969 >> v2:172.21.15.204:6812/28210 conn(0x7fdc4c0397b0 0x7fdc4c03bc60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009420 tx=0x7fdc50008040 comp rx=0 tx=0).stop 2024-09-15T01:50:04.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc66d5d640 10 monclient: shutdown 2024-09-15T01:50:04.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc66d5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:04.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc66d5d640 1 -- 172.21.15.178:0/2698595969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc600a0090 msgr2=0x7fdc6011cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc66d5d640 1 --2- 172.21.15.178:0/2698595969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc600a0090 0x7fdc6011cc80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009d30 tx=0x7fdc58037000 comp rx=0 tx=0).stop 2024-09-15T01:50:04.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc652fa640 1 -- 172.21.15.178:0/2698595969 reap_dead start 2024-09-15T01:50:04.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc66d5d640 1 -- 172.21.15.178:0/2698595969 shutdown_connections 2024-09-15T01:50:04.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc66d5d640 1 -- 172.21.15.178:0/2698595969 >> 172.21.15.178:0/2698595969 conn(0x7fdc600a61e0 msgr2=0x7fdc60062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:04.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc66d5d640 1 -- 172.21.15.178:0/2698595969 shutdown_connections 2024-09-15T01:50:04.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.325+0000 7fdc66d5d640 1 -- 172.21.15.178:0/2698595969 wait complete. 2024-09-15T01:50:04.339 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T01:50:04.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.431+0000 7f49c899d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:04.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.431+0000 7f49c899d640 10 monclient: build_initial_monmap 2024-09-15T01:50:04.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.431+0000 7f49c899d640 10 monclient: monmap: 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:04.432178+0000 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:04.432178+0000 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:04.434 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:04.435 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:04.435 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:04.435 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:04.435 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:04.435 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.431+0000 7f49c899d640 1 Processor -- start 2024-09-15T01:50:04.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.431+0000 7f49c899d640 1 -- start start 2024-09-15T01:50:04.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.431+0000 7f49c899d640 10 monclient: init 2024-09-15T01:50:04.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:04.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 10 monclient: _add_conns ranks=[4,8,10,2,5,7,1,3,6,0,9] 2024-09-15T01:50:04.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 0x7f49c409e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 10 monclient(hunting): picked mon.noname-c con 0x7f49c40678d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:04.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 0x7f49c409ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 10 monclient(hunting): picked mon.noname-e con 0x7f49c406ea30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:04.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f49c409f180 0x7f49c41306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 10 monclient(hunting): picked mon.noname-f con 0x7f49c409f180 addr v2:172.21.15.204:3300/0 2024-09-15T01:50:04.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c409f590 con 0x7f49c409f180 2024-09-15T01:50:04.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f49c406d5b0 con 0x7f49c40678d0 2024-09-15T01:50:04.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4067ca0 con 0x7f49c406ea30 2024-09-15T01:50:04.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:04.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c899d640 10 monclient(hunting): authenticate will time out at 1809.182573s 2024-09-15T01:50:04.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c1d9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 0x7f49c409ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c259c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 0x7f49c409e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c1d9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 0x7f49c409ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40132/0 (socket says 172.21.15.178:40132) 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c259c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 0x7f49c409e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44832/0 (socket says 172.21.15.178:44832) 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c1d9b640 1 -- 172.21.15.178:0/3918279748 learned_addr learned my addr 172.21.15.178:0/3918279748 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c1d9b640 10 monclient(hunting): get_auth_request con 0x7f49c406ea30 auth_method 0 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c259c640 10 monclient(hunting): get_auth_request con 0x7f49c40678d0 auth_method 0 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c2d9d640 1 --2- 172.21.15.178:0/3918279748 >> v2:172.21.15.204:3300/0 conn(0x7f49c409f180 0x7f49c41306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c2d9d640 10 monclient(hunting): get_auth_request con 0x7f49c409f180 auth_method 0 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.432+0000 7f49c259c640 10 monclient(hunting): handle_auth_done global_id 45130 payload 274 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c259c640 1 -- 172.21.15.178:0/3918279748 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 msgr2=0x7f49c409ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c259c640 1 --2- 172.21.15.178:0/3918279748 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 0x7f49c409ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c259c640 1 -- 172.21.15.178:0/3918279748 >> v2:172.21.15.204:3300/0 conn(0x7f49c409f180 msgr2=0x7f49c41306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c259c640 1 --2- 172.21.15.178:0/3918279748 >> v2:172.21.15.204:3300/0 conn(0x7f49c409f180 0x7f49c41306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c259c640 10 monclient: _finish_hunting 0 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c259c640 1 monclient: found mon.noname-c 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c259c640 1 -- 172.21.15.178:0/3918279748 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c404bf60 con 0x7f49c40678d0 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c1d9b640 1 --2- 172.21.15.178:0/3918279748 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 0x7f49c409ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c259c640 1 --2- 172.21.15.178:0/3918279748 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 0x7f49c409e700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009880 tx=0x7f49b40090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=61f963046fc357e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c159a640 1 -- 172.21.15.178:0/3918279748 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f49b402ac50 con 0x7f49c40678d0 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c159a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c159a640 10 monclient: dump: 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.447 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c159a640 10 monclient: _finish_auth 0 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c159a640 1 -- 172.21.15.178:0/3918279748 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b402add0 con 0x7f49c40678d0 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c899d640 5 monclient: authenticate success, global_id 45130 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c899d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c159a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c899d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c159a640 1 -- 172.21.15.178:0/3918279748 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f49b4031ca0 con 0x7f49c40678d0 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c899d640 4 set_mon_vals no callback set 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.433+0000 7f49c159a640 10 monclient: dump: 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.449 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.450 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 10 monclient: shutdown 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 1 -- 172.21.15.178:0/3918279748 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 msgr2=0x7f49c409e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 1 --2- 172.21.15.178:0/3918279748 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 0x7f49c409e700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009880 tx=0x7f49b40090f0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 1 -- 172.21.15.178:0/3918279748 shutdown_connections 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 1 --2- 172.21.15.178:0/3918279748 >> v2:172.21.15.204:3300/0 conn(0x7f49c409f180 0x7f49c41306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 1 --2- 172.21.15.178:0/3918279748 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 0x7f49c409ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 1 --2- 172.21.15.178:0/3918279748 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 0x7f49c409e700 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 1 -- 172.21.15.178:0/3918279748 >> 172.21.15.178:0/3918279748 conn(0x7f49c40a4390 msgr2=0x7f49c4067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 1 -- 172.21.15.178:0/3918279748 shutdown_connections 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 1 -- 172.21.15.178:0/3918279748 wait complete. 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 10 monclient: build_initial_monmap 2024-09-15T01:50:04.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.434+0000 7f49c899d640 10 monclient: monmap: 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:04.452 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 1 Processor -- start 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 1 -- start start 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 10 monclient: init 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 10 monclient: _add_conns ranks=[4,8,7,6,1,5,2,3,0] 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 0x7f49c4133590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 10 monclient(hunting): picked mon.noname-c con 0x7f49c40678d0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 0x7f49c411d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 10 monclient(hunting): picked mon.noname-e con 0x7f49c406ea30 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f49c409f180 0x7f49c411d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 10 monclient(hunting): picked mon.noname-i con 0x7f49c409f180 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f49c41206b0 con 0x7f49c40678d0 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4140680 con 0x7f49c409f180 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4120d00 con 0x7f49c406ea30 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c899d640 10 monclient(hunting): authenticate will time out at 1809.185746s 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c1d9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 0x7f49c411d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c259c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 0x7f49c4133590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c1d9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 0x7f49c411d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40142/0 (socket says 172.21.15.178:40142) 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c259c640 10 monclient(hunting): get_auth_request con 0x7f49c40678d0 auth_method 0 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c1d9b640 1 -- 172.21.15.178:0/3992900305 learned_addr learned my addr 172.21.15.178:0/3992900305 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:04.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c259c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c1d9b640 10 monclient(hunting): get_auth_request con 0x7f49c406ea30 auth_method 0 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.435+0000 7f49c1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c259c640 10 monclient(hunting): handle_auth_done global_id 45139 payload 824 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c2d9d640 1 --2- 172.21.15.178:0/3992900305 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f49c409f180 0x7f49c411d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c259c640 1 -- 172.21.15.178:0/3992900305 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 msgr2=0x7f49c411d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c259c640 1 --2- 172.21.15.178:0/3992900305 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 0x7f49c411d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c259c640 1 -- 172.21.15.178:0/3992900305 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f49c409f180 msgr2=0x7f49c411d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c259c640 1 --2- 172.21.15.178:0/3992900305 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f49c409f180 0x7f49c411d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c259c640 10 monclient: _finish_hunting 0 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c259c640 1 monclient: found mon.noname-c 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c259c640 1 -- 172.21.15.178:0/3992900305 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b8009450 con 0x7f49c40678d0 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c2d9d640 10 monclient: get_auth_request con 0x7f49c409f180 auth_method 0 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c1d9b640 1 --2- 172.21.15.178:0/3992900305 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 0x7f49c411d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:04.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c2d9d640 1 --2- 172.21.15.178:0/3992900305 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f49c409f180 0x7f49c411d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c259c640 1 --2- 172.21.15.178:0/3992900305 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 0x7f49c4133590 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f49b4031000 tx=0x7f49b4032000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49b27fc640 1 -- 172.21.15.178:0/3992900305 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f49b402ac50 con 0x7f49c40678d0 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49b27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49b27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49b27fc640 10 monclient: dump: 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.457 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49b27fc640 10 monclient: _finish_auth 0 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49b27fc640 1 -- 172.21.15.178:0/3992900305 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b4033e00 con 0x7f49c40678d0 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c899d640 5 monclient: authenticate success, global_id 45139 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49b27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:04.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49b27fc640 1 -- 172.21.15.178:0/3992900305 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f49b403b870 con 0x7f49c40678d0 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49b27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49b27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49b27fc640 10 monclient: dump: 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.459 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c0d99640 4 set_mon_vals no callback set 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c899d640 10 monclient: _renew_subs 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c899d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c899d640 1 -- 172.21.15.178:0/3992900305 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b4009390 con 0x7f49c40678d0 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c899d640 10 monclient: _renew_subs 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c899d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:04.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.436+0000 7f49c899d640 1 -- 172.21.15.178:0/3992900305 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c406ccd0 con 0x7f49c40678d0 2024-09-15T01:50:04.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.437+0000 7f49b27fc640 1 -- 172.21.15.178:0/3992900305 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f49b4039040 con 0x7f49c40678d0 2024-09-15T01:50:04.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.437+0000 7f49c899d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:04.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.437+0000 7f49c899d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:04.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.437+0000 7f49c899d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:04.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.437+0000 7f49c899d640 1 -- 172.21.15.178:0/3992900305 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c404b720 con 0x7f49c40678d0 2024-09-15T01:50:04.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.437+0000 7f49b27fc640 1 --2- 172.21.15.178:0/3992900305 >> v2:172.21.15.204:6812/28210 conn(0x7f49a40397b0 0x7f49a403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.437+0000 7f49b27fc640 1 -- 172.21.15.178:0/3992900305 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f49b406b240 con 0x7f49c40678d0 2024-09-15T01:50:04.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.437+0000 7f49c1d9b640 1 --2- 172.21.15.178:0/3992900305 >> v2:172.21.15.204:6812/28210 conn(0x7f49a40397b0 0x7f49a403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.437+0000 7f49c1d9b640 10 monclient: get_auth_request con 0x7f49a40397b0 auth_method 0 2024-09-15T01:50:04.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.438+0000 7f49c1d9b640 1 --2- 172.21.15.178:0/3992900305 >> v2:172.21.15.204:6812/28210 conn(0x7f49a40397b0 0x7f49a403bc60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009fd0 tx=0x7f49b80091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.442+0000 7f49b27fc640 1 -- 172.21.15.178:0/3992900305 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f49b40446c0 con 0x7f49c40678d0 2024-09-15T01:50:04.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.442+0000 7f49b27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:04.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.442+0000 7f49b27fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:04.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.594+0000 7f49c899d640 1 -- 172.21.15.178:0/3992900305 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f49c405ed70 con 0x7f49a40397b0 2024-09-15T01:50:04.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.595+0000 7f49b27fc640 1 -- 172.21.15.178:0/3992900305 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f49c405ed70 con 0x7f49a40397b0 2024-09-15T01:50:04.597 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T01:50:04.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.596+0000 7f49c899d640 1 -- 172.21.15.178:0/3992900305 >> v2:172.21.15.204:6812/28210 conn(0x7f49a40397b0 msgr2=0x7f49a403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.596+0000 7f49c899d640 1 --2- 172.21.15.178:0/3992900305 >> v2:172.21.15.204:6812/28210 conn(0x7f49a40397b0 0x7f49a403bc60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009fd0 tx=0x7f49b80091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.597+0000 7f49c899d640 10 monclient: shutdown 2024-09-15T01:50:04.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.597+0000 7f49c899d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:04.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.597+0000 7f49c899d640 1 -- 172.21.15.178:0/3992900305 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 msgr2=0x7f49c4133590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.597+0000 7f49c899d640 1 --2- 172.21.15.178:0/3992900305 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 0x7f49c4133590 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f49b4031000 tx=0x7f49b4032000 comp rx=0 tx=0).stop 2024-09-15T01:50:04.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.597+0000 7f49c899d640 1 -- 172.21.15.178:0/3992900305 shutdown_connections 2024-09-15T01:50:04.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.597+0000 7f49c899d640 1 --2- 172.21.15.178:0/3992900305 >> v2:172.21.15.204:6812/28210 conn(0x7f49a40397b0 0x7f49a403bc60 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.597+0000 7f49c899d640 1 --2- 172.21.15.178:0/3992900305 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f49c409f180 0x7f49c411d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.597+0000 7f49c899d640 1 --2- 172.21.15.178:0/3992900305 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f49c406ea30 0x7f49c411d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.597+0000 7f49c899d640 1 --2- 172.21.15.178:0/3992900305 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f49c40678d0 0x7f49c4133590 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.597+0000 7f49c899d640 1 -- 172.21.15.178:0/3992900305 >> 172.21.15.178:0/3992900305 conn(0x7f49c40a4390 msgr2=0x7f49c409b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:04.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.597+0000 7f49c899d640 1 -- 172.21.15.178:0/3992900305 shutdown_connections 2024-09-15T01:50:04.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.597+0000 7f49c899d640 1 -- 172.21.15.178:0/3992900305 wait complete. 2024-09-15T01:50:04.609 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T01:50:04.609 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:50:04.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.704+0000 7f2811f1d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:04.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.704+0000 7f2811f1d640 10 monclient: build_initial_monmap 2024-09-15T01:50:04.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.704+0000 7f2811f1d640 10 monclient: monmap: 2024-09-15T01:50:04.706 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:04.706 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.706 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:04.705146+0000 2024-09-15T01:50:04.706 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:04.705146+0000 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:04.707 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:04.708 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.704+0000 7f2811f1d640 1 Processor -- start 2024-09-15T01:50:04.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 1 -- start start 2024-09-15T01:50:04.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 10 monclient: init 2024-09-15T01:50:04.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:04.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 10 monclient: _add_conns ranks=[2,1,3,7,0,6,5,9,10,4,8] 2024-09-15T01:50:04.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f280c130820 0x7f280c06e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 10 monclient(hunting): picked mon.noname-b con 0x7f280c130820 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:04.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f280c130bf0 addr v1:172.21.15.204:6789/0 2024-09-15T01:50:04.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c130fc0 0x7f280c09eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 10 monclient(hunting): picked mon.noname-g con 0x7f280c130fc0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280c04bf60 con 0x7f280c130bf0 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0a0100 con 0x7f280c130820 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c06d730 con 0x7f280c130fc0 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f2811f1d640 10 monclient(hunting): authenticate will time out at 1809.455564s 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f280c130820 0x7f280c06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f280c130820 0x7f280c06e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39614/0 (socket says 172.21.15.178:39614) 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 1 -- 172.21.15.178:0/3117798142 learned_addr learned my addr 172.21.15.178:0/3117798142 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 10 monclient(hunting): get_auth_request con 0x7f280c130820 auth_method 0 2024-09-15T01:50:04.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 10 monclient(hunting): handle_auth_done global_id 35543 payload 274 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 1 -- 172.21.15.178:0/3117798142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c130fc0 msgr2=0x7f280c09eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 1 --2- 172.21.15.178:0/3117798142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c130fc0 0x7f280c09eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 1 -- 172.21.15.178:0/3117798142 >> v1:172.21.15.204:6789/0 conn(0x7f280c130bf0 legacy=0x7f280c09e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 1 monclient: found mon.noname-b 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:04.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.705+0000 7f280b7fe640 1 -- 172.21.15.178:0/3117798142 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c130640 con 0x7f280c130820 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280b7fe640 1 --2- 172.21.15.178:0/3117798142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f280c130820 0x7f280c06e650 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009940 tx=0x7f27fc0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df83f050a020602d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280a7fc640 1 -- 172.21.15.178:0/3117798142 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f27fc02ac70 con 0x7f280c130820 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280a7fc640 10 monclient: dump: 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.719 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f2811f1d640 5 monclient: authenticate success, global_id 35543 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f2811f1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280a7fc640 1 -- 172.21.15.178:0/3117798142 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc02adf0 con 0x7f280c130820 2024-09-15T01:50:04.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f2811f1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280a7fc640 1 -- 172.21.15.178:0/3117798142 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f27fc031ca0 con 0x7f280c130820 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f2811f1d640 4 set_mon_vals no callback set 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.706+0000 7f280a7fc640 10 monclient: dump: 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.721 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.722 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.722 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.722 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.722 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.722 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.722 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.722 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.722 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.722 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.707+0000 7f2811f1d640 10 monclient: shutdown 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.707+0000 7f2811f1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.707+0000 7f2811f1d640 1 -- 172.21.15.178:0/3117798142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f280c130820 msgr2=0x7f280c06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.707+0000 7f2811f1d640 1 --2- 172.21.15.178:0/3117798142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f280c130820 0x7f280c06e650 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009940 tx=0x7f27fc0091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.707+0000 7f2811f1d640 1 -- 172.21.15.178:0/3117798142 shutdown_connections 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.707+0000 7f2811f1d640 1 --2- 172.21.15.178:0/3117798142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c130fc0 0x7f280c09eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.707+0000 7f2811f1d640 1 --2- 172.21.15.178:0/3117798142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f280c130820 0x7f280c06e650 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.707+0000 7f2811f1d640 1 -- 172.21.15.178:0/3117798142 >> 172.21.15.178:0/3117798142 conn(0x7f280c0a42a0 msgr2=0x7f280c0a0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.707+0000 7f2811f1d640 1 -- 172.21.15.178:0/3117798142 shutdown_connections 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.707+0000 7f2811f1d640 1 -- 172.21.15.178:0/3117798142 wait complete. 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.707+0000 7f2811f1d640 10 monclient: build_initial_monmap 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.707+0000 7f2811f1d640 10 monclient: monmap: 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:04.723 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:04.724 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 1 Processor -- start 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 1 -- start start 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 10 monclient: init 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 10 monclient: _add_conns ranks=[3,7,0,1,8,2,4,6,5] 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c130820 0x7f280c0999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 10 monclient(hunting): picked mon.noname-g con 0x7f280c130820 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f280c130bf0 0x7f280c12f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 10 monclient(hunting): picked mon.noname-i con 0x7f280c130bf0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f280c130fc0 0x7f280c1419a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 10 monclient(hunting): picked mon.noname-a con 0x7f280c130fc0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:50:04.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c069860 con 0x7f280c130fc0 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0693e0 con 0x7f280c130820 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f280c11ea80 con 0x7f280c130bf0 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f2811f1d640 10 monclient(hunting): authenticate will time out at 1809.458809s 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f280bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f280c130fc0 0x7f280c1419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f280bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f280c130fc0 0x7f280c1419a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35138/0 (socket says 172.21.15.178:35138) 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f280bfff640 1 -- 172.21.15.178:0/2391814278 learned_addr learned my addr 172.21.15.178:0/2391814278 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f280bfff640 10 monclient(hunting): get_auth_request con 0x7f280c130fc0 auth_method 0 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f280bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f280bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f280bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f280b7fe640 1 --2- 172.21.15.178:0/2391814278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c130820 0x7f280c0999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f280affd640 1 --2- 172.21.15.178:0/2391814278 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f280c130bf0 0x7f280c12f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.708+0000 7f280affd640 10 monclient(hunting): get_auth_request con 0x7f280c130bf0 auth_method 0 2024-09-15T01:50:04.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280b7fe640 10 monclient(hunting): get_auth_request con 0x7f280c130820 auth_method 0 2024-09-15T01:50:04.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 10 monclient(hunting): handle_auth_done global_id 104229 payload 824 2024-09-15T01:50:04.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 1 -- 172.21.15.178:0/2391814278 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f280c130bf0 msgr2=0x7f280c12f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 1 --2- 172.21.15.178:0/2391814278 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f280c130bf0 0x7f280c12f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 1 -- 172.21.15.178:0/2391814278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c130820 msgr2=0x7f280c0999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 1 --2- 172.21.15.178:0/2391814278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f280c130820 0x7f280c0999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:50:04.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 1 monclient: found mon.noname-a 2024-09-15T01:50:04.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:04.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:50:04.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 1 -- 172.21.15.178:0/2391814278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc009450 con 0x7f280c130fc0 2024-09-15T01:50:04.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f280bfff640 1 --2- 172.21.15.178:0/2391814278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f280c130fc0 0x7f280c1419a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f280400bc30 tx=0x7f2804009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f28037fe640 1 -- 172.21.15.178:0/2391814278 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28040092e0 con 0x7f280c130fc0 2024-09-15T01:50:04.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f28037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f28037fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:50:04.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f28037fe640 10 monclient: dump: 2024-09-15T01:50:04.729 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.729 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.730 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f28037fe640 10 monclient: _finish_auth 0 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f28037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f2811f1d640 5 monclient: authenticate success, global_id 104229 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f28037fe640 1 -- 172.21.15.178:0/2391814278 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280400cce0 con 0x7f280c130fc0 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f2811f1d640 10 monclient: _renew_subs 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f2811f1d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f2811f1d640 1 -- 172.21.15.178:0/2391814278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c11ed60 con 0x7f280c130fc0 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f28037fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f28037fe640 1 -- 172.21.15.178:0/2391814278 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28040158f0 con 0x7f280c130fc0 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f28037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f28037fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f28037fe640 10 monclient: dump: 2024-09-15T01:50:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.732 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f2809ffb640 4 set_mon_vals no callback set 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f2811f1d640 10 monclient: _renew_subs 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f2811f1d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.709+0000 7f2811f1d640 1 -- 172.21.15.178:0/2391814278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c11e710 con 0x7f280c130fc0 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f28037fe640 1 -- 172.21.15.178:0/2391814278 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2804012070 con 0x7f280c130fc0 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f2811f1d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f2811f1d640 1 -- 172.21.15.178:0/2391814278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f280c04c4a0 con 0x7f280c130fc0 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f28037fe640 1 --2- 172.21.15.178:0/2391814278 >> v2:172.21.15.204:6812/28210 conn(0x7f27ec0397b0 0x7f27ec03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f28037fe640 1 -- 172.21.15.178:0/2391814278 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f280404b730 con 0x7f280c130fc0 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f28037fe640 1 --2- 172.21.15.178:0/2391814278 >> v2:172.21.15.178:6804/29233 conn(0x7f27ec0422c0 0x7f27ec044750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f28037fe640 1 -- 172.21.15.178:0/2391814278 --> v2:172.21.15.178:6804/29233 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27ec044de0 con 0x7f27ec0422c0 2024-09-15T01:50:04.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f28037fe640 1 -- 172.21.15.178:0/2391814278 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=265) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2804015650 con 0x7f280c130fc0 2024-09-15T01:50:04.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f28037fe640 10 monclient: handle_get_version_reply finishing 1 version 265 2024-09-15T01:50:04.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f280b7fe640 1 --2- 172.21.15.178:0/2391814278 >> v2:172.21.15.204:6812/28210 conn(0x7f27ec0397b0 0x7f27ec03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f280b7fe640 10 monclient: get_auth_request con 0x7f27ec0397b0 auth_method 0 2024-09-15T01:50:04.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f280affd640 1 --2- 172.21.15.178:0/2391814278 >> v2:172.21.15.178:6804/29233 conn(0x7f27ec0422c0 0x7f27ec044750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.710+0000 7f280affd640 10 monclient: get_auth_request con 0x7f27ec0422c0 auth_method 0 2024-09-15T01:50:04.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.711+0000 7f280affd640 1 --2- 172.21.15.178:0/2391814278 >> v2:172.21.15.178:6804/29233 conn(0x7f27ec0422c0 0x7f27ec044750 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.711+0000 7f280b7fe640 1 --2- 172.21.15.178:0/2391814278 >> v2:172.21.15.204:6812/28210 conn(0x7f27ec0397b0 0x7f27ec03bc60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009420 tx=0x7f27fc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.711+0000 7f28037fe640 1 -- 172.21.15.178:0/2391814278 <== osd.1 v2:172.21.15.178:6804/29233 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f27ec044de0 con 0x7f27ec0422c0 2024-09-15T01:50:04.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.722+0000 7f2811f1d640 1 -- 172.21.15.178:0/2391814278 --> v2:172.21.15.178:6804/29233 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f280c04bc80 con 0x7f27ec0422c0 2024-09-15T01:50:04.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.722+0000 7f28037fe640 1 -- 172.21.15.178:0/2391814278 <== osd.1 v2:172.21.15.178:6804/29233 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f280c04bc80 con 0x7f27ec0422c0 2024-09-15T01:50:04.735 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:50:04.735 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:50:04.735 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.722+0000 7f2811f1d640 1 -- 172.21.15.178:0/2391814278 >> v2:172.21.15.178:6804/29233 conn(0x7f27ec0422c0 msgr2=0x7f27ec044750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.722+0000 7f2811f1d640 1 --2- 172.21.15.178:0/2391814278 >> v2:172.21.15.178:6804/29233 conn(0x7f27ec0422c0 0x7f27ec044750 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.722+0000 7f2811f1d640 1 -- 172.21.15.178:0/2391814278 >> v2:172.21.15.204:6812/28210 conn(0x7f27ec0397b0 msgr2=0x7f27ec03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.722+0000 7f2811f1d640 1 --2- 172.21.15.178:0/2391814278 >> v2:172.21.15.204:6812/28210 conn(0x7f27ec0397b0 0x7f27ec03bc60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009420 tx=0x7f27fc032000 comp rx=0 tx=0).stop 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.722+0000 7f2811f1d640 10 monclient: shutdown 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.722+0000 7f2811f1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.722+0000 7f2811f1d640 1 -- 172.21.15.178:0/2391814278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f280c130fc0 msgr2=0x7f280c1419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.722+0000 7f2811f1d640 1 --2- 172.21.15.178:0/2391814278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f280c130fc0 0x7f280c1419a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f280400bc30 tx=0x7f2804009840 comp rx=0 tx=0).stop 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.722+0000 7f280bfff640 1 -- 172.21.15.178:0/2391814278 reap_dead start 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.723+0000 7f2811f1d640 1 -- 172.21.15.178:0/2391814278 shutdown_connections 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.723+0000 7f2811f1d640 1 -- 172.21.15.178:0/2391814278 >> 172.21.15.178:0/2391814278 conn(0x7f280c0a42a0 msgr2=0x7f280c0679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.723+0000 7f2811f1d640 1 -- 172.21.15.178:0/2391814278 shutdown_connections 2024-09-15T01:50:04.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.723+0000 7f2811f1d640 1 -- 172.21.15.178:0/2391814278 wait complete. 2024-09-15T01:50:04.738 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T01:50:04.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.829+0000 7f8a1a11d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:04.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.829+0000 7f8a1a11d640 10 monclient: build_initial_monmap 2024-09-15T01:50:04.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.829+0000 7f8a1a11d640 10 monclient: monmap: 2024-09-15T01:50:04.832 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:04.832 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.832 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:04.830660+0000 2024-09-15T01:50:04.832 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:04.830660+0000 2024-09-15T01:50:04.832 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:04.832 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.832 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:04.832 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:04.832 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:04.833 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:04.833 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:04.833 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:04.833 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:04.833 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:04.833 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:04.833 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:04.833 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:04.833 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.830+0000 7f8a1a11d640 1 Processor -- start 2024-09-15T01:50:04.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.830+0000 7f8a1a11d640 1 -- start start 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.830+0000 7f8a1a11d640 10 monclient: init 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 10 monclient: _add_conns ranks=[2,6,5,8,0,1,4,7,3,10,9] 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1409c020 0x7f8a1409ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 10 monclient(hunting): picked mon.noname-b con 0x7f8a1409c020 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 0x7f8a14067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 10 monclient(hunting): picked mon.noname-d con 0x7f8a14067130 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8a14067a40 0x7f8a141393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 10 monclient(hunting): picked mon.noname-h con 0x7f8a14067a40 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1409e220 con 0x7f8a1409c020 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1409b270 con 0x7f8a14067a40 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1409af90 con 0x7f8a14067130 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a1a11d640 10 monclient(hunting): authenticate will time out at 1809.581465s 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a137fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1409c020 0x7f8a1409ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 0x7f8a14067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a137fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1409c020 0x7f8a1409ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39628/0 (socket says 172.21.15.178:39628) 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 0x7f8a14067500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:35486/0 (socket says 172.21.15.178:35486) 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a137fe640 1 -- 172.21.15.178:0/2624440650 learned_addr learned my addr 172.21.15.178:0/2624440650 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a137fe640 10 monclient(hunting): get_auth_request con 0x7f8a1409c020 auth_method 0 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a137fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 10 monclient(hunting): get_auth_request con 0x7f8a14067130 auth_method 0 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a13fff640 1 --2- 172.21.15.178:0/2624440650 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8a14067a40 0x7f8a141393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a13fff640 10 monclient(hunting): get_auth_request con 0x7f8a14067a40 auth_method 0 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a13fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a13fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a13fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a13fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a13fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a13fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 10 monclient(hunting): handle_auth_done global_id 25944 payload 274 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 1 -- 172.21.15.178:0/2624440650 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8a14067a40 msgr2=0x7f8a141393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 1 --2- 172.21.15.178:0/2624440650 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8a14067a40 0x7f8a141393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 1 -- 172.21.15.178:0/2624440650 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1409c020 msgr2=0x7f8a1409ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 1 --2- 172.21.15.178:0/2624440650 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1409c020 0x7f8a1409ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 1 monclient: found mon.noname-d 2024-09-15T01:50:04.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a12ffd640 1 -- 172.21.15.178:0/2624440650 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1404bf60 con 0x7f8a14067130 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.831+0000 7f8a137fe640 1 --2- 172.21.15.178:0/2624440650 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1409c020 0x7f8a1409ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a12ffd640 1 --2- 172.21.15.178:0/2624440650 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 0x7f8a14067500 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009880 tx=0x7f8a080090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e91bb3d78f0d040a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a127fc640 1 -- 172.21.15.178:0/2624440650 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a0802abb0 con 0x7f8a14067130 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a127fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a127fc640 10 monclient: dump: 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.847 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a127fc640 10 monclient: _finish_auth 0 2024-09-15T01:50:04.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a127fc640 1 -- 172.21.15.178:0/2624440650 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0802ad30 con 0x7f8a14067130 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a1a11d640 5 monclient: authenticate success, global_id 25944 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a1a11d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a127fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a1a11d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a127fc640 1 -- 172.21.15.178:0/2624440650 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a08031ca0 con 0x7f8a14067130 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a1a11d640 4 set_mon_vals no callback set 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a127fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.832+0000 7f8a127fc640 10 monclient: dump: 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.849 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 10 monclient: shutdown 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 1 -- 172.21.15.178:0/2624440650 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 msgr2=0x7f8a14067500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 1 --2- 172.21.15.178:0/2624440650 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 0x7f8a14067500 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009880 tx=0x7f8a080090f0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 1 -- 172.21.15.178:0/2624440650 shutdown_connections 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 1 --2- 172.21.15.178:0/2624440650 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f8a14067a40 0x7f8a141393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 1 --2- 172.21.15.178:0/2624440650 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 0x7f8a14067500 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 1 --2- 172.21.15.178:0/2624440650 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1409c020 0x7f8a1409ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 1 -- 172.21.15.178:0/2624440650 >> 172.21.15.178:0/2624440650 conn(0x7f8a14068a60 msgr2=0x7f8a140a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 1 -- 172.21.15.178:0/2624440650 shutdown_connections 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 1 -- 172.21.15.178:0/2624440650 wait complete. 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 10 monclient: build_initial_monmap 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.833+0000 7f8a1a11d640 10 monclient: monmap: 2024-09-15T01:50:04.851 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:04.852 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 1 Processor -- start 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 1 -- start start 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 10 monclient: init 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 10 monclient: _add_conns ranks=[6,2,8,1,7,0,3,4,5] 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 0x7f8a141324c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 10 monclient(hunting): picked mon.noname-d con 0x7f8a14067130 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a14067a40 0x7f8a1411bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 10 monclient(hunting): picked mon.noname-b con 0x7f8a14067a40 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a1409c020 0x7f8a1411c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 10 monclient(hunting): picked mon.noname-e con 0x7f8a1409c020 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:04.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1409f2d0 con 0x7f8a14067a40 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1411f8c0 con 0x7f8a14067130 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141427d0 con 0x7f8a1409c020 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.834+0000 7f8a1a11d640 10 monclient(hunting): authenticate will time out at 1809.584955s 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a137fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 0x7f8a141324c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a14067a40 0x7f8a1411bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a137fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 0x7f8a141324c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:35490/0 (socket says 172.21.15.178:35490) 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a14067a40 0x7f8a1411bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39636/0 (socket says 172.21.15.178:39636) 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a137fe640 1 -- 172.21.15.178:0/840440276 learned_addr learned my addr 172.21.15.178:0/840440276 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a137fe640 10 monclient(hunting): get_auth_request con 0x7f8a14067130 auth_method 0 2024-09-15T01:50:04.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a137fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a13fff640 1 --2- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a1409c020 0x7f8a1411c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 10 monclient(hunting): get_auth_request con 0x7f8a14067a40 auth_method 0 2024-09-15T01:50:04.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a13fff640 10 monclient(hunting): get_auth_request con 0x7f8a1409c020 auth_method 0 2024-09-15T01:50:04.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a13fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:04.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a13fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:04.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a13fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:04.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a13fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:04.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a13fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:04.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a13fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:04.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 10 monclient(hunting): handle_auth_done global_id 35561 payload 824 2024-09-15T01:50:04.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 1 -- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a1409c020 msgr2=0x7f8a1411c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 1 --2- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a1409c020 0x7f8a1411c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 1 -- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 msgr2=0x7f8a141324c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:04.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 1 --2- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 0x7f8a141324c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:04.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:50:04.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 1 monclient: found mon.noname-b 2024-09-15T01:50:04.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:04.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:50:04.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 1 -- 172.21.15.178:0/840440276 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a04009450 con 0x7f8a14067a40 2024-09-15T01:50:04.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a137fe640 1 --2- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 0x7f8a141324c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:04.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a13fff640 1 --2- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a1409c020 0x7f8a1411c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a12ffd640 1 --2- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a14067a40 0x7f8a1411bee0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8a08031000 tx=0x7f8a08032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f89fb7fe640 1 -- 172.21.15.178:0/840440276 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a0802aea0 con 0x7f8a14067a40 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f89fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f89fb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f89fb7fe640 10 monclient: dump: 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.858 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.859 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f89fb7fe640 10 monclient: _finish_auth 0 2024-09-15T01:50:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f89fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:04.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f89fb7fe640 1 -- 172.21.15.178:0/840440276 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0803b870 con 0x7f8a14067a40 2024-09-15T01:50:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a1a11d640 5 monclient: authenticate success, global_id 35561 2024-09-15T01:50:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a1a11d640 10 monclient: _renew_subs 2024-09-15T01:50:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a1a11d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a1a11d640 1 -- 172.21.15.178:0/840440276 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a08009390 con 0x7f8a14067a40 2024-09-15T01:50:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f89fb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f89fb7fe640 1 -- 172.21.15.178:0/840440276 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a0803be60 con 0x7f8a14067a40 2024-09-15T01:50:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f89fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f89fb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:50:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f89fb7fe640 10 monclient: dump: 2024-09-15T01:50:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:04.860 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:04.861 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.861 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:04.861 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:04.861 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:04.861 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:04.861 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:04.861 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:04.861 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:04.861 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:04.861 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:04.861 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:04.862 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:04.862 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:04.862 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:04.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a1a11d640 10 monclient: _renew_subs 2024-09-15T01:50:04.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a1a11d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:04.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a1a11d640 1 -- 172.21.15.178:0/840440276 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1411f5e0 con 0x7f8a14067a40 2024-09-15T01:50:04.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.835+0000 7f8a11ffb640 4 set_mon_vals no callback set 2024-09-15T01:50:04.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.836+0000 7f89fb7fe640 1 -- 172.21.15.178:0/840440276 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f8a08039040 con 0x7f8a14067a40 2024-09-15T01:50:04.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.836+0000 7f8a1a11d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:04.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.836+0000 7f8a1a11d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:04.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.836+0000 7f8a1a11d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:04.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.836+0000 7f8a1a11d640 1 -- 172.21.15.178:0/840440276 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1404b720 con 0x7f8a14067a40 2024-09-15T01:50:04.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.837+0000 7f89fb7fe640 1 --2- 172.21.15.178:0/840440276 >> v2:172.21.15.204:6812/28210 conn(0x7f8a000397b0 0x7f8a0003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:04.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.837+0000 7f89fb7fe640 1 -- 172.21.15.178:0/840440276 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f8a0806b0b0 con 0x7f8a14067a40 2024-09-15T01:50:04.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.837+0000 7f8a137fe640 1 --2- 172.21.15.178:0/840440276 >> v2:172.21.15.204:6812/28210 conn(0x7f8a000397b0 0x7f8a0003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:04.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.837+0000 7f8a137fe640 10 monclient: get_auth_request con 0x7f8a000397b0 auth_method 0 2024-09-15T01:50:04.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.837+0000 7f8a137fe640 1 --2- 172.21.15.178:0/840440276 >> v2:172.21.15.204:6812/28210 conn(0x7f8a000397b0 0x7f8a0003bc60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009fd0 tx=0x7f8a040091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:04.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.841+0000 7f89fb7fe640 1 -- 172.21.15.178:0/840440276 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8a08044900 con 0x7f8a14067a40 2024-09-15T01:50:04.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.841+0000 7f89fb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:04.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.841+0000 7f89fb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:04.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.995+0000 7f8a1a11d640 1 -- 172.21.15.178:0/840440276 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a1404c4a0 con 0x7f8a000397b0 2024-09-15T01:50:04.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.996+0000 7f89fb7fe640 1 -- 172.21.15.178:0/840440276 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8a1404c4a0 con 0x7f8a000397b0 2024-09-15T01:50:04.998 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T01:50:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.997+0000 7f8a1a11d640 1 -- 172.21.15.178:0/840440276 >> v2:172.21.15.204:6812/28210 conn(0x7f8a000397b0 msgr2=0x7f8a0003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.997+0000 7f8a1a11d640 1 --2- 172.21.15.178:0/840440276 >> v2:172.21.15.204:6812/28210 conn(0x7f8a000397b0 0x7f8a0003bc60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009fd0 tx=0x7f8a040091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.997+0000 7f8a1a11d640 10 monclient: shutdown 2024-09-15T01:50:05.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.998+0000 7f8a1a11d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:05.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.998+0000 7f8a1a11d640 1 -- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a14067a40 msgr2=0x7f8a1411bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.998+0000 7f8a1a11d640 1 --2- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a14067a40 0x7f8a1411bee0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8a08031000 tx=0x7f8a08032000 comp rx=0 tx=0).stop 2024-09-15T01:50:05.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.998+0000 7f8a1a11d640 1 -- 172.21.15.178:0/840440276 shutdown_connections 2024-09-15T01:50:05.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.998+0000 7f8a1a11d640 1 --2- 172.21.15.178:0/840440276 >> v2:172.21.15.204:6812/28210 conn(0x7f8a000397b0 0x7f8a0003bc60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.998+0000 7f8a1a11d640 1 --2- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f8a1409c020 0x7f8a1411c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.998+0000 7f8a1a11d640 1 --2- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a14067a40 0x7f8a1411bee0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.998+0000 7f8a1a11d640 1 --2- 172.21.15.178:0/840440276 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f8a14067130 0x7f8a141324c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.998+0000 7f8a1a11d640 1 -- 172.21.15.178:0/840440276 >> 172.21.15.178:0/840440276 conn(0x7f8a14068a60 msgr2=0x7f8a140a1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:05.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.998+0000 7f8a1a11d640 1 -- 172.21.15.178:0/840440276 shutdown_connections 2024-09-15T01:50:05.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:04.998+0000 7f8a1a11d640 1 -- 172.21.15.178:0/840440276 wait complete. 2024-09-15T01:50:05.010 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T01:50:05.010 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:50:05.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.104+0000 7fe214f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:05.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.104+0000 7fe214f9d640 10 monclient: build_initial_monmap 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.104+0000 7fe214f9d640 10 monclient: monmap: 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:05.105622+0000 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:05.105622+0000 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:05.107 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:05.108 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:05.108 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:05.108 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:05.108 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:05.108 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 1 Processor -- start 2024-09-15T01:50:05.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 1 -- start start 2024-09-15T01:50:05.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 10 monclient: init 2024-09-15T01:50:05.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:05.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 10 monclient: _add_conns ranks=[4,2,7,5,8,3,0,9,6,1,10] 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe21009f8c0 0x7fe210132dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 10 monclient(hunting): picked mon.noname-c con 0x7fe21009f8c0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe21009fc90 0x7fe2100a0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 10 monclient(hunting): picked mon.noname-b con 0x7fe21009fc90 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe2100a0060 0x7fe2100a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 10 monclient(hunting): picked mon.noname-i con 0x7fe2100a0060 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21009e420 con 0x7fe21009fc90 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe21009d320 con 0x7fe21009f8c0 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100a6d30 con 0x7fe2100a0060 2024-09-15T01:50:05.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.105+0000 7fe214f9d640 10 monclient(hunting): authenticate will time out at 1809.855987s 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20dd9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe21009fc90 0x7fe2100a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe21009f8c0 0x7fe210132dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20dd9b640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe21009fc90 0x7fe2100a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39650/0 (socket says 172.21.15.178:39650) 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe21009f8c0 0x7fe210132dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44850/0 (socket says 172.21.15.178:44850) 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20dd9b640 1 -- 172.21.15.178:0/868482596 learned_addr learned my addr 172.21.15.178:0/868482596 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20dd9b640 10 monclient(hunting): get_auth_request con 0x7fe21009fc90 auth_method 0 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 10 monclient(hunting): get_auth_request con 0x7fe21009f8c0 auth_method 0 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20ed9d640 1 --2- 172.21.15.178:0/868482596 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe2100a0060 0x7fe2100a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20ed9d640 10 monclient(hunting): get_auth_request con 0x7fe2100a0060 auth_method 0 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 10 monclient(hunting): handle_auth_done global_id 45148 payload 274 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 1 -- 172.21.15.178:0/868482596 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe2100a0060 msgr2=0x7fe2100a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 1 --2- 172.21.15.178:0/868482596 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe2100a0060 0x7fe2100a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 1 -- 172.21.15.178:0/868482596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe21009fc90 msgr2=0x7fe2100a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 1 --2- 172.21.15.178:0/868482596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe21009fc90 0x7fe2100a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 10 monclient: _finish_hunting 0 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 1 monclient: found mon.noname-c 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20e59c640 1 -- 172.21.15.178:0/868482596 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21004bf60 con 0x7fe21009f8c0 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.106+0000 7fe20dd9b640 1 --2- 172.21.15.178:0/868482596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe21009fc90 0x7fe2100a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:05.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20e59c640 1 --2- 172.21.15.178:0/868482596 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe21009f8c0 0x7fe210132dc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe2000098b0 tx=0x7fe2000090f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=e83fdff85216660d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20d59a640 1 -- 172.21.15.178:0/868482596 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe20002ab90 con 0x7fe21009f8c0 2024-09-15T01:50:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20d59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20d59a640 10 monclient: dump: 2024-09-15T01:50:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.122 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.123 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.123 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20d59a640 10 monclient: _finish_auth 0 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20d59a640 1 -- 172.21.15.178:0/868482596 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20002ad10 con 0x7fe21009f8c0 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 5 monclient: authenticate success, global_id 45148 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20d59a640 1 -- 172.21.15.178:0/868482596 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe200031ca0 con 0x7fe21009f8c0 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 4 set_mon_vals no callback set 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20d59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe20d59a640 10 monclient: dump: 2024-09-15T01:50:05.124 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.125 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.126 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 10 monclient: shutdown 2024-09-15T01:50:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 1 -- 172.21.15.178:0/868482596 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe21009f8c0 msgr2=0x7fe210132dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 1 --2- 172.21.15.178:0/868482596 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe21009f8c0 0x7fe210132dc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe2000098b0 tx=0x7fe2000090f0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 1 -- 172.21.15.178:0/868482596 shutdown_connections 2024-09-15T01:50:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 1 --2- 172.21.15.178:0/868482596 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe2100a0060 0x7fe2100a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 1 --2- 172.21.15.178:0/868482596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe21009fc90 0x7fe2100a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 1 --2- 172.21.15.178:0/868482596 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe21009f8c0 0x7fe210132dc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.107+0000 7fe214f9d640 1 -- 172.21.15.178:0/868482596 >> 172.21.15.178:0/868482596 conn(0x7fe2100a61e0 msgr2=0x7fe2100a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.108+0000 7fe214f9d640 1 -- 172.21.15.178:0/868482596 shutdown_connections 2024-09-15T01:50:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.108+0000 7fe214f9d640 1 -- 172.21.15.178:0/868482596 wait complete. 2024-09-15T01:50:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.108+0000 7fe214f9d640 10 monclient: build_initial_monmap 2024-09-15T01:50:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.108+0000 7fe214f9d640 10 monclient: monmap: 2024-09-15T01:50:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:05.128 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:05.129 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.108+0000 7fe214f9d640 1 Processor -- start 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.108+0000 7fe214f9d640 1 -- start start 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.108+0000 7fe214f9d640 10 monclient: init 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 10 monclient: _add_conns ranks=[7,8,3,5,6,2,1,0,4] 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe21009f8c0 0x7fe21012fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 10 monclient(hunting): picked mon.noname-i con 0x7fe21009f8c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe21009fc90 0x7fe21011c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 10 monclient(hunting): picked mon.noname-e con 0x7fe21009fc90 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2100a0060 0x7fe21011ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 10 monclient(hunting): picked mon.noname-g con 0x7fe2100a0060 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:05.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe210142980 con 0x7fe2100a0060 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe21009d560 con 0x7fe21009f8c0 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe21009ca90 con 0x7fe21009fc90 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe214f9d640 10 monclient(hunting): authenticate will time out at 1809.859373s 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe21009fc90 0x7fe21011c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20e59c640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe21009f8c0 0x7fe21012fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe21009fc90 0x7fe21011c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40168/0 (socket says 172.21.15.178:40168) 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 1 -- 172.21.15.178:0/462365376 learned_addr learned my addr 172.21.15.178:0/462365376 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 10 monclient(hunting): get_auth_request con 0x7fe21009fc90 auth_method 0 2024-09-15T01:50:05.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20e59c640 10 monclient(hunting): get_auth_request con 0x7fe21009f8c0 auth_method 0 2024-09-15T01:50:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20e59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20ed9d640 1 --2- 172.21.15.178:0/462365376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2100a0060 0x7fe21011ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20ed9d640 10 monclient(hunting): get_auth_request con 0x7fe2100a0060 auth_method 0 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 10 monclient(hunting): handle_auth_done global_id 94571 payload 824 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 1 -- 172.21.15.178:0/462365376 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe21009f8c0 msgr2=0x7fe21012fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 1 --2- 172.21.15.178:0/462365376 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe21009f8c0 0x7fe21012fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 1 -- 172.21.15.178:0/462365376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2100a0060 msgr2=0x7fe21011ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 1 --2- 172.21.15.178:0/462365376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2100a0060 0x7fe21011ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 10 monclient: _finish_hunting 0 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 1 monclient: found mon.noname-e 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 1 -- 172.21.15.178:0/462365376 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204009450 con 0x7fe21009fc90 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20e59c640 1 --2- 172.21.15.178:0/462365376 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fe21009f8c0 0x7fe21012fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.109+0000 7fe20dd9b640 1 --2- 172.21.15.178:0/462365376 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe21009fc90 0x7fe21011c920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe20400ec80 tx=0x7fe2040091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 1 -- 172.21.15.178:0/462365376 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe204009730 con 0x7fe21009fc90 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 10 monclient: dump: 2024-09-15T01:50:05.134 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.135 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.136 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 10 monclient: _finish_auth 0 2024-09-15T01:50:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 1 -- 172.21.15.178:0/462365376 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20400eec0 con 0x7fe21009fc90 2024-09-15T01:50:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 1 -- 172.21.15.178:0/462365376 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe204013d70 con 0x7fe21009fc90 2024-09-15T01:50:05.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe214f9d640 5 monclient: authenticate success, global_id 94571 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe20cd99640 4 set_mon_vals no callback set 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 10 monclient: dump: 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.137 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe214f9d640 10 monclient: _renew_subs 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe214f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe214f9d640 1 -- 172.21.15.178:0/462365376 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe200009390 con 0x7fe21009fc90 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe214f9d640 10 monclient: _renew_subs 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe214f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:05.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe214f9d640 1 -- 172.21.15.178:0/462365376 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe210120160 con 0x7fe21009fc90 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe1fe7fc640 1 -- 172.21.15.178:0/462365376 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe20400cb60 con 0x7fe21009fc90 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe214f9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.110+0000 7fe214f9d640 1 -- 172.21.15.178:0/462365376 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe21004c4a0 con 0x7fe21009fc90 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.111+0000 7fe1fe7fc640 1 --2- 172.21.15.178:0/462365376 >> v2:172.21.15.204:6812/28210 conn(0x7fe1f00397e0 0x7fe1f003bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.111+0000 7fe1fe7fc640 1 -- 172.21.15.178:0/462365376 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fe20404bfd0 con 0x7fe21009fc90 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.111+0000 7fe1fe7fc640 1 --2- 172.21.15.178:0/462365376 >> v2:172.21.15.178:6808/29234 conn(0x7fe1f0042180 0x7fe1f0044610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.111+0000 7fe1fe7fc640 1 -- 172.21.15.178:0/462365376 --> v2:172.21.15.178:6808/29234 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1f0044ca0 con 0x7fe1f0042180 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.111+0000 7fe1fe7fc640 1 -- 172.21.15.178:0/462365376 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_get_version_reply(handle=1 version=265) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe20404e660 con 0x7fe21009fc90 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.111+0000 7fe1fe7fc640 10 monclient: handle_get_version_reply finishing 1 version 265 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.111+0000 7fe20e59c640 1 --2- 172.21.15.178:0/462365376 >> v2:172.21.15.204:6812/28210 conn(0x7fe1f00397e0 0x7fe1f003bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.111+0000 7fe20e59c640 10 monclient: get_auth_request con 0x7fe1f00397e0 auth_method 0 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.111+0000 7fe20ed9d640 1 --2- 172.21.15.178:0/462365376 >> v2:172.21.15.178:6808/29234 conn(0x7fe1f0042180 0x7fe1f0044610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.111+0000 7fe20ed9d640 10 monclient: get_auth_request con 0x7fe1f0042180 auth_method 0 2024-09-15T01:50:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.111+0000 7fe20ed9d640 1 --2- 172.21.15.178:0/462365376 >> v2:172.21.15.178:6808/29234 conn(0x7fe1f0042180 0x7fe1f0044610 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.111+0000 7fe20e59c640 1 --2- 172.21.15.178:0/462365376 >> v2:172.21.15.204:6812/28210 conn(0x7fe1f00397e0 0x7fe1f003bc90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe200009360 tx=0x7fe200032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.112+0000 7fe1fe7fc640 1 -- 172.21.15.178:0/462365376 <== osd.2 v2:172.21.15.178:6808/29234 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fe1f0044ca0 con 0x7fe1f0042180 2024-09-15T01:50:05.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.122+0000 7fe214f9d640 1 -- 172.21.15.178:0/462365376 --> v2:172.21.15.178:6808/29234 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe21004bc80 con 0x7fe1f0042180 2024-09-15T01:50:05.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.122+0000 7fe1fe7fc640 1 -- 172.21.15.178:0/462365376 <== osd.2 v2:172.21.15.178:6808/29234 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe21004bc80 con 0x7fe1f0042180 2024-09-15T01:50:05.141 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:50:05.141 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:50:05.141 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:50:05.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe214f9d640 1 -- 172.21.15.178:0/462365376 >> v2:172.21.15.178:6808/29234 conn(0x7fe1f0042180 msgr2=0x7fe1f0044610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe214f9d640 1 --2- 172.21.15.178:0/462365376 >> v2:172.21.15.178:6808/29234 conn(0x7fe1f0042180 0x7fe1f0044610 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe214f9d640 1 -- 172.21.15.178:0/462365376 >> v2:172.21.15.204:6812/28210 conn(0x7fe1f00397e0 msgr2=0x7fe1f003bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe214f9d640 1 --2- 172.21.15.178:0/462365376 >> v2:172.21.15.204:6812/28210 conn(0x7fe1f00397e0 0x7fe1f003bc90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe200009360 tx=0x7fe200032000 comp rx=0 tx=0).stop 2024-09-15T01:50:05.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe214f9d640 10 monclient: shutdown 2024-09-15T01:50:05.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe214f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:05.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe214f9d640 1 -- 172.21.15.178:0/462365376 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe21009fc90 msgr2=0x7fe21011c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe214f9d640 1 --2- 172.21.15.178:0/462365376 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe21009fc90 0x7fe21011c920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe20400ec80 tx=0x7fe2040091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe20ed9d640 1 -- 172.21.15.178:0/462365376 reap_dead start 2024-09-15T01:50:05.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe214f9d640 1 -- 172.21.15.178:0/462365376 shutdown_connections 2024-09-15T01:50:05.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe214f9d640 1 -- 172.21.15.178:0/462365376 >> 172.21.15.178:0/462365376 conn(0x7fe2100a61e0 msgr2=0x7fe2100a2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:05.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe214f9d640 1 -- 172.21.15.178:0/462365376 shutdown_connections 2024-09-15T01:50:05.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.123+0000 7fe214f9d640 1 -- 172.21.15.178:0/462365376 wait complete. 2024-09-15T01:50:05.144 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T01:50:05.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.234+0000 7f2f67d9d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:05.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.234+0000 7f2f67d9d640 10 monclient: build_initial_monmap 2024-09-15T01:50:05.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.234+0000 7f2f67d9d640 10 monclient: monmap: 2024-09-15T01:50:05.237 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:05.237 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.237 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:05.235685+0000 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:05.235685+0000 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:05.238 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:05.239 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.235+0000 7f2f67d9d640 1 Processor -- start 2024-09-15T01:50:05.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.235+0000 7f2f67d9d640 1 -- start start 2024-09-15T01:50:05.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.235+0000 7f2f67d9d640 10 monclient: init 2024-09-15T01:50:05.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:05.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 10 monclient: _add_conns ranks=[0,10,1,3,7,6,9,5,8,2,4] 2024-09-15T01:50:05.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2f600a2930 addr v1:172.21.15.178:6789/0 2024-09-15T01:50:05.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2f60132de0 0x7f2f60137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 10 monclient(hunting): picked mon.noname-f con 0x7f2f60132de0 addr v2:172.21.15.204:3300/0 2024-09-15T01:50:05.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2f601380e0 addr v1:172.21.15.204:6789/0 2024-09-15T01:50:05.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6004bf60 con 0x7f2f600a2930 2024-09-15T01:50:05.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6009dd80 con 0x7f2f601380e0 2024-09-15T01:50:05.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f600a2d00 con 0x7f2f60132de0 2024-09-15T01:50:05.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:05.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 10 monclient(hunting): authenticate will time out at 1809.986150s 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f65b39640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2f600a2930 0x7f2f6009faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:37034/0 (socket says 172.21.15.178:37034) 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f65b39640 1 -- 172.21.15.178:0/602630310 learned_addr learned my addr 172.21.15.178:0/602630310 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f65338640 1 --2- 172.21.15.178:0/602630310 >> v2:172.21.15.204:3300/0 conn(0x7f2f60132de0 0x7f2f60137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600715838 0 0) 0x7f2f6004bf60 con 0x7f2f600a2930 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient(hunting): my global_id is 104238 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f480034e0 con 0x7f2f600a2930 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f65338640 10 monclient(hunting): get_auth_request con 0x7f2f60132de0 auth_method 0 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f65338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f65338640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f65338640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 <== mon.0 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2861057130 0 0) 0x7f2f480034e0 con 0x7f2f600a2930 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 >> v2:172.21.15.204:3300/0 conn(0x7f2f60132de0 msgr2=0x7f2f60137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 --2- 172.21.15.178:0/602630310 >> v2:172.21.15.204:3300/0 conn(0x7f2f60132de0 0x7f2f60137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 >> v1:172.21.15.204:6789/0 conn(0x7f2f601380e0 legacy=0x7f2f6013a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient: _finish_hunting 0 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 monclient: found mon.noname-a-legacy 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.178:6789/0 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f601331b0 con 0x7f2f600a2930 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient: _finish_auth 0 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 <== mon.0 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2029261281 0 0) 0x7f2f50001c70 con 0x7f2f600a2930 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 5 monclient: authenticate success, global_id 104238 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:05.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f65338640 1 --2- 172.21.15.178:0/602630310 >> v2:172.21.15.204:3300/0 conn(0x7f2f60132de0 0x7f2f60137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient: dump: 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.252 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] but i'm connected to v1:172.21.15.178:6789/0, reconnecting 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 >> v1:172.21.15.178:6789/0 conn(0x7f2f600a2930 legacy=0x7f2f6009faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient: _add_conns ranks=[3,8,7,4,6,1,2,5,0] 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f48004220 0x7f2f48004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient(hunting): picked mon.g con 0x7f2f48004220 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f48004de0 0x7f2f48005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient(hunting): picked mon.e con 0x7f2f48004de0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f48009cb0 0x7f2f4800c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient(hunting): picked mon.i con 0x7f2f48009cb0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600a2d00 con 0x7f2f48004220 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4800ca10 con 0x7f2f48009cb0 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4800cb80 con 0x7f2f48004de0 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f64b37640 10 monclient(hunting): _finish_auth 0 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f67d9d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.236+0000 7f2f65338640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f48004de0 0x7f2f48005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 10 monclient(hunting): get_auth_request con 0x7f2f48004de0 auth_method 0 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f6633a640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f48009cb0 0x7f2f4800c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65b39640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f48004220 0x7f2f48004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f6633a640 10 monclient(hunting): get_auth_request con 0x7f2f48009cb0 auth_method 0 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f6633a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f6633a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f6633a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65b39640 10 monclient(hunting): get_auth_request con 0x7f2f48004220 auth_method 0 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65b39640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T01:50:05.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f6633a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f6633a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f6633a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-15T01:50:05.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65b39640 1 -- 172.21.15.178:0/602630310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f48004220 msgr2=0x7f2f48004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T01:50:05.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65b39640 1 -- 172.21.15.178:0/602630310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f48004220 msgr2=0x7f2f48004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T01:50:05.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 10 monclient(hunting): handle_auth_done global_id 104238 payload 293 2024-09-15T01:50:05.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65b39640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f48004220 0x7f2f48004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T01:50:05.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 1 -- 172.21.15.178:0/602630310 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f48009cb0 msgr2=0x7f2f4800c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:05.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7f2f48009cb0 0x7f2f4800c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 1 -- 172.21.15.178:0/602630310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f48004220 msgr2=0x7f2f48004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65b39640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f48004220 0x7f2f48004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f48004220 0x7f2f48004780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 10 monclient: _finish_hunting 0 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 1 monclient: found mon.e 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f6633a640 1 -- 172.21.15.178:0/602630310 reap_dead start 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 1 -- 172.21.15.178:0/602630310 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480034e0 con 0x7f2f48004de0 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 10 monclient: _finish_auth 0 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.237+0000 7f2f65338640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f48004de0 0x7f2f48005250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400bbb0 tx=0x7f2f54008040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=b9ff6adb778643a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f5402aca0 con 0x7f2f48004de0 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f64b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f64b37640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f64b37640 10 monclient: dump: 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.258 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f67d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5402e070 con 0x7f2f48004de0 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f64b37640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f67d9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f64b37640 1 -- 172.21.15.178:0/602630310 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f5402ae40 con 0x7f2f48004de0 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f67d9d640 4 set_mon_vals no callback set 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f64b37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f64b37640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f64b37640 10 monclient: dump: 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.260 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f67d9d640 10 monclient: shutdown 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f67d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f67d9d640 1 -- 172.21.15.178:0/602630310 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f48004de0 msgr2=0x7f2f48005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f67d9d640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f48004de0 0x7f2f48005250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400bbb0 tx=0x7f2f54008040 comp rx=0 tx=0).stop 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f67d9d640 1 -- 172.21.15.178:0/602630310 shutdown_connections 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f67d9d640 1 --2- 172.21.15.178:0/602630310 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f2f48004de0 0x7f2f48005250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f67d9d640 1 -- 172.21.15.178:0/602630310 >> 172.21.15.178:0/602630310 conn(0x7f2f600a5ed0 msgr2=0x7f2f600a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f67d9d640 1 -- 172.21.15.178:0/602630310 shutdown_connections 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.238+0000 7f2f67d9d640 1 -- 172.21.15.178:0/602630310 wait complete. 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient: build_initial_monmap 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient: monmap: 2024-09-15T01:50:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:05.262 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 1 Processor -- start 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 1 -- start start 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient: init 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient: _add_conns ranks=[0,1,6,4,8,2,3,5,7] 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f48004de0 0x7f2f601417c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient(hunting): picked mon.noname-a con 0x7f2f48004de0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f6009c0c0 0x7f2f6009c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient(hunting): picked mon.noname-f con 0x7f2f6009c0c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2f60135a30 0x7f2f60135e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient(hunting): picked mon.noname-d con 0x7f2f60135a30 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:05.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6009c9d0 con 0x7f2f48004de0 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f601375f0 con 0x7f2f6009c0c0 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6009d630 con 0x7f2f60135a30 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f67d9d640 10 monclient(hunting): authenticate will time out at 1809.989899s 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.239+0000 7f2f65b39640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f48004de0 0x7f2f601417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f48004de0 0x7f2f601417c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35152/0 (socket says 172.21.15.178:35152) 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 1 -- 172.21.15.178:0/4238205540 learned_addr learned my addr 172.21.15.178:0/4238205540 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 10 monclient(hunting): get_auth_request con 0x7f2f48004de0 auth_method 0 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f6633a640 1 --2- 172.21.15.178:0/4238205540 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2f60135a30 0x7f2f60135e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f6633a640 10 monclient(hunting): get_auth_request con 0x7f2f60135a30 auth_method 0 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f6633a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f6633a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f6633a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f6633a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f6633a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f6633a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65338640 1 --2- 172.21.15.178:0/4238205540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f6009c0c0 0x7f2f6009c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65338640 10 monclient(hunting): get_auth_request con 0x7f2f6009c0c0 auth_method 0 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65338640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65338640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 10 monclient(hunting): handle_auth_done global_id 104247 payload 824 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 1 -- 172.21.15.178:0/4238205540 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2f60135a30 msgr2=0x7f2f60135e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 1 --2- 172.21.15.178:0/4238205540 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2f60135a30 0x7f2f60135e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 1 -- 172.21.15.178:0/4238205540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f6009c0c0 msgr2=0x7f2f6009c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 1 --2- 172.21.15.178:0/4238205540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f6009c0c0 0x7f2f6009c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 10 monclient: _finish_hunting 0 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 1 monclient: found mon.noname-a 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:50:05.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 1 -- 172.21.15.178:0/4238205540 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6009f870 con 0x7f2f48004de0 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f6633a640 1 --2- 172.21.15.178:0/4238205540 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2f60135a30 0x7f2f60135e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f65b39640 1 --2- 172.21.15.178:0/4238205540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f48004de0 0x7f2f601417c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f50007ae0 tx=0x7f2f50005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5dffb640 1 -- 172.21.15.178:0/4238205540 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f5000eaa0 con 0x7f2f48004de0 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5dffb640 10 monclient: dump: 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.267 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5dffb640 10 monclient: _finish_auth 0 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f67d9d640 5 monclient: authenticate success, global_id 104247 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5dffb640 1 -- 172.21.15.178:0/4238205540 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5000ec20 con 0x7f2f48004de0 2024-09-15T01:50:05.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f67d9d640 10 monclient: _renew_subs 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f67d9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f67d9d640 1 -- 172.21.15.178:0/4238205540 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f60096d80 con 0x7f2f48004de0 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5dffb640 1 -- 172.21.15.178:0/4238205540 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2f5000ed90 con 0x7f2f48004de0 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5dffb640 10 monclient: dump: 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.269 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f67d9d640 10 monclient: _renew_subs 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f67d9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f5ffff640 4 set_mon_vals no callback set 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.240+0000 7f2f67d9d640 1 -- 172.21.15.178:0/4238205540 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f601425d0 con 0x7f2f48004de0 2024-09-15T01:50:05.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.241+0000 7f2f5dffb640 1 -- 172.21.15.178:0/4238205540 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2f5000e620 con 0x7f2f48004de0 2024-09-15T01:50:05.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.241+0000 7f2f5dffb640 1 --2- 172.21.15.178:0/4238205540 >> v2:172.21.15.204:6812/28210 conn(0x7f2f4c0397b0 0x7f2f4c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.241+0000 7f2f5dffb640 1 -- 172.21.15.178:0/4238205540 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f2f5004b800 con 0x7f2f48004de0 2024-09-15T01:50:05.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.241+0000 7f2f67d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:05.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.241+0000 7f2f67d9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:05.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.241+0000 7f2f67d9d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-09-15T01:50:05.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.241+0000 7f2f67d9d640 1 -- 172.21.15.178:0/4238205540 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f60096d80 con 0x7f2f48004de0 2024-09-15T01:50:05.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.242+0000 7f2f65338640 1 --2- 172.21.15.178:0/4238205540 >> v2:172.21.15.204:6812/28210 conn(0x7f2f4c0397b0 0x7f2f4c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.242+0000 7f2f65338640 10 monclient: get_auth_request con 0x7f2f4c0397b0 auth_method 0 2024-09-15T01:50:05.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.242+0000 7f2f65338640 1 --2- 172.21.15.178:0/4238205540 >> v2:172.21.15.204:6812/28210 conn(0x7f2f4c0397b0 0x7f2f4c03bc60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2f54035000 tx=0x7f2f54036000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.246+0000 7f2f5dffb640 1 -- 172.21.15.178:0/4238205540 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2f60096d80 con 0x7f2f48004de0 2024-09-15T01:50:05.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.246+0000 7f2f5dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:05.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.246+0000 7f2f5dffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:05.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.401+0000 7f2f67d9d640 1 -- 172.21.15.178:0/4238205540 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f6005ed70 con 0x7f2f4c0397b0 2024-09-15T01:50:05.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.402+0000 7f2f5dffb640 1 -- 172.21.15.178:0/4238205540 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2f6005ed70 con 0x7f2f4c0397b0 2024-09-15T01:50:05.405 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T01:50:05.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 1 -- 172.21.15.178:0/4238205540 >> v2:172.21.15.204:6812/28210 conn(0x7f2f4c0397b0 msgr2=0x7f2f4c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 1 --2- 172.21.15.178:0/4238205540 >> v2:172.21.15.204:6812/28210 conn(0x7f2f4c0397b0 0x7f2f4c03bc60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2f54035000 tx=0x7f2f54036000 comp rx=0 tx=0).stop 2024-09-15T01:50:05.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 10 monclient: shutdown 2024-09-15T01:50:05.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:05.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 1 -- 172.21.15.178:0/4238205540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f48004de0 msgr2=0x7f2f601417c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 1 --2- 172.21.15.178:0/4238205540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f48004de0 0x7f2f601417c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f50007ae0 tx=0x7f2f50005bc0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 1 -- 172.21.15.178:0/4238205540 shutdown_connections 2024-09-15T01:50:05.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 1 --2- 172.21.15.178:0/4238205540 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f2f60135a30 0x7f2f60135e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 1 --2- 172.21.15.178:0/4238205540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f6009c0c0 0x7f2f6009c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 1 --2- 172.21.15.178:0/4238205540 >> v2:172.21.15.204:6812/28210 conn(0x7f2f4c0397b0 0x7f2f4c03bc60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 1 --2- 172.21.15.178:0/4238205540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f48004de0 0x7f2f601417c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 1 -- 172.21.15.178:0/4238205540 >> 172.21.15.178:0/4238205540 conn(0x7f2f600a5ed0 msgr2=0x7f2f600a49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:05.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 1 -- 172.21.15.178:0/4238205540 shutdown_connections 2024-09-15T01:50:05.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.404+0000 7f2f67d9d640 1 -- 172.21.15.178:0/4238205540 wait complete. 2024-09-15T01:50:05.417 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T01:50:05.417 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:50:05.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.511+0000 7f6181b1d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:05.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.511+0000 7f6181b1d640 10 monclient: build_initial_monmap 2024-09-15T01:50:05.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.511+0000 7f6181b1d640 10 monclient: monmap: 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:05.512411+0000 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:05.512411+0000 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:05.514 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:05.515 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:05.515 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:05.515 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:05.515 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:05.515 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 1 Processor -- start 2024-09-15T01:50:05.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 1 -- start start 2024-09-15T01:50:05.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 10 monclient: init 2024-09-15T01:50:05.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:05.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 10 monclient: _add_conns ranks=[3,6,2,1,5,7,9,8,4,10,0] 2024-09-15T01:50:05.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f617c09f8c0 0x7f617c132dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 10 monclient(hunting): picked mon.noname-g con 0x7f617c09f8c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:05.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f617c09fc90 0x7f617c0a0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 10 monclient(hunting): picked mon.noname-d con 0x7f617c09fc90 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617c0a0060 0x7f617c0a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 10 monclient(hunting): picked mon.noname-b con 0x7f617c0a0060 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617c09e420 con 0x7f617c0a0060 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617c09d320 con 0x7f617c09f8c0 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f617c0a6d30 con 0x7f617c09fc90 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f6181b1d640 10 monclient(hunting): authenticate will time out at 1810.262793s 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f617bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617c0a0060 0x7f617c0a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f617affd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f617c09fc90 0x7f617c0a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f617bfff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617c0a0060 0x7f617c0a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39664/0 (socket says 172.21.15.178:39664) 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f617bfff640 1 -- 172.21.15.178:0/970278315 learned_addr learned my addr 172.21.15.178:0/970278315 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f617bfff640 10 monclient(hunting): get_auth_request con 0x7f617c0a0060 auth_method 0 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f617bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f617bfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f617bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f617affd640 10 monclient(hunting): get_auth_request con 0x7f617c09fc90 auth_method 0 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f617affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f617affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.512+0000 7f617affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617b7fe640 1 --2- 172.21.15.178:0/970278315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f617c09f8c0 0x7f617c132dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617b7fe640 10 monclient(hunting): get_auth_request con 0x7f617c09f8c0 auth_method 0 2024-09-15T01:50:05.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 10 monclient(hunting): handle_auth_done global_id 35579 payload 274 2024-09-15T01:50:05.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 1 -- 172.21.15.178:0/970278315 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f617c09fc90 msgr2=0x7f617c0a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:05.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 1 --2- 172.21.15.178:0/970278315 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f617c09fc90 0x7f617c0a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 1 -- 172.21.15.178:0/970278315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f617c09f8c0 msgr2=0x7f617c132dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 1 --2- 172.21.15.178:0/970278315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f617c09f8c0 0x7f617c132dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 10 monclient: _finish_hunting 0 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 1 monclient: found mon.noname-b 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 1 -- 172.21.15.178:0/970278315 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c04bf60 con 0x7f617c0a0060 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617affd640 1 --2- 172.21.15.178:0/970278315 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f617c09fc90 0x7f617c0a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617b7fe640 1 --2- 172.21.15.178:0/970278315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f617c09f8c0 0x7f617c132dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.513+0000 7f617bfff640 1 --2- 172.21.15.178:0/970278315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617c0a0060 0x7f617c0a0d70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6174009f80 tx=0x7f6174009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=499729201fbe9555 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f617a7fc640 1 -- 172.21.15.178:0/970278315 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6174007880 con 0x7f617c0a0060 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f617a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f617a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f617a7fc640 10 monclient: dump: 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.528 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f617a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f617a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 5 monclient: authenticate success, global_id 35579 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f617a7fc640 1 -- 172.21.15.178:0/970278315 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617402d050 con 0x7f617c0a0060 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f617a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 4 set_mon_vals no callback set 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f617a7fc640 1 -- 172.21.15.178:0/970278315 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6174030360 con 0x7f617c0a0060 2024-09-15T01:50:05.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f617a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f617a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f617a7fc640 10 monclient: dump: 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.530 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 10 monclient: shutdown 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 1 -- 172.21.15.178:0/970278315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617c0a0060 msgr2=0x7f617c0a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 1 --2- 172.21.15.178:0/970278315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617c0a0060 0x7f617c0a0d70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6174009f80 tx=0x7f6174009840 comp rx=0 tx=0).stop 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 1 -- 172.21.15.178:0/970278315 shutdown_connections 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 1 --2- 172.21.15.178:0/970278315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617c0a0060 0x7f617c0a0d70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 1 --2- 172.21.15.178:0/970278315 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f617c09fc90 0x7f617c0a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 1 --2- 172.21.15.178:0/970278315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f617c09f8c0 0x7f617c132dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 1 -- 172.21.15.178:0/970278315 >> 172.21.15.178:0/970278315 conn(0x7f617c0a61e0 msgr2=0x7f617c0a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 1 -- 172.21.15.178:0/970278315 shutdown_connections 2024-09-15T01:50:05.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.514+0000 7f6181b1d640 1 -- 172.21.15.178:0/970278315 wait complete. 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.515+0000 7f6181b1d640 10 monclient: build_initial_monmap 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.515+0000 7f6181b1d640 10 monclient: monmap: 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:05.532 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.515+0000 7f6181b1d640 1 Processor -- start 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.515+0000 7f6181b1d640 1 -- start start 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.515+0000 7f6181b1d640 10 monclient: init 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient: _add_conns ranks=[0,2,3,1,5,8,6,7,4] 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617c09f8c0 0x7f617c09b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient(hunting): picked mon.noname-a con 0x7f617c09f8c0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617c09fc90 0x7f617c12f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient(hunting): picked mon.noname-b con 0x7f617c09fc90 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f617c0a0060 0x7f617c11c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient(hunting): picked mon.noname-g con 0x7f617c0a0060 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c09d530 con 0x7f617c09f8c0 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617c11ecb0 con 0x7f617c09fc90 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617c135120 con 0x7f617c0a0060 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient(hunting): authenticate will time out at 1810.266103s 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617c09f8c0 0x7f617c09b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617c09fc90 0x7f617c12f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617c09f8c0 0x7f617c09b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35158/0 (socket says 172.21.15.178:35158) 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617b7fe640 1 -- 172.21.15.178:0/2820114825 learned_addr learned my addr 172.21.15.178:0/2820114825 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617b7fe640 10 monclient(hunting): get_auth_request con 0x7f617c09f8c0 auth_method 0 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 10 monclient(hunting): get_auth_request con 0x7f617c09fc90 auth_method 0 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 10 monclient(hunting): handle_auth_done global_id 35588 payload 824 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617bfff640 1 --2- 172.21.15.178:0/2820114825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f617c0a0060 0x7f617c11c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 1 -- 172.21.15.178:0/2820114825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f617c0a0060 msgr2=0x7f617c11c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 1 --2- 172.21.15.178:0/2820114825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f617c0a0060 0x7f617c11c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 1 -- 172.21.15.178:0/2820114825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617c09f8c0 msgr2=0x7f617c09b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 1 --2- 172.21.15.178:0/2820114825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617c09f8c0 0x7f617c09b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 10 monclient: _finish_hunting 0 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 1 monclient: found mon.noname-b 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 1 -- 172.21.15.178:0/2820114825 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6170009390 con 0x7f617c09fc90 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617bfff640 10 monclient: get_auth_request con 0x7f617c0a0060 auth_method 0 2024-09-15T01:50:05.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617b7fe640 1 --2- 172.21.15.178:0/2820114825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617c09f8c0 0x7f617c09b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617bfff640 1 --2- 172.21.15.178:0/2820114825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f617c0a0060 0x7f617c11c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f617affd640 1 --2- 172.21.15.178:0/2820114825 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617c09fc90 0x7f617c12f800 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f617000ea80 tx=0x7f61700090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f61637fe640 1 -- 172.21.15.178:0/2820114825 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6170009670 con 0x7f617c09fc90 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f61637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f61637fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f61637fe640 10 monclient: dump: 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.537 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f61637fe640 10 monclient: _finish_auth 0 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f61637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f61637fe640 1 -- 172.21.15.178:0/2820114825 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617000ce90 con 0x7f617c09fc90 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 5 monclient: authenticate success, global_id 35588 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient: _renew_subs 2024-09-15T01:50:05.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 1 -- 172.21.15.178:0/2820114825 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6174009ae0 con 0x7f617c09fc90 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f61637fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f61637fe640 1 -- 172.21.15.178:0/2820114825 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61700138f0 con 0x7f617c09fc90 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f61637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f61637fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f61637fe640 10 monclient: dump: 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.539 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient: _renew_subs 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6181b1d640 1 -- 172.21.15.178:0/2820114825 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617c11d9e0 con 0x7f617c09fc90 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.516+0000 7f6179ffb640 4 set_mon_vals no callback set 2024-09-15T01:50:05.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.517+0000 7f6181b1d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.517+0000 7f6181b1d640 1 -- 172.21.15.178:0/2820114825 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f617c04c4a0 con 0x7f617c09fc90 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.517+0000 7f61637fe640 1 -- 172.21.15.178:0/2820114825 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f6170013470 con 0x7f617c09fc90 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.517+0000 7f61637fe640 1 --2- 172.21.15.178:0/2820114825 >> v2:172.21.15.204:6812/28210 conn(0x7f61680397e0 0x7f616803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.517+0000 7f61637fe640 1 -- 172.21.15.178:0/2820114825 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f617004bec0 con 0x7f617c09fc90 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.518+0000 7f61637fe640 1 --2- 172.21.15.178:0/2820114825 >> v2:172.21.15.204:6801/28752 conn(0x7f6168042180 0x7f6168044610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.518+0000 7f61637fe640 1 -- 172.21.15.178:0/2820114825 --> v2:172.21.15.204:6801/28752 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6168044ca0 con 0x7f6168042180 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.518+0000 7f61637fe640 1 -- 172.21.15.178:0/2820114825 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_get_version_reply(handle=1 version=265) v2 ==== 24+0+0 (secure 0 0 0) 0x7f617004e550 con 0x7f617c09fc90 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.518+0000 7f61637fe640 10 monclient: handle_get_version_reply finishing 1 version 265 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.518+0000 7f617b7fe640 1 --2- 172.21.15.178:0/2820114825 >> v2:172.21.15.204:6812/28210 conn(0x7f61680397e0 0x7f616803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.518+0000 7f617b7fe640 10 monclient: get_auth_request con 0x7f61680397e0 auth_method 0 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.518+0000 7f617bfff640 1 --2- 172.21.15.178:0/2820114825 >> v2:172.21.15.204:6801/28752 conn(0x7f6168042180 0x7f6168044610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.518+0000 7f617bfff640 10 monclient: get_auth_request con 0x7f6168042180 auth_method 0 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.518+0000 7f617b7fe640 1 --2- 172.21.15.178:0/2820114825 >> v2:172.21.15.204:6812/28210 conn(0x7f61680397e0 0x7f616803bc90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f616c010370 tx=0x7f616c008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.518+0000 7f617bfff640 1 --2- 172.21.15.178:0/2820114825 >> v2:172.21.15.204:6801/28752 conn(0x7f6168042180 0x7f6168044610 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.519+0000 7f61637fe640 1 -- 172.21.15.178:0/2820114825 <== osd.3 v2:172.21.15.204:6801/28752 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f6168044ca0 con 0x7f6168042180 2024-09-15T01:50:05.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.529+0000 7f6181b1d640 1 -- 172.21.15.178:0/2820114825 --> v2:172.21.15.204:6801/28752 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f617c04bc80 con 0x7f6168042180 2024-09-15T01:50:05.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.529+0000 7f61637fe640 1 -- 172.21.15.178:0/2820114825 <== osd.3 v2:172.21.15.204:6801/28752 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f617c04bc80 con 0x7f6168042180 2024-09-15T01:50:05.542 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:50:05.543 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:50:05.543 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:50:05.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f6181b1d640 1 -- 172.21.15.178:0/2820114825 >> v2:172.21.15.204:6801/28752 conn(0x7f6168042180 msgr2=0x7f6168044610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f6181b1d640 1 --2- 172.21.15.178:0/2820114825 >> v2:172.21.15.204:6801/28752 conn(0x7f6168042180 0x7f6168044610 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f6181b1d640 1 -- 172.21.15.178:0/2820114825 >> v2:172.21.15.204:6812/28210 conn(0x7f61680397e0 msgr2=0x7f616803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f6181b1d640 1 --2- 172.21.15.178:0/2820114825 >> v2:172.21.15.204:6812/28210 conn(0x7f61680397e0 0x7f616803bc90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f616c010370 tx=0x7f616c008040 comp rx=0 tx=0).stop 2024-09-15T01:50:05.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f6181b1d640 10 monclient: shutdown 2024-09-15T01:50:05.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f6181b1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:05.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f6181b1d640 1 -- 172.21.15.178:0/2820114825 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617c09fc90 msgr2=0x7f617c12f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f6181b1d640 1 --2- 172.21.15.178:0/2820114825 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617c09fc90 0x7f617c12f800 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f617000ea80 tx=0x7f61700090f0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f617bfff640 1 -- 172.21.15.178:0/2820114825 reap_dead start 2024-09-15T01:50:05.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f6181b1d640 1 -- 172.21.15.178:0/2820114825 shutdown_connections 2024-09-15T01:50:05.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f6181b1d640 1 -- 172.21.15.178:0/2820114825 >> 172.21.15.178:0/2820114825 conn(0x7f617c0a61e0 msgr2=0x7f617c131090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:05.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f6181b1d640 1 -- 172.21.15.178:0/2820114825 shutdown_connections 2024-09-15T01:50:05.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.530+0000 7f6181b1d640 1 -- 172.21.15.178:0/2820114825 wait complete. 2024-09-15T01:50:05.545 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T01:50:05.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.636+0000 7fe507c34640 10 monclient: get_monmap_and_config 2024-09-15T01:50:05.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.636+0000 7fe507c34640 10 monclient: build_initial_monmap 2024-09-15T01:50:05.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.636+0000 7fe507c34640 10 monclient: monmap: 2024-09-15T01:50:05.638 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:05.637170+0000 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:05.637170+0000 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:05.639 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:05.640 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:05.640 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:05.640 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.636+0000 7fe507c34640 1 Processor -- start 2024-09-15T01:50:05.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 1 -- start start 2024-09-15T01:50:05.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 10 monclient: init 2024-09-15T01:50:05.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:05.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 10 monclient: _add_conns ranks=[6,4,1,9,8,2,7,3,5,10,0] 2024-09-15T01:50:05.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe5000a2930 0x7fe500132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 10 monclient(hunting): picked mon.noname-d con 0x7fe5000a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:05.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe50009faa0 0x7fe50009fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 10 monclient(hunting): picked mon.noname-c con 0x7fe50009faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:05.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe5000a08c0 addr v1:172.21.15.204:6789/0 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe50004bf60 con 0x7fe5000a08c0 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe50009e0f0 con 0x7fe50009faa0 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe50009dd80 con 0x7fe5000a2930 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe507c34640 10 monclient(hunting): authenticate will time out at 1810.387563s 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5051cf640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe50009faa0 0x7fe50009fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5051cf640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe50009faa0 0x7fe50009fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44854/0 (socket says 172.21.15.178:44854) 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5059d0640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe5000a2930 0x7fe500132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5051cf640 1 -- 172.21.15.178:0/3150635131 learned_addr learned my addr 172.21.15.178:0/3150635131 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5051cf640 10 monclient(hunting): get_auth_request con 0x7fe50009faa0 auth_method 0 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5051cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5051cf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5051cf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5059d0640 10 monclient(hunting): get_auth_request con 0x7fe5000a2930 auth_method 0 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5059d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5059d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5059d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5059d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5059d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.637+0000 7fe5059d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5051cf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5051cf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5051cf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5059d0640 10 monclient(hunting): handle_auth_done global_id 25980 payload 274 2024-09-15T01:50:05.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5059d0640 1 -- 172.21.15.178:0/3150635131 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe50009faa0 msgr2=0x7fe50009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5059d0640 1 --2- 172.21.15.178:0/3150635131 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe50009faa0 0x7fe50009fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5059d0640 1 -- 172.21.15.178:0/3150635131 >> v1:172.21.15.204:6789/0 conn(0x7fe5000a08c0 legacy=0x7fe5000a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5059d0640 10 monclient: _finish_hunting 0 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5059d0640 1 monclient: found mon.noname-d 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5059d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5059d0640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5059d0640 1 -- 172.21.15.178:0/3150635131 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5000a2d00 con 0x7fe5000a2930 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5059d0640 1 --2- 172.21.15.178:0/3150635131 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe5000a2930 0x7fe500132de0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009880 tx=0x7fe4fc0090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=53d1e92c2500474b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5049ce640 1 -- 172.21.15.178:0/3150635131 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4fc02ac50 con 0x7fe5000a2930 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5049ce640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5049ce640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.638+0000 7fe5049ce640 10 monclient: dump: 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.650 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.651 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe5049ce640 10 monclient: _finish_auth 0 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe5049ce640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe5049ce640 1 -- 172.21.15.178:0/3150635131 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4fc02add0 con 0x7fe5000a2930 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 5 monclient: authenticate success, global_id 25980 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe5049ce640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe5049ce640 1 -- 172.21.15.178:0/3150635131 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4fc031ca0 con 0x7fe5000a2930 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 4 set_mon_vals no callback set 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe5049ce640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe5049ce640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe5049ce640 10 monclient: dump: 2024-09-15T01:50:05.652 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.653 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 10 monclient: shutdown 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 1 -- 172.21.15.178:0/3150635131 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe5000a2930 msgr2=0x7fe500132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 1 --2- 172.21.15.178:0/3150635131 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe5000a2930 0x7fe500132de0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009880 tx=0x7fe4fc0090f0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 1 -- 172.21.15.178:0/3150635131 shutdown_connections 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 1 --2- 172.21.15.178:0/3150635131 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fe50009faa0 0x7fe50009fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 1 --2- 172.21.15.178:0/3150635131 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fe5000a2930 0x7fe500132de0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 1 -- 172.21.15.178:0/3150635131 >> 172.21.15.178:0/3150635131 conn(0x7fe5000a5ed0 msgr2=0x7fe5000a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 1 -- 172.21.15.178:0/3150635131 shutdown_connections 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.639+0000 7fe507c34640 1 -- 172.21.15.178:0/3150635131 wait complete. 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.640+0000 7fe507c34640 10 monclient: build_initial_monmap 2024-09-15T01:50:05.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.640+0000 7fe507c34640 10 monclient: monmap: 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:05.655 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.640+0000 7fe507c34640 1 Processor -- start 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.640+0000 7fe507c34640 1 -- start start 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.640+0000 7fe507c34640 10 monclient: init 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 10 monclient: _add_conns ranks=[1,8,3,2,7,4,6,0,5] 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe50009faa0 0x7fe50009b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 10 monclient(hunting): picked mon.noname-f con 0x7fe50009faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe5000a08c0 0x7fe500131890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 10 monclient(hunting): picked mon.noname-e con 0x7fe5000a08c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5000a2930 0x7fe50011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 10 monclient(hunting): picked mon.noname-g con 0x7fe5000a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:05.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50011ec90 con 0x7fe50009faa0 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe500142ba0 con 0x7fe5000a2930 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe50009d630 con 0x7fe5000a08c0 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 10 monclient(hunting): authenticate will time out at 1810.391147s 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe5000a08c0 0x7fe500131890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe5000a08c0 0x7fe500131890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40180/0 (socket says 172.21.15.178:40180) 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 1 -- 172.21.15.178:0/404930738 learned_addr learned my addr 172.21.15.178:0/404930738 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 10 monclient(hunting): get_auth_request con 0x7fe5000a08c0 auth_method 0 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5059d0640 1 --2- 172.21.15.178:0/404930738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe50009faa0 0x7fe50009b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5059d0640 10 monclient(hunting): get_auth_request con 0x7fe50009faa0 auth_method 0 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5061d1640 1 --2- 172.21.15.178:0/404930738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5000a2930 0x7fe50011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5059d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5059d0640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5059d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 10 monclient(hunting): handle_auth_done global_id 94580 payload 824 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 1 -- 172.21.15.178:0/404930738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5000a2930 msgr2=0x7fe50011c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 1 --2- 172.21.15.178:0/404930738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5000a2930 0x7fe50011c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 1 -- 172.21.15.178:0/404930738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe50009faa0 msgr2=0x7fe50009b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 1 --2- 172.21.15.178:0/404930738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe50009faa0 0x7fe50009b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 10 monclient: _finish_hunting 0 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 1 monclient: found mon.noname-e 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 1 -- 172.21.15.178:0/404930738 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc009390 con 0x7fe5000a08c0 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5061d1640 10 monclient: get_auth_request con 0x7fe5000a2930 auth_method 0 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5061d1640 1 --2- 172.21.15.178:0/404930738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5000a2930 0x7fe50011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe5051cf640 1 --2- 172.21.15.178:0/404930738 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe5000a08c0 0x7fe500131890 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000e880 tx=0x7fe4f00091d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f5ffb640 1 -- 172.21.15.178:0/404930738 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4f000ee70 con 0x7fe5000a08c0 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f5ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f5ffb640 10 monclient: dump: 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f5ffb640 10 monclient: _finish_auth 0 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 5 monclient: authenticate success, global_id 94580 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f5ffb640 1 -- 172.21.15.178:0/404930738 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f000cdc0 con 0x7fe5000a08c0 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 10 monclient: _renew_subs 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe507c34640 1 -- 172.21.15.178:0/404930738 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50011e9b0 con 0x7fe5000a08c0 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f5ffb640 1 -- 172.21.15.178:0/404930738 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4f00138f0 con 0x7fe5000a08c0 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f5ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f5ffb640 10 monclient: dump: 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.662 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.642+0000 7fe507c34640 10 monclient: _renew_subs 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.641+0000 7fe4f7fff640 4 set_mon_vals no callback set 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.642+0000 7fe507c34640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.642+0000 7fe507c34640 1 -- 172.21.15.178:0/404930738 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50011dbf0 con 0x7fe5000a08c0 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.642+0000 7fe4f5ffb640 1 -- 172.21.15.178:0/404930738 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fe4f0013470 con 0x7fe5000a08c0 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.643+0000 7fe4f5ffb640 1 --2- 172.21.15.178:0/404930738 >> v2:172.21.15.204:6812/28210 conn(0x7fe4e40397b0 0x7fe4e403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.643+0000 7fe4f5ffb640 1 -- 172.21.15.178:0/404930738 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fe4f004bd50 con 0x7fe5000a08c0 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.643+0000 7fe5059d0640 1 --2- 172.21.15.178:0/404930738 >> v2:172.21.15.204:6812/28210 conn(0x7fe4e40397b0 0x7fe4e403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.643+0000 7fe507c34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.643+0000 7fe5059d0640 10 monclient: get_auth_request con 0x7fe4e40397b0 auth_method 0 2024-09-15T01:50:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.643+0000 7fe507c34640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:05.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.643+0000 7fe507c34640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:05.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.643+0000 7fe507c34640 1 -- 172.21.15.178:0/404930738 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5000a2400 con 0x7fe5000a08c0 2024-09-15T01:50:05.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.643+0000 7fe5059d0640 1 --2- 172.21.15.178:0/404930738 >> v2:172.21.15.204:6812/28210 conn(0x7fe4e40397b0 0x7fe4e403bc60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc034000 tx=0x7fe4fc032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.647+0000 7fe4f5ffb640 1 -- 172.21.15.178:0/404930738 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe5000a2400 con 0x7fe5000a08c0 2024-09-15T01:50:05.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.647+0000 7fe4f5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:05.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.647+0000 7fe4f5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:05.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.803+0000 7fe507c34640 1 -- 172.21.15.178:0/404930738 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe50005ed70 con 0x7fe4e40397b0 2024-09-15T01:50:05.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.803+0000 7fe4f5ffb640 1 -- 172.21.15.178:0/404930738 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe50005ed70 con 0x7fe4e40397b0 2024-09-15T01:50:05.805 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T01:50:05.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 1 -- 172.21.15.178:0/404930738 >> v2:172.21.15.204:6812/28210 conn(0x7fe4e40397b0 msgr2=0x7fe4e403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 1 --2- 172.21.15.178:0/404930738 >> v2:172.21.15.204:6812/28210 conn(0x7fe4e40397b0 0x7fe4e403bc60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc034000 tx=0x7fe4fc032000 comp rx=0 tx=0).stop 2024-09-15T01:50:05.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 10 monclient: shutdown 2024-09-15T01:50:05.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:05.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 1 -- 172.21.15.178:0/404930738 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe5000a08c0 msgr2=0x7fe500131890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 1 --2- 172.21.15.178:0/404930738 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe5000a08c0 0x7fe500131890 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000e880 tx=0x7fe4f00091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 1 -- 172.21.15.178:0/404930738 shutdown_connections 2024-09-15T01:50:05.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 1 --2- 172.21.15.178:0/404930738 >> v2:172.21.15.204:6812/28210 conn(0x7fe4e40397b0 0x7fe4e403bc60 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 1 --2- 172.21.15.178:0/404930738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5000a2930 0x7fe50011c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 1 --2- 172.21.15.178:0/404930738 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fe5000a08c0 0x7fe500131890 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 1 --2- 172.21.15.178:0/404930738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe50009faa0 0x7fe50009b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 1 -- 172.21.15.178:0/404930738 >> 172.21.15.178:0/404930738 conn(0x7fe5000a5ed0 msgr2=0x7fe50009e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:05.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 1 -- 172.21.15.178:0/404930738 shutdown_connections 2024-09-15T01:50:05.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.805+0000 7fe507c34640 1 -- 172.21.15.178:0/404930738 wait complete. 2024-09-15T01:50:05.818 INFO:tasks.ceph:Scrubbing osd.4 2024-09-15T01:50:05.818 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:50:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.913+0000 7f9d55f5d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.913+0000 7f9d55f5d640 10 monclient: build_initial_monmap 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.913+0000 7f9d55f5d640 10 monclient: monmap: 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:05.914229+0000 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:05.914229+0000 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:05.917 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 1 Processor -- start 2024-09-15T01:50:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 1 -- start start 2024-09-15T01:50:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 10 monclient: init 2024-09-15T01:50:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 10 monclient: _add_conns ranks=[2,1,0,9,8,7,5,10,3,6,4] 2024-09-15T01:50:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d5009f8c0 0x7f9d50132dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 10 monclient(hunting): picked mon.noname-b con 0x7f9d5009f8c0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9d5009fc90 addr v1:172.21.15.204:6789/0 2024-09-15T01:50:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9d500a0060 addr v1:172.21.15.178:6789/0 2024-09-15T01:50:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d5004bf60 con 0x7f9d500a0060 2024-09-15T01:50:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d5009d320 con 0x7f9d5009fc90 2024-09-15T01:50:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5009e420 con 0x7f9d5009f8c0 2024-09-15T01:50:05.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d55f5d640 10 monclient(hunting): authenticate will time out at 1810.664697s 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d4f7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d5009f8c0 0x7f9d50132dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d4f7fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d5009f8c0 0x7f9d50132dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:39684/0 (socket says 172.21.15.178:39684) 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d4f7fe640 1 -- 172.21.15.178:0/2490686983 learned_addr learned my addr 172.21.15.178:0/2490686983 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d4f7fe640 10 monclient(hunting): get_auth_request con 0x7f9d5009f8c0 auth_method 0 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d4f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d4f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d4f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d4f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d4f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.914+0000 7f9d4f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4f7fe640 10 monclient(hunting): handle_auth_done global_id 35597 payload 274 2024-09-15T01:50:05.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4f7fe640 1 -- 172.21.15.178:0/2490686983 >> v1:172.21.15.204:6789/0 conn(0x7f9d5009fc90 legacy=0x7f9d500a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4f7fe640 1 -- 172.21.15.178:0/2490686983 >> v1:172.21.15.178:6789/0 conn(0x7f9d500a0060 legacy=0x7f9d500a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4f7fe640 10 monclient: _finish_hunting 0 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4f7fe640 1 monclient: found mon.noname-b 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 1 -- 172.21.15.178:0/2490686983 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744279341 0 0) 0x7f9d5004bf60 con 0x7f9d500a0060 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4f7fe640 1 -- 172.21.15.178:0/2490686983 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d500a6d30 con 0x7f9d5009f8c0 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4f7fe640 1 --2- 172.21.15.178:0/2490686983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d5009f8c0 0x7f9d50132dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9d400098c0 tx=0x7f9d400091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a824097d0b4d0c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 1 -- 172.21.15.178:0/2490686983 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9d4002ac10 con 0x7f9d5009f8c0 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:50:05.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 10 monclient: dump: 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.929 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 10 monclient: _finish_auth 0 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 1 -- 172.21.15.178:0/2490686983 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4002ad90 con 0x7f9d5009f8c0 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d55f5d640 5 monclient: authenticate success, global_id 35597 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d55f5d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d55f5d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 1 -- 172.21.15.178:0/2490686983 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9d40031ca0 con 0x7f9d5009f8c0 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d55f5d640 4 set_mon_vals no callback set 2024-09-15T01:50:05.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.915+0000 7f9d4e7fc640 10 monclient: dump: 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.931 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.916+0000 7f9d55f5d640 10 monclient: shutdown 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.916+0000 7f9d55f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.916+0000 7f9d55f5d640 1 -- 172.21.15.178:0/2490686983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d5009f8c0 msgr2=0x7f9d50132dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.916+0000 7f9d55f5d640 1 --2- 172.21.15.178:0/2490686983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d5009f8c0 0x7f9d50132dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9d400098c0 tx=0x7f9d400091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.916+0000 7f9d55f5d640 1 -- 172.21.15.178:0/2490686983 shutdown_connections 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.916+0000 7f9d55f5d640 1 --2- 172.21.15.178:0/2490686983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d5009f8c0 0x7f9d50132dc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.916+0000 7f9d55f5d640 1 -- 172.21.15.178:0/2490686983 >> 172.21.15.178:0/2490686983 conn(0x7f9d500a61e0 msgr2=0x7f9d500a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.916+0000 7f9d55f5d640 1 -- 172.21.15.178:0/2490686983 shutdown_connections 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.916+0000 7f9d55f5d640 1 -- 172.21.15.178:0/2490686983 wait complete. 2024-09-15T01:50:05.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.916+0000 7f9d55f5d640 10 monclient: build_initial_monmap 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.916+0000 7f9d55f5d640 10 monclient: monmap: 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:05.933 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 1 Processor -- start 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 1 -- start start 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 10 monclient: init 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 10 monclient: _add_conns ranks=[1,3,4,2,5,7,8,6,0] 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d5009f8c0 0x7f9d50130f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 10 monclient(hunting): picked mon.noname-f con 0x7f9d5009f8c0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d5009fc90 0x7f9d5009b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 10 monclient(hunting): picked mon.noname-g con 0x7f9d5009fc90 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d500a0060 0x7f9d5009b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 10 monclient(hunting): picked mon.noname-c con 0x7f9d500a0060 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d50131ca0 con 0x7f9d5009f8c0 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5009ca90 con 0x7f9d5009fc90 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5009d530 con 0x7f9d500a0060 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.917+0000 7f9d55f5d640 10 monclient(hunting): authenticate will time out at 1810.667989s 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d5009f8c0 0x7f9d50130f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d500a0060 0x7f9d5009b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d5009f8c0 0x7f9d50130f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:52798/0 (socket says 172.21.15.178:52798) 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4f7fe640 1 -- 172.21.15.178:0/4283708175 learned_addr learned my addr 172.21.15.178:0/4283708175 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 10 monclient(hunting): get_auth_request con 0x7f9d500a0060 auth_method 0 2024-09-15T01:50:05.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4f7fe640 10 monclient(hunting): get_auth_request con 0x7f9d5009f8c0 auth_method 0 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4effd640 1 --2- 172.21.15.178:0/4283708175 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d5009fc90 0x7f9d5009b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4effd640 10 monclient(hunting): get_auth_request con 0x7f9d5009fc90 auth_method 0 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4effd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 10 monclient(hunting): handle_auth_done global_id 45166 payload 824 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 1 -- 172.21.15.178:0/4283708175 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d5009fc90 msgr2=0x7f9d5009b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 1 --2- 172.21.15.178:0/4283708175 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d5009fc90 0x7f9d5009b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 1 -- 172.21.15.178:0/4283708175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d5009f8c0 msgr2=0x7f9d50130f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 1 --2- 172.21.15.178:0/4283708175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d5009f8c0 0x7f9d50130f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 10 monclient: _finish_hunting 0 2024-09-15T01:50:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 1 monclient: found mon.noname-c 2024-09-15T01:50:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 1 -- 172.21.15.178:0/4283708175 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d40009450 con 0x7f9d500a0060 2024-09-15T01:50:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4ffff640 1 --2- 172.21.15.178:0/4283708175 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d500a0060 0x7f9d5009b570 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800c810 tx=0x7f9d48001550 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d377fe640 1 -- 172.21.15.178:0/4283708175 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9d4800aac0 con 0x7f9d500a0060 2024-09-15T01:50:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d377fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d377fe640 10 monclient: dump: 2024-09-15T01:50:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.938 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d377fe640 10 monclient: _finish_auth 0 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d377fe640 1 -- 172.21.15.178:0/4283708175 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4800ac80 con 0x7f9d500a0060 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d55f5d640 5 monclient: authenticate success, global_id 45166 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d55f5d640 10 monclient: _renew_subs 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d55f5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d55f5d640 1 -- 172.21.15.178:0/4283708175 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5011e630 con 0x7f9d500a0060 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d377fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d377fe640 1 -- 172.21.15.178:0/4283708175 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9d4800adf0 con 0x7f9d500a0060 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d377fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d377fe640 10 monclient: dump: 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:05.940 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:05.941 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d55f5d640 10 monclient: _renew_subs 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d55f5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d55f5d640 1 -- 172.21.15.178:0/4283708175 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5011e070 con 0x7f9d500a0060 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.918+0000 7f9d4dffb640 4 set_mon_vals no callback set 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.919+0000 7f9d55f5d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.919+0000 7f9d55f5d640 1 -- 172.21.15.178:0/4283708175 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d5004c4a0 con 0x7f9d500a0060 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.919+0000 7f9d377fe640 1 -- 172.21.15.178:0/4283708175 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9d48003030 con 0x7f9d500a0060 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.919+0000 7f9d377fe640 1 --2- 172.21.15.178:0/4283708175 >> v2:172.21.15.204:6812/28210 conn(0x7f9d3c0397b0 0x7f9d3c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.919+0000 7f9d377fe640 1 -- 172.21.15.178:0/4283708175 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f9d4804ba10 con 0x7f9d500a0060 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.920+0000 7f9d377fe640 1 --2- 172.21.15.178:0/4283708175 >> v2:172.21.15.204:6800/28753 conn(0x7f9d3c0422c0 0x7f9d3c044750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.920+0000 7f9d377fe640 1 -- 172.21.15.178:0/4283708175 --> v2:172.21.15.204:6800/28753 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d3c044de0 con 0x7f9d3c0422c0 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.920+0000 7f9d377fe640 1 -- 172.21.15.178:0/4283708175 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_get_version_reply(handle=1 version=265) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d48050240 con 0x7f9d500a0060 2024-09-15T01:50:05.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.920+0000 7f9d377fe640 10 monclient: handle_get_version_reply finishing 1 version 265 2024-09-15T01:50:05.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.920+0000 7f9d4f7fe640 1 --2- 172.21.15.178:0/4283708175 >> v2:172.21.15.204:6812/28210 conn(0x7f9d3c0397b0 0x7f9d3c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.920+0000 7f9d4f7fe640 10 monclient: get_auth_request con 0x7f9d3c0397b0 auth_method 0 2024-09-15T01:50:05.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.920+0000 7f9d4effd640 1 --2- 172.21.15.178:0/4283708175 >> v2:172.21.15.204:6800/28753 conn(0x7f9d3c0422c0 0x7f9d3c044750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:05.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.920+0000 7f9d4effd640 10 monclient: get_auth_request con 0x7f9d3c0422c0 auth_method 0 2024-09-15T01:50:05.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.920+0000 7f9d4f7fe640 1 --2- 172.21.15.178:0/4283708175 >> v2:172.21.15.204:6812/28210 conn(0x7f9d3c0397b0 0x7f9d3c03bc60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9d40000c00 tx=0x7f9d40032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.920+0000 7f9d4effd640 1 --2- 172.21.15.178:0/4283708175 >> v2:172.21.15.204:6800/28753 conn(0x7f9d3c0422c0 0x7f9d3c044750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:05.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.921+0000 7f9d377fe640 1 -- 172.21.15.178:0/4283708175 <== osd.4 v2:172.21.15.204:6800/28753 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f9d3c044de0 con 0x7f9d3c0422c0 2024-09-15T01:50:05.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.931+0000 7f9d55f5d640 1 -- 172.21.15.178:0/4283708175 --> v2:172.21.15.204:6800/28753 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9d5004bc80 con 0x7f9d3c0422c0 2024-09-15T01:50:05.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d377fe640 1 -- 172.21.15.178:0/4283708175 <== osd.4 v2:172.21.15.204:6800/28753 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9d5004bc80 con 0x7f9d3c0422c0 2024-09-15T01:50:05.944 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:50:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:50:05.944 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:50:05.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d55f5d640 1 -- 172.21.15.178:0/4283708175 >> v2:172.21.15.204:6800/28753 conn(0x7f9d3c0422c0 msgr2=0x7f9d3c044750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d55f5d640 1 --2- 172.21.15.178:0/4283708175 >> v2:172.21.15.204:6800/28753 conn(0x7f9d3c0422c0 0x7f9d3c044750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:05.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d55f5d640 1 -- 172.21.15.178:0/4283708175 >> v2:172.21.15.204:6812/28210 conn(0x7f9d3c0397b0 msgr2=0x7f9d3c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d55f5d640 1 --2- 172.21.15.178:0/4283708175 >> v2:172.21.15.204:6812/28210 conn(0x7f9d3c0397b0 0x7f9d3c03bc60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9d40000c00 tx=0x7f9d40032000 comp rx=0 tx=0).stop 2024-09-15T01:50:05.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d55f5d640 10 monclient: shutdown 2024-09-15T01:50:05.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d55f5d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:05.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d55f5d640 1 -- 172.21.15.178:0/4283708175 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d500a0060 msgr2=0x7f9d5009b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:05.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d55f5d640 1 --2- 172.21.15.178:0/4283708175 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f9d500a0060 0x7f9d5009b570 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800c810 tx=0x7f9d48001550 comp rx=0 tx=0).stop 2024-09-15T01:50:05.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d4ffff640 1 -- 172.21.15.178:0/4283708175 reap_dead start 2024-09-15T01:50:05.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d55f5d640 1 -- 172.21.15.178:0/4283708175 shutdown_connections 2024-09-15T01:50:05.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d55f5d640 1 -- 172.21.15.178:0/4283708175 >> 172.21.15.178:0/4283708175 conn(0x7f9d500a61e0 msgr2=0x7f9d50135980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:05.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d55f5d640 1 -- 172.21.15.178:0/4283708175 shutdown_connections 2024-09-15T01:50:05.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:05.932+0000 7f9d55f5d640 1 -- 172.21.15.178:0/4283708175 wait complete. 2024-09-15T01:50:05.946 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-15T01:50:06.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.038+0000 7fc1b859d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:06.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.038+0000 7fc1b859d640 10 monclient: build_initial_monmap 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.038+0000 7fc1b859d640 10 monclient: monmap: 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:06.039068+0000 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:06.039068+0000 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:06.041 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:06.042 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:06.042 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:06.042 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:06.042 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:06.042 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:06.042 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.038+0000 7fc1b859d640 1 Processor -- start 2024-09-15T01:50:06.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.038+0000 7fc1b859d640 1 -- start start 2024-09-15T01:50:06.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.038+0000 7fc1b859d640 10 monclient: init 2024-09-15T01:50:06.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:06.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 10 monclient: _add_conns ranks=[10,3,6,4,0,5,8,9,2,1,7] 2024-09-15T01:50:06.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc1b00678d0 0x7fc1b009e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 10 monclient(hunting): picked mon.noname-f con 0x7fc1b00678d0 addr v2:172.21.15.204:3300/0 2024-09-15T01:50:06.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1b006ea30 0x7fc1b009ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 10 monclient(hunting): picked mon.noname-g con 0x7fc1b006ea30 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:06.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b009f180 0x7fc1b01306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 10 monclient(hunting): picked mon.noname-d con 0x7fc1b009f180 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:06.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b009f5c0 con 0x7fc1b00678d0 2024-09-15T01:50:06.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b006d5b0 con 0x7fc1b006ea30 2024-09-15T01:50:06.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0067ca0 con 0x7fc1b009f180 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b859d640 10 monclient(hunting): authenticate will time out at 1810.789509s 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b009f180 0x7fc1b01306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b009f180 0x7fc1b01306b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:35514/0 (socket says 172.21.15.178:35514) 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 1 -- 172.21.15.178:0/1954928071 learned_addr learned my addr 172.21.15.178:0/1954928071 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 10 monclient(hunting): get_auth_request con 0x7fc1b009f180 auth_method 0 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6339640 1 --2- 172.21.15.178:0/1954928071 >> v2:172.21.15.204:3300/0 conn(0x7fc1b00678d0 0x7fc1b009e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6339640 10 monclient(hunting): get_auth_request con 0x7fc1b00678d0 auth_method 0 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b5b38640 1 --2- 172.21.15.178:0/1954928071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1b006ea30 0x7fc1b009ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b5b38640 10 monclient(hunting): get_auth_request con 0x7fc1b006ea30 auth_method 0 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b5b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b5b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b5b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 10 monclient(hunting): handle_auth_done global_id 25989 payload 274 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 1 -- 172.21.15.178:0/1954928071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1b006ea30 msgr2=0x7fc1b009ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 1 --2- 172.21.15.178:0/1954928071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1b006ea30 0x7fc1b009ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 1 -- 172.21.15.178:0/1954928071 >> v2:172.21.15.204:3300/0 conn(0x7fc1b00678d0 msgr2=0x7fc1b009e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:06.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 1 --2- 172.21.15.178:0/1954928071 >> v2:172.21.15.204:3300/0 conn(0x7fc1b00678d0 0x7fc1b009e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 10 monclient: _finish_hunting 0 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 1 monclient: found mon.noname-d 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6b3a640 1 -- 172.21.15.178:0/1954928071 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b004bf60 con 0x7fc1b009f180 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.039+0000 7fc1b6339640 1 --2- 172.21.15.178:0/1954928071 >> v2:172.21.15.204:3300/0 conn(0x7fc1b00678d0 0x7fc1b009e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b6b3a640 1 --2- 172.21.15.178:0/1954928071 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b009f180 0x7fc1b01306b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009940 tx=0x7fc1a00091d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c337b528efd16ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b5337640 1 -- 172.21.15.178:0/1954928071 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1a002ad00 con 0x7fc1b009f180 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b5337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b5337640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b5337640 10 monclient: dump: 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.054 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.055 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b5337640 10 monclient: _finish_auth 0 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b5337640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b5337640 1 -- 172.21.15.178:0/1954928071 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a002ae80 con 0x7fc1b009f180 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b859d640 5 monclient: authenticate success, global_id 25989 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b859d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b5337640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b859d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b859d640 4 set_mon_vals no callback set 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b5337640 1 -- 172.21.15.178:0/1954928071 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1a0031ca0 con 0x7fc1b009f180 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b5337640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b5337640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.040+0000 7fc1b5337640 10 monclient: dump: 2024-09-15T01:50:06.056 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.057 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 10 monclient: shutdown 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 1 -- 172.21.15.178:0/1954928071 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b009f180 msgr2=0x7fc1b01306b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 1 --2- 172.21.15.178:0/1954928071 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b009f180 0x7fc1b01306b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009940 tx=0x7fc1a00091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 1 -- 172.21.15.178:0/1954928071 shutdown_connections 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 1 --2- 172.21.15.178:0/1954928071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1b006ea30 0x7fc1b009ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 1 --2- 172.21.15.178:0/1954928071 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b009f180 0x7fc1b01306b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 1 --2- 172.21.15.178:0/1954928071 >> v2:172.21.15.204:3300/0 conn(0x7fc1b00678d0 0x7fc1b009e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 1 -- 172.21.15.178:0/1954928071 >> 172.21.15.178:0/1954928071 conn(0x7fc1b00a4390 msgr2=0x7fc1b0067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 1 -- 172.21.15.178:0/1954928071 shutdown_connections 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 1 -- 172.21.15.178:0/1954928071 wait complete. 2024-09-15T01:50:06.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 10 monclient: build_initial_monmap 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.041+0000 7fc1b859d640 10 monclient: monmap: 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:06.059 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 1 Processor -- start 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 1 -- start start 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 10 monclient: init 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 10 monclient: _add_conns ranks=[6,4,7,0,3,8,1,5,2] 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b00678d0 0x7fc1b01336c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 10 monclient(hunting): picked mon.noname-d con 0x7fc1b00678d0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc1b006ea30 0x7fc1b011d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 10 monclient(hunting): picked mon.noname-c con 0x7fc1b006ea30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:06.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc1b009f180 0x7fc1b011d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 10 monclient(hunting): picked mon.noname-i con 0x7fc1b009f180 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b006ccd0 con 0x7fc1b006ea30 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b006c900 con 0x7fc1b00678d0 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0120ac0 con 0x7fc1b009f180 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b859d640 10 monclient(hunting): authenticate will time out at 1810.792552s 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b00678d0 0x7fc1b01336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b00678d0 0x7fc1b01336c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:35528/0 (socket says 172.21.15.178:35528) 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 1 -- 172.21.15.178:0/3483196925 learned_addr learned my addr 172.21.15.178:0/3483196925 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:06.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 10 monclient(hunting): get_auth_request con 0x7fc1b00678d0 auth_method 0 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b5b38640 1 --2- 172.21.15.178:0/3483196925 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc1b006ea30 0x7fc1b011d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b5b38640 10 monclient(hunting): get_auth_request con 0x7fc1b006ea30 auth_method 0 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b5b38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b5b38640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b5b38640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b5b38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b5b38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6b3a640 1 --2- 172.21.15.178:0/3483196925 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc1b009f180 0x7fc1b011d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b5b38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6b3a640 10 monclient(hunting): get_auth_request con 0x7fc1b009f180 auth_method 0 2024-09-15T01:50:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6b3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6b3a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6b3a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6b3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6b3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6b3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 10 monclient(hunting): handle_auth_done global_id 25998 payload 824 2024-09-15T01:50:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 1 -- 172.21.15.178:0/3483196925 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc1b009f180 msgr2=0x7fc1b011d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 1 --2- 172.21.15.178:0/3483196925 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc1b009f180 0x7fc1b011d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 1 -- 172.21.15.178:0/3483196925 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc1b006ea30 msgr2=0x7fc1b011d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 1 --2- 172.21.15.178:0/3483196925 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc1b006ea30 0x7fc1b011d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 10 monclient: _finish_hunting 0 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 1 monclient: found mon.noname-d 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b6339640 1 -- 172.21.15.178:0/3483196925 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a0009450 con 0x7fc1b00678d0 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.042+0000 7fc1b5b38640 1 --2- 172.21.15.178:0/3483196925 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc1b006ea30 0x7fc1b011d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1b6339640 1 --2- 172.21.15.178:0/3483196925 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b00678d0 0x7fc1b01336c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800d1d0 tx=0x7fc1a8009a90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1ae7fc640 1 -- 172.21.15.178:0/3483196925 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1a8014820 con 0x7fc1b00678d0 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1ae7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1ae7fc640 10 monclient: dump: 2024-09-15T01:50:06.064 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.065 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1ae7fc640 10 monclient: _finish_auth 0 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1b859d640 5 monclient: authenticate success, global_id 25998 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1ae7fc640 1 -- 172.21.15.178:0/3483196925 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a800f040 con 0x7fc1b00678d0 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1ae7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1ae7fc640 1 -- 172.21.15.178:0/3483196925 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1a8014e00 con 0x7fc1b00678d0 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1b859d640 10 monclient: _renew_subs 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1b859d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1b859d640 1 -- 172.21.15.178:0/3483196925 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b011ecb0 con 0x7fc1b00678d0 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1ae7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1b4b36640 4 set_mon_vals no callback set 2024-09-15T01:50:06.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1ae7fc640 10 monclient: dump: 2024-09-15T01:50:06.067 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.067 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.067 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.067 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.067 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.067 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.067 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.067 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.067 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.067 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.067 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.067 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1b859d640 10 monclient: _renew_subs 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1b859d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.043+0000 7fc1b859d640 1 -- 172.21.15.178:0/3483196925 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b0134bf0 con 0x7fc1b00678d0 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.044+0000 7fc1ae7fc640 1 -- 172.21.15.178:0/3483196925 <== mon.6 v2:172.21.15.178:3303/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fc1a8014980 con 0x7fc1b00678d0 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.044+0000 7fc1ae7fc640 1 --2- 172.21.15.178:0/3483196925 >> v2:172.21.15.204:6812/28210 conn(0x7fc19c0397b0 0x7fc19c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.044+0000 7fc1ae7fc640 1 -- 172.21.15.178:0/3483196925 <== mon.6 v2:172.21.15.178:3303/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fc1a804bfc0 con 0x7fc1b00678d0 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.044+0000 7fc1b859d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.044+0000 7fc1b859d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:06.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.044+0000 7fc1b859d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.178:3303/0 2024-09-15T01:50:06.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.044+0000 7fc1b859d640 1 -- 172.21.15.178:0/3483196925 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b0120c80 con 0x7fc1b00678d0 2024-09-15T01:50:06.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.044+0000 7fc1b5b38640 1 --2- 172.21.15.178:0/3483196925 >> v2:172.21.15.204:6812/28210 conn(0x7fc19c0397b0 0x7fc19c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.044+0000 7fc1b5b38640 10 monclient: get_auth_request con 0x7fc19c0397b0 auth_method 0 2024-09-15T01:50:06.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.045+0000 7fc1b5b38640 1 --2- 172.21.15.178:0/3483196925 >> v2:172.21.15.204:6812/28210 conn(0x7fc19c0397b0 0x7fc19c03bc60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4007fa0 tx=0x7fc1a4008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.048+0000 7fc1ae7fc640 1 -- 172.21.15.178:0/3483196925 <== mon.6 v2:172.21.15.178:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc1a801a020 con 0x7fc1b00678d0 2024-09-15T01:50:06.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.048+0000 7fc1ae7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:06.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.048+0000 7fc1ae7fc640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:06.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.203+0000 7fc1b859d640 1 -- 172.21.15.178:0/3483196925 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1b004bc80 con 0x7fc19c0397b0 2024-09-15T01:50:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.203+0000 7fc1ae7fc640 1 -- 172.21.15.178:0/3483196925 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc1b004bc80 con 0x7fc19c0397b0 2024-09-15T01:50:06.206 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 4 to deep-scrub 2024-09-15T01:50:06.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 1 -- 172.21.15.178:0/3483196925 >> v2:172.21.15.204:6812/28210 conn(0x7fc19c0397b0 msgr2=0x7fc19c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 1 --2- 172.21.15.178:0/3483196925 >> v2:172.21.15.204:6812/28210 conn(0x7fc19c0397b0 0x7fc19c03bc60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4007fa0 tx=0x7fc1a4008040 comp rx=0 tx=0).stop 2024-09-15T01:50:06.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 10 monclient: shutdown 2024-09-15T01:50:06.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:06.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 1 -- 172.21.15.178:0/3483196925 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b00678d0 msgr2=0x7fc1b01336c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 1 --2- 172.21.15.178:0/3483196925 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b00678d0 0x7fc1b01336c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800d1d0 tx=0x7fc1a8009a90 comp rx=0 tx=0).stop 2024-09-15T01:50:06.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 1 -- 172.21.15.178:0/3483196925 shutdown_connections 2024-09-15T01:50:06.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 1 --2- 172.21.15.178:0/3483196925 >> v2:172.21.15.204:6812/28210 conn(0x7fc19c0397b0 0x7fc19c03bc60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 1 --2- 172.21.15.178:0/3483196925 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fc1b009f180 0x7fc1b011d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 1 --2- 172.21.15.178:0/3483196925 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fc1b006ea30 0x7fc1b011d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 1 --2- 172.21.15.178:0/3483196925 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fc1b00678d0 0x7fc1b01336c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 1 -- 172.21.15.178:0/3483196925 >> 172.21.15.178:0/3483196925 conn(0x7fc1b00a4390 msgr2=0x7fc1b009cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:06.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 1 -- 172.21.15.178:0/3483196925 shutdown_connections 2024-09-15T01:50:06.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.206+0000 7fc1b859d640 1 -- 172.21.15.178:0/3483196925 wait complete. 2024-09-15T01:50:06.219 INFO:tasks.ceph:Scrubbing osd.5 2024-09-15T01:50:06.219 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-15T01:50:06.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.313+0000 7fdbf4a2b640 10 monclient: get_monmap_and_config 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.313+0000 7fdbf4a2b640 10 monclient: build_initial_monmap 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.313+0000 7fdbf4a2b640 10 monclient: monmap: 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:06.314722+0000 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:06.314722+0000 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:06.316 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:06.317 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:06.317 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:06.317 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:06.317 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:06.317 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:06.317 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:06.317 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.314+0000 7fdbf4a2b640 1 Processor -- start 2024-09-15T01:50:06.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.314+0000 7fdbf4a2b640 1 -- start start 2024-09-15T01:50:06.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.314+0000 7fdbf4a2b640 10 monclient: init 2024-09-15T01:50:06.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:06.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 10 monclient: _add_conns ranks=[7,8,9,6,10,3,0,1,2,4,5] 2024-09-15T01:50:06.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdbf009f8c0 0x7fdbf0132dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 10 monclient(hunting): picked mon.noname-i con 0x7fdbf009f8c0 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:50:06.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009fc90 0x7fdbf00a0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 10 monclient(hunting): picked mon.noname-e con 0x7fdbf009fc90 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:06.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdbf00a0060 0x7fdbf00a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 10 monclient(hunting): picked mon.noname-a con 0x7fdbf00a0060 addr v2:172.21.15.178:3300/0 2024-09-15T01:50:06.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf009e420 con 0x7fdbf00a0060 2024-09-15T01:50:06.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf009d320 con 0x7fdbf009f8c0 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf00a6d30 con 0x7fdbf009fc90 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbf4a2b640 10 monclient(hunting): authenticate will time out at 1811.065216s 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbedd9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009fc90 0x7fdbf00a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdbf00a0060 0x7fdbf00a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbedd9b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009fc90 0x7fdbf00a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40196/0 (socket says 172.21.15.178:40196) 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdbf00a0060 0x7fdbf00a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35172/0 (socket says 172.21.15.178:35172) 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbedd9b640 1 -- 172.21.15.178:0/37629011 learned_addr learned my addr 172.21.15.178:0/37629011 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbedd9b640 10 monclient(hunting): get_auth_request con 0x7fdbf009fc90 auth_method 0 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbedd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbedd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbedd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 10 monclient(hunting): get_auth_request con 0x7fdbf00a0060 auth_method 0 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbee59c640 1 --2- 172.21.15.178:0/37629011 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdbf009f8c0 0x7fdbf0132dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbedd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbedd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbedd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbee59c640 10 monclient(hunting): get_auth_request con 0x7fdbf009f8c0 auth_method 0 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbee59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbee59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbee59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbee59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 10 monclient(hunting): handle_auth_done global_id 104274 payload 274 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 1 -- 172.21.15.178:0/37629011 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009fc90 msgr2=0x7fdbf00a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 1 --2- 172.21.15.178:0/37629011 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009fc90 0x7fdbf00a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 1 -- 172.21.15.178:0/37629011 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdbf009f8c0 msgr2=0x7fdbf0132dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 1 --2- 172.21.15.178:0/37629011 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdbf009f8c0 0x7fdbf0132dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 10 monclient: _finish_hunting 0 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 1 monclient: found mon.noname-a 2024-09-15T01:50:06.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:06.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:50:06.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbeed9d640 1 -- 172.21.15.178:0/37629011 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf004bf60 con 0x7fdbf00a0060 2024-09-15T01:50:06.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.315+0000 7fdbedd9b640 1 --2- 172.21.15.178:0/37629011 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009fc90 0x7fdbf00a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:06.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbeed9d640 1 --2- 172.21.15.178:0/37629011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf00a0060 0x7fdbf00a0d70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009e90 tx=0x7fdbe8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11665504e51675f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbed59a640 1 -- 172.21.15.178:0/37629011 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdbe80079d0 con 0x7fdbf00a0060 2024-09-15T01:50:06.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbed59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbed59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:50:06.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbed59a640 10 monclient: dump: 2024-09-15T01:50:06.330 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.330 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.330 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.330 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbed59a640 10 monclient: _finish_auth 0 2024-09-15T01:50:06.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbed59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbed59a640 1 -- 172.21.15.178:0/37629011 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe802d050 con 0x7fdbf00a0060 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbf4a2b640 5 monclient: authenticate success, global_id 104274 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbf4a2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbed59a640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbed59a640 1 -- 172.21.15.178:0/37629011 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdbe8007b90 con 0x7fdbf00a0060 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbed59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbed59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbed59a640 10 monclient: dump: 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.332 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbf4a2b640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.316+0000 7fdbf4a2b640 4 set_mon_vals no callback set 2024-09-15T01:50:06.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 10 monclient: shutdown 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/37629011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf00a0060 msgr2=0x7fdbf00a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 1 --2- 172.21.15.178:0/37629011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf00a0060 0x7fdbf00a0d70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009e90 tx=0x7fdbe8009780 comp rx=0 tx=0).stop 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/37629011 shutdown_connections 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 1 --2- 172.21.15.178:0/37629011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbf00a0060 0x7fdbf00a0d70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 1 --2- 172.21.15.178:0/37629011 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009fc90 0x7fdbf00a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 1 --2- 172.21.15.178:0/37629011 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdbf009f8c0 0x7fdbf0132dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/37629011 >> 172.21.15.178:0/37629011 conn(0x7fdbf00a61e0 msgr2=0x7fdbf00a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/37629011 shutdown_connections 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/37629011 wait complete. 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 10 monclient: build_initial_monmap 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.317+0000 7fdbf4a2b640 10 monclient: monmap: 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:06.334 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:06.335 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 1 Processor -- start 2024-09-15T01:50:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 1 -- start start 2024-09-15T01:50:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 10 monclient: init 2024-09-15T01:50:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 10 monclient: _add_conns ranks=[8,6,7,2,3,0,1,5,4] 2024-09-15T01:50:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009f8c0 0x7fdbf0130e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 10 monclient(hunting): picked mon.noname-e con 0x7fdbf009f8c0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdbf009fc90 0x7fdbf011d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 10 monclient(hunting): picked mon.noname-d con 0x7fdbf009fc90 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 1 --2- >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdbf00a0060 0x7fdbf011de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 10 monclient(hunting): picked mon.noname-i con 0x7fdbf00a0060 addr [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf009ca90 con 0x7fdbf009fc90 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 1 -- --> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf009d530 con 0x7fdbf00a0060 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf0142830 con 0x7fdbf009f8c0 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbf4a2b640 10 monclient(hunting): authenticate will time out at 1811.068639s 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009f8c0 0x7fdbf0130e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009f8c0 0x7fdbf0130e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:40200/0 (socket says 172.21.15.178:40200) 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 1 -- 172.21.15.178:0/2534218721 learned_addr learned my addr 172.21.15.178:0/2534218721 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 10 monclient(hunting): get_auth_request con 0x7fdbf009f8c0 auth_method 0 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbedd9b640 1 --2- 172.21.15.178:0/2534218721 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdbf009fc90 0x7fdbf011d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbedd9b640 10 monclient(hunting): get_auth_request con 0x7fdbf009fc90 auth_method 0 2024-09-15T01:50:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbedd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbedd9b640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbedd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbedd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbedd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbedd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbeed9d640 1 --2- 172.21.15.178:0/2534218721 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdbf00a0060 0x7fdbf011de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 10 monclient(hunting): handle_auth_done global_id 94598 payload 824 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 1 -- 172.21.15.178:0/2534218721 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdbf00a0060 msgr2=0x7fdbf011de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 1 --2- 172.21.15.178:0/2534218721 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdbf00a0060 0x7fdbf011de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 1 -- 172.21.15.178:0/2534218721 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdbf009fc90 msgr2=0x7fdbf011d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 1 --2- 172.21.15.178:0/2534218721 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fdbf009fc90 0x7fdbf011d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 10 monclient: _finish_hunting 0 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 1 monclient: found mon.noname-e 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.318+0000 7fdbee59c640 1 -- 172.21.15.178:0/2534218721 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4009450 con 0x7fdbf009f8c0 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbeed9d640 10 monclient: get_auth_request con 0x7fdbf00a0060 auth_method 0 2024-09-15T01:50:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbeed9d640 1 --2- 172.21.15.178:0/2534218721 >> [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] conn(0x7fdbf00a0060 0x7fdbf011de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbee59c640 1 --2- 172.21.15.178:0/2534218721 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009f8c0 0x7fdbf0130e40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000ea20 tx=0x7fdbe00090f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbde7fc640 1 -- 172.21.15.178:0/2534218721 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdbe000ce90 con 0x7fdbf009f8c0 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbde7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbde7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbde7fc640 10 monclient: dump: 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbde7fc640 10 monclient: _finish_auth 0 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbde7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbde7fc640 1 -- 172.21.15.178:0/2534218721 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe0009600 con 0x7fdbf009f8c0 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbf4a2b640 5 monclient: authenticate success, global_id 94598 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbf4a2b640 10 monclient: _renew_subs 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbf4a2b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:06.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/2534218721 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe8009a20 con 0x7fdbf009f8c0 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbde7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbde7fc640 1 -- 172.21.15.178:0/2534218721 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdbe00138f0 con 0x7fdbf009f8c0 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbde7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbde7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbde7fc640 10 monclient: dump: 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.342 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbecd99640 4 set_mon_vals no callback set 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbf4a2b640 10 monclient: _renew_subs 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbf4a2b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.319+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/2534218721 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf011f4d0 con 0x7fdbf009f8c0 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbf4a2b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/2534218721 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdbf004c4a0 con 0x7fdbf009f8c0 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbde7fc640 1 -- 172.21.15.178:0/2534218721 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdbe0013470 con 0x7fdbf009f8c0 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbde7fc640 1 --2- 172.21.15.178:0/2534218721 >> v2:172.21.15.204:6812/28210 conn(0x7fdbd00397b0 0x7fdbd003bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbde7fc640 1 -- 172.21.15.178:0/2534218721 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fdbe004bfb0 con 0x7fdbf009f8c0 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbde7fc640 1 --2- 172.21.15.178:0/2534218721 >> v2:172.21.15.204:6808/28754 conn(0x7fdbd0042150 0x7fdbd00445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbde7fc640 1 -- 172.21.15.178:0/2534218721 --> v2:172.21.15.204:6808/28754 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbd0044c70 con 0x7fdbd0042150 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbde7fc640 1 -- 172.21.15.178:0/2534218721 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_get_version_reply(handle=1 version=265) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbe004e640 con 0x7fdbf009f8c0 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbde7fc640 10 monclient: handle_get_version_reply finishing 1 version 265 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbedd9b640 1 --2- 172.21.15.178:0/2534218721 >> v2:172.21.15.204:6812/28210 conn(0x7fdbd00397b0 0x7fdbd003bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbedd9b640 10 monclient: get_auth_request con 0x7fdbd00397b0 auth_method 0 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbeed9d640 1 --2- 172.21.15.178:0/2534218721 >> v2:172.21.15.204:6808/28754 conn(0x7fdbd0042150 0x7fdbd00445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.320+0000 7fdbeed9d640 10 monclient: get_auth_request con 0x7fdbd0042150 auth_method 0 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.321+0000 7fdbedd9b640 1 --2- 172.21.15.178:0/2534218721 >> v2:172.21.15.204:6812/28210 conn(0x7fdbd00397b0 0x7fdbd003bc60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdbf011d890 tx=0x7fdbe40091d0 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.321+0000 7fdbeed9d640 1 --2- 172.21.15.178:0/2534218721 >> v2:172.21.15.204:6808/28754 conn(0x7fdbd0042150 0x7fdbd00445e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.321+0000 7fdbde7fc640 1 -- 172.21.15.178:0/2534218721 <== osd.5 v2:172.21.15.204:6808/28754 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fdbd0044c70 con 0x7fdbd0042150 2024-09-15T01:50:06.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.331+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/2534218721 --> v2:172.21.15.204:6808/28754 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdbf004bc80 con 0x7fdbd0042150 2024-09-15T01:50:06.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.332+0000 7fdbde7fc640 1 -- 172.21.15.178:0/2534218721 <== osd.5 v2:172.21.15.204:6808/28754 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fdbf004bc80 con 0x7fdbd0042150 2024-09-15T01:50:06.345 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-15T01:50:06.346 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T01:50:06.346 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-15T01:50:06.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.332+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/2534218721 >> v2:172.21.15.204:6808/28754 conn(0x7fdbd0042150 msgr2=0x7fdbd00445e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.332+0000 7fdbf4a2b640 1 --2- 172.21.15.178:0/2534218721 >> v2:172.21.15.204:6808/28754 conn(0x7fdbd0042150 0x7fdbd00445e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.332+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/2534218721 >> v2:172.21.15.204:6812/28210 conn(0x7fdbd00397b0 msgr2=0x7fdbd003bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.333+0000 7fdbf4a2b640 1 --2- 172.21.15.178:0/2534218721 >> v2:172.21.15.204:6812/28210 conn(0x7fdbd00397b0 0x7fdbd003bc60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdbf011d890 tx=0x7fdbe40091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.333+0000 7fdbf4a2b640 10 monclient: shutdown 2024-09-15T01:50:06.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.333+0000 7fdbf4a2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:06.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.333+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/2534218721 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009f8c0 msgr2=0x7fdbf0130e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.333+0000 7fdbf4a2b640 1 --2- 172.21.15.178:0/2534218721 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fdbf009f8c0 0x7fdbf0130e40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000ea20 tx=0x7fdbe00090f0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.333+0000 7fdbeed9d640 1 -- 172.21.15.178:0/2534218721 reap_dead start 2024-09-15T01:50:06.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.333+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/2534218721 shutdown_connections 2024-09-15T01:50:06.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.333+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/2534218721 >> 172.21.15.178:0/2534218721 conn(0x7fdbf00a61e0 msgr2=0x7fdbf00a2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:06.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.333+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/2534218721 shutdown_connections 2024-09-15T01:50:06.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.333+0000 7fdbf4a2b640 1 -- 172.21.15.178:0/2534218721 wait complete. 2024-09-15T01:50:06.348 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-15T01:50:06.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.439+0000 7fde02b9d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:06.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.439+0000 7fde02b9d640 10 monclient: build_initial_monmap 2024-09-15T01:50:06.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.439+0000 7fde02b9d640 10 monclient: monmap: 2024-09-15T01:50:06.441 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:06.441 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:06.440184+0000 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:06.440184+0000 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:06.442 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:06.443 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:06.443 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 1 Processor -- start 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 1 -- start start 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 10 monclient: init 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 10 monclient: _add_conns ranks=[10,6,5,0,1,7,2,3,4,9,8] 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fddfc0a2930 0x7fddfc132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 10 monclient(hunting): picked mon.noname-f con 0x7fddfc0a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fddfc09faa0 0x7fddfc09fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 10 monclient(hunting): picked mon.noname-d con 0x7fddfc09faa0 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddfc0a08c0 0x7fddfc0a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 10 monclient(hunting): picked mon.noname-h con 0x7fddfc0a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddfc09e0f0 con 0x7fddfc0a2930 2024-09-15T01:50:06.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc09dd80 con 0x7fddfc0a08c0 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc0a2d00 con 0x7fddfc09faa0 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde02b9d640 10 monclient(hunting): authenticate will time out at 1811.190604s 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fddfc09faa0 0x7fddfc09fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fddfc09faa0 0x7fddfc09fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:35536/0 (socket says 172.21.15.178:35536) 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 1 -- 172.21.15.178:0/74849692 learned_addr learned my addr 172.21.15.178:0/74849692 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 10 monclient(hunting): get_auth_request con 0x7fddfc09faa0 auth_method 0 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde00939640 1 --2- 172.21.15.178:0/74849692 >> v2:172.21.15.204:3300/0 conn(0x7fddfc0a2930 0x7fddfc132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fde0113a640 1 --2- 172.21.15.178:0/74849692 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddfc0a08c0 0x7fddfc0a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 10 monclient(hunting): handle_auth_done global_id 26016 payload 274 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 1 -- 172.21.15.178:0/74849692 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddfc0a08c0 msgr2=0x7fddfc0a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 1 --2- 172.21.15.178:0/74849692 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddfc0a08c0 0x7fddfc0a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.440+0000 7fddfbfff640 1 -- 172.21.15.178:0/74849692 >> v2:172.21.15.204:3300/0 conn(0x7fddfc0a2930 msgr2=0x7fddfc132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfbfff640 1 --2- 172.21.15.178:0/74849692 >> v2:172.21.15.204:3300/0 conn(0x7fddfc0a2930 0x7fddfc132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfbfff640 10 monclient: _finish_hunting 0 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfbfff640 1 monclient: found mon.noname-d 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfbfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:50:06.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfbfff640 1 -- 172.21.15.178:0/74849692 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc04bf60 con 0x7fddfc09faa0 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fde00939640 10 monclient: get_auth_request con 0x7fddfc0a2930 auth_method 0 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fde00939640 1 --2- 172.21.15.178:0/74849692 >> v2:172.21.15.204:3300/0 conn(0x7fddfc0a2930 0x7fddfc132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfbfff640 1 --2- 172.21.15.178:0/74849692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fddfc09faa0 0x7fddfc09fe70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fddec009f20 tx=0x7fddec009840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f821c25680384129 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfb7fe640 1 -- 172.21.15.178:0/74849692 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fddec007880 con 0x7fddfc09faa0 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfb7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfb7fe640 10 monclient: dump: 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.454 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfb7fe640 10 monclient: _finish_auth 0 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fde02b9d640 5 monclient: authenticate success, global_id 26016 2024-09-15T01:50:06.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfb7fe640 1 -- 172.21.15.178:0/74849692 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec02d050 con 0x7fddfc09faa0 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fde02b9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fde02b9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fde02b9d640 4 set_mon_vals no callback set 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfb7fe640 1 -- 172.21.15.178:0/74849692 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fddec030360 con 0x7fddfc09faa0 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfb7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.441+0000 7fddfb7fe640 10 monclient: dump: 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.456 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 10 monclient: shutdown 2024-09-15T01:50:06.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 1 -- 172.21.15.178:0/74849692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fddfc09faa0 msgr2=0x7fddfc09fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 1 --2- 172.21.15.178:0/74849692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fddfc09faa0 0x7fddfc09fe70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fddec009f20 tx=0x7fddec009840 comp rx=0 tx=0).stop 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 1 -- 172.21.15.178:0/74849692 shutdown_connections 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 1 --2- 172.21.15.178:0/74849692 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddfc0a08c0 0x7fddfc0a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 1 --2- 172.21.15.178:0/74849692 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7fddfc09faa0 0x7fddfc09fe70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 1 --2- 172.21.15.178:0/74849692 >> v2:172.21.15.204:3300/0 conn(0x7fddfc0a2930 0x7fddfc132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 1 -- 172.21.15.178:0/74849692 >> 172.21.15.178:0/74849692 conn(0x7fddfc0a5ed0 msgr2=0x7fddfc0a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 1 -- 172.21.15.178:0/74849692 shutdown_connections 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 1 -- 172.21.15.178:0/74849692 wait complete. 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 10 monclient: build_initial_monmap 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.442+0000 7fde02b9d640 10 monclient: monmap: 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:06.458 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 1 Processor -- start 2024-09-15T01:50:06.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 1 -- start start 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 10 monclient: init 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 10 monclient: _add_conns ranks=[1,3,5,2,6,0,7,8,4] 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddfc09faa0 0x7fddfc09ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 10 monclient(hunting): picked mon.noname-f con 0x7fddfc09faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddfc0a08c0 0x7fddfc1317d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 10 monclient(hunting): picked mon.noname-g con 0x7fddfc0a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddfc0a2930 0x7fddfc11c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 10 monclient(hunting): picked mon.noname-h con 0x7fddfc0a2930 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc11ecd0 con 0x7fddfc09faa0 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc11f200 con 0x7fddfc0a08c0 2024-09-15T01:50:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc0a2390 con 0x7fddfc0a2930 2024-09-15T01:50:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.443+0000 7fde02b9d640 10 monclient(hunting): authenticate will time out at 1811.193878s 2024-09-15T01:50:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde00939640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddfc09faa0 0x7fddfc09ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddfc0a2930 0x7fddfc11c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddfc0a08c0 0x7fddfc1317d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddfc0a2930 0x7fddfc11c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3302/0 says I am v2:172.21.15.178:44226/0 (socket says 172.21.15.178:44226) 2024-09-15T01:50:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde00939640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddfc09faa0 0x7fddfc09ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.178:52820/0 (socket says 172.21.15.178:52820) 2024-09-15T01:50:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 1 -- 172.21.15.178:0/2279305846 learned_addr learned my addr 172.21.15.178:0/2279305846 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 10 monclient(hunting): get_auth_request con 0x7fddfc0a2930 auth_method 0 2024-09-15T01:50:06.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde00939640 10 monclient(hunting): get_auth_request con 0x7fddfc09faa0 auth_method 0 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde00939640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde00939640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde00939640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddfbfff640 10 monclient(hunting): get_auth_request con 0x7fddfc0a08c0 auth_method 0 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddfbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddfbfff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddfbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddfbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddfbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddfbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde00939640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde00939640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde00939640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 10 monclient(hunting): handle_auth_done global_id 7943 payload 824 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 1 -- 172.21.15.178:0/2279305846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddfc0a08c0 msgr2=0x7fddfc1317d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 1 --2- 172.21.15.178:0/2279305846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddfc0a08c0 0x7fddfc1317d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 1 -- 172.21.15.178:0/2279305846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddfc09faa0 msgr2=0x7fddfc09ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 1 --2- 172.21.15.178:0/2279305846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddfc09faa0 0x7fddfc09ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 10 monclient: _finish_hunting 0 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 1 monclient: found mon.noname-h 2024-09-15T01:50:06.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.204:3302/0 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 1 -- 172.21.15.178:0/2279305846 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec009ae0 con 0x7fddfc0a2930 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde0113a640 1 --2- 172.21.15.178:0/2279305846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddfc0a2930 0x7fddfc11c530 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdde400c8e0 tx=0x7fdde4008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddf8ff9640 1 -- 172.21.15.178:0/2279305846 <== mon.5 v2:172.21.15.204:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdde4007aa0 con 0x7fddfc0a2930 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddf8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddf8ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddf8ff9640 10 monclient: dump: 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.464 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddf8ff9640 10 monclient: _finish_auth 0 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddf8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:06.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde02b9d640 5 monclient: authenticate success, global_id 7943 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddf8ff9640 1 -- 172.21.15.178:0/2279305846 <== mon.5 v2:172.21.15.204:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde4007c40 con 0x7fddfc0a2930 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde02b9d640 10 monclient: _renew_subs 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde02b9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde02b9d640 1 -- 172.21.15.178:0/2279305846 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc09b9b0 con 0x7fddfc0a2930 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddf8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddf8ff9640 1 -- 172.21.15.178:0/2279305846 <== mon.5 v2:172.21.15.204:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdde4007db0 con 0x7fddfc0a2930 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddf8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddf8ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddfaffd640 4 set_mon_vals no callback set 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fddf8ff9640 10 monclient: dump: 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.466 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde02b9d640 10 monclient: _renew_subs 2024-09-15T01:50:06.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde02b9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:50:06.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.444+0000 7fde02b9d640 1 -- 172.21.15.178:0/2279305846 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddfc142990 con 0x7fddfc0a2930 2024-09-15T01:50:06.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.445+0000 7fddf8ff9640 1 -- 172.21.15.178:0/2279305846 <== mon.5 v2:172.21.15.204:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fdde40075c0 con 0x7fddfc0a2930 2024-09-15T01:50:06.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.445+0000 7fde02b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:06.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.445+0000 7fde02b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:06.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.445+0000 7fde02b9d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.204:3302/0 2024-09-15T01:50:06.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.445+0000 7fde02b9d640 1 -- 172.21.15.178:0/2279305846 --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddfc04b720 con 0x7fddfc0a2930 2024-09-15T01:50:06.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.446+0000 7fddf8ff9640 1 --2- 172.21.15.178:0/2279305846 >> v2:172.21.15.204:6812/28210 conn(0x7fdddc0396e0 0x7fdddc03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.446+0000 7fddf8ff9640 1 -- 172.21.15.178:0/2279305846 <== mon.5 v2:172.21.15.204:3302/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fdde404c070 con 0x7fddfc0a2930 2024-09-15T01:50:06.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.446+0000 7fde00939640 1 --2- 172.21.15.178:0/2279305846 >> v2:172.21.15.204:6812/28210 conn(0x7fdddc0396e0 0x7fdddc03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.446+0000 7fde00939640 10 monclient: get_auth_request con 0x7fdddc0396e0 auth_method 0 2024-09-15T01:50:06.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.446+0000 7fde00939640 1 --2- 172.21.15.178:0/2279305846 >> v2:172.21.15.204:6812/28210 conn(0x7fdddc0396e0 0x7fdddc03bb90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fddf400acc0 tx=0x7fddf4008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.451+0000 7fddf8ff9640 1 -- 172.21.15.178:0/2279305846 <== mon.5 v2:172.21.15.204:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fdde407de40 con 0x7fddfc0a2930 2024-09-15T01:50:06.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.451+0000 7fddf8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:06.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.451+0000 7fddf8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:06.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.603+0000 7fde02b9d640 1 -- 172.21.15.178:0/2279305846 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fddfc04c4a0 con 0x7fdddc0396e0 2024-09-15T01:50:06.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.604+0000 7fddf8ff9640 1 -- 172.21.15.178:0/2279305846 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fddfc04c4a0 con 0x7fdddc0396e0 2024-09-15T01:50:06.607 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 5 to deep-scrub 2024-09-15T01:50:06.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 1 -- 172.21.15.178:0/2279305846 >> v2:172.21.15.204:6812/28210 conn(0x7fdddc0396e0 msgr2=0x7fdddc03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 1 --2- 172.21.15.178:0/2279305846 >> v2:172.21.15.204:6812/28210 conn(0x7fdddc0396e0 0x7fdddc03bb90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fddf400acc0 tx=0x7fddf4008040 comp rx=0 tx=0).stop 2024-09-15T01:50:06.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 10 monclient: shutdown 2024-09-15T01:50:06.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:06.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 1 -- 172.21.15.178:0/2279305846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddfc0a2930 msgr2=0x7fddfc11c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 1 --2- 172.21.15.178:0/2279305846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddfc0a2930 0x7fddfc11c530 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdde400c8e0 tx=0x7fdde4008040 comp rx=0 tx=0).stop 2024-09-15T01:50:06.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 1 -- 172.21.15.178:0/2279305846 shutdown_connections 2024-09-15T01:50:06.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 1 --2- 172.21.15.178:0/2279305846 >> v2:172.21.15.204:6812/28210 conn(0x7fdddc0396e0 0x7fdddc03bb90 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 1 --2- 172.21.15.178:0/2279305846 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7fddfc0a2930 0x7fddfc11c530 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 1 --2- 172.21.15.178:0/2279305846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddfc0a08c0 0x7fddfc1317d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 1 --2- 172.21.15.178:0/2279305846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddfc09faa0 0x7fddfc09ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 1 -- 172.21.15.178:0/2279305846 >> 172.21.15.178:0/2279305846 conn(0x7fddfc0a5ed0 msgr2=0x7fddfc135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:06.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 1 -- 172.21.15.178:0/2279305846 shutdown_connections 2024-09-15T01:50:06.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.606+0000 7fde02b9d640 1 -- 172.21.15.178:0/2279305846 wait complete. 2024-09-15T01:50:06.618 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:50:06.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.709+0000 7fb2594c0640 10 monclient: get_monmap_and_config 2024-09-15T01:50:06.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.709+0000 7fb2594c0640 10 monclient: build_initial_monmap 2024-09-15T01:50:06.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.709+0000 7fb2594c0640 10 monclient: monmap: 2024-09-15T01:50:06.711 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:06.711 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.711 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:06.709976+0000 2024-09-15T01:50:06.711 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:06.709976+0000 2024-09-15T01:50:06.711 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:06.711 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.711 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:06.712 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:06.712 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:06.712 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:06.712 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:06.712 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:06.712 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:06.712 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:06.712 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:06.712 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:06.712 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:06.712 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.709+0000 7fb2594c0640 1 Processor -- start 2024-09-15T01:50:06.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.709+0000 7fb2594c0640 1 -- start start 2024-09-15T01:50:06.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.709+0000 7fb2594c0640 10 monclient: init 2024-09-15T01:50:06.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:06.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 10 monclient: _add_conns ranks=[10,0,4,2,1,7,5,9,3,8,6] 2024-09-15T01:50:06.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb2540a2930 0x7fb254132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 10 monclient(hunting): picked mon.noname-f con 0x7fb2540a2930 addr v2:172.21.15.204:3300/0 2024-09-15T01:50:06.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb25409faa0 addr v1:172.21.15.178:6789/0 2024-09-15T01:50:06.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409fe70 0x7fb2540a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 10 monclient(hunting): picked mon.noname-c con 0x7fb25409fe70 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:06.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25404bf60 con 0x7fb25409faa0 2024-09-15T01:50:06.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25409dd80 con 0x7fb2540a2930 2024-09-15T01:50:06.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2540a2d00 con 0x7fb25409fe70 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2594c0640 10 monclient(hunting): authenticate will time out at 1811.460269s 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409fe70 0x7fb2540a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2527fc640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fb25409faa0 0x7fb2540a08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:37058/0 (socket says 172.21.15.178:37058) 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2527fc640 1 -- 172.21.15.178:0/3752051350 learned_addr learned my addr 172.21.15.178:0/3752051350 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 10 monclient(hunting): get_auth_request con 0x7fb25409fe70 auth_method 0 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb252ffd640 1 --2- 172.21.15.178:0/3752051350 >> v2:172.21.15.204:3300/0 conn(0x7fb2540a2930 0x7fb254132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb252ffd640 10 monclient(hunting): get_auth_request con 0x7fb2540a2930 auth_method 0 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb252ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb252ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb252ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 10 monclient(hunting): handle_auth_done global_id 45184 payload 274 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 1 -- 172.21.15.178:0/3752051350 >> v2:172.21.15.204:3300/0 conn(0x7fb2540a2930 msgr2=0x7fb254132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb251ffb640 1 -- 172.21.15.178:0/3752051350 <== mon.0 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773077155 0 0) 0x7fb25404bf60 con 0x7fb25409faa0 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 1 --2- 172.21.15.178:0/3752051350 >> v2:172.21.15.204:3300/0 conn(0x7fb2540a2930 0x7fb254132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 1 -- 172.21.15.178:0/3752051350 >> v1:172.21.15.178:6789/0 conn(0x7fb25409faa0 legacy=0x7fb2540a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 10 monclient: _finish_hunting 0 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 1 monclient: found mon.noname-c 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb2537fe640 1 -- 172.21.15.178:0/3752051350 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb254133440 con 0x7fb25409fe70 2024-09-15T01:50:06.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.710+0000 7fb251ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-09-15T01:50:06.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb2537fe640 1 --2- 172.21.15.178:0/3752051350 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409fe70 0x7fb2540a0fd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009e60 tx=0x7fb24c009780 comp rx=0 tx=0).ready entity=mon.4 client_cookie=4f84ef4e82e3b05b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb251ffb640 1 -- 172.21.15.178:0/3752051350 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb24c0079d0 con 0x7fb25409fe70 2024-09-15T01:50:06.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb251ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb251ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:06.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb251ffb640 10 monclient: dump: 2024-09-15T01:50:06.724 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.724 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.724 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.724 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.724 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.724 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.725 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.725 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.725 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.725 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.725 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.725 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.725 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.725 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.725 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.725 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb251ffb640 10 monclient: _finish_auth 0 2024-09-15T01:50:06.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb251ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:06.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb251ffb640 1 -- 172.21.15.178:0/3752051350 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c02d050 con 0x7fb25409fe70 2024-09-15T01:50:06.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb2594c0640 5 monclient: authenticate success, global_id 45184 2024-09-15T01:50:06.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb2594c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:06.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb251ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:06.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb251ffb640 1 -- 172.21.15.178:0/3752051350 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb24c007b90 con 0x7fb25409fe70 2024-09-15T01:50:06.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb2594c0640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:06.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb2594c0640 4 set_mon_vals no callback set 2024-09-15T01:50:06.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb251ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb251ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:06.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.711+0000 7fb251ffb640 10 monclient: dump: 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.727 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.712+0000 7fb2594c0640 10 monclient: shutdown 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.712+0000 7fb2594c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.712+0000 7fb2594c0640 1 -- 172.21.15.178:0/3752051350 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409fe70 msgr2=0x7fb2540a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.712+0000 7fb2594c0640 1 --2- 172.21.15.178:0/3752051350 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409fe70 0x7fb2540a0fd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009e60 tx=0x7fb24c009780 comp rx=0 tx=0).stop 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.712+0000 7fb2594c0640 1 -- 172.21.15.178:0/3752051350 shutdown_connections 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.712+0000 7fb2594c0640 1 --2- 172.21.15.178:0/3752051350 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409fe70 0x7fb2540a0fd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.712+0000 7fb2594c0640 1 --2- 172.21.15.178:0/3752051350 >> v2:172.21.15.204:3300/0 conn(0x7fb2540a2930 0x7fb254132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.712+0000 7fb2594c0640 1 -- 172.21.15.178:0/3752051350 >> 172.21.15.178:0/3752051350 conn(0x7fb2540a5ed0 msgr2=0x7fb2540a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.712+0000 7fb2594c0640 1 -- 172.21.15.178:0/3752051350 shutdown_connections 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.712+0000 7fb2594c0640 1 -- 172.21.15.178:0/3752051350 wait complete. 2024-09-15T01:50:06.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.712+0000 7fb2594c0640 10 monclient: build_initial_monmap 2024-09-15T01:50:06.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.712+0000 7fb2594c0640 10 monclient: monmap: 2024-09-15T01:50:06.729 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:06.729 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:06.729 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:06.729 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:06.729 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:06.729 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:06.729 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:06.729 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:06.729 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 1 Processor -- start 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 1 -- start start 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 10 monclient: init 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 10 monclient: _add_conns ranks=[4,8,3,0,7,5,1,2,6] 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409faa0 0x7fb25409ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 10 monclient(hunting): picked mon.noname-c con 0x7fb25409faa0 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb25409fe70 0x7fb2541316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 10 monclient(hunting): picked mon.noname-e con 0x7fb25409fe70 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2540a2930 0x7fb25411c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 10 monclient(hunting): picked mon.noname-g con 0x7fb2540a2930 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25409d630 con 0x7fb2540a2930 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb254137000 con 0x7fb25409faa0 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb25411edc0 con 0x7fb25409fe70 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2594c0640 10 monclient(hunting): authenticate will time out at 1811.463704s 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb252ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409faa0 0x7fb25409ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb252ffd640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409faa0 0x7fb25409ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3302/0 says I am v2:172.21.15.178:44890/0 (socket says 172.21.15.178:44890) 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb252ffd640 1 -- 172.21.15.178:0/3800612963 learned_addr learned my addr 172.21.15.178:0/3800612963 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb252ffd640 10 monclient(hunting): get_auth_request con 0x7fb25409faa0 auth_method 0 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb252ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb252ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb252ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2527fc640 1 --2- 172.21.15.178:0/3800612963 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb25409fe70 0x7fb2541316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2527fc640 10 monclient(hunting): get_auth_request con 0x7fb25409fe70 auth_method 0 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2527fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2527fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb252ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb252ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb252ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2527fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2527fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2527fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.713+0000 7fb2537fe640 1 --2- 172.21.15.178:0/3800612963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2540a2930 0x7fb25411c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb252ffd640 10 monclient(hunting): handle_auth_done global_id 45193 payload 824 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb252ffd640 1 -- 172.21.15.178:0/3800612963 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb25409fe70 msgr2=0x7fb2541316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb252ffd640 1 --2- 172.21.15.178:0/3800612963 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb25409fe70 0x7fb2541316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb252ffd640 1 -- 172.21.15.178:0/3800612963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2540a2930 msgr2=0x7fb25411c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb252ffd640 1 --2- 172.21.15.178:0/3800612963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2540a2930 0x7fb25411c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb252ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb252ffd640 1 monclient: found mon.noname-c 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb252ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:06.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb252ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3302/0 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb252ffd640 1 -- 172.21.15.178:0/3800612963 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c009a20 con 0x7fb25409faa0 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb2537fe640 10 monclient: get_auth_request con 0x7fb2540a2930 auth_method 0 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb2537fe640 1 --2- 172.21.15.178:0/3800612963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2540a2930 0x7fb25411c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb2527fc640 1 --2- 172.21.15.178:0/3800612963 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb25409fe70 0x7fb2541316e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb252ffd640 1 --2- 172.21.15.178:0/3800612963 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409faa0 0x7fb25409ad50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb24400d890 tx=0x7fb244008040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb24affd640 1 -- 172.21.15.178:0/3800612963 <== mon.4 v2:172.21.15.178:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb24400de20 con 0x7fb25409faa0 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb24affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb24affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb24affd640 10 monclient: dump: 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.734 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.735 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.735 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.735 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.735 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.735 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.735 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.735 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.735 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.736 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.736 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.736 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb24affd640 10 monclient: _finish_auth 0 2024-09-15T01:50:06.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb24affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:06.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb24affd640 1 -- 172.21.15.178:0/3800612963 <== mon.4 v2:172.21.15.178:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24400bd50 con 0x7fb25409faa0 2024-09-15T01:50:06.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb2594c0640 5 monclient: authenticate success, global_id 45193 2024-09-15T01:50:06.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb2594c0640 10 monclient: _renew_subs 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb2594c0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb2594c0640 1 -- 172.21.15.178:0/3800612963 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25409bd00 con 0x7fb25409faa0 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb24affd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb24affd640 1 -- 172.21.15.178:0/3800612963 <== mon.4 v2:172.21.15.178:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb2440148b0 con 0x7fb25409faa0 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb24affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb24affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb24affd640 10 monclient: dump: 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:06.737 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb2517fa640 4 set_mon_vals no callback set 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb2594c0640 10 monclient: _renew_subs 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb2594c0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.714+0000 7fb2594c0640 1 -- 172.21.15.178:0/3800612963 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25411e800 con 0x7fb25409faa0 2024-09-15T01:50:06.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.715+0000 7fb24affd640 1 -- 172.21.15.178:0/3800612963 <== mon.4 v2:172.21.15.178:3302/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7fb244014430 con 0x7fb25409faa0 2024-09-15T01:50:06.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.715+0000 7fb2594c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:06.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.715+0000 7fb2594c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:06.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.715+0000 7fb2594c0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3302/0 2024-09-15T01:50:06.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.715+0000 7fb2594c0640 1 -- 172.21.15.178:0/3800612963 --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25404b720 con 0x7fb25409faa0 2024-09-15T01:50:06.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.715+0000 7fb24affd640 1 --2- 172.21.15.178:0/3800612963 >> v2:172.21.15.204:6812/28210 conn(0x7fb2340397b0 0x7fb23403bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:06.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.715+0000 7fb24affd640 1 -- 172.21.15.178:0/3800612963 <== mon.4 v2:172.21.15.178:3302/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7fb24404c300 con 0x7fb25409faa0 2024-09-15T01:50:06.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.715+0000 7fb2527fc640 1 --2- 172.21.15.178:0/3800612963 >> v2:172.21.15.204:6812/28210 conn(0x7fb2340397b0 0x7fb23403bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:06.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.715+0000 7fb2527fc640 10 monclient: get_auth_request con 0x7fb2340397b0 auth_method 0 2024-09-15T01:50:06.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.716+0000 7fb2527fc640 1 --2- 172.21.15.178:0/3800612963 >> v2:172.21.15.204:6812/28210 conn(0x7fb2340397b0 0x7fb23403bc60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb23c000c00 tx=0x7fb23c009040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:06.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.720+0000 7fb24affd640 1 -- 172.21.15.178:0/3800612963 <== mon.4 v2:172.21.15.178:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb244014a30 con 0x7fb25409faa0 2024-09-15T01:50:06.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.720+0000 7fb24affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:06.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.720+0000 7fb24affd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:06.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.876+0000 7fb2594c0640 1 -- 172.21.15.178:0/3800612963 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb25404c4a0 con 0x7fb2340397b0 2024-09-15T01:50:06.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.879+0000 7fb24affd640 1 -- 172.21.15.178:0/3800612963 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+93378 (secure 0 0 0) 0x7fb25404c4a0 con 0x7fb2340397b0 2024-09-15T01:50:06.881 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:50:06.882 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-09-15T01:50:06.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.881+0000 7fb2594c0640 1 -- 172.21.15.178:0/3800612963 >> v2:172.21.15.204:6812/28210 conn(0x7fb2340397b0 msgr2=0x7fb23403bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.881+0000 7fb2594c0640 1 --2- 172.21.15.178:0/3800612963 >> v2:172.21.15.204:6812/28210 conn(0x7fb2340397b0 0x7fb23403bc60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb23c000c00 tx=0x7fb23c009040 comp rx=0 tx=0).stop 2024-09-15T01:50:06.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.881+0000 7fb2594c0640 10 monclient: shutdown 2024-09-15T01:50:06.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.881+0000 7fb2594c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:06.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.881+0000 7fb2594c0640 1 -- 172.21.15.178:0/3800612963 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409faa0 msgr2=0x7fb25409ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:06.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.881+0000 7fb2594c0640 1 --2- 172.21.15.178:0/3800612963 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409faa0 0x7fb25409ad50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb24400d890 tx=0x7fb244008040 comp rx=0 tx=0).stop 2024-09-15T01:50:06.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.881+0000 7fb2594c0640 1 -- 172.21.15.178:0/3800612963 shutdown_connections 2024-09-15T01:50:06.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.881+0000 7fb2594c0640 1 --2- 172.21.15.178:0/3800612963 >> v2:172.21.15.204:6812/28210 conn(0x7fb2340397b0 0x7fb23403bc60 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.881+0000 7fb2594c0640 1 --2- 172.21.15.178:0/3800612963 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7fb25409fe70 0x7fb2541316e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.881+0000 7fb2594c0640 1 --2- 172.21.15.178:0/3800612963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2540a2930 0x7fb25411c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.881+0000 7fb2594c0640 1 --2- 172.21.15.178:0/3800612963 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7fb25409faa0 0x7fb25409ad50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:06.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.881+0000 7fb2594c0640 1 -- 172.21.15.178:0/3800612963 >> 172.21.15.178:0/3800612963 conn(0x7fb2540a5ed0 msgr2=0x7fb25409e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:06.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.882+0000 7fb2594c0640 1 -- 172.21.15.178:0/3800612963 shutdown_connections 2024-09-15T01:50:06.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:06.882+0000 7fb2594c0640 1 -- 172.21.15.178:0/3800612963 wait complete. 2024-09-15T01:50:06.894 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":803,"stamp":"2024-09-15T01:50:06.387286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562077696,"kb_used":4795144,"kb_used_data":557282552,"kb_used_omap":18904,"kb_used_meta":0,"kb_avail":557282552,"statfs":{"total":575567560704,"available":570657333248,"internally_reserved":0,"allocated":570657333248,"data_stored":570657333248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19358691,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002661"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288825+0000","last_change":"2024-09-15T01:36:25.227464+0000","last_active":"2024-09-15T01:44:24.288825+0000","last_peered":"2024-09-15T01:44:24.288825+0000","last_clean":"2024-09-15T01:44:24.288825+0000","last_became_active":"2024-09-15T01:36:25.227161+0000","last_became_peered":"2024-09-15T01:36:25.227161+0000","last_unstale":"2024-09-15T01:44:24.288825+0000","last_undegraded":"2024-09-15T01:44:24.288825+0000","last_fullsized":"2024-09-15T01:44:24.288825+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:44:14.933447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":97,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161809+0000","last_change":"2024-09-15T01:36:25.228477+0000","last_active":"2024-09-15T01:44:24.161809+0000","last_peered":"2024-09-15T01:44:24.161809+0000","last_clean":"2024-09-15T01:44:24.161809+0000","last_became_active":"2024-09-15T01:36:25.228216+0000","last_became_peered":"2024-09-15T01:36:25.228216+0000","last_unstale":"2024-09-15T01:44:24.161809+0000","last_undegraded":"2024-09-15T01:44:24.161809+0000","last_fullsized":"2024-09-15T01:44:24.161809+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:33:26.644405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288874+0000","last_change":"2024-09-15T01:36:25.228316+0000","last_active":"2024-09-15T01:44:24.288874+0000","last_peered":"2024-09-15T01:44:24.288874+0000","last_clean":"2024-09-15T01:44:24.288874+0000","last_became_active":"2024-09-15T01:36:25.228024+0000","last_became_peered":"2024-09-15T01:36:25.228024+0000","last_unstale":"2024-09-15T01:44:24.288874+0000","last_undegraded":"2024-09-15T01:44:24.288874+0000","last_fullsized":"2024-09-15T01:44:24.288874+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:35.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":97,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161781+0000","last_change":"2024-09-15T01:36:25.230047+0000","last_active":"2024-09-15T01:44:24.161781+0000","last_peered":"2024-09-15T01:44:24.161781+0000","last_clean":"2024-09-15T01:44:24.161781+0000","last_became_active":"2024-09-15T01:36:25.229786+0000","last_became_peered":"2024-09-15T01:36:25.229786+0000","last_unstale":"2024-09-15T01:44:24.161781+0000","last_undegraded":"2024-09-15T01:44:24.161781+0000","last_fullsized":"2024-09-15T01:44:24.161781+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:15:18.249275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":92,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160052+0000","last_change":"2024-09-15T01:36:25.233254+0000","last_active":"2024-09-15T01:44:24.160052+0000","last_peered":"2024-09-15T01:44:24.160052+0000","last_clean":"2024-09-15T01:44:24.160052+0000","last_became_active":"2024-09-15T01:36:25.232778+0000","last_became_peered":"2024-09-15T01:36:25.232778+0000","last_unstale":"2024-09-15T01:44:24.160052+0000","last_undegraded":"2024-09-15T01:44:24.160052+0000","last_fullsized":"2024-09-15T01:44:24.160052+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:35:24.397702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.289191+0000","last_change":"2024-09-15T01:36:25.227383+0000","last_active":"2024-09-15T01:44:24.289191+0000","last_peered":"2024-09-15T01:44:24.289191+0000","last_clean":"2024-09-15T01:44:24.289191+0000","last_became_active":"2024-09-15T01:36:25.227142+0000","last_became_peered":"2024-09-15T01:36:25.227142+0000","last_unstale":"2024-09-15T01:44:24.289191+0000","last_undegraded":"2024-09-15T01:44:24.289191+0000","last_fullsized":"2024-09-15T01:44:24.289191+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:22:59.700920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":170,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159692+0000","last_change":"2024-09-15T01:36:25.231017+0000","last_active":"2024-09-15T01:44:24.159692+0000","last_peered":"2024-09-15T01:44:24.159692+0000","last_clean":"2024-09-15T01:44:24.159692+0000","last_became_active":"2024-09-15T01:36:25.230723+0000","last_became_peered":"2024-09-15T01:36:25.230723+0000","last_unstale":"2024-09-15T01:44:24.159692+0000","last_undegraded":"2024-09-15T01:44:24.159692+0000","last_fullsized":"2024-09-15T01:44:24.159692+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:51:39.782818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160643+0000","last_change":"2024-09-15T01:36:25.228490+0000","last_active":"2024-09-15T01:44:24.160643+0000","last_peered":"2024-09-15T01:44:24.160643+0000","last_clean":"2024-09-15T01:44:24.160643+0000","last_became_active":"2024-09-15T01:36:25.228252+0000","last_became_peered":"2024-09-15T01:36:25.228252+0000","last_unstale":"2024-09-15T01:44:24.160643+0000","last_undegraded":"2024-09-15T01:44:24.160643+0000","last_fullsized":"2024-09-15T01:44:24.160643+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:56:14.136629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":98,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161798+0000","last_change":"2024-09-15T01:36:25.227553+0000","last_active":"2024-09-15T01:44:24.161798+0000","last_peered":"2024-09-15T01:44:24.161798+0000","last_clean":"2024-09-15T01:44:24.161798+0000","last_became_active":"2024-09-15T01:36:25.226915+0000","last_became_peered":"2024-09-15T01:36:25.226915+0000","last_unstale":"2024-09-15T01:44:24.161798+0000","last_undegraded":"2024-09-15T01:44:24.161798+0000","last_fullsized":"2024-09-15T01:44:24.161798+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:18:34.776247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":118,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159771+0000","last_change":"2024-09-15T01:36:25.229923+0000","last_active":"2024-09-15T01:44:24.159771+0000","last_peered":"2024-09-15T01:44:24.159771+0000","last_clean":"2024-09-15T01:44:24.159771+0000","last_became_active":"2024-09-15T01:36:25.229665+0000","last_became_peered":"2024-09-15T01:36:25.229665+0000","last_unstale":"2024-09-15T01:44:24.159771+0000","last_undegraded":"2024-09-15T01:44:24.159771+0000","last_fullsized":"2024-09-15T01:44:24.159771+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:55:30.051647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":176,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159528+0000","last_change":"2024-09-15T01:36:25.227907+0000","last_active":"2024-09-15T01:44:24.159528+0000","last_peered":"2024-09-15T01:44:24.159528+0000","last_clean":"2024-09-15T01:44:24.159528+0000","last_became_active":"2024-09-15T01:36:25.227397+0000","last_became_peered":"2024-09-15T01:36:25.227397+0000","last_unstale":"2024-09-15T01:44:24.159528+0000","last_undegraded":"2024-09-15T01:44:24.159528+0000","last_fullsized":"2024-09-15T01:44:24.159528+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:52.384874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160501+0000","last_change":"2024-09-15T01:36:25.227223+0000","last_active":"2024-09-15T01:44:24.160501+0000","last_peered":"2024-09-15T01:44:24.160501+0000","last_clean":"2024-09-15T01:44:24.160501+0000","last_became_active":"2024-09-15T01:36:25.226969+0000","last_became_peered":"2024-09-15T01:36:25.226969+0000","last_unstale":"2024-09-15T01:44:24.160501+0000","last_undegraded":"2024-09-15T01:44:24.160501+0000","last_fullsized":"2024-09-15T01:44:24.160501+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:11:23.636488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.289013+0000","last_change":"2024-09-15T01:36:26.228988+0000","last_active":"2024-09-15T01:44:24.289013+0000","last_peered":"2024-09-15T01:44:24.289013+0000","last_clean":"2024-09-15T01:44:24.289013+0000","last_became_active":"2024-09-15T01:36:26.228683+0000","last_became_peered":"2024-09-15T01:36:26.228683+0000","last_unstale":"2024-09-15T01:44:24.289013+0000","last_undegraded":"2024-09-15T01:44:24.289013+0000","last_fullsized":"2024-09-15T01:44:24.289013+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:42.069204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":87,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160502+0000","last_change":"2024-09-15T01:36:26.228676+0000","last_active":"2024-09-15T01:44:24.160502+0000","last_peered":"2024-09-15T01:44:24.160502+0000","last_clean":"2024-09-15T01:44:24.160502+0000","last_became_active":"2024-09-15T01:36:26.228429+0000","last_became_peered":"2024-09-15T01:36:26.228429+0000","last_unstale":"2024-09-15T01:44:24.160502+0000","last_undegraded":"2024-09-15T01:44:24.160502+0000","last_fullsized":"2024-09-15T01:44:24.160502+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:41:46.249246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":106,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288775+0000","last_change":"2024-09-15T01:36:26.226092+0000","last_active":"2024-09-15T01:44:24.288775+0000","last_peered":"2024-09-15T01:44:24.288775+0000","last_clean":"2024-09-15T01:44:24.288775+0000","last_became_active":"2024-09-15T01:36:26.225738+0000","last_became_peered":"2024-09-15T01:36:26.225738+0000","last_unstale":"2024-09-15T01:44:24.288775+0000","last_undegraded":"2024-09-15T01:44:24.288775+0000","last_fullsized":"2024-09-15T01:44:24.288775+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:28.575953+0000","last_clean_scrub_stamp":"2024-09-15T01:35:28.575953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:49:14.676091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":86,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288958+0000","last_change":"2024-09-15T01:36:25.225943+0000","last_active":"2024-09-15T01:44:24.288958+0000","last_peered":"2024-09-15T01:44:24.288958+0000","last_clean":"2024-09-15T01:44:24.288958+0000","last_became_active":"2024-09-15T01:36:25.225608+0000","last_became_peered":"2024-09-15T01:36:25.225608+0000","last_unstale":"2024-09-15T01:44:24.288958+0000","last_undegraded":"2024-09-15T01:44:24.288958+0000","last_fullsized":"2024-09-15T01:44:24.288958+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:35:27.569993+0000","last_clean_scrub_stamp":"2024-09-15T01:35:27.569993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:16:43.688463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":100,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159580+0000","last_change":"2024-09-15T01:36:25.260030+0000","last_active":"2024-09-15T01:44:24.159580+0000","last_peered":"2024-09-15T01:44:24.159580+0000","last_clean":"2024-09-15T01:44:24.159580+0000","last_became_active":"2024-09-15T01:36:25.233270+0000","last_became_peered":"2024-09-15T01:36:25.233270+0000","last_unstale":"2024-09-15T01:44:24.159580+0000","last_undegraded":"2024-09-15T01:44:24.159580+0000","last_fullsized":"2024-09-15T01:44:24.159580+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_clean_scrub_stamp":"2024-09-15T01:31:59.015115+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:28:21.577888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":258,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159895+0000","last_change":"2024-09-15T01:36:25.233515+0000","last_active":"2024-09-15T01:44:24.159895+0000","last_peered":"2024-09-15T01:44:24.159895+0000","last_clean":"2024-09-15T01:44:24.159895+0000","last_became_active":"2024-09-15T01:36:25.230032+0000","last_became_peered":"2024-09-15T01:36:25.230032+0000","last_unstale":"2024-09-15T01:44:24.159895+0000","last_undegraded":"2024-09-15T01:44:24.159895+0000","last_fullsized":"2024-09-15T01:44:24.159895+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:44:35.523507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076976900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.161688+0000","last_change":"2024-09-15T01:36:25.227415+0000","last_active":"2024-09-15T01:44:24.161688+0000","last_peered":"2024-09-15T01:44:24.161688+0000","last_clean":"2024-09-15T01:44:24.161688+0000","last_became_active":"2024-09-15T01:36:25.226499+0000","last_became_peered":"2024-09-15T01:36:25.226499+0000","last_unstale":"2024-09-15T01:44:24.161688+0000","last_undegraded":"2024-09-15T01:44:24.161688+0000","last_fullsized":"2024-09-15T01:44:24.161688+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:35:59.548191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037511000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160610+0000","last_change":"2024-09-15T01:36:25.231459+0000","last_active":"2024-09-15T01:44:24.160610+0000","last_peered":"2024-09-15T01:44:24.160610+0000","last_clean":"2024-09-15T01:44:24.160610+0000","last_became_active":"2024-09-15T01:36:25.231212+0000","last_became_peered":"2024-09-15T01:36:25.231212+0000","last_unstale":"2024-09-15T01:44:24.160610+0000","last_undegraded":"2024-09-15T01:44:24.160610+0000","last_fullsized":"2024-09-15T01:44:24.160610+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:16:13.072580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046258900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":296,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160823+0000","last_change":"2024-09-15T01:36:25.236741+0000","last_active":"2024-09-15T01:44:24.160823+0000","last_peered":"2024-09-15T01:44:24.160823+0000","last_clean":"2024-09-15T01:44:24.160823+0000","last_became_active":"2024-09-15T01:36:25.226718+0000","last_became_peered":"2024-09-15T01:36:25.226718+0000","last_unstale":"2024-09-15T01:44:24.160823+0000","last_undegraded":"2024-09-15T01:44:24.160823+0000","last_fullsized":"2024-09-15T01:44:24.160823+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:07:25.773736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093022000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":255,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.289138+0000","last_change":"2024-09-15T01:36:25.227607+0000","last_active":"2024-09-15T01:44:24.289138+0000","last_peered":"2024-09-15T01:44:24.289138+0000","last_clean":"2024-09-15T01:44:24.289138+0000","last_became_active":"2024-09-15T01:36:25.227361+0000","last_became_peered":"2024-09-15T01:36:25.227361+0000","last_unstale":"2024-09-15T01:44:24.289138+0000","last_undegraded":"2024-09-15T01:44:24.289138+0000","last_fullsized":"2024-09-15T01:44:24.289138+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:15:42.847085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031034999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.159956+0000","last_change":"2024-09-15T01:36:25.230157+0000","last_active":"2024-09-15T01:44:24.159956+0000","last_peered":"2024-09-15T01:44:24.159956+0000","last_clean":"2024-09-15T01:44:24.159956+0000","last_became_active":"2024-09-15T01:36:25.229674+0000","last_became_peered":"2024-09-15T01:36:25.229674+0000","last_unstale":"2024-09-15T01:44:24.159956+0000","last_undegraded":"2024-09-15T01:44:24.159956+0000","last_fullsized":"2024-09-15T01:44:24.159956+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:12:26.442496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077134000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":85,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.288724+0000","last_change":"2024-09-15T01:36:25.225614+0000","last_active":"2024-09-15T01:44:24.288724+0000","last_peered":"2024-09-15T01:44:24.288724+0000","last_clean":"2024-09-15T01:44:24.288724+0000","last_became_active":"2024-09-15T01:36:25.225361+0000","last_became_peered":"2024-09-15T01:36:25.225361+0000","last_unstale":"2024-09-15T01:44:24.288724+0000","last_undegraded":"2024-09-15T01:44:24.288724+0000","last_fullsized":"2024-09-15T01:44:24.288724+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:25:49.332027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041942500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":245,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-15T01:44:24.160207+0000","last_change":"2024-09-15T01:36:26.437860+0000","last_active":"2024-09-15T01:44:24.160207+0000","last_peered":"2024-09-15T01:44:24.160207+0000","last_clean":"2024-09-15T01:44:24.160207+0000","last_became_active":"2024-09-15T01:36:26.437594+0000","last_became_peered":"2024-09-15T01:36:26.437594+0000","last_unstale":"2024-09-15T01:44:24.160207+0000","last_undegraded":"2024-09-15T01:44:24.160207+0000","last_fullsized":"2024-09-15T01:44:24.160207+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:50:29.143741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065783200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640475,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799576,"kb_used_data":92880040,"kb_used_omap":3214,"kb_used_meta":0,"kb_avail":92880040,"statfs":{"total":95927926784,"available":95109160960,"internally_reserved":0,"allocated":95109160960,"data_stored":95109160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3291775,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.39500000000000002,"5min":0.28999999999999998,"15min":0.24299999999999999},"max":{"1min":0.65500000000000003,"5min":0.73499999999999999,"15min":0.95099999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59699999999999998,"5min":0.68100000000000005,"15min":0.76700000000000002},"last":0.41099999999999998}]},{"osd":1,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.28899999999999998,"15min":0.28299999999999997},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.91800000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.30499999999999999},"max":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.72899999999999998},"last":0.45600000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.23300000000000001},"max":{"1min":0.629,"5min":0.69199999999999995,"15min":0.73399999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.50600000000000001},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.27700000000000002},"max":{"1min":0.629,"5min":0.67200000000000004,"15min":0.81000000000000005},"last":0.372}]},{"osd":3,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.27200000000000002,"5min":0.224,"15min":0.20699999999999999},"max":{"1min":0.52600000000000002,"5min":0.63400000000000001,"15min":0.96799999999999997},"last":0.432},{"interface":"front","average":{"1min":0.377,"5min":0.39700000000000002,"15min":0.38600000000000001},"min":{"1min":0.28199999999999997,"5min":0.23200000000000001,"15min":0.216},"max":{"1min":0.54700000000000004,"5min":0.64700000000000002,"15min":0.86199999999999999},"last":0.55500000000000005}]},{"osd":4,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.378,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":1.0289999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40699999999999997,"15min":0.40300000000000002},"min":{"1min":0.26400000000000001,"5min":0.249,"15min":0.22},"max":{"1min":0.54300000000000004,"5min":0.61799999999999999,"15min":0.67200000000000004},"last":0.48899999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799000,"kb_used_data":92880616,"kb_used_omap":3258,"kb_used_meta":0,"kb_avail":92880616,"statfs":{"total":95927926784,"available":95109750784,"internally_reserved":0,"allocated":95109750784,"data_stored":95109750784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3336786,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.48999999999999999},"min":{"1min":0.39800000000000002,"5min":0.33500000000000002,"15min":0.219},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.39700000000000002,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.66600000000000004,"5min":0.69399999999999995,"15min":0.70799999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53600000000000003,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.25600000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.52800000000000002},"min":{"1min":0.47599999999999998,"5min":0.36499999999999999,"15min":0.28999999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.60099999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.71599999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69199999999999995,"5min":0.876,"15min":0.876},"last":0.54600000000000004}]},{"osd":3,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.36699999999999999},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.22700000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.40699999999999997,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.216},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.3540000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.39600000000000002,"15min":0.376},"min":{"1min":0.27900000000000003,"5min":0.26600000000000001,"15min":0.22700000000000001},"max":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":1.2709999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.39600000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.24199999999999999},"max":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":1.2250000000000001},"last":0.53300000000000003}]}]},{"osd":3,"up_from":11,"seq":47244640475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798996,"kb_used_data":92880620,"kb_used_omap":3033,"kb_used_meta":0,"kb_avail":92880620,"statfs":{"total":95927926784,"available":95109754880,"internally_reserved":0,"allocated":95109754880,"data_stored":95109754880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3106015,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.24299999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.69399999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.29899999999999999},"max":{"1min":0.64900000000000002,"5min":0.66400000000000003,"15min":0.81599999999999995},"last":0.45300000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.27900000000000003},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.79300000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.37,"5min":0.34399999999999997,"15min":0.29899999999999999},"max":{"1min":0.58799999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.442}]},{"osd":2,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.74299999999999999},"last":0.375},{"interface":"front","average":{"1min":0.501,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.30099999999999999},"max":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.75800000000000001},"last":0.40500000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36099999999999999,"15min":0.35999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.46200000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.313},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.61699999999999999,"5min":0.65900000000000003,"15min":0.70599999999999996},"last":0.33000000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.53200000000000003,"5min":0.57099999999999995,"15min":0.65500000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.39000000000000001,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.49299999999999999,"5min":0.64600000000000002,"15min":0.70399999999999996},"last":0.34699999999999998}]}]},{"osd":2,"up_from":11,"seq":47244640475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798996,"kb_used_data":92880620,"kb_used_omap":3051,"kb_used_meta":0,"kb_avail":92880620,"statfs":{"total":95927926784,"available":95109754880,"internally_reserved":0,"allocated":95109754880,"data_stored":95109754880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3125031,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.32600000000000001},"min":{"1min":0.23200000000000001,"5min":0.20100000000000001,"15min":0.17499999999999999},"max":{"1min":0.47999999999999998,"5min":0.48499999999999999,"15min":0.53000000000000003},"last":0.16300000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.375,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.23999999999999999,"15min":0.20200000000000001},"max":{"1min":0.501,"5min":0.53000000000000003,"15min":0.59999999999999998},"last":0.26300000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.33200000000000002,"15min":0.33500000000000002},"min":{"1min":0.20300000000000001,"5min":0.187,"15min":0.182},"max":{"1min":0.46100000000000002,"5min":0.52500000000000002,"15min":0.60599999999999998},"last":0.191},{"interface":"front","average":{"1min":0.377,"5min":0.373,"15min":0.373},"min":{"1min":0.24299999999999999,"5min":0.222,"15min":0.20699999999999999},"max":{"1min":0.53300000000000003,"5min":0.57099999999999995,"15min":0.67400000000000004},"last":0.252}]},{"osd":3,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.28899999999999998},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.78900000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61299999999999999,"5min":0.77900000000000003,"15min":1.0129999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.55400000000000005,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":0.63500000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":0.371},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.27600000000000002},"max":{"1min":0.59699999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.32200000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.27800000000000002},"max":{"1min":0.67300000000000004,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.314},{"interface":"front","average":{"1min":0.504,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.30099999999999999},"max":{"1min":0.623,"5min":0.75900000000000001,"15min":0.87},"last":0.36299999999999999}]}]},{"osd":1,"up_from":11,"seq":47244640476,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799576,"kb_used_data":92880040,"kb_used_omap":3055,"kb_used_meta":0,"kb_avail":92880040,"statfs":{"total":95927926784,"available":95109160960,"internally_reserved":0,"allocated":95109160960,"data_stored":95109160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3129006,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33700000000000002,"15min":0.33300000000000002},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.188},"max":{"1min":0.47399999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.35499999999999998,"15min":0.35299999999999998},"min":{"1min":0.27800000000000002,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.68200000000000005},"last":0.28499999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.37},"min":{"1min":0.29799999999999999,"5min":0.245,"15min":0.23799999999999999},"max":{"1min":0.501,"5min":0.51400000000000001,"15min":0.73399999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28899999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.70299999999999996},"last":0.313}]},{"osd":3,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.46999999999999997,"5min":0.34499999999999997,"15min":0.30099999999999999},"max":{"1min":0.68700000000000006,"5min":0.72099999999999997,"15min":0.76800000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.46800000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.76700000000000002},"last":0.53500000000000003}]},{"osd":4,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.49099999999999999,"5min":0.40500000000000003,"15min":0.35599999999999998},"max":{"1min":0.70599999999999996,"5min":0.77700000000000002,"15min":0.80600000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.308},"max":{"1min":0.71699999999999997,"5min":0.76800000000000002,"15min":0.81899999999999995},"last":0.56799999999999995}]},{"osd":5,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.32600000000000001},"max":{"1min":0.69099999999999995,"5min":0.748,"15min":0.89400000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.53800000000000003},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.32200000000000001},"max":{"1min":0.64400000000000002,"5min":0.69699999999999995,"15min":0.81000000000000005},"last":0.499}]}]},{"osd":0,"up_from":11,"seq":47244640475,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799000,"kb_used_data":92880616,"kb_used_omap":3291,"kb_used_meta":0,"kb_avail":92880616,"statfs":{"total":95927926784,"available":95109750784,"internally_reserved":0,"allocated":95109750784,"data_stored":95109750784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3370078,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32600000000000001,"15min":0.33100000000000002},"min":{"1min":0.22,"5min":0.188,"15min":0.157},"max":{"1min":0.503,"5min":0.52100000000000002,"15min":0.75600000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.376,"15min":0.379},"min":{"1min":0.254,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.47999999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.38700000000000001}]},{"osd":2,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34699999999999998,"15min":0.36899999999999999},"min":{"1min":0.25,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.46300000000000002,"5min":0.54800000000000004,"15min":0.69499999999999995},"last":0.376},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40000000000000002,"15min":0.38100000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.19},"max":{"1min":0.55100000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.41399999999999998}]},{"osd":3,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.28399999999999997},"max":{"1min":0.58299999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51300000000000001,"15min":0.51400000000000001},"min":{"1min":0.40500000000000003,"5min":0.30599999999999999,"15min":0.26400000000000001},"max":{"1min":0.64000000000000001,"5min":0.745,"15min":0.77400000000000002},"last":0.46300000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.625,"5min":0.71799999999999997,"15min":0.72099999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59699999999999998,"5min":0.69999999999999996,"15min":0.752},"last":0.48099999999999998}]},{"osd":5,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65700000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.439},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.46800000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66900000000000004,"5min":0.73899999999999999,"15min":0.75600000000000001},"last":0.49099999999999999}]}]}],"pool_statfs":[]}} 2024-09-15T01:50:06.902 INFO:tasks.ceph:pgid 10.4 last_scrub_stamp 2024-09-15T01:35:28.575953+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=28, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.902 INFO:tasks.ceph:pgid 9.7 last_scrub_stamp 2024-09-15T01:35:27.569993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=27, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.902 INFO:tasks.ceph:pgid 10.5 last_scrub_stamp 2024-09-15T01:35:28.575953+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=28, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.902 INFO:tasks.ceph:pgid 9.6 last_scrub_stamp 2024-09-15T01:35:27.569993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=27, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.903 INFO:tasks.ceph:pgid 10.6 last_scrub_stamp 2024-09-15T01:35:28.575953+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=28, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.903 INFO:tasks.ceph:pgid 9.5 last_scrub_stamp 2024-09-15T01:35:27.569993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=27, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.903 INFO:tasks.ceph:pgid 10.7 last_scrub_stamp 2024-09-15T01:35:28.575953+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=28, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.903 INFO:tasks.ceph:pgid 9.4 last_scrub_stamp 2024-09-15T01:35:27.569993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=27, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.903 INFO:tasks.ceph:pgid 10.0 last_scrub_stamp 2024-09-15T01:35:28.575953+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=28, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.903 INFO:tasks.ceph:pgid 9.3 last_scrub_stamp 2024-09-15T01:35:27.569993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=27, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.904 INFO:tasks.ceph:pgid 10.1 last_scrub_stamp 2024-09-15T01:35:28.575953+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=28, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.904 INFO:tasks.ceph:pgid 9.2 last_scrub_stamp 2024-09-15T01:35:27.569993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=27, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.904 INFO:tasks.ceph:pgid 10.2 last_scrub_stamp 2024-09-15T01:35:28.575953+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=28, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.904 INFO:tasks.ceph:pgid 9.1 last_scrub_stamp 2024-09-15T01:35:27.569993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=27, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.904 INFO:tasks.ceph:pgid 10.3 last_scrub_stamp 2024-09-15T01:35:28.575953+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=28, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.905 INFO:tasks.ceph:pgid 9.0 last_scrub_stamp 2024-09-15T01:35:27.569993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=35, tm_sec=27, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.905 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T01:31:59.015115+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=31, tm_sec=59, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.905 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-15T01:32:03.044904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=32, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.905 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T01:32:03.044904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=32, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.905 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-15T01:32:03.044904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=32, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.905 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T01:32:03.044904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=32, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.906 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-15T01:32:03.044904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=32, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.906 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T01:32:03.044904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=32, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.906 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-15T01:32:03.044904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=32, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.906 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T01:32:03.044904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=32, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:06.906 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T01:50:26.907 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:50:27.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.006+0000 7f976159d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:27.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.006+0000 7f976159d640 10 monclient: build_initial_monmap 2024-09-15T01:50:27.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.006+0000 7f976159d640 10 monclient: monmap: 2024-09-15T01:50:27.008 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:27.007138+0000 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:27.007138+0000 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:27.009 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:27.010 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:27.010 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:27.010 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:27.010 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:27.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.006+0000 7f976159d640 1 Processor -- start 2024-09-15T01:50:27.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 1 -- start start 2024-09-15T01:50:27.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 10 monclient: init 2024-09-15T01:50:27.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 10 monclient: _add_conns ranks=[9,5,6,0,10,2,7,4,1,3,8] 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f975c0678d0 0x7f975c09e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 10 monclient(hunting): picked mon.noname-a con 0x7f975c0678d0 addr v2:172.21.15.178:3300/0 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f975c06ea30 0x7f975c09ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 10 monclient(hunting): picked mon.noname-h con 0x7f975c06ea30 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f975c09f180 0x7f975c1306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 10 monclient(hunting): picked mon.noname-d con 0x7f975c09f180 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975c09f5c0 con 0x7f975c0678d0 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f975c06d5b0 con 0x7f975c06ea30 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f975c067ca0 con 0x7f975c09f180 2024-09-15T01:50:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f976159d640 10 monclient(hunting): authenticate will time out at 1831.757638s 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975affd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f975c0678d0 0x7f975c09e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975b7fe640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f975c09f180 0x7f975c1306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975affd640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f975c0678d0 0x7f975c09e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47624/0 (socket says 172.21.15.178:47624) 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975affd640 1 -- 172.21.15.178:0/3974816552 learned_addr learned my addr 172.21.15.178:0/3974816552 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975b7fe640 10 monclient(hunting): get_auth_request con 0x7f975c09f180 auth_method 0 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975affd640 10 monclient(hunting): get_auth_request con 0x7f975c0678d0 auth_method 0 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f975b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f9753fff640 1 --2- 172.21.15.178:0/3974816552 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f975c06ea30 0x7f975c09ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f9753fff640 10 monclient(hunting): get_auth_request con 0x7f975c06ea30 auth_method 0 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f9753fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f9753fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.007+0000 7f9753fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975affd640 10 monclient(hunting): handle_auth_done global_id 104292 payload 274 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975affd640 1 -- 172.21.15.178:0/3974816552 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f975c09f180 msgr2=0x7f975c1306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975affd640 1 --2- 172.21.15.178:0/3974816552 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f975c09f180 0x7f975c1306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975affd640 1 -- 172.21.15.178:0/3974816552 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f975c06ea30 msgr2=0x7f975c09ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975affd640 1 --2- 172.21.15.178:0/3974816552 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f975c06ea30 0x7f975c09ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975affd640 10 monclient: _finish_hunting 0 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975affd640 1 monclient: found mon.noname-a 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975affd640 1 -- 172.21.15.178:0/3974816552 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c04bf60 con 0x7f975c0678d0 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975b7fe640 1 --2- 172.21.15.178:0/3974816552 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f975c09f180 0x7f975c1306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975affd640 1 --2- 172.21.15.178:0/3974816552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f975c0678d0 0x7f975c09e700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f974c009910 tx=0x7f974c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f468d48dcb15e7e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975a7fc640 1 -- 172.21.15.178:0/3974816552 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f974c02ad20 con 0x7f975c0678d0 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975a7fc640 10 monclient: dump: 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:27.024 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:27.024 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:27.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975a7fc640 10 monclient: _finish_auth 0 2024-09-15T01:50:27.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:27.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975a7fc640 1 -- 172.21.15.178:0/3974816552 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c031820 con 0x7f975c0678d0 2024-09-15T01:50:27.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f976159d640 5 monclient: authenticate success, global_id 104292 2024-09-15T01:50:27.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f976159d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:27.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:27.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f976159d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:27.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975a7fc640 1 -- 172.21.15.178:0/3974816552 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f974c02aea0 con 0x7f975c0678d0 2024-09-15T01:50:27.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:27.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f976159d640 4 set_mon_vals no callback set 2024-09-15T01:50:27.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T01:50:27.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.008+0000 7f975a7fc640 10 monclient: dump: 2024-09-15T01:50:27.025 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:27.025 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:27.025 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.009+0000 7f976159d640 10 monclient: shutdown 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.009+0000 7f976159d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.009+0000 7f976159d640 1 -- 172.21.15.178:0/3974816552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f975c0678d0 msgr2=0x7f975c09e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.009+0000 7f976159d640 1 --2- 172.21.15.178:0/3974816552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f975c0678d0 0x7f975c09e700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f974c009910 tx=0x7f974c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.009+0000 7f976159d640 1 -- 172.21.15.178:0/3974816552 shutdown_connections 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.009+0000 7f976159d640 1 --2- 172.21.15.178:0/3974816552 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f975c09f180 0x7f975c1306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.009+0000 7f976159d640 1 --2- 172.21.15.178:0/3974816552 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f975c06ea30 0x7f975c09ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.009+0000 7f976159d640 1 --2- 172.21.15.178:0/3974816552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f975c0678d0 0x7f975c09e700 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.009+0000 7f976159d640 1 -- 172.21.15.178:0/3974816552 >> 172.21.15.178:0/3974816552 conn(0x7f975c0a4390 msgr2=0x7f975c067130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.009+0000 7f976159d640 1 -- 172.21.15.178:0/3974816552 shutdown_connections 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.009+0000 7f976159d640 1 -- 172.21.15.178:0/3974816552 wait complete. 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient: build_initial_monmap 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient: monmap: 2024-09-15T01:50:27.027 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:27.028 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 1 Processor -- start 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 1 -- start start 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient: init 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient: _add_conns ranks=[8,4,5,1,6,2,3,7,0] 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f975c0678d0 0x7f975c13e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient(hunting): picked mon.noname-e con 0x7f975c0678d0 addr [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f975c06ea30 0x7f975c13d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient(hunting): picked mon.noname-c con 0x7f975c06ea30 addr [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f975c09f180 0x7f975c141880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient(hunting): picked mon.noname-h con 0x7f975c09f180 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:27.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:27.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 1 -- --> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f975c06c900 con 0x7f975c06ea30 2024-09-15T01:50:27.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:27.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f975c069e00 con 0x7f975c09f180 2024-09-15T01:50:27.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:27.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 1 -- --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f975c122a80 con 0x7f975c0678d0 2024-09-15T01:50:27.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:27.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f976159d640 10 monclient(hunting): authenticate will time out at 1831.760891s 2024-09-15T01:50:27.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.010+0000 7f975affd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f975c0678d0 0x7f975c13e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:27.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9753fff640 1 --2- >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f975c06ea30 0x7f975c13d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:27.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 1 --2- >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f975c0678d0 0x7f975c13e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3304/0 says I am v2:172.21.15.178:42370/0 (socket says 172.21.15.178:42370) 2024-09-15T01:50:27.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 1 -- 172.21.15.178:0/800683222 learned_addr learned my addr 172.21.15.178:0/800683222 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:27.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 10 monclient(hunting): get_auth_request con 0x7f975c0678d0 auth_method 0 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9753fff640 10 monclient(hunting): get_auth_request con 0x7f975c06ea30 auth_method 0 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9753fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9753fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9753fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975b7fe640 1 --2- 172.21.15.178:0/800683222 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f975c09f180 0x7f975c141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975b7fe640 10 monclient(hunting): get_auth_request con 0x7f975c09f180 auth_method 0 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:27.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:27.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:27.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:27.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9753fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:27.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9753fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:27.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9753fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:27.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:27.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:27.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:27.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 10 monclient(hunting): handle_auth_done global_id 94616 payload 824 2024-09-15T01:50:27.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 1 -- 172.21.15.178:0/800683222 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f975c09f180 msgr2=0x7f975c141880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:27.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 1 --2- 172.21.15.178:0/800683222 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f975c09f180 0x7f975c141880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 1 -- 172.21.15.178:0/800683222 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f975c06ea30 msgr2=0x7f975c13d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 1 --2- 172.21.15.178:0/800683222 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f975c06ea30 0x7f975c13d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 10 monclient: _finish_hunting 0 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 1 monclient: found mon.noname-e 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.178:3304/0 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 1 -- 172.21.15.178:0/800683222 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9754009a20 con 0x7f975c0678d0 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9753fff640 1 --2- 172.21.15.178:0/800683222 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f975c06ea30 0x7f975c13d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f975affd640 1 --2- 172.21.15.178:0/800683222 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f975c0678d0 0x7f975c13e330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f974c000c00 tx=0x7f974c032000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9752ffd640 1 -- 172.21.15.178:0/800683222 <== mon.8 v2:172.21.15.178:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f974c033d30 con 0x7f975c0678d0 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9752ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9752ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9752ffd640 10 monclient: dump: 2024-09-15T01:50:27.033 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:27.034 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:27.034 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:27.034 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:27.034 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:27.034 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:27.034 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:27.034 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:27.034 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:27.034 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:27.034 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:27.034 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:27.034 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:27.035 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:27.035 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:27.035 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:27.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9752ffd640 10 monclient: _finish_auth 0 2024-09-15T01:50:27.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9752ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:27.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9752ffd640 1 -- 172.21.15.178:0/800683222 <== mon.8 v2:172.21.15.178:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c03b3f0 con 0x7f975c0678d0 2024-09-15T01:50:27.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9752ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:27.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9752ffd640 1 -- 172.21.15.178:0/800683222 <== mon.8 v2:172.21.15.178:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f974c03b9e0 con 0x7f975c0678d0 2024-09-15T01:50:27.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f976159d640 5 monclient: authenticate success, global_id 94616 2024-09-15T01:50:27.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9752ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9752ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T01:50:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9759ffb640 4 set_mon_vals no callback set 2024-09-15T01:50:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f9752ffd640 10 monclient: dump: 2024-09-15T01:50:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f976159d640 10 monclient: _renew_subs 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f976159d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f976159d640 1 -- 172.21.15.178:0/800683222 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c009450 con 0x7f975c0678d0 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f976159d640 10 monclient: _renew_subs 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f976159d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:27.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.011+0000 7f976159d640 1 -- 172.21.15.178:0/800683222 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975c13ee40 con 0x7f975c0678d0 2024-09-15T01:50:27.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.012+0000 7f9752ffd640 1 -- 172.21.15.178:0/800683222 <== mon.8 v2:172.21.15.178:3304/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f974c039040 con 0x7f975c0678d0 2024-09-15T01:50:27.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.012+0000 7f976159d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:27.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.012+0000 7f976159d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:27.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.012+0000 7f976159d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.178:3304/0 2024-09-15T01:50:27.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.012+0000 7f976159d640 1 -- 172.21.15.178:0/800683222 --> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975c05ed70 con 0x7f975c0678d0 2024-09-15T01:50:27.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.013+0000 7f9752ffd640 1 --2- 172.21.15.178:0/800683222 >> v2:172.21.15.204:6812/28210 conn(0x7f973c0397b0 0x7f973c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:27.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.013+0000 7f9752ffd640 1 -- 172.21.15.178:0/800683222 <== mon.8 v2:172.21.15.178:3304/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f974c06b5a0 con 0x7f975c0678d0 2024-09-15T01:50:27.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.013+0000 7f9753fff640 1 --2- 172.21.15.178:0/800683222 >> v2:172.21.15.204:6812/28210 conn(0x7f973c0397b0 0x7f973c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:27.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.013+0000 7f9753fff640 10 monclient: get_auth_request con 0x7f973c0397b0 auth_method 0 2024-09-15T01:50:27.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.013+0000 7f9753fff640 1 --2- 172.21.15.178:0/800683222 >> v2:172.21.15.204:6812/28210 conn(0x7f973c0397b0 0x7f973c03bc60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f975c068240 tx=0x7f9744008040 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:27.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.017+0000 7f9752ffd640 1 -- 172.21.15.178:0/800683222 <== mon.8 v2:172.21.15.178:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f974c0434e0 con 0x7f975c0678d0 2024-09-15T01:50:27.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.017+0000 7f9752ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:27.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.017+0000 7f9752ffd640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:27.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.171+0000 7f976159d640 1 -- 172.21.15.178:0/800683222 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f975c04c4a0 con 0x7f973c0397b0 2024-09-15T01:50:27.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.174+0000 7f9752ffd640 1 -- 172.21.15.178:0/800683222 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+93751 (secure 0 0 0) 0x7f975c04c4a0 con 0x7f973c0397b0 2024-09-15T01:50:27.177 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:50:27.177 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-09-15T01:50:27.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 1 -- 172.21.15.178:0/800683222 >> v2:172.21.15.204:6812/28210 conn(0x7f973c0397b0 msgr2=0x7f973c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:27.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 1 --2- 172.21.15.178:0/800683222 >> v2:172.21.15.204:6812/28210 conn(0x7f973c0397b0 0x7f973c03bc60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f975c068240 tx=0x7f9744008040 comp rx=0 tx=0).stop 2024-09-15T01:50:27.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 10 monclient: shutdown 2024-09-15T01:50:27.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:27.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 1 -- 172.21.15.178:0/800683222 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f975c0678d0 msgr2=0x7f975c13e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:27.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 1 --2- 172.21.15.178:0/800683222 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f975c0678d0 0x7f975c13e330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f974c000c00 tx=0x7f974c032000 comp rx=0 tx=0).stop 2024-09-15T01:50:27.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 1 -- 172.21.15.178:0/800683222 shutdown_connections 2024-09-15T01:50:27.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 1 --2- 172.21.15.178:0/800683222 >> v2:172.21.15.204:6812/28210 conn(0x7f973c0397b0 0x7f973c03bc60 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:27.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 1 --2- 172.21.15.178:0/800683222 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f975c09f180 0x7f975c141880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:27.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 1 --2- 172.21.15.178:0/800683222 >> [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] conn(0x7f975c06ea30 0x7f975c13d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:27.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 1 --2- 172.21.15.178:0/800683222 >> [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] conn(0x7f975c0678d0 0x7f975c13e330 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:27.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 1 -- 172.21.15.178:0/800683222 >> 172.21.15.178:0/800683222 conn(0x7f975c0a4390 msgr2=0x7f975c09c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:27.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 1 -- 172.21.15.178:0/800683222 shutdown_connections 2024-09-15T01:50:27.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:27.176+0000 7f976159d640 1 -- 172.21.15.178:0/800683222 wait complete. 2024-09-15T01:50:27.189 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":813,"stamp":"2024-09-15T01:50:26.391773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562077696,"kb_used":4795164,"kb_used_data":557282552,"kb_used_omap":18974,"kb_used_meta":0,"kb_avail":557282532,"statfs":{"total":575567560704,"available":570657312768,"internally_reserved":20480,"allocated":570657333248,"data_stored":570657333248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19430356,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002823"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":97,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:11.715566+0000","last_change":"2024-09-15T01:50:11.715534+0000","last_active":"2024-09-15T01:50:11.715566+0000","last_peered":"2024-09-15T01:50:11.715566+0000","last_clean":"2024-09-15T01:50:11.715566+0000","last_became_active":"2024-09-15T01:36:25.227161+0000","last_became_peered":"2024-09-15T01:36:25.227161+0000","last_unstale":"2024-09-15T01:50:11.715566+0000","last_undegraded":"2024-09-15T01:50:11.715566+0000","last_fullsized":"2024-09-15T01:50:11.715566+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:11.715465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:11.715465+0000","last_clean_scrub_stamp":"2024-09-15T01:50:11.715465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:37:29.706523+0000","scrub_duration":0.0020818830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":108,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:12.750058+0000","last_change":"2024-09-15T01:50:12.750035+0000","last_active":"2024-09-15T01:50:12.750058+0000","last_peered":"2024-09-15T01:50:12.750058+0000","last_clean":"2024-09-15T01:50:12.750058+0000","last_became_active":"2024-09-15T01:36:25.228216+0000","last_became_peered":"2024-09-15T01:36:25.228216+0000","last_unstale":"2024-09-15T01:50:12.750058+0000","last_undegraded":"2024-09-15T01:50:12.750058+0000","last_fullsized":"2024-09-15T01:50:12.750058+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:12.749978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:12.749978+0000","last_clean_scrub_stamp":"2024-09-15T01:50:12.749978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:41:18.296244+0000","scrub_duration":0.0032490700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":97,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:07.674817+0000","last_change":"2024-09-15T01:50:07.674785+0000","last_active":"2024-09-15T01:50:07.674817+0000","last_peered":"2024-09-15T01:50:07.674817+0000","last_clean":"2024-09-15T01:50:07.674817+0000","last_became_active":"2024-09-15T01:36:25.228024+0000","last_became_peered":"2024-09-15T01:36:25.228024+0000","last_unstale":"2024-09-15T01:50:07.674817+0000","last_undegraded":"2024-09-15T01:50:07.674817+0000","last_fullsized":"2024-09-15T01:50:07.674817+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:07.674727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:07.674727+0000","last_clean_scrub_stamp":"2024-09-15T01:50:07.674727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:18:11.325944+0000","scrub_duration":0.0015531200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":108,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:06.778196+0000","last_change":"2024-09-15T01:50:06.778167+0000","last_active":"2024-09-15T01:50:06.778196+0000","last_peered":"2024-09-15T01:50:06.778196+0000","last_clean":"2024-09-15T01:50:06.778196+0000","last_became_active":"2024-09-15T01:36:25.229786+0000","last_became_peered":"2024-09-15T01:36:25.229786+0000","last_unstale":"2024-09-15T01:50:06.778196+0000","last_undegraded":"2024-09-15T01:50:06.778196+0000","last_fullsized":"2024-09-15T01:50:06.778196+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:06.778081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:06.778081+0000","last_clean_scrub_stamp":"2024-09-15T01:50:06.778081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:41:05.421522+0000","scrub_duration":0.001913238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":103,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:13.118106+0000","last_change":"2024-09-15T01:50:13.118083+0000","last_active":"2024-09-15T01:50:13.118106+0000","last_peered":"2024-09-15T01:50:13.118106+0000","last_clean":"2024-09-15T01:50:13.118106+0000","last_became_active":"2024-09-15T01:36:25.232778+0000","last_became_peered":"2024-09-15T01:36:25.232778+0000","last_unstale":"2024-09-15T01:50:13.118106+0000","last_undegraded":"2024-09-15T01:50:13.118106+0000","last_fullsized":"2024-09-15T01:50:13.118106+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:13.118031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:13.118031+0000","last_clean_scrub_stamp":"2024-09-15T01:50:13.118031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:32:28.135514+0000","scrub_duration":0.001530794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":97,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:10.431074+0000","last_change":"2024-09-15T01:50:10.431048+0000","last_active":"2024-09-15T01:50:10.431074+0000","last_peered":"2024-09-15T01:50:10.431074+0000","last_clean":"2024-09-15T01:50:10.431074+0000","last_became_active":"2024-09-15T01:36:25.227142+0000","last_became_peered":"2024-09-15T01:36:25.227142+0000","last_unstale":"2024-09-15T01:50:10.431074+0000","last_undegraded":"2024-09-15T01:50:10.431074+0000","last_fullsized":"2024-09-15T01:50:10.431074+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:10.430987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:10.430987+0000","last_clean_scrub_stamp":"2024-09-15T01:50:10.430987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:44:34.394061+0000","scrub_duration":0.001846798,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":181,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:15.955104+0000","last_change":"2024-09-15T01:50:15.955069+0000","last_active":"2024-09-15T01:50:15.955104+0000","last_peered":"2024-09-15T01:50:15.955104+0000","last_clean":"2024-09-15T01:50:15.955104+0000","last_became_active":"2024-09-15T01:36:25.230723+0000","last_became_peered":"2024-09-15T01:36:25.230723+0000","last_unstale":"2024-09-15T01:50:15.955104+0000","last_undegraded":"2024-09-15T01:50:15.955104+0000","last_fullsized":"2024-09-15T01:50:15.955104+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:15.955008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:15.955008+0000","last_clean_scrub_stamp":"2024-09-15T01:50:15.955008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:56:26.610655+0000","scrub_duration":0.0016696580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":99,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:15.314924+0000","last_change":"2024-09-15T01:50:15.314899+0000","last_active":"2024-09-15T01:50:15.314924+0000","last_peered":"2024-09-15T01:50:15.314924+0000","last_clean":"2024-09-15T01:50:15.314924+0000","last_became_active":"2024-09-15T01:36:25.228252+0000","last_became_peered":"2024-09-15T01:36:25.228252+0000","last_unstale":"2024-09-15T01:50:15.314924+0000","last_undegraded":"2024-09-15T01:50:15.314924+0000","last_fullsized":"2024-09-15T01:50:15.314924+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:15.314836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:15.314836+0000","last_clean_scrub_stamp":"2024-09-15T01:50:15.314836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:35:59.924412+0000","scrub_duration":0.0018829020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":109,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:05.817753+0000","last_change":"2024-09-15T01:50:05.817732+0000","last_active":"2024-09-15T01:50:05.817753+0000","last_peered":"2024-09-15T01:50:05.817753+0000","last_clean":"2024-09-15T01:50:05.817753+0000","last_became_active":"2024-09-15T01:36:25.226915+0000","last_became_peered":"2024-09-15T01:36:25.226915+0000","last_unstale":"2024-09-15T01:50:05.817753+0000","last_undegraded":"2024-09-15T01:50:05.817753+0000","last_fullsized":"2024-09-15T01:50:05.817753+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:05.817683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:05.817683+0000","last_clean_scrub_stamp":"2024-09-15T01:50:05.817683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:21:15.575297+0000","scrub_duration":0.0015299910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":129,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:09.144108+0000","last_change":"2024-09-15T01:50:09.144081+0000","last_active":"2024-09-15T01:50:09.144108+0000","last_peered":"2024-09-15T01:50:09.144108+0000","last_clean":"2024-09-15T01:50:09.144108+0000","last_became_active":"2024-09-15T01:36:25.229665+0000","last_became_peered":"2024-09-15T01:36:25.229665+0000","last_unstale":"2024-09-15T01:50:09.144108+0000","last_undegraded":"2024-09-15T01:50:09.144108+0000","last_fullsized":"2024-09-15T01:50:09.144108+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:09.144016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:09.144016+0000","last_clean_scrub_stamp":"2024-09-15T01:50:09.144016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:01:59.924448+0000","scrub_duration":0.0020523759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":187,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:10.048231+0000","last_change":"2024-09-15T01:50:10.048199+0000","last_active":"2024-09-15T01:50:10.048231+0000","last_peered":"2024-09-15T01:50:10.048231+0000","last_clean":"2024-09-15T01:50:10.048231+0000","last_became_active":"2024-09-15T01:36:25.227397+0000","last_became_peered":"2024-09-15T01:36:25.227397+0000","last_unstale":"2024-09-15T01:50:10.048231+0000","last_undegraded":"2024-09-15T01:50:10.048231+0000","last_fullsized":"2024-09-15T01:50:10.048231+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:10.048137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:10.048137+0000","last_clean_scrub_stamp":"2024-09-15T01:50:10.048137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:18:37.403671+0000","scrub_duration":0.0017503429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":99,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:13.281646+0000","last_change":"2024-09-15T01:50:13.281627+0000","last_active":"2024-09-15T01:50:13.281646+0000","last_peered":"2024-09-15T01:50:13.281646+0000","last_clean":"2024-09-15T01:50:13.281646+0000","last_became_active":"2024-09-15T01:36:25.226969+0000","last_became_peered":"2024-09-15T01:36:25.226969+0000","last_unstale":"2024-09-15T01:50:13.281646+0000","last_undegraded":"2024-09-15T01:50:13.281646+0000","last_fullsized":"2024-09-15T01:50:13.281646+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:13.281577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:13.281577+0000","last_clean_scrub_stamp":"2024-09-15T01:50:13.281577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:20:18.111636+0000","scrub_duration":0.0011427259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":97,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:12.438021+0000","last_change":"2024-09-15T01:50:12.437972+0000","last_active":"2024-09-15T01:50:12.438021+0000","last_peered":"2024-09-15T01:50:12.438021+0000","last_clean":"2024-09-15T01:50:12.438021+0000","last_became_active":"2024-09-15T01:36:26.228683+0000","last_became_peered":"2024-09-15T01:36:26.228683+0000","last_unstale":"2024-09-15T01:50:12.438021+0000","last_undegraded":"2024-09-15T01:50:12.438021+0000","last_fullsized":"2024-09-15T01:50:12.438021+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:12.437908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:12.437908+0000","last_clean_scrub_stamp":"2024-09-15T01:50:12.437908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:50:28.490080+0000","scrub_duration":0.001907688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":98,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:16.278045+0000","last_change":"2024-09-15T01:50:16.278015+0000","last_active":"2024-09-15T01:50:16.278045+0000","last_peered":"2024-09-15T01:50:16.278045+0000","last_clean":"2024-09-15T01:50:16.278045+0000","last_became_active":"2024-09-15T01:36:26.228429+0000","last_became_peered":"2024-09-15T01:36:26.228429+0000","last_unstale":"2024-09-15T01:50:16.278045+0000","last_undegraded":"2024-09-15T01:50:16.278045+0000","last_fullsized":"2024-09-15T01:50:16.278045+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:16.277912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:16.277912+0000","last_clean_scrub_stamp":"2024-09-15T01:50:16.277912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:25:44.185617+0000","scrub_duration":0.001927944,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":117,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:15.680464+0000","last_change":"2024-09-15T01:50:15.680421+0000","last_active":"2024-09-15T01:50:15.680464+0000","last_peered":"2024-09-15T01:50:15.680464+0000","last_clean":"2024-09-15T01:50:15.680464+0000","last_became_active":"2024-09-15T01:36:26.225738+0000","last_became_peered":"2024-09-15T01:36:26.225738+0000","last_unstale":"2024-09-15T01:50:15.680464+0000","last_undegraded":"2024-09-15T01:50:15.680464+0000","last_fullsized":"2024-09-15T01:50:15.680464+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:15.680362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:15.680362+0000","last_clean_scrub_stamp":"2024-09-15T01:50:15.680362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:01:48.118870+0000","scrub_duration":0.001709716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":97,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:09.410437+0000","last_change":"2024-09-15T01:50:09.410410+0000","last_active":"2024-09-15T01:50:09.410437+0000","last_peered":"2024-09-15T01:50:09.410437+0000","last_clean":"2024-09-15T01:50:09.410437+0000","last_became_active":"2024-09-15T01:36:25.225608+0000","last_became_peered":"2024-09-15T01:36:25.225608+0000","last_unstale":"2024-09-15T01:50:09.410437+0000","last_undegraded":"2024-09-15T01:50:09.410437+0000","last_fullsized":"2024-09-15T01:50:09.410437+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:09.410354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:09.410354+0000","last_clean_scrub_stamp":"2024-09-15T01:50:09.410354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:15.825237+0000","scrub_duration":0.001899525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":101,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:44:29.159148+0000","last_change":"2024-09-15T01:36:25.260030+0000","last_active":"2024-09-15T01:44:29.159148+0000","last_peered":"2024-09-15T01:44:29.159148+0000","last_clean":"2024-09-15T01:44:29.159148+0000","last_became_active":"2024-09-15T01:36:25.233270+0000","last_became_peered":"2024-09-15T01:36:25.233270+0000","last_unstale":"2024-09-15T01:44:29.159148+0000","last_undegraded":"2024-09-15T01:44:29.159148+0000","last_fullsized":"2024-09-15T01:44:29.159148+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:31:59.015115+0000","last_clean_scrub_stamp":"2024-09-15T01:31:59.015115+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"265'2","reported_seq":270,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:20.093946+0000","last_change":"2024-09-15T01:50:20.093921+0000","last_active":"2024-09-15T01:50:20.093946+0000","last_peered":"2024-09-15T01:50:20.093946+0000","last_clean":"2024-09-15T01:50:20.093946+0000","last_became_active":"2024-09-15T01:36:25.230032+0000","last_became_peered":"2024-09-15T01:36:25.230032+0000","last_unstale":"2024-09-15T01:50:20.093946+0000","last_undegraded":"2024-09-15T01:50:20.093946+0000","last_fullsized":"2024-09-15T01:50:20.093946+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-09-15T01:50:20.093859+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-09-15T01:50:20.093859+0000","last_clean_scrub_stamp":"2024-09-15T01:50:20.093859+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:03.602901+0000","scrub_duration":0.002496683,"objects_trimmed":0,"snaptrim_duration":0.00076976900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:07.819352+0000","last_change":"2024-09-15T01:50:07.819333+0000","last_active":"2024-09-15T01:50:07.819352+0000","last_peered":"2024-09-15T01:50:07.819352+0000","last_clean":"2024-09-15T01:50:07.819352+0000","last_became_active":"2024-09-15T01:36:25.226499+0000","last_became_peered":"2024-09-15T01:36:25.226499+0000","last_unstale":"2024-09-15T01:50:07.819352+0000","last_undegraded":"2024-09-15T01:50:07.819352+0000","last_fullsized":"2024-09-15T01:50:07.819352+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:07.819277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:07.819277+0000","last_clean_scrub_stamp":"2024-09-15T01:50:07.819277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:23:42.579404+0000","scrub_duration":0.001625727,"objects_trimmed":0,"snaptrim_duration":0.00037511000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":99,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:06.227796+0000","last_change":"2024-09-15T01:50:06.227770+0000","last_active":"2024-09-15T01:50:06.227796+0000","last_peered":"2024-09-15T01:50:06.227796+0000","last_clean":"2024-09-15T01:50:06.227796+0000","last_became_active":"2024-09-15T01:36:25.231212+0000","last_became_peered":"2024-09-15T01:36:25.231212+0000","last_unstale":"2024-09-15T01:50:06.227796+0000","last_undegraded":"2024-09-15T01:50:06.227796+0000","last_fullsized":"2024-09-15T01:50:06.227796+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:06.227713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:06.227713+0000","last_clean_scrub_stamp":"2024-09-15T01:50:06.227713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:41:46.522757+0000","scrub_duration":0.001627975,"objects_trimmed":0,"snaptrim_duration":0.00046258900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"265'3","reported_seq":308,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:06.028446+0000","last_change":"2024-09-15T01:50:06.028427+0000","last_active":"2024-09-15T01:50:06.028446+0000","last_peered":"2024-09-15T01:50:06.028446+0000","last_clean":"2024-09-15T01:50:06.028446+0000","last_became_active":"2024-09-15T01:36:25.226718+0000","last_became_peered":"2024-09-15T01:36:25.226718+0000","last_unstale":"2024-09-15T01:50:06.028446+0000","last_undegraded":"2024-09-15T01:50:06.028446+0000","last_fullsized":"2024-09-15T01:50:06.028446+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-09-15T01:50:06.028377+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-09-15T01:50:06.028377+0000","last_clean_scrub_stamp":"2024-09-15T01:50:06.028377+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:02:09.417710+0000","scrub_duration":0.0030518149999999998,"objects_trimmed":0,"snaptrim_duration":0.00093022000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":266,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:08.387173+0000","last_change":"2024-09-15T01:50:08.387134+0000","last_active":"2024-09-15T01:50:08.387173+0000","last_peered":"2024-09-15T01:50:08.387173+0000","last_clean":"2024-09-15T01:50:08.387173+0000","last_became_active":"2024-09-15T01:36:25.227361+0000","last_became_peered":"2024-09-15T01:36:25.227361+0000","last_unstale":"2024-09-15T01:50:08.387173+0000","last_undegraded":"2024-09-15T01:50:08.387173+0000","last_fullsized":"2024-09-15T01:50:08.387173+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:08.387067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:08.387067+0000","last_clean_scrub_stamp":"2024-09-15T01:50:08.387067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:23.598441+0000","scrub_duration":0.0014306060000000001,"objects_trimmed":0,"snaptrim_duration":0.00031034999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":102,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:15.113653+0000","last_change":"2024-09-15T01:50:15.113631+0000","last_active":"2024-09-15T01:50:15.113653+0000","last_peered":"2024-09-15T01:50:15.113653+0000","last_clean":"2024-09-15T01:50:15.113653+0000","last_became_active":"2024-09-15T01:36:25.229674+0000","last_became_peered":"2024-09-15T01:36:25.229674+0000","last_unstale":"2024-09-15T01:50:15.113653+0000","last_undegraded":"2024-09-15T01:50:15.113653+0000","last_fullsized":"2024-09-15T01:50:15.113653+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:15.113566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:15.113566+0000","last_clean_scrub_stamp":"2024-09-15T01:50:15.113566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:13:40.272496+0000","scrub_duration":0.0018433779999999999,"objects_trimmed":0,"snaptrim_duration":0.00077134000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":96,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:13.655562+0000","last_change":"2024-09-15T01:50:13.655541+0000","last_active":"2024-09-15T01:50:13.655562+0000","last_peered":"2024-09-15T01:50:13.655562+0000","last_clean":"2024-09-15T01:50:13.655562+0000","last_became_active":"2024-09-15T01:36:25.225361+0000","last_became_peered":"2024-09-15T01:36:25.225361+0000","last_unstale":"2024-09-15T01:50:13.655562+0000","last_undegraded":"2024-09-15T01:50:13.655562+0000","last_fullsized":"2024-09-15T01:50:13.655562+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:13.655486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:13.655486+0000","last_clean_scrub_stamp":"2024-09-15T01:50:13.655486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:23:15.849882+0000","scrub_duration":0.001290636,"objects_trimmed":0,"snaptrim_duration":0.00041942500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":246,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:44:29.159766+0000","last_change":"2024-09-15T01:36:26.437860+0000","last_active":"2024-09-15T01:44:29.159766+0000","last_peered":"2024-09-15T01:44:29.159766+0000","last_clean":"2024-09-15T01:44:29.159766+0000","last_became_active":"2024-09-15T01:36:26.437594+0000","last_became_peered":"2024-09-15T01:36:26.437594+0000","last_unstale":"2024-09-15T01:44:29.159766+0000","last_undegraded":"2024-09-15T01:44:29.159766+0000","last_fullsized":"2024-09-15T01:44:29.159766+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:32:03.044904+0000","last_clean_scrub_stamp":"2024-09-15T01:32:03.044904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065783200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640479,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799576,"kb_used_data":92880040,"kb_used_omap":3227,"kb_used_meta":0,"kb_avail":92880040,"statfs":{"total":95927926784,"available":95109160960,"internally_reserved":0,"allocated":95109160960,"data_stored":95109160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3305321,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.39500000000000002,"5min":0.28999999999999998,"15min":0.24299999999999999},"max":{"1min":0.65500000000000003,"5min":0.73499999999999999,"15min":0.95099999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59699999999999998,"5min":0.68100000000000005,"15min":0.76700000000000002},"last":0.55200000000000005}]},{"osd":1,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.28899999999999998,"15min":0.28299999999999997},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.91800000000000004},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.30499999999999999},"max":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.72899999999999998},"last":0.57799999999999996}]},{"osd":2,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.23300000000000001},"max":{"1min":0.629,"5min":0.69199999999999995,"15min":0.73399999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.50600000000000001},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.27700000000000002},"max":{"1min":0.629,"5min":0.67200000000000004,"15min":0.81000000000000005},"last":0.52700000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.27200000000000002,"5min":0.224,"15min":0.20699999999999999},"max":{"1min":0.52600000000000002,"5min":0.63400000000000001,"15min":0.96799999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.39700000000000002,"15min":0.38600000000000001},"min":{"1min":0.28199999999999997,"5min":0.23200000000000001,"15min":0.216},"max":{"1min":0.54700000000000004,"5min":0.64700000000000002,"15min":0.86199999999999999},"last":0.48599999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.378,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":1.0289999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40699999999999997,"15min":0.40300000000000002},"min":{"1min":0.26400000000000001,"5min":0.249,"15min":0.22},"max":{"1min":0.54300000000000004,"5min":0.61799999999999999,"15min":0.67200000000000004},"last":0.51100000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640479,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799000,"kb_used_data":92880616,"kb_used_omap":3271,"kb_used_meta":0,"kb_avail":92880616,"statfs":{"total":95927926784,"available":95109750784,"internally_reserved":0,"allocated":95109750784,"data_stored":95109750784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3350414,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.48999999999999999},"min":{"1min":0.39800000000000002,"5min":0.33500000000000002,"15min":0.219},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.39700000000000002,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.66600000000000004,"5min":0.69399999999999995,"15min":0.70799999999999996},"last":0.54500000000000004}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53600000000000003,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.25600000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.52800000000000002},"min":{"1min":0.47599999999999998,"5min":0.36499999999999999,"15min":0.28999999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.56799999999999995}]},{"osd":2,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.71599999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69199999999999995,"5min":0.876,"15min":0.876},"last":0.52100000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.38900000000000001,"15min":0.36699999999999999},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.22700000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.40699999999999997,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.216},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.3540000000000001},"last":0.49099999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.39600000000000002,"15min":0.376},"min":{"1min":0.27900000000000003,"5min":0.26600000000000001,"15min":0.22700000000000001},"max":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":1.2709999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.39600000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.24199999999999999},"max":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":1.2250000000000001},"last":0.505}]}]},{"osd":3,"up_from":11,"seq":47244640479,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798996,"kb_used_data":92880620,"kb_used_omap":3045,"kb_used_meta":0,"kb_avail":92880620,"statfs":{"total":95927926784,"available":95109754880,"internally_reserved":0,"allocated":95109754880,"data_stored":95109754880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3118296,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.24299999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.69399999999999995},"last":0.437},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.29899999999999999},"max":{"1min":0.64900000000000002,"5min":0.66400000000000003,"15min":0.81599999999999995},"last":0.51900000000000002}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.27900000000000003},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.79300000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.37,"5min":0.34399999999999997,"15min":0.29899999999999999},"max":{"1min":0.58799999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.74299999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.30099999999999999},"max":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.75800000000000001},"last":0.42499999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36099999999999999,"15min":0.35999999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.46200000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.61699999999999999,"5min":0.65900000000000003,"15min":0.70599999999999996},"last":0.375}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.53200000000000003,"5min":0.57099999999999995,"15min":0.65500000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.39000000000000001,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.49299999999999999,"5min":0.64600000000000002,"15min":0.70399999999999996},"last":0.34100000000000003}]}]},{"osd":2,"up_from":11,"seq":47244640479,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799004,"kb_used_data":92880620,"kb_used_omap":3065,"kb_used_meta":0,"kb_avail":92880612,"statfs":{"total":95927926784,"available":95109746688,"internally_reserved":8192,"allocated":95109754880,"data_stored":95109754880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3138640,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.32600000000000001},"min":{"1min":0.23200000000000001,"5min":0.20100000000000001,"15min":0.17499999999999999},"max":{"1min":0.47999999999999998,"5min":0.48499999999999999,"15min":0.53000000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.375,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.23999999999999999,"15min":0.20200000000000001},"max":{"1min":0.501,"5min":0.53000000000000003,"15min":0.59999999999999998},"last":0.38100000000000001}]},{"osd":1,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.33200000000000002,"15min":0.33500000000000002},"min":{"1min":0.20300000000000001,"5min":0.187,"15min":0.182},"max":{"1min":0.46100000000000002,"5min":0.52500000000000002,"15min":0.60599999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.373,"15min":0.373},"min":{"1min":0.24299999999999999,"5min":0.222,"15min":0.20699999999999999},"max":{"1min":0.53300000000000003,"5min":0.57099999999999995,"15min":0.67400000000000004},"last":0.36199999999999999}]},{"osd":3,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.28899999999999998},"max":{"1min":0.65000000000000002,"5min":0.75700000000000001,"15min":0.78900000000000003},"last":0.503},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61299999999999999,"5min":0.77900000000000003,"15min":1.0129999999999999},"last":0.498}]},{"osd":4,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.55400000000000005,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":0.63500000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.27600000000000002},"max":{"1min":0.59699999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44700000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56200000000000006,"15min":0.53100000000000003},"min":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.27800000000000002},"max":{"1min":0.67300000000000004,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.53900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.30099999999999999},"max":{"1min":0.623,"5min":0.75900000000000001,"15min":0.87},"last":0.47599999999999998}]}]},{"osd":1,"up_from":11,"seq":47244640480,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799588,"kb_used_data":92880040,"kb_used_omap":3065,"kb_used_meta":0,"kb_avail":92880028,"statfs":{"total":95927926784,"available":95109148672,"internally_reserved":12288,"allocated":95109160960,"data_stored":95109160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3138887,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33700000000000002,"15min":0.33300000000000002},"min":{"1min":0.26100000000000001,"5min":0.21099999999999999,"15min":0.188},"max":{"1min":0.47399999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.35499999999999998,"15min":0.35299999999999998},"min":{"1min":0.27800000000000002,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.68200000000000005},"last":0.41599999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.37},"min":{"1min":0.29799999999999999,"5min":0.245,"15min":0.23799999999999999},"max":{"1min":0.501,"5min":0.51400000000000001,"15min":0.73399999999999999},"last":0.498},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.28899999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.70299999999999996},"last":0.435}]},{"osd":3,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.46999999999999997,"5min":0.34499999999999997,"15min":0.30099999999999999},"max":{"1min":0.68700000000000006,"5min":0.72099999999999997,"15min":0.76800000000000002},"last":0.626},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.46800000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.76700000000000002},"last":0.61099999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.49099999999999999,"5min":0.40500000000000003,"15min":0.35599999999999998},"max":{"1min":0.70599999999999996,"5min":0.77700000000000002,"15min":0.80600000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.308},"max":{"1min":0.71699999999999997,"5min":0.76800000000000002,"15min":0.81899999999999995},"last":0.64000000000000001}]},{"osd":5,"last update":"Sun Sep 15 01:49:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.32600000000000001},"max":{"1min":0.69099999999999995,"5min":0.748,"15min":0.89400000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.53800000000000003},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.32200000000000001},"max":{"1min":0.64400000000000002,"5min":0.69699999999999995,"15min":0.81000000000000005},"last":0.54600000000000004}]}]},{"osd":0,"up_from":11,"seq":47244640479,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799000,"kb_used_data":92880616,"kb_used_omap":3299,"kb_used_meta":0,"kb_avail":92880616,"statfs":{"total":95927926784,"available":95109750784,"internally_reserved":0,"allocated":95109750784,"data_stored":95109750784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3378798,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32600000000000001,"15min":0.33100000000000002},"min":{"1min":0.22,"5min":0.188,"15min":0.157},"max":{"1min":0.503,"5min":0.52100000000000002,"15min":0.75600000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.376,"15min":0.379},"min":{"1min":0.254,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.47999999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41899999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34699999999999998,"15min":0.36899999999999999},"min":{"1min":0.25,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.46300000000000002,"5min":0.54800000000000004,"15min":0.69499999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.40000000000000002,"15min":0.38100000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.19},"max":{"1min":0.55100000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45700000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.28399999999999997},"max":{"1min":0.58299999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51300000000000001,"15min":0.51400000000000001},"min":{"1min":0.40500000000000003,"5min":0.30599999999999999,"15min":0.26400000000000001},"max":{"1min":0.64000000000000001,"5min":0.745,"15min":0.77400000000000002},"last":0.56499999999999995}]},{"osd":4,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.625,"5min":0.71799999999999997,"15min":0.72099999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.40699999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59699999999999998,"5min":0.69999999999999996,"15min":0.752},"last":0.52700000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54900000000000004},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65700000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.46800000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66900000000000004,"5min":0.73899999999999999,"15min":0.75600000000000001},"last":0.60599999999999998}]}]}],"pool_statfs":[]}} 2024-09-15T01:50:27.192 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T01:31:59.015115+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=31, tm_sec=59, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:27.192 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T01:32:03.044904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=32, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=1, tm_min=50, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T01:50:27.192 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T01:50:47.193 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T01:50:47.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.297+0000 7f9ffd39d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:47.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.297+0000 7f9ffd39d640 10 monclient: build_initial_monmap 2024-09-15T01:50:47.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.297+0000 7f9ffd39d640 10 monclient: monmap: 2024-09-15T01:50:47.300 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:47.298833+0000 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:47.298833+0000 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:47.301 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:47.302 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:47.302 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:47.302 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:47.302 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:47.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.298+0000 7f9ffd39d640 1 Processor -- start 2024-09-15T01:50:47.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.298+0000 7f9ffd39d640 1 -- start start 2024-09-15T01:50:47.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.298+0000 7f9ffd39d640 10 monclient: init 2024-09-15T01:50:47.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:47.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 10 monclient: _add_conns ranks=[6,9,5,7,4,1,10,8,2,0,3] 2024-09-15T01:50:47.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff80a2930 0x7f9ff8132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 10 monclient(hunting): picked mon.noname-d con 0x7f9ff80a2930 addr [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] 2024-09-15T01:50:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9ff809faa0 0x7f9ff809fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 10 monclient(hunting): picked mon.noname-a con 0x7f9ff809faa0 addr v2:172.21.15.178:3300/0 2024-09-15T01:50:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 1 --2- >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9ff80a08c0 0x7f9ff80a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 10 monclient(hunting): picked mon.noname-h con 0x7f9ff80a08c0 addr [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] 2024-09-15T01:50:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff809e0f0 con 0x7f9ff809faa0 2024-09-15T01:50:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:47.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 1 -- --> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff809dd80 con 0x7f9ff80a08c0 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 1 -- --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80a2d00 con 0x7f9ff80a2930 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ffd39d640 10 monclient(hunting): authenticate will time out at 1852.049112s 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff80a2930 0x7f9ff8132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9fee7fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9ff809faa0 0x7f9ff809fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9fee7fc640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9ff809faa0 0x7f9ff809fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:55468/0 (socket says 172.21.15.178:55468) 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 1 --2- >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff80a2930 0x7f9ff8132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3303/0 says I am v2:172.21.15.178:49892/0 (socket says 172.21.15.178:49892) 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9fee7fc640 1 -- 172.21.15.178:0/910120866 learned_addr learned my addr 172.21.15.178:0/910120866 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9fee7fc640 10 monclient(hunting): get_auth_request con 0x7f9ff809faa0 auth_method 0 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9fee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9fee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:47.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9fee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 10 monclient(hunting): get_auth_request con 0x7f9ff80a2930 auth_method 0 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9fee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9fee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9fee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff77fe640 1 --2- 172.21.15.178:0/910120866 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9ff80a08c0 0x7f9ff80a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff77fe640 10 monclient(hunting): get_auth_request con 0x7f9ff80a08c0 auth_method 0 2024-09-15T01:50:47.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 10 monclient(hunting): handle_auth_done global_id 26034 payload 274 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 1 -- 172.21.15.178:0/910120866 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9ff80a08c0 msgr2=0x7f9ff80a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 1 --2- 172.21.15.178:0/910120866 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9ff80a08c0 0x7f9ff80a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 1 -- 172.21.15.178:0/910120866 >> v2:172.21.15.178:3300/0 conn(0x7f9ff809faa0 msgr2=0x7f9ff809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 1 --2- 172.21.15.178:0/910120866 >> v2:172.21.15.178:3300/0 conn(0x7f9ff809faa0 0x7f9ff809fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 10 monclient: _finish_hunting 0 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 1 monclient: found mon.noname-d 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.178:3303/0 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9ff6ffd640 1 -- 172.21.15.178:0/910120866 --> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff804bf60 con 0x7f9ff80a2930 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.299+0000 7f9fee7fc640 1 --2- 172.21.15.178:0/910120866 >> v2:172.21.15.178:3300/0 conn(0x7f9ff809faa0 0x7f9ff809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:47.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff6ffd640 1 --2- 172.21.15.178:0/910120866 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff80a2930 0x7f9ff8132de0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009850 tx=0x7f9fe80090f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=884ad4ea3097a0fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff67fc640 1 -- 172.21.15.178:0/910120866 <== mon.6 v2:172.21.15.178:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9fe802abc0 con 0x7f9ff80a2930 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff67fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff67fc640 10 monclient: dump: 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:47.316 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff67fc640 10 monclient: _finish_auth 0 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff67fc640 1 -- 172.21.15.178:0/910120866 <== mon.6 v2:172.21.15.178:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe802ad40 con 0x7f9ff80a2930 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ffd39d640 5 monclient: authenticate success, global_id 26034 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ffd39d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:47.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ffd39d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff67fc640 1 -- 172.21.15.178:0/910120866 <== mon.6 v2:172.21.15.178:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9fe8031ca0 con 0x7f9ff80a2930 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ffd39d640 4 set_mon_vals no callback set 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.300+0000 7f9ff67fc640 10 monclient: dump: 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:47.318 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 10 monclient: shutdown 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 1 -- 172.21.15.178:0/910120866 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff80a2930 msgr2=0x7f9ff8132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 1 --2- 172.21.15.178:0/910120866 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff80a2930 0x7f9ff8132de0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009850 tx=0x7f9fe80090f0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 1 -- 172.21.15.178:0/910120866 shutdown_connections 2024-09-15T01:50:47.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 1 --2- 172.21.15.178:0/910120866 >> [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] conn(0x7f9ff80a08c0 0x7f9ff80a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 1 --2- 172.21.15.178:0/910120866 >> v2:172.21.15.178:3300/0 conn(0x7f9ff809faa0 0x7f9ff809fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 1 --2- 172.21.15.178:0/910120866 >> [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] conn(0x7f9ff80a2930 0x7f9ff8132de0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 1 -- 172.21.15.178:0/910120866 >> 172.21.15.178:0/910120866 conn(0x7f9ff80a5ed0 msgr2=0x7f9ff80a62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 1 -- 172.21.15.178:0/910120866 shutdown_connections 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 1 -- 172.21.15.178:0/910120866 wait complete. 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 10 monclient: build_initial_monmap 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.301+0000 7f9ffd39d640 10 monclient: monmap: 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:47.320 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 1 Processor -- start 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 1 -- start start 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 10 monclient: init 2024-09-15T01:50:47.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 10 monclient: _add_conns ranks=[1,3,2,7,5,8,0,4,6] 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ff809faa0 0x7f9ff809afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 10 monclient(hunting): picked mon.noname-f con 0x7f9ff809faa0 addr [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80a08c0 0x7f9ff8131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 10 monclient(hunting): picked mon.noname-g con 0x7f9ff80a08c0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80a2930 0x7f9ff811c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 10 monclient(hunting): picked mon.noname-b con 0x7f9ff80a2930 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80a2390 con 0x7f9ff809faa0 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff809d630 con 0x7f9ff80a2930 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff81429f0 con 0x7f9ff80a08c0 2024-09-15T01:50:47.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ffd39d640 10 monclient(hunting): authenticate will time out at 1852.052769s 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ff77fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80a2930 0x7f9ff811c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ff77fe640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80a2930 0x7f9ff811c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42804/0 (socket says 172.21.15.178:42804) 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ff77fe640 1 -- 172.21.15.178:0/2936648399 learned_addr learned my addr 172.21.15.178:0/2936648399 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ff77fe640 10 monclient(hunting): get_auth_request con 0x7f9ff80a2930 auth_method 0 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ff77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ff77fe640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ff77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9fee7fc640 1 --2- 172.21.15.178:0/2936648399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80a08c0 0x7f9ff8131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ff6ffd640 1 --2- 172.21.15.178:0/2936648399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ff809faa0 0x7f9ff809afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9fee7fc640 10 monclient(hunting): get_auth_request con 0x7f9ff80a08c0 auth_method 0 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9fee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:47.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9fee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9fee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ff6ffd640 10 monclient(hunting): get_auth_request con 0x7f9ff809faa0 auth_method 0 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ff6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ff6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.302+0000 7f9ff6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:47.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 10 monclient(hunting): handle_auth_done global_id 35606 payload 824 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 1 -- 172.21.15.178:0/2936648399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80a08c0 msgr2=0x7f9ff8131840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 1 --2- 172.21.15.178:0/2936648399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80a08c0 0x7f9ff8131840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 1 -- 172.21.15.178:0/2936648399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ff809faa0 msgr2=0x7f9ff809afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 1 --2- 172.21.15.178:0/2936648399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ff809faa0 0x7f9ff809afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 10 monclient: _finish_hunting 0 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 1 monclient: found mon.noname-b 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 1 -- 172.21.15.178:0/2936648399 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe0009450 con 0x7f9ff80a2930 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff77fe640 1 --2- 172.21.15.178:0/2936648399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80a2930 0x7f9ff811c530 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009fd0 tx=0x7f9ff0009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fef7fe640 1 -- 172.21.15.178:0/2936648399 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ff0009a40 con 0x7f9ff80a2930 2024-09-15T01:50:47.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fef7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fef7fe640 10 monclient: dump: 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:47.326 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fef7fe640 10 monclient: _finish_auth 0 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ffd39d640 5 monclient: authenticate success, global_id 35606 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fef7fe640 1 -- 172.21.15.178:0/2936648399 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff000ee90 con 0x7f9ff80a2930 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ffd39d640 10 monclient: _renew_subs 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ffd39d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ffd39d640 1 -- 172.21.15.178:0/2936648399 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe8009390 con 0x7f9ff80a2930 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fef7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:47.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fef7fe640 1 -- 172.21.15.178:0/2936648399 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ff0014d20 con 0x7f9ff80a2930 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fef7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9fef7fe640 10 monclient: dump: 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:47.328 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ff5ffb640 4 set_mon_vals no callback set 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ffd39d640 10 monclient: _renew_subs 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ffd39d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.303+0000 7f9ffd39d640 1 -- 172.21.15.178:0/2936648399 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff811e960 con 0x7f9ff80a2930 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.304+0000 7f9fef7fe640 1 -- 172.21.15.178:0/2936648399 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f9ff0014420 con 0x7f9ff80a2930 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.304+0000 7f9fef7fe640 1 --2- 172.21.15.178:0/2936648399 >> v2:172.21.15.204:6812/28210 conn(0x7f9fd80397e0 0x7f9fd803bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.304+0000 7f9fef7fe640 1 -- 172.21.15.178:0/2936648399 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f9ff004c310 con 0x7f9ff80a2930 2024-09-15T01:50:47.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.304+0000 7f9ffd39d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:47.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.304+0000 7f9ffd39d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:47.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.304+0000 7f9ffd39d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:47.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.304+0000 7f9ff6ffd640 1 --2- 172.21.15.178:0/2936648399 >> v2:172.21.15.204:6812/28210 conn(0x7f9fd80397e0 0x7f9fd803bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.304+0000 7f9ffd39d640 1 -- 172.21.15.178:0/2936648399 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff804b720 con 0x7f9ff80a2930 2024-09-15T01:50:47.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.304+0000 7f9ff6ffd640 10 monclient: get_auth_request con 0x7f9fd80397e0 auth_method 0 2024-09-15T01:50:47.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.305+0000 7f9ff6ffd640 1 --2- 172.21.15.178:0/2936648399 >> v2:172.21.15.204:6812/28210 conn(0x7f9fd80397e0 0x7f9fd803bc90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8034000 tx=0x7f9fe8032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:47.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.309+0000 7f9fef7fe640 1 -- 172.21.15.178:0/2936648399 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9ff001e460 con 0x7f9ff80a2930 2024-09-15T01:50:47.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.309+0000 7f9fef7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:47.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.309+0000 7f9fef7fe640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:47.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.459+0000 7f9ffd39d640 1 -- 172.21.15.178:0/2936648399 --> v2:172.21.15.204:6812/28210 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ff805ed70 con 0x7f9fd80397e0 2024-09-15T01:50:47.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.463+0000 7f9fef7fe640 1 -- 172.21.15.178:0/2936648399 <== mgr.4121 v2:172.21.15.204:6812/28210 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+93921 (secure 0 0 0) 0x7f9ff805ed70 con 0x7f9fd80397e0 2024-09-15T01:50:47.465 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-15T01:50:47.466 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-09-15T01:50:47.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.464+0000 7f9ffd39d640 1 -- 172.21.15.178:0/2936648399 >> v2:172.21.15.204:6812/28210 conn(0x7f9fd80397e0 msgr2=0x7f9fd803bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:47.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 1 --2- 172.21.15.178:0/2936648399 >> v2:172.21.15.204:6812/28210 conn(0x7f9fd80397e0 0x7f9fd803bc90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8034000 tx=0x7f9fe8032000 comp rx=0 tx=0).stop 2024-09-15T01:50:47.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 10 monclient: shutdown 2024-09-15T01:50:47.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:47.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 1 -- 172.21.15.178:0/2936648399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80a2930 msgr2=0x7f9ff811c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:47.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 1 --2- 172.21.15.178:0/2936648399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80a2930 0x7f9ff811c530 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009fd0 tx=0x7f9ff0009900 comp rx=0 tx=0).stop 2024-09-15T01:50:47.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 1 -- 172.21.15.178:0/2936648399 shutdown_connections 2024-09-15T01:50:47.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 1 --2- 172.21.15.178:0/2936648399 >> v2:172.21.15.204:6812/28210 conn(0x7f9fd80397e0 0x7f9fd803bc90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 1 --2- 172.21.15.178:0/2936648399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80a2930 0x7f9ff811c530 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 1 --2- 172.21.15.178:0/2936648399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80a08c0 0x7f9ff8131840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 1 --2- 172.21.15.178:0/2936648399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ff809faa0 0x7f9ff809afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 1 -- 172.21.15.178:0/2936648399 >> 172.21.15.178:0/2936648399 conn(0x7f9ff80a5ed0 msgr2=0x7f9ff8135900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:47.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 1 -- 172.21.15.178:0/2936648399 shutdown_connections 2024-09-15T01:50:47.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.465+0000 7f9ffd39d640 1 -- 172.21.15.178:0/2936648399 wait complete. 2024-09-15T01:50:47.478 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":823,"stamp":"2024-09-15T01:50:46.396335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":562077696,"kb_used":4795144,"kb_used_data":557282552,"kb_used_omap":18983,"kb_used_meta":0,"kb_avail":557282552,"statfs":{"total":575567560704,"available":570657333248,"internally_reserved":0,"allocated":570657333248,"data_stored":570657333248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19439182,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002761"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":97,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:11.715566+0000","last_change":"2024-09-15T01:50:11.715534+0000","last_active":"2024-09-15T01:50:11.715566+0000","last_peered":"2024-09-15T01:50:11.715566+0000","last_clean":"2024-09-15T01:50:11.715566+0000","last_became_active":"2024-09-15T01:36:25.227161+0000","last_became_peered":"2024-09-15T01:36:25.227161+0000","last_unstale":"2024-09-15T01:50:11.715566+0000","last_undegraded":"2024-09-15T01:50:11.715566+0000","last_fullsized":"2024-09-15T01:50:11.715566+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:11.715465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:11.715465+0000","last_clean_scrub_stamp":"2024-09-15T01:50:11.715465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:37:29.706523+0000","scrub_duration":0.0020818830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":108,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:12.750058+0000","last_change":"2024-09-15T01:50:12.750035+0000","last_active":"2024-09-15T01:50:12.750058+0000","last_peered":"2024-09-15T01:50:12.750058+0000","last_clean":"2024-09-15T01:50:12.750058+0000","last_became_active":"2024-09-15T01:36:25.228216+0000","last_became_peered":"2024-09-15T01:36:25.228216+0000","last_unstale":"2024-09-15T01:50:12.750058+0000","last_undegraded":"2024-09-15T01:50:12.750058+0000","last_fullsized":"2024-09-15T01:50:12.750058+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:12.749978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:12.749978+0000","last_clean_scrub_stamp":"2024-09-15T01:50:12.749978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:41:18.296244+0000","scrub_duration":0.0032490700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":97,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:07.674817+0000","last_change":"2024-09-15T01:50:07.674785+0000","last_active":"2024-09-15T01:50:07.674817+0000","last_peered":"2024-09-15T01:50:07.674817+0000","last_clean":"2024-09-15T01:50:07.674817+0000","last_became_active":"2024-09-15T01:36:25.228024+0000","last_became_peered":"2024-09-15T01:36:25.228024+0000","last_unstale":"2024-09-15T01:50:07.674817+0000","last_undegraded":"2024-09-15T01:50:07.674817+0000","last_fullsized":"2024-09-15T01:50:07.674817+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:07.674727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:07.674727+0000","last_clean_scrub_stamp":"2024-09-15T01:50:07.674727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:18:11.325944+0000","scrub_duration":0.0015531200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":108,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:06.778196+0000","last_change":"2024-09-15T01:50:06.778167+0000","last_active":"2024-09-15T01:50:06.778196+0000","last_peered":"2024-09-15T01:50:06.778196+0000","last_clean":"2024-09-15T01:50:06.778196+0000","last_became_active":"2024-09-15T01:36:25.229786+0000","last_became_peered":"2024-09-15T01:36:25.229786+0000","last_unstale":"2024-09-15T01:50:06.778196+0000","last_undegraded":"2024-09-15T01:50:06.778196+0000","last_fullsized":"2024-09-15T01:50:06.778196+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:06.778081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:06.778081+0000","last_clean_scrub_stamp":"2024-09-15T01:50:06.778081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:41:05.421522+0000","scrub_duration":0.001913238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":103,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:13.118106+0000","last_change":"2024-09-15T01:50:13.118083+0000","last_active":"2024-09-15T01:50:13.118106+0000","last_peered":"2024-09-15T01:50:13.118106+0000","last_clean":"2024-09-15T01:50:13.118106+0000","last_became_active":"2024-09-15T01:36:25.232778+0000","last_became_peered":"2024-09-15T01:36:25.232778+0000","last_unstale":"2024-09-15T01:50:13.118106+0000","last_undegraded":"2024-09-15T01:50:13.118106+0000","last_fullsized":"2024-09-15T01:50:13.118106+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:13.118031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:13.118031+0000","last_clean_scrub_stamp":"2024-09-15T01:50:13.118031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:32:28.135514+0000","scrub_duration":0.001530794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":97,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:10.431074+0000","last_change":"2024-09-15T01:50:10.431048+0000","last_active":"2024-09-15T01:50:10.431074+0000","last_peered":"2024-09-15T01:50:10.431074+0000","last_clean":"2024-09-15T01:50:10.431074+0000","last_became_active":"2024-09-15T01:36:25.227142+0000","last_became_peered":"2024-09-15T01:36:25.227142+0000","last_unstale":"2024-09-15T01:50:10.431074+0000","last_undegraded":"2024-09-15T01:50:10.431074+0000","last_fullsized":"2024-09-15T01:50:10.431074+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:10.430987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:10.430987+0000","last_clean_scrub_stamp":"2024-09-15T01:50:10.430987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:44:34.394061+0000","scrub_duration":0.001846798,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":181,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:15.955104+0000","last_change":"2024-09-15T01:50:15.955069+0000","last_active":"2024-09-15T01:50:15.955104+0000","last_peered":"2024-09-15T01:50:15.955104+0000","last_clean":"2024-09-15T01:50:15.955104+0000","last_became_active":"2024-09-15T01:36:25.230723+0000","last_became_peered":"2024-09-15T01:36:25.230723+0000","last_unstale":"2024-09-15T01:50:15.955104+0000","last_undegraded":"2024-09-15T01:50:15.955104+0000","last_fullsized":"2024-09-15T01:50:15.955104+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:15.955008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:15.955008+0000","last_clean_scrub_stamp":"2024-09-15T01:50:15.955008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:56:26.610655+0000","scrub_duration":0.0016696580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":99,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:15.314924+0000","last_change":"2024-09-15T01:50:15.314899+0000","last_active":"2024-09-15T01:50:15.314924+0000","last_peered":"2024-09-15T01:50:15.314924+0000","last_clean":"2024-09-15T01:50:15.314924+0000","last_became_active":"2024-09-15T01:36:25.228252+0000","last_became_peered":"2024-09-15T01:36:25.228252+0000","last_unstale":"2024-09-15T01:50:15.314924+0000","last_undegraded":"2024-09-15T01:50:15.314924+0000","last_fullsized":"2024-09-15T01:50:15.314924+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:15.314836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:15.314836+0000","last_clean_scrub_stamp":"2024-09-15T01:50:15.314836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:35:59.924412+0000","scrub_duration":0.0018829020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":109,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:05.817753+0000","last_change":"2024-09-15T01:50:05.817732+0000","last_active":"2024-09-15T01:50:05.817753+0000","last_peered":"2024-09-15T01:50:05.817753+0000","last_clean":"2024-09-15T01:50:05.817753+0000","last_became_active":"2024-09-15T01:36:25.226915+0000","last_became_peered":"2024-09-15T01:36:25.226915+0000","last_unstale":"2024-09-15T01:50:05.817753+0000","last_undegraded":"2024-09-15T01:50:05.817753+0000","last_fullsized":"2024-09-15T01:50:05.817753+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:05.817683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:05.817683+0000","last_clean_scrub_stamp":"2024-09-15T01:50:05.817683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:21:15.575297+0000","scrub_duration":0.0015299910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":129,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:09.144108+0000","last_change":"2024-09-15T01:50:09.144081+0000","last_active":"2024-09-15T01:50:09.144108+0000","last_peered":"2024-09-15T01:50:09.144108+0000","last_clean":"2024-09-15T01:50:09.144108+0000","last_became_active":"2024-09-15T01:36:25.229665+0000","last_became_peered":"2024-09-15T01:36:25.229665+0000","last_unstale":"2024-09-15T01:50:09.144108+0000","last_undegraded":"2024-09-15T01:50:09.144108+0000","last_fullsized":"2024-09-15T01:50:09.144108+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:09.144016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:09.144016+0000","last_clean_scrub_stamp":"2024-09-15T01:50:09.144016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:01:59.924448+0000","scrub_duration":0.0020523759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":187,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:10.048231+0000","last_change":"2024-09-15T01:50:10.048199+0000","last_active":"2024-09-15T01:50:10.048231+0000","last_peered":"2024-09-15T01:50:10.048231+0000","last_clean":"2024-09-15T01:50:10.048231+0000","last_became_active":"2024-09-15T01:36:25.227397+0000","last_became_peered":"2024-09-15T01:36:25.227397+0000","last_unstale":"2024-09-15T01:50:10.048231+0000","last_undegraded":"2024-09-15T01:50:10.048231+0000","last_fullsized":"2024-09-15T01:50:10.048231+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:10.048137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:10.048137+0000","last_clean_scrub_stamp":"2024-09-15T01:50:10.048137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:18:37.403671+0000","scrub_duration":0.0017503429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":99,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:13.281646+0000","last_change":"2024-09-15T01:50:13.281627+0000","last_active":"2024-09-15T01:50:13.281646+0000","last_peered":"2024-09-15T01:50:13.281646+0000","last_clean":"2024-09-15T01:50:13.281646+0000","last_became_active":"2024-09-15T01:36:25.226969+0000","last_became_peered":"2024-09-15T01:36:25.226969+0000","last_unstale":"2024-09-15T01:50:13.281646+0000","last_undegraded":"2024-09-15T01:50:13.281646+0000","last_fullsized":"2024-09-15T01:50:13.281646+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:13.281577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:13.281577+0000","last_clean_scrub_stamp":"2024-09-15T01:50:13.281577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:20:18.111636+0000","scrub_duration":0.0011427259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":97,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:12.438021+0000","last_change":"2024-09-15T01:50:12.437972+0000","last_active":"2024-09-15T01:50:12.438021+0000","last_peered":"2024-09-15T01:50:12.438021+0000","last_clean":"2024-09-15T01:50:12.438021+0000","last_became_active":"2024-09-15T01:36:26.228683+0000","last_became_peered":"2024-09-15T01:36:26.228683+0000","last_unstale":"2024-09-15T01:50:12.438021+0000","last_undegraded":"2024-09-15T01:50:12.438021+0000","last_fullsized":"2024-09-15T01:50:12.438021+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:12.437908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:12.437908+0000","last_clean_scrub_stamp":"2024-09-15T01:50:12.437908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:50:28.490080+0000","scrub_duration":0.001907688,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":98,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:16.278045+0000","last_change":"2024-09-15T01:50:16.278015+0000","last_active":"2024-09-15T01:50:16.278045+0000","last_peered":"2024-09-15T01:50:16.278045+0000","last_clean":"2024-09-15T01:50:16.278045+0000","last_became_active":"2024-09-15T01:36:26.228429+0000","last_became_peered":"2024-09-15T01:36:26.228429+0000","last_unstale":"2024-09-15T01:50:16.278045+0000","last_undegraded":"2024-09-15T01:50:16.278045+0000","last_fullsized":"2024-09-15T01:50:16.278045+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:16.277912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:16.277912+0000","last_clean_scrub_stamp":"2024-09-15T01:50:16.277912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:25:44.185617+0000","scrub_duration":0.001927944,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":117,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:15.680464+0000","last_change":"2024-09-15T01:50:15.680421+0000","last_active":"2024-09-15T01:50:15.680464+0000","last_peered":"2024-09-15T01:50:15.680464+0000","last_clean":"2024-09-15T01:50:15.680464+0000","last_became_active":"2024-09-15T01:36:26.225738+0000","last_became_peered":"2024-09-15T01:36:26.225738+0000","last_unstale":"2024-09-15T01:50:15.680464+0000","last_undegraded":"2024-09-15T01:50:15.680464+0000","last_fullsized":"2024-09-15T01:50:15.680464+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:15.680362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:15.680362+0000","last_clean_scrub_stamp":"2024-09-15T01:50:15.680362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:01:48.118870+0000","scrub_duration":0.001709716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":97,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:09.410437+0000","last_change":"2024-09-15T01:50:09.410410+0000","last_active":"2024-09-15T01:50:09.410437+0000","last_peered":"2024-09-15T01:50:09.410437+0000","last_clean":"2024-09-15T01:50:09.410437+0000","last_became_active":"2024-09-15T01:36:25.225608+0000","last_became_peered":"2024-09-15T01:36:25.225608+0000","last_unstale":"2024-09-15T01:50:09.410437+0000","last_undegraded":"2024-09-15T01:50:09.410437+0000","last_fullsized":"2024-09-15T01:50:09.410437+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:09.410354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:09.410354+0000","last_clean_scrub_stamp":"2024-09-15T01:50:09.410354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:15.825237+0000","scrub_duration":0.001899525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"265'74","reported_seq":113,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:29.190518+0000","last_change":"2024-09-15T01:50:29.190474+0000","last_active":"2024-09-15T01:50:29.190518+0000","last_peered":"2024-09-15T01:50:29.190518+0000","last_clean":"2024-09-15T01:50:29.190518+0000","last_became_active":"2024-09-15T01:36:25.233270+0000","last_became_peered":"2024-09-15T01:36:25.233270+0000","last_unstale":"2024-09-15T01:50:29.190518+0000","last_undegraded":"2024-09-15T01:50:29.190518+0000","last_fullsized":"2024-09-15T01:50:29.190518+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'74","last_scrub_stamp":"2024-09-15T01:50:29.190410+0000","last_deep_scrub":"265'74","last_deep_scrub_stamp":"2024-09-15T01:50:29.190410+0000","last_clean_scrub_stamp":"2024-09-15T01:50:29.190410+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:17:09.837923+0000","scrub_duration":0.0050183399999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"265'2","reported_seq":270,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:20.093946+0000","last_change":"2024-09-15T01:50:20.093921+0000","last_active":"2024-09-15T01:50:20.093946+0000","last_peered":"2024-09-15T01:50:20.093946+0000","last_clean":"2024-09-15T01:50:20.093946+0000","last_became_active":"2024-09-15T01:36:25.230032+0000","last_became_peered":"2024-09-15T01:36:25.230032+0000","last_unstale":"2024-09-15T01:50:20.093946+0000","last_undegraded":"2024-09-15T01:50:20.093946+0000","last_fullsized":"2024-09-15T01:50:20.093946+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'2","last_scrub_stamp":"2024-09-15T01:50:20.093859+0000","last_deep_scrub":"265'2","last_deep_scrub_stamp":"2024-09-15T01:50:20.093859+0000","last_clean_scrub_stamp":"2024-09-15T01:50:20.093859+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:49:03.602901+0000","scrub_duration":0.002496683,"objects_trimmed":0,"snaptrim_duration":0.00076976900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:07.819352+0000","last_change":"2024-09-15T01:50:07.819333+0000","last_active":"2024-09-15T01:50:07.819352+0000","last_peered":"2024-09-15T01:50:07.819352+0000","last_clean":"2024-09-15T01:50:07.819352+0000","last_became_active":"2024-09-15T01:36:25.226499+0000","last_became_peered":"2024-09-15T01:36:25.226499+0000","last_unstale":"2024-09-15T01:50:07.819352+0000","last_undegraded":"2024-09-15T01:50:07.819352+0000","last_fullsized":"2024-09-15T01:50:07.819352+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:07.819277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:07.819277+0000","last_clean_scrub_stamp":"2024-09-15T01:50:07.819277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:23:42.579404+0000","scrub_duration":0.001625727,"objects_trimmed":0,"snaptrim_duration":0.00037511000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":99,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:06.227796+0000","last_change":"2024-09-15T01:50:06.227770+0000","last_active":"2024-09-15T01:50:06.227796+0000","last_peered":"2024-09-15T01:50:06.227796+0000","last_clean":"2024-09-15T01:50:06.227796+0000","last_became_active":"2024-09-15T01:36:25.231212+0000","last_became_peered":"2024-09-15T01:36:25.231212+0000","last_unstale":"2024-09-15T01:50:06.227796+0000","last_undegraded":"2024-09-15T01:50:06.227796+0000","last_fullsized":"2024-09-15T01:50:06.227796+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:06.227713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:06.227713+0000","last_clean_scrub_stamp":"2024-09-15T01:50:06.227713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:41:46.522757+0000","scrub_duration":0.001627975,"objects_trimmed":0,"snaptrim_duration":0.00046258900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"265'3","reported_seq":308,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:06.028446+0000","last_change":"2024-09-15T01:50:06.028427+0000","last_active":"2024-09-15T01:50:06.028446+0000","last_peered":"2024-09-15T01:50:06.028446+0000","last_clean":"2024-09-15T01:50:06.028446+0000","last_became_active":"2024-09-15T01:36:25.226718+0000","last_became_peered":"2024-09-15T01:36:25.226718+0000","last_unstale":"2024-09-15T01:50:06.028446+0000","last_undegraded":"2024-09-15T01:50:06.028446+0000","last_fullsized":"2024-09-15T01:50:06.028446+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'3","last_scrub_stamp":"2024-09-15T01:50:06.028377+0000","last_deep_scrub":"265'3","last_deep_scrub_stamp":"2024-09-15T01:50:06.028377+0000","last_clean_scrub_stamp":"2024-09-15T01:50:06.028377+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:02:09.417710+0000","scrub_duration":0.0030518149999999998,"objects_trimmed":0,"snaptrim_duration":0.00093022000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":266,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:08.387173+0000","last_change":"2024-09-15T01:50:08.387134+0000","last_active":"2024-09-15T01:50:08.387173+0000","last_peered":"2024-09-15T01:50:08.387173+0000","last_clean":"2024-09-15T01:50:08.387173+0000","last_became_active":"2024-09-15T01:36:25.227361+0000","last_became_peered":"2024-09-15T01:36:25.227361+0000","last_unstale":"2024-09-15T01:50:08.387173+0000","last_undegraded":"2024-09-15T01:50:08.387173+0000","last_fullsized":"2024-09-15T01:50:08.387173+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:08.387067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:08.387067+0000","last_clean_scrub_stamp":"2024-09-15T01:50:08.387067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:23.598441+0000","scrub_duration":0.0014306060000000001,"objects_trimmed":0,"snaptrim_duration":0.00031034999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":102,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:15.113653+0000","last_change":"2024-09-15T01:50:15.113631+0000","last_active":"2024-09-15T01:50:15.113653+0000","last_peered":"2024-09-15T01:50:15.113653+0000","last_clean":"2024-09-15T01:50:15.113653+0000","last_became_active":"2024-09-15T01:36:25.229674+0000","last_became_peered":"2024-09-15T01:36:25.229674+0000","last_unstale":"2024-09-15T01:50:15.113653+0000","last_undegraded":"2024-09-15T01:50:15.113653+0000","last_fullsized":"2024-09-15T01:50:15.113653+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:15.113566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:15.113566+0000","last_clean_scrub_stamp":"2024-09-15T01:50:15.113566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:13:40.272496+0000","scrub_duration":0.0018433779999999999,"objects_trimmed":0,"snaptrim_duration":0.00077134000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":96,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:13.655562+0000","last_change":"2024-09-15T01:50:13.655541+0000","last_active":"2024-09-15T01:50:13.655562+0000","last_peered":"2024-09-15T01:50:13.655562+0000","last_clean":"2024-09-15T01:50:13.655562+0000","last_became_active":"2024-09-15T01:36:25.225361+0000","last_became_peered":"2024-09-15T01:36:25.225361+0000","last_unstale":"2024-09-15T01:50:13.655562+0000","last_undegraded":"2024-09-15T01:50:13.655562+0000","last_fullsized":"2024-09-15T01:50:13.655562+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:13.655486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:13.655486+0000","last_clean_scrub_stamp":"2024-09-15T01:50:13.655486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:23:15.849882+0000","scrub_duration":0.001290636,"objects_trimmed":0,"snaptrim_duration":0.00041942500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":256,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-15T01:50:34.156058+0000","last_change":"2024-09-15T01:50:34.156033+0000","last_active":"2024-09-15T01:50:34.156058+0000","last_peered":"2024-09-15T01:50:34.156058+0000","last_clean":"2024-09-15T01:50:34.156058+0000","last_became_active":"2024-09-15T01:36:26.437594+0000","last_became_peered":"2024-09-15T01:36:26.437594+0000","last_unstale":"2024-09-15T01:50:34.156058+0000","last_undegraded":"2024-09-15T01:50:34.156058+0000","last_fullsized":"2024-09-15T01:50:34.156058+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T01:50:34.155968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T01:50:34.155968+0000","last_clean_scrub_stamp":"2024-09-15T01:50:34.155968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:14:32.943332+0000","scrub_duration":0.0014129539999999999,"objects_trimmed":0,"snaptrim_duration":0.00065783200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":0},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640483,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799576,"kb_used_data":92880040,"kb_used_omap":3230,"kb_used_meta":0,"kb_avail":92880040,"statfs":{"total":95927926784,"available":95109160960,"internally_reserved":0,"allocated":95109160960,"data_stored":95109160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3308492,"internal_metadata":0},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.39300000000000002,"5min":0.28999999999999998,"15min":0.24299999999999999},"max":{"1min":0.64700000000000002,"5min":0.65600000000000003,"15min":0.95099999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.49399999999999999,"15min":0.48899999999999999},"min":{"1min":0.32000000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.59099999999999997,"5min":0.68100000000000005,"15min":0.70199999999999996},"last":0.59499999999999997}]},{"osd":1,"last update":"Sun Sep 15 01:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.496},"min":{"1min":0.30099999999999999,"5min":0.28899999999999998,"15min":0.28299999999999997},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.91800000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.501,"15min":0.496},"min":{"1min":0.40799999999999997,"5min":0.31900000000000001,"15min":0.30499999999999999},"max":{"1min":0.61399999999999999,"5min":0.63500000000000001,"15min":0.69099999999999995},"last":0.55700000000000005}]},{"osd":2,"last update":"Sun Sep 15 01:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.34300000000000003,"5min":0.307,"15min":0.23300000000000001},"max":{"1min":0.63,"5min":0.68500000000000005,"15min":0.73399999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.504},"min":{"1min":0.33400000000000002,"5min":0.33200000000000002,"15min":0.27700000000000002},"max":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.81000000000000005},"last":0.57199999999999995}]},{"osd":3,"last update":"Sun Sep 15 01:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.376,"15min":0.38100000000000001},"min":{"1min":0.29099999999999998,"5min":0.224,"15min":0.20699999999999999},"max":{"1min":0.501,"5min":0.63400000000000001,"15min":0.96799999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.433,"5min":0.40500000000000003,"15min":0.38700000000000001},"min":{"1min":0.33900000000000002,"5min":0.23200000000000001,"15min":0.22},"max":{"1min":0.58799999999999997,"5min":0.64700000000000002,"15min":0.86199999999999999},"last":0.46200000000000002}]},{"osd":4,"last update":"Sun Sep 15 01:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.38700000000000001},"min":{"1min":0.31900000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.53300000000000003,"5min":0.58899999999999997,"15min":1.0289999999999999},"last":0.441},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.34399999999999997,"5min":0.249,"15min":0.22},"max":{"1min":0.59199999999999997,"5min":0.61799999999999999,"15min":0.67200000000000004},"last":0.498}]}]},{"osd":4,"up_from":11,"seq":47244640483,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799000,"kb_used_data":92880616,"kb_used_omap":3271,"kb_used_meta":0,"kb_avail":92880616,"statfs":{"total":95927926784,"available":95109750784,"internally_reserved":0,"allocated":95109750784,"data_stored":95109750784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3350414,"internal_metadata":0},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:50:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.5,"15min":0.49099999999999999},"min":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.219},"max":{"1min":0.56699999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.503},"min":{"1min":0.39000000000000001,"5min":0.32900000000000001,"15min":0.27400000000000002},"max":{"1min":0.63100000000000001,"5min":0.69399999999999995,"15min":0.70799999999999996},"last":0.56000000000000005}]},{"osd":1,"last update":"Sun Sep 15 01:50:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.51300000000000001},"min":{"1min":0.42399999999999999,"5min":0.311,"15min":0.25600000000000001},"max":{"1min":0.70099999999999996,"5min":0.752,"15min":0.752},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57299999999999995,"15min":0.53500000000000003},"min":{"1min":0.45200000000000001,"5min":0.36499999999999999,"15min":0.29899999999999999},"max":{"1min":0.72799999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.58099999999999996}]},{"osd":2,"last update":"Sun Sep 15 01:50:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.495,"15min":0.48799999999999999},"min":{"1min":0.41299999999999998,"5min":0.33300000000000002,"15min":0.28899999999999998},"max":{"1min":0.55500000000000005,"5min":0.70499999999999996,"15min":0.71599999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.51700000000000002},"min":{"1min":0.432,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69499999999999995,"5min":0.876,"15min":0.876},"last":0.52700000000000002}]},{"osd":3,"last update":"Sun Sep 15 01:50:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39500000000000002,"15min":0.36899999999999999},"min":{"1min":0.316,"5min":0.23999999999999999,"15min":0.22700000000000001},"max":{"1min":0.56699999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.41599999999999998,"15min":0.38900000000000001},"min":{"1min":0.33500000000000002,"5min":0.251,"15min":0.216},"max":{"1min":0.59899999999999998,"5min":0.81000000000000005,"15min":1.3540000000000001},"last":0.42899999999999999}]},{"osd":5,"last update":"Sun Sep 15 01:50:37 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.38},"min":{"1min":0.33100000000000002,"5min":0.26600000000000001,"15min":0.22900000000000001},"max":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":1.2709999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41999999999999998,"15min":0.39900000000000002},"min":{"1min":0.308,"5min":0.26800000000000002,"15min":0.24199999999999999},"max":{"1min":0.53300000000000003,"5min":0.61599999999999999,"15min":1.2250000000000001},"last":0.46500000000000002}]}]},{"osd":3,"up_from":11,"seq":47244640483,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798996,"kb_used_data":92880620,"kb_used_omap":3045,"kb_used_meta":0,"kb_avail":92880620,"statfs":{"total":95927926784,"available":95109754880,"internally_reserved":0,"allocated":95109754880,"data_stored":95109754880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3118296,"internal_metadata":0},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:50:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49199999999999999,"15min":0.47699999999999998},"min":{"1min":0.35699999999999998,"5min":0.28799999999999998,"15min":0.24299999999999999},"max":{"1min":0.626,"5min":0.68899999999999995,"15min":0.69399999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.36399999999999999,"5min":0.33200000000000002,"15min":0.29899999999999999},"max":{"1min":0.63100000000000001,"5min":0.66400000000000003,"15min":0.746},"last":0.40200000000000002}]},{"osd":1,"last update":"Sun Sep 15 01:50:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.27900000000000003},"max":{"1min":0.60099999999999998,"5min":0.69399999999999995,"15min":0.79300000000000004},"last":0.435},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.501,"15min":0.48699999999999999},"min":{"1min":0.35999999999999999,"5min":0.34399999999999997,"15min":0.29899999999999999},"max":{"1min":0.68100000000000005,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.42799999999999999}]},{"osd":2,"last update":"Sun Sep 15 01:50:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.51500000000000001,"15min":0.50600000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":0.56999999999999995,"5min":0.70199999999999996,"15min":0.74299999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.5,"15min":0.49199999999999999},"min":{"1min":0.379,"5min":0.316,"15min":0.30099999999999999},"max":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.75800000000000001},"last":0.38800000000000001}]},{"osd":4,"last update":"Sun Sep 15 01:50:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.26600000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.39000000000000001,"15min":0.376},"min":{"1min":0.28999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.70599999999999996},"last":0.34000000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:50:38 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.377},"min":{"1min":0.30199999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.65500000000000003},"last":0.371},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.39400000000000002,"15min":0.38700000000000001},"min":{"1min":0.29799999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.63100000000000001,"5min":0.64600000000000002,"15min":0.70399999999999996},"last":0.32500000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640483,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":798996,"kb_used_data":92880620,"kb_used_omap":3065,"kb_used_meta":0,"kb_avail":92880620,"statfs":{"total":95927926784,"available":95109754880,"internally_reserved":0,"allocated":95109754880,"data_stored":95109754880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3138690,"internal_metadata":0},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:50:36 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.32400000000000001,"15min":0.32500000000000001},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.42399999999999999,"5min":0.48499999999999999,"15min":0.53000000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.36299999999999999,"15min":0.35699999999999998},"min":{"1min":0.25600000000000001,"5min":0.23999999999999999,"15min":0.20200000000000001},"max":{"1min":0.45300000000000001,"5min":0.53000000000000003,"15min":0.59999999999999998},"last":0.433}]},{"osd":1,"last update":"Sun Sep 15 01:50:36 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.32100000000000001,"15min":0.33200000000000002},"min":{"1min":0.191,"5min":0.187,"15min":0.182},"max":{"1min":0.42999999999999999,"5min":0.52500000000000002,"15min":0.60599999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36399999999999999,"15min":0.374},"min":{"1min":0.23599999999999999,"5min":0.222,"15min":0.20699999999999999},"max":{"1min":0.495,"5min":0.57099999999999995,"15min":0.67400000000000004},"last":0.46600000000000003}]},{"osd":3,"last update":"Sun Sep 15 01:50:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.501,"15min":0.51100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68200000000000005,"5min":0.75700000000000001,"15min":0.78900000000000003},"last":0.498},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.316,"5min":0.249,"15min":0.249},"max":{"1min":0.72499999999999998,"5min":0.77900000000000003,"15min":1.0129999999999999},"last":0.48599999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:50:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.33900000000000002,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":0.65200000000000002,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.70499999999999996,"5min":0.75600000000000001,"15min":0.78000000000000003},"last":0.52900000000000003}]},{"osd":5,"last update":"Sun Sep 15 01:50:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.27800000000000002},"max":{"1min":0.79200000000000004,"5min":2.9969999999999999,"15min":2.9969999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.35799999999999998,"5min":0.34000000000000002,"15min":0.312},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.87},"last":0.57799999999999996}]}]},{"osd":1,"up_from":11,"seq":47244640484,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799576,"kb_used_data":92880040,"kb_used_omap":3069,"kb_used_meta":0,"kb_avail":92880040,"statfs":{"total":95927926784,"available":95109160960,"internally_reserved":0,"allocated":95109160960,"data_stored":95109160960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3143202,"internal_metadata":0},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 01:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33300000000000002,"15min":0.33200000000000002},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.188},"max":{"1min":0.45700000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.254,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.41899999999999998,"5min":0.54100000000000004,"15min":0.68200000000000005},"last":0.35399999999999998}]},{"osd":2,"last update":"Sun Sep 15 01:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.26900000000000002,"5min":0.245,"15min":0.23799999999999999},"max":{"1min":0.498,"5min":0.51400000000000001,"15min":0.60899999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36099999999999999,"15min":0.36199999999999999},"min":{"1min":0.26400000000000001,"5min":0.25,"15min":0.21199999999999999},"max":{"1min":0.435,"5min":0.50900000000000001,"15min":0.70299999999999996},"last":0.34499999999999997}]},{"osd":3,"last update":"Sun Sep 15 01:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.435,"5min":0.34499999999999997,"15min":0.30099999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.27300000000000002},"max":{"1min":0.61699999999999999,"5min":0.71299999999999997,"15min":0.75600000000000001},"last":0.49199999999999999}]},{"osd":4,"last update":"Sun Sep 15 01:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.38},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.308},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.52100000000000002}]},{"osd":5,"last update":"Sun Sep 15 01:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.39700000000000002,"5min":0.35999999999999999,"15min":0.32600000000000001},"max":{"1min":0.65400000000000003,"5min":0.748,"15min":0.75800000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.32200000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.753},"last":0.505}]}]},{"osd":0,"up_from":11,"seq":47244640483,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93679616,"kb_used":799000,"kb_used_data":92880616,"kb_used_omap":3300,"kb_used_meta":0,"kb_avail":92880616,"statfs":{"total":95927926784,"available":95109750784,"internally_reserved":0,"allocated":95109750784,"data_stored":95109750784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3380088,"internal_metadata":0},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 01:50:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.33700000000000002,"15min":0.33300000000000002},"min":{"1min":0.214,"5min":0.188,"15min":0.184},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.75600000000000001},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.379,"15min":0.38},"min":{"1min":0.27900000000000003,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.59599999999999997,"5min":0.61099999999999999,"15min":0.79200000000000004},"last":0.27700000000000002}]},{"osd":2,"last update":"Sun Sep 15 01:50:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.36799999999999999},"min":{"1min":0.22800000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53600000000000003,"5min":0.54800000000000004,"15min":0.69499999999999995},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.38400000000000001},"min":{"1min":0.28199999999999997,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.58499999999999996,"5min":0.58799999999999997,"15min":0.82199999999999995},"last":0.312}]},{"osd":3,"last update":"Sun Sep 15 01:50:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.44500000000000001,"5min":0.316,"15min":0.28399999999999997},"max":{"1min":0.71999999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52800000000000002,"15min":0.51800000000000002},"min":{"1min":0.42099999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.66000000000000003,"5min":0.745,"15min":0.745},"last":0.48099999999999998}]},{"osd":4,"last update":"Sun Sep 15 01:50:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.46999999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.46000000000000002,"5min":0.34000000000000002,"15min":0.28100000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.56699999999999995}]},{"osd":5,"last update":"Sun Sep 15 01:50:29 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.439,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.84399999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.62}]}]}],"pool_statfs":[]}} 2024-09-15T01:50:47.481 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T01:50:47.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.571+0000 7f2e92f9d640 10 monclient: get_monmap_and_config 2024-09-15T01:50:47.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.571+0000 7f2e92f9d640 10 monclient: build_initial_monmap 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.571+0000 7f2e92f9d640 10 monclient: monmap: 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:50:47.572578+0000 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:50:47.572578+0000 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:0: v1:172.21.15.178:6789/0 mon.noname-a-legacy 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:1: v1:172.21.15.204:6789/0 mon.noname-f-legacy 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:47.574 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:47.575 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:47.575 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:47.575 INFO:teuthology.orchestra.run.smithi178.stderr:9: v2:172.21.15.178:3300/0 mon.noname-a 2024-09-15T01:50:47.575 INFO:teuthology.orchestra.run.smithi178.stderr:10: v2:172.21.15.204:3300/0 mon.noname-f 2024-09-15T01:50:47.575 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 1 Processor -- start 2024-09-15T01:50:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 1 -- start start 2024-09-15T01:50:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 10 monclient: init 2024-09-15T01:50:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 10 monclient: _add_conns ranks=[1,2,3,4,0,6,8,5,7,10,9] 2024-09-15T01:50:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2e8c0a2330 addr v1:172.21.15.204:6789/0 2024-09-15T01:50:47.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 0x7f2e8c132a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f2e8c09fff0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e8c137ba0 0x7f2e8c139f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 10 monclient(hunting): picked mon.noname-g con 0x7f2e8c137ba0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8c04bf60 con 0x7f2e8c0a2330 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c0a1d80 con 0x7f2e8c09fff0 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c09e0d0 con 0x7f2e8c137ba0 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.572+0000 7f2e92f9d640 10 monclient(hunting): authenticate will time out at 1852.322933s 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 0x7f2e8c132a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 0x7f2e8c132a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42820/0 (socket says 172.21.15.178:42820) 2024-09-15T01:50:47.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 1 -- 172.21.15.178:0/825333378 learned_addr learned my addr 172.21.15.178:0/825333378 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 10 monclient(hunting): get_auth_request con 0x7f2e8c09fff0 auth_method 0 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e9153a640 1 --2- 172.21.15.178:0/825333378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e8c137ba0 0x7f2e8c139f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e9153a640 10 monclient(hunting): get_auth_request con 0x7f2e8c137ba0 auth_method 0 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e9153a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e9153a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e9153a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:47.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 10 monclient(hunting): handle_auth_done global_id 35615 payload 274 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 1 -- 172.21.15.178:0/825333378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e8c137ba0 msgr2=0x7f2e8c139f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 1 --2- 172.21.15.178:0/825333378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e8c137ba0 0x7f2e8c139f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 1 -- 172.21.15.178:0/825333378 >> v1:172.21.15.204:6789/0 conn(0x7f2e8c0a2330 legacy=0x7f2e8c09f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 10 monclient: _finish_hunting 0 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 1 monclient: found mon.noname-b 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e83fff640 1 -- 172.21.15.178:0/825333378 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0a2700 con 0x7f2e8c09fff0 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.573+0000 7f2e9153a640 1 --2- 172.21.15.178:0/825333378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e8c137ba0 0x7f2e8c139f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e83fff640 1 --2- 172.21.15.178:0/825333378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 0x7f2e8c132a20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009940 tx=0x7f2e7c0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ef42d03459f2519 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e837fe640 1 -- 172.21.15.178:0/825333378 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e7c02ad00 con 0x7f2e8c09fff0 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:47.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e837fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e837fe640 10 monclient: dump: 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:47.587 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e837fe640 10 monclient: _finish_auth 0 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 5 monclient: authenticate success, global_id 35615 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e837fe640 1 -- 172.21.15.178:0/825333378 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c02ae80 con 0x7f2e8c09fff0 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e837fe640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 10 monclient: get_monmap_and_config success 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e837fe640 1 -- 172.21.15.178:0/825333378 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e7c031ca0 con 0x7f2e8c09fff0 2024-09-15T01:50:47.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 4 set_mon_vals no callback set 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e837fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e837fe640 10 monclient: dump: 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:47.589 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:47.590 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 10 monclient: shutdown 2024-09-15T01:50:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 1 -- 172.21.15.178:0/825333378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 msgr2=0x7f2e8c132a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 1 --2- 172.21.15.178:0/825333378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 0x7f2e8c132a20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009940 tx=0x7f2e7c0091d0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 1 -- 172.21.15.178:0/825333378 shutdown_connections 2024-09-15T01:50:47.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 1 --2- 172.21.15.178:0/825333378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e8c137ba0 0x7f2e8c139f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 1 --2- 172.21.15.178:0/825333378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 0x7f2e8c132a20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 1 -- 172.21.15.178:0/825333378 >> 172.21.15.178:0/825333378 conn(0x7f2e8c06e860 msgr2=0x7f2e8c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 1 -- 172.21.15.178:0/825333378 shutdown_connections 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.574+0000 7f2e92f9d640 1 -- 172.21.15.178:0/825333378 wait complete. 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.575+0000 7f2e92f9d640 10 monclient: build_initial_monmap 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.575+0000 7f2e92f9d640 10 monclient: monmap: 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 0 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 0.000000 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:created 0.000000 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 0 (unknown) 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 1 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-09-15T01:50:47.591 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.noname-f 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-b 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.noname-g 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.noname-c 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.noname-h 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.noname-d 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.noname-i 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.noname-e 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.575+0000 7f2e92f9d640 1 Processor -- start 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.575+0000 7f2e92f9d640 1 -- start start 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.575+0000 7f2e92f9d640 10 monclient: init 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 10 monclient: _reopen_session rank -1 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 10 monclient: _add_conns ranks=[2,0,3,8,5,1,4,6,7] 2024-09-15T01:50:47.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 0x7f2e8c130590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f2e8c09fff0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e8c0a2330 0x7f2e8c0997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 10 monclient(hunting): picked mon.noname-a con 0x7f2e8c0a2330 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e8c137ba0 0x7f2e8c099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 10 monclient(hunting): picked mon.noname-g con 0x7f2e8c137ba0 addr [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c0a2af0 con 0x7f2e8c0a2330 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c09d420 con 0x7f2e8c09fff0 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 10 monclient(hunting): start opening mon connection 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c09b830 con 0x7f2e8c137ba0 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 10 monclient(hunting): _renew_subs 2024-09-15T01:50:47.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e92f9d640 10 monclient(hunting): authenticate will time out at 1852.326271s 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 0x7f2e8c130590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e8c0a2330 0x7f2e8c0997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 0x7f2e8c130590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:42828/0 (socket says 172.21.15.178:42828) 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 1 -- 172.21.15.178:0/2628473900 learned_addr learned my addr 172.21.15.178:0/2628473900 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 10 monclient(hunting): get_auth_request con 0x7f2e8c09fff0 auth_method 0 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e83fff640 10 monclient(hunting): get_auth_request con 0x7f2e8c0a2330 auth_method 0 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e83fff640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e9153a640 1 --2- 172.21.15.178:0/2628473900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e8c137ba0 0x7f2e8c099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e9153a640 10 monclient(hunting): get_auth_request con 0x7f2e8c137ba0 auth_method 0 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e9153a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e9153a640 10 monclient(hunting): _init_auth method 2 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e9153a640 10 monclient(hunting): _init_auth creating new auth 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e9153a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e9153a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e9153a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-15T01:50:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 10 monclient(hunting): handle_auth_done global_id 35624 payload 824 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 1 -- 172.21.15.178:0/2628473900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e8c137ba0 msgr2=0x7f2e8c099d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 1 --2- 172.21.15.178:0/2628473900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e8c137ba0 0x7f2e8c099d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 1 -- 172.21.15.178:0/2628473900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e8c0a2330 msgr2=0x7f2e8c0997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 1 --2- 172.21.15.178:0/2628473900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e8c0a2330 0x7f2e8c0997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 10 monclient: _finish_hunting 0 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 1 monclient: found mon.noname-b 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.178:3301/0 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 1 -- 172.21.15.178:0/2628473900 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c009450 con 0x7f2e8c09fff0 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e83fff640 1 --2- 172.21.15.178:0/2628473900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e8c0a2330 0x7f2e8c0997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e90d39640 1 --2- 172.21.15.178:0/2628473900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 0x7f2e8c130590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800c590 tx=0x7f2e88007eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e80ff9640 1 -- 172.21.15.178:0/2628473900 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e88012ab0 con 0x7f2e8c09fff0 2024-09-15T01:50:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.576+0000 7f2e80ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e80ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e80ff9640 10 monclient: dump: 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e80ff9640 10 monclient: _finish_auth 0 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e80ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e80ff9640 1 -- 172.21.15.178:0/2628473900 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e88012c70 con 0x7f2e8c09fff0 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e92f9d640 5 monclient: authenticate success, global_id 35624 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e80ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e92f9d640 10 monclient: _renew_subs 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e80ff9640 1 -- 172.21.15.178:0/2628473900 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e88012e20 con 0x7f2e8c09fff0 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e92f9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e92f9d640 1 -- 172.21.15.178:0/2628473900 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8c11c6a0 con 0x7f2e8c09fff0 2024-09-15T01:50:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e80ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e80ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e82ffd640 4 set_mon_vals no callback set 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e80ff9640 10 monclient: dump: 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:epoch 1 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:fsid 6ff069e2-2827-44d3-9460-72f9fc8c094b 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:last_changed 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:created 2024-09-15T01:31:38.100732+0000 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:min_mon_release 17 (quincy) 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:election_strategy: 3 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.f 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.b 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:3: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.g 2024-09-15T01:50:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:4: [v2:172.21.15.178:3302/0,v1:172.21.15.178:6791/0] mon.c 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:5: [v2:172.21.15.204:3302/0,v1:172.21.15.204:6791/0] mon.h 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:6: [v2:172.21.15.178:3303/0,v1:172.21.15.178:6792/0] mon.d 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:7: [v2:172.21.15.204:3303/0,v1:172.21.15.204:6792/0] mon.i 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:8: [v2:172.21.15.178:3304/0,v1:172.21.15.178:6793/0] mon.e 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e92f9d640 10 monclient: _renew_subs 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e92f9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e92f9d640 1 -- 172.21.15.178:0/2628473900 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8c09bef0 con 0x7f2e8c09fff0 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.577+0000 7f2e80ff9640 1 -- 172.21.15.178:0/2628473900 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 44975+0+0 (secure 0 0 0) 0x7f2e88012630 con 0x7f2e8c09fff0 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.578+0000 7f2e80ff9640 1 --2- 172.21.15.178:0/2628473900 >> v2:172.21.15.204:6812/28210 conn(0x7f2e780397b0 0x7f2e7803bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.578+0000 7f2e80ff9640 1 -- 172.21.15.178:0/2628473900 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(265..265 src has 240..265) v4 ==== 6799+0+0 (secure 0 0 0) 0x7f2e8804b5f0 con 0x7f2e8c09fff0 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.578+0000 7f2e92f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.578+0000 7f2e92f9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.578+0000 7f2e92f9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.578+0000 7f2e92f9d640 1 -- 172.21.15.178:0/2628473900 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8c05ed70 con 0x7f2e8c09fff0 2024-09-15T01:50:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.578+0000 7f2e83fff640 1 --2- 172.21.15.178:0/2628473900 >> v2:172.21.15.204:6812/28210 conn(0x7f2e780397b0 0x7f2e7803bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T01:50:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.578+0000 7f2e83fff640 10 monclient: get_auth_request con 0x7f2e780397b0 auth_method 0 2024-09-15T01:50:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.578+0000 7f2e83fff640 1 --2- 172.21.15.178:0/2628473900 >> v2:172.21.15.204:6812/28210 conn(0x7f2e780397b0 0x7f2e7803bc60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009420 tx=0x7f2e7c032000 comp rx=0 tx=0).ready entity=mgr.4121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T01:50:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.582+0000 7f2e80ff9640 1 -- 172.21.15.178:0/2628473900 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2e88011380 con 0x7f2e8c09fff0 2024-09-15T01:50:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.582+0000 7f2e80ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T01:50:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.582+0000 7f2e80ff9640 10 monclient: _finish_command 1 = system:0 2024-09-15T01:50:47.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.741+0000 7f2e92f9d640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-15T01:50:47.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.741+0000 7f2e92f9d640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-15T01:50:47.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.741+0000 7f2e92f9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.178:3301/0 2024-09-15T01:50:47.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.741+0000 7f2e92f9d640 1 -- 172.21.15.178:0/2628473900 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f2e8c11c830 con 0x7f2e8c09fff0 2024-09-15T01:50:47.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.801+0000 7f2e80ff9640 1 -- 172.21.15.178:0/2628473900 <== mon.2 v2:172.21.15.178:3301/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2e88019ec0 con 0x7f2e8c09fff0 2024-09-15T01:50:47.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.801+0000 7f2e80ff9640 10 monclient: handle_config config(1 keys) v1 2024-09-15T01:50:47.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.801+0000 7f2e82ffd640 4 set_mon_vals no callback set 2024-09-15T01:50:47.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.801+0000 7f2e82ffd640 10 set_mon_vals mon_health_to_clog = false 2024-09-15T01:50:47.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.802+0000 7f2e80ff9640 1 -- 172.21.15.178:0/2628473900 <== mon.2 v2:172.21.15.178:3301/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) v1 ==== 123+0+0 (secure 0 0 0) 0x7f2e88019300 con 0x7f2e8c09fff0 2024-09-15T01:50:47.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.802+0000 7f2e80ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-15T01:50:47.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.802+0000 7f2e80ff9640 10 monclient: _finish_command 2 = system:0 2024-09-15T01:50:47.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 1 -- 172.21.15.178:0/2628473900 >> v2:172.21.15.204:6812/28210 conn(0x7f2e780397b0 msgr2=0x7f2e7803bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 1 --2- 172.21.15.178:0/2628473900 >> v2:172.21.15.204:6812/28210 conn(0x7f2e780397b0 0x7f2e7803bc60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009420 tx=0x7f2e7c032000 comp rx=0 tx=0).stop 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 10 monclient: shutdown 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 1 -- 172.21.15.178:0/2628473900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 msgr2=0x7f2e8c130590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 1 --2- 172.21.15.178:0/2628473900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 0x7f2e8c130590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800c590 tx=0x7f2e88007eb0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 1 -- 172.21.15.178:0/2628473900 shutdown_connections 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 1 --2- 172.21.15.178:0/2628473900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e8c137ba0 0x7f2e8c099d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 1 --2- 172.21.15.178:0/2628473900 >> v2:172.21.15.204:6812/28210 conn(0x7f2e780397b0 0x7f2e7803bc60 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 1 --2- 172.21.15.178:0/2628473900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e8c0a2330 0x7f2e8c0997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 1 --2- 172.21.15.178:0/2628473900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e8c09fff0 0x7f2e8c130590 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 1 -- 172.21.15.178:0/2628473900 >> 172.21.15.178:0/2628473900 conn(0x7f2e8c06e860 msgr2=0x7f2e8c131890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 1 -- 172.21.15.178:0/2628473900 shutdown_connections 2024-09-15T01:50:47.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-15T01:50:47.804+0000 7f2e92f9d640 1 -- 172.21.15.178:0/2628473900 wait complete. 2024-09-15T01:50:47.817 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T01:50:47.817 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T01:50:47.818 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T01:50:47.818 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:47.829 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T01:50:47.829 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T01:50:47.829 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:47.881 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T01:50:47.881 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T01:50:47.881 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:47.933 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T01:50:47.934 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T01:50:47.934 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:47.947 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T01:50:47.947 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T01:50:47.947 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:48.000 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T01:50:48.001 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T01:50:48.001 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:48.053 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T01:50:48.054 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T01:50:48.054 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T01:50:48.054 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:48.118 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T01:50:48.119 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T01:50:48.119 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T01:50:48.119 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:48.131 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T01:50:48.132 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T01:50:48.132 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:48.189 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T01:50:48.189 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T01:50:48.189 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:48.245 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T01:50:48.245 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-09-15T01:50:48.246 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:48.302 INFO:tasks.ceph.mon.d:Stopped 2024-09-15T01:50:48.302 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-15T01:50:48.302 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:48.357 INFO:tasks.ceph.mon.e:Stopped 2024-09-15T01:50:48.357 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-15T01:50:48.357 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:48.370 INFO:tasks.ceph.mon.f:Stopped 2024-09-15T01:50:48.370 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-15T01:50:48.370 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:48.425 INFO:tasks.ceph.mon.g:Stopped 2024-09-15T01:50:48.426 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-09-15T01:50:48.426 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:48.484 INFO:tasks.ceph.mon.h:Stopped 2024-09-15T01:50:48.484 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-15T01:50:48.485 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T01:50:48.538 INFO:tasks.ceph.mon.i:Stopped 2024-09-15T01:50:48.539 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T01:50:48.539 DEBUG:teuthology.orchestra.run.smithi178:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v ScrubResult | egrep -v 'scrub mismatch' | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(AUTH_BAD_CAPS\)' | head -n 1 2024-09-15T01:50:48.583 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi178.front.sepia.ceph.com 2024-09-15T01:50:48.583 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T01:50:48.797 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi178.front.sepia.ceph.com 2024-09-15T01:50:48.797 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T01:50:48.860 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi178.front.sepia.ceph.com 2024-09-15T01:50:48.860 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T01:50:48.920 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi204.front.sepia.ceph.com 2024-09-15T01:50:48.920 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T01:50:49.062 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi204.front.sepia.ceph.com 2024-09-15T01:50:49.062 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T01:50:49.132 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi204.front.sepia.ceph.com 2024-09-15T01:50:49.132 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T01:50:49.183 INFO:tasks.ceph:Archiving mon data... 2024-09-15T01:50:49.184 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/data/mon.a.tgz 2024-09-15T01:50:49.184 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-09-15T01:50:49.200 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.y6r9WxsAC5 2024-09-15T01:50:49.200 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.y6r9WxsAC5 2024-09-15T01:50:49.449 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.y6r9WxsAC5 2024-09-15T01:50:49.533 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.y6r9WxsAC5 is 999KB 2024-09-15T01:50:49.645 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.y6r9WxsAC5 2024-09-15T01:50:49.660 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/data/mon.b.tgz 2024-09-15T01:50:49.661 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-09-15T01:50:49.715 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.QOqsUfPdJu 2024-09-15T01:50:49.715 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.QOqsUfPdJu 2024-09-15T01:50:49.921 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.QOqsUfPdJu 2024-09-15T01:50:50.003 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.QOqsUfPdJu is 1004KB 2024-09-15T01:50:50.081 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.QOqsUfPdJu 2024-09-15T01:50:50.097 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/data/mon.c.tgz 2024-09-15T01:50:50.097 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-09-15T01:50:50.151 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.qij0tULxRX 2024-09-15T01:50:50.151 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.qij0tULxRX 2024-09-15T01:50:50.355 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.qij0tULxRX 2024-09-15T01:50:50.438 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.qij0tULxRX is 1006KB 2024-09-15T01:50:50.516 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.qij0tULxRX 2024-09-15T01:50:50.531 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-d to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/data/mon.d.tgz 2024-09-15T01:50:50.532 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-09-15T01:50:50.586 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.OoVGkQ2SBp 2024-09-15T01:50:50.586 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-d -- . > /tmp/tmp.OoVGkQ2SBp 2024-09-15T01:50:50.793 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.OoVGkQ2SBp 2024-09-15T01:50:50.876 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.OoVGkQ2SBp is 1003KB 2024-09-15T01:50:50.954 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.OoVGkQ2SBp 2024-09-15T01:50:50.969 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-e to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/data/mon.e.tgz 2024-09-15T01:50:50.969 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-09-15T01:50:51.024 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.dknFpbCK6R 2024-09-15T01:50:51.024 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-e -- . > /tmp/tmp.dknFpbCK6R 2024-09-15T01:50:51.226 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.dknFpbCK6R 2024-09-15T01:50:51.308 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.dknFpbCK6R is 1001KB 2024-09-15T01:50:51.414 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.dknFpbCK6R 2024-09-15T01:50:51.429 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-f to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/data/mon.f.tgz 2024-09-15T01:50:51.429 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-09-15T01:50:51.448 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.LiXoIFbSr3 2024-09-15T01:50:51.448 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-f -- . > /tmp/tmp.LiXoIFbSr3 2024-09-15T01:50:51.735 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.LiXoIFbSr3 2024-09-15T01:50:51.830 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.LiXoIFbSr3 is 997KB 2024-09-15T01:50:51.986 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.LiXoIFbSr3 2024-09-15T01:50:52.002 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-g to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/data/mon.g.tgz 2024-09-15T01:50:52.002 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-09-15T01:50:52.056 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.jlpN9PeIzt 2024-09-15T01:50:52.057 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-g -- . > /tmp/tmp.jlpN9PeIzt 2024-09-15T01:50:52.265 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.jlpN9PeIzt 2024-09-15T01:50:52.349 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.jlpN9PeIzt is 1004KB 2024-09-15T01:50:52.425 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.jlpN9PeIzt 2024-09-15T01:50:52.441 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-h to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/data/mon.h.tgz 2024-09-15T01:50:52.441 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-09-15T01:50:52.495 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.SB2EJcG3Wg 2024-09-15T01:50:52.496 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-h -- . > /tmp/tmp.SB2EJcG3Wg 2024-09-15T01:50:52.704 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.SB2EJcG3Wg 2024-09-15T01:50:52.787 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.SB2EJcG3Wg is 1005KB 2024-09-15T01:50:52.892 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.SB2EJcG3Wg 2024-09-15T01:50:52.908 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-i to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/data/mon.i.tgz 2024-09-15T01:50:52.909 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-09-15T01:50:52.963 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.KuAJ2QBXZQ 2024-09-15T01:50:52.963 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-i -- . > /tmp/tmp.KuAJ2QBXZQ 2024-09-15T01:50:53.302 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.KuAJ2QBXZQ 2024-09-15T01:50:53.385 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.KuAJ2QBXZQ is 2MB 2024-09-15T01:50:53.505 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.KuAJ2QBXZQ 2024-09-15T01:50:53.521 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T01:50:53.521 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T01:50:53.524 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T01:50:53.586 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-15T01:50:53.587 DEBUG:teuthology.orchestra.run.smithi178:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T01:50:53.595 DEBUG:teuthology.orchestra.run.smithi204:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T01:50:53.623 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T01:50:53.653 INFO:teuthology.orchestra.run.smithi204.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T01:50:53.658 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T01:50:53.659 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/remote/smithi178/crash 2024-09-15T01:50:53.660 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T01:50:53.696 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/remote/smithi204/crash 2024-09-15T01:50:53.697 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T01:50:53.722 INFO:tasks.ceph:Compressing logs... 2024-09-15T01:50:53.722 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T01:50:53.742 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T01:50:54.102 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T01:50:54.102 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38267.log 2024-09-15T01:50:54.102 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34844.log 2024-09-15T01:50:54.102 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31300.log 2024-09-15T01:50:54.102 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36921.log 2024-09-15T01:50:54.102 INFO:teuthology.orchestra.run.smithi178.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38267.log.gz 2024-09-15T01:50:54.102 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38897.log 2024-09-15T01:50:54.102 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31300.log: gzip/var/log/ceph/ceph-client.admin.36921.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41499.log 2024-09-15T01:50:54.102 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38897.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.34844.log.gz 2024-09-15T01:50:54.102 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41835.log 2024-09-15T01:50:54.103 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.31300.log.gz 2024-09-15T01:50:54.103 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.36921.log.gz 2024-09-15T01:50:54.103 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41499.log: gzip 89.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.38897.log.gz -- 2024-09-15T01:50:54.103 INFO:teuthology.orchestra.run.smithi178.stderr: /var/log/ceph/ceph-client.admin.35669.log 2024-09-15T01:50:54.103 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39065.log 2024-09-15T01:50:54.103 INFO:teuthology.orchestra.run.smithi178.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41499.log.gz 2024-09-15T01:50:54.103 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31216.log 2024-09-15T01:50:54.103 INFO:teuthology.orchestra.run.smithi178.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.41835.log.gz 2024-09-15T01:50:54.103 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43451.log 2024-09-15T01:50:54.103 INFO:teuthology.orchestra.run.smithi178.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35669.log.gz 2024-09-15T01:50:54.104 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32036.log 2024-09-15T01:50:54.104 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43451.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.39065.log.gz 2024-09-15T01:50:54.104 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36669.log 2024-09-15T01:50:54.104 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.31216.log.gz 2024-09-15T01:50:54.104 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32036.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43451.log.gz 2024-09-15T01:50:54.104 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33709.log 2024-09-15T01:50:54.104 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36669.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32036.log.gz 2024-09-15T01:50:54.104 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39713.log 2024-09-15T01:50:54.104 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36215.log 2024-09-15T01:50:54.104 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.36669.log.gz 2024-09-15T01:50:54.104 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37761.log 2024-09-15T01:50:54.105 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.33709.log.gz 2024-09-15T01:50:54.105 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32078.log 2024-09-15T01:50:54.105 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36215.log: /var/log/ceph/ceph-client.admin.37761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29551.log 2024-09-15T01:50:54.105 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.39713.log.gz 2024-09-15T01:50:54.105 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32078.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36215.log.gz 2024-09-15T01:50:54.105 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33254.log 2024-09-15T01:50:54.105 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.37761.log.gz 2024-09-15T01:50:54.105 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32918.log 2024-09-15T01:50:54.105 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.29551.log: /var/log/ceph/ceph-client.admin.33254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42358.log 2024-09-15T01:50:54.105 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.32078.log.gz 2024-09-15T01:50:54.105 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32918.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29551.log.gz 2024-09-15T01:50:54.106 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40219.log 2024-09-15T01:50:54.106 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.33254.log.gz 2024-09-15T01:50:54.106 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42358.log: -- /var/log/ceph/ceph-client.admin.30009.log 2024-09-15T01:50:54.106 INFO:teuthology.orchestra.run.smithi178.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32918.log.gz 2024-09-15T01:50:54.106 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42148.log 2024-09-15T01:50:54.106 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.42358.log.gz 2024-09-15T01:50:54.106 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30009.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40219.log.gz 2024-09-15T01:50:54.106 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41373.log 2024-09-15T01:50:54.106 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30265.log 2024-09-15T01:50:54.106 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42148.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30009.log.gz 2024-09-15T01:50:54.106 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41162.log 2024-09-15T01:50:54.107 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36753.log 2024-09-15T01:50:54.107 INFO:teuthology.orchestra.run.smithi178.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.42148.log.gz 2024-09-15T01:50:54.107 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.41373.log.gz 2024-09-15T01:50:54.107 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41120.log 2024-09-15T01:50:54.107 INFO:teuthology.orchestra.run.smithi178.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30265.log.gz 2024-09-15T01:50:54.107 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37719.log 2024-09-15T01:50:54.107 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.41162.log.gz 2024-09-15T01:50:54.107 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41331.log 2024-09-15T01:50:54.107 INFO:teuthology.orchestra.run.smithi178.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.36753.log.gz 2024-09-15T01:50:54.107 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35753.log 2024-09-15T01:50:54.107 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.41120.log.gz 2024-09-15T01:50:54.107 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38519.log 2024-09-15T01:50:54.108 INFO:teuthology.orchestra.run.smithi178.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37719.log.gz 2024-09-15T01:50:54.108 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38182.log 2024-09-15T01:50:54.108 INFO:teuthology.orchestra.run.smithi178.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.41331.log.gz 2024-09-15T01:50:54.108 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42906.log 2024-09-15T01:50:54.108 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.35753.log.gz 2024-09-15T01:50:54.108 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38182.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T01:50:54.108 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.38519.log.gz 2024-09-15T01:50:54.108 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34339.log 2024-09-15T01:50:54.108 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42906.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.38182.log.gz 2024-09-15T01:50:54.108 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37425.log 2024-09-15T01:50:54.108 INFO:teuthology.orchestra.run.smithi178.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42906.log.gz 2024-09-15T01:50:54.109 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30880.log 2024-09-15T01:50:54.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34339.log: /var/log/ceph/ceph-client.admin.37425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39629.log 2024-09-15T01:50:54.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41415.log 2024-09-15T01:50:54.109 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.34339.log.gz 2024-09-15T01:50:54.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35795.log 2024-09-15T01:50:54.109 INFO:teuthology.orchestra.run.smithi178.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30880.log.gz 2024-09-15T01:50:54.109 INFO:teuthology.orchestra.run.smithi178.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.37425.log.gz 2024-09-15T01:50:54.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41415.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39629.log.gz 2024-09-15T01:50:54.109 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41078.log 2024-09-15T01:50:54.109 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39166.log 2024-09-15T01:50:54.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35795.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.41415.log.gz 2024-09-15T01:50:54.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30007.log 2024-09-15T01:50:54.110 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38056.log 2024-09-15T01:50:54.110 INFO:teuthology.orchestra.run.smithi178.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.35795.log.gz 2024-09-15T01:50:54.110 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.41078.log.gz 2024-09-15T01:50:54.110 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37635.log 2024-09-15T01:50:54.110 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34129.log 2024-09-15T01:50:54.110 INFO:teuthology.orchestra.run.smithi178.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.39166.log.gz 2024-09-15T01:50:54.110 INFO:teuthology.orchestra.run.smithi178.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30007.log.gz 2024-09-15T01:50:54.110 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5/var/log/ceph/ceph-client.admin.37635.log: --verbose -- /var/log/ceph/ceph-client.admin.35921.log 2024-09-15T01:50:54.110 INFO:teuthology.orchestra.run.smithi178.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.38056.log.gz 2024-09-15T01:50:54.110 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37551.log 2024-09-15T01:50:54.110 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.37635.log.gz 2024-09-15T01:50:54.111 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35963.log 2024-09-15T01:50:54.111 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35921.log: 90.5%/var/log/ceph/ceph-client.admin.37551.log: -- replaced with /var/log/ceph/ceph-client.admin.34129.log.gz 2024-09-15T01:50:54.111 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41541.log 2024-09-15T01:50:54.111 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35963.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35921.log.gz 2024-09-15T01:50:54.111 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32708.log 2024-09-15T01:50:54.111 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.37551.log.gz 2024-09-15T01:50:54.111 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35627.log 2024-09-15T01:50:54.111 INFO:teuthology.orchestra.run.smithi178.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.35963.log.gz 2024-09-15T01:50:54.111 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30438.log 2024-09-15T01:50:54.111 INFO:teuthology.orchestra.run.smithi178.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41541.log.gz 2024-09-15T01:50:54.111 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41751.log 2024-09-15T01:50:54.112 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30438.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.32708.log.gz 2024-09-15T01:50:54.112 INFO:teuthology.orchestra.run.smithi178.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35627.log.gz 2024-09-15T01:50:54.112 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33086.log 2024-09-15T01:50:54.112 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43242.log 2024-09-15T01:50:54.112 INFO:teuthology.orchestra.run.smithi178.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30438.log.gz 2024-09-15T01:50:54.112 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30263.log 2024-09-15T01:50:54.112 INFO:teuthology.orchestra.run.smithi178.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.41751.log.gz 2024-09-15T01:50:54.112 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30558.log 2024-09-15T01:50:54.112 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.33086.log.gz 2024-09-15T01:50:54.112 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38351.log 2024-09-15T01:50:54.113 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30558.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43242.log.gz 2024-09-15T01:50:54.113 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42570.log 2024-09-15T01:50:54.113 INFO:teuthology.orchestra.run.smithi178.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30263.log.gz 2024-09-15T01:50:54.113 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38351.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30558.log.gz 2024-09-15T01:50:54.113 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40742.log 2024-09-15T01:50:54.113 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31510.log 2024-09-15T01:50:54.113 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42570.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.38351.log.gz 2024-09-15T01:50:54.113 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37173.log 2024-09-15T01:50:54.113 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31510.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42570.log.gz 2024-09-15T01:50:54.113 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31426.log 2024-09-15T01:50:54.113 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.40742.log.gz 2024-09-15T01:50:54.113 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32498.log 2024-09-15T01:50:54.114 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.31510.log.gz 2024-09-15T01:50:54.114 INFO:teuthology.orchestra.run.smithi178.stderr:gzip/var/log/ceph/ceph-client.admin.31426.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33625.log 2024-09-15T01:50:54.114 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32498.log: 90.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.37173.log.gz 2024-09-15T01:50:54.114 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30498.log 2024-09-15T01:50:54.114 INFO:teuthology.orchestra.run.smithi178.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31426.log.gz 2024-09-15T01:50:54.114 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41982.log 2024-09-15T01:50:54.114 INFO:teuthology.orchestra.run.smithi178.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.32498.log.gz 2024-09-15T01:50:54.114 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30796.log 2024-09-15T01:50:54.114 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30498.log: /var/log/ceph/ceph-client.admin.41982.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.33625.log.gz 2024-09-15T01:50:54.114 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30012.log 2024-09-15T01:50:54.115 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30796.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30498.log.gz 2024-09-15T01:50:54.115 INFO:teuthology.orchestra.run.smithi178.stderr:gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41982.log.gz 2024-09-15T01:50:54.115 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.31048.log 2024-09-15T01:50:54.115 INFO:teuthology.orchestra.run.smithi178.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30796.log.gz 2024-09-15T01:50:54.115 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31384.log 2024-09-15T01:50:54.115 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5/var/log/ceph/ceph-client.admin.31048.log: --verbose -- /var/log/ceph/ceph-client.admin.35543.log 2024-09-15T01:50:54.115 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31384.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30012.log.gz 2024-09-15T01:50:54.115 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34003.log 2024-09-15T01:50:54.115 INFO:teuthology.orchestra.run.smithi178.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31048.log.gz 2024-09-15T01:50:54.115 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42190.log 2024-09-15T01:50:54.115 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28887.log 2024-09-15T01:50:54.115 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.31384.log.gz 2024-09-15T01:50:54.116 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34003.log: /var/log/ceph/ceph-client.admin.42190.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.35543.log.gz 2024-09-15T01:50:54.116 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36585.log 2024-09-15T01:50:54.116 INFO:teuthology.orchestra.run.smithi178.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34003.log.gzgzip 2024-09-15T01:50:54.116 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.32288.log 2024-09-15T01:50:54.116 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.28887.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.42190.log.gz 2024-09-15T01:50:54.116 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41583.log 2024-09-15T01:50:54.116 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40050.log 2024-09-15T01:50:54.116 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32288.log: /var/log/ceph/ceph-client.admin.41583.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.36585.log.gz 2024-09-15T01:50:54.116 INFO:teuthology.orchestra.run.smithi178.stderr:gzip 92.0% -- replaced with /var/log/ceph/ceph-client.admin.28887.log.gz 2024-09-15T01:50:54.117 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39755.log 2024-09-15T01:50:54.117 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.32288.log.gz 2024-09-15T01:50:54.117 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39419.log 2024-09-15T01:50:54.117 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39755.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.41583.log.gz 2024-09-15T01:50:54.117 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40952.log 2024-09-15T01:50:54.117 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.40050.log.gz 2024-09-15T01:50:54.117 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30264.log 2024-09-15T01:50:54.117 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39419.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.39755.log.gz 2024-09-15T01:50:54.117 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43284.log 2024-09-15T01:50:54.117 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39335.log 2024-09-15T01:50:54.117 INFO:teuthology.orchestra.run.smithi178.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.39419.log.gz 2024-09-15T01:50:54.118 INFO:teuthology.orchestra.run.smithi178.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.40952.log.gz 2024-09-15T01:50:54.118 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37930.log 2024-09-15T01:50:54.118 INFO:teuthology.orchestra.run.smithi178.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30264.log.gz 2024-09-15T01:50:54.118 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38981.log 2024-09-15T01:50:54.118 INFO:teuthology.orchestra.run.smithi178.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43284.log.gz 2024-09-15T01:50:54.118 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29788.log 2024-09-15T01:50:54.118 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38981.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.39335.log.gz 2024-09-15T01:50:54.118 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37215.log 2024-09-15T01:50:54.118 INFO:teuthology.orchestra.run.smithi178.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.37930.log.gz 2024-09-15T01:50:54.118 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.29788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37341.log 2024-09-15T01:50:54.118 INFO:teuthology.orchestra.run.smithi178.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.38981.log.gz 2024-09-15T01:50:54.118 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37215.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29788.log.gz 2024-09-15T01:50:54.119 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32120.log 2024-09-15T01:50:54.119 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36879.log 2024-09-15T01:50:54.119 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37341.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.37215.log.gz 2024-09-15T01:50:54.119 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38645.log 2024-09-15T01:50:54.119 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36879.log: gzip 90.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.37341.log.gz --verbose 2024-09-15T01:50:54.119 INFO:teuthology.orchestra.run.smithi178.stderr: -- /var/log/ceph/ceph-client.admin.31808.log 2024-09-15T01:50:54.119 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.32120.log.gz 2024-09-15T01:50:54.119 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39545.log 2024-09-15T01:50:54.119 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.36879.log.gz 2024-09-15T01:50:54.119 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34297.log 2024-09-15T01:50:54.119 INFO:teuthology.orchestra.run.smithi178.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.38645.log.gz 2024-09-15T01:50:54.120 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39545.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31808.log.gz 2024-09-15T01:50:54.120 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30750.log 2024-09-15T01:50:54.120 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34297.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.39545.log.gz 2024-09-15T01:50:54.120 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T01:50:54.120 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42527.log 2024-09-15T01:50:54.120 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30750.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34297.log.gz 2024-09-15T01:50:54.120 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34423.log 2024-09-15T01:50:54.120 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42527.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30750.log.gz 2024-09-15T01:50:54.120 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40994.log 2024-09-15T01:50:54.120 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42064.log 2024-09-15T01:50:54.120 INFO:teuthology.orchestra.run.smithi178.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42527.log.gz 2024-09-15T01:50:54.121 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33296.log 2024-09-15T01:50:54.121 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42064.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34423.log.gz 2024-09-15T01:50:54.121 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39251.log 2024-09-15T01:50:54.121 INFO:teuthology.orchestra.run.smithi178.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40994.log.gz 2024-09-15T01:50:54.121 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33296.log: gzip 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42064.log.gz -5 2024-09-15T01:50:54.121 INFO:teuthology.orchestra.run.smithi178.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34507.log 2024-09-15T01:50:54.121 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37467.log 2024-09-15T01:50:54.121 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39251.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33296.log.gz 2024-09-15T01:50:54.121 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33002.log 2024-09-15T01:50:54.121 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.39251.log.gz 2024-09-15T01:50:54.121 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37467.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34507.log.gz 2024-09-15T01:50:54.121 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36795.log 2024-09-15T01:50:54.122 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38687.log 2024-09-15T01:50:54.122 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33002.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.37467.log.gz 2024-09-15T01:50:54.122 INFO:teuthology.orchestra.run.smithi178.stderr:gzip/var/log/ceph/ceph-client.admin.36795.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41457.log 2024-09-15T01:50:54.122 INFO:teuthology.orchestra.run.smithi178.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33002.log.gz 2024-09-15T01:50:54.122 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40429.log 2024-09-15T01:50:54.122 INFO:teuthology.orchestra.run.smithi178.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.36795.log.gz 2024-09-15T01:50:54.122 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40658.log 2024-09-15T01:50:54.122 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.38687.log.gz 2024-09-15T01:50:54.122 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35585.log 2024-09-15T01:50:54.122 INFO:teuthology.orchestra.run.smithi178.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41457.log.gz 2024-09-15T01:50:54.122 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33667.log 2024-09-15T01:50:54.123 INFO:teuthology.orchestra.run.smithi178.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.40429.log.gz 2024-09-15T01:50:54.123 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38435.log 2024-09-15T01:50:54.123 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.40658.log.gz 2024-09-15T01:50:54.123 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40008.log 2024-09-15T01:50:54.123 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.35585.log.gz 2024-09-15T01:50:54.123 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32540.log 2024-09-15T01:50:54.123 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.33667.log.gz 2024-09-15T01:50:54.123 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34886.log 2024-09-15T01:50:54.123 INFO:teuthology.orchestra.run.smithi178.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.38435.log.gz 2024-09-15T01:50:54.123 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32330.log 2024-09-15T01:50:54.123 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.40008.log.gz 2024-09-15T01:50:54.124 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32162.log 2024-09-15T01:50:54.124 INFO:teuthology.orchestra.run.smithi178.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.32540.log.gz 2024-09-15T01:50:54.124 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38855.log 2024-09-15T01:50:54.124 INFO:teuthology.orchestra.run.smithi178.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34886.log.gz 2024-09-15T01:50:54.124 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42442.log 2024-09-15T01:50:54.124 INFO:teuthology.orchestra.run.smithi178.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.32330.log.gz 2024-09-15T01:50:54.124 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32204.log 2024-09-15T01:50:54.124 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.32162.log.gz 2024-09-15T01:50:54.124 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34213.log 2024-09-15T01:50:54.124 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32204.log: 90.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43158.log 2024-09-15T01:50:54.124 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38855.log.gz 2024-09-15T01:50:54.125 INFO:teuthology.orchestra.run.smithi178.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.42442.log.gz 2024-09-15T01:50:54.125 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42484.log 2024-09-15T01:50:54.125 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.32204.log.gz 2024-09-15T01:50:54.125 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36341.log 2024-09-15T01:50:54.125 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43158.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.34213.log.gz 2024-09-15T01:50:54.125 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32666.log 2024-09-15T01:50:54.125 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36341.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43158.log.gz 2024-09-15T01:50:54.125 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 90.6% -- replaced with /var/log/ceph/ceph-client.admin.42484.log.gz 2024-09-15T01:50:54.125 INFO:teuthology.orchestra.run.smithi178.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34087.log 2024-09-15T01:50:54.125 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31342.log 2024-09-15T01:50:54.125 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.36341.log.gz 2024-09-15T01:50:54.125 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37047.log 2024-09-15T01:50:54.126 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.32666.log.gz 2024-09-15T01:50:54.126 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36299.log 90.3% -- replaced with /var/log/ceph/ceph-client.admin.34087.log.gz 2024-09-15T01:50:54.126 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:54.126 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.31342.log.gz 2024-09-15T01:50:54.126 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43200.log 2024-09-15T01:50:54.126 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37677.log 2024-09-15T01:50:54.126 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.37047.log.gz 2024-09-15T01:50:54.126 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35249.log 2024-09-15T01:50:54.126 INFO:teuthology.orchestra.run.smithi178.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.36299.log.gz 2024-09-15T01:50:54.126 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37677.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43200.log.gz 2024-09-15T01:50:54.127 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41247.log 2024-09-15T01:50:54.127 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40303.log 2024-09-15T01:50:54.127 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35249.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.37677.log.gz 2024-09-15T01:50:54.127 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36131.log 2024-09-15T01:50:54.127 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40303.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.41247.log.gz 2024-09-15T01:50:54.127 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.35249.log.gz 2024-09-15T01:50:54.127 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39209.log 2024-09-15T01:50:54.127 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39293.log 2024-09-15T01:50:54.127 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36131.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40303.log.gz 2024-09-15T01:50:54.127 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33793.log 2024-09-15T01:50:54.127 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39293.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39209.log.gz 2024-09-15T01:50:54.128 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41919.log 2024-09-15T01:50:54.128 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33793.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T01:50:54.128 INFO:teuthology.orchestra.run.smithi178.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.39293.log.gz 2024-09-15T01:50:54.128 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41919.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.36131.log.gz 2024-09-15T01:50:54.128 INFO:teuthology.orchestra.run.smithi178.stderr:gzip 90.4% -- replaced with /var/log/ceph/ceph-client.admin.33793.log.gz -5 2024-09-15T01:50:54.128 INFO:teuthology.orchestra.run.smithi178.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36089.log 2024-09-15T01:50:54.128 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.c.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.41919.log.gz 2024-09-15T01:50:54.128 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37131.log 2024-09-15T01:50:54.128 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42316.log 2024-09-15T01:50:54.128 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42990.log 2024-09-15T01:50:54.128 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42316.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36089.log.gz 2024-09-15T01:50:54.128 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31468.log 2024-09-15T01:50:54.129 INFO:teuthology.orchestra.run.smithi178.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.37131.log.gz 2024-09-15T01:50:54.129 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42990.log: -- /var/log/ceph/ceph-client.admin.31174.log 2024-09-15T01:50:54.129 INFO:teuthology.orchestra.run.smithi178.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.42316.log.gz 2024-09-15T01:50:54.129 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41625.log 2024-09-15T01:50:54.129 INFO:teuthology.orchestra.run.smithi178.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42990.log.gz/var/log/ceph/ceph-client.admin.31174.log: 2024-09-15T01:50:54.129 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34760.log 2024-09-15T01:50:54.129 INFO:teuthology.orchestra.run.smithi178.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31468.log.gz 2024-09-15T01:50:54.129 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5/var/log/ceph/ceph-client.admin.41625.log: --verbose -- 90.5% -- replaced with /var/log/ceph/ceph-client.admin.31174.log.gz 2024-09-15T01:50:54.129 INFO:teuthology.orchestra.run.smithi178.stderr: /var/log/ceph/ceph-client.admin.42274.log 2024-09-15T01:50:54.129 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41036.log 2024-09-15T01:50:54.130 INFO:teuthology.orchestra.run.smithi178.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.41625.log.gz/var/log/ceph/ceph-client.admin.42274.log: 2024-09-15T01:50:54.130 INFO:teuthology.orchestra.run.smithi178.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34760.log.gzgzip 2024-09-15T01:50:54.130 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40532.log 2024-09-15T01:50:54.130 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36257.log 2024-09-15T01:50:54.130 INFO:teuthology.orchestra.run.smithi178.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.42274.log.gz 2024-09-15T01:50:54.130 INFO:teuthology.orchestra.run.smithi178.stderr: 95.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T01:50:54.130 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41036.log: /var/log/ceph/ceph-client.admin.40532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33877.log 2024-09-15T01:50:54.130 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36257.log: gzip -5 89.7% -- replaced with /var/log/ceph/ceph-client.admin.40532.log.gz 2024-09-15T01:50:54.130 INFO:teuthology.orchestra.run.smithi178.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36543.log 2024-09-15T01:50:54.130 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.41036.log.gz 2024-09-15T01:50:54.130 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40387.log 2024-09-15T01:50:54.130 INFO:teuthology.orchestra.run.smithi178.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.36257.log.gz 2024-09-15T01:50:54.131 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35711.log 2024-09-15T01:50:54.131 INFO:teuthology.orchestra.run.smithi178.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.33877.log.gz 2024-09-15T01:50:54.131 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34381.log 2024-09-15T01:50:54.131 INFO:teuthology.orchestra.run.smithi178.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36543.log.gz 2024-09-15T01:50:54.131 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33499.log 2024-09-15T01:50:54.131 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34381.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.40387.log.gz 2024-09-15T01:50:54.131 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31090.log 2024-09-15T01:50:54.131 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.35711.log.gz 2024-09-15T01:50:54.131 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40134.log 2024-09-15T01:50:54.131 INFO:teuthology.orchestra.run.smithi178.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34381.log.gz 2024-09-15T01:50:54.132 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30011.log 2024-09-15T01:50:54.132 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37846.log 2024-09-15T01:50:54.132 INFO:teuthology.orchestra.run.smithi178.stderr: 91.5% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33499.log.gz 2024-09-15T01:50:54.132 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31090.log.gz 2024-09-15T01:50:54.132 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43116.log 2024-09-15T01:50:54.132 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37846.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30011.log.gz 2024-09-15T01:50:54.132 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35459.log 2024-09-15T01:50:54.132 INFO:teuthology.orchestra.run.smithi178.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.40134.log.gz 2024-09-15T01:50:54.132 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43116.log: gzip -5 90.2% -- replaced with /var/log/ceph/ceph-client.admin.37846.log.gz 2024-09-15T01:50:54.132 INFO:teuthology.orchestra.run.smithi178.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30014.log 2024-09-15T01:50:54.132 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38309.log 2024-09-15T01:50:54.132 INFO:teuthology.orchestra.run.smithi178.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43116.log.gz 2024-09-15T01:50:54.133 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31892.log 2024-09-15T01:50:54.133 INFO:teuthology.orchestra.run.smithi178.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.35459.log.gz 2024-09-15T01:50:54.133 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38813.log 2024-09-15T01:50:54.133 INFO:teuthology.orchestra.run.smithi178.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30014.log.gz 2024-09-15T01:50:54.133 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42106.log 2024-09-15T01:50:54.133 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.38309.log.gz 2024-09-15T01:50:54.133 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32750.log 2024-09-15T01:50:54.133 INFO:teuthology.orchestra.run.smithi178.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.31892.log.gz 2024-09-15T01:50:54.133 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31132.log 2024-09-15T01:50:54.133 INFO:teuthology.orchestra.run.smithi178.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42106.log.gz 2024-09-15T01:50:54.133 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36005.log 2024-09-15T01:50:54.134 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31132.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.38813.log.gz 2024-09-15T01:50:54.134 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38393.log 2024-09-15T01:50:54.134 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.32750.log.gz 2024-09-15T01:50:54.134 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33961.log 2024-09-15T01:50:54.134 INFO:teuthology.orchestra.run.smithi178.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31132.log.gz 2024-09-15T01:50:54.134 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38393.log: gzip -5 --verbose -- 90.2% /var/log/ceph/ceph-client.admin.30010.log 2024-09-15T01:50:54.134 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36005.log.gz 2024-09-15T01:50:54.134 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29737.log 2024-09-15T01:50:54.134 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.38393.log.gz 2024-09-15T01:50:54.134 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39839.log 2024-09-15T01:50:54.134 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9%/var/log/ceph/ceph-client.admin.29737.log: -- replaced with /var/log/ceph/ceph-client.admin.33961.log.gz 2024-09-15T01:50:54.134 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39461.log 2024-09-15T01:50:54.135 INFO:teuthology.orchestra.run.smithi178.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30010.log.gz 2024-09-15T01:50:54.135 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31762.log 2024-09-15T01:50:54.135 INFO:teuthology.orchestra.run.smithi178.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29737.log.gz 2024-09-15T01:50:54.135 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40616.log 2024-09-15T01:50:54.135 INFO:teuthology.orchestra.run.smithi178.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.39839.log.gz 2024-09-15T01:50:54.135 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32414.log 2024-09-15T01:50:54.135 INFO:teuthology.orchestra.run.smithi178.stderr:gzip 90.4% -- replaced with /var/log/ceph/ceph-client.admin.39461.log.gz 2024-09-15T01:50:54.135 INFO:teuthology.orchestra.run.smithi178.stderr: -5/var/log/ceph/ceph-client.admin.40616.log: --verbose -- /var/log/ceph/ceph-client.admin.32834.log 2024-09-15T01:50:54.135 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32414.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31762.log.gz 2024-09-15T01:50:54.135 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30555.log 2024-09-15T01:50:54.135 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.40616.log.gz 2024-09-15T01:50:54.136 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32834.log: gzip -5 90.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.32414.log.gz 2024-09-15T01:50:54.136 INFO:teuthology.orchestra.run.smithi178.stderr: /var/log/ceph/ceph-client.admin.30266.log 2024-09-15T01:50:54.136 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39587.log 2024-09-15T01:50:54.136 INFO:teuthology.orchestra.run.smithi178.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.32834.log.gz 2024-09-15T01:50:54.136 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34718.log 2024-09-15T01:50:54.136 INFO:teuthology.orchestra.run.smithi178.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30555.log.gz 2024-09-15T01:50:54.136 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39587.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30266.log.gz 2024-09-15T01:50:54.136 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T01:50:54.136 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34718.log: gzip 90.5% -- replaced with /var/log/ceph/ceph-client.admin.39587.log.gz 2024-09-15T01:50:54.136 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.31258.log 2024-09-15T01:50:54.136 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39881.log 2024-09-15T01:50:54.137 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31258.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.34718.log.gz 2024-09-15T01:50:54.137 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.e.log 2024-09-15T01:50:54.137 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35837.log 2024-09-15T01:50:54.137 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.31258.log.gz 2024-09-15T01:50:54.137 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.e.log: gzip -5 --verbose 90.7% -- replaced with /var/log/ceph/ceph-client.admin.39881.log.gz 2024-09-15T01:50:54.137 INFO:teuthology.orchestra.run.smithi178.stderr: -- /var/log/ceph/ceph-client.admin.36425.log 2024-09-15T01:50:54.137 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32792.log 2024-09-15T01:50:54.137 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36425.log: 90.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29712.log 2024-09-15T01:50:54.137 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35837.log.gz 2024-09-15T01:50:54.137 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32792.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.36425.log.gz 2024-09-15T01:50:54.138 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36711.log 2024-09-15T01:50:54.138 INFO:teuthology.orchestra.run.smithi178.stderr: 89.8%/var/log/ceph/ceph-client.admin.29712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33044.log 2024-09-15T01:50:54.138 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32792.log.gz 2024-09-15T01:50:54.138 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36711.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29712.log.gz 2024-09-15T01:50:54.138 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43074.log 2024-09-15T01:50:54.138 INFO:teuthology.orchestra.run.smithi178.stderr:gzip 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36711.log.gz 2024-09-15T01:50:54.138 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42864.log 2024-09-15T01:50:54.138 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43074.log: /var/log/ceph/ceph-client.admin.33044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30690.log 2024-09-15T01:50:54.138 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42864.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43074.log.gz 2024-09-15T01:50:54.138 INFO:teuthology.orchestra.run.smithi178.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.33044.log.gz 2024-09-15T01:50:54.139 INFO:teuthology.orchestra.run.smithi178.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38771.log 2024-09-15T01:50:54.139 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42864.log.gz 2024-09-15T01:50:54.139 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37888.log 2024-09-15T01:50:54.139 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30838.log 2024-09-15T01:50:54.139 INFO:teuthology.orchestra.run.smithi178.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30690.log.gz 2024-09-15T01:50:54.139 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38771.log: /var/log/ceph/ceph-client.admin.37888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34633.log 2024-09-15T01:50:54.139 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30838.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.38771.log.gz 2024-09-15T01:50:54.139 INFO:teuthology.orchestra.run.smithi178.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.37888.log.gz 2024-09-15T01:50:54.139 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35375.log 2024-09-15T01:50:54.139 INFO:teuthology.orchestra.run.smithi178.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30838.log.gz 2024-09-15T01:50:54.139 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39671.log 2024-09-15T01:50:54.140 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34633.log: /var/log/ceph/ceph-client.admin.35375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34465.log 2024-09-15T01:50:54.140 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39671.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34633.log.gz 2024-09-15T01:50:54.140 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.35375.log.gz 2024-09-15T01:50:54.140 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T01:50:54.140 INFO:teuthology.orchestra.run.smithi178.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.39671.log.gz 2024-09-15T01:50:54.140 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37972.log 2024-09-15T01:50:54.140 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37089.log 2024-09-15T01:50:54.140 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.34465.log.gz 2024-09-15T01:50:54.140 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32624.log 2024-09-15T01:50:54.140 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37089.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.37972.log.gz 2024-09-15T01:50:54.140 INFO:teuthology.orchestra.run.smithi178.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37089.log.gz 2024-09-15T01:50:54.141 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34591.log 2024-09-15T01:50:54.141 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41667.log 2024-09-15T01:50:54.141 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34591.log: /var/log/ceph/ceph-client.admin.32624.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34591.log.gz 89.9% -- replaced with /var/log/ceph/ceph-client.admin.32624.log.gz 2024-09-15T01:50:54.141 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:54.141 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30418.log 2024-09-15T01:50:54.141 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37299.log 2024-09-15T01:50:54.141 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30418.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.41667.log.gz 2024-09-15T01:50:54.141 INFO:teuthology.orchestra.run.smithi178.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41793.log 2024-09-15T01:50:54.141 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30418.log.gz 2024-09-15T01:50:54.141 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32246.log 2024-09-15T01:50:54.141 INFO:teuthology.orchestra.run.smithi178.stderr: 89.5%/var/log/ceph/ceph-client.admin.41793.log: -- replaced with /var/log/ceph/ceph-client.admin.37299.log.gz 2024-09-15T01:50:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43369.log 2024-09-15T01:50:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32246.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.41793.log.gz 2024-09-15T01:50:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35005.log 2024-09-15T01:50:54.142 INFO:teuthology.orchestra.run.smithi178.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32246.log.gz 2024-09-15T01:50:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33212.log 2024-09-15T01:50:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35005.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43369.log.gz 2024-09-15T01:50:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37509.log 2024-09-15T01:50:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33212.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.35005.log.gz 2024-09-15T01:50:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31934.log 2024-09-15T01:50:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37509.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33212.log.gz 2024-09-15T01:50:54.142 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31720.log 2024-09-15T01:50:54.142 INFO:teuthology.orchestra.run.smithi178.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.37509.log.gz 2024-09-15T01:50:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36383.log 2024-09-15T01:50:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31720.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31934.log.gz 2024-09-15T01:50:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27433.log 2024-09-15T01:50:54.143 INFO:teuthology.orchestra.run.smithi178.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.31720.log.gz 2024-09-15T01:50:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41289.log 2024-09-15T01:50:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.tmp-client.admin.27433.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27433.log.gz 2024-09-15T01:50:54.143 INFO:teuthology.orchestra.run.smithi178.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.36383.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40177.log 2024-09-15T01:50:54.143 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T01:50:54.143 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41289.log: /var/log/ceph/ceph-client.admin.40177.log: 90.0% 90.5% -- replaced with /var/log/ceph/ceph-client.admin.41289.log.gz 2024-09-15T01:50:54.144 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40177.log.gz 2024-09-15T01:50:54.144 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35501.log 2024-09-15T01:50:54.144 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39377.log 2024-09-15T01:50:54.144 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35501.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.35501.log.gz 2024-09-15T01:50:54.144 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33170.log 2024-09-15T01:50:54.144 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39377.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.39377.log.gz 2024-09-15T01:50:54.144 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31636.log 2024-09-15T01:50:54.144 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33170.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33170.log.gz 2024-09-15T01:50:54.144 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42612.log 2024-09-15T01:50:54.144 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31636.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.31636.log.gz 2024-09-15T01:50:54.144 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37383.log 2024-09-15T01:50:54.145 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42612.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.42612.log.gz 2024-09-15T01:50:54.145 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41877.log 2024-09-15T01:50:54.145 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37383.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.37383.log.gz 2024-09-15T01:50:54.145 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32876.log 2024-09-15T01:50:54.145 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41877.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.41877.log.gz 2024-09-15T01:50:54.145 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38477.log 2024-09-15T01:50:54.145 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32876.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.32876.log.gz 2024-09-15T01:50:54.145 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40092.log 2024-09-15T01:50:54.145 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38477.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.38477.log.gz 2024-09-15T01:50:54.145 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40868.log 2024-09-15T01:50:54.146 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40092.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.40092.log.gz 2024-09-15T01:50:54.146 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30964.log 2024-09-15T01:50:54.146 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40868.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.40868.log.gz 2024-09-15T01:50:54.146 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32372.log 2024-09-15T01:50:54.146 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30964.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30964.log.gz 2024-09-15T01:50:54.146 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41205.log 2024-09-15T01:50:54.146 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32372.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.32372.log.gz 2024-09-15T01:50:54.146 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36627.log 2024-09-15T01:50:54.146 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41205.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.41205.log.gz 2024-09-15T01:50:54.146 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30689.log 2024-09-15T01:50:54.147 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36627.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.36627.log.gz 2024-09-15T01:50:54.147 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29231.log 2024-09-15T01:50:54.147 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30689.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30689.log.gz 2024-09-15T01:50:54.147 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32456.log 2024-09-15T01:50:54.147 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.29231.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29231.log.gz 2024-09-15T01:50:54.147 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36963.log 2024-09-15T01:50:54.147 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32456.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32456.log.gz 2024-09-15T01:50:54.147 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33583.log 2024-09-15T01:50:54.147 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36963.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36963.log.gz 2024-09-15T01:50:54.147 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39797.log 2024-09-15T01:50:54.147 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33583.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33583.log.gz 2024-09-15T01:50:54.148 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40784.log 2024-09-15T01:50:54.148 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39797.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.39797.log.gz 2024-09-15T01:50:54.148 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34802.log 2024-09-15T01:50:54.148 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40784.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.40784.log.gz 2024-09-15T01:50:54.148 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37804.log 2024-09-15T01:50:54.148 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34802.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.34802.log.gz 2024-09-15T01:50:54.148 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31850.log 2024-09-15T01:50:54.148 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37804.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.37804.log.gz 2024-09-15T01:50:54.148 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35417.log 2024-09-15T01:50:54.148 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31850.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.31850.log.gz 2024-09-15T01:50:54.148 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39923.log 2024-09-15T01:50:54.149 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35417.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.35417.log.gz 2024-09-15T01:50:54.149 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34676.log 2024-09-15T01:50:54.149 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39923.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.39923.log.gz 2024-09-15T01:50:54.149 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.d.log 2024-09-15T01:50:54.149 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34676.log: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.34676.log.gz 2024-09-15T01:50:54.149 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34549.log 2024-09-15T01:50:54.149 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43032.log 2024-09-15T01:50:54.149 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33919.log 2024-09-15T01:50:54.149 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43032.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34549.log.gz 2024-09-15T01:50:54.149 INFO:teuthology.orchestra.run.smithi178.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43032.log.gz 2024-09-15T01:50:54.150 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40574.log 2024-09-15T01:50:54.150 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33919.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.33919.log.gz 2024-09-15T01:50:54.150 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29656.log 2024-09-15T01:50:54.150 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40574.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.40574.log.gz 2024-09-15T01:50:54.150 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40700.log 2024-09-15T01:50:54.150 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.29656.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29656.log.gz 2024-09-15T01:50:54.150 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43326.log 2024-09-15T01:50:54.150 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40700.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.40700.log.gz 2024-09-15T01:50:54.150 INFO:teuthology.orchestra.run.smithi178.stderr: 89.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T01:50:54.150 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30557.log 2024-09-15T01:50:54.150 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43326.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43326.log.gz 2024-09-15T01:50:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39966.log 2024-09-15T01:50:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30557.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30557.log.gz 2024-09-15T01:50:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35879.log 2024-09-15T01:50:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39966.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.39966.log.gz 2024-09-15T01:50:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35291.log 2024-09-15T01:50:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35879.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35879.log.gz 2024-09-15T01:50:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38603.log 2024-09-15T01:50:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35291.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.35291.log.gz 2024-09-15T01:50:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42654.log 2024-09-15T01:50:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38603.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.38603.log.gz 2024-09-15T01:50:54.151 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29830.log 2024-09-15T01:50:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42654.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42654.log.gz 2024-09-15T01:50:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34045.log 2024-09-15T01:50:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.29830.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.29830.log.gz 2024-09-15T01:50:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T01:50:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34045.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34045.log.gz 2024-09-15T01:50:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36047.log 2024-09-15T01:50:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37005.log 2024-09-15T01:50:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36047.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.36047.log.gz 2024-09-15T01:50:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41709.log 2024-09-15T01:50:54.152 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37005.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.37005.log.gz 2024-09-15T01:50:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34255.log 2024-09-15T01:50:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41709.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.41709.log.gz 2024-09-15T01:50:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38939.log 2024-09-15T01:50:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34255.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34255.log.gz 2024-09-15T01:50:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35333.log 2024-09-15T01:50:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38939.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.38939.log.gz 2024-09-15T01:50:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42696.log 2024-09-15T01:50:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35333.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35333.log.gz 2024-09-15T01:50:54.153 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42738.log 2024-09-15T01:50:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42696.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42696.log.gz 2024-09-15T01:50:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31594.log 2024-09-15T01:50:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42738.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42738.log.gz 2024-09-15T01:50:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30922.log 2024-09-15T01:50:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31594.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31594.log.gz 2024-09-15T01:50:54.154 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40910.log 2024-09-15T01:50:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30922.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30922.log.gz 2024-09-15T01:50:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39023.log 2024-09-15T01:50:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40910.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.40910.log.gz 2024-09-15T01:50:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30256.log 2024-09-15T01:50:54.160 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39023.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.39023.log.gz 2024-09-15T01:50:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38224.log 2024-09-15T01:50:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30256.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30256.log.gz 2024-09-15T01:50:54.161 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31006.log 2024-09-15T01:50:54.164 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38224.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38224.log.gz 2024-09-15T01:50:54.175 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42780.log 2024-09-15T01:50:54.182 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31006.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31006.log.gz 2024-09-15T01:50:54.188 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38729.log 2024-09-15T01:50:54.189 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36027.log 2024-09-15T01:50:54.189 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.36954.log 2024-09-15T01:50:54.189 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35186.log 2024-09-15T01:50:54.189 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36027.log: /var/log/ceph/ceph-client.foo.36954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32920.log 2024-09-15T01:50:54.189 INFO:teuthology.orchestra.run.smithi204.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36027.log.gz 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29767.log 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.foo.36954.log.gz 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr:gzip/var/log/ceph/ceph-client.admin.32920.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32822.log 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35186.log.gz 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34825.log 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35955.log 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32920.log.gz 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.29767.log.gz 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32678.log 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32822.log.gz 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34017.log 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34825.log.gz 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32484.log 2024-09-15T01:50:54.190 INFO:teuthology.orchestra.run.smithi204.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35955.log.gz 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34607.log 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32678.log.gz 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33384.log 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34017.log.gz 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36387.log 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32484.log.gz 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35642.log 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34607.log.gz 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30099.log 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33384.log.gz 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35642.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.36387.log.gz 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-client.bar.37241.log 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30099.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35642.log.gz 2024-09-15T01:50:54.191 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29674.log 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.bar.37241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32195.log 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30099.log.gz 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.bar.37241.log.gz 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.29674.log: /var/log/ceph/ceph-client.admin.32147.log 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36435.log 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29674.log.gz 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30856.log 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32195.log.gz 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36171.log 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32147.log.gz 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30186.log 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36435.log.gz 2024-09-15T01:50:54.192 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32388.log 2024-09-15T01:50:54.193 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30856.log.gz 2024-09-15T01:50:54.193 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33720.log 2024-09-15T01:50:54.193 INFO:teuthology.orchestra.run.smithi204.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36171.log.gz 2024-09-15T01:50:54.193 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36547.log 2024-09-15T01:50:54.193 INFO:teuthology.orchestra.run.smithi204.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30186.log.gz 2024-09-15T01:50:54.193 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35546.log 2024-09-15T01:50:54.193 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32388.log.gz 2024-09-15T01:50:54.193 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28621.log 2024-09-15T01:50:54.193 INFO:teuthology.orchestra.run.smithi204.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33720.log.gz 2024-09-15T01:50:54.193 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35546.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.g.log 2024-09-15T01:50:54.193 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36547.log.gz 2024-09-15T01:50:54.194 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.28621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34371.log 2024-09-15T01:50:54.194 INFO:teuthology.orchestra.run.smithi204.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35546.log.gz 2024-09-15T01:50:54.194 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.g.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31863.log 2024-09-15T01:50:54.194 INFO:teuthology.orchestra.run.smithi204.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28621.log.gz 2024-09-15T01:50:54.194 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35018.log 2024-09-15T01:50:54.194 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32342.log 2024-09-15T01:50:54.194 INFO:teuthology.orchestra.run.smithi204.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34371.log.gz 2024-09-15T01:50:54.194 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33286.log 2024-09-15T01:50:54.194 INFO:teuthology.orchestra.run.smithi204.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.31863.log.gz 2024-09-15T01:50:54.194 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32031.log 2024-09-15T01:50:54.194 INFO:teuthology.orchestra.run.smithi204.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35018.log.gz 2024-09-15T01:50:54.194 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34419.log 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32342.log.gz 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30033.log 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33286.log.gz 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33310.log 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32031.log.gz 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34419.log: /var/log/ceph/ceph-client.admin.30033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35691.log 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34419.log.gz 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33262.log 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30033.log.gz 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35691.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33310.log.gz 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33408.log 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30880.log 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35691.log.gz 2024-09-15T01:50:54.195 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32896.log 2024-09-15T01:50:54.196 INFO:teuthology.orchestra.run.smithi204.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33262.log.gz 2024-09-15T01:50:54.196 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30880.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33408.log.gz 2024-09-15T01:50:54.196 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30607.log 2024-09-15T01:50:54.196 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35114.log 2024-09-15T01:50:54.196 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30880.log.gz 2024-09-15T01:50:54.196 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34113.log 2024-09-15T01:50:54.196 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32896.log.gz 2024-09-15T01:50:54.196 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31414.log 2024-09-15T01:50:54.196 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30607.log.gz 2024-09-15T01:50:54.196 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29819.log 2024-09-15T01:50:54.196 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35114.log.gz 2024-09-15T01:50:54.196 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34113.log: /var/log/ceph/ceph-client.admin.31414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34504.log 2024-09-15T01:50:54.197 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29819.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34113.log.gz 2024-09-15T01:50:54.197 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37193.log 2024-09-15T01:50:54.197 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31414.log.gz 2024-09-15T01:50:54.197 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35594.log 2024-09-15T01:50:54.197 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34504.log: /var/log/ceph/ceph-client.admin.37193.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29819.log.gz 2024-09-15T01:50:54.197 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33620.log 2024-09-15T01:50:54.197 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6%/var/log/ceph/ceph-client.admin.35594.log: -- replaced with /var/log/ceph/ceph-client.admin.34504.log.gz 2024-09-15T01:50:54.197 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33844.log 2024-09-15T01:50:54.197 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37193.log.gz 2024-09-15T01:50:54.197 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35042.log 2024-09-15T01:50:54.197 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33620.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35594.log.gz 2024-09-15T01:50:54.197 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37569.log 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33620.log.gz 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29741.log 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35042.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33844.log.gz 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.37569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30162.log 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31250.log 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35042.log.gz 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37569.log.gz 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34191.log 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29741.log.gz 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37217.log 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30162.log.gz 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30583.log 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31250.log.gz 2024-09-15T01:50:54.198 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.bar.37217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30904.log 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34191.log.gz 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30583.log: 88.8% -- replaced with /var/log/ceph/ceph-client.bar.37217.log.gz 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31715.log 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35618.log 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31715.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30583.log.gz 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31510.log 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30904.log.gz 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33116.log 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35618.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31715.log.gz 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr:gzip/var/log/ceph/ceph-client.admin.31510.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31957.log 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33116.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35618.log.gz 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30210.log 2024-09-15T01:50:54.199 INFO:teuthology.orchestra.run.smithi204.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31510.log.gz 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36339.log 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33116.log.gz 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35787.log 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31957.log.gz 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30953.log 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30210.log.gz 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35450.log 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36339.log.gz 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35979.log 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30953.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35787.log.gz 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31100.log 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30953.log.gz 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29604.log 2024-09-15T01:50:54.200 INFO:teuthology.orchestra.run.smithi204.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35450.log.gz 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31100.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35979.log.gz 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37532.log 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34065.log 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31100.log.gz 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.37532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35138.log 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.29604.log.gz 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29582.log 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37532.log.gz 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33769.log 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34065.log.gz 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33430.log 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35138.log.gz 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30120.log 2024-09-15T01:50:54.201 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33769.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29582.log.gz 2024-09-15T01:50:54.202 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33214.log 2024-09-15T01:50:54.202 INFO:teuthology.orchestra.run.smithi204.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33769.log.gz 2024-09-15T01:50:54.202 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34705.log 2024-09-15T01:50:54.202 INFO:teuthology.orchestra.run.smithi204.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33430.log.gz 2024-09-15T01:50:54.202 INFO:teuthology.orchestra.run.smithi204.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30120.log.gz 2024-09-15T01:50:54.202 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29981.log 2024-09-15T01:50:54.202 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34801.log 2024-09-15T01:50:54.202 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33214.log.gz 2024-09-15T01:50:54.202 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36836.log 2024-09-15T01:50:54.202 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34705.log.gz 2024-09-15T01:50:54.202 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33238.log 2024-09-15T01:50:54.202 INFO:teuthology.orchestra.run.smithi204.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29981.log.gz 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34994.log 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34801.log.gz 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36836.log: /var/log/ceph/ceph-client.admin.33238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33921.log 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36836.log.gz 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34395.log 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33238.log.gz 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30487.log 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.34994.log.gz 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34395.log: gzip -5 --verbose -- 89.4% /var/log/ceph/ceph-client.admin.34293.log 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33921.log.gz 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30077.log 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34395.log.gz 2024-09-15T01:50:54.203 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29696.log 2024-09-15T01:50:54.204 INFO:teuthology.orchestra.run.smithi204.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30487.log.gz 2024-09-15T01:50:54.204 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32848.log 2024-09-15T01:50:54.204 INFO:teuthology.orchestra.run.smithi204.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34293.log.gz 2024-09-15T01:50:54.204 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36571.log 2024-09-15T01:50:54.204 INFO:teuthology.orchestra.run.smithi204.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30077.log.gz 2024-09-15T01:50:54.204 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36003.log 2024-09-15T01:50:54.204 INFO:teuthology.orchestra.run.smithi204.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29696.log.gz 2024-09-15T01:50:54.204 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31793.log 2024-09-15T01:50:54.204 INFO:teuthology.orchestra.run.smithi204.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32848.log.gz 2024-09-15T01:50:54.204 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33332.log 2024-09-15T01:50:54.204 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36571.log.gz 2024-09-15T01:50:54.204 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35258.log 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36003.log.gz 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.31793.log.gz 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.32532.log 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33332.log: /var/log/ceph/ceph-client.admin.35258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30655.log 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33332.log.gz 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36595.log 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.35258.log.gz 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31366.log 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30655.log: /var/log/ceph/ceph-client.admin.36595.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32532.log.gz 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29560.log 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31366.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30655.log.gz 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30463.log 2024-09-15T01:50:54.205 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36595.log.gz 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30439.log 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29560.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31366.log.gz 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29843.log 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29560.log.gz 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35883.log 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30463.log.gz 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29843.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30439.log.gz 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32508.log 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29843.log.gz 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33090.log 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35883.log.gz 2024-09-15T01:50:54.206 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32604.log 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33090.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32508.log.gz 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32604.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33090.log.gz 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36619.log 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34559.log 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32604.log.gz 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34922.log 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32556.log 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36619.log.gz 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36860.log 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32556.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34559.log.gz 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31839.log 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34922.log.gz 2024-09-15T01:50:54.207 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33142.log 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32556.log.gz 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33166.log 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36860.log.gz 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32103.log 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31839.log.gz 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33598.log 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33142.log.gz 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35474.log 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33166.log.gz 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33694.log 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32103.log.gz 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30511.log 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33598.log.gz 2024-09-15T01:50:54.208 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35234.log 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30511.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.35474.log.gz 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36291.log 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33694.log.gz 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37263.log 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35234.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30511.log.gz 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36075.log 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35234.log.gz 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.bar.37263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29418.log 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36291.log.gz 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36075.log: 89.0% -- replaced with /var/log/ceph/ceph-client.bar.37263.log.gz 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35859.log 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31200.log 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29418.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36075.log.gz/var/log/ceph/ceph-client.admin.35859.log: 2024-09-15T01:50:54.209 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34631.log 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29418.log.gz 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31200.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.35859.log.gz 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36051.log 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31342.log 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34631.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31200.log.gz 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30282.log 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34631.log.gz 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31342.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36051.log.gz 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30535.log 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35210.log 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31342.log.gz 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32944.log 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30282.log.gz 2024-09-15T01:50:54.210 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35210.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30535.log.gz 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31076.log 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36411.log 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32944.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.35210.log.gz 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28581.log 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.bazar.36825.log 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32944.log.gz 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31076.log.gz 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr:gzip/var/log/ceph/ceph-client.admin.28581.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31152.log 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.bazar.36825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34347.log 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36411.log.gz 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28581.log.gz 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31152.log: 89.6% -- replaced with /var/log/ceph/ceph-client.bazar.36825.log.gz 2024-09-15T01:50:54.211 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35522.log 2024-09-15T01:50:54.212 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35330.log 2024-09-15T01:50:54.212 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31152.log.gz 2024-09-15T01:50:54.212 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35522.log: gzip -5 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34347.log.gz 2024-09-15T01:50:54.212 INFO:teuthology.orchestra.run.smithi204.stderr: --verbose -- /var/log/ceph/ceph-client.admin.29793.log 2024-09-15T01:50:54.212 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31296.log 2024-09-15T01:50:54.212 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35522.log.gz 2024-09-15T01:50:54.212 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37285.log 88.9% 2024-09-15T01:50:54.212 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35330.log.gz 2024-09-15T01:50:54.212 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31296.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29793.log.gz 2024-09-15T01:50:54.212 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35306.log 2024-09-15T01:50:54.212 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.bar.37285.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31296.log.gz 2024-09-15T01:50:54.212 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31767.log 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32992.log 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.bar.37285.log.gz 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31767.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35306.log.gz 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.32057.log 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32220.log 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.31767.log.gz 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32057.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.32992.log.gz -5 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34753.log 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32057.log.gz 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32436.log 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32220.log: /var/log/ceph/ceph-client.admin.34753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36643.log 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32220.log.gz 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.36998.log 2024-09-15T01:50:54.213 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34753.log.gz 2024-09-15T01:50:54.214 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31003.log 2024-09-15T01:50:54.214 INFO:teuthology.orchestra.run.smithi204.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32436.log.gz 2024-09-15T01:50:54.214 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.foo.36998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33869.log 2024-09-15T01:50:54.214 INFO:teuthology.orchestra.run.smithi204.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36643.log.gz 2024-09-15T01:50:54.214 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33745.log 2024-09-15T01:50:54.214 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30559.log 2024-09-15T01:50:54.214 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31003.log.gz 2024-09-15T01:50:54.214 INFO:teuthology.orchestra.run.smithi204.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.foo.36998.log.gz 2024-09-15T01:50:54.214 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30011.log 2024-09-15T01:50:54.214 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33869.log.gz 2024-09-15T01:50:54.214 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34729.log 2024-09-15T01:50:54.214 INFO:teuthology.orchestra.run.smithi204.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33745.log.gz 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr: 96.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32125.log 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30559.log.gz 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34729.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30011.log.gz 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35282.log 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33454.log 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34729.log.gz 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33670.log 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32125.log.gz 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35739.log 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35282.log.gz 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.36884.log 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33454.log.gz 2024-09-15T01:50:54.215 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33478.log 2024-09-15T01:50:54.216 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33670.log.gz 2024-09-15T01:50:54.216 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32750.log 2024-09-15T01:50:54.216 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.foo.36884.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35739.log.gz 2024-09-15T01:50:54.216 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31126.log 2024-09-15T01:50:54.216 INFO:teuthology.orchestra.run.smithi204.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.foo.36884.log.gz 2024-09-15T01:50:54.216 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35715.log 2024-09-15T01:50:54.216 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32750.log: /var/log/ceph/ceph-client.admin.31126.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33478.log.gz 2024-09-15T01:50:54.216 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30757.log 2024-09-15T01:50:54.216 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32750.log.gz 2024-09-15T01:50:54.216 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32630.log 2024-09-15T01:50:54.216 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31126.log.gz 2024-09-15T01:50:54.217 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33795.log 2024-09-15T01:50:54.217 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32630.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35715.log.gz 2024-09-15T01:50:54.217 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34137.log 2024-09-15T01:50:54.217 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30757.log.gz 2024-09-15T01:50:54.217 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33795.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32630.log.gz 2024-09-15T01:50:54.217 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32007.log 2024-09-15T01:50:54.217 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31318.log 2024-09-15T01:50:54.217 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34137.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33795.log.gz 2024-09-15T01:50:54.217 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36195.log 2024-09-15T01:50:54.217 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31318.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34137.log.gz 2024-09-15T01:50:54.217 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.36908.log 2024-09-15T01:50:54.217 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32007.log.gz 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.36932.log 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.foo.36908.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31318.log.gz 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31438.log 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36195.log.gz 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.foo.36932.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.h.log 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.foo.36908.log.gz 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31438.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.i.log 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.foo.36932.log.gz 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.h.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30367.log 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31438.log.gz 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.i.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33993.log 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32580.log 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37556.log 2024-09-15T01:50:54.218 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32580.log: 90.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36363.log 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30367.log.gz 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33993.log.gz 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.bar.37556.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32580.log.gz 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.bar.37556.log.gz 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31272.log 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34897.log 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36267.log 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.36363.log.gz 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34897.log: gzip -5 --verbose -- 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31272.log.gz 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr: /var/log/ceph/ceph-client.admin.29442.log 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36267.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34897.log.gz 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29720.log 2024-09-15T01:50:54.219 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29442.log: gzip 90.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.33044.log 2024-09-15T01:50:54.220 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36267.log.gz 2024-09-15T01:50:54.220 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29720.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29442.log.gz 2024-09-15T01:50:54.220 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.32702.log 2024-09-15T01:50:54.220 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33044.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29720.log.gz 2024-09-15T01:50:54.220 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31052.log 2024-09-15T01:50:54.220 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32702.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29392.log 2024-09-15T01:50:54.220 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33044.log.gz 2024-09-15T01:50:54.220 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30680.log 2024-09-15T01:50:54.220 INFO:teuthology.orchestra.run.smithi204.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32702.log.gz 2024-09-15T01:50:54.220 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32079.log 2024-09-15T01:50:54.220 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5%/var/log/ceph/ceph-client.admin.29392.log: -- replaced with /var/log/ceph/ceph-client.admin.31052.log.gz/var/log/ceph/ceph-client.admin.30680.log: 2024-09-15T01:50:54.221 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35666.log 2024-09-15T01:50:54.221 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34320.log 2024-09-15T01:50:54.221 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5%/var/log/ceph/ceph-client.admin.35666.log: -- replaced with /var/log/ceph/ceph-client.admin.30680.log.gz 2024-09-15T01:50:54.221 INFO:teuthology.orchestra.run.smithi204.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.29392.log.gz 2024-09-15T01:50:54.221 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32079.log.gz 2024-09-15T01:50:54.221 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.37182.log 2024-09-15T01:50:54.221 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35666.log.gz 2024-09-15T01:50:54.221 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31933.log 2024-09-15T01:50:54.221 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34320.log: gzip/var/log/ceph/ceph-client.foo.37182.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36739.log 2024-09-15T01:50:54.221 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31933.log: 89.5% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34320.log.gz 2024-09-15T01:50:54.221 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.foo.37182.log.gz 2024-09-15T01:50:54.221 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31741.log 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32968.log 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36739.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31933.log.gz 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.31741.log: -- /var/log/ceph/ceph-client.admin.34970.log 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32968.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36739.log.gz 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33819.log 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr: 89.9%/var/log/ceph/ceph-client.admin.34970.log: -- replaced with /var/log/ceph/ceph-client.admin.31741.log.gz 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32968.log.gz 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34681.log 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35162.log 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34970.log.gz 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.34681.log: -- 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33819.log.gz 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35162.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34681.log.gz 2024-09-15T01:50:54.222 INFO:teuthology.orchestra.run.smithi204.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35162.log.gz 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr: /var/log/ceph/ceph-client.admin.34532.log 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30631.log 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37307.log 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36315.log 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34532.log.gz 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.bar.37307.log: gzip -5 --verbose 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30631.log.gz 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr: -- /var/log/ceph/ceph-client.admin.30141.log 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36315.log: 89.4% -- replaced with /var/log/ceph/ceph-client.bar.37307.log.gz 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31889.log 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2%/var/log/ceph/ceph-client.admin.30141.log: -- replaced with /var/log/ceph/ceph-client.admin.36315.log.gz 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36147.log 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31889.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31028.log 2024-09-15T01:50:54.223 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30141.log.gz 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28541.log 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31889.log.gz 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.31028.log: -- /var/log/ceph/ceph-client.admin.31176.log 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36147.log.gz 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.28541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34849.log 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31028.log.gz 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31176.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.28541.log.gz 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36801.log 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34849.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31176.log.gz 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30807.log 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36801.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34849.log.gz 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29491.log 2024-09-15T01:50:54.224 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30807.log: gzip -5 --verbose 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36801.log.gz 2024-09-15T01:50:54.225 INFO:teuthology.orchestra.run.smithi204.stderr: -- /var/log/ceph/ceph-client.admin.36099.log 2024-09-15T01:50:54.225 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35835.log 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30807.log.gz 2024-09-15T01:50:54.225 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:50:54.225 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36099.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.29491.log.gz 2024-09-15T01:50:54.225 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34946.log 2024-09-15T01:50:54.225 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35835.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36099.log.gz 2024-09-15T01:50:54.225 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-client.foo.36976.log 2024-09-15T01:50:54.225 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34946.log: gzip -5 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35835.log.gz 2024-09-15T01:50:54.225 INFO:teuthology.orchestra.run.smithi204.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33018.log 2024-09-15T01:50:54.225 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.foo.36976.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34946.log.gz 2024-09-15T01:50:54.225 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35907.log 2024-09-15T01:50:54.225 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33018.log: 88.6% -- replaced with /var/log/ceph/ceph-client.foo.36976.log.gz 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30258.log 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35907.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33018.log.gz 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32872.log 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30258.log: gzip -5 89.4% --verbose -- /var/log/ceph/ceph-client.admin.33552.log 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35907.log.gz 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32872.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30258.log.gz 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33552.log: gzip 88.6% -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32872.log.gz 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.5.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33552.log.gz 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37667.log 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31224.log 2024-09-15T01:50:54.226 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.37667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31911.log 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr: 88.1%/var/log/ceph/ceph-client.admin.31224.log: -- replaced with /var/log/ceph/ceph-client.admin.37667.log.gz 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31224.log.gz 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29650.log 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36459.log 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29650.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31911.log.gz 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr:gzip 89.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.29925.log 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29650.log.gz 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36459.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36459.log.gz 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37643.log 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28461.log 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.37643.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29925.log.gz 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36667.log 2024-09-15T01:50:54.227 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2%/var/log/ceph/ceph-client.admin.28461.log: -- replaced with /var/log/ceph/ceph-client.admin.37643.log.gz 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28461.log.gz 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36243.log 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34218.log 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36667.log: /var/log/ceph/ceph-client.admin.36243.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36667.log.gz 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36243.log.gz 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37329.log 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29955.log 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.bar.37329.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34218.log.gz 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31815.log 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29955.log: 93.1% -- replaced with /var/log/ceph/ceph-client.bar.37329.log.gz 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29955.log.gz 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34242.log 2024-09-15T01:50:54.228 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mgr.x.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31815.log.gz 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32412.log 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34242.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34242.log.gz 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29538.log 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32412.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.32412.log.gz 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33945.log 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29538.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29538.log.gz 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36219.log 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33945.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33945.log.gz 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29467.log 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36219.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36219.log.gz 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32726.log 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29467.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29467.log.gz 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30306.log 2024-09-15T01:50:54.229 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32726.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32726.log.gz 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33503.log 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30306.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30306.log.gz 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35498.log 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33503.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33503.log.gz 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31390.log 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35498.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35498.log.gz 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32318.log 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31390.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31390.log.gz 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35402.log 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32318.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32318.log.gz 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30391.log 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35402.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35402.log.gz 2024-09-15T01:50:54.230 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31983.log 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30391.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30391.log.gz 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34777.log 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31983.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31983.log.gz 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35426.log 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34777.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34777.log.gz 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33528.log 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35426.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35426.log.gz 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28665.log 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33528.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33528.log.gz 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32460.log 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.28665.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.28665.log.gz 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31641.log 2024-09-15T01:50:54.231 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32460.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32460.log.gz 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37158.log 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31641.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31641.log.gz 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29516.log 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.37158.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.37158.log.gz 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32776.log 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29516.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29516.log.gz 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29899.log 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32776.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32776.log.gz 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33644.log 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29899.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29899.log.gz 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35378.log 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33644.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33644.log.gz 2024-09-15T01:50:54.232 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35354.log 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35378.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35378.log.gz 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33068.log 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35354.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35354.log.gz 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33895.log 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33068.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33068.log.gz 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35811.log 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33895.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33895.log.gz 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30928.log 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35811.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35811.log.gz 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34164.log 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30928.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30928.log.gz 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32366.log 2024-09-15T01:50:54.233 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34164.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34164.log.gz 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35763.log 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32366.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32366.log.gz 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36691.log 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35763.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35763.log.gz 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32294.log 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36691.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36691.log.gz 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33969.log 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32294.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32294.log.gz 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32270.log 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33969.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33969.log.gz 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36715.log 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32270.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32270.log.gz 2024-09-15T01:50:54.234 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30978.log 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36715.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36715.log.gz 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31486.log 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30978.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30978.log.gz 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30415.log 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31486.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31486.log.gz 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31534.log 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30415.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30415.log.gz 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30055.log 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31534.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31534.log.gz 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34657.log 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30055.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30055.log.gz 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33358.log 2024-09-15T01:50:54.235 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34657.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34657.log.gz 2024-09-15T01:50:54.236 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32171.log 2024-09-15T01:50:54.236 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33358.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33358.log.gz 2024-09-15T01:50:54.236 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36123.log 2024-09-15T01:50:54.236 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32171.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32171.log.gz 2024-09-15T01:50:54.236 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29628.log 2024-09-15T01:50:54.236 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36123.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36123.log.gz 2024-09-15T01:50:54.236 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34266.log 2024-09-15T01:50:54.236 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29628.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29628.log.gz 2024-09-15T01:50:54.236 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31689.log 2024-09-15T01:50:54.236 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34266.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34266.log.gz 2024-09-15T01:50:54.236 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36483.log 2024-09-15T01:50:54.236 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31689.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31689.log.gz 2024-09-15T01:50:54.237 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33190.log 2024-09-15T01:50:54.237 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36483.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36483.log.gz 2024-09-15T01:50:54.237 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30234.log 2024-09-15T01:50:54.237 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33190.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33190.log.gz 2024-09-15T01:50:54.237 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29869.log 2024-09-15T01:50:54.237 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30234.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.30234.log.gz 2024-09-15T01:50:54.237 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28501.log 2024-09-15T01:50:54.237 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.29869.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29869.log.gz 2024-09-15T01:50:54.237 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32245.log 2024-09-15T01:50:54.237 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.28501.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28501.log.gz 2024-09-15T01:50:54.237 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.f.log 2024-09-15T01:50:54.237 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32245.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32245.log.gz 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30831.log 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.f.log: 89.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32654.log 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30831.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.30831.log.gz 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33576.log 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32654.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32654.log.gz 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34089.log 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.33576.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33576.log.gz 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34583.log 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34089.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34089.log.gz 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37618.log 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34583.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34583.log.gz 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32800.log 2024-09-15T01:50:54.238 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.37618.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37618.log.gz 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35090.log 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.32800.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32800.log.gz 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35066.log 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35090.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35090.log.gz 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30781.log 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35066.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35066.log.gz 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30731.log 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30781.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.30781.log.gz 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30731.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30731.log.gz 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31665.log 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35931.log 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31665.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31665.log.gz 2024-09-15T01:50:54.239 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34041.log 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35931.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35931.log.gz 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31462.log 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34041.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34041.log.gz 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35570.log 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31462.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31462.log.gz 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34873.log 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35570.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35570.log.gz 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37593.log 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.34873.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34873.log.gz 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31617.log 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.37593.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37593.log.gz 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30706.log 2024-09-15T01:50:54.240 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.31617.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.31617.log.gz 2024-09-15T01:50:54.241 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.30706.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30706.log.gz 2024-09-15T01:50:54.241 INFO:teuthology.orchestra.run.smithi204.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T01:50:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35207.log 2024-09-15T01:50:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38729.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.38729.log.gz 2024-09-15T01:50:54.245 INFO:teuthology.orchestra.run.smithi178.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42780.log.gz 2024-09-15T01:50:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40345.log 2024-09-15T01:50:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.35207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36837.log 2024-09-15T01:50:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40345.log: 89.9% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.35207.log.gz 2024-09-15T01:50:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38014.log 2024-09-15T01:50:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36837.log: -- replaced with /var/log/ceph/ceph-client.admin.40345.log.gz 2024-09-15T01:50:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42400.log 2024-09-15T01:50:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38014.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.38014.log.gz 2024-09-15T01:50:54.245 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.36837.log.gz 2024-09-15T01:50:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29872.log 2024-09-15T01:50:54.251 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43411.log 2024-09-15T01:50:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.29872.log: 89.6% 90.5% -- replaced with /var/log/ceph/ceph-client.admin.42400.log.gz 2024-09-15T01:50:54.262 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38561.log 2024-09-15T01:50:54.263 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43411.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43411.log.gz 2024-09-15T01:50:54.264 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29872.log.gz 2024-09-15T01:50:54.273 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30257.log 2024-09-15T01:50:54.274 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38561.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.38561.log.gz 2024-09-15T01:50:54.283 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32960.log 2024-09-15T01:50:54.293 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30257.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38098.log 2024-09-15T01:50:54.296 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32960.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30257.log.gz 2024-09-15T01:50:54.301 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33835.log 2024-09-15T01:50:54.305 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38098.log: -- replaced with /var/log/ceph/ceph-client.admin.32960.log.gz 2024-09-15T01:50:54.310 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40261.log 2024-09-15T01:50:54.314 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33835.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.38098.log.gz 2024-09-15T01:50:54.315 INFO:teuthology.orchestra.run.smithi204.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.g.log.gz 2024-09-15T01:50:54.318 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33377.log 2024-09-15T01:50:54.323 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40261.log: 89.9% 90.5% -- replaced with /var/log/ceph/ceph-client.admin.33835.log.gz 2024-09-15T01:50:54.332 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40261.log.gz 2024-09-15T01:50:54.339 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32582.log 2024-09-15T01:50:54.340 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33377.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33377.log.gz 2024-09-15T01:50:54.348 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33751.log 2024-09-15T01:50:54.360 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.32582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39503.log 2024-09-15T01:50:54.361 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33751.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.33751.log.gz 2024-09-15T01:50:54.361 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.32582.log.gz 2024-09-15T01:50:54.369 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31678.log 2024-09-15T01:50:54.382 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39503.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.39503.log.gz 2024-09-15T01:50:54.387 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33541.log 2024-09-15T01:50:54.387 INFO:teuthology.orchestra.run.smithi204.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.i.log.gz 2024-09-15T01:50:54.388 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31678.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31678.log.gz 2024-09-15T01:50:54.396 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37257.log 2024-09-15T01:50:54.397 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.33541.log: 92.4% -- replaced with /var/log/ceph/ceph-mon.e.log.gz 2024-09-15T01:50:54.397 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40826.log 2024-09-15T01:50:54.398 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37257.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.37257.log.gz 2024-09-15T01:50:54.399 INFO:teuthology.orchestra.run.smithi178.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33541.log.gz 2024-09-15T01:50:54.400 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42232.log 2024-09-15T01:50:54.401 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40826.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.40826.log.gz 2024-09-15T01:50:54.401 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42948.log 2024-09-15T01:50:54.401 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42232.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.42232.log.gz 2024-09-15T01:50:54.403 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33128.log 2024-09-15T01:50:54.403 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42948.log: 92.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T01:50:54.403 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42822.log 2024-09-15T01:50:54.403 INFO:teuthology.orchestra.run.smithi178.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42948.log.gz/var/log/ceph/ceph-client.admin.33128.log: 2024-09-15T01:50:54.405 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.33128.log.gz 2024-09-15T01:50:54.405 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38140.log 2024-09-15T01:50:54.405 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42822.log: -- /var/log/ceph/ceph-client.admin.31552.log 2024-09-15T01:50:54.405 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38140.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42822.log.gz 2024-09-15T01:50:54.407 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37593.log 2024-09-15T01:50:54.407 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31552.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.38140.log.gz 2024-09-15T01:50:54.407 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34171.log 2024-09-15T01:50:54.407 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5%/var/log/ceph/ceph-client.admin.37593.log: -- replaced with /var/log/ceph/ceph-client.admin.31552.log.gz 2024-09-15T01:50:54.407 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36173.log 2024-09-15T01:50:54.408 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.34171.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.37593.log.gz 2024-09-15T01:50:54.408 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34171.log.gz 2024-09-15T01:50:54.408 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36173.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36173.log.gz 2024-09-15T01:50:54.421 INFO:teuthology.orchestra.run.smithi178.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mon.d.log.gz 2024-09-15T01:50:54.463 INFO:teuthology.orchestra.run.smithi178.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T01:50:54.495 INFO:teuthology.orchestra.run.smithi178.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T01:50:54.589 INFO:teuthology.orchestra.run.smithi178.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T01:50:54.610 INFO:teuthology.orchestra.run.smithi178.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T01:50:54.626 INFO:teuthology.orchestra.run.smithi178.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T01:50:54.628 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-15T01:50:54.628 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m0.873s 2024-09-15T01:50:54.628 INFO:teuthology.orchestra.run.smithi178.stderr:user 0m5.317s 2024-09-15T01:50:54.629 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m0.415s 2024-09-15T01:50:54.662 INFO:teuthology.orchestra.run.smithi204.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T01:50:54.683 INFO:teuthology.orchestra.run.smithi204.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T01:50:54.718 INFO:teuthology.orchestra.run.smithi204.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T01:50:54.889 INFO:teuthology.orchestra.run.smithi204.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mon.f.log.gz 2024-09-15T01:50:55.208 INFO:teuthology.orchestra.run.smithi204.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mon.h.log.gz 2024-09-15T01:50:55.209 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-15T01:50:55.210 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m1.432s 2024-09-15T01:50:55.210 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m6.059s 2024-09-15T01:50:55.210 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.434s 2024-09-15T01:50:55.210 INFO:tasks.ceph:Archiving logs... 2024-09-15T01:50:55.210 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/remote/smithi178/log 2024-09-15T01:50:55.211 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T01:50:56.086 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/remote/smithi204/log 2024-09-15T01:50:56.087 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T01:50:57.113 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T01:50:57.124 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T01:50:57.124 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T01:50:57.127 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T01:50:57.203 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T01:50:57.212 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T01:50:57.213 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:50:57.216 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T01:50:57.230 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-09-15T01:50:57.233 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-09-15T01:50:57.234 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T01:50:57.235 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-09-15T01:50:57.235 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 -23us[-2947ns] +/- 45ms 2024-09-15T01:50:57.235 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -8800ns[-8800ns] +/- 77ms 2024-09-15T01:50:57.235 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 +17us[ +37us] +/- 68ms 2024-09-15T01:50:57.235 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:50:57.237 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T01:50:57.237 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-09-15T01:50:57.237 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 -22us[ -17us] +/- 46ms 2024-09-15T01:50:57.237 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 +24us[ +27us] +/- 76ms 2024-09-15T01:50:57.237 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +38us[ +43us] +/- 69ms 2024-09-15T01:50:57.237 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T01:50:57.238 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T01:50:57.249 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T01:50:57.249 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T01:50:57.290 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T01:50:57.366 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T01:50:57.454 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T01:50:57.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:50:57.485 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T01:50:57.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:50:57.514 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T01:50:57.523 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T01:50:57.531 INFO:teuthology.task.internal:Duration was 1525.118258 seconds 2024-09-15T01:50:57.532 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T01:50:57.540 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T01:50:57.540 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T01:50:57.543 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T01:50:57.581 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T01:50:57.595 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T01:50:57.987 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T01:50:57.987 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-09-15T01:50:57.987 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T01:50:58.047 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-09-15T01:50:58.048 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T01:50:58.078 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T01:50:58.078 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T01:50:58.090 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T01:50:58.147 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T01:50:58.147 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T01:50:58.157 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T01:50:58.289 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T01:50:58.302 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T01:50:58.302 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T01:50:58.327 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T01:50:58.352 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T01:50:58.369 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:50:58.372 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:50:58.393 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-09-15T01:50:58.415 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-09-15T01:50:58.429 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:50:58.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:50:58.466 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T01:50:58.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T01:50:58.485 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T01:50:58.495 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T01:50:58.496 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/remote/smithi178 2024-09-15T01:50:58.497 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T01:50:58.543 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905043/remote/smithi204 2024-09-15T01:50:58.543 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T01:50:58.576 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T01:50:58.577 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T01:50:58.579 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T01:50:58.627 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T01:50:58.638 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T01:50:58.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T01:50:58.647 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T01:50:58.647 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T01:50:58.650 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T01:50:58.664 INFO:teuthology.orchestra.run.smithi178.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 01:50 /home/ubuntu/cephtest 2024-09-15T01:50:58.682 INFO:teuthology.orchestra.run.smithi204.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 01:50 /home/ubuntu/cephtest 2024-09-15T01:50:58.686 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T01:50:58.696 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T01:50:58.719 INFO:teuthology.run:Summary data: description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_latest} thrashers/force-sync-many workloads/rados_mon_workunits} duration: 1525.118257522583 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-15T01:50:58.720 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T01:50:58.805 INFO:teuthology.run:pass